STAT 583: ADVANCED STATISTICAL THEORY
Spring 2024
Instructor: Marco Carone
HOMEWORK 1 (WITH SOLUTIONS)
1) A partial Slutsky’s theorem for weak convergence
Let F be a given function class. Let X1 , X2 , . . . and Y1 , Y2 , . . . be two sequences of ℓ∞ (F)-valued
random variables, and suppose that ∥Xn − Yn ∥F tends to zero in probability. Show that if Xn
converges weakly to X in ℓ∞ (F) relative to ∥ · ∥F for some ℓ∞ (F)-valued random variable X, then
Yn also converges weakly to X in ℓ∞ (F) relative to ∥ · ∥F .
Solution:
Below, we denote ∥ · ∥F simply by ∥ · ∥ for brevity. We note that, for any bounded, ∥ · ∥-Lipschitz
function h : ℓ∞ (F) → R, the triangle inequality can be used to write
|E[h(Yn )] − E[h(X)]| ≤ |E[h(Yn )] − E[h(Xn )]| + |E[h(Xn )] − E[h(X)]| .
For the first summand, denoting by M0 the bound on the image of |h|, the Lipschitz property can
be used to write that
|E[h(Yn )] − E[h(Xn )]| = |E[h(Yn ) − h(Xn )]| ≤ E|h(Yn ) − h(Xn )|
≤ E[min{2M0 , L0 ∥Yn − Xn ∥}] .
Since min{2M0 , L0 ∥Zn − Z̄n ∥} is a non-negative, bounded random variable tending to zero in
probability, its expectation also tends to zero. For the second summand, we note that |E[h(Xn )] −
E[h(X)]| tends to zero because Xn converges weakly to X relative to ∥ · ∥. Thus, we find that
|E[h(Yn )] − E[h(X)]| tends to zero, and so, Yn also converges weakly to X relative to ∥ · ∥.
2) Studying the evaluation of the empirical process on a random function
p
Let F be a class satisfying that supf ∈F ρP (f ) < ∞, where ρP : f 7→ P (f − P f )2 is the standard
deviation seminorm. For any δ > 0, define the class
D(δ) := {f − g : f, g ∈ F, ρP (f − g) < δ} .
Based on results presented in class, F is P -Donsker if and only if both:
1. (F, ρP ) is totally bounded, in the sense that, for each ϵ > 0, there exist f1 , f2 , . . . , fm ∈ F
with m < ∞ such that, for each f ∈ F, ρP (f − fj ) < ϵ for some j ∈ {1, 2, . . . , m};
√
2. the empirical process Gn := n (Pn − P ) is asymptotically uniformly equicontinuous on F,
in the sense that ∥Gn ∥D(δn ) = oP (1) for each deterministic sequence δn ↓ 0.
Show each of the following facts:
(a) If F is P -Donsker and ∆1 , ∆2 , . . . is a sequence of non-negative real-valued random variables
such that ∆n = oP (1), then ∥Gn ∥D(∆n ) also tends to zero in probability.
1
(b) If F is P -Donsker and h1 , h2 , . . . is a sequence of random functions in F such that ρP (hn −
h0 ) = oP (1) for some h0 ∈ F, then Gn (hn − h0 ) = oP (1).
(c) If F ⊆ L2 (P ) is P -Donsker and h1 , h2 , . . . is a sequence of random functions in L2 (P ) such
that P (hn ∈ F) → 1 and P (hn − h0 )2 = oP (1) for some h0 ∈ F, then Gn (hn − h0 ) = oP (1).
Solution:
(a) First, we note that for any deterministic sequence δn ↓ 0 we can write
P ∥Gn ∥D(∆n ) > ϵ = P ∥Gn ∥D(∆n ) > ϵ, ∆n ≤ δn + P ∥Gn ∥D(∆n ) > ϵ, ∆n > δn
≤ P ∥Gn ∥D(δn ) > ϵ + P (∆n > δn ) .
Suppose that for any η > 0 it is possible to select a sequence δn ↓ 0 such that P (∆n > δn ) ≤ η for
all n. For any such sequence, using the inequality derived above, we have that
0 ≤ lim sup P ∥Gn ∥D(∆n ) > ϵ ≤ lim sup P ∥Gn ∥D(δn ) > ϵ + η = η .
n n
Since η > 0 was chosen arbitrarily, we may then conclude that lim supn P ∥Gn ∥D(∆n ) > ϵ = 0
using that ∥Gn ∥D(δn ) = oP (1), thereby establishing that ∥Gn ∥D(∆n ) = oP (1).
To complete the argument, we must show that it is always possible to construct a sequence δn ↓ 0
such that P (∆n > δn ) ≤ η for all n. For each n, we define
δn∗ := inf{u : P (∆n > u) ≤ η} and δn∗∗ := sup{u : P (∆n > u) ≥ η2 } .
It must be that δn∗∗ → 0. Otherwise, there would exist some ε > 0 and a subsequence n1 , n2 , . . .
of increasing natural numbers such that δn∗∗k > ε for all k. This would imply that 0 ≤ P (∆nk >
δn∗∗k ) ≤ P (∆nk > ε) → 0 since ∆n = oP (1). However, we also have that P (∆nk > δn∗∗k ) ≥ η2 > 0
by definition, yielding a contradiction. As such, we have that δn∗∗ → 0, and since 0 ≤ δn∗ ≤ δn∗∗ , it
follows that δn∗ → 0. Finally, defining δn := min{δ1∗ , δ2∗ , . . . , δn∗ }, we note that δn ↓ 0, as required.
(b) Set ∆n := ρP (hn − h0 ) and note that ∆n is a sequence of non-negative real-valued random
variables such that ∆n = oP (1), and so, ∥Gn ∥D(∆n ) = oP (1) by (a). Since we have that |Gn (hn −
h0 )| ≤ ∥Gn ∥D(∆n ) in view of the fact that hn −h0 lies in D(∆n ), it follows that Gn (hn −h0 ) = oP (1).
(c) First, we note that P (hn − h0 )2 = oP (1) implies that ρP (hn − h0 ) = oP (1). This follows because
ρP (f )2 = P (f − P f )2 = P f 2 − (P f )2 ≤ P f 2 , and so, ρP (hn − h0 )2 ≤ P (hn − h0 )2 . Now, for any
ϵ > 0, we can write
P (|Gn (hn − h0 )| > ϵ) = P (|Gn (hn − h0 )| > ϵ, hn ∈ F) + P (|Gn (hn − h0 )| > ϵ, hn ∈
/ F)
≤ P (|Gn (hn − h0 )| > ϵ, hn ∈ F) + P (hn ∈
/ F)
≤ P ∥Gn ∥D(γn ) > ϵ, hn ∈ F + P (hn ∈ / F)
≤ P ∥Gn ∥D(γn ) > ϵ + P (hn ∈ / F) ,
p
where γn := P (hn − h0 )2 = oP (1). So, by (a), we find that
0 ≤ lim sup P (|Gn (hn − h0 )| > ϵ) = lim sup P ∥Gn ∥D(γn ) > ϵ + lim sup P (hn ∈
/ F) = 0 .
n n n
3) Examples of the permanence property of Donsker classes
Let F be a P -Donsker class of functions from X to R. Show each of the following facts:
2
(a) If G ⊂ F, then G is also P -Donsker.
(b) If ∥P ∥F < ∞ and there exists some δ > 0 such that f ≥ δ pointwise for each f ∈ F, then
G := {x 7→ 1/f (x) : f ∈ F} is also P -Donsker.
Solution: (a) Because F is P -Donsker, for any f1 , f2 , . . . , fm ∈ F, it is true that {Gn fj : j =
1, 2, . . . , m} ⇝ {Gfj : j = 1, 2, . . . , m}. Consequently, for any g1 , g2 , . . . , gm ∈ G ⊂ F, it is true that
{Gn f gj : j = 1, 2, . . . , m} ⇝ {Ggj : j = 1, 2, . . . , m}. Moreover, as F is P -Donsker, there exists a
pseudometric ρ for which F is totally bounded and Gn is asymptotically uniformly ρ-equicontinuous
over F. As G ⊂ F, G must also be totally bounded with respect to ρ. To see this, fix ϵ > 0 and let
C := {f1 , f2 , . . . , fm } ⊂ F be an ϵ-cover of F. For each j = 1, 2, . . . , m, if fj ∈ / G, remove fj from C,
and if there exists some gj ∈ G such that ρ(fj , gj ) < ϵ, add gj to C. The resulting modification of C
is an 2ϵ-cover of G. As such, for any ϵ > 0, N (2ϵ, G, ρ) is bounded above by N (ϵ, F, ρ) < ∞, and so,
G is totally bounded with respect to ρ. Finally, Gn is asymptotically uniformly ρ-equicontinuous
over G as well.
(b) Consider the function ϕ : z 7→ 1/z. This function is Lipschitz on [δ, ∞) for any δ > 0. Hence,
by the theorem stated on slide 17 of Chapter 0, the class {ϕ ◦ f : f ∈ F} is P -Donsker.