Scribd
Upload
0 views9 pages

Problem Set 2

This document contains solutions to various statistical problems, focusing on concepts such as sample means, variance, and the Central Limit Theorem (CLT). It includes exercises that demonstrate the application of asymptotic tools like the Law of Large Numbers (LLN) and the Delta Method to derive estimators and their distributions. The document also discusses properties of estimators, including bias and variance, in the context of different probability distributions.

Uploaded by

cristobalignaciomh
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0 views9 pages

Problem Set 2

This document contains solutions to various statistical problems, focusing on concepts such as sample means, variance, and the Central Limit Theorem (CLT). It includes exercises that demonstrate the application of asymptotic tools like the Law of Large Numbers (LLN) and the Delta Method to derive estimators and their distributions. The document also discusses properties of estimators, including bias and variance, in the context of different probability distributions.

Uploaded by

cristobalignaciomh
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Problem Set 2

Cristóbal Medina Hernández


April 2026

Below I provide my solutions to Problem Set 2. Whenever possible, I use the asymptotic
tools discussed in class, especially the LLN, the CLT, the Continuous Mapping Theorem,
and the Delta Method.

Exercise 1
Let n
1X
X̄n = Xi
n i=1
and n
1 X
s2n = (Xi − X̄n )2 .
n − 1 i=1
Suppose an additional observation Xn+1 becomes available.

(a) Show that


nX̄n + Xn+1
X̄n+1 = .
n+1
Starting from the definition of the sample mean with n + 1 observations,
n+1 n
!
1 X 1 X
X̄n+1 = Xi = Xi + Xn+1 .
n + 1 i=1 n + 1 i=1

Since n
X
Xi = nX̄n ,
i=1

it follows that
nX̄n + Xn+1
X̄n+1 = .
n+1

1
(b) Show that
n
(n − 1)s2n + n+1 (Xn+1 − X̄n )2
s2n+1 = .
n

By definition,
n+1
1X
s2n+1 = (Xi − X̄n+1 )2 .
n i=1
We write
n+1
X n
X
(Xi − X̄n+1 )2 = (Xi − X̄n+1 )2 + (Xn+1 − X̄n+1 )2 .
i=1 i=1

For the first term, note that

Xi − X̄n+1 = (Xi − X̄n ) + (X̄n − X̄n+1 ),

so
n
X n
X n
X
2 2
(Xi − X̄n+1 ) = (Xi − X̄n ) + 2(X̄n − X̄n+1 ) (Xi − X̄n ) + n(X̄n − X̄n+1 )2 .
i=1 i=1 i=1

Since n
X
(Xi − X̄n ) = 0,
i=1

this becomes n n
X X
2
(Xi − X̄n+1 ) = (Xi − X̄n )2 + n(X̄n − X̄n+1 )2 .
i=1 i=1

Using the definition of s2n ,


n
X
(Xi − X̄n )2 = (n − 1)s2n .
i=1

Now, from part (a),


nX̄n + Xn+1
X̄n+1 = ,
n+1
so
X̄n − Xn+1 Xn+1 − X̄n
X̄n − X̄n+1 = =− .
n+1 n+1
Hence,
(Xn+1 − X̄n )2
(X̄n − X̄n+1 )2 = .
(n + 1)2
Also,
nX̄n + Xn+1 n
Xn+1 − X̄n+1 = Xn+1 − = (Xn+1 − X̄n ),
n+1 n+1

2
so  2
2 n
(Xn+1 − X̄n+1 ) = (Xn+1 − X̄n )2 .
n+1
Substituting these into the expression for the sum of squared deviations,
n+1 2
(Xn+1 − X̄n )2

X
2 2 n
(Xi − X̄n+1 ) = (n − 1)sn + n 2
+ (Xn+1 − X̄n )2 .
i=1
(n + 1) n + 1

Factoring the last two terms,


n2
 
n
= (n − 1)s2n + + (Xn+1 − X̄n )2
(n + 1)2 (n + 1)2
n(n + 1) n
= (n − 1)s2n + 2
(Xn+1 − X̄n )2 = (n − 1)s2n + (Xn+1 − X̄n )2 .
(n + 1) n+1
Finally, dividing by n,
n
(n − 1)s2n + n+1
(Xn+1 − X̄n )2
s2n+1 = .
n

Exercise 2
The target parameter is
θ = exp(E[X]) = eµ .
A natural plug-in estimator is

θ̂ = exp(X̄n ) = eX̄n .

To check whether it is unbiased, note that in general

E[eX̄n ] ̸= eE[X̄n ] = eµ .

Since the exponential function is convex, Jensen’s inequality implies

E[eX̄n ] ≥ eE[X̄n ] = eµ = θ.

Therefore, the estimator is not unbiased in general; it has positive bias.


Its variance is 2
V ar(θ̂) = V ar(eX̄n ) = E[e2X̄n ] − E[eX̄n ] .
Without imposing an additional distributional assumption on Xi , this expression cannot
be simplified further using only µ and σ 2 . Thus, the natural estimator is

θ̂ = eX̄n ,

it is generally biased, and its variance is


2
V ar(θ̂) = E[e2X̄n ] − E[eX̄n ] .

3
Exercise 3
Consider the plug-in estimator

θ̂ = exp(X̄n ), θ = eµ .

(a) What is the probability limit of θ̂?


By the Law of Large Numbers,
p
X̄n →
− µ.
Since the exponential function is continuous, the Continuous Mapping Theorem implies
p
θ̂ = eX̄n →
− eµ = θ.

Hence,
plim θ̂ = θ = eµ .

(b) What is the distribution limit of θ̂?


Since convergence in probability to a constant implies convergence in distribution to the
same constant,
d
θ̂ →
− θ.
Therefore,
d
θ̂ →
− θ.

√
(c) What is the distribution limit of n(θ̂ − θ)?
By the CLT,
√ d
− N (0, σ 2 ).
n(X̄n − µ) →
Let
g(x) = ex .
Then
g ′ (µ) = eµ .
Applying the Delta Method,
√ d
− N 0, [g ′ (µ)]2 σ 2 .

n(g(X̄n ) − g(µ)) →

That is,
√ d
n(eX̄n − eµ ) →
− N 0, e2µ σ 2 .


Hence,
√ d
− N (0, e2µ σ 2 ).
n(θ̂ − θ) →

4
Exercise 4
Assume now that Xi > 0, E[Xi ] = µ > 0, and V ar(Xi ) = σ 2 .

(a) What is the probability limit of ln(X̄n )?


By the Law of Large Numbers,
p
X̄n →
− µ.
Since µ > 0 and ln(·) is continuous on (0, ∞), the Continuous Mapping Theorem gives
p
ln(X̄n ) →
− ln(µ).

Therefore,
plim ln(X̄n ) = ln(µ).

(b) What is the distribution limit of


√ 
n ln(X̄n ) − ln(µ) ?

Again, by the CLT,


√ d
− N (0, σ 2 ).
n(X̄n − µ) →
Let
g(x) = ln(x),
so that
1
g ′ (µ) = .
µ
By the Delta Method,
 2 !
√ d 1
σ2 .

n g(X̄n ) − g(µ) →
− N 0,
µ

Hence,
√ σ2
 
 d
n ln(X̄n ) − ln(µ) →
− N 0, 2 .
µ

Exercise 5
We want to show that Pn
i=1 Xi p 1
p P n 2
− √ ,
→
n i=1 Xi 2
where E[Xi ] = 1 and V ar(Xi ) = 1.
First, note that
V ar(X) = E[X 2 ] − (E[X])2 .

5
Since V ar(X) = 1 and E[X] = 1, it follows that

1 = E[X 2 ] − 1,

so
E[X 2 ] = 2.
Now rewrite the expression in terms of sample moments:
n
X n
X
Xi = nX̄n , Xi2 = nX 2 n ,
i=1 i=1

where n
1X 2
X 2n = X .
n i=1 i
Hence, Pn
i=1 Xi nX̄n X̄
p P n = q = p n .
n i=1 Xi2 n(nX 2 n ) X 2n

By the Law of Large Numbers,


p
X̄n →
− E[X] = 1

and
p
− E[X 2 ] = 2.
X 2n →
Applying the Continuous Mapping Theorem to the function
a
g(a, b) = √ ,
b
we obtain
X̄ p 1
p n → − √ .
X 2n 2
Therefore,
Pn
i=1 Xi p 1
p P n 2
− √ .
→
n i=1 Xi 2

Exercise 6
Let Xi be independent Bernoulli(p) random variables.

6
(a) Show that
√ d
n(X̄n − p) →
− N (0, p(1 − p)).
Since Xi ∼ Bernoulli(p),
E[Xi ] = p, V ar(Xi ) = p(1 − p).
By the Central Limit Theorem,
√ d
n(X̄n − p) →
− N (0, p(1 − p)).
Hence,
√ d
n(X̄n − p) →
− N (0, p(1 − p)).

(b) For p ̸= 1/2, show that


√  d
− N 0, (1 − 2p)2 p(1 − p) .

n X̄n (1 − X̄n ) − p(1 − p) →

Let
g(x) = x(1 − x) = x − x2 .
Then
g ′ (x) = 1 − 2x, g ′ (p) = 1 − 2p.
From part (a),
√ d
n(X̄n − p) →
− N (0, p(1 − p)).
Applying the Delta Method,
√  d
− N 0, [g ′ (p)]2 p(1 − p) .

n g(X̄n ) − g(p) →
Since
g(X̄n ) = X̄n (1 − X̄n ), g(p) = p(1 − p),
we obtain
√  d
− N 0, (1 − 2p)2 p(1 − p) .

n X̄n (1 − X̄n ) − p(1 − p) →

Exercise 7
Define 1/k
mk = E[|X|k ] .

(a) Propose an estimator for mk


A natural plug-in estimator is
n
!1/k
1X
m̂k = |Xi |k .
n i=1

7
√
(b) Find the asymptotic distribution of n(m̂k − mk )
Let
Yi = |Xi |k , θ = E[Yi ] = E[|X|k ], h(u) = u1/k .
Then !
n
1X
mk = h(θ), m̂k = h Yi .
n i=1
If E[|X|2k ] < ∞, then V ar(Yi ) < ∞, and by the CLT,
n
!
√ 1X d
n Yi − θ → − N (0, V ar(Yi )) = N (0, V ar(|X|k )).
n i=1

Now
1 1/k−1
h′ (u) = u .
k
Evaluating at θ,
1 1/k−1
h′ (θ) = θ .
k
Applying the Delta Method,
2 !
√

d 1
n(m̂k − mk ) →
− N 0, θ1/k−1 V ar(|X|k ) .
k

Therefore,
√
 
d 1 k 2(1/k−1)
 k
n(m̂k − mk ) →
− N 0, 2 E[|X| ] V ar(|X| ) .
k
Since mk = (E[|X|k ])1/k , an equivalent expression is

√
 
d 1 2(1−k) k
n(m̂k − mk ) → − N 0, 2 mk V ar(|X| ) .
k

(c) Find the asymptotic distribution when k = 2


When k = 2,
n
!1/2
1/2 1X 2
m2 = E[X 2 ] , m̂2 = X .
n i=1 i
Using the general formula,
√ V ar(X 2 )
 
d
n(m̂2 − m2 ) →
− N 0, .
4m22
Hence,
√ V ar(X 2 )
 
d
n(m̂2 − m2 ) →
− N 0, .
4m22

8
(d) State the regularity conditions under which the Lindeberg–Levy
CLT holds
The Lindeberg–Levy CLT holds when:

1. X1 , . . . , Xn are independent and identically distributed,

2. E[X] is finite,

3. V ar(X) is finite.

Equivalently, for the CLT applied to Yi = |Xi |k , we need

E[|X|2k ] < ∞.

You might also like