Statistical Inference - Comprehensive Final 1 of 11

Final Exam - Advanced Set 1

Detailed Solutions

Remark. The solutions are intentionally detailed. Each likelihood, statistic, and limiting distribution is
derived step by step.
Statistical Inference - Comprehensive Final 2 of 11

Solution to Problem 1

The joint density is

n n n
! !
x2i xi x2 1X
 
−3n
Y Y
i
L(θ; x) = 3
exp − = θ exp − xi .
i=1
2θ θ i=1
2 θ i=1

Thus the data enter the likelihood through

n
X
T = Xi .
i=1

By the factorization theorem, T is sufficient for θ.

Since a sum of independent gamma variables with common scale is gamma,

T ∼ Gamma(3n, θ),

with density
1
fT (t; θ) = t3n−1 e−t/θ , t > 0.
Γ(3n)θ3n
To prove completeness, suppose Eθ [g(T )] = 0 for every θ > 0. Then

t3n−1 −t/θ
Z ∞
0= g(t) e dt.
0 Γ(3n)θ3n

Let y = 1/θ. Since θ−3n = y 3n ,

y 3n
Z ∞
0= g(t)t3n−1 e−yt dt
Γ(3n) 0

for all y > 0. Hence Z ∞

g(t)t3n−1 e−yt dt = 0, y > 0.
0
By uniqueness of the Laplace transform,

g(t)t3n−1 = 0 a.e. on (0, ∞),

and therefore g(T ) = 0 almost surely. Hence T is complete.

Next,
Eθ [T ] = 3nθ.
Therefore
T
δ(T ) =
3n
is unbiased for θ. Since T is complete and sufficient, the Lehmann-Scheffe theorem implies

T
is the MVUE of θ.
3n

The log-likelihood, ignoring constants not involving θ, is

T
ℓ(θ) = −3n log θ − .
θ
Statistical Inference - Comprehensive Final 3 of 11

Differentiating,
3n T
ℓ′ (θ) = − + 2.
θ θ
Setting ℓ′ (θ) = 0 gives
−3nθ + T = 0,
so
T
θ̂ = .
3n
The second derivative is
3n 2T
ℓ′′ (θ) = − 3.
θ2 θ
For one observation, the log-likelihood is
X
ℓ1 (θ) = −3 log θ − + C(X).
θ
Thus
3 2X
ℓ′′1 (θ) = 2
− 3.
θ θ
Using Eθ [X] = 3θ,
3 2(3θ) 3
 
I(θ) = −Eθ [ℓ′′1 (θ)] =− 2 − = .
θ θ3 θ2
Hence !
√ 1 θ2
 
d
n(θ̂ − θ) −
→ N 0, = N 0, .
I(θ) 3
Let g(θ) = log θ. Then g ′ (θ) = 1/θ. By the Delta method,
 2 2 !
√ 1 θ 1
 
d
n{log θ̂ − log θ} −
→ N 0, = N 0, .
θ 3 3

Therefore
√ 1
 
d
n{log θ̂ − log θ} −
→ N 0, .
3
Statistical Inference - Comprehensive Final 4 of 11

Solution to Problem 2

The likelihood is
n
( )
1 X
L(µ, σ 2 ) = (2πσ 2 )−n/2 exp − 2 (xi − µ)2 .
2σ i=1
Under the unrestricted model, the MLEs are
n
2 1X
µ̂ = X̄, σ̂ = (Xi − X̄)2 .
n i=1

Let n
X
SSE = (Xi − X̄)2 .
i=1

Then the maximized unrestricted likelihood is proportional to

−n/2
SSE

(σ̂ 2 )−n/2 e−n/2 = e−n/2 .
n

Under H0 : µ = µ0 , only σ 2 is unknown. The restricted MLE is

n
1X
σ̂02 = (Xi − µ0 )2 .
n i=1

Let n
X
SSE0 = (Xi − µ0 )2 .
i=1

The restricted maximized likelihood is proportional to

−n/2
SSE0

e−n/2 .
n
Therefore the likelihood ratio is
n/2
supH0 L SSE

Λ= = .
supH0 ∪H1 L SSE0

Now decompose
n
X
SSE0 = (Xi − X̄ + X̄ − µ0 )2 = SSE + n(X̄ − µ0 )2 .
i=1

Let √
2 SSE n(X̄ − µ0 )
s = , T = .
n−1 s
Then
SSE0 n(X̄ − µ0 )2 T2
=1+ =1+ .
SSE SSE n−1
Therefore !−n/2
T2
Λ= 1+ .
n−1
Statistical Inference - Comprehensive Final 5 of 11

Since Λ is a decreasing function of T 2 , the LRT rejects for large |T |. Under H0 ,

T ∼ tn−1 .

Thus the exact size α LRT rejects when

|T | > tn−1,1−α/2 .

This is exactly the usual two-sided one-sample t-test.

For asymptotic tests, the parameter of interest is µ and σ 2 is nuisance. The usual large-sample Wald
statistic is
n(X̄ − µ0 )2
Wn = .
s2
d
→ χ21 .
Under H0 , Wn −
The likelihood ratio statistic is
!
T2
LRn = −2 log Λ = n log 1 + .
n−1

Since log(1 + u) = u + o(u) and T 2 = Op (1) under H0 ,

(  )
T2 1 d
LRn = n + op = T 2 + op (1) −
→ χ21 .
n−1 n

The score statistic can be written using the efficient score for µ after estimating the nuisance parameter
under H0 . The restricted variance estimator is
n
1X
σ̂02 = (Xi − µ0 )2 .
n i=1

The score for µ is

n
1 X
Uµ (µ, σ 2 ) = (Xi − µ).
σ 2 i=1
Evaluated at the restricted MLE,
n(X̄ − µ0 )
Uµ (µ0 , σ̂02 ) = .
σ̂02
The corresponding score statistic is asymptotically equivalent to

n(X̄ − µ0 )2 d 2
Rn = −
→ χ1 .
σ̂02

All three tests have the same limiting null distribution because they are all quadratic approximations to
the same log-likelihood around µ0 . In regular models,

{ℓ′ (µ0 )}2

2{ℓ(µ̂) − ℓ(µ0 )} = + op (1) = In (µ̂ − µ0 )2 + op (1).
In (µ0 )

This identity explains the first-order equivalence of likelihood ratio, score, and Wald tests.
Statistical Inference - Comprehensive Final 6 of 11

Solution to Problem 3

The likelihood is n
X
Y n−Y
L(p) = p (1 − p) , Y = Xi .
i=1
Under the unrestricted model, the MLE is
Y
p̂ = .
n
Under H0 , p = p0 . Hence the likelihood ratio is

pY0 (1 − p0 )n−Y
Λ(Y ) = .
p̂Y (1 − p̂)n−Y

The symbol in the denominator is simply p̂ = Y /n, so explicitly,

pY0 (1 − p0 )n−Y
Λ(Y ) =
(Y /n)Y (1 − Y /n)n−Y

with the usual conventions when Y = 0 or Y = n. The exact LRT rejects for small values of Λ(Y ). Since
Y is discrete, the exact critical region is formed by including values of Y for which Λ(Y ) is smallest until
the null probability is at most α. Under H0 ,

Y ∼ Binomial(n, p0 ).

Thus the size is computed exactly from binomial probabilities.

For large samples, the Wald statistic is

n(p̂ − p0 )2
Wn = .
p̂(1 − p̂)
p
Under H0 , p̂ →
− p0 and
√ d
n(p̂ − p0 ) −
→ N (0, p0 (1 − p0 )).
Therefore
d
→ χ21 .
Wn −

The score function is

Y n−Y Y − np
U (p) = − = .
p 1−p p(1 − p)
The Fisher information is
n
In (p) = .
p(1 − p)
The score statistic is
U (p0 )2 (Y − np0 )2 n(p̂ − p0 )2
Rn = = = .
In (p0 ) np0 (1 − p0 ) p0 (1 − p0 )
Therefore
d
→ χ21 .
Rn −
This is the square of the usual one-sample proportion z statistic:
p̂ − p0
Z=p , Rn = Z 2 .
p0 (1 − p0 )/n
Statistical Inference - Comprehensive Final 7 of 11

The likelihood ratio statistic is

p̂ 1 − p̂
    
LRn = −2 log Λ(Y ) = 2 Y log + (n − Y ) log .
p0 1 − p0
By Wilks’ theorem,
d
→ χ21
LRn −
under H0 . More directly, a second-order Taylor expansion of the log-likelihood around p0 gives

{ℓ′ (p0 )}2

2{ℓ(p̂) − ℓ(p0 )} = + op (1),
In (p0 )

which is exactly the score statistic up to op (1).

Statistical Inference - Comprehensive Final 8 of 11

Solution to Problem 4

The joint density is

n
!
−n 1X
L(θ; x) = θ exp − xi , xi > 0.
θ i=1
Therefore n
X
T = Xi
i=1
is sufficient by the factorization theorem. Also,

T ∼ Gamma(n, θ),

with density
1
fT (t; θ) = tn−1 e−t/θ , t > 0.
Γ(n)θn
The same Laplace transform argument used for gamma families shows completeness: if

Eθ [g(T )] = 0 for all θ > 0,

then Z ∞
g(t)tn−1 e−yt dt = 0 for all y > 0,
0

so g(t)tn−1 = 0 a.e. Hence T is complete sufficient.

The MLE of θ is
T
θ̂ = X̄ = .
n
By functional invariance of the MLE,

η̂ = e−a/θ̂ = e−a/X̄ .

To find the MVUE of η(θ) = Prθ (X1 > a), start from the unbiased estimator

1{X1 > a}.

Indeed,
Eθ [1{X1 > a}] = Pr(X1 > a) = e−a/θ .
θ
By Rao-Blackwellization and Lehmann-Scheffe,

E[1{X1 > a} | T ]

is the MVUE.
For exponential samples, conditional on T = t,
X1
| T = t ∼ Beta(1, n − 1).
T
The density of U = X1 /T is

fU (u) = (n − 1)(1 − u)n−2 , 0 < u < 1.

Statistical Inference - Comprehensive Final 9 of 11

Thus, if t > a, Z 1
a
 
Pr(X1 > a | T = t) = Pr U > = (n − 1)(1 − u)n−2 du.
t a/t

Computing the integral,

Z 1 n−1
a

n−2
(n − 1)(1 − u) du = 1 − .
a/t t
If t ≤ a, the conditional probability is zero. Therefore
n−1
a

δ(T ) = 1 − 1{T > a}
T

is the MVUE of e−a/θ .

For the asymptotic distribution, first note that for an exponential mean parameter,
√ √ d
n(θ̂ − θ) = → N (0, θ2 ).
n(X̄ − θ) −

Let
g(θ) = e−a/θ .
Then
a
g ′ (θ) = e−a/θ .
θ2
By the Delta method,
√ d
 2 
→ N 0, g ′ (θ) θ2 .

n{g(θ̂) − g(θ)} −
Hence !
√ d a2
→ N 0, 2 e−2a/θ .
n(η̂ − η) −
θ
Statistical Inference - Comprehensive Final 10 of 11

Solution to Problem 5

The log-likelihood is
n
X
ℓ(λ) = {−λ + Xi log λ − log(Xi !)} = −nλ + Y log λ + C,
i=1

where n
X
Y = Xi .
i=1

The score function is

Y
U (λ) = ℓ′ (λ) = −n + .
λ
The second derivative is
Y
ℓ′′ (λ) = − .
λ2
Since Eλ (Y ) = nλ,
n
In (λ) = −Eλ [ℓ′′ (λ)] = .
λ
Thus the Fisher information in one observation is
1
I(λ) = .
λ
Solving U (λ) = 0 gives
Y
−n + = 0,
λ
so
Y
λ̂ = X̄ = .
n

The Wald statistic is

n(λ̂ − λ0 )2
Wn = In (λ̂)(λ̂ − λ0 )2 = .
λ̂
The score statistic is

U (λ0 )2 (−n + Y /λ0 )2 (Y − nλ0 )2 n(λ̂ − λ0 )2

Rn = = = = .
In (λ0 ) n/λ0 nλ0 λ0

The likelihood ratio statistic is

LRn = 2{ℓ(λ̂) − ℓ(λ0 )}.
Since
ℓ(λ) = −nλ + Y log λ + C,
we get h i
LRn = 2 −nλ̂ + Y log λ̂ + nλ0 − Y log λ0 .

Using Y = nλ̂,
" ! #
λ̂
LRn = 2n λ̂ log − λ̂ + λ0 .
λ0
Statistical Inference - Comprehensive Final 11 of 11

Under H0 , by the CLT,

√ d
n(λ̂ − λ0 ) −
→ N (0, λ0 ).
Therefore √
n(λ̂ − λ0 ) d
√ −
→ N (0, 1),
λ0
and hence (√ )2
n(λ̂ − λ0 ) d
Rn = √ → χ21 .
−
λ0
p
Because λ̂ →
− λ0 under H0 , replacing λ0 by λ̂ in the denominator does not change the limit, so
d
→ χ21 .
Wn −

By Wilks’ theorem,
d
→ χ21 .
LRn −

The Taylor expansion explanation is as follows. Around λ0 ,

1
ℓ(λ̂) = ℓ(λ0 ) + U (λ0 )(λ̂ − λ0 ) − In (λ0 )(λ̂ − λ0 )2 + op (1).
2
The likelihood equation gives
√
0 = U (λ̂) = U (λ0 ) − In (λ0 )(λ̂ − λ0 ) + op ( n|λ̂ − λ0 |).

Thus
U (λ0 )
λ̂ − λ0 = + op (n−1/2 ).
In (λ0 )
Substituting this into the expansion yields

U (λ0 )2
LRn = + op (1) = Rn + op (1).
In (λ0 )

Also
Wn = In (λ0 )(λ̂ − λ0 )2 + op (1) = Rn + op (1).
Therefore Wald, score, and likelihood ratio tests are asymptotically equivalent.