Statistical Inference - Comprehensive Final 1 of 10

Final Exam - Advanced Set 2

Detailed Solutions

Remark. The solutions are intentionally detailed. Each likelihood, statistic, and limiting distribution is
derived step by step.
Statistical Inference - Comprehensive Final 2 of 10

Solution to Problem 1

The joint density is

L(θ; x) = θ−n 1{0 < X(1) , X(n) ≤ θ}.
The likelihood depends on the sample through X(n) = maxi Xi , so by the factorization theorem

T = X(n)

is sufficient.
The density of T = X(n) is
ntn−1
fT (t; θ) = , 0 < t < θ.
θn
To prove completeness, suppose Eθ [g(T )] = 0 for all θ > 0. Then

ntn−1
Z θ
0= g(t) dt.
0 θn
Multiplying by θn /n gives
Z θ
g(t)tn−1 dt = 0 for every θ > 0.
0
Differentiating with respect to θ,
g(θ)θn−1 = 0
for almost every θ > 0. Therefore g(T ) = 0 a.s., so T is complete.
Since
n
θ,
Eθ [X(n) ] =
n+1
an unbiased estimator based on the complete sufficient statistic is

n+1
δ(T ) = X(n) .
n

By Lehmann-Scheffe, this is the MVUE of θ. The MLE is

θ̂M LE = X(n) .

Thus the MLE is biased downward, while the MVUE corrects this bias by multiplying by (n + 1)/n.
Now consider 2X̄. Since
θ θ2
E[Xi ] = , Var(Xi ) = ,
2 12
WLLN gives
p θ
X̄ →
− ,
2
so
p
2X̄ →
− θ.
By the CLT, !
√ θ θ2
 
d
n X̄ − −
→ N 0, .
2 12
Statistical Inference - Comprehensive Final 3 of 10

Multiplying by 2,
!
√ dθ2
n(2X̄ − θ) −
→ N 0, .
3

So 2X̄ is a regular asymptotically normal estimator.

The MLE has a different limiting distribution. For 0 < y < n,
!
n(θ − X(n) ) y
  
Pr >y = Pr X(n) <θ 1− .
θ n

Since X(n) /θ has cdf un on 0 < u < 1,

n
y y
   
Pr X(n) < θ 1 − = 1− .
n n
Letting n → ∞, n
y

1− → e−y .
n
Hence
n(θ − X(n) ) d
−
→ Exponential(1).
θ
This is not a normal limit, showing that the uniform endpoint problem is nonregular.
For testing H0 : θ = θ0 against H1 : θ > θ0 , large values of X(n) are evidence against H0 . Under H0 ,
n
c

Pr(X(n) ≤ c) = , 0 < c < θ0 .
θ0 θ0
Choose cα so that
Pr(X(n) > cα ) = α.
θ0

Then n
cα

1− = α,
θ0
so
cα = θ0 (1 − α)1/n .
The exact level α test rejects when
X(n) > θ0 (1 − α)1/n .
Statistical Inference - Comprehensive Final 4 of 10

Solution to Problem 2

Let n
X
T = Xi .
i=1
The joint pmf is
n
P 1
Pr(X1 = x1 , . . . , Xn = xn ) = e−nλ λ
Y
xi
.
i=1
xi !
By the factorization theorem, T is sufficient. Also

T ∼ Poisson(nλ).

The Poisson family is complete: if Eλ [g(T )] = 0 for all λ > 0, then

∞
(nλ)t
g(t)e−nλ
X
0= .
t=0
t!

Multiplying by enλ ,
∞
X (nλ)t
0= g(t)
t=0
t!
for all λ > 0. By uniqueness of power series, g(t) = 0 for all t. Hence T is complete.
An unbiased estimator of η(λ) = Pr(X1 = 2) is

1{X1 = 2}.

Therefore the MVUE is

E[1{X1 = 2} | T ].
Conditional on T = t, the vector (X1 , . . . , Xn ) has multinomial distribution with total t and cell
probabilities 1/n, . . . , 1/n. Hence
1
 
X1 | T = t ∼ Binomial t, .
n
Thus !   t−2
2
t 1 1
Pr(X1 = 2 | T = t) = 1−
2 n n
for t ≥ 2, and zero for t < 2. Therefore
!   T −2
T 1 2 1
δ(T ) = 1− 1{T ≥ 2}
2 n n

is the MVUE.
The MLE of λ is
λ̂ = X̄.
By functional invariance,
λ̂2 e−λ̂ X̄ 2 e−X̄
η̂ = = .
2 2
Statistical Inference - Comprehensive Final 5 of 10

For the Delta method, define

λ2 e−λ
g(λ) = .
2
Then
1 e−λ
g ′ (λ) = {2λe−λ − λ2 e−λ } = λ(2 − λ).
2 2
For a Poisson sample,
√ d
n(λ̂ − λ) −
→ N (0, λ).
Therefore √  
d
→ N 0, λ[g ′ (λ)]2 .
n{g(λ̂) − g(λ)} −
Hence !
√ d λ3 (2 − λ)2 e−2λ
n(η̂ − η) −
→ N 0, .
4
Statistical Inference - Comprehensive Final 6 of 10

Solution to Problem 3

The unrestricted likelihood is

  
n m
1 X
 
L = (2πσ 2 )−(n+m)/2 exp − 2  (Xi − µ1 )2 +
X
(Yj − µ2 )2  .
 2σ 
i=1 j=1

Under the unrestricted model,

µ̂1 = X̄, µ̂2 = Ȳ ,
and
SSEU
σ̂ 2 = ,
n+m
where n m
X X
2
SSEU = (Xi − X̄) + (Yj − Ȳ )2 .
i=1 j=1

Under H0 : µ1 = µ2 = µ, the restricted MLE of the common mean is the pooled mean

nX̄ + mȲ
µ̂0 = Z̄ = .
n+m
The restricted error sum of squares is
n
X m
X
SSE0 = (Xi − Z̄)2 + (Yj − Z̄)2 .
i=1 j=1

The restricted MLE of σ 2 is SSE0 /(n + m). Therefore

(n+m)/2
SSEU

Λ= .
SSE0
Now decompose
nm
SSE0 = SSEU + (X̄ − Ȳ )2 .
n+m
Thus #−(n+m)/2
nm
− Ȳ )2
"
n+m (X̄
Λ= 1+ .
SSEU
Let
SSEU
Sp2 = .
n+m−2
The pooled two-sample t statistic is
X̄ − Ȳ
T = p .
Sp 1/n + 1/m
Since
nm (X̄ − Ȳ )2
(X̄ − Ȳ )2 = ,
n+m 1/n + 1/m
we have
nm
n+m (X̄ − Ȳ )2 T2
= .
SSEU n+m−2
Statistical Inference - Comprehensive Final 7 of 10

Hence !−(n+m)/2
T2
Λ= 1+ .
n+m−2
Because this is decreasing in T 2 , the LRT rejects for large |T |. Under H0 ,

T ∼ tn+m−2 .

Thus the exact size α LRT rejects when

|T | > tn+m−2,1−α/2 .

The asymptotic Wald test uses

µ̂1 − µ̂2 = X̄ − Ȳ ,
with estimated variance
1 1
 
Sp2 + .
n m
Thus the Wald statistic is
(X̄ − Ȳ )2
W = = T 2.
Sp2 (1/n + 1/m)
For large samples,
d
→ χ21 .
W −
Statistical Inference - Comprehensive Final 8 of 10

Solution to Problem 4

The density of one observation is

1
f (x; θ) = xα−1 e−x/θ , x > 0.
Γ(α)θα

Let n
X
T = Xi .
i=1
Ignoring constants not involving θ,
T
ℓ(θ) = −nα log θ − .
θ
The score is
nα T
U (θ) = ℓ′ (θ) = − + 2.
θ θ
The likelihood equation is
−nαθ + T = 0,
so
T
θ̂ = .
nα
The second derivative is
nα 2T
ℓ′′ (θ) = − 3.
θ2 θ
Since E(T ) = nαθ,
nα
In (θ) = −E[ℓ′′ (θ)] = .
θ2
Thus I(θ) = α/θ2 per observation.
For H0 : θ = θ0 , the Wald statistic is

nα(θ̂ − θ0 )2
Wn = In (θ̂)(θ̂ − θ0 )2 = .
θ̂2
The score statistic is
2
U (θ0 )2 −nα/θ0 + T /θ02 (T − nαθ0 )2
Rn = = = .
In (θ0 ) nα/θ02 nαθ02
The likelihood ratio statistic is
LRn = 2{ℓ(θ̂) − ℓ(θ0 )}.
Because T = nαθ̂, !
θ̂ 1 1
 
ℓ(θ̂) − ℓ(θ0 ) = −nα log −T − .
θ0 θ̂ θ 0

Since T /θ̂ = nα, " ! #

θ̂ θ̂
ℓ(θ̂) − ℓ(θ0 ) = nα − log −1+ .
θ0 θ0
Therefore " !#
θ̂ θ̂
LRn = 2nα − 1 − log .
θ0 θ0
Statistical Inference - Comprehensive Final 9 of 10

d
→ χ21 under H0 .
Under regularity conditions, Wn , Rn , LRn −
Now, since Xi ∼ Gamma(α, θ0 ) under H0 ,

T ∼ Gamma(nα, θ0 ).

A gamma variable with shape k and scale θ0 satisfies

2T
∼ χ22k .
θ0
Thus
2T
∼ χ22nα .
θ0
This gives an exact test. For a two-sided alternative, the LRT rejects when T is too small or too large
relative to nαθ0 ; the exact critical constants are chosen from the chi-square distribution of 2T /θ0 .
The exact chi-square test is finite-sample exact. The asymptotic likelihood ratio test uses
d
→ χ21 .
LRn −

For large n, both procedures reject for values of T far from nαθ0 , and the rejection regions become
asymptotically equivalent.
Statistical Inference - Comprehensive Final 10 of 10

Solution to Problem 5

The MLE of p is
p̂ = X̄.
By functional invariance,
p̂
 
ψ̂ = log .
1 − p̂
Assume 0 < p̂ < 1.
By the CLT,
√ d
n(p̂ − p) −
→ N (0, p(1 − p)).
Let
p
 
g(p) = log .
1−p
Then
1 1 1
g ′ (p) = + = .
p 1−p p(1 − p)
By the Delta method,
√ 1
   
d  ′ 2
n(ψ̂ − ψ) −
→ N 0, g (p) p(1 − p) = N 0, .
p(1 − p)

Thus an estimated standard error for ψ̂ is

s
1
SE(
d ψ̂) = .
np̂(1 − p̂)

An approximate 100(1 − α)% Wald confidence interval for ψ is

s
1
ψ̂ ± z1−α/2 .
np̂(1 − p̂)

To convert this into a confidence interval for p, use the inverse logit transformation

eψ
p= .
1 + eψ
If the confidence interval for ψ is [L, U ], then the corresponding interval for p is
" #
eL eU
L
, .
1+e 1 + eU

Because the inverse logit is increasing, the endpoints transform directly.