Tutorial 6 – MTL390

Problems

1. Let X1 , X2 , . . . , Xn be a random sample from the probability density function:

fθ (x) = exp{−(x − θ)}, x>θ

Show that the first order statistic X(1) is a complete sufficient statistic and is independent of the
sample variance S 2 .

2. Let X ∼ NB(1; θ) and define

d(θ) = Pθ (X = 0).
Let X1 , X2 , . . . , Xn be an i.i.d. sample from this distribution. Find the UMVUE of d(θ).

3. Let X be a hypergeometric random variable with probability mass function

 −1   
N M N −M
PM (X = x) = ,
n x n−x

where
max(0, M + n − N ) ≤ x ≤ min(M, n).

(a) Find the UMVUE of M when N is assumed to be known.

(b) Does there exist an unbiased estimator of N when M is known?

4. A random sample of size n is taken from the lognormal distribution with pdf
 
1 −1 1 2
f (x; µ, σ) = √ x exp − 2 (log x − µ) , x > 0.
σ 2π 2σ

Find the method of moments estimators for µ and σ 2 .

5. Let X1 , X2 , . . . , Xn be a random sample from the distribution with pdf

−1 (x−α)
f (x; α, β) = β −1 e−β , α < x < ∞,

where −∞ < α < ∞ and β > 0.

(a) Find the maximum likelihood estimators (MLEs) of (α, β).

(b) Find the MLE of Pα,β {X1 ≥ 1}.

6. Let X be a Cauchy-distributed random variable with PDF

1 1
f (x; θ) = , −∞ < x < ∞, −∞ < θ < ∞.
π 1 + (x − θ)2

Find the Cramér–Rao lower bound for the estimation of the location parameter θ.

7. Let X1 , X2 , . . . , Xn be i.i.d. from the uniform distribution U (a, b), −∞ < a < b < ∞. Prove that,
using the method of moments, the estimators of a and b are, respectively,
r r
3(n − 1) 2 3(n − 1) 2
â = X̄ − S , b̂ = X̄ + S .
n n

1
 8. Let X1 , X2 , . . . , Xn be a random sample from the geometric distribution with PMF
(
q x (1 − q), x = 0, 1, 2, . . . , 0 ≤ q ≤ 1,
p(x; q) =
0, otherwise.

(a) Find the MLE q̂ of q.

n
X
(b) Show that Xi is a complete sufficient statistic for q.
i=1
(c) Determine the minimum variance unbiased estimator of q.
n
X
9. Let X1 , X2 , . . . , Xn be a random sample from Bernoulli’s B(1, p). Prove that Xi is a complete
i=1
statistic. Also, find a UMVUE for the parameter p and for p2 .

10. Let X1 , X2 , . . . , Xn be a random sample from a Poisson distribution with mean θ. Find the minimum
variance unbiased estimator of θ2 .

11. Consider a random sample of size 8 from the Gamma distribution with PDF
 5
 λ 4 −λx
x e , x > 0,
f (x; λ) = 24
0, elsewhere,

1
where λ > 0. If = 2, find the value of the Cramér–Rao lower bound (CRLB) for the variance of
λ
1
any unbiased estimator of 2 .
λ
12. Consider a 4-faced die with probabilities
1 1 1 1
(2 + θ), (1 − θ), (1 − θ), θ
4 4 4 4
for outcomes A, B, C, and D, respectively. In 200 independent throws of the die, the observed
frequencies for A, B, C, and D are 120, 20, 40, and 20, respectively.

(a) Find the maximum likelihood estimator (MLE) of θ.

(b) Find the Fisher information for θ.

13. Prove that the method of moments estimator is consistent for the estimation of r > 0 in the Gamma
family with density
e−x xr−1
f (x) = , 0 < x < ∞.
Γ(r)

14. (a) Let (X1 , X2 , . . . , Xn ) denote a random sample of size n from a distribution that is N (µ, σ 2 ).
Prove that the sample mean X and the sample variance S 2 are independent.
(b) Find an statistic S that is independent of every ancillary statistic but not complete.

15. Let X1 , X2 , . . . , Xn be an iid sample from a Uniform distribution U (0, θ), where θ > 0. The maxi-
n−1
mum order statistic T = X(n) is a complete and sufficient statistic for θ with pdf fT (t) = ntθn for
0 < t < θ.

(a) Find the Uniformly Minimum Variance Unbiased Estimator (UMVUE) for θ and calculate its
Mean Squared Error (MSE).

2
 (b) Then, consider a class of estimators of the form Wc = cT . Find the constant c that minimizes
the MSE of Wc and show that this optimal biased estimator achieves a strictly lower MSE than
the UMVUE for all n ≥ 1.

16. Suppose we have a single observation vector X = (X1 , X2 , . . . , Xp )T from a p-dimensional multi-
variate normal distribution Np (θ, Ip ), where p ≥ 3. We wish to estimate the mean vector θ.

(a) Calculate the total Mean Squared Error (MSE) of the standard unbiased estimator, θ̂ ub = X.
 
p−2
(b) Consider the James-Stein estimator defined as θ̂ JS = 1 − ||X|| 2 X. Assuming the true
parameter vector is exactly the origin
h (θ i= 0), calculate the total MSE of θ̂ JS .
2 2 1 1
Hint: If θ = 0, ||X|| ∼ χp , and E ||X|| 2 = p−2 .
(c) Draw a conclusion comparing the two estimators when p = 500.

17. Let X1 , X2 , . . . , Xn be a random sample from the normal distribution with both mean and variance
equal to an unknown parameter θ.

(a) Is there a sufficient statistic?

(b) What is the MLE for θ?
(c) What is the Cramér–Rao lower bound?