ST102 Exercise 13
In this exercise you will practise aspects of estimator properties and an example of method
of moments estimation. Note that Question 1 is simply the proof of the mean squared error
decomposition into the variance and bias squared of the estimator – just copy the proof you
have! Question 2 considers the mean and variance of a linear combination of random variables.
Question 3 requires you to explore the properties of an estimator of the parameter of a Bernoulli
distribution. Question 4 requires you to find an unbiased estimator – follow the hint closely.
Finally, Question 5 requires you to derive method of moments estimators for different
parameterisations of the exponential distribution.
You should attempt these questions ahead of the corresponding class. This exercise is not to
be subnitted. It will be covered by your class teacher in your thirteenth class, which will take
place in the week commencing Monday 9 February 2026.
1. Show that if E(θb2 ) < ∞, it holds that:
2
MSE(θ)
b = Var(θ)
b + Bias(θ)
b
where Bias(θ) b − θ.
b = E(θ)
Hint: Yes, this is just the proof on page 26 of the Winter term lecture material. Yes, you
simply need to copy it to answer the question. Yes, this decomposition is important!
2. Let X1 , X2 , . . . , Xk be independent random variables, and a1 , a2 , . . . , ak be constants.
Show that:
k k
P P
(a) E ai Xi = ai E(Xi )
i=1 i=1
k k
a2i Var(Xi ).
P P
(b) Var ai Xi =
i=1 i=1
Hint: use the result in Example 7.6 on page 29 of the Winter term lecture material.
3. Let {X1 , X2 , . . . , Xn } be a random sample from a Bernoulli distribution where:
P (Xi = 1) = π = 1 − P (Xi = 0) for all i = 1, 2, . . . , n.
Let:
X1 + X2 + · · · + Xn
π
b = X̄ =
n
be an estimator of π.
(a) Find the mean squared error of π
b, i.e. MSE(b
π ). Is π
b an unbiased estimator of π? Is
π
b a consistent of π?
1
(b) Let Y = X1 + X2 + · · · + Xn . Find the probability distribution of Y .
Hint: See Section 5.10.5 of the Autumn term lecture material on distributions of
sums of random variables.
(c) Find the sampling distribution of π b = Y /n (which, recall, is simply the probability
distribution of π
b).
Hint: Notice that π b is just a rescaling of Y due to the division by the sample size n.
4.* Suppose that a random variable X has a Poisson distribution with unknown rate
parameter λ, where λ > 0. Find a statistic g(X), i.e. some known function of X, which
will be an unbiased estimator of eλ .
Hint: If E(g(X)) = eλ , then:
∞
X g(x) e−λ λx
= eλ .
x!
x=0
Consider multiplying both sides of this equation by eλ , and use the series expansion of the
exponential function (given at the top of page 77 of the Autumn term lecture material).
5.* Let {X1 , X2 , . . . , Xn } be a random sample from the exponential distribution with
probability density function:
(
µ−1 e−x/µ for x ≥ 0
f (x; µ) =
0 otherwise
where µ > 0 is an unknown parameter.
(a) Compute the mean and the variance of Xi using integration.
(b) Find the method of moments estimator of µ, and compute its mean squared error.
(c) Suppose that we reparameterise the distribution such that the density function is
now written in the form of:
(
λe−λx for x ≥ 0
f (x; λ) =
0 otherwise
(i.e. we set λ = 1/µ). Find the method of moments estimator of λ.