B4 Estimation and Inference I
A. Zisserman, Hilary Term 2007
1. A continuous random variable has the pdf
p(x) = kx2 (1 − x) 0≤x≤1
Compute the following values: (a) k; (b) its mean; (c) its median (NB only an equation for the median is required);
(d) its mode.
2. Joint probabilities for the discrete random variables X, Z are given by
X=1 X=2 X=3
Z =1 0.2 0.07 0.06
Z =2 0.09 0.1 0.01
Z =3 0.02 0.12 0.09
Z =4 0.06 0.08 0.1
(a) Check the normalisation of the joint distribution
(b) Compute the marginal probability distributions for X, Z.
(c) Compute the conditional distribution P (Z|X = 3)
(d) Use Bayes’ rule to compute P (X = 3|Z = 4) using only information in the previous 2 parts; then check the
answer by direct calculation from the table above.
3. (a) Show that cov(x1 , x2 ) = E[x1 x2 ] − E[x1 ]E[x2 ]
(b) If the independent continuous variables x1 , x2 are normally distributed with means and variances µi , σi2 , i =
1, 2, derive the distribution of their sum x1 + x2 . Hence show that the variance of the average of N such
variables each with variance σ 2 is σ 2 /N .
(c) Now suppose that the variables x1 , x2 are normal but not independent, and have correlation coefficient ρ. Find
the covariance and information matrices of the vector variable x = (x1 , x2 )T . Write down the joint density
function including the normalisation constant.
(d) Find the conditional density function p(x2 |x1 ) including the normalisation constant.
4. The data of question 2 could be taken as characterising a noisy sensor with output Z, intended to measure a state
X.
(a) Given that a measurement Z = 3 has been taken, what is the MLE for X?
(b) Now, if conditional probabilities P (Z = i|X = j) are as in part (a) but prior probabilities (obtained from
elsewhere) are as follows:
P (X = 1) = 0.2, P (X = 2) = 0.5, P (X = 3) = 0.3
find the MAP estimate for X.
5. Independent sensors have outputs z1 , z2 , i = 1, 2 measuring a state x which are unbiased and normally distributed
with standard deviation σi , i = 1, 2.
(a) Derive the maximum likelihood estimate of x given single measurements z1 , z2 from each sensor.
(b) If also the state itself is known to have a prior distribution that is normal N (x, σ02 ) what is the posterior density
for x?
(c) If, instead, the prior density is given by the exponential distribution
a exp(−ax) if x ≥ 0
p(x) =
0 otherwise
and only a single sensor reading z1 is made, what is the MAP estimate for x?
(d) Suppose in part b) that
x = 10, σ0 = 0.3, σ1 = σ2 = 0.4
and that the sensor readings are
z1 = 10.5 z2 = 12.0.
What is the MAP estimate for x, based on these data?
1
Partial answers
p
3. c) p(X) = K −1 exp − 12 (x − x̄)T S(x − x̄) with K = 2πσ1 σ2 1 − ρ2 and
1 1/σ12 −ρ/σ1 σ2
S=
1 − ρ2 −ρ/σ1 σ2 1/σ22
3. d) x2 |x1 ∼ N (x̄2 , σ2′ ) with
σ2 p
x̄2 = µ2 + ρ(x1 − µ1 ) and σ2′ = σ2 1 − ρ2 .
σ1
4. a) X̂ = 3; b) X̂ = 2.
5. c)
z1 − aσ12 if z1 ≥ aσ12
x̂ =
0 otherwise