Tutorial on Probability and Statistics
COMP64101
COMP 64102 - Weeks 1 and 2
October 23and
February and74, 2025
1 Probability
(P1) Answer the following.
– Under what circumstances is an event A independent of itself?
– By considering events concerned with independent tosses of a red die and a blue die, give examples
of events A, B and C which are not independent, but nevertheless are such that every pair of them
is independent.
– By considering events concerned with three independent tosses of a coin, and supposing that A
and B both represent tossing a head on the first trial, give examples of events A, B and C which
are such that P(ABC) = P(A)P(B)P(C) although no pair of them is independent.
(P2) Whether certain mice are black or brown depends on a pair of genes, each of which is either B or b.
If both members of the pair are alike, the mouse is said to be homozygous, and if they are different it
is said to be heterozygous. The mouse is brown only if it is homozygous bb. The offspring of a pair
of mice have two such genes, one from each parent, and if the parent is heterozygous, the inherited
gene is equally likely to be B or b. Suppose that a black mouse results from a mating between two
heterozygotes.
– What are the probabilities that this mouse is homozygous and that it is heterozygous? Now
suppose that this mouse is mated with a brown mouse, resulting in seven offspring, all of which
turn out to be black.
– Use Bayes’ Theorem to find the probability that the black mouse was homozygous BB.
– Recalculate the same probability by regarding the seven offspring as seven observations made
sequentially, treating the posterior after each observation as the prior for the next.
(P3) Suppose that X ∼ Bin(N, µ), wheren N is large and µ is small, but N−λ
µ = λ has an intermediate value.
Use the exponential limit 1 + nk → ek to show that P(X = 0) ∼ = e and P(X = 1) ∼ = λe−λ . Extend
this result to show that X is such that x
∼ λ e−λ ,
p(x) =
x!
that is, X is approximately distributed as a Poisson variable of mean λ.
(P4) Suppose that the random variable X has a negative binomial distribution, X ∼ NegBin(N, µ), so that
N +x−1 n
p(x) = µ (1 − µ)x .
x
Find the mean and variance of X.
1
(P5)
(P4) Let X and Y be continuous random variables defined on the same probability space. Call pX and pY
their respective marginal
R densities, pX,YR the joint density, and pX|Y the
R conditional density of X given
Y . Let E(X) := xpX (x)dx, E(Y ) := ypY (y)dy, and E(X | Y ) := xpX|Y (x | y)dx. Assume that
they all exist and are integrable. Then, show the tower property, that is,
E [E(X | Y )] = E(X).
2 Statistics
(S1) Suppose we are given the following twelve observations from a Normal distribution:
15.644, 16.437, 17.287, 14.448, 15.308, 15.169, 18.123, 17.635, 17.259, 16.311, 15.390, 17.252.
and we are told that the variance σ 2 = 1. Find a 90% HDR for the posterior distribution of the mean
assuming the usual reference prior, i.e. a Uniform.
(S2) With the same data as in the previous question, what is the predictive distribution for a possible future
observation x̃?
(S3) Compute the MLE for a univariate Normal likelihood N (µ, σ 2 ). Now, put on your Bayesian hat and
assume that σ 2 is known, but µ is not. Define the conjugate prior, obtain the posterior, and find the
MAP estimator.
(S4) Suppose that your prior beliefs about the probability µ of success in Bernoulli trials have mean 1/3 and
variance 1/32. Give a 95% posterior HDR for µ given that you have observed 8 successes in 20 trials.
(S4) Suppose the data is modeled as i.i.d. Exp(θ), and the prior is θ ∼ Gamma(a, b), i.e.
(S5)
ba a−1 −bθ
p(θ) = θ e .
Γ(a)
We know that the posterior is
p(θ | x1 , . . . , xn ) = Gamma(a′ , b′ ),
Pn
where a′ = a + n and b′ = b + i=1 xi . What is the posterior predictive density p(xn+1 |x1 , . . . , xn )?
What is the marginal likelihood p(x1 , . . . , xn )?