DA1004-Problem Set 3

April 18, 2025

1. Consider the following variant of Buffon’s needle problem, which was investigated by
Laplace. A needle of length l is dropped on a plane surface that is partitioned in
rectangles by horizontal lines that are a apart and vertical lines that are b apart.
Suppose that the length of the needle l satisfies l < a and l < b.

• What is the expected number of rectangle sides crossed by the needle?

• What is the probability that the needle will cross at least one side of some rect-
angle?

2. Estimating an expected value by simulation using samples of another ran-

dom variable. Let Y1 , . . . , Yn be independent random variables drawn from a common
and known PDF fY . Let S be the set of all possible values of Yi , S = {y | fY (y) > 0}.
Let X be a random variable with known PDF fX , such that fX (y) = 0, for all y ∈ / S.
Consider the random variable
n
1 X fX (Yi )
Z= Yi .
n i=1 fY (Yi )

Show that
E[Z] = E[X].

3. We have a stick of unit length, and we consider breaking it into three pieces using one
of the following three methods:

(i) We choose randomly and independently two points on the stick using a uniform
PDF, and we break the stick at these two points.
(ii) We break the stick at a random point chosen by using a uniform PDF, and then
we break the piece that contains the right end of the stick at a random point
chosen by using a uniform PDF.
(iii) We break the stick at a random point chosen by using a uniform PDF, and then
we break the larger of the two pieces at a random point chosen by using a uniform
PDF.

For each of the methods (i), (ii), and (iii), what is the probability that the three pieces
we are left with can form a triangle?

1
4. Let the random variables X and Y have a joint PDF which is uniform over the triangle
with vertices at (0, 0), (0, 1), and (1, 0).
(a) Find the joint PDF of X and Y .
(b) Find the marginal PDF of Y .
(c) Find the conditional PDF of X given Y .
(d) Find E[X | Y = y], and use the total expectation theorem to find E[X] in terms
of E[Y ].
(e) Use the symmetry of the problem to find the value of E[X].
5. Let X, Y , and Z be three random variables with joint PDF fX,Y,Z . Show the multi-
plication rule:
fX,Y,Z (x, y, z) = fX|Y,Z (x | y, z)fY |Z (y | z)fZ (z).

6. Estimating an expected value by simulation. Let fX (x) be a PDF such that for
some nonnegative scalars a, b, and c, we have fX (x) = 0 for all x outside the interval
[a, b], and xfX (x) ≤ c for all x. Let Yi , i = 1, . . . , n, be independent random variables
with values generated as follows: a point (Vi , Wi ) is chosen at random (according to
a uniform PDF) within the rectangle whose corners are (a, 0), (b, 0), (a, c), and (b, c),
and if Wi ≤ Vi fX (Vi ), the value of Yi is set to 1, and otherwise it is set to 0. Consider
the random variable
Y1 + · · · + Yn
Z= .
n
Show that
E[X] 1
E[Z] = and var(Z) ≤ .
c(b − a) 4n
In particular, we have var(Z) → 0 as n → ∞.
7. The sum of a random number of random variables. You visit a random number
N of stores and in the ith store, you spend a random amount of money Xi . Let
T = X1 + X 2 + · · · + XN
be the total amount of money that you spend. We assume that N is a positive integer
random variable with a given PMF, and that the Xi are random variables with the
same mean E[X] and variance var(X). Furthermore, we assume that N and all the Xi
are independent. Show that
E[T ] = E[X]E[N ], and var(T ) = var(X)E[N ] + (E[X])2 var(N ).

8. A defective coin minting machine produces coins whose probability of heads is a random
variable P with PDF (
pep , p ∈ [0, 1],
fP (p) =
0, otherwise.
A coin produced by this machine is selected and tossed repeatedly, with successive
tosses assumed independent.

2
 (a) Find the probability that a coin toss results in heads.
(b) Given that a coin toss resulted in heads, find the conditional PDF of P .
(c) Given that the first coin toss resulted in heads, find the conditional probability
of heads on the next toss.

9. Let X and Y be independent continuous random variables with PDFs fX and fY ,

respectively, and let Z = X + Y .

(a) Show that fZ|X (z | x) = fY (z − x). Hint: Write an expression for the conditional
CDF of Z given X, and differentiate.
(b) Assume that X and Y are exponentially distributed with parameter λ. Find the
conditional PDF of X, given that Z = z.
(c) Assume that X and Y are normal random variables with mean zero and variances
σx2 and σy2 , respectively. Find the conditional PDF of X, given that Z = z.

10. The metro train arrives at the station near your home every quarter hour starting at
6:00 a.m. You walk into the station every morning between 7:10 and 7:30 a.m., with
the time in this interval being a random variable with given PDF. Let X be the elapsed
time, in minutes, between 7:10 and the time of your arrival. Let Y be the time that
you have to wait until you board a train. Calculate the CDF of Y in terms of the CDF
of X and differentiate to obtain a formula for the PDF of Y .

11. Let X and Y be independent random variables, uniformly distributed in the interval
[0, 1]. Find the CDF and the PDF of |X − Y |.

12. Let X and Y be the Cartesian coordinates of a randomly chosen point (according to a
uniform PDF) in the triangle with vertices at (0, 1), (0, −1), and (1, 0). Find the CDF
and the PDF of |X − Y |.

13. Two points are chosen randomly and independently from the interval [0, 1] according
to a uniform distribution. Show that the expected distance between the two points is
1/3.

14. Find the PDF of Z = X + Y , when X and Y are independent exponential random
variables with common parameter λ.

15. Let X and Y be independent random variables with PMFs


( 
 1/2, if y = 0,

1/3, if x = 1, 2, 3,  1/3, if y = 1,
pX (x) = pY (y) =
0, otherwise, 

 1/6, if y = 2,

0, otherwise.

Find the PMF of Z = X + Y , using the convolution formula.

3
16. Schwarz inequality. Show that for any random variables X and Y , we have

(E[XY ])2 ≤ E[X 2 ]E[Y 2 ].

17. Suppose that X and Y are random variables with the same variance. Show that X − Y
and X + Y are uncorrelated.

18. Consider four random variables, W , X, Y , Z, with

E[W ] = E[X] = E[Y ] = E[Z] = 0,

var(W ) = var(X) = var(Y ) = var(Z) = 1,

and assume that W , X, Y , Z are pairwise uncorrelated. Find the correlation coeffi-
cients ρ(R, S) and ρ(R, T ), where

R = W + X, S = X + Y, and T = Y + Z.

19. A retired professor comes to the office at a time which is uniformly distributed between
9 a.m. and 1 p.m., performs a single task, and leaves when the task is completed. The
duration of the task is exponentially distributed with parameter
1
λ(y) = ,
5−y
where y is the length of the time interval between 9 a.m. and the time of his arrival.

(a) What is the expected amount of time that the professor devotes to the task?
(b) What is the expected time at which the task is completed?
(c) The professor has a Ph.D. student who on a given day comes to see him at a
time that is uniformly distributed between 9 a.m. and 5 p.m. If the student does
not find the professor, he leaves and does not return. If he finds the professor,
he spends an amount of time that is uniformly distributed between 0 and 1 hour.
The professor will spend the same total amount of time on his task regardless of
whether he is interrupted by the student.
What is the expected amount of time that the professor will spend with the
student and what is the expected time at which he will leave his office?

20. We toss n times a biased coin whose probability of heads, denoted by q, is the value of
a random variable Q with given mean µ and positive variance σ 2 . Let Xi be a Bernoulli
random variable that models the outcome of the ith toss (i.e., Xi = 1 if the ith toss is
a head). We assume that X1 , . . . , Xn are conditionally independent, given Q = q. Let
X be the number of heads obtained in the n tosses.

(a) Find E[Xi ] and E[X].

(b) Find cov(Xi , Xj ). Are X1 , . . . , Xn independent?

4
 (c) Use the law of total variance to find var(X). Verify your answer using the covari-
ance result of part (b).

21. The Bivariate Normal PDF. The (zero mean) bivariate normal PDF is of the form
fX,Y (x, y) = ce−q(x,y) ,
where the exponent term q(x, y) is a quadratic function of x and y,
,
x2 2ρxy y 2
q(x, y) = 2 − + 2 2(1 − ρ2 ),
σx σx σy σy

σx and σy are positive constants, ρ is a constant that satisfies −1 < ρ < 1, and c is a
normalizing constant.

(a) By completing the square, rewrite q(x, y) in the form (αx − βy)2 + γy 2 , for some
constants α, β, and γ.
(b) Show that X and Y are zero mean normal random variables with variance σx2 and
σy2 , respectively.
(c) Find the normalizing constant c.
(d) Show that the conditional PDF of X given that Y = y is normal, and identify its
conditional mean and variance.
(e) Show that the correlation coefficient of X and Y is equal to ρ.
(f) Show that X and Y are independent if and only if they are uncorrelated.
(g) Show that the estimation error E[X | Y ] − X is normal with mean zero and
variance (1 − ρ2 )σx2 , and is independent from Y .

22. The polar coordinates of two independent normal random variables. Let X
and Y be independent standard normal random variables. The pair (X, Y ) can be
described in polar coordinates in terms of random variables R ≥ 0 and Θ ∈ [0, 2π], so
that
X = R cos Θ, Y = R sin Θ.

(a) Show that Θ is uniformly distributed in [0, 2π], that R has the PDF
2 /2
fR (r) = re−r , r ≥ 0,
and that R and Θ are independent. (The random variable R is said to have a
Rayleigh distribution.)
(b) Show that R2 has an exponential distribution with parameter 1/2.

23. A stock market trader buys 100 shares of stock A and 200 shares of stock B. Let X
and Y be the price changes of A and B, respectively, over a certain time period, and
assume that the joint PMF of X and Y is uniform over the set of integers x and y
satisfying:
−2 ≤ x ≤ 4, −1 ≤ y − x ≤ 1.

5
 (a) Find the marginal PMFs and the means of X and Y .
(b) Find the mean of the trader’s profit.

26. PMF of the minimum of several random variables. On a given day, your golf
score takes values from the range 101 to 110, with probability 0.1, independent of other
days. Determined to improve your score, you decide to play on three different days
and declare as your score the minimum X of the scores X1 , X2 , and X3 on the different
days.

(a) Calculate the PMF of X.

(b) By how much has your expected score improved as a result of playing on three
days?

27. The multinomial distribution. A die with r faces, numbered 1, . . . , r, is rolled a

fixed number of times n. The probability that the ith face comes up on any one roll is
denoted pi , and the results of different rolls are assumed independent. Let Xi be the
number of times that the ith face comes up.

(a) Find the joint PMF pX1 ,...,Xr (k1 , . . . , kr ).

(b) Find the expected value and variance of Xi .
(c) Find E[Xi Xj ] for i ̸= j.

28. Computational problem. Here is a probabilistic method for computing the area of a
given subset S of the unit square. The method uses a sequence of independent random
selections of points in the unit square [0, 1] × [0, 1], according to a uniform probability
law. If the ith point belongs to the subset S, the value of a random variable Xi is set
to 1, and otherwise it is set to 0. Let X1 , X2 , . . . be the sequence of random variables
thus defined, and for any n, let
X1 + X2 + · · · + Xn
Sn = .
n
(a) Show that E[Sn ] is equal to the area of the subset S, and that var(Sn ) diminishes
to 0 as n increases.
(b) Show that to calculate Sn , it is sufficient to know Sn−1 and Xn , so the past values
of Xk , k = 1, . . . , n − 1, do not need to be remembered. Give a formula.
(c) Write a computer program to generate Sn for n = 1, 2, . . . , 10000, using the com-
puter’s random number generator, for the case where the subset S is the circle
inscribed within the unit square. How can you use your program to measure
experimentally the value of π?
(d) Use a similar computer program to calculate approximately the area of the set of
all (x, y) that lie within the unit square and satisfy 0 ≤ cos(πx) + sin(πy) ≤ 1.

29. A spider and a fly move along a straight line. At each second, the fly moves
a unit step to the right or to the left with equal probability p, and stays where it is

6
 with probability 1 − 2p. The spider always takes a unit step in the direction of the
fly. The spider and the fly start D units apart, where D is a random variable taking
positive integer values with a given PMF. If the spider lands on top of the fly, it’s the
end. What is the expected value of the time it takes for this to happen?

30. Entropy and uncertainty. Consider a random variable X that can take n values,
x1 , . . . , xn , with corresponding probabilities p1 , . . . , pn . The entropy of X is defined to
be n
X
H(X) = − pi log pi .
i=1

(All logarithms in this problem are with respect to base two.) The entropy H(X)
provides a measure of the uncertainty about the value of X. To get a sense of this, note
that H(X) ≥ 0 and that H(X) is very close to 0 when X is “nearly deterministic,”
i.e., takes one of its possible values with probability very close to 1 (since we have
p log p ≈ 0 if either p ≈ 0 or p ≈ 1).
The notion of entropy is fundamental in information theory, which originated with C.
Shannon’s famous work and is described in many specialized textbooks. For example,
it can be shown that H(X) is a lower bound to the average number of yes-no questions
(such as “is X = x1 ?” or “is X < x5 ?”) that must be asked in order to determine
the value of X. Furthermore, if k is the average number of questions required to
determine the value of a string of independent identically distributed random variables
X1 , X2 , . . . , Xn , then, with a suitable strategy, k/n can be made as close to H(X) as
desired, when n is large.
Pn
(a) Show that if q1 , . . . , qn are nonnegative numbers such that i=1 qi = 1, then
n
X
H(X) ≤ − pi log qi ,
i=1

with equality if and only if pi = qi for all i. As a special case, show that H(X) ≤
log n, with equality if and only if pi = 1/n for all i.
Hint: Use the inequality ln α ≤ α − 1, for α > 0, which holds with equality if and
only if α = 1; here ln α stands for the natural logarithm.
(b) Let X and Y be random variables taking a finite number of values, and having
joint PMF pX,Y (x, y). Define
 
XX pX,Y (x, y)
I(X, Y ) = pX,Y (x, y) log .
x y
pX (x)pY (y)

Show that I(X, Y ) ≥ 0, and that I(X, Y ) = 0 if and only if X and Y are
independent.
(c) Show that
I(X, Y ) = H(X) + H(Y ) − H(X, Y ),

7
 where XX
H(X, Y ) = − pX,Y (x, y) log pX,Y (x, y),
x y
X X
H(X) = − pX (x) log pX (x), H(Y ) = − pY (y) log pY (y).
x y

(d) Show that

I(X, Y ) = H(X) − H(X|Y ),
where X X
H(X|Y ) = − pY (y) pX|Y (x|y) log pX|Y (x|y).
y x

Note: H(X|Y ) may be viewed as the conditional entropy of X given Y , that

is, the entropy of the conditional distribution of X given that Y = y, averaged
over all possible values y. Thus, the quantity I(X, Y ) = H(X) − H(X|Y ) is the
reduction in the entropy (uncertainty) on X, when Y becomes known. It can be
therefore interpreted as the information about X that is conveyed by Y , and is
called the mutual information of X and Y .