1 MFCS 2026: Problem Set I
1. Consider a probability distribution on the X − Y plane with a probability density function f XY ( x, y)
that satisfies the following conditions:
a The density is zero outside the rectangle defined by (0, 0), ( a, 0), (0, b), ( a, b).
b For a given x value, the density the density f XY ( x, y) remains the same for all y.
c The density at ( x, y) is proportional to the x value and f XY (0, 0) = 0.
Solve the following
1. Find f X ( x ), f Y (y), f XY ( x, y), the distribution functions FX ( x ), FY (y), FXY ( x, y); and statistical
parameters E( X ), E(Y ), Var ( X ), Var (Y ) and Cov( X, Y ).
2. Find the mean and variance of the random variable Z = X + Y. Find f Z (z) and FZ (z). Find
Pr ( Z ≤ min{ a, b}).
3. Suppose that condition (a) is changed so that the density is zero outside the triangle
(0, 0), ( a, 0), (0, b), what will be the density function.
X −Y
4. Let U = X + Y
2 and V = 2 . Find f UV ( u, v ), f U ( u ), f V ( v ), E (V ), Var (U ), Var (V ), Cov (U, V ).
5. Suppose Condition (c) of f XY is changed to “The density at ( x, y) is proportional to the dis-
tance of x from a, and f XY ( a, 0) = 0, ” find f X ( x ), f Y (y), f XY ( x, y) and the distribution func-
tions FX ( x ), FY (y), FXY ( x, y).
2. Let X, Y be jointly distributed random variables with mean µ X and µY . Let c, d be constants. Prove
the following.
• Var ( X + c) = Var ( X ).
• Cov( X, Y ) = Cov( X − c, Y − d).
• Cov( aX, bY ) = abCov( X, Y )
3. Let a, b be two vectors in Rn . Find the scalar value α such that the length || a − αb||2 is minimized.
(Hint: Differentiate with respect to α to find the minimum. You will get an expression in terms of
( a, b) and ||b||. This is a standard technique that will be useful through the next few questions.)
4. This question must be attempted after the previous question. Suppose X, Y be jointly distributed
random variables. Suppose we want to find a real number α that minimizes E(( X − αY )2 ). Let
Cov( X, Y ) = σXY , Var ( X ) = σX2 and Var (Y ) = σY2 . Find the optimal α in terms of σXY and σY2 .
5. Let X be an arbitrary random variable. Suppose we want to approximate X with a constant c. One
of the natural choices for optimizing the error is to minimize the mean squared error E( X − c)2 .
Show that the optimal value of c that minimizes the mean squared error is c = E( X ).
6. Let X, Y be arbitrary random variables. Suppose we want to approximate X with an expression
of the form αY + β for some real numbers α, β. As with the previous question, we would like to
determine α, β that minimizes the mean squared error E( X − αY − β)2 . Find the optimal values of
α, β. (Hint: From the previous question, if the value of α is given, the the value of β must be E( X −
αY ), reducing the problem to a single variable optimization over the random variables X − E( X )
and Y − E(Y ), which was solved earlier.)
7. Let X be Gaussian with mean µ and variance σ2 . Let t > 0 be any real number. Consider the
1 2
random variable etX . Show that E(etX ) = etµ+ 2 (tσ) . (Hint: After suitable transformations, you will
be able to reduce it to the evaluation of a standard Gaussian integral.)
8. This question uses the previous question. Let X be Gaussian with mean µ and variance σ2 . Let
Y = X − µ and let ϵ > 0.
tσ2
• Show that for any t > 0 Pr (Y > ϵ) = Pr (etY > etϵ ) < e 2 −tϵ . (Hint: First note that etY is
monotone increasing, injective and always positive. Hence use Markov’s inequality.)
tσ2 − ϵ2
• Find the value of t minimizing e 2 −tϵ . Hence conclude that Pr ( X − µ > ϵ) < e 2σ2 . This shows
that the probability of a Gaussian deviating far from mean decreases exponentially fast.