Scribd
Upload
44 views2 pages

Variance and Covariance of Random Variables

This document provides 14 practice problems related to concepts in probability and statistics including: 1) Jointly distributed random variables, covariance, correlation, and independence. 2) Moment generating functions and their relationship to moments. 3) Transformations of random variables and their distributions. 4) Properties of uncorrelated and independent random variables. 5) Multinomial distributions, expectations, and variances. 6) Bounds on probabilities involving means and variances. The problems cover both theoretical concepts and practical applications involving distributions like uniform, geometric, and normal.

Uploaded by

SHAH YASH RAMESHBHAI
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
44 views2 pages

Variance and Covariance of Random Variables

This document provides 14 practice problems related to concepts in probability and statistics including: 1) Jointly distributed random variables, covariance, correlation, and independence. 2) Moment generating functions and their relationship to moments. 3) Transformations of random variables and their distributions. 4) Properties of uncorrelated and independent random variables. 5) Multinomial distributions, expectations, and variances. 6) Bounds on probabilities involving means and variances. The problems cover both theoretical concepts and practical applications involving distributions like uniform, geometric, and normal.

Uploaded by

SHAH YASH RAMESHBHAI
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

SC222: Tutorial Sheet 4

Problems based on Jointly Distributed Random Variables, Expectation, Vari-


ance, Covariance and the Weak Law of Large Numbers.

Pb 1) The joint PMF of a discrete random vector (X1 , X2 ) is given by the following table

x2 \ x1 −1 0 1
0 1/9 2/9 1/9
1 1/9 2/9 1/9
2 0 1/9 0

a) Determine the covariance of X1 and X2 .


b) Calculate the correlation coefficient ρX1 ,X2 = √ Cov(X1 ,X2 ) of X1 and X2 .
Var(X1 )Var(X2 )

c) Are X1 and X2 independent random variables? Justify your answer.

Pb 2) The moment generating function of a random variable X is a function MX (t) of a free


parameter t, defined by MX (t) = E[etX ] (if it exists).

(i) Compute the moment generating functions for the following distributions.
a) Geometric distribution with parameter p.
b) Uniform distribution over the interval [a, b].
(ii) If the moment generating function exists for a random variable X, then show
that the nth moment about the origin (or E[X n ]) can be found by evaluating the
nth derivative of the moment generating function at t = 0.

Pb 3) Suppose that we have a resistance R. We know that the value of R follows a uniform
law between 900 and 1100 Ω. What is the density of the corresponding conductance
G = 1/R ?

Pb 4) (Universality of the uniform distribution) Let X be a real valued random vari-


able and let U ∼ U ([0, 1]). Since FX : R → [0, 1] is not always one-to-one, therefore,
we define FX−1 as
FX−1 (u) = sup{x ∈ R : FX (x) ≤ u}.
Show that FX−1 (U ) and X has the same distribution.

Pb 5) If X and Y are two independent random variables, then so are g(X) and h(Y ).

Pb 6) Two random variables X and Y are said to be uncorrelated if their covariance is 0.


Suppose X and Y are independent uniformly distributed random variables over the
common interval [0, 1]. Define Z = X + Y and W = X − Y . Show that Z and W are
not independent, but uncorrelated random variables.
Pb 7) Let the joint PDF of random variables X and Y be defined as
π π
fX,Y (x, y) = kcos(x + y) for 0 ≤ x ≤ ,0 ≤ y ≤ .
4 4
Determine the constant k and the marginal probability density functions (fX (x) and fY (y))
of X and Y . Are the random variables X and Y are orthogonal? Justify. (The ran-
dom variables X and Y are said to be orthogonal if the mathematical expectation
E[XY ] = 0.)

Pb 8) Let X and Y be linearly dependent real valued random variables. Show that X and
Y are not independent (in the probability sense.)

Pb 9) (Multinomial Distribution) Let Ω be a sample space associated with a random


experiment E, and let B1 , B2 , .., Bn be a partition of Ω. Assume that we perform m
independent repetitions of the experiment E and that the probability pk = P [Bk ] is
constant from one repetition to another. If Xk denotes the number of times that the
event Bk has occurred among the m repetitions, for k = 1, 2, .., n, then, determine the
joint PMF of the random vector (X1 , X2 , ..., Xn ) and Cov(Xi , Xj ). Also, calculate the
Xn
expectation and variance of the random variable X = n1 Xn .
k=1
√ √
Pb 10) Show that if X ≥ 0 and E(X) = µ then P (X ≥ µ) ≤ µ.
2
Pb 11) Let X have variance σX and Y have variance σY2 . Show that −1 ≤ ρX,Y ≤ 1. Further,
argue that, if ρX,Y = 1 or −1, then X and Y are related by Y = a + bX, where b > 0
if ρX,Y = 1 and b < 0 if ρX,Y = −1.

Pb 12) Consider n independent trials, each of which results in any of the outcomes i, i =
3
X
1, 2, 3, with respective probabilities p1 , p2 , p3 , pi = 1. Let Ni denote the number of
i=1
trials that result in outcome i, and show that Cov(N1 , N2 ) = −np1 p2 . Also explain
why it is intuitive that this covariance is negative.

Pb 13) Suppose that X is a random variable with mean and variance both equal to 20. What
can be said about P [0 ≤ X ≤ 40]?.

Pb 14) From past experience, a professor knows that the test score of a student taking her
final examination is a random variable with mean 75.

(a) Give an upper bound to the probability that a student’s test score will exceed
85.
(b) Suppose in addition the professor knows that the variance of a student’s test
score is equal to 25. What can be said about the probability that a student will
score between 65 and 85?

Common questions

Powered by AI

If X and Y are linearly dependent, they can be expressed as Y = aX + b for real constants a and b. This direct relationship indicates perfect linear predictability of one variable given the other, implying dependence. Consequently, the probability of any value occurring for Y is entirely determined by X, reflecting a non-independent nature as statistical independence requires the joint distribution to equal the product of marginals. Thus, X and Y cannot be independent random variables.

Z and W are uncorrelated if Cov(Z, W) = Cov(X+Y, X-Y) = 0. Since we have Cov(X, X) = Var(X), Cov(X, -Y) = -Cov(X, Y), and similarly for Y, they cancel out leading to a covariance of zero, making them uncorrelated. However, independence implies joint distributions factor as products of marginals, which doesn't necessarily follow from lack of correlation. Such mappings Z = X + Y and W = X - Y exhibit dependent structures, e.g., W = X - Y implies knowing W provides information about Z, hence dependence despite zero covariance.

Chebyshev's inequality provides a bound on the probability that a random variable deviates from its mean. For a random variable X with mean μ and variance σ², Chebyshev's inequality states P(|X - μ| ≥ kσ) ≤ 1/k² for any k > 0. This inequality suggests that no more than 1/k² of values lie more than k standard deviations from the mean. It offers a non-parametric estimate, applying to distributions with finite mean and variance, allowing probabilistic bounds when distribution specifics are unknown.

The transformation F^{-1}_X(U) involves using the inverse cumulative distribution function (CDF) for a random variable X and is applied to a uniform random variable U over [0, 1]. This inverse transform sampling exploits the property that if U is uniformly distributed, then applying the inverse CDF of X generates a random variable with the same distribution as X. Since the uniform distribution is continuous and the CDF is non-decreasing, F^{-1}_X(U) produces a variable equivalent in distribution to X.

In a multinomial distribution, the sum of probabilities of all outcomes equals 1, and the total number of iterations m is constant, so an increase in one category’s count implies a decrease in others. This introduction of dependency leads to a negative covariance between different outcomes, intuitively showing competition for probability mass or counts under a fixed trial scenario. Consequently, if one category occurs more frequently, fewer chances are left for others, explaining negative covariance.

To determine the covariance of X1 and X2, we use the formula Cov(X1, X2) = E[X1X2] - E[X1]E[X2]. First, find E[X1], E[X2], and E[X1X2] using the joint PMF. E[X1] is the sum of x1 * P(X1 = x1, X2 = x2) over all x1, x2 pairs. E[X2] is computed similarly. To find E[X1X2], sum x1 * x2 * P(X1 = x1, X2 = x2) over all pairs. Finally, substitute these expectations into the covariance formula.

The joint PMF of a multinomial distribution with random variables X1, X2,...,Xn resulting from m independent experiments is given by P(X1=x1, X2=x2,...,Xn=xn) = m!/(x1!x2!...xn!) * (p1^x1) * (p2^x2) * ... * (pn^xn). Covariance between any two variables Xi and Xj is -mpi pj for i ≠ j due to negative association, since increasing one Xi implies reducing the number of remaining counts spread among other categories. This follows because ∑i Xi = m, maintaining constant total trials.

To solve for the constant k, integrate the joint PDF over the given bounds 0 ≤ x ≤ π/4, 0 ≤ y ≤ π/4 and set the integral equal to 1 (property of PDFs). Compute integral ∫0^(π/4)∫0^(π/4) kcos(x+y) dy dx, and solve for k. For marginal PDFs, integrate joint PDF fX,Y over one variable. E.g., fX(x) = ∫0^(π/4) kcos(x + y) dy and fY(y) = ∫0^(π/4) kcos(x + y) dx. These integrals yield marginal PDFs for X and Y.

The correlation coefficient ρX,Y equals -1 or 1 when there is a perfect linear relationship between X and Y. Specifically, when ρX,Y = 1, Y = a + bX where b > 0, signifying a perfect positive linear relationship. When ρX,Y = -1, Y = a + bX where b < 0, indicating a perfect negative linear relationship. This shows that the data points lie exactly on a line without scatter, depicting complete predictability from one variable to the other.

The moment generating function of a random variable X, denoted by MX(t), is defined as E[etX]. The existence of MGF facilitates the calculation of moments. Specifically, the nth moment about the origin, E[X^n], can be found by evaluating the nth derivative of MX(t) at t = 0, i.e., E[X^n] = M^(n)(0). MGFs characterize distributions and moments, given that they exist within an interval around t = 0.

You might also like