Notes on Random Variables

Probability & Statistics, Winter 2025

Contents:
• Random Variables and their properties
• Probability Distribution Functions
• Mean and Variance of random variables
• Joint Probability Distribution and covariance
• Chebyshev’s Theorem

1 Random Variables and their properties:

Consider the statistical experiment of rolling two dice, where the possible outcomes are ordered pairs of numbers
between 1 to 6 appearing on the upper face of each die. It is a valid statistical experiment with uncertainty, as
the particular outcome before rolling the dice is unknown. The sample space (S or Ω) of the experiment is the
set {(1, 1)(1, 2), . . . , (6, 6)}.
If we want to find the probability of the event E that both the numbers appearing on top must be even, then the
probability is P (E) = |E| 9 1
|S| = 36 = 4 , where E = {(2, 2), (2, 4), . . . , (6, 6)}.
This can also be done by assigning a numerical description to each element of the sample space S or sample point.
Sometimes, it is more important or convincing to think of the numerical value of the desired or favourable sample
points.
Suppose we count how many even numbers are in each of these sample points as a function X. Then X(1, 1) =
0, X(1, 2) = 1, X(4, 6) = 2 and so on.
In general, we can ask about the probability of X taking on different values using the following notation:

• P (X = 0) = 1/4 (1, 3), (3, 3), . . . , (5, 5)

• P (X = 0) = 1/2 (1, 2), (3, 2), . . . , (5, 6)
• P (X = 2) = 1/4 (2, 2), (2, 4), . . . , (6, 6)
• P (X > 2) = 0

Where each possible value of X represents an event that is a subset of the sample space for the given experiment,
and P (X = x) is the corresponding probability of that event.
Definition 1. A random variable is a function that associates a real number with each element in the sample
space, i.e., X : S → R.
We shall use a capital letter, say X, Y or Z, to denote a random variable and its corresponding small letter,
x in this case, for one of its values. Using random variables is a convenient notation technique that assists in
decomposing problems. There are many different types of random variables (indicator, binary, choice, Bernoulli,
etc). The two main families of random variable types are discrete and continuous.
Definition 2. A discrete random variable is a variable that can take only a countable number of distinct values.
For example: Dead/alive, number on a die, Children in a family, No. of defective items in a box, etc.
Example 3. Consider a statistical experiment involving the testing of various types of electronic components.
Suppose 3 such components are tested and checked if they are non-defective (N ) or defective (D). Then the sample
space is S = {N N N, N N D, N DN, . . . , DDD}. To verify the reliability of component production, it is essential
to identify the occurrence of defective components. Subsequently, the random variable X can be viewed as the
values assumed is the number of defective components, when three components are tested. Then, the discrete
random variable X is defined as X : S → {0, 1, 2, 3} ⊊ R.

1
Definition 4. A continuous random variable takes an infinite number of possible values in a given interval. For
example: Height, Weight, blood pressure, real numbers between 1 to 5, etc.

Example 5. Let a point be chosen in a circle C with radius r. Let X denote the distance of the point from the
centre of the circle. Then X is a continuous random variable defined as X : S → [0, r] ⊊ R, where the sample
space S is the set of all points in the circle C.

Note 6. Let X and Y be random variables on the same sample space S. Then for a real number k, we define the
functions on S defined by X + Y , X + k, kX, and XY ,

• (X + Y )(s) = X(s) + Y (s) • (kX)(s) = kX(s)

• (X + k)(s) = X(s) + k • (XY )(s) = X(s)Y (s).

More generally, for any polynomial or exponential function h(x), we define h(X) to be the function on S
defined by [h(X)](s) = h[X(s)]. It can be shown that these are also random variables. (This is trivial in the case
that every subset of S is an event.)
The short notation P (X = a) and P (a ≤ X ≤ b) will be used, respectively, for the probability that ”X maps into
a” and ”X maps into the interval [a, b]”, that is:

P (X = a) = P ({s ∈ S : X(s) = a}) and P (a ≤ X ≤ b) = P ({s ∈ S : a ≤ X(s) ≤ b})

2 Probability Distribution Functions:

A probability distribution shows how the total probability is shared among the possible values of a random
variable. It tells us how likely each value (or range of values) is to occur. Probability distributions help us study
and summarize random outcomes in a clear mathematical way.

2.1 Discrete Probability Distribution:

Definition 7. All possible values of a random variable that it can take, along with their associated probabilities
is called a probability distribution.
There are two types of probability distributions: a probability function is called the Probability Mass Function
(PMF) in the case of discrete probability distributions and the Probability Density Function (PDF) in the case
of continuous probability distributions.

Definition 8. Let X be a discrete random variable. Then a function f (x) = P (X = x) or pX (x) = P (X = x) is

Probability Mass Function (PMF) of the random variable X, if, for each possible outcome x, i) f (x) ≥ 0,
called aP
and ii) f (x) = 1.
x

There are two ways to present the probability distribution graph of X. One is by the bar chart, and the other
is by the histogram shown in the following figure. Observe that the sum of the lengths of the bars in the bar chart
is 1, whereas the sum of the areas of the rectangles in the histogram is 1. One may view the histogram as making
the random variable continuous, where X = 1 means X lies between 0.5 and 1.5.

Figure 1: Probability Mass Function (Bar Chart & Histogram) [3].

2
Example 9. A shipment of 15 similar smart boards to a retail outlet contains 2 that are defective. If UIT makes
a random purchase of 3 of these smart boards for the new classrooms in D block, find the probability distribution
for the number of defectives.
Solution: Let X be the random variable that assumes values x, the possible number of defective smart boards
purchased by UIT. Then x can take values 0, 1, 2. Now,

(20)(133) (2)(13) (2)(13)

f (0) = 15 , f (1) = 1 15 2 , f (2) = 2 15 1 .
(3) (3) (3)
Hence, the probability distribution of the discrete random variable X is

x 0 1 2
f (x) 22 35
12
35
1
35
P 22 12 1
Also, f (x) is a PMF as f (x) ≥ 0, ∀x ∈ R and f (x) = 35 + 35 + 35 = 1.
x
Example 10. Let be the sample space when a pair of fair dice is tossed, and let X and Y be the random variables
denoting the maximum of the numbers appearing, i.e. X(a, b) = max(a, b), and the sum of the numbers, i.e.
Y (a, b) = a + b, respectively. Find the distributions of X and Y .
Solution: The sample space S is an equiprobable space with 36 points of ordered pairs (ai , bj ) such that
1 ≤ ai , bj ≤ 6.
Consider x = 1, 2, . . . , 6 be the outputs of the random variable X. Then, for example, only one point (1, 1)
1
has maximum value 1; hence f (x = 1) = 36 ; three points, (1, 2), (2, 2), (2, 1), have maximum value 2; hence
3
f (x = 2) = 36 and so on.
Accordingly, the following is the distribution f (x) of X:

x 1 2 3 4 5 6 Total
1 3 5 7 9 11
f (x) 36 36 36 36 36 36 1
Now, consider y = 2, . . . , 12 be the outputs of the random variable Y . Then, for example, only one point (1, 1)
1 1
has value 2; hence g(y = 2) = 36 ; two points, (1, 2), (2, 1), have value 3; hence g(y = 3) = 18 and so on.
Accordingly, the following is the distribution pY (y) of Y :

y 2 3 4 5 6 7 8 9 10 11 12 Total
1 1 1 1 5 1 5 1 1 1 1
g(y) 36 18 12 9 36 6 36 9 12 18 36 1

2.2 Continuous Probability Distribution:

Definition 11. Let X be a continuous random variable. Then the function f (X) = P (a ≤ X ≤ b) or pX (x) =
P (a ≤ X ≤ b) is called the Probability Density Function (PDF) of the random variable X, if f (x) ≥ 0 for all
+∞
R
x ∈ R and f (x)dx = 1.
−∞

We shall concern ourselves with computing probabilities for various intervals of continuous random variables
such as P (a < X < b), P (X ≥ c), and so forth. Note that when X is continuous, P (a ≤ X ≤ b) = P (X =
a) + P (a < X < b) + P (X = b) = P (a < X < b). That is, it does not matter whether we include an endpoint of
the interval or not. This is not true, though, when X is discrete.
Although the probability distribution of a continuous random variable cannot be presented in tabular form,
it can be stated as a formula or the PDF f (x) or pX (x). Since X is defined over a continuous sample space, it
is possible for the pX (x) to have a finite number of discontinuities. However, most density functions that have
practical applications in the analysis of statistical data are continuous, and their graphs may take any of several
forms. Because areas will be used to represent probabilities and probabilities are positive numerical values, the
density function must lie entirely above the x axis. A probability density function is constructed so that the
area under its curve bounded by the x axis is equal to 1 when computed over the range of X for which f (x)
or pX (x) is defined. Should this range of X be a finite interval, it is always possible to extend the interval to
include the entire set of real numbers by defining f (x) to be zero at all points in the extended portions of the
interval. In the following figure, the probability that X assumes a value between a and b is equal to the shaded
area under the density function between the ordinates at x = a and x = b, and from integral calculus is given by
Rb
P (a < X < b) = f (x)dx.
a

3
 Figure 2: Probability Density Function on the interval [a, b] or P (a < X < b) [1].

Example 12. Suppose that the error in the reaction temperature, in ℃, for a controlled laboratory experiment
is a continuous random variable X having the distribution function
 x2
f (x) = 3 , −1 < x < 2,
0, elsewhere.

(a) Verify that f (x) is a density function.

(b) Find P (0 < X ≤ 1).

+∞
R −1
R R2
Solution: (a) Obviously, pX (x) ≥ 0 ∀ x ∈ R. Also, by definition, f (x)dx = f (x)dx + f (x)dx +
−∞ −∞ −1
+∞ R2 x2 3
= [ x9 ]2−1 = 8
− (− 19 ) = 1. So the given probability function is a PDF.
R
f (x)dx = 3 dx 9
2 −1
R1 x2 3
(b) P (0 < X ≤ 1) = 3 dx = [ x9 ]10 = 91 .
0

Example 13. The total number of hours, measured in units of 100 hours, that a family runs a vacuum cleaner
over a period of one year is a continuous random variable X that has the distribution function

 x, 0 < x < 1,
f (x) = 2 − x, 1 ≤ x < 2
0, elsewhere.


(a) Verify that f (x) is a density function.

(b) Find the probability that over a period of one year, a family runs their vacuum cleaner between 50 and 100
hours.
+∞
R R1 R2
Solution: (a) Obviously, f (x) ≥ 0 ∀ x ∈ R. Also, by definition, f (x)dx = xdx + (2 − x))dx = 1. So
−∞ 0 1
the given probability function is a PDF.
R1 2
(b) As the total number of hours, measured in units of 100 hours, we obtain P (0.5 < X < 1) = xdx = [ x2 ]10.5 =
0.5
1 (0.5)2
2 − 2 = 0.375.

2.3 Cumulative Distribution Function or CDF:

There are many problems where we may wish to compute the probability that the observed value of a random
variable X will be less than or equal to some real number x. Writing F (x) = P (X ≤ x) for every real number x,
we define F (x) to be the cumulative distribution function of the random variable X.

Definition 14. The cumulative distribution function

P F (x) or FX (x) of a discrete random variable X with prob-
ability distribution f (x) is F (x) = P (X ≤ x) = f (t), for −∞ < x < ∞.
t≤x

4
Similarly, the cumulative distribution function F (x) of a continuous random variable X with density function
Rx
f (x) is F (x) = P (X ≤ x) = f (t)dt, for −∞ < x < ∞.
−∞

As an immediate consequence, for a discrete random variable X taking finite values x1 , x2 , . . . , xn , we can
write f (xk ) = F (xk ) − F (xk−1 ), for 2 ≤ k ≤ n.
Also, for a continuous random variable X, P (a < X < b) = F (b) − F (a) and f (x) = dFdx(x) .
One should pay particular notice to the fact that the cumulative distribution function is a monotone nonde-
creasing function defined not only for the values assumed by the given random variable but for all real numbers,
i.e., F (a) ≤ F (b) whenever a ≤ b. Also, lim F (x) = 0 and lim F (x) = 1.
x→−∞ x→∞

Example 15. Consider the problem stated in Example 9. The corresponding CDF can be calculated as,

 0, x < 0,
 22

F (x) = 35 , 0 ≤ x < 1
34
, 1≤x<2
 35


1, x ≥ 2.
34 22 12
Also, P (X = 1) = F (1) − F (0) = 35 − 35 = 35 . The cumulative distribution function F (x) is shown below.

Figure 3: Discrete cumulative distribution function.

Example 16. For the density function of Example 12, find F (x), and use it to evaluate P (0 < X ≤ 1).

Solution: For −1 < x < 2,

Zx Zx
t2 t3 x3 + 1
f (t)dt = dt = [ ]x−1 = .
3 9 9
−∞ −1

Therefore,

 0, x < −1,
F (x) = x3 +1
9 , −1 ≤ x < 2
x ≥ 2.

1,
2 1
Also, P (0 < X ≤ 1) = F (1) − F (0) = 9 − 9 = 91 . The cumulative distribution function F (x) is shown below.

Figure 4: Continuous cumulative distribution function [1].

5
3 Mean or Expectation and Variance of random variables:
3.1 Mean or Expectation:
If two dice are rolled 10 times in total (5 times each) and X is the number of times the number 6 appears, then
the values of X are 0, 1, and 2. Suppose that the experiment yields no time 6, one time 6, and two times 6, a
total of 5, 4, and 1 times, respectively. The average number of 6 appearing per throw of the two dice is then

(0)(5) + (1)(4) + (2)(1)

= 0.6.
10
This is the average value of the data, yet it is not a possible outcome of {0, 1, 2}. Hence, an average is not
necessarily a possible outcome for the experiment.
Let us now restructure our computation for the average number of 6 appearing to have the following equivalent
form:
5 4 1
(0) + (1) + (2) = 0.6.
10 10 10
The numbers 5/10, 4/10, and 1/10 are the fractions of the total throws resulting in 0, 1, and 2 times 6 appearing,
respectively. These fractions represent the relative frequencies of the different values of X in our experiment. In
fact, we can calculate the mean, or average, of a set of data by knowing the distinct values that occur and their
relative frequencies, without needing to know the total number of observations in our set of data. Therefore, if
5/10, or 1/2, of the throws result in 6 appearing 0 time, 2/5 of the throws result in 6 appearing 1 time, and 1/10
of the throws result in 6 appearing 2 time, the mean number of 6 appearing per throw would be 0.6 no matter
whether the total number of throws were 10, 100, or even 1000.
This method of relative frequencies is used to calculate the average number of appearances of the number 6 per
throw of two dice that we might expect in the long run. We shall refer to this average value as the mean of the
random variable X or the mean of the probability distribution of X and write it as µX or simply as µ when it
is clear to which random variable we refer. It is also common among statisticians to refer to this mean as the
mathematical expectation, or the expected value of the random variable X, and denote it as E[X].

Definition 17. Let X be a random variable with probability distribution f (x). The mean, or expected value, of
X is
X
µ = E[X] = xf (x), when X is discrete
x

and
Z∞
µ = E[X] = xf (x)dx, when X is continuous.
−∞

Therefore, in mathematical expectation, the expected value is calculated by using the probability distribution
and the mean is usually understood as a “centre” value of the underlying distribution if we use the expected value.

Example 18. A player tosses a fair die. If a prime number, 2, 3, or 5, occurs, the player wins that number of
dollars, but if a nonprime number occurs, the player loses that number of dollars. The distribution of the game
follows:

x 2 3 5 −1 −4 −6 Total
1 1 1 1 1 1
f (x) 36 36 36 36 36 36 1
The negative numbers −1, −4, and −6 correspond to the fact that the player loses if a nonprime number occurs.
The expected value of the game is
1 1 1 1 1 1 1
µ = E[X] = (2) + (3) + (5) + (−1) + (−4) + (−6) = − .
6 6 6 6 6 6 6
Thus, the game is unfavourable to the player, since the expected value E[X] is negative.

6
Example 19. Let X be the random variable that denotes the life in hours of a certain electronic device. The
probability density function is
 20000
x3
, x > 100,
f (x) =
0, elsewhere.
Find the expected life of this type of device.

Solution: Using the definition of E[X], we obtain

Z∞ Z100 Z∞
20000 20000 ∞
µ = E[X] = xf (x)dx = x.0dx + x dx = [− ] = 200.
x 3 x 100
−∞ −∞ 100

Therefore, we can expect this type of device to last, on average, 200 hours.

3.1.1 Properties of Expectation:

• E[c] = c where c is a constant.

• E[aX ± b] = aE[X] ± b.

• E[X + Y ] = E[X] + E[Y ], provided E[X] and E[Y ] exist.

• Expectations preserve linearity which means that E[aX + bY ] = aE[X] + bE[Y ], where X is the random
variable and a, b are constants.

• E[XY ] = E[X]E[Y ], if X and Y are independent random variables.

Theorem 20. Let X be a random variable with probability distribution f (x). The expected value of the random
variable g(X) is
X
µg(x) = E[g(X)] = g(x)f (x), when X is discrete
x

and
Z∞
µg(X) = E[g(X)] = g(x)f (x)dx, when X is continuous.
−∞

3.2 Variance and Standard Deviation:

The mean, or expected value, of a random variable X is of special importance in statistics because it describes
where the probability distribution is centred. By itself, however, the mean does not give an adequate description of
the shape of the distribution. We also need to characterize the variability in the distribution. The most important
measure of variability of a random variable X is obtained by considering g(X) = (X −µ)2 . The quantity is referred
to as the variance of the random variable X or the variance of the probability distribution of X and is denoted
by V ar(X) or the symbol σX 2 . Eventually, σ or σ is the standard deviation (S.D.) of the random variable X.
X

Definition 21. Let X be a random variable with probability distribution f (x) and mean µ. The variance of X is
X
σ 2 = E[(X − µ)2 ] = (x − µ)2 f (x), when X is discrete
x

and
Z∞
2 2
σ = E[(X − µ) ] = (x − µ)2 f (x)dx, when X is continuous.
−∞

The quantity x − µ is called the deviation of an observation from its mean. Since the deviations are squared
and then averaged, σ 2 will be much smaller for a set of x values that are close to µ than it will be for a set of
values that vary considerably from µ.

7
Theorem 22. The variance of a random variable X is σ 2 = E[X 2 ] − µ2 .
Proof. For the discrete case, we can write
σ 2 =E[(X − µ)2 ]
=E[X 2 − 2Xµ + µ2 ]
=E[X 2 ] − E[2Xµ] + E[µ2 ]
=E[X 2 ] − 2µE[X] + µ2 ]
=E[X 2 ] − 2µ2 + µ2
=E[X 2 ] − µ2 .
For the continuous case, the proof is step by step the same, with summations replaced by integrations.

Example 23. Consider the problem stated in Example 18. From the previous calculation, we obtain µ = − 16 .
Now,
1 1 1 1 1 1 91
E[X 2 ] = (2)2 + (3)2 + (5)2 + (−1)2 + (−4)2 + (−6)2 = .
6 6 6 6 6 6 6
Hence,
91 1
σ 2 = E[X 2 ] − µ2 = − (− )2 ≈ 15.14
6 6
and the standard deviation
p
S.D. = σ = V ar(X) ≈ 3.9.
Example 24. The weekly demand for a drinking-water product, in thousands of litres, from a local chain of
efficiency stores is a continuous random variable X having the probability density

2(x − 1), 1 < x < 2,
f (x) =
0, elsewhere.
Find the mean and variance of X.
Solution: Considering the definitions of the mean µ and variance σ 2 , we first obtain,

Z2 Z2 Z2
5
µ = E[X] = 2 x(x − 1))dx = 2 x2 dx − 2 xdx =
3
1 1 1
and
Z2 Z2 Z2
2 2 3 17
E[X ] = 2 x (x − 1))dx = 2 x dx − 2 x2 dx = .
6
1 1 1
Hence,
17 5 1
σ 2 = E[X 2 ] − µ2 = − ( )2 =
6 3 18
and the standard deviation
p
σ= V ar(X) ≈ 0.23.

3.2.1 Properties of Variance:

• V ar(c) = 0 where c is a constant. • V ar(cX) = c2 V ar(X).

• V ar(X + c) = V ar(X). • V ar(aX ± b) = a2 V ar(X).

Standardized Random Variable: Let X be a random variable with mean µ and standard deviation σ > 0.
Then the standardized random variable Z is defined by

X −µ
Z=
σ
Theorem 25. The standardized random variable Z has mean µZ = 0 and standard deviation σz = 1.

8
4 Joint Probability Distribution:
The discussion so far has been restricted to one-dimensional sample spaces, where we recorded the outcomes of
an experiment as values assumed by a single random variable. There will be situations, however, where we may
find it desirable to record the simultaneous outcomes of several random variables.
Let a fair coin be tossed three times.
Define the random variables:

X = number of heads, Y = number of tails.

Since each outcome is equally likely, X + Y = 3. Now, the sample space is represented as,

Outcome X (Heads) Y (Tails)

HHH 3 0
HHT 2 1
HT H 2 1
T HH 2 1
HT T 1 2
T HT 1 2
TTH 1 2
TTT 0 3
Note that each outcome or the sample points in the sample space has equal probability
1
P (s) = , ∀ s ∈ S.
8
Also, the corresponding probability distribution table can be represented as

X\Y 0 1 2 3
1
0 0 0 0 8
3
1 0 0 8 0
3
2 0 8 0 0
1
3 8 0 0 0

Each cell in the table represents the joint probability (P (X = x, Y = y). The sum of all probabilities is 18 + 38 +
3 1
8 + 8 = 1.
Again, for example, if a 10-wheeler is to have its tyres serviced and X represents the number of miles these
tyres have been driven, and Y represents the number of tyres that need to be replaced, then f (15000, 4) is the
probability that the tyres are used over 15000 kms and the truck needs 4 new tyres.

4.1 Discrete Joint Distribution:

If X and Y are two discrete random variables, the probability distribution for their simultaneous occurrence can be
represented by a function with values f (x, y) for any pair of values (x, y) within the range of the random variables
X and Y . It is customary to refer to this function as the joint probability distribution of X and Y . Hence, in the
discrete case, f (x, y) = P (X = x, Y = y); that is, the values f (x, y) give the probability that outcomes x and y
occur at the same time

Definition 26. The function f (x, y) is a joint probabilityPdistribution

P or joint mass function of the discrete
random variables X and Y if i) f (x, y) ≥ 0 ∀(x, y), and ii) f (x, y) = 1.
P Px y
For any region A in the xy plane, P [(X, Y ) ∈ A] = f (x, y).
A

Example 27. Two electives are selected at random by a student from a list that contains 3 computer science
electives, 2 mathematics electives, and 3 AIML electives. If X is the number of AIML electives selected, and Y is
the number of math electives selected, find (a) the joint probability function f (x, y),
(b) P [(X, Y ) ∈ A], where A is the region {(x, y)| x + y ≤ 1}.

Solution: The possible pairs of values (x, y) are (0, 0), (0, 1), (1, 0), (1, 1), (0, 2), and (2, 0).

9
 (a) Here f (0, 0) represents the probability that 2 computer science electives are selected. This can be done in
3

(32) 3
ways and hence f (0, 0) = = 28 . Now, f (0, 1) represents the probability that 1 math elective and
2 (82)
(2)(3) 3
1 computer science elective are selected. Hence, f (0, 1) = 1 8 1 = 14 . Similarly, f (1, 1) represents the
(2)
(3)(2) 3
probability that 1 AIML elective and 1 math elective are selected. Hence, f (1, 1) = 1 8 1 = 14 and so on.
( 2)
By calculating the probabilities for all other cases, we can construct the following probability distribution
table.

X\Y 0 1 2
3 3 1
0 28 14 28

9 3
1 28 14 0

3
2 28 0 0

Also, using the counting principle, we obtain the following formula for the joint mass function,

3

2
3


x y 2−x−y
f (x, y) = 8

,
2
for x = 0, 1, 2; y = 0, 1, 2; and 0 ≤ x + y ≤ 2.

(b) The probability that (X, Y ) fall in the region A is P [(X, Y ) ∈ A] = P (X + Y ≤ 1) = f (0, 0) + f (0, 1) +
3 3 9 9
f (1, 0) = 28 + 14 + 28 = 14 .

Note 28. Verify that the given joint probability distribution function is a joint mass function!!

4.2 Continuous Joint Distribution:

When X and Y are continuous random variables, the joint density function f (x, y) is a surface lying above the
xy plane, and P [(X, Y ) ∈ A], where A is any region in the xy plane, is equal to the volume of the right cylinder
bounded by the base A and the surface.

Definition 29. The function f (x, y) is a joint density function of the continuous random variables X and Y if i)
R∞ R∞
f (x, y) ≥ 0 ∀(x, y), and ii) f (x, y) dx dy = 1.
−∞ −∞ RR
For any region A in the xy plane, P [(X, Y ) ∈ A] = A f (x, y) dx dy.

Example 30. On a randomly selected day, let X and Y , respectively, be the proportions of the time that the
eat-in and parcel facilities are in use in Anticlock cafeteria, and suppose that the joint probability distribution
function of these random variables is
 2
f (x, y) = 3 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
0, elsewhere.

a) Verify that it is a joint density function.

b) Find the probability that, on the day, the eat-in facility is in use for more than three-fourths of the time,
while the parcel facility is in use for less than half of the time.

c) Find the probability that, on the day, the proportion of time the eat-in facility is in use is greater than the
proportion of time the parcel facility is in use.

d) Find the probability that, on the day, the parcel facility window is busy less than one-half of the time.

10
 Solution: (a) Obviously, f (x, y) ≥ 0 ∀ x, y ∈ R. Also, by definition,
Z ∞ Z ∞ Z 1Z 1
2
f (x, y) dx dy = (x + 2y) dx dy
−∞ −∞ 0 0 3
Z 1 Z 1 
2
= (x + 2y) dx dy
0 3 0

11
2 x2

Z
= + 2xy dy
0 3 2 0
Z 1 
1 4y
= + dy
0 3 3
1
y 2y 2

= + =1
3 3 0
So the given probability function is a joint density function.
(b) The probability that, on the day, the eat-in facility is in use for more than three-fourths of the time, while the
parcel facility is in use for less than half of the time is,
  Z 1/2 Z 1
3 1 2
P X> ,Y < = (x + 2y) dx dy
4 2 0 3/4 3
Z 1/2 1 !
2 x2

= + 2yx dy
0 3 2 3/4
Z 1/2  
7 y
= + dy
0 48 3
1/2
7y y 2

11
= + =
48 6 0 96
c) The probability that, on the day, the proportion of time the eat-in facility is in use is greater than the
proportion of time the parcel facility is in use is,
Z 1Z x
2
P (X > Y ) = (x + 2y) dy dx
0 0 3
Z 1
2 x
= xy + y 2 0 dx
0 3
Z 1 Z 1
2 2 4 2
= (2x ) dx = x dx
0 3 0 3

4 3 1 4
 
= x =
9 0 9
d) The probability that, on the day, the parcel facility window is busy less than one-half of the time is,
  Z 1/2 Z 1
1 2
P Y < = (x + 2y) dx dy
2 0 0 3
Z 1/2  
1 4y
= + dy
0 3 3
1/2
y 2y 2

1
= + =
3 3 0 3
Given the joint probability distribution f (x, y) of the random variables X and Y , the probability distribution
g(x) of X alone is obtained by summing f (x, y) over the values of Y . Similarly, the probability distribution h(y)

11
of Y alone is obtained by summing f (x, y) over the values of X. We define g(x) and h(y) to be the marginal
distributions of X and Y , respectively. When X and Y are continuous random variables, summations are replaced
by integrals.
The term marginal is used here because, in the discrete case, the values of g(x) and h(y) are just the marginal
totals of the respective columns and rows when the values of f (x, y) are displayed in a rectangular table.

X\Y y1 y2 ··· ym Sum

x1 f (x1 , y1 ) f (x1 , y2 ) · · · f (x1 , ym ) g(x1 )
x2 f (x2 , y1 ) f (x2 , y2 ) · · · f (x2 , ym ) g(x2 )
.. .. .. .. .. ..
. . . . . .
xn f (xn , y1 ) f (xn , y2 ) · · · f (xn , ym ) g(xn )
Sum h(y1 ) h(y2 ) ··· h(ym )
Definition 31. The marginal distributions of X alone and of Y alone are
X X
g(x) = f (x, y) and h(y) = f (x, y)
y x

for the discrete case, and

Z∞ Z∞
g(x) = f (x, y) dy and h(y) = f (x, y) dx
−∞ −∞

for the continuous case.

Example 32. Consider the problem stated in Example 27. The probability distribution table, considering P (X =
x, Y = y) = f (x, y), can be represented as

X\Y 0 1 2 g(x)
3 3 1 10
0 28 14 28 28
9 3 15
1 28 14 0 28
3 3
2 28 0 0 28
15 12 1
h(y) 28 28 28
Then the marginal distribution of the random variable X alone is,

10
 28 ,
 if x = 0,
g(x) = 1528 , if x = 1,

3
28 , if x = 2,
P
where g(x) = f (x, y).
y
The marginal distribution of the random variable Y alone is,

15
 28 ,
 if y = 0,
h(y) = 12 28 , if y = 1,

1
28 , if y = 2,
P
where h(y) = f (x, y).
x P P
One can also verify that g(x) = 1, and h(y) = 1.
y x

Example 33. Consider the problem stated in Example 30 and the joint density function,

 2 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
f (x, y) = 3
0, elsewhere.
Then the marginal distribution of the random variable X alone is,
Z 1
2
g(x) = (x + 2y) dy
0 3

12
 2 1 2
= xy + y 2 0 = (x + 1), 0≤x≤1
3 3
Hence, g(x) can be represented as

 2 (x + 1), 0 ≤ x ≤ 1,
g(x) = 3
0, elsewhere.

Similarly, the marginal distribution of the random variable Y alone is,

Z 1
2
h(y) = (x + 2y) dx
0 3
1
2 x2

1 4y
= + 2yx = + , 0≤y≤1
3 2 0 3 3
Hence, 
 1 + 4y , 0 ≤ y ≤ 1,
h(y) = 3 3
0, elsewhere.
Note: Verify that both the marginal distributions are pdfs of X and Y , respectively!!

Independent random variables: Let X and Y be two random variables, discrete or continuous, with joint
probability distribution f (x, y) and marginal distributions g(x) and h(y), respectively. The random variables X
and Y are said to be statistically independent if and only if f (x, y) = g(x)h(y) for all (x, y) within their range.
Equivalently, this can be represented as P {X ∈ A, Y ∈ B} = P {X ∈ A}P {Y ∈ B}, i.e., X and Y are independent
if, for all A and B, the events EA = {X ∈ A} and EB = {Y ∈ B} are independent.

4.3 Mean and Covariance of joint distribution:

The concept of mathematical expectation can be extended to the case of two random variables, X and Y , with
the joint probability distribution f (x, y).

Definition 34. Let X and Y be random variables with joint probability distribution f (x, y). The mean, or
expected value, of the random variable g(X, Y ) is
XX
µg(X,Y ) = E[g(X, Y )] = g(x, y)f (x, y),
x y

if X and Y are discrete, and

Z ∞ Z ∞
µg(X,Y ) = E[g(X, Y )] = g(x, y)f (x, y) dx dy
−∞ −∞

if X and Y are continuous.

Example 35. Consider the problem stated in Example 27. Recall that the joint mass function is given by,
3 2


3

x y 2−x−y
f (x, y) = 8

,
2
for x = 0, 1, 2; y = 0, 1, 2; and 0 ≤ x + y ≤ 2.
As given, there are 3 AIML electives, 2 Mathematics electives, and 3 Computer Science electives, and two
electives are selected at random.
The probability distribution table, considering P (X = x, Y = y) = f (x, y), can be represented as

X\Y 0 1 2 g(x)
3 3 1 10
0 28 14 28 28
9 3 15
1 28 14 0 28
3 3
2 28 0 0 28
15 12 1
h(y) 28 28 28

13
 Therefore, the expected value is X
E(X + Y ) = (x + y)f (x, y).
x,y

3 9 6 6 3 1
E(X + Y ) = 0 · +1· +1· +2· +2· +2·
28 28 28 28 28 28
35 5
== .
28 4
Alternatively, one can use the linearity of expectation E(X + Y ) = E(X) + E(Y ).
Then,
3 3 2 1
E(X) = 2 · = , E(Y ) = 2 · = .
8 4 8 2
3 1 5
E(X + Y ) =
+ = .
4 2 4
So, if a student repeatedly selects two electives at random from the given list, then in the long run, an average
of 1.25 electives selected will be from AIML or Mathematics.
Also, we can calculate the expected value of XY for the discrete random variables X and Y ,
XX
E[XY ] = xy f (x, y).
x y

Since XY = 0 unless X = 1 and Y = 1, we have

E[XY ] = (1)(1)f (1, 1).

Now,
3

2

1 6 3
f (1, 1) = 8

1 = = .
2
28 14
Hence,
3
E[XY ] =
.
14
One can note that we can calculate the expected values of X and Y individually.
The possible values of X are 0, 1 and 2.
5 3 5 3





2 10 1 1 15 2 3
P (X = 0) = 8
= , P (X = 1) = 8
= , P (X = 2) = 8

= .
2
28 2
28 2
28
X
E[X] = xP (X = x)
x
10 15 3
=0·
+1· +2·
28 28 28
21 3
= = .
28 4
Similarly, the possible values of X are 0, 1 and 2.
6 2 6 2





2 15 1 1 12 2 1
P (Y = 0) = 8
= , P (Y = 1) = 8
= , P (Y = 2) = 8

= .
2
28 2
28 2
28
X
E[Y ] = yP (Y = y)
y
15 12 1
=0· +1· +2·
28 28 28
14 1
= = .
28 2
Therefore, E[X]E[Y ] = 34 · 12 = 3
8 ̸= 3
14 = E[XY ] and hence, the random variables X and Y are not independent.

14
Example 36. Consider the example stated in Example 30. We can compute E[X + Y ] directly using the joint
PDF 
 2 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
f (x, y) = 3
0, elsewhere.
Then,
Z 1Z 1
E[X + Y ] = (x + y)f (x, y) dy dx
0 0
Z 1Z 1
2
= (x + y) · (x + 2y) dy dx
0 0 3
Z 1Z 1
2
x2 + 3xy + 2y 2 dy dx


=
3 0 0
2 1
Z Z 1 Z 1 Z 1 
= x2 dy + 3xy dy + 2y 2 dy dx
3 0 0 0 0
2 1
Z  
3x 2
= x2 + + dx
3 0 2 3
1
2 x3 3x2 2x

= + +
3 3 4 3 0
 
2 1 3 2
= + +
3 3 4 3
2 21
= ·
3 12
7
= .
6
Note 1: Verify that E[X + Y ] = E[X] + E[Y ]!! Also, check whether X and Y are independent random
variables!!

4.4 Covariance of random variables:

Recall the properties of variance of a random variable X, and we obtain,

Var(X + X) = Var(2X)
= 22 Var(X)
= 4 Var(X)
̸= Var(X) + Var(X).

This represents that the variance of a random variable does not follow the linearity property, in general.
Therefore, we define the concept of covariance for two random variables X and Y . If g(X, Y ) = (X −µX )(Y −µY ),
where µX = E(X) and µY = E(Y ), the expected value of E[g(X, Y )] yields an expected value called the covariance
of X and Y , which we denote by σXY or Cov(X, Y ).

Definition 37. Let X and Y be random variables with joint probability distribution f (x, y). The covariance of
X and Y is
XX
Cov(X, Y ) = σXY = E[(X − µX )(Y − µY )] = (x − µX )(y − µY )f (x, y)
x y

if X and Y are discrete, and

Z∞ Z∞
Cov(X, Y ) = σXY = E[(X − µX )(Y − µY )] = (x − µX )(y − µY )f (x, y) dx dy
−∞ −∞

if X and Y are continuous.

15
 The covariance between two random variables is a measure of the nature of the association between the two.
If large values of X often result in large values of Y or small values of X result in small values of Y , positive
(X − µX ) will often result in positive (Y − µY ) and negative (X − µX ) will often result in negative (X − µX ).
Thus, the product (X − µX )(Y − µY ) will tend to be positive. On the other hand, if large X values often result
in small Y values, the product (X − µX )(Y − µY ) will tend to be negative. The sign of the covariance indicates
whether the relationship between two dependent random variables is positive or negative. When X and Y are
statistically independent, it can be shown that the covariance is zero. The converse, however, is not generally
true. Two variables may have zero covariance and still not be statistically independent. Note that the covariance
only describes the linear relationship between two random variables. Therefore, if the covariance between X and
Y is zero, X and Y may have a nonlinear relationship, which means that they are not necessarily independent.
A useful expression for Cov(X, Y ) can be obtained by expanding the right side of the definition. This yields

Cov(X, Y ) = E[(X − µX )(Y − µY )]

= E[XY − µX Y − µY X + µX µY ]
= E[XY ] − µX E[Y ] − µY E[X] + µX µY
= E[XY ] − µX µY − µY µX + µX µY
= E[XY ] − E[X]E[Y ].

4.4.1 Properties of covariance:

• Cov(X, X) = V ar(X) • Cov(kX, Y ) = kCov(X, Y ), for constant k

• Cov(X, Y ) = Cov(Y, X) • Cov(X + Z, Y ) = Cov(X, Y ) + Cov(Z, Y )

Note: V ar(X + Y ) = V ar(X) + V ar(Y ) + 2Cov(X, Y ).

Theorem 38. If X and Y are independent random variables, then Cov(X, Y ) = 0.

Example 39. Consider the problem stated in Example 27. From the calculations done in Example 35, we obtain,
3
E[XY ] = 14 , E[X] = 43 and E[Y ] = 12 . Therefore,
3 31 9
Cov(X, Y ) = σXY = E[XY ] − E[X]E[Y ] = − =− .
14 4 2 56
Example 40. Consider the example stated in Example 30. Recall the joint pdf,

 2 (x + 2y), 0 ≤ x ≤ 1, 0 ≤ y ≤ 1,
f (x, y) = 3
0, elsewhere.
Then we obtain the required values to compute the Cov(X, Y ).
Z 1Z 1
E[X] = xf (x, y) dy dx
0 0
Z 1Z 1
2
= x · (x + 2y) dy dx
0 0 3
2 1 1 2
Z Z
= (x + 2xy) dy dx
3 0 0
2 1 2
Z


= x + x dx
3 0
 
2 1 1
= +
3 3 2
5
= .
9

16
 Z 1Z 1
E[Y ] = yf (x, y) dy dx
0 0
Z 1Z 1
2
= y · (x + 2y) dy dx
0 0 3
2 1 1
Z Z
= (xy + 2y 2 ) dy dx
3 0 0
2 1 x 2
Z  
= + dx
3 0 2 3
 
2 1 2
= +
3 4 3
11
= .
18
Z 1Z 1
E[XY ] = xy f (x, y) dy dx
0 0
Z 1Z 1
2
= xy · (x + 2y) dy dx
0 0 3
Z 1Z 1
2
= (x2 y + 2xy 2 ) dy dx
3 0 0
2 1 x2 2x
Z  
= + dx
3 0 2 3
 
2 1 1
= +
3 6 3
1
= .
3
Finally, the covariance of X and Y is,

Cov(X, Y ) = E[XY ] − E[X]E[Y ]

  
1 5 11
= −
3 9 18
1 55
= −
3 162
54 − 55
=
162
1
=− .
162
Although the covariance between two random variables does provide information regarding the nature of the
relationship, the magnitude of σXY does not indicate anything regarding the strength of the relationship, since
σXY is not scale-free. Its magnitude will depend on the units used to measure both X and Y . There is a scale-free
version of the covariance called the correlation coefficient that is used widely in statistics.
Correlation Coefficient: Let X and Y be random variables with covariance σXY and standard deviations
σX and σY , respectively. The correlation coefficient of X and Y is
σXY
ρXY = .
σX σY
It can be noted that σXY is free of the units of X and Y . The correlation coefficient satisfies the inequality
−1 ≤ σXY ≤ 1. It assumes a value of zero when σXY = 0.

5 Chebyshev’s Theorem:
As discussed earlier, the interpretation of the mean of a random variable is the point of concentration of most of the
values as the outputs of the discrete random variable or the area/region of high density as the continuous random
variable. Now, the variance of a random variable tells us something about the variability of the observations
about the mean. So, the interrelation between the mean and variance of a probability distribution provides a

17
lower bound for the probability that the values of X lies with a specific range. A more precise statement of this
expectation is given by the following inequality, named after the Russian mathematician P. L. Chebyshev.
Theorem 41. Chebyshev’s Theorem: The probability that any random variable X will assume a value within k
standard deviations of the mean is at least 1 − k12 . Then,
1
P (µ − kσ < X < µ + kσ) ≥ 1 − ,
k2
i.e.,
1
P (|X − µ| < kσ) ≥ 1 − .
k2
Consider a probability distribution with mean µ and S.D. σ. Now we are interested in finding the probability
of the event that X falls within the range [µ − kσ, µ + kσ], i.e., P (µ − kσ < X < µ + kσ) or P (|X − µ| < kσ).
There are two important observations considering the quantities σ and k.

• If a random variable has a small variance or standard deviation, we would expect most of the values to
be grouped around the mean. Therefore, the probability that the random variable assumes a value within
a certain interval about the mean is greater than for a similar random variable with a larger standard
deviation. If we think of probability in terms of area, we would expect a continuous distribution with a large
value of σ to indicate a greater variability, and therefore, we should expect the area to be more spread out.
A distribution with a small standard deviation should have most of its area close to µ. The following figure
represents the fact.

Figure 5: Variability of continuous observations about the mean [1].

We can argue the same way for a discrete distribution. The area in the probability histogram is spread out
much more than that in the first case, indicating a more variable distribution of measurements or outcomes.

Figure 6: Variability of discrete observations about the mean [1].

So, intuitively, for a probability distribution, the S.D.(σ) primarily controls the width of the probability
distribution.
• Now, Chebyshev’s Theorem states that for a fixed value of k, the proportion of values lying within [µ −
kσ, µ + kσ] is at least 1 − k12 , regardless of how small or large σ is.
Also, a large value of k means that the interval [µ − kσ, µ + kσ] becomes very wide. As k increases, this
interval expands and can cover almost the entire range of the distribution. Chebyshev’s Theorem reflects
this behaviour through the bound
1
P (µ − kσ < X < µ + kσ) ≥ 1 − .
k2

18
 As k → ∞,
1
1− → 1,
k2
which means that the guaranteed probability inside the interval approaches 1, i.e., it guarantees that at
least a proportion 1 − k12 of the total probability lies in that region. For large k, this guaranteed proportion
becomes arbitrarily close to 1. So, intuitively, k controls the probability of the event that X within the
range [µ − kσ, µ + kσ]. The following figure represents the phenomenon.

Figure 7: Principle of Chebyshev’s Theorem in any distribution.

For k = 2, the theorem states that the random variable X has a probability of at least 1 − 212 = 43 of
falling within two standard deviations of the mean. That is, three-fourths or more of the observations of
any distribution lie in the interval µ ± 2σ. Similarly, the theorem says that at least eight-ninths of the
observations of any distribution fall in the interval µ ± 3σ and so on.

Interval around the mean Minimum proportion inside

µ ± 2σ ≥ 75%
µ ± 3σ ≥ 88.89%
µ ± 4σ ≥ 93.75%

Note: An equivalent form of Chebyshev’s Theorem can be stated as, if X is a random variable with mean µ
and variance σ 2 , then for any value k > 0,

σ2
P {|X − µ| ≥ k} ≤ .
k2
Example 42. Suppose that it is known that the number of items produced in a factory during a week is a random
variable with a mean of 50. If the variance of a week’s production is known to equal 25, then what can be said
about the probability that this week’s production will be between 40 and 60?

Solution: Given that the mean and variance of the weekly production are

µ = 50, σ 2 = 25,

we have √
σ= 25 = 5.
Now,
50 − 40 = 10 = 2 × 5 = 2σ, 60 − 50 = 10 = 2 × 5 = 2σ.
Hence,
40 = µ − 2σ and 60 = µ + 2σ.
By Chebyshev’s Theorem,
1 3
P (µ − 2σ ≤ X ≤ µ + 2σ) ≥ 1 − = .
22 4
Therefore,
P (40 ≤ X ≤ 60) ≥ 0.75
and so the probability that this week’s production will be between 40 and 60 is at least 0.75.

19
Example 43. From past experience, the faculty knows that the test score of a student taking her final examination
in Probability & Statistics is a random variable with mean 70. In addition, the professor knows that the variance
of a student’s test score is equal to 25. What can be said about the probability that a student will score between
55 and 85?

Solution: Given that the mean and variance of the test scores are

µ = 70, σ 2 = 25,

we have √
σ= 25 = 5.
Now,
70 − 55 = 15 = 3 × 5 = 3σ, 85 − 70 = 15 = 3 × 5 = 3σ.
Hence,
55 = µ − 3σ and 85 = µ + 3σ.
By Chebyshev’s inequality,
1 8
P (µ − 3σ ≤ X ≤ µ + 3σ) ≥ 1 − 2
= .
3 9
Therefore,
8
P (55 ≤ X ≤ 85) ≥ ≈ 0.889.
9
So, at least 88.89% of students are guaranteed to score between 55 and 85, regardless of the actual distribution
of test scores.

References:

1. Walpole RE, Myers RH, Myers SL, Ye K. Probability and statistics for engineers and scientists. New York:
Macmillan; 1993 Jan.

2. Ross SM. Introduction to Probability and Statistics for Engineers and Scientists. Academic Press; 2020 Sep
11.

3. Lipschutz, S., & Schiller, J. J. (2014). Schaum’s Outline of Theory and Problems of Introduction to Proba-
bility and Statistics. New Delhi: McGraw-Hill.

4. MIT 6.041SC, 2013, Lecture 5-8, [Link] 0ek6vyAu27e8

5. Stanford Engineering, CS109, 2022, Lecture 6, [Link]

20