Mekelle University Ethiopian Institute of Technology Faculty of ECE

Chapter 3
Two Random Variables
3.1 Introduction
In many applications it is important to study two or more rvs defined on the same sample space.
In this chapter, we first consider the case of two rvs, their associated distribution, and some
properties, such as independence of the rvs. These concepts are then extended to the case of
many rvs defined on the same sample space.

Consider an experiment E whose outcomes lie in a sample space, S. A two-dimensional random

variable is a mapping of the points in the sample space to ordered pairs {x, y}. Usually, when
dealing with a pair of random variables, the sample space naturally partitions itself so that it can
be viewed as a combination of two simpler sample spaces S1 and S2.

The overall sample space of the experiment could then be viewed as S = S1 × S2. For any
outcome s ∈ S of this experiment, the pair of random variables (X, Y) is merely a mapping of the
outcomes to a pair of numerical values (x(s), y(s)).

Let S be the sample space of a random experiment. Let X and Y be two rvs. Then the pair (X, Y)
is called a bivariate rv (or two-dimensional random vector) if each of X and Y associates a real
number with every element of S. Thus, the bivariate rv (X, Y) can be considered as a function
that to each point ξ in S assigns a point (x, y) in the plane. The range space of the bivariate rv (X,
Y) is denoted by Rxy and defined by

*( ) ( ) ( ) +

If the rvs X and Y are each, by themselves, discrete rvs, then (X, Y) is called a discrete bivariate
rv. Similarly, if X and Y are each, by themselves, continuous rvs, then (X, Y) is called a
continuous bivariate rv. If one of X and Y is discrete while the other is continuous, then (X, Y) is
called a mixed bivariate rv. Figure 3.1 (X, Y) as a function from S to the plane.

Chapter 3 Page 1
Mekelle University Ethiopian Institute of Technology Faculty of ECE

3.2 Joint cumulative distribution functions

The joint cumulative distribution function of a pair of random variables, {X, Y}, is FXY(x, y) =
P(X ≤ x, Y ≤ y). That is, the joint cdf is the joint probability of the two events {X ≤ x} and {Y ≤
y}. Therefore, the joint cumulative distribution function (or joint cdf) of X and Y, denoted by
FXY(x, y), is the function defined by

FXY(x, y) = P(X ≤ x, Y ≤ y) (3.1)

The event (X ≤ x, Y ≤ y) in Eq. (3.1) is equivalent to the event A ∩ B, where A and B are events
of S defined by

A = {ξ: X(ξ) ≤ x} and B = {ξ: Y(ξ) ≤ y} (3.2)

and P(A) = FX(x) P(B) = FY(y)

Thus, FXY(x, y) = P(A∩B) (3.3)

If, for particular values of x and y, A and B were independent events of S, then

FXY(x, y) = P(A ∩ B) = P(A)P(B) = FX(x) FY(y) (3.4)

Independent random variables: Two rvs X and Y will be called independent if

FXY(x, y) = FX(x) FY(y) (3.5)

for every value of x and y.

Properties of FXY(x, y): The joint cdf of two rvs has many properties analogous to those of the
cdf of a single rv.

1) 0 ≤ FXY(x, y) ≤ 1
2) If x1 ≤ x2, and y1 ≤ y2, then

FXY(x1, y1) ≤ FXY(x2, y1) ≤ FXY(x2, y2)

FXY(x1, y1) ≤ FXY(x1, y2) ≤ FXY(x2, y2)

3) ( ) ( )

4) ( ) ( )

( ) ( )

5) ( ) ( ) ( )

( ) ( ) ( )

6) P(x1 < X ≤ x2, Y ≤ y) = FXY(x2, y) - FXY(x1, y)

P(X ≤ x, y1 < Y ≤ y2) = FXY(x, y2) - FXY(x, y1)

Chapter 3 Page 2
Mekelle University Ethiopian Institute of Technology Faculty of ECE
7) If x1 ≤ x2 and y1 ≤ y2, then

FXY(x2, y2) - FXY(x1, y2) - FXY(x2, y1) + FXY(x1, y1) ≥ 0

Note that the left-hand side of this equation is equal to P(x1 < X ≤ x2, y1 < Y ≤ y2).

Marginal distribution functions: Now

( ) ( ) ( )

since the condition y ≤ ∞ is always satisfied. Then

( ) ( ) ( ) (3.6)

Similarly,

( ) ( ) ( ) (3.7)

FX(x) and FY(y) on Equations (3.6) and (3.7), are referred to as the marginal cdfs of X and Y,
respectively.

Example 3.1: Given two random variables X and Y with the joint cdf ( ), and marginal
cdfs ( ) and ( ), respectively, compute the joint probability that X is greater than a and Y
is greater than b.

Solution:

P(X > a, Y > b) = P({X > a} {Y > b}) = (*̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅

+ * +)

= 1 – P(*̅̅̅̅̅̅̅̅̅̅̅+ ̅̅̅̅̅̅̅̅̅̅̅
* +) = 1 – P({X ≤ a} {Y ≤ b})

= 1 – {P(X ≤ a) + P(Y ≤ b) – P(* + * +)}

= 1 – FX(a) – FY(b) + FXY(a, b)

Example 3.2: The joint cdf of a bivariate rv (X, Y) is given by

( )( )
( ) {

a) Find the marginal cdfs of X and Y.

b) Show that X and Y are independent.
c) Find P(X ≤ 1, Y ≤ 1), P(X ≤ 1), P(Y > 1), and P(X > x, Y > y).

Solution:

a) The marginal cdfs of X and Y are

( ) ( ) 2

( ) ( ) {

Chapter 3 Page 3
Mekelle University Ethiopian Institute of Technology Faculty of ECE
b) Since FXY(x,y) = FX(x)FY(y), X and Y are independent.
c) P(X ≤ 1, Y ≤ 1) = FXY(1,1) = ( )( )
P(X ≤ 1) = FX(1) = ( )
P(Y > 1) = 1 – FY(1) =
P(X > x, Y > y) = 1 – FX(x) – FY(y) + FXY(x,y) =

Example 3.3: The joint cdf of a bivariate rv (X, Y) is given by

( )

a) Find the marginal cdfs of X and Y.

b) Find the conditions on p1, p2, and p3, for which X and Y are independent.

Solution:

a) The marginal cdfs of X and Y are given by

( ) {

( ) {

b) For X and Y to be independent, we must have FXY(x, y) = FX(x)FY(y). Thus, for 0 ≤ x <
a, 0 ≤ y < b, we must have p1 = p2p3, for X and Y to be independent.

Joint probability mass functions: Let (X, Y) be a discrete bivariate rv, and let (X, Y) take
on the values (xi, yj) for a certain allowable set of integers i and j. Let

PXY(xi, yj) = P(X = xi, Y = yj) (3.8)

The function PXY(xi, yj) is called the joint probability mass function (joint pmf) of (X, Y).
Properties of PXY(xi, yj):

1) 0 ≤ PXY(xi, yj) ≤ 1
2) ∑ ∑ ( )
3) ,( )∈ - ∑( )∈ ∑ ( )

where the summation is over the points (xi, yj) in the range space RA corresponding to the event
A. The joint cdf of a discrete bivariate rv (X, Y) is given by

( ) ∑ ∑ ( ) (3.9)

Chapter 3 Page 4
Mekelle University Ethiopian Institute of Technology Faculty of ECE
Marginal probability mass functions: Suppose that for a fixed value X = xi, the rv Y can take on
only the possible values yj (j = 1, 2, … , n). Then

( ) ( ) ∑ ( ) (3.10)

where the summation is taken over all possible pairs (xi, yj) with xi fixed. Similarly,

( ) ( ) ∑ ( ) (3.11)

where the summation is taken over all possible pairs (xi, yj) with yj fixed. The pmfs PX(xi) and
PY(yj), when obtained by Equations (3.10) and (3.11), are referred to as the marginal pmfs of X
and Y, respectively.

Independent random variables: If X and Y are independent rv's, then

PXY(xi, yj) = PX(xi)PY(yj) (3.12)

Example 3.4: Two fair dice are thrown. Consider a bivariate rv (X, Y). Let X = 0 or 1 according
to whether the first die shows an even number or an odd number of dots. Similarly, let Y = 0 or 1
according to the second die.

a) Find the range Rxy of (X, Y).

b) Find the joint pmf of (X, Y).

Solution:

a) The range of (X, Y) is RXY = {(0,0), (0,1), (1,0), (1,1)}

b) It is clear that X and Y are independent and

P(X=0) = P(X=1) = 3/6 = ½ and P(Y=0) = P(Y=1) = 3/6 =1/2

PXY(x,y) = PX(x)PY(y) = ¼

Example 3.5: Consider the binary communication channel shown in Figure 3.2. Let (X, Y) be a
bivariate rv, where X is the input to the channel and Y is the output of the channel. Let P(X = 0)
= 0.5, P(Y = 1|X = 0) = 0.1, and P(Y = 0|X = 1) = 0.2.

a) Find the joint pmfs of (X, Y).

b) Find the marginal pmfs of X and Y.
c) Are X and Y independent?

Figure 3.2: Binary communication channel.

Chapter 3 Page 5
Mekelle University Ethiopian Institute of Technology Faculty of ECE
Solution:

a) The joint pmfs are

P(X=1) = 1 – P(X=0) = 1 – 0.5 = 0.5

P(Y=0|X=0) = 1 - P(Y = 1|X = 0) = 1 - 0.1 = 0.9

P(Y=1|X=1) = 1 - P(Y = 0|X = 1) = 1 - 0.2 = 0.8

P(X=0, Y=0) = P(Y=0|X=0)P(X=0) = 0.9*0.5 = 0.45

P(X=0, Y=1) = P(Y = 1|X = 0)P(X=0) = 0.1*0.5 = 0.05

P(X=1, Y=0) = P(Y=0|X=1)P(X=1) = 0.2*0.5 = 0.10

P(X=1, Y=1) = P(Y = 1|X = 1)P(X=1) = 0.8*0.5 = 0.40

Hence the joint pmfs of (X, Y) is

PXY(0,0) = 0.45, PXY(0,1) = 0.05, PXY(1,0) = 0.1, and PXY(1,1) = 0.4

b) The marginal pmfs are

( ) ∑ ( )

( ) ∑ ( )

( ) ∑ ( )

( ) ∑ ( )

c) Using the results of a and b

(PXY(0,0) = 0.45) ≠ ( ( ) ( ) = 0.5*0.55 = 0.275)

Therefore, X and Y are not independent.

Joint probability density functions: Let (X, Y) be a continuous bivariate rv with cdf
FXY(x, y) and let
( )
( ) (3.13)

The function fXY(x, y) is called the joint probability density function (joint pdf) of (X, Y). By
integrating Equation (3.13), we have

( ) ∫ ∫ ( ) (3.14)

Chapter 3 Page 6
Mekelle University Ethiopian Institute of Technology Faculty of ECE
Properties of fXY(x, y):

1) fXY(x, y) ≥ 0
2) ∫ ∫ ( )
3) ( ) is continuous for all values of x or y except possibly a finite set.
4) ,( )∈ - ∬ ( )

5) P(a < X ≤ b, c < Y ≤ d) = ∫ ∫ ( )

Marginal probability density functions:

( ) ∫ ∫ ( ) ( )

( )
( ) ∫ ( )

( ) ∫ ( ) (3.15)

( ) ∫ ( ) (3.16)

The pdfs fX(x) and fY(y), when obtained by Equations (3.15) and (3.16), are referred to as the
marginal pdfs of X and Y, respectively.

Independent random variables: If X and Y are independent rvs, then

( ) ( ) ( )

( ) ( ) ( )

( ) ( ) ( ) (3.17)

analogous with Equation (3.12) for the discrete case. Thus, we say that the continuous rvs X and
Y are independent rvs if and only if Equation (3.17) is satisfied.

Example 3.6: The joint pdf of a bivariate rv (X, Y) is given by

( )
( ) 2

where k is a constant.

a) Find the value of k.

b) Find the marginal pdfs of X and Y.
c) Are X and Y independent?

Solution:

a) The value of k is

Chapter 3 Page 7
Mekelle University Ethiopian Institute of Technology Faculty of ECE

∫ ∫ ( ) ∫ ∫ ( ) ∫ ( )

∫ ( ) ( )

Thus k = 1/8

b) The marginal pdfs are

( ) ∫ ( ) ∫ ( ) ( ) { ( )

( ) ∫ ( ) ∫ ( ) ( ) { ( )

c) Since fXY(x,y) ≠ fX(x)fY(y), X and Y are not independent

Example 3.7: The joint pdf of a bivariate rv (X, Y) is given by

( ) 2

where k is a constant.

a) Find the value of k.

b) Are X and Y independent?
c) Find P(X + Y < 1).

Solution:

a) The value of k is

∫ ∫ ( ) ∫ ∫ ( ) ∫ ( )

∫ . / ( )

Thus, k = 4.

b) To determine whether X and Y are independent, first find the marginal pdfs of X and Y.

( ) {∫

( ) {∫

Chapter 3 Page 8
Mekelle University Ethiopian Institute of Technology Faculty of ECE
( ) ( ) ( ), therefore X and Y are independent.

c) The region in the x - y plane corresponding to the event (X +. Y < 1) is shown in Figure
3.3(b) as a shaded area. Then

Figure 3.3 X – Y plane

( ) ∫ ∫ ∫ ( ) ∫ ( )

3.3 Conditional distributions

Conditional probability mass functions: If (X, Y) is a discrete bivariate rv with joint pmf PXY(xi,
yj), then the conditional pmf of Y, given that X = xi, is defined by
( )
( | ) ( ) (3.18)
( )

Similarly, we can define PX|Y(xi|yj) as

( )
( | ) ( ) (3.19)
( )

Properties of ( | ):

1) 0 ≤ ( | )≤1 2) ∑ ( | )
Notice that if X and Y are independent, then by Equation (3.12),

( | ) ( ) (3.20)

( | ) ( ) (3.21)

Conditional probability density functions: If (X, Y) is a continuous bivariate rv with joint pdf
fXY(x, y), then the conditional pdf of Y, given that X = x, is defined by
( )
( ) ( ) (3.22)
( )

Similarly, we can define fX|Y(x|y) as

( )
( ) ( ) (3.23)
( )

Chapter 3 Page 9
Mekelle University Ethiopian Institute of Technology Faculty of ECE
Properties of ( ):

1) ( )
2) ∫ ( )

As in the discrete case, if X and Y are independent, then by Equation (3.17),

( ) ( ) and ( ) ( ) (3.24)

Example 3.8: Consider the bivariate rv (X, Y) with joint and marginal pmfs given by

( ) { ( )

( ) ( )

( ) ( )

a) Find the conditional pmfs PY|X(yj|xi) and PX|Y(xi|yj).

b) Find P(Y = 2|X = 2) and P(X = 2|Y = 2).

Solution:

a) The conditional pmfs are

( )
( | )
( )

( )
( | )
( )

b) Using the results of part (a)

( ) ( )

( ) ( )

Example 3.9: Find the conditional pdfs fy|x(y|x) and fx|y(x|y) for the bivariate rv (X, Y) whose
joint and marginal pdfs are given below

( ) 2

( )

( )

Solution:

Chapter 3 Page 10
Mekelle University Ethiopian Institute of Technology Faculty of ECE
( )
( )
( )

( )
( )
( )

3.4 Covariance and correlation coefficient

The (k, n)th moment of a bivariate rv (X, Y) is defined by

∑ ∑ ( )
, - { (3.25)
∫ ∫ ( )

If n = 0, we obtain the kth moment of X, and if k = 0, we obtain the nth moment of Y. Thus,

, - , -

If (X, Y) is a discrete bivariate rv, then

∑∑ ( ) ∑ ∑ ( ) ∑ ( )

∑∑ ( ) ∑ ∑ ( ) ∑ ( )

Similarly, if (X, Y) is a continuous bivariate rv, then

∫ ∫ ( ) ∫ ,∫ ( ) - ∫ ( )

∫ ∫ ( ) ∫ ,∫ ( ) - ∫ ( )

The (1,1)th joint moment of (X,Y), , - is called the correlation of X and Y. If E(XY)
= 0, then we say that X and Y are orthogonal. The covariance of X and Y, denoted by Cov(X, Y)
or σXY is defined by

Cov(X, Y) = σXY = E[(X - μX)(Y - μY)] (3.26)

Expanding Equation (3.26), we obtain

Cov(X, Y) = E[XY] – E[X]E[Y] (3.27)

If Cov(X, Y) = 0, then we say that X and Y are uncorrelated. From Equation (3.27), we see that
X and Y are uncorrelated if

E[XY] = E[X]E[Y] (3.28)

Note that if X and Y are independent, then it can be shown that they are uncorrelated, but the
converse is not true in general; that is, the fact that X and Y are uncorrelated does not, in general,

Chapter 3 Page 11
Mekelle University Ethiopian Institute of Technology Faculty of ECE
imply that they are independent. The correlation coefficient, denoted by ρ(X,Y) or ρxy, is defined
by
( )
( ) (3.29)

It can be shown that | ρxy | ≤ 1 and -1 ≤ ρxy ≤ 1. Note that the correlation coefficient of X and Y
is a measure of linear dependence between X and Y.

Conditional means and conditional variances: If (X, Y) is a discrete bivariate rv with

joint pmf PXY(xi, y), then the conditional mean (or conditional expectation) of Y, given that X =
xi, is defined by

, - ∑ ( ) (3.30)

The conditional variance of Y, given that X = xi, is defined by

( ) 0( ) 1 ∑ ( ) ( ) (3.31)

which can be reduced to

( ) , - , - (3.32)

The conditional mean of X, given that Y = yj and the conditional variance of X, given that Y = yj
are given by similar expressions. Note that the conditional mean of Y, given that X = xi, is a
function of xi alone. Similarly, the conditional mean of X, given that Y = yj, is a function of yj
alone.

If (X, Y) is a continuous bivariate rv with joint pdf fXY(x, y), the conditional mean of Y, given
that X = x, is defined by

, - ∫ ( ) (3.33)

The conditional variance of Y, given that X = x, is defined by

( ) 0( ) 1 ∫ ( ) ( ) (3.34)

which can be reduced to

( ) , - , - (3.35)

The conditional mean of X, given that Y = y, and the conditional variance of X, given that Y = y,
are given by similar expressions. Note that the conditional mean of Y, given that X = x, is a
function of x alone. Similarly, the conditional mean of X, given that Y = y, is a function of y
alone.

Example 3.10: Let (X, Y) be a bivariate rv with the joint pdf

( )

Chapter 3 Page 12
Mekelle University Ethiopian Institute of Technology Faculty of ECE
Show that X and Y are not independent but are uncorrelated.

Solution:

The marginal pdfs of rvs X and Y are

( ) ∫ ( )

( ∫ ∫ )
√ √ √
Noting that the integrand of the first integral in the above expression is the pdf of N(0; 1) and the
second integral in the above expression is the variance of N(0; 1), we have

( ) ( ) -∞ < x < ∞
√

Since fxy(x, y) is symmetric in x and y, we have

( ) ( ) -∞ < y < ∞
√

Now fxy(x,y) ≠ fx(x)fy(y), and hence X and Y are not independent.

( ) ∫ ( ) ( ) ∫ ( )

since for each integral the integrand is an odd function.

( ) ∫ ∫ ( )

The integral vanishes because the contributions of the second and the fourth quadrants cancel
those of the first and the third. Thus, E(XY) = E(X)E(Y), and so X and Y are uncorrelated.

3.5 N - variate random variables

In previous sections, the extension from one rv to two rvs has been made. The concepts can be
extended easily to any number of rvs defined on the same sample space. In this section we
briefly describe some of the extensions.

Definitions:

Given an experiment, the n-tuple of rvs (X1, X2, … , Xn) is called an n-variate rv (or n -
dimensional random vector) if each Xi, i = 1, 2, … , n, associates a real number with every
sample point ξ ϵ S. Thus, an n-variate rv is simply a rule associating an n-tuple of real numbers
with every ξ ϵ S.

Let (X1, X2, … , Xn) be an n-variate rv on S. Then its joint cdf is defined as

Chapter 3 Page 13
Mekelle University Ethiopian Institute of Technology Faculty of ECE
( ) ( ) (3.36)

Note that

( )

The marginal joint cdfs are obtained by setting the appropriate Xi’s to in Equation (3.36). For
example,

( ) ( )

A discrete n-variate rv will be described by a joint pmf defined by

( ) ( ) (3.37)

The probability of any n-dimensional event A is found by summing Eq. (3.37) over the points in
the n-dimensional range space RA corresponding to the event A:

,( )∈ - ∑( )∈ ∑ ( ) (3.38)

Properties of ( ):

1) 0 ≤ ( )≤1
2) ∑ ∑ ( )

The marginal pmf's of one or more of the rvs are obtained by summing Eq. (3.37) appropriately.
For example,

( ) ∑ ( ) (3.39)

Conditional pmfs are defined similarly. For example,

( )
( ) (3.40)
( )

A continuous n-variate rv will be described by a joint pdf defined by

( )
( ) (3.41)

Then

( ) ∫ ∫ ∫ ( ) (3.42)

The marginal pdfs of one or more of the rvs are obtained by integrating Equation (3.41)
appropriately. For example,

( ) ∫ ( )

( ) ∫ ∫ ( )

Chapter 3 Page 14
Mekelle University Ethiopian Institute of Technology Faculty of ECE
Conditional pdfs are defined similarly. For example,
( )
( ) (3.43)
( )

The rvs X1, X2, …, Xn are said to be mutually independent if

( ) ∏ ( ) (3.44)

for the discrete case, and

( ) ∏ ( ) (3.45)

for the continuous case.

The mean (or expectation) of Xi in (X1, X2, …, Xn) is defined as

∑ ∑ ( )
( ) { (3.46)
∫ ∫ ( )

The variance of Xi is defined as

( ) ,( ) - (3.47)

The covariance of Xi and Xj is defined as

( ) ,( )( )- (3.48)

The correlation coefficient of Xi and Xj is defined as

( )
(3.49)

3.6 Special distributions

A. Multinomial distribution: The multinomial distribution is an extension of the
binomial distribution. An experiment is termed a multinomial trial with parameters p1, p2, …, pk,
if it has the following conditions: The experiment has k possible outcomes that are mutually
exclusive and exhaustive, say A1, A2, …, Ak. And P(Ai) = pi i = 1, … , k and ∑ .

Consider an experiment which consists of n repeated, independent, multinomial trials with

parameters p1, p2, …, pk. Let Xi be the rv denoting the number of trials which result in Ai. Then
(X1, X2, …, Xk) is called the multinomial rv with parameters (n, p1, p2, …, pk) and its pmf is
given by:

( ) (3.50)

for xi = 0, 1, …, n, i = 1, 2, …, k, such that ∑ = n. Note that when k = 2, the multinomial

distribution reduces to the binomial distribution.

Chapter 3 Page 15
Mekelle University Ethiopian Institute of Technology Faculty of ECE
B. Bivariate normal distribution: A bivariate rv (X, Y) is said to be a bivariate normal
(or gaussian) rv if its joint pdf is given by
( )
( ) (3.51)
( )

Where ( ) [. / . /. / . / ] (3.52)

and μX, μY, , are the means and variances of X and Y, respectively. It can be shown that ρ
is the correlation coefficient of X and Y and that X and Y are independent when p = 0.

C. N-variate normal distribution: Let (X1, X2, …, Xn) be an n-variate rv defined on a

sample space S. Let X be an n-dimensional random vector expressed as an n x 1 matrix:

[ ]

Let x be an n-dimensional vector (n x 1 matrix) defined by

[ ]

The n-variate rv (X1, X2, …, Xn) is called an n-variate normal rv if its joint pdf is given by

( ) ( )
( ) (3.53)
( )

where T denotes the transpose, μ is the vector mean, K is the covariance matrix given by

( )
, - [ ] [ ]
( )

[ ] ( )

and det(K) is the determinant of the matrix K. Note that:

fX(x) = ( ).

Chapter 3 Page 16