MULTIPLE RANDOM VARIABLES

Multiple Random Variables

In many applications we have to deal with more than two random variables. For example,
in the navigation problem, the position of a space craft is represented by three random variables
denoting the x, y and z coordinates. The noise affecting the R, G, B channels of color video may
be represented by three random variables. In such situations, it is convenient to define the vector-
valued random variables where each component of the vector is a random variable.

In this lecture, we extend the concepts of joint random variables to the case of multiple
random variables. A generalized analysis will be presented for random variables defined on the
same sample space.

Example1: Suppose we are interested in studying the height and weight of the students in a
class. We can define the joint RV ( X , Y ) where X represents height and Y represents the
weight.

Example 2 Suppose in a communication system X is the transmitted signal and Y is the

corresponding noisy received signal. Then ( X , Y ) is a joint random variable.

Joint Probability Distribution Function:

Recall the definition of the distribution of a single random variable. The event { X  x} was used
to define the probability distribution function FX ( x). Given FX ( x), we can find the probability of any
event involving the random variable. Similarly, for two random variables X and Y , the event
{ X  x, Y  y}  { X  x}  {Y  y} is considered as the representative event.
The probability P{ X  x, Y  y} ( x, y )  2
is called the joint distribution function of the
random variables X and Y and denoted by FX ,Y ( x, y ).
Properties of Joint Probability Distribution Function:

The joint CDF satisfies the following properties:

1. FX(x)=FXY(x,∞) , for any x (marginal CDF of X);
Proof:
{X  x}  {X  x}  {Y  }
 FX ( x)  P {X  x}  P { X  x, Y  }  FXY ( x, )

Similarly FY ( y)  FXY (, y).

2. FY(y)=FXY(∞,y), for any y (marginal CDF of Y);

3. FXY(∞,∞)=1;
4. FXY(−∞,y)=FXY(x,−∞)=0;
5. P(x1<X≤x2,y1<Y≤y2)= FXY(x2,y2)−FXY(x1,y2)−FXY(x2,y1)+FXY(x1,y1);
6. if X and Y are independent, then FXY(x,y)=FX(x)FY(y)
7. FX ,Y ( x1 , y1 )  FX ,Y ( x2 , y2 ) if x1  x2 and y1  y2
Proof:
 If x1  x2 and y1  y2 ,
{ X  x1 , Y  y1}  { X  x2 , Y  y2 }
:
 P{ X  x1 , Y  y1 }  P{ X  x2 , Y  y2 }
 FX ,Y ( x1 , y1 )  FX ,Y ( x2 , y2 )

Example1:

Consider two jointly distributed random variables X and Y with the joint CDF
(1  e2 x )(1  e y ) x  0, y  0
FX ,Y ( x, y )  
0 otherwise

(a) Find the marginal CDFs

(b) Find the probability P{1  X  2, 1  Y  2}
Solution:
1  e 2 x x  0
FX ( x)  lim FX ,Y ( x, y )  
y 
0 elsewhere
(a)
1  e y y  0
FY ( y )  lim FX ,Y ( x, y )  
x 
0 elsewhere
(b)
P{1  X  2, 1  Y  2}  FX ,Y (2, 2)  FX ,Y (1,1)  FX ,Y (1, 2)  FX ,Y (2,1)
 (1  e4 )(1  e2 )  (1  e2 )(1  e1 )  (1  e2 )(1  e2 )  (1  e4 )(1  e1 )
=0.0272

Jointly distributed discrete random variables

If X and Y are two discrete random variables defined on the same probability space
( S , F , P) such that X takes values from the countable subset R X and Y takes values from the
countable subset RY . Then the joint random variable ( X , Y ) can take values from the countable subset in
RX  RY . The joint random variable ( X , Y ) is completely specified by their joint probability mass
function
p X ,Y ( x, y )  P{s | X ( s)  x, Y ( s)  y}, ( x, y )  RX  RY .
Given p X ,Y ( x, y ), we can determine other probabilities involving the random variables X and Y .
Remark
 p X ,Y ( x, y )  0 for ( x, y )  RX  RY

   p X ,Y ( x, y )  1
( x , y ) RX  RY

This is because
  p X ,Y ( x, y )  P( {x, y})
( x , y ) RX  RY ( x , y )RX  RY

=P( RX  RY )
=P{s | ( X ( s), Y ( s))  ( RX  RY )}
=P( S )  1
  Marginal Probability Mass Functions: The probability mass functions p X ( x) and
pY ( y ) are obtained from the joint probability mass function as follows

pX ( x)  P{ X  x} RY
=  pX ,Y ( x, y)
yRY

and similarly
pY ( y )   p X ,Y ( x, y )
xRX

These probability mass functions p X ( x) and pY ( y) obtained from the joint probability mass
functions are called marginal probability mass functions.
Example Consider the random variables X and Y with the joint probability mass function as
tabulated in Table . The marginal probabilities are as shown in the last column and the last row

X 0 1 2 pY ( y )
Y
0 0.25 0.1 0.15 0.5
1 0.14 0.35 0.01 0.5
p X ( x) 0.39 0.45

Joint Probability Density Function

If X and Y are two continuous random variables and their joint distribution function is continuous
in both x and y, then we can define joint probability density function f X ,Y ( x, y ) by
2
f X ,Y ( x, y)  FX ,Y ( x, y), provided it exists.
xy
x y
Clearly FX ,Y ( x, y )    f X ,Y (u , v)dvdu
 
Properties of Joint Probability Density Function:

 f X ,Y ( x, y ) is always a non-negative quantity. That is,

f X ,Y ( x, y)  0 ( x, y)  2

 
   f X ,Y ( x, y )dxdy  1
 
 Marginal probability density functions can be defined as

 The probability of any Borel set B can be obtained by

P( B)   f X ,Y ( x, y)dxdy
( x , y )B
Marginal Distribution and density Functions:
The probability distribution functions of random variables X and Y obtained from joint
distribution function is called ad marginal distribution functions. i.e.
FX(x)=FXY(x,∞) , for any x (marginal CDF of X);

Proof:
{X  x}  {X  x}  {Y  }
 FX ( x)  P {X  x}  P { X  x, Y  }  FXY ( x, )

Similarly FY ( y)  FXY (, y).

The marginal density functions f X ( x) and fY ( y) of two joint RVs X and Y are given by the
derivatives of the corresponding marginal distribution functions. Thus
f X ( x)  dx
d
FX ( x)
 d
dx
FX ( x, )
x 
 d
dx  (  f X ,Y (u, y ) dy ) du
 

  f X ,Y ( x, y ) dy


and similarly fY ( y )   f X ,Y ( x, y ) dx


The marginal CDF and pdf are same as the CDF and pdf of the concerned single random variable. The
marginal term simply refers that it is derived from the corresponding joint distribution or density function
of two or more jointly random variables.

Example2: The joint density function f X ,Y ( x, y ) in the previous example is

2
f X ,Y ( x, y )  FX ,Y ( x, y )
xy
2
 [(1  e 2 x )(1  e  y )] x  0, y  0
xy
 2e 2 x e  y x  0, y  0

Example3: The joint pdf of two random variables X and Y are given by
f X ,Y ( x, y )  cxy 0  x  2, 0  y  2
 0 otherwise
(i) Find c.
(ii) Find FX , y ( x, y )
(iii) Find f X ( x) and fY ( y ).
(iv) What is the probability P(0  X  1, 0  Y  1) ?
   2 2
 
 
f X ,Y ( x, y )dydx  c 
0 
0
xydydx
1
c 
4
1 y x
4 0 0
FX ,Y ( x, y )  uvdudv

x2 y 2

16
2
xy
f X ( x)   dy 0  y  2
0
4
x
 0 y2
2
Similarly
y
fY ( y )  0 y2
2

P(0  X  1, 0  Y  1)
 FX ,Y (1,1)  FX ,Y (0, 0)  FX ,Y (0,1)  FX ,Y (1, 0)
1
= 000
16
1
=
16

Conditional Distribution and Density functions

We discussed conditional probability in an earlier lecture. For two events A and B with P( B)  0 , the
conditional probability P  A / B  was defined as
P  A  B
P  A / B 
P  B

Clearly, the conditional probability can be defined on events involving a random variable X.

Conditional distribution function

Consider the event  X  x and any event B involving the random variable X. The conditional
distribution function of X given B is defined as

FX  x / B   P  X  x / B 
P  X  x  B 
 P  B  0
P  B
Properties of Conditional distribution function

We can verify that FX  x / B  satisfies all the properties of the distribution function. Particularly.
 FX   / B   0 and FX   / B   1.
 0  FX  x / B   1
  FX  x / B  is a non-decreasing function of x.
P( x1  X  x2  / B)  P({ X  x2 }/ B)  P({ X  x1}/ B)

 FX ( x2 / B)  FX ( x1 / B)
Conditional density function
In a similar manner, we can define the conditional density function f X  x / B  of the random variable X
given the event B as

d
fX  x / B  FX  x / B 
dx
Properties of Conditional density function:

All the properties of the pdf applies to the conditional pdf and we can easily show that
 f X  x / B  0

  f X  x / B dx  FX   / B   1

x
 FX  x / B    f X  u / B du


P( x1  X  x2  / B)  FX ( x2 / B)  FX ( x1 / B)
 x2

  f  x / B dx
x1
X

Let (X, Y ) be a discrete bivariate random vector with joint pmf f(x, y) and marginal pmfs f X(x) and
fY (y). For any x such that P(X = x) = fX(x) > 0, the conditional pmf of Y given that X = x is the function
of y denoted by f(y|x) and defined by

For any y such that P(Y = y) = fY (y) > 0, the conditional pmf of X given that Y = y is the function of x
denoted by f(x|y) and defined by

Example 1: Suppose X is a random variable with the distribution function FX  x  . Define B   X  b .

Then
P  X  x  B 
FX  x / B  
P  B
P  X  x   X  b 

P  X  b
P  X  x   X  b 

FX  b 
Case 1: x<b
Then
P  X  x   X  b
FX  x / B  
FX  b 
P  X  x  FX  x 
 
FX  b  FX  b 

d FX  x  f X  x 
And f X  x / B   
dx FX  b  f X  b 

Case 2: x  b

P  X  x   X  b
FX  x / B  
FX  b 
P  X  x FX  b 
  1
FX  b  FX  b 

d
and f X  x / B   FX  x / B   0
dx

FX  x / B  and f X  x / B  are plotted in the following figures.

FX ( x / B)

FX ( x)

b x
 f X ( x / B)

f X ( x)

b x

Example 2 Suppose X is a random variable with the distribution function FX  x  and B   X  b .

Then
P  X  x  B 
FX  x / B  
P  B
P  X  x   X  b 

P  X  b
P  X  x   X  b 

1  FX  b 
For x  b,  X  x {X  b}  . Therefore,

FX  x / B   0 xb
For x  b,  X  x {X  b}  {b  X  x} Therefore,
P b  X  x 
FX  x / B  
1  FX  b 
F  x   FX  b 
 X
1  FX  b 

Thus,
0 xb

FX  x / B    FX  x   FX  b 
 1  F b otherwise
 X

The corresponding pdf is given by

 0 xb

fX  x / B   fX  x
1  F  b  otherwise
 X
Example4:
 Conditional Probability Distribution Function

Consider two continuous jointly random variables and with the joint probability
distribution function We are interested to find the conditional distribution function of
one of the random variables on the condition of a particular value of the other random variable.

We cannot define the conditional distribution function of the random variable on the
condition of the event by the relation

as in the above expression. The conditional distribution function is defined in the

limiting sense as follows:

Conditional Probability Density Function

is called the conditional probability density function of

given

Let us define the conditional distribution function .

The conditional density is defined in the limiting sense as follows

 Because,

The right hand side of the highlighted equation is

Similarly we have

Two random variables are statistically independent if for all

Example 2 X and Y are two jointly random variables with the joint pdf given by

find,

(a)
(b)
(a)

Solution:
Since

We get

Independent Random Variables (or) Statistical Independence

Let and be two random variables characterized by the joint distribution function

and the corresponding joint density function

Then and are independent if and are independent events.
Thus,
 and equivalently

Density function of Sum of Two Random Variables:

We are often interested in finding out the probability density function of a function of two or
more RVs. Following are a few examples.

• The received signal by a communication receiver is given by

where is received signal which is the superposition of the message signal and the noise .

• The frequently applied operations on communication signals like modulation,

demodulation, correlation etc. involve multiplication of two signals in the form Z = XY.

We have to know about the probability distribution of in any analysis of . More formally,
given two random variables X and Y with joint probability density function and a
function we have to find .

In this lecture, we shall address this problem.

 We consider the transformation

Consider the event corresponding to each z. We can find a variable subset

such that .

Figure 1

Probability density function of Z = X + Y .

Consider Figure 2
Figure 2

We have

Therefore, is the colored region in the Figure

 OPERATIONS ON MULTIPLE RANDOM VARIABLES

Expected Values of Functions of Random Variables

Introduction:

In this Part of Unit we will see the concepts of expectation such as mean, variance, moments,
characteristic function, Moment generating function on Multiple Random variables. We are already
familiar with same operations on Single Random variable. This can be used as basic for our topics
we are going to see on multiple random variables.

Function of joint random variables:

If g(x,y) is a function of two random variables X and Y with joint density function f x,y(x,y) then the
expected value of the function g(x,y) is given as

Similarly, for N Random variables X1, X2, . . . XN With joint density function fx1,x2, . . . Xn(x1,x2, . . .
xn), the expected value of the function g(x1,x2, . . . xn) is given as

Properties :

The properties of E(X) for continuous random variables are the same as for discrete ones:
1. If X and Y are random variables on a sample space Ω
then E(X + Y ) = E(X) + E(Y ). (linearity I)

2. If a and b are constants then E(aX + b) = aE(X) + b.

 If is a function of a discrete random variable then

Suppose is a function of continuous random variables then the

expected value of is given by

Thus can be computed without explicitly determining .

We can establish the above result as follows.

Suppose has roots at . Then

Where

Is the differential region containing The mapping is illustrated in Figure 1

for .

Figure 1
Note that

As is varied over the entire axis, the corresponding (non-overlapping) differential regions
in plane cover the entire plane.

Thus,

If is a function of discrete random variables , we can similarly show that

Example 1 The joint pdf of two random variables is given by

 Find the joint expectation of

Example 2 If

Proof:

Thus, expectation is a linear operator.

Example 3

Consider the discrete random variables discussed in Example 4 in lecture [Link]

joint probability mass function of the random variables are tabulated in Table . Find the joint
expectation of .
Remark

(1) We have earlier shown that expectation is a linear operator. We can generally write

Thus
(2) If are independent random variables and ,then

Joint Moments of Random Variables

Just like the moments of a random variable provide a summary description of the random
variable, so also the joint moments provide summary description of two random variables. For
two continuous random variables , the joint moment of order is defined as

And the joint central moment of order is defined as

 where and

Remark
(1) If are discrete random variables, the joint expectation of order and is
defined as

(2) If and , we have the second-order moment of the random variables

given by

(3) If are independent,

Covariance of two random variables

The covariance of two random variables is defined as

Cov(X, Y) is also denoted as .

Expanding the right-hand side, we get

The ratio is called the correlation coefficient.

 If then are called positively correlated.

If then are called negatively correlated

If then are uncorrelated.

We will also show that To establish the relation, we prove the following result:

For two random variables

Proof:

Consider the random variable

Non-negativity of the left-hand side implies that its minimum also must be nonnegative.

For the minimum value,

so the corresponding minimum is

Since the minimum is nonnegative,

Now
 Thus

Uncorrelated random variables

Two random variables are called uncorrelated if

Recall that if are independent random variables, then

then

Thus two independent random variables are always uncorrelated.

Note that independence implies uncorrelated. But uncorrelated generally does not imply
independence (except for jointly Gaussian random variables).

Joint Characteristic Functions of Two Random Variables

The joint characteristic function of two random variables X and Y is defined by

If and are jointly continuous random variables, then

 Note that is same as the two-dimensional Fourier transform with the basis function

instead of

is related to the joint characteristic function by the Fourier inversion formula

If and are discrete random variables, we can define the joint characteristic function in terms
of the joint probability mass function as follows:

Properties of the Joint Characteristic Function

The joint characteristic function has properties similar to the properties of the chacteristic
function of a single random variable. We can easily establish the following properties:

1.
2.
3. If and are independent random variables, then

4. We have,