CEN 343 Chapter 3: Multiple random
variables
Chapter 3: Multiple Random Variables
CLO3 Define multiple random variables in terms of their PDF and CDF and
calculate joint moments such as the correlation and covariance.
Objectives
1. Introduce Joint distribution, joint density,
conditional distribution and density, Statistical
independence
2. Introduce Expectations, correlations and joint
characteristic functions
1
CEN 343 Chapter 3: Multiple random
variables
1. Vector Random Variables
• Let two random variables X with value x and Y with value y are defined on a sample space S,
then the random point (x, y) is a random vector in the XY plane.
• In the general case where N r.v's. X1, X2, . . . XN are defined on a sample space S, they
become N-dimensional random vector or N-dimensional r.v.
2. Joint distribution
• Let two events = ≤ = ≤ , (X and Y two random variables ) with
probability distribution functions ( and ( , respectively:
( = ( ≤ (1
( = ( ≤ (2
• The probability of the joint event ≤ , ≤ , which is a function of the members and
is called the joint probability distribution function ≤ , ≤ = ( ∩ . It is
given as follows:
, ( , = ( ≤ , ≤ (3
• If X and Y are two discrete random variables, where X have N possible values and Y have
M possible values , then:
, ( , = ∑" !∑ ! ( , ( − ( − (4
Probability of the joint Unit step function
event = , =
• If X and Y are two continues random variables, then:
2
CEN 343 Chapter 3: Multiple random
variables
* '
, ( , = $ $% , ( ,& & (5
() ()
Example 1:
Let X and Y be two discrete random variables. Let us assume that the joint space has only three
possible elements (1, 1), (2, 1) and (3, 3). The probabilities of these events are:
(1,1 = 0.2, (2, 1 = 0.3, (3, 3 = 0.5.
Find and plot , ( , .
Solution:
Proprieties of the joint distribution ./,0 (1, 2 :
1. ./,0 (−∞, −∞ = ./,0 (−∞, 2 = ./,0 (1, −∞ = 4
2. ./,0 (∞, ∞ = 5
3. 4 ≤ ./,0 (1, 2 ≤ 5
4. 6 15 < / ≤ 18 , 25 < 0 ≤ 28 = ./,0 (18 , 28 − ./,0 (15 , 28 − ./,0 (18 , 25 +
./,0 (15, 25
5. ./,0 (1, 2 is a non decreasing function of both 1 and 2
3
CEN 343 Chapter 3: Multiple random
variables
Marginal distributions:
• The marginal distribution functions of one random variable is expressed as:
./,0 (1, ∞ = .: (1
./,0 (∞, 2 = .0 (2
Example 2:
; = (1,1 , (2,1 , (3,3
(1,1 = 0.2, (2,1 = 0.3, (3,3 = 0.5
Find , ( , and the marginal distributions <( and =( of example 1
Solution:
4
CEN 343 Chapter 3: Multiple random
variables
3. Joint density
• For two continuous random variables X and Y, the joint probability density function is given
by:
>? ,( ,
%, ( , = (6
> >
• If X and Y, are tow discrete random variables then the joint probability density function is
given by:
"
%, ( , = AA ( , B( − B( − (7
! !
Proprieties of the joint density function D/,0 (1, 2 :
1. % , ( , ≥0
) )
2. F() F() % , ( , =1
* '
3. , ( , = F() F() % , ( ,
' ) * )
4. ( = F() F() % , ( , , ( = F() F() % , ( ,
* '
5. ! < ≤ ? , ! < ≤ ? = F* G F' G % , ( ,
H H
) )
6. % ( = F() % , ( , ,% ( = F() % , ( , (Marginal density
function)
Example 3:
Let us consider the following joint probability density function of two random variables X and Y:
%, ( , = ( ( I− ( +1
Find the marginal probability density functions % ( and % (
5
CEN 343 Chapter 3: Multiple random
variables
Solution:
4. Conditional distribution and density
• The conditional distribution function of a random variable X, given event B with ( ≠
0 is:
≤ ∩
( | = ≤ | = (8
(
• The corresponding conditional density function is:
( |
%( | = (9
• Often, we are interested in computing the distribution function of one random variable X
conditioned by the fact that the second variable has some specific values.
4.1 Point conditioning: The Radom variable has some specific value
• For continuous random variables:
%, ( ,
% ( |y = (10
% ,(
6
CEN 343 Chapter 3: Multiple random
variables
Also, we have:
OP,Q (',*
%( | = OP, ('
(11
• For discrete random variables:
Suppose we have:
% (y = A ( R B( − R (12
R !
%, ( , = ∑"
T ! ∑R ! S T, R UB( − T BS − RU (13
Assume = V is the specific value of y, then:
" ( T, V
( | = V = A ( − T (14
T ! ( V
and
" ( T, V
%( | = V = A B( − T (15
T ! ( V
Example 4:
Let us consider the joint pdf: % , ( , = ( ( I ('(*W!
Find % ( | if the marginal pdf of X is given by: % ( = ( I ('
Solution:
%, ( , and % ( are nonzero only for >0 > 0, % ( | is nonzero only for >
0 > 0, therefore, we keep u(x).
7
CEN 343 Chapter 3: Multiple random
variables
Example 5:
What represents the corresponding figure ?
Find %' ( | = Z
Solution:
8
CEN 343 Chapter 3: Multiple random
variables
4.2 Interval conditioning: The Radom variable in the interval [ < ≤ \
[ \ are real numbers.
*
F* % , ( ,
]
, ( , \ − , ( , [
%' ( | [ < ≤ \ = ^
*] = *] (16
F* % ( F* % (
^ ^
and
* '
F* F() % , ( , & &
]
'( | < ≤ = (17
^
[ \ *])
F* F() % , ( ,& &
^
Example 6:
Let % , ( , = ( ( I ('(*W! (18
_(*
%( = ( I (' , %( = (*W! G
Find %( | ≤ ?
Solution 6:
9
CEN 343 Chapter 3: Multiple random
variables
5. Statistical independence
• Two random variables X and Y are independent if:
, ( , = ( ( (19
This means that
%, ( , =% ( %( (20
• Note that if X and Y are independent, then
% ( | )=% ( ) % ( | ) = %* ( ) (21)
Example 7:
In example 6, are X and Y independent?
Solution
10
CEN 343 Chapter 3: Multiple random
variables
⟹ aI bc d IeI I c.
Example 8:
f h
!
Let %, ( , )= ( ) ( )I (g(i are X and Y independent?
!?
Solution
6. Sum of two random variables
• Here the problem is to determine the probability density function of the sum of two
independent random variables X and Y:
j= + (22)
• The resulting probability density function of j can be shown to be the convolution of the
density functions of X and Y:
Convolution
%k (l ) = %' ( ) ∗ % ( )
) )
%k (l) = $ % ( )% (l − ) = $ % ( )% (l − ) (23)
() ()
11
CEN 343 Chapter 3: Multiple random
variables
Proof:
Example 9:
Let us consider two independent random variables X and Y with the following pdfs:
!
% ( ) = [ [ ( ) − ( − )]
!
% ( ) = \ [ ( ) − ( − p)] Where 0 < <p
Find the pdf of W=X+Y
12
CEN 343 Chapter 3: Multiple random
variables
%j (l)
Solution:
1
p
a b a+b w
7. Central Limit Theorem
The probability distribution function of the sum of a large number of random variables
approaches a Gaussian distribution.
13
CEN 343 Chapter 3: Multiple random
variables
8. Expectations and Correlations
• If q(1, 2) is a function of two random variables X and Y ( their joint probability density
function D/,0 (1, 2)). Then the expected value of q(1, 2), a function of the two random
variables X and Y is given by:
∞ ∞
r[q(/, 0)] = $ $ q(1, 2)D/,0 (1, 2)s1s2 (8t)
(∞ (∞
Example 10
Let u( , ) = + p , find v[u( , )]?
Solution:
• If we have n functions of random variables q5 (1, 2), q8 (1, 2), … , qx (1, 2) yz{|:
r[q5(1, 2) + q8 (1, 2) + … + qx (1, 2)] = r[q5 (1, 2)] + r[q8 (1, 2)] + ⋯ + r[qx (1, 2)]
Which means that the expected value of the sum of the functions is equal to the sum of the
expected values of the functions.
14
CEN 343 Chapter 3: Multiple random
variables
8.1 Joint Moments about the origin
) )
• V = vn Vo
=$ $ V
% , , (25)
() ()
We have the following proprieties:
• € = v[ ] are the moments of X
•€V = v[ V
] are the moments of Y
The sum n+k is called the order of the moments. ( •?€ , •€? and •!! are called second
order moments).
•!€ = v[ ] and •€! = v[ ] are called first order moments.
8.2 Correlation
• The second-order joint moment •!! is called the correlation between X and Y.
• The correlation is a very important statistic and its denoted by • :
) )
• = v[ ] = •!! = $ $ %, ( , ) (26)
() ()
• If • = v [ ]v [ ] then X and Y are said to be uncorrelated.
• If X and Y are independent then % , (x, y) = % ( )% ( ) and
∞ ∞
• =$ %( ) '$ %( ) = v [ ]v [ ] (27)
() ()
• Therefore, if X and Y are independent cℎI they are uncorrelated.
• However, if X and Y are uncorrelated, it is not necessary that they are independent.
15
CEN 343 Chapter 3: Multiple random
variables
• If • = 0 cℎI Y are called orthogonal.
Example 11
Let v[ ] = 3, •' ? = 2, let also = −6 + 22
Find • ?
Are X and Y Orthogonal?
Are X and Y uncorrelated?
Solution:
8.3 Joint central moments
• The joint central moment is defined as:
V = v[( − • ) ( − • )V ]
) )
= $ $ ( − • ) ( − • )V % , ( , ) (28)
() ()
We note that:
?€ = v [( − • )? ] = • ?
is the variance of X.
16
CEN 343 Chapter 3: Multiple random
variables
‘? = vn − • ?o = • ?
is the variance of Y.
8.4 Covariance
• The second order joint central moment ’55 is called the covariance of X and Y and denoted
by “ .
) )
“ = !! = vn − • − • o = F() F() − • −• % , , (29)
We have
“ = v[ ] − •• = • − v[ ]v[ ] (30)
• If X and Y are uncorrelated (or independent), then “ =0
• If X and Y are orthogonal, then “ − ••
• If X and Y are correlated, then the correlation coefficient ” measures the degree of correlation
between X and Y:
–HH ™PQ (• (•
•= = = v[› œ› œ] (31)
—–G˜ –˜G šP šQ šP šQ
It is important to note that: −1 ≤ • ≤ 1
Example 12
Let u = +p find •• ? when X and Y are uncorrelated
17
CEN 343 Chapter 3: Multiple random
variables
Solution:
9 Joint Characteristic functions
• The joint characteristic function of two random variables is defined by:
) ) ¤¥H f¦¤¥G h
∅ , Ÿ! , Ÿ? )=v IR¡H WR¡G
¢ = F() F() I £ % , , (32)
Ÿ! and Ÿ? are real numbers.
• By setting Ÿ! = 0 ba Ÿ? = 0, we obtain the marginal characteristic function:
∅ (Ÿ! ) = ∅ , (Ÿ! , 0)
∅ (Ÿ? ) = ∅ , (0, Ÿ? ) (33)
• The joint moments are obtained as :
18
CEN 343 Chapter 3: Multiple random
variables
¨©¦ª ∅P,Q (¡H ,¡G )
• V = −§ WV
| ¡H €,¡G € (34)
¨¥H © ¨¥H ª
Example 1
8 (-® 8
Let us consider the joint characteristic function: ∅/,0 («5 , «8 ) = ¬(8«5 8
Find E[X] and E[Y]
Are X and Y uncorrelated?
Solution:
19