Chapter 4

BIVARIATE (JOINT) DISTRIBUTIONS

4.1 Introduction
In this chapter we shall be concerned with the bivariate or joint distributions, that is,
with situations where we are interested at the same time in a pair of r.v.'s defined over a
joint sample space.
If X and Y are discrete r.v.'s, we write the probability that X will take on the value x
and Y will take on the value y as
h(x,y) = P(X=x , Y=y)
Thus P(X=x,Y=y) is the probability of the intersection of the events {X=x} and {Y=y}.

Definition 4.1
If X and Y are discrete r.v.'s with possible pairs of values
(xi, yj ) , i = 1, 2, .... ; j = 1, 2,...
the function h(x,y) defined by
h(x,y ) = P(X = x , Y = y) for x = x1, x2,... ; y = y1, y2, ....
is called the joint p.m.f. (joint probability mass function) of X and Y.
The table containing the possible values of X and Y together with their joint
probabilities (see table 4.1) is called the joint probability distribution.

X x1 x2 x3 ...
Y
y1 h(x1, y1) h(x2, y1) h(x3, y1) ...
y2 h(x1, y2) h(x2, y2) h(x3, y2) ...
y3 h(x1, y3) h(x2, y3) h(x3, y3) ...
.. ... ... ... ...
.. ... ... ... ...

Table 4.1 The Joint Probability distribution of X and Y

- 61 -
Definition 4.2
A bivariate function h(x,y) can serve as a joint p.m.f. of a pair of r.v.'s X and Y iff it
satisfies the conditions
1. h(x,y)  0 for all x and y
2.   h(xi , yj ) = 1 , where the double summation extends over all possible pairs
i j

(xi, yj ) of (X, Y).

Example 4.1
Determine the value of the constant k so that the following function,
h(x,y) = k x y for x = 1, 2, 3 ; y = 1, 2, 3
can serve as a joint p.m.f. of two r.v.'s X and Y, then find P(X + Y  4).
Solution
To satisfy the first condition of theorem 4.1, the constant k must be nonnegative,
and to satisfy the second condition, we must have
3 3

 k x y =1
x =1 y =1

i.e { k + 2k + 3k + 2k + 4k + 6k + 3k + 6k + 9k } = 1
Thus, we must have 36k = 1 and therefore k = 1/36 and the joint p.m.f. can be also
represented by the following table

x 1 2 3
y
1 1/36 2/36 3/36
2 2/36 4/36 6/36
3 3/36 6/36 9/36
Now,
P(X + Y  4) = 1 - P(X + Y < 4)
= 1 - {h(1, 1) + h(1, 2) + h(2, 1)}
= 1 - {1/36 + 2/36 + 2/36}
= 31/36.

Definition 4.3
A bivariate function f(x,y), defined over the xy-plane, is called a joint probability
density function (joint p.d.f.) of the continuous r.v.'s X and Y iff

- 62 -
 P [ (X , Y )  A ] =  f(x , y)dx dy
A

for any region A in the xy-plane.

Definition 4.4
A bivariate function f(x,y) can serve as a joint p.d.f. of a pair of continuous r.v.'s X
and Y iff it satisfies the conditions
1. f(x,y)  0 for all x and y

2.  f ( x , y ) dx dy = 1
-

Example 4.2
Two electronic components of a missile system work in harmony for the success of
the total system. Let X and Y denote the life in hours of the two components. The joint
density of X and Y is
 k (x2  y 2 ) , 0 x1 , 0 y 1
f (x,y) = 
 0 , elsewhere
(a) Determine the value of the constant k.
(b) Compute P(0 < X < 0.5 , 0 < Y < 0.5).
Solution

(a) Since f(x,y) is a joint p.d.f, then it must satisfy -   f(x , y) dx dy = 1 , i.e.
1 1
 1 1

0 0 k (x  y )dx dy  1     k (x 2  y 2 )dx  dy  1
2 2

0  0 
1 1
k  (x / 3  xy ) |  dy  1  k  (1/ 3  y 2 )  dy  1
3 2 1
0
0 0
1
k (y / 3  y 3 / 3)  0 dy  1  k(2 / 3)  1  k  3 / 2

(b)
1 1 1
2 2 2
3 (x 2  y 2 )dx dy = 3  (x 3 / 3  xy 2 ) | 2  dy
1

P(0< X < ½ , 0< Y < ½)=  2 2   0 


0 0 0
1

 32  24
1 y  1 y 3 / 3 2  1
2  0 16

- 63 -
4.2 Marginal Distributions
If X and Y are discrete r.v.'s with joint p.m.f. h(x,y), the p.m.f. , called the
marginal p.m.f. of X is given by
g X(xi ) =  h (xi , y j ) , i = 1 , 2 , ...
j

Correspondingly, the following function

g Y(y j ) =  h (xi , y j ) , j = 1 , 2 , ...
i

is called the marginal p.m.f. of Y.

Example 4.3
If the joint p.m.f. of the r.v.'s X and Y is given by the following table

x 0 1 2
y
0 3/28 9/28 3/28
1 3/14 3/14 0
2 1/28 0 0
Find the probability distributions of X alone (marginal pmf of X) and that of Y
alone(marginal pmf of Y).
Solution
From the joint probability distribution table, the marginal probability distribution
tables of X and Y are given by

x 0 1 2
5 15 3
gX(x)
14 28 28

y 0 1 2
15 3 1
gY(y)
28 7 28

When X and Y are continuous r.v.'s, the pmf's are replaced by pdf's, the summations are
replaced by integrals and we get

- 64 -
Definition 4.3
If X and Y are continuous r.v.'s with joint p.d.f. f(x,y), the function given by


f X (x) =  f(x,y) dy , -  < x < .

-

for every possible value x of X, is called the marginal p.d.f. of X. Correspondingly, the
marginal p.d.f. of Y is given by


f Y (y) =  f(x,y) dx , -  < y < .

-

Example 4.4
Consider the joint p.d.f. given in example 4.2,
 32 (x2  y 2 ) , 0  x  1, 0  y  1
f (x,y) = 
 0 , elsewhere
find the marginal p.d.f.'s of X and Y.
Solution
The marginal p.d.f. of X is given by,
 13 2 2
  (x  y ) d y 0 < x < 1 
 3 (x 2  1 ) 0<x<1
2 3
f X (x) =  f(x, y) dy =  0 2 =
 0 otherwise 
- 0 otherwise

Since f(x,y) is symmetric in x and y, then, the marginal p.d.f. of Y is given by,

 13 2 2
  2 (x  y ) d x 0 < y < 1  2 (y  3 ) 0 < y < 1
 3 2 1
f Y (y) =  f(x, y) d x =  0 =
 0 otherwise  0
- otherwise

4.3 Conditional Distributions

As we know, the conditional probability of event A given that event B has
been occurred is
P (A  B)
P( A|B )= , P (B ) > 0
P(B )
Suppose now that A and B are the events X = xi and Y = yj , so that we can write,

- 65 -
 P (X = xi , Y = y j ) h (xi , y j )
P (X = xi | Y = y j ) = = , g Y (y j ) > 0
P (Y = y j ) g Y (y j )
Denoting this conditional probability by f(xi|yj) to indicate that xi is a variable and yj is
fixed.

Definition 4. 4
If X and Y are discrete r.v.'s with joint p.m.f. h(x,y), the conditional p.m.f. of X
given Y = yj is given by
h (xi , y j )
f (xi | y j ) = , g Y ( y j ) > 0.
g Y (y j )
for all possible values xi of X, where gY(yj) is the marginal p.m.f. of Y evaluated at y = yj.
Similarly, the conditional p.m.f. of Y given X = xi is given by
h (xi , y j )
f (y j | xi ) = , g X (xi ) > 0.
g X (xi )
for all possible values yj of Y.

Example 4.5
With reference to examples 4.3, the conditional distribution of X given that Y = 1, is
3 / 14 1 3 / 14 1
f (0 | 1) = = , f (1 | 1) = = and f (2 |1) = 0 .
3/7 2 3/7 2
i.e
x 0 1 2
1 1
f(x|y=1)) 0
2 2

When X and Y are continuous r.v.'s, the probability mass functions are replaced by
the probability density functions, and we get
When X and Y are continuous r.v.'s, the probability mass functions are replaced by
the probability density functions, and we get

Definition 4.5
If X and Y are continuous r.v.'s with joint p.d.f. f(x,y), the conditional p.d.f. of X

- 66 -
given Y = y is given by
f(x , y)
f(x | y) = , f (y) > 0.
f (y)
Y
Y

for all possible values x of X, where fY(y) is the marginal p.d.f. of Y evaluated at Y = y.
Similarly, the conditional p.d.f. of Y given X = x is given by
f(x , y)
f(y | x) = , f (x) > 0.
f X (x) X

for all possible values y of Y.

4.4 Independence
When we are dealing with two or more r.v.'s, questions of independence are usually
of great importance. As we know the two events A and B are said to be independent iff
P(A  B) = P(A) . P(B)
Similarly, for the two r.v.'s X and Y we have,

Definition 4.4
If X and Y are two discrete r.v.'s with joint p.m.f. h(x,y) and marginal p.m.f.'s gX(x)
and gY(y) respectively, then X and Y are said to be independent iff
h(xi , yj) = gX(xi) . gY(yj) , for all i and j.
More generally, the discrete r.v.'s X1 , X2 , ... , Xk are said to be independent iff
h( x1, x2,... , x ) = g X1 (x1 ) g X2 (x2 ) ... g Xk (xk ) for all possible values x1, x 2,... , xk .
k

Similarly, if f(x , y ) is the joint p.d.f. of the continuous r.v.'s X and Y and fX(x) is the
marginal p.d.f. X and fY(y) is the marginal p.d.f. Y, then the r.v.'s X and Y are
independent iff

f (x, y) = f X (x) fY (y)  x&y

Example 4.6
The r.v.'s X and Y defined in example 4.3 are dependent, since for example
h(0,0) = 3/28 , gX(0) = 5/12 and gY(0) = 15/28

- 67 -
hence,
h(0,0)  gX(0).gY(0)
Note that the r.v.'s X and Y defined in examples 4.1 are independent (verify).
Also the r.v.'s X and Y defined in examples 4.2 and 4.4 are dependent, since
 3 (x 2  y 2 ) , 0 x1 , 0 y 1
f (x,y) =  2  f X (x). f Y (y)
 0 , elsewhere

4.5 Expectations
The concept of a mathematical expectation can be easily extended to situations
involving more than one r.v. For instance, if Z is the r.v. whose values are related to those
of the two r.v.'s X and Y by means of the equation z = g(x,y), we have the following
definition;
Definition 4.6
If X and Y are discrete r.v.'s with joint p.m.f. h(x,y), then the expected value of g(X,Y)
is,
E [g (X, Y) ] =   g ( xi , y ) f(xi , y )
j j
i j

Similarly, if X and Y are continuous r.v.'s with joint p.m.f. h(x,y), then the expected value
of g(X,Y) is,
 
E [ g (X , Y) ] =   g(x , y) f(x , y ) dx dy
- -

In particular, if g(X,Y) = XY, then

   x i yj f( x i , yj ) , if X & Y are discrete

 i j
E[ X Y ] =  
   x y f(x,y) dx dy , if X & Y are continuous

 -
Generalization of this definition to functions of any finite number of r.v.'s is
straightforward.

Example 4.7
With reference to example 4.3, the expected value of XY is

- 68 -
E[XY] =  x y h(x , y)
x y

3 9 3 3 3 1 3
= 0*0* +1*0* + 2*0* + 0*1* +1*1* + 2*1*0 + 0*2* +1*2*0 + 2*2*0 =
28 28 14 14 14 28 14
Also, with reference to example 4.2, the expected value of XY is
 1 1 9
E[ X Y ] =   x y f(x , y ) d x d y =   xy. 32 (x2  y 2 )dx dy 
- 0 0 16

4.5 Covariance and Correlation Coefficient

The covariance of the two r.v.'s X and Y, denoted by cov(X,Y) or σxy is defined as

σxy = cov(X , Y) = E[(X - μX )(Y - μY )]

The correlation coefficient of the r.v.'s X and Y, denoted by ρ(X , Y) or ρxy is defined to
be
cov ( X , Y ) 
( X, Y ) = = xy
var ( X ) var ( Y ) x y
Both the covariance and the correlation coefficient of the r.v.'s X and Y are measures of a
linear relationship of X and Y in the following sense:
Cov(X , Y) will be positive when (X - μX) and (Y - μY) tend to have the same sign with
high probability; and cov(X , Y) will be negative when (X - μX) and (Y - μY) tend to have
the opposite signs with high probability. cov(X , Y) tends to measure the linear relationship
of X and Y; however, its actual magnitude does not have much meaning since it depends
on the variability of X and Y. The correlation coefficient removes, in a sense, the
individual variability of each X and Y by dividing the covariance by the product of
standard deviations , and thus the correlation coefficient is a better measure of the linear
relationship of X and Y than is the covariance. Also, the correlation coefficient is unitless
and, as we shall see below,
-1  ρ(X , Y)  1
Remark:
Cov(X , Y) = E[(X - μX )(Y - μY )] = E[XY] - μX μY
Proof
Cov(X ,Y) = E[(X - μX )(Y - μY)]
= E[XY - μX Y - X μY + μX μY ]

- 69 -
 = E[XY] - μX E(Y) - E(X) μY + μX μY
= E[XY] - μX μY

Some Properties of Covariance

(1) Cov  X , Y   Cov Y , X 
(2) Cov  X , X   Var  X 
(3) Cov  X , a   0, for any constant a.
(4) For any constants a and b;
Cov a X , b y   a b Cov  X ,Y 
(5) If X1 , X2 and Y are random variables, then
Cov  X1  X2 , Y   Cov  X1 ,Y   Cov  X2 ,Y 
This property can be generalized as follows;
 
 X ,  Y    Cov  X , Y 
n m n m
Cov  i j i j
 i 1 j1  i 1 j1

(6) For any two r.v.'s X and Y we have

Var  X  Y   Var  X   Var  Y   2Cov  X, Y 

Example 4.9
With reference to examples 4.3 and 4.7, we have
10 15 3 21 3
 X = E [ X ] =  x gx ( x ) = 0 x + 1x + 2x = 
x 28 28 28 28 4
15 3 1 1
Y = E [ Y ] =  y gy ( y ) = 0 x + 1x + 2x =
y 28 7 28 2
and
3
E[ X Y ] =  x y h ( x , y ) =
x y 14
Therefore,
cov(X , Y) = E[XY] - μX μY = -9/56

In order to find the correlation coefficient, we have to calculate, first, the standard
deviations σx and σy.

- 70 -
 5 15 3 27
E[ X 2] =  x 2g X (x) = 02  + 12  + 22  =
x 14 28 28 28
Hence,
2
27  3  45
 = var (X) = E [ X ] -  = -  =  0.4
2 2 2
X
28  4  112
X

Similarly (note that distribution of X and Y is symmetric i.e X and Y have the same mean
and the same variance),
9
y = var ( Y ) =
2

28
Finally,
cov ( X , Y ) - 9 / 28 1
 ( X ,Y ) = = =-  - 0.447
var (X) var ( Y ) 45 9 5
.
112 28

Some Properties of Correlation Coefficient

(1)  (Y, X)   (X, Y)
(2)  (X  a, Y  b)   (X, Y).
(3)  (aX,bY)   (X, Y)
(4)  (X, X)  1

Theorem 4.3
If X and Y are independent, then
a- E[XY] = E[X].E[Y]
b- cov(X , Y) = 0

It follows also that

ρ(X , Y) = 0
Definition 4. 6
The r.v.'s X and Y are defined to be uncorrelated iff,
ρ(X , Y) = 0

Remark:
It is of interest to note that the independence of two r.v.'s implies a zero covariance

- 71 -
(uncorrelated), but a zero covariance does not necessarily imply their independence.

4.6 Sums of Random Variables

It is interested and important to know means, variances and covariances of sums or
linear combinations of n r.v.'s.

Theorem 4.4
For r.v.'s X1 , X2 , ... , Xn ,
E   Xi  =  E  Xi 
n n

i = 1  i = 1
and
var   Xi  =  var  Xi  + 2   cov( Xi , X j )
n n

i = 1  i = 1 i< j

For n = 2, we have
var (X1  X2 ) = var  X1  + var  X2   2 cov( X1 , X2 )
and
var(a1X1  a2 X2 ) = a12 var  X1  + a22 var  X2   2a1a 2 cov(X1 , X2 )
Corollary 4.1
If the r.v.'s X1 , X2 , ... , Xn are uncorrelated or independent, then
 n  n
var   X i  =  var X i 
 i =1  i =1
Theorem 4.5
Let X1 , X2 ,..., Xn and Y1 , Y2 ,..., Yn be two sets of r.v.'s, and a1 , a2 ,..., an and
b1 , b2 ,..., bn be two sets of any constants, then
n n
 n n
cov   a i X i ,  b j Yj  =  a i b j cov ( X i , Yj )
i = 1 j=1  i =1 j=1

Example 4.10
If var(X) = 1, var(Y) = 5, var(Z) = 2, cov(X,Y) = -2, cov(X,Z) = -1, and Y and Z are
independent, find var(3X-Y+2Z) and covariance of U=X-2Y+3Z and V=-2X+3Y+4Z.
Solution
var(3X-Y+2Z) = 9 var(X) + var(y) + 4 var(Z) +
2 {-3 cov(X , Y) + 6 cov(X , Z) - 2 cov(Y , Z) }

- 72 -
 = 9(1) + 5 + 4(2) + 2{ -3(-2) + 6(-1) + 0} = 22.
Cov(U,V) = cov(X-2Y+ 3Z , -2X + 3Y + 4Z)
= -2 var(X) + 3 cov(X , Y) + 4 cov(X , Z)
+ 4 cov( X , Y) - 6 var(Y) - 8 cov(Y , Z)
- 6 cov(X , Z) + 9 cov(Y , Z) + 12 var(Z ) = -20.

<+><+><+><+><+><+><+><+><+>

- 73 -
 EXERCISES
[1] A box contains 4 red chips, 3 white chips and 2 blue chips. A random sample of size 3
is drawn without replacement. Let X denotes the number of white chips in the
sample and Y, the number of blue. Write down a formula for the joint p.m.f. of X
and Y.

[2] If the joint p.m.f. of X and Y is given by

h(x , y) = c(x2 + y2) , x = -1, 0, 1, 3 and y = -1, 2, 3.
Find the constant c, the marginal distributions of X and Y, cov(X,Y) and ρ(X,Y).

[3] Determine the constant k so that

 k x (x  y) , 0 < x < 1 , 0  y < 2
f (x,y)=
 0 , o.w.
can serve as a joint probability density function.

[4] If X and Y have the joint probability density function given by:
k ,  1 < x < 1, 0  y  1
f(x, y) = 
0 , o.w.
Find: a- The value of k, b- Cov(3X-7,2Y+3).

[5] If the joint probability distribution of X and Y is given by the following table

Y -2 -1 2 4
X
1 0.1 0.2 0.0 0.1
2 0.0 0.1 0.1 0.2
3 0.1 0.0 0.1 0.0

Find: P(Y>X) , COV(X,Y) and VAR(2X-5Y+7).

[6] If var(X1)=5, var(X2)=4, var(X3)=7, cov(X1,X2)=3, cov(X1,X3)=-2, and X2 and X3 are

independent. Find the correlation coefficient of
Y = X1 - 2X2 + 3X3 and Z = -2X1 + 3X2 + 4X3.

[7] Circle the correct answer from each of the following multiple choice questions

- 74 -
i. If the joint p.m.f. of X and Y is given by
h(x , y) = k(x2 + y2) , x = -1, 0, 3 and y = -2, 1, 3.
where k is a constant, then P(X+Y≤ 3) is
a. 53/72 b. 19/72 c. 11/18 d. None of the above.

ii. If the joint p.m.f. of X and Y is given by:

h(x , y) = kx(x + y) , x = 1, 2, 3 and y = -1, 0, 2.
where k is a constant, then P(X+Y >2) is
a. 10/48 b. 25/48 c. 45/48 d. None of the above.
iii. If var(X)=5, var(Y)=4, and cov(X,Y)=3 then var(3X-2Y+5) is:
a. 22 b. 25 c. 43 d. None of the above.
iv. If the joint probability density function of X and Y is given by:
k , 0 < x < 1,  1  y  1
f(x, y) = 
0 , o.w.
then the value of the constant k is
a. 2 b. 1/2 c. 1/4 d. None of the above.

<+><+><+><+><+><+><+><+><+>

- 75 -