Chapter 2 Random Variables

Let (, F, P) be a probability model for an experiment,

and X a function that maps every   , to a unique
point x  R, the set of real numbers. Since the outcome 
is not certain, so is the value X ( )  x . Thus if B is some
subset of R, we may want to determine the probability of
“ X ( )  B ”. To determine this probability, we can look at
the set A  X 1 ( B )   that contains all    that maps
into B under the function X. 

A
X ( )
x B R
1
Fig. 2.1
Obviously, if the set A  X 1 ( B ) also belongs to the
associated field F, then it is an event and the probability of
A is well defined; in that case we can say
Probabilit y of the event " X ( )  B "  P ( X 1 ( B )). (2-1)

However, X 1 ( B ) may not always belong to F for all B, thus

creating difficulties. The notion of random variable (r.v)
makes sure that the inverse mapping always results in an
event so that we are able to determine the probability for
any B  R .
Random Variable (r.v): A finite single valued function X ( )
that maps the set of all experimental outcomes  into the
set of real numbers R is said to be a r.v, if the set  | X ( )  x 
is an event ( F ) for every x in R.
2
Alternatively X is said to be a r.v, if X 1 ( B )  F where B
represents semi-definite intervals of the form {  x  a}
and all other sets that can be constructed from these sets by
performing the set operations of union, intersection and
negation any number of times. The Borel collection B of
such subsets of R is the smallest -field of subsets of R that
includes all semi-infinite intervals of the above form. Thus
if X is a r.v, then
 | X ( )  x   X  x  (2-2)

is an event for every x. What about  a  X  b ,  X  a ?

Are they also events ? In fact with b  a since { X  a }
and  X  b  are events,  X  a c   X  a  is an event and
hence  X  a   X  b   {a  X  b} is also an event.
3
  1 
Thus, 

a 
n
 X  a 

is an event for every n.
Consequently

 1  (2-3)
 

a 
n
 X  a   { X  a}

n 1

is also an event. All events have well defined probability.

Thus the probability of the event  | X (  )  x  must
depend on x. Denote
P  | X ( )  x  FX ( x )  0. (2-4)

The role of the subscript X in (2-4) is only to identify the

actual r.v. FX (x) is said to the Probability Distribution
Function (PDF) associated with the r.v X.

4
Distribution Function: Note that a distribution function
g(x) is nondecreasing, right-continuous and satisfies
g ( )  1, g ( )  0, (2-5)

i.e., if g(x) is a distribution function, then

(i) g ( )  1, g ( )  0,
(ii) if x 1  x 2 , then g ( x 1 )  g ( x 2 ), (2-6)
and

(iii) g ( x )  g ( x ), for all x.

We need to show that FX (x) defined in (2-4) satisfies all

properties in (2-6). In fact, for any r.v X,
5
(i) F X (  )  P   | X ( )    P ( )  1 (2-7)

and F X (  )  P   | X ( )     P ( )  0 . (2-8)
(ii) If x 1  x 2 , then the subset ( , x1 )  ( , x2 ).
Consequently the event  | X ( )  x 1    | X ( )  x 2 ,
since X (  )  x 1 implies X (  )  x 2 . As a result

F X ( x1 )  P  X ( )  x1   P  X ( )  x 2   F X ( x 2 ), (2-9)

implying that the probability distribution function is

nonnegative and monotone nondecreasing.
(iii) Let x  x n  x n 1    x 2  x1 , and consider the event
Ak   | x  X ( )  x k . (2-10)
since
x  X ( )  x k   X ( )  x    X ( )  x k , (2-11) 6
using mutually exclusive property of events we get
P ( Ak )  P  x  X ( )  x k   F X ( x k )  F X ( x ). (2-12)

But  Ak 1  Ak  Ak 1  , and hence


lim Ak   Ak   and hence lim P ( Ak )  0 . (2-13)
k  k 
k 1
Thus
lim P ( Ak )  lim FX ( x k )  FX ( x )  0 .
k  k 

But lim x k  x  , the right limit of x, and hence

k

FX ( x  )  FX ( x ), (2-14)

i.e., FX (x) is right-continuous, justifying all properties of a

distribution function.
7
Additional Properties of a PDF
(iv) If FX ( x0 )  0 for some x 0 , then FX ( x )  0, x  x0 . (2-15)

This follows, since FX ( x0 )  P  X ( )  x0   0 implies X ( )  x 0 

is the null set, and for any x  x0 ,  X ( )  x  will be a subset
of the null set.
(v) P  X ( )  x   1  FX ( x ). (2-16)

We have  X ( )  x   X ( )  x    , and since the two events

are mutually exclusive, (16) follows.
(vi) P x1  X ( )  x2   FX ( x2 )  FX ( x1 ), x2  x1. (2-17)

The events  X ( )  x1  and { x1  X ( )  x 2 } are mutually

exclusive and their union represents the event  X ( )  x 2 .
8
(vii) P  X ( )  x   FX ( x )  FX ( x  ). (2-18)

Let x1  x   ,   0, and x 2  x . From (3-17)

lim P  x    X ( )  x   FX ( x )  lim FX ( x   ), (2-19)
 0  0

or
P  X ( )  x   FX ( x )  FX ( x  ). (2-20)

According to (2-14), FX ( x0 ), the limit of FX ( x ) as x  x0

from the right always exists and equals FX ( x0 ). However the
left limit value FX ( x0 ) need not equal FX ( x0 ). Thus FX ( x )
need not be continuous from the left. At a discontinuity
point of the distribution, the left and right limits are
different, and from (2-20)
P  X ( )  x0   FX ( x0 )  FX ( x0 )  0 . (3-21) 9
Thus the only discontinuities of a distribution function F X ( x )
are of the jump type, and occur at points x 0 where (2-21) is
satisfied. These points can always be enumerated as a
sequence, and moreover they are at most countable in
number.
Example 2.1: X is a r.v such that X ( )  c ,   .Find FX (x).
Solution: For x  c,  X ( )  x    , so that FX ( x )  0, and
for x  c, X ( )  x  , so that FX ( x)  1. (Fig.2.2)
FX (x)
1

x
c
Fig. 2.2

Example 2.2: Toss a coin.   H , T . Suppose the r.v X is

10
F
such that X (T )  0 , X ( H )  1 . Find X (x ).
Solution: For x  0,  X ( )  x    , so that FX ( x )  0.

FX(x)

1
q
x
1
Fig.2.3
•X is said to be a continuous-type r.v if its distribution
function FX (x ) is continuous. In that case F X ( x  )  F X ( x ) for
all x, and from (2-21) we get P X  x  0 .
•If F X ( x ) is constant except for a finite number of jump
discontinuities(piece-wise constant; step-type), then X is
said to be a discrete-type r.v. If xi is such a discontinuity
point, then from (2-21)
p i  P X  x i   F X ( x i )  F X ( x i ). (2-22) 11
 From Fig.2.2, at a point of discontinuity we get
P  X  c   FX ( c )  FX ( c  )  1  0  1 .
and from Fig.2.3,
P  X  0   FX ( 0 )  FX ( 0  )  q  0  q.

Example:2.3 A fair coin is tossed twice, and let the r.v X

represent the number of heads. Find FX ( x ).
Solution: In this case    HH , HT , TH , TT , and
X ( HH )  2 , X ( HT )  1, X ( TH )  1, X ( TT )  0 .
x  0, X ( )  x    F X ( x )  0,
1
0  x  1, X ( )  x    TT   F X ( x )  P  TT  P (T ) P (T )  ,
4
3
1  x  2 , X ( )  x    TT , HT , TH  F X ( x )  P  TT , HT , TH  ,
4
x  2, X ( )  x     F X ( x )  1 . (Fig. 3.4)
12
From Fig.3.4, P X  1  F X (1)  F X (1 )  3 / 4  1 / 4  1 / 2 .
FX (x)
1
3/ 4
1/ 4
x
1 2
Fig. 3.4

Probability density function (p.d.f)

The derivative of the distribution function FX (x ) is called
the probability density function f X ( x ) of the r.v X. Thus
 dF X ( x )
f X ( x)  . (2-23)
dx
Since
dF X ( x ) F ( x   x )  FX ( x )
 lim X  0, (2-24)
dx  x  0 x
from the monotone-nondecreasing nature of F X ( x ), 13
it follows that f X ( x )  0 for all x. f X ( x ) will be a
continuous function, if X is a continuous type r.v.
However, if X is a discrete type r.v as in (2-22), then its
f (x )
p.d.f has the general form (Fig. 3.5) X
p i

x
f X (x)   p i ( x  x i ) , (2-25) xi
i Fig. 2.5
where xi represent the jump-discontinuity points in FX ( x ).
As Fig. 2.5 shows f X ( x ) represents a collection of positive
discrete masses, and it is known as the probability mass
function (p.m.f ) in the discrete case. From (2-23), we
also obtain by integration
x
FX ( x)   f x ( u ) du . (2-26)


Since F X (  )  1, (2-26) yields


f x ( x ) dx  1 , (2-27)
  14
which justifies its name as the density function. Further,
from (2-26)
x2
P  x 1  X ( )  x 2  F X ( x 2 )  F X ( x1 )   f X ( x ) dx . (2-28)
x1

Thus the area under f X ( x ) in the interval ( x1 , x 2 ) represents

the probability in (3-28).
FX (x) fX (x)
1

x1 x2 x x1 x2 x
(a) (b)
Fig. 2.6

Often, [Link] are referred by their specific density functions -

both in the continuous and discrete cases - and in what
follows we shall list a number of them in each category.
15
Continuous-type random variables
1. Normal (Gaussian): X is said to be normal or Gaussian
r.v, if
1 2 2
f X (x)  e ( x ) / 2
. (2-29)
2
2 

This is a bell shaped curve, symmetric around the

parameter  , and its distribution function is given by
x 1  ( y   ) 2 / 2 2  x  (2-30)
FX (x)   e dy  G  ,

2  2
  
where G ( x )   1 e dy is often tabulated. Since f X (x)
x
 y2 / 2

2


depends on two parameters  and  2 , the notation X  N(, 2 )

will be used to represent (3-29). f (x) X

x

16
Fig. 3.7
2. Uniform: X  U(a, b), a  b, if (Fig. 2.8)
 1
fX (x)   b  a
, a  x  b, (2.31)
 0, otherwise.

3. Exponential: X   () if (Fig. 3.9)

 1  x / 
e , x  0,
fX (x)    (2-32)
 0, otherwise.

f X (x) f X (x)
1
ba
x x
a b
Fig. 2.8 Fig. 2.9
17
4. Gamma: X  G(,  ) if (  0,   0) (Fig. 2.10)
f X ( x)
 1
 x x/
 e , x  0,
f X ( x )    ( )   (3-33)
 0, otherwise. x
Fig. 2.10
If   n an integer  ( n )  ( n  1 )!. f X ( x)

5. Beta: X   (a, b) if (a  0, b  0) (Fig. 2.11)

x
0 1
 1 a 1 b 1 Fig. 2.11
 x ( 1  x ) , 0  x  1 ,
f X ( x )    (a , b)
 (2-34)
0, otherwise.
where the Beta function  ( a , b ) is defined as
1
 (a, b)   u a  1 ( 1  u ) b  1 du . (2-35)
0

18
6. Chi-Square: X   2 (n), if (Fig. 3.12)
fX ( x)
 1
 n/2 x n / 2 1e  x / 2 , x  0 ,
f X ( x )   2  (n / 2)
 0, otherwise. (2-36) x
Fig. 2.12

Note that  2 (n) is the same as Gamma (n / 2, 2).

7. Rayleigh: X  R( 2 ), if (Fig. 2.13) fX (x)

 x  x 2 / 2  2
e , x  0,
f X ( x)   2
 0, otherwise. (2-37) x
Fig. 2.13

8. Nakagami – m distribution:
 2  m  m 2 m 1  mx 2 / 
   x e , x0
f X ( x )   ( m )   
(2-38)
 0 otherwise
 19
 f X (x)

9. Cauchy: X  C(, ), if (Fig. 2.14)

 /  x
fX (x)  2 2
,    x   . (2-39)
  (x  ) Fig. 2.14
10. Laplace: (Fig. 2.15)
1  | x |/ 
fX (x)  e ,    x   . (2-40)
2
11. Student’s t-distribution with n degrees of freedom (Fig 2.16)
 ( n 1) / 2
 ( n  1 ) / 2   2
t 
fT (t )   1   ,    t   . (2-41)
n  (n / 2)  n 

fX (x) fT ( t )

x t
Fig. 2.15 Fig. 2.16 20
Discrete-type random variables
1. Bernoulli: X takes the values (0,1), and
P ( X  0)  q, P ( X  1)  p . (2-43)

2. Binomial: X  B(n, p), if (Fig. 3.17)

n  k nk
P ( X  k )    p q , k  0 ,1 , 2 ,  , n . (2-44)
k 
3. Poisson: X  P() , if (Fig. 3.18)
k
P ( X  k )  e  , k  0 ,1 , 2 ,  ,  . (2-45)
k!

P(X  k) P(X  k)

k
12 n
21
Fig. 2.17 Fig. 2.18
4. Hypergeometric:
m  N m 
   
k   n k 
   
P( X  k )  N 
, max(0, m  n  N )  k  min( m, n ) (2-46)
 
n 
 

5. Geometric: X  g ( p ) if
P ( X  k )  pq k , k  0 ,1 , 2 ,  ,  , q  1  p. (2-47)
6. Negative Binomial: X ~ NB (r, p), if
 k  1 r kr
P(X  k)    p q , k  r , r  1,  . (2-48)
 r 1
7. Discrete-Uniform:
1
P(X  k)  , k  1, 2 ,  , N . (2-49)
N
We conclude this lecture with a general distribution due 22
23
24
The pdf of a continuous random variable is given by:

kx , 0  x 1
f X ( x)  
0 , otherwise

where k is a constant.

a. Determine the value of k .

b. Find the corresponding cdf of X .

c. Find P (1 / 4  X  1)

d . Evaluate the mean and variance of X .

25
  1
a.  f X ( x ) dx  1   kxdx  1
 0
2
 x 1
 k    1
 2 0
k
 1
2
k  2

2 x , 0  x 1
 f X ( x)  
0, otherwise
26
b. The cdf of X is given by :
x
FX ( x)   f X ( u ) du


Case 1 : for x  0
F X ( x )  0 , since f X ( x )  0 , for x  0
Case 2 : for 0  x  1
x x
2 x
FX ( x)   f X ( u ) du   2 udu  u  x2
0 0 0

27
Case 3 : for x  1
1 1
2 1
FX ( x)   f X ( u ) du   2 udu  u 1
0 0 0
 The cdf is given by
0, x 0
 2
FX ( x)   x , 0  x 1
1, x 1


28
c. P (1 / 4  X  1)
i . Using the pdf
1 1
P (1 / 4  X  1)   f X ( x ) dx   2 x dx
1/ 4 1/ 4

2 1
 P (1 / 4  X  1)  x  15 / 16
1/ 4
 P (1 / 4  X  1)  15 / 16
ii . Using the cdf
P (1 / 4  X  1)  F X (1)  F X (1 / 4 )
 P (1 / 4  X  1)  1  (1 / 4 ) 2  15 / 16
 P (1 / 4  X  1)  15 / 16
29
d. Mean and Variance
i . Mean
1 1
 X  E(X )   xf X ( x ) dx   2 x 2 dx
0 0

2x3 1
  X   2/3
3 0
ii . Variance
2 2
 X  Var ( X )  E ( X )  [ E ( X )] 2
1 1
2 2
E(X )   x f X ( x ) dx   2 x 3 dx  1 / 2
0 0
2
  X  Var ( x )  1 / 2  ( 2 / 3 ) 2  1 / 18

30
31
 Example 1 Y  aX  b (2-50)
Solution: Suppose a  0 .
 y b  y b
FY ( y)  PY ( )  y   PaX ( )  b  y   P X ( )    FX  . (2-51)
 a   a 
and
1  y b (2-52)
fY ( y)  f X  .
a  a 

On the other hand if a  0 , then

 y b
FY ( y)  PY ( )  y   PaX ( )  b  y   P X ( )  
 a 
 y b 
 1  FX  , (2-53)
 a 
and hence
1  y b
fY ( y)   fX  .
a  a  (2-54)
32
 1  y b
fY ( y)  fX  . (2-55)
|a |  a 
Example 2 Y  X 2. (2-56)

FY ( y )  P Y ( )  y   P X 2 ( )  y . (2-57)

If y  0 , then the event  X 2 ( )  y  , and hence

FY ( y )  0, y  0. (2-58)

For y  0 , from Fig. 5.1, the event {Y ( )  y }  { X 2 ( )  y }

is equivalent to { x1  X ( )  x 2 }.
Y  X2
y

X
x1 x2 33
Hence

FY ( y )  P  x1  X ( )  x 2   F X ( x 2 )  F X ( x1 )
 FX ( y )  FX (  y ), y  0. (2-59)

By direct differentiation, we get

 1

fY ( y )   2 y X

f ( y )  f X ( 
y) , y  0,

(2-60)
0, otherwise .

If f X (x) represents an even function, then reduces to

1
fY ( y ) 
y
fX  y  U ( y ). (2-61)

In particular if X  N ( 0 ,1), so that

1  x2 /2
fX (x)  e , (2-61)
2
34
 Example 4

 X  c, X  c,
 (2-62)
Y  g ( X )   0,  c  X  c,
 X  c, X   c.


35
In this case
P (Y  0)  P (  c  X ( )  c )  FX ( c )  FX (  c ). (2-63)
For y  0 , we have x  c , and Y ( )  X ( )  c so that
FY ( y )  P Y ( )  y   P ( X ( )  c  y )
 P  X ( )  y  c   FX ( y  c ), y  0 . (2-64)
Similarly y  0 , if x   c , and Y (  )  X (  )  c so that
FY ( y )  P Y ( )  y   P ( X ( )  c  y )
 P  X ( )  y  c   FX ( y  c ), y  0. (2-65)
Thus
 f X ( y  c ), y  0,

f Y ( y )  [ FX ( c )  FX (  c )] ( y ), (2-66)
 f ( y  c ), y  0.
 X
g( X ) FX (x ) FY ( y )
c
c X
x y
(a) (b) (c) 36
Fig. 5.2
Example 5 : Half-wave rectifier
 x, x  0,
Y  g ( X ); g ( x)   (2-67) Y
 0, x  0.
In this case
X
P (Y  0)  P ( X ( )  0)  FX ( 0). (2-68)

and for y  0 , since Y  X ,

FY ( y )  P Y ( )  y   P X ( )  y   FX ( y ). (2-69)

Thus
 f X ( y ), y  0,

f Y ( y )   FX (0) ( y ) y  0,  f X ( y )U ( y )  FX (0) ( y ). (2-70)
 0, y  0,

37
Note: As a general approach, given Y  g ( X ), first sketch
the graph y  g ( x ), and determine the range space of y.
Suppose a  y  b is the range space of y  g ( x ).
Then clearly for y  a, FY ( y)  0, and for y  b, FY ( y)  1, so
that FY ( y) can be nonzero only in a  y  b. Next, determine
whether there are discontinuities in the range space of y. If
so evaluate PY ( )  yi  at these discontinuities. In the
continuous region of y, use the basic approach
FY ( y )  P g ( X ( ))  y 

and determine appropriate events in terms of the r.v X for

every y. Finally, we must have FY ( y ) for   y  , and
obtain
dFY ( y )
fY ( y )  in a  y  b.
dy 38
However, if Y  g(X ) is a continuous function, it is easy to
establish a direct procedure to obtain fY (y). A continuos
function g(x) with g(x) nonzero at all but a finite number
of points, has only a finite number of maxima and minima,
and it eventually becomes monotonic as | x | . Consider a
specific y on the y-axis, and a positive increment  y as
shown in Fig below
g ( x)
y  y

x
x 1 x1  x1 x 3 x3  x3
x 2  x 2 x 2

fY (y) for Y  g(X ), 39

we can write
y  y
P y  Y ( )  y   y    f Y ( u ) du  f Y ( y )   y . (2-72)
y

But the event y  Y ( )  y   y  can be expressed in terms

of X ( ) as well. To see this, referring back to Fig. 5.4, we
notice that the equation y  g(x) has three solutions x1 , x2 , x3
(for the specific y chosen there). As a result
when y  Y ( )  y   y , the r.v X could be in any one of the
three mutually exclusive intervals
{x1  X ( )  x1  x1}, {x2  x2  X ( )  x2} or {x3  X ( )  x3  x3}.

Hence the probability of the event in is the sum of the

probability of the above three events, i.e.,
P y  Y ( )  y   y   P { x1  X ( )  x1   x1 }
 P { x 2   x 2  X ( )  x 2 }  P { x 3  X ( )  x 3   x 3 } .(2-73) 40
For small y, xi , making use of the approximation in this we
get
fY ( y )y  f X ( x1 ) x1  f X ( x2 )(  x2 )  f X ( x3 )x3. (2-75)

In this case, x1  0, x2  0 and x3  0, so that (5-28) can be

rewritten as
| xi | 1
fY ( y )   f X ( xi )  f X ( xi ) (2-76)
i y i y / xi

and as y  0, we can be expressed as

1 1
fY ( y )   f X ( xi )   f X ( xi ). (2-77)
i dy / dx x i g ( xi )
i

The summation index i in (5-30) depends on y, and for every

y the equation y  g ( xi ) must be solved to obtain the total
number of solutions at every y, and the actual solutions x1 , x2 ,
all in terms of y. 41
For example, if Y  X 2 , then for all y  0, x1   y and x2   y
represent the two solutions for each y. Notice that the
solutions xi are all in terms of y so that the right side of the
fun is only a function of y. Referring back to the example Y  X2
(Example 2) here for each y  0, there are two solutions
given by x1   y and x2   y . ( fY ( y )  0 for y  0 ).
Moreover YX 2

dy dy y
 2 x so that 2 y
dx dx x  xi
X
and using (2-59) we get x1 x2

 1
 
fY ( y )   2 y

f X ( y )  f X ( y ) , y  0,
(2-78)
 0, otherwise ,

which agrees with (2-60). 42