Addis Ababa University

Addis Ababa Institute of Technology

Probability and Random Process

Chapter 2: Random Variables

 Random Variables
Outline
 Introduction
 The Cumulative Distribution Function
 Probability Density and Mass Functions
 Expected Value, Variance and Moments
 Some Special Distributions
 Functions of One Random Variable

Semester-I, 2015 By Habib M. 2

Introduction
 In many probabilistic models, the outcomes are of a numerical
nature e.g.
 Stock price value
 Outcome of rolling of a die
 GPA of a randomly selected student
 Voltage at a wireless receiver
 Life time of an electric bulb
 And in other experiments the outcomes are not numerical, but
they may be associated with some numerical value of interest
 Tossing a coin, T = 0, H = 1

Semester-I, 2015 By Habib M. 3

Introduction
 A random variable X is a function that assigns a real number
X(ω) to each outcome ω in the sample space Ω of a random
experiment.
 The sample space Ω is the domain of the random variable and the
set RX of all values taken on by X is the range of the random
variable.
 Thus, RX is the subset of all real numbers.


A
X ( )  x
Re al Line
x B

Semester-I, 2015 By Habib M. 4

Introduction Cont’d……
 If X is a random variable, then {ω: X(ω)= x}={X= x} is an event
for every X in RX.
Example: Consider a random experiment of tossing a fair coin three
times. The sequence of heads and tails is noted and the sample
space Ω is given by:
  {HHH , HHT , HTH , THH , THT , HTT , TTH , TTT }

Let X be the number of heads in three coin tosses. X assigns

each possible outcome ω in the sample space Ω a number from
the set RX={0, 1, 2, 3}.
 : HHH HHT HTH THH THT HTT TTH TTT
X ( ) : 3 2 2 2 1 1 1 0

Semester-I, 2015 By Habib M. 5

 Examples of Random Variables
[Link] an experiment involving two rolls of a die, the following are
examples of random variables:
1) The sum of the two rolls.
2) The number of sixes in the two rolls.
3) The second roll raised to the fifth power.
2. In an experiment involving the transmission of a message,
1) the time needed to transmit the message,
2) the number of symbols received in error,
3) and the delay with which the message is received are
all random variables.

Semester-I, 2015 By Habib M. 6

 Types of Random Variables
 There are two basic types of random variables.
i. Continuous Random Variable
Is a variable whose value is obtained by measuring. Examples
Height of students in class
Time it takes to get to school
ii. Discrete Random Variable
A discrete random variable has a countable number of possible
values. Examples
Number of shop visitors in a given period of time

Semester-I, 2015 By Habib M. 7

 The Probability Mass Function
The most important way to characterize a random variable is through
the probabilities of the values that it can take. For a discrete
random variable X, these are captured by the probability mass
function (PMF for short) of X, denoted PX.
In particular, if x is any possible value of X, the probability mass of
x, denoted PX(x), is the probability of the event {X = x} consisting
of all outcomes that give rise to a value of X equal to x:

For example, let the experiment consist of two independent

tosses of a fair coin, and let X be the number of heads obtained.
Then the PMF of X is

Semester-I, 2015 By Habib M. 8

 PMF contd…

How to Calculate the PMF of a Random Variable

For each possible value x of X
1. Collect all the possible outcomes that give rise to the event {X = x}.
2. Add their probabilities to obtain pX(x).

Semester-I, 2015 By Habib M. 9

 PMF Contd…

Properties of the pmf, PX (x):

i. 0  PX ( x )  1,
ii. P
k
X ( xk )  1

Semester-I, 2015 By Habib M. 10

 Continuous RVs and the
Probability Density Function
• A random variable X is called continuous if its probability law
can be described in terms of a nonnegative function fX, called
the probability density function of X, or PDF for short,
which satisfies

for every subset B of the real line

In particular, the probability that the value of X falls within an

interval is

Semester-I, 2015 By Habib M. 11

 PDF Contd…

• Properties of the PDF

i. For all values of X , f X ( x)  0


ii. 

f X ( x)dx  1
x2
iii. P( x1  X  x2 )   f X ( x)dx
x1

Semester-I, 2015 By Habib M. 12

 PDF Contd…
Example. Continuous Uniform Random Variable.
A gambler spins a wheel of fortune, continuously calibrated
between 0 and 1, and observes the resulting number. Assuming
that all subintervals of [0,1] of the same length are equally likely,
this experiment can be modeled in terms a random variable X
with PDF

What is the value of c?

Semester-I, 2015 By Habib M. 13

The Cumulative Distribution Function

 The cumulative distribution function (cdf) of a random variable

X is defined as the probability of the event {X≤ x}.
FX ( x)  P( X  x)
Properties of the cdf, FX(x):
 The cdf has the following properties.

i. FX ( x) is a non - negative function, i.e.,

0  FX ( x)  1
ii. lim FX ( x)  1
x 
iii. lim FX ( x)  0
x 

Semester-I, 2015 By Habib M. 14

The Cumulative Distribution Function Cont’d…..

iv. FX ( x) is a non - decreasing function of X , i.e.,

If x1  x2 , then FX ( x1 )  FX ( x2 )

v. P( x1  X  x2 )  FX ( x2 )  FX ( x1 )
vi. P( X  x)  1  FX ( x)

Example:
Find the cdf of the random variable X which is defined as the
number of heads in three tosses of a fair coin.

Semester-I, 2015 By Habib M. 15

The Cumulative Distribution Function
Solution:
 We know that X takes on only the values 0, 1, 2 and 3 with
probabilities 1/8, 3/8, 3/8 and 1/8 respectively.
 Thus, FX(x) is simply the sum of the probabilities of the
outcomes from the set {0, 1, 2, 3} that are less than or equal to x.

0, x  0
1 / 8, 0  x  1

 FX ( x)  1 / 2, 1  x  2
7 / 8, 2  x  3

1, x  3

Semester-I, 2015 By Habib M. 16

Calculating the Cumulative Distribution Function

 The cdf of a continuous random variable X can be obtained by

integrating the pdf, i.e.,
x
FX ( x)   f X (u )du


 Similarly, the cdf of a discrete random variable X can be obtained by

using the formula:

FX ( x)  P
xk  x
X ( xk )U ( x  xk )

Semester-I, 2015 By Habib M. 17

Expected Value, Variance and Moments
i. Expected Value (Mean)
 The expected value (mean) of a continuous random variable X,
denoted by μX or E(X), is defined as:


 X  E ( X )   xf X ( x)dx


 Similarly, the expected value of a discrete random variable X is

given by:

 X  E ( X )   xk PX ( xk )
k

 Mean represents the average value of the random variable

in a very large number of trials.
Semester-I, 2015 By Habib M. 18
Expected Value, Variance and Moments Cont’d…..
ii. Variance
 The variance of a continuous random variable X, denoted by
σ2X or VAR(X), is defined as:

 2 X  Var ( X )  E[( X   X ) 2 ]

 2
X  Var ( X )   ( x   X ) 2 f X ( x)dx


 Expanding (x-μX )2 in the above equation and simplifying the

resulting equation, we will get:
 2 X  Var ( X )  E ( X 2 )  [ E ( X )]2
The Variance provides a measure of dispersion of X around its
mean.
Semester-I, 2015 By Habib M. 19
 Expected Value, Variance and Moments Cont’d…..
 The variance of a discrete random variable X is given by:

 2 X  Var( X )   ( xk   X ) 2 PX ( xk )
k

 The standard deviation of a random variable X, denoted by σX, is

simply the square root of the variance, i.e.,

 X  E ( X   X ) 2  Var( X )

[Link]
 The nth moment of a continuous random variable X is defined as:

E ( X )   x n f X ( x)dx ,
n
n 1

Semester-I, 2015 By Habib M. 20
Expected Value, Variance and Moments Cont’d…..

 Similarly, the nth moment of a discrete random variable X is

given by:

E ( X )   xk PX ( xk ) , n 1
n

 Mean of X is the first moment of the random variable X.

Semester-I, 2015 By Habib M. 21

Expected Value, Variance and Moments Cont’d…..

 Example1: Suppose there are 5 students and they scored the

following points in a given test.
S1 = 10, S2 = 8, S3 = 10, S4 = 6, S5 = 1
If X represents the score of a student, find mean and
variance of X.
 Example2: Repeat the above example for the following scores
S1 = 7, S2 = 8, S3 = 6, S4 = 6.5, S5 = 7.5

What do you observe?

Semester-I, 2015 By Habib M. 22

Some Useful Inequalities
Markov Inequality
If a random variable X can only take nonnegative values, then

 Example: Let X be uniformly distributed on the interval [0, 4]

and note that E[X] = 2. Then, the Markov inequality asserts that

Semester-I, 2015 By Habib M. 23

Some Useful Inequalities
Chebyshev Inequality
If X is a random variable with mean µ and variance σ2, the

 Example: let X be uniformly distributed on [0, 4]. Let us use the

Chebyshev inequality to bound the probability that |X - 2| ≥ 1.
We have σ2 = 16/12 = 4/3, and

Semester-I, 2015 By Habib M. 24

Some Special Distributions

i. Continuous Probability Distributions

1. Normal (Gaussian) Distribution
 The random variable X is said to be normal or Gaussian
random variable if its pdf is given by:
1  ( x   ) 2 / 2 2
f X ( x)  e .
2 2
 The corresponding CDF is given by:
x 1  x 
FX ( x)   e  ( y   ) 2 / 2 2
dy  G 

2 2
  
x 1  y2 / 2
where G ( x)   e dy

2
Semester-I, 2015 By Habib M. 25
Some Special Distributions Cont’d……
 The normal or Gaussian distribution is the most common
continuous probability distribution. f X (x)

x

Fig. Normal or Gaussian Distribution

 Example: Suppose the amount of light (in lumens) emitted by a

particular brand of 40W light bulbs is normally distributed with
and mean of 450 lumens and a standard deviation of 20 lumens.
If we randomly select one bulb, what is the probability that
it will emit at least 400 lumens of light?

Semester-I, 2015 By Habib M. 26

 Some Special Distributions Cont’d……

2. Uniform Distribution f X (x)

1
 1 ba
 , a xb
f X ( x)   b  a a b
x

 0, otherwise. Fig. Uniform Distribution

Is used to model a situation where all intervals of the same length

on the distributions support are equally probable.
Example: In analog-to-digital conversion, a quantization error
occurs. And this error has an approximately uniform distribution.

Semester-I, 2015 By Habib M. 27

Some Special Distributions Cont’d……

f X (x)

 1 x / 
 e , x  0, x
f X ( x)   

 0, otherwise. Fig. Exponential Distribution

Semester-I, 2015 By Habib M. 28

 Some Special Distributions Cont’d……

4. Gamma Distribution

 x 1 x / 
 e , x  0,
f X ( x )   ( )  


 0, otherwise.
5. Beta Distribution
 1
 x a 1 (1  x) b 1 , 0  x  1,
f X ( x )    ( a, b)

 0, otherwise.
where
1
 ( a , b)  0
u a 1 (1  u ) b 1 du.

Semester-I, 2015 By Habib M. 29

 Some Special Distributions Cont’d……

6. Rayleigh Distribution

 x  x 2 / 2 2
 2e , x  0,
f X ( x )  

 0, otherwise.

7. Cauchy Distribution
 /
f X ( x)  ,    x  .
  (x  )
2 2

8. Laplace Distribution
1 |x|/ 
f X ( x)  e ,    x  .
2
Semester-I, 2015 By Habib M. 30
Some Special Distributions Cont’d….

i. Discrete Probability Distributions

1. Bernoulli Distribution
P ( X  0)  q, P( X  1)  p.
This distribution is used to model the outcome of an experiment in
which there are only two possible outcomes.
Example: A coin tossing
Here the possible outcomes are Head and Tail only

Semester-I, 2015 By Habib M. 31

Some Special Distributions Cont’d….

2. Binomial Distribution (n, p)

 n  k n k
P( X  k )  
k p q , k  0,1,2,  , n.
 

Is the discrete probability distribution of the number of successes

in a sequence of n independent trails with p being the probability
of success of each trial.
Example: Suppose a biased coin comes up heads with probability
of 0.3 when tossed. What is the probability of achieving 4 heads
after six tosses?

Semester-I, 2015 By Habib M. 32

Some Special Distributions Cont’d….
3. Poisson Distribution

k
P ( X  k )  e  , k  0,1,2, , .
k!
Where λ is the average number of events per interval
Expresses the probability of a given number of events occurring in
in a fixed interval of time or space if these events occur with a
known constant rate and independent of the time since the last
event.
Example: Customers visit a given shop randomly at an average
rate of 120 customers per hour. What is the probability of having
only 20 customers at a given hour in the shop?
Semester-I, 2015 By Habib M. 33
Some Special Distributions Cont’d….
4. Geometric distribution

Is the probability distribution of the number X of Bernoulli trails

needed to get one success. The pmf function gives the probability
that the first occurrence of success requires k independent trials.
Example: A newlywed couple plans to have children, and will
continue until the first girl. What is the probability they will end up
having 5 children?

Semester-I, 2015 By Habib M. 34

Some Special Distributions Cont’d….

5. Hypergeometric Distribution
m  N m 
   
k   n k 
P( X  k )    
N 

, max(0, m  n  N )  k  min( m, n )
 
n 
 

6. Negative Binomial Distribution

 k  1 r k  r
P( X  k )    p q , k  r , r  1, .
 r 1

Semester-I, 2015 By Habib M. 35

Random Variable Examples

Example-1:
The pdf of a continuous random variable is given by:
kx , 0  x 1
f X ( x)  
0 , otherwise

whe re k is a constant.

a. Determine the value of k .

b. Find the corresponding cdf of X .

c. Find P (1 / 4  X  1)

d . Evaluate the mean and variance of X .

Semester-I, 2015 By Habib M. 36
Random Variable Examples Cont’d……
Solution:
 1
a.  
f X ( x ) dx  1  0
kxdx  1
 x2 1
 k    1
 2 0
k
 1
2
k  2

2 x, 0  x 1
 f X ( x)  
0, otherwise
Semester-I, 2015 By Habib M. 37
Random Variable Examples Cont’d……
Solution:
b. The cdf of X is given by :
x
FX ( x)   
f X (u ) du
Case 1 : for x  0
FX ( x)  0, since f X ( x)  0, for x  0
Case 2 : for 0  x  1
x x x
FX ( x)   f X (u ) du   2udu  u  x2
2
0 0 0

Semester-I, 2015 By Habib M. 38

Random Variable Examples Cont’d……
Solution:

Case 3 : for x  1
1 1 1
FX ( x )   f X (u ) du   2udu  u 1
2
0 0 0
 The cdf is given by
0, x0
 2
FX ( x )   x , 0  x 1
1, x 1


Semester-I, 2015 By Habib M. 39

Random Variable Examples Cont’d……
Solution:
c. P (1 / 4  X  1)
i. Using the pdf
1 1
P (1 / 4  X  1)   f X ( x) dx   2 xdx
1/ 4 1/ 4

1
 P (1 / 4  X  1)  x 2
 15 / 16
1/ 4
 P (1 / 4  X  1)  15 / 16
ii. Using the cdf
P (1 / 4  X  1)  FX (1)  FX (1 / 4)
 P (1 / 4  X  1)  1  (1 / 4) 2  15 / 16
 P (1 / 4  X  1)  15 / 16
Semester-I, 2015 By Habib M. 40
Random Variable Examples Cont’d……
Solution:
d. Mean and Variance
i. Mean
1 1
 X  E ( X )   xf X ( x) dx   2 x 2 dx
0 0

2 x3 1
 X   2/3
3 0
ii. Variance
 X 2  Var ( X )  E ( X 2 )  [ E ( X )]2
1 1
E ( X )   x f X ( x ) dx   2 x 3 dx  1 / 2
2 2
0 0

  X  Var ( x )  1 / 2  ( 2 / 3) 2  1 / 18
2

Semester-I, 2015 By Habib M. 41

Random Variable Examples Cont’d……..

Example-2:
Consider a discrete random variable X whose pmf is given by:

1 / 3 , xk  1, 0, 1
PX ( xk )  
0 , otherwise

Find the mean and variance of X .

Semester-I, 2015 By Habib M. 42

Random Variable Examples Cont’d……
Solution:
i. Mean
1
 X  E( X )  x
k  1
k PX ( xk )  1 / 3(1  0  1)  0

ii. Variance
 X 2  Var ( X )  E ( X 2 )  [ E ( X )]2
1
E( X )   k X k     ]  2/3
2 2 2 2 2
x P ( x ) 1 / 3[( 1) ( 0) (1)
k  1

  X  Var ( x)  2 / 3  (0) 2  2 / 3
2

Semester-I, 2015 By Habib M. 43

 Functions of One Random Variable
 Let X be a continuous random variable with pdf fX(x) and suppose
g(x) is a function of the random variable X defined as:
Y  g(X )

 We can determine the cdf and pdf of Y in terms of that of X.

 Consider some of the following functions.
aX  b
sin X X2

1
Y  g( X ) |X |
X
X
log X
eX | X | U ( x)

Semester-I, 2015 By Habib M. 44

 Functions of a Random Variable Cont’d…..
 Steps to determine fY(y) from fX(x):
Method I:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. Determine the cdf of Y using the following basic approach.

FY ( y)  P(Y  y)  P( g ( X )  y)

3. Obtain fY(y) from FY(y) by using direct differentiation, i.e.,

dFY ( y )
fY ( y ) 
dy

Semester-I, 2015 By Habib M. 45

 Functions of a Random Variable Cont’d…..
Method II:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. If Y=g(X) is one to one function and has an inverse
transformation x=g-1(y)=h(y), then the pdf of Y is given by:
dx dh( y )
fY ( y )  f X ( x)  f X [h( y )]
dy dy
3. If Y=g(x) is not one-to-one function, then the pdf of Y can be
obtained as follows.
i. Find the real roots of the function Y=g(x) and denote them by xi

Semester-I, 2015 By Habib M. 46

 Functions of a Random Variable Cont’d…..
ii. Determine the derivatives, dxi of each root xi , i.e. ,

dy

iii. Find the pdf of Y by using the following formula.

dxi
fY ( y )   f X ( xi )
i dy

Semester-I, 2015 By Habib M. 47

Examples on Functions of One Random Variable

Examples:

a. Let Y  aX  b. Find fY ( y ).
b. Let Y  X 2 . Find fY ( y ).
1
c. Let Y  . Find fY ( y ).
X

Semester-I, 2015 By Habib M. 48

Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b

i. Using Method  I

Suppose that a  0

 y b
Fy ( y )  P (Y  y )  P (aX  b  y )  P X  
 a 

 y b
FY ( y )  FX  
 a 
dFY ( y ) 1  y b
 f Y ( y)   fX   (i )
dy a  a 

Semester-I, 2015 By Habib M. 49

Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b

i. Using Method  I

On the other if a  0, then

 y b
Fy ( y )  P(Y  y )  P(aX  b  y )  P X  
 a 

 y b
FY ( y )  1  FX  
 a 
dFY ( y ) 1  y b
 f Y ( y)    fX   (ii)
dy a  a 
Semester-I, 2015 By Habib M. 50
Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b

i. Using Method  I

From equations (i ) and (ii) , we obtain :

1  y b
f Y ( y)  fX  , for all a
a  a 

Semester-I, 2015 By Habib M. 51

Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b
ii. Using Method  II
The function Y  aX  b is one - to - one
y b
For any y, x   h( y ) is the principal solution
a
dx dh( y ) 1 dx 1
   
dy dy a dy a
dh y   y b
f X h( y )   fY ( y ) 
dx 1
fY ( y )  f X ( x)  fX  
dy dy a  a 

Semester-I, 2015 By Habib M. 52

Examples on Functions of One Random Variable…..
Solutions:

b. The function Y  X 2 is not one - to - one and the range

space of Y is y  0
For each y  0, there are two solutions given by
x1   y and x 2  y

Semester-I, 2015 By Habib M. 53

Examples on Functions of One Random Variable…..
Solutions:
dx1 1 dx1 1
b.    and
dy 2 y dy 2 y

dx2 1 dx2 1
  
dy 2 y dy 2 y

dxi dx1 dx2

f Y ( y)   f X ( xi )  f Y ( y )  f X ( x1 )  f X ( x2 )
i dy dy dy

 1
2 y f X  y   f  y ,
X y0
 f Y ( y)  

 0, otherwise
Semester-I, 2015 By Habib M. 54
Examples on Functions of One Random Variable…..
Solutions:
1
c. The function Y  is one - to - one and the range
X
space of Y is IR /0

1
For any y, x   h( y ) is the principal solution
y
dx dh( y ) 1
  2
dy dy y

dh y  1
f X h( y )   f Y ( y )  2 f X  
dx 1
f Y ( y)  f X ( x) 
dy dy y  y

1
IR /0
1
 f Y ( y)  2
f X 
  ,
y y
Semester-I, 2015 By Habib M. 55
Assignment-II

1. The continuous random variable X has the pdf given by:

k (2 x  x 2 ) , 0  x  2
f X ( x)  
0 , otherwise

whe re k is a constant.

Find :

a. the value of k .

b. the cdf of X .

c. P ( X  1)

d . the mean and variance of X .

Semester-I, 2015 By Habib M. 56
Assignment-II Cont’d…..

2. The cdf of continuous random variable X is given by:

0 , x0


F ( x )  k x , 0  x  1
X


1, x 1

whe re k is a constant.

Determine :

a. the value of k .

b. the pdf of X .

c. the mean and variance of X .

Semester-I, 2015 By Habib M. 57
Assignment-II Cont’d…..

3. The random variable X is uniform in the interval [0, 1]. Find

the pdf of the random variable Y if Y=-lnX.

Semester-I, 2015 By Habib M. 58