Chapter One

Random Variables and Probability Distributions

Unit Objective

At the end of this chapter you will be able to understand

 The meaning of a random variable and its types;

 The different kinds of special probability distributions and density functions;

 How to derive joint, marginal, and conditional probability distributions

1. Introduction

The unit is subdivided into three sections. The first section discusses random variables. In this
section a random variable is defined, the two types of a random variable are discussed, The
Expected Value of a Random Variable and Moments, Conditional Probability Distributions and
Independence, and, finally Product Moments and Conditional Expectations.

PRETEST:

Can you think of a variable which has known probabilities of assuming each of its possible
values?

1.2. Random Variable and their distributions

 Define a random variable;

 Tell the difference between a continuous and a discrete random variable;

 Compute the expected value of a random variable and moments;

 Understand what conditional probability distribution is and how it is related to

independence/dependence of events;

 Explain product moments and conditional expectations.

1|Page
DKU, Department of Economics
This section is divided into four subsections dealing with the subject matter of the section and
one other provided for review of the first four subsections. The structure of the sections is given
in the following diagram.

1.1 The concept of a random variable

1.2 Discrete random variables and probability

distributions
Section 1:
Basics of 1.3 Continous random variables and probability density
Probability functions

1.4 The expected value of a randome variable and

moments

Section Review

1.2.1. The Concept of a Random Variable

In many applied problems, we are usually interested only in one aspect or perhaps two or three
aspects of an experiment. For instance, in the experiment of rolling a pair of dice, we may be
interested only in the total score; in the experiment of tossing a coin several times, we may be
interested in the number of heads; etc.

In any experiment, therefore, the experimenter is interested in numbers that are associated with
outcomes of situations involving an element of chance, and such numbers are called random
variables.

For Example, consider the experiment of rolling a pair of dice, assuming that each 36 possible
outcome has the probability of 1/36. Suppose to each sample point, we associate a number that
gives the „total score‟, which is the sum of scores on the first and second die. Accordingly the
outcome (1, 1) will be associated with the number 2, (1,2) with 3, (2,3) with 5, etc. Hence the

2|Page
DKU, Department of Economics
„total score‟ is a random variable, since it associates a number with each element of the sample
space S. This means it defines a function over the points/elements of a sample space.

In general, a random variable is defined as follows. If S is a sample space with a probability

measure and X is a real-valued function defined over the elements of S, then X is called a
random variable. We shall denote random variables by capital letters and their values by lower
case letters. Accordingly X = x is therefore interpreted as the set of elements of S for which the
random variable X takes on the value x.

Example: Consider a balanced coin that is tossed three times. Suppose that X denotes a random
variable “the number of heads.” List the elements of S and the corresponding x values of the r.v
X, and compute the probability of the event that the r.v X takes on the value of 2!

Solution: The listing of all possible outcomes of the given experiment along with the
corresponding values of the r.v X is given in the table below:

Elements of S X Probabilities

HHH 3 1/8

HHT 2 1/8

HTH 2 1/8

THH 2 1/8

HTT 1 1/8

THT 1 1/8

TTH 1 1/8

TTT 0 1/8

Since each outcome is equally likely and mutually exclusive, the probability value of 1/8 can be
associated with each of the sample points as in the last column of the table. To calculate the
probability of the event that X = 2, we can make use of the fundamental law of probabilities (i.e.,
the probability of an event is the sum of the individual probabilities of the equally likely and

3|Page
DKU, Department of Economics
mutually exclusive outcomes constituting the event) as follows. P(X=2) = P(HHT) + P(HTH) +
P(THH) = 1/8 +1/8 + 1/8 = 3/8.

1.2.2. Discrete random variables and probability distributions

Random variables could be discrete or continuous depending on the sample space associated
with them. Random variables defined over discrete sample spaces and hence taking on discrete
values are called discrete [Link]. The random variables considered so far are discrete.

A. Probability distribution

If X is a discrete rv, the function f(x) = P(X=x) for x within the range of X is called the
probability distribution of X. Alternatively, it is a listing of all the possible outcomes of an
experiment and the probability associated with each outcome.

Example: Construct the probability distribution for the experiment of tossing of a balanced coin
three times considered in the preceding example.

Solution: The possible values that X takes on are 0, 1, 2, and 3 (refer back the preceding
example). Using the fundamental law of probabilities as explained before, we obtain the
probability distribution below.

X P(X=x)

0 1/8

1 3/8

2 3/8

3 1/8

Theorem: A function can serve as a probability distribution of a discrete r.v X iff its values f(x)
satisfies the conditions:

 f ( x )  0 , for each value within its domain

 f ( x)  1
 x , where summation extends over all the values within its domain

4|Page
DKU, Department of Economics
Example: Find a formula for the probability distribution of the number of heads obtained in
three tosses of a balanced die in the preceding example.

Solution: From the solution in the example above, the denominator for the P(X=x) terms is
uniformly 8, while the numerators take on the values 1,3,3,1; which are indeed the binomial
 3
 
 3   3  3   3
f ( x)   
        x
coefficients  0  ,  1  ,  2  and  3  . We can therefore write 8 for x  0,1,2,3 as the
formula to our distribution.
x3
f ( x) 
Example: Check whether 15 for x = 1,2 and 3 can serve as the probability distribution
of some random variable.

There are many problems in which it is of interest to know the probability that the value of a r.v
is less than or equal to some real number x.

B. Cumulative Distribution Function

F ( x)  P( X  x)   f (t )
If X is a discrete r.v, the function t 0 for    x   where f(t) is the
value of the p.d.f of X at t is called the Cumulative Distribution Function (CDF) of X.

Example: Find the CDF for the number of head obtained in three tosses of a balanced coin
referred to in the preceding example.

Solution: From the preceding example we have f(0) = 1/8, f(1) = 3/8, f(2) = 3/8, and f(3) = 1/8.
Now,

F(0) = P(X  0)  P(X  0)  f(0)  1 / 8

F(1) = P( X  1)  P( X  0)  P( X  1)  f (0)  f (1)  1 / 2

F(2) = P( X  2)  P( X  0)  P( X  1)  P( X  2)  f (0)  f (1)  f (2)  7 / 8

F(3) = P( X  3)  P( X  0)  P( X  1)  P( X  2)  P( X  3)  1

5|Page
DKU, Department of Economics
 1 / 8 x  0
1 / 2 x  1

F ( x)  
7 / 8 x  2
 1 x3
Therefore,

Note that the CDF obtained above is defined only at values of x =0, 1, 2, and 3 (possible values
that the r.v X takes on). But it should also be logical to determine F at values like 4 or 2.5. For

instance F(4) = 1, since F(4) = P( X  4) , which is the same as F(3). Similarly, F(2.5) = F(2) =
7/8. So, it sounds logical therefore to define our CDF over the set of all real numbers as given
below:

 0 x0
1 / 8 0  x  1

F ( x)  1 / 2 1  x  2
7 / 8 2  x  3

 1 x3

Example: Suppose two balls are selected at random in succession without replacement from a
box containing five red and three blue balls. If x is the r.v for the number of red balls selected,
find the CDF of X.

x1  x2  ....  xn
Theorem: If the range of a r.v X consists of values , then f ( x1 )  F ( x1 ) and
f ( xi )  F ( xi )  F ( xi 1 )
, for i  2,3,..., n .

Example: Given

 0 x 1
1 / 3 1  x  4

F ( x)  1 / 2 4  x  6
5 / 6 6  x  10

 1 x  10 ,

Find (a) P(X = 4), (b) P (2  x  6) , (c) P(X< ), (d) P( X  4) , (e) P(X = 10), (f) P ( x  6) , and
(g) the p.d.f of X.

6|Page
DKU, Department of Economics
Solution: It would be helpful to transform the CDF given to

1 / 3 x  1
1 / 2 x  4

F ( x)  
5 / 6 x  6
 1 x  10

 F(1) = 1/3, F(4) = ½, F(6) = 5/6, F(10) = 1

P( X  4)  P( X  4)  P( X  4)  F (4)  F (1)  1 / 6 1

P(2  X  6)  P( X  6)  P( X  2)  F (6)  F (2)  1 / 2

P( X  6)  P( X  6)  P( X  6)  F (6)  [ F (6)  F (4)]  1 / 2 2

P( X  4)  1  P( X  4)  1  F (1)  2 / 3

P( X  10 )  P( X  10 )  P( X  10 )  F (10 )  F (4)  1 / 6

P( X  6)  F (6)  5 / 6

X takes on values 1, 4, 6, and 10. Using the theorem stated above,

f(1) = F(1) = 1/3

f(4) = F(4) – F(1) = 1/6

f(6) = F(6) – F(4) = 1/3

f(10 = F(10) – F(6) = 1/6

So the p.d.f is given as in the table below.3

x f(x)=P(X=x)

1 1/3

1
Note that P ( X  4)  f (4)  F (4)  F (1)  1 / 6
2
Or simply X < 6 is the third line in F(X) which is associated with F(4) = 1/2
3
Note that the probabilities in problems a to f are solved using F(x). It is also possible to do so from f(x).

7|Page
DKU, Department of Economics
 4 1/6

6 1/3

10 1/6

Theorem: P (a  X  b)  F (b)  F (a )

Theorem: F ()  0 , F ()  1

1.2.3. Continuous Random Variables and Probability Density Functions

Unlike discrete rvs that take only specific values, continuous [Link] can take any value in a real
interval. Continuous [Link] are therefore are [Link] that can take on continuous values or are defined
over continuous sample spaces. Continuous sample spaces and continuous [Link] arise, when we
deal with quantities that are measured on a continuous scale, like measurements of temperature,
speed of a car, amount of toxic substances in blood, etc.

A. Probability Density Function

A function with values f(x), defined over the set of real numbers, is called a probability density
b
P(a  X  b)   f ( x)dx
function of a continuous r.v X iff a for any real constants a and b with
a  b . Sometimes it is also called probability density or simply density.

NB: For continuous [Link], probabilities are always associated with intervals. Hence the probability
that a continuous r.v will take on any given value, say c, would be zero. That is
c
P( X  c)  P(c  X  c)   f ( x)dx  0.
c

Theorem: If X is a continuous r.v and a and b are real constants with a  b , then P (a  X  b) =
P ( a  X  b) = P ( a  X  b) = P ( a  X  b) .

The theorem above directly follows from the remark preceding it.

Theorem: A function can serve as a probability density of a continuous r.v X if its values f(x)
satisfy the conditions:

8|Page
DKU, Department of Economics
f ( x )  0 for    x  



f ( x)dx  1

ke 3 x x0
f ( x)  
Example: If X is a continuous r.v with density  0 otherwise , find the value of k and

also P (0.5  X  1) .

1.2.4. Cumulative Density Function

If X is a continuous r.v and the values of its density at t is f(t), then the function given by

x
F ( x)  P( X  x)   f (t )dt
 for    x   is called the cumulative distribution (CDF) of X.

Theorem: Given the density f(x) and the CDF F(x), then P (a  X  b)  F (b)  F (a ) where a
dF ( x)
f ( x) 
and b are real constants with a  b , then dx where the derivative exists.

Example: Derive the CDF and use it to calculate P (0.5  X  1) for the density

3e 3 x x0
f ( x)  
 0 otherwise .

Example: Find the probability density for the r.v whose CDF is given by

0 x0

F ( x)   x 0  x  1
1 x 1
 .

1.3. The Expected Value of a Random Variable and Moments

If an insurance agent tells us that a 35-year-old woman in Ethiopia can expect to live 20 more
years, this doesn‟t mean that anyone really expects a 35-year-old woman to live until her 55th
birthday and then die the next day. Some 35-year-old women will live another 10 years, some

9|Page
DKU, Department of Economics
will live another 15 years, some will live another 30, …, and the life expectancy of ‟20 more
years‟ will have to be interpreted as an average, namely as a mathematical expectation4.

1.3.2. Expected Value

Given a r.v X and f(x) is the value of its probability distribution/density function at x, then the
expected value of X is

E ( X )   xf ( x)
x , if X is discrete r.v


E ( X )   xf ( x)dx
 , if X is continuous r.v

Example: A lot of 12 TV sets includes 2 with white cords. If 3 of the sets are chosen at random
for shipment to a hotel, how many sets with white cord can the shipper expect to send to the
hotel?

Solution: Let X be the number of TV sets with white cords sent to the hotel, then x = 0, 1, and 2
are the possible values that X can assume, which are discrete.

E ( X )   xf ( x)  0.P( X  0)  1.P( X  1)  2.P( X  2).

We want to calculate x

 210 12 6
P(X  0)       
But, 0 3  3 11 . Since there are 2 TV sets with white cord and 10

without, the event that X = 0 occurs if none from the two white and all three from the ten non-
white TV sets are selected, resulting in term in the numerator. The total number of ways of
selecting three out of twelve would be 12 combination 3, hence yielding the term given in the
denominator of the expression given above. The probabilities for X = 1 and X = 2 can be
calculated the same way and the resulting p.d.f is shown in the table below.
X P(X = x)
0 6/11
1 9/22
2 1/22

4
For more on how the concept of mathematical expectation originated, read John E. Freund‟s „Modern elementary
statistics‟ book.

10 | P a g e
DKU, Department of Economics
The required expected value will therefore be 0.(6/11)+1(9/22)+2(1/22) = ½
 4
x dx 0  x  1
f ( x)   (1  x 2 )
 0 Otherwise
Example: Suppose the probability density of X is given as: .
Find the expected value of X.

Sometimes we may be interested in the expected values of functions of the r.v X. For instance, if
the r.v Y whose values are related to those of X by means of the equation y = g(x), which for
simplicity can be denoted by g(X). To determine the expected value of such a r.v, we use the
following theorem.

Theorem: If X is a r.v and f(x) is the value of its probability distribution/density function at x, the
expected value of g(X) is given by

E[g(X )]   g(x)f(x)
x , if X is discrete r.v


E[ g ( X )]   g ( x) f ( x)dx
 , if X is continuous r.v

Example: If X is the r.v representing the score from an experiment of rolling a balanced die, find
the expected value of g(X) = 2X2+1.

 3X  e  x x0
E e 4  f ( x)   .
Example: Find   , if 0 otherwise

Theorem: For any constants a and b, E(ax+b) = aE(x)+b and E(b) = b

n  n
E  ci g i ( X )   ci Eg i ( X )
Theorem: If c1, c2,….,cn are constants, then  i 1  i 1

The above theorems are of much help in simplifying the calculation of the expected values of
random variables and functions of random variables, as can easily be seen in the following
example.

11 | P a g e
DKU, Department of Economics
 1  x 0  x  1
f ( x)   ,
Example: Given the the density of X as  0 elsewhere find E(Xn) and E[(2X+1)3]

for positive integer n.

Solution: Moments

The rth moment about the origin: The rth moment about the origin of a r.v X, denoted by  r , is
'

 r'  E ( X r )   x r f ( x)
r
the expected value of X , i.e., x for r = 0, 1, 2, …., when X is discrete and

 r'  E ( X r )   x r f ( x)dx
 for r = 0, 1, 2, …., when X is continuous.

Note that for r = 0,  0  E ( X )  E (1)  1. For r = 1, 1  E ( X )  E ( X ), i.e., the first moment

' 0 ' 1

about the origin is just the expected value of the r.v X. It is called the mean of the distribution of
X, or simply the mean of X, and it is denoted by  .

The rth moment about the mean: The rth moment about the mean of a r.v X, denoted by  r , is the
 r  E[( X   ) r ]   ( x   ) r f ( x)
r
expected value of X , i.e., x for r = 1, 2,… when X is discrete

 r  E[( X   ) r ]   ( x   ) r f ( x)dx
and  for r = 1, 2, …., when X is continuous.

Note that for r = 0,  0  E[( X   ) 0 ]  E (1)  1. For r = 1,

1  E[( X   )1 ]   ( x   ) f ( x)   xf ( x)   f ( x)  E ( x)    0
x x x

The second moment about the mean (  2 ) is called the variance of the distribution of X or simply

variance of X and is denoted by  , Var(X) or V(X).  , which is the positive square root of 
2 2

is called the standard deviation.

The variance shows how spread/dispersed the distribution of a random variable about its mean.
A small variance indicates that it is likely to get a value close to the mean, while a large value of
variance suggests that there is a greater probability of getting a value that is not close to the
mean.

12 | P a g e
DKU, Department of Economics
The third moment about the mean,
 3 is used to measure the symmetry or skewness (lack of

symmetry) of a distribution. The bench mark value for skewness is zero. A positive value of
skewness indicates that the distribution is skewed towards values greater than the mean (i.e.,
skewed towards the right side) and a negative value indicates that the distribution is skewed
towards the left side.

The 4th moment about the mean,  4 is used to measure the kurtosis of a distribution. The bench
mark value is 3, which corresponds to that of the normal distribution. A value of kurtosis higher
than 3 indicates that the distribution is flatter compared to the normal distribution, and a smaller
value indicates a higher peak (relative to the normal distribution) around the mean value. The
normal distribution will be discussed in the next chapter.

Theorem:  2   2'   2

Example: Calculate the variance of the r.v X that represents the score rolled with a balanced die.

Section Reflection

1. What is a random variable?

2. What is the difference between discrete and continuous random variable?

3. What is the difference between probability distribution function and probability density
function?

4. What is the first moment about the origin?

5. What do we call the second moment about the mean?

6. A fair coin is tossed independently four times, and let X be the r.v. defined on the usual
sample space S for this experiment as follows: X(s) = the number of H‟s in s.
a. What is the set of values of X?
b. What is the distribution of X?
c. What is the partition of S induced by X?

13 | P a g e
DKU, Department of Economics