Chapter Three: Random Variables

1 Real Random Variable (R.V.)

Definition 1
Let (Ω, Θ, P) be a probability space. A real random variable is any mapping X : Ω → R
such that, for any interval I ⊆ R, the pre-image
X −1 (I) = {ω ∈ Ω : X(ω) ∈ I} ∈ Θ.
The set of values taken by X, denoted as X(Ω), is called the set of observables associated
with X.
Remark 1: If Θ = P(Ω) (the power set of Ω), any mapping from Ω to R is a real random
variable. This is often the case in discrete spaces.

Definition 2
The cumulative distribution function (CDF) F of a real random variable X defined on
(Ω, Θ, P) is given by:
F (x) = P(X ≤ x) = P({ω ∈ Ω : X(ω) ≤ x}).
Thus,the distribution function specifies, for all real values x, the probability that the
random variable is less than or equal to x.

2 Discrete Real Random Variable (D.R.V.)

Definition
A real random variable X is discrete if the set X(Ω) is countable. Note that X(Ω) can
either be finite, e.g., {x1 , x2 , . . . , xn }, or countably infinite, e.g., {x1 , x2 , . . .}.
Example:Suppose that our experiment consists of tossing 3 fair coins. If we let Y denote
the number of heads that appear, then Y is a random variable taking on one of the values
0, 1, 2, and 3 with respective probabilities:
P {Y = 0} = P {(t, t, t)} = (1/8)
P {Y = 1} = P {(t, t, h), (t, h, t), (h, t, t)} = (3/8)
P {Y = 2} = P {(t, h, h), (h, t, h), (h, h, t)} = (3/8)
P {Y = 3} = P {(h, h, h)} = (1/8)

1
2.1 Probability Mass Function (PMF)
Let X be a discrete random variable. The probability mass function (PMF) of X is
defined as:
PX (x) = P(X = x), x ∈ X(Ω).
For a discrete random variable X with PMF PX ,
X
PX (x) = 1.
x∈X(Ω)

Note that if a is a value that X never takes, then PX (a) = 0.

2.2 Cumulative Distribution Function (CDF)

If the values of X are ordered as x1 < x2 < · · · < xn , then for all x ∈ R, the cumulative
distribution function FX (x) is given by:
X
FX (x) = P(X ≤ x) = PX (xi ).
xi ≤x

Properties: For a discrete random variable X with cumulative distribution function FX ,

the following hold:
1. FX is a non decreasing step function.
2. The value of FX is constant in the intervals [xi−1 , xi ) and then takes a step(or jump)
of size PX (xi ) at xi .
3. For any a, b ∈ R with a < b:

P(a < X ≤ b) = FX (b) − FX (a).

4. For any a ∈ R:
P(X > a) = 1 − P(X ≤ a) = 1 − FX (a)

5. lim FX (x) = 0 and lim FX (x) = 1.

x→−∞ x→+∞

Remark: The cumulative distribution function FX is uniquely determined by the proba-

bility law PX , and conversely, PX can be derived from FX .
Example: Let X be a discrete random variable with the following pmf:
1 1 1 1
PX (1) = , PX (2) = , PX (3) = , PX (4) = .
4 2 8 8
then its cumulative distribution function is:


 0 x<1

1/4 1≤x<2



FX (x) = 3/4 2≤x<3

7/8 3≤x<4





1 x≥4

2
3 Continuous Real Random Variable
A continuous random variable takes a range of possible values. We say that a real random
variable X is continuous if there exists a continuous function f (called the probability
density function, PDF) such that for any two reals a ≤ b:
Z b
P (a ≤ X ≤ b) = f (x) dx
a

The pdf always satisfies the following properties:

ˆ f (x) ≥ 0 for x ∈ R.
R∞
ˆ −∞ f (x) dx = 1.

Remark: If we let a = b in the precedent integral, we get

Z a
P (X = a) = P (a ≤ X ≤ a) = f (x) dx = 0.
a

In words, this equation states that the probability that a continuous random variable will
assume any fixed value is zero.

Example: Suppose that X is a continuous random variable whose probability density

function is given by:
(
C(4x − 2x2 ) 0 < x < 2
f (x) =
0 otherwise.
What is the value of C?
R∞
Since f is a probability density function, we must have −∞
f (x) dx = 1 then:
Z 2
C 4x − 2x2 dx = 1
0
x=2
2x3

2
C 2x − =1
3 x=0
3
C=
8

3.1 Cumulative Distribution Function

The cumulative distribution function FX (.) is defined by:
Z x
FX (x) = P(X ≤ x) = f (t)dt.
−∞

Properties: The CDF of a continuous r.v has the following properties:

ˆ FX (.) is continuous and non decreasing on R.
ˆ ∀x ∈ R, d
dx
F (x) = f (x).

3
 ˆ For any a, b ∈ R with a < b:
P (a ≤ X ≤ b) = FX (b) − FX (a).

ˆ For any a ∈ R:
P (X < a) = P (X ≤ a) = F (a).

ˆ lim FX (x) = 0 and lim FX (x) = 1.

x→−∞ x→+∞

4 Expectation
One of the most important concepts in probability theory is that of the expectation of a
random variable. The expected value,of X, denoted by E[X], is defined by:
(P
xPX (x) if X is discrete,
E[X] = R ∞x∈X(Ω)
−∞
xf (x) dx if X is continuous.
The expected value of X is a weighted average of the possible values that X can take on,
each value being weighted by the probability that X assumes it.
Example: We roll two standard 6-sided dice. You win 1000DZD if the sum is 2 and
lose 100DZD otherwise. Would you take this bet?
The probability of a 2 is p = 1/36 and the probability of obtaining something other than
2 is 1 − p = 35/36. If we denote by G the random variable representing the possible gain
then: (
+1000 with probability p
G=
−100 otherwise
The expected value of G is E[G] = (1000)(1/36) + (−100)(35/36) = −69.44. Then this is
a losing bet on average.

4.1 Expectation of a function of a random variable

For any real-valued function g(.), we have that:
 P
x∈X(Ω) g(x)PX (x) if X is discrete,


E[g(X)] = R∞
 g(x)f (x) dx if X is continuous.


−∞

The precedent result allows us to compute E[g(X)] even when we don’t know the distri-
bution function of g(X). In particular, for two constants a and b, we have that
E[aX + b] = aE[X] + b.

Example: Let X denote a random variable that takes on any of the values −1, 0, and 1
with respective probabilities:
P (X = −1) = .2 P (X = 0) = .5 P (X = 1) = .3
The value of E[X 2 ] is given by:
E[X 2 ] = (−1)2 (.2) + 02 (.5) + 12 (.3) = 0.5

4
5 Moments
The expected value of a random variable X, E[X], is also referred to as the first moment
of X. The quantityE[X n ], n ≥ 1, is called the nth moment of X given by:
 P
n
x∈X(Ω) x PX (x) if X is discrete,


E[X n ] = R∞
n
 x f (x) dx if X is continuous.


−∞

5.1 Central Moments

The nth central moment of a random variable X is defined as the expected value of the
nth power of the deviations of X from its mean µ = E[X]. It is given by:
 P
n
x∈X(Ω)(x − µ) PX (x) if X is discrete,


µn (X) = E[(X − µ)n ] = R∞
n
 (x − µ) f (x) dx if X is continuous.


−∞

6 Variance
If X is a random variable with expected value E[X], then the variance of X, denoted by
Var(X), is defined by:
V ar(X) = E[(X − E[X])2 ] = µ2 (X)
An alternative formula to compute Var(X) is:
V ar(X) = E[X 2 ] − (E[X])2
For any two constants a and b we have that:
V ar(aX + b) = a2 V (X).

6.1 Standard Deviation

The standard deviation of a random variable X, denoted σ(X), is the square root of the
variance: p
σ(X) = Var(X).

7 Moment Generating Function

The moment generating function (MGF) of X, denoted MX (t), is defined for all real
values of t by: (P
etx PX (x) if X is discrete,
MX (t) = R ∞x∈X(Ω)
−∞
etx f (x) dx if X is continuous.
We call MX (t) the moment generating function because for any n ≥ 1, the nth moment
of X can be obtained by differentiating MX (t) n times and evaluating at t = 0:
(n)
E[X n ] = MX (0).

5
Properties:
ˆ The moment generating function of the sum of independent random variables equals
the product of the individual moment generating functions.
ˆ The moment generating function uniquely determines the distribution. That is, if
MX (t) exists and is finite in a neighborhood of t = 0, then the distribution of X is
uniquely determined.

8 Distribution of a function of a random variable

8.1 Discrete variable case
Let X(Ω), denote the set of possible values of the discrete r.v X and PX its associated
p.m.f. The r.v Y = g(X) is a discrete variable with range Y (Ω) = g(X(Ω)) and its p.m.f
is given by:  P
 PX (x) if y ∈ Y (Ω)
PY (y) = x:g(x)=y
0 otherwise.

8.2 Continuous variable case

Let X be a continuous random variable having probability density function fX . Sup-
pose that g(x) is a strictly monotonic (increasing or decreasing), differentiable (and thus
continuous) function of x. Then the random variable Y defined by Y = g(X) has a
probability density function given by:
(
d −1
fX [g −1 (y)] dy g (y) if y = g(x) for some x,
fY (y) =
0 if y ̸= g(x) for all x.

where g −1 (y) is defined to equal that value of x such that g(x) = y.

Example: Let X be a continuous nonnegative random variable with density function

f , and let Y = X n . Find fY , the probability density function of Y .
If g(x) = xn , then
g −1 (y) = y 1/n
and
d −1 1
{g (y)} = y 1/n−1 .
dy n
Hence we obtain, for y ≥ 0,
1 1/n−1
fY (y) = y f (y 1/n ).
n

Remark: Even in the case where the function g is not invertible, we can determine the
distribution of g(X). For example, if X has a probability density fX , then Y = |X| has

6
a density function that is obtained as follows:
For y ≥ 0,

FY (y) = P (Y ≤ y)
= P (|X| ≤ y)
= P (−y ≤ X ≤ y)
= FX (y) − FX (−y)

Hence, on differentiation, we obtain

fY (y) = fX (y) + fX (−y) for y ≥ 0.

9 Some usual probability models

9.1 Discrete distributions
9.1.1 Bernoulli r.v
Suppose that a trial, or an experiment, whose outcome can be classified as either a success
or a failure is performed. If we let X = 1 when the outcome is a success and X = 0 when
it is a failure, then the probability mass function of X is given by:

PX (0) = P {X = 0} = 1 − p
PX (1) = P {X = 1} = p

where p is the probability that the trial is a success. The r.v X is said to be a Bernoulli
random variable with parameter p denoted by X ∼ B (p). Its mean and variance are
given by:
E(X) = p and V ar(X) = p − p2 = p(1 − p)

9.1.2 Binomial r.v

Suppose that n independent trials, each of which results in a success with probability p
or in a failure with probability 1 − p, are to be performed. If X represents the number of
successes that occur in the ntrials then its pmf is given by:

PX (k) = Cnk pk (1 − p)n−k k = 0, 1, ..., n

then X is said to be a binomial random variable with parameters (n, p) denoted by

X ∼ B (n, p). Its mean and variance are given by:

E(X) = np and V ar(X) = np(1 − p).

Example: It is known that screws produced by a certain company will be defective with
probability .01, independently of one another. The company sells the screws in packages
of 10 and offers a money-back guarantee that at most 1 of the 10 screws is defective.
What proportion of packages sold must the company replace? If X is the number of

7
defective screws in a package, then X ∼ B(10, .01). Hence, the probability that a package
will have to be replaced is

P (X > 1) = 1 − P (X ≤ 1) = 1 − P {X = 0} − P {X = 1}
0
= 1 − C10 (.01)0 (.99)10 − C10
1
(.01)1 (.99)9
= .004

Thus, only .4 percent of the packages will have to be replaced.

9.1.3 Poisson r.v

A random variable X, taking on one of the values 0, 1, 2, ..., is said to be a Poisson random
variable with parameter λ denoted X ∼ P(λ) , if for some λ > 0

λk
PX (k) = e−λ k = 0, 1, 2, ...,
k!
Its mean and variance are:

E(X) = λ and V ar(X) = λ.

Example: If the number of accidents occurring on a highway each day is a Poisson

random variable with parameter λ = 3, what is the probability that no accidents occur
today?
PX (0) = e−3 ≈ 0.05

Properties:
ˆ If a r.v X ∼ B (n, p), when n is large (n ≥ 30) and p is small enough (p ≤ 0.1) to
make np moderate, the number of successes occurring is approximately a Poisson
random variable with parameter λ = np.
ˆ For n random poisson variables X1 , X2 , · · · , Xn with respective parameters λ1 , λ2 , · · · , λn
Pn Pn
then Xi ∼ P( λi ).
i=1 i=1

9.1.4 Geometric r.v

Suppose that independent trials, each having probability p > 0 of being a success, are
performed until a success occurs. If we let X be the number of trials required until the
first success, then X is said to be a geometric random variable with parameter p denoted
by X ∼ G(p). Its probability mass function is given by:

PX (k) = (1 − p)k−1 p k = 1, 2, ...,

Its mean and variance are:

1 1−p
E(X) = and V ar(X) = .
p p2

8
9.2 Continuous distributions
9.2.1 Uniform r.v
A random variable X is said to be uniformly distributed over the interval [a, b] denoted
X ∼ U[a,b] if its probability density function is given by:

 1 if a ≤ x ≤ b
f (x) = b − a
0 otherwise

The uniform distribution arises in practice when we suppose a certain random variable
is equally likely to be near any value in the interval [a, b]. The c.d.f of a uniform random
variable is given by:

x 
 0 if x < a
Z x − a
F (x) = f (t)dt = if a ≤ x < b

 b−a
−∞ 
1 if x ≥ b

Its mean and variance are:

a+b (b − a)2
E(X) = and V ar(X) = .
2 12

9.2.2 Exponential r.v

A continuous random variable whose probability density function is given, for some λ > 0,
by: (
λe−λx if x ≥ 0
f (x) =
0 otherwise
is said to be an exponential random variable (or, more simply, is said to be exponentially
distributed) with parameter λ denoted X ∼ E(λ). The cumulative distribution function
F (x) of an exponential random variable is given by:
Zx
F (x) = P (X ≤ x) = f (t)dt
−∞
Zx
= λe−λt dt
0
= 1 − eλx , x ≥ 0.

with:
1 1
E(X) = and V ar(X) = 2 .
λ λ
In practice, the exponential distribution often arises as the distribution of the amount of
time until some specific event occurs, such as the time until a machine failure, customer
arrival, or radioactive decay...

9
 Figure 1: Exponential density.

Example: Suppose that the length of a phone call in minutes is an exponential random
variable with mean 10. If someone arrives immediately ahead of you at a public telephone
booth, find the probability that you will have to wait (a) more than 10 minutes;(b)
between 10 and 20 minutes

9.2.3 Gamma r.v

A random variable is said to have a gamma distribution with parameters(a, b),a > 0,
b > 0, denoted X ∼ γ(a, b) if its density function is given by
 −bx a−1
 be (bx) if x ≥ 0
f (x) = Γ(a)
0 otherwise


where Γ(a), called the gamma function, is defined as

Z∞
Γ(a) = e−x xα−1 dx
0

and:
a a
E(X) = and V ar(X) = .
b b2

Remark:
ˆ Integration of Γ(a) by parts yields: Γ(a) = (a − 1)Γ(a − 1).
ˆ For n ∈ N∗ we have : Γ(n) = (n − 1)!.
 
b
ˆ If a r.v X ∼ γ(a, b) then the r.v Y = cX ∼ γ a, , c > 0.
c
ˆ For a = 1 and b > 0, a r.v X ∼ γ(1, b) = E(b)

10
 Figure 2: Gamma density.

9.2.4 Normal r.v

We say that X is a normal random variable, or simply that X is normally distributed,
with parameters µ and σ 2 , denoted X ∼ N (µ, σ 2 ) if the density of X is given by:
1 1 x−µ 2
f (x) = √ e− 2 ( σ ) , with x ∈ R
2πσ
with:
E(X) = µ and V ar(X) = σ 2 .
This density function is a bell-shaped curve that is symmetric about µ.

Figure 3: Normal (Gaussian) density with µ = 0 and σ 2 = 1.

Some examples of random phenomena obeying this behavior are the heights, the weights,
the velocity in any direction of a molecule in gas,and the error made in measuring a
physical quantity...

Remak:
ˆ If a r.v X ∼ N (µ, σ 2 ) then the r.v Y = aX + b ∼ N (aµ + b, a2 σ 2 ) in particular the
r.v Z = X−µσ
∼ N (0, 1) such a random variable is said to be a standard, or a unit,
normal random variable.

11
ˆ We denote the cumulative distribution function of a standard normal distribution
Φ(x). That is, Z x
1 1 2
Φ(x) = √ e− 2 t dt
−∞ 2π

ˆ For any x ∈ R we have Φ(−x) = 1 − Φ(x)

ˆ Whenever X ∼ N (µ, σ 2 ) it follows that its distribution function can be expressed
in terms of Φ(.) as:
   
X −µ x−µ x−µ
FX (x) = P (X ≤ x) = P ≤ =Φ
σ σ σ

12
 Table 1: Area Φ(x) Under the Standard Normal Curve to the Left of X.
X .00 .01 .02 .03 .04 .05 .06 .07 .08 .09
0.0 .5000 .5040 .5080 .5120 .5160 .5199 .5239 .5279 .5319 .5359
0.1 .5398 .5438 .5478 .5517 .5557 .5596 .5636 .5675 .5714 .5753
0.2 .5793 .5832 .5871 .5910 .5948 .5987 .6026 .6064 .6103 .6141
0.3 .6179 .6217 .6255 .6293 .6331 .6368 .6406 .6443 .6480 .6517
0.4 .6554 .6591 .6628 .6664 .6700 .6736 .6772 .6808 .6844 .6879
0.5 .6915 .6950 .6985 .7019 .7054 .7088 .7123 .7157 .7190 .7224
0.6 .7257 .7291 .7324 .7357 .7389 .7422 .7454 .7486 .7517 .7549
0.7 .7580 .7611 .7642 .7673 .7704 .7734 .7764 .7794 .7823 .7852
0.8 .7881 .7910 .7939 .7967 .7995 .8023 .8051 .8078 .8106 .8133
0.9 .8159 .8186 .8212 .8238 .8264 .8289 .8315 .8340 .8365 .8389
1.0 .8413 .8438 .8461 .8485 .8508 .8531 .8554 .8577 .8599 .8621
1.1 .8643 .8665 .8686 .8708 .8729 .8749 .8770 .8790 .8810 .8830
1.2 .8849 .8869 .8888 .8907 .8925 .8944 .8962 .8980 .8997 .9015
1.3 .9032 .9049 .9066 .9082 .9099 .9115 .9131 .9147 .9162 .9177
1.4 .9192 .9207 .9222 .9236 .9251 .9265 .9279 .9292 .9306 .9319
1.5 .9332 .9345 .9357 .9370 .9382 .9394 .9406 .9418 .9429 .9441
1.6 .9452 .9463 .9474 .9484 .9495 .9505 .9515 .9525 .9535 .9545
1.7 .9554 .9564 .9573 .9582 .9591 .9599 .9608 .9616 .9625 .9633
1.8 .9641 .9649 .9656 .9664 .9671 .9678 .9686 .9693 .9699 .9706
1.9 .9713 .9719 .9726 .9732 .9738 .9744 .9750 .9756 .9761 .9767
2.0 .9772 .9778 .9783 .9788 .9793 .9798 .9803 .9808 .9812 .9817
2.1 .9821 .9826 .9830 .9834 .9838 .9842 .9846 .9850 .9854 .9857
2.2 .9861 .9864 .9868 .9871 .9875 .9878 .9881 .9884 .9887 .9890
2.3 .9893 .9896 .9898 .9901 .9904 .9906 .9909 .9911 .9913 .9916
2.4 .9918 .9920 .9922 .9925 .9927 .9929 .9931 .9932 .9934 .9936
2.5 .9938 .9940 .9941 .9943 .9945 .9946 .9948 .9949 .9951 .9952
2.6 .9953 .9955 .9956 .9957 .9959 .9960 .9961 .9962 .9963 .9964
2.7 .9965 .9966 .9967 .9968 .9969 .9970 .9971 .9972 .9973 .9974
2.8 .9974 .9975 .9976 .9977 .9977 .9978 .9979 .9979 .9980 .9981
2.9 .9981 .9982 .9982 .9983 .9984 .9984 .9985 .9985 .9986 .9986
3.0 .9987 .9987 .9987 .9988 .9988 .9989 .9989 .9989 .9990 .9990
3.1 .9990 .9991 .9991 .9991 .9992 .9992 .9992 .9992 .9993 .9993
3.2 .9993 .9993 .9994 .9994 .9994 .9994 .9994 .9995 .9995 .9995
3.3 .9995 .9995 .9995 .9996 .9996 .9996 .9996 .9996 .9996 .9997
3.4 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9997 .9998

To read the values of a standard normal distribution from table 1, locate the row corre-
sponding to the first part of the value and the column corresponding to the second part.
The value where the row and column intersect gives the cumulative probability up to
that value. This probability represents the area under the curve to the left of that value
in a standard normal distribution.
For example, if you want to find the cumulative probability for a value of 1.28 that is
Φ(1.28), look for the row labeled 1.2 and the column labeled 0.08. The intersection gives
a value of approximately Φ(1.28) = 0.8997, meaning the cumulative probability up to 1.28

13
is about 0.8997, or 89.97%.

14