Chapter 3: Random Variables and Probability

Distributions

These notes are written for second-year students with Calculus I–II (including multivariable
calculus). We avoid measure-theory language as much as possible.

3.1 Concept of a random variable

3.1.1 What is a random variable?
Example 3.1.1

Toss a fair coin once. If it lands Heads you win $1, and if it lands Tails you lose $1.

S = {H, T }, X(H) = 1, X(T ) = −1.

So X : S → {−1, 1} ⊂ R is a random variable.

Definition 3.1.1: Random variable

Let S be a sample space (the set of all outcomes of an experiment) equipped with a probability
rule P. A random variable is a real-valued function

X : S → R.

We usually use capital letters (X, Y, Z, . . . ) for random variables and lowercase letters
(x, y, z, . . . ) for their numerical values.

Example 3.1.2: Number of heads in 4 tosses

Toss a fair coin 4 times and let X be the number of Heads. Then S consists of 24 sequences
such as HHT T , and
X(HHHH) = 4, . . . , X(T T T T ) = 0,
so the possible values are {0, 1, 2, 3, 4}.

Example 3.1.3: Toss until the first Head

Toss a fair coin repeatedly until the first Head occurs. Let X be the number of tosses. Then
the outcomes look like
H, T H, T T H, T T T H, . . .
and X(H) = 1, X(T H) = 2, X(T T H) = 3, . . . so X takes values in {0, 1, 2, 3, . . .}.

1
Chapter 3: Random Variables 2

Example 3.1.4: Uniform random variable on [0, 1]

Let S = [0, 1] and suppose we pick s ∈ [0, 1] “equally likely.” Define X(s) = s. Then
X : S → R is a random variable.

Remark 3.1.1: Discrete vs. continuous (informal)

• X is discrete if it takes at most countably many values, e.g. {0, 1, 2, . . .} (see Section 3.2).
• X is continuous if it takes values on an interval and its CDF is continuous. (see
Section 3.3 for continous random variables with density functions.)

3.1.2 Distribution and CDF

Definition 3.1.2: Distribution of a random variable
Let X : S → R be a random variable. For a set A ⊆ R, define


P(X ∈ A) := P {s ∈ S : X(s) ∈ A} .

The set function A 7→ P(X ∈ A) is called the (probability) distribution of X.

Definition 3.1.3: Same distribution

d
We say X and Y have the same distribution (written X = Y ) if

P(X ∈ A) = P(Y ∈ A) for all A ⊆ R.

Definition 3.1.4: Cumulative distribution function (CDF)

The cumulative distribution function (CDF) of a random variable X is

FX (x) := P(X ≤ x), x ∈ R.

SWhen the random variable X is clear from the context, we often drop the subscript and
simply write F (x) when the random variable is clear from context.

Remark 3.1.2: CDF determines the distribution

If two random variables have the same CDF, then they have the same distribution. So
knowing FX is enough to determine probabilities of the form P(X ∈ A).

Example 3.1.5

Toss a coin with P(H) = 0.3 and P(T ) = 0.7. Define

(
1, if Heads occurs,
X=
−1, if Tails occurs.
Chapter 3: Random Variables 3

Then 
x < −1,
0,


FX (x) = P(X ≤ x) = 0.7, −1 ≤ x < 1,

x ≥ 1.

1,

3.1.3 Properties of the CDF

Theorem 3.1.1: Basic properties of a CDF

Let F (x) = FX (x) = P(X ≤ x). Then:

(i) 0 ≤ F (x) ≤ 1 for all x ∈ R.
(ii) lim F (x) = 0 and lim F (x) = 1.
x→−∞ x→+∞

(iii) F is non-decreasing: if x ≤ y then F (x) ≤ F (y).

(iv) F is right-continuous and has left limits. We write

F (x− ) := lim F (y) = P(X < x).

y↑x

(v) For a < b,

P(a < X ≤ b) = F (b) − F (a), P(X > a) = 1 − F (a).

Remark 3.1.3: When does a function qualify as a CDF?

A function F : R → [0, 1] that is non-decreasing, right-continuous, and has limx→−∞ F (x) = 0

and limx→+∞ F (x) = 1 is the CDF of some random variable.

3.2 Discrete probability distributions

Definition 3.2.1: Discrete random variable and possible values
A random variable X is discrete if it takes at most countably many values. A number x ∈ R
is called a possible value of X if P(X = x) > 0.

Definition 3.2.2: Probability mass function (pmf)

If X is discrete, its probability mass function (pmf) is

fX (x) := P(X = x).

Remark 3.2.1: Basic facts about a pmf

Let f be the pmf of a discrete random variable X.
(1) 0 ≤ f (x) ≤ 1 for all x.
Chapter 3: Random Variables 4

X
(2) f (x) = 1 (sum over all possible values).
x

(3) The CDF is a step function:

X
FX (x) = P(X ≤ x) = f (y).
y≤x

(4) For any set A of real numbers,

X
P(X ∈ A) = f (x).
x∈A

(5) The jump size of F at x equals the point mass:

f (x) = P(X = x) = FX (x) − FX (x− ).

Example 3.2.1: Binomial distribution: heads in 4 fair tosses

Let X be the number of Heads in 4 independent fair coin tosses. Then
!
4  1 4
P(X = k) = 2 , k = 0, 1, 2, 3, 4.
k

k 0 1 2 3 4
1 4 6 4 1
P(X = k) 16 16 16 16 16

For example,
1+4+6 11
P(X ≤ 2.5) = P(X ≤ 2) = P(X = 0) + P(X = 1) + P(X = 2) = = .
16 16

3.3 Continuous distributions

3.3.1 Motivating example: uniform on [0, 1]
Example 3.3.1

Pick a number “equally likely” from [0, 1] and call it X.

• For any fixed s ∈ [0, 1], P(X = s) = 0.
• P(0 ≤ X ≤ 12 ) = 21 .
• More generally, for 0 ≤ a ≤ b ≤ 1, P(a ≤ X ≤ b) = b − a.
A density that produces these probabilities is
(
1, 0 ≤ x ≤ 1,
f (x) =
0, otherwise.
Chapter 3: Random Variables 5

Rx
Then FX (x) = −∞ f (t) dt equals

x < 0,
0,


FX (x) = x, 0 ≤ x ≤ 1,


1, x > 1.

3.3.2 Probability density functions

Definition 3.3.1: Probability density function (pdf)

A piecewise continuous function f : R → [0, ∞) is called a probability density function

(pdf) of a random variable X if
Z
P(X ∈ A) = f (x) dx for every set A ⊂ R.
A

Remark 3.3.1
If X has pdf f and CDF F :
Z x
(1) F (x) = f (t) dt and F is continuous
−∞

(2) For which f is continuous, F 0 (x) = f (x) (Fundamental Theorem of Calculus).

(3) For every x ∈ R, P(X = x) = 0.
(4) Therefore, endpoint choices do not matter:
Z b
f (x) dx = P(a < X < b) = P(a ≤ X ≤ b) = P(a < X ≤ b) = P(a ≤ X < b).
a
Z
P(X ∈ A) = f (x) dx.
A

Remark 3.3.2
R∞
Any piecewise continuous function f such that f (x) ≥ 0 and −∞ f (x) dx = 1 can serve as a
pdf of some random variable.

Example 3.3.2: A piecewise continous pdf

Let 
x,

 0 < x < 1,
f (x) = ce−cx , x ≥ 1,


0, otherwise.
Suppose f is a pdf of some random variable.
Chapter 3: Random Variables 6

R∞
1) Find c. We require −∞ f (x) dx = 1:
Z 1 Z ∞
1
1= x dx + ce−cx dx = + e−c .
0 1 2

So e−c = 21 , hence c = ln 2.
2) Find the CDF. For x < 0, F (x) = 0. For 0 ≤ x < 1,

x2
Z x
F (x) = t dt = .
0 2
For x ≥ 1, Z x
1 1
ce−ct dt = + e−c − e−cx = 1 − e−cx .


F (x) = +
2 1 2
Using c = ln 2, this becomes F (x) = 1 − 2−x for x ≥ 1.
1


3) Compute P 2 <X<2 .

(1/2)2 1 1 5
       
1 1 −2c
P 2 < X < 2 = F (2) − F 2 = 1−e − = 1− − = .
2 4 8 8

3.3.3 Quick summary: CDF, pmf, and pdf

The table below is a quick “translation guide” between the discrete and continuous cases.

Concept Discrete (X has pmf f ) Continuous (X has pdf f )

X Z
Probabilities P(X ∈ A) = f (x) P(X ∈ A) = f (x) dx
x∈A A
X Z x
CDF F (x) = f (y) F (x) = f (t) dt
y≤x −∞
Point probability P(X = x) = f (x) (can be > 0) P(X = x) = 0
Recover f from F f (x) = F (x) − F (x− ) f (x) = F 0 (x) (when differentiable)

For joint distributions (X, Y ), replace sums by double sums in the discrete case and single
integrals by double integrals in the continuous case:
 X

 fX,Y (x, y), discrete,

(x,y)∈A
P (X, Y ) ∈ A = ZZ
fX,Y (x, y) dx dy,


 continuous.
A
Chapter 3: Random Variables 7

3.4 Joint probability distributions

3.4.1 Joint distributions: discrete case
Definition 3.4.1: Joint pmf

Let X and Y be discrete random variables. A function fX,Y (x, y) is called a joint pmf of
(X, Y ) if
(i) P(X = x, Y = y) = fX,Y (x, y) for all x, y,
(ii) fX,Y (x, y) ≥ 0 for all x, y,
XX
(iii) fX,Y (x, y) = 1.
x y
(fX,Y represents joint pmf of (X, Y ). If it is clear, we use f (x, y) instead of fX,Y (x, y).)

Definition 3.4.2: Joint CDF

The joint CDF of (X, Y ) is

FX,Y (x, y) = P(X ≤ x, Y ≤ y).

If (X, Y ) is discrete, then XX

FX,Y (x, y) = fX,Y (s, t).
s≤x t≤y

Example 3.4.1: Two balls without replacement: joint pmf

Suppose an urn contains 3 blue balls, 2 red balls, and 4 green balls. We select 2 balls at
random without replacement. Let

X = # of red balls, Y = # of green balls.

Then X ∈ {0, 1, 2} and Y ∈ {0, 1, 2} but only pairs with x + y ≤ 2 are possible.
There are 92 = 36 equally likely unordered draws. For x, y ≥ 0 with x + y ≤ 2,

2
4
3

x y 2−x−y
P(X = x, Y = y) = 9
.
2

P(X = x, Y = y) y=0 y=1 y=2

1 1 1
x=0 12 3 6
1 2
x=1 6 9 0
1
x=2 36 0 0

Question: Find P(Y = 0).

1 1 1 5
P(Y = 0) = P(X = 0, Y = 0) + P(X = 1, Y = 0) + P(X = 2, Y = 0) = + + = .
12 6 36 18
Chapter 3: Random Variables 8

3.4.2 Marginal distributions

Definition 3.4.3: Marginal pmfs (discrete)

If (X, Y ) has joint pmf fX,Y , then the marginal pmfs are
X X
fX (x) = P(X = x) = fX,Y (x, y), fY (y) = P(Y = y) = fX,Y (x, y).
y x

Remark 3.4.1: Do marginals determine the joint distribution?

In general, no. Knowing fX and fY does not uniquely determine fX,Y , unless we also know
that X and Y are independent (Section 3.5.2). (Can you construct an example?)

3.4.3 Joint distributions: continuous case

Definition 3.4.4: Joint pdf

Let X and Y be continuous random variables. A function fX,Y (x, y) is a joint pdf of (X, Y )
if
(i) fX,Y (x, y) ≥ 0,
Z ∞ Z ∞
(ii) fX,Y (x, y) dx dy = 1,
−∞ −∞

(iii) for every region A ⊆ R2 (e.g. a box, A = [a1 , a2 ] × [b1 , b2 ]),

ZZ


P (X, Y ) ∈ A = fX,Y (x, y) dx dy.
A

Remark 3.4.2: Joint CDF and derivatives

If (X, Y ) has joint pdf fX,Y , then the joint CDF is
Z x Z y
FX,Y (x, y) = P(X ≤ x, Y ≤ y) = fX,Y (a, b) db da.
−∞ −∞

Under mild smoothness conditions (e.g. for which fX,Y (x, y) is continous),

∂2
fX,Y (x, y) = FX,Y (x, y).
∂x ∂y

Definition 3.4.5: Marginal pdfs (continuous)

If (X, Y ) has joint pdf fX,Y , then

Z ∞ Z ∞
fX (x) = fX,Y (x, y) dy, fY (y) = fX,Y (x, y) dx.
−∞ −∞
Chapter 3: Random Variables 9

Example 3.4.2: Uniform distribution on a square

Let (X, Y ) be uniformly distributed on [0, 2] × [0, 2]. Then the joint pdf is

1, (x, y) ∈ [0, 2] × [0, 2],
4
fX,Y (x, y) =
0, otherwise.

1) Compute P(X + Y ≤ 1). The set {(x, y) : x + y ≤ 1} inside the square is a right triangle
with area 12 . Because the density is constant 14 ,

1 1 1 1
ZZ
P(X + Y ≤ 1) = dx dy = · = .
x+y≤1 4 4 2 8

2) Find the marginal densities. For x ∈ [0, 2],

Z 2
1 1
fX (x) = dy = ,
0 4 2
1
and similarly fY (y) = 2 for y ∈ [0, 2]. So X ∼ U nif orm(0, 2) and Y ∼ U nif orm(0, 2).

Example 3.4.3

Let (X, Y ) have joint pdf

(
Cxy, 0 < x < 1, 0 < y < 1, x + y < 1,
fX,Y (x, y) =
0, otherwise.

1) Find C.
Z 1 Z 1−x
C
ZZ
1= fX,Y (x, y) dx dy = Cxy dy dx = ,
0 0 24
so C = 24.
1
2) Compute P(X ≤ Y ). The condition X ≤ Y inside x + y < 1 forces 0 ≤ x ≤ 2 and
y ∈ [x, 1 − x], so
Z 1/2 Z 1−x
1
P(X ≤ Y ) = 24xy dy dx = .
0 x 2
3) Find the marginals. For 0 < x < 1,
Z 1−x
fX (x) = 24xy dy = 12x(1 − x)2 ,
0

and similarly, for 0 < y < 1,

fY (y) = 12y(1 − y)2 .
Chapter 3: Random Variables 10

3.5 Conditional distributions and independence

3.5.1 Conditional distributions
Remark 3.5.1: Reminder: conditional probability

For events A and B with P(B) > 0,

P(A ∩ B)
P(A | B) = .
P(B)

Definition 3.5.1: Conditional pmf and pdf

Discrete case. If (X, Y ) has joint pmf fX,Y and fY (y) > 0, the conditional pmf of X
given Y = y is
fX,Y (x, y)
P(X = x | Y = y) = fX|Y (x | y) = .
fY (y)
Continuous case. If (X, Y ) has joint pdf fX,Y and fY (y) > 0, the conditional pdf of X
given Y = y is
fX,Y (x, y)
fX|Y (x | y) = .
fY (y)

Remark 3.5.2: How to use conditional distributions

• Discrete: P(X ∈ A | Y = y) = | y) and | y) = 1.
P P
x∈A fX|Y (x x fX|Y (x
R R∞
• Continuous: P(X ∈ A | Y = y) = A fX|Y (x | y) dx and −∞ fX|Y (x | y) dx = 1.
• Contional pmfs and conditional pdfs are pmfs and pdfs respectively.

Example 3.5.1
Roll a fair die and let Y be the number shown on the die. If the result is Y = y, then toss a
biased coin y times, where P(Head) = p. Let X be the number of Heads.
1) Find P(X = x, Y = y).
!
1 y x
P(X = x, Y = y) = P(Y = y) P(X = x | Y = y) = p (1 − p)y−x ,
6 x

for y ∈ {1, 2, 3, 4, 5, 6} and x = 0, 1, . . . , y (and 0 otherwise).

2) Find P(X = 4). Only y ≥ 4 contribute:
6
X p4 h i
P(X = 4) = P(X = 4, Y = y) = 1 + 5(1 − p) + 15(1 − p)2 .
y=4
6
Chapter 3: Random Variables 11

3.5.2 Independence
Definition 3.5.2: Independence of events
Events A and B are independent if

P(A ∩ B) = P(A) P(B).

In general, A1 , . . . , An are independent if for any 1 ≤ i1 < i2 < · · · < ik ≤ n

P(Ai1 ∩ · · · ∩ Aik ) = P(Ai1 ) · · · P P (Aik )

Theorem 3.5.1
(i) If an event A is independent of any event (including itself), then P(A) = 0 or 1.
(ii) If P(A) = 0 or 1, then A is independent of any event.

Definition 3.5.3: Independence of random variables

Random variables X and Y are independent if for all sets A, B ⊆ R,

P(X ∈ A, Y ∈ B) = P(X ∈ A) P(Y ∈ B).

More generally, X1 , . . . , Xn are independent if

n
Y
P(X1 ∈ A1 , . . . , Xn ∈ An ) = P(Xi ∈ Ai ) for all A1 , . . . , An ⊆ R.
i=1

Theorem 3.5.2: Factorization test for independence

Suppose (X, Y ) has a joint pmf or joint pdf fX,Y and marginal pmf/pdf fX , fY . Then X and
Y are independent if and only if

fX,Y (x, y) = fX (x) fY (y) for all x, y ∈ R.

Example 3.5.2

Suppose (X, Y ) has joint pmf

e−2
fX,Y (x, y) = , x = 0, 1, 2, . . . , y = 0, 1, 2, . . .
x! y!
Then
e−1 e−1
fX,Y (x, y) = ,
x!
|{z} y!
|{z}
fX (x) fY (y)

so X and Y are independent. (In fact, X ∼ Poisson(1) and Y ∼ Poisson(1).)

Chapter 3: Random Variables 12

Example 3.5.3

In Example 3.4.3, the joint pdf is fX,Y (x, y) = 24xy on the triangle x + y < 1. But

fX (x)fY (y) = 12x(1 − x)2 12y(1 − y)2 6= 24xy,

  

so X and Y are not independent.

Example 3.5.4: Max and min of two independent uniforms

Let X and Y be independent and uniformly distributed on [0, 1]. Equivalently, the joint pdf
is fX,Y (x, y) = 1 on [0, 1] × [0, 1] and 0 otherwise.
(a) Let Z = max(X, Y ). Find the distribution of Z. For 0 ≤ z ≤ 1,

FZ (z) = P(max(X, Y ) ≤ z) = P(X ≤ z, Y ≤ z) = z 2 .

So 
0,

 z < 0,
FZ (z) = z2, 0 ≤ z ≤ 1, fZ (z) = FZ0 (z) = 2z (0 < z < 1).


1, z > 1,
(b) Let W = min(X, Y ). Find the distribution of W . For 0 ≤ w ≤ 1,

P(W > w) = P(X > w, Y > w) = (1 − w)2 ,

so
FW (w) = P(W ≤ w) = 1 − (1 − w)2 = 2w − w2 .
Hence fW (w) = 2 − 2w for 0 < w < 1.