STA 711: Lecture 3 – Random variables

Probability & Measure Theory

Duke University, Fall 2023

Author: Galen Reeves

Last Modified: September 12, 2023

Outline of lecture:
3.1 Random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
3.1.1 Function of random variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.1.2 Random vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.1.3 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2.1 Discrete random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2.2 Absolutely continuous random variables . . . . . . . . . . . . . . . . . . . . . 5
3.2.3 Infinite coin toss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Measurability & Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3.1 Canonical Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.3.2 The Inverse CDF method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3.3 Lebesgue decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.1 Random variables

• Let X be a function mapping from a set Ω to another set E

X : Ω → E, ω 7→ X(ω), ∀ω ∈ Ω

• For such a function, the set inverse X −1 is a set-valued function mapping from the power set
2E to the power set 2Ω according to

X −1 : 2E → 2Ω , X −1 (B) = {ω : X(ω) ∈ B}, ∀B ∈ 2E

We also use the compact notation:

[X ∈ B] := {ω : X(ω) ∈ B}

• Set-inverses are well-behaved functions from one class of sets to another:

◦ X −1 (E) = Ω
◦ X −1 (B c ) = {ω : X(w) ∈
/ B} = [X −1 (B)]c
◦ X −1 (A ∩ B) = {ω : X(ω) ∈ A, X(w) ∈ B} = X −1 (A) ∩ X −1 (B).
◦ More generally, for any collection {Aα } ⊂ E, countable or not,
[ [  \ \ 
X −1 (Aα ) = X −1 Aα and X −1 (Aα ) = X −1 Aα
α α α α
2 STA 711: Lecture 3

◦ Thus,
E is a field on E =⇒ X −1 (E) is a field on Ω
E is a σ-field on E =⇒ X −1 (E) is a σ-field on Ω.
• Recall that a measurable space (Ω, F) consists of a set Ω and a σ-algebra F defined on Ω.
In order to define a random variable, we restrict our attention to functions defined from one
measurable space (Ω, F) to another measurable space (E, E).
• Definition: A function X : (Ω, F) → (E, E) is measurable if its inverse maps elements of E
to elements of F, i.e,
X −1 (B) ⊂ F, ∀B ∈ E ⇐⇒ X is F\E-measurable
This condition can be expressed compactly as X −1 (E) ⊂ F.
• Warning: The backslash character “\” in this notation is unrelated to the backslash character
that appears in the common notation for set exclusion, A \ B := A ∩ B c .
• Definition: A real-valued random variable X is a measurable function from measurable
space (Ω, F) to (R, B), i.e., the real line R equipped with the Borel σ-algebra B,
X : (Ω, F) → (R, B) is F\B measurable

• Example: Let Ω = {a, b, c}. Define a real-valued random variable

X(ω) = 1{a} (ω)
For what choices of F is this function F\B measurable?
F = 2Ω =⇒ yes
F = σ({a}) =⇒ yes
F = σ({b}) =⇒ no, X −1 ({0}) = {b, c} is not in σ({b})

• Definition: More generally, an E-valued random variable X is a measurable function from

measurable space (Ω, F) to another measurable space (E, E).
• We often consider the case where (E, d) is a metric space and E is the Borel σ-algebra
generated by the open balls in (E, d).

3.1.1 Function of random variable

• Question: For a random variable X : (Ω, F) → (E, E) and function g : E → R, under what
conditions is Y = g(X) a random variable?

[illustration]
• Answer: We need Y to be a measurable mapping from (Ω, F) to (R, B) where B are the
Borel sets. Hence, for every B ∈ B, we need
Y −1 (B) = {ω : g(X(ω)) ∈ B} = {ω : X(ω) ∈ g −1 (B)} ⊂ F
This occurs if g −1 (B) ∈ E for all B. So the condition we need is that g is E\B-measurable.
• Pretty much every function g : E → R you’ll ever encounter is Borel measureable. In particu-
lar, a real-valued function g(x) is Borel measurable if it is continuous, or right-continuous, or
piecewise continuous, or monotonic, or the countable limits, suprema, etc. of such functions.
STA 711: Lecture 3 3

3.1.2 Random vectors

• Denote by R2 the set of points (x, y) in the plane, and by B 2 the sigma field generated by
rectangles of the form {(x, y) : a < x ≤ b, c < y ≤ d} = (a, b] × (c, d]
• Definition: A measurable mapping X : (Ω, F) → (R2 , B 2 ) is called a (two-dimensional)
random vector, or simply an R2 -valued random variable. The components X1 , X2 of a random
vector are each random variables.
• Conversely, for any two random variables X1 : (Ω, F) → (R, B) and X2 : (Ω, F) → (R, B) the
two-dimensional vector (X1 , X2 ) is a measurable mapping from (Ω, F) to (R2 , B 2 ) and thus
it is random vector.
• Similarly, if X1 , . . . , Xn are RVs, then X = (X1 , . . . , Xn ) is an Rn -valued RV.
• Question: If {Xn } are RVs is the supremum a RV?
X ∗ = sup Xn
n
◦ Yes, observe that \
{ω : X ∗ (ω) ≤ r} = {ω : Xn (ω) ≤ r} ∈ F
n
◦ The σ-field generated by {{ω : X ∗ (ω) ≤ r} : r ∈ R} is the Borel σ-field B.
◦ Hence, X ∗−1 (B) ⊂ F and so X ∗ is a RV.
• By same arguments, the following are also RVs
X
Xn , inf Xn sup Xn lim inf Xn lim sup Xn
n n n n
n

• Note, the event that Xn converges is in F and so we can find its probability:
{ω : Xn (ω) converges} = {ω : lim sup Xn − lim inf Xn (ω) = 0}
n n

This is one point where countable additivity (and not just finite additivity) of P is crucial,
and where F must be a sigma field (and not just a field).

3.1.3 Distributions
• Recall that a probability space consists of the triple (Ω, F, P) where Ω is a set, F is a σ-field
on Ω, and P is a probability measure defined for each element of F.
• Definition: An E-valued random variable X on a probability space (Ω, F, P) induces a
probability measure µX on (E, E), called the distribution measure (or simply the distribution),
according to
µX (B) := P[X ∈ B]
This is denoted variously as µX = P ◦ X −1 or PX −1 .
• Note that the distribution measure defines a new probability space given by (µX , E, E).
• Definition: A real-valued random variable X defined on a probability space (Ω, F, P) induces
a distribution function FX according to
FX (x) = P(X ≤ x)

• From the results in the previous lecture, we know that the distribution function FX defines
the distribution measure µX uniquely on (R, B).
4 STA 711: Lecture 3

3.2 Examples
3.2.1 Discrete random variables
• Definition: A random variable is called discrete if takes only countably-many values.

• For a discrete random variable X defined on (Ω, F), let {bj } be an enumeration of its possible
values. The collection of sets {Λj } ⊂ F defined by

Λj = X −1 ({bj }) = {ω : X(ω) = bj }

forms a countable partition of Ω, i.e.,

[
Ω= Λi , and Λi ∩ Λj = ∅, ∀i 6= j
i

Hence, a discrete random variable can be expressed as

X
X(ω) = bi 1Λi (ω)
i

where (
1 if ω ∈ Λ
1Λ (ω) :=
0 if ω ∈
/Λ

is the indicator function of Λ ∈ F.

• The distribution function is

X
FX (x) = P(X ≤ x) = P(Λj )1{bj ≤x}
j

and the probability distribution is

X
µX (B) = P(X ∈ B) = P(Λj )1{bj ∈B}
j

• If X is a discrete random variable X and g : R → R is an arbitrary function (measurable or

not) it follows that
Y = g(X)

is also a random variable (i.e., it is measurable mapping from (Ω, F) to (R, B)) that can be
expressed as
X X [
Y (ω) = g(bj )1Λj (ω) = ai 1Ai (ω), Ai = {Λj : g(bj ) = ai }
j i i

The distribution function is

    X
µY (B) = P Y ∈ B = P ∪ {Λj : g(bj ) ∈ B} = P(Λj )1{g(bj )∈B}
j
STA 711: Lecture 3 5

3.2.2 Absolutely continuous random variables

• Definition: A distribution function F is absolutely continuous if there exists a function f
on R such that f ≥ 0 and Z x
F (x) = f (s) ds ∀x ∈ R
−∞

(This is the Lebesgue integral, which we will learn more about later on.) The function f is
called that probability density function (pdf).

• A random variable X is said to be absolutely continuous if its distribution function is abso-

lutely continuous. The distribution measure is
Z Z
µX (B) := P[X ∈ B] = f (x) dx = f (x)1B (x) dx
B R

• Question: Let X is an absolutely continuous random variable with distribution function

F (x) = P(X ≤ x)

and let φ be a function from R to R. Is Y = φ(X) a random variable? Is it absolutely

continuous?

◦ Y is a random variable if φ is a Borel function, i.e., it is measurable with respect to the

Borel sets.
◦ If φ is strictly increasing and differentiable then for each y ∈ R there exists x such that
φ(x) = y and so distribution function G of Y satisfies

G(y) := P[Y ≤ y]
= P[φ(X) ≤ φ(x)]
= P[X ≤ x]
= F (x)

By absolute continuity of F , we can write

Z x
G(φ(x)) = f (s) ds
−∞

and differentiating both sides gives

f (x)
G0 (φ(x))φ0 (x) = f (x) ⇐⇒ G0 (y) =
φ0 (x) x : g(x)=y

◦ If φ is strictly decreasing and differentiable then

Z ∞
G(y) = P[φ(X) ≤ φ(x)] = P[X > x] = 1 − F (x) = f (s) ds
x

and so
f (x) f (x)
G0 (y) = − =
φ0 (x) x : g(x)=y |φ0 (x)| x : g(x)=y
6 STA 711: Lecture 3

◦ More generally, if φ is everywhere differentiable with a derivative φ0 (x) that vanishes on

at most countably many points then φ(X) has pdf g given by the change of variable
formula:
X f (x)
g(y) = 0 (x)|
, if {x : φ0 (x) = 0} is countable
−1
|φ
x∈φ (y)

• Example If X ∼ N (0, 1) then

1 2
f (x) = √ e−x /2
2π
If φ(x) = |x| then

−1,
 x<0
0
φ (x) = +1 x>0

DNE, x = 0


The pdf for |X| is

X f (x) p 2
0
= 2/πey /2 1y>0
|φ (x)|
x:|x|=y

3.2.3 Infinite coin toss

• Let Ω = (0, 1], F = Borel sets, and P = Lebesgue measure, i.e.,
P((a, b]) = b − a, 0≤a≤b≤1

• Each ω ∈ Ω can be represented by its binary expansion

∞
X
ω= δi 2−i , δi ∈ {0, 1} ⇐⇒ ω = 0.δ1 δ2 δ3 . . . , δi ∈ {0, 1}
i=1

where δn is the n-th “bit’

• There is some ambiguity because 1/2 can be written as either 0.10000 . . . or 0.0111111 . . .
◦ The first representation corresponds to the rule
δn (ω) = b2n ωc (mod 2)
such that
{ω : δ1 (ω) = 1} = [1/2, 1)
◦ We will use the second representation, which corresponds to the rule
δn (ω) = (d2n ωe − 1) (mod 2).
such that
{ω : δ1 (ω) = 1} = (1/2, 1]
• What is the distribution of δn ?
P({ω : δ1 (ω) = 1}) = P((1/2, 1]) = 1/2
P({ω : δ2 (ω) = 1}) = P((1/4, 1/2] ∪ (3/4, 1]) = 1/4 + 1/4 = 1/2
P(δn = 1) = 2n−1 × 2−n = 1/2
This is a sequence of Bernoulli random variables.
STA 711: Lecture 3 7

• What is the distribution of

∞
X
X= 2−n δn = ω
n=1

This is the uniform distribution on (0, 1]. We write X ∼ Un(0, 1).

• It is possible to construct countably many independent uniform variables:

∞
X
X1 = 2−n δ2n
n=1
X∞
X2 = 2−n δ3n
n=1
X∞
X3 = 2−n δ5n
n=1
X∞
X4 = 2−n δ7n
n=1

where the base for the i-th variable is the i-th prime.

• For Ω = (0, 1] let us define

δn (ω) = (d2n ωe − 1) (mod 2)

and the random variables

n
X n
X
Sn := δj , Yn := 2Sn − n = (2δj − 1)
j=1 j=1

• For each n ∈ N, the σ-field generated by the first n terms is the same:

Fn := σ({δj : j ≤ n}) = σ({Sj : j ≤ n}) = σ({Yj : j ≤ n}) = σ({(0, j/2n ] : j ≤ n})

• Question: Is δ10 measurable over Fn ?

◦ If n ≥ 10 then yes.
◦ If n < 10 then no.

3.3 Measurability & Observability

• Definition: For random variable X : (Ω, F) → (E, E), the σ-field generated by X, is the
smallest σ-field G such that X is G\E-measurable. It is often denoted by FX or σ(X):

σ(X) = X −1 (E) = {{ω : X(ω) ∈ B} : B ∈ B} = {[X ∈ B] : B ∈ B}

• Example: If (Ω, F) = (R, B) and X(ω) = 1{(−∞,0]} (ω) then

σ(X) = {∅, R, (−∞, 0], (0, ∞)}

• Proposition: Let X and Y be random variables on (Ω, F). Then, σ(X) ⊂ σ(Y ) if an only
if X = g(Y ) for some Borel function g.
8 STA 711: Lecture 3

• Proof of if: If g is a Borel function the g −1 (B) ∈ B for all B ∈ B. Thus

σ(X) = {{ω : X(ω) ∈ B} : B ∈ B}

= {{ω : g(Y (ω)) ∈ B} : B ∈ B}
= {{ω : Y (ω) ∈ g −1 (B)} : B ∈ B}
⊂ {{ω : Y (ω) ∈ B 0 } : B 0 ∈ B} since g −1 (B) ∈ B
= σ(Y )

• Proof of only if:

◦ Suppose σ(X) ⊂ σ(Y ). We show there exists a Borel function g such that X = g(Y ).
◦ The assumption σ(X) ⊂ σ(Y ) means that for each n ∈ N and j ∈ Z, there exists Bjn ⊂ B
such that
n o
Y −1 (Bjn ) = {ω : Y (ω) ∈ Bjn } = ω : 2jn ≤ X(ω) < j−1
2n ⊂ σ(X)

◦ For each n ∈ N, the collection {Bjn : j ∈ Z} is disjoint.

◦ Define
X j
g n (y) := 1B n (y)
2n j
j∈Z

◦ By construction,
1
g n (Y ) ≤ X < g n (Y ) +
2n
Hence X = g(Y ) where g := lim sup gn . (Note that g is a Borel function because it is
the limsup of Borel functions).

3.3.1 Canonical Space

• Example: How should we choose Ω to model the outcomes of two dice?

◦ If we care only about the maximum value need #(Ω) ≥ 6

◦ If we care only about the sum need #(Ω) ≥ 11
◦ want both values and distinguishable need #(Ω) ≥ 36
◦ want both values and indistinguishable need #(Ω) ≥ 21

• Example continued: Let Ω = (0, 1] with Lebesgue P.

◦ Let the red die be given by

R(ω) = d6ωe

◦ Can construct an independent green die according to

G(ω) = (d36ωe − 1) (mod 6) + 1

◦ Can construct another independent blue die according to

B(ω) = (d216ωe − 1) (mod 6) + 1

STA 711: Lecture 3 9

• Definition: If X : (Ω, F, P) → (E, E) is an E-valued RV, then one choice of (Ω, F, P) that
always works is

Ω := E, F := E, P := µX , µX (B) = P(X ∈ B)

• For dice example with one die this is

#(A)
Ω = {1, 2, 3, 4, 5, 6}, F = 2Ω , P(A) =
6

3.3.2 The Inverse CDF method

• Let (Ω, F, P) = ((0, 1], B, P) be the unit interval the Borel sets and the Lebesgue measure.

• Let F (x) be any distribution function, i.e., non-decreasing, right-continuous function on R

with limits F (−∞) = 0 and F (∞) = 1.

• Define the left-continuous inverse

F ← (ω) := inf{x ∈ R : F (x) ≥ ω},

• Define random variable on (Ω, F, P)

X(ω) = F ← (ω)

Then, X is a random variable on (Ω, F, P) with distribution function F , because for any
x ∈ R,
{ω : X(ω) ≤ x} = (0, F (x)]

• Example:

F (x)
1

−2 −1 0 1 2 3 4 5 6 7
x
10 STA 711: Lecture 3

3
F ← (x)
2

ω
0 a c
b 1

−1

−2

• Example: Let X ∼ Bern(p). Then

F (x)
1

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2

x

Observe that
σ(X) = {∅, (0, p], (p, 1], Ω}

3.3.3 Lebesgue decomposition

• Recall that δn are independent Bernoulli 1/2 variables. Define
∞
X
Y := 2δn 3−n
n=1

Then, Y has a singular continuous distribution function F such that

STA 711: Lecture 3 11

1. F is continuous
2. F 0 (x) = 0 wherever it exists.

F (x) 0.5

0
0 1/9 2/9 1/3 2/3 7/9 8/9 1
x

• Proposition: Let F be any distribution function. Then there exits almost unique numbers
pd , pac , psc and distribution functions Fd , Fac , and Fsc such that

F = pd Fd + pac Fac + psc Fsc

and a random variable with distribution function F can be written as a mixture

P(discrete) = pd , P(abs cont) = pac , P(sing cont) = psc

• These are almost unique because if, say, psc is zero then there are no constraints on Fsc .