Chapter 3

Conditional Probabilities, Independence, Product Measures

If one considers the analytical methods to which this theory has given rise, the truth of
the principles on which it rests, the fine and delicate logic required in applying them to
the solution of problems, the public institutions that rely upon it, and the extensions
it has received and may still receive through its application to the most important
questions of natural philosophy and the moral sciences; if one further observes that, even
in matters that cannot be subjected to calculation, it provides the soundest insights
that can guide our judgments, and teaches us to guard against the illusions that so
often lead us astray, one sees that there is no science more worthy of our meditation,
and none that would be more useful to introduce into the system of public education.

Pierre Simon de Laplace, Théorie Analytique des Probabilités

Until now we have treated probability theory largely as a part of analysis. In this chapter we
come to central concepts that mathematically establish the independence of probability theory as a
discipline.

1 Conditional Probabilities

Let (Ω, F, P ) be an arbitrary probability space and let A, B ∈ F be two events. The probability of
A ∩ B, i.e. the simultaneous occurrence of both events, satisfies

P (A ∩ B) ≤ min(P (A), P (B)).

What interests us now is how information about the event B affects our assumptions about the
event A. We therefore define conditional probability.
Definition 3.1. Let (Ω, F, P ) be a probability space and let A, B ∈ F with P (B) > 0. Then
P (A ∩ B)
P (A | B) ≡ (3.1)
P (B)
is called the conditional probability of A given B.
This definition is intuitive and compatible with the frequentist interpretation. If P is an empirical
distribution, then P (A | B) is the frequency with which A occurs among all experiments whose
outcome lies in B.
The conditional probability has two important properties.
Theorem 3.2. Let B ∈ F with P (B) > 0.

(i) The conditional probability P (· | B) defines a probability measure on (Ω, F), with

P (B | B) = 1.

(ii) Let (Bn )n∈N ⊂ F be pairwise disjoint with

[
Bn = Ω, P (Bn ) > 0 ∀n.
n∈N

Then, for every A ∈ F, X

P (A | Bn )P (Bn ) = P (A). (3.2)
n∈N

1
Proof. First we check that P (· | B) is a probability measure. Clearly

P (B | B) = 1, P (∅ | B) = 0.

Furthermore,
P (Ac ∩ B)
P (Ac | B) =
P (B)
P (B \ (A ∩ B))
=
P (B) (3.3)
P (B) − P (A ∩ B)
=
P (B)
= 1 − P (A | B).
Finally, if (An ) is a pairwise disjoint sequence, then
!
∩ B)
S
[ P (( n An )
P An | B =
n P (B)
(3.4)
X P (An ∩ B) X
= = P (An | B),
n P (B) n

so (i) follows.
For (ii), X X
P (A | Bn )P (Bn ) = P (A ∩ Bn )
n∈N n∈N
 
[ (3.5)
= P A ∩ Bn 
n∈N

= P (A ∩ Ω) = P (A).
□
If A ⊂ B, then
P (A)
P (A | B) = .
P (B)
Conditional probability can therefore be viewed as a probability measure on (B, F ∩ B), where

F ∩ B ≡ {A ∩ B : A ∈ F}. (3.6)

The definition of conditional probability naturally leads to the concept of stochastic independence.
Definition 3.3. Two events A, B ∈ F are called independent if and only if

P (A ∩ B) = P (A)P (B). (3.7)

If P (A) > 0 and P (B) > 0, this is equivalent to

P (A | B) = P (A), P (B | A) = P (B). (3.8)

More generally, a family of events (Ai )i∈I , I ⊆ N, is independent if for every finite family of indices

1 ≤ i1 < i2 < · · · < im ∈ I

we have
m m
!
\ Y
P Aik = P (Aik ). (3.9)
k=1 k=1

2
It is not enough to require that Ai and Aj be independent for every i ̸= j. In that case the events are
called pairwise independent. Pairwise independence is strictly weaker than mutual independence.
Under our definition, null events are independent of every event. This case is of course not
particularly interesting.
A trivial corollary of the definition of conditional probability is Bayes’ famous formula.
Theorem 3.4 (Bayes’ formula). Let A, B ∈ F with P (A) > 0 and P (B) > 0. Then

P (B)
P (B | A) = P (A | B) . (3.10)
P (A)

Proof. Immediate from (3.1). □

The formula is of great importance in statistics. Thomas Bayes (1702–1761) derived this relation in
a special case in his work Essay towards solving a problem in the doctrine of chances. The chapter
notes that the formula can produce results that are far from intuitive.
Example (the two-envelope problem). Two closed envelopes contain amounts X and 2X. A
player chooses one envelope and may open it. Afterwards, he may keep the money or switch to the
other envelope.
A naive argument says that one should always switch. If the observed amount is x, one informally
assigns probability 1/2 each to x/2 and 2x in the other envelope, giving expected value
x 5
+ x = x > x.
4 4
The argument is wrong because it assumes that, independently of the observed x, the two possibilities
are equally likely.
Now suppose X is a random variable and Z is an independent random variable taking values 1 and
2 with equal probability. The observed amount is then

XZ.

After observing XZ = x, the relevant quantity is

P (Z = 1 | ZX = x).

By Bayes’ formula,

P (XZ = x | Z = 1)P (Z = 1)
P (Z = 1 | ZX = x) = . (3.11)
P (ZX = x)

Since Z = 1 implies ZX = X and X and Z are independent,

P ({ZX = x} ∩ {Z = 1})
P (ZX = x | Z = 1) =
P (Z = 1)
(3.12)
P ({X = x} ∩ {Z = 1})
= = P (X = x).
P (Z = 1)

Moreover,
1 1
P (ZX = x) = P (2X = x) + P (X = x).
2 2
Therefore
1
P (Z = 1 | ZX = x) = P (X=x/2)
. (3.13)
1+ P (X=x)

3
Hence the conditional expected content of the other envelope, given that the first contains x, is
x
2x P (Z = 1 | ZX = x) + P (Z = 2 | XZ = x),
2
which can be written as
x 3x 1
+ P (X=x/2)
. (3.14)
2 2 1+
P (X=x)

Whether this is larger or smaller than x depends on

P (X = x/2)
.
P (X = x)
Under reasonable assumptions this ratio tends to infinity as x → ∞. In that limit the conditional
expectation tends to x/2, so switching at large values of x is disadvantageous.

2 Independent Random Variables

We have already defined independence for events. We now extend the concept to σ-algebras.
Definition 3.5. Let (Ω, F, P ) be a probability space and let E, G ⊂ F be sub-σ-algebras. We say
that E and G are independent if for every A ∈ E and B ∈ G, the events A and B are independent.
We now define independence of random variables. We first need the σ-algebra generated by a
random variable.
Definition 3.6. Let (Ω, F) be a measure space and let X : Ω → R be measurable. Let σ(X) be
the smallest sub-σ-algebra of F with respect to which X is measurable. We call σ(X) the σ-algebra
generated by X.
Lemma 3.7. Let X be a real-valued random variable on (Ω, F). Then

σ(X) = X −1 (B(R)),

the collection of all preimages of Borel sets.

Proof. For X to be measurable with respect to σ(X), every preimage of a Borel set must belong to
σ(X), so
X −1 (B(R)) ⊆ σ(X).
On the other hand, X −1 (B(R)) itself is a σ-algebra: X −1 (R) = Ω; complements are preserved,

(X −1 (B))c = X −1 (B c ), (3.15)

and countable unions are preserved,

!
−1 −1
[ [
X (Bn ) = X Bn . (3.16)
n n

Thus it is the smallest such σ-algebra, so

σ(X) = X −1 (B(R)).

□
Definition 3.8. Let (Ω, F, P ) be a probability space and let X1 , X2 be random variables. They
are called independent if and only if the σ-algebras σ(X1 ) and σ(X2 ) are independent.
Since σ(X) = X −1 (B), two random variables X1 , X2 are independent exactly when, for all Borel
sets B1 , B2 ,
P ({X1 ∈ B1 } ∩ {X2 ∈ B2 }) = P (X1 ∈ B1 )P (X2 ∈ B2 ). (3.17)

4
Lemma 3.9. Let (Ω, F, P ) be a probability space and let X1 , X2 be random variables. Then X1
and X2 are independent if and only if for all measurable functions g1 , g2 : R → R for which
Z
|gi (Xi )| dP < ∞,
Ω

we have Z Z  Z 
g1 (X1 )g2 (X2 ) dP = g1 (X1 ) dP g2 (X2 ) dP . (3.18)
Ω Ω Ω

Proof. First suppose X1 and X2 are independent. It suffices to establish (3.18) for indicator
functions. For Ai ∈ B(R), Z
1Ai (Xi ) dP = P (Xi ∈ Ai ), (3.19)
Ω
while Z
1A1 (X1 )1A2 (X2 ) dP = P ({X1 ∈ A1 } ∩ {X2 ∈ A2 })
Ω (3.20)
= P (X1 ∈ A1 )P (X2 ∈ A2 ).
Linearity gives the result for positive simple functions. Approximate general positive measurable
functions by increasing sequences of simple functions and use monotone convergence twice:
Z Z
gi (Xi ) dP = lim hn(i) (Xi ) dP, (3.21)
Ω n→∞ Ω

and Z Z
g1 (X1 )g2 (X2 ) dP = lim h(1) (2)
n (X1 )hn (X2 ) dP. (3.22)
Ω n→∞ Ω

For simple functions, independence gives

Z Z  Z 
h(1) (2)
n (X1 )hn (X2 ) dP = h(1)
n (X1 ) dP h(2)
n (X2 ) dP . (3.23)
Ω Ω Ω

Taking limits proves (3.18) for positive functions, and decomposition into positive and negative
parts gives the general integrable case.
Conversely, choose gi to be indicator functions. Then (3.18) immediately implies independence of
X1 and X2 . □
A related property that is much weaker than independence is uncorrelatedness.
Definition 3.10. Let (Ω, F, P ) be a probability space and let X1 , X2 be random variables. They
are called uncorrelated if
Z Z  Z 
X1 X2 dP = X1 dP X2 dP . (3.24)
Ω Ω Ω

Uncorrelatedness is much easier to check than independence and is therefore sometimes used as a
first indication of independence. However, it is not conclusive.
For example, let X, Y be independent, identically distributed random variables, and define
Z+ = X + Y, Z− = X − Y.
Then Z+ and Z− are uncorrelated, but in general they are not independent. For the Bernoulli
distribution with p = 1/2,
P (Z− = 0 | Z+ = 2) = 1, P (Z− = 0 | Z+ = 1) = 0, (3.25)
which immediately rules out independence.
There is exactly one class of distributions in which uncorrelatedness is equivalent to independence:
Gaussian distributions.
Definition 3.11. Let (Ω, F, P ) be a probability space.

5
 (i) A finite family X1 , . . . , Xn of random variables is called independent if for all Borel sets
B1 , . . . , Bn ,
n n
!
\ Y
P {Xk ∈ Bk } = P (Xk ∈ Bk ). (3.26)
k=1 k=1

(ii) A family (Xn )n∈N is called independent if every finite subfamily is independent.

3 Product Spaces

Independent random variables can be constructed explicitly. Consider two probability spaces

(Ω1 , F1 , P1 ), (Ω2 , F2 , P2 ),

and measurable functions

X1 : Ω1 → R, X2 : Ω2 → R.
The idea is to construct a probability space on Ω1 × Ω2 and extend X1 , X2 to this space so that
they become independent random variables.
Definition 3.12. The product σ-algebra F1 ⊗ F2 is the smallest σ-algebra on Ω1 × Ω2 containing
all sets of the form
C = A × B, A ∈ F1 , B ∈ F2 .
Sets of the form A × B are sometimes called rectangles. They form an intersection-stable generator
because
(A1 × B1 ) ∩ (A2 × B2 ) = (A1 ∩ A2 ) × (B1 ∩ B2 ).

For C ∈ F1 ⊗ F2 and x ∈ Ω1 , y ∈ Ω2 , define the sections

Cx ≡ {y ∈ Ω2 : (x, y) ∈ C}, (3.27)

and
Cy ≡ {x ∈ Ω1 : (x, y) ∈ C}. (3.28)
Similarly, for a measurable function X on Ω1 × Ω2 define

Xx (y) ≡ X(x, y), Xy (x) ≡ X(x, y).

Lemma 3.13. With the above definitions:

(i) For every C ∈ F1 ⊗ F2 and every x ∈ Ω1 , y ∈ Ω2 ,

Cx ∈ F2 , Cy ∈ F1 .

(ii) For every measurable function

X : Ω1 × Ω2 → R
and every x ∈ Ω1 , y ∈ Ω2 ,

Xx is F2 -measurable, Xy is F1 -measurable.

Proof. Fix x ∈ Ω1 and define

Cx ≡ {C ∈ F1 ⊗ F2 : Cx ∈ F2 }. (3.29)

6
Every rectangle A × B belongs to Cx , since Cx = B if x ∈ A and Cx = ∅ otherwise. One checks
directly that Cx is a σ-algebra. Since it contains the generator of F1 ⊗ F2 ,

Cx = F 1 ⊗ F 2 .

Thus (i) holds.

For measurable D ∈ B(R),
Xx−1 (D) = {y ∈ Ω2 : Xx (y) ∈ D}
= {y ∈ Ω2 : X(x, y) ∈ D} (3.30)
= (X −1 (D))x ,
which belongs to F2 by (i). Hence (ii). □
Theorem 3.14. Let P1 , P2 be probability measures on (Ω1 , F1 ) and (Ω2 , F2 ).

(i) There exists a unique probability measure

P ≡ P1 ⊗ P2

called the product measure, on F1 ⊗ F2 such that

P (A × B) = P1 (A)P2 (B) (3.31)

for all A ∈ F1 , B ∈ F2 .

(ii) For C ∈ F1 ⊗ F2 , Z Z
P (C) = P2 (Cx ) dP1 (x) = P1 (Cy ) dP2 (y). (3.32)
Ω1 Ω2

Proof. Uniqueness follows because the rectangles form an intersection-stable generator of F1 ⊗ F2 .

For existence define Z
P (C) ≡ P2 (Cx ) dP1 (x). (3.33)
Ω1

We must first check measurability of x 7→ P2 (Cx ). Let

G ≡ {C ∈ F1 ⊗ F2 : P2 (Cx ) is F1 -measurable}. (3.34)

For a rectangle A × B,
P2 ((A × B)x ) = 1A (x)P2 (B),
which is measurable. One checks that G is a Dynkin system: Ω1 × Ω2 ∈ G; differences are preserved
because
(D \ C)x = Dx \ Cx ;
and disjoint countable unions are preserved using
!
[ [
Ci = (Ci )x , (3.35)
i x i

together with σ-additivity, ! !

[ X
P2 Ci = P2 ((Ci )x ). (3.36)
i x i

Since G contains the intersection-stable generator, Dynkin’s theorem gives

G = F1 ⊗ F 2 .

7
Thus (3.33) is well-defined.
For a rectangle, Z
P (A × B) = P2 (B) 1A (x) P1 (dx)
Ω1 (3.37)
= P1 (A)P2 (B).
Finally, for a pairwise disjoint family (Ci ),
! Z ! !
[ [
P Ci = P2 Ci P1 (dx)
i Ω1 i x
XZ
= P2 ((Ci )x ) P1 (dx) (3.38)
i Ω1
X
= P (Ci ),
i

so P is σ-additive, and P (Ω1 × Ω2 ) = 1. Thus P is the desired probability measure. The second
formula in (3.32) is obtained analogously; equality follows from uniqueness. □
If Xi are random variables on (Ωi , Fi ), define
e 1 (x, y) ≡ X1 (x),
X e 2 (x, y) ≡ X2 (y).
X

Then X
e 1 and X
e 2 are independent random variables on

(Ω1 × Ω2 , F1 ⊗ F2 , P1 ⊗ P2 ).

This is the canonical construction of independent random variables. Iteration extends the construc-
tion to arbitrary finite products.
Example. Consider tossing n coins. The state space of each coin is

Ωi = {0, 1}.

The state space of the n tosses is

Ω1 × · · · × Ωn = {0, 1}n .

Each individual coin has a Bernoulli distribution with parameter p. The coordinate random
variables
Xi (ω1 , . . . , ωn ) = ωi
are independent and identically distributed under the n-fold product measure.
Example. Let Ω = R. Then Rn is a product space with the product Borel σ-algebra. The Gaussian
measure with density
n
!
1 X x2i
exp − (3.39)
(2π)n/2 ni=1 σi 2
Q
i=1
2σ i

is a product measure. The coordinates of the vector

X = (x1 , . . . , xn )

are then independent random variables.

Independent random variables are a fundamental building block of probability theory. In everyday
language, independence is often informally identified with randomness; for example, one assumes
that successive roulette outcomes are independent and regards this as part of the random character
of the game.

8
Example (Winning with conditional probability). Alice writes two numbers on two pieces of
paper. She tosses a fair coin and, depending on the outcome, shows Bob one of the two numbers.
Let the shown number be y and the hidden number be x. Bob must guess whether x > y or x < y.
Alice offers him a bet at odds 1 : 2. Should Bob accept?
Yes. Bob can answer correctly with probability strictly greater than 1/2.
A strategy is the following. Bob draws a random number

Z ∼ N (0, 100).

He compares Z with the observed y: if Z ≥ y, he guesses y < x; if Z < y, he guesses x < y.

To formalize this, let x0 < x1 and let B be a Bernoulli random variable with parameter 1/2 on
(Ω1 , F1 , P1 ). Bob only observes
Y = xB .
His task is to estimate B. Introduce a second probability space (Ω2 , F2 , P2 ) and construct on it a
Gaussian random variable Z. On

(Ω1 × Ω2 , F1 ⊗ F2 , P1 ⊗ P2 )

the variables B and Z are independent.

Define
A ≡ 1{Z<Y } . (3.41)
Then A depends only on the observed Z and Y , but
1
P (A = B) > . (3.40)
2

Indeed,
P (A = B) = P ({Z < Y } ∩ {B = 1}) + P ({Z ≥ Y } ∩ {B = 0})
1
= (P2 (Z < x1 ) + P2 (Z ≥ x0 ))
2
1 1 1
= + P2 (x0 ≤ Z < x1 ) > .
2 2 2

4 Fubini’s Theorem

One of the most important tools for computing complicated integrals on product spaces is the
interchange of the order of integration. The conditions allowing this are supplied by a theorem
named after Guido Fubini (1879–1943).
The first result concerns positive functions. Surprisingly, no additional integrability assumptions
are needed.
Theorem 3.15 (Fubini–Tonelli). Let (Ω1 , F1 , P1 ) and (Ω2 , F2 , P2 ) be probability spaces, and let

f : (Ω1 × Ω2 , F1 ⊗ F2 ) → R

be positive and measurable. Define

Z Z
h(x) ≡ f (x, y) dP2 (y), g(y) ≡ f (x, y) dP1 (x). (3.42)
Ω2 Ω1

Then h is F1 -measurable, g is F2 -measurable, and

Z Z Z
f d(P1 ⊗ P2 ) = h dP1 = g dP2 . (3.43)
Ω1 ×Ω2 Ω1 Ω2

9
Proof. For an indicator function f = 1C , the measurability of the sections follows from Theorem
3.14:
h(x) = P2 (Cx ), (3.44)
g(y) = P1 (Cy ). (3.45)
Linearity gives the result for simple functions. Every positive measurable function is a monotone
limit of simple functions, giving the general case. The integral identity is immediate for indicators
from Theorem 3.14, then follows for simple functions by linearity and for general positive measurable
functions by monotone convergence. □
Theorem 3.16 (Fubini–Lebesgue). Let

f : (Ω1 × Ω2 , F1 ⊗ F2 ) → (R, B(R))

be measurable and absolutely integrable with respect to P1 ⊗ P2 . Then:

(i) f (x, y) is absolutely integrable with respect to P2 for P1 -almost every x, and vice versa.

(ii) The functions

Z Z
h(x) ≡ f (x, y) dP2 (y), g(y) ≡ f (x, y) dP1 (x) (3.46)
Ω2 Ω1

are well-defined except possibly on null sets and are absolutely integrable with respect to P1
and P2 , respectively.

(iii) Z Z Z
f d(P1 ⊗ P2 ) = h(x) dP1 (x) = g(y) dP2 (y). (3.47)
Ω1 ×Ω2 Ω1 Ω2

Proof. Apply Theorem 3.15 to |f |:

Z Z  Z
|f (x, y)| dP2 (y) dP1 (x) = |f | d(P1 ⊗ P2 ) < ∞. (3.48)
Ω1 Ω2 Ω1 ×Ω2

Thus the inner integral can be infinite only on a P1 -null set. Write

f = f + − f −.

Then Z
±
h (x) = f ± (x, y) dP2 (y), (3.49)
Ω2
Z
g ± (y) = f ± (x, y) dP1 (x), (3.50)
Ω1
are finite and measurable outside null sets. Defining the functions arbitrarily, say as 0, on those
null sets does not affect the integral. Moreover,
Z Z Z 
|h(x)| dP1 (x) ≤ |f (x, y)| dP2 (y) dP1 (x) < ∞. (3.51)
Ω1 Ω1 Ω2

Finally use
Z Z Z
f d(P1 ⊗ P2 ) = +
f d(P1 ⊗ P2 ) − f − d(P1 ⊗ P2 ), (3.52)
Ω1 ×Ω2 Ω1 ×Ω2 Ω1 ×Ω2

and apply Fubini–Tonelli to the two nonnegative parts. □

The fact that P1 , P2 are probability measures is not essential; the theorems also hold for general
σ-finite measures.

10
The absolute integrability assumption is essential in general. Without it, changing the order of
integration can lead to a false conclusion.
Exercise. Show that the hypotheses of Fubini’s theorem are not satisfied for

f (x, y) = 2e−2xy − e−xy

on (0, ∞) × (0, 1) with respect to Lebesgue measure.

5 Infinite Products

Ultimately we would like to speak about the distribution of arbitrarily, hence infinitely, many
random experiments, such as infinitely many coin tosses. Is this really difficult?
Take finite spaces Ωi , for example Ωi = {0, 1}. What should be the appropriate σ-algebra on the
infinite product
∞
Y
Ωi ?
i=1
For finite products one might first think of using the full power set. But note that

{0, 1}N

is isomorphic to the interval [0, 1] via

∞
ωi 2−i .
X
ω = (ω1 , ω2 , . . .) 7−→
i=1

In particular, the infinite product is uncountable. Constructing a probability space on it with the
full power set would implicitly attempt to do the same for R, running into the familiar difficulties.
We therefore need a smaller σ-algebra, analogous to the Borel σ-algebra on R.
A practical point of view is useful. Even if we perform infinitely many coin tosses, we are initially
interested only in the first n tosses for finite n. Thus the σ-algebra should certainly contain all sets
describing finitely many coordinates, and should contain nothing beyond what is forced by closure.
Definition 3.17. Let (Ωi , Fi ), i ∈ N, be measure spaces and let
∞
Y
e ≡
Ω Ωi
i=1

be the infinite product space. The product σ-algebra Fe on Ωe is the smallest σ-algebra containing
all subsets of the form Y Y
A= Ai × Ωj , (3.53)
i∈I j ∈I
/

where Ai ∈ Fi and I = (i1 , . . . , ik ) ⊂ N is finite. Such sets are called cylinder sets.
Theorem 3.18. Let (Ωi , Fi , Pi ), i ∈ N, be probability spaces. Then there exists a unique probability
measure O
Pe ≡ Pi
i
e F)
called the product measure, on (Ω, e such that for every cylinder set of the form (3.53),
Y
Pe (A) = Pi (Ai ). (3.54)
i∈I

The existence proof is not given here. Uniqueness follows from Theorem 2.15 because cylinder sets
form an intersection-stable generating class.

11
The product σ-algebra contains a very rich class of sets, but is still smaller than the full power set,
even when Ωi is finite. In nature it is analogous to the Borel σ-algebra.
Theorem 3.19. Let Ωi , i ∈ N, be metric spaces (for example Ωi = R), and let B(Ωi ) be the
corresponding Borel σ-algebras. Then the infinite product space
Y
Ω
e = Ωi
i

can be equipped with a metric such that the product σ-algebra is the Borel σ-algebra associated
with that metric, i.e. it is generated by the open sets of the product topology.
In other words, the product σ-algebra contains all open sets, and therefore all closed sets, of the
product topology. For our purposes: do not be afraid of infinite product spaces; they are no worse
than the real numbers.
Exercise. Use the isomorphism
∞
ωi 2−i ,
X
I : {0, 1}N → [0, 1], I(ω) =
i=1

and the example of a non-Borel set from Chapter 2 to construct a set in {0, 1}N that is not contained
in the product σ-algebra.
The construction of infinite product spaces now allows us to construct infinite sequences of random
variables.
Definition 3.20. Let (Ω, F, P ) be a probability space. A sequence

X = (Xn )n∈N

of measurable maps Ω → R is called a random sequence, or a stochastic process with discrete time.
A random sequence is called a sequence of independent random variables if the family (Xn )n∈N is
independent.
Theorem 3.18 shows that countably many independent random variables can be constructed on a
suitable probability space. Take the product space
e F,
(Ω, e Pe )

and measurable maps Xn : Ωn → R. Define

e n (ω) ≡ Xn (ωn ),
X n ∈ N.

These random variables are independent and have the same distributions as the original Xn .
One may also define a stochastic process as a measurable map

X : (Ω, F) → (RN , B(RN )).

It is then a sequence of independent random variables precisely when the pushforward measure

PX ≡ P ◦ X −1

is a product measure on (RN , B(RN )). If the component distributions are also identical, the sequence
is called a sequence of independent identically distributed random variables (i.i.d.).
The definition of a family of independent real-valued random variables can equivalently be written
as follows. The sequence
X = (Xi , i ∈ N)

12
is independent if and only if for every n ∈ N and arbitrary measurable functions

hi : R → R,
" n # n
Y Y
E hi (Xi ) = E[hi (Xi )]. (3.55)
i=1 i=1

Exercise. Prove the statement in the preceding remark.

Infinite sequences of independent random variables are among the most important building blocks
of probability theory. In particular, they allow us to model repeated identical random experiments,
such as repeated coin tosses and roulette games.

13