Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Example: Balancing Vectors . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Discrete Probability Spaces and Distributions . . . . . . . . . 11
1.3 Poisson Approximation to the Binomial . . . . . . . . . . . . . . 29
1.4 Independence, Conditional Distributions . . . . . . . . . . . . . 38
1.5 Applications of the Linearity of Expectation . . . . . . . . . . 56

2 Second Moment Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . 71

2.1 Variance and Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.2 Classical Law of Large Numbers . . . . . . . . . . . . . . . . . . . . 78
2.3 Bernstein Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.4 Cliques in the Erdős–Rényi Random Graph . . . . . . . . . . . 86
2.5 Hardy–Ramanujan Theorem . . . . . . . . . . . . . . . . . . . . . . . 95

3 Exponential Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

3.1 Hoeffding Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.2 Johnson–Lindenstrauss Lemma . . . . . . . . . . . . . . . . . . . . . 112
3.3 Hoeffding–Chernoff Inequality . . . . . . . . . . . . . . . . . . . . . 118
3.4 Azuma Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
3.5 Applications of Azuma Inequality . . . . . . . . . . . . . . . . . . . 132

4 Gaussian Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

4.1 Gaussian Distributions on R . . . . . . . . . . . . . . . . . . . . . . . 143
4.2 Gaussian Distributions on Rn . . . . . . . . . . . . . . . . . . . . . . . 154
4.3 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
4.4 Distributions Related to Gaussian . . . . . . . . . . . . . . . . . . . 179
4.5 Simple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 188

5 Finite State Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

5.1 Definitions and Basic Properties . . . . . . . . . . . . . . . . . . . . 201
5.2 Stationary Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
5.3 Convergence Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
5.4 Reversible Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . 229

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
 PREFACE

This book is intended for students who have completed proof-based

courses in Advanced Calculus and Linear Algebra. In addition to the
standard topics (such as the Poisson approximation to the binomial, law
of large numbers, central limit theorem, Markov chains, and simple linear
regression), several other topics and results that are accessible at this level
and that fit into a one semester course are covered:
• first moment method with some applications, such as cliques in the
Erdős–Rényi random graph, and an upper bound on the typical longest
increasing subsequence of a random permutation;
• second moment method with applications to Bernstein’s polynomials,
cliques in the Erdős–Rényi random graph, and the Hardy–Ramanujan
theorem;
• Hoeffding’s inequality, and the Johnson–Lindenstrauss lemma;
• the Hoeffding–Chernoff inequality, and the generalization ability of
classification algorithms;
• Azuma’s inequality with several examples, such as the chromatic
number of the Erdős–Rényi random graph, max-cut in sparse random
graphs, and the Hamming distance on the hypercube.
A knowledge of Lebesgue integration is not assumed, although, when
discussing continuous distributions, I tried to give some idea of why
learning about it is something to look forward to. A number of exercises
are included throughout and at the end of each section.
In the second edition, a number of explanations were clarified and
some mistakes were corrected.

Dmitry Panchenko,
Toronto, Canada
Chapter 1
Introduction

1.1 Example: Balancing Vectors

We will begin this chapter with an example that on the

surface appears unrelated to Probability. We will then go
through various steps of this example and reformulate them
using probabilistic terminology and notation, introducing
such notions as the probability space, probability measure,
random variable, expectation, independence, sample space,
change of measure, distribution. The general definitions will
appear in the following sections, but an illustration of these
concepts on a simple example will, hopefully, make things
clearer later on.
Example 1.1.1 (Balancing vectors). Let us consider n 2
vectors on the unit sphere in Rn ,

v1 , . . . , vn 2 Rn , |vi | = 1 for all i = 1, . . . , n.

We denote the length of a vector v 2 Rn by |v|. Consider the

following question. Among all linear combinations of these
vectors v = ±v1 ± . . . ± vn with ±1 coefficients, can we find
a choice of signs such that
p
|v|  n?

1
2 1 Introduction
p
Can we always choose the signs in such a way that |v| pn?
Notice that if vectors vi are orthogonal then, clearly, |v| = n
for all choices of signs, but here we do not assume that they
are orthogonal. We will now show that the answer to both
questions is yes.
Let us consider n variables e1 , . . . , en each taking two pos-
sible values ±1, which will represent possible choices of
signs above. Then the question can be rephrased as follows.
Can we find
n
e = (e1 , . . . , en ) 2 1, 1 (1.1)

such that the vector v(e) := e1 v1 + . . . + en vn has length

p
|v(e)|  n?

We will answer this question by computing the average of

|v(e)|2 over all 2n choices of signs:
1 1
n Â
2 e2{ 1,1}n
|v(e)| 2
= n Â
2 e2{ 1,1}n
|e1 v1 +. . .+en vn |2 . (1.2)

First of all, if we rewrite in terms of scalar products,

n
2
|v(e)| = |e1 v1 + . . . + en vn | = 2
 ei e j (vi , v j ),
i, j=1

then the average can be written as

n
1
2n e2{Â1,1}n i,Â
ei e j (vi , v j )
j=1
n h1 i
=  (vi , v j ) n  ei e j , (1.3)
i, j=1 2 e2{ 1,1}n

where we simply interchanged the order of summation. If

i = j then ei e j = 1 and
1.1 Example: Balancing Vectors 3

1 1 2n
2n e2{Â1,1}n
ei e j = n  1 = n = 1.
2 e2{ 1,1}n 2

For i 6= j, notice that for each choice of ei , e j 2 { 1, +1}

there are 2n 2 choices of the remaining n 2 coordinates,
which do not affect the value of ei e j . Therefore,

1 2n 2 1
Â
2n e2{ 1,1}n
ei e j = Â
2n e ,e 2{ 1,1}
e i e j = Â eie j
22 e ,e 2{
i j i j 1,1}
⇣1 ⌘ ⇣1 ⌘
2 e 2{Â1,1} 2 e 2{Â1,1}
= ei ⇥ e j = 0 ⇥ 0 = 0. (1.4)
i j

This means that the average in (1.3) is

n
1
2n e2{Â1,1}n  (vi, vi) = n,
2
|v(e)| =
i=1

because we assumed that all |vi | = 1. Since the average is

n, all |v(e)|2 can not be strictly bigger than n, because the
average would be strictly bigger than [Link] shows that there
exists e such that |v(e)|2  n. By the same logic, there exists
a choice of signs e such that |v(e)|2 n. t
u
Let us introduce several key concepts using the setting of
the above example. In the probabilistic language, the set
n
W := 1, 1 (1.5)

that appeared in (1.1) consisting of vectors e = (e1 , . . . , en ) is

called a probability space, and elements of W are also called
outcomes. To be more precise, in order for W to be called a
probability space, it must have a probability, or probability
measure, associated with it. In the above example, the mea-
sure implicitly appeared when we talked about averaging.
Namely, if we let
1
P(e) = n (1.6)
2
 4 1 Introduction

represent the probability of an individual outcome e and, for

any subset A ✓ W, let
card(A)
P(A) = (1.7)
2n
represent the probability of A, then P is an example of
a probability measure. In this particular case, it is called
a uniform measure on { 1, 1}n , because all outcomes are
‘equally likely’. We can think of P as a function describing
the chances of individual outcomes e1 , . . . , en if we choose
them ‘at random’ in the everyday sense of the word, say, by
flipping a coin.
Informally, probability (measure) is a function that assigns
to subsets of W values in [0, 1] and satisfies certain basic
properties. The general definition is more involved (we will
say more in Chapter 4), but for now let us mention several
properties that are obvious in the above setting, where W is a
finite set:
(i) P(A) 2 [0, 1] for any set A ✓ W.
(ii) P(W) = 1.
(iii) for any disjoint sets A1 , A2 ✓ W,

P(A1 [ A2 ) = P(A1 ) + P(A2 ).

The last property is called additivity, or finite additivity, of

probability, because it extends by induction to any finite
number of disjoint sets A1 , . . . , An ✓ W,

P(A1 [ . . . [ An ) = P(A1 ) + . . . + P(An ). (1.8)

The second property, P(W) = 1, is important because 12 = 1,

which allows us to take products of probability spaces, as
well as leading to natural analogues of the Fubini theorem
via the notion of conditional distributions, as we will see in
Section 1.4 below.
1.1 Example: Balancing Vectors 5

Exercise 1.1.1. Show that the above properties (i), (ii), (iii)
imply:
1. P(Ac ) = 1 P(A), where Ac = W \ A is the complement of
A in W.
2. P(A [ B) = P(A) + P(B) P(A \ B) for any sets A, B ✓ W.
3. P(A [ B)  P(A) + P(B) for any sets A, B ✓ W.
4. P(A1 [ . . . [ An )  Âni=1 P(Ai ) for any sets A1 , . . . , An ✓ W.
The last inequality is called the union bound, and it will be
used often.
Next, in the probabilistic language, the function

|v(e)| = |e1 v1 + . . . + en vn |

in the above example is called a random variable. In fact,

any function
f: W!R (1.9)
is called a random variable, so a ‘random variable’ is just
another name for a ‘function’ in the probabilistic language.
(Again, it is not quite correct that all functions are random
variables, but we will discuss this more in Chapter 4 when
we talk about continuous distributions). If we think about W
as the set of possible outcomes e of some random experiment
(for example, flipping n coins) then the function f (e) can be
thought of as some numerical attribute of this random out-
come, which, hopefully, explains somewhat why functions
are called random variables.
The average that appeared in (1.2) is called the expectation
(or sometimes called mathematical expectation, or expected
value) of the random variable f (e) = |v(e)|2 and is denoted
by E f . More generally, given f : W ! R,

Ef = Â f (e)P(e), (1.10)
e2W

which is well defined because W is finite. Despite the fact that

the expected value in the above example is just the average
6 1 Introduction

with weights P(e) = 1/2n , it is also common to write it using

integral notation,
Z
Ef = f (e) dP(e), (1.11)
W

to emphasize that its meaning is not much different than the

familiar integrals in Calculus. For example, the interchange
of summation in (1.3) is the first example of the linearity of
expectation.
Exercise 1.1.2. Show that the definition (1.10) implies the
following.
1. E(a f1 + b f2 ) = aE f1 + bE f2 for any f1 , f2 : W ! R and
any a, b 2 R.
2. If f 0 then E f 0. If f1 f2 then E f1 E f2 .
3. Given a subset A ✓ W, consider an indicator function of A,
⇢
1, if e 2 A
I(e 2 A) =
0, if e 62 A.
Then,
E I(e 2 A) = P(A). (1.12)
Next, let us mention that the computation in the equation
(1.4) was an illustration of another fundamental concept in
Probability – independence. First of all, notice that e1 , . . . ,
en can be viewed as random variables on W, because ei is a
function of e. To emphasize this and to avoid confusion, let
us give these functions a different name,

pi (e) = ei . (1.13)

For any specific choice of signs a1 , . . . , an 2 { 1, +1}, let us

consider an event
n o
A = e 2 W : p1 (e) = a1 , . . . , pn (e) = an .
1.1 Example: Balancing Vectors 7

Typically, a more concise notation will be used for sets like

this, n o
A = p1 = a1 , . . . , pn = an .
In other words, we will often omit an explicit reference
to the variable e 2 W when dealing with random variables
(i.e. functions) on W. The above set A consists of one point
(a1 , . . . , an ), so it might look a bit strange to use such a con-
voluted way to describe it. However, in this language we can
say what it means for the random variables p1 , . . . , pn to be
independent. It means that
n
P p1 = a1 , . . . , pn = an = ’ P(pi = ai ) (1.14)
i=1

for any a1 , . . . , an . We will discuss the meaning of this defi-

nition in more detail later in this chapter, but for now let us
see why it holds in the balancing vectors example. By (1.6),
the left hand side is 1/2n . What is P(pi = ai )? The set

{pi = ai } = {e : pi (e) = ai }

consists of all the vectors e with the ith coordinate fixed,

equal to ai . There are 2n 1 such vectors so, according to our
definition (1.7), the probability of this set is
2n 1 1
P(pi = ai ) = = ,
2n 2
which implies that (1.14) is satisfied.
Exercise 1.1.3. Show that (1.14) implies that

P(p1 = a1 , p2 = a2 ) = P(p1 = a1 )P(p2 = a2 ).

Let us now show that the computation in the equation (1.4)

can be rewritten as

Epi p j = Epi Ep j = 0 · 0 = 0, (1.15)

8 1 Introduction

where the first equality is an important consequence of inde-

pendence. By definition,

Epi p j = Â pi(e)p j (e)P(e).

e2W

To make connection with (1.14), we will need to rewrite this

sum in a different way (this is a very important idea of the
change of variables). Let us first consider all possible values
(ai , a j ) that (pi (e), p j (e)) can take, then for each of these
values consider the set
n o
A(ai , a j ) = e 2 W : pi (e) = ai , p j (e) = a j ,

and partition the space W into a disjoint union

[
W= A(ai , a j ).
ai ,a j

Then the sum over e can be written as sum over each set
A(ai , a j ) and then the sum over these sets. In this particular
case, pi (e) = ei takes only two values ±1, so

Epi p j = Â pi(e)p j (e)P(e)

e2W
=  ai a j  P(e)
ai ,a j =±1 e2A(ai ,a j )
= Â ai a j P(A(ai , a j ))
ai ,a j =±1
= Â ai a j P(pi = ai , p j = a j ).
ai ,a j =±1

By independence, we can continue to write

Epi p j = Â ai a j P(pi = ai )P(p j = a j )

ai ,a j =±1
= Â ai P(pi = ai ) Â a j P(p j = a j ).
ai =±1 a j =±1
1.1 Example: Balancing Vectors 9

It remains to check that the factors on the right hand side are
equal to Epi and Ep j . By a similar calculation,

Epi =  pi(e)P(e) =  ai  P(e)

e2W ai =±1 e:pi (e)=ai
= Â ai P(pi = ai ), (1.16)
ai =±1

as desired, so we checked the first equality in (1.15) from the

definitions. For the second equality, as before, since there
are 2n 1 different e such that pi (e) = ei = ai , the probability
P(pi = ai ) = 1/2 and
1 1
Epi = (+1) + ( 1) = 0.
2 2
Finally, let us mention some terminology associated with
the last few equations, namely, the sample space and change
of variables. For example, when we looked at the random
variable pi (e) = ei just now, we looked at all possible values
ai it could take (the range of pi ) and calculated the proba-
bilities P(pi = ai ) for all such values. In this case, the set of
values was { 1, 1} and the probabilities were both 1/2. One
can think of the pair,
1 1
W0 = { 1, 1}, P0 ( 1) = and P0 (1) = , (1.17)
2 2
and a new probability space, and it is called the sample space
of the random variable pi . Probability measure P0 is called
the distribution (or sometimes law) of the random variable
pi . In other words, the sample space consists of all possible
values of the random variable and their probabilities.
The process by which the probabilities P0 were computed
is called the change of variables, because P0 (ai ) is just the
probability P on the original space W of the set

pi = ai = e : pi (e) = ai ,
10 1 Introduction

which can be also viewed as the pre-image pi 1 (ai ) of ai . For

this reason, we can write
⇣ ⌘
P (ai ) = P(pi = ai ) = P pi (ai ) =: P pi 1 (ai ). (1.18)
0 1

The distribution P0 = P pi 1 on the sample space is called

the image measure of P by the map pi . Notice that (1.16) can
be rewritten as
Epi = Â ai P0 (ai ), (1.19)
ai =±1
in terms of the distribution of pi . This average with the
weights given by P0 can be taken as another definition of
the expectation of the random variable, and (1.19) is called
the change of variables formula for the expectation.
As a final comment, let us mention that independence is an
analogue of the fact that the area of a rectangle is the product
of the lengths of its sides,

Area([a, b] ⇥ [c, d]) = Length([a, b]) ⇥ Length([c, d]).

The consequence of this in Calculus is that

ZZ Z 1 Z 1
f (x)g(y) dxdy = f (x) dx g(y) dy.
[0,1]2 0 0

Similarly, Epi p j = Epi Ep j above was the consequence of

independence.
Exercise 1.1.4. What is the sample space and distribution of
the random variable f (e) = e1 + e2 + e3 in the above exam-
ple? Calculate its expectation using the linearity of expecta-
tion, and using the distribution.
Exercise 1.1.5. Suppose we have 50 vectors of length 3 and
50 vectors of length 5 in R200 . If we do not know anything
about these vectors except their lengths, can we find x such
that there exist linear combinations v of these vectors with
±1 coefficients such that |v|  x and such that |v| x?
1.2 Discrete Probability Spaces and Distributions 11

1.2 Discrete Probability Spaces and Distributions

In this section, we will discuss discrete probability spaces

and distributions. Discrete spaces W consist of finitely many
or countably many elements,

W = w1 , w2 , . . . , wn , . . . , (1.20)

so they can be indexed by natural numbers. A probability

measure on such a space is defined by assigning probabilities
to individual outcomes,

P(wn ) = pn , (1.21)

which satisfy the conditions

pn 0, and  pn = 1. (1.22)
n 1

For a subset A ✓ W, its probability is

P(A) = Â P(w). (1.23)

w2A

The order of summation here does not matter, because the

probabilities are nonnegative.
Exercise 1.2.1. If the series Ân 1 an is absolutely convergent
then, for any bijection p : N ! N, the series Ân 1 ap(n) gives
the same answer. Hint: See the proof of Lemma 1.2 below.
For example, we can always sum in the order w’s were
enumerated in (1.20) and, if wn is not in A, replace the term
P(wn ) by zero in the summation,
•
P(A) = Â P(wn) I(wn 2 A). (1.24)
n=1

This implies all the properties in Exercise 1.1.1 (check!) or,

for example, the monotonicity property of probability:
12 1 Introduction

if A ✓ B then P(A)  P(B). (1.25)

In the last section, we denoted a generic random variable

by f to emphasize that it is nothing but a real-valued function
on our space. It is more common, however, to denote generic
random variables by capital letters, for example, X,Y or Z.
Of course, this is not required and in particular situations
more suitable or evocative notation may be used (such as
p, g, z, h, h, e etc.).
If the set W is finite then all the definitions in the previous
section, as well as the basic properties of probability and ex-
pectation, stay the same. If the set W is countably infinite, the
definition of expectation in (1.10),

EX = Â X(w)P(w), (1.26)
w2W

is formally the same, but it is used only for random variables

X that satisfy

E|X| = Â |X(w)| P(w) < •. (1.27)

w2W

Otherwise, EX is assumed undefined. Random variables that

satisfy (1.27) are called integrable. One reason we have to
assume the absolute convergence is to make sure that the
definition of expectation does not depend on the order of
summation, because the elements of W are not necessarily
ordered. This follows from Exercise 1.2.1.
Since the order of summation does not matter, (1.26)
makes sense and we can use basic properties of the series
to derive basic properties of expectation, for example, the
linearity and monotonicity of expectation.
Exercise 1.2.2. If E|X1 | < • and E|X2 | < • then, for any
a, b 2 R,
E(aX1 + bX2 ) = a EX1 + b EX2 . (1.28)
1.2 Discrete Probability Spaces and Distributions 13

Exercise 1.2.3. If E|X| < • and E|Y | < •, and X  Y, then

EX  EY.
The entire Section 1.5 will be devoted to applications of
the linearity of expectation. Notice that the representation
(1.24) is a way of writing P(A) = E I(w 2 A), as in (1.12)
in the previous section.
Given a random variable X : W ! R, since W is countably
infinite, the set of values {a1 , a2 , . . .} that X can take (range
of X) is either finite or countably infinite. In particular, as in
(1.18), we can define the image measure on this set by
1
P0 (an ) = P w : X(w) = an = P X (an ) . (1.29)

Standard notation for this P0 is P X 1 , and it is called the

distribution of X.
If a probability space W ✓ R is a subset of the real line then
the probability P can also be called a distribution, because it
is the distribution of the identity function X(w) = w,

P0 (an ) = P w : X(w) = w = an = P(an ).

We will also often use the word distribution for P(X = an )

when X is a vector consisting of several random variables. As
in the equation (1.19), we can use the change of variables to
rewrite the expectation of X in terms of its distribution.
Lemma 1.1 (Change of variables). If E|X| < • then EX in
(1.26) can be rewritten as

EX = Â anP(X = an) = Â anP0(an). (1.30)

n 1 n 1

Proof. To show this, let us enumerate the points in the set

X 1 (an ) in an arbitrary order,
1
X (an ) = w : X(w) = an = wnm : 1  m  Mn .
14 1 Introduction

Some (or all) of these sets could be infinite, in which case

Mn = •. Let us rewrite, using (1.23) and that an = X(wnm )
for all m 1,
• • Mn
 anP(X = an) =  an  P(wnm)
n=1 n=1 m=1
• Mn
= Â Â anP(wnm)
n=1 m=1
• Mn
= Â Â X(wmn)P(wnm).
n=1 m=1

This looks very similar to the definition of the expectation

(1.26), except that now the elements of W are enumerated
by the double index (n, m) and we have the double sum. To
finish the proof, it remains to apply the following lemma,
which will be useful to us in other ways. t
u
Lemma 1.2. For any bijection p : N ⇥ N ! N,
• • •
 cn =   cp(n,m)
n=1 n=1 m=1

if either all cn 0 or the series on either side is absolutely

convergent.
Here, for simplicity, we take the second sum over 1  m < •,
but the same argument will work when 1  m  Mn .
Proof. Let us consider the case cn 0 first. In one direction,
for any K 1, there exists N 1 such that
K N N • •
 ck   cp(n,m)    cp(n,m),
k=1 n=1 m=1 n=1 m=1

because p is a bijection and 1, . . . , K must appear somewhere

on the list of p(n, m). Letting K ! • proves that the left
hand side is less than or equal to the right hand side. In the
1.2 Discrete Probability Spaces and Distributions 15

opposite direction, for any N, M 1 there exists K 1 such

that
N M K •
  cp(n,m)   ck   ck ,
n=1 m=1 k=1 k=1
for example, by taking K to be the largest among p(n, m) for
n  N, m  M. Letting M ! • first and then letting N ! •
proves the inequality in the other direction.
In the second case, suppose, for example, that • n=1 cn is
absolutely convergent. Let us write each cn as the difference
cn = an bn where an = |cn | I(cn 0) and bn = |cn | I(cn  0).
By the first part of the proof, we know that
• • • • • •
 an =  ap(n,m) ,  bn =   bp(n,m).
n=1 n=1 m=1 n=1 n=1 m=1

Since • •
n=1 (an +bn ) = Ân=1 |cn | < •, all the series above are
convergent and, subtracting the two equations, we obtain the
same equality for (cn ). t
u
The proof also shows that if cn 0 then one side is +•
only if the other side is +•.
In addition to showing that we can compute EX in terms
of the distribution of X as in (1.30), this lemma immediately
implies that, if the set W is countably infinite, we can extend
finite additivity property (1.8) to countable additivity.
S
Exercise 1.2.4. If W is countably infinite and A = n 1 An
for disjoint An ✓ W then

P(A) = Â P(An). (1.31)

n 1

Moreover, if E|X| < • then

⇥ ⇤ ⇥ ⇤
E X I(w 2 A) = Â E X I(w 2 An ) . (1.32)
n 1

One simple consequence of countable additivity is that the

tail probability P(X t) goes to zero as t goes to infinity,
16 1 Introduction

lim P(X t) = 0. (1.33)

t!+•

To see this, for integer n 0, we can write the set {X n}

as a disjoint union
[
{X n} = {m  X < m + 1},
m n

and, using countable additivity, we can write

•
P(X n) = Â P(m  X < m + 1).
m=n

This means that P(X n) is the tail of convergent series, so

it must go to zero. For non-integer t, by the monotonicity
property of probability, P(X t)  P(X btc), so the tail
probability must go to zero.
It will sometimes be convenient to rewrite the expectation
EX of a nonnegative random variable X in terms of its tail
probability P(X t). If X takes integer values k 0 and if
we write k = Âkm=1 1 then
• • k
EX = Â kP(X = k) = Â Â P(X = k)
k=1 k=1 m=1
• • •
= Â Â P(X = k) = Â P(X m), (1.34)
m=1 k=m m=1

where the interchange of the order of summation is justified

by the previous lemma. For X 0 that do not necessarily
take only integer values, we have the following analogue.
Lemma 1.3. For any nonnegative random variable X 0,
Z •
EX = P(X t) dt. (1.35)
0

In particular, one side is equal to +• only if the other side

is.
1.2 Discrete Probability Spaces and Distributions 17
R
Proof. If we express an = 0• I(t  an ) dt and plug into the
formula (1.30), we can formally write
Z •
EX = Â I(t  an )P(X = an ) dt
n 1 0
Z •
= Â I(t  an)P(X = an) dt
0 n 1
Z • Z •
= Â P(X = an) dt =
0 an t 0
P(X t) dt,

where in the last equality we used countable additivity of

probability in (1.31). If you are familiar with the Lebesgue
integration then the second step (interchanging the series and
integral) is justified by the Monotone Convergence Theorem.
If you are only familiar with the Riemann integral, we have R•
to say a bit
R more. First, if we write the improper integral 0
as Âm 1 mm 1 , we can use the previous lemma to interchange
the order of summation,
Z m
EX = ÂÂ I(t  an )P(X = an ) dt
n 1m 1 m 1
Z m
= ÂÂ I(t  an )P(X = an ) dt.
m 1n 1 m 1

If we write the series over n 1 as the limit of partial sums,

we can continue to write
Z m N
EX = Â N!•
lim
m 1 n=1
 I(t  an)P(X = an) dt
m 1
Z m
= Â lim
N!• m 1
fN (t) dt,
m 1

where we denoted
N
fN (t) := Â I(t  an)P(X = an).
n=1
18 1 Introduction

By the countable additivity of probability in (1.31),

•
lim fN (t) =
N!•
 I(t  an)P(X = an) = P(X t).
n=1

Moreover, this convergence is uniform, because

P(X t) fN (t) = Â I(t  an)P(X = an)

n>N
 Â P(X = an) ! 0 (1.36)
n>N

as N ! • as the tail of convergent series. This implies that

Z m Z m
lim fN (t) dt = P(X t) dt
N!• m 1 m 1

and, therefore, we again showed that

Z m Z •
EX = Â P(X t) dt = P(X t) dt.
m 1 m 1 0

This finishes the proof. t

u
Exercise 1.2.5. For any nonnegative random variable X 0
and n > 0, show that
Z •
n
EX = nt n 1 P(X t) dt. (1.37)
0

Hint: use (1.35).

Exercise 1.2.6. Show that if, for some c, e > 0 and t0 0,
c
P |X| t  for all t t0 ,
t 2+e
then EX 2 < •.
As we mentioned in the previous section (see equation
(1.14)), random variables X1 , . . . , Xn that satisfy
1.2 Discrete Probability Spaces and Distributions 19

n
P(X1 = x1 , . . . , Xn = xn ) = ’ P(Xi = xi ) (1.38)
i=1

for all possible values x1 , . . . , xn that they can take are called
independent. We will discuss independence and its mean-
ing in great detail later in this chapter, in Section 1.4, and in
this section simply use the definition (1.38). Next, let us give
some examples of probability spaces and distributions.
Example 1.2.1 (Bernoulli distribution). A distribution on
{0, 1} with the probabilities

P(1) = p and P(0) = 1 p

for some p 2 [0, 1] is called the Bernoulli distribution, and is

denoted B(p). A random variable X on any probability space
with this distribution is called a Bernoulli random variable.
This means that

P(X = 1) = p and P(X = 0) = 1 p, (1.39)

and is denoted by X ⇠ B(p). This can also be written as

P(X = x) = px (1 p)1 x
for x 2 {0, 1}. (1.40)

The expected value of the Bernoulli random variable X is

EX = 1 · p + 0 · (1 p) = p. (1.41)

Bernoulli distribution can be used to model a flip of a coin,

or an outcome of a random experiment with two possible
outcomes. The case p = 1/2 corresponds to a fair coin and
p 6= 1/2 is a biased coin. t
u
Example 1.2.2 (Products of Bernoulli). Given

p1 , . . . , pn 2 [0, 1]

for some n 2, the probability measure on the set of vectors

x = (x1 , . . . , xn ) 2 {0, 1}n given by
20 1 Introduction

n
P(x1 , . . . , xn ) = ’ pxi i (1 pi )1 xi
(1.42)
i=1

is called product of Bernoulli and is denoted by ⌦in B(pi ).

The probabilities add up to one because
n n
 n’ pxi i (1 pi ) 1 xi
=’ Â pxi i (1 pi )1 xi
= 1.
x2{0,1} i=1 i=1 xi 2{0,1}

A random vector X = (X1 , . . . , Xn ) consisting of n random

variables on some probability space has this distribution if
n
P(X = x) = ’ pxi i (1 pi )1 xi
(1.43)
i=1

for x = (x1 , . . . , xn ) 2 {0, 1}n . One way to construct such a

random vector is to take the probability space W = {0, 1}n
and simply let Xi (x) = xi be the ith coordinate xi of x 2 W.
In other words, X can be defined as the identity function on
its own sample space (see Remark 1.1 below). However, for
the rest of this example, X can be defined on any probability
space as long as it satisfies (1.43).
If we fix xi and consider the set
n o
n
A = y 2 {0, 1} : yi = xi

of all vectors with the ith coordinate fixed to xi then, by the

finite additivity of probability,
n
P(Xi = xi ) = Â P(X = y) = Â ’ pyi i (1 pi )1 yi
y2A y2A i=1
y
= pxi i (1 pi )1 xi
’ Â p j j (1 p j )1 yj
j6=i y j 2{0,1}

= pxi i (1 pi )1 xi
.
1.2 Discrete Probability Spaces and Distributions 21

In other words, each Xi has Bernoulli distribution B(pi ) and,

therefore,
n
P(X = x) = ’ P(Xi = xi ). (1.44)
i=1
If we recall the definition (1.38), we showed that a random
vector X = (X1 , . . . , Xn ) with the distribution given by the
product of Bernoulli (1.43) has independent coordinates Xi ⇠
B(pi ). This is not surprising since the two definitions look
almost identical, but we had to check that P(Xi = xi ) is the
corresponding ith factor in (1.43).
If all parameters pi = p 2 [0, 1] then X1 , . . . , Xn also have
the same distribution and are called independent identically
distributed, or i.i.d. for short. In this case, we can rewrite

P(X = x) = pÂin xi (1 p)n Âin xi . (1.45)

Instead of ⌦in B(p), we write B(p)⌦n . We will discuss in

Section 1.4 why the definition in (1.38) corresponds to our
intuitive notion of independence. t
u
Remark 1.1. Let us emphasize an important point. Random
variables defined on different probability spaces can have the
same distribution. For example, a Bernoulli random variable
can be defined on its sample space {0, 1} or it can be defined
as one of the coordinates Xi on the product space {0, 1}n as
in the previous example. In fact, a random variable defined
on any space W and taking two values {0, 1} (i.e. an indica-
tor of some set) is a Bernoulli random variable. Whenever
we are interested to study properties of a particular distribu-
tion, we can work with a random variable with this distri-
bution defined on any probability space. For example, when
we derived the equation (1.44) from (1.43), the calculation
depended only on the distribution of the vector X in (1.43)
and not on the particular probability space on which X is de-
fined. For this reason, it is quite common in Probability not
to specify a particular probability space and use generic no-
22 1 Introduction

tation P and E for probability measures and expectations on

any probability space.
On the other hand, it is often convenient to represent a
distribution using a specific random variable, sometimes de-
fined on a specific probability space. For example, in the ex-
ample of the Binomial distribution below, we will see that it
is much easier to compute its expectation using the linearity
of expectation and representation via the sum of Bernoulli
random variables. In the next section, we will see another
important example when a specific probability space con-
struction is also quite useful.
Example 1.2.3 (Erdős–Rényi random graph). Let us take
p 2 [0, 1] and consider a set

V = {v1 , . . . , vn }

of n elements, called vertices. For each pair of vertices vi

and v j for i 6= j, we draw an edge between them with the
probability p. We do this independently for different pairs of
vertices, for example by flipping a coin for each edge. The
random graph obtained in this way is denoted by G(n, p).
One can, for example, view this random graph as a model of
a group of people, where any two people like each other with
probability p and dislike each other with probability 1 p.
Of course, mathematically this means that we consider
i.i.d. Bernoulli random variables from the previous example,
✓ ◆
⌦m n
(X1 , . . . , Xm ) ⇠ B(p) with m = ,
2

since n2 is the number of distinct pairs of vertices, and we

say that the edge number k is present if the corresponding
Xk = 1. Although this model appears to be a special case of
the previous example, the graph structure will allow us to
consider various interesting functions of the edge indicators
X1 , . . . , Xm , for example, the number of triangles. t
u
1.2 Discrete Probability Spaces and Distributions 23

Example 1.2.4 (Binomial distribution). Let us consider n

i.i.d. Bernoulli random variables X1 , . . . , Xn ⇠ B(p). Then the
distribution of their sum

Sn = X1 + . . . + Xn

is called a binomial distribution, denoted B(n, p). The sum

takes values in the set

{0, 1, 2, . . . , n}.

What is the probability P(Sn = k)? We can break the set

{Sn = k} into a disjoint union of sets {X = (X1 , . . . , Xn ) = x}
over all x 2 {0, 1}n with exactly k coordinates equal to 1 and
sum their probabilities. For each such vector x, by (1.45),

P(X = x) = pk (1 p)n k .

Since there are ✓ ◆

n n!
=
k k!(n k)!
possible ways to choose which k coordinates are equal to 1,
✓ ◆
n k
P(Sn = k) = p (1 p)n k . (1.46)
k
This distribution on {0, . . . , n} is called the binomial distri-
bution B(n, p). The name comes from the binomial formula,
n ✓ ◆
n k n k
(a + b)n = Â ab ,
k=0 k

which for a = p, b = 1 p implies that the probabilities in

(1.46) add up to one. By linearity of expectation and (1.41),

ESn = EX1 + . . . + EXn = np. (1.47)

24 1 Introduction

We can also compute the expectation using the distribution

formula (1.46),
n ✓ ◆
n k
ESn = Â k p (1 p)n k , (1.48)
k=0 k

which we will leave as an exercise. Notice how much easier

the computation using the linearity of expectation was. This
will often be useful in situations when a random variable can
be written as a sum of indicators of events with probabilities
that can be computed more easily than the distribution of the
sum. t
u
Exercise 1.2.7. Show that the right hand side of (1.48) is np.
Example 1.2.5 (Multinomial distribution). As an analogue
of a coin flip, let us imagine rolling a die with k 2 sides that
have probabilities

p1 , . . . , pk > 0 such that p1 + . . . + pk = 1.

In other words, we consider any distribution on k outcomes.

If we roll the die n times, or roll n dice, the analogue of the
Binomial random variable is the number of times each side
comes up, which is a vector that belongs to the set
n o
W = (n1 , . . . , nk ) : n j 0 for j  k, n1 + . . . + nk = n .

The probability measure on this set given by

n!
P(n1 , . . . , nk ) = pn11 · · · pnk k (1.49)
n1 ! · · · nk !
is called a multinomial distribution. The product pn11 · · · pnk k
is the analogue of the product of Bernoulli, and the factor in
front represents the number of different sequences of rolls
that result in n1 , . . . , nk outcomes of each type. This factor
comes from the formula,
1.2 Discrete Probability Spaces and Distributions 25
✓ ◆✓ ◆✓ ◆ ✓ ◆
n! n n n1 n n1 n2 nk
= ··· ,
n1 ! · · · nk ! n1 n2 n3 nk
where we choose n1 dice with the first side up, then choose
n2 out of the remaining n n1 dice with the second side up,
and so on. t
u
Example 1.2.6 (Geometric distribution). Let 0 < p  1
and let us imagine that we toss a {0, 1}-valued coin until
the first time 1 comes up. The number N of coin tosses can
be any integer k 1 and

P(N = k) = (1 p)k 1
p, (1.50)

by (1.45), because in terms of the i.i.d. Bernoulli random

variables X1 , . . . , Xk the event {N = k} can be expressed as

{N = k} = {X1 = 0, . . . , Xk 1 = 0, Xk = 1}.

The sum of these probabilities is 1 by the geometric series

formula.
Exercise 1.2.8. Show that the expectation of the geometric
random variable (1.50) is
1
EN = . (1.51)
p
We see that, for example, if a probability of 1 in a biased
coin is 1/100, on average we ‘expect’ to toss the coin 100
times until we see 1. t
u
Example 1.2.7 (Poisson distribution). Consider l > 0. The
distribution
lk l
P(k) = e for k = 0, 1, 2, . . . (1.52)
k!
on nonnegative integers is called a Poisson distribution with
mean l , and denoted by Poiss(l ). The probabilities add up
26 1 Introduction

to one by the Taylor series for exponential. The word ‘mean’

is just another name for ‘expectation’, so the parameter l > 0
equals to the expectation of this distribution.
Exercise 1.2.9. Show that the mean of a Poisson random
variable X ⇠ Poiss(l ) is

EX = l . (1.53)

The Poisson distribution arises as an approximation of the

binomial B(n, p) distribution in the regime when p ⇡ l /n,
as will be explained in the next section.
Let us prove a very important stability property of the
Poisson distribution. Consider two random variables X1 and
X2 defined on the same probability space W and suppose that
X1 , X2 are independent and have Poisson distributions with
means l1 and l2 > 0 :

P(X1 = n, X2 = m) = P(X1 = n)P(X2 = m)

l1n l1 l2m l2
= e e . (1.54)
n! m!
We can construct such pair X1 and X2 as the coordinates on
W = {0, 1, . . .} ⇥ {0, 1, . . .}, in the same way we constructed
the product of Bernoulli above. Then the following holds.
Lemma 1.4 (Stability of Poisson). The sum X = X1 + X2 of
two independent Poiss(l1 ) and Poiss(l2 ) random variables
has Poiss(l1 + l2 ) distribution.
Proof. The sum can take only integer values n 0, and it is
equal to n, X1 + X2 = n, if and only if X1 = m and X2 = n m
for some m = 0, 1, . . . , n. Therefore,
1.2 Discrete Probability Spaces and Distributions 27

n
P(X = n) = Â P(X1 = m, X2 = n m)
m=0
n lm l2n m
= Â 1
m!
e l1
(n m)!
e l2
m=0
1⇣ n n! ⌘
= Â
n! m=0 m!(n m)!
l1m l2n m
e (l1 +l2 )

(l1 + l2 )n (l1 +l2 )

= e ,
n!
where the last equality is the binomial formula. t
u
One can show by induction that the following holds.
Exercise 1.2.10. Consider k 2 independent Poisson ran-
dom variables X1 , . . . , Xk with the parameters l1 , . . . , lk > 0,
n
k lj j
P(X1 = n1 , . . . , Xk = nk ) = ’ e lj
.
j=1 n j!

Show that their sum has Poiss(l1 + . . . + lk ) distribution.

Hint: To use the induction, show that if the random vari-
ables X1 , . . . , Xk are independent in the sense of (1.38) then
X1 + X2 , X3 , . . . , Xn are also independent.
Exercise 1.2.11. Show that if a random variable X satisfies
|X|  10 then |EX|  10.
Exercise 1.2.12. Find the event D such that

(D [ A)c [ (D [ Ac )c = B.

Exercise 1.2.13. Prove that limx!+• E|X| I(|X| x) = 0 if

E|X| < •. Hint: use (1.32).
Exercise 1.2.14. Three soccer players are trying out for a
soccer team. They will get one chance to take a penalty kick
to get a spot on the team. Each player has the probability 3/5
of making the goal, independently of each other. What is the
probability at least one of them will make the team?
28 1 Introduction

Exercise 1.2.15. Suppose W = {a, b, c, d}, P({a, b}) = 0.6,

P({b, c}) = 0.3, and P({c, d}) = 0.4. Describe the set of all
possible probabilities of a, b, c, and d.
Exercise 1.2.16. Four people play a game, where all of them
simultaneously roll a six sided die. The winner is determined
to be the first person to roll a one; then that person is elimi-
nated from the game. They roll again if more than one person
rolls a one. After that, the second place winner is determined
to be the first person who rolls a one, then that person is elim-
inated from the game. The third place winner is determined
to be the first person who rolls a one, then the game ends.
What is the expected number of rolls until the game ends?
1.3 Poisson Approximation to the Binomial 29

1.3 Poisson Approximation to the Binomial

Let us first notice how Poiss(l ) distribution arises as a limit

of B(n, p) when
l
p= and n ! •. (1.55)
n
If we rewrite the binomial probability of any fixed integer
k 0 with the choice of p = l /n, we see that
✓ ◆
n k
p (1 p)n k
k
n! ⇣ l ⌘k ⇣ l ⌘n k
= 1
k!(n k)! n n
lk n n 1 n k+1 ⇣ l ⌘n k
= · · ··· · 1
k! n n n n
l k
! e l,
k!
as n ! •, because k is fixed, k factors in the middle of the
form (n `)/n converge to one, and the last factor converges
to e l . We will generalize this in two ways. First, we will
prove an explicit error bound that works simultaneously for
any set of outcomes rather than for one fixed k. Moreover,
instead of the Binomial distribution, we will consider sums
of independent Bernoulli B(pi ) with possibly different pi . As
we alluded to in the previous section, our argument will be
based on a special coupling construction on some specific
probability space.
Given l > 0 and a set A ✓ {0, 1, 2, . . .}, let

lk
Poissl (A) = Â e l
(1.56)
k2A k!

be the Poiss(l ) probability of the set A. Our main result will

be the following.
30 1 Introduction

Theorem 1.1. Consider independent Bernoulli Xi ⇠ B(pi )

random variables with pi 2 (0, 1) for i  n defined on the
same probability space, and let

Sn = X1 + . . . + Xn and l = p1 + . . . + pn .

Then, for any subset A ✓ {0, 1, 2, . . .},

n
P(Sn 2 A) Poissl (A)  Â p2i . (1.57)
i=1

When all pi = p 2 (0, 1), the sum Sn has binomial B(n, p)

distribution, while l = np and the equation (1.57) becomes
l2 2
P(Sn 2 A) Poissl (A)  np = . (1.58)
n
If the right hand side is small, for example, if n ! • and l is
fixed, then binomial B(n, p) distribution is approximated by
the Poisson Poiss(l ) distribution in the strong sense that the
probabilities of all events are close.
Example 1.3.1. The chances of winning the jackpot in Lotto
Max are p = 1/28, 633, 528. If n = 20, 000, 000 tickets are
sold then the number of jackpot winners has the Binomial
B(n, p) distribution. (Of course, we ignore people’s prefer-
ences for their birthdays, or many people using the same for-
tune cookie.) Then, if
20, 000, 000
l = np = ⇡ 0.69848,
28, 633, 528
we can approximate the probability that no one wins a jack-
pot by
Poissl (0) = e l ⇡ 0.49734,
and the probability of exactly one winning ticket by
l
Poissl (1) = l e ⇡ 0.34738.
1.3 Poisson Approximation to the Binomial 31

The error of this approximation is bounded by

2l2 8
np = ⇡ 2.44 ⇥ 10
n
for each outcome, but also for any collection of outcomes.
For example, the same error is valid if we approximate the
probability of at most one winning ticket by e l + l e l ⇡
0.84472256. t
u
Example 1.3.2 (Sparse Erdős–Rényi graph). Let G(n, p)
be the Erdős–Rényi random graph with p = ln for a fixed l
and large n. This is called an example of a sparse Erdős–
Rényi random graph. If N is the number of edges connected
to a particular vertex, say v1 , then, since N ⇠ B(n 1, p), its
distribution can be approximated by Poiss(l ). t
u
Proof (Theorem 1.1). First, we will construct (on the same
probability space) independent Bernoulli Xi ⇠ B(pi ) random
variables and independent Poisson Yi ⇠ Poiss(pi ) random
variables coupled in a special way. Later on we will use the
stability property of Poisson,

Sn0 = Y1 + . . . +Yn ⇠ Poiss(l ), (1.59)

proved in Lemma 1.4 and Exercise 1.2.10 at the end of last

section. It is important to point out right away that P Sn 2 A
does not depend on the particular probability space on which
the random variables X1 , . . . , Xn are defined and depends only
on their distribution

P(X1 = xn , . . . , Xn = xn ) = ’ pxi i (1 pi )1 xi
,
in

so the construction below on a particular probability space

implies the result in full generality.
First, to explain what we have in mind, let us show how
to construct a pair X ⇠ B(p) and Y ⇠ Poiss(p) on the same
probability space in such a way that X and Y are close in
32 1 Introduction

some sense when p is small. Let us start with the usual

sample space of the Poisson random variable Y ⇠ Poiss(p),
namely,

pk p
W = {0, 1, 2, . . .} with P(k) = e .
k!
Notice that P(0) = e p is bigger than the probability 1 p
that a Bernoulli random variable X is equal to 0,
p
1 p<e for p 6= 0,

because 1 x is the tangent line to e x at zero. This means

that we can split e p into two positive numbers,
p p
e = (1 p) + (e 1 + p),

one of them being the probability that X = 0. This suggests

an idea that, in order to accommodate the Bernoulli random
variable X on the same probability space, we can split the
outcome 0 into two outcomes, say 1 and 0, and enlarge the
probability space,

W+ = { 1, 0, 1, 2, . . .}. (1.60)

We assign new probabilities to 1 and 0 and denote them

p
P p ( 1) = 1 p, P p (0) = e 1 + p. (1.61)

All the other probabilities will be untouched and denoted

pk p
P p (k) = e for k 1. (1.62)
k!
The way the probability space (W+ , P p ) was constructed
makes it obvious what will be X and what will be Y . For
w 2 W+ , we define a Bernoulli random variable
1.3 Poisson Approximation to the Binomial 33
⇢
0, if w = 1,
X = X(w) = (1.63)
1, if w 0,
and we define a Poisson random variable
⇢
0, if w = 1 or 0,
Y = Y (w) = (1.64)
w, if w 1.
What do we accomplish by this construction? Notice that
X(w) = Y (w) if and only if w = 1 or 1. Therefore,
p
P X =Y =1 p + pe
1 p + p(1 p) = 1 p2 ,

where we again used that 1 pe p. This implies that

P(X 6= Y )  p2 (1.65)

and, when p is small, the square makes it of an even smaller

order, so X and Y are equal with high probability.
Armed with this coupling construction for one pair (X,Y ),
how do we construct independent Xi ⇠ B(pi ) for i  n and
independent Yi ⇠ Poiss(pi ) on the same probability space, so
that each pair is coupled as we just described? Of course, we
will use the product space construction that we have already
seen for products of Bernoulli (i.e. defining random variables
in terms of coordinates on the product space), which will be
generalized in the next section. We will take
n
W = W+ , (1.66)

denote its elements by w = (w1 , . . . , wn ), and let

n
P(w) = ’ P pi (wi ). (1.67)
i=1

We will then define Xi and Yi in terms of the coordinate wi ,

Xi (w) = X(wi ) and Yi (w) = Y (wi ), (1.68)

34 1 Introduction

where X and Y were defined in (1.63) and (1.64). One can

check that: (a) Xi ⇠ B(pi ), and (b) they are independent over
i  n, by using the same calculation as in the Example 1.2.2
about the products of Bernoulli. For example, to calculate

P(X1 = x1 , . . . , Xn = xn )

for any x = (x1 , . . . , xn ) 2 {0, 1}n , we can sum P(w) over all
w 2 {X1 = x1 , . . . , Xn = xn }. If we denote

A(0) := { 1}, A(1) := {0, 1, 2, . . .},

then the definition (1.63) means that

Xi = xi () wi 2 A(xi ),

so the sum should be taken over w = (w1 , . . . , wn ) in

A(x) := A(x1 ) ⇥ · · · ⇥ A(xn ).

Since the probabilities are positive, Lemma 1.2 allows us to

sum over one coordinate wi at a time and, because P(w) is
given by the product (1.67), we will get

P(X1 = x1 , . . . , Xn = xn ) = Â P(w)
w2A(x)
n n
= Â ’ P pi (wi) = ’ Â P pi (wi )
w2A(x) i=1 i=1 wi 2A(xi )
n n
= ’ P pi A(xi ) = ’ pxi i (1 pi )1 xi
,
i=1 i=1

where the last equality follows from the definition of P pi .

This distribution appeared in Example 1.2.2, which means
that the random variables X1 , . . . , Xn are independent and
each Xi has Binomial distribution B(pi ).
A similar calculation for Y1 , . . . ,Yn also shows that each
Yi ⇠ Poiss(pi ) and they are independent (check!).
1.3 Poisson Approximation to the Binomial 35

If the sum Sn of Bernoulli random variables and the sum

Sn0of Poisson random variables are not equal then at least
one of the summands Xi is not equal to Yi , which means that
[
Sn 6= Sn0 ✓ Xi 6= Yi .
in

By the union bound, we get that

n
P Sn 6= Sn0  Â P Xi 6= Yi .
i=1

By construction and (1.65),

P Xi 6= Yi  p2i (1.69)

(exercise below asks to fill in the details) and, therefore,

n
6 Sn0 ) 
P(Sn = Â p2i .
i=1

Finally, using that (we leave it as an exercise below)

P Sn 2 A P Sn0 2 A  P Sn 6= Sn0

and using (1.59) finishes the proof. t

u
The Poisson distribution is often a good model for the
number of occurrences of certain events, when there is a
large number of opportunities and a small probability for an
event to occur at a given moment, such as: a number of shark
attacks in a given year, number of wrong number phone calls,
number of goals scored by a hockey player in a game, etc.
For example, in a given month, there are many phone calls
made to numbers relatively close to a given phone number
and there is always a small chance of misdial, so the total
will be a sum of many Bernoulli B(pi ) with small pi . We
might not know the numbers n and pi , but we may be able
to estimate the expected number l = Âin pi from previous
36 1 Introduction

experience and, as a result, estimate the probabilities using

the Poiss(l ) distribution.
For example, in his 9 seasons with Edmonton Oilers,
Wayne Gretzky had 1, 669 points in 696 games at a rate of
l = 2.397988 per game. The table below shows the number
of games broken down by the actual number of points, as
well as the corresponding Poisson approximation

lk l
696 ⇥ e .
k!
The agreement is quite remarkable.

Points 0 1 2 3 4 5 6 7 8 9
Games 69 155 171 143 79 57 14 6 2 0
Poisson 63.27 151.71 181.9 145.4 87.17 41.81 16.71 5.72 1.72 0.46

Exercise 1.3.1. Prove that, for any two variables X,Y on the
same (discrete) probability space,

P X 2A P Y 2A  P X 6= Y

for any subset A ✓ R.

Exercise 1.3.2. Fill in the details in the proof of (1.69) above.
Hint: if you are not sure what to do, take a look at Example
1.4.1 in the next section.
Exercise 1.3.3. An emperor orders preparations for a mas-
sive banquet for all of his 250 nobles. The Chancellor pre-
pares enough seats for 245 nobles knowing that the proba-
bility a noble will not come is 0.05. Use the Poisson approx-
imation to compute the probability that the Chancellor will
get to keep his head? (Or what is the probability that there
will be enough seats?) Estimate the error of approximation.
Exercise 1.3.4. Although many large and ferocious creatures
wander around the African savannah, none are quite as im-
1.3 Poisson Approximation to the Binomial 37

posing as the Hippopotamus. In fact, many smaller creatures

are often trampled by this massive, lumbering neighbour. On
average, three black mambas are trampled by Hippopotami
per year. What is the probability that no black mambas will
be squashed this year?
Exercise 1.3.5. The Kicker for the Dallas Cowboys scores
an average of 2 Field goals per game. Over this players 150
game career, what is the probability that in at least one game
he scored exactly 6 Field goals?
Exercise 1.3.6. When you bet on black in Roulette, your
chances are 18/38. Suppose also that bets over $250 are not
allowed by the casino. You decide to play the following strat-
egy: you start with a $1 bet and double the bet until either you
win (the same amount as the bet) or the bet exceeds $250;
then you start again with a $1 bet and repeat. We will call
this sequence of bets in the strategy until restart with a $1
bet ‘one round’. If you play 1000 rounds of this strategy,
what is the probability that your total winnings/losses are
0? Compute the exact formula and then compare it with
Poisson approximation.
38 1 Introduction

1.4 Independence, Conditional Distributions

Two events A, B ✓ W on a (discrete) probability space (W, P)

are called independent if

P(A \ B) = P(A)P(B). (1.70)

This definition is motivated by the following related concept.

If P(B) > 0 then the conditional probability of A given B is
defined as
P(A \ B)
P(A | B) := . (1.71)
P(B)
It represents the proportion (in the sense of probability) of
the event A inside B. From the two definitions it is clear that
if P(B) > 0 then A and B are independent if and only if

P(A | B) := P(A). (1.72)

In other words, if we know that an outcome w is in the set

B, this information does not alter the chances that w is in A.
This justifies the term ‘independent events’. We use the more
symmetric definition (1.70), because it makes sense even if
one or both events have zero probability.
Exercise 1.4.1. If A and B are independent, show that Ac and
B are also independent. Hint: write B = (B \ A) [ (B \ Ac )
and use that B \ A and B \ Ac are disjoint.
Given n 2, events Ai ✓ W for i  n on (W, P) are called
independent if, for any subset of indices I ✓ {1, . . . , n},
⇣\ ⌘
P Ai = ’ P(Ai ). (1.73)
i2I i2I

As in the above exercise, this implies that

⇣\ ⌘
P A⇤i = ’ P(A⇤i ), (1.74)
i2I i2I
1.4 Independence, Conditional Distributions 39

where each A⇤i is either Ai or Aci , which can be proved by

induction on the number of ⇤ = c. For example, if we know
(1.74) for all intersections with at most one ⇤ = c then

P Ac1 \ A2 \ Ac3 = P Ac1 \ A2 P Ac1 \ A2 \ A3

=P Ac1 P A2 P Ac1 P A2 P A3
=P Ac1 P A2 1 P A3
=P Ac1 P A2 P Ac3 .

The general case is similar. In its turn, (1.74) implies that

P(Ai | B) = P(Ai ) (1.75)

for any event B given by an intersection of A⇤j over any subset

of indices j 6= i. This means that any information that an
outcome w belongs to or does not belong to some sets A j
does not affect the probability that w belongs to another set
Ai . This definition is stronger than pairwise independence,
which is requiring any pair of sets (Ai , A j ) to be independent.
Exercise 1.4.2. Consider a regular tetrahedron die painted
blue, red and green on three sides and painted in all three
colours on the fourth side. If the die is equally likely to land
on any side, show that the appearances of these colours on
the side it lands on are pairwise-independent but not inde-
pendent.
Random variables X1 , . . . , Xn defined on (W, P) are called
independent if
n
P(X1 = a1 , . . . , Xn = an ) = ’ P(Xi = ai ) (1.76)
i=1

for all possible values a1 , . . . , an that these random variables

can take. This definition is equivalent to asking that, for ar-
bitrary subsets of outcomes Ai ,
40 1 Introduction

n
P(X1 2 A1 , . . . , Xn 2 An ) = ’ P(Xi 2 Ai ), (1.77)
i=1

which follows from (1.76) by summing over outcomes ai of

Xi in the set Ai . If we take the set Ai = R then the event
{Xi 2 R} = W has probability one and can be omitted on
both sides of the equation. This means that the definition of
independence of n random variables X1 , . . . , Xn automatically
includes independence for any subfamily of these random
variables, for example, X1 , X3 , X4 .
In the proof of Theorem 1.1 in the previous section, we
constructed independent random variables with Bernoulli
and Poisson distributions. The same construction can be used
to define independent random variables X1 , . . . , Xn on the
same probability space W with arbitrary distributions.
Example 1.4.1 (Product space construction). Let (Wi , Pi )
for i  n be arbitrary discrete probability spaces. Then the
space
W = W1 ⇥ · · · ⇥ Wn (1.78)
with the probability measure P on it given by
n
P(w) = P(w1 , . . . , wn ) = ’ Pi (wi ) (1.79)
i=1

is called the product of the above probability spaces. Here,

we denoted the elements of W by w = (w1 , . . . , wn ). If we
consider any subsets Ai ✓ Wi and consider the rectangle

A = A1 ⇥ · · · ⇥ An

in the product space W then its probability equals

1.4 Independence, Conditional Distributions 41

n
P(A) = Â P(w) = Â ’ Pi(wi)
w2A w2A i=1
n n
=’ Â Pi (wi ) = ’ Pi Ai , (1.80)
i=1 wi 2Ai i=1

which is the product of probabilities Pi (Ai ) of its sides.

One consequence of this is that any random variables
X1 , . . . , Xn : W ! R on this product space such that each Xi
depends only on the coordinate wi ,

Xi (w) = fi (wi ) for some fi : Wi ! R,

are independent, because the event {X1 = x1 , . . . , Xn = xn } is

a rectangle with sides { fi = xi }, so
n
P(X1 = x1 , . . . , Xn = xn ) = ’ Pi fi = xi . (1.81)
i=1

On the other hand, the event

Xi = xi = Xi = xi and X j 2 R for j 6= i

is a rectangle with the side { fi = xi } on the ith coordinate and

sides W j on the coordinates j 6= i, because no constraints are
imposed on these w j . Since P j (W j ) = 1, by (1.80),

P(Xi = xi ) = Pi fi = xi . (1.82)

Together, the equations (1.81) and (1.82) show that

n
P(X1 = x1 , . . . , Xn = xn ) = ’ P Xi = xi ,
i=1

which proves the claim that X1 , . . . , Xn are independent. If

we want these random variables to have specific prescribed
distributions then we can choose each space (Wi , Pi ) to be
the sample space of Xi with Xi (wi ) = wi . t
u
42 1 Introduction

We have already used independence in important ways in

the previous sections. Let us now discuss various additional
key consequences of the definition.
Let us partition random variables X1 , . . . , Xn into m disjoint
groups,
m
[
{1, . . . , n} = Ik ,
k=1
and consider random variables Yk for k  m given by some
functions
Yk = fk (Xi )i2Ik (1.83)
of random variables Xi that belong to the group k. Then the
following intuitive statement holds. A special example of this
already appeared in Exercise 1.2.10.
Lemma 1.5 (Grouping Lemma). If the random variables
X1 , . . . , Xn are independent then Y1 , . . . ,Ym defined in (1.83)
are also independent.
Proof. Let us consider the event {Y1 2 A1 , . . . ,Ym 2 Am } for
some sets Ak ✓ R. Let
n o
1
Bk = fk (Ak ) = (xi )i2Ik : fk (xi )i2Ik 2 Ak

be the set of all vectors that are mapped by fk into Ak . If

we use the notation xk = (xi )i2Ik and X k = (Xi )i2Ik for the
coordinates inside the kth group then

Yk 2 Ak = fk (X k ) 2 Ak = X k 2 Bk .

If we denote B = B1 ⇥ · · · ⇥ Bm then

P Y1 2 A1 , . . . ,Ym 2 Am = ÂP X 1 = x1 , . . . , X m = xm .
x2B

By independence, each term

P X 1 = x1 , . . . , X m = xm = P X 1 = x1 · · · P X m = xm
1.4 Independence, Conditional Distributions 43

(why?), so the above sum can be rewritten as

ÂP X 1 = x1 · · · P X m = xm
x2B
= Â P X 1 = x1 · · ·
m
 P X m = xm
1
x 2B1 x 2Bm
1
= P X 2 B1 · · · P X m 2 Bm
= P Y1 2 A1 · · · P Ym 2 Am .

This proves that

P Y1 2 A1 , . . . ,Ym 2 Am = P Y1 2 A1 · · · P Ym 2 Am ,

so Y1 , . . . ,Ym are independent. t

u
Example 1.4.2 (Stability of Poisson, revisited). Recall the
stability property of the Poisson distribution at the end of
Section 1.2. From the results of Section 1.3, we now know
that the Poisson distribution is a certain limit of the Binomial
distribution. Let us take some l1 , l2 > 0 and, for simplicity,
suppose that their ratio is rational,
l1 r1
=
l2 r2
for some integers r1 , r2 1. Take large n 1 and set
l1 l2
p := = 2 (0, 1).
nr1 nr2
By the results in the previous section, we know that if

m1 = nr1 , m2 = nr2 , m = m1 + m2 ,

and X1 , . . . , Xm ⇠ B(p) are i.i.d. Bernoulli then

Y = X1 + . . . + Xm1 ⇠ B(m1 , p) ⇡ Poiss(l1 ),

Z = Xm1 +1 + . . . + Xm ⇠ B(m2 , p) ⇡ Poiss(l2 ),
44 1 Introduction

while their sum

Y + Z ⇠ B(m, p) ⇡ Poiss(mp) = Poiss(l1 + l2 ).

By the Grouping lemma, Y and Z are independent, so this

approximation of the Poisson by independent flips of a coin
gives an intuitive explanation for the stability property of
Poisson. t
u
Next, we will show the following.
Lemma 1.6. If random variables X and Y are independent
and E|X| < •, E|Y | < • then E|XY | < • and

EXY = EXEY. (1.84)

Proof. To prove this, we will essentially apply Lemma 1.2

that allows us to rewrite

EXY = Â X(w)Y (w)P(w)

w2W

in an arbitrary order, or as a double summation. For possible

values an and bm of the random variables X and Y , let us
consider the event

Wnm = w : X(w) = an ,Y (w) = bm

and write, using Lemma 1.2 and independence of X and Y,

 X(w)Y (w)P(w) =   an bm P(w) (1.85)

w2W n,m w2Wnm
= Â an bm P(X = an ,Y = bm )
n,m
= Â an bm P(X = an )P(Y = bm )
n,m
= Â an P(X = an ) Â bm P(Y = bm )
n m
= EXEY.
1.4 Independence, Conditional Distributions 45

We can apply Lemma 1.2, because all the series on the right
hand side are absolutely convergent by our assumption that
E|X| < •, E|Y | < •. This finishes the proof. t
u
Of course, this lemma can be extended by induction, using
grouping lemma, to show that

E ’ Xi = ’ EXi , (1.86)
in in

when X1 , . . . , Xn are independent and all E|Xi | < •.

Exercise 1.4.3. If X1 , . . . , Xn are i.i.d. Bernoulli B(p), com-
pute E(X1 + . . . + Xn )2 .
We can repeat the calculation (1.85) for any function
f (X,Y ) instead of the product XY to obtain the following.
Theorem 1.2 (Fubini’s Theorem). If the random variables
X and Y are independent and E| f (X,Y )| < • then
h i
E f (X,Y ) = Â Â f (an , bm )P(Y = bm ) P(X = an ), (1.87)
n m

where the sum is over possible values an and bm of X and Y.

In other words, we can first fix X = an and average over the
distribution of Y and then average over an with respect to the
distribution of X. This is an analogue of the formula
ZZ Z 1 hZ 1 i
f (x, y) dxdy = f (x, y) dy dx
[0,1]2 0 0

in Calculus.
Remark 1.2. In the calculation (1.85), the fact that X and Y
are random variables and not, for example, random vectors
consisting of several coordinates each was never used. This
means that we can apply it to random vectors, as long as they
are independent of each other. Fubini’s theorem will some-
times be used in this way.
46 1 Introduction

One can also rewrite (1.87) by expressing the averages

over the distributions of X and Y in terms of averages over
the points w on our probability space W. We need two copies
of w because we have two averages. We leave this simple
observation as an exercise.
Exercise 1.4.4. Show that if the random variables X and Y
are independent and E| f (X,Y )| < • then
h i
E f (X,Y ) = Â Â f X(w),Y (w ) P(w ) P(w).
0 0
w2W w 0 2W

What is the analogue of Fubini’s theorem when X and Y

are not independent? As in the above calculation (1.85), we
can still write

E f (X,Y ) = Â f X(w),Y (w) P(w)

w2W
= Â f (an , bm )P(X = an ,Y = bm ),
n,m

but we have to stop here if we do not have independence.

However, if we multiply and divide by P(X = an ), we can
write this as
h P(X = an ,Y = bm ) i
  f (an, bm) P(X = an) P(X = an ),
n m

which looks quite similar to Fubini’s theorem. Recall that,

by (1.71),
P(X = an ,Y = bm )
= P Y = bm | X = an (1.88)
P(X = an )
is the conditional probability of the event Y = bm given that
X = an . This gives us a generalization of the Fubini theorem
for non-independent random variables.
1.4 Independence, Conditional Distributions 47

Theorem 1.3 (Fubini’s Theorem II). If E| f (X,Y )| < •

then the expectation of f (X,Y ) can be written as

E f (X,Y ) (1.89)
h i
= Â Â f (an , bm )P Y = bm | X = an P(X = an ),
n m

where the sum is over possible values an , bm of X and Y.

For any fixed an , the conditional probability
P(X = an ,Y = bm )
P Y = bm | X = an = (1.90)
P(X = an )
in (1.88) viewed as a function on the set of values {bm } is
called the conditional distribution of Y given X = an . It is
a probability measure because the sum over all values bm is
obviously equal to one.
The expectation of a function g(Y ) of Y with respect to
this conditional distribution, denoted
⇥ ⇤
E g(Y ) | X = an := Â g(bm )P Y = bm | X = an ,
m

is called the conditional expectation of g(Y ) given X = an .

Notice that the sum over m inside the bracket in (1.89) is a
conditional expectation
⇥ ⇤
E f (an ,Y ) | X = an := Â f (an , bm )P Y = bm | X = an
m

of f (an ,Y ) given X =⇥ an . This conditional

⇤ expectation can
also be denoted by E f (X,Y ) | X = an , because X is fixed
to be an .
In this terminology, we can describe the Fubini formula
(1.89) as a two-step process. We first fix the value of X = an
and average f (an ,Y ) over possible values of Y with respect
to the conditional distribution (1.90). This average (called
conditional expectation) depends on an , so it can be viewed
48 1 Introduction

as some function h(an ), and we average it with respect to the

distribution of X,

E f (X,Y ) = Â h(an )P(X = an ) = Eh(X).

n

Quite often, by nature of the problem, the distribution

P(X = an ) and the conditional distribution (1.90) are known
or defined first, and then the joint distribution of (X,Y ) is
computed as

P(X = an ,Y = bm ) = P Y = bm | X = an P(X = an ). (1.91)

In this case, the representation of E f (X,Y ) in (1.89) is very

natural, because, for a fixed value of X = an , we average
first with respect to known conditional distribution and then
average with respect to the distribution of X, which is called
the marginal distribution of X. This type of construction of
the pair (X,Y ) is sometimes called a two-stage experiment.
Let us consider an example.
Example 1.4.3. Suppose we want to predict the number of
shark attacks in Florida over the next summer. This depends
on the number of people going swimming, which depends on
the weather. When the weather is typical, the average num-
ber of shark attacks is 10; when the weather is colder than
normal, the average number of shark attacks is 8; when the
weather is warmer than normal, the average number of shark
attacks is 12. The forecast predicts warmer than usual sum-
mer with probability 60%, typical weather with probability
30%, and colder weather with probability 10%.
First, let us write down a mathematical model for this
problem. We have two sources of randomness. The first one
is the weather, which determines the average number L of
shark attacks and, according to the forecast,

P(L = 8) = 0.1, P(L = 10) = 0.3, P(L = 12) = 0.6.

1.4 Independence, Conditional Distributions 49

The second random variable is the number N of shark at-

tacks, which can be modelled by the Poisson distribution
Poiss(l ) once we know the mean l . Since the average L
is random itself, what we are modelling here is precisely the
conditional distribution
lk l
P N=k|L=l = e .
k!
By (1.91), the joint distribution of (L, N) equals
8
8k 8
>
>
< 0.1 · k! e , if l = 8,
k
P(N = k, L = l ) = 0.3 · 10 k! e
10 , if l = 10,
>
>
: k
0.6 · 12
k! e
12 , if l = 12.

We can use Fubini’s Theorem 1.3 to compute expectation of

any (integrable) function of (L, N) and, in particular,

EN = 0.1 · 8 + 0.3 · 10 + 0.6 · 12 = 9.8,

because the conditional distribution of N for a fixed L = l is

Poiss(l ), so its conditional expectation equals to l . We can
similarly compute the probabilities of events, for example,
• •
8k 10k
P(N 10) = 0.1 Â e 8
+ 0.3 Â e 10
k=10 k! k=10 k!
•
12k
+ 0.6 Â k! e 12
⇡ 0.645,
k=10

because, it terms of (L, N) the event N 10 can be written

as (L, N) 2 {8, 10, 12} ⇥ {10, 11, . . .}. t
u
Remark 1.3. Notice how in this example we talked about
L and N as random variables without defining a probabil-
ity space W on which they are defined and specifying how
L = L(w) and N = N(w) are defined as functions on this
space. This is because we are only interested in their distri-
50 1 Introduction

butions or, in other words, in probabilities of values that these

random variables model, so the precise probability space is
not important. In the situations like this, we can always think
of the sample space as our probability space and random
variables as the coordinates on this space. For example, when
we consider R2 and denote its elements by (x, y), we can
think of x as the first coordinate on R2 but also as a function
f (x, y) = x. In our example, the sample space was

{8, 10, 12} ⇥ {0, 1, 2, . . .}

and L and N can be viewed as the coordinates on this space.

On the other hand, the advantage of not mentioning the
precise probability space on which these random variables
are defined is that it does not really matter. For example, if
L was a function of not only the weather forecast but also
economic forecast (think of beach vacations) and came out as
an output of some complicated model on another probability
space, for the calculations regarding the shark attacks only
the distribution on the values {8, 10, 12} would matter. t
u
Example 1.4.4 (Colouring property of Poisson). Let us
consider one more two-stage experiment. Let N be a Poiss(l )
random variable. Given N = n, we flip n coins with Bernoulli
B(p) distribution and we let N1 and N2 be the number of
coins taking values 1 and 0 correspondingly. Another way to
say it is that the conditional distribution of N1 given N = n is
binomial B(n, p),
✓ ◆
n k
P N1 = k | N = n = p (1 p)n k
k
for 1  k  n, and N2 = N N1 .
Lemma 1.7 (Poisson colouring). The random variables N1
and N2 are independent and have distributions Poiss(pl )
and Poiss((1 p)l ).
1.4 Independence, Conditional Distributions 51

If a chicken lays Poisson number of eggs and we colour them

red or blue by tossing a coin then knowing the number of red
eggs gives no information about the number of blue eggs and,
moreover, both are Poisson. Also, notice that their means
agree with the linearity of expectations, since

l = EN = EN1 + EN2 = pl + (1 p)l .

Proof. By (1.91), the joint distribution of (N1 , N) is

✓ ◆
n k n k ln l
P N1 = k, N = n = p (1 p) ⇥ e .
k n!
Since N1 + N2 = N, we can also write this as
✓ ◆
n k ln l
P N1 = k, N2 = n k = p (1 p)n k ⇥ e .
k n!
Making the change of variables m = n k,
✓ ◆
k+m k m l k+m l
P N1 = k, N2 = m = p (1 p) ⇥ e .
k (k + m)!
The right hand side can be rewritten as

(k + m)! k l k+m
p (1 p)m ⇥ e l
k!m! (k + m)!
(pl ) k (pl )m
= e pl ⇥ e (1 p)l ,
k! m!
where we recognize the Poisson probabilities Poiss pl (k) and
Poiss(1 p)l (m). Hence,

P N1 = k, N2 = m = Poiss pl (k) ⇥ Poiss(1 p)l (m).

Summing both sides over m 0 or over k 0 gives that

52 1 Introduction

P(N1 = k) = Poiss pl (k),

P(N2 = m) = Poiss(1 p)l (m),

so these random variables have distributions Poiss(pl ) and

Poiss((1 p)l ). This implies that

P N1 = k, N2 = m = P N1 = k P N2 = m ,

which means that N1 and N2 are independent. This finishes

the proof. t
u
Two-stage constructions can be extended to any number
of stages. For example, consider the following exercise.
Exercise 1.4.5. Show that

P(A \ B \C) = P(C | A \ B)P(B | A)P(A),

when the events A and A \ B have positive probabilities.

Once you have done this exercise, it should be clear that the
joint distribution formula (1.91) can be extended to longer
vectors of random variables (X1 , . . . , Xn ),

P X1 = x1 , . . . , Xn = xn
n 1
= ’ P Xk+1 = xk+1 | X1 = x1 , . . . , Xk = xk , (1.92)
k=0

where the term for k = 0 is just P(X1 = x1 ). In other words, if

we know the distribution of the next outcome Xk+1 once the
preceding outcomes X1 = x1 , . . . , Xk = xk are revealed, this
allows us to reconstruct the joint probabilities of the entire
sequence recursively.
Example 1.4.5 (Markov chains). A sequence X1 , . . . , Xn is
called a Markov chain if

P Xk+1 = xk+1 | X1 = x1 , . . . , Xk = xk
= P Xk+1 = xk+1 | Xk = xk , (1.93)
1.4 Independence, Conditional Distributions 53

which means that the conditional distribution of the next

outcome Xk+1 given the preceding outcomes X1 = x1 , . . . ,
Xk = xk depends only on the most recent outcome Xk = xk .
This property is called a Markov property of the sequence,
also called a memoryless property, in the sense that we do
not need to remember the entire past and only need to know
the most recent outcome to know the chances of the next
outcome. The conditional distribution in (1.93) is also called
transition probability. Markov chain is called homogeneous
if transition probabilities P(Xk+1 = a | Xk = b) do not de-
pend on the index k. The entire Chapter 5 will be devoted to
homogeneous Markov chains. t
u
Exercise 1.4.6. If X and Y are independent, fill in the blanks
in their joint distribution
X\Y 1 2 3
1 ? ? 0.1
2 0.24 0.16 0.4

and compute P(Y = 2 | X = 1).

Exercise 1.4.7 (Bayes’ Formula). Show that, for possible
values a and b of the random variables X and Y,
P(X = a | Y = b) P(Y = b)
P Y =b|X =a = .
Âm P(X = a | Y = bm ) P(Y = bm )
Exercise 1.4.8. If X1 , X2 , and X3 are independent Bernoulli
B(1/2) random variables, find the conditional distribution of
X1 + X2 + X3 given X1 X2 X3 = 0.
Exercise 1.4.9. Charlie met a pretty girl at the local coffee
shop Friday night. She wanted to see Charlie again, so she
wrote her phone number on his hand; her number was 854-
2564. When Charlie got home he realized that his hands were
sweating more than usual, and two of the numbers wore off.
Unfortunately, Charlie could only remember that the girl’s
phone number had a lot of fours and fives, so when he calls
54 1 Introduction

her the next day he dials either four or five in place of the
numbers that wore away, and he flips his lucky coin twice
to pick the numbers (heads - four, tails - five). What is the
probability that he guessed right and calls the girl?
Exercise 1.4.10. In a Probability class, 20% of the students
are failing. Of the entire class, 60% of students are both pass-
ing and going to tutorials. Given that a student is passing,
what is the probability that she is going to tutorials?
Exercise 1.4.11. In the Example 1.4.3, what is the condi-
tional distribution P(L = l | N = 15)? In other words, if
we did not follow the weather in Florida over the summer
and later saw on the news that there were 15 shark attacks,
how should the probabilities of normal, colder and warmer
weather be updated given this information?
Exercise 1.4.12. Bill drives to work 50% of the time and
walks the rest of the time. If Bill drives, he is speeding 5mph
60% of the time, 10mph 10% of the time and not speeding
30% of the time. He is always late when he walks or drives
within speed limit, he is always on time when he is speeding
10mph, and he is late 50% of the time when he is speeding
only 5mph. If he is running late, what is the probability he is
walking.
Exercise 1.4.13. Generalize the colouring property of the
Poisson to more than two colours using the multinomial dis-
tribution in Example 1.2.5 in Section 1.2.
Exercise 1.4.14. Show that the Markov property in (1.93) is
equivalent to

P X1 = x1 , . . . , Xk 1 = xk 1 , Xk+1 = xk+1 | Xk = xk
= P X1 = x1 , . . . , Xk 1 = xk 1 | Xk = xk
⇥ P Xk+1 = xk+1 | Xk = xk .
1.4 Independence, Conditional Distributions 55

This can be expressed by saying that the past outcomes X1 =

x1 , . . . , Xk 1 = xk 1 and the future outcome Xk+1 = xk+1 are
conditionally independent given the present Xk = xk .
Exercise 1.4.15. If X1 , . . . , Xn are independent and all Xi 0,
show that
Z •h n i
E max Xi = 1 ’ P(Xi < t) dt.
in 0 i=1

Hint: use Lemma 1.3.

56 1 Introduction

1.5 Applications of the Linearity of Expectation

When we introduced the binomial distribution in Section 1.2,

we saw that computing its expectation was much easier using
the linearity of expectation. Similarly, whenever a random
variable N takes integer values n 0 and can be represented
as a sum of indicators of some events Ai ✓ W,
n
N = N(w) = Â I(w 2 Ai ),
i=1

then its expectation can be computed using the linearity of

expectation,
n n
EN = Â E I(w 2 Ai ) = Â P(Ai ).
i=1 i=1

It is often easier to compute the probabilities P(Ai ) than to

compute the probabilities P(N = n) and to use the formula
EN = Ân 1 nP(N = n).
Example 1.5.1 (Functions on a finite set). Let us consider
the set W of all functions

w : 1, . . . , n ! 1, . . . , n .

Choosing a function at random from W means that we assign

them equal probabilities
1
P(w) = .
nn
Let N = card R(w) be the cardinality of the range of w,

R(w) = w(1), . . . , w(n) .

One can also think of this as throwing n balls into n boxes

at random and N being the number of non-empty boxes. If
we want to compute the distribution P(N = k) then we need
1.5 Applications of the Linearity of Expectation 57

to count how many functions have the range of cardinality

exactly k. On the other hand, if we only want to know EN
we can represent
n
N = Â I i 2 R(w) ,
i=1

so that, by symmetry,
n
EN = Â P i 2 R(w) = nP 1 2 R(w) .
i=1

To calculate the probability that 1 2 R(w), we can write it as

(n 1)n
P 1 2 R(w) = 1 P 1 62 R(w) = 1 ,
nn
because the number of functions w : {1, . . . , n} ! {2, . . . , n}
is (n 1)n . Therefore,
⇣ ⇣ 1 ⌘n ⌘ ⇣ 1⌘
EN = n 1 1 ⇠n 1 ,
n e
where ⇠ means that the ratio goes to one. t
u
To compute the distribution P(N = k) in the previous
example, we need to use the inclusion-exclusion principle,
which is a generalization of Exercise 1.1.1, part 2.
Lemma 1.8 (Inclusion-exclusion principle). For any events
A1 , . . . , An ✓ W,

P [in Ai = Â P(Ai) Â P Ai \ A j
i i< j
+ Â P Ai \ A j \ Ak + . . .
i< j<k
. . . + ( 1)n 1 P \in Ai . (1.94)

Proof. Let us begin by writing

58 1 Introduction

(1 x1 ) · · · (1 xn ) = 1 Â xi + Â xix j
i i< j

 n
xi x j xk + . . . + ( 1) x1 · · · xn ,
i< j<k

where we multiplied out the product and wrote all possible

ways to choose factors xi and factors 1.
Exercise 1.5.1. Prove the above identity by induction on n.
To continue the proof, let us regroup the terms in the above
identity,

1 (1 x1 ) · · · (1 xn ) =  xi  xix j
i i< j
+ Â xi x j xk + . . . + ( 1) n 1
x1 · · · xn .
i< j<k

Let us take xi = I(w 2 Ai ) and rewrite the left hand side as

1 ’ 1 I(w 2 Ai ) = 1 ’ I(w 2 Aci)

in in
⇣ \ ⌘ ⇣ ⇣ \ ⌘c ⌘
c
=1 I w 2 Ai = I w 2 Aci
in in
⇣ [ ⌘
= I w 2 Ai .
in

Therefore, taking expectation of the left hand side we get

P([in Ai ). On the other hand, taking expectation of the right
hand side, by linearity of expectation we get the right hand
side of (1.94), and this finishes the proof. t
u
Example 1.5.2 (Functions on a finite set, continued). Let
us now compute the probability P(N = k). First of all, there
are nk ways to choose k values out of {1, . . . , n} that will
constitute the range R(w) when N(w) = k. Then we need
to count how many functions are there with this range. By
symmetry, we can assume that the range is {1, . . . , k}. Let us
denote the set of nk functions
1.5 Applications of the Linearity of Expectation 59

w : 1, . . . , n ! 1, . . . , k

by Wk . Then the question is how many surjective functions

are there in Wk , i.e. whose range covers all the k values.
Again, we can think of this as the number of ways to place n
balls into k boxes so that no box is left empty. It will be eas-
ier to count the complement, namely, all functions w 2 Wk
whose range misses at least one of the values. Let us con-
sider the event
n o
Ai = w 2 Wk : i 62 R(w) ,

where R(w) is the range of w, and let Pk be the uniform

probability on Wk ,
card(A)
Pk (A) = . (1.95)
nk
The set of all functions w 2 Wk whose range misses at least
one of the values is just A1 [ . . . [ Ak , and we can apply the
inclusion-exclusion principle to this union. For any `  k and
any indices 1  i1 < . . . < i`  k,
(k `)n
Pk Ai1 \ . . . \ Ai` = ,
kn
because the number of functions in Wk that are not allowed
to take values i1 < . . . < i` is (k `)n . There are k` choices
of such indices so, by inclusion-exclusion principle,
k ✓ ◆ n
` 1 k (k `)
Pk A1 [ . . . [ Ak = Â ( 1) .
`=1 ` kn

By (1.95), to get the cardinality, we need to multiply by kn ,

and recalling that we counted over the complement, we get
that the number of surjective functions in Wk is
60 1 Introduction

k ✓ ◆ k ✓ ◆
k k
kn  ( 1)` 1 ` (k `)n =  ( 1)`
`
(k `)n .
`=1 `=0

Finally, multiplying by the number nk of ways to choose k

values out of {1, . . . , n}, we get that the number of functions
in W with N(w) = k is
✓ ◆ k ✓ ◆
n ` k
k `=0 Â ( 1)
`
(k `)n

and, therefore, we finally get

✓ ◆ ✓ ◆
1 n k k
P(N = k) = n
n k `=0 Â ( 1)`
`
(k `)n .

Using this formula together with EN = Ânk=0 kP(N = k), it is

much harder to get to the simple answer for EN we obtained
above by the linearity of expectation. t
u
In many problems it is not feasible to compute the distri-
bution of N, but we can obtain useful information about the
tail probabilities P(N k) by computing the expected num-
ber of events that ‘witness’ that N k. Let us look at a couple
of examples.
Example 1.5.3 (Longest increasing subsequence of a ran-
dom permutation). Let Sn be the set of all permutations s
of n elements, i.e. all bijections

s : 1, . . . , n ! 1, . . . , n .

Let P be the uniform probability on Sn such that P(s ) =

1/n!. If, for some i1 < . . . < ik ,

s (i1 ) < . . . < s (ik ),

we call this an increasing subsequence of length k. Let

us define L = L(s ) to be the maximum length among all
1.5 Applications of the Linearity of Expectation 61

increasing subsequences of s . For example, the permuta-

tion (1, 5, 2, 4, 3) has L = 3 with subsequences (1, 2, 4) and
(1, 2, 3) having length 3. To compute the distribution of L or
even EL is too difficult, and for a number of years it was an
open problem to show that
EL
lim p = 2.
n!• n

In thispexample we will show that, for most permutations, p

L  3 n and, as a result, the expectation EL  4 n.
Let Nk = Nk (s ) be the number of increasing subsequences
of length k in the permutation s . In other words, Nk is the
number of different i1 < . . . < ik on which s is increasing.
Let us compute the expectation ENk . If
n o
Ik = i = (i1 , . . . , ik ) : 1  i1 < . . . < ik  n

is the set of all possible choices of k different indices then

we can write

Nk = ÂI s (i1 ) < . . . < s (ik )

i2Ik

and, by the linearity of expectation and symmetry,

ENk = ÂP s (i1 ) < . . . < s (ik )

i2Ik
✓ ◆
n
= P s (1) < . . . < s (k) .
k
The last probability equals
1
P s (1) < . . . < s (k) =
k!
which can be argued by symmetry, since all possible orders
among s (1), . . . , s (k) are equally likely. Or, we can simply
count the number of permutations s such that s (1) < . . . <
62 1 Introduction

s (k), which equals

✓ ◆
n n!
(n k)! = ,
k k!

because there are nk ways to choose values s (1), . . . , s (k)

(and one way to arrange them in the increasing order) and
(n k)! ways to arrange the remaining n k values. Dividing
by n! we again get that the probability is k!1 . This implies
✓ ◆
n 1
ENk = . (1.96)
k k!
We can already use this to get some information about the
length L of the longest increasing subsequence, so we will
come back to this a little later in this section. t
u
Example 1.5.4 (Cliques in Erdős–Rényi random graph).
If V = {v1 , . . . , vn } is the set of vertices of a graph G, a subset
of vertices W ✓ V is called a clique if all vertices in W are
connected by edges in the graph. The clique number w(G)
of the graph G is the size k of the largest clique.
In this example we will consider the Erdős–Rényi random
graph G = G(n, p) from Example 1.2.3 in Section 1.2 and
show that its clique number is typically (meaning, with prob-
ability close to 1) not bigger than of order O(log n). We will
do this by counting the expected number of cliques of a given
size.
Let Nk be the number of cliques of size k in G and let us
compute ENk . There are nk subsets of size k and, for a given
subset W of k vertices to be a clique, all of the 2k edges
between these vertices must be present, which happens with
probability
k
P(W is a clique) = p(2) .
As in the previous example, representing Nk as the sum of
indicators over all such W and using the linearity of expecta-
tion, we get
1.5 Applications of the Linearity of Expectation 63
✓ ◆
n (k )
ENk = p2 . (1.97)
k
We will extract some information from this formula a little
later in this section. t
u
To extract information from the formulas we proved in the
last two examples, we will need one result. It is absolutely
fundamental and will be the starting point of many much
more sophisticated results later.
Theorem 1.4 (Chebyshev’s inequality). For any random
variable X and any x > 0,
EX I(X x)
P(X x)  . (1.98)
x
Moreover, if X 0 then, for any x > 0,
EX
P(X x)  . (1.99)
x
Proof. We can write

X I(X x) x I(X x),

because the indicator I(X x) is non-zero only when X x.

Taking expectation of both sides and using the monotonicity
property of expectation, we get

EX I(X x) xE I(X x) = xP(X x).

Dividing both sides by x finishes the proof of (1.98). If X 0

then X I(X x)  X and, taking expectations,

EX I(X x)  EX.

In this case, (1.98) implies (1.99). t

u
The proof of Chebyshev’s inequality (1.99) looks almost
trivial, but its historical significance and its usefulness can
not be overstated.
64 1 Introduction

In this section, we will use it for integer-valued random

variables N 0, for which {N > 0} = {N 1}. Using
Chebyshev’s inequality with x = 1,

P(N > 0) = P(N 1)  EN. (1.100)

This tells us that if the expectation EN is very small then

P(N = 0) = 1 P(N > 0) is close to 1, so for most outcomes
w 2 W, we have N = N(w) = 0.
Another result we will need is Stirling’s formula,
p ⇣ n ⌘n
n! ⇠ 2pn (1.101)
e
as n ! •, where as usual ⇠ means that the ratio of two sides
goes to one. We will not prove this here in order not to get
sidetracked by a somewhat technical result. However, let us
mention that it is very easy to show that
p ⇣ n ⌘n
n! ⇠ c n
e
for some constant c > 0, which was discovered by Abraham p
de Moivre, and it requires more work to show that c = 2p,
which was proved by James Sterling. There are a number of
different
p proofs of this fact. For our applications below, the
factor 2pn in the Stirling’s formula will be irrelevant, so
we can just as well use the formula with some unknown c.
Also, there exist precise estimates in Stirling’s formula, for
example, for n 1,
1 p
⇣ n ⌘n 1 p
⇣ n ⌘n
e 12n+1 2pn  n!  e 12n 2pn . (1.102)
e e
Now we go back to our two previous examples.
Example 1.5.5 (Longest increasing subsequence, part 2).
We showed in (1.96) that the expectation of the number Nk
of increasing subsequences of length k in a random uniform
permutation of {1, . . . , n} is
1.5 Applications of the Linearity of Expectation 65
✓ ◆
n 1 n! nk
ENk = =  . (1.103)
k k! (k!)2 (n k)! (k!)2

By Stirling’s formula, k! ( ke )k , so
⇣ e ⌘2k ⇣ epn ⌘2k
ENk  nk = . (1.104)
k k
By Chebyshev’s inequality (1.100),
⇣ epn ⌘2k
P(Nk > 0)  ,
k
p
and if we take k = 3 n then
⇣ ⌘ ⇣ e ⌘6pn p
n/2
P N3pn > 0  e ,
3
where in the last inequality we used that (e/3)6  e 1/2 to
simplify the expression. First of all, this means that it is very
unlikelypthat there exists even one increasing subsequence of
length 3 n in a random permutation. Of course, in this case
the length
p L of the longest increasing subsequence is smaller
than 3 n, so we showed that
p p
P L 3 n  e n/2 . (1.105)

From this, we can also estimate the expectation EL. Since L

never exceeds n,
h p p i
L = L I L < 3 n +I L 3 n
p p
 3 n+nI L 3 n .

Taking expectation of both sides, we get

p p p p
n/2 p
EL  3 n + nP L 3 n  3 n + ne  4 n,
66 1 Introduction

although the inequality (1.105) is, of course, a much stronger

statement. t
u
Next, we will come back to the example of cliques in the
Erdős–Rényi random graph. We will have to carry out some
tedious calculations, but it is a good idea to see at least one
example which show that the linearity of expectation can
yield non-trivial information when it is not clear right away.
These calculations will also prepare us for Section 2.4 in the
next chapter, where we will obtain further information using
the second moment method.
Example 1.5.6 (Cliques in Erdős–Rényi graph, part 2).
Let us recall the formula in (1.97), which states that
✓ ◆
n (k ) n!
f (k) := p2 = pk(k 1)/2 (1.106)
k k!(n k)!
is the expectation of the number Nk of cliques of size k in the
Erdős–Rényi random graph. For a fixed k, this number goes
to infinity when n ! •, so we expect many cliques of small
size. If we find k = k(n) for which this expectation becomes
small, Chebyshev’s inequality will tell us that it is unlikely
to have cliques of that size. Since
n!
f (k + 1) := pk(k+1)/2 ,
(k + 1)!(n k 1)!
it is easy to check that
f (k + 1) n k k
= p. (1.107)
f (k) k+1
Notice that this ratio is decreasing in k, it is bigger than 1
for small k, and eventually it becomes smaller than 1. This
means that f (k) is first increasing and then decreasing. Since

f (1) = n and f (n) = pn(n 1)/2

⌧ 1,

there is a unique point k0 such that

1.5 Applications of the Linearity of Expectation 67

f (k0 ) 1 > f (k0 + 1). (1.108)

This is the transition point where the expected number of

cliques of size k becomes smaller than 1 and, heuristically,
the graph is unlikely to have cliques of size bigger than k0 .
We will make this more precise below, but first we would
like to understand how k0 looks like. We can not write down
the exact formula for k0 , but we can accurately estimate its
location.
Lemma 1.9 (Clique number). For large n, we have
2
k0 = k0 (n) ⇠ log n. (1.109)
log(1/p)
As usual, ⇠ means that the ratio of two sides goes to 1. In
fact, we will give precise explicit bounds in the proof.
Proof. We will use simple bounds for binomial coefficients:
⇣ n k ⌘k ✓n◆ (n k + 1) · · · n
 =  nk ,
k k 1···k
which imply the bounds for f (k):
⇣n ⌘k
1 pk(k 1)/2  f (k)  nk pk(k 1)/2
. (1.110)
k
The right hand side is smaller than 1 if np(k 1)/2 < 1. Taking
logarithms, this is the same as k > bn , where we introduce
the notation
2
bn := c p log n + 1, where c p := . (1.111)
log(1/p)
This means that f (k) 1 can hold only if k  bn , which
proves that k0  bn . Similarly, taking logarithms, the left
hand side of (1.110) is 1 if and only if
⇣n ⌘
k  c p log 1 + 1.
k
68 1 Introduction

Since f (k0 + 1) < 1, k0 + 1 violates this inequality, so

⇣ n ⌘
k0 + 1 c p log 1 +1
k0 + 1
and, therefore,
⇣ n ⌘
k0 c p log 1 .
k0 + 1
Since we have already shown that k0  bn , this implies that
⇣ n ⌘
k0 c p log 1 .
bn + 1
Thus, we obtained explicit bounds on k0 ,
⇣ n ⌘
c p log 1  k0  bn . (1.112)
bn + 1
It is easy to see that the ratio of the two sides goes to 1,
because we can rewrite the left hand side as
⇣ n ⌘
c p log 1 = c p log n c p log(bn + 1)
bn + 1
⇣ bn + 1 ⌘
+ c p log 1
n
and notice that the last two terms are much smaller than the
first when n ! •. This finishes the proof of (1.109). t
u
To continue with our example, we will now check that as
soon as the expectation f (k) of the number Nk of cliques of
size k in (1.106) crosses level 1 as defined in (1.108), i.e. for
k > k0 , the expectation f (k) becomes small very quickly. By
(1.107),
f (k + 1)  npk f (k).
Also notice that, for any e > 0,
(2 e) log n 1
k () pk  2 e . (1.113)
log(1/p) n
1.5 Applications of the Linearity of Expectation 69

Since, by the above lemma, for any e > 0, k0 satisfies this

condition for large enough n, we get that, for k k0 ,
1
f (k + 1)  npk f (k)  f (k).
n1 e
Repeating this recursively m 1 times, we get
1
f (k + m)  e)
f (k).
nm(1
In particular, since f (k0 + 1) < 1,
1
f (k0 + m + 1)  e)
.
nm(1
By Chebyshev’s inequality (1.100),
1
P Nk0 +m+1 > 0  e)
. (1.114)
nm(1
This means that, it is unlikely that the random graph has
cliques of size k0 + 2, and it gets even more unlikely to have
cliques of size k0 + 1 + m as m grows. t
u
Remark 1.4. To summarize, we used together the linearity
of expectation and Chebyshev’s inequality to show that the
longest increasing subsequencep in a random permutation
is typically not bigger than 3 n and the clique number
w(G(n, p)) of the Erdős–Rényi random graph is typically
2 log n
not bigger than log(1/p) . In both cases, we were counting the
number of subsets in a certain configuration and, as the size
of subsets got bigger, there was a competition between the
large number of such subsets and the small probability that
the subset appears in a special configuration. Eventually, the
small probability dominated, which allowed us to conclude
that typically there are no such large subsets. t
u

Exercise 1.5.2. Give an example of integer valued random

variable N 0 such that P(N > 0) = e 10 and EN = e10 .
70 1 Introduction

Exercise 1.5.3. Suppose there were n pairs of animals in

Noah’s ark and m animals died. Compute the expectation of
the number of complete pairs left.
Exercise 1.5.4. Let us say that four vertices {v1 , v2 , v3 , v4 } in
a graph G form a square if there are exactly 4 edges present
among 6 possible edges between them and these edges form
a cycle. Compute the expected number of squares in the
Erdős–Rényi random graph G(n, p).
Exercise 1.5.5. Six geese are flying overhead. Three hunters
each picks one at random, kills it with probability 0.3 and
misses with probability 0.7. What is the expected number of
geese killed.
Exercise 1.5.6. (Matching problem) After a party on a rainy
evening, n gentlemen are not in a position to recognize their
umbrellas and everyone takes one at random. What is the
probability that at least one takes his own umbrella? Hint:
use the inclusion-exclusion principle.
Exercise 1.5.7. What is the expected number of cycles in a
random permutation of {1, . . . , n}? Hint: write N = Âni=1 X1i ,
where Xi = length of the cycle including i.
Exercise 1.5.8. In a permutation (p1 , . . . , pn ) of {1, . . . , n},
we say that pk is a record if pk > pi for i = 1, . . . , k 1. What
is the expected number of records in a random permutation?
Chapter 2
Second Moment Calculations

So far, we have only considered discrete probability spaces

and distributions, and in the next two chapters we implicitly
assume that all distributions are discrete. However, many of
the results in these chapters rely only on basic properties of
probability and also apply to non-discrete probability spaces.
In Chapter 4, we will introduce and study some continuous
distributions, and it will be clear that many general results in
Chapter 2 and Chapter 3 still apply.

2.1 Variance and Covariance

For k 1, the expectation EX k is called the kth moment of X,

which is defined if E|X|k < •. In particular, EX 2 is called the
second moment, and all results in this chapter will be based
on the calculations of second moments of various random
variables. A quantity related to the second moment EX 2 is
the variance of X,

Var(X) = E(X EX)2 , (2.1)

which is the second momentp of the random variable X EX.

The square root of variance, Var(X), is called the standard
deviation.

71
72 2 Second Moment Calculations

Subtracting the expected value EX from a random variable

X is called centering, because the expectation of X EX is
equal to zero,

E(X EX) = EX EX = 0.

Any random variable whose expectation is zero, EX = 0, is

called centred.
If we denote µ = EX, we can rewrite the variance as

Var(X) = E(X µ)2 = E(X 2 2µX + µ 2 )

= EX 2 2µ 2 + µ 2 = EX 2 µ 2 = EX 2 (EX)2 ,

so the variance can be computed in two ways,

Var(X) = E(X EX)2 = EX 2 (EX)2 . (2.2)

Of course, we need to assume that both EX 2 and EX are

well defined. In fact, it is enough to assume that EX 2 < •,
because it implies that E|X| < •. Indeed,

|X| = |X| I(|X| < 1) + |X| I(|X| 1)  1 + X 2 ,

because |X|  X 2 when |X| 1, and taking expectations of

both sides gives E|X| < •. Another way to see this is using
the following inequality that will be useful to us later on.
Lemma 2.1 (Jensen’s inequality). If f : R ! R is convex
and EX is well-defined then

f (EX)  E f (X). (2.3)

If f is concave, the inequality is reversed.

Proof. Let µ = EX. Since f (x) is convex, its tangent line at
x = µ (or subdifferential line) is below f (x), so

f (x) f (µ) + f 0 (µ)(x µ).

Plugging in the random variable X gives

 2.1 Variance and Covariance 73

f (X) f (µ) f 0 (µ)(X µ) 0.

Therefore, taking expectations,

E f (X) f (µ) f 0 (µ)(EX µ) = E f (X) f (µ) 0,

which is exactly (2.3). It is possible that this expectation is

undefined, but, because we are taking the expectation of a
non-negative random variable, the only way it is undefined
is if E f (X) = +•. In this case, (2.3) also holds. The proof
for concave f is similar. t
u
Note that the above proof works for functions f defined on
any interval, as long as X takes values on the same interval.
For example, for nonnegative X 0, we could consider a
convex of concave function f on [0, •).
Example 2.1.1. Suppose that EX 2p< •. Take Y = X 2 , so that
EY is well-defined. Since f (x) = x is concave on [0, •),
p p
EY E Y .
p
In other words, EX 2 E|X|. This is another way to see
that EX 2 < • implies that E|X| < •. t
u
We leave some simple examples of computing variance as
an exercise.
Exercise 2.1.1. Check that:
(a) If X ⇠ B(p) then Var(X) = p(1 p).
(b) If X ⇠ Poiss(l ) then Var(X) = l .
Hint: compute EX(X 1) first.
(c) If EX 2 < • then Var(aX + b) = a2 Var X for all a, b 2 R.
Another quantity related to the second moment is called
covariance and it involves two random variables. Given two
random variables X and Y on the same probability space,
their covariance is defined by

Cov(X,Y ) = E(X EX)(Y EY ) = EXY EXEY, (2.4)

74 2 Second Moment Calculations

where the last equality can be checked by multiplying out

(X EX)(Y EY ) = XY (EX)Y (EY )X + (EX)(EY )

and taking the expectation of both sides. Of course, in the

definition of covariance we assume that all the expectations
are well defined, namely, E|XY |, E|X|, E|Y | < •. One way to
ensure this is to assume that the second moments are finite,
EX 2 , EY 2 < •, because
1
|XY |  (X 2 +Y 2 ).
2
In this case, we also have the following.
Lemma 2.2 (Cauchy-Schwarz inequality). For any random
variables X and Y on the same probability space,
1/2 1/2
E|XY |  EX 2 EY 2 . (2.5)

Proof. The proof follows by a standard argument. If we let

a = EX 2 , b = EXY, and c = EY 2 , then, for all t 2 R,

0  E(tX Y )2 = at 2 2bt + c.

Quadratic function is nonnegative if it has at most one root,

so the discriminant D = 4b2 4ac  0. Therefore, b2  ac
and |EXY |2  EX 2 EY 2 . Using this for |X| and |Y | instead of
X and Y proves (2.5). t
u
Example 2.1.2 (Uncorrelated random variables). If two
random variables X and Y are independent and their expec-
tations are well defined, E|X|, E|Y | < •, then

Cov(X,Y ) = 0. (2.6)

This follows from Lemma 1.6 in Section 1.4. Of course, the

random variables X and Y do not have to be independent for
(2.6) to hold, and random variables with zero covariance are
called uncorrelated.
2.1 Variance and Covariance 75

For example, consider two independent random variables

e and Z such that
1
P(e = 1) = P(e = 1) = ,
2
1
P(Z = ±2) = P(Z = ±3) = ,
4
and let X = Z,Y = eZ. Then

EX = EZ = 0, EY = EeEZ = 0,

and
Cov(X,Y ) = EXY = EeZ 2 = EeEZ 2 = 0,
so X and Y are uncorrelated. But they are not independent
because |X| = |Y |, so knowing the value of one determines
the other one up to a sign, while independence means that
the chances of all outcomes should not be affected. t
u
Example 2.1.3 (Variance of the sum). Consider random
variables X1 , . . . , Xn on the same probability space such that
all EXi2 < •. If Sn = Âni=1 Xi then multiplying out,
⇣ n ⌘2
2
Sn ESn = Â (Xi EXi )
i=1
n
= Â (Xi EXi )(X j EX j ),
i, j=1

and taking expectations on both sides gives

n
Var(Sn ) = Â Cov(Xi , X j ). (2.7)
i, j=1

When i = j, Cov(Xi , Xi ) = Var(Xi ) and, when i 6= j, the terms

(i, j) and ( j, i) are the same, so covariance of the sum can be
also written as
76 2 Second Moment Calculations

n
Var(Sn ) = Â Var(Xi ) + 2 Â Cov(Xi , X j ). (2.8)
i=1 i< j

If the random variables X1 , . . . , Xn are uncorrelated then

n
Var(Sn ) = Â Var(Xi ). (2.9)
i=1

In particular, by (2.6), this holds when they are independent.

Moreover, if X1 , . . . , Xn are i.i.d. then

Var(Sn ) = n Var(X1 ), (2.10)

because all their variances are equal. t

u
Example 2.1.4 (Variance of the Binomial). The variance of
the binomial random variable X ⇠ B(n, p) can be computed
by definition, but it is much easier if we recall that the sum
of n i.i.d. Bernoulli B(p) random variables is binomial, so

Var(X) = np(1 p),

by (2.10) and Exercise 2.1.1 (a). t

u
Exercise 2.1.2. Show that | Cov(X,Y )|  [Var(X) Var(Y )]1/2 .
Exercise 2.1.3. Let G = G(n, p) be the Erdős-Rényi random
graph. Let N be the number of vertices among {v3 , . . . , vn }
that are connected by edges to both v1 and v2 . Compute
Var(N).
Exercise 2.1.4. Let e1 , . . . , en be i.i.d. random variables such
that P(ei = ±1) = 12 . Let X = Âi< j ei e j . Compute Var(X).
Exercise 2.1.5. Compute variance of the geometric random
variable N with the distribution P(N = k) = (1 p)k 1 p for
k 1.
Exercise 2.1.6. Suppose that the random variables X1 and X2
are independent, EX j = µ j and Var(X j ) = s 2j for j = 1, 2.
What is the variance of X1 X2 ? Is it well-defined?
2.1 Variance and Covariance 77

Exercise 2.1.7. If E|X| p < • for p > 0, show that E|X|q < •
for 0  q  p.
Exercise 2.1.8. Suppose that a random variable X takes four
values 2, 1, 1, 2 with equal probabilities 14 and let Y = X 2 .
Show that X and Y are uncorrelated but not independent.
Exercise 2.1.9. Suppose that the random variables X1 , . . . , Xn
are i.i.d. and P(Xi = ±1) = 12 . For each i, let us pick an index
e(i) 2 {1, . . . , n} \ {i} uniformly at random, independently
over i  n, and independently of X1 , . . . , Xn . Compute the
variance of Xe(1) + . . . + Xe(n) .
78 2 Second Moment Calculations

2.2 Classical Law of Large Numbers

When we flip a fair coin 100 times, we expect the number

of Heads to be close to 50 or, in other words, the proportion
of Heads to be close to its probability 0.5. In this section,
we will make a quantitative mathematical statement of this
kind, called the Law of Large Numbers. In the next chapter,
we will strengthen it using more sophisticated methods, but
a simple argument based on the second moment calculations
that we will use here is still very useful in many situations
where ‘simple’ calculations is all one can do.
Let us recall Chebyshev’s inequality in Theorem 1.4 in
Section 1.5. A straightforward consequence is the following
inequality, which is also called Chebyshev’s inequality.
Theorem 2.1 (Chebyshev’s inequality II). If X has finite
variance then
Var(X)
P |X EX| x  , (2.11)
x2
for any x > 0.
Proof. Consider Y = (X EX)2 , which is a non-negative
random variable. Then |X EX| x if and only if Y x2
and, by Theorem 1.4,
EY Var(X)
P Y x2  = .
x2 x2
This finishes the proof. t
u
By definition, variance Var(X) measures deviations of a
random variable X from its expectation EX by computing the
expectation of the square of this deviation (X EX)2 . Small
variance indicates that X is typically close to its expected
value EX, and the inequality (2.11) gives precise meaning
to this statement. Namely, the probability that this deviation
exceeds x > 0 is smaller than Var(X)/x2 . To state our first
2.2 Classical Law of Large Numbers 79

application, given random variables X1 , . . . , Xn , let us denote

their average by
1 n
X n = Â Xi . (2.12)
n i=1
Theorem 2.2 (Law of large numbers). If random variables
X1 , . . . , Xn are i.i.d., µ = EX1 and s 2 = Var(X1 ) < • then,
for any e > 0,
s2
P |X n µ| e  . (2.13)
ne 2
Proof. By the properties of variance proved in the previous
section,
1 n n Var(X1 ) s 2
Var(X n ) = 2 Â Var(Xi ) = = .
n i=1 n2 n

Then Chebyshev’s inequality (2.11) with X = X n and x = e

becomes exactly (2.13) because EX n = EX1 = µ. t
u
Because s 2 /ne 2 ! 0 as n ! •, Chebyshev’s inequality
(2.11) implies that the average X n differs from its expectation
µ by more than an arbitrarily small e > 0 only with small
probability when n gets large. For example, the variance of
a fair coin B(1/2) equals s 2 = 1/4 and the expectation is
µ = 0.5. Therefore,
1
P |X n 0.5| e  . (2.14)
4ne 2
For example, the probability that the proportion of Heads dif-
fers from 0.5 by more than, say e = 0.01, is bounded by
2500
P |X n 0.5| 0.01  .
n
Later on we will prove quantitatively stronger inequalities,
but qualitatively this statement is already quite useful as we
will see in the next section. Also, because the calculations
80 2 Second Moment Calculations

in the above proof of the Law of Large Numbers were quite

simple, assumptions can be easily relaxed, which we leave
as an exercise below.
Remark 2.1. Law of large numbers if the first manifestation
of a very general idea in Probability and Analysis called the
“concentration of measure phenomenon”. In the law of large
numbers, the average function
x1 + . . . + xn
f (x1 , . . . , xn ) =
n
was shown to concentrate near a constant (its expectation),
under certain assumptions on the coordinates, for example,
independence. It turns out that many other functions with
large number of variables exhibit this behaviour. Later we
will see other examples of interesting non-linear functions
that also concentrate.

Exercise 2.2.1. Suppose that random variables X1 , . . . , Xn are

uncorrelated, µ = EX n and Var(Xi )  s 2 for all i  n. Then
show (2.13) still holds.
Exercise 2.2.2. In the setting of the previous exercise, show
that, for any d > 0,
⇣ r ⌘
s 1
P |X n µ| p  d.
n d
Exercise 2.2.3. Suppose that Xk for k 1 are independent,
but not identically distributed, random variables such that
1 1
P(Xk = ± k) = , P(Xk = 0) = 1 .
2k log(2k) k log(2k)

Show that limn!• P(|X n | e) = 0 for any e > 0, where X n

is the average 1n Ânk=1 Xk .
Exercise 2.2.4. If X ⇠ Poiss(l ), show that, for any e > 0,
2.2 Classical Law of Large Numbers 81
⇣ X ⌘ 1
P 1 e  .
l l e2
Exercise 2.2.5. (Law of Large Numbers for U-statistics) Let
X1 , . . . , Xn be i.i.d. such that EX1 = µ and s 2 = Var(X1 ) < •,
and consider the random variable
✓ ◆ 1
n
U :=
2 Â XiX j .
1i< jn

Prove an explicit bound on P(|U µ 2| e) and show that it

goes to zero as n ! •.
Exercise 2.2.6. Suppose that j : R ! R is a strictly positive
nondecreasing function, and suppose that j(X) is integrable.
Prove that, for any x 2 R,
Ej(X)
P(X x)  .
j(x)
Exercise 2.2.7. Suppose that the random variables X1 , . . . ,
Xn are i.i.d. and E|X1 | p < • for some p > 0. Show that

lim P max |Xi | en1/p = 0

n!• in

for any e > 0. Hint: Use the union bound, then Chebyshev’s
inequality in the form (1.98), and then Exercise 1.2.13.
82 2 Second Moment Calculations

2.3 Bernstein Polynomials

The Weierstrass approximation theorem states that any con-

tinuous function on a closed interval [a, b] can be approx-
imated uniformly by polynomials. In this section, we will
use the Law of Large Numbers from the previous section to
give one explicit construction of approximating polynomials
called Bernstein’s polynomials. One can make a change of
variables to scale the interval [a, b] to [0, 1], so we will only
consider the interval [0, 1].
If f : [0, 1] ! R is a continuous function on [0, 1] then the
Bernstein polynomial of order n associated to this function
is defined by
n ⇣ k ⌘✓n◆
Bn (x) := Â f xk (1 x)n k . (2.15)
k=0 n k

Our goal will be to show that Bn (x) uniformly approximates

f (x) on the interval [0, 1].
Theorem 2.3. If f : [0, 1] ! R is continuous then

lim max f (x) Bn (x) = 0. (2.16)

n!• x2[0,1]

Proof. Given p 2 [0, 1], let X1 , . . . , Xn be i.i.d. Bernoulli B(p)

random variables and let
Sn
Sn = X1 + . . . + Xn and X n = .
n
Since the sum Sn has binomial distribution B(n, p),
⇣S ⌘ n ⇣ k ⌘✓n◆
=Âf
n
E f (X n ) = E f pk (1 p)n k
n k=0 n k

is Bernstein’s polynomial Bn (x) evaluated at x = p. The

statement of the theorem is that this expectation is close to
f (p) uniformly over p 2 [0, 1]. First,
2.3 Bernstein Polynomials 83

Bn (p) f (p) = E f (X n ) f (p)

⇥ ⇤
= E f (X n ) f (p)  E f (X n ) f (p) ,

because |EX|  E|X|, which follows from the monotonicity

property of expectation,

|X|  X  |X| =) E|X|  EX  E|X|,

or from Jensen’s inequality (2.3). Take any e > 0 and write

f (X n ) f (p) = f (X n ) f (p) I |X n p|  e
+ f (X n ) f (p) I |X n p| > e .

Since |X n p|  e in the first indicator, we can bound the

first term by the modulus of continuity of f ,

D(e) := max f (x) f (y) .

|x y|e

For the second term, we use that a continuous function f on

[0, 1] is bounded by some constant, | f |  C, so

f (X n ) f (p) I |X n p| > e  2C I |X n p| > e .

Putting two bounds together,

f (X n ) f (p)  D(e) + 2C I |X n p| > e ,

and taking expectations on both sides,

E f (X n ) f (p)  D(e) + 2CP |X n p| > e .

The probability in the last term is where we finally use

Chebyshev’s inequality (2.13) and, since the variance of
B(p) is p(1 p)  1/4, we showed that
C
Bn (p) f (p)  D(e) + . (2.17)
2ne 2
84 2 Second Moment Calculations

This upper bound does not depend on p, so maximizing over

p on the left hand side and then taking the limit,

lim max f (p) Bn (p)  D(e).

n!• p2[0,1]

A continuous function on [0, 1] is uniformly continuous, so

the modulus of continuity D(e) # 0 as e # 0. Since the upper
bound was proved for any e > 0, this finishes the proof. tu

Exercise 2.3.1. If f (x) = x2 on [0, 1], compute its Bernstein

polynomials Bn (x) and show that
1
|Bn (x) x2 |  .
4n
Exercise 2.3.2. Suppose that a differentiable function f on
[0, 1] is bounded by some constant, | f |  C, and its derivative
is also bounded, | f 0 |  D. Show that

3D2/3C1/3
max Bn (x) f (x)  .
x2[0,1] 2n1/3
Hint: start with (2.17).
Exercise 2.3.3. (Multivariate Bernstein polynomials) Con-
sider a continuous f : [0, 1]m ! R and show that
⇣k ✓ ◆
km ⌘ n ki
 ’
1
f , . . . , xi (1 xi )n ki
0k1 ,...,km n n n im ki
! f (x1 , . . . , xm )

as n ! •, uniformly on [0, 1]m . Hint: consider independent

Binomial random variables B(n, xi ), or find a way to use in-
duction.
Exercise 2.3.4. Suppose that f : [0, •) ! R is continuous
and uniformly bounded, and define
2.3 Bernstein Polynomials 85

• ⇣ k ⌘ (nx)k
Pn (x) := Âf n k!
e nx
.
k=0

Show that
lim max f (x) Pn (x) = 0
n!• x2[a,b]

for any finite interval 0  a < b < •. Hint: In the proof of

(2.16), take X1 , . . . , Xn to be independent Poisson P(l ) in-
stead of Bernoulli B(p).
86 2 Second Moment Calculations

2.4 Cliques in the Erdős–Rényi Random Graph

In this section we will continue discussing the example of

cliques in the Erdős–Rényi graph from Section 1.5. The main
purpose of this section is to illustrate that such a simple tool
as Chebyshev’s inequality can still be quite useful when a
random variable is complicated but computing or estimating
its variance is doable, even if it requires some effort. When
things get complicated, simple things is all one can do.
Let us recall what we did in Section 1.5. We considered the
number Nk of cliques of size k in the Erdős–Rényi random
graph G(n, p). We showed that its expectation is
✓ ◆
n (k )
f (k) = ENk = p2 . (2.18)
k
Then we considered the unique transition point k0 such that

f (k0 ) 1 > f (k0 + 1), (2.19)

which means that at k = k0 there is a transition when the

expected number of cliques of size k > k0 becomes smaller
than 1. We showed that, for m 1,
1
P Nk0 +m+1 > 0  e)
, (2.20)
nm(1
which means that it is very unlikely to have cliques of size
k0 + 2 and bigger. We also analyzed the formula for f (k) and
showed that
2 log n
k0 = k0 (n) ⇠ (2.21)
log(1/p)
in the sense that their ratio goes to 1 when n gets large.
In this section, we would like to understand what happens
for k < k0 . First, we will see from the calculations we have
already done in Section 1.5 that the expected number of such
cliques is large. Then we will carry out the second moment
calculation and use Chebyshev’s inequality to see that Nk
2.4 Cliques in the Erdős–Rényi Random Graph 87

concentrates, similarly to the law of large numbers, which

means that the number of such cliques is large not only on
average but typically (with probability close to 1).
First, let us look at the expected number f (k) of cliques of
size k for k < k0 . More precisely, let us take small e > 0 and
consider k in the range
(2 e) log n
 k < k0 . (2.22)
log(1/p)
Exponentiating the first inequality, it is equivalent to
1
pk  e
,
n2
which we have already seen in (1.113). If we recall the for-
mula (1.107) for the ratio of two consecutive expectations,
for such k, this ratio is small:
f (k + 1) n k k 1
= p  npk  1 e .
f (k) k+1 n

Since f (k0 ) 1, this implies that f (k0 1) n1 e , and

f (k0 2) n2(1 e) , and, by induction, for any fixed m 1,
e)
f (k0 m) nm(1 . (2.23)

This shows that the expected number of cliques of size k in

the range (2.22) is large. Next, we will compute the variance
of Nk and apply Chebyshev’s inequality.
Remark 2.2. A common way to use Chebyshev’s inequality
(2.11) when the expectation EX > 0 is positive is to take
x = d EX for small d > 0, so
Var(X)
P |X EX| d EX  . (2.24)
d 2 (EX)2

If the variance is much smaller that (EX)2 ,

88 2 Second Moment Calculations

Var(X) ⌧ (EX)2 , (2.25)

then the probability in (2.24) is small. Since we can rewrite

the opposite inequality |X EX| < d EX as
X
1 d  1+d,
EX
the probability of the complement of (2.24),
⇣ X ⌘ Var(X)
P 1 d  1+d 1 , (2.26)
EX d 2 (EX)2
is close to 1. Especially when EX is large, this tells us that the
random variable X is close to its expectation when viewed on
the right scale, because typical deviations of X from EX are
small relative to EX. This concentration around expectation,
assuming (2.25) holds, can be viewed as an analogue of the
law of large numbers. t
u
We already know that f (k) = ENk is large for k in the
range (2.22). If we can show

Var(Nk ) ⌧ (ENk )2 = f (k)2

then we can conclude that Nk is also typically large, by

(2.26). Let

Vk = W ✓ V : card(W ) = k

be the collection of all subsets of vertices W of size k, and let

us represent

Nk = Â IW , where IW := I(W is a clique). (2.27)

W 2Vk

In Section 2.1 we saw that the variance of the sum can be

written as
2.4 Cliques in the Erdős–Rényi Random Graph 89

Var(Nk ) = Â0 Cov(IW , IW 0 ). (2.28)

W,W 2Vk

If the subsets W and W 0 do not intersect or only share one

vertex then they do not have any common edges and, hence,
the indicators IW , IW 0 are functions of disjoint sets of edges.
Since all edges are independent of each other in the Erdős–
Rényi random graph, by the Grouping Lemma in Section 1.4,
IW and IW 0 are independent and, therefore, their covariance
is equal to zero. This means that in (2.28) we only need to
sum over pairs (W,W 0 ) such i = card(W \W 0 ) 2. For such
pairs, we will use that

Cov(IW , IW 0 ) = E IW IW 0 E IW E IW 0
 E IW IW 0 = P(W, W 0 are cliques).

The probability that W and W 0 are both cliques depends on

how many vertices they have in common. If the cardinality
i = card(W \W 0 ) 2 then
k k i
P(W, W 0 are cliques) = p(2) p(2) (2) ,

because 2k edges that must be present in W include 2i

edges in W 0 , which leaves 2k i
2 additional edges in
W 0 . Next, how many pairs W,W 0 2 Vk are there such that
i = card(W \W 0 ) 2? There are
✓ ◆✓ ◆✓ ◆
n k n k
k i k i

such pairs, because there are nk ways to select vertices in

W , then there are ki ways to select i vertices in W that will
be shared with W 0 , and then there are nk ki ways to select
k i vertices in W 0 that are not shared and are chosen from
n k remaining vertices. Summing over intersection sizes
i = card(W \W 0 ) 2 in (2.28),
90 2 Second Moment Calculations
✓ ◆✓ ◆✓
k ◆
n k n k (k ) (k ) ( i )
Var(Nk )  Â p2 p2 2
i=2 k i k i
k k n k
i
= f (k) 2
 i
n
k i
p (2) , (2.29)
i=2 k

where we used the formula (2.18). If we denote the terms in

the last sum by
k n k
i
i k i
a(i) := n p ( 2) (2.30)
k

then Chebyshev’s inequality (2.24) implies

1 k
P |Nk ENk | d ENk  2 Â a(i), (2.31)
d i=2

and it remains to show that the sum on the right hand side is
small for k in the range (2.22).
The rest of the calculation is just a tedious analysis of the
sequence a(i) and does not really contain any clever idea. It
simply requires experimentation and time. We will proceed
in two steps:
1. We will check that endpoints a(2) and a(k) are both small.
2. We will check that other a(i) are dominated by a(2) and
a(k).
Step 1. Cancelling out common factors in the factorials,
k n k
2
2 k 2
a(2) = n p (2)
k
k2 (k 1)2 (n
2k + 3) · · · (n k) 1
= · ·
2 (n k + 1) · · · n p
4
k (n k)k 2 1 k 4 1 1 (log n)4
 · · = · ·  cp ,
2 (n k)k p 2 (n k)2 p n2
 2.4 Cliques in the Erdős–Rényi Random Graph 91

for some constant c p that depends on p, because k is of order

O(log n) in the range (2.22). This shows that a(2) is small.
Next,
1 k 1
a(k) = n p (2) = ,
k
f (k)
which, as we checked in (2.23), is also small in the range
(2.22). t
u
Step 2. When analyzing the binomial coefficients and fac-
torials, it is often helpful to study the ratio of two neighbours
to understand their behaviour. This is what we will do here.
For 2  i  k 1, let us consider the ratio of two consecutive
numbers a(i),

a(i + 1) (k i)2
b(i) := = p i. (2.32)
a(i) (i + 1)(n 2k + i + 1)
This looks much simpler than the original sequence a(i) and
we can derive some of its basic properties.
Lemma 2.3. Suppose that k is in the range (2.22) and take
2  i  k 1. Then the following statements hold.
1 log n
(a) For i  3 log(1/p) the sequence b(i) < 1.
log n
(b) For i 32 log(1/p) the sequence b(i) > 1.
1 log n 3 log n
(c) For 3 log(1/p) < i < 2 log(1/p) the sequence b(i) is strictly
increasing.
Proof. (a) First of all, in the range (2.22), n 2k +i+1 n/2
and
2k2 i (log n)2 i
b(i)  p  ap p , (2.33)
n n
for some constant a p . Since

i 1 log n
p  n1/3 () i  , (2.34)
3 log(1/p)
in this range of i we get
92 2 Second Moment Calculations

(log n)2
b(i)  a p 2/3 < 1.
n
(b) Next, for 2  i  k 1, we can bound b(i) from below by

(k i)2 i 1 i
b(i) = p p .
(i + 1)(n 2k + i + 1) kn
Since
i 3 log n
p n3/2 () i , (2.35)
2 log(1/p)
in this range of i and for k in the range (2.22) we get

n1/2
b(i) > 1. (2.36)
k
(c) In the intermediate range of i in between (2.34) and
(2.35),
1 log n 3 log n
<i< , (2.37)
3 log(1/p) 2 log(1/p)
and for k in the range (2.22), both i and k i are of order
O(log n). Therefore,
b(i + 1) 1
⇡ > 1,
b(i) p
so in this range the sequence b(i) is strictly increasing. t
u
These three properties imply that there is a unique i0 such
that b(i)  1 for i < i0 and b(i) > 1 for i i0 . This implies
that a(i)  a(3) for i  i0 and a(i)  a(k 1) for i > i0 . As
a result,
k
 a(i)  a(2) + a(k) + k a(3) + a(k 1) .
i=2

Using (2.33) one more time, we see that

2.4 Cliques in the Erdős–Rényi Random Graph 93

a p (log n)2
a(3)  2 a(2),
p n
and, using (2.36), we see that
k k0 log n
a(k 1)  p a(k)  p a(k)  b p p a(k),
n n n
for some constant b p . Finally, adding up these inequalities
and using that k = O(log n),
k ⇣ (log n)2 ⌘
 a(i)  a(2) + a(k) 1 + d p pn  2 a(2) + a(k) .
i=2

This finishes Step 2. t

u
Combining the two steps, we can summarize what we
proved as follows.
Theorem 2.4. Let e, d > 0. For k in the range
(2 e) log n 2 log n
 k < k0 ⇠ , (2.38)
log(1/p) log(1/p)
where k0 was defined in (2.19), the number Nk of cliques of
size k satisfies
⇣ ⌘ 2 ⇣ (log n)4 1 ⌘
P |Nk f (k)| d f (k)  2 c p + . (2.39)
d n2 f (k)

Since we saw in (2.23) that f (k0 m) nm(1 e) , the right

hand side goes to zero as n goes to infinity. As we mentioned
before, this means that, with probability close to 1, the ratio
Nk / f (k) is close to 1, so the number of cliques of size k is
typically large, just as its expectation f (k). Together with
(2.20), this shows that there is a transition occurring at k = k0
where the typical number of cliques goes from being large
for smaller k to zero for larger k.
94 2 Second Moment Calculations

Exercise 2.4.1. Let w(G) be the size of the largest clique in

the graph G. Show that
2 log n
Ew(G(n, p)) ⇠ .
log(1/p)
Exercise 2.4.2. A subset W of vertices in a graph is called an
independent set if there are no edges between any vertices in
W . What can you say about the typical size of the largest
independent set in the Erdős–Rényi random graph G(n, p)
for large n? Hint: how do independent sets relate to cliques
if we switch present and absent edges?
Exercise 2.4.3. Suppose there were n pairs of animals in
Noah’s ark and m animals died. Compute the variance of the
number of complete pairs of animals left.
Exercise 2.4.4. Compute the variance of the number N3 of
triangles in the Erdős–Rényi random graph.
Exercise 2.4.5. (Matching problem) After a party on a rainy
evening, n gentlemen are not in a position to recognize their
umbrellas and everyone takes one at random. What is the
expectation and variance of the number of correct umbrellas
taken?
Exercise 2.4.6. If EX 2 < •, show that
Var(|X|)
P(X = 0)  .
E(X 2 )
Hint: rewrite this in a way that follows from the Cauchy-
Schwarz inequality.
2.5 Hardy–Ramanujan Theorem 95

2.5 Hardy–Ramanujan Theorem

In this section we will give an example of the second moment

calculation from Number Theory. For integer n 1, let w(n)
be the number of distinct prime factors of n,

w(n) = card p  n : p is prime, p | n . (2.40)

Hardy–Ramanujan theorem states the following.

Theorem 2.5 (Hardy–Ramanujan). For any sequence y(n)
such that y(n) ! • as n ! •, the proportion of numbers
N 2 {1, . . . , n} that satisfy
p
w(N) log log n  y(n) log log n (2.41)

goes to 1.
In other words, one can take y(n) that goes to infinity arbi-
trarily slowly, and for most natural numbers N in between 1
and n, the number w(N) of distinct prime divisors of N is
close to log
plog n in the sense that their difference is smaller
than y(n) log log n, which is relatively small compared to
log log n. Of course, log log n itself grows very slowly, so the
effect becomes noticeable only for very large n.
The proof of the Hardy–Ramanujan theorem will rely on
Chebyshev’s inequality and the following result that we will
prove below.
Theorem 2.6 (Mertens’ first and second theorem). The
following holds as n ! •,
log p
 p
= log n + O(1) , (2.42)
pn
1
 p = log log n + O(1) , (2.43)
pn

where the sum is over prime numbers p  n.

96 2 Second Moment Calculations

First, let us prove the Hardy–Ramanujan theorem.

Proof (Theorem 2.5). Let N be a uniform random variable
on the set {1, . . . , n},
1
P(N = k) = for k = 1, . . . , n.
n
Then, what we want to prove is that
⇣ p ⌘
P w(N) log log n > y(n) log log n ! 0.

As usual, we will represent w(N), which counts the number

of prime divisors of N, as the sum of indicators

w(N) = Â I(p | N),

pn

where the sum is over pprime numbers p  n.

First of all, let m = n and consider a modified sum

wm (N) = Â I(p | N),

pm
p
where we do not count prime divisor(s) of N bigger that n.
There could be no more than one such prime divisor because,
if there were two, their product would exceed n. This means
that |wm (N) w(N)|  1 and it is enough to prove that
⇣ p ⌘
P wm (N) log log n > y(n) log log n ! 0.

Expectation of one indicator I(p | N) in the above sum is

1 n 1j n k
E I(p | N) = Â I(p | k) = ,
n k=1 n p

because the number of k  n divisible by p equals b np c.

Therefore,
2.5 Hardy–Ramanujan Theorem 97

1j n k
Ewm (N) = Â E I(p | N) = Â .
pm pm n p

Since b np c  n
p  b np c + 1, the last sum

1j n k 1
 n p =  p + O(1) .
pm pm

(Or even O (1).) Using the second Mertens’ theorem (2.43),

1
Ewm (N) = Âp
+ O(1) (2.44)
pm
p
= log log n + O(1) = log log n + O(1) .

Next, let us compute the variance of wm (N). Using the for-

mula (2.8) in Section 2.1 for the variance of the sum in terms
of the sum of covariances,
⇣ ⌘
Var wm (N) = Â Var I(p | N) (2.45)
pm
⇣ ⌘
+ Â Cov I(p | N), I(q | N) .
1p6=qm

First of all, since the indicator I(p | N) is a Bernoulli B(x)

random variable with
✓ ◆
1j n k 1 1
x = E I(p | N) = = +O ,
n p p n
and the variance of Bernoulli B(x) is x(1 x), the variance
of this indicator is
⇣ ⌘ 1⇣ ✓ ◆
1⌘ 1
Var I(p | N) = 1 +O .
p p n
Then the sum of variances is
98 2 Second Moment Calculations
⇣ ⌘ 1 1
 Var I(p | N) =   2
+ O(1)
pm pm p pm p
= log log n + O(1) , (2.46)

where for the first sum we again used the second Mertens’
theorem (2.43), and the second sum is bounded by
•
1
 j2
< •.
j=1

Next, let us compute the covariances in (2.45),

⇣ ⌘
Cov I(p | N), I(q | N)
= E I p | N, q | N E I(p | N)E I(q | N).

The last two expectations we already computed, and the first

expectation is
1 n 1j n k
E I p | N, q | N = Â I p | k, q | k = ,
n k=1 n pq

because the number of k  n divisible by both prime numbers

p and q equals to the number of k  n divisible by pq, which
n
equals b pq c. Therefore,
⇣ ⌘ 1j n k 1j n k 1jnk
Cov I(p | N), I(q | N) = ·
n pq n p n q
1 n 1⇣ n ⌘ 1⇣n ⌘
 1 · 1
n pq n p n q
1⇣ 1 1⌘ 1 1⇣ 1 1⌘
= +  + .
n p q n2 n p q
This shows that the events that a random integer N  n is
divisible by a prime p or prime q are almost uncorrelated. In
particular, the sum of covariances in (2.45) is bounded by
2.5 Hardy–Ramanujan Theorem 99
⇣ ⌘
 Cov I(p | N), I(q | N)
1p6=qm
1⇣ 1 1⌘
 Â +
1p6=qm n p q
m 1 m 1
 Âp nÂq
n pm
+
qm
2 ⇣ ⌘
= p log log n + O(1) = O (1) , (2.47)
n
where in the last line we again usedpthe second Mertens’
theorem (2.43) and the fact that m = n. Combining (2.46)
and (2.47), the variance of the sum in (2.45) is

Var wm (N) = log log n + O(1) . (2.48)

By Chebyshev’s inequality (2.11),

⇣ p ⌘
P wm (N) log log n > y(n) log log n
log log n + O(1)
 ! 0,
y 2 (n) log log n
since y(n) ! •. This finishes the proof. t
u
It remains to prove Mertens’ theorem. In the proof, we will
need the following result.
Lemma 2.4. For any integer n 1,

’ p  24n, (2.49)
pn

where the product is taken over primes p  n.

One can prove a better inequality using the prime number
theorem, but here we will use only elementary arguments.
Proof. Let us consider the following binomial coefficient
100 2 Second Moment Calculations
✓ ◆
2n (n + 1) · · · (2n)
= .
n 1···n
All the primes between n + 1  p  2n are in the numerator
and, since this binomial coefficient is integer, other factors in
the numerator will cancel with n! in the denominator, which
implies that ✓ ◆
2n
’ p n .
n+1p2n

This binomial coefficient is smaller than 22n because it is just

one term in
2n ✓ ◆
2n k 2n k
22n = (1 + 1)2n = Â 11 ,
k=0 k

so we showed that ’n+1p2n p  22n . For n = 2k , this gives

’
k+1
p  22 .
2k +1p2k+1

Multiplying these equations over k = 0, . . . , m, we get

’m+1 p  22+2 +...+2

2 m+1 m+2
 22 .
p2

Given n 1, let us take m such that 2m < n  2m+1 . Then,

this equation implies

’ p  ’m+1 p  22
m+2
 24n ,
pn p2

which finishes the proof. t

u
Finally, we can prove Mertens’ theorem.
Proof (Theorem 2.6). (Part 1) Let us consider the factorial
n! = 1 · 2 · · · (n 1) · n and let us count how many times the
factor p appears in this product, for a given prime number
p  n. First of all, each the following numbers
2.5 Hardy–Ramanujan Theorem 101
jnk
p, 2p, 3p, . . . , p
p
contains a factor p. This gives b np c factors of p, but some of
these numbers also contain at least two factors of p, that is,
they are divisible by p2 . These numbers are
jnk
p , 2p , 3p , . . . , 2 p2 .
2 2 2
p
Since the first factor of p was already counted, this gives
additional b pn2 c factors of p. However, some of these are di-
visible by p3 , giving us another b pn3 c factors of p, and if we
continue in the same fashion, the total number of factors of
p in n! is
jnk j n k j n k jnk
f (n, p) := + 2 + 3 +...+ k , (2.50)
p p p p
where pk is the last power of p less than or equal to n. If we
write n! using prime number decomposition, we showed that
n! = ’ pn p f (n,p) and, taking logarithms,

log(n!) = Â f (n, p) log p.

pn

To see where we are going, notice that by Stirling’s formula

(1.101) in Section 1.5,

log(n!) = n log n + O(n) ,

and dividing both sides by n, we get

f (n, p)
log n + O(1) = Â n
log p. (2.51)
pn

Since
f (n, p) 1 j n k 1 j n k 1j n k
= + +...+ ,
n n p n p2 n pk
102 2 Second Moment Calculations

this is starting to look like (2.42), because the first term in

this sum looks like 1/p. More precisely, the difference can
be bounded by
f (n, p) 1 1 1 1 1 2
 + 2 +...+ k  + 2,
n p n p p n p
using the geometric series formula. Adding up over p  n,
f (n, p) log p 1 log p
 n
log p    log p +  2 .
pn pn p n pn pn p

By the previous lemma, the first term

1 1 1
Â
n pn
log p = log ’ p  log 24n  4 log 2.
n n
pn

The second term is bounded by •j=1 log j/ j2 < •, so the

sum of two terms is bounded by a constant. Therefore, (2.51)
implies that
log p
log n + O(1) = Â + O(1)
pn p

and finishes the proof of the first equation (2.42).

(Part 2) The second equation (2.43) follows from the first
one. Equation (2.42) can also be written for all real x 2,
log p
 = log x + O(1) , (2.52)
px p

instead of only integer x = n, because rounding to the nearest

integer changes both sides by at most a constant. Next, we
will use that, for p  n,
Z n
dt 1 n 1 1
= = .
p t log2 t logt p log p log n
Using this, we can write
2.5 Hardy–Ramanujan Theorem 103
Z n
1 log p 1 log p ⇣ 1 dt ⌘
 p =  p · log p =  p log n + p t log2 t
pn pn pn
Z
1 log p log p n dt
= Â
log n pn p
+Â
t log2 t
,
pn p p

By (2.52), the first term is O(1) . The second term can be

rewritten as
Z n Z n
log p dt log p dt
 p I(p  t)
t log2 t
= Â I(p  t)
t log2 t
pn 2 2 pn p
Z n
log p dt
= Â 2
.
2 pt p t log t

Using (2.52) again, this equals

Z n Z n
dt dt
logt + O(1) = + O(1)
2 t log2 t 2 t logt
n
= log logt + O(1) = log log n + O(1) .
2

This finishes the proof of (2.43). t

u
Exercise 2.5.1. If we denote Ĩ(p | N) = I(p | N) E I(p | N),
compute the 3-point correlation
h i
E Ĩ(p | N) Ĩ(q | N) Ĩ(r | N)

for three distinct prime numbers p, q, and r.

Chapter 3
Exponential Inequalities

3.1 Hoeffding Inequality

Using Chebyshev’s inequality, we showed that if i.i.d. ran-

dom variables X1 , . . . , Xn have finite variance,

s 2 = EX12 < •,

then their average

Sn 1 n
Xn = = Â Xi
n n i=1

is close to the expectation p = EX1 , with high probability.

This is known as the law of large numbers. In this chapter
we will prove several quantitatively stronger results under
stronger assumptions on the random variables. More pre-
cisely, instead of only assuming finite variance we will work
with random variables such that

Eel X < • for all l 2 R. (3.1)

For example, this holds if a random variable X is bounded

by a constant. To take advantage of this, we will be using
the exponential form of Chebyshev’s inequality, known as
Markov’s inequality.

105
106 3 Exponential Inequalities

Lemma 3.1 (Markov’s inequality). For any l 0,

lt
P X t e Eel X , (3.2)

assuming that Eel X < •.

Proof. For positive l > 0, the inequality X t is equivalent
to el X elt , so
Eel X
P X t = P el X elt  ,
elt
by the usual Chebyshev’s inequality, since el X is positive. If
l = 0 then the right hand side of (3.2) is equal to 1, so the
inequality still holds. t
u
First, we will consider independent flips of a fair coin. It
will be convenient to rescale the usual values {0, 1} of the
Bernoulli random variable to { 1, 1}. In other words, we
consider i.i.d. random variables e1 , . . . , en such that
1
P en = 1 = P en = 1 = . (3.3)
2
These are usually called Rademacher random variables. The
first classical exponential inequality that we will prove is the
following.
Theorem 3.1 (Hoeffding’s inequality). For any t 0 and
any constant a1 , . . . , an 2 R,
⇣ n ⌘ ⇣ t2 ⌘
P Â eiai t  exp
2 Âni=1 a2i
(3.4)
i=1

and ⇣ n ⌘ ⇣ ⌘
t2
P Â e i ai t  2 exp
2 Âni=1 ai 2
. (3.5)
i=1

Proof. We begin by using Markov’s inequality (3.2),

3.1 Hoeffding Inequality 107
⇣ n ⌘ n
P Â eiai t e lt
Ee l Âni=1 ei ai
=e lt
’ Eel eiai ,
i=1 i=1

where in the last step we used Lemma 1.6 and (1.86). One
factor in this product equals
1 1
Eel ei ai = el ai + e l ai
= cosh(l ai ).
2 2
2 /2
Next, we will use the inequality cosh(x)  ex , which can
be seen by comparing the Taylor series,
• •
ex + e x 1 2k 1
=Â x  Â k x2k = ex /2 .
2
cosh(x) =
2 k=0 (2k)! k=0 2 k!
2 a2 /2
This implies that Eel ei ai  el i and
⇣ n ⌘ ⇣ l 2 n 2⌘
P Â eiai t  exp lt + Â ai .
2 i=1
i=1

This holds for any l 0, so we can minimize over l 0.

Taking derivative in l , the critical point is equal to
n
t
t +l  a2i = 0 () l = Ân 2
,
i=1 i=1 ai

which is nonnegative, because t 0. Plugging back this l ,

we get (3.4).
To prove (3.5), let us notice that, by symmetry, ei has the
same distribution as ei , because (3.3) implies that
1
P en = 1 = P en = 1 = . (3.6)
2
This means that we can apply (3.4) to ( ei ), which reads
⇣ n ⌘ ⇣ t2 ⌘
P Â eiai  t  exp
2 Âni=1 a2i
. (3.7)
i=1
108 3 Exponential Inequalities

Since
n n o n n o [n n o
|  ei ai | t =  eiai t  eiai  t ,
i=1 i=1 i=1

the union bound in the Exercise 1.1.1 implies (3.5). t

u
Example 3.1.1 (Law of large numbers for a fair coin). If
in the inequality (3.5) we take ai = 1n , we get that
⇣ 1 n ⌘
P Â ei
n i=1
t  2e nt 2 /2
.

Since the random variables Xi := (ei + 1)/2 take values 0 and

1 with probability 1/2 each, X1 , . . . , Xn are i.i.d. Bernoulli
B(1/2) corresponding to independent flips of a fair coin.
Since ei = 2Xi 1, we can rewrite the above inequality as
⇣ 1 n 1 t⌘
P Â Xi
n i=1 2 2
 2e nt 2 /2
.

Making the change of variables t = 2e, we get

2ne 2
P |X n 0.5| e  2e . (3.8)

Compare this with the application of Chebyshev’s inequality

in (2.14) in Section 2.2,
1
P |X n 0.5| e  . (3.9)
4ne 2
We can see that the dependence on n is dramatically better in
Hoeffding’s inequality, which is crucial for its many applica-
tions. For the sake of illustration, if we take n = 10, 000 and
e = 0.02, the two inequalities give

P |X 10,000 0.5| 0.02  0.00067 vs. 0.0625.

3.1 Hoeffding Inequality 109

When we get to the Central Limit Theorem, we will see that

the dependence on n and e in Hoeffding’s inequality is, in
fact, nearly optimal. t
u
Example 3.1.2 (Higher moments of Rademacher sums).
The following consequence of Hoeffding’s inequality will be
crucial in the application in the next section. Let us consider
a1 , . . . , an 2 R such that

a21 + . . . + a2n = 1 (3.10)

and let us consider the random variable

X := e1 a1 + . . . + en an . (3.11)

By symmetry, its odd moments are equal to zero,

2k+1
EX 2k+1 = E e1 a1 + . . . + en an =0

(prove this carefully). Its second moment is equal to

EX 2 = a21 + . . . + a2n = 1. (3.12)

Now we will see how Hoeffding’s inequality allows us to

control even moments EX 2k . By Lemma 1.3 and Exercise
1.2.5 in Section 1.2,
Z • Z •
2k 2k 1 t 2 /2
EX = 2kt P(|X| t) dt  4kt 2k 1
e dt,
0 0

where we used Hoeffding’s inequality t)  2e t 2 /2 .

p P(|X|
Making the change of variables t = 2u,
Z •
2k k+1
EX  k2 uk 1
e u
du.
0

The last integral is the Gamma function G(k) = (k 1)!,

which can be easily checked by induction on k, using in-
tegration by parts (see Section 4.4). Therefore, we showed
that, for k 1,
110 3 Exponential Inequalities

2k
EX 2k = E e1 a1 + . . . + en an  2k+1 k!, (3.13)

if the condition a21 + . . . + a2n = 1 holds.

Let us now consider the random variable
2
Y = X2 1 = e1 a1 + . . . + en an 1. (3.14)

In the next section, we will need the following consequence

of the estimate (3.13), which will be used in a very important
geometric application.
Lemma 3.2. If (3.10) holds then, for any 0  l  14 ,
2 2
EelY  e16l and Ee lY
 e16l . (3.15)

Proof. Let us start with the following simple observation: for

all x 2 R,
•
x2 (x+ )k
e  1+x+ + Â
x
, (3.16)
2 k=3 k!
where x+ = max(x, 0). If x 0 then x+ = x and the inequal-
ity is actually an equality, since the right hand side becomes
the Taylor series of ex . If x  0 then x+ = 0 and the inequal-
ity becomes ex  1 + x + x2 /2, which we leave as a simple
exercise below. Using (3.16), we can write

l2 2 • lk
EelY
 1 + l EY + EY + Â E(Y+ )k .
2 k=3 k!

First of all, EY = EX 2 1 = 0 by (3.12). Next,

EY 2 = EX 4 2EX 2 + 1 = EX 4 1  EX 4 .

Also, since Y+ = max(X 2 1, 0)  X 2 , we have E(Y+ )k 

EX 2k and, therefore,
• •
l2 lk lk
EelY  1 + EX 4 + Â EX 2k = 1 + Â EX 2k .
2 k=3 k! k=2 k!
3.1 Hoeffding Inequality 111

Here, we finally use the moment estimate (3.13),

•
l k k+1
Ee lY
 1+ Â 2 k!
k=2 k!
•
8l 2
= 1+ Â l 2 k k+1
= 1+  1 + 16l 2 .
k=2 1 2l

since this is a geometric series and 2l  12 by assumption.

To finish the proof of the first inequality in (3.15), it remains
to use that 1 + x  ex . The proof of the second inequality is
almost the same and we leave it as an exercise. t
u
Exercise 3.1.1. Estimate the probability that in 100, 000 flips
of a fair coin, the number of Heads will deviate from 50, 000
by more that 500.
Exercise 3.1.2. Show that, for k 1,
2k
E e1 a1 + . . . + en an  2k+1 k!(a21 + . . . + a2n )k

for any a1 , . . . , an 2 R.
Exercise 3.1.3. Show that

x x2
e  1+x+ for x  0.
2
Exercise 3.1.4. Prove the second inequality in (3.15).
Exercise 3.1.5 (*). Suppose that X1 , X10 , . . . , Xn , Xn0 are inde-
pendent and, for all i  n, Xi and Xi0 have the same distribu-
tion. Prove that
⇣n n ⌘
1/2
P  (Xi Xi0 ) > 2t  (Xi Xi0 )2  e t.
i=1 i=1

Hint: think about a way to introduce Rademacher random

variables ei into the problem and then use Hoeffding’s in-
equality.
112 3 Exponential Inequalities

3.2 Johnson–Lindenstrauss Lemma

In this section, we will give one classical application of the

results in the previous section. Let us consider N 1 and a
set
V = v1 , . . . , vm ✓ RN (3.17)
of m points in RN . The dimension N here can be arbitrar-
ily large, and we should think of the number of points m as
also being large. The Johnson–Lindenstrauss lemma states
that there exists a linear map from RN into Euclidean space
Rn of possibly much lower dimension n that preserves the
distances between all the points in the set V up to a small
relative error. This is called a low-distortion embedding. It
was discovered in a work on functional analysis, but it found
many applications to computational algorithms in various
fields as a preprocessing step to reduce the dimensionality
of high-dimensional data. Here is the precise statement.
Theorem 3.2 (Johnson–Lindenstrauss lemma). Given m
points V = {v1 , . . . , vm } in RN , any e 2 (0, 1), and
128
n> log m, (3.18)
e2
there exists a linear map f : RN ! Rn such that
p k f (vk ) f (v` )k p
1 e  1+e (3.19)
kvk v` k
for all 1  k < `  m, where k·k denotes the Euclidean norm.
The dimension n in (3.18) that we are allowed to choose de-
pends on the distortion parameter e and the number of points
m, but it does not depend on N. The dependence on m is log-
arithmic, so the dimension can be relatively small even when
the number of points is very large. The constant 128 is not
optimized here and can, for example, be improved to 8.
3.2 Johnson–Lindenstrauss Lemma 113

There are various proofs of the Johnson–Lindenstrauss

lemma, but most of them use probabilistic constructions. We
will consider a random matrix
2 3
e11 e12 e13 . . . e1N
6e21 e22 e23 . . . e2N 7
E =6 7
4 ... ... ... . . . ... 5 , (3.20)
en1 en2 en3 . . . enN
where the entries ei j are i.i.d. Rademacher random variables,
and show that the linear map
1
f (x) = p E x (3.21)
n
satisfies the low-distortion property (3.19) with positive prob-
ability. This means that there exists at least one matrix with
±1 entries that satisfies (3.19), if the dimension n is properly
chosen. This is another example of application the proba-
bilistic method idea that we saw in Section 1.1.
First of all, let us denote, for 1  k < `  m,
vk v`
ak` = . (3.22)
kvk v` k
Squaring (3.19), it can be rewritten for f in (3.21) as
1
1 e  kE ak` k2  1 + e (3.23)
n
for all 1  k < `  m. Since we rescaled so that the length
kak` k = 1, all the vectors ak` belong to the unit sphere in RN ,
ak` 2 SN 1 .
For the moment, let us consider an arbitrary vector

a = (a1 , . . . , aN ) 2 SN 1
,

let us denote the ith coordinate of E a by

114 3 Exponential Inequalities

Xi (a) := (E a)i = ei1 a1 + . . . + eiN aN , (3.24)

and let us denote

2
Yi (a) := Xi2 (a) 1 = ei1 a1 + . . . + eiN aN 1. (3.25)

With this notation, (3.23) can be rewritten as

1 n 2
1 e Â Xi (a)  1 + e,
n i=1
or
1 n
 Yi(a)  e.
n i=1
This looks like the law of large numbers for Y1 (a), . . . ,Yn (a),
because EYi (a) = 0, and in the last section we prepared
what will be needed to control the probability of this event.
Namely, we showed in Lemma 3.2 that, for any 0  l  14 ,
2 2
EelYi (a)  e16l and Ee lYi (a)
 e16l . (3.26)

In addition, observe that, by the Grouping Lemma 1.5 in

Section 1.4, the random variables Y1 (a), . . . ,Yn (a) are inde-
pendent, because they are functions of different rows of the
matrix E , which are all independent by construction. This
independence will be crucial in the proof below.
As a consequence of the estimate (3.26) on the exponen-
tial moments of Yi (a), we obtain the following exponential
inequality.
Lemma 3.3. For any e 2 (0, 1) and a = (a1 , . . . , aN ) 2 SN 1,

⇣ 1 n ⌘
P Â Yi(a)
n i=1
e  2e ne 2 /64
. (3.27)

Proof. In one direction, using Markov’s inequality, for any

0  l  14 ,
3.2 Johnson–Lindenstrauss Lemma 115
⇣1 n ⌘ ⇣ n ⌘
P Â
n i=1
Yi (a) e =P Â Yi(a) ne
i=1
nl e n
e Eel Âi=1 Yi (a)
n
by independence =e nl e
’ EelYi(a)
i=1
nl e+16nl 2
using (3.26) e .

Since this upper bound holds for any 0  l  14 , we can

e
minimize it over such l . Critical point is equal to l = 32
and, since it belongs to (0, 14 ), plugging it in we get
⇣1 n ⌘
P Â Yi(a)
n i=1
e e ne 2 /64
.

Exactly the same calculation with Yi (a) instead of Yi (a)

gives
⇣1 n ⌘ ⇣1 n ⌘
n nÂ
ne 2 /64
P Yi (a)  e =P ( Yi (a)) e e ,
i=1 i=1

and the claim (3.27) follows by the union bound, just like in
the proof of Hoeffding’s inequality. t
u
We are now ready to prove the Johnson–Lindenstrauss
Lemma.
Proof (Theorem 3.2). Recall that we would like to show that
there exists a matrix E such that (3.23) holds for all 1  k <
`  m or, in other words,
1 n
 Yi(ak`)  e
n i=1

holds for all 1  k < `  m. The complement of this is

116 3 Exponential Inequalities
n 1 n o
9 k < ` such that  Yi (ak` ) e
n i=1
[ n 1 n o
=
n  Yi(ak`) e .
1k<`m i=1

By the union bound in Exercise 1.1.1 and previous lemma,

the probability of this event is bounded by the sum of proba-
bilities
⇣ 1 n ⌘
 P Â
n i=1
Y i (a k` ) e  Â 2e ne 2 /64
1k<`m 1k<`m
ne 2 /64
 m2 e .

Therefore, we showed that

⇣ 1 n ⌘
P 8 1  k < `  m, Â Yi (ak` )  e 1 m2 e ne 2 /64
.
n i=1
(3.28)
2
This probability is strictly positive if m2 e ne /64 < 1, which
is the same as
128
n > 2 log m.
e
This is precisely the assumption in (3.18). The fact that the
probability is positive means that there exists E such that the
low-distortion condition (3.23) is satisfied. t
u
Remark 3.1. In the above proof, the logarithmic dependence
on the number of points came from the inequality
ne 2 /64
m2 e < 1,

which can be viewed as a competition between the large

number of pairs (of order m2 ) and the exponentially small
2
estimate e ne /64 on the probability that a given pair violates
the low-distortion condition. This exponential dependence
on n is exactly what resulted in the logarithmic dependence
on m.
3.2 Johnson–Lindenstrauss Lemma 117

By increasing the dimension n, we can make sure that the

random matrix p1n E yields a low-distortion embedding not
just with positive probability, but with high probability close
to 1. We leave this observation as an exercise.
Exercise 3.2.1. Show that, for d 2 (0, 1), if the dimension
128 m
n> log p
e2 d
then the map f (x) = p1n E x in the above proof satisfies (3.19)
with probability at least 1 d .
Suppose that we also want to control the distortion of the
areas of triangles formed by any three points vi , v j and vk .
This can be done as follows. First, the areas of all triangles in
the same (two-dimensional) plane are distorted by the same
factor under a linear transformation. For each three points,
we can add another point vi jk in the same plane, which forms
an isosceles right triangle with vi , v j . This means that, instead
of m points, we now have m + m3  m3 points on the list.
We can now find f as in Lemma 3.2 for this new enlarged list
of points. Since the dependence on m is logarithmic, we lose
a factor of 3 in the bound (3.18) for n. It remains to solve
the following exercise to see that this procedure allows us to
control the distortion of areas.
Exercise 3.2.2. Suppose that three points vi , v j and vk in the
setting of Lemma 3.2 form an isosceles right triangle D.
Show that
p Area( f (D))
1 5e   1 + e.
Area(D)
118 3 Exponential Inequalities

3.3 Hoeffding–Chernoff Inequality

The proof of Hoeffding’s inequality strongly relied on the

symmetry of the Rademacher random variables and it does
not apply, for example, to flips of a biased coin B(p) with
p 6= 1/2. In this section, we will prove another inequality,
called the Hoeffding–Chernoff Inequality, which applies to
any bounded random variables. Bounded random variables
can be rescaled to take values in [0, 1], so it is enough to
consider this case.
Let X1 , . . . , Xn be independent random variables taking
values in [0, 1], with common expectation p = EXi 2 [0, 1].
They do not need to have the same distribution. Consider the
function of p 2 (0, 1) and q 2 [0, 1],
q 1 q
D(qkp) := q log + (1 q) log , (3.29)
p 1 p
called the Kullback-Leibler divergence, or relative entropy.
If q = 0 or q = 1, we understand that 0 · log 0 = 0, so the
function is continuous in q. Notice that D is not symmetric
in its arguments, D(qkp) 6= D(pkq). First, let us check that
this function is nonnegative.
Lemma 3.4. For any p, q 2 (0, 1), D(qkp) 0. It is equal to
0 only if p = q.
Proof. Since log x is convex and log x 1 x (its tangent
line at x = 1) with equality only at x = 1,
q 1 q
D(qkp) = q log + (1 q) log
p 1 p
⇣ p ⌘ ⇣ 1 p⌘
= q log + (1 q) log
q 1 q
⇣ p ⌘ ⇣ 1 p⌘
q 1 + (1 q) 1 = 0,
q 1 q
with equality only if p = q. t
u
3.3 Hoeffding–Chernoff Inequality 119

Theorem 3.3 (Hoeffding–Chernoff’s inequality). Suppose

that the random variables X1 , . . . , Xn are independent, each
Xi 2 [0, 1], and p = EXi . Then
nD(p+tkp)
P Xn p+t  e (3.30)

for any 0  t  1 p, and

nD(1 p+tk1 p)
P Xn  p t e (3.31)

for any 0  t  p.
The dependence on n in these inequalities is exponential, just
like in Hoeffding’s inequality, which is much better than in
Chebyshev’s inequality. Also, notice that the probability on
the left hand side of (3.30) is zero when p + t > 1, because
the average X n can never exceed 1, so t  1 p is not really
a constraint there. Similarly, t  p in (3.31) is not really a
constraint.
The factor D(p + tkp) in the exponent in the upper tail
bound (3.30) is not the same as D(1 p + tk1 p) in the
exponent in the lower tail bound (3.31), so these bound are
not symmetric. Heuristically, they should not be the same
since, for example, when p < 1/2, there is ‘less room’ for the
deviations below p between 0 and p than above p between
p and 1. However, one can simplify both bounds and obtain
more intuitive (although slightly worse) bounds
2nt 2
P Xn p+t  e , (3.32)

and
2nt 2
P Xn  p t e , (3.33)

which we will leave as a simple exercise below.

Proof (Theorem 3.3). Using the convexity of et , we can write
for x 2 [0, 1]:
120 3 Exponential Inequalities

el x = ex·l +(1 x)·0

 xel + (1 x)e0 = 1 x + xel .

Since each Xi 2 [0, 1], this implies that

Eel Xi  1 EXi + EXi el

=1 p + pel .

Using this together with Markov’s inequality, for any l 0,

⇣ n ⌘
P Xn p+t = P Â Xi n(p + t)
i=1
l n(p+t) n
e Eel Âi=1 Xi
n
by independence = e l n(p+t)
’ Eel Xi
i=1
l n(p+t) n
e 1 p + pel .

This upper bound holds for any l 0 and we can minimize

it over l 0. Equivalently, we can minimize its logarithm,

l n(p + t) + n log 1 p + pel ! minimize .

l 0

To find the critical point, we set the derivative equal to zero

npel
n(p + t) + =0
1 p + pel
and solve for l . We find that
(1 p)(p + t)
el = .
p(1 p t)
This is 1 because p + t p and 1 p 1 p t, which
implies that l 0 as required in the application of Markov’s
inequality. Therefore, we can plug this back into the upper
bound,
3.3 Hoeffding–Chernoff Inequality 121
⇣ n ⌘
P Â Xi n(p + t)
i=1
h⇣ p(1 p t) ⌘ p+t ⇣ (1 p)(p + t) ⌘in
 1 p+
(1 p)(p + t) 1 p t
h⇣ p ⌘ p+t ⇣ 1 p ⌘1 p t in
=
p+t 1 p t
h ⇣ p+t 1 p t ⌘i
= exp n (p + t) log + (1 p t) log ,
p 1 p

which is precisely e nD(p+tkp) . This finishes the proof of

(3.30).
To prove (3.31), we can consider the random variables
Zi = 1 Xi 2 [0, 1] with EZi = 1 p. If we apply (3.30) to
these random variables, we get
nD(1 p+tk1 p)
P Zn 1 p+t  e .

Since

Zn = 1 Xn 1 p + t () X n  p t,

this proves (3.31). t

u
Let us describe one typical application of the Hoeffding–
Chernoff inequality.
Example 3.3.1 (The generalization error of classification
algorithms). The goal of a classification algorithm is to
classify data. For example, given an image as an input, an
algorithm outputs a label describing the image content, for
example, ‘dog’ or ‘cat’. We can think of an image as a pair
(X,Y ), where X is the vector of pixel values and Y is one of
possible labels. In applications, the algorithms will be given
X and will output Y as some function Y = f (X). To find
a good candidate f (x), however, requires that the algorithm
has access to training data, which is a collection of exam-
ples (Xi ,Yi ) for i  n, for which the label is known. Given
122 3 Exponential Inequalities

the training data, an algorithm will attempt to find a function

f (x) such that the proportion of incorrectly predicted labels
is as small as possible,
1 n
 I Yi 6= f (Xi)
n i=1
! minimize .

Sometimes, a different loss function L(Y, f (X)) may be used

instead of the indicator function above,
1 n
En ( f ) := Â L Yi , f (Xi ) ! minimize .
n i=1

This average is called the empirical error of the classifier f ,

and different algorithms will minimize this error over differ-
ent classes of functions F to find a good candidate f . Sup-
pose an algorithm succeeds in finding a function f with small
empirical error on the training examples provided. Does this
mean that it will be good at classifying future examples with
unknown labels?
One typical framework to study this question is to assume
that the examples are coming from some (unknown) proba-
bility distribution P over a population of possible examples
W, and that the training data (Xi ,Yi ) for i  n consists of
i.i.d. observations from this distribution P. Then, the ability
to classify future examples, called the generalization ability,
is measured by the generalization error, which is the expec-
tation of the loss function

E( f ) := EL Y, f (X) .

Does the small value of En ( f ) mean that E( f ) is also small?

For a fixed function f this looks exactly like the law of
large numbers, but the problem is that the choice of the func-
tion f itself depends on the training data, so we can not use
the law of large numbers directly. Think of the example when
an algorithm is allowed to use all possible functions f . Then
3.3 Hoeffding–Chernoff Inequality 123

the algorithm simply memorizes the training examples and

there is no guarantee that it will have small generalization er-
ror on the future data, just like a student memorizing a bunch
of examples but lacking understanding. This is called over-
fitting in Statistics.
Suppose that the algorithm chooses from a large, but finite,
number of possible classifiers

F = f1 , f2 , . . . , fN .

Let us also suppose that the loss function L takes values in

[0, 1], for example, an indicator function above. Then, for a
fixed f 2 F , the second inequality in (3.33) implies that
2ne 2
P En ( f )  E( f ) e e .

This shows that the complement E( f ) < En ( f ) + e will have

probability close to one, at least for large n. This means that,
with high confidence, if the training error En ( f ) was small,
we expect the generalization error E( f ) also to be small.
Now we want to make the same statement simultaneously
for all f 2 F . Since
n o
9 f 2 F such that En ( f )  E( f ) e
[n o
= En ( f )  E( f ) e ,
f 2F

the union bound implies that

⇣ ⌘
P 9 f 2 F , En ( f )  E( f ) e
 Â P En ( f )  E( f ) e  Ne 2ne 2
.
f 2F

Then the complement of this event satisfies

⇣ ⌘
2ne 2
P 8 f 2 F , En ( f ) E( f ) e 1 Ne ,
124 3 Exponential Inequalities

or, written differently,

⇣ ⌘
2ne 2
P 8 f 2 F , E( f )  En ( f ) + e 1 Ne . (3.34)

Let us denote the last term by d ,

2ne 2
d := Ne . (3.35)

If e and d are both small then, with probability close to 1, no

matter what f 2 F the classification algorithm selected, its
generalization error E( f ) will be bounded by En ( f ) + e, so
it will be small if the training error En ( f ) is small. Another
way to use this is to solve (3.35) for n,
1 N
n= log , (3.36)
2e 2 d
and to say that we need n training examples in order to be
(1 d ) ⇥ 100% confident that the generalization error E( f )
is smaller that En ( f ) + e. The exponential dependence on n
in the Hoeffding–Chernoff inequality was crucial to obtain
the logarithmic dependence on N and d in (3.36) of the size
n of the training set.
Of course, classification algorithms usually optimize over
infinite function sets F . In this case, one can often find a
finite subset of functions that approximate all functions in F
within a given error, and then apply the Hoeffding–Chernoff
inequality to this set. There are many other ideas involved
in the analysis of the generalization ability of classification
algorithms, including various analogues of the Hoeffding–
Chernoff inequality. t
u

Exercise 3.3.1. (Variational representation) Prove that, for

p, q 2 (0, 1),
h ⇣ ⌘i
a b
D(qkp) = sup aq + b(1 q) log e p + e (1 p) .
a,b2R
3.3 Hoeffding–Chernoff Inequality 125

Exercise 3.3.2. Prove that the function f (p, q) = D(qkp) is

convex on (0, 1)2 . Hint: use the previous exercise.
Exercise 3.3.3. Prove (3.33) by showing that D(p + tkp)
2t 2 for t  1 p. Hint: compare second derivatives.
Exercise 3.3.4. Prove that for 0 < p  1/2 and 0  t < p,

t2
D(1 p + tk1 p) ,
2p(1 p)
so ⇣1 n ⌘ ⇣ ⌘
nt 2
P Â
n i=1
Xi  p t  exp
2p(1 p)
.

Hint: compare second derivatives.

Exercise 3.3.5. Suppose that the random variables X1 , . . . , Xn
are independent, Xi 2 [a, b] and µ = EXi . How can we use the
Hoeffding–Chernoff inequality to bound P(X n µ + t) for
0  t  b µ and P(X n  µ t) for 0  t  µ a?
Exercise 3.3.6. Suppose that the loss function L 2 [0, 1] and
the class of functions F utilized by a classification algorithm
are such that the following approximation property holds for
some constant V > 0. For any e > 0, we can find N = ( e1 )V
functions f1 , . . . , fN in F such that, for any f 2 F ,

sup L(y, f (x)) L(y, f j (x))  e

x,y

for some j  N. What should the training data size n be in

order to guarantee with probability at least 1 d that the
generalization error E( f )  En ( f ) + e for all f 2 F ?
126 3 Exponential Inequalities

3.4 Azuma Inequality

Until now we have considered sums of independent random

variables. In this section, we will prove a concentration in-
equality that deals with non-linear functions

Z = f (X1 , . . . , Xn ) (3.37)

of independent random variables, or random vectors, Xi . For

applications (we will see several in the next section), it will
be convenient to suppose that each entry Xi can itself be a
vector consisting of several random variables, as long as
n
P(X1 = x1 , . . . , Xn = xn ) = ’ P(Xi = xi ), (3.38)
i=1

for any possible (vector) values x1 , . . . , xn that these vectors

X1 , . . . , Xn can take.
We will assume the following stability condition on the
function f :

f (x1 , . . . , xi , . . . , xn ) f (x1 , . . . , xi0 , . . . , xn )  ai (3.39)

for all i  n, for some constants a1 , . . . , an . This means that

changing the ith coordinate of the function f while keeping
all the other coordinates fixed can change its value by not
more than ai . In particular, the function f is bounded, which
we assume throughout this section. Our main result is the
following.
Theorem 3.4 (Azuma’s inequality). If (3.39) holds then
⇣ t2 ⌘
P f Ef t  exp (3.40)
2 Âni=1 a2i
for any t 0.
In the proof we will finally use the Fubini Theorem 1.2 from
Section 1.4 in an essential way. It states that if the random
3.4 Azuma Inequality 127

variables X are Y and independent then

h i
E f (X,Y ) = Â Â f (ak , b` )P(Y = b` ) P(X = ak ), (3.41)
k `

where the sum is over possible values ak and b` of X and

Y, which means that we can first fix the value of X = an
and average with respect to the distribution of Y , and then
average the result over the distribution of X. The proof of
Fubini’s theorem was very simple, based on the rearrange-
ment of the summation over possible outcomes, and the same
proof works for random vectors without any changes. For ex-
ample, if we split (X1 , . . . , Xn ) into two vectors (X1 , . . . , Xi )
and (Xi+1 , . . . , Xn ), for simplicity of notation denote a =
(x1 , . . . , xi ) and b = (xi+1 , . . . , xn ), and use that the indepen-
dence in (3.38) implies that

P (X1 , . . . , Xn ) = (a, b) (3.42)

= P (X1 , . . . , Xi ) = a P (Xi+1 , . . . , Xn ) = b ,

then the Fubini theorem allows to write E f (X1 , . . . , Xn ) as

  f (ak , b`)P (Xi+1 , . . . , Xn ) = b` P (X1 , . . . , Xi ) = ak ,

k `

where the sum is over all possible values of a = (x1 , . . . , xi )

and b = (xi+1 , . . . , xn ). It will be convenient to simplify the
notation, namely, denote the average over the coordinates
(xi+1 , . . . , xn ) by Ei . In other words,

Ei f (X1 , . . . , Xn ) (3.43)
= Â f ((X1 , . . . , Xi ), b` )P (Xi+1 , . . . , Xn ) = b` ,
`

which is a function of (X1 , . . . , Xi ). Averaging these remain-

ing coordinates with respect to their (marginal) distribution
P((X1 , . . . , Xi ) = ak ) is the second step in the Fubini theorem,
so it can be written in this more compact notation as
128 3 Exponential Inequalities
⇥ ⇤
E f (X1 , . . . , Xn ) = E Ei f (X1 , . . . , Xn ) . (3.44)

Notice that we can also use the Fubini theorem in the follow-
ing way,
⇥ ⇤
Ei 1 f (X1 , . . . , Xn ) = Ei 1 Ei f (X1 , . . . , Xn ) , (3.45)

because averaging in the coordinates (xi , . . . , xn ) can also be

done in two steps, first averaging over (xi+1 , . . . , xn ) and then
averaging over xi .
We will need the following observation in the proof of
Azuma’s inequality.
Lemma 3.5. If a random variable X satisfies |X|  1 and
EX = 0 then, for any l 0,
2 /2
Eel X  el .

Proof. We can write l X as a convex combination

1+X 1 X
lX = l+ ( l ),
2 2
because (1 + X)/2, (1 X)/2 2 [0, 1] and their sum is equal
to 1. By the convexity of ex ,
1+X l 1 X
el X  e + e l
,
2 2
and, taking expectations and using that EX = 0, we get
1 1
Eel X  el + e l
= cosh(l ).
2 2
The result follows, because we checked in the proof of the
2
Hoeffding inequality that cosh(l )  el /2 . t
u
We are now ready to prove Azuma’s inequality.
Proof (Theorem 3.4). The main idea of the proof will be to
represent the quantity of interest, f E f , as a sum of n terms,
3.4 Azuma Inequality 129

where we will average one coordinate at a time (instead of

averaging them all at once) and then compare two consecu-
tive averages. The stability condition will allow us to control
the difference of two consecutive averages and, moreover,
the average of each increment will be zero. Then, we will
apply the previous lemma.
Recalling the notation Ei in (3.43), we denote

Yi = Ei f (X1 , . . . , Xn ) Ei 1 f (X1 , . . . , Xn ) (3.46)

for i = 1, . . . , n. As we mentioned above, this is the difference

of two consecutive averages. What will be crucial to us is that
the stability condition (3.39) implies that

|Yi |  ai , (3.47)

which we will leave as a simple exercise below. Intuitively,

this should be clear, since the average Ei 1 differs from Ei
only in that Ei 1 also averages the coordinate Xi while it re-
mains fixed in Ei , so the corresponding terms in the averages
differ by not more than ai . Notice that, since

E0 f (X1 , . . . , Xn ) = E f (X1 , . . . , Xn )

(because E0 is the average with respect to X1 , . . . , Xn ) and

En f (X1 , . . . , Xn ) = f (X1 , . . . , Xn )

(because En is the placeholder for empty average), by the

telescoping sum,

Y1 + . . . +Yn = f (X1 , . . . , Xn ) E f (X1 , . . . , Xn ). (3.48)

This is called a martingale-difference representation. The

name comes from the fact that, by (3.45),

Ei 1Yi = Ei 1 f (X1 , . . . , Xn ) Ei 1 f (X1 , . . . , Xn ) = 0,

130 3 Exponential Inequalities

i.e. the average of each term Yi with respect to the last coor-
dinate Xi that it depends on is zero.
Let us now take l 0 and start with Markov’s inequality
⇣ n ⌘
P f Ef t =P Â Yi t e lt
EelY1 +...+lYn .
i=1

Using (3.44) with i = n 1,

h i
lY1 +...+lYn lY1 +...+lYn 1 lYn
Ee = E En 1 e e
h i
lY1 +...+lYn 1 lYn
=E e En 1 e , (3.49)

because En 1 is the average in Xn , and the terms Y1 , . . . ,Yn 1

do not depend on the coordinate Xn .
Let X = Yn /an , which we view as a function of Xn only,
with all other coordinates fixed. Since we showed above that
|X|  1 and En 1 X = 0, Lemma 3.5 gives
2 /2
En 1 elYn = En 1 el an X  e(l an ) .

Plugging this into (3.49), we get

2 /2
EelY1 +...+lYn  e(l an ) EelY1 +...+lYn 1 .

We can now proceed in exactly the same fashion to average

Xn 1 , Xn 2 , and so on, removing one by one the terms Yn 1 ,
Yn 2 , . . ., and in the end we get
n 2 /2
EelY1 +...+lYn  eÂi=1 (l ai ) .

This proves that

⇣ n ⌘ ⇣ l 2 n 2⌘
P Â Yi t  exp lt + Â ai .
2 i=1
i=1

Optimizing over l 0 yields the claim (3.40), so the proof

is complete. t
u
3.4 Azuma Inequality 131

Exercise 3.4.1. Show that (3.38) implies (3.42).

Exercise 3.4.2. Check carefully that (3.45) holds.
Exercise 3.4.3. Prove the equation (3.47)
Exercise 3.4.4. Among all the random variables X such that
|X|  100 and EX = 0, which one has the largest exponential
moment EeX ?
132 3 Exponential Inequalities

3.5 Applications of Azuma Inequality

In this section, we will show several applications of Azuma’s

inequality.
Example 3.5.1 (Chromatic number of the Erdős–Rényi
graph). Consider the Erdős–Rényi random graph G(n, p) on
n vertices. Let c(G(n, p)) be the chromatic number of this
graph, which is the smallest number of colours needed to
colour the vertices so that no two adjacent vertices share the
same colour. Let us denote the set of vertices by {v1 , . . . , vn }
and let

ei, j = I the edge between vi and v j is present .

Then, all ei, j are independent Bernoulli B(p) random vari-

ables, by the definition of the Erdős–Rényi random graph.
For i = 2, . . . , n, let us denote by Xi = (e1,i , e2,i . . . , ei 1,i ) the
vector of indicators of edges between the vertex vi and ver-
tices v1 , . . . , vi 1 . The vectors X2 , . . . , Xn are independent be-
cause they consist of indicators of disjoint sets of edges, and
the chromatic number is a function

c(G(n, p)) = f (X2 , . . . , Xn ),

since these vectors include indicators of all edges in the

graph. To apply Azuma’s inequality, we need to determine
the stability constants a2 , . . . , an . Notice that if we replace Xi
with another value Xi0 = (e01,i , e02,i . . . , e0i 1,i ), this means that
we modify some edges between vi and v1 , . . . , vi 1 . The chro-
matic number can not increase by more than 1, because we
can always assign a new colour to the vertex vi , so

f (X2 , . . . , Xi0 , . . . , Xn ) f (X2 , . . . , Xi , . . . , Xn )  1.

By the same logic,

f (X2 , . . . , Xi , . . . , Xn ) f (X2 , . . . , Xi0 , . . . , Xn )  1,

3.5 Applications of Azuma Inequality 133

and this shows that the stability condition (3.39) holds with
ai = 1. Therefore, Azuma’s inequality implies (when applied
to both f and f )
⇣ ⌘ t2
P c(G(n, p)) Ec(G(n, p)) t  2e 2(n 1) .
p
For example, if we take t = 2n log n, we get
⇣ p ⌘ 2
P c(G(n, p)) Ec(G(n, p))  2n log n 1 ,
n
so, with
p high probability, the chromatic number will be
within 2n log n from its expected value Ec(G(n, p)). It is
known (but non-trivial) that this expected value
n 1
Ec(G(n, p)) ⇠ log ,
2 log n 1 p
p
so the deviation 2n log n is of a smaller order compared
to the expectation. This shows that the chromatic number is
typically of the same order as its expectation, which gives us
an example of the law of large numbers for a very non-trivial
functional of independent random variables. t
u
Example 3.5.2 (Balls in boxes). Suppose that we throw n
balls into m boxes at random, so that the probability that
a ball lands in any given box is 1/m, and independently
of each other. Let N be the number of non-empty boxes.
If Xi 2 {1, . . . , m} is the box number in which the ith ball
lands then X1 , . . . , Xn are independent random variables and
N = card{X1 , . . . , Xn } is the number of distinct boxes hit.
As in the Example 1.5.1 in Section 1.5, it is easy to com-
pute, using the linearity of expectation, that
⇣ ⇣ 1 ⌘n ⌘
EN = m 1 1 . (3.50)
m
When n is large and m = an for some fixed a > 0 then
134 3 Exponential Inequalities

1/a
EN ⇠ na 1 e .

Changing the value of one box Xi changes N by at most 1,

so the stability constants are all equal to ai = 1. As in the
previous example,
t2
P N EN t  2e 2n , (3.51)

by Azuma’s inequality, and

p 2
P N EN  2n log n 1 .
n
So the typical number of non-empty boxes in this case is
close to its expectation, relatively speaking. t
u
Example 3.5.3 (Max-Cut of sparse random graph). This
example is quite similar to the previous one, only balls and
boxes have a different meaning and, instead of the number
of non-empty boxes, we consider a much more complicated
function.
Let V = {v1 , . . . , vn } be the set of vertices of a graph and
E be its set of edges. Max-Cut problem is to divide vertices
into two groups in a way that maximizes the number of edges
between the two groups. This is a very important problem in
computer science that has many applications, for example,
to layout of electronic circuitry, and as a reformulations of
various combinatorial optimization problems. For example,
imagine that we have a group of n people and edges represent
people that dislike each other. Then our goal is to separate
them into two groups in such a way that as many ‘enemies’
as possible are in the opposite groups.
Here we will consider a specific model of a random graph,
and will show that the maximal number of edges between
the two groups concentrates around its expectation. We will
select edges randomly, but using a different procedure than
in the Erdős–Rényi graph. We will take
3.5 Applications of Azuma Inequality 135

m = dn

possible edges for some fixed d > 0 and, as usual, we think of

n as being large. Then we place each of these m edges at ran-
dom among the set of n2 possible edges, independently of
each other. Each pair of vertices (a possible location to place
an edge) represents a box, and edges represent balls. The ran-
dom variables X1 , . . . , Xm that describe between which ver-
tices each edge is placed take n2 possible values and are all
independent. This model produces what is called a sparse
graph, because the number of edges m = nd is small rela-
tive to n2 and there is a fixed average number of edges per
vertex, d. As in the above example, let

ei, j = I the edge between vi and v j is present .

It is possible that more than one edge is placed between two

vertices, in which case we keep one of them.
The function that we will consider on the above sparse
random graph is called Max-Cut. It is defined as follows.
If we want to split all vertices into two groups, one way to
encode this is to assign each vertex one of the two labels
{ 1, 1}. Then all vertices with the same label belong to the
same group. In other words, each vector

s = s1 , . . . , sn 2 { 1, 1}n

describes a possible cut of the graph into two groups. Let

E(s ) be the number of present edges connecting the vertices
in opposite groups,
n o
E(s ) = card i < j : ei, j = 1, si 6= s j . (3.52)

Another way to represent E(s ) is as follows. Notice that

si 6= s j only if si s j = 1, otherwise, si s j = 1. Therefore,
I(si 6= s j ) = (1 si s j )/2 and
136 3 Exponential Inequalities

1
E(s ) = Â ei, j 1
2 i<
si s j . (3.53)
j

Then the value M of the Max-Cut corresponds to a way to

cut the graph so that the number of edges between the two
groups is as large as possible,

M = max E(s ). (3.54)

s

Recall that M is a random variable that depends on the posi-

tions X1 , . . . , Xm of our possible m = dn edges.
If we change the placement Xi of one edge, this can change
the maximum M by at most 1, because, for each possible
cut (configuration s ), moving one edge might increase or
decrease the number of edges between the two groups by at
most 1. This means that the stability condition in Azuma’s
inequality holds with ai = 1 and, therefore,
t2 t2
P M EM t  2e 2m = 2e 2dn . (3.55)
p
If we take t = 2dn log n, we get
⇣ p ⌘ 2
P M EM  2dn log n 1 .
n
p
Is the deviation 2dn log n of smaller order than EM? It
turns out that there exists a cut such that at least half of
all edges are between the vertices that belong to opposite
groups. To see this, choose labels s1 , . . . , sn to be i.i.d.
Rademacher, which means that we assign each vertex to one
of the two groups { 1, +1} at random. Then

M = max E(s ) EE(s )

s
1 1
= Â ei, j 1
2 i<
Esi s j = Â ei, j ,
2 i<
j j
3.5 Applications of Azuma Inequality 137

which is exactly one half of all the present edges. Taking the
expectations on both sides, we can write
1
EM Â P there is an edge between vi and v j
2 i< j
✓ ◆⇣ ⇣
1 n 1 ⌘dn ⌘
= 1 1 n ,
2 2 2

where on the right hand side we have the analogue of (3.50).

Using the inequality 1 x  e x , we can write
⇣ 1 ⌘dn dn/(n2) 2d
1 n  e  e n

2
2d 2d 2 ⇣1⌘
=1 + 2 +O 3 ,
n n n
where in the second line we used Taylor’s theorem for ex at
zero. Plugging this in the above inequality, we get
✓ ◆
1 n ⇣ 2d 2d 2 ⇣ 1 ⌘⌘
EM +O 3
2 2 n n2 n
dn d + d 2 ⇣ 1 ⌘
= +O .
2 2 n
This shows that (up to the smaller order
p terms) EM is at least
dn/2. This means that the deviation 2dn log n in Azuma’s
inequality above is of a smaller order, so the typical Max-Cut
value M is relatively close to its expectation.
By the way, it is a major open problem to compute the
limit limn!• EM/n in terms of d. t
u
Example 3.5.4 (Hamming cube). For any x, y 2 {0, 1}n , the
number of coordinates of x and y that differ,
n
r(x, y) = Â I(xi 6= yi ), (3.56)
i=1
138 3 Exponential Inequalities

is called the Hamming distance between x and y. The func-

tion r is called the Hamming metric on {0, 1}n . We will
now use Azuma’s inequality to show that, given a subset
A ✓ {0, 1}n that contains a positive proportion of all points,

card(A) > e2n (3.57)

for some e > 0, most points in {0, 1}n are relatively close
pto
the set A, namely, within the Hamming distance of order pn.
In other words, for most points in {0, 1}n , only of order n
coordinates are different from one of the points in A.
Given t > 0, let us denote by
n o
n
B(A,t) = x 2 {0, 1} : r(x, y)  t for some y 2 A (3.58)

the set of all points within Hamming distance t from A. For

e, d 2 (0, 1), let us denote
r r
1 1
te,d = 2 log + 2 log .
e d
p
Then the following lemma shows that te,d n-neighbourhood
of A contains at least (1 d ) proportion of all points in
{0, 1}n .
Lemma 3.6. For e, d 2 (0, 1), if card(A) > e2n then
p
card B A,te,d n (1 d )2n . (3.59)

Proof. Let X = (X1 , . . . , Xn ) be a vector consisting of i.i.d.

Bernoulli B(1/2) random variables, which means that, for
any subset S ✓ {0, 1}n ,
card(S)
P X 2S = .
2n
Let us consider a random variable Z = miny2A r(X, y) equal
to the Hamming distance from X to the set A. Changing one
coordinate Xi to Xi0 can change Z by at most 1, and Azuma’s
3.5 Applications of Azuma Inequality 139

inequality (applied to Z and Z) implies that

p 2
P Z  EZ t n  e t /2 , (3.60)
p 2
P Z EZ + t n  e t /2 . (3.61)
q
If in the first inequality we take te = 2 log e1 then
p
P Z  EZ te n  e.

This directionpof Azuma’s inequality allows us to conclude

that EZp te n because of the following observation. If
EZ te n > 0 then the above probability would be bigger
than P(Z = 0). However, since Z = 0 if and only if X 2 A,
card(A)
P(Z = 0) = P(X 2 A) = > e,
2n
by our assumption, which is a contradiction. Therefore,
p
EZ  te n.

The second inequality then implies

p p p t 2 /2
P Z te n + t n  P Z EZ + t n  e .
q
In this inequality, we take td = 2 log d1 then
p p
P Z te n + td n  d ,

which implies that

p p
P(Z  te n + td n) d.
1
p
Since this event is exactly the setpB(A,te,d n) of all points
within the Hamming distance te,d n from A, this finishes the
proof. t
u
140 3 Exponential Inequalities

Exercise 3.5.1 (Empirical process). Given a family F of

functions f : R ! [0, 1] and independent random variables
X1 , . . . , Xn , consider
n
1
Z := p sup
n f 2F Â f (Xi ) E f (Xi ) .
i=1

Prove that P(|Z EZ| t)  2e t 2 /2 for all t 0.

Exercise 3.5.2. In a graph with n vertices and m = dn edges,
if di is the number of edges connected to the vertex vi , show
that 1n Âni=1 di = 2d.
Exercise 3.5.3 (3-SAT). Suppose that a debate class has a
large number of students, n. The professor creates m = 2n
debate teams, each composed of 3 members, and he does it
by assigning each student to 6 different teams.
Each member of each team is assigned to defend a posi-
tion from the platform of one of two major political parties,
Party A or Party B. The professor does not know the party
affiliations of the students, and we suppose that each stu-
dent belongs to either Party A or Party B with probability
1/2 each, independently of each other. Given their turn, each
team will select only one of its members to defend his or her
assigned position. However, if it turns out that all three mem-
bers have been assigned positions of the opposing party (not
the one to which they belong), the team will skip their turn.
Let N be the number of speeches delivered in class. What
is the expectation EN? Use Azuma’s inequality to show that,
with high probability, the number of speeches N will be rel-
atively close to EN.
Exercise 3.5.4. Suppose we throw n balls into m boxes at
random, and let N be the number of boxes with at least two
balls in them. Compute EN. What is the limit limn!• EN/n
when m = an for a fixed a > 0? What does Azuma’s in-
equality say about |N EN|?
Chapter 4
Gaussian Distributions

In Example 3.1.1, we have seen that Hoeffding’s inequality

for the average
Sn X1 + . . . + Xn
Xn = =
n n
of i.i.d. X1 , . . . , Xn Bernoulli B(1/2) random variables im-
plies that, for x 0,
2nx2
P |X n 1/2| > x  2e .
t
If we make the change of variables x = p
2 n
and denote
p
Zn := 2 n(X n 1/2),

we can rewrite this inequality as

t 2 /2
P |Zn | t  2e .

The expectation and variance of Bernoulli B(1/2) random

variables equal
1 1
µ = EX1 = , s 2 = Var(X1 ) =
2 4
and, with this notation, we can represent Zn as

141
142 4 Gaussian Distributions
p
n Sn nµ
Zn = (X n µ) = p .
s ns 2
We mentioned in the Example 3.1.1 that the dependence on
t in Hoeffding’s inequality is nearly optimal, which will be
demonstrated in this chapter once we prove the Central Limit
Theorem (CLT). For Zn as above, the CLT states that
Z t
1 x2 /2
lim P(Zn  t) = p e dx, (4.1)
n!• 2p •

for all t 2 R. In other words, instead of upper bounds on the

probabilities, we will get precise asymptotic formulas in the
limit n ! •. For example, (4.1) will imply
t Z Z •
1 x2 /2 1 x2 /2
lim P(|Zn | t) = p e dx + p e dx
n!• 2p Z • 2p t
1 •
x2 /2
= 2p e dx,
2p t
by symmetry of the integrand. For large t, one can show the
following.
Exercise 4.0.1. As t ! •,
Z •
1 x2 /2 1 t 2 /2
p e dx ⇠ p e
2p t t 2p
in the sense that their ratio goes to 1. Hint: use L’Hospital’s
rule.
This shows that Hoeffding’s inequality was quite precise
p
for large t, because it was only missing a factor 1/t 2p,
2
which is large compared to e t /2 . It is important to men-
tion that, while it might look like the CLT is a more precise
result than Hoeffding’s inequality, it is an asymptotic state-
ment, whereas Hoeffding’s inequality holds for all n. For this
reason, Hoeffding’s inequality is often more useful but, on
the other hand, it is difficult to argue that the Central Limit
Theorem is a mathematically more beautiful statement.
4.1 Gaussian Distributions on R 143

4.1 Gaussian Distributions on R

The function that appeared in the integral in (4.1),

1 x2 /2
p(x) = p e , (4.2)
2p
is called the density of standard Gaussian distribution. This
is our first example of a continuous probability space and
a continuous distribution on the real line. In contrast with
discrete probability spaces, the space of possible outcomes
W = R is not finite or countable, and the probability of a
subset of outcomes A ✓ R (an event) is not defined as a sum
but as an integral
Z Z
1 x2 /2
g(A) := p(x) dx = p e dx. (4.3)
A 2p A

The probability measure g is called the standard Gaussian

measure on R, and we use the notation g instead of the
usual P to save P for more generic situations. This defini-
tion means that probabilities of individual outcomes are all
zero, g({x}) = 0, and probability of an interval [a, b] is
Z b
g([a, b]) = p(x) dx. (4.4)
a

Compared to discrete distributions, not every subset A ✓ W

can be an event, because for some sets A the integral may be
undefined. Allowed choices of A depend on whether we use
the Lebesgue integral or Riemann integral. For now, we will
work only with nice sets such as intervals of the form

[a, b], [a, b), (a, b], (a, b),

where a and b can be ±•, or finite unions of intervals, in

which case the Riemann integral is well defined.
144 4 Gaussian Distributions

Since the Lebesgue integral has better properties than

Riemann integral, you can think of all the integrals as the
Lebesgue integral if you are familiar with it. But even if we
work with the Riemann integral, we can notice that at least
some of the properties of probability measures that we ob-
served in the setting of discrete probability spaces still hold
here. For example,
Z •
g(R) = p(x) dx = 1. (4.5)
•

To see this, one can use the following replica trick,

⇣ 1 Z ⌘2
2 x2 /2
g(R) = p e dx
2p
ZZ
R
1 2 2
= e (x +y )/2 dxdy
2p R2
Z Z
1 2p • r2 /2
= e r drdq
2p 0 0
Z Z •
1 2p 2
= dq e r /2 r dr = 1,
2p 0 0

where we used polar coordinates as dx dy = r dr dq . We also

have finite additivity,

g(A1 [ . . . [ An ) = g(A1 ) + . . . + g(An ) (4.6)

for disjoint sets A1 , . . . , An , as long as these sets are nice, for

example, finite unions of intervals. This holds by the usual
linearity of integral.
However, if we would like to have countable additivity as
in Section 1.2 for discrete spaces, working with the Riemann
integral is not enough. For example, if we take a sequence
of one point sets {qn }, where Q = {q1 , q2 , . . .} are rational
numbers on the interval [0, 1] (enumerated in an arbitrary or-
der) then the Riemann integral over one point is zero, but
the Riemann integral of the indicator of Q is undefined (Q
is not rectifiable), while the Lebesgue integral is well de-
4.1 Gaussian Distributions on R 145

fined and equal to zero. Countable additivity is satisfied by

the Lebesgue integral over the class of Lebesgue measurable
sets, which can play the role of events on this probability
space.
Let g : R ! R be the identity function g(x) = x. On the
space W with the measure g, this function may be called
a standard Gaussian random variable, or random variable
with the standard Gaussian distribution (by analogy with the
definitions on discrete spaces), because

P(g 2 A) = g x : g(x) 2 A = g(A). (4.7)

Exercise 4.1.1. Show that g(x) = x also has the standard

Gaussian distribution.
Recall that, on discrete probability spaces, any function
was called a random variable. However, on a continuous
space such as (R, g), just like not every set A can be called
an event, a function X : R ! R has to be nice enough to be
called a random variable. The reason for this is that, if we
want to calculate the probability that X takes values in some
set A (for example, an interval),

P(X 2 A) = g(x : X(x) 2 A) (4.8)

Z
1
=g X (A) = p(x) dx,
X 1 (A)

we need to be able to integrate over the pre-image X 1 (A).

If we are using the Lebesgue integral then random variables
are (Borel) measurable functions. In general, the Lebesgue
integral is much better suited for Probability, but, since we
do not assume the familiarity with Lebesgue integration, we
simply have to limit ourselves to a nicer class of functions.
We will call a function f : R ! R nice if
1. f is piecewise-continuous (and bounded on compacts);
2. the pre-image f 1 (A) of any interval A is a rectifiable set.
146 4 Gaussian Distributions

We can be even more restrictive and require that f 1 (A) for

any interval A is a finite union of intervals.
For nice functions, we can calculate their distributions. Let
us consider several examples.
Example 4.1.1. Let us first consider the function

X(x) = I(x 0) I(x < 0),

which takes two values { 1, +1}. Then

Z •
1
P(X = +1) = g(x : x 0) = p(x) dx = ,
0 2
Z 0
1
P(X = 1) = g(x : x < 0) = p(x) dx = .
• 2
This means that X is a familiar Rademacher random variable,
now defined on a continuous probability space (R, g). t
u
Example 4.1.2. Next, let us consider X(x) = |x|. Obviously,
X takes values in [0, •), but how do we describe probabilities
P(X 2 A)? If 0  a  b,

X 2 [a, b] = x : |x| 2 [a, b] = [ b, a] [ [a, b]

and, therefore, by symmetry of the Gaussian density p(x),

Z b
P(X 2 [a, b]) = 2g([a, b]) = 2p(x) dx.
a

Since X can not take negative values, if we define

⇢
0, x < 0,
pX (x) = 2p(x) I(x 0) =
2p(x), x 0,
then, for any a  b, we can write
Z b
P(a  X  b) = pX (x) dx.
a

If we add up over disjoint intervals, we get

4.1 Gaussian Distributions on R 147
Z
P(X 2 A) = pX (x) dx,
A

for any finite union of intervals A. The distribution of X is

encoded by the function pX (x) via an integral, in the same
way as the standard Gaussian distribution above. t
u
When the probabilities P(X 2 A) are given by
Z
P(X 2 A) = pX (x) dx, (4.9)
A

for an integrable function pX (x) 0 such that

Z
pX (x) dx = 1,
R

we say that X has continuous distribution with the density

pX (x) on the real line. More precisely, such distributions
are called absolutely continuous, but for simplicity we will
call them continuous. Notice that, if we replace the Gaussian
density p(x) in (4.3) with pX (x), we get another example (or
as many examples as we like) of a non-discrete probability
space, (R, P).
Example 4.1.3 (Exponential distribution). A probability
distribution with the density
lx
pX (x) = l e I(x 0) (4.10)

is called the exponential distribution with parameter l . t

u
Example 4.1.4 (Uniform distribution). A probability dis-
tribution with the density
1
pX (x) = I(a  x  b), (4.11)
b a
for • < a < b < •, is called the uniform distribution on
the interval [a, b]. t
u
The distribution P(X 2 A) can be encoded by a function
148 4 Gaussian Distributions

F(x) = P(X  x), (4.12)

which is called the cumulative distribution function (c.d.f.)

of X. For example,

P(a < X  b) = P(X  b) P(X  a) = F(b) F(a).

If F(x) is continuous at x = a then

P(X = a)  P(a e < X  a + e)

= F(a + e) F(a e)

and, letting e # 0 shows that P(X = a) = 0. If the function

F(x) is differentiable and the derivative F 0 (x) = pX (x) is
piecewise-continuous then, by the Fundamental Theorem of
Calculus,
Z b
P(a < X  b) = F(b) F(a) = pX (x) dx,
a

so pX (x) is the density of the distribution of X. To find the

density, we can find the c.d.f. first and differentiate it.
Example 4.1.5. Let X = g2 , where g(x) = x is a standard
Gaussian random variable on (R, g). For x > 0,

F(x) = P(X  x) = P(g2  x)

Z p
p p x
= P( x  g  x) = p p(t) dt.
x

Taking derivatives and using the symmetry of the Gaussian

density p(x),
p 1 p 1
F 0 (x) = p( x) p p( x) p
2 x 2 x
p 1 1
= p( x) p = p e x/2 .
x 2px
Since X does not take negative values, if we define
4.1 Gaussian Distributions on R 149
(
0, x  0,
pX (x) = p 1 e x/2 ,
2px
x > 0,

then we showed that pX (x) is the density of the distribution

of X. This distribution is called G( 12 , 12 ) distribution, and we
will come back to it in Section 4.4 t
u
Example 4.1.6. If µ 2 R, s > 0 and X = µ + s g, one can
check as in the previous example that X has density
1 (x µ)2
pµ,s (x) = p e 2s 2 . (4.13)
2ps
This distribution, denoted N(µ, s 2 ), is called the Gaussian
distribution with mean µ and variance s 2 (we will see why
in a second). When µ = 0, we denote p0,s (x) by ps (x). t u
The expectation or expected value of a random variable
X : R ! R on the probability space (R, g) is defined by
Z
EX = X(x)p(x) dx, (4.14)
R

assuming that E|X| < • (to make sure that the integrals of
the positive and negative parts of X are well defined). For
example, for g(x) = x,
Z
Eg = xp(x) dx = 0,
R

because the function xp(x) is odd, and E|g| < •. Also,

Z • Z •
2 1 2 x2 /2 1 x2 /2
Eg = p x e dx = p x d( e )
2p Z • 2p •
1 •
x2 /2
=p e dx = 1,
2p •

using integration by parts. The expectation and variance of

the standard Gaussian random variable g is Eg = 0 and
Var(g) = 1. This implies that, for X = µ + s g ⇠ N(µ, s 2 ),
150 4 Gaussian Distributions

EX = µ, Var(X) = E(s g)2 = s 2 ,

so the meaning of the parameter µ, s 2 in N(µ, s 2 ) is the

expectation and variance of X.
The process by which we computed the distribution of X
in the examples above is called the change of variables in the
sense of measures, where the function X does not have to be
one-to-one (injective). When the distribution of X turned out
to be continuous, we found that
Z Z
P(X 2 A) = p(x) dx = pX (t) dt, (4.15)
X 1 (A) A

for some density function pX (t). This change of variables

formula can be extended to expectations.
Lemma 4.1 (Change of variables for integrals). If (4.15)
holds and E|X| < • then
Z Z
EX = X(x)p(x) dx = t pX (t) dt. (4.16)
R R

This is the analogue of the change of variables in the setting

of discrete spaces in the equation (1.30) in Section 1.2.
Proof. First of all, if we write X as a sum of positive and
negative parts,

X = X I(X > 0) + X I(X  0),

the density pX (t) on [0, •) is the density of the positive part,

and on ( •, 0) it is the density of the negative part. If we can
prove (4.16) separately for the positive and negative parts,
the claim follows by adding them up. This means that it is
enough to assume that X > 0 and prove that
Z Z •
EX = X(x)p(x) dx = t pX (t) dt. (4.17)
R 0

Let us take large M > 0 and divide the interval (0, M] into
many small subintervals
4.1 Gaussian Distributions on R 151

0 = y0 < y1 < . . . < ym = M,

where the length of each interval is less than e > 0. Let us

use the change of variables formula (4.15) for one interval
A = (yi , yi+1 ]. If, for i = 0, . . . , m 1, we denote
⇣ ⌘ n o
1
Ai := X (yi , yi+1 ] = x 2 R : yi < X(x)  yi+1 ,

then (4.15) states that

Z Z
p(x) dx = pX (t) dt.
Ai (yi ,yi+1 ]

Since we restricted ourselves to nice functions, the sets Ai are

rectifiable and the integrals are well defined. This equation
can also be written as
Z
I yi < X(x)  yi+1 p(x) dx
RZ
= I(yi < t  yi+1 )pX (t) dt.
R

If we multiply both sides by yi and sum over i, we get

Z m 1
Â
R i=0
yi I yi < X(x)  yi+1 p(x) dx
Z m 1
=
R i=0
 yi I(yi < t  yi+1)pX (t) dt.
If we denote the sum in the last integral by
m 1
ye (t) := Â yi I(yi < t  yi+1),
i=0

this equation can be written as

Z Z
ye X(x) p(x) dx = ye (t)pX (t) dt. (4.18)
R R
152 4 Gaussian Distributions

Notice that the step function ye (t) is an approximation of the

identity function on (0, M],

ye (t) t I(0 < t  M)  e,

because ye (t) = yi when yi < t  yi+1  yi + e. Therefore,

letting e # 0 in (4.18), we get
Z
X(x) I(0 < X(x)  M)p(x) dx (4.19)
R
Z Z M
= t I(0 < t  M)pX (t) dt = t pX (t) dt.
R 0

By the Monotone Convergence Theorem for the Riemann (or

Lebesgue) integral, letting M " +• implies (4.17), because
the limit function X(x) on the left hand side was assumed to
be integrable. t
u
This result can be generalized to functions f (X) of the
random variable X.
Lemma 4.2 (Change of variables for integrals). If (4.15)
holds and E| f (X)| < • then
Z
E f (X) = f (t)pX (t) dt. (4.20)
R

The proof is identical, only instead of approximating the

identity function by step functions, we approximate f (t) by
step functions of the form
m 1
fe (t) = Â yi I(yi < f (t)  yi+1).
i=0

Also, instead of A = (yi , yi+1 ], now we apply the change of

variables formula (4.15) to the set
1
B= f (A) = t 2 R : yi < f (t)  yi+1 .

The set X 1 (B) should be rectifiable and, since

4.1 Gaussian Distributions on R 153

1 1 1
X (B) = X (f (A)) = ( f X) 1 (A),

this poses some restrictions on f . When we assumed that

the random variable X is a nice function, we assumed that
X 1 (B) is rectifiable when B is a union of intervals. This
means that we can consider f such that the pre-image of an
interval is a finite union of intervals. In applications, many
functions that we deal with satisfy this condition. However,
this is another reason to learn about Lebesgue integration,
where a composition f X of two nice (Borel measurable)
functions is automatically nice (Borel measurable), and we
do not need to make unnatural assumptions in the change of
variables formula.
Exercise 4.1.2. Compute the expectation of a uniform and
exponential random variable.
Exercise 4.1.3. What is the density of the random variable
X = g3 ?
Exercise 4.1.4. Compute the expectation E|X|, where X ⇠
N(0, s 2 ).
Exercise 4.1.5. Show that the moments of the standard Gaus-
sian distribution are Eg2k 1 = 0, and

Eg2k = (2k 1)!! = 1 · 3 · 5 · · · (2k 1)

for integer k 1. Hint: use integration by parts.

Exercise 4.1.6. (Gaussian integration by parts) If g is the
standard Gaussian random variable, prove that

EgF(g) = EF 0 (g)

and both sides are well defined if F is continuously differen-

tiable and |F 0 (x)|  2 + e|x| . Hint: use integration by parts.
154 4 Gaussian Distributions

4.2 Gaussian Distributions on Rn

If for x = (x1 , . . . , xn ) 2 Rn we denote its length by

|x| = (x12 + . . . + xn2 )1/2

then
1 |x|2 /2
pn (x) = e (4.21)
(2p)n/2
is called the density of the standard Gaussian distribution
on Rn . If we recall the standard Gaussian density p(x) on the
real line defined in (4.2), we see that

pn (x) = p(x1 )p(x2 ) · · · p(xn ). (4.22)

In other words, pn is the product of one-dimensional Gaus-

sian densities. We define the standard Gaussian measure gn
on Rn by
Z Z
gn (A) = pn (x) dx = pn (x1 , . . . , xn ) dx1 . . . dxn . (4.23)
A A

As in the previous section, we assume that the set A is nice,

so that the integral is well defined. By Fubini’s theorem,
Z
n
gn (R ) = p(x1 ) · · · p(xn ) dx1 . . . dxn
Rn
n Z
=’ p(xi ) dxi = 1.
i=1 R

In other words, Fubini’s theorem tells us that a product of

densities on R is a density on Rn in the sense that it satisfies
basic properties of a probability measure.
This definition as a product corresponds to independence
of random variables that played such an important role in
all the previous chapters in the setting of discrete probability
spaces. Specifically, in the case of the space W = Rn with the
probability measure gn , the coordinate functions
4.2 Gaussian Distributions on Rn 155

g1 (x) = x1 , . . . , gn (x) = xn (4.24)

can be viewed as independent standard Gaussian random

variables, because, for any nice sets Ai for i  n, by Fubini’s
theorem,

P(g1 2 A1 , . . . , gn 2 An ) = gn (A1 ⇥ · · · ⇥ An ) (4.25)

Z n Z n
= pn (x) dx = ’ p(xi ) dxi = ’ g(Ai ).
A1 ⇥···⇥An i=1 Ai i=1

If we take all sets equal to R except for one, Ai , we get that

P(gi 2 Ai ) = g(Ai ) and, therefore,
n
P(g1 2 A1 , . . . , gn 2 An ) = ’ P(gi 2 Ai ).
i=1

As before, this can be expressed by saying that g1 , . . . , gn

are independent random variables with the standard Gaus-
sian distribution N(0, 1).
Given coordinate functions gi (x) = xi in (4.24), consider a
vector-valued function

g = (g1 , . . . , gn ) : Rn ! Rn ,

which is just the identity function g(x) = x on Rn . Given a

rectifiable set A ✓ Rn , similarly to (4.25), we can write
Z
P(g 2 A) = gn (A) = pn (x) dx. (4.26)
A

By analogy with continuous random variables, we say that a

random vector g has density pn (x). Such random vectors are
called standard Gaussian random vectors.
As in the previous section, in order to call a function

X : Rn ! R
156 4 Gaussian Distributions

a random variable, we can either assume that it is measurable

if we use the Lebesgue integral or, otherwise, we have to
restrict ourselves to nice functions. We assume that X 1 (A)
is a rectifiable set for any interval A, and that X is piecewise-
continuous and bounded on compacts, so that all the integrals
that we consider are well defined.
For any random variable X : Rn ! R on the space (Rn , gn ),
its expectation is defined by
Z
EX = X(x)pn (x) dx, (4.27)
Rn

assuming that E|X| < •.

To compute the distribution of X, i.e. the probabilities

P(X 2 A) = gn (x : X(x) 2 A)
Z
1
= gn X (A) = pn (x) dx
X 1 (A)

for intervals A ✓ R, we can try to compute the cumulative

distribution function F(x) = P(X  x) first and reconstruct
the probabilities of other sets from there. As we have seen
in the previous section, X might have a discrete distribution
or continuous distribution or, in principle, it can have a more
general distribution. If we find that its distribution has the
density pX (t), this means that
Z Z
P(X 2 A) = pn (x) dx = pX (t) dt, (4.28)
X 1 (A) A

for some function pX (t) 0 such that

Z
pX (t) dt = 1.
R

This is again the change of variables formula. As in Lemma

4.2 in the last section, the change of variables formula in
(4.28) can be extended to expectations.
4.2 Gaussian Distributions on Rn 157

Lemma 4.3 (Change of variables for integrals). If (4.28)

holds and E| f (X)| < • then
Z
E f (X) = f (t)pX (t) dt. (4.29)
R

The proof is identical to the proof of Lemma 4.2, as long as

f satisfies certain conditions. In the proof, we needed that
( f X) 1 (A) = X 1 ( f 1 (A)) is rectifiable for any interval
A. Or, if we are using the Lebesgue integral, we can take f
to be Borel measurable.
Remark 4.1. In this and the last section, we considered two
probability spaces (W, P), namely, (R, g) and (Rn , gn ). In
both settings, given a random variable X : W ! R, we said
that if its distribution can be written as
Z
P(X 2 A) = pX (t) dt (4.30)
A

for some density function pX (t) then

Z
E f (X) = f (t)pX (t) dt. (4.31)
R

The proof of this does not depend much on what (W, P) is

and in a more advanced Real Analysis or Measure Theory
class this type of change of variables formula is proved in
much more generality. t
u
Let us consider an example of the change of variables,
which will be the most important example to us. It describes
the stability property of Gaussian distributions.
Theorem 4.1 (Gaussian stability). If a = (a1 , . . . , an ) 2 Rn
then the random variable

X = (a, g) = a1 g1 + . . . + an gn

has the Gaussian distribution N(0, |a|2 ) with variance |a|2 .

158 4 Gaussian Distributions

Since ai gi ⇠ N(0, a2i ), this means that the sum of indepen-

dent Gaussian random variables is also Gaussian with the
variance equal to the sum of variances. We will see that this
property will be at the centre of the proof of the Central Limit
Theorem below.
Proof. We can assume that a 6= 0, since otherwise there is
nothing to prove. We will now compute the probability
Z
P(X  t) = P (a, g)  t = pn (x) dx,
H

where t 2 R and H is a half-space H = {x : (a, x)  t}.

Let us denote q1 = a/|a|, and let us take arbitrary vectors
q2 , . . . , qn so that q1 , q2 , . . . , qn form an orthonormal basis of
Rn . Let Q be the n ⇥ n matrix with rows q1 , . . . , qn . Then Q is
an orthogonal matrix, det Q = 1 and |Qx| = |x| for all x 2 Rn .
If we make the change of variables y = Qx then, since
1
y1 = (q1 , x) = (a, x),
|a|
the half-space H can be written as H = {y : |a|y1  t} in the
coordinates y. Since Q is orthogonal and |y| = |Qx| = |x|,
1 |x|2 /2 1 |y|2 /2
pn (x) = e = e = pn (y).
(2p)n/2 (2p)n/2
Since det Q = 1, the Jacobian determinant in the change of
variables equals to 1 and
Z Z
pn (x) dx = pn (y) dy
H ZH
= p(y1 ) · · · p(yn ) dy1 · · · dyn
|a|y1 t
Z
(by Fubini) = p(y1 ) dy1 = P |a|g1  t ,
|a|y1 t
4.2 Gaussian Distributions on Rn 159

since the constraint |a|y1  t in the integral depends only on

the first coordinate y1 , so the coordinates y2 , . . . , yn integrated
to 1. We showed that

P(X  t) = P(|a|g1  t)

and, since we already know that |a|g1 has the distribution

N(0, |a|2 ), this finishes the proof. t
u
Example 4.2.1. If we use Lemma 4.3 above together with
the Gaussian stability property, we get that, for any

a = (a1 , . . . , an ) 2 Rn

and a function f that satisfies the conditions of Lemma 4.3,

E f (a, g) = E f (a1 g1 + . . . + an gn )
Z • Z • t2
1 2|a|2
= f (t)p|a| (t) dt = p f (t)e dt.
• 2p|a| •
For example, in the proof of the central limit theorem, we
will use this formula with a smooth bounded monotone f .
Since the pre-image f 1 (A) of an interval A is an interval
and, since X = (a, g), the pre-image ( f X) 1 (A) is a region
bounded by parallel hyperplanes, which is rectifiable.
As we emphasized in the Remark 4.1, the expectation
E f (X) depends only on the distribution of X and not on the
specific form of X or probability space on which it is defined,
so another way to write the above formula is

E f (a, g) = E f |a|g1 , (4.32)

because |a|g1 also has the distribution N(0, |a|2 ). t

u

General Gaussian distributions on Rn . Let us consider a

linear transformation of the standard Gaussian vector,

X = (X1 , . . . , Xn )T = Ag = A(g1 , . . . , gn )T , (4.33)

160 4 Gaussian Distributions

where A is an n ⇥ n matrix. It turns out that the distribution of

the random vector X depends on A only through the matrix

C = AAT , (4.34)

and it is called the Gaussian distribution with covariance C,

denoted N(0,C). Before we explain this, let us mention that
the covariance of a random vector X is the matrix
h i
Cov(X) = Cov(Xi , X j ) , (4.35)
i, jn

assuming that all the entries are well defined. If µ = EX =

(EX1 , . . . , EXn )T then the covariance matrix can be written

Cov(X) = E[(X µ)(X µ)T ].

Since EXi = 0 for X = Ag,

Cov(Ag) = EAg(Ag)T = EAggT AT

= A(EggT )AT = AAT = C,

where we used that EggT = [Egi g j ]i, jn = I is the identity

matrix.
Let us now describe what the distribution N(0,C) looks
like. Let us first consider the invertible case det(C) 6= 0.
Lemma 4.4. If X = Ag, C = AAT and det(C) 6= 0 then the
distribution of X has density
1 1 1 T 1
pC (x) = p e 2x C x . (4.36)
(2p)n/2 det(C)
Of course, as in (4.26) above, by analogy with (absolutely)
continuous random variables, when we say that a random
vector X has density pC (x), this means that
Z
P(X 2 D) = pC (x) dx (4.37)
D
4.2 Gaussian Distributions on Rn 161

for any nice set D on Rn .

Proof (of Lemma 4.4). For any nice set D on Rn , for example
a rectangle, we can write
Z
1 1 |x|2 /2
P(Ag 2 D) = P(g 2 A D) = e dx.
A 1 D (2p)n/2

If we make the change of variables y = Ax, or x = A 1 y, then

Z
1 1 y|2 /2 1
P(Ag 2 D) = e |A
dy.
D (2p)n/2 | det(A)|
We have

det(C) = det(AAT ) = det(A) det(AT ) = det(A)2 ,

and

|A 1 y|2 = (A 1 y)T (A 1 y) = yT (AT ) 1 A 1 y

= yT (AAT ) 1 y = yT C 1
y.

Therefore, we showed that

Z
1 1 1 T 1
P(Ag 2 D) = p e 2y C y dy
D (2p)n/2 det(C)
Z
= pC (y) dy,
D

which means that (4.36) is the density of the distribution

N(0,C). t
u
Example 4.2.2 (Orthogonal maps). In particular, if Q is
an orthogonal matrix and X = Qg then C = QQT = I and
Lemma 4.4 implies that
Z
1 |x|2 /2
P(X 2 D) = e dx. (4.38)
D (2p)n/2
162 4 Gaussian Distributions

In other words, X = Qg has the density pI (x) = pn (x), so X

is also a standard Gaussian random vector. t
u
If detC = 0, or det A = 0, the range of the map Ax is a
proper subspace of Rn , so the distribution of X = Ag can
not have a density on Rn . However, one can still see that it
depends only on C and describe it constructively as follows.
This is based on the following exercises.
Exercise 4.2.1. Show that a covariance matrix Cov(X) is
symmetric and nonnegative definite.
Exercise 4.2.2. If C = AAT and C = QDQT is an eigen-
decomposition of C, for some orthogonal matrix Q and D =
diag(l1 , . . . , ln ), then
p QD1/2 R for some orthogonal R
A =p
and D1/2 = diag( l1 , . . . , ln ). Hint: use that B := QT A
satisfies BBT = D to represent it as B = D1/2 R.
In the notation of the last exercise, if g̃ = Rg and q1 , . . . , qn
are the column vectors of Q then
p p
X = Ag = QD1/2 g̃ = l1 g̃1 q1 + . . . + ln g̃n qn .

Therefore, in the orthonormal p basis qp 1 , . . . , qn , the random

vector X has coordinates l1 g̃1 , . . . , ln g̃n . In this repre-
sentation, only the matrix R in g̃ = Rg depends on A. How-
ever, since R is orthogonal, by the previous lemma, g̃ has the
standard Gaussian distribution gn , which does notpdepend on
p matrix A. More precisely, the coordinates l1 g̃1 , . . . ,
the
ln g̃n are independent and have Gaussian distributions with
variances l1 , . . . , ln . When det(C) = 0, some of its eigenval-
ues will be zero, say, lk+1 = . . . = ln = 0. Then the distri-
bution will be concentrated on the subspace spanned by the
first k vectors q1 , . . . , qk , and it will have the density
k
1
f (x1 , . . . , xk ) = ’ p e xi2 /2li
i=1 2pli
4.2 Gaussian Distributions on Rn 163

on this subspace, where x1 , . . . , xk are the coordinates in the

basis q1 , . . . , qk . Notice that this description does not depend
on the specific choice of A as long as AAT = C, because the
eigenvalues and eigenvectors depend only on C. t
u
Exercise 4.2.3. Consider the matrix

21
C= .
11

Find a two-by-two matrix A such that Ag ⇠ N(0,C), where

g = (g1 , g2 )T ⇠ g2 .
Exercise 4.2.4. Given a1 , . . . , an 2 R, consider the matrix
⇥ ⇤
C = ai a j 1i, jn .

Find an n ⇥ n matrix A such that Ag ⇠ N(0,C), where g =

(g1 , . . . , gn )T ⇠ gn .
Exercise
⇥ 4.2.5. ⇤Given 0 < t1 < . . . < tn , consider the matrix
C = min(ti ,t j ) 1i, jn . Show that the Gaussian distribution
N(0,C) has density on Rn .
Exercise 4.2.6. Suppose that X = (X1 , . . . , Xn ) ⇠ N(0,C), C
is invertible, and C1i = 0 for i = 2, . . . , n. Prove that X1 and
(X2 , . . . , Xn ) are independent, i.e.

P X1 2 A1 , X2 2 A2 , . . . , Xn 2 An
= P X1 2 A1 P X2 2 A2 , . . . , Xn 2 An .

Exercise 4.2.7. If a21 + . . . + a2n = 1, compute the expectation

E(a1 g1 + . . . + an gn )k for integer k 1.
Exercise 4.2.8. If (g1 , g2 ) is a standard Gaussian random
vector on R2 , compute the density pX (t) of the distribution
of X = g21 + g22 .
164 4 Gaussian Distributions

4.3 Central Limit Theorem

Let us consider random variables X1 , . . . , Xn defined on the

same probability space (W, P), and let us suppose that they
are independent and have distributions P1 , . . . , Pn . As usual,
this means that

P(X1 2 A1 , . . . , Xn 2 An ) = P(X1 2 A1 ) · · · P(Xn 2 An )

= P1 (A1 ) · · · Pn (An ).

Is it possible to construct random variables X1 , . . . , Xn with

these prescribed properties? This can be done using the same
product space construction as in Chapter 1.
In previous chapters we often worked with independent
discrete random variables, but since in this chapter we have
encountered continuous random variables, we can suppose
that some of the random variables X1 , . . . , Xn are discrete and
some are continuous. If a random variable Xi has discrete
distribution, let Wi be the set of all the values that Xi can
take. If Xi has continuous distribution, let Wi = R. Then we
can take the space W to be the product space

W = W1 ⇥ · · · ⇥ Wn , (4.39)

and define the probability P on W by

P(A1 ⇥ · · · ⇥ An ) = P1 (A1 ) · · · Pn (An ), (4.40)

where Ai can be any subset of Wi in the discrete case, and Ai

is a union of intervals when Wi = R. Actually, this definition
only specifies the probabilities of rectangles A1 ⇥ . . . ⇥ An in
the product space W, so how do we compute probabilities of
more general sets? Before we address this, let us first men-
tion that the random variables X1 , . . . , Xn are defined on this
space W as the coordinate functions

Xi (w) = Xi (w1 , . . . , wn ) = wi . (4.41)

4.3 Central Limit Theorem 165

Then

P(X1 2 A1 , . . . , Xn 2 An ) = P(w1 2 A1 , . . . , wn 2 An )
= P(A1 ⇥ · · · ⇥ An ) = P1 (A1 ) · · · Pn (An ),

as we wished. Such construction of random variables as

coordinates on the product space is the simplest way to
construct independent random variables X1 , . . . , Xn with pre-
scribed distributions P1 , . . . , Pn .
To extend (4.40) to more general sets A, we can define the
probability P(A) as an integral over the set A,
Z
P(A) = dP1 (w1 ) · · · dPn (wn ), (4.42)
A

where we understand the integral in the Fubini sense (in-

tegrating
R one coordinate at a time) and where each integral
R the sum  . . . Pi (wi ) when Pi
Wi . . . dPi (wi ) is understood as
is discrete and as the integral . . . pXi (wi ) dwi with respect to
the density of Xi when Pi is continuous. Of course, the set A
should be such that all these consecutive integrals are well
defined.
More generally, an integral of a function f : W ! R on the
above product space (W, P),
Z
Ef = f (w1 , . . . , wn ) dP1 (w1 ) · · · dPn (wn ), (4.43)
W

is also understood in the Fubini sense. We assume that the

function is nice with respect to the continuous coordinates,
so the integrals are well defined, and, as usual, assume that
E| f | < •.
Example 4.3.1. Suppose that e is a Rademacher random
variable, g is a standard Gaussian random variable, and e
and g are independent. Then, if we constructed them as the
coordinates on the product space W = { 1, 1} ⇥ R, we can,
for example, compute
166 4 Gaussian Distributions
Z
1h i
E f (g + e) = f (x 1) + f (x + 1) p(x)dx, (4.44)
R2

where we first averaged over the values ±1 of e and then

integrated with respect to the standard Gaussian distribution
of g.
What if the random variables e and g were defined on
a different probability space instead of the product space?
Then we could still compute probabilities and expectations
using the formula (4.43), as if e and g were defined on the
product space; this is an analogue of the change of variables
formula that we proved for a single random variable (discrete
or continuous), because we integrate over the range of these
random variables with respect to their distributions. The idea
of the proof is, basically, the same but in such generality it is
better left for a more advanced class. t
u
Now that we have seen how to define independent random
variables with arbitrary prescribed distributions on the same
probability space, we are ready to prove the Central Limit
Theorem. There are a number of different proofs of the CLT,
but the proof below via Lindeberg’s method is, perhaps, the
most transparent. It clearly demonstrates that the Gaussian
distribution emerges in the limit due of its stability property
proved in Theorem 4.1 in the last section.
Again, let X1 , . . . , Xn be independent random variables (on
the same space) with distributions P1 , . . . , Pn . Denote

µi = EXi and si2 = Var(Xi ) < •. (4.45)

In other words, we suppose that their variances are finite. We

will also assume that
E|Xi |3
K<• (4.46)
si3
for all i  n for some constant K > 0. Let
4.3 Central Limit Theorem 167

1 n Xi µi
Zn = p  si .
n i=1
(4.47)

When the random variables are i.i.d. then µi = µ, si2 = s 2 ,

and we can write (4.47) as
p
Sn nµ n(X n µ)
Zn = p = , (4.48)
ns s

where Sn = X1 + . . . + Xn and X n = Sn /n. In the i.i.d. case,

the assumption (4.46) reduces to E|X1 |3 < • and with some
more work it can be removed. Here is the main result of this
chapter.
Theorem 4.2 (Central Limit Theorem). If the condition
(4.46) holds then, for all t 2 R,
Z t
1 x2 /2
lim P(Zn  t) = p e dx. (4.49)
n!• 2p •

Since the right hand side is P(g  t) for g ⇠ N(0, 1), this
type of statement is called convergence of distributions, or
convergence of the random variables Zn to g in distribution.
Remark 4.2. In general, we say that a sequence of random
variables Xn converges in distribution to a random variable X
if their cumulative distribution functions Fn (t) = P(Xn  t)
converge to F(t) = P(X  t) for all points on continuity t of
the c.d.f. F(t). Convergence is not required at the points of
discontinuity, where P(X = t) > 0. For example, consider
‘non-random’ random variables Xn = 1/n. They converge
to X = 0 in the usual Calculus sense, so it seems natural
that they should converge in distribution, but P(Xn  0) = 0,
which does not converge to P(X  0) = 1. An equivalent
definition of convergence in distribution is that

lim E f (Xn ) = E f (X)

n!•
168 4 Gaussian Distributions

for all continuous bounded functions f on R. Here, we will

not discuss the general theory of convergence in distribution,
although the argument that shows the equivalence of the two
definitions we mentioned is quite simple and its variant will
appear in the proof of the CLT. t
u
Proof (Theorem 4.2). Before we start the main argument of
the proof, let us reduce the claim to a statement which will
be more suitable to us.
First, since the probability of a set is also an expectation
of its indicator, we can write

P(Zn  t) = E I(Zn  t).

However, instead of working with an indicator, it will be con-

venient to work with a smooth function that approximates
the indicator, because this will allow us to use the Taylor
theorem. Given small e > 0, let us consider a smooth non-
increasing function j(x) such that (see Figure 4.1)

ϕ (x)

t t +ε x

Fig. 4.1 Smooth approximation j(x) of an indicator function I(x  t).

I(x  t)  j(x)  I(x  t + e), (4.50)

and |j 000 (x)|  c, for some constant c that may depend on e.

In other words, j(x) equals 1 for x  t, j(x) equals 0 for
x t + e, and smoothly decreases in between t and t + e.
4.3 Central Limit Theorem 169

Such function is easy to construct, so we will leave it as an

exercise. Then, in order to prove the theorem, all we need to
show is
Z
lim Ej(Zn ) = Ej(g) = j(x)p(x) dx, (4.51)
n!• R

where g is a standard Gaussian random variable and p(x) is

its density,
1 2
p(x) = p e x /2 .
2p
The reason why this is enough is because, by (4.50),

E I(Zn  t)  Ej(Zn )  E I(Zn  t + e).

The first inequality implies, assuming (4.51),

lim sup E I(Zn  t)  lim Ej(Zn )

n!• n!•
Z Z t+e
= j(x)p(x) dx  p(x) dx,
R •

where the last inequality holds because j(x)  I(x  t + e).

The second inequality implies, assuming (4.51),

lim inf E I(Zn  t + e) lim Ej(Zn )

n!• n!•
Z Z t
= j(x)p(x) dx p(x) dx,
R •

where the last inequality holds because j(x) I(x  t).

Since t here is arbitrary, using this with t e gives
Z t e
lim inf E I(Zn  t) p(x) dx.
n!• •

Together these two inequalities imply that E I(Zn  t) is

squeezed in the limit in between
170 4 Gaussian Distributions
Z t e Z t+e
p(x) dx  lim E I(Zn  t)  p(x) dx.
• n!• •

Since e > 0 was arbitrary, letting e # 0 yields (4.49).

We will now focus on proving (4.51). To simplify the no-
tation, let us introduce
Xi µi
Yi = ,
si
so that
1 n
Zn = p  Yi .
n i=1
All the assumptions on Xi ’s translate into

EYi = 0, EYi2 = 1, and E|Yi |3  K < •. (4.52)

Let us now suppose that the random variables X1 , . . . , Xn are

constructed on the same probability space together with i.i.d.
standard Gaussian random variables g1 , . . . , gn . The discus-
sion at the beginning of this section explains how this can be
done. We can just think of g1 , . . . , gn as Xn+1 , . . . , X2n and use
the product space with 2n coordinates instead of n. Consider
the random variable
1 n
g = p  gi . (4.53)
n i=1

By the stability property of Gaussians proved in Theorem

4.1, the distribution of g is standard Gaussian, g ⇠ N(0, 1),
and, in particular, by the change of variables formula in
Lemma 4.3,
⇣ 1 n ⌘ Z
Ej p  gi = j(x)p(x) dx.
n i=1 R

In other words, this choice of g agrees with the notation in

(4.51), where g also denoted standard Gaussian, and, in order
4.3 Central Limit Theorem 171

to prove (4.51), it is enough to show that the difference

⇣ 1 n ⌘ ⇣ 1 n ⌘
Ej p  Yi Ej p  gi (4.54)
n i=1 n i=1

is getting small for large n.

The idea of Lindeberg’s method is to compare
1 n 1 n
Zn = p  Yi and g = p  gi
n i=1 n i=1
p p
by ‘taking small steps’ and by replacing Yi / n with gi / n
one by one, as follows. This is where the stability property
of Gaussians was so crucial, allowing us to represent g as a
sum of similar small increments. For 1  i  n + 1, we define
1
Ti = p g1 + . . . + gi 1 +Yi + . . . +Yn ,
n
so that at the first step T1 = Zn and at the last step Tn+1 = g.
Then we can write (4.54)
n
Ej(Zn ) Ej(g) = Â Ej(Ti ) Ej(Ti+1 )
i=1

as a telescoping sum of ‘small increments’

p becausepTi differs
from Ti+1 only in one term, Yi / n instead of gi / n. They
share all the other terms, which we will denote by
1
Si = p g1 + . . . + gi 1 +Yi+1 + . . . +Yn ,
n
so that the consecutive terms are
Yi gi
Ti = Si + p , Ti+1 = Si + p .
n n
The telescoping sum representation implies that
172 4 Gaussian Distributions

n
Ej(Zn ) Ej(g)  Â Ej(Ti ) Ej(Ti+1 ) .
i=1

Since |j 000 (x)|  c, if we expand j(Ti ) and j(Ti+1 ) around

Si using the Taylor polynomials of order 2, we get

Yi Yi2 c|Yi |3
j(Ti ) j(Si ) j (Si ) p
0
j (Si )
00
 3/2 ,
n 2n 6n
gi g2i c|gi |3
j(Ti+1 ) j(Si ) j (Si ) p j (Si )
000
 3/2 .
n 2n 6n
Now we open the absolute values by writing b  a  b
instead of |a|  b and take expectations. When we take the
expectations of the first and second order terms,
h Yi i h 00 Yi2 i h 0 gi i h 00 g2i i
E j (Si ) p , E j (Si )
0
, E j (Si ) p , E j (Si ) ,
n 2n n 2n
we notice that Si is a function of the coordinates that do not
include Yi and gi and, by independence and the Fubini for-
mula (4.43),
h Yi i Yi
E j (Si ) p = Ej 0 (Si )E p = 0,
0
n n
since EYi = 0, and
h gi i gi
E j (Si )
0
p = Ej 0 (Si )E p = 0,
n n
since Egi = 0. As we can see, because we subtracted the
expectation in Yi = (Xi µi )/si , the first order terms in the
Taylor expansions match. Similarly,
h Yi2 i Yi2 1
E j (Si )
00
= Ej (Si )E = Ej 00 (Si )
00
2n 2n 2n
since EYi2 = 1, and
4.3 Central Limit Theorem 173

h g2i i g2i 1
E j (Si )
00
= Ej (Si )E = Ej 00 (Si )
00
2n 2n 2n
since Eg2i = 1. Because we scaled by si in Yi = (Xi µi )/si ,
the second order terms in the Taylor expansions match. The
zeroth order terms were the same, Ej(Si ), and, as a result,
the difference is only in the third order error terms,
c(E|Yi |3 + E|gi |3 )
Ej(Ti ) Ej(Ti+1 )  .
6n3/2
By (4.52), E|Yi |3  K, and E|gi |3 is the same for all i  n, so
it is just another constant and
c0
Ej(Ti ) Ej(Ti+1 ) 
n3/2
for some constant c0 . These third order terms are of smaller
order 1/n3/2 than the number of ‘steps’ in the telescoping
sum, n, and we finally get
n
c0
Ej(Zn ) Ej(g)  Â Ej(Ti ) Ej(Ti+1 )  p .
i=1 n

This finishes the proof. t

u
Example 4.3.2. Suppose that all we know about the check-
out time at a grocery store is that it has a mean of µ = 5
minutes and a standard deviation of s = 2 minutes. Let us
estimate the probability that a checker will serve at least 50
customers during her 4-hour shift. In other words, what is
the probability that 50 customers will be served in under 240
minutes? Let Ti be the time it takes to serve the ith customer,
and Sn = Âni=1 Ti . Then S50  240 is equivalent to
S50 50 · 5 240 50 · 5
Z50 = p  p ⇡ 0.7.
2 50 2 50
174 4 Gaussian Distributions

By the central limit theorem, the probability of this event can

be approximated by
Z 0.7
1 x2 /2
P(Z50  0.7) ⇡ p e dx ⇡ 0.242.
2p •

So, we can estimate the chance to be about 24%. t

u
Sometimes one is interested in a sequence of indepen-
dent random variables X1 , . . . , Xn such that all their distri-
butions also depend on n. In other words, for each n, we
consider Xn1 , . . . , Xnn , which are independent but, otherwise,
the whole sequence varies with n. This setting is called the
triangular array. Let us denote
n
µni = EXni , sni2 = Var(Xni ) < •, D2n = Â sni2 . (4.55)
i=1

Instead of (4.47), we will consider a different rescaling,

Sn ESn 1 n
Zn = p = Â Xni
Var(Sn ) Dn i=1
µni . (4.56)

For i.i.d. sequences, the two definitions of Zn are the same.

We still have EZn = 0 and Var(Zn ) = 1, but the variance of
each summand, sni2 /D2n , is not necessarily 1/n as was the
case in (4.47). When does Zn satisfy the CLT?
One condition that ensures this is
1 n 3
lim
n!• D3
 E Xni
n i=1
µni = 0, (4.57)

which is called the Lyapunov condition.

Theorem 4.3 (CLT for triangular arrays). The central limit
theorem (4.49) holds for the triangular arrays that satisfy the
Lyapunov condition (4.57).
The proof is almost identical to the above, so we will leave it
as an exercise. One small change is to consider ani = sni /Dn
4.3 Central Limit Theorem 175

and, instead of (4.53), represent g = Âni=1 ani gi . At some

point, when controlling one of the error terms in the Taylor
expansion, one also has to use that
⇣ ⌘
3 2 3/2 3
sni = E Xni µni  E Xni µni ,

which follows from Jensen’s inequality.

Example 4.3.3 (Records in a random permutation). Given
a permutation (p1 , . . . , pn ) of {1, . . . , n}, we say that pk is a
record if it is greater than the preceding values,

pk > pi for i = 1, . . . , k 1.

For example, in a permutation (3, 5, 2, 6, 4, 1), there are three

records: p1 = 3, p2 = 5 and p4 = 6. We will show that the
number Nn of records in a random permutation of {1, . . . , n}
satisfies the central limit theorem:
⇣ N ln n ⌘ Z t
n 1 2
lim P p t = p e x /2 dx. (4.58)
n!• ln n 2p •
Let us consider the indicator that pk is a record,
⇢
1, if pk is a record,
rk =
0, otherwise.
Then Nn = Ânk=1 rk . In order to apply Theorem 4.3, we will
prove that r1 , . . . , rn are independent random variables and
⇣1⌘
rk ⇠ B for k  n. (4.59)
k
This might be counterintuitive, because, observing many
records at the beginning of the permutation might suggest
that the chances of further records are affected, but this is
not the case. We will check this carefully, but the basic idea
is that, no matter what the values {p1 , . . . , pk } are, only their
order and not their values determines the records (r1 , . . . , rk ),
176 4 Gaussian Distributions

so knowing the records in the first k positions gives us no

information about the values.
Our goal is to show that, for any x1 , . . . , xn 2 {0, 1},
⇣ 1 ⌘xk ⇣
n
1 ⌘1 xk
P r1 = x1 , . . . , rn = xn =’ 1 .
k=1 k k

Equivalently, if we multiply both sides by n! and denote by

#(A) the number of permutations in the set A, we want to
show that
n
# r1 = x1 , . . . , rn = xn = ’ (k 1)1 xk
. (4.60)
k=1

It is easier to explain the proof of this formula on a specific

example. Suppose that n = 6 and the sequence of records is

(r1 , r2 , r3 , r4 , r5 , r6 ) = (1, 1, 0, 1, 0, 0).

First of all, since the last record is at k = 4, we must have

p4 = n = 6, because 6 will always be a record and there can
be no records after 6. Second, once we know that p4 = 6, the
numbers p5 and p6 can be anything but 6. There are 5 ⇥ 4
ways to choose these numbers. Third, no matter what p1 , p2
and p3 are, the number of ways to arrange them that will
result in (r1 , r2 , r3 ) = (1, 1, 0) is the same; in other words,
the specific values do not matter, since we simply compare
which one is bigger than the other. This is a good place to
use induction. If we assume that we already proved the above
formula for n = 3, the number of permutations of any three
distinct numbers that result in (r1 , r2 , r3 ) = (1, 1, 0) is

#(r1 = 1, r2 = 1, r3 = 0) = 00 ⇥ 10 ⇥ 21 = 2.

Multiplying this by 4 ⇥ 5 ways to choose p5 and p6 gives

2 ⇥ 4 ⇥ 5, which agrees with the formula (4.60). The proof
of the general case is exactly the same.
4.3 Central Limit Theorem 177

To apply Theorem 4.3, we need to compute the mean and

variance of Nn = Ânk=1 rk , and check the Lyapunov condition
(4.57). Since the mean and variance of the Bernoulli B(p)
random variable is p and p(1 p), and the random variables
r1 , . . . , rn are independent,
1n n
1⇣ 1⌘
ENn = Â , Var(Nn ) = Â 1 .
k=1 k k=1 k k

In particular, both quantities are within a constant of ln n,

ENn ln n  c, Var(Nn ) ln n  c, (4.61)

and D3n = Var(Nn )3/2 ⇠ (ln n)3/2 . Next,

n
1 3 n ⇣k 1 ⌘3 1 n ⇣ ⌘3
1 k 1
 E rk k
= Â k
+Â
k k=1 k k
.
k=1 k=1

The second sum is bounded by a constant, and the first sum is

bounded by ENn  ln n + 1. This implies that the Lyapunov
condition (4.57) is satisfied and, therefore,
⇣ N EN ⌘ Z t
n n 1 2
lim P p t = p e x /2 dx.
n!• Var(Nn ) 2p •
Together with (4.61), this implies (4.58). t
u
Exercise 4.3.1. If the random variables X1 , . . . , Xn are i.i.d.
Bernoulli
p B(p), what is the limit limn!• P |X n p| tn if
limn!• ntn = t 2 [0, •)?

p with mean l > 0,

Exercise 4.3.2. If X ⇠ Poiss(l ) is Poisson
what is the limit liml !• P |X l | a l for a > 0? Hint:
recall the stability property of Poisson.
Exercise 4.3.3. 24% of the residents in a community are
members of a minority group but among the 96 people called
for jury duty only 13 are. Does this data indicate that minori-
ties are less likely to be called for jury duty?
178 4 Gaussian Distributions

Exercise 4.3.4. Members of the ASF fraternity each drink a

random number of beers with mean 6 and standard deviation
3. If there are 81 fraternity members, how much should they
buy so that using the normal approximation they are 95%
sure they will not run out?
Exercise 4.3.5. Prove Theorem 4.3.
Exercise 4.3.6. Consider a triangular array of i.i.d. Bernoulli
Xn1 , . . . , Xnn ⇠ B(pn ), where pn varies with n. If pn ! 0 but
npn ! +•, show that Zn in (4.56) satisfies the CLT. Check
that the condition (4.46) fails.
Exercise 4.3.7. Let X1 , X2 , . . . be independent random vari-
ables such that Xi ⇠ B(pi ) for some pi 2 [0, 1]. If the series
•i=1 pi (1 pi ) diverges, show that Zn in (4.56) satisfies the
CLT.
Exercise 4.3.8. Let X1 , X2 , . . . be independent random vari-
ables such that P(Xi = ±ia ) = 12 for some a > 12 . Show
that Zn in (4.56) satisfies the CLT.
4.4 Distributions Related to Gaussian 179

4.4 Distributions Related to Gaussian

In this section we will introduce several distributions related

to Gaussian: Gamma distribution, chi-squared distribution,
F-distribution, and Student’s t-distribution. This will give us
another chance to study various transformations of random
variables (such as sums and ratios), but the main reason we
discuss these distributions is their central role in the analysis
of Simple Linear Regression in the next section.
Gamma distribution. Let us recall the definition of the
Gamma function G(a) for a > 0,
Z •
G(a) = xa 1
e x
dx. (4.62)
0

If we divide both sides by G(a) and, given b > 0, make the

change of variables x = b y, we get
Z •
1 a 1 x
1= x e dx
0 G(a)
Z • a
b
= ya 1
e by
dy.
0 G(a)
Therefore, if we define
(
ba a 1 bx
f (x; a, b ) = G(a) x e , x>0
(4.63)
0, x0

then f (x; a, b ) is a probability density function, since it is

nonnegative and integrates to one. The distribution with the
density f (x; a, b ) is called Gamma distribution with param-
eters a and b , and it is denoted by G(a, b ).
Next, let us recall some properties of G(a). If a > 1 then,
using integration by parts,
180 4 Gaussian Distributions
Z • Z •
G(a) = xa 1
e x dx = xa 1
d( e x )
0 Z 0•
•
a 1
=x ( e x) ( e x )(a 1)xa 2
dx
Z • 0 0
1) 1
= (a 1) x(a e x dx = (a 1)G(a 1).
0
R•
Since G(1) = 0 e x dx = 1, by induction, we get that

G(n) = (n 1)!

for integer n 1. Using G(a) = (a 1)G(a 1), one can

easily check the following.
Exercise 4.4.1. Show that the kth moment of X ⇠ G(a, b )
equals
(a + k 1) · · · a
EX k = .
bk
In particular, the expectation, second moment and variance
are
a (a + 1)a a
EX = , EX 2 = , Var(X) = .
b b2 b2
The most important property of Gamma distribution to us
will be the following stability property.
Lemma 4.5. If the random variables X1 , . . . , Xn are indepen-
dent and X1 ⇠ G(a1 , b ), . . . , Xn ⇠ G(an , b ) then their sum
X1 + . . . + Xn has distribution G(a1 + . . . + an , b ).
Notice that the scaling parameter b is the same. This lemma
is based on the classical convolution formula for the density
of the sum of two independent random variables whose dis-
tributions have densities.
Lemma 4.6. If X1 , X2 are independent and their distributions
have densities f (x), g(x) then X1 + X2 has density
Z •
h(x) = f (x y)g(y) dy. (4.64)
•
4.4 Distributions Related to Gaussian 181

Proof. Let us compute the cumulative distribution function

of X1 + X2 . By independence and the Fubini representation
(4.42) in the previous section,
ZZ
P(X1 + X2  t) = I(x + y  t) f (x)g(y) dxdy
2
ZZR
= I(x  t y) f (x)g(y) dxdy
2
Z •R hZ t y i
= f (x)g(y) dx dy
Z •• hZ t• i
{x = z y} = f (z y)g(y) dz dy
Z t• hZ •• i
{switch order} = f (z y)g(y) dy dz
Z t• •

= h(z) dz,
•

where h(x) coincides with (4.64). This shows that h is the

density of X1 + X2 and finishes the proof. t
u
Using this convolution formula, we can prove Lemma 4.5.
Proof (Lemma 4.5). Let us start with X1 ⇠ G(a1 , b ) and X2 ⇠
G(a2 , b ) and compute the distribution of their sum X1 + X2 .
Since

f (xy; a1 , b ) f (y; a2 , b )
b a1 +a2
= e b x (x y)a1 1 a2 1
y
G(a1 )G(a2 )
when both x y 0 and y 0, and 0 otherwise, by (4.64),
the density of X1 + X2 is
Z x
b a1 +a2 bx
h(x) = e (x y)a1 1 a2 1
y dy.
G(a1 )G(a2 ) 0

Making the change of variables y = xz, this equals

182 4 Gaussian Distributions
Z 1
b a1 +a2
xa1 +a2 1
e bx
(1 z)a1 1 a2 1
z dz.
G(a1 )G(a2 ) 0

The factors xa1 +a2 1 e b x that depend on x look exactly like

the ones in the Gamma density f (x; a1 + a2 , b ) and, because
h(x) is a density and must integrate to 1, we must have
b a1 +a2 a1 +a2 1 bx
h(x) = f (x; a1 + a2 , b ) = x e .
G(a1 + a2 )
This proves the claim in the case of two Gamma random
variables. As a byproduct, we also showed that
Z 1
G(a1 )G(a2 )
(1 z)a1 1 a2 1
z dz = .
0 G(a1 + a2 )
This integral is known as the Beta function B(a1 , a2 ), so
G(a1 )G(a2 )
B(a1 , a2 ) = .
G(a1 + a2 )
For the sum of more than two, one can proceed by induction
on n, although it is a good idea to check the details. t
u
Exercise 4.4.2. Complete the induction step at the end of the
above proof. Hint: use the change of variables formula (think
about Remark 4.1), or consider the proof of the convolution
formula above.
One can also use the convolution formula to prove the
Gaussian stability property in Theorem 4.1.
Exercise 4.4.3. Prove Theorem 4.1 using the convolution
formula (4.64). Hint: Recall notation in Example 4.1.6, and
check that
Z •
ps (x) = ps1 (x y)ps2 (y) dy,
•

where s 2 = s12 + s22 . Then use induction.

4.4 Distributions Related to Gaussian 183

Chi-squared distribution. If g1 , . . . , gn are i.i.d. standard

Gaussian random variables then the distribution of the sum

g21 + . . . + g2n (4.65)

is called the chi-squared (or chi-square) distribution with n

degrees of freedom, and it is denoted cn2 . In the Example
4.1.5 in Section 4.1, we showed that the square X = g2 of a
standard Gaussian random variable g has density
1 1 1 x
pX (x) = p x 2 e 2 I(x > 0).
2p
Comparing with the definition of the Gamma distribution in
(4.63), wepcan see that ppX (x) is the density of G( 12 , 12 ). The
constantsp 2G( 12 ) and 2p have no choice but to agree, so
G( 12 ) = p.
By the stability property of the Gamma distribution in
Lemma 4.5, we see that
⇣n 1⌘
2
cn = G , . (4.66)
2 2
This is a good place to mention one more time that here we
implicitly use the change of variables formula in Lemma 4.3
in Section 4.2. When we compute, for example, the c.d.f.
P(g21 + . . . + g2n  t), by definition, we integrate
Z
I(x12 + . . . + xn2  t)p(x1 ) · · · p(xn ) dx1 · · · dxn ,
Rn

where p(x) is the standard Gaussian density. The change of

variables formula in Lemma 4.3 allows us to replace
Z Z
. . . p(xi ) dxi by . . . f (ti ) dti ,

where f (ti ) is the G(1/2, 1/2) density f (ti ; 1/2, 1/2), by

making the change of variables ti = xi2 and using that, for
g ⇠ N(0, 1), the distribution of g2 is G(1/2, 1/2). Once the
184 4 Gaussian Distributions

integral is rewritten in terms of the Gamma densities,

Z
I(t1 + . . . + tn  t) f (t1 ) · · · f (tn ) dt1 · · · dtn ,
Rn

only then we apply the stability property in Lemma 4.5.

F-distribution. Consider two independent random variables
with the distributions
⇣k 1⌘ ⇣m 1⌘
2 2
X ⇠ ck = G , and Y ⇠ cm = G , .
2 2 2 2
The distribution of the ratio
X/k
Z= (4.67)
Y /m
is called F-distribution with degrees of freedom k and m, and
is denoted by Fk,m . (It is also known as the Fisher–Snedecor
distribution.) In other words, if

g1 , . . . , gk , g̃1 , . . . , g̃m

are i.i.d. standard Gaussian, then

m g21 + . . . + g2k
k g̃21 + . . . + g̃2m
has distribution Fk,m . We will show that the density of Fk,m is

G( k+m
2 ) k k+m
fk,m (x) = kk/2 mm/2 x 2 1
(m + kx) 2 (4.68)
G( 2k )G( m2 )
for x > 0, and zero otherwise. We will need the following
lemma.
Lemma 4.7. If two positive random variables X,Y > 0 have
densities f (x) and g(x), then Z = X/Y has density
Z •
h(x) = f (xy)g(y)y dy for x > 0. (4.69)
0
4.4 Distributions Related to Gaussian 185

Proof. For t > 0,

Z • hZ ty i
P(Z  t) = P(X  tY ) = f (x)g(y) dx dy
0 0
Z • hZ t i
{x = zy} = f (zy)g(y)y dz dy
Z0t hZ 0• i
{switch order} = f (zy)g(y)y dy dz
Z0t 0

= h(z)dz,
0

where h(x) coincides with (4.69). This finishes the proof. t

u
To compute the density of (4.67), let us first compute the
density of mk Z = YX . The densities of X and Y are
k m
( 12 ) 2 k
1 1 ( 12 ) 2 m 1
f (x) = x 2 e 2x and g(y) = m y 2 1
e 2y
G( 2k ) G( 2 )

correspondingly, for x > 0 and y > 0. By the previous lemma,

the density of X/Y is
Z • 1 2k m
(2) k
1 1 ( 12 ) 2 m 1
fX/Y (x) = (xy) 2 e 2 xy y2 1
e 2y y dy
0 G( 2k ) G( m2 )
k+m Z •
( 12 ) 2 k
1 k+m
1 1
= x 2 y 2 e 2 (x+1)y dy.
G( 2k )G( m2 ) 0

If we make the change of variables z = 12 (x + 1)y, we get

Z •
1 k
1 k+m k+m
1 z
fX/Y (x) = k x 2 (1 + x) 2 z 2 e dz
G( 2 )G( m2 ) 0

G( k+m
2 ) k
1 k+m
= x2 (1 + x) 2 ,
G( 2k )G( m2 )
where we used (4.62). Finally, we obtain (4.68) using that
186 4 Gaussian Distributions
⇣ kx ⌘ k
fZ (x) = fX/Y .
m m
Student’s t-distribution. If g0 , g1 , . . . , gn are i.i.d. standard
Gaussian then the distribution of
g0
T=q (4.70)
1 2 2
n (g1 + . . . + gn )

is called the t-distribution with n degrees of freedom, and it

is denoted by tn .
To compute the density fT (x) of T, we only need to notice
that T 2 has F1,n -distribution and that the distribution of T is
symmetric, so fT (x) = fT ( x). If we write, for t > 0,

2P(0  T  t) = P( t  T  t) = P(T 2  t 2 )

as integrals of the densities, we get

Z t Z t2
2 fT (x) dx = f1,n (x) dx.
0 0

Making the change of variables x = y2 on the right hand side,

Z t Z t
2 fT (x) dx = f1,n (y2 )2y dy.
0 0

Taking derivatives of both side with respect to t, for t > 0,

2 G( n+1
2 )1 ⇣ t2 ⌘ n+1
2
fT (t) = f1,n (t )t = 1 p 1+ . (4.71)
G( 2 )G( n2 ) n n

Since fT ( t) = fT (t), the same formula holds for all t 2 R.

It is not difficult to show that
1 t 2 /2
lim fT (t) = p(t) = p e , (4.72)
n!• 2p
4.4 Distributions Related to Gaussian 187

but we will skip the technical details here. Intuitively this

should be true, because the denominator in (4.70) is close
to 1 by the law of large numbers, so the random variable T
should be close to a standard Gaussian in distribution (see
exercise below).
Exercise 4.4.4. If X ⇠ G(a, b ), show that, for t < b ,
⇣ b ⌘a
EetX = .
b t
Exercise 4.4.5. If X and Y are independent random variables
with the densities pX (x) = pY (x) = I(0  x  1) (uniform on
[0, 1]), compute the density of their sum X +Y.
Exercise 4.4.6. If X ⇠ cn2 and P(X  cn ) = p 2 (0, 1), prove
that limn!• cnn = 1. Hint: Represent X as g21 + . . . + g2n and
use the law of large numbers.
Exercise 4.4.7. If X ⇠ Fk,m and Y ⇠ Fm,k , show that, for any
c > 0, P(X  c) = P(Y 1/c).
Exercise 4.4.8. If X ⇠ tn , show that E|X|a < • for a < n and
and E|X|a = • (undefined) for a n.
Exercise 4.4.9. In the notation of (4.70), show that

lim P(T  t) = P(g0  t)

n!•

for all t 2 R. Hint: use the Law of Large Numbers for the
denominator in (4.70).
188 4 Gaussian Distributions

4.5 Simple Linear Regression

Simple linear regression (SLR) is a classical statistical model

that deals with the data of the form

(X1 ,Y1 ), . . . , (Xn ,Yn ), (4.73)

where Xi ,Yi 2 R, and one expects or observes a roughly linear

dependence of the variable Y on X. For example, here is a
graph of carbon monoxide content (mg) vs. nicotine content
(mg) in 25 brands of cigarettes.
35

30
Carbon monoxide

25

20

15

10

0
Nicotine
−5
0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Fig. 4.2 Carbon monoxide content (mg) vs. nicotine content (mg). Solid line: least-
squares line.

The SLR model is

Yi = b0 + b1 Xi + ei , (4.74)

where the function

y = f (x) = b0 + b1 x (4.75)

is called a regression line (whose parameters b0 and b1

are unknown) and where e1 , . . . , en represent noise modelled
by i.i.d. Gaussian N(0, s 2 ) random variables with unknown
variance s 2 . In this model, variable X is called the predictor,
or independent variable, and Y is called the response, or de-
4.5 Simple Linear Regression 189

pendent variable. We will assume that not all Xi ’s are equal,

because it does not make sense to use a linear function as a
model in that case.
We have three unknown parameters, b0 , b1 , and s 2 , and
we want to estimate them using the sample (4.73). Values
X1 , . . . , Xn can be either random or non random, but from the
point of view of the SLR model they are treated as fixed and
non random, and it is assumed that the randomness comes
entirely from the noise variables ei . Another way to look at
it is that each Yi has Gaussian distribution N(b0 + b1 Xi , s 2 )
with the density
1 1
b0 b1 Xi )2
fi y; b0 , b1 , s 2 = p
(y
e 2s 2
2ps
with three unknown parameters b0 , b1 , s 2 , and that Y1 , . . . ,Yn
are independent.
In Statistics, one of the most important methods of esti-
mation of unknown parameters is the Maximum Likelihood
Estimation. Consider the likelihood function
n
`(b0 , b1 , s ) := ’ fi Yi ; b0 , b1 , s 2 ,
2
(4.76)
i=1

which is the joint density of the random vector (Y1 , . . . ,Yn )

evaluated at the observed values Y1 , . . . ,Yn . We maximize it
over the unknown parameters,

`(b0 , b1 , s 2 ) ! maximize . (4.77)

b0 ,b1 ,s 2

If the maximum is achieved on some b̂0 , b̂1 , ŝ 2 , these values

are called the maximum likelihood estimates (MLE) of the
unknown parameters. In our case,
1 1
Âni=1 (Yi b0 b1 Xi )2
`(b0 , b1 , s 2 ) = n e 2s 2 .
(2p) 2 s n
190 4 Gaussian Distributions

First of all, no matter what s is, we need to minimize

n
L := Â (Yi b0 b1 Xi )2 (4.78)
i=1

over b0 , b1 . The parameters that minimize L correspond to

the famous least-squares line. Let us find critical points:
n
∂L
∂ b0
= Â 2(Yi (b0 + b1 Xi )) = 0,
i=1
n
∂L
∂ b1
= Â 2(Yi (b0 + b1 Xi ))Xi = 0.
i=1

If we introduce the notation

1 n 1 n 1 n 2 1 n
X = Â Xi , Y = Â Yi , X = Â Xi , XY = Â XiYi
2
n i=1 n i=1 n i=1 n i=1

then the critical point conditions can be rewritten as

b0 + b1 X = Y ,
b0 X + b1 X 2 = XY .

Solving for b0 and b1 we obtain the MLE,

XY XY
b̂1 := , b̂0 := Y b̂1 X. (4.79)
X2 X2
Since we assumed that not all Xi ’s are equal, the denominator

sx2 := X 2 X 2 6= 0. (4.80)

It is called the sample variance of X1 , . . . , Xn .

Then we can maximize `(b̂0 , b̂1 , s 2 ) over s 2 . Ignoring the
constant (2p)n/2 and taking the logarithm, we would like to
maximize
 4.5 Simple Linear Regression 191

1 n
n log s  (Yi
2s 2 i=1
b̂0 b̂1 Xi )2 .

Setting the derivative in s to zero, we get the MLE

1 n
ŝ 2 := Â (Yi
n i=1
b̂0 b̂1 Xi )2 . (4.81)

Interestingly, up to a factor 1/n, the estimate ŝ 2 turned out

to be the minimum value of the function in (4.78) that we
minimized in order to find b̂0 and b̂1 ,
n
1
ŝ = min  (Yi
2
b0 b1 Xi )2 . (4.82)
n b0 ,b1 i=1

This observation will play a crucial role below. In order to

understand how these MLE estimates relate to the actual un-
known parameters b0 , b1 and s 2 , we will need to figure out
their joint distribution.
Theorem 4.4. The joint distribution of MLE (b̂0 , b̂1 , ŝ 2 ) is
described by the following properties.
(a) The random vector (b̂0 b0 , b̂1 b1 ) is Gaussian N(0,C),
where the covariance matrix

s 2 sx2 + X 2 X
C= . (4.83)
nsx2 X 1

(b) ŝ 2 is independent of b̂0 and b̂1 .

2
(c) nsŝ2 has cn2 2 -distribution with n 2 degrees of freedom.
Notice that, if the sample variance sx2 stays bounded with n
then the variances of b̂0 b0 and b̂1 b1 are of order 1/n, by
(4.83). Chebyshev’s inequality tells us that the estimates b̂ j
are close to unknown parameters b j with probability close
to one. However, we will do even better than Chebyshev’s
inequality by constructing precise confidence intervals for
these unknown parameters below.
192 4 Gaussian Distributions

Proof. First of all, let us recenter the estimates b̂0 and b̂1 and
work with
ĝ0 = b̂0 b0 , ĝ1 = b̂1 b1 . (4.84)
Let us also recall the SLR model equation (4.74), because it
will be more convenient to work with the random variables
ei instead of Yi , since, by definition, we can think of them as

e = (e1 , . . . , en )T = s g = s (g1 , . . . , gn )T ,

where g = (g1 , . . . , gn )T ⇠ gn has the standard Gaussian dis-

tribution on Rn . We leave it as a (simple) exercise to check
that, with this recentering, the MLE equations (4.79) and
(4.81) are equivalent to
Xe Xe
ĝ1 = , ĝ0 = e ĝ1 X (4.85)
sx2
and
1 n
ŝ = Â (ei
2
ĝ0 ĝ1 Xi )2 . (4.86)
n i=1
Let us also record the analogue of (4.82),
n
1
ŝ = min  (ei
2
g0 g1 Xi )2 . (4.87)
n g0 ,g1 i=1

The main idea will be to rewrite all the quantities above

in terms of Z = Ag for some orthogonal matrix A such that
ĝ0 , ĝ1 will be linear combinations of the first two coordinates
Z1 , Z2 of Z and ŝ 2 will depend only on Z3 , . . . , Zn . Using the
fact that an orthogonal transformation Z = Ag of a standard
Gaussian vector g on Rn is also standard Gaussian, all the
claims will follow immediately from this representation in a
new orthogonal basis.
Let us rewrite ĝ0 and ĝ1 as follows. Let us represent e in
terms of the scalar product
4.5 Simple Linear Regression 193

e1 + . . . + en 1
e= = p (~a1 , e), (4.88)
n n
where the vector
⇣ 1 1 ⌘T
T
~a1 = (a11 , . . . , a1n ) = p , . . . , p .
n n
Similarly, let us represent

√
nσ̂

!a2
γ̂0!1 + γ̂1 !X
!a1

Fig. 4.3 A decomposition of e into orthogonal components.

Xe Xe 1
ĝ1 = =p (~a2 , e), (4.89)
sx2 nsx
where the vector
1 T
~a2 = (a21 , . . . , a2n )T = p X1 X, . . . , Xn X .
nsx
We can write
1 X
ĝ0 = p (~a1 , e) p (~a2 , e). (4.90)
n nsx
Notice that both vectors ~a1 and ~a2 have length 1, and they
are orthogonal to each other,
194 4 Gaussian Distributions

n
Xi X
|~a1 | = |~a2 | = 1, (~a1 ,~a2 ) = Â = 0.
i=1 nsx

Next, if we consider vectors

~1 = (1, . . . , 1)T and ~X = (X1 , . . . , Xn )T ,

then the equations (4.86) and (4.87) can be written as

1 2
ŝ 2 = e ĝ0~1 ĝ1~X
n
1 2
= min e g0~1 g1~X .
n g0 ,g1
In particular, this implies that
2
ĝ0 , ĝ1 = argmin e g0~1 g1~X . (4.91)
g0 ,g1

This equation states that ĝ0~1 + ĝ1 ~X is the closest point to

e in the subspace spanned by ~1 and ~X, so it must coincide
with the orthogonal projection of e onto this subspace. On
the other hand, this subspace is also spanned by ~a1 and ~a2 ,
Span{~1, ~X} = Span{~a1 ,~a2 }, because

1 1 ~ X
~a1 = p ~1, ~a2 = p X p ~1,
n nsx nsx
and p p
~1 = n~a1 , ~X = n X ~a1 + sx ~a2 .
Since ~a1 ,~a2 form an orthonormal basis of this subspace, the
orthogonal projection of e onto this subspace is

ĝ0~1 + ĝ1~X = (~a1 , e)~a1 + (~a2 , e)~a2 . (4.92)

Another crucial consequence of this observation is that, by

the Pythagorean theorem,
4.5 Simple Linear Regression 195

|e|2 = |e ĝ0~1 ĝ1~X|2 + (~a1 , e)2 + (~a2 , e)2

= nŝ 2 + (~a1 , e)2 + (~a2 , e)2 . (4.93)

All the claims now follow from the fact that an orthogonal
transformation of the standard Gaussian vector (g1 , . . . , gn )
is standard Gaussian. Let us choose vectors ~a3 , . . . ,~an so that
~a1 , . . . ,~an is an orthonormal basis of Rn and the matrix A
with rows ~aT1 , . . . ,~aTn ,
2 3
a11 a12 · · · a1n
6a21 a22 · · · a2n 7
A=6 7
4 ... ... . . . ... 5 ,
an1 an2 · · · ann
is orthogonal. By Lemma 4.4 in Section 4.2, we know that

Z = (Z1 , . . . , Zn )T = Ag ⇠ gn

and, since e = s g, we have s Z = Ae. With this notation, the

equations (4.89) and (4.90) become
s sX
ĝ0 = p Z1 p Z2 . (4.94)
n nsx
s
ĝ1 = p Z2 , (4.95)
nsx

and, using that s 2 |Z|2 = |Ae|2 = |e|2 , the equation (4.93)

becomes

s 2 |Z|2 = nŝ 2 + s 2 Z12 + s 2 Z22 , (4.96)

which can be rewritten as

nŝ 2
= |Z|2 Z12 Z22 = Z32 + . . . + Zn2 . (4.97)
s 2

This representation proves claim (c), by the definition of

cn2 2 -distribution. Claim (b) holds because the coordinates
196 4 Gaussian Distributions
2
of Z are i.i.d., nsŝ2 is a function of Z3 , . . . , Zn , while ĝ0 and ĝ1
are functions of Z1 , Z2 . If we denote

s sx X
B= p ,
nsx 0 1
the system of equations (4.94), (4.95) can be written as
(ĝ0 , ĝ1 )T = B(Z1 , Z2 )T and, therefore, (ĝ0 , ĝ1 )T ⇠ N(0,C)
with the covariance
s 2  2
sx + X 2 X
C = BBT = .
nsx2 X 1
This finishes the proof. t
u

Confidence intervals for parameters of SLR. Next, using

Theorem 4.4, we will construct the confidence intervals for
unknown parameters of the SLR model, b0 , b1 and s 2 . For
example, we showed that
nŝ 2 ⇣n 1⌘
⇠ cn2 2 =G 1, ,
s 2 2 2
and the density of cn2 2 is
⇣ n 1⌘ 1 n x
1
f x; 1, = n 1 G( n
x2 e 2 .
2 2 22 2 1)
For n > 0, the density is 0 at x = 0 and goes to 0 as x ! •,
and it is easy to check that its unique maximum is at x =
n 2. This means that the most unlikely outcomes x under
this distribution (having smallest density) are near x = 0 and
x = •. To select more likely values in the middle, a common
choice is to take a confidence level a < 1, say a = 0.95, and
take constants c1 and c2 such that
1 a 1 a
cn2 2 (0, c1 ) = and cn2 2 (c2 , +•) = . (4.98)
2 2
4.5 Simple Linear Regression 197

0.4

0.35

0.3

0.25

0.2

0.15

0.1
1−α
0.05 1−α 2
2
0
0 c1 5 10 c2 15 20 25

Fig. 4.4 Tails of chi-squared cn2 2 -distribution.

Then the interval [c1 , c2 ] has cn2 2 probability a,

⇣ nŝ 2 ⌘
P c1  2  c2 = a.
s
Solving this for s 2 , we find that the event
nŝ 2 2 nŝ 2
s  (4.99)
c2 c1
occurs with probability a. This interval [nŝ 2 /c2 , nŝ 2 /c1 ] is
called the a confidence interval for the parameter s 2 .
Similarly, we can find the a confidence intervals for b1
and b0 . By Theorem 4.4,
⇣ ns 2 ⌘1/2 nŝ 2
x
(b̂1 b1 ) ⇠ N(0, 1) and 2 ⇠ cn2 2 ,
s 2 s
and the two random variables are independent. Therefore, by
the definition of t-distribution, the ratio
⇣ ns 2 ⌘1/2 .⇣ 1 nŝ 2 ⌘1/2
x
(b̂1 b1 )
s2 n 2 s2
has Student’s tn 2 -distribution with n 2 degrees of free-
dom. Simplifying, we get
198 4 Gaussian Distributions
p
n 2sx
(b̂1 b1 ) ⇠ tn 2 .
ŝ
Notice how the unknown parameter s 2 cancelled out. The

0.4

0.3

0.2

1−α 1−α
0.1 2 2

0
−6 −4 −c 0 c 4 6

Fig. 4.5 Tails of tn 2 -distribution.

density of t-distribution in (4.71) is symmetric around x = 0,

has maximum at x = 0, and is decreasing as x ! ±•. This
means that the most likely outcomes are around zero and,
in order to construct a confidence interval, we select c such
that tn 2 ( c, c) = a. Then, with probability a,
p
n 2sx
c  (b̂1 b1 )  c, (4.100)
ŝ
and, solving for b1 , we obtain the a confidence interval

ŝ ŝ
b̂1 c p , b̂1 + c p . (4.101)
n 2sx n 2sx
Similarly, one can find the a confidence interval for b0 ,
2 s s 3
ŝ 2 ⇣ X 2 ⌘ ŝ 2 ⇣ X 2 ⌘
4 b̂0 c 1 + 2 , b̂0 + c 1+ 2 5.
n 2 sx n 2 sx
4.5 Simple Linear Regression 199

Simultaneous confidence set for the SLR regression line.

Instead of constructing confidence intervals for b0 and b1
separately, we can also construct a simultaneous confidence
set for (b0 , b1 ) as follows. Let us recall the equations (4.94)
and (4.95) in the proof of Theorem 4.4,
s sX s
ĝ0 = p Z1 p Z2 , ĝ1 = p Z2 ,
n nsx nsx

where ĝ j = b̂ j b j . Solving them for Z1 , Z2 ,

p p
n n
Z1 = ĝ0 + X ĝ1 = (b̂0 b0 ) + X(b̂1 b1 ) ,
s
p p s
nsx nsx
Z2 = ĝ1 = (b̂1 b1 ).
s s
Random variables Z1 , Z2 are independent standard Gaussian,
also independent of
nŝ 2
⇠ cn2 2 .
s 2

Therefore, Z12 + Z22 has c22 -distribution and, by the definition

of F-distribution in the previous section,
Z12 + Z22 . nŝ 2 1
F :=
2 s2 n 2
n 2 2 h i
2
= sx (b̂1 b1 )2 + (b̂0 b0 ) + X(b̂1 b1 )
2ŝ 2

has F2,n 2 distribution with degrees of freedom 2 and n 2.

Given a 2 (0, 1), if we define c by F2,n 2 (0, c) = a then,
with probability a,
n 2h 2 2 2
i
s (b̂1 b1 ) + (b̂0 b0 ) + X(b̂1 b1 )  c.
2ŝ 2 x
This inequality defines an ellipse centred at (b̂0 , b̂1 ), which
is a simultaneous a confidence set for the pair (b0 , b1 ).
200 4 Gaussian Distributions

Exercise 4.5.1. Check that (4.79) and (4.79) are equivalent

to (4.85) and (4.85).
Exercise 4.5.2. Show that the constants c1 and c2 in (4.98)
c
(that depend on n) satisfy limn!• nj = 1. Hint: use (4.65)
and Chebyshev’s inequality.
Exercise 4.5.3. Check directly from the definitions that the
equation (4.92) holds.
Exercise 4.5.4. How does c in (4.100) behave as n ! •?
Hint: use (4.72) or Exercise 4.4.9.
Chapter 5
Finite State Markov Chains

5.1 Definitions and Basic Properties

In the Example 1.4.5 in Section 1.4, we discussed how the

joint distribution of discrete random variables X1 , . . . , Xn can
be constructed sequentially

P X1 = x1 , . . . , Xn = xn
n 1
= ’ P Xk+1 = xk+1 | X1 = x1 , . . . , Xk = xk , (5.1)
k=0

where the first factor for k = 0 is just P(X1 = x1 ) and

P Xk+1 = xk+1 | X1 = x1 , . . . , Xk = xk

is the conditional probability that Xk+1 = xk+1 given X1 = x1 ,

. . . , Xk = xk . A sequence of random variables X1 , . . . , Xn , . . . is
called a Markov chain if this conditional probability depends
only on the last outcome Xk = xk in the condition,

P Xk+1 = xk+1 | X1 = x1 , . . . , Xk = xk
= P Xk+1 = xk+1 | Xk = xk . (5.2)

This is called a Markov property, or memoryless property.

201
202 5 Finite State Markov Chains

In this chapter, we will discuss only finite state Markov

chains when all Xk ’s take values in a finite set

S = s1 , . . . , sm . (5.3)

Elements of this set are called states.

The conditional probabilities in (1.93) are called transi-
tion probabilities, and Markov chain is called homogeneous
if the transition probabilities P(Xk+1 = b | Xk = a) for a, b 2 S
are the same for all k. A Markov chain is almost by default
assumed to be homogeneous, and it is stated explicitly when
it is not. Here we will discuss only homogeneous Markov
chains.
The transition matrix P is the matrix
h i h i
P := pi j = p(si , s j ) , (5.4)
1i, jn 1i, jn

whose entries are the transition probabilities

pi j = p(si , s j ) = P Xk+1 = s j | Xk = si . (5.5)

It is customary (and convenient in terms of notation) to start

the Markov chain at ‘time zero’, i.e. k = 0. The distribution
of X0 , which we will denote by µ,

µi = µ(si ) := P(X0 = si ) for si 2 S, (5.6)

is called the initial distribution of the Markov chain. With

this notation, the definition of the (homogeneous finite state)
Markov chain can be rewritten as

P X0 = x0 , X1 = x1 , . . . , Xn = xn
= µ(x0 )p(x0 , x1 )p(x1 , x2 ) · · · p(xn 1 , xn ). (5.7)

The initial distribution µ will vary depending on how we

want to start the chain, so we can also express this definition
entirely in terms of the transition matrix,
5.1 Definitions and Basic Properties 203

P X1 = x1 , . . . , Xn = xn | X0 = x0
= p(x0 , x1 )p(x1 , x2 ) · · · p(xn 1 , xn ). (5.8)

If we start the chain in the state x0 , then the probability that

the chain will visit states x1 , . . . , xn in the next n steps is the
product of transition probabilities along the path. If we mul-
tiply by µ(x0 ), we recover the previous equation.
It is also convenient to visualize transition probabilities by
a transition graph. For example, the transition matrix
2 3
0.25 0.25 0 0 0.5 0 0
6 0.4 0 0 0 0.6 0 0 7
6 7
6 0 0.3 0 0.7 0 0 0 7
6 7
P=6 0 0 0.25 0 0.75 0 0 7 (5.9)
6 7
6 0 0.9 0 0 0.1 0 0 7
4 0 0 0 0 0 0 0.55
0 0 0 0 0 0.5 0
can be depicted as in Figure 5.1, with dots representing
states, and arrows representing non-zero transition probabil-
ities. Notice that, for example, p12 6= p21 .

0.3
s2
0.25 | 0.4
s7 s3
0.5 0.5
s6 0.6 | 0.9
0.25 0.7 | 0.25
s1
0.1

s4

0.5 s5
0.75

Fig. 5.1 Transition graph of a Markov chain with the transition matrix (5.9). States
s3 , s4 are inessential, and essential states are divided into two clusters, {s1 , s2 , s5 } and
{s6 , s7 }.
204 5 Finite State Markov Chains

This graphical representation of transition probabilities

suggests that we can think of Markov chains as random
walks on the set of states, where at each step we pick one
of the neighbours of the current state si with probabilities pi j
and move there. If pii 6= 0, we can end up staying at si .
A state si is called inessential if we can reach from it an-
other state s j with positive probability (not necessarily in one
step), from which we cannot come back to si . For example,
in Figure 5.1, the states s3 and s4 are inessential. Such states
are also called transient, although this terminology is better
suited for infinite state Markov chains. For finite state chains,
once we reach s j , we can never come back to si , so for the
long-term behaviour of the Markov chain these states do not
matter.
Two states si and s j are called communicating, denoted
si $ s j , if we can go from each of them to the other with
positive probability. If

Si = s 2 S : si $ s

is the set of states communicating with si , and if si communi-

cates with s j then, obviously, Si = S j , because any state that
communicates with si communicates with s j and vice versa.
This means that communicating states can be divided into
disjoint clusters. For example, in Figure 5.1, there are two
clusters of essential states, {s1 , s2 , s5 } and {s6 , s7 }, and one
cluster of inessential states, {s3 , s4 }. A state si is called ab-
sorbing state if pii = 1, because, once we reach this state, we
can never leave. Transition probabilities on a given cluster of
essential states define a Markov chain on that cluster.
Markov chain is called irreducible if all its states com-
municate with each other, which means that there are no
inessential states and there is only one cluster of essen-
tial states. Below we will mostly study irreducible Markov
chains.
5.1 Definitions and Basic Properties 205

The transition matrix P is much more useful than just as a

representation of the transition probabilities. If we denote

pi j (n) = Pn ij
(5.10)

the (i, j)th element of Pn then the following holds.

Lemma 5.1 (n-step transition probabilities). For all n 1,

P(Xn = s j | X0 = si ) = pi j (n) (5.11)

and, in particular,
n
P(Xn = s j ) = Â µi pi j (n), (5.12)
i=1

where µ is the initial distribution defined in (5.6).

If we write the last equation in the vector form,

P(Xn = s1 ), . . . , P(Xn = sm ) = µPn , (5.13)

where µ = (µ1 , . . . , µm ), we can compute the distribution of

the chain at time n by multiplying the initial distribution µ
by Pn . It is important that we multiply the row vector µ by
Pn on the right and, for example, Pn µ T does not have the
same meaning. The equation (5.11) also shows that
m
 pi j (n) = 1,
j=1

so Pn can be viewed as an n-step transition matrix.

Proof. We start with n = 2. In order to compute

P(X2 = s j | X0 = si ),

we can sum over possible outcomes of X1 = sk ,

206 5 Finite State Markov Chains

m
P(X2 = s j | X0 = si ) = Â P(X1 = sk , X2 = s j | X0 = si)
k=1
m m
= Â p(si, sk )p(sk , s j ) = Â pik pk j ,
k=1 k=1

which is the (i, j)th entry of P2 . The general case follows by

induction, using a similar calculation. Assuming that (5.11)
holds,

P(Xn+1 = s j | X0 = si )
m
= Â P(Xn = sk , Xn+1 = s j | X0 = si)
k=1
m
= Â P(Xn+1 = s j | Xn = sk )P(Xn = sk | X0 = si)
k=1
m
= Â pik (n)pk j = pi j (n + 1).
k=1

If we multiply this by µi = P(X0 = si ) and sum over i  m,

we get (5.12) for Xn+1 , so the proof of the induction step is
complete. t
u
The period di of a state si is defined by

di = gcd n 1 : pii (n) > 0 , (5.14)

the greatest common divisor all the times n when a walk

starting at si can return to si . If di = 1 then the state is called
aperiodic. We call a Markov chain aperiodic if all periods
di = 1. It turns out that for irreducible chains this is the same
as requiring only one period di to be equal to 1.
Lemma 5.2. If a Markov chain is irreducible then all periods
di are equal.
Proof. Let us suppose that s j can be reached from si in N
steps and si can be reached from s j in M steps with positive
probabilities, pi j (N) > 0 and p ji (M) > 0. If p j j (n) > 0 then
5.1 Definitions and Basic Properties 207

pii (N + M + n) pi j (N)p j j (n)p ji (M) > 0,

because we can reach s j in N steps, then come back to s j

in n steps, and then reach si in M steps, so the probability to
return to si in N + M + n steps is positive. This is also true for
n = 0, because we can go to s j and come back to si in N + M
steps. Since di is the period of si , by definition, di divides all
numbers N + M + n as above. In particular, it divides N + M,
which implies that it divides all n such that p j j (n) > 0. This
means that di divides d j , because d j is the greatest common
divisor of all such n. Similarly, d j divides di , so they must be
equal. t
u
The following property of irreducible aperiodic Markov
chains will be very important in Section 5.3.
Lemma 5.3. If a Markov chain is irreducible and aperiodic
then there exists N 1 such that, for all n N,

pi j (n) > 0 for all i, j. (5.15)

In other words, for large n, all the entries of Pn are strictly

positive.
Proof. Let T (s1 ) = {n 1 : p11 (n) > 0} be the set of all
times the chain starting at s1 can come back to s1 . Let d 1
be the smallest positive integer that can be written as

d = a1 n1 + . . . + ak nk ,

for k 1, ni 2 T (s1 ) and ai 2 Z. Then d must divide all

n 2 T (s1 ), because if it does not divide some n 2 T (s1 ) then

n = ad + r,

where the remainder r 1 is strictly less than d. However,

r=n ad = n a1 n1 ... ak nk
208 5 Finite State Markov Chains

is also a linear combination with integer coefficients of the

times in T (s1 ), which contradicts that d was the smallest
such number. This proves that d divides the period of s1 and,
since the chain is aperiodic, d = 1. We showed that

1 = a1 n1 + . . . + ak nk , (5.16)

for some k 1, ni 2 T (s1 ) and ai 2 Z. Some ai of course can

be negative.
Let us take the largest |ai | and, for certainty, suppose that
|a1 | is the largest. If

N1 = |a1 |n1 (n1 + . . . + nk ),

we will now show that all n N1 can be written as a linear

combinations
n = c1 n1 + . . . + ck nk ,
with non-negative integer coefficients ci . This will prove that
p11 (n) > 0 for such n, because the chain starting from the
state s1 can come back to s1 in ni number of steps, so we just
need to repeat these loops ci times for all i  k to see that the
chain can come back in n steps with positive probability.
Let us first consider any n between N1 and N1 + n1 , which
means that n = N1 + ` for some `  n1 . By (5.16), we can
write

n = N1 + ` = |a1 |n1 (n1 + . . . + nk )

+ `(a1 n1 + . . . + ak nk )
= c1 n1 + . . . + ck nk ,

where

ci = |a1 |n1 + `ai |a1 |` + `ai = (|a1 | + ai )` 0

as we wished, because we assumed that |a1 | is greater or

equal to |ai |. Since
5.1 Definitions and Basic Properties 209

N1 + n1 = (|a1 |n1 + 1)n1 + |a1 |n1 (n2 + . . . + nk ),

we can repeat the same argument for n in between N1 + n1

and N1 + 2n1 , and so on.
Similarly, for each state si , we can find some Ni such that
pii (n) > 0 for n Ni . If we take N 0 = max(N1 , . . . , Nm ) then
pii (n) > 0 for all i and all n N 0 . Since the chain is irre-
ducible, we can always go from one state si to another state
s j in under m steps with positive probability (see exercise
below), which implies that pi j (n) > 0 for all n N 0 + m. t u
Exercise 5.1.1. Let U1 ,U2 , . . . be i.i.d. random variables with
the uniform distribution on {1, . . . , m} and let

Xn = max Ui .
in

Show that X1 , X2 , . . . is a Markov chain and find its transition

probabilities. What are the essential states of this chain?
Exercise 5.1.2. Suppose that m while balls and m black balls
are mixed together and divided evenly between two baskets.
At each step two balls are chosen at random, one from each
basket, and switched. We say that the system is in the state si
if there are i white balls in the first basket. Find the transition
probabilities of this chain.
Exercise 5.1.3. If a Markov chain on m states is irreducible,
show that, for any i, j  m, pi j (k) > 0 for some k  m.
Exercise 5.1.4. If a Markov chain is irreducible and p11 > 0,
what is the period d2 of the state s2 ?
Exercise 5.1.5. If a Markov chain (Xn )n 0 is irreducible and
has period d, show that Yn = Xdn for n 0 is a Markov chain
and that it is aperiodic. Does it have to be irreducible?
Exercise 5.1.6. Let N ⇠ Poiss(l ) be a Poisson random vari-
able. Let us run N independent Markov chains starting from
the state s1 , and let Nn (i) be the number of these chains in
the state si at time n. Show that Nn (i) ⇠ Poiss(l p1i (n)).
210 5 Finite State Markov Chains

5.2 Stationary Distributions

In Lemma 5.1 we showed that if the vector µ = (µ1 , . . . , µm )

describes the distribution of the chain X0 at time zero, then
µPn is the distribution of Xn at time n. The distribution µ on
the set of states is called stationary if

µ = µP, (5.17)

i.e. the distribution of X1 is the same as that of X0 . Of course,

this implies that µ = µPn , so the distribution of all Xn is
the same. We will show that a stationary distribution always
exists, and for an irreducible Markov chain it is unique. For
irreducible chains, we will also derive a representation of the
stationary distribution in terms of expected return times. To
find a stationary distribution in practice, one can just solve a
system of linear equations µ = µP.
Lemma 5.4 (Existence). For any finite state Markov chain,
there exists at least one stationary distribution.
Proof. Let us consider a sequence of matrices
1
An = (1 + P + . . . + Pn ).
n
Since each matrix Pk is a transition matrix and its rows sum
up to 1, the rows of An also sum up to one, so its entries are
all between 0 and 1. By the Bolzano–Weierstrass theorem,
there exists a convergent subsequence Ank ! A. Notice that
the limit A is also a transition matrix because the entries must
be nonnegative and rows add up to 1. If we consider
1
An P = (P + . . . + Pn + Pn+1 ),
n
we see that the difference goes to a zero matrix,
1
An P An = ( 1 + Pn+1 ) ! 0.
n
5.2 Stationary Distributions 211

This implies that

A = lim Ank = lim Ank P = AP,

k!• k!•

which shows that A = AP. This means that each row a of

this matrix satisfies a = aP. Since each row is a probability
distribution on states (entries are nonnegative and add up to
one), each row is a stationary distribution. t
u
Notice that if the stationary distribution is unique, all the
rows of A in the above proof must be equal.
Next, we will show the following.
Lemma 5.5 (Uniqueness). If a Markov chain is irreducible
then the stationary distribution is unique.
Proof. Previous lemma shows that P I is not invertible,
since there exists a non-zero solution of µ(P I) = 0. Let
us first show that, when P is the transition matrix of an irre-
ducible Markov chain, the rank of P I is m 1. Consider
the linear system (P I)vT = 0 for v = (v1 , . . . , vn ). Let us
show that irreducibility implies that

v1 = v2 = . . . = vn .

Consider the largest coordinate of v, let us say, v1 . The first

equation in the system reads
m
 p1 j v j = v1.
j=1

On the other hand, since all v j  v1 and p1 j v j  p1 j v1 ,

m m
v1 = Â p1 j v j  Â p1 j v1 = v1.
j=1 j=1

So the equality is actually an equality, and all p1 j v j = p1 j v1 .

This means that v j = v1 for all j such that p1 j 6= 0. In other
words, all the states s j that can be reached from the state
212 5 Finite State Markov Chains

s1 in one step must have v j = v1 . Then, repeating the same

calculation for these immediate neighbours we can see that
all their neighbours also much have v j = v1 . Since the chain
is irreducible, we can reach all states in finitely many steps,
which proves that all v j ’s are equal.
This proves that the kernel of P I is one dimensional,

ker(P I) = c(1, . . . , 1) : c 2 R , (5.18)

spanned by the vector (1, . . . , 1), which means that the rank
of P I equals to m 1. On the other hand, if we had more
than one stationary distribution, the system µ = µP would
have at least two linearly independent solutions, which would
mean that the rank of P I would be not greater than m 2.
This proves that the stationary distribution is unique. t
u
Next, we will derive a more descriptive formula for the
stationary distribution of irreducible Markov chains. First, let
us give a heuristic non-rigorous explanation and then check
carefully that our intuitive guess is correct. In the proof of
Lemma 5.4 we considered the sequence of matrices
1
An = (1 + P + . . . + Pn )
n
and showed that any limit A over some subsequence consists
of rows which are stationary distributions. For irreducible
chains, we just showed that the stationary distribution µ is
unique, so we must have that
2 3 2 3
µ µ1 · · · µm
A = 4 ... 5 = 4 ... . . . ... 5 .
µ µ1 · · · µm
In particular, this means that the limits over all convergent
subsequences are equal to A and, therefore, the limit exists
over the entire sequence,
5.2 Stationary Distributions 213
2 3
µ1 · · · µm
1
lim (1 + P + . . . + Pn ) = 4 ... . . . .. 5 .
.
n!• n
µ1 · · · µm

Since the (i, j)th entry of Pk is

(Pk )i j = pi j (k) = P(Xk = s j | X0 = si ),

this can be written as

1 n
lim  P(Xk = s j | X0 = si ) = µ j . (5.19)
n!• n
k=1

If we multiply both sides by P(X0 = si ), sum over i  m and

use that
m
 P(Xk = s j | X0 = si)P(X0 = si) = P(Xk = s j ),
i=1

we get
1 n
lim  P(Xk = s j ) = µ j . (5.20)
n!• n
k=1
In other words, this equation holds no matter how we start
the chain, i.e. no matter what the distribution of X0 is. Since

P(Xk = s j ) = E I(Xk = s j ),

by the linearity of expectation,

1 n
lim E Â I(Xk = s j ) = µ j . (5.21)
n!• n
k=1

Clearly, this equation can be interpreted by saying that µ j is

the expected proportion of time the chain visits the state s j ,
if we looks at these proportions over long periods of time.
There is another interpretation of the last equation. Since
it does not matter how we start the chain, let us start it in
the state s1 . Then, let us divide the time between 1 and n into
214 5 Finite State Markov Chains

intervals between consecutive visits of the state s1 . Let us say

that M visits of s1 occurred between 1 and n, with intervals
T1 , T2 , . . . , TM , so that

T1 + . . . + TM ⇡ n

in the sense that their ratio is close to 1. Let N1 ( j), . . . , NM ( j)

be the number of times we visited s j in between these con-
secutive returns to s1 , so that
n
N1 ( j) + . . . + NM ( j) ⇡ Â I(Xk = s j ).
k=1

Then
1 n N1 ( j) + . . . + NM ( j)
Â
n k=1
I(Xk = s j ) ⇡
T1 + . . . + TM
.

If we know that the Markov chain is visiting s1 , the future

does not depend on the past because of the memoryless prop-
erty. This suggests that what happens in between consecutive
visits is independent of each other and the pairs

(N1 ( j), T1 ), . . . , (NM ( j), TM )

are actually independent and identically distributed. To make

this statement rigorous, once needs to prove the so called
strong Markov property, which we are not going to go into
here. However, if we use it as a guiding intuition and di-
vide both numerator and denominator above by M, the law
of large numbers tells us that
N1 ( j) + . . . + NM ( j) EN1 ( j)
⇡ .
T1 + . . . + TM ET1
This suggests another representation,
EN1 ( j)
µj = , (5.22)
ET1
5.2 Stationary Distributions 215

where we assume that the chain starts in the state s1 ,

T1 = min n 1 : Xn = s1 (5.23)

is the first return time to s1 , and

T1 1
N1 ( j) = Â I(Xk = s j ) (5.24)
k=0

is the number of times the chain visits s j before returning to

s1 .
The first return time T1 is an example of a stopping time.
In general, an integer valued random variable T 0 is called
a stopping time if the event {T = n} depends only on X0 , . . . ,
Xn . In other words, we decide whether to stop at time n based
on what was observed up to time n. The return time T1 to s1
is a stopping time in this sense, because

{T1 = n} = {X1 6= s1 , . . . , Xn 1 6= s1 , Xn = s1 }.

Notice also that N1 (1) = 1, because the chain is in the state

s1 only at time zero before the return, so (5.22) implies that
1
µ1 = . (5.25)
ET1
Since it did not matter how we start the chain, if we start it at
s j and define

T j = min n 1 : Xn = s j (5.26)

then the same logic suggests that

1
µj = . (5.27)
ET j
In order to prove the representation (5.22), we need to
check two things. First, we will need to check that ET1 < •.
Because the number of visits to different states before time
216 5 Finite State Markov Chains

T1 adds up to T1 ,
m
 N1( j) = T1,
j=1

and therefore N1 ( j)  T1 , proving ET1 < • would also imply

that EN1 ( j) < •. This would show that the numbers in (5.22)
are well defined, and they define a distribution because
m m
EN1 ( j) ET1
 µj =  = = 1.
j=1 j=1 ET1 ET1

After that we will need to check that µP = µ, so it is the

stationary distribution.
Lemma 5.6. If a Markov chain is irreducible then ET1 < •,
where T1 defined in (5.23) is the first return time to the state
s1 for the chain that starts at s1 .
In the proof we will use the Markov property (5.2) in the
following way. Let us consider two vectors

X 1 = (X1 , . . . , Xt1 ), X 2 = (Xt1 +1 , . . . , Xt2 )

consisting of the Markov chain up to time t1 and in between

t1 and t2 . Let us consider some subsets

A ✓ {s1 , . . . , sm }t1 , B ✓ {s1 , . . . , sm }t2 t1

of possible outcomes for these two blocks of the Markov

chain, and consider the conditional probability

P(X 2 2 B | X 1 2 A).

We can not use the Markov property directly, because the

condition is a set, but we can rewrite this as
5.2 Stationary Distributions 217

2 P(X 2 2 B, X 1 2 A)
1
P(X 2 B | X 2 A) = (5.28)
P(X 1 2 A)
P(X 2 2 B, X 1 = a)
=Â
a2A P(X 1 2 A)
P(X 2 2 B | X 1 = a)P(X 1 = a)
=Â
a2A P(X 1 2 A)
P(X 2 2 B | Xt1 = at1 )P(X 1 = a)
=Â 1 2 A)
,
a2A P(X

where in the last step we used the Markov property to write

the condition in terms of the last outcome Xt1 = at1 . This
is very useful because, for example, if we can control the
probability
P(X 2 2 B | Xt1 = at1 )  p
for all possible outcomes Xt1 = at1 then the above equation
implies that
P(X 1 = a)
P(X 2 B | X 2 A)  p Â
2 1
1
= p. (5.29)
a2A P(X 2 A)

In other words, Markov property can be used in this way

even if the condition is a set.
Proof (of Lemma 5.6). Since the chain is irreducible, if the
chain is in the state si at time k, Xk = si , we can reach the
state s1 in under m steps, which means that

pi1 (`) = P(Xk+` = s1 | Xk = si ) > 0

for some `  m. This implies that the probability that we do

not reach the state s1 in the next m steps is strictly smaller
than 1,
218 5 Finite State Markov Chains

P(Xk+1 6= s1 , . . . , Xk+m 6= s1 | Xk = si )
 P(Xk+` 6= s1 | Xk = si )
= 1 P(Xk+` = s1 | Xk = si ) < 1.

This is true for any si , so

P(Xk+1 6= s1 , . . . , Xk+m 6= s1 | Xk = si )  1 e, (5.30)

for some small enough e > 0, for all i  m.

Using Markov property, this will imply that the probability
that the chain does not come back to s1 in Nm steps (i.e. N
cycles of m steps) will be smaller than (1 e)N ,

P(T1 > Nm)  (1 e)N . (5.31)

To see this, let us write

P(T1 > Nm) = P(X1 6= s1 , . . . , X(N 1)m 6= s1 , . . . , XNm 6= s1 )

and let us decompose this sequence into two blocks

X 1 = (X1 , . . . , X(N 1)m ), X 2 = (X(N 1)m+1 , . . . , XNm ).

If we take
1)m
A = {s2 , . . . , sm }(N , B = {s2 , . . . , sm }m

to be the sets consisting of all the paths that avoid the state
s1 and have lengths (N 1)m and m then

P(T1 > Nm) = P(X 2 2 B, X 1 2 A)

= P(X 2 2 B | X 1 2 A)P(X 1 2 A).

The equation (5.30) implies that, no matter what the value of

X(N 1)m is, the conditional probability that all coordinates of
X 2 avoid the state s1 is smaller than 1 e. By the discussion
before the proof and (5.29), the Markov property implies
5.2 Stationary Distributions 219

P(X 2 2 B | X 1 2 A)  1 e,

and, therefore,

P(T1 > Nm)  (1 e)P(X 1 2 A)

= (1 e)P(T1 > (N 1)m).

By induction on N, the equation (5.31) follows.

Next, let us denote d = (1 e)1/m < 1 and consider any
k 1. Take N 0 such that Nm < k  (N + 1)m. Then

P(T1 k)  P(T1 > Nm)  (1 e)N = d Nm  d k m

.

By the formula (1.34) in Section 1.2 and (5.31),

• •
d1 m
ET1 = Â P(T1 k)  Âd k m
=
1 d
< •,
k=1 k=1

which finishes the proof. t

u
Recall the equations 5.22), (5.23), (5.24),
EN1 ( j)
µj = , (5.32)
ETn1 o
T1 = min n 1 : Xn = s1 , (5.33)
T1 1
N1 ( j) = Â I(Xk = s j ), (5.34)
k=0

where we assumed that the chain starts in the state s1 . To

check that our heuristic discussion above gives the correct
answer, it remains to prove the following.
Theorem 5.1 (Representation of stationary distribution).
If a Markov chain is irreducible then µ defined in (5.32) is
the stationary distribution.
Proof. Showing that µ = µP is equivalent to showing
220 5 Finite State Markov Chains

m
EN1 ( j) = Â pi j EN1 (i), (5.35)
i=1

because the two equations only differ by a factor of 1/ET1 .

To prove (5.35), first let us consider j 6= 1. Because the chain
starts at s1 , we have X0 = s1 6= s j and, in the definition of
N1 ( j), we can start the summation from k = 1,
T1 1 T1 1
N1 ( j) = Â I(Xk = s j ) = Â I(Xk = s j ).
k=0 k=1

We can also write this as

•
N1 ( j) = Â I(Xk = s j , k < T1)
k=1
•
= Â I(Xk = s j , k 1 < T1 ), (5.36)
k=1

where we used that

{Xk = s j , k < T1 } = {Xk = s j , k 1 < T1 },

because Xk = s j 6= s1 , so it is impossible that T1 = k. When

j = 1, the equation (5.36) also holds, for a different reason.
In this case, N1 (1) = 1 by definition, while the right hand
side of (5.36) equals to 1 because k 1 < T1 is the same as
k  T1 and, between times 1 and T1 , the chain visits s1 once
at k = T1 .
If we take expectations of both sides of (5.36),
•
EN1 ( j) = Â P(Xk = s j , k 1 < T1 ) (5.37)
k=1
• m
= Â Â P(Xk 1 = si , Xk = s j , k 1 < T1 ),
k=1 i=1

where we also partitioned the event into different outcomes

Xk 1 = si . The key observation we need to make is that the
5.2 Stationary Distributions 221

event

{k 1 < T1 } = X1 6= s1 , . . . , Xk 1 6= s1

depends only on the random variables X1 , . . . , Xk 1 up to

time k 1. Let us write one term in the above sum by condi-
tioning on the first k 1 random variables,

P(Xk 1 = si , Xk = s j , k 1 < T1 )
= P(Xk = s j | Xk 1 = si , k 1 < T1 )
⇥ P(Xk 1 = si , k 1 < T1 ).

If we use the Markov property in the form (5.28), we can

drop the condition k 1 < T1 and rewrite this as

P(Xk 1 = si , Xk = s j , k 1 < T1 )
= P(Xk = s j | Xk 1 = si )P(Xk 1 = si , k 1 < T1 )
= pi j P(Xk 1 = si , k 1 < T1 ).

Plugging this back into (5.37) and rearranging terms,

• m
EN1 ( j) = ÂÂ pi j P(Xk 1 = si , k 1 < T1 )
k=1 i=1
m •
=  pi j  P(Xk 1 = si , k 1 < T1 )
i=1 k=1
m •
{` = k 1} =  pi j  P(X` = si , ` < T1 )
i=1 `=0
m T1 1
= Â pi j E Â I(X` = si)
i=1 `=0
m
= Â pi j EN1 (i).
i=1

This proves (5.35) and finishes the proof. t

u
222 5 Finite State Markov Chains

Exercise 5.2.1. Show that if a Markov chain has two differ-

ent stationary distributions then there exist infinitely many
stationary distributions.
Exercise 5.2.2. Find the stationary distribution of Markov
chain with the transition matrix
2 3
0 0.5 0.5
P = 4 0 0 1 5.
0.5 0.5 0
If the chain starts at s2 , what is the expectation of the first
return time to s2 ?
Exercise 5.2.3. If µ = (µ1 , . . . , µm ) is the stationary distribu-
tion of an irreducible Markov chain, show that µi 6= 0 for all
1  i  m.
Exercise 5.2.4 (Another proof of uniqueness). Consider an
irreducible Markov chain and suppose that there exist two
stationary distributions µ 1 and µ 2 . Let j be any minimizer

µ 1j µi1
= min 2 .
µ 2j im µ
i

Show that
µ 1j µi1
= 2
µ 2j µi
for any i such that pi j > 0. From this, derive that µ 1 = µ 2 .
Exercise 5.2.5. If p1 j = p2 j for all j and
(
s j , if Xn = s j for j 3,
Yn =
s⇤ , if Xn = s j for j  2,

show that Y1 ,Y2 , . . . is a Markov chain on the new state space

S = {s⇤ , s3 , . . . , sm }.
 5.2 Stationary Distributions 223

Exercise 5.2.6. If a Markov chain is irreducible, prove that

all moments ET1p < • for p 1, where T1 is defined in
(5.23). Hint: use Exercise 1.2.5 in Section 1.2.
Exercise 5.2.7. In Toronto, it rains (or snows) on about 38%
of days per year. Let us consider a Markov chain model of
weather with two states s1 = ‘rain’, s2 = ‘no rain’, and the
transition matrix

p 1 p
P= ,
1 q q

for some p, q 2 (0, 1), so that if it rains today then it will rain
tomorrow with probability p, and if it does not rain today
then it will not rain tomorrow with probability q. Find all p
and q for which the expected proportion of rainy days over
long periods of time is 0.38.
Exercise 5.2.8. Suppose that the state s1 is inessential and let
S1 be the cluster of states that communicate with s1 . Suppose
that the chain starts at s1 , and let T be the first time we exit
the cluster S1 ,

T = min n 1 : Xn 62 S .

Prove that ET < •.

Exercise 5.2.9. Let us consider an irreducible Markov chain
X0 , X1 , . . ., and let A ✓ S be a non-empty subset of its states.
If the chain starts at si then let

TA (si ) = min n 0 : Xn 2 A

be the first time the chain visits one of the states in A and let
ti = ETA (si ) be its expectation. Prove that
(a) If si 2 A then ti = 0.
(b) If si 62 A then ti = 1 + Âmj=1 pi j t j .
(c) The equations in parts (a) and (b) determine t = (t1 , . . . ,tm )
uniquely. Hint: consider equation (5.18).
224 5 Finite State Markov Chains

5.3 Convergence Theorem

In Lemma 5.4 in the last section we considered the sequence

of matrices
1
An = (1 + P + . . . + Pn )
n
and showed that any subsequential limit A consists of rows
which are stationary distributions. For irreducible chains, we
also showed that the stationary distribution µ is unique and,
therefore,
2 3
µ1 · · · µm
1
lim (1 + P + . . . + Pn ) = 4 ... . . . ... 5 .
n!• n
µ1 · · · µm
Can we expect a stronger statement that
2 3
µ1 · · · µm
lim Pn = 4 ... . . . ... 5?
n!•
µ1 · · · µm
First, we need to check that this is, indeed, a stronger state-
ment.
Exercise 5.3.1. If the limit limn!• Pn exists then the limit
limn!• 1n (1 + P + . . . + Pn ) exists and is the same.
It is easy to see that this can not be true in general. For
example, if a Markov chain has two states s1 , s2 and it has
the transition probabilities p12 = p21 = 1 then,
 
0 1 10
P2n = , P2n+1 = .
10 01
Another way to put it is that, if we start the chain in the state
si , it can be back in si only at even times. The problem with
this example is that this chain has period 2, which results in
this periodic behaviour. However, if the chain is aperiodic
then the convergence holds.
5.3 Convergence Theorem 225

Theorem 5.2 (Convergence Theorem). If a Markov chain

is irreducible and aperiodic then
2 3
µ1 · · · µm
lim Pn = 4 ... . . . ... 5 , (5.38)
n!•
µ1 · · · µm

where µ = (µ1 , . . . , µm ) is its stationary distribution.

Notice that this is equivalent to the following statement: if
n = (n1 , . . . , nm ) is any distribution on the set of states then

lim nPn = µ. (5.39)

n!•

In one direction, multiplying the right hand side of (5.38) by

n on the left, obviously, gives µ. In the other direction, if we
take n = (0, . . . , 1, . . . , 0) where the ith coordinate is 1, then

lim nPn = µ
n!•

is the ith row of the limit limn!• Pn , so all the rows of this
limit are equal to µ.
Recall that if n is the initial distribution of the chain, i.e.
the distribution of X0 , then nPn is the distribution of Xn .
Hence, the convergence theorem in the form (5.39) states that
the distribution of Xn converges to the stationary distribution
µ no matter what the initial distribution is.
Proof (Theorem 5.2). We will be proving the formula (5.39).
Let us consider the L1 -distance between vectors on Rm ,
m
kx yk1 = Â |xi yi |.
i=1

Let us notice that multiplying by any transition matrix P on

the right does not increase this distance, because
226 5 Finite State Markov Chains

m m
kxP yPk1 = Â Â pi j (xi yi )
j=1 i=1
m m
 Â Â pi j |xi yi |
j=1 i=1
m m
=Â Â pi j |xi yi |
i=1 j=1
m
= Â |x j y j | = kx yk1 .
i=1

If we apply this to distributions n and µ and use that µ = µP,

we get that
knP µk1  kn µk1 .
This suggests that the distributions get closer to each other
after one step but, in order to prove convergence, we would
like to have a strict inequality and then iterate it.
This is where the assumption that our chain is aperiodic
comes into play. By Lemma 5.3 in Section 5.1, for large
enough N, all entries pi j (N) of PN are strictly positive. Let
e > 0 be the smallest among its entries. If we apply the above
calculation to the transition matrix PN and distributions n
and µ, the improvement comes from the fact that
m
 e(ni µi ) = e(1 1) = 0
i=1

and, therefore,
m m
 pi j (N)(ni µi ) =  (pi j (N) e)(ni µi ).
i=1 i=1

Notice that all pi j (N) e 0, because e was the smallest

among all entries, so
5.3 Convergence Theorem 227

m m
knP N N
µP k1 = Â Â pi j (N)(ni µi )
j=1 i=1
m m
= Â Â (pi j (N) e)(ni µi )
j=1 i=1
m m
 Â Â (pi j (N) e)|ni µi |
j=1 i=1
m m
=Â Â (pi j (N) e)|ni µi |
i=1 j=1
m
= (1 me) Â |n j µ j |.
i=1

Thus, knPN µPN k1  (1 me)kn µk1 and, by induction,

knPNk µPNk k1  (1 me)k kn µk1 .

For powers in between the multiples of N, of the form Nk + `

for `  N, we use

knPNk+` µPNk+` k1  (1 me)k knP` µP` k1

 (1 me)k kn µk1 .

This proves that knPn µPn k1 ! 0 for any two distributions

n and µ. When µ is the stationary distribution, µPn = µ, so
knPn µk1 ! 0. This finishes the proof. t
u
Exercise 5.3.2. Given p 2 (0, 1), consider a Markov chain
with the transition probabilities

pi,i 1 =1 p, pi,i+1 = p for i = 2, . . . , m 1

and p1,1 = 1 p, p1,2 = p, pm,m 1 = 1 p, pm,m = p. What

is the limit limn!• Pn . Hint: solve the system µ = µP ex-
plicitly.
228 5 Finite State Markov Chains

Exercise 5.3.3. Show that if a Markov chain is irreducible

and aperiodic then, for any function f : S ! R, the expected
value E f (Xn ) converges.
Exercise 5.3.4. Suppose that m people sit at the round table
and all have plates with rice in front of them. At the same
time, each person divides his of her rice in half and evens
out one half with the half of the person on the right (i.e. they
split the total of their halves evenly), and another half with
the person on the left. If they keep repeating this, what will
happen in the long run?
Exercise 5.3.5. Let us consider the L• norm on Rm ,

kx yk• = max |xi yi |.

im

If P is a Markov transition matrix P, show that

kPx Pyk•  kx yk• .

5.4 Reversible Markov Chains 229

5.4 Reversible Markov Chains

Let us consider a Markov chain with the transition matrix

P. A probability distribution µ = (µ1 , . . . , µm ) on the set of
states is called reversible for the chain if

µi pi j = µ j p ji (5.40)

for all i and j. A Markov chain is called reversible if it has a

reversible distribution. The equations (5.40) are called the
detailed balance equations. If we start the chain with the
distribution µ then (5.40) states that the first two outcomes
(si , s j ) and (s j , si ) are equally likely. Moreover,

P(X0 = si , X1 = s j , X2 = sk )
= µi pi j p jk = p ji µ j p jk = p ji pk j µk
= P(X0 = sk , X1 = s j , X2 = si ),

which means that the first three outcomes (si , s j , sk ) and

(sk , s j , si ) are equally likely. The same calculation show that
any sequence of outcomes and the same sequence in reverse
order are equally likely, hence the name reversible. It is easy
to check that a reversible distribution is stationary.
Lemma 5.7. A reversible distribution µ is stationary.
Proof. Summing (5.40) over i, we get
m m
 µi pi j = µ j  p ji = µ j ,
i=1 i=1

which finishes the proof. t

u
There are a number of examples, where the equations
(5.40) are easy to guess or easy to check, so this is one good
way to find a stationary distribution. Let us consider some
classical examples of reversible Markov chains.
230 5 Finite State Markov Chains

Example 5.4.1 (Birth-and-death processes). Let us con-

sider a Markov chain that in one step moves from a state si
only to si+1 or si 1 . Of course, since our state space is finite,
this means that from s1 it only moves up to s2 and from sm
it only moves down to sm 1 . The chain can also stay in each
state si . Another way to say this is that

pi j > 0 if |i j| = 1, and pi j = 0 if |i j| > 1.

The probabilities pii may be positive or zero. Such Markov

chains are called birth-and-death chains, and they are all re-
versible. To find a reversible distribution, let us set µ1 = x as
an unknown variable. If (5.40) holds then
µ1 p12 p12
µ2 = = x.
p21 p21
Given µ2 , we can find µ3 ,
µ2 p23 p12 p23
µ3 = = x,
p32 p21 p32
and we can continue, by induction,
p12 pi 1,i
µi = ··· x.
p21 pi,i 1
Since all µi are of the form ci x for some positive ci > 0, to
find x we can use that the probabilities add up to 1, so
1
x= .
c1 + . . . + cm
Of course, the same calculation works for any chain with the
transition graph that looks like a single path between two
‘endpoint’ vertices, and we can relabel the states of such
chain to turn it into a formal birth-and-death chain. t
u
Example 5.4.2 (The Ehrenfest model of diffusion). One
example of a birth-and-death Markov chain is Ehrenfest’s
5.4 Reversible Markov Chains 231

model of diffusion. Suppose there are m particles inside

two connected containers. The particles can move between
the two containers, and suppose that the next particle that
moves is chosen uniformly at random. This process can
be described by a Markov chain with m + 1 states si for
i = 0, . . . , m, where the system is in the state i if there are
i particles in the first container and m i in the second. The
transition probabilities are given by
i m i
pi,i 1 = , pi,i+1 = . (5.41)
m m
Since this is a birth-and-death chain, it is reversible and
it turns out that its reversible distribution is the Binomial
B(m, 21 ) distribution,
✓ ◆
m 1
µi =
i 2m .
We will leave it as an exercise to check that the detailed bal-
ance equations (5.40) are satisfied. t
u
Example 5.4.3 (Random walks on graphs). Let us consider
a connected graph G on the set of vertices {s1 , . . . , sm }. This
means that all vertices are connected by a path on edges of
the graph. Let Ni be the number of neighbours of si . Let us
consider a Markov chain with the transition probabilities
⇢1
, if s j is a neighbour of si ,
pi j = Ni (5.42)
0, otherwise.
In other words, in each state si the chain picks a neighbour
uniformly at random and moves there. A distribution µ is
reversible for this chain if
1 1
µi = µj .
Ni Nj
232 5 Finite State Markov Chains

Therefore, all the ratios Nµii must be equal to the same constant
c, so µi = cNi . Since the probabilities must add up to 1, the
constant c = N1 , where N = N1 + . . . + Nm , and
Ni
µi = . (5.43)
N
Since the graph is connected, the Markov chain is irreducible
and µ is the unique stationary distribution. t
u
The most important examples of reversible Markov chains
appear in the context of Markov Chain Monte Carlo (MCMC)
algorithms. Let us describe one such algorithm.
Example 5.4.4 (Metropolis algorithm). As in the previous
example, let us consider a connected graph G on the set of
vertices S = {s1 , . . . , sm }. Suppose that we are interested in
some probability distribution µ on S, but µ is defined by a
complicated model that makes the computation of its coordi-
nates µi impractical. On the other hand, for any neighbours
si and s j on the graph, µi and µ j are related in some simple
way, so that we know the ratio
µi
ri j =
µj
without knowing µi and µ j . Of course, given ri j , the distri-
bution µ can in principle be reconstructed by setting µ1 = x
as unknown parameter, then finding µi = ri1 x in terms of x
for all its neighbours, and propagating this through the graph
to express all µi = xci in terms of x. Then we find
1
x= ,
c1 + . . . + cm
because the probabilities must add up to one. However, in
many applications the graph is so large (say m = 220 ) that it
is not even feasible to add up m numbers.
If we only know the above ratios ri j , how can we compute,
for example, the expectation of some function f : S ! R on
5.4 Reversible Markov Chains 233

S with respect to the distribution µ,

m
Eµ f := Â f (si )µi ,
i=1

without computing µ? The idea of the Markov Chain Monte

Carlo method is to combine the law of large numbers with
the convergence theorem for Markov chains. Suppose that
we can construct a Markov chain with the stationary distri-
bution µ, whose transition probabilities depends only on the
ratios ri j . Then we can pick the initial state at random and
run the chain for a large number of steps, n, to obtain a ran-
dom variable Xn with the distribution close to µ. If we repeat
this procedure N times, we will get N i.i.d. random variables
Xn1 , . . . , XnN with the distribution close to µ. By the law of
large numbers,

1 N m
 f (Xn ) ⇡ E f (Xn ) ⇡  f (si)µi.
N k=1
k 1
i=1

In other words, this procedure allows us to approximate the

expectation of f with respect to the distribution µ, when it is
not feasible to compute µ directly.
Here is a classical construction of a Markov chain, called
the Metropolis chain, with the stationary distribution µ and
the transition probabilities that depend only on the ratios ri j .
Let us write i ⇠ j to denote that si and s j are neighbours. For
each vertex si , we define
1 ⇣µ N ⌘
j i
pi j = min ,1 (5.44)
Ni µi N j
if j ⇠ i and define

pii = 1 Â pi j . (5.45)
j⇠i

All other pi j = 0. Notice that pii 0, because

234 5 Finite State Markov Chains

1
 pi j   Ni = 1,
j⇠i j⇠i

so this definition makes sense. Notice also that the transition

probabilities depend only on the ratios ri j .
To show that µ is the stationary distribution of this Markov
chain, we will check that µ satisfies the detailed balance
equations (5.40). For i = j there is nothing to check, and if
i 6⇠ j then both sides are zero, so we only need to check this
for i ⇠ j. If µ j Ni  µi N j then, by (5.44),
1 1 µi N j µi
pi j = and p ji = = ,
Ni N j µ j Ni µ j Ni
which immediately implies (5.40). The case µ j Ni µi N j is
exactly the same, so the Metropolis chain has the stationary
distribution µ.
If at least one pii > 0 then the chain is aperiodic. If the
chain is periodic, a slight modification will make it periodic
without affecting the stationary distribution µ (see exercise
below). Then the convergence theorem applies and we can
use this chain to produce an i.i.d. sample with the distribution
close to µ. t
u
Exercise 5.4.1. Show that if µ is a reversible distribution for
P, it is also a reversible distribution for l I + (1 l )P for any
l 2 [0, 1].
Exercise 5.4.2. Show that if µ is a reversible distribution for
P, it is also a reversible distribution for Pn .
Exercise 5.4.3. Show that a birth-and-death Markov chain is
aperiodic if and only if at least one pii > 0.
Exercise 5.4.4. Show that a Markov chain with the following
transition matrix is not reversible:
2 3
0 0.75 0.25
P = 40.25 0 0.755 .
0.75 0.25 0
5.4 Reversible Markov Chains 235

Exercise 5.4.5. Check that the Binomial distribution B(m, 12 )

satisfies the detailed balance equations (5.40) for the Ehren-
fest chain (5.41).
Exercise 5.4.6. Suppose that two containers contain a total
of m while balls and m black balls. At each step a ball is
chosen at random from all 2m balls and put into the other
container. We say that a system is in the state si if there are
i white balls in the first container. Find the transition proba-
bilities of this chain and compute limn!• Pn .
Exercise 5.4.7. Consider the Ehrenfest chain (5.41) and let
Dn be the difference of particles in the two containers at time
n. This means that if the chain is in the state si at time n then
Dn = 2i m. Prove that
m 2 ⇣ m 2 ⌘n
EDn+1 = EDn = ED0 .
m m
Exercise 5.4.8. Consider a random walk on the following
graph:
s3

s2 s4

s1 s6 s5

If we start the chain at s1 , what is the expected time of the

first return to s1 ? What is the expected number of visits to s4
before the first return to s1 ? Hint: recall the results in Section
5.2.
References

1. Alon, N.; Spencer, J.H.: The probabilistic method. Wiley Series in Discrete
Mathematics and Optimization. John Wiley & Sons, Inc., Hoboken, NJ, 2016.
2. Boucheron, S.; Lugosi, G.; Massart, P.: Concentration inequalities. A non-
asymptotic theory of independence. Oxford University Press, Oxford, 2013.
3. Borovkov, A. A.: Probability theory. Universitext. Springer, London, 2013.
4. Durrett, R.: Elementary probability for applications. Cambridge University
Press, Cambridge, 2009.
5. Feller, W.: An introduction to probability theory and its applications. Vol. I. John
Wiley & Sons, Inc., New York-London-Sydney,1968.
6. Grimmett, G.R.; Stirzaker, D. R.: One thousand exercises in probability. Oxford
University Press, Oxford, 2001.
7. Häggström, O.: Finite Markov chains and algorithmic applications. London
Mathematical Society Student Texts, 52. Cambridge University Press, Cam-
bridge, 2002.
8. Kingman, J. F. C.: Poisson processes. Oxford University Press, New York, 1993.
9. Levin, D. A.; Peres, Y.; Wilmer, E.L.: Markov chains and mixing times. Amer-
ican Mathematical Society, Providence, RI, 2009.
10. Rozanov, Y. A.: Probability theory. A concise course. Dover Publications, Inc.,
New York, 1977.

237
Index

absolute convergence, 11 clique number, 62

Azuma inequality, 126 concentration of measure, 80
3-SAT, 140 conditional distribution, 47
balls in boxes, 134 Fubini theorem, 47
chromatic number, 133 conditional expectation, 47
empirical process, 140 conditional probability, 38
Hamming cube, 139 confidence intervals, 196
max-cut, 136 confidence level, 196
confidence set, 199
balancing vectors, 1 continuous distribution, 147
balls in boxes, 59, 133 convergence in distribution, 167
Bernoulli distribution, 19 convolution, 180
variance, 73 countable additivity, 15
Bernstein polynomials, 82 coupling, 29
multivariate, 84 covariance, 73
biased coin, 19 random vector, 160
binomial distribution, 23 cumulative distribution function, 148
Poisson approximation, 30
binomial formula, 23 density, 143, 147
birth-and-death chains, 230 convolution, 180
Bolzano–Weierstrass theorem, 210 of ratio, 184
standard Gaussian, 143
Cauchy-Schwarz inequality, 74, 94 detailed balance equations, 229
centering, 72 distribution, 9, 13
central limit theorem, 167 F-distribution, 184, 199
Lyapunov condition, 174 t-distribution, 186
change of variables, 8, 9, 13, 150, 183 Bernoulli, 19
expectation, 152, 156 binomial, 23
Chebyshev’s inequality, 63, 65, 69, 78, chi-squared, 183
87, 99 continuous, 147
chi-squared distribution, 183, 191, 196 exponential, 147
classification, 121 Gamma, 179
clique, 62, 86 Gaussian, 149

239
240 Index

geometric, 25 Hamming cube, 137

marginal, 48 Hardy–Ramanujan theorem, 95
multinomial, 24 Hoeffding inequality, 106, 141
Poisson, 25 Hoeffding–Chernoff inequality, 119
Rademacher, 106
standard Gaussian, 143 i.i.d., 21, 76
standard Gaussian on Rn , 154 image measure, 10, 13
uniform, 147 inclusion-exclusion principle, 57
balls in boxes, 59
independence, 6, 19, 38, 154, 164
Ehrenfest model of diffusion, 230
conditional, 55
ellipse, 199
convolution, 180
empirical error, 122
covariance, 74
empirical process, 140
expectation, 44, 45
Erdős–Rényi graph, 22
pairwise, 39
chromatic number, 132
independent set, 94
cliques, 62, 66, 86
indicator function, 6
sparse, 31, 135
initial distribution, 202
expectation, 5, 12, 149, 150
change of variables, 152, 156 Jensen’s inequality, 72, 83, 175
conditional, 47 Johnson–Lindenstrauss lemma, 112
countable additivity, 15 joint distribution, 48
geometric, 25
linearity, 6, 12 Kullback-Leibler divergence, 118
monotonicity, 12 variational formula, 124
Poisson, 26
standard Gaussian, 149 law of large numbers, 79, 233
tail probability formula, 16, 219 least-squares line, 190
exponential distribution, 147 likelihood function, 189
Lindeberg’s method, 166
linearity of expectation
fair coin, 19, 78, 106
balls in boxes, 56
finite additivity, 4
longest increasing subsequence, 60, 64
Fisher–Snedecor distribution, 184
loss function, 122
Fubini, 165
low-distortion embedding, 112
Fubini theorem, 45, 47, 127
Lyapunov condition, 174
Gamma distribution, 179 marginal distribution, 48
stability, 180 Markov chain, 52, 201
Gamma function,
p 179 aperiodic, 206
G(1/2) = p, 183 birth-and-death, 230
Gaussian distribution, 143, 149 convergence theorem, 225
on Rn , 154 Ehrenfest model, 230
orthogonal transformation, 161, 195 finite state, 202
stability, 157 homogeneous, 202
with covariance C, 160, 191 initial distribution, 202
Gaussian integration by parts, 153 irreducible, 204
Gaussian moments, 153 random walks on graphs, 231
generalization error, 121, 122 reversible, 229
geometric distribution, 25 states, 202
variance, 76 stationary distribution, 210
Grouping lemma, 42 transition graph, 203
Index 241

transition matrix, 202 Rademacher sums, 109

transition probability, 202 moments, 109
Markov Chain Monte Carlo, 232 random matrix, 113
Markov property, 53, 201, 217 random permutation, 60, 175
Markov’s inequality, 106, 130 random variables, 5
martingale-difference representation, Bernoulli, 19
129 centered, 72
matching problem, 70, 94 i.i.d., 21
max-cut, 134, 136 independent, 7, 164
maximum likelihood estimators, 189, integrable, 12
190 Rademacher, 106, 113, 146
memoryless property, 53 standard Gaussian, 145
Mertens’ theorems, 95 uncorrelated, 74, 98
Metropolis algorithm, 232 random walks on graphs, 231
moments, 71 regression line, 188
Gamma distribution, 180 relative entropy, 118
Gaussian, 153 replica trick, 144
Rademacher sums, 109 Riemann vs. Lebesgue, 144
tail probability formula, 18
multinomial distribution, 24 sample space, 9, 20
sample variance, 190
nice functions, 145 second moment, 71
Noah’s ark, 70, 94 simple linear regression, 188
noise, 188 confidence intervals, 196
distribution of estimates, 191
orthogonal transformation, 161, 195 predictor, 188
outcomes, 3 response, 188
sparse graph, 31, 135
pairwise independence, 39 standard deviation, 71
period, 206 states, 202
Poisson approximation, 30 absorbing, 204
lottery, 30 aperiodic, 206
shark attacks, 35 clusters, 204
Wayne Gretzky, 36 communicating, 204
Poisson distribution, 25 inessential, 204
colouring property, 50 period, 206
stability, 26, 27, 31, 43 transient, 204
variance, 73 stationary distribution, 210
probability, 3, 11 convergence, 225
conditional, 38 existence, 210
countable additivity, 15 representation, 219
finite additivity, 4 uniqueness, 211, 222
monotonicity, 11 Stirling’s formula, 64, 101
uniform, 4 stopping time, 215
union bound, 5 return time, 215
probability space, 3, 11 strong Markov property, 214
product of Bernoulli, 19 Student t-distribution, 186, 197
product space, 33, 40, 164
tail probability, 15, 16, 60
Rademacher, 146 Toronto weather, 223
242 Index

training data, 121 typically, 62, 87

transition graph, 203
transition matrix, 202 uniform distribution, 147
transition point, 67, 86 union bound, 5, 35
transition probability, 53, 202
n-step, 205 variance, 71
triangular array, 174 of the sum, 75
two-stage experiment, 48 standard Gaussian, 149
Poisson colouring, 50
shark attacks, 48 Weierstrass theorem, 82