Measure Theoretic Probability

(condensed lecture notes)

Tinbergen Institute

P.J.C. Spreij

this version: September 21, 2021

Preface
In these notes we explain the measure theoretic foundations of modern probability.
The notes are used during a course that had as one of its principal aims a swift
introduction to measure theory as far as it is needed in modern probability, e.g. to
define concepts as conditional expectation.
Everyone with a basic notion of mathematics and probability would understand
what is meant by f (x) and P(A). In the former case we have the value of some function
f evaluated at its argument. In the second case, one recognizes the probability of an
event A. Look at the notations, they are quite similar and this suggests that also P is
a function, defined on some domain to which A belongs. This is indeed the point of
view that we follow. We will see that P is a function -a special case of a measure- on a
collection of sets, that satisfies certain properties, a σ-algebra. In general, a σ-algebra
Σ will be defined as a suitable collection of subsets of a given set S. A measure µ will
then be a map on Σ, satisfying some defining properties. This gives rise to considering a
triple, to be called a measure space, (S, Σ, µ). We will develop probability theory in the
context of measure spaces and because of tradition and some distinguished features,
we will write (Ω, F, P) for a probability space instead of (S, Σ, µ). Given a measure
space we will develop in a rather abstract sense integrals of functions defined on S. In
a probabilistic context, these integrals have the meaning of expectations. The general
setup provides us with two big advantages. In the definition of expectations, we don’t
have to distinguish anymore between random variables having a discrete distribution
and those who have what is called a density. In the first case, expectations are usually
computed as sums, whereas in the latter case, Riemann integrals are the tools. We
will see that these are special cases of the more general notion of Lebesgue integral.
Another advantage is the availability of convergence theorems. In analytic terms, we
will see that integrals of functions converge to the integral of a limit function, given
appropriate conditions and an appropriate concept of convergence. In a probabilistic
context, this translates to convergence of expectations of random variables. We will
see many instances, where the foundations of the theory can be fruitfully applied to
fundamental issues in probability theory.
The present set of lecture notes, composed for the course on Measure theory and
asymptotic statistics at the Tinbergen Institute is a short version of the extended set
Measure theoretic probability.
Amsterdam, October 2018
Peter Spreij
Contents
1 σ-algebras and measures 1
1.1 σ-algebras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Null sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 π- and d-systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.5 Probability language . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Measurable functions and random variables 9

2.1 General setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Integration 16
3.1 Integration of simple functions . . . . . . . . . . . . . . . . . . . . . . 16
3.2 A general definition of integral . . . . . . . . . . . . . . . . . . . . . . 19
3.3 Integrals over subsets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4 Expectation and integral . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.5 Lp -spaces of random variables . . . . . . . . . . . . . . . . . . . . . . . 26
3.6 Lp -spaces of functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4 Product measures 32
4.1 Product of two measure spaces . . . . . . . . . . . . . . . . . . . . . . 32
4.2 Application in Probability theory . . . . . . . . . . . . . . . . . . . . . 35
4.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Derivative of a measure 41
5.1 Absolute continuity and singularity . . . . . . . . . . . . . . . . . . . . 41
5.2 The Radon-Nikodym theorem . . . . . . . . . . . . . . . . . . . . . . . 42
5.3 Decomposition of a distribution function . . . . . . . . . . . . . . . . . 43
5.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6 Conditional expectation 46
6.1 A simple, finite case . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.2 Conditional expectation for X ∈ L1 (Ω, F, P) . . . . . . . . . . . . . . . 47
6.3 Conditional probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . 50
6.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
1 σ-algebras and measures
In this chapter we lay down the measure theoretic foundations of probability
theory. We start with some general notions and show how these are instrumental
in a probabilistic environment.

1.1 σ-algebras
Definition 1.1 Let S be a non-empty set. A collection Σ0 ⊂ 2S is called an
algebra (on S) if
(i) S ∈ Σ0 ,
(ii) E ∈ Σ0 ⇒ E c ∈ Σ0 ,
(iii) E, F ∈ Σ0 ⇒ E ∪ F ∈ Σ0 .

Notice that always ∅ belongs to an algebra, since ∅ = S c . Of course property

(iii) extends to finite unions by induction. Moreover, in an algebra we also
have E, F ∈ Σ0 ⇒ E ∩ F ∈ Σ0 , since E ∩ F = (E c ∪ F c )c . Furthermore
E \ F = E ∩ F c ∈ Σ0 . It follows that an algebra is closed under finitely many
of the usual set operations.
S
Definition 1.2 Let S be a non-empty set.S∞A collection Σ ⊂ 2 is called a
σ-algebra (on S) if it is an algebra and n=1 En ∈ Σ as soon as En ∈ Σ
(n = 1, 2 . . .).

Alternatively, a collection
S∞ Σ is a σ-algebra if (i) and (ii) of Definition 1.1 are
valid together with n=1 En ∈ Σ as soon as En ∈ Σ (n = 1, 2 . . .). It follows
that an algebra is closed under countably many of the usual set operations.
If Σ is a σ-algebra on S, then (S, Σ) is called a measurable space and the
elements of Σ are called measurable sets. We shall ‘measure’ them in the next
section.
If C is any collection of subsets of S, then by σ(C) we denote the smallest σ-
algebra containing C. This means that σ(C) is the intersection of all σ-algebras
that contain C (see Exercise 1.1). If Σ = σ(C), we say that C generates Σ. The
union of two σ-algebras Σ1 and Σ2 on a set S is usually not a σ-algebra. We
write Σ1 ∨ Σ2 for σ(Σ1 ∪ Σ2 ).
One of the most relevant σ-algebras of this course is B = B(R), the Borel sets
of R. Let O be the collection of all open subsets of R with respect to the usual
topology (in which all intervals (a, b) are open). Then B := σ(O). Of course,
one similarly defines the Borel sets of Rd , and in general, for a topological space
(S, O), one defines the Borel-sets as σ(O). Borel sets can in principle be rather
‘wild’, but it helps to understand them a little better, once we know that they
are generated by simple sets. The σ-algebra B(R) is also generated by all open
intervals, or all closed intervals. An alternative generating collection is given
next.

Proposition 1.3 Let I = {(−∞, x] : x ∈ R}. Then σ(I) = B.

1
Proof We prove the two obvious inclusions, starting with σ(I) ⊂ B. Since
(−∞, x] = ∩n (−∞, x + n1 ) ∈ B, we have I ⊂ B and then also σ(I) ⊂ B, since
σ(I) is the smallest σ-algebra that contains I. (Below we will use this kind of
arguments repeatedly).
For the proof of the reverse inclusion we proceed in three steps. First we
observe that (−∞, x) = ∪n (−∞, x − n1 ] ∈ σ(I). Knowing this, we conclude
that (a, b) = (−∞, b) \ (−∞, a] ∈ σ(I). Let then G be an arbitrary open set.
Since G is open, for every x ∈ G there exists a rational εx > 0 such that
(x − 2εx , x + 2εx ) ⊂ G. Consider (x − εx , x + εx ) and choose a rational qx in
this interval, note that |x − qx | ≤ εx . It follows that x ∈ (qx − εx , qx + εx ) ⊂
(x − 2εx , x + 2εx ) ⊂ G. Hence G ⊂ ∪x∈G (qx − εx , qx + εx ) ⊂ G, and so
G = ∪x∈G (qx − εx , qx + εx ). But the union here is in fact a countable union,
since there are only countably many qx and εx . (Note that the arguments
above can be used for any metric space with a countable dense subset to get
that an open G is a countable union of open balls.) It follows that G ∈ σ(I),
hence O ⊂ σ(I), and therefore (recall B is the smallest σ-algebra containing O)
B ⊂ σ(I). 

An obvious question to ask is whether every subset of R belongs to B = B(R).

The answer is no. It is a fact, albeit not easy to prove, that the cardinality
of B(R) is the same as the cardinality of R, from which the negative answer
follows.

1.2 Measures
Let Σ0 be an algebra on a set S, and Σ be a σ-algebra on S. We consider
mappings µ0 : Σ0 → [0, ∞] and µ : Σ → [0, ∞]. Note that ∞ is allowed as a
possible value.
We call µ0 finitely additive if µ0 (∅) = 0 and if µ0 (E ∪ F ) = µ0 (E) + µ0 (F )
for every pair of disjoint sets E and F in Σ0 . Of course this addition rule then
extends to arbitrary finite unions of disjoint sets. The mapping P µ0 is called
σ-additive or countably additive, if µ0 (∅) = 0 and if µ0 (∪n En ) = n µ0 (En ) for
every sequence (En ) of disjoint sets of Σ0 whose union is also in Σ0 . σ-additivity
is defined similarly for µ, but then we don’t have to require that ∪n En ∈ Σ.
This is true by definition.

Definition 1.4 Let (S, Σ) be a measurable space. A countably additive map-

ping µ : Σ → [0, ∞] is called a measure. The triple (S, Σ, µ) is called a measure
space.

Some extra terminology follows. A measure is called finite if µ(S) < ∞. It

is called σ-finite, if we can write S = ∪n Sn , where the Sn are measurable sets
and µ(Sn ) < ∞. If µ(S) = 1, then µ is called a probability measure.
Measures are used to ‘measure’ (measurable) sets in one way or another.
Here is a simple example. Let S = N and Σ = 2N (we often take the power set
as the σ-algebra on a countable set). Let τ (we write τ instead of µ for this

2
special case) be the counting measure: τ (E) = |E|, the cardinality of E. One
easily verifies that τ is a measure, and it is σ-finite, because N = ∪n {1, . . . , n}.
A very simple measure is the Dirac measure. Consider a measurable space
(S, Σ) and single out a specific x0 ∈ S. Define δ(E) = 1E (x0 ), for E ∈ Σ (1E is
the indicator function of the set E, 1E (x) = 1 if x ∈ E and 1E (x) = 0 if x ∈ / E).
Check that δ is a measure on Σ.
Another example is Lebesgue measure, whose existence is formulated below.
It is the most natural candidate for a measure on the Borel sets on the real line.
Theorem 1.5 There exists a unique measure λ on (R, B) with the property
that for every interval I = (a, b] with a < b it holds that λ(I) = b − a.
The proof of this theorem can be found in the literature, or in the extended
version of these notes. We take this existence result for granted. One remark
is in order. One can show that B is not the largest σ-algebra for which the
measure λ can coherently be defined. On the other hand, on the power set of
R it is impossible to define a measure that coincides with λ on the intervals.
Here are the first elementary properties of a measure.
Proposition 1.6 Let (S, Σ, µ) be a measure space. Then the following hold
true (all the sets below belong to Σ).
(i) If E ⊂ F , then µ(E) ≤ µ(F ).
(ii) µ(E ∪ F ) ≤ µ(E) + µ(F ).
Pn
(iii) µ(∪nk=1 Ek ) ≤ k=1 µ(Ek )
If µ is finite, we also have
(iv) If E ⊂ F , then µ(F \ E) = µ(F ) − µ(E).
(v) µ(E ∪ F ) = µ(E) + µ(F ) − µ(E ∩ F ).
Proof The set F can be written as the disjoint union F = E ∪ (F \ E). Hence
µ(F ) = µ(E) + µ(F \ E). Property (i) now follows and (iv) as well, provided µ
is finite. To prove (ii), we note that E ∪ F = E ∪ (F \ (E ∩ F )), a disjoint union,
and that E ∩ F ⊂ F . The result follows from (i). Moreover, (v) also follows, if
we apply (iv). Finally, (iii) follows from (ii) by induction. 
Measures have certain continuity properties.
Proposition 1.7 Let (En ) be a sequence in Σ.
(i) If the sequence is increasing, with limit E = ∪n En , then µ(En ) ↑ µ(E) as
n → ∞.
(ii) If the sequence is decreasing, with limit E = ∩n En and if µ(En ) < ∞
from a certain index on, then µ(En ) ↓ µ(E) as n → ∞.
Proof (i) Define D1 = E1 and Dn = En \ ∪n−1 k=1 Ek for n ≥ 2. Then the
Dn are disjoint,
Pn E n = ∪
P∞
n
k=1 D k for n ≥ 1 and E = ∪∞k=1 Dk . It follows that
µ(En ) = k=1 µ(Dk ) ↑ k=1 µ(Dk ) = µ(E).
To prove (ii) we assume without loss of generality that µ(E1 ) < ∞. Define
Fn = E1 \ En . Then (Fn ) is an increasing sequence with limit F = E1 \ E. So
(i) applies, yielding µ(E1 ) − µ(En ) ↑ µ(E1 ) − µ(E). The result follows. 

3
Corollary 1.8 Let (S, Σ, µ) be a measure P∞space. For an arbitrary sequence
(En ) of sets in Σ, we have µ(∪∞ E
n=1 n ) ≤ n=1 µ(En ).

Proof Exercise 1.2. 

Remark 1.9 The finiteness condition in the second assertion of Proposition 1.7
is essential. Consider N with the counting measure τ . Let Fn = {n, n + 1, . . .},
then ∩n Fn = ∅ and so it has measure zero. But τ (Fn ) = ∞ for all n.

1.3 Null sets

Consider a measure space (S, Σ, µ) and let E ∈ Σ be such that µ(E) = 0. If N
is a subset of E, then it is fair to suppose that also µ(N ) = 0. But this can only
be guaranteed if N ∈ Σ. Therefore we introduce some new terminology. A set
N ⊂ S is called a null set or µ-null set, if there exists E ∈ Σ with E ⊃ N and
µ(E) = 0. The collection of null sets is denoted by N , or Nµ since it depends
on µ. In Exercise 1.5 you will be asked to show that N is a σ-algebra and to
extend µ to Σ̄ = Σ∨N . If the extension is called µ̄, then we have a new measure
space (S, Σ̄, µ̄), which is complete, all µ̄-null sets belong to the σ-algebra Σ̄.

1.4 π- and d-systems

In general it is hard to grab what the elements of a σ-algebra Σ are, but often
collections C such that σ(C) = Σ are easier to understand. In ‘good situations’
properties of Σ can easily be deduced from properties of C. This is often the
case when C is a π-system, to be defined next.

Definition 1.10 A collection I of subsets of S is called a π-system, if I1 , I2 ∈ I

implies I1 ∩ I2 ∈ I.

It follows that a π-system is closed under finite intersections. In a σ-algebra, all

familiar set operations are allowed, at most countably many. We will see that it
is possible to disentangle the defining properties of a σ-algebra into taking finite
intersections and the defining properties of a d-system. This is the content of
Proposition 1.12 below.

Definition 1.11 A collection D of subsets of S is called a d-system, if the

following hold.
(i) S ∈ D.
(ii) If E, F ∈ D such that E ⊂ F , then F \ E ∈ D.
(iii) If En ∈ D for n ∈ N, and En ⊂ En+1 for all n, then ∪n En ∈ D.

Proposition 1.12 Σ is a σ-algebra iff it is a π-system and a d-system.

Proof Let Σ be a π-system and a d-system. We check the defining conditions

of a σ-algebra. (i) Since Σ is a d-system, S ∈ Σ. (ii) Complements of sets
in Σ are in Σ as well, again because Σ is a d-system. (iii) If E, F ∈ Σ, then

4
E ∪ F = (E c ∩ F c )c ∈ Σ, because we have just shown that complements remain
in Σ and because Σ is a π-system. Then Σ is also closed under finite unions.
Let E1 , E2 , . . . be a sequence in Σ. We have just showed that the sets Fn =
∪ni=1 Ei ∈ Σ. But since the Fn form an increasing sequence, also their union
is in Σ, because Σ is a d-system. But ∪n Fn = ∪n En . This proves that Σ is a
σ-algebra. Of course the other implication is trivial. 

If C is a collection of subsets of S, then by d(C) we denote the smallest d-system

that contains C. Note that it always holds that d(C) ⊂ σ(C). In one important
case we have equality. This is known as Dynkin’s lemma.
Lemma 1.13 Let I be a π-system. Then d(I) = σ(I).

Proof Suppose that we would know that d(I) is a π-system as well. Then
Proposition 1.12 yields that d(I) is a σ-algebra, and so it contains σ(I). Since
the reverse inclusion is always true, we have equality. Therefore we will prove
that indeed d(I) is a π-system.
Step 1. Put D1 = {B ∈ d(I) : B ∩ C ∈ d(I), ∀C ∈ I}. We claim that
D1 is a d-system. Given that this holds and because, obviously, I ⊂ D1 , also
d(I) ⊂ D1 . Since D1 is defined as a subset of d(I), we conclude that these
two collections are the same. We now show that the claim holds. Evidently
S ∈ D1 . Let B1 , B2 ∈ D1 with B1 ⊂ B2 and C ∈ I. Write (B2 \ B1 ) ∩ C as
(B2 ∩ C) \ (B1 ∩ C). The last two intersections belong to d(I) by definition of
D1 and so does their difference, since d(I) is a d-system. For Bn ↑ B, Bn ∈ D1
and C ∈ I we have (Bn ∩ C) ∈ d(I) which then converges to B ∩ C ∈ d(I). So
B ∈ D1 .
Step 2. Put D2 = {C ∈ d(I) : B ∩ C ∈ d(I), ∀B ∈ d(I)}. We claim, again,
(and you check) that D2 is a d-system. The key observation is that I ⊂ D2 .
Indeed, take C ∈ I and B ∈ d(I). The latter collection is nothing else but D1 ,
according to step 1. But then B ∩ C ∈ d(I), which means that C ∈ D2 . It now
follows that d(I) ⊂ D2 , but then we must have equality, because D2 is defined
as a subset of d(I). The equality D2 = d(I) and the definition of D2 together
imply that d(I) is a π-system, as desired. 

Sometimes another version of Lemma 1.13 is useful.

Corollary 1.14 The assertion of Lemma 1.13 is equivalent to the following
statement. Let I be a π-system and D be a d-system. If I ⊂ D, then σ(I) ⊂ D.

Proof Suppose that I ⊂ D. Then d(I) ⊂ D. But d(I) = σ(I), according to

Lemma 1.13. Conversely, let I be a π-system. Then I ⊂ d(I). By hypothesis,
one also has σ(I) ⊂ d(I), and the latter is always a subset of σ(I). 

All these efforts lead to the following very useful theorem. It states that any
finite measure on Σ is characterized by its action on a rich enough π-system.
We will meet many occasions where this theorem is used.

5
Theorem 1.15 Let I be a π-system and Σ = σ(I). Let µ1 and µ2 be finite
measures on Σ with the properties that µ1 (S) = µ2 (S) and that µ1 and µ2
coincide on I. Then µ1 = µ2 (on Σ).

Proof The whole idea behind the proof is to find a good d-system that contains
I. The following set is a reasonable candidate. Put D = {E ∈ Σ : µ1 (E) =
µ2 (E)}. The inclusions I ⊂ D ⊂ Σ are obvious. If we can show that D is a
d-system, then Corollary 1.14 gives the result. The fact that D is a d-system is
straightforward to check, we present only one verification. Let E, F ∈ D such
that E ⊂ F . Then (use Proposition 1.6 (iv)) µ1 (F \ E) = µ1 (F ) − µ1 (E) =
µ2 (F ) − µ2 (E) = µ2 (F \ E) and so F \ E ∈ D. 

Remark 1.16 In the above proof we have used the fact that µ1 and µ2 are
finite. If this condition is violated, then the assertion of the theorem is not
valid in general. Here is a counterexample. Take N with the counting measure
µ1 = τ and let µ2 = 2τ . A π-system that generates 2N is given by the sets
Gn = {n, n + 1, . . .} (n ∈ N).

1.5 Probability language

In Probability Theory, one usually writes (Ω, F, P) instead of (S, Σ, µ), and
one then speakes of a probability space. On one hand this is merely change of
notation and language. We still have that Ω is a set, F a σ-algebra on it, and P
a measure, but in this case, P is a probability measure (often also simply called
probability), P(Ω) = 1. In probabilistic language, Ω is often called the set of
outcomes and elements of F are called events. So by definition, an event is a
measurable subset of the set of all outcomes.
A probability space (Ω, F, P) can be seen as a mathematical model of a random
experiment. Consider for example the experiment consisting of tossing two
coins. Each coin has individual outcomes 0 and 1. The set Ω can then be
written as {00, 01, 10, 11}, where the notation should be obvious. In this case,
we take F = 2Ω and a choice of P could be such that P assigns probability 14
to all singletons. Of course, from a purely mathematical point of view, other
possibilities for P are conceivable as well.
A more interesting example is obtained by considering an infinite sequence
of coin tosses. In this case one should take Ω = {0, 1}N and an element ω ∈ Ω
is then an infinite sequence (ω1 , ω2 , . . .) with ωn ∈ {0, 1}. It turns out that one
cannot take the power set of Ω as a σ-algebra, if one wants to have a nontrivial
probability measure defined on it. As a matter of fact, this holds for the same
reason that one cannot take the power set on (0, 1] to have a consistent notion
of Lebesgue measure. This has everything to do with the fact that one can set
up a bijective correspondence between (0, 1) and {0, 1}N . Nevertheless, there
is a good candidate for a σ-algebra F on Ω. One would like to have that sets
like ‘the 12-th outcome is 1’ are events. Let C be the collection of all such sets,
C = {{ω ∈ Ω : ωn = s}, n ∈ N, s ∈ {0, 1}}. We take F = σ(C) and all sets
{ω ∈ Ω : ωn = s} are then events. One can show that there indeed exists a

6
probability measure P on this F with the nice property that for instance the set
{ω ∈ Ω : ω1 = ω2 = 1} (in the previous example it would have been denoted by
{11}) has probability 14 .
Having the interpretation of F as a collection of events, we now introduce two
special events. Consider a sequence of events E1 , E2 , . . . and define
∞ [
\ ∞
lim sup En := En
m=1 n=m
[∞ \ ∞
lim inf En := En .
m=1 n=m

Note that the sets Fm = ∩n≥m En form an increasing sequence and the sets
Dm = ∪n≥m En form a decreasing sequence. Clearly, F is closed under taking
limsup and liminf. The terminology is explained by (i) of Exercise 1.4. In prob-
abilistic terms, lim sup En is described as the event that the En occur infinitely
often, abbreviated by En i.o. Likewise, lim inf En is the event that the En occur
eventually. The former interpretation follows by observing that ω ∈ lim sup En
iff for all m, there exists n ≥ m such that ω ∈ En . In other words, a particular
outcome ω belongs to lim sup En iff it belongs to some (infinite) subsequence of
(En ). S∞ T ∞
The terminology to call m=1 n=m En the lim inf of the sequence is justified
in Exercise 1.4. In this exercise, indicator functions of events are used, of which
we here recall the definition. If E is an event, then the function 1E is defined
by 1E (ω) = 1 if ω ∈ E and 1E (ω) = 0 if ω ∈ / E.

1.6 Exercises
1.1 Prove the following statements.
(a) The intersection of an arbitrary family of d-systems is again a d-system.
(b) The intersection of an arbitrary family of σ-algebras is again a σ-algebra.
(c) If C1 and C2 are collections of subsets of Ω with C1 ⊂ C2 , then d(C1 ) ⊂ d(C2 ).
1.2 Prove Corollary 1.8.
1.3 Prove the claim that D2 in the proof of Lemma 1.13 forms a d-system.
1.4 Consider a measure space (S, Σ, µ). Let (En ) be a sequence in Σ.
(a) Show that 1lim inf En = lim inf 1En .
(b) Show that µ(lim inf En ) ≤ lim inf µ(En ). (Use Proposition 1.7.)
(c) Show also that µ(lim sup En ) ≥ lim sup µ(En ), provided that µ is finite.
1.5 Let (S, Σ, µ) be a measure space. Call a subset N of S a (µ, Σ)-null set
if there exists a set N 0 ∈ Σ with N ⊂ N 0 and µ(N 0 ) = 0. Denote by N the
collection of all (µ, Σ)-null sets. Let Σ∗ be the collection of subsets E of S for
which there exist F, G ∈ Σ such that F ⊂ E ⊂ G and µ(G \ F ) = 0. For E ∈ Σ∗
and F, G as above we define µ∗ (E) = µ(F ).

7
 (a) Show that Σ∗ is a σ-algebra and that Σ∗ = Σ ∨ N (= σ(N ∪ Σ)).
(b) Show that µ∗ restricted to Σ coincides with µ and that µ∗ (E) doesn’t
depend on the specific choice of F in its definition.
(c) Show that the collection of (µ∗ , Σ∗ )-null sets is N .
1.6 Let G and H be two σ-algebras on Ω. Let C = {G ∩ H : G ∈ G, H ∈ H}.
Show that C is a π-system and that σ(C) = σ(G ∪ H).
Ω
1.7
P Let Ω be a countable set.P Let F = 2 and let p : Ω → [0, 1] satisfy
ω∈Ω p(ω) = 1. Put P(A) = ω∈A p(ω) for A ∈ F. Show that P is a probability
measure.
1.8 Let Ω be a countable set. Let A be the collection of A ⊂ Ω such that A
or its complement has finite cardinality. Show that A is an algebra. What is
d(A)?

1.9 Show that a finitely additive map µ : Σ0 → [0, ∞] is countably

T additive if
µ(Hn ) → 0 for every decreasing sequence of sets Hn ∈ Σ0 with n Hn = ∅. If
µ is countably additive, do we necessarily
T have µ(Hn ) → 0 for every decreasing
sequence of sets Hn ∈ Σ0 with n Hn = ∅?
1.10 Consider the collection Σ0 of subsets of R that can be written as a finite
union of disjoint intervals of type (a, b] with −∞ ≤ a ≤ b < ∞ or (a, ∞). Show
that Σ0 is an algebra and that σ(Σ0 ) = B(R).
1.11 Let S = {1, 2, 3, 4}, C = {{1, 2}, {3}}, F = σ(C). A measure µ is fixed by
µ({1, 2}) = 0, µ({3}) = 1, µ({4}) = 1.
(a) Give all sets in F (there are 8 of them) together with their measure.
(b) Show that {1} and {1, 2} are null sets. Do they belong to F?
(c) Let F 0 be the power set of S (all subsets of S). Put µ0 ({1}) = 0 and
µ0 (E) = µ(E) for all E ∈ F. Show that µ0 is the unique extension of µ to
F 0.
(d) What are the µ0 -null sets? Are they elements of F 0 ?

1.12 Consider R together with the Borel sets and the Lebesgue measure λ.
(a) Show that all finite and countable sets have measure zero. What is λ(Q)?
P
(b) Is λ(R) = x∈R λ({x}) (whatever the sum could mean)?

8
2 Measurable functions and random variables
In this chapter we define random variables as measurable functions on a proba-
bility space and derive some properties.

2.1 General setting

Let (S, Σ) be a measurable space. Recall that the elements of Σ are called
measurable sets. Also recall that B = B(R) is the collection of all the Borel sets
of R. Finally, recall some notation. For a mapping h : S → R and B ⊂ R, the
set {s ∈ S : h(s) ∈ B} is denoted h−1 [B].
Definition 2.1 A mapping h : S → R is called measurable if h−1 [B] ∈ Σ for
all B ∈ B.
It is clear that this definition depends on B and Σ. When there are more
σ-algebras in the picture, we sometimes speak of Σ-measurable functions, or
Σ/B-measurable functions, depending on the situation. If S is a topological
space with a topology T and if Σ = σ(T ), a measurable function h is called a
Borel measurable function.
Remark 2.2 Consider E ⊂ S. Recall that the indicator function of E is defined
by 1E (s) = 1 if s ∈ E and 1E (s) = 0 if s ∈
/ E. Check that 1E is a measurable
function iff E is a measurable set.
Sometimes one wants to extend the range of the function h to [−∞, ∞]. If this
happens to be the case, we extend B with the singletons {−∞} and {∞}, and
work with B̄ = σ(B ∪ {{−∞}, {∞}}). We call h : S → [−∞, ∞] measurable if
h−1 [B] ∈ Σ for all B ∈ B̄.
Below we will often use the shorthand notation {h ∈ B} for the set {s ∈ S :
h(s) ∈ B}. Likewise we also write {h ≤ c} for the set {s ∈ S : h(s) ≤ c}. Many
variations on this theme are possible.
Proposition 2.3 Let (S, Σ) be a measurable space and h : S → R.
(i) If C is a collection of subsets of R such that σ(C) = B, and if h−1 [C] ∈ Σ
for all C ∈ C, then h is measurable.
(ii) If {h ≤ c} ∈ Σ for all c ∈ R, then h is measurable.
(iii) If S is topological and h continuous, then h is measurable with respect to
the σ-algebra generated by the open sets.
(iv) If h is measurable and another function f : R → R is Borel measurable
(B/B-measurable), then f ◦ h is measurable as well.
Proof (i) Put D = {B ∈ B : h−1 [B] ∈ Σ}. One easily verifies that D is a
σ-algebra and it is evident that C ⊂ D ⊂ B. It follows that D = B.
(ii) This is an application of the previous assertion. Take C = {(−∞, c] : c ∈
R}.
(iii) Take as C the collection of open sets and apply (i).
(iv) Take B ∈ B, then f −1 [B] ∈ B since f is Borel. Because h is measurable,
we then also have (f ◦ h)−1 [B] = h−1 [f −1 [B]] ∈ Σ. 

9
Remark 2.4 There are many variations on the assertions of Proposition 2.3
possible. For instance in (ii) we could also use {h < c}, or {h > c}. Further-
more, (ii) is true for h : S → [−∞, ∞] as well. We proved (iv) by a simple
composition argument, which also applies to a more general situation. Let
(Si , Σi ) be measurable spaces (i = 1, 2, 3), h : S1 → S2 is Σ1 /Σ2 -measurable
and f : S2 → S3 is Σ2 /Σ3 -measurable. Then f ◦ h is Σ1 /Σ3 -measurable.

The set of measurable functions will also be denoted by Σ. This notation is

of course a bit ambiguous, but it turns out, that no confusion can arise. Re-
mark 2.2, in a way justifies this notation. The remark can, with the present
convention, be rephrased as 1E ∈ Σ iff E ∈ Σ. Later on we often need the set
of nonnegative measurable functions, denoted Σ+ .
Fortunately, the set Σ of measurable functions is closed under elementary
operations.

Proposition 2.5 We have the following properties.

(i) The collection Σ of Σ-measurable functions is a vector space and products
of measurable functions are measurable as well.
(ii) Let (hn ) be a sequence in Σ. Then also inf hn , sup hn , lim inf hn , lim sup hn
are in Σ, where we extend the range of these functions to [−∞, ∞]. The
set L, consisting of all s ∈ S for which limn hn (s) exists as a finite limit,
is measurable.

Proof (i) If h ∈ Σ and λ ∈ R, then λh is also measurable (use (ii) of the

previous proposition for λ 6= 0). To show that the sum of two measurable
functions is measurable, we first note that {(x1 , x2 ) ∈ R2 : x1 + x2 > c} =
∪q∈Q {(x1 , x2 ) ∈ R2 : x1 > q, x2 > c − q} (draw a picture!). But then we also
have {h1 +h2 > c} = ∪q∈Q ({h1 > q}∩{h2 > c−q}), a countable union. To show
that products of measurable functions are measurable is left as Exercise 2.1.
(ii) Since {inf hn ≥ c} = ∩n {hn ≥ c}, it follows that inf hn ∈ Σ. To sup hn
a similar argument applies, that then also yield measurability of lim inf hn =
supn inf m≥n hm and lim sup hn . To show the last assertion we consider h :=
lim sup hn − lim inf hn . Then h : S → [−∞, ∞] is measurable. The assertion
follows from L = {lim sup hn < ∞} ∩ {lim inf hn > −∞} ∩ {h = 0}. 

For later use we present the Monotone Class Theorem.

Theorem 2.6 Let H be a vector space of bounded functions, with the following
properties.
(i) 1 ∈ H.
(ii) If (fn ) is a nonnegative sequence in H such that fn+1 ≥ fn for all n, and
f := lim fn is bounded, then f ∈ H.
If, in addition, H contains the indicator functions of sets in a π-system I, then
H contains all bounded σ(I)-measurable functions.

10
Proof Put D = {F ⊂ S : 1F ∈ H}. One easily verifies that D is a d-system,
and that it contains I. Hence, by Corollary 1.14, we have Σ := σ(I) ⊂ D. We
will use this fact later in the proof.
Let f be a bounded, σ(I)-measurable function. Without loss of generality,
we may assume that f ≥ 0 (add a constant otherwise), and f < K for some real
constant K. Introduce the functions fn defined by fn = 2−n b2n f c. In explicit
terms, the fn are given by
n
X−1
K2
fn (s) = i2−n 1{i2−n ≤f <(i+1)2−n } (s).
i=0

Then we have for all n that fn is a bounded measurable function, fn ≤ f , and

fn ↑ f (check this!). Moreover, each fn lies in H. To see this, observe that
{i2−n ≤ f < (i + 1)2−n } ∈ Σ, since f is measurable. But then this set is also
an element of D, since Σ ⊂ D (see above) and hence 1{i2−n ≤f <(i+1)2−n } ∈ H.
Since H is a vector space, linear combinations remain in H and therefore fn ∈ H.
Property (ii) of H yields f ∈ H. 

2.2 Random variables

We return to the setting of Section 1.5 and so we consider a set (of outcomes) Ω
and F a σ-algebra (of events) defined on it. In this setting Definition 2.1 takes
the following form.

Definition 2.7 A function X : Ω → R is called a random variable if it is

(F-)measurable.

Following the tradition, we denote random variables by X (or other capital

letters), rather than by h, as in the previous sections. By definition, random
variables are nothing else but measurable functions with respect to a given σ-
algebra F. Given X : Ω → R, let σ(X) = {X −1 [B] : B ∈ B}. Then σ(X)
is a σ-algebra, and X is a random variable in the sense of Definition 2.7 iff
σ(X) ⊂ F. It follows that σ(X) is the smallest σ-algebra on Ω such that X is
a random variable. See also Exercise 2.2.
If we have a collection of mappings X := {Xi : Ω → R|i ∈ I}, then we denote
by σ(X) the smallest σ-algebra on Ω such that all the Xi become measurable.
See Exercise 2.3.
Having a probability space (Ω, F, P), a random variable X, and the measurable
space (R, B), we will use these ingredients to endow the latter space with a
probability measure. Define µ : B → [0, 1] by

µ(B) := P(X ∈ B) = P(X −1 [B]). (2.1)

It is straightforward to check that µ is a probability measure on B. Commonly

used alternative notations for µ are PX , or LX , LX . This probability measure is
referred to as the distribution of X or the law of X. Along with the distribution

11
of X, we introduce its distribution function, usually denoted by F (or FX ,
or F X ). By definition it is the function F : R → [0, 1], given by F (x) =
µ((−∞, x]) = P(X ≤ x).
Proposition 2.8 The distribution function of a random variable is right con-
tinuous, non-decreasing and satisfies limx→∞ F (x) = 1 and limx→−∞ F (x) = 0.
The set of points where F is discontinuous is at most countable.

Proof Exercise 2.4. 

The fundamental importance of distribution functions in probability is based

on the following proposition.

Proposition 2.9 Let µ1 and µ2 be two probability measures on B. Let F1 and

F2 be the corresponding distribution functions. If F1 (x) = F2 (x) for all x, then
µ1 = µ2 .

Proof Consider the π-system I = {(−∞, x] : x ∈ R} and apply Theorem 1.15.



This proposition thus states, in a different wording, that for a random variable
X, its distribution, the collection of all probabilities P(X ∈ B) with B ∈ B, is
determined by the distribution function FX .
We call any function on R that has the properties of Proposition 2.8 a distri-
bution function. Note that any distribution function is Borel measurable (sets
{F ≥ c} are intervals and thus in B). Below, in Theorem 2.10, we justify this
terminology. We will see that for any distribution function F , it is possible
to construct a random variable on some (Ω, F, P), whose distribution function
equals F . This theorem is founded on the existence of the Lebesgue measure λ
on the Borel sets B[0, 1] of [0, 1], see Theorem 1.5.
We now give a probabilistic translation of this theorem. Consider (Ω, F, P) =
([0, 1], B[0, 1], λ). Let U : Ω → [0, 1] be the identity map. The distribution of U
on [0, 1] is trivially the Lebesgue measure again, in particular the distribution
function F U of U satisfies F U (x) = x for x ∈ [0, 1] and so P(a < U ≤ b) =
F U (b) − F U (a) = b − a for a, b ∈ [0, 1] with a ≤ b. Hence, to the distribu-
tion function F U corresponds a probability measure on ([0, 1], B[0, 1]) and there
exists a random variable U on this space, such that U has F U as its distri-
bution function. The random variable U is said to have the standard uniform
distribution.
The proof of Theorem 2.10 (Skorokhod’s representation of a random variable
with a given distribution function) below is easy in the case that F is continuous
and strictly increasing (Exercise 2.6), given the just presented fact that a random
variable with a uniform distribution exists. The proof that we give below for
the general case just follows a more careful line of arguments, but is in spirit
quite similar.

12
Theorem 2.10 Let F be a distribution function on R. Then there exists a
probability space and a random variable X : Ω → R such that F is the distri-
bution function of X.

Proof Let (Ω, F, P) = ((0, 1), B(0, 1), λ). We define X − (ω) = inf{z ∈ R :
F (z) ≥ ω}. Then X − (ω) is finite for all ω and X − is Borel measurable function,
so a random variable, as this follows from the relation to be proven below, valid
for all c ∈ R and ω ∈ (0, 1),

X − (ω) ≤ c ⇔ F (c) ≥ ω. (2.2)

This equivalence can be represented as {X − ≤ c} = [0, F (c)]. It also shows that

X − serves in a sense as an inverse function of F . We now show that (2.2) holds.
The implication F (c) ≥ ω ⇒ X − (ω) ≤ c is immediate from the definition of
X − . Conversely, let z > X − (ω). Then F (z) ≥ ω, by definition of X − . We now
take a sequence of zn > X − (ω) and zn ↓ X − (ω). Since F is right continuous, we
obtain F (X − (ω)) ≥ ω. It trivially holds that F (X − (ω)) ≤ F (c) if X − (ω) ≤ c,
because F is non-decreasing. Combination with the previous inequality yields
F (c) ≥ ω. This proves (2.2). In order to find the distribution function of X − ,
we compute P(X − ≤ c) = P([0, F (c)]) = λ([0, F (c)]) = F (c). 

2.3 Independence
Recall the definition of independent events. Two events E, F ∈ F are called
independent if the product rule P(E ∩ F ) = P(E)P(F ) holds. In the present
section we generalize this notion of independence to independence of a sequence
of events and to independence of a sequence of σ-algebras. It is even convenient
and elegant to start with the latter.

Definition 2.11 We have the following different definitions of independence,

in decreasing order of generality.
(i) A sequence of σ-algebras F1 , F2 , . Q
. . is called independent, if for every n
n
it holds that P(E1 ∩ · · · ∩ En ) = i=1 P(Ei ), for all choices of Ei ∈ Fi
(i = 1, . . . , n).
(ii) A sequence of random variables X1 , X2 , . . . is called independent if the
σ-algebras σ(X1 ), σ(X2 ), . . . are independent.
(iii) A sequence of events E1 , E2 , . . . is called independent if the random vari-
ables 1E1 , 1E2 , . . . are independent.

The above definition also applies to finite sequences. For instance, a finite se-
quence of σ-algebras F1 , . . . , Fn is called independent if the infinite sequence
F1 , F2 , . . . is independent in the sense of part (ii) of the above definition, where
Fm = {∅, Ω} for m > n. It follows that two σ-algebras F1 and F2 are indepen-
dent, if P(E1 ∩ E2 ) = P(E1 )P(E2 ) for all E1 ∈ F1 and E2 ∈ F2 . It also follows
that two events E and F are independent iff σ(E) and σ(F ) are independent
σ-algebras, this is Exercise 2.12.

13
 To check independence of two σ-algebras, Theorem 1.15 is again helpful. It
tells you that for two σ-algebras to be independent, it is sufficient to check the
product rule for generating π-systems.

Proposition 2.12 Let I and J be π-systems and suppose that for all I ∈ I
and J ∈ J the product rule P(I ∩ J) = P(I)P(J) holds. Then the σ-algebras
σ(I) and σ(J ) are independent.

Proof Put G = σ(I) and H = σ(J ). We define for each I ∈ I the finite
measures µI and νI on H by µI (H) = P(H ∩I) and νI (H) = P(H)P(I) (H ∈ H).
Notice that µI and νI coincide on J by assumption and that µI (Ω) = P(I) =
νI (Ω). Theorem 1.15 yields that µI (H) = νI (H) for all H ∈ H.
Now we consider for each H ∈ H the finite measures µH and ν H on G
defined by µH (G) = P(G ∩ H) and ν H (G) = P(G)P(H). By the previous step,
we see that µH and ν H coincide on I. Invoking Theorem 1.15 again, we obtain
P(G ∩ H) = P(G)P(H) for all G ∈ G and H ∈ H. 

We next present an important consequence concerning two random variables

X1 and X2 for which we need some notation. Put FX : R2 → [0, 1], defined by
FX (x1 , x2 ) = P({X1 ≤ x1 } ∩ {X2 ≤ x2 }). The function FX is called the joint
distribution function of X1 and X2 .
Corollary 2.13 Let X1 , X2 be random variables defined on some (Ω, F, P).
Then X1 and X2 are independent iff P({X1 ≤ x1 } ∩ {X2 ≤ x2 }) = P(X1 ≤
x1 )P(X2 ≤ x2 ) for all x1 , x2 ∈ R. In terms of the distribution functions this can
be written as FX (x1 , x2 ) = FX1 (x1 )FX2 (x2 ) for all x1 , x2 ∈ R.

Proof Combine Proposition 2.12 and Exercise 2.2. 

2.4 Exercises
2.1 If h1 and h2 are Σ-measurable functions on (S, Σ, µ), then h1 h2 is Σ-
measurable too. Show this.
2.2 Let X be a random variable. Show that Π(X) := {{X ≤ x} : x ∈ R} is
a π-system and that it generates σ(X). Formulate a similar statement for a
two-dimensional random vector X.
2.3 Let {Yγ : γ ∈ C} be an arbitrary collection of random variables and {Xn :
n ∈ N} be a countable collection of random variables, all defined on the same
probability space.
(a) Show that σ{Yγ : γ ∈ C} = σ{Yγ−1 (B) : γ ∈ C, B ∈ B}.
S∞
(b) Let Xn = σ{X1 , . . . , Xn } (n ∈ N) and A = n=1 Xn . Show that A is an
algebra and that σ(A) = σ{Xn : n ∈ N}.
2.4 Prove Proposition 2.8.

14
2.5 Let F be a σ-algebra on Ω with the property that for all F ∈ F it holds
that P(F ) ∈ {0, 1}. Let X : Ω → R be F-measurable. Show that for some c ∈ R
one has P(X = c) = 1. (Hint: P(X ≤ x) ∈ {0, 1} for all x.)
2.6 Let F be a strictly increasing and continuous distribution function. Let U
be a random variable defined on some (Ω, F, P) having a uniform distribution
on [0, 1] and put X = F −1 (U ). Show that X is F-measurable and that it has
distribution function F .
2.7 Let F be a distribution function and put X + (ω) = inf{x ∈ R : F (x) > ω}.
Show that (next to X − ) also X + has distribution function F and that P(X + =
X − ) = 1 (Hint: P(X − ≤ q < X + ) = 0 for all q ∈ Q). Show also that X + is a
right continuous function and Borel-measurable.
2.8 Consider a probability space (Ω, F, P). Let I1 , I2 , I3 be π-systems on Ω
with the properties Ω ∈ Ik and Ik ⊂ F, for all k. Assume that for all Ik ∈ Ik
(k = 1, 2, 3)

P(I1 ∩ I2 ∩ I3 ) = P(I1 )P(I2 )P(I3 ).

Show that σ(I1 ), σ(I2 ), σ(I3 ) are independent.

2.9 Let G1 , G2 , . . . be sub-σ-algebras of a σ-algebra F on a set Ω and let G =
σ(G1 ∪ G2 ∪ . . .).
(a) Show that Π = {Gi1 ∩ Gi2 ∩ . . . ∩ Gik : k ∈ N, ik ∈ N, Gij ∈ Gij } is a
π-system that generates G.
(b) Assume that (Ω, F, P) is a probability space and that G1 , G2 , . . . is an in-
dependent sequence. Let M and N be disjoint subsets of N and put
M = σ(Gi , i ∈ M ) and N = σ(Gi , i ∈ N ). Show that M and N are
independent σ-algebras.
2.10 Consider an independent sequence X1 , X2 , . . .. Let Fn = σ(X1 , . . . , Xn )
and Tn = σ(Xn+1 , Xn+2 , . . .), n ≥ 1. Let I be the collection of events of the
type {X1 ∈ B1 , . . . , Xn ∈ Bn }, with the Bi Borel sets in R. Show that I is a
π-system that generates Fn . Find a π-system that generates Tn and show that
Fn and Tn are independent. (Use Proposition 2.12.)
2.11 Consider an infinite sequence of coin tosses. We take Ω = {H, T }∞ , a
typical element ω is an infinite sequence (ω1 , ω2 , . . .) with each ωn ∈ {H, T },
and F = σ({ω ∈ Ω : ωn = w}, w ∈ {H, T }, n ∈ N). Define functions Xn by
Xn (ω) = 1 if ωn = H and Xn (ω) = 0 if ωn = T .
(a) Show that all Xn are random variables, i.e. everyone of them is measurable.
Pn
(b) Let Sn = i=1 Xi . Show that also Sn is a random variable.
(c) Let p ∈ [0, 1] and Ep = {ω ∈ Ω : limn→∞ n1 Sn (ω) = p}. Show that Ep is
an F-measurable set.
2.12 Show that two events E and F are independent iff σ(E) and σ(F ) are
independent σ-algebras

15
3 Integration
In elementary courses on Probability Theory, there is usually a distinction be-
tween random variables X having a discrete distribution, on N say, and those
having a P density. In the former case we have for the expectation ER X the ex-
pression k k P(X = k), whereas in the latter case one has E X = xf (x) dx.
This distinction is annoying and not satisfactory from a mathematical point of
view. Moreover, there exist random variables whose distributions are neither
discrete, nor do they admit a density. Here is an example. Suppose Y and Z,
defined on the same (Ω, F, P), are independent random variables. Assume that
P(Y = 0) = P(Y = 1) = 12 and that Z has a standard normal distribution. Let
X = Y Z and F the distribution function of X. Easy computations (do them!)
yield F (x) = 21 (1[0,∞) (x) + Φ(x)). We see that F has a jump at x = 0 and is
differentiable on R \ {0}, a distribution function of mixed type. How to compute
E X in this case?
In this section we will see that expectations are special cases of the unifying
concept of Lebesgue integral, a sophisticated way of addition. Lebesgue integrals
have many advantages. It turns out that Riemann integrable functions (on
a compact interval) are always Lebesgue integrable w.r.t. Lebesgue measure
and that the two integrals are the same. Also sums are examples of Lebesgue
integral. Furthermore, the theory of Lebesgue integrals allows for very powerful
limit theorems. Below we work with a measure space (S, Σ, µ).

3.1 Integration of simple functions

Bearing in mind the elementary formula for the area of a rectangle and the
interpretation of the Riemann integral of a positive function as the area under
its graph, it is natural to define the integral of a multiple of an indicator function
a · 1E as a · µ(E), for E ∈ Σ. This should be seen as the product of height times
width, the classical formula, with height equal to a and width equal to the
measure of E, µ(E). We extend this definition to the class of simple functions.

Definition 3.1 A function f : S → [0, ∞) is called a nonnegative simple func-

tion, if it has a representation as a finite sum
n
X
f= ai 1Ai , (3.1)
i=1

where ai ∈ [0, ∞) and Ai ∈ Σ. The class of all nonnegative simple functions is

denoted by S+ .
The representation in (3.1) is inherently non-unique. Take n = 2, a1 = a2 = a,
A1 and A2 disjoint sets and think of a representation of f with n = 1. Note that
a simple function is measurable. Since we remember that Riemann integrals are
linear operators and knowing the definition of integral for an indicator function,
we now present the definition of the integral of f ∈ S+ .

16
Definition 3.2 Let f ∈ S+ . The (Lebesgue) integral of f with respect to the
measure µ is defined as
Z X n
f dµ := ai µ(Ai ), (3.2)
i=1

when f has representation (3.1).

R
Other notations that we often use for this integral are f (s) µ(ds) and µ(f ).
Note that if f = 1A , then µ(f ) = µ(1A ) = µ(A), so there is a bit of ambiguity in
the notation, but also a reasonable level of consistency. Note that µ(f ) ∈ [0, ∞]
and also that the above summation is well defined, since all quantities involved
are nonnegative, although possibly infinite.
For products ab for a, b ∈ [0, ∞], we use the convention ab = 0, when a = 0.

It should be clear that this definition of integral is, at first sight, troublesome.
As the representation of a simple function is not unique, one might wonder if
the just defined integral takes on different values for different representations.
This would be very bad, and fortunately it is not the case.
Proposition 3.3 Let f be a nonnegative simple function. Then the value of
the integral µ(f ) is independent of the chosen representation.
Proof Step 1. Let f be given by (3.1) and define φ : S → {0, 1}n by φ(s) =
(1A1 (s), . . . , 1An (s)). Let {0, 1}n = {u1 , . . . , um } where m = 2n and put Uk =
φ−1 (uk ). Then the collection {U1 , . . . , Um } is a measurable partition of S (the
sets Uk are measurable). We will also need the sets Si = {k : Uk ⊂ Ai } and
Tk = {i : Uk ⊂ Ai }. Note that these sets are dual in the sense that k ∈ Si iff
i ∈ Tk .
Below we will use the fact Ai = ∪k∈Si Uk , when we rewrite (3.1). We obtain
by interchanging the summation order
X X X
f= ai 1Ai = ai ( 1Uk )
i i k∈Si
X X
= ( ai )1Uk . (3.3)
k i∈Tk

Now apply the definition

P Pof µ(f ) by using the representation of f given by (3.3).
This gives µ(f ) = k ( i∈TPk ai )µ(UP k ). Interchanging
P the summation order, we
see that this is equal to i ai ( k∈Si µ(Uk )) = i ai µ(Ai ), which coincides
with (3.2). We conclude that if f is given by (3.1), we can also represent f in
a similar fashion by using a partition, and that both representations give the
same value for the integral.
Step 2: Suppose that we have two representations of a simple function f ,
one is as in (3.1) with the collection of Ai a measurable partition of S. The
other one is
m
X
f= bj 1Bj , (3.4)
j=1

17
where the Bj form a measurable partition of S as well. We obtain a third
measurable partition of S by taking the collection of all intersections Ai ∩ Bj .
Notice that if s ∈ Ai ∩ Bj , then f (s) = ai = bj and so we have the implication
Ai ∩Bj 6= ∅ ⇒ ai = bj . We compute the integral of f according to thePdefinition.
Of course, this yields (3.2) by using the representation (3.1) of f , but j bj µ(Bj )
if we use (3.4). Rewrite
X X X X
bj µ(Bj ) = bj µ(∪i (Ai ∩ Bj )) = bj µ(Ai ∩ Bj )
j j j i
XX XX
= bj µ(Ai ∩ Bj ) = ai µ(Ai ∩ Bj )
i j i j
X X X
= ai µ(Ai ∩ Bj ) = ai µ(∪j (Ai ∩ Bj ))
i j i
X
= ai µ(Ai ),
i

which shows that the two formulas for the integral are the same.
Step 3: Take now two arbitrary representations of f of the form (3.1)
and (3.4). According to step 1, we can replace each of them with a repre-
sentation in terms of a measurable partition, without changing the value of the
integral. According to step 2, each of the representations in terms of the par-
titions also gives the same value of the integral. This proves the proposition.


Corollary 3.4 Let f ∈ S+ and suppose Pn that f assumes the different values
0 ≤ a1 , . . . , an < ∞. Then µ(f ) = i=1 ai µ({f = ai }). If f is indentically zero,
then µ(f ) = 0.
Pn
Proof We have the representation f = i=1 ai 1{f =ai } . The expression for
µ(f ) follows from Definition 3.2, which is unambiguous by Proposition 3.3. The
result for the zero function follows by the representation f = 0 × 1S and the
convention ab = 0 for a = 0 and b ∈ [0, ∞].


Example 3.5 Here is an instructive example. Let (S, Σ, µ) = (N, 2N , τ ), with

counting measure τ . A function f on N can be identified with a sequence (fi ).
Then f can be represented in a somewhat cumbersome way (but it just means
that the value f (k) of the function f at the variable k equals the number fk )
by
∞
X
f (k) = fi 1{i} (k).
i=1

For now, we assume fi = 0 for i > n and fi ≥ 0Pfor i ≤ n; obviously, f is a

n
simple function. Since τ ({i}) = 1, we get τ (f ) = i=1 fi , nothing else but the
finite sum of the fi . In this case, integration is just summation. Of course, a

18
different representation would yield the same answer. A generalization occurs
when the set of values {fi : i ∈ N} is finite. Then f is still a simple function if
the fi are nonnegative, as in Corollary 3.4. But note that now it may happen
that τ (f ) = ∞ (think of all fi equal to one).

Example 3.6 Let (S, Σ, µ) = ([0, 1], B([0, 1]), λ) and f the indicator of the
rational numbers in [0, 1], f = 1Q∩[0,1] . We know that λ(Q ∩ [0, 1]) = 0 and it
follows that λ(f ) = 0. This f is a nice example of a function that is not Riemann
integrable, whereas its Lebesgue integral trivially exists and has a very sensible
value.

We say that a property of elements of S holds almost everywhere (usually abbre-

viated by a.e. or by µ-a.e.), if the set for which this property does not hold, has
measure zero. For instance, we say that two measurable functions are almost
everywhere equal, if µ({f 6= g}) = 0. Elementary properties of the integral are
listed below.
Proposition 3.7 Let f, g ∈ S+ and c ∈ [0, ∞).
(i) If f ≤ g a.e., then µ(f ) ≤ µ(g).
(ii) If f = g a.e., then µ(f ) = µ(g).
(iii) µ(f + g) = µ(f ) + µ(g) and µ(cf ) = cµ(f ).
P
Proof (i)PRepresent f and g by means of measurable partitions, f = i ai 1Ai
and g = j bj 1Bj . We have {f > g} = ∪i,j:ai >bj Ai ∩ Bj , and since µ({f >
g}) = 0, we have that µ(Ai ∩Bj ) = 0 if ai > bj . It follows that for all i and j, the
inequality ai µ(Ai ∩ Bj ) ≤ bj µ(Ai ∩ Bj ) holds. We use this in the computations
below.
X
µ(f ) = ai µ(Ai )
i
XX
= ai µ(Ai ∩ Bj )
i j
XX
≤ bj µ(Ai ∩ Bj )
i j
X
= bj µ(Bj ).
j

Assertion (ii) follows by a double application of (i), whereas (iii) can also be
proved by using partitions and intersections Ai ∩ Bj . 

3.2 A general definition of integral

We start with a definition, in which we use that we already know how to integrate
simple functions.

19
Definition 3.8 Let f be a nonnegative measurable function. The integral of
f is defined as µ(f ) := sup{µ(h) : h ≤ f, h ∈ S+ }, where µ(h) is as in Defini-
tion 3.2.

Notice that for functions f ∈ S+ , Definition 3.8 yields for µ(f ) the same as
Definition 3.2 in the previous section. Thus there is no ambiguity in notation
by using the same symbol µ. We immediately have some extensions of results
in the previous section.

Proposition 3.9 Let f, g ∈ Σ+ . If f = 0 a.e., then µ(f ) = 0. If f ≤ g a.e.,

then µ(f ) ≤ µ(g), and if f = g a.e., then µ(f ) = µ(g).

Proof Let f ∈ Σ+ , f = 0 a.e. Take h ∈ S+ with h ≤ f . From this inequality we

obtain {h > 0} ⊂ {f > 0}, and hence µ({h > 0}) ≤ µ({f > 0}), but the latter
measure is zero and hence h = 0 a.e. By Corollary 3.4 and Proposition 3.7(ii),
µ(h) = 0. Therefore, µ(f ), being the supremum of those µ(h), is also zero.
Let f, g ∈ Σ+ and N = {f > g}. Take h ∈ S+ with h ≤ f . Then also
h1N , h1N c ∈ S+ and by Proposition 3.7(iii) and the fact that h1N = 0 a.e., we
then have µ(h) = µ(h1N ) + µ(h1N c ) = µ(h1N c ). Moreover,

h1N c ≤ f 1N c ≤ g1N c ≤ g.

By definition of µ(g) (as a supremum), we obtain µ(h) ≤ µ(g). By taking the

supremum in this inequality over all h, we get µ(f ) ≤ µ(g), which gives the first
assertion. The other one immediately follows. 

Example 3.10 We extend the situation of Example P∞3.5, by allowing infinitely

many fi Pto be positive. The result will be τ (f ) = i=1 fi , classically defined as
n
limn→∞ i=1 fi . Check that this is in agreement with Definition 3.8. See also
Exercise 3.1.

The following will frequently be used.

Lemma 3.11 Let f ∈ Σ+ and suppose that µ(f ) = 0. Then f = 0 a.e.

Proof Because µ(f ) = 0, it holds that µ(h) = 0 for all nonnegative simple
functions with h ≤ f . Take hn = n1 1{f ≥1/n} , then hn ∈ S+ and hn ≤ f .
The equality µ(hn ) = 0 implies µ({f ≥ 1/n}) = 0. The result follows from
{f > 0} = ∪n {f ≥ 1/n} and Corollary 1.8. 

We now present the first important limit theorem, the Monotone Convergence
Theorem.
Theorem 3.12 Let (fn ) be a sequence in Σ+ , such that fn+1 ≥ fn a.e. for
each n. Let f = lim sup fn . Then µ(fn ) ↑ µ(f ) ≤ ∞.

Proof We first consider the case where fn+1 (s) ≥ fn (s) for all s ∈ S, so (fn ) is
increasing everywhere. Then f (s) = lim fn (s) for all s ∈ S, possibly with value

20
infinity. It follows from Proposition 3.9, that µ(fn ) is an increasing sequence,
bounded by µ(f ). Hence we have ` := lim µ(fn ) ≤ µ(f ).
We show that we actually have an equality. Take h ∈ S+ with h ≤ f ,
c ∈ (0, 1) and put En = {fn ≥ ch}. The sequence (En ) is obviously increasing
and we show that its limit is S. Let s ∈ S and suppose that f (s) = 0. Then
also h(s) = 0 and s ∈ En for every n. If f (s) > 0, then eventually fn (s) ≥
cf (s) ≥ ch(s), and so s ∈ En . This shows that ∪n En = S. Consider the chain
of inequalities

` ≥ µ(fn ) ≥ µ(fn 1En ) ≥ cµ(h1En ). (3.5)

P
Suppose that h has representation (3.1). Then µ(h1En ) = i ai µ(Ai ∩En ). This
is a finite sum of nonnegative numbers and hence the limit of it for n → ∞ can
be taken inside the sum and thus equals µ(h), since En ↑ S and the continuity
of the measure (Proposition 1.7). From (3.5) we then conclude ` ≥ cµ(h), for
all c ∈ (0, 1), and thus ` ≥ µ(h). Since this holds for all our h, we get ` ≥ µ(f )
by taking the supremum over h. This proves the first case.
Next we turn to the almost everywhere version. Let Nn = {fn > fn+1 },
by assumption µ(Nn ) = 0. Put N = ∪n Nn , then also µ(N ) = 0. It follows
that µ(fn ) = µ(fn 1N c ). But on N c we have that f = f 1N c and similarly
µ(f ) = µ(f 1N c ). The previous case can be applied to get µ(fn 1N c ) ↑ µ(f 1N c ),
from which the result follows. 

Example 3.13 Here is a nice application of Theorem 3.12. Let f ∈ Σ+ and,

for each n ∈ N, put En,i = {i2−n ≤ f < (i+1)2−n } (i ∈ In := {0, . . . , n2n −1}),
similar to the sets in the proof of Theorem 2.6. Put also En = {f ≥ n}. Note
that the sets En,i and En are in Σ. Define
X
fn = i2−n 1En,i + n1En .
i∈In

These fn form an increasing sequence in Σ+ , even in S+ , with limit f . Theo-

rem 3.12 yields µ(fn ) ↑ µ(f ). We have exhibited a sequence of simple functions
with limit f , that can be used to approximate µ(f ).

Proposition 3.14 Let f, g ∈ Σ+ and α, β > 0. Then µ(αf + βg) = αµ(f ) +

βµ(g) ≤ ∞.

Proof Exercise 3.2. 

We proceed with the next limit result, known as Fatou’s lemma.

Lemma 3.15 Let (fn ) be an arbitrary sequence in Σ+ . Then lim inf µ(fn ) ≥
µ(lim inf fn ). If there exists a function h ∈ Σ+ such that fn ≤ h a.e., and
µ(h) < ∞, then lim sup µ(fn ) ≤ µ(lim sup fn ).

Proof Put gn = inf m≥n fm . We have for all m ≥ n the inequality gn ≤ fm .

Then also µ(gn ) ≤ µ(fm ) for m ≥ n, and even µ(gn ) ≤ inf m≥n µ(fm ). We want

21
to take limits on both side of this inequality. On the right hand side we get
lim inf µ(fn ). The sequence (gn ) is increasing, with limit g = lim inf fn , and by
Theorem 3.12, µ(gn ) ↑ µ(lim inf fn ) on the left hand side. This proves the first
assertion. The second assertion follows by considering f¯n = h − fn ≥ 0. Check
where it is used that µ(h) < ∞. 

Remark 3.16 Let (En ) be a sequence of sets in Σ, and let fn = 1En and h = 1.
The statements of Exercise 1.4 follow from Lemma 3.15.

We now extend the notion of integral to (almost) arbitrary measurable functions.

Let f ∈ Σ. For (extended) real numbers x one defines x+ = max{x, 0} and
x− = max{−x, 0}. Then, for f : S → [−∞, ∞], one defines the functions f +
and f − by f + (s) = f (s)+ and f − (s) = f (s)− . Notice that f = f + − f − and
|f | = f + + f − . If f ∈ Σ, then f + , f − ∈ Σ+ .

Definition 3.17 Let f ∈ Σ and assume that µ(f + ) < ∞ or µ(f − ) < ∞. Then
we define µ(f ) := µ(f + ) − µ(f − ). If both µ(f + ) < ∞ and µ(f − ) < ∞, we
say that f is integrable. The collection of all integrable functions is denoted by
L1 (S, Σ, µ). Note that f ∈ L1 (S, Σ, µ) implies that |f | < ∞ µ-a.e.

Proposition 3.18 The following natural properties hold.

(i) Let f, g ∈ L1 (S, Σ, µ) and α, β ∈ R. Then αf + βg ∈ L1 (S, Σ, µ) and
µ(αf + βg) = αµ(f ) + βµ(g). Hence µ can be seen as a linear operator
on L1 (S, Σ, µ).
(ii) If f, g ∈ L1 (S, Σ, µ) and f ≤ g a.e., then µ(f ) ≤ µ(g).
(iii) Triangle inequality: If f ∈ L1 (S, Σ, µ), then |µ(f )| ≤ µ(|f |).

Proof Exercise 3.3. 

Example 3.19 Let (S, Σ, µ) = ([0, 1], B([0, 1]), λ), where λ is Lebesgue mea-
sure. Assume that f ∈ C[0, 1]. Exercise 3.5 yields that f ∈ L1 ([0, 1], B([0, 1]), λ)
R1
and that λ(f ) is equal to the Riemann integral 0 f (x) dx. This implication fails
to hold if we replace [0, 1] with an unbounded interval, see Exercise 3.6.
On the other hand, one can even show that every function that is Riemann
integrable over [0, 1], not only a continuous function, is Lebesgue integrable too.
More knowledge is required for a precise statement and its proof.

The next theorem is known as the Dominated Convergence Theorem, also called
Lebesgue’s Convergence Theorem.

Theorem 3.20 Let (fn ) ⊂ Σ and f ∈ Σ. Assume that fn (s) → f (s) for all s
outside a set of measure zero. Assume also that there exists a function g ∈ Σ+
such that supn |fn | ≤ g a.e. and that µ(g) < ∞. Then µ(|fn − f |) → 0 (often
L1
denoted fn → f ), and hence µ(fn ) → µ(f ).

22
Proof The second assertion easily follows from the first one, which we prove
now for the case that fn → f everywhere. One has the inequality |f | ≤ g,
whence |fn − f | ≤ 2g. The second assertion of Fatou’s lemma immediately
yields lim sup µ(|fn − f |) ≤ 0, which is what we wanted. The almost everywhere
version is left as Exercise 3.4. 

Many results in integration theory can be proved by what is sometimes called

the standard machine. This ‘machine’ works along the following steps. First
one shows that results hold true for an indicator function, then one extends
this by a linearity argument to nonnegative simple functions. Invoking the
Monotone Convergence Theorem, one can then prove the results for nonnegative
measurable functions. In the final step one shows the result to be true for
functions in L1 (S, Σ, µ) by splitting into positive and negative parts.

3.3 Integrals over subsets

This section is in a sense a prelude to the theorem of Radon-Nikodym, Theo-
rem 5.4. Let f ∈ Σ+ and E ∈ Σ. Then we may define
Z
f dµ := µ(1E f ). (3.6)
E

An alternative approach is to look at the measurable space (E, ΣE ), where

ΣE = {E ∩F : F ∈ Σ} (check that this a σ-algebra on E). Denote the restriction
of µ to ΣE by µE . Then (E, ΣE , µE ) is a measure space. We consider integration
on this space.
Proposition 3.21 Let f ∈ Σ and denote by fE its restriction to E. Then
fE ∈ L1 (E, ΣE , µE ) iff 1E f ∈ L1 (S, Σ, µ), in which case the identity µE (fE ) =
µ(1E f ) holds.

Proof Exercise 3.7. 

Let f ∈ Σ+ . Define for all E ∈ Σ

Z
ν(E) = µ(1E f ) (= f dµ). (3.7)
E

One verifies (Exercise 3.8) that ν is a measure on (S, Σ). We want to compute
ν(h) for h ∈ Σ+ . For measurable indicator functions we have by definition
that the integral ν(1E ) equals ν(E), which is equal to µ(1E f ) by (3.7). More
generally we have

Proposition 3.22 Let f ∈ Σ+ and h ∈ Σ. Then h ∈ L1 (S, Σ, ν) iff hf ∈

L1 (S, Σ, µ), in which case one has ν(h) = µ(hf ).

Proof Exercise 3.9. 

23
 R R
For the measure ν above, Proposition 3.22 states that h dν = hf dµ, valid
dν
for all h ∈ L1 (S, Σ, ν). The notation f = dµ is often used and looks like
a derivative. We will return to thisR in Chapter
R 5, where we discuss Radon-
Nikodym derivatives. The equality h dν = hf dµ now takes the appealing
form Z Z
dν
h dν = h dµ.
dµ

RExample 3.23 Let (S, Σ, µ) = (R, B, λ), f ≥ 0, Borel measurable, ν(E) =

1E f dλ and hf ∈ L1 (R, B, λ). Then
Z ∞
ν(h) = h(x)f (x) dx,
−∞

where the equality is valid under conditions as for instance in Example 3.19.
R
Remark 3.24 If f is continuous, see Example 3.19, then x 7→ F (x) = [0,x] f dλ
defines a differentiable function on (0, 1), with F 0 (x) = f (x). This follows
from theR theory of Riemann integrals. We adopt the conventional notation
x
F (x) = 0 f (u) du. This case can be generalized as follows. If f ∈ L1 (R, B, λ),
Rx
then (using a similar notational convention) x 7→ F (x) = −∞ f (u) du is well
defined for all x ∈ R. Moreover, F is at (Lebesgue) almost all points x of
R differentiable with derivative F 0 (x) = f (x). The proof of this result, the
fundamental theorem of calculus for the Lebesgue integral, is not given here.

3.4 Expectation and integral

The whole point of this section is that the expectation of a random variable is a
Lebesgue integral. Indeed, consider a probability space (Ω, F, P), and let X be
a (real) random variable defined on it. Recall that X : Ω → R is by definition
a measurable function. Making the switch between the notations (S, Σ, µ) and
(Ω, F, P), one has the following notation for the integral of X w.r.t. P
Z
P(X) = X dP, (3.8)
Ω

provided that the integral is well defined, which is certainly the case if X is
nonnegative or if P(|X|) < ∞. Other often used notations for the integral in
(3.8) are PX and E X. The latter is the favorite one among probabilists and
one speaks of the Expectation of X. Note also that E X is always defined when
X ≥ 0 almost surely. The latter concept meaning almost everywhere w.r.t. the
probability measure P. We abbreviate almost surely by a.s.
Example 3.25 Let (Ω,PF, P) = (N, 2N , P), where P is defined by P({n}) = pn ,
where all pn ≥ 0 and pn = 1. Let (xn ) be a sequence of nonnegative real
numbers and define the random variable X, a function on Ω = N, by X(n) = xn .
In a spirit
P∞similar to what we have seen in Examples 3.5 and 3.10, we get
EX = n=1 xn pn . Let us switch to a different approach. Let ξ1 , ξ2 , . . . be

24
the different elements of the set {x1 , x2 , . . .} and put Ei = {j : xj = ξi },
i ∈ N. Notice
P that {X = ξi } = Ei and that
P the Ei form a partition
P of N with
P(Ei ) = j∈Ei pj . It follows that E X = i ξi P(Ei ), or E X = i ξi P(X = ξi ),
the familiar expression for the expectation.
If h : R → R is Borel measurable, then Y := h ◦ X (we also write Y = h(X))
is a random variable as well. There are two recipes to compute E Y . One is of
course the direct application of the definition of expectation to Y . But we also
have
Proposition 3.26 Let X be a random variable, and h : R → R Borel mea-
surable. Let PX be the distribution of X. Then h ◦ X ∈ L1 (Ω, F, P) iff
h ∈ L1 (R, B, PX ), in which case
Z
E h(X) = h dPX . (3.9)
R

Proof Exercise 3.10. 

It
R follows from this proposition that one can also compute E Y as E Y =
Y X
R
R
y P ( dy), and of course E X = R
x P ( dx).
Example 3.27 Suppose there exists f ≥ 0, Borel-measurable such that for all
B ∈ B one has PX (B) = λ(1B f ), in which case it is said that X has a density
f . Then, provided that the expectation is well defined, Example 3.23 yields
Z
E h(X) = h(x)f (x) dx,
R

another familiar formula for the expectation of h(X).

We conclude that the definition of expectation as a Lebesgue integral w.r.t. a
probability measure as in (3.8) yields the familiar formulas, sums for discrete
random variables and Riemann integrals for random variables having an ordi-
nary density function, as special cases. So, we see that the Lebesgue integral
serves as a unifying concept for expectation. At least as important is that we can
use the powerful convergence theorems (obtained for integrals) of Section 3.2
for expectations as well. Notice that every real constant (function) has a well
defined, and trivially finite, expectation. Therefore one can in pertaining cases
apply the Dominated Convergence Theorem (Theorem 3.20) with the function g
equal to a constant. Here is a simple example of the application of the Monotone
Convergence Theorem.

Example 3.28 Let (Xn ) be a sequence

P of nonnegative random variables, so all
E Xn ≤ ∞ are well defined. ThenP Xn isP a well defined random P variable as
well, nonnegative,
P and we have E ( X n ) = E Xn . Moreover if E Xn < ∞,
then Xn < ∞ a.s. Verification of these assertions is straightforward and left
as Exercise 3.11.

25
The next two propositions have proven to be very useful in proofs of results in
Probability Theory.

Proposition 3.29 Let X be a real valued random variable and g : R → [0, ∞]

an increasing function. Then E g(X) ≥ g(c)P(X ≥ c).

Proof This follows from the inequality g(X)1{X≥c} ≥ g(c)1{X≥c} . 

The inequality in Proposition 3.29 is known as Markov’s inequality. An example

is obtained by taking g(x) = x+ and by replacing X with |X|. One gets E |X| ≥
cP(|X| ≥ c). For the special case where g(x) = (x+ )2 , it is known as Chebychev’s
inequality. This name is especially used, if we apply it with |X − E X| instead
of X. For c ≥ 0 we then obtain Var X ≥ c2 P(|X − E X| ≥ c).
We now turn to a result that is known as Jensen’s inequality, Proposition 3.30
below. Recall that a function g : G → R is convex, if G is a convex set and if
for all x, y ∈ G and α ∈ [0, 1] one has

g(αx + (1 − α)y) ≤ αg(x) + (1 − α)g(y).

We consider only the case where G is an interval.

Let us first give a property of a convex function. For all x in the interior of
G there is a number d(x) such that for all z ∈ G it holds that

g(z) − g(x) ≥ d(x)(z − x). (3.10)

Verify this property graphically. The d(x) are also called subgradients of g. The
following proposition (Jensen’s inequality) is now easy to prove, and you check
where we use in the proof the fact that P is a probability measure.
Proposition 3.30 Let g : G → R be convex and X a random variable with
P(X ∈ G) = 1. Assume that E |X| < ∞ and E |g(X)| < ∞. Then

E g(X) ≥ g(E X).

Proof We exclude the trivial case P(X = x0 ) = 1 for some x0 ∈ G. Since

P(X ∈ G) = 1, we have E X ∈ Int G (Exercise 3.15) and (3.10) with x = E X
and z replaced with X holds a.s. So, in view of (3.10),

g(X) − g(E X) ≥ d(E X)(X − E X).

Take expectations to get E g(X) − g(E X) ≥ 0. 

3.5 Lp -spaces of random variables

In this section we introduce the p-norms and the spaces of random variables
with finite p-norm. We start with a definition.

Definition 3.31 Let 1 ≤ p < ∞ and X a random variable on (Ω, F, P). If

E |X|p < ∞, we write X ∈ Lp (Ω, F, P) and ||X||p = (E |X|p )1/p .

26
The notation || · || suggests that we deal with a norm. In a sense, this is correct,
but we will not explain this until the end of this section. It is however obvious
that Lp := Lp (Ω, F, P) is a vector space, since |X + Y |p ≤ (|X| + |Y |)p ≤
2p (|X|p + |Y |p ).
In the special case p = 2, we have for X, Y ∈ L2 , that |XY | = 12 ((|X|+|Y |)2 −
X −Y 2 ) has finite expectation and is thus in L1 . Of course we have |E (XY )| ≤
2

E |XY |. For the latter we have the famous Cauchy-Schwarz inequality.

Proposition 3.32 Let X, Y ∈ L2 . Then E |XY | ≤ ||X||2 ||Y ||2 .

Proof If E Y 2 = 0, then Y = 0 a.s. (Lemma 3.11), so also XY = 0 a.s. and

there is nothing to prove. Assume then that E Y 2 > 0 and let c = E |XY |/E Y 2 .
One trivially has E (|X| − c|Y |)2 ≥ 0. But, by the choice of c, the left hand side
|)2
equals E X 2 − (E E|XY
Y2 . 

Proposition 3.32 tells us that X, Y ∈ L2 (Ω, F, P) is sufficient to guarantee that

the product XY is integrable. For independent X and Y weaker integrability
assumptions suffice and the product rule for probabilities of intersections extends
to a product rule for expectations.

Proposition 3.33 Let X, Y ∈ L1 (Ω, F, P) be independent random variables.

Then XY ∈ L1 (Ω, F, P) and E (XY ) = E X · E Y .

Proof The standard machine easily gives E (1A Y ) = P(A) · E Y for A an event
independent of Y . Assume that X ∈ S+ . Since X is integrable we can assume
that it is finite, and thus bounded by a constantP c. Since then |XY | ≤ c|Y |,
n
we
Pn obtain E |XY | < ∞. If we represent X as i=1 ai 1Ai , then E (XY ) =
i=1 ai P(Ai )E Y readily follows and thus E (XY ) = E X · E Y . The proof may
be finished by letting the standard machine operate on X. 

We continue with some properties of Lp -spaces. First we have monotonicity of

norms.

Proposition 3.34 Let 1 ≤ p ≤ r and X ∈ Lr (Ω, F, P), then X ∈ Lp (Ω, F, P)

and ||X||p ≤ ||X||r .

Proof It follows from the trivial inequality |u| ≤ 1+|u|a , valid for u ∈ R and a ≥
1, that |X|p ≤ 1+|X|r , by taking a = r/p, and hence X ∈ Lp (Ω, F, P). Observe
that x → |x|a is convex. We apply Jensen’s inequality to get (E |X|p )a ≤
E (|X|pa ), from which the result follows. 

3.6 Lp -spaces of functions

In the previous section we have introduced the Lp -spaces for random variables
defined on a probability space (Ω, F, P). In the present section, we consider
in some more generality the spaces Lp (S, Σ, µ). For completeness, we give the
definition, which is of course completely analogous to Definition 3.31.

27
Definition 3.35 Let 1 ≤ p < ∞ and f a measurable function on (S, Σ, µ). If
µ(|f |p ) < ∞, we write f ∈ Lp (S, Σ, µ) and ||f ||p = (µ(|f |p ))1/p .

Occasionally, it is useful to work with ||f ||p for p = ∞. It is defined as follows.

For f ∈ Σ we put

||f ||∞ := inf{m ∈ R : µ({|f | > m}) = 0},

with the convention inf ∅ = ∞. It is clear that |f | ≤ ||f ||∞ a.e. We write
f ∈ L∞ (S, Σ, µ) if ||f ||∞ < ∞.
Here is the first of two fundamental inequalities, known as Hölder’s inequality.

Theorem 3.36 Let p, q ∈ [1, ∞], f ∈ Lp (S, Σ, µ) and g ∈ Lq (S, Σ, µ). If

1 1 1
p + q = 1, then f g ∈ L (S, Σ, µ) and ||f g||1 ≤ ||f ||p ||g||q .

Proof Notice first that for p = 1 or p = ∞ there is basically nothing to prove.

So we assume p, q ∈ (1, ∞). We give a probabilistic proof by introducing a
conveniently chosen probability measure and by using Jensen’s inequality. We
assume without loss of generality that f, g ≥ 0 a.e. If ||f ||p = 0, then f = 0 a.e.
in view of Lemma 3.11 and we have a trivial inequality. Let then 0 < ||f ||p < ∞.
We now define a probability measure P on Σ by
µ(1E f p )
P(E) = .
µ(f p )

Put h(s) = g(s)/f (s)p−1 if f (s) > 0 and h(s) = 0 otherwise. Jensen’s inequality
gives (P(h))q ≤ P(hq ). We compute

µ(f g)
P(h) = ,
µ(f p )
and
µ(1{f >0} g q ) µ(g q )
P(hq ) = ≤ .
µ(f p ) µ(f p )
Insertion of these expressions into the above version of Jensen’s inequality yields
(µ(f g))q µ(g q )
≤ ,
(µ(f p ))q µ(f p )

whence (µ(f g))q ≤ µ(g q )µ(f p )q−1 . Take q-th roots on both sides and the result
follows. 

Remark 3.37 For p = 2 Theorem 3.36 yields the Cauchy-Schwarz inequal-

ity ||f g||1 ≤ ||f ||2 ||g||2 for square integrable functions. Compare to Proposi-
tion 3.32.

We now give the second fundamental inequality, Minkowski’s inequality.

28
Theorem 3.38 Let f, g ∈ Lp (S, Σ, µ) and p ∈ [1, ∞]. Then ||f + g||p ≤ ||f ||p +
||g||p .
Proof The case p = ∞ is almost trivial, so we assume p ∈ [1, ∞). To exclude
another triviality, we suppose ||f + g||p > 0. Note the following elementary
relations.
|f + g|p = |f + g|p−1 |f + g| ≤ |f + g|p−1 |f | + |f + g|p−1 |g|.
Now we take integrals and apply Hölder’s inequality to obtain
µ(|f + g|p ) ≤ µ(|f + g|p−1 |f |) + µ(|f + g|p−1 |g|)
≤ (||f ||p + ||g||p )(µ(|f + g|(p−1)q )1/q
= (||f ||p + ||g||p )(µ(|f + g|p )1/q ,

because (p − 1)q = p. After dividing by (µ(|f + g|p )1/q , we obtain the result,
because 1 − 1/q = 1/p. 
Recall the definition of a norm on a (real) vector space X. One should have
||x|| = 0 iff x = 0, ||αx|| = |α| ||x|| for α ∈ R (homogeneity) and ||x + y|| ≤
||x|| + ||y|| (triangle inequality). For || · ||p homogeneity is obvious, the triangle
inequality has just been proved under the name Minkowski’s inequality and
we also trivially have f = 0 ⇒ ||f ||p = 0. But, conversely ||f ||p = 0 only
implies f = 0 a.e. This annoying fact disturbs || · ||p being called a genuine
norm. This problem can be circumvented by identifying a function f that is
zero a.e. with the zero function. The proper mathematical way of doing this is
by defining the equivalence relation f ∼ g iff µ({f 6= g}) = 0. By considering
the equivalence classes induced by this equivalence relation one gets the quotient
space Lp (S, Σ, µ) := Lp (S, Σ, µ)/ ∼. One can show that || · ||p induces a norm
on this space in the obvious way. We don’t care too much about these details
and just call || · ||p a norm and Lp (S, Σ, µ) a normed space, thereby violating a
bit the standard mathematical language.
A desirable property of a normed space, (a version of) completeness, holds for
Lp spaces. We give this result for Lp (Ω, F, P).
Theorem 3.39 Let p ∈ [1, ∞]. The space Lp (Ω, F, P) is complete in the fol-
lowing sense. Let (Xn ) be a Cauchy-sequence in Lp : ||Xn − Xm ||p → 0 for
n, m → ∞. Then there exists a limit X ∈ Lp such that ||Xn − X||p → 0. The
limit is unique in the sense that any other limit X 0 satisfies ||X − X 0 ||p = 0.
Proof Omitted. 
Remark 3.40 Notice that it follows from Theorem 3.39, and the discussion
preceding it, that Lp (Ω, F, P) is a truly complete normed space, called a Ba-
nach space. The same is true for Lp (S, Σ, µ) (p ∈ [1, ∞]), for which you need
Exercise 3.12. For theR special case p = 2 we endow L2 (S, Σ, µ) with the in-
ner product hf, gi := f g dµ, and it is then called a Hilbert space. Likewise
L2 (Ω, F, P) is a Hilbert space with inner product hX, Y i := E XY .

29
3.7 Exercises
3.1 Let (x1 , x2 , . . .) be a sequence of nonnegative real numbers, let ` : N → N
be a bijection and define the sequence (y1 , y2 , . . .) by yk = x`(k) . Let for each
n the n-vector y n be given by y n = (y1 , . . . , yn ). Consider then for each n
a sequence of numbers xn defined by xnk = xk if xk is a coordinate of y n .
n n
Otherwise
P∞ P∞xk = 0. Show that xk ↑ xk for every k as n → ∞. Show that
put
k=1 yk = k=1 xk .

3.2 Prove Proposition 3.14.

3.3 Prove Proposition 3.18 (assume Proposition 3.14). Show also that |µ(f )| ≤
µ(|f |), if f ∈ L1 (S, Σ, µ).
3.4 Prove the ‘almost everywhere’ version of Theorem 3.20 by using the ‘every-
where’ version.
3.5 In this exercise λ denotes Lebesgue measure on the Borel sets of [0, 1]. Let
R1
f : [0, 1] → R be continuous. Then the Riemann integral I := 0 f (x) dx exists
(this is standard Analysis). But also the Lebesgue integral of f exists. (Explain
why.). Construct (use the definition of the Riemann integral) an increasing
sequence of simple functions hn with limit f satisfying hn ≤ f and λ(hn ) ↑ I.
Prove that λ(f ) = I.
3.6 Let f :R[0, ∞) → R be given by f (x) = sinx x for x > 0 and f (0) = 1. Show
∞
that I := 0 f (x) dx exists as an improper Riemann integral (i.e. the limit
RT
limT →∞ 0 f (x) dx exists and is finite), but that f ∈ / L1 ([0, ∞), B([0, ∞)), λ).
π
In Exercise 4.9 you compute that I = 2 .
3.7 Prove Proposition 3.21 by means of the standard machinery.
3.8 Verify that ν defined in (3.7) is a measure.

3.9 Prove Proposition 3.22. Hint: The standard machine works.

3.10 Prove Proposition 3.26. Hint: Use the standard machinery for h.
3.11 Give the details for Example 3.28.
3.12 Give the proof of Theorem 3.39 for an arbitrary measure space Lp (S, Σ, µ)
and p ∈ [0, ∞) (it requires minor modifications). Give also the proof of com-
pleteness of L∞ (S, Σ, µ).
3.13 This exercise concerns a more general version of Theorem 3.20. Let (fn ) ⊂
Σ and f = lim sup fn and assume that fn (s) → f (s) for all s outside a set of
measure zero. Assume also there exist functions g, gn ∈ Σ+ such that |fn | ≤ gn
a.e. with gn (s) → g(s) for all s outside a set of measure zero and that µ(gn ) →
µ(g) < ∞. Show that µ(|fn − f |) → 0.

30
3.14 Let (S, Σ, µ) be a measurable space, Σ0 a sub-σ-algebra of Σ and µ0 be the
restriction of µ to Σ0 . Then also (S, Σ0 , µ0 ) is a measurable space and integrals of
Σ0 -measurable functions can be defined according to the usual procedure. Show
that µ0 (f ) = µ(f ), if f ≥ 0 and Σ0 -measurable. Show also that L1 (S, Σ, µ)∩Σ0 =
L1 (S, Σ0 , µ0 ).

3.15 Let G be an interval, X a random variable. Assume P(X ∈ G) = 1 and

E |X| < ∞. If X is not degenerate (P(X = x) < 1 for all x ∈ G), show that
E X ∈ Int G.
3.16 Let f ∈ L∞ (S, Σ, µ) and suppose that µ({f 6= 0}) < ∞. We will see that
limp→∞ kf kp = kf k∞ .
(a) Show that lim supp→∞ kf kp ≤ kf k∞ .
(b) Show that lim inf p→∞ kf kp ≥ kf k∞ . Hint: for ε > 0 it holds that µ({f >
kf k∞ − ε}) > 0.
(c) Show also that kf kp converges monotonically to kf k∞ if µ is a probability
measure.

31
4 Product measures
So far we have consideredR measure spaces (S, Σ, µ) and we have looked at inte-
grals of the type µ(f ) = f dµ. Here f is a function of ‘one’ variable (depends
on how you count and what the underlying set S is). Suppose that we have two
measure spaces (S1 , Σ1 , µ1 ) and (S2 , Σ2 , µ2 ) and a function f : S1 × S2 → R.
Is it possible to integrate such a function of two variables w.r.t. some measure,
that has to be defined on some Σ-algebra of S1 × S2 . There is a natural way
of constructing this σ-algebra and a natural construction of a measure on this
σ-algebra. Here is a setup with some informal thoughts.
Take f : S1 × S2 → R and assume R any good notion of measurability and
integrability. Then µ(f (·, s2 )) := f (·, s2 ) dµ1 defines a function of s2 and so
we’d like to take the integral w.r.t. µ2 . We could as well have gone the other way
round (integrate first w.r.t. µ2 ), and the questions are whether these integrals
are well defined and whether both approaches yield the same result.
Here is a simple special case, where the latter question has a negative an-
swer. We have seen that integration w.r.t. counting measure is nothing else but
addition. What we have outlined above is in this context just interchanging the
summation order. P So P if (an,m ) P
is a P
double array of real numbers, the above is
about whether n m an,m = m n an,m . This is obviously true if n and m
run through a finite set, but things can go wrong for indices from infinite sets.
Consider for example

 1 if n = m + 1
an,m = −1 if m = n + 1
0 else.


P P
One
P P easily verifies m a1,m = −1, m an,m = P n ≥ 2 and hence we find
0, if P
n a
m n,m = −1. Similarly one shows that m n an,m = +1. In order
that interchanging of the summation order yields the P same
P result, additional
conditions have to be imposed. We will see that m n |an,m | < ∞ is a
sufficient condition. As a side remark we note that this case has everything
to do with a well known theorem by Riemann that says that a series of real
numbers is absolutely convergent iff it is unconditionally convergent.

4.1 Product of two measure spaces

Our aim is to construct a measure space (S, Σ, µ) with S = S1 × S2 . First
we construct Σ. It is natural that ‘measurable rectangles’ are in Σ. Let R =
{E1 × E2 : E1 ∈ Σ1 , E2 ∈ Σ2 }. Obviously R is a π-system, but in general not a
σ-algebra on S. Therefore we define Σ := σ(R), the product σ-algebra of Σ1 and
Σ2 . A common notation for this product σ-algebra, also used below for similar
cases, is Σ = Σ1 × Σ2 .
Alternatively, one can consider the projections πi : S → Si , defined by
πi (s1 , s2 ) = si . It is easy to show that Σ coincides with the smallest σ-algebra
that makes these projections measurable.

32
 Next to the projections, we now consider embeddings. For fixed s1 ∈ S1 we
define es1 : S2 → S by es1 (s2 ) = (s1 , s2 ). Similarly we define es2 (s1 ) = (s1 , s2 ).
One easily checks that the embeddings es1 are Σ2 /Σ-measurable and that the
es2 are Σ1 /Σ-measurable (Exercise 4.1). As a consequence we have the following
proposition.

Proposition 4.1 Let f : S → R be Σ-measurable. Then the marginal map-

pings s1 7→ f (s1 , s2 ) and s2 7→ f (s1 , s2 ) are Σ1 -, respectively Σ2 -measurable,
for any s2 ∈ S2 , respectively s1 ∈ S1 .

Proof This follows from the fact that a composition of measurable functions is
also measurable. 

Remark 4.2 The converse statement of Proposition 4.1 is in general not true,
but a counterexample is beyond the scope of these lecture notes; see the full
version for details. There are functions f : S → R that are not measurable
w.r.t. the product σ-algebra Σ, although the mappings s1 7→ f (s1 , s2 ) and s2 7→
f (s1 , s2 ) are Σ1 -, respectively Σ2 -measurable. Counterexamples are not obvious,
see below for a specific one. Fortunately, there are also conditions that are
sufficient to have measurability of f w.r.t. Σ, when measurability of the marginal
functions is given. See Exercise 4.8.

Having constructed the product σ-algebra Σ, we now draw our attention to

the construction of the product measure µ on Σ, denoted by µ1 × µ2 . We will
construct µ such that the property µ(E1 × E2 ) = µ1 (E1 )µ2 (E2 ) holds. This
justifies the name product measure.
Until later notice we assume that the measures µ1 and µ2 are finite.
Consider a bounded Σ-measurable function f . We know that the mappings
si 7→ f (s1 , s2 ) are Σi -measurable and therefore the integrals w.r.t. µi are well
defined (why?). Let then
Z
f
I1 (s1 ) = f (s1 , s2 )µ2 (ds2 )
Z
I2f (s2 ) = f (s1 , s2 )µ1 (ds1 ).

Lemma 4.3 Let f be a bounded Σ-measurable function. Then the mappings

Iif : Si → R are Σi -measurable (i = 1, 2). Moreover we have the identity

µ1 (I1f ) = µ2 (I2f ), (4.1)

or, in a more appealing notation,

Z Z Z Z



f (s1 , s2 )µ2 (ds2 ) µ1 (ds1 ) = f (s1 , s2 )µ1 (ds1 ) µ2 (ds2 ). (4.2)
S1 S2 S2 S1

33
Proof We use the Monotone Class Theorem, Theorem 2.6, and so we have
to find a good vector space H. The obvious candidate is the collection of all
bounded Σ-measurable functions f that satisfy the assertions of the lemma.
First we notice that H is indeed a vector space, since sums of measurable
functions are measurable and by linearity of the integral. Obviously, the con-
stant functions belong to H. Then we have to show that if fn ∈ H, fn ≥ 0 and
fn ↑ f , where f is bounded, then also f ∈ H. Of course here the Monotone
Convergence Theorem comes into play. First we notice that measurability of
the Iif follows from measurability of the Iifn for all n. Theorem 3.12 yields
that the sequences Iifn (si ) are increasing and converging to Iif (si ). Another
application of this theorem yields that µ1 (I1fn ) converges to µ1 (I1f ) and that
µ2 (I2fn ) converges to µ2 (I2f ). Since µ1 (I1fn ) = µ2 (I2fn ) for all n, we conclude
that µ1 (I1f ) = µ2 (I2f ), whence f ∈ H.
Next we check that H contains the indicators of sets in R. A quick com-
putation shows that for f = 1E1 ×E2 one has I1f = 1E1 µ2 (E2 ), which is Σ1 -
measurable, I2f = 1E2 µ1 (E1 ), and µ1 (I1f ) = µ2 (I2f ) = µ1 (E1 )µ2 (E2 ). Hence
f ∈ H. By Theorem 2.6 we conclude that H coincides with the space of all
bounded Σ-measurable functions. 

It follows from Lemma 4.3 that for all E ∈ Σ, the indicator function 1E sat-
isfies the assertions of the lemma. This shows that the following definition is
meaningful.

Definition 4.4 We define µ : Σ → [0, ∞) by µ(E) = µ2 (I21E ) for E ∈ Σ.

In Theorem 4.5 below (known as Fubini’s theorem) we assert that this defines
a measure and it also tells us how to compute integrals w.r.t. this measure in
terms of iterated integrals w.r.t. µ1 and µ2 .

Theorem 4.5 The mapping µ of Definition 4.4 has the following properties.
(i) It is a measure on (S, Σ). Moreover, it is the only measure on (S, Σ) with
the property that µ(E1 × E2 ) = µ1 (E1 )µ1 (E2 ). It is therefore called the
product measure of µ1 and µ2 and often written as µ1 × µ2 .
(ii) If f ∈ Σ+ , then

µ(f ) = µ2 (I2f ) = µ1 (I1f ) ≤ ∞. (4.3)

(iii) If f ∈ L1 (S, Σ, µ), then Equation (4.3) is still valid and µ(f ) ∈ R.

Proof (i) It is obvious that µ(∅)P = 0. If (En ) is a disjoint sequence in Σ with

n
union E, then we have 1E = limn i=1 1Ei . Linearity of the integral and Mono-
tone Convergence (applied two times) show that µ is σ-additive. Uniqueness of
µ follows from Theorem 1.15 applied to the π-system R.
(ii) We use the standard machine. The two equalities in (4.3) are by defini-
tion of µ valid for f = 1E , when E ∈ Σ. Linearity of the integrals involved show

34
that it is true for nonnegative simple functions f and Monotone Convergence
yields the assertion for f ∈ Σ+ .
(iii) Of course, here we have to use the decomposition f = f + − f − . The
tricky details are left as Exercise 4.2. 
Theorem 4.5 has been proved under the standing assumption that the initial
measures µ1 and µ2 are finite. The results extend to the case where both these
measures are σ-finite. The approach is as follows. Write S1 = ∪∞ i
i=1 S1 with the
S1 ∈ Σ1 and µ1 (S1 ) < ∞. Without loss of generality, we can take the S1i disjoint.
i i

Take a similar partition (S2j ) of S2 . Then S = ∪i,j Sij , where the Sij := S1i × S2j ,
form a countable disjoint union as well. Let Σij = {E ∩ Sij : E ∈ Σ}. On each
measurable space (Sij , Σij ) the above results apply and one has e.g. identity of
the involved integrals by splitting the integration over the sets Sij and adding
up the results.
We note that if one goes beyond σ-finite measures (often a good thing to
do if one wants to have counterexamples), the assertion may no longer be true.
Let S1 = S2 = [0, 1] and Σ1 = Σ2 = B[0, 1]. Take µ1 equal to Lebesgue measure
and µ2 the counting measure, the latter is not σ-finite. It is a nice exercise to
show that ∆ := {(x, y) ∈ S : x = y} ∈ Σ. Let f = 1∆ . Obviously I1f (s1 ) ≡ 1
and I2f (s2 ) ≡ 0 and the two iterated integrals in (4.3) are 1 and 0. So, more or
less everything above concerning product measures fails in this example.
We close the section with a few remarks on products with more than two factors.
The construction of a product measure space carries over, without any problem,
to products of more than two factors, as long as there are finitely many. This
results in product spaces of the form (S1 × . . . × Sn , Σ1 × . . . × Σn , µ1 × . . . × µn )
under conditions similar to those of Theorem 4.5. The product σ-algebra is
again defined as the smallest σ-algebra that makes all projections measurable.
Existence of product measures is proved in just the same way as before, using an
induction argument. Note that there will be many possibilities to extend (4.1)
and (4.2), since there are n! different integration orders. We leave the details to
the reader.

4.2 Application in Probability theory

In this section we consider real valued random variables, as well as real random
vectors. The latter require a definition. Consider a probability space (Ω, F, P)
and a map X : Ω → E, where E is some other set. Let E be a σ-algebra on
E. If the map X is F/E measurable, X is also called a random element of E.
If E is a vector space, we call X in such a case a random vector. Notice that
this definition depends on the σ-algebras at hand, which we don’t immediately
recognize in the term random vector.
An obvious example of a vector space is R2 . Suppose we have two random
variables X1 , X2 : Ω → R. We can consider the map X = (X1 , X2 ) : Ω → R2 ,
defined by X(ω) = (X1 (ω), X2 (ω)) and it is natural to call X a random vector.
To justify this terminology, we need a σ-algebra on R2 and there are two obvious
candidates, the Borel σ-algebra B(R2 ) generated by the ordinary open sets (as in

35
Section 1.1), and, continuing our discussion of the previous section, the product
σ-algebra B(R) × B(R).

Proposition 4.6 It holds that B(R2 ) = B(R) × B(R).

Proof The projections πi : R2 → R are continuous and thus B(R2 )-measurable.

Since B(R) × B(R) is the smallest σ-algebra for which the projections are mea-
surable, we have B(R) × B(R) ⊂ B(R2 ). Conversely, if G is open in R2 , it is
the countable union of (open) rectangles in R (similar to the proof of Proposi-
tion 1.3) and hence G ∈ B(R) × B(R), which yields the other inclusion. 

Remark 4.7 Observe that the proof of B(R) × B(R) ⊂ B(R2 ) generalizes to
the situation, where one deals with two topological spaces with the Borel sets.
For the proof of the other inclusion, we used (and needed) the fact that R is
separable under the ordinary topology. In a general setting one might have the
strict inclusion of the product σ-algebra in the Borel σ-algebra on the product
space (with the product topology).

We now know that there is no difference between B(R2 ) and B(R) × B(R). This
facilitates the use of the term 2-dimensional random vector and we have the
following easy to prove corollary.
Corollary 4.8 Let X1 , X2 : Ω → R be given. The vector mapping X =
(X1 , X2 ) : Ω → R2 is a random vector iff the Xi are random variables.

Proof Exercise 4.3. 

Remark 4.9 Let X1 , X2 be two random variables. We already knew that X1 +

X2 is a random variable too. This also follows from the present results. Let
f : R2 → R be a continuous function. Then it is also B(R2 )-measurable, and by
Corollary 4.8 and composition of measurable functions, f (X1 , X2 ) is a random
variable as well. Apply this with f (x1 , x2 ) = x1 + x2 .

Recall that we defined in Section 2.2 the distribution, or the law, of a random
variable. Next to random variables, one can also look at random vectors. These
are mappings X : Ω → Rn of which the components, written as Xi for i =
1, . . . , n, are random variables. Also random vectors have distributions, now
probability measures on the Borel sets of Rn . So, for the Borel sets B in Rn ,
B ∈ B(Rn ), we define PX (B) = P(X ∈ B) and as in the one-dimensional case,
PX is a probability measure on B(Rn ), the joint distribution of X, also called
(joint) law of X. The marginal distribution of e.g. X 1 (for the other components
something similar applies) is given by PX1 (E) = PX (E ×Rn−1 ), with E ∈ B(R).
Random vector have distribution functions as well, also called joint dis-
tribution functions. The latter are functions F : Rn → [0, 1], defined by
F (x1 , . . . , xn ) = P({X1 ≤ x1 } ∩ · · · {Xn ≤ xn }) for all (x1 , . . . , xn ) ∈ Rn .
We also use the notation FX for such a distribution function. Note that one
has (in obvious notation) FX1 (x1 ) = P({X1 ≤ x1 } ∩ Rn−1 ), which gives the

36
(marginal ) distribution function of X1 . For the other components one similarly
has the (marginal) distribution functions FXi .
Let us specialize to n = 2. Then we have the relations PX (B1 ×R) = P(X1 ∈
B1 ) = PX1 (B1 ) for the marginal distribution, or marginal law, of X1 . The joint
distribution function F = FX : R2 → [0, 1] can be written as

F (x1 , x2 ) = PX ((−∞, x1 ] × (−∞, x2 ]) = P(X1 ≤ x1 , X2 ≤ x2 ).

Notice that, for instance, FX1 (x1 ) = limx2 →∞ F (x1 , x2 ), also denoted F (x1 , ∞).
An important case happens Rif there exists a nonnegative B(R2 )-measurable
function f such that PX (E) = E f d(λ × λ), for all E ∈ B(R2 ). In that case,
f is called the (joint) density
R of X. The obvious marginal density fX1 of X1
is defined by fX1 (x1 ) = f (x1 , x2 ) λ( dx2 ). One similarly defines the marginal
density of X2 . Check these are indeed densities in the sense of Example 3.27.
Independence (of random variables) had to do with multiplication of probabili-
ties (see Definition 2.11), so it should in a natural way be connected to product
measures.

Proposition 4.10 Two random variables X1 , X2 on (Ω, F, P) are independent

iff the joint distribution P(X1 ,X2 ) is the product measure PX1 × PX2 . This in
turn happens iff F (x1 , x2 ) = FX1 (x1 )FX2 (x2 ), for all x1 , x2 ∈ R. Assume further
that (X1 , X2 ) has a joint probability density function f . Let f1 and f2 be the
(marginal) probability density functions of X1 and X2 respectively. Then X1
and X2 are independent iff f (x1 , x2 ) = f1 (x1 )f2 (x2 ) for all (x1 , x2 ) except in a
set of λ × λ-measure zero.

Proof Exercise 4.4. 

The results of the present section (Proposition 4.6, Corollary 4.8, Proposi-
tion 4.10) have obvious extensions to higher dimensional situations. We leave
the formulation to the reader.

Remark 4.11 Suppose one is given a random variable X, defined on a given

(Ω, F, P). Sometimes one needs an additional random variable Y having a
specified distribution. It may happen that the given probability space is not
rich enough to have such a random variable well defined. Suppose Ω = {0, 1}
and X(ω) = ω, having a Bernoulli distribution for P defined on the power set
of Ω with P({1}) = p. Clearly, it is impossible to define on this Ω a random
variable having more than two different outcomes. Extending the probability
space to a suitable product space offers a way out, see Exercise 4.13, from which
it even follows that X and Y are independent.

4.3 Exercises
4.1 Show that the embeddings es1 are Σ2 /Σ-measurable and that the es2 are
Σ1 /Σ-measurable. Also prove Proposition 4.1.

37
4.2 Prove part (iii) of Fubini’s theorem (Theorem 4.5) for f ∈ L1 (S, Σ, µ) (you
already know it for f ∈ Σ+ ). Explain why s1 7→ f (s1 , s2 ) is in L1 (S1 , Σ1 , µ1 )
for all s2 outside a set N of µ2 -measure zero and that I2f is well defined on N c .

4.3 Prove Corollary 4.8.

4.4 Prove Proposition 4.10.
4.5 A two-dimensional random vector (X, Y ) is said to have a density f w.r.t.
the Lebesgue measure on B(R)2 is for every set B ∈ B(R2 ) one has
Z Z
P((X, Y ) ∈ B) = f (x, y) dx dy.
B

Define
Z
fX (x) = f (x, y) dy.
R

Show that for all B ∈ B(R) one has

Z
X
P (B) = fX (x) dx.
B

4.6 Let X and Y be independent random variables on some probability space

(Ω, F, P). Let FX and FY be their distribution functions and µX and µY their
laws. Put Z = X + Y and FZ its distribution function.
R
(a) Show that FZ (z) = R FX (z − y) µY (dy).
(b) Assume that FX admits a density fX (w.r.t. Lebesgue measure). Show
that also FZ admits a density, which can be taken to be
Z
fZ (z) := fX (z − y) µY (dy).
R

4.7 If Z1 , Z2 , . . . is a sequence of nonnegative random variables, then

∞
X ∞
X
E Zk = E Zk . (4.4)
k=1 k=1

(a) Show that this follows from Fubini’s

P∞ theorem (as an alternative P∞ to the
arguments of Exercise 3.11). If k=1 E Zk < ∞, what is P( k=1 Zk =
∞)?
(b) Formulate a result similar to (4.4) for random variables Zk that may assume
negative values as well.
4.8 Let f be defined on R2 such that for all a ∈ R the function y 7→ f (a, y)
is Borel measurable and such that for all b ∈ R the function x 7→ f (x, b) is
continuous.

38
 (a) Show that for all a, b, c ∈ R the function (x, y) 7→ bx + cf (a, y) is Borel-
measurable on R2 .
(b) Let ani = i/n, i ∈ Z, n ∈ N. Define
X ani − x x − ani−1
f n (x, y) = 1(ani−1 ,ani ] (x)( f (ani−1 , y) + n f (ani , y)).
i
ani n
− ai−1 ai − ani−1

Show that the f are Borel-measurable on R2 and conclude that f is Borel-

n

measurable on R2 .
4.9 Show that for t > 0
Z ∞
1
sin x e−tx dx = .
0 1 + t2
Although x 7→ sinx x doesn’t belong L1 ([0, ∞), B([0, ∞)), λ), show that one can
use Fubini’s theorem to compute the improper Riemann integral
Z ∞
sin x π
dx = .
0 x 2

4.10 Let F, G : R → R be nondecreasing and right-continuous. Similar to the

case of distribution functions, these generate measures µF and µG on the Borel
sets satisfying R F (b) − F (a). Integrals w.r.t µF are commonly
R e.g. µF ((a, b]) =
denoted by f dF instead of f dµF .
(a) Use Fubini’s theorem to show the integration by parts formula, valid for
all a < b,
Z Z
F (b)G(b) − F (a)G(a) = F (s−) dG(s) + G(s) dF (s),
(a,b] (a,b]

where F (s−) = limu↑s F (u). Hint: integrate 1(a,b]2 and split the square
into a lower and an upper triangle.
(b) The above displayed formula is not symmetric in F and G. Show that it
can be rewritten in the symmetric form
F (b)G(b) − F (a)G(a) =
Z Z
F (s−) dG(s) + G(s−) dF (s) + [F, G](b) − [F, G](a),
(a,b] (a,b]
P
where [F, G](t) = a<s≤t ∆F (s)∆G(s) (for t ≥ a), with ∆F (s) = F (s) −
F (s−). Note that this sum involves at most countably many terms and is
finite.
4.11 Let F be the distribution function of a nonnegative random variable X
and α >R 0. Show (use Exercise 4.10 for instance, or write E X α = E f (X), with
x
f (x) = 0 αy α−1 dy) that
Z ∞
α
EX = α xα−1 (1 − F (x)) dx.
0

39
4.12 Let I be an arbitrary uncountable index set. For each Q i there is a proba-
bility space (Ωi , Fi , Pi ). Define the product σ-algebra F on i∈I Ωi as for the
case that I is Qcountable. Call a set C a countable cylinder if it can be written
as a product i∈I Ci , with Ci ∈ Fi and Ci a strict subset of Ωi for at most
countably many indices i.
(a) Show that the collection of countable cylinders is a σ-algebra, that it con-
tains the measurable rectangles and that every set in F is in fact a count-
able cylinder.
Q
(b) Let F = i∈I Ci ∈ F and let IF beQ the set of indices i for which Ci is a
strict subset of Ωi . Define P(F ) := i∈IF Pi (Ci ). Show that this defines
a probability measure on F with the property that P(πi−1 [E]) = Pi (E) for
every i ∈ I and E ∈ Fi .
4.13 Let X be a random variable, defined on some (Ω, F, P). Let Y be random
variable defined on another probability space (Ω0 , F 0 , P0 ). Consider the product
space, with the product σ-algebra and the product probability measure. Rede-
fine X and Y on the product space by X(ω, ω 0 ) = X(ω) and Y (ω, ω 0 ) = Y (ω 0 ).
Show that the redefined X and Y are independent and that their marginal
distributions are the same as they were originally.

40
5 Derivative of a measure
The topics of this chapter are absolute continuity and singularity of a pair of
measures. The main result is a kind of converse of Proposition 3.22, known as
the Radon-Nikodym theorem, Theorem 5.4.

5.1 Absolute continuity and singularity

We start this section with the definition of absolute continuity and singularity
for two measures. The former is connected to Section 3.3.
Definition 5.1 Let µ and ν be measures on a measurable space (S, Σ). We say
that ν is absolutely continuous w.r.t. µ (notation ν  µ), if ν(E) = 0 for every
E ∈ Σ with µ(E) = 0. Two arbitrary measures µ and ν on (S, Σ) are called
mutually singular (notation ν ⊥ µ) if there exist disjoint sets E and F in Σ such
that ν(A) = ν(A ∩ E) and µ(A) = µ(A ∩ F ) for all A ∈ Σ.
An example of absolute continuity is provided by the measures ν and µ of (3.7),
ν  µ. See also Proposition 5.3 below. Note that for two mutually singu-
lar measures µ and ν one has ν(F ) = µ(E) = 0, where E and F are as in
Definition 5.1.
Proposition 5.2 Let µ, νa and νs be measures on (S, Σ). Assume that νa  µ
and νs ⊥ µ. Put
ν = νa + νs . (5.1)
Suppose that ν also admits the decomposition ν = νa0 + νs0 with νa0  µ and
νs0 ⊥ µ. Then νa0 = νa and νs0 = νs .
Proof Omitted. 
The content of Proposition 5.2 is that the decomposition (5.1) of ν, if it exists,
is unique. We will see in Section 5.2 that, given a σ-finite measure µ, such a
decomposition exists for any σ-finite measure ν and it is called the Lebesgue
decomposition of ν w.r.t. µ. We extend the definition of the measure ν as given
in (3.7) to the real and complex case.
Proposition 5.3 Let µ be a measure on (S, Σ) and h a nonnegative measurable
function on S. Then the map ν : Σ → [0, ∞] defined by
ν(E) = µ(1E h) (5.2)
is a measure on (S, Σ) that is absolutely continuous w.r.t. µ.
Proof See Exercise 3.8. 
The Radon-Nikodym theorem of the next section states that every measure ν
that is absolutely continuous w.r.t. µ is of the form (5.2). We will use in that
case the notation
dν
h= .
dµ

41
5.2 The Radon-Nikodym theorem
As an appetizer for the Radon-Nikodym theorem (Theorem 5.4) we consider
a special case. Let S be a finite or countable set and Σ = 2S . Let µ be a
σ-finite measure on (S, Σ) and ν another, finite, measure such that ν  µ.
ν({x})
Define h(x) = µ({x}) if µ({x}) > 0 and zero otherwise. It is easy to verify that
1
h ∈ L (S, Σ, µ) and

ν(E) = µ(1E h), ∀E ⊂ S. (5.3)

Observe that we have obtained an expression like (5.2), but now starting from
the assumption ν  µ. The principal theorem on absolute continuity (and
singularity) is the following.

Theorem 5.4 Let µ be a σ-finite measure and let ν be a finite measure. Then
there exists a unique decomposition ν = νa + νs and a nonnegative function
h ∈ L1 (S, Σ, µ) such that νa (E) = µ(1E h) for all E ∈ Σ (so νa  µ) and
νs ⊥ µ. Moreover, h is unique in the sense that any other h0 with this property
is such that µ({h 6= h0 }) = 0. The function h is called the Radon-Nikodym
derivative of νa w.r.t. µ and is often written as
dνa
h= .
dµ
Proof Omitted. 

Remark 5.5 If ν is a σ-finite measure, then the Radon-Nikodym theorem is

still true with the exception that we only have µ(h1Sn ) < ∞, where the Sn form
a measurable partition of S such that ν(Sn ) < ∞ for all n. Notice that in this
case we may still take h ≥ 0.

Remark 5.6 The function h of Theorem 5.4, the Radon-Nikodym derivative

of νa w.r.t. µ, is also called the density of νa w.r.t. µ. If λ is Lebesgue measure
on (R, B) and ν is the law of a random variable XRthat is absolutely continuous
dν
w.r.t. λ, we have that F (x) := ν((−∞, x]) = (−∞,x] f dλ, where f = dλ .
Traditionally, the function f was called the density of X, and we see that calling
a Radon-Nikodym derivative a density is in agreement with this tradition, but
also extends it.

Theorem 5.4 is often used for probability measures Q and P with Q  P. Write
Z = dQ
dP and note that E Z = 1. It is immediate from Proposition 3.22 that for
X ∈ L1 (Ω, F, Q) one has

EQ X = E [XZ], (5.4)

where EQ is used to denote expectation under the probability measure Q.

42
5.3 Decomposition of a distribution function
In elementary probability one often distinguishes between distribution functions
that are of pure jump type (for discrete random variables) and those that admit
an ordinary density. These can both be recognized as examples of the following
result.

Proposition 5.7 Let F be a distribution function, F : R → R. Then there

exists a purely discontinuous right-continuous nondecreasing function Fd , with
limx→−∞ Fd (x) = 0, a nonnegative Borel-measurable function f and a nonde-
creasing continuous function Fs with limx→−∞ Fs (x) = 0 such that the decom-
position

F = Fd + Fs + Fac ,
Rx
holds true, with Fac defined by Fac (x) = −∞
f (y) dy. Such a decomposition is
unique.

Proof Since F is increasing, it has at most countably many discontinuities,

collected in a set D. Define
X
Fd (x) = ∆F (y).
y∈D∩(−∞,x]

One verifies that Fd has the asserted properties, the set of discontinuities of Fd is
also D and that Fc := F −Fd is continuous. Up to the normalization constant 1−
Fd (∞), Fc is a distribution function if Fd (∞) < 1 and equal to zero if Fd (∞) = 1.
Hence there exists a subprobability measure µc on B such that µc ((−∞, x]) =
Fc (x), Theorem 2.10. According to the Radon-Nikodym theorem, we can split
µc = µac + µs , where µac is absolutely continuous w.r.t. Lebesgue measure
λ. Hence,R there exists a λ-a.e. unique function f in L1+ (R, B, λ) such that
µac (B) = B f dλ. 

We have already encountered two examples, where the above decomposition

consists of a single term only. If a P
random variable X has a discrete distribution,
there are xk , k = 1, 2, . . . with k≥1 P(X = xk ) = 1, F = Fd , and if the
distribution function admits a density f , then F = Fac . Another extreme case
occurs when F is the distribution function of Exercise 5.2, then F = Fs . Think of
an example of a random variable for which all three terms in the decomposition
of Proposition 5.7 are nontrivial.

5.4 Exercises
5.1 Let X be a symmetric Bernoulli distributed random variable (P(X = 0) =
P(X = 1) = 21 ) and Y uniformly distributed on [0, θ] (for some arbitrary θ > 0).
Assume that X and Y are independent.
(a) Show that the laws Lθ (θ > 0) of XY are not absolutely continuous w.r.t.
Lebesgue measure on R.

43
 (b) Find a fixed dominating σ-finite measure µ such that Lθ  µ for all θ and
determine the corresponding Radon-Nikodym derivatives.
5.2 Let X1 , X2 , . . . be an iid sequence of Bernoulli random variables, defined on
some probability space (Ω, F, P) with P(X1 = 1) = 12 . Let
∞
X
X= 2−k Xk .
k=1

(a) Find the distribution of X.

(b) A completely different situation occurs when we ignore the odd numbered
random variables. Let
∞
X
Y =3 4−k X2k ,
k=1

where the factor 3 only appears for esthetic reasons. Show that the distri-
bution function F : [0, 1] → R of Y is constant on ( 14 , 43 ), that F (1 − x) =
1 − F (x) and that it satisfies F (x) = 2F (x/4) for x < 41 .
(c) Make a sketch of F and show that F is continuous, but not absolutely
continuous w.r.t. Lebesgue measure.
R (Hence there is no Borel measurable
function f such that F (x) = [0,x] f (u) du, x ∈ [0, 1]).

5.3 Let f ∈ L1 (S, Σ, µ) be such that µ(1E f ) = 0 for all E ∈ Σ. Show that
µ({f 6= 0}) = 0. Conclude that the function h in the Radon-Nikodym theorem
has the stated uniqueness property.
5.4 Let µ and ν be σ-finite measures and φ an arbitrary measure on a measurable
space (S, Σ). Assume that φ  ν and ν  µ. Show that φ  µ and that
dφ dφ dν
= .
dµ dν dµ

dν
5.5 Let ν and µ be σ-finite measures on (S, Σ) with ν  µ and let h = dµ , the
standing assumptions in this exercise. Show that ν({h = 0}) = 0. Show that
µ({h = 0}) = 0 iff µ  ν. What is dµ
dν if this happens?

5.6 Let µ and ν be σ-finite measures and φ a finite measure on (S, Σ). Assume
that φ  µ and ν  µ with Radon-Nikodym derivatives h and k respectively.
Let φ = φa + φs be the Lebesgue decomposition of φ w.r.t. µ. Show that (ν-a.e.)
dφa h
= 1{k>0} .
dν k

5.7 Consider the measurable space (Ω, F) and a measurable map X : Ω → Rn

(Rn is endowed with the usual Borel σ-algebra B n ). Consider two probability
measure P and Q on (Ω, F) and let PX and QX be the corresponding dis-
tributions (laws) on (Rn , B n ). Assume that PX and QX are both absolutely

44
continuous w.r.t. some σ-finite measure (e.g. Lebesgue measure), with corre-
sponding Radon-Nikodym derivatives (in this context often called densities) f
and g respectively, so f, g : Rn → [0, ∞). Assume that g > 0. Show that for
F = σ(X) it holds that P  Q and that (look at Exercise 5.6) the Radon-
Nikodym derivative here can be taken as the likelihood ratio

dP f (X(ω))
ω 7→ (ω) = .
dQ g(X(ω))

5.8 Show that the two displayed formulas in Exercise 4.10 are valid for functions
F and G that are of bounded variation over some interval (a, b]. The integrals
should be taken in the Lebesgue-Stieltjes sense.

5.9 Let a random variable X have distribution function F with the decomposi-
tion as in Proposition 5.7.
(a) Suppose that Fs = 0. Assume that E X is well defined. How would one
compute this expectation practically? See also the introductory paragraph
of Chapter 3 for an example where this occurs, and compute for that case
E X explicitly. Verify the answer by exploiting the independence of Y and
Z.
(b) As an example of the other extreme case, suppose that F is the distribution
of Y as in Exercise 5.2, so F = Fs . What is E Y here?

45
6 Conditional expectation
6.1 A simple, finite case
Let X be a random variable with values in {x1 , . . . , xn } and Y a random variable
with values in {y1 , . . . , ym }. The conditional probability

P(X = xi , Y = yj )
P(X = xi |Y = yj ) :=
P(Y = yj )

is well defined if P(Y = yj ) > 0. Otherwise we define it to be zero. We write

Ej for {Y = yj }. The conditional expectation x̂j := E [X|Ej ] is then
X
x̂j = xi P(X = xi |Ej ).
i

We define now a new random variable X̂ by

X
X̂ = x̂j 1Ej .
j

Since X̂ = x̂j on each event {Y = yj }, we call X̂ the conditional expectation

of X given Y . It has two remarkable properties. First we see that X̂ is σ(Y )-
measurable. The second property, which we prove below, is

E X̂1Ej = E X1Ej ,

the expectation of X̂ over the set Ej is the same as the expectation of X over
that set. We show this by simple computation. Note first that the values of
X1Ej are zero and xi , the latter reached on the event {X = xi } ∩ Ej that has
probability P({X = xi } ∩ Ej ). Note too that X̂1Ej = x̂j 1Ej . We then get

E X̂1Ej = x̂j P(Ej )

X
= xi P({X = xi }|Ej )P(Ej )
i
X
= xi P({X = xi } ∩ Ej )
i
= E X1Ej .

Every event E ∈ σ(Y ) is a finite union of events Ej . It then follows that

E X̂1E = E X1E , ∀E ∈ σ(Y ). (6.1)

The just described two properties of the conditional expectation will lie at the
heart of a more general concept, conditional expectation of a random variable
given a σ-algebra, see Section 6.2.
The random variable X̂ is a.s. the only σ(Y )-measurable random variable that

46
satisfies (6.1). Indeed, suppose that Z is σ(Y )-measurable and that E Z1E =
E X1E , ∀E ∈ σ(Y ). Let E = {Z > X̂}. Then (Z − X̂)1E ≥ 0 and has
expectation zero since E ∈ σ(Y ), so we have (Z − X̂)1{Z>X̂} = 0 a.s. Likewise
we get (Z − X̂)1{Z<X̂} = 0 a.s. and it then follows that Z − X̂ = 0 a.s.

6.2 Conditional expectation for X ∈ L1 (Ω, F, P)

Let (Ω, F, P) be a probability space and G a sub-σ-algebra of F. Assume that
X ∈ L1 (Ω, F, P). Inspired by the results of the previous section we adopt the
following definition.
Definition 6.1 A random variable X̂ is called a version of the conditional
expectation E [X|G], if it is G-measurable and if
E X̂1G = E X1G , ∀G ∈ G. (6.2)
If G = σ(Y ), where Y is a random variable, then we usually write E [X|Y ]
instead of E [X|σ(Y )].
Theorem 6.2 If X ∈ L1 (Ω, F, P), then a version of the conditional expectation
E [X|G] exists and moreover, any two versions are a.s. equal.
Proof For any G ∈ G we define ν + (G) := E X + 1G and ν − (G) := E X − 1G We
have seen that ν + and ν − are finite measures on the measurable space (Ω, G).
Moreover, ν +  P and ν −  P on this space. According to the Radon-Nikodym
Theorem 5.4 there exist nonnegative G-measurable functions ξ + and ξ − such
that ν + (G) = E ξ + 1G and ν − (G) = E ξ − 1G . These functions are a.s. unique.
Then X̂ = ξ + − ξ − is a version of E [X|G]. 
Remark 6.3 The ξ + and ξ − in the above proof are in general not equal to the
positive and negative parts X̂ + and X̂ − of X̂. Think of a simple example.
Remark 6.4 It is common to call a given version of E [X|G] the conditional
expectation of X given G, but one should take care with this custom. In fact
one should consider E [X|G] as an equivalence class of random variables, where
equivalence Y1 ∼ Y2 for G-measurable functions means that P(Y1 = Y2 ) = 1. As
such one can consider E [X|G] as an element of L1 (Ω, G, P). Later on we will
often identify a version X̂ of E [X|G] with E [X|G].
Remark 6.5 One can also define versions of conditional expectations for ran-
dom variables X with P(X ∈ [0, ∞]) = 1 without requiring that E X < ∞.
Again this follows from the Radon-Nikodym theorem. The definition of condi-
tional expectation can also be extended to e.g. the case where X = X + − X − ,
where E X − < ∞, but E X + = ∞.
Let us present the most relevant properties of conditional expectation. As be-
fore, we let X ∈ L1 (Ω, F, P), G a sub-σ-algebra of F and X̂ is a version of
E [X|G]. Other random variables below that are versions of a conditional ex-
pectation given G are similarly denoted with a ‘hat’.

47
Proposition 6.6 The following elementary properties hold.
(i) If X ≥ 0 a.s., then X̂ ≥ 0 a.s. If X ≥ Y a.s., then X̂ ≥ Ŷ a.s.
(ii) E X̂ = E X.
(iii) If a, b ∈ R and if X̂ and Ŷ are versions of E [X|G] and E [Y |G], then
aX̂ + bŶ is a version of E [aX + bY |G].
(iv) If X is G-measurable, then X is a version of E [X|G].

Proof (i) Let G = {X̂ < 0}. Then we have from (6.2) that 0 ≥ E 1G X̂ =
E 1G X ≥ 0. Hence 1G X̂ = 0 a.s.
(ii) Take G = Ω in (6.2).
(iii) Just verify that E 1G (aX̂ + bŶ ) = E 1G (aX + bY ), for all G ∈ G.
(iv) Obvious. 

We have taken some care in formulating the assertions of the previous theorem
concerning versions. Bearing this in mind and being a bit less precise at the
same time, one often phrases e.g. (iii) as E [aX + bY |G] = aE [X|G] + bE [Y |G].
Some convergence properties are listed in the following theorem.

Theorem 6.7 The following convergence properties for conditional expectation

given a fixed sub-σ-algebra hold.
(i) If (Xn ) is an a.s. increasing sequence of nonnegative random variables,
then the same holds for versions (X̂n ). If moreover Xn ↑ X a.s., then
X̂n ↑ X̂ a.s. (monotone convergence for conditional expectations)
(ii) If (Xn ) is a sequence of a.s. nonnegative random variables, and (X̂n ) are
corresponding versions of the conditional expectations, then
lim inf n→∞ X̂n ≥ `ˆ a.s., where `ˆ is a version of the conditional expectation
of ` := lim inf n→∞ Xn . (Fatou’s lemma for conditional expectations)
(iii) If (Xn ) is a sequence of random variables such that for some X one has
Xn → X a.s. and if there is a random variable Y such that E Y < ∞ and
|Xn | ≤ Y a.s. for all n. Then X̂n → X̂ a.s. (dominated convergence for
conditional expectations)

Proof (i) From the previous theorem we know that the X̂n form a.s. an increas-
ing sequence. Let X̂ := lim sup X̂n , then X̂ is G-measurable and X̂n ↑ X̂ a.s.
We verify that this X̂ is a version of E [X|G]. But this follows by application of
the Monotone Convergence Theorem to both sides of E 1G Xn = E 1G X̂n for all
G ∈ G.
(ii) and (iii) These properties follow by mimicking the proofs of the ordinary
versions of Fatou’s Lemma and the Dominated Convergence Theorem, Exer-
cises 6.4 and 6.5. 

Theorem 6.8 Additional properties of conditional expectations are as follows.

(i) If H is a sub-σ-algebra of G, then any version of E [X̂|H] is also a version
of E [X|H] and vice versa (tower property).

48
 (ii) If Z is G-measurable such that ZX ∈ L1 (Ω, F, P), then Z X̂ is a version
of E [ZX|G]. We write ZE [X|G] = E [ZX|G].
(iii) Let X̂ be a version of E [X|G]. If H is independent of σ(X) ∨ G, then X̂
is a version of E [X|G ∨ H]. In particular, E X is a version of E [X|H] if
σ(X) and H are independent.
(iv) Let X be a G-measurable random variable and let the random variable
Y be independent of G. Assume that h ∈ B(R2 ) is such that h(X, Y ) ∈
L1 (Ω, F, P). Put ĥ(x) = E [h(x, Y )]. Then ĥ is a Borel function and ĥ(X)
is a version of E [h(X, Y )|G].
(v) If c : R → R is a convex function and E |c(X)| < ∞, then c(X̂) ≤ C,
a.s., where C is any version of E [c(X)|G]. We often write c(E [X|G]) ≤
E [c(X)|G] (Jensen’s inequality for conditional expectations).
(vi) ||X̂||p ≤ ||X||p , for every p ≥ 1.

Proof (i) Let X̃ be a version of E [X̂|H]. By definition, we have E 1H X̃ =

E 1H X̂, for all H ∈ H. But since H ⊂ G, it also holds that E 1H X̂ = E 1H X,
by (6.2). Hence X̃ is a version of E [X|H].
(ii) We give the proof for bounded Z. Certainly ZX is integrable in this
case and its conditional expectation exists. Without loss of generality we may
then even assume that Z ≥ 0 a.s. (Add a constant c to Z to have Z + c ≥ 0,
if this is not the case and the result will follow from the case of nonnegative
Z). Assume first that also X is nonnegative. If Z = 1G for some G ∈ G, then
the result directly follows from the definition. By linearity the assertion holds
for nonnegative simple Z. For arbitrary Z ≥ 0, we choose simple Zn such that
Zn ↑ Z. Since we know (in the sense of versions) Zn X̂ = E [Zn X|G], we apply
Theorem 6.7 (i)) to settle the case for X ≥ 0. If X is arbitrary, linearity yields
the assertion by applying the previous results for X + and X − .
(iii) It is sufficient to show this for nonnegative X. Let G ∈ G and H ∈
H. By the independence assumption, we have E 1G 1H X = E 1G X P(H) and
E 1G 1H X̂ = E 1G X̂ P(H). It follows that E 1G 1H X = E 1G 1H X̂, since X̂ is
version of E [X|G]. Recall from Exercise 1.6 that the collection C := {G ∩ H :
G ∈ G, H ∈ H} is a π-system that generates G ∨ H. Observe that E 7→ E 1E X
and E 7→ E 1E X̂ both define measures on G ∨ H and that these measures have
been seen to coincide on C. It follows from Theorem 1.15 that these measures
are the same. The second statement follows by taking G = {∅, Ω}.
(iv) We use the Monotone Class Theorem, Theorem 2.6 and for simplicity of
notation we take X and Y real valued. Let V be the collection of all bounded
measurable functions for which the statement holds true. Using (iii), one easily
checks that h = 1B×C ∈ V , where B, C are Borel sets in R. The sets B ×C form
a π-system that generates B(R2 ). The collection V is obviously a vector space
and the constant functions belong to it. Let (hn ) be an increasing sequence
of nonnegative functions in V that converge to some bounded function h. If
ĥn (x) = E hn (x, Y ) and ĥ(x) = E h(x, Y ), then we also have ĥn (x) ↑ ĥ(x) for all
x by the Monotone Convergence Theorem. We will see that ĥ(X) is a version of
E [h(X, Y )|G]. Let G ∈ G. For all n it holds that E 1G ĥn (X) = E 1G hn (X, Y ).

49
Invoking the Monotone Convergence Theorem again results in E 1G ĥ(X) =
E 1G h(X, Y ). Since all ĥn (X) are G-measurable, the same holds for ĥ(X) and
we conclude that h ∈ V . The remainder of the proof is Exercise 6.11.
(v) Since c is convex, there are sequences (an ) and (bn ) in R such that
c(x) = sup{an x + bn : n ∈ N}, ∀x ∈ R. Hence for all n we have c(X) ≥
an X + bn and by the monotonicity property of conditional expectation, we
also have C ≥ an X̂ + bn a.s. If Nn is the set of probability zero, where this
inequality is violated, then also P(N ) = 0, where N = ∪∞n=1 Nn . Outside N we
have C ≥ supn (an X̂ + bn ) = c(X̂).
(vi) The statement concerning the p-norms follows upon choosing c(x) = |x|p
in (v) and taking expectations. 

Let P : L1 (Ω, F, P) → L1 (Ω, G, P) be the linear map that transforms X into

E [X|G]. If X̂ is a version of E [X|G], then it is also a version of E [X̂|G]. So, we
get P 2 = P , meaning that P is a projection. In the next proposition we give
this a geometric interpretation in a slightly narrower context.

Proposition 6.9 Let X ∈ L2 (Ω, F, P) and G a sub-σ-algebra of F. If X̂ is a

version of E [X|G], then X̂ ∈ L2 (Ω, G, P) and

E (X − Y )2 = E (X − X̂)2 + E (X̂ − Y )2 , ∀Y ∈ L2 (Ω, G, P).

Hence, E (X −Y )2 ≥ E (X − X̂)2 , ∀Y ∈ L2 (Ω, G, P). Conditional expectations of

square integrable random variables can thus be viewed as orthogonal projections
onto L2 (Ω, G, P).

Proof Exercise 6.3. 

We conclude this section with the following loose statement, whose message
should be clear from the above results. A conditional expectation is a random
variable that has properties similar to those of ordinary expectation.

6.3 Conditional probabilities

Let F ∈ F and G a sub-σ-algebra of F. We define P(F |G) := E [1F |G], the
conditional probability of F given G. So a version of P(F |G) is a G-measurable
random variable P̂(F ) that satisfies

P(F ∩ G) = E [P̂(F )1G ], ∀G ∈ G.

Likewise, one can define conditional distributions of a random variable X. For

a Borel set B one defines PX (B|G) := P(X −1 [B]|G).
Of course all versions of P(F |G) are almost surely equal. Moreover, if F1 , F2 , . . .
is a sequence of disjoint events, and P̂(Fn ) are versions of the conditional prob-
P∞
abilities, then one easily shows that n=1 P̂(Fn ) is a version of the conditional

50
probability P(∪∞ ∞ ∞
n=1 Fn |G). So, if P̂(∪n=1 Fn ) is any version of P(∪n=1 Fn |G), then
outside a set N of probability zero, we have
∞
X
P̂(∪∞
n=1 Fn ) = P̂(Fn ). (6.3)
n=1

A problem is that the set N in general depends on the sequence of events

F1 , F2 , . . . Since there are usually uncountably many of such sequences, it is not
clear (and in fact not always true!) that there is one (fixed) set of probability
zero such that outside this set for all disjoint sequences (Fn ) the equality (6.3)
holds true. But if it does, this means that for every F ∈ F, there exists a
random variable P̂(F ) that is a version of P(F |G) and such that for all ω outside
a set N with P(N ) = 0 the map F 7→ P̂(F )(ω) is a probability measure on F.
In this case, the map

F × Ω 3 (F, ω) 7→ P̂(F )(ω)

is called a regular conditional probability given G.

In the above setup for regular conditional probabilities, relation (6.3) is
assumed to hold outside a set N of probability zero. Of course, if N = ∅, this
relation holds everywhere. But also of N 6= ∅, this relation can be turned into
one that is everywhere true. Suppose that N 6= ∅. Redefine P̂ by taking P̂(F )(ω)
as given on N c , but for all ω ∈ N we take instead P̂(·)(ω) as any fixed probability
measure on F (for instance a Dirac measure). Since we change the map P̂(F ) on
the null set N only, we keep on having a conditional probability of F , whereas
(6.3) now holds everywhere. One easily checks that the modification P̂(·)(·)
enjoys the following properties. For any fixed ω, P̂(·)(ω) is a probability measure,
whereas for any fixed F ∈ F, P̂(F )(·) is a G-measurable function. These two
properties are often cast by saying that (F, ω) 7→ P̂(F )(ω) is a probability kernel
defined on F × Ω.
As mentioned before, regular conditional probabilities do not always exist.
But when it happens to be the case, conditional expectations can be computed
through integrals.

Theorem 6.10 Let X be a (real) random variable with law PX , a probability

measure on (R, B). There exists a regular conditional distribution of X given G.
That is, there exists a probability kernel P̂X on B × Ω with the property that
P̂X (B) is a version of P(X −1 [B]|G) for all B ∈ B.

Proof We split the proof into two parts. First we show the existence of a
conditional distribution function, after which we show that it generates a regular
conditional distribution of X given G.
We will construct a conditional distribution function on the rational num-
bers. For each q ∈ Q we select a version of P(X ≤ q|G), call it G(q). Let
Erq = {G(r) < G(q)}. Assume that r > q. Then {X ≤ r} ⊃ {X ≤ q} and hence
G(r) ≥ G(q) a.s. and so P(Erq ) = 0. Hence we obtain that P(E) = 0, where

51
E = ∪r>q Erq . Note that E is the set where the random variables G(q) fail to be
increasing in the argument q. Let Fq = {inf r>q G(r) > G(q)}. Let {q1 , q2 , . . .}
be the set of rationals strictly bigger then q and let rn = inf{q1 , . . . , qn }.
Then rn ↓ q, as n → ∞. Since the indicators 1{X≤rn } are bounded, we
have G(rn ) ↓ G(q) a.s. It follows that P(Fq ) = 0, and then P(F ) = 0, where
F = ∪q∈Q Fq . Note that F is the event on which G(·) is not right-continuous.
Let then H be the set on which limq→∞ G(q) < 1 or limq→−∞ G(q) > 0. By a
similar argument, we have P(H) = 0. On the set Ω0 := (E ∪ F ∪ H)c , the ran-
dom function G(·) has the properties of a distribution function on the rationals.
Note that Ω0 ∈ G. Let F 0 be an arbitrary distribution function and define for
x∈R

F̂ (x) = 1Ωc0 F 0 (x) + 1Ω0 inf G(q).

q>x

It is easy to check that F̂ (·) is a distribution function for each hidden argument
ω. Moreover, F̂ (x) is G-measurable and since inf q>x 1{X≤q} = 1{X≤x} , we
obtain that F̂ (x) is a version of P(X ≤ x|G). This finishes the proof of the
construction of a conditional distribution function of X given G.
For every ω, the distribution function F̂ (·)(ω) generates a probability mea-
sure PX (·)(ω) on (R, B(R)). Let C be the class of Borel-measurable sets B for
which PX (B) is a version of P(X ∈ B|G). It follows that all intervals (−∞, x]
belong to C. Moreover, C is a d-system. By virtue of Dynkin’s Lemma 1.13,
C = B(R). 

Proposition 6.11 Let X be a random variable and h : R → R be a Borel-

measurable function. Let P̂X be a regular conditional distribution of X given
G. If h(X) ∈ L1 (Ω, F, P), then
Z
h(x) P̂X (dx) (6.4)

is a version of the conditional expectation E [h(X)|G].

Proof Consider the collection H of all Borel functions h for which (6.4) is a
version of E [h(X)|G]. Clearly, in view of Theorem 6.10 the indicator functions
1B for B ∈ B(R) belong to H and so do linear combinations of them. If
h ≥ 0, then we can find nonnegative simple functions hn that convergence to h
in a monotone way. Monotone convergence for conditional expectations yields
h ∈ H. If h is arbitrary, we split as usual h = h+ − h− and apply the previous
step. 

Once more we emphasize that regular conditional probabilities in general don’t

exist. The general definition of conditional expectation would be pointless if
every conditional expectation could be computed by Proposition 6.11. The
good news is that in most common situations Proposition 6.11 can be applied.
In Exercise 6.8 you find an explicit expression for the regular conditional
distribution of a random variable X given another random variable Y .

52
6.4 Exercises
6.1 Let (Ω, F, P) be a probability space and let A = {A1 , . . . , An } be a partition
1
of Ω, where the Ai belong to F. Let X P∈n L (Ω, F, P) and G = σ(A). Show that
any version of E [X|G] is of the form i=1 ai 1Ai and determine the ai .
6.2 Let Y be a (real) random variable or random vector on a probability space
(Ω, F, P). Assume that Z is another random variable that is σ(Y )-measurable.
Use the standard machine to show that there exists a Borel-measurable function
h on R such that Z = h(Y ). Conclude that for integrable X it holds that
E [X|Y ] = h(Y ) for some Borel-measurable function h.
6.3 Prove Proposition 6.9.
6.4 Prove the conditional version of Fatou’s lemma, Theorem 6.7(ii).
6.5 Prove the conditional Dominated Convergence theorem, Theorem 6.7(iii).
6.6 Let (X, Y ) have a bivariate normal distribution with E X = µX , E Y = µY ,
2
Var X = σX , Var Y = σY2 and Cov (X, Y ) = c. Let
c
X̂ = µx + (Y − µY ).
σY2

Show that E (X − X̂)Y = 0. Show also (use a special property of the bivariate
normal distribution) that E (X − X̂)g(Y ) = 0 if g is a Borel-measurable function
such that E g(Y )2 < ∞. Conclude that X̂ is a version of E [X|Y ].
6.7 Let X, Y ∈ L1 (Ω, F, P) and assume that E [X|Y ] = Y and E [Y |X] = X (or
rather, versions of them are a.s. equal). Show that P(X = Y ) = 1. Hint: Start
to work on E (X − Y )1{X>z,Y ≤z} + E (X − Y )1{X≤z,Y ≤z} for arbitrary z ∈ R.
6.8 Let X and Y be random variables and assume that (X, Y ) admits a density
f w.r.t. Lebesgue measure on (R2 , B(R2 )). Let fY be the marginal density of
Y . Define fˆ(x|y) by
(
f (x,y)
if fY (y) > 0
fˆ(x|y) = fY (y)
0 else.

h(x)fˆ(x|y) dx. Show that ĥ(Y ) is a

R
Assume that E |h(X)| < ∞. Put ĥ(y) = R
version of E [h(X)|Y ]. Show also that
Z
P̂(E) = fˆ(x|Y ) dx
E

defines a regular conditional probability on B(R) given Y . What is the excep-

tional set N of Section 6.3?
6.9 Consider a probability space (Ω, F, P). Two random variables X and Y are
called conditionally independent given a sub-σ-algebra G of F if for all bounded
Borel functions f, g : R → R it holds that E [f (X)g(Y )|G] = E [f (X)|G]E [g(Y )|G].

53
 (a) Show that X and Y are conditionally independent given G iff for every
bounded measurable function f : R → R it holds that E [f (X)|σ(Y ) ∨ G] =
E [f (X)|G].
(b) Show (by examples) that in general conditional independence is not implied
by independence, nor vice versa.
(c) If X and Y are given random variables, give an example of a σ-algebra G
that makes X and Y conditionally independent.
(d) Propose a definition of conditional independence of two σ-algebras H1 and
H2 given G that is such that conditional independence of X and Y given
G can be derived from it as a special case.
6.10 (Hölder’s inequality for conditional expectations) Let X ∈ Lp (Ω, F, P) and
Y ∈ Lq (Ω, F, P), where p, q ∈ [1, ∞], p1 + 1q = 1. Then

E [|XY | G] ≤ (E [|X|p |G])1/p (E [|Y |q |G])1/q . (6.5)

It is sufficient to show this for nonnegative X and Y , so assume X, Y ≥ 0 a.s.

(a) Let U = E [X p |G] and V = E [Y q |G] and H = {U, V > 0}. Suppose that

1H E [XY |G] ≤ 1H (E [X p |G])1/p (E [Y q |G])1/q ]. (6.6)

Show that Hölder’s inequality (6.5) follows from (6.6).

(b) Show that

E [XY |G]
E [1G 1H ≤ E [1G 1H ]
UV
holds for all G ∈ G and deduce (6.6).
6.11 Finish the proof of Theorem 6.8 (iv), i.e. show that the assertion also holds
without the boundedness condition on h.

54
Index
algebra, 1 null set, 4
σ-algebra, 1
product, 31 probability, 6
conditional, 49
distribution, 11 regular conditional probability, 50
joint, 35 probability kernel, 50
regular conditional distribution, 50
distribution function, 12 Radon-Nikodym derivative, 41
decomposition, 42 random variable, 11
distribution:conditional, 49 random vector, 34

event, 6 simple function, 16

expectation, 24 space
conditional, 46 Lp , 29
version, 46 Lp , 27
complete measure space, 4
independence, 13 measurable, 1
inequality measure, 2
Chebychev, 25 probability, 6
Hölder, 27 standard machine, 22
Jensen, 26 system
Markov, 25 π-system, 4
Minkowski, 28 d-system, 4
integral
Lebesgue, 17 theorem
Riemann, 22 dominated convergence, 22
Fubini, 33
law, 11 Lebesgue, 22
lemma monotone class theorem, 10
Fatou, 21 monotone convergence, 20
Radon-Nikodym, 41
measurable Skorokhod’s representation, 13
Borel
function, 9
set, 1
function, 9
set, 1
measure, 2
absolutely continuous, 40
Lebesgue, 3
Lebesgue decomposition, 40
probability, 6
product, 33
singular, 40

55