Probability & Statistics I Chapter6: Mathematical expectation and moments

Definition 6.1 Mathematical expectation of 𝑋, if it exists, is the value denoted by 𝐸(𝑋)

such that:
Discrete case :
𝑛

𝐸(𝑋) = ∑ 𝑥𝑖 ℙ(𝑋 = 𝑥𝑖 ).
𝑖=1
Continuous case :
+∞

𝐸(𝑋) = ∫ 𝑥𝑓𝑋 (𝑥)𝑑𝑥.

−∞
𝐸(𝑋) is also known as the mean of 𝑋.
6-1- Expectation of a function of a random variable

Let 𝑍 = 𝑔(𝑋) be a random variable which is a function of the random variable 𝑋. To

calculate 𝐸(𝑍) we can first determine its distribution (i.e. 𝑃𝑧 or 𝑓𝑧 ) from that of 𝑋.
However, it is possible to show that we can calculate 𝐸(𝑍) directly from the distribution
of 𝑋.

Proposition 6.1 Let 𝑔(𝑋) be a function of the random variable 𝑋. Then:

Discrete case :
𝑛

𝐸(𝑔(𝑋)) = ∑ 𝑔(𝑥𝑖 )ℙ(𝑋 = 𝑥𝑖 ).

𝑖=1

Continuous case :
+∞

𝐸(𝑔(𝑋)) = ∫ 𝑔(𝑥)𝑓𝑋 (𝑥)𝑑𝑥.

−∞
Example 6.1
Consider 𝑋 with a uniform distribution over the interval [0,1]. Let be the function
𝑍 = 𝑋2.
- Determine 𝐸(𝑍).
1st method:
+∞

𝐸(𝑍) = ∫ 𝑧𝑓𝑍 (𝑧𝑖 )𝑑𝑧

−∞
First, we look for the distribution of 𝑍:
1, 0≤𝑥≤1
𝑓(𝑥) = {
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
0, 𝑥<0
𝐹(𝑥) = { 𝑥, 0≤𝑥<1
1, 𝑥≥1
𝐹𝑍 (𝑧) = 𝑃(𝑍 ≤ 𝑧)
= 𝑃(𝑋 2 ≤ 𝑧)
= 𝑃(−√𝑧 ≤ 𝑋 ≤ √𝑧)
= 𝐹𝑋 (√𝑧) − 𝐹𝑋 (−√𝑧)

Dr. Sara Bouredja, bouredja@[Link] 1

Probability & Statistics I Chapter6: Mathematical expectation and moments

Since 𝑃(𝑋 < 0) = 0. Therefore: 𝐹𝑍 (𝑧) = 𝐹𝑋 (√𝑧) − 0 = √𝑧 for 𝑧 ∈ [0,1]. Then

1
𝑓𝑍 (𝑧) = 𝐹 ′ 𝑍 = for 𝑧 ∈ [0,1].
2√𝑧
1
, 𝐼𝑓 𝑧 ∈ [0,1]
𝑓𝑍 (𝑧) = {2√𝑧 ;
0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
1 1
𝐸(𝑍) = ∫0 𝑧 𝑑𝑧 2√𝑧
1 1
= ∫0 √𝑧√𝑧 2√𝑧 𝑑𝑧
1 1
= ∫ √𝑧 𝑑𝑧
2 0
1
1 2 3
= [ 𝑧2]
2 3 0
1
=
3
2nd method:
We can calculate directly :
𝐸(𝑍) = 𝐸(𝑔(𝑋)) = 𝐸(𝑋 2 )
1
= ∫0 𝑥 2 𝑓𝑋 (𝑥)𝑑𝑥
1
= ∫0 𝑥 2 . 1𝑑𝑥
1 1
= [ 𝑥3]
3 0
1
=
3
6-2- Moments
Definition 6.2 The simplest moment of order 𝑟 of the random variable 𝑋, where 𝑟 is a
positive integer, is the value (if it exists) :
+∞

𝜇𝑟 = 𝐸(𝑋 𝑟 ) = ∫ 𝑥 𝑟 𝑓𝑋 (𝑥)𝑑𝑥
−∞
The shape characteristics are based more on the centred moments, i.e. the mathematical
expectations of the powers of 𝑋 − 𝐸(𝑋), or 𝑋 − 𝜇.
Definition 6.3 The centred moment of order 𝑟 of the random variable 𝑋 , where 𝑟 is a
positive integer, is the value (if it exists):
𝜇′𝑟 = 𝐸((𝑋 − 𝜇)𝑟 )

• For 𝑟 = 1 we have : 𝜇′1 = 𝐸(𝑋 − 𝜇) = 𝐸(𝑋) − 𝜇 = 𝜇 − 𝜇 = 0 which characterises

the centring of 𝑋.
• For 𝑟 = 2 we have : 𝜇′2 = 𝐸((𝑋 − 𝜇)2 ) = 𝐸(𝑋 2 − 2𝑋𝜇 + 𝜇 2 )
= 𝐸(𝑋 2 ) + 𝐸(𝜇 2 ) − 2𝜇𝐸(𝑋)
= 𝐸(𝑋 2 ) + 𝜇 2 − 2𝜇𝜇
= 𝐸(𝑋 2 ) − 𝜇 2
= 𝐸(𝑋 2 ) − (𝐸(𝑋))2
= 𝑉(𝑋) which is a dispersion
characteristic of the distribution, as
in descriptive statistics.

Dr. Sara Bouredja, bouredja@[Link] 2

Probability & Statistics I Chapter6: Mathematical expectation and moments

6-3- Conditional expectation

Discrete case
𝑛

𝐸(𝑋\𝑌 = 𝑌) = ∑ 𝑥𝑖 ℙ𝑋\𝑌 (𝑋 = 𝑥𝑖 )
𝑖=1
𝑚

𝐸(𝑌\𝑋 = 𝑥) = ∑ 𝑦𝑗 ℙ𝑌\𝑋 (𝑌 = 𝑦𝑗 )
𝑗=1

Continuous case
+∞ +∞
𝑓(𝑋,𝑌) (𝑥, 𝑦)
𝐸(𝑋\𝑌 = 𝑦) = ∫ 𝑥 𝑓𝑋\𝑌 𝑑𝑥 = ∫ 𝑥 𝑑𝑥
𝑓𝑌 (𝑦)
−∞ −∞

+∞ +∞
𝑓(𝑋,𝑌) (𝑥, 𝑦)
𝐸(𝑌\𝑋 = 𝑥) = ∫ 𝑦 𝑓𝑌\𝑋 𝑑𝑦 = ∫ 𝑦 𝑑𝑦
𝑓𝑋 (𝑥)
−∞ −∞

Example 6.2 Let 𝑓 be the joint density of 𝑋 and 𝑌 defined by :

𝑥
−
𝑒 𝑦 𝑒 −𝑦
𝑓(𝑥, 𝑦) = , 0 < 𝑥 < +∞, 0 < 𝑦 < +∞.
𝑦
- Calculate 𝐸[𝑋|𝑌 = 𝑦].

First we calculate the conditional density:

𝑓𝑋\𝑌 (𝑥,𝑦)
𝑓𝑋\𝑌 (𝑥, 𝑦) =
𝑓𝑌 (𝑦)
𝑓(𝑥,𝑦)
= +∞
∫−∞ 𝑓(𝑥,𝑦)𝑑𝑥
𝑥
1 −𝑦 −𝑦
𝑒 𝑒
𝑦
= 𝑥
+∞ 1 −𝑦 −𝑦
∫−∞ 𝑦𝑒 𝑒
𝑥
1 −𝑦 −𝑦
𝑒 𝑒
𝑦
=
𝑒 −𝑦
𝑥
1 −𝑦
= 𝑒 This is the probability density of a
𝑦
1
random variable that follows an exponential distribution with parameter .
𝑦
1 1
𝑋\𝑌 ↷ 𝐸𝑋𝑃 ( ). Consequently, 𝐸[𝑋|𝑌 = 𝑦] = 1 = 𝑦.
𝑦
𝑦
+∞
Or, 𝐸[𝑋|𝑌 = 𝑦] = ∫−∞ 𝑥 𝑓𝑋\𝑌 𝑑𝑥
𝑥
+∞ 1 −
= ∫0 𝑥 𝑦 𝑒 𝑦 𝑑𝑥

=𝑦

Dr. Sara Bouredja, bouredja@[Link] 3

Probability & Statistics I Chapter6: Mathematical expectation and moments

6-4- Conditional variance

The conditional variance of 𝑋 can be defined in the same way as the conditional
expectation. We obtain:
2
𝑉(𝑋\𝑌) = 𝐸 [(𝑋 − 𝐸(𝑋\𝑌)) \𝑌]

= 𝐸(𝑋 2 \𝑌) − [𝐸(𝑋\𝑌)]2.

6-5- Moment generating function
The moment generating function interest us in the measure, it can facilitate the calculation
of a distribution's moments.
Definition 6.4 The moments generating function of the random variable 𝑋, if it exists, is
the function :
Ψ𝑋 (𝑡) = 𝐸(𝑒 𝑡𝑋 ).

Proposition 6.2 The moment of order 𝑟 of the random variable 𝑋 is given by :

(𝑟)
𝜇𝑟 = Ψ𝑋 (0).
(𝑟)
Where Ψ𝑋 is the derivative of order 𝑟 of Ψ𝑋 . In particular the mathematical
expectation (𝜇1 ) of 𝑋 is the value of the first derivative Ψ𝑋′ for 𝑡 = 0.
(1)
𝜇1 = 𝐸(𝑋) = Ψ𝑋 (0).

Example 6.3 The continuous exponential distribution, which depends on a

parameter 𝜆 > 0, takes the density:
𝜆𝑒−𝜆𝑥 , 𝑖𝑓 𝑥 ≥ 0;
𝑓(𝑥) = {
0, 𝑖𝑓 𝑥 < 0.

- Calculate the moments generating function of a random variable 𝑋.

- Calculate the expectation and variance.

Let's calculate the moments generating function of a random variable 𝑋 which follows
the exponential distribution :
Ψ𝑋 (𝑡) = 𝐸(𝑒 𝑡𝑋 )
+∞
= ∫0 𝑒 𝑡𝑥 𝜆𝑒 −𝜆𝑥 𝑑𝑥
+∞
= 𝜆 ∫0 𝑒 (𝑡−𝜆)𝑥 𝑑𝑥
1 +∞
= 𝜆[ 𝑒 (𝑡−𝜆)𝑥 ] for convergence, we assume that 𝑡 − 𝜆 < 0 → 𝑡 < 𝜆
𝑡−𝜆 0
−𝜆
=
𝑡−𝜆
𝜆
=
𝜆−𝑡
The expectation:
𝜇1 = 𝐸(𝑋) = Ψ′𝑋 (0)
𝜆
Ψ′𝑋 (𝑡) =
(𝜆 − 𝑡)2
1
→ 𝐸(𝑋) =
𝜆
The variance :
𝑉(𝑋) = 𝐸(𝑋 2 ) − 𝐸(𝑋)2
Dr. Sara Bouredja, bouredja@[Link] 4
Probability & Statistics I Chapter6: Mathematical expectation and moments

𝜇2 = 𝐸(𝑋 2 ) = Ψ′′𝑋 (0)

2(𝜆−𝑡)𝜆 2𝜆
Ψ′′𝑋 (𝑡) = 4 =
(𝜆−𝑡) (𝜆−𝑡)3
2𝜆 2
→ 𝐸(𝑋 2 ) = = .
𝜆3 𝜆2
2 1 1
We obtain 𝑉(𝑋) = − =
𝜆2 𝜆2 𝜆2

Theorem 9 let (𝑋, 𝑌) be a pair of independent random variables, each having a moment
generating function. Then the sum 𝑋 + 𝑌 has a moment generating function and we have:
Ψ𝑋+𝑌 (𝑡) = Ψ𝑋 (𝑡) Ψ𝑌 (𝑡)

Proof 6.1 we have

Ψ𝑋+𝑌 (𝑡) = 𝐸[𝑒 𝑡(𝑋+𝑌) ] = 𝐸[𝑒 𝑡𝑋 𝑒 𝑡𝑌 ].
Since 𝑋 and 𝑌 are independent, so the variables 𝑒 𝑡𝑋 and 𝑒 𝑡𝑌 are also independent. Then :
Ψ𝑋+𝑌 (𝑡) = 𝐸[𝑒 𝑡(𝑋+𝑌) ]
= 𝐸[𝑒 𝑡𝑋 𝑒 𝑡𝑌 ]
= 𝐸[𝑒 𝑡𝑋 ] 𝐸[𝑒 𝑡𝑌 ]
= Ψ𝑋 (𝑡) Ψ𝑌 (𝑡)

Corollary 6.1 the product of two moment generating functions is a moment generating
function.

Dr. Sara Bouredja, bouredja@[Link] 5