Class: MSc

Subject : Probability & Statistics

Chapter: Unit 2 Chapter 1
Chapter Name: Generating Functions

1
Today’s Agenda
1. Introduction
2. Moment Generating Functions
3. Cumulant Generating Functions
4. Linear Functions

2
1 Introduction

Topic Generating
Functions

Moment Cumulant
Types Generating
Functions
Generating
Functions

3
2 Moment Generating Functions
A moment generating function (MGF) can be used to generate moments (about the origin) of the
distribution of a random variable (discrete or continuous). Using MGF will simplify many calculations.

Definition: The MGF – written as 𝑀𝑥 (t), for a random variable X is given by,
𝑴𝒙 (t) = E[𝒆(𝒕𝑿) ]
for all values of t for which the expectations exist.

To find 𝑀𝑥 (t)

If X is discrete then, 𝑀𝑥 (t) = σ𝑥 𝒆(𝒕𝑿) P(X = x)

If X is continuous then, 𝑀𝑥 (t) = ‫𝑥𝑑 𝑥 𝑓 )𝑿𝒕(𝒆 ׬‬

4
2.1 Finding Moments through Derivatives
The method is to differentiate the MGF with respect to t and then set t = 0 , the rth derivative giving the rth
moment about the origin.

𝑑 𝑡𝑋
𝑀′𝑋 (𝑡) = 𝐸 𝑋𝑒 𝑡𝑋 , since 𝑀𝑋 (𝑡) = 𝐸 𝑒 𝑡𝑋 and 𝑒 = 𝑋𝑒 𝑡𝑋 .
𝑑𝑡

⇒ 𝑀𝑋′ (0) = 𝐸 𝑋𝑒 0 = 𝐸[𝑋]

Similarly for higher orders moments.

𝑀𝑋′′ 𝑡 = 𝐸 𝑋 2 𝑒 𝑡𝑋 ⇒ 𝑀𝑋′′ 0 = 𝐸(𝑋 2 )

𝑀𝑋′′′ 𝑡 = 𝐸 𝑋 3 𝑒 𝑡𝑋 ⇒ 𝑀𝑋′′′ 0 = 𝐸(𝑋 3 )

The formulae for the mean and variance are:

Mean = 𝑀𝑋′ (0) = 𝐸[𝑋]

Var = 𝑀𝑋′′ 0 - [𝑀𝑋′ (0)]2 5

2.2 Finding Moments through Series expansion
Expanding the exponential function and taking expected values throughout (a procedure which is justifiable
for the distributions here) gives:

𝑡𝑋
𝑡2 2 𝑡3 3
𝑀𝑋 (𝑡) =𝐸 𝑒 = 𝐸 1 + 𝑡𝑋 + 𝑋 + 𝑋 + ⋯
2! 3!
𝑡2 𝑡3
= 1 + 𝑡𝐸[𝑋] + 𝐸 𝑋 + 𝐸 𝑋 3 + ⋯
2
2! 3!

from which it is seen that the 𝑟 𝑡ℎ moment of the distribution about the origin, 𝐸 𝑋 r , is obtainable as the
𝑡r
coefficient of in the power series expansion of the MGF.
r!

6
2.3 Use of MGFs
• If the distribution of a random variable X is known, in theory at least, all moments of the distribution
that exist can be calculated. If the moments are specified, then the distribution can be identified.

• Without going deeply into mathematical rigour, it can in fact be said that if all moments of a
random variable exist (and if they satisfy a certain convergence condition) then the sequence of
moments uniquely determines the distribution of X.

• Further, if a moment generating function has been found, then there is a unique distribution with
that MGF. Thus an MGF can be recognised as the MGF of a particular distribution. (There is a one-
to-one correspondence between MGFs and distributions with MGFs).

7
2.3 Important examples – discrete distributions
The MGFs for some of the distributions introduced earlier are found as follows.

Discrete Uniform: (𝑒 𝑡/𝐾 )(1 − 𝑒 𝑘𝑡 )/(1 − 𝑒 𝑡 ) for t≠0

Binomial: (𝑞 + 𝑝𝑒 𝑡 ) 𝑛

𝑝𝑒 𝑡
Negative binomial: [1−𝑞𝑒 𝑡 ] 𝑘
𝑡
Poisson: 𝑒 𝜆(𝑒 −1)

8
2.3 Important examples – continuous distributions
The MGFs for some of the distributions introduced earlier are found as follows.

𝑒 𝑏𝑡 −𝑒 𝑎𝑡
Continuous Uniform:
𝑡(𝑏−𝑎)

𝑡
Gamma: (1 − )−𝛼 for t < 𝜆
𝜆

If t ≥ 𝜆, then the power in the exponential factor in the integral is positive and therefore the

answer is infinite. So, the MGF does not exist in this case.

chi-square: (1 − 2𝑡)−𝑣/2
1
(𝜇𝑡+2𝜎2 𝑡 2 )
Normal: 𝑒

9
Question
CT3 April 2005 Q3

Claim sizes in a certain insurance situation are modelled by a distribution with moment generating function
M(t) given by

Show that E[𝑋 2 ] = 600 and find the value of E[𝑋 3 ].

10
Question

11
Question
CT3 April 2007 Q6

Consider the discrete random variable X with probability function

(i) Show that the moment generating function of the distribution of X is given by

(i) Determine E[X] using the moment generating function given in part (i).

12
Solution

13
3 Cumulant Generating Functions
The cumulant generating function, 𝑪𝑿 (𝒕) , of a random variable X is given by:
𝑪𝑿 (𝒕) = ln 𝑴𝑿 (𝒕)
and so 𝑴𝒙 𝒕 = 𝒆𝑪𝒙(𝒕) .
As a result, if 𝐶𝑋 (𝑡) is known it is easy to determine 𝑀𝑋 (𝑡).

14
3.1 Finding Moments
If we differentiate 𝐶𝑋 (𝑡) = ln 𝑀𝑋 (𝑡), we obtain:

𝑀′𝑥 (𝑡)
𝐶𝑥′ (𝑡) =
𝑀𝑥 (𝑡)
and:

𝑀𝑋′′ (t)𝑀𝑋 (t) − 𝑀𝑋′ (t) 2

𝐶𝑋′′ t =
𝑀𝑋2 (t)
Now 𝑀𝑥 (0) = 1 so:
′
𝑀𝑋 0 𝐸[𝑋]
𝐶𝑋′ 0 − = = 𝐸[𝑋]
𝑀𝑋 0 1

and:

𝑀𝑋′′ (0)𝑀𝑋 (0) − 𝑀𝑋′ (0) 2

𝐶𝑋′′ (0) =
𝑀𝑋2 (0)
𝐸 𝑋 2 (1) − (𝐸[𝑋])2
=
12
= var[𝑋] 15
4 Linear Functions

Suppose X has MGF 𝑀𝑋 (t) and the distribution of a linear function Y = a + bX is of interest. The MGF
of Y, 𝑀𝑌 (t) say, can be obtained from that of X as follows:

𝑴𝒀 (t) = E[𝒆𝒕𝒀 ] = E[𝒆𝒕(𝒂+𝒃𝑿) ] = 𝒆𝒂𝒕 E[𝒆𝒃𝒕𝑿 ] = 𝒆𝒂𝒕 𝑴𝑿 (bt)

16
Question
CS1 September 2021 Q7

17
Solution

18
Summary
Moment Generating Functions
𝑀𝑋 (𝑡) = 𝐸 𝑒 𝑡𝑋 = σ𝑥 𝑒 𝑡𝑥 𝑃(𝑋 = 𝑥) or ‫𝑥𝑑)𝑥(𝑓 𝑥𝑡 𝑒 𝑥׬‬

𝐸(𝑋) = 𝑀𝑋′ (𝑂)

Var (𝑋) = 𝑀𝑋′′ (𝑂) − 𝑀𝑋′ (𝑂) 2

𝑡2 𝑡3
𝑀𝑋 (𝑡) = 1 + 𝑡𝐸(𝑋) + 2! 𝐸 𝑋 2 + 3! 𝐸 𝑋 3 + ⋯

Cumulant Generating Functions

𝐶𝑋 𝑡 = ln 𝑀𝑋 (𝑡)
𝐸(𝑋) = 𝐶𝑋′ (𝑂)
var(𝑋) = 𝐶𝑋′′ (𝑂)
skew(𝑋) = 𝐶𝑋′′ ′(𝑂) 19
Summary
Linear Transformations
If 𝑌 = 𝑎𝑋 + 𝑏 then,
𝑀𝑌 (𝑡) = 𝑒 𝑎𝑡 𝑀𝑋 (𝑏𝑡) and 𝐶𝑌 (𝑡) = 𝑎𝑡 + 𝐶𝑋 (𝑏𝑡)

The uniqueness property means that if two variables have the same MGF and CGF, then they have the
same distribution.

20