Moment Generating Function (MGF)
Moments
• k-th moment:
µk = E[X k ]
• k-th central moment:
µ′k = E[(X − E[X])k ]
For each integer k, the k-th moment of X is defined as above.
Moment Generating Function (MGF)
Let X be a random variable with cumulative distribution function FX (x). The moment generating
function (mgf) of X, denoted by MX (t), is defined provided that the expectation exists for t in
some neighborhood of 0. That is, there exists h > 0 such that, for all t in −h < t < h, E(etX )
exists.
X
MX (t) = E[etX ] = etx pX (x)
all x
Properties of the Moment Generating Function
• If MX (t) exists in a neighborhood around t = 0, then all moments of X exist.
• The k-th moment of X about the origin can be obtained by differentiating MX (t) k times
and evaluating at t = 0:
(k)
µk = E[X k ] = MX (0)
(k)
where MX (0) denotes the k-th derivative of MX (t) at t = 0.
Example
Suppose X is a discrete random variable with probability mass function pX (x). Then the MGF
is: X
MX (t) = E[etX ] = etx pX (x)
x
If X is continuous with probability density function fX (x), the MGF is:
Z ∞
MX (t) = E[etX ] = etx fX (x) dx
−∞
1
Using MGFs to Find Moments
• The MGF can be used to compute moments of X.
• For the k-th moment about the origin:
(k)
µk = E[X k ] = MX (0)
• For example, the first moment (mean) is
′
E[X] = MX (0)
and the second central moment (variance) is
′′ ′
Var(X) = E[(X − E[X])2 ] = MX (0) − (MX (0))2
MGFs and Sums of Independent Random Variables
• Let X1 , X2 , . . . , Xn be independent random variables with MGFs MX1 (t), MX2 (t), . . . , MXn (t).
• Then the MGF of the sum S = X1 + X2 + · · · + Xn is:
MS (t) = MX1 (t) · MX2 (t) · · · · · MXn (t)
• This property allows us to determine the distribution of sums of independent random
variables using their MGFs.
MGFs of Common Distributions
The moment generating function (MGF) can also be used to understand the behavior of common
probability distributions. Here are some important examples:
1. Exponential Distribution
Let X ∼ Exp(λ), with probability density function (pdf) fX (x) = λe−λx , x > 0.
The MGF is: Z ∞
λ
MX (t) = E[etX ] = etx λe−λx dx = , t<λ
0 λ−t
Notes:
• The condition t < λ ensures the integral converges.
• Differentiating MX (t) gives the moments:
′ 1 ′′ ′ 1
E[X] = MX (0) = , Var(X) = MX (0) − (MX (0))2 =
λ λ2
2. Poisson Distribution
e−λ λx
Let X ∼ Poisson(λ) with probability mass function (pmf) P (X = x) = x! , x = 0, 1, 2, . . .
The MGF is:
∞
X e−λ λx t
tX
MX (t) = E[e ] = etx = eλ(e −1)
x!
x=0
Notes:
• Differentiating gives the mean and variance: E[X] = λ and Var(X) = λ.
2
3. Normal Distribution
2 2
Let X ∼ N (µ, σ 2 ) with pdf fX (x) = √ 1 e−(x−µ) /(2σ ) .
2πσ 2
The MGF is:
tX 1 2 2
MX (t) = E[e ] = exp µt + σ t
2
Notes:
• This compact form makes it easy to compute moments:
E[X] = µ, Var(X) = σ 2
• The MGF of a sum of independent normal random variables is also normal.
Summary of MGFs for Common Distributions
Distribution Parameter(s) MGF MX (t) Mean & Variance
λ
Exponential λ>0 MX (t) = , t<λ E[X] = λ1 , Var(X) = 1
λ2
λ−t
Poisson λ>0 MX (t) = exp{λ(et − 1)} E[X] = λ, Var(X) = λ
Normal µ, σ 2 MX (t) = exp{µt + 12 σ 2 t2 } E[X] = µ, Var(X) = σ 2
Binomial n, p MX (t) = (1 − p + pet )n E[X] = np, Var(X) = np(1 − p)
pet
Geometric p MX (t) = 1−(1−p)e t , t < − ln(1 − p) E[X] = p1 , Var(X) = 1−p
p2
Table 1: MGFs and moments of some common distributions
3
Examples of MGFs for Common Distributions
1. Exponential Distribution
Let X ∼ Exp(λ). The pdf is fX (x) = λe−λx , x > 0, and the MGF is:
λ
MX (t) = , t<λ
λ−t
Example 1: Find the mean and variance of X ∼ Exp(2)
Let X ∼ Exp(2) with MGF:
2
MX (t) = , t<2
2−t
We will find the mean, second moment, and variance step by step.
Step 1: First Derivative → Mean
The mean can be obtained from the first derivative of the MGF:
′
E[X] = MX (0)
Step 1A: Differentiate MX (t)
MX (t) = 2(2 − t)−1
Using the power rule:
d −n
[u ] = −nu−n−1 · u′ , u = 2 − t, n = 1, u′ = −1
dt
′
MX (t) = 2 · (−1)(2 − t)−2 · (−1) = 2(2 − t)−2
Step 1B: Evaluate at t = 0
′ 1 1
E[X] = MX (0) = 2(2 − 0)−2 = 2 · =
4 2
So the mean is E[X] = 12 .
Step 2: Second Derivative → Second Moment
The second moment is obtained from the second derivative:
′′
E[X 2 ] = MX (0)
′ (t)
Step 2A: Differentiate MX
′
MX (t) = 2(2 − t)−2
Again, using the power rule:
d −2
[u ] = −2u−3 · u′ = −2(2 − t)−3 · (−1) = 2(2 − t)−3
dt
Multiply by 2:
′′
MX (t) = 4(2 − t)−3
Step 2B: Evaluate at t = 0
′′ 4 1
E[X 2 ] = MX (0) = 4(2)−3 = =
8 2
So the second moment is E[X 2 ] = 21 .
4
Step 3: Compute Variance
2
2 1 2 1 1 1 1
Var(X) = E[X ] − (E[X]) = − = − =
2 2 2 4 4
✓ Variance is Var(X) = 41 .
Step 4: Summary Table
Quantity Formula Value
Mean E[X] = MX ′ (0) 1/2
Second moment 2
E[X ] = MX ′′ (0) 1/2
Variance Var(X) = E[X 2 ] − (E[X])2 1/4
Example 2: Using the MGF of an Exponential Distribution
Let X ∼ Exp(5) with MGF:
5
MX (t) = , t<5
5−t
Step 1: First Derivative → Mean
The mean is obtained from the first derivative of the MGF:
′
E[X] = MX (0)
Step 1A: Differentiate MX (t)
MX (t) = 5(5 − t)−1
Using the power rule:
d −1
[u ] = −1 · u−2 · u′ , u = 5 − t, u′ = −1
dt
′
MX (t) = 5 · (−1)(5 − t)−2 · (−1) = 5(5 − t)−2
Step 1B: Evaluate at t = 0
′ 1 1
E[X] = MX (0) = 5(5 − 0)−2 = 5 · =
25 5
So the mean is E[X] = 15 .
Step 2: Second Derivative → Second Moment
The second moment is obtained from the second derivative of the MGF:
′′
E[X 2 ] = MX (0)
′ (t)
Step 2A: Differentiate MX
′
MX (t) = 5(5 − t)−2
Using the power rule again:
d −2
[u ] = −2u−3 · u′ , u = 5 − t, u′ = −1
dt
5
′′
MX (t) = 5 · (−2)(5 − t)−3 · (−1) = 10(5 − t)−3
Step 2B: Evaluate at t = 0
′′ 1 2
E[X 2 ] = MX (0) = 10(5)−3 = 10 · =
125 25
2
So the second moment is E[X 2 ] = 25 .
Step 3: Compute Variance
2
2 2 2 1 2 1 1
Var(X) = E[X ] − (E[X]) = − = − =
25 5 25 25 25
1
Variance is Var(X) = 25 .
Step 4: Summary Table
Quantity Formula Value
Mean E[X] = MX ′ (0) 1/5
Second moment 2
E[X ] = MX ′′ (0) 2/25
Variance Var(X) = E[X ] − (E[X])2
2 1/25
—
2. Poisson Distribution
Example 1: Using the MGF of a Poisson Distribution
Let X ∼ Poisson(λ) with pmf:
e−λ λx
P (X = x) =
, x = 0, 1, 2, . . .
x!
The moment generating function (MGF) is:
MX (t) = exp{λ(et − 1)}
We will find the mean and variance for X ∼ Poisson(3)
Step 1: First Derivative → Mean
The mean is obtained from the first derivative of the MGF:
′
E[X] = MX (0)
Step 1A: Differentiate MX (t)
t −1)
MX (t) = e3(e
Using the chain rule:
d u
[e ] = eu · u′ , u = 3(et − 1), u′ = 3et
dt
′ t −1) t −1)
MX (t) = e3(e · 3et = 3et · e3(e
Step 1B: Evaluate at t = 0
′ 0 −1)
E[X] = MX (0) = 3 · e0 · e3(e = 3 · 1 · e0 = 3
So the mean is E[X] = 3.
6
Step 2: Second Derivative → Second Moment
The second moment is obtained from the second derivative:
′′
E[X 2 ] = MX (0)
′ (t)
Step 2A: Differentiate MX
′ t −1)
MX (t) = 3et · e3(e
Use the product rule:
d t −1)
[f · g] = f ′ g + f g ′ , f = 3et , g = e3(e
dt
t −1)
f ′ = 3et , g ′ = 3et · e3(e
t −1)
t
t t
′′
MX (t) = f ′ g + f g ′ = 3et · e3(e + 3et · 3et · e3(e −1) = 3et e3(e −1) + 9e2t e3(e −1)
′′ t −1)
(t) = e3(e 3et + 9e2t
MX
Step 2B: Evaluate at t = 0
′′
E[X 2 ] = MX (0) = e3(1−1) 3 · 1 + 9 · 12 = 3 + 9 = 12
Step 3: Compute Variance
Var(X) = E[X 2 ] − (E[X])2 = 12 − 32 = 12 − 9 = 3
So the variance is Var(X) = 3.
Step 4: Summary Table
Quantity Formula Value
Mean E[X] = MX ′ (0) 3
Second moment 2
E[X ] = MX ′′ (0) 12
Variance Var(X) = E[X 2 ] − (E[X])2 3
Example (General): X ∼ Poisson(λ)
Let X ∼ Poisson(λ) with pmf
e−λ λx
P (X = x) = , x = 0, 1, 2, . . .
x!
The MGF is
MX (t) = exp{λ(et − 1)}.
We compute the mean and variance from derivatives of the MGF.
7
Step 1: First derivative → Mean
′
E[X] = MX (0).
Differentiate MX (t). Let u(t) = λ(et − 1), so MX (t) = eu(t) and u′ (t) = λet . By the chain
rule,
′ t
MX (t) = eu(t) · u′ (t) = λet eλ(e −1) .
Evaluate at t = 0:
′ 0 −1)
E[X] = MX (0) = λe0 eλ(e = λ · 1 · e0 = λ.
Step 2: Second derivative → Second moment
′′
E[X 2 ] = MX (0).
′ (t) = λet eλ(e t −1)
Differentiate MX using the product rule:
t −1)
t
t
′′
(t) = λet eλ(e + λet λet eλ(e −1) = eλ(e −1) λet + λ2 e2t .
MX
Evaluate at t = 0:
′′
E[X 2 ] = MX (0) = eλ(1−1) λ · 1 + λ2 · 1 = λ + λ2 .
Step 3: Variance
Var(X) = E[X 2 ] − (E[X])2 = (λ + λ2 ) − λ2 = λ.
Summary
Quantity Value
Mean E[X] = λ
Second moment E[X 2 ] = λ + λ2
Variance Var(X) = λ
Example 3: X ∼ Poisson(5), find mean
t −1)
1. MGF: MX (t) = e5(e
′ (0) = 5
2. Mean: E[X] = MX
3. Normal Distribution
Example: X ∼ N (2, 4)
Let X ∼ N (µ, σ 2 ). The MGF of a normal distribution is
1 2 2
MX (t) = exp µt + σ t .
2
For this example, µ = 2 and σ 2 = 4. So the MGF becomes
1
MX (t) = exp 2t + (4)t = exp 2t + 2t2 .
2
2
We now find the mean and variance from derivatives of the MGF.
8
Step 1: First derivative → mean
The mean is
′
E[X] = MX (0).
Differentiate:
′ d h 2t+2t2 i
MX (t) = e .
dt
Apply the chain rule. Let u(t) = 2t + 2t2 . Then MX (t) = eu(t) and
u′ (t) = 2 + 4t.
So,
′ 2
MX (t) = u′ (t) eu(t) = (2 + 4t) e 2t+2t .
Evaluate at t = 0:
′
E[X] = MX (0) = (2 + 0) e0 = 2.
Step 2: Second derivative → second moment
′′
E[X 2 ] = MX (0).
2
′ (t) = (2 + 4t)e 2t+2t using the product rule:
Differentiate MX
′′ 2 2
MX (t) = 4e 2t+2t + (2 + 4t)(2 + 4t)e 2t+2t .
Factor out the exponential:
′′ 2
(t) = e 2t+2t 4 + (2 + 4t)2 .
MX
Evaluate at t = 0:
′′
E[X 2 ] = MX (0) = e0 4 + 22 = 4 + 4 = 8.
Step 3: Variance
Var(X) = E[X 2 ] − (E[X])2 = 8 − 22 = 8 − 4 = 4.
Summary
Quantity Value
Mean E[X] = 2
Second moment E[X 2 ] = 8
Variance Var(X) = 4
Example: X ∼ N (0, 1)
Let X ∼ N (µ, σ 2 ). The MGF of a normal distribution is
1 2 2
MX (t) = exp µt + σ t .
2
For this example, µ = 0 and σ 2 = 1. So the MGF becomes
2
1 2 t
MX (t) = exp 0 · t + · 1 · t = exp .
2 2
We now find the mean and variance using derivatives of the MGF.
9
Step 1: First derivative → mean
The mean is
′
E[X] = MX (0).
Differentiate:
′ d h t2 /2 i
MX (t) = e .
dt
Let u(t) = t2 /2. Then MX (t) = eu(t) and
u′ (t) = t.
So by the chain rule:
′ 2 /2
MX (t) = u′ (t)eu(t) = t et .
Evaluate at t = 0:
′
E[X] = MX (0) = 0 · e0 = 0.
Step 2: Second derivative → second moment
The second moment is
′′
E[X 2 ] = MX (0).
′ (t) = tet2 /2
Differentiate MX using the product rule:
2
2
′′
MX (t) = 1 · et /2 + t t et /2 .
Simplify:
′′ 2 /2
(t) = et 1 + t2 .
MX
Evaluate at t = 0:
′′
E[X 2 ] = MX (0) = e0 (1 + 0) = 1.
Step 3: Variance
Var(X) = E[X 2 ] − (E[X])2 = 1 − 02 = 1.
Summary
Quantity Value
Mean E[X] = 0
Second moment E[X 2 ] = 1
Variance Var(X) = 1
10