Moment generating functions (mgf)
Reference: 1. Meyer PL. Introductory probability and statistical applications. Oxford and IBH Publishing; 1965.
2. Johnson, Richard A., Irwin Miller, and John E. Freund. "Probability and statistics for engineers." (2000).
NOTE:
1. 𝐸(𝑋) is coefficient of t In 𝑀𝑋 (𝑡).
𝑡2
2. 𝐸(𝑋 2 ) is coefficient of In 𝑀𝑋 (𝑡).
2!
𝑡𝑛
3. 𝐸(𝑋 𝑛 ) is coefficient of In 𝑀𝑋 (𝑡).
𝑛!
4. 𝐸(𝑋 𝑛 ) = 𝑀𝑋𝑛 (0)
Example:
𝑒 −|𝑥|
1. Suppose that X has pdf 𝑓(𝑥) = , −∞ < 𝑥 < ∞ then find E(X) and V(X) using mgf.
2
Solution:
∞
𝑀𝑋 (𝑡) = ∫ 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥
−∞
∞ 𝑒 −|𝑥|
= ∫−∞ 𝑒 𝑡𝑥 2 𝑑𝑥
0 𝑒𝑥 ∞ 𝑒− 𝑥 0 𝑒 (1+𝑡)𝑥 ∞ 𝑒 −(1−𝑡)𝑥
= ∫−∞ 𝑒 𝑡𝑥 2 𝑑𝑥 + ∫0 𝑒 𝑡𝑥 2 𝑑𝑥= ∫−∞ 2
𝑑𝑥 + ∫0 2
𝑑𝑥
1
𝑀𝑋 (𝑡) = , -1< t <1
1−𝑡 2
𝑀𝑋1 (0) = 0
𝑀𝑋2 (0) = 2
E(X)= 0 and V(X)=2.
1 2𝑡 2
Or: 𝑀𝑋 (𝑡) = 1−𝑡 2 = 1 + 𝑡 2 + 𝑡 4 + 𝑡 6 + ⋯ = 1 + + 𝑡4 + 𝑡6 + ⋯
2
𝑡2
Cofficient of “t” E(X)= 0. Cofficient of “ 2 ” V(X)= 2-0=2
2. Let X be a random variable taking the values 0,1,2,… and 𝑓(𝑥) = 𝑎𝑏 𝑥 , 𝑎, 𝑏 > 0 & a+b=1. Find mgf
of X. If 𝐸(𝑋) = 𝑚1, 𝐸(𝑋 2 ) = 𝑚2 then S.T 𝑚2= 𝑚1 (2𝑚1 + 1).
Solution:
1 𝑎
𝑀𝑋 (𝑡) = ∑∞ 𝑥 𝑡𝑥
0 𝑎𝑏 𝑒 = 𝑎 ∑∞ 𝑡 𝑥
0 (𝑏𝑒 ) = a 1−𝑏𝑒 𝑡 = 1−𝑏𝑒 𝑡
𝑎𝑏
𝐸(𝑋) = 𝑀𝑋1 (0) =
(1 − 𝑏)2
(1 + 𝑏)𝑎𝑏
𝐸(𝑋 2 ) = 𝑀𝑋2 (0) =
(1 − 𝑏)3
Given,
𝐸(𝑋) = 𝑚1, 𝐸(𝑋 2 ) = 𝑚2
To Prove, 𝑚2= 𝑚1 (2𝑚1 + 1).
𝑎𝑏 𝑎𝑏 𝑎𝑏 2𝑎𝑏+1+𝑏 2 −2𝑏
Consider, 𝑚1 (2𝑚1 + 1) = (1−𝑏)2
(2 (1−𝑏)2 + 1) =(1−𝑏) 2
( (1−𝑏)2
)
𝑎𝑏 𝑎𝑏
= (1−𝑏)4 (2𝑏(𝑎 − 1) + 1 + 𝑏 2 )= = (1−𝑏)4 (2𝑏(−𝑏) + 1 + 𝑏 2 )= 𝑚2 .
Moment generating function for Gamma Distribution
∞ 𝛼𝑟 ∞
𝑀𝑋 (𝑡) = ∫−∞ 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥 = Γ(𝑟) ∫0 𝑒 −(𝛼−𝑡)𝑥 𝑥 𝑟−1 𝑑𝑥
𝑑𝑣
Substitute, 𝑥(𝛼 − 𝑡) = 𝑣 then 𝑑𝑥 = 𝛼−𝑡
𝛼𝑟 ∞ 𝑣 𝑟−1 𝑑𝑣
𝑀𝑋 (𝑡) = Γ(𝑟) ∫0 𝑒 −𝑣 (𝛼−𝑡) 𝛼−𝑡
𝛼𝑟 ∞ −𝑣 𝑟−1
= 𝑟 ∫0
𝑒 (𝑣) 𝑑𝑣
Γ(𝑟)(𝛼−𝑡)
𝛼𝑟 𝛼 𝑟
=Γ(𝑟)(𝛼−𝑡)𝑟 Γ(𝑟) = (𝛼−𝑡)
𝑟
𝐸(𝑋) =
𝛼
2 )= 𝑟(𝑟+1)
𝐸(𝑥 2
𝛼
𝑟
V(X)=
𝛼2
Moment generating function for Exponential Distribution
Note: When we sub r=1 in gamma distribution we get exponential distribution.
∞ ∞
𝑀𝑋 (𝑡) = ∫0 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥 = 𝜆 ∫0 𝑒 −(𝜆−𝑡)𝑥 𝑑𝑥
𝜆
𝑀𝑋 (𝑡) = (𝜆−𝑡), 𝑡 < 𝜆
1
𝐸(𝑋) =
𝜆
1
𝑉(𝑋) = 2
𝜆
Or:
Moment generating function for chi square Distribution
𝒏 𝟏
Special case of Gamma distribution: 𝒓 = and 𝜶 = in 𝚪 function we get 𝝌𝟐 distribution.
𝟐 𝟐
A continuous random variable X is said to have a chi-square distribution if its PDF is given by.
∞
𝑀𝑋 (𝑡) = ∫ 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥
−∞
1 𝑛
𝛼𝑟 ∞ (𝑡− )𝑥
= 𝑛
𝑛 ∫−∞ 𝑒 2 𝑥 2 −1 𝑑𝑥
Γ( )2 2
2
𝑛
𝑀𝑋 (𝑡) = (1 − 2𝑡)− 2
𝐸(𝑋) =n and V(X)= 2n
Moment generating function for Uniform Distribution
𝟏
𝒂≤𝒙≤𝒃
𝒇(𝒙) = {(𝒃 − 𝒂)
𝟎 ∞ 𝒆𝒍𝒔𝒆 𝒘𝒉𝒆𝒓𝒆
𝑀𝑋 (𝑡) = ∫ 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥
−∞
1 ∞
= ∫ 𝑒 𝑡𝑥 𝑑𝑥
𝑏−𝑎 −∞
1
= 𝑡(𝑏−𝑎) (𝑒 𝑏𝑡 − 𝑒 𝑎𝑡 )
Expand and get the suitable coefficient in the expanssion to find E(X) and V(X).
(𝒂+𝒃)
Mean E(X) = 𝟐
𝟐 (𝒂𝟐 + 𝒃𝟐 +𝒂𝒃)
E(𝑿 ) = 𝟑
(𝒃−𝒂)𝟐
Variance V(X) = E(𝑿𝟐 ) – [𝑬(𝑿)]𝟐 = 𝟏𝟐
2
[𝑥−(𝑡σ2 +μ] ∞ 2 1
By substituting, 𝑣 = [ ] 𝑎𝑛𝑑 𝑑𝑥 = 𝜎 2 𝑑𝑣 and ∫−∞ 𝑒 −𝑥 𝑑𝑥 = Γ(2) = √𝜋
𝜎2
Solution:
Properties of mgf
Reproductive Properties of mgf
Problems:
1. Find the mgf of random variable X which is uniformly distributed with an interval
(-a, a) and hence find 𝐸(𝑋 2𝑛 ).
𝑒 𝑎𝑡 −𝑒 −𝑎𝑡 1
Solution: 𝑋~ 𝑈(−𝑎, 𝑎) = we know, 𝑀𝑋 (𝑡) = 𝑡(𝑏−𝑎) (𝑒 𝑏𝑡 − 𝑒 𝑎𝑡 )
(𝑎+𝑎)𝑡
𝑒 𝑎𝑡 −𝑒 −𝑎𝑡
= (2𝑎)𝑡
, by expanding
𝑡 2𝑛 𝑎2𝑛
𝐸(𝑋 2𝑛 ) = 𝑐𝑜𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑡 𝑜𝑓 =
(2𝑛)! (2𝑛+1)
2. If X is normally distributed with mean 𝜇 and variance, 𝜎 2 then show that
𝐸(𝑋 − 𝜇)2𝑛 = 1.3.5 … . (2𝑛 − 1)𝜎 2𝑛 .
𝜎2 𝑡 2
2 ), 𝜇𝑡+
Solution: Given 𝑋~𝑁(𝜇, 𝜎 𝑀𝑋 (𝑡) = 𝑒 2
Let Y= (𝑋 − 𝜇)
𝑡 2𝑛
To get E(𝑌 2𝑛 )= The coefficient of (2𝑛)! in 𝑀𝑌 (𝑡).
𝑀𝑌 (𝑡) = 𝐸 (𝑒 𝑡𝑦 )= 𝐸(𝑒 𝑡(𝑥−𝜇) )
= 𝐸(𝑒 𝑡𝑥 )𝐸(𝑒 −𝜇𝑡 )
= 𝑀𝑋 (𝑡) 𝐸(𝑒 −𝜇𝑡 )
𝜎2 𝑡2 𝜎2 𝑡2 2 3
𝜎2𝑡 2 1 𝜎2𝑡 2 1 𝜎2𝑡 2
𝑀𝑌 (𝑡) = 𝑒 𝜇𝑡+ 2 𝑒 −𝜇𝑡 = 𝑒 2 =1+( )+ ( ) + 3! ( ) +⋯
2 2! 2 2
𝑡 2𝑛 𝜎 2𝑛 (2𝑛)! (2𝑛)!
E(𝑌 2𝑛 )= The coefficient of (2𝑛)! in 𝑀𝑌 (𝑡)= = * 𝜎 2𝑛
2𝑛 𝑛! 2𝑛 𝑛!
(2𝑛)(2𝑛−1)(2𝑛−2)…(3)(2)(1)
= * 𝜎 2𝑛
(2𝑛)(2𝑛−2)(2𝑛−4)…(6)(4)(2)
= 1.3.5 … . (2𝑛 − 1)𝜎 2𝑛 .
3. Let 𝑋1 ~𝜒 2 (3), 𝑋2 ~𝜒 2 (5) and Z= 𝑋1 + 𝑋2 where 𝑋1 𝑎𝑛𝑑 𝑋2 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡 random
variables. Find 𝑀𝑍 (𝑡) and 𝑉(𝑍) 𝑎𝑛𝑑 𝑝𝑑𝑓 𝑜𝑓 𝑍.
Solution:
3
𝑋1 ~𝜒 2 (3) gives 𝑀𝑋1 (𝑡) = (1 − 2𝑡) − 2
5
𝑋2 ~𝜒 2 (5) gives 𝑀𝑋2 (𝑡) = (1 − 2𝑡) – 2
8
𝑀𝑍 (𝑡) = 𝑀𝑋1 (𝑡)𝑀𝑋2 (𝑡) = (1 − 2𝑡) − 2 ~𝜒 2 (8)
Therefore, 𝑉(𝑍) = 2𝑛 = 2 (8) = 16. Where 𝑛 = 8.
𝑛 𝑍 𝑍
−1 − −
𝑍2 𝑒 2 𝑍3𝑒 2
, 𝑍>0
f(Z)= { 2𝑛2 Γ(𝑛/2) = { 24Γ(4) , 𝑍 > 0
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
4. Let 𝑋1 , 𝑋2 𝑎𝑛𝑑 𝑋3 𝑎𝑟𝑒 3 independent random variable having normal distributions
with parameter (4, 1), (5,2), (7, 3) respectively. Let 𝑌 = 2𝑋1 + 2𝑋2 + 𝑋3 . Find the pdf
𝑌−𝜇 2
of 𝑉 = ( ) , where 𝜇 𝑎𝑛𝑑 𝜎 are the mean and standard deviation of Y.
𝜎
Solution:
𝜎2𝑡 2
𝜇𝑡+
𝑀𝑋 (𝑡) = 𝑒 2
𝑌~𝑁(2 ∗ 4 + 2 ∗ 5 + 1 ∗ 7, 2 ∗ 1 + 22 ∗ 2 + 1 ∗ 3) 2
𝑌~𝑁(25, 15)
15𝑡 2
25𝑡+
𝑀𝑌 (𝑡) = 𝑒 2
We have,
𝑌−25 2
Therefore,𝑉 = ( ) = 𝑍 2 ~𝜒 2 (1)
√15
𝑣 1 𝑣 1
− − − −
𝑒 2 𝑣 2 𝑒 2 𝑣 2
1 , 𝑣>0 , 𝑣>0
f(V)= { 1
22 Γ( ) ={ √𝜋 2 2
1
2
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
3
5. If a random variable X has mgf 𝑀𝑋 (𝑡) = then find the standard deviation of the
3−𝑡
random variable X.
Solution:
3 3 1 𝑡 −1 𝑡 𝑡 2
𝑀𝑋 (𝑡) = = 𝑡 = 𝑡 = (1 − ) =1+ +( ) +⋯
3−𝑡 3(1−3) (1−3) 3 3 3
1
𝐸(𝑋) = 𝑐𝑜𝑒𝑓 𝑜𝑓 𝑡 𝑖𝑛 𝑀𝑋 (𝑡) =
3
𝑡2
2)
1 2
𝐸(𝑋 = 𝑐𝑜𝑒𝑓 𝑜𝑓 𝑖𝑛 𝑀𝑋 (𝑡) = 2! =
2! 9 9
1 1
𝑉(𝑋) = . 𝐻𝑒𝑛𝑐𝑒, 𝜎 =
9 3
6. Let X be random variable having probability mass function p(x=k) = p(1 − 𝑝)𝑘−1 ,
𝑘 = 1,2,3 … 𝑛. Find 𝑀𝑋 (𝑡) and V(X).
Solution:
𝑀𝑋 (𝑡) = ∑𝑛1 𝑒 𝑡𝑥 𝑝(𝑋 = 𝑥)
= ∑𝑛1 𝑒 𝑡𝑥 p(1 − 𝑝)𝑥−1
𝑝
= ∑𝑛1 𝑒 𝑡𝑥 (1 − 𝑝)𝑥
𝑝−1
𝑝
= ∑𝑛1(𝑒 𝑡 (1 − 𝑝))𝑥 , Expanding
1−𝑝
𝑝
= {(𝑒 𝑡 (1 − 𝑝)) + (𝑒 𝑡 (1 − 𝑝))2 + ⋯ }
1−𝑝
𝑝
= (𝑒 𝑡 (1 − 𝑝)){1 + (𝑒 𝑡 (1 − 𝑝)) + (𝑒 𝑡 (1 − 𝑝))2 + ⋯ }
1−𝑝
𝑝 𝑡
1 𝑝𝑒 𝑡
𝑀𝑋 (𝑡) = ∗ (𝑒 (1 − 𝑝)) ∗ =
1−𝑝 1 − 𝑒 𝑡 (1 − 𝑝) 1 − 𝑒 𝑡 (1 − 𝑝)
𝑝𝑒 𝑡
𝑀𝑋1 (𝑡) = [1−𝑒 𝑡(1−𝑝)]2,
1
at t=o, E(X)=
𝑝
and
1 2(1 − 𝑝)
𝐸 (𝑋 2 ) =
+
𝑝 𝑝2
1−𝑝
𝑉(𝑋) = 2
𝑝
7. If X has pdf f(x)= 𝜆𝑒 −𝜆(𝑥−𝑎) , 𝑋 ≥ 𝑎, find the mgf of X and hence find V(X).
Ans: Given is an exponential distribution.
∞
𝑀𝑋 (𝑡) = ∫ 𝑒 𝑡𝑥 𝑓(𝑥) 𝑑𝑥
−∞
∞
= ∫ 𝑒 𝑡𝑥 𝜆𝑒 −𝜆(𝑥−𝑎) 𝑑𝑥
𝑎
∞
= 𝜆 ∫ 𝑒 𝑡𝑥 𝑒 −𝜆(𝑥−𝑎) 𝑑𝑥
𝑎
𝜆𝑎 ∞
= 𝜆𝑒 ∫𝑎 𝑒 −(𝜆−𝑡)𝑥 𝑑𝑥
𝜆𝑎
𝑒 −(𝜆−𝑡)𝑥
= 𝜆𝑒
−(𝜆 − 𝑡)
𝑒 −(𝜆−𝑡)𝑎 𝜆𝑒 𝑎𝑡
= 𝜆𝑒 𝜆𝑎 (𝜆−𝑡)
= ,𝜆>𝑡
𝜆−𝑡
1 1
E(X)= 𝑎 + and V(X)=
𝜆 𝜆2
𝑡
8. If the mgf of discrete random variable is 𝑒 4(𝑒 −1) then find P(𝑋 = 𝜇 + 𝜎) where 𝜇 and
𝜎 are the mean and S.D of X.
𝑡
Solution: 𝑀𝑋 (𝑡) = 𝑒 4(𝑒 −1)
𝑋~𝑝(𝛼) where 𝛼 = 4
Therefore E(X)= V(X)= 4,
Which gives 𝜇 = 4 𝑎𝑛𝑑 𝜎 = 2.
𝑒 −𝛼 𝛼 𝑘
P(X=k)= , k= 0,1,2,….,n
𝑘!
For, 𝛼 = 4,
𝑒 −4 4𝑘
P(X=k)=
𝑘!
𝑒 −4 46
To find P(𝑋 = 𝜇 + 𝜎)= P(𝑋 = 6)= = 0.1042.
6!