NPTEL- Probability and Distributions

MODULE 7
LIMITING DISTRIBUTIONS
LECTURE 40
Topics
7.1 CONVERGENCE IN DISTRIBUTION AND
PROBABILITY
7.1.1 Poisson Approximation to Binomial distribution
7.2 THE WEAK LAW OF LARGE NUMBERS (WLLN)
AND THE CENTRAL LIMIT THEOREM (CLT)
7.2.1 Random Walk
7.2.2 Justification of Relative Frequency Method of Assigning Probabilities

Proposition 1.1

Let 𝑐𝑛 𝑛≥1 be a sequence of positive real numbers such that lim𝑛→∞ 𝑐𝑛 = 𝑐 ∈ ℝ. Then
𝑐𝑛 𝑛
lim 1 + = 𝑒𝑐 .
𝑛→∞ 𝑛
Proof. We know that

𝑥2
𝑥− ≤ ln 1 + 𝑥 ≤ 𝑥, ∀𝑥 > 0
2
𝑐𝑛 2 𝑐𝑛
⇒ 𝑐𝑛 − ≤ 𝑛 ln 1 + ≤ 𝑐𝑛 , 𝑛 = 1, 2, …
2𝑛 𝑛
𝑐𝑛
⇒ lim 𝑛 ln 1 + =𝑐 (on taking limits on both sides)
𝑛 →∞ 𝑛
𝑐𝑛 𝑛
⇒ lim ln 1 + =𝑐
𝑛→∞ 𝑛
𝑐𝑛 𝑛
⇒ lim 1 + = 𝑒𝑐 . ▄
𝑛→∞ 𝑛

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 1

NPTEL- Probability and Distributions

7.1.1 Poisson Approximation to Binomial distribution

Example 1.13

Let 𝑋𝑛 ~ Bin 𝑛, 𝜃𝑛 , where 𝜃𝑛 ∈ 0, 1 , 𝑛 = 1, 2, …, and let lim 𝑛𝜃𝑛 = 𝜃 > 0. Show

𝑛→∞
𝑑
that 𝑋𝑛 → 𝑋, as 𝑛 → ∞, where 𝑋 ~ P 𝜃 , the Poisson distribution with mean 𝜃.

Solution. Note that the m.g.f. of 𝑋 is

𝑡 −1)
𝑀 𝑡 = 𝑒 𝜃(𝑒 , 𝑡 ∈ ℝ,

and the m.g.f. of 𝑋𝑛 is

𝑀𝑛 𝑡 = 1 − 𝜃𝑛 + 𝜃𝑛 𝑒 𝑡 𝑛

𝑛
𝑐𝑛 𝑡
= 1+ , 𝑡 ∈ ℝ,
𝑛

where 𝑐𝑛 𝑡 = 𝑛𝜃𝑛 𝑒 𝑡 − 1 , 𝑡 ∈ ℝ, 𝑛 = 1, 2, …. Clearly lim 𝑐𝑛 𝑡 = 𝜃 𝑒 𝑡 − 1 , ∀𝑡 ∈ ℝ.

𝑛→∞
Now using Proposition 1.1 we get

𝑒 𝑡 −1
lim 𝑀𝑛 𝑡 = 𝑒 𝜃 = 𝑀 𝑡 , ∀𝑡 ∈ ℝ.
𝑛→∞

𝑑
Using Theorem 1.5(i) we conclude that 𝑋𝑛 → 𝑋~ 𝑃 𝜃 , as 𝑛 → ∞. ▄

7.2 THE WEAK LAW OF LARGE NUMBERS (WLLN)

AND THE CENTRAL LIMIT THEOREM (CLT)
1 𝑛
Let 𝑋𝑛 𝑛≥1 be a sequence of i.i.d. random variables and let 𝑋𝑛 = 𝑛 𝑖=1 𝑋𝑖 , 𝑛 = 1,2, …,
be the corresponding sequence sample means. In this section we will study the
convergence behavior of the sequence 𝑋𝑛 𝑛≥1 of sample means.

Theorem 2.1
1 𝑛
Let 𝑋𝑛 𝑛≥1 be a sequence of i.i.d. random variables and let 𝑋𝑛 = 𝑛 𝑖=1 𝑋𝑖 , 𝑛 = 1,2, ….

𝑝
(i) (WLLN) Suppose that 𝐸 𝑋1 = 𝜇 is finite. Then 𝑋𝑛 → 𝜇, as 𝑛 → ∞.
(ii) (CLT) suppose that 0 < Var 𝑋1 = 𝜎 2 < ∞. Then
𝑛 𝑋𝑛 − 𝜇 𝑑
𝑍𝑛 ≝ → 𝑍 ~ 𝑁 0,1 , as 𝑛 → ∞.
𝜎

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 2

NPTEL- Probability and Distributions

Proof.

(i) As the proof for the case Var 𝑋1 = ∞ is quite involved, for simplicity, we assume
that Var 𝑋1 = 𝜎 2 < ∞. Then
𝑛 𝑛
1 1
𝐸 𝑋𝑛 =𝐸 𝑋𝑖 = 𝐸 𝑋𝑖 = 𝐸 𝑋1 = 𝜇
𝑛 𝑛
𝑖=1 𝑖=1
and
𝑛 𝑛
1 1 𝜎 2 𝑛→∞
Var 𝑋𝑛 = Var 𝑋𝑖 = 2 Var 𝑋𝑖 = 0.
𝑛 𝑛 𝑛
𝑖=1 𝑖=1

𝑝
Using Theorem 1.4 it follows that 𝑋𝑛 → 𝜇, as 𝑛 → ∞.

(ii) For simplicity we will assume that the common m.g.f. 𝑀 ∙ of 𝑋1 , 𝑋2 , … is finite in
an interval – 𝑎, 𝑎 for some 𝑎 > 0. Then, by Theorem 3.4, Module 3, 𝜇𝑟′ = 𝐸 𝑋1𝑟
𝑑𝑟
is finite for each 𝑟 ∈ 1, 2, ⋯ and 𝜇𝑟′ = 𝐸 𝑋1𝑟 = 𝑀 𝑟
0 = 𝑀 𝑡 ,𝑟 =
𝑑𝑡 𝑟 𝑡=0
𝑋 𝑖 −𝜇
1, 2, …. Let 𝑌𝑖 = , 𝑖 = 1, … , 𝑛. Then 𝑌1 , 𝑌2 , … are i.i.d. random variables with
𝜎
mean 0 and variance 1. Let 𝑀𝑌 ∙ denote the common m.g.f. of 𝑌1 , 𝑌2 , …, so that
𝜇𝑡 𝑡
𝑀𝑌 𝑡 = 𝑒 − 𝜎 𝑀 , −𝑎𝜎 < 𝑡 < 𝑎𝜎,
𝜎

𝜇 𝑀 1 (0)
𝑀𝑌1 0 = − + = 0 = 𝐸 𝑌1
𝜎 𝜎
𝜇 2 2𝜇 1
and 𝑀𝑌2 0 = 𝑀 0 − 𝑀 1
0 + 𝑀 2
0 = 1 = 𝐸 𝑌12 .
𝜎 𝜎2 𝜎2
Let 𝜓2 : −𝑎𝜎, 𝑎𝜎 → ℝ be such that

𝑡2
𝑀𝑌 𝑡 = 𝑀𝑌 0 + 𝑡𝑀𝑌1 0 + 𝑀𝑌2 0 + 𝜓2 𝑡 , 𝑡 ∈ −𝑎𝜎, 𝑎𝜎 (2.1)
2

𝑀𝑌 𝑡 − 𝑀𝑌 0 − 𝑡𝑀𝑌1 0
i. e., 𝜓2 𝑡 = − 𝑀𝑌2 0 , 𝑡 ∈ −𝑎𝜎, 𝑎𝜎 , 𝑡 ≠ 0.
𝑡2
2

Using L’ Hospital rule 0/0 form we get

1 1
𝑀𝑌 𝑡 − 𝑀𝑌 0
lim 𝜓2 𝑡 = lim − 𝑀𝑌2 0
𝑡→0 𝑡→0 𝑡

= 𝑀𝑌2 0 − 𝑀𝑌2 0

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 3

NPTEL- Probability and Distributions

= 0. (2.2)

𝑛 𝑋𝑛 −𝜇 1 𝑛
The m.g.f. of 𝑍𝑛 = = 𝑖=1 𝑌𝑖 is
𝜎 𝑛

𝑡 𝑛
𝑖=1 𝑌𝑖
𝑀𝑛 𝑡 = 𝐸 𝑒 𝑛

𝑛
𝑡𝑌 𝑖
=𝐸 𝑒 𝑛

𝑖=1

𝑛
𝑡𝑌 𝑖
= 𝐸 𝑒 𝑛 (𝑌𝑖 s are independent)
𝑖=1

𝑛
𝑡
= 𝑀𝑌 (𝑌𝑖 s are i. i. d. )
𝑛
𝑛
𝑡 𝑡2 𝑡
= 𝑀𝑌 0 + 𝑀𝑌1 0 + 𝑀𝑌2 0 + 𝜓2 (using 2.1 )
𝑛 2𝑛 𝑛
𝑛
𝑡2 𝑡
= 1+ 1 + 𝜓2 , 𝑡 ∈ − 𝑛𝑎𝜎, 𝑛𝑎𝜎 , 𝑛 = 1, 2, ….
2𝑛 𝑛

Now using (2.2) and Proposition 1.1 we get

𝑡2
lim 𝑀𝑛 𝑡 = 𝑒 2 = 𝐾 𝑡 , say, 𝑡 ∈ ℝ.
𝑛→∞

Note that 𝐾 𝑡 , 𝑡 ∈ ℝ, is the m.g.f. of 𝑍 ~ 𝑁 0,1 . Using Theorem 1.5 (i) we conclude
𝑑
that 𝑍𝑛 → 𝑍 ~ 𝑁 0,1 , as 𝑛 → ∞. ▄

Remark 2.1

(i) The WLLN implies that the sample mean, based on a random sample from any
parent distribution, can be made arbitrarily close to the population mean in
probability by choosing sufficiently large sample size.
(ii) The CLT states that, irrespective of the nature of the parent distribution, the
probability distribution of a normalized version of the sample mean, based on a
random sample of large size, is approximately normal. For this reason the normal
distribution is quite important in the field of Statistics. ▄

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 4

NPTEL- Probability and Distributions

7.2.1 Random Walk

Example 2.1

Consider a drunkard, who having missed his bus from the bus stand, starts walking
towards his residence. Every second he either moves half a meter forward or half a meter
1
backward from his current position, each with probability 2 . Assuming that steps are
taken independently, find the (approximate) probability that after fifteen minutes the
drunkard will be within 30 meters form the bus stand.

Solution. Note that in 15 minutes (= 900 seconds) the drunkard will take 900 steps. Let
𝑌𝑖 be the size (in meters) of the i-th step, 𝑖 = 1, 2, … ,900. Then 𝑌1 , 𝑌2 , … are i.i.d. random
variables with

1 1 1
𝑃 𝑌1 = − =𝑃 𝑌1 = = ,
2 2 2

and 𝑌 = 900𝑖=1 𝑌𝑖 is the position of the drunkard after 15 minutes. The desired
probability is

1 1
𝑃 𝑌 ≤ 30 =𝑃 − ≤ 𝑌900 ≤ ,
30 30
1 900 𝑌 1
where 𝑌900 = 𝑖=1 𝑌𝑖 = . Note that 𝐸 𝑌1 = 0 and Var 𝑌1 = 𝐸 𝑌12 = = 𝜎 2 ,
900 900 4
say. By the CLT

900 𝑌900 − 0 approx

𝑍900 = ~ 𝑁 0,1 ,
1
2
approx
i. e., 𝑍900 = 60 𝑌900 ~ 𝑁 0,1 .

The desired probability is

𝑃 𝑌 ≤ 30 = 𝑃 −2 ≤ 𝑍900 ≤ 2
approx .
= Φ 2 − Φ −2

= 2Φ 2 − 1

= 2 × .9772 − 1

= .9544. ▄

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 5

NPTEL- Probability and Distributions

7.2.2 Justification of Relative Frequency Method of Assigning Probabilities

Example 2.2

Suppose that we have independent repetitions of a random experiment under identical

conditions. Further suppose that we are interested in assigning probability, say 𝑃 𝐸 , to
an event 𝐸. To do this we repeat the random experiment a large (say 𝑁) number of times.
Define

1, if 𝑖 − th trial results in occurrence of 𝐸

𝑌𝑖 = , 𝑖 = 1, … , 𝑁.
0, otherwise

Then 𝑌1 , 𝑌2 , … are i.i.d. random variables with common mean 𝜇 = 𝐸 𝑌1 = 𝑃 𝐸 . Also

𝑓𝑁 𝐸 = number of times event 𝐸 occurs in first 𝑁 trials

𝑁

= 𝑌𝑖
𝑖=1

and the relative frequency of event 𝐸 in first 𝑁 trials is

𝑁
𝑓𝑁 𝐸 1
𝑟𝑁 𝐸 = = 𝑌𝑖 = 𝑌𝑁 , say ∙
𝑁 𝑁
𝑖=1

The WLLN implies that

𝑝
𝑟𝑁 𝐸 = 𝑌𝑁 → 𝜇 = 𝑃 𝐸 , as 𝑁 → ∞.

Thus the WLLN justifies the relative frequency approach to assign probabilities. ▄

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 6