Chapter 7: Introduction

Suppose X1 , X2 , . . . , X n is a random sample on a random variable X which has a N(µ, σ2 )

Ín
distribution. Denote the sample mean by X̄ n  1
n i1 X i . Then it is well know that
X̄ n ∼ N(µ, σ2 /n). What if X does not have a normal distribution?
Suppose X1 , X2 , . . . , X n is a random sample on a random variable X which has a
Statistical Inference, September 2, 2020

Ín
EXP(θ) distribution. Denote the sample sum by n X̄ n  i1 X i . Then it is well know that
n X̄ n ∼ GAM(θ, n). What if X does not have an exponential distribution?
Suppose X1 , X2 , . . . , X n is a random sample on a random variable X which has a
distribution defined by the pdf f X (x). Further, suppose that t n  t(x1 , . . . , x n ) is a function
of x 1 , . . . , x n such that Tn  t(X1 , . . . , X n ) is a random variable. Several special forms of
Ín Ín
Tn are Tn  X̄ n  1
n i1 X i , Tn  S2n  1
n−1 i1 (X i − X̄ n )2 , Tn  X(1) , Tn  X(n) , etc. These
random variables play a key role in obtaining exact procedures for estimation, confidence
interval and testing of unknown parameters of the distribution.
In some cases the pdf of Tn is obtained easily, but there are many important cases
where the derivation is not tractable.
In many of these, it is possible to obtain useful approximate results that apply when n
is large. These results are based on the notions of convergence in distribution and limiting
distribution.

Chapter 7: Sequences of Random Variables

Consider a sequence of random variables Y1 , Y2 , . . . with a corresponding sequence of

CDFs G1 (y), G2 (y), . . . so that for each n  1, 2, . . .

G n (y)  P[Yn ≤ y].

1 Convergence in distribution

If the CDF of Yn is G n (y) for each n  1, 2, . . . , .. and if for some CDF GY (y) of a random
variable Y,
lim G n (y)  GY (y)
n→∞

1
 for all values y at which GY (y) is continuous, then the sequence Y1 , Y2 , . . . , .. is said to
d
converge in distribution to Y, denoted by Yn → Y. The distribution corresponding to the
CDF GY (y) is called the limiting distribution of Yn .
Example 7.2.1 of the book Let X1 , . . . , X n , be a random sample from a uniform distribution,
X ∼ UNIF(0, 1). Then,
f X (x)  1, 0 < x < 1,
Statistical Inference, September 2, 2020

and zero otherwise, and



 0, if x ≤ 0


FX (x)  x, if 0 < x < 1

 1, if x ≥ 1.


Further, let Yn  X n:n , the largest order statistic. Then, it follows that the G n (y) of Yn is

G n (y)  [FX (y)]n  y n , 0<y<1

and zero if y ≤ 0 and one if y ≥ 1. Of course, when 0 < y < 1, y n approaches 0 as n

approaches ∞, and when y ≤ 0 or y ≥ 1, G( y) is a sequence of constants, with respective
limits 0 or 1. Thus, (
0, if y < 1
lim G n (y) 
n→∞ 1, if y ≥ 1

The degenerate random variable

A random variable X is degenerate if, for some constant µ, P(X  µ)  1. The CDF of
X is given by (
0, if x < µ
FX (x) 
1, if x ≥ µ.
The moment generating function of X is

MX (t)  exp(µt)

and
Var(X)  0.

Now, let Y be a degenerate random variable with P(Y  1)  1. Then

(
0, if y < 1
GY (Y) 
1, if y ≥ 1.

2
 converge in distribution to Y G(y), denoted by - Y. The distribution corre-
sponding to the CDF G(y) is called the limiting distribution of }.

Fxmph1 7.2.1 Let X1, ..., X, be a random sample from a uniform distribution, X. UNIF(O, 1),
and let }Ç = the largest order statistic. From the results of Chapter 6, it
follows that
Now, one can check thatthe CDF of is
G(y)y' lim G n (y)  GY (y)
O<y<1 (7.2.3)
n→∞
zero if y O and one if y 1. Of course, when O <y < 1, yfl approaches O as n
for all values y at which GY (y) is continuous.
approaches x, and when y O or y ? 1, G(y) is a sequence of constants, with
This situation is illustrated in Figure 1 (Figure 7.1 in the book), which shows G(y) and
G(y) for n = 2, 5, and 10.
Statistical Inference, September 2, 2020

FIGURE 7.1 Comparison of CDFs G(y) with limiting degenerate CDF G(y)
G,1(y) G(y)

5
Yo y2

y
o o

Figure 1: Comparison of CDFs G n (y) with limiting degenerate CDF GY (y)

d
Thus, Yn  X n:n → Y where the random variable Y has a degenerate distribution with
P[Y  1]  1. In other words we can say that the nth order statistic from a uniform (0, 1)
distribution converges to a degenerate random variable. Or the nth order statistic from a
ression, OCR, web optimization
uniform using
(0, 1) has a limiting a watermarked
distribution evaluationatcopy
which degenerates 1. of CVISION PDFCompressor
Example 7.2.2 of the book Let X1 , . . . , X n , be a random sample from an exponential
distribution, X ∼ EXP(θ). Then,
1  x
f X (x)  exp − , x > 0, θ>0
θ θ
and zero otherwise, and
(
0, if x≤0
FX (x) 
1 − exp − θ , if
x
x > 0.

Further, let Yn  X1:n , the smallest order statistic. Then, it follows that the G n (y) of Yn is
 ny 
G n (y)  1 − [1 − FX (y)]  1 − exp −
n
, y>0
θ

3
 y

and zero if y ≤ 0. We have limn→0 G n (y)  1 if y > 0 because exp − θ < 1 in this case.
Thus, the limit is zero if y < 0 and one if y > 0. Also, notice that the limit at y  O is zero.
Thus

(
0, if y ≤ 0
lim G n (y) 
n→∞ 1, if y > 0
Statistical Inference, September 2, 2020

Observe that the limiting function is not only discontinuous at y  0 but also not even
continuous from the right at y  0, which is a requirement of a CDF.
Now, define the CDF of a degenerate random variable Y as
(
0, if y < 0
GY (Y) 
1, if y ≥ 0.

Now, note that the right hand side (limiting function) of limn→∞ G n (y) and GY (Y) except
for one point y  0 but this is not a problem, because the definition of convergence in
distribution requires only that the limiting function agrees with a CDF at its points of
continuity and y  0 is the point of discontinuity of GY (Y).
d
Thus, X1:n → Y where the random variable Y has a degenerate distribution with
P[Y  0]  1. That is the first order statistic from an exponential distribution converges to
a degenerate random variable.

2 Stochastic convergence

A sequence of random variables, Y1 , Y2 , . . . , . . . is said to converge stochastically to a constant

c if it has a limiting distribution that is degenerate at c.

3 Non-degenerate limiting distributions

In earlier examples we have seen that limiting distribution are degenerate. But, not all
limiting distributions are degenerate, as seen in the next example. The following limits
are useful in many problems:
 c  nb
lim 1 +  exp(cb),
n→∞ n
  nb
c d(n)
lim 1 + +  exp(cb) if lim d(n)  0.
n→∞ n n n→∞

4
 The Pareto distribution
A random variable X is said to have a Pareto distribution with parameters θ and κ,
denoted by X ∼ PAR(θ, κ) if its density is given by
κ
f X (x; θ, κ)  , x > 0, θ > 0, κ > 0.
θ(1 + x/θ)κ+1
Statistical Inference, September 2, 2020

The CDF is given by x  −κ

FX (x; θ, κ)  1 − 1 + , x > 0.
θ

Example 7.2.3 of the book Let X1 , . . . , X n , be a random sample from an exponential

distribution, X ∼ PAR(1, 1), and let Yn  nX1:n . The CDF of X is

FX (x)  1 − (1 + x)−1 , x>0

so the CDF of Yn , is

G n (y)  P[Yn ≤ y]
 P[nX1:n ≤ y]
h yi
 P X1:n ≤
n  i
h y n
 1 − 1 − FX
n
 y  −n
1− 1+ , y > 0.
n
Now, taking he limit of G n (y) as n → ∞, for y > 0, we get
 y  −n
lim G n (y)  1 − lim 1 +  1 − exp(−y).
n→∞ n→∞ n
WE know that if Y ∼ EXP(1), then
(
0, if y ≤ 0
GY (Y) 
1 − exp(−y), if y > 0.
Now, observe that
lim G n (y)  GY (Y).
n→∞

his is illustrated in Figure 2 (Figure 7.2 in the book), which shows the graphs of GY (y)
and G n (y) for n  1, 2, and 5.
Thus the limiting distribution of nX1:n is exponential which is a non-degenerate dis-
tribution.

5
 which is the CDF of an exponential distribution, EXP(1) This is illustrated in
Figure 7.2, which shows the graphs of G(y) and G,,(y) for n = 1, 2, and 5.

FIGURE 7.2 Comparison of CDFs G(y) with 'imiting CDF G(y)

G(y) = i - e

Ql (Y) G2(y) G,(y)

Statistical Inference, September 2, 2020

FigureThe
2: Comparison of CDFs
following example G n (y)
shows thatwith limitingofCDF
a sequence GY (y)
random variables need not
have a limiting distribution.
4 Limiting distribution does not exit

The following example shows that a sequence of random variables need not have a limiting
distribution.
Example 7.2.4 of the book Let X1 , . . . , X n , be a random sample from an exponential
distribution, X ∼ PAR(1, 1), and let Yn  X n:n . The CDF of X is
compression, OCR, web optimization using a watermarked
−1 x evaluation copy of CVISION PDFComp
FX (x)  1 − (1 + x)  , x>0
1+x
so the CDF of Yn , is

G n (y)  P[Yn ≤ y]
 P[X n:n ≤ y]

n
 FX y

 n
y
 , y > 0,
1+ y

and zero otherwise. Because y/(l + y) < 1, we have limn→∞ G n (y)  0 for all y > 0 which
can not be a CDF because it does not approach one as y → ∞.