Order Statistics (Math 313)

Prof. Magdy E. El-Adll

Department of Mathematics

Faculty of Science

Helwan University

2021
Contents

1 An Introduction 2

2 Basic Distribution Theory 3

2.1 Chapter Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.2 The Distribution of an Order Statistic . . . . . . . . . . . . . . . 5

1
Chapter 1

An Introduction

2
Chapter 2

Basic Distribution Theory

3
Order Statistics (Math 313) Prof. Magdy E. El-Adll

2.1 Chapter Objectives

After completing this chapter, you should be able to:

1. Identify types of random variables.

2. Distinguish between independent and dependent random variables.

3. Clarify the concept of random sample.

4. Identify the basic terms of order statistics.

5. Derive the distribution of an order statistic.

6. Demonstrate the importance of the median and the range.

7. Establish the distributions of some functions of order statistics.

4
Order Statistics (Math 313) Prof. Magdy E. El-Adll

2.2 The Distribution of an Order Statistic

Suppose that X1, X2, ..., Xn, are independent and identically distribut-

ed random variables (iid) with common probability density function

(pdf) f (x) and cumulative distribution function (cdf) F (x). Arrange

(X1, X2, ..., Xn) in an increasing order of magnitude and let

X1:n < X2:n < ... < Xn:n, be the ordered values. The rth elemen-

t of this sequence,Xr:n, is called the rth order statistic. The first and

last order statistics, X1:n and Xn:n, are the minimum and maximum of

(X1, X2, ..., Xn), respectively. The minimum and maximum order statis-

tics are called the extremes.

Order statistics are very important in practice and especially the min-

imum, X1:n, and the maximum, Xn:n, because they are the critical values

used in engineering, physics, medicine, etc.

5
Order Statistics (Math 313) Prof. Magdy E. El-Adll

Let X1, X2, ..., Xn, be a random sample of size n from a continuous

population with pdf f (x) and cdf F (x). Then, the joint pdf and the joint

cdf of X1, X2, ..., Xn, respectively, are

n
Y
f (x1, x2, ..., xn) = f (xi) (2.2.1)
i=1

and
n
Y
F (x1, x2, ..., xn) = F (xi). (2.2.2)
i=1

Clearly, X1:n = min{X1, X2, ..., Xn} and Xn:n = max{X1, X2, ..., Xn}.

The cdf of the maximum order statistic, Xn:n, denoted by Fn:n can be

derived as

6
Order Statistics (Math 313) Prof. Magdy E. El-Adll

Fn:n(x) = P (Xn:n 6 x)

= P (max{X1, X2, ..., Xn} 6 x)

= P (X1 6 x, X2 6 x, ..., Xn 6 x)
n
Y
= P (Xi 6 x) (by the independence of {X1, X2, ..., Xn})
i=1
Yn
= F (x) (by the identical of {X1, X2, ..., Xn})
i=1

= [F (x)]n , −∞ < x < ∞. (2.2.3)

Therefore, the cdf of the maximum, Xn:n, can be obtained by raising

the original cdf, F , to the nth power. The pdf of the sample maximum,

Xn:n, is obtained by differentiating (2.2.3) with respect to x, namely,

fn:n(x) = n [F (x)]n−1 f (x), −∞ < x < ∞. (2.2.4)

The cdf of the minimum order statistic, X1:n, denoted by F1:n, can be

7
Order Statistics (Math 313) Prof. Magdy E. El-Adll

derived in a similar way as follows:

F1:n(x) = P (X1:n 6 x)

= P (min{X1, X2, ..., Xn} 6 x)

= 1 − P (min{X1, X2, ..., Xn} > x)

= 1 − P (X1 > x, X2 > x, ..., Xn > x)

n
Y
=1− P (Xi > x) (by the independence of {X1, X2, ..., Xn})
i=1
Yn
=1− (1 − F (x)) (by the identical of {X1, X2, ..., Xn})
i=1

= 1 − [1 − F (x)]n , −∞ < x < ∞. (2.2.5)

The pdf of the sample maximum, X1:n, is obtained by differentiating

(2.2.5) with respect to x, as

f1:n(x) = n [1 − F (x)]n−1 f (x), −∞ < x < ∞. (2.2.6)

8
Order Statistics (Math 313) Prof. Magdy E. El-Adll

The cdf of the rth order statistic, Xr:n, can be obtained as follows:

Fr:n(x) = P (Xr:n 6 x)

= P ( at least r of {X1, X2, ..., Xn} are less than or equal to x)

n
X
= P ( exactly i random variables among {X1, X2, ..., Xn}
i=r

are less than or equal to x)

n  
X n
= [F (x)]i[1 − F (x)]n−i, −∞ < x < ∞. (2.2.7)
i=r
i

Exercise 1. Prove the identity:

n   Z z
X n i n!
z (1 − z)n−i = tr (1 − t)n−r dt. (2.2.8)
i=r
i (r − 1)!(n − r)! 0

The incomplete beta function denoted by Iz (a, b) is defined as

Z z
1
Iz (a, b) = ta−1(1 − t)b−1dt, (2.2.9)
B(a, b) 0

9
Order Statistics (Math 313) Prof. Magdy E. El-Adll

where, B(a, b) is the beta function given by

Z 1
Γ(a)Γ(b)
B(a, b) = ta−1(1 − t)b−1dt = , a > 0, b > 0,
0 Γ(a + b)

and Γ(a) is gamma function defined by

Z ∞
Γ(a) = ta−1e−tdt.
0

If a is an integer, Γ(a) = (a − 1)!. Moreover, it can be shown that

√
Γ(a + 1) = aΓ(a) and Γ(1/2) = π.

By (2.2.7), (2.2.8), and (2.2.9), the cdf of the rth order statistic, Xr:n,

can be written as

Fr:n(x) = IF (x)(r, n − r + 1), −∞ < x < ∞. (2.2.10)

We shall now assume that Xi is a continuous random variable with pdf

f (x) = F 0(x). Therefore, the pdf of the rth order statistic, Xr:n, can be

10
Order Statistics (Math 313) Prof. Magdy E. El-Adll

obtained by (2.2.9), and (2.2.10) as

Z F (x)
1 d
fr:n(x) = tr−1(1 − t)n−r dt
B(r, n − r + 1) dx 0
1
= [F (x)]r−1[1 − F (x)]n−r f (x), − ∞ < x < ∞.
B(r, n − r + 1)

(2.2.11)

Recall that (from elementary calculus)

Z F (x)
d
H(t)dt = H(F (x))F 0(x).
dx 0

Example 2.2.1. Let U1, U2, ..., Un be a random sample of size n from

the uniform distribution on the interval (0, 1). That is F (u) = u,

0 < u < 1. Find the pdf, the mean and the variance of Ur:n.

Th pdf of Ur:n can be obtained from (2.2.11)

1
fr:n(u) = ur−1(1 − u)n−r
B(r, n − r + 1)
n!
= ur−1(1 − u)n−r , 0 < u < 1.
(r − 1)!(n − r)!
11
Order Statistics (Math 313) Prof. Magdy E. El-Adll

That is Ur:n follows beta distribution with parameters r and n − r + 1.

The m−moment of Ur:n can be obtained directly as

Z 1
m
E [U ] = umfr:n(u)du
0
Z 1
n!
= um+r−1(1 − u)n−r du
(r − 1)!(n − r)! 0
n!(m + r − 1)!
= (Prove). (2.2.12)
(r − 1)!(m + n)!

Therefore, the mean and the variance of Ur:n, respectively are given by

r r(n − r + 1)
E[Ur:n] = and V ar(Ur:n) = . (2.2.13)
n+1 (n + 1)2(n + 2)

For r = 1, we have

1 n
E[U1:n] = and V ar(U1:n) = .
n+1 (n + 1)2(n + 2)

For r = n, we have

n n
E[Un:n] = and V ar(Un:n) = .
n+1 (n + 1)2(n + 2)

12