MTL390: Statistical Methods
Instructure: Dr. Biplab Paul
January 17, 2025
Lecture 7
Sample Moments and Their Distributions (Cont.)
Sampling Distributions Associated with Normal Pop-
ulations
The sampling distribution of a statistic depends on the population distribution from
which the samples are taken. In this section, we discuss the sampling distributions of
some statistics that are based on a random sample drawn from a normal distribution.
Here we define the chi-square, t-, and F -distributions and study their properties.
Theorem 1. Let X1 , X2 , . . . , Xn be independent random variables, where Xi ∼ N (µi , σi2 ).
Let a1 , a2 , . . . , an be real constants. Then the linear combination:
n
X
Y = ai X i
i=1
is normally distributed with:
n
X n
X
µY = ai µ i and σY2 = a2i σi2 .
i=1 i=1
Proof. The moment-generating function (MGF) of Y is:
h Pn i
MY (t) = E[etY ] = E et i=1 ai Xi .
By independence of Xi ’s:
n
Y
MY (t) = E[etai Xi ].
i=1
Using the definition of the MGF for each Xi :
1 2 2
MXi (t) = exp tµi + t σi ,
2
we substitute tai into the MGF of Xi :
n
Y 1 2 2
MY (t) = exp ai tµi + (ai t) σi .
i=1
2
1
Simplify the product:
n n
!
X 1 X
MY (t) = exp t ai µi + t2 a2i σi2 ,
i=1
2 i=1
Pn
which is the mgf of a normal random variable with mean mean i=1 ai µi and variance
P n 2 2
i=1 ai σi .
Corollary 1.1. If X1 , X2 , . . . , Xn is a random sample of size n from N (µ, σ 2 ), then the
sample mean
n
1X
X̄ = Xi
n i=1
is normally distributed with
2 σ2
µX̄ = µ, σX̄ = .
n
Using the z-transformation, the sample mean can be converted into a standard normal
random variable
X̄ − µ
Z= √ ∼ N (0, 1).
σ/ n
Example A car manufacturing company claims that the gas mileage for its new line of
hybrid cars averages 60 miles per gallon with a standard deviation of 4 miles per gallon.
A random sample of 16 cars yielded a mean of 57 miles per gallon. If the company’s
claim is correct, what is the probability that the sample mean is less than or equal to 57
miles per gallon? Comment on the validity of the company’s claim. Assume the sample
of 16 measurements comes from a normal population.
Solution: Let X represent the gas mileage for the new hybrid car (in miles per gallon).
According to the company’s claim, X ∼ N (60, 42 ). From corollary 1.1, the sampling
distribution of the sample mean X̄ for a sample of size n = 16 is
42
X̄ ∼ N 60, = N (60, 1).
16
The probability is
X̄ − µ
P (X̄ ≤ 57) = P ( ≤ −3) = P (Z ≤ −3).
σX̄
From standard normal tables or software:
P (Z ≤ −3) ≈ 1 − 0.999 = 0.001.
Therefore, if the company’s claim is correct, it is very unlikely that the mean value of
the random sample of 16 cars will be 57 miles per gallon. Because the mean is indeed 57
miles per gallon, we conclude that the company’s claim is very likely not true.
2
Chi-square distribution
A random variable X is said to have a chi-square distribution (χ2 -distribution) with n
degrees of freedom, where n is a positive integer, if its probability density function (PDF)
is given by (
1 −x/2 (n/2)−1
n/2 e x , 0 < x < ∞,
fX (x) = Γ(n/2)2
0, x < 0.
Figure 1: Chi-Square Densities
We write X ∼ χ2 (n) for a χ2 random variable with n degrees of freedom (df). If
X ∼ χ2 (n), then:
E[X] = n, Var(X) = 2n.
The moment-generating function (MGF) of X is:
1
MX (t) = (1 − 2t)−n/2 , t< .
2
It is a very useful distribution in statistical inference problems.
Theorem 2. Let X1 , X2 , . . . , Xk be independent χ2 -random variables with n1 , n2 , . . . , nk
degrees of freedom, respectively. Then the sum
k
X
V = Xi
i=1
is χ2 -distributed with n1 + n2 + · · · + nk degrees of freedom.
Proof. The proof is left as an exercise. Use MGF.
3
Theorem 3. Let X1 and X2 be independent random variables. Suppose X1 ∼ χ2 (n1 ),
and let Y = X1 + X2 , where Y ∼ χ2 (n) with n > n1 . Then
X2 = Y − X 1
is a χ2 -random variable with n − n1 degrees of freedom.
Proof. The proof is left as an exercise
Theorem 4. If X is a standard normal random variable, then X 2 is a chi-square random
variable with 1 degree of freedom.
Proof. Because X ∼ N (0, 1), the moment-generating function of X 2 is given by
Z ∞
2 1 2 1
MX 2 (t) = etx √ e−x /2 dx = (1 − 2t)−1/2 , for t < .
−∞ 2π 2
This matches the moment-generating function of a chi-square distribution with 1 degree
of freedom. Thus, X 2 ∼ χ2 (1).
Theorem 5. Let the random sample X1 , X2 , . . . , Xn be from a N (µ, σ 2 ) distribution.
Then
Xi − µ
Zi = , i = 1, 2, . . . , n
σ
are independent standard normal random variables, and
n n 2
X
2
X Xi − µ
Zi =
i=1 i=1
σ
has a χ2 -distribution with n degrees of freedom. In particular, if X1 , X2 , . . . , Xn are
independent standard normal random variables, then
n
X
2
Y = Xi2
i=1
is χ2 -distributed with n degrees of freedom.
Example Let the random variables X1 , X2 , . . . , X5 be from an N (5, 1) distribution.
Find a number a such that:
5
!
X
P (Xi − 5)2 ≤ a = 0.90.
i=1
Solution: We know
5
X
(Xi − 5)2 ∼ χ2 (5).
i=1
From the chi-square table, the upper-tail area corresponding to α = 0.10 (90% cumulative
probability) with 5 degrees of freedom is χ20.10,5 = 9.23635. Thus,
5
!
X
P (Xi − 5)2 ≤ 9.23635 = 0.90.
i=1
Therefore, a = 9.23635.
4
Theorem 6. If X1 , X2 , . . . , Xn is a random sample from a normal population with mean
µ and variance σ 2 , then:
(a) Pn
i=1 (Xi − X̄)2
(n − 1)S 2
=
σ2 σ2
has a χ2 -distribution with (n − 1) degrees of freedom.
(b) X̄ and S 2 are independent.
Proof. (a) We know that σ12 ni=1 (Xi − µ)2 has a chi-square distribution with n degrees
P
of freedom. Thus,
n n
1 X 2 1 X 2
(X i − µ) = X i − X̄ + X̄ − µ
σ 2 i=1 σ 2 i=1
Since n
X
(Xi − X̄)(X̄ − µ) = 0,
i=1
we have
n n
!
1 X X
= 2 (Xi − X̄)2 + (X̄ − µ)2
σ i=1 i=1
2 2
(n − 1)S (X̄ − µ)
= 2
+ .
σ σ 2 /n
The left-hand side of this equation has a chi-square distribution with n degrees of
X̄−µ
√ ∼ N (0, 1), we have
freedom. Also, since σ/ n
2
X̄ − µ
√ ∼ χ2 (1),
σ/ n
implies,
(n − 1)S 2
2
∼ χ2 (n − 1).
σ
(b) The proof is beyond the scope of this course. A rigorous proof depends on the
geometric properties of the multivariate normal distribution. We first need to show
that the sample mean X̄ and the deviations (X1 − X̄, X2 − X̄, . . . , Xn − X̄) are
independent. This can be demonstrated using the moment-generating function
(MGF) of X̄, X1 − X̄, X2 − X̄, . . . , and Xn − X̄.
Example Let X1 , X2 , . . . , X10 be a random sample from a normal distribution with
σ = 0.8. Find two positive numbers a and b such that the sample variance S 2 satisfies
2
P a ≤ S 2 ≤ b = 0.90.
(n−1)S 2
Solution: Because ∼ χ2 (n − 1), we have:
σ2
(n − 1)S 2
2
(n − 1)a (n − 1)b
P a≤S ≤b =P ≤ ≤ .
σ2 σ2 σ2
5
The desired values can be found by setting the upper-tail area and lower-tail area each
equal to 0.05. Using the chi-square table with n − 1 = 9 degrees of freedom:
9b
= 16.919 (from χ20.05,9 ),
0.8
which implies:
16.919 × 0.8
b= = 1.50.
9
Similarly:
9a
= 3.325 (from χ20.95,9 ),
0.8
which implies:
3.325 × 0.8
a= = 0.295.
9
Thus:
P 0.295 ≤ S 2 ≤ 1.50 = 0.90.
Note: This is not the only interval that satisfies
P a ≤ S 2 ≤ b = 0.90,
but it is a convenient one.