Distribution of the Sample Proportion and Sampling Distributions

Sampling from a Finite Population Without Replacement

The proof is the same as shown in the earlier case.

1
Analysis of the Sampling Distribution of the Sample Proportion

Hence the sample proportion P is a linear combination of X.

Sampling from an Infinite Population or From a Finite Population with Replacement

2
II. Sampling from a Finite Population Without Replacement

3
The proof is the same for the earlier case.

4
The standard error of P

Result 1: Limit of Binomial is Normal Distribution

Let the random variable X have a Binomial distribution with parameters n and p and for a fixed a<b

P a 
X  np 
 
 b  P a npq  np  X  b npq  np   ( b )   ( a ) as n (41)
 npq 

Result 2: Limit of Poisson is Normal Distribution

Let the random variable X have a Poisson distribution with parameter  and for a fixed a<b

P a 
X 


 
 b  P a     X  b      ( b )   ( a ) as  (40)
 

SAMPLING FROM NORMAL POPULATIONS

Normal distribution plays a predominant role in quantitative economic analysis. In this section the
sampling distributions of most commonly used statistics like sample mean and sample variances and
their ratios based on samples from normal distribution are discussed. These results are true only when
we are sampling from normal distribution. That is the f(x) mentioned above has to be a normal
distribution with a certain mean and variance.

Let us assume that the samples X1, X2,…,Xn are from a normal distribution with mean μ and variance
σ2, with the common density

( x )2
1 
f ( x;  ,  ) 
2
e 2 2
;    x   ,     ,   0
2 

5
There are 5 major results of our interest here. These results introduce three more distributions, namely
those of (i) t distribution,(ii) Chi square distribution and (iii) F distribution, which are of paramount
importance in econometric analysis. What one needs to know is the way the statistics are defined and
the structure of their densities.

X 1  X 2  ...  X n
Result 1: The sampling distribution of sample mean X  , when the values
n
of population mean μ and population variance σ2 are known.

X 1  X 2  ...  X n
The sampling distribution of X  is also normal but with mean μ and variance
n
σ2/n. Its density, therefore is

( x  )2

1
f ( x;  , 2 )  e 2 ( 2 / n )
;    x   ,     ,   0
2 ( / n)

( X  )
or, equivalently, the distribution of Z  is the standard normal distribution. Therefore the
( / n )
standard error of X is  / n .

Remark:

1. Recall that the central limit theorem asserts that, when sampling from any distribution, (that is,
whatever be the common distribution of X, but with finite mean and variance), the distribution of the
sample mean is approximately normal with mean μ and variance σ2/n. The above result asserts the
distribution of the sample mean is exactly normal with mean μ and variance σ2/n if the sample is from
a normal distribution.

2. The above result is a particular case of a more general result:

n n
Result 1(a) : The sampling distribution of   i X i is also normal but with mean    i and
i 1 i 1
n
variance  2 
i 1
2
i where  i are known constants.

USES: The result given above is useful in testing the mean of a normal distribution when the variance
is known. It is also useful in the construction of confidence interval for the population mean in normal
samples, when the variance is known.
X 1  X 2  ...  X n
Result 2: The sampling distribution of sample mean X  , when the
n
population mean μ is known but the population variance σ2 is unknown.

When σ2 is unknown, the distribution of X is not exactly normal. W. S. Gosset (under the
pseudonym Student), in the early 1900s, discovered the exact distribution of a standardized version of
n

(X
i 1
i  X )2
X . Let S2 be the sample variance, that is, S2 = . Gosset derived the sampling
n 1
distribution of the statistic

6
 (X  ) (X  )
t=  (n  1)
{S / (n  1)} S
as a Student t-distribution (or t-distribution), with (n-1) degrees of freedom. Its density is given by

(n / 2) 1 1
f (t )   t  
((n  1) / 2) (n  1)  t2 
n/2

1  
 (n  1) 
As mentioned before if p is an integer Γ(p)= (p-1)..2.1=(p-1)!, and when p is not an integer it is the

value of the integral: ( p)   e  x x p 1dx ; p  0
0
Result 3: The sampling distribution of the statistic

( X i  )2 ( X i  ) 
n n n2

 n2   =     zi
 2

i 1 2 i 1    i 1
( X i  )
where, zi  is the standardized variable.

when the of population mean μ and the population variance σ2 are both known. This statistic is
also known by the name “Chi-square” statistic.

Note that this statistic is the sum of squares of deviations of each observation from the population
mean, when each of such squared deviation is divided by the population variance. Alternatively it is
the sum of squares of n independent standard normal variables. Its pdf is that of a “chi-square random
variable” with n degrees of freedom, and is given by

1
f ( 2 )  e   / 2 (  2 ) ( n2) / 2 ; 2 0
2

2 n/2
 ( n / 2)

Note that this statistic is a non-negative random variable and hence its distribution spreads only on the
positive (right) side of zero. This distribution belongs to the family of Gamma distributions. The
shape of this distribution varies with the value of n. The percentage points of this distribution are
available in a tabular format for different values of n.

Result 4 : The sampling distribution of the statistic

2
n
(X i  X )2 n
( X  X ) 
 2
=  i 
2 
n
i 1 i 1  
when the of population mean μ is unknown but the population variance σ2 is known.

Its pdf is also that of a “chi-square random variable” but with (n-1) degrees of freedom, and is given
by

1
f ( 2 )  e   / 2 (  2 ) ( n 3) / 2 ; 2 0
2

( n 1) n / 2
2 ((n  1) / 2)

Note that the pdf is the same as that given under Result 3 except that n is replaced by (n-1). As said
before under Result 3, note that this is also a non negative random variable and hence its distribution
spreads only on the positive (right) side of zero. This distribution also belongs to the family of

7
Gamma distributions. The shape of this distribution varies with the value of (n-1). The percentage
points of this distribution can be read out from the table of Chi-square distribution.

Result 4a: If X and S2 are the sample mean and variance respectively from a random sample of
size n from a normal population with mean μ and variance σ2 then X and S2 are independent and the
(n  1) S 2
random variable has a chi-square distribution with n-1 degrees of freedom.
2

Result 4b: If Y and Z are independent random variables such that Y has a Chi-square distribution
with  degrees of freedom and Z has a standard normal distribution then the distribution of the
Z
random variable T  is the t-distribution with  degrees of freedom.
Y /

Result 5: The sampling distribution of the statistic of the type

(  12 / m)
F=
(  22 / n)

where (i)  12 is a chisqure statistic with m degrees of freedom, (ii)  22 is a chisqure statistic
with n degrees of freedom and (iii)  12 and  22 are independent.

The sampling distribution of the above mentioned F statistic has the pdf

((m  n) / 2)  m  x ( m2) / 2
m/2

f ( x)    ; 0 x
(m / 2)(n / 2)  n  [1  (m / n) x]( m n ) / 2

The above distribution is also known as Snedecor’s F-Distribution with m and n degrees of freedom.

F is a non-negative random variable, with support on (0,∞). This is also known as a Type II Beta
distribution. The pdf depends on the values of m and n, which are essentially the degrees of the Chi-
square statistics in the numerator and the denominator of the F statistic. Percentage points of the F
distributions are available in a tabular format for different values of m and n.

An interesting result is that 1/F also has an F distribution with degrees of freedom interchanged to
(n,m). This is helpful to read the percentage points of the distribution from the F tables, since the
tables are available only for m ≥ n.