Sampling Distribution

Elmer S. Poliquit

USC - ESPoliquit
1 / 31
Section 3.1 Sampling Distribution

The distribution of all possible values that can be assumed by some

statistic, computed from samples of the same size randomly drawn from
the same population, is called the sampling distribution of that statistic.
Sampling Distribution Construction
1 From a finite population of size N, randomly draw all possible
samples of size n.
2 Compare the statistic of interest for each sample.
3 List in one column the different distinct observed values of the
statistic, and in another column list the corresponding frequency of
occurrence of each distinct observed value of the statistic.

USC - ESPoliquit
2 / 31
Section 3.1 Sampling Distribution

Distribution of the Sample Mean

Suppose we have a population of size N = 4, consisting of the ages of four
children who are outpatients in a community health center. The ages are
6, 8, 10, and 12. The population mean with the population variance:

age=c(6,8,10,12)
mu_age=mean(age)
sigma2_age=((6-9)ˆ2+(8-9)ˆ2+(10-9)ˆ2+(12-9)ˆ2)/4
mu_age; sigma2_age

[1] 9

[1] 5

USC - ESPoliquit
3 / 31
Section 3.1 Sampling Distribution
Distribution of the Sample Mean
All possible samples of size, n = 2 from a population of size, N = 4

The sample above or below the principal diagonal result when sampling is
without replacement. USC - ESPoliquit
4 / 31
Section 3.1 Sampling Distribution
Distribution of the Sample Mean
When sampling is with replacement, there are 16 possible samples.

In general when sampling is with replacement, the number of possible

samples is N 2 (i.e. 42 = 16).
USC - ESPoliquit
5 / 31
Section 3.1 Sampling Distribution
Sampling Distribution of x̄ : Mean
Distribution of the Mean
4.0
3.5
3.0
Frequency

2.5
2.0
1.5
1.0

6 7 8 9 10 11 12

Age

The sampling distribution of x̄ has the same value as the mean of the
original population.
6(1) + 7(2) + 8(3) + 9(4) + 10(3) + 11(2) + 12(1)
x̄ = = 9.
16 USC - ESPoliquit
6 / 31
Section 3.1 Sampling Distribution
Sampling Distribution of x̄ : Mean

Age=c(6,7,7,8,8,8,9,9,9,9,9,10,10,10,11,11,12)
var(Age)

[1] 2.5

The variance of the sampling distribution (2.5) is not equal to the

population variance (5) so there is an error in the sampling distribution.
To get the standard error of the mean from a sample size, n:
σ
√ .
n

It is the ratio of the standard deviation of the sample and the square root
of the sample size, n.
USC - ESPoliquit
7 / 31
Section 3.1 Sampling Distribution

Sampling Distribution of x̄ : Mean

The standard error of the mean of sample size, n = 2:

Age=c(6,7,7,8,8,8,9,9,9,9,9,10,10,10,11,11,12)
var(Age)

[1] 2.5

USC - ESPoliquit
8 / 31
Section 3.1 Sampling Distribution

When sampling is from a normally distributed population, the

distribution of the sample means will possess the following properties:
1 The distribution of x̄ will be normal.
2 The mean, µx̄ , of the distribution of x̄ will be equal to the mean of
the population from which the sample was drawn.
3 The variance, σx̄ 2 , of the distribution of x̄ will be equal to the
variance of the population divided by the sample size.

σ2
σx̄ 2 =
n

USC - ESPoliquit
9 / 31
Section 3.1 Sampling Distribution

Sampling from Nonnormally Distributed Population

The Central Limit Theorem
Given a population of any nonnormal functional form with a mean µ and
finite variance σ 2 , the sampling distribution of x̄ , computed from sample
of size n from the population, will have mean µ and variance σ 2 /n and will
be approximately normally distributed when the sample size is large.
A mathematical formulation of the central limit theorem is that the
−µ)
distribution of (x̄ √ approaches a normal distribution with standard mean
s/ n
0 and variance 1 as n → ∞.

USC - ESPoliquit
10 / 31
Section 3.1 Sampling Distribution

Sampling from Nonnormally Distributed Population

Sampling without Replacement When sampling is without replacement
from a finite population, the sampling distribution of x̄ will have mean µ
and variance
σ2 N − n
σx̄ 2 = × .
n n−1
The factor N−n
n−1 is called the finite population correction and can be
ignored when the sample size is small in comparison with the population
size. When the population is much larger then the sample, the difference
between variances will be negligible.

USC - ESPoliquit
11 / 31
Section 3.1 Sampling Distribution
Summary
Sampling is from a normally distributed population with a known
population variance:
a. The mean of the distribution is equal to the population mean, µx̄ = µ.
b. The standard deviation of the distribution is equal to the ratio of
population standard deviation and the square root of the sample size,
σx̄ = √σn .
c. The sampling distribution of x̄ is normal.
Sampling is from a nonnormally distributed population with known
population variance:
a. The mean of the distribution is equal to the population mean, µx̄ = µ.
b. The standard deviation of the distribution is equal to the ratio of
population standard deviation and the square root of the sample size,
σx̄ = √σn when Nn ≤ 0.05 and σx̄ = √σn × N−n
n−1 , otherwise.
c. The sampling distribution of x̄ is approximately normal. USC - ESPoliquit
12 / 31
Section 3.1 Sampling Distribution
Simulation - Uniform Distribution This a simulation of 100,000
numbers between 0 and 10 inclusive taken in 4 different ways - 1, 10, 50,
and 100 ways.
Mean of Sample Means is 5 Mean of Sample Means is 5
Sd of Sample Means is 2.89 Sd of Sample Means is 0.91

0.4
Density

Density
0.06

0.2
0.00

0.0
0 2 4 6 8 10 0 2 4 6 8 10
Mean from a Sample of Size 1 Mean from a Sample of Size 10
Simulation with 100,000 Trials Simulation with 100,000 Trials

Mean of Sample Means is 5 Mean of Sample Means is 5

Sd of Sample Means is 0.41 Sd of Sample Means is 0.29
1.2
0.8
Density

Density

0.6
0.4
0.0

0.0

0 2 4 6 8 10 0 2 4 6 8 10
Mean from a Sample of Size 50 Mean from a Sample of Size 100
USC - ESPoliquit
Simulation with 100,000 Trials Simulation with 100,000 Trials
13 / 31
Section 3.1 Sampling Distribution
Applications
1 Suppose it is known that in a certain large human population cranial
length is approximately normally distributed with a mean of 185.6
mm and a standard deviation of 12.7 mm. What is the probability
that a random sample of size 10 from this population will have a
mean greater than 190?
Solution
The standard deviation is known, so we look for the z-score using
x − µx̄ x − µx̄
z= = √
σx̄ σ/ n

190 − 185.6
z= 12.7 = 1.0955 ≈ 1.10
√
10
Hence,
P(z > 1.10) = 1 − 0.8642 = 0.1357 or 13.57%.
USC - ESPoliquit
14 / 31
Section 3.1 Sampling Distribution
2 If the mean and standard deviation of serum iron values for healthy
men are 120 and 15 micrograms per 100 ml, respectively, what is the
probability that a random sample of 50 normal men will yield a mean
between 115 and 125 micrograms per 100 ml?
Solution
Since we have a sample size greater than 30, we make use of the central
limit theorem and transform the resulting approximately normal sampling
distribution of x̄ .
15
σx̄ = √ ≈ 2.1213
50
115 − 120 125 − 120
 
P(115 ≤ x̄ ≤ 125) = P ≤z ≤
2.1213 2.1213
round(pnorm(2.36, mean = 0, sd = 1)-
pnorm(-2.36, mean = 0, sd = 1),4)

[1] 0.9817
USC - ESPoliquit
15 / 31
Section 3.1 Sampling Distribution

2 If the mean and standard deviation of serum iron values for healthy
men are 120 and 15 micrograms per 100 ml, respectively, what is the
probability that a random sample of 50 normal men will yield a mean
between 115 and 125 micrograms per 100 ml?

Solution Continues

115 − 120 125 − 120

 
P(115 ≤ x̄ ≤ 125) = P ≤z ≤
2.1213 2.1213

Thus,
115 − 120 125 − 120
 
P(115 ≤ x̄ ≤ 125) = P ≤z ≤ = 0.9817.
2.1213 2.1213

USC - ESPoliquit
16 / 31
Section 3.1 Sampling Distribution
Distribution of the Difference Between Two Sample Means
Assume that each population is normally distributed.
The difference between two sample means is x̄1 − x̄2 .
Convert z as
(x̄1 − x̄2 ) − (µ1 − µ2 )
z= r .
σ12 σ22
n1 + n2

The two samples of sizes n1 and n2 are independent.

Sampling from Nonnormal Population
Take large samples, since when the sample sizes are large the central limit
theorem applies and the distribution of the difference between two sample
means is at least approximately normally distributed with a mean equal to
µ1 − µ2 and a variance of s
σ12 σ22
+ .
n1 n2
USC - ESPoliquit
17 / 31
Section 3.1 Sampling Distribution

Distribution of the Sample Proportion (p̂)

The mean of the distribution, µp̂ , that is, the average of all possible
sample proportions, will be equal to the true population proportions,
p, and the variance of the distribution, σp̂2 , will be equal to
p(1 − p)/n.
The z-score formula is
p̂ − p
z=q .
p(1−p)
n

Correction for Continuity

The normal approximation may be improved by the correction for
continuity, a device that makes an adjustment for the fact that a discrete
distribution is being approximated by a continuous distribution.

USC - ESPoliquit
18 / 31
Section 3.1 Sampling Distribution

Distribution of the Sample Proportion (p̂)

Suppose we let x = np̂, the number in the sample with the characteristic
of interest when the proportion p̂. To apply the correction for continuity,
compute
x +0.5
−p
zc = qn
p(1−p)
n

for x < np or zc for x > np.

The correction for continuity will not make a greater deal of difference
when n is large.

USC - ESPoliquit
19 / 31
Section 3.1 Sampling Distribution

Distribution of the Difference Between Two Sample Proportions

(p̂1 − p̂2 )
If independent random samples of size n1 and n2 are drawn from two
populations of dichotomous variables where the proportions of
observations with the characteristic of interest in the two populations are
p1 and p2 , respectively.
The z-score formula is
(p̂1 − p̂2 ) − (p1 − p2 )
z= q .
p1 (1−p1 ) p2 (1−p2 )
n1 + n2

USC - ESPoliquit
20 / 31
3.2 Estimation and Confidence Interval

Statistical inference is the procedure by which one reaches a conclusion

about a population on the basis of the information contained in a sample
drawn from that population.

A point estimate is a single numerical value used to estimate the

corresponding population parameter.
An interval estimate consists of two numerical values defining a
range of values that, with a specified degree of confidence.
The sampled population is the population from which one actually
draws a sample.
The target population is the population about which one wishes to
make an inference.

USC - ESPoliquit
21 / 31
3.2 Estimation and Confidence Interval
In general, an interval estimate may be expressed as follows:

estimator ± (reliability coefficient × standard error)

In particular, when sampling is from a normal distribution with known

variance, an interval estimate for µ may be expressed as

x̄ ± z(1−α/2) σx̄

where z(1−α/2) is the value of z to the left of which lies 1 − α/2 and to the
right of which lies α/2 of the area under its curve. The z(1−α/2) σx̄ is
called the margin of error.
Practical Interpretation
When sampling from a normally distributed population with known
standard deviation, we are 100(1 − α) percent confident that the single
computed interval, x̄ ± z(1−α/2) σx̄ , contains the population mean µ.
USC - ESPoliquit
22 / 31
3.2 Estimation and Confidence Interval

Trimmed Means
Estimators that are insensitive to outliers are called robust estimator.
Another robust measure and estimation of central tendency is the trimmed
mean. For a set of sample data containing n measurements we calculate
the 100α percent trimmed mean as follows:
1 Order the measurements.
2 Discard the smallest 100α percent and the largest 100α percent of
the measurements. The recommended value of α is something
between 0.1 and 0.2.
3 Compute the arithmetic mean of the remaining measurements.

USC - ESPoliquit
23 / 31
3.3 The t Distribution
When the sample size is large, say greater than 30, the sample
standard deviation s is substantial in approximating the population
standard deviation σ.
For small samples, an alternative, known as Student’s t distribution,
usually shortened to t distribution, is available.
Properties of t Distribution
1 It has a mean 0.
2 It is symmetrical about the mean.
3 In general, it has a variance greater than 1, but the variance
approaches 1 as the sample size becomes large where the degree of
freedom df > 2 and the variance is df /(df − 2).
4 The variable t ranges from −∞ to ∞.
5 Compared to the normal distribution the t distribution is less peaked
in the center and has higher tails.
6 The t distribution approaches the normal distribution as n − 1
approaches to infinity. USC - ESPoliquit
24 / 31
3.3 The t Distribution

When sampling is from a normal distribution whose standard deviation, σ,

is unknown, the 100(1 − α) percent confidence interval for the population
mean, µ, is given by
s
x̄ ± t(1−α/2) √ .
n

USC - ESPoliquit
25 / 31
3.4 Determination of Sample Size for
Estimating Means
If sampling is to be with replacement, from an infinite population, or from
a population that is sufficiently large to warrant our ignoring the finite
population, the sample size,

z 2σ2
n=
(se)2
where se is the standard error.
When sampling is without replacement from a small finite population, the
finite population correction is required and the sample,

Nz 2 σ 2
n= .
(se)2 (N − 1) + z 2 σ 2
There are three items of information needed: (1) the desired width of the
confidence interval for z, (2) the level of confidence desired for z, and (3)
the magnitude of the population variance for σ 2 . USC - ESPoliquit
26 / 31
3.4 Determination of Sample Size for
Estimating Means
Example
Assume that the researcher would like an interval about 10 grams wide;
that is, the estimate should be within about 5 grams of the population
mean in either direction. In other words, a margin of error of 5 grams is
desired. Let us also assume that a confidence coefficient of 0.95 is decided
on and that, from past experience, the researcher feels that the population
standard deviation is probably about 20 grams.

z = qnorm(1-0.05/2);z

[1] 1.959964
sigma = 20;SE = 5
ceiling((zˆ2*sigmaˆ2)/SEˆ2)

[1] 62
USC - ESPoliquit
27 / 31
3.4 Determination of Sample Size for
Estimating Proportions

Assuming that the random sampling and conditions warranting

approximate normality of the distribution of p̂ leads to the following
formula for n sampling is with replacement, when sampling is from an
infinite population, or when the sampled population is large enough to
make use of the finite population correction unnecessary,

z 2 p(1 − p)
n= .
(se)2

If the finite population correction cannot be disregarded, the proper

formula for n is
Nz 2 p(1 − p)
n=
(se)2 (N − 1) + z 2 p(1 − p)

USC - ESPoliquit
28 / 31
3.4 Determination of Sample Size for
Estimating Proportions

Example
A survey is being planned to determine what proportion of families in a
certain area are medically indigent. It is believed that the proportion
cannot be greater than 0.35. A 95 percent confidence interval is desired
with d=0.05. What size sample of families should be selected?

z = qnorm(1-0.05/2)
p = 0.35
SE= 0.05
ceiling((zˆ2*p*(1-p))/SEˆ2)

[1] 350

USC - ESPoliquit
29 / 31
References

Walpole, Ronald E. Probability and Statistics for Engineers and

Scientists. New York, USA: MacMillan Publishing Co., Inc., 1985
Daniel, Wayne W. & Cross, Chad L., BIOSTATISTICS A Foundation
for Analysis in the Health Sciences. 10th ed., John Wiley & Sons, Inc,
2013(text)
Sprinthall, Richard C., Basic Statistical Analysis. 8th ed., USA:
Progressive Publishing Alternatives, 2007
Downie, N. M. and R. W. Heath. Basic Statistical Methods. New
York, USA: McGraw-Hill Publishing [Link]., 1977
Walpole, Ronald E. Introduction to Statistics. 3rd ed., New York,
USA: MacMillan Publishing Co., 1981
Snedecor and Cochran. Statistical Methods. Lexington, Mass.:
Addison-Wesley Publishing Com Inc., 1984.
Joaquim P. Marques de Sá, Applied Statistics Using SPSS,
STATISTICA, MATLAB and R
USC - ESPoliquit
30 / 31
USC - ESPoliquit
31 / 31