Chapter Five

5. Sampling and Sampling Distributions

5.1. Introduction
Sample and Population: A population is the collection of all the elements of interest whereas a sample
is a subset of the population.
Example: In s study of academic performance of first year economics students. The GPA of all second
year economics students is the population. The GPA of some of the second year economics students is the
sample.
Population can be finite and infinite population. Finite population mean that the population has limited
size, that is to say, there is a whole number (N) that tells us how many items are there in the population.
Infinite population is a population in which it is theoretically impossible to observe all the elements.
Sometimes it is possible and practical to examine every person or item in the population we wish to study.
We call this complete enumeration or census.
Why “Sample” the Population? Why not study the whole population?
• The physical impossibility of checking all items in the population
• The cost of studying all the items in a population
• The sample results are usually adequate
• Contacting the whole population would often be time-consuming
• The destructive nature of certain tests (e.g., study of light bulb life)
Types of Sampling:
In statistics, there are two methods of selecting samples from populations: Random or probability
sampling, and Non-random, non-probability or judgment sampling.
Probability (Random) Sampling: - is sampling when all items (i.e., each element) in the population have a
chance of being chosen in the sample and the probability of each element of the population included in
the sample is known.
Non-probability (Non-random/Judgment) sampling:- is a sampling methodology where personal
knowledge and opinion play major role in identifying which elements of the population are to be included
in the sample, and the probability of an element from the population to be included in the sample is not
known.
Statisticians advocate Probability Sampling (not judgment sampling). A probability sample is a sample
selected in such a way that each item or person in the population being studied has a known likelihood of
being included in the sample. If we use judgment sampling we will have no idea about the accuracy of
our estimates since we have no idea about the quality of judgments. Probability sampling enables us to
construct probabilistic error bounds. The aim of sampling is to get a sample, which is representative of

1|Page
the population.
Methods of Probability Sampling
a. Simple Random Sample (SRS): A sample formulated so that each item or person and each subset in
the population has the same chance of being included. (e.g., from N items, probability that any n is
selected is equal to1/ N .) A simple way to implement this is to use a lottery or computer program.
For example: we can mark N cards and write names of items on these cards shuffle the cards and
select n cards. This will yield a simple random sample of size n.
Note that: a sample can be made with replacement or without replacement. With replacement means a
sample element picked, should be put back immediately before the second element is chosen but not in the
case of without replacement.
b. Systematic Random Sampling (SysRS): The items or individuals of the population are arranged in
some order. A random starting point is selected (by lottery) and then every k th member of the
population is selected.
Example:- If there are N=1000 stores along an avenue and we want to select n=100 stores in the
sample, k =N /n∨10. We shuffle only the first k, and select one, say #4. Now on we systematically
select stores by adding k, 2k, 3k, 4k etc to 4. So a systematic sample will have store #4, 14, 24, 34, 44,
54 etc.

c. Stratified Random Sampling (StrRS)): A population is first divided into subgroups, called strata, and
a sample is selected from each stratum (e.g., 70% males, 30% females). If a sample of 10 is selected,
(n=10) 70% of n =7, so select 7 males and 3 females. In general, if N is population size, N1 is stratum
1(female), N2 is stratum 2 (males), n is sample size desired. Sample should have (N1/N)*n from
stratum 1 and so on. Thus Females number in sample is (N1/N)*n, Males in the sample is (N2/N)*n
Example: Population has 25 students of whom 15 are white and 10 black. A stratified sample of size 10
should have how many whites / blacks?
Answer: Let N=population size, N1=blacks=10, N2=whites =15, n=sample size=10. Then the sample

will have ( NN1 )× n=( 1025 ) ×10=4 blacks and ( NN2 )× n=( 1525 ) ×10=6 whites. Verify that 6+4=10.
We have a representative sample
d. Cluster Sampling: is sampling in which one divides the elements in the population in to a number of
clusters or groups. One then begins by choosing at random a sample of these clusters, after which a
simple random sample of the elements in each chosen cluster is selected. Sometimes, this is referred
as two stage cluster sampling. It works if clusters are as heterogeneous as the population.
Most of the time students face difficulty in differentiating stratified and cluster sampling. The main
distinguishing criteria of stratified from cluster sampling is that in the case of stratified sampling the

2|Page
population is divided in to well-defined groups, where each group has homogeneity with in itself but
wider heterogeneity (or variation) among the groups. In the case of cluster sampling, the situation is the
reveres for stratified sampling.
Why probability sampling? It uses for statistical inference.
5.2. Sample statistics
Mathematically, we can describe samples and populations by using measures such as the mean,
median, mode, and standard deviation. When these terms describe the characteristics of a sample,
they are called statistics. When they describe the characteristics of a population, they are referred
parameters.
Example: If the mean CGPA of all first year economics students is 3.20. In this case, 3.20 is the
characteristics of the population “All first year economics students” and is termed as population
parameter. On the other hand, if we say that the mean CGPA of the first year economics students
at Dire Dawa University is 3.20, we are using 3.20 to describe a characteristic of the sample
“First year economics students at Dire Dawa University”. In this case, 3.20 would be a sample
statistics.
If we are convinced that the sample statistics are accurate estimate of the population
characteristics, we could use sample statistics to estimate the population parameter without
measuring the entirety of the items under study. In order to be consistent, statisticians use lower
case roman letters to denote sample statistics, and Greak or capital letters to denote population
parameters.
Name of symbol Population (parameter) Sample (Statstic)
Mean , average x μ
Variance, standard deviation 2 2
s ,s σ ,σ
Correlation coefficient R ρ
Slope of least square regression b1 β1
Intercept of least square regression B0 β0
Number (in sample, in population) N N

A sampling error is the difference between a sample statistic and its corresponding parameter. We can
make probabilistic statements about this sampling error only if we have a probability sample (not
judgment sample).
5.3. Sampling From Normal Distribution
Sampling distribution is a probability distribution of all the values of sample statistics.

3|Page
In general, sampling distribution is for any sample statistic (mean, median, mode, standard deviation, etc)
defined over a sample space consisting of all possible samples of size n from the available population of
size N.

a. Distribution of Mean and Proportions

Sampling Distribution of the Sample Mean: - The sampling distribution of the sample means is a
probability distribution consisting of all possible sample means based on specified sample sizes selected
from the population. The sampling distribution yields the probability of occurrence associated with each
sample mean over the set of all possible sample mean numbers.
Example:-
A firm has five branches of product distribution (A, B, C, D, and E). Assume at the end of last week the
amount of distribution in thousands at each branch is given by
A 22, B 26, C 30, D 26, and E 22.
If n=2, two partners are selected randomly, how many different samples are possible? This is the
combination of 5 objects taken 2 at a time. That is, 5C2= 5!/(2!3!)=10. There are 10 possible samples.
Note that: The number of different samples of size n that can be drown without replacement from
N!
a population of N elements equals to N Cn =
( N−n ) ! × n!
Ten sample means are given below: (e.g. if the sample has A and B, sample mean is 24)

AB 24, AC 26, AD 24, AE 22, BC 28, BD 26, BE 24, CD 28, CE 26, DE 24

Exercise: draw a picture with freq on vertical axis for sampling distribution of means. Note above that

mean of A and C is 26, B and D is 26 and mean of C and E is also 26, which means the x =26 repeats
itself three times (has frequency 3). We find following list of frequencies:

x =22 with freq= 1, probability 1/10=0.1

x =24 with freq= 4, probability 4/10= 0.4

x =26 with freq= 3, probability 3/10=0.3

x =28 with freq= 2, probability 2/10=0.2. This is almost the sampling distribution of means

4|Page
 Sampling distribution
0.5
0.4
0.3
probability 0.2
0.1
0
22 24 26
sample mean

Total freq uency =10. If we divide individual frequencies by total frequency we get “relative frequency”
or probability. These probabilities add up to one, so we have a probability distribution. The above
information says that the probability that sample mean is 22 is 2 out of 10 or 0.2. The sampling
distribution is simply this probability distribution defined over all possible samples of size n from the
population of size N. In the real world problems N will be large (e.g. 80 million Ethiopia population) and
n will be also be large (e.g., 1000 people surveyed) and (N C n) will be astronomical number. Then the
sampling distribution can only be imagined. We have chosen a simple example of N=5, n=2 so that the
entire sampling distribution can be explicitly computed and visualized. This is a sampling distribution of
all possible sample means. Now the random variable is x , it is no longer just X.
What are the properties of the sampling distribution of sample means x ?
 Mean of the sample means is equal with the population mean: For our simple example we can
explicitly calculate the mean of means or Expected value of means or E(x )=¿
· The mean of the sample means is obtained by weighting each sample mean by its frequency=
[(22)(1) + (24)(4) + (26)(3) + (28)(2)]/10=25.2 Since we know the value of every observation in
the population in this (impractical) simple example, we have the directly calculated population
mean  = (22+26+30+26+22)/5 = (25.2). Note that in the real world we usually cannot find,
we can only make inferences about it from sample mean x . Observe that the grand mean of all 10
sample means (25.2) is equal to the population mean (25.2).
· Since E( x )= , we say that Sample mean x is an UNBIASED estimator of population mean  .
In general, such verification is difficult and one needs to use advanced theory.
 Give the population mean (μ)., population standard deviation (σ ), the sample size (n) and
population size (N); the standard deviation of the sample mean(σ x ) is given as follows:
σ
σ x= ………………………………………………. For infinite population
√n
5|Page
 σ x=
σ
×
√
N−n
√ n N−1
………………………………… For finite population

A population is said to be infinite when it is not possible to list or count all the elements included in the
population, (i.e., when the elements are unlimited). Or, in the cases when the elements in the population
are limited, the population may be considered as infinite when the sample size is small and as rule of
thumb statisticians consider the population as infinite when n ≤ 5 % of N and a population is said to be

finite whenn ≥ 0.05 N . The value

√ N −n
N −1
is referred as finite population correction factor.

 The sampling distribution of the mean is normally distributed regardless of the population from
which it is drawn.
Whenever the population has a normal probability distribution, the sampling distribution of the mean ( x
) is a normal probability distribution for any sample size.
In this case E(x )=μ(x )=μ
σ n
σ x= For ≤ 0.05 and
√n N

√
σ N−n for n ≥ 0.05
σ x= ×
√n N−1 N

It is customary to distinguish between usual standard deviation (SD) and that of a sampling distribution
(SE).The standard error of the sample means is the standard deviation of the sampling distribution of the
sample means. The range spread of the sampling distribution decreases as n increases. In fact the variance
can be proved to be inversely proportional to n.

Sampling distribution of the mean from Unknown or Non-normal population: central Limit Theorem
(CLT) is one of the most important theorems in statistics. In selecting simple random samples of size n
from a population, the sampling distribution of the sample mean ( x ) can be approximated by a normal
probability distribution as the sample size becomes large. The significance of the central limit
theorem is that it permits us to use sample statistics to make inferences about population
parameters without knowing anything about the shape of the frequency distribution of that
population other than what we can get from the sample.
Generally Polya in 1920's showed that EVEN IF x is NOT NORMAL, if n ≥ 30 the process of averaging
(is so helpful) that it yields normality of the sampling distribution of ( X ) with the variance given below.
For a population with a mean  and variance 2, the sampling distribution of all possible means of all
possible samples of size n generated from that population will be approximately normally distributed

6|Page
X N {,(2/n)[(N n)/(N 1)]} assuming sufficiently large n. (n 30). If N is large the finite population
correction term [( N n)/(N 1)] is close to 1 and can be ignored. Then, this formula simplifies to

X N {,(2/n)

Example1: A population of 100 elements has a mean of 19.2 and standard deviation of 1. What are the
mean and standard deviation of the sampling distribution of the mean for samples of size 25?
Solution:
E ( x )=μ ( x )=μ=19.2
n 25
= =0.25≥ 0.05
N 100

√ √
100−25 1 √ 75
σ N−n = 1
→ σ x= × × = × ≅ 0.174
√n N−1 √ 25 100−1 5 √ 99
Example 2: A library Checks out an average of μ=320 books per day, with a standard deviation of
σ =75 books. Consider a sample of 30 days of operation, with x being the sample mean number of books
checked out per day. What is the probability that the sample mean for 30 days will be between 300 and
340 books?
Given: μ=320, σ =75
Required: P ¿)
Solution: This is the case of sampling distribution of x . Since the population is normal, the sampling
distribution of x is also normal. So, using the normal probability distribution way of computing
probability; P ¿) should be converted in to standard normal probability distribution.
σ 75
σ x= = =13.69
√ n √30
Let x 1=300 and x 2=340
x1−μ 300−320
z 1= = =−1.46
σx 13.69
x2 −μ 340−320
z 2= = =1.46
σx 13.69
P(300< x<340)=P (−1.46< z <1.46) From the normal distribution table this is equal to:
P (−1.46< z <1.46 ) =2× 0.4279=0.8558
Example 3: The distribution of annual earnings of all economics graduates with zero years’ experience is
skewed negatively, as shown in figure (a) below. This distribution has a mean of 19,000 Birr and a

7|Page
standard deviation of 2,000 Birr If we draw a random sample of 30 fresh economics graduates, what is the
probability that their earnings will average more than 19,750 Birr annually?
Given: μ=19 ,000 , σ =2 , 000
Required: P ¿)
Solution: In order to answer this question, first let’s calculate the standard error of the mean.

σ 2000
σ x= = =365.148
√ n √30

(
P ¿)= P z >
19 ,750−19 , 000
365.148 )
= p ( z >2.05 )

¿ 0.5−p ( 0< z< 2.05 )

¿ 0.5−0.4798=0.0202
Sampling distribution of Proportion:

Consider a population of N = 5 given numbers 3, 6, 9, 12, and 15. Let’s take even numbers, the
proportion of even numbers is 2/5 = 0.4. Consider a sample of size 3 (n = 3) that are drawn from the
population the samples, sample proportions are given in the following table:
Sample 3,6,9 3,6,12 3,6,15 3,9,12 3,9,15 3,12,15 6,9,12 6,12,15 6,9,15 9,12,15

Sample 1/3 2/3 1/3 1/3 0/3 1/3 2/3 2/3 1/3 1/3
proportion
Given table above, we can construct the probability distribution of the sample proportions as shown in
table below.
Table: - Probability Distribution of sample proportion ( P)
Probability of P 0.1 0.6 0.3
Sample Proportion 0/3 1/3 2/3
Sampling distribution of the proportion is the probability distribution of all possible values of the
sample proportion ( P).
0 1 2
The mean of the sample proportion is ¿ 0.1 × + 0.6 × + 0.3× =0.4
3 3 3
Properties of the sampling distribution of the proportion (P):
 The expected value of the sample proportion ( p) is equal to the population proportion P.
Symbolically: E ( p) = P. Where E ( p) = is the expected value of the random variable ( p ).
P = is the population proportion.

8|Page
  Just as with the standard deviation of the sample means¿), the standard deviation of the
sample proportion (σ p) also depends on whether the population is finite or infinite. It
follows that the standard deviation of the sample proportion is:

σ p=
√ P(1−P)
n
…………………………………For infinite population (n> 0.05 N ¿

σ p=
√ P(1−P)
n
×
√N −n …………………………For finite population (n ≤ 0.05 N ¿
N −1
Where σ p is the standard deviation of (P) and P is any given population proportion.
When the population from which the samples are drawn is normal, the form of the sampling distribution
of the sample proportion is also normal. In the other case, when the population is non-normal or unknown
applying the central limit theorem the sampling distribution of the sample proportion ( P ) can be
approximated by normal probability distribution provided that these two condition are fulfilled:
nP ≥ 5 and n ( 1 – p ) ≥5 Where n is the sample size, and P is any given population proportion.
Example: A population proportion is 0.4. A simple random sample of size 200 will be taken and the
sample proportion P will be used to estimate the population proportion. What is the probability that the
sample proportion will be within ±0.03 of the population proportion?
Given: P=0.4, n=200
Required: P(0.37 < P<0.43)
Solution:

σ p=
√ 0.4 (0.6)
200
≅ 0.035

Here let P1=0.37 and P2= 0.43

P 1−P 0.37−0.4
z 1= = =−0.857
σP 0.035
P 2−P 0.43−0.4
z 2= = =0.857
σP 0.035
P ( 0.37< P< 0.43 )= p ( z 1< Z < z 2 )
¿ P(−0.857 <Z < 0.857)
From normal distribution table P ( Z <0.86 )=0.3051 ¿
P (−0.857< Z <0.857 )=2× 0.3051=0.6102
b. Distribution of the sample Variance: Let x1, x2, . . . , xn be a random sample from a population. The
sample variance is

9|Page
 n
1
2
s= × ∑ (x −x)
n−1 i=1 i
The square root of the sample variance is called the sample standard deviation. The sample variance is
different for different random samples from the same population.

 The Sampling Distribution of Sample Variances ( s2) has mean σ2

2
E( s ¿¿ 2)=σ ¿

 If the population distribution is normal then

4
2σ
Var (s¿¿ 2)= ¿
n−1

 If the population distribution is normal then

( n−1 ) s2
σ2

Has a 2 distribution with n−1 degrees of freedom

c. Small Sample Distributions

In the case of small samples (n ≤ 30), the distribution of the mean x does not depend to a considerable
extent on the form of the population distribution with the central limit thermo no longer applicable, more
information concerning the population is required for the development of statistical procedures. The
appropriate method of inference depends on the restriction meet by the population distribution.
x−μ σ
 If we assume that the population distribution is normal, then z= where σ x = has a
σx √n
standard normal distribution.
 But if the population standard deviation is unknown , we consider the random variable obtained
by replacing the unknown σ by the sample standard deviation s x, giving
x−μ s
t= , where s x =
sx √n
This random variable does not follow a standard normal distribution. However its distribution is
known and is in fact a member of family of distributions called student`s t.

10 | P a g e
 Given a random sample of n observations, with mean x and standard deviation s x , from a normal
x−μ
population with mean μ, the random variable t= follows a student`s t distribution with n-1
sx
degree of freedom.

Note that: x=
1
× x 2 ∑ (xi −x)
∑ i, and s = n−1
2

11 | P a g e