Chapter 2: Simple Random Sampling

Simple random sampling: Simple random sampling is the technique of drawing a sample
in such a way that each unit of the population has an equal and independent chance of being
selected/included in the sample.
Simple random sampling without replacement: If a unit is selected and noted and not
returned to the population and this procedure is repeated till n distinct units are selected,
then it gives rise to a simple random sample of n units, which is called the simple random
sampling without replacement.

Simple random sampling with replacement: If a unit is selected and noted and then
returned to the population before the next drawing is made and this procedure is repeated
n times, then it gives rise to a simple random sample of n units, which is called the simple
random sampling with replacement. Occasionally, sampling with replacement is referred
to as unrestricted sampling.
Properties of simple random sampling:
(1) In sampling with replacement, the probability of selecting a specified element ui on
each of the n draws from a population of N elements is: ( )
p ui =
1
N
; i = 1, 2,..., N .

(2) For sampling without replacement (SRSWOR), the possible number of different
N
combinations of n elements formed from N elements is  . For sampling with
n 
replacement (SRSWR), the possible number of different combinations of n elements
formed from N elements is: N n .

(3) In sampling without replacement, the probability of selecting a specified element ui on

any draw is equal to the probability of selecting it on the first draw. To verify this, we note
1
that the probability that the specified unit is selected on the first draw is: p1 = . The second
N
draw is conditional upon the first draw, since the sample is being drawn without
replacement. Thus, the probability that the specified unit is selected on the second draw is
clearly the product of the probability of the event A , which is not selected at the first draw
and the conditional probability of the event B , which is selected at the second draw. That
 N − 1  1  1
is, p2 =   = . Similarly, the third draw is conditional upon the two previous
 N  N − 1  N
draws.
Thus, the probability that the specified unit is selected on the third draw is clearly the
product of the probability of the event A and B , which are not selected at the first two
 Simple Random Sampling

draws and the conditional probability of the event C , which is selected at the third draw.
 N − 1  N − 2  1  1
That is, p3 =    =
 N  N − 1  N − 2  N

Similarly, for the r th draw, the probability is:

 N − 1  N − 2   N − r + 1  1  1
pr =   . . .   =
 N  N − 1   N − r + 2  N − r + 1  N

This shows that the probability of selecting a specified element ui on any draw is equal to
the probability of selecting it on the first draw irrespective of whether the elements are
drawn with replacement or without replacement.

(4) In simple random sampling of n elements drawn from a population of N elements, the
n
probability of any of the elements being selected is .
N
To verify this, we note that the probability of any of the elements being selected is the sum
of the probabilities that is selected at the first draw, second draw and so on. That is,
1 1 1 n
+ +. . . + = .
N N N N

(5) In simple random sampling, each possible combination of n different elements drawn
from a population of N elements has the same probability of being selected for the sample
1
and is equal to: .
N
 
n 
To verify this, we note that at the first draw, the probability that one of the n specified
n
units will be selected is . At the second draw, the probability that one of the remaining
N
n −1
( n − 1) specified units will be selected is . At the n th draw, the probability is
N −1
n − ( n − 1) 1
= .
N − ( n − 1) N − ( n − 1)

N
So, there are   combinations of elements, any one of which may constitute the sample.
n 
Then, each combination has the probability of selection equal to

n n −1 1 n ( n − 1) . . . 1 ( N − n ) ! = n! ( N − n ) ! = 1 1
  . . .  = = . In
N N −1 N − ( n − 1) N ( N − 1) . . . N − ( n − 1) ( N − n ) ! N! N! N
n ! ( N − n ) !  n 

arriving at the above probability, we have assumed that sampling was done without
replacement.

Dr. Mohd. Muzibur Rahman 2

Professor, Department of Statistics and Data Science, JU
 Simple Random Sampling

(6) When sampling is done without replacement, the sample size n cannot exceed the
population size N , but when sampling is done with replacement, the sample size n can be
of any size.

Estimation of population mean and total: The frequent objective of a sample survey is
to estimate the population mean and population total to draw inference about a population
from information contained in a sample. Suppose, the values obtained for any specified
items in the population of size N is denoted by: y 1 , y 2 , . . . , y N . The corresponding values
for the units in the sample of size n is denoted by: y1, y 2 , . . . , y n . The formula used for the
totals and means of the population and sample are summarized below:

N n
Y =  y i = Population total y =  y i = Sample total
i =1 i =1
N n
 yi Y
 yi y
Y = i =1 = = Population mean y= i =1 = = Sample mean
N N n n

= Y = N Y = y = n y = Yˆ = N y

N
 (yi −Y )
2

 2
= i =1
N
N n
 (yi −Y )  (yi − y )
2 2

S2= i =1 s2 = i =1
N −1 n −1

= ( N − 1) S 2 = N 2

Theorem: The sample mean y for a simple random sample of size n is an unbiased
estimator of the population mean Y . Symbolically, E ( y ) = Y .
Proof:
n N  n  N
1
 y i p ( y i )
n n n
 yi  ( ) E yi
i =1  i =1 
   y i 
N  Y
  i =1  i =1 nY
y= i =1 = E ( y ) = i =1 = = = i =1 = =Y
n n n n n n

Yˆ = N y ( )
= E Yˆ = N E ( y ) = N Y = Y

Theorem: In simple random sampling of n units without replacement (SRSWOR) from a

population of N units, the variance of the sample mean is given by:

𝑆2 𝑁−𝑛 1 1
we get 𝑉𝑎𝑟(𝑦̅𝑛 ) = . = (𝑛 − 𝑁) . 𝑆 2 ----------- (1)
𝑛 𝑁
𝑛 𝑆2 𝑆2
𝑉𝑎𝑟(𝑦̅𝑛 ) = (1 − 𝑁) . = (1 − 𝑓).
𝑛 𝑛

Dr. Mohd. Muzibur Rahman 3

Professor, Department of Statistics and Data Science, JU
 Simple Random Sampling

𝑛
Note: 𝑓 = 𝑁 is called the sampling fraction and consequently, The factor (1 − 𝑓) is called
the finite population correction factor (f.p.c.). If the population size is N is very large or if
𝑛
n is small compared with N then 𝑓 = 𝑁 → 0 and consequently 𝑓. 𝑝. 𝑐. → 1.

The standard error (S.E) of the sampling distribution of 𝑦̅𝑛 is given by:
𝑁−𝑛 𝑆
𝑆. 𝐸. (𝑦̅𝑛 ) = √ .
𝑁 √𝑛
Usually S is not known and in that case we replace 𝑆 2 by its unbiased estimate 𝑠 2 and we
get
𝑁−𝑛 𝑠 𝑠
𝐸𝑠𝑡. [𝑆. 𝐸(𝑦̅𝑛 )] = √ . = √(1 − 𝑓).
𝑁 √𝑛 √𝑛
If we consider the sampling is done with replacement (SRSWR), from the population with
variance 𝜎 2 , then
𝑛 𝑛
1 1
𝑉𝑎𝑟(𝑦̅𝑛 ) = 𝑉𝑎𝑟( ∑ 𝑦𝑖 ) = 2 ∑ 𝑉𝑎𝑟 (𝑦𝑖 )
𝑛 𝑛
𝑖=1 𝑖=1

the covariance terms vanish since in SRSWR all the draws are independent and
consequently 𝑦𝑖 (𝑖=1,2,3,…….𝑛) are independently and identically distributed (i.i.d.) with the
1 𝜎2
same variance 𝜎 2 . So, the variance under SRSWR is: 𝑉𝑎𝑟 (𝑦̅𝑛 ) = 𝑛2 ∑𝑛𝑖=1 𝜎 2 = …. (2)
𝑛
1
But 𝜎 2 = 𝑁 ∑𝑁 ̅ 2 2 2
𝑖=1(𝑌𝑖 − 𝑌𝑁 ) or, 𝑁𝜎 = (𝑁 − 1)𝑆 , now from the equation (2) we can write

𝑁−1
∴ 𝑉𝑎𝑟(𝑦̅𝑛 ) = . 𝑆2 ---------------(3)
𝑛𝑁

Now, comparing equation (1) and (3) we observed that the variance of the sample mean
under sampling with replacement (SRSWR) is greater than variance of the sample mean
under sampling without replacement (SRSWOR).

Example: Consider a population of 6 units with values 1, 2, 3, 4, 5 and 6. Write down all
possible samples of 2 (without replacement) from this population and verify that sample
mean is an unbiased estimate of the population mean. Also calculate it’s sampling variance
and verify that:
i) it aggress with the formula for the variance of the sample mean
ii) this variance is less than the variance obtained from sampling with replacement.
Solution: Let us consider
Y 1 2 3 4 5 6
Y2 1 4 9 16 25 36

Dr. Mohd. Muzibur Rahman 4

Professor, Department of Statistics and Data Science, JU
 Simple Random Sampling

We have,  Y = 21 and  Y 2
= 91 .
1 21
Again, we have population mean, Y =
N
 Y=
6
= 3.5
1 1 1
Popn. mean square, 𝑆 2 = 𝑁−1 ∑(𝑌 − 𝑌̅)2 = 𝑁−1 [∑ 𝑌 2 − 𝑁𝑌̅ 2 ] = 5 [91 − 6 × 12.25] = 3.5
N −1 2 5
Population variance  2 = S =  3.5 = 2.917
N 6
The total number of samples of size n=2 from a population of N=6 units is 6 C2 = 15 , as
enumerated in the following table:
Sample No. Sample val. (y) Sample mean ( y ) y −Y ( y − Y )2
1 (1, 2) 1.5 -2.0 4.00
2 (1, 3) 2.0 -1.5 2.25
3 (1, 4) 2.5 -1.00 1.00
4 (1, 5) 3.0 -0.5 0.25
5 (1, 6) 3.5 0.0 0.0
6 (2, 3) 2.5 -1.0 1
7 (2, 4) 3.0 -0.5 0.25
8 (2, 5) 3.5 0 0
9 (2, 6) 4.0 0.5 0.25
10 (3, 4) 3.5 0 0
11 (3, 5) 4.0 0.5 0.25
12 (3, 6) 4.5 1.0 1.0
13 (4, 5) 4.5 1.0 1.0
14 (4, 6) 5.0 1.5 2.25
15 (5, 6) 5.5 2.0 4.0

Total 52.5 0 17.50

E( y ) =
y i
=
52.5
= 3.5 = Y
N
Cn 15
Hence the sample mean is an unbiased estimate of the population mean.
2
1 15
V ( y) = 
15 i =1
( yi − Y ) =
17.50
15
= 1.167.....(1)

Dr. Mohd. Muzibur Rahman 5

Professor, Department of Statistics and Data Science, JU
 Simple Random Sampling

(i) In SRSWOR, the variance of the sample mean is given by the formula:
𝑁−𝑛 6−2
𝑉(𝑦̅) = 𝑆 2 = 6×2 × 3.5 = 1.167, which is equal to the equation (1).
𝑁𝑛

2 2.917
(ii) We have in SRSWOR, V ( y ) = 1.167 and in SRSWR, V ( y ) = = = 1.458 .
n 2
Hence V ( y ) SRSWR  V ( y ) SRSWOR .

Example: Consider a population of 4 units with values 1, 2, 3, 4. Write down the all
possible samples 2 (with replacement) from the population. Then find the population mean
and population variance. Also compute the mean and standard deviation of the sampling
distribution of mean.
Solution: Let us consider
Y 1 2 3 4
Y2 1 4 9 16

We get,  Y = 10 and  Y 2
= 30 .
1 10
Now, the population mean, Y =
N
Y = 4
= 2.5

Popn. mean square,

1 1 1 5
𝑆2 = ∑(𝑌 − 𝑌̅)2 = [∑ 𝑌 2 − 𝑁𝑌̅ 2 ] = [30 − 4 × (2.5)2 ] = = 1.67
𝑁−1 𝑁−1 3 3
N −1 2 3 5
And population variance  2 = S =  = 1.25
N 4 3
The total number of samples of size n=2 from a population of N=4 units is N n = 42 = 16 ,
as enumerated in the following table:
Sample No. Sample val. (y) Sample mean (y) y −Y ( y − Y )2
1 (1, 1) 1.0 -1.5 2.25
2 (1, 2) 1.5 -1.0 1
3 (1, 3) 2.0 -0.5 0.25
4 (1, 4) 2.5 0.0 0.0
5 (2, 1) 1.5 -1.5 2.25
6 (2, 2) 2.0 -0.5 0.25
7 (2, 3) 2.5 0.0 0.0
8 (2, 4) 3.0 0.5 0.25
9 (3, 1) 2.0 -0.5 0.25
10 (3, 2) 2.5 0.0 0.0

Dr. Mohd. Muzibur Rahman 6

Professor, Department of Statistics and Data Science, JU
 Simple Random Sampling

11 (3, 3) 3.0 0.5 0.25

12 (3, 4) 3.5 1.0 1.0
13 (4, 1) 2.5 0.0 0.0
14 (4, 2) 3.0 0.5 0.25
15 (4,3) 3.5 1.0 1.0
16 (4, 4) 4.0 1.5 2.25

Total 40.0 0 10.0

So, the mean of the sampling distribution of mean under SRSWR is:

E( y ) =
y i
=
40
= 2.5 and the variance under SRSWR is:
n
N 16
N −1 2 4 −1 5 5
V ( y) = S =  = = 0.625
Nn 4 2 3 8
Therefore, the mean and the variance of the sampling distribution of mean is 2.5 and
0.625 respectively.

Advantages of simple random sampling:

1) It is simple to conceptualize.
2) It provides foundation for much of statistical theory.
3) It provides a basis to which other methods can be compared.
4) Since, the sampling units are selected at random giving each unit an equal chance of
being selected, the element of subjectivity or personal bias is completely eliminated.

Disadvantages of simple random sampling:

1) It requires an up-to-date frame from which samples are to be drawn. So, all units in the
population must be identified and labeled prior to sampling. This process is potentially so
expensive and time consuming that it becomes unrealistic to implement.
2) Sampled individuals may be so widely dispersed that visiting each selected individual
may be extremely expensive and time consuming.

3) Certain subgroups in the population may be totally overlooked or may be over

represented in the sample as a result of chance factor. In either case, the estimated
parameters are likely to be in error.
4) When the population measurements vary considerably in size, then simple random
sample produces larger variances than other methods of sampling.

Dr. Mohd. Muzibur Rahman 7

Professor, Department of Statistics and Data Science, JU