Sampling Distributions

Content
1. The Concept of a Sampling Distribution

2. Properties of Sampling Distributions:

Unbiasedness

3. The Sampling Distribution of the Sample

Mean and the Central Limit Theorem

2
Statistic and Sampling Distributions

Section 5.1 3
 Parameters of a
Population Distribution
➢Probability distribution (Chapter 4)
• Mathematical models for the population
• Random variables defined on sample space Ω
• Sample space contains all possible outcomes

4
 Parameters of a
Population Distribution
➢ Parameter (population level)
• Numerical quantities that describe probability distributions.
• It is based on all the observations in the population (either
unknown or given by the assumption)

5
 Parameters of a
Population Distribution
➢Parameters (discrete distribution)
• Bernoulli: ( )
• Binomial: ( , )
• ……

6
𝐵
𝑒
𝑟
𝑝
𝐵
𝑖
𝑛
𝑛
𝑝
 Parameters of a
Population Distribution
➢Parameters (continuous distribution)
2
• Normal: ( , )
• Uniform: U( , )
• ……

7
𝑁
𝜇
𝜎
𝑎
𝑏
 Parameters of a
Population Distribution
➢Parameters (any distribution)
• Not limited to parameter that shows up in the
definition
• – mean, – variance, ……

8
𝜇𝜎
 Sample Statistics
➢Sample Statistic
• A number computed from a sample
• Is used to make inference about the parameters
of a population

➢ Statistics is a sample quantity

➢ Parameter is a population quantity

9
 Common Statistics & Parameters
Sample Statistic Population Parameter

Mean X µ

Standard
Deviation S σ

Variance S2 σ2

Sample statistics are based on (sample size) random

observations, thus they are also random variables.
10
𝑛
 Statistics & Parameters
Common Distributions
Distribution Population Sample Statistics
Parameters

Sample statistics are based on (sample size) random observations ,

thus ^, ^, ^, ^, ^ , ^ are also random variables.
2
11
𝑏
𝜎
𝒏
𝑎
𝑛
𝑝
𝜇
𝑛𝜴
 Sample Statistics are Random

➢ A sample statistic is
a random variable
• E.g., sample mean
+…+
¯ = 1
• : independently and
identically distributed
random variable from
the same population
𝑛
𝑋
12
𝑖
𝑛
𝑋
𝑋
𝑋
 Distribution of a Sample Statistics

➢ Sampling distribution:
• Probability distribution of
sample statistics
• Sampling distribution of
sample mean:
✓ List of all possible
✓ [ ¯ , ( ¯ )] pairs

(60 ≤ ¯ ≤ 70) = ???

13
𝑝
𝑋
𝑋
𝑃
𝑋
 Example: Coin-Tossing = 2
➢ Toss a fair coin once
➢Let be the number of heads observed.
➢A sample of = 2 tosses.
➢ Find the sampling distribution of the sample mean ¯ .

Steps:
• List all the possible values of ¯
• Find the probability of ¯ taking a value
𝑛
𝑋
𝑋
𝑋
𝑋𝑛
 Example: Coin-Tossing = 2

• P(x̄ = 0) = P(X = 0) = 1/4

• P(x̄ = 0.5) = P(X = 1) = 2/4
• P(x̄ = 1) = P(X = 2) = 1/4
𝑛
 Finding a Sampling Distribution of
Sample Mean

For small sample size

Sample: { }
=1
Steps:
1
List all the possible values of ¯ =
➢ ∑
=1

➢ Find the probability of ¯ taking a value

• Possible outcomes in the sample space
𝑖
𝑛
𝑖
𝑖
𝑋
𝑋
𝑖
𝑋
𝑋
𝑛
𝑛
𝑛
 Example: Coin-Tossing = 5
➢Toss = 5 biased coins (each with head
probability )

{0
1 h h
➢ For = for 1 ≤ ≤ 5
h 1−

1
Sample Mean: ¯ =
➢ ∑
=1
➢ Sampling Distribution ?
𝑖
𝑛
𝑡
𝑎
𝑖
𝑙
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
𝑖
𝑖
𝑋
𝑋
𝑋
𝑛
17
𝑒
𝑎
𝑑
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
𝑝
𝑛
𝑖
𝑛
 Example: Coin-Tossing = 5
{0
1 h h
➢ For = for 1 ≤ ≤ 5
h 1−
1
Sample Mean: ¯ =
➢ ∑
=1

∑
Sample Sum: S = ~ ( = 5, )
➢
=1
x_bar p(x_bar)
0 (when (1-p)^5
0.2 (when C(1,5) p (1-p)^4
0.4 (when C(2,5) p^2 (1-p)^3
0.6 (when C(3,5) p^3 (1-p)^2
0.8 (when C(4,5) p^4 (1-p)
𝑖
𝑖
𝑛
𝑡
𝑎
𝑖
𝑙
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
𝑖
𝑖
𝑖
𝑛
𝑋
𝑋
𝑋
𝐵
𝑋
𝑖
𝑛
𝑛
𝑝
1 (when p^5 18
𝑒
𝑎
𝑑
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
𝑖
𝑛
𝑛
 x_bar p(x_bar)
0 (when (1-p)^5
0.2 (when C(1,5) p (1-p)^4
0.4 (when C(2,5) p^2 (1-p)^3
0.6 (when C(3,5) p^3 (1-p)^2
0.8 (when C(4,5) p^4 (1-p)
1 (when p^5

➢ Expectation of sample mean:

( ¯ ) = ∑ ¯ ( ¯ ) = (population mean)
¯

➢ = {0
1 h h
⇒ ( )=
h 1−
𝑎
𝑙
𝑙
𝑥
𝑡
𝑎
𝑖
𝑙
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
➢ ( ¯ )= ( )
𝑖
𝑋
19
𝑖
𝑖
𝐸
𝑋
𝑥
𝑝
𝑥
𝑝
𝐸
𝐸
𝑋
𝑋
𝐸
𝑋
𝑝
𝑒
𝑎
𝑑
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
 x_bar p(x_bar)
0 (when (1-p)^5
0.2 (when C(1,5) p (1-p)^4
0.4 (when C(2,5) p^2 (1-p)^3
0.6 (when C(3,5) p^3 (1-p)^2
0.8 (when C(4,5) p^4 (1-p)
1 (when p^5
➢ Variance/Standard Deviation of sample mean:
(1 − )
Var( ¯ ) = ( ¯ − ( )) ( ¯ ) =
2
¯
∑
¯
(1 − )
¯ = (¯)= (¯ ) = =

{0
1 h h
➢ =
h 1−
⇒ = (1 − )

( )
( ¯ )=
𝑛
➢
𝑛
𝑎
𝑙
𝑙
𝑥
𝑛
𝑡
𝑎
𝑖
𝑙
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
𝑛
𝑋
𝑖
𝑖
𝜎
𝑠
𝑑
𝑋
𝑉
𝑎
𝑟
𝑋
20
𝑋
𝑋
𝑋
𝑥
𝐸
𝑋
𝑝
𝑥
𝜎
𝑝
𝑝
𝑖
𝑝
𝑝
𝜎
𝑠
𝑠
𝑑
𝑑
𝑋
𝑋
𝑒
𝑎
𝑑
𝑤
𝑖
𝑡
𝑝
𝑟
𝑜
𝑏
𝑝
𝑝
𝑝
 Sampling Distribution
The sampling distribution of a sample statistic:
(calculated from a sample of n measurements) is
the probability distribution of the sample statistic.

21
 Developing a
Sampling Distribution
➢ Assume there is a population …
➢Random variable, D
A B C
is age of individuals

➢ Values of x: 18, 20,

22, 24 (years)

➢ Each value has probability

0.25

22
𝑋
 Developing a
Sampling Distribution
(continued)

Summary Measures for the Population Distribution:

P(y)
.3

.2
.1

0
18 20 22 24 y

Uniform Distribution

23
 Developing a
Sampling Distribution
Now consider all possible samples of size =2

1st 2nd Observation

Obs 18 20 22 24 16 Sample Means

18 18,18 18,20 18,22 18,24

20 20,18 20,20 20,22 20,24 1st 2nd Observation
22 22,18 22,20 22,22 22,24 Obs 18 20 22 24
18
24 24,18 24,20 24,22 24,24
20
22
24 24
𝑛
 Developing a
Sampling Distribution
Sampling Distribution of All Sample Means

Sample Means
16 Sample Means
Distribution
1st 2nd Observation
Obs 18 20 22 24 18
19
18 18 19 20 21 20

20 19 20 21 22 21
22
22 20 21 22 23 23
24
24
25 21 22 23 24
 Developing a
Sampling Distribution
Sampling Distribution of Sample Means

X_bar Probability P(X )

18 1/16 .3
19 2/16
20 3/16 .2
21 4/16
.1
22 3/16
23 2/16 0
18 19 20 21 22 23 24
24 1/16 X
26 (no longer uniform)
 Developing a
Sampling Distribution
➢ Summary Measures of this Sampling Distribution:

18 1/16 ( ¯ ) = ∑ ¯ ( ¯ ) = 21
19 2/16 ¯

(¯)= ( ¯ − ( ¯ )) ( ¯ ) = 1.58
2
∑
20 3/16
21 4/16
¯
22 3/16
23 2/16
24 1/16
𝑎
𝑙
𝑙
𝑥
𝑎
𝑙
𝑙
𝑥
27
𝑠
𝑑
𝑋
𝑥
𝐸
𝑋
𝑝
𝑥
𝐸
𝑋
𝑥
𝑝
𝑥
 Comparing the Population with its
Sampling Distribution
Sample Means Distribution
Population
=2
µ X = 21 σ X = 2.236 E ( X ) = 21 σ X = 1.58
P(x)
p( X )
.3 .3

.2 .2

.1 .1

0 0
18 20 22 24 X 18 19 20 21 22 23 24
X
¯ =
√2
𝑋
28
𝜎
𝑛
𝑋
𝜎
 Simulating a
Sampling Distribution

➢For large sample size

• Use computer program to simulate a sample
distribution
• Basically the same as what we did just now
• Large table and more calculation
• Textbook, P. 277, Example 5.3

29
𝑛
 Properties of Sampling Distributions

Section 5.2 30
 Point Estimator
➢ Population from Distribution with the unknown parameter (e.g., mean of a
normal)

➢ Sample independent data points from the same underlying distribution

with mean and standard deviation (i.i.d.)
– 1, 2, …,

➢ A point es mator: rule (not a number) on how to use the samples( 1, …, ) to

calculate a sample number as an estimate of population parameter
1
¯ =
• ∑
=1
• Since each is random, the estimator is a random variable

➢ Estimate: calculated number/value using the rule (estimator) based on one

sample
1
𝑖
1, …, ∑
Observed , Estimate: ¯ =
𝑖
𝑛
•
𝑛
𝑖
31
𝑋
𝑋
𝑖
=1
𝑥
𝑥
𝑛
𝑖
𝑛
ti
𝑋
𝑋
𝜇𝑛𝜇𝜎
𝑛
𝑋
𝑋
𝑋
𝑋
𝑥
𝑥
𝑛
𝑛
 Properties of the Sampling Distribution of
¯

➢ Expectation of the sample mean = population mean

(unbiased)
µ X = E (X ) = µ.
➢ Standard deviation of the sample mean (standard
error of sample mean)
σ Standard deviation of the population distribution
σX = =
n Square root of sample size
• Less than population standard deviation

32
𝑿
 Unbiased Estimates

➢ Unbiased estimator of the parameter:

• Mean of the sampling distribution of a
sample statistic = population parameter
• Example: estimator ¯ , ¯ = (¯)=

➢ Biased estimator of the parameter:

• Mean of the sampling distribution of a
sample statistic ≠ population parameter
𝑋
33
𝑋
𝜇
𝐸
𝑋
𝜇
 Unbiased Estimates
➢ Statistic is an unbiased estimator of a parameter :
• ( )=

➢ Statistic is a biased estimator of the parameter :

• ( )≠

34
𝐴𝜃𝐸
𝐸
𝐵𝜃
𝐴
𝐵
𝜃
𝜃
 Efficiency of an Estimate
➢ Statistic and are both unbiased

➢ Statistic is more efficient

• smaller standard deviation (better)

➢ MVUE
Minimum variance
unbiased estimator

35
𝐴𝐵𝐴
 2
• Population has unknown mean μ and variance σ
• Draw many independent samples of size n.
1 n
n∑
Define the estimator, e.g. X̄ = Xi → es mate μ
•
i=1
n
1
For sample # 1, compute x̄(1) =
∑
xi
• n i=1
(1)
• x̄ one “observation” of X̄
• different samples give different observation values.
x̄(2), x̄(3) . . .
(1) (2) (3)
• x̄ ,x̄ , x̄ . . . are the data points for random variable X̄
• Sampling distribution: the distribution of X̄
• the collection of all possible X̄ values and their probability.36
ti
 Example: Biased and Unbiased Estimators
In an casino game, there are 3 different possible values for X, that
is -5, 0, 5. Below is a probability distribution of X.
The sampling distribution of the sample mean ¯ and the sample
median for random samples of n = 3
𝑋
𝑀
Example: Biased and Unbiased Estimators

…… …… …… ……

Entire table is in Section 5.1 of the textbook.

 Example: Biased and Unbiased Estimators (cont)

The sampling distribution of the sample mean ¯

and were found to be:
𝑋
𝑀
 Example: Biased and Unbiased Estimators (cont)

(a) Show that ¯ is an unbiased estimator of μ.

E(X̄) = (-5)(1/729) 1 2
μ = − 5 * + 5 * = 0.556
+ (-3.33)(18/729) 9 9
+ (-1.67)(114/729)
+ ( 0)(288/729)
+ ( 1.67)(228/729)
+ ( 3.33)( 72/729)
+ ( 5)( 8/729) = 0.556
𝑋
 Example: Biased and Unbiased Estimators (cont)

(b) Show that is a biased estimator of .

𝑀𝜇
 Example: Biased and Unbiased Estimators (cont)

(c) Find the standard deviation of the sampling

distribution of ¯ and . Which statistics would appear
to be a better estimation of ?
𝑋
𝑀𝜇
 ∑ ∫all x
E(X) = xp(x), = xf(x)dx
•

∫
2 2
∑
• var(X) = (x − E(X)) p(x), = (x − E(X)) f(x)dx

2 2
• var(X) = E(X ) − (E(X))

∫
2 2
•
E(X ) = x f(x)dx

• cov(X, Y ) = E[(X − E(X))(Y − E(Y ))]

43