Probability and Statistics 2

Chapter 3 - Part 1

Parameter Estimation

Point Estimation Problems

Dr. Eman Ahmed Farag

Lecturer at Faculty of Science

Helwan University
Dr. Eman Farag Reminder: Parameters and Statistics

- A parameter is a population measure (e.g. , 2).

- A statistic is a sample function (e.g. sample mean,
sample variance). Hence, statistics may be regarded
as random variables.
- Statistics are used to estimate parameters and are
called point estimators.
- A point estimate of a parameter is a single numerical
value of a respective estimator.
- The standard deviation of an estimator is called the
standard error.

2
Dr. Eman Farag Statistical Inference
- In particular applications, the collected data usually allow a
certain population and this population has a probability
distribution usually with known type ( Binomial, Poisson,
normal, exponential,…), but with unknown parameters (µ,
σ,…).
- Statistical inference consists of those methods by which one
makes inferences or generalizations about a population.
- The statistical inference is concerned mainly with the
following two problems
1. Estimating or (approximating) the unknown parameters.
2. Testing some statistical hypothesis about these population
parameters.
A parameter is an unknown constant, but an estimator is a
statistic.
3
Dr. Eman Farag Estimation

Estimation is divided into

1. The classical method
The classical method of estimating a population
parameter, whereby inferences are based strictly on
information obtained from a random sample selected
from the population, and
2. The Bayesian method
The Bayesian method which utilizes prior subjective
knowledge about the probability distribution of the
unknown parameters in conjunction with the
information provided by the sample data.

Statistical inference
may be divided into two major areas :
(a) Estimation (b) Testing of hypotheses.
4
Dr. Eman Farag Classical Methods of Estimation
Point Estimates

A point estimate of some population parameter 𝜃 is a single

value 𝜃෠ of a statistic Θ
෡.

For example
The value 𝑥ҧ of the statistic Xത , computed from a sample of size n,
is a point estimate of the population parameter μ.

The desirable properties for good estimator

1- Unbiased estimator
which means that
𝐸 Θ෡ = 𝜃.
2- Minimum Variance
(Efficient Estimator)

5
Dr. Eman Farag Unbiased Estimator

Definition the bias of the point estimator

Let Θ
෡ = ℎ 𝑋1 , 𝑋2 , … , 𝑋𝑛 be a point estimator for 𝜃. The bias
of point estimator Θ෡ is defined by
෡ =𝐸 Θ
𝐵 Θ ෡ − 𝜃.

Definition Unbiased estimator

෡ is said to be an unbiased estimator of the
A statistic Θ
parameter θ if
𝜇Θ෡ =𝐸 Θ෡ = 𝜃.
Or, ෡ = 0,
𝐵 Θ for all possible values of 𝜃.
In general, we would like to have a bias that is close to 0,
෡ is close to the real value of 𝜃.
indicating that on average, Θ
We would like the sampling distribution of Θ ෡ to have a mean
equal to the parameter estimated. An estimator possessing this
property is said to be unbiased.
6
Dr. Eman Farag Example 1

Let 𝑋1 , 𝑋2 , 𝑋3 , ..., 𝑋𝑛 be a random sample.

Show that the sample mean
𝑋1 + 𝑋2 + 𝑋3 + … + 𝑋𝑛
෡
Θ=𝑋= ത
𝑛
is an unbiased estimator of 𝜃 = 𝐸 𝑋𝑖 .
Solution
We have 𝐵 Θ ෡ =𝐸 Θ ෡ −𝜃
= 𝐸 𝑋ത − 𝜃
= 𝐸 𝑋𝑖 − 𝜃
= 0.

7
Dr. Eman Farag Example 2

Show that S2 is an unbiased estimator of the parameter σ2 .

Solution: We showed that

8
Dr. Eman Farag

9
 Dr. Eman Farag Variance of a Point Estimator

Definition Efficient Estimator

If we consider all possible unbiased estimators of some parameter
θ, the one with the smallest variance is called the most efficient
estimator of θ.

Figure: Sampling
distributions of different
estimators of θ

Figure illustrates the sampling distributions of three different estimators, Θ ෡1, Θ෡ 2,

and Θ ෡ 3 , all estimating θ. It is clear that only Θ
෡ 1 and Θ෡ 2 are unbiased, since their
distributions are centered at θ. The estimator Θ ෡ 1 has a smaller variance than Θ
෡ 2 and
is therefore more efficient. Hence, our choice for an estimator of θ, among the three
considered, would be Θ ෡1. 10
Dr. Eman Farag The Mean Squared Error (MSE) of a
point estimator Θ෡
෡
Definition The Mean Squared Error (MSE) of a point estimator Θ
The Mean Squared Error (MSE) of a point estimator Θ෡ , shown by
𝑀𝑆𝐸 Θ ෡ , is defined as
෡ =𝐸 Θ
𝑀𝑆𝐸 Θ ෡−𝜃 2 .

We suspect that Θ ෡ is probably not as good as the sample mean

ത Therefore, we need other measures to ensure that an
𝑋.
estimator is a "good" estimator. A very common measure is the
mean squared error defined by 𝐸 Θ ෡ −𝜃 2

Note that
Θ෡ − 𝜃 is the error that we make when we estimate 𝜃 by Θ ෡.
Thus, the MSE is a measure of the distance between Θ ෡ and 𝜃 ,
and a smaller MSE is generally indicative of a better estimator.

11
Dr. Eman Farag Point Estimators for Mean and Variance

Given a population of size N, We need to find its mean  and

variance 𝜎 2 . The number N is too big, so we pick a sample of
reasonable (?) size n. Find the sample mean and sample variance.
How good is 𝑋asത an estimate of ?
How good is s2 as an estimate of 𝜎 2 ?
Point Estimation of the population Mean
Parameter : population mean 
Data : 𝑋1 , … , 𝑋2 ( sample of size n )
Estimator : 𝑋ത (Sample mean)
It is unbiased and efficient estimator.
Point Estimation of the population Variance 𝝈𝟐
Parameter : population Variance σ2
Data : 𝑋1 , … , 𝑋2 ( sample of size n )
Estimator : S2 (Sample Variance)
It is unbiased and efficient estimator.
12
Dr. Eman Farag Example 3

Consider the sample 1, 2, 3, 4 drawn from a population with

unknown mean µ and variance σ2 . Find estimates for µ and σ2 .
Solution

1 + 2 + 3 + 4 10
=X = = = 2.5
4 4
2

 =s =
2 2 1 4

n −1 1
(
X i −X )
1 
(1 − 2.5 ) + ( 2 − 2.5 ) + ( 3 − 2.5 ) + ( 4 − 2.5 ) 
2 2 2 2
=
4 −1  
1 5
= . 5 = = 1.66
3 3

13
Dr. Eman Farag Example 4

Let T be the time that is needed for a specific task in a factory to

be completed. In order to estimate the mean and variance of T, we
observe a random sample T1 , T2 , T3 , … , T6 . Thus, T𝑖 's are i.i.d.
and have the same distribution as T. We obtain the following
values (in minutes): 18, 21, 17, 16, 24, 20.
Find the values of the sample mean, the sample variance, and the
sample standard deviation for the observed sample.
Solution
The sample mean is
𝑇1 + 𝑇2 + 𝑇3 +𝑇4 + 𝑇5 + 𝑇6
ത
𝑇=
6
18+ 21+ 17+16+ 24+20
= = 19.333.
6
The sample variance is given by
1
s2 = σ6𝑘=1 𝑇𝑘 − 19.333 2 = 8.67.
6−1
Finally, the sample standard deviation is given by: 𝑠 = 𝑠 2 = 2.94.14
Dr. Eman Farag Example 5

Let X be the height of a randomly chosen individual from a population. In

order to estimate the mean and variance of X, we observe a random
sample X1 , X2 , X3 , … , X7 . Thus, 𝑋𝑖 's are i.i.d. and have the same
distribution as X. We obtain the following values (in centimeters):
166.8, 171.4, 169.1, 178.5, 168.0, 157.9, 170.1
Find the values of the sample mean, the sample variance, and the
sample standard deviation for the observed sample.
Solution
The sample mean is
𝑋1 + 𝑋2 + 𝑋3 +𝑋4 + 𝑋5 + 𝑋6 + 𝑋7
𝑋ത =
7
166.8+171.4+169.1+178.5+168.0 + 157.9 + 170.1
= = 168.8.
7
The sample variance is given by
1
s =2 σ7𝑘=1 𝑋𝑘 − 168.8 2 = 37.7.
7−1
Finally, the sample standard deviation is given by: 𝑠 = 𝑠 2 = 6.1.
15
 Any Question?
Tutorial # 4: pp 282 – 286 odd Number

16