Chapter 7: Introduction to
Point Estimation and Testing
Probability and Statistics for Science and Engineering
with Examples in R
Silvia Sharna
(1) Point Estimation
• (Point) estimator: a statistic intended for estimating a parameter
• (Point) estimate: an observed value of the estimator
7.1+5.5+6.6
𝑥1 = 7.1, 𝑥2 = 5.5, 𝑥3 = 6.6 ⟹ 𝑥ҧ = = 6.40 → estimate of µ
3
𝑥ҧ 𝑖𝑠 𝑡ℎ𝑒 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑜𝑟 𝑎𝑛𝑑 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓𝑥ҧ 𝑖𝑠 𝑡ℎ𝑒 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒.
2 2 ത 2
σ(𝑋𝑖 −𝑋) σ 𝑋𝑖2 − σ 𝑋𝑖 2 /𝑛
𝜎ො = 𝑆 = = : estimator of 𝜎 2
𝑛−1 𝑛−1
σ 𝑥𝑖2 − σ 𝑥𝑖 2 /𝑛 σ 𝑥𝑖2 − 𝑛𝑥ҧ 2 7.12 + 5.52 + 6.62 − 3(6.40)2
= = = 0.67
𝑛−1 𝑛−1 3−1
→ estimate
Standard error (s.e.): standard deviation of an estimator
𝜎2
𝐸 𝑋ത = 𝜇, 𝑉𝑎𝑟 𝑋ത =
𝑛
𝜎 𝜎
𝜎𝑋ത = ⟹ s. e. 𝑋ത =
𝑛 𝑛
Unbiased Estimators
An estimator 𝜃 is an unbiased estimator of 𝜃 if 𝐸 𝜃 = 𝜃 for all 𝜃.
If 𝜃 is not unbiased (biased), then 𝐸 𝜃 − 𝜃 is called bias.
Example 7.2∶ 𝑋~Bin 𝑛, 𝑝 , 𝑛 = 25
Find an unbiased estimator of 𝑝.
𝐸 𝑋 = 𝑛𝑝
𝑋 1 1
𝐸 = 𝐸 𝑋 = ∙ 𝑛𝑝 = 𝑝
𝑛 𝑛 𝑛
𝑋
∴ 𝑝Ƹ = is an unbiased estimator of 𝑝.
𝑛
𝑥 15 3
If 𝑥 = 15, then = = is the estimate.
𝑛 25 5
𝑋
• 𝑋~Bin 𝑛, 𝑝 ⟹ 𝑝Ƹ = is an unbiased estimator of p.
𝑛
𝑋
The s.e. of 𝑝Ƹ = :
𝑛
𝑋 𝑉𝑎𝑟 𝑋 𝑛𝑝 1 − 𝑝
𝜎𝑝ො = 𝑉𝑎𝑟 = = = 𝑝(1 − 𝑝)/𝑛
𝑛 𝑛2 𝑛2
s.e. estimator: 𝜎ො𝑝ො = 𝑝(1
Ƹ − 𝑝)/𝑛
Ƹ
s.e. estimate: 0.6 (0.4)/25 = 0.098
Example 7.3: In a presidential election poll conducted the day before
the election, a sample of 1500 voters were asked for their intended
vote. From this sample, 45% preferred the candidate from a
conservative party. Assume everyone in the sample expressed a
preference.
The unbiased estimate of this ratio is 0.45, and the estimate of s.e. is
(0.45)(0.55)/1500 = 0.013.
• Let 𝑋1 , ⋯ , 𝑋𝑛 be a random sample from a distribution with
mean 𝜇 and variance 𝜎 2 . Then
𝜇Ƹ = 𝑋ത is an unbiased estimator of 𝜇 and
ത 2
σ(𝑋𝑖 −𝑋)
𝜎ො 2 = 𝑆2 = is an unbiased estimator of 𝜎 2 .
𝑛−1
(2) Tests of Hypotheses
Null Hypothesis (𝑯𝟎 ): The null hypothesis is the default assumption. We
assume 𝐻0 is true unless the data gives strong evidence against it.
Alternative Hypothesis (𝐻𝑎 ):The alternative hypothesis is what we are
checking for. This is the statement we look for evidence in favor of.
(2) Tests of Hypotheses
A statistical hypothesis is a statement about a population parameter.
(A) The null and the alternative hypotheses
Example 7.4: The cure rate for a given disease using a standard
medication is 35%. The cure rate of a new drug is claimed to be better.
A sample has 𝑛 = 18 patients.
𝑋: number of patients who were cured
Is there substantial evidence that the new drug has a higher cure rate
than the standard medication?
𝐻0 : The new drug is not better: 𝑝 ≤ 0.35 (null hypothesis)
𝐻1 : The new drug is better: 𝑝 > 0.35 (alternative hypothesis)
Choice of 𝑯𝟎 & 𝑯𝟏
When our goal is to establish an assertion, the negation of the
assertion is 𝐻0 , and the assertion itself is 𝐻1 .
If 𝐻0 is true, we would expect 6 or fewer cures out of 18.
If 𝐻1 is true, we would expect more than 6 cures out of 18.
Decision rule for a test of the null hypothesis:
Reject 𝐻0 if 𝑋 > 𝑐
Do not reject 𝐻0 if 𝑋 ≤ 𝑐
(𝑋: test statistic; {𝑋 > 𝑐}: Rejection region)
• Test: decision rule that tells us when to reject 𝐻0 , when not
to reject 𝐻0
• Test statistic: a statistic whose value serves to determine the
action
• Rejection region: the set of values of a test statistic which 𝐻0
is to be rejected
(B) The Two Types of Errors
Type I error: reject 𝐻0 when 𝐻0 is true
Type II error: fail to reject 𝐻0 when 𝐻1 is true
Example 7.4 (continued): 𝐻0 : 𝑝 ≤ 0.35 vs. 𝐻1 : 𝑝 > 0.35, 𝑛 =18
Rejection region: 𝑅: 𝑋 ≥ 10
Determine the type of error that can occur and calculate the error probability
when (a) 𝑝 = 0.3, (b) 𝑝 = 0.6.
(a) 𝑝 = 0.3 ⟹𝐻0 is true. Type I error: Reject 𝐻0
𝑃(type I error given 𝑝 = 0.3) = 𝑃(𝑋 ≥ 10 given 𝑝 = 0.3)
𝑋 ~ Bin(18, 0.3)
𝑃(𝑋 ≥ 10) = 1 − 𝑃(𝑋 ≤ 9) = 1 − 0.979 = 0.021
𝑃(type I error given 𝑝 = 0.3 ) = 0.021
(b) 𝑝 = 0.6 ⟹ 𝐻1 is true. Type II error: Accept 𝐻0
𝑋 ~ Bin(18, 0.6)
𝑃(type II error given 𝑝 = 0.6) = 𝑃(𝑋 ≤ 9 given 𝑝 = 0.6) = 0.263
Example 7.4 (Continued):
𝑝 in 𝐻0
Type I error Probability 0.2 0.3 0.35
𝑷(𝑿 ≥ 𝟏𝟎) 0.001 0.021 0.060
Level of significance (significance level; 𝛼):
The max type I error probability of a test
In the above example, 𝛼 = 0.060.
Type II error probability: 𝛽 = 𝑃(type II error)
Power: 1 − 𝛽
Example 7.5: The lifetime of a certain type of car battery: 𝑁 5, 0.92 (yr.)
A new kind of battery is designed to increase the average lifetime.
𝐻0 : 𝜇 = 5, 𝐻1 : 𝜇 > 5
2
𝑋1 , ⋯ , 𝑋25 ത
𝑋~𝑁 𝜇, 𝜎
ൗ = 𝑁 𝜇, 0.182
𝑛
Suppose rejection region: 𝑋ത ≥ 5.42 . Calculate the type I error rate
and also the type II error rates when 𝜇 = 5.2 or 5.5.
𝛼 = 𝑃(type I error) = 𝑃(𝐻0 is rejected when it is true)
= 𝑃(𝑋ത ≥ 5.42 when 𝑋ത ~𝑁 5, 0.182 )
5.42−5
=𝑃 𝑍≥ = 𝑃 𝑍 ≥ 2.33 = Փ −2.33 = 0.01
0.18
𝛽(5.2) = 𝑃 Type II error when 𝜇 = 5.2
= 𝑃(do not reject 𝐻0 when 𝜇 = 5.2 )
= 𝑃 𝑋ത < 5.42 when 𝑋~𝑁(5.2,
ത 0.182 )
5.42−5.2
=𝑃 𝑍< = Փ 1.22 = 0.8888
0.18
Power: 1 − 0.8888 = 0.1112
5.42−5.5
𝛽(5.5) = 𝑃 𝑍 < = Փ −0.44 = 0.33
0.18
Power: 1 − 0.33 = 0.67
Suppose an experiment and 𝑛 are fixed. Then decreasing the
size of the rejection region ⟹ smaller 𝛼 ⟹ larger 𝛽
Testing Hypothesis (overall), 5 Steps:
Step 1 Setup 𝐻0 & 𝐻1 ←question
Step 2 𝛼 = ?
Step 3 Choose the test statistic and critical (rejection) region
Step 4 Substitute the values in the test statistic
Decision
Step 5 Calculate the 𝑝-value
• The p-value is the probability of obtaining a value for the test
statistic that is more extreme than the value actually observed.
(Probability is calculated under the null hypothesis 𝐻0 .)
• Meaning of the p-value:
Reject 𝐻0 if 𝑝 ≤ 𝛼
Do not reject 𝐻0 if 𝑝 > 𝛼