H YPOTHESIS T ESTING

C ONCEPTS AND A PPLICATIONS

W HY D O W E N EED H YPOTHESIS T ESTING ?

In practice we never observe the entire population. We draw a sample and must decide
whether patterns in the sample reflect genuine population-level truths or are merely due to
random chance.
Running Example
A government agency surveys 250 households across two districts to study income
inequality.
125 urban households, 125 rural households.
Questions we want to answer:

• Is mean urban income truly above the poverty line of BDT 20,000/month, or could the
observed difference be due to sampling variation?
• Is there a genuine income gap between urban and rural households, or could the
observed difference arise by chance?

1
T HE C ORE I DEA : S IGNAL VS . N OISE

Signal
genuine effect
Real?
Observed Hypothesis test
difference decides
Chance? Noise
random variation

A hypothesis test quantifies how surprising the observed result would be if there were truly
no effect in the population. If the result is sufficiently surprising, we conclude the effect is
real.

2
S AMPLING D ISTRIBUTION AND
C ENTRAL L IMIT T HEOREM
F ROM P OPULATION TO S AMPLE

Population The Sampling Problem

All households in the country. If we drew a different sample of 250
True mean µ and SD σ are unknown. households, we would get a different x̄.

We can never observe every unit. How much does x̄ vary from sample to
sample?
Sample This variation is described by the
250 randomly selected households. sampling distribution of x̄.
Sample mean x̄ and SD s are observed.
x̄ is our best estimate of µ.

3
T HE S AMPLING D ISTRIBUTION OF X̄

Population distribution of income Sampling distribution of X̄

·10−5 Many samples ·10−4
of n = 250
Density

Density
Income (BDT) ·104 x̄ (BDT) ·104

4
T HE C ENTRAL L IMIT T HEOREM (CLT)

Statement
Regardless of the shape of the population distribution, the sampling distribution of the
sample mean X̄ approaches a normal distribution as the sample size n increases:

σ2
 
approx
X̄ ∼ N µ, for large n
n

Why It Matters Rule of Thumb

Even if household incomes are n ≥ 30: CLT approximation is generally
right-skewed, the mean income from a adequate.
sample of 250 follows an approximately With n = 250 households, the CLT applies
normal distribution. This allows us to comfortably regardless of the income
compute probabilities and build distribution’s shape.
confidence intervals using the normal (or
t) table. 5
CLT: E FFECT OF S AMPLE S IZE
·10−3

n = 10
n = 30
n = 100
n = 250
Density

1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 3.2

Sample mean x̄ (BDT) ·10 4

Larger samples produce narrower sampling distributions: x̄ is closer to µ with high probability.

6
S TANDARD E RROR
S TANDARD E RROR

Definition
The standard error (SE) of the sample mean is the standard deviation of its sampling
distribution:
σ
SE(X̄) = √
n
In practice, σ is unknown and replaced by the sample SD s:

c (X̄) = √s
SE
n

Interpretation SE vs. SD
SE is the typical distance between the SD measures spread of individual
sample mean x̄ and the true population observations around the mean.
mean µ. SE measures spread of the sample mean
Smaller SE ⇒ more precise estimate. around the population mean. 7
S TANDARD E RROR : W HAT A FFECTS I T ?

s
SE = √
n

Factor Effect on SE Reason

Increase n SE ↓ More data ⇒ more precise estimate

Increase s SE ↑ More dispersed population ⇒ less precision

√
Double n SE ×1/ 2 Halving SE requires quadrupling n

Running Example
Urban households: s = 7,209, n = 125
√
c = 7,209 / 125 = 7,209 / 11.18 = 644.8 BDT/month
SE
The sample mean urban income is typically within ± BDT 644.8 of the true population
8
mean.
C ONFIDENCE I NTERVALS AND
M ARGIN OF E RROR
C ONFIDENCE I NTERVAL : C ONCEPT

Definition
A confidence interval (CI) is a range of plausible values for the unknown population
parameter, constructed from the sample:

x̄ ± t∗α/2,n−1 × SE
c

where t∗ is the critical value from the t-distribution with n − 1 degrees of freedom.

Correct Interpretation
A 95% CI means: if we repeated this sampling procedure many times and built a CI each
time, 95% of those intervals would contain the true population mean µ.
It does not mean there is a 95% chance that µ lies in this particular interval.

9
C ONFIDENCE I NTERVAL : V ISUAL I NTUITION
µ = 29,018

Sample number

2.5 2.7 2.9 3.1 3.3 3.5

·104

Out of 10 intervals, 9 capture the true mean (teal). One misses (red). At 95% confidence, 5% of intervals will miss.

10
M ARGIN OF E RROR

Definition
The margin of error (ME) is the half-width of the confidence interval:
s
ME = t∗α/2, n−1 × SE
c = t∗
α/2, n−1 × √
n

The CI is then: x̄ ± ME.

Running Example: Urban Household Income

x̄ = 29,018,
s = 7,209, n = 125, ∗
t0.025, 124 = 1.984
√
c = 7,209/ 125 = 644.8
SE
ME = 1.984 × 644.8 = 1,279 BDT
95% CI: 29,018 ± 1,279 = (27,739, 30,297) BDT/month

11
W IDTH OF CI: E FFECT OF C ONFIDENCE L EVEL

± 1,688
Confidence level

99%
x̄ = 29,018
95%
± 1,279
90%
± 1,068

2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4
Urban mean monthly income (BDT) ·104

Higher confidence ⇒ wider interval. There is a trade-off between precision and certainty.

12
H YPOTHESIS T ESTING :
I NTUITION AND F RAMEWORK
T HE I NTUITION B EHIND H YPOTHESIS T ESTING

Suppose a friend claims: “Average urban household income is exactly BDT 20,000/month.”

Your survey found x̄ = 29,018.

The Question
If the true mean really were BDT 20,000, how likely would it be to observe a sample mean
as large as BDT 29,018 from n = 125 households purely by chance?

If probability is high If probability is very low

The data are consistent with the claim. We It is very unlikely the claim is true. We
cannot rule it out. reject the claim and conclude the true
mean is different from BDT 20,000.

13
T HE H YPOTHESIS T ESTING F RAMEWORK

Step Action

1. State hypotheses Formulate H0 (null) and H1 (alternative)

2. Choose α Set significance level (e.g. α = 0.05)

3. Collect data Obtain sample and compute test statistic

4. Compute p-value Probability of observing data this extreme under H0

5. Decision Reject H0 if p < α; fail to reject if p ≥ α

6. Conclusion State finding in plain language

Important Language
We never “accept H0 ”. Failing to reject H0 means only that there is insufficient evidence to
conclude otherwise. Absence of evidence is not evidence of absence. 14
N ULL AND A LTERNATIVE H YPOTHESES

Null Hypothesis H0
The default claim — usually a statement of no effect, no difference, or a specific value.
The test asks: are the data consistent with H0 ?

Alternative Hypothesis H1
The claim we want to support — what we believe is actually true. H1 is accepted only if
the data provide strong evidence against H0 .

Important Language
We never say “accept H0 ”. Failing to reject H0 only means there is insufficient evidence to
conclude otherwise.

15
T YPES OF H YPOTHESES : E XAMPLES

Test type H0 H1

One-sample (vs. value) µ = µ0 µ ̸= µ0 (two-sided)

One-sample (upper) µ ≤ µ0 µ > µ0

One-sample (lower) µ ≥ µ0 µ < µ0

Two-sample (difference) µ1 = µ2 µ1 ̸ = µ2

Running Example
Question 1: Is mean urban income different from the poverty line BDT 20,000?
H0 : µurban = 20,000 vs H1 : µurban ̸= 20,000
Question 2: Is there an urban-rural income gap?
H0 : µurban = µrural vs H1 : µurban ̸= µrural 16
T YPE I AND T YPE II E RRORS
T WO WAYS TO B E W RONG

A hypothesis test can produce two types of error:

True state of the world

H0 is true H0 is false

2*Decision Reject H0 Type I error (α) Correct (Power = 1 − β)

Fail to reject H0 Correct Type II error (β)

Terminology
• Type I error = false positive = α = significance level.
• Type II error = false negative = β.
• Power = 1 − β = probability of correctly detecting a real effect. 17
T YPE I AND T YPE II E RRORS : D ESCRIPTION

Type I Error (False Positive)

Definition: Rejecting H0 when it is actually true.
Probability: α (the significance level you chose).
Example: Concluding that mean urban income exceeds the poverty line of BDT 20,000
when it actually does not.
Control: Choose a smaller α (e.g. 0.01 instead of 0.05).

Type II Error (False Negative)

Definition: Failing to reject H0 when it is actually false.
Probability: β (depends on sample size and effect size).
Example: Missing a genuine income gap between urban and rural households because the
sample was too small.
Remedy: Increase sample size to raise power = 1 − β.
18
S TATISTICAL P OWER

Definition
Power = 1 − β is the probability of correctly rejecting H0 when it is false. It measures the
test’s ability to detect a genuine effect.

Factor Effect on power

Larger sample size n Power ↑

Larger true effect size |µ1 − µ0 | Power ↑
Smaller population SD σ Power ↑
Larger α (less strict) Power ↑ (but more Type I errors)

The Trade-off
Reducing α (being stricter about false positives) increases β (more false negatives) if sample
size is fixed. The only way to reduce both simultaneously is to increase the sample size. 19
T YPE I AND II E RRORS : V ISUAL

c
Null hypothesis distribution Alternative hypothesis distribution

(1 − α) (1-β)

−4 −2 0 2 4 6 8
Test statistic

c = critical value. Type I (α): reject H0 when true (right tail of navy curve). Type II (β): fail to reject H0 when false
(left tail of teal curve).
20
S IGNIFICANCE L EVEL AND T EST
S TATISTIC
S IGNIFICANCE L EVEL α

Definition
The significance level α is the maximum probability of a Type I error the researcher is
willing to tolerate. It is chosen before seeing the data.

α Critical t (n = 125) Use when

0.10 ±1.657 Exploratory research; large cost of missing effect

0.05 ±1.984 Standard in most applied research
0.01 ±2.616 Policy decisions; high cost of false alarm

Choosing α
There is no universally correct α. The choice depends on the cost of a Type I error relative
to a Type II error. In clinical trials, α = 0.01 is common. In social science, α = 0.05 is the
most widely used convention. 21
T HE p- VALUE

Definition
The p-value is the probability of observing a test statistic as extreme as (or more extreme
than) the one computed from the sample, assuming H0 is true.
 
p = P |T | ≥ |tobs | H0

If p < α If p ≥ α
The observed result is unlikely under H0 . The observed result is plausible under H0 .
Reject H0 : evidence supports H1 . Fail to reject H0 : insufficient evidence.

Common Misconception
The p-value is not the probability that H0 is true. It is the probability of the data (or more
extreme data) given that H0 is true. These are very different statements.
22
T HE T EST S TATISTIC

One-Sample t-Statistic
Measures how many standard errors the sample mean lies from the hypothesised value µ0 :

x̄ − µ0 x̄ − µ0
t = = √
SE
c s/ n

Under H0 : µ = µ0 , this follows a t-distribution with n − 1 degrees of freedom.

Running Example
H0 : µurban = 20,000 vs H1 : µurban ̸= 20,000
t = (29,018 − 20,000) / 644.8 = 9,018 / 644.8 = 13.99
∗
Since t = 13.99 ≫ t0.025, 124 = 1.984: strongly reject H0 . p < 0.001.

23
C RITICAL R EGION AND p- VALUE : V ISUAL

t = 13.99
(far right)

Fail to reject H0
Reject H0 Reject H0
α/2 = 0.025 α/2 = 0.025

−3 −1.984 0 +1.984 3
t-statistic (df = 124)

Shaded red regions are the rejection zone (α = 0.05, two-sided). Our t = 13.99 falls far beyond the right critical
value +1.984.
24
T YPES OF H YPOTHESIS T ESTS
O NE -S AMPLE t-T EST

Purpose
Test whether the population mean equals a specific value µ0 .

x̄ − µ0
H0 : µ = µ 0 t= √ ∼ tn−1
s/ n

Running Example
Is mean urban income equal to the poverty line of BDT 20,000?
t = 13.99, p < 0.001.
Conclusion: We reject H0 at any conventional significance level. Mean urban income is
statistically significantly above the poverty line of BDT 20,000/month.

25
O NE -S IDED VS . T WO -S IDED T ESTS

Type H1 When to use

Two-sided µ ̸ = µ0 No prior expectation about di-

rection of difference

One-sided (upper) µ > µ0 Theory or prior evidence sug-

gests mean is higher

One-sided (lower) µ < µ0 Theory suggests mean is

lower

Choose Before Seeing the Data

The direction of H1 must be decided before collecting data. Switching to a one-sided test
after observing the data inflates the Type I error rate and is considered p-hacking.

Effect on Critical Value

26
Two-sided at α = 0.05: |t| > 1.984, One-sided at α = 0.05: t > 1.657 (upper tail only).
O THER C OMMON T ESTS

Test Question When to use

One-sample t-test Is µ equal to a known Continuous outcome vs. bench-

value? mark

Two-sample t-test Are µ1 = µ2 ? Comparing two independent

groups

Paired t-test Is mean difference = 0? Before-after or matched pairs

Proportion z-test Is p = p0 or p1 = p2 ? Binary outcomes (yes/no, em-

ployed/unemployed)

F-test (ANOVA) Are all group means Comparing 3 or more groups si-
equal? multaneously

Chi-square test Are two variables inde- Categorical variables in a cross-

pendent? tabulation
27
S TATA L AB
D ATASET

Dataset: income_data.csv 250 households, 2 districts, single cross-section.

Variable Description Variable Mean SD

hh_id Household ID income 23,157 8,805

area 1 = urban, 0 = rural age_head 43.4 10.1
income Monthly income (BDT) educ_head 7.9 3.7
age_head Age of household head hh_size 4.7 1.5
educ_head Years of education
hh_size Household size

28
S TEP 1: L OAD D ATA AND S UMMARISE

* Load data
import delimited "income_data.csv", clear

* Label the area variable

label define area_lbl 1 "Urban" 0 "Rural"
label values area area_lbl

* Overall summary
summarize income age_head educ_head hh_size

* Summary by area
bysort area: summarize income

Reading summarize Output

Note the mean, SD, min, and max for each group. The difference in group means is the
starting point for all subsequent hypothesis tests. 29
S TEP 1 O UTPUT: S UMMARY BY A REA

Group n Mean (BDT) SD (BDT) SE (BDT)

Urban 125 29,018 7,209 645

Rural 125 17,296 5,869 525

All 250 23,157 8,805 557

Observation
Urban households earn BDT 11,722 more per month on average. But is this difference
statistically significant, or could it have arisen by chance in a sample of 250 households?

30
S TEP 2: V ISUALISE THE D ISTRIBUTION

* Histogram by area
twoway (histogram income if area==1, color(navy%60) ///
frequency) ///
(histogram income if area==0, color(gold%60) ///
frequency), ///
title("Monthly Income Distribution by Area") ///
xtitle("Monthly income (BDT)") ///
ytitle("Frequency") ///
legend(label(1 "Urban") label(2 "Rural"))

* Kernel density
twoway (kdensity income if area==1, lcolor(navy) lwidth(medthick)) ///
(kdensity income if area==0, lcolor(gold) lwidth(medthick)), ///
title("Income Density by Area") ///
xtitle("Monthly income (BDT)") ///
legend(label(1 "Urban") label(2 "Rural"))
31
S TEP 3: C ONFIDENCE I NTERVAL FOR M EAN I NCOME

* 95% CI for mean income, all households

ci means income

* 95% CI by area
ci means income if area==1 // Urban
ci means income if area==0 // Rural

* 90% and 99% CIs for comparison

ci means income if area==1, level(90)
ci means income if area==1, level(99)

Reading ci means Output

Stata reports the sample mean, SE, and lower/upper bounds of the CI. Compare the 90%,
95%, and 99% intervals to see how confidence level trades off against interval width.

32
S TEP 3 O UTPUT: C ONFIDENCE I NTERVALS

Group Level Mean SE Lower Upper

Urban 90% 29,018 645 27,950 30,086

Urban 95% 29,018 645 27,739 30,297
Urban 99% 29,018 645 27,330 30,706

Rural 95% 17,296 525 16,254 18,337

The 95% CI for urban income is (27,739; 30,297). The 95% CI for rural income is (16,254;
18,337). The two intervals do not overlap: strong visual evidence of a genuine difference.

33
S TEP 4: O NE -S AMPLE t-T EST

* H0: mean urban income = 20,000 (poverty line)

* H1: mean urban income != 20,000 (two-sided)
ttest income == 20000 if area==1

* One-sided version: H1: mu > 20000

ttest income == 20000 if area==1, unpaired

Reading ttest Output

Stata reports the observed mean, hypothesised value, t-statistic, degrees of freedom, and
three p-values: one for each direction (< µ0 , ̸= µ0 , > µ0 ). For a two-sided test, use the
middle p-value.

34
S TEP 4 O UTPUT: O NE -S AMPLE t-T EST

Test: H0 : µurban = 20,000 vs H1 : µurban ̸= 20,000

Mean SE 95% CI

Observed 29,018 645 (27,739; 30,297)

Under H0 20,000

t(124) = 13.99 p < 0.001 (two-sided)

Conclusion
We reject H0 at α = 0.001. Mean urban household income of BDT 29,018 is statistically
significantly higher than the poverty line of BDT 20,000 (t = 13.99, p < 0.001).

35
S TEP 5: T WO -S AMPLE t-T EST

* H0: mean income same in urban and rural

* H1: mean income different (two-sided)
* by(area) splits the sample by area
ttest income, by(area)

* Welch’s t-test (does not assume equal variances)

ttest income, by(area) welch

Equal vs. Unequal Variances

The standard two-sample t-test assumes equal population variances. welch relaxes this
assumption. Since urban SD (7,209) is notably larger than rural SD (5,869), Welch’s test is
appropriate here.

36
S TEP 5 O UTPUT: T WO -S AMPLE t-T EST

Test: H0 : µurban = µrural vs H1 : µurban ̸= µrural

Group n Mean (BDT) SD SE

Urban 125 29,018 7,209 645

Rural 125 17,296 5,869 525

Difference 11,722 831

t(248) = 14.10 p < 0.001 95% CI for difference: (BDT 10,073; BDT 13,372)

Conclusion
Urban households earn BDT 11,722 more per month than rural households (t = 14.10,
p < 0.001).

37
S UMMARY

Key Concepts Stata Commands

• CLT: X̄ ∼ N (µ, σ2 /n). • summarize

√ • ci means
• SE = s/ n: precision of x̄.
• CI: plausible range for µ. • ttest
• ME: half-width of CI. • ttest ..., by(group)
• Type I (α): false positive. • ttest ..., welch
• Type II (β): false negative. • histogram, kdensity
• Power = 1 − β: detect real effects.
• p-value: evidence against H0 .

Decision Rule
Reject H0 if p < α. Interpret the p-value as evidence against H0 , not as the probability that
H0 is true. 38
R EFERENCES

• Moore, D.S., McCabe, G.P., & Craig, B.A. (2017). Introduction to the Practice of Statistics.
9th ed. W.H. Freeman.
• Wooldridge, J.M. (2019). Introductory Econometrics: A Modern Approach. 7th ed. Cengage.
Chapter 4.
• Freedman, D., Pisani, R., & Purves, R. (2007). Statistics. 4th ed. W.W. Norton.
• StataCorp (2023). Stata Base Reference Manual. Release 18. College Station, TX:
StataCorp LLC.

39