Econometrics
F. Machado
Motivation
The Gaussian
LRM Econometrics
The t Test
Inference in the Linear Regression Model
The p-value
Confidence
Intervals
The F-test
José A. Ferreira Machado João Valle e Azevedo
Nova School of Business and Economics
Fall Semester, 2024-2025
Hypothesis testing in the MLRM
Econometrics
Suppose you want to test whether a variable is important
F. Machado
in explaining variation in the dependent variable:
Motivation E.g., is the effect of tenure on wages statistically
The Gaussian significant (ie, different from zero)? Is the effect of height
LRM
on wages statistically significant?
The t Test
The p-value Or suppose you want to test whether a coefficient has a
Confidence particular value
Intervals
E.g., is the effect of one additional year of schooling on
The F-test
expected monthly wages equal to 200?
Need to take into account the sampling distribution of our
estimators
We will check whether under the maintained hypothesis
(or null hypothesis) the observed values of certain test
statistics are likely
If they are not we reject the null
The normality assumption
Econometrics
F. Machado y = β0 + β1 x1 + β2 x2 + ... + βk xk + u
Motivation
The Gaussian
LRM
Assumption MLR.6 (Normality)
The t Test
The p-value
The distribution of the population error u is independent
Confidence
of x1 , x2 , ..., xk and u is normally distributed with mean 0
Intervals and variance σ 2 . We write
The F-test
u ∼ Normal(0, σ 2 )
Independence assumption is stronger than MLR.4 (Zero
Conditional Mean) assumption. Actually, it implies MLR.4
Also, normality and independence imply MLR.5 so that
all the results regarding unbiasedness and variance of the
estimators remain valid
Three important results abut Normal r.v.’s
Econometrics
Linear functions of a normal rv
F. Machado
Let a and b be constants.
Motivation If Z ∼ Normal(µZ , σZ2 ) then
The Gaussian
LRM (a + bZ ) ∼ Normal(a + bµZ , b 2 σZ2 )
The t Test
The p-value Sum of normal rv’s
Confidence Pp
Intervals Let Zi ∼ Normal(µi , σi2 ) and W = i=1 Zi . Then
The F-test
2
W ∼ Normal(µW , σW )
Sum of squares of normal rv’s
Pp 2
Let Zi be independent Normal(0, 1) and U = i=1 Zi . Then
U ∼ χ2(p)
The Gaussian Linear Model
Econometrics
Assumptions MLR.1 through MLR.6 are the Gaussian or
F. Machado
Classical Linear Model (CLM) assumptions
Motivation Under the CLM assumptions, OLS is not only BLUE, but
The Gaussian
LRM
is the minimum variance unbiased estimator: no other
The t Test
unbiased estimator has a variance smaller than OLS
The p-value We can summarize the population assumptions of CLM as
Confidence follows
Intervals
The F-test
y |X ∼ Normal(β0 + β1 x1 + β2 x2 + ... + βk xk , σ 2 )
Normality is unrealistic in many cases (e.g., wages cannot
be negative but under the normality assumption of u we
can get negative wages)
However, most results would hold in large samples without
the normality assumption
Illustration of the Gaussian model
Econometrics
F. Machado
Motivation
y
The Gaussian f(y|x)
LRM
The t Test
The p-value .E(y|x) = b + b x
0 1
Confidence
Intervals .
The F-test
Normal
distributions
x1 x2
Figure: The homoskedastic normal distribution with a single
explanatory variable
Sampling Distribution of the Least Squares
Estimators
Econometrics
F. Machado Since the OLS estimators are a linear function of the error
Motivation term u, then (conditional on the x’s):
The Gaussian
LRM Theorem
The t Test Under the CLM assumptions, conditional on the sample values
The p-value
of the independent variables,
Confidence
Intervals
β̂j ∼ Normal[βj , Var (β̂j )],
The F-test
Therefore,
(β̂j − βj )
∼ Normal(0, 1)
sd(β̂j )
where sd stands for standard deviation (squared root of the
variance, derived in previous classes)
Econometrics
F. Machado
Now, the σ 2 that appears in the expression for the
Motivation standard deviation of the estimators must be estimated
The Gaussian
LRM Also, conditional on the x’s (n − k − 1)σ̂ 2 /σ 2 ∼ χ2n−k−1
The t Test which implies:
The p-value
(β̂j − βj ) (β̂j − βj ) sd(β̂j )
Confidence =
Intervals se(β̂j ) sd(β̂j ) se(β̂j )
The F-test
(β̂j − βj ) σ
=
sd(β̂j ) σ̂
Normal(0, 1)
≡ q 2 ∼ tn−k−1
χn−k−1
n−k−1
Econometrics
F. Machado
Motivation
The Gaussian
Theorem
LRM
Under the CLM assumptions MLR.1 through MLR.6,
The t Test
The p-value
(β̂j − βj )
Confidence ∼ tn−k−1 ,
Intervals se(β̂j )
The F-test
where k+1 is the number of unknown parameters in the
population model y = β0 + β1 x1 + ... + βk xk + u (k slope
parameters and the intercept β0 )
Performing a test on a single coefficient
Econometrics
F. Machado
Set the null hypothesis (and the alternative)
E.g., H0 : βj = 0 (coefficient on experience in our wage
Motivation
regression) and H1 : βj > 0
The Gaussian
LRM Choose a significance level (Probability of rejecting the
The t Test null if the null is actually true)
The p-value
E.g., α = 0.05
Confidence
Intervals Look at the sampling distribution of the ”test statistic” t
The F-test (random variable) involving the parameter:
(β̂j − βj )
t= ∼ t(n−k−1) ,
se(β̂j )
Under the null hypothesis, the test statistic should be
”small” across samples. Reject the null if the observed
value of the test statistic is very unlikely (very large)
Performing a test on a single coefficient
One-side alternatives
Econometrics
F. Machado One-side Tests
Motivation For one-sided tests where the alternative is favored if tobs
The Gaussian is large and positive (e.g., H1 : βj > 0), reject the null if
LRM
the observed test statistic, tobs , is larger than c, where c is
The t Test implicitly given by: Prob[t > c|H0 is true]=α
The p-value For one-sided tests where the alternative is favored if tobs
Confidence is large and negative (e.g., H1 : βj < 0), reject the null if
Intervals
the observed test statistic, tobs , is smaller than -c, where c
The F-test
is implicitly given by: Prob[t < −c|H0 is true]=α
For two-sided tests, where the alternative is favored if tobs
is large in absolute value (e.g., H1 : βj 6= 0), reject the null
if the absolute value of observed test statistic, tobs , is
larger than c, where c is implicitly given by:
Prob[|t| > c|H0 is true]=α
Illustration
One-Sided Alternative
Econometrics H0 : βj = 0 H1 : βj > 0
F. Machado
Motivation
The Gaussian
LRM
Fail to reject
The t Test the null
The p-value
Confidence
Intervals
The F-test
(1-α)
Reject the null
Figure: Rejection region for a 5% significance level for alternative
H1 : βj > 0
Illustration
Two-Sided Alternative
Econometrics H0 : βj = 0 H1 : βj 6= 0
F. Machado
Motivation
The Gaussian
LRM
The t Test Fail to reject
the null
The p-value
Confidence
Intervals
The F-test
(1-α)
Reject the null Reject the null
α/2 α/2
Figure: Rejection region for a 5% significance level for alternative
H1 : βj 6= 0
Tests on a single coefficient
Example
Econometrics
F. Machado
Independent Variable Coefficient Estimate Standard Error t ratio
Motivation
Intercept 5.33815 0.01218 438.36
The Gaussian
LRM Education (in years) 0.07614 0.00079 96.75
The t Test Labor Market Experience (in years) 0.03093 0.00087 35.38
The p-value Square of Labor Market Experience (in -0.00038 0.000018 -20.64
years)
Confidence
n 11064
Intervals
R2 0.4774
The F-test
Figure: Dependent Variable: Log of Wages
The ”t ratios” are the observed values of the test statistic
for testing βj = 0
E.g. 96.75=0.07614/0.00079
Tests on a single coefficient
Example (cont)
Econometrics
Choose α = 0.05
F. Machado Test H0 : βj = 0 against H1 : βj 6= 0 (coefficient on
Motivation education)
The Gaussian 0.07614 − 0
LRM tobs = = 96.75
The t Test
0.00079
The p-value
|t| >1.96 ⇒ Reject the null: the coefficient for education
Confidence
is significant at 5% significance level
Intervals We use Normal approximation since n is large
The F-test
Fail to reject the
null
Reject the null Reject the null
-c=-1.96 c=1.96
Tests on a single coefficient
Example (cont)
Econometrics
Choose α = 0.05
F. Machado Test H0 : βj = 0 against H1 : βj > 0 (clearly more
Motivation reasonable...)
The Gaussian 0.07614 − 0
LRM tobs = = 96.75
The t Test
0.00079
The p-value
|t| >1.645 ⇒ Reject the null: the coefficient for education
Confidence
is significant at 5% significance level
Intervals We use Normal approximation since n is large
The F-test
Fail to reject the
null
Reject the null
c=1.645
Tests on a single coefficient
Example (cont)
Econometrics
Choose α = 0.05
F. Machado Test H0 : βj = 0.07 against H1 : βj 6= 0.07 (coefficient on
Motivation education)
The Gaussian 0.07614 − 0.07
LRM tobs = = 7.772
0.00079
The t Test
The p-value
|t| >1.96 ⇒ Reject the null: the coefficient for education
Confidence
Intervals
is significant at 5% significance level
We use Normal approximation since n is large
The F-test
Fail to reject the
null
Reject the null Reject the null
-c=-1.96 c=1.96
p-value
Econometrics
p-value: Given the observed value of the t statistic, what
F. Machado would be the smallest significance level at which the null
Motivation H0 : βj = 0 would be rejected against the alternative
The Gaussian H1 : βj 6= 0?
LRM
It is given by:
The t Test
The p-value Prob[|t| > |tobs | | H0 true]
Confidence
Intervals
1-“p-value”
The F-test
“p-value”/2 “p-value”/2
-tobs tobs
p-value
Example
Econometrics
F. Machado
Motivation
The Gaussian
LRM
The t Test
The p-value
Confidence
Intervals
The F-test
Confidence Intervals
Econometrics
F. Machado
Motivation
The Gaussian
A (1 − α)% confidence interval is defined as:
LRM
The t Test
β̂j ± c × se(β̂j )
The p-value
Confidence where c is the (1 − α2 ) percentile in a tn−k−1 distribution
Intervals
The F-test
If the hypothesized value of a parameter (bj ) is inside the
confidence interval, we would not reject the null βj = bj
against βj 6= bj at the significance level α
Testing multiple exclusion restrictions
Econometrics
Unrestricted model:
F. Machado y = β0 + β1 x1 + β2 x2 + β3 x3 + ... + βk xk + u
Motivation
The Gaussian H0 : βk−q+1 = βk−q+2 = ... = βk = 0 H1 : Not H0
LRM
The t Test Restricted model:
The p-value y = β0 + β1 x1 + β2 x2 + β3 x3 + ... + βk−q xk−q + u
Confidence
Intervals
Under the null:
(SSRr − SSRur )/q
The F-test Fstatistic = ∼ F(q,n−k−1)
SSRur /(n − k − 1)
r stands for restricted and ur for unrestricted, q is number of
restrictions
Does SSRur decrease enough compared to SSRr ? If Fobs is ”too”
large we reject the null
Testing multiple exclusion restrictions
Alternative formulation
Econometrics
F. Machado
Motivation H0 : βk−q+1 = βk−q+2 = ... = βk = 0 H1 : NotH0
The Gaussian
LRM
The t Test (SSRr − SSRur )/q
Fstatistic = ∼ F(q,n−k−1)
The p-value SSRur /(n − k − 1)
Confidence
Intervals 2 − R 2 )/q
(Rur r
The F-test Fstatistic = 2 )/(n − k − 1)
∼ F(q,n−k−1)
(1 − Rur
Obtained by dividing the numerator and the denominator
above by SST
This is different from testing significance of each
coefficient individually!! It is a test of joint significance
Testing multiple exclusion restrictions
Overall significance of the model
Econometrics
F. Machado
Motivation H0 : β1 = β2 = ... = βk = 0 H1 : NotH0
The Gaussian
LRM Under the null use:
The t Test
The p-value
Confidence
(SST − SSR)/k
Intervals
F =
SSR/(n − k − 1)
The F-test
SSE /k
=
SSR/(n − k − 1)
R 2 /k
= ∼ F(k,n−k−1)
(1 − R 2 )/(n − k − 1)
Testing general linear restrictions: in the practice sessions!
Testing multiple exclusion restrictions
F test
Econometrics
Reject the null if the observed test statistic, Fobs , is
F. Machado larger than c, where c is implicitly given by:
Motivation Prob[F > c|H0 istrue] = α
The Gaussian
LRM
The t Test
Fail to Reject
The p-value the null
Confidence
Intervals
The F-test 1-α
Reject the null
α
c
Example
Econometrics H0 : β 2 = β 3 = 0
F. Machado
Independent Variable Coefficient Estimate Standard Error t ratio
Motivation
The Gaussian
Unrestricted model
LRM Intercept 5.33815 0.01218 438.36
The t Test Education (in years) 0.07614 0.00079 96.75
The p-value Labor Market Experience (in years) 0.03093 0.00087 35.38
Square of Labor Market Experience (in -0.00038 0.000018 -20.64
Confidence
years)
Intervals
R2 0.4774
The F-test
Mean Square Error 0.11342
Restricted model
Intercept 5.88400 0.00729 807.45
Education (in years) 0.06046 0.00081 75.05
R2 0.3374
Mean Square Error 0.14379
Figure: Dependent Variable: Log of monthly wage, n=11064
Example (Cont.)
Econometrics
F. Machado
Motivation
The Gaussian α = 0.05
LRM
The t Test
The p-value H0 : β 2 = β 3 = 0
Confidence
Intervals
The F-test
(0.4774 − 0.3374)/2
Fstatistic =
(1 − 0.4774)/(11064 − 3 − 1)
= 1581.4 > 3.00 ⇒ Reject H0