ECO2009 Econometrics I

Lecture 5 Multiple Regression Analysis: Inference (Chapter 4)

Yaein Baek

Sogang University

Spring 2026

ECO2009 Econometrics I Lecture 5 1 / 43

Sampling Distributions of the OLS Estimators

We studied the expected value and variance of the OLS estimators

In order to perform statistical inference, we need to know the full
sampling distribution of the OLS estimators
Statistical inference: To infer the properties of the population
▶ Hypothesis tests about population parameters
▶ Construction of confidence intervals

ECO2009 Econometrics I Lecture 5 2 / 43

Normality Assumption

Assumption MLR.6 (Normality of error terms)

The population error u is independent of the explanatory variables
X1 , . . . , Xk and is normally distributed with mean zero and variance σ 2 :

u ∼ N(0, σ 2 )

Much stronger than our previous assumptions.

It implies MLR.4 (ZCM) and MLR.5 (homoskedasticity).
Assumptions MLR.1 through MLR.6 are called the classical linear
model (CLM) assumptions.

ECO2009 Econometrics I Lecture 5 3 / 43

Normality Assumption

Under the normality assumption (MLR.6),

Y |(X1 , . . . , Xk ) ∼ N(β0 + β1 X1 + · · · + βk Xk , σ 2 )

ECO2009 Econometrics I Lecture 5 4 / 43

Normality Assumption

How restrictive is the normality assumption?

The error term is the sum of “many” different unobserved factors, we
can invoke the central limit theorem (CLT) to conclude u has an
approximate normal distribution.
Problems:
▶ How many different factors? Number large enough?
▶ Possibly very heterogenuous distributions of individual factors
▶ How independent are the different factors?
The normality of the error term is an empirical question.

ECO2009 Econometrics I Lecture 5 5 / 43

Normality Assumption

Examples where normality cannot hold:

▶ Skewed distribution
⋆ The distribution of wages is usually right-skewed
▶ Truncated distribution
⋆ Minimum wage
▶ Discrete dependent variable
⋆ Number of arrests (takes on a small number of integer values)
⋆ Unemployment (indicator variable, takes on only 1 or 0)
In some cases, normality can be achieved through transformations of
the dependent variable
▶ Use log(wage) instead of wage
For the purposes of statistical inference, the assumption of normality
can be replaced by a large sample size (Chapter 5)

ECO2009 Econometrics I Lecture 5 6 / 43

Normality Assumption

Figure: Log-transformation example

Original Data Log-transformation

ECO2009 Econometrics I Lecture 5 7 / 43

Sampling Distributions of the OLS Estimators

Gauss-Markov assumptions: MLR.1-MLR.5

Classical linear model (CLM) assumptions: MLR.1-MLR.6

Theorem 4.1 Normal sampling distributions

Under the CLM assumptions MLR.1 through MLR.6, conditional on the
sample values of the independent variables X,

β̂j ∼ N(βj , Var(β̂j ))

σ2
where Var(β̂j ) = P
(X −X̄ 2 2 . Therefore
i ij j ) (1−Rj )

β̂j − βj
∼ N(0, 1).
sd(β̂j )

ECO2009 Econometrics I Lecture 5 8 / 43

Testing Hypotheses about a Single Population Parameter

The population model satisfies the CLM assumptions

Y = β0 + β1 X1 + · · · + βk Xk + u

Note that the variance of the error term σ 2 is unknown,

P 2 so replace
2 1
sd(β̂j ) to se(β̂j ) using the estimator σ̂ = n−k−1 i ûi

Theorem 4.2 t distribution for the standardized estimators

Under the CLM assumptions MLR.1 through MLR.6, conditional on the
sample values of the independent variables X

β̂j − βj
∼ tn−k−1 .
se(β̂j )

where n − k − 1 is the degrees of freedom (df) of the t distribution.

ECO2009 Econometrics I Lecture 5 9 / 43

Testing Hypotheses about a Single Population Parameter

The sampling distribution of the OLS estimator (Theorem 4.2) allows

us to test hypothesis involving the βj

H0 : βj = 0
▶ Xj has no effect on the expected value of Y , after controlling for all
other independent variables

Example: Wage equation

log(wage) = β0 + β1 educ + β2 exper + β3 tenure + u

▶ H0 : β2 = 0 ⇐⇒ Holding education and tenure fixed, experience has no
effect on wage.
▶ If β2 > 0, then experience increases productivity (and thus wage).

ECO2009 Econometrics I Lecture 5 10 / 43

Testing Hypotheses about a Single Population Parameter

Recall that
β̂j − βj
∼ tn−k−1 .
se(β̂j )
The t statistic (or t ratio) of β̂j

β̂j
tβ̂j ≡
se(β̂j )

If the null hypothesis H0 : βj = 0 is true,

tβ̂j ∼ tn−k−1

which follows the tn−k−1 distribution under the null hypothesis

ECO2009 Econometrics I Lecture 5 11 / 43

Testing Hypotheses about a Single Population Parameter

The t statistic of β̂j

β̂j
tβ̂j =
se(β̂j )

If the estimated coefficient β̂j is far away from zero, the less likely it is that
the null hypothesis holds true. Being “far away” from zero is measured by
the variability of β̂j , i.e. its standard deviation.
The t statistic measures how many estimated standard deviations the
estimated coefficient is away from zero.
Values of tβ̂j sufficiently far way from zero will result in a rejection of H0 .
The rejection rule depends on the alternative hypothesis and the chosen
significance level of the test.
▶ Goal: Define a rejection rule so that, if it is true, H0 is rejected only
with a small probability (= significance level, e.g. 5%)

ECO2009 Econometrics I Lecture 5 12 / 43

Testing Hypotheses about a Single Population Parameter

Two-sided alternative:
H1 : βj ̸= 0
One-sided alternative:
H1 : βj > 0
or
H1 : βj < 0
In most cases, we are interested in two-sided alternatives
▶ We may not have any prior information on the sign of βj
In some cases, we may want to focus on one-sided alternatives

ECO2009 Econometrics I Lecture 5 13 / 43

Testing against One-Sided Alternatives

Reject the null hypothesis in favor of the

alternative hypothesis H1 : βj > 0 if the
estimated coefficient is “too large” (i.e.
larger than a critical value).
Construct the critical value c so that, if the
null hypothesis is true, it is rejected in, for
example, 5% of the cases.
In the given example, c0.05 = 1.701 is the
point of the t-distribution with 28 degrees
of freedom that is exceeded in 5% of the
cases.
Reject the null if tβ̂j > c0.05

ECO2009 Econometrics I Lecture 5 14 / 43

Testing against One-Sided Alternatives
Example 4.1 Hourly wage equation

Test whether, after controlling for education and tenure, higher work
experience leads to higher hourly wages.

Test H0 : βexper = 0 against H1 : βexper > 0

The t statistic
0.0041
texper = ≈ 2.41
0.0017
df = n − k − 1 = 526 − 3 − 1 = 522 is large, the standard normal
approximation applies

ECO2009 Econometrics I Lecture 5 15 / 43

Testing against One-Sided Alternatives
Example 4.1 Hourly wage equation

Test H0 : βexper = 0 against H1 : βexper > 0

texper = 2.41, df = 522
The critical value is c0.05 = 1.645 at the 5% significance level

texper > c0.05

We reject the null hypothesis. The effect of experience on hourly

wage is statistically greater than zero at the 5% significance level.
We can also check that the null hypothesis is rejected at the 1%
significance level (c0.01 = 2.326)

ECO2009 Econometrics I Lecture 5 16 / 43

Testing against One-Sided Alternatives

Reject the null hypothesis in favor of the

alternative hypothesis if the estimated
coefficient is “too small” (i.e. smaller than
a critical value).
Construct the critical value so that, if the
null hypothesis is true, it is rejected in, for
example, 5% of the cases.
In the given example, −c0.05 = −1.734 is
the point of the t distribution with 18
degrees of freedom so that 5% of the cases
are below the point.
Reject the null if tβ̂j < −c0.05

ECO2009 Econometrics I Lecture 5 17 / 43

Testing against One-Sided Alternatives
Example 4.2 Student performance and school size

Test whether smaller school size leads to better student performance.

Test H0 : βenroll = 0 against H1 : βenroll < 0

The t statistic
−0.00020
tenroll = ≈ −0.91
0.00022
df = n − k − 1 = 408 − 3 − 1 = 404 is large, the standard normal
approximation applies

ECO2009 Econometrics I Lecture 5 18 / 43

Testing against One-Sided Alternatives
Example 4.2 Student performance and school size

Test H0 : βenroll = 0 against H1 : βenroll < 0

tenroll = −0.91, df = 404
The critical value is −c0.05 = −1.645 at the 5% significance level

tenroll > −c0.05

We fail to reject the null hypothesis that there is no effect of school

size on student performance at the 5% significance level.
The null hypothesis is also not rejected at the 10% significance level.

ECO2009 Econometrics I Lecture 5 19 / 43

Testing against Two-Sided Alternatives

Reject the null hypothesis in favour of the

alternative hypothesis if the absolute value
of the estimated coefficient is too large.
Construct the critical value so that, if the
null hypothesis is true, it is rejected in, for
example, 5% of the cases.
In the given example, ±c0.025 = ±2.06 are
the points of the t-distribution with 25
degrees of freedom so that 5% of the cases
lie in the two tails.
Reject the null if |tβ̂j | > c0.025

ECO2009 Econometrics I Lecture 5 20 / 43

Testing against Two-Sided Alternatives
Example 4.3 Determinants of college GPA

Test whether the variables of the college GPA model are statistically
significant.

Test H0 : βj = 0 against H1 : βj ̸= 0
df = n − k − 1 = 141 − 3 − 1 = 137 is large, the standard normal
approximation applies.
The critical value at the 1% significance level is c0.005 = 2.575

ECO2009 Econometrics I Lecture 5 21 / 43

Testing against Two-Sided Alternatives
Example 4.3 Determinants of college GPA

Test H0 : βj = 0 against H1 : βj ̸= 0
The critical value at the 1% significance level is c0.005 = 2.575

|thsGPA | = 4.38 > 2.575

|tACT | = 1.36 < 2.575
|tskipped | = | − 3.19| > 2.575

The effects of hsGPA and skipped are significantly different from zero
at the 1% significance level. The effect of ACT is not significantly
different from zero, not even at the 10% significance level.

ECO2009 Econometrics I Lecture 5 22 / 43

Testing against Two-Sided Alternatives

“Statistically significant”” variables in a regression

If a regression coefficient is different from zero in a two-sided test, the
corresponding variable is said to be “statistically significant”.
If the number of degrees of freedom is large enough so that the
normal approximation applies, the following rules of thumb apply:

|tβ̂j | > 1.645 ⇒ statistically significant at 10% level

|tβ̂j | > 1.96 ⇒ statistically significant at 5% level
|tβ̂j | > 2.575 ⇒ statistically significant at 1% level

ECO2009 Econometrics I Lecture 5 23 / 43

Testing more general hypotheses about a regression coefficient

Null hypothesis:
H0 : βj = aj
where aj is the hypothesized value of the coefficient
The t statistic:
β̂j − aj
t=
se(β̂j )
The test works exactly as before, except that the hypothesized value
is subtracted from the estimate when forming the statistic.

ECO2009 Econometrics I Lecture 5 24 / 43

Testing more general hypotheses about a regression coefficient
Example 4.4 Campus crime and enrollment

An interesting hypothesis is whether crime increases by one percent if

enrollment is increased by one percent.

H0 : βlog(enroll) = 1, H1 : βlog(enroll) ̸= 1
df = 97 − 2 = 95, the two-sided 5% critical value for a t95
distribution is c0.025 = 1.985
1.27 − 1
t= ≈ 2.45 > 1.985
0.11
The null hypothesis is rejected at the 5% level, the elasticity of crime
with respect to enrollment is statistically different from one.

ECO2009 Econometrics I Lecture 5 25 / 43

Computing p-Values for t Tests

If the significance level is made smaller and smaller, there will be a

point where the null hypothesis cannot be rejected anymore.
The reason is that, by lowering the significance level, one wants to
avoid more and more to make the error of rejecting a correct H0 .
The smallest significance level at which the null hypothesis is still
rejected, is called the p-value of the hypothesis test.
A small p-value is evidence against the null hypothesis because one
would reject the null hypothesis even at small significance levels.
A large p-value is evidence in favor of the null hypothesis.

ECO2009 Econometrics I Lecture 5 26 / 43

Computing p-Values for t Tests

The p-value is the significance level at

which one is indifferent between rejecting
and not rejecting the null hypothesis.
In the two-sided case, the p-value is thus
the probability that the t-distributed
variable takes on a larger absolute value
than the realized value of the test statistic,
e.g. P(|t| > 1.85) = 2 × 0.0359 = 0.0718.
From this, it is clear that a null hypothesis
is rejected if and only if the corresponding
p-value is smaller than the significance level.
For example, for a significance level of 5%
the t statistic would not lie in the rejection
region.

ECO2009 Econometrics I Lecture 5 27 / 43

Economic versus Statistical Significance

The statistical significance of a variable Xj is determined entirely by the size

of tβ̂j , whereas the economic significance of Xj is related to the size (and
sign) of β̂j
The fact that a coefficient is statistically significant does not necessarily
mean it is economically or practically significant.
Guidelines:
1 Check for statistical significance. If the variable is statistically
significant, discuss the magnitude of the coefficient to get an idea of its
practical or economic importance.
2 If a variable is statistically insignificant at the usual levels (10%, 5%, or
1%), one may think of dropping it from the regression. If the sample
size is small, effects might be imprecisely estimated so that the case for
dropping insignificant variables is less strong.
3 If a variable is statistically and economically important but has the
“wrong” sign, the regression model might be misspecified.

ECO2009 Econometrics I Lecture 5 28 / 43

Confidence Intervals

Confidence intervals (CI) provide a range of likely values for the

population parameter.
Under the CLM assumptions, we can compute the CI from simple
manipulation of the result in Theorem 4.2

β̂j − βj
∼ tn−k−1
se(β̂j )
!
β̂j − βj
P −cα/2 < < cα/2 = 1 − α,
se(β̂j )

where α ∈ (0, 1) and cα/2 is the (1 − α/2)-percentile of the tn−k−1

distribution.
(1 − α) is the confidence level and cα/2 is the critical value of a
two-sided test.

ECO2009 Econometrics I Lecture 5 29 / 43

Confidence Intervals
This is equivalent to
 
P β̂j − cα/2 · se(β̂j ) < βj < β̂j + cα/2 · se(β̂j ) = 1 − α

The 100(1 − α)% confidence interval for βj :

h i
β̂j − cα/2 · se(β̂j ), β̂j + cα/2 · se(β̂j )

Interpretation of the confidence interval:

▶ The bounds of the interval are random.
▶ In repeated samples, the interval that is constructed in the above way
will cover the population regression coefficient in 95% of the cases.
Relationship between confidence intervals and hypotheses tests

aj ∈
/ CI ⇒ Reject H0 : βj = aj in favor of H1 : βj ̸= aj

ECO2009 Econometrics I Lecture 5 30 / 43

Confidence Intervals
Example 4.8 Model of firms’ R&D expenditures

n = 32, R 2 = 0.918, df = 32 − 2 − 1 = 29 ⇒ c0.025 = 2.045

The 95% confidence interval of βlog(sales)
[1.084 − 2.045 × 0.060, 1.084 + 2.045 × 0.060] = [0.961, 1.21]
▶ The effect of sales on R&D is significantly different from zero at the
5% level because zero is outside the interval.
The 95% confidence interval of βprofmarg
[0.0217 − 2.045 × 0.0218, 0.0217 + 2.045 × 0.0218] = [−0.0045, 0.0479]
▶ We fail to reject the null H0 : βprofmarg = 0 in favor of
H1 : βprofmarg ̸= 0 at 5% level because zero is lies in the interval.
ECO2009 Econometrics I Lecture 5 31 / 43
Testing Linear Restrictions

Hypotheses associated with more than one parameter

▶ Single linear combination of parameters
▶ Exclusion restrictions
▶ General linear restrictions
Restricted and unrestricted models
We assume the CLM assumptions(MLR.1-MLR.6) hold

ECO2009 Econometrics I Lecture 5 32 / 43

Testing a Single Linear Restriction
Example: Return to education at two-year vs. at four-year colleges
The population includes working people with a high school degree
and the model is

The hypothesis of interest is whether one year at a junior college is

worth one year at a university against the one-sided alternative

H0 : β 1 = β 2
H1 : β1 < β2

A possible test statistic would be

β̂1 − β̂2
t=
se(β̂1 − β̂2 )

ECO2009 Econometrics I Lecture 5 33 / 43

Testing a Single Linear Restriction
Example: Return to education at two-year vs. at four-year colleges

Two questions
▶ What is the distribution of the test statistic?
▶ What is the standard error of (β̂1 − β̂2 )?
Another way to test the hypothesis
▶ Rewrite the model: Let θ = β1 − β2 , then

log(wage) = β0 + β1 jc + β2 univ + β3 exper + u

= β0 + (θ + β2 )jc + β2 univ + β3 exper + u
= β0 + θjc + β2 (jc + univ ) + β3 exper + u

▶ Estimate the model and perform a t-test for θ = 0

ECO2009 Econometrics I Lecture 5 34 / 43

Testing Multiple Linear Restrictions
Testing Exclusion Restrictions

We want to test whether a group of variables has no effect on the

dependent variable

H0 : β3 = 0, β4 = 0, β5 = 0 against H1 : H0 is not true

We need a way to test the exclusion restrictions jointly

ECO2009 Econometrics I Lecture 5 35 / 43

Testing Multiple Linear Restrictions
Testing Exclusion Restrictions

We compare the SSR from the restricted model with the SSR from
the unrestricted model.
Setup
▶ SSRr : SSR from the restricted model
▶ SSRur : SSR from the unrestricted model
▶ q: number of restrictions (= dfr − dfu )
Test statistic:
(SSRr − SSRur )/q
F ≡ ∼ Fq,n−k−1
SSRur /(n − k − 1)

▶ The relative increase of the sum of squared residuals when going from
H1 to H0 follows a F distribution, if the null hypothesis H0 is correct

ECO2009 Econometrics I Lecture 5 36 / 43

Testing Multiple Linear Restrictions
Testing Exclusion Restrictions

Estimation of the unrestricted model

SSRur = 183.186, dfur = n − k − 1 = 353 − 5 − 1 = 347

ECO2009 Econometrics I Lecture 5 37 / 43

Testing Multiple Linear Restrictions
Testing Exclusion Restrictions

Estimation of the restricted model

SSRr = 198.311, q = 3
F statistic
(SSRr − SSRur )/q
F =
SSRur /(n − k − 1)
(198.311 − 183.186)/3
= ≈ 9.55
183.186/(353 − 5 − 1)

ECO2009 Econometrics I Lecture 5 38 / 43

Testing Multiple Linear Restrictions

A F -distributed variable only takes on

positive values. This corresponds to the
fact that the sum of squared residuals can
only increase if one moves from H1 to H0 .
Choose the critical value cα so that we
incorrectly reject the null hypothesis in only
100α% of the cases.

In the previous example, F ∼ F3,347 under the null hypothesis so the

critical value at 1% level is c0.01 = 3.78.
F = 9.55 > 3.78 and p-value = P(F > 9.55) ≈ 0
The three variables are “jointly significant”

ECO2009 Econometrics I Lecture 5 39 / 43

Testing Multiple Linear Restrictions

The R 2 form of the F statistic

2 − R 2 )/q
(Rur r
F = 2
(1 − Rur )/(n − k − 1)

Note that this is valid only when the dependent variables are the
same across models
▶ Although this form is convenient for testing exclusion restrictions, it
cannot be applied for testing all linear restrictions

ECO2009 Econometrics I Lecture 5 40 / 43

Testing Multiple Linear Restrictions
Testing Exclusion Restrictions: Overall Significance

Test of overall significance of a regression

Y = β0 + β1 X1 + · · · + βk Xk + u

The null hypothesis states that the explanatory variables are not
useful at all in explaining the dependent variable

H0 : β1 = β2 = · · · = βk = 0, H1 : H0 is not true

Restricted model: Y = β0 + u
F statistic
(SSRr − SSRur )/q R 2 /k
F = = ∼ Fk,n−k−1
SSRur /(n − k − 1) (1 − R 2 )/(n − k − 1)

ECO2009 Econometrics I Lecture 5 41 / 43

Testing General Linear Restrictions
Example: Test whether house price assessments are rational

If house price assessments are rational, a 1% change in the

assessment should be associated with a 1% change in price. In
addition, other known factors should not influence the price once the
assessed value has been controlled for.

H0 : β1 = 1, β2 = 0, β3 = 0, β4 = 0, H1 : H0 is not true

ECO2009 Econometrics I Lecture 5 42 / 43

Testing General Linear Restrictions
Example: Test whether house price assessments are rational

Unrestricted regression:

log(price) = β0 +β1 log(assess)+β2 log(lotsize)+β3 log(sqrtft)+β4 bdrms +u

Restricted regression:

log(price) = β0 + log(assess) + u
log(price) − log(assess) = β0 + u

F statistic
(SSRr − SSRur )/q (1.880 − 1.822)/4
F = = ≈ 0.661
SSRur /(n − k − 1) 1.822/(88 − 4 − 1)

F ∼ F4,83 under the null ⇒ c0.05 = 2.50 ⇒ Fail to reject H0

ECO2009 Econometrics I Lecture 5 43 / 43