CHAPTER 4

STATISTICAL INFERENCE IN MULTIPLE REGRESSION

Lecture notes prepared for students — plain-language explanations with worked numerical examples

Why do we need this chapter?

Whenever you run a regression (e.g. the reg command in Stata), the software always returns a set of
estimated coefficients β̂ . But those are just estimates from ONE particular sample. Draw a different
sample, and the coefficients will change somewhat. The key question is: are these coefficients
statistically trustworthy, or could they simply be the result of sampling luck?

Statistical inference is the toolkit that answers exactly this question — using probability theory to test
whether a claim (a hypothesis) about the true population coefficient β is consistent with the data we
actually observed.

This chapter walks through the six most common types of hypothesis tests used in multiple regression:
● 4.1 — Testing an individual partial regression coefficient (does Xj affect Y?)
● 4.2 — Testing the overall significance of the model (are all slope coefficients jointly zero?)
● 4.3 — Testing whether two coefficients are equal to each other
● 4.4 — Testing whether the coefficients satisfy a linear restriction (Restricted Least Squares)
● 4.5 — Testing the stability of the model across time periods/groups (the Chow test)
● 4.6 — Testing the functional form of the model: linear or log–linear? (the MWD test)
💡 Key idea: What every test in this chapter has in common: we always compute a test statistic (t or F)
from the data, then compare it against a critical value taken from a probability table. If the computed
statistic is "unusually large" relative to the critical value, we reject H0.

4.1. Testing hypotheses about an individual partial regression

coefficient
This is the most basic and most frequently used test. The question being asked: "Does Xj actually affect
the expected value of Y, holding the other variables constant?"

a) Hypotheses
H0: βj = 0 H1: βj ≠ 0
H0 says: "holding the other variables constant, Xj has NO effect on Y." If we manage to reject H0, that is
statistical evidence that Xj really does have an effect.
b) The t test statistic
t = ( β̂j − 0 ) / se(β̂j)
Intuition: t measures how many standard errors away from zero the estimated coefficient β̂ j is. If β̂ j is
large while its standard error is small, t will be large — meaning it's unlikely that the true β is actually
zero (because if β were truly zero, it would be very unlikely to randomly obtain such a large β̂ by
chance).

This t statistic follows a Student's t distribution with (n − k) degrees of freedom, where n is the number
of observations and k is the total number of parameters estimated (including the intercept).

c) Decision rule
● If |computed t| > critical t (looked up from the t table with df = n−k at significance level α) →
REJECT H0 → Xj is statistically significant.
● If |computed t| ≤ critical t → DO NOT reject H0 → not enough evidence that Xj has an effect.
⚠️A note on language: Never write "H0 is accepted"! The correct phrasing in classical hypothesis testing
is "we fail to reject H0 at the x% level." We can never have enough evidence to prove H0 is true with
certainty — only enough (or not enough) evidence to reject it.

d) Statistical significance vs. economic significance — don't confuse the two!

Statistical significance is determined by the size of the t-statistic — it tells us how confident we are that
β ≠ 0.

Economic significance depends on the size and sign of the estimated coefficient itself — it tells us
whether the effect is meaningful in practical, real-world terms.

A variable can be highly statistically significant (very large t, e.g. from a huge sample) yet have a
coefficient too small to matter economically — and vice versa.

e) Three forms of the test (two-tail / one-tail)

Depending on the research question, we choose one of the following three forms. β0 is a specific value
we want to compare against (usually β0 = 0):
Test type H0 H1 Rejection region
Two-tail βj = β0 βj ≠ β0 |t0| > t(n−k), α/2
Right-tail βj ≤ β0 βj > β0 t0 > t(n−k), α
Left-tail βj ≥ β0 βj < β0 t0 < −t(n−k), α

A two-tail test is used when we only want to know "is it different from zero" (we don't care about the
direction). A one-tail test is used when economic theory already predicts the sign of β in advance (e.g. "a
higher price should lower demand," so we expect a negative coefficient).
f) Confidence Interval (CI)
Besides hypothesis testing, we can also construct a confidence interval for βi — a range of values we
"trust" contains the true population parameter, rather than just giving a single point estimate.

β̂i − tα/2·se(β̂i) < βi < β̂i + tα/2·se(β̂i)

If the confidence level is 95%, then tα/2 is the 97.5th percentile of the t(n−k) distribution. The quantity
(1 − α) is called the confidence coefficient, while α (0 < α < 1) is called the level of significance.
🔗 Connection: Hypothesis testing and confidence intervals are really two sides of the same coin: if β0
(usually 0) does NOT fall inside the 95% confidence interval for βj, that is equivalent to rejecting H0: βj =
0 at the 5% significance level.

Worked example: Determinants of college GPA (colGPA)

Model: colGPA = β1 + β2·hsGPA + β3·ACT + u, where hsGPA = high-school GPA, ACT = standardized ACT
score.

Question: Holding ACT constant, does hsGPA affect colGPA?

H0: β2 = 0 H1: β2 ≠ 0
Stata regression output gives: β̂ 2 = 0.4534 ; se(β̂ 2) = 0.0958. Computing t:

t = (0.4534 − 0) / 0.0958 = 4.73

With df = n − k = 141 − 3 = 138, the critical t value from the table at the 1% significance level (two-tail) is
2.61.

Comparison: |4.73| > 2.61 → REJECT H0 at the 1% level. Conclusion: hsGPA has a statistically significant
effect on colGPA.

Full regression output (n = 141 students):

Variable Coefficient Std. Error t P>|t|
hsGPA 0.4535 0.0958 4.73 0.000
ACT 0.0094 0.0108 0.87 0.383
_cons 1.2863 0.3408 3.77 0.000

Note the ACT variable: t = 0.87, very small, with P-value = 0.383 >> 0.05 → we do NOT reject H0: β(ACT)
= 0. That is, after controlling for hsGPA, the ACT score has NO statistically significant effect on college
GPA. R² = 0.1764 means the model explains only 17.64% of the variation in colGPA — fairly low, but
typical of cross-sectional data on human behavior.
4.2. Testing the overall significance of the sample regression (the
F test)
The t test in section 4.1 only answers the question for ONE variable at a time. But sometimes we want to
ask a broader question: "Do all the independent variables in the model, taken TOGETHER, help explain
anything about Y?" This is the overall significance test, and it is simultaneously a test of the significance
of R².

a) Hypotheses
For the model: Yi = β1 + β2X2i + β3X3i + ... + βkXki + ui

H0: β2 = β3 = ... = βk = 0 H1: not all slope coefficients are zero

b) The F test statistic

F = [ R² (n − k) ] / [ (1 − R²)(k − 1) ]
Here k is the total number of parameters estimated (including the intercept). Intuition: F grows larger as
R² grows larger — that is, the more of Y's variation the model explains relative to the unexplained
portion (1 − R²).

c) Decision rule
● If F > critical F = Fα,(k−1, n−k) → REJECT H0 → the model is jointly significant (at least one
variable has an effect).
● If F ≤ critical F → do not reject H0 → the model does not explain anything meaningful about Y.

Continuing example: the GPA model

From the ANOVA table: R² = 0.1764, n = 141, k = 3 (2 independent variables + intercept).

F = (0.1764 × 138) / [(1 − 0.1764) × 2] = 14.78

Critical value: F0.05,(2,138) = 3.062. Since 14.78 > 3.062 → REJECT H0. Conclusion: the model is jointly
statistically significant — hsGPA and ACT together do genuinely help explain colGPA (even though ACT
alone was not significant, as shown in section 4.1).
📌 Stata note: This is exactly the "Prob > F = 0.0000" figure Stata always prints in the top-right corner of
the regression output — that is the p-value of this overall F test.

4.3. Testing the equality of two regression coefficients

Sometimes the research question isn't "is β equal to zero" but rather "are these two coefficients equal
to each other." Example: do general work experience (exper) and tenure at the current employer
(tenure) have the SAME effect on wages?
Consider the model: Yi = β1 + β2X2i + β3X3i + β4X4i + ui

H0: β3 = β4 (i.e. β3 − β4 = 0) H1: β3 ≠ β4

Option 1 — the t test

t = [ (β̂3 − β̂4) − (β3 − β4) ] / se(β̂3 − β̂4)
under H0, (β3 − β4) = 0, and the standard error of the difference between the two coefficients is
computed as:

se(β̂3 − β̂4) = √[ var(β̂3) + var(β̂4) − 2cov(β̂3, β̂4) ]

⚠️Common pitfall: Pay attention to the MINUS sign in front of the covariance term! This is the single
most common mistake students make when computing this by hand — you can't just add the two
standard errors together. To get cov(β̂ 3, β̂ 4), you need the coefficient covariance matrix (in Stata: the
command estat vce after reg).

Option 2 — the F test

Equivalent to the t test above, we can also use an F test (in fact F = t² when there is a single restriction).
Rule: if the computed F exceeds the critical F at the chosen significance level, reject H0.

Option 3 — using Stata's built-in command

Instead of computing by hand, Stata lets you test this directly with:

test exper = tenure

Stata automatically pulls the covariance from the e(V) matrix and returns the F result immediately —
much faster and more accurate than doing it by hand.

Example: the model wage = f(educ, exper, tenure)

Regression output (n = 526):
Variable Coefficient Std. Error t
educ 0.5990 0.0513 11.68
exper 0.0223 0.0121 1.85
tenure 0.1693 0.0216 7.82
_cons −2.8727 0.7290 −3.94

Partial covariance matrix (from estat vce): var(β̂ exper)=0.000145; var(β̂ tenure)=0.000468;
cov(β̂ exper,β̂ tenure) = −0.000132.

se(β̂exper − β̂tenure) = 0.029635

t = (0.0223 − 0.1693) / 0.029635 = −4.958
With df = 522, critical t (5%, two-tail) ≈ 2. Since |−4.958| > 2 → REJECT H0. Conclusion: the effects of
"general work experience" (exper) and "tenure at the current employer" (tenure) on wages are
statistically significantly different — tenure at the current firm has a much stronger effect.

Confirming with the Stata command:

test exper = tenure → F(1, 522) = 24.58, Prob > F = 0.0000

Since 24.58 > F0.05,(1,522) = 3.85 (and indeed 24.58 ≈ (−4.958)²) → same conclusion: reject the
hypothesis that the two coefficients are equal.

4.4. Restricted Least Squares (RLS) — Testing linear equality

restrictions
This is a more general version of section 4.3: sometimes economic theory imposes a MORE COMPLEX
restriction across coefficients — not simply "equal to each other," but a linear combination equal to
some constant.

a) A classic example: the Cobb–Douglas production function

Yi = β1 X2iβ2 X3iβ3 e^Ui
where Y = output, X2 = labor, X3 = capital. Taking logs of both sides converts this into the familiar linear
form for OLS estimation:

lnYi = β0 + β2 lnX2i + β3 lnX3i + ui (β0 = lnβ1)

Economic theory on "constant returns to scale" (if both labor and capital increase by the same
proportion, output increases by exactly that same proportion) corresponds to the restriction:

β2 + β3 = 1
This is an example of a linear equality restriction. There are two ways to test it.

b) Option 1 — the t-Test Approach (directly on the combination of coefficients)

Estimate the unrestricted equation as usual, then test:

t = [ (β̂2+β̂3) − 1 ] / √[ var(β̂2) + var(β̂3) + 2cov(β̂2,β̂3) ]

(note this time it's a PLUS sign in front of cov, because we're looking at the SUM of two coefficients, not
the difference). If |t| exceeds the critical t → reject the hypothesis of constant returns to scale.

c) Option 2 — the F-Test Approach using Restricted Least Squares

From β2 + β3 = 1, we get β2 = 1 − β3. Substituting into the original model and rearranging algebraically:
 lnYi = β0 + (1−β3)lnX2i + β3lnX3i + ui ⇒ ln(Yi/X2i) = β0 +
β3·ln(X3i/X2i) + ui
This is called the restricted regression equation (RLS) — instead of regressing Y on the two separate
variables X2 and X3, we regress the "output/labor" ratio on the "capital/labor" ratio. The restriction has
already been "baked into" the model.

Once we have the RSS from both models (unrestricted and restricted), we compute:

F = [ (RSSR − RSSUR) / m ] / [ RSSUR / (n − k) ]

Where: RSSUR = residual sum of squares of the UNRESTRICTED model; RSSR = residual sum of squares of
the RESTRICTED model; m = number of restrictions (here m=1); k = number of parameters in the
unrestricted model; n = number of observations.

An equivalent formula using R²:

F = [ (R²UR − R²R) / m ] / [ (1 − R²UR) / (n − k) ]

● If F > critical F → REJECT H0 → the restriction is NOT consistent with the data.
● If F ≤ critical F → DO NOT reject H0 → the restriction is consistent with the data; the (simpler)
restricted model may be accepted.
⚠️A cautionary note (p. 269 of the textbook): The R² of the restricted and unrestricted models can
ONLY be compared when both models have the SAME dependent variable. In the example above, we
must use the RSS formula (not the plain R² formula) once the dependent variable changes from lnY to
ln(Y/X2).

Example: The Cobb–Douglas production function for the Mexican economy, 1955–1974
With n = 20 observations (GDP, employment, fixed capital), the unrestricted regression gives:

ln GDPt = −1.6524 + 0.3397 lnLabort + 0.8460 lnCapitalt

R²UR = 0.9951 ; RSSUR = 0.0136

The restricted regression (output/labor ratio on capital/labor ratio):

ln(GDP/Labor)t = −0.4947 + 1.0153 ln(Capital/Labor)t

R²R = 0.9777 ; RSSR = 0.0166. Here m = 1 (one restriction), k = 3, n = 20 → n−k = 17.

F = [(0.0166 − 0.0136)/1] / [0.0136/17] = 3.75

Critical value: F1,17,0.05 = 4.45. Since 3.75 < 4.45 → DO NOT REJECT H0. Conclusion: the data for the
Mexican economy during this period are consistent with the hypothesis of constant returns to scale.

Example 2 (general F testing): the wage model, restriction β(exper) = 0

This illustrates that the t test in section 4.1 can itself be viewed as a special case of the restricted F test
(m = 1 restriction, β(exper)=0).

Unrestricted model (3 variables): RSSUR, n=526, k=4, R²UR = 0.3064.

Restricted model (dropping the exper variable entirely — forcing β(exper)=0): reg wage educ tenure
gives R²R = 0.3019.
R² Denominator df
Unrestricted (educ, exper, tenure) 0.3064 n−k = 522
Restricted (educ, tenure) 0.3019 —

Applying the F formula above with m=1, k=4, n−k=522 yields a result matching F=(1.85)²≈3.4 from the t
test on the exper variable in the full regression — demonstrating that a single-variable t test and a
single-restriction F test are EQUIVALENT.

Example 3: The demand for chicken — testing TWO restrictions jointly

Model: lnYt = β1 + β2 lnX2t + β3 lnX3t + β4 lnX4t + β5 lnX5t + ui, where Y = per-capita chicken
consumption, X2 = real disposable income, X3 = price of chicken, X4 = price of pork, X5 = price of beef.

Hypothesis to test: chicken consumption is NOT affected by the prices of pork and beef, i.e.:

H0: β4 = β5 = 0 (m = 2 joint restrictions)

Unrestricted model (all 4 variables): R²UR = 0.9823. Restricted model (only income and chicken price
remain): R²R = 0.9801. n = 23, k = 5 → n−k = 18.

F = [(0.9823 − 0.9801)/2] / [(1 − 0.9823)/18] = 1.1224

Critical value: F0.05(2,18) = 3.55. Since 1.1224 < 3.55 → DO NOT REJECT H0.

Conclusion: there is no evidence that chicken demand depends on pork and beef prices in this sample.
The restricted (smaller) model — income and chicken price only — can be accepted as representing the
demand for chicken.

4.5. Testing for structural / parameter stability — the Chow test

An implicit assumption when estimating a regression is that the relationship between Y and X stays
CONSTANT throughout the sample period. But in reality, an economic shock (a crisis, a policy change, a
war...) can completely alter this relationship between two periods, or between two different groups of
subjects (e.g. northern vs. southern region).

The Chow test answers: "Should we estimate ONE common model for the entire dataset, or should we
split it into SEPARATE models for each period/group?"

a) Three regressions to estimate

Period 1: Yt = λ1 + λ2Xt + u1t (RSS1, df = n1 − k)
Period 2: Yt = γ1 + γ2Xt + u2t (RSS2, df = n2 − k)
 Pooled (both periods): Yt = α1 + α2Xt + ut (RSS3 = RSSR, df =
n1+n2−k)
The pooled regression (the third one) is the restricted model — because it implicitly imposes λ1=γ1 and
λ2=γ2 (i.e. it assumes the two periods are identical).

b) Procedure
● Step 1: Estimate the pooled regression (8.7.3), obtain RSSR = RSS3, df = n1+n2−k.
● Step 2: Estimate period 1 separately, obtain RSS1, df = n1−k.
● Step 3: Estimate period 2 separately, obtain RSS2, df = n2−k.
● Step 4: Compute the unrestricted RSS: RSSUR = RSS1 + RSS2, df = n1+n2−2k.

F = [ (RSSR − RSSUR) / k ] / [ RSSUR / (n1+n2−2k) ]

● Step 5: Compute the F ratio.

● Step 6: Compare the computed F against the critical F. If F > critical F → REJECT the hypothesis of
parameter stability → the two periods have a genuinely different Y-X relationship (estimate
them separately); otherwise, the pooled model is acceptable.

Example: Savings – Disposable Income relationship in the United States, 1970–1995

Question: has the savings-income relationship undergone a structural change between the two periods
1970–1981 (n1=12) and 1982–1995 (n2=14)? k=2 (intercept + slope).
Model Result RSS df
Period 1970–1981 Ŷt = 1.0161 + 0.0803Xt 1785.03 10
Period 1982–1995 Ŷt = 153.49 + 0.0148Xt 10,005.22 12
Pooled 1970–1995 Ŷt = 62.42 + 0.0376Xt 23,248.30 24

RSSUR = RSS1+RSS2 = 1785.03 + 10,005.22 = 11,790.25; RSSR =

RSS3 = 23,248.30
F = [(23,248.30 − 11,790.25)/2] / [11,790.25/22] = 10.69
Critical value at the 1% level, df(2,22): F0.01(2,22) = 5.72. Since 10.69 > 5.72 → STRONGLY REJECT H0.

Conclusion: the U.S. savings–income relationship underwent a substantial STRUCTURAL CHANGE

between 1970–1981 and 1982–1995 — the slope coefficient dropped sharply (from 0.08 down to just
0.015), indicating that the sensitivity of savings to income weakened considerably. A single pooled
model should not be used for these 26 years.
4.6. Testing the functional form of the regression model — the
MWD test
A common practical question when building a model: should you choose a LINEAR form (Y = β1+β2X+u)
or a LOG–LINEAR form (lnY = β1+β2lnX+u)? These two models cannot be compared directly using R²
because they have different dependent variables (Y vs. lnY). The MWD test (MacKinnon, White,
Davidson) solves exactly this problem.

Procedure
H0: the true model is linear H1: the true model is log–linear
● Step I: Estimate the LINEAR model, obtain the fitted values Ŷ (call this Yf).
● Step II: Estimate the LOG–LINEAR model, obtain the fitted values ln Ŷ (call this lnf).
● Step III: Compute Z1 = lnYf − lnf (the log of the linear model's fitted values, minus the log
model's fitted log-values).
● Step IV: Regress Y on the ORIGINAL X's plus Z1. If the coefficient of Z1 is statistically significant (t
test) → REJECT H0 → the linear model is NOT adequate.
● Step V: Compute Z2 = (antilog of lnf) − Yf.
● Step VI: Regress lnY on the LOGS of the X's plus Z2. If the coefficient of Z2 is statistically
significant → REJECT H1 → the log–linear model is NOT adequate.
🔎 Interpreting the results: There are four possible outcomes: (1) reject only H0 → choose log–linear; (2)
reject only H1 → choose linear; (3) reject both → neither form fits well, look for another specification;
(4) reject neither (as in the example below) → the data aren't strong enough to distinguish between
them, and either form may be used depending on the purpose of interpretation.

Example: The demand for roses in Detroit

Y = quantity of roses (dozens), X2 = wholesale price of roses, X3 = wholesale price of carnations (a
substitute good).

Linear model: Ŷt = 9734.22 − 3782.20·X2t + 2815.25·X3t (R² = 0.771)

Log–linear model: ln Ŷt = 9.2278 − 1.7607·lnX2t + 1.3398·lnX3t (R² = 0.729)

After computing Z1 and re-estimating the linear model with Z1 added:

Ŷt = 9727.57 − 3783.06X2t + 2817.72X3t + 85.23 Z1t (t on Z1 =

0.0207)
The coefficient of Z1 has t = 0.0207 — VERY small, NOT statistically significant → we do NOT reject H0
(the linear model is acceptable).

Similarly, after computing Z2 and re-estimating the log model with Z2 added:

ln Ŷt = 9.1486 − 1.9699 lnXt + 1.5891 lnX2t − 0.0013 Z2t (t on Z2 =

−1.6612)
The coefficient of Z2 is also NOT statistically significant at the 5% level → we do NOT reject H1 (the log–
linear model is also acceptable).
📝 Practical takeaway: Conclusion of this example: the data are not powerful enough to cleanly
distinguish between the two functional forms — both "survive" the test. In this situation, an analyst
typically chooses the form based on the intended use: if the slope coefficient is meant to represent an
elasticity directly, the log–linear form is usually preferred because it's easier to interpret economically.

Appendix — A quick recap of common regression functional

forms
This section briefly reviews the functional forms directly related to the MWD test in section 4.6, to make
sure the meaning of the slope coefficient is clear in each case.

1) The Log–linear (Log–Log / exponential) model

Yi = β1 Xiβ2 e^ui ⇔ lnYi = lnβ1 + β2 lnXi + ui
β2 is the ELASTICITY of Y with respect to X: the percentage change in Y for a given (small) percentage
change in X. This is precisely why the log–log form is so popular in economics (demand elasticity, supply
elasticity, production functions, etc.).

2) The semilog Log–Lin model (used to measure growth rates)

lnYt = β1 + β2 t + ut
β2 gives the RELATIVE (percentage) growth rate of Y per unit of time. Example, using expenditure-on-
services data: ln EXSt = 8.3226 + 0.00705t → expenditure on services grows on average by 0.705% per
quarter.

Compare this to the simple linear trend model Yt = β1+β2t+ut: EXSt = 4111.5 + 30.67t → expenditure on
services increases on average by 30.67 billion dollars per quarter (an ABSOLUTE amount, not a
percentage).

3) The semilog Lin–Log model

Yi = β1 + β2 lnXi + ui
β2 gives the ABSOLUTE change in Y when X changes by 1%. Example, food expenditure in India:
FoodExpi = −1283.9 + 257.27·lnTotalExpi → a 1% increase in total expenditure raises food expenditure
on average by about 2.57 rupees.
🧠 Memory trick: Quick rule of thumb: LOG on the Y side → talks about the PERCENTAGE change in Y.
LOG on the X side → talks about the response to a 1% change in X. LOG on both sides → the slope
coefficient is an elasticity.
Chapter 4 summary
The table below summarizes all six types of tests covered in this chapter — use it for quick review before
homework or exams:
Sectio Question being tested Statistic used When to use it
n
4.1 Does one variable Xj affect Y? t = β̂ j/se(β̂ j) Evaluating a single
variable
4.2 Are all variables jointly F = R²(n-k)/[(1-R²)(k-1)] Evaluating the whole
significant? model
4.3 Are two coefficients equal? t or F on (β̂ i−β̂ j) Comparing two
variables
4.4 Do coefficients satisfy a linear F = (RSSR−RSSUR)/m ÷ Testing an economic
restriction? RSSUR/(n-k) hypothesis (e.g.
constant returns to
scale)
4.5 Is the model stable across Chow-type F on pooled vs. Time-series data
periods/groups? separate RSS suspected of a
structural break
4.6 Should the form be linear or t on the added variables Z1, Choosing the
log-linear? Z2 (MWD test) appropriate functional
form

Tips for homework and exams

● Always determine the degrees of freedom (df) before looking up a table value — different df
give different critical values for both t and F.
● For F tests involving RSS/R², always check whether the restricted and unrestricted models share
the same dependent variable — if not, use RSS instead of R².
● Always write out H0 and H1 explicitly before computing anything — it's easy to get the rejection
direction backwards otherwise.
● Your final conclusion should connect back to the ECONOMIC meaning of the variable, not stop at
a mechanical "reject" or "do not reject."

— End of Chapter 4 —