ECONOMETRICS

Model Specification
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Consequences of variable misspecification

True model

Y = 1 +  2 X 2 + u Y = 1 +  2 X 2 +  3 X 3 + u

Coefficients are biased (in

Correct specification,
Yˆ = b1 + b2 X 2 no problems
general). Standard
Fitted model

errors are invalid.

Yˆ = b1 + b2 X 2
Correct specification,
+ b3 X 3 no problems

In this sequence we will investigate the consequences of including an irrelevant variable in

a regression model.

1
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Consequences of variable misspecification

True model

Y = 1 +  2 X 2 + u Y = 1 +  2 X 2 +  3 X 3 + u

Coefficients are biased (in

Correct specification,
Yˆ = b1 + b2 X 2 no problems
general). Standard
Fitted model

errors are invalid.

Coefficients are
Yˆ = b1 + b2 X 2 unbiased (in general),
but inefficient (big Correct specification,
+ b3 X 3 variance). no problems
Standard errors are
valid (in general)

The effects are different from those of omitted variable misspecification. In this case the
coefficients in general remain unbiased, but they are inefficient. The standard errors remain
valid, but are needlessly large.
2
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3

These results can be demonstrated quickly.

3
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3
Y = 1 +  2 X 2 + 0 X 3 + u

Rewrite the true model adding X3 as an explanatory variable, with a coefficient of 0. Now the
true model and the fitted model coincide. Hence b2 will be an unbiased estimator of 2 and
b3 will be an unbiased estimator of 0.
4
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3
Y = 1 +  2 X 2 + 0 X 3 + u

 u2 1
 =2

b2
(
 2i 2
X − X )2
1 − r 2
X2 ,X3

However, the variance of b2 will be larger than it would have been if the correct simple
regression had been run because it includes the factor 1 / (1 – r2), where r is the correlation
between X2 and X3.
5
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3
Y = 1 +  2 X 2 + 0 X 3 + u

 u2 1
 =
2

b2
(
 2i 2
X − X )2
1 − r 2
X2 ,X3

The estimator b2 using the multiple regression model will therefore be less efficient than the
alternative using the simple regression model.

6
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3
Y = 1 +  2 X 2 + 0 X 3 + u

 u2 1
 =2

b2
(
 2i 2
X − X )2
1 − r 2
X2 ,X3

The intuitive reason for this is that the simple regression model exploits the information
that X3 should not be in the regression, while with the multiple regression model you find
this out from the regression results.
7
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3
Y = 1 +  2 X 2 + 0 X 3 + u

 u2 1
 =2

b2
(
 2i 2
X − X )2
1 − r 2
X2 ,X3

The standard errors remain valid, because the model is formally correctly specified, but
they will tend to be larger than those obtained in a simple regression, reflecting the loss of
efficiency.
8
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

Y = 1 +  2 X 2 + u

Yˆ = b1 + b2 X 2 + b3 X 3
Y = 1 +  2 X 2 + 0 X 3 + u

 u2 1
 =
2

b2
(
 2i 2
X − X )2
1 − r 2
X2 ,X3

These are the results in general. Note that if X2 and X3 happen to be uncorrelated, there will
be no loss of efficiency after all.

9
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

. reg LGFDHO LGEXP LGSIZE

Source | SS df MS Number of obs = 868

---------+------------------------------ F( 2, 865) = 460.92
Model | 138.776549 2 69.3882747 Prob > F = 0.0000
Residual | 130.219231 865 .150542464 R-squared = 0.5159
---------+------------------------------ Adj R-squared = 0.5148
Total | 268.995781 867 .310260416 Root MSE = .388

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2866813 .0226824 12.639 0.000 .2421622 .3312003
LGSIZE | .4854698 .0255476 19.003 0.000 .4353272 .5356124
_cons | 4.720269 .2209996 21.359 0.000 4.286511 5.154027
------------------------------------------------------------------------------

The analysis will be illustrated using a regression of LGFDHO, the logarithm of annual
household expenditure on food eaten at home, on LGEXP, the logarithm of total annual household
expenditure, and LGSIZE, the logarithm of the number of persons in the household.
10
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

. reg LGFDHO LGEXP LGSIZE

Source | SS df MS Number of obs = 868

---------+------------------------------ F( 2, 865) = 460.92
Model | 138.776549 2 69.3882747 Prob > F = 0.0000
Residual | 130.219231 865 .150542464 R-squared = 0.5159
---------+------------------------------ Adj R-squared = 0.5148
Total | 268.995781 867 .310260416 Root MSE = .388

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2866813 .0226824 12.639 0.000 .2421622 .3312003
LGSIZE | .4854698 .0255476 19.003 0.000 .4353272 .5356124
_cons | 4.720269 .2209996 21.359 0.000 4.286511 5.154027
------------------------------------------------------------------------------

The source of the data was the 1995 US Consumer Expenditure Survey. The sample size was 868.

11
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

. reg LGFDHO LGEXP LGSIZE LGHOUS

Source | SS df MS Number of obs = 868

---------+------------------------------ F( 3, 864) = 307.22
Model | 138.841976 3 46.2806586 Prob > F = 0.0000
Residual | 130.153805 864 .150640978 R-squared = 0.5161
---------+------------------------------ Adj R-squared = 0.5145
Total | 268.995781 867 .310260416 Root MSE = .38812

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2673552 .0370782 7.211 0.000 .1945813 .340129
LGSIZE | .4868228 .0256383 18.988 0.000 .4365021 .5371434
LGHOUS | .0229611 .0348408 0.659 0.510 -.0454214 .0913436
_cons | 4.708772 .2217592 21.234 0.000 4.273522 5.144022
------------------------------------------------------------------------------

Now add LGHOUS, the logarithm of annual expenditure on housing services. It is safe to
assume that LGHOUS is an irrelevant variable and, not surprisingly, its coefficient is not
significantly different from zero.
12
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

. reg LGFDHO LGEXP LGSIZE LGHOUS . cor LGHOUS LGEXP LGSIZE

(obs=869)
Source | SS df MS Number of obs = 868
---------+------------------------------ | LGHOUS
F( 3, LGEXP LGSIZE
864) = 307.22
Model | 138.841976 3 46.2806586 --------+---------------------------
Prob > F = 0.0000
Residual | 130.153805 864 .150640978 lGHOUS| 1.0000
R-squared = 0.5161
---------+------------------------------ LGEXP| 0.8137 1.0000 = 0.5145
Adj R-squared
Total | 268.995781 867 .310260416 LGSIZE| 0.3256
Root MSE0.4491 = 1.0000
.38812

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2673552 .0370782 7.211 0.000 .1945813 .340129
LGSIZE | .4868228 .0256383 18.988 0.000 .4365021 .5371434
LGHOUS | .0229611 .0348408 0.659 0.510 -.0454214 .0913436
_cons | 4.708772 .2217592 21.234 0.000 4.273522 5.144022
------------------------------------------------------------------------------

It is however highly correlated with LGEXP (correlation coefficient 0.81), and also, to a
lesser extent, with LGSIZE (correlation coefficient 0.33).

13
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

. reg LGFDHO LGEXP LGSIZE

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2866813 .0226824 12.639 0.000 .2421622 .3312003
LGSIZE | .4854698 .0255476 19.003 0.000 .4353272 .5356124
_cons | 4.720269 .2209996 21.359 0.000 4.286511 5.154027
------------------------------------------------------------------------------

. reg LGFDHO LGEXP LGSIZE LGHOUS

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2673552 .0370782 7.211 0.000 .1945813 .340129
LGSIZE | .4868228 .0256383 18.988 0.000 .4365021 .5371434
LGHOUS | .0229611 .0348408 0.659 0.510 -.0454214 .0913436
_cons | 4.708772 .2217592 21.234 0.000 4.273522 5.144022
------------------------------------------------------------------------------

Its inclusion does not cause the coefficients of those variables to be biased.

14
VARIABLE MISSPECIFICATION II: INCLUSION OF AN IRRELEVANT VARIABLE

. reg LGFDHO LGEXP LGSIZE

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2866813 .0226824 12.639 0.000 .2421622 .3312003
LGSIZE | .4854698 .0255476 19.003 0.000 .4353272 .5356124
_cons | 4.720269 .2209996 21.359 0.000 4.286511 5.154027
------------------------------------------------------------------------------

. reg LGFDHO LGEXP LGSIZE LGHOUS

------------------------------------------------------------------------------
LGFDHO | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------+--------------------------------------------------------------------
LGEXP | .2673552 .0370782 7.211 0.000 .1945813 .340129
LGSIZE | .4868228 .0256383 18.988 0.000 .4365021 .5371434
LGHOUS | .0229611 .0348408 0.659 0.510 -.0454214 .0913436
_cons | 4.708772 .2217592 21.234 0.000 4.273522 5.144022
------------------------------------------------------------------------------

But it does increase their standard errors, particularly that of LGEXP, as you would expect,
reflecting the loss of efficiency.

15
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 78.89
Model | 42.4015936 2 21.2007968 Prob > F = 0.0000
Residual | 144.30605 537 .26872635 R-squared = 0.2271
-------------+------------------------------ Adj R-squared = 0.2242
Total | 186.707643 539 .34639637 Root MSE = .51839

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1092273 .0091576 11.93 0.000 .0912382 .1272164
WEIGHT85 | .0024192 .0006402 3.78 0.000 .0011616 .0036769
_cons | .9194011 .1609538 5.71 0.000 .6032248 1.235577
------------------------------------------------------------------------------

Here is a regression of the logarithm of hourly earnings on years of schooling and weight in
pounds. The weight coefficient implies than an extra pound leads to 0.24% increase in
earnings, so four extra pounds leads to a 1% increase. Can you really believe this?
1
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 78.89
Model | 42.4015936 2 21.2007968 Prob > F = 0.0000
Residual | 144.30605 537 .26872635 R-squared = 0.2271
-------------+------------------------------ Adj R-squared = 0.2242
Total | 186.707643 539 .34639637 Root MSE = .51839

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1092273 .0091576 11.93 0.000 .0912382 .1272164
WEIGHT85 | .0024192 .0006402 3.78 0.000 .0011616 .0036769
_cons | .9194011 .1609538 5.71 0.000 .6032248 1.235577
------------------------------------------------------------------------------

Perhaps not, but the t statistic is very highly significant. What is going on?

2
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 78.89
Model | 42.4015936 2 21.2007968 Prob > F = 0.0000
Residual | 144.30605 537 .26872635 R-squared = 0.2271
-------------+------------------------------ Adj R-squared = 0.2242
Total | 186.707643 539 .34639637 Root MSE = .51839

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1092273 .0091576 11.93 0.000 .0912382 .1272164
WEIGHT85 | .0024192 .0006402 3.78 0.000 .0011616 .0036769
_cons | .9194011 .1609538 5.71 0.000 .6032248 1.235577
------------------------------------------------------------------------------

Older people tend to have more work experience, which increases their earnings. They also
tend to weigh more. This could be an explanation.

3
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S EXP WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 3, 536) = 70.13
Model | 52.6290507 3 17.5430169 Prob > F = 0.0000
Residual | 134.078593 536 .250146628 R-squared = 0.2819
-------------+------------------------------ Adj R-squared = 0.2779
Total | 186.707643 539 .34639637 Root MSE = .50015

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1222516 .0090671 13.48 0.000 .1044402 .140063
EXP | .0324871 .0050807 6.39 0.000 .0225066 .0424676
WEIGHT85 | .0016163 .0006303 2.56 0.011 .0003781 .0028545
_cons | .318147 .1815401 1.75 0.080 -.0384704 .6747644
------------------------------------------------------------------------------

Here we have controlled for work experience. The weight coefficient is lower, but still
almost significant at the 1% level. Can you think of any other variable that might be
correlated with both earnings and weight?
4
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S EXP MALE WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 4, 535) = 64.31
Model | 60.6259571 4 15.1564893 Prob > F = 0.0000
Residual | 126.081686 535 .235666703 R-squared = 0.3247
-------------+------------------------------ Adj R-squared = 0.3197
Total | 186.707643 539 .34639637 Root MSE = .48546

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1197587 .0088112 13.59 0.000 .10245 .1370674
EXP | .0282462 .0049849 5.67 0.000 .0184538 .0380386
MALE | .2953164 .0506962 5.83 0.000 .1957283 .3949045
WEIGHT85 | -.0006213 .0007224 -0.86 0.390 -.0020404 .0007978
_cons | .6269889 .1840109 3.41 0.001 .2655164 .9884614
------------------------------------------------------------------------------

The MALE dummy is such a variable. When it is included, the weight effect disappears.

5
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S EXP MALE WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 4, 535) = 64.31
Model | 60.6259571 4 15.1564893 Prob > F = 0.0000
Residual | 126.081686 535 .235666703 R-squared = 0.3247
-------------+------------------------------ Adj R-squared = 0.3197
Total | 186.707643 539 .34639637 Root MSE = .48546

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1197587 .0088112 13.59 0.000 .10245 .1370674
EXP | .0282462 .0049849 5.67 0.000 .0184538 .0380386
MALE | .2953164 .0506962 5.83 0.000 .1957283 .3949045
WEIGHT85 | -.0006213 .0007224 -0.86 0.390 -.0020404 .0007978
_cons | .6269889 .1840109 3.41 0.001 .2655164 .9884614
------------------------------------------------------------------------------

The point of this example is that model misspecification – variable misspecification or

indeed any kind of misspecification – in general will invalidate the regression diagnostics,
and as a consequence the diagnostics may lead you to the wrong conclusions.
6
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 78.89
Model | 42.4015936 2 21.2007968 Prob > F = 0.0000
Residual | 144.30605 537 .26872635 R-squared = 0.2271
-------------+------------------------------ Adj R-squared = 0.2242
Total | 186.707643 539 .34639637 Root MSE = .51839

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1092273 .0091576 11.93 0.000 .0912382 .1272164
WEIGHT85 | .0024192 .0006402 3.78 0.000 .0011616 .0036769
_cons | .9194011 .1609538 5.71 0.000 .6032248 1.235577
------------------------------------------------------------------------------

In the original model, we had two kinds of variable misspecification. We omitted EXP and
MALE, and we included the irrelevant variable WEIGHT85.

7
 VARIABLE MISSPECIFICATION III: CONSEQUENCES FOR DIAGNOSTICS

. reg LGEARN S WEIGHT85

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 78.89
Model | 42.4015936 2 21.2007968 Prob > F = 0.0000
Residual | 144.30605 537 .26872635 R-squared = 0.2271
-------------+------------------------------ Adj R-squared = 0.2242
Total | 186.707643 539 .34639637 Root MSE = .51839

------------------------------------------------------------------------------
LGEARN | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
S | .1092273 .0091576 11.93 0.000 .0912382 .1272164
WEIGHT85 | .0024192 .0006402 3.78 0.000 .0011616 .0036769
_cons | .9194011 .1609538 5.71 0.000 .6032248 1.235577
------------------------------------------------------------------------------

Including an irrelevant variable is one of the few types of misspecification that does not
lead to the invalidation of the regression diagnostics. However, omitting relevant variables
certainly does. This is why the t statistic in the original specification misled us.
8
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u

Suppose that a variable Y is hypothesized to depend on a set of explanatory variables X2, ...,
Xk as shown above, and suppose that for some reason there are no data on X2.

1
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u

As we have seen, a regression of Y on X3, ..., Xk would yield biased estimates of the
coefficients and invalid standard errors and tests.

2
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z

Sometimes, however, these problems can be reduced or eliminated by using a proxy

variable in the place of X2. A proxy variable is one that is hypothesized to be linearly related
to the missing variable. In the present example, Z could act as a proxy for X2.
3
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z

The validity of the proxy relationship must be justified on the basis of theory, common
sense, or experience. It cannot be checked directly because there are no data on X2.

4
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

If a suitable proxy has been identified, the regression model can be rewritten as shown.

5
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

We thus obtain a model with all variables observable. If the proxy relationship is an exact
one, and we fit this relationship, most of the regression results will be rescued.

6
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same

The estimates of the coefficients of X3, ..., Xk will be the same as those that would have been
obtained if it had been possible to regress Y on X2, ..., Xk.

7
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same

2. S.e. and t for b3, ..., bk same

The standard errors and t statistics of the coefficients of X3, ..., Xk will be the same as those
that would have been obtained if it had been possible to regress Y on X2, ..., Xk.

8
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same

2. S.e. and t for b3, ..., bk same
3. R2 same

R2 will be the same as it would have been if it had been possible to regress Y on X2, ..., Xk.

9
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same

2. S.e. and t for b3, ..., bk same
3. R2 same
4. Not possible to obtain an estimate of 2, unless  known

The coefficient of Z will be an estimate of 2, and so it will not be possible to obtain an
estimate of 2, unless you are able to guess the value of .

10
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same

2. S.e. and t for b3, ..., bk same
3. R2 same
4. Not possible to obtain an estimate of 2, unless  known
5. t statistic for Z same as that for X2

However the t statistic for Z will be the same as that which would have been obtained for X2
if it had been possible to regress Y on X2, ..., Xk, and so you are able to assess the
significance of X2, even if you are not able to estimate its coefficient.
11
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z
Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same

2. S.e. and t for b3, ..., bk same
3. R2 same
4. Not possible to obtain an estimate of 2, unless  known
5. t statistic for Z same as that for X2
6. Not possible to obtain an estimate of 1

It will not be possible to obtain an estimate of 1 since the intercept in the revised model is
(1+2), but usually 1 is of relatively little interest, anyway.

12
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z (approximation)

Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same (approximations)

2. S.e. and t for b3, ..., bk same
3. R2 same
4. Not possible to obtain an estimate of 2, unless  known
5. t statistic for Z same as that for X2
6. Not possible to obtain an estimate of 1

It is generally more realistic to hypothesize that the relationship between X2 and Z is

approximate, rather than exact. In that case the results listed above will hold approximately.

13
 PROXY VARIABLES

Y =  1 +  2 X 2 +  3 X 3 + ... +  k X k + u
X 2 =  + Z (approximation)

Y =  1 +  2 ( + Z ) +  3 X 3 + ... +  k X k + u
= ( 1 +  2  ) +  2 Z +  3 X 3 + ... +  k X k + u

Comparison of regression with Z instead of X2

1. b3, ..., bk same (approximations)

2. S.e. and t for b3, ..., bk same
3. R2 same
4. Not possible to obtain an estimate of 2, unless  known
5. t statistic for Z same as that for X2
6. Not possible to obtain an estimate of 1

However, if Z is a poor proxy for X2, the results will effectively be subject to measurement
error (see Chapter 8). Further, it is possible that some of the other X variables will try to act
as proxies for X2, and there will still be a problem of omitted variable bias.
14
 PROXY VARIABLES

S =  1 +  2 ASVABC +  3 INDEX + u

The use of a proxy variable will be illustrated with an educational attainment model. We will
suppose that educational attainment depends jointly on cognitive ability and family
background.
15
 PROXY VARIABLES

S =  1 +  2 ASVABC +  3 INDEX + u

As usual, ASVABC will be used as the measure of cognitive ability. However, there is no
‘family background’ variable in the data set. Indeed, it is difficult to conceive how such a
variable might be defined.
16
 PROXY VARIABLES

S =  1 +  2 ASVABC +  3 INDEX + u
INDEX =  + 1 SM +  2 SF

Instead, we will try to find a proxy. One obvious variable is the mother's educational
attainment, SM. However, father's educational attainment, SF, may also be relevant. So we
will hypothesize that the family background index depends on both.
17
 PROXY VARIABLES

S =  1 +  2 ASVABC +  3 INDEX + u
INDEX =  + 1 SM +  2 SF
S =  1 +  2 ASVABC +  3 ( + 1 SM +  2 SF ) + u
= ( 1 +  3  ) +  2 ASVABC +  3 1 SM +  3  2 SF + u

Thus we obtain a relationship expressing S as a function of ASVABC, SM, and SF.

18
 PROXY VARIABLES

. reg S ASVABC SM SF

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 3, 536) = 104.30
Model | 1181.36981 3 393.789935 Prob > F = 0.0000
Residual | 2023.61353 536 3.77539837 R-squared = 0.3686
-------------+------------------------------ Adj R-squared = 0.3651
Total | 3204.98333 539 5.94616574 Root MSE = 1.943

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

Here is the corresponding regression using EAEF Data Set 21.

19
 PROXY VARIABLES

. reg S ASVABC

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 1, 538) = 274.19
Model | 1081.97059 1 1081.97059 Prob > F = 0.0000
Residual | 2123.01275 538 3.94612035 R-squared = 0.3376
-------------+------------------------------ Adj R-squared = 0.3364
Total | 3204.98333 539 5.94616574 Root MSE = 1.9865

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .148084 .0089431 16.56 0.000 .1305165 .1656516
_cons | 6.066225 .4672261 12.98 0.000 5.148413 6.984036
------------------------------------------------------------------------------

Here is the regression of S on ASVABC alone.

20
 PROXY VARIABLES

. reg S ASVABC SM SF

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

. reg S ASVABC

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .148084 .0089431 16.56 0.000 .1305165 .1656516
_cons | 6.066225 .4672261 12.98 0.000 5.148413 6.984036
------------------------------------------------------------------------------

A comparison of the regressions indicates that the coefficient of ASVABC is biased

upwards if we make no attempt to control for family background.

21
 PROXY VARIABLES

. reg S ASVABC SM SF

. cor ASVABC SM SF
------------------------------------------------------------------------------
S | Coef. Std. Err. (obs=570)
t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 | ASVABC.1063528
0.000 SM SF
.1450646
SM | .0492424 .0390901 --------+---------------------------
1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 ASVABC|
3.48 1.0000 .04688
0.001 .1684851
_cons | 5.370631 .4882155 11.00 SM| 0.4202 4.41158
0.000 1.0000 6.329681
SF| 0.4090 0.6241 1.0000
------------------------------------------------------------------------------

. reg S ASVABC

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .148084 .0089431 16.56 0.000 .1305165 .1656516
_cons | 6.066225 .4672261 12.98 0.000 5.148413 6.984036
------------------------------------------------------------------------------

This is what we should expect. Both SM and SF are likely to have positive effects on
educational attainment, and they are both positively correlated with ASVABC.

22
 PROXY VARIABLES

. reg S ASVABC SM SF LIBRARY SIBLINGS

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 5, 534) = 63.21
Model | 1191.57546 5 238.315093 Prob > F = 0.0000
Residual | 2013.40787 534 3.77042672 R-squared = 0.3718
-------------+------------------------------ Adj R-squared = 0.3659
Total | 3204.98333 539 5.94616574 Root MSE = 1.9418

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1245327 .0099875 12.47 0.000 .104913 .1441523
SM | .0388414 .039969 0.97 0.332 -.0396743 .1173571
SF | .1035001 .0311842 3.32 0.001 .0422413 .1647588
LIBRARY | -.0355224 .2134634 -0.17 0.868 -.4548534 .3838086
SIBLINGS | -.0665348 .0408795 -1.63 0.104 -.1468392 .0137696
_cons | 5.846517 .5681221 10.29 0.000 4.730489 6.962546
------------------------------------------------------------------------------

LIBRARY (a dummy variable equal to 1 if anyone in the family owned a library card when the
respondent was 14) and SIBLINGS (number of brothers and sisters of the respondent) are
two other variables in the data set which might act as proxies for family background.
23
 PROXY VARIABLES

. reg S ASVABC SM SF LIBRARY SIBLINGS

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 5, 534) = 63.21
Model | 1191.57546 5 238.315093 Prob > F = 0.0000
Residual | 2013.40787 534 3.77042672 R-squared = 0.3718
-------------+------------------------------ Adj R-squared = 0.3659
Total | 3204.98333 539 5.94616574 Root MSE = 1.9418

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1245327 .0099875 12.47 0.000 .104913 .1441523
SM | .0388414 .039969 0.97 0.332 -.0396743 .1173571
SF | .1035001 .0311842 3.32 0.001 .0422413 .1647588
LIBRARY | -.0355224 .2134634 -0.17 0.868 -.4548534 .3838086
SIBLINGS | -.0665348 .0408795 -1.63 0.104 -.1468392 .0137696
_cons | 5.846517 .5681221 10.29 0.000 4.730489 6.962546
------------------------------------------------------------------------------

The LIBRARY variable was one of three variables included in the National Longitudinal
Survey of Youth to help pick up the influence of family background on education.
Surprisingly, it has a negative coefficient, but it is not significant.
24
 PROXY VARIABLES

. reg S ASVABC SM SF LIBRARY SIBLINGS

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 5, 534) = 63.21
Model | 1191.57546 5 238.315093 Prob > F = 0.0000
Residual | 2013.40787 534 3.77042672 R-squared = 0.3718
-------------+------------------------------ Adj R-squared = 0.3659
Total | 3204.98333 539 5.94616574 Root MSE = 1.9418

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1245327 .0099875 12.47 0.000 .104913 .1441523
SM | .0388414 .039969 0.97 0.332 -.0396743 .1173571
SF | .1035001 .0311842 3.32 0.001 .0422413 .1647588
LIBRARY | -.0355224 .2134634 -0.17 0.868 -.4548534 .3838086
SIBLINGS | -.0665348 .0408795 -1.63 0.104 -.1468392 .0137696
_cons | 5.846517 .5681221 10.29 0.000 4.730489 6.962546
------------------------------------------------------------------------------

There is a tendency for parents who are ambitious for their children to limit their number
(have few kids), so SIBLINGS should be expected to have a negative coefficient. It does,
but it is also not significant.
25
 PROXY VARIABLES

. reg S ASVABC SM SF LIBRARY SIBLINGS

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 5, 534) = 63.21
Model | 1191.57546 5 238.315093 Prob > F = 0.0000
Residual | 2013.40787 534 3.77042672 R-squared = 0.3718
-------------+------------------------------ Adj R-squared = 0.3659
Total | 3204.98333 539 5.94616574 Root MSE = 1.9418

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1245327 .0099875 12.47 0.000 .104913 .1441523
SM | .0388414 .039969 0.97 0.332 -.0396743 .1173571
SF | .1035001 .0311842 3.32 0.001 .0422413 .1647588
LIBRARY | -.0355224 .2134634 -0.17 0.868 -.4548534 .3838086
SIBLINGS | -.0665348 .0408795 -1.63 0.104 -.1468392 .0137696
_cons | 5.846517 .5681221 10.29 0.000 4.730489 6.962546
------------------------------------------------------------------------------

There are further background variables which may be relevant for educational attainment:
faith, ethnicity, and region of residence. These variables are supplied in the data set, but it
will be left to you to experiment with them.
26
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

In the last sequence it was argued that educational attainment might be related to cognitive
ability and family background, with mother's and father's educational attainment proxying
for the latter.
1
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 3, 536) = 104.30
Model | 1181.36981 3 393.789935 Prob > F = 0.0000
Residual | 2023.61353 536 3.77539837 R-squared = 0.3686
-------------+------------------------------ Adj R-squared = 0.3651
Total | 3204.98333 539 5.94616574 Root MSE = 1.943

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

However, when we run the regression using Data Set 21, we find the coefficient of mother's
education is not significant.

2
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF
. cor SM SF
Source | SS df MS Number of obs =
(obs=540) 540
-------------+------------------------------ F(| 3, 536)
SM = 104.30
SF
Model | 1181.36981 3 393.789935 Prob > F = 0.0000
--------+------------------
Residual | 2023.61353 536 3.77539837 R-squared
SM| 1.0000 = 0.3686
-------------+------------------------------ Adj R-squared
SF| 0.6241 =1.0000
0.3651
Total | 3204.98333 539 5.94616574 Root MSE = 1.943

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

As was noted in one of the sequences for Chapter 3, this might be due to multicollinearity,
because mother's education and father's education are correlated.

3
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

In the discussion of multicollinearity, several measures for alleviating the problem were
suggested, among them the use of an appropriate theoretical restriction.

4
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

4 = 3

In particular, in the case of the present model, it was suggested that the impact of parental
education might be the same for both parents, that is, that 3 and 4 might be equal.

5
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

4 = 3

S =  1 +  2 ASVABC +  3 ( SM + SF ) + u
=  1 +  2 ASVABC +  3 SP + u
SP = SM + SF

If this is the case, the model may be rewritten as shown. We now have a total parental
education variable, SP, instead of separate variables for mother’s and father’s education,
and the multicollinearity caused by the correlation between the latter has been eliminated.
6
 F TEST OF A LINEAR RESTRICTION

. g SP=SM+SF

. reg S ASVABC SP

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 156.04
Model | 1177.98338 2 588.991689 Prob > F = 0.0000
Residual | 2026.99996 537 3.77467403 R-squared = 0.3675
-------------+------------------------------ Adj R-squared = 0.3652
Total | 3204.98333 539 5.94616574 Root MSE = 1.9429

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469
SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014
_cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608
------------------------------------------------------------------------------

Here is the regression with SP replacing SM and SF.

7
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

. reg S ASVABC SP

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469
SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014
_cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608
------------------------------------------------------------------------------

A comparison of the regressions reveals that the standard error of the coefficient of SP is
much smaller than those of SM and SF, and consequently its t statistic is higher. Its
coefficient is a compromise between those of SM and SF, as might be expected.
8
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

. reg S ASVABC SP

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469
SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014
_cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608
------------------------------------------------------------------------------

However, the use of a restriction will lead to a gain in efficiency only if the restriction is
valid. If it is not valid, its use will lead to biased coefficients and invalid standard errors and
tests.
9
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1257087 .0098533 12.76 0.000 .1063528 .1450646
SM | .0492424 .0390901 1.26 0.208 -.027546 .1260309
SF | .1076825 .0309522 3.48 0.001 .04688 .1684851
_cons | 5.370631 .4882155 11.00 0.000 4.41158 6.329681
------------------------------------------------------------------------------

. reg S ASVABC SP

------------------------------------------------------------------------------
S | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
ASVABC | .1253106 .0098434 12.73 0.000 .1059743 .1446469
SP | .0828368 .0164247 5.04 0.000 .0505722 .1151014
_cons | 5.29617 .4817972 10.99 0.000 4.349731 6.242608
------------------------------------------------------------------------------

Do the coefficients of SM and SF in the unrestricted regression look as if they satisfy the
restriction? Not really, in this case. The coefficient of SM is much smaller than that of SF,
but then it should be noted that the standard errors are quite large.
10
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 3, 536) = 104.30
Model | 1181.36981 3 393.789935 Prob > F = 0.0000
Residual | 2023.61353 536 3.77539837 R-squared = 0.3686
-------------+------------------------------ Adj R-squared = 0.3651
Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------

. reg S ASVABC SP

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 156.04
Model | 1177.98338 2 588.991689 Prob > F = 0.0000
Residual | 2026.99996 537 3.77467403 R-squared = 0.3675
-------------+------------------------------ Adj R-squared = 0.3652
Total | 3204.98333 539 5.94616574 Root MSE = 1.9429
------------------------------------------------------------------------------

We will now perform a proper test. The imposition of a restriction makes it more difficult for
the regression model to fit the data because there is one fewer parameter to adjust. There
will therefore be an increase in RSS (and a decrease in R2) when it is imposed.
11
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 3, 536) = 104.30
Model | 1181.36981 3 393.789935 Prob > F = 0.0000
Residual | 2023.61353 536 3.77539837 R-squared = 0.3686
-------------+------------------------------ Adj R-squared = 0.3651
Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------

. reg S ASVABC SP

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 156.04
Model | 1177.98338 2 588.991689 Prob > F = 0.0000
Residual | 2026.99996 537 3.77467403 R-squared = 0.3675
-------------+------------------------------ Adj R-squared = 0.3652
Total | 3204.98333 539 5.94616574 Root MSE = 1.9429
------------------------------------------------------------------------------

If the restriction is valid, the deterioration in the fit should be a small, random amount.
However, if the restriction is invalid, the distortion caused by its imposition will lead to a
significant deterioration in the fit.
12
 F TEST OF A LINEAR RESTRICTION

. reg S ASVABC SM SF

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 3, 536) = 104.30
Model | 1181.36981 3 393.789935 Prob > F = 0.0000
Residual | 2023.61353 536 3.77539837 R-squared = 0.3686
-------------+------------------------------ Adj R-squared = 0.3651
Total | 3204.98333 539 5.94616574 Root MSE = 1.943
------------------------------------------------------------------------------

. reg S ASVABC SP

Source | SS df MS Number of obs = 540

-------------+------------------------------ F( 2, 537) = 156.04
Model | 1177.98338 2 588.991689 Prob > F = 0.0000
Residual | 2026.99996 537 3.77467403 R-squared = 0.3675
-------------+------------------------------ Adj R-squared = 0.3652
Total | 3204.98333 539 5.94616574 Root MSE = 1.9429
------------------------------------------------------------------------------

In the present case, we can see that the increase in RSS is very small, and hence we are
unlikely to reject the restriction.

13
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

4 = 3

S =  1 +  2 ASVABC +  3 ( SM + SF ) + u
=  1 +  2 ASVABC +  3 SP + u
SP = SM + SF

H 0 :  4 =  3 , H1 :  4   3

The null hypothesis is that the restriction is valid, and the alternative one is that it is invalid.

14
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

4 = 3

S =  1 +  2 ASVABC +  3 ( SM + SF ) + u
=  1 +  2 ASVABC +  3 SP + u
SP = SM + SF

H 0 :  4 =  3 , H1 :  4   3

( RSS R − RSSU ) / 1 2027.00 − 2023.61

F (1, n − k ) = = = 0.90
RSSU /( n − k ) 2023.61 / 536

The test statistic is a member of the family of F tests where the numerator is the
improvement in the fit on relaxing the restriction, divided by the cost of relaxing it (one
degree of freedom, because one additional parameter has to be estimated).
15
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

4 = 3

S =  1 +  2 ASVABC +  3 ( SM + SF ) + u
=  1 +  2 ASVABC +  3 SP + u
SP = SM + SF

H 0 :  4 =  3 , H1 :  4   3

( RSS R − RSSU ) / 1 2027.00 − 2023.61

F (1, n − k ) = = = 0.90
RSSU /( n − k ) 2023.61 / 536

The denominator of the test statistic is RSS after making the improvement (that is, RSS for
the unrestricted model), divided by n – k, the number of degrees of freedom remaining. k is
the number of parameters in the unrestricted model.
16
 F TEST OF A LINEAR RESTRICTION

S =  1 +  2 ASVABC +  3 SM +  4 SF + u

4 = 3

S =  1 +  2 ASVABC +  3 ( SM + SF ) + u
=  1 +  2 ASVABC +  3 SP + u
SP = SM + SF

H 0 :  4 =  3 , H1 :  4   3

( RSS R − RSSU ) / 1 2027.00 − 2023.61

F (1, n − k ) = = = 0.90
RSSU /( n − k ) 2023.61 / 536

The F statistic is 0.90. An F statistic below 1 is never significant (look at the F table), so we
do not reject H0. The restriction appears to be valid. At least, it is not rejected by the data.

17
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

The F test approach to testing a restriction may be extended to cover the case where we
wish to test whether several restrictions are valid simultaneously.

1
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

Suppose that there are p restrictions. Let RSSU be RSS for the fully unrestricted model and
RSSR be RSS for the model where all p restrictions have been imposed. The test statistic is
then as shown.
2
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

The numerator is the reduction in RSS comparing the fully restricted model with the
unrestricted model, divided by the number of degrees of freedom lost when the restrictions
are relaxed.
3
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

The denominator is the RSS for the unrestricted model, divided by the number of degrees of
freedom remaining when that model is fitted. k is the number of parameters in the
unrestricted model.
4
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

The t test approach can be used, as before, to test individual restrictions in isolation.

5
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

Y = 1 +  2 X 2 +  3 X 3 +  4 X 4 +  5 X 5 + u

5 = 0

You will often encounter references to zero restrictions. This just means that a particular
parameter is hypothesized to be equal to zero, for example, 5 in the model above. Taken in
isolation, the appropriate test is of course the t test.
6
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

Y = 1 +  2 X 2 +  3 X 3 +  4 X 4 +  5 X 5 + u

5 = 0

It can be considered to be a special case of the t test of a restriction discussed above where
there is no need for reparameterization. The test statistic is just the t statistic for the
parameter in question.
7
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

Y = 1 +  2 X 2 +  3 X 3 +  4 X 4 + u RSSU
Y = 1 +  2 X 2 + u RSS R

H0 :  3 = 4 = 0
H1 :  3  0 or 4  0 or both 3 and 4  0

( RSS R − RSSU ) / 2
F ( 2, n − k ) =
RSSU / (n − k )

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

Likewise the testing of multiple zero restrictions can be thought of as a special case of the
testing of multiple restrictions. The example shown is for a model where there are two zero
restrictions.
8
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

Y = 1 +  2 X 2 +  3 X 3 +  4 X 4 + u RSSU
Y = 1 +  2 X 2 + u RSS R

H0 :  3 = 4 = 0
H1 :  3  0 or 4  0 or both 3 and 4  0

( RSS R − RSSU ) / 2
F ( 2, n − k ) =
RSSU / (n − k )

( RSS R − RSSU ) / p
F ( p, n − k ) =
RSSU / (n − k )

The F test of the joint explanatory power of a group of explanatory variables discussed in
Section 3.5 in the text can be thought of in this way.

9
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

Even the F statistic for the equation as a whole can be treated as a special case. Here the
unrestricted and restricted models are as shown.

10
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

Fitting restricted model: b1 = Y

When we fit the restricted model, we find that the OLS estimator of 1 is the sample mean of
Y (see Exercise 1.3).

11
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

Fitting restricted model: b1 = Y

Yˆi = b1 = Y for all i

Hence the fitted value of Y in all observations is equal to the sample mean of Y.

12
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

Fitting restricted model: b1 = Y

Yˆi = b1 = Y for all i
TSS = ESS + RSS

Now we know that for any OLS regression, TSS = ESS + RSS.

13
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

Fitting restricted model: b1 = Y

Yˆi = b1 = Y for all i
TSS = ESS + RSS
(
 i
Y − Y ) 2
=  i(Yˆ − Y )2
+ i
e 2
= 0 + i
e 2

Hence TSS = RSS for the restricted regression.

14
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

Fitting restricted model: b1 = Y

Yˆi = b1 = Y for all i
TSS = ESS + RSS
(
 i
Y − Y ) 2
=  i(Yˆ − Y )2
+ i
e 2
= 0 + i
e 2

Obviously, if there are no explanatory variables, none of the variation in Y is explained by

the model and so RSS is equal to TSS.

15
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

( RSS R − RSSU ) / (k − 1)
F (k − 1, n − k ) =
RSSU / (n − k )
(TSS − RSSU ) / (k − 1) ESSU / (k − 1)
= =
RSSU / (n − k ) RSSU / (n − k )

Here is the F statistic for the comparison of the unrestricted model with all of the X
variables and the restricted model with only the intercept.

16
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

( RSS R − RSSU ) / (k − 1)
F (k − 1, n − k ) =
RSSU / (n − k )
(TSS − RSSU ) / (k − 1) ESSU / (k − 1)
= =
RSSU / (n − k ) RSSU / (n − k )

We have just seen that RSS from the restricted version is equal to TSS.

17
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

( RSS R − RSSU ) / (k − 1)
F (k − 1, n − k ) =
RSSU / (n − k )
TSS = ESSU + RSSU (TSS − RSSU ) / (k − 1) ESSU / (k − 1)
= =
RSSU / (n − k ) RSSU / (n − k )

Now we refer to the decomposition of TSS in the case of the unrestricted regression. This
is similar to the decomposition for the restricted model, with the difference that RSSU will in
be smaller than RSSR and ESSU will be positive, instead of zero.
18
 MULTIPLE RESTRICTIONS AND ZERO RESTRICTIONS

k
Unrestricted model: Y = 1 +   j X j + u RSSU
j =2
Restricted model: Y = 1 + u RSSR

Restrictions:  2 =  3 = ... =  k = 0
Hypotheses: H 0 :  2 =  3 = ... =  k = 0
H 1 : at least one of the slope coefficients  0

( RSS R − RSSU ) / (k − 1)
F (k − 1, n − k ) =
RSSU / (n − k )
TSS = ESSU + RSSU (TSS − RSSU ) / (k − 1) ESSU / (k − 1)
= =
RSSU / (n − k ) RSSU / (n − k )

Given the decomposition for the unrestricted version, we can rewrite the F statistic as
shown. This is the expression for the F statistic for the equation as a whole that was given
in Section 3.5.
19