RS - Econometrics I - Lecture 3

Lecture 3
Specification

OLS Estimation - Assumptions

• CLM Assumptions
(A1) DGP: y = X  +  is correctly specified.
(A2) E[|X] = 0
(A3) Var[|X] = σ2 IT
(A4) X has full column rank –rank(X)=k-, where T ≥ k.

Q: What happens when (A1) is not correctly specified?

• In this lecture, we look at (A1), always in the context of linearity. Are

we omitting a relevant regressor? Are we including an irrelevant
variable? What happens when we impose restrictions in the DGP?

1
RS - Econometrics I - Lecture 3

Specification Errors: Omitted Variables

• Omitting relevant variables: Suppose the correct model is
y = X11 + X22 +  –i.e., with two sets of variables.
But, we compute OLS omitting X2. That is,
y = X11 +  <= the “short regression.”

Some easily proved results:

(1) E[b1|X] = E [(X1X1)-1X1 y] = 1 + (X1X1)-1X1X22  1. So,
unless X1X2 =0, b1 is biased. The bias can be huge. It can reverse the
sign of a price coefficient in a “demand equation.”

(2) Var[b1|X] ≤ Var[b1.2|X]. (The latter is the northwest submatrix of

the full covariance matrix.) The proof uses M, the residual maker. We
get a smaller variance when we omit X2.

Specification Errors: Omitted Variables

• We get a smaller variance when we omit X2.

Interpretation: Omitting X2 amounts to using extra information –i.e.,

2 = 0. Even if the information is wrong, it reduces the variance.

(3) MSE
b1 may be more “precise.”
Precision = Mean squared error
= variance + squared bias.
Smaller variance but positive bias. If bias is small, may still favor the
short regression.

Note: Suppose X1X2 = 0. Then the bias goes away. Interpretation,

the information is not “right,” it is irrelevant. b1 is the same as b1.2.

2
RS - Econometrics I - Lecture 3

Omitted Variables Example: Gasoline Demand

• We have a linear model for the demand for gasoline:
G = PG 1 + Y 2 + ,

Q: What happens when you wrongly exclude Income (Y)?

E[b1|X] = 1 + Cov[ Price, Income] 2

Var[ Price]
In time series data, 1 < 0, 2 > 0 (usually)
Cov[Price, Income] > 0 in time series data.
 The short regression will overestimate the price coefficient.

In a simple regression of G (demand) on a constant and PG, the Price

Coefficient (1) should be negative.

Estimation of a ‘Demand’ Equation (Greene):

Shouldn’t the Price Coefficient be Negative?

3
RS - Econometrics I - Lecture 3

Estimation of a ‘Demand’ Equation (Greene):

Multiple Regression - Theory Works.
Ordinary least squares regression ............
LHS=G Mean = 226.09444
Standard deviation = 50.59182
Number of observs. = 36
Model size Parameters = 3
Degrees of freedom = 33
Residuals Sum of squares = 1472.79834
Standard error of e = 6.68059
Fit R-squared = .98356
Adjusted R-squared = .98256
Model test F[ 2, 33] (prob) = 987.1(.0000)
--------+------------------------------------------------------
Variable| Coefficient Standard Error t-ratio P[|T|>t]
--------+------------------------------------------------------
Constant| -79.7535*** 8.67255 -9.196 .0000
Y| .03692*** .00132 28.022 .0000
PG| -15.1224*** 1.88034 -8.042 .0000
--------+------------------------------------------------------

• Note: Income is helping us to identify a demand equation –i.e., with a

negative slope for the price variable.

Specification Errors: Irrelevant Variables

• Irrelevant variables . Suppose the correct model is
y = X11 +  –i.e., with one set of variables.
But, we estimate
y = X11 + X22 +  <= the “long regression.”

Some easily proved results: Including irrelevant variables just reverse

the results: It increases variance -the cost of not using information-;
but does not create biases.

 Since the variables in X2 are truly irrelevant, then 2 = 0,

so E[b1.2|X] = 1.

4
RS - Econometrics I - Lecture 3

Specification Errors: Irrelevant Variables

• A simple example
Suppose the correct model is: y = 1 + 2 X2 + 
But, we estimate: y = 1 + 2 X 2 + 3 X 3 + 

• Unbiased: given that 3 =0  E[b2|X]= 2

• Efficiency:
2 1 2
 b22   
 X 2i  X2 2 1  rX2 2 , X 3
 X 2i  X2 2
Note: These are the results in general. Note that if X2 and X3 are
uncorrelated, there will be no loss of efficiency after all.

Other Models
• Looking ahead to nonlinear models: neither of the preceding results
extend beyond the linear regression model.

“Omitting relevant variables from a model is always costly. (No

exceptions.) The benign result above almost never carries over to
more involved nonlinear models.” (Greene)

5
RS - Econometrics I - Lecture 3

Specification and Functional Form: Non-linearity

• In the context of OLS estimation, we can introduce some non-
linearities: quadratic, cubic and interaction effects can be easily
estimated by OLS. For example:
y = 1 + 2 X2 + 3 X22 + 4 X2 X3 + 

• Partial effects , 𝜕y/𝜕X2, (and standard errors) can be different. In

the above model
𝜕y/𝜕X2 = 2 + 2 3 X2 + 4 X3 2

Note: Recall that in a simple linear model:

y = 1 + 2 X 2 + 3 X 3 + 
the partial effect is equal to the i coefficient:
𝜕y/𝜕X2 = 2.

Specification and Functional Form: Non-linearity

• The estimator of partial effects and their variances are different
from bi and Var[bi|X] in the presence of non-linearities

Example: Quadratic Effect

P o p u la tio n E s tim a to r s
y  1   2 x   3 x   4 z  
2
yˆ  b1  b 2 x  b 3 x 2  b 4 z
E[ y | x, z]
x    2  2 3 x ˆ x  b 2  2 b 3 x
x
E s tim a to r o f th e v a r ia n c e o f ˆ x
E s t .V a r [ ˆ x ]  V a r [ b 2 ]  4 x 2 V a r [ b 3 ]  4 x C o v [ b 2 , b 3 ]

Note: Now, the partial effect and the variance are a function of the
data! Usually, an average is used in the estimation.

6
RS - Econometrics I - Lecture 3

Application (Greene): Log Income Equation

----------------------------------------------------------------------
Ordinary least squares regression ............
LHS=LOGY Mean = -1.15746 Estimated Cov[b1,b2]
Standard deviation = .49149
Number of observs. = 27322
Model size Parameters = 7
Degrees of freedom = 27315
Residuals Sum of squares = 5462.03686
Standard error of e = .44717
Fit R-squared = .17237
--------+-------------------------------------------------------------
Variable| Coefficient Standard Error b/[Link]. P[|Z|>z] Mean of X
--------+-------------------------------------------------------------
AGE| .06225*** .00213 29.189 .0000 43.5272
AGESQ| -.00074*** .242482D-04 -30.576 .0000 2022.99
Constant| -3.19130*** .04567 -69.884 .0000
MARRIED| .32153*** .00703 45.767 .0000 .75869
HHKIDS| -.11134*** .00655 -17.002 .0000 .40272
FEMALE| -.00491 .00552 -.889 .3739 .47881
EDUC| .05542*** .00120 46.050 .0000 11.3202
--------+-------------------------------------------------------------
At Average Age = x = 43.5272.
Estimated Partial effect = .066225 – 2(.00074) x 43.5272 = .00018.
Estimated Variance 4.54799e-6 + 4(43.5272)2*(5.87973e-10) + 4(43.5272)*
(-5.1285e-8) = 7.4755086e-08.
Estimated standard error = .00027341.

Specification and Functional Form: Non-linearity

Example: Interactive Effect

P o p u la t i o n E s t im a t o r s
y  1   2 x   3 z   4 xz   yˆ  b1  b 2 x  b 3 z  b 4 x z
E [ y | x, z]
x   2  4z ˆ x  b 2  b 4 z
x
E s t im a t o r o f t h e v a r ia n c e o f ˆ x
E s t .V a r [ ˆ x ]  V a r [ b 2 ]  z 2 V a r [ b 4 ]  2 z C o v [ b 2 , b 4 ]

7
RS - Econometrics I - Lecture 3

Application (Greene): Interaction Effect

----------------------------------------------------------------------
Ordinary least squares regression ............
LHS=LOGY Mean = -1.15746
Standard deviation = .49149
Number of observs. = 27322
Model size Parameters = 4
Degrees of freedom = 27318
Residuals Sum of squares = 6540.45988
Standard error of e = .48931
Fit R-squared = .00896
Adjusted R-squared = .00885
Model test F[ 3, 27318] (prob) = 82.4(.0000)
--------+-------------------------------------------------------------
Variable| Coefficient Standard Error b/[Link]. P[|Z|>z] Mean of X
--------+-------------------------------------------------------------
Constant| -1.22592*** .01605 -76.376 .0000
AGE| .00227*** .00036 6.240 .0000 43.5272
FEMALE| .21239*** .02363 8.987 .0000 .47881
AGE_FEM| -.00620*** .00052 -11.819 .0000 21.2960
--------+-------------------------------------------------------------
Do women earn more than men (in this sample?) The +.21239 coefficient on
FEMALE would suggest so.
But, the female “difference” –i.e., partial effect- is: +.21239 - .00620*Age.

At average Age, the effect is: .21239 - .00620 * (43.5272) = -.05748.

OLS Subject to Restrictions

• Restrictions: Theory imposes certain restrictions on parameters.
Examples:
(1) Dropping variables from the equation. That is, certain
coefficients in b forced to equal 0. (Is variable x3=size significant?”)
(2) Adding up conditions: Sums of certain coefficients must equal
fixed values. Adding up conditions in demand systems. Constant
returns to scale in production functions (α + β = 1 in a Cobb-Douglas
production function).
(3) Equality restrictions: Certain coefficients must equal other
coefficients. Using real vs. nominal variables in equations.

• Common formulation:
Minb {S(xi, θ) =Σi ei2 = e′e = (y – Xb)′ (y – Xb)} s.t. Rb = q

8
RS - Econometrics I - Lecture 3

Restricted Least Squares

• In practice, restrictions can usually be imposed by solving them out.

(1) Dropping variables –i.e., force a coefficient to equal zero.

  yi  b1 xi1  b2 xi 2  b3 xi 3 2
n
Problem: min b s.t. b3  0
i 1

  yi  b1 xi1  b2 xi 2 2
n
min b
i 1
. Do least squares subject to b +b +b =1. Then, b = 1-
(2) Adding up. 1 2 3 3
b1-b2. Make the substitution so (y – x3) = b1(x1 – x3) + b2(x2 – x3) + e.
i 1 ( yi  xi3 )  b1( xi1  xi3 )  b2 ( xi 2  xi3 )2
n
Problem: Minb

(3) Equality. If b3 = b2, then y = b1x1 + b2x2 + b2x3 + e

= b1x1 + b2(x2+x3) + e

i 1  yi  b1xi1  b2 ( xi 2  xi3 )2

n
Problem: Minb

Restricted Least Squares

• Theoretical results provide insights and the foundation of several
tests.

• Programming problem:
Minimize wrt b L* = (y – Xb)(y – Xb) s.t. Rb = q

• Quadratic programming problem

 Minimize a quadratic criterion s.t. a set of linear restrictions.
- Concave programming problem, all binding constraints. No
need for Kuhn-Tucker
- Solve using a Lagrangean formulation

• The Lagrangean approach (the 2 is for convenience).

Min b, L* = (y – Xb)(y – Xb) + 2 (Rb – q)

9
RS - Econometrics I - Lecture 3

Restricted Least Squares

• The Lagrangean approach
Min b, L* = (y – Xb)(y – Xb) + 2 (Rb – q)
f.o.c:
L*/b = -2X(y-Xb*) + 2R = 0  -X(y – Xb*) + R = 0
L*/ = 2(Rb* – q) = 0.  (Rb* – q) = 0

Then, from the 1st equation (and assuming full rank for X):
–Xy + XXb* + R = 0  b* = (XX)-1Xy – (XX)-1R
= b – (XX)-1R
Premultiply both sides by R and then subtract q
Rb* – q = Rb - R(XX)-1R - q
0 = –R(XX)-1R + (Rb –q)

Solving for    = [R(XX)-1R]-1 (Rb – q)

Substituting in b*  b* = b – (XX)-1R[R(XX)-1R]-1(R b – q)

Linear Restrictions
• Q: How do linear restrictions affect the properties of the least
squares estimator?
Model ( DGP): y = X + 
Theory (information): R – q = 0

Restricted LS estimator: b* = b – (XX)-1R[R(XX)-1R]-1(Rb – q)

1. Unbiased?
E[b*|X] =  - (XX)-1R[R(XX)-1R]-1 E[(Rb – q)|X] = 
2. Efficiency?
Var[b*|X] = 2(XX)-1 – 2 (XX)-1R[R(XX)-1R]-1 R(XX)-1
Var[b*|X] = Var[b|X] – a nonnegative definite matrix <
Var[b|X]
3. b* may be more “precise.”
Precision = Mean squared error = variance + squared bias.

10
RS - Econometrics I - Lecture 3

Linear Restrictions

1. b* = b – Cm, m = the “discrepancy vector” Rb – q.

Note: If m = 0  b* = b. (Q: What does m = 0 mean?)

2.  = [R(XX)-1R]-1(Rb – q) = [R(XX)-1R]-1m
When does  = 0? What does this mean?

3. Combining results: b* = b – (XX)-1R

4. Recall: ee = (y – Xb)(y – Xb) ≤ e*e* = (y – Xb*)(y – Xb*)

 Restrictions cannot increase R2  R2 ≥ R2*

Linear Restrictions - Interpretation

• Two cases
- Case 1: Theory is correct: R – q = 0 (restrictions hold).
b* is unbiased & Var[b*|X] ≤ Var[b|X]
- Case 2: Theory is incorrect: R - q  0 (restrictions do not hold).
b* is biased & Var[b*|X] ≤ Var[b|X].

• Interpretation
- The theory gives us information.
Bad information produces bias (away from “the truth.”)
Any information, good or bad, makes us more certain of our
answer. In this context, any information reduces variance.

11
RS - Econometrics I - Lecture 3

Linear Restrictions - Interpretation

- What about ignoring information (theory)?
Not using the correct information does not produce bias.
Not using information foregoes the variance reduction.

12