IV: Violations of Classical Assumptions of a Linear Regression Model

4.1 Test of equality of Coefficients two Regression Lines (Test of Stability)

What is known as the Rao-Chow test applied on time series data is referred to as test of stability
of parameters over time. To motivate the analysis for a test of stability of parameters, suppose we
have a time series data set on consumption of food, and measures of prices and income for a
country over time. Suppose also that there was an important event that has occurred in between,
such as a shift in policy or a war or any big event which would have a bearing on consumption.
The main question we ask is, would such an event affect the parameters that are estimated?
Suppose we formulate two multiple regression equations as follows.
Y 1 i = β 11 +β 12 X 2i +β 13 X 3i +⋯+ β1 k X ki+ε 1 i ; i=1 , 2 ,⋯, n1
Y 2 i = β 21+ β22 X 2 i +β 23 X 3i +⋯+β 2 k X ki +ε 2i ; i=1 , 2 ,⋯, n2
In both equations Y represents consumption and (X2, X3, ..., Xk) represent factors affecting
consumption. The first subscript refers to the period, thus 1 represents the period before the event
at hand and 2 is for periods after the event. Thus, for sample before the event we have
Y 1 = X 1 β 1 +ε 1
where Y 1 is ( n1 x 1 ) vector, X 1 is ( n1 xk ) matrix, ε 1 is ( n1 x1 ) vector, and β 1 is ( kx1 ) vector
Var ( ε 1 ) = σ 21 I n
1

and for sample 2 (i.e., period after the event) we have

Y 2 = X 2 β 2 +ε 2
where Y 2 is ( n2 x 1 ) vector, X 2 is ( n 2 xk ) matrix, ε 2 is ( n2 x 1 ) vector, and β 2 is ( kx 1 ) vector
Var ( ε 2 ) = σ 22 I n
2

We want to test whether these consumption (demand) functions are identical (i.e., have identical
parameters) or whether they have different parameters (i.e., they are different). We do this by
introducing the following null hypothesis, namely,
H0: β1 = β2 against the alternative H1: β1  β2

Note that this needs one basic assumption which has to be fulfilled in order to proceed. Namely,
2 2 2 2
we need to assume that two variances are equal. That is to test H 0: σ 1=σ 2 against H1: σ 1≠σ 2 ,

86
which can be tested by the F test as follows: The calculated F statistic is

F cal=
σ^ 21
σ^ 22
and under H 0 ,
σ^ 21
σ^ 22
∼ F (( n −k ) , ( n −k ))
1 2 where σ^ 12=
RSS1
n1 −k
∧ σ^ 22=
RSS 2
n2 −k
2 2
Assuming that our null hypothesis (H0: σ 1=σ 2 ) is accepted, we can then proceed to test the
hypotheses: H0: β1 = β2 against the alternative H1: β1  β2, i.e., whether the two regression
functions have the same parameters.
Procedure for testing this hypothesis:
^ ^
Step1: Estimate the two models separately by OLS and obtain β 1 and β 2 . Use this to obtain the

residual sum of squares for equations, i.e., RSS1 = ∑ e 21i and RSS 2=∑ e 22i
From these obtain the unrestricted sum of squares, which is given by
URSS=RSS1 + RSS2 , this will have (n + n - 2k) degrees of freedom.
1 2

^
Step2: Estimate the model under H0: β1 = β2 (= β) by OLS and obtain β . Thus we have,

Y 1 = X 1 β 1 +ε 1 ¿ } ¿ ¿ ⇒ [ Y 1 ¿] ¿ ¿ ¿
¿
Use the results, β^ , obtained here to derive the restricted residuals sum of squares (RRSS). These
residuals are restricted because we have forced the β’s in the two equations to be identical.

Therefore, we have RRSS = ∑ e20 i , this would be with (n 1 + n2 - k) degrees of freedom.

Step 3: The test for equality of the parameters of these two regression equations is given by

F cal =
( RRSS −URSS ) / k
URSS / [ ( n1−k ) + ( n2 −k ) ]
and
( RRSS −URSS ) / k
URSS / [ ( n1−k ) + ( n2 −k ) ]
∼ F ( ( n1 −k ) , ( n2 −k ) )

Example: The following are results obtained from a data set on US for two periods, before and
after second world war (the data set is for 1927-1962 and for 1942-1947 was omitted because it
was war period). The model was specified as
ln Q = β 1 + β2 ln P+ β3 ln Y +ε
Where lnQ is log of demand for food, lnP is log of price of food, lnY is log of income. Since the
period spans over a long period and includes the 2 nd WW, two models were specified for before

87
(1927-41) and after (1948-62) the war. Thus, we have
ln Q = β 11+ β12 ln P+β 13 ln Y +ε 1 for the first period, and

ln Q = β 21 + β 22 ln P+ β 23 ln Y +ε 2 for the second period.

The results of the exercise gave us the following result (t-ratios in parenthesis):

lnQ = 4.555 −0.235 lnP+0.243 lnY +e1 ¿ ¿¿→¿

(22.67 ) ( 4.41 ) (10.63 ) } for the first period, and

lnQ = −5.052−0.237 ln P+0.141 lnY +e2 ¿ ¿¿→¿

( 5.61) ( 1.54 ) (3.03) } for the second period.
2 2 2 2
We first test for the equality of variances, i.e., H0: σ 1=σ 2 against H1: σ 1≠σ 2 at  = 2%.
RSS1 0. 001151 RSS2 0 . 0005444
σ^ 21 = = & σ^ 22 = =
n1 −3 12 n2−3 12

σ^ 21 0 . 001151
F cal = = ≃ 2. 11 but F 0( 12. 01,12 ) = 4 .16
σ^ 22 0. 0005444
Thus
Now, since the calculated Fcal is less than the tabulated F, we accept the null at  = 2%, i.e., the
variances are equal.
Thus, the URSS = 0.001151 + 0.000544 = 0.001695
Pooling all observations one obtains
ln Q = 4 . 050−0. 12 ln P+0 .242 ln Y +e0
( 29. 65 ) ( 2. 95 ) (0 . 048 )
2
R =0 . 9731 , RSS=0 . 002866 , n=30 , df =26
The test we apply tries to distinguish which model to use.
( RRSS −URSS ) /k ( 0. 002866−0 . 001695 ) / 3 0. 002866−0 . 001695 24
F cal = = = x
URSS / [ ( n1 −k ) + ( n2 −k ) ] 0. 001695 / [ (15−3 ) + ( 15−3 ) ] 0. 001695 3
0 . 001171
= x 8 ≃ 5 .53
0 . 001695
0. 05
From the tables of F, F (3 , 24 ) = 3 .01 at 10% level of significance and is 4.72 at 2%. Thus, the
null is rejected because F calculated is greater than F tabulated. Thus, there is no case for
pooling. Though the second parameter (the price elasticity) is almost the same in the two models,

88
the third parameter (the income elasticity) is different.

89
4.2 Specification Errors
Assume that on the basis of some criteria we arrived at a model that we assumed to be correct,
and let this model be:

Y i = β1 + β 2 X 2i + β3 X 3 i + β 4 X 4 i +U 1 i EQ 1

Where Y is Output and X2, X3 and X4 are various inputs.

Now, suppose for some reason a researcher decides to use the following equation in estimating
the above model:

Y i = α 1 +α 2 X 2i + α 3 X 3 i +U 2i EQ 2

Since equation 1 is assumed to be the correct model, and then adopting equation 2 would
constitute a specification error termed as "Omitting a relevant variable (X4)". Hence, the error
term U2i in equation 2 is in fact:

U 2 i = U 1 i+ β4 X4 i EQ 3

Suppose again that another researcher uses the following model:

Y i = θ1 +θ 2 X 2i + θ3 X 3 i +θ 4 X 4 i +θ5 X 5i +U 3i EQ 4

If equation 1 is again assumed to be the true model, then equation 3 also constitutes a
specification error that we call "inclusion of irrelevant or unnecessary variable (X5)" in the
true model. Given that equation 1 is the true model then it follows that θ 5 should be zero.
However, if θ5 is different from zero as per the specification in equation 4, then in this case the
error term is in fact:

U 3 i = U 1i−θ 5 X 5 i EQ 5

Now, assume also that yet another researcher postulates the following specification:

ln Y i = β 1 + β 2 X 2 i + β 3 X 3 i + β 4 X 4 i +U 4i EQ 6

In relation to the true model given in equation 1 above, equation 6 would also constitutes a
specification bias where the bias here is termed as the use of "wrong functional form". In the
true model the dependent variable appears linearly, whereas in equation 6 it appears log-linearly.

89
Finally, consider again another researcher who uses the following model:
¿ ¿ ¿ ¿ ¿ ¿ ¿ ¿ ¿
Y i = β 1 +β 2 X 2 i +β 3 X 3 i +β 4 X 4 i +U i EQ 7
¿ ¿ ¿ ¿
Y i =Y i + ε i X 2i =X 2 i +ω 2i X 3 i= X 3i + ω3 i X 4 i =X 4i + ω 4i , and
Where ε i , and ω 2i , ω3i and ω 4i are the errors of measurement .
Equation 7 states that instead of the true values of the variables, we use their proxies, which may
contain errors of measurement. Hence the error that we commit in using equation 7 is termed as
the "errors of measurement bias".

In summary, having once specified a model as the correct model, one is likely to commit one or
more of the following specification errors:
1. Omission of a relevant variable
2. Inclusion of irrelevant variable
3. Adopting a wrong functional form
4. Errors of measurement.

Omission of Relevant Variables (Under fitting a Model)

Suppose the true model is:

Y i = β1 + β 2 X 2i + β3 X 3 i +ui EQ 8

but for some reason a researcher fits the following model, and thus commits the specification
error of excluding a relevant variable:

Y i = α 1 +α 2 X 2i + v i EQ 9

Consequences of omitting X3 are as follows:

1. If the left-out variable X3 is correlated with the included variable X 2, that is, r23 is

nonzero, both α^ 1 and α^ 2 are biased as well as inconsistent, and the bias does not
disappear no matter how large the sample may be. Fitting EQ9 yields the following
estimators of the two unknown parameters of the model

90
 α^ 2 =
∑ x 2i y i and α^ 1 = Ȳ −α^ 2 X̄ 2
∑ x22 i
If the true model is as given in EQ8, then
∑ x2 i ( β 2 x 2i + β 3 x 3 i +ui−ū )
α^ 2 = as y i = β 2 x 2i + β 3 x 3 i +ui −ū
∑ x22 i
∑ x 2i x 3 i +∑ x 2i u
= β 2+ β 3 i
as ∑ x2 i ¯u = 0
∑ x 22i ∑ x 22 i
∑ x 2i u i
= β 2 + β 3 β32 +
∑ x 22i
∴ E ( α^ 2) = β 2 + β 3 β32 +
∑ x2 i E ( ui )
∑ x22 i
= β 2 + β 3 β32 ; as E ( ui ) =0 ∀ i
Similarly,
α^ 1 = Ȳ −α^ 2 X̄ 2
= ( β 1 + β 2 X̄ 2 + β 3 X̄ 3 + ū ) −α^ 2 X̄ 2

(
= ( β 1 + β 2 X̄ 2 + β 3 X̄ 3 + ū ) − β 2 + β3 β32 +
∑ x 2i ui
∑ x 22i ) X̄ 2

X̄ 2 ∑ x 2 i u i
= β 1 + β 3 X̄ 3 −β 3 β 32 X̄ 2 + ū−
∑ x 22i
X̄ 2 ∑ x 2i E( ui )
∴ E ( α^ 1 ) = β1 + β 3 X̄ 3 −β 3 β 32 X̄ 2 + E ( ū )−
∑ x 22 i
= β 1 + β 3 X̄ 3−β 3 β 32 X̄ 2 as E (ū )=0 and E ( ui ) = 0 ∀ i
p lim α^ 2≠β 2 p lim α^ 1 ≠β1
From the above derivation, it can clearly be seen that n →∞ and n→∞ , and

hence both α^ 1 and α^ 2 are biased and inconsistent estimators of β1 and β2, respectively.

2. Even if X2 and X3 are uncorrelated (r23=0), α^ 1 is still biased, although α^ 2 is now

unbiased.

91
 3. The disturbance variance σ2 is incorrectly estimated. Disturbance variance of the true

σ^ 2u =
∑ u^ 2i σ^ 2v =
∑ v^ 2i
model is n−3 , this is to be estimated by n−2 and
β 23 [ ∑ x 23 i−β 32 ∑ x 2 i x 2 i ]
E ( σ^ 2v ) = σ 2 + ≠ σ2
n−2
4. The conventionally measured variance of α^ 2 is a biased estimator of the variance of the
^
true estimator β 2 .
σ2
Var ( β^ 2 ) =
^ (1−r 223 ) ∑ x 22i
Variance of the true estimator β 2 is and the conventionally
σ2
Var ( α^ 2 ) =
^
α
measured variance of 2 is ∑ x 22 i .
5. In consequence, the usual confidence interval and hypothesis-testing procedures are
likely to give misleading conclusions about the statistical significance of the estimated
parameters.

Inclusion of Irrelevant Variables (Over fitting a Model)

Now let us assume that

Y i = β1 + β 2 X 2i + ui EQ 10

is the true model, but a researcher fits the following model:

Y i = α 1 +α 2 X 2i + α 3 X 3 i +ν i EQ 11

and thus commits the specification error of including an irrelevant variable in the model. The
consequences of this specification error are as follows:
1. The OLS estimators of the parameters of the incorrect model are all unbiased and
E(α
^ 1 ) = β 1 , E ( α^ 2 ) = β 2 , and E ( α^ 3 ) = β 3 = 0
consistent, that is, This
can be shown as follows:
If the true model is as given in EQ 10 and the researcher estimates EQ 11, then

92
 ^2 =
α
∑ x23 i ∑ x 2i y i −∑ x 2i x 3i ∑ x 3i y i
(∑ )
2

∑ x22 i ∑ x 23i − x 2i x 3i

=
∑ x 23i ∑ x 2 i Y i−∑ x 2 i x 3 i ∑ x 3i Y i
(∑ )
2

∑ x 22i ∑ x 23 i− x 2i x 3 i

∑ x 23i ∑ x 2 i ( β 1 + β 2 X 2 i +ui ) −∑ x 2i x 3i ∑ x 3 i ( β 1 + β 2 X 2 i +ui )

=

(∑ )
2

∑ x 22i ∑ x 23 i− x 2i x 3 i

[ (∑ )]
2
β2 ∑ x 22i ∑ x 23 i− x 2i x 3 i + ∑ x 23i ∑ x 2i ui − ∑ x 2i x 3i ∑ x 3 i ui
=

(∑ )
2

∑ x 22i ∑ x 23 i− x 2i x 3 i

= β 2+
∑ x 23 i ∑ x2 i ui−∑ x2 i x 3i ∑ x 3i ui
(∑ )
2

∑ x 22 i ∑ x 23 i− x 2i x 3 i

∑ x 23i ∑ x 2 i E ( ui ) −∑ x 2i x 3i ∑ x 3 i E ( ui )
∴ E ( α^ 2 ) = β 2 +

(∑ )
2

∑ x 22i ∑ x 23 i− x2 i x 3 i

= β2 as E ( ui ) = 0 ∀i

^3 =
α
∑ x 22i ∑ x 3i y i −∑ x 2i x 3i ∑ x 2i y i
(∑ )
2

∑ x 22i ∑ x 23i − x 2 i x 3i

=
∑ x 22i ∑ x 3 i Y i−∑ x 2 i x 3 i ∑ x 2i Y i
(∑ )
2

∑ x 22i ∑ x 23 i− x 2i x 3 i

∑ x 22i ∑ x 3 i ( β 1 + β 2 X 2 i +ui ) −∑ x 2i x 3i ∑ x 2 i ( β 1 + β 2 X 2 i +ui )

=

(∑ )
2

∑ x 22i ∑ x 23 i− x 2i x 3 i

=
∑ x 22i ∑ x 3 i u i−∑ x 2 i x 3 i ∑ x 2i ui
(∑ )
2

∑ x 22i ∑ x 23 i− x 2i x 3 i

^ 3) =
∑ x 22i ∑ x 3 i E ( ui ) −∑ x 2i x 3i ∑ x 2i E ( u i )
∴ E (α

(∑ )
2

∑ x 22i ∑ x 23 i− x2 i x3 i

= 0 as E ( ui ) = 0 ∀i

93
α^ 1 = Ȳ −α^ 2 X̄ 2 −α^ 3 X̄ 3
= ( β 1 + β 2 X̄ 2 + ū ) −α^ 2 X̄ 2 −α^ 3 X̄ 3
∴ E ( α^ 1 ) = β1 + β 2 X̄ 2 + E ( ū )−E ( α^ 2 ) X̄ 2−E ( α^ 3 ) X̄ 3
= β 1 + β 2 X̄ 2−β 2 X̄ 2 as E ( ū ) =0 , E ( α^ 2 ) =β 2 and E ( α^ 3 )=β 3 =0
= β1

2. The error variance σ2 is correctly estimated, that is, E ( σ v ) = σ .

^2 2

3. The usual confidence interval and hypothesis-testing procedures remain valid.

4. However, the estimated α's will be generally inefficient in a sense that their variances will
generally larger than those estimators of the β's of the true model.
σ2 σ2
Var ( β^ 2 ) = Var ( ^ 2) =
α
Recall that , ∑ x 22i and ( 1−r 223) ∑ x22 i
Var ( α^ 2 ) 1
=
^ )
Var ( β 1−r 223 0≤r 223≤1 , it follows that Var ( α^ 2 ) ≥Var ( β^ 2 )
Therefore, 2 . Since , that is,
^
the variance of α^ 2 is generally greater than the variance of β 2 even though ( 2 )
^ =β 2
E α
.
The implications of this finding is that inclusion of the unnecessary variable X 3 makes the

variance of α^ 2 larger than necessary, thereby making α^ 2 less precise. This is also true ofα^ 1 .
In conclusion here would be it is better to include irrelevant variable than to omit the relevant
ones. But this philosophy is not to be especially used because addition of unnecessary
variables will lead to loss in efficiency of the estimators and may also lead to the problem of
multicollinearity, not to mention the loss of degrees of freedom. Therefore, in general, the
best approach is to include only explanatory variables that, on the theoretical grounds,
directly influence the dependent variable and that are not accounted for by other included
variables.

Tests for Omitted Variables and Incorrect Functional Form

In practice we are never sure that the model adopted for empirical testing is the true model.
Here the idea is to check whether the chosen model is adequate or not. In determining the
2
adequacy of the model, we look at some broad features of the results, such as the R̄ value,

94
 the estimated t-ratios, the signs of the estimated coefficients in relation to their prior
expectations, and the like. If these diagnostic tests are reasonably good, we proclaim that the
chosen model is a fair representation of reality. By the same token, if the results do not look
encouraging, then we may begin to worry about model adequacy and look for remedies: may
be we have omitted important variable or used wrong functional form, or have not first
differenced the time series (to remove serial correlation), and so on. To aid us in determining
whether model inadequacy is because of omission of relevant variable or wrong functional
form, we can use the following methods.

To illustrate, let us consider the total cost of production function, where the true total cost
function is assumed to be a cubic function as given below:

Y i = β1 +β 2 X i +β 3 X i2 +β 4 X 3i +u 1i
Where Y = total cost and X = output EQ 12

but a researcher fits the following quadratic function:

Y i = α 1 +α 2 X i +α 3 X i2 +u2 i EQ 13

and another researcher fits the following linear function:

Y i = λ1 + λ2 X i + u3i EQ 14

Ramsey's RESET Test: Ramsey has proposed a general test of specification error called
RESET (regression specification error test). To illustrate how the RESET test is used to detect
for a specification error, now, assume that a researcher fits a linear model for the total cost of
production where the true model is a cubic function.

Now, if we plot the residuals obtained from the linear regression function given in EQ 14 against
Y^ i , the estimated Y from this model, the residuals will show a pattern in which their mean
i

^ ^
changes systematically with Y i . This would suggest that if we introduce Y i in some form as a
regressor(s) in EQ 14, in fact it should increase R 2. If the increase in R2 is statistically significant,
it would suggest that the linear cost function in EQ 14 was misspecified. This is essentially the

95
ideas behind the RESET test. The steps involved in RESET are as follows:
^
1. From the chosen model, for example EQ 14, obtain the estimated Yi, that is, Y i .
^
2. Rerun EQ 14 by introducing Y i in some form as an additional regressor(s). Suppose is it
observed that the plot of the residuals against the estimated Y i's show a curve-linear
^2 ^3
relationship, one can introduce Y i and Y i as additional regressors. Thus, we may run the
following regression:

Y i = β1 + β 2 X i + β 3 Y^ 2i + β 4 Y^ 3i +ui EQ 15
2 2
3. Let the R2 obtained from EQ 15 be R new and that obtained from EQ 14 be Rold . Then one
can use the F test as follows:
( R2new − R2old ) / number of new regressors
F =
( 1- R2new ) / ( n− number of parameters in the new model )
to find out if the increase in the R2 from using EQ 15 is statistically significant.
4. If the computed Value is significant at some specified level of significance, one can
accept the hypothesis that the model in EQ 14 is misspecified.

To provide a numerical illustration, suppose we have ten observations on total cost and
output, and obtained the following estimated regression equations (standard errors in
parenthesis):

Y i = 166 . 467 +19 . 933 X i + u^ 3 i , R 2 = 0 . 8409

( 19 . 021) ( 3. 066 ) EQ 16

Y i = 2140. 7223 +476 .6557 X i−0 . 09187 Y^ 2i +0. 000119 Y^ 3i + u^ i

(132. 00044 ) ( 33 . 3951 ) ( 0. 0062 ) (0 . 0000074 )
2
R = 0 . 9983 EQ 17
^2 ^3
Note: Y i and Y i in EQ 17 are obtained from EQ 16.
Now applying the F-test introduced earlier, we obtain:

( 0 . 9983−0. 8409 ) /2
F cal = = 284 . 4035
( 1−0 . 9983 ) / ( 10−4 ) EQ 18

96
It can easily be verified that this F-value is highly significant, indicating that the model in EQ 16
is misspecified.

One advantage of RESET is that it is easy to apply, for the reason that it does not specify what
the alternative model is. But that is also its disadvantage because knowing that a model is
misspecified does not help us in choosing a better alternative model.

Lagrange Multiplier (LM) Test for Adding Variables

This is an alternative to Ramsey's RESET test. To illustrate this, we will continue with
preceding illustrative examples.
If we compare the linear cost function in EQ 14 with cubic cost function given in EQ 12, the
former is the restricted version of the latter. The restricted regression assumes that the
coefficients of the squared and cubed output terms are equal to zero. To test this, the LM test
proceeds as follows:
^i.
1. Estimate the restricted regression (EQ 14) by OLS and obtain the residuals, u
2. If in fact the unrestricted regression in EQ 12 is the true regression, the residuals obtained
2 3
from EQ 14 should be related to the squared and cubed output terms, that is, X i and X i .
^ i obtained in step 1 on all the regressors (including
3. This suggests that we regress the u
those in the restricted regression), which in the present case means:

u^ i = α 1 +α 2 X i +α 2 X i2 +α 3 X 3i +v i
Where v is an error term with the usual properties . EQ 19
4. For large sample size, it can be shown that n (the sample size) times the R 2 estimated
from the auxiliary regression in EQ 19 follows the chi-square distribution with degrees of
freedom equal to the number of restrictions imposed by the restricted regression.
Symbolically,

nR
2
~ 2
χ ( number of restrictions )
asy EQ 20
5. If the chi-square value obtained from EQ 20 exceeds the critical chi-square value at the
chosen level of significance, we reject the restricted regression.

97
For the example we started with, the regression results are as follows:

Y^ i = 166 . 467 +19 . 933 X i

( 19 . 021) ( 3. 066 ) EQ 21

When the residuals from EQ 21 are regressed as suggested in step 3, we obtain the following
results (standard errors in parenthesis):

u^ i = −24 . 7 +43 . 5443 X i −12. 9615 X 2i + 0 .9396 X 2i + ν^ i , R 2 = 0 . 9896

( 6. 375 ) ( 4 .779 ) ( 0 . 986 ) ( 0. 059 ) EQ 22

Although our sample size of 10 is by no means regarded as large, just to illustrate the LM test,
obtain nR2 = 10*0.9896 = 9.896. From the chi-square table we observe that for 2 degrees of
freedom the 1% critical chi-square value is about 9.21. Therefore, the observed value of 9.896 is
significant at 1% level of significance, and our conclusion would be to reject the restricted
regression, which is the same conclusion as that we reached based on the Ramsey RESET test.

98
4.3 Dummy Variable Regression
Usually data is classified according to categories. Often, such categories are qualitative in nature
and specify unique characteristics of the members of the observations. For instance, if we have
data on outputs and inputs of firms, the producing unit may be classified as large or small. If we
have data on wages of workers in a firm, we may characterize the recipients of these wages as
male and female, etc. We may then argue, hypothesize that large firms are more productive than
small ones, or female workers earn less than their male counterparts. Note that what we are
saying here is that, given the same attributes in the right-hand side of the equation, the left hand
side of the equation will give larger or smaller value depending on the categories of the firms or
workers. What we are proposing here is the following. Suppose the general model we specify is
the following
Y i = β1 + β 2 X i + ε i
with the following characteristics
Y i = β1 + β 2 X i + ε i : Small firms/female workers
Y i = β1 +γ + β 2 X i + ε i : Large firms/ male workers
Thus, the problem is, how do we take into account for the increment in output/wage equivalent to
γ, when the slope parameter is the same. This can be shown more clearly using the following
graph:

Large firms/
Y Male workers

Small firms/ Female workers

X1 X

98
99
This is one of the main areas where the notion of dummy variables is used. To illustrate
this,
Let Di = 0 for small firms /female workers
Di = 1 for large firms /male workers
Say we have n1 small firms/ female workers and n2 large firms/ male workers in our
sample. The total number of observations in our sample is, therefore, equal to n =n 1+n2.
Then our model reduces to:
Y i = β1 + β 2 X i + ε i ; i=1 , 2 ,⋯, n1
Y i = β1 +γ + β 2 X i + ε i ; i=n1 +1 , n1 +2 ,⋯,n
Y i = β1 +γ Di + β2 X i +ε i ; i=1 , 2 ,⋯, n
To estimate 1, 2 and  we regress Y on X and D. Suppose we ignore such a difference
and regress the equation without the dummy variable, then we would have a biased
estimators because we have omitted a relevant variable. To see this, observe the
following figure.

Y Regression ignoring categories

Large firms/ Male workers

* Small firms/ Female workers

*
*
**..

X
Notice the parameter estimates would be different if we use dummy compared to the case
where we don’t. Now, given that the model should include a dummy variable, we have:
Y i = β1 +γ Di + β2 X i +ε i
We minimize the sum of squared residuals with respect to the parameters in order to
n n
∑ ∑ (Y i − β^ 1− γ^ Di − β^ 2 X i )
2
min e 2i =
obtain the OLS estimators, i.e., i =1 i=1

100
This results in the following first order conditions:
n
∂ ∑ ei2 n
i=1
= 0 ⇒ 2 ∑ ( Y i− β^ 1 −^γ D i− β^ 2 X i ) (−1 ) = 0
∂ β^ 1 i=1
n n
⇒ ∑ Y i−n β^ 1 −n2 ^γ − β^ 2 ∑ X i = 0
i=1 i=1
n2
⇒ Y = β^ 1 + γ^ + β^ 2 X EQ1
n
n
∂ ∑ ei2 n
i=1
= 0 ⇒ 2 ∑ ( Y i− β^ 1 −^γ D i− β^ 2 X i ) (−X i ) = 0
∂ β^ 2 i=1
n n n n
= ⇒ ∑ X i Y i − β^ 1 ∑ X i −^γ ∑ Di X i− β^ 2 ∑ X 2i = 0
i=1 i=1 i=1 i=1
n n n n
= ⇒ ∑ X i Y i − β^ 1 ∑ X i −γ^ ∑ X i− β^ 2 ∑ X i2 = 0 EQ2
i=1 i=1 i=n 1 i=n1+1
n
∂ ∑ ei2 n
i=1
= 0 ⇒ 2 ∑ ( Y i− β^ 1 −^γ D i− β^ 2 X i ) (−D i ) = 0
∂ γ^ i=1
n n n n
= ⇒ ∑ Di Y i− β^ 1 ∑ Di −γ^ ∑ Di − β^ 2 ∑ D i X i = 0
2

i=1 i=1 i=1 i=1

n n
= ⇒ ∑ Y i − β^ 1 n2− γ^ n2 − β^ 2 ∑ Xi = 0
i=n1 +1 i=n1+ 1

= ⇒ γ^ = Ȳ 2 - β^ 1 - β^ 2 X̄ 2 EQ3

Substituting EQ3 into EQ1, we get

n2
Ȳ = β^ 1 +
n
( Ȳ 2− β^ 1− β^ 2 X̄ 2 )+ β^ 2 X̄
⇒ Ȳ −
n2
n
Ȳ 2 = 1−
n2
n ( ) ( n2
β^ 1 + X̄ − X̄ 2 β^ 2
n )
n1 n1 n1
⇒ β^ 1 = Ȳ 1− X̄ 1 β^ 2
n n n
^ ^
⇒ β 1 = Ȳ 1 − β2 X̄ 1 EQ4

Substituting EQ4 into EQ3 gives

γ^ = Ȳ 2 −( Ȳ 1− β^ 2 X̄ 1 )− β^ 2 X̄ 2
= ( Ȳ −Ȳ ) − β^ ( X̄ − X̄ )
2 1 2 2 1 EQ5

Substituting the results in EQ4 and EQ5 into EQ2 yields:

101
 n1 n

∑ ( Y 1i −Ȳ 1 )( X 1i − X̄ 1 ) + ∑ ( Y 2i −Ȳ 2 )( X 2i − X̄ 2 )
i=1 i=n1 +1
β^ 2 = n
1 n

∑ ( X 1 i− X̄ 1 ) + ∑ ( X 2 i− X̄ 2 )2
2

i=1 i=n1

Interpretation and Testing

The results obtained earlier shows that for group2, we have:
Y i = β1 + β 2 X i + ε i ; ∀ i i=1 , 2 ,⋯, n1
and for group2, we have:
Y i = ( β 1 +γ ) + β 2 X i +ε i ; ∀ i i=n1 +1 , 2 ,⋯, n

Combining both equations yield:

Y i = β1 +γ Di + β2 X i +ε i ; ∀ i i=1 , 2,⋯, n
where D i = 0 for group1, and
= 1 for gropu2
If we want to test that the intercept for both groups is identical, we introduce the
following hypothesis:
γ^
H 0 : γ =0 against H 1 : γ > 0 . We know that under H 0 SE ( ^γ )
^¿ ∼t ( n−3 )
¿
. Thus we use the t

test in order to test whether the intercepts are different or equal. Not that if we accept H 0 ,
it implies that the dummy variable is not necessary in our model, i.e., categorization of

the data is not necessary. On the other hand if H 0 is rejected, meaning that  is
significantly different from zero in the statistical sense, then our categorization is correct,
and there are two different regression equations and not one in our model.

We can also introduce a different dummy variable for group1 and model our regression
as follows
Y i = β1 + β 2 X i + ε i : Small firms/female workers
Y i = δ + β 2 X i +ε i : Large firms/ male workers

In this formulation δ = β 1 + γ
Combining the two equations give us:

102
Y i = β1 Z i +δ D i + β 2 X i + ε i ; ∀ i i=1 , 2 ,⋯, n
where Z i = 1 & D i = 0 for group1, and
Z i = 0 & D i = 1 for group2
Note that in this formulation we do not have a constant (intercept) term. Actually
introducing an intercept into the last model would lead to what is known as the dummy
variable trap, which is the result of the fact that given
Y i = α + β 1 Z i +δ Di + β 2 X i +ε i ; ∀ i i=1 ,2 ,⋯, n

This is so because Zi + D i 1 i. e . , col ( 1 ) = col (2 )+ col (3 ) . This is what is known as

perfect multicollinearity, we shall treat shortly. However, the issue of dummy variables
(categories in data) can be generalized to the case of more than two categories. Think
about it, and try to formulate a model with multiple (say three) categories.

4.4 1Multicollinearity
Multicollinearity is a phenomenon that may be observed in multiple linear regressions. It
results in:
1. high estimates of standard errors (small t-ratios) and a high R2.
2. a wide confidence intervals for parameter estimates.
There are basically two reasons for multicollinearity. Either
1. there is little variation in the explanatory variables, or
2. there is high correlation among the explanatory variables.
Now, for the general linear regression model, we know that
σ2
Var ( β^ j )=
∑ x 2j ( 1−R2j ) 2
, where R j is the square of multiple correlation coefficient
between Xj and the other explanatory variables. If X2, X3, ..., Xk are orthogonal (i.e.,

β^ 2 =
∑ x2 y
, β^ 3 =
∑ x3 y , .. . , β^ = ∑ x k y
uncorrelated) or R2j =0 , then ∑ x 22 ∑ x 23 k
∑ x 2k and
σ2
Var ( β j )=
^
∑ x 2j ∀ j j = 2, 3, ..., k
Var ( β^ j ) Var ( β^ j )
Based on this fact, we use the ratio of to the orthogonal to obtain a

103
measure of the degree of multicolliniearity in the model, which is known as the Variance
Inflation Factor (VIFj) for the jth explanatory variable. That is,
Var ( β^ j ) σ2 σ2 1
VIF j = = / = ≥ 1
Var orth ( β^ j ) ∑ x 2j ( 1−R 2j ) ∑ x j ( 1−R2j )
2

2
There is a positive relationship between R j and VIFj, which is depicted in the following
table.

R2j VIFj
0.1 1.11
0.2 1.25
0.5 2
0.8 5
0.9 10
0.95 20

VIFj

1
2
Rj
1

1Detection of Multicollinearity
1. The estimates are unstable: thus, though R2 is high, the parameter estimates will
be unstable and insignificant. Actually, if R2j = 1 it follows that the parameter
estimates will be indeterminate.
2. Deletion/addition of an explanatory variable causes large changes in the estimates
of the remaining coefficients.
Suppose that Y = 2 + 3X2 + X3 and that X3 = 2X2 - 1. There is no disturbance term in the

104
equation for Y, but that is not important. Suppose that we have the six observations
shown below.
X2 X3 Y
10 19 51
11 21 56
12 23 61
13 25 66
14 27 71
15 29 76
The three variables are plotted as line graphs below. Looking at the data it is impossible
to tell whether the changes in Y are caused by changes in X2, by changes in X3, or jointly
by changes in both X2 and X3.
80

70

60

50

40

30

20

10

0
1 2 3 4 5 6

X2 X3 Y Change in X2 Change in X3 Change in Y

10 19 51 1 2 5
11 21 56 1 2 5
12 23 61 1 2 5
13 25 66 1 2 5
14 27 71 1 2 5
15 29 76 1 2 5
Numerically Y increases by 5 in each observation. X2 changes by one. Hence the true
relationship could have been Y = 1 + 5X2. However, it can also be seen that X3 increases
by 2 in each observation. Hence the true relationship could have been Y = 3.5
+[Link] two possibilities are special cases of Y = 3.5 -2.5p + 5pX2 +2.5(1-p)X3,
which would fit the relationship for any value of p. There is no way that regression
analysis, or any other technique, could determine the true relationship from this infinite

105
set of possibilities. What would happen if you tried to run a regression when there is an
exact (perfect) linear relationship among the explanatory variables?

Given the model:

Y i =β 1 +β 2 X 2 i +β 3 X 3 i +ui where X 3i =μ+λX 2 i

We will investigate, using the model with two explanatory variables shown above.
[Note: A disturbance term has now been included in the true model, but it makes no
difference to the analysis.]

^
The multiple regression coefficient β 2 is calculated as:
^ =
β
∑ x 2i y i ∑ x 23 i−∑ x 2 i x 3 i ∑ x 3i y i
(∑
2

)
2

∑ x22 i ∑ x 23i − x 2i x 3i

Assume that X3i =μ+ λX2i, where λ is non-zero constant, the estimator of β2 can be written
as:
^ =
β
∑ x 2i y i λ 2 ∑ x 22i − λ2 ∑ x 22 i ∑ x2 i y i =
0

(∑ )
2 2 0
λ 2 ∑ x 22i ∑ x 22i − λ 2
2
x 2i
- Indeterminate
It turns out that both the numerator and the denominator are equal to zero. The
regression coefficient is not defined (determinate).

The presence of an exact relationship among the explanatory variables in a regression is

unusual. When this occurs, it is typically because there is a logical error in the
specification.

Possible Measures for Alleviating Multicollinearity

What can you do about this problem if you encounter it? We will discuss some possible
measures, looking at the model with two explanatory variables. Before doing this, two
important points should be emphasized.
1. Multicollinearity does not cause the regression coefficients to be biased. Their
probability distributions are still centred around the true values, if the regression
specification is correct, but they have unsatisfactorily (unnecessarily) large
variances.
2. The standard errors and t tests remain valid. However, the standard errors are
larger than they would have been in the absence of multicollinearity, warning us
that the regression estimates are erratic.

106
Since the problem of multicollinearity is caused by the population variances of the
coefficients being unsatisfactorily large, we will seek for ways of reducing the variances.
We will look at the various components of the population variance. We might be able to
reduce it by bringing more variables into the model and reducing the population variance
of the disturbance term.

σ 2 = n Varσ ( X ) ¿ 1−r1
2 2
σ
V ar ( β^ 2 )= β^ 2
=
( 1−r 22 ,3 ) ∑ x 22 i
2
2 2 ,3

1. Reduce σ^ by including further relevant variables in the model.

2

2. Increase the number of observations.

Surveys: increase the budget. Use clustering. Time series: use quarterly instead of
annual data The next factor to look at is n, the number of observations. If you are
working with cross-section data (individuals, households, enterprises, etc) and you are
undertaking a survey, you could increase the size of the sample by negotiating a bigger
budget. Alternatively, you could make a fixed budget go further by using a technique
known as clustering. You divide the country geographically by zip code or postal area.

You select a number of these randomly, perhaps using stratified random sampling to
make sure that metropolitan, other urban and rural areas are properly represented. You
then confine the survey to the areas selected. This reduces the travel time and cost of the
fieldworkers, allowing them to interview a greater number of respondents. If you are
working with time series data, you may be able to increase the sample by working with
shorter time intervals for the data, for example quarterly or even monthly data instead of
annual data.
3. Increase Var(X2): A third possible way of reducing the problem of multicollinearity
might be to increase the variance of the explanatory variables. This is possible only at the
design stage of a survey.

For example, if you were planning a household survey with the aim of investigating how
expenditure patterns vary with income, you should make sure that the sample included
relatively rich and relatively poor households as well as middle-income households.

107
4. Reduce
r 2 , 3 : Another possibility might be to reduce the correlation between the
explanatory variables. This is possible only at the design stage of a survey and even then
it is not easy.
5. Combine the correlated variables: If the correlated variables are similar conceptually,
it may be reasonable to combine them into some overall index.
6. Drop some of the correlated variables: Drop some of the correlated variables, if they
have insignificant coefficients. However, this approach to multicollinearity is dangerous
because some of the variables dropped may truly belong to the model and their omission
may cause omitted variable bias.
7. Empirical restriction: A further way of dealing with the problem of multicollinearity
is to use extraneous information, if available, concerning the coefficient of one of the
variables.
For example, suppose that Y in
ln Y t =β 1 + β2 ln X t + β 3 ln Pt + ut

is the demand for a category of consumer expenditure, X is aggregate disposable personal

income, and P is a price index for the category.
To fit a model of this type you would use time series data. If X and P are highly
correlated, which is often the case with time series variables, the problem of
multicollinearity might be eliminated in the following way.
Obtain data on income and expenditure on the category from a household survey and
regress Y/ on X/. (The /
marks are to indicate that the data are household data, not
aggregate data.)
ln Y ¿i =β ¿1 +β ¿2 ln X i¿ +ν i
This is a simple regression because there will be relatively little variation in the price paid
^¿
β
by the households, obtain 2 .
Now assuming that cross-sectionally estimated income elasticity is the same as that
would be obtained from a time series analysis, we may write the preceding time series
regression as:

108
 ¿
Y t = β 1 +β 2 ln Pt +ut
Where Y ¿t = ln Y t − β^ ¿2 ln X t , represents that value of Y after having removed the effect of X (income )
This is a simple regression, so multicollinearity has been eliminated. There are some
^
problems with this technique. First, the β 2 coefficients may be conceptually different in
time series and cross-section contexts. Second, since we subtract the estimated income
^ ¿ ln X
β
component 2 t , not the true income component β 2 ln X t , from ln Y t when constructing
¿
Y t , we have introduced an element of measurement error in the dependent variable.

8. Theoretical restriction: is the use of a theoretical restriction, which is defined as a

theoretical relationship among the parameters of a regression model.
It will be explained using an educational attainment model as an example. Suppose that
we hypothesize that highest grade completed, S, depends on ASVABC, and highest grade
completed by the respondent's mother and father, SM and SF, respectively.

Mother's education is generally held to be at least, if not more, important than father's
education for educational attainment, so this outcome is unexpected.
S= β1 + β 2 ASVABC+ β3 SM +β 4 SF +u
β 3 =β 4
S= β1 + β 2 ASVABC+ β3 (SM + SF )+u
¿ β 1 + β 2 ASVABC+ β 3 SP+u

4.5 Hetroscedasticity
An important assumption maintained so far in our analysis is that of constant variance
(assumption of common variance). Or, stated differently, we assumed that the equation
has a common variance 2. This is known as the homoscedastisity assumption. In this
section attempt will be made to examine the validity of this assumption and find out what
happens if this assumption is not fulfilled. Some of the questions that we need to answer
are:
1. What is the nature of heterscedasticity?
2. What are its consequences?

109
3. How does one detect it?
4. What are the remedial measures?
The Nature of Heteroscedasticity

Sources (causes) of heteroscedasticity:

 due to large variation in the size of entities for which data are collected
 Due to outliers, which can increase or decrease variance

This sequence introduces the topic of heteroscedasticity, which relates to the distribution
of the disturbance term in a regression model. We will discuss it in the context of the

regression model: Y i =β 1 + β 2 X i + ε i

To keep the diagram uncluttered, we will suppose that we have a sample of only five
observations, the X values of which are as shown. If there were no disturbance term in the
model, the observations would lie on the line as shown below.

E Y  1  2 X

1

X X2 X3 X4 X5 X
1

Now we take account of the effect of the disturbance term. It will displace each
observation in the vertical dimension, since it modifies the value of Y without affecting

110
 Y
Y

Y=1+2X
1

X
X.
The disturbance term in each observation is hypothesized to be drawn randomly from a
given distribution. In the diagram, three assumptions are being made.

One is that the expected value of  in each observation is 0. The second is that the
distribution in each observation is normal. We are not concerned with either of these and
we will assume them to be true.

The third is that the distribution is the same for each observation. In the present case, that
means that the normal distributions shown all have the same variance.

If this condition is satisfied, the disturbance term is said to be homoscedastic (Greek for
same scattering).

Each observation is then potentially (before the sample is drawn) an equally reliable
guide to the location of the line
Y = β1 + β 2 X +ε
Once the sample has been drawn, some observations will lie closer to the line than others,
but we have no way of anticipating in advance which ones these will be.

111
 Y
Y

Y=1+2X
1

Now consider the situation illustrated by the diagram above. The distribution of ε
associated with each observation still has expected value 0 and is normal. However, the
variance is no longer constant. Obviously, observations where ε has low variance, like
that for X1, will tend to be better guides to the underlying relationship than those like that
for X5, where it has a relatively high variance. When the distribution is not the same for
each observation, the disturbance term is said to be subject to heteroscedasticity.

There are two major consequences of heteroscedasticity. One is that the standard errors
of the regression coefficients are estimated wrongly and the t tests (and F test) are
invalid.

The other is that OLS is an inefficient estimation technique. An alternative technique,

which gives relatively high weight to, the relatively low-variance observations should

tend to yield more accurate estimates.

Effects/Consequences of Heteroskedasticity
Heteroscedasticity implies that the variance of error terms is not constant. Thus,

Var ( ε i ) = σ 2i

112
The main effects of this are:
1. The least squares estimators are still unbiased, but they are inefficient, and
2. Variance of the estimators are biased–thus the tests of significance based on them are
invalid.
To see this consider the simple regression model in deviation form
y i = β x i +ε i , with Var ( ε i ) = σ 2i

The OLS estimator of the slope parameter is

∑ x i yi ∑ xi ( β xi + εi ) ∑ xi εi
β^ = = = β+
∑ x 2i ∑ x 2i ∑ x 2i
Since E(i) = 0 and and X are independent, it follows that
∑ x i E ( εi )
E ( β^ ) = β + = β
∑ x 2i
Therefore, the OLS estimator is unbiased.
3. However, hetroscedasticity increases the variance of the error term and hence it does
affect the minimum variance property.
If i’s are mutually independent, i.e., uncorrelated,

[ ( β−
^ β) ]
2
^) = E
Var ( β

[( ) ] [ ∑
∑
]
2
xi ε i ∑ x 2i ε 2i + ∑ ∑ x i ε i x j ε j
i≠ j
= E = E

∑ ( )
2
2 2
xi xi

∑ x 2i E ( ε 2i ) + ∑ ∑ x i x j E ( ε i ε j ) ∑ x 2i σ 2i
i≠ j
= =

(∑ ) (∑ )
2 2
2 2
xi xi

OLS estimation in the presence of heteroscedasticity results in estimators that are still
linear and unbiased but less efficient, i.e., the OLS estimator does not possess the
minimum variance in the class of linear unbiased estimators. This is one violation of one
of the classical assumptions of homoscedasiticity (Common Variance of the disturbance
terms). Hence, in the presence of heteroscedasticity, OLS estimators are no longer BLUE
as OLS estimation does not make use of the "information" contained in the unequal
variability of the dependent variable Y.

113
  Thus the OLS estimators are inefficient and hence no longer BLUE
 Thus, OLS in the presence of heteroscedasticity, causes standard errors to be
inappropriate and hence any inferences we make could be misleading.
4. t-test and F-test are invalid in the presence of heteroscadisticity
5. OLS estimation technique is inappropriate. Thus, we need alternative estimation
technique known as “Generalized Least Squares-GLS”. This estimation technique
transforms the original variables in such a way that the transformed variables satisfy the
assumptions of classical model and then apply the OLS estimation method and is
therefore capable of producing estimators that are BLUE. The estimators thus obtained
are known as GLS estimators.
In short, GLS is OLS on the transformed variables that satisfy the assumptions of CLRM

Now, suppose that we know the functional form of the variance up to a multiplicative
constant, say, given by

σ 2i = σ 2 z 2i

and zi's are known. Then dividing the original equation by zi, we get
yi xi εi
= β +
zi zi zi

()
¿ ¿ ¿ ¿ 1 εi σ 2 z 2i
y i = β xi + εi ; Var ( ε i )= Var = 2 Var ( ε i )= 2 =σ 2
zi zi zi

β^ be
¿
Let the parameter estimate in this model: it is known as the Weighted Least
Squares (WLS) or GLS estimators because we are using zi's as weights that deflate the
equation. The WLS estimator is BLUE.
Proof:

^¿ = ∑ x¿i y ¿i ∑ ( y i / zi )( x i / z i ) ∑ ( x i / z i ) ( βx i / z i + ε ¿i )
β = =
∑( / z ) ∑( /z )
¿ 2 2 2
∑ ( xi ) xi xi
i i

( / zi ) +∑ ( x / z ) ε
2
β ∑ xi
¿

=
i i i
= β+
∑ ¿
( xi / zi ) εi
∑ ( xi / zi ) ∑ ( xi / zi )
2 2

Noting that E() = 0, it follows that

114
 ( )
∑ ( x i / z i ) ε ¿i ∑ ( x i / z i ) E ( ε ¿i )
E ( β^ ¿ ) = E β + = β+ = β
∑ ( xi / z i ) ∑ ( xi / zi )
2 2

Thus, the WLS is an unbiased estimator of . In addition, the WLS is more efficient than
the OLS estimator. By this we mean that the variance of WLS estimator is less than the
variance of the OLS estimator.
Proof: Now the variance of the WLS estimator is

[( ∑
)]
2

( /z )ε
xi i
¿
i
^ ¿) = E
Var ( β [ ( β^ ¿
^ ¿))
−E ( β
2
] = E

∑(
2

xi / zi )

[( ∑
∑ ( xi / zi ) ( ε i )

]
2
¿ 2
+ ∑ ∑ ( x i / z i ) ( x j / z j ) ε ¿i ε ¿j
i≠ j
= E

( x / zi ) )
2 2

∑ ( x / zi ) E ( ε ) +∑ ∑ ( x / z ) ( x / z ) E ( ε ε )
2
¿ 2 ¿ ¿
i i i i j j i j
i≠ j σ2
= =

( ∑ ( / zi ) ) ∑ ( xi / zi )
2 2 2

xi

and saw earlier that the variance of the OLS estimator is

^) =
Var ( β
∑ x 2i σ 2i
(∑ )
2
2
xi

^ ) = σ2
Var ( β
∑ x 2i z 2i
(∑ )
2
2
xi
Substituting σ 2i = σ 2
z 2i in this formula we have
To obtain the relative efficiency of WLS to that of OLS, we just obtain the ratio of the
two variances, which is

( / )
2
^ ¿)
Var ( β σ 2 /∑ x i z i
=

(∑ )
^)
Var ( β 2
2

σ ∑ xi zi /
2 2 2
xi

(∑ )
2
2
xi
=
∑ ( xi / z i ) ∑ x 2i z 2i
2

Let ai = xi/zi, and bi = xizi, then we can write this expression as

115
 (∑ ) (∑ )
2 2
2
xi ai b i
=

(∑ ( / ) ) ∑ ∑ a2i ∑ b 2i
2
xi zi x2i z 2i

It can easily be show that this value is less than unity. It will be equal to unity if and only
if
1. ai and bi are proportional, that is xi/zi and xizi are proportional, implying that their
multiplication will always give a constant, or
2. zi2 is a constant for all i, implying that the error terms are constant (homoscedastic).
3. Variances of the estimators are biased and thus tests of significance based on them are
invalid.
Conclusion:
If the error terms are not homoscedastic
a) OLS estimators are unbiased
b) the OLS estimators are less efficient (i.e., have higher variance) than the WLS.

Detection of Hetroscedasticity
The first, and simplest, way of detecting hetroscedasticity is to cross-plot the residuals
versus the explanatory variable. If the residuals from the levels cross-tabulated against
the explanatory variable indicate that there is larger variation in the residuals as the
explanatory variable increases, then one may suspect the presence of heteroscedasticity.

To detect the type of relationship that exists between the explanatory variable and the
residual, we have to check the correlations (even use regression) between the two

116
variables. Recall however, that the residuals and the explanatory variable(s) are
uncorrelated. Thus any regression of the residual and the explanatory variable would give
no significant relationship. Consequently, the following tests that are based on regression
have been suggested in the literature.

1. Glejser test: amounts to running the OLS regression first and then obtaining the
^ i , and then regressing the absolute values of the u
residuals u ^ i on the X variable that
2
is thought to be closely associated with σ i using the functional forms:
|^ui| = β 1 + β 2 X i + ν i
|^ui| = β 1 + β 2 √ X i + ν i
1
|^ui| = β 1 + β 2 +ν i
Xi
1
|^ui| = β 1 + β 2 +ν
√ Xi i
Where νi is the error term.
Glejser has found that for large samples these models give generally satisfactory results
in detecting heterscedasticity.

2. Goldfield and Quandt test: This test is applicable if one assumes that the
2
heteroscedastic variance, σ i , is positively related to one of the explanatory variables
in the regression model. For simplicity, consider a two variables regression model of
the form:
Y i = β1 + β 2 X i +ui
2
Suppose σ i is positively related to Xi as
σ 2i = σ 2 X 2i
2
Where σ is constant
2
If this assumption appropriate, it would mean σ i would be larger, the larger the values
of Xi. If this turns out to be the case, heterscedasticity is most likely to be present in the
model. To test this explicitly, the following steps are suggested:

117
Step1. Rank or order the observations in ascending order of magnitude of values of Xi.
Step2. Omit c central observations, where c is specified a priori, and then divide the
remaining (n-c) observations into two groups of each of (n-c)/2 observations.
Step3. Fit separate OLS regressions to the first (n-c)/2 observations and the last (n-c)/2
observations, and obtain the respective residual sum of squares, RSS1 and RSS2.
( n−c ) ( n-c- 2 k )
−k or
These residual sum of squares each have 2 2 degrees of
freedom, where k is the number of parameters to be estimated, including the
intercept.
RSS 2 / df
f =
Step4. Compute the ratio RSS 1 / df .

If ui are assumed to be normally distributed, and if the assumption of homoscedasticity is

valid, then it can be shown that f follows the F-distribution with numerator and
denominator df each of (n-c-2k)/2.

The central observations are omitted to sharpen or accentuate the difference between the
small variance group and large variance group. The ability of Goldfield-Quandt test to
detect the presence of heteroscedasticity depends on how the "c" central observations are
chosen, and on identifying the correct X variable with which to order the observations.
Goldfield and Quandt suggest omitting about a quarter of the central observations.

3. Breusch-Pagan-Godfrey test: limitations of the Goldfield-Quandt test can be

avoided by using this test. To illustrate this test, consider a k-variables linear
regression model:

Y i = β1 + β 2 X 2i +⋯+ β k X ki +ui EQ 23
2
Assume that the error variance σ i is described as:

σ 2i = f ( α 1 +α 2 Z 2 i +⋯+α m Z mi ) EQ 24
2
that is, σ i is some function of the non-stochastic variables Z's; some or all of the X's
can serve as Z's. Specifically you may assume that

118
 σ 2i = α 1 + α 2 Z 2i +⋯+ α m Z mi EQ 25
2 2
that is, σ i , a linear function of the Z's. If α2 = α3 = ····· = αm = 0, σ i = α1, which is
2
constant. Therefore, to test whether σ i is homoscedastic, one can test the hypothesis
that H0: α2 = α3 = ····· = αm = 0. The actual test procedure for the Breusch-Pagan-
Godfrey test is as follows:

Step1. Estimate EQ1 by OLS and obtain the residualsu^ 1 , u^ 2 , ⋯ u^ n .

~2 =
Step2. Obtain σ ∑ u^ 2i /n , the maximum likelihood estimate of σ 2 .
Step3.
^ 2 ~2
Construct variables pi defined as pi = ui / σ
Step4. Regress Pi on the Z's as

pi = α 1 + α 2 Z 2i +⋯+ α m Z mi+ ν i EQ 26
Step5. Obtain the ESS (explained summ of squares) from step 4 and define
1
Θ = ( ESS )
2 EQ 27
Assuming ui are normally distributed, one can show that if there is homoscedasticity and

Θ ~ 2
χ m−1
if the sample size is sufficiently large, then asy

Therefore, if the computed chi-square value exceeds the critical value at the chosen level
of significance then on can reject the hypothesis of homoscedasticity.

4. White's Heteroscedasticity test: This does not rely on the normality assumption of
the error terms and is easy to apply. For the purpose of illustration, consider a three
variables linear regression model:

Y i = β1 + β 2 X 2i + β3 X 3 i +ui EQ 28
The White test proceeds as follows:

Step1. ^i.
Estimate the model in EQ 6 and obtain the residuals, u
Step2. Run the following auxiliary regression:

u^ 2i = α 1 +α 2 X 2 i +α 3 X 3 i +α 4 X 22i + α 5 X 23i + α 6 X 2 i X 3i + ν i EQ 29

119
 Higher powers of regressors can also be introduced in this regression, and then
obtain the R2 from this auxiliary regression.
Step3. Under the null hypothesis that there is no heteroscedasticity, it can be
shown that the sample size (n) times R2 obtained from the auxiliary regression
asymptotically follows the chi-square distribution with degrees of freedom equal to
the number of regressors (excluding the constant term) in the auxiliary regression.
That is

nR
2
~ 2
χ df
asym EQ 30
Step4. If the chi-square value obtained in EQ 8 exceeds the critical chi-square
value at the chosen level of significance, the conclusion is that there is
heteroscedasticity.

Remedial Measures

Heteroscedasticity does not destroy the unbiasedness and consistency properties of the
OLS estimators, but they are not efficient, even asymptotically. This inefficiency makes
the usual hypothesis testing procedure dubious and as a result, remedial measures called
2 2
for. There are two approaches to remedies: when σ i is known and when σ i is not
known.
2 2
When σ i is known: If σ i is known, the most straightforward method of correcting for
heteroscedasticity is by means of weighted least squares so that the estimators thus
obtained to be BLUE.

For the purpose of ease of exposition, let us consider a two variables linear regression
model of the following form:

Y i = β1 + β 2 X i +ui
Where E ( ui ) = 0 E ( u2i ) = σ 2i and Cov ( ui , u j ) = 0 ∀ i≠ j

To correct for heterscedasticity, divide the above regression equation through by σ i to

obtain:

120
 ( ) ( )( )
Yi X X u
= β1 0 i + β2 i + i Where X 0 i = 1 for each i
σi σi σi σi

For ease of exposition, write the above regression equation as

¿ ¿ ¿ ¿ ¿ ¿
Y i = β 1 X 0i +β 2 X i +ui

Where the starred or transformed variables are the original variables divided by σ i , and
¿ ¿
β
the notations 1
and β 2 , the parameters of the transformed model, are used to distinguish

them from the usual OLS parameters β 1 and β 2 .

¿
The variance of the transformed error term ui is

] = E (σ [ )]
2
ui
[
Var ( ui ) = E ( ui )
¿ ¿ 2

1
= 2
E [ u2i ]
σi
1 2
= 2 σi = 1
σi

which is constant. Hence, the variance of the transformed error term is now
homoscedastic. Therefore, OLS applied to the transformed variables results in estimators
that are BLUE. Weighted least squares amounts to applying OLS on the transformed
variables that satisfy the standard least-squares assumptions.
2 2
When σ i is not known: When the true σ i are not known one alternative way to deal
with heteroscedasticity is to obtain the White's Heteroscedasticity-Consistent Variances
and Standard errors of the estimators of each parameter values. Asymptotically valid
statistical inferences can be made about the true parameter values using this estimate.

The other alternative remedy to heteroscedasticity is to make a plausible assumption

about heteroscedasticity pattern. To illustrate this, let us consider the two-variables linear
regression model:

121
 Y i = β1 +β 2 X i +ui
Where E ( ui ) = 0 E ( u2i ) = σ 2i and Cov ( ui , u j ) = 0 ∀ i≠ j

Assumptions about pattern of heteroscedasticity:

Assumption 1: The error variance is proportional to X i , i.e., E (u i ) = σ X i

2 2 2 2

Transform the original model by dividing through by Xi to obtain

Yi β1 ui
= + β 2+
Xi Xi Xi
1
= β1 + β2 +ν i
Xi

where νi is the transformed error term, equal to ui/Xi. It can easily be shown that:

[( ) ]
2
ui 1 1
2 ( i )
Var ( νi )=E ( ν 2i ) = E = E u 2 = 2 σ2 Xi2
Xi Xi Xi
= σ2
Now the variance of νi is homoscedastic, and hence one can proceed to apply OLS to the
transformed equation by regressing Yi/Xi on 1/Xi.

Assumption 2: The error variance is proportional to Xi: E (u i ) = σ X i

2 2

If this is the case with regard to the pattern of heteroscedasticity, then the original model

can be transformed by dividing through by √ X i as follows:

Yi β1 ui
= + β2 √ Xi+
√ Xi √ Xi √ Xi
1
= β1 + β2 √ X i+ νi
√ Xi
where ν i = u i / √ X i and where X i > 0 , Var ( ν i )=σ 2

Given assumption 2 above, it can easily be shown that variance of ν i is homoscedastic,

and hence one can proceed to apply OLS to the transformed model.

122
Assumption 3: The error variance is proportional to the square of the mean value of Y,
i.e.,

[ E (Y ) ] .
2

E( u 2i ) = σ 2
i

If the patter of heteroscedasticity is as postulated in assumption 3 above, to correct for

heteroscedasticity transform the original model as follows:

Yi β1 Xi ui
= + β2 +
E (Y i ) E ( Y i) E (Y i ) E (Y i )

( )
1 X
= β1 + β2 i +ν i
E ( Y i) E (Y i )

where νi = ui/E(Yi), it can easily be shown that E ( ν i ) = σ implying that the

2 2

disturbance terms νi are homoscedastic. However, this transformation is inoperational as

^ ^ ^
the E(Yi) depends on unknown parameters β1 and β2. But we know Y i = β1 + β 2 X i is
an estimator of E(Yi). Hence, we may proceed in correcting for heterscedasticity in two
steps:

First run the usual OLS regression, disregarding the heterscedasticity problem, and obtain
Y^ i . Second, using the estimatedY^ i , we transform the original model as follows:

( )
Yi 1 X
= β1 + β 2 i +ν i
Y^ i Y^ i Y^ i
where ν = u / Y^ i i i

^
and run the above transformed model using OLS. Although Y i is not exactly the same as
E(Yi), they are consistent estimators, as the sample size increases indefinitely, they
converge to the true E(Yi). Hence, the above transformation will perform satisfactory if
the sample size is sufficiently large.

Assumption 4: A log transformation such as ln Y i = β 1 + β 2 ln X i +u i very often

123
reduces heteroscedasticity when compared with the regressionY i = β1 + β 2 X i +ui .

This result arises because log transformation compresses the scales in which the variables
are measured. The additional advantage of the log transformation is that the slope
coefficient β2 measures elasticity of Y with respect to X, which can easily be interpreted.

4.6 Autocorrelation
The term autocorrelation may be defined as correlation between members of series of
observations ordered in time (as in time series data) or space (as in cross-sectional data).
The classical linear regression assumes that such autocorrelation does not exist in the
E ( ui u j ) =0 for all i≠ j
disturbancesui , i.e., . However, if there is such dependence, we
E ( ui u j ) ≠0 for i≠ j
have autocorrelation. Symbolically, .

Autocorrelation versus Serial Correlation: Although it is now a common practice to treat

these two terms synonymously, the term autocorrelation is defined as “lag correlation of a
given series with itself, lagged by a number of time units,” where as the term serial
correlation may be defined as “lag correlation between two different series.”
E ( u t ut− j )≠0 for j≠0
Symbolically, Autocorrelation reefers to the fact that while
E ( u t ν t− j ) ≠0 for j≠0
serial correlation refers to
Consider a two-variable linear regression model of the form:
Y t = β 0 + β1 X t +ut t=1, 2 , ⋯⋯, T
Autocorrelation is present in the residuals if
E ( u t ut−s ) ≠ 0 for s>1
i. e . , cov ( us , u t ) ≠ 0 for |s|≠t
The autocovariance at lag s is defined by
γ s = E ( u t ut −s ) s=0 , ±1 , ±2 ,⋯

At lag zero, i.e., s=0, we have γ 0 = E (u t ) = σ

2 2
i . e ., constant variance
.
The Autocorrelation coefficient at lag s is defined by:

124
 γs
ρs =
γ0

Note that ρ0 = 1
The γ's and the ρ's are symmetric in s (the lag) and do not depend on t (time), i.e., they
only depend on the lags.
The covariance matrix of the u's is now:

[( ) ] [ ]
u1 u21 u1 u2 ⋮ u 1u T
2
u uu u ⋮ u 2u T
E ( uu ) = E 2 (u1 u2 ⋯ uT ) = E 2 1 2
'

⋮ ⋯ ⋯ ⋯ ⋯
ut uT u 1 u T u 2 ⋮ u2T

[ ]
[ γ0 γ 1 γ 2 ⋯ γT −1 ¿ ][ γ1 γ 0 γ1 ⋯ γ T−2 ¿ ] [⋯ ⋯ ⋯ ⋯ ⋯ ¿ ] ¿ ¿¿
E ( u21) E ( u1 u 2 ) ⋮ E ( u 1 u T ) ¿
E (u2 u1 ) E ( u22) ⋮ E ( u 2 uT )
= =
⋯ ⋯ ⋯ ⋯
E (uT u1 ) E (uT u2 ) ⋮ E ( u2T )
¿
The autocorrelation matrix is given by:

E (uu') [ 1 ρ ρ2 ⋯ ρT−1 ¿] [ ρ 1 ρ ⋯ ρT−2 ¿] [⋯ ⋯ ⋯ ⋯ ⋯ ¿] ¿

=¿ ¿
γ0 ¿
Given that there is temporal dependence in the disturbance terms, how do we best model
it? There are three main types of time series process that we could use:
1. Autoregressive process of order p AR(p):
ut = φ1 ut−1 +φ 2 ut−2 +⋯⋯+φ p ut− p +ε t
Where E ( ε t ) = 0 Var ( ε t ) = σ 2ε and Cov ( ε t , ε s ) = 0 for s≠t
2. Moving Average process of order q MA(q):
ut = ε t + θ1 ε t−1 +θ 2 ε t−2 +⋯⋯+θq ε t−q
Where E ( ε t ) = 0 Var ( ε t ) = σ 2ε and Cov ( ε t , ε s ) = 0 for s≠t
3. Autoregressive-Moving Average Process ARMA(p, q):
ut = φ1 ut−1 +φ 2 ut−2 +⋯⋯+φ p ut− p +ε t +θ1 ε t−1 +θ 2 ε t−2 +⋯⋯+θq ε t−q
Where E ( ε t ) = 0 Var ( ε t ) = σ 2ε and Cov ( ε t , ε s ) = 0 for s≠t
This is a very general process and may be difficult to estimate. However, we can usually

125
be fairly confident of capturing realistic dynamics in the residuals by considering low
order AR process. Hence, we will confine ourselves (our attention) to the AR(1) model:
ut = ρ ut −1 + ε t

To ensure that this process is stationary, we require, |ρ| < 1 in statistical sense.
Definition: A weakly Stationary Process has a constant and finite mean and variance, and
a set of (auto) covariances which are independent of time and are functions of the lag
length, s.
Consider the following regression model:
Y t = β 1 + β 2 X 2t +⋯+ β k X kt +ut where ut = ρut−1 + ε t
and E ( ε t ) = 0 , Var ( ε t ) = σ 2ε for all t , Cov ( ε t , ε t−s ) for all s≠0
The regression equation can be written matrix form as follows:

Y = Xβ+u
E (ε ) = 0 E (ε ε ) = σ2 I T [u0 ¿] [u1 ¿] [u2 ¿] [⋮¿] ¿ ¿¿
'
u = ρu−1+ε
Where u−1 = ¿
¿
Here, we know that

2 σ 2ε
Var ( ut ) = σu =
1−ρ 2
and E ( u u ) = σ u Ω
'' 2

Where Ω=¿ 1 ρ ρ ⋯ ρ ρ 1 ρ ⋯ ρ ¿][⋯ ⋯ ⋯ ⋯ ⋯¿]¿¿

[ ] [
2 T−1 T−2
¿
¿
If we were to estimate by OLS:
i. Still unbiased
ii. Invalid inferences because the usual standard errors would be wrong.
Hence, use GLS, which is fine if ρ (and hence Ω) are known.

126
Estimation when ρ is known:

Consider the model: Y t = β 0 + β1 X t + ut EQ 31

ut = ρ ut −1 + ε t where E ( ε t ) =0 , Var ( ε t )=σ 2 for all t, Cov ( ε t , ε t − j ) =0 for all j≠0

Multiply EQ 1 by ρ and lag by (1) period to get:

ρY t−1 = ρ β 0 + ρβ1 X t−1 + ρu t−1 EQ 32

Subtract EQ 2 from EQ 1 to obtain:

Y t − ρY t −1 = β 0 ( 1−ρ ) + β 1 ( X t −ρ X t −1 ) +u t −ρ ut −1

¿ ¿ ¿
Y t = β 0 +β 1 X t +ε t EQ 33

Equation (3) satisfies the usual assumptions of the classical linear regression model, and
hence once can apply OLS, which is BLUE.

Estimation when ρ is unknown:

In this case, we need to estimate ρ along with β 0 and β1. One method is to use the
COCHRAN ORCUTT procedure:
^ ^
Step1. Apply OLS to Y t =β 0 + β 1 X t + ut and obtainut =Y t − β 0− β1 X t , t=1, 2, ..., T.
^

Step2. Run the regressionu^ t = ρ u^ t −1 + ε t , that is, estimate ρ by

T
∑ u^ t u^ t −1
t =2
ρ^ = T
∑ u^ 2t −1
t =2
¿ ¿ ¿
¿ ¿
Y =β 0 +β 1 X t +ε t .
Step3. Compute Y t =Y t − ρ^ Y t−1 , X t =Y t − ρ^ X t−1 and apply OLS to t
The resulting estimates are consistent and more efficient than OLS estimators.
In practice, we could iterate the above procedure, i.e. use the residuals from step(3) to

compute ρ^ in step 2 and repeat the process until the estimates converges, i.e., until

|ρ^ i − ρ^ i−1| is small. This procedure is available in Stata and other Econometric packages.
Another alternative method is to use the Maximum Likelihood estimation technique.

127
Testing for autocorrelation (1st order)
The most commonly used test is due to Durbin and Watson. The statistic is given by:

∑ ( u^ t −u^ t −1 )
T 2

u^ t =Y t −X t β^
t =2 ¿
d = T
where
∑ u^ 2t
t =1

It can be shown that 0 ≤ d ≤ 4.

If we expand the square in d, we obtain:
T T T
∑ u^ 2t ∑ u^ 2t−1 2 ∑ u^ t u^ t−1
t=2
d = T
+ t=2
T
− T
t=2

∑ u^ 2t ∑ u^ 2t ∑ u^ 2t
t=1 t=1 t=1
≈ 2 ( 1- { ^ρ ¿ )
1
ρ^ ≈ 1− d
Or equivalently, 2
The exact distribution is difficult to calculate - there is an indeterminate region.
NB dL is the lower bound both are obtained from the table
dU is the upper bound

1. To test the hypothesis that: H 0 : ρ=0 against H 0 : ρ>0

Reject H 0 if 0< d <d L
Do not reject H 0 if d u < d <4−du
Inclonclusive region if d L < d <d u

2. To test the hypothesis that: H 0 : ρ=0 against H 0 : ρ<0

Reject H 0 if 4−d L< d <4
Do not reject H 0 if d u < d <4−du
Inclonclusive region if 4−d u <d <4−d L
NB dL is the lower bound both are obtained from the table
dU is the upper bound

128
 Reject H0 in Inconclusive Accept H0 (No Reject H0 in
favour of +ve region autocorrelation) Inconclusive favour of –ve
autocorrelation region autocorrelatio
n
d
0 dL du 2 4- du 4-dL 4

129
4.7 DYNAMIC MODELS
So far we have considered purely static models of the form:
Y t = β 0 + β1 X t + ut
What implications does this have for the underlying behaviour of economic agents?
What is the effect on Y of a change in X? If X increases by 1 unit in period t, then Y
will increase by β1 units instantaneously. This is unrealistic. In the real world adjustment
does not take place immediately - there are lags involved. These lags may be
due to:
i. time to feed through the economy
ii. lack of information
iii. search cost
iv. adjustment costs
Suppose it takes three periods for a change in X to feed through into Y given by the
model of the form:
Y t = β 0 + β1 X t + β2 X t −1 + β 3 X t−2 + ε t
The effect on Y of a one-time change in X is as follows:
Period Effect (Cumulative)
t β1
t+1 β1+ β2
t+2 β1+ β2+ β3
This type of model is known as a DISTRIBUTED LAG MODEL. In general we have a
model of the form:
Y t = α + β 0 X t + β 1 X t −1 + β 2 X t −2 + ⋯ + β k X t −k + ε t
k
= α + ∑ β i X t −i +ε t
i=0

The immediate effect of a change in X is given by β0 - this is the impact multiplier.

The effect next period is given by (β0+ β1), then (β0+ β1+ β2) and so on - these
are known as interim multipliers.
k
∑ βi
The total effect of any change in X is given by - i =0 , this is the long run or total
multiplier.

130
The easiest way to estimate Distributed Lag (DL) model is by OLS but note that
estimation is based on T-K observations.
Example: t Xt Xt-1 Xt-2
1 12 - -
2 10 12 -
3 14 10 12
4 15 14 10
Suppose we have an infinite distributed lag model in variables of the following form:

Y t = α + β 0 X t + β 1 X t −1 + β 2 X t −2 + ⋯ + ε t EQ 1

Where the lag length is not defined, that is, how far into the past we want to go. One
alternative estimation mechanism of such a model is to use the KOYCK approach to
distributed lag model. If the β's are all of the same sign, Koyck assumes that they decline
or decay geometrically as follows:

βk = β0 λk k = 0, 1, 2, ⋯ EQ 2

Where λ, such that 0 < λ <1, is known as the rate of decline or decay, of the distributed
lag and where 1- λ is known as the speed of adjustment. By assuming that λ<1, we are
attaching lesser weight to the distant β's than current ones, and also insures that the sum
of the β's, which gives the long-run multipliers, is finite, namely:
∞ ∞ ∞ β0
∑ βk = ∑ β 0 λk = β 0 ∑ λ k =
1−λ
k =0 k =0 k =0

Using equation (2), equation (1) can be written as:

Y t = α + β 0 X t + β 0 λ X t −1 + β 0 λ2 X t −2 + ⋯ +ε t EQ 3

Still the model in equation (3) is not amenable to easy estimation. Now lag equation (3)
by one period and multiply both sides of the resulting model by λ to obtain:

λY t −1 = αλ+ β 0 λ X t −1 + β 0 λ 2 X t −2 + β 0 λ3 X t −3 + ⋯ + λε t −1 EQ 4

Now subtract equation (4) from equation (3) to obtain:

Y t − λY t −1 = α ( 1−λ ) + β 0 X t + ( ε t − λ ε t −1 )

Or rearranging,

131
Y t = α ( 1−λ )+ β0 X t + λ Y t −1 + ν t EQ 5

Where νt = εt-λεt-1, a moving average of εt and εt-1.

AUTOREGRESSIVE MODELS
These are models containing lagged dependent variables as regressors.

Example: Y t = α 0 + α 1 Y t −1 + β0 X t +ε t

An AR(1) model may be written as:

Y t = α 0 + α 1 Y t −1 + ε t which does not depend on Xt.
E(α
^ 1) ¿ α1
The use of OLS on autoregressive models is biased in finite samples, i.e., , but
this bias disappears as the sample size gets larger. That is, OLS is consistent.
NB. A special case of the AR(1) model is the random walk, obtained by setting α 0 = 0

and α1 = 1, i.e., Y t = Y t −1 + ε t .

If α0 ≠ 0, we have a random walk with drift: Y t = α 0 + Y t −1 +ε t

However, these models are non-stationary models. The condition for stationarity is:
‌α1‌< 1.
Why may we be interested in such dynamic models? What is the motivation or
justification for their use?

THE ADAPTIVE EXPECTATIONS MODEL

Consider the following model:
¿
Y t = β 0 + β1 X t +ut EQ 6
¿
Where X t : is equilibrium/desired/ long-run/ expected/normal values of Xt.
¿ ¿
Now X t is not observable, but Xt is observable; so we need some way of relating X t to
Xt. we might suggest the following adjustment mechanism:

X ¿t −X ¿t −1 = γ ( X t − X ¿t −1 )
EQ 7

Where γ is an adjustment parameter - typically we would expect 0 < γ ≤ 1, i.e.,

expectations are revised every period by a fraction γ of the difference between current

132
value of Xt and previous period's expectations..
¿
How can we use this information? Solve equation (7) for X t :
¿ ¿
X t = γ X t + ( 1−γ ) X t −1
and substitute into equation (6) to get:
¿
Y t = β 0 + β1 γ X t +β 1 ( 1−γ ) X t−1 +u t EQ 8
¿
which still includes X t−1 : now lag equation (8) by 1 period and multiply by (1-γ) to
obtain:

( 1−γ ) Y t−1 = β 0 (1−γ ) +β1 (1−γ ) X ¿t−1 + ( 1−γ ) ut−1 EQ 9

Now, subtracting equation (9) from equation (8) gives:

Y t = ( 1−γ ) Y t−1 + γβ 0 + γβ 1 X t +ut − ( 1−γ ) u t−1

or
Y t = ( 1−γ ) Y t −1 + γβ 0 + γβ 1 X t + ν t EQ 10

Where νt = ut-(1-γ)ut-1, which is a moving average error term depending on (1-γ).

For the purpose of estimation, equation (10) above can be written as:
Y t = α 0 + α 1 X t + α 2 Y t−1 + ν t where α 0 =γ β 0 , α 1=γ β 1 ∧ α 2 =1−γ

Estimates of γ , β 0 and β1 can be obtained from estimates of α 0 , α 1 and α 2 as follows:

{{ ^ 0 α^ 1
α
α^ 0=^γ β^ 0 ¿} α^ 1=^γ β^ 1¿}¿¿ ⇒ ¿ β^ 0= ¿ β^1= ¿ ¿¿
THE PARTIAL ADJUSTMENT MODEL
1−^α2 1−^α2
(Stock adjustment Model):
Here we assume that the desired level of Y (Y*) is a function of the observable X, i.e.,
¿
Y t = β 0 + β 1 X t +ut EQ 11

133
 ¿
Where Y t is the desired/expected level of Y.
In this model, it is assumed that Yt adjusts as follows:

Y t −Y t −1 = δ ( Y ¿t −Y t −1 )
EQ 12

Where δ is the adjustment parameter: 0 < δ ≤ 1, i.e., the actual change in Y is some
proportion δ of the desired change in Y.

Solving equation (12) for Yt gives:

¿
Y t = δ Y t + ( 1−δ ) Y t−1 EQ 13
¿
Substitute equation (11) into equation (13) forY t
Y t = δ β 0 +δ β 1 X t + ( 1−δ ) Y t −1 + δ ut
For estimation purpose the above equation can be written as:
Y t = α 0 + α 1 X t + α 2 Y t−1 + ν t where α 0 =δ β 0 , α 1=δ β1 ∧ α 2 =1−δ

Estimates of δ , β 0 and β 1 can be obtained from estimates of α 0 , α 1 and α 2 as follows:

{{ ^ 0 α^ 1
α
α^ 0=δ^ β^ 0 ¿}α^ 1=δ^ β^ 1 ¿}¿¿ ⇒ ¿ β^ 0= ¿ β^ 1= ¿ ¿¿
1−α^2 1−α^2
Economic theory typically has little to say about dynamic adjustment, but the partial
adjustment and adaptive expectations model go some way towards overcoming these
drawbacks.

From our discussions so far we have the following three models:

Koyck Model (Infinite distributed lag model):

Y t = α ( 1−λ )+ β0 X t + λY t −1 + ( ε t −λ ε t −1 )
EQ 14

Adaptive Expectations Model:

134
 Y t = ( 1−γ ) Y t −1 + γβ 0 + γβ 1 X t + [ u t −( 1−γ ) ut −1 ]
EQ 15

Partial Adjustment Model:

Y t = δ β 0 +δ β 1 X t + ( 1−δ ) Y t −1 + δ ut EQ 16

All these models have the following common form:

Y t = α 0 + α 1 X t + α 2 Y t −1 + ν t EQ 17
They are all autoregressive in nature. The classical least squares estimation cannot be
directly applicable for such models for two reasons:
i. Presence of stochastic explanatory variable
ii. Possibility of serial correlation
For the above models, OLS estimators are not only biased but also not consistent as the
stochastic explanatory variable in the regression model is correlated with the stochastic
disturbance term.

THE METHOD OF INSTRUMENTAL VARIABLES (IV)

OLS cannot be applied to Koyck, adapative expectations and partial adjustment models
for the simple reason that the explanatory variable Y t-1 tends to be correlated with error
terms. If some how this correlation can be removed, one can apply OLS to obtain
consistent estimators.

Let us suppose we can find a proxy variable for Y t-1 that is highly correlated with Y t-1 but
uncorrelated with the error term νt, where νt is the error term appearing in the Koyck, or
adpative expectations or partial adjustment models. Such a variable is called an
instrumental variable (IV). One suggestion could be to use Xt-1 as an instrument for Yt-1
and further suggestion is that the parameters of the model in equation (17) can be
obtained by solving the following normal equations:

135
 T +1 T +1 T +1
∑ Yt = T α^ 0 + α^ 1 ∑ X t + α^ 2 ∑ Y t−1
t=2 t=2 t=2
T +1 T +1 T +1 T +1
∑ Y t X t = α^ 0 ∑ X t + α^ 1 ∑ X 2t +α^ 2 ∑ Y t−1 X t
t=2 t=2 t=2 t=2
T +1 T +1 T +1 T+1
∑ Y t X t−1 = α^ 0 ∑ X t−1 +α^ 1 ∑ X t X t−1 +α^ 2 ∑ Y t−1 X t−1
t=2 t=2 t=2 t=2

Durbin-Watson statistic for testing first order autocorrelation is no longer appropriate if

the model contains lagged dependent variable as a regressor (i.e., Yt-1).

TESTING FOR AUTOCORRELATION IN MODELS WITH LAGGED

DEPENDENT VARIABLE:
Consider the model,
Y t = α 0 + α 1 X t + α 2 Y t −1 + ut

Since Yt-1 is included as a regressor, DW is not appropriate. In this case, the appropriate
test statistic is the Durbin's h-statistic. Durbin's h-statistic to test for 1st order
autocorrelation is given by:

h = ^ρ
√ T
1−T . Var ( α^ 2 )
Where T - Number of observations used for estimation
ρ^ - Estimate of ρ in ut = ρ ut-1+εt
In large samples, if ρ = 0 then h has a N(0, 1) distribution, i.e., H 0: ρ = 0 (no first order
auto correlation).
Decision rule: if ‌h ‌> Z(α/2) then reject H0 at α% level of significance, do not reject H 0 if
otherwise.
There is an easy way to compute h, since we know that:
1
ρ^ ≈ 1− d
2
where d is the DW statistic; thus
1
h ≈ 1− d
2 ( ) √ 1−T .TVar ( α^ ) 2

136
 T . Var ( α^ 2 )
NB. This test statistic can not be used if ≥ 1. Alternatively, one can use
Breusch-Godfrey test of higher order autocorrelation, also known as the Lagrange
multiplier test.

137