Chapter Three: Multiple Linear Regression Models

Pre-test Questions
1. What are multiple regression models?
2. How do you think multiple linear regressions are different from simple linear regression
model?
3. Why are multiple regression models advantageous over simple linear regression model?
4. Do you think the estimation and inferences in multiple regression similar with those in
simple linear regression?

Concept and Notations of Multiple Regression Models

Simple linear regression model (also called the two-variable model) is extensively discussed
in the previous section. Such models assume that a dependent variable is influenced by only
one explanatory variable. However, many economic variables are influenced by several
factors or variables. Hence, simple regression models are unrealistic. There is no more
practicality of such models except simple to understand. Very good examples, for this
argument, are demand and supply in which they have several determinants each.
Adding more variables to the simple linear regression model leads us to the discussion of
multiple regression models i.e. models in which the dependent variable (or regressand)
depends on two or more explanatory variables, or regressors. The multiple linear regression
(population regression function) in which we have one dependent variable Y, and k
explanatory variables, ε is given by

E ( u i u j )=0 for i ≠ j 3.1

2 βi vi lnσ 2=lnσ2+βlnXi+vi
Where, Var(ei )=σ Xi e the intercept = value of i when all X’s are zero
2
lne 2=lnσ +βln Xi+vi
i = are partial slope coefficients
X's = the random term
^
¿^
¿ X

X ' s is the amount of change in

22
^
¿X + β ¿

¿ + ^
1
β ¿ i

In this model, for example, when changes by one unit,

0 1
¿^ =
Yi
β ¿
¿ ei=Y i−Y ¿

(n−c) (n−c)
−k
keeping the effect of other variables constant. Similarly, 2 is the amount of change in 2
n n n
2 ∂ ∑( Y −β X −β )
^ ^ 2 2 ∂∑(Y−β^ X−β^ )2 2 ∂∑(Y−β^ X−β^ )2
∂∑ei =1 0 1 2 ∂∑ei =1 0 1 2 ∂∑ei =1 0 1 2

when changes by one unit, keeping the effect of other variables constant. The other slopes
= =04 .5 4.5¿ ¿^ = =0 4.6 4.6¿ ¿^ = =04 .7 4.7¿
¿^ ^
∂β ∂β^ ∂β^
∂ β0 0 ∂ β1 1 ∂ β2 2

are also interpreted in the same way.

Although multiple regression equation can be fitted for any number of explanatory variables
(equation 3.1), the simplest possible regression model, three-variable regression will be
presented for the sake of simplicity. It is characterized by one dependent variable (Y) and two
explanatory variables (X1 and X2). The model is given by:

10
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 ( n−c − 2 k )
v 1= v 2=
2 3.2
−

= the intercept = value of Y when both yi=Yi−Ȳ 4.13are zero

x 1 i = X 1 i− X 1 4 .1
−
x2i=X2i−X2 4.12

¿¿ changes by one unit, keeping the effect of Var ¿¿ constant.

( )( ) (
∑ x1 y 1 ∑ x22 − ∑ x2 y ∑ x1 x 2
)( )
¿^ = 4.14
( )( ) (
∑ x 12 ∑ x 22 − ∑ x 1 x2 2
)
β1 ¿
(∑ x2 y)(∑ x1)−(∑ x1 y )(∑ x1 x2)
2

= the change in Y, when

¿^ = 4.15 ¿ ¿
(∑ x12)(∑ x22)−(∑ x1 x2)2
β2
− − −
¿^ = Y −^β X −^β X 4.16
1 1 2 2
β0

¿^

= the change in Y, when Var ¿¿ changes by one unit, keeping the effect of
¿^
β1 ¿ β2 ¿ constant
Assumptions of the Multiple Linear Regression
Each econometric method that would be used for estimation purpose has its own assumptions.
Knowing the assumptions and their consequence if they are not maintained is very important
for the econometrician. In the previous section, there are certain assumptions underlying the
multiple regression model, under the method of ordinary least squares (OLS). Let us see them
one by one.
Assumption 1: Randomness of ui - the variable u is a real random variable.
[ ] ∑ e 2i
∑x
2
^¿2 1
u ∑ x2∑ x2−(∑ x x )2 ¿^ =
n−3

Assumption 2: Zero mean of - the random variable has a zero mean for each value of
1 2 12
¿^ =
δ 2u ¿
δ ¿
Var ( β 2) ¿

^¿
√ ^
¿
¿^ )= Var ( β ) ¿
0
β0 ¿ i.e. SE ( β0 ¿

¿^

Assumption 3: Homoscedasticity of the random term - the random term β 1 ¿ has constant
√ ¿^
¿^ )= Var(β )¿ ¿^
β2 ¿
1

variance. In other words, the variance of each SE( β ¿ is the same for all the 1 values.
√ ^
¿)
^
¿= Var ( β ¿
2
SE ( β2 ) ¿

2 2 2
Assumption 4: Normality of(R ) - the values of each R are normally distributed R
Assumption 5: No autocorrelation or serial independence of the random terms - the
¿^

∑ y2
R2 = 2
successive values of the random term are not strongly correlated. The values of ∑y
4.17¿

^
¿ as: ^
¿∑ x y

R2 are independent of the values of any other

2
¿^ and β ¿ ∑ x y+ β
2 ¿^
12

(corresponding to (corresponding to ).
1 2
β1 ¿ y
R2 = β
∑
4.18 ¿

R2
2
Assumption 6: Independence of R - every disturbance term R2 is independent of the
explanatory variables. R2
2 2 ( n− 1 )

Assumption 7: No errors of measurement in theR =1−(1−R)(n−k) ady 4.20

- the explanatory variables are measured
without error.
2
Assumption 8: No perfect multicollinearity among the R - the explanatory variables are not
perfectly linearly correlated.
Assumption 9: Correct specification of the model - the model has no specification error in that
all the important explanatory variables appear explicitly in the function and the mathematical

11
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
form is correctly defined (linear or non-linear form and the number of equations in the
model).
Estimation of Partial Regression Coefficients
The process of estimating the parameters in the multiple regression model is similar with that
of the simple linear regression model. The main task is to derive the normal equations using
the same procedure as the case of simple regression. Like in the simple linear regression
model case, OLS and Maximum Likelihood (ML) methods can be used to estimate partial
regression coefficients of multiple regression models. But, due to their simplicity and
popularity, OLS methods can be used. The OLS procedure consists in so choosing the values
of the unknown parameters that the residual sum of squares is as small as possible.
Under the assumption of zero mean of the random term, the sample regression function will
look like the following.

R2 3.3
We call this equation, the fitted equation. Subtracting (3.3) from (3.2), we obtain:

R2 3.4
The method of ordinary least squares (OLS) or classical least square (CLS) involves obtaining

the values R2 is minimum.

2
The values of R is minimum is obtained by differentiation this
sum of squares with respect to these coefficients and equate them to zero. That is,

2
(R )
Solving equations (3.5), (3.6) and (3.7) simultaneously, we obtain the system of normal
equations given as follows:

12
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 2
R ,
Then, letting

2
R
H 0 : β i =0
H 1 : β i≠ 0

The above three equations (3.8), (3.9) and (3.10) can be solved using Matrix operations or
simultaneously to obtain the following estimates:

^¿ )¿¿¿
Reject H 0 if SE ( β i ¿ ¿
¿
Variance and Standard errors of OLS Estimators
Estimating the numerical values of the parameters is not enough in econometrics if the data
are coming from the samples. The standard errors derived are important for two main
purposes: to establish confidence intervals for the parameters and to test statistical hypotheses.
They are important to look into their precision or statistical reliability. An estimator cannot be
used for any purpose if it is not a good estimator. The precision of an estimator is measured by
observing the standard error of the estimator.
Like in the case of simple linear regression, the standard errors of the coefficients are vital in
statistical inferences about the coefficients. We use standard the error of a coefficient to
construct confidence interval estimate for the population regression coefficient and to test the
significance of the variable to which the coefficient is attached in determining the dependent
variable in the model. In this section, we will see these standard errors. The standard error of a
coefficient is the positive square root of the variance of the coefficient. Thus, we start with
defining the variances of the coefficients.

13
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 ¿^
βi
t cal= ^¿ ¿¿
SE(β i )

Variance of the intercept

Reject H 0 if |t cal|¿ t α ,n−3

2 3.17

Variance of
H0

H 0 3.18
H0:β1=β2=0
Variance of H1:βi≠0, at least for one i.
∑ ^
y 2

k −1
F ca l=
∑ e 2

n −k 3.19
Where,

F cal 3.20
Equation 3.20 here gives the estimate of the variance of the random term. Then, the standard
errors are computed as follows:
¿^

Standard error of SSE=∑ y ¿

2

3.21

Standard error of k−1

^
¿

MSE =
∑ y2
¿
k −1 3.22
MSE
F=
Standard error of MSR

SSR=∑ e2 3.23
Note: The OLS estimators of the multiple regression model have properties which are parallel
to those of the two-variable model.

14
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
Coefficient of Multiple Determination
In simple regression model we have discussed about the coefficient of determination and its
interpretation. In this section, we will discuss the coefficient of multiple determination which
has an equivalent role with that of the simple model. As coefficient of determination is the
square of the simple correlation in simple model, coefficient of multiple determination is the
square of multiple correlation coefficient.

The coefficient of multiple determination is the measure of the proportion of the

variation in the dependent variable that is explained jointly by the independent variables in the
n−k
model. One minus is called the coefficient of non-determination. It gives the proportion of
the variation in the dependent variable that remains unexplained by the independent variables
MSR=
∑ e2
n−k
in the model. As in the case of simple linear regression, is the ratio of the explained
variation to the total variation. Mathematically:

SST =∑ y 2
¿^
SSE=∑ y2=∑ (Y^ i−Ȳ)2=Explained sum ofsquares.¿SSR=∑ y2i =∑ (Y i−Y^ )2=Unexplainedsumof squares.

Or n−1 can also be given in terms of the slope coefficients SST=∑ y =∑(Y −Ȳ) =Totalsumofsquares.
2 2
i

SST=SSE+SSR SSE
4. 34
In simple linear regression, the higher the MSE= k−1 means the better the model is determined by the
explanatory variable in the model. In multiple linear regression, however, every time we insert
SSR
MSR=
additional explanatory variable in the model, the n−k increases irrespective of the improvement
MSR
Fcal= ≈Fα(k−1,n−k)
in the goodness-of- fit of the model. That means high MSE may not imply that the model is
good.
Thus, we adjust the Rej ct H0 if Fcal¿Fα(k−1,n−k)as follows:

F α ( k −1 , n−k )
Where, k = the number of explanatory variables in the model.

In multiple linear regression, therefore, we better interpret the adjusted α than the ordinary
2 ∑
or the unadjusted R , . We have known that the value of ∑ is always between zero and one.
2 y^ 2
R=
y2

But the adjusted ∑ ^y2= R2∑ y2can lie outside this range even to be negative.
2 ∑ e2
In the case of simple linear regression, R =1− ∑ y is the square of linear correlation coefficient. Again
2

as the correlation coefficient lies between -1 and +1, the coefficient of determination ∑
x y=1043.25 1

15
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 ∑ x2 y=−509
lies between 0 and 1. The of multiple linear regression also lies between 0 and +1. The
adjusted ∑
x1 x2=960.6667
however, can sometimes be negative when the goodness of fit is poor. When the
∑ y2=1536.25
adjusted value is negative, we considered it as zero and interpret as no variation of the
dependent variable is explained by regressors.
Confidence Interval Estimation
Confidence interval estimation in multiple linear regression follows the same formulae and
procedures that we followed in simple linear regression. You are, therefore, required to
practice finding the confidence interval estimates of the intercept and the slopes in multiple
regression with two explanatory variables.
∑ e 2i
^^^
% confidence interval for β0=Ȳ−β1X̄−β2X̄=53.2−(0.2536 )−( 0.2139)=75.4012is given as
1401 . 223 1401 .223
¿^ =
(∑ x1 y1)(∑ x2)−(∑x2 y)(∑ x1x2) (1043.25)(2500.667)−(-509)(960.667) 2608821+488979.4
2
¿^ = =
(∑x12)(∑x22)−(∑x1x2)2 (49206.92)(2500.667)−(960.667)2
=
123050121−922880.51
= = =155. 69143
β1 ¿ =
309 780 0.2
=0.02536 5 n−3 12−3 9

Please recall that 100(1- where

12 127 24 1

2
δ ¿
(∑ x2 y)(∑x1)−(∑x1y)(∑x1x2)=(-509)(49206.92)−(1043.25)(960.667)=-2504632-1002216 ¿
2
¿^ =
(∑x12)(∑x22)−(∑x1x2)2 (49206.92)(2500.667)−(960.667)2
123050121−922880.51

β2 =
-2 604 853 8
12 127 24 1
=−0.2132 9 u

k is the number of parameters to be estimated or the number of variables (both dependent and
explanatory)
Interpretation of the confidence interval: Values of the parameter lying in the interval are
¿^

plausible with 100(1- β 1 ¿ % confidence.

Hypothesis Testing in Multiple Regression
Hypothesis testing is important to draw inferences about the estimates and to know how
representative the estimates are to the true population parameter. Once we go beyond the
simple world of the two-variable linear regression model, hypothesis testing assumes several
interesting forms such as the following.
a) Testing hypothesis about an individual partial regression coefficient;
b) Testing the overall significance of the estimated multiple regression model (finding
out if all the partial slope coefficients are simultaneously equal to zero);
c) Testing if two or more coefficients are equal to one another;
d) Testing that the partial regression coefficients satisfy certain restrictions
e) Testing the stability of the estimated regression model over time or in different cross-
sectional units
f) Testing the functional form of regression models.
These and other types of hypotheses tests can be referred from different Econometrics books.
For the case in point, we will confine ourselves to the major ones.
Testing individual regression coefficients
The tests concerning the individual coefficients can be done using the standard error test or
the t-test. In all the cases the hypothesis is stated as:

√ ^
Var ¿¿
¿^
¿^ )= Var(β )=√0.003188=0.056462 ¿ ¿
β2 ¿
1
SE( β1 ¿
a) b)

16
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
In a) we will like to test the hypothesis that X 1 has no linear influence on Y holding other
variables constant. In b) we test the hypothesis that X2 has no linear relationship with Y
holding other factors constant. The above hypotheses will lead us to a two-tailed test however,
one-tailed test might also be important. There are two methods for testing significance of
individual regression coefficients.
a) Standard Error Test: Using the standard error test we can test the above hypothesis.
Thus the decision rule is based on the relationship between the numerical value of
the parameter and the standard error of the same.

[ ]
∑x
2
^¿2 1 =155 .69143(
49206 .92
)=0.0627
u 2 2 −(∑ x x )2 122127241
∑ x ∑x
1 2 12 ¿^
¿^ =

(i) IfVar ( β )
δ

2
¿
¿ , we accept the null hypothesis, i.e. the estimate of β 2 ¿is not statistically
significant.
(r xi x j )2 ¿R2
Conclusion: The coefficient is not statistically significant. In other words, it does not
have a significant influence on the dependent variable.
1
VIF= 2
23
(ii) If ),
we fail to accept H0, i.e., we reject the null hypothesis in favour of the
( 1−r

alternative hypothesis meaning the estimate of i has a significant influence on the dependent
variable.
Generalisation: The smaller the standard error, the stronger is the evidence that the estimates
are statistically significant.
(b) t-test
The more appropriate and formal way to test the above hypothesis is to use the t-
test. As usual we compute the t-ratios and compare them with the tabulated t-
values and make our decision.

Therefore:
r 23
2
23
Decision Rule: accept H0 if r

^
Otherwise, reject the null hypothesis. Rejecting Y i means, the coefficient being
¿

tested is significantly different from 0. Not rejecting β 1 , on the other hand, means
we don’t have sufficient evidence to conclude that the coefficient is different from
0.

17
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
Testing the Overall Significance of Regression Model
Here, we are interested to test the overall significance of the observed or estimated regression
line, that is, whether the dependent variable is linearly related to all of the explanatory
variables. Hypotheses of such type are often called joint hypotheses. Testing the overall
significance of the model means testing the null hypothesis that none of the explanatory
variables in the model significantly determine the changes in the dependent variable. Put in
other words, it means testing the null hypothesis that none of the explanatory variables
significantly explain the dependent variable in the model. This can be stated as:
¿ ∑ x i y i
¿
β 1= bu t y i= Y i− Y an d substit u t e thi s in pla c e of yi
∑ x 2
i
¿ ¿
¿ ∑ x i( Y i −Y ) ∑ xi Y i Y∑ xi
β 1= = − bu t ∑ x i=0
∑ x 2
i ∑ 2
xi ∑ xi
2

¿ ∑ x iY i
β 1=
∑ x 2
i
x i
Le t K i=
∑ x
2
i
¿
β 1= ∑ K i Y i

K i ha s th e follow i n g propert i e s ;

∑ ∑ ∑ x 0
K i= 0 . Th i s is beca u s e K i= = =0
∑ x
2
i ∑ x
2
i
∑ x i X i
¿
∑ K i X i= ∑ K i x i =1 . Th i s is beca u s e ∑ K i X i = bu t x i= X i− X an d substit u t e
∑ x 2
i
in t o th e form u l a
¿ ¿
∑ ( X − X ) X ∑ X 2
X ∑ X ¿
=
i i
=
i
−
i
. Bu t ∑ X =n X an d substit u t e in th e form u l a
∑ 2
xi ∑ 2
xi ∑ xi
2 i

¿ ¿

∑ X
2
n X 2
∑ X
2
i −n X
2
¿ ∑ xi
2
=
i
− = . Bu t ∑ X 2
i −n X 2
=∑ x 2
i = =1
∑ xi
2
∑ x
2
i ∑ x
2
i ∑ xi
2

1 x x 2
∑ x2 1
∑ K 2
i = . Th i s is beca u s e K i=
i
an d K 2
i =
i
. Theref o re , ∑ K 2
i =
i
=
∑ x 2
i ∑ x 2
i ( ∑ x 2
i )2 ( ∑ xi 2
)2 ∑ x 2
i

The test statistic for this test is given by:

¿
β 1=∑ K i Y i . But Y i=β 0 +β 1 X i +U i
=∑ K i (β 0+β 1 X i +U i )
=∑ (β 0 K i +β i K i X i +KU i )
=β0 ∑ K i +β 1∑ K i X i +∑ K i U i

¿
β1 =β 1+∑ K i U i
¿
E(β1 )=E(β1 +∑ K i U i )
= E(β 1 )+E(∑ K i U i )
¿
E(β 1 )=β 1+∑ K i E(U i ). But by theassumptionthat E(U i )=0

¿
E(β 1 )=β 1

Where, k is the number of explanatory variables in the model.

The results of the overall significance test of a model are summarized in the analysis of
variance (ANOVA) table as follows.
¿
Source of Sum of squares Degrees of Mean sum of β1
variation freedom squares
¿ ¿ ¿
¿ ¿ ¿ ¿
Var ( β 1 )=E(( β1 −E( β 1 ) )2 . But we have seen that E( β 1 )=β 1

Regression
¿ ¿
=E( β 1 −β 1 )2 . But β 1 −β 1=∑ K i U i

=E( ∑ K i U i )2 =E( ∑ K i2 X i2 +2 ∑ K i U i U j )
i≠ j

=∑ K i E(U i )+2 ∑ K i E(U i U j ).

2 2

But E (U i U j )=0 and E (U i )2 =δ U2 and is cons tan t by assumption 3

β1
=∑ K i δU

β0
2 2

β0
=δ U2 ∑ K i2 . But ∑ K i2=1 x2
∑ i
¿ δ U2
Therefore , Var ( β 1 )=
∑ x i2

¿
¿
¿ ¿ ¿ ¿ ¿
β 0 =Y − β 1 X . But β1 = ∑ K i Y i

Residual
¿ ¿
=Y − X ∑ K i Y i

β0
¿
1
=∑ ( −X Ki) Y i
n
1 ¿
=∑ (

β0
− X K i )( β 0 + β 1 X i + U i )
n
β0 β1 Xi Ui ¿ ¿ ¿
=∑ ( + + − X β 0 K i − X β1 K i X i− X K i U i )
n n n
∑ β 0 + β 1∑ X i +∑ U i − X ¿ ¿ ¿
β 0 ∑ K i − X β1 ∑ K i X i− X
=
n n n
∑ Ki Ui
nβ 0 ¿ ¿ ¿
= + β 1 X − β1 X − X ∑ K i U i
n
¿ ¿
β 0 = β 0− X ∑ Ki Ui

¿
Total β0 β0

The values in this table are explained as follows:

¿
β 0
These three sums of squares are related in such a way that
¿ ¿ ¿ ¿
1 ¿
Var ( β 0 )= E (( β 0− E ( β 0 ))2
. But β 0 =∑ (
n
− X K i ) Y i
¿
1 ¿
Var ( β 0 )=∑ (
n
− X K i )2
Var (Y i ) . But Var ( Y i )= δ2
U which is cons tan t .
1 ¿
= δ 2
U ∑ (
n
− X K i )2

1 ¿ ¿
=δ 2
U ∑ (
n 2
−2 X K i + X 2
K 2
i )

=δ 2
(
∑ 1
− 2 X ∑
¿
K +
¿
X 2
∑ K 2
U 2 i i
n
2 n ¿
2 1
=δ U ( + X )
n2
∑ x 2
i
¿
2
2 1 X
= δU ( + )
n ∑ xi
2
¿
∑ x2
i +n X 2 ¿
=δ 2
U (
n ∑ 2
x i
) . But ∑ x 2
i + n X 2
=∑ X 2
i

Therefore , Var ( β
¿
)=
δ ∑
2
U X 2
i
0
n ∑ x 2
i

This implies that the total sum of squares is the sum of the explained (regression) sum of
squares and the residual (unexplained) sum of squares. In other words, the total variation in
the dependent variable is the sum of the variation in the dependent variable due to the
variation in the independent variables included in the model and the variation that remained

18
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
unexplained by the explanatory variables in the model. Analysis of variance (ANOVA) is the
technique of decomposing the total sum of squares into its components. As we can see here,
the technique decomposes the total variation in the dependent variable into the explained and
the unexplained variations. The degrees of freedom of the total variation are also the sum of
the degrees of freedom of the two components. By dividing the sum of squares by the
corresponding degrees of freedom, we obtain what is called the Mean Sum of Squares
(MSS).
The Mean Sum of Squares due to regression, errors (residual) and Total are calculated as the
Sum of squares and the corresponding degrees of freedom (look at column 3 of the above
ANOVA table.
The final table shows computation of the test statistic which can be computed as follows:

δU
¿
2
=
∑ e2
i
n −2 [The F statistic follows F distribution]
Y =β 0 + β 1 X +U i

δ U2
i i
¿ ¿ ¿
Y =β 0 + β 1 X +U

¿ ¿ ¿
¿
y i =β 1 xi + U i −U

∑ x 2i E ( β1−β 1 )2. But E( β 1−β 1)2 is the var iance of β 1 which isgiven as
¿
y i =β 1 xi
¿
ei = y i − y i
¿ ¿
ei =β 1 x i +U −U −β 1 xi

∑ x 2i
i
¿ ¿
ei =U i −U −( β1 − β 1 ) xi
¿ ¿ ¿ ¿
2
ei =( U i −U )2
+( β 1 −β 1 )2 2
xi −2 ( β 1− β1 ) xi (U i −U )
¿ ¿ ¿ ¿
∑ 2
ei =∑ (U −U
i )2
+( β 1− β 1 )2
∑ xi2
− 2 ( β1 −β 1 ) ∑ xi ( U i −U )

The test rule: where is the value to be read

¿ ¿ ¿ ¿
=∑ 2
Ui −U ∑ U i +( β 1− β1 )2
∑ xi2
−2 ( β 1− β 1 ) ∑ xi ( U
i− U )
( ∑ Ui ) 2 ¿ ¿ ¿
E ( ∑ 2
ei ) = E (∑ Ui2
− )+∑ xi2
E ( β 1 − β1 )2
−2 E (( β 1 −β1 )∑ x i ( U i −U ) )

δ 2U
n
Now let us see the ex p e ctation o ne by on e .
( ∑ Ui )2 ∑ Ui 2
+2 ∑ Ui U j
E (∑ = E (∑ i≠ j

¿ ∑ x i2
2
Ui − ) Ui2
− ( ) )
n n

=δ U2
=∑ E ( Ui )2
−
∑ E ( U i )2 +2 ∑ E( Ui U j )
n
∑ E( Ui ) 2
+ 0

∑
=∑ δ 2
U −
n

x 2i
=∑ 2
δU −
∑ 2
δU
n
nδ 2
U
=n 2
δ U −
n
=n 2
δ U − δU2

¿ ¿
−2 E((β1 −β1 ) ∑ xi (Ui −U ))
∑ xi U i

from the F- distribution table at a given level.

¿ ¿ ¿
¿−2E((β1−β1 )∑ xi U i−U ∑ xi . But ∑ xi =0 and β1−β1 =∑ Ki Ui =
∑ xi2
∑ i i. xU)
xU
¿−2E( ∑i i
∑ xi2
¿−2E((
∑ xi Ui )2 )
∑ xi2
¿−2E(
∑ xi2 Ui2 +2 i≠∑j xi Ui U j )
∑ xi2 ∑ xi2
∑ xi2 E(Ui2 ) + ∑
2 xi E(Ui U j )
i≠ j
¿−2(
∑ xi2 ∑ xi2
∑ x2 δ2
¿−2( 2i U +0)
∑ xi
∑ x2
¿−2δU2 ( 2i )=−2δ U2
∑ xi

Relationship between F and R2

¿
∑ e2i
^
β ^
^ ∧ β
α
You may recall that δ 2U =
n−k is given by and
We also know that

^
β ^=
β
Σx i y i
Σx 2 =
Σx i ( Y −Ȳ )
=
Σx i Y −Ȳ Σx i
,
Hence, i which means Σx 2i Σx 2i

The formula for F is:

Σxi=∑(X−X̄)=∑X−nX̄=nX̄−nX̄=0
Σx i Y
^=
⇒β 2
Σx i
That means the calculated F can also be expressed in terms of the coefficient of determination.

19
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
Testing the Equality of two Regression Coefficients
Given the multiple regression equation:
x i
2
= K i
Σx i

We would like to test the hypothesis:

^ ^
or ∴ β=ΣKiY−−−−−−−−−−−−−−−−−−−−−−−−−−(2.23)vs. ⇒ β=K1Y1+K2Y2+K3Y3+−−−−+KnY n is not true
(i=1,2,.....n)
The null hypothesis says that the two slope coefficients are equal.
Example: If Y is quantity demanded of a commodity, X1 is the price of the commodity and
X2 is income of the consumer. The hypothesis suggests that the price and income elasticity of
demand are the same.
We can test the null hypothesis using the classical assumption that

^
∴β t distribution with N - K degrees of freedom.
Where K = the total number of parameters estimated.

The α ^ is given as
^ =Σ
α (1 n− X̄ k i ) Y i

Thus the t-statistic is:

^
α
Decision: Reject H0 if tcal. > ttab.

Note: Using similar procedures one can also test linear equality restrictions, for example
^
^ ∧ β
α
and other restrictions.
Illustration: The following table shows a particular country’s the value of imports (Y), the
level of Gross National Product(X1) measured in arbitrary units, and the price index of
imported goods (X2), over 12 years period.
Table 1: Data for multiple regression examples
Yea 196 196 196 196 196 196 196 196 196 196 197 197
r 0 1 2 3 4 5 6 7 8 9 0 1
Y 57 43 73 37 64 48 56 50 39 43 69 60
X1 220 215 250 241 305 258 354 321 370 375 385 385
X2 125 147 118 160 128 149 145 150 140 115 155 152

a) Estimate the coefficients of the economic relationship and fit the model.
To estimate the coefficients of the economic relationship, we compute the entries given in
Table 2

20
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 21
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
Table 2: Computations of the summary statistics for coefficients for data of Table 1
Year Y X1 X2 x1 x2 Y X12 x22 x1y x2y x1x2 y2
1960 57 220 125 -86.5833 -15.3333 3.75 7496.668 235.1101 -324.687 -57.4999 1327.608 14.0625
1961 43 215 147 -91.5833 6.6667 -10.25 8387.501 44.44489 938.7288 -68.3337 -610.558 105.0625
1962 73 250 118 -56.5833 -22.3333 19.75 3201.67 498.7763 -1117.52 -441.083 1263.692 390.0625
1963 37 241 160 -65.5833 19.6667 -16.25 4301.169 386.7791 1065.729 -319.584 -1289.81 264.0625
1964 64 305 128 -1.5833 -12.3333 10.75 2.506839 152.1103 -17.0205 -132.583 19.52731 115.5625
1965 48 258 149 -48.5833 8.6667 -5.25 2360.337 75.11169 255.0623 -45.5002 -421.057 27.5625
1966 56 354 145 47.4167 4.6667 2.75 2248.343 21.77809 130.3959 12.83343 221.2795 7.5625
1967 50 321 150 14.4167 9.6667 -3.25 207.8412 93.44509 -46.8543 -31.4168 139.3619 10.5625
1968 39 370 140 63.4167 -0.3333 -14.25 4021.678 0.111089 -903.688 4.749525 -21.1368 203.0625
1969 43 375 115 68.4167 -25.3333 -10.25 4680.845 641.7761 -701.271 259.6663 -1733.22 105.0625
1970 69 385 155 78.4167 14.6667 15.75 6149.179 215.1121 1235.063 231.0005 1150.114 248.0625
1971 60 385 152 78.4167 11.6667 6.75 6149.179 136.1119 529.3127 78.75022 914.8641 45.5625
Sum 639 3679 1684 0.0004 0.0004 0 49206.92 2500.667 1043.25 -509 960.6667 1536.25
Mean 53.25 306.5833 140.3333 0 0 0

22
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
From Table 2, we can take the following summary results.

α ∧ β
^
θ
θ
^
E( θ)−θ=the amount of bias
The summary results in deviation forms are then given by:

^
θ θ
^
E( θ)−θ=0⇒E( ^
θ)=θ α^ ∧ β^
α ∧ β Ε( β)^ =β and Ε(α^ )=α

^
The coefficients are then obtained as follows.

Ε( β
β
^ )= β .
^
The fitted model is then written as: β=ΣkY i = 75.40512 + 0.025365X1 - 0.21329X2
b) Compute the variance and standard errors of the slopes.
First, you need to compute the estimate of the variance of the random term as follows

=Σk i ( α + βX i +U i )
23
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 =αΣk +βΣk X +Σk u
Variance of i i i i i

but Σki =0 and Σki X i=1

Σx Σ(X−X̄) ΣX−nX̄
Σki= 2i= 2 = 2
Standard error of Σx i Σx i Σx i

n X̄ −n X̄
= =0
Σx 2
i

Variance of
⇒∑ ki=0
Σx i X i Σ ( X − X̄ ) Xi
Σk i X i= =
Σx 2
i Σx 2
i
ΣX2−X̄ ΣX ΣX2−nX̄2
= 2 2 = 2 2=1
Standard error of ΣX −nX̄ ΣX −nX̄

⇒ ∑ k i X i =1.............................
Similarly, the standard error of the intercept is found to be 37.98177. The detail is left for you as
an exercise.
c) Calculate and interpret the coefficient of determination.
We can use the following summary results to obtain the R2.

 yˆ 2
135.0262

β^=β+Σkiui⇒ β^−β=Σkiui− − − − − − − − − − − − −(2.26)

^
Ε( β)=E(β)+Σk i E(ui ), (The sum of the above two). Then,

k i

^
Ε( β )=β
or
d) Compute the adjusted R2.

24
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
 Ε ( u i )= 0
e) Construct 95% confidence interval for the true population parameters (partial regression
coefficients).[Exercise: Base your work on Simple Linear Regression]
f) Test the significance of X1 and X2 in determining the changes in Y using t-test.
The hypotheses are summarized in the following table.

Coefficient Hypothesi Estimate Std. error Calculated t Conclusion

s
1 H0: 1=0
H1: 10
0.025365 0.056462
^
β
We do not
reject H0 since
tcal<ttab
2 H0: 2=0
H1: 20
-0.21329 0.25046
β We do not
reject H0 since
tcal<ttab

The critical value (t 0.05, 9) to be used here is 2.262. Like the standard error test, the t- test revealed
that both X1 and X2 are insignificant to determine the change in Y since the calculated t values
are both less than the critical value.
Exercise: Test the significance of X 1 and X2 in determining the changes in Y using the standard
error test.
g) Test the overall significance of the model. (Hint: use  = 0.05)
This involves testing whether at least one of the two variables X 1 and X2 determine the changes
in Y. The hypothesis to be tested is given by:

^
α
The ANOVA table for the test is give as follows:
Source of Sum of Squares Degrees of Mean Sum of Squares Ε( α^ )=α
variation freedom
Regression
α^ =Σ ( n− X̄ k i ) Y i 1 1 1
1
[ ]i
=Σ (1n−X̄ki)(α+βXi+Ui) Y =α+βX +U
i i =3-1=2
=α+β n ΣXi+ n Σui−α X̄ Σki−β X̄Σki Xi−X̄ Σkiui n Σui−X̄Σkiui
=α+

Ε(α^ )=α− − − − − − − − − − − − − − −(2.28)

Σu i− X̄ Σk i ui =∑( n−X̄ki)uiΕ(^α)=α+1nΣΕ(ui)−X̄Σki Ε(ui)=12-
1 1
Residual ⇒ α−α=
^ n
3=9

Total ∴ α
^ α =12-
1=11

25
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
The tabulated F value (critical value) is F(2, 11) = 3.98
In this case, the calculated F value (0.4336) is less than the tabulated value (3.98). Hence, we do
not reject the null hypothesis and conclude that there is no significant contribution of the
variables X1 and X2 to the changes in Y.
h) Compute the F value using the R2.

^
α and ^
β
Dummy Variable Regression Models
There are four basic types of variables we generally encounter in empirical analysis. These are:
nominal, ordinal, interval and ratio scale variables. In preceding sections, we have encountered
ratio scale variables. However, regression models do not deal only with ratio scale variables; they
can also involve nominal and ordinal scale variables. In regression analysis, the dependent
variable can be influenced by nominal variables such as sex, race, color, geographical region etc.
models where all regressors are nominal (categorical) variables are called ANOVA (Analysis of
Variance) models. If there is mixture of nominal and ratio scale variables, the models are called
ANCOVA (Analysis of Covariance) models. Look at the following example.
Illustration: The following model represents the relationship between geographical location and
teachers’ average salary in public schools. The data were taken from 50 states for a single year.
The 50 states were classified into three regions: Northeast, South and West. The regression
models looks like the following.
α and β
Where Yi = the (average) salary of public school teachers in state i
D1i = 1 if the state is in the Northeast
= 0 otherwise (i.e. in other regions of the country)
D2i = 1 if the state is in the South
= 0 otherwise (i.e. in other regions of the country)
Note that the above regression model is like any multiple regression model considered
previously, except that instead of quantitative regressors, we have only qualitative (dummy)
regressors. Dummy regressors take value of 1 if the observation belongs to that particular
category and 0 otherwise.
Note also that there are 3 states (categories) for which we have created only two dummy
variables (D1 and D2). One of the rules in dummy variable regression is that if there are m
categories, we need only m-1 dummy variables. If we are suppressing the intercept, we can have
m dummies but the interpretation will be a bit different.

26
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES
The intercept value represents the mean value of the dependent variable for the bench mark
category. This is the category for which we do not assign a dummy (in our case, West is a bench
mark category). The coefficients of the dummy variable are called differential intercept
coefficients because they tell us by how much the value of the intercept that receives the value of
1 differs from the intercept coefficient of the benchmark category.
^
α and ^
β
^
α and β^
β ^
p  value (0.000) (0.233) (0.0349) R 2 0.0901

From the above fitted model, we can see that mean salary of public school teachers in the West is
about $26,158.62. The mean salary of teachers in the Northeast is lower by $1734.47 than those
of the West and those in the South is lower by $3264.42. Doing this, we will find the average
salaries in the latter two regions are about $24,424 and $22,894.
In order to know the statistical significance of the mean salary differences, we can run the tests
we have discussed in previous sections. The other results can also be interpreted the way we
discussed previously.

Learning Activities:
Fitting a multiple linear regression model and conducting significance test (both model and
parameter estimates) where data is to be provided

Summary
Multiple regression model is an extension of the two variable regression model with new
concepts involved and more practicality. Such models can be used for the purpose of mean and
individual prediction. This is the simplest possible multiple linear regression model is the three
variable regression model. R2 and adjusted R2 are overall measures of how well the chosen
model fits the data. Hypothesis testing in multiple linear regression models include testing the
individual statistical significance of partial regression coefficients, testing overall significance of
model and others. Dummy variables classify set of samples into subgroups based on qualities or
attributes. It is common to see regression models with dummy variables as regressors. The
coefficients should be interpreted very carefully

27
JIMMA, HARAMAYA, HAWASSA, AMBO, ADAMA, WOLAYITA SODO & SEMERA UNIVERSITIES