Wollo University, Department of Economics

CHAPTER FOUR
MULTIPLE REGRESSION ANALYSIS: INFERENCE AND PREDICTION

Before going to deal about hypothesis testing and interval estimation we need to compute the
mean and variance of the estimates. Since the OLS estimators are normally distributed they are
unbiased estimator of the population parameter and therefore their expected value is equal to the
corresponding population parameter. Given the multiple regression model with two explanatory
variable:
E ( ˆ0 )   0 , E ( ˆ1 )  1 , and E ( ˆ2 )   2 .......... .......... ....( 4.1)

Therefore, the mean of the estimates ˆ0 , ˆ1 andˆ 2 are the corresponding population parameters

 0 , 1 and 2 respectively.

Similarly, we can derive variances and standard errors of these estimates in a manure as follows:

2 1 X 1  X 2i  X 2  X 1i  2 X 1 X 2  X 1i X 2i 
2 2 2 2
ˆ
Var  (  0 )   U   .........( 4.2)
 X 1i  X 2i  ( X 1i X 2i ) 2
2 2
 n 

2  X 2i 
2

Var  ( ˆ1 )   U  2 
.......... .......... .......... .......( 4.3)
  X 1i  X 2i  ( X 1i X 2i ) 
2 2

2  X 1i 
Var  ( ˆ 2 )   U  2 
.......... .......... .......... .......( 4.4)
  X 1i  X 2i  ( X 1i X 2i ) 
2 2

Where  U   U i / n  k , n is number of samples, K is number of parameters.

2 2

The corresponding standard errors are:

Se( ˆ0 )  Var ( ˆ0 ) .......... .......... .......... .......... .......... ....( 4.5)

Se( ˆ1 )  Var ( ˆ1 ) .......... .......... .......... .......... .......... ....( 4.6)

Se( ˆ 2 )  Var ( ˆ 2 ) .......... .......... .......... .......... .......... ....( 4.7)

Econometrics Lecture Notes 1

 Wollo University, Department of Economics

4.1 Testing Hypothesis about a Single Population Parameter: The t-test

In hypothesis testing involving the population parameter  j , the primary interest is laying on

testing the null hypothesis:

H0   j  0

Where j represents any of the k-independent variables

This hypothesis testing is suited for testing simple hypothesis and hence it is called classical
hypothesis testing. As an example, consider the wage equation which is affected by variables
education (educ), experience (exp) and tenure (tenu):
log(wage)   0  1educ   2 exp  3tenu  U .......... .......... .....( 4.8)

From this wage equation, for example, the null hypothesis H 0 :  2  0 means that once
education and tenure have been accounted for, the number of years in work force (exp.) has no
effect on the hourly wage. Note that the alternative hypothesis will reveals the opposite one.

The statistic used to test the null hypothesis (against any alternative hypothesis) is called the t-
statistics or the t-ratio and can be defined as:

t s  t ˆ  ( ˆ j   j ) / Se(  j )
j

 ( ˆ j  0) / Se(  j )

 ˆ j / Se(  j )......... .......... .......... .......... .......... .......( 4.9)

Testing Other Hypothesis About  j

Although H 0 :  j  0 is the most common hypothesis, we sometimes want to test whether  j is equal

to some other given constant. Two common examples are  j  1 and  j  1 , generally, if the null

hypothesis is stated as:

H0 :  j  a j

Where a j is our hypothesized value of  j , then the appropriate t-statistics is:

Econometrics Lecture Notes 2

 Wollo University, Department of Economics

(estimate  hypothesized value)

tS 
s tan dard error of the estimate
( ˆ j  a j )
 .......... .......... .......... .......... .......... .......... .......... .......( 4.10)
Se( ˆ j )

Note that the computation of t-critical and the decision rules are exactly the same as before for
the respective types of tailed tests. The only difference is the computation of t-statistics.

In addition to the t-statistics, the theoretical or critical value of t, tc, at some significant level, 
and degree of freedom, n-k-1, is required so as to determine the decision rule for rejecting or
accepting the null hypothesis.

The critical values of t are directly obtained at the t-distribution table using the level of
significance and the degree of freedom. As the degrees of freedom in the t-distribution get large,
the t-distribution approaches to the normal distribution. For example, when n-k-1=120, the 5%
critical value for one tailed test is 1.658 compared with the standard normal value of 1.645. these
are close enough for practical purposes; for degrees of freedom greater than 120, one can use the
standard normal critical values.

Some treatment of regression analysis define the t-statistics as the absolute value so that the t-
statistics is always positive. However, this practice has the draw backs of making testing against
one-sided alternative clumsy. Therefore, throughout this test for one sided case, the t-statistics
always has the same sign as the corresponding OLS coefficient estimate.

In order to determine the decision rule for rejecting or accepting the null hypothesis H 0 :  j  0 ,

we need to make the relevant alternative hypothesis. There are two alternatives in which the
alternative hypothesis is formulated.

Econometrics Lecture Notes 3

 Wollo University, Department of Economics

i. One-Sided Alternative
Positive Alternative: it is the right tailed test in which the hypothesized population parameter  j is

positive so that alternative hypothesis is formulated as: H 0 :  j  0 . Since  j is greater than zero, the t-

statistics is expected to be positive.

Decision rule:
If ts > tc, then reject H 0

If ts < tc, then accept H 0

Negative Alternative: it is the left-tailed test in which the hypothesized population parameter is negative
so that the alternative hypothesis is formulated as: H1 :  j  0 . Since  j is less than zero, but the t-

statistics is expected to be negative. Thus, to compare this negative t-statistics with the t-critical value we
must get the negative t-critical though it is always positive on the table.
Decision rule:
If ts < -tc, then reject H 0

If ts > -tc, then accept H 0

Illustration
Suppose that the estimated wage equation with standard errors in parentheses, is:
log(wagˆe)  2.274  0.00046educ  0.48 exp 0.0002ten
(6.113) (0.00010) (0.40) (0.00022)......... .......... .........( 4.11)
n  408, R 2  0.0541
Test the significance of education (for right tailed test) and the significance of tenure (for left
tailed) at 5% of significant level.
 Significance of education;

t s  ˆ1 / se( ˆ1 )  0.00046/ 0.0001  4.6

t c  t ,n k 1  t 0.05, 404  1.645

(K represents the number of slope parameters)

H0
 Since ts > tc we reject so that education is statistically significant in determining wage.
 Significance of tenure:

Econometrics Lecture Notes 4

 Wollo University, Department of Economics

t s  ˆ3 / se( ˆ3 )  0.0002 / 0.00022  0.91

t c  t ,n k 1 , t 0.05, 404  1.645

 Since ts > -tc we accept H 0 so that tenure is not statistically significant in determining wage.

ii. Two-Sided Alternatives

This refers to the two tailed test in which the hypothesized population parameter is either
positive or negative but different from zero. Therefore, the alternative hypothesis is formulated
as:
H1 :  j  0
When the alternative is two-sided we are interested in the absolute value of t-statistics. Thus the decision
rule for the null hypothesis H 0 :  j  0 against the alternative hypothesis is:

If /ts/ > tc, then reject H 0

If /ts/ < tc, then accept H 0

Note that to find the t-critical value for two-tailed test the significant level α has to be divided by
2 so as to share the significance level for each tail of the t-distribution equally. For the sake of
illustration we can test the significance of experience in the earlier estimated wage equation
(4.11).

t s  ˆ 2 / se( ˆ 2 )  0.48 / 0.4  1.2

t s  t / 2,n k 1 , t 0.025, 404  1.96

 Since /ts/ < tc we accept H 0 and hence experience is not statistically significant in determining wage at

5% level of significance.

Econometrics Lecture Notes 5

 Wollo University, Department of Economics

Computing p-values for t-test

So far, we have see hypothesis testing of a single parameter using t-test by comparing the critical
and computed values of t at different significance level and degree of freedom, Rather than
testing at different significance levels, we can use the p-values for testing the parameter. The p-
value refers to the smallest probability of rejecting the null hypothesis when it is true. Implying,
it is the smallest significance level at which the null hypothesis would be rejected.

We obtain the p-value by computing the probability that at random variable with certain degree
of freedom. Thus, we reject the null-hypothesis when the value of t distributed random variable
denoted by T with n-k-1 degree of freedom is larger than the value of t-statistics in absolute
value, /t/.
P(/T/ > /t/)…………………………………….(3.44)
Therefore, p-value is the significance level of test when we use the value of t-statistics as the
critical value for test.

However, in most cases the p-value is computed using the econometric package STATA but not
available in table. Once the p-values have been computed, a classical test can be carried out at
any desired level. If  denotes the significance level of the test (in decimal form), then H 0 is

rejected if p-value <  .

Computing p-value for one-sided alternatives is also quite simple. Although some regression
packages only compute p-values for two-sided alternatives, the p-values can be obtained by just
dividing the two-sided p-value by 2.

Illustration
Suppose you estimate a regression model and obtain a p-value = 0.086 for a particular parameter estimate

ˆ1 and given the 5% of significance level test the significance of the parameter  1 at one-tailed and two
tailed alternatives.

Econometrics Lecture Notes 6

 Wollo University, Department of Economics

For two-tailed test or classical test we reject H 0 , since P / 2   while for one-tailed test

(i.e, H 0 : 1  0; H 1 : 1  0) we accept H 0 , since p   .

Note:
Note: (1) when H 0 is not rejected we prefer to use the language “we fail to reject H 0 at x%

level,” rather than “ H 0 is accepted at x% level. (2) The statistical significance is distincted from

economic or practical significance. That is, the statistical significance of a variable X j is

determined entirely by the size of t-statistics, where as the economic/practical significance of a

variable is related to the size (and sign) of ˆ j .

Confidence Interval
Confidence interval also called interval estimates for the population provide a range or likely values for
the population parameter  j . For example, a 95% confidence interval (CI) for the unknown  j is given

by:

ˆ j  t / 2 . Se( ˆ j )   j  ˆ j  t / 2 . Se( ˆ j )......... ...(3.45)

From this CI the lower and upper bounds of the confidence interval are

 j  ˆ j  t / 2 . Se( ˆ j ) and  j  ˆ j  t / 2 . Se( ˆ j ) respectively

As an example, for df = n-k-1 = 25 a 95% CI for ˆ j which is assumed to be -0.056 (its estimated value)

with Se( ˆ j )  0.02 is given by:

- 0.056 - t 0.025, 25 .Se(  j )   j  -0.056 + t 0.025,25 .Se(  j )

- 0.056 - (2.06) (0.02)   j  -0.056 + (2.06) (0.02).............. .......... .......... .......... .(3.46)
- 0.0972   j  - 0.0148

Econometrics Lecture Notes 7

 Wollo University, Department of Economics

The lower & upper bounds of the CI are -0.0972 & 0.0148 respectively so that the unknown
population parameter  j whose estimated value is – 0.056 would lie in the interval (-0.0972,-

0.0148) for 95% samples. That is, if 100 samples of size 29 are selected and 100 confidence
intervals like (3.46) are constructed, we expect 95 of them to contain the true population
parameter  j .

3.5.3 Overall Significance Hypothesis Test: The F-Test

Throughout the previous section we were concerned with testing the significance of a single
parameter to point out the partial effect of individual parameters on the dependent variable. But
now we try to point out the joint effect of one or more coefficients in the multiple regression
model using F-test. For the sake of simplicity let’s take the simplified form of multiple
regression with three variables (i.e. with two independent variables) by considering the following
null hypothesis:
H 0 : 1   2  0

This null hypothesis is a joint hypothesis that 1 and  2 are jointly of simultaneously equal to zero. A
test of such a hypothesis is called overall significance hypothesis test.

Unlike testing individual hypothesis using the t-test, two or more parameters are tested jointly
using F-test in which its value is calculated through the analysis of variance technique
(ANOVA). That is, computing the TSS, ESS and RSS first with their respective degree of
freedoms as n-1, k-1, n-k where n is the total number of observations and k is the total number of
parameters to be estimated including the intercept term.
Thus, F is computed using the formula:
ESS / k  1
FS 
RSS / n  k
Once the F-statistics (FS) is compute using the above formula we can test the overall significance
by comparing this calculated value with its critical value (FC) obtained from the F-table at the
numerator degree of freedom (k-1) and denominator degree of freedom (n-k), at α level of
significance. Therefore, the decision rule will be:
If Fs > Fc, then reject H 0

Econometrics Lecture Notes 8

 Wollo University, Department of Economics

If Fs < Fc, then failed to reject H 0

As an example of testing the overall significance, consider the following model to explain
students score interms of factors such as class size (classize) and teachers’ compensation
(tchcomp).
Score   0  1classize   2 tchcomp

Given 20 number of observations and the TSS &ESS are 363 and 257 respectively, test the
statistical significance of the slope parameters jointly at 1% level of significance.
ESS / k  1 257 / 3  1 257 / 2
FS     20.60
RSS / n  k 106 / 20  3 106 / 17
FC  F ,ndf ,ddf  F0.01, 2,17  6.11
Since Fs > Fc we reject H0 so that the factors class size and teachers compensation are
statistically significant in affecting students performance.

Relationship between R2 and F

There is an intimate relationship between the multiple coefficient of determination R2 and the
overall test F-test. In this regard, the R2 from of the formula expression for F is produced as
follow:
ESS / k  1
F 
RSS / n  k
ESS (n  k )
 .
RSS (k  1)
ESS / TSS (n  k )
 .
RSS / TSS (k  1)
R 2 (n  k ) ESS RSS
 , R2  , 1 R2  .......... .......... .......... .......... .....( 3.47)
1  R 2 (k  1) TSS TSS
Equation 3.47 shows that R2 & F are related and both vary directly. That is, the larger the R 2 ,
the greater the F-value.

Thus, the F-test, which is a measure of the overall significance of the estimated regression is
equivalent to a test of significance of R2. Implying that testing the null hypothesis
H 0 : 1   2  0 using F-test is equivalent to testing the null hypothesis H0 : R2 = 0.

Econometrics Lecture Notes 9

 Wollo University, Department of Economics

Note that there are two types of models in this lesson: unrestricted and restricted models. The
unrestricted model is a model with potential k-independent variables (and k+1 number of
parameters: adding one for the intercept) where as the restricted model is a model with less than
k-independent variables due to drop out of some independent variables. Accordingly, the
unrestricted and restricted models have the unadjusted R-squared, R2 and the adjusted R-squared,
R 2 respectively.

The adjusted R-squared is computed as:

RSS / n  k
R 2  1
TSS / n  1
RSS (n  1)
 1 .
TSS (n  k )
TSS  ESS (n  1)
 1 .
TSS (n  k )
TSS ESS (n  1)
 1 (  ).
TSS TSS (n  k )
 1  (1  R 2 ).(n  k ), ESS  R 2 .......... .......... .......... .......... ...(3.48)
k 1 TSS

It is R 2 but not R2 which helps to test the significance for exclusion of some variables from the
unrestricted model. In other words, the overall significance of the unrestricted model (with more
explanatory variables) is compared with the overall significance of restricted model (with fewer
explanatory variables) using the adjusted R-squared.

Computing P-values for F-test

Since the F-distribution depends on the numerator and denominator degree of freedoms, it is
difficult to come up with a reliable decision simply by looking at the statistical & critical values
of F. Therefore P-value is useful in reporting the outcome of F-tests.
In F-testing context, the P-value is defined as:
P-value = P (f > F)……………………………… (3.49)
Where f refers and F random variable with (k-1 , n-k) d.f and F refers to the actual value of the
test statistics. The p-value still has the same interpretation as it did for t-statistics: it is the
probability of rejecting H0 when it is true so that a small P-value is an evidence to reject H0

Econometrics Lecture Notes 10

 Wollo University, Department of Economics

comparing with level of significance. As with t-testing, once the p-value has been computed, the
F-test can be carried out at any significance level.
 If p < α, then reject H0
 If p > α, then fail to reject H0

Econometrics Lecture Notes 11