Chapter two

Simple Linear Regression

Regression analysis is concerned with the study of the dependence of one variable, the

dependent variable, on one or more other variables, the explanatory variables, with a view to

estimating and/or predicting the (population) mean or average value of the former in terms of

the known or fixed (in repeated sampling) values of the latter.

1
Simple Linear Regression Model
 Stochastic relationship with one explanatory variable is
called simple linear regression model.

Yi     xi  ui
    
the dependent var iable the regression line randomvar iable

2
Conti.. Graphically

3
Conti..
 The scatter of observations represents the true relationship

between Y and X.
 The line represents the exact part of the relationship and the
deviation of the observation from the line represents the random
component of the relationship.

4
cont...
The first component in the bracket is the part of Y explained
by the changes in X and the second is the part of Y not
explained by X, that is to say the change in Y is due to the
random influence term.

5
Ui
Assumptions of the Classical Linear Stochastic Regression Model

 The classical economists made important assumption in their

analysis of regression

A. The model is linear in parameter

B. Ui is a random real variable

C. The mean value of the random variable(U) in any particular period is zero

D. The variance of the random variable(U) is constant in each period (The

assumption of homoscedasticity)

E. The random variable (U) has a normal distribution

6
Conti..
F) The random terms of different observations are independent. (The
assumption of no autocorrelation)

G) The set of fixed values in the hypothetical process of repeated sampling

which underlies the linear regression model.

H) The random variable (U) is independent of the explanatory variables.

I) The explanatory variables are measured without error

7
 A) model is linear in parameter

classical assumed that the model should be linear in the parameters

regardless of whether the explanatory and the dependent variables
are linear or not.
This is because if the parameters are non-linear it is difficult to
estimate them since their value is not known
 but you are given with the data of the dependent and independent
variable.
8
 .

i
Conti…
Y    x  u
Example 1.

is linear in both parameters and the variables, so it satisfies the

assumption.
ln Y    ln x  u
2.

is linear only in the parameters. Since the classical worry on

the parameters, the model satisfies the assumption.

9
B) U is real random variables
i

This means that the value which u may assume in any one period

depends on chance;

 it may be positive, negative or zero.

 Every value has a certain probability of being assumed by u in any

particular instance.
10
C) The mean value of the random variable(U) in any particular period is zero

This means that for each value of x, the random variable(u) may assume various
values, some greater than zero and some smaller than zero,

 But if we considered all the possible and negative values of u, for any given value
of X, they would have on average value equal to zero. In other words the positive
and negative values of u cancel each other.
Mathematically, EQ (2.3)

11
D) The variance of the random variable(U) is constant in
each period (The assumption of homoscedasticity )

12
Conti…
 For all values of X, the u’s will show the same dispersion
around their mean.
 In Fig.2.c this assumption is denoted by the fact that the
values that u can assume lie with in the same limits,
irrespective of the value of X.
 For , u can assume any value with in the range AB;

 For , u can assume any value with in the range CD which is

equal to AB and so on.
13
Conti…
Mathematically;

2 2 2
Var(U i ) E[U i  E (U i )] E (U i ) 
SINCE

E (U i ) 0
This constant variance is called homoscedasticity assumption and the
constant variance itself is called homoscedastic variance.
14
Conti…
• This means the values of u (for each x) have a bell
shaped symmetrical distribution about their zero mean
and constant variance , i.e.

U i N (0,  2 ) ………………………………………..……2.4

15
E) The random terms of different observations are
independent. (The assumption of no autocorrelation)
This means the value which the random term assumed in one period
does not depend on the value which it assumed in any other period

 E (ui u j ) 0 …………………………..…. (2.5)

16
F) The set of fixed values in the hypothetical
process of repeated sampling which underlies
the linear regression model.

This means that, in taking large number of samples on Y

and X, the values are the same in all samples, but the
values do differ from sample to sample, and so of course

do the values of .

17
F) The random variable (U) is
independent of the explanatory variables.
 This means there is no correlation between the random
variable and the explanatory variable.
 If two variables are unrelated their covariance is zero.

18
Conti…
Proof:-
cov(XU ) [( X i  ( X i )][U i  (U i )]

[( X i  ( X i )(U i )] given E (U i ) 0

( X iU i )  ( X i )(U i )

( X iU i )

 X i (U i ) , given that the xi are fixed

0
19
G) The explanatory variables are measured without error

 U absorbs the influence of omitted variables and

possibly errors of measurement in the y’s.

 i.e., we will assume that the regressors are error free,
while y values may or may not include errors of
measurement.

20
We can now use the above assumptions to derive the following basic concepts.

A. The dependent variable is normally distributed.

i.e.  Yi ~ N(   x i ),  2  ……………… (2.7)

Proof:
Mean: (Y )    xi  u i 
   X i Since (u i ) 0

Variance: Var (Yi ) Yi  (Yi ) 2

   X i  u i  (   X i ) 
2

(u i ) 2

 2 (since (u i ) 2  2 )

 var(Yi )  2 ………………….(2.8)

21
Conti..
The shape of the distribution of Yi is determined by the shape of the
distribution of ui which is normal by assumption 4.
Since α and β, being constant, they don’t affect the distribution of
Y.i

 Furthermore, the values of the explanatory variable, are a set of fixed

values by assumption 5 and therefore don’t affect the shape of the
distribution .
22
 Methods of estimation
Specifying the model and stating its underlying assumptions are the first stage of any
econometric application.
The next step is the estimation of the numerical values of the parameters of economic
relationships.
The parameters of the simple linear regression model can be estimated by various
methods. Three of the most commonly used methods are:

1. Ordinary least square method (OLS)

2. Maximum likelihood method (MLM)

3. Method of moments (MM)

23
 .

The ordinary least square (OLS) method

 The model is called the true relationship between Y and X because Y

and X represent their respective population value are called the true
parameters since they are estimated from the population value of Y and
X.
 But, it is difficult to obtain the population value of Y and X.

because of technical or economic reasons. So we are forced to take the

sample value of Y and X.
 The parameters estimated from the sample value of Y and X are called
the estimators of the true parameter and are symbolized as and .
24
Conti…
 The model is called estimated relationship between Y
and X since are estimated from the sample of Y and X and
represents the sample counterpart of the population
The ˆ  ˆX i  ei , is called estimated relationship between Y and X since
model Yidisturbance
random .
ˆ and ˆ are estimated from the sample of Y and X and ei represents the sample
counterpart of the population random disturbance U i .
Estimation of  and  by least square method (OLS) or classical least square

(CLS) involves finding values for the estimates ˆ and ˆ which will minimize the
sum of square of the squared residuals (  ei2 ).

25
Conti..
From the estimated relationship Yi ˆ  ˆX i  ei , we obtain:

ei Yi  (ˆ  ˆX i ) …………………………… (2.6)

ˆX ) 2 ………………………. (2.7)
 i  i
e 2
 (Y  
ˆ   i

To find the values of ˆ and ˆ that minimize this sum, we have to partially

differentiate  i with respect to ˆ and ˆ and set the partial derivatives equal to
e 2

zero.
 ei2
1.  2 (Yi  ˆ  ˆX i ) 0.......................................................( 2.8)
ˆ

Rearranging this expression we will get: Y i n  ˆX i …… (2.9)

If you divide (2.9) by ‘n’ and rearrange, we get

ˆ Y  ˆX ..........................................................................(2.10) 26
Conti..
 ei2
2.  2 X i (Yi  ˆ  ˆX ) 0..................................................(2.11)
ˆ

Note: at this point that the term in the parenthesis in equation 2.8and 2.11 is the
residual, e Yi  ˆ  ˆX i . Hence it is possible to rewrite (2.8) and (2.11) as

 2 ei 0 and  2 X i ei 0 . It follows that;

e i 0 and X e i i 0............................................(2.12)

If we rearrange equation (2.11) we obtain;

ˆX 2 ……………………………………….(2.13)
 i i
Y X ˆ X i   i

27
Conti..
Equation (2.9) and (2.13) are called the Normal Equations. Substituting the values
of ̂ from (2.10) to (2.13), we get:
ˆX )  ˆX 2
 i i
Y X   X i (Y   i

Y X i  ˆXX i  ˆX i2

ˆ (X 2  XX )
 i i
Y X  Y  X i   i i

XY  nXY = ̂ ( X i2  nX 2)

XY  nXY
ˆ  2 ………………….(2.14)
 X i  nX 2

28
Conti…
Equation (2.14) can be rewritten in somewhat different way as follows;
( X  X )(Y  Y ) ( XY  XY  XY  XY )

XY  Y X  XY  nXY

XY  nY X  nXY  nXY

( X  X )(Y  Y ) XY  n X Y               (2.15)

( X  X ) 2 X 2  nX 2                  (2.16)

Substituting (2.15) and (2.16) in (2.14), we get

( X  X )(Y  Y )
ˆ 
( X  X ) 2

29
Conti…
Now, denoting ( X i  X ) as xi , and (Yi  Y ) as yi we get;

 x y
ˆ  2 i ……………………………………… (2.17)
i

xi

The expression in (2.17) to estimate the parameter coefficient is termed is the

formula in deviation form.

30
 Estimation of a function with zero intercept
Suppose it is desired to fit the line Yi   X i  U i , subject to the restriction  0.

to estimate ̂ , the problem is put in a form of restricted minimization problem and

then Lagrange method is applied.
n
We minimize: ei2  (Yi  ˆ  ˆX i ) 2
i 1

Subject to: ˆ 0
31
Conti…
The composite function then becomes
Z  (Yi  ˆ  ˆX i ) 2  ˆ , where  is a Lagrange multiplier.

We minimize the function with respect to ˆ , ˆ , and 

Z
 2(Yi  ˆ  ˆX i )   0         (i )
ˆ
Z
 2(Yi  ˆ  ˆX i ) ( X i ) 0         (ii )
ˆ

z
 2 0                    (iii )

32
Conti…
Substituting (iii) in (ii) and rearranging we obtain:
X i (Yi  ˆX i ) 0

Yi X i  ˆX i 0
2

ˆ  X i Yi ……………………………………..(2.18)
X i2
This formula involves the actual values (observations) of the variables and not their
deviation forms, as in the case of unrestricted value of ̂ .

33
Statistical Properties of Least Square Estimators

There are various econometric methods with which we may

obtain the estimates of the parameters of economic
relationships.
 We would like to an estimate to be as close as the value of
the true population parameters i.e. to vary within only a small
range around the true parameter.

34
Conti….
How are we to choose among the different econometric
methods, the one that gives ‘good’ estimates?
We need some criteria for judging the ‘goodness’ of an
estimate.

35
Conti…
Closeness’ of the estimate to the
population parameter is measured by the
mean and variance or standard deviation
of the sampling distribution of the
estimates of the different econometric
methods.
36
Cont..
We assume the usual process of repeated
sampling i.e. we assume that we get a very
large number of samples each of size ‘n’.
we compute the estimates' from each sample,
and for each econometric method and we form
their distribution.

37
Conti….
We next compare the mean (expected value)
and the variances of these distributions and we
choose among the alternative estimates the one
whose distribution is concentrated as close as
possible around the population parameter.

38
 PROPERTIES OF OLS ESTIMATORS

The ideal or optimum properties that the OLS estimates

possess may be summarized by well known theorem known
as the Gauss-Markov Theorem.
Statement of the theorem: “Given the assumptions of the classical
linear regression model, the OLS estimators, in the class of linear and
unbiased estimators, have the minimum variance, i.e. the OLS estimators
are BLUE.
39
Conti…
 According to the this theorem, under the basic assumptions of

the classical linear regression model, the least squares

estimators are linear, unbiased and have minimum variance (i.e.

are best of all linear unbiased estimators).

 Some times the theorem referred as the BLUE theorem i.e.

Best, Linear, Unbiased Estimator. An estimator is called BLUE

if:

40
A)Linear:
a linear function of the a random variable, such as, the dependent
variable Y.

 ̂  K1Y1  K 2Y2  K 3Y3      K nYn

 ̂ is linear in Y

where

41
B. Unbiasedness
Proposition: ˆ & ˆ are the unbiased estimators of the true parameters  & 

From your statistics course, you may recall that if ˆ is an estimator of  then
E (ˆ)   the amount of bias and if ˆ is the unbiased estimator of  then bias =0 i.e.

E (ˆ)   0  E (ˆ) 

In our case, ˆ & ˆ are estimators of the true parameters  &  .To show that they
are the unbiased estimators of their respective parameters means to prove that:
( ˆ )  and (ˆ ) 
42
 C) Minimum variance:

• It has a minimum variance in the class of linear and unbiased

estimators. An unbiased estimator with the least variance is known
as an efficient estimator.

43
 Statistical test of Significance of the OLS Estimators
(First Order tests)

After the estimation of the parameters and the determination of the least

square regression line, we need to know how ‘good’ is the fit of this line to the

sample observation of Y and X, that is to say we need to measure the dispersion

of observations around the regression line.

This knowledge is essential because the closer the observation to the line, the

better the goodness of fit, i.e. the better is the explanation of the variations of Y

by the changes in the explanatory variables.

44
Cont.….
 We divide the available criteria into three groups: the
theoretical a priori criteria, the statistical criteria, and the
econometric criteria.
 Under this section, our focus is on statistical criteria (first
order tests). the two most commonly used first order tests in
econometric analysis are

45
Cont.…..
A) The coefficient of determination (the square of the correlation
coefficient i.e. R2). This test is used for judging the explanatory
power of the independent variable(s).

B) The standard error tests of the estimators. This test is used for
judging the statistical reliability of the estimates of the regression
coefficients.

46
 Tests of the ‘ Goodness of fit’ with R2
A. R2 shows the percentage of total variation of the dependent variable that can be
explained by the changes in the explanatory variable(s) included in the model.

B. To elaborate this let’s draw a horizontal line corresponding to the mean value of
the dependent variable (see figure‘d’ below).

By fitting the line Yˆ ˆ 0  ˆ1 X we try to obtain the explanation of the variation of
the dependent variable Y produced by the changes of the explanatory variable X.

47
Conti…

.Y
Y = e Y  Yˆ
YY = Yˆ Yˆ ˆ 0  ˆ1 X

= Yˆ  Y
Y.

X
Figure ‘d’. Actual and estimated values of the dependent variable Y 48
Conti……
As can be seen from fig.(d) above, represents measures the variation of the

sample observation value of the dependent variable around the mean.

However the variation in Y that can be attributed the influence of X, (i.e. the

regression line) is given by the vertical distance .

The part of the total variation in Y about that can’t be attributed to X is equal to

which is referred to as the residual variation.

 e which is referred to as the residual variation.

49
Conti…
In summary:
ei Yi  Yˆ = deviation of the observation Yi from the regression line.

yi Y  Y = deviation of Y from its mean.

yˆ Yˆ  Y = deviation of the regressed (predicted) value ( Yˆ ) from the mean.

50
Conti…
Now, we may write the observed Y as the sum of the predicted value ( Yˆ ) and the
residual term (ei.).
Yi  Yˆ  ei
 predicted Yi

Observed Yi Re sidual

From equation (2.34) we can have the above equation but in deviation form
y  yˆ  e . By squaring and summing both sides, we obtain the following expression:

51
Conti…
y 2 ( yˆ 2  e) 2

y 2 ( yˆ 2  ei2  2 yei )

2
y i  ei2  2yˆei

But ŷei = e(Yˆ  Y ) e(ˆ  ˆxi  Y )

ˆei  ˆexi  Yˆei

(but ei 0, exi 0 )

  yˆe 0 ………………………………………………(2.46)
52
Conti….
Therefore;
y i2  yˆ 2  ei2 ………………………………...(2.47)
 
Total Explained Un exp lained
var iation var iation var ation

OR,
Total sum of Explained sum Re sidual sum
 
square of square of square
                
TSS ESS RSS

i.e
TSS  ESS  RSS ……………………………………….(2.48)
53
Conti…
Mathematically; the explained variation as a percentage of the total variation is
explained as:
ESS yˆ 2
 2 ……………………………………….(2.49)
TSS y

From equation (2.37) we have yˆ ̂x . Squaring and summing both sides give us

yˆ 2 ˆ 2 x 2                        (2.50)

We can substitute (2.50) in (2.49) and obtain:

ˆ 2 x 2
ESS / TSS  2
…………………………………(2.51)
y
54
CONTI…
2 2
  xy   x ˆ x i y i
 2  i
2
, Since   2
 x  y x i

xy xy
 2 2 ………………………………………(2.52)
x  y

Comparing (2.52) with the formula of the correlation coefficient:

r = Cov (X,Y) / x2x2 = xy / nx2x2 = xy / ( x 2 y 2 )1/2 ………(2.53)

Squaring (2.53) will result in: r2 = ( xy )2 / ( x 2 y 2 ). ………….(2.54)

55
Conti….
Comparing (2.52) and (2.54), we see exactly the expressions. Therefore:
xy xy
ESS/TSS  2 2 = r2
x y

From (2.48), RSS=TSS-ESS. Hence R2 becomes;

2
TSS  RSS RSS  e
R2  1  1  i2 ………………………….…………(2.55)
TSS TSS y
From equation (2.55) we can drive;
RSS ei2 y i2 (1  R 2 )                             (2.56)

The limit of R2: The value of R2 falls between zero and one. i.e. 0 R 2 1 .
56
 Interpretation of R2

A) suppose , this means that the regression line gives a good fit
to the observed data since this line explains 90% of the total
variation of the Y value around their mean.

B) The remaining 10% of the total variation in Y is

unaccounted for by the regression line and is attributed to the
factors included in the disturbance variable

57
B) TESTING THE SIGNIFICANCE OF OLS
PARAMETERS
To test the significance of the OLS parameter estimators we need the following:
 Variance of the parameter estimators
 Unbiased estimator of  2

 The assumption of normality of the distribution of error term.

We have already derived that:
58
Conti….
ˆ 2
 var( ˆ ) 
x 2
ˆ 2 X 2
 var(ˆ ) 
n x 2
2
 e RSS
  
ˆ 2

n 2 n 2
For the purpose of estimation of the parameters the assumption of normality is not
used, but we use this assumption to test the significance of the parameter estimators;
because the testing methods or procedures are based on the assumption of the
normality assumption of the disturbance term.
59
Cont.…
Hence before we discuss on the various testing methods it is important to see
whether the parameters are normally distributed or not.
We have already assumed that the error term is normally distributed with mean zero
and variance  2 , i.e. U i ~ N(0, 2 ) . Similarly, we also proved that

Yi ~ N[(  x),  2 ] . Now, we want to show the following:

ˆ   2

1.  ~ N  , 2 
 x 

  2 X 2 
2. ˆ ~ N  , 
2 
 n x 
60
Cont.…
To show whether ˆ and ˆ are normally distributed or not, we need to make use of
one property of normal distribution. “........ any linear function of a normally
distributed variable is itself normally distributed.”
ˆ k i Yi k1Y1  k 2 Y2i  ....  k n Yn

ˆ wi Yi w1Y1  w2 Y2i  ....  wn Yn

Since ˆ and ˆ are linear in Y, it follows that

ˆ   2
   2 X 2 
 ~ N  , 2  ; ˆ ~ N  , 2 

 x   n x  61
 Conti…
The OLS estimates ˆ and ˆ are obtained from a sample of observations on Y and
X. Since sampling errors are inevitable in all estimates, it is necessary to apply test
of significance in order to measure the size of the error and determine the degree of
confidence in order to measure the validity of these estimates. This can be done by
using various tests. The most common ones are:
i) Standard error test ii) Student’s t-test iii) Confidence interval

62
 Conti…
All of these testing procedures reach on the same conclusion. Let us now see these
testing methods one by one.
i) Standard error test
This test helps us decide whether the estimates ˆ and ˆ are significantly different
from zero, i.e. whether the sample from which they have been estimated might have
come from a population whose true parameters are zero.  0 and / or  0 .
Formally we test the null hypothesis
H 0 :  i 0 against the alternative hypothesis H 1 :  i 0

The standard error test may be outlined as follows.

First: Compute standard error of the parameters.
63
Conti…
SE ( ˆ )  var( ˆ )

SE (ˆ )  var(ˆ )

Second: compare the standard errors with the numerical values of ˆ and ˆ .
Decision rule:
 If SE ( ˆi )  1 2 ˆi , accept the null hypothesis and reject the alternative

hypothesis. We conclude that ̂ i is statistically insignificant.

 If SE ( ˆi )  1 2 ˆi , reject the null hypothesis and accept the alternative

hypothesis. We conclude that ̂ i is statistically significant.

64
Conti…
The acceptance or rejection of the null hypothesis has definite economic meaning.
Namely, the acceptance of the null hypothesis  0 (the slope parameter is zero)
implies that the explanatory variable to which this estimate relates does not in fact
influence the dependent variable Y and should not be included in the function, since
the conducted test provided evidence that changes in X leave Y unaffected. In other
words acceptance of H 0 implies that the relation ship between Y and X is in fact
Y   (0) x  , i.e. there is no relationship between X and Y.
Numerical example: Suppose that from a sample of size n=30, we estimate the
following supply function.
Q  120  0.6 p  ei
SE : (1.7) (0.025)
65
 Conti…
Test the significance of the slope parameter at 5% level of significance using the
standard error test.
SE ( ˆ ) 0.025

( ˆ ) 0.6

1
2 ˆ 0.3

This implies that SE ( ˆi )  1 2 ˆi . The implication is ̂ is statistically significant at

5% level of significance.
Note: The standard error test is an approximated test (which is approximated from
the z-test and t-test) and implies a two tail test conducted at 5% level of significance.
66
Cont.…
i) Standard error test
This test helps us decide whether the estimates ˆ and ˆ are significantly different
from zero, i.e. whether the sample from which they have been estimated might have
come from a population whose true parameters are zero.  0 and / or  0 .
Formally we test the null hypothesis
H 0 :  i 0 against the alternative hypothesis H 1 :  i 0

The standard error test may be outlined as follows.

First: Compute standard error of the parameters.

SE ( ˆ )  var( ˆ )

SE (ˆ )  var(ˆ )

Second: compare the standard errors with the numerical values of ˆ and ˆ .
67
Conti…
Decision rule:
 If SE (ˆi )  1 2 ˆi , accept the null hypothesis and reject the alternative

hypothesis. We conclude that ̂ i is statistically insignificant.

 If SE (ˆi )  1 2 ˆi , reject the null hypothesis and accept the alternative

hypothesis. We conclude that ̂ i is statistically significant.

68
Conti…
The acceptance or rejection of the null hypothesis has definite economic meaning.
Namely, the acceptance of the null hypothesis  0 (the slope parameter is zero)
implies that the explanatory variable to which this estimate relates does not in fact
influence the dependent variable Y and should not be included in the function, since
the conducted test provided evidence that changes in X leave Y unaffected. In other
words acceptance of H0 implies that the relation ship between Y and X is in fact
Y   (0) x  , i.e. there is no relationship between X and Y.
Numerical example: Suppose that from a sample of size n=30, we estimate the
following supply function.
Q  120  0.6 p  ei
SE : (1.7) (0.025)

Test the significance of the slope parameter at 5% level of significance using the
standard error test. 69
 Conti…
SE ( ˆ ) 0.025

( ˆ ) 0.6

1
2 ˆ 0.3

This implies that SE ( ˆi )  1

2 ˆi . The implication is ̂ is statistically significant at
5% level of significance.
Note: The standard error test is an approximated test (which is approximated from
the z-test and t-test) and implies a two tail test conducted at 5% level of significance.
i) Student’s t-test
Like the standard error test, this test is also important to test the significance of the
parameters. From your statistics, any variable X can be transformed into t using the
general formula:
70
Conti..
X 
t , with n-1 degree of freedom.
sx

Where  i  value of the population mean

s x  sample estimate of the population standard deviation

( X  X ) 2
sx 
n 1
n  sample size
We can derive the t-value of the OLS estimates
ˆi   
t ˆ  
SE ( ˆ ) 
 with n-k degree of freedom.
ˆ   
tˆ 
SE (ˆ ) 

71
Conti…
Where:
SE = is standard error
k = number of parameters in the model.
Since we have two parameters in simple linear regression with intercept different
from zero, our degree of freedom is n-2. Like the standard error test we formally
test the hypothesis: H 0 :  i 0 against the alternative H 1 :  i 0 for the slope
parameter; and H 0 :  0 against the alternative H 1 :  0 for the intercept.

To undertake the above test we follow the following steps.

Step 1: Compute t*, which is called the computed value of t, by taking the value of
 in the null hypothesis. In our case  0 , then t* becomes:

ˆ  0 ˆ
t*  
ˆ
SE (  ) SE ( ˆ )
72
 cont..
Step 2: Choose level of significance. Level of significance is the probability of
making ‘wrong’ decision, i.e. the probability of rejecting the hypothesis when it is
actually true or the probability of committing a type I error. It is customary in
econometric research to choose the 5% or the 1% level of significance. This means
that in making our decision we allow (tolerate) five times out of a hundred to be
‘wrong’ i.e. reject the hypothesis when it is actually true.
73
 Conti…
Step 3: Check whether there is one tail test or two tail test. If the inequality sign in
the alternative hypothesis is , then it implies a two tail test and divide the chosen
level of significance by two; decide the critical rejoin or critical value of t called t c.
But if the inequality sign is either > or < then it indicates one tail test and there is no
need to divide the chosen level of significance by two to obtain the critical value of
to from the t-table.
Example:
If we have H 0 :  i 0
against: H 1 :  i 0
74
 Cont..
Then this is a two tail test. If the level of significance is 5%, divide it by two to
obtain critical value of t from the t-table.
Step 4: Obtain critical value of t, called t c at 
2 and n-2 degree of freedom for two
tail test.
Step 5: Compare t* (the computed value of t) and tc (critical value of t)
 If t*> tc , reject H0 and accept H1. The conclusion is ̂ is statistically
significant.
 If t*< tc , accept H0 and reject H1. The conclusion is ̂ is statistically
insignificant.
75
Conti..
Numerical Example:
Suppose that from a sample size n=20 we estimate the following consumption
function:
C  100  0.70  e
(75.5) (0.21)

The values in the brackets are standard errors. We want to test the null hypothesis:
H 0 :  i 0 against the alternative H 1 :  i 0 using the t-test at 5% level of

significance.
the t-value for the test statistic is
76
Conti…
ˆ  0 ˆ 0.70
t*   = 3 . 3
SE ( ˆ ) SE ( ˆ ) 0.21

a. Since the alternative hypothesis (H1) is stated by inequality sign ( ) ,it is a

two tail test, hence we divide 
2 0.05 2 0.025 to obtain the critical value of

‘t’ at  2 =0.025 and 18 degree of freedom (df) i.e. (n-2=20-2). From the
t-table ‘tc’ at 0.025 level of significance and 18 df is 2.10.
b. Since t*=3.3 and tc=2.1, t*>tc. It implies that ̂ is statistically significant.

77
III) Confidence interval
Rejection of the null hypothesis doesn’t mean that our estimate ˆ and ˆ is the
correct estimate of the true population parameter  and  . It simply means that
our estimate comes from a sample drawn from a population whose parameter  is
different from zero.
In order to define how close the estimate to the true parameter, we must construct
confidence interval for the true parameter, in other words we must establish limiting
values around the estimate with in which the true parameter is expected to lie within
a certain “degree of confidence”. In this respect we say that with a given probability
the population parameter will be with in the defined confidence interval (confidence
limits). 78
Conti…
We choose a probability in advance and refer to it as confidence level (interval
coefficient). It is customarily in econometrics to choose the 95% confidence level.
This means that in repeated sampling the confidence limits, computed from the
sample, would include the true population parameter in 95% of the cases. In the
other 5% of the cases the population parameter will fall outside the confidence
interval.
In a two-tail test at  level of significance, the probability of obtaining the specific
t-value either –tc or tc is  2 at n-2 degree of freedom. The probability of obtaining
79
Cont..
ˆ  
any value of t which is equal to at n-2 degree of freedom is
SE ( ˆ )

1  2   2  i.e. 1   .
i.e. Pr t c  t*  t c  1   …………………………………………(2.57)

ˆ  
but t*  …………………………………………………….(2.58)
ˆ
SE (  )

Substitute (2.58) in (2.57) we obtain the following expression.

 ˆ   
Pr  t c   t c  1   ………………………………………..(2.59)
SE ( ˆ)

 

 
Pr  SE ( ˆ )t c  ˆ    SE ( ˆ )t c 1        by multiplyin g SE ( ˆ )

Pr  ˆ  SE ( ˆ )t      ˆ  SE ( ˆ )t  1        by subtractin g ˆ
c c

Pr  ˆ  SE ( ˆ )    ˆ  SE ( ˆ )t  1        by multiplyin g by  1
c


Pr ˆ  SE ( ˆ )t    ˆ  SE ( ˆ )t  1        int erchanging
c c
80
Cont.…
Numerical Example:
Suppose we have estimated the following regression line from a sample of 20
observations.
Y 128.5  2.88 X  e
(38.2) (0.85)

The values in the bracket are standard errors.

a. Construct 95% confidence interval for the slope of parameter
b. Test the significance of the slope parameter using constructed confidence
interval.

81
Cont.…
Solution:
a. The limit within which the true  lies at 95% confidence interval is:

ˆ SE ( ˆ )t c

ˆ 2.88

SE ( ˆ ) 0.85

t c at 0.025 level of significance and 18 degree of freedom is 2.10.

 ˆ SE ( ˆ )t c 2.88 2.10(0.85) 2.88 1.79.

The confidence interval is:

(1.09, 4.67)
The value of  in the null hypothesis is zero which implies it is out side the
confidence interval. Hence  is statistically significant.
82
 Cont..
2.2.3 Reporting the Results of Regression Analysis

The results of the regression analysis derived are reported in conventional formats.
It is not sufficient merely to report the estimates of  ’s. In practice we report
2
regression coefficients together with their standard errors and the value of R . It
has become customary to present the estimated equations with standard errors
placed in parenthesis below the estimated parameter values. Sometimes, the

83
 Conti…
placed in parenthesis below the estimated parameter values. Sometimes, the
estimated coefficients, the corresponding standard errors, the p-values, and some
other indicators are presented in tabular form.
2
These results are supplemented by R on ( to the right side of the regression
equation).

84
Thanks
85