CHAPTER TWO

SIMPLE LINEAR REGRESSION

Introduction
As we have already mentioned in chapter one, one of the roles of econometrics is to lend
empirical support to economic theories and hypothesis. There are various ways of econometric
analysis used to achieve this objective. Regression analysis is the most common and appropriate
technique of econometric analysis. Regression analysis refers to estimating functions showing
the relationship between two or more variables and corresponding tests. Thus, the purpose of this
chapter is to introduce the concept of simple linear regression analysis.

2.1. Concept of Regression Function

2.1.1. Introduction to Simple Linear Regression
Correlation analysis is concerned for expressing and measuring the closeness of variables. It does
not identify the cause and effect relationship between variables. Regression is suitable in
identifying the cause and effect relationships of variables. In this section, we will discuss one of
the simplest form of regression; that is simple regression.

Regression analysis is concerned with the study of relationship between one variable (known as
dependent variable) and one or more other variables (known as independent variable(s)). A
central point in regression analysis is estimating regression functions accompanied by some
preceding and some succeeding steps in the econometrics methodology. Such function is
generally given as follows, equation 3.1.

Y = f ( Xi )
2.1
The equation means that the variable ‘Y’ is a function of the other variable(s) given by ‘Xi’.
Where, ‘i’ = 1, 2, 3, …………….
Note that the variable on the left hand side represented by ‘Y’ is known as dependent variable,
explained variable and/or endogenous variable. The variable(s) on the right hand side
represented by ‘Xi’ is also known as independent variable(s), explanatory variable(s) and/or
exogenous variable(s) demonstrated as follows.

1|Page
 ⏟
Y = f ( X 1 , X 2 , X 3 ,. . .. .. . .. . )
⏟
⇑ ⇑
dependent var iable independent var iables
exp lained variable exp lanatory var iables
endogenous variable exogenous var iables 2.2
Note also that regression function doesn’t always imply causal relationship among the variables.
Why regression functions? What are the objectives of the regression analysis?
The major objectives and uses of a regression function are:
1) to estimate mean or average value of the dependent variable, given the value of
independent variable(s);
2) to test hypothesis about sign and magnitude of relationship between the dependent
variable and one or more independent variable(s);
3) to predict or forecast future value(s) of the dependent variable which is in turn used in
policy formulation; and
4) Combination of any two or more of the above objectives.

Having this much about regression function in general, now let’s see some more points about our
current topic ‘simple linear regression analyses. Let’s see the meaning of the words ‘simple’
and ‘linear’ in the topic. In simple linear regression function or analysis, the term ‘simple’ refers
to the fact that we use only two variables (one dependent and another independent variable). If
the number of independent or explanatory variables is greater than one, we don’t use the term
‘simple’. Instead we use the term ‘multiple’ as we will see in the next chapter.

Most econometric models got their model specification from economic theories since as we
mentioned in chapter one, coefficients of an econometric model are constants of economic
theory. Now let us discuss the simple linear regression using an example from economic
theories. Consider the theory of demand. The theory of demand in its simplest form postulates
that there exists a negative relationship between price and quantity demanded, ceteris paribus.
As price rises quantity demanded decreases and vice versa. Following the econometric procedure
our task is specification of the demand model: Determination of dependent and independent

2|Page
variables, the number of equations of the model and their precise mathematical form. Economic
theory provides the following information with regard to demand function.
a) The dependent variable is quantity demanded and the independent variable is price.
Y i=f ( X i)

Where Y i = quantity demanded and X i =price

b) Economic theory does not specify whether demand is studied using one equation or more
elaborated form of simultaneous equations. Let us we use one equation system.

c) Economic theory is not clear about the mathematical form (linear or non-linear) of the demand

function. Let we choose a linear function. Therefore, the demand function is given as
Y i=α + β X i----------------------------------------------------------------------------------------- (2.3)
d) The above form of demand function implies that the relation between Y and X is exact. That
means the whole variation in Y is due to the changes in X only and there are no other factors or
variables affecting the dependent variable except X. If this were true, all quantity-price pairs
would fall on the straight line if we plot it in the X-Y plane. However, if we gather information
from the market and plot it in the X-Y plane, all the price-quantity pairs do not fall on the
straight line. Most of the pairs would lie on the straight line, some of the points would lie above
the straight line and some of the points would also lie below the straight line. The deviation of
points from the straight line may be attributed to several reasons. These are:
1. Omitted Variables from the Function: In economics, each variable is influenced by a very
large number of factors. However, in the model not each and every factor or variable can be
included in the function due to several factors
a) Some of the factors may not even known by a person
b) Some of the factors may not be statistically measured
c) Some of the factors are random appearing in unpredictable way and time
d) Some of the factors have very small influence on the dependent variable
e) Some of the factors may not be having reliable statistical data.
Therefore, in most cases, only few variables are explicitly included in the model.
2. Misspecification of the Mathematical Form of the Model: Misspecification of the model
can also result in the deviation of points from the straight line.

3|Page
4. Errors of Aggregation: We often use aggregate data in which we add magnitudes referring to
individual behavior which are dissimilar. In this case, those variables expressing individual
peculiarities are missing.
5. Errors of Measurement: The deviation of points around the line may be due to the errors of
measurements of variables which are inevitable due to the methods of collecting and processing
statistical information.

In order to take into account the above sources of errors, we include in econometric functions a
random variable which is usually denoted by letter ‘U’ and is called the error term or the
disturbance term or the random term.
Y i = α + β X i+U i--------------------------------------------------- (2.4)
The true relationship which connects the variables involved is split in two parts: A part
represented by the line and a part represented by the random term. Such type of models is called
stochastic or probabilistic models and is familiar in econometrics. The above model shows that
the relationship between the two variables is inexact and the total variation of the dependent
variable is split into two additive components: Explained variation and residual variation which
can be shown as
TSS=ESS + RSS
The variation in the dependent variable is not hundred percent explained by the variation in the
explanatory variable. Thus, the variation in the dependent variable expressed as the sum of
explained variation and random variation given as follows.
Yi = α + βX i + Ui
⏟ ⏟ ⏟
(Variation in Yi) = (Systematic variation) + (Random variation)
(Variation in Yi) = (Explained variation) + (Unexplained variation)

Let’s discuss the second term ‘linear’ in our topic. What is the meaning of the term ‘linear’? Is it
the same as its meaning in mathematics? The answer is no! The meaning linear regression
equation in econometrics is different from linear equation in mathematics. The difference
between the two can better be captured in comparing ‘linearity’ in variables and ‘linearity’ in
parameters.

4|Page
According to mathematical definition, a function is said to be linear in variables if the
conditional expectation of the dependent variable given the values of the independent variable is
a linear function of the variable. Linearity in variables is represented by combination of variables
with the same power and their power is of degree one. Variables are linearly combined (the
variables have the same degree). The following are examples of linear equations in variables.
Y i=α + β X i +U i ----------------------------------------------------------- (2.5)
Y i=α +2 β X i+ U i --------------------------------------------------------- (2.6)

etc

As opposed to mathematical functions, in econometrics linearity doesn’t necessarily mean

linearity in variables. It basically means linearity in parameters. For instance if the variation in
the dependent variable ‘Yi’ is better explained by the square the explanatory variable (X i2) than
its actual values (Xi), the same concept of linear regression analysis can be used since all the
necessary transformations are made before estimation (for instance, squaring all values of X i).
Thus, the term ‘linearity’ in regression analysis emphasizes the linearity in parameters. The
function is said to be linear parameters if the conditional expectation of a function for given
values of the variable is a linear function of its parameters. For instances,
Y i=α + β X i +U i ----------------------------------------------------------- (2.7)
2
Y i=α + β X i +U i ---------------------------------------------------------- (2.8)

etc

5|Page
2.2. Estimating Simple Linear Regression Function
In econometrics research, the second step or stage is to find the numerical values of the
population parameters which can be used for several purposes. The magnitude of the estimator is
highly required by analysts, policy formulators and forecasters in executing their jobs. In this
section, we will discuss the method of estimating parameters of simple linear regression
functions. There are various methods of estimating regression functions such as Method of
Moment, Method of Least Square or Ordinary Least Square (OLS) method and the Maximum
Likelihood (MLE) method. First we give highlight on these three methods and we discuss OLS
method in detail due to its advantage over the Method of Moment and Maximum Likelihood
Method. The ordinary least square method is the easiest and the most commonly used method as
opposed to the maximum likelihood (MLH) method and Moment Method which are limited by
their assumptions. For instance, the MLH method is valid only for large sample as opposed to the
OLS method which can be applied to smaller samples. Owing to this merit, our discussion
mainly focuses on the ordinary least square (OLS).

This section is divided into three subsections. In the first subsection we will explain the Method
of Moment and OLS techniques of estimating regression functions along with classical
conditions or assumptions of OLS method. In the second section we will entirely concentrate on
the OLS method of regression analysis; and finally we will see the related concepts including
measures of goodness of fit and prediction with simple linear regression function.
2.2.1. Method of Moment
Since our objective is to get the estimates of the unknown parameters α ∧β in equation 2.3, we
have to make some assumptions about the error term, U i. These are
i) Zero mean: E(U i) = 0 for all i
ii) Constant variance : Var (U ¿ ¿i)¿ =δ 2u for all i
iii) Independence: U i∧U j are independent for all i≠ j . i.e. Cov (U i ,U j ) = 0
iv) Independence of X j: U i∧ X j are independent for all i≠ j. i.e. Cov (X i , U i) = 0
v) Normality: U i are normally distributed with zero mean and constant (common)
variance. U i ̴ N (0 , δ 2u ).
The assumptions we have made about the error term u imply that

6|Page
 E(U i) = 0 and Cov (X i , U i) = E( X i U i) =0
In the method of moments, we replace these conditions by their sample counterparts. Let α^ ∧ β^
be the estimators for α ∧β , respectively. The sample counterpart of U i is the estimated error e i
(which is called the residual), defined as
e i = Y i - Y^i , but Y^i= α^ + β^ X i
e i = Y i− α^ − ^β X i ---------------------------------------------------------(2.9)

The two equations to determine α^ ∧ β^ are obtained by replacing the population assumptions by
their sample counterparts. That is, for E(U i) = 0 and Cov (X i , U i) = 0, the counterpart
assumptions are E(ei ) = 0 and Cov (X i , ei ) = 0.

n
Note: In these and hereafter ∑ denotes ∑ ❑
i=1

To do so, taken the summation of both sides in equation (2.9)

∑ ei= ∑ (Y i− α^ − ^β X i ) ,
But, E(ei ) =
∑ e i = 0. This imply that ∑ e =0
i
n
Then ∑ (Y i− α^ − ^β X i ) = 0
∑ Y i−n α^ − ^β ∑ X i n α^ = ∑ Y i− ^β ∑ X i α^ = Y − ^β X ----------------- (2.10)

By assumption IV above, E( X i ei ) =
∑ X i ei = 0. This also imply that ∑ X e =0
i i
n
∑ X i ei =0, but e i = Y i− α^ − ^β X i

∑ X i (Y i− α^ − β^ X i ) = 0
∑ Y i X i −α^ ∑ X i− ^β ∑ X 2i =0----------------------------------------------------------- (2.11)

n ∑ Y i X i−∑ Y i ∑ X i ∑ yi xi
^β = or ^β =
n ∑ X −¿ ¿ ∑ X 2i
2
i

While estimating the unknown parametersα ∧β , we have derived two equations (equation 2.10
and 2.11). These equations are called normal equations.

2.2.2. Ordinary Least Square Method (OLS) and Classical Assumptions

7|Page
There are two major ways of estimating regression functions. These are the (ordinary) least
square (OLS) method and maximum likelihood (MLH) method. Both the methods are basically
similar to their application to point estimations that you may be aware of in statistics courses.

The (Ordinary) least square (OLS) method of estimating parameters or regression function is

about finding or estimating values of the parameters ( α and β ) of the simple linear regression
function given below (equation (2.2)’’ for which the errors or residuals are minimized. Thus, it is
about minimizing the residuals or the errors.

Y i=α + β X i +U i ----------------------------------------------------------- (2.12)

You may recall that the above identity represents population regression function (to be estimated
from total enumeration of data from the entire population. But, most of the time it is difficult to
generate population data owing to several reasons; and most of the time we use sample data and
we estimate sample regression function. Thus, we use the following sample regression function
for the derivation of the parameters and related analysis.

Y^i= α^ + β^ X i----------------------------------------------------------- (2.13)

Before discussing the details of the OLS estimation techniques, let’s see the major conditions
known as classical assumptions that are necessary for the validity of the analysis, interpretations
and conclusions of the regression function.

i) Classical Assumptions
1. The error terms ‘U i ’ are randomly distributed or the disturbance terms are not correlated.
This means that there is no systematic variation or relation among the value of the error

terms (U i and U j) where i = 1, 2, 3, …….., j = 1, 2, 3, ……. and i≠ j . This is

represented by zero covariance among the error terms: ¿ , U j ¿=0 for i≠ j . Note that the
same argument holds for residual terms when we use sample data or sample regression

function. Thus, Cov ¿ , e j ¿=0 for i≠ j

Otherwise, the error terms do not serve an adjustment purpose; rather, it causes an
autocorrelation problem. This problem will be discussed in detail in chapters four.

8|Page
 2. The disturbance terms ‘U i ’ have zero mean. The sum of the disturbance terms is zero. The
deviation or the values of some of the disturbance terms are negative; some are zero and
∑Ui
some are positive and the sum or the average is zero. That is, E(U ¿¿ i)¿= . Multiplying
n
both sides by (sample size ‘n’) we obtain the following. ⇒ E(U ¿¿ i)=∑U i=0 ¿. The
same argument is true for sample regression function and so for residual terms given as
follows ∑ e i=0 . If this condition is not met, then the position of the regression function (or
curve) will not be the same as where it is supposed to be. This results in an upward (if the
mean of the error term or residual term is positive) or down ward (if the mean of the error
term or residual term is negative) shift in the regression function. For instance, suppose we
have the following regression function.
Y i=α + β X i +U i
E(Y ¿¿ i)=E (Y i / X i )=E ( α + β X i )+ E(U ¿¿ i)¿ ¿
E(Y ¿¿ i)=E (Y i / X i )=α + β X i ¿ if E (U ¿¿ i)¿ =0
Otherwise the estimated models will be biased and causes the regression function to shift.
For instance, if E(U ¿¿ i)>0 ¿ (or positive) it is going to shift up the estimation from the true
representative model. Similar argument is true for residual term of sample regression
function.

3. The disturbance terms have constant variance in each period. This is given as follows:
Var (U ¿ ¿i)¿ = E ¿ ¿2 = δ2 = δ 2u. This assumption is known as the assumption of
homoscedasticity. If this condition is not fulfilled or if the variance of the error terms varies
as sample size changes or as the value of explanatory variables changes, then this leads to
Heteroscedasticity problem. This problem is given as follows: Var (U ¿ ¿i)¿ =δ 2i = δ 2ui. i = 1, 2,
3, 4... Heteroscedasticity problem along its detection methods and remedial measures will be
addressed in chapters four as well.

4. Explanatory variables ‘Xi’ and disturbance terms ‘U i ’ are uncorrelated or independent. All
the co-variances of the successive values of the error term are equal to zero. This condition
is given by the following identity. Cov ¿ , U j ¿=0 . It is followed from this that ∑ e i X i=0 .
The value in which the error term assumed in one period does not depend on the value in

9|Page
 which it assumed in any other period. If this condition is not met by our data or variables,
our regression function and conclusions to be drawn from it will be invalid.

5. The explanatory variable Xi is fixed in repeated samples. Each value of X i does not vary for
instance owing to change in sample size. This means the explanatory variables are non-
random and hence distributional free variable.

6. Linearity of the model in parameters. The simple linear regression requires linearity in
parameters; but not necessarily linearity in variables. The same technique can be applied to
estimate regression functions of the following forms: Y = f (X ); Y = f (X 2); Y = f
(X 3); Y = f (X – kX ); and so on. What is important is transforming the data as required.

7. Normality assumption. The disturbance term Ui is assumed to have a normal distribution

with zero mean and a constant variance. This assumption is given as follows: U i ̴ N (0 , δ 2u ).
This assumption is a combination of zero mean of error term assumption and
homoscedasticity assumption. This assumption or combination of assumptions is used in
testing hypotheses about significance of parameters as we will see in the fourth chapter. It
is also useful in both estimating parameters and testing their significance in maximum
likelihood method.

8. Explanatory variables should not be perfectly, linearly and/or highly correlated. Using
explanatory variables which are highly or perfectly correlated in a regression function
causes a biased function or model. It also results in multicollearity problem (to be discussed
chapter four.

9. The variables are measured without error (the data are error free). Since wrong data leads
to wrong conclusion, it is important to make sure that our data is free from any type of
error. There must also be correct aggregation of values of related variables (macro
variables). Aggregate variables such as Gross Domestic Product (GDP), Trade Balance,
Balance of Payment (BOP), etc. should be correctly calculated from their components.

10. The relationship between variables (or the model) is correctly specified. For instance all the
necessary variables are included in the model. The variables are in the form that best

10 | P a g e
 describes the functional relationship. For instance, “Y = f (X 2)” may better reflects the
relationship between Y and X than “Y = f (X )”.

Note that some of these assumptions or conditions, (those which imply to more than one
explanatory variables), are meant for the next chapters (along with all the other assumptions or
conditions). So, we may not restate these conditions in the next chapter even if they are
required there also. Next let’s directly go to our discussion about techniques of estimating the
simple linear regression model.

ii) Estimation of OLS Model and Distribution of the Dependent variable Y

The dependent variable is normally distributed with the following mean and variance.

Mean of Y =E(Y i )=E (α+βX i +U i )

=E (α )+E( βX i )+E(U i )
= α+βX i

Var ( Y i )=E( ( Y i−E ( Y i ) )2

And the variance of Y is given as =E( α + βX i +U i −α −βX i )2 =E( U i )2=δ 2u

Therefore, the variance of the dependent variable is the same with the variance of the error term
which is constant.

Yi N ( α + βX i , δ
u
2 )
˜

Estimating a linear regression function using the Ordinary Least Square (OLS) method is simply
about calculating the parameters of the regression function for which the sum of square of the
error terms is minimized. Suppose we want to estimate the following equation
Y i=α + β X i +U i ----------------------------------------------------------- (2.14)
Since most of the time we use sample (or it is difficult to get population data) the corresponding
sample regression function is given as follows.
Y^i= α^ + β^ X i----------------------------------------------------------------- (2.15)

From this identity, we solve for the residual term

' e i ' , square both sides and then take sum of
both sides. These three steps are given (respectively as follows.

11 | P a g e
 e i = Y i- Y^i = Y^i-α^ - ^β X i ------------------------------------------------- (2.16)

∑ e2i = ∑ (Y i− α^ − ^β X i )2------------------------------------------ (2.17)

Where,
∑ e i2 = RSS= Residual Sum of Squares.
The method of OLS involves finding the estimates of the intercept and the slope for which the
sum squares given by Equation 2.17 is minimized. To minimize the residual sum of squares we
take the first order partial derivatives and equate them to zero.

That is, the partial derivative with respect to α^ :

∂∑ ei
2
= 2∑ ( Y i− α^ – ^β X i ) (−1) =0 ------------------------------------------ (2.18)
∂ α^

∑ ( Y i− α^ – ^β X i ) =0 -------------------------------------------------- (2.19)
∑ Y i−n α^ − ^β ∑ X i =0 ------------------------------------------------ (2.20)
∑ Y i=n α^ + ^β ∑ X i ---------------------------------------------------- (2.21)
Where n is the sample size.

Partial derivative With respect to ^β

∂∑ ei
2

= 2∑ ( Y i− α^ – ^β X i ) (− X i) =0 ------------------------------------------ (2.22)
∂ β^

∑ ( Y i X i−α^ X i – ^β X 2i ) =0 ------------------------------------------ (2.23)

∑ Y i X i =α^ ∑ X i+ β^ ∑ X 2i =0 --------------------------------------- (2.24)
∑ (Y i− α^ − ^β X i )
2
Note that the equation is a composite function and we should apply a chain

rule in finding the partial derivatives with respect to the parameter estimates.
Equations 2.21 and 2.24 are together called the system of normal equations. Solving the system
of normal equations simultaneously we obtain:
n ∑ Y i X i−∑ Y i ∑ X i
^β = ------------------------------------------------------ (2.25)
n ∑ X i −¿ ¿
2

^β ∑ Y i X i−n Y X
=
∑ X 2i −n X 2
12 | P a g e
And α^ = Y − ^β X ------------------------------------------------------------ (2.26)
The regression coefficients can also be obtained by simple formulae by taking the deviations
between the original values and their means. Now, if
x i= X i− X̄ (2.27)
and
y i =Y i−Ȳ (2.28)
Then,

^β ∑ yi xi
= --------------------------------------------------------------------- (2.29)
∑ X 2i
α^ = Y − ^β X -------------------------------------------------------------------------------- (2.30)
Example 2.2. The following table gives the quantity supplied (in tons) and its price (pound per
ton) for a commodity over a period of twelve years.
Yi 69 76 52 56 57 77 58 55 67 53 72 64
Xi 9 12 6 10 9 10 7 8 12 6 11 8
a) Fit the simple linear regression equation; Y = f(X) and Interpret your result.
b) Test the significance of each parameter estimates
c) Determine the 95% confidence interval for the slope.
d) Predict the value of Y when X is 45.
Solution: The slope and the intercept are estimated to be:

∑ xy 156
^
β= = =3 .23
48
∑ x2
α^ = Y − ^β X = 63-(3.23)(9) = 33.75
Thus the fitted regression function is given by:

Y^ i = 33 .75 + 3 . 23 X i
Interpretation: The value of the intercept term, 33.75, implies that the value of the
dependent variable ‘Y’ is 33.75 when the value of the explanatory variable is zero. The
^
value of the slope coefficient ( β=3 . 23 ) is a measure of the marginal change in the
dependent variable ‘Y’ when the value of the explanatory variable increases by one. For

13 | P a g e
 instance, in this model, the value of ‘Y’ increases on average by 3.23 units when ‘X’
increases by one.

2.3. Residuals and Goodness of fit

After estimating the parameters and determining the regression line, we need to judge the
explanatory power as well as the statistical reliability of the regression of Y on X. There are two
most commonly used tests in econometrics. These are:
2
1. The square of correlation coefficient,r , which is used for judging the explanatory power of
the linear regression of Yon X or on X’s. The square of correlation coefficient in simple
2
regression is known as the coefficient of determination and is given by R . The coefficient of
determination measures the goodness of fit of the line of regression on the observed sample
values of Y and X.
2. The standard error test of the parameter estimates and is applied for judging the statistical
reliability of the estimates. This test measures the degree of confidence that we may attribute to
the estimates (section 2.6).

2.3.1. The Coefficient of determination (R2)

The coefficient of determination is the measure of the amount or proportion of the total variation
of the dependent variable that is determined or explained by the model or the presence of the
explanatory variable in the model. The total variation of the dependent variable is split in two
additive components: a part explained by the model and a part represented by the random term.
¿
Total var iation in Y i=∑ (Y i −Y )2
¿ ¿
Total exp lained var iation=∑ (Y i −Y )
2

Total un explained var iation=∑ e2i

The total variation of the dependent variable is given in the following form: TSS=ESS + RSS,
which means total sum of square of the dependent variable is split into explained sum of square
and residual sum of square.

14 | P a g e
 ¿
ei = y i− y i
¿
y i = y i +e i
¿ ¿
y 2i = y i2 +e 2i +2 y i ei
¿ ¿
∑ yi2=∑ yi2+∑ e 2i +2 ∑ y i e i
¿
But ∑ y i e i=0
¿
Therefore , ∑ yi2= ∑ y i2+∑ e 2i
The coefficient of determination is given by the formula

∑ (Y^ i−Ȳ )2 ∑ ^y i2 Explained Variation in Y

R2 = = =
Total Variation in Y
∑ (Y i−Ȳ )2 ∑ y i2
(2.31)
¿ ¿
Since ∑ y 2i = β1 ∑ x i y i the coefficient of det er min ation can also be given as
¿
β 1∑ x i y i
R2 =
∑ y 2i or

Unexplained Variation in Y
∑ (Y i−Y^ )2
i ∑ ei 2

2
R =1− =1− =1−
Total Variation inY
∑ (Y i− Ȳ ) 2
∑ yi 2

(2.32)
The higher the coefficient of determination is the better the fit. Conversely, the smaller the
coefficient of determination is the poorer the fit. That is why the coefficient of determination is
used to compare two or more models. One minus the coefficient of determination is called the
coefficient of non-determination, and it gives the proportion of the variation in the dependent
variable that remained undetermined or unexplained by the model.

Example 2.3. Refer to Example 2.2. Determine how much percent of the variations in the
quantity supplied is explained by the price of the commodity and what percent remained
unexplained.
15 | P a g e
Using Equation 2.26:

∑ e i2 387
2
R =1− =1− =1−0 . 43=0. 57
894
∑ y i2
This result shows that 57% of the variation in the quantity supplied of the commodity under
consideration is explained by the variation in the price of the commodity; and the rest 37%
remained unexplained by the price of the commodity. In other word, there may be other
important explanatory variables left out that could contribute to the variation in the quantity
supplied of the commodity, under consideration.

2.4. Properties of OLS Estimates and Gauss-Markov Theorem

There are various econometric methods with which we may obtain the estimates of the
parameters of economic relationships. We would like to an estimate to be as close as the value of
the true population parameters i.e. to vary within only a small range around the true parameter.
How are we to choose among the different econometric methods, the one that gives ‘good’
estimates? We need some criteria for judging the ‘goodness’ of an estimate.

‘Closeness’ of the estimate to the population parameter is measured by the mean and variance or
standard deviation of the sampling distribution of the estimates of the different econometric
methods. We assume the usual process of repeated sampling i.e. we assume that we get a very
large number of samples each of size ‘n’; we compute the estimates ^β ’s from each sample, and
for each econometric method and we form their distribution. We next compare the mean
(expected value) and the variances of these distributions and we choose among the alternative
estimates the one whose distribution is concentrated as close as possible around the population
parameter. The ideal or optimum properties that the OLS estimates possess may be summarized
by well known theorem known as the Gauss-Markov Theorem.

Statement of the theorem: “Given the assumptions of the classical linear regression model, the
OLS estimators, in the class of linear and unbiased estimators, have the minimum variance, i.e.
the OLS estimators are BLUE. According to this theorem, under the basic assumptions of the

16 | P a g e
classical linear regression model, the least squares estimators are linear, unbiased and have
minimum variance (i.e. are best of all linear unbiased estimators). Sometimes the theorem
referred as the BLUE theorem i.e. Best, Linear, and Unbiased Estimator. An estimator is
called BLUE if (the detailed proofs of these properties are presented below):
a. Linear: a linear function of the random variable, such as, the dependent variable Y.
b. Unbiased: its average or expected value is equal to the true population parameter.
c. Minimum variance: It has a minimum variance in the class of linear and unbiased
estimators. An unbiased estimator with the least variance is known as an efficient
estimator.
^
a. Linearity: (for β )
^
Proposition: α^ ∧ β are linear in Y.
^
Proof: From (2.17) of the OLS estimator of β is given by:
Σx i y i Σxi (Y −Ȳ ) Σx i Y −Ȳ Σx i
^
β= = = ,
Σx2i Σx 2i Σx 2i

(but Σ xi= ∑ ( X− X̄ )=∑ X−n X̄ =n X̄−n X̄=0 )

Σxi Y xi
⇒ β^ = =K i
Σx 2i ; Now, let Σx 2i (i=1,2,.....n)
^
∴ β=ΣK i Y −−−−−−−−−−−−−−−−−−−−−−−−−−(2. 33)

⇒ β^ =K 1 Y 1 +K 2 Y 2 +K 3 Y 3 +−−−−+K n Y n

∴ β^ is linear in Y
Exercise: Show that α^ is linear in Y
b. Unbiasedness:
^
Proposition: α^ ∧ β are the unbiased estimators of the true parameters α ∧ β
^
From your statistics course, you may recall that if θ is an estimator of θ then

E( θ^ )−θ=the amount of bias and if θ^ is the unbiased estimator of θ then bias =0 i.e.

E( θ^ )−θ=0 ⇒ E ( θ)=θ
^ ^
In our case, α^ ∧ β are estimators of the true parameters α ∧ β .To

17 | P a g e
show that they are the unbiased estimators of their respective parameters means to prove that:

Ε( β^ )=β and Ε( α^ )=α

^ ^
 Proof (1): Prove that β is unbiased i.e. Ε( β )=β .
^
We know that β=Σ kY i =Σk i (α + βX i +U i )

=αΣk i + βΣk i X i + Σk i u i ,

but Σk i =0 and Σk i X i=1

Σx i Σ ( X − X̄ ) ΣX − n X̄ = n X̄ −n X̄ =0
Σk i = = =
Σx i 2
Σx 2
i Σx 2
i Σx2i

⇒ ∑ k i =0 …………………………………………………………………(2.34)
Σx i X i Σ ( X − X̄ ) Xi
Σki X i= =
Σx 2i Σx 2i

ΣX 2− X̄ ΣX ΣX 2−n X̄ 2
= = =1
ΣX 2 −n X̄ 2 ΣX 2−n X̄ 2
⇒ ∑ k i X i =1............................. ……………………………………………(2.35)
^
β=β ^ β=Σk u −−−−−−−−−−−−−−−−−−−−−−−−−(2 . 36 )
+Σki ui ⇒ β− i i

Ε( β^ )=E( β )+Σk i E (ui ), Since k i are fixed

Ε( β^ )=β , since Ε(u i )=0

^
Therefore, β is unbiased estimator of β .

 Proof(2): prove that α^ is unbiased i.e.: Ε ( α^ )=α

From the proof of linearity property under [Link] (a), we know that:
α^ =Σ ( n− X̄ k i ) Y i
1

=Σ [( 1
n
− X̄ k i ) ( α + βX i +U i ) ], Since Y i =α + βX i +U i
=α + β 1 n ΣX i + 1 n Σu i−α X̄ Σk i −β X̄ Σki X i− X̄ Σk i ui

=α + 1 n Σu i − X̄ Σki ui ⇒ α^ −α =1 n Σu i− X̄ Σk i ui
,

18 | P a g e
 =∑ ( 1 n − X̄ k i )ui
……………………(2.37)

Ε( α^ )=α + 1 n ΣΕ( ui )− X̄ Σk i Ε( ui )

Ε( α^ )=α−−−−−−−−−−−−−−−−−−−−−−−−−−−−−(2 .24 )
∴ α^ is an unbiased estimator of α .

^
c. Minimum variance of α^ and β
Now, we have to establish that out of the class of linear and unbiased estimators of α ∧β , α^ ∧ β^
possess the smallest sampling variances. For this, we shall first obtain variance of α^ ∧ β^ and then
establish that each has the minimum variance in comparison of the variances of other linear and
unbiased estimators obtained by any other econometric methods than OLS.
a. Variance of ^β
^
var( β )=Ε ( β−Ε( β^ ))2=Ε ( β−β
^ )2
……………………………………(2.39)
Substitute (2.22) in (2.25) and we get
var( β^ )=E ( ∑ k i ui )2
=Ε [ k 21 u21 +k 22 u22 +. . .. .. . .. .. .+k 2n u2n +2k 1 k 2 u1 u 2 +.. . .. ..+2 k n−1 k n un−1 u n ]

=Ε [ k 21 u21 +k 22 u22 +. . .. .. . .. .. .+k 2n u2n ]+Ε [2 k 1 k 2 u1 u2 +.. .. . ..+2 k n−1 k n un−1 u n ]

=Ε ( ∑ k 2i u2i )+Ε( Σk i k j u i u j ) i≠ j
=Σk 2i Ε(u2i )+2 Σki k j Ε(u i u j )=σ 2 Σk2i (Since Ε(u i u j ) =0)
Σx i Σx2i 1
Σk i = Σki2= =
Σx 2i , and therefore, ( Σx2i )2 Σx2i

σ2
∴ var ( β^ )=σ 2 Σk2i = 2
Σx i ……………………………………………..(2.40)

^
b. Variance of α
var( α^ )=Ε ( ( α−Ε(
^ α) ) 2

19 | P a g e
 =Ε ( α^ −α )2 −−−−−−−−−−−−−−−−−−−−−−−−−−(2. 27 ) (2.41)
Substituting equation (2.37) in (2.41), we get

var( α^ ) =Ε [ Σ ( ]
2
− X̄ k i ) u2i
1
n

2
=∑ ( 1 n − X̄ k i ) Ε( ui )2

=σ 2 Σ( 1 n − X̄ k i )2
2
=σ2 Σ ( 1 − X̄ k i + X̄ 2 k 2i )
n2 n

=σ 2 Σ ( 1 n −2 X̄ n Σki + X̄ 2 Σk2i )
, Since∑ k i=0
=σ 2 ( 1 n + X̄ 2 Σk 2i )

1 X̄ 2 Σx2i 1
=σ 2 ( + ) Σki2= =
n ∑x 2 ( Σx2i )2 Σx2i
i , Since
Again:

( )
1 X̄ 2 Σx 2i +n X̄ 2 ΣX 2
+ = =
n Σx 2 nΣx2i nΣx 2i
i

( ) ( )…………………………………………(2.28)
1 X̄ 2 ΣX 2i
∴ var ( α^ )=σ 2 n
+ =σ 2
Σx 2i nΣx 2i

We have computed the variances OLS estimators. Now, it is time to check whether these
variances of OLS estimators do possess minimum variance property compared to the variances
other estimators of the true α ∧β , other than α^ ∧ β^ .
To establish that α^ ∧ β^ possess minimum variance property, we compare their variances with

that of the variances of some other alternative linear and unbiased estimators of α ∧β , say α∗¿ ¿

and β∗¿ ¿. Now, we want to prove that any other linear and unbiased estimator of the true
population parameter obtained from any other econometric method has larger variance that that

OLS estimators. Lets first show minimum variance of β^ and then that of α^ .
^
1. Minimum variance of β

Suppose: β∗¿ ¿ an alternative linear and unbiased estimator of β and;

Let β∗¿ Σw i Y i ......................................... ………………………………(2.29)

20 | P a g e
 where , w i≠k i ; but: w i=k i +c i

β∗¿ Σw i ( α + βX i +ui ) Since Y i =α + βX i +U i

=αΣwi +βΣw i X i +Σw i ui

∴ Ε( β∗)=αΣwi + βΣw i X i ,since Ε(u i )=0

Since β∗¿ ¿is assumed to be an unbiased estimator, then for β∗¿ ¿is to be an unbiased estimator of
β , there must be true that Σw i=0 and Σw i X=1 in the above equation.

But, w i=k i +c i
Σw i=Σ(k i +c i )=Σk i + Σc i

Therefore, Σci =0 since Σki =Σw i=0

Again Σw i X i=Σ (k i +c i ) X i=Σk i X i + Σci X i

Since Σw i X i=1 and Σki X i=1 ⇒ Σc i X i=0 .

From these values we can drive Σci x i=0 , where xi =X i − X̄

Σci x i=∑ ci ( X i − X̄ )= Σci X i + X̄ Σci

Since Σc i x i=1 Σc i =0 ⇒ Σc i x i =0
Thus, from the above calculations we can summarize the following results.
Σw i=0 , Σw i x i=1 , Σci =0 , Σci X i =0
^
To prove whether β has minimum variance or not lets compute var( β∗) to compare with
var( β^ ) . var( β∗)=var( Σw i Y i )

=Σw 2 var (Y i )
i

∴ var( β∗)=σ 2 Σw 2i since Var (Y i )=σ 2

Σw 2 = Σ ( k i + ci )2 =Σk 2i + 2 Σk i ci + Σc 2i
But, i

Σc i x i
Σk i ci = =0
⇒ Σw 2i =Σk2i + Σc 2i Since Σx 2i

21 | P a g e
 2 2 2 2 2 2 2
Therefore, var ( β∗)=σ ( Σk i + Σci )⇒ σ Σk i + σ Σci
var ( β∗)=var ( β^ )+ σ 2 Σc2i
2 2
Given that ci is an arbitrary constant, σ Σc i is a positive i.e it is greater than zero. Thus
^
var( β∗)>var( β^ ) . This proves that β possesses minimum variance property. In the similar
^ ) possesses
way we can prove that the least square estimate of the constant intercept (α
minimum variance.
Exercise 2: Check for α^
The variance of the random variable (Ui)
2
You may observe that the variances of the OLS estimates involve σ , which is the population
variance of the random disturbance term. But it is difficult to obtain the population data of the
2
disturbance term because of technical and economic reasons. Hence it is difficult to compute σ ;
this implies that variances of OLS estimates are also difficult to compute. But we can compute
2 ^ 2 computed from the
these variances if we take the unbiased estimate of σ which is σ
sample value of the disturbance term ei from the expression:
Σe2i
σ^ 2u=
n−2 …………………………………..2.42

^
To use σ^ in the expressions for the variances of α^ and β , we have to prove whether
2

∑ e i2
^ 2 )= E(
E( σ )=σ
2
σ^ 2 is the unbiased estimator of σ 2 , i.e., n−2

To prove this we have to compute

∑ ei 2 from the expressions of Y, Y^ , y , ^y and ei .
Proof:
Y i =α^ + β^ X i + ei

^ β^ x
Y^ = α+
⇒Y =Y^ +e i ……………………………………………………………(2.43)
^
⇒ e i=Y i−Y ……………………………………………………………(2.44)

22 | P a g e
Summing (2.43) will result the following expression
ΣY i =Σyi + Σei

ΣY i =Σ Y^ i sin ce ( Σei )=0

Dividing both sides the above by ‘n’ will give us
ΣY Σ Y^ i
= ^
n n  Ȳ =Ȳ −−−−−−−−−−−−−−−−−−−−(2 . 45 )
Putting (2.43) and (2.45) together and subtract
Y =Y^ + e
Ȳ =Ȳ^

⇒(Y −Ȳ )=( Y^ −Ȳ^ )+e

⇒ y i= ^y i + e ………………………………………………(2.46)

From (2.46):
e i= y i− ^y i ………………………………………………..(2.47)

Where the y’s are in deviation form.

Now, we have to express y i and y^ i in other expression as derived below.

From: Y i =α + βX i +U i
Ȳ =α+β X̄ + Ū
We get, by subtraction
y i =(Y i −Ȳ )=βi ( X i− X̄ )+(U i−Ū )=βx i +(U−Ū )
⇒ y i=βx +(U −Ū ) …………………………………………………….(2.48)

Note that we assumed earlier that , Ε(u )=0 , i.e in taking a very large number samples we

expect U to have a mean value of zero, but in any particular single sample Ū is not
necessarily zero.
Similarly: From;
^ β^ x
Y^ = α+
^ β^ x̄
Ȳ = α+
23 | P a g e
 We get, by subtraction

Y^ −Ȳ^ = β^ ( X− X̄ )
⇒ ^y = β^ x …………………………………………………………….(2.49)
Substituting (2.48) and (2.49) in (2.47) we get
e i=βx i +(ui −ū)− β^ x i

= (ui −ū)−( β^ i−β )x i

The summation over the n sample values of the squares of the residuals over the ‘n’
samples yields:
^
Σe2i =Σ [(u i−ū )−( β−β )x i ] 2
^ β )2 x −2 (u −ū )( β−β
=Σ [( ui −ū )2 +( β− ^ ) xi ]
2 ii

^ β )2 Σx −2[ ( β−β
=Σ ( ui −ū )2 +( β− ^ ) Σxi ( ui −ū) ]
2 i

Taking expected values we have:

^
Ε( Σe 2i )=Ε [ Σ ( ui −ū )2 ]+ Ε[ ( β−β ^ β ) Σx ( u −ū ) ]
)2 Σx 2 ]−2 Ε [( β−
i i i
……………(2.50)
The right hand side terms of (2.50)may be rearranged as follows
2 2
a. Ε [ Σ(u−ū ) ]=Ε( Σui −ū Σui )

=Ε ( Σu2i −
( Σui )2
n )
1
=ΣΕ(u2i )− Ε (Σu)2
n
=nσ 2 − 1n Ε( u1 +u2 +. . .. .. .+ui )2 2 2
since Ε( u i )=σ u
=nσ 2 − 1n Ε( Σu 2i +2 Σui u j )

=nσ 2 − 1n ( ΣΕ ( u2i )+ 2 Σu i u j ) i≠ j

=nσ 2 − 1n nσ 2u − 2n ΣΕ( ui u j )

=nσ 2u −σ 2u (given Ε (ui u j )=0)

24 | P a g e
 =σ 2u (n−1) ……………………………………………..(2.51)
^
Ε [( β−β ^
)2 Σx 2 ]=Σx 2i . Ε ( β−β )2
b. i

Given that the X’s are fixed in all samples and we know that
^ β )2=var ( β^ )=σ 2 1
Ε( β− u
Σx2
2 1
2 ^ 2 2 σu ^
Hence Σx i . Ε( β−β ) =Σx i. Σx 2 Σx 2i . Ε( β−β )2 =σ 2u

……………………………………………(2.40)
^ ^
c. -2 Ε [( β−β )Σxi (u i−ū )]=−2 Ε[( β−β )(Σx i u i−ū Σxi )]
^
= -2 Ε [( β−β )(Σx i u i )] ,sin ce ∑ x i =0
^
But from (2.36) ,( β−β )=Σk i ui and substitute it in the above expression, we will
get:
^
-2 Ε [( β−β )Σxi (u i−ū )=−2 Ε( Σk i ui )( Σxi ui )]

= -2
Ε
[( Σx i u i
Σx 2
i
)
( Σx i ui )
] ,since
k i=
xi
∑ x i2

=−2 Ε
[ ( Σx i ui )2
Σx 2
i
]
[ ]
Σx 2 u 2 +2 Σx i x j ui u j
i i
=−2 Ε
Σx 2
i

=−2
[ Σx 2 Ε (u 2 ) +2 Σ ( x x ) Ε ( u u )
Σx
i

i
2
i j i j
Σx 2
i
i≠ j
]
Σx 2 Ε( u 2 )
i
=−2 ( given Ε( ui u j )=0 )
Σx
i2

=−2 Ε (u 2i )=−2 σ 2 …………………………………………………….(2.52)

25 | P a g e
Consequently, Equation (2.50) can be written interms of (2.50), (2.51) and (2.52) as

follows: Ε ( Σe i )=( n−1 ) σ u +σ −2 σ u=( n−2) σ u ………………………….(2.53)

2 2 2 2 2

From which we get

Ε ( )
Σe 2i
n−2
2
= E( σ^ u )=σ u
2

………………………………………………..(2.54)
Σe2i
σ^ 2u=
Since n−2

Σe 2i
σ^ 2=
Thus, n−2 is unbiased estimate of the true variance of the error term(σ 2 ).

The conclusion that we can drive from the above proof is that we can substitute

2 Σe 2i
σ^ =
n−2 for (σ 2 ) in the variance expression of α^ and β^ , since E( σ^ 2 )=σ 2 .
^
Hence the formula of variance of α^ and β becomes;
2
σ^ 2 Σei
Var ( β^ )= 2
Σxi = ( n−2) ∑ x i2 ……………………………………(2.55)

∑ e i2 ∑ X i2
Var ( α^ )=σ^
2
( )
ΣX 2i
nΣx2i
=
n( n−2) ∑ x 2
i ……………………………(2.56)

Note:
∑ ei 2 can be computed as∑ ei 2=∑ y i 2− β^ ∑ x i y i .

2.5. Maximum Likelihood Method

The Maximum Likelihood Method (MLM) is another method of obtaining the estimates of the
parameters of the population from a random sample. The MLM chooses among all the possible
estimates of parameters those values which make the possibility of obtaining an observed sample
is as large as possible.

Assumptions of Maximum Likelihood Method

i) The form of the distribution of the population Y i’s is assumed to be known. In
particular, we assume that the distribution of Yi’s is normal

26 | P a g e
 ii) The sample must be random and each Ui must be independent of any values of U i .
E(U i U j ) =0
iii) Any random sample must be representative of the population

Definition: The function which defines the joint (total) probability of any sample being
observed is called likelihood function. The general expression of likelihood function is given
as:

L= f ¿ , ⏟
β 0 , β 1 , β 2 ,−−−−−−−−−β k ¿

Varables Parameters
−1

( )
n
¿¿
1 2δ
2

L= . e u

√2 π δ2u
∂L
=o
∂ β0
−1

( )
n ¿¿
1 2
−1 2
2 u ∑ i 0 1 i
( Y − β^ − β^ X )(−1)
2δ
.e u
.2. .δ (
√2 π δu
2

∑ ( Y i− β^0− β^1 X i ) =0-------------------------------------------------- (2.33)

Similarly, by taking the partial derivative of fuction L with respect to β 1 ¿ = o ), we will arrive

at ,
^
β0 = Y − ^
β1 X
n ∑ Y i X i−∑ Y i ∑ X i ∑ yi xi
^
β1 = or ^
β1 =
n ∑ X i −¿ ¿ ∑ X 2i
2

2.6. Confidence Interval and Hypothesis Testing

2.6.1. Testing the significance of a given regression coefficient
Since the sample values of the intercept and the coefficient are estimates of the true population
parameters, we have to test them for their statistical reliability. The significance of a model can
be seen in terms of the amount of variation in the dependent variable that it explains and the

27 | P a g e
significance of the regression coefficients. There are different tests that are available to test the
statistical reliability of the parameter estimates. The following are the common ones;
A) The standard error test
B) The students t-test
A) The Standard Error Test
This test first establishes the two hypotheses that are going to be tested which are commonly
known as the null and alternative hypotheses. The null hypothesis addresses that the sample is
coming from the population whose parameter is not significantly different from zero while the
alternative hypothesis addresses that the sample is coming from the population whose parameter
is significantly different from zero. The two hypotheses are given as follows:
H0: βi=0
H1: βi≠0
The standard error test is outlined as follows:
1. Compute the standard deviations of the parameter estimates using the above formula for
variances of parameter estimates. This is because standard deviation is the positive square root of
the variance.

√ δ 2U
¿
se( β 1 )=
∑ x 2i

√ δ 2U ∑ X 2i
¿
se( β 0 )=
n ∑ x 2i
2. Compare the standard errors of the estimates with the numerical values of the estimates and
make decision.
A) If the standard error of the estimate is less than half of the numerical value of the estimate,
¿
1 ¿
se( β i )< ( β i )
we can conclude that the estimate is statistically significant. That is, if 2 , reject the
null hypothesis and we can conclude that the estimate is statistically significant.
B) If the standard error of the estimate is greater than half of the numerical value of the estimate,
¿
1 ¿
se( β i )> ( β i )
the parameter estimate is not statistically reliable. That is, if 2 , ad conclude that
accept the null hypothesis and conclude that the estimate is not statistically significant.
Example 2.4: The regression shows the estimated regression of supply on price

28 | P a g e
¿
Y i =33 .75+3 . 25 X i
(8 . 3) (0 . 9) , where the numbers in parenthesis are standard errors. Test the statistical
significance of the estimates using standard error test.
Solution: The following information is given for decision.
¿
β 0 =33 .75
¿
se( β 0 )=8. 3
¿
β 1=3. 25
¿
se( β 1 )=0 . 9
¿ ¿ ¿

Testing for β 1 : Since the standard error of β 1 is less than half of the value of β 1 , we have to
¿

reject the null hypothesis and conclude that the parameter estimate β 1 is statistically significant.
¿ ¿ ¿

Testing for: β 0 Since the standard error of β 0 is less than half of the numerical value of β 0 , we
¿

have to reject the null hypothesis and conclude that β 0 is statistically significant.

B) The Student T-Test

In conditions where Z-test is not applied (in small samples), t-test can be used to test the
statistical reliability of the parameter estimates. The test depends on the degrees of freedom that
the sample has. The test procedures of t-test are similar with that of the z-test. The procedures are
outlined as follows;
1. Set up the hypothesis. The hypotheses for testing a given regression coefficient is given by:
H 0 : β i =0
H 1 : β i≠0 (2.33)
2. Determine the level of significance for carrying out the test. We usually use a 5% level
significance in applied econometric research.
3. Determine the tabulated value of t from the table with n-k degrees of freedom, where k is the
number of parameters estimated.
4. Determine the calculated value of t. The test statistic (using the t- test) is given by:

29 | P a g e
 β^ i
t cal =
se ( β^ i ) (2.34)
The test rule or decision is given as follows:
Reject H0 if
|t cal|≥tα /2 , n−k (2.35)
Example 3.6. Refer to Example 2.2. Is price of the commodity significant in determining the
quantity supplied of the commodity under consideration? Use =0.05.
The hypothesis to be tested is:
H 0 : β 1 =0
H 1 : β 1 ≠0
^ ^
As we found in Example 2.2, β 1=3. 25 , se ( β1 )=0. 8979
Then
β^ i 3. 25
t cal = = =3 . 62
se ( β^ i ) 0. 8979

The tabulated value as given in Example 2.2 is 2.228. since calculated t is greater than the
tabulated value, we reject the null hypothesis and conclude that the price of the commodity is
significant in determining the quantity supplied for the [Link] that when the degrees of
freedom is large, we can conduct t-test without consulting the t-table in finding the theoretical
value of t. This rule is known as “2t-rule”. The t-table shows that the values of t changes very

slowly if the degrees of freedom (n-k) are greater than 8. For example the value of
t 0.025 changes
from 2.30 (when n-k=8) to 1.96(when n-k=∞). The change from 2.30 to 1.96 is obviously very
slow. Consequently, we can ignore the degrees of freedom (when they are greater than 8) and say

that the theoretical value of

t cal is 2.0. Thus, a two tail test of a null hypothesis at 5% level of
significance can be reduced to the following rules.

1. If
t cal is greater than 2 or less than -2, we reject the null hypothesis

2. If
t cal is less than 2 or greater than -2, accept the null hypothesis.
2.6.2. Confidence Interval Estimation of the regression Coefficients

30 | P a g e
In the above section, we have seen how to test the reliability of parameter estimates. But one
thing that must be clear is that rejecting the null hypothesis does not mean that the parameter
estimates are correct estimates of the true population parameters. It means that the estimate
comes from the sample drawn from the population whose population parameter is significantly
different from zero. In order to define how close to the estimate the true parameter lies, we must
construct a confidence interval for the parameter. Like we constructed confidence interval
estimates for a given population mean, using the sample mean (in Introduction to Statistics), we
can construct 100(1- ) % confidence intervals for the sample regression coefficients. To do so
we need to have the standard errors of the sample regression coefficients. The standard error of a
given coefficient is the positive square root of the variance of the coefficient. Thus, we have
discussed that the formulae for finding the variances of the regression coefficients are given as.
^
Variance of the intercept (( β 0 )

∑ X i2
var( β^ 0 )=δ
u2
n∑ x 2
i
(2.36)
^
Variance of the slope (( β 1 )

1
var( β^ 1 )=δ 2
u
∑ x i2
(2.37)
Where,

∑ e i2
δ 2=
u n−k (2.38)
is the estimate of the variance of the random term and k is the number of parameters to be
estimated in the model.

The standard errors are the positive square root of the variances, as repeatedly defined above and
100(1- ) % confidence interval for the slope is:
31 | P a g e
¿ ¿ ¿ ¿
β 1−t α ( n−k )(se ( β 1 ))≤β 1 ≤β 1 +t α (n−k )( se ( β1 ) )
2 2

β 1= β^ 1 ±t α /2 ,n−k ( se( β^ 1 )) (2.39)

And for the intercept:

β 0 = β^ 0±t α /2 , n−k (se( β^ 0 )) (2.40)

Example 3.7: From example 2.2 above, determine the 95% confidence interval for the slope.

∑ xy 156
^
β= = =3 .23
48
∑ x2

β^ 0 =Ȳ − β^ 1 X̄ =63−(3 .25 )(9 )=33 .75

∑ e i2 387 387
δ 2= = = =38 .7
u n−k 12−2 10

1 1
var( β^ 1 )=δ 2 =38 .7 ( )=0 . 80625
u 48
∑ x2
The standard error of the slope is:

se( β^ 1 )=√ var( β^ 1 )=√ 0. 80625=0 .8979

The tabulated value of t for degrees of freedom 12-2=10 and /2=0.025 is 2.228.

Hence the 95% confidence interval for the slope is given by:

β^ 1=3. 25±(2 .228 )(0 . 8979)=3 . 25±2=( 3 .25−2 , 3 . 25+2 )= (1 . 25 , 5 . 25 )

2.7. Prediction Using Simple Linear Regression Model

32 | P a g e
Once estimated parameters are proved to be significant and valid, then it can be used to forecast
or predict future values of the objective variable. Prediction or forecasting future values of
economic variables such as demand, cost and so on is a crucial contribution of econometric
analysis in economic decision making. For instance, in fixing future price, determination of
production level, making investment decision and so on.

Hence, predicting the future values of the dependent variable is one of the key tasks in
econometric analysis. The estimated regression equation Y^i= α^ + β^ X i is used for predicting the
values of Y for a given values of X. To proceed with, let X 0 be the given value of X. Then we
predict the corresponding value of Y0 of Y by
Y^0= α^ + β^ X 0 ---------------------------------------------------------------------- (2.41)
The true value of Y is given by
Y 0=α + β X 0 +U 0 where U 0 is error term
Hence, the prediction error is
Y^0−Y 0 = ( α^ – α )+ ¿
Since E ( α^ – α )=0 , E ( ^β−β ) =0∧E (U ¿¿ 0)=0¿ , we have
E ( Y^0−Y 0 ) =0
This equation shows that the predictant given by equation (2.35) is unbiased. Note that the
predictant is unbiased in the sense that E ¿ since both Y^0∧Y 0 are random variables.
Example 2.8: From example 2.2, predict the value of Y when X is 45

Y^ i = 33 .75 + 3. 23 X i =33.75+ (3.23) (45) = 179.1

Solution:

That means when X assumes a value of 45, Y on average is expected to be 179.1.

33 | P a g e