200916 – Economic and Financial
Modelling
Week 2 :
The Simple Regression Model
Text: Chapter 4
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Definition of the simple linear
regression model
Explains variable y in terms of variable x
Dependent
variable Error term
Intercept Slope Independent
parameter variable
Definition of the simple linear
regression model examples
Example: house price and land size
Number of bathrooms,
bedrooms etc.
Measures the effect of land size on
house price, holding all other factors fixed
Example: a simple wage equation
Labour force experience,
tenure with current employer,
work ethic, intelligence etc.
Measures the change in hourly wage
given another year of education,
holding all other factors fixed
When is there a causal
interpretation?
Conditional mean independence assumption
The explanatory variable must not
contain information about the mean
of the unobserved factors
Example: wage equation
e.g. intelligence
The conditional mean independence assumption is unlikely to hold because
individuals with more education will also be more intelligent on average.
Population regression
function (PFR)
The conditional mean independence assumption implies that:
This means that the average value of the dependent variable can be
expressed as a linear function of the explanatory variable
Population regression
function (PFR) (cont.)
Population
regression function
For individuals with x = x2,
the average value of y is
β0 + β1 x
Deriving the ordinary least
squares estimates
In order to estimate the regression model one needs data
A random sample of n observations
If the random sample is n = 15, a scatterplot will be produced with
population regression function
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Deriving the ordinary least
squares estimates (cont. 1)
Fit as good as possible a regression line through the data points:
Fitted
e.g., the i-th
regression line
data point (xi ,yi )
Deriving the ordinary least
squares estimates (cont. 2)
Regression residuals
Minimise sum of squared regression residuals
Ordinary Least Squares (OLS) estimates
Example of simple regression obtained
using real data
CEO Salary and return on equity
Salary in Return on equity
$ thousands of CEO‘s firm
Fitted regression:
Intercept If return on equity increases by 1%
then salary is predicted to change by $18 501
Example of simple regression obtained
using real data (cont.)
Fitted regression
line (depends on
sample)
Unknown population
regression line
Properties of OLS on any
sample of data
Fitted values and residuals
Given and we can predict y
Fitted or predicted Deviations from regression
values line (= residuals)
Deviations from Correlation between Sample averages of y
regression line sum up to deviations and regressors and x lie on regression
zero is zero line
Example: CEO salary and return
on equity
E.g., CEO number 11‘s salary was
$875.372 lower than predicted
using information on his firm‘s
return on equity
Goodness-of-fit
How well does the explanatory variable explain the dependent variable?
Measures of variation
Total sum of squares, Explained sum of squares, Residual sum of squares,
represents total variation represents variation represents variation not
in dependent variable explained by regression explained by regression
Decomposition of total variation
Total Explained Unexplained
variation part part
Proving SST = SSE + SSR
Goodness-of-fit measure
(R-squared)
R-squared measures the fraction of the total variation that is explained by the
regression
• If the model does not contain an intercept parameter, then the measure R2 given
above is no longer appropriate
Intrepretation of R-squared
The fraction (percentage) of the sample variation in y that is explained by x
What does a low/high R-squared value mean?
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
CEO Salary and return on equity
The regression explains only 1.3 %
of the total variation in salaries
Caution: Using R-squared as the main gauge of success in econometrics can lead to
trouble
The effect of changing units of measurement
on OLS Statistics
Salary was measured in thousands of dollars (e.g. $857 000 = $857), ROE
was measured in percentage
What would happen if salary were measured in dollars and ROE in
percentage?
Would R-squared be affected?
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Incorporating nonlinearities:
semi-logarithmic form
Regression of log wages on years of eduction
Natural logarithm of wage
This changes the interpretation of the regression coefficient:
Fitted regression
The wage increases by 8.3 % for
every additional year of education (=
return to education)
Growth rate of wage is 8.3%
per year of education
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Unbiasedness of OLS:
assumptions
Assumption SLR.1 (Linear in parameters)
In the population, the relationship
between y and x is linear
Assumption SLR.2 (Random sampling)
The data is a random sample
drawn from the population
Each data point therefore follows
the population equation
Unbiasedness of OLS:
assumptions (cont.)
Assumption SLR.3 sample variation in the explanatory variable
The sample outcomes on
x are not all the same
Assumption SLR.4 Zero conditional mean
The error has an expected value of zero given
any value of the explanatory variable
If any of the assumptions fail, then the OLS estimators will be biased
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Estimating unbiased OLS estimators
The question is what the estimators will estimate on average and how large their
variability in repeated samples is:
Variances of the OLS estimators
Depending on the sample, the estimates will be nearer or farther away
from the true population values
How far can we expect our estimates to be away from the true
population values on average
Sampling variability is measured by the estimator‘s variances
Assumption SLR.5 (Homoskedasticity)
The value of the explanatory variable must contain no information about
the variability of the unobserved factors
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Graphical illustration of
homoskedasticity
The variability of the
unobserved influences does not
dependent on the value of the
explanatory variable
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
An example for heteroskedasticity:
wage and education
The variance of the unobserved
determinants of wages
increases
with the level of education
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Under the assumptions SR1-SR4 of the linear regression model,
the OLS estimators are unbiased.
Under the assumptions SR1-SR5 of the linear regression model,
the OLS estimators have the smallest variance of all linear and
unbiased estimators of β1 and β2. They are the Best Linear
Unbiased Estimators (BLUE).
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.