Chapter 3
Further issues with
the classical linear regression model
1
Meaning of "Linear" Regression
Interpreting coefficients : Case 1
One unit increase in x will change y by 1 units
regress mpg weight foreign,r
Linear regression Number of obs = 74
F( 2, 71) = 73.81
Prob > F = 0.0000
R-squared = 0.6627
Root MSE = 3.4071
Robust
mpg Coef. Std. Err. t P>t [95% Conf. Interval]
weight -.0065879 .0005462 -12.06 0.000 -.007677 -.0054988
foreign -1.650029 1.132566 -1.46 0.150 -3.908301 .6082424
_cons 41.6797 1.797553 23.19 0.000 38.09548 45.26392
N:B: an increase in weight by 1lb decreases meleage by 0.0066
Interpreting coefficients : Case 2
One per cent increase in x will change y by B1/100 units
gen lnweight = ln(weight)
regress mpg lnweight foreign,r
Linear regression Number of obs = 74
F( 2, 71) = 87.61
Prob > F = 0.0000
R-squared = 0.6886
Root MSE = 3.2737
Robust
mpg Coef. Std. Err. t P>t [95% Conf. Interval]
lnweight -19.95213 1.507408 -13.24 0.000 -22.95781 -16.94644
foreign -2.05233 1.048791 -1.96 0.054 -4.143561 .0389006
_cons 181.1005 12.17115 14.88 0.000 156.8319 205.3691
N:B: 1% increase in weight will decrease mpg by 0.19 units
Interpreting coefficients : Case 3
One unit increase in x will change y by 100 x B1 percent
gen lnmpg = ln(mpg)
regress lnmpg weight foreign,r
Linear regression Number of obs = 74
F( 2, 71) = 98.46
Prob > F = 0.0000
R-squared = 0.7349
Root MSE = .13583
Robust
lnmpg Coef. Std. Err. t P>t [95% Conf. Interval]
weight -.0003189 .000023 -13.86 0.000 -.0003647 -.000273
foreign -.1028503 .0429981 -2.39 0.019 -.188586 -.0171145
_cons 4.017887 .0721133 55.72 0.000 3.874097 4.161676
Increase in weight by 1lbs will decrease mpg by 0.03189 %
Interpreting coefficients: Case 4
One per cent increase in x will change y by 1 percent
The B1 can be interpreted as elasticity– the responsiveness of y
to a 1% increase in x
regress lnmpg lnweight foreign,r
Linear regression Number of obs = 74
F( 2, 71) = 94.22
Prob > F = 0.0000
R-squared = 0.7400
Root MSE = .13452
Robust
lnmpg Coef. Std. Err. t P>t [95% Conf. Interval]
lnweight -.9460359 .069638 -13.59 0.000 -1.08489 -.8071817
foreign -.1154278 .0418563 -2.76 0.007 -.198887 -.0319686
_cons 10.60704 .558801 18.98 0.000 9.492819 11.72125
A 1% increase in weight will decrease mpg by 0.95%
Goodness of Fit Statistics
We would like some measure of how well our regression model actually fits
the data.
We have goodness of fit statistics to test this: i.e. how well the sample
regression function (srf) fits the data.
The most common goodness of fit statistic is known as R2. One way to define
R2 is to say that it is the square of the correlation coefficient between y and .
For another explanation, recall that what we are interested in doing y$ is
explaining the variability of y about its mean value, , i.e. the total sum of
squares, TSS:
TSS yt y
2
We can split the TSS into two parts, the part which we have explained (known
as the explained sum of squares, ESS) and the part which we did not explain
using the model (the RSS).
Goodness of Fitness Statistics (cont’d)
That is, TSS = ESS + RSS
ty y 2
tˆ
y y 2
uˆt2
t t t
Our goodness of fit statistic is
ESS
R2
TSS
But since TSS = ESS + RSS, we can also write
ESS TSS RSS RSS
R2 1
TSS TSS TSS
R2 must always lie between zero and one. To understand this, consider two
extremes
RSS = TSS i.e. ESS = 0 so R2 = ESS/TSS = 0
ESS = TSS i.e. RSS = 0 so R2 = ESS/TSS = 1
8
The Limit Cases: R2 = 0 and R2 = 1
yt
yt
xt xt
9
Problems with R2 as a Goodness of Fit Measure
There are a number of them:
1. R2 is defined in terms of variation about the mean of y so that if a
model is reparameterised (rearranged) and the dependent variable
changes, R2 will change.
2. R2 never falls if more regressors are added. to the regression, e.g.
consider:
Regression 1: y = 1 + 2x2 + 3x3 + u
Regression 2: y = 1 + 2x2 + 3x3 + 4x4 + u
R2 will always be at least as high for regression 2 relative to regression 1.
3. R2 quite often takes on values of 0.9 or higher for time series
regressions.
10