Econometrics I

Chapter 2
The Simple Regression Model
By Kefyalew Endale, Assistant Professor, Department of Economics,
Addis Ababa University
Email: ekefyalew@[Link]
October 2024
Main reference-Wooldrigde (2013). Introductory Econometrics, A
modern Approach, 5th edition

by Kefyalew Endale (AAU) 1

The Simple regression model: Introduction
• This model can be used to study the relationship between two variables (Y and X).
Examples Y X

1. Cumulative GPA Number hours spent on studying

2. Crop yield (kg/ha) Fertilizer used in kg/ha
3. Wage Years of schooling
• Three issues in formulating the regression of Y on X
• How do we allow for the other factors to affect Y?
• What is the functional relationship between Y and X?
• How can we be sure we are capturing the Ceterius paribus r/p between Y and X?

by Kefyalew Endale (AAU) 2

Introduction CONT’D
• Specifies the relationship between Y and X as a linear relationship
• 𝑌 = 𝛽0 + 𝛽1 𝑋 + 𝜀 … … … … … .2.1
• Equation (2.1) is called the two variable linear regression model or
bivariate regression model
• 𝛽1 -is the slope (which measures the impact of a unit change in X on Y)
• 𝛽0 -is the intercept which captures the value of Y when X takes a zero
value.
• Intercept doesn’t show relationships between X and Y but it is used in
predicting the values of Y for a given values of X

by Kefyalew Endale (AAU) 3

 Terminologies in Simple Linear Regression Model
• The most frequently used terminologies for Y and X are “dependent” and “Independent”
variables, respectively. But there are other terms for Y and X
Y X
Explained variable Explanatory variable
Response variable Control variable
Predicted variable Predictor variable
Regressand variable Regressor variable

• 𝜀-is the error term or disturbance in the relationship which represents factors other than X
that affect Y
• In the simple regression model such as the one in equation (2.1) all factors other than “X”
which affect “Y” are considered as “unobserved factors”

by Kefyalew Endale (AAU) 4

 Methods of Estimation: Ordinary Least Square (OLS)
1. OLS (Ordinary Least Squares)-is the most basic and most commonly used
regression technique
𝑌𝑖 = 𝛽0 + 𝛽1 𝑋𝑖 + 𝜀𝑖
• We want to estimate 𝑌𝑖 = 𝛽0 + 𝛽1 𝑋𝑖
• “^” or hat denotes a sample estimate of the unobservable true population
value
• OLS permits the estimation of 𝛽0 𝑎𝑛𝑑 𝛽1 such that the sum of squared
residuals (RSS) are minimized.
• 𝑒𝑖 = 𝑌𝑖 − 𝑌𝑖
• OLS minimizes 𝑛𝑖=1 𝑒𝑖2 𝑜𝑟 𝑚𝑖𝑛𝑖𝑚𝑖𝑧𝑒𝑠 𝑒12 + 𝑒22 + ⋯ + 𝑒𝑛2

by Kefyalew Endale (AAU) 5

OLS CONT’D
• Why we use OLS?
1. It is easy to compute the parameters (one can easily compute them
even by hand)
2. Theoretically appropriate: The method involves finding parameters
which minimizes the sum square of errors. Because we want the
difference between the actual and the predicted values of the
dependent variables to be as small as possible.
3. Its useful properties
a. The regression line passes through the averages of Y and X
b. The sum of the residuals is zero
c. OLS generated estimates can be said to be “best” under a set of restrictive
assumptions.
by Kefyalew Endale (AAU) 6
OLS CONT’D
• 𝑀𝑖𝑛 𝑛𝑖=1 𝑒𝑖2 , wrt 𝛽0 and 𝛽1 =𝑀𝑖𝑛 𝑛
𝑖=1(𝑌𝑖 − 𝛽0 − 𝛽1 𝑋𝑖 )2

𝜕𝑅𝑆𝑆 𝑛
• Focs, = -2 𝑖=1(𝑌𝑖 − 𝛽0 − 𝛽1 𝑋𝑖 ) =0 …………………………….(1)
𝜕𝛽0
𝜕𝑅𝑆𝑆 𝑛
= -2 𝑖=1 𝑌𝑖 − 𝛽0 − 𝛽1 𝑋𝑖 𝑋𝑖 =0 ……………………….(2)
𝜕𝛽1
From 1: 𝛽0 = 𝑌 − 𝛽1 𝑋…………(3)
From 2: 𝑌𝑖 𝑋𝑖 − 𝑛𝛽0 𝑋 − 𝛽1 𝑛𝑖=1 𝑋𝑖2 = 0 …….[4]
Substituting [3] in [4] and rearrangement yields

𝑛
𝑖=1 𝑌𝑖 𝑋𝑖 − 𝑛𝑌 𝑋
𝛽1 = 𝑛 2 2
𝑋
𝑖=1 𝑖 − 𝑛 𝑋
by Kefyalew Endale (AAU) 7
Example 1. regression
X of resale price of used refrigerator (Y) in USD on the
age of refrigerator in years (this is a hypothetical data)
𝑖 𝑖 - ( - )2
205 6 1230 1.9 3.61
195 6 1170 1.9 3.61
210 6 1260 1.9 3.61
340 2 680 -2.1 4.41
299 2 598 -2.1 4.41
230 5 1150 0.9 0.81
270 4 1080 -0.1 0.01
243 5 1215 0.9 0.81
340 1 340 -3.1 9.61
240 4 960 -0.1 0.01

)
=
=

257.2 4.1 9683 30.9

𝛽 1 -27.9
𝛽 0 371.6

by Kefyalew Endale (AAU) 8

. reg y x

Source SS df MS Number of obs = 10

F( 1, 8) = 118.53
Model 24057.8913 1 24057.8913 Prob > F = 0.0000
Residual 1623.70874 8 202.963592 R-squared = 0.9368
Adj R-squared = 0.9289
Total 25681.6 9 2853.51111 Root MSE = 14.247

y Coef. Std. Err. t P>|t| [95% Conf. Interval]

x -27.90291 2.562889 -10.89 0.000 -33.81295 -21.99288

_cons 371.6019 11.4329 32.50 0.000 345.2376 397.9663

by Kefyalew Endale (AAU) 9

 Assumptions about 𝜀 and prediction in SLR (Gasu-Markov assumption)
• Assumptions about 𝜀 [continued from equation 2.1in slide number 3]
1. 𝐸 𝜀 = 0: the average of the error term in the population is zero
This is not a restrictive assumption so long as the model has an intercept term.
2. X and 𝜀 are uncorrelated or Cov 𝑋𝜀 = 0 or their linear correlation is
zero.
3. The expected value of 𝜀 does not depend on the value of the regressors
(or X). i.e E 𝜀/𝑋 = 𝐸(𝜀)
• The 3rd assumption implies that 𝜀 is the same across all slices of the
population
E(𝜀/ X= 1 Yr of refrigerator use ) =E(𝜀/ X= 6 Yr of refregirator use)= 𝐸(𝜀)

by Kefyalew Endale (AAU) 10

Assumptions & Prediction CONT’D
• If the conditional mean of 𝜀 is zero or E[𝜀/𝑋]=0, then E[𝑌/𝑋]= 𝛽0 + 𝛽1 𝑋
• This equation says the average value of Y changes with X.
• It doesn’t say Y=𝛽0 + 𝛽1 𝑋

• For any given value of X, the distribution of Y is centered about 𝛽0 + 𝛽1 𝑋

by Kefyalew Endale (AAU) 11
Assumptions & Prediction CONT’D
• Eg. E[𝑌]=371.6-27.9X
• Then, when the years of service is 5 years, the expected resale value
becomes 371.6 – 27.9*5=$232.1.
• This does not mean that the resale value of every used refrigerator
used for 5 years is $232.1
• Some will value more than $232.1
• Others will value below $232.1
• Whether the actual resale value is above or below $232.1 depends on
the other factors such as brand type and current condition.

by Kefyalew Endale (AAU) 12

 Marginal Effects of independent variables
• The change in the dependent variable is given by
• ∆𝑌 = 𝛽0 + 𝛽1 ∆𝑋 + ∆𝜀………………………..2.2
• Ceterius paribus implies that all factors other than X are kept constant
(∆𝜀=0 in the case of simple regression).
• If there is another explanatory variable (Z) in addition to X, then
ceterius paribus assumption implies that ∆𝑍=0 and ∆𝜀 = 0
• Then, the marginal impact of X on Y is given by
Δ𝑦
• = 𝛽1
Δ𝑥
• Interpretation, as X changes by 1 unit, Y changes by 𝛽1 unit.

by Kefyalew Endale (AAU) 13

Marginal Effect CONT’D

• Eg1. 𝛽1 =-27.9 as the age of used refrigerator increases by 1 year, the resale
value of refrigerator decreases by $28.

• Eg2. Wheat Yield (kg/ha)= 1069 + 3.93 Fertilizer (kg/ha)+ 𝜀

t-ratio (10.06) (6.72)
𝛽0 =1069: It means when the quantity of fertilizer is 0, the wheat yield per hectare
of land is 1069kg
𝛽1 = 3.93 slope . It means as the farmer increases the quantity of fertilizer per
hectare of land by 1 kg, the wheat yield increases by 3.93 Kg.
Questions
1-by how many kilograms will the wheat yield increase if the farmer raises the
quantity of fertilizer from 20 (kg/ha) to 25(kg/ha)?
2. What is the predicted value of the wheat yield when fertilizer is 25 kg/ha
3. Is the coefficient of fertilizer statistically significant? Why?
by Kefyalew Endale (AAU) 14
Marginal Effects CONT’D
• Limitation of Simple Linear regression-it assumes that the impacts are linear.
• Raising fertilizer from 5kg/ha to 6kg/ha could have larger marginal effect
compared to raising it from 120kg/ha to 121kg/ha
• Eg2. 𝑤𝑎𝑔𝑒 = 𝛽0 + 𝛽1 𝑌𝑒𝑎𝑟𝑠 𝑜𝑓 𝑠𝑐ℎ𝑜𝑜𝑙𝑖𝑛𝑔 + 𝜀
• This model assumes linear impact of years of schooling on
wages. But in practice years of schooling has non-linear impacts
• For instance, increasing years of schooling from grade 7 to
grade 8 might have smaller impact on wage than raising it from
3rd year undergraduate dropout to a BA degree graduate.
• Solution-including non-linear terms such as squares and
interaction terms.

by Kefyalew Endale (AAU) 15

 Marginal Effects CONT’D
Interpretations of 𝛽1 in different models
1. Log-Log Model: both the dependent and independent variables are in
logarithm
l𝑛𝑌 = 𝛽0 + 𝛽1 𝑙𝑛𝑋 + 𝜀
𝑑[l𝑛𝑌] = 𝛽1 𝑑[𝑙𝑛𝑋]
𝑑 𝑑
= 𝛽1
𝑑
100 ∗
𝛽1 =
𝑑
100 ∗
Interpretation of 𝛽1 :
a. For 𝛽1 >0: a 1% increase in X is associated with a 𝛽1 % increase in Y
b. For 𝛽1 <0: a 1% increase in X is associated with a 𝛽1 % decrease in Y
by Kefyalew Endale (AAU) 16
 Marginal Effects CONT’D
2. Log-Level Model: Y is in logarithm whereas X is in Level
l𝑛𝑌 = 𝛽0 + 𝛽1 𝑋 + 𝜀
𝑑[l𝑛𝑌] = 𝛽1 𝑑𝑋
𝑑
= 𝛽1 𝑑𝑋
Multiplying both sides by 100 and rearranging them yields
𝑑𝑦
100∗ %∆𝑌
𝑦
100*𝛽1 = =
𝑑𝑥 𝑑𝑥
• Interpretation:
• For 𝛽1 >0; a one unit increase in X is associated with a 𝛽1 × 100 % increases
in Y
• For 𝛽1 <0; a one unit increase in X is associated with a 𝛽1 × 100% decrease
in Y by Kefyalew Endale (AAU) 17
 Marginal Effects CONT’D
3. Level-Log Model: Y is in level but X is in logarithm
𝑌 = 𝛽0 + 𝛽1 𝑙𝑛𝑋 + 𝜀
𝑑[𝑌] = 𝛽1 𝑑[𝑙𝑛𝑋]
𝑑𝑋
d𝑌 = 𝛽1
𝑋
𝑑𝑌
𝛽1 = 𝑑𝑋
𝑋
• Dividing both sides by 100 yields the following
𝜷 𝒅
=𝒅
∗
• Interpretation:
𝛽1
• For 𝛽1 >0; a 1% increase in X is associated with a unit increases in Y
100
𝛽1
• For 𝛽1 <0; a 1% increase in X is associated with a unit decrease in Y
100
by Kefyalew Endale (AAU) 18
Introduction to STATA
• STATA is a widely used statistical package for estimating relationships
between variables and evaluating program impacts.
• It has the following parts
• Variable Window-it displays the list of variables in the dataset
• Command window-used to execute a single STATA command
• Results window (interface)-displays the outcomes of the STATA commands
• Review window-shows the list of executed commands
• Dofile editor-helps to save and execute several stata commands at once.
• Data editor-used to edit the data in use
• Data browser-used to browse the data
• Variable manager-used to manage variables such as variable names and labels

by Kefyalew Endale (AAU) 19

Introduction to STATA CONT’D
Data Variable
Dofile editor
editor manager

Result
Data Window
Variable
browser
window
Review
window

20
by Kefyalew Endale (AAU)
 Introduction to STATA CONT’D
• Basic stata commands
• des-to describe variables in the dataset
• sum-to summarize the variables
• asdoc sum: to generate summary statistics in Microsoft word
• sort-to sort the variables in increasing or decreasing order
• Eg. Sort hhid
• label-to give value labels for the variables
• Eg. label var hhid “unique household identification number”
• Rename: to change the variable name
• Rename assetv assetvalue
• tab-to tabulate the variables
• Eg. tab toxen
• gen-to generate new variables from the existing variables
• Eg. gen hhagesq=hhage*hhage
gen lnyield =ln(yield)
• reg-to regress one variable on another
• Eg. reg yield fertha
by Kefyalew Endale (AAU) 21
STATA session (LAB)
• Using the wheat Yield data
1. describe the variables
2. Briefly interpret the summary of each variable (the number of observations,
averages, minimum, and maximums)
3. Generate the natural logarithms of yield and fertilizer variables (note: if a variable
has many zeros in its level, you should add one before transforming it into logarithm to
avoid missing values in the generated variable)
4. Estimate the following models for the regressions of yield (kg/ha) on fertilizer
(kg/ha)
a. The level-level model
b. Log-log model
c. Log-level model
d. Level-log models
5. Interpret the coefficient of fertilizer (kg/ha) for each of the estimated model

by Kefyalew Endale (AAU) 22

Examples of regression results
(1) (2) (3) (4)
VARIABLES level-level log-log log-level level-log
fertha 3.937*** 0.00256***
(6.720) (8.405)
lnfertha 0.122*** 167.8***
(5.757) (4.144)
Constant 1,070*** 6.626*** 6.795*** 903.0***
(10.06) (65.38) (123.0) (4.678)
Observations 522 516 516 522
R-squared 0.080 0.061 0.121 0.032
t-statistics in parentheses, *** p<0.01, ** p<0.05, * p<0.1

by Kefyalew Endale (AAU) 23

Interpretation of the coefficients
• Column (1): a 1 kg/ha increase in fertilizer use increases wheat yield
by 3.93 kg/ha, given all other factors kept constant
• Column (2): a one percent increase in fertilizer use increases wheat
yield by 0.12 percent, given all other factors kept constant
• Column (3): a 1 kg/ha increase in fertilizer use increases wheat yield
by 0.25%, given all other factors kept constant
• Column (4): a one percent increase in fertilizer use increases wheat
yield by 1.67kg/ha, given all other factors kept constant

by Kefyalew Endale (AAU) 24

 Expected values of 𝜷 𝐚𝐧𝐝𝜷
−1
• 𝛽1 = ( ′ ) ( ′ )
• We know =𝑌𝑖 − 𝑌 = 𝛽1 (𝑋𝑖 − 𝑋) + ε, HENCE
′ −1 ′ ′ −1 ′ ′ −1 ′
• 𝛽1 = ( ) (𝛽1 + ε)=( ) ( ) 𝛽1 +( ) ε
′ −1 ′
• 𝛽1 =𝛽1 +( ) ε
−1
-> E(𝛽1 )=𝛽1 +E(( ′ ) ′ ε) = 𝛽 (because E( ′ ε) = 0)
1
• 𝛽0 = 𝑌-𝛽1 𝑋=𝛽0 + 𝛽1 𝑋+𝑢 − 𝛽1 𝑋=𝛽0 +(𝛽1 -𝛽1 ) 𝑋
->E(𝛽0 )= 𝛽0 + 𝑋 E(𝛽1 -𝛽1 )=𝛽0 (Because E(𝛽1 -𝛽1 )=0)

by Kefyalew Endale (AAU) 25

 Variances of 𝛽0 and 𝛽1
• Y =𝛽0 + 𝛽1 𝑋 +e
• E(Y/X)=𝛽0 + 𝛽1 𝑋
• Var (Y/x) =E[𝑌 − 𝐸(𝑌/𝑋)]2 =E[𝑒/x − 𝐸(𝑒/ )]2 =E[𝑒/x − 0]2 =RSS/n-k
′ −1 ′ ′ −1 ′ ′ −1 ′ −1
• Var(𝛽1 )=E(𝛽1 -𝛽1 )sq=Var (( ) ε)=( ) ( ) var (ε/ ) =( ) RSS/n-k
−1
• Var (𝛽0 )=E(𝛽0 -𝛽0 )sq=var ((𝛽1 -𝛽1 ) 𝑋)=𝑋 2 𝑣𝑎𝑟(𝛽1 )=𝑋 2 ( ′ ) RSS/n-k
′ −1 1
• Note: ( ) =
(𝑋𝑖 −𝑋)2
• For the refrigerator example, (𝑋𝑖 − 𝑋)2 𝑜𝑟( ′
) =30.9, 𝑋= 4.1, and
RSS/n-k=1623.7/8=202.96,
• Hence, Var(𝛽1 )=202.96/30.9=6.56 and
Standard error of 𝛽1 = SE(𝛽1 ) = 6.56 =2.56
• Var(𝛽0 )=𝑋 2 𝑣𝑎𝑟(𝛽1 )= 4.1*4.1*6.56=110.27,
 Standard error of 𝛽0 = SE(𝛽0 ) = 110.27 = 10.7
by Kefyalew Endale (AAU) 26
Confidence Intervals
• Constructing confidence interval around your estimated coefficient(s).
• The formula used to construct a confidence interval for the regression
coefficient is given by:
(𝑡𝑤𝑜−𝑠𝑖𝑑𝑒𝑑)
𝐶𝐼 = 𝛽𝑘 +𝑡𝑐𝑣 ∗ 𝑆𝐸(𝛽𝑘 )
• In other words, the X% confidence interval around the estimated coefficient is
equal to the estimated coefficient plus or minus the product of the critical
value (using a two-sided test) times the standard error of the estimated
coefficient.
• Confidence intervals are usually either 90%, 95%, or 995 intervals
(𝑡𝑤𝑜−𝑠𝑖𝑑𝑒𝑑)
• For a 95% confidence interval, 𝑡𝑐𝑣 =2
• For example, the 95% CI for fertilizer in level-level yield regression is
calculated as 3.93 − 2 ∗ 0.586 , 3.93 + 2 ∗ 0.586 = 2.78, 5.09
• Which means that the researcher is 95% confident that the true coefficient of
fertilizer variables is between 2.78 and 5.09]
by Kefyalew Endale (AAU) 27
 Statistical Significance
• Estimated parameters are the estimates for the “true” relationships and
there is uncertainty associated with the estimates.
• The uncertainty about each parameter is measured by its standard error or
the standard deviation of the coefficient.
• The larger the standard errors, the larger the uncertainty in the parameter.
• Standard errors are used to test hypothesis.
• t-statistic which is the ratio of the coefficient to the standard error is used
in the decision whether to reject the null hypothesis or not.
Eg. The fertilizer regression
Null hypothesis (H0): Fertilizer has no effect on wheat yield (𝛽1 = 0)
Alternative hypothesis (H1): 𝛽1 >0
Rule of thumb: Reject the null of the absolute value of the t-statistic is greater than 2
or when the coefficient is at least twice as much as its standard deviation.

by Kefyalew Endale (AAU) 28

 Statistical Significance CONT’D
• In the wheat yield regression, the t-statistics for fertilizer is 6.72. Which
means the coefficient of fertilizer is 6.72 times the standard error. Hence,
the null hypothesis is rejected.
• The results suggest that fertilizer use has significant positive impact on crop
yield.
• Other methods to check statistical significance
• In addition to the t-ratio, you can use the information in the confidence interval and
P-values to conduct your hypothesis testing.
A. Using Confidence Interval: reject the null if there is no zero in the CI. For
example, there is no zero in the 95% CI [2.78, 5.09] for fertilizer variable
B. Using P-values: reject the null if the P-value is less than 0.1. Eg. The p-value for
fertilizer variable in the level-level regression is 0.000 (which is less than 0.1)

by Kefyalew Endale (AAU) 29

Statistically Testing for joint level of significance
• The F-ratio provide a test of the significance of all the independent
variables (other than the constant term) taken together.
• Ho: none of the regressors have significant coefficient
• H1: One or more of the regressors have significant coefficient.
• Reject the null if the P-value <0.1
𝐸𝑥𝑝𝑙𝑎𝑖𝑛𝑒𝑑 𝑠𝑢𝑚 𝑜𝑓 𝑠𝑞𝑢𝑎𝑟𝑒𝑠/(𝑘−1)
F=
𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 𝑠𝑢𝑚 𝑜𝑓𝑠𝑞𝑢𝑎𝑟𝑒𝑠/(𝑛−𝑘)
Where k-the number of estimated parameters in the model
n=the total number of observations in the model
- In the level-level yield regression: ESS=83483098.2, k=2, RSS=961441603,
and n=520
𝟖𝟑𝟒𝟖𝟑 𝟗𝟖. /( − )
- Thus, F= =45
961441603/(𝟓 − )
- P-value=0.000----->The null hypothesis should be rejected
by Kefyalew Endale (AAU) 30
 Goodness of Fit
• Evaluates how measures how well a regression model fits the data.
• The smaller the residual sum of squares, the better is the goodness of fit.
• In linear regressions, goodness of fit is measured by the coefficient of
determination (R-square)
• Total sum of squares=Estimated sum of squares + Residual sum of
squares
• TSS=ESS+RSS
• 𝑇𝑆𝑆 = 𝑛𝑖=1(𝑌𝑖 − 𝑌)2
• ESS= 𝑛𝑖=1(𝑌𝑖 − 𝑌)2
• RSS= 𝑛𝑖=1(𝑌𝑖 − 𝑌𝑖 )2
𝐸𝑆𝑆 𝑅𝑆𝑆
• R-sq= =1−
𝑇𝑆𝑆 𝑇𝑆𝑆
• OLS Maximizes R-square by minimizing the RSS
by Kefyalew Endale (AAU) 31
 𝑖 𝑖 - ( - )2 - ( - )2 𝑖=371.6-27.9 𝑖 ( 𝑖 ) ( 𝑖 )2 𝑒𝑖 = 𝑖- 𝑖 𝑒𝑖2

205 6 1230 1.9 3.61 -52.2 2724.84 204.20 -53.00 2808.89 0.80 0.64
195 6 1170 1.9 3.61 -62.2 3868.84 204.20 -53.00 2808.89 -9.20 84.66
210 6 1260 1.9 3.61 -47.2 2227.84 204.20 -53.00 2808.89 5.80 33.63
340 2 680 -2.1 4.41 82.8 6855.84 315.80 58.60 3434.08 24.20 585.59
299 2 598 -2.1 4.41 41.8 1747.24 315.80 58.60 3434.08 -16.80 282.27
230 5 1150 0.9 0.81 -27.2 739.84 232.10 -25.10 629.96 -2.10 4.41
270 4 1080 -0.1 0.01 12.8 163.84 260.00 2.80 7.85 10.00 99.98
243 5 1215 0.9 0.81 -14.2 201.64 232.10 -25.10 629.96 10.90 118.79
340 1 340 -3.1 9.61 82.8 6855.84 343.70 86.50 7482.42 -3.70 13.70
240 4 960 -0.1 0.01 -17.2 295.84 260.00 2.80 7.85 -20.00 400.04
10 10
10
( )2 ( )2 𝑒𝑖2
) 𝑖
= 𝑖=1 𝑖=1
= 𝑖=1

257.2 4.1 9683 30.9 25681.60 24052.87 1623.71

𝛽1 -27.9
𝛽0 371.6
ESS 24053 Rsq ESS/TSS 0.9368
RSS 1623.71 Rsq 1-RSS/TSS 0.9368
TSS 25676.6
by Kefyalew Endale (AAU) 32
 R-square and Adjusted R-Sq
• R-Square can be increased just by including more explanatory variables.
• Or an equation with more independent variables will always have a better (or
equal) fit as measured by R-square
Eg Model A: Weight=103.4+6.38Height, R-Sq=0.74
Model B: Weight =102.35+6.36Height+0.02PostOfficeBoxNumber, R-Sq =0.75
The R-Square result suggests that Model B is better than A.
• But in reality Post office Box number is unrelated to weight and hence Model B
is illegitimate.
• Hence, it is necessary to check the adjusted R-square in addition to the R-sq.

by Kefyalew Endale (AAU) 33

 R-square and Adjusted R-Sq CONT’D
𝑛−1
• Adjusted R2=1-(1-R2)
𝑛−𝑘−1
• Where n-k-1 is the degree of freedom. Degree of freedom tells the number of
information available for estimating the model.
-n-the total number of observations
-k-number of parameters for the regressors (x1, x2, …xk)
-1-for the constant term
-Rsq could increase when more regressors are included. But the degree of freedom
decreases as K increases.
-Hence Adjusted R2 penalizes the researcher for including irrelevant explanatory
variable.
-Adjusted R2 might increase, decrease or stay constant depending on the
contribution of the added variable to the R2 and the loss in the degree of freedom
(n-k-1).

by Kefyalew Endale (AAU) 34