Topic 2: OLS estimation

PG Econometrics (Computation)

Toshiaki Aizawa

Acknowledgement
• This note is mainly based on the course text book.
– Florian Heiss “Using R for Introductory Econometrics” Createspace Independent Pub, 2016
∗ Free version available on the web;
∗ [Link]

OLS estimation
Simple OLS Regression
• We are concerned with estimating the population parameters β0 and β1 of the simple linear regression
model
y = β0 + β1 x + u
from a random sample of y and x.
• The ordinary least squares (OLS) estimators are

β̂0 = y − β̂1 x
Cov(x, y)
β̂1 =
V ar(x)

OLS regression line

• Based on these estimated parameters, the OLS regression line is

ŷ = β̂0 + β̂1 x

• For a given sample, we just need to calculate the four statistics y, x, Cov(x, y), and V ar(x) and plug
them into these equations.

Example 2.1 -OLS regression line

• For the population of chief executive officers, let y be annual salary (salary) in thousands of dollars.
• Let x be the average return on equity (roe) for the CEO’s firm for the previous three years.
– (Return on equity is defined in terms of net income as a percentage of common equity.)
– For example, if roe=10, then average return on equity is 10 percent.

1
 • To study the relationship between this measure of firm performance and CEO compensation, we
postulate the simple model
salary = β0 + β1 roe + u
.
• The data set [Link] contains information on 209 CEOs for the year 1990.
# Read Data
library(wooldridge)
data(ceosal1, package='wooldridge')
dim(ceosal1)

## [1] 209 12

• In this sample, the average annual salary is $1,281,120, with the smallest and largest being $223,000
and $14,822,000, respectively.
# Descriptive statistics
mean(ceosal1$salary)

## [1] 1281.12
min(ceosal1$salary)

## [1] 223
max(ceosal1$salary)

## [1] 14822

• Our model is
salary = β0 + β1 roe + u
where salary is the salary of a CEO in thousand dollars and roe is the return on investment in percent.
• We also calculate the four statistics so we can reproduce the OLS formulas by hand.
data(ceosal1, package='wooldridge')

# ingredients to the OLS formulas

cov(ceosal1$roe,ceosal1$salary)

## [1] 1342.538
var(ceosal1$roe)

## [1] 72.56499
mean(ceosal1$salary)

## [1] 1281.12
mean(ceosal1$roe)

## [1] 17.18421

2
 β̂0 = y − β̂1 x
Cov(x, y)
β̂1 =
V ar(x)

# manual calculation of OLS coefficients

( b1hat <- cov(ceosal1$roe,ceosal1$salary)/var(ceosal1$roe) )

## [1] 18.50119
( b0hat <- mean(ceosal1$salary) - b1hat*mean(ceosal1$roe) )

## [1] 963.1913

Linear model
• If the values of the dependent variable are stored in the vector y and those of the regressor are in the
vector x, we can calculate the OLS coefficients as
lm( y ~ x )

• The name of the command lm comes from the abbreviation of linear model.
• Its argument y~x is called a formula in R lingo.
data(ceosal1, package='wooldridge')
# OLS regression
lm( salary ~ roe, data=ceosal1 )

##
## Call:
## lm(formula = salary ~ roe, data = ceosal1)
##
## Coefficients:
## (Intercept) roe
## 963.2 18.5
• lm returns its results in a special version of a list.

• We can store these results in an object using code like:

myolsres <- lm( y ~ x )

• This will create an object with the name myolsres.

Example 2.2 -Linear model

• We are using the data set [Link].
• We are interested in studying the relation between education and wage, and our regression model is

wage = β0 + β1 education + u.
data(wage1, package='wooldridge')
# OLS regression:
lm(wage ~ educ, data=wage1)

3
##
## Call:
## lm(formula = wage ~ educ, data = wage1)
##
## Coefficients:
## (Intercept) educ
## -0.9049 0.5414
• Does the sign of educ make sense to you?

Regression line plot

• Given the results from a regression, plotting the regression line is straightforward.
• The command abline(...) can add a line to a graph.

Example 2.3 -Regression line plot

• This command demonstrates how to store the regression results in a variable CEOregres and then use
it as an argument to abline to add the regression line to the scatter plot.
data(ceosal1, package='wooldridge')
# OLS regression
CEOregres <- lm( salary ~ roe, data=ceosal1 )

# Scatter plot (restrict y axis limits)

plot(ceosal1$roe, ceosal1$salary, ylim=c(0,4000))
# Add OLS regression line
abline(CEOregres)

4
 3000
ceosal1$salary

1000
0

0 10 20 30 40 50

ceosal1$roe

Coefficients, Fitted Values, and Residuals

• The object returned by lm contains all relevant information on the regression.
• After defining the regression results object CEOregres, we can see the names of its components and
access the first component coefficients with
names(CEOregres)
CEOregres$coefficients

• Another way to interact with objects like this is through generic functions.
• As an example, the number of observations n is returned with nobs(myolsres) if the regression results
are stored in the object myolsres.
nobs(CEOregres)

## [1] 209

• Obviously, we are interested in the OLS coefficients.

• They can be obtained as myolsres$coefficients.
– An alternative is the generic function: coef(myolsres).
CEOregres$coefficients

## (Intercept) roe
## 963.19134 18.50119

5
coef(CEOregres)

## (Intercept) roe
## 963.19134 18.50119

• The coefficient vector has names attached to its elements.

• The name of the intercept parameter β̂0 is “(Intercept)” and the name of the slope parameter β̂1 is the
variable name of the regressor x.
• In this way, we can access the parameters separately.
• Given these parameter estimates, calculating the predicted values ŷi and residuals ûi for each observation
i = 1, ..., n is easy:

ŷi = β̂0 + β̂1 xi

ûi = yi − ŷi

Example 2.4 -Coefficients, Fitted Values, and Residuals-

• If the values of the dependent and independent variables are stored in the vectors y and x, respectively,
we can estimate the model and do the calculations of these equations for all observations jointly using
the code:
myolsres <- lm( y ~ x )
bhat <- coef(myolsres)
yhat <- bhat["(Intercept)"] + bhat["x"] * x
uhat <- y - yhat

• We can also use a more black-box approach which will give exactly the same results using the generic
functions fitted and resid on the regression results object:
myolsres <- lm( y ~ x )
bhat <- coef(myolsres)
yhat <- fitted(myolsres)
uhat <- resid(myolsres)

Example 2.5 -Adding logarithmic terms to the model

• We study the relationship between the sales of a firm and the salary of its CEO using a log-log
specification:
data(ceosal1, package='wooldridge')
# Estimate log-log model
lm( log(salary) ~ log(sales), data=ceosal1 )

##
## Call:
## lm(formula = log(salary) ~ log(sales), data = ceosal1)
##
## Coefficients:
## (Intercept) log(sales)
## 4.8220 0.2567

6
 • How can we interpret the coefficients?

Multiple Regression estimation

• Consider the population regression model:

y = β0 + β1 x 1 + β 2 x 2 + β3 x 3 + · · · + βk x k + u

• Suppose the variables y, x1 , x2 , x3 , ... contain the respective data of our sample.
• We estimate the model parameters by OLS using the command
lm(y ~ x1+x2+x3+...)

• The tilde ~ again separates the dependent variable from the regressors which are now separated using a
+ sign.

• We can add options as before.

– For example if the data are contained in a data frame df, we should add the option data=df.
• The constant is again automatically added unless it is explicitly suppressed using lm(y ~
0+x1+x2+x3+...).
• We can store the estimation results in a variable myres using the code myres <- lm(...) and then
use this variable for further analyses.
• For a typical regression output including a coefficient table, call summary(myres).

Example 2.6 -Multiple Regression estimation-

• This example relates the college GPA (colGPA) to the high school GPA (hsGPA) and achievement test
score (ACT) for a sample of 141 students.
data(gpa1, package='wooldridge')
myres <- lm(colGPA ~ hsGPA+ACT+skipped, data=gpa1)

# Display full table:

(sumres <- summary(myres))

##
## Call:
## lm(formula = colGPA ~ hsGPA + ACT + skipped, data = gpa1)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.85698 -0.23200 -0.03935 0.24816 0.81657
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.38955 0.33155 4.191 4.95e-05 ***
## hsGPA 0.41182 0.09367 4.396 2.19e-05 ***
## ACT 0.01472 0.01056 1.393 0.16578
## skipped -0.08311 0.02600 -3.197 0.00173 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.3295 on 137 degrees of freedom
## Multiple R-squared: 0.2336, Adjusted R-squared: 0.2168
## F-statistic: 13.92 on 3 and 137 DF, p-value: 5.653e-08

# Manually confirm the formulas:

regtable <- sumres$coefficients
regtable

## Estimate Std. Error t value Pr(>|t|)

7
## (Intercept) 1.38955383 0.33155354 4.191039 4.950269e-05
## hsGPA 0.41181617 0.09367422 4.396260 2.192050e-05
## ACT 0.01472023 0.01056487 1.393319 1.657799e-01
## skipped -0.08311314 0.02599853 -3.196840 1.725431e-03

# Extract coefficients and SE

bhat <- regtable[,1]
se <- regtable[,2]
# Reproduce t statistic
( tstat <- bhat / se )

## (Intercept) hsGPA ACT skipped

## 4.191039 4.396260 1.393319 -3.196840

Pipe Operators in R
What Is a Pipe Operator?
A pipe operator passes the result on its left-hand side to the function on its right-hand side.
Conceptually:
x %>% f()

is equivalent to
f(x)

The key advantage is that code can be read from top to bottom, following the logic of the analysis.

The %>% Pipe

The most widely used pipe in R comes from the tidyverse package.

Example

data %>%
filter(x > 0) %>% # select the subset of the data
mutate(y = log(x)) # make a new variable

Using . (the Placeholder)

By default, the left-hand side is passed to the first argument of the function.
If this is not desired, the placeholder . can be used.
df %>%
lm(y ~ x, data = .)

Here, . represents the object coming from the left side of the pipe.

The Base R Pipe |>

Since R 4.1.0, R includes a native pipe operator: |>.

8
Example

x |>
log() |>
mean()

This is equivalent to:

mean(log(x))

A Typical Data Analysis Example

df %>%
filter(gender == "male") %>%
group_by(education) %>%
summarise(mean_income = mean(income))

Interpretation:
“Using df → restrict to males → group by education → compute mean income.”
For more details about tidyverse and pipe operators, see Wickham et al. (2024)