Simple Linear Regression

Econometrics

Lecture Notes

S. Chanda

2026
Simple Linear Regression Lecture Notes

Contents

1 Introduction and Motivation 2

2 The Method of Least Squares 2

3 The Correlation Coefficient and Coefficient of Determination 3

3.1 Pearson’s Correlation Coefficient, r . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2 Coefficient of Determination, r2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.3 Adjusted r2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

4 Variance of the Error Term 4

5 Worked Example 1 (Fully Solved) 5

6 Worked Example 2 (Fully Solved) 7

7 Worked Example 3: Interpreting a Negative Relationship 8

8 Summary of Key Formulas 8

9 Past Paper Style Questions 10

1
Simple Linear Regression Lecture Notes

1 Introduction and Motivation

In many practical situations we want to know how one variable (the dependent variable, Y ) re-
sponds to changes in another variable (the independent or explanatory variable, X). Examples
include:

• Sales (Y ) versus advertising expenditure (X)

• Consumption (Y ) versus income (X)
• Crop yield (Y ) versus rainfall (X)

Simple linear regression models the relationship between X and Y using a straight line:

Yi = α + βXi + εi

where α is the intercept, β is the slope, and εi is the random error term capturing all factors affecting
Y other than X.

Classical Assumptions of the Linear Regression Model

1. The relationship between X and Y is linear in parameters.

2. The error term εi has mean zero: E(εi ) = 0.
3. Homoscedasticity: V ar(εi ) = σ 2 (constant) for all i.
4. No autocorrelation: Cov(εi , εj ) = 0 for i ̸= j.
5. X is non-stochastic (fixed in repeated sampling) and uncorrelated with εi .
6. Normality of error εi ∼ N (0, σ 2 ).

2 The Method of Least Squares

We estimate α and β by choosing values a (estimate of α) and b (estimate of β) that minimise the
sum of squared residuals:
Xn
SSE = (Yi − a − bXi )2
i=1

Minimising with respect to a and b gives the Ordinary Least Squares (OLS) normal equations,
whose solutions are:

OLS Formulas
P P P
XY − nX̄ Ȳ (Xi − X̄)(Yi − Ȳ ) xy
b1 = P 2 2
== P 2
= P 2 b0 = Ȳ − b1 X̄
X − nX̄ (Xi − X̄) x

where x = X − X̄ and y = Y − Ȳ are deviations from the sample means, and

P P
X Y
X̄ = , Ȳ =
n n

The fitted (estimated) regression line is Ŷ = a+bX, and the residual for observation i is ei = Yi −Ŷi .

A Note on Notation
Different textbooks use different symbols for the same two OLS coefficients. All of the following

2
Simple Linear Regression Lecture Notes

mean exactly the same thing:

a ≡ b0 ≡ α̂ (intercept) b ≡ b1 ≡ β̂ (slope)

So the slope is often written as

P P
(X − X̄)(Y − Ȳ ) xy
b1 = P 2
= P 2
(X − X̄) x

and the intercept as b0 = Ȳ − b1 X̄. This notation is common in econometrics texts that write
the model as Yi = b0 + b1 Xi + ei . Whichever symbols are used in a given exam paper, apply
exactly the same formulas and computational steps shown throughout these notes.

Interpretation:

• b (slope): the average change in Y for a one-unit increase in X.

• a (intercept): the predicted value of Y when X = 0 (only meaningful if X = 0 is within or

near the observed range of data).

3 The Correlation Coefficient and Coefficient of Determination

3.1 Pearson’s Correlation Coefficient, r

P P P P
xy n XY − X Y
r = pP P =p P
x2 y 2 [n X 2 − ( X)2 ][n Y 2 − ( Y )2 ]
P P P

r measures the strength and direction of the linear association between X and Y , and −1 ≤ r ≤ 1.

3.2 Coefficient of Determination, r2

(Ŷi − Ȳ )2
P
2 Explained Sum of Squares (ESS) SSE
r = =P 2
=1−
Total Sum of Squares (TSS) (Yi − Ȳ ) T SS

r2 is the proportion of the total variation in Y that is explained by the regression on X. It lies
between 0 and 1 (0 ≤ r2 ≤ 1); multiplying by 100 gives the percentage of variation explained.

3.3 Adjusted r2

Adding more explanatory variables to a regression can never decrease r2 , even if the added variables
are irrelevant. The adjusted r2 penalises the addition of regressors that do not genuinely improve
the fit:
 
2 2 n−1
r̄ = 1 − (1 − r )
n−k−1
where n = number of observations, k = number of explanatory (independent) variables (for
simple regression, k = 1).

Note: r̄2 ≤ r2 always, and r̄2 can be negative for a very poor fit, whereas r2 cannot.

3
Simple Linear Regression Lecture Notes

4 Variance of the Error Term

Since the true error variance σ 2 is unknown, it is estimated from the sample residuals using:
P 2 P 2 P
2 SSE ei y − b xy
σ̂ = s2e = = =
n−2 n−2 n−2

The divisor is n − 2 (not n) because two degrees of freedom are used up in estimating a and b.
s2e is also
p called the residual mean square or mean squared error (MSE). Its square root,
se = s2e , is the standard error of the regression (SER), used later to compute standard
errors of a and b for hypothesis testing.
Useful computational identity:

( xy)2
X X X P
2 2
SSE = y −b xy = y − P 2
x

4
Simple Linear Regression Lecture Notes

5 Worked Example 1 (Fully Solved)

Example 1: Advertising Expenditure and Sales

A company records advertising expenditure (X, in $000) and sales revenue (Y , in $000) over 6
months:

X 52 65 60 78 80 85
Y 10 15 14 20 24 28

Required: (i) r2 and adjusted r2 ; (ii) intercept a and slope b; (iii) variance of the error term.
Step 1: Basic sums.
X X
n = 6, X = 420, X̄ = 70, Y = 111, Ȳ = 18.5

Step 2: Deviations table.

X Y x = X − X̄ y = Y − Ȳ xy x2 y2
52 10 −18 −8.5 153.0 324 72.25
65 15 −5 −3.5 17.5 25 12.25
60 14 −10 −4.5 45.0 100 20.25
78 20 8 1.5 12.0 64 2.25
80 24 10 5.5 55.0 100 30.25
85 28 15 9.5 142.5 225 90.25
Totals 425.0 838 227.5

Step 2: Slope and intercept.

P
xy 425
b= P 2 = = 0.5072
x 838

a = Ȳ − bX̄ = 18.5 − (0.5072)(70) = −17.00

Ŷ = −17.00 + 0.5072 X
Step 3: Correlation, r2 , adjusted r2 .
P
xy 425 425
r = pP P =√ = = 0.9734
x 2 y 2 838 × 227.5 436.63

r2 = (0.9734)2 = 0.9474 (94.74%)

 
2 6−1
r̄ = 1 − (1 − 0.9474) = 1 − (0.0526)(1.25) = 0.9343 (93.43%)
6−1−1
Step 4: Variance of the error term.
X X
SSE = y2 − b xy = 227.5 − (0.5072)(425) = 227.5 − 215.59 = 11.91

SSE 11.91
σ̂ 2 = = = 2.978
n−2 4
Conclusion: About 94.7% of the variation in sales is explained by advertising expenditure;
the fit is very strong, and even after adjusting for sample size and the single regressor, 93.4% of
variation remains explained. On average, sales rise by $0.507 (thousand) for every $1 (thousand)
increase in advertising.

5
Simple Linear Regression Lecture Notes

Step-by-Step: Computing b1 Alone

It is worth isolating the slope calculation on its own, since it is the step students most often rush.
Using the Example 1 data again:
Finding b1 Step by Step
Step 1 – Find the means.
P P
X 420 Y 111
X̄ = = = 70 Ȳ = = = 18.5
n 6 n 6
Step 2 – Find the deviations (X − X̄) and (Y − Ȳ ) for every observation (see deviations
table above).
Step 3 – Multiply the two deviations in each row, then add them all up. This gives
the numerator:
X
(X − X̄)(Y − Ȳ ) = 153 + 17.5 + 45 + 12 + 55 + 142.5 = 425

Step 4 – Square each (X − X̄) value only, then add them up. This gives the denominator:
X
(X − X̄)2 = 324 + 25 + 100 + 64 + 100 + 225 = 838

Step 5 – Divide numerator by denominator.

425
b1 = = 0.5072
838
Step 6 – Use b1 to find b0 .

b0 = Ȳ − b1 X̄ = 18.5 − (0.5072)(70) = −17.00

! The numerator multiplies an X-deviation by a Y -deviation (cross-product) – it is not

squared.
! The denominator uses only X-deviations, and they are squared.
! A common exam mistake is accidentally squaring (Y − Ȳ ) instead of, or in addition to,
(X − X̄) in the denominator – always double check which variable is squared.

6
Simple Linear Regression Lecture Notes

6 Worked Example 2 (Fully Solved)

Example 2: Study Hours and Examination Score

A lecturer collects data on hours studied (X) and examination score out of 100 (Y ) for 5
students:

Student 1 2 3 4 5
X (hours) 2 4 5 6 8
Y (score) 50 55 65 70 80

Step 1: Sums.
X X
n = 5, X = 25, X̄ = 5, Y = 320, Ȳ = 64

Step 2: Deviations table.

X Y x y xy x2 y2
2 50 −3 −14 42 9 196
4 55 −1 −9 9 1 81
5 65 0 1 0 0 1
6 70 1 6 6 1 36
8 80 3 16 48 9 256
Totals 105 20 570

Step 3: Slope and intercept.

105
b= = 5.25 a = 64 − (5.25)(5) = 64 − 26.25 = 37.75
20

Ŷ = 37.75 + 5.25X
Step 4: Correlation and r2 .
105 105 105
r=√ =√ = = 0.9834
20 × 570 11400 106.77

r2 = 0.9671 (96.71%)
Step 5: Adjusted r2 (n = 5, k = 1).
 
2 4
r̄ = 1 − (1 − 0.9671) = 1 − (0.0329)(1.333) = 0.9561 (95.61%)
3

Step 6: Variance of the error term.

SSE = 570 − (5.25)(105) = 570 − 551.25 = 18.75

18.75 18.75
σ̂ 2 =
= = 6.25
5−2 3
Interpretation: Each additional hour of study is associated with a 5.25-point increase in
examination score, on average. About 96.7% of the variation in scores is explained by study
hours.

7
Simple Linear Regression Lecture Notes

7 Worked Example 3: Interpreting a Negative Relationship

Example 3: Price and Quantity Demanded

The price of a good (X, $) and quantity demanded (Y , units) over 5 weeks:

X 10 12 15 18 20
Y 90 80 65 55 40

X X
n = 5, X = 75, X̄ = 15, Y = 330, Ȳ = 66

X Y x y xy x2 y2
10 90 −5 24 −120 25 576
12 80 −3 14 −42 9 196
15 65 0 −1 0 0 1
18 55 3 −11 −33 9 121
20 40 5 −26 −130 25 676
Totals −325 68 1570

−325
b= = −4.779 a = 66 − (−4.779)(15) = 66 + 71.69 = 137.69
68
Ŷ = 137.69 − 4.779X

−325 −325
r=√ = = −0.9950 r2 = 0.9901 (99.01%)
68 × 1570 326.65
 
2 4
r̄ = 1 − (1 − 0.9901) = 0.9868 (98.68%)
3
SSE = 1570 − (−4.779)(−325) = 1570 − 1553.2 = 16.8
16.8
σ̂ 2 = = 5.6
3
Note: The negative slope confirms the law of demand — as price rises, quantity demanded
falls. The negative r reflects an inverse (but still very strong) linear relationship; r2 is unaffected
by the sign of r.

8 Summary of Key Formulas

8
Simple Linear Regression Lecture Notes

Quantity Formula
P P P P
xy n XY − X Y
Slope b= =
x2 n X 2 − ( X)2
P P P

Intercept a = Ȳ − bX̄
P
xy
Correlation coefficient r = pP P
x 2 y2
ESS SSE
Coefficient of determination r2 = =1−
T SS T SS 
n−1
Adjusted r2 2 2
r̄ = 1 − (1 − r )
n−k−1
P 2 P
Error sum of squares SSE = y − b xy
SSE
Variance of error term σ̂ 2 =
n−2
√
Standard error of regression se = σ̂ 2

9
Simple Linear Regression Lecture Notes

9 Past Paper Style Questions

Question 1 (Past Paper Style — 13 marks)

The following data relate to the number of years of experience (X) and monthly salary in $’00
(Y ) of 6 employees at a firm:

X 1 3 4 6 8 9
Y 12 18 20 28 34 38

Required:

(i) Compute r2 and the adjusted r2 . (4 marks)

(ii) Compute the intercept and slope coefficients. (5 marks)

(iii) Compute the variance of the error term. (4 marks)

Question 2 (Past Paper Style — 15 marks)

A researcher wants to study the relationship between the number of hours of training (X)
received by machine operators and the number of defective items (Y ) they produce per day:

X 5 8 10 12 15 18
Y 22 18 15 11 8 4

Required:

(i) State and briefly explain three assumptions of the classical linear regression model. (3
marks)

(ii) Estimate the regression equation of Y on X. (5 marks)

(iii) Compute and interpret r, r2 , and adjusted r2 . (5 marks)

(iv) Estimate the variance of the error term and comment on the goodness of fit. (2 marks)

Question 3 (Past Paper Style: Short Answer, 6 marks)

(a) Distinguish between r2 and adjusted r2 , explaining why the latter is preferred when com-
paring models with different numbers of explanatory variables. (3
marks)

(b) Explain, with reference to degrees of freedom, why the variance of the error term in simple
linear regression is divided by (n − 2) rather than n. (3 marks)

10