A Comprehensive Guide to Simple Linear Regression

Formulas
Statistical Reference

November 17, 2025

Contents
1 Basic Statistical Measures (Univariate) 2
1.1 Mean (x̄) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Sample Variance (s2x ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Sample Standard Deviation (sx ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3.1 Example: Basic Stats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Bivariate Statistical Measures 2

2.1 Sample Covariance (Cov(x, y) or sxy ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Pearson Correlation Coefficient (r) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Simple Linear Regression (Ordinary Least Squares) 3

3.1 The Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2 Method 1: Raw Data Formulas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2.1 Slope (β̂1 or b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2.2 Intercept (β̂0 or a) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.3 Method 2: Deviation / Covariance Formulas . . . . . . . . . . . . . . . . . . . . . . . 3
3.3.1 Slope (β̂1 or b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.3.2 Intercept (β̂0 or a) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.4 Coefficient of Determination (R2 ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.4.1 Conceptual Formula (ANOVA) . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.4.2 Simple Calculation Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

4 Comprehensive Worked Example 5

4.1 Step 1: Create a Calculation Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2 Step 2: Basic Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.3 Step 3: Solve using Method 1 (Raw Data) . . . . . . . . . . . . . . . . . . . . . . . . . 5
4.4 Step 4: Solve using Method 2 (Deviation Data) . . . . . . . . . . . . . . . . . . . . . . 6
4.5 Step 5: Calculate Correlation (r) and R2 . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.5.1 Correlation (r) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.5.2 Coefficient of Determination (R2 ) . . . . . . . . . . . . . . . . . . . . . . . . . 6

1
Simple Linear Regression

1 Basic Statistical Measures (Univariate)

Given a set of data x = {x1 , x2 , . . . , xn } with n observations.

1.1 Mean (x̄)

The average of the dataset. Pn
i=1 xi
x̄ =
n

1.2 Sample Variance (s2x )

A measure of how spread out the data is from its mean. We use the sample variance (dividing
by n − 1) as it’s an unbiased estimator of the population variance.
Pn
(xi − x̄)2
s2x = i=1
n−1

1.3 Sample Standard Deviation (sx )

The square root of the variance, returning the spread to the original units.
sP
n 2
i=1 (xi − x̄)
p
sx = s2x =
n−1

1.3.1 Example: Basic Stats

Data: x = {2, 4, 6}

• n=3
2+4+6 12
• x̄ = 3 = 3 =4
(2−4)2 +(4−4)2 +(6−4)2 (−2)2 +02 +22 4+0+4
• s2x = 3−1 = 2 = 2 =4
√
• sx = 4=2

2 Bivariate Statistical Measures

Given a paired dataset (x, y) = {(x1 , y1 ), (x2 , y2 ), . . . , (xn , yn )}.

2.1 Sample Covariance (Cov(x, y) or sxy )

Measures how two variables move together.
Pn
i=1 (xi − x̄)(yi − ȳ)
Cov(x, y) = sxy =
n−1

Page 2
Simple Linear Regression

2.2 Pearson Correlation Coefficient (r)

Standardizes the covariance to a unitless value between -1 and 1, measuring the strength and
direction of the *linear* relationship.
Formula using Covariance:
Cov(x, y) sxy
r= =
sx sy sx sy
Computational Formula (Raw Data):
P P P
n xi yi − ( xi )( yi )
r=q P
[n x2i − ( xi )2 ][n yi2 − ( yi )2 ]
P P P

3 Simple Linear Regression (Ordinary Least Squares)

3.1 The Model
We want to find the best-fitting line
Pŷ = a + bx (or ŷ = β̂0 + β̂1 x) that minimizes the sum of
squared errors (SSE), where SSE = (yi − ŷi )2 .

• β̂0 or a: The y-intercept (predicted value of y when x = 0).

• β̂1 or b: The slope (change in y for a one-unit change in x).

3.2 Method 1: Raw Data Formulas

These formulas are fast for calculation but hide the statistical concepts.

3.2.1 Slope (β̂1 or b)

P P P
n xi yi − ( xi )( yi )
β̂1 = b =
n x2i − ( xi )2
P P

3.2.2 Intercept (β̂0 or a)

The intercept formula is the same for both methods. It is calculated *after* the slope.

β̂0 = a = ȳ − β̂1 x̄

3.3 Method 2: Deviation / Covariance Formulas

These formulas show the ”why” behind the calculation.

3.3.1 Slope (β̂1 or b)

This formula is derived from minimizing the SSE.

Pn
(x − x̄)(yi − ȳ)
Pn i
β̂1 = b = i=1 2
i=1 (xi − x̄)

Note the relationship to covariance and variance:

Cov(x, y) × (n − 1) sxy
β̂1 = = 2
Var(x) × (n − 1) sx

Page 3
Simple Linear Regression

Also, note the relationship to the correlation coefficient r:

 
sy
β̂1 = r
sx

3.3.2 Intercept (β̂0 or a)

(Same as Method 1)
β̂0 = a = ȳ − β̂1 x̄

3.4 Coefficient of Determination (R2 )

R2 (or ”R-squared”) measures the proportion of the variance in the dependent variable (y) that
is predictable from the independent variable (x). It is a value between 0 and 1.

3.4.1 Conceptual Formula (ANOVA)

(ŷi − ȳ)2
P
SSR
2
R = =P
SST (yi − ȳ)2
Where:

(yi − ȳ)2
P
• SST (Total Sum of Squares) =

• SSR (Regression Sum of Squares) = (ŷi − ȳ)2

P

• SSE (Error Sum of Squares) = (yi − ŷi )2

P

• And SST = SSR + SSE

3.4.2 Simple Calculation Method

For **simple** linear regression (one x variable), R2 is simply the square of the Pearson cor-
relation coefficient (r).
R2 = r 2

Page 4
Simple Linear Regression

4 Comprehensive Worked Example

Let’s use the data from your lecture notes (page 20).
• x = {1, 2, 3, 4, 5} (Advertising)
• y = {1, 1, 2, 2, 4} (Sales)

4.1 Step 1: Create a Calculation Table

This table contains all sums needed for both methods.

xi yi xi − x̄ yi − ȳ (xi − x̄)2 (yi − ȳ)2 (xi − x̄)(yi − ȳ) x2i yi2 xi yi

1 1 -2 -1 4 1 2 1 1 1
2 1 -1 -1 1 1 1 4 1 2
3 2 0 0 0 0 0 9 4 6
4 2 1 0 1 0 0 16 4 8
5 4 2 2 4 4 4 25 16 20
P P P P P P P P P P
= 15 = 10 =0 =0 = 10 =6 =7 = 55 = 26 = 37

Table 1: Calculation Table for Regression

4.2 Step 2: Basic Statistics

• n=5
P
xi 15
• x̄ = n = 5 =3
P
• ȳ = nyi = 10 5 =2
2
P
(x −x̄) 10
• s2x = n−1 i
= 4 = 2.5
√
• sx = 2.5 ≈ 1.581
(yi −ȳ)2
P
6
• s2y = n−1 = 4 = 1.5
√
• sy = 1.5 ≈ 1.225

4.3 Step 3: Solve using Method 1 (Raw Data)

P P P
n xi yi − ( xi )( yi )
β̂1 =
n x2i − ( xi )2
P P

5(37) − (15)(10)
=
5(55) − (15)2
185 − 150 35
= = = 0.7
275 − 225 50

β̂0 = ȳ − β̂1 x̄
= 2 − (0.7)(3)
= 2 − 2.1 = −0.1
Regression Equation: ŷ = −0.1 + 0.7x (This matches your notes on page 20)

Page 5
Simple Linear Regression

4.4 Step 4: Solve using Method 2 (Deviation Data)

P
(xi − x̄)(yi − ȳ)
β̂1 =
(xi − x̄)2
P

7
= = 0.7
10

β̂0 = ȳ − β̂1 x̄
= 2 − (0.7)(3) = −0.1
Regression Equation: ŷ = −0.1 + 0.7x (The result is identical)

4.5 Step 5: Calculate Correlation (r) and R2

4.5.1 Correlation (r)

Using the raw data formula:

P P P
xi yi − ( xi )( yi )
n
r=q P
[n x2i − ( xi )2 ][n yi2 − ( yi )2 ]
P P P

5(37) − (15)(10)
=p
[5(55) − (15)2 ][5(26) − (10)2 ]
35 35 35
=p =√ ≈ ≈ 0.9036
[50][30] 1500 38.73

4.5.2 Coefficient of Determination (R2 )

Method A: Squaring r (Easiest)

R2 = r2 = (0.9036)2 ≈ 0.8166
Method B: Conceptual Formula (SST/SSR) (As shown in your notes, page 21)
1. Find SST: SST = (yi − ȳ)2 = 6 (from table)
P

2. Find predicted ŷ values:

• = −0.1 + 0.7(1) = 0.6
ŷ1
• = −0.1 + 0.7(2) = 1.3
ŷ2
• = −0.1 + 0.7(3) = 2.0
ŷ3
• = −0.1 + 0.7(4) = 2.7
ŷ4
• = −0.1 + 0.7(5) = 3.4
ŷ5
3. Find SSR: SSR = (ŷi − ȳ)2
P

SSR = (0.6 − 2)2 + (1.3 − 2)2 + (2.0 − 2)2 + (2.7 − 2)2 + (3.4 − 2)2
= (−1.4)2 + (−0.7)2 + (0)2 + (0.7)2 + (1.4)2
= 1.96 + 0.49 + 0 + 0.49 + 1.96 = 4.9

4. Calculate R2 :
SSR 4.9
R2 = = ≈ 0.8166
SST 6
Both methods for R2 yield the same result, which matches your notes. This means that ≈
81.66% of the variation in Sales (y) can be explained by the variation in Advertising (x).

Page 6