Simple Linear Regression

Grace Carmichael

Department of Statistical Sciences

University of Cape Town

2024

Simple Linear Regression 1 / 51

Outline

1 Recap of Foundational Concepts in Statistics

2 The Problem We Want to Solve:

3 Example Problem

4 Correlation Analysis

5 Simple Linear Regression

Simple Linear Regression 2 / 51

 Recap of Foundational Concepts in Statistics

Recap of Foundational Concepts in Statistics

Simple Linear Regression 3 / 51

 Recap of Foundational Concepts in Statistics

Population versus Sample

When we refer to a numerical descriptor for a population we refer to

it as a parameter, where a numerical descriptor for a sample is
referred to as a statistic.
We use sample statistics to approximate population parameters.

Simple Linear Regression 4 / 51

 Recap of Foundational Concepts in Statistics

Statistical Inference

Statistical inference is the attempt to reach a conclusion concerning a

complete set of observations (the population) using only a subset thereof
(a sample).
It is important to note that this sample needs to be representative of
the population in order to make accurate inference.
We make use of sampling distributions to make inference.
Statistical inference is conducted with the help of hypothesis testing.

Simple Linear Regression 5 / 51

 Recap of Foundational Concepts in Statistics

Hypothesis Testing

Hypothesis testing allows us to make statements about a population from

a sample of that population. It involves the following basic steps:
Step 1: Define the null hypothesis (H0 ) This is the hypothesis of no statistical significance.
Step 2: Define the alternative hypothesis (Ha ) This is the hypothesis of statistical
significance.
Step 3: Define the significance level (α) This is the type one error rate (probability of
falsely rejecting H0 ). Typically α = 0.05 or α = 0.01 are sufficiently low.
Step 4: Calculate the test statistic This will be calculated differently depending on the test
being conducted.
Step 5: Find the p-value This is the probability of getting a result as or more extreme than
the observed test statistic, assuming H0 is true. A precise p-value can be
generated using software or an approximate one using tables by hand.
Step 6: Make a conclusion If p-value is ≤ α, then we reject H0 and conclude statistical
significance of our result. Otherwise, we fail to reject H0 and conclude no
statistical significance (this means that we can’t make any statements
about the population from our sample result).

Simple Linear Regression 6 / 51

 The Problem We Want to Solve:

The Problem We Want to Solve:

Simple Linear Regression 7 / 51

 The Problem We Want to Solve:

Describing the relationship between two variables

How strong is the relationship? (so we want to be able to quantify it)

Is this observed relationship likely real or just due to chance?
Can we explain the impact that changing one variable has on another
variable?
Can we predict the value of one variable from another variable?

Simple Linear Regression 8 / 51

 Example Problem

Example Problem

Simple Linear Regression 9 / 51

 Example Problem

Lecture attendance example

As part of an experiment, a lecturer recorded the overall course marks and
number of lectures attended for 20 students in the course that they teach.
The results of this experiment are shown below:

Number of lectures Marks

1 46 80

90
2 10 20
3 38 59
4 27 34

80
5 45 71
6 26 55

70
7 35 50
8 45 78
9 48 81
60

10 20 28
marks

11 30 50
50

12 27 47
13 38 77
40

14 12 18
15 28 41
16 40 79
30

17 38 68
18 47 88
20

19 36 66
20 40 70 10 20 30 40

lectures

Simple Linear Regression 10 / 51

 Correlation Analysis

Correlation Analysis

Simple Linear Regression 11 / 51

 Correlation Analysis

Correlation Analysis as a method to solve our problem

Correlation is a measure of strength and direction of a linear relationship

between two variables.
Correlation is bounded between -1 and 1.
Correlation does not have a unit.
Correlation can not be used to predict one variable from another.

Simple Linear Regression 12 / 51

 Correlation Analysis

Correlation coefficient

Correlation is measured using the correlation coefficient (typically the

Pearson correlation coefficient).
The population correlation coefficient (ρ) measures the direction
and strength of the association between the full set of two variables.
The sample correlation coefficient (r) is an estimate of ρ and
measures the direction and strength of the association between the
two variables in a sample of the population.
The sample correlation coefficient is given by:
!
(xi → x̄)(yi → ȳ) SSxy
r = "! ! ="
(xi → x̄)2 (yi → ȳ)2 SSx SSy

Simple Linear Regression 13 / 51

 Correlation Analysis

Test your understanding

Calculate the correlation coefficient between X and Y for the following 3
observations:
X Y
2 3
3 1
4 2

Simple Linear Regression 14 / 51

 Correlation Analysis

Examples of data with different correlation coefficients

r = 0.16 r=1 r = 0.94

y1

y2

y3
x x x

r = −1 r = −0.8 r = 0.08
y4

y5

y6

x x x2
Simple Linear Regression 15 / 51
 Correlation Analysis

Correlation Analysis with our Example

Number of lectures Marks

1 46 80 x̄ = 33.8 and ȳ = 58
2 10 20
3 38 59
4 27 34
5 45 71
6 26 55
7 35 50 SSxy 4280
8 45 78 r=" =√ = 0.95
9 48 81
10 20 28
SSx SSy 2345.2 × 8660
11 30 50
12 27 47
13 38 77
14 12 18
15 28 41
16 40 79
17 38 68
18 47 88
19 36 66
20 40 70

Simple Linear Regression 16 / 51

 Grace
Correlation Analysis Carmichael

Example in R 2024-01-29

#-------------------------------------------------------------------------------
# Perform Correlation Analysis on example data using first principals
#-------------------------------------------------------------------------------

x <- lectures
y <- marks

# calculate means of x and y

xbar <- mean(x)
ybar <- mean(y)

# calculate sum of squares

SSxy <- sum((x-xbar)*(y-ybar))
SSx <- sum((x-xbar)ˆ2)
SSy <- sum((y-ybar)ˆ2)

# calculate correlation
(r <- SSxy/sqrt(SSx*SSy))

## [1] 0.9497185

#-------------------------------------------------------------------------------
# Confirm result using base R cor() function
#-------------------------------------------------------------------------------
# cor() just takes the variables as inputs and outputs the correlation
cor(x,y)

## [1] 0.9497185

Simple Linear Regression 17 / 51

 Correlation Analysis

Inference on correlation coefficient (is it significant?)

Step 1: Define the null hypothesis H0 : ρ = 0

Step 2: Define the alternative hypothesis Ha : ρ $= 0
Step 3: Define the significance level α = 0.05
Step 4: Calculate the test statistic
√ √
r n→2 0.9497185 × 20 → 2
tstat = √ ∼ tn−2 tstat = √
1 → r2 1 → 0.94971852

tstat = 12.87 ∼ t18

Simple Linear Regression 18 / 51

 Correlation Analysis

Step 5: Find the p-value

#-----------------------------------------------------------------------
# Perform hypothesis test on correlation
#-----------------------------------------------------------------------

# Using the t-stat that we calculated by hand

2*pt(q=12.87,df=18, [Link] = F)
2.5% 2.5%

-12.87 -2.101 2.101 12.87 ## [1] 1.622399e-10

# using [Link]() which conducts the entire hypothesis test for you
[Link](x,y)

##
## Pearson’s product-moment correlation
##
## data: x and y
## t = 12.869, df = 18, p-value = 1.625e-10
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.8748863 0.9802637
## sample estimates:
## cor
## 0.9497185

Simple Linear Regression 19 / 51

 Correlation Analysis

Step 6: Make a conclusion the p-value is sufficiently small to reject H0

(p-value < 0.05). Thus, it can be concluded that there is
some significant linear relationship between course marks and
the number of lectures attended by the students.

Simple Linear Regression 20 / 51

 Correlation Analysis

Limitations of correlation analysis

Recall the questions we want to answer about our two variables of interest:

How strong is the relationship? (so we want to be able to quantify it)

Is this observed relationship likely real or just due to chance?
Can we explain the impact that changing one variable has on another
variable?
Can we predict the value of one variable from another variable?

Correlation analysis allows us to answer the first two questions but the
limitation of correlation analysis is the inability to quantify the impact of
one variable on another and to predict the value of one variable from the
other.

Simple Linear Regression 21 / 51

 Simple Linear Regression

Simple Linear Regression

Simple Linear Regression 22 / 51

 Simple Linear Regression

Simple Linear Regression as a method to solve our problem

Simple linear regression is another method used to describe the relationship

between two variables. It is different to correlation analysis in that:
There is a dependent and independent variable. The dependent
variable is the one we want to explain and the independent variable is
the one we want to use to explain the dependent variable.
It allows us to explain the impact of changing the independent
variable on the dependent variable.
It allows us to predict the dependent variable from the independent
variable (if our model fit is good).

Simple Linear Regression 23 / 51

 Simple Linear Regression

Simple Linear Regression Model

y = mx + c

Population model Sample model

yi = β 0 + β 1 x i + $ i ŷi = β̂0 + β̂1 xi

Where: Where:
i refers to a specific observation i refers to a specific observation
β0 is the intercept parameter ŷi is the predicted value of the
β1 is the slope parameter dependent variable
$i is the error β̂0 is the estimated regression
The error term accounts for any intercept
variability in the response that is not β̂1 is the estimated regression
due to the independent variable. slope
Assume that E[$i ] = 0 and that
$i ∼ N (0, σ 2 )
Simple Linear Regression 24 / 51
 Simple Linear Regression

Estimating the β parameters

The algorithm used to find the optimal values of β0 and β1 in a simple
linear regression model is called the Ordinary Least Squares (OLS)
algorithm. It works by minimizing the sum of the squared error terms, ie:
n
#
Minimize $2i
i=1

Where:
$i = yi → ŷi
So:
n
# n $
# %2
$2i = yi → (βˆ0 + βˆ1 xi )
i=1 i=1

We can express $i in terms of the

! β estimates and can thus find values of
the β estimates that minimize ni=1 $2i . These will be the optimal β values
for the linear relationship we are trying to model.
Simple Linear Regression 25 / 51
 Simple Linear Regression

90
80
70
60
marks

ε
50

ε j
40
30
20

10 20 30 40

lectures

Simple Linear Regression 26 / 51

 Simple Linear Regression

90
80
70
60
marks

50
40
30
20

10 20 30 40

lectures

Simple Linear Regression 27 / 51

 SLR_example
Simple Linear Regression
Grace Carmichael

Performing Linear Regression in R 2024-01-31

fit <- lm(marks~lectures)

summary(fit) β̂0 →→ (Intercept) estimate
##
β̂1 →→ lectures estimate
## Call:

So the regression equation for the

## lm(formula = marks ~ lectures)
##
## Residuals:
##
##
Min 1Q
-11.590 -5.215
Median
-0.240
3Q
4.348
Max
11.335 example is given by:
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
##
##
(Intercept) -3.6851
lectures 1.8250
5.0333 -0.732 0.474
0.1418 12.869 1.62e-10 ***
ŷ = β̂0 + β̂1 (lectures)
## ---
## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
##
## Residual standard error: 6.868 on 18 degrees of freedom ŷ = →3.6851 + 1.8250(lectures)
## Multiple R-squared: 0.902, Adjusted R-squared: 0.8965
## F-statistic: 165.6 on 1 and 18 DF, p-value: 1.625e-10

Interpretation of β0 (intercept) On average, a student’s course mark is

-3.6851 when a student attends no lectures (this is not
contextually practical as a student’s mark cannot be < 0).
Interpretation of β1 (slope) On average, there is a 1.8250 mark increase in
student marks for the course for each additional lecture
attended.
Simple Linear Regression 28 / 51
 Simple Linear Regression

General interpretation of β coefficients

Interpretation of β0 (intercept) It is the average estimated value of y

when x = 0.
Interpretation of β1 (slope) It is the average estimated change in y for a
unit increase in x. If β1 is positive then the change is an
increase and negative means a decrease (It is important to
be specific).

Simple Linear Regression 29 / 51

 Simple Linear Regression

Assessing the accuracy of the model

To assess the overall accuracy of the model estimates, we can use a
measure called the Residual Standard Error (RSE). This measures the
standard deviation of the model residuals (average amount that the
response will deviate from the regression line).
&!
n 2
i=1 $i
RSE =
n→2
Higher RSE Lower RSE
30

15
20
10

10
y

y
0

5
−10

0
−20

0 5 10 15 20 0 5 10 15 20
x x

Simple Linear Regression 30 / 51

 Simple Linear Regression

Assessing the accuracy of the β estimates

The standard error of an estimate indicates how different the population estimate
is likely to be from the sample estimate. A large standard error relative to the size
of the estimate is an indication that the estimate may not be an accurate
reflection of the true population parameter.
RSE RSE
se(β1 ) = "! =√
(xi → x̄) 2 SSx

Smaller se(β1 ) Larger se(β1 )

Where each line on the plot is the regression line from a different sample.
Simple Linear Regression 31 / 51
 Simple Linear Regression

Testing the significance of β1 estimate

We are usually mostly interested in the significance of the slope estimate
(β1 ), as this is the estimate that tells us whether there is a relationship
between the independent and dependent variables. We can do a
hypothesis test on this estimate as follows:
Step 1: Define the null hypothesis H0 : β1 = 0
Step 2: Define the alternative hypothesis Ha : β1 $= 0
Step 3: Define the significance level α = 0.05
Step 4: Calculate the test statistic

βˆ1 → β1 βˆ1 1.825

tstat = = ∼ tn−2 tstat =
se(βˆ1 ) se(βˆ1 ) 0.1418

tstat = 12.87 ∼ t18

Simple Linear Regression 32 / 51
 Simple Linear Regression
Grace Carmichael

Step 5: Find the p-value 2024-02-03

#--------------------------------------------------------------------
# Perform hypothesis test on slope cofficient
#--------------------------------------------------------------------

# Using the t-stat that we calculated by hand

2*pt(q=12.87,df=18, [Link] = F)

## [1] 1.622399e-10

# using the output from lm()

2.5% 2.5%
fit <- lm(marks~lectures)
summary(fit)
-12.87 -2.101 2.101 12.87

##
## Call:
## lm(formula = marks ~ lectures)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.590 -5.215 -0.240 4.348 11.335
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.6851 5.0333 -0.732 0.474
## lectures 1.8250 0.1418 12.869 1.62e-10 ***
## ---
## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
##
## Residual standard error: 6.868 on 18 degrees of freedom
## Multiple R-squared: 0.902, Adjusted R-squared: 0.8965
## F-statistic: 165.6 on 1 and 18 DF, p-value: 1.625e-10

Simple Linear Regression 33 / 51

 Simple Linear Regression

Step 6: Make a conclusion the p-value is sufficiently small to reject H0

(p-value < 0.05). Thus, it can be concluded that there is
some significant linear relationship between course marks and
the number of lectures attended by the students.

Simple Linear Regression 34 / 51

 Simple Linear Regression 2024-02-12

Confidence Intervals for β Estimates ##

## Call:
## lm(formula = marks ~ lectures)
##
We can get a range of values that we are Residuals:
relatively
Min
sure that
##
##
1Q Median 3Q
the
Max
true β
parameters fall into (with a given level of confidence). ##
-11.590 -5.215 -0.240
##
4.348 11.335

## Coefficients:

We calculate this range using: ##

##
Estimate Std. Error t value Pr(>|t|)
(Intercept) -3.6851 5.0333 -0.732 0.474
## lectures 1.8250 0.1418 12.869 1.62e-10 ***
## ---
CI = βˆj ± t α ##
,df
2 ##
× se(β̂j )
Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’

## Residual standard error: 6.868 on 18 degrees of freedom

We can calculate the 95% confidence interval for β (the slope) as follows:
##
##
Multiple R-squared: 0.902, Adjusted R-squared: 0.8965
1 on 1 and 18 DF, p-value: 1.625e-10
F-statistic: 165.6

#---------------------------------------------------------------
# Calculate confidence interval of regression estimates in R

CI = βˆ1 ± t α2 ,18 × se(βˆ1 )

#---------------------------------------------------------------

confint(fit)

CI = 1.825 ± 2.101 × 0.1418 ## 2.5 % 97.5 %

## (Intercept) -14.259806 6.889518
CI = [1.527, 2.123] ## lectures 1.527061 2.122947

Be careful not to interpret the confidence interval as a 95% probability

that the parameter is between the interval, this is WRONG! It really just
means that if we resample our population we expect 95% of the estimates
to be within our interval.
Simple Linear Regression 35 / 51
 Simple Linear Regression

Checking overall model significance

We have seen that the significance of the β estimates can be assessed. We

can also check the overall model significance by assessing if our model is
significantly different to a null model (a model with just an intercept).

Source of df SS Mean Square F

Variation
!
Regression
SSreg MSreg
1 SSreg = i (ŷi → ȳ)2 MSreg = 1 MSresid
!
Residual n→2 SSresid = i (yi → ŷi )2 MSresid = SSresid
n−2
!
Total n→1 SStot = i (yi → ȳ)2
We can use the F-statistic to then perform an F-test.
√
Note: MSresid = RSE

Simple Linear Regression 36 / 51

 Simple Linear Regression

In R the test for overall model significance

GraceisCarmichael
automatically done when
running lm() and the results can be viewed2024-01-31
by summarising the model
object:
fit <- lm(marks~lectures)
summary(fit)

##
## Call:
## lm(formula = marks ~ lectures)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.590 -5.215 -0.240 4.348 11.335
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.6851 5.0333 -0.732 0.474
## lectures 1.8250 0.1418 12.869 1.62e-10 ***
## ---
## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
##
## Residual standard error: 6.868 on 18 degrees of freedom
## Multiple R-squared: 0.902, Adjusted R-squared: 0.8965
## F-statistic: 165.6 on 1 and 18 DF, p-value: 1.625e-10

Simple Linear Regression 37 / 51

 Simple Linear Regression

Coefficient of determination (R2 )

The coefficient of determination is a measure of model fit. It is calculated

using: !
2 SSreg (ŷi → ȳ)2
R = = !i 2
SStot i (yi → ȳ)

The coefficient of determination (or R-squared) is the square of the

sample correlation coefficient.
It describes the proportion of variation in the response variable that is
explained by the explanatory variable.
It is bounded between 0 and 1.
A low value of R2 (closer to 0) indicates a poor model fit (only a
small proportion of the variation in y is explained by x) and a large
value (closer to 1) indicates a good fit.
Simple Linear Regression 38 / 51
 Simple Linear Regression
SLR_example
Coefficient of determination in R
Grace Carmichael
The coefficient of determination can also be found in the model summary
2024-01-31
in R.
fit <- lm(marks~lectures)
summary(fit)

##
## Call:
## lm(formula = marks ~ lectures)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.590 -5.215 -0.240 4.348 11.335
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) -3.6851 5.0333 -0.732 0.474
## lectures 1.8250 0.1418 12.869 1.62e-10 ***
## ---
## Signif. codes: 0 ’***’ 0.001 ’**’ 0.01 ’*’ 0.05 ’.’ 0.1 ’ ’ 1
##
## Residual standard error: 6.868 on 18 degrees of freedom
## Multiple R-squared: 0.902, Adjusted R-squared: 0.8965
## F-statistic: 165.6 on 1 and 18 DF, p-value: 1.625e-10

Simple Linear Regression 39 / 51

 Simple Linear Regression

Model Checking

When fitting linear regression models there are some assumptions that we
make about the relationship we are modelling.
1 We assume that the relationship between the dependent and
independent variable is linear.
2 We assume that the errors in our model are normally distributed with
a mean of 0. ($i ∼ N (0, σ 2 )).
3 We assume errors have constant variance (no heteroscedasticity)
4 We assume errors are independent (ie. there is no pattern in the
residuals).
We need to check that the models we fit do not violate any of these
assumptions because if they do then linear regression is not an appropriate
model for the data.

Simple Linear Regression 40 / 51

 Simple Linear Regression

Assessing linearity

To ensure that the relationship we are modelling is linear we can just plot
a scatter plot of x and y and look if it looks linear.

Simple Linear Regression 41 / 51

 Simple Linear Regression

Assessing normality of residuals

There are two ways to check this assumption (you should always do both).
1. QQ-plot 2. Histogram
Normal Q−Q Plot Histogram of Residuals

50
2

40
1
Sample Quantiles

30
Frequency
0

20
−1
−2

10
−3

−3 −2 −1 0 1 2 3 0 −3 −2 −1 0 1 2 3

Theoretical Quantiles Residuals

We want the points to follow the We want the histogram to be

red line (a bit of deviation in the bell-shaped (should look normally
tails is alright). distributed).
Simple Linear Regression 42 / 51
 Simple Linear Regression

Assessing constant variance of errors

We can check this assumption by plotting the residuals versus the fitted
values. If there is no pattern (just random scatter) then this assumption is
met. 2
1
Residuals

0
−1
−2
−3

6 8 10 12 14

Fitted values

Simple Linear Regression 43 / 51

 Simple Linear Regression

Assessing independence of errors

We can check this assumption by plotting errors against the independent
variable. If there is no pattern then the errors are independent (don’t
change with x). 2
1
Residuals

0
−1
−2
−3

−4 −2 0 2 4

Simple Linear Regression 44 / 51

 Simple Linear Regression

An example of a model that meets assumptions

Normal Q−Q Plot Histogram of resids

50
2

40
Sample Quantiles

Frequency

30
0

20
−1
−2

10
−3

0
−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3

Theoretical Quantiles Residuals

2

2
1

1
Residuals

Residuals
0

0
−1

−1
−2

−2
−3

−3

6 8 10 12 14 −4 −2 0 2 4

Fitted values x

Simple Linear Regression 45 / 51

 Simple Linear Regression

An example of a model that violates assumptions

Normal Q−Q Plot Histogram of resids

15

30
25
10
Sample Quantiles

Frequency

20
5

15
0

10
−5

5
−10

0
−3 −2 −1 0 1 2 3 −10 −5 0 5 10 15

Theoretical Quantiles Residuals

15

15
10

10
Residuals

Residuals
5

5
0

0
−5

−5
−10

−10

24 26 28 30 32 −4 −2 0 2 4

Fitted values x

Simple Linear Regression 46 / 51

 Simple Linear Regression

Prediction

Linear regression can be used to quantify the linear relationship between

two variables but it can also be used to predict the value of the dependent
variable from the value of the independent variable.
The predicted values from a linear regression lie on the fitted line (ie.
if you supply an x value, the predicted y value is that of the
corresponding point on the fitted line.)
Prediction should only be done once you have confirmed that your
model fits the data well and has passed all of the model checks.
You cannot predict outside of the range of your data (for example, if
you only have x values ranging from 0 to 10, you can’t predict for an
x of 15).

Simple Linear Regression 47 / 51

 Simple Linear Regression

Predicting marks based on lecture attendance

Equation for a fitted line:
ŷ = β0 + β1 x
For our example the fitted line is given by:

! = β0 + β1 × lectures
mark
! = →3.6851 + 1.825 × lectures
mark

If we want to know the predicted mark of a student who attends 35

lectures for the course we can do the following:

! = →3.6851 + 1.825 × 35
mark
! = 60.19
mark

So a student who attends 35 lectures can expect a mark of 60.19% for the
course.
Simple Linear Regression 48 / 51
 Simple Linear Regression

Confidence and Prediction interval

We saw that we can use a confidence interval to provide a level of
confidence around our β estimates. We can also get a level of confidence
around our predictions.
We can predict the average y value for a given x value or we can predict
the y value for a specific individual given their x value. These actual
predictions will be the same if the x value is the same but the intervals will
differ.
CI for average y: PI for individual y:
& &
1 (xp → x̄)2 1 (xp → x̄)2
ŷ ± t α2 ,n−2 s! +! 2
ŷ ± t α2 ,n−2 s! 1 + + ! 2
n i (xi → x̄) n i (xi → x̄)

The CI for the average y will always be narrower than the PI for an
individual y. This is because predicting for an individual is always more
uncertain that predicting the mean.
Simple Linear Regression 49 / 51
 Simple Linear Regression
Grace Carmichael

Prediction in R 2024-02-23

#-------------------------------------------------------------------------------
#Predicting for a single individual
#-------------------------------------------------------------------------------

ind1_lectures <- 35
predict(fit, newdata = list(lectures=ind1_lectures), interval = "prediction")

## fit lwr upr

## 1 60.19001 45.40081 74.9792

#-------------------------------------------------------------------------------
#Predicting over lecture range
#-------------------------------------------------------------------------------

lectures_new <- [Link](lectures = 10:48)

predict(fit, newdata=lectures_new)

## 1 2 3 4 5 6 7 8
## 14.56490 16.38990 18.21491 20.03991 21.86492 23.68992 25.51492 27.33993
## 9 10 11 12 13 14 15 16
## 29.16493 30.98994 32.81494 34.63995 36.46495 38.28995 40.11496 41.93996
## 17 18 19 20 21 22 23 24
## 43.76497 45.58997 47.41498 49.23998 51.06498 52.88999 54.71499 56.54000
## 25 26 27 28 29 30 31 32
## 58.36500 60.19001 62.01501 63.84001 65.66502 67.49002 69.31503 71.14003
## 33 34 35 36 37 38 39
## 72.96503 74.79004 76.61504 78.44005 80.26505 82.09006 83.91506

Simple Linear Regression 50 / 51

 Simple Linear Regression

A note on causality

It is important to remember that there being correlation between two

variables does NOT imply that one variable causes a change in the other
variable.
We can identify that there is a relationship but we can’t determine
directionality or causality in the relationship (for example, there could be a
third unmeasured variable that affects both variables creating the observed
relationship, in this case neither variable causes the other).

Simple Linear Regression 51 / 51