Module 10: Inference in
Simple Linear Regression
ISOM2500 BUSINESS STATISTICS
(L1 & L2, Spring 2025)
Jason HO
Contents
Inference in Regression
Confidence Interval for the Mean Response
Prediction Interval for a New Observation
2
Procedure in applying SLR (Recall from Module 9)
1. Produce a scatterplot of y on x
• Check for linearity
2. Find the LS equation, and obtain residuals
3. Perform regression diagnostics Return to this after Module 10
4. Test the slope parameter (Module 10)
5. Interpret the LS equation
• intercept and slope estimates, R2
6. Make prediction of new observations (Module 10)
3
INFERENCE IN
REGRESSION
4
Population VS sample (from Module 9, Slide 33)
unobserved observed
Theory: SLR model Mean of Y for X = x Reality
Y
Y
Population Sample
The purple line
& the red line
are
NOT the same
Normal with
equal variance σ2
X X
5
Inference in regression
Recall: Three parameters, and , identify/characterize the
population described by SLR
• The LS method provides their estimates:
Often, we have from data, but it does NOT imply
A fitted model/equation that survives in the regression diagnostics
(end of Module 9) provides the foundation for statistical inference
6
Validity of SLR
After getting the LS line, the validity test, a 2-sided test of H0:
is the first and foremost test to carry out
1. Fail to reject H0: The SLR reduces to
─ Estimate of mean of y is (i.e., X is of no use)
2. Able to reject H0:
• Call X a significant/useful predictor of Y
• Use the fitted equation (based on X) to estimate
mean of Y, and predict any new observation Y
7
Parameters and inference
In Modules 6-8, in estimation of a normal population mean :
1. is estimated by the sample mean
2. Inference of is based on sampling distribution of
─ With , the standardized sample mean
or when is
unknown
In SLR: Parameters in population Statistics from data
estimate, or
, point estimates of , LS estimates
• Inference: based on sampling distributions of and 8
Sampling distribution of Module 9, Slide 45
The LS estimate is unbiased for (i.e., )
The estimated standard errors are given by
in terms of (1) SE of the regression , (2) sample SD of the x
values , (3) sample size n
The sampling distribution of the standardized is given by
9
Confidence interval for the slope
A 95% confidence interval for the parameter in SLR is the
interval
where the t quantile
(obtainable in MS Excel by
[Link]( )) is defined Area
from a Student’s t-distribution = 0.975
with n-2 degrees of freedom,
as in Module 7, Slide 17
10
Hypothesis test for the slope
2-sided test: , for any hypothesized value
• Observed test statistic
(also called t-statistic) :
• Reject H0 at level α if , or
p-value =
1-sided test:
• Same observed test statistic as above
• Reject H0 if or for
or for
11
Equivalent inferences in SLR
We reject the claim (i.e., H0 below) that the parameter in SLR
equals any value with 5% chance of committing Type I error (or
with 95% confidence)
If any of the following is correct:
1. The absolute value of the t-statistic (in Slide 11) is larger
than ;
2. The p-value computed from the t-statistic (in Slide 11) is
less than 0.05; or
3. lies outside the 95% ci for (in Slide 10)
12
Hypothesis test for the validity of SLR
The validity of SLR can be done by testing
• Observed test statistic:
, or
• Reject H0 at level α if
p-value =
Alternatively, it is
equivalent to test if the true correlation of X and
Y (introduced in Module 9, Slide 13) is different from zero
• Observed test statistic:
(resulting in the same t value)
• Same rejection rule as above
13
Example 1: CEO compensation and sales
The scatterplots shown below and the regression outputs (given at next
Slide) summarize the relationship between total compensation of CEOs and
net sales at 201 finance companies. Both variables are on a log10 scale
Y=
X= 14
Example 1 (cont’d) – using regression outputs
from computer softwares
The
coefficient of determination R2 = 0.4037 . That is, 40% of variation in
compensation can be explained by sales (with both variables in log10 scale)
The error SD, , is estimated by SE of the regression
The LS estimates are , with respective
estimated SEs
15
Example 1 (cont’d 2) – validity test
3rd & 4th columns give the t-statistic and p-value to perform a 2-sided test to
check if the slope (or intercept) is different from zero, e.g., in row 2, we have
1. The hypotheses:
2. The t-statistic:
3. The p-value:
4. Reject H0 at any significance level α
5. Conclude that is different from zero, i.e., is a significant
predictor of 16
Example 1 (cont’d 3) – t-test for
A CEO claimed to her Board of Directors that in the model (i.e., elasticity
of compensation with respect to net sales) is 1/2. Let’s test the claim:
1. The hypotheses:
2. The t-statistic:
3. The p-value:
4. Do not reject H0 at 5% level
5. Conclude that 1/2 is a plausible value for the elasticity 17
Example 1 (cont’d 4) – constructing ci
A 95% ci for the elasticity is
We are 95% confident that the true elasticity is between 0.418 and 0.588
Itis possible to use this ci to repeat the last 2-sided test (in Slide 17)
of whether the elasticity is different from 1/2:
• As 1/2 lies inside the above ci for , we do not reject H0: = 1/2
at 5% level, and we conclude that the elasticity is possibly 1/2
18
Regression output by MS Excel for Example 1
Coefficient of determination
Estimate of SD of error
[out of scope]
95% ci for
t-statistic and p-value for the 2-tailed test of H0:
19
Example 2: locating a franchise outlet
Consider where to locate a new gas station
Q1: On a highway or another location with lower traffic?
Does traffic volume affect sales of gas?
Q2: We want to choose between two sites via predicting their sales volumes
• One site is on a highway that averages 40,000 “drives-by” a day
• Another averages 32,000
How much more gas can we expect to sell at the busier location?
20
Example 2 (cont’d) – a regression analysis
approach
We won’t find many stations that have 32,000 and 40,000 drives-by a day,
but possibly be able to find sales at stations with various levels of traffic
volume so as to use all data to fit a regression model of Y on X
• Y: (average) sales of gas per day (in 1,000 gallons)
• X: (average) daily traffic volume (in 1,000 cars)
Data:Collect sales data and traffic volume during a recent month at n = 80
franchise outlets
Theintercept sets a baseline of gas sales that occur regardless of traffic
intensity (perhaps due to loyal customers)
The slope measures the mean sales per passing car
21
Example 2 (cont’d 2) – linearity in the
scatterplot
The scatterplot below suggests that the association is linear
Noobvious lurking variable – These 80 stations are in similar areas with
comparable prices
22
Example 2 (cont’d 3) – residual diagnostics
Residual plot and histogram of residuals reveal:
• Residuals (1) are evidently independent, (2) have similar variances,
and (3) are nearly normal with zero mean
23
Example 2 (cont’d 4) – validity of the LS
equation
The 2-sided t-test for H0: is highly significant ( p-value < 0.0001 ):
• Traffic volume is a significant predictor of sales
Coefficient of determination R2 = 54.86% :
• The LS equation explains ~55% of total variation in daily sales by
daily traffic volume
24
Example 2 (cont’d 5) – A ci for 8,000 x for Q2
The parameter, 8,000 x , gives the difference in expected daily gasoline
sales between a station on a highway location with 40,000 drives-by and
another with 32,000 drives-by
A 99% ci for is given by
A 99% ci for 8000 is given by
• We are 99% confident that a station located at a site with 40,000 drives-
by per day sell on average from 1,392 to 2,392 more gallons of gasoline
daily than a location with 32,000 drives-by per day
25
CONFIDENCE INTERVAL
FOR THE MEAN AND
PREDICTION INTERVAL
26
Population & its Population of all costs
distribution (Recall)
Sub-population Cost of all
An illustration using of diamonds
the Example: diamonds with
with weight weight X=0.4
X=0.4
Sub-populations formed
by different (infinitely
many) values of weight (X)
Independent
What is a ci for the
Cost
mean in a certain
Sub-population
sub-population? of diamonds
with weight X=0.5
27
Pop. line:
A ci for the mean response
LS line:
The SLR states that the mean response for any given value x0 of X
as
To estimate it, instead of using the point estimate given by the
fitted value,
a 95% ci for the mean response at a given value x0 of X is given by
28
Example 3: ci for the mean response ci for the mean, Slide 28
Consider the “Locating a Franchise Outlet” Example (in Slide 20)
What is an interval estimate of the mean daily gasoline
sales for outlets with 40,000 drives-by per day?
The LS equation is given by
Sample mean &
A 95% ci for the mean daily sales for outlets with traffic = 40 is sample variance
of traffic volume
of the 80
stations
We are 95% confident that the mean daily gasoline sales for outlets with
40,000 drives-by per day is between 7,791 and 8,471 gallons
29
Population & its Population of all costs
distribution (Recall)
Sub-population
An illustration using of diamonds
Cost of all
the Example: diamonds with
with weight weight X=0.4
X=0.4
Sub-populations formed
by different (infinitely
many) values of weight (X)
Independent
What is an interval ynew
estimate for a random Cost
draw ynew from a Sub-population
of diamonds
certain sub-population? with weight X=0.5
30
The prediction interval ci for the mean, Slide 28
– an interval estimate for a new observation
One powerful use of SLR is to predict the value ynew of a new
observation knowing its X-value given as xnew
The SLR states:
Estimated by
(identical to in Slide 28 with )
A 95% prediction interval for a new observation at a given
value xnew of X is given by
31
Example 4: managing natural
resources
Overfishing endangered many fish species (e.g., bluefin tuna, cod)
Consider how number of days that fishing boats trapping Dungeness crabs
can operate in water (limited by the Canadian government) and the size of
catch are related based on n = 28 yearly data (1980-2007) near Vancouver
Island
• Response Y: Dungeness weight (in 1,000 lbs)
• Predictor X: days of effort or number of limited boat-days
The slope of a regression equation of Y on X indicates the average catch
per additional day a boat operates
32
Example 4 (cont’d) – linear relationship
Positive linear association between harvest weight and effort
Ignore the
PINK region at
the moment
33
Example 4 (cont’d 2) – computer output of the
LS equation
A time plot of the residuals shows no clear pattern, suggesting
independence of residuals
34
Example 4 (cont’d 3) – residual diagnostics
The residual plot confirms independence and similar variances of residuals
Histogramof residuals is perhaps NOT bell-shaped, but with such a small
sample these data could be normally distributed
35
Example 4 (cont’d 4) – days of effort as a
significant predictor
The LS equation
explains 84% of variation in catch (Y) with days of effort (X)
RejectH0: at any significant level as p-value < 0.0001 , and conclude
that days of effort (X) is a significant predictor of Dungeness weight
• The regression model of weight on days of effort provides a better
prediction of the catch weight than we can generate without knowing
days of effort
36
Example 4 (cont’d 5) – a prediction interval
for the catch
Suppose that a 7,500 days of effort will be imposed in the coming year
We can predict the catch (with xnew = 7500) by
A 95% prediction interval of the catch (with xnew = 7500) is given by
pi for a new response, Slide 31
with , , and
We are 95% confident that the catch next year will be from 908.42 to
1,438.08 thousand pounds when 7,500 days of effort are to be imposed
37
Example 4 (cont’d 6) – 95% prediction bands
The 2 lines enclosing the region in PINK below are called 95% prediction
bands, constructed by joining endpoints of PI’s at every value of X
1,438.08
*
*
908.42 If SLR holds true,
*
* 95% of the data
*
should be within
A 95% pi of the catch (with xnew = 7500)
7,500 the 95%
marked as *
*
prediction bands
Obtain and mark PI’s as *
for all values of X 38
Takeaway
Sampling distribution of the slope estimate
Inference in regression
• Validity of the SLR model
• Confidence interval & hypothesis test of slope
Confidence interval for the mean response
Prediction interval for a new response
39