Regression Models in Management Analysis
Regression Models in Management Analysis
Fourteenth Edition
Chapter 4
Regression Models
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Learning Objectives
After completing this chapter, students will be able to:
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Learning Objectives
After completing this chapter, students will be able to:
6 Use computer software for regression analysis.
7. Develop a multiple regression model and use it for
prediction purposes.
8 Use dummy variables to model categorical data.
9 Determine which variables should be included in a
multiple regression model.
10 Transform a nonlinear function into a linear one for use
in regression.
11 Understand and avoid mistakes commonly made in the
use of regression analysis.
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Introduction
Regression analysis—a very valuable tool for managers
Understand the relationship between variables
Predict the value of one variable based on another
variable
Simple linear regression models have only two variables,
a dependent and an independent variable
Multiple regression models have three or more variables,
one dependent variable, and two or more independent
variables
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Introduction
The variable to be predicted is called the dependent
variable or response variable
Value depends on the value of the independent
variable(s)
Explanatory or predictor variable
A scatter diagram or scatter plot is often used to
investigate the relationship between variables
Independent variable is normally plotted on the X axis
Dependent variable is normally plotted on the Y axis
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Simple Linear Regression
Regression models are used We estimate with sample data
to test relationships between and make predictions using
variables
Yˆ b0 b1 X
Y 0 1
where:
where:
Yˆ predicted Y
Y = dependent variable
X = independent variable
X = independent variable
b0 estimate of 0
B0 intercept
b1 estimate of 1
B1 slope of regression line
random error
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction
Triple A Construction renovates old homes, and the dollar volume of
renovation work is dependent on the area payroll
Table 4.1 Triple A Construction Figure 4.1 Scatter Diagram of
Company Sales and Local Payroll Triple A Construction Company
Triple Local
Data
A’s Sales Payroll
($100,000s) ($100,000,000s)
6 3
8 4
9 6
5 4
4.5 2
9.5 5
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction
Predict sales based on area payroll
Y Sales X Area payroll
The line Figure 4.1 minimizes the errors
Error = Actual value (Predicted va lue)
e Y Yˆ
Regression analysis minimizes the sum of squared
errors
Least-squares regression
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction
Formulas for simple linear regression, intercept and slope
Yˆ b0 b1 X
X
X
average mean of X values
n
Y
Y
average mean of Y values
n
b1
X X Y Y
X X
2
b0 Y b1 X
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction
Table 4.2 Regression Calculations for Triple A Construction
( X X )(Y Y )
Left parenthesis X minus X bar right parenthesis left parenthesis Y minus Y bar right parenthesis
(3 4)(6 7) 1
Left parenthesis 3 minus 4 right parenthesis left parenthesis 6 minus 7 right parenthesis equals 1
(4 4)(8 7) 0Left parenthesis 4 minus 4 right parenthesis left parenthesis 8 minus 7 right parenthesis equals 0
(6 4)(9 7) 4Left parenthesis 6 minus 4 right parenthesis left parenthesis 9 minus 7 right parenthesis equals 4
(4 4)(5 7) 0Left parenthesis 4 minus 4 right parenthesis left parenthesis 5 minus 7 right parenthesis equals 0
(2 4)(4.5 7) 5
Left parenthesis 2 minus 4 right parenthesis left parenthesis 4.5 minus 7 right parenthesis equals 5
(5 4)(9.5 7) 2.5
Left parenthesis 5 minus 4 right parenthesis left parenthesis 9.5 minus 7 right parenthesis equals 2.5
Y 42
Summation Y equals 42. Y bar equals start fraction 42 over 6 end fraction equals 7
X 24
Summation X equals 24. X bar equals start fraction 24 over 6 end fraction equals 4
(X X )2 10
Summation left parenthesis X minus X bar right parenthesis squared equals 10
( X X )(Y Y ) 12.5
Summation left parenthesis X minus X bar right parenthesis left parenthesis Y minus Y bar right parenthesis equals 12.5
Y 42 / 6 7 X 24 / 6 4
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction
Regression calculations
X
X
24
4
6 6
b1
X X Y Y 12.5
1.25
X X
2
10
b0 Y b1 X 7 1.25 4 2
Therefore Yˆ 2 1.25 X
What are Triple A’s sales next year if the payroll is $600
million?
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Measuring the Fit of the Regression
Model
Regression models can be developed for any variables
X and Y
How helpful is the model in predicting Y?
With average error, positive and negative errors cancel
each other out
Three measures of variability
SST – Total variability about the mean
SSE – Variability about the regression line
SSR – Total variability that is explained by the model
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Measuring the Fit of the Regression
Model
2
Sum of squares total SST = Y Y
2
Sum of squares error SSE = e Y Yˆ 2
2
Sum of squares regression SSR = Yˆ Y
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Measuring the Fit of the Regression
Model
Table 4.3 Sum of Squares for Triple A Construction
Y X Y - Y 2 Left parenthesis Y minus Y bar right parenthesis squared
Yˆ
Y cap
2 1.25(3) 5.75
2 plus 1.25 left parenthesis 3 right parenthesis equals 5.75
0.0625 1.563
2 1.25(4) 7.00
2 plus 1.25 left parenthesis 4 right parenthesis equals 7.00
1 0
2 1.25(6) 9.50
2 plus 1.25 left parenthesis 6 right parenthesis equals 9.50
0.25 6.25
2 1.25(4) 7.00
2 plus 1.25 left parenthesis 4 right parenthesis equals 7.00
4 0
4.5 2 (4.5 7)2 6.25 Left parenthesis 4.5 minus 7 right parenthesis squared equals 6.25
2 +1.25(2) 4.50
2 plus 1.25 left parenthesis 2 right parenthesis equals 4.50
0 6.25
9.5 5 (9.5 7)2 6.25 left parenthesis 9.5 minus 7 right parenthesis squared equals 6.25
2 1.25(5) 8.25
2 plus 1.25 left parenthesis 5 right parenthesis equals 8.25
1.5625 1.563
Y 7
Y bar equals 7 blank
(Y Y ) 2
22.5
Summation left parenthesis Y minus Y bar right parenthesis squared equals 22.5. S S T equals 22.5. blank
(Y Yˆ )2 6.875
Summation left parenthesis Y minus Y cap right parenthesis squared equals 6.875. S S T equals 6.875.
(Yˆ Y ) 2
15.625
Summation left parenthesis Y cap minus Y bar right parenthesis squared equals 15.625. S S R equals 15.625.
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Measuring the Fit of the Regression
Model
Figure 4.2 Deviations from the Regression Line and from
the Mean
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Coefficient of Determination
The proportion of the variability in Y explained by the
regression equation
The coefficient of determination is r 2 .
2 SSR SSE
r 1
SST SST
For Triple A Construction
2 15.625
r 0.6944
22.5
About 69% of the variability in Y is explained by the
equation based on payroll (X)
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Correlation Coefficient
An expression of the strength of the linear relationship
Always between 1 and 1
The correlation coefficient is r
r r 2
For Triple A Construction
r 0.6944 0.8333
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Four Values of the Correlation Coefficient
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Assumptions of the Regression Model
With certain assumptions about the errors, statistical
tests can be performed to determine the model’s
usefulness
1. Errors are independent
2. Errors are normally distributed
3. Errors have a mean of zero
4. Errors have a constant variance
A plot of the residuals (errors) often highlights glaring
violations of assumptions
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Residual Plots
Figure 4.4A Pattern of
Errors Indicating
Randomness
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Variance and Standard Deviation
Errors are assumed to have a constant variance 2
This is estimated suing mean squared error (MSE) s2 .
SSE
s 2 MSE =
n k 1
where
n = number of observations in the sample
k = number of independent variables
The standard error of the estimate (standard deviation of
the regression) is s.
What is the MSE for Triple A Construction?
What is the standard error of the estimate?
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Testing the Model for Significance
We start with the general linear model Y 0 1 X
If 1 0, the null hypothesis is that there is no relationship
between X and Y
The alternate hypothesis 1 0 is that a linear
relationship exists
The F statistic is based on the ratio of MSE and MSR
degrees of freedom for the numerator df1 k
MSE
F degrees of freedom for the denominator
MSR
df2 n k 1
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Testing the Model for Significance
1. If there is very little error, the MSE would be small, and
the F statistic would be large—the model is useful
2. If the F statistic is large, the significance level (p-value)
will be low,—unlikely would have occurred by chance
3. When the F value is large, we can reject the null
hypothesis and accept that there is a linear relationship
between X and Y
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Steps in a Hypothesis Test
1. Specify null and alternative hypotheses
H0 : 1 0
H1 : 1 0
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Steps in a Hypothesis Test
4. Make a decision using one of the following methods
a. Reject the null hypothesis if the test statistic is greater than the
F value from the table in Appendix D. Otherwise, do not reject the
null hypothesis:
Reject if Fcalculated F , df , df
1 2
df1 k
df2 n k 1
b. Reject the null hypothesis if the observed significance level, or
p-value, is less than the level of significance . Otherwise, do not
reject the null hypothesis:
p-value P (F calculated test statistic)
Reject if p-value
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction Example
Step 1
H0 : 1 0 (no linear relationship between X and Y)
H1 : 1 0 (linear relationship exists between X and Y)
Step 2
Select 0.05
Step 3
– Calculate the value of the test statistic
SSR 15.6250
MSR 15.6250
k 1
MSR 15.6250
F 9.09
MSE 1.7188
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction Example
Step 4
Reject the null hypothesis if the test statistic is greater
than the F value in Appendix D
df1 k 1
df2 n k 1 6 1 1 4
The value of F associated with a 5% level of
significance and with degrees of freedom 1 and 4 is
found in Appendix D.
F0.05, 1, 4 7.71
Fcalculated 9.09
Reject H0 because 9.09 7.71
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Triple A Construction
Figure 4.5 F Distribution for Triple A Construction Test for
Significance
DF SS MS F Significance F
MSR / MSE
Start fraction M S R over M S E end fraction
P (F MSR/MSE)
P left parenthesis F is greater than M S R over M S E right parenthesis
Residual n k 1
n minus k minus 1
Total n 1 n minus 1
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Multiple Regression Analysis
Extensions to the simple linear model
Models with more than one independent variable
Y 0 1 X1 2 X 2 K k X k
where
Y = dependent variable (response variable)
xi i th independent variable (predictor or explanatory
variable)
0 intercept (value of Y when all X i 0)
1 coefficient of the ith independent variable
k = number of independent
variables
random error
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Multiple Regression Analysis
To estimate these values, a sample is taken the following
equation developed
Yˆ b0 b1 X1 b2 X 2 K bk X k
where
Yˆ predicted value of Y
b0 sample intercept (an estimate of 0 )
b1 sample coefficient of the ith variable (an
estimate of i )
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Develop a model to determine the suggested listing price for houses
based on the size and age of the house
Yˆ b0 b1 X1 b2 X 2
where
Yˆ predicted value of dependent variable (selling price)
b0 Y intercept
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Table 4.5 Jenny Wilson Real Estate Data
Selling Square
Age Condition
Price ($) Footage
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Table 4.5 [Continued]
Selling Square
Age Condition
Price ($) Footage
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example (4 of 10)
Program 4.4A Input Screen for Jenny Wilson Realty
Multiple Regression in Excel
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Program 4.4B Excel Output Screen for Jenny Wilson Realty Multiple
Regression Example
Yˆ b0 b1 X1 b2 X 2
146,630.89 43.28 X1 2898.69 X 2
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Evaluating the Multiple Regression Model
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Evaluating the Multiple Regression Model
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Full model is statistically significant
Useful in predicting selling price
p-value for F test = 0.002 r 2 0.6719
Are both variables significant? H0 : 1 0
For X1 (square footage) H1 : 1 0
For a 0.05, p-value = 0.0013 null hypothesis is rejected
For X1 (age)
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Binary or Dummy Variables
1. Binary (or dummy or indicator) variables are special
variables created for qualitative data
2. A dummy variable is assigned a value of 1 if a particular
condition is met and a value of 0 otherwise
3. The number of dummy variables must equal one less
than the number of categories of the qualitative variable
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
A better model can be developed if information about the
condition of the property is included
X 3 1 if the house is in excellent condition, X 3 0
otherwise
X 4 1 if the house is in mint condition, and X 4 0
otherwise.
1. Two dummy variables are used to describe the three
categories of conditions
2. No variable is needed for “good” condition since if both
X 3 0 and X 4 0, the house is in good condition.
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Program 4.5A Input Screen for Jenny Wilson Realty
Example with Dummy Variables in Excel
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Program 4.5B Output Screen for Jenny Wilson Realty
Example with Dummy Variables in Excel
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Jenny Wilson Realty Example
Program 4.5B Output Screen for Jenny Wilson Realty
Example with Dummy Variables in Excel 2016
Coefficient of determination, r 2 0.898
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Model Building
2 SSR SSE
The formula for r r2 1
SST SST
2 SSE n k 1
The formula for adjusted r 2
Adj r 1
SST n 1
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Model Building
1. Stepwise regression systematically adds or deletes
independent variables
2. A forward stepwise procedure puts the most
significant variable in first, adds the next variable that
will improve the model the most
3. Backward stepwise regression begins with all the
independent variables and deletes the least helpful
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Model Building
1. In some cases, variables contain duplicate information
2. When two independent variables are correlated, they
are said to be collinear
3. When more than two independent variables are
correlated, multicollinearity exists
4. When multicollinearity is present, hypothesis tests for
the individual coefficients are not valid but the model
may still be useful
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Colonel Motors
In some situations, variables are not linearly related
Transformations may be used to turn a nonlinear model into a
linear model
Table 4.6 Automobile Weight Versus MPG
Weight Weight
MPG (1,000s LB.) MPG (1,000s LB.)
12 4.58 20 3.18
13 4.66 23 2.68
15 4.02 24 2.65
18 2.53 33 1.70
19 3.09 36 1.95
19 3.11 42 1.92
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Colonel Motors
Figure 4.6A Linear model for MPG data
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Colonel Motors
Program 4.6 Excel Output with Linear Regression Model
for MPG Data
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Colonel Motors
Figure 4.6B Nonlinear Model for MPG Data
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Colonel Motors
The nonlinear model is quadratic
The easiest approach—develop a new variable
X 2 weight
2
New model
Yˆ b0 b1 X1 b2 X 2
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Colonel Motors
Program 4.7 Excel Output with Nonlinear Regression
Model for MPG Data
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Cautions and Pitfalls in Regression
Analysis
5. A t-test for the intercept b0 may be ignored as this point
is often outside the range of the model
6. A linear relationship may not be the best relationship,
even if the F test is significant
7. Even though a relationship is statistically significant it
may not have any practical value
8. A nonlinear relationship can exist even if a linear
relationship does not
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved
Copyright
Copyright © 2024, 2018, 2015 Pearson Education, Inc. All Rights Reserved