Liudmyla Feurstein
QUANTITATIVE BUSINESS METHODS
UNIT 7 – ADVANCED REGRESSION ANALYSIS AND
OTHER FORECASTING METHODS
PROFESSOR: HAHSEM ZARAFAT
The Economist
[Link]
“In every way that people, firms, or
governments act and plan, they
are making implicit forecasts
about the future”.
Introduction to Forecasting
• What is forecasting?
– Primary function is to predict the future using (time
series related or other) data we have in hand.
• Why are we interested?
– Affects the decisions we make today.
• Examples of Forecasting Applications:
– forecast demand for products and services
– forecast availability/need for manpower
– forecast inventory and materiel needs daily
– forecast movements in stock prices, interest rates…
Forecasting Methods:
An Overview
• There are many forecasting methods available, and
little agreement exists as to the best forecasting
method.
• The methods are typically divided into three groups:
1. Judgmental (subjective, qualitative) methods
2. Econometric (causal, regression-based) methods
3. Extrapolation (or time series) methods
Qualitative Approaches to
Forecasting
• Delphi Approach
– A panel of experts, each of whom is physically
separated from the others and is anonymous, is
asked to respond to a sequential series of
questionnaires.
– After each questionnaire, the responses are tabulated
and the information and opinions of the entire group
are made known to each of the other panel members
so that they may revise their previous forecast
response.
– The process continues until some degree of
consensus is achieved.
Qualitative Approaches to
Forecasting (2)
• Scenario Writing
– Scenario writing consists of developing a
conceptual scenario of the future based on a
well defined set of assumptions.
– After several different scenarios have been
developed, the decision maker determines
which is most likely to occur in the future and
makes decisions accordingly.
– Also: best-case or worst-case scenarios (e.g.
setting up cut-off points)
Qualitative Approaches to
Forecasting (3)
• Subjective or Interactive Approaches
– These techniques are often used by
committees or panels seeking to develop new
ideas or solve complex problems.
– They often involve "brainstorming sessions".
– It is important in such sessions that any ideas
or opinions be permitted to be presented
without regard to its relevancy and without fear
of criticism.
Quantitative Approaches to Forecasting
• . A time series is a set of observations measured at successive
points in, or over successive periods of time.
• Quantitative forecasting approaches are based on an analysis
of historical data concerning one or more time series.
• If the data used are limited to past values of the series we are
trying to forecast, the procedure is called a time series method.
• If the data used involve other time series that are believed to be
related to the time series being forecasted, the procedure is
called a causal method
Econometric Models
• Econometric models, also called causal or regression-
based models, use regression to forecast a time series
variable by using other explanatory time series variables.
• Causal regression models present mathematical challenges,
including:
– What are the appropriate “lags” for the regression equation?
– Should lags be included as explanatory variables?
– How to deal with autocorrelation (correlation of a variable with itself)
and cross-correlation (correlation of a variable with another lagged
variable)?
Causal Methods:
AUTOREGRESSION
(Simple Linear Regression)
• Inferential forecasting model (time-series)
• Model: E(Yt) = β0 + β1t
• Relates time series, Yt, to time, t
• Cautions
– Risky to extrapolate (i.e. to forecast beyond observed data)
– Does not account for cyclical effects
Simple Linear Regression Example
The data shows the average
undergraduate tuition at all 4–
year institutions for the years
1996–2004 (Source: U.S. Dept.
of Education). Use least–
squares regression to fit a linear
model.
Forecast the tuition for 2005 (t =
11) and compute a 95%
prediction interval for the
forecast.
Simple Linear Regression Solution
From R
Yˆt 7997.533 528.158t
Simple Linear Regression Solution
$15,000
Yˆt 7997.533 528.158t
$14,000
$13,000
Tuition
$12,000
$11,000
$10,000
$9,000
$8,000
1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005
Year
Simple Linear Regression Solution
Forecast tuition for 2005 (t = 11) with
95% prediction interval:
Autocorrelation
• Time series data may have errors that are not
independent
• Time series residuals: Rˆt Yt Yˆt
• Correlation between residuals at different points in
time or between values related to their past values
(autocorrelation)
• 1st order correlation: Correlation between
neighboring residuals (times t and t + 1)
Durbin–Watson Test
• H0: No first–order autocorrelation of residuals
• Ha: Positive first–order autocorrelation of
residuals
• Test Statistic (for residuals)
n
2
ˆ
R ˆ
Rt t 1
d t 2
n
Rt
ˆ
t 1
2
Interpretation of Durbin-Watson d-
Statistic
Rˆ Rˆ
n
t t 1
d t 2
n
Range of d : 0 d 4
t
ˆ
R 2
t 1
1. If the residuals are uncorrelated, then d ≈ 2.
2. If the residuals are positively autocorrelated, then
d < 2, and if the autocorrelation is very strong,
d ≈ 0.
3. If the residuals are negatively autocorrelated, then
d >2, and if the autocorrelation is very strong,
d ≈ 4.
Durbin–Watson Test Example
Use the Durbin–Watson test to test for the
presence of autocorrelation in the tuition data. Use
α = .05.
Durbin–Watson Solution
Extrapolation Methods
• Extrapolation methods are quantitative methods that use
past data of a time series variable to forecast future values of
the variable.
• Many methods are available, for example:
– Autoregression
– Moving averages
– Exponential smoothing
• Extrapolation methods look for patterns in the historical series
and then extrapolate these patterns into the future.
• Complex methods are not always superior to simple methods.
Components of Time Series Data
• Trend component: If the observations increase or decrease
regularly through time.
– Linear trend occurs if the observations increase by about
the same amount from one period to the next.
– Exponential trend occurs when observations increase at a
tremendous rate.
– S-shape trend occurs when it takes a while for
observations to start increasing, but then a rapid increase
occurs, leveling off to a fairly constant level.
• Seasonal component: Occurs when observations stay
relatively high during a certain period of time, then drop off for
a period, and when this pattern repeats (is predictable) from
one period to the next.
Components of Time Series Data
(Continued)
• Cyclic component occurs when business cycles affect many
variables in similar ways.
– Cyclic component is more difficult to predict than the
seasonal component, because seasonal variation is more
regular.
– The length of the business cycle varies, sometimes
substantially.
• Random variation (or simply white noise) is the
unpredictable component that gives most time series graphs
their irregular, zigzag appearance.
– Random variation adds a certain amount of unpredictability
to almost all time series.
Time Series Decomposition
Autoregression Models
• A regression-based extrapolation method that
regresses the current value of the time series on
past (lagged) values is called autoregression.
• Auto means that the explanatory variables in the
equation are lagged values of the dependent
variable, so that the dependent variable is
regressed on lagged versions of itself.
Moving Averages
• Simplest and one of the most frequently used extrapolation
methods is the moving averages method.
• The essence of the method is to average the observations of
the past few periods, where the number of terms in the
average is the span.
• The role of the span is important. If the span is large, then the
extreme values have relatively little effect on the forecasts.
• Use of span requires some judgment:
– If ups and downs in the series appear to be random noise,
then use a relatively large span.
– If each up and down in the data is predictable, use a
smaller span to forecast future fluctuations.
Example: Gasoline Sales
The data below show the number of gallons of
gasoline (in 1,000s) sold by a gasoline distributor in
Bennington, Vermont, over the past 12 weeks. The
time series appears to be stable over time, so
smoothing methods are applicable.
Week Sales Week Sales
1 17 7 20
2 21 8 18
3 19 9 22
4 23 10 20
5 18 11 15
6 16 12 22
Example: Gasoline Sales
Moving Average
To forecast gasoline sales for week 13 (in 1,000s)
using a 3-week moving average, we need to compute
the average of sales for weeks 10, 11, and 12. The week
13 forecast (F13) is:
20 15 22
F13 Moving Average (weeks 10 12) 19
3
Smoothing Methods
• Weighted Moving Average Method
In the weighted moving average method
for computing the average of the most recent
n periods, the more recent observations are
typically given more weight than older
observations. For convenience, the weights
usually sum to 1.
Example: Gasoline Sales
Three-Period Weighted Moving Average
To forecast gasoline sales (in 1,000s) for week 13
using a 3-week weighted moving average, we will
compute the weighted average of sales for weeks 10,
11, and 12 using the weights of 1/6, 2/6, and 3/6. The
week 13 forecast (F13) is:
F13 = 1/6(20) + 2/6(15) + 3/6(22)
= 3.333 + 5.000 + 11.000
= 19.333
Exponential Smoothing
• Exponential smoothing allows to solve
problems created by the moving averages
method:
– Forecast is based on a weighted average of
past observations, with more weight on the
more recent observations.
– Requires very little data storage, and is easy to
understand conceptually.
• Simple exponential smoothing is appropriate for a
series with no pronounced trend or seasonality.
• Holt’s method is appropriate for a series with
trend but no seasonality.
Weights in Exponential Smoothing:
Simple Exponential Smoothing
• Every exponential model has at least one
smoothing constant, which is always a number
between 0 and 1.
• Simple exponential smoothing has a single
constant denoted by α.
• The level of the series at time t is an estimate
of where the series would be at time t if there
were no random noise.
Smoothing Methods
• Exponential Smoothing
Ft+1 = aYt + (1 – a)Ft
where:
Ft+1 = forecast of the time series for period t+1
Yt = actual value of the time series in period
t
Ft = forecast of the time series for period t
a = smoothing constant (0 < a < 1)
(To start the calculations, we let F1 equal the
actual value of the time series in period 1.)
Example: Gasoline Sales
• Exponential Smoothing (a = .2, 1 - a = .8)
F1 = 17
F2 = .2Y1 + .8F1 = .2(17) + .8(17) = 17.00
F3 = .2Y2 + .8F2 = .2(21) + .8(17) = 17.80
F4 = .2Y3 + .8F3 = .2(19) + .8(17.80) = 18.04
F5 = .2Y4 + .8F4 = .2(23) + .8(18.04) = 19.03
F6 = .2Y5 + .8F5 = .2(18) + .8(19.03) = 18.83
F7 = .2Y6 + .8F6 = .2(16) + .8(18.83) = 18.26…
Holt’s Model for Trend
• When there is a trend in the series, Holt’s method deals with it
explicitly by including a trend term, Tt, and a corresponding
smoothing constant β.
• The interpretation of Lt is exactly the same as before.
• The interpretation of Tt is that it represents an estimate of the
change in the series from one period to the next.
• Formulas for Holt’s exponential smoothing method:
Seasonal Models
• Seasonality is the consistent month-to-month (or quarter-to-
quarter) differences that occur each year.
• The easiest way to check for seasonality is graphically: Look for a
regular pattern of ups and/or downs in particular months or
quarters.
• There are three methods of dealing with seasonality.
– Winter’s exponential smoothing model.
– Deseasonalizing the data, and then follow with any other
forecasting method.
– Multiple regression with dummy variables for seasons.
• Seasonal models are additive or multiplicative.
– Additive models find seasonal indexes that are added to the
monthly average.
– In multiplicative models, a based forecast is multiplied by an
appropriate seasonal index.
Stationary Seasonal Effects
A d d itiv e S e a s o n a l E ffe c ts
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
T im e P e r io d
M u ltip lic a tiv e S e a s o n a l E ffe c ts
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
T im e P e r io d
Winters’ Exponential Smoothing
Method
• Similar to Holt’s model, but also has seasonal
indexes and a corresponding smoothing constant γ
(gamma).
• The smoothing constant gamma controls how quickly
the method reacts to observed changes in the
seasonality pattern.
Deseasonalizing:
The Ratio-to-Moving-Averages
Method
• To deseasonalize an observation (assuming a multiplicative
model of seasonality), divide it by the appropriate seasonal
index.
• To find the exact index, divide the period’s observation by
the average of all observations for all periods.
Seasonal Regression Models
• Takes into account secular trend and seasonal effects
(seasonal component)
• Uses multiple regression models
• Dummy variables to model seasonal component
• E(Yt) = β0 + β1t + β2Q1 + β3Q2 + β4Q3
where
1 if quarter i
Qi
0 if not quarter i
• Remember “omit one category” rule?
Measures of Accuracy
• The forecast error is the difference between the actual value and
the forecast, Et-k,t = Yt - Ft-k,t .
• Forecasting software packages report several summary measures
of the forecast errors, including:
• Mean Absolute Error:
• Root Mean Square Error:
• Mean Absolute Percentage Error:
Package “forecast” in R
The Difficulty with Long-Term Forecasts –an
afterthought!
SIMULATION
Simulation Modeling
• Simulation is one of the most frequently employed
management science techniques.
• It is typically used to model random processes that
are too complex to be solved by analytical methods.
• One begins a simulation by developing a
mathematical statement of the problem.
• Input values for the model as well as probability
estimates for the random variables must then be
determined.
Random Variables
• Random variable values are utilized in the model through
a technique known as Monte Carlo simulation.
• Each random variable is mapped to a set of numbers so
that each time one number in that set is generated, the
corresponding value of the random variable is given as an
input to the model.
• The main assumption of this approach is that a randomly
chosen sample tends to exhibit the same properties as the
population from which it as drawn.
• The mapping is normally done in such a way that the
likelihood that a particular number is chosen is the same
as the probability (frequency distribution) that the
corresponding value of the random variable occurs.
Pseudo-Random Numbers
• Because a computer program generates
random numbers for the mapping according
to some formula, the numbers are not truly
generated in a random fashion.
• However, using standard statistical tests, the
numbers can be shown to appear to be
drawn from a random process.
• These numbers are called pseudo-random
numbers.
Simulation Applications
New Product Development
…to determine the probability that a new product
will be profitable.
Airline Overbooking
…to determine the number of reservations an
airline should accept for a particular flight.
Inventory Policy
…to choose an inventory policy that will provide
good customer service at a reasonable cost.
Waiting Lines
…to determine the waiting times for customers at a
bank’s ATM.
Advantages of Using Simulation
Among the advantages of simulation is the ability to
gain insights into the model solution which may be
impossible to attain through other techniques.
Also, once the simulation has been developed, it
provides a convenient experimental laboratory to
perform "what if" and sensitivity analysis.
Disadvantages of Using Simulation
A large amount of time may be required to develop
the simulation.
There is no guarantee that the solution obtained will
actually be optimal.
Simulation is, in effect, a trial and error method of
comparing different policy inputs.
It does not determine if some input which was not
considered could have provided a better solution for
the model.
Homework Assignment
Reading:
1
W “Basic Business Statistics: Concepts and Applications”
Chapter 16
“Forecasting: principles and practice”
Free online book
[Link]