Stat 602: Time series and forecasting Econometrics
1.1 Time Series
A time series is a set of observations that are recorded over a period of time. The
observations are denoted by 𝑥𝑥1 , 𝑥𝑥2 , … , 𝑥𝑥𝑛𝑛 or 𝑥𝑥𝑡𝑡 , 𝑡𝑡 = 1,2, … , 𝑛𝑛.
A time series is a sequence of observations taken over time. A time series can be
regarded as a sample from a stochastic process.
A time series can be univariate (a single characteristic recorded over time) or
multivariate (vector of observations recorded over time). In this module, the focus will
be on univariate time series that are observed at discrete points in time.
Time series are found in many fields such as:
1. Economics (monthly unemployment figures)
2. Meteorology (rainfall)
3. Medicine (electrocardiograms)
4. Agriculture (annual crop production figures)
5. Astronomy (solar activity)
6. Industrial processes (hourly viscosity readings)
7. Any field that takes observations over time, e.g.:
• the daily closing share price of Pick n Pay quoted on the JSE
• the weekly absenteeism rates for an organisation
• the daily occupancy rates at the Holiday Inn, Newlands
• the weekly pedestrian flows through the Eastgate Shopping Mall
• the monthly company turnover for Woolworths
• the quarterly value of new car sales published by NAAMSA
• the annual net immigration figures from the Department of Home Affairs.
1
Examples of some time series plots
Preliminary analysis of time series:
• Plot the series (observations) against time
• Identify components/unusual features of the series to describe the series.
• Decide on a tentative model/ method to describe the series.
1.2 Components of a Time Series
Generally, the following four underlying forces, individually or collectively determine
the variation in a time series in any period.
• Trend denoted by 𝑇𝑇𝑇𝑇
• Cyclical variation denoted by 𝐶𝐶𝐶𝐶
• Seasonal variation denoted by 𝑆𝑆𝑆𝑆
• Irregular or random variation denoted by 𝐼𝐼𝐼𝐼
Trend refers to the upward and downward pattern of a time series giving a reflection
of the long-term growth or decline in a time series. It is the general direction in which
the series would move (up, down, oscillate around a fixed level) over a long period of
time.
2
Cyclical variation describes alternating periods of relative expansion and contraction
or recession (in economic time series) usual of a year or more. Cycles vary in length
and intensity as each is caused by a host of factors that may differ in their effect from
one cycle to the next.
Seasonal variation refers to a periodical pattern in a time series that completes itself
in a calendar year and is repeated yearly thereafter. A period is defined as the time
during which one cycle of repetitive movements is completed. E.g. monthly rainfall
data. The period is one year.
3
Irregular variation refers to random and erratic movement in a time series and has
no regular pattern. The component of the series behaves in an unpredictable fashion.
1.3 Decomposition of a Time Series
The main aim of time series analysis is to isolate the influence of the four components
of the actual series.
The multiplicative time series model is used to analyse the influence of these four
components on the actual series.
The multiplicative time series model is defined as:
Actual 𝑦𝑦 = 𝑇𝑇𝑇𝑇 × 𝐶𝐶𝐶𝐶 × 𝑆𝑆𝑆𝑆 × 𝐼𝐼𝐼𝐼
1.4 Trend Analysis
The long-term trend in a time series can be isolated by removing the medium- and
short-term fluctuations (i.e. cycles, seasonal and random) in the series. This will result
in either a smooth curve or a straight line, depending on the method chosen.
Two methods for trend isolation can be used:
• the moving average method, which produces a smooth curve. A moving
average removes the short-term fluctuations in a time series by taking
successive averages of groups of observations.
4
• regression analysis, which results in a straight-line trend
1.5 Seasonal Analysis
Seasonal analysis isolates the influence of seasonal forces on a time series
1.6 Objectives of Time Series analysis
There are many objectives of time series analysis. Generally, comprise description,
explanation, prediction and control. In this module only, description and prediction
aspects will be considered. The description is based on the time series plot e.g.
electricity consumption in SA show an increasing trend and the highest consumption
peak in winter. Prediction/Forecasting-Much business planning depends on some
prediction/forecast values. So, given an observed time series, it is imperative that the
analyst should be able to predict the future values reliably.
The purpose of time series analysis is to identify any recurring patterns in a time series,
quantify these patterns through building a statistical model and then use the statistical
model to prepare forecasts to estimate future values of the time series.
1.7 Fundamental Concepts in Time Series
Stochastic process
A stochastic process is a model that describes the probability structure of a sequence
of observations
An observed set of random variables {𝑥𝑥𝑡𝑡 : 𝑡𝑡 = 1,2, … , 𝑛𝑛} can be regarded as a
realization (sample size n) from the infinite set of random variables {𝑋𝑋𝑡𝑡 : 𝑡𝑡 =
0, ±1, ±2, … , 𝑛𝑛} . Such a set of random variables is called a stochastic process. Its
values evolve in time according to probabilistic laws.
Means and Autocovariances
A stochastic process can be described in a simple way, i.e. by use of the first moment
the mean and its second moment called the autocovariance function.
The mean is given by 𝐸𝐸(𝑋𝑋𝑡𝑡 ) = 𝜇𝜇𝑡𝑡 .
The variance is given by 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡 ) = 𝜎𝜎𝑡𝑡2 .
The autocovariance function (𝛾𝛾𝑘𝑘 ) is defined as
𝛾𝛾𝑘𝑘 = 𝐶𝐶𝐶𝐶𝐶𝐶(𝑋𝑋𝑡𝑡 , 𝑋𝑋𝑡𝑡+𝑘𝑘 ) = 𝐸𝐸[(𝑋𝑋𝑡𝑡 − 𝜇𝜇)(𝑋𝑋𝑡𝑡+𝑘𝑘 − 𝜇𝜇)]
The autocorrelation function (ACF) is the correlation between 𝑋𝑋𝑡𝑡 and 𝑋𝑋𝑡𝑡+𝑘𝑘 , defined as
𝐶𝐶𝐶𝐶𝐶𝐶 (𝑋𝑋𝑡𝑡 , 𝑋𝑋𝑡𝑡+𝑘𝑘 ) 𝛾𝛾𝑘𝑘
𝜌𝜌𝑘𝑘 = =
�𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡 ) �𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡+𝑘𝑘 ) 𝛾𝛾0
We note that 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡 ) = 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡+𝑘𝑘 ) = 𝛾𝛾0 .
Properties of the autocovariances and autocorrelation functions:
5
• 𝛾𝛾0 = 𝐸𝐸[(𝑋𝑋𝑡𝑡 − 𝜇𝜇)]2 = 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡 )
• 𝛾𝛾𝑘𝑘 = 𝐸𝐸[(𝑋𝑋𝑡𝑡 − 𝜇𝜇)(𝑋𝑋𝑡𝑡+𝑘𝑘 − 𝜇𝜇)] = 𝐸𝐸[(𝑋𝑋𝑡𝑡−𝑘𝑘 − 𝜇𝜇)(𝑋𝑋𝑡𝑡 − 𝜇𝜇)] = 𝛾𝛾−𝑘𝑘
𝛾𝛾
• 𝜌𝜌0 == 𝛾𝛾0 = 1
0
𝛾𝛾𝑘𝑘 𝛾𝛾−𝑘𝑘
• 𝜌𝜌𝑘𝑘 = 𝛾𝛾0
= 𝛾𝛾0
= 𝜌𝜌−𝑘𝑘
• |𝜌𝜌𝑘𝑘 | ≤ 1 and |𝛾𝛾𝑘𝑘 | ≤ 𝛾𝛾0
Partial autocorrelation function (PACF) is the correlation of 𝑋𝑋𝑡𝑡 and 𝑋𝑋𝑡𝑡+𝑘𝑘 with their linear
dependency on the intervening variables 𝑋𝑋𝑡𝑡+1 , … , 𝑋𝑋𝑡𝑡+𝑘𝑘−1 removed.
𝜌𝜌𝑘𝑘 = 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶(𝑋𝑋𝑡𝑡 , 𝑋𝑋𝑡𝑡+𝑘𝑘 |𝑋𝑋𝑡𝑡+1 , … , 𝑋𝑋𝑡𝑡+𝑘𝑘−1 )
Stationarity
A stochastic process {𝑋𝑋𝑡𝑡 } is said to be weakly (or second order) stationary if:
• 𝐸𝐸(𝑋𝑋𝑡𝑡 ) = 𝜇𝜇 < ∞. Constant mean.
• 𝑉𝑉𝑉𝑉𝑉𝑉(𝑋𝑋𝑡𝑡 ) = 𝜎𝜎 2 . Constant variance.
• 𝐶𝐶𝐶𝐶𝐶𝐶(𝑋𝑋𝑡𝑡 , 𝑋𝑋𝑡𝑡−𝑘𝑘 ) =a function of only the time difference 𝑘𝑘. The autocovariance
function is constant over time, depends only on the time lag 𝑘𝑘.
(Thus, all statistical measures do no depend on 𝑡𝑡)
A time series is said to be stationary if:
• there is no systematic change in the mean (no trend)
• there is no systematic change in variance
• it contains no strictly periodic variations.
Example: Suppose 𝑍𝑍𝑡𝑡 = 5 + 2𝑡𝑡 + 𝑋𝑋𝑡𝑡 , where 𝑋𝑋𝑡𝑡 is a mean zero stationary series.
(a) Find the mean function for 𝑍𝑍𝑡𝑡 .
(b) Is 𝑍𝑍𝑡𝑡 stationary?
Solution
(a) 𝐸𝐸(𝑍𝑍𝑡𝑡 ) = 5 + 2𝑡𝑡, since 𝐸𝐸(𝑋𝑋𝑡𝑡 ) = 0.
(b) No, the mean is not constant. It is a function of 𝑡𝑡.
White noise process
A process 𝑎𝑎𝑡𝑡 of independent identically distributed random variables is known a white
noise process with constant mean [𝐸𝐸(𝑎𝑎𝑡𝑡 ) = 0] and constant variance [𝑉𝑉𝑉𝑉𝑉𝑉(𝑎𝑎𝑡𝑡 ) = 𝜎𝜎𝑎𝑎2 ].
Example: Calculate the autocovariance function and autocorrelation function of the
white noise process.
Solution
The autocovariance function:
𝛾𝛾0 = 𝐶𝐶𝐶𝐶𝐶𝐶 (𝑎𝑎𝑡𝑡 , 𝑎𝑎𝑡𝑡 ) = 𝑉𝑉𝑉𝑉𝑉𝑉(𝑎𝑎𝑡𝑡 ) = 𝜎𝜎𝑎𝑎2
𝛾𝛾1 = 𝐶𝐶𝐶𝐶𝐶𝐶 (𝑎𝑎𝑡𝑡 , 𝑎𝑎𝑡𝑡+1 ) = 0
6
𝛾𝛾𝑘𝑘 = 𝐶𝐶𝐶𝐶𝐶𝐶 (𝑎𝑎𝑡𝑡 , 𝑎𝑎𝑡𝑡+𝑘𝑘 ) = 0
𝑎𝑎𝑡𝑡 ′𝑠𝑠 are uncorrelated and therefore the covariance = 0. Thus
𝜎𝜎 2 , 𝑘𝑘 = 0
𝛾𝛾𝑘𝑘 = � 𝑎𝑎
0, 𝑘𝑘 ≠ 0.
The autocorrelation function (ACF):
𝜌𝜌0 = 1
𝛾𝛾1 0
𝜌𝜌1 = = 2=0
𝛾𝛾0 𝜎𝜎𝑎𝑎
𝛾𝛾2 0
𝜌𝜌2 = = 2 = 0.
𝛾𝛾0 𝜎𝜎𝑎𝑎
Thus
1, 𝑘𝑘 = 0
𝜌𝜌𝑘𝑘 = �
0, 𝑘𝑘 ≠ 0.
The backshift operator
The Backshift Operator, called 𝐵𝐵 shift time back one-time unit to form a new series.
𝐵𝐵(𝑌𝑌𝑡𝑡 ) = 𝑌𝑌𝑡𝑡−1
𝐵𝐵[𝐵𝐵(𝑌𝑌𝑡𝑡 )] = 𝐵𝐵(𝑌𝑌𝑡𝑡−1 ) = 𝑌𝑌𝑡𝑡−2
This can be written as
𝐵𝐵 2 (𝑌𝑌𝑡𝑡 ) = 𝑌𝑌𝑡𝑡−2
And more generally
𝐵𝐵 𝑛𝑛 (𝑌𝑌𝑡𝑡 ) = 𝑌𝑌𝑡𝑡−𝑛𝑛 .
Invertibility
To ensure uniqueness of the parameters in linear process (time series model), the
process must be invertible.
Linear Difference Equations
Let 𝑍𝑍𝑡𝑡 − 2𝑍𝑍𝑡𝑡−1 + 𝑍𝑍𝑡𝑡−2 = 0
The auxiliary equation is given by 𝐶𝐶(𝐵𝐵) = 1 − 2𝐵𝐵 + 𝐵𝐵 2 = 0.