MA23214

Stationary Time Series

ARIMA models

Identification

Estimation and

Forecasting
 Time Series

A time series is a set of observations made at

specified times and arranged in a chronological
order.
Common examples of time series data include

 Stock market prices recorded every minute.

 Daily temperature recordings in a city.
 Monthly sales data for a retail store.
 Quarterly GDP of a country.
 Yearly rainfall data in a specific region.
Uses of Time Series
 Time series is used to predict future values based on previously
observed values.
 Time series analysis is used to identify the fluctuation in
economics and business.
 It helps in the evaluation of current achievements.
 Time series is used in pattern recognition, signal processing,
weather forecasting and earthquake prediction.
 Components of Time Series
The factors that are responsible for bringing about changes in a time series are called the components of time
series.

Secular Seasonal Cyclical Irregular

Trend Variations Variations Variations
Secular Trend
 Secular Trend is also called long term trend or simply trend.
 The secular trend refers to the general tendency of data to grow or decline over a long
period of time.
Mathematically the secular trend may be classified into two types
1. Linear Trend
2. Curvi-Linear Trend or Non-Linear Trend.

If one plots the trend values for the time series

on a graph paper and if it gives a straight line
then it is called a linear trend i.e. in linear trend
the rate of change is constant where as in non-
linear trend there is varying rate of change.
Let’s consider the population of a city over 10 years:

The population increases steadily over the years. This upward movement
shows a positive secular trend.
Seasonal Variations
Seasonal variations are fluctuations within a year over different seasons.
Estimation of seasonal variations requires that the time series data are recorded at
even intervals such as quarterly, monthly, weekly or daily, depending on the
nature of the time series. Changes due to seasons, weather conditions and social
customs are the primary causes of seasonal variations.
The main objective of the measurement of seasonal variation is to study their
effect and isolate them from the trend.

There occur seasonal fluctuations in a time series due

to two factors.
 Due to natural forces
 Manmade convention.
Electricity usage increases during summer due to air conditioners and
coolers, and again in winter in colder regions due to heating appliances.
This cyclical rise and fall each year is a seasonal variation in power de
mand.
Cyclical Variations

Cyclical variations refer to periodic movements in the time series about the trend line,
described by upswings and downswings. They occur in a cyclical fashion over an
extended period of time (more than a year).
A construction firm earns more during years of economic boom but sees lower
profits during recession years. These ups and downs over 4–8 years show
cyclic variation.
Irregular Variation

This type of fluctuations occurs in random way or irregular ways which are unforeseen
, unpredictable and due to some irregular circumstances which are beyond the control
of human being such as earth quakes, wars, floods, famines, lockouts, etc. These
factors affect the time series in the irregular ways.

 Auto regressive models(AR)

 Moving average models (MA)
 Auto regressive moving average models (ARMA)
 Auto regressive integrated moving average models(ARIMA)
During the COVID-19 outbreak, airline travel dropped sharply due to lockdowns and
travel bans. This sudden drop was not seasonal, cyclic, or part of a trend—it was
an irregular variation caused by a global crisis.
 Mathematical Model
There are two approaches to the decomposition of time series data

Additive model Multiplicative model

𝒀𝒕 = 𝑻𝒕 + 𝑺𝒕 + 𝑪𝒕 + 𝑹𝒕 𝒀𝒕 = 𝑻𝒕 × 𝑺𝒕 × 𝑪𝒕 × 𝑹𝒕
A small beverage company records its monthly sales in 1000’s of units across the
year
Month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Sales 120 118 132 130 142 144 155 152 163 165 170 180

Perform seasonal decomposition of the time series using

(i) Additive models

(ii) Multiplicative models
Months Sales Trend Additive Multiplicative
𝒀𝒕 𝑻𝒕 𝑺𝒕 = 𝒀𝒕 − 𝑻𝒕 Irregular 𝑺𝒕 = 𝒀𝒕 /𝑻𝒕 Irregular
𝒀𝒕−𝟏 + 𝒀𝒕 + 𝒀𝒕+𝟏 𝑬𝒕
=
𝟑 𝑬𝒕 = 𝒀𝒕 ÷ (𝑻𝒕
= 𝒀𝒕 − 𝑻𝒕 − 𝑺𝒕 × 𝑺𝒕 )

Jan 120 - - - - -
Feb 118 123.33 -5.33 0 0.9568 1
Mar 132 126.69 5.33 0 1.0419 1
Apr 130 134.67 -4.67 0 0.9653 1
May 142 138.67 3.33 0 1.0244 1
Jun 144 147 -3 0 0.9796 1
Jul 155 150.33 4.67 0 1.0311 1
Aug 152 156.67 -4.67 0 0.9702 1
Sep 163 160 3 0 1.0188 1
Months Sales Trend Additive Multiplicative
𝒀𝒕 𝑻𝒕 𝑺𝒕 = 𝒀𝒕 − 𝑻𝒕 Irregular 𝑺𝒕 = 𝒀𝒕 /𝑻𝒕 Irregular
𝒀𝒕−𝟏 + 𝒀𝒕 + 𝒀𝒕+𝟏 𝑬𝒕
=
𝟑 𝑬𝒕 = 𝒀𝒕 ÷ (𝑻𝒕
= 𝒀𝒕 − 𝑻𝒕 − 𝑺𝒕 × 𝑺𝒕 )

Oct 165 166 -1 0 0.9940 1

Nov 170 174 -1.67 0 0.977 1
Dec 180 - - - - -
The monthly production (in units) of a factory over a year is given below:

Month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Sales 200 210 230 250 270 300 310 305 295 280 260 240

Perform seasonal decomposition of the time series using

(i) Additive models
(ii) Multiplicative models
A retail store records its quarterly sales (in ₹ lakhs) for a single year as follows::
Quarter Q1 Q2 Q3 Q4
Sales 80 100 90 130

Perform seasonal decomposition of the time series using

(i) Additive models
(ii) Multiplicative models
Stationary Processes
A time series is said to be stationary if there is no systematic
change in mean, variance and covariance of the observations
over a period of time.
 𝑛
𝑦𝑖
𝑦=
𝑁
𝑖=1

𝑁−𝑘
𝑦𝑡 − 𝑦 𝑦𝑡+𝑘 − 𝑦
Auto covariance 𝑟𝑘 =
𝑁−𝑘
𝑖=1

𝑁 𝑦𝑡 −𝑦 𝑦𝑡 −𝑦 𝑁 𝑦𝑡 −𝑦 2
If 𝑘 = 0 ⇒ 𝑟0 = 𝑖=1 = 𝑖=1 = 𝜎𝑌2
𝑁 𝑁

𝑟𝑘
Auto correlation 𝜌𝑘 =
𝑟0
Determine the autocorrelation coefficient 𝜌1 and auto covariance 𝑟1
1.6,0.8,1.2,0.5,0.6,1.5,0.8,1.2,0.5 and 1.3.
Given Data:
47, 64, 23, 71, 38, 64, 55, 41, 59, 48
Calculate:
Mean (𝑥 )
Auto covariance at lag 1 (𝑟1 )
Autocorrelation coefficient at lag 1 (𝜌1 )

Final Answers:
Mean = 51
Autocovariance 𝛾1 ≈ -149.7
Autocorrelation coefficient 𝜌1 ≈ -0.79
 Correlogram

A Correlogram is a graphical representation of the autocorrelation function (A

CF) of a time series.
Calculate 𝜌1 , 𝜌2 , 𝜌3 , 𝜌4 and 𝜌5 and hence plot a correlogram for the data
47, 64, 23, 71, 38, 64, 55, 41, 59, 48.
𝑦𝑡∗𝑘 = 𝑦𝑡 − 𝑦 𝑦𝑡+𝑘 − 𝑦

𝑦𝑡∗1 = 𝑦𝑡 − 𝑦 𝑦𝑡+1 − 𝑦
𝑦𝑡∗2 = 𝑦𝑡 − 𝑦 𝑦𝑡+2 − 𝑦

𝑦𝑡∗3 = 𝑦𝑡 − 𝑦 𝑦𝑡+3 − 𝑦

𝑦𝑡∗4 = 𝑦𝑡 − 𝑦 𝑦𝑡+4 − 𝑦

𝑦𝑡∗5 = 𝑦𝑡 − 𝑦 𝑦𝑡+5 − 𝑦
Linear Stationary Process
A linear stationary process in time series is a series whose statistical
properties (mean, variance, covariance) do not change over time, and
can be represented as a linear combination of past error terms.
Two particular cases of the linear stationary processes.

Moving Average (MA) Process

Autoregressive (AR) Process

Moving Average (MA) Process
The economic indicators are affected by many random events such as
government decisions, strikes and shortages of raw materials, etc. They
have immediate effects as well as effects of lower magnitude in past
periods. Such processes have been successfully modelled by moving
average processes.
 First Order
Moving Average (MA) Processes
 Second order
Moving Average MA(2) process
 Invertibility
the autocorrelation coefficients should satisfy the condition of invertibility, which is satisfied
Classify the following MA process, find its auto correlation coefficients
𝑋𝑡 = 𝑎𝑡 + 0.74𝑎𝑡−1 − 0.19𝑎𝑡−2
Classify the following MA process, find its auto correlation coefficients
𝑋𝑡 = 𝑎𝑡 + 0.74𝑎𝑡−1 − 0.19𝑎𝑡−2
 Autoregressive Models

An autoregressive model is when a value from a time series is regressed on

previous values from that same time series.
First Order Autoregressive {AR(1)} (Markov) Process

Autocorrelations

For stationarity
 Second order Autoregressive {AR (2)} process

This process is obtained by taking p = 2 and the model is

𝜌3 = 𝛼1 𝜌2 + 𝛼2 𝜌1

𝜌4 = 𝛼1 𝜌3 + 𝛼2 𝜌2

𝜌5 = 𝛼1 𝜌4 + 𝛼2 𝜌3

For stationarity
Determine whether the processes are stationary or not:
For an autoregressive AR (p) process, the partial autocorrelation function
(pacf) is defined as the value of the last coefficient 𝛼𝑝.

The AR (1) process, pacf (1) 𝛼1 = 𝜌1

𝜌2 − 𝜌12
For AR (2), the pacf is 𝑝𝑎𝑐𝑓 2 = 2 = 𝛼2
1 − 𝜌1
 Fitting an Autoregressive Process
2
 Calculate the range ± 𝑁
 The partial autocorrelation function values 𝒑𝒂𝒄𝒇 (𝟏) = 𝝆𝟏 = 𝜶𝟏

𝟏 𝝆𝟏 𝝆𝟐
𝝆𝟐 − 𝝆𝟐𝟏 𝝆𝟏 𝟏 𝝆𝟏
𝒑𝒂𝒄𝒇 𝟐 = = 𝜶𝟐 𝝆 𝝆𝟏 𝝆𝟑
𝟏 − 𝝆𝟐𝟏 𝒑𝒂𝒄𝒇 𝟑 = 𝟐
𝟏 𝝆𝟏 𝝆𝟐
= 𝜶𝟑
𝝆𝟏 𝟏 𝝆𝟏
𝝆𝟐 𝝆𝟏 𝟏

 Determine how many pacf values lie in the range defined in step 1
If the 𝜌𝑘 lies in the range then 𝑘 − 1 is the order of the AR model
Suppose for a time series of length N = 100, the three autocorrelation coefficients
are 𝛼1 = 0.806, 𝛼2 = 0.428, 𝛼3 = 0.070. Calculate the pacfs and estimate the order
of autoregressive model to be fitted.
Suppose that the correlogram of a time series consisting of 100 observations
has 𝑟1 = 0.31, 𝑟2 = 0.37, 𝑟3 = – 0.05, 𝑟4 = 0.06, 𝑟5 = – 0.21, 𝑟6 = 0.11,
𝑟7 = 0.08, 𝑟8 = 0.05, 𝑟9 = 0.12, 𝑟10 = – 0.01
Suggest an ARIMA model which may be appropriate for this case.
AUTOREGRESSIVE INTEGRATED MOVING AVERAGE (ARIMA) MODELS

A finite order moving average process can be written as an infinite order

autoregressive process. Similarly, a finite order autoregressive process can
be written as an infinite order moving average process. We would like to fit
a model, which has the least number of parameters. This property is called
parsimony (most economical). Hence, a combination of autoregressive
(AR) and moving average (MA) models may turn out to be the most
parsimonious.

We represent a combination of AR(p) and MA(q) model as ARMA(p, q) and write

AUTOREGRESSIVE INTEGRATED MOVING AVERAGE (ARIMA) MODELS

If a time series is non-stationary because of changes in mean, we can take

the difference of successive observations. The modified series is more likely
to be stationary. Sometimes more than one difference of successive
observations is required to get a modified stationary model. Such a model is
called an integrated model because the stationary model that is fitted to the
modified series has to be summed or integrated to provide a model for the
original non-stationary series.
where 𝛻 is the difference operator. This is called the difference of order 1.
AR component 𝜙 𝐵 𝑊𝑡

MA component 𝜃 𝐵 𝑎𝑡

It is denoted by ARIMA (p, d, q).

𝑜𝑟
The conditions of stationarity and invertibility are

the roots of Φ (𝐵) = 0 and 𝜃(𝐵) = 0 … must lie outside the unit circle.
So the modulus of roots of B must be greater than one.
For the model 1 − 0.2𝐵 2 𝑋𝑡 = 1 − 0.5B at , find 𝑝, 𝑑, 𝑞 and
express it as a ARIMA (p,d,q). Determine whether the process
is stationary and invertible.
For the model 1 − 𝐵 𝑋𝑡 = 1 − 0.5B at , find 𝑝, 𝑑, 𝑞 and
express it as a ARIMA (𝑝, 𝑑, 𝑞). Determine whether the process
is stationary and invertible.
Identify each of the following models based on the values of p,d,q. Specify the order
And the model of the parameter.

𝑋𝑡 = 10 + 𝑋𝑡−1 + 𝑎𝑡 + 0.6𝑎𝑡−1
Identify each of the following models based on the values of p,d,q. S[ecify the order
And the model of the parameter.

𝑋𝑡 = 3 + 1.25 𝑋𝑡−1 − 0.25𝑋𝑡−2 + 𝑎𝑡 + 0.6𝑎𝑡−1

Forecasting the results of an ARIMA model

Analyse the ARIMA model 1 − 0.2𝐵 1 − 𝐵 𝑋𝑡 = 1 − 0.5B at and forecast three

future values if 𝑋4 = 8 and 𝑋5 = 10
Analyse the ARIMA model 1 − 0.2𝐵 2 𝑋𝑡 = 1 − 0.3B at and forecast three
future values if 𝑋5 = 10 and 𝑋6 = 13.
Thank you