Time Series Analysis

Dr. Maryam Habadi

STAT 454
1
4 Autoregressive Model: AR(P)

´ The idea behind the autoregression models AR(P) is to explain the

present value of a series, !! , by a function of " past values
5
!!"#, !!"$, … , !!"% .

´ Autoregressive models are remarkably flexible at handling a wide

range of different time series patterns.
 OH
5 Autoregressive Model

An Autoregressive Model of order p AR(P) is written as :

!! = '#!!"# + '$!!"$ + ⋯ + '% !!"% + *!

Where: Ly 4Yt My 2 x OpDt Pt Et

' is the coefficient ( parameters)

P is the order of the model

*! random error *! ~, 0, . $
u
6
´A useful form follows by using the backshift operator to
write the AR(p) model as:
*! = !! − '#!!"# − '$!!"$ − ⋯ − '% !!"%

and applying 0!! = !!"#, we get

1 − '#0 − '$0$ − ⋯ − '% 0% !! = *!

or, using the concise notation we writeLt AYE 1

*! = Φ 0 !! Bea I

where LEE YE
Φ 0 = 1 − '#0 − '$0$ − ⋯ − '% 0%
 Ez AB Gt Et ACB Yt
Special case: AR(1)
Lift
7
II ee
off I
It has the following form:
q Yt_MT
*! = Φ 0 !!
Where 456418
Φ 0 = 1 − '#0
Bdt Yt
∴ *! = 1 − '#0 !!

Yt g py
ad E Tt 9 Ye

µ
 p yet babe rt Et
8 Special case: AR(2)order y
EYE2
Z EE Ye 4,8 1
It has the following form: Ee
*! = Φ 0 !! KEITH
EE B Yt
Where
Φ 0 = 1 − '#0 − '$0$
∴ *! = 1 − '#0 − '$0$ !!

Ye ABYE Az BLYE z

Ye 0 It i 02 Yet
 É
9 Notes
Autorgrsiney
´ The ACF of an AR(p) model is an exponential decay. T
b
´ The PACF of an AR(P) model is significant till P lags.
4 a AKA no of the
p ´ Partial autocorrelation plots are a commonly used tool for
identifying the order of an autoregressive model (p).

HE YTeria
 imp 2,2 QUIZ A
Example: ingassistantpace oh
me ai
´ Identify the order of the autoregressive models

using the ACF and PACF plots a

it
PAH
3
y
µ b

10 1,11 6,1 bits

11 Answer
´ Figure (a): The ACF has an exponential decay while the PACF is

I
significant at lag p⟹ Autoregressive model

´ Because lag 1 is significant in PACF ⟹ P = 1 ⟹ 67(1)

´ Figure (b): The ACF has an exponential decay while the PACF is
significant at lag p⟹ Autoregressive model

2
´ Because lag 1 is significant in PACF ⟹ P = 1 ⟹ 67(1)

´ The different between the two figures is that the first figure, the ACF is
decreasing slowly while the rate of decay is higher in the second ACF
⟹ the value of the paramenter ' will be different.
 D I x it's a 1 I
12 Note
s3 ARE
Is
´ Yule Walker equations were used to estimate the parameters of the AR(P) models.

´ The AR(P) models always stationarity models under a sufficient condition:

F
´ An AR(1) process is stationary if &: < 1

´ An AR(2) process is stationary under the following three conditions:

&: + &; < 1

o
&: − &; < 1
−1 < &; < 1

don't talk to me
 on seated Wim
O
13 The Moving Average Models: MA(q)
si errors É
Xi tax
´ The moving average models use past errors 3 I
as explanatory
0
variables.

´ Rather than using past values of the forecast variable (AR), a

moving average model uses past forecast errors in a regression-like
model. ___a
 The Moving Average Models: MA(q)
14
The moving average model of order q MA(q) has the general form:
!! = < + *! + =#HE
*!"# + =$*!"$ + ⋯ + =+ *!"+

155 Jew
5
84.4 + o d Wl is
Smk !! = < + *! + > =, *!",
,-#
108
410.1
Where

<: mean of the series ; =, ? = 1,2, … , A are the parameters of the

O O
models

*: is the random error *! ~, 0, . $

o
 I
15 In terms of the backward shift operator, the MA(q) model is Bette
!! = < + 1 + =#B + =$B$ + ⋯ + =+ B+ *!

+
G
!! = < + 1 + > =, B. *!
,-#

!! = < + Θ B *!
where Ttunction
+ no of paranters
Θ B = 1 + > =, B.
,-#
 once I was sevenyears old
16 Special cases:

9É
• The first order moving average MA(1) 9
yesh E

!! = < + *! + =#*!"# I 0 B Et
or Je M
!! = < + 1 + =#B *!

• The second order moving average MA(2)

BIo
!! = < + *! + =#*!"# + =$*!"$ fEÉI
IDK
y !! = < + 1 + =#B + =$B$ *!
metres
math
 job PALF
Notes
ACF
f jowl
17 if
if ou IN Ag tu Ey
´ A MA(q) models is always stationary for any values of the parameters =.
IE
´ The ACF of an MA(q) model is significant till q lags.
E
on
E orders s
ACE s
´ The PACF of an MA(q) model is an exponential decay.
MA aw
´ The ACF is an excellent tool in identifying the order of an MA(q) process,

because it is expected to "cut off" after lag q

S ACF SI MA JIE
orders
JI I
 p g
18 Note:

es
A moving average model is used for forecasting future values, while
moving average smoothing is used for estimating the trend-cycle of
past values.
 MULI I Acfsig
become
untelllage
Example (2)
a
MALI
´ Identify the order of the moving
average model using the ACF and
2 site
PACF plots Tiki est MACZ
A
ÉÉÉz
b
May
µ ugh
11
y
19
µF
 lag 9
movies
Answer verge a MF
20
´ Figure (a): The ACF is significant at lag 1 while the PACF has an

Me exponential decay ⟹ Moving Average model

´ Because lag 1 is significant in ACF ⟹ q = 1 ⟹ MA(1)

!! = < + *! + =#*!"#

´ Figure (b): The ACF is significant until lag 2 while the PACF has
sinusoidal ⟹ ⟹ Moving Average model
Ei
´ Because the ACF is significant until lag 2 ⟹ q = 2 ⟹ MA(2)
´ The coefficient of lag 1 is positive and lag 2 is negative
!! = < + *! + =#*!"# − =$*!"$

on a M I O B OzB Et
 231 É E M1 2 41 0501g
21 The ARMA models
Autoregressive !"($) and moving average &!(') can be effectively combined to form
t
a general and useful time series models knowns as !"&! ($, ') models. It can be
represented as follows:
A em
)E = + + -E + .: )EF: + .; )EF; + ⋯ + .G )EFG + 0: -EF: + 0; -EF; + ⋯ + 0H -EFH
j
Ek
the same G H

)E = + + -E + 1 .I )EFI + 1 0K -EFK
IJ: KJ:

Where $ and ' refers to the autoregressive and moving average terms.
 Applying backshift operator in ARMA(p, q) models and it is defined as
22
follows:

2)E = )EF:

For ARMA(p, q) models:

of G
AR(p): -E = Φ 2 )E ⟹ Φ 2 = 1 − ∑IJ: .I 2 I
J
B
H
MA(q): )E = Θ 9 -E ⟹ Θ 9 = 1 + ∑KJ: 0K 9 K
Ct B et
E's B E

414ft
Then, ARMA(p, q)

Φ 2 )E = C + Θ 9 -E
I
23 Behavior of ACF and PACF

at 551,01 is
Model ACF PACF

MA(q) significant spike at lag q Exponential decay and/or

damped sinusoidal
order it to God
AR (p) Exponential decay and/or significant spike at lag p
damped sinusoidal
order it is 00 af
ARMA Exponential decay and/or Exponential decay and/or
(p,q) damped sinusoidal damped sinusoidal
 maybe I term
24 Modelling procedure
WE
When fitting an ARMA model to a set of (non-seasonal) time series data, the
following procedure provides a useful general approach.

1. Plot the data and identify any unusual observations.

It is I fi
2. If necessary, transform the data to stabilize the variance.
if non constant ve
trans I a
3. If the data are non-stationary, take first differences of the data until the
__
data are stationary.
aww
8 1
25
Continue..
AR D MA 9
4. Examine the ACF/PACF: Is an ARMA(p, 0) or ARMA(0,q) model

Smh i i t o b w ji y ou s
appropriate?
I d
AIL
5. Try your chosen model(s) and use the AIC to search for a better model.
MSE I
j 81 A 6. Check the residuals from your chosen model

7. Calculate forecasts.
 i
26 White Noise Time Series
56 IT E o g
we
HIM E 1,5 6 653 53 It ts ji g
´ A time series is white noise if the variables are independent and
identically distributed with a mean of zero and has constant
É
variance. (it is a sequence of random numbers and uncorrelated
observations).

´ If your time series is white noise, it cannot be predicted.

 b J1
diff 114W I do lost
27 ARIMA(p, d, q) gets's
6
I
diff I MA AR DESE ED
´ If we combine differencing with autoregression and a moving
w it average model, we obtain an ARIMA(p, d, q) model. ARIMA is
diff stand for Autoregressive Integrated Moving Average.
I l
´ When no differencing is involved, the abbreviation ARMA may be
ARMI used.

past p sic'd AI
values ul Ell
0
 ARIMA(p, d, q) 21 1
ARimac
28

´ The full ARIMA(p, d, q) model can be written as:

L L
)EL = + + -E + .: )EF: + .; )EF; L
+ ⋯ + .G )EFG + 0: -EF: + 0; -EF; + ⋯ + 0H -EFH

where
TART EM
)EL is the differenced series (it may have been differenced more than once)

P: order of the autoregressive part

d: degree of the first differencing involved

q: order of the moving average part.

YI c 4 94 1
Y C Et 0.8873 Et
 To D 6
29 Special cases of ARIMA models

´ Many of the models we have already discussed are special cases of

the ARIMA model

ARIMA (0,0,0) White noise

ARIMA (p,0,0) Autoregression ARCP É
ARIMA (0,0,q) Moving Average
on
MACE
ARIMA Pio 9 ARMA P 9
 30
BYE Yt 1
It is much easier to work with the backshift notation. The general form of

did
ARIMA can be written in backshift notation as

guy
Anima W e 1 − '#B − ⋯ − '% B% 1 − B / !! = 1 + =#B + ⋯ + =+ B+ *!
backshifts y
opretor where ME ARIMA JI
B
67 " : 1 − '#B − ⋯ − '% B%

R6 A : 1 + =#B + ⋯ + =+ B+
d differences ∶ 1 − B / !!

l
BP
OB 425 Ap l Bdy it o B o Bt top Et
 order it is 0 I
ACF and PACF plots p w
31 g
ja ´ It is possible to use the ACF plot, and the closely related PACF plot, to determine
o d9 Jb I appropriate values for p and q.
MA Da
Sig unteng
DEAL
´ The data may follow an ARIMA(p,d,0) model if the ACF and PACF plots of the
differenced data show the following patterns:
PACF
exp dear • the ACF is exponentially decaying or sinusoidal;
or sin
• there is a significant spike at lag p in the PACF, but none beyond lag p.
Ex2
2
´ The data may follow an ARIMA(0,d,q) model if the ACF and PACF plots of the
differenced data show the following patterns:

• the PACF is exponentially decaying or sinusoidal;

• there is a significant spike at lag q in the ACF, but none beyond lag q.
 non seasnd u N UG del S
but data
32 for e Seasonal ARIMA models
SARIMANI I
I 061 Ed I zig
ARIMA models are also capable of modelling a wide range of seasonal
data.

A seasonal ARIMA model is formed by including additional seasonal

terms in the ARIMA models
33
YIMseason
The analytical form
period for monthlys
of seasonal ARIMA ", Z, A [, \, ] ! is defined by
Pant

It B#
Atma
Φ" '$ B 1−B % 1 − B # & ^' = =( B Θ) B# *' ,

E
where p, d and q as defined before, also , P is the order of the seasonal
autoregressive process, D is the order of the seasonal differencing, Q is the

Eres
order of the seasonal moving average, and the subindex S refers to the
seasonal period, with monthly data S=12; for quarterly data S=4.
34
ΦP B + , ', B , =- B , Θ. B + are defined as follows:

The non-seasonal components we have:

AR: ', B = 1 − '1 B − '2 B2 − ⋯ − ', B ,

s

MA: =- B = 1 + =1 B + =2 B2 + ⋯ + =- B .

The seasonal components are:

É
Seasonal AR: ΦP B + = 1 − Φ1 B + − Φ2 B2+ − ⋯ − Φ1 B1+

Seasonal MA:Θ. B + = 1 + Θ1 B + + Θ2 B2+ + ⋯ + Θ. B.+ .

 Example spical case
35
B I B ye I O B i t B Et
l OB l G BYy CI
ARIMA 1, 1, 1 1, 1, 1 2 is for quarterly data (S=4), and can be written as:

I 0 1 − ' B 1 − Φ B2 1 − B a 1 − B2 a! = 1 + = B 1 + Θ B # *
g I 3 3 ' 3 3 '

Ra
SARIMA P d 9 P D QI model
81
D
I
A

paratorsD

NGSJ
 i
What if More Than One Model Looks Okay?
36

my D Sometimes more than one model can seem to work for the same dataset.
j G D When that’s the case, some things you can do to decide between the models
Paratrs
I d N are:
AI C E E ´ Possibly choose the model with the fewest parameters.
ME U
NI I I ´ Examine MSE of forecast values. Pick the model with the generally
lowest MSE for predictions of the future.

´ Compare models with regard to AIC. Lower values of AIC is desirable.

o

finished
I'm finished too
0