STA457: Time Series Analysis

Lecture 12

Lijia Wang

Department of Statistical Sciences

University of Toronto

Lijia Wang (UofT) STA457: Time Series Analysis 1 / 19

Overview

Last Time:
1 Integrated ARMA (ARIMA) models
2 Building ARIMA models
Today:
1 Regression with Auto-correlated Errors
2 Multiplicative Seasonal ARIMA Models

Lijia Wang (UofT) STA457: Time Series Analysis 2 / 19

Outline

1 Regression with Auto-correlated Errors

2 Multiplicative Seasonal ARIMA Models

Lijia Wang (UofT) STA457: Time Series Analysis 3 / 19

Regression with Auto-correlated Errors: Introduction

Definition: We consider the regression model with correlated errors xf

r
X
yt = βj ztj + xf ,
j=1

where xf is a process with some covariance function γx (s, t). Such a

model is called Regression with Auto-correlated Errors.

Lijia Wang (UofT) STA457: Time Series Analysis 4 / 19

Procedure to identify a model

1 First, run an ordinary regression of yt on zt1 , · · · , ztr (acting as if the

errors are uncorrelated). Retain the residuals,
r
X
x̂t = yt − βj ztj .
j=1

2 Identify ARMA model(s) for the residuals x̂t .

3 Run weighted least squares (or MLE) on the regression model with
autocorrelated errors using the model specified in step (ii).
4 Inspect the residuals ŵt for whiteness, and adjust the model if
necessary.

Lijia Wang (UofT) STA457: Time Series Analysis 5 / 19

If the error is AR(p)

If the error term has an AR(p) representation:

ϕ(B)xt = wt

Multiplying the regression equation through by the transformation ϕ(B)

yields,
Xr
ϕ(B)yt = βj ϕ(B)zt,j + ϕ(B)xt .
j=1

Lijia Wang (UofT) STA457: Time Series Analysis 6 / 19

Example: Mortality, Temperature and Pollution

We consider the following analyses relating mean adjusted temperature Tr ,

and particulate levels Pt to cardiovascular mortality Mt . We consider the
regression model

Mt = β1 + β2 t + β3 Tr + β4 Tr2 + β5 Pt + xt

where, for now, we assume that xt is white noise.

Lijia Wang (UofT) STA457: Time Series Analysis 7 / 19

Sample ACF and PACF of the residuals

Figure: Sample ACF and PACF of the mortality residuals indicating an AR(2)
process.
Lijia Wang (UofT) STA457: Time Series Analysis 8 / 19
Fit the correlated error model
Our next step is to fit the correlated error model, but where xt is AR(2).

xt = ϕ1 xt−1 + ϕ2 xt−2 + wt

and wt is white noise.

Lijia Wang (UofT) STA457: Time Series Analysis 9 / 19

Outline

1 Regression with Auto-correlated Errors

2 Multiplicative Seasonal ARIMA Models

Lijia Wang (UofT) STA457: Time Series Analysis 10 / 19

Multiplicative Seasonal ARIMA Models

In this section, we introduce several modifications made to the ARIMA

model to account for seasonal and nonstationary behavior. Often, the
dependence on the past tends to occur most strongly at multiples of some
underlying seasonal lag s.

Lijia Wang (UofT) STA457: Time Series Analysis 11 / 19

ARMA(P, Q)s

The pure seasonal autoregressive moving average model, say,

ARMA(P, Q)s , can be written by

ΦP (B s )xi = ΘQ (B s )wi ,

where the operators

ΦP (B s ) = 1 − Φ1 B s − Φ2 B 2s − · · · − ΦP B Ps

and
ΘQ (B s ) = 1 + Θ1 B s + Θ2 B 2s + · · · + ΘQ B Qs
are the seasonal autoregressive operator and the seasonal moving
average operator of orders P and Q, respectively, with seasonal period s.

Lijia Wang (UofT) STA457: Time Series Analysis 12 / 19

Properties of the ARMA(P, Q)s

Analogous to the properties of nonseasonal ARMA models, the pure

seasonal ARMA(P, Q)s is causal only when the roots of ΦP (B s ) lie
outside the unit circle, and it is invertible only when the roots of ΘQ (B s )
lie outside the unit circle.

Lijia Wang (UofT) STA457: Time Series Analysis 13 / 19

first-order seasonal AR

first-order seasonal autoregressive series that might run over months could
be written as
(1 − ΦB 12 )xt = wt
or
xt = Φxt−12 + wt .

This model exhibits the series xt in terms of past lags at the multiple
of the yearly seasonal period s = 12 months.
It is clear from the above form that estimation and forecasting for
such a process involves only straightforward modifications of the unit
lag case already treated.
In particular, the causal condition requires |Φ| < 1.

Lijia Wang (UofT) STA457: Time Series Analysis 14 / 19

first-order seasonal MA

first-order seasonal moving average series could be written as

xt = wt + Θwt−12 .
We can verify the auto-covariance that,

γ(0) = (1 + Θ2 )σ 2
γ(±12) = Θσ 2
γ(h) = 0 Otherwise

Lijia Wang (UofT) STA457: Time Series Analysis 15 / 19

Behavior of the ACF and PACF for pure SARMA models

Table: Behavior of the ACF and PACF for Pure SARMA Models
AR(P)s MA(Q)s ARMA(P, Q)s
Tails off at lags ks, Cuts off after
ACF Tails off at lags ks
k = 1, 2, . . . lag Qs
Tails off at lags ks,
PACF Cuts off after lag Ps Tails off lags ks
k = 1, 2, . . .

Note that the values of ACF and PACF at nonseasonal lags h ̸= ks, for
k = 1, 2, · · · , are zero.

Lijia Wang (UofT) STA457: Time Series Analysis 16 / 19

ARMA(p, q) × (P, Q)s

In general, we can combine the seasonal and nonseasonal operators into a

multiplicative seasonal autoregressive moving average model, denoted by

ARMA(p, q) × (P, Q)s

and write
ΦP (B S )ϕ(B)xt = ΘQ (B S )θ(B)wt .

Lijia Wang (UofT) STA457: Time Series Analysis 17 / 19

Example

Consider an ARMA(0, 1) × (1, 0)12 model

xt = ϕxt12 + wt + θwt1 ,

where |ϕ| < 1 and |θ| < 1. Find the ACF of xt .

Lijia Wang (UofT) STA457: Time Series Analysis 18 / 19

SARIMA model

Definition: The multiplicative seasonal autoregressive integrated moving

average model, or SARIMA model is given by

ΦP (B s ) ϕ(B)∇D d s
s ∇ xt = δ + ΘQ (B ) θ(B)wt ,

where wt is the usual Gaussian white noise process. The general model is
denoted as ARIMA (p, d, q)(P, D, Q)s .
The ordinary AR and MA components are represented by polynomials
ϕ(B) and θ(B) of orders p and q, respectively
The seasonal AR and MA components are represented by ΦP (B s )
and ΘQ (B s ) of orders P and Q
Ordinary and seasonal difference components by ∇d = (1 − B)d and
∇D s D
s = (1 − B )

Lijia Wang (UofT) STA457: Time Series Analysis 19 / 19