LECTURE 1

Definition of A Time Series

• A time series is a sequential set of data points, measured typically over successive
times. It is mathematically defined as a set of vectors x(t),t = 0,1,2,... where t
represents the time elapsed . The variable x(t) is treated as a random variable. The
measurements taken during an event in a time series are arranged in a proper
chronological order.
A time series containing records of a single variable is termed as univariate.

But if records of more than one variable are considered, it is termed as

multivariate. A time series can be continuous or discrete. In a continuous time
series observations are measured at every instance of time, whereas a discrete time
series contains observations measured at discrete points of time. For example
temperature readings, flow of a river, concentration of a chemical process etc. can
be recorded as a continuous time series.

Usually in a discrete time series the consecutive observations are recorded at

equally spaced time intervals such as hourly, daily, weekly, monthly or yearly
time separations. The variable being observed in a discrete time series is assumed
to be measured as a continuous variable using the real number scale. Furthermore
a continuous time series can be easily transformed to a discrete one by merging
data together over a specified time interval.

Components of a Time Series

• A time series in general is supposed to be affected by four main components,

which can be separated from the observed data. These components are:
Trend, Cyclical, Seasonal and Irregular components. A brief description of
these four components is given here.

• The general tendency of a time series to increase, decrease or stagnate over a

long period of time is termed as Secular Trend or simply Trend. Thus, it can
be said that trend is a long term movement in a time series. For example,
series relating to population growth, number of houses in a city etc. show
 upward trend, whereas downward trend can be observed in series relating to
mortality rates, epidemics, etc.
• Seasonal variations in a time series are fluctuations within a year during the
season. The important factors causing seasonal variations are: climate and
weather conditions, customs, traditional habits, etc. For example sales of ice-
cream increase in summer, sales of woolen cloths increase in winter. Seasonal
variation is an important factor for businessmen, shopkeeper and producers for
making proper future plans.
• The cyclical variation in a time series describes the medium-term changes in
the series, caused by circumstances, which repeat in cycles. The duration of a
cycle extends over longer period of time, usually two or more years. Most of
the economic and financial time series show some
• kind of cyclical variation. For example a business cycle consists of four
phases, viz.
• i) Prosperity, ii) Decline, iii) Depression and iv) Recovery.

• Irregular or random variations in a time series are caused by unpredictable

influences, which are not regular and also do not repeat in a particular pattern.
These variations are caused by incidences such as war, strike, earthquake,
flood, revolution, etc. There is no defined statistical technique for measuring
random fluctuations in a time series.
• Considering the effects of these four components, two different types of
models are generally used for a time series viz. Multiplicative and Additive
models.
Multiplicative Model: Y(t) = T(t)× S(t)×C(t)× I (t).
Additive Model: Y(t) = T(t) + S(t) + C(t) + I (t).
 • Here Y(t) is the observation and T(t) , S(t) ,C(t) and I (t) are respectively the
trend, seasonal, cyclical and irregular variation at time t.
Multiplicative model is based on the assumption that the four components of a time
series are not necessarily independent and they can affect one another; whereas in the
additive model it is assumed that the four components are independent of each other.

Examples of Time Series Data

• Time series observations are frequently encountered in many domains such as
business, economics, industry, engineering and science. Depending on the nature
of analysis and practical need, there can be various different kinds of time series.
To visualize the basic pattern of the data, usually a time series is represented by a
graph, where the observations are plotted against corresponding time. Below we
show two time series plots:

Weekly BP/USD exchange rate series (1980-1993)

Monthly international airline passenger series (Jan. 1949-Dec. 1960)

• In time series forecasting, past observations are collected and analyzed to
develop a suitable mathematical model which captures the underlying data
generating process for the series.
• The future events are then predicted using the model. This approach is
particularly useful when there is not much knowledge about the statistical
pattern followed by the successive observations or when there is a lack of a
satisfactory explanatory model. Time series forecasting has important
applications in various fields. Often valuable strategic decisions and
precautionary measures are taken based on the forecast results. Thus making a
good forecast, i.e. fitting an adequate model to a time series is very important.
Over the past several decades many efforts have been made by researchers for
the development and improvement of suitable time series forecasting models.

Time Series and Stochastic Process

• A time series is non-deterministic in nature, i.e. we cannot predict with
certainty what will occur in future. Generally a time series {x(t),t = 0,1, 2,...}
is assumed to follow certain probability model which describes the joint
distribution of the random variable Xt. The mathematical expression
describing the probability structure of a time series is termed as a stochastic
process. Thus the sequence of observations of the series is actually a sample
realization of the stochastic process that produced it.

• A usual assumption is that the time series variables are independent and
identically distributed (i.i.d) following the normal distribution. An interesting
point is that time series are in fact not exactly i.i.d; they follow more or less
some regular pattern in long term. For example if the temperature today of a
particular city is extremely high, then it can be reasonably presumed that
tomorrow’s temperature will also likely to be high. This is the reason why
time series forecasting using a proper technique, yields result close to the
actual value.
1. Univariate modelling of stationary time series

Univariate time series models attempt to model/predict a financial series, using only
the information contained in its own past values.

Multivariate time series models attempt to model/predict several financial series. The
information contained in the past (and possibly current) values of all of the series is
used to model/predict each of the series.

A regression model with the following specification:

yt = 10 + 20xt + 1yt–1 + 21xt–1 + t

A widely used type of univariate time series model is based on a simplified version of
this regression, from which we remove xt and its lagged value xt–1, and specify a
regression model for yt in which the only explanatory variable is its own lagged value,
yt–1:

yt = 10 + 1yt–1 + t

This specification is known as a first-order autoregressive model, or AR(1) model:

 The model is ‘autoregressive’, because the only systematic determinant of the
evolution of yt is its own past value yt–1 (like an autobiography, in which the
subject describes his/her own past history).
 The model is ‘first-order’, because only one lagged value of the series yt is
used as a explanatory variable.

It is also possible to specify higher-order autoregressive models, containing higher-

order lagged values (yt–2, yt–3, ... etc.) on the right-hand-side. A second-order
autoregressive model, or AR(2) model, is specified as follows:

yt = 10 + 1yt–1 + 2yt–2 + t

An AR(2) model allows for a richer and more complex dynamic pattern in the time
series evolution of the series yt than an AR(1) model.

Time series models are often atheoretical  the specification is not based closely on
any particular theoretical model. Instead, the objective is to describe empirically
observable patterns in the evolution of a series over time.

An important class of univariate time series models is the family of Autoregressive

Moving Average (ARMA) models (Box and Jenkins, 1976).

In practice, ARMA models are often used for purposes of forecasting.

Stationarity and non-stationarity

ARMA modelling is applicable to the class of stationary time series variables. Figure
4.1 provides the intuition for the property of stationarity.
Case 1: yt is stationary.
Case 2: yt contains a deterministic trend and is therefore non-stationary. But if yt
is de-trended (by fitting the linear regression y t  â  b̂t  e t and using et
instead of yt), the de-trended series et is stationary.
Case 3: Although untrended, yt is non-stationary because over time it has a
tendency to ‘wander’ (infinitely far in either direction) from its initial value.
Case 4: yt contains a deterministic trend, and tends to increase over time. But yt also
displays a tendency to ‘wander’ away from the deterministic trend. De-
trending (as described above) does not induce stationarity.

More formally, a time series is said to be covariance stationary (also known as

weakly stationary) under the following conditions:
1. E(yt) =  for all t,
2. var(yt) = E(yt–)(yt–) = 2 <  for all t,
3. cov(yt,yt–k) = E(yt–)(yt–k–) = k for all t.

These conditions require:

 constancy over time of the mean and variance of yt,
 k, the autocovariance between any two observations k periods apart, depends
only on k (and not on t).

There are other forms of stationarity. In these notes, however, the term ‘stationarity’
will refer to the property of covariance stationarity (unless otherwise stated).

In Figure 4.1, we can see more formally the distinction between the four cases:

Case 1 E(yt) and var(yt) are both constant over time,

Case 2 E(yt) is increasing over time, but var(yt) is constant over time,
Case 3 E(yt) is constant over time, but var(yt) is increasing over time,
Case 4 E(yt) and var(yt) are both increasing over time.

Autocorrelation function (ACF) and partial autocorrelation function (PACF)

k=k/0 for k=0,1,2, ... is known as the autocorrelation function (where 0 = 2).

T
 ( y t  y)( y t k  y)
The sample autocorrelation function is ˆ k  t  k 1
T
,
 ( y t  y)
2

t 1
T
where y   y t / T
t 1

If the probability distribution of yt is normal, the asymptotic distribution of ̂ k is:

̂ k ~ N(0,1/T).
 Figure 4.1 Stationary and non-stationary time series variables

CASE 1 CASE 2
yt untrended yt trended & non-stationary;
& stationary but the de-trended series
is stationary

t t
CASE 3 CASE 4
yt yt
untrended trended & non-stationary;
& non-stationary the de-trended series is
also non-stationary

t t

Mean Variance

Case 1 Constant Constant

Case 2 Increases – but constant Constant
after detrending
Case 3 Constant Increases
Case 4 Increases Increases
This result provides the basis for significance tests for autocorrelation coefficients:

 To test H0:k=0 against H1:k0,  = Tˆ k ~ N(0,1)

 Accept H0 if –1.96    1.96

m
 To test H0:k=0 for k=1...m (against H1:k0 for some k),  = T  ˆ 2k ~ 2(m)
k 1

 Accept H0 if    (= 5% critical value from 2(m)).

2
0.05

In this test,  is known as the Box and Pierce (1970) Q-statistic.

 A modified version of the Box-Pierce test, with superior small sample

properties, was developed by Ljung and Box (1978).

In the Ljung-Box test, H0 is the same as in the Box-Pierce test.

m  ˆ2
 = T(T  2)  k ~ 2(m). The decision rule is the same.
k 1 T  k

As we have seen, the autocorrelation function, denoted k, measures the correlation
between yt and yt–k.

The partial autocorrelation function, denoted kk, measures the correlation between
yt and yt–k after controlling for the effects on yt of all the intermediate lags of this
series (i.e. yt–1, yt–2, ... , yt–k+1).

The theoretical formulae for kk (the true PACF) are complex. But the computation of
the sample PACF, denoted ̂ kk , is simple to describe, as follows:

̂11 is the estimated coefficient on yt–1 in the autoregression:

y  ˆ  ˆ y  e
t 0 11 t 1 t

̂ 22 is the estimated coefficient on yt–2 in the autoregression:

ty  ˆ  ˆ y  ˆ y
0 21 t 1 22 t 2  et
̂ 33 is the estimated coefficient on yt–3 in the autoregression:
ty  ˆ  ˆ y  ˆ y
0 31 t 1 32 t 2  ˆ 33 y t 3  e t
and so on.

White noise, moving average and autoregressive processes

If yt is stationary and k=0 for all k>0, yt is known as a white noise process.

For a white noise process, each observation is uncorrelated with every other
observation, or successive values of the series are completely random.
We can write yt =  + ut, where {ut} is a sequence of independent and identically
distributed random variables with E(ut)=0 and var(ut)=  0 .  denotes the mean of the
2

series {yt}. In this formulation, {ut} and {yt} are both white noise processes.

A moving average process is one where the current value of the variable is a linear
combination of white noise processes.

yt =  + ut + 1ut–1 + 2ut–2 + ... + qut–q

is a q-th order moving average process, denoted MA(q).

We can also write: yt =  + (1 + 1L+ 2L2 + ... + qLq)ut

where Liut = ut–i denotes the lag operator.

An MA(q) process has the following properties (see Brooks p236-8 for derivations):
1. E(yt) = ,
var(yt) = 2 = 0 = (1  1   2  ...   q ) 0 ,
2 2 2 2
2.
3. cov(yt,yt–k) = k > 0 for k=1...q; k = 0 for k>q (see below).

An MA(q) process is always stationary.

An autoregressive process is one where the current value of the variable is a linear
combination of the values the variable took in previous periods, plus a white noise
disturbance term.

yt = 0+ 1yt–1 + 2yt–2 + ... + pyt–p + ut

is a p-th order autoregressive process, denoted AR(p).

We can also write: yt = 0 + (1L + 2L2+ ... + pLp)yt + ut

A stationary AR(p) process has the following properties:

1. E(yt) = 0/(1+1+2+...+p),
2. var(yt) = 0 is finite,
3. cov(yt,yt–k) = k0 as k (see below).

A necessary condition for an AR(p) process to be stationary is that the roots of the
characteristic equation must lie outside the unit circle.

Using lag operator notation, an AR(p) process with zero mean can be written:

yt = 1yt–1 + 2yt–2 + ... + pyt–p + ut

 yt = (1L + 2L2 + ... + pLp)yt + ut

 (1 – 1L – 2L2 – ... – pLp)yt = ut

A more concise notation is: (L)yt = ut where (L) = (1 – 1L – 2L2 –...– pLp)

i.e. (L) denotes a p’th order polynomial in L.

The characteristic equation is the polynomial formed by the terms in the lag operator
on the left-hand-side of this expression:

(z) = 1 – 1z – 2z2 – ... – pzp = 0

The roots of the characteristic equation are the values of z which solve this equation.
These roots may be real or complex.

In general, the solution(s) can be written z=a+bi, where i=  1 :

 The root z is said to lie outside the unit circle if a2+b2>1.
 The root z is said to lie on the unit circle if a2+b2=1.

If the roots of the characteristic equation are complex, the series {yt} will follow a
regular cyclical pattern.

In the following examples, we will exclude this case by assumption, and we will only
consider cases where the roots of the characteristic equation are real ( b=0, so
z=a+bi=a).

Therefore the condition for the stationarity of the AR(p) process is that the absolute
values of the roots of the characteristic equation must exceed one.

If the absolute value of one (or more) of the roots equals one, the process is non-
stationary. In this case, the process is said to contain one (or more) unit roots.

(We will also exclude by assumption cases where one or more of these roots is less
than one).

Examples

(i) AR(1) process: yt = 0.8yt–1 + ut

 (1 – 0.8L)yt = ut

Characteristic equation is 1 – 0.8z = 0

The root (solution) is z=1/0.8 or z=1.25

z>1, so the series {yt} is stationary.

(ii) AR(1) process: yt = yt–1 + ut

 (1 – L)yt = ut

Characteristic equation is 1–z=0

The root (solution) is z=1, so the series {yt} is non-stationary, and contains
one unit root.

(iii) AR(2) process: yt = 0.6yt–1 + 0.3yt–2 + ut

 (1 – 0.6L – 0.3L2)yt = ut

Characteristic equation is 1 – 0.6z – 0.3z2 = 0

The roots (solutions) are z1=1.08, z2=–3.08

|z1|, |z2|>1, so the series {yt} is stationary.

(iv) AR(2) process: yt = 0.7yt–1 + 0.3yt–2 + ut

 (1 – 0.7L – 0.3L2)yt = ut

Characteristic equation is 1 – 0.7z – 0.3z2 = 0

The roots (solutions) are z1=1, z2=–3.33

|z1|=1, |z2|>1, so the series {yt} is non-stationary, and contains one unit root.

(v) AR(2) process: yt = 2yt–1 – yt–2 + ut

 (1 – 2L + L2)yt = ut

Characteristic equation is 1 – 2z + z2 = 0

The roots (solutions) are z1=1, z2=1

|z1|=1, |z2|=1, so the series {yt} is non-stationary, and contains two unit roots.

In all of these examples, the AR(p) process yt = 1yt–1 + 2yt–2 + ... + pyt–p + ut is:
 stationary if 1+2+ ... +p < 1
 non-stationary if 1+2+ ... +p = 1

This suggests a test of H0:1+2 + ... + p=1 can be used as a test for stationarity or
non-stationarity. This result forms the basis for unit root tests.
ACF and PACF for stationary AR(p) and MA(q) processes

The following table describes the behaviour of the autocorrelation function and the
partial autocorrelation function for a stationary AR(p) process and for an MA(q)
process, if these functions are evaluated over successive values of k.

AR(p) MA(q)

ACF k0 as k k>0 for kq, k=0 for k>q

PACF kk>0 for kp, kk=0 for k>p kk0 as k

For AR(p), the ACF decays gradually as k, while the PACF drops suddenly to
zero between k=p and k=p+1.

For MA(q), the ACF drops suddenly to zero between k=q and k=q+1, while the PACF
decays gradually as k.

Figure 4.2 illustrates. These patterns in the ACF and PACF imply that the sample
ACF and sample PACF contain useful information, which can help determine whether
to fit an autoregressive model or a moving average model, and the order of the chosen
model (p or q).

Box-Jenkins ARMA modelling

The relationships summarised in the previous table enabled Box and Jenkins (1976) to
develop a number of visual conventions for ARMA modelling (fitting autoregressive
or moving average models to stationary time series variables).

 Look at plots of the sample ACF and sample PACF as k increases.

 If the ACF decays gradually and the PACF drops suddenly after k=p, fit an
AR(p) model.
 If the ACF drops suddenly after k=q and the PACF decays gradually, fit an
MA(q) model.
 If the ACF and PACF both decay gradually, fit an ARMA(p,q) model.

Box and Jenkins also describe a hybrid specification, known as ARMA(p,q), which
contains both autoregressive and moving average components. For example:

ARMA(1,1): yt = 0 + 1yt–1 + ut + 1ut–1

ARMA(1,2): yt = 0 + 1yt–1 + ut + 1ut–1 + 2ut–2
ARMA(2,1): yt = 0 + 1yt–1 + 2yt–2 + ut + 1ut–1
ARMA(2,2): yt = 0 + 1yt–1 + 2yt–2 + ut + 1ut–1 + 2ut–2

... and so on.

For an ARMA(p,q) process, both the ACF and PACF decay progressively (and quite
quickly) towards zero as k.
 Figure 4.2 Sample ACF and PACF: sketch diagrams

ACF PACF
k  kk

k k

Case 1: ACF tails off, PACF cuts off at k=2, suggests AR(2) model is required

k  kk

k k

Case 2: ACF cuts off at k=3, PACF tails off, suggests MA(3) model is required
Box-Jenkins suggested the following visual conventions for ARMA modelling (fitting
autoregressive or moving average models) to a stationary time series variable yt:

 Look at plots of the sample ACF and sample PACF as k increases.

 If the ACF decays progressively toward zero and the PACF drops suddenly to
zero after k=p, fit an AR(p) model.
 If the ACF drops suddenly to zero after k=q and the PACF decays
progressively toward zero, fit an MA(q) model.
 If the ACF and PACF both decay progressively toward zero, fit an
ARMA(p,q) model. With this type of model, there is no easy guide for the
selection of p and q. However, it is common practice to restrict p and q to very
low values. ARMA(1,1) often provides a good representation for processes
containing both autoregressive and moving average components.

Having fitted the selected model (AR(p), MA(q) or ARMA(p,q)), use a Box-Pierce or
Ljung-Box test to determine whether the residuals from the fitted model are white
noise.

 If the residuals are white noise, the model may be satisfactory (although you
could still be guilty of over-fitting: using a more complex or higher-order
specification than you need in order to obtain white noise residuals).
 If the residuals are not white noise, by definition the residuals must contain
some non-random component. The model has not succeeded in describing all
of the non-random variation in yt. Try fitting a different (perhaps higher-order)
specification.

What if the ACF does not decay progressively (and quite quickly) towards zero, and
the ACF also does not drop suddenly to zero? In other words, what if the ACF
remains large (and most likely positive) over many high values of k? Then it is likely
the series yt is non-stationary.
Technical Appendix (non-examinable)

Wold’s decomposition theorem and the invertibility condition

This Appendix contains more information about the connections between

autoregressive and moving average processes.

As we have seen, any stationary AR(p) process can be written:

(1 – 1L – 2L2 – ... – pLp)yt = ut

A more concise notation is: (L)yt = ut

where (L) = 1 – 1L – 2L2 – ... – pLp

i.e. (L) is a p’th order polynomial in L.

According to Wold’s decomposition theorem, we can also write: yt = (L)–1ut

where (L)–1 = (1 – 1L – 2L2 – ... – pLp) –1

(L)–1 is also a polynomial in L, but with an infinite number of higher-order terms:

(L)–1 = 1 + 1L + 2L2 + ...

Therefore yt = (L)–1ut = (1 + 1L + 2L2 + ... )ut

 yt = ut + 1ut–1 + 2ut–2 + ...

Therefore Wold’s decomposition theorem demonstrates that for any stationary AR(p)
process, there is an equivalent MA() process.

Similar reasoning can be used to demonstrate that for any MA(q) process which
satisfies the invertibility condition, there exists an equivalent AR() process.

The MA(q) process is: yt = (1 + 1L + 2L2 + ... + qLq)ut

A more concise notation is: yt = (L)ut

 (L)–1yt = ut where (L)–1 = (1 + 1L + 2L2 + ... )

 yt is a stationary AR() process.

The invertibility condition for the MA(q) process is that the absolute value of the
roots of the characteristic equation (z)=0 must exceed one. (This is similar to the
condition for the stationarity of an AR(p) process, which we examined previously).