Chapter 3, Part II: Autoregressive Models

Another simple time series model is the f irst order autoregression , denoted by AR(1). The

series {xt } is AR(1) if it satisfies the iterative equation (called a dif f erence equation )

xt = αxt −1 + εt , (1)

where {εt } is a zero-mean white noise. We use the term autoregression since (1) is actually a linear

regression model for xt in terms of the explanatory variable xt −1. That is, xt is being modeled as a

regression on its own past. We will see that εt is uncorrelated with past values of the AR series xt .

Thus, εt represents the new contribution to xt , and we can think of {εt } as a series of random shocks ,

or innovations .

The definition (1) is implicit, since xt is defined in terms of its own past. It is useful to try to

write the AR (1) process explicitly in terms of present and past innovations. Substituting for xt −1 in (1)

gives

xt = α[αxt −2 + εt −1] + εt = εt + αεt −1 + α2xt −2 . (2)

Substituting for xt −2 in (2) gives

xt = εt + αεt −1 + α2[αxt −3 + εt −2] = εt + αεt −1 + α2εt −2 + α3xt −3 .

Continuing this process, we see that for any N ,

N −1
xt = Σ
j =0
α j εt −j + αN xt −N . (3)

The value of the parameter α strongly affects the behavior of the AR(1) process. Suppose that N

in equation (3) is large, so that the underlying series {xt } started a long time before our observations

x 1 , . . . , xn were formed. Then if we have −1 < α < 1 , the last term in (3) will be negligible and the

weight given to shocks which occurred a long time ago will also be extremely small. The resulting

series {xt } will be stationary. If, on the other hand, we have e α e > 1 , the last term in (3) will be large

in magnitude and the weights given to distant shocks will be much greater than those given to more

recent ones. The model is then said to be explosive , since the series mean and variance both explode as

t grows. The explosive model is not considered useful for economic time series. Finally, if α = 1 , we
 -2-

obtain a useful model called the random walk which is neither explosive nor stationary. It will be dis-

cussed later.

Here, we assume that −1 < α < 1 , and that N is very large, so that (3) is effectively

∞
xt = Σ
j =0
α j εt −j . (4)

Equation (4) is called an MA (∞) representation for the AR (1) process, since it expresses xt as a moving

average of an infinite number of present and past shocks. It follows from (4) that E [xt εt +1] = 0 . Thus

the future shock is uncorrelated with the present data. More generally, we have

E [xt εt +k ] = 0

for all t and all positive k . Thus, for example, the present shock is uncorrelated with all past series

values. Also, all future shocks are uncorrelated with all present and past series values.

Another consequence of (4) is that

var εt
var (xt ) = E [xt 2] = (1 + α2 + α4 + α6 + . . . ) var εt = hhhhhh .
1 − α2

The covariance between xt and xt −1 is

cov (xt , xt −1) = E [xt xt −1] = E [(αxt −1 + εt ) (xt −1)]

= α E [xt 2−1 ] + E [εt xt −1] = α var (xt ) .

Thus, the correlation between xt and xt −1 is

cov (xt , xt −1)

corr (xt , xt −1) = hhhhhhhhhhhh = α ,
var xt

so the degree of smoothness of {xt } is determined by α : {xt } is most smooth for α near +1 , very

unsmooth for α near −1 .

A similar argument shows that

corr (xt , xt −k ) = αk

for all k . Thus, there is always some correlation between present and future values, but this correlation
 -3-

dies down as we look further into the future. The implication is that future values are always foreca-

stable, but forecasting becomes more difficult (inaccurate) as the lead time increases.

Forecasting for AR models is achieved by the same strategy used earlier for MA models: obtain

an expression for the desired future value in terms of the {xt } and the {εt }, and then replace all unk-

nown terms by their optimal forecasts (which may be zero). Specifically, for one-step prediction in the

AR(1) model, we have

xn +1 = αxn + εn +1 .

The optimal forecast of εn +1 is zero since εn +1 is uncorrelated with all present and past values of {xt }.

Thus, the optimum forecast of xn +1 is

f n , 1 = αxn .

The one-step forecast error is

en , 1 = xn +1 − f n , 1 = εn +1 ,

so the sequence of one-step errors is a white noise. For two-step prediction, use the relation

xn +2 = αxn +1 + εn +2 ,

replace εn +2 by 0 , and replace xn +1 by its optimal forecast f n , 1 to obtain

f n , 2 = αf n , 1 = α2xn .

In general, the optimal h -step forecast is

f n , h = αh x n .

As the lead time h increases, the forecast approaches zero (i.e., the series mean).

A generalization of the AR(1) model is the p ’th order autoregression AR(p), generated by

xt = α1xt −1 + α2xt −2 + . . . + αp xt −p + εt .

The solution to this difference equation depends on the p starting values of {xt } and on the {εt } series.

The AR(p) series will be stationary if the largest root θ of the equation (in the complex variable z )
 -4-

z p = α1z p −1 + α2z p −2 + . . . + αp −1z + αp

satisfies e θ e < 1 . In this case, the correlation function corr (xt , xt −k ) (which is, because of stationarity, a

function of k alone) will approximately lie in the region ± A e θ e k , if k is not too small. This helps

determine the shape of the plot of the correlation function against k . (More about this later.)

To forecast an AR(p) model with known parameters, use the usual strategy. For one-step fore-

casts, use

xn +1 = (α1xn + . . . + αp xn −p +1) + εn +1 ,

and replace εn +1 by zero to obtain the forecast

f n , 1 = α1xn + . . . + αp xn −p +1 .

For two-step forecasts, use

xn +2 = α1xn +1 + (α2xn + . . . + αp xn −p +2) + εn +2 ,

replace εn +2 by zero and replace xn +1 by its optimal forecast f n , 1 to obtain

f n , 2 = α1 f n , 1 + (α2xn + . . . + αp xn −p +2) .

h-step forecasts are obtained similarly.

 Yen Per U.S. Dollar Mean Adjusted Data and
240 Jan 1985 to July 1992 Random Walk Prediction

80
Exchange Rate

60
200

40
20
160

0
-20
120

1985 1988 1991 1985 1988 1991

Adjusted Data and This Month’s vs Last

AR(1) Prediction, .5 x(t-1) Month’s Exchange Rate
••
•• •
240

•
••
80
60

•
This Month

•• •
200
40

•
••
20

••
160

•••••• •
••••••••
0

••
•••••••
•
• ••••
•
-20

••••
••
•
• •••
•
•••••••••
120

1985 1988 1991 120 160 200 240

Last Month