Week 2
Numerical Representation of Time Series Data
Definition: A very important type of time series is a stationary time series. A time series is said to be
strictly stationary if its properties are not affected by a change in the time origin. That is, if the joint
probability distribution of the observations. 𝑋 , 𝑋 ,𝑋 ,…,𝑋 is exactly the same as the joint
probability distribution of the observations 𝑋 ,𝑋 ,𝑋 ,…,𝑋 then the time series is
strictly stationary. When k= 0 the stationarity assumption means that the probability distribution 𝑋
is the same for all time periods and can be written as 𝑓(𝑥). The index k represents the lag property of
the time series, namely, the lead variables.
Stationary implies a type of statistical equilibrium or stability in the data. Consequently, the time series
has a constant mean defined in the usual way as
𝜇 = 𝐸(𝑦) = 𝑥𝑓(𝑥)𝑑𝑥
and constant variance defined as
𝜎 = 𝑉(𝑋) = (𝑥 − 𝜇) 𝑓(𝑥)𝑑𝑥
The sample mean and sample variance are used to estimate these parameters. If the observations in the
time series are 𝑥 , 𝑥 , 𝑥 , … , 𝑥 then the sample mean is
∑ 𝑥
𝜇 =𝑦=
𝑇
and the sample variance is
1
𝜎 =𝑠 = (𝑥 − 𝑥̅ )
𝑇
Note that the divisor in the sample variance formula is T rather than the more familiar T – 1. This is the
common convention in many time series applications, and because T is usually not small, there will be
little difference between using T instead of T – 1.
Autovariance and Autocorrelation Functions
Autocovariance
If a time series is stationary this means that the joint probability distribution of any two observations,
say 𝑋 𝑎𝑛𝑑 𝑋 is the same for any two time periods t and t + k that are separated by the same interval
1
k, which is an useful information about this joint distribution. and hence about the nature of the time
series, can be obtained by plotting a scatter diagram of all of the data pair. 𝑋 𝑎𝑛𝑑 𝑋 that are separated
by the same interval k. The interval k is called the lag.
The covariance between 𝑋 and its value at another time period, say, 𝑋 is called the autocovariance
at lag k, defined by
𝛾 = 𝐶𝑜𝑣(𝑋 , 𝑋 ) = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)]
Or
For sample autocovariance
1
𝛾 = (𝑥 − 𝑥̅ )(𝑥 − 𝑥̅ ), 𝑘 = 0,1,2, … , 𝑇
𝑇
The collection of the values of 𝛾 , k = 0, 1, 2, ... is called the autocovariance function. Note that the
autocovariance at lag k = 0 is just the variance of the time series.
Ex: compute autocovariance for k=0,1 and 2.
Table: The number of sales of a product at a supermarket in each day
Day 1 2 3 4 5 6 7 8 9 10
(t)
Sales 25 34 12 45 67 12 90 36 37 21
(𝑋)
𝑋 25 34 12 45 67 12 90 36 37 21
For k=0, autocovariance is actually called the variance and is denoted by
𝛾 = 𝛾 = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)] = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)] = 𝐸[(𝑋 − 𝜇) ]
1
= (𝑥 − 𝑥̅ )
𝑇
First average is computed by
∑ 𝑥 25 + 34 + 12 + ⋯ + 37 + 21
𝜇 =𝑦= = = 37.9
𝑛 10
Then variance is computed by
1
𝛾 =𝜎 = (𝑥 − 𝑥̅ )
𝑇
2
𝑥 (𝑥 − 𝑥̅ )
25 (25 − 37.9)
34 (34 − 37.9)
12 (12 − 37.9)
45 (45 − 37.9)
67 (67 − 37.9)
12 (12 − 37.9)
90 (90 − 37.9)
36 (36 − 37.9)
37 (37 − 37.9)
21 (21 − 37.9)
Total=379 and mean=37.9 Total=5424.9
1 5424.9
𝛾 =𝜎 = (𝑥 − 𝑥̅ ) = = 542.49
𝑇 10
For k=1,
𝛾 = 𝛾 = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)] = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)]
1
𝛾 = (𝑥 − 𝑥̅ )(𝑥 − 𝑥̅ )
𝑇
Where t=1,2,…,10
𝑥 𝑥 𝑥 − 𝑥̅ 𝑥 − 𝑥̅ (𝑥 − 𝑥̅ )(𝑥 − 𝑥̅ )
25 34 (25-37.9) (34-37.9)
50,31
34 12 (34-37.9) (12-37.9)
101,01
12 45 (12-37.9) (45-37.9)
-183,89
45 67 (45-37.9) (67-37.9)
206,61
67 12 (67-37.9) (12-37.9)
-753,69
12 90 (12-37.9) (90-37.9)
-1349,39
90 36 (90-37.9) (36-37.9)
-98,99
36 37 (36-37.9) (37-37.9)
1,71
37 21 (37-37.9) (21-37.9)
15,21
21 none (21-37.9) none Total=-2011.11
−2011.11
𝛾 = = −201.11
10
3
For k=2,
𝛾 = 𝛾 = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)] = 𝐸[(𝑋 − 𝜇)(𝑋 − 𝜇)]
1
𝛾 = (𝑥 − 𝑥̅ )(𝑥 − 𝑥̅ )
𝑇
Where t=1,2,…,10
𝑥 𝑥 𝑥 − 𝑥̅ 𝑥 − 𝑥̅ (𝑥 − 𝑥̅ )(𝑥 − 𝑥̅ )
25 12 (25-37.9) (12-37.9)
-154,8
34 45 (34-37.9) (45-37.9)
-175,5
12 67 (12-37.9) (67-37.9)
-1735,3
45 12 (45-37.9) (12-37.9)
85,2
67 90 (67-37.9) (90-37.9)
2619
12 36 (12-37.9) (36-37.9)
-932,4
90 37 (90-37.9) (37-37.9)
1927,7
36 21 (36-37.9) (21-37.9)
-39,9
37 none none none
Total=1594
21 none none none
1594
𝛾 = = 159.4
10
Autocorrelation
Definition: The collection of the values of 𝜌 , 𝑘 = 01,2,3, … is called the autocorrelation function (ACF).
Note that by definition 𝜌 = 1. Also. the ACF is independent of the scale of measurement of the time
series, so it is a dimensionless quantity. Furthermore, 𝜌 = 𝜌 that is, the autocorrelation function is
symmetric around zero. so it is only necessary to compute the positive (or negative) half.
If a time series has a finite mean and autocovariance function it is said to be second-order stationary
(or weakly stationary of order 2). If, in addition. the joint probability distribution of the observations
at all times is multivariate normal. then that would be sufficient to result in a time series that is strictly
stationary.
The formula for autocorrelation is defined as follows:
𝐶𝑜𝑣(𝑋 , 𝑋 ) 𝛾
𝜌 = =
𝑉(𝑋 ) 𝛾
For sample autocorrelation,
4
∑ (𝑥 − 𝑥̅ )(𝑥 − 𝑥̅ ) 𝛾
𝜌 = =
∑ (𝑥 − 𝑥̅ ) 𝛾
The variance of the sample autocorrelation is defined by
1
𝑉(𝜌 ) ≅
𝑇
Remark: A good general rule of thumb is that at least 50 observations are required to give a reliable
estimate of the ACF, and the individual sample autocorrelations should be calculated up to lag K. where
K is about T /4.
Use of Data Transformation and Adjustment
Data transformations are useful in many aspects of statistical work, often for stabilizing the variance
of the data. Non-constant variance is quite common in time series data. A very popular type of data
transformation to deal with non-constant variance is the power family of transformations given by
[(𝑥 + 1) − 1]/𝜆, 𝑥 ≥ 0, 𝜆 ≠ 0
⎧
⎪ log(𝑥 + 1), 𝑥 ≥ 0, 𝜆 = 0
𝑓 (𝑥 , 𝜆) = (−𝑥 + 1) −1
⎨− , 𝑥 < 0, 𝜆 ≠ 2
⎪ 2−𝜆
⎩ − log(−𝑥 + 1) , 𝑥 < 0, 𝜆 ≠ 2
If λ=1 there is no transformation. Typical values of λ used with time series data are λ = 0.5 (a square
root transformation). λ = 0 (the log transformation). λ = -0.5 (reciprocal square root transformation),
and 𝜆 = −1 (inverse transformation).
Ex: Compute transformations for λ=1, 0.5, 0, -0.5 and -1
Table: The number of sales of a product at a supermarket in each day
Day 1 2 3 4 5 6 7 8 9 10
(t)
Sales 25 34 12 45 67 12 90 36 37 21
(𝑋)
𝑋 25 34 12 45 67 12 90 36 37 21
5
for λ=1
𝑋 (𝑥 + 1) − 1
1
25
25
34
34
12
12
45
45
67
67
12
12
90
90
36
36
37
37
21
21
for λ=0.5
𝑋 (𝑥 + 1) . − 1
0.5
25
8,19
34
9,83
12
5,21
45
11,56
67
14,49
12
5,21
90
17,07
36
10,16
37
10,32
21
7,38
for λ=0
𝑋 log(𝑥 + 1)
25
1.41
34
1.54
12
1.11
45
1.66
6
67
1.83
12
1.11
90
1.95
36
1.56
37
1.57
21
1.34
for λ=-0.5
𝑋 (𝑥 + 1) . − 1
−0.5
25
1,60
34
1,66
12
1,44
45
1,7
67
1,75
12
1,44
90
1,79
36
1,67
37
1,67
21
1,57
for λ=-1
𝑋 (𝑥 + 1) − 1
−1
25
0,96
34
0,97
12
0,92
45
0,97
67
0,98
12
0,92
90
0,98
36
0,97
37
0,97
21
0,95