1

Quantitative Methods

CHAPTER 4 Introduction to Linear Regression

Linear Regression a.k.a. Linear Least Squares assumes a linear relationship between the dependent and the
independent variables. Linear Regression computes a straight line that best fits the observations; it chooses
values for the intercept 'b0' and slope 'b1', that minimize the sum of the squared vertical distances between
the observations and the regression line.

ACTUAL Y =b +b X +ε = 1, 2, 3 ... n
0 1

Whereas,
th
Y : i observation of the dependent variable Y. The dependent variable is also referred to as the
'Explained Variable', 'Endogenous Variable' or 'Predicted Variable'.
th
X : i observation of the independent variable X. The independent variable is also referred to as
the 'Explanatory Variable', 'Exogenous Variable' or 'Predicting Variable'.
b : Regression Intercept. It is the value of dependent variable, if value of the independent
0
Regression variable is '0'. The intercept term in this regression is called the stock's Ex-post Alpha. It is a
Coefficients measure of excess risk-adjusted returns a.k.a. Jensen's p = R p - [R f + (R m- R f )]
b : Regression Slope Coefficient. It is the change in the dependent variable for a 1-unit change
1
in the independent variable. The slope coefficient in a regression like this is called Stock's
Beta and it measures the relative amount of systematic risk in returns.
th
ε : Residual for the i observation, also referred to as the 'Disturbance Term', 'Error Term' or
'Unexplained Deviation'. It represents the portion of the dependent variable that cannot be
explained by the independent variable.

^ ^ ^
PREDICTED Y =b +b X = 1, 2, 3 ... n
0 1

Whereas,
^
Y : Estimated (Fitted Parameters) value of Y given X
^ ^ ^
b : Estimated Intercept. b0 = Y - b1 X ; The intercept equation highlights the fact that the
I

0
regression line passes through a point with coordinates equal to the mean of the
independent and dependent variables.
^ ^ 2
b : Estimated Slope Coefficient. b1 = Cov XY / o X
I

The sum of the squared vertical distances between the estimated and actual Y-values is referred to as the
'Sum of Squared Errors' (SSE). Thus, regression line is the line that minimizes the SSE. This explains why
simple linear regression is frequently referred to as 'Ordinary Least Squares' (OLS) regression and the values
^
estimated by the estimated regression equation Y is called Least Squares estimates. Here are some
assumptions below:
 2

Assumption 1: A linear relationship exists between the dependent and the independent variable. This
requirement means that b and b are raised to the first power only and that neither b nor b is multiplied or
divided by another regression parameter (as in b /b ). The requirement doesn't exclude X from being raised
to a power other than 1. If the relationship between the independent and dependent variables is non-linear
in the parameters, then estimating that relation with a linear regression model will produce invalid results.

b x
Y =b e1 +ε
0 Non-Linear (Not Allowed) x
2
Y =b +b X +ε Linear (Allowed)
0 1

Assumption 2: The independent variable X is not random. If the independent variable is random, we can still
rely on the results of regression models given the crucial assumption that the error term is uncorrelated with
the independent variable.

Assumption 3: The expected value of the error term is 0; E(ε) = 0.

2 2
Assumption 4: The variance of the error term is the same for all observations; E(ε ) = o ε . It is also known as

I
the 'Homoscedasticity' assumption i.e. violation.

Assumption 5: The error term is independently distributed, that the error for one observation is not
correlated with that of another observation. Related to 'Serial Correlation' i.e. violation.

Assumption 6: The error term is normally distributed. If the regression errors are not normally distributed,
we can still use regression analysis. Econometricians who dispense with the normality assumption use Chi-
Square tests of hypothesis rather than F-tests.

An unbiased forecast can be expressed as E(Actual Change - Predicted Change) = 0. If the forecasts are
unbiased, the intercept 'b 0' should be 0 and the slope 'b 1' should be 1, then the error term
[Actual Change - b 0- b1 (Predicted Change)] will have an expected value of 0, as required by Assumption 3 of
the linear regression model.

lllll ANOVA
Analysis of Variance (ANOVA) is a statistical procedure for analyzing the total variability of the
dependent variable. In regression analysis, we use ANOVA to determine the usefulness of the
independent variable or variables in explaining variation in the dependent variable.

1. Total Sum of Squares (TSS): measures the total variation in the dependent variable.

n 2
TSS = Σ (Y - Y)
I

2. Regression Sum of Squares (RSS): measures the explained variation in the dependent variable.
n ^ 2
RSS = Σ (Y - Y)
I

3. Sum of Squared Errors (SSE): measures the unexplained variation in the dependent variable. It is also
known as the Sum of Squared Residuals or the Residual Sum of Squares.
n ^ 2
SSE = Σ (Y - Y )
1

Total Variation = Explained Variation + Unexplained Variation

(TSS) (RSS) (SSE)
 3

Fig 1: Components of
Total Variation
Y
Y .
^
^ ^ ^ (Y - Y ) = SSE
Y = b0 + b1 X

. . (Y - Y) = TSS

I
.
^

..
(Y - Y) = RSS

I
.
. . .
Y
I

. . . . Actual
^
b0
X Predicted

I
Fig 2: ANOVA Table
df ss ss / df

Degrees of Sum of
Source of Variation Mean Sum of Squares
Freedom Squares

Regression k RSS MSR = RSS

(Explained) k

Error n-k-1 SSE MSE = SSE

(Unexplained) n-k-1

TOTAL n-1 TSS

n: no. of observations
k: no. of independent variables

lllll Variations Explained

lllll T-Statistic
T-value measures the size of the difference relative to the variation in your sample data. When
T-Stat is very small it indicates that none of the autocorrelations are significantly different than 0.

^
t = b1 - b1 df = n - k - 1
s ^b
1

H0 : What you don't believe in (wants to reject)

Fig 3: Null and Alternative Hypothesis H : Something you believe in (wants to accept)
1

a. H0 : μ = 0 Ha : μ ≠ 0 b. H0 : μ ≤ 0 Ha : μ > 0 c. H0 : μ ≥ 0 Ha : μ < 0

accept accept accept

IIlll lllII lllII IIlll
reject reject reject reject
 4

ρ-value: is the smallest level of significance for which the null hypothesis can be rejected.

ρ< : Reject H
0
ρ> : Accept H0

Example 1:
The estimated slope coefficient of the ABC plc. is 0.64 with standard error equal to 0.26. Assuming that the
sample has 36 observations, determine if the estimated slope coefficient is significantly different than 0 at a
5% level of significance.
Reject H 0
H0 : b 1 = 0 Ha : b1 ≠ 0

^
T-test = b1 - b1 = 0.64 - 0 = 2.46 IIlll lllII.
s ^b 0.26 2.03 -2.03 -2.46
1

The critical two-tailed t-values are ± 2.03 (df = n - k -1 = 36 - 1 - 1 = 34). Because t > t c i.e. 2.46 > 2.03, we
reject the null hypothesis and conclude the slope is different from 0.

If none of the independent variables in a regression model helps explain the dependent variable,
the slope coefficients should all equal '0'. In a multiple regression however, we cannot test the null
hypothesis that all slope coefficients equal '0' based on T-tests that each individual slope coefficient
equals '0', because the individual tests don't account for the effects of interactions among the
independent variables. To test the null hypothesis that all of the slope coefficients in the multiple
regression model are jointly equal to '0' (H 0 : b1 = b2 = ... b k = 0) against the alternative hypothesis
that at least one slope coefficient is not equal to '0'; we must use an F-test. The F-test is viewed as a
test of the regression's overall significant.

lllll F-Statistic
F-test assesses how well the set of independent variables as a group explains the variation in the
dependent variable. That is, the F-Stat is used to test whether at least one of the independent
variables explains a significant portion of the variation of the dependent variable. This is always a
one-tailed test, despite the fact that it looks like it should be a two-tailed test because there is an
equal sign in the null hypothesis. If the regression model does a good job of explaining variation in
the dependent variable, then this ratio should be high.

F = MSR = RSS / k df =k
denominator
MSE = SSE / n - k - 1 df numerator = n - k - 1

For simple linear regression, F = t2b is

1
H0 : b1 = 0 Ha : b1 ≠ 0

For multiple regression, F = t2b is

1
H0 : b1 = b2 = b3 = b4 = 0 Ha : At least one b ≠ 0

Example 2:
An analyst runs a regression of monthly value stock returns on five independent variables over 60 months.
The TSS is 460 and the SSE is 170. Test the null hypothesis at the 5% significance level that all five
independent variables are equal to 0.
 5

H0 : b1 = b2 = b3 = b4 = b5 = 0 Ha : At least one b ≠ 0 Reject H

0

.
F-test = MSR = 58 = 18.41
llIII
MSE 3.15 0
2.4 18.41

The critical F-value for 5 and 54 degrees of freedom at a 5% significance level is significantly 2.4. Therefore,
we can reject the null hypothesis and conclude that at least one of the five independent variables is
significantly different than 0.

lllll Standard Error of Estimate

The Standard Error of Estimate (SEE) measures the degree of variability of the actual Y-values
relative to the estimates Y-values from a regression equation. The SEE gauges the 'fit' of the
regression line. The smaller the standard error, the better fit. The SEE is the standard deviation of
the error terms in the regression. As such, SEE is also referred to as the Standard Error of the
Residual or Standard Error of the Regression. In regressions, the relationship between the
independent and dependent variables can be strong or weak; relative to total variability, SEE will be
low if the relationship is very strong and high if the relationship is weak. Any violation of an
assumption that affects the error term will ultimately affect the coefficient standard error, cause
coefficient standard error is calculated using SEE.
n 2 1/2
SEE = MSE = SSE = Σ (ε^ )
n-k-1 n-k-1

2 2
lllll Coefficient of Determination: R and Adjusted: R a

2
1. R : It is defined as the percentage of the total variation in the dependent variable explained by
2
the independent variable. R explains the correlation between predicted and actual values of
2
dependent variable. For example, an R of 0.63 indicates that the variation of the independent
variable explains 63% of the variation in the dependent variable.

2 2
R = TSS - SSE = RSS = 1 - SSE Higher the better; k R
TSS TSS TSS

Regression output often includes multiple R (correlation coefficient), which is the correlation
between actual values of Y and forecasted values of Y. (Multiple R is the square root of R2 ).

2
For simple linear regression (i.e. one independent variable), the coefficient of determination R
2
may be computed by simply squaring the correlation coefficient 'r' i.e. R = r 2. This approach is
not appropriate when more than one independent variable is used in the regression.

2 2
For multiple regression, R by itself may not be reliable, this is because R almost always increases
as variables are added to the model, even if the marginal contribution of the new variables is not
statistically significant (if we add regression variables to the model, the amount of unexplained
variation will decrease and RSS will increase, if the new independent variable explains any of the
unexplained variation in the model. Such a reduction occurs when the new independent variable
is even slightly correlated with the dependent variable and is not a linear combination of other
2
independent variables in the regression). Consequently, a relatively high R may reflect the impact
of a large set of independent variables rather than how well the set explains the dependent
variable. This problem is often referred to as overestimating the regression.

2 2
2. Ra : Some financial analysts use an alternative measure of goodness of fit called Ra . To overcome
 6

the problem of overestimating the impact of additional variables on the explanatory power of a
2
regression model, many researchers recommend adjusting R for the number of independent
variables 'k'.

2 2
Ra = 1 - n - 1 x (1 - R )
n-k-1

2
As k overly R a ; should not add independent variables beyond
% 2
k*. When a new independent variable is added, Ra can decrease if
2
R adding that variable results in only a small increase in R 2 . When
2 2 2
k ≥ 1, then R ≥ R a. In fact R a can be negative (effectively consider
its value 0), although R is always non-negative. In addition, R2a may
2

2
be less than 0, if R2 is low enough. Furthermore, we must be aware
Ra 2
that a high R a doesn't necessarily indicate that the regression is
well specified in the sense of including the correct set of variables.
0 k One reason for caution is that a high R 2a may reflect peculiarities
k*
i.e. weirdness of the dataset used to estimate the regression.

Goodness of 'fit' (SEE) and Coefficient of Determination are different values for the same concept.
The Coefficient of Variation is not directly part of regression model.

Example 3:
Part a: An analyst runs a regression of monthly value stock returns on five independent variables over 60
2 2
months. The TSS is 460 and the SSE is 170. Calculate the R and Ra .
2
Part b: Suppose the analyst now adds four more independent variables to the regression and the R
increases to 65%. Identify which model the analyst should most likely prefer.

2 2
a. R = 460 - 170 = 0.63 or 63% Ra = 1 - 60 - 1 x (1 - 0.63) = 59.6%
460 60 - 5 - 1

2 2
b. R = 65% (given) Ra = 1 - 60 - 1 x (1 - 0.65) = 58.7%
60 - 9 - 1

2 2
With nine independent variables, even though the R has increased from 63% to 65%, the R a has decreased
2
from 59.6% to 58.7%. The analyst would prefer the first model because the Ra is higher and the model has
five independent variables as opposed to nine.

lllll Confidence Intervals

Estimated Regression Coefficient ± (t c ) . (Coefficient Standard Error)

SEE Standard Error CI widens tc

SEE Standard Error CI tightens tc

Standard Error of Slope Coefficient

^
b1± (t c x S ^b )
1

^
Y ± (t c x S f )
Standard Error of Forecast
 7

Whereas,
2 2 2 2
S f : SEE 1 + 1 + (X - X) ; S x is variance of the independent variable.

I
2
n (n - 1) Sx
2
SY = TSS is variance dependent variable.
n-1

Rule: H0 : b1 = 0 and Ha : b1 ≠ 0. If the confidence interval (CI) at the desired level of significance doesn't
include 0, the H0 is rejected and the coefficient is said to be statistically different from 0.

Example 4:
Coldplay forecasts the excess return on the S&P 500 for June 2017 to be 5% and the 95% CI for the predicted
value of the excess return on VIGRX for June 2017 to be 3.9% to 7.7% (b 0 : 0.0023 and b 1 : 1.1163). The
standard error of the forecast is closest to?

^ ^ ^
Y=b +b X
^ 0 1
Y = 0.0023 + 1.1163 (0.05) = 0.058115

0.039 = 0.058115 + 2.03 (S f ) to 0.058115 - 2.03 (S f ) = 0.077 S f = 0.0093

NOTES

1. The distinction between SSE and SEE: SSE is the sum of the squared residuals, while SEE is the standard
deviation of the residuals.

2. Limitations of regression analysis:

(a) Linear relationships can change overtime. This is referred to as 'Parameter Instability'.
(b) Public knowledge of regression relationship may negate their future usefulness.
(c) If the assumptions underlying regression analysis don't hold, the interpretation and tests of hypothesis
may not be valid.

^ ^
3. Prediction must be based on the parameters' estimated values (b0 and b1 ) in relation to the hypothesized
population values.

4. Cov = Σ (X - X) (Y - Y) Past Sample

I

XY
n-1

Cov XY= Σ P (X - X) (Y - Y) Future Sample

I

The covariance between two random variables is a statistical measure of the degree to which the two
variables move together. The covariance captures the linear relationship between two variables.

5. rXY= Cov XY
ox oY
I

The correlation coefficient is a measure of the strength of the linear relationship between two variables.

6. Regression analysis uses two principle types of data:

(a) Cross-Sectional: data involve many observations on X and Y for the same time period.
(b) Time Series: data use many observations from different time periods for the same company or country.
A mix of time series and cross sectional data is also known as 'Panel Data'.
 8

CHAPTER 5 Multiple Regression

Multiple Regression is regression analysis with more than one independent variable. It is used to quantify the
influence of two or more independent variables on a dependent variable.

ACTUAL Y = b0 + b1 X 1 + b2 X2 + ..... + bk X k + ε = 1, 2, 3 ... k

^ ^ ^ ^ ^
PREDICTED Y = b0 + b1 X 1 + b2 X2 + ..... + bk X k

Assumptions for multiple regression are almost exactly the same as those for the single variable linear
regression model, except assumption 2 and 3. Here are some changes under multiple regression model:

Assumption 2: The independent variables (X 1 , X2 , ... , X k) are not random. Also, no exact linear elation exists
between two or more of the independent variables. If this part of assumption 2 is violated, then we cannot
compute linear regression, may encounter problems if two or more of the independent variables or
combinations thereof are highly correlated. Such a high correlation is known as 'Multicollinearity' i.e. violation

Assumption 3: The expected value of the error term, conditional on the independent variables is 0.
E ( ε| X , X , ... , X ) = 0.
1 2 k

lllll Qualitative Factors

Qualitative Independent Variable i.e. Dummy Variables capture the effect of binary independent
variables. Whereas, Qualitative Dependent Variables (Categorical Dependent Variables) require
methods other than OLS i.e. Probit, Logit or Discriminant Analysis.

lllll Dummy Variables

There are occasions when the independent variable is binary in nature, it is either 'on' or 'off'. One
type of qualitative variable is called a Dummy Variable, takes on a value of 1 if a particular condition
is true and 0 if that condition is false. Not all qualitative variables are simply dummy variables. For
example, in a trinomial choice model i.e. a model with three choices; a qualitative variable might
have value of 0, 1 or 2. Whenever we want to distinguish between n classes, we must use n -1
dummy variables to avoid 'Multicollinearity'. Otherwise, the regression assumption of no exact
linear relationship between independent variables would be violated.

Consider the following regression equation for explaining quarterly EPS in terms of the quarter of
their occurrence:

EPS = b + b Q + b Q + b Q + ε df = n - 1
t 0 1 1t 2 2t 3 3t t

Whereas,
EPS t : Quarterly observation of Earning Per Share.
Q1t : 1 if period t is the first quarter, Q1t : 0 otherwise.
Q2t : 1 if period t is the second quarter, Q2t : 0 otherwise.
Q3t : 1 if period t is the third quarter, Q 3t : 0 otherwise.
b0 : Average value of EPS for the fourth quarter.
b1 , b2 , b3 : Estimate the difference in EPS on average between the respective quarter
(i.e. quarter 1, 2 or 3) and the omitted quarter (the fourth quarter in this case).
Think of the omitted class as the reference point.
 9

Example 1:
Some developing nations are hesitant to open their equity markets to foreign investments because they fear
that rapid inflows and outflows of foreign funds will increase volatility. You want to test whether the volatility
of returns of stocks traded on the Bombay Stock Exchange (BSE) increased after July 1993, when foreign
institutional investors were first allowed to invest in India. Your dependent variable is a measure of return
volatility of stocks traded on the BSE; your independent variable is a dummy variable that is coded 1 if foreign
investment was allowed during the month and 0 otherwise.

Coefficient Standard Error T-test

Intercept 0.0133 0.002 6.5351

Dummy -0.0075 -0.0027 -2.7604
n = 95

a. State null and alternative hypothesis for the slope coefficient of the dummy variables that are consistent
with testing your stated belief about the effect of opening the equity markets on stock return volatility.

H 0: b 1≥ 0 Ha : b 1 < 0

b. Determine whether you can reject the null hypothesis at the 5% significance level in a one-sided test of
significance. Reject H 0

df = n - k - 1 = 95 - 1 - 1 = 93 . IIII
-2.7604 -1.661
We reject the null hypothesis because the dummy variable takes on a value of 1 when foreign investment is
allowed, we can conclude that the volatility was lower with foreign investment.

c. According to the estimated regression equation, what is the level of return volatility before and after the
market-opening event?

Before: Y = 0.0133 - 0.0075 (0) = 0.0133

After: Y = 0.0133 - 0.0075 (1) = 0.0058

lllll Probit Model lllll Logit Model

0 0

Y = f (b0 + b1 X 1+ b 2 X 2 ) + ε Y = f (b0 + b 1 X1 + b 2X2 ) + ε

= f (x) = Φ (1 + e-x) = f (x) = 1

(1 + e-x)
Event Happens
-1
Y = Φ (P) Y = In P
(1 - P)
Event Doesn't Happen
 10

Probit Model i.e. Probit regression, which is Logit Model i.e. Logistic regression is based
based on the normal distribution, estimates on the logistic distribution also called a log
the probability that Y = 1 (a condition is adds ratio. Logistic regression is widely
fulfilled) given the values of the used in machine learning, where the
independent variables. objective is classification. Logistic
regression assumes a logistic distribution
for the error term; this distribution is
similar in shape to the normal distribution
but has heavier tails.

In most cases it makes no difference which one is used (probit or logit). Both functions increase
relatively quickly at x = 0 and relatively slowly at extreme values of x. Both functions lie between 0 and 1.
In econometrics, probit and logit models are traditionally viewed as models suitable when the
dependent variable is not fully observed.

lllll Discriminant Model

Discriminant Analysis yields a linear function similar to a regression equation, which can then be
used to create an overall score. Based on the score, an observation can be classified into bankrupt
or not bankrupt category. Discriminant model makes use of financial ratios as the independent
variables to predict the qualitative dependent variable bankruptcy.

Examine the individual coefficients using T-tests, determine the validity of the model with the F-test, the R2
and look out for Heteroskedasticity, Serial Correlation and Multicollinearity.

Fig 3: Violations of Assumptions

A. Heteroskedasticity B. Serial Correlation C. Multicollinearity

Heteroskedasticity occurs when Serial Correlation a.k.a. When one of the independent
the variance of the residuals is Autocorrelation refers to the variables is an exact linear
not the same across all situation in which the residual combination of other
observations in the sample. terms or regression errors are independent variables, it
This happens when there are correlated with one another. It becomes mechanically
sub-samples that are more is a relatively common problem impossible to estimate the
spread out than the rest of the with time series data. Any effect regression. That case, known as
sample. of serial correlation appears 'Perfect Collinearity' is much
only in the regression less of a practical concern than
(a) (b) coefficient standard errors. If multicollinearity.
Homoskedastic Heteroskedastic one of the independent Multicollinearity occurs when
2
[Var. (ε ) = o ] [With errors] variables is a lagged value of two or more independent
I

High residual

x . x . value the dependent variable, then variables (or combinations of

.... . .... . . .
.. . .... .. . . serial correlation in the error independent variables) are
.. ... .. . . .. . .. . .. .. .
.. .. term will cause all the highly (but not perfectly)
. . .. . ....... ... Low residual
.. . value parameter estimates from correlated with each other.
Y Y
linear regression to be Multicollinearity if a serious
No relationship On average,
between, value of regression
inconsistent and they will not practical concern because
independent residuals grow be valid estimates of the true approximate linear
variables and larger as the size parameter. relationships among financial
regression of the independent variables are common.
residuals variable increases
 11

When one of the independent

(i) (ii) (a) (a) variables is an exact linear
Unconditional Conditional Positive Sc. Negative Sc. combination of other
[Var. (ε ) ≠ o 2 ] [Var. (ε /x) ≠ independent variables, it
I
x x
const.] . . +.
+
Unconditional +
. .+ +. becomes mechanically
. +.
+
Time
+
Time
Heteroskedasticity The type of . .
-
... - .- .- impossible to estimate the
- -
occurs when the heteroskedasticity
regression. That case, known as
-

heteroskedasticity that causes the Y Y

'Perfect Collinearity' is much
is not related to most problems for Positive Sc. is serial Negative Sc. occurs
the level of statistical inference correlation in when a positive
less of a practical concern than
independent is Conditional which a positive error in one period multicollinearity.
variables; which Heteroskedasticity. error for one increases the Multicollinearity occurs when
means that it It is the error observation probability of two or more independent
doesn't variance that is increases the observing a
variables (or combinations of
systematically correlated with chance of positive negative error in
or with changes (conditional on) independent variables) are
error for another the next period and
in the value of the the values of the observation. It also vice versa. highly (but not perfectly)
independent independent means that a correlated with each other.
variables. While variables in the negative error for Multicollinearity if a serious
this is a violation of regression. It one observation practical concern because
the equal variance identifies non- increases the
assumption, it constant volatility
approximate linear
chance of a
usually causes no related to prior negative error for relationships among financial
major problems period's volatility. another variables are common.
with the Conditional observation. First
regression. It heteroskedasticity Order Sc. means
refers to general is not predictable the sign of the
structural changes by nature. error term tends to
in volatility that are persist from one
not related to prior period to the next.
period's volatility,
Unconditional
heteroskedasticity
is predictable and
can relate to
variables that are
cyclical by nature.

^ ^
T-test = b = Reliable Positive Sc. T-test = b = Unreliable
(unreliable) S ^ ^ (unreliable) S ^
b Unreliable T-test = b = Reliable b Unreliable
(unreliable) S^ Unreliable
b
^ ^
^
^

F-test = MSR F-test = MSR

(unreliable)MSE MSE

Negative Sc.
OLS standard errors need not Type I Error: Reject H 0
be underestimates of actual when it is true.
standard errors, if negative Sc. Type II Error: Fail to Reject H 0
when it is false.
is present in the regression.

1. Examining scatter plots of 1. Durbin-Watson: is used to 1. The most common way to

the residuals. detect the presence of serial detect multicollinearity is the
correlation. situation where T-tests
 12

2. Breusch-Pagan Chi-Square (a) Small Sample indicate that none of the

Test: which calls for the individual coefficient is
^ ^ 2
regression of the squared DW = Σ (ε t - ε t-1 ) df = k significantly different than 0,
residuals on the independent Σ (ε^ t2 ) while the F-test is statistically
variables. If conditional significant and the R2 is high.
^ ^ ^
heteroskedasticity is present, = [Var (ε t ) - 2 Cov (εt, ε t-1 ) This suggests that the
^ ^
the independent variables +Var (ε t-1 )] / Var (ε t ) variables together explain
will significantly contribute to much of the variation in the
If the variance of the error is constant
the explanation of the dependent variable, but the
through time, then we expect
squared residuals. Var (ε^t ) = o^ ε for all 't', where we use o^ 2ε
2 individual independent

I
to represent the estimate of the constant variables don't. The only way
2 2 error variance. If in addition, the errors
BP X Test = n x R df = k this can happen is when the
residuals are also not serially correlated, then we
(one-tailed test)
expect Cov (ε^ t , ε^t-1) = 0. In that case DW is independent variables are
2
R from a second approximately equal to: highly correlated with each
regression of the
squared residuals from ^ 2 ^ 2 other. If the absolute value of
o ε- 0 + o ε =2
I

I
the first regression on 2 the sample correlation
the independent o^ ε
I
between any two
variables.
Therefore we can test the null hypothesis independent variables in the
H0 : No Conditional Heteroskedasticity that the errors are not serially correlated
H1 : Conditional Heteroskedasticity by testing whether the DW test differs
regression is greater than
significantly from 2 0.7, multicollinearity is a
Conditional potential problem. However,
Heteroskedasticity is only a (a) Large Sample this only works if there are
2
problem if the R and the BP exactly two independent
Test statistic are too large. DW ≈ 2 (1 - r) df = k variables. If there are more
than two independent
'r' correlation coefficient
between residuals from variables, while individual
one period and those variables may not be highly
from the previous period.
correlated, linear
H0 : No Positive Serial Correlation combinations might lead to
H1 : Positive Serial Correlation
multicollinearity (conflicting
T-test and F-test statistics).
Reject H0
x
Accept H0 High pairwise correlations
Positive Sc. Negative Sc.
Inconclusive among the independent
0 0 variables are not a necessary
dL dU
condition for multicollinearity
(Lower) (Upper)
and low pairwise correlations
No Autocorrelation (r = 0)
don't mean that
DW ≈ 2 (1 - 0) = 2 (DW = 2) multicollinearity is not a
problem.
Positive Serial Correlation (r > 0)
DW ≈ 2 (1 - 1) = 0 (DW < 2)

Negative Serial Correlation (r < 0)

DW ≈ 2 (1 - (- 1) = 4 (DW > 2)

Corrections: Corrections: Corrections:

1. Calculate 'Robust Standard 1. Adjust the coefficient 1. The most common method
Errors' i.e. 'White Corrected standard errors, using the to correct for
Standard Errors' or 'Hansen Method' i.e. 'Newey multicollinearity is to omit
'Heteroskedasticity - & West Method'. These adjust one or more of the
Consistent Standard Errors'. standard errors upwards correlated independent
These robust standard errors using the Hansen method, variables.
 13

are then used to recalculate which are then used in

the T-statistic using the hypothesis testing of the
original regression regression coefficients. Only
coefficients, if there is an use the 'Hansen Method' if
evidence of serial correlation is a
heteroskedasticity. problem. The 'White
Corrected Standard Errors'
2. Use 'Generalized Least are preferred, if only
Squares', which attempts to heteroskedasticity is a
eliminate the problem. If both conditions
heteroskedasticity by are present. use the 'Hansen
modifying the original Method'.
equation.
2. It is to explicitly incorporate
the time series nature of the
data e.g. include a seasonal
term (this can be tricky).

a. Heteroskedasticity: variance of error term is constant.

b. Serial Correlation: errors are not serially correlated.
c. Multicollinearity: no exact linear relationship among X's.

Example 2:
A variable is regressed against three other variables X, Y, Z. Which of the following would not be an indication
of multicollinearity? X is closely related to:
A. 3y + 2z
B. 9y - 4z + 3
2
C. y

lllll Model Specification

Principles of Model Specification:

(a) The model should be grounded in cogent economic reasoning.
(b) The functional form chosen for the variables in the regression should be appropriate given the
nature of the variables.
(c) The model should be parsimonious i.e. accomplishing a lot with little.
(d) The model should be examined for violations of regression assumptions before being accepted.
(e) The model should be tested and be found useful out of sample before being accepted.

There are three broad categories of 'Model Misspecification' or ways in which the regression model can be
specified incorrectly, each with several subcategories:

1. Misspecification of Functional Form

- Important variables are omitted: If the omitted independent variable (X 2) is correlated with the remaining
independent variable (X 1 ), then the error term in the model will be correlated with (X 1) and the estimated
values of the regression coefficients a 0 and a 1 would be biased and inconsistent, so will the standard error
of those coefficients.

Y = b 0+ b 1 X 1+ b 2X 2+ ε

Y = a 0 + a1 X 1+ ε b 0≠ a 0
 14

- Variables should be transformed: Sometimes analysts fail to account for curvature or non-linearity in the
relationship between the dependent variable and one or more of the independent variables, instead
specifying a linear relation among variables. We should also consider whether economic theory suggests a
non-linear relation. We may be able to correct the misspecification by taking the natural logarithm (In) of
the variable we want to represent as a proportional change.

- Data is improperly pooled: Suppose the relationship between returns and the independent variables
during the first three-years is actually different than relationship in the second three-year period i.e.
regression coefficients are different from one period to the next. By pooling the data and estimating one
regression over the entire period, rather estimate two separate regressions over each of the subperiods.
If we have misspecified the model, the predictions of portfolio returns will be misleading.

2. Time Series Misspecification

- Lagged dependent variable is used as an independent variable: If the error term in the regression model
is serially correlated as a result of the lagged dependent variable (which is common in time series
regression), then this model misspecification will result in biased and inconsistent regression estimates
and unreliable hypothesis tests. If lagged dependent variables help explain and not cause serial
correlation in residuals, they are okay!
- A function of the dependent variable is used as an independent variable - 'Forecasting the Past':
Sometimes as a result of the incorrect dating of variables.
- Independent variables are measured with error: Common example being when an independent variable
is measured with error is when we want to use 'Expected Inflation' in our regression but use 'Actual
Inflation' as a proxy.

3. Other Time Series Misspecifications that result in Non-Stationary

Means that a variable's properties such as mean and variance are not constant through time e.g.
consumption and GDP, random walks or exchange rates.

NOTES

1. If expected value of the sample mean is equal to the population mean, the sample mean is therefore an
unbiased estimator of the population mean. A consistent estimator is one for which the accuracy of the
parameter estimate increases as the sample size increases.

2. Predictions in multiple regression model are subject to both parameter estimate uncertainty and
regression model uncertainty.

3. Y = 5 + 4.2 (Beta) - 0.05 (Alpha) + ε. One unit increase in Beta risk is associated with a 4.2% increase in
return, while a $1 bn. increase in Alpha implies a 0.05% decrease in return.
 15

CHAPTER 6 Time Series Analysis

Time Series is a set of observations on a variable's outcomes in different time periods e.g. quarterly sales for
a particular company during the past 5 years.

Fig 1: Linear Vs. Log-Linear Trend Models

Linear Trend Model Log-Linear Trend Model

Yt In (Y t )

.. . .
Linear TM Log-Linear TM
Yt
Transformed
.
..
In (Y t )

Data ..
. . .
... . . . .
Raw Data

.
. . . . ..
0 Time 0 Time

A Linear Trend is a time series pattern that can A Log-Linear Trend works well in fitting time
be graphed using a straight line. A downward series that have exponential growth. Positive
sloping line indicates a negative trend, while an exponential growth means that the time series
upward sloping line indicates a positive trend. tend to increase at some constant rate of growth
↱Trend Coefficient i.e. the observations will form a convex curve ().
Yt = b 0 + b1 (t) + ε t t = 1, 2, ... T Negative exponential growth means that the
data tends to decrease at some constant rate of
^ ^ ^
Yt = b0 + b1 (t) decay i.e. the plotted time series will be concave
curve )(.
When the variable increases over time by a
b0 + b1 (t) + ε t
constant amount, a Linear Trend Model is most Y =e t = 1, 2, ... T
appropriate.
In (Yt ) = In (eb0 + b1 (t) )

Yt = e In (Yt )
^
^ In (Yt )
Yt = e

When a variable grows at a constant rate, a Log-

Linear Trend Model is most appropriate.

If on the other hand, the data plots with a non-linear curved shape, then the residuals from a Linear Trend
Model will be persistently positive or negative for a period of time. In this case, the Log-Linear Trend Model
may be more suitable. In other words, when the residuals from a Linear Trend Model are serially correlated, a
Log-Linear Trend Model maybe more appropriate. However, it may be the case that even a Log-Linear Trend
Model is not appropriate in the presence of serial correlation. In this case, we will want to turn to an
Autoregressive Model.

If Linear Model exhibits Autocorrelation, try Log-Linear

If Log-Linear Model exhibits Autocorrelation, try Autoregression
 16

Fig 2: Covariance Stationary Vs. Non-Stationary

Stationary Non-Stationary

A time series is Covariance Stationary if its mean, Data points are often Non-Stationary or have
variances and covariances with lagged and means, variances and covariances that change
leading values don't change over time. over time.

E.g. Strict Stationary, Second-Order Stationary E.g. Trends, Cycles, Random Walks or
(Weak-Stationary), Trend Stationary and combinations of three.
Difference Stationary Models.

In order to receive consistent-reliable results the non-stationary data needs to be transformed into stationary
data.

Time series is Covariance Stationary if it satisfies the following three conditions:

1. Constant and Finite Expected Value: The expected value of the time series is constant over time. E(y t ) = μ
and |μ| < ∞, t = 1, 2, 3, ... T. We refer to this value as the 'Mean Reverting Level'. All covariance stationary
AR(1) time series have a finite mean reverting level, when the absolute value of the lag coefficient is less
than 1 i.e. |b 1|< 1.

(a) If time series Current Value > Mean; Current Value above Mean AR (1): x t = b 0 + b1 x t-1
As per (c): x t = b 0 + b1 x t
(b) If time series Current Value < Mean; Current value below Mean
^
(c) If time series — Current Value = Mean; Next value of the time series will equal its current value x t = x t-1

Decline: x t > x t+1 ; x t > b0 Rise: x t < x t+1 ; x t < b 0 Same: x t = x t+1 ; x t = b0
(1 - b 1 ) (1 - b1 ) (1 - b1 )

2. Constant and Finite Variance: The time series' volatility around its mean doesn't change over time.

3. Constant and Finite Covariance between values at any given lag: The covariance of the time series with
itself for a fixed no. of periods in the past or future must be constant and finite in all periods.

Both 2 and 3: The covariance of a random variable with itself is its variance
Covariance (yt , y t ) = Var (y t )

Stationary in the past doesn't guarantee stationary in the future. There is always the possibility that a well
specified model will fail with the state of change in time. Models estimated with shorter time series are
usually more stable than those with longer time series because a longer sample period increases the chance
that the underlying economic process has changed. Thus, there is a trade off between the increased
statistical reliability when using longer time periods and the increased stability of the estimates when using
shorter periods.
 17

lllll Autoregressive Model

When the dependent variable is regressed against one or more lagged values of itself, the resultant
model is called as an Autoregressive Model (AR).

BACKWARD AR (1) x t = b 0+ b 1x t-1+ ε t t = 1. 2, ... T

No. of lagged values include
AR (2) x t = b 0 + b 1x t-1+ b 2 x t-2+ ε t
↱as independent variable
To calculate AR (p) x t = b 0+ b 1 x t-1+ b2 x t-2+ ... + bp x t-p+ ε t
the successive ↳Time
forecasts ^ ^ ^
FORWARD AR (1) x t+1 = b0 + b1 x t
referred to as ^ ^
x^ t+2 = b 0 + b 1 x t+1
↷

the 'Chain Rule

of Forecasting'

This implies that multi-period forecasts are more uncertain than single period forecasts.

Testing Autoregressive Model (AR)

lllll Autocorrelation [Stationary]

If the residuals have significant autocorrelation, the AR model that produced the residuals is not the
best model for the time series being analyzed. We can estimate an AR model using ordinary least
squares (OLS) if the time series is covariance stationary and the errors are uncorrelated.
Unfortunately, our previous DW test statistic is invalid when the independent variables include past
values of the dependent variable. Residual autocorrelations drag to '0' as the no. of lags increases.

Step 1: Start with AR (1): x t = b0 + b 1x t-1+ ε t

Step 2: Calculate the autocorrelations of the model's residuals i.e. the level of correlation between
the forecast errors from one period to the next.

ρ k = Cov (x t , x t-1) = E [(x t - μ) (x t-k - μ)] When k = 1

o 2x o 2x
I

Where 'E' stands for the expected value. Note that we have the relationship
2
Cov (x t , xt-k ) ≤ o x with equality holding when k = 0. This means that the absolute value of
I

ρ k ≤ 1.

ρ k = Σ [(xt - x) (x t-k - x)]

I

t=k+1
Σ (x t - x) 2
I

t=1

Whenever we refer to autocorrelation without qualification, we mean autocorrelation of the

time series itself rather than autocorrelation of the error term.

ρε,k = Cov (ε t , εt-k ) = E [(ε t - 0) (ε t-k- 0)] = E (ε t . εt-k )

o 2ε o 2ε o 2ε
I
I

Step 3: Test whether the autocorrelations are significantly different from 'o': If the model is
correctly specified, none of the autocorrelations will be statistically significant. To test for
significance, a T-test is used to test the hypothesis that the correlations of residuals are 0.

t = ρ εt , εt-k Autocorrelation df = n - k -1
1/ n Standard Error

H 0 : |Autocorrelations| = 0 H1 : |Autocorrelations| > 0

 18

lllll Unit Root: Dickey and Fuller

Remember, if an AR (1) model has a coefficient of 1, it has a 'unit root' and no finite mean reverting
level i.e. it is not covariance stationary. By definition, all Random Walks with or without a drift term
have unit roots. Dickey and Fuller (DF) transform the AR (1) model to run a simple regression.

AR (1) x t = b 0+ b1 x t-1+ ε Substract x t-1from both sides

↷
x t - x t-1= b 0 + b1 x t-1- x t-1+ ε
x t - x t-1= b 0+ (b 1 - 1) x t-1+ ε

Rather than directly testing whether the original coefficient is different from 1, they test whether
the new transformed coefficient (b1 - 1) is different from 0 using a modified T-test. In their actual
test, Dickey and Fuller use the variable 'g' = b 1 - 1.

H0 : g = 0 ;Time series has Unit Root

Not Stationary

H1 : g < 0 ;Time series doesn't have a Unit Root

Stationary

lllll Autoregressive Conditional Heteroskedasticity

Autoregressive Conditional Heteroskedasticity (ARCH) exists if the variance of the residuals in one
period is dependent on the variance of the residuals in a previous period. At times, however, this
assumption is violated and the variance of the error term is not constant. In such a situation, ARMA
models will be incorrect and our hypothesis tests would be invalid.

↱ Error Term
ε^t = a 0 + a1 ε^t-1 + μ t
2 2
ARCH (1)
^2 ^ ^ ^2
ot+1= a 0+ a 1 ε t
I

H0 : a1 = 0 ;The variance is constant from period to period

H1 : a1 ≥ or ≤ 0 ;The variance increases (decreases) over time i.e. the error terms
exhibit heteroskedasticity

If a time series model has been determined to contain ARCH errors, regression procedures that correct
for heteroskedasticity such as Generalized Least Squares (GLS) must be used in order to develop a
predictive model. Otherwise, the standard error of the model's coefficients will be incorrect, leading to
invalid conclusions. Engle and other researchers have suggested many generalizations of the ARCH (1)
model which include ARCH (p) and generalized autoregressive conditional heteroskedasticity (GARCH)
models. GARCH models are similar to ARMA models of the error variance in a time series. Just like
ARMA models, GARCH models can be finicky and unstable.

In-Sample Forecasts: are within the range of data i.e. time period.
^
Errors: (Yt - Yt )

Out-of-Sample Forecasts: are made outside of the sample period.

Out-of-sample forecast accuracy is important
because the future is always out-of-sample.
Errors: RMSE i.e. Root Mean Squared Error (Square root
of the average of the squared errors). The model
with the smallest RMSE is judged most accurate.
 19

Example 1:
To qualify as a covariance stationary process, which of the following doesn't have to be true?
A. Covariance (x t , x t-2) = Covariance (x t , xt+2) The covariance between any two
observations equal distance apart will be
B. E (x t ) = E (xt+1)
equal e.g. t and t-2 observations with
C. Covariance (x t , x t-1) = Covariance (x t , x t-2) t and t+2 observations.

Example 2:
Suppose the following model describes changes in the unemployment rate: UER t = - 0.0405 - 0.4674 UER t-1
The current change (first difference) in the unemployment rate is 0.03. Assume that the mean reverting level
for changes in the unemployment rate is -0.0276.

a. What is the best prediction for the next change? - 0.0405 - 0.4674 (0.03) = - 0.0545 (y )
t+1

b. What is the prediction of the change following the next change?

Using chain rule of forecasting: - 0.0405 - 0.4674 (-0.0545) = - 0.0150 (yt+2)

c. Explain your answer to Part 'b' in terms of equilibrium?

The answer to Part 'b' is quite close to the mean reverting level of - 0.0276 (-0.0405/0.4674). A stationary
time series may need many periods to return to its equilibrium, mean reverting level.

Example 3:
Table below gives actual sales, log of sales and changes in the log of sales of Cisco Systems for the period
1Q: 2001 to 4Q: 2001.

Quarters / Yr. Actual Sales Log of Sales Actual Values Forecast Values
In (Sales t ) In (Sales t-1)

1Q: 2001 6519 8.7825 0.1308

2Q: 2001 6748 8.8170 0.0345
3Q: 2001 4728 8.4613 - 0.3557
4Q: 2001 4298 8.3659 - 0.0954
1Q: 2002 x (4391) x (8.3872) x (0.0213)
2Q: 2002 x (4738) x (8.4633) x (0.0761)

Forecast the first and second quarter sales of Cisco Systems for 2002 using the regression
In (Sales t ) = 0.0661 + 0.4698 . In (Sales t-1)

Step 1: Calculate forecast values of In (Sales t ) with the help of above regression.
1Q 2002: In Sales 1Q 2002 = 0.0661 + 0.4698 (-0.0954) = 0.0213
2Q 2002: In Sales 2Q 2002 = 0.0661 +0.4698 (0.0213) = 0.0761

Step 2: Calculate Log of Sales with the help of Step 1

1Q 2002: In Sales 1Q 2002 = 8.3659 + 0.0213 = 8.3872
2Q 2002: In Sales 2Q 2002 = 8.3872 + 0.0761 = 8.4633

Step 3: Calculate Actual Sales

8.3872
1Q 2002: e = 4391
8.4633
2Q 2002: e = 4738
 20

Example 4:
Table below gives the actual change in the Log of Sales of Cisco Systems from 1Q: 2001 to 4Q: 2001, along
with the forecasts from the regression model In (Sales t ) = 0.0661 + 0.4698 . In (Sales t-1) estimated using
data from 3Q: 1991 to 4Q: 2000. Note the observations after the fourth quarter of 2000 are out-of-sample.

Date Actual Values: In Sales t Forecast Values: In Sales t

1Q: 2001 0.1308 0.1357

2Q: 2001 0.0345 0.1299
3Q: 2001 - 0.3557 0.1271
4Q: 2001 - 0.0954 0.1259

a. Calculate the RMSE for the out-of-sample forecast errors.

Error Squared Error

1Q: 2001 - 0.0049 0
2Q: 2001 - 0.0954 0.0091
3Q: 2001 - 0.4828 0.2331
4Q: 2001 - 0.2213 0.0490
0.2912 Sum
0.0728 Average; RMSE = 0.0728 = 0.2698

b. Compare the forecasting performance of the model given with that of another model having an
out-of-sample RMSE of 20%

The model with the RMSE of 20% has greater accuracy in forecasting than model in Part a, which has an
RMSE of 27%

Example 5:
Based on the regression output below, the forecasted value of quarterly sales for March 2016 for PoweredUP
is closest to?

AR (1) Regression Output Quarterly Sales Data

Coefficient Bn

Intercept 0.0092 Dec 2015 (S t-1) $ 3.868

In S t-1- In S t-2 - 0.1279 Sept 2015 (S t-2 ) $ 3.780
In S t-4- In S t-5 0.7239 June 2015 (St-3 ) $ 3.692
Mar 2014 (S t-4 ) $ 3.836
Dec 2014 (St-5 ) $ 3.418

The quarterly sales for March 2016 is calculated as follows:

In S t - In St-1 = b 0+ b1 (In S t-1- In St-2 ) + b 2(In St-4 - In S t-5 )

In S t - In 3.868 = 0.0092 - 0.1279 (In 3.868 - In 3.780) + 0.7239 (In 3.836 - In 3.418)
In S t = 1.35274 + 0.0092 - 0.1279 (0.02301) + 0.7239 (0.11538)
In S t = 1.44251
S t = e 1.44251
= 4.231
 21

Example 6:
David Brice, CFA has used AR (1) model to forecast the next period's interest rate to be 0.08. The AR (1) has a
positive slope coefficient. If the interest rate is the mean reverting process with an unconditional mean a.k.a
mean reverting level, equal to 0.09, then which of the following could be his forecast for two periods ahead?
A. 0.081 Brice makes more distant forecast, each forecast will be closer to the
unconditional mean, so the two period forecast would be between
B. 0.072
0.08 and 0.09; therefore 0.081 is the only possible outcome.
C. 0.113

lllll Random Walks [Non-Stationary]

Random Walks Random Walks with a Drift

AR (1): x t = b0 + b1 x t-1+ ε t AR (1): x t = b 0 + b1 x t-1+ ε t

Since b0 = 0; x t = x t-1+ ε t Since b0 ≠ 0; x t = b0 + x t-1+ ε t

b1 = 1 b1 = 1
↶

↶
(Unit Root) (Unit Root)

No Mean Reverting Level for Random Walk because, Undefined Mean Reverting Level for Random Walk
x t = b0 = 0 = 0 = 0 with Drift because, x t = b0 = b 0
1 - b1 1- 1 0 1 - b1 0

(i) E (ε t ) = 0: The expected value of each error term is 0.

2
(ii) E (ε 2t ) = o : The variance of the error term is constant.
I

(iii) E (εi ε j ) = 0; If i ≠ j: There is no serial correlation in the error terms.

For a time series that is not covariance stationary, the least squares regression procedure that we have been
using to estimate an AR (1) model will not work without transforming the data.

b1 < 1: Stationary
b1 = 1: Non-Stationary 'Unit Root'
b1 > 1: 'Explosive Root'

Testing Random Walk Model

lllll First Differencing

If we believe a time series is a random walk i.e. has a unit root, we can transform the data to a
covariance stationary time series using a procedure called 'First Differencing'. It is because, it
substracts the value of the time series in the first prior period from the current value of the time
series. Note that by taking first differences, you model the change in the value of the dependent
variable.

AR (1) y t = b0 + b1 y t-1+ ε t

If b0 = b 1 = 0, then yt = ε t and ε t = x t - x t-1

x
yt = xt - x t-1 = ε t

This transformed time series has a finite mean reverting level of b 0 = 0 = 0 and is therefore
covariance stationary. 1 - b1 1 - 0
 22

This is how first differencing

removes the upward trend.
Linear
Trend

After 1st
Differencing
y =x -x
t t t-1

lllll Seasonality
Seasonality in a time series is a pattern that tends to repeat from year to year. One example is monthly
sales data for a retailer. Given that sales data normally vary accordingly to the time of year, we might
expect this month's sales (x t ) to be related to sales for the same month last year (x t-12 ). To adjust for
seasonality in an AR model, an additional lag of the dependent variable (corresponding to the same
period in the previous year) is added to the original model as another independent variable. For
example, if quarterly data are used, the seasonal lag is 4; if monthly data is used, the seasonal lag is 12.

Suppose for example, we model a particular quarterly time series using an AR (1) model,
x t = b 0 + b1 x t-1+ ε t . If the time series had significant seasonality, this model would not be correctly
specified. The seasonality would be easy to detect because the seasonal autocorrelation in the case of
quarterly data, the 4th autocorrelation of the error term would differ significantly from 0. Suppose this
quarterly model has significant seasonality. In this case, we might include a seasonal lag in the
autoregressive model and estimate x t = b 0 + b1 x t-1+ b2 xt-2 + ε t , to test whether including the seasonal
lag in the autoregressive model would eliminate statistically significant autocorrelation in the error
term.

lllll Moving Average Time Series Model

Suppose you are analyzing the long-term trend in the past sales of a company. In order to focus on the
trend, you may find it useful to remove short-term fluctuations or noise by smoothing out the time
series of sales. One technique to smooth out period-to-period fluctuations is by:

n-period moving average = x t + x t-1 + ... + x t-n+1

n

Moving Average always lags large movements in the

actual data i.e. slowly rises and slowly falls. Though often
useful in smoothing out a time series, it may not be the
best predictor of the future. A main reason for this is that
Moving Average
a simple moving average gives equal weight to all the
periods in the moving average.

Example 7:
Suppose we want to compute the four-quarter moving average of AstraZeneca's sales as of the beginning of
the first quarter of 2012. AstraZeneca's sales in the previous four quarters were 1Q 2011: $ 8490 m, 2Q 2011:
$8601 m, 3Q 2011: $ 8405 m and 4Q 2011: $ 8872 m.
 23

The 4 Quarter Moving Average = 8490 + 8601 + 8405 + 8872 = $ 8592 m

of 1st Quarter of 2012 4

MA (1) x t = ε t + θ εt-1 E (εt) = 0

2 2
E (εt) = o

I
MA (1) i.e. moving average model places different weights on the two terms Cov (εt, εs) = E (εt. εs) = 0
in the moving average ( 1 on ε t and θ on εt-1 ). Because the expected value of t≠s
x t is 0 in all periods and ε t is uncorrelated with its own part values, the first
autocorrelation is not equal to 0, but the second and higher autocorrelations
are equal to 0. Further analysis shows all autocorrelations except for the first
will be equal to 0 in an MA (1) model. Thus for an MA (1) process, any value x t
is correlated with x t-1 and xt+1 but with no other time series values; we could
say that an MA (1) model has a memory of one period.

MA (p) x t = εt + θ 1 ε t-1+ ... + θp ε t-q E (εt) = 0

2 2
E (εt) = o

I
For an MA (q) model, the first 'q' autocorrelations will be significantly Cov (εt, εs) = E (εt. εs) = 0
different from 0 and all autocorrelations beyond that will be equal to 0; an t≠s
MA (q) model has a memory of 'q' periods.

MA (0) x t= μ + ε t E (εt) = 0
2 2
E (εt) = o

I
MA (0) time series in which we allow the mean to be 'non-zero' which also Cov (εt, εs) = E (εt. εs) = 0
means that the time series is not predictable. t≠s
AR MA

ARMA (p, q) x t = b 0+ b1 x t-1+ ... + bp xt-p + ε t + θ1 ε t-1 + ... + θ qε t-q E (εt) = 0

E (ε2t) = o 2
'b1 , b2 ... b p' are the autoregressive parameters and 'θ 1 , θ 2 ... θ q' are the I
Cov (εt, εs) = E (εt. εs) = 0
moving average parameters. Estimating and using ARMA models have several t≠s
limitations. First, the parameters in ARMA models can be very unstable.
Second, choosing the right ARMA model is more of an art than a science. The
criteria for deciding on 'p' and 'q' for a particular time series are far from
perfect. Moreover, even after a model is selected, that model may not
forecast well. Thirdly, even some of the strongest advocates of ARMA models
admit that these models should not be used with fewer than 80 observations
and they don't recommend using ARMA models for predicting quarterly sales
or gross margins for a company using even 15 years of quarterly data.

lllll Cointegration
Occasionally an analyst will run a regression using two time series i.e. time series utilizing two different
variables. For example, using the market model to estimate the equity beta for a stock, an analyst
regresses a time series of the stock's returns (y t ) on a time series of returns for the market (x t ).
Cointegration means that two time series are economically linked (related to the same macro variable)
or follow the same trend and that relationship is not expected to change.

y t = b0 + b 1 x t + ε

whereas,
y t : Value of time series y at time t
xt : Value of time series x at time t
 24

If neither data series has a unit root : VALID

If only one data series has a unit root: INVALID
If both data series have unit root & they are cointegrated : VALID
If both data series have unit root & they are not cointegrated : INVALID

Fig 3: Time Series Analysis Summary