Summary Topic 4

Topic 4 Classical Normal Linear Regression Model (CNLRM)

Add in assumptions: The error terms must be normally distributed

4.1 The probability Distribution of Disturbance

• If the error terms are normally distributed, then all the parameter also normally
distributed.
• Can apply F-test & t-test in hypothesis testing on β1 & β2

4.2 The normality Assumption for error term, µi

• Mean = 0
E(µi/Xi) =0
• Constant variance
Var (𝝁𝒊 ) = σ2
• No correlation between error term
Var (𝝁𝒊 ) = σ2

4.3 Why the Normality Assumption? Why need?

• Hope that µi is small and at best random
• Central limit theorem (CLT): large number of independent & identically distributed
random variables
When keep adding in independent variables, at the end, the model will normally distributed.
• When assume there is normally distributed under OLS, also assume estimated β1 & β2
is normally distributed as well.
• If without normality assumptions, cannot proceed to hypothesis testing.

4.4 Properties of OLS Estimators (β1 & β2) under the Normality Assumption
• Unbiased
• Minimum variance
• Consistency: Sample size increase, estimator will tend to close to population value
• Normally distributed

4.5 Normality Test: Jarque-Bera Test

𝑺𝟐 (𝑲−𝟑)𝟐
• 𝑱𝑩 = 𝒏 [ 𝟔 + ] , For normally distributed variable, S=0, K=3
𝟐𝟒
Critical value P value approach
H0: The error terms are normally H0: The error terms are normally
distributed distributed
H1: The error terms are not normally H1: The error terms are not normally
distributed distributed
α = 0.05 α = 0.05
X2 α, 2 -
Reject H0 if the test statistic is more than Reject H0 if the p value is less than α =
the 5.991. Otherwise, do not reject H0. 0.05. Otherwise, do not reject H0.
JB test P-value
Do not reject H0 since the test statistic is Do not reject H0 since the p value is
more than the 5.991. larger than α = 0.05.
The error terms are normally distributed. The error terms are normally distributed.

JXT
Summary Topic 4

4.6 Maximum Likelihood

• Alternative method of OLS
• Assumption: Error term must be normally distributed
• In large sample, both are same. If sample size is not huge enough, the estimator’s
variance of error term will be biased.
• Can be applied to non-linear in parameters

JXT
Summary Topic 5

Topic 5 Interval Estimation and Hypothesis Testing

5.1 Introduction
• Ideas of interval estimation and hypothesis testing to models involving 3 or more
variables
• 𝒀𝒊 = 𝜷𝟏 + 𝜷𝟐 𝑿𝟐𝒊 + 𝜷𝟑 𝑿𝟑𝒊 + 𝝁𝒊
5.2 The normality assumption once again
• In order to proceed to hypothesis testing, need to make sure all the models fulfil the
normality assumptions
✓ Constant variance
✓ Error term is normally distribution
✓ All the estimators will be normally distributed
• Therefore, the OLS estimators are BLUE

Condition 1: Individual partial regression coefficient (t-test)

Question: Is there a significant relationship between Yt and X4t at α = 0.05?
1. H0: β4 = 0
H1: β4 ≠ 0
2. Critical value: ta/2, n-k-1 = t0.05/2, 16-5-1 = ±2.228
3. Decision rule: Reject H0 is the test statistic is larger than the upper critical value or
smaller than the lower critical value. Otherwise, do not reject H0.
4. Test statistic = t = -4.364 (from the table)
5. Decision: Reject H0. The test statistic is less than lower critical value.
6. Conclusion: we have enough evidence to conclude that β4 ≠ 0. Unemployment rate
(X4t) is significantly related to annual sales (Yt).

Condition 2: The overall significance of the estimated multiple

regression model (F-test)
Question: Is the model (5.2) significant to explain annual sales (Yt) at α = 0.05?
1. H0: β2 = β3 = β4 = β5 = β6 = 0
H1: Not all slope coefficients are simultaneously zero
2. Critical value: d.f1 = k. d.f2 = n-k-1 F (α, k, n-k-1): F (0.05, 5, 15-5-1) = 3.33
3. Decision rule: Reject H0 if the F statistic is larger than the critical value. Otherwise,
do not reject H0.
𝑹𝟐 /𝒌 𝟎.𝟖𝟐𝟐𝟕𝟓⁄𝟓
4. Test statistic: 𝐅 = (𝟏−𝑹𝟐 )/(𝒏−𝒌−𝟏) = (𝟏−𝟎.𝟖𝟐𝟐𝟕𝟓)/(𝟏𝟔−𝟓−𝟏) = 𝟗. 𝟐𝟖𝟑𝟓
5. Decision: Reject H0. The F statistic (9.2835) is more than critical value (3.333).
6. Conclusion: The model is significant to explain annual sales.

JXT
Summary Topic 5

Condition 3: Whether to add more or exclude some variables on a

multiple regression model (F-test)
*Unrestricted model & restricted model
Question: is it worth adding X5 and X6 on the model (α = 0.05)?
1. H0: β5 = β6 = 0 (2 or more X variables)
H1: At least one β is different from zero.
H0: β6 = 0 (one X variables)
H1: β6 ≠ 0
2. Critical value: df1 = number of new X, df2 = (n-k-1) UR: F (α, df1, df2) = F (0.05, 2, 16-5-1) = 4.1
3. Decision rule: Reject H0 if the F statistic is larger than the critical value. Otherwise,
do not reject H0.
𝑹𝟐𝑼𝑹 −𝑹𝟐𝑹 ⁄𝒏𝒐. 𝒐𝒇 𝒏𝒆𝒘 𝑿 (𝟎.𝟖𝟐𝟐𝟕𝟓−𝟎.𝟔𝟎𝟏𝟐𝟓𝟒)⁄𝟐
4. Test statistic: 𝐅 = (𝟏−𝑹𝟐𝑼𝑹 )/(𝒏−𝒌−𝟏)𝑼𝑹
= (𝟏−𝟎.𝟖𝟐𝟐𝟕𝟓)/(𝟏𝟔−𝟓−𝟏) = 𝟔. 𝟐𝟒𝟖𝟐
5. Decision: Reject H0. The F statistic (6.2482) is more than critical value (4.1).
6. Conclusion: Adding X5 and X6 can improve model.

Condition 4: Two or more coefficients are equal (t-test)

Question: are the partial regression coefficients for X3i and X4i the same?
1. H0: β3 = β4
H1: β3 ≠ β4
2. Critical value: 𝒕𝜶⁄ = 𝒕𝟎.𝟎𝟓⁄ = ± 𝟐. 𝟒𝟒𝟕
𝟐 , 𝒏−𝒌−𝟏 𝟐 , 𝟏𝟎−𝟑−𝟏
3. Decision rule: Reject H0 is the test statistic is larger than the upper critical value or
smaller than the lower critical value. Otherwise, do not reject H0.
̂ 𝟑 −𝜷
(𝜷 ̂ 𝟒) −𝟏𝟐.𝟗𝟔𝟏𝟓−𝟎.𝟗𝟑𝟗𝟓
4. Test statistic: 𝒕 = = = −𝟏𝟑. 𝟑𝟏𝟑
̂ 𝟑 )+𝒗𝒂𝒓(𝜷
√𝒗𝒂𝒓(𝜷 ̂ 𝟒 )−𝟐𝒄𝒐𝒗(𝜷
̂ 𝟑 ,𝜷
̂ 𝟒) √(𝟎.𝟗𝟖𝟓𝟕)𝟐 +(𝟎.𝟎𝟓𝟗)𝟐 −𝟐(−𝟎.𝟎𝟓𝟕𝟔)

5. Decision: Reject H0. The test statistic (-13.313) is less than lower critical value (-2.447).
6. Conclusion: we have enough evidence to conclude that β3 ≠ β4.

Condition 5: Testing linear restriction (F-test)

Question: Is there a constant return to scale at α = 0.05?
1. H0: β2 + β3 = 1
H1: β2 + β3 ≠ 1
2. Critical value: df1 = 1, df2 = (n-k-1) UR: 𝑭𝜶 , 𝒅𝒇𝟏 , 𝒅𝒇𝟐 = 𝑭(𝟎.𝟎𝟓, 𝟏, 𝟐𝟎−𝟐−𝟏) = 𝟒. 𝟒𝟓
3. Decision rule: Reject H0 if the F statistic is larger than the critical value. Otherwise,
do not reject H0.
(𝑹𝑺𝑺𝑹 −𝑹𝑺𝑺𝑼𝑹 )⁄𝒅𝒇𝟏 (𝟎.𝟎𝟏𝟔𝟔−𝟎.𝟎𝟏𝟑𝟔)⁄𝟏
4. Test statistic: 𝐅 = 𝑹𝑺𝑺 /𝒅𝒇
= (𝟎.𝟎𝟏𝟑𝟔)/(𝟐𝟎−𝟐−𝟏) = 𝟑. 𝟕𝟓
𝑼𝑹 𝟐
5. Decision: Do not reject H0. The F statistic (3.75) is less than critical value (4.45).
6. Conclusion: There is constant return to scale (β2 + β3 = 1).

JXT
Summary Topic 5

Condition 6: Testing ‘stability’ of the estimated regression model

over time or in different cross-sectional unit (Chow test)
• Testing for structural change that due to external forces
• Unrestricted model: 2000 – 2007 (Before Crisis)
• Unrestricted model: 2008 – 2015 (After Crisis)
• Restricted model: 2000 – 2015 (Whole Period)
• Chow test assumes
✓ Error term is normally distributed – Normality Test (JB)
✓ Same variance – Homoscedasticity test
✓ Error term is independently distributed
Question: Is there structural change over the period of 2000 through 2015?
1. H0: There is no structural change
H1: There is structural change
2. Critical value: 𝑭(𝜶, 𝒌+𝟏,𝒏𝟏 +𝒏𝟐 −𝟐𝒌−𝟐)
3. Decision Rule: Reject H0 is the test statistic is more than the critical value.
Otherwise, do not reject H0.
(𝑹𝑺𝑺𝑹 −𝑹𝑺𝑺𝑼𝑹 )⁄𝒌+𝟏
4. Test statistic: 𝑭 = 𝑹𝑺𝑺
𝑼𝑹 /(𝒏𝟏 +𝒏𝟐 −𝟐𝒌−𝟐)

JXT
Summary Topic 5
Condition 1 Condition 2 Condition 3 Condition 4 Condition 5
Individual Overall Add/Exclude 2 or more coefficient same (t- Linear restriction & constant return
(t-test) (F-test) (F-test) test) to scale
(F-test)
H0: β4 = 0 H0: β2 = β3 = β4 = β5 = 0 H0: β5 = β6 = 0 H0: β3 = β4 H0: β2 + β3 = 1
H1: β4 ≠ 0 H1: Not all slope H1: At least one β is H1: β3 ≠ β4 H1: β2 + β3 ≠ 1
coefficients are different from zero.
simultaneously zero

α = 0.05 α = 0.05 α = 0.05 α = 0.05 α = 0.05

𝒕𝜶/𝟐 , (𝒏−𝒌−𝟏) 𝑭𝜶 , 𝒌 , (𝒏−𝒌−𝟏) 𝑭𝜶 , 𝒏𝒆𝒘 𝑿 , (𝒏−𝒌−𝟏)𝑼𝑹 𝒕𝜶⁄ , 𝒏−𝒌−𝟏 𝑭𝜶 , 𝟏 , (𝒏−𝒌−𝟏)𝑼𝑹
𝟐
Reject H0 if the test statistic Reject H0 if the F statistic Reject H0 if the F statistic Reject H0 is the test statistic Reject H0 if the F statistic is larger
is larger than the upper is larger than the critical is larger than the critical is larger than the upper than the critical value. Otherwise,
critical value or smaller than value. Otherwise, do not value. Otherwise, do not critical value or smaller than do not reject H0.
the lower critical value. reject H0. reject H0. the lower critical value.
Otherwise, do not reject H0. Otherwise, do not reject H0.
̂𝟑 − 𝜷
̂ 𝟒)
T = Coefficient / Standard 𝑹𝟐 /𝒌 𝑹𝟐𝑼𝑹 − 𝑹𝟐𝑹 ⁄𝒏𝒐. 𝒐𝒇 𝒏𝒆𝒘 𝑿 𝒕=
(𝜷 (𝑹𝑺𝑺𝑹 − 𝑹𝑺𝑺𝑼𝑹 )⁄𝒅𝒇𝟏
𝐅= 𝐅= ̂ 𝟑 ) + 𝒗𝒂𝒓(𝜷
̂ 𝟒 ) − 𝟐𝒄𝒐𝒗(𝜷
̂ 𝟑, 𝜷
̂ 𝟒) 𝐅=
Error (𝟏 − 𝑹𝟐 )/(𝒏 − 𝒌 − 𝟏) (𝟏 − 𝑹𝟐𝑼𝑹 )/(𝒏 − 𝒌 − 𝟏)𝑼𝑹 √𝒗𝒂𝒓(𝜷
𝑹𝑺𝑺𝑼𝑹 /𝒅𝒇𝟐
Reject H0 since the test Reject H0 since the test Reject H0 since the test Reject H0 since the test Reject H0 since the test statistic is
statistic is less than lower statistic is more than statistic is more than statistic is less than lower more than critical value.
critical value. critical value. critical value. critical value.

We have enough evidence The model is significant to Adding X5 and X6 can We have enough evidence to There is constant return to scale
to conclude that β4 ≠ 0. explain annual sales. improve model. conclude that β3 ≠ β4. (β2 + β3 = 1).
Unemployment rate (X4t) is
significantly related to
annual sales (Yt).

! F-test always should be more than 1

JXT
Summary Topic 5
Condition 6
Normality test (JB test) Homoscedasticity test (F-test) Chow test (F-test)
H0: The error terms are normally H0: There is homoscedasticity H0 : β i = 0
distributed H1: There is heteroscedasticity H1 : β i ≠ 0
H1: The error terms are not
normally distributed

α = 0.05 α = 0.05 α = 0.05

X2 α, 2 Fα, (n-k-1)1, (n-k-1)2 Fα, k+1, n1+n2-2k-2
Reject H0 if the test statistic is Reject H0 if the test statistic is Reject H0 if the test statistic is
more than the 5.991. Otherwise, more than the xxxx. Otherwise, more than the xxx. Otherwise, do
do not reject H0. do not reject H0. not reject H0.
𝑺𝟐 (𝑲 − 𝟑)𝟐 𝑹𝑺𝑺𝟐/𝒅𝒇𝟏 (𝑹𝑺𝑺𝒓 − 𝑹𝑺𝑺𝒖𝒓)/𝒅𝒇𝟏
𝑱𝑩 = 𝒏 [ + ] 𝑭= 𝑭=
𝟔 𝟐𝟒 𝑹𝑺𝑺𝟏/𝒅𝒇𝟐 𝑹𝑺𝑺𝒖𝒓/𝒅𝒇𝟐
Do not reject H0 since the test Do not reject H0 since the test Do not reject H0 since the test
statistic is more than the 5.991. statistic is more than the xxxx. statistic is more than the xxxx.
The error terms are normally There is homoscedasticity. There is a structural change.
distributed.
! Chow test: RSSUR = RSS1 + RSS2
3 approaches of t-test
Critical Value P-value approach Confidence Interval
H0: βi = 0 H0: βi = 0 H0 : β i = 0
H1: βi ≠ 0 H1: βi ≠ 0 H1 : β i ≠ 0

α = 0.05 α = 0.05 α = 0.05

𝒕𝜶/𝟐 , (𝒏−𝒌−𝟏) - -
Reject H0 is the test statistic is Reject H0 if the p value is less Reject H0 if the hypothesized value
larger than the upper critical than α=0.05. Otherwise, do not of zero falls outside the interval.
value or smaller than the lower reject H0. Otherwise, do not reject H0.
critical value. Otherwise, do not
reject H0.
T = Coefficient / Standard Error P-value C. I. = βi ± 𝑡𝛼/2 , (𝑛−𝑘−1) (Seβ)
Reject H0 since the test statistic is Reject H0 since the p value is less Reject H0 if the hypothesized value
less than lower critical value. than α=0.05. of zero falls outside the interval.

Do not reject H0 since the test Do not reject H0 since the p value Do not reject H0 if the
statistic falls in between lower is more than α=0.05. hypothesized value of zero falls
and upper critical value inside the interval.
We can/cannot conclude that βi is We can/cannot conclude that βi is We can/cannot conclude that βi is
different to zero. Xi is significant. different to zero. Xi is significant. different to zero. Xi is significant.

JXT
Summary Topic 6

Topic 6 Extension of the Linear Regression Model

6.1 Introduction
• Regression through the origin
• Units of measurements
• Functional form
6.2 Comparison between regression through the origin & regression with intercept
• We cannot directly be comparable with R2 between without intercept model and with
intercept model.
• If do not have intercept, they would not show the R2
• Under some special features of model, we want to apply zero intercept, you are
required to have strong justification. If not, advised to have intercept model.
• Specification bias will occur if ignore the topic with intercept.
6.3 Scaling and Unit of Measurement
Example 1 Example 2 Example 3
̂ 𝒊 = −𝟗𝟐𝟔. 𝟎𝟗 + 𝟎. 𝟐𝟓𝟑𝟓𝑿𝒊
𝒀
̂ 𝒊 = 𝒃𝒊𝒍𝒍𝒊𝒐𝒏
𝒀
𝑿𝒊 = 𝒃𝒊𝒍𝒍𝒊𝒐𝒏
Y = billion to million Y = unchanged Y = billion to million
X = billion to million X = billion to million X = unchanged
Step 1: Step 1: Step 1:
Wy = 1000 Wy = 1 Wy = 1000
Wx = 1000 Wx = 1000 Wx = 1
Step 2: Step 2: Step 2:
̂ ∗𝟏 = 𝟏𝟎𝟎𝟎 (−𝟗𝟐𝟔. 𝟎𝟗)
𝜷 ̂ ∗𝟏 = 𝟏 (−𝟗𝟐𝟔. 𝟎𝟗)
𝜷 ̂ ∗𝟏 = 𝟏𝟎𝟎𝟎 (−𝟗𝟐𝟔. 𝟎𝟗)
𝜷
= -926090 = -926.09 = -926090

𝟏𝟎𝟎𝟎 𝟏 𝟏𝟎𝟎𝟎
̂ ∗𝟐 = (
𝜷 ) 𝟎. 𝟐𝟓𝟑𝟓 ̂ ∗𝟐 = (
𝜷 ) 𝟎. 𝟐𝟓𝟑𝟓 ̂ ∗𝟐 = (
𝜷 ) 𝟎. 𝟐𝟓𝟑𝟓
𝟏𝟎𝟎𝟎 𝟏𝟎𝟎𝟎 𝟏
= 0.2535 = 0.0002535 = 253.50
Step 3: Step 3: Step 3:
̂ 𝒊 = −𝟗𝟐𝟔𝟎𝟗𝟎 + 𝟎. 𝟐𝟓𝟑𝟓𝑿𝒊
𝒀 ̂ 𝒊 = −𝟗𝟐𝟔. 𝟎𝟗 + 𝟎. 𝟎𝟎𝟎𝟐𝟓𝟑𝟓𝑿𝒊
𝒀 ̂ 𝒊 = −𝟗𝟐𝟔𝟎𝟗𝟎 + 𝟐𝟓𝟑. 𝟓𝟎𝑿𝒊
𝒀
Step 4: Step 4: Step 4:
The GPDI will decrease by The GPDI will decrease by The GPDI will decrease by
$926090 in million if GDP = $926.09 in billion if GDP = $926090 in million if GDP =
0. 0. 0.

If GDP increase by 1 If GDP increase by 1 If GDP increase by 1 billion,

million, on average, Y will million, on average, Y will on average, Y will increase
increase by $0.2535 million increase by $0.0002535 by $253.50 million
billion

JXT
Summary Topic 6

6.4 Functional Forms of Regression Models

Model Descriptive Title

𝒍𝒐𝒈(𝒀𝒊 ) = 𝒍𝒐𝒈(𝜷𝟏 ) + 𝜷𝟐 𝒍𝒐𝒈(𝑿𝒊 ) + 𝝁𝒊 log-log; double-log; log-linear model

𝒍𝒐𝒈(𝒀𝒊 ) = 𝜷𝟏 + 𝜷𝟐 𝑿𝒊 + 𝝁𝒊 Semi-logarithmic: log-lin model

𝒀𝒊 = 𝜷𝟏 + 𝜷𝟐 𝒍𝒐𝒈(𝑿𝒊 ) + 𝝁𝒊 Semi-logarithmic: lin-log model
𝟏 Reciprocal
𝒀𝒊 = 𝜷𝟏 + 𝜷𝟐 ( ) + 𝝁𝒊
𝑿𝒊
𝟏 Logarithmic reciprocal
𝒍𝒐𝒈(𝒀𝒊 ) = 𝜷𝟏 + 𝜷𝟐 ( ) + 𝝁𝒊
𝑿𝒊
• When interpret log parameter, need to mention in percentage (%)
1. ln (AB) = ln A + ln B
2. ln (A/B) = ln A - ln B
3. ln (Ak) = k ln A
Log linear model Log-Lin model (β x 100) Lin-log model (β/100)
Measure the elasticity of Y Measure the growth rate of
with respect to X certain economic variables
̂ 𝒊) = −𝟕. 𝟓𝟒 + 𝟎. 𝟔𝟑 𝒍𝒐𝒈 (𝑿𝒊 )
𝒍𝒐𝒈(𝒀 ̂ 𝒊) = −𝟕. 𝟓𝟒 + 𝟎. 𝟔𝟑 (𝑿𝒊 )
𝒍𝒐𝒈(𝒀 ̂ 𝒊 ) = −𝟕. 𝟓𝟒 + 𝟎. 𝟔𝟑 𝒍𝒐𝒈 (𝑿𝒊 )
(𝒀

𝒓𝒆𝒍𝒂𝒕𝒊𝒗𝒆 𝒄𝒉𝒂𝒏𝒈𝒆 𝒊𝒏 𝒀 %∆𝒀 𝒓𝒆𝒍𝒂𝒕𝒊𝒗𝒆 𝒄𝒉𝒂𝒏𝒈𝒆 𝒊𝒏 𝒀 𝒂𝒃𝒔𝒐𝒍𝒖𝒕𝒆 𝒄𝒉𝒂𝒏𝒈𝒆 𝒊𝒏 𝒀

𝜷𝟐 = = 𝜷𝟐 = 𝜷𝟐 =
𝒓𝒆𝒍𝒂𝒕𝒊𝒗𝒆 𝒄𝒉𝒂𝒏𝒈𝒆 𝒊𝒏 𝑿 %∆𝑿
𝒂𝒃𝒔𝒐𝒍𝒖𝒕𝒆 𝒄𝒉𝒂𝒏𝒈𝒆 𝒊𝒏 𝑿 𝒓𝒆𝒍𝒂𝒕𝒊𝒗𝒆 𝒄𝒉𝒂𝒏𝒈𝒆 𝒊𝒏 𝑿

𝜷𝟐=𝟎.𝟔𝟐𝟔𝟔 𝜷𝟐=𝟎.𝟔𝟐𝟔𝟔 𝜷𝟐=𝟎.𝟔𝟐𝟔𝟔

If total personal income If total personal income If total personal income
increases by 1%, on increases by 1percentage increases by 1%, on
average, expenditure point, on average, average, expenditure
increases by 0.63%. expenditure increases by increases by 0.006
62.3% percentage point
[0.6266 x 100]. [0.6266/ 100].

X&Y – percentage change X – point change X – percentage change

Y – percentage change Y – point change

• Reciprocal Model
̂ 𝒊 ) = 𝟖𝟏. 𝟕𝟗𝟒𝟒 + 𝟎. 𝟔𝟑 (𝟏/𝑿𝒊 )
(𝒀
As per capita GNP increases indefinitely, child mortality approaches its asymptotic
value of 82 deaths per thousand.
6.4.1 Percentage Change vs Percentage Point Change
E.g., The unemployment rate from 6% increase to 8%.
8%−6%
Log - Percentage change: 33% ( )
6%
Normal - Percentage point change: 2% (8%-6%)

JXT
Summary Topic 7
Topic 7 Multicollinearity (Independent Variables are correlated)
7.1 What is Multicollinearity?
• All explanatory variables are highly correlated with one another.
• If present, the regression model cannot tell which X are affect Y
7.2 Types of Multicollinearities
• Perfect collinearity
• Imperfect collinearity
7.3 Why lead to perfect collinearity/Sources of perfect collinearity
Normally multicollinearity occur in time series. The model shares same trend.

• The data collection method employed

✓ Limited range and don’t have much choice
• Model specification
✓ Solution: Adding in polynomial terms (express more than 2 algebraic terms)
✓ Exp: x3, x2
• An overdetermined model
✓ Independent variables more than number of observation (k > n)
✓ Exp: Medical research (10 patients, more information)
7.4 Theoretical Consequences of Multicollinearity
Question: Is multicollinearity a problem for BLUE?
Answer: No. OLS estimators still unbiased.

• When come to sampling, X variables are fixed in the repeated sample

• When come to OLS, 𝐄(𝜷 ̂ ) = 𝜷) as long as the number of samples increases
• OLS estimators: unbiased, efficient (minimum variance), correctly estimated standard errors.
• ̂ ) with small standard error/variance
Effect: Hard to get (𝜷

7.5 Practical Consequences of Multicollinearity

1. Large variances and covariances
• 𝒓𝟐𝑿𝟐 𝑿𝟑 = 𝟏
• Variances of 𝛽̂2 and 𝛽̂3 are infinite
2. Wider Confidence intervals
• 𝒓𝟐𝑿𝟐 𝑿𝟑 increase
• Standard errors increase
• Confidence interval larger
• Probability of not rejecting a false hypothesis high
3. Insignificant T ratio
• 𝒓𝟐𝑿𝟐 𝑿𝟑 increase
• Standard errors increase
• Probability of not rejecting a H0 increase
4. High R2 is but few significant t Ratios
• Insignificant t values
• High R2 (exceed 0.9)
• Significant F
5. OLS estimators and standard errors can be sensitive to small changes in the data`
• Increase in 𝒔𝒆(𝜷 ̂ 𝟏 ), 𝒔𝒆(𝜷
̂ 𝟐 ), and 𝒓𝟐𝑿 𝑿
𝟏 𝟐

JXT
Summary Topic 7
• Change in 𝒄𝒐𝒗(𝜷
̂ 𝟏, 𝜷
̂ 𝟐)

7.6 Detection of Multicollinearity

✓ Question of degree and not of kind of presence or absence
✓ Features of sample and not of population

1. High R2 but few significant t ratios

• T insignificant
• F significant
2. High pair-wise correlation among X variables
• 𝒓𝟐𝑿𝟏 𝑿𝟐 is high
3. VIF and TOL
R2 is taken from auxiliary model
Variance Inflation Factor (VIF) Tolerance (TOL)
Formula 𝟏 𝟏
𝑽𝑰𝑭 = 𝑻𝑶𝑳𝒋 = = (𝟏 − 𝒓𝟐𝑿𝟐 𝑿𝟑 )
(𝟏 − 𝑹𝟐𝑿𝟏 𝑿𝟐 ) 𝑽𝑰𝑭𝒋
Perfect collinearity Infinite 0
(R2 = 1)
No collinearity 1 1
(R2 = 0)
High collinearity >10 Closer to zero
(R2 > 0.9)

7.7 Remedial Measure

• Prior information (x3, x2)
• Combining time series data and cross-sectional data (panel data)
• Dropping a variable but results in specification bias (Drop highly collinearity variable)
• Transformation of variables (x2-x2-t-1)
• Increase sample size
• Reducing collinearity in polynomial regressions

JXT
Summary Topic 8
Topic 8 Heteroscedasticity (Variance not constant)
8.1 Nature of heteroscedasticity
• CLRM assumes the error term should have same variance
• Homoscedasticity: 𝒗𝒂𝒓(𝝁𝒊 |𝑿𝟏𝒊 , … , 𝑿𝒌𝒊 ) = 𝝈𝟐
• Heteroscedasticity: 𝒗𝒂𝒓(𝝁𝒊 |𝑿𝟏𝒊 , … , 𝑿𝒌𝒊 ) = 𝝈𝟐𝒊

8.2 Why does Heteroscedasticity occur?

• Error-learning models
✓ Number of errors become more consistent
✓ Variance expected to decrease
✓ Exp: The number of hours of typing practice increase, the average of typing error as well as variance
decrease
• Human behaviors
✓ Exp: Disposition/Savings on Income
Income increase, more extra income, more scope of choice of the disposition of income (Variance
increase with income)
✓ Exp: Growth oriented company unlikely to show more variability in their dividend payout ratio
than the established company
Growth oriented company: High risk high return, dividend to shareholder quite volatile, variance
volatile
• Data collecting techniques
• Outliers
• Misspecification
8.3 Consequences
• Unbiased and consistent (No longer BLUE)
• Variance no longer minimum – Inefficient (Bias)
• Standard error is no longer valid
• T and F statistics unreliable
• Conclusion may be misleading
8.4 Detection of Heteroscedasticity
Regress the residual on Xi in the following auxiliary model
nR2 & d.f.: R2 & d.f. [Prob Chi-Square(x)] take from auxiliary model
White test Park test Glejser test Breusch-Pagan test
RESID^2 LOG(UTSQ) ARESID RESID^2
1. 𝐻0 : There is no heteroscedasticity
𝐻1 : There is heteroscedasticity
2. Critical value: 𝜒𝛼 , 𝑘 = 𝜒0.05 , 5 = 11.07
3. Decision rule: Reject 𝐻0 if the test statistic (𝑛. 𝑅 2 ) is larger than the critical value. Otherwise, do
not reject 𝐻0 .
4. Test statistic: 𝑛. 𝑅 2 = 8.19 (from the table)
5. Decision: Do not reject 𝐻0 . The test statistic (8.19) is less than the critical value (11.07).
6. Conclusion: There is no heteroscedasticity. (There is homoscedasticity.)

8.5 Remedial Measures

• Generalized Least Square (GLS)/ Weighted Least Squares (WLS)
• If heteroscedasticity present, re-estimates the model by applying GLS method
• Produce a new set parameter estimates (correct covariance and t-statistics)
• GLS: Divide each term in the model by standard deviation of error term
𝒀𝒊 𝟏 𝑿𝟐𝒊 𝑿𝟑𝒊 𝑿𝒌𝒊 𝝁𝒊
= 𝜷𝟏 + 𝜷𝟐 + 𝜷𝟑 + ⋯ + 𝜷𝒌 +
𝝈𝒊 𝝈𝒊 𝝈𝒊 𝝈𝒊 𝝈𝒊 𝝈𝒊
JXT
Summary Topic 8
• WLS: 𝒘𝒊 = 𝝈𝟏
𝒊
𝒘𝒊 𝒀𝒊 = 𝜷𝟏 𝒘𝒊 + 𝜷𝟐 𝑿𝟐𝒊 𝒘𝒊 + 𝜷𝟑 𝑿𝟑𝒊 𝒘𝒊 + ⋯ + 𝜷𝒌 𝑿𝒌𝒊 𝒘𝒊 + 𝝁𝒊 𝒘𝒊

JXT
Summary Topic 9
Topic 9: Autocorrelation (Error term correlated)
9.1 Introduction
• Cross sectional data: Spatial autocorrelation
• Time series data: Serial Correlation

9.2 The nature of autocorrelation

• CLRM assumes no autocorrelation
• Error term between two period not correlated
Exp: Labor output (Time Series)
If there is a labor strike, the labor strike only affect in a quarter, cannot carry forward to next quarter.
Exp: The effect of family income on consumption expenditures (Cross-Sectional)
The effect of increase in one family income on its consumption expenditures family income is not
expected to affect the consumption expenditures of another family.
• Autocorrelation: 𝒄𝒐𝒗(𝝁𝒊 , 𝝁𝒋 ) ≠ 𝟎 𝒊≠𝒋
• No autocorrelation: 𝒄𝒐𝒗(𝝁𝒊 , 𝝁𝒋 ) = 𝟎 𝒊≠𝒋

9.3 Types of Autocorrelations

Pure Serial Correlation
𝝆=𝟎 No serial correlation
𝝆>𝟎 Positive serial correlation Same sign from one period to next
𝝆<𝟎 Negative serial correlation Negative to positive and back again
Impure Serial Correlation
• Caused by specification bias (omitted variables/incorrect functional)
✓ Omitted the important variable
✓ Capture by error term
✓ Lead to serial correlation in future
9.4 Consequences of Auto correlation
• OLS unbiased and consistent
• Inefficient – Variance understate or overstate
Understated Overstated
Smaller standard error Larger standard error
Larger t statistic Smaller t statistic
Variables not important Variables not important
Hypothesis testing no valid Hypothesis testing no valid
9.5 Detecting Autocorrelation: Durbin-Watson d test
• Assumptions of d statistic
✓ Included the intercept
✓ Independent variables are non-stochastic (fixed)
✓ Disturbances are generated by first-order autoregressive
✓ Error term is normally distributed
• Features of the test
✓ Never test the one side null hypothesis
✓ ‘the inconclusive’

JXT
Summary Topic 9

Two sided Positive Negative

𝐻0 : 𝜌 = 0 (No serial correlation) 𝐻0 : 𝜌 ≤ 0 (No positive) 𝐻0 : 𝜌 ≥ 0 (No negative)
𝐻1 : 𝜌 ≠ 0 (Serial correlation) 𝐻1 : 𝜌 > 0 (Positive) 𝐻1 : 𝜌 < 0 (Negative)

if d < dL, reject H0. if d < dL, reject H0. if d > 4 – dL, reject H0.
if d > 4 - dL, reject H0. if d > dU, do not reject H0. if d < 4 - dU, do not reject H0.
if dU < d < 4 - dU, do not reject H0. if dL ≤ d ≤ dU, inconclusive. if 4 - dU ≤ d ≤ 4 - dL,
Otherwise, inconclusive. inconclusive.

9.6 Remedial Measure

• Make sure the model is not misspecification
• Pure autocorrelation – GLS method to transform model
• Large sample – Newey-West method

JXT
Summary Topic 7, 8, 9
Problem Multicollinearity Heteroscedasticity Autocorrelation
Nature Independent Variables are correlated Variance of error term not constant Error terms are correlated
Types • Perfect collinearity - • Pure serial correlation
• Imperfect collinearity • Impure serial correlation
Reason • The data collection method employed • Error-learning models -
• Model specification • Human behaviors
• An overdetermined model • Data collecting techniques
• Outliers
• Misspecification

Consequences • OLS: unbiased, efficient, correctly estimated • OLS: Unbiased and consistent • OLS: Unbiased and consistent,
standard errors • Variance no longer minimum inefficient
• Hard to get (𝜷̂ ) with small standard error/variance • Standard error is no longer valid • Smaller standard error
• Large variances and covariances • T and F statistics unreliable • Larger t statistics
• Wider Confidence intervals • Conclusion may be misleading
• Insignificant T ratio
• High R2 is but few significant t Ratios
• OLS estimators and standard errors can be sensitive
to small changes in the data
Detection • High R2 but few significant t ratios • White test • Durbin-Watson d test
• High pair-wise correlation among X variables • Park test
• VIF and TOL • Glejser test
• Breusch-Pagan test
Remedial • Prior information • Generalized Least Square (GLS) • GLS method – transform model
measure • Combining time series data and cross-sectional data • Weighted Least Squares (WLS) (Pure)
• Dropping a variable but results in specification bias • Newey-West method (large
• Transformation of variables sample)
• Increase sample size
• Reducing collinearity in polynomial regressions

JXT
Interpretation & Formula List
Interpretation
̂ 𝟏 : The (𝑌̂𝑖 ) equals - $3.3384 if (𝑋2𝑖 ) and (𝑋3𝑖 ) equal zero.
𝜷
̂ 𝟐 : If (𝑋3𝑖 ) increases by 1, on average, the (𝑌𝑖 ) increases by β2, ceteris paribus.
𝜷
𝑹𝟐 : There are % of the variation in 𝑌𝑖 is explained by the variation in (𝑋2𝑖 ) and (𝑋3𝑖 ).
̅ 𝟐 : There are % of the variation in 𝑌𝑖 is explained by the variation in (𝑋2𝑖 ) and (𝑋3𝑖 ), after taking into account
𝑹
the degrees of freedom.
Log-Linear model: If the x increases by 1%, on average, the y increase by 1%
Log-lin model: If the x increases by 1 percentage point, on average, the y increase by 1% (x 100)
Lin-log model: If the x increases by 1%, on average, the y increase by 1 percentage point (÷ 100)
Topic 4
𝑺𝟐 (𝑲 − 𝟑)𝟐
𝑱𝑩 = 𝒏 [ + ]
𝟔 𝟐𝟒

Topic 5
Condition Critical Value Degree of freedom
Condition 1: Coefficient / Standard Error t (a/2, n-k-1) df = n-k-1
𝑹𝟐 /𝒌 F (α, k, n-k-1) df1 = k
Condition 2: 𝐅 = (𝟏−𝑹𝟐 )/(𝒏−𝒌−𝟏)
df2 = n-k-1
𝑹𝟐𝑼𝑹 −𝑹𝟐𝑹 ⁄𝒏𝒐. 𝒐𝒇 𝒏𝒆𝒘 𝑿 F (α, df1, df2) df1 = number of new X
Condition 3: 𝐅 = (𝟏−𝑹𝟐𝑼𝑹 )/(𝒏−𝒌−𝟏)𝑼𝑹 df2 = (n-k-1) UR
̂ 𝟑 −𝜷
(𝜷 ̂ 𝟒)
Condition 4: 𝒕 = t (a/2, n-k-1) df = n-k-1
̂ 𝟑 )+𝒗𝒂𝒓(𝜷
√𝒗𝒂𝒓(𝜷 ̂ 𝟒 )−𝟐𝒄𝒐𝒗(𝜷
̂ 𝟑 ,𝜷
̂ 𝟒)

̂ 𝒊 −𝜷
(𝜷 ̂ 𝒋 )−(𝜷𝒊 −𝜷𝒋 𝟎)
𝒕= 𝒔𝒆(𝜷 ̂ 𝒊 −𝜷
̂ 𝒋)

(𝑹𝑺𝑺𝑹 −𝑹𝑺𝑺𝑼𝑹 )⁄𝒅𝒇𝟏 F (α, df1, df2) df1 = 1 (Fixed)

Condition 5: 𝐅 = 𝑹𝑺𝑺𝑼𝑹 /𝒅𝒇𝟐
df2 = (n-k-1)
𝑹𝟐𝑼𝑹 −𝑹𝟐𝑹 ⁄𝒅𝒇𝟏
=
(𝟏−𝑹𝟐𝑼𝑹 )/𝒅𝒇𝟐

(𝑹𝑺𝑺𝑹 −𝑹𝑺𝑺𝑼𝑹 )⁄𝒌+𝟏 𝑭(𝜶, 𝒌+𝟏,𝒏𝟏 +𝒏𝟐 −𝟐𝒌−𝟐)

Condition 6: 𝑭 = 𝑹𝑺𝑺
𝑼𝑹 /(𝒏𝟏 +𝒏𝟐 −𝟐𝒌−𝟐)

Topic 6

𝒘𝒀 ̂ ∗𝟏 = 𝒘𝒀 𝜷
𝜷 ̂𝟏
̂ ∗𝟐 = (
𝜷 ̂
)𝜷
𝒘𝑿 𝟐
If the increase by 1 unit, on average, the y
If x = 0, the y is xx (in units) increase by xx.

Topic 7
𝟏 𝟏
𝑽𝑰𝑭 = 𝑻𝑶𝑳𝒋 = = (𝟏 − 𝒓𝟐𝑿𝟐 𝑿𝟑 )
(𝟏 − 𝑹𝟐𝑿𝟏𝑿𝟐 ) 𝑽𝑰𝑭𝒋

JXT
Interpretation & Formula List
Topic 8
Test statistic: 𝒏. 𝑹𝟐

Topic 9
Two sided Positive Negative
if d < dL, reject H0. if d < dL, reject H0. if d > 4 – dL, reject H0.
if d > 4 - dL, reject H0. if d > dU, do not reject H0. if d < 4 - dU, do not reject H0.
if dU < d < 4 - dU, do not reject H0. if dL ≤ d ≤ dU, inconclusive. if 4 - dU ≤ d ≤ 4 - dL, inconclusive.
Otherwise, inconclusive.

JXT
Tutorial question (Theory)
T6 Classical Normal Linear Regression Model
1. Why is it important to know the normality assumption?
With the normality assumption, the probability distribution of OLS estimators can be easily derived. This is
because one property of the normal distribution is that any linear function of normally distributed variables is
itself normally distributed.
OLS estimators are a linear function of µi. Therefore, if µi is normally distributed, so these estimators are also
normally distributed and make our task of hypothesis testing very straightforward.
More importantly, if we are dealing with a small sample, the normality assumption assumes a critical role. In
large samples, t and F statistics have approximately the t and F probability distribution, so that the t and F tests
that are based on the assumption that the error terms are normally distributed can still be applied validly.
T10 Multicollinearity
1. Would you suspect multicollinearity in the estimated regression?
Yes. It is due to high R2 but few significant t ratios.
If R2 is high, in excess of 0.8, the F-test in most cases will reject the hypothesis that the partial slope
coefficients are simultaneously equal to zero, but the individually t-test will show that none or very few of
partial slope coefficients are statistically different from zero.
In this model, R2 = 0.906646 and F = 148.9161 with a P-value of 0.000. this mean that the model is statistically
significant to explain the percentage change (log) of petroleum consumption in the state.
Institutively, we may expect the increase in population may result in an increase in the purchase of motor
vehicle. Hence, the population and the motor vehicle registration in the model might be highly correlated and
cause the problem of multicollinearity.
2. Other than the indicators you used in part (a), what are other ways of detecting collinearity between
two explanatory variables?
High pair-wise correlation coefficient
If an r is high absolute value, then we know that these 2 particular X are quite correlated, and that
multicollinearity is a potential problem.
Some researchers pick an arbitrary number, such as 0.8 and become concerned about the multicollinearity
anytime the absolute value of pair wise correlation coefficients exceeds 0.8.
Tolerance & Variance inflation factor
The coefficient of determination in the regression of regressor of regressor Xj on the remaining regressor (r2)
in the model increase if there is high collinearity. VIF also increase and in the limit, it can be infinite if r2
increase towards infinity.
As a rule of thumb, if the VIF of a variable exceeds 10, which will happen if r2 increases towards unity.
One could use TOLj as a measure of multicollinearity in view of its intimate connection with VIF. The closer
TOJ is to zero, the greater the degree of collinearity of that variable with other regressors and otherwise.
3. Is the high pair wise correlation being good indicator to suspect multicollinearity? (FA famous
question)
No, this is because sometime the linear of the multicollinearity occurs when only we join more than 2
independent variables.
JXT
Tutorial question (Theory)

T11&12 Heteroscedasticity
1. Consequences of getting heteroscedasticity
The OLS estimators for the coefficients are still unbiased and consistent.
This is because none of the explanatory variables is correlated with the error term.
Heteroscedasticity affects the distribution of coefficients increasing the variances of the distributions and
therefore making the estimators of the OLS method inefficient (because it violates the minimum variance
property).
Heteroscedasticity causes the OLS method to underestimate the variances (and standard errors) of estimators
and hence leading to higher-than-expected values of t statistics and F statistics. Therefore, heteroscedasticity
has a wide impact on hypothesis testing: neither the t statistics nor the F statistics are reliable any more for
hypothesis testing we will reject the null hypothesis too often.
T13 & 14 Autocorrelation
1. Consequences of getting autocorrelation in the estimation
The OLS estimators are still unbiased and consistent. Both unbiasedness and consistency do not depend on
the assumption of no autocorrelation. They are inefficient hence they are no longer BLUE. The usual OLS
formula will understate or overstate the variance of the estimator. As the variance of OLS estimators are
understated, the smaller standard error produces a larger t statistic. Some variables that are not
important/insignificant may be considered as significant. The hypothesis testing is no longer valid. An
opposite effect occurs when 𝒗𝒂(𝜷 ̂) are overstated.
2. Types of specification bias
• Omitting a variable
• Irrelevant variable
• Adoption of wrong functional form (Types of the model)
• Error of measurement (X or Y are wrongly measured)

JXT
Past Year (Revision)
September 2020
Section A Q1(b) Interpretation, individually significant, Model significant
(i) Interpretation
β2: If the relative price in two countries increase by 1 unit, on average, the exchange rate will increase by
1.250 CNY/$, holding other variables.

β3: If the interest rate in China increases by 1 percentage point, on average, the exchange rate decreases by
0.86CNY/$, holding another variable constant.

Adjusted R2: 64% of the total variation in the exchange rate of RMB to the US dollar is explained by the
variation in the relative price in the two countries and interest rate in China after taking into account the
degree of freedom.

(ii) Condition 1: Individually significant

H0: β2 = 0

H1: β2 ≠ 0

Critical value: ta/2, n-k-1 = t0.05/2, 21-2-1 = ±2.101

Reject H0 is the test statistic is larger than the upper critical value or smaller than the lower critical
value. Otherwise, do not reject H0.

Test statistic = t = 1.250/0.196 = 6.3776

Reject H0 since the test statistic is higher than upper critical value.

There is sufficient evidence show the ratio of the US Consumer Price Index to the Chinese Consumer Price
Index, the relative price in the two countries is significantly in this model.

(iii) Condition 2: Model significant

H0: β2 = β3 = 0

H1: Not all slope coefficients are simultaneously zero

Critical value: F (0.05, 2, 18) = 3.55

Reject H0 if the F statistic is larger than the critical value. Otherwise, do not reject H0.

𝑹𝟐 /𝒌
Test statistic: 𝐅 = (𝟏−𝑹𝟐 )/(𝒏−𝒌−𝟏) = 𝟐𝟐. 𝟑𝟔

Reject H0 since the F statistic (22.36) is more than critical value (3.55).

There is sufficient evidence to conclude that the model is significant.

JXT
Past Year (Revision)

September 2021
Section B Q1(b) Heteroscedasticity
𝐻0 : There is no heteroscedasticity

𝐻1 : There is heteroscedasticity

Significant value = 0.05

Decision rule: Reject 𝐻0 if the p value is less than significant value (0.05). Otherwise, do not reject 𝐻0 .

Decision: Reject 𝐻0 since the p-value is less than the significant value.

Conclusion: There is heteroscedasticity.

Autocorrelation
𝐻0 : 𝜌 = 0 (No serial correlation)
𝐻1 : 𝜌 ≠ 0 (Serial correlation)
Significant value = 0.05
Critical value = dL = 1.549, dU = 1.616, d = 0.238
Decision rule:
if d < dL, reject H0.
if d > 4 - dL, reject H0.
if dU < d < 4 - dU, do not reject H0.
Otherwise, inconclusive.
Test Statistic:
4 - dL = 2.451
4 - dU = 2.484
Reject 𝐻0 since d (0.238) less than dL (1.549).
There is autocorrelation.

JXT