Quantitative Methods
Analysis of Variance
and Goodness of Fit
Introduction to Linear Regression
Analysis of Variance (ANOVA)
Yi
Y n
Yˆi b̂0 b̂1X i SSE (Yi Yˆi )2
ˆi
Y i1
SST
n
SSR (Yˆi Y )2
i 1
Y
SSE + SSR = SST
n
SST (Yi Y )2
b̂0 i 1
© Kaplan, Inc.
X 1
1
Analysis of Variance and Goodness of Fit
Mean Square Regression and Error
SSR
mean square regression (MSR) =
k
SSE
mean square error (MSE) =
(n – k – 1)
For simple regression, k = 1 (number of independent variables)
SSR SSE
MSR = MSE =
1 (n – 2)
2
© Kaplan, Inc.
Analysis of Variance and Goodness of Fit
ANOVA Table for k = 1
Source of Df Sum of Squares Mean Square
Variation
Regression 1 SSR SSR/1 = MSR
(explained)
Error n-2 SSE SSE/(n-2) = MSE
(unexplained)
Total n-1 SST
These columns sum vertically.
3
© Kaplan, Inc.
2
Analysis of Variance and Goodness of Fit
Using the ANOVA Table for ABC Data: Example
Source of Df Sum of Squares Mean Square
Variation
Regression ? 0.0076 ?
(explained)
Error ?-2 0.0406 ?
(unexplained)
Total ?-1 ?
ABC data: N = 3 years of monthly return observations 4
© Kaplan, Inc.
Analysis of Variance and Goodness of Fit
ANOVA Table for ABC: Solution
Source of Df Sum of Mean Square
Variation Squares
Regression 1 0.0076 SSR 0.0076
(explained)
Error 34 0.0406 SSE 0.0012
(unexplained)
Total 35 0.0482 SST
5
© Kaplan, Inc.
3
Analysis of Variance and Goodness of Fit
The Coefficient of Determination (R2)
R2 measures the percentage of total variation in Y variable explained
by the variation in X (SSR / SST).
An R2 of 0.63 means the variation in X can explain 63% of
the variation in Y.
For simple regression R 2 = correlation2xy
6
© Kaplan, Inc.
Analysis of Variance and Goodness of Fit
R2 for the ABC Regression: Example
Coefficient of determination (R2) = SSR / SST
Total variation (SST) = explained (SSR) + unexplained (SSE)
Given SSR = 0.0076, SSE = 0.0406
SSR 0.0076
0.1577 15.8%
SST 0.0406 0.0076
7 -1
© Kaplan, Inc.
4
Analysis of Variance and Goodness of Fit
Standard Error of the Estimate (SEE)
Standard error of the estimate (SEE) measures the accuracy of
predicted values from the regression equation.
SEE is the standard deviation of the error term.
SSE
S EE = = MSE
(n 2)
Lower the SEE, the better the model fit
8
© Kaplan, Inc.
Analysis of Variance and Goodness of Fit
Standard Error of the Estimate (SEE)
Calculate the SEE based on the ANOVA table for ABC:
0.0406
SEE = = 0.0012 0.035
34
9 -1
© Kaplan, Inc.
5
Analysis of Variance and Goodness of Fit
The F-Statistic
Tests whether the independent variables explain variation in the
dependent variable (i.e., test of overall model significance)
H0: slope coefficient = 0
HA: slope coefficient ≠ 0
One-tailed test (significance is right-hand tail probability)
Reject H0 if F-statistic exceeds critical value
Critical F determined by two degrees of freedom
(numerator and denominator)
10
© Kaplan, Inc.
Analysis of Variance and Goodness of Fit
The F-Statistic
SSR
k MSR
F= =
SSE MSE
(n – k – 1)
with k = 1
numerator df = 1
denominator df = n –2
11
© Kaplan, Inc.
6
Analysis of Variance and Goodness of Fit
The F-Statistic
Perform F-test for 36 observations at 5% significance:
SSR = 0.0076, SSE = 0.0406 H0: b1 = 0 vs. Ha: b1 ≠ 0
MSR = SSR / k = 0.0076
MSE = SSE / (n – 2) = 0.0406 / (36 – 2) = 0.0012
MSR 0.0076 Critical F-value for 1 and
F 6.333 34 df at 5% is 4.13
MSE 0.0012
Test stat 6.33 > critical stat 4.13; therefore, reject H0
12 -6
© Kaplan, Inc.
Analysis of Variance and Goodness of Fit
Regression Coefficient t-Test
A test of H0 : b1 0 vs. Ha : b1 0 is a t-test with (n – 2) degrees of
freedom for a simple regression.
bˆ 1 – b1 estimate hypothesized bˆ 1 – 0 slope
tb1 =
sbˆ standard error sbˆ standard error
1 1
SEE
Standard error of the slope coefficient = sb̂1
Xi – X
n 2
i=1
13
© Kaplan, Inc.
7
Analysis of Variance and Goodness of Fit
Regression Coefficient t-Test
Estimated slope for ABC (b1) = 0.64 with an SE = 0.26
Assuming n = 36, determine if slope is significant at 5% level:
Degrees of freedom (df)
0.64 – 0
tb1 = 2.46 are 36 – 2 = 34
0.26
Critical two-tailed t-stat (34 df Computed stat 2.46 > critical
and 5% significance) is 2.03 stat 2.03; therefore, null is
rejected
14 -4
© Kaplan, Inc.