0% found this document useful (0 votes)
4 views13 pages

Regression Senior Notes

The document provides a comprehensive overview of linear regression basics, including key terms, formulas, and examples for understanding how to predict a dependent variable using an independent variable. It covers the purpose of simple linear regression, the calculation of coefficients, and the interpretation of results, including the significance of regression coefficients through ANOVA and goodness of fit measures. Additionally, it discusses the importance of predicted values, confidence intervals, and different functional forms of regression.

Uploaded by

liguanest12
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
4 views13 pages

Regression Senior Notes

The document provides a comprehensive overview of linear regression basics, including key terms, formulas, and examples for understanding how to predict a dependent variable using an independent variable. It covers the purpose of simple linear regression, the calculation of coefficients, and the interpretation of results, including the significance of regression coefficients through ANOVA and goodness of fit measures. Additionally, it discusses the importance of predicted values, confidence intervals, and different functional forms of regression.

Uploaded by

liguanest12
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Excellent choice ✅

We’ll go module by module — written in Roman numerals (I, II, III, etc.) — just like a proper
CFA-style study summary.

Here’s what I’ll do for you:

 Cover every formula, concept, and example from the PDF.


 Use clear bullet points, definitions, and short notes for memorization.
 Add Professor’s Notes, Examples, and Exam Tips where needed.
 No detail will be skipped.

Let’s begin with Module I (10.1): Linear Regression Basics.

Ⅰ. MODULE 10.1 — LINEAR REGRESSION BASICS


🎯 Learning Outcome (LOS 10.a)

Describe a simple linear regression model, how the least squares criterion is used to estimate
regression coefficients, and interpret these coefficients.

1. Purpose of Simple Linear Regression

 Explains variation in a dependent variable (Y) using one independent variable (X).
 "Variation" means how much values deviate from the mean — not the same as variance.
 Regression answers:
➤ “What explains the fluctuations in Y?”

2. Key Terms

Term Also Known As Meaning


Explained / Endogenous / Variable whose variation is being
Dependent Variable (Y)
Predicted explained
Independent Variable Explanatory / Exogenous /
Variable used to explain changes in Y
(X) Predictor

3. Example
 Predicting stock returns (Y) using GDP growth (X).
→ Stock returns are dependent; GDP is independent.
 Another example:
→ S&P 500 excess returns (X) → explain ABC stock’s excess returns (Y).

4. Scatter Plot & Correlation

 Plot ABC returns (Y) vs. S&P 500 returns (X).


 Example correlation ≈ 0.40 → positive relation.

5. Simple Linear Regression Equation

[
Y_i = b_0 + b_1 X_i + \varepsilon_i
]
where:

 ( Y_i ): actual dependent variable


 ( X_i ): independent variable
 ( b_0 ): intercept (Y-value when X = 0)
 ( b_1 ): slope coefficient (change in Y per unit change in X)
 ( \varepsilon_i ): residual/error term

6. Estimated Regression Line (Line of Best Fit)

[
\hat{Y_i} = b_0 + b_1 X_i
]

 The hat (ˆ) indicates a predicted value.


 Line minimizes Sum of Squared Errors (SSE):
[
SSE = \sum (Y_i - \hat{Y_i})^2
]
 The method that minimizes SSE is called Ordinary Least Squares (OLS).

7. Formulas for Coefficients


[
b_1 = \frac{Cov(X, Y)}{Var(X)}
]
[
b_0 = \bar{Y} - b_1 \bar{X}
]
→ The regression line always passes through the mean point ((\bar{X}, \bar{Y})).

8. Example Calculation

Given:

 Cov(S&P, ABC) = 0.000336


 Var(S&P) = 0.000522
 Mean S&P = –2.70%, Mean ABC = –4.05%

Then:
[
b_1 = \frac{0.000336}{0.000522} = 0.64
]
[
b_0 = -4.05% - (0.64)(-2.70%) = -2.3%
]

Equation:
[
\hat{Y} = -2.3% + 0.64 X
]

If S&P 500 return = –7.8%,


[
\hat{Y} = -2.3 + 0.64(-7.8) = -7.3%
]
Residual = Actual – Predicted = ( 1.1 - (-7.3) = 8.4% )

9. Interpretation

 Intercept (b₀ = –2.3%): Expected return on ABC when S&P = 0.


 Slope (b₁ = 0.64): When S&P excess return ↑1%, ABC’s excess return ↑0.64%.
10. Professor’s Note

 In finance, the slope (b_1) = Beta (β) of the stock.


o β < 1 → less volatile than market
o β = 1 → average risk
o β > 1 → more volatile
 Statistical significance (tested by t-test or F-test) determines if X truly explains Y.

11. Assumptions of Simple Linear Regression (LOS 10.b)

1. Relationship between X and Y is linear.


2. Variance of residuals is constant (homoskedasticity).
3. Residuals are independent (no autocorrelation).
4. Residuals are normally distributed.

12. Common Issues

 Nonlinearity → use residual plots to detect.


 Heteroskedasticity → unequal variance of errors.
 Serial Correlation (Dependence) → residuals show patterns over time.
 Non-normality → affects hypothesis testing validity.
 Outliers → extreme values distort regression fit.

🧠 Module Quiz 10.1

1. Slope = 10 → For each 1-unit change in X, Y changes by 10 units. ✅


2. Least likely assumption: Dependent variable uncorrelated with residuals ❌ (should be
independent variable).

That’s Module I (10.1) fully covered — every key line, example, and formula from your PDF ✅

Shall I move to Module II (10.2 — ANOVA and Goodness of Fit) next?

Perfect 👌
Let’s continue with Module Ⅱ (10.2): Analysis of Variance (ANOVA) & Goodness of Fit —
this part connects regression output with how well the model explains Y.
We’ll cover formulas, interpretations, F-test, t-test, and examples — exactly as in your PDF.
Ⅱ. MODULE 10.2 — ANALYSIS OF VARIANCE
(ANOVA) & GOODNESS OF FIT
🎯 Learning Outcomes

LOS 10.c:
Calculate and interpret measures of fit, and evaluate tests of fit and regression coefficients.

LOS 10.d:
Describe how ANOVA is used in regression, interpret ANOVA results, and compute the
Standard Error of Estimate (SEE).

1. Purpose of ANOVA in Regression

 ANOVA (Analysis of Variance) breaks down the total variation in the dependent
variable ( Y ) into:
o Explained variation (due to regression)
o Unexplained variation (due to residuals)

This helps evaluate how well X explains Y.

2. Key Sums of Squares

Term Symbol Meaning / Formula


Total variation of Y around its
Total Sum of Squares ( SST = \sum (Y_i - \bar{Y})^2 )
mean
Regression Sum of ( SSR = \sum (\hat{Y_i} - \ Variation explained by the
Squares bar{Y})^2 ) regression line
( SSE = \sum (Y_i - \
Error Sum of Squares Unexplained variation (residuals)
hat{Y_i})^2 )

Relationship:
[
SST = SSR + SSE
]

Total variation = Explained + Unexplained variation


3. Mean Squares

Measure Formula Interpretation


Mean Square Regression (MSR) ( MSR = SSR / k ) Explained variation per variable
Mean Squared Error (MSE) ( MSE = SSE / (n - k - 1) ) Average unexplained variation
SEE (Standard Error of Estimate) ( SEE = \sqrt{MSE} ) Standard deviation of residuals

For simple linear regression (k = 1):


→ Degrees of freedom (df):

 Regression: 1
 Error: ( n - 2 )
 Total: ( n - 1 )

4. Coefficient of Determination (R²)

[
R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}
]

 Measures the % of total variation in Y explained by X.


 Example: ( R^2 = 0.63 ) → 63% of variation in Y explained by X.
 For simple regression, ( R^2 = r^2 ) (the square of the correlation coefficient).

5. Example — Using the ANOVA Table

Source of Variation df Sum of Squares Mean Squares


Regression (Explained) 1 0.0076 0.0076
Error (Unexplained) 34 0.0406 0.0012
Total 35 0.0482 —

Compute:
[
R^2 = \frac{0.0076}{0.0482} = 0.158 = 15.8%
]
[
SEE = \sqrt{MSE} = \sqrt{0.0012} = 0.035
]
✅ So, the model explains 15.8% of the variation, with a residual standard deviation of 3.5%.
6. F-Test (Overall Significance of Regression)

Used to test whether the model as a whole explains a significant portion of Y’s variation.

Formula:
[
F = \frac{MSR}{MSE} = \frac{SSR/k}{SSE/(n - k - 1)}
]

Hypotheses:
[
H_0: b_1 = 0 \quad (\text{no linear relationship})
]
[
H_1: b_1 \neq 0 \quad (\text{significant relationship})
]

Decision Rule:
Reject ( H_0 ) if ( F > F_{crit} ) at chosen α level.
It’s always a one-tailed test.

Degrees of freedom:

 Numerator = k (number of Xs = 1)
 Denominator = n – k – 1 = n – 2

7. Example — F-Test Calculation

From the earlier table:

[
F = \frac{MSR}{MSE} = \frac{0.0076}{0.0012} = 6.33
]
df₁ = 1, df₂ = 34

At 5% significance → Critical ( F_{0.05,1,34} ≈ 4.1 )


→ Since 6.33 > 4.1, Reject ( H_0 ).
✅ The slope (b₁) is significantly different from zero — meaning S&P 500 returns do explain
ABC stock returns.
8. t-Test (Significance of Individual Coefficient)

Tests whether the slope ( b_1 ) differs significantly from a hypothesized value (often 0).

Formula:
[
t = \frac{b_1 - b_{1,0}}{SE(b_1)}
]
Degrees of freedom: ( n - 2 )

Decision Rule:
Reject ( H_0: b_1 = 0 ) if (|t| > t_{crit}).

Example:

 ( b_1 = 0.64 ), ( SE(b_1) = 0.26 ), ( n = 36 )


[
t = \frac{0.64 - 0}{0.26} = 2.46
]
From table: ( t_{crit} (df = 34, 5%) = 2.03 )
✅ ( 2.46 > 2.03 ) → Reject ( H_0 ), slope is significant.

🔹 In simple regression, F-test ≡ t-test²


(they test the same null: b₁ = 0)

9. Exam Tips

 ( SEE ) → Lower = better fit.


 ( R^2 ) → Higher = stronger explanatory power.
 ( F )-stat significant → regression model explains Y overall.
 ( t )-stat significant → specific slope (β) is meaningful.
 For CFA Level I, interpretation > calculation.

🧠 Module Quiz 10.2

Question Correct Answer Explanation


1 A df = n – 2 for both parameter & predicted Y confidence intervals
2 A To test intercept significance → ( H_0: a_0 = 0 )
3 C Variation explained by X = Regression Sum of Squares (SSR)
That’s Module Ⅱ (ANOVA and Goodness of Fit) — complete, clean, and exam-focused ✅

Would you like me to continue with Module Ⅲ (10.3 — Predicted Values and Functional
Forms of Regression) next?

Excellent 🌟
Now let’s finish strong with the last part — Module Ⅲ (10.3): Predicted Values &
Functional Forms of Regression.
This one focuses on forecasting (predicted Y values), confidence intervals, and different
regression models (log-linear, lin-log, etc.).

We’ll go point by point — all formulas, interpretations, and examples from your CFA text are
included.

Ⅲ. MODULE 10.3 — PREDICTED VALUES &


FUNCTIONAL FORMS OF REGRESSION
🎯 Learning Outcomes

LOS 10.e: Calculate and interpret predicted values and prediction intervals for a dependent
variable.
LOS 10.f: Describe different functional forms of simple linear regressions.

1. Predicted Value (Forecast)

 The regression equation can be used to forecast Y, given a forecast for X.

[
\hat{Y} = b_0 + b_1 X_f
]
Where:

 ( \hat{Y} ): predicted (forecasted) value of dependent variable


 ( b_0, b_1 ): estimated regression coefficients
 ( X_f ): forecasted value of the independent variable

2. Example — Forecasting
Regression equation for ABC stock:
[
ABC = -2.3% + 0.64(\text{S&P 500 excess return})
]

If forecasted S&P 500 excess return = 10%,


then:
[
\hat{Y} = -2.3 + 0.64(10) = 4.1%
]

✅ Predicted ABC excess return = 4.1%

3. Confidence (Prediction) Interval for Forecasts

We don’t just want a single forecast — we want a range that shows how confident we are in that
prediction.

Formula:
[
\hat{Y} \pm t_{\alpha/2, n-2} \times s_f
]

Where:

 ( t_{\alpha/2, n-2} ) = two-tailed critical t-value at desired confidence level


 ( s_f ) = standard error of forecast

4. Standard Error of Forecast (sₓ)

Variance formula:
[
s_f^2 = SEE^2 \left(1 + \frac{1}{n} + \frac{(X_f - \bar{X})^2}{\sum (X_i - \bar{X})^2}\right)
]

Where:

 ( SEE ) = standard error of estimate


 ( X_f ) = forecasted X value
 ( \bar{X} ) = mean of X
 ( n ) = number of observations
⚠️CFA Level I exams rarely ask you to calculate this — the standard error of the forecast (sₓ)
is usually given.

5. Example — Prediction Interval

Given:

 Regression: ( \hat{Y} = -2.3 + 0.64X )


 Forecast ( X = 10% )
 ( s_f = 3.67 )
 ( n = 36 ) → ( df = 34 )
 ( t_{0.025,34} = 2.03 )

Then:
[
\hat{Y} = -2.3 + 0.64(10) = 4.1%
]
[
\text{Prediction interval: } 4.1 \pm (2.03)(3.67)
]
[
4.1 \pm 7.5 \Rightarrow [-3.4%, 11.6%]
]

✅ Interpretation: We are 95% confident that ABC’s excess return will lie between –3.4% and
+11.6% when S&P 500 excess return = 10%.

6. Functional Forms of Regression

Linear regression assumes the relationship between X and Y is linear.


If it’s not, we can transform variables (using logarithms) to make it linear.

7. Common Functional Forms

Model Type Equation Interpretation of ( b_1 )


Linear–Linear ( Y = b_0 + b_1 X + \ Absolute change in Y for one-unit change in
(Lin–Lin) varepsilon ) X
Log–Linear (Log– ( \ln(Y) = b_0 + b_1 X + \ Relative (%) change in Y for an absolute
Lin) varepsilon ) change in X
Linear–Log (Lin– ( Y = b_0 + b_1 \ln(X) + \ Absolute change in Y for a relative (%)
Model Type Equation Interpretation of ( b_1 )
Log) varepsilon ) change in X
Log–Log (Double ( \ln(Y) = b_0 + b_1 \ln(X) Relative (%) change in Y for a relative (%)
Log) + \varepsilon ) change in X (elasticity)

8. Example of Nonlinear Relationship

Suppose EPS of a company grows 10% annually (nonlinear trend).


Plotting EPS vs. Time gives a curved pattern → violates linearity assumption.

✅ Solution → Apply log transformation:

 Take natural log of EPS → makes the relationship linear.


 Regression: ( \ln(EPS) = b_0 + b_1(\text{Time}) )

This gives a log–lin model, where ( b_1 ) ≈ growth rate.

9. Visual Example (Conceptually)

 Original EPS vs Time: Curved upward (exponential growth).


 Log(EPS) vs Time: Straight line (linear relationship).
→ Now OLS regression can be applied correctly.

🧠 Module Quiz 10.3

Correct
Question Explanation
Answer
90% CI → ( \hat{Y} = 40 ), SE = 2.5, ( z = 1.65 ): ( 40 ± 1.65(2.5) =
1 C
35.9 \text{ to } 44.1 )
Lin–Log model: absolute change in Y for a relative (% log) change in
2 B
X

🧾 Key Concepts Summary

Concept Formula / Description


Regression Equation ( Y = b_0 + b_1 X + \varepsilon )
Least Squares Criterion Minimizes ( SSE = \sum (Y - \hat{Y})^2 )
Coefficient of Determination ( R^2 = SSR/SST = 1 - SSE/SST )
Standard Error of Estimate ( SEE = \sqrt{MSE} = \sqrt{SSE/(n-2)} )
Concept Formula / Description
Predicted Value ( \hat{Y} = b_0 + b_1 X_f )
Confidence Interval ( \hat{Y} \pm t_{\alpha/2,n-2}(s_f) )
Log–Lin Model Relative ΔY / Absolute ΔX
Lin–Log Model Absolute ΔY / Relative ΔX
Log–Log Model Relative ΔY / Relative ΔX

✅ End of Module Ⅲ (10.3) Summary

You now have all three modules (Ⅰ, Ⅱ, Ⅲ) of CFA Level I – Regression Analysis fully
explained and organized — nothing missed.

Would you like me to now make a 1-page condensed revision sheet (cheat sheet) with only
formulas, key points, and exam triggers from all 3 modules?

You might also like