SSTB031 — COMPLETE STUDY NOTES
Chapter 3: Simple & Multiple Linear Regression + Multicollinearity
Formulas · Tests · Confidence Intervals · Interpretations · Model Selection
1. SIMPLE LINEAR REGRESSION (SLR)
1.1 The Model
Population model:
Yi = β₀ + β₁Xᵢ + εᵢ
Estimated (fitted) model:
Ŷ = b₀ + b₁X
• β₀ = population y-intercept (estimated by b₀)
• β₁ = population slope (estimated by b₁)
• εᵢ = random error term, assumed ~ N(0, σ²)
1.2 Matrix Approach — Estimating Coefficients
Set up matrices:
[ ]
1 X₁
1 X₂
X =
⋮ ⋮
1 Xₙ
[]
Y₁
Y₂
Y =
...
Yₙ
Least Squares Estimator:
b̂ = (X'X)⁻¹ X'Y
Key matrix products to compute:
• X'X → [ ΣXᵢn ΣXᵢ
ΣXᵢ ² ]
• X'Y → [ ΣXᵢYᵢ
ΣYᵢ
]
• (X'X)⁻¹ → invert the 2×2 matrix
⚠️NOTE: For MLR with k predictors, X has columns: 1, X₁, X₂, ..., Xₖ
1.3 ANOVA Table for SLR
Source SS df MS F
F=
Regression SSR = b'X'Y − (1/n)Y'11'Y 1 MSR = SSR/1
MSR/MSE
MSE =
Error SSE = Y'Y − b'X'Y n−2 —
SSE/(n−2)
Total SST = Y'Y − (1/n)Y'11'Y n−1 — —
Key relationships:
SST = SSR + SSE
MSE = s² = SSE/(n−p) where p = number of parameters (2 for SLR)
R² = SSR/SST (coefficient of determination)
📌 INTERPRETATION: R² = 0.82 means 82% of the variation in Y is explained by the regression
model.
1.4 Hypothesis Test — Overall Significance (F-Test)
Hypotheses:
H₀: β₁ = 0 (no linear relationship)
H₁: β₁ ≠ 0 (linear relationship exists)
Test statistic:
F_calc = MSR / MSE ~ F(1, n−2)
Decision Rule: Reject H₀ if F_calc > F_crit(α; 1, n−2)
📌 INTERPRETATION: If we reject H₀, there is sufficient evidence at the α level that a significant
linear relationship exists between X and Y.
1.5 t-Test for Individual Coefficient
H₀: β₁ = 0 vs H₁: β₁ ≠ 0
t_calc = b̂ ₁ / s(b̂ ₁) ~ t(n−2)
s(b̂ ₁) = √[MSE × (X'X)⁻¹₍₁₁₎] (diagonal element of variance-covariance
matrix)
Relationship: t²_calc = F_calc (for SLR only)
📌 INTERPRETATION: Reject H₀ if |t_calc| > t_crit(α/2, n−2). Conclude that X significantly predicts
Y.
1.6 Confidence Intervals for Coefficients
b̂ ᵢ ± t(α/2, n−p) × √Var(bᵢ)
Var(bᵢ) = MSE × [(X'X)⁻¹]ᵢᵢ (diagonal element)
📌 INTERPRETATION: A 95% CI for β₁ = (L, U) means: we are 95% confident the true slope lies
between L and U. If the interval does NOT contain 0, β₁ is significant at the 5% level.
1.7 Interpreting Coefficients (SLR)
📌 INTERPRETATION: b₀ (intercept): The estimated mean value of Y when X = 0. May not be
meaningful if X = 0 is outside the data range.
📌 INTERPRETATION: b₁ (slope): For every 1-unit increase in X, Y changes by b₁ units on
average, holding all else constant.
1.8 Residuals and Studentised Residuals
eᵢ = Yᵢ − Ŷᵢ (ordinary residual)
Studentised residual: tᵢ = eᵢ / [s√(1 − hᵢᵢ)]
where hᵢᵢ = xᵢ'(X'X)⁻¹xᵢ (leverage / hat value)
⚠️NOTE: If |studentised residual| > 2, the observation may be an outlier.
📌 INTERPRETATION: A large studentised residual (|t| > 2 or 3) flags the observation as a potential
outlier in Y. This does NOT automatically mean it is influential.
1.9 Outlier Detection — Confidence Interval for Individual Observation
Ŷ_new ± t(α/2, n−2) × s × √[hᵢᵢ + 1]
⚠️NOTE: If the actual observed value Yᵢ lies OUTSIDE this 99% prediction interval, classify it as
an outlier.
1.10 Influence Measures
Cook's Distance
Dᵢ = (eᵢ²/(p × MSE)) × (hᵢᵢ / (1−hᵢᵢ)²)
OR: Dᵢ = (Ŷ−Ŷ₍ᵢ₎)'(Ŷ−Ŷ₍ᵢ₎) / (p × MSE)
📌 INTERPRETATION: Dᵢ > 1 (or > 4/n as a guideline) suggests observation i has substantial
influence on the estimated coefficients. The larger Dᵢ, the more influential the point.
Leverage (Hat Value)
hᵢᵢ = xᵢ'(X'X)⁻¹xᵢ
High leverage threshold: hᵢᵢ > 2p/n
📌 INTERPRETATION: High leverage means the X-value is far from the mean of X. A high-
leverage point has the POTENTIAL to influence the regression line, but does not necessarily do so.
1.11 Lack of Fit Test
SSE = SSPE + SSLOF
Where:
• SSPE = Pure Error SS (from repeated X values)
• SSLOF = SS Lack of Fit = SSE − SSPE
F_LOF = (SSLOF / df_LOF) / (SSPE / df_PE) ~ F(df_LOF, df_PE)
H₀: Model is adequate (no lack of fit) vs H₁: Model is inadequate
📌 INTERPRETATION: If F_LOF > F_crit, reject H₀ — the simple linear model is NOT adequate for
the data. Consider adding higher-order terms or transformations.
1.12 Residual Analysis Plots
• Normal Q-Q plot / Normal probability plot: Points should fall roughly on a straight line →
normality assumption holds.
• Residuals vs Fitted values: Should show no pattern (random scatter) → constant variance
(homoscedasticity).
• Histogram of residuals: Should be approximately bell-shaped.
• I-chart of residuals: Should show no trends or runs.
📌 INTERPRETATION: If residuals show a funnel shape (variance increases with fitted values), the
constant variance assumption is violated. If residuals show a curve, linearity is questionable.
1.13 Correlation Test
H₀: ρ = 0 vs H₁: ρ ≠ 0
t_calc = r√(n−2) / √(1−r²) ~ t(n−2)
where r = correlation coefficient = √R² (with sign of b₁)
📌 INTERPRETATION: If |t_calc| > t_crit, there is significant linear correlation between X and Y.
The sign of r indicates direction.
2. MULTIPLE LINEAR REGRESSION (MLR)
2.1 The Model
Yᵢ = β₀ + β₁X₁ᵢ + β₂X₂ᵢ + ... + βₖXₖᵢ + εᵢ
Estimated model:
Ŷ = b₀ + b₁X₁ + b₂X₂ + ... + bₖXₖ
In matrix notation:
Y = Xβ + ε → b̂ = (X'X)⁻¹X'Y
2.2 Interpreting MLR Coefficients
📌 INTERPRETATION: b₀: Estimated mean Y when ALL Xᵢ = 0 (may not be practically meaningful).
📌 INTERPRETATION: bᵢ (slope for Xᵢ): For every 1-unit increase in Xᵢ, Y changes by bᵢ units ON
AVERAGE, HOLDING ALL OTHER VARIABLES CONSTANT. This 'holding constant' is the KEY
difference from SLR.
⚠️NOTE: In MLR, the slope values can differ dramatically from simple regression slopes due to
correlations among predictors.
2.3 Sums of Squares
SST = Y'Y − (1/n)Y'11'Y
SSR = b'X'Y − (1/n)Y'11'Y
SSE = Y'Y − b'X'Y = SST − SSR
(1/n)Y'11'Y = nȲ² = (ΣYᵢ)²/n
2.4 ANOVA Table for MLR
Source SS df MS F
p−1 (= MSR =
Regression SSR F = MSR/MSE
k) SSR/(p−1)
MSE =
Error SSE n−p —
SSE/(n−p)
Total SST n−1 — —
Where: p = total number of parameters = k + 1 (including intercept), k = number of predictors
s = √MSE (standard error of estimate)
📌 INTERPRETATION: s = √MSE: On average, the predicted values differ from the actual values
by approximately s units. Smaller s → better fitting model.
2.5 Overall F-Test
H₀: β₁ = β₂ = ... = βₖ = 0 (model has no predictive value)
H₁: At least one βᵢ ≠ 0
F_calc = MSR/MSE ~ F(k, n−p)
Reject H₀ if F_calc > F_crit(α; k, n−p) OR if p-value < α
📌 INTERPRETATION: Rejecting H₀ means at least ONE predictor significantly explains Y. The
overall model is useful.
2.6 Coefficient of Determination
R² = SSR/SST (multiple coefficient of determination)
R²_adj = 1 − [(1−R²)(n−1)/(n−p)] = 1 − [SSE/(n−p)] / [SST/(n−1)]
📌 INTERPRETATION: R² = 0.85: 85% of the variation in Y is explained by ALL predictors together.
R²_adj penalises for adding useless predictors — use this when comparing models with different
numbers of variables.
⚠️NOTE: R² always increases when you add predictors, even useless ones. R²_adj may decrease
when useless predictors are added — so R²_adj is preferred for model comparison.
2.7 Variance-Covariance Matrix
V̂ ar(b̂ ) = MSE × (X'X)⁻¹
From this matrix:
• Diagonal elements: Var(bᵢ) → SE(bᵢ) = √Var(bᵢ)
• Off-diagonal: Cov(bᵢ, bⱼ)
SE(bᵢ) = √[MSE × (X'X)⁻¹ᵢᵢ]
2.8 t-Tests for Individual Coefficients
H₀: βⱼ = 0 vs H₁: βⱼ ≠ 0
t_calc = b̂ ⱼ / SE(b̂ ⱼ) ~ t(n−p)
Reject H₀ if |t_calc| > t_crit(α/2, n−p)
📌 INTERPRETATION: If we reject H₀ for βⱼ, Xⱼ makes a SIGNIFICANT contribution to predicting Y,
GIVEN all other variables already in the model.
⚠️NOTE: Failing to reject H₀ for a variable suggests it adds little additional explanatory power
once other variables are accounted for.
2.9 Confidence Intervals for βᵢ
b̂ ᵢ ± t(α/2, n−p) × SE(b̂ ᵢ)
📌 INTERPRETATION: A 95% CI for βᵢ = (L, U): We are 95% confident the true partial slope lies
between L and U. If 0 is NOT in the interval, Xᵢ is significant at 5%.
2.10 Partial F-Test (Testing Subsets of Variables)
Testing contribution of one variable (e.g., X₁) given X₂ is in the model
SSR(X₁|X₂) = SSR(X₁ and X₂) − SSR(X₂)
SSR(X₂|X₁) = SSR(X₁ and X₂) − SSR(X₁)
F_partial = SSR(Xⱼ | all others) / MSE_full ~ F(1, n−p)
H₀: Xⱼ does NOT significantly improve the model given all other predictors.
H₁: Xⱼ DOES significantly improve the model.
📌 INTERPRETATION: If F_partial > F_crit(1, n−p), keep Xⱼ in the model — it adds significant
predictive power beyond what the other variables already provide.
General Nested Model F-Test (testing a GROUP of variables)
F = [(SSE_R − SSE_F) / (df_R − df_F)] / MSE_F
Where: R = reduced model, F = full (complete) model
Critical value: F(df_R − df_F, df_F)
Hypotheses example — H₀: β₂ = β₃ = 0 vs H₁: at least one ≠ 0
📌 INTERPRETATION: If F_calc > F_crit, the additional terms in the full model ARE significant —
keep them. If not, the reduced model is adequate.
2.11 Prediction / Confidence Intervals in MLR
Point estimate: Ŷₕ = Xₕ'b̂
SE of fit: s.e.(Ŷₕ) = √[MSE × Xₕ'(X'X)⁻¹Xₕ]
95% CI for mean Y given Xₕ: Ŷₕ ± t(α/2, n−p) × s.e.(Ŷₕ)
95% PI for individual Y: Ŷₕ ± t(α/2, n−p) × √[MSE(1 + Xₕ'(X'X)⁻¹Xₕ)]
📌 INTERPRETATION: CI: We are 95% confident the AVERAGE Y for all units with these X values
lies in this interval. PI: We are 95% confident a SINGLE NEW observation's Y lies in this (wider)
interval.
2.12 Coefficients of Partial Determination (r²_Yj·others)
r²_Y1.2 = SSR(X₁|X₂) / SSE(X₂) [contribution of X₁ given X₂]
r²_Y2.1 = SSR(X₂|X₁) / SSE(X₁) [contribution of X₂ given X₁]
📌 INTERPRETATION: r²_Y1.2 = 0.70: Among the variation in Y not already explained by X₂, an
additional 70% is explained by X₁. This measures the UNIQUE contribution of each variable.
3. DUMMY (INDICATOR) VARIABLES
3.1 Definition and Setup
Used to include CATEGORICAL variables in regression. A dummy variable takes only 0 or 1.
• 1 = presence of a characteristic
• 0 = absence (reference/baseline category)
⚠️NOTE: For a categorical variable with c categories, you need c−1 dummy variables. Using all c
dummies causes perfect multicollinearity (the dummy variable trap).
Example: Gender: X₃ = 1 if female, 0 if male
Model: Ŷ = b₀ + b₁X₁ + b₂X₂ + b₃X₃
3.2 Interpreting Dummy Variable Coefficients
📌 INTERPRETATION: b₃ for gender dummy (1=female, 0=male): Holding all other variables
constant, females earn b₃ more/less than males on average. If b₃ < 0, females earn less on
average.
📌 INTERPRETATION: The intercept b₀ represents the estimated mean Y for the REFERENCE
GROUP (dummy = 0) when all continuous predictors = 0.
3.3 Swimming Pool / Yes-No Example
Model: AssessedValue = b₀ + b₁(Size) + b₂(Pool) where Pool = 1 if yes, 0 if no
📌 INTERPRETATION: b₂: Houses WITH a pool are valued b₂ thousand dollars MORE than
houses without a pool, holding size constant.
4. MODEL SELECTION CRITERIA
4.1 Mallows' Cₚ Statistic
Cₚ = SSEₖ/MSE_all + 2(k+1) − n
Where:
• SSEₖ = SSE for the subset model with k predictors
• MSE_all = MSE from the full model with ALL predictors
• k = number of predictors in the subset model
Selection rules:
• Choose models where Cₚ ≈ k+1 (little bias)
• Cₚ >> k+1 indicates substantial bias (important variables missing)
• Cₚ < k+1 is due to sampling error — treat as no bias
• The full model ALWAYS gives Cₚ = k+1
⚠️NOTE: Among models with Cₚ ≈ k+1, prefer the one with SMALLEST Cₚ and fewest predictors
(parsimony).
📌 INTERPRETATION: A model with 2 predictors giving Cₚ = 2.68 ≈ 3 (= k+1 = 2+1) is preferred —
minimal bias and simpler than the full model.
4.2 Adjusted R²
R²_adj = 1 − [(1−R²)(n−1)/(n−p)]
📌 INTERPRETATION: Unlike R², Adjusted R² can DECREASE when unhelpful predictors are
added. Higher R²_adj = better model. Use R²_adj when comparing models with DIFFERENT
numbers of predictors.
4.3 Variable Selection Methods
Backward Elimination
• Start with ALL variables in the model
• Remove the variable with the LARGEST p-value (if > α_remove, e.g. 0.10)
• Refit and repeat until all remaining variables are significant
Forward Selection
• Start with NO variables
• Add the variable that most improves the model (lowest p-value < α_enter)
• Repeat until no remaining variable meets entry criterion
Stepwise Selection
• Combination: variables can enter OR leave at each step
• Stops when no variable can be added or removed based on criteria
📌 INTERPRETATION: Different methods may yield different final models. Always validate the
chosen model. The 'best' model balances fit and parsimony.
4.4 Akaike Information Criterion (AIC)
AIC = n·ln(SSE) + 2p − n·ln(n)
📌 INTERPRETATION: Smaller AIC = better model. AIC penalises for adding parameters,
balancing fit and model complexity.
5. MULTICOLLINEARITY / COLLINEARITY
5.1 Definition
Multicollinearity occurs when two or more predictors in the model are HIGHLY CORRELATED with
each other, making it difficult to isolate their individual effects on Y.
5.2 Consequences
• Inflated standard errors for coefficients
• Unstable (unreliable) coefficient estimates
• Overall model may be significant (F-test) but individual t-tests are not
• Coefficients may have unexpected signs (opposite to theory)
• Large changes in coefficients when predictors are added/removed
• R² may be artificially inflated
⚠️NOTE: Multicollinearity does NOT affect the model's ability to PREDICT Y — it only affects the
INTERPRETATION of individual coefficients.
5.3 Detection Method 1: Correlation Matrix
Threshold: |rᵢⱼ| > 0.8 suggests HIGH correlation between Xᵢ and Xⱼ
📌 INTERPRETATION: If two predictors have |r| > 0.8, they provide largely the same information.
Including both may cause multicollinearity issues.
5.4 Detection Method 2: Coefficient of Multiple Determination (R²ₖ)
For each predictor Xₖ, regress Xₖ on ALL OTHER predictors. Compute R²ₖ.
Threshold: R²ₖ > 0.7 = potential problem; R²ₖ > 0.9 = serious problem
📌 INTERPRETATION: R²ₖ = 0.956 for X₁: 95.6% of X₁'s variation is explained by the other
predictors — severe multicollinearity. Tests on β₁ are unreliable.
5.5 Detection Method 3: Variance Inflation Factor (VIF) and Tolerance
VIF_k = 1 / (1 − R²ₖ) = 1 / TOL_k
TOL_k = 1 − R²ₖ
Statistic No Problem Potential Problem Serious Problem
VIF =1 >4 > 10
Tolerance (TOL) =1 < 0.25 < 0.1
R²ₖ =0 > 0.7 > 0.9
📌 INTERPRETATION: VIF = 22.64 for X₁: The variance of b̂ ₁ is inflated by a factor of 22.64 due to
multicollinearity. This is severe — t-tests for β₁ are not trustworthy.
📌 INTERPRETATION: VIF = 1.009 (from SPSS output): Near 1, so multicollinearity is NOT a
problem for this predictor. The t-test result can be trusted.
5.6 Remedies for Multicollinearity
• Remove one of the highly correlated predictors from the model
• Combine correlated predictors into a single new variable (e.g., index)
• Increase sample size (larger n reduces impact of multicollinearity)
• Redesign the study to avoid correlations between predictors
• Use ridge regression or other regularisation techniques
⚠️NOTE: Dropping a correlated variable may cause specification bias if theory says it should be in
the model.
6. INTERPRETING COMPUTER OUTPUT (SAS / MINITAB /
SPSS)
6.1 Key Output Elements
Output Item What It Means
Root MSE / s Standard error of estimate = √MSE
R-Square Proportion of variation in Y explained by model
Adj R-Sq Adjusted R² (penalised for number of predictors)
Coeff Var Coefficient of variation = (s/Ȳ) × 100%
F Value / Pr > F Overall F-test: Pr < α → model is significant
t Value / Pr > |t| Individual t-test: Pr < α → that predictor is significant
Parameter Estimate The estimated regression coefficient b̂ ᵢ
Standard Error SE(b̂ ᵢ) = used for t-test and CI
VIF Variance Inflation Factor — checks multicollinearity
Tolerance = 1/VIF — checks multicollinearity
6.2 SAS Output Specific
• Corrected Total DF = n−1; Model DF = k (number of predictors); Error DF = n−k−1
• 'Pr > F' < α → overall model significant
• 'Pr > |t|' < α → individual predictor significant
• 95% CL Predict = 95% prediction interval for individual observations
6.3 Residual Diagnostic Plots
• Residuals vs Predicted: random scatter = OK; pattern = problem
• Normal probability / Q-Q plot: straight line = normality holds
• RStudent (studentised residuals) vs Predicted: |RStudent| > 2 → potential outlier
• Cook's D plot: large values → influential observations
• Leverage plot: hᵢᵢ > 2p/n → high leverage
📌 INTERPRETATION: All diagnostic plots should show random, structureless patterns for model
assumptions to hold.
7. AUTOCORRELATION (Serial Correlation)
7.1 Durbin-Watson Test
Tests for first-order autocorrelation in the residuals (time series data):
H₀: No autocorrelation (ρ = 0) vs H₁: Positive autocorrelation (ρ >
0)
DW = Σᵢ₌₂ⁿ (eᵢ − eᵢ₋₁)² / Σeᵢ²
Decision using DW statistic (compare to dL and dU from tables):
• DW < dL → Reject H₀ (positive autocorrelation present)
• dL < DW < dU → Inconclusive
• DW > dU → Fail to reject H₀ (no positive autocorrelation)
• DW ≈ 2 → No autocorrelation
📌 INTERPRETATION: DW = 1.85 (close to 2): Little evidence of autocorrelation — the
independence assumption for residuals appears satisfied.
📌 INTERPRETATION: DW = 0.5: Strong evidence of POSITIVE autocorrelation — consecutive
residuals are correlated. The model underestimates then overestimates in runs.
⚠️NOTE: Autocorrelation invalidates the standard errors and t/F tests. Remedy: use Cochrane-
Orcutt procedure or add lagged variables.
8. QUICK REFERENCE — ALL FORMULAS
8.1 SLR Formula Sheet
Model: Ŷ = b₀ + b₁X
b̂ = (X'X)⁻¹X'Y
SST = ΣYᵢ² − n·Ȳ² OR Y'Y − (1/n)Y'11'Y
SSR = b'X'Y − (1/n)Y'11'Y
SSE = Y'Y − b'X'Y = SST − SSR
MSE = SSE/(n−2); R² = SSR/SST
F = MSR/MSE; t = b̂ ᵢ/SE(b̂ ᵢ)
CI for βᵢ: b̂ ᵢ ± t(α/2, n−2) × SE(b̂ ᵢ)
Studentised residual: tᵢ = eᵢ / [s√(1−hᵢᵢ)]
Cook's D: Dᵢ = (eᵢ²/[p·MSE]) × hᵢᵢ/(1−hᵢᵢ)²
High leverage: hᵢᵢ > 2p/n
8.2 MLR Formula Sheet
Model: Ŷ = b₀ + b₁X₁ + b₂X₂ + ... + bₖXₖ
b̂ = (X'X)⁻¹X'Y
SST = Y'Y − (1/n)Y'11'Y
SSR = b'X'Y − (1/n)Y'11'Y
SSE = SST − SSR = Y'Y − b'X'Y
F_overall = MSR/MSE ~ F(k, n−k−1)
R²_adj = 1 − [SSE/(n−p)] / [SST/(n−1)]
V̂ ar(b̂ ) = MSE × (X'X)⁻¹
t = b̂ ᵢ/SE(b̂ ᵢ) ~ t(n−p)
Partial F = SSR(Xⱼ|others) / MSE
r²_Yj.others = SSR(Xⱼ|others) / SSE_without_Xⱼ
Cₚ = SSEₖ/MSE_all + 2(k+1) − n
VIF_k = 1/(1−R²ₖ); TOL = 1−R²ₖ
AIC = n·ln(SSE) + 2p − n·ln(n)
8.3 Decision Rules Summary
Test Reject H₀ when Conclusion if Rejected
Overall F F_calc > F_crit OR p-
Model is significant (≥1 predictor useful)
(MLR/SLR) value < α
|t_calc| > t_crit OR p <
Individual t That predictor is significant given others
α
That variable adds significant explanatory
Partial F F_partial > F_crit(1, n−p)
power
F_calc > F(df_R−df_F,
Nested F Additional terms in full model are needed
df_F)
Durbin-Watson DW < dL Positive autocorrelation present
LOF test F_LOF > F_crit Model form is inadequate
Significant linear correlation between X
Correlation t |t_calc| > t_crit
and Y
8.4 Multicollinearity Summary
Check Threshold Action if Exceeded
Consider removing one of the correlated
Correlation |rᵢⱼ| > 0.8
variables
> 0.7 (moderate), > 0.9
R²ₖ Multicollinearity is a problem for variable k
(severe)
> 4 (moderate), > 10 Do not trust t-test for this variable's
VIF_k
(severe) coefficient
< 0.25 (moderate), < 0.1 Same as VIF — consider dropping the
Tolerance
(severe) variable
9. STEP-BY-STEP EXAM PROCEDURE GUIDES
9.1 When asked to fit a model using matrix approach
1. Write out the X matrix (column of 1s + predictor columns) and Y vector
2. Compute X'X and X'Y
3. Find (X'X)⁻¹ (given or compute 2×2 inverse: swap diagonals, negate off-diagonals, divide by
determinant)
4. Compute b̂ = (X'X)⁻¹ X'Y
5. Write the fitted equation and interpret b₀ and b₁
9.2 When asked to complete an ANOVA table
6. Note what is given: ΣYᵢ², n, ΣXᵢYᵢ (or SST, SSR, SSE)
7. Compute SST = Y'Y − (ΣYᵢ)²/n
8. Compute SSR = b'X'Y − (ΣYᵢ)²/n
9. SSE = SST − SSR
10. df: Regression = k, Error = n−p, Total = n−1
11. MS = SS/df for regression and error
12. F = MSR/MSE
9.3 When asked to test significance at α
13. State H₀ and H₁ clearly
14. Calculate test statistic (F or t)
15. Find critical value from tables (or compare to p-value)
16. Make decision: Reject or Fail to Reject H₀
17. State conclusion in CONTEXT of the problem
9.4 When asked to check multicollinearity
18. Check the correlation matrix — flag |r| > 0.8
19. For each predictor Xₖ, note the VIF from output (or compute if R²ₖ is given)
20. Check Tolerance = 1/VIF
21. Interpret: VIF > 10 or TOL < 0.1 = severe collinearity
22. Recommend dropping or combining the problematic variable(s)
9.5 When asked about model selection (Cₚ, R²_adj, backward/stepwise)
23. List Cₚ for each model size — identify models where Cₚ ≈ k+1
24. Among those, choose the one with smallest Cₚ and fewest variables
25. Confirm using R²_adj — higher is better
26. Check that remaining variables are individually significant
27. State final model equation with interpretation
── These notes cover ALL tutorial questions in SSTB031 Tutorial 3 & Tutorial 5 (MLR) ──