Dr.
Khalaf Sultan Regression Analysis (Stat 332)
Now
V ar (e ) V ar ((I H )Y ) (I H ) var(Y )(I H )
(I H ) 2 I (I H ) '
2 (I H )I (I H ) '
2 (I H ) MSE (I H ),
where
MSE SSE /( n 2) e ' e /( n 2).
SSE e 'e .
Analysis of Variance Results
122
Dr. Khalaf Sultan Regression Analysis (Stat 332)
hence
SSE Y 'Y b ' X 'Y .
Remark: SSTO=SSE+SSR
Example
For Toluca Company example, find the SSE, SSTO and SSR
123
Dr. Khalaf Sultan Regression Analysis (Stat 332)
1
SSTO Y 'Y (Y ' JY ) 2745173 2437970
n
= 307203
1
SSR=b'X'Y- (Y ' JY ) 2690348-243790=252378
n
Remark:
SSTO=t(y)%*%y-t(y)%*%J%*%y/25
SSR=t(b)%*%t(x)%*%y-t(y)%*%J%*%y/25
Sums of Squares as Quadratic Forms
124
Dr. Khalaf Sultan Regression Analysis (Stat 332)
125
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Recall:
The simple linear regression model and be formed in the matrix form as
Y X ,
E ( ) 0, V ar ( ) 2 I ,
E (Y ) X , V ar (Y ) 2 I ,
where
b0 0
b (X ' X ) 1 X 'Y
b1 1
Y X b X HY , H X (X ' X ) 1 X '
V ar (e ) V ar ((I H )Y ) (I H ) var(Y )(I H )
(I H ) 2 I (I H ) '
2 (I H )I (I H ) '
2 (I H ) MSE (I H ),
126
Dr. Khalaf Sultan Regression Analysis (Stat 332)
MSE SSE /( n 2) e ' e /( n 2).
SSE e 'e
SSE Y 'Y b ' X 'Y .
1
SSTO Y 'Y (Y ' JY )
n
1
SSR=b'X'Y- (Y ' JY )
n
127
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Inferences in Regression Analysis
Lemma
E ()
) MSE (X ' X ) 1
V ar (
Proof
) E (X ' X ) 1 X 'Y (X ' X ) 1 X ' E (Y )
E (
(X ' X ) 1 X ' X B
I
.
This show that the Least square estimate of is an unbiased estimator.
128
Dr. Khalaf Sultan Regression Analysis (Stat 332)
) V ar [(X ' X ) 1 X 'Y ]
V ar (
(X ' X ) 1 X 'V ar (Y )[(X ' X ) 1 X ']'
(X ' X ) 1 X ' 2 [(X ' X ) 1 X ']'
2 (X ' X ) 1 X ' X (X ' X ) 1
2 (X ' X ) 1 I
2 (X ' X ) 1
MSE (X ' X ) 1.
This can be written as
) MSE (X ' X )1
Var (
Or
129
Dr. Khalaf Sultan Regression Analysis (Stat 332)
) MSE (X ' X ) 1
Var (
Example:
For Toluca Company example by matrix methods, we get
) 2 (X ' X )1
Var (
130
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Mean Response
V ar (Y h ) V ar (X 'h b ) V ar [X 'h (X ' X ) 1 X 'Y ]
X 'h (X ' X ) 1 X 'V ar (Y )[X 'h (X ' X ) 1 X ']'
X 'h (X ' X ) 1 X ' 2 [X 'h (X ' X ) 1 X ']'
2 X 'h (X ' X ) 1 X ' X (X ' X ) 1 X h
2 X 'h (X ' X ) 1 X h
MSE (X 'h (X ' X ) 1 X h ).
131
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Example
For Toluca Company example, the variance of the mean of the response
when X=65 can be calculated using the matrix form as
V ar (Yh ) X 'h V ar(b )X h
Which is the same result as that obtained before.
Prediction of New Observation
Proceeding similarly, we get
V ar (Y new ) MSE (1 X 'h (X ' X ) 1 X h ).
132
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Multiple Linear Regression
In This chapter, we generalized the simple linear regression model as
This model can be specialized for different cases as follows
133
Dr. Khalaf Sultan Regression Analysis (Stat 332)
1- Simple linear model when p=2
Y 0 1X 1 ,
2- Model with some Qualitative Predictor Variables
Y 0 1X 1 2 X 2 ,
This model cab be used in different applications such as:
3- Polynomial regression
134
Dr. Khalaf Sultan Regression Analysis (Stat 332)
4- Transformed Variables
5- Interaction Effects
135
Dr. Khalaf Sultan Regression Analysis (Stat 332)
6-Combination of Cases
General Linear Regression Model in Matrix Terms
136
Dr. Khalaf Sultan Regression Analysis (Stat 332)
137
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Estimation of Regression Coefficients
Hence
0
b 0
b
b (X ' X ) 1 X 'Y
1 1
b p 1
p 1
Y X b X HY , H X (X ' X ) 1 X '
138
Dr. Khalaf Sultan Regression Analysis (Stat 332)
The prove is similar to the simple linear model in the matrix form.
Example: Multiple Regression with Two Predictor Variables (Dwaine
Studios)
Dwaine Studios, Inc., operates portrait studios in 21 cities of medium size.
These studios specialize in portraits of children. The company is
considering an expansion into other cities of medium size and wishes to
investigate whether sales (Y)- thousands- in a community can be
predicted from the number of persons aged 16 or younger in the
community (X1)- thousands- and the per capita disposable personal
income in the community (X2 )- thousands. Data on these variables for
the most recent year for the 21 cities in which Dwaine Studios is now
operating are shown below:
city X1 X2 Y
1 68.5 16.7 174.4
2 45.2 16.8 164.4
3 91.3 18.2 244.2
4 47.8 16.3 154.6
5 46.9 17.3 181.6
6 66.1 18.2 207.5
7 49.5 15.9 152.8
8 52 17.2 163.2
9 48.9 16.6 145.4
10 38.4 16 137.2
11 87.9 18.3 241.9
12 72.8 17.1 191.1
13 88.4 17.4 232
14 42.9 15.8 145.3
15 52.5 17.8 161.1
16 85.7 18.4 209.7
17 41.3 16.5 146.4
18 51.7 16.3 144
19 89.6 18.1 232.6
20 82.7 19.1 224.1
21 52.3 16 166.5
139
Dr. Khalaf Sultan Regression Analysis (Stat 332)
# How to read txt file in R
mat <- scan('[Link]')
mat <- matrix(mat, ncol = 3, byrow = TRUE)
X1=mat[,1]
X2=mat[,2]
Y=mat[,3]
n=length(mat[,1])
one=[Link](rep(1, n))
X=cbind(one,X1,X2)
b=solve(t(X)%*%X)%*%t(X)%*%Y
model=lm(Y~X1+X2)
summary(model)
then we obtain
Y 68.9 1.46X 1 9.37 X 2.
The model coefficients can be interpreted as:
1- There is -68.9 of the sales when X1 and X2 are zeros
2- When in population size increases by one unit (thousand), the sales
increases by 1.46 thousand with fixed income.
3- When in income increases by one unit (thousand), the sales
increases by 9.37 thousand with fixed population zise.
140
Dr. Khalaf Sultan Regression Analysis (Stat 332)
Multiple Linear Regression
141
Dr. Khalaf Sultan Regression Analysis (Stat 332)
142