AST 303

Linear Regression Analysis

—Lecture 06: Inferential Issues in Regression

Analysis—

Syed Shahadat Hossain, Ph.D.

shahadat@[Link]

Professor
Institute of Statistical Research and Training
University of Dhaka, Bangladesh

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI and Test of Hypothesis: General structure

Recall: The t-statistic
When X1 , X2 , . . . , Xn are n random samples from a X ∼ N(θ, θ2 ). If θ̂ is a
linear unbiased estimator of θ, we can state
θ̂ − E [θ̂]
t= ∼ tn−1 .
\
SE [θ̂]
Confidence Interval for a parameter θ
Clearly, a 100(1 − α)% CI for θ can be given by
h i
θ̂ − tn−1; β0 SE [θ̂], θ̂ + tn−1; β0 SE [θ̂] .
2 2

Now, consider testing

H0 : θ = θ 0
against Ha : θ < θ0 , Ha : θ ̸= θ0 , or Ha : θ > θ0 .
θ̂ − θ0
under H0 , t = ∼ tn−1
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
\
SE [θ̂]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Inferencial Issues in Regression Model

Recall: Population Regression Model

The Population regression model is expressed by the following equation
Yi = β0 + β1 xi + ϵi , (1)
⇒ E (Yi |Xi ) = β0 + β1 Xi
⇒ V (Yi |Xi ) = V (ϵi ) = σ 2 .

where
ϵi being normally distributed, Now we have for give X , Y1 , Y2 , . . . , Yn ,
a random samples of size n from a Y ∼ N(β0 + β1 Xi , σ 2 ).
We have already obtained the point estimators of model parameters
β0 and β1 using LS method and an estimator of σ 2 using ANOVA
method.
An interval estimator of β0 and β1 can also be generated to describe
and validate the regression model.
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of β0

The t-statistic
We have the LS estimator β̂0 is a linear function of yi s, given by
n
X 1 h i
β̂0 = vi yi , with vi = − x̄wi , E β̂0 = β0 ,
n
 i=1 2
s 
x̄ 2
 
h i
2 1 x̄ h i
2
1
and V β̂0 = σ + Pn 2
⇒ SE β̂0 = σ + Pn 2
.
n i=1 (xi − x̄) n i=1 (xi − x̄)
n
1 X
We also have σˆ2 = (yi − ŷi )2 = MSE
n−2
i=1
v  
u
u
u  
\h i u 1 x̄ 2 
⇒ SE β̂0 = uMSE  + n  .
u
u  n X 2

(xi − x̄)
t  
i=1
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of β1

The t-statistic
We have the LS estimator β̂1 is a linear function of yi s, given by
n
X (xi − x̄) h i
β̂1 = wi yi , with wi =
n , E β̂1 = β1 ,
X
i=1 2
(xi − x̄)
i=1 v
h i σ2 h i u σ2
and V β̂1 = n ⇒ SE β̂1 = u n .
u
X uX
2 2
(xi − x̄) t (xi − x̄)
i=1 n i=1
1 X
We also have σˆ2 = (yi − ŷi )2 = MSE
n−2
i=1
v v
\h i u σ̂ 2 u MSE
⇒ SE β̂1 = u n =u n .
u u
uX u X
2 2
t (xi − x̄) t (xi − x̄)
i=1 i=1
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of Predicted Mean Response

We have the variance of the predicted mean response at a given

(observed) value X = x0 is given by

Ŷ (x0 ) = β̂0 + β̂1 x0

h 2
i
with Var(Ŷ (x0 )) = σ 2 n1 + P(x(x0 −x̄) 2
i −x̄)
h i r h i
2
⇒ SE Ŷ (x0 ) = σ 2 n1 + P(x(x0 −x̄)
−x̄)2
,
i

n
1 X
We also have σˆ2 = (yi − ŷi )2 = MSE
n−2
i=1
\
h i r h 2
i r h i
(x −x̄) (x0 −x̄)2
⇒ SE Ŷ (x0 ) = σ̂ n + P(x −x̄)2 = MSE n1 +
2 1 0 P
(xi −x̄)2
.
i

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of Predicted individual Response

We have the variance of the predicted individual response at predicted
response (Ŷnew ) for a new observation at a specific arbitrary X-value
(xnew ) is

Ỹnew (xnew ) = β̂0 + β̂1 xnew

(xnew − x̄)2
 
h i
2 1
with Var Ỹnew (xnew ) = σ 1 + + Pn 2
n i=1 (xi − x̄)
s 
(xnew − x̄)2

h i
2
1
⇒ SE Ỹnew (xnew ) = σ 1+ + n P 2
,
n i=1 (xi − x̄)
s 
(xnew − x̄)2

ˆ2
h \ i
2
1
with σ = MSE ; SE Ỹnew (xnew ) = σ̂ 1 + + Pn 2
n i=1 (xi − x̄)
s
(xnew − x̄)2
 
1
= MSE 1 + + Pn 2
.
n i=1 (xi − x̄)
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI and Test of Hypothesis for β0 (Cont.)

The t-statistic
β̂0 − E [β̂0 ]
Now, t = ∼ tdf
\
SE [β̂0 ]
β̂0 − β0
⇒t = v   ∼ tn−2 [MSE df is (n − 2)] ,
u
u
u  
u
x̄ 2
1 
uMSE  n + n
u  
X 
(x − x̄)2 
u 
i
t
i=1

 − α)% CI for
Clearly, a 100(1 v β0 can

be given by

u
u
 u  
 u  1 x̄ 2 
β̂0 ± tn−2; α2 uMSE  + .
 u 
 u n X n 
2
(xi − x̄) 
 t  
Syed Shahadat Hossain, Ph.D. shahadat@[Link] i=1
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI and Test of Hypothesis for β0 (Cont.)

In regression analysis, it is often a fundamenta question that whether the
parameters used in the model have significantly non-zero values. The
reason is that, with 0 model parameters, the model does not serve it’s
purpose of explaining the dependent variable using one or more
independent variables. The following hypothesis is, therefore crucial in
regression analysis:
H0 : β 0 = 0
against Ha : β0 < 0 , Ha : β0 ̸= 0 , or Ha : β0 > 0 .
Now, consider testing the above hypothesis, we have under H0 ,
β̂0
t= v
u   ∼ tn−2 .
u
u  
u 1 x̄ 2 
uMSE  + n
u 
 n X 
2
u
t  (xi − x̄) 
Syed Shahadat Hossain, Ph.D. shahadat@[Link] i=1
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI and Test of Hypothesis for β1 (Cont.)

The t-statistic
β̂1 − E [β̂1 ] β̂1 − β1
Now, t = ∼ tdf ⇒ t = v ∼ tdf
\ u n MSE
u
SE [β̂1 ]
uX
t (xi − x̄)2
i=1
v
u n
uX
(β̂1 − β1 )t (xi − x̄)2
i=1
⇒t= √ ∼ tn−2 [MSE df is (n − 2)] ,
MSE
Clearly,a 100(1 − α)% CI for β1 can be given by 
 v v 
 u MSE u MSE 
β̂1 − tn−2; 2 u n , β̂1 + tn−2; 2 u n  .
 α
u α
u 
 uX 2
u X 
2
 t (xi − x̄) t (xi − x̄)
i=1 i=1
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI and Test of Hypothesis for β1 (Cont.)

In regression analysis, it is often a fundamenta question that whether the

parameters used in the model have significantly non-zero values. The
reason is that, with 0 model parameters, the model does not serve it’s
purpose of explaining the dependent variable using one or more
independent variables. The following hypothesis is, therefore crucial in
regression analysis:
H0 : β 1 = 0
against Ha : β1 < 0 , Ha : β1 ̸= 0 , or Ha : β1 > 0 .
Now, consider testing the above hypothesis, we have under H0 ,
v
u n
uX
β̂1 t (xi − x̄)2
i=1
t= √ ∼ tn−2 .
MSE
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI for predicted mean response at a X = x0

With the estimated mean response at a X = x0 ,
Ŷ (x0 ) = E [Y |X = x0 ] = β̂0 + β̂1 x0 ,
we can deduce the following:
The t-statistic
Ŷ (x0 ) − E [Ŷ (x0 )]
Now, t = ∼ tdf
SE\ [Ŷ (x )] 0
Ŷ (x0 ) − Y (x0 )
⇒t = r h i ∼ tn−2 [MSE df is (n − 2)] ,
1 (x0 −x̄)2
MSE n + (x −x̄)2
P
i

Clearly, a 100(1 − α)% CI for Y (x0 ) can be given by

r !
h 2
i
1 (x0 −x̄)
Ŷ (x0 ) ± tn−2; α2 MSE n + P(x −x̄)2 .
i

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

CI for predicted individual response at a X = xnew

With the estimated predicted individual response at a X = xnew ,
Ŷnew (xnew ) = β̂0 + β̂1 xnew ,
we can deduce the following:
The t-statistic
Ỹnew (xnew ) − E [Ỹnew (xnew )]
Now, t = ∼ tdf
SE [Ỹ \ (x )]
new new
Ỹ (x ) − Ynew (xnew )
⇒t = rnewh new i ∼ tn−2 [MSE df is (n − 2)] ,
2 1 (xnew −x̄)2
σ̂ 1 + n + n (x −x̄)2
P
i=1 i

Clearly, a 100(1 − α)% CI for Ynew (xnew ) can be given by

s  !
2
1 (xnew − x̄)2
Ỹnew (xnew ) ± tn−2; α2 σ̂ 1 + + Pn 2
.
n i=1 (xi − x̄)
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

ANOVA F-Test for β1

In regression analysis, testing H0 : β1 = 0 gives the interpretation on

whether the independent variable has any influence on the dependent
variable or not. Only rejecting the H0 indicates that the independent
variable has an influence on the dependent variable. The test is often done
by the t-test described earlier. The test can also be conducted using an
ANOVA F -test. Recall the ANOVA table along with the expected sum of
squares:
Source of d.f. Sum of Expected Mean Expected
Variation d.f. Squares SS SS Mean SS
2 SST
Total n−1 SST (n − 1)σ (n−1) σ2
SSR
Regression 2-1=1 SSR σ2 1 σ2
SSE
Error n−1−1 SSE= (n − 2)σ 2 (n−2) σ2
= n − 2 SST-SSR

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

ANOVA F-Test for β1

Recall:
A sum of squared, independent and identically distributed (i.i.d.) standard
normal variables follows a chi-squared distribution.
Let Z1 , Z2 , . . . , Zn be i.i.d. standard normal random variables,
Zi ∼ N(0, 1).
The sum of their squares, U = ni=1 Zi2 , follows a central chi-squared
P
distribution with n degrees of freedom, denoted as U ∼ χ2n .
A chi-squared distribution is a special case of the gamma distribution,
where χ2n ≡ Γ(α = n/2, β = 2).
The standardized version of the sample sum of squares
n
(n − 1)S 2 1 X
= (Xi − X̄ )2
σ2 σ2
i=1
also follows a central chi-squared distribution with n − 1 degrees of
freedom, where n is the sample size.
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of Predicted Mean Response

We have the variance of the predicted mean response at a given

(observed) value X = x0 is given by

Ŷ (x0 ) = β̂0 + β̂1 x0

h 2
i
with Var(Ŷ (x0 )) = σ 2 n1 + P(x(x0 −x̄) 2
i −x̄)
h i r h i
2
⇒ SE Ŷ (x0 ) = σ 2 n1 + P(x(x0 −x̄)
−x̄)2
,
i

n
1 X
We also have σˆ2 = (yi − ŷi )2 = MSE
n−2
i=1
\
h i r h 2
i r h i
(x −x̄) (x0 −x̄)2
⇒ SE Ŷ (x0 ) = σ̂ n + P(x −x̄)2 = MSE n1 +
2 1 0 P
(xi −x̄)2
.
i

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of Predicted individual Response

We have the variance of the predicted individual response at predicted
response (Ŷnew ) for a new observation at a specific arbitrary X-value
(xnew ) is

Ỹnew (xnew ) = β̂0 + β̂1 xnew

(xnew − x̄)2
 
h i
2 1
with Var Ỹnew (xnew ) = σ 1 + + Pn 2
n i=1 (xi − x̄)
s 
(xnew − x̄)2

h i
2
1
⇒ SE Ỹnew (xnew ) = σ 1+ + n P 2
,
n i=1 (xi − x̄)
s 
(xnew − x̄)2

ˆ2
h \ i
2
1
with σ = MSE ; SE Ỹnew (xnew ) = σ̂ 1 + + Pn 2
n i=1 (xi − x̄)
s
(xnew − x̄)2
 
1
= MSE 1 + + Pn 2
.
n i=1 (xi − x̄)
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

ANOVA F-Test for β1

n
X n
X
With SSR = (ŷi − ȳ )2 , and SSE = (yi − ŷi )2 ,
i=1 i=1
SSE being location standardized sample sums of squares, dividing it
by σ 2 will produce standardized version of the sample sum of squares.
That is, the quantity SSE
σ2
follows a χ2 distribution with n − 2 degrees
of freedom, where n is the number of observations.
If the null hypothesis (H0 ) that ’the regression coefficients (β1 ) is
zero’ is true since in that casee the predictor variables have no effect
on the response. Under H0 , the SSR also becomes a sum of squared,
scaled normal variables. Hence the quantity SSR σ2
follows a χ2
distribution with 1 degrees of freedom.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

ANOVA F-Test for β1

The F-statistic as a ratio of chi-square variables

The F-distribution is defined as the ratio of two independent
chi-square random variables, each divided by its degrees of freedom.
SSR
/dfSSR
σ2
From the results in regression set-up, the ratio SSE
/dfSSE
will follow a
σ2
F -distribution as, under the null hypothesis, each of these SS follows
χ2 distribution.
With the given degrees of freedoms, we can write

(SSR/σ 2 )/1
F = .
(SSE /σ 2 )/(n − 2)

The σ 2 terms cancel out, leaving F = MSR

MSE follows a F -distribution
wuth 1, (n − 2) d.f, where MSR and MSE denote Mean Square
Regression and Mean Square Error, respectively.
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

ANOVA F-Test for β1

The F-statistic as a ratio of chi-square variables

We can re-write the ANOVA table to facilitate the F -test as:

Source of d.f. Sum of Mean F -value Sig.

Variation d.f. Squares SS (p-value)
Regression 1 SSR MSR = SSR
1 F0 = MSR
MSE P(F1,n−2 > F0 )
SSE
Error n−2 SSE MSE = (n−2)
Total n−1 SST

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Least Square Estimation

Example
A pediatrician tested the cholesterol levels of several children. The
readings of weights (in Kg) and cholesterol levels of 10 children are
presented in the following table.
Subject Weight Cholesterol Subject Weight Cholesterol
number in KG (x) levels (y ) number in KG (x) levels (y )
1 16 230 6 23 228
2 17 223 7 24 232
3 18 220 8 24 228
4 19 230 9 25 227
5 21 229 10 28 237

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Least Square Estimation (cont.)

Example (cont.)
We intend to fit a regression of cholesterol levels on weights (in Kg) of the
children. The following computations are done:
Subject Weight Cholesterol x2 y2 xy
number in KG (x) levels (y )
1 16 230 256 52900 3680
2 17 223 289 49729 3791
3 18 220 324 48400 3960
4 19 230 361 52900 4370
5 21 229 441 52441 4809
6 23 228 529 51984 5244
7 24 232 576 53824 5568
8 24 228 576 51984 5472
9 25 227 625 51529 5675
10 28 237 784 56169 6636
Total 215 2284 4761 521860 49205
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Least Square Estimation (cont.)

Example (cont.)
We
P get from Pthe computation:
P 2 P 2 P
x = 215; y = 2284; x = 4761; y = 521860; xy = 49205 .
We compute the regression coefficients:

xy − n1
P P P
x y)
β̂1 = P 2 1 P 2
x − n ( x)
1
49205 − 10 × 215 × 2284)
=  1
= 0.7148
4761 − 10 × (215)2
1X 1X
β̂0 = y − β̂ x
n n
1 1
= × 2284 − 0.7148 × × 215 = 213.03 .
10 10

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Least Square Estimation (cont.)

Example (cont.)
The fitted line is given by

ŷ = α̂ + β̂x = 213.03 + 0.7148x

⇒ Cholesterol
\ level = 213.03 + 0.7148Weight .

For example, estimated (predicted) cholesterol level of a child with weight

21Kg (case number 5) would be
\ level
Cholesterol = 213.03 + 0.7148Weight
= 213.03 + 0.7148 × 21 = 228 .

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Least Square Estimation (cont.)

Example (cont.)
The fitted line can be drawn as:

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

The sums of squares

Example
The sums of squares can be obtained as:
n
( ni=1 yi )2
X P
2
SST = yi −
n
i=1
(2284)2
= 521860 − = 194.4 .
10
" n #
( ni=1 xi )2
X P
2 2
SSR = β̂ xi −
n
i=1
(215)2
 
2
= (0.7148) × 4761 − = 70.7651 .
10

Hence SSE = SST − SSR = 194.4 − 70.7651 = 123.6349 .

Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimated Standard Error of β̂0

n
1 X 123.6349
We have σˆ2 = (yi − ŷi )2 = MSE = = 15.45 ;
n−2 10 − 2
i=1
Pn
x 215
x̄ = i=1 = = 21.5 ;
n 10
n n
( ni=1 xi )2 (215)2
X X P
2 2
(xi − x̄) = xi − = 4761 − = 138.5 .
n 10
i=1 i=1

Hence v
u  
u
u  2
 s
21.52
 
\h i u 1 21.5  1
SE β̂0 = uMSE  + n  = 15.45 10 + 138.5
u  
u n X 2
t (xi − x̄)
i=1
= 7.28 .
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

95% CI for β0

We compute
 from the data, ha 100(1 − α)% CI for β0 as
\i
β̂0 ± tn−2; α2 SE β̂0

= (213.03 ± 2.7515 × 7.28) [As t10−2; .05 = t8;.025 = 2.7515]

2

= (192.99, 233.06) .

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

95% CI for β0

Now, consider testing the hypothesis,

H0 : β0 = 0
against Ha : β0 ̸= 0 .

We have under H0 , β̂0

t=
\ h i ∼ tn−2 .
SE β̂0
Computed value of t is,
β̂0 213.03
t= = = 29.26 [p − value = 0.00000000101] .
\ h i 7.28
SE β̂0

The p-value is very small, we can reject the null hypothesis.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimated Standard Error of β̂1

We have v
r
\h i u MSE 15.45
SE β̂1 = u n = = 0.334 .
u
uX 2
138.5
t (xi − x̄)
i=1
A 100(1 − α)%
 CI for β1 can be given by
\h i \h i
β̂1 − tn−2; 2 SE β̂1 , β̂1 + tn−2; 2 SE β̂1
α α

= (0.7148 − 2.7515 × 0.334, 0.7148 + 2.7515 × 0.334)

[As t10−2; .05 = t8;.025 = 2.7515]
2

= (−0.204201, 1.633801) .

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimated Standard Error of β̂1

The following hypothesis can also be testeds:

H0 : β1 = 0
against Ha : β1 ̸= 0 .
Now, consider testing the above hypothesis, we have under H0 ,
β̂1
t=
\ h i ∼ tn−2 .
SE β̂1
Computed value of t is,
β̂1 0.7148
t= = = 2.14
\ h i 0.334
SE β̂1
[p-value = 0.064776403] .

Since p-value > 0.05 is very small, we can not reject the null hypothesis.
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

ANOVA F-Test for β1

Computed ANOVA Table

The following hypothesis can also be tested using an F -test:

H0 : β1 = 0
against Ha : β1 ̸= 0 .

We can compute the following:

Source of d.f. Sum of Mean F -value Sig.
Variation d.f. Squares SS (p-value)
Regression 1 70.7651 70.7651 45.789 0.00
Error n − 2 123.6349 15.45
Total n−1 194.4

The p-value is very small, we can reject the null hypothesis.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]

AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of Predicted Mean Response

Let us consider the Case number 6, with Weight = 23 kg and Observed
response of Cholesterol = 228 .
A predicted mean response at the given value X = 23 is given by
Ŷ (X = 23) = β̂0 + β̂1 × x0 = 213.03 + 0.7148 × 23 = 229.47
r
\
h i h 2
i
with SE Ŷ (x0 ) = MSE n1 + P(x(x0 −x̄) 2
i −x̄)
s  
1 (23− 215 )2
= 15.45 × 10 + 10
138.5 = 1.3401 .

Hence a 100(1 − α)% CI for Y (x0 ) can be given by

r !
h 2
i
(x −x̄)
Ŷ (x0 ) ± tn−2; α2 MSE n1 + P(x0 −x̄)2
i

= (229.47 ± 2.7515 × 1.3401) = (225.78, 233.16) .

Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Estimating Standard error of Predicted individual Response

Let the predicted individual response (Ŷnew ) for a new observation at a
specific arbitrary X-value (xnew = 20) is of interest.
We haveỸnew (xnew ) = Ŷnew (xnew ) = β̂0 + β̂1 xnew
= Ŷnew (20) = 213.03 + +0.7148 × 20 = 227.326
s
(xnew − x̄)2
 
h \ i 1
with SE Ỹnew (xnew ) = MSE 1 + + Pn 2
n i=1 (xi − x̄)
v " #
215 2
u
u 1 (20 − 10 )
= t15.45 × 1 + + = 4.1528 .
10 138.5
Now, a 100(1 − α)% CI for Ynew (xnew ) can be given by
s  !
1 (x − x̄)2
new
Ỹnew (xnew ) ± tn−2; α2 σ̂ 2 1 + + Pn 2
n i=1 (xi − x̄)
(227.326 ± 2.7515 × 4.1528) = (215.90, 238.75) .
Syed Shahadat Hossain, Ph.D. shahadat@[Link]
AST 303 Linear Regression Analysis —Lecture 06: Inferential Issues in Regression Analysis—

Confidence and Prediction Band

The confidence
Intervals for both
Predicted Mean
Responses and
Predicted individual
Responses can be
drawn on a graph
against different
values of x variable to
show how the interval
gets wider as the
prediction point gets
away from the mean
of x.

Syed Shahadat Hossain, Ph.D. shahadat@[Link]