- ....

___ • .4""'•

't-·

INFLUENTIAL OBSERVATIONS
IN LINEAR REGRESSION

by

R. Dennis Cook

Technical Report #282

Department of Applied Statistics

University of Minnesota
Saint Paul, Minnesota 55108

February, 1977

'.j· .··
'~:; ;~~-
')_·
 Abstract

Characteristics of observations which cause them to be important

in a least squares analysis of data arising from a non-designed experiment

are investigated and related to residual variances, residual correlations

and the convex hull of the observed values of the independent variables.

It is shown how deleting an observation can substantially alter an

analysis by changing the partial F-test, studentized residuals, residual

variances, convex hull of the independent variables and the estimated

parameter vector. Outliers are discussed briefly and an example is

presented.
lo INTRODUCTION

Consider the full rank linear regression model

where Y is an [Link] vector .of observators, Xis an nxp full rank matrix

sf known constants,! is a pxl vector of unknown parameters and e.

is an nxl vector of randomly distributed errors such that E(_!) = .Q_

2
and Y._(£,) = l q . Experimental designs for these models are usually

constructed by first specifying the design space (eogo some closed convex

subset of RP) and then choosing the design points within the space so

that, in some optimal sense, the maximum amount of information will be

obtained. For example, D-optimal designs produce coafidence ellipsoids

for ! with minimal volume. The analysis of designed experim~nts .is

usually well-known and straightforward. Also, the design space can be

and usually is ignored during the analysis.

Unfortunately the idea of a design space is completely absent during

most analyses of data arising from experiments in which the selection of

the design points was relegated to Nature. In non-designed experiments

the configuration of the design points in the factor space can have an

important effect on the analysiso It has become increasingly apparent that

failure to examine these configurations can result in severe misinterpre-

tations and loss of informationo

Behnken and Draper (1972) have noted that a wide variation in the

variance of the residuals reflects a nonhomogeneous spacing of the design

points. Box and Draper (1975) suggest that for a design to be insensitive

to outliers the variances of the residuals should be constanto The comments

 - 2 -

of Davies and Hutton (1975) also reflect the opinion that the residual

variances should be examinedo

Cook "(1977) developed a measure based on confidence ellipsoids for

judging the contribution of each data point to the determination of the

least squares estimate of ~-- The measure combines information from the

studentized residuals and the residual variances, and shows that design

points with relatively small residual variances will tend to be the more

importanto Huber (1975) mentioned that small residual variances

typically correspond to "outlying" design points.

A close examination of the residual variances seems necessary in the

analysis of any non-designed experiment. (In many designed experiments

the residual variances are constant). However, the role that such an

examination should play is somewhat vague. Recall that the covariance

matrix of the residual vector,

-
R={r.),
].
from a least squares analysis is

given by

where

,. ,..
R =Y-Y=Y-~

Apart from a proportionately constant, the residual variances are

determined by the design points (ioe. the rows of X). ,We find, it

convenient to refer to the smallest convex set containing all design

points as the independent variable hull (IVH)o Of course, the IVH

and the design space are not in general the sameo

 - 3 -

Generally, in this note we discuss the role that the residual

variances, residual correlations and the IVH play in the analysis

of non-designed experiments. In Section 2 the measure proposed by

Cook (1977) is reviewed, some comments on its use are given, and

relationships between the residual variances and the IVH are

discussed. Consequences of deleting an observation are discussed in

Section 3. Residual correlations and outliers are discussed in Sections 4

and 5. An example is presented.

 - 4 -

2. INFLUENTIAL OBSERVATIONS
Cook (1977) proposed that the importance of the ith data point be

measured by first computing the least squares estimate of ~ with and

without the point an~, second, measuring the distances, D., between the
1.

two estimates as a monotonic function of descriptive levels of significance.

,..
Specifically, let ! denote the least squares estimate of l based on
,..
the full data set and the analogous estimate without the ith
l(--i)
point. Then

( ,.. " ) I I (" "' )

! - ~(-i) ! ! t - !(-i)
D.1. =
..... 2 (1)
ps

2
where s = R'R/(n-p). A large value of D. indicates that the associated
-- 1.

point has a strong influence on the least squares estimate of l· The

magnitude of the distance between A and

.t::..
B,
~-i)
is assessed by comparing

D. to the probability points of the central F-distribution~with p and

1.

n-p degrees of freedom. For example, suppose that D. is equal to the

1.

50% probability point, then the removal of the ith data point moves the

least squares estimate to the edge of the usual 50% elliptical confidence
,..
region for ·f centered at ~:

Let I
v •. = x.(X I
X) "1x. where x.I is the ith row of X.. The
1.l. -l. - - -l. -l.

quantities controlling the measure defined in equation (1) may be seen

in an equivalent form that depends only on the full data set:

2 ,..
t. V(yi)
l.
D.1. = -
p
(2)
v(ri)
 - 5 -

where t.l. = r l../sJf - v..

l.l.
is the ith studentized residual, vc;..)
].
= r/v ].].
..
2
is the variance of the ith predicted value and V(r.) = a (1 - v .. ) is
l. ].].

the variance of the ith residualo Clearly, D. can be large if either

l.
2
ti or V(y.)/(V(r.)) is large. These two components seem to measure the
]. l.

importance of two characteristics of each data point. The

value of t~ depends on the degree to which the ith observation conforms

to the assumed model. It will be shown later that t~ is a monotonic

l.

function of the likelihood ratio test statistic for the hypothesis that

the ith observation is an outlier. Thus, a large ·value of D. arising

l.

because of a large value of t~ is an indication that the ith observation

].

does not conform to the assumed model.

The ratio V(Y.)/V(r.)

]. ].
depends only on the design points and

reflects characteristics of the ith point relative to the IVH.

This ratio is a monotonically increasing function of V• ••

].].
The.
,..
particular characteristics which cause v.. and hence V(y.) /V(.r.)
]. ]. l. ~

to be large may be seen by noting that for all ~ in the IVH (x

need not be a design point)

. I . 1· -1
X (XX)
- - - - <- max. v ..•
X
].
].].
(3)

This follows because the levels of constant value of the quadratic form

on the left of relationship (3) are ellipsoids and the ellipsoid passing

through the design point corresponding to max v..

].].
must contain the

IVH. Expression (3) shows that the point with the largest

variance of a predicted value must lie on the boundary of the IVH

 - 6 -

,...
Thus, large values of V (y . ) /V ( r . ) indicate "outlying" design
1. J.

pointso Of course, the point with the largest prediction variance need not be the

one whose Euclidean distance from the center of the design is the greatest

since the values of ~'(X'X)-lx depend on the density of the points in the

IVH. If x. is any design point with k replicates then

-J

. I I -1 1
X • (X X) x. < -k (4)
-J - - -J -

This is easily justified by first noting that it is certainly true

for k=l and assuming that it is true for k=k0 • It can be shown to be

true for k=k + 1 by using the relationship

0

(X'X )-1 = (X'X + x.x~) -1

-+-+ -- -J-J

(X'X)- 1x.x~(X'X)-l
= (X'X)-1 - - -J J
1
1 + x~(X'X)- x.
-J - - J

where X and ~ are the design matrices for k0 and k0 + 1 replicates

of x .•
-J
Generally we may anticipate that the design point corresponding to
~

max~v .. (max V(y.)/V(r.)) will. lie on the boundary of the IVH in a region
1.1 J. 1.

where the density of the design [Link]··relative low. ·rn non-designed

experiments it is hot uncommon

. .
for max v 1.1.
.• to be close to one.

The influence of a design point can also be seen by considering the

variance of a predicted value conditional on the corresponding observed

value. Letting x denote a design point with k replicated

-r
observations, y , j = 1,k, it can be demonstrated by induction that
r.
J
 - 7 -

V(x; f \Yr . , j = l,k) = cr 2 Vrr (1 - kV

rr
) o

This conditional variance will be small when x lies on the boundary

-r
of the IVH (k v is large) or when x lies in the interior
rr -r
and k is large (v is small). Generally, the remaining points will
rr
have little influence over the predictions around a design point with a

small conditional variance.

Some additional insight about the dispersion of the points within

the IVH may be obtained by inspecting the ordered values, ·v ( ii) .•

Since the ith design point lies on an ellipse whose value is v.. any
l.l.

"large gaps" between the individual elements indicates a corresponding

gap in the spacing of the design pointso A large gap may be taken as an

indication of a region in the design space with relatively inadequate coverageo

Equation (3) also provides a guide to the region in th~ factor·

space where a final model may be appropri_ate for the purpose of predictiono

Prediction at any point ~ for which (3) does not hold may be

tantamount to extrapolation. In this case ~ cannot be in the IVH.

Using this rule it is possi~le to have extreme situations in which the

design points are virtually the only points for which the model is

appropriate for prediction.

The previous discussion relates the properties of Di to the behavior

of its individual components. It is also of interest to relate the

behavior of D. to its components simultaneously. If a point appears to

1

be an outlier (t~ is large) and at the same time lies in a high density
l.
~

region of the IVI-1 (V(y.)/V(r.) is small) then it may be irrelevant whether

l. l.

the point is excluded or not at least as far as estimating f. is concernedo

~ ~

A small value of Di indicates that t and ~(-i) are essentially the

same. The same conclusion may hold if a point lies on the boundary of

the IVH and fits the model extremely well.

 - 8 .,.

3o CONSEQUENCES OF DELETING AN OBSERVATION

The measure discussed in the previous section provides a method for

judging the importance of each observation through the implicit deletion

of the observationso It does not, however, remove the necessity of

actually deleting highly influential observations and examining the new

solutiono When this is done it must be remembered that the removal of

any observation on the boundary of the IVH may affect a considerable

change in the hµll • . Changing the IVH by removing an observation

on its boundary·may cause observations that were previou~ly

judged to be noninfluential to become influentialo Such behavior

indicates that the solution is not stable across the full IVH

This should be cause for concern since the final form of the solution

depends on a few points whose presence or absence was originally left to

chanceo It may be desirable in such c_ircumstances to post-design the

experiment by deleting the observations and confining inference to a

smaller region of the factor spaceo Of course, the influential

observations shoul_d not be completely forgotten but should perhaps be the

subject for future investigationo Our main contention is that the use of

such observations without an independent verification of their authenticity

or a well grounded: firm belief in the model i~ not a·sound practice.

3ol Distance Measures

At present, sequential deletion of observations seems the most

expedient method for detecting multiple influential pointso In what

follows we examine the effects of deleting a single influential pointo

Let D.J ( -l.• ) denote the value of the distance measure for the jth point

based on the data set from which the ith point has been removed, i~j.
 - 9 -

The characteristics of D. ( . ) seem best understood by expressing it in

J '""l.
terms of the full data seto

Let
. -1
V. . = X~ (X 1 X)
X.
l.J -1. -- - -J
and

)-1
wki, - ~ ~{-i)!(-i)
- '( I I
~J,

where

x ' . x . = -x 'x- -
-( -1.)-(-1.)
x. x ~
-1.-1.
2
and x' is the·rth row of X. Also, let 8 (-i) denote the usual
-r
2
estimate of a based the data set with the ith point removedo

The following relationships will be useful:

v .. =w • ./(1 +w .• ) (5)
l.J l.J l.l.
2
v .. = w .• - w• ./(1 + w •• ) (6)
JJ JJ l.J ].].
,._ ,._ I -1 I"°
A-A( ")=(XX)
~ .i::. -1. - -
x.[y.]. - -1.-
-1.
x.~)/(1 - v ].].
.. ) (7)

2 2 ,,.. 2,
(n-p)s = (n-1-p)s ( -1.") + (y.]. - -1.~
x.A),(1 - v ].].
.. ) o (8)

Expressions (5) and (6) are easily verified using the identity

, -1 , ·, -1
(!'X)-1
' -1 <!c -i)!(-i)) ~i~i <!c -i~(-i) >
= (X(-i)~-i)) 1 + wl.1.
••

Expression (7) was shown by Cook (1977) and expression (8) was shown by

Beckman and Trussell (1974)0 The derivations are straightforward and will

not be repeated hereo

 - 10 -

From equation (2),

2
DJ"( -J.• ) = t.( .)w . ./p(l
J - i JJ
- w •. )
JJ
(9)

2
where tJ·c .)-i
denotes the studentized residual for the jth point in the

data set with the ith point removed. We now relate equation (9) to the

full data set by dealing with t~( ") and w .. separately. Consider
J -J. JJ
first w .. : Using equations (5) and (6) it is easily verified that
JJ

2
V • • = W• • .,. p •• (1 - V •• )
JJ JJ J.J JJ

where denotes the correlation coefficient between the ith and jth
p ••
J.J
residuals in the full data set. It follows that

2 2
v .. (1 - p •• ) + p ••
w • ./(1- w •• ) =
]] J.J J.] (10)
2 0
JJ JJ (1 - v .. )(l - p •• )
JJ J.J

Clearly, this ratio will be large if either is large.

A large value of v.. would have been, detected in the analysis of the
JJ
full data set. Thus, if the variance of the jth predicted value increases

substantially when the ith observation is deleted it must be due to a large

correlation between the ith and jth residuals in the full data set. We

see also that if the residual correlations are negligible then the variances

of the predicted values will remain essentially unchanged when any point

is deleted.

Next,

(y. - x~~( .))2

2 = J -1 _-1._
t.J ( -J.. ) 2
s( .)(1 - w •• )
-1. JJ
 - 11 -

Using the results in equations (5) - (7) a little algebra will verify that

2 2 2 2
t.J ( -1.. ) = s [t. p •• t.] /s( .)(1
J.J 1. col.
J

where t. and t. are studentized residuals from the full data seto Using
1. J
equation (8) this reduces to

2
2
(n - p - l)(t. - p •• t.)
1 1.J 1.
(11)
J -1..) =
t.( 2 2
(n - p t. )(1 p •• )
]. 1.J

Notice that if the residual correlations are negligible then the

deletion of any point with a studentized residual larger than one will

cause all remaining studentized residuals to increaseo

Finally, combining (10) and (11) we have,

2 2 2
(n - p - 1) [ t. ""' p •• t.] V •• ( 1 - p •• ) + p 1.· .
= J 1.1 1. 11 · 1.1 1_ (12)
D.J ( -1.• ) 2
(n - p - ti)(l - p~j) (1 - vjj)p

The rather complicated form of this expression makes definite

predictions concerning its behavior difficulto Two general observations

seem worthy of mention, however: If the residual correlations are all

2
negligible then D.( ") '2= D.
J -1. J
for all j ~ [Link] t.
l. -
> lo Also, a

large residual correlation can cause D.( .) to increase substantiallyo

J c:oJ.

It is clear from the previous discussion that residual correlations

can play a substantial role in the isolation of influential observationso

It seems natural to question those characteristics of two observations which

cause their residuals to be highly correlatedo This will be considered in

Section 4o
 - 12 -

3o2 Partial F-Tests

Partial F-tests for the hypothesis that the individual coefficients

of ! are zero are commonly used to simplify the original model. When

using this procedure it is not uncommon to find that whether a particular

coefficient is retained in the model depends on the presence of a single

observation. This behavior seems particularly prevalent when the model

contains polynomial terms.o

Let ~k denote the kth component of ! and define

where dk is the kth diagonal element of (!_'X)-lo The partial

F-statistic, Fk, for the hypothesis that ~k is zero can be expressed as

,..
Further, let ~k(-i), Tk(-i), dk(-i), and Fk_(-i) denote the analogous
quantities based on the data set without the ith observationo

The factors controlling the behavior of the partial F-statistics can

be seen by expressing F~(-i) in terms that depend only on the complete

data set. We consider the three components comprising Fk(-i)

separately: Using equation (7),

sk(
I"'
") = sk
-]. I"'
- ck.r./(1
1. 1. - v 1.1.
.. )

where

C = e'(X 1X) ""'lx.

ki -k - - -l.
 - 13.,.

and ..=k is a pxl vector with a 1 in the kth position and zeros elsewhereo

Also,

2
dk(-i) = dk + cki/(l - vii) 0

2 derives immediately from equation (8).

The desired expression for s(-i)
After substituting these three forms into

"2 2
Fk(-i) = ~k(-i)/ 8 (-i)dk(-i)

a little algebra will verify that

2
Tk V.. \ 2
(n-p-l)t. l t.i
l.
-v <r-~;.:) 1
(12a)
Fk(-i) - 2 2
(1 +y v . ./(1-v .. ))
(n-p-t.) J.l. l.l.
l.

,... I ,..
where y denotes the correlation between ~k and x. ~.
-1. -

Recall that v .. /(1-v .. ) appears in the expression of the distance

l.l. l.l.

measure, D., and will be relatively large for points on the boundary of
l.

the IVH. The term (n-p-l)t~/(n-p-t~)

l. 1.
is monotonic in t~]. and

may be used to test whether the ith observ~tion is an outlier. In

fact, under the null hypothesis it is distributed as an F random variable

with 1 and n-p-1 degrees of freedom. (This will be discussed in more

detail in Section 5.)

It seems clear from inspection of equation (12a) that almost

anything can happen to the partial F-statistics when an observation is

removed. Two general observations seem particularly interesting, however:

Suppose that the deleted observation appears to be an outlier (t~ is

1.
~
2
large) and that y (v •. /1-v .. ) is negligibleo The latter supposition
l.1. 1.l.
 - 14 -

would hold when y is small or the deleted observation lies in a dense

region of the design space. (Empirical investigations indicate that

typically y is not negligible by itself.) In this case,

Thus, deleting an observation with ·t~ > 1 in a dense region of the

l.

design space will tend to increase all paitial F-statisticso

Next, consider the deletion of a point that fits the model quite
2
well (t.
].
<
- 1). In this case,

. \ 2
[Tk - yt. (v . ./1-v .. ) ]
~ ]. l.l. l.l.
2
1 +y V• ./1-v ..
u. 11.

Using this approximation it is easily verified that

2
•
[VTkv . ./(1-v •. )
l.l. .].].
+ t.]
l.
t~].
1 + V 2v l.l.
. ./1-v ..
l.l.

Thus, we can generally expect all partial F-statistics greater than one

to decrease when a conforming point on the boundary of the design spac~

is deletedo
 - 15 -

4. RESIDUAL CORRELATIONS
2
The squared correlation coefficient, p •• , between the ith and jth
l.J

residuals can be expressed as,

2 2
p • • = v. ./ ( 1-v .. )( 1-v. . ) o (13)
l.J l.J l.l. JJ

2
To investigate the causes of a large value for p •• , we shall hold the
1.J
jth design point fixed and find how to choose the ith design point so

that the correlation between the associated residuals is maximized.

2
Specifically, we consider sup p.. where the supremum is to be taken
1.J

over sowe convex subset of the factor space that consists of all

permissible values for the ith design pointo The required calculation
2
is facilitated by writing p •• in terms of explicit quadratic forms in
l.J

X. o Using equations (5) and (6) it is easily verified that

-1.
2
w•.
2 l.
Pij = 2
(14)
( 1-+w •.• ) ( 1-w . j ) + w. •
1.1. J 1.J

2
w. . and w •• are quadratic forms in
-1.
x. while w •• is independent
1.J 1.l. JJ
of x. and may be considered constanto If the model contains a constant
-1.

term the first component of x. is constrained to be 1 and the

-1.

supremum must be taken with respect to the last p-1 components of x. O

-1.

Let x; = (1, z;) and, assuming that the independent variables in the

reduced data set are measured around their means,

1
0
/
(X{-i) X(-i))
-1
= ( n:1 A )·
 ... 16 -

The correlation may now be expressed in a more manageable form by

substituting

w ••
J.J
=--
1
n- 1
+z!Az.
-J -1. -

into equation (14) •.

The largest possible value for p7.J.J will obviously depend on the

subset of the factor space over which the supremum is takeno Lacking

definite guidance we choose a subset that seems reasonable and is, at

least, expedient: For an arbitrary positive constant c, the supremum

will be taken over G(c) = {.~\.~'~ ~ ~ c}. Of course, c may be chosen

so that G contains the IVH corresponding to the reduced data

seto From .equation (14) it is not difficult to see that the supremum
2
must be obtained on the boundary of G(min[c, n (w .. - 1/(n-l))])o It
JJ
follows that

c' Q~.
2
sup Pij = (15)
[Link]
-l.
( 1-W .. ){ 1+ _1.._l
JJ n-
+ C
1
) + C
I
Q:JJ.
where

1 1 ~
Q .• =
JJ
+ ( w .. -
JJ
-)2
n-1
<'~-l)Jc I

and

2 1
c' = min(c, n (w .. - )) o
JJ n- 1
 - 17 -

This value is attained at

r
z. = z.(c'/(w .. - n-l)) o
\ If the model does
-1. -J JJ
not contain a constant term the analogous expression is obtained by

setting c'=c and l/n-1 to zeroo

This result shows that the higher correlations will arise

between points on the boundary of the IVH and proportional

points in the interioro Moreover, since equation (15) · is monotonically

increasing in w •• we see that the highest correlations should occur

JJ
between replicated design points on the boundary of the IVH.

This observation confirms a conclusion reaching in Section 3; namely,

if ona of two highly correlated observations is deleted· the·

remaining observation is likely to become extremely important.

If one of two replicates of a design point on the boundary of the

IVH is deleted the. remaining point will stand alone and,

thus, may become extremely important.

The previous discussion shows the general relationship between

points whose residuals are the most highly correlatedo It may be,

however, that the highest possible correlation is quite smallo If n is

2 then a convenient approximation for
large and c' = n (w .• - 1/n.. 1)
JJ
equation (15) is ·.

2
sup p •• ~ w. . •
[Link] l.J JJ
-1.

Since equation (15) is monotonically increasing in c ~ c',

 - 18 -

for all io This in combination with equation (4) justifies the

previous comment. Equation (16) may be used as a rough guide to the

behavior of the correlations between any influential point and all

other pointso

Finally, it is worth noting that the correlations between

residuals corresponding to replicated design points are ~v].].

.. /1-vJ.l.
..•
 - 19 -

5. OUTLIERS

Outliers play an important role in the concept of influential

observationso The approaches used in the detection of outliers are

·manifold and no pretence of an exhaustive presentation is madeo In

this section we consider briefly the problem of detecting multiple

outlying points as it relates to the previous discussiono

Consider the model,

.!_=!i+!+.!-

All quantities are as previously defined except that ! is a vector

consisting of n-1, zeros and .e, < n-p unknown parameterso Without

loss of generality we may assume that we wish to test the last .e,

.observations as outliers. The unknown parameters in e will now occupy

the last .e, positions. Generally, we are concerned with estimation of

e and the test of H:0 = Oo The development is greatly facilitated

by reparameterizing the model to obtain the following form:

! = ! ~ + Jze ~ + .! (17)

-1
where ct = (~'X) !1 (X J?.. + ~)

T = (I
....... - ____ X}-l -X')
X(X 1........

z = (o(n-.e,)x1,)
I.e,x.e,

and !.e, is the i,xl vector consisting of the last .e, components of 80
 - 20 -

Since X and T are orthogonal the least squares estimate of ~t

can be written as,

,..
ft = (Z'T'Tz)-l
--~ --- z'T'Y

= (f '.'.!.__)-1 Z'TY. (18)

It can be shown that Z 1TZ is positive definite as long as [X'f!l

,..
is of full rank. Alternatively, !t may be written as

,.. -1
!t = (!_t) RJ,

where T.e, is the submatrix of T consisting of the last t rows and

columns, and !t is a vector of the last t components of Rt residual

vector from the model with e = O.

,..
The reduction in the sums of squares due to fitting ft is

/ -1
R:t T.e, R:t (19)

and the usual normal theory F-statistic for the hypothesis

H:8:t = 0 is
. alR
Rif.J. --'J, -1 n-t-p
FJ, = R;R - l',,r,, RJ, t
---
The dependence on the studentized residuals and residual correlations

can be seen by writing Ft in the alternate form,

 - 21 -

(20)
-1
n-p-t!o t
-lll-L -t

where ti and .e_L are the matrices of studentized residuals and residual
correlations for the last L observations. For L ·= 1 we have

which is clearly a montonically increasing function of the last

studentized residual,

The above presentation is conditional on the!_ priori specification

of the observations to be tested. It is perhaps·more common to ask for

the L most likely outlying points. When this is the case the quadratic form in

equation (19) must be computed for all possible combinations of

points. The combination producing the maximum value is then chosen to

be tested using the· statistic

(21)

which, of course, has the distribution of the maximum of

correlated F-random variableso It is important to notice that the two

most likely outlying points will not, in general, correspond to the two

points with the largest studentized residualso The residual correlations

can produce two seemingly unlikely candidates as the most outlying

valueso

When L ~ 2 and J?_L is strongly diagonal the well-known

approximation
 C3 22 -

-1
.e...e, :!= 21 - .e.1,

can be used to display the effects of the residual correlations in

determining the critical points: Substituting this approximation

into equation (19) we obtain,

t. t .p •.
]. J l.J

where the summations are over those observations in the subset in

question.

In short, if two or more outlying values are suspected the residual

correlations should be inspected. The observations, if any, which are

disturbing the analysis may not be the ones with the larger

studentized residuals. Points on the boundary of the IVH will

generally be associated with higher residual correlations. At the

very least, it seems wise to give these points special attention.

 - 23 -

6. EXAMPLE

Daniel and Wood (1971) considered a set of data on the oxidation- of

ammonia to nitric acid. The original data set is from Brownlee (1965) and

consists of 21 observations with three possible explanatory variables. After a

reasonably extensive analysis, Daniel and [Link] that 4 observations

(1, 3, 4 and 2l)·were-outliers and that one of the explanatory variables was

not needed. Their final [Link] a linear and quadratic term for one

explanatory variable, a linear term for the other and was based on 17

"valid" observationso

In this example we adopt the final model of Daniel and Wood but

include all data points. For ease of reference the data set has been

given in Table lo The general purpose of this example is to illustrate

selected results of the previous sections by considering 6 selected

subsets of the data from Table lo Tables 2, 3, and 4 give the values

of D.,
].
v ].].
.. , and t.,
].
respectively, for the six data sets. Table 5

gives the estimated coefficients, partial F-statistics, and mean

square error for each data seto Note that the last data set used in

each table consists of the observations that Daniel and Wood judged

valid.

Consider first the complete data set. Inspection of the first

column of Table 2 reveals that observation 21 is the most influentialo

Removal of this observation would move the least estimate of ~ to

,.
the edge of a 40% confidence ellipse for ~ based on ~· The reasons

for this importance can be obtained from Tables 3 and 4. The four

largest v .. values are,

l.l.

Observations 1 and 2 lie on the edge of the IVH

 - 24 -

and are replicates. The spacing of these values indicates two gaps in

the coverage of the design space. Observation 21 has the third largest

value of v .. and, thus, lies near the edge of the IVH.

11

Moreover, from Table 4 we see that it has the largest studentized

residual. Using Lund's (1975) tables of critical values we see that

observation 21 may be declared an outlier at the 0.1 level but not

at the 0.05 level of significance. Thus, there is some evidence to

suggest that it is an outlier, although it is not overwhelming.

We are now faced with the decision to declare observation 21 an

·outlier or accept the data set as it stands. It must be remembered

that when inspecting the studentized residuals we are implicitly assuming the

existence of at most one outlier, (i.e •. the vector e in equation (16)

has at most one nonzero component). The two most likely candidates as

outliers are found, using the quadratic form in equation (19), to be

4 and 21. These observations also have the two largest studentized

residuals. The value of the statistic in equation (21) is

max F = 2.82 which is about the 90% probability point of the

2
F-distribution with 2 and 15 degrees of freedom. Since this probability

point is an upper bound for the probability point of the distribution of

max F , we cannot justify the simultaneous deletion of observations 4

2
and [Link] significance levels alone.

In this example, we shall treat observation 21 as an outlier. Now,

the conditional hypothesis H:ai = O\e 21 ~ 0 might be of interest. It

seems intuitively obvious and is easy to show that the tests of this

hypot~esis is the same as the test of H:e. = 0 based on the data set
1
 - 25 -

with observation 21 removed. The second column in each table shows the

results after the removal of observation 21. Note that now observation 4

appears to be an outlier although it is not the most influential,

~

max Di= n2 = 0.593. A comparison of the first two columns in Table 3

shows that all values of v .. increased when observation 21 was

ii

deleted. Of course, this was predicted by equation {lO)o Also, the

values in the second column of Table 3 may be taken as upper bounds

on the residual correlations P!, 21 (i = "i, 20) in the full data seto

Next, considering the data set·with observations 4 and 21 deleted,

we find ourselves in a situation similar to that of the full data seto

Observation 2 is the most influential, n = lo33o Removal of this

2
observation will move the latest estimate of ~ to the edge of a 70%

confidence ellipseo The reasons for this importance are that observation 2

lies on the edge of the IVH and appears to be an outlier. (It

may be rejected at the Oo05 level.) From Table 5 we see that now the
2 A
coefficient of x , ~ , is highly significanto The increase in the
1 3
A
partial F-statistic for ~ is due to the large value of t and a
3 4
A A
small value of the correlation between ~
3
and ~~ (y = 0957) in

the data set with only observation 21 deleted.

Recall that observation 2 is very influential and is one of two

replicates (1 and 2) on the edge of the IVH. The residual

correlation between observations 1 and 2 is p

12 = -0421/(1 - .421) = -00727.
Thus, when entertaining the removal of observation 2 we should anticipate

that the characteristics of the IVH may change considerably and

that observation 1 will become more influentialo Inspection of the fourth

column in each table shows this change. Notice that now the gaps .. in the

spacing of the design points have widened considerablyo

 - 26 -

In the latest data set (2, 4, and 21 deleted) design points 1 and

3 are approximately proportional and thus have a very high residual

correlation, p = -Oo988o (If the third design point were replaced

13
with a replicate of the first the residual correlation would increase to

-00993.) This high residual correlation suggests that we should anticipate

extreme changes if observation 3 were deletedo ~he fifth column in each

table shows the results of deleting observation 3o

For comparison, we have included in the last column of each table

the results based on the subset of the data that Daniel and Wood judged

valido Notice that observation 2 is extremely influentialo The removal

of this observation would move the least squares estimates beyond the

edge of a 99.95% confidence ellipseo This observation fits the model

quite well and is important because it stands alone on the edge of the

IVH. From this we can anticipate [Link] it were removed all

partial F-statistics would decreaseo In fact, when observation 2 is

removed the first three partial F-statistics are all less than one while

F decreases but remains fairly largeo It appears that for the final
4
data set of Daniel and Wood the quadratic term is needed to model a single

observation.
 REFERENCES

[1] Beckman, R.J. and Trussell, H.J., (1974). The distribution of an

arbitrary studentized residual and the effects of updating
in multiple regression. Jo Amero Statisto Assoc. 69, 199-2010

[2] Behnken, D.W. and Draper, N.R., (1972). Residuals and their variance
patterns. Technometrics 14, 101-lllo

[3] Box, [Link] and Draper, N.R., (1975)0 Robust designo Biometrika
62, 347-3520

[4] Cook, R.D., (1977). Detection of influential observations in

linear regression. Technometrics 19, 15-18.

[5] Daniel, Co and Wood, F.S., (1971)0 Fitting Equations to Data.

John Wiley & Sons, Inco, New York.

[6] Davies, R.B. and Hutton, B., (1975)0 The effects of errors in the
independent variables in linear regression. Biometrika g,
383-3910

[7] Huber, P.J., (1975). Robustness and designs. A Survey of

Statistical Design and Linear Models, North-Holland, Amsterdam.

[8] Longley, J.W., (1967)0 An appraisal of least squares programs for

the electronic computer from the point of view of the usero
J. Amero Statist. Assoc. g, 819-8410

[9] Lund, R.E., (1975)0 Tables for an approximate test for outliers
in linear models. Technometrics 17, 473~476.
 TABLE 1

DATA ON THE OXIDATION OF AMONIA

TO NITRIC ACID

Observation Air Flow (Air Flow) 2 Cooling Water Stack

Number 2
Inlet Temperature Loss
xl xl x2 y

1 80 6400 27 42
2 80 6400 27 37
3 75 5625 25 37
4 62 3844 24 28
5 62 3844 22 18
6 62 3844 23 18
7 62 3844 24 19
8 62 3844 24 20
9 58 3364 23 15
10 58 3364 18 14
11 58 3364 18 14
12 58 3364 17 13
13 58 3364 18 11
14 58 3364 19 12
15 50 2500 18 8
16 50 2500 18 7
17 50 2500 19 8
18 so 2500 19 8
19 50 2500 20 9
20 56 3136 20 15
21 70 4900 20 15
 TABLE 2

Distance Measures, D., Based on Selected

l..
Subsets of Observations From Table 1.

Observations Deleted
Observation None (21) (4,21) (2 ,4, 21) (1,2,4,21) (1,3,4,21)
1 0.162 0.107 0.210 2.042 * *
2 0.193 0.593 1.331 * * 12.175
3 0.125 0.123 0.333 0.403 21.160 *
4 0.304 0.539 * * * *
5 0.003 0.012 0.003 0.006 0.006 0.001
6 0.021 0.037 0.019 0.031 0.033 0.021
7 0.042 0.050 0.007 0.010 0.008 0.003
8 0.014 0.014 0.010 0.023 0.034 0.056
9 0.043 0.040 0.010 0.008 0.002 0.028
10 0.028 0.008 0.013 0.030 0.049 0.051
11 0.028 0.008 0.013 0.030 0.049 0.051
12 0.062 0.009 0.002 0.009 0.019 0.028
13 0.001 0.044 0.107 0.177 0.190 0.213
14 0.001 0.019 0.034 0.054 0.055 0.068
15 0.002 0.007 0.005 0.005 0.002 0.006
16 0.002 0.001 0.007 0.019 0.035 0.027
17 0.004 0.000 0.000 0.001 0.003 0.003
18 0.004 0.000 0.000 0.001 0.003 0.003
19 0.008 0.001 0.008 0.011 0.007 0.006
20 0.008 0.010 0.037 0.078 0.117 0.088
21 0.699 * * * * *
 TABLE 3
Values of v .. Based on Selected Subsets
11
of Observations From Table 1.

Observations Deleted
Observation None (21) (4,21) (2,4,21) (1,2,4,21) (1,3,4,21)
1 0.409 0.421 0.421 o. 727 * *
2 0.409 0.421 0.421 * * 0.993
3 0.176 0.199 0.201 0.308 0.983 *
4 0 ._191 0.192 * * * *
5 0.103 0.108 0.125 0.125 0.125 0.131
6 0.134 o. 134 0.164 0.164 0.165 0.169
7 0.191 0.192 0.238 0.239 0.240 0.242
8 0.191 0.192 0.238 0.239 0.240 0.242
9 0.163 0.170 0.206 0.208 0.218 0.208
10 0.139 0.175 0.175 0.176 0.179 0.179
11 0.139 0.175 0.175 0.176 0.179 0.179
12 0.212 0.272 0.275 0.276 0.279 0.280
13 0.139 0.175 0.175 0.176 0.179 0.179
14 0.092 0.110 0.111 0.112 o. 116 0.113
15 0.188 0.189 0.191 0.191 0.195 0.193
16 0.188 0.189 · 0.191 0.191 0.195 0.193
17 0.187 0.195 0.195 0.195 0.198 0.197
18 0.187 0.195 0.195 0.195 0.198 0.197
19 0.212 0.232 0.234 0.234 0.236 0.237
20 0.064 0.064 0.069 0.070 0.076 0.070
21 ' 0.288 * * * * *
 TABLE 4

Studentized Residuals, t., Based on

1
Selected Subsets of Observations from Table 1.
Observations Deleted
Observation None (21) (4,21) (2,4,21) (1,2,4,21) (1,3,4,21)
1 0.97 0.77 1.08 -1. 75
* *
2 -1.06 -1.81 -2.71 0.57
* *
3 1.54 1.40 2.30 1.91 1.21 *
4 2.27 3.01 * * * *
5 -0.31 -0.63 -0.31 -0.40 -0.40 -0.12
6 -0.73 -0.97 -0.62 -0.80 -0.82 -0.64
7 -0.84 -0.92 -0.30 -0.35 -0.31 -0.18
8 -0.50 -0.48 0.36 -0.54 0.66 0.84
9 -0.94 -0.89 -0.39 -0.36 -0.18 -0.66
10 0.84 0.38 0.49 0.75 0.94 0.96
11 0.84 0.38 0.49 0.75 0.94 0.96
12 0.96 0.31 0.16 0.30 0.45 0.53
13 -0.17 -0.91 -1.42 -1.82 -1.87 -1.98
14 -0.25 -0. 79 -1.04 -1.30 -1.29 -1.41
15 0.17 0.34 0.29 0.29 0.19 0.32
16 -0.17 -0.10 -0.35 -0.57 -0.76 -0.67
17 -0.26 -0.01 -0.01 -0.10 -0.23 -0.21
18 -0.26 -0.01 -0 .. 01 -0 . 10 -0.23 -0.21
19 -0.35 0.09 0.33 0.37 0.31 0.27
20 0.68 0.75 1.42 2.04 2.38 2.16
21 -2.63 * * * * *
,.

TABLE 5
A

Estimated Coefficients, $k, Partial F-

Statistics, Fk, and Mean Square Error, MSE, Based
on Selected Subsets of Observations from Table 1.

Observations Deleted
None (21) (4 21) (2 4 21)
A : A A A

Term ak Fk ak Fk ak Fk f3k Fk
1 -14.30 0.19 -25.90 1.00 -3.74 0.04 13.26 0.85
xl · -0.46 0.21 0.07 0.01 -0.51 0.80 -1.10 5.95

X2 0.009 1.27 0.006 0.97 -0. 011 6.49 I 0.017 21.34

1
x2 1.25 11.63 0.79 6.00 0.47 4.19 I 0.46 7.15

MSE 10.33 6.51 3.02 I 1.65

1 2 4 21) ( l ,]_,_4, 21)

A A

Term ak Fk ak Flc'
1 33.)4 3.96 -15.41 1.5
xl -1.82 10.61 - 0.07 0.03
X2 0.023 24.09 0.007 4.60
1
x2 0.44 7.81 0.53 12.27

MSE 1.39 1.26