14:39 12/12/25 Regression Analysis By Example Using R

CHAPTER 3
SIMPLE LINEAR REGRESSION

3.1 INTRODUCTION

We start with the simple case of studying the

relationship between a response variable and a
predictor variable . Since we have only one predictor
variable, we shall drop the subscript in and use for
simplicity. We discuss covariance and correlation
coefficient as measures of the direction and strength of
the linear relationship between the two variables. A
simple linear regression model is then formulated and
the key theoretical results are given without
mathematical derivations, but illustrated by numerical
examples. Readers interested in mathematical
derivations are referred to the bibliographic notes at the
end of the chapter, where books that contain a formal
development of regression analysis are listed.

3.2 COVARIANCE AND CORRELATION

COEFFICIENT

Suppose we have observations on subjects consisting

of a dependent or response variable and an
explanatory variable . The observations are usually
recorded as in Table 3.1. We wish to measure both the
direction and the strength of the relationship between

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 1/63

14:39 12/12/25 Regression Analysis By Example Using R

and . Two related measures, known as the covariance

and the correlation coefficient, are developed below.

Table 3.1 Notation for the Data Used in Simple

Regression and Correlation

Observation Response Variable Predictor

Number
1
2

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 2/63

14:39 12/12/25 Regression Analysis By Example Using R

Figure 3.1 Graphical illustration of the correlation

coefficient.

On the scatter plot of versus , let us draw a vertical

line at and a horizontal line at , as shown in Figure
3.1, where

(3.1)

are the sample mean of and , respectively. The two

lines divide the graph into four quadrants. For each
point in the graph, compute the following quantities:

, the deviation of each observation from the

mean of the response variable,

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 3/63

14:39 12/12/25 Regression Analysis By Example Using R

, the deviation of each observation from the

mean of the predictor variable, and

the product of the above two quantities, .

It is clear from the graph that the quantity is

positive for every point in the first and second quadrants
and is negative for every point in the third and fourth
quadrants. Similarly, the quantity is positive for
every point in the first and fourth quadrants and is
negative for every point in the second and third
quadrants. These facts are summarized in Table 3.2.

If the linear relationship between and is positive (as

increases also increases), then there are more
points in the first and third quadrants than in the second
and fourth quadrants. In this case, the sum of the last
column in Table 3.2 is likely to be positive because there
are more positive than negative quantities. Conversely,
if the relationship between and is negative (as
increases decreases), then there are more points in
the second and fourth quadrants than in the first and
third quadrants. Hence the sum of the last column in
Table 3.2 is likely to be negative. Therefore, the sign of
the quantity

(3.2)

which is known as the covariance between and ,

indicates the direction of the linear relationship between

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 4/63

14:39 12/12/25 Regression Analysis By Example Using R

and . If cov , then there is a positive

relationship between and , but if cov , then the
relationship is negative. Unfortunately, cov does not
tell us much about the strength of such a relationship
because it is affected by changes in the units of
measurement. For example, we would get two different
values for the cov if we report and/or in terms of
thousands of dollars instead of dollars. To avoid this
disadvantage of the covariance, we standardize the
data before computing the covariance. To standardize
the data, we first subtract the mean from each
observation then divide by the standard deviation, that
is, we compute

Table 3.2 Algebraic Signs of the Quantities and

Quadrant
1
2
3
4

(3.3)

where

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 5/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.4)

is the sample standard deviation of . It can be shown

that the standardized variable in (3.3) has mean zero
and standard deviation one. We standardize in a
similar way by subtracting the mean from each
observation and then divide by the standard deviation
. The covariance between the standardized and
data is known as the correlation coefficient between
and and is given by

(3.5)

Equivalent formulas for the correlation coefficient are

(3.6)

(3.7)

Thus, cor can be interpreted either as the

covariance between the standardized variables or the
ratio of the covariance to the standard deviations of the
two variables. From (3.5), it can be seen that the
correlation coefficient is symmetric, that is,
.

Unlike cov , cor is scale invariant, that is, it does

not change if we change the units of measurements.
Furthermore, cor satisfies

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 6/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.8)

These properties make the cor a useful quantity for

measuring both the direction and the strength of the
relationship between and . The magnitude of cor
measures the strength of the linear relationship between
and . The closer cor is to 1 or , the stronger is
the relationship between and . The sign of cor
indicates the direction of the relationship between and
. That is, or implies that and are positively
related. Conversely, cor implies that and are
negatively related.

Table 3.3 Data Set with a Perfect Nonlinear

Relationship Between and , Yet cor

1 7 46 2 41 3
14 6 49 1 34 4
25 5 50 0 25 5
34 4 49 1 14 6
41 3 46 2 17

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 7/63

14:39 12/12/25 Regression Analysis By Example Using R

Figure 3.2 Scatter plot of versus in Table 3.3.

Note, however, that cor does not necessarily

mean that and are not related. It only implies that
they are not linearly related because the correlation
coefficient measures only linear relationships. In other
words, the cor can still be zero when and are
nonlinearly related. For example, and in Table 3.3
have the perfect nonlinear relationship
1
(graphed in Figure 3.2) , yet cor .

Furthermore, like many other summary statistics, the cor

can be substantially influenced by one or a few
outliers in the data. To emphasize this point, Anscombe
(1973) has constructed four data sets, known as
Anscombe quartet, each with a distinct pattern, but each
having the same set of summary statistics (e.g., the
same value of the correlation coefficient). The pairwise
scatter plots with the fitted regression lines are given in
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 8/63
14:39 12/12/25 Regression Analysis By Example Using R

Figure 3.3. The data can be found at the Book's

2
Website. It can also be obtained from the R package
datasets by typing data(anscombe). An analysis based
exclusively on an examination of summary statistics,
such as the correlation coefficient, would have been
unable to detect the differences in patterns.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 9/63

14:39 12/12/25 Regression Analysis By Example Using R

Figure 3.3 Scatter plots of Anscombe's data with the

fitted regression lines.

An examination of Figure 3.3 shows that only the first

set, whose plot is given in (a), can be described by a
linear model. The plot in (b) shows the second data set
is distinctly nonlinear and would be better fitted by a
quadratic function. The plot in (c) shows that the third

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 10/63

14:39 12/12/25 Regression Analysis By Example Using R

data set has one point that distorts the slope and the
intercept of the fitted line. The plot in (d) shows that the
fourth data set is unsuitable for linear fitting, the fitted
line being determined essentially by one extreme
observation. Therefore, it is important to examine the
scatter plot of versus before interpreting the
numerical value of cor .

3.3 EXAMPLE: COMPUTER REPAIR DATA

As an illustrative example, consider a case of a

company that markets and repairs small computers. To
study the relationship between the length of a service
call and the number of electronic components in the
computer that must be repaired or replaced, a sample of
records on service calls was taken. The data consist of
the length of service calls in minutes (the response
variable) and the number of components repaired (the
predictor variable). The data are presented in Table 3.4.
The Computer Repair data can also be found at the
Book's Website. We use this data set throughout this
chapter as an illustrative example. The quantities
needed to compute , , cov , and cor are
shown in Table 3.5. We have

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 11/63

14:39 12/12/25 Regression Analysis By Example Using R

and

The above quantities can be obtained using R as

follows: First download the file [Link]
from the Book's Website in your Working Directory (see
Section 2.6.1) in an object named df (see Section 2.6.5).
Then use the following code:

attach(df) # To make the variable names in df

available

colMeans(df)

cov(Minutes,Units)

cor(Minutes,Units)

detach(df) # To de-attach the object df

Before drawing conclusions from this value of cor ,

we should examine the corresponding scatter plot of
versus . This plot is given in Figure 3.4. The high value
of cor is consistent with the strong linear
relationship between and exhibited in Figure 3.4.
We therefore conclude that there is a strong positive
relationship between repair time and units repaired.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 12/63

14:39 12/12/25 Regression Analysis By Example Using R

Table 3.4 Length of Service Calls (in Minutes) and

Number of Units Repaired

Row Minutes Units Row Minutes Units

1 23 1 8 97 6
2 29 2 9 109 7
3 49 3 10 119 8
4 64 4 11 149 9
5 74 4 12 145 9
6 87 5 13 154 10
7 96 6 14 166 10

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 13/63

14:39 12/12/25 Regression Analysis By Example Using R

Table 3.5 Quantities Needed for Computation of

Correlation Coefficient Between Length of Service Calls,
, and Number of Units Repaired,

1 23 1 74.21 5507.76 25 371.07

2 29 2 68.21 4653.19 16 272.86
3 49 3 48.21 2324.62 9 144.64
4 64 4 33.21 1103.19 4 66.43
5 74 4 23.21 538.90 4 46.43
6 87 5 10.21 104.33 1 10.21
7 96 6 1.21 0 1.47 0 0.00
8 97 6 0.21 0 0.05 0 0.00
9 109 7 11.79 1 138.90 1 11.79
10 119 8 21.79 2 474.62 4 43.57
11 149 9 51.79 3 2681.76 9 155.36
12 145 9 47.79 3 2283.47 9 143.36
13 154 10 56.79 4 3224.62 16 227.14
14 166 10 68.79 4 4731.47 16 275.14
Total 1361 84 0.00 0 27,768.36 114 1768.00

Although cor is a useful quantity for measuring the

direction and the strength of linear relationships, it
cannot be used for prediction purposes, that is, we
cannot use cor to predict the value of one variable
given the value of the other. Furthermore, cor
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 14/63
14:39 12/12/25 Regression Analysis By Example Using R

measures only pairwise relationships. Regression

analysis, however, can be used to relate one or more
response variable to one or more predictor variables. It
can also be used in prediction. Regression analysis is
an attractive extension to correlation analysis because it
postulates a model that can be used not only to
measure the direction and the strength of a relationship
between the response and predictor variables, but also
to numerically describe that relationship. We discuss
simple linear regression models in the rest of this
chapter. Chapter 4 is devoted to multiple regression
models.

Figure 3.4 Computer repair data: scatter plot of minutes

versus units.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 15/63

14:39 12/12/25 Regression Analysis By Example Using R

3.4 THE SIMPLE LINEAR REGRESSION MODEL

The relationship between a response variable and a

3
predictor variable is postulated as a linear model

(3.9)

where and are constants called the model

regression coefficients or parameters, and is a
random disturbance or error. It is assumed that in the
range of the observations studied, the linear equation
(3.9) provides an acceptable approximation to the true
relation between and . In other words, is
approximately a linear function of , and measures
the discrepancy in that approximation. In particular
contains no systematic information for determining
that is not already captured in . The coefficient ,
called the slope, may be interpreted as the change in
for unit change in . The coefficient , called the
constant coefficient or intercept, is the predicted value of
when .

According to (3.9), each observation in Table 3.1 can be

written as

(3.10)

where represents the th value of the response

variable , represents the th value of the predictor
variable , and represents the error in the
approximation of .
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 16/63
14:39 12/12/25 Regression Analysis By Example Using R

Regression analysis differs in an important way from

correlation analysis. The correlation coefficient is
symmetric in the sense that cor is the same as cor
. The variables and are of equal importance. In
regression analysis the response variable is of
primary importance. The importance of the predictor
lies on its ability to account for the variability of the
response variable and not in itself per se. Hence is
of primary importance.

Returning to the Computer Repair Data example,

suppose that the company wants to forecast the number
of service engineers that will be required over the next
few years. A linear model,

(3.11)

is assumed to represent the relationship between the

length of service calls and the number of electronic
components in the computer that must be repaired or
replaced. To validate this assumption, we examine the
graph of the response variable versus the explanatory
variable. This graph, shown in Figure 3.4, suggests that
the straight line relationship in (3.11) is a reasonable
assumption.

3.5 PARAMETER ESTIMATION

Based on the available data, we wish to estimate the

parameters and . This is equivalent to finding the
straight line that gives the best fit (representation) of the
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 17/63
14:39 12/12/25 Regression Analysis By Example Using R

points in the scatter plot of the response versus the

predictor variable (see Figure 3.4). We estimate the
parameters using the popular least squares method,
which gives the line that minimizes the sum of squares
4
of the vertical distances from each point to the line.
The vertical distances represent the errors in the
response variable. These errors can be obtained by
rewriting (3.10) as

(3.12)

The sum of squares of these distances can then be

written as

(3.13)

The values of and that minimize are given by

(3.14)

and

(3.15)

Note that we give the formula for before the formula

for because uses . The estimates and are
called the least squares estimates of and because
they are the solution to the least squares method, the
intercept and the slope of the line that has the smallest
possible sum of squares of the vertical distances from
each point to the line. For this reason, the line is called
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 18/63
14:39 12/12/25 Regression Analysis By Example Using R

the least squares regression line. The least squares

regression line is given by

(3.16)

Note that a least squares line always exists because we

can always find a line that gives the minimum sum of
squares of the vertical distances. In fact, as we shall see
later, in some cases a least squares line may not be
unique. These cases are not common in practice.

For each observation in our data we can compute

(3.17)

These are called the fitted values. Thus, the th fitted

value, , is the point on the least squares regression
line (3.16) corresponding to . The vertical distance
corresponding to the th observation is

(3.18)
5
These vertical distances are called the ordinary least
squares residuals. One property of the residuals in
(3.18) is that their sum is zero [see Exercise 3.5(a)].
This means that the sum of the distances above the line
is equal to the sum of the distances below the line.

Using the Computer Repair data and the quantities in

Table 3.5, we have

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 19/63

14:39 12/12/25 Regression Analysis By Example Using R

and

Then the equation of the least squares regression line is

(3.19)

This least squares line is shown together with the

scatter plot of Minutes versus Units in Figure 3.5. The
fitted values in (3.17) and the residuals in (3.18) are
shown in Table 3.6.

The coefficients in (3.19) can be computed using the

function lm(), where lm stands for Linear Models. The
syntax for lm() is:

lm( ).

This code assumes that both Y and X are in the dataset

df. The in this code means that Y is a function of X, in
the same sense as . In the current example, we
use reg <– lm(Minutes Units, data = df). This code
indicates that the output of lm() is saved in an object
named reg (short for regression). Typing print(reg) after
this code will produce the following output:

reg <- lm(Minutes ∼ Units, data = df)

print(reg)
Call:
lm(formula = Minutes ∼ Units, data = df)
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 20/63
14:39 12/12/25 Regression Analysis By Example Using R

Coefficients:
(Intercept) Units
4.162 15.509

Figure 3.5 Computer repair data: plot of minutes versus

uUnits with the fitted regression line.

The last line in the above output gives the regression

coefficients and , respectively. These coefficients
can be interpreted in physical terms. The constant
(intercept) term represents the setup or startup time for
each repair and is approximately four minutes. The
coefficient of Units represents the increase in the length
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 21/63
14:39 12/12/25 Regression Analysis By Example Using R

of a service call for each additional component that has

to be repaired. From the data given, we estimate that it
takes about 15.5 minutes for each additional component
that has to be repaired. For example, the length of a
service call in which four components had to be repaired
is obtained by substituting Units = 4 in the equation of
the regression line (3.19) and obtaining
. Since Units = 4, corresponds to
two observations in our data set (observations 4 and 5),
the value 66.198 is the fitted value for both observations
4 and 5, as can be seen from Table 3.6. Note, however,
that since observations 4 and 5 have different values for
the response variable Minutes, they have different
residuals.

R knows that the object reg computed above is a lm()

output. Accordingly, the fitted values ( ) and residuals (
) in Table 3.6 can be computed using the commands:
fitted(reg) and residuals(reg), respectively.

We should note here that by comparing (3.2), (3.7), and

(3.14), an alternative formula for can be expressed as

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 22/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.20)

Table 3.6 Fitted Values, , and Ordinary Least Squares

Residuals, , for Computer Repair Data

1 1 23 19.67 3.33 8 6 97 97.21

2 2 29 35.18 9 7 109 112.72
3 3 49 50.69 10 8 119 128.23
4 4 64 66.20 11 9 149 143.74 5.26
5 4 74 66.20 7.80 12 9 145 143.74 1.26
6 5 87 81.71 5.29 13 10 154 159.25
7 6 96 97.21 14 10 166 159.25 6.75

from which it can be seen that , cov , and cor

have the same sign. This makes intuitive sense because
positive (negative) slope means positive (negative)
correlation.

So far in our analysis we have made only one

assumption, namely, that and are linearly related.
This assumption is referred to as the linearity
assumption. This is merely an assumption or a
hypothesis about the relationship between the response
and predictor variables. An early step in the analysis
should always be the validation of this assumption. We
wish to determine if the data at hand support the
assumption that and are linearly related. An informal
way to check this assumption is to examine the scatter
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 23/63
14:39 12/12/25 Regression Analysis By Example Using R

plot of the response versus the predictor variable,

preferably drawn with the least squares line
superimposed on the graph (see Figure 3.5). This graph
can be obtained in R using:

attach(df)
plot(Units,Minutes,pch=19)
reg<- lm(Minutes ∼ Units,data = df)
abline(reg)
detach(df)

If the scatter of points resembles a straight line, then we

conclude that the linearity assumption is reasonable and
continue with our analysis. But, if we observe a
nonlinear pattern, we will have to take corrective action.
For example, we may reexpress or transform the data
before we continue the analysis. Data transformation is
discussed in Chapter 7.

The least squares estimators have several desirable

properties when some additional assumptions hold. The
required assumptions are stated in Chapter 5. The
validity of these assumptions must be checked before
meaningful conclusions can be reached from the
analysis. Chapter 5 also presents methods for the
validation of these assumptions. Using the properties of
least squares estimators, one can develop statistical
inference procedures (e.g., confidence interval

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 24/63

14:39 12/12/25 Regression Analysis By Example Using R

estimation, tests of hypothesis, and goodness-of-fit

tests). These are presented in Sections 3.6–3.9.

3.6 TESTS OF HYPOTHESES

As stated earlier, the usefulness of as a predictor of

can be measured informally by examining the
correlation coefficient and the corresponding scatter plot
of versus . A more formal way of measuring the
usefulness of as a predictor of is to conduct a test of
hypothesis about the regression parameter . Note that
the hypothesis means that there is no linear
relationship between and . A test of this hypothesis
requires the following assumption. For every fixed value
of , the 's are assumed to be independent random
quantities normally distributed with mean zero and a
common variance . With these assumptions, the
6
quantities and are unbiased estimates of and ,
respectively. Their variances are

(3.21)

and

(3.22)

Furthermore, the sampling distributions of the least

squares estimates and are normal with means
and and variance as given in (3.21) and (3.22),
respectively.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 25/63

14:39 12/12/25 Regression Analysis By Example Using R

The variances of and depend on the unknown

parameter . So, we need to estimate from the data.
An unbiased estimate of is given by

(3.23)

where SSE is the sum of squares of the residuals

(errors). The square root of , , is called the residual
standard error. The number in the denominator of
(3.23) is called the degrees of freedom ( ). It is equal to
the number of observations minus the number of
estimated regression coefficients.

Replacing in (3.21) and (3.22) by in (3.23), we get

unbiased estimates of the variances of and . An
estimate of the standard deviation is called the standard
error (s.e.) of the estimate. Thus, the standard errors of
and are

(3.24)

and

(3.25)

respectively, where is the square root of in (3.23).

The standard error of is a measure of how precisely
the slope has been estimated. The smaller the standard
error, the more precise the estimator.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 26/63

14:39 12/12/25 Regression Analysis By Example Using R

The R object reg from the lm() call actually includes the
quantities in (3.23)–(3.25), among others. To see this, if
we type summary(reg), we will get

summary(reg)
Call:
lm(formula = Minutes ∼ Units, data = df)
Residuals:
Min 1Q Median 3Q Max
-9.2318 -3.3415 -0.7143 4.7769 7.8033
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.162 3.355 1.24 0.239
Units 15.509 0.505 30.71 8.92e-13 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 5.392 on 12 degrees of freedo

m
Multiple R-squared: 0.9874, Adjusted R-squared: 0.9
864
F-statistic: 943.2 on 1 and 12 DF, p-value: 8.916e-13

In the above output, after the call, a summary of the

residuals is given followed by what is called the
Coefficients Table. This table shows the estimated
regression coefficients (4.162 and 15.509) and their
standard errors (3.355 and 0.505), respectively. The first
line below the Coefficients Table shows that the
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 27/63
14:39 12/12/25 Regression Analysis By Example Using R

Residual standard error is 5.392 on 12 ( ) degrees of

freedom. The other values in the above output will be
explained shortly.

With the sampling distributions of and , we are now

in position to perform statistical analysis concerning the
usefulness of as a predictor of . Under the normality
assumption, an appropriate test statistic for testing the
null hypothesis against the alternative
is the -Test,

(3.26)

The statistic is distributed as a Student's with

degrees of freedom. The test is carried out by
comparing this observed value with the appropriate
critical value obtained from the Student -distribution,
which is , where is a specified significance level.
Note that we divide by 2 because we have a two-
sided alternative hypothesis. The value of can be
obtained by R using the command qt( , , [Link]
= FALSE). Accordingly, is to be rejected at the
significance level if

(3.27)

where denotes the absolute value of .

A criterion equivalent to that in (3.27) is to compare the

-value for the -Test with and reject if

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 28/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.28)

where , called the -value, is the probability that a

random variable having a Student distribution with
degrees of freedom is greater than (the absolute
value of the observed value of the -Test). Figure 3.6 is
a graph of the density function of a -distribution. The -
value is the sum of the two shaded areas under the
curve. Letting abs.t = abs , the -value can be
obtained using R by

2 * pt(abs.t, , [Link] = FALSE).

Figure 3.6 Graph of the probability density function of a

-distribution. The -value for the -Test is the shaded
areas under the curve.

Note that we multiplied by 2 in the above line because

the pt() gives only one side. However, the -value is
computed and supplied as part of the lm() output. In the
summary(reg) above, the -values and -values are

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 29/63

14:39 12/12/25 Regression Analysis By Example Using R

given for both the intercept and the slope under the
columns labeled t value and Pr(>|t|), respectively. Note
that the -value for the slope is nearly zero (8.92e–13);
hence, is rejected at practically any small
significant level. This is indicated in the summary(reg)
output above by three [Link] would mean that is
likely to be different from 0, and hence the predictor
variable Units is a statistically significant predictor of the
response variable Minutes.

To complete the picture of hypotheses testing regarding

regression parameters, we give here tests for three
other hypotheses that may arise in practice.

Testing :

The above -Test can be generalized to test the more

general hypothesis , where is a constant
chosen by the investigator, against the two-sided
alternative . The appropriate test statistic in this
case is the -Test,

(3.29)

Note that when , the -Test in (3.29) reduces to the

-Test in (3.26). The statistic in (3.29) is also
distributed as a Student's with degrees of
freedom. Thus, is rejected if (3.27) holds [or,
equivalently, if (3.28) holds].

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 30/63

14:39 12/12/25 Regression Analysis By Example Using R

For illustration, using the Computer Repair data, let us

suppose that the management expected the increase in
service time for each additional unit to be repaired to be
12 minutes. Do the data support this conjecture? The
answer may be obtained by testing against
. The appropriate statistic is

with 12 degrees of freedom. The critical value for this

test is . This value is obtained using
the R command

qt(0.05/2, 12, [Link] = FALSE)

Since , the result is highly significant, leading

to the rejection of the null hypothesis. The
management's estimate of the increase in time for each
additional component to be repaired is not supported by
the data. Their estimate is too low.

Testing :

The need for testing hypotheses regarding the

regression parameter may also arise in practice. More
specifically, suppose we wish to test against
the alternative , where is a constant chosen
by the investigator. The appropriate test in this case is
given by

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 31/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.30)

If we set , a special case of this test is obtained as

(3.31)

which tests against the alternative .

The least squares estimates of the regression

coefficients, their standard errors, the -Tests for testing
that the corresponding coefficient are zero, and the -
values are usually given as part of the regression output
by statistical packages. These values are usually
displayed in a table such as the one in Table 3.7. This
table is known as the coefficients table. To facilitate the
connection between a value in the table and the formula
used to obtain it, the equation number of the formula is
given in parentheses.

As an illustrative example, Table 3.8 shows a part of the

regression output for the Computer Repair data in Table
3.4. Thus, for example, , the s.e. , and
hence . The critical value for this test
using , for example, is . The is
much larger than its critical value 2.18. Consequently,
according to (3.27), is rejected, which means
that the predictor variable Units is a statistically
significant predictor of the response variable Minutes.
This conclusion can also be reached using (3.28) by

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 32/63

14:39 12/12/25 Regression Analysis By Example Using R

observing that the -value ( ) is much less than

indicating very high significance.

Table 3.7 Standard Regression Output

Variable Coefficient s.e. -Test -

(Formula) (Formula) (Formula) Value
Constant (3.15) (3.24) (3.31)
(3.14) (3.26)
(3.25)

Equation number of the corresponding formulas are

given in parentheses.

Table 3.8 Regression Output for Computer Repair Data

Variable Coefficient s.e. -Test -Value

Constant 4.162 3.355 1.24 0.2385
Units 15.509 0.505 30.71 0.0000

A Test Using Correlation Coefficient

As mentioned above, a test of against

can be thought of as a test for determining whether the
response and the predictor variables are linearly related.
We used the -Test in (3.26) to test this hypothesis. An
alternative test, which involves the correlation coefficient
between and , can be developed. Suppose that the
population correlation coefficient between and is
denoted by . If , then and are linearly related.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 33/63

14:39 12/12/25 Regression Analysis By Example Using R

An appropriate test for testing against is

given by

(3.32)

where cor is the sample correlation coefficient

between and , defined in (3.6), which is considered
here to be an estimate of . The -Test in (3.32) is
distributed as a Student's with degrees of
freedom. Thus, is rejected if (3.27) holds [or,
equivalently, if (3.28) holds]. Again if is rejected,
it means that there is a statistically significant linear
relationship between and .

It is clear that if no linear relationship exists between

and , then . Consequently, the statistical tests for
and should be identical. Although the
statistics for testing these hypotheses given in (3.26)
and (3.32) look different, it can be demonstrated that
they are indeed algebraically equivalent.

3.7 CONFIDENCE INTERVALS

To construct confidence intervals for the regression

parameters, we also need to assume that the 's have a
normal distribution, which will enable us to conclude that
the sampling distributions of and are normal, as
discussed in Section 3.6. Consequently, the
confidence interval for is given by

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 34/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.33)

where is the percentile of a distribution

with degrees of freedom. Similarly, limits of the
confidence interval for are given by

(3.34)

The confidence interval in (3.34) has the usual

interpretation, namely, if we were to take repeated
samples of the same size at the same values of and
construct, for example, 95% confidence intervals for the
slope parameter for each sample, then 95% of these
intervals would be expected to contain the true value of
the slope.

From Table 3.8 we see that a 95% confidence interval

for is

(3.35)

That is, the incremental time required for each broken

unit is between 14 and 17 minutes. The calculation of
confidence interval for in this example is left as an
exercise for the reader.

Note that the confidence limits in (3.33) and (3.34) are

constructed for each of the parameters and ,
separately. This does not mean that a simultaneous
(joint) confidence region for the two parameters is
rectangular. Actually, the simultaneous confidence
region is elliptical. This region is given for the general
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 35/63
14:39 12/12/25 Regression Analysis By Example Using R

case of multiple regression in the Appendix to Chapter 4

in (A.15), of which the simultaneous confidence region
for and is a special case.

3.8 PREDICTIONS

The fitted regression equation can be used for

prediction. We distinguish between two types of
predictions:

1. The prediction of the value of the response variable

which corresponds to any chosen value, , of the
predictor variable.

2. The estimation of the mean response , when

.

For the first case, the predicted value is

(3.36)

The standard error of this prediction is

(3.37)

Hence, the confidence limits for the predicted value with

confidence coefficient are given by

(3.38)

For the second case, the mean response is estimated

by

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 36/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.39)

The standard error of this estimate is

(3.40)

from which it follows that the confidence limits for with

confidence coefficient are given by

(3.41)

Note that the point estimate of is identical to the

predicted response . This can be seen by comparing
(3.36) with (3.39). The standard error of is, however,
smaller than the standard error of and can be seen by
comparing (3.37) with (3.40). Intuitively, this makes
sense. There is greater uncertainty (variability) in
predicting one observation (the next observation) than in
estimating the mean response when . The
averaging that is implied in the mean response reduces
the variability and uncertainty associated with the
estimate.

To distinguish between the limits in (3.38) and (3.41),

the limits in (3.38) are sometimes referred to as the
prediction or forecast limits, whereas the limits given in
(3.41) are called the confidence limits.

Suppose that we wish to predict the length of a service

call in which four components had to be repaired. If
denotes the predicted value, then from (3.36) we get

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 37/63

14:39 12/12/25 Regression Analysis By Example Using R

with a standard error that is obtained from (3.37) as

On the other hand, if the service department wishes to

estimate the expected (mean) service time for a call that
needed four components repaired, we would use (3.39)
and (3.40), respectively. Denoting by , the expected
service time for a call that needed four components to
be repaired, we have

with a standard error

With these standard errors we can construct confidence

intervals using (3.38) and (3.41), as appropriate.

As can be seen from (3.37), the standard error of

prediction increases the farther the value of the
predictor variable is from the center of the actual
observations. Care should be taken when predicting the
value of Minutes corresponding to a value for Units that
does not lie close to the observed data. There are two
dangers in such predictions. First, there is substantial
uncertainty due to the large standard error. More
important, the linear relationship that has been
estimated may not hold outside the range of
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 38/63
14:39 12/12/25 Regression Analysis By Example Using R

observations. Therefore, care should be taken in

employing fitted regression lines for prediction far
outside the range of observations. In our example we
would not use the fitted equation to predict the service
time for a service call which requires that 25
components be replaced or repaired. This value lies too
far outside the existing range of observations.

3.9 MEASURING THE QUALITY OF FIT

After fitting a linear model relating to , we are

interested not only in knowing whether a linear
relationship exists, but also in measuring the quality of
the fit of the model to the data. The quality of the fit can
be assessed by one of the following highly related
(hence, somewhat redundant) ways:

1. When using the tests in (3.26) or (3.32), if is

rejected, the magnitude of the values of the test (or
the corresponding -values) gives us information
about the strength (not just the existence) of the
linear relationship between and . Basically, the
larger the (in absolute value) or the smaller the
corresponding -value, the stronger the linear
relationship between and . These tests are
objective but they require all the assumptions
stated earlier, specially the assumption of normality
of the 's.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 39/63

14:39 12/12/25 Regression Analysis By Example Using R

2. The strength of the linear relationship between

and can also be assessed directly from the
examination of the scatter plot of versus
together with the corresponding value of the
correlation coefficient cor in (3.6). The closer
the set of points to a straight line [the closer cor
to 1 or ], the stronger the linear relationship
between and . This approach is informal and
subjective, but it requires only the linearity
assumption.

3. Examine the scatter plot of versus . The closer

the set of points to a straight line, the stronger the
linear relationship between and . One can
measure the strength of the linear relationship in
this graph by computing the correlation coefficient
between and , which is given by

(3.42)

where is the mean of the response variable and

is the mean of the fitted values. In fact, the scatter
plot of versus and the scatter plot of versus
are redundant because the patterns of points in the
two graphs are identical. The two corresponding
values of the correlation coefficient are related by
the following equation:

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 40/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.43)

Note that cor cannot be negative (why?), but

cor can be positive or negative [ ].
Therefore, in simple linear regression, the scatter
plot of versus is redundant. However, in
multiple regression, the scatter plot of versus is
not redundant. The graph is very useful because,
as we shall see in Chapter 4, it is used to assess
the strength of the relationship between and the
set of predictor variables .

4. Although scatter plots of versus and cor are

redundant in simple linear regression, they give us
an indication of the quality of the fit in both simple
and multiple regressions. Furthermore, in both
simple and multiple regressions, cor is related
to another useful measure of the quality of fit of the
linear model to the observed data. This measure is
developed as follows. After we compute the least
squares estimates of the parameters of a linear
model, let us compute the following quantities:

(3.44)

where SST stands for the total sum of squared

deviations in from its mean , SSR denotes the
sum of squares due to regression, and SSE
represents the sum of squared residuals (errors).
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 41/63
14:39 12/12/25 Regression Analysis By Example Using R

The quantities , , and are depicted

in Figure 3.7 for a typical point . The line
is the fitted regression line based on all
data points (not shown on the graph) and the
horizontal line is drawn at . Note that for every
point , there are two points, , which lies on
the fitted line, and which lies on the line .

A fundamental equality, in both simple and multiple

regressions, is given by

(3.45)

This equation arises from the description of an

observation as

= +( )
Observed = Fit + Deviation from fit.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 42/63

14:39 12/12/25 Regression Analysis By Example Using R

Figure 3.7 Graphical illustration of various

quantities computed after fitting a regression line to
data.

Subtracting from both sides, we obtain

=( ) +( )
Deviation from = Deviation due to + Residual.
mean fit

Accordingly, the total sum of squared deviations in

can be decomposed into the sum of two
quantities, the first, SSR, measures the quality of
as a predictor of , and the second, SSE,
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 43/63
14:39 12/12/25 Regression Analysis By Example Using R

measures the error in this prediction. Therefore, the

ratio can be interpreted as the proportion
of the total variation in that is accounted for by
the predictor variable . Using (3.45), we can
rewrite as

(3.46)

Additionally, it can be shown that

(3.47)

In simple linear regression, is equal to the square

of the correlation coefficient between the response
variable and the predictor or to the square of
the correlation coefficient between the response
variable and the fitted values . The definition
given in (3.46) provides us with an alternative
interpretation of the squared correlation coefficients.
The goodness-of-fit index, , may be interpreted as
the proportion of the total variability in the response
variable that is accounted for by the predictor
variable . Note that because SSE SST.
If is near 1, then accounts for a large part of
the variation in . For this reason, is known as
the coefficient of determination because it gives us
an idea of how the predictor variable accounts for
(determines) the response variable . The same
interpretation of will carry over to the case of
multiple regression.
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 44/63
14:39 12/12/25 Regression Analysis By Example Using R

Using the Computer Repair data, the fitted values,

and the residuals in Table 3.6, the reader can verify
that , from which it follows that
. The same value of can be
computed using (3.46). Verify that SST =
27,768.348 and SSE = 348.848. So that

The value indicates that nearly 99% of the

total variability in the response variable (Minutes) is
accounted for by the predictor variable (Units). The
high value of indicates a strong linear relationship
between servicing time and the number of units
repaired during a service call.

We reemphasize that the regression assumptions

should be checked before drawing statistical
conclusions from the analysis (e.g., conducting tests of
hypothesis or constructing confidence or prediction
intervals) because the validity of these statistical
procedures hinges on the validity of the assumptions.
Chapter 5 presents a collection of graphical displays
that can be used for checking the validity of the
assumptions. We have used these graphs for the
computer repair data and found no evidence that the
underlying assumptions of regression analysis are not in
order. In summary, the 14 data points in the Computer
Repair data have given us an informative view of the

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 45/63

14:39 12/12/25 Regression Analysis By Example Using R

repair time problem. Within the range of observed data,

we are confident of the validity of our inferences and
predictions.

3.10 REGRESSION LINE THROUGH THE ORIGIN

We have considered fitting the model

(3.48)

which is a regression line with an intercept. Sometimes,

it may be necessary to fit the model

(3.49)

a line passing through the origin. This model is also

called the no-intercept model. The line may be forced to
go through the origin because of subject matter theory
or other physical and material considerations. For
example, distance traveled as a function of time should
have no constant. Thus, in this case, the regression
model in (3.49) is appropriate. Many other practical
applications can be found where model (3.49) is more
appropriate than (3.48). We shall see some of these
examples in Chapter 8.

The least squares estimate of in (3.49) is

(3.50)

The th fitted value is

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 46/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.51)

and the corresponding residual is

(3.52)

The standard error of the is

(3.53)

where

(3.54)

Note that the degree of freedom for SSE is , not ,

as is the case for a model with an intercept.

Note that the residuals in (3.52) do not necessarily add

up to zero as is the case for a model with an intercept
[see Exercise 3.11(c)]. Also, the fundamental identity in
(3.45) is no longer true in general. For this reason, some
quality measures for models with an intercept such as
in (3.46) are no longer appropriate for models with no
intercept. The appropriate identity for the case of models
with no intercept is obtained by replacing in (3.45) by
zero. Hence, the fundamental identity becomes

(3.55)

from which is redefined as

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 47/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.56)

This is the appropriate form of for models with no

intercept. Note, however, that the interpretations for the
two formulas of are different. In the case of models
with an intercept, can be interpreted as the proportion
of the variation in that is accounted for by the
predictor variable after adjusting by its mean. For
models without an intercept, no adjustment of is
made. For example, if we fit (3.49) but use the formula
for in (3.46), it is possible for to be negative in
some cases [see Exercise 3.11(d)]. Therefore, the
correct formula and the correct interpretation should be
used.

The formula for the -Test in (3.29) for testing

against the two-sided alternative continues to
hold but with the new definitions of and s.e. in
(3.50) and (3.53), respectively.

As we mentioned earlier, models with no intercept

should be used whenever they are consistent with the
subject matter (domain) theory or other physical and
material considerations. In some applications, however,
one may not be certain as to which model should be
used. In these cases, the choice between the models
given in (3.48) and (3.49) has to be made with care.
First, the goodness of fit should be judged by comparing
the residual mean squares ( ) produced by the two
models because it measures the closeness of the
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 48/63
14:39 12/12/25 Regression Analysis By Example Using R

observed and predicted values for the two models.

Second, one can fit model (3.48) to the data and use the
-Test in (3.31) to test the significance of the intercept. If
the test is significant, then use (3.48), otherwise use
(3.49).

An excellent exposition of regression models through

the origin is provided by Eisenhauer (2003) who also
alerts the users of regression models through the origin
to be careful when fitting these models using computer
software programs because some of them give incorrect
and confusing results for the case of regression models
through the origin.

3.11 TRIVIAL REGRESSION MODELS

In this section we give two examples of trivial regression

models, that is, regression equations that have no
regression coefficients. The first example arises when
we wish to test for the mean of a single variable
based on a random sample of observations , , ,
. Here we have against . Assuming that
is normally distributed with mean and variance ,
the well-known one-sample -Test

(3.57)

can be used to test , where is the sample standard

deviation of . Alternatively, the above hypotheses can
be formulated as

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 49/63

14:39 12/12/25 Regression Analysis By Example Using R

(3.58)

where . Thus, Model 1 indicates that and

Model 2 indicates that . The least squares estimate
of in Model 2 is , the th fitted value is , and the
th residual is (See Exercise. 3.15). It follows
then that an estimate of is

(3.59)

which is the sample variance of . The standard error of

is then , which is the familiar standard error
of the sample mean . The -Test for testing Model 1
against Model 2 is

(3.60)

which is the same as the one-sample -Test in (3.57).

The second example occurs in connection with the

paired two-sample -Test. For example, to test whether
a given diet is effective in weight reduction, a random
sample of people is chosen and each person in the
sample follows the diet for a specified period of time.
Each person's weight is measured at the beginning of
the diet and at the end of the period. Let and
denote the weight at the beginning and at the end of diet
period, respectively. Let be the difference
between the two weights. Then is a random variable
with mean and variance . Consequently, testing
whether or not the diet is effective is the same as testing
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 50/63
14:39 12/12/25 Regression Analysis By Example Using R

against . With the definition of and

assuming that is normally distributed, the well-known
paired two-sample -Test is the same as the test in
(3.57). This situation can be modeled as in (3.58) and
the test in (3.60) can be used to test whether the diet is
effective in weight reduction.

The above two examples show that the one-sample and

the paired two-sample tests can be obtained as special
cases using regression analysis.

3.12 BIBLIOGRAPHIC NOTES

The standard theory of regression analysis is developed

in a number of good text books, some of which have
been written to serve specific disciplines. Each provides
a complete treatment of the standard results. The books
by Snedecor and Cochran (1980), Fox (1984), and
Kmenta (1986) develop the results using simple algebra
and summation notation. The development in Searle
(1971), Rao (1973), Seber (1977), Myers (1990), Sen
and Srivastava (1990), Green (1993), Graybill and Iyer
(1994), Draper and Smith (1998), Yan and Su (2009),
and Montgomery et al. (2012) leans more heavily on
theory and/or matrix algebra.

EXERCISES

3.1 Using the data in Table 3.5:

a. Compute Var and Var .

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 51/63

14:39 12/12/25 Regression Analysis By Example Using R

b. Prove or verify that .

c. Prove or verify that any standardized variable

has a mean of 0 and a standard deviation of 1.

d. Prove or verify that the three formulas for cor

in (3.5), (3.6), and (3.7) are identical.

e. Prove or verify the inequalities in (3.8).

f. Prove or verify that the three formulas for in

(3.14) and (3.20) are identical.

3.2 Explain why you would or wouldn't agree with

each of the following statements:

a. The cov can take values between and

.

b. If cov or cor , one can conclude

that there is no relationship between and .

c. The least squares line fitted to the points in the

scatter plot of versus has a zero intercept
and a unit slope.

3.3 Using the regression output in Table 3.8, test

the following hypotheses using :

a. versus

b. versus

c. versus

d. versus

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 52/63

14:39 12/12/25 Regression Analysis By Example Using R

3.4 Using the regression output in Table 3.8,

construct the 99% confidence interval for .

3.5 When fitting the simple linear regression model

to a set of data using the least squares
method, each of the following statements can be
proven to be true. Prove each statement
mathematically or demonstrate its correctness
numerically (using the data in Table 3.4):

a. The sum of the ordinary least squares

residuals is zero.

b. The two tests in (3.26) and (3.32) are

equivalent.

c. The scatter plot of versus and the scatter

plot of versus have identical patterns.

d. The correlation coefficient between and

must be nonnegative.

3.6 Using the data in Table 3.4, and the fitted values
and the residuals in Table 3.6, verify that:

a.

b. SST = 27,768.348

c. SSE = 348.848

3.7 Verify that the four pairs in Anscombe's data

give identical results for the following quantities:
(a) and (b)

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 53/63

14:39 12/12/25 Regression Analysis By Example Using R

(c) (d) The -Test

3.8 When fitting a simple linear regression model

to a set of data using the least squares
method, suppose that was not rejected.
This implies that the model can be written simply
as:

Table 3.9 Regression Output When Is Regressed

on for Labor Force Participation Rate of Women

Variable Coefficient s.e. -Test -Value

Constant 0.203311 0.0976 2.08 0.0526
0.656040 0.1961 3.35 0.0038
df = 17

. The least squares estimate of is

. (Can you prove that?)

a. What are the ordinary least squares residuals

in this case?

b. Show that the ordinary least squares residuals

sum up to zero.

3.9 Let and denote the labor force participation

rate of women in 1972 and 1968, respectively, in
each of 19 cities in the United States. The
regression output for this data set is shown in Table
3.9. It was also found that SSR = 0.0358 and SSE

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 54/63

14:39 12/12/25 Regression Analysis By Example Using R

= 0.0544. Suppose that the model

satisfies the usual regression assumptions.

a. Compute Var and .

b. Suppose that the participation rate of women in

1968 in a given city is 45%. What is the
estimated participation rate of women in 1972
for the same city?

c. Suppose further that the mean and variance of

the participation rate of women in 1968 are 0.5
and 0.005, respectively. Construct the 95%
confidence interval for the estimate in (b).

d. Construct the 95% confidence interval for the

slope of the true regression line, .

e. Test the hypothesis: versus at

the 5% significance level.

f. If and were reversed in the above

regression, what would you expect to be?

3.10 One may wonder if people of similar heights

tend to marry each other. For this purpose, a
sample of newly married couples was selected. Let
be the height of the husband and be the height
of the wife. The heights (in centimeters) of
husbands and wives can be found at the Book's
Website.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 55/63

14:39 12/12/25 Regression Analysis By Example Using R

a. Compute the covariance between the heights

of the husbands and wives.

b. What would the covariance be if heights were

measured in inches rather than in centimeters?

c. Compute the correlation coefficient between

the heights of the husband and wife.

d. What would the correlation be if heights were

measured in inches rather than in centimeters?

e. What would the correlation be if every man

married a woman exactly 5 centimeters shorter
than him?

f. We wish to fit a regression model relating the

heights of husbands and wives. Which one of
the two variables would you choose as the
response variable? Justify your answer.

g. Using your choice of the response variable in

Exercise 3.10(f), test the null hypothesis that
the slope is zero.

h. Using your choice of the response variable in

3.10(f), test the null hypothesis that the
intercept is zero.

3.11 Consider fitting a simple linear regression

model through the origin, , to a set of data
using the least squares method.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 56/63

14:39 12/12/25 Regression Analysis By Example Using R

a. Give an example of a situation where fitting the

model (3.49) is justified by theoretical or other
physical and material considerations.

b. Show that least squares estimate of is as

given in (3.50).

c. Show that the residuals will not

necessarily add up to zero.

d. Give an example of a data set and in

which in (3.46) but computed from fitting
(3.49) to the data is negative.

e. Which goodness of fit measures would you use

to compare model (3.49) with model (3.48)?

3.12 The gold medal performance in the Men's

Long Jump (measured in meters) for the Summer
Olympic games can be found in the file
[Link] in the Book's Website. The
data are for the period from 1896 to 2022. Note that
Olympic games were not played in some years due
to wars.

a. Compute the number of gold medals for each

country.

b. Construct the scatter plot of the [Link]

versus Year.

c. What is the minimum [Link] during these

Olympic Series and in which year it occurred?

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 57/63

14:39 12/12/25 Regression Analysis By Example Using R

d. How many Olympic games were skipped due

to wars during this period and which years?

e. Fit a simple regression model between the

[Link] and Year and show the results.

f. From the regression output, compute the

variance of the variable [Link].

g. Which two years have the largest (in absolute

value) residuals?

h. What proportion of variability in the [Link]

that can be explained by the variable Year?

i. According to this model, what is your estimate

of the [Link] in the years during which the
Olympic games did not take place?

j. What is the minimum [Link] during these

Olympic Series and in which year it occurred?

k. Construct the confidence interval for the

intercept of the regression line.

l. At the level, test the null hypothesis that

there is no linear relationship between
[Link] and Year. (Specify the two
hypotheses, compute the test-statistics, and
indicate your decision).

m. At the level, test the null hypothesis that

slope of the true regression line is equal to 1.
(Specify the two hypotheses, compute the test-

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 58/63

14:39 12/12/25 Regression Analysis By Example Using R

statistic, compute the -value, and indicate

your decision).

3.13 The gold medal performance in the Women's

Long Jump (measured in meters) for the Summer
Olympic games can be found in the file
[Link] in the Book's Website. The
data are for the period from 1948 to 2022. Repeat
Exercise 3.12 using the data for women.

3.14 In order to investigate the feasibility of starting

a Sunday edition for a large metropolitan
newspaper, information was obtained from a
sample of 34 newspapers concerning their daily
and Sunday circulations (in thousands) (Source:
Gale Directory of Publications, 1994). The data can
be found in the file [Link] at the Book's
Website.

a. Construct a scatter plot of Sunday circulation

versus daily circulation. Does the plot suggest
a linear relationship between daily and Sunday
circulation? Do you think this is a plausible
relationship?

b. Fit a regression line predicting Sunday

circulation from daily circulation.

c. Obtain the 95% confidence intervals for and

.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 59/63

14:39 12/12/25 Regression Analysis By Example Using R

d. Is there a significant relationship between

Sunday circulation and daily circulation? Justify
your answer by a statistical test. Indicate what
hypothesis you are testing and your
conclusion.

e. What proportion of the variability in Sunday

circulation is accounted for by daily circulation?

f. Provide an interval estimate (based on 95%

level) for the average Sunday circulation of
newspapers with daily circulation of 500,000.

g. The particular newspaper that is considering a

Sunday edition has a daily circulation of
500,000. Provide an interval estimate (based
on 95% level) for the predicted Sunday
circulation of this paper. How does this interval
differ from that given in (f)?

h. Another newspaper being considered as a

candidate for a Sunday edition has a daily
circulation of 2,000,000. Provide an interval
estimate for the predicted Sunday circulation
for this paper. How does this interval compare
with the one given in (g)? Do you think it is
likely to be accurate?

3.15 Let be a sample drawn from a normal

population with unknown mean and unknown

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 60/63

14:39 12/12/25 Regression Analysis By Example Using R

variance . One way to estimate is to fit the

linear model

(3.61)

and use the least squares (LS), that is, to

minimize the sum of squares, .
Another way is to use the least absolute
value (LAV), that is, to minimize the sum of
absolute value of the vertical distances,
.

a. Show that the least squares estimate of is

the sample mean .

b. Show that the LAV estimate of is the sample

median.

c. State one advantage and one disadvantage of

the sample mean.

d. State one advantage and one disadvantage of

the sample median.

e. Which of the above two estimates of would

you choose? Why?

3.16 An alternative to the least squares method is

the orthogonal regression method. According to the
orthogonal regression method, the estimated
regression coefficients in the simple regression
model are obtained by minimizing the sum of
squares of the perpendicular distances from each
[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 61/63
14:39 12/12/25 Regression Analysis By Example Using R

point to the regression line. Show that the intercept

and the slope of the line that minimizes the sum of
the squared orthogonal distances are obtained by
finding and that minimize the function

(3.62)

Unlike the least squares criterion, there is no

closed-form solution to the minimization problem in
(3.62). A solution, however, can be obtained using
iterative methods. This is one reason for the
popularity of the least squares method.

NOTES

1 The R code for producing this and all other graphs in

this book can be found at the Book's Website at
[Link]

2 [Link]

3 The adjective linear has a dual role here. It may be

taken to describe the fact that the relationship
between and is linear. More generally, the word
linear refers to the fact that the regression
parameters, and , enter (3.9) in a linear fashion.
Thus, for example, is also a linear model
even though the relationship between and is
quadratic.

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 62/63

14:39 12/12/25 Regression Analysis By Example Using R

4 An alternative to the vertical distance is the

perpendicular (shortest) distance from each point to
the line. The resultant line is called the orthogonal
regression line.

5 To be distinguished from other types of residuals to

be presented later.

6 An estimate is said to be an unbiased estimate of a

parameter if the expected value of is equal to .

[Link] Analysis By Example Using R (12_12_2025 14：33：51).html 63/63