Math Refresher C Fundamentals of Mathematical Statistics 733

Before we turn to hypothesis testing, it is useful to review the various population and sample
quantities that measure the spreads in the population distributions and the sampling distributions of
the estimators. These quantities appear often in statistical analysis, and extensions of them are impor-
tant for the regression analysis in the main text. The quantity s is the (unknown) population standard
deviation; it is a measure of the spread in the distribution of Y. When we divide s by !n, we obtain
the sampling standard deviation of Y (the sample average). While s is a fixed feature of the popula-
tion, sd 1 Y 2 5 s/!n shrinks to zero as n S `: our estimator of m gets more and more precise as the
sample size grows.
The estimate of s for a particular sample, s, is called the sample standard deviation because it
is obtained from the sample. (We also call the underlying random variable, S, which changes across
different samples, the sample standard deviation.) Like y as an estimate of m, s is our “best guess”
at s given the sample at hand. The quantity s/!n is what we call the standard error of y, and it is
our best estimate of s/!n. Confidence intervals for the population parameter m depend directly on
se 1 y 2 5 s/!n. Because this standard error shrinks to zero as the sample size grows, a larger sample
size generally means a smaller confidence interval. Thus, we see clearly that one benefit of more data
is that they result in narrower confidence intervals. The notion of the standard error of an estimate,
which in the vast majority of cases shrinks to zero at the rate 1/!n, plays a fundamental role in
hypothesis testing (as we will see in the next section) and for confidence intervals and testing in the
context of multiple regression (as discussed in Chapter 4).

C-6 Hypothesis Testing

So far, we have reviewed how to evaluate point estimators, and we have seen—in the case of a popu-
lation mean—how to construct and interpret confidence intervals. But sometimes the question we are
interested in has a definite yes or no answer. Here are some examples: (1) Does a job training program
effectively increase average worker productivity? (see Example C.2); (2) Are blacks discriminated
against in hiring? (see Example C.3); (3) Do stiffer state drunk driving laws reduce the number of
drunk driving arrests? Devising methods for answering such questions, using a sample of data, is
known as hypothesis testing.

C-6a Fundamentals of Hypothesis Testing

To illustrate the issues involved with hypothesis testing, consider an election example. Suppose there
are two candidates in an election, Candidates A and B. Candidate A is reported to have received 42%
of the popular vote, while Candidate B received 58%. These are supposed to represent the true per-
centages in the voting population, and we treat them as such.
Candidate A is convinced that more people must have voted for him, so he would like to investi-
gate whether the election was rigged. Knowing something about statistics, Candidate A hires a con-
sulting agency to randomly sample 100 voters to record whether or not each person voted for him.
Suppose that, for the sample collected, 53 people voted for Candidate A. This sample estimate of 53%
clearly exceeds the reported population value of 42%. Should Candidate A conclude that the election
was indeed a fraud?
While it appears that the votes for Candidate A were undercounted, we cannot be certain. Even if
only 42% of the population voted for Candidate A, it is possible that, in a sample of 100, we observe
53 people who did vote for Candidate A. The question is: How strong is the sample evidence against
the officially reported percentage of 42%?
One way to proceed is to set up a hypothesis test. Let u denote the true proportion of the popula-
tion voting for Candidate A. The hypothesis that the reported results are accurate can be stated as
H0: u 5 .42 [C.28]

58860_em_appC_hr_714-[Link] 733 10/18/18 4:52 PM

 734 Math Refresher

This is an example of a null hypothesis. We always denote the null hypothesis by H0. In hypothesis
testing, the null hypothesis plays a role similar to that of a defendant on trial in many judicial systems:
just as a defendant is presumed to be innocent until proven guilty, the null hypothesis is presumed to
be true until the data strongly suggest otherwise. In the current example, Candidate A must present
fairly strong evidence against (C.28) in order to win a recount.
The alternative hypothesis in the election example is that the true proportion voting for
Candidate A in the election is greater than .42:
H1: u . .42. [C.29]
In order to conclude that H0 is false and that H1 is true, we must have evidence “beyond reason-
able doubt” against H0. How many votes out of 100 would be needed before we feel the evidence is
strongly against H0? Most would agree that observing 43 votes out of a sample of 100 is not enough
to overturn the original election results; such an outcome is well within the expected sampling varia-
tion. On the other hand, we do not need to observe 100 votes for Candidate A to cast doubt on H0.
Whether 53 out of 100 is enough to reject H0 is much less clear. The answer depends on how we
quantify “beyond reasonable doubt.”
Before we turn to the issue of quantifying uncertainty in hypothesis testing, we should head off
some possible confusion. You may have noticed that the hypotheses in equations (C.28) and (C.29)
do not exhaust all possibilities: it could be that u is less than .42. For the application at hand, we are
not particularly interested in that possibility; it has nothing to do with overturning the results of the
election. Therefore, we can just state at the outset that we are ignoring alternatives u with u , .42.
Nevertheless, some authors prefer to state null and alternative hypotheses so that they are exhaustive,
in which case our null hypothesis should be H0: u # .42. Stated in this way, the null hypothesis is a
composite null hypothesis because it allows for more than one value under H0. [By contrast, equa-
tion (C.28) is an example of a simple null hypothesis.] For these kinds of examples, it does not mat-
ter whether we state the null as in (C.28) or as a composite null: the most difficult value to reject if
u # .42 is u 5 .42. (That is, if we reject the value u 5 .42, against u . .42, then logically we must
reject any value less than .42.) Therefore, our testing procedure based on (C.28) leads to the same test
as if H0: u # .42. In this text, we always state a null hypothesis as a simple null hypothesis.
In hypothesis testing, we can make two kinds of mistakes. First, we can reject the null hypothesis
when it is in fact true. This is called a Type I error. In the election example, a Type I error occurs if
we reject H0 when the true proportion of people voting for Candidate A is in fact .42. The second kind
of error is failing to reject H0 when it is actually false. This is called a Type II error. In the election
example, a Type II error occurs if u . .42 but we fail to reject H0.
After we have made the decision of whether or not to reject the null hypothesis, we have either
decided correctly or we have committed an error. We will never know with certainty whether an error
was committed. However, we can compute the probability of making either a Type I or a Type II error.
Hypothesis testing rules are constructed to make the probability of committing a Type I error fairly
small. Generally, we define the significance level (or simply the level) of a test as the probability of a
Type I error; it is typically denoted by a. Symbolically, we have
a 5 P 1 Reject H0 0 H0 2 . [C.30]
The right-hand side is read as: “The probability of rejecting H0 given that H0 is true.”
Classical hypothesis testing requires that we initially specify a significance level for a test. When
we specify a value for a, we are essentially quantifying our tolerance for a Type I error. Common val-
ues for a are .10, .05, and .01. If a 5 .05, then the researcher is willing to falsely reject H0 5% of the
time, in order to detect deviations from H0.
Once we have chosen the significance level, we would then like to minimize the probability of a
Type II error. Alternatively, we would like to maximize the power of a test against all relevant alter-
natives. The power of a test is just one minus the probability of a Type II error. Mathematically,
p 1 u 2 5 P 1 Reject H0 0 u 2 5 1 2 P 1 Type II 0 u 2 ,

58860_em_appC_hr_714-[Link] 734 10/18/18 4:52 PM

 Math Refresher C Fundamentals of Mathematical Statistics 735

where u denotes the actual value of the parameter. Naturally, we would like the power to equal unity
whenever the null hypothesis is false. But this is impossible to achieve while keeping the significance
level small. Instead, we choose our tests to maximize the power for a given significance level.

C-6b Testing Hypotheses about the Mean in a Normal Population

In order to test a null hypothesis against an alternative, we need to choose a test statistic (or statistic,
for short) and a critical value. The choices for the statistic and critical value are based on convenience
and on the desire to maximize power given a significance level for the test. In this subsection, we
review how to test hypotheses for the mean of a normal population.
A test statistic, denoted T, is some function of the random sample. When we compute the sta-
tistic for a particular outcome, we obtain an outcome of the test statistic, which we will denote by t.
Given a test statistic, we can define a rejection rule that determines when H0 is rejected in
favor of H1. In this text, all rejection rules are based on comparing the value of a test statistic, t, to a
critical value, c. The values of t that result in rejection of the null hypothesis are collectively known
as the rejection region. To determine the critical value, we must first decide on a significance level
of the test. Then, given a, the critical value associated with a is determined by the distribution of T,
assuming that H0 is true. We will write this critical value as c, suppressing the fact that it depends
on a.
Testing hypotheses about the mean m from a Normal 1 m, s2 2 population is straightforward. The
null hypothesis is stated as
H0: m 5 m0, [C.31]
where m0 is a value that we specify. In the majority of applications, m0 5 0, but the general case is no
more difficult.
The rejection rule we choose depends on the nature of the alternative hypothesis. The three alter-
natives of interest are
H1: m . m0, [C.32]
H1: m , m0, [C.33]
and
H1: m 2 m0. [C.34]
Equation (C.32) gives a one-sided alternative, as does (C.33). When the alternative hypothesis is
(C.32), the null is effectively H0: m # m0, because we reject H0 only when m . m0. This is appropri-
ate when we are interested in the value of m only when m is at least as large as m0. Equation (C.34)
is a two-sided alternative. This is appropriate when we are interested in any departure from the null
hypothesis.
Consider first the alternative in (C.32). Intuitively, we should reject H0 in favor of H1 when the
value of the sample average, y, is “sufficiently” greater than m0. But how should we determine when y
is large enough for H0 to be rejected at the chosen significance level? This requires knowing the prob-
ability of rejecting the null hypothesis when it is true. Rather than working directly with y, we use its
standardized version, where s is replaced with the sample standard deviation, s:
t 5 !n 1 y 2 m0 2 /s 5 1 y 2 m0 2 /se 1 y 2 , [C.35]
where se 1 y 2 5 s/!n is the standard error of y. Given the sample of data, it is easy to obtain t. We
work with t because, under the null hypothesis, the random variable
T 5 !n 1 Y 2 m0 2 /S

58860_em_appC_hr_714-[Link] 735 10/18/18 4:52 PM

 736 Math Refresher

FIGURE C.5 Rejection region for a 5% significance level test against the one-sided
alternative m . m0.

area = .95

area = .05

0
c rejection

has a tn21 distribution. Now, suppose we have settled on a 5% significance level. Then, the critical
value c is chosen so that P 1 T . c 0 H0 2 5 .05; that is, the probability of a Type I error is 5%. Once we
have found c, the rejection rule is
t . c, [C.36]
where c is the 100 1 1 2 a 2 percentile in a tn21 distribution; as a percent, the significance level is
100 # a%. This is an example of a one-tailed test because the rejection region is in one tail of the t dis-
tribution. For a 5% significance level, c is the 95th percentile in the tn21 distribution; this is illustrated
in Figure C.5. A different significance level leads to a different critical value.
The statistic in equation (C.35) is often called the t statistic for testing H0: m 5 m0. The t statistic
measures the distance from y to m0 relative to the standard error of y, se 1 y 2 .

EXAMPLE C.4 Effect of Enterprise Zones on Business Investments

In the population of cities granted enterprise zones in a particular state [see Papke (1994) for Indiana],
let Y denote the percentage change in investment from the year before to the year after a city became
an enterprise zone. Assume that Y has a Normal 1 m, s2 2 distribution. The null hypothesis that enter-
prise zones have no effect on business investment is H0: m 5 0; the alternative that they have a posi-
tive effect is H1: m . 0. (We assume that they do not have a negative effect.) Suppose that we wish to
test H0 at the 5% level. The test statistic in this case is
y y
t5 5 . [C.37]
s/!n se 1 y 2
Suppose that we have a sample of 36 cities that are granted enterprise zones. Then, the critical value is
c 5 1.69 (see Table G.2), and we reject H0 in favor of H1 if t . 1.69. Suppose that the sample yields
y 5 8.2 and s 5 23.9. Then, t < 2.06, and H0 is therefore rejected at the 5% level. Thus, we conclude

58860_em_appC_hr_714-[Link] 736 10/18/18 4:52 PM

 Math Refresher C Fundamentals of Mathematical Statistics 737

that, at the 5% significance level, enterprise zones have an effect on average investment. The 1% criti-
cal value is 2.44, so H0 is not rejected at the 1% level. The same caveat holds here as in Example C.2:
we have not controlled for other factors that might affect investment in cities over time, so we cannot
claim that the effect is causal.

The rejection rule is similar for the one-sided alternative (C.33). A test with a significance level
of 100 # a% rejects H0 against (C.33) whenever
t , 2 c; [C.38]
in other words, we are looking for negative values of the t statistic—which implies y , m0—that are
sufficiently far from zero to reject H0.
For two-sided alternatives, we must be careful to choose the critical value so that the signifi-
cance level of the test is still a. If H1 is given by H1: m 2 m0, then we reject H0 if y is far from m0 in
absolute value: a y much larger or much smaller than m0 provides evidence against H0 in favor of H1.
A 100 # a% level test is obtained from the rejection rule
0 t 0 . c, [C.39]
where 0 t 0 is the absolute value of the t statistic in (C.35). This gives a two-tailed test. We must now
be careful in choosing the critical value: c is the 100 1 1 2 a/2 2 percentile in the tn21 distribution. For
example, if a 5 .05, then the critical value is the 97.5th percentile in the tn21 distribution. This ensures
that H0 is rejected only 5% of the time when it is true (see Figure C.6). For example, if n 5 22, then
the critical value is c 5 2.08, the 97.5th percentile in a t21 distribution (see Table G.2). The absolute
value of the t statistic must exceed 2.08 in order to reject H0 against H1 at the 5% level.
It is important to know the proper language of hypothesis testing. Sometimes, the appropriate
phrase “we fail to reject H0 in favor of H1 at the 5% significance level” is replaced with “we accept
H0 at the 5% significance level.” The latter wording is incorrect. With the same set of data, there are

FIGURE C.6 Rejection region for a 5% significance level test against the two-sided
alternative H1: m 2 m0.

area = .95

area = .025 area = .025

0
rejection rejection
region –c c region

58860_em_appC_hr_714-[Link] 737 10/18/18 4:52 PM

 738 Math Refresher

usually many hypotheses that cannot be rejected. In the earlier election example, it would be logically
inconsistent to say that H0: u 5 .42 and H0: u 5 .43 are both “accepted,” because only one of these
can be true. But it is entirely possible that neither of these hypotheses is rejected. For this reason, we
always say “fail to reject H0” rather than “accept H0.”

C-6c Asymptotic Tests for Nonnormal Populations

If the sample size is large enough to invoke the central limit theorem (see Section C-3), the mechanics
of hypothesis testing for population means are the same whether or not the population distribution is
normal. The theoretical justification comes from the fact that, under the null hypothesis,
a
T 5 !n 1 Y 2 m0 2 /S | Normal 1 0,1 2 .
Therefore, with large n, we can compare the t statistic in (C.35) with the critical values from a stan-
dard normal distribution. Because the tn21 distribution converges to the standard normal distribution
as n gets large, the t and standard normal critical values will be very close for extremely large n.
Because asymptotic theory is based on n increasing without bound, it cannot tell us whether the stan-
dard normal or t critical values are better. For moderate values of n, say, between 30 and 60, it is tra-
ditional to use the t distribution because we know this is correct for normal populations. For n!.!120,
the choice between the t and standard normal distributions is largely irrelevant because the critical
values are practically the same.
Because the critical values chosen using either the standard normal or t distribution are only
approximately valid for nonnormal populations, our chosen significance levels are also only approxi-
mate; thus, for nonnormal populations, our significance levels are really asymptotic significance lev-
els. Thus, if we choose a 5% significance level, but our population is nonnormal, then the actual
significance level will be larger or smaller than 5% (and we cannot know which is the case). When the
sample size is large, the actual significance level will be very close to 5%. Practically speaking, the
distinction is not important, so we will now drop the qualifier “asymptotic.”

EXAMPLE C.5 Race Discrimination in Hiring

In the Urban Institute study of discrimination in hiring (see Example C.3) using the data in AUDIT,
we are primarily interested in testing H0: m 5 0 against H1: m , 0 where m 5 u B 2 u W is the differ-
ence in probabilities that blacks and whites receive job offers. Recall that m is the population mean of
the variable Y 5 B 2 W, where B and W are binary indicators. Using the n 5 241 paired compari-
sons in the data file AUDIT, we obtained y 5 2.133 and se 1 y 2 5 .482/!241 < .031. The t statistic
for testing H0: m 5 0 is t 5 2.133/.031 < 24.29. You will remember from Math Refresher B that
the standard normal distribution is, for practical purposes, indistinguishable from the t distribution
with 240 degrees of freedom. The value 24.29 is so far out in the left tail of the distribution that we
reject H0 at any reasonable significance level. In fact, the .005 (one-half of a percent) critical value
(for the one-sided test) is about 22.58. A t value of 24.29 is very strong evidence against H0 in favor
of H1. Hence, we conclude that there is discrimination in hiring.

C-6d Computing and Using p-Values

The traditional requirement of choosing a significance level ahead of time means that different
researchers, using the same data and same procedure to test the same hypothesis, could wind up with
different conclusions. Reporting the significance level at which we are carrying out the test solves this
problem to some degree, but it does not completely remove the problem.

58860_em_appC_hr_714-[Link] 738 10/18/18 4:52 PM

 Math Refresher C Fundamentals of Mathematical Statistics 739

To provide more information, we can ask the following question: What is the largest significance
level at which we could carry out the test and still fail to reject the null hypothesis? This value is
known as the p-value of a test (sometimes called the prob-value). Compared with choosing a signifi-
cance level ahead of time and obtaining a critical value, computing a p-value is somewhat more diffi-
cult. But with the advent of quick and inexpensive computing, p-values are now fairly easy to obtain.
As an illustration, consider the problem of testing H0: m 5 0 in a Normal 1 m, s2 2 population.
Our test statistic in this case is T 5 !n # Y/S, and we assume that n is large enough to treat T as hav-
ing a standard normal distribution under H0. Suppose that the observed value of T for our sample is
t 5 1.52. (Note how we have skipped the step of choosing a significance level.) Now that we have
seen the value t, we can find the largest significance level at which we would fail to reject H0. This
is the significance level associated with using t as our critical value. Because our test statistic T has a
standard normal distribution under H0, we have
p-value 5 P 1 T . 1.52 0 H0 2 5 1 2 F 1 1.52 2 5 .065, [C.40]
where F 1 # 2 denotes the standard normal cdf. In other words, the p-value in this example is simply the
area to the right of 1.52, the observed value of the test statistic, in a standard normal distribution. See
Figure C.7 for illustration.
Because the p-value 5 .065, the largest significance level at which we can carry out this test and
fail to reject is 6.5%. If we carry out the test at a level below 6.5% (such as at 5%), we fail to reject H0.
If we carry out the test at a level larger than 6.5% (such as 10%), we reject H0. With the p-value at
hand, we can carry out the test at any level.
The p-value in this example has another useful interpretation: it is the probability that we observe
a value of T as large as 1.52 when the null hypothesis is true. If the null hypothesis is actually true, we
would observe a value of T as large as 1.52 due to chance only 6.5% of the time. Whether this is small
enough to reject H0 depends on our tolerance for a Type I error. The p-value has a similar interpreta-
tion in all other cases, as we will see.
Generally, small p-values are evidence against H0, because they indicate that the outcome of the
data occurs with small probability if H0 is true. In the previous example, if t had been a larger value,
say, t 5 2.85, then the p-value would be 1 2 F 1 2.85 2 < .002. This means that, if the null hypoth-
esis were true, we would observe a value of T as large as 2.85 with probability .002. How do!we

FIGURE C.7 The p-value when t 5 1.52 for the one-sided alternative m # m0.

area = .065
= p-value

0 1.52

58860_em_appC_hr_714-[Link] 739 10/18/18 4:52 PM

 740 Math Refresher

interpret this? Either we obtained a very unusual sample or the null hypothesis is false. Unless we
have a very small tolerance for Type I error, we would reject the null hypothesis. On the other hand, a
large p-value is weak evidence against H0. If we had gotten t 5 .47 in the previous example, then the
p-value 5 1 2 F 1 .47 2 5 .32. Observing a value of T larger than .47 happens with probability .32,
even when H0 is true; this is large enough so that there is insufficient doubt about H0, unless we have
a very high tolerance for Type I error.
For hypothesis testing about a population mean using the t distribution, we need detailed tables
in order to compute p-values. Table G.2 only allows us to put bounds on p-values. Fortunately, many
statistics and econometrics packages now compute p-values routinely, and they also provide calcula-
tion of cdfs for the t and other distributions used for computing p-values.

EXAMPLE C.6 Effect of Job Training Grants on Worker Productivity

Consider again the Holzer et al. (1993) data in Example C.2. From a policy perspective, there are two
questions of interest. First, what is our best estimate of the mean change in scrap rates, m? We have
already obtained this for the sample of 20 firms listed in Table C.3: the sample average of the change
in scrap rates is 21.15. Relative to the initial average scrap rate in 1987, this represents a fall in the
scrap rate of about 26.3% 1 21.15/4.38 < 2.263 2 , which is a nontrivial effect.
We would also like to know whether the sample provides strong evidence for an effect in the
population of manufacturing firms that could have received grants. The null hypothesis is H0: m 5 0,
and we test this against H1: m , 0, where m is the average change in scrap rates. Under the null, the
job training grants have no effect on average scrap rates. The alternative states that there is an effect.
We do not care about the alternative m . 0, so the null hypothesis is effectively H0: m $ 0.
Because y 5 21.15 and se 1 y 2 5 .54, t 5 21.15/.54 5 22.13. This is below the 5% critical
value of 21.73 (from a t19 distribution) but above the 1% critical value, 22.54. The p-value in this
case is computed as
p-value 5 P 1 T19 , 22.13 2 , [C.41]

FIGURE C.8 The p-value when t 5 22.13 with 19 degrees of freedom for the one-sided alter-
native m , 0.

area = p-value = .023

–2.13 0

58860_em_appC_hr_714-[Link] 740 10/18/18 4:52 PM

 Math Refresher C Fundamentals of Mathematical Statistics 741

where T19 represents a t distributed random variable with 19 degrees of freedom. The inequality is
reversed from (C.40) because the alternative has the form in (C.33). The probability in (C.41) is the
area to the left of 22.13 in a t19 distribution (see Figure C.8).
Using Table G.2, the most we can say is that the p-value is between .025 and .01, but it is closer
to .025 (because the 97.5th percentile is about 2.09). Using a statistical package, such as Stata®, we
can compute the exact p-value. It turns out to be about .023, which is reasonable evidence against H0.
This is certainly enough evidence to reject the null hypothesis that the training grants had no effect at
the 2.5% significance level (and therefore at the 5% level).

Computing a p-value for a two-sided test is similar, but we must account for the two-sided nature
of the rejection rule. For t testing about population means, the p-value is computed as
P 1 0 Tn21 0 . 0 t 0 2 5 2P 1 Tn21 . 0 t 0 2 , [C.42]
where t is the value of the test statistic and Tn21 is a t random variable. (For large n, replace Tn21 with
a standard normal random variable.) Thus, compute the absolute value of the t statistic, find the area
to the right of this value in a tn21 distribution, and multiply the area by two.
For nonnormal populations, the exact p-value can be difficult to obtain. Nevertheless, we can
find asymptotic p-values by using the same calculations. These p-values are valid for large sample
sizes. For n larger than, say, 120, we might as well use the standard normal distribution. Table G.1 is
detailed enough to get accurate p-values, but we can also use a statistics or econometrics program.

EXAMPLE C.7 Race Discrimination in Hiring

Using the matched pairs data from the Urban Institute in the AUDIT data file (n 5 241), we obtained
t 5 24.29. If Z is a standard normal random variable, P 1 Z , 24.29 2 is, for practical purposes, zero.
In other words, the (asymptotic) p-value for this example is essentially zero. This is very strong evi-
dence against H0.

Summary of How to Use p-Values:

(i) Choose a test statistic T and decide on the nature of the alternative. This determines whether
the rejection rule is t . c, t , 2c, or 0 t 0 . c.
(ii) Use the observed value of the t statistic as the critical value and compute the correspond-
ing significance level of the test. This is the p-value. If the rejection rule is of the form t . c, then
p-value 5 P 1 T . t 2 . If the rejection rule is t , 2c, then p-value 5 P 1 T , t 2 ; if the rejection rule is
0 t 0 . c, then p-value 5 P 1 0 T 0 . 0 t 0 2 .
(iii) If a significance level a has been chosen, then we reject H0 at the 100 # a% level if
p-value , a. If p-value $ a, then we fail to reject H0 at the 100 # a% level. Therefore, it is a small
p-value that leads to rejection of the null hypothesis.

C-6e The Relationship between Confidence Intervals

and Hypothesis Testing
Because constructing confidence intervals and hypothesis tests both involve probability statements, it
is natural to think that they are somehow linked. It turns out that they are. After a confidence interval
has been constructed, we can carry out a variety of hypothesis tests.
The confidence intervals we have discussed are all two-sided by nature. (In this text, we will
have no need to construct one-sided confidence intervals.) Thus, confidence intervals can be used to

58860_em_appC_hr_714-[Link] 741 10/18/18 4:52 PM

 742 Math Refresher

test against two-sided alternatives. In the case of a population mean, the null is given by (C.31), and
the alternative is (C.34). Suppose we have constructed a 95% confidence interval for m. Then, if the
hypothesized value of m under H0, m0, is not in the confidence interval, then H0: m 5 m0 is rejected
against H1: m 2 m0 at the 5% level. If m0 lies in this interval, then we fail to reject H0 at the 5% level.
Notice how any value for m0 can be tested once a confidence interval is constructed, and because a con-
fidence interval contains more than one value, there are many null hypotheses that will not be rejected.

EXAMPLE C.8 Training Grants and Worker Productivity

In the Holzer et al. example, we constructed a 95% confidence interval for the mean change in scrap
rate m as 3 22.28, 2.02 4 . Because zero is excluded from this interval, we reject H0: m 5 0 against
H1: m 2 0 at the 5% level. This 95% confidence interval also means that we fail to reject H0: m 5 22
at the 5% level. In fact, there is a continuum of null hypotheses that are not rejected given this confi-
dence interval.

C-6f Practical versus Statistical Significance

In the examples covered so far, we have produced three kinds of evidence concerning population
parameters: point estimates, confidence intervals, and hypothesis tests. These tools for learning about
population parameters are equally important. There is an understandable tendency for students to
focus on confidence intervals and hypothesis tests because these are things to which we can attach
confidence or significance levels. But in any study, we must also interpret the magnitudes of point
estimates.
The sign and magnitude of y determine its practical significance and allow us to discuss the
direction of an intervention or policy effect, and whether the estimated effect is “large” or “small.”
On the other hand, statistical significance of y depends on the magnitude of its t statistic. For testing
H0: m 5 0, the t statistic is simply t 5 y/se 1 y 2 . In other words, statistical significance depends on the
ratio of y to its standard error. Consequently, a t statistic can be large because y is large or se 1 y 2 is
small. In applications, it is important to discuss both practical and statistical significance, being aware
that an estimate can be statistically significant without being especially large in a practical sense.
Whether an estimate is practically important depends on the context as well as on one’s judgment, so
there are no set rules for determining practical significance.

EXAMPLE C.9 Effect of Freeway Width on Commute Time

Let Y denote the change in commute time, measured in minutes, for commuters in a metropolitan area
from before a freeway was widened to after the freeway was widened. Assume that Y | Normal 1 m,s2 2 .
The null hypothesis that the widening did not reduce average commute time is H0: m 5 0; the alterna-
tive that it reduced average commute time is H1: m , 0. Suppose a random sample of commuters of
size n 5 900 is obtained to determine the effectiveness of the freeway project. The average change
in commute time is computed to be y 5 23.6, and the sample standard deviation is s 5 32.7; thus,
se 1 y 2 5 32.7/!900 5 1.09. The t statistic is t 5 23.6/1.09 < 23.30, which is very statistically sig-
nificant; the p-value is about .0005. Thus, we conclude that the freeway widening had a statistically
significant effect on average commute time.
If the outcome of the hypothesis test is all that were reported from the study, it would be mis-
leading. Reporting only statistical significance masks the fact that the estimated reduction in average
commute time, 3.6 minutes, seems pretty meager, although this depends to some extent on what the
average commute time was prior to widening the freeway. To be up front, we should report the point
estimate of 23.6, along with the significance test.

58860_em_appC_hr_714-[Link] 742 10/18/18 4:52 PM

 5
HYPOT H ESI S T E ST S A N D
MOD EL SELECTI O N
§
5.1 INTRODUCTION

The linear regression model is used for three major purposes: estimation and
prediction,!which were the subjects of the previous chapter, and hypothesis testing. In
this chapter, we examine some applications of hypothesis tests using the linear regression
model. We begin with the methodological and statistical theory.!Some of this theory was
developed in Chapter 4 (including the idea of a pivotal statistic in Section 4.7.1) and in
Appendix C.7. In Section 5.2, we will extend the methodology to hypothesis testing based
on the regression model. After the theory is developed, Sections 5.3 through 5.5 will
examine some applications in regression modeling. This development will be concerned
with the implications of restrictions on the parameters of the model, such as whether a
variable is relevant (i.e., has a nonzero coefficient) or whether the regression model itself
is supported by the data (i.e., whether the data seem consistent with the hypothesis that
all of the coefficients are zero). We will primarily be concerned with linear restrictions in
this discussion. We will turn to nonlinear restrictions in Section 5.5. Section 5.6 considers
some broader types of hypotheses, such as choosing between two competing models, for
example, whether a linear or a loglinear model is better suited to the data. In each of
the cases so far, the testing procedure attempts to resolve a competition between two
theories for the data; in Sections 5.2 through 5.5 between a narrow model and a broader
one and in Section 5.6, between two arguably equal models. Section 5.7 illustrates a
particular specification test, which is essentially a test of a proposition such as the model
is correct versus the model is inadequate. This test pits the theory of the model against
some other unstated theory. Finally, Section 5.8 presents some general principles and
elements of a strategy of model testing and selection.

5.2 HYPOTHESIS TESTING METHODOLOGY

We begin the analysis with the regression model as a statement of a proposition,

y = XB + E. (5-1)
To consider a specific application, Examples 4.3 and 4.5 depicted the auction prices of
paintings,
ln Price = b1 + b2 ln Size + b3 Aspect Ratio + e. (5-2)
Some questions might be raised about the model in (5-2), fundamentally, about
the variables. It seems natural that fine art enthusiasts would be concerned about
aspect ratio, which is an element of the aesthetic quality of a painting. But the idea
113

M05_GREE1366_08_SE_C05.indd 113 2/24/17 7:23 PM

 114 PART I � The Linear Regression Model

that size should be an element of the price is counterintuitive, particularly weighed

against the surprisingly small sizes of some of the world’s most iconic paintings
such as the Mona Lisa (30″ high and 21″ wide) or Dali’s Persistence of Memory
(only 9.5″ high and 13″ wide). A skeptic might question the presence of lnSize in the
equation or, equivalently, the nonzero coefficient, b2. To settle the issue, the relevant
empirical question is whether the equation specified appears to be consistent with the
data—that is, the observed sale prices of paintings. In order to proceed, the obvious
approach for the analyst would be to fit the regression first and then examine the
estimate of b2. The test, at this point, is whether b2 in the least squares regression
is zero or not. Recognizing that the least squares slope is a random variable that
will never be exactly zero even if b2 really is, we would soften the question to be
whether the sample estimate seems to be close enough to zero for us to conclude
that its population counterpart is actually zero, that is, that the nonzero value we
observe is nothing more than noise that is due to sampling variability. Remaining to
be answered are questions including: How close to zero is close enough to reach this
conclusion? What metric is to be used? How certain can we be that we have reached
the right conclusion? (Not absolutely, of course.) How likely is it that our decision
rule, whatever we choose, will lead us to the wrong conclusion? This section will
formalize these ideas. After developing the methodology in detail, we will construct
a number of numerical examples.

5.2.1 RESTRICTIONS AND HYPOTHESES

The approach we will take is to formulate a hypothesis as a restriction on a model. Thus,

in the classical methodology considered here, the model is a general statement and a
hypothesis is a proposition that narrows that statement. In the art example in (5-2),
the narrower statement is (5-2) with the additional statement that b2 = 0—without
comment on b1 or b3. We define the null hypothesis as the statement that narrows the
model and the alternative hypothesis as the broader one. In the example, the broader
model allows the equation to contain both ln Size and Aspect Ratio—it admits the
possibility that either coefficient might be zero but does not insist upon it. The null
hypothesis insists that b2 = 0 while it also makes no comment about b1 or b3. The formal
notation used to frame this hypothesis would be
ln Price = b1 + b2 ln Size + b3AspectRatio + e,
H0 : b2 = 0, (5-3)
H1 : b2 ≠ 0.
Note that the null and alternative hypotheses, together, are exclusive and exhaustive.
There is no third possibility; either one or the other of them is true, not both.
The analysis from this point on will be to measure the null hypothesis against the
data. The data might persuade the econometrician to reject the null hypothesis. It would
seem appropriate at that point to accept the alternative. However, in the interest of
maintaining flexibility in the methodology, that is, an openness to new information,
the appropriate conclusion here will be either to reject the null hypothesis or not to
reject it. Not rejecting the null hypothesis is not equivalent to accepting it—though
the language might suggest so. By accepting the null hypothesis, we would implicitly
be closing off further investigation. Thus, the traditional, classical methodology leaves

M05_GREE1366_08_SE_C05.indd 114 2/24/17 7:23 PM

 CHAPTER 5 � Hypothesis Tests and Model Selection 115

open the possibility that further evidence might still change the conclusion. Our testing
methodology will be constructed so as either to
Reject H0 : The data appear to be inconsistent with the hypothesis with a reasonable
degree of certainty.
Do not reject H0 : The data appear to be consistent with the null hypothesis.

5.2.2 NESTED MODELS

The general approach to testing a hypothesis is to formulate a statistical model that

contains the hypothesis as a restriction on its parameters. A theory is said to have
testable implications if it implies some testable restrictions on the model. Consider, for
example, a model of investment, It,
ln It = b1 + b2it + b3 ∆pt + b4 ln Yt + b5t + et, (5-4)
which states that investors are sensitive to nominal interest rates, it, the rate of inflation,
∆pt, (the log of) real output, ln Yt, and other factors that trend upward through time,
embodied in the time trend, t. An alternative theory states that “investors care about
real interest rates.” The alternative model is
ln It = b1 + b2(it - ∆pt) + b3 ∆pt + b4 ln Yt + b5t + et. (5-5)
Although this new model does embody the theory, the equation still contains both
nominal interest and inflation. The theory has no testable implication for our model.
But, consider the stronger hypothesis, “investors care only about real interest rates.”
The resulting equation,
ln It = b1 + b2(it - ∆pt) + b4 ln Yt + b5t + et, (5-6)
is now restricted; in the context of (5-4), the implication is that b2 + b3 = 0. The stronger
statement implies something specific about the parameters in the equation that may or
may not be supported by the empirical evidence.
The description of testable implications in the preceding paragraph suggests (correctly)
that testable restrictions will imply that only some of the possible models contained in
the original specification will be valid; that is, consistent with the theory. In the example
given earlier, (5-4) specifies a model in which there are five unrestricted parameters
(b1, b2, b3, b4, b5). But (5-6) shows that only some values are consistent with the theory,
that is, those for which b3 = -b2. This subset of values is contained within the unrestricted
set. In this way, the models are said to be nested. Consider a different hypothesis, “investors
do not care about inflation.” In this case, the smaller set of coefficients is (b1, b2, 0, b4, b5).
Once again, the restrictions imply a valid parameter space that is “smaller” (has fewer
dimensions) than the unrestricted one. The general result is that the hypothesis specified
by the restricted model is contained within the unrestricted model.
Now, consider an alternative pair of models: Model0 : “Investors care only about
inflation”; Model1 : “Investors care only about the nominal interest rate.” In this case, the
two parameter vectors are (b1, 0, b3, b4, b5) by Model0 and (b1, b2, 0, b4, b5) by Model1.
The two specifications are both subsets of the unrestricted model, but neither model is
obtained as a restriction on the other. They have the same number of parameters; they
just contain different variables. These two models are nonnested. For the present, we are
concerned only with nested models. Nonnested models are considered in Section 5.6.

M05_GREE1366_08_SE_C05.indd 115 2/24/17 7:23 PM

 116 PART I � The Linear Regression Model
5.2.3 TESTING PROCEDURES

In the example in (5-2), intuition suggests a testing approach based on measuring the
data against the hypothesis. The essential methodology provides a reliable guide to
testing hypotheses in the setting we are considering in this chapter. Broadly, the analyst
follows the logic, “What type of data will lead me to reject the hypothesis?” Given the
way the hypothesis is posed in Section 5.2.1, the question is equivalent to asking what
sorts of data will support the model. The data that one can observe are divided into a
rejection region and an acceptance region. The testing procedure will then be reduced to
a simple up or down examination of the statistical evidence. Once it is determined what
the rejection region is, if the observed data appear in that region, the null hypothesis is
rejected. To see how this operates in practice, consider, once again, the hypothesis about
size in the art price equation. Our test is of the hypothesis that b2 equals zero. We will
compute the least squares slope. We will decide in advance how far the estimate of b2
must be from zero to lead to rejection of the null hypothesis. Once the rule is laid out,
the test, itself, is mechanical. In particular, for this case, b2 is far from zero if b2 7 b0+
2
or b2 6 b0- 2 . If either case occurs, the hypothesis is rejected. The crucial element is that
the rule is decided upon in advance.

5.2.4 SIZE, POWER, AND CONSISTENCY OF A TEST

Because the testing procedure is determined in advance and the estimated coefficient(s)
in the regression are random, there are two ways the Neyman–Pearson method can
make an error. To put this in a numerical context, the sample regression corresponding
to (5-2) appears in Table 4.7. The estimate of the coefficient on ln Area is 1.31638 with
an estimated standard error of 0.09205. Suppose the rule to be used to test is decided
arbitrarily (at this point—we will formalize it shortly) to be: If b2 is greater than +1.0
or less than -1.0, then we will reject the hypothesis that the coefficient is zero (and
conclude that art buyers really do care about the sizes of paintings). So, based on this
rule, we will, in fact, reject the hypothesis. However, because b2 is a random variable,
there are the following possible errors:
Type I error: b2 = 0, but we reject the hypothesis that b2 = 0.
The null hypothesis is incorrectly rejected.
Type II error: b2 ≠ 0, but we do not reject the hypothesis that b2 = 0.
The null hypothesis is incorrectly retained.
The probability of a Type I error is called the size of the test. The size of a test is the
probability that the test will incorrectly reject the null hypothesis. As will emerge later,
the analyst determines this in advance. One minus the probability of a Type II error is
called the power of a test. The power of a test is the probability that it will correctly reject
a false null hypothesis. The power of a test depends on the alternative. It is not under
the control of the analyst. To consider the example once again, we are going to reject
the hypothesis if $ b2 $ 7 1. If b2 is actually 1.5, then based on the results we’ve seen, we
are quite likely to find a value of b2 that is greater than 1.0. On the other hand, if b2 is
only 0.3, then it does not seem likely that we will observe a sample value greater than
1.0. Thus, again, the power of a test depends on the actual parameters that underlie the
data. The idea of power of a test relates to its ability to find what it is looking for.

M05_GREE1366_08_SE_C05.indd 116 2/24/17 7:23 PM

 CHAPTER 5 � Hypothesis Tests and Model Selection 117

A test procedure is consistent if its power goes to 1.0 as the sample size grows to
infinity. This quality is easy to see, again, in the context of a single parameter, such as
the one being considered here. Because least squares is consistent, it follows that as
the sample size grows, we will be able to learn the exact value of b2, so we will know if
it is zero or not. Thus, for this example, it is clear that as the sample size grows, we will
know with certainty if we should reject the hypothesis. For most of our work in this text,
we can use the following guide: A testing procedure about the parameters in a model
is consistent if it is based on a consistent estimator of those parameters. Nearly all our
work in this book is based on consistent estimators. Save for the latter sections of this
chapter, where our tests will be about the parameters in nested models, our tests will be
consistent as well.

5.2.5 A METHODOLOGICAL DILEMMA: BAYESIAN VERSUS CLASSICAL TESTING

As we noted earlier, the testing methodology we will employ here is an all-or-nothing

proposition. We will determine the testing rule(s) in advance, gather the data, and either
reject or not reject the null hypothesis. There is no middle ground. This presents the
researcher with two uncomfortable dilemmas. First, the testing outcome, that is, the
sample data might be uncomfortably close to the boundary of the rejection region.
Consider our example. If we have decided in advance to reject the null hypothesis if
b2 7 1.00, and the sample value is 0.9999, it will be difficult to resist the urge to reject the
null hypothesis anyway, particularly if we entered the analysis with a strongly held belief
that the null hypothesis is false. That is, intuition notwithstanding, we are unconvinced
that art buyers really do care about size. Second, the methodology we have laid out here
has no way of incorporating other studies. To continue our example, if we were the tenth
team of analysts to study the art market, and the previous nine had decisively rejected
the hypothesis that b2 = 0, we will find it very difficult not to reject that hypothesis even
if our evidence suggests, based on our testing procedure, that we should not.
This dilemma is built into the classical testing methodology. There is a middle
ground. The Bayesian methodology!that we will discuss in Chapter 16 does not face
this dilemma because Bayesian analysts never reach a firm conclusion. They merely
update their priors. Thus, in the first case noted, in which the observed data are close
to the boundary of the rejection region, the analyst will merely be updating the prior
with slightly less persuasive evidence than might be hoped for. But the methodology
is comfortable with this. For the second instance, we have a case in which there is a
wealth of prior evidence in favor of rejecting H0. It will take a powerful tenth body of
evidence to overturn the previous nine conclusions. The results of the tenth study (the
posterior results) will incorporate not only the current evidence, but the wealth of prior
data as well.

5.3 THREE APPROACHES TO TESTING HYPOTHESES

We will consider three approaches to testing hypotheses, Wald tests, fit based tests, and
Lagrange multiplier tests. The hypothesis characterizes the population. If the hypothesis
is correct, then the sample statistics should mimic that description. To continue our
earlier example, if the hypothesis that states that a certain coefficient in a regression
model equals zero is correct, then the least squares estimate of that coefficient should

M05_GREE1366_08_SE_C05.indd 117 2/24/17 7:23 PM

 118 PART I � The Linear Regression Model

be close to zero, at least within sampling variability. The tests will follow that logic as
follows:

� Wald tests: The hypothesis states that B obeys some restriction(s), which we might
state generally as c(B) = 0. The least squares estimator, b, is a consistent estimator
of B. If the hypothesis is correct, then c(b) should be close to zero. For the example
of a single coefficient, if the hypothesis that bk equals zero is correct, then bk should
be close to zero. The Wald test measures how close c(b) is to zero. The Wald test is
based on estimation of the unrestricted model—the test measures how close the
estimated unrestricted model is to the hypothesized restrictions.
� Fit based tests: We obtain the best possible fit—highest R2 (or smallest sum of
squared residuals)—by using least squares without imposing the restrictions.
Imposing the restrictions will degrade the fit of the model to the data. For example,
when we impose bk = 0 by leaving xk out of the model, we should expect R2 to fall.
The empirical device to use for testing the hypothesis will be a measure of how much
R2 falls when we impose the restrictions. This test procedure compares the fit of the
restricted model to that of the unrestricted model.
� Lagrange multiplier (LM) tests: The LM test is based on the restricted model. The
logic of the test is based on the general result that with the restrictions imposed,
if those restrictions are incorrect, then we will be able to detect that failure in
a measurable statistic. For the example of a single coefficient, bk, in a multiple
regression, the LM approach for the test will be based on the residuals from the
regression that omits xk. If bk actually is not zero, then those residuals, say ei(k),
which contain bkxik, will be correlated with xk. The test statistic will be based on
that correlation. The test procedure is based on the estimates of the restricted
model.

IMPORTANT ASSUMPTIONS
To develop the testing procedures in this section, we will begin by assuming homosce-
dastic, normally distributed disturbances—Assumptions A4 and A6 in Table 4.1.!As
we saw in Chapter 4, with these assumptions, we are able to obtain the exact distribu-
tions of the test statistics. In Section 5.4, we will develop an alternative set of results
that allows us to proceed without Assumptions A4 and A6. It is useful to keep the
distinction between the underlying theory of the testing procedures and the practical
mechanics of inferences based on asymptotic approximations and robust covariance
matrices. Robust inference is an improvement on the received procedures based on
large-sample approximations to conventional statistics that allow conclusions to be
drawn in a broader set of circumstances. For example, the conventional “F statistic”
examined in Section 5.3.1B derives specifically from Assumptions A4 and A6.
Cameron and Miller (2015, Sec. VII.A) in their survey of cluster robust inference!(see
Section 4.5.3) examine reconstruction of the F statistic in the broader context of
nonnormality and clustered sampling.

The general linear hypothesis is a set of J restrictions on the linear regression model,
y = XB + E.

M05_GREE1366_08_SE_C05.indd 118 2/24/17 7:23 PM

 CHAPTER 5 � Hypothesis Tests and Model Selection 119

The restrictions are written

r11b1 + r12b2 + g + r1KbK = q1
r21b1 + r22b2 + g + r2KbK = q2 (5-7)
g
rJ1b1 + rJ2b2 + g + rJKbK = qJ.
The general case can be written in the matrix form,
RB = q. (5-8)
Each row of R is the coefficients in one of the restrictions. Typically, R will have only one
or a few rows and numerous zeros in each row. The hypothesis implied by the restrictions
is written
H0 : RB - q = 0, H1 : RB - q ≠ 0.
Some examples would be as follows:
1. One of the coefficients is zero, bj = 0,
R = [0 0 g 1 0 g 0]; q = 0.
2. Two of the coefficients are equal, bk = bj,
R = [0 0 1 g -1 g 0]; q = 0.
3. A set of the coefficients sum to one, b2 + b3 + b4 = 1,
R = [0 1 1 1 0 g]; q = 1.
4. A subset of the coefficients are all zero, b1 = 0, b2 = 0, and b3 = 0,

1 0 0 0 g 0 0
R = C0 1 0 0 g 0 S = [I $ 0]; q = C 0 S .
0 0 1 0 g 0 0
5. Several linear restrictions, b2 + b3 = 1, b4 + b6 = 0, and b5 + b6 = 0,

0 1 1 0 0 0 1
R = C0 0 0 1 0 1S; q = C0S.
0 0 0 0 1 1 0
6. All the coefficients in the model except the constant term are zero,
R = [0$ IK-1]; q = 0.
The matrix R has K columns to be conformable with B, J rows for a total of
J restrictions, and full row rank, so J must be less than or equal to K. The rows of
R!must be linearly independent. Although it does not violate the condition, the case
of J = K must also be ruled out. If the K coefficients satisfy J = K restrictions, then
R! is square and nonsingular and B = R-1q. There is no estimation or inference
problem. The restriction RB = q imposes J restrictions on K otherwise free
parameters. Hence, with the restrictions imposed, there are, in principle, only K - J
free parameters remaining.

M05_GREE1366_08_SE_C05.indd 119 2/24/17 7:23 PM

 120 PART I � The Linear Regression Model

We will want to extend the methods to nonlinear restrictions. In example 5.6 below,
the hypothesis takes the form H0 : bj/bk = bl/bm. The general nonlinear hypothesis
involves a set of J possibly nonlinear restrictions,
c(B) = q, (5-9)
where c(B) is a set of J nonlinear functions of B. The linear hypothesis is a special case.
The counterpart to our requirements for the linear case are that, once again, J be strictly
less than K, and the matrix of derivatives,
G(B) = 0c(B)/0B′, (5-10)
have full row rank. This means that the restrictions are functionally independent. In
the linear case, G(B) is the matrix of constants, R, that we saw earlier and functional
independence is equivalent to linear independence. We will consider nonlinear
restrictions in detail in Section 5.5. For the present, we will restrict attention to the
general linear hypothesis.

5.3.1 WALD TESTS BASED ON THE DISTANCE MEASURE

The Wald test is the most commonly used procedure. It is often called a significance test.
The operating principle of the procedure is to fit the regression without the restrictions,
and then assess whether the results appear, within sampling variability, to agree with
the hypothesis.

5.3.1.a Testing a Hypothesis about a Coefficient

The simplest case is a test of the value of a single coefficient. Consider, once again, the
art market example in Section 5.2. The null hypothesis is
H0 : b2 = b02,
where b02 is the hypothesized value of the coefficient, in this case, zero. The Wald distance
of a coefficient estimate from a hypothesized value is the distance measured in standard
deviation units. For this case, the distance of bk from b0k would be
bk - b0k
Wk = . (5-11)
2s2S kk
As we saw in (4-45), Wk !(which we called zk before) has a standard normal distribution
assuming that E[bk] = b0k. Note that if E[bk] is not equal to b0k, then Wk still has a normal
distribution, but the mean is not zero. In particular, if E[bk] is b1k which is different from
b0k, then
b1k - b0k
E{Wk $ E[bk] = b1k} = . (5-12)
2s2S kk
(For example, if the hypothesis is that bk = b0k = 0, and bk does not equal zero, then
the expected value of Wk = bk/2s2S kk will equal b1k/2s2S kk, which is not zero.) For
purposes of using Wk to test the hypothesis, our interpretation is that if bk does equal b0k,
then bk will be close to b0k, with the distance measured in standard error units. Therefore,
the logic of the test, to this point, will be to conclude that H0 is incorrect—should be
rejected—if Wk is “large” in absolute value.

M05_GREE1366_08_SE_C05.indd 120 2/24/17 7:23 PM

 CHAPTER 5 � Hypothesis Tests and Model Selection 121

Before we determine a benchmark for large, we note that the Wald measure
suggested here is not usable because s2 is not known. It is estimated by s2. Once
again,!invoking our results from Chapter 4, if we compute Wk using the sample estimate
of s2, we obtain
bk - b0k
tk = . (5-13)
2s2S kk
Assuming that bk does indeed equal b0k, that is, “under the assumption of the null
hypothesis,” tk has a t distribution with n - K degrees of freedom. [See (4-47).] We can
now construct the testing procedure. The test is carried out by determining in advance
the desired confidence with which we would like to draw the conclusion—the standard
value is 95%. Based on (5-13), we can say that
Prob{-t *(1-a/2),[n-K] 6 tk 6 +t *(1-a/2),[n-K]},
where t*(1-a/2),[n-K] is the appropriate critical value from the t table. By this construction,
if the null hypothesis is true, then finding a sample value of tk that falls outside this range
is unlikely. The test procedure states that it is so unlikely that we would conclude that
it could not happen if the hypothesis were correct, so the hypothesis must be incorrect.
A common test is the hypothesis that a parameter equals zero—equivalently, this is
a test of the relevance of a variable in the regression. To construct the test statistic, we set
b0k to zero in (5-13) to obtain the standard t ratio, tk = bk/sbk. This statistic is reported in
the regression results in several of our earlier examples,!such as Example 4.10 where the
regression results for the model in (5-2) appear. This statistic is usually labeled the t ratio
for the estimator bk. If $ bk $ /sbk 7 t(1-a/2),[n-K], where t(1-a/2),[n-K] is the 100(1 - a/2)
% critical value from the t distribution with (n - K) degrees of freedom, then the
null hypothesis that the coefficient is zero is rejected and the coefficient (actually, the
associated variable) is said to be statistically significant. The value of 1.96, which would
apply for the 95% significance level in a large sample, is often used as a benchmark value
when a table of critical values is not immediately available. The t ratio for the test of the
hypothesis that a coefficient equals zero is a standard part of the regression output of
most computer programs.
Another view of the testing procedure is useful. Also based on (4-48) and (5-13),
we formed a confidence interval for bk as bk { t * sk. We may view this interval as the
set of plausible values of bk with a confidence level of 100(1-a)%, where we choose a,
typically 5%. The confidence interval provides a convenient tool for testing a hypothesis
about bk, because we may simply ask whether the hypothesized value, b0k, is contained in
this range of plausible values. The complement of the confidence interval is the rejection
region for this test.
Example 5.1 Art Appreciation
Regression results for the model in (5-3) based on a sample of 430 sales of Monet
paintings!appear in Table 4.7 in Example 4.9. The estimated coefficient on ln Area is 1.33372
with an estimated standard error of 0.09205. The distance of the estimated coefficient from
zero is 1.31638/0.092 - 5 = 14.16. Because this is far larger than the 95% critical value of
1.96, we reject the hypothesis that b2 equals zero; evidently buyers of Monet paintings do care
about size. In contrast, the coefficient on Aspect Ratio is - 0.09623 with an estimated standard
error of 0.16706, so the associated t ratio for the test of H0 : b3 = 0 is only - 0.61. Given that
this is well under 1.96, we conclude that art buyers (of Monet paintings) do not care about the

M05_GREE1366_08_SE_C05.indd 121 2/24/17 7:23 PM

 122 PART I � The Linear Regression Model

aspect ratio of the paintings. As a final consideration, we examine another (equally bemusing)
hypothesis, whether auction prices are inelastic H0 : b2 … 1 or elastic H1 : b2 7 1 with respect
to area. This is a one-sided test. Using our guideline for formulating the test, we will reject the
null hypothesis if the estimated coefficient is sufficiently larger than 1.0. To maintain a test of
size 0.05, we will then place all of the area for the rejection region to the right of 1.0; the critical
value from the table is 1.645. The test statistic is (1.31638 - 1)/0.09205 = 3.437 7 1.645.
Thus, we will reject this null hypothesis as well.

Example 5.2 Earnings Equation

Appendix Table F5.1 contains the 753 observations used in Mroz’s (1987) study of the labor
supply behavior of married women. Of the 753 individuals in the sample, 428 were participants
in the formal labor market. For these individuals, we will fit a semilog earnings equation of the
form!suggested in Example 2.2:
ln earnings = b1 + b2age + b3age2 + b4education + b5kids + e,
where earnings is hourly wage times hours worked, education is measured in years of
schooling, and kids is a binary variable which equals one if there are children under 18 in
the household. (See the data description in Appendix F for details.) Regression results are
shown in Table 5.1. There are 428 observations and 5 parameters, so the t statistics have
(428- 5) = 423 degrees of freedom. For 95% significance levels, the standard normal value of
1.96 is appropriate when the degrees of freedom are this large. By this measure, all variables
are statistically significant and signs are consistent with expectations. It will be interesting
to investigate whether the effect of kids is on the wage or hours, or both. We interpret the
schooling variable to imply that an additional year of schooling is associated with a 6.7%
increase in earnings. The quadratic age profile suggests that for a given education level and
family size, earnings rise to a peak at -b2/(2b3) which is about 43 years of age, at which point
they begin to decline. Some points to note: (1) Our selection of only those individuals who had
positive hours worked is not an innocent sample selection mechanism. Because individuals

TABLE 5.1 Regression Results for an Earnings Equation

Sum of squared residuals: 599.4582
R2 based on 428 observations 0.040944
Standard error of the regression: 1.19044
Variable Coefficient Standard Error t Ratio
Constant 3.24009 1.7674 1.833
Age 0.20056 0.08386 2.392
Age2 -0.0023147 0.00098688 -2.345
Education 0.067472 0.025248 2.672
Kids - 0.35119 0.14753 -2.380
Estimated Covariance Matrix for b(e−n = times 10−n)
Constant Age Age2 Education Kids
Constant 3.12381
Age -0.13409 0.0070325
Age2 0.0016617 -8.23237e- 5 9.73928e-7
Education -0.0092609 5.08549e- 5 -4.96761e-7 0.00063729
Kids 0.026749 -0.0026412 3.84102e-5 -5.46193e- 5 0.021766

M05_GREE1366_08_SE_C05.indd 122 2/24/17 7:23 PM

 CHAPTER 5 � Hypothesis Tests and Model Selection 123

chose whether or not to be in the labor force, it is likely (almost certain) that earnings potential
was a significant factor, along with some other aspects we will consider in Chapter 19. (2) The
earnings equation is a mixture of a labor supply equation—hours worked by the individual—
and a labor demand outcome—the wage is, presumably, an accepted offer. As such, it is
unclear what the precise nature of this equation is. Presumably, it is a hash of the equations
of an elaborate structural equation system.!(See Example 10.1 for discussion.)
5.3.1.b The F Statistic

We now consider testing a set of J linear restrictions stated in the null hypothesis,
H0 :RB - q = 0,
against the alternative hypothesis,
H1 :RB - q ≠ 0.
Given the least squares estimator b, our interest centers on the discrepancy vector
Rb - q = m. It is unlikely that m will be exactly 0. The statistical question is whether
the deviation of m from 0 can be attributed to sampling variability or whether it is
significant. Because b is normally distributed [see Section 4.3.6] and m is a linear function
of b, m is also normally distributed. If the null hypothesis is true, then RB - q = 0 and
m has mean vector
E[m$ X] = RE[b $ X] - q = RB - q = 0
and covariance matrix
Var[m$ X] = Var[Rb - q $ X] = R{Var[b $ X]}R′ = R[s2(X′X) -1]R′.
We can base a test of H0 on the Wald criterion. Conditioned on X, we find:
W = m′{Var[m$ X]} -1m
= (Rb - q)′{R[s2(X′X) -1]R′}-1 (Rb - q) (5-14)
∼ x2[J].
The statistic W has a chi-squared distribution with J degrees of freedom if the hypothesis
is correct.1 Intuitively, the larger m is—that is, the worse the failure of least squares to
satisfy the restrictions—the larger the chi-squared statistic. Therefore, a large chi-squared
value will weigh against the hypothesis.
The chi-squared statistic in (5-14) is not usable because of the unknown s2. By using
s instead of s2 and dividing the result by J, we obtain a usable F statistic with J and
2

n - K degrees of freedom,
W s2
F = = (Rb - q)′{R[s2(X′X) -1]R′}-1 (Rb - q)/J. (5-15)
J s2
The F statistic for testing the general linear hypothesis is simply the feasible Wald
statistic, divided by J:
(Rb - q)′{R[s2(X′X) -1]R′}-1 (Rb - q)
F[J, n - K $ X] = . (5-16)
J
1
This calculation is an application of the full rank quadratic form!of Section B.11.6. Note that although the
chi-squared distribution is conditioned on X, it is also free of X.

M05_GREE1366_08_SE_C05.indd 123 2/24/17 7:23 PM

 124 PART I � The Linear Regression Model

For testing one linear restriction of the form

H0 : r1b1 + r2b2 + g + rKbK = r′B = q,
(usually, some of the r’s will be zero), the F statistic is
(Σ jrjbj - q)2
F[1, n - K] = .
Σ j Σ krjrk Est. Cov[bj, bk]
If the hypothesis is that the jth coefficient is equal to a particular value, then R has a
single row with a one in the jth position and zeros elsewhere, R[s2(X′X) -1]R′ is the jth
diagonal element of the estimated covariance matrix, and Rb - q is (bj - q). The F
statistic is then
(bj - q)2
F[1, n - K] = .
Est. Var[bj]
Consider an alternative approach. The sample estimate of r′B is
r1b1 + r2b2 + g + rKbK = r′b = qn .
If qn differs significantly from q, then we conclude that the sample data are not consistent
with the hypothesis. It is natural to base the test on
qn - q
t = . (5-17)
se(qn )
We require an estimate of the standard error of qn . Because qn is a linear function of b
and we have an estimate of the covariance matrix of b, s2(X′X) -1, we can estimate the
variance of qn with
Est. Var[qn $ X] = r′[s2 (X′X) -1]r.
The denominator of t is the square root of this quantity. In words, t is the distance in
standard error units between the hypothesized function of the true coefficients and
the same function of the estimates of them. If the hypothesis is true, then the estimates
should reflect that, at least within the range of sampling variability. Thus, if the absolute
value of the preceding t ratio is larger than the appropriate critical value, then doubt is
cast on the hypothesis.
There is a useful relationship between the statistics in (5-16) and (5-17). We can write
the square of the t statistic as
(qn - q)2 (r′b - q){r′[s2(X′X) -1]r} -1(r′b - q)
t2 = = . (5-18)
Var(qn - q $ X) 1
It follows, therefore, that for testing a single restriction, the t statistic is the square root
of the F statistic that would be used to test that hypothesis. (The sign of the t statistic is
lost, of course.)
Example 5.3 Restricted Investment Equation
Section 5.2.2 suggested a theory about the behavior of investors: They care only about real
interest rates. If investors were only interested in the real rate of interest, then equal increases
in interest rates and the rate of inflation would have no independent effect on investment.
The null hypothesis is
H0 : b2 + b3 = 0.

M05_GREE1366_08_SE_C05.indd 124 2/24/17 7:23 PM