Chapter Two

Simple Regression Model

2.1 Interpretation of Regression

Regression analysis is concerned with the study of the dependence of one variable, the dependent
variable, on one or more other variables, the explanatory variables, with a view to estimating and/or
predicting the mean or average value of the former in terms of the known or fixed values of the
latter. Our objective in regression analysis is to find out how the average value of the dependent
variable (or regressand) varies with the given value of the explanatory variable (or regressor).

A simple regression model is a model in which the dependent variable (the regressand) is related
to a single explanatory variable (the regressor). It is also called the two-variable regression model
or bivariate regression model because it relates the two variables X and Y.

The variables Y and X have several different names used interchangeably, as follows: Y is called
the dependent variable, the explained variable, the response variable, the predicted variable,
outcome, controlled variable or the regressand; X is called the independent variable, the
explanatory variable, the control variable, the predictor variable, covariate or the regressor.

2.2 The Concept of Population Regression Function (PRF)

The Population Regression Function (PRF) expresses the expected (mean) value of the dependent
variable Y given the value of the explanatory variable X. The expected value, or expectation, or
population mean of a random variable Y is denoted by the symbol E(Y). E(Y | X), which is read
as the expected value of Y given the value of X.

Example: Let us consider a total population of 60 families in a hypothetical community and their
weekly income (X) and weekly consumption expenditure (Y), both in dollars. The 60 families are
divided into 10 income groups (from $80 to $260) and the weekly expenditures of each family in
the various groups are as shown in the table.

Page | 1
 Table 2.1: Weekly Family Income and Consumption

Corresponding to the weekly income level of $80, the mean consumption expenditure is $65, while
corresponding to the income level of $200, it is $137. In all we have 10 mean values for the 10 sub
populations of Y. We call these mean values conditional expected values, as they depend on the
given values of the (conditioning) variable X. It is important to distinguish these conditional
expected values from the unconditional expected value of weekly consumption expenditure, E(Y).
If we add the weekly consumption expenditures for all the 60 families in the population and divide
this number by 60, we get the number $121.20 ($7272/60), which is the unconditional mean, or
expected, value of weekly consumption expenditure, E(Y); it is unconditional in the sense that in
arriving at this number we have disregarded the income levels of the various families.

Figure 2.1: Conditional distribution of expenditure for various levels of income

Page | 2
The dark circled points in Figure 2.1 show the conditional mean values of Y against the various X
values. If we join these conditional mean values, we obtain what is known as the population
regression line (PRL), or more generally, the population regression curve. More simply, it is the
regression of Y on X. The adjective “population” comes from the fact that we are dealing in this
example with the entire population of 60 families.

Geometrically, then, a population regression curve is simply the locus of the conditional means of
the dependent variable for the fixed values of the explanatory variable(s). More simply, it is the
curve connecting the means of the subpopulations of Y corresponding to the given values of the
regressor X. This figure shows that for each X (i.e., income level) there is a population of Y values
(weekly consumption expenditures) that are spread around the (conditional) mean of those Y
values. For simplicity, we are assuming that these Y values are distributed symmetrically around
their respective (conditional) mean values. And the regression line (or curve) passes through these
(conditional) mean values.

Therefore, it is clear that each conditional mean E(Y | Xi) is a function of Xi, where Xi is a given
value of X. Symbolically,

where f(Xi) denotes some function of the explanatory variable X. In our example, E(Y | Xi) is a
linear function of Xi. This is known as the conditional expectation function (CEF) or population
regression function (PRF) or population regression (PR) for short. It states merely that the expected
value of the distribution of Y given Xi is functionally related to Xi. In simple terms, it tells how
the mean or average response of Y varies with X. What form does the function f (Xi) assume? This
is an important question because in real situations we do not have the entire population available
for examination. The functional form of the PRF is therefore an empirical question, although in
specific cases theory may have something to say. For example, an economist might posit that
consumption expenditure is linearly related to income. Therefore, as a first approximation or a
working hypothesis, we may assume that the PRF E(Y | Xi) is a linear function of Xi, say, of the
type

(1)

Page | 3
2.3 Stochastic Specification of PRF (Stochastic PRF)

In real life, not all observations of Y lie exactly on this mean line. There are many unobserved that
cause actual 𝑌𝑖 values to deviate from their expected values. To account for this random variation,
we add a stochastic (random) error term, ui.

Therefore, we can express the deviation of an individual Yi around its expected value as follows:

(2)

where the deviation ui is an unobservable random variable taking positive or negative values.
Technically, ui is known as the stochastic disturbance or stochastic error term. How do we interpret
Equation 2? We can say that the expenditure of an individual family, given its income level, can
be expressed as the sum of two components: (a) E(Y | Xi), which is simply the mean consumption
expenditure of all the families with the same level of income. This component is known as the
systematic, or deterministic, component, and (b) ui, which is the random, or nonsystematic,
component. We shall examine shortly the nature of the stochastic disturbance term, but for the
moment assume that it is a surrogate or proxy for all the omitted or neglected variables that may
affect Y but are not (or cannot be) included in the regression model.

If E(Y | Xi) is assumed to be linear in Xi, Eq. 2 may be written as

(3)

Equation 3 posits that the consumption expenditure of a family is linearly related to its income
plus the disturbance term. Thus, the individual consumption expenditures, given X =$80 can be
expressed as

Page | 4
Now if we take the expected value of Eq. 2 on both sides, we obtain

(4)
where use is made of the fact that the expected value of a constant is that constant itself. Notice
carefully that in Equation 4, we have taken the conditional expectation, conditional upon the given
X’s. Since E(Yi | Xi) is the same thing as E(Y | Xi), Eq. 4 implies that

2.4 Sample Regression Function (SRF)

Now, analogously to the PRF that underlies the population regression line, we can develop the
concept of the sample regression function (SRF) to represent the sample regression line. The
sample counterpart of Eq. 1 PRF may be written as

Note that an estimator, also known as a (sample) statistic, is simply a rule or formula or method
that tells how to estimate the population parameter from the information provided by the sample
at hand. A particular numerical value obtained by the estimator in an application is known as an
estimate. It should be noted that an estimator is random, but an estimate is nonrandom.

Now just as we expressed the PRF in two equivalent forms, Eq. 1 and Eq. 3, we can express the
SRF in Equation 2.6.1 in its stochastic form as follows:

Page | 5
where, in addition to the symbols already defined, denotes the (sample) residual term.
Conceptually is analogous to ui and can be regarded as an estimate of ui. It is introduced in
the SRF for the same reasons as ui was introduced in the PRF.

To sum up, then, we find our primary objective in regression analysis is to estimate the PRF

on the basis of the SRF

2.5 The Assumptions of Classical Linear Regression Model

The Gaussian, standard, or classical linear regression model (CLRM), which is the cornerstone of
most econometric theory, makes assumptions. It is classical in the sense that it was developed first
by Gauss in 1821 and since then has served as a norm or a standard against which may be compared
the regression models that do not satisfy the Gaussian assumptions.

i) The regression model is linear in the parameters

The regression model is linear in the parameters (the parameters are raised to the first power only)
though it may or may not be linear in the variables.

For example:
a) Yi = β1 + β2Xi + 𝑢𝑖 is linear in both parameters and variable X
b) Yi = β1 + β2X𝑖2 + 𝑢𝑖 and Yi = β1 + β2lnXi + 𝑢𝑖 are linear in parameters but not in the variable
because the variable X appears with a power or index of 2 and with the natural logarithm.
c) Yi = β1 + β22 Xi + 𝑢𝑖 is nonlinear in parameter, but linear in variable X

ii) Zero Mean Value of Disturbance 𝑢𝑖

The mean or expected value of the random disturbance term ui given the value of Xi is zero.

iii) Homoscedasticity or Constant Variance of 𝑢𝑖 :

The variance of the error or disturbance term is the same regardless of the value of X.

Page | 6
This states that the variance of 𝑢𝑖 for each Xi (i.e., the conditional variance of ui) is some positive
constant number equal to σ2. It represents the assumption of homoscedasticity, or equal (homo)
spread (scedasticity) or equal variance. Put simply, the variation around the regression line (which
is the line of average relationship between Y and X) is the same across the X values; it neither
increases nor decreases as X varies. In contrast, where the conditional variance of the Y population
varies with X. This situation is known appropriately as heteroscedasticity, or unequal spread, or
variance.

iv) No Autocorrelation between the Disturbances

Given any two X values, Xi and Xj (i ≠ j), the correlation between any two 𝑢𝑖 and 𝑢𝑗 are sampled
independently.

In words, this postulates that the disturbances ui and uj are uncorrelated. Technically, this is the
assumption of no serial correlation, or no autocorrelation.

v) Zero covariance between 𝑢𝑖 and Xi, or E(𝑢𝑖 , Xi) = 0.

This states that the disturbance u and explanatory variable X are uncorrelated. The rationale for
this assumption is as follows: When we expressed the PRF, we assumed that X and u (which may
represent the influence of all the omitted variables) have separate (and additive) influence on Y.
But if X and u are correlated, it is not possible to assess their individual effects on Y. This

Page | 7
assumption is automatically fulfilled if X variable is nonrandom or nonstochastic and cov(ui,
Xi)=[Xi − E(Xi)]E[ui − E(ui)] = 0 holds.

vi) Variability in X values

The X values in a given sample must not all be the same. Technically, var (X) must be a positive
number. Furthermore, there can be no outliers in the values of the X variable, that is, extreme
values that are very large/small in relation to the rest of the observations.

vii) X values are fixed in repeated sampling

Values taken by the regressor X are considered fixed in repeated samples. More technically, X is
assumed to be nonstochastic.

viii) There is no perfect multicollinearity

That is, there are no perfect linear relationships among the explanatory variables.

ix) The number of observations n must be greater than the number of parameters to be
estimated. Alternatively, the number of observations n must be greater than the number
of explanatory variables.

2.6 Methods of Estimation

The parameters of the simple linear regression model can be estimated by various methods. Three
of the most commonly used methods are: Ordinary least squares method (OLS), Maximum
likelihood method (MLM), and Method of Moments (MM). By and large, it is the method of OLS
that is used extensively in regression analysis primarily because it is intuitively appealing and
mathematically much simpler.

2.6.1 Ordinary Least Squares Method

The method of ordinary least squares is attributed to Carl Friedrich Gauss, a German
mathematician.

Recall the stochastic PRF:

Page | 8
However, the PRF is not directly observable. We estimate it from the SRF:

̂𝑖 is the estimated (conditional mean) value of Yi.

where 𝑌

But how is the SRF itself determined? To see this, let us proceed as follows.

̂1 and 𝛽
Estimation of 𝛽 ̂2 by the OLS method involves finding values for the estimates 𝛽
̂1 and 𝛽
̂2
̂1 and 𝛽
which minimize the sum of the squared residuals. The values of 𝛽 ̂2 that give the minimum
sum of the squared residuals are called the least squares estimates of β1 and β2.

̂1 and 𝛽
To find the values of 𝛽 ̂2 that minimize this sum, we have to partially differentiate
̂1 and 𝛽
with respect to 𝛽 ̂2 and set the partial derivatives equal to zero.

…………………………… (2.6.1)
̂1 and 𝛽
Differentiating Eq. (2.6.1) partially with respect to 𝛽 ̂2, we obtain

……………………(2.6.2)

………………... (2.6.3)
By setting the partial derivatives equal to zero, from Eq. (2.6.2), we obtain

… … … … … … … … … … … … … . . … . (2.6.4)

Page | 9
From Eq. (2.6.3), we obtain

Since

………………………………... (2.6.5)

Where are the sample means of X and Y and where we define and

. Henceforth, we adopt the convention of letting the lowercase letters denote

deviations from mean values.

Hence, alternatively, can be expressed as

……………………………..………….. (2.6.6)

Page | 10
2.7 Properties of Least-Squares Estimators: The Gauss–Markov Theorem

The least squares estimates possess some ideal or optimum properties. These properties are
contained in the well-known Gauss–Markov theorem. To understand this theorem, we need to
consider the best linear unbiasedness property of an estimator. An estimator, say the OLS estimator
̂2, is said to be a best linear unbiased estimator (BLUE) of β2 if the following hold:
𝛽

i) It is linear, that is, a linear function of a random variable, such as the dependent variable
Y in the regression model.
ii) ̂2), is equal to the true value,
It is unbiased, that is, its average or expected value, E( 𝛽
population parameter β2.
iii) It has minimum variance in the class of all such linear unbiased estimators; an unbiased
estimator with the least variance is known as an efficient estimator.

Given the assumptions of the classical linear regression model, the least-squares estimators, in the
class of unbiased linear estimators, have minimum variance, that is, they are BLUE.

i) Linearity

̂2, from Eq. (2.6.6), we have,

For 𝛽

Page | 11
This shows that is a linear estimator because it is a linear function of Y; actually it is a
weighted average of Yi with ki serving as the weights. Incidentally, note these properties of the
weights ki:

For

Let So,

Hence, is also a linear function of the observed Yi.

NB.

ii) Unbiasedness
̂1 and 𝛽
𝛽 ̂2 are the unbiased estimators of the true parameters β1 and β2 respectively if the following
conditions are fulfilled.

Page | 12
 ̂2, we know that
For 𝛽

Take conditional expectation

Hence, is an unbiased estimator of β2.

̂1, We know that

For 𝛽

Take conditional expectation

Hence , is an unbiased estimator of β1.

iii) Minimum variance

Assignment: Show that the estimators and have also minimum variance in the class of all
linear unbiased estimators. Show the proof of variance of the random variable 𝑢𝑖

Page | 13
2.8 The Coefficient of Determination r2: A Measure of “Goodness of Fit”

The coefficient of determination r2 is a summary measure that tells how well the sample regression
line fits the data. It measures the overall goodness of fit of the regression model. It tells what
proportion of the variation in the dependent variable is explained by the explanatory variable. This
r2 lies between 0 and 1; the closer it is to 1, the better is the fit.

Figure 2.1: The Ballentine view of r2: (a) r2 = 0; (f ) r2 = 1.

In this figure, the circle Y represents variation in the dependent variable Y and the circle X
represents variation in the explanatory variable X. The term variation and variance are different.
Variation means the sum of squares of the deviations of a variable from its mean value. Variance
is this sum of squares divided by the appropriate degrees of freedom. In short, variance =
variation/df. The overlap of the two circles (the shaded area) indicates the extent to which the
variation in Y is explained by the variation in X(say, via an OLS regression). The greater the extent
of the overlap, the greater the variation in Y is explained by X. The r2 is simply a numerical measure
of this overlap. In the figure, as we move from left to right, the area of the overlap increases, that
is, successively a greater proportion of the variation in Y is explained by X. In short, r2 increases.
When there is no overlap, r2 is obviously zero, but when the overlap is complete, r2 is 1, since 100
percent of the variation in Y is explained by X. As we shall show shortly, r2 lies between 0 and 1.

Page | 14
To compute this r2, we proceed as follows: Recall that

Take the mean of both sides, Since, Hence,

Now, subtract the mean equation from the original equation

In the deviation form

Now, square and sum both sides:

………………………….. Eq. (2.8.1)

Recall, the sample regression

Take summation both sides

Divide both sides by n

………….………………………………..Eq. (2.8.2)

Subtract Equation (2.8.2) from the sample regression, we obtain

Page | 15
From Equation (2.8.1):
: total variation of the actual Y values about their sample mean, which may
be called the total sum of squares (TSS).
: variation of the estimated Y values about
their mean , which appropriately may be called the sum of squares due to regression
[i.e., due to the explanatory variable(s)], or explained by regression, or simply the explained sum
of squares (ESS).
: residual or unexplained variation of the Y values about the regression line, or simply
the residual sum of squares (RSS). Thus,

and shows that the total variation in the observed Y values about their mean value can be
partitioned into two parts, one attributable to the regression line and the other to random forces
because not all actual Y observations lie on the fitted line.

Page | 16
The quantity r2 thus defined is known as the (sample) coefficient of determination and is the most
commonly used measure of the goodness of fit of a regression line. Verbally, r2 measures the
proportion or percentage of the total variation in Y explained by the regression model.

Two properties of r2 may be noted:

• It is a nonnegative quantity.
• Its limits are 0 ≤ r2 ≤ 1. An r2 of 1 means a perfect fit, that is, for each i. On the
other hand, an r2 of zero means that there is no relationship between the regressand and the
regressor whatsoever (i.e., ). In this case, that is, the best
prediction of any Y value is simply its mean value. In this situation, therefore the regression
line will be horizontal to the X axis.

Although r2 can be computed directly from its definition, it can be obtained more quickly from the
following formula (the explained variation as a percentage of the total variation)

………Eq. (2.8.3)

Given the definition of r2, we can express ESS and RSS discussed earlier as follows:

Page | 17
Coefficient of Correlation (r)

It is a quantity closely related to but conceptually very much different from r2. It is a measure of
the degree of association between two variables. It can be computed

… Eq. (2.8.4)

which is known as the sample correlation coefficient. Some of the properties of r are as follows.

• It can be positive or negative, the sign depending on the sign of the term in the numerator
of Equation Eq. (2.8.4), which measures the sample covariation of two variables.
• It lies between the limits of −1 and +1; that is, −1 ≤ r ≤ 1.
• It is symmetrical in nature; that is, the coefficient of correlation between X and Y(rXY) is
the same as that between Y and X(rYX).
• It is independent of the origin and scale; that is, if we define and
where a > 0, b > 0, and c and d are constants, then r between X ∗ and Y∗
is the same as that between the original variables X and Y.
• If X and Y are statistically independent, the correlation coefficient between them is zero;
but if r = 0, it does not mean that two variables are independent. In other words, zero
correlation does not necessarily imply independence.
• It is a measure of linear association or linear dependence only; it has no meaning for
describing nonlinear relations. In correlation analysis, we treat any (two) variables
symmetrically; there is no distinction between the dependent and explanatory variables.
• Although it is a measure of linear association between two variables, it does not necessarily
imply any cause-and-effect relationship. In the regression context, r2 is a more meaningful
measure than r, for the former tells us the proportion of variation in the dependent variable
explained by the explanatory variable(s) and therefore provides an overall measure of the
extent to which the variation in one variable determines the variation in the other.

Page | 18
2.9 Hypothesis Testing

Hypothesis testing is concerned with testing whether a given observation or finding is compatible
with some stated hypothesis or not. The stated hypothesis is known as the null hypothesis and is
denoted by the symbol H0. It is a statement of no effect or relationship between variables. The null
hypothesis is usually tested against an alternative hypothesis (also known as maintained
hypothesis) denoted by H1. H1 is a statement that says there is an effect or relationship between
variables. The theory of hypothesis testing is concerned with developing rules or procedures for
deciding whether to reject or not reject the null hypothesis. H0 represents the default or maintained
assumption about a parameter. H₁ contradicts the null hypothesis; it is what we conclude if we
reject H₀.

When we reject the null hypothesis, we say that our finding is statistically significant. On the other
hand, when we do not reject the null hypothesis, we say that our finding is not statistically
significant. Hypotheses can be formulated as follows:

Two-Sided (Two-tail) Test:

H0: β2 = 0 H₁: β₂ ≠ 0
One-sided (Right-tail) Test:
H0: β2 ≤ 0 H₁: β₂ > 0
One-sided (Left-tail) Test:
H0: β2 ≥ 0 H₁: β₂ < 0

2.9.1 The Confidence-Interval Approach

Two-Sided or Two-Tail Test

Assume that we want to find out how “close” is, say, to β2. For this purpose, we try to find
out two positive numbers δ and α, the latter lying between 0 and 1, such that the probability that
the random interval contains the true β2 is 1 − α. Symbolically,

………………………Eq. (2.9.1)

Such an interval, if it exists, is known as a confidence interval; 1 − α is known as the confidence

coefficient; and α (0 <α<1) is known as the level of significance. The end points of the confidence
interval are known as the confidence limits (also known as critical values), being the

Page | 19
lower confidence limit and the upper confidence limit. In practice, α and 1 − α are often
expressed in percentage forms as 100α and 100(1 − α) percent.

• Eq. (2.9.1) does not say that the probability of β2 lying between the given limits is 1 −α.
Since β2, although an unknown, is assumed to be some fixed number, either it lies in the
interval or it does not. What Eq. (2.9.1) states is that the probability of constructing an
interval that contains β2 is 1 − α.
• The interval in Eq. (2.9.1) is a random interval; that is, it will vary from one sample to the
next because it is based on , which is random.
• Since the confidence interval is random, the probability statements attached to it should be
understood in the long-run sense, that is, repeated sampling. More specifically, Eq. (2.9.1)
means: If in repeated sampling confidence intervals like it are constructed a great many
times on the 1 − α probability basis, then, in the long run, on the average, such intervals
will enclose in 1 − α of the cases the true value of the parameter.
• As noted, the interval in Eq. (2.9.1) is random so long as is not known. But once we
have a specific sample and once we obtain a specific numerical value of , the
interval in Eq. (2.9.1) is no longer random; it is fixed. In this case, we cannot make the
probabilistic statement in Eq. (2.9.1); that is, we cannot say that the probability is 1 − α
that a given fixed interval includes the true β2. In this situation, β2 is either in the fixed
interval or outside it. Therefore, the probability is either 1 or 0.

Confidence Interval for β2:

Under the classical linear regression assumptions (including normality), the OLS estimators
and are normally distributed. Therefore, the standardized form follows a t distribution and
the t-statistic for is

The t-statistic follows a t distribution with (n – k) degrees of freedom. k is the number of

parameters in a model.

Instead of using the normal distribution, we can use the t distribution to establish a confidence
interval for β2 as follows:
Page | 20
 ………………………..Eq. (2.9.2)

Where tα/2 is the value of the t variable obtained from the t distribution for α/2 level of significance
and n − 2 df; it is often called the critical t value at α/2 level of significance. The level of
significance is also known as the probability of committing a Type I error. A Type I error (false
positive) consists in rejecting a true hypothesis (falsely rejecting a null hypothesis), whereas a Type
II error (false negative) consists in accepting a false hypothesis (falsely accepting a null
hypothesis). The symbol α is also known as the size of the (statistical) test. Always bear in mind
that there is a 100α percent chance that the confidence interval does not contain β2 under H0 even
though the hypothesis is correct. In short, there is a 100α percent chance of committing a Type I
error. Thus, if α = 0.05, there is a 5 percent chance that we could reject the null hypothesis even
though it is true. Whether we reject or do not reject the null hypothesis depends critically on α, the
level of significance or the probability of committing a Type I error—the probability of rejecting
the true hypothesis. α is commonly fixed at the 1, 5, or, at most, 10 percent levels.

Substitution of the t-statistic into Equation 2.9.2 yields

……………… Eq. (2.9.3)

Rearrange Equation Eq. (2.9.3) by the following way (multiply both sides by 𝑠𝑒(𝛽̂2 ), subtract 𝛽̂2,
multiply the entire inequality by -1 where the inequality signs flip and rearrange). We obtain

….Eq. (2.9.4)

Equation Eq. (2.9.4) provides a 100(1 − α) percent confidence interval for β2, which can be written
more compactly as 100(1 − α)% confidence interval for β2:

Page | 21
100(1 − α)% confidence interval for β1:

An important feature of the confidence intervals: the width of the confidence interval is
proportional to the standard error of the estimator. That is, the larger the standard error, the larger
is the width of the confidence interval. Put differently, the larger the standard error of the estimator,
the greater is the uncertainty of estimating the true value of the unknown parameter. Thus, the
standard error of an estimator is often described as a measure of the precision of the estimator (i.e.,
how precisely the estimator measures the true population value).

Decision Rule: Construct a 100(1 − α)% confidence interval for 𝛽1 𝑎𝑛𝑑 𝛽2. For 𝛽2 , if the 𝛽2 under
H0 falls within this confidence interval, do not reject H0, but if it falls outside this interval, reject
H0. The same is true for 𝛽1 .

2.9.2 The Test-of-Significance Approach: The t Test

An alternative but complementary approach to the confidence-interval method of testing statistical

hypotheses is the test-of-significance approach. Broadly, speaking, a test of significance is a
procedure by which sample results are used to verify the truth or falsity of a null hypothesis. The
key idea behind tests of significance is that of a test statistic (estimator) and the sampling
distribution of such a statistic under the null hypothesis. The decision to accept or reject H0 is made
on the basis of the value of the test statistic obtained from the data at hand.

Under the normality assumption the variable

………………..…………….Eq. (2.9.5)

follows the t distribution with n − 2 df. If the value of true β2 is specified under the null hypothesis,
the t value of Eq. (2.9.5) can readily be computed from the available sample, and therefore it can
serve as a test statistic. And since this test statistic follows the t distribution, confidence-interval
statements such as the following can be made:

Page | 22
where is the value of β2 under H0 and where and are the values of t (the
critical t values) obtained from the t table for (α/2) level of significance and n – 2 df.

Rearranging the above Equation, we obtain

……………….. Eq. (2.9.6)

which gives the interval in which will fall with 1 − α probability, given β2 = 𝛽2∗ . In the
language of hypothesis testing, the 100(1 − α)% confidence interval established in Equation 2.9.6
is known as the region of acceptance (of the null hypothesis) and the region(s) outside the
confidence interval is (are) called the region(s) of rejection (of H0) or the critical
region. The confidence limits, the endpoints of the confidence interval, are also called critical
values. The intimate connection between the confidence-interval and test-of-significance
approaches to hypothesis testing can now be seen by comparing Eq.(2.9.4) with Eq. (2.9.6). In the
confidence-interval procedure, we try to establish a range or an interval that has a certain
probability of including the true but unknown β2,whereas in the test-of-significance approach we
hypothesize some value for β2 and try to see whether the computed lies within reasonable
(confidence) limits around the hypothesized value.

Table 2.2: The t Test of Significance: Decision Rules

Notes: is the hypothesized numerical value of β2. |t | means the absolute value of t. tα or tα/2
means the critical t value at the α or α/2 level of significance. df: degrees of freedom, (n − 2) for
the two-variable model, (n − 3) for the three-variable model, and so on. The same procedure holds
to test hypotheses about β1.

Page | 23
In short, to undertake the above test, we follow the following steps.

Step 1: Compute t-statistic

Step 2: Choose level of significance.
Step 3: Check whether there is one tail test or two tail test.
Step 4: Obtain critical value of t
Step 5: Compare t (the computed value of t) and critical value of t

2.9.3 The Exact Level of Significance: The p Value

The p value (i.e., probability value) is defined as the lowest significance level at which a null
hypothesis can be rejected. It is also known as the observed or exact level of significance or the
exact probability of committing a Type I error.

Decision Rule: fix α at some level and reject the null hypothesis if the p value is less than α.

Page | 24