Statistics for Economics and Business with

RStudio
— PART II: S TATISTICAL I NFERENCE —

Xavier Vilà
Universitat Autònoma de Barcelona

Universitat Autònoma de Barcelona

2 Notes on Statistics II

Attribution-Noncommercial-Share Alike 4.0 International

(CC BY-NC-SA 4.0)
You are free to:
• Share: copy and redistribute the material in any medium or format
• Adapt: remix, transform, and build upon the material
The licensor cannot revoke these freedoms as long as you follow the li-
cense terms.s
Under the following terms:
• Attribution. You must give appropriate credit, provide a link to the
license, and indicate if changes were made. You may do so in any
reasonable manner, but not in any way that suggests the licensor en-
dorses you or your use.
• Noncommercial. You may not use the material for commercial pur-
poses.
• Share Alike. If you remix, transform, or build upon the material,
you must distribute your contributions under the same license as the
original.
No additional restrictions — You may not apply legal terms or technolog-
ical measures that legally restrict others from doing anything the license
permits.
Notices:
• You do not have to comply with the license for elements of the ma-
terial in the public domain or where your use is permitted by an ap-
plicable exception or limitation.
• No warranties are given. The license may not give you all of the
permissions necessary for your intended use. For example, other
rights such as publicity, privacy, or moral rights may limit how you
use the material.
This is a human-readable summary of the Legal Code (the full license)
available in [Link]

Copyright © 1998-2025 Xavier Vilà.

Contents

1 Introduction to Inferential Statistics and Estimation 9

1.1 Inferential Statistics: Definition and Methods . . . . . . . . . . . . . 11
1.2 Main sample statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2.1 Sample Mean . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.2 Sample Variance . . . . . . . . . . . . . . . . . . . . . . . . 17
1.2.3 Sample Proportion . . . . . . . . . . . . . . . . . . . . . . . 18
1.3 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 19
RStudio Activity 1.3.1 - The Central Limit Theorem . . . . . . . . 20
1.4 Point and interval estimation . . . . . . . . . . . . . . . . . . . . . . 24
1.4.1 Point estimation . . . . . . . . . . . . . . . . . . . . . . . . 25
1.4.2 Interval estimation . . . . . . . . . . . . . . . . . . . . . . . 25
[Link] Confidence Interval for the mean . . . . . . . . . . 26
[Link] Confidence Interval for the variance . . . . . . . . . 28
[Link] Confidence Interval for the proportion . . . . . . . 30
RStudio Activity [Link] - Interval Estimation . . . . . . . 31
1.5 Properties of estimators . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.5.1 Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.5.2 Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
[Link] Unbiased Estimators . . . . . . . . . . . . . . . . . 40
[Link] Biased Estimators . . . . . . . . . . . . . . . . . . 40
1.5.3 Asymptotic properties . . . . . . . . . . . . . . . . . . . . . 41
[Link] Asymptotically unbiased estimators . . . . . . . . . 41
[Link] Consistent Estimators . . . . . . . . . . . . . . . . 42
RStudio Activity 1.5.1 - Consistent Estimators . . . . . . . . . . . 42
1.6 Methods of point estimation . . . . . . . . . . . . . . . . . . . . . . 45
1.6.1 Maximum Likelihood estimation . . . . . . . . . . . . . . . . 46
1.6.2 Method of moments . . . . . . . . . . . . . . . . . . . . . . 48
RStudio Activity 1.6.1 - Maximum Likelihood estimation . . . . . 49

3
4 CONTENTS

2 Parametric Hypothesis Testing 53

2.1 Concept of parametric test: null hypothesis and alternative hypothesis 53
2.2 Test statistic and error types . . . . . . . . . . . . . . . . . . . . . . 55
2.3 Tests on the mean, variance and proportion . . . . . . . . . . . . . . 57
2.3.1 Hypothesis Test for the Population Mean (µ) . . . . . . . . . 57
RStudio Activity [Link] - Hypothesis Test for the Popula-
tion Mean . . . . . . . . . . . . . . . . . . . . . . 66
2.3.2 Hypothesis Test for the Population Variance (σ 2 ) . . . . . . . 70
RStudio Activity [Link] - Hypothesis Test for the Popula-
tion Variance . . . . . . . . . . . . . . . . . . . . . 75
2.3.3 Hypothesis Test for the Population Proportion (π) . . . . . . . 77
RStudio Activity [Link] - Hypothesis Test for the Popula-
tion Proportion . . . . . . . . . . . . . . . . . . . . 82
2.4 Tests for comparing samples . . . . . . . . . . . . . . . . . . . . . . 84
2.4.1 Test for the Difference of Means . . . . . . . . . . . . . . . . 85
2.4.2 Test for the Comparison of Variances . . . . . . . . . . . . . 93
2.4.3 Test for the Difference of Proportions . . . . . . . . . . . . . 99
RStudio Activity 2.4.1 - Tests for comparing samples - Preliminaires103
2.5 The p_value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
2.5.1 Concept, definition and computation . . . . . . . . . . . . . . 104
2.5.2 Test conclusion based on the p_value . . . . . . . . . . . . 109
RStudio Activity 2.5.1 - Tests for comparing samples . . . . . . . 112
2.6 Analysis of Variance . . . . . . . . . . . . . . . . . . . . . . . . . . 120
2.6.1 Basic Framework . . . . . . . . . . . . . . . . . . . . . . . . 120
2.6.2 Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
2.6.3 Test statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
2.6.4 Distribution of the test statistic . . . . . . . . . . . . . . . . . 123
2.6.5 Test conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 123
2.6.6 The ANOVA table . . . . . . . . . . . . . . . . . . . . . . . 124
RStudio Activity 2.6.1 - Analysis of Variance . . . . . . . . . . . . 126

3 Goodness-of-fit and analysis of the relationship between variables 133

3.1 Tests of goodness-of-fit . . . . . . . . . . . . . . . . . . . . . . . . . 134
2
3.1.1 χ test of goodness-of-fit for discrete variables . . . . . . . . 134
RStudio Activity [Link] - Tests of goodness-of-fit for dis-
crete variables . . . . . . . . . . . . . . . . . . . . 137
CONTENTS 5

3.1.2 Kolmogorov-Smirnov test of goodness-of-fit for continuous vari-

ables. The normality test. . . . . . . . . . . . . . . . . . . . . 142
RStudio Activity [Link] - Contrast de la bondat d’ajust per
variables contínues . . . . . . . . . . . . . . . . . 146
3.2 Analysis of the relationship between variables . . . . . . . . . . . . . 149
3.2.1 Test of independence between qualitative variables . . . . . . 150
RStudio Activity [Link] - Test of independence between
qualitative variables . . . . . . . . . . . . . . . . . 153
3.2.2 Analysis of the correlation between quantitative variables. The
correlation coefficient. . . . . . . . . . . . . . . . . . . . . . 157
RStudio Activity [Link] - Test of independence between
quantitative variables . . . . . . . . . . . . . . . . 164

4 Introduction to the regression model 171

4.1 Objective of the model . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.2 Hypothesis of the model specification . . . . . . . . . . . . . . . . . 173
4.3 Estimation by Ordinary Least Squares . . . . . . . . . . . . . . . . . 175
4.3.1 Data in differences with respect to the mean . . . . . . . . . . 176
4.3.2 The OLS estimators . . . . . . . . . . . . . . . . . . . . . . 177
4.3.3 Estimation of the variance of the error term . . . . . . . . . . 178
4.3.4 Properties of the estimators OLS . . . . . . . . . . . . . . . . 178
[Link] Properties of β̂2 . . . . . . . . . . . . . . . . . . . 179
[Link] Properties of β̂1 . . . . . . . . . . . . . . . . . . . 179
4.4 Model testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
4.4.1 Confidence intervals for β1 and β2 . . . . . . . . . . . . . . . 181
4.4.2 Hypothesis testing for β1 and β2 . . . . . . . . . . . . . . . . 181
4.5 Coefficient of the goodness-of-fit . . . . . . . . . . . . . . . . . . . 182
4.5.1 The Coefficient of determination (or goodness-of-fit) . . . . . 182
4.5.2 Relationship between the correlation analysis and the regres-
sion analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 184
4.6 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
4.6.1 Point estimation of Yn+1 . . . . . . . . . . . . . . . . . . . . 185
4.6.2 Estimation by intervals of Yn+1 . . . . . . . . . . . . . . . . 185
4.6.3 Estimation of the effect on Y of changes in X . . . . . . . . . 186
RStudio Activity 4.1 - Regression Analysis . . . . . . . . . . . . . . . . 190
6 CONTENTS

A Statistical Tables 211

A.1 The Standard Normal Distribution . . . . . . . . . . . . . . . . . . . 212
A.2 The t − student Distribution . . . . . . . . . . . . . . . . . . . . . . 213
A.3 The χ2 Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
A.4 The Snedecor ’s F Distribution . . . . . . . . . . . . . . . . . . . . . 215
A.5 Values for the Kolmogorov-Smirnov test . . . . . . . . . . . . . . . . 218

B Reading RStudio Outputs 219

B.1 Summary(aov)output reading (ANOVA) . . . . . . . . . . . . . . 220
B.2 summary(lm) output reading (Linear Regression) . . . . . . . . . 220
B.3 stargazer(lm) output reading (Linear Regression) . . . . . . . . 221
Introduction

This textbook is intended to be used following the text "Statistics for Economics and
Business with RStudio - PART I: D ESCRIPTIVE S TATISTICS AND R ANDOM VARIABLES", so
that the reader will already be familiar with the basic functionalities of RStudio and
the necessary statistical concepts. This document is not intended to be an introductory
manual to R and RStudio, but an invitation to use R and RStudio to better understand
how to do statistical inference in the fields of economics and business.
At the end of some sections of the text, some “RStudio Activities” are pro-
posed to experiment in a practical way with the concepts explained. It is strongly rec-
ommended to read the comments (fragments starting with the # symbol) in the R code
because, in addition to explaining what is being done, they can give clues about the syn-
tax used. Similarly, the “R Notes” section that concludes some of these RStudio
Activities expands the explanation on some specific R commands or techniques.
The structure and contents of these two manuals ("Statistics with RStudio I" and "Statis-
tics with RStudio II") largely adhere to the sequence of statistics courses that are usu-
ally followed in undergraduate degrees. economics, business administration and other
related degrees, introducing RStudio as a tool for experimentation and better under-
standing of the topics covered.

7
8 CONTENTS
Chapter 1

Introduction to Inferential
Statistics and Estimation

T HE BLUNDER is to estimate, —
“Eternity is Then,”
We say, as of a station.
Meanwhile he is so near,
He joins me in my ramble,
Divides abode with me,
No friend have I that so persists
As this eternity

E MILY D ICKINSON

Think of a researcher who seeks to explain some fact in the real world. For instance,
imagine Newton trying to explain why apples fall. As a more familiar example, imagine
an economist trying to explain why unemployment does exist.
Usually, the task of a researcher consists of three parts:

1. Observe the world in order to determine the problem to study and gather infor-
mation about it

2. Think about the problem

3. Produce an explanation (Theory or model) for the problem.

4. Test and validate the model proposed.

9
10 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Statistics becomes extremely important for the first of these four items.1

It is clear that, in order to study a "real problem", the researcher must observe the "real"
world. Nevertheless, it is also clear that no researcher can observe the whole reality.
Newton can not observe all the falling apples, neither can an economist interview the
whole population of a country to determine the unemployment rate. It is thus necessary
to somehow "summarize" the reality, but this task has to be done so that such "sum-
mary" closely fits the reality being studied. Then, and only then, the conclusions drawn
from the "summary" can be applied reliably to the whole population.

Statistics (more precisely, statistical inference) is a collection of techniques by means

of which we can draw conclusions about a reality from the study of a summary of
that reality

Hereafter we will study in detail how this is done.

Chapter one reviews the main concepts used in statistics and deals with the first ap-
proach on how to draw conclusions about some real issues based on what we observe
in the summary.

Chapters two and three introduce more sophisticated techniques to make inferences
about the reality using some of the more elemental results seen in Chapter two.

Finally, chapter four introduces the linear regression analysis, a technique widely used
in the economic analysis (and other sciences) to study the relationship between vari-
ables.

It is worth saying that a clear understanding of the topics in Chapter one is important
in order to easily understand what other chapters deal with, and also to get an global
idea of the whole process of statistical inference.

It is important to understand that statistics is based on probabilistic techniques. Hence,

any statistical conclusion drawn from this kind of summary will not be true for sure
when applied to the whole reality, but only with a certain probability. For instance,
when an electoral survey is conducted it is clear that its results will not exactly coincide
with the results in the final election. Nevertheless, if the survey is "well done", that is, if
the summary of the reality (which in this case is the set of people interviewed) closely
represents the whole reality (which in this case is the whole population that has the right
to vote), then the survey result will be close to the final results with a high probability

In the sections below we will see which are the basic ingredients of any statistical
analysis and its probabilistic features

1 Very often the researcher does not start up by gathering information using statistical techniques. On the

contrary, in many cases his initial activity consists of detecting general patterns of behavior for a given fact.
From here, researchers are able to build up an abstract theory in order to explain the phenomenon at study.
This is, for example, Newton’s way, and also the way Economic Theory works. Once this "abstract theory"
is logically constructed, statistical techniques are often used to check whether such theory fits the reality, as
we will see in Chapter 4.
1.1. INFERENTIAL STATISTICS: DEFINITION AND METHODS 11

1.1 Inferential Statistics: Definition and Inference Meth-

ods

Statistical inference is mainly built upon four main concepts, which will be defined
and described below. These concepts are closely related to each other and it is very
important to clearly understand each of them and not to mistake one by the other.

Population Is the set of elements that are the object of study. The goal is to draw some
conclusion regarding some specific feature of this population.

Example 1.1.1 All the apples in the world. The feature at study is whether
apples fall down or not.

Example 1.1.2 Labor force in the European Union. The feature at study
is whether workers are unemployed or not.

Example 1.1.3 Production of Intel chips in a given day. The feature at

study is whether chips are faulty or not.

Sample Subset of the Population used to draw conclusions about the population

Example 1.1.4 50 apples in Newton’s garden.

Example 1.1.5 Unemployment statistics at the European Union.

Example 1.1.6 25 Intel chips manufactured in a given day.

Parameter Is the feature of the population that we want to learn something about.
This feature has to be a numerical one2 and, obviously, its true value must be
unknown3

Example 1.1.7 What is the proportion of falling apples.

Example 1.1.8 What is the unemployment rate at the European Union

Example 1.1.9 What is the proportion of faulty chips among those pro-
duced in a given day.

Statistic Computation made using the elements in the sample and used to get an ap-
proximation to the true value of the parameter. It is important to notice that
this value will be known (since we will compute it) and will be used to draw
conclusions on the true value of the parameter, which is unknown and is what
is of interest to us.
2 Although non numerical features can be studied as well, the techniques used in such cases are different

from those that we will see here. Nevertheless, Chapter four will introduce some of these analysis.
3 For otherwise it will not be necessary to do any statistical analysis at all !
12 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Example 1.1.10 Proportion of falling apples among the 50 sampled ap-

ples in Newton’s garden.

Example 1.1.11 Unemployment rate among the workers interviewed in

the unemployment statistics in the European Union.

Example 1.1.12 Proportion of faulty chips among the 25 selected chips

produced in a given day.

From this four main concepts, the process of statistical inference works as follows:

1. Using sampling techniques that will be explained below, a sample is selected

from the population that is going to be studied.

2. From this sample, the proper computations are done in order to obtain a statistic.

3. From this statistic, using some statistical inference technique that we will see in
other chapters, some conclusions are drawn regarding the unknown population
parameter that represents the feature of the population that is to be studied.

This process can be represented as in Figure 1.1

Population Parameter
(unknown)
Statistical
Sampling Inference

Statistic
Sample
(known)

Figure 1.1: The process of Statistical Inference

We can now provide a better definition for Statistics (or Statistical Inference).
Definition 1.1.13 Statistical Inference is a subject whose main objective is to draw
conclusions regarding a population through the study of one sample by means of
probabilistic techniques.

1.2 Definition, Characteristics and Distribution of the

main sample statistics: mean, variance and propor-
tion
Once the sample is obtained (we will always assume that using a Simple Random
Sampling), the process of working with it and drawing conclusions begins.
In this sense, the main task is now to obtain a statistic, one of the main concepts
in statistical inference. We will use it to draw conclusions regarding the unknown
population parameter that is of interest to us.
1.2. MAIN SAMPLE STATISTICS 13

The definition that follows reminds us what a statistic is (as introduced in the previous
section). Then, the concept of estimate is defined. Although these two concepts are
very similar and closely related it is very important to notice that they are not the same
thing.

Definition 1.2.1 A statistic (or estimator)a is a formula that uses the values in the
sample at hand (observations) in order to produce an approximation to the true
value of an unknown population parameter.
a The fact that the the same "object" can have two names should not be misleading. Depending on the

kind of analysis that we want to perform, the same "formula" is referred to with one name or the other. In
Chapter 2 we will use the term estimator, while in the chapters that follow we will rather use the name
statistic. It is always the same idea, but used purposes for different .

Definition 1.2.2 An estimate (or estimation) is the particular value of an estimator

that is obtained from a particular sample.

Hence, a statistic (or estimator) is not a number but a formula while an estimate is the
number that is obtained when the formula (the statistic) is applied to the observations
of the specific sample that we have at hand.
At this point, it becomes crucial to understand that, given that the sample is obtained
by means of a random technique, the statistic will produce different estimates with
different probabilities (depending on the specific sample that is finally "selected" at
random). To put it more formally, a statistic is a Random Variable, that is, a variable
that takes different values with different probabilities In this sense, an estimate is a
specific realization of this random variable. The following example aims at clarifying
this idea.

Example 1.2.3 We want to know the average number of cars per family in a given
population. To keep the example simple, we will assume that the population is very
small, only 4 families,
Population = {A, B, C, D}

Let us now assume that family A owns one car, families B and C have 2 cars each,
and family D has 4. 4
For our study, we want to obtain a random sample of size 2. We can then compute the
average number of cars in the sample and use it to infer some conclusion regarding
the true average in the population. Hence, the sample mean (or just mean, for short)
will play the role of statistic in this example, and we can use it to draw conclusions on
the true population parameter that is of interest to us: the average number of cars per
family in the whole population, that is, the population mean.
Table 1.2 summarizes:

1. The 6 possible samples than can be the result of a random sampling process on
this population,
4 Obviously, we are supposed not to know that information for otherwise there will be no need for any

kind of statistical analysis !!

14 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

2. for each of the possible samples, the probability of being selected (all of them
will have the same probability as we are assuming SRS)
3. the estimate value that would result from applying the sample average formula
to the corresponding sample

Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 Sample 6

Elements {A, B} {A, C} {A, D} {B, C} {B, D} {C, D}
1 1 1 1 1 1
Probability 6 6 6 6 6 6
Estimate 1.5 1.5 2.5 2 3 3

Table 1.2: Possible samples, probability for each sample, and estimate value in each
case.

In this example we can see how the statistic at use (sample mean) can take 4 different
values, depending on which of the six possible samples is selected by the SRS. We see,
for example, that the value 1.5 corresponds to two possible samples (Sample 1 and
Sample 2). Hence, since each sample has the same probability of being selected ( 61 ),
the probability that the statistic takes the value 1.5 is:

1 1 1
P (statistic = 1.5) = P (Sample 1) + P (Sample 2) = + =
6 6 3
Next, we summarize the possible values the statistic can take an what is the probability
associated to each of them:

1


 1.5 p= 3
1
2 p=

statistic value = 6
1

 2.5 p= 6
1
3 p=

3

In this example, we have seen how the statistic can take different values (4 in this case)
with different probabilities. Hence, the statistic is a random variable

Hence, given that statistics are random variables, it will be useful to know their main
properties and, specially, the probability distributions of the ones that are more fre-
quently used. In this sense, the main statistics (or estimators) that are studied are the
sample mean, the sample variance, and the sample proportion.
In all cases, we will assume that a sample of size n has been obtained by means of a
SRS. The elements of the sample will be denoted by
{x1 , x2 , · · · xn }

Also, we will assume that the sample has been selected form a population that follows
a given distribution5 . To know this distribution is very important as it will influence
5 Note
that when we talk about "population distribution" we are abusing the language. Properly speaking
we should say "distribution of the variable of interest that we are studying in the reference population". For
simplicity, however, we just say "population distribution" (or "distribution of the population”). Similarly,
the expression “sampling distribution” (or “distribution of the sample”) is used instead of “distribution of
observed values in the sample”, which would be more appropriate.
1.2. MAIN SAMPLE STATISTICS 15

the sampling result and, hence, the possible values of the statistic as we have seen in
the previous example. Indeed, in that example we have seen that the population is dis-
tributed so that there is 1 element with 1 car, 2 elements with 2 cars, and 1 element with
4 cars. Therefore, if we pick the sample element xi at random from this population, we
will have that:  1
if a = 1
 41


2 if a = 2
p(xi = a) = 1
if a = 4
 4


0 otherwise

This is, in this case, the distribution of the population. Figure 1.2 shows it.

Figure 1.2: Population distribution in example 1.3.3

In generala , we will assume that the S AMPLE has been obtained by means of
a SRS from a population distributed according to a N ORMAL D ISTRIBUTION
with some P OPULATION M EAN µ and some P OPULATION VARIANCE σ 2 .
a There are special cases that we will discuss in due time

The above means that for any two numbers a and b, we have that for any element in
our sample xi ,

p(a ≤ xi ≤ b) = p(a − µ ≤ xi − µ ≤ b − µ) =

a−µ xi − µ b−µ a−µ b−µ

= p( ≤ ≤ ) = p( ≤Z≤ )
σ σ σ σ σ

where Z represents the Standard Normal distribution, usually denoted by N (0, 1),
whose associated probabilities are found in tables. Graphically, Figure 1.3 shows it
16 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Figure 1.3: Normal Distribution

We turn next to the study of the distributions of the three main statistics. These, as we
have discussed above, will depend on the distribution of the population from which we
obtain the sample, and also on the mean µ and variance σ 2 of that population. For each
case, we will also be interested in knowing what is the expectation and the variance of
these statistics.

1.2.1 Sample Mean

The Sample mean, denoted by X̄, is the statistic that is obtained from the sample using
the formula:
n
1X
X̄ = xi
n i=1

It is normally used to infer conclusions regarding the true value of the Population mean
µ. Its distribution depends on the characteristics of both the population and the sample

1. If the population is Normal, that is, Xi ∼ N (µ, σ 2 ) ∀i, then we have that

σ2
X̄ ∼ N (µ, )
n
because of the sample mean being a linear combination of Normal random vari-
ables

2. If the population is not Normal but the sample is big enough then:

X̄ − µ
q ∼ N (0, 1) (approx.)
σ2
n

because of the Central Limit Theorem that will be introduced later on.

3. If the population is not Normal and the sample is small, then the distribution
of the sample mean X̄ is unknown in general, and will depend on the specific
distribution of that population.
1.2. MAIN SAMPLE STATISTICS 17

4. If the population variance σ 2 is unknown and the population is Normal, then

X̄ − µ
q ∼ tn−1
S2
n

where S 2 is the Sample variance (that we will see next) and tn−1 is the t −
student distribution with n − 1 degrees of freedom, which is very similar to the
N (0, 1) distribution and whose values can also be found in tables.

We turn next to the study of the expectation and variance of this statistic. To do so,
we will use the mathematical properties of the expectation and variance of a random
variable. 6 As usual, we will assume that the sample has been obtained from a popu-
lation with population mean µ and population variance σ 2 . That is, E(xi ) = µ and
V (xi ) = σ 2 for any element xi in the sample. Then,
n n n
1X 1X 1X 1
E(X̄) = E( xi ) = E(xi ) = µ = nµ = µ
n i=1 n i=1 n i=1 n

n n n n
1X 1 X 1 X 1 X 2 1 σ2
V (X̄) = V ( xi ) = 2 V ( xi ) = 2 V (xi ) = 2 σ = 2 nσ 2 =
n i=1 n i=1
n i=1 n i=1 n n

Therefore, for the case of the sample mean X̄ we have that

σ2
E(X̄) = µ, V (X̄) =
n

1.2.2 Sample Variance

The Sample variance, denoted by S 2 , is the statistic that is obtained from the sample
using the formula:

1 X
S2 = (xi − X̄)2
n−1

It is normally used to infer conclusions regarding the true value of the Population vari-
ance σ 2 . Its distribution depends on the characteristics of the population.

1. If the population is Normal, (Xi ∼ N (µ, σ 2 ) ∀i), then:

(n − 1)S 2
∼ χ2n−1
σ2
where χ2n−1 is the chi-square distribution with n − 1 degrees of freedom. (This
distribution corresponds to the sum of n − 1 squared standard Normal distribu-
tions)
6 For instance, the expectation of the sum of random variables is the sum of expectations, and so.
18 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

2. If the population is not Normal, then the distribution of the sample variance is
unknown in general, even for large samples.

Since we only know the distribution of the sample variance when the population is
Normal, we will use the fact that in that case its distribution is χ2n−1 to find the expec-
tation and variance easily. In this sense, we know that for any χ2 variable we have that
E(χ2n−1 ) = n − 1 and V (χ2n−1 ) = 2(n − 1). Hence, we will assume the the sample
has been obtained from a Normal population with sample mean µ and sample variance
σ 2 . That is, xi ∼ N (µ σ 2 ) for any element xi in the sample. Hence:

(n − 1)S 2
∼ χ2n−1
σ2

and therefore
(n − 1)S 2 (n − 1)
E( 2
)=n−1⇒ E(S 2 ) = n − 1 ⇒ E(S 2 ) = σ 2
σ σ2

(n − 1)S 2 (n − 1)2 2 2 2σ 4
V( ) = 2(n − 1) ⇒ V (S ) = 2(n − 1) ⇒ V (S ) =
σ2 (σ 2 )2 n−1

Therefore, for the case of the sample variance S 2 we have that

2σ 4
E(S 2 ) = σ 2 , V (S 2 ) =
n−1

1.2.3 Sample Proportion

The Sample proportion is a special case. It is used when we are interested in knowing
which is the true proportion of elements in a population that have a given characteristic.
For instance, it might be of interest to know what is the proportion of smokers among
the second year students in this school (in this case, the characteristic that is of interest
is "whether a student smokes or not"), or what is the proportion of faulty Intel chips in
a day (in this case, the characteristic of interest is "whether a chip is faulty or not")
Sample proportion, denoted by π̂, is the statistic that is obtained from the sample using
the formula:

n
X xi
π̂ =
i=1
n
where xi = 1 if the i-th element in the sample has the characteristic that we are study-
ing and xi = 0 if it does not.
Sample proportion is normally used to infer conclusions regarding the true population
sample π. In this case, the population is never Normal since each observation xi comes
from a Bernoulli random variable. Indeed, let us assume that we are looking at a
population of 100 individuals out of which 45 are smokers. That is, the true population
1.3. CENTRAL LIMIT THEOREM 19

proportion is 45% or π = 0.45. Imagine that from this population we want to obtain a
sample of size 10. It is clear that for any element xi of the sample we will have that:
45
p(xi = 1) = = 0.45
100

55
p(xi = 0) = = 0.55
100
Hence, we see that each element xi in the sample follows a Bernoulli distribution with
parameter π (where π is the true and unknown population proportion
Pn
It can be shown then that π̂ = i=1 xi /n is a Binomial random variable. Also, given
that when samples are large a Binomial distribution can be approximated by a Normal
distribution, we can conclude that, in general:

1. If the sample is large enough (nπ̂(1 − π̂) > 5), then (approx.):

π(1 − π)
π̂ ∼ N (π, )
n
This approximation is better the closer to 0, 5 is π and the larger is the sample
2. If the sample is not large, then the approximation is very bad.

With regards to the expectation and variance of the sample proportion, we have:

π(1 − π)
E(π̂) = π, V (π̂) =
n

1.3 Central Limit Theorem

This theorem presents a mathematical fact that is of the highest importance in statistical
inference. Basically, the theorem states that the sum of identical random variables,
whatever their distribution is, approximates a Normal random variable.
From a practical point of view this result allows us to work with the sample mean as
if it were a Normal random variable even when the population from which we obtain
the sample does not follow a Normal distribution. Moreover, the larger the sample the
better the approximation.
Formally,

Theorem 1.3.1 Let X1 , X2 , . . . , Xn be a series of independent random variables with

identical distribution, expectation µ and variance σ 2 . Then, when n is large enough,
the random variable
n
1X
X̄ = Xi
n i=1

2 σ2
follows, approximately, a Normal distribution with µX̄ = µ and σX̄ = n
20 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

As a general rule, it is considered that n > 30 is a large enough sample for a good
approximation to a Normal distribution. Thus, we often use the transformation
X̄ − µ
Z= q ∼ N (0, 1)
σ2
n

in many techniques in statistical inference

RStudio Activity 1.3.1 - The Central Limit Theorem

We will use RStudio to illustrate the conclusion of the Central Limit Theorem.
Basically, we will see how, starting with a random variable that follows an arbitrary dis-
tribution, as we keep adding replicas of this variable the sum of all of them approaches
a Normal distribution.

• In this example, this variable, that we will initially label as X1 , will follow an
exponential distribution with parameter λ = 20. We will simulate 10.000 real-
izations of this variable and will construct a histogram of frequencies to view its
distribution.
• We will keep adding replicas of this variable, X1 +X2 +X3 +· · · , simulating in
each case 10.000 realizations of this sum, and will verify that as the sum grows
large the histogram of frequencies approaches more and more the graphical rep-
resentation of a Normal distribution.

We can use now the R console to type7 and run the chunk below to:

1. Simulate 10.000 realizations of one exponential variable with parameter λ = 20

and graphically represent the frequencies of the results obtained by means of a
histogram. We will label this case SUM _1, as there is only one variable in the
sum: SUM_1 = X1
2. Simulate 10.000 realizations of 10 exponential variables with parameter λ = 20,
sum them up, and graphically represent the frequencies of the results obtained by
means of a histogram. We will label this case SUM _10, as there are 10 variables
in the sum:: SUM_10 = X1 + X2 + X3 + · · · X10
3. Simulate 10.000 realizations of 100 exponential variables with parameter λ =
20, sum them up, and graphically represent the frequencies of the results ob-
tained by means of a histogram. We will label this case SUM _100, as there is
only one variable in the sum: SUM_100 = X1 + X2 + X3 + · · · X100
4. Simulate 10.000 realizations of 1000 exponential variables with parameter λ =
20, sum them up, and graphically represent the frequencies of the results ob-
tained by means of a histogram. We will label this case SUM _1000, as there is
only one variable in the sum: SUM_1000 = X1 + X2 + X3 + · · · X1000

At the end we will check whether the histograms behave as expected according to the
Central Limit Theorem and approach a Normal distribution.

7 It is not necessary to copy and paste the lines that start with "#". The # symbol in R is a “comment”.

Anything written to the right of # will be ignored by R.

1.3. CENTRAL LIMIT THEOREM 21

### Simulate 10.000 realizations of one exponential

### variable with parameter 20
SUM_1 <- rexp(10000,20)

### Create the corresponding histogram

hist(SUM_1)

Histogram of SUM_1
5000
Frequency

0 2000

0.0 0.1 0.2 0.3 0.4 0.5

SUM_1

### Simulate 10.000 realizations of 10 exponential

### variables with parameter 20. Sum them up
### and give the name SUM_10 to the sum
SUM_10 <- 0
for (i in 1:10) {SUM_10 <- SUM_10 +rexp(10000,20)}

### Create the corresponding histogram

hist(SUM_10)
22 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Histogram of SUM_10

2500
Frequency

1000
0

0.2 0.4 0.6 0.8 1.0 1.2

SUM_10

### Simulate 10.000 realizations of 100 exponential

### variables with parameter 20. Sum them up
### and give the name SUM_100 to the sum
SUM_100 <- 0
for (i in 1:100) {SUM_100 <- SUM_100 +rexp(10000,20)}

### Create the corresponding histogram

hist(SUM_100)

Histogram of SUM_100
1500
Frequency

500
0

3 4 5 6 7

SUM_100

### Simulate 10.000 realizations of 1000 exponential

### variables with parameter 20. Sum them up
1.3. CENTRAL LIMIT THEOREM 23

### and give the name SUM_1000 to the sum

SUM_1000 <- 0
for (i in 1:1000) {SUM_1000 <- SUM_1000 +rexp(10000,20)}

### Create the corresponding histogram

hist(SUM_1000)

Histogram of SUM_1000
2000
Frequency

1000
0

44 46 48 50 52 54 56

SUM_1000

We notice that, indeed, the histograms approach the graphical representation of a Nor-
mal distribution.

R Hints

• For convenience, in this activity instead of working with a sample obtained

from a database stored in some .xls or .csv file, we have generated a
totally random sample. The advantages of this technique, which is an ex-
ample of the so called "Monte Carlo method" used to solve many problems,
are various:
1. We avoid having to load an external data file into RStudio and the
problems that this often causes.
2. We can generate large or small samples as you like, and also as many
samples as needed.
3. We can generate random samples from a population with a known
distribution, population mean, and population variance. This, as we
have seen in the section 1.2 , will be very useful as it will allow us to
generate samples with the appropriate characteristics for the subject
we are dealing with in each case.
24 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Note that we will use this same technique in all the RStudio activities in this
textbook. If, on the other hand, you have a database and want to perform
any of these activities with it, just import it into RStudio and replace the
names of the randomly generated samples with the names of the variables
of your interest in the database.
To generate these samples from known distributions, remember the RStudio
commands corresponding to each case:

Discrete Distributions
Generates a radon sample of a given size
rbinom(size,1, p)
from a Bernoulli b(p) distribution
Generates a radon sample of a given size
rbinom(size,n, p)
from a Binomial B(n, p) distribution
Generates a radon sample of a given size
rpois(size,λ)
from a Poisson P(λ) distribution
Continuous Distributions
Generates a radon sample of a given size
rnorm(size,µ, σ)
from a Normal distribution N (µ, σ 2 )
Generates a radon sample of a given size
rt(size,df )
from a t-student distribution with df degrees of freedom
Generates a radon sample of a given size
rchisq(size,df )
from a χ2 distribution with df degrees of freedom

• In R,the keyword for is used to create “loops”, that is, cycles on which
the same command (or group of commands) is repeated a given number of
times. For instance, in the code above,

for (i in 1:10) {SUM_10 <- SUM_10 +

rexp(10000,20)}

creates a loop (defined by the curly brackets “{“ and “}”) with 10 repetitions
(1:10) of the command SUM_10 <- SUM_10 + rexp(10000,20)
(which takes care of the sum of the simulated exponential variables))

End of Activity RStudio 1.3.1

1.4 Point and interval estimation

Statistical estimation is the simplest inference technique. It allows for a quick approx-
imation to the true value of the parameter of interest.
Thus, its objective is to produce a first approximated measure of the parameter we
want to study. This measure will be improved later on by means of more elaborated
statistical inference techniques.
We will learn how to use the statistics learned previously to produce conclusions (very
preliminary at this point) regarding the true population parameters. Point estimation
and confidence intervals will be the techniques that we will use.
1.4. POINT AND INTERVAL ESTIMATION 25

Later in this section we will investigate the main properties of these estimators, as well
as other more advanced topics like Maximum likelihood estimation and the method of
moments which will allow us to design good estimators for the case we do not know
which one to use.

1.4.1 Point estimation

A point estimation is the simplest method to produce estimations for a population pa-
rameter, that is, an approximation to its true value. To obtain a point estimation or
estimate we just need to apply our estimator to the specific sample at hand.

Example 1.4.1 Imagine that we want to obtain an approximation to the true value of
the population mean µ of a given population. For what we have seen before, we know
that the sample mean X̄ is a good estimator of µ for it is unbiased8 . Hence, this will
be the estimator we use. Imagine that the sample we have is

Sample = {1, 2, 3, 4}

Then
1+2+3+4
X̄ = = 2.5
4

Hence, in this case the point estimation (or estimate) we get for µ is 2.5

A point estimation has the advantage of being an easy and quick method of estimation.
On the other hand, it does not provide much information about the parameter, and is
not very accurate either.
In the example above:

• The value of X̄ that we have found suggests that the true value of the population
mean µ will be “around” 2.5

• We do not know, though, if it will be larger or smaller

• We do not know if it will be near 2.5 or not.

• We do not know anything about the accuracy of our estimation

Such “lack” of precision can be somehow fixed with the next method of estimation

1.4.2 Interval estimation

We will use now the knowledge we have about the probability distribution of the sample
statistics to supplement the point estimation with additional information. In this way,
we will produce an interval that will contain, with some probability, the true value of
the unknown population parameter.
8 We will see later that, moreover, it is the most efficient estimator of the population mean
26 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

That is, we will be able now to "measure" the accuracy of our estimation. In this sense,
the outcome of an interval estimation will be something similar to (for the case of the
mean):
µ ∈ [2.25 , 2.75] with probability 95%

The intervals obtained using this method are called confidence intervals, and the prob-
ability that the interval contains the population parameter is the confidence level, usu-
ally denoted by 1 − α.

[Link] Confidence Interval for the mean

We will see next how to build the confidence interval for the case when we need to
produce an estimation for the population mean µ

Case I: Normal Population (or large sample) and σ 2 known

We know that in this case,

X̄ − µ
q ∼ N (0, 1)
σ2
n

hence
X̄ − µ
p(−z1− α2 ≤ q ≤ z1− α2 ) = 1 − α
σ2
n

where z1− α2 is the value that corresponds to a N (0, 1) whose left tail contains an area
of 1 − α2 . That is,
α
P (Z ≤ z1− α2 ) = 1 −
2

where Z represents a N (0, 1) and this value can be found in tables.

Doing some algebra inside the inequalities we get,
r r
σ2 σ2
p(−X̄ − z1− α2 ≤ −µ ≤ −X̄ + z1− α2 )=1−α
n n

multiplying by −1 we reverse the "direction" of the inequalities, and hence

r r
σ2 σ2
p(X̄ + z1− α2 ≥ µ ≥ X̄ − z1− α2 )=1−α
n n

at the end we get the interval we were looking for,

r r
σ2 σ2
µ ∈ [X̄ − z1− α2 , X̄ + z1− α2 ] with probability 1 − α
n n
1.4. POINT AND INTERVAL ESTIMATION 27

Example 1.4.2 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from
a Normal population with unknown mean and variance σ 2 = 1.000.000. Construct
a confidence interval with a confidence level of 95% for the population mean µ if we
know that the sample mean is X̄ = 26.000.
If the confidence level is 95% we have that 1 − α = 0.95. Hence, α = 0.05 and
α
2 = 0.025. Therefore,
α
1 − = 0.975
2

The interval will be of the form

r r
σ2 σ2
[X̄ − z1− α2 , X̄ + z1− α2 ]
n n

where all the values are known except for the values Z that correspond to a Normal
distribution. In this case we have to look up the tables for the value

Z1− α2 = Z0.975

That is, the value of a N (0, 1) that has to its left a probability of 0.975. In the tables
we find
Z0.975 = 1.96

Thus, r r
σ2 σ2
[X̄ − z1− α2 , X̄ + z1− α2
]=
n n
r r
1.000.000 1.000.000
= [26.000 − 1.96 , 26.000 + 1.96 ]
100 100

Doing the computations, we finally get

µ ∈ [25.804, 26.196] with a probability of 95%

Case II: Normal Population (or large sample) and σ 2 unknown

In the previous case we need to know the true value of the population variance σ 2 in
order to compute the interval. This is highly unusual. To overcome this problem we
can replace σ 2 by its unbiased estimator S 2 . The only difference is that now we can
not use the N (0, 1), but the t − student with n − 1 degrees of freedom.

r r
S2 S2
µ ∈ [X̄ − t1− α2 , X̄ + t1− α2 ] with probability 1 − α
n n

where t1− α2 is the value that corresponds to a t − student whose left tail contains an
area of 1 − α2 and that can be found in tables as well.
(when n is large, then t1− α2 is approximately equal to a z1− α2 )
28 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Example 1.4.3 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from a
Normal population with unknown mean and variance. Construct a confidence interval
with a confidence level of 95% for the population mean µ if we know that the sample
mean is X̄ = 26.000 and the sample variance is S 2 = 980.000
If the confidence level is 95% we have that 1 − α = 0.95. Hence, α = 0.05 and
α
2 = 0.025. Therefore,
α
1 − = 0.975
2

The interval will be of the form

r r
S2 S2
[X̄ − t1− α2 , X̄ + t1− α2 ]
n n

where all the values are known except for the valuest that correspond to a t − student
with n − 1 = 99 degrees of freedom. In this case we have to look up the tables for the
value
t1− α2 = t0.975

That is, the value of a t − student with 99 degrees of freedom that has to its left a
probability of 0.975. In the tables we find (since 99 degrees of freedom does not appear
in the tables we take the nearest value, 100 degrees of freedom)

t0.975 (99) = 1.984

Thus r r
S2 S2
[X̄ − t1− α2 , X̄ + t1− α2 ]=
n n
r r
980.000 980.000
= [26.000 − 1.984 , 26.000 + 1.984 ]
100 100

Doing the computations, we finally get

µ ∈ [25.803, 56, 26.196, 42] with a probability of 95%

[Link] Confidence Interval for the variance

In a similar manner, we can also construct a confidence interval for the case of the
population variance. We must remember, though, that in this case the population must
follow a Normal distribution for otherwise the distribution of the sample variance S 2
would be unknown. We know the2 that
(n − 1)S 2
∼ χ2n−1
σ2

and hence
(n − 1)S 2
p(χ α2 ≤ ≤ χ1− α2 ) = 1 − α
σ2
1.4. POINT AND INTERVAL ESTIMATION 29

where χ α2 is the value of a χ2n−1 whose left tail contains an area of α2 and that can be
found in tables. Similarly, χ1− α2 is the value of a χ2n−1 whose left tail contains an area
of 1 − α2 .
As before, we can work the inequalities out to obtain

1 σ2 1
p( ≥ ≥ )=1−α
χ2α (n − 1)S 2 χ1− α2

(n − 1)S 2 (n − 1)S 2
p( ≥ σ2 ≥ )=1−α
χ α2 χ1− α2

that is,
(n − 1)S 2 (n − 1)S 2
σ2 ∈ [ , ] with probability 1 − α
χ1− α2 χ α2

Example 1.4.4 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from a
Normal population with unknown mean and variance. Construct a confidence interval
with a confidence level of 95% for the population variance σ 2 if we know that the
sample variance is S 2 = 4.800
If the confidence level is 95% we have that 1 − α = 0.95. Hence, α = 0.05 and
α
2 = 0.025. Therefore,
α
1 − = 0.975
2
The interval will be of the form

(n − 1)S 2 (n − 1)S 2
[ , ]
χ1− α2 χ α2

where all the values are known except for the values χ that correspond to a chi-square
with n − 1 = 99 degrees of freedom. In this case we have to look up the tables for the
value
χ1− α2 = χ0.975 and χ α2 = χ0.025

That is, the values of a chi-square with 99 degrees of freedom that have to its left a
probability of 0.975 i 0.025 respectively, In the tables we find

χ0.975 = 129.561 i χ0.025 = 74.222

Thus,
(n − 1)S 2 (n − 1)S 2
[ , ]=
χ1− α2 χ α2

99·4.800 99·4.800
=[ , ]
129.561 74.222
Doing the computations, we finally get

σ 2 ∈ [3667.77, 6402.41] with a probability of 95%

30 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

[Link] Confidence Interval for the proportion

The case of the proportion is special for, as said before, the approximation to the Nor-
mal requires a large sample (nπ(1 − π) > 5)
Then we will have
π(1 − π)
π̂ ∼ N (π, )
n
and, similarly as in the case of the confidence interval for the mean, we get:
r r
π̂(1 − π̂) π̂(1 − π̂)
π ∈ [π̂ − z1− 2α , π̂ + z1− 2
α ] with probability 1 − α
n n

Example 1.4.5 In an random sample of 1000 people, 450 declare that they smoke on
a regular basis. Construct a confidence interval with a confidence level of 95% for the
proportion of smokers, π, in the population from which the sample has been obtained.
If the confidence level is 95% we have that 1 − α = 0.95. Hence, α = 0.05 and
α
2 = 0.025. Therefore,
α
1 − = 0.975
2

Let us first compute the sample proportion, that is, the proportion of smokers in the
sample. In this case
450
π̂ = = 0.45
1000

The interval will be of the form

r r
π̂(1 − π̂) π̂(1 − π̂)
[π̂ − z1− α2 , π̂ + z1− α2 ]
n n

where all the values are known except for the values Z that correspond to a Normal
distribution. In this case we have to look up the tables for the value

Z1− α2 = Z0.975

That is, the value of a N (0, 1) that has to its left a probability of 0.975. In the tables
we find
Z0.975 = 1.96

Thus, r r
π̂(1 − π̂) π̂(1 − π̂)
[π̂ − z1− α2 , π̂ + z1− α2 ]=
n n
r r
0.45(1 − 0.45) 0.45(1 − 0.45)
= [0.45 − 1.96 , 0.45 + 1.96 ]
1000 1000

Doing the computations, we finally get

π ∈ [0.4191, 0.4808] with a probability of 95%

1.4. POINT AND INTERVAL ESTIMATION 31

RStudio Activity [Link] - Interval Estimation

We will use RStudio to compute confidence intervals for the three cases studied above:
the population mean, the population variance and the population proportion.
The script for this activity is:

1. Confidence interval for the population mean in a Normal population (or large
sample) and σ 2 known
2. Confidence interval for the population mean in a Normal population (or large
sample) and σ 2 unknown
3. Confidence interval for the population variance in a Normal population.

4. Confidence interval for the population proportion.

In each case we will do the following:

(a) Generate a random sample (from a Normal distribution in the cases of the mean
and variance, from a Bernoulli distribution in the case of the proportion).
(b) Compute the corresponding confidence interval in each case by calculating it step
by step with RStudio using the formulas we have seen in the previous sections.
(c) Compute the same confidence interval but now directly using RStudio commands
tailored for this. We will see that, in practice, this is the best option.
1. Confidence interval for the population mean in a Normal population (or large
sample) and σ 2 known
We will find a 95% confidence interval for the population mean µ.
(a) Let us generate a random sample of size n = 2500, that will be labeled
SAMPLE _1, form a Normal distribution with population mean µ = 1000
and standard deviation σ = 200. That is, a N (1000, 40000).
[Link](1)
SAMPLE_1 <- rnorm(2500,1000,200)

(b) Recall the formula for the interval in this case:

" r r #
σ2 σ2
µ ∈ X̄ − z1− α2 , X̄ + z1− α2
n n
which can be written as:
 
µ ∈ X̄ − error, X̄ + error

where r
σ2
error = z1− α2
n
α
If the confidence level (1 − α) is 95%, then 1 − 2 = 0.975
r
σ2
error = z0.975
n
32 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

We will use RStudio to compute this interval using the above formula.
In the code that follows each element in the formula is identified with the
names in this table:

X̄ MEAN σ2 VARIANCE

z0.975 Z _ VALUE n SAMPLE _ SIZE

### Compute the elements of the formula

MEAN = mean(SAMPLE_1)
VARIANCE = 40000 ### assuming it's known
SAMPLE_SIZE = 2500
Z_VALUE <- qnorm(0.975,0,1)
ERROR = Z_VALUE * sqrt(VARIANCE/SAMPLE_SIZE)

### Now we can compute the left (LEFT_END)

### and right extremes (RIGHT_END) of the interval
### according to the formula

LEFT_END <- MEAN - ERROR

RIGHT_END <- MEAN + ERROR

### Show the interval

paste("[",LEFT_END,",",RIGHT_END,"]")

## [1] "[ 990.291173064218 , 1005.97088494054 ]"

Thus we have: µ ∈ [990.2911731, 1005.9708849]

(c) We will use now the library TeachingDemos to compute the same inter-
val, but in a much simpler way. We must first load the library into RStu-
dio, and then the command [Link]() will take care of the computation.
We only need to enter the value of σ (stdev) and the confidence level
([Link])
library(TeachingDemos)

[Link](SAMPLE_1, stdev=200, [Link]=0.95)

##
## One Sample z-test
##
## data: SAMPLE_1
## z = 249.53, n = 2500, Std. Dev. = 200, Std. Dev. of the sample mean =
## 4, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
## 990.2912 1005.9709
## sample estimates:
## mean of SAMPLE_1
## 998.131

The output of this command produces much information, most of which

will be of interest later on. For the moment, let’s pay attention to the “95
percent confidence interval” result to notice that we have ex-
actly the same interval that we had computed before, only that much faster:
µ ∈ [990.2912, 1005.9709]
1.4. POINT AND INTERVAL ESTIMATION 33

It is clear that this is the command to use in RStudio to find confidence

intervals instead of computing the formula step-by-step.
2. Confidence interval for the population mean in a Normal population (or large
sample) and σ 2 unknown
In a similar way we will compute a 95% confidence interval for the population
mean µ when the population variance σ 2 is unknown and hence must use the
sample variance S 2 .
(a) As before, let us generate a random sample of size n = 2500, that will now
be labeled SAMPLE _2, form a Normal distribution with population mean
µ = 1000 and standard deviation σ = 200. That is, a N (1000, 40000).
[Link](1)
SAMPLE_2 <- rnorm(2500,1000,200)

(b) Recall the formula for the interval in this case (unknown population variance):
" r r #
S2 S2
µ ∈ X̄ − t1− α2 , X̄ + t1− α2
n n

Note that we are using the sample variance S 2 (for σ 2 is unknown) and,
consequently, the t − student distribution instead the Normal distribution.
As before, this can be written as:
 
µ ∈ X̄ − error, X̄ + error

with: r
S2
error = t1− α2
n
For a 95% confidence level we have 1 − α = 0.95. Thus α = 0.05 and
1 − α2 = 0.975
r
S2
error = t0.975
n
We will use RStudio to compute this interval using the above formula. In
the code that follows each element in the formula is identified with the
names in this table:

X̄ MEAN S2 VARIANCE

t0.975 T _ VALUE n SAMPLE _ SIZE

### Compute the elements of the formula

MEAN = mean(SAMPLE_2)

##### We must now comupute the sample

### variance S^2
VARIANCE = var(SAMPLE_2)
34 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

SAMPLE_SIZE = 2500

### We must now use a t-student

### with n-1 = 2499 degrees of freedom
T_VALUE <- qt(0.975,2499)

ERROR = T_VALUE * sqrt(VARIANCE/SAMPLE_SIZE)

### Now we can compute the left (LEFT_END)

### and right extremes (RIGHT_END) of the interval
### according to the formulal

LEFT_END <- MEAN - ERROR

RIGHT_END <- MEAN + ERROR

### Show the interval

paste("[",LEFT_END,",",RIGHT_END,"]")

## [1] "[ 989.996685681592 , 1006.26537232316 ]"

Thus we have: µ ∈ [989.9966857, 1006.2653723]

(c) We will use TeachingDemos now to find the interval. In this case the
command to use is [Link]() since we are using a la t-student distribu-
tion. Also, we do not need to enter stdev as it is unknown. RStudio will
take care of computing the sample standard deviation (sd) when running
the [Link]() command.
[Link](SAMPLE_2,[Link]=0.95)

##
## One Sample t-test
##
## data: SAMPLE_2
## t = 240.62, df = 2499, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
## 989.9967 1006.2654
## sample estimates:
## mean of x
## 998.131

Again, we get the same interval as when computed using the formula, but
in a much easier way.

µ ∈ [989.9967, 1006.2654]

3. Confidence interval for the population variance in a Normal population.

We will now find a 95% confidence interval for the population variance σ 2 .

(a) Again, generate a random sample of size n = 2500, that will now be labeled
SAMPLE _3, form a Normal distribution with population mean µ = 1000
and standard deviation σ = 200. That is, a N (1000, 40000).
[Link](3)
SAMPLE_3 <- rnorm(2500,1000,200)
1.4. POINT AND INTERVAL ESTIMATION 35

(b) Recall the formula for the interval in this case:

(n − 1)S 2 (n − 1)S 2
σ2 ∈ [ , ]
χ1− α2 χ α2

Now we need to use a χ2 distribution with n − 1 degrees of freedom.

As before, for a 95% level we have 1 − α = 0.95, α = 0.05 α2 = 0.025,
and 1 − α2 = 0.975
We will us RStudio to compute the formula. In the code that follows each
element in the formula is identified with the names in this table:
χ0.975 CHI _ VALUE _0.975 S2 VARIANCE
χ0.025 CHI _ VALUE _0.025 n SAMPLE _ SIZE

### Compute the elements of the formula

VARIANCE = var(SAMPLE_3)

SAMPLE_SIZE = 2500

CHI_VALUE_0.975 <- qchisq(0.975,2499)

CHI_VALUE_0.025 <- qchisq(0.025,2499)

### Now we can compute the left (LEFT_END)

### and right extremes (RIGHT_END) of the interval
### according to the formula

LEFT_END <-((SAMPLE_SIZE-1)*VARIANCE)/CHI_VALUE_0.975
RIGHT_END<-((SAMPLE_SIZE-1)*VARIANCE)/CHI_VALUE_0.025

### Show the interval

paste("[",LEFT_END,",",RIGHT_END,"]")

## [1] "[ 37744.9222190065 , 42172.4533145744 ]"

Thus we have: σ 2 ∈ [3.7744922 × 104 , 4.2172453 × 104 ]

(c) Again, using TeachingDemos we can find the same interval easily. The
command to use is [Link]() in this case.
[Link](SAMPLE_3,[Link]=0.95)

##
## One sample Chi-squared test for variance
##
## data: SAMPLE_3
## X-squared = 99625727, df = 2499, p-value < 2.2e-16
## alternative hypothesis: true variance is not equal to 1
## 95 percent confidence interval:
## 37744.92 42172.45
## sample estimates:
## var of SAMPLE_3
## 39866.24

Again, the same interval as when using the formula

σ 2 ∈ [37744.92, 42172.45]
36 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

4. Confidence interval for the population proportion.

We will finally construct a 95% confidence interval for the population propor-
tion π.
We must remember that in this case the population of origin does not follow
a Normal distribution but a Bernoulli: any element in the population takes the
value 1 if it has the characteristic that we are studying, and 0 otherwise, being π
the proportion of elements in the population that have such characteristic.
Therefore, we will use a Bernoulli to generate the sample, with xi = 1 if the
i−th element in the sample has the characteristic of interest and xi = 0 if it does
not.
(a) Generate a sample of size n = 2500, that we will now label SAMPLE _4,
from a Bernoulli distribution with parameter p = 31 . That is, a b(1/3).
[Link](4)
SAMPLE_4 <- rbinom(2500,1,1/3)

(b) The formula for the interval in this case is:

" r r #
π̂(1 − π̂) π̂(1 − π̂)
π ∈ π̂ − z1− α2 , π̂ + z1− α2
n n
which once more we like to write as:

π ∈ [π̂ − error, π̂ + error]

In this case we have that,

r
π̂(1 − π̂)
error = z1− α2
n
Once again, for a 95% confidence level it follows that α = 0.05 and 1 −
α
2 = 0.975
In the code that follows each element in the formula is identified with the
names in this table
π̂ PROPORTION
z0.975 Z _ VALUE
n SAMPLE _ SIZE

### Compute the elements of the formula

PROPORTION = mean(SAMPLE_4)

SAMPLE_SIZE = 2500

Z_VALUE <- qnorm(0.975,0,1)

ERROR=Z_VALUE*sqrt((PROPORTION*(1-PROPORTION))/SAMPLE_SIZE)

### Now we can compute the left (LEFT_END)

### and right extremes (RIGHT_END) of the interval
### according to the formula
1.4. POINT AND INTERVAL ESTIMATION 37

LEFT_END <- PROPORTION - ERROR

RIGHT_END <- PROPORTION + ERROR

### Show the interval

paste("[",LEFT_END,",",RIGHT_END,"]")

## [1] "[ 0.307625444789947 , 0.344374555210053 ]"

Thus we have: π ∈ [0.3076254, 0.3443746]

(c) Once more, using T EACHING D EMOS we will find the same interval. We
will now use the command [Link]() again (as in the first case) as the
formula of this interval uses the values that correspond to a standard Nor-
mal distribution
### Recall that in the case of a Bernoulli
### the variance is p(1-p). Thus, the
### standard deviation is sqrt(p(1-p))
### In this example we are using a
### Bernoulli with p=1/3

[Link](SAMPLE_4,stdev=sqrt((1/3)*(1-1/3)),[Link]=0.95)

##
## One Sample z-test
##
## data: SAMPLE_4
## z = 34.578, n = 2.5000e+03, Std. Dev. = 4.7140e-01, Std. Dev. of the
## sample mean = 9.4281e-03, p-value < 2.2e-16
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
## 0.3075213 0.3444787
## sample estimates:
## mean of SAMPLE_4
## 0.326

Not surprisingly, we have the same interval9

π ∈ [0.3075213, 0.3444787]

R Hints

• Notice that in the first chunk of code, when the random sample is generated,
we use the command

[Link](1)

The use of this command is to initialize (seed) in a specific way (1 in this

case) the algorithm that generates the random sample. Thus, as long as the
"seed" is 1, the generated sample will always be the same. If we change
this seed, for example,

[Link](27)

then the sample will be different.

9 There is a small difference as, remember, the use of a Normal distribution here is just an approximation.
38 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

This is very useful to keep the sample, and thus the results obtained, from
changing every time the code is run.
For example, making this textbook would be very difficult if every time
we edit it and change something, the results change when saving the new
version and generate a new PDF file (which is when the code is run). There
would be no way to comment on them!
If you want to work with different random samples, simply don’t use this
[Link] command and the sample will be different each time.

End of Activity RStudio [Link]

1.5 Properties of estimators: bias, efficiency and con-

sistency
Once the main statistics and their probabilistic features (i.e. probability distribution,
expectation and variance) are known, we focus in this chapter on the "good" properties
that we would like the estimators to have in order for them to provide good approxi-
mations to the parameters of interest. In this sense, an estimator might, among others,
satisfy the properties of being unbiased, efficient, and consistent that we will see next.
In what follows we will use the generic notation θ̂ in reference to a population param-
eter θ, that is not specified.

1.5.1 Bias

Definition 1.5.1 Let θ̂ be an estimator of the population parameter θ. The bias of θ̂

is defined as the difference between the expected value of the estimator and the true
value of the population parameter

B(θ̂) = E(θ̂) − θ

Definition 1.5.2 An estimator θ̂ is said to be an unbiased estimator of the population

parameter θ if its bias is zero, that is:

B(θ̂) = 0 (or E(θ̂) = θ)

Example 1.5.3 Let {x1 , x2 , . . . , xn } be a random sample drawn from a population

with population mean µ. Then, for the sample mean X̄ we have:

E(X̄) = µ

Thus,
1.5. PROPERTIES OF ESTIMATORS 39

X̄ is and unbiased estimator of µ

Example 1.5.4 Let {x1 , x2 , . . . , xn } be a random sample drawn from a population

with population variance σ 2 . Then, for the population variance S 2 we have:

E(S 2 ) = σ 2

Thus,

S 2 is an unbiased estimator of σ 2

Example 1.5.5 Let {x1 , x2 , . . . , xn } be a random sample drawn from a population

with population proportion π. Then, for the sample proportion π̂ we have:

E(π̂) = π

Thus,

π̂ is an unbiased estimator of π

The interpretation of the unbiased property is simple. For what we have seen in the
previous chapter, we know that an estimator is a random variable, that is, takes different
values with different probabilities. Hence, it is clear that it is highly unlikely that the
specific value (estimate) that we get once we apply the sample to the estimator exactly
coincides with the true parameter value. What the unbiased property means is that the
above is true "in the sense of expectation". In other words, although when we apply
the specific sample we have to the estimator the estimate will not coincide (in general)
with the true value of the parameter, if we had 100 different samples to apply to the
estimator then the average of the 100 different estimates produced would be very close
to the true parameter value. This kind of approximation would be more precise the
larger is the number of samples to use.
We can compare an estimator with a "shooter" whose target is the true value of the pa-
rameter. A good "shooter" (unbiased) always aims at the center of the target, although
there is always a small probability that the shot slightly deviates from the center. On
the contrary, a bad "shooter" (biased) never aims at the center of the target.

1.5.2 Efficiency

The efficiency criterion for an estimator, that we will see next , has two different ver-
sions depending on whether the estimator is biased or unbiased.
40 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

[Link] Unbiased Estimators

Definition 1.5.6 Let θ̂1 and θ̂2 be two unbiased estimators of θ. Then, the more
efficient estimator is that of the lesser variance.

[Link] Biased Estimators

Definition 1.5.7 Let θ̂1 and θ̂2 be any two estimators of θ. Then, the more efficient
estimator is that of the lesser Mean Quadratic Error (M QE) where:

M QE(θ̂) = E(θ̂ − θ)2 = V (θ̂) + B(θ̂)2

It is easy to see that, in fact, the second "version" contains the first one as a special
case. Indeed, it an estimator has zero bias’s then its M QE and Variance are equal.

Example 1.5.8 Let us consider the following two alternative estimators of the popu-
lation mean µ which will be applied to a sample of size n obtained from a population
with population mean µ and population variance σ 2
x1 + x2 + x3
µ̂1 =
3

x1 + x2
µ̂2 =
2
Let us check first the bias’s of each of these estimators:
x1 + x2 + x3
B(µ̂1 ) = E(µ̂1 ) − µ = E( )−µ=
3
1
= (E(x1 ) + E(x2 ) + E(x3 )) − µ =
3
1
= 3µ − µ = µ − µ = 0
3

x1 + x2
B(µ̂2 ) = E(µ̂2 ) − µ = E( )−µ=
2
1
= (E(x1 ) + E(x2 )) − µ =
2
1
= 2µ − µ = µ − µ = 0
2
Hence, both estimators and unbiased. Let us now check which one has less variance:
x1 + x2 + x3
V (µ̂1 ) = V ( )=
3
1
= (V (x1 ) + V (x2 ) + V (x3 )) =
9
1 2 σ2
= 3σ =
9 3
1.5. PROPERTIES OF ESTIMATORS 41

x1 + x2
V (µ̂2 ) = V( )=
2
1
= (V (x1 ) + V (x2 )) =
4
1 2 σ2
= 2σ =
4 2

2
σ2
Therefore, µ̂1 is more efficient as it has less variance ( σ3 < 2 )

The intuition behind the efficiency of an estimator is also clear. If we compare an

unbiased estimator with a "good shooter" (as we have done before) that always aims
at the center of the target, then an estimator is more efficient than another one if it
"trembles" less. In other words, the more efficient estimator is the one whose values
are more concentrated around the mean.

1.5.3 Asymptotic properties

Very often it becomes very difficult to find efficient estimators for a specific parameter.
In this case we look at the so called asymptotic properties, that consist of the properties
that the estimators have when the sample is as large as needed. In this sense, we will
introduce the asymptotic bias and the asymptotic efficiency or consistency.

[Link] Asymptotically unbiased estimators

Definition 1.5.9 An estimator θ̂ of the population parameter θ is said to be asymp-

totically unbiased if its bias vanishes as the sample size goes to infinity. Formally, θ̂
is an unbiased estimator of θ if

lim B(θ̂) = 0
n→∞

Example 1.5.10 Let us consider the following estimator of the population variance
(σ 2 ) Pn
2 (xi − X̄)2
S̃ = i=1
n

It is easy to check that if Pn

− X̄)2
i=1 (xi
S2 =
n−1

then
n−1 2
S̃ 2 = S
n

and hence
n−1 2 n−1 n−1 2
E(S̃ 2 ) = E( S )= E(S 2 ) = σ
n n n
42 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Therefore
n−1 2 σ2
B(S̃ 2 ) = E(S̃ 2 ) − σ 2 = σ − σ2 = −
n n

That is, S̃ 2 is a biased estimator of σ 2 since E(S̃ 2 ) ̸= σ 2 . Nevertheless, S̃ 2 is an

asymptotically unbiased estimator of σ 2 , for its bias vanishes as the sample grows.
Indeed,
σ2
lim B(S̃ 2 ) = lim − =0
n→∞ n→∞ n

[Link] Consistent Estimators

The property of consistency not only considers the behavior of the bias as the sample
grows large, but also looks at the variance. That is, consistency refers to the behavior
of the M QE of the estimator as the sample size goes to infinity.

Definition 1.5.11 An estimator θ̂ of the population parameter θ is said to be consis-

tent it its Mean Quadratic Error vanishes as the size of the sample goes to infinity.
Formally, θ̂ is a consistent estimator of θ if

lim EQM (θ̂) = 0

n→∞

Example 1.5.12 Let us consider the estimator of σ 2 that we have seen before, S̃ 2 . We
2
already know that it it a biased estimator for σ 2 and that its bias is B(S̃ 2 ) = − σn . We
will compute now its variance in order to study the behavior of its EQM as the sample
size goes to infinity

n−1 2 n−1 2 (n − 1)2 2(σ 2 )2 2(n − 1)σ 4

V (S̃ 2 ) = V ( S )=( ) V (S 2 ) = =
n n n2 n−1 n2

Hence
2(n − 1)σ 4 σ2 2 (2n − 1)σ 4
EQM (S̃ 2 ) = V (S̃ 2 ) + B(S̃ 2 )2 = + (− ) =
n2 n n2

and then
(2n − 1)σ 4
lim EQM (S̃ 2 ) = lim =0
n→∞ n→∞ n2

Therefore, S̃ 2 is a consistent estimator of σ 2

RStudio Activity 1.5.1 - Consistent Estimators

We will use RStudio now to study the consistency of the two population variance esti-
mators seen above:
Pn 2
i=1 (xi −X̄)
• The estimator S 2 = n−1 , and
Pn 2
i=1 (xi −X̄)
• The biased estimator S̃ 2 = n
1.5. PROPERTIES OF ESTIMATORS 43

The sript for this activity is:

1. We will generate 1000 samples, each of them of size n (n will vary from 10 to
500 observations), of a Normal variable with mean µ =0 and variance σ 2 =
400, N (0, 400).
2. For each of these samples we will calculate the Quadratic Error of the Variance
with the formula S 2 (QE) and also with the biased formula S̃ 2 (QE_biassed)
3. Once we have these 1000 samples of size n, and their corresponding QE and
QE_biassed, we will calculate the mean of these Quadratic Errors, that is,
the Mean Quadratic Error (MQE) and the Mean Quadratic Error of the biased
estimator (MQE_biased)
4. Finally, we will check if, according to the consistency property, these MQEs
approach zero as n (sample size) increases; and which of the two estimators,
S 2 or S̃ 2 , exhibits a better behavior.
Given that in this experiment we must use two loops (one that goes from 1 to 1000 for
the generation of the samples, and another that goes from 10 to 500 to vary the size of
these samples ), items 1, 2, and 3 in this script must be done simultaneously in a single
chunk of code in RStudio.
First, we need to create a database ([Link]) to store the results obtained
[Link](3)
### We create a database (data frame), which we will call MQE_DATA,
### with two variables: MQE and MQE_biased.
### The database will be empty for now.
### Then we will add the results (MQE and MQE_biased) of each
### one of the samples
MQE_DATA <- [Link](MQE=double(),MQE_biased=double())

### Let's start the loop where the 'n' (sample size)
### will range from 1 to 500 (in step increments of 1)

for (n in seq(10,500, 1))

{
### Reset QEs to zero
QE <- 0
QE_biased <- 0

### Let's start the loop where the 1000

### samples of size 'n' will be generated
for (i in 1:1000)
{
SAMPLE <- rnorm(n,0,20) ### recall that if the variance is 400
### the standard deviation is 20

### Compute both QEs

### Note that the 'var()' command in R
### correspond to the unbiased formula S^2
QE<-QE + var(SAMPLE)-400
QE_biased<-QE_biased + (var(SAMPLE)*(length(SAMPLE)-1)/length(SAMPLE))-400
}

### Compute the means(MQEs) of the 1000 values

MQE = QE / 1000
MQE_biased = QE_biased / 1000

### Add these MQE and MQE_biased to the data base

MQE_DATA[nrow(MQE_DATA)+1,]=c(MQE,MQE_biased)
}
44 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

We now graphically analyze the behavior of these estimators as the sample size in-
creases, from 10 to 500.
### Represent graphically the evolution of MQE
plot(MQE_DATA$MQE,pch=10,type="l",col="blue",
xlab="Sample Size",ylab="MQE",ylim=c(-40,10))

### Add the evolution of EQM_biased

lines(MQE_DATA$MQE_biased,pch=10, col="red")

### Add a line at zero level

### to check the efficiency
abline(0,0)
10
0
MQE

−20
−40

0 100 200 300 400 500

Sample Size

We see how, indeed, the behavior of these estimators is as expected

• The S 2 estimator has a low MQE, even when the samples are small (it is more
efficient than S̃ 2 ). Furthermore, this MQE quickly approaches 0 as the sample
grows large. That is, it is consistent.
• The biased estimatorS̃ 2 exhibits a noticeable MQE when the samples are small (it
is clearly less efficient thanS 2 ), but also approaches 0 as the sample grows, it is
consistent as well.

Notes d’R

• For this activity we had to create a database to save the results that were pro-
duced during the simulation and thus be able to represent them graphically
at the end. A database is just a table with as many columns as variables to
1.6. METHODS OF POINT ESTIMATION 45

consider. In this case we had two variables, MQE and MQE_biased, and
what we have done is to save the result obtained for each different value of
the sample size. Since this value ranged from 10 to 500, in this database,
which we have named MQE_DATA, there are two columns with 490 values
for each of the two corresponding variables.
– To create the database, initially empty, we used the command
EQM_DATA <-
[Link](EQM=double(),EQM_biased=double()),
which creates the database ([Link]) with the two indicated
variables, without any value for the moment. The double()a option
that appears indicates that these variables will be real numbers
– Once created, we can view its content, if we want, by entering the
following command in the RStudio console:
View(EQM_DATA)
A new tab will appear with the database (table) and its values:

a The reason for using the double expression to refer to a real number comes from the origins of

programming. This indicated that the number corresponding to this variable would have a "double"
component: an integer part and a decimal part

End of Activity RStudio 1.5.1

1.6 Methods of point estimation: maximum likelihood

and method of moments
We have seen that when we need to produce estimations for population parameters
that are "standard", (µ σ 2 , π), there are good estimators at hand: (X̄, S 2 , π̂). We have
studied the main features and properties of these estimators. The problem appears when
we need to estimate a different population parameter (for instance the median or the
kurtosis) for which do not have a "candidate" for estimator. The Maximum Likelihood
method and the method of moments provide techniques to build good estimators of a
given population parameter.
46 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

1.6.1 Maximum Likelihood estimation

The intuition of this method is as follows: After performing a totally random sam-
pling (SRS) we obtain a specific sample, and there must be a reason for it (since we
could have obtained a different one). Well, probably we have obtained this specific
sample because the parameter value we want to estimate is such that the sample we
have obtained is the one with the highest probability of been selected. In this sense,
the maximum likelihood method finds the value of the parameter that maximizes the
probability of obtaining the sample at hand. The process takes three steps, starting
with the sample we have, {x1 , x2 , · · · xn } and the probability density function of the
population that contains the parameter (θ) we want to estimate, f (x; θ). We will first
introduce the general method, and later we offer an example to clarify it.
Imagine that we want to estimate the parameter θ of a population with a distribution
given by f (x; θ) using the sample that we have obtained {x1 , x2 , · · · xn }. These are
the 3 steps:

Step 1 B UILD THE L IKELIHOOD FUNCTION

The Likelihood function is the "formula" that computes the probability of having ob-
tained the sample we have conditional on the population parameter we want to estimate.
In other words, is a function (denoted by L) that depends on both the sample obtained
and the parameter we want to estimate:

L(x1 , x2 , · · · xn ; θ) = P (X1 = x1 , X2 = x2 , · · · Xn = xn ; θ)

Since the sample has been obtained from a population with a probability distribution
given by f (x; θ) and that the elements in the sample are independent from each other,
the joint probability P (X1 = x1 , X2 = x2 , · · · Xn = xn ; θ) can be computed as

P (X1 = x1 , X2 = x2 , · · · Xn = xn ; θ) = f (x1 ; θ) · f (x2 ; θ) · . . . · f (xn ; θ)

hence,
n
Y
L(x1 , x2 , · · · xn ; θ) = f (x1 ; θ) · f (x2 ; θ) · . . . · f (xn ; θ) = f (xi ; θ)
i=1

Step 2 Apply logarithms

The functional form of the likelihood function is often involved (the product of func-
tions), and working directly with it is rather difficult. Hence, using logarithms we can
simplify the function so that it becomes easier to deal with. Therefore, in this step we
simply apply “ln” and then use the properties of logarithms in order to simplify the
form of the likelihood function
n
Y n
X
ln L(x1 , x2 , · · · xn ; θ) = ln f (xi ; θ) = ln f (xi ; θ)
i=1 i=1

Step 3 M AXIMIZE
1.6. METHODS OF POINT ESTIMATION 47

The last step is to maximize the likelihood function, that is, to find the value of θ that
maximizes the function L (the probability of having obtained the sample we have).
Thus, we must compute the derivative of the likelihood function L with respect to
the parameter θ and make it equal to zero to find the value of θ that maximizes it.
Usually this is complicated, that is why we have applied logarithms in Step 2. Indeed,
since the function "logarithm" is strictly increasing, the value of θ that maximizes ln L
maximizes L as well. Hence, in practice, what we do is:
∂ ln L(x1 , · · · xn ; θ)
=0
∂θ

and from here we find the value of θ solves the above equation. The solution will be
the maximum likelihood estimator of θ, usually denoted by θ̂M L

Example 1.6.1 L ET {x1 , x2 , · · · xn } BE A SAMPLE ( INDEPENDENT ) OBTAINED FROM

A N ORMAL POPULATION WITH POPULATION MEAN µ AND POPULATION VARIANCE
σ 2 . F IND THE MAXIMUM LIKELIHOOD ESTIMATOR OF µ.
First, let us remember what is the probability density function corresponding to a
N (µ, σ 2 ):
1 1 x−µ 2
f (x; µ, σ 2 ) = √ e− 2 ( σ )
2πσ

Step 1 L IKELIHOOD F UNCTION

n
Y 1 1 xi −µ 2
L(x1 , x2 , · · · xn ) = e− 2 ( σ ) =
√
i=1
2πσ
 n
1 Pn xi −µ 2
· e− 2 i=1 ( σ )
1
= √
2πσ

This would be hard to work with !. That’s why we need to use logarithms.

Step 2 L OGARITHMS
 n 
1 − 21
Pn
( xiσ−µ )
2
ln L(x1 , · · · xn ) = ln √ ·e i=1

2πσ

It still looks hard, but after using some of the properties of logarithms10 the simplifica-
tion will be important
 n   n
1 Pn xi −µ 2 1 Pn xi −µ 2
· e− 2 i=1 ( σ ) = ln √ + ln e− 2 i=1 ( σ ) =
1 1
ln √
2πσ 2πσ

n n 2
xi − µ
 
1 1X
= ln √ − ln e
2πσ 2 i=1 σ

Hence n n 2
xi − µ
 
1 1X
ln L(x1 , · · · xn ) = ln √ −
2πσ 2 i=1 σ
10 The logarithm of the product is the sum of logarithms, etc.
48 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

Pas 3 M AXIMIZE

We have to compute the derivative of ln L(x1 , · · · , xn ) with respect to µ and equate it

to zero.
n n  2 !
∂ ln L(x1 , · · · xn ) 1 X xi − µ

∂ 1
= ln √ − =
∂µ ∂µ 2πσ 2 i=1 σ

n n 2
xi − µ
 
∂ 1 ∂ 1X
= ln √ − =
∂µ 2πσ ∂µ 2 i=1 σ

n 2 n n
xi − µ xi − µ xi − µ
    
1X ∂ 1X 1 X
=0− =− 2 (− ) =
2 i=1 ∂µ σ 2 i=1 σ σ i=1
σ2

Hence,
n n n
∂L(x1 , · · · xn ) xi − µ
 
X 1 X X
=0⇒ =0⇒ ( x i − µ) = 0
∂µ i=1
σ2 σ 2 i=1 i=1

and finally,
n n n Pn
X X X
i=1 xi
xi = µ⇒ xi = nµ ⇒ µ =
i=1 i=1 i=1
n

That is, the maximum likelihood estimator of the population mean µ is the sample mean
X̄ Pn
xi
µ̂M L = i=1 = X̄
n

1.6.2 Method of moments

Consider a population distributed according to the density function f (x, θ), where θ is
the unknown population parameter that we want to study.
The method of moments proceeds in 3 simple steps

Step 1 Compute the expectation of X according to the density function above and
make it equal to the average µ of the population
Z
µ = E(X) = xf (x, θ)dx

The result of this integral will be a function of the parameter θ. Hence we should
have something as
µ = g(θ)

Step 2 Since we know that X̄ is a “good” estimator of µ, we just set µ = X̄ , that is:

X̄ = g(θ)
1.6. METHODS OF POINT ESTIMATION 49

Step 3 Finally, just “inverting” the function g we can express θ as a function of X̄

and we are done ! We have found an estimator for θ that is called the method of
moments estimator θ̂M M
θ̂M M = g −1 (X̄)

Example 1.6.2 Consider a population distributed according to the density function

(θ + 1)xθ

0≤x≤1
f (x, θ) =
0 otherwise

Find the Method of Moments estimator of θ

—————

Step 1 Expectation
Z 1 Z 1
θ
µ = E(x) = x(θ + 1)x dx = (θ + 1) x(θ+1) dx =
0 0

1
x(θ+2)

(θ + 1)
= (θ + 1) =
θ+2 0 (θ + 2)

Hence, we can write

θ+1
µ=
θ+2

Step 2 Use the estimation of µ

θ+1
X̄ =
θ+2

Step 3 Solve for θ

1 − 2X̄
θ̂M M =
X̄ − 1

RStudio Activity 1.6.1 - Maximum Likelihood estimation

We will now use RStudio to find a maximum likelihood estimate of the parameter λ of
an exponential distribution.

Recall that we can use the exponential distribution to measure the elapsed time between
the occurrence of two independent events. In this example we will simulate the time
that passes between the arrival of a flight at the control tower of El Prat airport and the
arrival of the next one, to end up estimating the average time between flights.

The script for this activity will be:

50 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

1. Generate a random sample from an exponential distribution, which will rep-

resent our sample for the study. The values obtained will be interpreted as
observations of the elapsed time (in minutes) between one flight and the next
one.
2. Find the maximum likelihood estimate of the parameter λ of the exponential
distribution from the data in the generated sample.
3. Compute the "mean time elapsed between flight and flight" using the property
we already know that, in the case of the exponential distribution,
1
E(X) =
λ

1. Generate a sample of size n = 1883, that will be labeled FLIGHTS _ SAMPLE,

from an exponential distribution with parameter λ = 0.65. That is, an exp(0.65).
[Link](10)
FLIGHTS_SAMPLE <- rexp(1883,0.65)

We can take a look at the 5 first values in the sample (elapsed time in minutes)
head(FLIGHTS_SAMPLE,5)

## [1] 0.02300986 1.41572493 1.15716760 2.42314131 0.35639787

2. We next load the package fitdistrplus11 into RStudio and use the com-
mand fitdist to find the maximum likelihood estimate of λ. The output of
this command will be stored in a variable that we call MLE_Estimate.
library(fitdistrplus)
MLE_Estimate <- fitdist(data = FLIGHTS_SAMPLE,
distr = 'exp', method = 'mle')

In this command we have used as options the distribution we are dealing with
(’exp’, for exponential in this case) and the estimation method we want to use
(’mle’, per Maximum Likelihood Estimation)
We now ask RStudio to show the result of the estimation.
MLE_Estimate$estimate

## rate
## 0.6408337

Thus, the estimate of the parameter λ found by maximum likelihood is:

λ̂M V =0.6408337

3. Finally, use the relation E(X) = λ1 to find the estimate of the average elapsed
time between flights 1/0.6408337 = 1.5604672.
That is, the estimation is 1.5604672 minutes between one flight and the next one.
11 If it is not installed on your system you can do so by using the command
[Link]("fitdistrplus", dependencies=TRUE)
1.6. METHODS OF POINT ESTIMATION 51

Comment for the smart reader

Yes indeed, you are right.
If we wanted to estimate "the average time between flights" we could simply have
calculated the sample average of the values obtained in the sample:
### Compute the sample mean
mean(FLIGHTS_SAMPLE)

## [1] 1.560467

Clearly, it gives the same result (1.5604672) as we had obtained by doing the maximum
likelihood estimation.
This is because, as in the case of the Normal distribution that we saw in example 1.6.1,
the sample mean is also a maximum likelihood estimator of the population mean in the
case of the exponential distribution. In other words:
1
λ̂M V =
X

The interest of this activity, though, was to work in RStudio with the maximum likeli-
hood estimation technique, and not so much to find the specific value of the average.

R Hints

• In this activity, when we wanted RStudio to show us the result of the maxi-
mum likelihood estimation (which was stored in the MLE_Estimate vari-
able) we used the command

MLE_Estimate$estimate

The reason for this is that the variable MLE_Estimate is a variable that
stores a lot of information about the maximum likelihood procedure that we
performed. To see this, if in the console of RStudio we enter the command

View(MLE_Estimate)

a new tab with the following information will appear :

52 CHAPTER 1. INTRODUCTION TO INFERENTIAL STATISTICS

We note in the second row of this table that estimate is the "component"
that contains the information about the value of the estimate. To access this
value we use the $ sign followed by the name of the component we want to
consult. In other words:
MLE_Estimate$estimate

## rate
## 0.6408337

End of Activity RStudio 1.6.1

Chapter 2

Parametric Hypothesis Testing

This house, this poem ...

This fresh hypothesis

I AIN C RICHTON S MITH

2.1 Concept of parametric test: null hypothesis and al-

ternative hypothesis

So far we have learned how to approximate the value of a population parameter by

means of some estimation technique. In many cases, though, what is of interest is
not knowing what is the value of the parameter but rather some question regarding the
parameter. For instance: Is the average wage in Cerdanyola this year greater than in the
previous year ? Does this Intel chip works at 3000 MHz ? Is the proportion of smokers
in the UAB smaller than in the UPF ?
In all these cases, we are interested in testing whether a belief, idea, or conjecture
regarding the population parameter seems true or not. To do so, to test our hypothesis,
we must base our analysis upon the data we have, the sample, because this is the only
information we have regarding the population. We can now be more precise in the
definition of "hypothesis testing"

Definition 2.1.1 Hypothesis testing is a statistical technique by means of which we can

verify whether the data in the sample backs up, or not, a specific hypothesis established
on some population parameter.

53
54 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

In general, the "structure" of the hypothesis testing technique can be decomposed into
6 steps. To understand how the procedure works, let’s imagine that we want to test
whether the population parameter θ equals θ0 or not. In this case, the 6 steps we have
mentioned above are:

1. To specify the Null Hypothesis. This is the hypothesis that we believe is true and
that we want to test if the data supports it.
H0 : θ = θ 0

2. To specify an Alternative Hypothesis, which represents what is true when the

Null Hypothesis is false. This Alternative Hypothesis may have four different
specifications, depending on the information we have on the population parame-
ter we are studying.

H1 : θ ̸= θ0

or H1 : θ < θ0

or H1 : θ > θ0

or H1 : θ = θ1
The first kind of Alternative Hypothesis is the more general. It corresponds to
the case when there is no information at all regarding the population parameter.
In this sense, if the parameter is not what we believe (θ0 , the Null Hypothesis),
then we simply specify than it is different from what we think.
The second kind corresponds to the case when there is some information regard-
ing the parameter we are studying. In this specific case such information points
out that if the parameter is not what we think, then it must be smaller (for some
reason it is known that it can not be larger).
The third kind of Alternative Hypothesis is the opposite to the previous case. We
use this specification when the information says that if the parameter is not what
we think then it must have a larger value.
Finally, the last specification, which is rather rare, corresponds to the case in
which there is a lot of information regarding the parameter. In this case, we
know that either the parameter takes the value we believe (θ0 ) or it takes another
specific value θ1
Later we will see that the first kind of Alternative Hypothesis produces a two-
tailed test, whereas the second a the third correspond to a left-tail test and a right-
tail test respectively. Finally, the last kind of hypothesis produces a left-tail test
or a right-tail test depending on whether θ1 < θ0 or θ1 > θ0 respectively.
3. To specify a test statistic (TS) and to compute the observed value of the test
statistic, (OVTS) using the data in the sample.
In practice, what distinguishes one hypothesis test from another is the test statis-
tic used. Hence, we will see these test statistics in detail when we introduce each
specific test.
2.2. TEST STATISTIC AND ERROR TYPES 55

4. To determine what is the probability distribution of the test statistic in the pre-
vious step under the assumption that the Null Hypothesis is true. This, as in the
previous step, depends on what kind of test we are conducting. Hence, we will
see the details later.

5. To define a Rejection Area, (RA) of size α (level of significance). This is the

place where the test actually takes place. For this, we need to use the tables
that correspond to the distribution determined in step 4 to find a region with
the property that if the null hypothesis is true then the probability that the test
statistic lies within this RA is α.

p(T S ∈ RA) = α
Notice that when we say “”Rejection Area of size α” we are referring to its size
in terms of probability, not in terms of the length of the interval that determines
such area.
In general, this Rejection area consists of only one tail (left- or right-tailed tests)
of size α or can be split into two symmetric tails of size α2 each.

6. Finally, the last step consists of, simply, verifying whether the Observed Value
of the Test Statistic, (OVTS) lies, or not, inside the Rejection Area. Therefore,

(a) If the OVTS is inside the RA ⇒ THE NULL HYPOTHESIS IS REJECTED

(b) If the OVTS is NOT inside the RA ⇒ THE NULL HYPOTHESIS IS NOT
REJECTED

Notice that the final conclusion is always of the form "REJECT" or "DON’T
REJECT" the Null Hypothesis. The term "ACCEPT" is never used. The reason
is as follows: if the test results in the Null Hypothesis being rejected, then the
interpretation is that the data we have is in contradiction with the hypothesis, and
thus we can say that we have "empirical evidence" against the hypothesis. In the
same sense, if the output of the test is that the Null Hypothesis is NOT rejected,
then we interpret this as not having "enough empirical evidence" against the
hypothesis.
This is similar, for example, to the procedure that is followed in a trial in the
court. In principle, because of the presumption of innocence, we must main-
tain the hypothesis that the defendant is not guilty. If the police, however, finds
enough "empirical evidence" that incriminates him or her, then we REJECT the
hypothesis of innocence and the defendant is condemned as guilty. On the con-
trary, if the police does not find enough empirical evidence, the conclusion must
be that WE CAN NOT REJECT the hypothesis of innocence, which does not
imply ACCEPTING that the defendant is not guilty, only that we have not been
able to prove the contrary.

2.2 Test statistic and error types

As we have seen in the general structure of any hypothesis test, step 3 (specification
of the test statistic) is what makes a test different from another test. For instance, if
56 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

we want to test some hypothesis regarding the population mean µ it seems natural to
consider a test statistic based on the sample mean X̄, whereas if we need to test a
hypothesis referred to the population proportion π we will use a test statistic based on
the sample proportion π̂.
In each case we will need to recall the distribution of the corresponding statistic, as
seen in section 1.2, and its properties.

Sample statistic Distribution Expectation Variance

X̄−µ
q ∼ N (0, 1)
σ2
n
σ2
1 Pn
Mean X̄ = n i=1 xi σ 2 known E(X̄) = µ V (X̄) = n
X̄−µ
q ∼ tn−1
S2
n
σ2 unknown

1 Pn (n−1)S 2 2σ 4
Variance S2 = n−1 i=1 (xi − X̄)2 σ2
∼ χ2n−1 E(S 2 ) = σ 2 V (σ 2 ) = n−1

1 Pn π(1−π) π(1−π)
Proportion π̂ = n i=1 xi π̂ ∼ N (π, n
) E(π̂) = π V (π̂) = 2

Finally, before getting into the study of each specific test, we will see what type of
errors might occur when doing a hypothesis test. Indeed, given that tests are based
on probabilistic tools, their conclusions will be certain only with a given probability.
Therefore, it could be the case that the conclusions of a test are wrong. In this sense
the conclusion of a test might be erroneous for:

• R EJECTING a null hypothesis which is actually true

• N OT REJECTING a null hypothesis which is actually false

The first type of error is known as E RROR OF TYPE I, whereas the second is an E RROR
OF TYPE II.

The well known anti spam filters to eliminate junk e-mail work in this way. Each time a
new e-mail enters, the filter tests the hypothesis “this is a normal message” and reaches
a conclusion. Such conclusion is correct most of the times and the message is properly
tagged. Sometimes, though, the filter makes a mistake and:

• Tags as “junk” a message that actually is “normal” (Error of type I: erroneously

rejects the hypothesis that the the message is “normal”)

• Tags as “normal” a message that actually is “junk” (Error of type II: does not
reject, erroneously, the hypothesis that the message is “normal”)
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 57

H0 true H0 false

Reject H0 α 1−β

Not Reject H0 1−α β

Table 2.1: Types of errors

In table 2.1 we observe the different types of errors and the probability of each of them
Notice that the probability of the error of type I, α, is the level of significance of the
test. Indeed, as we have seen in the definition of the rejection area (RA), we know that
if the null hypothesis is true then the test statistic (TS) is within this RA (and thus the
null hypothesis will be rejected) with probability α.

p(EC ∈ ZR) = α

Therefore, α is the probability of rejecting the null hypothesis when it is actually true.
The probability of the error of type II, β, it depends on the true value of the parameter
to test.

2.3 Tests on the population mean, population variance

and population proportion
We will see next the three basic types of hypothesis testing:

3.3.1 Hypothesis test on the population mean µ

3.3.2 Hypothesis test on the population variance σ 2
3.3.3 Hypothesis test on the population proportion π

We will learn that all three cases share a common structure, the "six steps" we have
seen above. The difference is, mainly, the test statistic that will be used in each case.
Also, in each case the test can be of one or two tails, depending on the form of the
corresponding alternative hypothesis.

2.3.1 Hypothesis Test for the Population Mean (µ)

1. N ULL H YPOTHESIS
Is the value of the population mean we want to test (µ0 = value to test)

H0 : µ = µ0
58 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false. Depends on
what information we have regarding the population mean. There are 4 cases

I NFORMATION REGARDING µ T EST TYPE

H1 : µ ̸= µ0 General case. No information re- Two Tails test
garding µ. Hence, if it is not equal
to µ0 we can only say that it is dif-
ferent
H1 : µ > µ0 We have some information right-tail test
regardingµ. This information
states that if it is not equal to µ0 the
it is larger
H1 : µ < µ0 We have some information regard- left-tail test
ing µ. This information states that if
it is not equal to µ0 then it is smaller
H1 : µ = µ1 We have a lot of information regard- right-tail test if µ1 > µ0
ing µ. We know that if it is not equal or left-tail test if
to µ0 then it must be equal to the µ1 < µ0
value µ1

3. T EST S TATISTIC
The Test Statistic (TE) to use in this case depends on whether we know the
population variance σ 2 or not.

σ 2 KNOWN σ 2 UNKNOWN

TE= X̄−µ
q o
σ2
TE= X̄−µ
q o
S2
n n

In any of these cases, the observed value of the test statistic (OVTS) is obtained
by plugging the values into the corresponding formula, where

X̄ Sample Mean
µ0 Null Hypothesis value
σ2 Population Variance (if known)
S2 Sample Variance (if σ 2 is unknown)
n Sample size

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As we have learned in previous chapters, we have that if the null hypothesis is
true, that is, if µ = µ0 then

σ 2 KNOWN σ 2 UNKNOWN

X̄−µ X̄−µ
q o
σ2
∼ N (0, 1) q o
S2
∼ tn−1
n n
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 59

5. R EJECTION A REA of size α

The way to determine the Rejection Area will different depending on whether
the test is of one or two tails.

(a) T WO TAILS TEST. Corresponds to the case when the Alternative hypothesis
is like H1 : µ ̸= µ0

α/2 α/2

−Z1−α/2 0 Z1−α/2
−τ1−α/2 t1−α/2

Figure 2.1: Rejection Area in a two tails test

The limit values of the rejection area, Z1− α2 or t1− α2 can be found using
the tables of the N (0, 1) or t − student with n − 1 degrees of freedom
respectively depending on whether we know σ 2 or not. (See Figure 2.1)
(b) R IGHT- TAIL TEST. Corresponds to the case when we have an alternative
hypothesis of the type H1 : µ > µ0 (or the type H1 : µ = µ1 i µ1 > µ0 )

0 Z1−α
t1−α

Figure 2.2: Rejection Area in one right-tail test

60 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

The limit values of the rejection area, Z1−α i t1−α can be found in the
tables of the N (0, 1) or t − student with n − 1 degrees of freedom respec-
tively depending on whether we know σ 2 or not. (See Figure 2.2)
(c) L EFT- TAIL TEST. Corresponds to the case when we have an alternative
hypothesis of the type H1 : µ < µ0 (or H1 : µ = µ1 i µ1 < µ0 )

−Z1−α 0
−τ1−α

Figure 2.3: Rejection Area in one left-tail test

The limit values of the rejection area, Z1−α i t1−α can be found in the
tables of the N (0, 1) or t − student with n − 1 degrees of freedom respec-
tively, depending on whether we know σ 2 or not. (See Figure 2.3)
6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA. If it does, we then say that
the test rejects the N ULL H YPOTHESIS. If it does not belong to the rejection
area, then we say that the test D OES NOT REJECT THE N ULL H YPOTHESIS .

Let us see some examples

Example 2.3.1 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from
a Normal population with unknown mean and variance σ 2 = 1000000. Test the hy-
pothesis that the population mean µ equals 25000 if we know that the sample mean is
X̄ = 26000. (significance level α = 0.05)

1. N ULL H YPOTHESIS
Is the value of the population mean we want to test
(µ0 = 25000 in this case)
H0 : µ = 25000

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 61

Since we do not have any information about the possible value of µ if the null
hypothesis is false, we choose the more general form of the alternative hypothesis

H1 : µ ̸= 25000

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case, knowing σ 2 , is

X̄ − µo
TS = q
σ2
n

The observed value of the test statistic (OVTS) is obtained by plugging the values
into the corresponding formula

26000 − 25000 1000

OV T S = q = = 10
1000000 100
100

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As we have learned in previous chapters, we have that if the null hypothesis is
true, that is, if µ = 25.000 then

X̄ − µo
q ∼ N (0, 1)
σ2
n

5. R EJECTION A REA of size α = 0.05

The way to determine the Rejection Area will different depending on whether the
test is of one or two tails.
In this case we have a T WO TAILS TEST as the Alternative hypothesis is of the
form H1 : µ ̸= µ0

α/2 α/2

−Z1−α/2 0 Z1−α/2
−τ1−α/2 t1−α/2

The limit values of the rejection area, Z1− α2 can be found using the tables of the
N (0, 1)
62 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

α
Thus, if α = 0.05 then 2 = 0.025 and:
α
1− = 0.975
2
We have to look up the tables for the value
Z1− α2 = Z0.975

That is, the value of a N (0, 1) that has to its left a probability of 0.975. In the
tables:
Z0.975 = 1.96
Thus,

α/2 α/2

-1.96 0 1.96

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
Clearly, in this case it does. We have
OV T S = 10 > 1.96
We then say that the test rejects the N ULL H YPOTHESIS.

Example 2.3.2 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from a
Normal population with unknown mean and variance σ 2 = 230400. Test the hypoth-
esis that the population mean µ equals 1000 against the alternative of being less than
that if we know that the sample mean is X̄ = 950. (significance level α = 0.05)

1. N ULL H YPOTHESIS
Is the value of the population mean we want to test
(µ0 = 1000 in this case)
H0 : µ = 1000
2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
Since we have some information saying that if the null hypothesis is false then µ
will be smaller, we choose the alternative hypothesis
H1 : µ < 1000
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 63

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case, knowing σ 2 , is

X̄ − µo
TS = q
σ2
n

The observed value of the test statistic (OVTS) is obtained by plugging the values
into the corresponding formula
950 − 1000 −50
OV T S = q = = −1.042
230400 48
100

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As we have learned in previous chapters, we have that if the null hypothesis is
true, that is, if µ = 1.000 then

X̄ − µo
q ∼ N (0, 1)
σ2
n

5. R EJECTION A REA of size α = 0.05

The way to determine the Rejection Area will different depending on whether the
test is of one or two tails.
In this case we have a L EFT- TAIL TEST as the Alternative hypothesis is of the
form H1 : µ < µ0

−Z1−α 0
−τ1−α

The limit values of the rejection area, Z1−α can be found using the tables of the
N (0, 1)
In this case, if α = 0.05 then

1 − α = 0.95

We have to look up the tables for the value

Z1−α = Z0.95
64 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

That is, the value of a N (0, 1) that has to its left a probability of 0.95. In the
tables:
−Z0.95 = −1.645

Thus,

-1.645 0

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
Clearly, in this case it does not. We have

OV T S = −1.042 > −1.645

We then say that the test does not reject the N ULL H YPOTHESIS.

Example 2.3.3 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from
a Normal population with unknown mean and variance . Test the hypothesis that the
population mean µ equals 1000 against the alternative of being larger if we know that
the sample mean is X̄ = 1100 and the sample variance is S 2 = 200000. (significance
level α = 0.05)

1. N ULL H YPOTHESIS
Is the value of the population mean we want to test
(µ0 = 1000 in this case)
H0 : µ = 1000

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
Since we have some information saying that if the null hypothesis is false then µ
will be larger, we choose the alternative hypothesis

H1 : µ > 1000
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 65

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case, not knowing σ 2 , is

X̄ − µo
TS = q
S2
n

The observed value of the test statistic (OVTS) is obtained by plugging the values
into the corresponding formula
1100 − 1000 100
OV T S = q = = 2.236
200000 44.72
100

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As we have learned in previous chapters, we have that if the null hypothesis is
true, that is, if µ = 1.000 then

X̄ − µo
q ∼ tn−1 = t99
S2
n

5. R EJECTION A REA of size α = 0.05

The way to determine the Rejection Area will different depending on whether the
test is of one or two tails.
In this case we have a R IGHT- TAIL TEST as the Alternative hypothesis is of the
form H1 : µ > µ0

0 Z1−α
t1−α

The limit values of the rejection area, t1−α can be found using the tables of the
tn−1 = t99
In this case, if α = 0.05 then

1 − α = 0.95

We have to look up the tables for the value

t1−α = t0.95
66 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

That is, the value of a t − student that has to its left a probability of 0.95. In the
tables:
t0.95 = 1.66

Thus,

0 1.66

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
Clearly, in this case it does. We have

OV T S = 2.236 > 1.66

We then say that the test rejects the N ULL H YPOTHESIS.

RStudio Activity [Link] - Hypothesis Test for the Population Mean

We will conduct a hypothesis test for the population mean µ.
The script for this activity is:

1. Hypothesis test for the population mean in a Normal population (or large sam-
ple) and σ 2 known
2. Hypothesis test for the population mean in a Normal population (or large sam-
ple) and σ 2 unknown
In a similar ways as in the RStudio Activity corresponding to confidence intervals in
the previous chapter, what we will do in each case is:

(a) Generate a random sample from a Normal distribution.

(b) Compute the Observed Value of the Test Statistic (OVTS) step by step with RStu-
dio using the formulas we have seen in the previous sections.

(c) Find the critical value(s) that determine the Rejection Area (RA) using RStudio.

(d) Conclude the test by comparing the computed OVTS to the RA found above.
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 67

Note that, as seen when computing confidence intervals in the previous chapter, RStu-
dio contains commands that are specially useful to do hypothesis tests. We will intro-
duce these commands later on, once the important concept of p_value in introduced
in section 2.5.

1. Hypothesis test for the population mean in a Normal population (or large sam-
ple) and σ 2 known
(a) Generate a sample of size n = 2500, that will be called SAMPLE _1, from
a Normal distribution with population mean µ = 1000 and standard devi-
ation σ = 200. That is, a N (1000, 40000). As a matter of fact, this will
be exactly the same sample that we have used when computing confidence
intervals for the mean in the previous chapter1 .
[Link](1)
SAMPLE_1 <- rnorm(2500,1000,200)

The test to conduct using this sample is:

H0 : µ = 1000
H1 : µ ̸= 1000

(b) Remember the formula for the Test Statistic in this case:

X̄ − µo
TS = q
σ2
n

We will use RStudio to compute the Observed Value of the Test Statistic
(OVTS). In the code that follows we will use the following terms for each
element in the formula:

X̄ MEAN σ2 VARIANCE

µ0. NULL _ VALUE n SAMPLE _ SIZE

### Compute the elements in the formula

MEAN = mean(SAMPLE_1)
VARIANCE = 40000 ### assuming it is known
SAMPLE_SIZE = 2500
NULL_VALUE <- 1000 ### value of the null hypothesis

### Compute the OVTS

OVTS <- (MEAN - NULL_VALUE) /
sqrt(VARIANCE/SAMPLE_SIZE)

### Show the OVTS

paste("OVTS = ", OVTS)

## [1] "OVTS = -0.467242749405557"

We thus have: OVTS =-0.4672427

1 Remember that by using the same random seed, we generate the same random sample
68 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

(c) We will now find the critical value(s) that determine the Rejection Area
(RA). In this case, because of the type of alternative hypothesis, this is
a two-tails test and thus we will have two critical values, one to the right
and one to the left.

α/2 α/2

−Z1−α/2 0 Z1−α/2
−τ1−α/2 t1−α/2

By symmetry, though, we only need to find of the values: z1−α/2

With a signification level of 0.05 we have 1 − α2 = 0.975. Therefore, we
must find the value z0.975
## Note that when using 'qnorm', if no
### options are given for the mean and the variance
### the default is to use a standard N(0,1)
RA <- qnorm(0.975)

### Show the value for RA

paste("RA = ", RA)

## [1] "RA = 1.95996398454005"

In this case, since the OVTS is negative we have to consider the left tail of
the test. Thus, the relevant side of the Rejection Area is given by the value
-1.959964 to the left.
(d) Since -1.959964 = RA <OVTS = -0.4672427 it is clear that the OVTS is not
in the RA and thus the conclusion is:

The data does not reject the null hypothesis µ = 1000

2. Hypothesis test for the population mean in a Normal population (or large sam-
ple) and σ 2 unknown

(a) Generate a sample of size n = 2500, that will be called SAMPLE _1, from a
Normal distribution with population mean µ = 1000 and standard devia-
tion σ = 200. That is, a N (1000, 40000). This is exactly the same sample
that we have used before
[Link](1)
SAMPLE_1 <- rnorm(2500,1000,200)
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 69

The test, again, is:

H0 : µ = 1000
H1 : µ ̸= 1000

(b) Remember the formula for the Test Statistic in this case: (σ 2 unknown):

X̄ − µo
TS = q
S2
n

We will use RStudio to compute the Observed Value of the Test Statistic
(OVTS). In the code that follows we will use the following terms for each
element in the formula:

X̄ MEAN S2 VARIANCE

µ0. NULL _ VALUE n SAMPLE _ SIZE

The only difference with respect to the previous case is that we will use S 2
instead of σ 2 .

### Compute the elements in the formula

MEAN = mean(SAMPLE_1)
VARIANCE = var(SAMPLE_1) ### now we must compute it
SAMPLE_SIZE = 2500
NULL_VALUE <- 1000 ### value of the null hypothesis

### Compute the OVTS

OVTS <- (MEAN - NULL_VALUE) /
sqrt(VARIANCE/SAMPLE_SIZE)

### Show the OVTS

paste("OVTS = ", OVTS)

## [1] "OVTS = -0.450545389602778"

Thus, in this case: OVTS =-0.4505454

(c) We will now find the critical value(s) that determine the Rejection Area
(RA). In this case, because of the type of alternative hypothesis, this is
a two-tails test and thus we will have two critical values, one to the right
and one to the left. Moreover, since σ 2 is unknown and we are using S 2
the Test Statistics follows a t − student distribution with n − 1 = 2499
degrees of freedom.
70 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

α/2 α/2

−Z1−α/2 0 Z1−α/2
−τ1−α/2 t1−α/2

As before, by symmetry we only need to find one value: t0.975

## Note that in the 'qt' command
## we have to specify the degrees of freedom
RA <- qt(0.975,2499)

### Show the value for RA

paste("RA = ", RA)

## [1] "RA = 1.96091372487254"

In this case, since the OVTS is negative we have to consider the left tail of
the test. Thus, the relevant side of the Rejection Area is given by the value
-1.9609137 to the left.
(d) Since -1.9609137 = RA <OVTS = -0.4505454 it is clear that the OVTS is
not in the RA and thus the conclusion is:

The data does not reject the null hypothesis µ = 1000

End of Activity RStudio [Link]

2.3.2 Hypothesis Test for the Population Variance (σ 2 )

1. N ULL H YPOTHESIs
Is the value of the population variance that we want to test. (σ02 = value to test)

H0 : σ 2 = σ02

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false. Depends on
what information we have regarding the population mean. There are 4 cases
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 71

I NFORMATION REGARDING σ 2 T EST T YPE

2
H1 : σ ̸= σ02 General case. There is no informa- Two Tails Test
tion about σ 2 . Therefore, if it does
not equal σ02 the only think we can
say is that it will be different
H1 : σ 2 > σ02 We have some information about σ 2 Right-Tail Test
indicating that if it is not equal to σ02
then it must be greater
H1 : σ 2 < σ02 We have some information about σ 2 Left-Tail Test
indicating that if it is not equal to σ02
then it must be smaller
H1 : σ 2 = σ12 We have some information about σ 2 Right-Tail Test if σ12 > σ02
indicating that if it is not equal to and Left-Tail Test if σ12 <
σ02 then it must be equal to another σ02
value σ12

3. T EST S TATISTIC
The Test Statistic (TE) to use in this case is:
(n − 1)S 2
T.E. =
σ02
The Observed Value of the Test Statistic (OVTS) is obtained by substituting the
corresponding values in the formula, where

σ02 Null Hypothesis Value

S2 Sample Variance
n Sample Size

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

From what we know from the previous chapter, if the Null Hypothesis is true,
that is, if σ 2 = σ02 then
(n − 1)S 2
∼ χ2n−1
σ02
5. R EJECTION A REA of size α
The way to determine the Rejection Area depends on the test being of one or two
tails, that is, depending on what is the Alternative Hypothesis.
(a) T WO -TAILS T EST. Corresponds to the case when the Alternative Hypoth-
esis is of the type H1 : σ 2 ̸= σ02

The limit values in the Rejection Area, χ21− α and χ2α , can be found using
2 2
the tables of a χ2 with n − 1 degrees of freedom. (See Figure 2.4)
(b) R IGHT-TAIL T EST. Corresponds to the case when the Alternative Hypoth-
esis is of the type H1 : σ 2 > σ02 (also of the type H1 : σ 2 = σ12 i σ12 > σ02 )
72 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

α/2

α/2

χα/2 χ1−α/2

Figure 2.4: Rejection Area in a Two-Tails Test

The limit value in the Rejection Area, χ21−α , can be found in the tables of
a χ2 with n − 1 degrees of freedom. (See Figure 2.5)
(c) L EFT-TAIL T EST. Corresponds to the case when the Alternative Hypothe-
sis is of the type H1 : σ 2 < σ02 (also of the type H1 : σ 2 = σ12 i σ12 < σ02 )

The limit value in the Rejection Area, χ2α , can be found in the tables of a
χ2 with n − 1 degrees of freedom. (See Figure 2.6)
6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA. If it does, we then say that
the test rejects the N ULL H YPOTHESIS. If it does not belong to the rejection
area, then we say that the test D OES NOT REJECT THE N ULL H YPOTHESIS .

Example 2.3.4 Let {x1 , x2 , · · · , x100 } be a random sample of size 100 drawn from
a Normal population with unknown mean and variance. Test the hypothesis that the
population variance σ 2 equals 230400 if we know that the sample variance is S 2 =
200000. (significance level α = 0.05)

1. N ULL H YPOTHESIS
Is the value of the population variance we want to test (σ02 = 230400in this case)
H0 : σ 2 = 230400

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
Since we do not have any information about the possible value of σ 2 if the null
hypothesis is false, we choose the more general form of the alternative hypothesis
H1 : σ 2 ̸= 230400
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 73

χ1−α

Figure 2.5: Rejection Area in a Right-Tail Test

3. T EST S TATISTIC

The Test Statistic (TS) to use in this case is:

(n − 1)S 2
TS =
σ02

The Observed Value of the Test Statistic (OVTS) is obtained by substituting the
corresponding values in the formula:

99·200000
OV T S = = 85.94
230400

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

From what we know from the previous chapter, if the Null Hypothesis is true, that
is, if σ 2 = 230400 then
(n − 1)S 2
∼ χ2n−1
σ02

5. R EJECTION A REA of size α = 0.05

The way to determine the Rejection Area will different depending on whether the
test is of one or two tails.

In this case we have a Two-TAILS T EST. It corresponds to the case when the
Alternative Hypothesis is of the type H1 : σ 2 ̸= σ02
74 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

χα

Figure 2.6: Rejection Area in a Left-Tail Test

α/2

α/2

χα/2 χ1−α/2

The limit values in the Rejection Area, χ21− α and χ2α , can be found using the
2 2
tables of a χ2 with n − 1 = 99 degrees of freedom.
α α
Thus, if α = 0.05 then 2 = 0.025 and 1 − 2 = 0.975
We have to look up the tables for the values

χ21− α2 = χ20.975
χ2α2 = χ20.025

In the tables:

χ21− α2 = χ20.975 = 129.561

χ2α2 = χ20.025 = 74.22
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 75

Thus,

α/2

α/2

74.22 129.561

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
Clearly, in this case it does NOT. We have

74.22 < 85.94 < 129.561 ⇒ OV T S ̸∈ RA

We then say that the test does not reject the N ULL H YPOTHESIS.

RStudio Activity [Link] - Hypothesis Test for the Population Variance

We will now conduct a hypothesis test for the population variance σ 2 .
As in the case of the mean, we will:

(a) Generate a random sample from a Normal distribution.

(b) Compute the Observed Value of the Test Statistic (OVTS) step by step with RStu-
dio using the formulas we have seen in the previous sections.
(c) Find the critical value(s) that determine the Rejection Area (RA) using RStudio.
(d) Conclude the test by comparing the computed OVTS to the RA found above.

As said before, after studying the p_value in section 2.5 we will explore other com-
mands that RStudio has to do hypothesis tests.

(a) Generate a sample of size n = 2500, that will be called SAMPLE _4, from a Normal
distribution with population mean µ = 1000 and standard deviation σ = 200.
That is, a N (1000, 40000). As a matter of fact, this will be exactly the same
sample that we have used when computing confidence intervals for the mean in
the previous chapter.
76 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

[Link](3)
SAMPLE_4 <- rnorm(2500,1000,200)

The hypothesis to test in this example are:

H0 : σ 2 = 40000
H1 : σ 2 < 40000

(b) Recall the formula for the test statistic:

(n − 1)S 2
TS =
σ02

We will use RStudio to compute the Observed Value of the Test Statistic (OVTS).
In the code that follows we will use the following terms for each element in the
formula:

S2 VARIANCE n SAMPLE _ SIZE 2

σ0. NULL _ VALUE

### Compute the elements in the formula

VARIANCE = var(SAMPLE_4)
SAMPLE_SIZE = 2500
NULL_VALUE <- 40000 ### value of the null hypothesis

### Compute the OVTS

OVTS <- ((SAMPLE_SIZE - 1 )*VARIANCE) /
NULL_VALUE

### Show the OVTS

paste("OVTS = ", OVTS)

## [1] "OVTS = 2490.64316596707"

Hence: VOEC =2490.643166

(c) We will now find the critical value(s) that determine the Rejection Area (RA). In
this case, because of the type of alternative hypothesis, we have a left tail test.
Also, remember that in this case we must use χ2 distribution with n − 1 = 2499
degrees of freedom.
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 77

χα

If the signification level is 0.05 we will need to find χ0.05

## Note that in the 'qchisq' command

## we have to specify the degrees of freedom
RA <- qchisq(0.05,2499)
### Show the value for RA
paste("RA = ", RA)

## [1] "RA = 2383.86262966576"

(d) We have that 2383.8626297 = RA <OVTS = 2490.643166 and thus the OVTS is
not in the RA. Hence:

The data does not reject the null hypothesis σ 2 = 40000

End of Activity RStudio [Link]

2.3.3 Hypothesis Test for the Population Proportion (π)

1. N ULL H YPOTHESIS
Is the value of the Population Proportion that we want to test. (π0 = valor a contrastar)

H0 : π = π0

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false. Depends on
what information we have regarding the population proportion. There are 4 cases
78 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

I NFORMATION REGARDING π T EST T YPE

H1 : π ̸= π0 General case. There is no information re- Two Tails Test
garding π. Hence, if it is not equal to π0
the only thing we can say is that it is differ-
ent
H1 : π > π 0 We have some information regarding π in- Right-Tail Test
dicating that if it is not equal to π0 , then it
must be greater
H1 : π < π 0 We have some information regarding π in- Left-Tail Test
dicating that if it is not equal to π0 , then it
must be smaller
H 1 : π = π1 We have some information regarding π in- This is a Right-Tail Test if π1 > π0
dicating that if it is not equal to π0 , then it and a Left-Tail Test if π1 < π0
must be equal to another value π1

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case is:

π̂ − π0
TS = q
π0 (1−π0 )
n

The Observed Value of the Test Statistic (OVTS) is obtained when the elements
in the formula are replaced by their corresponding values from the sample, where

π̂ Sample Proportion
π0 Null Hypothesis Value
n Sample Size

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As we have seen in previous chapters, if it is true that π = π0 then2

π̂ − π0
q ∼ N (0, 1)
π0 (1−π0 )
n

5. R EJECTION A REA of sizeα

As in the other tests, the determination of the Rejection Area depends on the test
type

(a) T WO TAILS T EST. Corresponds to the case when the Alternative Hypoth-
esis is of the type H1 : π ̸= π0

The limit values of the Rejection Area, Z1− α2 and −Z1− α2 , can be found
in the table of the N (0, 1). (See Figure 2.7)
(b) R IGHT-TAIL T EST. Corresponds to the case when the Alternative Hypoth-
esis is of the type H1 : π > π0 (or H1 : π = π1 i π1 > π0 )
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 79

α/2 α/2

−Z1−α/2 0 Z1−α/2

Figure 2.7: Rejection Area in a Two Tails Test

0 Z1−α

Figure 2.8: Rejection Area in a Right-Tail Test

The limit value of the Rejection Area, Z1−α , can be found in the tables of
the N (0, 1). (See Figure 2.8)
(c) L EFT-TAIL T EST. Corresponds to the case when the Alternative Hypothe-
sis is of the type H1 : π < π0 (or H1 : π = π1 i π1 < π0 )

The limit value in the Rejection Area, Z1−α , can be found in the tables of
the N (0, 1). (See Figure 2.9)

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA. If it does, we then say that
2 Remember that this is an approximation, which is better the larger is the sample n. More specifically,

we need nπ0 (1 − p0 ) > 5

80 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

−Z1−α 0

Figure 2.9: Rejection Area in a Left-Tail Test

the test rejects the N ULL H YPOTHESIS. If it does not belong to the rejection
area, then we say that the test D OES NOT REJECT THE N ULL H YPOTHESIS .

Example 2.3.5 In the last political survey (Barómetro de Opinión) of the Centro de
Investigaciones Sociológicas, 40% of 2477 people interviewed declared that they would
vote to the PP party in a new election. Test the hypothesis that the PP could win more
than 35% of the votes. (significance level α = 0.05)

1. N ULL H YPOTHESIS
Is the value of the population proportion we want to test (π0 = 0.35 in this case)

H0 : π = 0.35

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
Since in this case we want to test if the PP could win more than 35% of the vote
we use the alternative hypothesis

H1 : π > 0.35

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case is.

π̂ − π0
TS = q
π0 (1−π0 )
n

The Observed Value of the Test Statistic (OVTS) is obtained when the elements
in the formula are replaced by their corresponding values from the sample

0.4 − 0.35
OVTS = q = 5.22
0.35(1−0.35)
2.477
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 81

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As we have seen in previous chapters, if it is true that π = π0 then
π̂ − π0
q ∼ N (0, 1)
π0 (1−π0 )
n

Let us check the condition nπ0 (1 − π0 ) > 5

nπ0 (1 − π0 ) = 2477(0.35)(1 − 0.35) = 563.52

5. R EJECTION A REA of size α = 0.05

The way to determine the Rejection Area will different depending on whether the
test is of one or two tails.
In this case we have a R IGHT- TAIL TEST as the Alternative hypothesis is of the
form H1 : π > π0

0 Z1−α

The limit values of the rejection area, Z1−α can be found using the tables of the
N (0, 1)
In this case, if α = 0.05 then
1 − α = 0.95

We have to look up the tables for the value

Z1−α = Z0.95 = 1.645

Thus,

0 1.645
82 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
Clearly, in this case it does. We have

OV T S = 5.22 > 1.645 ⇒ OV T S ∈ RA

We then say that the test rejects the N ULL H YPOTHESIS.

Notice that in this case we are rejecting the hypothesis H0 : π = 0.35 against
the alternative H1 : π > 0.35.
Thus, the conclusion is that the survey does not reject that the PP could win more
than 35% of the vote

RStudio Activity [Link] - Hypothesis Test for the Population Proportion

To end this section we will see an example of a hypothesis test for the population
proportion π.
As seen when computing confidence intervals for the proportion, in this case the pop-
ulation is not distributed as a Normal distribution but as a Bernoulli: any element in
the population takes the value 1 if it has the characteristic that we are studying, and 0
otherwise, being π the proportion of elements in the population that have such charac-
teristic.
Therefore, once again, we will use a Bernoulli to generate the sample, with xi = 1 if
the i−th element in the sample has the characteristic of interest and xi = 0 if it does
not.
The same as in the previous cases, we will:

(a) Generate a random sample (from a Bernoulli distribution in this case)

(b) Compute the Observed Value of the Test Statistic (OVTS) step by step with RStu-
dio using the formulas we have seen in the previous sections.
(c) Find the critical value(s) that determine the Rejection Area (RA) using RStudio.
(d) Conclude the test by comparing the computed OVTS to the RA found above.

Let’s do it.

(a) Generate a sample of size n = 2500, that will be called SAMPLE _6, from a
Bernoulli distribution with parameter p = 31 . That is, a b(1/3).
[Link](6)
SAMPLE_6 <- rbinom(2500,1,1/3)

The test in this case will be:

1
H0 : π =
4
1
H1 : π >
4
2.3. TESTS ON THE MEAN, VARIANCE AND PROPORTION 83

(b) The formula of the Test Statistic in this case is:

π̂ − π0
TS = q
π0 (1−π0 )
n

We will use RStudio to compute the Observed Value of the Test Statistic (OVTS).
In the code that follows we will use the following terms for each element in the
formula:

π̂ PROPORTION n SAMPLE _ SIZE π 0. NULL _ VALUE

### Compute the elements in the formula

PROPORTION <- mean(SAMPLE_6)
SAMPLE_SIZE = 2500
NULL_VALUE <- 1/4

### Compute the OVTS

OVTS <- (PROPORTION - NULL_VALUE ) /
sqrt(NULL_VALUE*(1-NULL_VALUE) / SAMPLE_SIZE)

### Show the OVTS

paste("OVTS = ", OVTS)

## [1] "OVTS = 10.0228006731319"

We thus have: OVTS = 10.0228007

(c) We will now find the Rejection Area (RA). Because of the type of alternative hy-
pothesis this will be a right-tail test

0 Z1−α

Hence, for a significance level of 0.05 we need to find z1−0.05 = z0.95

RA <- qnorm(0.95)

### Show the value for RA

paste("RA = ", RA)

## [1] "RA = 1.64485362695147"

84 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

(d) Given that: 1.6448536 = RA < OVTS = 10.0228007 we conclude that the OVTS
is inside the RA and hence:
1
The data does reject the null hypothesis π = 4

1
Hence, it seems that π > 4

End of Activity RStudio [Link]

2.4 Tests for comparing samples

In many cases what is of interest is not a question regarding some population parameter
(as in the previous section), but rather to compare one parameter in one population with
the corresponding parameter in another population. For instance, we might want to test
whether the average income in Cerdanyola this year is equal or greater that in the
previous year, or if the average income in Cerdanyola is equal to that in Sant Cugat.
That is, now we are interested in COMPARING POPULATION PARAMETERS BETWEEN
TWO POPULATIONS , either two different populations (as when comparing the average
income in Cerdanyola and Sant Cugat) or the same population at two different dates or
after some action (as when comparing the average income in Cerdanyola this year with
that of the previous year).
In any of these cases, what we do is a T WO S AMPLES T EST. Now we have two
populations (Population 1 and Population 2) each one with its corresponding popula-
tion parameters (µ1 , σ12 and π1 for the first population and µ2 , σ22 i π2 for the second
population). We then draw two independent samples from each of these populations
(Sample 1 and Sample 2) which might have different sizes (n1 and n2 ). From these
samples we compute the corresponding Sample Statistics that will be used to perform
the tests (X̄1 , S12 and π̂1 for the first sample and X̄2 , S22 i π̂2 for the second sample)
Summarized, the information we can gather is:

Population 1 Population 2
µ1 , σ12 and π1 µ2 , σ22 and π2
Sample 1 Sample 2
x11 x12
x21 x22
.. ..
. .
xn1 1 x n2 2
X̄1 , S12 and π̂1 X̄2 , S22 and π̂2

From here, we can do tests regarding:

1. The difference between the means of the two populations: µ1 − µ2

2. The comparison between the variances of the two populations σ12 − σ22
3. The difference between the proportions of the two populations π1 − π2
2.4. TESTS FOR COMPARING SAMPLES 85

2.4.1 Test for the Difference of Means

We want to test if the difference between the means of two populations equals some
specific value δ0 or not (δ0 = 0 if we want to test if the means are equal to each other).
For instance, we could test if the average income in Cerdanyola and Sant Cugat are
equal to each other (µ1 − µ2 = 0). Another example would be to test whether the
average sleeping time after taking a new pill equals (or, alternatively, is larger) than
without taking any pill.
For this test the six corresponding steps are:

1. N ULL H YPOTHESIS
Is the value for the difference that we want to test (δ0 = difference value to test)

H0 : µ1 − µ2 = δ0

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false. Depends on
what information we have regarding the population proportion. There are 4 cases

I NFORMATION REGARDING µ1 AND µ2 T EST T YPE

H1 : µ1 − µ2 ̸= δ0 General case. We have no information for Two Tails Test
the population means. Hence, if the differ-
ence is not equal to δ0 the only thing we can
say is that it is different
H1 : µ1 − µ2 > δ0 We have some information about the means Right-Tail Test
indicating that if the difference is not equal
to δ0 then it must be greater
H1 : µ1 − µ2 < δ0 We have some information about the means Left-Tail Test
indicating that if the difference is not equal
to δ0 then it must be smaller
H1 : µ1 − µ2 = δ1 We have some information about the means Right-Tail Test if δ1 > δ0 and
indicating that if the difference is not equal Left-Tail Test if δ1 < δ0
to δ0 then it must be equal to a known alter-
native value δ1

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case depends on whether the population
variances σ12 i σ22 are both known or not.

σ12 and σ22 KNOWN σ12 or σ22 UNKNOWN

TS= (X̄r1 −2X̄2 )−δ

2
o
TS= (X̄q1 −S2X̄2 )−δ
S2
o
σ1 σ +n
n1 + n2 n1 2
2

In any of these cases, the Observed Value of the Test Statistic (OVTS) is obtained
by replacing the corresponding values in the formula, where
86 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

X̄1 i X̄2 Sample Means

δ0 Null Hypothesis Value
σ12 i σ22 Population variances (if known)
S2 Common Sample Variance (if σ12 or σ22 are not known)
n1 i n2 Sample sizes

In the formulae above the common value for the Sample Variance, S 2 ( that we
use if we do NOT know either σ12 or
´ σ22 ) (or any of them) is computed as

(n1 − 1)S12 + (n2 − 1)S22

S2 =
n1 + n2 − 2

where S12 i S22 are the Sample Variances of the first and the second sample re-
spectively. The reason for using this common estimation of the sample variance
is that for the test to make sense the two populations must be somehow "homo-
geneous". Technically, this is equivalent to requiring that the two populations
have a similar population variance.

4. D ISTRIBUTION OF THE T EST S TATISTIC WHEN THE N ULL H YPOTHESIS IS

TRUE

If it is true that µ1 − µ2 = δ0 then

σ12 and σ22 KNOWN σ12 or σ22 UNKNOWN

(X̄1 −X̄2 )−δo (X̄1 −X̄2 )−δo

r
2
∼ N (0, 1) q
S2 S2
∼ tn1 +n2 −2
σ1 σ2
+ n2 n +n
1 2
n1 2

5. R EJECTION A REA of size α

The Rejection Area depends on whether we have a Two Tails Test, a Right-Tail
Test, or a Left-Tail Test. This, in turn, depends on what is the specification of the
Alternative Hypothesis.

(a) T WO TAILS T EST. Corresponds to the case when the Alternative Hypoth-
esis is of the type H1 : µ1 − µ2 ̸= δ0

The limit values of the Rejection Area, Z1− α2 or t1− α2 can be found in the
tables of a N (0, 1) or a t − student with n1 + n2 − 2 degrees of freedom
respectively, depending on whether we know the two population variances
or not as explained above (See Figure 2.10)
(b) R IGHT-TAIL T EST. Corresponds to the case when the Alternative Hypoth-
esis is of the type H1 : µ1 − µ2 > δ0 (or H1 : µ1 − µ2 = δ1 and δ1 > δ0 )

The limit value of the Rejection Area, Z1−α or t1−α can be found in the
tables of the N (0, 1) or the t−student with n1 +n2 −2 degrees of freedom
respectively depending on whether we know the two population variances
or not as explained before. (See Figure 2.11)
2.4. TESTS FOR COMPARING SAMPLES 87

α/2 α/2

−Z1−α/2 0 Z1−α/2
−τ1−α/2 t1−α/2

Figure 2.10: Rejection Area in a Two Tails Test

0 Z1−α
t1−α

Figure 2.11: Rejection Area in a Right-Tail Test

(c) L EFT-TAIL T EST. Corresponds to the case when the Alternative Hypothe-
sis is of the type H1 : µ1 − µ2 < δ0 (or H1 : µ1 − µ2 = δ1 and δ1 < δ0 )

The limit value of the Rejection Area, Z1−α or t1−α can be found in the
tables of the N (0, 1) or t − student with n1 − n2 − 2 degrees of freedom
respectively depending on whether we know the two population variances
or not. (See Figure 2.12)
6. T EST CONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA. If it does, we then say that
the test rejects the N ULL H YPOTHESIS. If it does not belong to the rejection
area, then we say that the test D OES NOT REJECT THE N ULL H YPOTHESIS .

Example 2.4.1 A consumer is thinking about what brand of energy-saving light bulbs
to buy. Manufacturers A and B both claim that the average life spam of the bulbs they
88 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

−Z1−α 0
−τ1−α

Figure 2.12: Rejection Area in a Left-Tail Test

produce is of 10000 hours. The consumer, however, reads a report on this type of light
bulbs in which, from a test with 50 bulbs of brand A and 45 bulbs of brand B, the results
show a mean life spam of 9430 hours for A and 9660 for brand B. Because of the way
these bulbs are manufactured, the variances are the same σ12 = σ22 = 10000. Can we
say that the two brands have the same life spam ?
In this case we have :

Population 1 (Brand A) Population 2 (Brand B)

µ1 = unknown, σ12 = 10000 µ2 = unknown, σ22 = 10000
Sample 1 (n1 = 50) Sample 2 (n2 = 45)
X̄1 = 9430 X̄2 = 9660

1. N ULL H YPOTHESIS
Is the value for the difference that we want to test (δ0 = 0 in this case)

H0 : µ1 − µ2 = 0

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
Since, in principle, both brands are apparently equal, we choose

H1 : µ1 − µ2 ̸= 0

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case that we know the population variances
σ12 and σ22 , is
2.4. TESTS FOR COMPARING SAMPLES 89

(X̄1 −X̄2 )−δo

TS = r
2
σ1 σ2
n1 + n2
2

The Observed Value of the Test Statistic (OVTS) is obtained by replacing the
corresponding values in the formula,
(9430 − 9660) − 0 −230
OV T S = q = = −11.19
10000 10000 20.55
50 + 45

4. D ISTRIBUTION OF THE T EST S TATISTIC WHEN THE N ULL H YPOTHESIS IS

TRUE
If it is true that µ1 − µ2 = 0 then

(X̄1 − X̄2 ) − δo
q 2 ∼ N (0, 1)
σ1 σ22
n1 + n2

5. R EJECTION A REA of size α

The Rejection Area depends on whether we have a Two Tails Test or a One-Tail
Test.
In this case we have a T WO -TAILS T EST since the alternative hypothesis is of
the form H1 : µ1 − µ2 ̸= 0

α/2 α/2

−Z1−α/2 0 Z1−α/2

The limit values of the rejection area, Z1− α2 and −Z1− α2 are found in the tables
of N (0, 1)
α
Thus, if α = 0.05 then 2 = 0.025 and:
α
1− = 0.975
2
Looking up the tables we find
Z1− α2 = Z0.975

Z0.975 = 1.96
Therefore,
90 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

α/2 α/2

-1.96 0 1.96

6. T EST CONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
In this case we find

OV T S = −11.19 < −1.96

We say that the test rejects the null hypothesis

Example 2.4.2 A manufacturer of conditioned glass for windows uses two different
machines to produce his glasses. The manufacturer suspected that one of the machines
(machine A) produces the glasses with a factor of insulation lower than the other (ma-
chine B). To test this hypothesis, the manufacturer checks the isolation levels of 50
glasses produced in each machine. The results show that the glasses produced in ma-
chine A have an average insulation of 0.81, whereas the average insulation of those
produced by machine B is of 0.85. Similarly, the corresponding sample variances are
S12 = 0.01 in the first case and S22 = 0.015 with respect to machine B. Is the manufac-
turer right or, on the contrary, the two machines produce the same results ?
In this case we have :

Population 1 (Machine A) Population 2 (Machine B)

µ1 = unknown, σ12 = unknown µ2 = unknown, σ22 = unknown
Sample 1 (n1 = 50) Sample 2 (n2 = 50)
X̄1 = 0.81, S12 = 0.01 X̄2 = 0.85, S22 = 0.015

1. N ULL H YPOTHESIS
Is the value for the difference that we want to test (δ0 = 0 in this case)

H0 : µ1 − µ2 = 0
2.4. TESTS FOR COMPARING SAMPLES 91

2. A LTERNATIVE H YPOTHESIS
Corresponds to what would be true if the null hypothesis is false.
Since the manufacturer suspects that µ1 < µ2 , we use an alternative hypothesis
of the form

H1 : µ1 − µ2 < 0

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case that we DO NOT know the population
variances is

(X̄1 −X̄2 )−δo

TS = q
S2 S2
n +n
1 2

where S 2 is the C OMMON S AMPLE VARIANCE S 2 computed as

(n1 − 1)S12 + (n2 − 1)S22

S2 =
n1 + n2 − 2

where S12 and S22 are the sample variances obtained in Sample 1 and Sample 2
respectively.
The Observed Value of the Test Statistic (OVTS) is obtained by replacing the
corresponding values in the formula.
We first compute the C OMMON S AMPLE VARIANCE S 2

(n1 − 1)S12 + (n2 − 1)S22 49 · 0.01 + 49 · 0.015

S2 = = = 0.0125
n1 + n2 − 2 50 + 50 − 2

Therefore,

(0.81 − 0.85) − 0 −0.04

OV T S = q = = −1.789
0.0125
+ 0.0125 0.022
50 50

4. D ISTRIBUTION OF THE T EST S TATISTIC WHEN THE N ULL H YPOTHESIS IS

TRUE

If it is true that µ1 − µ2 = 0 then

(X̄1 − X̄2 ) − δo
q ∼ tn1 +n2 −2
S2 S2
n1 + n2

5. R EJECTION A REA of size α

The Rejection Area depends on whether we have a Two Tails Test or a One-Tail
Test.
In this case we have a L EFT-TAIL T EST since the alternative hypothesis is of the
form H1 : µ1 − µ2 <0
92 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

0
−t1−α

The value that limits the rejection area, −t1−α , is found in the tables of tn1 +n2 −2 =
t98
Thus, if α = 0.05 then 1 − α = 0.95
We look up the tables for the value

t1−α = t0.95 = 1.66

Therefore,

0
−1.66

6. T EST CONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
In this case we find

OV T S = −1.789 < −1.66

We say that the test rejects the null hypothesis

That is, we are rejecting that the two means are equal. Hence, the manufacturer
seems to be right
2.4. TESTS FOR COMPARING SAMPLES 93

2.4.2 Test for the Comparison of Variances

In this case we only test if the two populations have the same variance or not. This is
special test for three reasons:

1. We can only test if the two variances are equal or not, that is, the Null Hypothesis
is always the same
H0 : σ12 = σ22
2. The test must be conducted following a strict order, that must be established in a
"extra" step before starting the usual 6 steps
3. This test is important because it allows to check if two different populations seem
to have the same variance. This, as we have seen before, is important for other
tests. Indeed, the test for the difference of means only makes sense if the two
populations are "homogeneous", that is, have a similar variance.

Hence, this "special" test will begin with an extra step (Step 0) where we establish the
order of the elements of the test.

0. E XTRA S TEP
We change the "denomination" of our two samples so that ALWAYS the sample
with the highest Sample Variance is the Sample 1, being the Sample 2 the one
with the lowest variance. This way, once we have followed this rule, we will
always have:
S12 > S22
1. N ULL H YPOTHESIS
Is always the same and, as said before, it consists of testing whether the two
variances are the same or not. Because of the special structure of this test, the
correct way to specify this hypothesis is:
σ12
H0 : =1
σ22
2. A LTERNATIVE H YPOTHESIS
As usual, it represents what is true when the Null Hypothesis is false. I this
specific case, there are only two possible specifications for this hypothesis (once
more, this is so because of the special structure of this test)

I NFORMATION REGARDING T EST T YPE

σ12 AND σ22
σ12
H1 : σ22
̸= 1 General Case. We have no informa- Two Tails Test
tion on σ12 nor about σ22 . Hence, if
they are not equal, the only thing we
can say is that they are different
σ12
H1 : σ22
>1 We have some information about σ12 Right-Tail Test
and σ22 indicating that if they are not
equal then one of them is greater.
Because of the “denomination“ in
Step 0, the greater will always be σ12
94 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

3. T EST S TATISTIC
In this case, the Test Statistic (TC) to use is:

S12
TS =
S22

The observed value of the Test Statistic (OVTS) is easily obtained replacing the
corresponding sample variances in the formula, where

S12 Sample Variance of Sample 1

S22 Sample Variance of Sample 2

Notice that, because of Step 0 we have that S12 > S22 , and hence we will always
find that OVTS> 1

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

In this case, the Test Statistic follows a distribution that is known as a F of
Snedecor. This distribution, the same as the t − student or the χ2 is also char-
acterized by its "degrees of freedom". Unlike those cases, though, the F −
snedecor has a "pair" of degrees of freedom, those corresponding to the nu-
merator and those corresponding to the denominator. Hence, the notation:

S12
∼ F(n1 −1,n2 −1)
S22
S2
indicates that the Test Statistic S12 follows a F − snedecor distribution with
2
n1 − 1 degrees of freedom in the numerator (that is, the size of the sample that
corresponds to S12 in the numerator minus 1) and n2 − 1 degrees of freedom
in the denominator (that is, the size of the sample that corresponds to S22 in the
denominator minus 1).
Remember that it is very important to keep the order established in Step 0, that
is, sample 1 corresponds to to the sample that has the highest sample variance. In
this sense, the "degrees of freedom in the numerator" is the size of such sample
minus 1: n1 −1. This is important when looking at the tables of the F −snedecor
in order to determine the Rejection Area.

5. R EJECTION A REA of size α

The Rejection Area depends on whether the test has one or two tails, as given by
the Alternative Hypothesis. In this special test, the tail that "matters" will always
be the Right-Tail, even if the test is a "Two Tails Test".

(a) T WO TAILS T EST. Corresponds to the case when we have an Alternative

σ2
Hypothesis of the type H1 : σ12 ̸= 1
2

For the limit values of the Rejection Area, F1− α2 and F α2 , we only need
to find F1− α2 in the tables of the F with n1 − 1 degrees of freedom in the
numerator and n2 − 1 degrees of freedom in the denominator. The other
value, F α2 , is not needed in any case since the OVTS is always > 1, Hence,
2.4. TESTS FOR COMPARING SAMPLES 95

α/2

α/2

(no cal buscar-lo) F1−α/2

Figure 2.13: Rejection Area in a Two Tails Test

if it falls into the Rejection Area, it will be on the Right-Tail. B ECAUSE OF

WHAT IS DONE IN S TEP 0 ( THE “ DENOMINATION ” OF THE SAMPLES ),
THE O BSERVED VALUE OF THE T EST S TATISTIC WILL NEVER BE IN
THE L EFT-TAIL. (See Figure 2.13)
(b) R IGHT-TAIL T EST. Corresponds to the case when the Alternative Hypoth-
σ2
esis is of the type H1 : σ12 > 1
2

F1−α

Figure 2.14: Rejection Area in a Right-Tail Test

The limit value of the Rejection Area, F1−α , can be found in the tables of
96 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

a F with n1 − 1 degrees of freedom in the numerator and n2 − 1 degrees

of freedom in the denominator. (See Figure 2.14)
6. T EST CONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA. If it does, we then say that
the test rejects the N ULL H YPOTHESIS. If it does not belong to the rejection
area, then we say that the test D OES NOT REJECT THE N ULL H YPOTHESIS .

Example 2.4.3 For the analysis of stock prices, the variance of the prices of different
shares is usually taken as a measure of the risk associated to each share. In this regard,
the observation of 50 prices of shares of company A resulted in a variance of 3.9,
whereas 100 shares of company B produced a variance of 4.1 in its prices. Can we
conclude that the shares of the two companies are equally risky ?
In this case we have :

Population 1 (Company A) Population 2 (Company B)

µ1 = unknown, σ12 = unknown µ2 = unknown, σ22 = unknown
Sample 1 (n1 = 50) Sample 2 (n2 = 100)
S12 = 3.9 S22 = 4.1

0. E XTRA S TEP
We change the "denomination" of our two samples so that ALWAYS the sample
with the highest Sample Variance is the Sample 1, being the Sample 2 the one
with the lowest variance.
Thus, we have to “flip” our samples so that:

Population 1 (Company B) Population 2 (Company A)

µ1 = unknown, σ12 = unknown µ2 = unknown, σ22 = unknown
Sample 1 (n1 = 100) Sample 2 (n2 = 50)
S12 = 4.1 S22 = 3.9

1. N ULL H YPOTHESIS
Is always the same and, as said before, it consists of testing whether the two
variances are the same or not.

σ12
H0 : =1
σ22
2. A LTERNATIVE H YPOTHESIS
As usual, it represents what is true when the Null Hypothesis is false. In this
case, since we do not have any information we use
σ12
H1 : ̸= 1
σ22
2.4. TESTS FOR COMPARING SAMPLES 97

3. T EST S TATISTIC

In this case, the Test Statistic (TS) to use is:

S12
TS =
S22

The observed value of the Test Statistic (OVTS) is easily obtained replacing the
corresponding sample variances in the formula

S12 4.1
OV T S = 2 = = 1.0513
S2 3.9

Notice that, because of Step 0 we have that S12 > S22 , and hence we will always
find that OVTS> 1

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

In this case, the Test Statistic follows a distribution that is known as a F of

Snedecor.
S12
∼ F(n1 −1,n2 −1)
S22

with n1 − 1 degrees of freedom in the numerator and n2 − 1 degrees of freedom

in the denominator. In this case,

S12
∼ F(n1 −1,n2 −1) = F(99,49)
S22

[R EMEMBER THAT IT IS VERY IMPORTANT TO KEEP THE ORDER ESTAB -

LISHED IN S TEP 0]

5. R EJECTION A REA of size α

The Rejection Area depends on whether the test has one or two tails, as given by
the Alternative Hypothesis.

I N THIS SPECIAL TEST, HOWEVER , THE TAIL THAT " MATTERS " WILL ALWAYS
BE THE R IGHT-TAIL , EVEN IF THE TEST IS A "T WO TAILS T EST ".

In this case we have a T WO TAILS T EST since we have an Alternative Hypothe-

σ2
sis of the type H1 : σ12 ̸= 1
2
98 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

α/2

α/2

(no cal buscar-lo) F1−α/2

For the limit values of the Rejection Area, F1− α2 and F α2 , we only need to find
F1− α2 in the tables of the F with n1 − 1 degrees of freedom in the numerator and
n2 − 1 degrees of freedom in the denominator.
α
Thus, if α = 0.05 then 2 = 0.025 and:
α
1− = 0.975
2

We look up the tables for the value

F1− α2 = F0.975

That is, the value of a F(99,49) that has to its left a probability of 0.975. In the
tables:
F0.975 = 1.66

α/2

α/2

(no cal buscar-lo) 1.66

6. T EST CONCLUSION
2.4. TESTS FOR COMPARING SAMPLES 99

Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC

(OVTS) falls, or not, inside the R EJECTION A REA.
Here we find
OV T S = 1.0513 < 1.66 ⇒ OV T S ̸∈ RA

Thus, we can NOT reject the hypothesis that the two variances are equal

2.4.3 Test for the Difference of Proportions

We now test what is the difference between the proportion of elements that have a
given characteristic in two populations. For instance, we can test if the proportion of
voters of the PP in Cerdanyola equals the proportion of voters of the PP in Sant Cugat
(π1 − π2 = 0). Another example would be to test if the proportion of people that
recovers from a given illness is bigger if they take a specific medicine than if they don’t
(in order to test the goodness of such medicine)
Te six steps for this test are as follows:

1. N ULL H YPOTHESIS
It is the value for the difference between the population proportions that we want
to test. (δ0 = difference to test)

H0 : π1 − π2 = δ0

2. A LTERNATIVE H YPOTHESIS
Represents what is true when the Null Hypothesis is false. As usual, its specifi-
cation depends on the information we have about the populations

I NFORMATION REGARDING T EST T YPE

π1 I π2
H1 : π1 − π2 ̸= δ0 General Case. We have no informa- Two Tails Test
tion about the population propor-
tions. Hence, we can only say that
if the difference is not δ0 then it is
different
H1 : π1 − π2 > δ0 We have some information indicat- Right-Tail Test
ing that if the difference is not δ0
then it must be bigger
H1 : π1 − π2 < δ0 We have some information indicat- Left-Tail Test
ing that if the difference is not δ0
then it must be smaller
H1 : π 1 − π 2 = δ 1 We have very specific informa- Right-Tail Test if
tion about the proportions indicat- δ1 > δ0 and Left-Tail
ing that if the difference is not equal Test if δ1 < δ0
to δ0 then it must be equal to a spe-
cific alternative value δ1
100 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

3. T EST S TATISTIC
The Test Statistic (TE) to use in this case is always the same.

(π̂1 − π̂2 ) − δ0
TS = q
π̂(1−π̂)
n1 + π̂(1−π̂)
n2

The Observed Value of the Test Statistic (OVTS) is obtained by replacing in the
formula the corresponding values, where

π̂1 and π̂2 Sample Proportions

δ0 Null Hypothesis Value
π̂ Common Sample Proportion
n1 i n2 Sample Sizes

The value of the Common Sample Proportion, π̂, is obtained from

n1 π̂1 + n2 π̂2
π̂ =
n1 + n2

which is equivalent to computing the proportion of elements in the two samples

(jointly) that have the characteristic that is of interest.
4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true
As seen in other cases, when is true that π1 − π2 = δ0 then
(π̂ − π̂2 ) − δ0
q 1 ∼ N (0, 1)
π̂(1−π̂) π̂(1−π̂)
n1 + n2

5. R EJECTION A REA of size α

The Rejection Area will be different depending on the type of test.
(a) T WO TAILS T EST. Corresponds to the case when we have an Alternative
Hypothesis of the type H1 : π1 − π2 ̸= δ0

The limit values of the Rejection Area, Z1− α2 and −Z1− α2 , can be found
in the table of a N (0, 1). (See Figure 2.15)
(b) R IGHT-TAIL T EST. Corresponds to the case when we have an Alternative
Hypothesis of the type H1 : π1 − π2 > δ0 (or H1 : π1 − π2 = δ1 and δ1 >
δ0 )

The limit value of the Rejection Area, Z1−α , can be found in the table of a
N (0, 1). (See Figure 2.16)
(c) L EFT-TAIL T EST. Corresponds to the case when the Alternative Hypothe-
sis is of the type H1 : π1 − π2 < δ0 (or H1 : π1 − π2 = δ1 and δ1 < δ0 )
2.4. TESTS FOR COMPARING SAMPLES 101

α/2 α/2

−Z1−α/2 0 Z1−α/2

Figure 2.15: Rejection Area in a Two Tails Test

0 Z1−α

Figure 2.16: Rejection Area in a Right-Tail Test

The limit value of the Rejection Area, Z1−α , can be found in the table of a
N (0, 1). (See Figure 2.17)

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA. If it does, we then say that
the test rejects the N ULL H YPOTHESIS. If it does not belong to the rejection
area, then we say that the test D OES NOT REJECT THE N ULL H YPOTHESIS .

Example 2.4.4 According to the WHO (World Health Organization) the proportion
of smokers in France among the population over 15 years of age is 27.6%, whereas
that rate is 29.1% in Spain. According to his information, can we conclude that the
proportion of smokers older than 15 in France is, at least, 1% lower than in Spain?
The sample sizes were 7000 people in France and 5000 in Spain.

1. N ULL H YPOTHESIS
102 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

−Z1−α 0

Figure 2.17: Rejection Area in a Left-Tail Test

It is the value for the difference between the population proportions that we want
to test. (δ0 = −0.01 in this case)

H0 : π1 − π2 = −0.01

2. A LTERNATIVE H YPOTHESIS
Represents what is true when the Null Hypothesis is false. As usual, its specifi-
cation depends on the information we have about the populations
In this case, since we want to test if π1 < π2 − 0.01, we use an alternative
hypothesis of the form

H1 : π1 − π2 < −0.01

3. T EST S TATISTIC
The Test Statistic (TS) to use in this case is always the same.

(π̂1 − π̂2 ) − δ0
TS = q
π̂(1−π̂)
n1 + π̂(1−π̂)
n2

The Observed Value of the Test Statistic (OVTS) is obtained by replacing in the
formula the corresponding values, where the value of the Common Sample Pro-
portion, π̂, is obtained from

n1 π̂1 + n2 π̂2 7000 · (0.276) + 5000 · (0.291)

π̂ = = = 0.28225
n1 + n2 7000 + 5000

Thus, we have,

(0.276 − 0.291) − (−0.01)

V OEC = q = −0.599943
0.28225(1−0.28225) 0.28225(1−0.28225)
7000 + 5000
2.4. TESTS FOR COMPARING SAMPLES 103

4. D ISTRIBUTION OF THE T EST S TATISTIC when the Null Hypothesis is true

As seen in other cases, when is true that π1 − π2 = δ0 then

(π̂ − π̂2 ) − δ0
q 1 ∼ N (0, 1)
π̂(1−π̂) π̂(1−π̂)
n1 + n2

5. R EJECTION A REA of size α

The Rejection Area will be different depending on the type of test.
In this case we have a L EFT-TAIL T EST, when the Alternative Hypothesis is of
the type H1 : π1 − π2 < δ0

−Z1−α 0

The limit value of the Rejection Area, −Z1−α , can be found in the table of a
N (0, 1).
Here, if α = 0.05 then 1 − α = 0.95
We look up the tables for the value

−Z1−α = −Z0.95 = −1.645

6. T EST C ONCLUSION
Finally, we have to check if the O BSERVED VALUE OF THE T EST S TATISTIC
(OVTS) falls, or not, inside the R EJECTION A REA.
Clearly, in this case it does NOT. We have

OV T S = −0.599943 > −1.645 ⇒ OV T S ∈

/ RA

We then say that the test does not reject the N ULL H YPOTHESIS.
Notice that in this case we are NOT rejecting the hypothesis H0 : π1 − π2 =
−0.01 against the alternative H1 : π1 − π2 < −0.01
Thus, the conclusion is that the survey rejects that the difference between the
proportion of smokers in France and Spain is larger than 1%

RStudio Activity 2.4.1 - Tests for comparing samples - Preliminaires

104 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

In the same way that in the section 2.3 we did hypothesis tests on the population mean,
variance and proportion, we could now do the same with the hypothesis tests on differ-
ences in means, variances and proportions. In other words, from randomly generated
samples we could calculate the observed values of the contrast statistics corresponding
to each case and check if they are (or not) in the rejection area.
But, besides the fact that the calculation of the observed values of the test statistic
would now be much more laborious, this is not the main potentiality of RStudio when
performing hypothesis contrasts. As we have commented in some of the examples
we have done, RStudio has much more useful and efficient commands for solving
hypothesis tests. These commands, however, rely on the use of a so called p_value.
Therefore, let’s first study this concept (p_value)and then perform some tests on
differences using the full potential of RStudio.
End of Activity RStudio 2.4.1

2.5 The p_value

2.5.1 Concept, definition and computation

Consider a hypothesis test, for instance, a right-tail test on the population mean:

H0 : µ = µ0
H1 : µ > µ0

with a significance level α = 0.05 that produces the outcome below:

Z1−α = Z0.95 = 1.645

OV T S = 1.66

The conclusion of the test, as seen in section 2.3.1, is that the null hypothesis is rejected.
Suppose, though, that the outcome were

Z1−α = Z0.95 = 1.645

OV T S = 4.37
2.5. THE P_VALUE 105

Clearly, the conclusion of the test will be the same: the null hypothesis is rejected.

Somehow, though, one might argue that the conclusion in the first case is very “weak”
as the difference between the value that determines the rejection area (1.645) and the
Observed Value of the Test Statistic (1.66) is rather tiny. A small variation in one
of the data points of the sample, or a slightly larger sample, might easily reverse the
conclusion of the test.

The conclusion in the second case, on the contrary, seems rather “robust”. The dif-
ference between the value that determines the rejection area (1.645) and the Observed
Value of the Test Statistic (4.37) is comfortably large. Only a substantial change in the
data at hand could possibly modify the conclusion of the test.

Consider now the following question:

Is there a way to asses the “reliability” of the conclusion? In other words,

given the outcome of the test, how far are we from obtaining the opposite
conclusion?

The quickest answer is, clearly, to measure this difference:

|Z1−α − OV T S|

Thus, we would have a difference of 0.015 in the first case and 2.725 in the second.

The problem, though, is that if we want to compare the outcomes of different tests,
this method of measuring the “reliability” can only be applied to test based on the
same distribution (the Normal distribution in this example). This is so because if we
consider tests whose test statistics follow different distributions we will obtain values
of different magnitudes. For instance, in the case of a test on the population variance
that uses the χ2 distribution we might have (see Example 2.3.4)
106 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

α/2

α/2

74.22 129.561

Hence, the method to measure this “distance to changing the conclusion” will not be
based on the absolute value of the difference between the value that determines the re-
jection area and the Observed Value of the Test Statistic but on the size3 of the rejection
area. In this sense, the question posed above

“Given the outcome of the test, how far are we from obtaining the opposite
conclusion?”

can be reformulated as

“Given the outcome of the test, how big should the rejection area be in
order to have the opposite conclusion?”

Then, if the difference between the “real” rejection area and this “hypothetical” rejec-
tion area that we refer to in the question above is large, we can say that we are far from
changing the conclusion of the test and, consequently, this conclusion is considered
robust. If, on the contrary, this difference is small, we might very well question the
reliability of the conclusion.

Notice that, because this measure is a probability (probability of rejecting the null
hypothesis when it is true), this measure can be used in any test of any kind based on
any distribution4 .

The so called p_value, then, is just the size of this “hypothetical” rejection area.

3 Remember that when we say “”Rejection Area of size α” we are referring to its size in terms of proba-

bility, not in terms of the length of the interval that determines such area.
4 It can be applied even in non-parametric test that we will study in the next chapter, and also in more

sophisticated tests.
2.5. THE P_VALUE 107

Definition 2.5.1 Given an arbitrary hypothesis test that results in an Observed

Value of the Test Statistic OV T S, the p_value of the test is defined as:

p_value = P (X ≥ OV T S) if the test is a rigth − tail test

p_value = P (X ≤ OV T S) if the test is a left − tail test
p_value = 2 · P (X ≥ |OV T S|) if the test is a two − tails test

where X represents the distribution that corresponds to the Test Statistic of the test
considered.

Notice that the definition of the p_value depends on the type of test. The graphs
shown below, that correspond to a test based on the Normal distribution (Z), illustrate
this fact.

Right-tail test

Left-tail test
108 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

Two-tails test

The two-tails case is, to certain extent, different form the other two cases. Given that
in these tests the rejection area is split in two tails of equal size, we find the p_value
by computing the probability of one of the tails (the right tail, for example, P (Z ≥
OV T S) ), we only get half of its value as seen in the graph above. It is for this reason
that in the definition 2.5.1 of the p_value this probability is multiplied by so that we
have the whole measure of the p_value.

p_value = 2 · P (Z ≥ OV T S) if the test is a two − tails test

Continuing with the example at the beginning of this section, in the fist case we should
consider a rejection area like

P (Z ≥ 1.66)
so that the conclusion of the test reverses from rejecting the null hypothesis to not
rejecting it
Graphically:

If we compute this probability we find:

p_value = P (Z ≥ 1.66) = 0.04846

2.5. THE P_VALUE 109

and we observe that it is very close to the size α = 0.05 of the original rejection are. We
can therefore say that a minor change in the test conditions might reverse its conclusion

In the second case, on the contrary, we have an OV T S = 4.37. Hence,

p_value = P (Z ≥ 4.37) = 0.00001

and we see that is very different from α = 0.05. Thus, the conditions of the test should
change considerably to have a different conclusion. In this sense, the conclusion of the
test can be considered robust.

2.5.2 Test conclusion based on the p_value

In the previous section we have seen the definition and how to compute the p_value.
Its main use, though, is to provide a method to determine the conclusion of any test,
regardless of its type or complexity, without having to compute the rejection area and
without the need for a graphical representation to help us in determining this conclu-
sion.

Let us consider the following two cases that correspond, once more, to a test based on
the Normal distribution (Z):

Case a

Size of the rejection area

110 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

Case b

Size of the rejection area

It is clear in the first case (Case a) that the conclusion is that the null hypothesis is
rejected, whereas in the second case (Case b) it is the contrary, the null hypothesis is
not rejected. If we look for the p_value = P (Z ≥ OV T S) in each case we have:

Case a

Size of the rejection area

Case b

Size of the rejection area

2.5. THE P_VALUE 111

We notice that,

Case a: p_value < α and the null hypothesis is rejected

Case b: p_value > α and the null hypothesis is not rejected

It should be clear that this will always occur.. That is, if p_value< α, then the
OVTS will be to the right of the value that determines the rejection area (case a) and,
thus, inside the rejection area. The conclusion, therefore, is that the null hypothesis is
rejected. If, on the contrary, we find that p_value> α then the OVTS will be to the
left of the value that determines the rejection area (case b) and the conclusion will be
that it is outside the rejection area and hence the null hypothesis is not rejected
This is so for the right-tail tests, that corresponds to the cases considered above. It is
easy to see, though, that we will have the same result for a left-tail test as follows.

Case c

Size of the rejection area

Case d

Size of the rejection area

Clearly, in the first case (Case c) the conclusion is rejection of the null hypothesis
whereas in the second case (Case d) it is the opposite (no rejection).
We notice that, again,
112 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

Case c: p_value < α and the null hypothesis is rejected

Case d: p_value > α and the null hypothesis is not rejected

Finally, a similar exercise for the two-tails test will produce the same result. We can
therefore enunciate the following fact that is of great use to determine the conclusion
of any hypothesis test.

Fact 2.5.2 Given an arbitrary hypothesis test with a significance level α we have
that:
p_value < α ⇔ The null hypothesis is rejected
p_value > α ⇔ The null hypothesis is not rejected

It is worth noticing that most of the statistical software packages and publications al-
ways provide the p_value of any test for the researcher to decide on the reliability of
the conclusion.
For instance, if after conducting an arbitrary5 hypothesis test we are informed that:

p_value = 0.032

we can rapidly conclude that:

• If we consider a significance level of α = 0.01, then we observe that p_value >

α and thus conclude that the null hypothesis is not rejected.

• If we consider a significance level of α = 0.05, then we observe that p_value <

α and thus conclude that the null hypothesis is rejected.

Hence, it is up to the researcher (or the reader of the research) criterion to decide on
the conclusion of the test.

RStudio Activity 2.5.1 - Tests for comparing samples

We have just seen that the p_value is a very useful tool for the realization of any
hypothesis test, regardless of how complex or involved it might be.
Once he have learned how to use the p_value, we can now go back to the tests for
comparing samples that we had intentionally postponed until now. The reason, as we
will see below, is that the specific RStudio commands to solve hypothesis tests rely on
the p_value concept.
We will see that some of these commands are not specific to the type of test in question
but to the distribution of its test statistic. Thus, for example, a test on the population
mean or a test on the difference in proportions can be performed with the same com-
mand since in both cases the distribution of the test statistic is N(0,1).
In this regard, the commands we will see first are:
5 Once again, this could be a right-tail test, a left-tail test, a two-tails test, a test on the variance, on the

mean, on the difference of means, an ANOVA test, etc. It could be any test.
2.5. THE P_VALUE 113

C OMMAND U SE
Tests where the Test Statistic follows a N (0, 1)
[Link]()
Examples: population mean (σ known), or difference of proportions
Tests where the Test Statistic follows a t − student
[Link]()
Examples: difference of population means (σ unknown)

As said above, this commands are generic, and can be used in any test based on a
N (0, 1) or a t − student distributions.
Moreover, there exist other commands that are specific for some tests6 :

C OMMAND U SE
[Link]() Test on the population variance
[Link]() Test for the comparison of population variances

The script for this activity is:

1. Test for the comparison of population means (when any of the population vari-
ances is unknown)
2. Test for the comparison of population variances
3. Test for the comparison of population proportions
We will notice that, although the commands used will be different, their syntax and
options are very similar.

1. Test for the comparison of population means (when any of the population vari-
ances is unknown)

Generate two samples of sizes n1 = 250 and n2 = 175, that will be labeled SAMPLE _1
and SAMPLE _2, from two Normal distributions with means µ1 = 1000 and µ2 = 1100
and population variances σ12 = σ22 = 40000. Notice that we are using these values
σ1 = σ2 to generate the samples only. Later, when doing the test, we will assume that
they are unknown.
[Link](15)
SAMPLE_1 <- rnorm(250,1000,200)
SAMPLE_2 <- rnorm(175,1100,200)

We will test if the two population means are equal or not:

H0 : µ1 − µ2 = 0
H1 : µ1 − µ2 ̸= 0

The Test Statistic (TS) for this case is:

(X̄1 − X̄2 ) − δo
TS = q
S2 S2
n1 + n2

6 And, in fact, there are even more specific commands for other tests, which we’ll see in due course
114 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

where
(n1 − 1)S12 + (n2 − 1)S22
S2 =
n1 + n2 − 2
It is clear that the step-by-step computation of the Observed Valued of the Test Statistic
(OVTS) would be tedious, even with RStudio.
Using the command [Link](), though, the task becomes much simpler. We use this
specific command because the Test Statistic follows a t − student distribution.
### We will use the command '[Link]' to do the test
### The outuput of the command will be stored in a
### variable that will be called "TestCompareMeans"
### for later inspecting the results

TestCompareMeans <- [Link](SAMPLE_1,SAMPLE_2,

alternative = c("[Link]"),
mu = 0,
paired = FALSE,
[Link] = TRUE)

It is important to learn the use of each of the options in this command as in other cases
the syntax will be very similar

Tells the type of alternative hypothesis. The options are:

alternative = "[Link]" → Two tails test
c("[Link]") "less" → Left-tail test
"greater" → Right-tail test
mu = 0 Tells the value of the null hypothesis
Tells is the samples are paired or not
paired = FALSE
We will normally assume that they are not (FALSE)
Tells if the twp population variances are equal
[Link] = TRUE
We will normally assume that they are (TRUE)

Let’s type the name of the variable TestCompareMeans where the outcome of the
command has been stored to view a summary.
### Show the result of the test

TestCompareMeans

##
## Two Sample t-test
##
## data: SAMPLE_1 and SAMPLE_2
## t = -6.0294, df = 423, p-value = 3.589e-09
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -163.94100 -83.33028
## sample estimates:
## mean of x mean of y
## 997.6335 1121.2691

Of the information provided, the following elements are of interest:

2.5. THE P_VALUE 115

t = -6.0293902 It is the OVTS

Corresponds to the “degrees of freedom”
df = 423
n1 + n2 − 2 = 250 + 175 − 2 = 423
p-value = 3.589e-09 Corresponds to the p_value
alternative hypothesis: Corresponds to the type of
true difference in means the alternative hypothesis
is not equal to 0 µ1 − µ2 ̸= 0

Notice that RStudio uses “exponential format ” to display the values of very small (or
very large) numbers, like the p_value. In this sense, recall that if x is real number and
c is an integer, then :

xec = x × 10c
Thus,
3.589
valor − p = 3.589e − 09 = 3.589 × 10−9 = = 0.00000003589
109

We thus have all the information to solve the test.

But, as we discussed before, we only need the p_value and to recall the result we
saw in the section 2.5.2 that summarizes the use of the p_value:
Given an arbitrary hypothesis test with a significance level α we have that:

p_value < α ⇔ The null hypothesis is rejected

p_value > α ⇔ The null hypothesis is not rejected

Thus, in our example, the p_value is so low that the conclusion is clear:
For any significance level α chosen from within the usual values (0.01, 0.05, 0.1) we
will have:

p_value < α ⇔ The null hypothesis is rejected

Therefore:

“The null hypothesis of equality between the population means is rejected by the data”

2. Test for the comparison of population variances

In a similar way we can test if two population variances are equal or not.
IMPORTANT: Because RStudio uses its own algorithms to do the test, in this case it
is not necessary to perform the "Previous Step" that we have seen in the section 2.4.2.
That is, there is no need to look at which of the sample variances is larger or change
the order of the samples.
Let’s generate two sample of sizes n1 = 250 a n2 = 175, labeled SAMPLE _1 and SAM -
PLE _2, from two Normal distributions with means µ1 = µ2 = 1000 and population
variances σ12 = 36100 and σ22 = 40000.
116 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

[Link](15)
SAMPLE_1 <- rnorm(250,1000,190)
SAMPLE_2 <- rnorm(175,1000,200)

We will test if the two variances are equal or not:

σ2 1
H0 : =1
σ22
σ2 1
H1 : 2 ̸= 1
σ2
The proper command to use now is [Link]() which corresponds to the test for
comparing variances.
### We will use the command '[Link]' to do the test
### The outuput of the command will be stored in a
### variable that will be called "TestCompareVariances"
### for later inspecting the results
TestCompareVariances <- [Link](SAMPLE_1,SAMPLE_2,
alternative = c("[Link]"),
ratio = 1)

We already know part of the options used in this command as we have used them
previously (test for the difference of the means). The only difference is that now the
option to specify the null hypothesis is ratio = 1, that is, the ratio between the two
variances is 1 (they are equal)
Le’s type the name of the variable TestCompareVariances where the results of
the test have been saved.
### Show the results

TestCompareVariances

##
## F test to compare two variances
##
## data: SAMPLE_1 and SAMPLE_2
## F = 0.94381, num df = 249, denom df = 174, p-value = 0.6728
## alternative hypothesis: true ratio of variances is not equal to 1
## 95 percent confidence interval:
## 0.7147128 1.2379367
## sample estimates:
## ratio of variances
## 0.9438128

The main results obtained are summarized in the table below:

F = 0.9438128 Corresponds to the OVTS

Corresponds to the “degrees of freedom” of the numerator
num df = 249
n1 − 1 = 250 − 1 = 249
Corresponds to the “degrees of freedom” of the denominator
denom df = 174
n2 − 1 = 175 − 1 = 174
p-value = 0.6728215 Corresponds to the p_value
2.5. THE P_VALUE 117

Once again, with the p_value we can conclude the test:

For any significance level α chosen from within the usual values (0.01, 0.05, 0.1) we
will have:

p_value > α ⇒ The null hypothesis is not rejected

That is,

“The data do not allow us to reject the null hypothesis of equality between the
population variances”

3. Test for the comparison of population proportions

We will end this section with a test for the comparison of population proportions. Re-
member that, in this case, we must consider a Bernoulli distribution when generating
the sample.
Generate two samples of size n1 = n2 = 500, labeled SAMPLE _1 and SAMPLE _2 ,
from two Bernoulli distributions of parameters p1 = 0.4 and p2 = 0.51. That is, b(0.4)
and b(0.51) .

[Link](6)
SAMPLE_1 <- rbinom(500,1,0.4)
SAMPLE_2 <- rbinom(500,1,0.51)

The contrast we want to run from this sample is if the proportion in the first population
is lower than the proportion in the second population by more than 10%. Since this is
an inequality, we will include it in the alternative hypothesis.

H0 : π1 − π2 = −0.1
H1 : π1 − π2 < −0.1

For this test we will use the command [Link]() as the Test Statistics follows a
N (0, 1) distribution.

### We will use the command '[Link]' to do the test

### The outuput of the command will be stored in a
### variable that will be called "TestCompareProportions"
### for later inspecting the results
TestCompareProportions <- [Link](SAMPLE_1,SAMPLE_2,
alternative = c("less"),
mu = -0.1,
paired = FALSE,
[Link] = TRUE)

We now type the name of the variable TestCompareProportions where the re-
sults have been stored to see a summary.
118 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

### Show the results

TestCompareProportions

##
## Two Sample t-test
##
## data: SAMPLE_1 and SAMPLE_2
## t = -1.0853, df = 998, p-value = 0.139
## alternative hypothesis: true difference in means is less than -0.1
## 95 percent confidence interval:
## -Inf -0.08242048
## sample estimates:
## mean of x mean of y
## 0.408 0.542

Of the information provided, the following elements are of interest:

t = -1.085256 Corresponds to the OVTS

Corresponds to the “degrees of freedom”
df = 998
n1 + n2 − 2 = 500 + 500 − 2 = 998
p-value = 0.1390352 Corresponds to the p_value

Knowing the p_value we have that:

For any significance level α chosen from within the usual values (0.01, 0.05, 0.1) we
will have:

p_value > α ⇒ The null hypothesis is not rejected

Therefore:

“The data do not allow us to reject the null hypothesis π1 − π2 = −0.1”

Therefore, the difference between the population proportions does not seem to be
greater than 10%.

Comment on one-sample tests

These RStudio commands that we have used here to conduct two-samples tests can
also be used when we only have one sample. The only difference will be that instead
of entering the names of the two samples in the corresponding command (for example,

TestCompareMeans <- [Link](SAMPLE_1,SAMPLE_2, ....)

we will use only one sample:

TestMean <- [Link](SAMPLE_1, ....)

2.5. THE P_VALUE 119

Thus, for example, the test on the population mean (when population variance is un-
known) that we have done "step by step" in the RStudio Activity at the end of the
section 2.3.1

H0 : µ = 1000
H1 : µ ̸= 1000
can be easily done as follows:
### Generate the sample

[Link](1)
SAMPLE_1 <- rnorm(2500,1000,200)

### Assuming that the population variance is unknown

### we will use the t-student test ('[Link]')
TestMean <- [Link](SAMPLE_1,
mu=1000,
alternative = c("[Link]"),
paired = FALSE,
[Link] = TRUE)

### Let's take a look at the output

TestMean

##
## One Sample t-test
##
## data: SAMPLE_1
## t = -0.45055, df = 2499, p-value = 0.6524
## alternative hypothesis: true mean is not equal to 1000
## 95 percent confidence interval:
## 989.9967 1006.2654
## sample estimates:
## mean of x
## 998.131

By looking at the results we see that they are the same as those obtained in that example,
Specifically, the OVTS obtained is -0.4505454 , as when it was calculated step by step
in the section 2.3.1.
R Hints

• In a very similar way to what was done in the RStudio activity at the end
of the section1.6 where a maximum likelihood estimate was found, in this
activity we have also used variables, for example TestCompareMeans,
to save the output of the RStudio command, and later see a summary of the
results.
But we could also access a structured detail of these results if in the RStudio
120 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

console we enter the command

View(TestCompareMeans)

A new tab will open in RStudio that will look like this:

We note in the second row of this table that statistic is the "element"
that contains the information about the OVTS, and in the fourth row we find
[Link] which clearly corresponds to the p_value. If, for whatever
reason, we need to access any of these values individually, we can use the
$ sign followed by the name of the element we want to consult. That is:
TestCompareMeans$[Link]

## [1] 3.589408e-09

End of Activity RStudio 2.5.1

2.6 Analysis of Variance

The ANalysis Of VAriance (ANOVA) between groups is a statistical technique that
allows to simultaneously compare more than two populations. For instance we can
compare the productivity of different types of wheat, the performance of several makes
of cars, etc. For each of these cases (factors or treatments) we focus on one specific
numerical feature, the response: the weight of wheat, the gas consumption of cars.
What we test with the ANOVA is whether there exists a relationship between the aver-
age response and the different factors: have all the varieties of wheat the same average
weight ? do all the car makes have the same consumption on average ?

2.6.1 Basic Framework

What we test now is if the means (responses) of all the populations (the different factors
or treatments) are the same. Let k be the number of populations. We will assume that
each of the populations (i = 1, . . . , k) is distributed according to a Normal distribution
with the same variance σ 2 :

x1 ∼ N (µ1 , σ 2 ) x2 ∼ N (µ2 , σ 2 ) ··· xk ∼ N (µk , σ 2 )

2.6. ANALYSIS OF VARIANCE 121

From each of the populations j a sample of size nj is obtained.

Notation
xij Observation ith from Sample j, (i = 1, . . . , nj ; j = 1, . . . , k).
N Number of observations
k
X
N= nj
j=1

X̄j Sample mean of the sample from population j.

nj
1 X
X̄j = xij
nj i=1

¯ Total mean (or mean of all observations)

X̄
k nj
¯ 1 XX n1 X̄1 + n2 X̄2 + · · · + nk X̄k
X̄ = xij =
N j=1 i=1 n1 + n2 + · · · + nk

One can show that, if all samples have equal length n1 = n2 = · · · = nk , then the total
¯ is the “mean of the means”
mean X̄

¯ = 1 (X̄ + · · · + X̄ )
X̄ 1 k
k

2.6.2 Test

The test to perform is of the form

H0 : µ1 = µ2 = · · · = µk
H1 : Not all the µj are equal

2.6.3 Test statistic

The test is based on the study of the deviations of each observation xij with respect to
¯:
the total mean X̄
¯
xij − X̄

These deviations can be decomposed into the sum of the deviations of each observation
with respect to the mean of the sample it belongs to plus the deviations of each of the
means of the the k with respect to the total mean:
¯ ) = (x − X̄ ) + (X̄ − X̄
(xij − X̄ ¯)
ij j j
122 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

As usual, we are interested in the sum of the deviations squared

nj
k X nj
k X k
¯ )2 ¯ )2 +
X X X
(xij − X̄ = (xij − X̄j )2 + nj (X̄j − X̄
j=1 i=1 j=1 i=1 j=1
nj
k X
¯)
X
+ 2 (xij − X̄j )(X̄j − X̄
j=1 i=1

It can be proved that the third term in this equation is equal to zero so that we have:
nj
k X nj
k X k
¯ )2 = ¯ )2
X X X
(xij − X̄ (xij − X̄j )2 + nj (X̄j − X̄
j=1 i=1 j=1 i=1 j=1

or, equivalently,

Total Variation= Variation Inside the Samples + Variation Between Samples

TV = VIS + VBS

where:
ni
k X
¯ )2
X
TV = (xij − X̄
j=1 i=1
nj
k X
X
V IS = (xij − X̄j )2
j=1 i=1

k
¯ )2
X
V BS = nj (X̄j − X̄
j=1

It is important to notice that:

2 V BS
1. Under the null hypothesis, SB = k−1 is an unbiased estimator of σ 2 . Hence,
2
(k − 1)SB V BS
∼ χ2k−1 ⇒ ∼ χ2k−1 (2.1)
σ2 σ2

V IS
2. If all variances are equal, SI2 = N −k is an unbiased estimator of σ 2 . Hence,

(N − k)SI2 V IS
∼ χ2k−1 ⇒ 2 ∼ χ2N −k (2.2)
σ2 σ

2
Therefore, if the null hypothesis is true, we should expect that SB /SI2 to be close to 1.
This is what will be used as test statistic:
2
SB
F∗ =
SI2
2.6. ANALYSIS OF VARIANCE 123

2.6.4 Distribution of the test statistic

To determine the distribution of such statistic we have,

V BS χ2k−1
∗S2 σ 2 (k−1) k−1
F = B2 = V IS
∼ χ2N −k
∼ F(k−1,N −k)
SI σ 2 (N −k) N −k

That is, the test statistic F ∗ follows a Snedecor’s F distribution with k − 1 degrees of
freedom in the numerator and N − k degrees of freedom in the denominator
2
SB
F∗ = ∼ F(k−1,N −k)
SI2

2.6.5 Test conclusion

This is always a right-tail test.

Once the Observed Value of the Test Statistic (OVTS) has been obtained, it has to be
compared to the value of the Snedecor’s F distributing for a given level of significance
α, F(k−1,N −k) (1 − α)7

F1−α

The rule to reach a test conclusion is:

• If V OEC < F(k−1,N −k) (1 − α) ⇒ The Null hypothesis IS NOT rejected

• Si V OEC > F(k−1,N −k) (1 − α) ⇒ The Null hypothesis IS rejected
7 Following the rule that we have been using, we denote F(k−1,N −k) (1 − α) the value of a F random
variable that has to its left an area of 1 − α (p(F < F(k−1,N −k) (1 − α)) = 1 − α). Notice though,
that depending of the tables at use they might inform about the complementary probability, that is, p(F >
F(k−1,N −k) (1 − α))
124 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

Variation Sum d.o.f Mean sums F

V BS
Pk ¯ )2
nj (X̄j − X̄ k−1 V BS 2
= SB
j=1 k−1

Pnj 2
Pk SB
V IS j=1 i=1 (xij − X̄j )2 N −k V IS
N −k = SI2 F∗ = SI2

TV
Pk Pni ¯ )2
− X̄ N −1
j=1 i=1 (xij

Table 2.2: ANOVA Table

2.6.6 The ANOVA table

To do the test, all the information is usually arranged in the so called ANOVA table8 .
The Observed Value of the Test Statistics is also computed within such table in order
to proceed with its comparison with the F tables. The ANOVA table is of the form:

8 ANalysis Of VAriance
2.6. ANALYSIS OF VARIANCE 125

Example 2.6.1 Three groups of lots of fish, 5 lots in each group, coming from 3 differ-
ent fish markets (factors), are weighted to verify if there exists differences in the weight
(response) of the lots sold at each market. The results obtained are:

Market 1 Market 2 Market 3

48 55 51
50 52 52
53 53 50
52 55 53
50 53 50

Is there any significant difference in the weight of the different lots ? (α = 0.05)
To do the test we must compute V BS and V IS to complete the ANOVA table.
In this case we have,

Samples k 3
Observations sample 1 n1 5
Observations sample 2 n2 5
Observations sample 3 n3 5
Total observations N = n1 + n2 + n3 15

From the computations of the sample means we have:

X̄1 50.6
X̄2 53.6
X̄3 51.2

¯
X̄ 51.8

From here we can compute

k
¯ )2 = 25.2
X
V BS = nj (X̄j − X̄
j=1
nj
k X
X
V IS = (xij − X̄j )2 = 29.2
j=1 i=1

Therefore,

Variation Sum d.o.f Mean sums F

V BS 25.2
V BS 25.2 k−1=2 k−1 = 2 = 12.6

V IS 29.2 N − k = 12 V IS
N −k = 29.2
12 = 2.43 F∗ = 12.6
2.43 = 5.18

TV 54.4 N − 1 = 14
126 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

That is, we have an OVTS

F ∗ = 5.18

We must use now the table of the Snedecor’s F distribution with k − 1 = 2 degrees of
freedom in the numerator and N − k = 12 degrees of freedom in the denominator to
look up the value that has to its left a probability of 1 − α = 0.95

F(2,12) (0.95) = 3.89

Hence, since
F ∗ = 5.18 > 3.89 = F(2,12) (0.95)

we find that the OVTS is inside the rejection area and therefore the null hypothesis of
all the population means being equal is rejected.
Thus, the conclusion of the test is that we reject the hypothesis that the lots of fish
weight the same in the three fish markets

RStudio Activity 2.6.1 - Analysis of Variance

We will now conduct an Analysis of Variance to find out, in a simulation, if the province
or residence of the first-year university students in Catalonia has an impact on the
university admission grade. Using the terminology of this technique, the province will
be the factor (or treatment) while the university admission grade will be the response.
We will first generate 4 different samples, one per province, that will be indexed ac-
cording to the table below:

P ROVINCE S AMPLE
Barcelona 1
Girona 2
Lleida 3
Tarragona 4

Samples will have different sizes (n1 = 1500, n2 = 800, n3 = 500, n4 = 700) and
will consist of two variables:

• Grade: Contains the university admission grade of each student in the sample.
For each sample, this variable will be generated from a Normal distribution with
means9

µ1 = 7.297
µ2 = 7.245
µ3 = 7.314
µ4 = 7.304

and standard deviation σi = 2 in all cases.

9 Data published by the “Canal Universitats de la Generalitat de Catalunya” for 2022.
2.6. ANALYSIS OF VARIANCE 127

• Province: Will take value 1, 2, 3 or 4 depending on the province (see table

above)

These samples will be stored in a data base that will be called UniversityAdmission,
to which we will give the appropriate structure for carrying out the test.
The goal of the analysis is to test if there exist differences in the average Grade de-
pending on the Province. In other words, test if the university admission Grade
depends on the Province of residence.

H0 : µ1 = µ2 = µ3 = µ4
H1 : Not all µi are equal

We first generate the sample for Barcelona. It is a good idea to read the comments in the
code to better understand the sample generation process. For the rest of the provinces,
the procedure will be the same.
[Link](25)

### BARCELONA ###

### Generate 1500 observations corresponding

### to Barcelona according to the parameter values:

Grade <- rnorm(1500,7.297,2)

### Create the data base "UniversityAdmissionBarcelona"

### with only one column for the moment: "Grade"

UniversityAdmissionBarcelona <- [Link](Grade)

### Add another column to this database

### with the "Province" indicator: (1)

UniversityAdmissionBarcelona$Province <- 1

Do the same for the other 3 provinces

### GIRONA ###

### Generate 800 observations corresponding

### to Girona according to the parameter values:

Grade <- rnorm(800,7.245,2)

### Create the data base "UniversityAdmissionGirona"

### with only one column for the moment: "Grade"

UniversityAdmissionGirona <- [Link](Grade)

### Add another column to this database

### with the "Province" indicator: (2)

UniversityAdmissionGirona$Province <- 2
128 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

### LLEIDA ###

### Generate 500 observations corresponding

### to Lleida according to the parameter values:

Grade <- rnorm(500,7.314,2)

### Create the data base "UniversityAdmissionLleida"

### with only one column for the moment: "Grade"

UniversityAdmissionLleida <- [Link](Grade)

### Add another column to this database

### with the "Province" indicator: (3)

UniversityAdmissionLleida$Province <- 3

### TARRAGONA ###

### Generate 700 observations corresponding

### to Tarragona according to the parameter values:

Grade <- rnorm(700,7.304,2)

### Create the data base "UniversityAdmissionTarragona"

### with only one column for the moment: "Grade"

UniversityAdmissionTarragona <- [Link](Grade)

### Add another column to this database

### with the "Province" indicator: (4)

UniversityAdmissionTarragona$Province <- 4

Once we have the four samples, one per province, we will join them (bind) in a single
data base, UniversityAdmission, using the rbind() command.
UniversityAdmission <- rbind(UniversityAdmissionBarcelona,
UniversityAdmissionGirona,
UniversityAdmissionLleida,
UniversityAdmissionTarragona)

We can type View(UniversityAdmission)in RStudio to see a table with the

3500 observations in the database.
To complete the preparation of the data base, we must make sure that the variable
Province , that we want to use as a factor, is really a factor in RStudio. What we
will do is transform this variable into a new variable called Province_f which will
be of type factor.
UniversityAdmission$Province_f<-[Link](UniversityAdmission$Province)

We are now ready to proceed with the Analysis of Variance10 . The command to do so
is simple:
10 Note that in a normal case all this prior preparation can be avoided. If we have a database saved in some

.xls or .csv file it is not necessary to do all the work of generating the database we made here
2.6. ANALYSIS OF VARIANCE 129

aov(response_variable ~ factor_variable)

In our case:

### We will store the outcomo of the command

### in a variable named "ANOVA_AdmissionGrade"

ANOVA_AdmissionGrade<-aov(UniversityAdmission$Grade~UniversityAdmission$Province_f)

The test is completed. To see the results we will use the command summary()

summary(ANOVA_AdmissionGrade)

## Df Sum Sq Mean Sq F value Pr(>F)

## UniversityAdmission$Province_f 3 31 10.496 2.596 0.0508 .
## Residuals 3496 14134 4.043
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The results are displayed in the same format, although with a different order of the
columns, than the ANOVA table that we saw in Table 2.2. You can see an explanation
on how to read this output in the Appendix B “Reading RStudio Outputs”. The most
relevant values are the OVTS (F value=2.596) and the p_value, which in this
table appears under the name of Pr(>F).

He have a p_value of 0.0508 . Therefore, the conclusion will be different depending

on the level of significance α chosen :

Significance Level Conclusion

α = 0.01 or α = 0.05 p_value > α ⇒Do not reject Ho
α = 0.1 p_value < α ⇒Reject Ho

The final result is not totally clear, the sample is not conclusive. In these cases a
graphical representation of the four samples can be useful. We will show a factored
boxplot.

boxplot(UniversityAdmission$Grade~UniversityAdmission$Province_f,
xlab="Province",ylab="Grade")
130 CHAPTER 2. PARAMETRIC HYPOTHESIS TESTING

15
10
Grade

5
0

1 2 3 4

Province

We see that there are no significant differences between the four samples. Therefore,
although not a rigorous conclusion, it appears that the factor (Province) does not
have a significant effect on the response (Grade)

R Hints

• We have used the command [Link]() to build a database with the

(simulated) admission grades of the four provinces. This command is very
useful when we need to organize the data in a table with several columns
(one for each variable or sample), so that RStudio can work with it as if it
were a database with many columns imported from an .xls or .csv file
• To view the results of the analysis of variance, that is, the output of the
aov() command, we have used the summary() command. This is a
"multipurpose" command in RStudio, which produces different results de-
pending on the context or the command for which you want to get the "sum-
mary". For example:
– If we apply it to a sample or to a variable, we will get a summary of
it:
summary(UniversityAdmissionGirona$Grade)

## Min. 1st Qu. Median Mean 3rd Qu. Max.

## 0.6153 5.7864 7.1588 7.1480 8.6286 13.6450
 131

– But if we use it with some RStudio command whose purpose is to

carry out some technique or make some calculations, it will present
us with its results in an orderly manner, as we have done before with
summary(ANOVA_AdmissionGrade)
Remember, as seen in the RStudio Activities of the previous chapter,
that the View() command applied to an RStudio command also
presents the results of that command, but in an exhaustive and un-
ordered manner. For example, if we type:
View(ANOVA_AdmissionGrade)
a new tab will show up in RStudio with this information:

which might be useful if we need to access one of the results in particular.

End of Activity RStudio 2.6.1

132 CHAPTER 3. GOODNESS-OF-FIT AND ...
Chapter 3

Goodness-of-fit and analysis of

the relationship between
variables

...
This is where reality lies
The event that lead to events, Causality
The observation and relation of cause and effects
Of this temporary world, lead to
An evolution of the mind and spirit
To be repeated in the next....

RYAN S ANDERS

In the previous chapter we have studied the main instances of the so called "paramet-
ric tests": we tested hypothesis regarding one specific numerical "parameter" of the
population.
In this chapter we will study tests of the kind named "non-parametric tests". That
is. we do not perform tests regarding a specific parameter but we test more general
hypothesis.
We will first see how to test if the data in our sample seems to come from a given
theoretical distribution. Next. we will introduce the concept of "relationship" between
data in two samples. This will be important for the next chapter. More specifically. we
will introduce the analysis of the correlation between samples.

133
134 CHAPTER 3. GOODNESS-OF-FIT AND ...

3.1 Tests of goodness-of-fit

The test of goodness-of-fit checks whether a given set of data (the sample) seems to
"fit" (and how "good" is this "fit") a specific probability distribution.
For instance. we can test if a casino roulette follows a uniform distribution (as expected
if the roulette is “fair”) or whether the distribution of the income per capita in a sample
collected in Cerdanyola fits what would be a Normal distribution with the same mean
and variance (this is sometimes referred to as the "normality test").
The idea is to test if the "frequencies" observed in the sample coincide with the "fre-
quencies" (probabilities) that we can compute using the probability distribution we are
considering. The test follows different techniques for discrete and for continuous vari-
ables.

3.1.1 χ2 test of goodness-of-fit for discrete variables

The χ2 test of goodness-of-fit checks if the data in the sample fits a given discrete
probability distribution.

1. The null hypothesis is always the same:

H0 : the observed absolut frequencies and the theoretical frequencies coincide

We will learn how to compute these frequencies in step 3 below.

2. The alternative hypothesis is always the same:

H1 : the observed absolut frequencies and the theoretical frequencies DO NOT coincide

3. Test statistic
To compute the test statistic we follow the following steps:
• Fist. to compute the “observed frequencies” nO . we have to count. for each
different value xi in the sample. how many observations take this value

nO (xi ) = Number of observations that take the value xi (3.1)

• Next we have to compute the corresponding “theoretical frequencies” nT .

To do so we must use the probability function that corresponds to the spe-
cific probability distribution we are testing. Thus. each case will have its
own way to compute these theoretical probabilities.
• Finally we compute the test statistic by means of the formula
k
X (nO (xi ) − nT (xi ))2
χ2 =
i=1
nT (xi )

where k corresponds to the number of different values found in the sample.

4. Distribution of the test statistic when the null hypothesis is true
In this test the test statistic follows a χ2 distribution with k − 1 degrees of free-
dom.
3.1. TESTS OF GOODNESS-OF-FIT 135

5. Rejection area of size α

This test is always a Right-Tail Test (only a tail on the right). In the χ2 table
we find the limit value for this rejection area depending on both the size of the
rejecting area α and the degrees of freedom k − 1

6. Test conclusion
Given the special features of this test. we only need to check if the Observed
Value of the Test Statistic χ2 is bigger or not that the limit value found in the
tables. If it is bigger. then we reject the Null Hypothesis that says that the
sample follows the proposed theoretical distribution. If it is smaller then we do
not reject that hypothesis

When to fully characterize the distribution referred to in the null hypothesis we need
to estimate some parameters of it (for example the parameter λ in the case of a Poisson
distribution) then the variable χ2 mentioned in step 4 of the above procedure will have
k − r − 1 degrees of freedom. where r is the number of parameters to be estimated.

Example 3.1.1 Looking at the winners in 1000 drawings of the lotto. we observe the
following frequencies regarding the occurrence of each digit.

Digit Frequency
0 97
1 102
2 98
3 103
4 94
5 108
6 95
7 101
8 96
9 106

Test if the data fits a uniform distribution over the digits{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}. that
is. if each digit is equally likely.

1. The null hypothesis is always the same:

H0 : the observed absolut frequencies and the theoretical frequencies coincide

where. in this case. the “theoretical frequency” is the one that corresponds to
the uniform probability distribution.

2. The alternative hypothesis is always the same:

H1 : the observed absolut frequencies and the theoretical frequencies DO NOT coincide

3. Test statistic
To compute the test statistic we follow the following steps:
136 CHAPTER 3. GOODNESS-OF-FIT AND ...

• Fist. to compute the “observed frequencies” nO . we have to count. for

each different value xi in the sample. how many observations take this
value

nO (xi ) = Number of observations that take the value xi

According to the information provided we have:

Digit (xi ) nO (xi )

0 97
1 102
2 98
3 103
4 94
5 108
6 95
7 101
8 96
9 106

• Next we have to compute what are the corresponding “theoretical frequen-

cies” if each value (digit) had the same probability (uniform distribution).
Since we have 1000 observations and 10 different digits. if the distribution
where uniform each digit should occur 100 times. Thus.

Digit (xi ) nT (xi )

0 100
1 100
2 100
3 100
4 100
5 100
6 100
7 100
8 100
9 100

• We finally compute the observed value of the test statistic. The table below
helps in doing so:
3.1. TESTS OF GOODNESS-OF-FIT 137

(nO (xi )−nT (xi ))2

Digit (xi ) no (xi ) nT (xi ) no (xi ) − nT (xi ) (no (xi ) − nT (xi ))2 nT (xi )
0 97 100 -3 9 0.09
1 102 100 2 4 0.04
2 98 100 -2 4 0.04
3 103 100 3 9 0.09
4 94 100 -6 36 0.36
5 108 100 8 64 0.64
6 95 100 -5 25 0.25
7 101 100 1 1 0.01
8 96 100 -4 16 0.16
9 106 100 6 36 0.36
SUM 2.04
Thus we get the following observed value of the test statistic:
10
X (nO (xi ) − nT (xi ))2
χ2 = = 2, 04
i=1
nT (xi )

4. Distribution of the test statistic when the null hypothesis is true

In this test the test statistic follows a χ2 distribution with k − 1 = 10 − 1 = 9
degrees of freedom

5. Rejection area of size α

This test is always a Right-Tail Test (only a tail on the right. In the χ2 table with
9 degrees of freedom we find the limit value for this rejection area depending on
its size α = 0.05 (that is. 1 − α = 0, 95)

χ29 (0, 95) = 16, 919

6. Test conclusion
We only need to check if the Observed Value of the Test Statistic χ2 is bigger or
not that the limit value found in the table. In this case.

χ2 = 2, 04 < 16, 919 = χ29 (0, 95)

Thus. we do not reject the hypothesis that the digits occur following a uniform
probability distribution: all of them are equally likely)

This test can also be applied to continuous variables when the corresponding data is
arranged in intervals. being k the number of such intervals.
Finally. as in many other cases. the sample size must be large for the test conclusion to
be reliable.

RStudio Activity [Link] - Tests of goodness-of-fit for discrete variables

There is a specific command to do this test RStudio : [Link](). Not surpris-
ingly, the name of the command has to do with the fact that the Test Statistic follows a
χ2 distribution.
138 CHAPTER 3. GOODNESS-OF-FIT AND ...

The script for this activity is:

1. Generate a random sample from a Normal distribution with mean µ = 5 and

variance σ 2 = 2.5
2. Proceed to “discretize” this sample by rounding up each element to the closest
integer value (Ex: from 3.463 to 3). The reason for doing this is that, remember,
we want to do a test for discrete variables.
3. Test if the generated (and “discretized”) sample fits (goodness-of-fit) a Bino-
mial distribution with parameters n = 10 and p = 12 . That is, a B(10, 12 )

The reason to choose those specific values of the parameters is that, remember, the
Normal distribution is a good approximation to the Binomial distribution when:

µ=n·p and σ 2 = n · p · (1 − p)

That is,
µ = n · p = 5 and σ 2 = n · p · (1 − p) = 2.5
Therefore, with this test we are somehow verifying the goodness of that fit as well.

1. Generate a random sample from a Normal distribution

The sample will have 500 observations.

[Link](2)
SAMPLE_1 <- rnorm(500,5,sqrt(2.5))

We can view the first 10 values

head(SAMPLE_1,10)

## [1] 3.581854 5.292272 7.510604 3.212719 4.873111 5.209375 6.119375 4.621004

## [9] 8.137729 4.780558

2. Discretize the sample.

We discretize the values in the sample with the command round(), that rounds
up each value to its closest integer.

SAMPLE_1_DISC <- round(SAMPLE_1)

Again, we can view the first 10 values

head(SAMPLE_1_DISC,10)

## [1] 4 5 8 3 5 5 6 5 8 5

We see that are the same first 10 values in the original sample, but “discretized”.

3. Test if the generated (and “discretized”) sample fits a Binomial distribution

To do the test we must first build two vectors:
3.1. TESTS OF GOODNESS-OF-FIT 139

(a) A vector of “observed” absolute frequencies” of the values in the (dis-

cretized) sample. This is what we have represented as nO (xi ) when in-
troducing this test in [3.1]
(b) A vector of probabilities (or “theoretical relative frequencies1 ) for each of
these values according to the Binomial distribution B(10, 21 ) that we want
to test.

We will generate these two vectors using RStudio commands

(a) For the vector of observed absolute frequencies we will use the command
table().
### We will first compute the absolute frequencies
### of the values in the (discretized) sample.
### We will call 'FREQUENCIES' to the result

FREQUENCIES <- table(SAMPLE_1_DISC)

### We can view the corresponding table of frequencies

FREQUENCIES

## SAMPLE_1_DISC
## 1 2 3 4 5 6 7 8 9 10
## 6 22 55 97 117 101 65 31 4 2

(b) To compute the probabilities that correspond to the Binomial distribution

we must first generate a sequence with the values observed in the sample
(that is, from 1 to 10)2 . We will the use the command dbinom(), that
computes Binomial probabilities, to construct the vector of probabilities.
### Generate the sequence

SEQUENCE<- seq(1,10)

### Generate the probabilities vector

### that we will call 'PROBABILITIES'

PROBABILITIES <- dbinom(SEQUENCE,10,1/2)

### View these PROBABILITIES

PROBABILITIES

## [1] 0.0097656250 0.0439453125 0.1171875000 0.2050781250 0.2460937500

## [6] 0.2050781250 0.1171875000 0.0439453125 0.0097656250 0.0009765625

Once we have these two vector, we can proceed with the test. The syntax of the
command below is self-explanatory. The option p= tells what is the vector of
probabilities to use, that is, the distribution with respect to which we want to test
the goodness of fit”.
1 Note that in the explanation of the test what we considered was a vector of "theoretical" absolute fre-

quencies nT (xi ). But if we use RStudio this vector must be of relative frequencies, that is, of probabilities.
2 Although a variable that follows a Binomial distribution can take the value zero (and therefore the correct

sequence would be from 0 to 10), we will only calculate the probabilities of the values that have appeared in
the generated sample. This is so because RStudio requires that the two vectors, the one of frequencies and
the one of probabilities, have the same dimension.
140 CHAPTER 3. GOODNESS-OF-FIT AND ...

But first we need to make an adjustment in the vector of probabilities, since there
is a problem. If we check we will see that, due to the limitation of the number of
decimals, the sum of the probabilities is not exactly 1, and to run the test RStudio
needs a vector of probabilities that add up to exactly 1. What we will do to fix
this problem is to "normalize" this vector dividing each probability by the sum
of all of them. This way the sum will be exactly 1.

TestGoodnesOfFit <- [Link](FREQUENCIES,

p = PROBABILITIES * (1/sum(PROBABILITIES)))

Let’s take a look to the results of the test

TestGoodnesOfFit

##
## Chi-squared test for given probabilities
##
## data: FREQUENCIES
## X-squared = 10.334, df = 9, p-value = 0.3241

We find a p_value = 0.3241 which is quite high. Thus, for any of the usual
values of α (0.01, 0.05, 0.1), we have that

p_value > α ⇒ The null hypothesis is not rejected

Therefore, the data does not contradict the hypothesis that the sample fits well a
Binomial B(10, 12 ) distribution.

On the other hand we now, because we have generated the sample, that the data
come from a Normal N (5, 2.5) distribution. In short, in this example we are also
checking that the Normal distribution is a good approximation of the Binomial
distribution. If we graphically represent the two distributions we observe:

hist(SAMPLE_1,freq=FALSE)
curve(dnorm(x, 5, sqrt(2.5)),
col="darkblue", lwd=2, add=TRUE, yaxt="n")
3.1. TESTS OF GOODNESS-OF-FIT 141

Histogram of SAMPLE_1
0.20
Density

0.10
0.00

0 2 4 6 8 10

SAMPLE_1

A part form the fact that one is continuous distribution and the other is a discrete
distribution, the representations are close to each other.

R Hints

• In this activity we have used some simple RStudio commands that some-
times are very useful for performing simple tasks:
– seq(a,b,c): Creates a sequence of consecutive integers starting
at a, increasing in value in steps of size c, and ending at b. For
example
seq(0,10,2)

## [1] 0 2 4 6 8 10

– table(NAME_VARIABLE): For discrete variables, it shows in a

simplified way the values that the variable takes and their absolute
frequencies
– hist(NAME_VARIABLE): For discrete variables, draws a simple
histogram representing the frequencies of the variable
– curve(<options>): Plots a curve according to the specified op-
tions.
142 CHAPTER 3. GOODNESS-OF-FIT AND ...

As with every command in RStudio, help(<command>) provides infor-

mation about its syntax and options.

End of Activity RStudio [Link]

3.1.2 Kolmogorov-Smirnov test of goodness-of-fit for continuous

variables. The normality test.

The Kolmogorov-Smirnov test of goodness-of-fit checks whether a given set of data

(the sample) seems to "fit" a specific continuous probability distribution. and how
good such "fit" is.
We include below. as an illustration. the so called normality test (also known as the
Lilliefors test) that checks whether some data fits a Normal distribution.
For instance. we can test whether the distribution of the income per capita in a sample
collected in Cerdanyola fits what would be a Normal distribution with the same mean
and variance as the ones that we can compute in that sample.
The idea is to test if the frequencies observed in the sample coincide with the theoretical
frequencies (probabilities) that we can compute using a Normal distribution with those
mean and variance.
Hence. the procedure focuses on looking at the differences between the "observed
frequencies" (in the sample) and the "theoretical frequencies" (according to a Normal
distribution) to determine if these differences are small enough as to conclude that.
indeed. the data in the sample seems to follow a distribution close to that of a Normal.
The procedure is as follows when we want to test if the data "fits" a Normal distribution
with mean= µ and variance= σ 2 . that is. N (µ, σ 2 )

1. The Null Hypothesis for this test is always the same:

H0 : the observed absolut frequencies and the theoretical frequencies coincide

In this case we will compare the "observed cumulative frequencies" in the sam-
ple. FO . with the "theoretical cumulative frequencies” (probability). FT . ac-
cording to a Normal distribution3 . How these frequencies are computed would
be explained later.

2. The Alternative Hypothesis also is always the same:

H1 : the observed absolut frequencies and the theoretical frequencies DO NOT coincide

3. Test Statistic
For this test. the computation of the Test Statistic is rather involved.
3 The test could also be done to check if the data behaves according to another distribution. like an

exponential. a Poisson. a Binomial. etc. Here we focus only on the "normality test". that is. to check if the
data in the sample behaves according to a Normal distribution.
3.1. TESTS OF GOODNESS-OF-FIT 143

First. for the observed frequencies FO . we must compute for each element in
the sample what is the proportion (or frequency) of elements that are "smaller or
equal" to that value

Number of elements in the sample smaller or equal than xi

FO (xi ) =
Total number of elements in the sample

Now we must compute (using the N (0, 1) tables) what are the corresponding
theoretical frequencies according to the Normal N (µ, σ 2 ) we are testing for:

X −µ xi − µ xi − µ
FT (xi ) = P (X ≤ xi ) = P ( √ ≤ √ ) = P (Z ≤ √ )
σ 2 σ 2 σ2

where Z ∼ N (0, 1)

Finally. we compute the differences between each of the "observed frequencies"

FO (xi ) and the corresponding "theoretical frequencies" FT (xi ) and then select
the "maximum" (in absolute value) of these differences. This value will be the
Observed value of the Test Statistic. That is. the Test Statistic for this test. that
we denote by K − S is given by:

K − S = max |FO (xi ) − FT (xi )|

and the corresponding Observed Value of the Test Statistic follows from the com-
putation of the differences and the selection of the "maximum" difference as
explained above.

4. Distribution of the Test Statistic when the Null Hypothesis is true

For this test. the distribution to use is a special one named the Kolmogorov-
Smirnov distribution. whose values are also in tables

5. Rejection Area of size α

This test is always a Right-Tail Test (only a tail on the right). In the Kolmogorov-
Smirnov tables we find the limit value for this rejection area depending on both
the size of the rejecting area α and the size of the sample n

6. Test conclusion

Given the special features of this test. we only need to check if the Observed
Value of the Test Statistic KS is bigger or not that the value found in the Kolmogorov-
Smirnov tables. If it is bigger. then we reject the Null Hypothesis that says that
the sample follows the distribution of a Normal. If it is smaller then we do not
reject that hypothesis

Example 3.1.2 In a recent survey conducted among 5000 families in Cerdanyola. the
following data regarding income was collected
144 CHAPTER 3. GOODNESS-OF-FIT AND ...

Income (C) Frequency

0-5000 33
5001-10000 78
10001-15000 224
15001-20000 456
20001-25000 753
25001-30000 952
30001-35000 959
35001-40000 753
40001-45000 459
45001-50000 218
50001-55000 82
55001-60000 28
>60000 5

Test whether the data follows a Normal distribution with mean 30.000 and standard
deviation 10.000

1. The Null Hypothesis for this test is always the same:

H0 : the observed absolut frequencies and the theoretical frequencies coincide

2. The Alternative Hypothesis also is always the same:

H1 : the observed absolut frequencies and the theoretical frequencies DO NOT coincide

3. Test Statistic

• First. for the observed frequencies FO . we must compute for each element
in the sample what is the proportion (or frequency) of elements that are
"smaller or equal" to that value
Number of elements in the sample smaller or equal than xi
FO (xi ) =
Total number of elements in the sample
According to the data at hand we have the following
Income (C) Frequency Cumulative Freq Fo
0-5000 33 33 0.0066
5001-10000 78 111 0.0222
10001-15000 224 335 0.0670
15001-20000 456 791 0.1582
20001-25000 753 1544 0.3088
25001-30000 952 2496 0.4992
30001-35000 959 3455 0.6910
35001-40000 753 4208 0.8416
40001-45000 459 4667 0.9334
45001-50000 218 4885 0.9770
50001-55000 82 4967 0.9934
55001-60000 28 4995 0.9990
>60000 5 5000 1.0000
3.1. TESTS OF GOODNESS-OF-FIT 145

• Now we must compute (using the N (0, 1) tables) what are the correspond-
ing theoretical frequencies according to the "Normal" N (30.000, 10.0002 )
we are testing for:
X − 30.000 xi − 30.000 xi − 30.000
FT (xi ) = P (X ≤ xi ) = P ( √ ≤ √ ) = P (Z ≤ )
10.000 2 10.000 2 10.000

where Z ∼ N (0, 1). In this case we have:

Income (C) Standardized Income FT

0-5000 -2.5 0.0062
5001-10000 -2.0 0.0228
10001-15000 -1.5 0.0668
15001-20000 -1.0 0.1587
20001-25000 -0.5 0.3085
25001-30000 0 0.5000
30001-35000 0.5 0.6915
35001-40000 1.0 0.8413
40001-45000 1.5 0.9332
45001-50000 2.0 0.9772
50001-55000 2.5 0.9938
55001-60000 3.0 0.9987
>60000 ∞ 1.0000
• Finally. we compute the differences between each of the observed frequen-
cies FO (xi ) and the corresponding theoretical frequencies FT (xi )

Income (C) Fo FT |Fo − FT |

0-5000 0.0066 0.0062 0.0004
5001-10000 0.0222 0.0228 0.0006
10001-15000 0.0670 0.0668 0.0002
15001-20000 0.1582 0.1587 0.0005
20001-25000 0.3088 0.3085 0.0003
25001-30000 0.4992 0.5000 0.0008
30001-35000 0.6910 0.6915 0.0005
35001-40000 0.8416 0.8413 0.0003
40001-45000 0.9334 0.9332 0.0002
45001-50000 0.9770 0.9772 0.0002
50001-55000 0.9934 0.9938 0.0004
55001-60000 0.9990 0.9987 0.0003
>60000 1.0000 1.0000 0.0000

The Test Statistic for this test. that we denote by K − S is given by:

K − S = max |FO (xi ) − FT (xi )|

and the corresponding Observed Value of the Test Statistic follows from the
computation of the differences and the selection of the "maximum" differ-
ence according to the table above.

K − S = max |FO (xi ) − FT (xi )| = 0.0008

146 CHAPTER 3. GOODNESS-OF-FIT AND ...

4. Distribution of the Test Statistic when the Null Hypothesis is true

For this test. the distribution to use is a special one named the Kolmogorov-
Smirnov distribution. whose values are also in tables

5. Rejection Area of size α

This test is always a Right-Tail Test (only a tail on the right). In the Kolmogorov-
Smirnov tables we find the limit value for this rejection area depending on both
the size of the rejecting area α = 0.05 and the size of the sample n = 5.000. In
this case
1.22 1.22 1.22
Limit V alue = √ = √ = = 0.01725
n 5000 70.71

6. Test conclusion
Now we only need to check if the Observed Value of the Test Statistic KS is bigger
or not that the value found in the Kolmogorov-Smirnov tables. In this case

K − S = 0.0008 < 0.01725

Thus. since it is smaller we do not reject the Normality hypothesis

Because of the importance of the Normal distribution. there are many “normality tests”.
the most important being the Shapiro-Wilk. Jarque-Bera. and Shapiro-Francia tests.
The Kolmogorov-Smirnov test applied to an arbitrary continuous distribution would be
very similar to normality test above. the only difference would be the way to compute
the “theoretical frequencies” FT (xi ).

RStudio Activity [Link] - Contrast de la bondat d’ajust per variables contínues

We will now practice the Kolmogorov-Smirnov (KS) test to check the Normality of a
sample which, once more, will consist of a simulated sample randomly generated. The
command to run the test will be:

[Link](sample, pnorm, mu, sigma)

where:

• sample is the name of the sample (or variable) used

• • pnorm is the distribution with respect to which one wants to test the fitness,
the Normal distribution in this case.

• mu and sigma are the parameters of the Normal distribution being consid-
ered. If they are not known, or no hypothesis is made about them, it is cus-
tomary to use their estimates from the sample: mu<-mean(sample) and
sigma<-sd(sample)

Generate a sample of size n = 500 from a N (100, 4) distribution.

3.1. TESTS OF GOODNESS-OF-FIT 147

[Link](2)
SAMPLE_1 <- rnorm(500,100,4)

We now conduct the KS test using the estimates of the mean (mu) and standard devia-
tion (sigma) obtained from the sample.

mu<-mean(SAMPLE_1)
sigma<-sd(SAMPLE_1)

[Link](SAMPLE_1, pnorm, mu, sigma)

##
## Asymptotic one-sample Kolmogorov-Smirnov test
##
## data: SAMPLE_1
## D = 0.022738, p-value = 0.9583
## alternative hypothesis: two-sided

In the information provided we find:

• D = 0.022738, that is the OVTS of the KS test

• p-value = 0.9583, that, obviously, is the p_value

Once again, the p_value is clearly above any of the values of α that we use normally.
Thus, the conclusion is clear:

p_value > α ⇒ The null hypothesis is not rejected

So it seems that the data fits the Normal distribution considered. Graphically,

hist(SAMPLE_1,freq=FALSE)
curve(dnorm(x,100,4), col="darkblue", lwd=2, add=TRUE, yaxt="n")
148 CHAPTER 3. GOODNESS-OF-FIT AND ...

Histogram of SAMPLE_1

0.08
Density

0.04
0.00

90 95 100 105 110

SAMPLE_1

Experimentally, just to see what happens when the data does not fit a Normal dis-
tribution, we will now repeat the same exercise, but generating the sample from an
exponential distribution with the same mean as before, 100. Therefore, given that in
the case of the exponential we have E(X) = 1/λ, the parameter that we will use is
λ = 1/100
Generate a new sample of size n = 500 from a exp(1/100) distribution.
[Link](2)
SAMPLE_2 <- rexp(500,1/100)

We again do a KS test using the estimates of the mean (mu) and standard deviation
(sigma) obtained from the sample.
mu<-mean(SAMPLE_2)
sigma<-sd(SAMPLE_2)

[Link](SAMPLE_2, pnorm, mu, sigma)

##
## Asymptotic one-sample Kolmogorov-Smirnov test
##
## data: SAMPLE_2
## D = 0.15259, p-value = 1.543e-10
## alternative hypothesis: two-sided

In the information provided in this case we have:

3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 149

• D = 0.15259, that is the OVTS of the KS test

• p-value = 1.543e-10, is the p_value

In this case, the p_value is extremely low4 , way below any of the usual values of α
. Thus,
p_value < α ⇒ The null hypothesis is rejected

It is clear that in this case the data does not fit the Normal distribution. Graphically

hist(SAMPLE_2,freq=FALSE)
curve(dnorm(x,100,4), col="darkblue", lwd=2, add=TRUE, yaxt="n")

Histogram of SAMPLE_2
0.004
Density

0.000

0 100 300 500

SAMPLE_2

End of Activity RStudio [Link]

3.2 Analysis of the relationship between variables

Consider two random samples corresponding to two different characteristics (param-
eters) of one population. or to the same parameter in two different populations. We
could. for instance. think of one sample with data about the unemployment rate in
the European Union and another sample with data about the inflation rate also in the
4 Recall that 1.543e-10 = 1.543 × 10−10 = 0.000000001543
150 CHAPTER 3. GOODNESS-OF-FIT AND ...

European Union. Then. we might wonder if there is a "relationship" between these

data. that is. if it is true that when the unemployment rate is high then the inflation
rate is low and vice versa. This type of questions is more ambitious for the economic
analysis than those addressed in previous chapters. Indeed. for the design of economic
policies it is very important to know what kind of relation exists among the different
economic variables.
In this sense. there are two types of relationships that we can observe between two
given variables:

1. Causal
We say that two variables have a "causal" relationship when changes in one of
the variables induce changes in the other one. For instance. it seems clear that
the lower is the interest rate the higher is the demand for mortgage loans. One
can say then that a fall in the interest rates causes a raise in the demand for loans

2. Spurious
We say that two variables have a "spurious" relationship when they seem to be
related but this relation is not causal but explained from some other factor. like
a third variable that is independently related to each of these two or some other
unknown factor. For instance. in the XVIIIth century Castilla one could observe
that the larger the number of storks the higher the number of new born. Since
there is no apparent “causality” in this relationship we must look for another
explanation. The fact is that the XVIIIth century Castilla was a rural society
based. mainly. on wheat growing. Grain was harvested by the end of June and
sold by July. Once monetary profits were collected. many people celebrated
weddings in August and September. From here. the natural cycle of human
reproduction explains why the births came along with the storks (March-April)

Once we now if two variables are related or not. it is very important to understand what
is the strength and the kind of the relationship they have. Indeed. even if two variables
are related to each other we can not use this relationship trying to influence one of the
variables by means of changes in the other if we do not know how it works

3.2.1 Test of independence between qualitative variables

The goal is to test if two qualitative (it might also be used for quantitative discrete)
random variables are independent or if. on the contrary. they are related.
Data observed from the two variables is arranged in a table called the contingency table.

y1 y2 ··· yc Total
x1 O11 O12 ··· O1c X1
x2 O21 O22 ··· O2c X2
.. .. .. .. .. ..
. . . . . .
xr Or1 Or2 ··· Orc Xr
Total Y1 Y2 ··· Yc n
3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 151

where
{x1 , . . . , xr } → Different values taken by the Xvariable
{y1 , . . . , yc } → Different values taken by the Y variable
Oij → Number of joint observations of xi and yj

From this table we can compute the so called table of expected frequencies. which
corresponds to the frequencies that one would observe if the variables were indeed
independent.

y1 y2 ··· yc Total
x1 E11 E12 ··· E1c
x2 E21 E22 ··· E2c
.. .. .. .. ..
. . . . .
xr Er1 Er2 ··· Erc
Total n

where each expected frequency Eij is computed using the formula:

Xi Yj Xi · Yj
Eij = n · p(X = xi ∩ Y = yj ) = n · p(X = xi ) · p(y = yj ) = n · · =
n n n

The formal comparison of this table of expected frequencies with the observed contin-
gency table drives this test
More specifically. the test consists of:
H0 : X is independent of Y
H1 : X and Y are related
The test statistic in this case is given by
r X c
X (Oij − Eij )2
P =
i=1 j=1
Eij

It can be shown that. under the null hypothesis. this statistic has a χ2 distribution
with (r − 1)(c − 1) degrees of freedom. This is always a right tail test. Hence. the
null hypothesis (independence) will be rejected whenever the observed value of the test
statistic P is bigger than the value in the tables corresponding to a χ21−α

Example 3.2.1 500 interviews were conducted among junior executives of several
firms. Such interviews asked about the yearly income and the degree earned at the
university. The data collected is shown in the table below

Income \ Degree Economics Not Economics Total

Less than 80.000 C 100 250 350
More than 80.000 C 100 50 150
Total 200 300 500
152 CHAPTER 3. GOODNESS-OF-FIT AND ...

Based on this data. can we conclude that the yearly income is related to the fact that
the individual earned an economics degree ?
Let X be the variable representing the yearly income. X ∈ {less than 80.000, more than 80.000}
and Y the variable that represents the degree earned. Y ∈ {economics, not economics}
The test to perform is:

H0 : X and Y are independent

H1 : X and Y are related

To do so. we must compute the Expected Frequency of each pair of data if the null
hypothesis were true
Xi Yj Xi · Yj
Eij = n · p(X = xi ∩ Y = yj ) = n · p(X = xi ) · p(y = yj ) = n · · =
n n n

According to the data collected we have the following table of Observed Frequencies
Oij

Income \ Degree Economics Not Economics Total

Less than 80.000 C O11 =100 O12 =250 X1 =350
More than 80.000 C O21 =100 O22 =50 X2 =150
Total Y1 =200 Y2 =300 n =500

Thus.
X1 · Y1 350 · 200
E11 = = = = 140
n 500
X1 · Y2 350 · 300
E12 = = = = 210
n 500
X2 · Y1 150 · 200
E21 = = = = 60
n 500
X2 · Y2 150 · 300
E22 = = = = 90
n 500

Hence we have the following table of Expected Frequencies Eij .

Income \ Degree Economics Not Economics Total

Less than 80.000 C E11 =140 E12 =210 X1 =350
More than 80.000 C E21 =60 E22 =90 X2 =150
Total Y1 =200 Y2 =300 n =500

To compute now the Observed Value of the Test Statistic

r X c
X (Oij − Eij )2
P =
i=1 j=1
Eij

we use the table below

3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 153

(Oij −Eij )2
Obs. Freq. Oij Exp. Freq. Eij Oij − Eij (Oij − Eij )2 Eij

1600
100 140 -40 1.600 140 = 11.43

1600
250 210 40 1.600 210 = 7.62

1600
100 60 40 1.600 60 = 26.67

1600
50 90 -40 1.600 90 = 17.78

P = 63.5

Thus. we have an Observed Value of the Test Statistic

P = 63.5

To complete the test we must check if this value P lies inside the Rejection Area deter-
mined by a χ2 with (r − 1)(c − 1) = (2 − 1)(2 − 1) = 1 degrees of freedom. Looking
up the tables of the value of χ21 that has to its left a probability of 0.95 (assuming
α = 0.05) we find
χ21 (0.95) = 3.84

Thus. since
P = 63.5 > 3.84 ⇒ P ∈ RA

we conclude that the empirical data rejects the null hypothesis of independence

The conclusion of the test is. therefore. that there seems to exist a relationship between
the degree earned at the University and the yearly income of junior executives

RStudio Activity [Link] - Test of independence between qualitative variables

To show how to carry out this test in RStudio, and given that (as in all the other RStudio
Activities in this textbook) the data will be generated randomly, we will do it in such a
way that it “allows for the possibility” that the two samples are related. At the end we
will experiment with this “possibility” of relationship (or independence) between the
samples.

The script for this activity is:

154 CHAPTER 3. GOODNESS-OF-FIT AND ...

1. Generate a first sample (SAMPLE_1) with 500 observations form a Binomial,

B(5, 1/2), distribution
2. Generate a sample (NOISE) of “white noise” (random noise) with 500 observa-
tions from a standard Normal, N (0, 1), distribution.

3. Generate a second sample (SAMPLE_2) by adding this “white noise” to the first
samplea :

SAMPLE _2 = SAMPLE _1 + NOISE

Theoretically, this noise does not add any relevant information to the first sam-
ple. Therefore, we should expect a clear relationship between the two samples:

SAMPLE_1 and SAMPLE_2 are NOT expected to be independent

4. Carry out the test of independence (χ2 ) between the two samples

5. Experiment with the size of the “noise” to learn about its effect.
a Asin previous cases, we will need to "round " (round()) this second sample since, remember, we
are working with a test for qualitative (discrete) variables.

Let us carry on.

1. Generate a first sample (SAMPLE_1) with 500 observations form a Binomial,

B(5, 1/2), distribution
[Link](2)

### Generate the first SAMPLE using a Binomial

SAMPLE_1 <- rbinom(500,5,1/2)

2. Generate a sample (NOISE) of “white noise” (random noise) with 500 observa-
tions from a standard Normal, N (0, 1), distribution.
### Generate the 'white noise' using a N(0,1)
NOISE <- rnorm(500,0,1)

3. Generate a second sample (SAMPLE_2) by adding this “white noise” to the first
sample
### Generate the second SAMPLE by adding the NOISE
### to SAMPLE 1 (and "round up" to make it discrete)
SAMPLE_2 <- round(SAMPLE_1 + NOISE)

We can now view a highly simplified dimensional table of frequencies (contin-

gency table) between the two samples.
table(SAMPLE_1,SAMPLE_2)

## SAMPLE_2
## SAMPLE_1 -1 0 1 2 3 4 5 6 7
3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 155

## 0 0 9 6 4 0 0 0 0 0
## 1 5 19 34 26 3 0 0 0 0
## 2 2 7 30 51 48 10 1 0 0
## 3 0 1 7 27 55 40 12 1 0
## 4 0 0 0 5 20 34 15 4 0
## 5 0 0 0 0 2 7 7 6 2

4. Carry out the test of independence (χ2 ) between the two samples
The command to use in this case is [Link]:

TestIndependenceDiscrete <- [Link](SAMPLE_1,SAMPLE_2)

TestIndependenceDiscrete

##
## Pearson's Chi-squared test
##
## data: SAMPLE_1 and SAMPLE_2
## X-squared = 449.59, df = 40, p-value < 2.2e-16

We get a very low p_value. In fact, the output tells us that it is below 2.2e-
16 = 0.0000000000000022. Hence, undoubtedly lower than any value of α we
might want to consider. Consequently, it is clear that:

p_value < α ⇒ The null hypothesis IS rejected

That is, the null hypothesis of independence is rejected and thus we conclude
that the two variables are related. As expected, adding a little bit of noise to one
sample generates another sample which is tightly related to the original one. The
question we will explore now is: how “small” should this noise be?

5. Experiment with the size of the “noise” to learn about its effect.
What we will to do is to increase the variance of the noise, its “volatility”. In
some sense, we are going to make it “more random”. To do this, now the sample
NOISE will be generated using a N (0, 25) distribution (25 times more variance
than before)
We follow the same procedure as before, but introducing this change:
Generate the samples

[Link](2)

### Generate the first SAMPLE using a Binomial

SAMPLE_1 <- rbinom(500,5,1/2)

### Generate noise, which will not be 'white'

### anymore using a N(0,25)

NOISE <- rnorm(500,0,5)

### Generate the second SAMPLE by adding the NOISE

### to SAMPLE 1 (and "round up" to make it discrete) S

SAMPLE_2 <- round(SAMPLE_1 + NOISE)

156 CHAPTER 3. GOODNESS-OF-FIT AND ...

We run the test again

TestIndependenceDiscrete <- [Link](SAMPLE_1,SAMPLE_2)

TestIndependenceDiscrete

##
## Pearson's Chi-squared test
##
## data: SAMPLE_1 and SAMPLE_2
## X-squared = 131.82, df = 140, p-value = 0.6767

Now we obtain a p_value much higher than before. As a matter of fact it is

higher than any value of α that we might use. Consequently,

p_value > α ⇒ The null hypothesis is not rejected

That is, we cannot reject the null hypothesis of independence between the two
samples.
The conclusion is that the noise is so disturbing now that the new sample has
resulted completely unlinked from the original one, and the two samples have
become independent from each other.

R Hints

• In this activity we have used the command round() to “round” a real

number to its closest integer value. RStudio has similar commands which,
depending on the case, allow for more control on how this rounding is done:

– ceiling(a): Rounds the number a to the nearest higher inte-

ger value.
– floor(a): Rounds the number a to the nearest lower integer
value.

For example:
a <- 3.42

round(a)

## [1] 3

ceiling(a)

## [1] 4

floor(a)

## [1] 3

End of Activity RStudio [Link]

3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 157

3.2.2 Analysis of the correlation between quantitative variables.

The correlation coefficient.

The goal now is to test if two continuous random variables are related or not. and what
is the degree of such relationship.

Let us consider a series of pairs of data corresponding to two continuous variables X

and Y

X Y
x1 y1
x2 y2
.. ..
. .
xn yn

If we plot these pairs of data we will obtain. basically. one of the four types of graphs
in figure 3.1. named dispersion graphs

Each of these plots corresponds to a specific type of relationship between the variables
X and Y . as explained in the table of figure 3.2

With the correlation analysis we want to find out:

1. What is the type of the relationship between the variables

2. What is the degree of the relationship between the variables

The so called Correlation Coefficient will inform about the type and degree of the
relationship. To understand how this coefficient is obtained we proceed as follows

1. First we compute the sign of the relationship. that will inform about the type

(a) Produce the series of data x̃i and ỹi . where

x̃i = (xi − X̄)

ỹi = (yi − Ȳ )

and notice that the sign of these differences with respect to the mean will
depend on where in the four regions defined in figure 3.3 the different pairs
of data are.
158 CHAPTER 3. GOODNESS-OF-FIT AND ...

Y Y

.. . .
.. . ..
... ...
.. ..
. .. ..
. ..
X X

Tipus A Tipus B

Y Y

. . .. . .
. . . . .. .
..
..
. . .. . . .
..
.
.. . . . ..
. .
X X

Tipus C Tipus D

Figure 3.1: Types of data dispersion

Figure 3.3: Regions in the dispersion graph

3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 159

Dispersion Type Type of Relationship between variables Comment

A Monotone increasing The two variables move in
the same direction. The
higher the value of one of
the variables. the higher the
value of the other
B Monotone Decreasing The two variables move in
opposite directions. The
higher the value of one of
the variables. the lower the
value of the other
C No relationship There is no relationship be-
tween the variables. For
some observations. high val-
ues in X coincide with high
values in Y . but in other
cases coincide with low val-
ues in Y
D Non-monotone Relationship There is a relationship be-
tween the variables. but in
some cases it is increasing
(like Type A) and in some
cases it is decreasing (like
Type B)

Figure 3.2: Types of relationship between the variables

Note that:
i. For points (pairs of data) in regions I and III we have x̃i y˜i > 0
ii. For points (pairs of data) in regions II and IV we have x̃i y˜i < 0
(b) Compute the sum of the product of all these differences
X
x̃i ỹi

For what we have seen in (a). we have that

P
i. x̃i ỹi > 0 means that there are more terms in the sum with x̃i y˜i > 0
than with x̃i y˜i < 0. That is. there are more points in regions I and
III than in regions II and IV. This shows that there exists a monotone
increasing relationship (Type A dispersion according to figure 3.1)
P
ii. x̃i ỹi < 0 means that there are more terms in the sum with x̃i y˜i < 0
than with x̃i y˜i > 0. That is. there are more points in regions II and
IV than in regions I and III. This shows that there exists a monotone
decreasing relationship (Type B dispersion according to figure 3.1)
P
iii. x̃i ỹi ≈ 0 means that there are. more or less. the same number of
terms with x̃i y˜i < 0 than with x̃i y˜i > 0. That is. there are point all
over the four regions. This might show that there is no relationship
(Type C dispersion according to figure 3.1) but also that there is a
non-monotone relationship (Type D dispersion according to figure 3.1)
160 CHAPTER 3. GOODNESS-OF-FIT AND ...

Table 3.4 illustrates these results.

Dispersion Type Type of Relationship between variables Plot

P
A ( x̃i ỹi > 0) Monotone Increasing

P
B ( x̃i ỹi < 0) Monotone Decreasing

P No relationship or
CoD( x̃i ỹi ≈ 0)
non-monotone relationship

P
Figure 3.4: Types of relationship according to x̃i ỹi

P
2. Hence. the sign of the x̃i ỹi identifies the type of the relationship. The value
of this sum. however. can not be taken as a measure of the degree of the relation-
ship. In this sense. we could have some data with a large number of observations
(n big) and
P another with a small number of observations (n small) and the the
value of x̃i ỹi would be large in the first case and small in the second; simply
because the sum has more elements. We could then reach the wrong conclusions
that the relationship is stronger in the first case than in the second
3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 161

P
To avoid this error we divide x̃i ỹi by the number of observations n
P
x̃i ỹi
n

3. Finally. there is one last issue that needs to be fixed. Consider two sets of data
consisting of exactly the observations of two variables with the only difference
is that one is measured in euros and the other in thousands of euros. Thus. the
values in the first data set are just the values in the second multiplied by a factor
of 1000. and it is clear that the “degree” of the relationship should
P be the same
in the two cases. Nevertheless. when we compute the sum x̃i ỹi . we will
find that its value will be larger for the first data set. simply because we are
summing values that are 1000 times larger. Again. we could then reach the
wrong conclusion that the relationship is stronger in the first case than in the
second. P
To avoid this error we can divide x̃ni ỹi by some factor that incorporates the
units of measurement of the data. In this sense. one uses the product of the
standard errors of the observations of the two variables. SX and SY :
P
x̃i ỹi
nSX SY

4. We finally obtain the expression for the correlation coefficient:

P
x̃i ỹi
r= (3.2)
nSX SY

where.

x̃i = (xi − X̄)

ỹi = (yi − Ȳ )
rP
x̃2i
SX =
n
rP
ỹi2
SY =
n

If we replace the values of SX and SY in (3.2) we have a simplified expression for r

P
x̃i ỹi
r = pP 2 pP 2 (3.3)
x̃i ỹi

One can show that −1 ≤ r ≤ 1. and its value can be interpreted as in the table in figure
3.5
The correlation coefficient r is an estimator of the population correlation coefficient ρ.
that measures the real correlation between two variables. In this sense. the same way
we have done with other parameters in previous chapters (like X̄ being an estimator of
162 CHAPTER 3. GOODNESS-OF-FIT AND ...

Value of r Interpretation
−1 ≤ r < 0 There is a monotone decreasing
relationship (Type B). The closer to -1r is.
the stronger the relationship
0<r≤1 There is a monotone increasing
relationship (Type A). The closer to 1r is.
the stronger the relationship
r≈0 A value of r close to 0 means that either
there is no relationship or that the
relationship is not monotone. We can not
tell whether it is of Type C or Type D

Figure 3.5: Interpretation ofr

the population mean µ). we can use r to do statistical inference regarding the value of
ρ (confidence intervals. hypothesis testing). To do so we need to know the probability
distribution of this estimator. In this sense. one can show that
r
n−2
t=r
1 − r2
follows a t − student distribution with n − 2 degrees of freedom.
This t statistic can be used to compute confidence intervals for ρ using the techniques
learned in chapter 1. or to conduct hypotheses tests like
H0 : ρ=0
H1 : ρ ̸= 0 ( or ρ > 0, or ρ < 0)
as seen in chapter 2.

Example 3.2.2 Let us consider the following set of data. that corresponds to the final
grade obtained in the midterm exam (variable X) and in final exam (variable Y ) by
the same class of 76 students of Statistics II.

1 2 3 4 5 6 7 8 9
MIDTERM_EXAM 7 8.5 3.25 1 6.75 5.25 5 10 4.5
FINAL_EXAM 8.21 7.16 6.33 0 6.3 6.21 6.17 9.21 6.32

10 11 12 13 14 15 16 17 18 19 20 21
8.5 6.25 9.75 4.25 5 6.75 1.75 2.5 3.75 5 8.25 3.5
8.66 8.68 9.68 7.53 6.63 6.67 3.53 6.77 2.35 5.9 7.01 5.76

22 23 24 25 26 27 28 29 30 31 32 33
9.25 2.5 8.5 5 7 5 9 3 7.5 6.75 3.8 3.25
7.44 2.53 9.01 6.44 8.24 7.46 8.28 4.75 8.64 8.06 6.48 5.95

34 35 36 37 38 39 40 41 42 43 44 45
5 2.25 9.5 2.5 3 6 9.75 6 5.5 6.25 4.5 8.55
7.73 0.72 9.52 5.05 4.46 8.47 9.63 7.59 7.36 7.68 6.12 9.09
3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 163

46 47 48 49 50 51 52 53 54 55 56 57
9.25 7.75 6.5 9.5 3 7.5 6 7.5 6.5 2.25 5.75 4.75
9.1 7.66 7.62 9.67 6.26 7.15 7.89 8.09 4.75 6.51 8.34 6.74

58 59 60 61 62 63 64 65 66 67 68 69
7 6.75 4.75 7 4 2.75 8 5.5 6.25 6.25 9.9 7.75
7.12 6.19 5.58 8.4 5.12 7.28 9.21 6.9 8.38 6.89 9.18 8.69

70 71 72 73 74 75 76
7.5 2 6 5 2.5 8.25 9.25
7.93 5.79 7.11 6.2 6.33 8.86 9.56

Graphically we have

To compute the correlation coefficient r:

1. X̄ and Ȳ .
7.0 + 8.5 + 3.25 + · · · + 9.25
X̄ = = 5.9
76
8.21 + 7.16 + 6.33 + · · · + 9.56
Ȳ = = 7.0
26

2. In the table below we have the computation of the sums

164 CHAPTER 3. GOODNESS-OF-FIT AND ...

xi yi X̄ Ȳ x̃i ỹi x̃2i ỹi2 x̃I y˜i

7 8.21 5.9 7.0 1.1 1.2 1.2 1.5 1.3

8.5 7.16 5.9 7.0 2.6 0.2 6.8 0.0 0.4
3.25 6.33 5.9 7.0 -2.7 -0.7 7.0 0.4 1.8
1 0 5.9 7.0 -4.9 -7.0 24.0 49.0 34.3
.. .. .. .. .. .. .. .. ..
. . . . . . . . .
5 6.2 5.9 7.0 -0.9 -0.8 0.8 0.6 0.7
2.5 6.33 5.9 7.0 -3.4 -0.7 11.6 0.4 2.3
8.25 8.86 5.9 7.0 2.4 1.9 5.5 3.5 4.4
9.25 9.56 5.9 7.0 3.4 2.6 11.2 6.6 8.6
P
−→ 0.00 0.00 413.13 275.94 263.28

3. Finally we compute the correlation coefficient

P
x̃i ỹi 263.28
r = pP 2 pP 2 = √ √ = 0.77978
x̃i ỹi 413.13 275.94
Thus. we find that there is a strong monotone increasing relationship between
the variables

RStudio Activity [Link] - Test of independence between quantitative variables

We will now do a similar exercise as in section 3.2.1, but using continuous variables in
this case.
The script is:

1. Generate a first sample (SAMPLE_1) with 500 observations form a Normal,

N (100, 25), distribution
2. Generate a sample (NOISE) of “white noise” (random noise) with 500 observa-
tions from a standard Normal, N (0, 1), distribution.
3. Generate a second sample (SAMPLE_2) by adding this “white noise” to the first
sample:

SAMPLE _2 = SAMPLE _1 + NOISE

Theoretically, this noise does not add any relevant information to the first sam-
ple. Therefore, we should expect a clear relationship between the two samples:

SAMPLE_1 and SAMPLE_2 are NOT expected to be independent

4. Carry out the test of independence (χ2 ) between the two samples
5. Experiment with the size of the “noise” to learn about its effect.
Let’s proceed.
3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 165

1. Generate a first sample (SAMPLE_1) with 500 observations form a Normal, N (100, 25),
distribution
[Link](2)

### Generate the first SAMPLE using a Normal distribution

SAMPLE_1 <- rnorm(500,100,5)

2. Generate a sample (NOISE) of “white noise” (random noise) with 500 observa-
tions from a standard Normal distribution.
### Generate the 'white noise' using a N(0,1)

NOISE <- rnorm(500,0,1)

3. Generate a second sample (SAMPLE_2) by adding this “white noise” to the first
sample
### Generate the second SAMPLE by adding the NOISE
### to SAMPLE 1
### Now we do not need to "round up" as we are in
### the case of continuous variables

SAMPLE_2 <- SAMPLE_1 + NOISE

We can now take a quick look at the possible relationship between the variables
by means of a scatter plot.
plot(SAMPLE_1,SAMPLE_2, pch=20,cex=0.5)
110
SAMPLE_2

100
90

90 95 100 105 110 115

SAMPLE_1
166 CHAPTER 3. GOODNESS-OF-FIT AND ...

Visually, the relationship seems quite clear. If it weren’t for the "noise" the graph
of the relationship would be a perfect diagonal, and we note that we are not too
far from that situation.

4. Carry out the test of independence (χ2 ) between the two samples
The command for this specific test is [Link] (for “correlation test”):

TestIndependenceContinuous <- [Link](SAMPLE_1,SAMPLE_2)

TestIndependenceContinuous

##
## Pearson's product-moment correlation
##
## data: SAMPLE_1 and SAMPLE_2
## t = 114.64, df = 498, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.9780715 0.9845227
## sample estimates:
## cor
## 0.9815747

The p_value is extremely low in this case, lower than 2.2e-16, clearly below
any conceivable value of α. Hence,

p_value < α ⇒ The null hypothesis IS rejected

In other words, we are rejecting the null hypothesis of independence and thus
concluding that there exists a relationship between the [Link] a matter of
fact, in the information provided by the test we can also find the value of the
Pearson’s correlation coefficient (cor) r = 0.9815747 (and also a 95% confi-
dence interval).
Therefore, as in the case of discrete variables, adding a small amount of “noise”
to one sample generates another sample very related to the original one.

5. Experiment with the size of the “noise” to learn about its effect.
Now we will generate NOISE using a N (0, 100) distribution (100 times more
variance than before).

[Link](2)

### Generate the first SAMPLE using a Normal distribution

SAMPLE_1 <- rnorm(500,100,5)

### Generate noise, which will not be so 'white' anymore

NOISE <- rnorm(500,0,10)

### Generate the second SAMPLE by adding the NOISE #

## to SAMPLE 1
### Now we do not need to "round up" as we are in
### the case of continuous variables

SAMPLE_2 <- SAMPLE_1 + NOISE

3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 167

Repeat the test.

TestIndependenceContinuous <- [Link](SAMPLE_1,SAMPLE_2)

TestIndependenceContinuous

##
## Pearson's product-moment correlation
##
## data: SAMPLE_1 and SAMPLE_2
## t = 10.709, df = 498, p-value < 2.2e-16
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## 0.3585471 0.5013071
## sample estimates:
## cor
## 0.4326352

The p_value is still very low. Thus,

p_value < α ⇒ The null hypothesis IS rejected

That is, despite the considerable increase in noise (100 times!) the hypothesis
of independence continues to be rejected. The reason is that, remember, this
technique is testing:

H0 : ρ = 0
H1 : ρ ̸= 0 (or ρ > 0, or ρ < 0)

Therefore, the correlation coefficient r needs to be really close to zero for the test
to conclude that the two variables are independent (ρ = 0), and now the value
r = 0.4326352 still is not close enough to zero.

The graphical representation now shows:

plot(SAMPLE_1,SAMPLE_2, pch=20,cex=0.5)
168 CHAPTER 3. GOODNESS-OF-FIT AND ...

130
SAMPLE_2

110
90
70

90 95 100 105 110 115

SAMPLE_1

We observe that, although a relationship can still be spotted, it is clearly weaker

than before.
To wrap up this exercise, let us now increase the standard deviation up to 50,
creating a much more noticeable “noise”. Now we have:
[Link](2)

### Generate the first SAMPLE using a Normal distribution

SAMPLE_1 <- rnorm(500,100,5)

### Generate a huge noise

NOISE <- rnorm(500,0,50)

### Generate the second SAMPLE by adding the NOISE

### to SAMPLE 1
### Now we do not need to "round up" as we are in
### the case of continuous variable

SAMPLE_2 <- SAMPLE_1 + NOISE

Let’s repeat the test

TestIndependenceContinuous <- [Link](SAMPLE_1,SAMPLE_2)
TestIndependenceContinuous

##
## Pearson's product-moment correlation
##
## data: SAMPLE_1 and SAMPLE_2
## t = 1.4706, df = 498, p-value = 0.142
## alternative hypothesis: true correlation is not equal to 0
3.2. ANALYSIS OF THE RELATIONSHIP BETWEEN VARIABLES 169

## 95 percent confidence interval:

## -0.02206034 0.15256829
## sample estimates:
## cor
## 0.06575745

Now the p_value is high enough not to reject the null hypothesis of indepen-
dence. Accordingly, the correlation coefficient is rather low, r = 0.06575745
Finally, the graphical representation shows that now it is very difficult to identify
any relationship between the two samples.
plot(SAMPLE_1,SAMPLE_2, pch=20,cex=0.5)
200
SAMPLE_2

100
0

90 95 100 105 110 115

SAMPLE_1

End of Activity RStudio [Link]

170 CHAPTER 3. GOODNESS-OF-FIT AND ...
Chapter 4

Introduction to the regression

model

Numbers are swirling in my head

I regret regression
But I have to graph instead
Of a gossip session.

XMXRGXNCY

4.1 Objective of the model

We want to study what is the relation that exists between two given variables X and
Y . For instance, if we plot data corresponding to the Gross Domestic Product in the
European Union between 1991 and 2003 against the Domestic Demand in the same
period we obtain the following graph:

171
172 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Figure 4.1: GDP and Domestic Demand data set

which (as seen in the Correlation Analysis topic) seems to indicate a strong monotone
positive relationship between X (Domestic Demand) and Y (Gross Domestic Product):
the higher the Domestic Demand, the higher the Gross Domestic Product.
In fact, the graph suggests that the relationship follows a straight line. That is, it “looks
like” if there was a line that goes through the points in the graph. The following graph
depicts such line:

Figure 4.2: GDP and Domestic Demand data set with fitted line
4.2. HYPOTHESIS OF THE MODEL SPECIFICATION 173

The main “goal” of the Regression Analysis i to obtain the equation of such line using
a set of data with observations of the variables {(x1 , y1 ), (x2 , y2 ), . . . , (xn , yn )}, and
to study its properties.
This equation will reflect the relationship that exists between the two variables, and
will be something like
Y = β1 + β2 X
where β1 is the vertical intercept and β2 is the slope of the line.
A relationship like the one described above is often referred to as a casual relationship
in the sense that the behavior of one of the variables explains the behavior of the other
variable. One says that the exogenous (or independent) variable (X in this case) causes
(or explains) the behavior of the endogenous (or dependent) variable (Y here).

4.2 Hypothesis of the model specification

Usually, though, this relationship is not perfect. In many cases the X variable can not
explain, alone, the behavior of the Y variable in full. As we can see in the figure 4.2,
the line can not pass through all the points that link X and Y . Even in the best case
there is always an error, a small difference between the real data and the line. This is
due, among other things, to missing variables that could explain part of the behavior of
Y and that we have not taken into account. The regression analysis technique assumes
that these errors are small and random, that is, that they are not very important and can
not be predicted1 . This is formalized by means of an error term (u) in the equation:

Y = β1 + β2 X + u

Thus, we have that the model of the simple regression analysis has two components,

1. β1 + β2 X → Part of the behavior of Y explained by X (Explained Variation)

2. u → Part of the behavior of Y not explained by X (Not Explained Variation)

The above defines the so called T HEORETICAL M ODEL [TM]

[TM] Yi = β1 + β2 Xi + ui (4.1)

where the subindexes i refer to the ith observation of the variables.

The basic assumption that the errors are small and random formalizes as:

1. ui ∼ N (0, σu2 ). That is, for each observation Xi , the error ui is a random
variable that follows a Normal distribution with E(ui ) = 0 (we expect the error
to be small) and V (ui ) = σu2 (all errors have the same variance). This last
assumption (equal variance) is called homoskedasticity. If this assumption is not
satisfied the model is said to be heterokedastic.
1 It they were important, they would invalidate the analysis. If they were not random, that is, could be

predicted, we should incorporate then in the analysis.

174 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

2. E(ui · uj ) = 0 when i ̸= j, that is, there is no relationship between errors of

different observations

3. The variable X is deterministic, its values are not random

4. E(Xi · ui ) = 0. There is no relationship between X and the errors u

5. The unknown parameters of the model, β1 , β2 i σu2 , are constant.

From these assumptions follows that Y is a random variable with a Normal distribution
characterized by

E(Yi ) = E(β1 + β2 Xi + ui ) = β1 + β2 Xi + E(ui ) = β1 + β2 Xi

V (Yi ) = V (β1 + β2 Xi + ui ) = 0 + 0 + V (ui ) = σu2

From here, the simple regression analysis proceeds by estimating the unknown param-
eters of the model, β1 , β2 and σu2 , using the data set at hand

{(x1 .y1 ), (x2 , y2 ), . . . , (xi , yi ), . . . , (xn , yn )}

To do so, one specifies the so called E STIMATED M ODEL [EM] as follows

[EM] Ŷi = β̂1 + β̂2 Xi (4.2)

Such estimated model is the equation of the line fitted to the data points in figure 4.2.

In this model,

• Ŷi is the estimate of the value of the Y variable that corresponds to the observa-
tion of Xi

• β̂1 is the estimate of the population parameter β1 . In the line, it corresponds to

the intercept

• β̂2 is the estimate of the population parameter β2 . In the line, it corresponds to

the slope

It is clear that for each value Xi there is a difference between the real value of Y (yi )
and the corresponding value according to the estimated line (ŷi ). This difference is the
residual (or error) of the regression and is denoted by ûi :

ûi = yi − ŷi (4.3)

4.3. ESTIMATION BY ORDINARY LEAST SQUARES 175

Figure 4.3: Fitted line and residuals

The objective is to find the line that better fits the data. That is, we have to find the E S -
TIMATED M ODEL that better approximates the T HEORETICAL M ODEL ; the estimates
β̂1 and β̂2 that make the residuals as small as possible (residual minimization)

4.3 Estimation by Ordinary Least Squares (OLS) and

its properties

In figure 4.3 we observe that some residuals are positive whereas others are negative.
Indeed, in some cases we have yi > ŷi while in other cases yi < ŷi . Hence, we should
not try to find the line that minimizes the sum of all these residuals. Positive errors
(ûi > 0) would cancel out with negative errors (ûi < 0), and we might end up with
an Estimated Model that, although having many errors, the sum of all of them would
result in a “total error” equal (or near) to zero.
176 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Figure 4.4: Positive and negative errors cancel out

The method of the Ordinary Least Squares (OLS) operates in this direction, but min-
imizing the sum of the errors squared. By doing so, the problem of the positive and
negative errors is eliminated: all the errors count as “positive”.
Therefore, we must look for the values of β̂1 and β̂2 that minimize û2i . Notice that,
P
from the definition of these errors in equation (4.3) and from the specification of the
estimated model (4.2), we can rewrite this sum as
n
X n
X n
X
û2i = (yi − ŷi )2 = (yi − (β̂1 + β̂2 xi ))2 (4.4)
i=1 i=1 i=1

4.3.1 Data in differences with respect to the mean

For convenience, we will often transform the original data set as follows.
Let X and Y be two variables of which the have the following observations:
{(x1 .y1 ), (x2 , y2 ), . . . , (xi , yi ), . . . , (xn , yn )}
From this data set we can compute the means (all sums are from i = 1 to n):
P P
xi yi
X̄ = , Ȳ =
n n
We finally produce a new data set “in differences”, that consists of a transformation of
the original data set by computing the differences with respect to the mean, that is,
x̃i = (xi − X̄)
ỹi = (yi − Ȳ )
The following sums will be used later on (again, all the sums Σ are from i = 1 to n):
X X X
x̃2i = (xi − X̄)2 = x2i − nX̄ 2 (4.5)
X X X
ỹi2 = (yi − Ȳ )2 = yi2 − nȲ 2 (4.6)
X X X
x̃i ỹi = (xi − X̄)(yi − Ȳ ) = xi yi − nX̄ Ȳ (4.7)
4.3. ESTIMATION BY ORDINARY LEAST SQUARES 177

4.3.2 The OLS estimators

We will now obtain the estimators β̂1 i β̂2 that minimize the sum of the squared errors
n
X n
X n
X
û2i = (yi − ŷi )2 = (yi − (β̂1 + β̂2 xi ))2
i=1 i=1 i=1

To do so we must take the derivatives of û2i with respect to β̂1 i β̂2 and make them
P
equal to zero. We will then obtain two equations, named the normal equations, that we
must solve for β̂1 and β̂2
P 2 
∂ ûi P
∂ β̂1
= 2 (y i − (β̂ 1 + β̂ 2 x i ))(−1) = 0 


P 2
=⇒
∂ ûi P 
= 2 (yi − (β̂1 + β̂2 xi ))(−xi ) = 0


∂ β̂
2

(yi − (βˆ1 + β̂2 xi )) = 0 

P  P P 
yi = nβ̂1 + β̂2 xi 
=⇒ =⇒
x2i
P P P P
(yi − (β̂1 + β̂2 xi ))xi = 0
 
yi xi = β̂1 xi + β̂2

Ȳ = β̂1 + β̂2 X̄ 

x2i
P P 
yi xi = β̂1 nX̄ + β̂2

Solving for β̂1 in the first equation we get

β̂1 = Ȳ − β̂2 X̄ (4.8)

and replacing this in the second equation,

X X
yi xi = nȲ X̄ − nβ̂2 X̄ 2 + β̂2 x2i

Reordering we have
X X
yi xi − nȲ X̄ = β̂2 ( x2i − nX̄ 2 )

and,finally, P
yi xi − nȲ X̄
β̂2 = P 2
xi − nX̄ 2

Using the expressions (4.7) and (4.5) we can rewrite this as

P
x̃i ỹi
β̂2 = P 2 (4.9)
x̃i
178 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

4.3.3 Estimation of the variance of the error term

Finally, to complete the estimation of all the parameters of the model, we must obtain
an expression for the variance of the error term, σu2 .
We will obtain such estimation from the variance of the errors (residuals) that we ob-
serve in the regression line obtained by OLS.2
P ¯2
(ûi − û)
σ̂u2 = Sû2 =
n−2

Notice now that from the first normal equation follows that
P ¯ = 0.
ûi = 0. Hence, û
Thus, from (4.4) we can compute

û2i (yi − (β̂1 + β̂2 xi ))2

P P
σ̂u2 = =
n−2 n−2

Replacing β̂1 by its OLS expression (4.8) and reordering we get

(yi − (Ȳ − β̂2 X̄ + β̂2 xi ))2 ((yi − Ȳ ) − β̂2 (xi − X̄))2

P P
σ̂u2 = = =
n−2 n−2
(ỹi − β̂2 x̃i )2 (ỹi2 − 2β̂2 ỹi x̃i + β̂22 x̃2i ) ỹi2 − 2β̂2 ỹi x̃i + β̂22 x̃2i
P P P P P
= = =
n−2 n−2 n−2

x̃2i . Therefore,
P P
Using now the OLS expression for β̂2 (4.9) we find that ỹi x̃i = β̂2

ỹi2 − 2β̂22 x̃2i + β̂22 x̃2i

P P P
σ̂u2 =
n−2

Thus, we finally have

ỹi2 − β̂22 x̃2i

P P
σ̂u2 = (4.10)
n−2

4.3.4 Properties of the estimators OLS

Once the expressions for the OLS estimators of β1 and β2 are obtained,
P
x̃i ỹi
β̂1 = Ȳ − β̂2 X̄; β̂2 = P 2
x̃i

we will study their main properties

2 Remember that to have and unbiased estimator of the variance we must divide by the degrees of freedom.

In this case we have n − 2 degrees of freedom since we work with two equations
4.3. ESTIMATION BY ORDINARY LEAST SQUARES 179

[Link] Properties of β̂2

1. Linearity. Notice that β̂2 can be written as a linear combination of the observed
values of the Y variable. Indeed, if we define3 ci as
x̃i
ci = P 2
x̃i
we can write X
β̂2 = ci ỹi

2. Normality. From the linearity property follows that β̂2 is the sum of linear trans-
formations of Normal random variables (the Yi ). Thus, β̂2 is a Normal random
variable as well.
3. Unbiased. Let us compute the expectation of β̂2
X X
E(β̂2 ) = ci E(ỹi ) = ci E(Yi − Ȳ ) =
P
X ui
= ci E(β1 + β2 Xi + ui − (β1 + β2 X̄ + )) =
n
x̃2
X X P
= ci β2 (Xi − X̄) = β2 ci x̃i = β2 P i2 = β2
x̃i
Therefore,
E(β̂2 ) = β2

4. Efficiency. Let us compute the variance of β̂2

X X X X
V (β̂2 ) = V ( ci ỹi ) = c2i V (Yi − Ȳ ) = c2i V (Yi ) = c2i V (ui ) =
P 2
X x̃ 1
= σu2 c2i = σu2 P i 2 = σu2 P 2
[ x̃i ] x̃i
Therefore,
1
V (β̂2 ) = σu2 P 2
x̃i
It can be shown that from all the linear, unbiased estimators of β2 , this is the
minimum variance

We can thus write

1
β̂2 ∼ N (β2 , σu2 P 2 )
x̃i

[Link] Properties of β̂1

Since β̂1 is a linear transformation of β̂2

β̂1 = Ȳ − β̂2 X̄

β̂1 will inherit the same properties as β̂2

3 Notice that these coefficients only depend on the observed values of X. Therefore, they are fixed (not

random) values in the model .

180 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

1. Linearity. Like β̂2 , we have that β̂1 can be written as a lineal combination of the
values of Y
2. Normality. From the linearity property follows that β̂_1 is the sum of linear
transformations of Normal random variables (the Yi ). Thus, β̂1 is a Normal
random variable as well
3. Unbiased. Let us compute the expectation of β̂1
P P
Yi Xi
E(β̂1 ) = E(Ȳ − β̂2 X̄) = E( − β̂2 )=
P n P n
(β1 + β2 Xi + ui ) Xi
= E( − β̂2 )=
P n P n
(β1 + β2 Xi ) Xi
= − β2 =
n P Pn
nβ1 Xi Xi
= + β2 − β2 = β1
n n n
Therefore,
E(β̂1 ) = β1

4. Efficiency. Let us compute the variance of β̂1

P
Yi
V (β̂1 ) = V (Ȳ − β̂2 X̄) = V (Ȳ ) + X̄ 2 V (β̂2 ) = V ( ) + X̄ 2 V (β̂2 ) =
n
1 X 1 1 X 2 1
= 2
V (Yi ) + X̄ 2 (σu2 P 2 ) = 2 σu + X̄ 2 (σu2 P 2 ) =
n x̃i n x̃i
2
n 2 1 1 X̄
= σ + X̄ 2 (σu2 P 2 ) = σu2 ( + P 2 )
n2 u x̃i n x̃i
Therefore,
1 X̄ 2
V (β̂1 ) = σu2 ( + P 2)
n x̃i
Again, it can be shown that from all the linear, unbiased estimators of β1 , this is
the minimum variance

We can thus write

1 X̄ 2
β̂1 ∼ N (β1 , σu2 ( + P 2 ))
n x̃i

4.4 Model testing

The expressions for β̂1 and β̂2 that we have found are estimators of the “true” popu-
lation parameters β1 and β2 of the theoretical model. Thus, we can use β̂1 and β̂2 to
construct confidence intervals for the true values of β1 and β2 , or to test hypothesis. To
do so we will use the statistics below and their corresponding probability distributions,
which are based on the distributions we have found for β̂1 and β̂2 .
β̂1 − β1
∼ tn−2 (4.11)
Sβ̂1
4.4. MODEL TESTING 181

β̂2 − β2
∼ tn−2 (4.12)
Sβ̂2
where s
q 1 X̄ 2
Sβ̂1 = Sβ̂2 = σ̂u2 ( + P 2) (4.13)
1 n x̃i
s
q 1
Sβ̂2 = Sβ̂2 = σ̂u2 P 2 (4.14)
2 x̃i

4.4.1 Confidence intervals for β1 and β2

Using the statistics (4.11) and (4.12) and the techniques we have learned in section
??, we find the confidence intervals for β1 and β2 below (for a given confidence level
1 − α):
β1 ∈ [β̂1 − t1− α2 Sβ̂1 , β̂1 + t1− α2 Sβ̂1 ] (4.15)

β2 ∈ [β̂2 − t1− α2 Sβ̂2 , β̂2 + t1− α2 Sβ̂2 ] (4.16)

4.4.2 Hypothesis testing for β1 and β2

1. Testing for β1 : To test the null hypothesis H0 : β1 = β10 against the alternative
H1 : β1 ̸= β10 we will use the test statistic

βˆ1 − β10
t(β1 ) =
Sβ̂1

which, under the null hypothesis, follows a t-student distribution with n − 2

degrees of freedom. Hence, as we have seen in Chapter 2,

|t(β1 )| < t1− α2 =⇒ We do not reject H0 : β1 = β10

|t(β1 )| > t1− α2 =⇒ We reject H0 : β1 = β10

2. Testing for β2 : To test the null hypothesis H0 : β2 = β20 against the alternative
H1 : β2 ̸= β20 we will use the test statistic

β̂2 − β20
t(β2 ) =
Sβ̂2

which, under the null hypothesis, follows a t-student distribution with n − 2

degrees of freedom. Hence, as we have seen in Chapter 2,

|t(β2 )| < t1− α2 =⇒ We do not reject H0 : β2 = β20

|t(β2 )| > t1− α2 =⇒ We reject H0 : β2 = β20

In this case, the following test is of special interest

H0 : β 2 = 0
182 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

H1 : β2 ̸= 0
Indeed, if we found that the hypothesis β2 = 0 is not rejected, we would have
that the theoretical model simplifies to

Yi = β1 + 0·Xi + ui

that is,
Yi = β1 + ui
and this would mean that there is no relationship between X and Y , that the
behavior of Y would be explained by the error term ui only, thus invalidating
our model.

4.5 Coefficient of the goodness-of-fit and relationship

between the correlation and the regression analysis

4.5.1 The Coefficient of determination (or goodness-of-fit)

Once we have the values of β̂1 i β̂2 that define the line that fits “better” to the data, we
want to know how good the fit is. We do so by means of the Coefficient of Determina-
tion.
The coefficient of determination (R2 ) measures what percentage of the behavior of
Y is explained by the estimated model. Therefore, the remaining percentage (1-R2 )
corresponds to the errors of the model, the behavior of Y that is unexplained
We want to explain the variation of Y with respect to its mean Ȳ , (Yi − Ȳ ). In order
to avoid positive and negative variations canceling each other, these deviations are
squared, (Yi − Ȳ )2 . Finally, to have a measure of the Total Variation, we sum all of
them: X
TV = (Yi − Ȳ )2 (4.17)
Expression (4.17) can be rewritten as
X
TV = ((Yi − Ŷi ) + (Ŷi − Ȳ ))2

where (Ŷi − Ȳ ) is, as we have seen, the behavior of Y that we can explain with the
estimated model, whereas (Yi − Ŷi ) are the errors ûi of the regression. Developing this
expression we have:
X
VT = ((yi − ŷi )2 + 2(yi − ŷi )(ŷi − Ȳ ) + (ŷi − Ȳ )2 ) =
X X X
= (yi − ŷi )2 + 2 (yi − ŷi )(ŷi − Ȳ ) + (ŷi − Ȳ )2 (4.18)
X X X
= û2i + 2 (yi − ŷi )(ŷi − Ȳ ) + (ŷi − Ȳ )2

From here, since

(Yi − Ŷi ) = ûi
and
(ŷi − Ȳ ) = (β̂1 + β̂2 xi ) − (β̂1 + β̂2 X̄) = β̂2 (xi − X̄) (4.19)
4.5. COEFFICIENT OF THE GOODNESS-OF-FIT 183

we can express the central term in the sum of (4.18) as

X X X X
2 (yi − ŷi )(ŷi − Ȳ ) = 2 ûi β̂2 (xi − X̄) = 2β̂2 ûi xi − 2β̂2 X̄ ûi

which equals zero since from the normal equations we have that

X
ûi = 0
X
ûi xi = 0

Therefore, we have that (4.18) summarizes to

X X
TV = û2i + (Ŷi − Ȳ )2

We finally note that û2i is the sum of the errors, that is, the “Not Explained Variation”
P

(NEV), whereas (Ŷi − Ȳi )2 is the “Explained Variation” (EV). We thus have
P

X X X
(Yi − Ȳ )2 = û2i + (Ŷi − Ȳ )2

or, equivalently,
T V = N EV + EV (4.20)

The coefficient of determination R2 computes what percentage of the Total Variation

is explained by the model, that is, what is the weight of the Explained Variation (EV)
in the Total Variation (TV):

(Ŷi − Ȳ )2
P
EV
R2 = =P
TV (Yi − Ȳ )2

From (4.19) we can write R2 as

P 2
β̂2 (xi − X̄)2 β̂22 (xi − X̄)2
P
2
R = P = P
(yi − Ȳ )2 (yi − Ȳ )2
Finally, using the data in differences we have a simpler expression

β̂22 x̃2i
P
R2 = P 2 (4.21)
ỹi

Notice that
0 ≤ R2 ≤ 1

Indeed,

1. R2 will always be non negative since we compute it using quadratic forms

2. R2 will always be less or equal to one since we compute it as a percentage

A value R2 = 0 corresponds to the non existence of a linear relationship between X

and Y whereas a value R2 = 1 shows a perfect linear relationship (100%)
184 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

4.5.2 Relationship between the correlation analysis and the regres-

sion analysis

From the data corresponding to the variables Y and X

{(x1 .y1 ), (x2 , y2 ), . . . , (xi , yi ), . . . , (xn , yn )}

we can compute the correlation coefficient r as we have seen in section3.2.2. There we

found the formula (3.3) P
x̃i ỹi
r = pP 2 pP 2
x̃i ỹi

If we square this formula we have

( x̃i ỹi )2
P
r2 = P 2 P 2
x̃i ỹi

Now, from the expression OLS (4.9) of β̂2

P
x̃i ỹi
β̂2 = P 2
x̃i

we have X X X X
x̃i ỹi = β̂2 x̃2i =⇒ ( x̃i ỹi )2 = β̂22 ( x̃2i )2

Replacing this into the formula for r2 we have

( x̃i ỹi )2 β̂ 2 ( x̃2 )2 β̂ 2 x̃2

P P P
r2 = P 2 P 2 = P2 2 Pi 2 = 2P 2 i
x̃i ỹi x̃i ỹi ỹi

that coincides with the formula (4.21) that we have for the coefficient of determination
R2 . Therefore, we have
R2 = r 2
That is, the coefficient of determination R2 found in the regression analysis coincides
correlation coefficient squared developed in the correlation analysis. Therefore, the
regression analysis embeds the correlation analysis, adding extra details in the study of
causality.

4.6 Forecasting
Once the regression model is estimated, it is customary to use it to produce predictions
about future values of the dependent variable Y using information about future values
of the independent variable X. In this sense, one forms a expectation about the value
of Y “conditional” upon a specific value of X.
Formally, if usingn pairs of observations of the variables X and Y , {(x1 , y1 ), (x2 , y2 ), . . . , (xn , yn )},
we have estimated the model
ŷi = β̂1 + β̂2 xi
4.6. FORECASTING 185

and now we have information about a future value of the variable X, (xn+1 ), we can
compute and expectation (prediction or forecast)) of the corresponding value (yn+1 ) of
the variable Y given this know value of X. That is to say,
E(Yn+1 /xn+1 )

Since the Theoretical Model [TM] says:

Yi = β1 + β2 xi + ui

computing the conditional expectation we obtain:

E(Yn+1 /xn+1 ) = E(β1 + β2 xn+1 + un+1 ) =
= E(β1 ) + E(β2 xn+1 ) + E(un+1 ) =
= β1 + β2 xn+1

given that, by assumption, E(un+1 ) = 0. Therefore:

E(Yn+1 /xn+1 ) = β1 + β2 xn+1

From here we can obtain estimations of future values of the Y variable in two different
ways: by point estimation or by interval estimation.

4.6.1 Point estimation of Yn+1

This is the simplest approximation to future values of the dependent variable Y . Indeed,
if we have the estimated model [EM]
ŷi = β̂1 + β̂2 xi

we can obtain a point estimation (ŷn+1 ) of the value of Y once xn+1 is known directly
from the estimated model:
ŷn+1 = β̂1 + β̂2 xn+1

This estimation, though, is very imprecise since, as explained in section 1.6, we do not
know if the actual value of Y will be close (or not) to such point estimation. In this
sense, an interval estimation will be more informative.

4.6.2 Estimation by intervals of Yn+1

As seen in section ??, one can show that a 1 − α confidence interval for the value of
Yn+1 is given by
Yn+1 ∈ [ŷn+1 − t1− α2 Sŷn+1 , ŷn+1 + t1− α2 Sŷn+1 ]

where t1− α2 is the value of a t − student with n − 2 degrees of freedom that has at its
left a probability of 1 − α2 , and the standard deviation, Sŷn+1 , of the estimator ŷn+1 is
given by: s 
(xn+1 − X̄)2

2
1
Sŷn+1 = σ̂u + P 2
n x̃i

The example below shows most of the tasks contained in a typical regression analysis
186 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

4.6.3 Estimation of the effect on Y of changes in X

Besides computing estimations of future values of the dependent variable Y , as seen
above, one might also be interested in knowing about the changes in this variable driven
by changes in the independent variable X. In this sense, if we consider the theoretical
model

[TM] Yi = β1 + β2 Xi + ui
dYi
and compute the change in Yi due to a change in Xi , that is, dXi , we have:

dYi
= β2
dXi
Hence, the coefficient β2 measures the effect on the dependent variable of changes in
the independent variable.
We can therefore use its OLS estimator, β̂2 , as an estimation of such effect. In this way,
we can say that if X varies in δ units the estimation of the corresponding change in Y
will be β̂2 · δ

dYi = β̂2 · δ

Notice that, since we are using an estimation of such effect, this result must be inter-
preted as the expected change (that is, on average) in the Y variable due to the change
in the X variable.

Example 4.6.1 Recall the example 3.2.2 in the previous chapter, where we had a data
set corresponding to the final grade obtained in the midterm exam (variable X) and in
final exam (variable Y ) by the same class of 76 students of Statistics II

1 2 3 4 5 6 7 8 9
MIDTERM_EXAM 7 8.5 3.25 1 6.75 5.25 5 10 4.5
FINAL_EXAM 8.21 7.16 6.33 0 6.3 6.21 6.17 9.21 6.32

10 11 12 13 14 15 16 17 18 19 20 21
8.5 6.25 9.75 4.25 5 6.75 1.75 2.5 3.75 5 8.25 3.5
8.66 8.68 9.68 7.53 6.63 6.67 3.53 6.77 2.35 5.9 7.01 5.76

22 23 24 25 26 27 28 29 30 31 32 33
9.25 2.5 8.5 5 7 5 9 3 7.5 6.75 3.8 3.25
7.44 2.53 9.01 6.44 8.24 7.46 8.28 4.75 8.64 8.06 6.48 5.95

34 35 36 37 38 39 40 41 42 43 44 45
5 2.25 9.5 2.5 3 6 9.75 6 5.5 6.25 4.5 8.55
7.73 0.72 9.52 5.05 4.46 8.47 9.63 7.59 7.36 7.68 6.12 9.09

46 47 48 49 50 51 52 53 54 55 56 57
9.25 7.75 6.5 9.5 3 7.5 6 7.5 6.5 2.25 5.75 4.75
9.1 7.66 7.62 9.67 6.26 7.15 7.89 8.09 4.75 6.51 8.34 6.74
4.6. FORECASTING 187

58 59 60 61 62 63 64 65 66 67 68 69
7 6.75 4.75 7 4 2.75 8 5.5 6.25 6.25 9.9 7.75
7.12 6.19 5.58 8.4 5.12 7.28 9.21 6.9 8.38 6.89 9.18 8.69

70 71 72 73 74 75 76
7.5 2 6 5 2.5 8.25 9.25
7.93 5.79 7.11 6.2 6.33 8.86 9.56

that produced the following computations

X̄ = 5.9
Ȳ = 7.0
X
x̃i ỹi = 263.28
X
x̃2i = 413.13
X
ỹi2 = 275.94

We can now compute the values for the OLS estimators of β̂1 and β̂2 . To do so we use
the formula (4.8) and (4.9)

β̂1 = Ȳ − β̂2 X̄

P
x̃i ỹi
β̂2 = P 2
x̃i

In this case we get:

263.28
β̂2 = = 0.637
413.13
β̂1 = 7.0 − (0.637)5.9 = 3.242

Thus, we obtain the E STIMATED M ODEL

Ŷi = 3.242 + 0.637Xi

Graphically,
188 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Let us compute now the coefficient of determination R2 using the formula (4.21)

β̂ 2 x̃2 (0.637)2 413.13

P
R2 = 2P 2 i = = 0.6075
ỹi 275.94

Therefore, the E STIMATED M ODEL explains 60.75% of the behavior of the Y variable.
We will construct now confidence intervals for β1 and β2 with a confidence level of
95% (1 − α) = 0.95. We use expressions (4.15) and (4.16):

β1 ∈ [β̂1 − t1− α2 Sβ̂1 , β̂1 + t1− α2 Sβ̂1 ]

β2 ∈ [β̂2 − t1− α2 Sβ̂2 , β̂2 + t1− α2 Sβ̂2 ]

We must use the tables of a t-student distribution with (n − 2) = 74 degrees of freedom
to look up for the value

t1− α2 = t1− 0.05

2
= t1−0.025 = t0.975 = 1.992

Next, we compute Sβ̂1 i Sβ̂2 using formulae(4.13) and (4.14)

s
q 1 X̄ 2
Sβ̂1 = Sβ2ˆ = σ̂u2 ( + P 2)
1 n x̃i
s
q 1
Sβ̂2 = Sβ̂2 = σ̂u2 P 2
2 x̃i

To do so we first need to compute σ̂u2 using (4.10)

ỹi2 − β̂22 x̃2i 275.94 − (0.637)2 413.13

P P
σ̂u2 = = = 1.463
n−2 74
4.6. FORECASTING 189

Thus, s r
1 X̄ 2 1 (5.9)2
Sβ̂1 = σ̂u2 (
+ P 2 ) = 1.463( + ) = 0.3775
n x̃i 76 413.13
s r
2
1 1
Sβ̂2 = σ̂u P 2 = 1.463( ) = 0.0595
x̃i 413.13
We can, finally, compute the intervals

β1 ∈ [3.242 − (1.992)(0.3775), 3.242 + (1.992)(0.3775)] = [2.490, 3.994]

β2 ∈ [0.637 − (1.992)(0.0595), 0.637 + (1.992)(0.0595)] = [0.518, 0.755]

To conclude, we test the hypothesis β2 = 0. It is customary to conduct this test to check
the relationship between X i Y as we have discussed at the end of section 4.4.2. Since
the value of R2 is rather high, we can expect that such hypothesis will be rejected
Formally, the test is
H0 : β2 = 0
H1 : β2 ̸= 0
The test statistic is
β̂2
t(β2 ) =
Sβ̂2
which, as we have seen, follows a t−student distribution with(n − 2) = 74 degrees of
freedom. When computing the confidence intervals we have already used this value:

t0.975 = 1.992

The observed value of the test statistic in this case is

0.637
t(β) = = 10.706
0.0595

Clearly, since
|t(β)| > t1− α2 =⇒ We reject H0 : β2 = 0
and we can say that X and Y are related
Let us assume now that we have information saying that for another student the vari-
able X(grade in the midterm exam) takes the value x77 = 7.1. What forecast can we
make about the corresponding value (y77 ) of the endogenous variable Y (grade in the
final exam)?
A point estimation can be easily computed using the estimated model found above

Ŷi = 3.242 + 0.637Xi

Thus, if x77 = 7.1 we have

ŷ77 = 3.242 + 0.637x77 =

= 3.242 + 0.637 · 7.1 = 7.7647
190 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Therefore, if x77 = 7.1 we can forecast that the variable Y will take the value ŷ77 =
7.7647 (approx.).
If we need more information about this forecast we may construct the corresponding
confidence interval. Thus, with a confidence level (1 − α) = 0.95 we have

Y77 ∈ [ŷ77 − t1− α2 Sŷ77 , ŷ77 + t1− α2 Sŷ77 ]

We have to look up the tables of a t-student with (n − 2) = 74 degrees of freedom for

the value
t1− α2 = t1− 0.05
2
= t1−0.025 = t0.975 = 1.992

We also need to compute Sŷ77

s
(x77 − X̄)2
 
1
Sŷ77 = σ̂u2 + P 2 =
n x̃i
s
(7.1 − 5.9)2
 
1
= 1.463 + = 0.156
76 413.13

Thus, we finally get:

Y77 ∈ [7.7647 − 1.992 · 0.156, 7.7647 + 1.992 · 0.156]

Y77 ∈ [7.454, 8.075]

That is, the value Y77 that corresponds to x77 = 7.1 will be between 7.454 and 8.075
with a 95% probability.
Imagine, finally, that one expects the X variable to go down by 2 units. What will be
the effect of this change on Y ?
We know that β2 measures the effect on Y of a change in X. If this change is of −2
units, the estimated change in Y will be:

β̂2 · (−2) = (0.637) · (−2) = −1.274

That is, one predicts that a decrease of 2 units in the variable X (grade in the midterm
exam) will produce a decrease of 1.274 units in the variable Y (grade in the final
exam).

RStudio Activity 4.1 - Regression Analysis

We will carry out now a complete regression analysis exercise. As in all the RStudio
Activities of this text, we will use a Monte Carlo method to generate the data, in this
case according to a theoretical model [MT] that we will specify.
The script for this activity is:
4.6. FORECASTING 191

1. Generation of the data series

2. Analysis of the relationship
3. Specification and OLS estimation of the regression model

4. Validation of the model

(a) Verification of the hypotheses of the model
(b) Significance of the parameters of the model
5. Prediction

1. Generation of the data series

As in all the other RStudio Activities, we will generate the data randomly, but
this time using a specific linear model:

Y = 1.3 + 2.1X + u

That is, we are considering a Theoretical Model [TM] where the real values of
the coefficients4 are: β1 = 1.3 and β2 = 2.1.
From here, the independent variable (exogenous) will be generated using a Nor-
2
mal distribution with mean µX = 100 and variance σX = 16. The error term
u, according to the regression analysis theory, will follow a Normal distribution
with mean µu = 0 and variance σu2 = 4.
Finally, the dependent variable (endogenous), Y , will be generated according to
the theoretical lineal model above.

[Link](4)

### Generate the independent variable

X <- rnorm(500,10,4)

### Generate the error term

u <- rnorm(500,0,2)

### Generate the dependent variable according

### to the Theoretical Model specified
Y <- 1.3 + 2.1*X + u

2. Analysis of the relationship

To check the prior interest in doing a regression analysis, it is recommended to
first perform a quick analysis of the relationship between the variables. Basically,
it’s about making a scatter plot to see if a possible linear relationship is detected
(or not), and calculating the correlation coefficient to get an idea of the intensity
of the possible relationship.

4 As in other cases, we are using these values just to generate the data. Then, we will pretend that they are

not known and will estimate them using the regression analysis technique, as if it were a real case with real
data.
192 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

### Scatter plot

plot (X,Y, pch=20,cex=0.5)

### Add the 'lowess' curbe

lines(lowess(X,Y), col="blue")

30
Y

10
0

0 5 10 15 20

### Compute the correlation coefficient

paste("Cor(X,Y) = ",cor(X,Y))

## [1] "Cor(X,Y) = 0.97307179852879"

The plot clearly suggests the existence of a linear relationship between the vari-
ables.
The blue curve shown in the graph is not a straight line, even though it looks like
it, and is therefore not the regression line we want to estimate. It is a curve called
"lowess line" that RStudio draws interactively looking for the line that best fits
the set of points. It is not, therefore, the result of an analytical expression (or
formula), but an indication of which would be the curve that best fits the data.
The conclusion is that if this curve is very close to a straight line, as in this case,
then it makes a lot of sense to consider doing a regression analysis with a linear
model.

3. Specification and OLS estimation of the regression model

Let us thus estimate the linear model below using the method of Ordinary Least
Squares (OLS):
Y = βˆ1 + βˆ2 X + û
The RStudio command to do so is easy: lm() (for linear model).
4.6. FORECASTING 193

As in other cases, the output of that command will be stored in a variable, that we
will call model. To show the results we will use he command summary(model)5
Notice that the order of the variables is important when entering them in the
command: the dependent variable Y goes first, followed by the ∼ symbol, and
finally the independent variable X.

### Run the OLS estimation

model <- lm(Y~X)

### Show the results

summary(model)

##
## Call:
## lm(formula = Y ~ X)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.1439 -1.1860 0.0192 1.3615 5.3502
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.08393 0.23817 4.551 6.72e-06 ***
## X 2.11383 0.02244 94.207 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.943 on 498 degrees of freedom
## Multiple R-squared: 0.9469, Adjusted R-squared: 0.9468
## F-statistic: 8875 on 1 and 498 DF, p-value: < 2.2e-16

The RStudio output provides a lot of information about the result of the esti-
mation. The details on how to interpret this output can be found in the in the
Appendix B “Reading RStudio Outputs”. For now, we will look only at the re-
sult of the parameter estimates. In the table labeled Coefficients, in the
Estimate column, we find the estimates of the coefficients α̂1 (Intercept)
and α̂2 (X), and in the first of the last three rows we have the estimate of the stan-
dard deviation of the residuals σ̂u (Residual standard error). Thus,
we have:
α̂1 = 1.08393, α̂2 = 2.11383, i σ̂u = 1.943
Notice that the results of the estimations are really closed to the values used to
generate the data. This would indicate that the OLS is good estimation method
in this context.
α1 = 1.3, α2 = 2.1, i σu = 2
Therefore, the estimated model [EM] found is:

Y = 1.08393 + 2.11383X + û

or, according to the notation that we have used in RStudio,

Y = 1.08393 + 2.11383X + res

5 This is the standard way RStudio has for presenting results. We will see another more sophisticated way
later
194 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Once again, let us represent graphically the relationship between the variables,
keeping the lowess line and adding the regression line that we have just estimated
with the command abline()
### Plot the relationship between X and Y
plot (X,Y, pch=20,cex=0.5)
lines(lowess(X,Y), col="blue")

### Add the regression line

abline(model, col="red")
30
Y

10
0

0 5 10 15 20

It is clear that the lowess line (blue) and the regression line (red) are almost
identical. This is so because, in this case, by the way the data is generated, the
relationship between X and Y is almost perfectly linear.
We present below a neater and clearer way of presenting the same information
provided by the summary(model) command. This is the stargazer com-
mand, available if the stargazer package is first installed in RStudio.
### Estimation results
### using 'stargazer'
library(stargazer)
stargazer(model, type="text")

##
## ===============================================
## Dependent variable:
## ---------------------------
## Y
## -----------------------------------------------
## X 2.114***
## (0.022)
##
4.6. FORECASTING 195

## Constant 1.084***
## (0.238)
##
## -----------------------------------------------
## Observations 500
## R2 0.947
## Adjusted R2 0.947
## Residual Std. Error 1.943 (df = 498)
## F Statistic 8,875.011*** (df = 1; 498)
## ===============================================
## Note: *p<0.1; **p<0.05; ***p<0.01

In this output is easy to identify the estimates for the constant coefficient α̂1
(Constant) and the coefficient of the independent variable α̂2 (X), as well
as the standard deviation of the residuals σ̂u (Residual Std. Error). A
guide for the interpretation of this output can be found in the Appendix B “Read-
ing RStudio Outputs”.

4. Model Validation

We will now carry out a diagnosis of the estimated model to determine the qual-
ity of its results. We will first look at the extent to which some of the basic
assumptions of the theory of regression analysis that we have seen in section
4.2 are satisfied. Then we will test the significance of the estimated coefficients
obtained and the explanatory power of the model.

(a) Verification of the hypothesis of the model

We will study whether the results obtained suggest the fulfillment of the
following three basic assumptions of the model:

i. Normality of the error term: ui ∼ N (0, σu2 ). The error ui is a random

variable that follows a Normal distribution with zero mean (that is, we
expect the error to be tiny) and constant variance σu2
ii. There is no relationship between the errors associated with different
observations: E(ui · uj ) = 0 si when i ̸= j.
iii. The independent variable X and the errors u are not related: E(Xi ·
ui ) = 0. ´

To do so, we will first use the command resid() to get the series of resid-
uals (û) of the estimated model, that we will label res, and will represent
it graphically.

### Get the residuals

res <- resid(model)

### Plot the residuals and add

### a horizontal line at zero
plot(seq(1,500),res, pch=20,cex=0.5, xlab="Observations")
abline(0,0)
196 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

4
res

0
−4

0 100 200 300 400 500

Observations

We notice that residuals behave according to what the theory predicts:

i. They oscillate around zero, with a variability (variance) that looks con-
stant and homogeneous

ii. Visually, one cannot detect any relationship between errors correspond-
ing to different observations.

iii. Their behavior seems totally random, without any clear pattern.

Just to provide a point of comparison with situations where these assump-

tions are not met we will intentionally generate:

i. A series of residuals in which the variance will not be homogeneous

(heteroskedasticity)

ii. A series of residuals in which a pattern of correlation between each

error and the error of the previous observation will be noticeable (au-
tocorrelation)

## Generate a series of residuals with

## non-homogenous variances

res_heterokedastic <- rnorm(500,0,seq(1,8,0.014))

## Plot these residuals

plot(seq(1,500),res_heterokedastic, pch=20,cex=0.5,
main="Heterokedastic Residuals",
xlab="Observations")
abline(0,0)
4.6. FORECASTING 197

res_heterokedastic Heterokedastic Residuals

−5 5 15
−20

0 100 200 300 400 500

Observations

We note that, despite oscillating around zero, the variance is increasing, it

is not constant. There is heteroskedasticity.
## Generate a series of residuals with autocorrelation
res_autocorrelation <- diffinv(rnorm(499))

## Plot these residuals

plot(seq(1,500),res_autocorrelation, pch=20,cex=0.5,
main="Autocorrelated residuals",
xlab="Observations")
abline(0,0)

Autocorrelated residuals
res_autocorrelation

20
0

0 100 200 300 400 500

Observations

In this case we note that each residual is very close to the previous residual.
There is, therefore, a relationship between residuals. There exists autocor-
relation.
198 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Let’s go back to the original series of residuals of our model, res. Natu-
rally, the proper way to test if the theoretical hypothesis of the model are
met is not a mere visual inspection. That might only suggest the existence
of a problem, but formal techniques are needed to corroborate (or deny) it.
In this sense, we will now do:

i. A Kolmogorov-Smirnov test of Normality6 of the errors u to verify

that, indeed, follow a Normal distribution with mean µu = 0 and con-
stant varianceσu2 . For this test we will take as the value of the variance
of the errors its estimate in the model, that is, the variance of the resid-
uals: σu2 = σ̂u2 , and the series of residuals res will be the sample to
use.

ii. A correlation analysis of the residuals using the command acf()

(auto correlation function) in RStudio

iii. A test of independence between quantitative variables7 to check whether

the variable X and the errors u are not related.

i. Test of Normality of the errors

We conduct a Kolmogorov-Smirnov test of Normality of the errors
using the residuals res

### KS test of Normality

[Link](res,pnorm,0,sd(res))
##
## Asymptotic one-sample Kolmogorov-Smirnov test
##
## data: res
## D = 0.024727, p-value = 0.9199
## alternative hypothesis: two-sided

As we can see, the p_value is quite high. Hence, for any value
α, we do not reject the null hypothesis of Normality. Therefore, the
hypothesis of Normality of the errors is fulfilled.
Another way to view this is by using a Q-Q (Quantile-to-Quantile)
plot, which compares two different distributions. In this case we will
compare the "quantiles" of the distribution of the residuals with the
"quantiles" of a Normal distribution (qqnorm command)

### Compare the series 'res' of residuals

### to a Normal distribution
qqnorm(res, pch=20,cex=0.5)

### Add the diagonal

qqline(res,col="red")

6 Remember the test studied in section 3.1.2.

7 Remember the test studied in section 3.2.2.
4.6. FORECASTING 199

Normal Q−Q Plot

Sample Quantiles

4
0
−4

−3 −2 −1 0 1 2 3

Theoretical Quantiles

The way to interpret this graph is that the further on the diagonal the
dots are, the more similar the two distributions will be. In this case,
the closer the points are to the diagonal, the closer the residuals will
adhere to a Normal distribution. And we see that, indeed, this is the
case.
Again, as an element of comparison, let’s look at what the graph would
look like if we plot the series of residuals that we constructed with
heteroskedasticity, res_heterokedastic
### Compare the series 'res_heterokedastic'
### of residuals to a Normal distribution
qqnorm(res_heterokedastic, pch=20,cex=0.5)

### Add the diagonal

qqline(res,col="red")

Normal Q−Q Plot

Sample Quantiles

−5 5 15
−20

−3 −2 −1 0 1 2 3

Theoretical Quantiles
200 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

Clearly, the data that corresponds to the residuals deviates from the
diagonal.

ii. Correlation analysis of the residuals.

We will now study the autocorrelation of the residuals. To do so the
command acf() command will calculate the correlation between
the residuals of each observation and the residuals of previous obser-
vations, and will show them graphically (correlogram)

### Study the correlation of the residuals

acf(res)

Series res
0.8
ACF

0.4
0.0

0 5 10 15 20 25

Lag

The length of each of the "bars" in this graph corresponds to the value
of the correlation coefficient between each residual and the residuals
of previous observations (up to 25 previous observations) represented
on the vertical axis (Lag).
We see that, logically, each residual is fully correlated with itself (bar
of length 1 above 0), but has a very low correlation (never exceeding
0.1) with residuals from previous observations.
Again for comparison, let’s look at what this correlogram would look
like with the series of residuals we constructed with autocorrelation,
res_autocorrelation:

### Study the correlation of the

### autocorrelated residuals
acf(res_autocorrelation)
4.6. FORECASTING 201

Series res_autocorrelation

0.8
ACF

0.4
0.0

0 5 10 15 20 25

Lag

Now the correlogram shows a high correlation between residuals.

iii. Test of independence between the variable X and the errors u
We will first represent graphically the observations of the variables X
and res, to check whether we spot a relationship visually. The lowess
line will be included.
plot (X,res, pch=20,cex=0.5)
lines(lowess(X,res), col="blue")
4
2
res

0
−4

0 5 10 15 20

We observe a very scattered cloud of points. Therefore, no relationship

is detected between the variables.
Let us run a test of independence between the variable X and the errors
u using the residuals res as the sample.
202 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

### Test of independence between

### the independent variable X and
### the residuals 'res' (rho=0)
[Link](X,res)
##
## Pearson's product-moment correlation
##
## data: X and res
## t = -4.7293e-16, df = 498, p-value = 1
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
## -0.08769059 0.08769059
## sample estimates:
## cor
## -2.119251e-17

The valor_p_value obtained has the highest possible value. There-

fore, the null hypothesis of independence between the independent
variable X and errors u is categorically NOT rejected.
(b) Significance of the parameters of the model
We now analyze the significance of the estimated coefficients and the ex-
planatory power of the estimated model, as discussed in Sections 4.4 i 4.5.
What we will do next is:
i. Compute confidence intervals for the coefficients
ii. Test the significance of the coefficient β2
iii. Study the goodness-of-fit of the model
i. Confidence intervals for the coefficients
In the output produced by RStudio when doing the OLS estimation
we have all the information needed to find confidence intervals for β1
and β2 as seen in 4.4.1. For example, in the information supplied by
stargazer we have all that we need: the estimates of the coeffi-
cients β̂1 and β̂2 , and their standard deviations Sβ̂1 and Sβ̂2 are printed
(in parenthesis) underneath.
From here we could calculate the corresponding formulas following
the same technique that we used in the RStudio activity of the section
1.4.2.
But RStudio has a specific command that makes this task much easier:
confint()
Let us try it.
confint(model,level=0.95)
## 2.5 % 97.5 %
## (Intercept) 0.6159873 1.551882
## X 2.0697411 2.157911

The reading of this output is clear:

β1 ∈ [0.6159873, 1.551882]
β2 ∈ [2.0697411, 2.157911]

with confidence level of 95%

ii. Test the significance of the coefficientβ2
4.6. FORECASTING 203

As seen in section 4.4.2, it is of the utmost importance to test the sig-

nificance of β2 . That is, to test:

H0 : β 2 = 0
H0 : β2 ̸= 0

Indeed, we do need that the null hypothesis β2 = 0 is rejected to en-

sure the validity of the model.
Once again, RStudio makes our work much easier. As a matter of
fact, the conclusion of this test is shown, visually, in the output of the
command when the model is estimated.
For example, in the output of stargazer we have:
stargazer(model,type="text")
##
## ===============================================
## Dependent variable:
## ---------------------------
## Y
## -----------------------------------------------
## X 2.114***
## (0.022)
##
## Constant 1.084***
## (0.238)
##
## -----------------------------------------------
## Observations 500
## R2 0.947
## Adjusted R2 0.947
## Residual Std. Error 1.943 (df = 498)
## F Statistic 8,875.011*** (df = 1; 498)
## ===============================================
## Note: *p<0.1; **p<0.05; ***p<0.01

Note that, next to the estimate of the coefficient of the independent

variable X, that is, β̂2 , three stars *** are displayed.
These stars indicate the "degree of significance" of the coefficient. The
key to its interpretation appears in the last line of the output:

Note: *p<0.1; **p<0.05; ***p<0.01

This information refers to the p_value of the test and tells us that:

The null hypothesis β2 = 0 is

* p_value < 0.1
rejected for α = 0.1 only
The null hypothesis β2 = 0 is
** p_value < 0.05
rejected for α = 0.1 and α = 0.05
The null hypothesis β2 = 0 is
*** p_value < 0.01
rejected for any value of α

Therefore, three stars *** is distinctive of the maximum significance

of the coefficient: whatever the level of significance α, the hypothesisβ2 =
0 is rejected, and therefore the model is valid.
204 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

If no star appears, it is an indication that the p_value is too high

(greater than 0.1) and, therefore, in no case would the hypothesisβ2 =
0 be rejected, thus invalidating the model
If we used summary instead of stargazer we would have:
summary(model)
##
## Call:
## lm(formula = Y ~ X)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.1439 -1.1860 0.0192 1.3615 5.3502
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.08393 0.23817 4.551 6.72e-06 ***
## X 2.11383 0.02244 94.207 < 2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.943 on 498 degrees of freedom
## Multiple R-squared: 0.9469, Adjusted R-squared: 0.9468
## F-statistic: 8875 on 1 and 498 DF, p-value: < 2.2e-16

In this case, next to the value of the coefficient estimate (2.11383 ),

we are given information about its standard deviation (0.02244), the
observed value of the t statistic to test the significance of the coeffi-
cient (94,207), and the corresponding p_value (lower than 2e-16).
We see that this is extremely small, and therefore indicates that the
hypothesisβ2 = 0is rejected for any significance level α.
In this output we also find at the end a code with "stars" that informs
about the degree of significance of the coefficient, but it is organized
in a somewhat confusing way and does not conform to the standards
in the academic literature on regression analysis, as does the output of
the stargazer command.
Once the model has been validated, we will see next the degree of this
validity, that is, what is the goodness-of-fit of the model.
iii. Goodness-of-fit of the model
Once again, the outputs obtained when the model is estimated already
provide us with the necessary information. For example, in the output
of stargazer it is easy to identify the value of the goodness-of-fit
coefficient 8

R2 0.947

That is, the goodness-of-fit coefficient of the model is R2 = 0.947.

The interpretation, as seen in section 4.5 is that the estimated model,
Y = 1.08393 + 2.11383X + û, explains 94.7% of the behavior of the
dependent variable Y . The goodness-of-fit of the model is, thus, very
high9 .
8 We will also find another value called "Adjusted R2", which is useful when considering several

explanatory variables (X1 , X2, , . . .), a case that goes beyond the scope of this textbook.
9 This result is not surprising since we have generated the data following a highly linear model. With real
4.6. FORECASTING 205

5. Prediction
We will conclude this RStudio Activity conducting a simple exercise of predic-
tion (or forecasting).
It consists of, given a value of the independent variable X, to compute a "pre-
diction" of the value that the dependent variable Y will take according to the
estimated model. That is, use this value of X in the estimated model equation
Ŷ = 1.08393 + 2.11383X to produce the corresponding prediction of the value
of Y , which we denoted by Ŷ in section 4.2
As expected, RStudio has a specific command for this task: predict().
For example, if we type predict(model) we will get a list of 500 values that
correspond to each of the 500 predictions Ŷ for each of the 500 values of the
variable X
predict(model)

## 1 2 3 4 5 6 7
## 24.0549231 17.6352551 29.7570938 27.2613918 36.0518424 28.0502283 11.3888649
## 8 9 10 11 12 13 14
## 20.4199931 38.2580161 37.2461136 27.0130081 22.3551076 25.4610611 21.8405468
## 15 16 17 18 19 20 21
## 22.5126506 23.6513676 32.0728510 21.8484366 21.3735491 19.8255848 35.2502540
## 22 23 24 25 26 27 28
## 23.6187491 33.2785395 33.1147984 27.2353187 19.8298199 32.8410761 29.9151615
## 29 30 31 32 33 34 35
## 14.3754350 32.7083009 23.5197811 31.1166046 15.8451097 9.6898351 29.5033266
## 36 37 38 39 40 41 42
## 18.8018566 20.2994128 30.1202620 18.2829035 16.8315707 33.5836598 23.7571317
## 43 44 45 46 47 48 49
## 33.1507796 7.9492308 15.2804445 14.9324870 23.0579483 19.0459163 28.3430246
## 50 51 52 53 54 55 56
## 7.0247833 16.6100447 16.9483977 21.5488784 25.9055346 38.8867618 17.1764188
## 57 58 59 60 61 62 63
## 17.5505783 28.1068043 20.9060086 33.6275392 13.1875073 31.2224496 11.1227331
## 64 65 66 67 68 69 70
## 39.6713610 23.3330781 20.2631985 18.8624330 29.7426135 26.6711140 20.7740275
## 71 72 73 74 75 76 77
## 23.5638562 18.1157490 14.1143524 23.7485234 28.3246776 19.0976177 24.2306535
## 78 79 80 81 82 83 84
## 16.5916208 15.4849450 21.7850815 33.1035652 20.4114943 17.3625471 9.7867504
## 85 86 87 88 89 90 91
## 13.4900773 11.1751299 15.1345158 12.6621741 25.3400728 20.5158889 11.4191918
## 92 93 94 95 96 97 98
## 15.4747566 23.5672851 27.4204957 28.0390040 21.8243642 41.9257732 17.3386987
## 99 100 101 102 103 104 105
## 30.4109804 19.8755467 28.0124034 21.2488751 19.2080888 21.3278637 22.6016924
## 106 107 108 109 110 111 112
## 7.6268752 35.3768472 28.7870002 12.9339798 7.6112623 25.8380060 28.5177153
## 113 114 115 116 117 118 119
## 29.5378997 24.8038427 21.2580974 25.8043033 15.4773168 17.1135233 36.7231309
## 120 121 122 123 124 125 126
## 16.1686344 21.0956472 13.7688772 38.0654064 19.3694730 30.4514911 30.5745803
## 127 128 129 130 131 132 133
## 14.2635840 25.1746652 17.1962041 2.0796073 31.3371730 27.8724104 14.0659915
## 134 135 136 137 138 139 140
## 5.5209625 17.2777537 30.4177269 26.8919944 21.5275451 8.0450701 32.4751593
## 141 142 143 144 145 146 147
## 30.6817254 28.3043402 15.0828391 27.4812704 16.1122756 18.4217163 12.1138276
## 148 149 150 151 152 153 154
## 25.5237574 17.8560733 29.9154137 29.6373667 15.3210110 35.2373875 33.8442277
## 155 156 157 158 159 160 161
## 18.1361800 26.8755710 14.9729074 16.2446215 4.4907666 31.5182669 25.1141970
## 162 163 164 165 166 167 168
## 26.3723073 14.3418596 10.1489339 30.4725854 9.1474098 22.3718831 15.6717236

data, it is very difficult to obtain goodness-of-fit coefficients that reach 0.8

206 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

## 169 170 171 172 173 174 175

## 20.2830360 19.9022107 37.4084602 18.1796049 17.1932940 3.1306406 36.4491504
## 176 177 178 179 180 181 182
## 22.8334345 18.5016573 27.5200634 15.4596759 12.6847300 13.5553724 22.8231279
## 183 184 185 186 187 188 189
## 25.4496787 8.5027172 38.2918610 16.1666822 25.4390970 25.9496551 24.3979632
## 190 191 192 193 194 195 196
## 20.7049029 16.3869554 22.2186199 27.0043533 12.0018713 19.2952124 16.7245811
## 197 198 199 200 201 202 203
## 14.7004267 34.7109316 12.1140890 37.0231609 32.4931075 9.1350729 19.6666127
## 204 205 206 207 208 209 210
## 31.0090121 15.7298595 35.1137651 1.7427101 26.9261199 31.5698202 23.6294687
## 211 212 213 214 215 216 217
## 20.3158425 20.2908991 20.0814029 39.7100748 35.6085465 13.4067730 22.1512768
## 218 219 220 221 222 223 224
## 10.9420760 23.4612015 15.5609082 19.7786322 27.8605239 21.0669363 24.1169380
## 225 226 227 228 229 230 231
## 31.6685169 10.0592872 27.5859499 18.0013509 31.7451707 21.3751028 12.8381431
## 232 233 234 235 236 237 238
## 27.7851445 21.8561138 27.5354168 14.3010189 19.2004094 21.8410896 19.2792936
## 239 240 241 242 243 244 245
## 19.1869613 25.5672420 13.9026477 22.0302546 14.7430538 26.3233592 15.1913505
## 246 247 248 249 250 251 252
## 32.1415417 22.7380991 23.5475943 26.1869356 26.8316951 25.5671314 14.4317875
## 253 254 255 256 257 258 259
## 16.5015676 5.7859689 23.2366889 32.9272631 12.9383825 25.9248271 27.4339854
## 260 261 262 263 264 265 266
## 22.3732881 11.7444747 17.9588535 33.3559843 28.9393436 26.3233062 42.2699898
## 267 268 269 270 271 272 273
## 19.8402930 23.9047621 29.0729800 21.1884613 30.7427691 26.5985092 20.2777168
## 274 275 276 277 278 279 280
## 20.8048002 1.2865098 17.1392530 22.4886097 32.5894510 16.3101210 9.5041594
## 281 282 283 284 285 286 287
## 20.3182209 22.1466324 9.7221975 6.5989158 15.0687470 32.8399238 10.8032901
## 288 289 290 291 292 293 294
## 24.9601502 23.2266428 27.7148972 27.9854798 43.5085551 24.7801356 16.6094739
## 295 296 297 298 299 300 301
## 22.7545057 12.1801445 18.6667107 15.9008366 21.1540595 13.6090547 -1.7873434
## 302 303 304 305 306 307 308
## 35.0021253 34.4177412 26.4952458 15.8190501 24.3295786 19.5431098 23.0924261
## 309 310 311 312 313 314 315
## 20.4329797 24.5097188 19.6264143 29.7463054 25.5032204 2.9915513 17.5721861
## 316 317 318 319 320 321 322
## 25.0156583 17.1647000 11.5512269 28.6761149 29.3328066 23.5600355 31.0706546
## 323 324 325 326 327 328 329
## 16.7021272 14.0990154 20.5214238 28.0866656 13.7835742 13.7699615 19.1502572
## 330 331 332 333 334 335 336
## 31.1650867 31.3451142 32.6898351 29.4439750 15.3434761 19.1120375 31.6725504
## 337 338 339 340 341 342 343
## 23.1837876 18.4634632 9.9981938 18.9716022 27.2372741 24.2941795 7.4609223
## 344 345 346 347 348 349 350
## 22.7500074 13.0394580 21.3130497 26.0371821 31.4754225 29.8823548 13.7879355
## 351 352 353 354 355 356 357
## 16.1695564 25.7954137 25.8827781 22.9369356 26.8903353 17.6752242 23.2854202
## 358 359 360 361 362 363 364
## 30.7103674 31.2849046 18.4143269 30.3261099 29.8685353 11.8109188 19.7662977
## 365 366 367 368 369 370 371
## 8.2531115 22.5385058 4.9507109 23.1630122 24.8535164 16.8200636 30.1090177
## 372 373 374 375 376 377 378
## 19.9166181 32.7223290 13.2537724 38.1867928 12.3456229 25.7650048 8.2499599
## 379 380 381 382 383 384 385
## 23.7179846 21.0052782 42.1995989 32.5120433 25.6321025 27.6135269 14.7046695
## 386 387 388 389 390 391 392
## 11.1017084 23.7283683 10.1834720 9.5080399 13.1293731 23.4210810 24.4391320
## 393 394 395 396 397 398 399
## 32.3171235 7.9590005 24.2518683 12.7139774 15.0334840 19.3580825 21.7173195
## 400 401 402 403 404 405 406
## 20.9210833 12.2270445 26.0800063 20.2074036 29.0892861 16.0442886 3.6688812
## 407 408 409 410 411 412 413
## 30.6905187 32.0927436 28.7297537 35.6491133 25.1929633 17.7042042 19.8134593
## 414 415 416 417 418 419 420
## 21.7763960 22.7670834 17.3293914 22.1157559 29.8798904 17.9037323 24.0458517
## 421 422 423 424 425 426 427
4.6. FORECASTING 207

## 17.2860791 11.9320125 23.6614661 23.3305688 20.1968040 17.4324431 22.5522970

## 428 429 430 431 432 433 434
## 6.0630760 22.0102835 27.8456010 33.4076980 49.0609090 6.4230973 10.8233423
## 435 436 437 438 439 440 441
## 41.7002929 24.3824421 29.5640922 25.2309276 12.5367883 24.7239628 16.3375816
## 442 443 444 445 446 447 448
## 32.2419244 35.1182153 25.7806042 35.4053783 35.9913799 23.6859046 19.3545108
## 449 450 451 452 453 454 455
## 11.4231410 22.7330649 12.6309037 21.6864604 16.0280403 31.3035162 34.7264431
## 456 457 458 459 460 461 462
## 0.8048308 15.2910419 24.9182967 14.0196991 18.9575712 21.9153479 28.8795556
## 463 464 465 466 467 468 469
## 25.8511571 27.7730277 15.8934210 22.3225839 17.0916403 34.1102356 16.8945195
## 470 471 472 473 474 475 476
## 14.8265086 12.1343353 25.3309680 16.3031463 18.6249448 30.7935375 25.7778878
## 477 478 479 480 481 482 483
## 33.4281970 31.0525969 16.7016251 8.1882813 9.4549237 24.8677114 6.1767548
## 484 485 486 487 488 489 490
## 17.1049749 19.2541206 13.6793641 18.4306294 29.5616908 22.8911287 16.5861523
## 491 492 493 494 495 496 497
## 21.3140746 26.2401090 15.1463172 25.8420541 16.7607406 27.0979246 21.4545428
## 498 499 500
## 20.7608513 18.3386213 17.3042125

This is not very useful, though.

Remember that, by definition of residual, we know that û = Y − Ŷ . Thus, using
the terminology used in the implementation of the model in RStudio, we could
calculate the residuals (res) of the regression by doing:

res = Y - predict(model)

But again, as said above, this is not very useful since we have seen before how
to easily calculate residuals with the command res(model).
The main use of the predict command is, if we have a list of new (or future)
values of the variable X, to calculate the corresponding predictions Ŷ of the
dependent variable.
To do so, we first will have to build a new database ([Link]) with the new
values of the variable X, and then use this new database to make the predictions
in our model.
For example, let us imagine that we want to calculate the predictions of the
dependent variable Y corresponding to the values of X: 4, 10, 15 and 23. We
would do this as follows:

### First create a data base with

### new values of X
[Link]<-[Link](X=c(4,10,15,23))

### Apply this new data base to

### our model to make predictions
predict(model, [Link])

## 1 2 3 4
## 9.539239 22.222195 32.791325 49.701933

The reading of the results is straightforward:

208 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL

X Ŷ
4 9.539239
10 22.222195
15 32.791325
23 49.701933

Moreover, the predict() command can also compute confidence intervals

for these predictions, as seen in section 4.6.2
### Add the option to compute confidence
### intervals to the command 'predict'
predict(model, [Link], interval="confidence", level=.95)

## fit lwr upr

## 1 9.539239 9.228747 9.84973
## 2 22.222195 22.051435 22.39295
## 3 32.791325 32.508462 33.07419
## 4 49.701933 49.099025 50.30484

We can now improve on the results that we had obtained before about the pre-
dictions:

X Ŷ 95% confidence interval for the prediction Ŷ

4 9.539239 [9.228747, 9.84973]

10 22.222195 [22.051435, 22.39295]
15 32.791325 [32.508462, 33.07419]
23 49.701933 [49.099025, 50.30484]

R Hints

• In this activity, when introducing the linear model, we have used the symbol
~. Its use is important as it indicates a precedence between two variables.
Thus, when we enter:

lm(Y ~ X)

we are indicating that Y precedes X, the order is important. In this case

(regression analysis) we are indicating that Y is the dependent variable and
X the independent variable.
This is not important, for example, when computing the correlation coeffi-
cient between the two variables as:

cor(X,Y) = cor(Y,X)

The order is not important in that case.

4.6. FORECASTING 209

In section 2.6 (Analysis of Variance), when creating a boxplot of the

university access Grade for each Province, we have also used the sym-
bol ∼ as we were looking for a graph in this sense, not a boxplot of the
Province for each access Grade. The order was important.

End of Activity RStudio 4.1

210 CHAPTER 4. INTRODUCTION TO THE REGRESSION MODEL
Appendix A

Statistical Tables

A.1 The Standard Normal Distribution

A.2 The t − student Distribution

A.3 The χ2 Distribution

A.4 The Snedecor’s F Distribution
A.5 Values for the Kolmogorov-Smirnov Test

211
212 APPENDIX A. STATISTICAL TABLES

A.1 The Standard Normal Distribution

z 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

0.0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359
0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753
0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141
0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517
0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879
0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224
0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549
0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852
0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133
0.9 0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389
1.0 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621
1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830
1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015
1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177
1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319
1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441
1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545
1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633
1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706
1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767
2.0 0.9773 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817
2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857
2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890
2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916
2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936
2.5 0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952
2.6 0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964
2.7 0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974
2.8 0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981
2.9 0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986
3.0 0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990
3.1 0.9990 0.9991 0.9991 0.9991 0.9992 0.9992 0.9992 0.9992 0.9993 0.9993
3.2 0.9993 0.9993 0.9994 0.9994 0.9994 0.9994 0.9994 0.9995 0.9995 0.9995
3.3 0.9995 0.9995 0.9995 0.9996 0.9996 0.9996 0.9996 0.9996 0.9996 0.9997
3.4 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9998
3.5 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998 0.9998
3.6 0.9998 0.9998 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999
3.7 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999
3.8 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999 0.9999
3.9 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000
A.2. THE t − student DISTRIBUTION 213

A.2 The t − student Distribution

α
n 0.90 0.95 0.975 0.990 0.995 0.999 0.9995
1 3.078 6.314 12.706 31.821 63.657 318.302 636.619
2 1.886 2.920 4.303 6.965 9.925 22.327 31.598
3 1.638 2.353 3.182 4.541 5.841 10.215 12.941
4 1.533 2.132 2.776 3.747 4.604 7.173 8.610
5 1.476 2.015 2.571 3.365 4.032 5.894 6.859
6 1.440 1.943 2.447 3.143 3.707 5.208 5.959
7 1.415 1.895 2.365 2.998 3.499 4.785 5.405
8 1.397 1.860 2.306 2.896 3.355 4.501 5.041
9 1.383 1.833 2.262 2.821 3.250 4.297 4.781
10 1.372 1.812 2.228 2.764 3.169 4.144 4.587
11 1.363 1.796 2.201 2.718 3.106 4.025 4.437
12 1.356 1.782 2.179 2.681 3.055 3.930 4.318
13 1.350 1.771 2.160 2.650 3.012 3.852 4.221
14 1.345 1.761 2.145 2.624 2.977 3.787 4.140
15 1.341 1.753 2.131 2.602 2.947 3.733 4.073
16 1.337 1.746 2.120 2.583 2.921 3.686 4.015
17 1.333 1.740 2.110 2.567 2.898 3.646 3.965
18 1.330 1.734 2.101 2.552 2.878 3.611 3.922
19 1.328 1.729 2.093 2.539 2.861 3.579 3.883
20 1.325 1.725 2.086 2.528 2.845 3.552 3.850
24 1.318 1.711 2.064 2.492 2.797 3.467 3.745
30 1.310 1.697 2.042 2.457 2.750 3.385 3.646
40 1.303 1.684 2.021 2.423 2.704 3.307 3.551
50 1.299 1.676 2.009 2.403 2.678 3.261 3.496
60 1.296 1.671 2.000 2.390 2.660 3.232 3.460
80 1.292 1.664 1.990 2.374 2.639 3.195 3.416
100 1.290 1.660 1.984 2.364 2.626 3.174 3.391
200 1.286 1.653 1.972 2.345 2.601 3.131 3.340
∞ 1.282 1.645 1.960 2.326 2.576 3.090 3.291
214 APPENDIX A. STATISTICAL TABLES

A.3 The χ2 Distribution

α
n 0.005 0.01 0.025 0.05 0.950 0.975 0.990 0.995 0.999
1.0 0.04 393 0.03 157 0.03 982 0.00393 3.841 5.024 6.635 7.879 10.828
2.0 0.010003 0.02010 0.05064 0.1026 5.991 7.378 9.210 10.597 13.816
3.0 0.07172 0.1148 0.2158 0.3518 7.815 9.348 11.345 12.838 16.266
4.0 0.2070 0.2971 0.4844 0.7107 9.488 11.143 13.277 14.860 18.467
5.0 0.4117 0.5543 0.8312 1.145 11.070 12.832 15.086 16.750 20.515
6.0 0.6757 0.8721 1.237 1.635 12.592 14.449 16.812 18.548 22.458
7.0 0.9893 1.239 1.690 2.167 14.067 16.013 18.475 20.278 24.322
8.0 1.344 1.646 2.180 2.733 15.507 17.535 20.090 21.955 26.124
9.0 1.735 2.088 2.700 3.325 16.919 19.023 21.666 23.589 27.877
10.0 2.156 2.558 3.247 3.940 18.307 20.483 23.209 25.188 29.588
11.0 2.603 3.053 3.816 4.575 19.675 21.920 24.725 26.757 31.264
12.0 3.074 3.571 4.404 5.226 21.026 23.337 26.217 28.300 32.909
13.0 3.565 4.107 5.009 5.892 22.362 24.736 27.688 29.819 34.528
14.0 4.075 4.660 5.629 6.571 23.685 26.119 29.141 31.319 36.123
15.0 4.601 5.229 6.262 7.261 24.996 27.488 30.578 32.801 37.697
16.0 5.142 5.812 6.908 7.962 26.296 28.845 32.000 34.267 39.252
17.0 5.697 6.408 7.564 8.672 27.587 30.191 33.409 35.718 40.790
18.0 6.265 7.015 8.231 9.390 28.869 31.526 34.805 37.156 42.312
19.0 6.844 7.633 8.907 10.117 30.144 32.852 36.191 38.582 43.820
20.0 7.434 8.260 9.591 10.851 31.410 34.170 37.566 39.997 45.315
21.0 8.034 8.897 10.283 11.591 32.671 35.479 38.932 41.401 46.797
22.0 8.643 9.542 10.982 12.338 33.924 36.781 40.289 42.796 48.268
23.0 9.260 10.196 11.689 13.091 35.172 38.076 41.638 44.181 49.728
24.0 9.886 10.856 12.401 13.848 36.415 39.364 42.980 45.559 51.179
25.0 10.520 11.524 13.120 14.611 37.652 40.646 44.314 46.928 52.620
26.0 11.160 12.198 13.844 15.379 38.885 41.923 45.642 48.290 54.052
27.0 11.808 12.879 14.573 16.151 40.113 43.195 46.963 49.645 55.476
28.0 12.461 13.565 15.308 16.928 41.337 44.461 48.278 50.993 56.892
29.0 13.121 14.256 16.047 17.708 42.557 45.722 49.588 52.336 58.301
30.0 13.787 14.953 16.791 18.493 43.773 46.979 50.892 53.672 59.703
40.0 20.707 22.164 24.433 26.509 55.758 59.342 63.691 66.766 73.402
50.0 27.991 29.707 32.357 34.764 67.505 71.420 76.154 79.490 86.661
60.0 35.534 37.485 40.482 43.188 79.082 83.298 88.379 91.952 99.607
70.0 43.275 45.442 48.758 51.739 90.531 95.023 100.425 104.215 112.317
80.0 51.172 53.540 57.153 60.391 101.879 106.629 112.329 116.321 124.839
90.0 59.196 61.754 65.647 69.126 113.145 118.136 124.116 128.299 137.208
100.0 67.328 70.065 74.222 77.929 124.342 129.561 135.807 140.169 149.449
A.4. THE SNEDECOR ’S F DISTRIBUTION 215

A.4 The Snedecor ’s F Distribution

These are the tables for the Snedecor’s F distribution with ν1 degrees of freedom in
the numerator and ν2 degrees of freedom in the denominator
The values in the table correspond to:

p(Fv1 ,v2 ≤ F ) = α

as in Figure A.1:

Figure A.1: Snedecor’s F distribution

The table has values for α = 0.95, α = 0.975 and α = 0.99. The top number in each
cell corresponds to α = 0.95, whereas the other two values are for α = 0.975 and α =
0.99 respectively
216 APPENDIX A. STATISTICAL TABLES

ν1
1 2 3 4 5 6 7 8
161 199 216 225 230 234 237 239
1 648 799 864 900 922 937 948 957 1
4052 5000 5403 5625 5764 5859 5928 5981
18.51 19.00 19.16 19.25 19.30 19.33 19.35 19.37
2 38.51 39.00 39.17 39.25 39.30 39.33 39.36 39.37 2
98.50 99.00 99.17 99.25 99.30 99.33 99.36 99.37
10.13 9.55 9.28 9.12 9.01 8.94 8.89 8.85
3 17.44 16.04 15.44 15.10 14.88 14.73 14.62 14.54 3
34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49
7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04
4 12.22 10.65 9.98 9.60 9.36 9.20 9.07 8.98 4
21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80
6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82
5 10.01 8.43 7.76 7.39 7.15 6.98 6.85 6.76 5
16.26 13.27 12.06 11.39 10.97 10.67 10.46 10.29
5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15
6 8.81 7.26 6.60 6.23 5.99 5.82 5.70 5.60 6
13.75 10.92 9.78 9.15 8.75 8.47 8.26 8.10
5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73
7 8.07 6.54 5.89 5.52 5.29 5.12 4.99 4.90 7
12.25 9.55 8.45 7.85 7.46 7.19 6.99 6.84
5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44
ν2 8 7.57 6.06 5.42 5.05 4.82 4.65 4.53 4.43 8 ν2
11.26 8.65 7.59 7.01 6.63 6.37 6.18 6.03
5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23
9 7.21 5.71 5.08 4.72 4.48 4.32 4.20 4.10 9
10.56 8.02 6.99 6.42 6.06 5.80 5.61 5.47
4.96 4.10 3.71 3.48 3.33 3.22 3.14 3.07
10 6.94 5.46 4.83 4.47 4.24 4.07 3.95 3.85 10
10.04 7.56 6.55 5.99 5.64 5.39 5.20 5.06
4.75 3.89 3.49 3.26 3.11 3.00 2.91 2.85
12 6.55 5.10 4.47 4.12 3.89 3.73 3.61 3.51 12
9.33 6.93 5.95 5.41 5.06 4.82 4.64 4.50
4.54 3.68 3.29 3.06 2.90 2.79 2.71 2.64
15 6.20 4.77 4.15 3.80 3.58 3.41 3.29 3.20 15
8.68 6.36 5.42 4.89 4.56 4.32 4.14 4.00
4.35 3.49 3.10 2.87 2.71 2.60 2.51 2.45
20 5.87 4.46 3.86 3.51 3.29 3.13 3.01 2.91 20
8.10 5.85 4.94 4.43 4.10 3.87 3.70 3.56
4.24 3.39 2.99 2.76 2.60 2.49 2.40 2.34
25 5.69 4.29 3.69 3.35 3.13 2.97 2.85 2.75 25
7.77 5.57 4.68 4.18 3.85 3.63 3.46 3.32
4.03 3.18 2.79 2.56 2.40 2.29 2.20 2.13
50 5.34 3.97 3.39 3.05 2.83 2.67 2.55 2.46 50
7.17 5.06 4.20 3.72 3.41 3.19 3.02 2.89
3.94 3.09 2.70 2.46 2.31 2.19 2.10 2.03
100 5.18 3.83 3.25 2.92 2.70 2.54 2.42 2.32 100
6.90 4.82 3.98 3.51 3.21 2.99 2.82 2.69
A.4. THE SNEDECOR ’S F DISTRIBUTION 217

ν1
9 10 12 15 20 25 50 100
241 242 244 246 248 249 252 253
1 963 969 977 985 993 998 1008 1013 1
6022 6056 6106 6157 6209 6240 6303 6334
19.38 19.40 19.41 19.43 19.45 19.46 19.48 19.49
2 39.39 39.40 39.41 39.43 39.45 39.46 39.48 39.49 2
99.39 99.40 99.42 99.43 99.45 99.46 99.48 99.49
8.81 8.79 8.74 8.70 8.66 8.63 8.58 8.55
3 14.47 14.42 14.34 14.25 14.17 14.12 14.01 13.96 3
27.35 27.23 27.05 26.87 26.69 26.58 26.35 26.24
6.00 5.96 5.91 5.86 5.80 5.77 5.70 5.66
4 8.90 8.84 8.75 8.66 8.56 8.50 8.38 8.32 4
14.66 14.55 14.37 14.20 14.02 13.91 13.69 13.58
4.77 4.74 4.68 4.62 4.56 4.52 4.44 4.41
5 6.68 6.62 6.52 6.43 6.33 6.27 6.14 6.08 5
10.16 10.05 9.89 9.72 9.55 9.45 9.24 9.13
4.10 4.06 4.00 3.94 3.87 3.83 3.75 3.71
6 5.52 5.46 5.37 5.27 5.17 5.11 4.98 4.92 6
7.98 7.87 7.72 7.56 7.40 7.30 7.09 6.99
3.68 3.64 3.57 3.51 3.44 3.40 3.32 3.27
7 4.82 4.76 4.67 4.57 4.47 4.40 4.28 4.21 7
6.72 6.62 6.47 6.31 6.16 6.06 5.86 5.75
3.39 3.35 3.28 3.22 3.15 3.11 3.02 2.97
ν2 8 4.36 4.30 4.20 4.10 4.00 3.94 3.81 3.74 8 ν2
5.91 5.81 5.67 5.52 5.36 5.26 5.07 4.96
3.18 3.14 3.07 3.01 2.94 2.89 2.80 2.76
9 4.03 3.96 3.87 3.77 3.67 3.60 3.47 3.40 9
5.35 5.26 5.11 4.96 4.81 4.71 4.52 4.41
3.02 2.98 2.91 2.85 2.77 2.73 2.64 2.59
10 3.78 3.72 3.62 3.52 3.42 3.35 3.22 3.15 10
4.94 4.85 4.71 4.56 4.41 4.31 4.12 4.01
2.80 2.75 2.69 2.62 2.54 2.50 2.40 2.35
12 3.44 3.37 3.28 3.18 3.07 3.01 2.87 2.80 12
4.39 4.30 4.16 4.01 3.86 3.76 3.57 3.47
2.59 2.54 2.48 2.40 2.33 2.28 2.18 2.12
15 3.12 3.06 2.96 2.86 2.76 2.69 2.55 2.47 15
3.89 3.80 3.67 3.52 3.37 3.28 3.08 2.98
2.39 2.35 2.28 2.20 2.12 2.07 1.97 1.91
20 2.84 2.77 2.68 2.57 2.46 2.40 2.25 2.17 20
3.46 3.37 3.23 3.09 2.94 2.84 2.64 2.54
2.28 2.24 2.16 2.09 2.01 1.96 1.84 1.78
25 2.68 2.61 2.51 2.41 2.30 2.23 2.08 2.00 25
3.22 3.13 2.99 2.85 2.70 2.60 2.40 2.29
2.07 2.03 1.95 1.87 1.78 1.73 1.60 1.52
50 2.38 2.32 2.22 2.11 1.99 1.92 1.75 1.66 50
2.78 2.70 2.56 2.42 2.27 2.17 1.95 1.82
1.97 1.93 1.85 1.77 1.68 1.62 1.48 1.39
100 2.24 2.18 2.08 1.97 1.85 1.77 1.59 1.48 10
2.59 2.50 2.37 2.22 2.07 1.97 1.74 1.60
218 APPENDIX A. STATISTICAL TABLES

A.5 Values for the Kolmogorov-Smirnov test

α
n 0.1 0.05 0.025 0.01 0.005
1 0.9000 0.9500 0.9750 0.9900 0.9950
2 0.6838 0.7764 0.8419 0.9000 0.9293
3 0.5648 0.6360 0.7076 0.7846 0.8290
4 0.4927 0.5652 0.6239 0.6889 0.7342
5 0.4470 0.5094 0.5633 0.6272 0.6685
6 0.4104 0.4680 0.5193 0.5774 0.6166
7 0.3815 0.4361 0.4834 0.5384 0.5758
8 0.3583 0.4096 0.4543 0.5065 0.5418
9 0.3391 0.3875 0.4300 0.4796 0.5133
10 0.3226 0.3687 0.4092 0.4566 0.4889
11 0.3083 0.3524 0.3912 0.4367 0.4677
12 0.2958 0.3382 0.3754 0.4192 0.4490
13 0.2847 0.3255 0.3614 0.4036 0.4325
14 0.2748 0.3142 0.3489 0.3897 0.4176
15 0.2659 0.3040 0.3376 0.3771 0.4042
16 0.2578 0.2947 0.3273 0.3657 0.3920
17 0.2504 0.2863 0.3180 0.3553 0.3809
18 0.2436 0.2785 0.3094 0.3457 0.3706
19 0.2373 0.2714 0.3014 0.3369 0.3612
20 0.2316 0.2647 0.2941 0.3287 0.3524
21 0.2262 0.2586 0.2872 0.3210 0.3443
22 0.2212 0.2528 0.2809 0.3139 0.3367
23 0.2165 0.2475 0.2749 0.3073 0.3295
24 0.2120 0.2424 0.2693 0.3010 0.3229
25 0.2079 0.2377 0.2640 0.2952 0.3166
26 0.2040 0.2332 0.2591 0.2896 0.3106
27 0.2003 0.2290 0.2544 0.2844 0.3050
28 0.1968 0.2250 0.2499 0.2794 0.2997
29 0.1935 0.2212 0.2457 0.2747 0.2947
30 0.1903 0.2176 0.2417 0.2702 0.2899
31 0.1873 0.2141 0.2379 0.2660 0.2853
32 0.1844 0.2108 0.2342 0.2619 0.2809
33 0.1817 0.2077 0.2308 0.2580 0.2768
34 0.1791 0.2047 0.2274 0.2543 0.2728
35 0.1766 0.2018 0.2242 0.2507 0.2690
36 0.1742 0.1991 0.2212 0.2473 0.2653
37 0.1719 0.1965 0.2183 0.2440 0.2618
38 0.1697 0.1939 0.2154 0.2409 0.2584
39 0.1675 0.1915 0.2127 0.2379 0.2552
40 0.1655 0.1891 0.2101 0.2349 0.2521
√ √ √ √ √
> 40 1.07/ n 1.22/ n 1.36/ n 1.52/ n 1.63/ n
Appendix B

Reading RStudio Outputs

B.1 summary(aov)output reading (ANOVA)

B.2 summary(lm) output reading (Linear Regression)

B.3 stargazer(lm) output reading (Linear Regression)

219
220 APPENDIX B. READING RSTUDIO OUTPUTS

B.1 Summary(aov)output reading (ANOVA)

p_value

B.2 summary(lm) output reading (Linear Regression)

B.3. STARGAZER(LM) OUTPUT READING (LINEAR REGRESSION) 221

B.3 stargazer(lm) output reading (Linear Regres-

sion)