Reproduction is permitted, provided the source is acknowledged.

2
Contents
1 Introduction to Statistics .................................................................................................................. 7
1.1 Subdivisions of Statistics......................................................................................................... 7
1.2 Basic Concepts of Statistics..................................................................................................... 7
1.3 Data Types............................................................................................................................... 8
1.4 What Is the Typical of Statistical Investigations? ................................................................... 8
2 Methods for Data Description and Exploration............................................................................... 9
2.1 Frequency Distribution ............................................................................................................ 9
2.1.1 Absolute and Relative Frequency .................................................................................... 9
2.1.2 Graphical Data Representation for Frequencies ............................................................ 10
2.1.3 Frequency Distribution Function ................................................................................... 10
2.1.4 Empirical Distribution Function .................................................................................... 10
2.2 Frequency Distribution for Grouped Data ............................................................................. 12
2.2.1 Absolute and Relative Class Frequency ........................................................................ 13
2.2.2 Frequency Distribution Function for Grouped Data...................................................... 14
2.2.3 Empirical Distribution Function for Grouped Data ....................................................... 15
2.3 Distribution: Symmetry and Skewness ................................................................................. 16
2.4 Measures of Location ............................................................................................................ 17
2.4.1 Mean Value ................................................................................................................... 17
2.4.2 Median ........................................................................................................................... 17
2.4.3 Mode.............................................................................................................................. 18
2.5 Measures of Dispersion ......................................................................................................... 18
2.6 Quantiles and Box-Plot ......................................................................................................... 19
2.6.1 Quantiles ........................................................................................................................ 19
2.6.2 Box-Plots ....................................................................................................................... 21
3 Bivariate Frequency Distribution Analysis ................................................................................... 22
3.1 Bivariate Frequency Tables ................................................................................................... 22
3.2 Contingency Tables ............................................................................................................... 24
3.2.1 Contingency Tables and Measures of Association ........................................................ 25
3.2.2 Pearson Contingency Coefficient .................................................................................. 26
4 Correlation and Regression ........................................................................................................... 29
4.1 Correlation ............................................................................................................................. 29
4.1.1 Spearman’s Rank Correlation Coefficient..................................................................... 32
4.1.2 Correlation and Causality .............................................................................................. 34
4.2 Regression ............................................................................................................................. 34
4.2.1 Linear Regression .......................................................................................................... 34

3
 4.2.2 Coefficient of Determination ......................................................................................... 36
4.2.3 Predictions by Using Linear Regression Models .......................................................... 36
4.2.4 Multiple Linear Regression ........................................................................................... 37
4.2.5 Coefficient of Determination for the Multiple Linear Regression ................................ 38
4.2.6 Transformation of Predictor Variables .......................................................................... 38
5 Probability ..................................................................................................................................... 39
5.1 Basic Concepts ...................................................................................................................... 39
5.2 Linkages of Events ................................................................................................................ 40
5.3 Classical Definition of Probability (Laplace Probability) ..................................................... 40
5.4 “Statistical” or “Empirical” Definition of Probability ........................................................... 41
5.5 Axioms for Probabilities and Calculation Rules ................................................................... 42
5.6 The Addition Law of Probability .......................................................................................... 43
5.7 Conditional Probability ......................................................................................................... 44
5.8 The Multiplication Law ......................................................................................................... 45
5.9 Independent Events ............................................................................................................... 46
5.10 Event Trees ............................................................................................................................ 47
5.11 Law of Total Probability and Bayes’ Theorem ..................................................................... 48
6 Probability Distribution of Random Variables .............................................................................. 51
6.1 Random Variable ................................................................................................................... 51
6.2 Probability Distribution of Discrete Random Variables........................................................ 51
6.2.1 Binomial Distribution .................................................................................................... 54
6.3 Probability Distribution of Continuous Random Variables .................................................. 57
6.3.1 Normal Distribution....................................................................................................... 58
6.3.2 Standard Normal Distribution ....................................................................................... 59
6.3.3 The Standard Score and the Normal Probabilities......................................................... 60
7 Parameter Estimation .................................................................................................................... 63
Point Estimation ................................................................................................................................ 63
Interval Estimation ............................................................................................................................ 63
7.1 Estimates for the Key Statistical Parameters ......................................................................... 64
7.2 Confidence Interval for the Unknown Mean of a Normal Distribution with Known Variance
68
7.3 Confidence Interval for the Unknown Mean of a Normal Distribution with Unknown
Variance ............................................................................................................................................ 70
7.4 Large-Sample Confidence Interval for the Unknown Mean ................................................. 72
7.5 Confidence Interval for the Unknown Variance of a Normal Distribution ........................... 73
8 Hypothesis Testing ........................................................................................................................ 76
8.1 Error Types ............................................................................................................................ 76
4
 8.2 Planning and Execution of a Statistical Test ......................................................................... 76
8.3 p-Value .................................................................................................................................. 80
8.4 Approximate Binomial Test .................................................................................................. 80
8.5 Z-Test for an Unknown Mean Value of a Normal Distribution with Known Variance ........ 83
8.6 t-Test for an Unknown Mean Value of a Normal Distribution with Unknown Variance ..... 85
8.7 Two-Sample Tests ................................................................................................................. 88
Literature ............................................................................................................................................... 90
Appendix ............................................................................................................................................... 91
Table 1: Cumulative Distribution function Φ (z) of the standard normal distribution ...................... 92
Table 2: Quantiles of the standard normal distribution ..................................................................... 93
Table 3: Quantiles of Student’s t-distribution ................................................................................... 93
Table 4: Quantiles of the chi-squared distribution ............................................................................ 94

5
6
Statistics is used wherever one wants to gain information by using data. It is thus used in almost all
areas of our everyday life - from natural sciences, technology, medicine, social sciences, life sciences,
economics to computer science and physics. The list of application areas is basically endless: market
analysis, customer data analysis, product placement analysis, product quality controls, risk
management, climate modelling, modelling of current and future distribution patterns of different
animal species, prediction of flight punctualities, medical tests, discrimination measurement in legal
processes, web marketing, software development, search for elementary particles, economic forecasts,
customer data analysis, etc. The many exciting and versatile statistical applications raise the question
of why toy models as coin toss or dice are used in statistical lessons. These simple examples enable a
thorough learning of the concepts that are inside many complex problems of the real world. Thus, e.g.
in the natural and economic sciences, similar to the coin throw, many phenomena are connected with
probability and uncertainties. Simple examples are the daily rainfall or the current share prices. Even if
you have a lot of information about the underlying processes, there is often a part of variability that
you cannot describe exactly. In order to describe the processes of interest best one can, data and
methods are needed. From the data, one wants to draw conclusions and make a decision using
different methods.
The role of the statistics can be summarized as follows:
Statistics is a scientific discipline that deals with the methods on topics such as collection,
preparation, analysis, interpretation and presentation of data.
Statistics is a steadily growing discipline. The method selection in the specific application depends on
the respective problem, whereby a certain method is often applicable for a multitude of problems.
In the following, the basic concepts of statistics are introduced.

The descriptive statistics are used to calculate the most important descriptive characteristics from the
data, or to graphically display and compress data. (Examples: parameters such as mean or standard
deviation, diagrams, tables, etc.)
The exploratory statistics (analytical statistics or data mining) are used to search for structures and
patterns in the data. (Examples: scatterplot, boxplot, principal component analysis, etc.)
The inferential statistics (mathematical statistics) provides the methods for deducing properties of an
underlying population by analysis of data samples. (Examples: prediction of the election results, rent
indices, creditworthiness, etc.)

Statistical unit - an object on which the data of interest are observed. (Examples: in case of the
private housing rental index, units are the apartments, in the case of credit approval processes, it is the
customers.)
Population - the quantity of all statistical units. (Examples: For the research question- “Rent indices in
Berlin”, all rental apartments in Berlin form the population)

7
Subpopulation – a part of the population. (Examples: the 2-bedroom apartments form a subpopulation
in the topic “Rent indices in Berlin”, the female customers form a subpopulation concerning the
“customer creditworthiness”.)
Sample - is the actual subpopulation surveyed.
Variable - observed characteristic of the statistical units in the sample. (Examples: For the topic “Rent
indices in Berlin” the net rent, the size of the apartment, the number of rooms, etc. can be of interest,
and for “customer creditworthiness” features, such as net income, gender, family status etc.)
Realization - concrete value of the variable.

Data can be subdivided into qualitative and quantitative data; numerical data can be continuous or
discrete:
Categorical or Qualitative Data - non-numerical quantities, verbal expressions. (Examples: eye
colour, first name, favourite film, political party.)
Quantitative Data - numerical measurements (Examples: size, weight, income, rest.)
Continuous data - data that can be any value in a given interval. (Examples: size, weight, income.)
Discrete Data - data with finite or countable infinite number of feature expressions. (Examples:
number of children, final grade, gender.)

The course of a statistical analysis can be divided into five phases.

Planning - In the planning phase, the questions for the investigation are formulated. Based on this, the
following questions are clarified: Which information is required? Which properties should be
investigated? What is the population? How should the properties be raised?
Data collection - The data required for the investigation can be collected in various ways, for example
in the form of surveys, observations or automatic recording.
Data preparation - In the data preparation phase, the data are merged to create the population. The
data can be ordered and possibly transformed. Plausibility checks are carried out, outliers are
identified and, if necessary, missing values are replaced.
Analysis - The methods of descriptive, explorative and inferential statistics are used in the analysis
phase.
Interpretation - The interpretation of the data and analysis results depends on the given question. The
goal is to condense the findings so that the relevant information can be derived.
These phases build on each other, but they do not necessarily go through in the given order. For
example, it is quite common to go back to data collection in terms of data preparation or from the
analysis phase to the data preparation.

8
The typical order of a statistical investigation starts with a description and exploration of the data. In
this process the data is depicted graphically and tabularly via diverse diagrams, frequency tables or
variation curves and the characteristics of interest are described by certain key figures such as the
mean. Some statistical studies end after the data description and exploration, and for some, this phase
is used for first impressions about the information content of the data and is therefore only a
preliminary study for further analysis or modeling steps. If you, for example, want to know how many
planes had a departure delay of less than 15 minutes between 15 and 30 minutes and 30 minutes at a
major airport this morning, one would first collect the relevant data. The next step is then to determine
the number of aircraft with the specified delay, the so-called absolute frequencies. Dividing the
absolute frequencies with the total number of aircraft in the record, you get the so-called relative
frequencies for flight delays. By dividing the sum of all flight delays with the total number of aircraft
in your sample, you get the average flight delay, i.e. the mean of the delays. Both, of passengers and
airline hope of course, that the average delay is very low and that more than 90% of all aircraft have a
delay of less than 15 minutes. If this is not the case in the morning, so it is highly likely that there may
be further delays in the afternoon, as many planes fly only between two airports. Thus already such
simple analyses have great practical value. Below, the basic methods for data description and
exploration are presented.

A frequency analysis is only useful if in the sample same characteristics occur multiple times (for
example, in a sample with the exam notes, the same notes often occur repeatedly). Starting from a
sample of size n with the values

𝑥1 , 𝑥2 , … , 𝑥𝑛 (e.g. exam notes for 10 students 1, 1, 1, 1, 1, 2, 2, 2, 3, 5; thus n=10),

in which k different values (characteristics) occur

𝑥1 , 𝑥2 , … , 𝑥𝑘 (for the upper example k=4 or 𝑥1 = 1; 𝑥2 = 2; 𝑥3 = 3; 𝑥4 = 5),

it describes the absolute number of times that each data point is included in the sample. Thus applies
to the sum of the absolute frequencies

∑𝑘𝑖=1 𝑛𝑖 = 𝑛1 + 𝑛2 + ⋯ + 𝑛𝑘 = 𝑛 (in the upper example 𝑛1 = 5; 𝑛2 = 3; 𝑛3 = 1; 𝑛4 = 1).

The relative frequency is calculated by dividing the absolute frequency with the number of sample
values (sample size) n:
𝑛𝑖 5
ℎ𝑖 = 𝑛
(in the upper example ℎ1 = 10 = 0.5; ℎ2 = 0.3; 𝑛3 = 0.1; 𝑛4 = 0.1).

It holds that

∑ ℎ𝑖 = ℎ1 + ℎ2 + ⋯ + ℎ𝑘 = 1.
𝑖=1

9
The most common forms of the graphs of frequency of characteristics are the rod, column, bar, and pie
charts (see Figure 1).
Pie chart: In a pie chart, the angle, and the surfaces of the circular sectors are proportional to the
absolute or the relative frequency.
Bar chart: In a bar chart, the characteristic values are plotted on the vertical axis and the beam length
corresponds to the relative or absolute frequencies.
Column chart: In a column and rod chart, the characteristic values are plotted on the horizontal axis
and the column, or the rod size is the relative or absolute frequency of the respective forms.

Figure 1: a) pie, b) bar and the c) column chart.

The frequency distribution function of a sample describes the distribution of individual sample values:
ℎ𝑖 𝑥 = 𝑥𝑖
𝑓(𝑥) = { 𝑓𝑜𝑟
0 𝑜𝑡ℎ𝑒𝑟 𝑥.
Thus, the value of the frequency function for all the sampled values occurring is equal to its relative
frequency, and all values which have not occurred in the sample have the function value zero.

The empirical distribution function F(x) is a cumulative distribution function describing the sum of the
relative frequencies of all sample values which are less than or equal to x. As such, F(x) describes
what proportion of data is less than or equal to an x of interest:

𝐹(𝑥) = ∑ 𝑓(𝑥𝑖 ).
𝑥𝑖 ≤𝑥

The empirical distribution function is a so-called step function and has for each of the k different
characteristic values of the sample examined (𝑥1 , 𝑥2 , … , 𝑥𝑘 ), each a jump of height f(xi)=hi. Thus, this
function has the final value one (see Figure 2).

10
 Figure 2: Empirical distribution function F(x) of a sample.

The frequency analysis of a sample can be illustrated by the following table:

Sample values Absolute Relative Empirical distribution
frequency frequency function
xi ni hi F(x)

x1 n1 h1= n1/n n1/n

ascending
order

x2 n2 h2= n2/n n1/n+ n2/n

… … … …
xC nC hC= nC/n 1
Table I: Frequency analysis.
Example:
The score obtained on a test was as follows:
63, 63, 76, 76, 76, 80, 80, 80, 82, 82, 84, 84, 85, 86, 86.
Determine the relative frequencies and draw them as pie and bar chart. Calculate and plot the
empirical distribution.
Solution: Sample length: n = 15; number of different values: k = 7.
Sample values Absolute Relative Empirical distribution
frequency frequency function
xi ni hi F(x)

63 2 h1= n1/n=2/15=0.13 n1/n=0.13

76 3 h2= n2/n=3/15=0.20 n1/n+n2/n=0.13+0.20=0.33
80 3 0.20 0.53
82 2 0.13 0.67
84 2 0.13 0.80
85 1 0.07 0.87
86 2 0.13 1

11
For large sample sizes with many different values, the values are grouped in so-called classes.

Figure 3: Classification of the sample values in classes.

The minimum value that can belong to a class is called the lower class limit and the maximum value,
the upper class limit. The centre of each class is called class mean 𝒙̃ . This is calculated from the
upper and lower class limits as

𝐶𝑙𝑎𝑠𝑠 𝑚𝑒𝑎𝑛 𝒙̃ = (𝑙𝑜𝑤𝑒𝑟 𝑐𝑙𝑎𝑠𝑠 𝑙𝑖𝑚𝑖𝑡 + 𝑢𝑝𝑝𝑒𝑟 𝑐𝑙𝑎𝑠𝑠 𝑙𝑖𝑚𝑖𝑡)/2.

General rules for the grouping of a large sample
a) First, the sample values are sorted by size, and the smallest (xmin) and the largest (xmax) sample
value are determined.
b) The classification should be selected in a way so that the class mean values are described by
simple numbers and the class widths equal (whenever possible).
c) The number of classes should be usually less than 30. As a rule of thumb, the number of
classes can be calculated as 𝐶 ≈ √𝑛 (for a sample with 50 < 𝑛 > 500). Another rule of
thumb for the class number C is 𝐶 ≤ 5 lg 𝑛.
12
 d) If a sample value corresponds precisely to one of the two boundary points of a class, it should
be considered in only one of the two classes.
Example:
The score obtained on a test was as follows:

63, 63, 76, 76, 76, 80, 80, 80, 82, 82, 84, 84, 85, 86, 86.
Group the values into classes and calculate the class averages.
Solution: To get a first orientation for the classification, one first determines the minimum and
maximum of the sample:

Min = 63  as the lowest class boundary one can select the first small round number = 60;

Max = 86  as the uppermost class limit one can select the first major round number = 90;

The rule of thumb provides a first estimate of the number of classes: C ≈ √n = √15 = 3.9 ≈ 4 classes.

Class width = (upper – lower class limit)/number of classes=(90-60)/4=7.5

By choosing 4 classes, the class boundaries or class means would have somewhat unfavourable
values- so would e.g. the boundaries of the first be 60 and 67.5, with the class mean 63.75. This is
computationally inconvenient because the class mean appears to be no "simple number". To simplify
the class division, one can choose e.g. 6 or 3 classes.

With 6 classes the class width is = (supreme class limit - lowest class limit) / number of classes = (90-
60) /6 = 5.
In 6 classes the class limits and mean values are as follows:
Class Lower class Upper class ̃𝑖
Class mean 𝒙
limit limit

Class 1 60 65 ̃1 =(lC1+uC1)/2 = (60+65)/2=62.5

𝒙
Class 2 65 70 67.5
Class 3 70 75 72.5
Class 4 75 80 77.5
Class 5 80 85 82.5
Class 6 85 90 87.5

The number of sample values that are in the i-th class is called the absolute class frequency 𝑛𝑖 . For
the sum of the absolute frequencies, it holds
𝐶

∑ 𝑛𝑖 = 𝑛1 + 𝑛2 + ⋯ + 𝑛𝐶 = 𝑛.
𝑖=1

The relative class frequency is calculated by dividing the absolute class frequency with the number of
sample values (sample size) n:

13
 𝑛𝑖
ℎ𝑖 = .
𝑛
Note that the sum of the relative frequencies should fulfil
𝐶

∑ ℎ𝑖 = ℎ1 + ℎ2 + ⋯ + ℎ𝐶 = 1.
𝑖=1

The absolute and relative class frequencies can be plotted in a variety of ways, such as bar or a column
chart. The relative class frequencies can also be plotted as a pie chart (see Figure 1).
Example:
Determine the absolute and the relative frequencies for the example from section 2.2 (points awarded
on a test) by splitting the data into 6 and 3 classes.
Solution: Sample size: n = 15. Note that if a data value corresponds precisely to one of the two
boundary limits of a class, one should count the value only to one of the two classes.
The absolute and the relative frequencies data grouping in 6 classes are listed in the following table:
Lower class Upper class Class mean Absolute class Relative class
limit limit frequency frequency
̃𝑖
𝒙 ni hi

>60 ≤65 ̃1 =(lC1+uC1)/2

𝒙 n1=2 h1= n1/n
=(60+65)/2=62.5 =2/15=0.13
>65 ≤70 67.5 0 0.00
>70 ≤75 72.5 0 0.00
>75 ≤80 77.5 6 0.40
>80 ≤85 82.5 5 0.33
>85 ≤90 87.5 2 0.13
The absolute and the relative frequencies for data grouping into 3 classes:
Lower class Upper class Class mean Absolute class Relative class
limit limit frequency frequency
̃𝑖
𝒙 ni hi

>60 ≤70 ̃1 =(lC1+uC1)/2

𝒙 n1=2 h1= n1/n
=(60+70)/2=65 =2/15=0.13
>70 ≤80 75 6 0.40
>80 ≤90 85 7 0.47

The frequency distribution function of a grouped sample describes the relative class frequency
depending on the class middle value of each class:

ℎ𝑖 ̃𝑖 , 𝑖 = 1,2, . . . , 𝐶,
𝑥=𝒙
𝑓(𝑥) = { 𝑓𝑜𝑟
0 𝑜𝑡ℎ𝑒𝑟 𝑥.

14
 Example:
Determine the frequency distribution function for the example from section 2.2 (points scored).
Solution: The frequency distribution function is defined by the following values:
Class 1 = 0.13; Class 2 = 0; Class 3 = 0; Class 4 = 0.40; Class 5 = 0.33; Class 6 = 0.13.

The empirical distribution function or cumulative distribution function F(x) of the grouped sample is
̃𝑖 are less than or equal x:
the sum of the relative class frequencies whose middle values 𝒙

̃𝑖 ).
𝐹(𝑥) = ∑ 𝑓(𝒙
̃𝑖 ≤𝑥
𝒙

The graphical representation of the distribution function F(x) of a grouped sample is in the form of a
step function.

Figure 4: Empirical distribution function F(x) of a grouped sample.

Example:
Determine the empirical distribution function for the example from section 2.2 (points scored) with 3
classes.
Solution: The empirical distribution function is equal to the sum of the relative frequency and is
defined by the following values: Class 1 = 0.13; Class 2 = 0.13 + 0.4 = 0.53; Class 3 = 1.
The frequency analysis of a grouped sample can be illustrated by the following table:
Lower class Upper class Class mean Absolute class Relative class Empirical distribution
limit limit frequency frequency function
̃
𝒙𝑖 ni hi F(x)

lC1 uC1 ̃1 =(lC1+uC1)/2

𝒙 n1 h1 = n1/n n1/n
ascending sorted

lC2 uC2 ̃2 =(lC2+uC2)/2

𝒙 n2 h2 = n2/n n1/n+ n2/n
… … … … … …

lCC uCC ̃𝐶 =(lCC+uCC)/2

𝒙 nC hC= nC/n 1

Table II: Frequency analysis of a grouped sample.

15
Example:
The current flat size (m2) of 40 students surveyed was as follows:

15 17 17 18 20 24 26 27 33 34
35 44 45 49 49 50 52 55 57 62
63 64 67 75 75 77 79 80 85 86
87 87 88 98 137 138 138 140 147 155

Determine the absolute and relative frequencies of this sample. Calculate the empirical distribution
function.
Solution: Sample length: n = 40.

Min = 15 ⇨ Lower class limit = 10; Max = 155 ⇨ Upper class limit = 160.
For a first estimate of the number of classes the rule of thumb is used:
C ≈ √𝑛 = √40 = 6.3 ≈ 6 classes.
Class width = (upper class limit − lower class limit )/number of classes = (160 − 10)/6 = 25.
Lower class Upper class Class mean Absolute class Relative class Empirical distribution
limit limit frequency frequency function
̃𝑖
𝒙 ni hi F(x)

10 35 ̃1 =(lC1+uC1)/2
𝒙 n1=11 h1= n1/n n1/n=0.28
=(10+35)/2=22.5 =11/40=0.28
35 60 47.5 8 0.20 n1/n+
n2/n=0.28+0.20=0.48
60 85 72.5 10 0.25 0.73
85 110 97.5 5 0.13 0.85
110 135 122.5 0 0.00 0.85
135 160 147.5 6 0.15 1.00

The graphs of frequency distributions give a first impression of the location and the data mean,
skewness, symmetry or outliers. A distribution is symmetric if the right and left half of the distribution
are nearly mirror images of each other. Significantly asymmetric distributions are called skewed
distributions. A distribution is right-skewed if the majority of the sample values is on the left side, and
a distribution is left-skewed, if the majority of the sample values is on the right side.

Figure 5: A symmetrical a), left-skewed b) and right-skewed c) distribution.

16
Measures of location describe numerically the central tendency of the data. The main measures of
location are the mean (arithmetic average), median and mode. The different measures of location are
suitable differently depending on the data.

The average value 𝑥̅ of a sample of size n is the arithmetic mean of the sample values:
𝑛
1 1
𝑥̅ = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ) = ∑ 𝑥𝑖 .
𝑛 𝑛
𝑖=1

The mean value is unsuitable for the qualitative data, with the exception of the data converted into
forms such as the binary form (0,1).
The mean is the best known and most commonly used measure of location. A disadvantage of the
mean is that it is strongly influenced by a few very large values or a few very small values.
Example:
The score obtained on a test was as follows:

63, 63, 76, 76, 76, 80, 80, 80, 82, 82, 84, 84, 85, 86, 86.
Determine the mean value.

1 1
Solution: 𝑥̅ = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ) = × 1183 = 78.9.
𝑛 15

Determining the mean for grouped data

The data that are available (for example, as a table with the class frequencies) only in grouped form,
the average can be calculated with the help of the class average values 𝒙 ̃𝑖 and the relative class
frequencies ℎ𝑖 :
𝐶 𝐶
𝑛𝑖 𝑥̃𝑖
𝑥̅ = ∑ = ∑ ℎ𝑖 𝑥̃𝑖 .
𝑛
𝑖=1 𝑖=1

Median is the value that lies in the middle of a data sample. To determine the median, the sample
values are first sorted in an ascending order. In practice, since the sample size n may be an even or an
odd number, in the odd case, the median is the value that lies in the middle of an ordered sample, and
for even n, the median is the arithmetic average of the two neighbouring middle sample values:

𝑥(𝑛+1)/2 , 𝑓𝑜𝑟 𝑛 𝑜𝑑𝑑,

𝑀𝑒𝑑𝑖𝑎𝑛 = {1
(𝑥 + 𝑥𝑛/2+1 ), 𝑓𝑜𝑟 𝑛 𝑒𝑣𝑒𝑛.
2 𝑛/2
The main characteristic of the median is that a half of the data lies below, and the other half lies above
the median. Compared to the mean value the median has the advantage that it is easier to interpret and
that it is not influenced by very large or very small values in the sample.

17
Example:
Determine the median for the example from section 2.4.1 (test score).
Solution: Since the number of sampling values is an odd number, so the median is
( 𝑛 + 1)/2 = (15 + 1)/2 = 8. value of the ordered sample=80.

The mode (also called the modal value) is the sample value that occurs in the sample most frequently.
If every characteristic has the frequency 1, there is no mode.

𝑀𝑜𝑑𝑒 (𝑚𝑜𝑑𝑎𝑙 𝑣𝑎𝑙𝑢𝑒) = 𝑐ℎ𝑎𝑟𝑎𝑐𝑡𝑒𝑟𝑖𝑠𝑡𝑖𝑐 𝑡ℎ𝑎𝑡 𝑜𝑐𝑐𝑢𝑟𝑠 𝑚𝑜𝑠𝑡 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑡𝑙𝑦.

The mode has the advantage that it can always be used (even with qualitative data). One disadvantage
is that it does not exist when all data values have the same frequency.
Example:
Determine the mode for the example from section 2.4.1 (achieved score on a test).
Solution: The sample values 76 and 80 have the same frequency and the largest. Thus, the mode is not
unique.
Determining the mode with grouped data
For the data that are present only in the grouped form (for example, as a table with the class
frequencies), firstly, the class with the highest frequency is determined. As a mode the class mean
value is then used.

The best-known measures of dispersion or a spread (variability) are the standard deviation S and the
variance 𝑆 2 . The variance measures the dispersion of data about its mean value, but has as a result of
squaring the disadvantage that it does not have the same unit as the data itself. The standard deviation
measures the dispersion of the data around the mean with the same unit.

The variance of the sample values 𝑥1 , 𝑥2 , … , 𝑥𝑛 is calculated as

𝑛
2
1
𝑆 = ∑(𝑥𝑖 − 𝑥̅ )2 ,
𝑛−1
𝑖=1

and the standard deviation of a sample as

𝑆 = √𝑆 2 .
Example:
The score obtained on a test was as follows:

63, 63, 76, 76, 76, 80, 80, 80, 82, 82, 84, 84, 85, 86, 86.
Determine the standard deviation.

Solution: Sample length: n = 15; mean: 𝑥̅ = 78.9.

18
 𝑛
2
1
𝑆 = ∑(𝑥𝑖 − 𝑥̅ )2
𝑛−1
𝑖=1
1
= × [(63 − 78.9)2 + (63 − 78.9)2 + ⋯ + (85 − 78.9)2 + (86 − 78.9)2 + (86 − 78.9)2 ]
15 − 1
1
= × 743.75 = 53.1
14

𝑆 = √𝑆 2 = √53.1 = 7.3

Calculation of the variance for grouped data

For the data that are available (for example, as a table with the class frequencies) only in grouped
form, the variance is calculated with the help of the class averages ̃
𝒙𝑖 and the relative class frequencies
ℎ𝑖 :
𝐾
2
𝑛
𝑆 = ̃𝑖 − 𝑥̅ )2 ℎ𝑖 .
∑(𝒙
𝑛−1
𝑖=1

Quantiles can be interpreted as threshold values dividing the data set in two parts. A certain proportion
of the values (p-percentage) is smaller than the quantile, the rest is greater ((1-p)- percentage). Thus,
for example, for the 25% quantile of all values, it applies that at least 25% of all values are smaller
than this value and at least 75% of all values are greater than this value. The median is thus the 50%
quantile. Other important quantiles are the lower and the upper quartile (25% and 75% quantile) and
deciles (10%, 20%, 30%, ..., 90% quantile), as well as the 5% and 95% quantile.
The p quantile (0 < p <1) of a distribution can be determined from the ordered list of data, taking into
account the following rule:
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑣𝑎𝑙𝑢𝑒𝑠 ≤ 𝑝 𝑞𝑢𝑎𝑛𝑡𝑖𝑙𝑒 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑣𝑎𝑙𝑢𝑒𝑠 ≥ 𝑝 𝑞𝑢𝑎𝑛𝑡𝑖𝑙𝑒
≥ 𝑝, ≥ 1 − 𝑝.
𝑛 𝑛
Quantiles can be estimated graphically using the empirical distribution function. For this purpose, one
inserts a horizontal line that intersects with the value p on the vertical axis. If the horizontal line then
also intersects with a vertical line of the step function, the p quantile can be determined uniquely. If it
encounters a step of the step function, the p quantile is defined as an average of the two sample values
defining the step (see Figure 6).

Figure 6: Example of graphic determination of quantiles.

19
One of the best known applications for quantiles comes from the paediatric, the branch of medicine
that deals with the medical care of baby’s, children and adolescents. Even for babies, data on weight,
length and head circumference are regularly collected. The Robert Koch Institute, for example, has
determined various quantiles for the body length from the observed data on the health of children and
adolescents aged 0 to 24 months. If one draws these quantiles on a chart with respect to the child age,
one gets the so-called percentile curves (Figure 7).

Figure 7: Percentile curves for body length (in cm) for boys aged 0 to 24 months (source: Robert
Koch-Institute, 2013).

Example:
Determine the 10% quantile, the 25% quantile, the 50% quantile, the 75% quantile and the 95%
quantile for the example of section 2.2.3 (the current apartment size for 40 students).

Solution: Sample length: n = 40.

The 10% quantile =0.1×n=0.1×40=4. Wert=18m2.

The 25% quantile =0.25×40 =10. Wert=34m2.
The 50% quantile =0.50×40 =20. Wert=62m2.
The 75% quantile =0.75×40 =30. Wert=86m2.
The 95% quantile =0.95×40 =38. Wert=140m2.
Comment: Please note that in the cases where p × n is an integer, alternatively, the mean value
between the value of p × n and the next upper value can be taken - in the case of the 10% quantile -

20
the average between 10th and 11th value. For discrete data, such a determining of a quantile makes
little sense, since the average value may not necessarily be an interpretable value (e.g. the average
value of exam marks 2.7 and 3 might not be particularly useful).

The convenient way for presenting the sample quantiles (25% quantile 50% quantile 75% quantile)
and the two extreme values (minimum and maximum) is called a box plot. The box plot describes the
essential characteristics of distributions in a very simple way. It is obtained by inscribing the minimum
and maximum values over the range of considered distribution (where outliers are excluded) and
connecting these with a line which is interrupted by a "box": this marks the area between the two
quartiles (25% and 75% quantile). The median of the distribution is shown as a line in the box.

Median
Smallest value of the Largest value of the
as a line in the box
sample (excl. sample (excl.
outliers) outliers)

25% quantile 75% quantile

start of the box end of the box

value range
Figure 8: Box-Plot.

Using the box plots one can very quickly get a sense of the distribution of sample values, such as
whether the distribution is symmetrical or not, and how wide the observations spread. In particular, the
box plot is very suitable for comparing different distributions.
Example:
The exam marks of 50 students were as follows:

Grade 1.0 1.3 1.7 2.0 2.3 2.7 3.0 3.3 3.7 4.0 5.0
Number
of 5 6 4 3 8 4 4 2 7 5 2
students

Draw the box plot.

Solution:
Min: 1.0 25% quantile: 1.7; 50% quantile: 2.3 75% quantile: 3.7 Max: 5.0

21
In many statistical applications, we are interested in two or more variables at the same time. For
example, one could investigate whether there is a relationship between exam points and the semester
of the students, or ask the question whether there is a difference in party preferences for men and
women. For such evaluations, it is necessary to collect multidimensional data, i.e. to collect different
variables. In this chapter we will deal mainly with bivariate or the two-dimensional data, i.e. data in
which two variables were collected together.

Bivariate frequency tables are tables containing the absolute or the relative frequencies of the
possible combinations of the two variables being studied. Such tables are commonly referred to as
bivariate frequency distribution.
Examples:
a) Party preference in a survey with 200 respondents:

x y SPD Grüne CDU/CSU Rest Sum

Female 35 25 25 15 100
Male 30 10 40 20 100
Sample Size: 200
b) Own car brand in a survey with 100 respondents:

x y VW Mercedes BMW Rest Sum

Age 18-25 3 2 2 18 25
Age 26-35 10 2 3 10 25
Age 36-55 10 3 10 2 25
Age > 55 5 10 5 5 25
Sample Size: 100
The data on the own car brand and ages consist of the absolute frequencies for all combinations of
the sample values. This data can be represented graphically as stacked columns:

Figure 9: Absolute frequencies of car brands per age category of the owner.
Although the absolute frequencies in the upper tables provide a good overview of which car brand in h
age group is popular, they might be less interesting in some applications than the relative numbers.
22
Dividing the values with the absolute frequency by the sample size (= 100 respondents), we obtain the
relative frequencies of the car brands per age category of the owner:

x y VW Mercedes BMW Rest Sum

Age 18-25 3% 2% 2% 18% 25%
Age 26-35 10% 2% 3% 10% 25%
Age 36-55 10% 3% 10% 2% 25%
Age > 55 5% 10% 5% 5% 25%
28% 17% 20% 35% 100%

Figure 10: Relative frequencies of car brands per age category of the owner.

The relative frequencies give only an indication of how often the different combinations of the
sample values occur. Another interesting question would be, for example, is there a difference in the
age structure across different car brands? To answer this question, you simply divide the number of
owners of the particular car in the particular age category (i.e. the absolute frequencies), by the total
number of owners for that brand. The results are the so-called conditional relative frequencies:

x y VW Mercedes BMW Rest

Age 18-25 11% 12% 10% 51%
Age 26-35 36% 12% 15% 29%
Age 36-55 36% 18% 50% 6%
Age > 55 18% 59% 25% 14%
Sum: 100% 100% 100% 100%

Figure 11: The conditional relative frequencies for different car brands.

23
The contingency table is a common way for analysing the bivariate (two-dimensional) frequency
distribution of two variables of interest. The contingency table is constructed so that each column
corresponds to realisations of one variable and each row corresponds to realisations of the other
[Link] entries in the table are the frequencies (absolute or relative frequency) for the possible
combinations of values of the two variables.
We extend the notation of the frequencies as follows:
xi, yj – Realisations or the values of the variables x and y
k – Number of different realisations of x (xi, i=1,…, k)
l – Number of realisations of y (yj, j=1,…, l)
nij – Absolute frequency of the common occurrence of xi and yj
hij – Relative frequency of the common occurrence of xi and yj
ni, hi – Absolute or relative marginal frequency for xi
nj, hj – Absolute or relative marginal frequency for yj

The contingency table for the absolute frequency is given by:

x y 𝑦1 𝑦2 … 𝑦𝑙 Sum
𝑥1 n11 n12 … n1l n1
𝑥2 n21 n22 … n2l n2
: : : … : :
𝑥𝑘 nk1 nk2 … nkl nk
Sum n1 n2 … nl n

The so-called absolute marginal frequencies ni and nj are obtained by summing the values in the
row or column.
It holds:
𝑙 𝑘 𝑘 𝑙

𝑛𝑖 = ∑ 𝑛𝑖𝑗 , 𝑛𝑗 = ∑ 𝑛𝑖𝑗 and 𝑛 = ∑ ∑ 𝑛𝑖𝑗 .

𝑗=1 𝑖=1 𝑖=1 𝑗=1

The placeholder „“ indicates that the marginal frequency does not consider the levels of the
corresponding variable. The marginal frequencies ni are the frequencies with which the levels of the
variable x are observed, regardless of the levels of the variable y. Accordingly, ni are the frequencies
with which the levels of y are observed, regardless of the levels of x.

24
Example:
A survey among 50 students of a statistical lecture shows that 10 of them live with their parents, while
40 students have moved out of home. If one adds at the end of the semester the results of the exam, the
following data arises:
xLives with parents yExam passed Frequency
Yes Yes 4
Yes No 6
No Yes 36
No No 4

Determine the contingency table for the variables "Lives with parents" and "Exam passed"!
Solution:
Exam passed
Lives with Yes No Sum
parents
Yes 4 6 4+6 = 10
No 36 4 36+4 = 40
Sum 40 10 50

From the contingency tables various information can be derived. Dividing all the entries by the total
number of observations "n", the result is a contingency table with relative frequencies hij. If, instead, in
each row, the values are divided by the corresponding marginal frequency ni, the relative frequency
of y results separately for the realisations or the values of x (the so-called conditional relative
frequency). Similarly, to find the relative frequency of x one calculates separately according to the
realisations of y.
Example:
a) For the above example, determine the contingency table with relative frequencies.
b) What percentage of students who live with their parents, has passed the exam? What proportion is
among the students who have moved out? Interpret the result.
Solution: a) The contingency table with relative frequencies is:
Exam passed
Lives with Yes No Sum
parents
Yes 8% 12% 20%
No 72% 8% 80%
Sum 80% 20% 100%

25
b) To answer the question, one calculates the conditional relative frequencies of the variable "Exam
passed" for the students who live with their parents and those who have moved out. For this purpose,
one divides the corresponding absolute frequencies by the related marginal frequency ni (the entries in
the first row by 10 and in the second by 40). The line totals 100% each, the column sum makes in this
case no sense.
Exam passed
Lives with Yes No Sum
parents
Yes 40% 60% 100%
No 90% 10% 100%
Sum 100%
Interpretation: The percentage of students who have passed the exam is significantly higher among
those who have moved out from home (90%) than among those who live with their parents (40%).
This suggests that there is a connection between the housing and the exam results.

To examine the question of whether there is a connection between the levels of the two variables x and
y (it is also called association), one can consider the squared contingency. For this purpose the so-
called 𝜒 2 -coefficient is formed:
𝑘 𝑙 𝑛𝑖 ⋅ 𝑛𝑗 2 𝑘 𝑙 2
(𝑛𝑖𝑗 − 𝑛 ) (𝑛𝑖𝑗 − 𝑛̃𝑖𝑗 ) 𝑛𝑖 × 𝑛𝑗
2
𝜒 = ∑∑ 𝑛𝑖 ⋅ 𝑛𝑗 = ∑∑ ; 𝑛̃𝑖𝑗 =
𝑛̃𝑖𝑗 𝑛
𝑖=1 𝑗=1 𝑛 𝑖=1 𝑗=1

If no relationship between the variables is present, i.e. they are independent, in each cell of the
contingency table for the absolute frequencies nij exactly the value 𝑛̃𝑖𝑗 = 𝑛𝑖 ⋅ 𝑛𝑗 /𝑛 is expected.
2
Therefore, 𝜒 is the square sum of the relative deviations among the observed frequencies and for the
case of independent variables expected frequencies.
With increasing sample size the values of 𝜒 2 are going to increase. Apart from the special case of
completely independent variables (here applies 𝜒 2 = 0) one cannot draw a direct conclusion from the
value of 𝜒 2 . Therefore, we consider the Pearson contingency coefficient:

𝜒2
𝐾=√ 2 .
𝜒 +𝑛

The Pearson contingency coefficient is always less than one. The maximum value that it takes
(𝐾𝑘𝑚𝑎𝑥 ) depends on the number of occurrences of the considered levels. To account for this, the
corrected Pearson contingency coefficient is considered:

𝐾 min(𝑘, 𝑙) − 1
𝐾𝑐𝑜𝑟𝑟 = ; 𝐾𝑘𝑚𝑎𝑥 = √ , 0 ≤ 𝐾𝑘𝑜𝑟𝑟 ≤ 1.
𝐾𝑘𝑚𝑎𝑥 min(𝑘, 𝑙)

It holds that
k – Number of rows of the contingency table (or number of different realisations of the variable x)
l – Number of columns of the contingency table (or number of different realisations of the variable y).

26
The corrected Pearson contingency coefficient can take values between 0 to 1, whereas 𝐾𝑐𝑜𝑟𝑟 = 1
corresponds to the strongest possible connection.
The measures of association 𝜒 2 , K and 𝐾𝑐𝑜𝑟𝑟 have a comparative nature. That is, that the values are
not clearly interpretable except for some special cases (for example, 𝐾𝑘𝑜𝑟𝑟 = 1). Only a comparison of
the values of two or more sub-populations allows an interpretation of the strength of the relationship
among the studied variables. It should be noted that e.g. 𝜒 2 is sensitive to the sample size, which is
why the selected sub-populations should have the same scope as possible. To give information about a
direction of action of the two variables x and y (x is large, then y is also large), none of the dimensions
of the measures (𝜒 2 , K and 𝐾𝑐𝑜𝑟𝑟 ) can be used.
Example:
Does the above example outline a connection between the fact that a student lives with his parents and
passed the exam? Calculate the corrected Pearson contingency coefficient!
Solution:
For clarity, here that is repeated "true" contingency table:

Exam passed
Lives with Yes No Sum
parents
Yes 4 6 10
No 36 4 40
Sum 40 10 50

In the first step 𝜒 2 is determined. For this, the contingency table of the expected values in case of
independence of the two variables is calculated (𝑛̃𝑖𝑗 = 𝑛𝑖 ⋅ 𝑛𝑗 /𝑛).

Exam passed
Lives with Yes No Sum
parents
Yes 10 ⋅ 40 10 ⋅ 10
=8 =2 10
50 50
No 40 ⋅ 40 40 ⋅ 10
= 32 =8 40
50 50
Sum 40 10 50

Now 𝜒 2 has to be determined:

𝑘 𝑙 𝑛𝑖 ⋅ 𝑛𝑗 2
(𝑛𝑖𝑗 − 𝑛 ) = (4 − 8) + (6 − 2) + (36 − 32) + (4 − 8)
2 2 2 2
𝜒2 = ∑ ∑ 𝑛𝑖 ⋅ 𝑛𝑗 8 2 32 8
𝑖=1 𝑗=1 𝑛
= 2 + 8 + 0.5 + 2 = 12.5.
Thus the Pearson contingency coefficient can be calculated (𝑛 = 50):

27
 𝜒2 12.5 12.5
𝐾=√ 2 =√ =√ = 0.45.
𝜒 +𝑛 12.5 + 50 62.5

Since both variables have two levels, the number of rows in the contingency table is 𝑘 = 2 and the
number of columns is 𝑙 = 2, and thus the minimum of the two dimensions of the contingency table is
min(𝑘, 𝑙) = min(2,2) = 2. 𝐾𝑚𝑎𝑥 is then given by

min(𝑘, 𝑙) − 1 2−1 1
𝐾𝑚𝑎𝑥 = √ =√ =√ .
min(𝑘, 𝑙) 2 2

The corrected Pearson contingency coefficient can now be determined:

𝐾
𝐾𝑐𝑜𝑟𝑟 = = √2 ⋅ 0.45 = 0.63.
𝐾𝑚𝑎𝑥
Interpretation: There is a medium high connection between the fact that a student lives with his
parents and the result in the exam.

28
In many statistical applications, the key question is whether or not there is a relationship between two
parameters or variables. The list of practical examples is nearly endless: the question of whether there
is a clear link between the rent and apartment size for a particular region, sales figures in a store and
the time of day, the resale value of a car and its age or mileage, the number of products sold per day
and price difference compared to competitors' offers, etc. If a relationship between the two parameters
of interest has been detected, the question is often whether the detected relationship has a certain
direction. For example, one can assume that the more you invest in your own education, the higher the
starting salary is. This means that there is a relationship between the duration of the education and the
starting salary, and that this relationship is positive (the larger the one, the bigger the other). If one
were to measure the strength of the relationship or dependences by means of appropriate correlations,
or to describe the context mathematically, the question is how strong the association is, and whether
the dependence can be defined by a mathematical equation. For the upper example "duration of the
education vs. entry level salary"- whether this relationship is weak or strong, whether it is
approximately linear or non-linear, and whether it is possible to calculate a forecast for the future entry
level salary in a particular industry by means of a suitable sample and a correlation and regression
analysis.

The value pairs of two variables can be graphically represented by a scatter plot. The resulting pattern
of the points in the scatter plot, provides initial indications of possible relationship between the two
variables (see Figure 12).

Figure 12: Different scatter plots.

For example, it is obvious from the patterns shown in Figure 13 that there is a relationship between the
average annual temperature at a location and its altitude. It is often of interest not only to know
whether there is a relationship between the two variables, but also how strong this relationship is. The
question of the strength of the context can be answered by means of a correlation analysis.

29
Figure 13: Scatter plot for the relationship between the average annual temperature at a place and its
altitude above sea level. Bravais-Pearson’s Correlation Coefficient

Correlation analysis is used to quantify the relationship between two variables, i.e., to measure the
strength of association between two variables. A known measure of the strength of the linear
relationship is the empirical correlation coefficient of Bravais-Pearson r. It is defined by:
𝑆𝑥𝑦
𝑟= , whereas
𝑆𝑥 𝑆𝑦
𝑛
1
𝑆𝑥𝑦 = ∑(𝑥𝑖 − 𝑥̅ ) × (𝑦𝑖 − 𝑦̅) is the 𝐞𝐦𝐩𝐢𝐫𝐢𝐜𝐚𝐥 𝐜𝐨𝐯𝐚𝐫𝐢𝐚𝐧𝐜𝐞 and
𝑛−1
𝑖=1
𝑛 𝑛
1 1
𝑆𝑥 = √ ∑(𝑥𝑖 − 𝑥̅ )2 and 𝑆𝑦 = √ ∑(𝑦𝑖 − 𝑦̅)2 ,
𝑛−1 𝑛−1
𝑖=1 𝑖=1
1
which are for the standard deviations of the variables x and y. Since the factor 𝑛−1
occurs in the
numerator and denominator of r, it can be shortened. Thus the empirical correlation coefficient of
Bravais-Pearson can be reduced to:
∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ ) × (𝑦𝑖 − 𝑦̅)
𝑟= .
√∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ )2 × √∑𝑛𝑖=1(𝑦𝑖 − 𝑦̅)2

The empirical covariance is the sum of the deviation products (𝑥𝑖 − 𝑥̅ )(𝑦𝑖 − 𝑦̅) and can be both a
positive and a negative number:

(𝐱𝐢 − 𝐱̅) (𝐲𝐢 − 𝐲̅) (𝐱𝐢 − 𝐱̅)(𝐲𝐢 − 𝐲̅)

positive positive positive
negative positive negative
negative negative positive
negative positive negative
Table III: Signs of products in the calculation of the empirical covariance.

The sign of the empirical correlation coefficient r is determined by the sign of the empirical
covariance. Here, r is always between -1 and 1 (−1 ≤ 𝑟 ≤ 1).

30
 r<0 r>0

r≈0 r≈0

Figure 14: Correlation coefficient for the different structures of point clouds.

Positive correlation (r > 0): In the case of a positive correlation, the general trend in the drawn points
is from the lower left to the upper right, or the higher the values of a variable, the higher the values
of the others (see Figure 14). Thus, e.g. a positive correlation between the size of an apartment and the
rental price is given.

Negative correlation (r < 0): In the case of a negative correlation, the general trend in the drawn
points is from the upper left to the lower right, or the higher the values of a variable, the lower the
values of the others (see Figure 14). Thus, e.g. a negative correlation between the age of a car and its
resale value is given.

No correlation (r ≈ 𝟎): There is no linear correlation (see Figure 14).

The closer the are to a straight line in a scatter plot, the closer the correlation coefficient of Bravais-
Pearson is to 1 when the line has a positive slope. Conversely, the closer the points are to a straight
line in a scatter plot, the closer the correlation coefficient of Bravais-Pearson is to -1 when the line has
a negative slope.
The strength of the correlations can be roughly divided into:

“weak correlation“ if |r| < 0.5

“medium correlation“ if 0.5 ≤ |r| < 0.8 and

“strong correlation“ if |r| ≥ 0.8.

Example:
The temporal development of a car’s resale value in 103 € was as follows:

First registration,
1 year 2 years 3 years
t=0 years
50 45 33 30

How strong is the linear relationship between the resale value and the age of the car?
Solution: The strength of a linear relationship can be quantified by using the Bravais-Pearson
correlation coefficient as
31
 𝑆𝑥𝑦 ∑𝑛𝑖=1(𝑥𝑖 − 𝑥
̅) × (𝑦𝑖 − 𝑦
̅)
𝑟= = .
𝑆𝑥 𝑆𝑦 2 2
√∑𝑛𝑖=1(𝑥𝑖 − 𝑥
̅) × √∑𝑛𝑖=1(𝑦𝑖 − 𝑦
̅)

n = 4 (a total of 4 data observation pairs are available)

𝑥̅ = (0 + 1 + 2 + 3)/4 = 1.5 = mean of x

𝑦̅ = (50 + 45 + 33 + 30)/4 = 39.5 = mean of y

𝑛

∑(𝑥𝑖 − 𝑥̅ ) × (𝑦𝑖 − 𝑦̅)

𝑖=1
= [( 0 − 1.5)(50 − 39.5) + (1 − 1.5)(45 − 39.5) + ( 2 − 1.5)(33 − 39.5) + (3 − 1.5)(30 − 39.5)]
= −36
𝑛

∑(𝑥𝑖 − 𝑥̅ )2 = (0 − 1.5)2 + (1 − 1.5)2 + (2 − 1.5)2 +(3 − 1.5)2 = 5

𝑖=1
𝑛

∑(𝑦𝑖 − 𝑦̅)2 = (50 − 39.5)2 + (45 − 39.5)2 + (33 − 39.5)2 +(30 − 39.5)2 = 273
𝑖=1

𝑆𝑥𝑦 ∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ ) × (𝑦𝑖 − 𝑦̅) −36

𝑟= = = = −0.97
𝑆 𝑥 𝑆𝑦 √∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ )2 × √∑𝑛𝑖=1(𝑦𝑖 − 𝑦̅)2 √5 × √273

The Bravais-Pearson correlation coefficient indicates a strong negative correlation!

The correlation coefficient of Spearman is suitable for the variables whose values can be ordered (e.g.,
grades in the school system, position in a company, or product evaluation). Thus, Spearman’s
correlation coefficient is often used in evaluations of sociological surveys, where an arrangement of
the answers is meaningful. In contrast to the correlation coefficient of Bravais-Pearson which
measures the strength of the linear relationship, the correlation coefficient of Spearman measures the
strength of the monotonic relationship. Moreover, Spearman’s correlation coefficient is robust against
outliers.
The correlation coefficient of Spearman is based on the ranked variables. Specifically, each observed
value of both variables is assigned a rank which corresponds to its place in the ascending ordered
sample. However, some variables might have identical values, thus slightly complicating the ranking
process. Namely, to each repeating sample value the average value of the eligible ranks is assigned.
The procedure is illustrated by the following example:
Suppose that 4 students were asked for their statistics grade and their shoe size, and the answers were
as follows:
Student 1 Student 2 Student 3 Student 4
Variable x = Grade 2.3 1.7 1 1.3
Variable y = Shoe size 41 44 39 41

The ordered statistics are 1, 1.3, 1.7 and 2.3 and thus the smallest grade occupies the position 1 (thus
rank 1) and the largest the place 4 (rank 4). The ordered shoe sizes are 39, 41, 41, 44 and thus the shoe

32
size 41 occupies the positions 2 and 3. The rank of the shoe size 41 is then the arithmetic mean (2 + 3)
/2=2.5. The largest shoe size has the rank 4 in the ordered sample.
The ranks of the variables x and y, 𝑟𝑔(𝑥𝑖 ), 𝑟𝑔(𝑦𝑖 ) are as follows:

Student 1 Student 2 Student 3 Student 4

Variable x = Grade; 𝑟𝑔(𝑥𝑖 ) 4 3 1 2

Variable y = Shoe size; 𝑟𝑔(𝑦𝑖 ) 2.5 4 1 2.5

Spearman's correlation coefficient is the Bravais-Pearson correlation coefficient applied to the ranking
pairs 𝑟𝑔(𝑥𝑖 ), 𝑟𝑔(𝑦𝑖 ). Since the sum of all assigned ranks is equal to the sum of all numbers from 1 to n,
the following applies to the mean values of the ranks:
𝑛
𝑛+1 𝑛+1 (𝑛 + 1) × 𝑛
̅̅̅̅̅̅̅̅ =
𝑟𝑔(𝑥) ̅̅̅̅̅̅̅̅ =
, 𝑟𝑔(𝑦) , since ∑ 𝑖 = .
2 2 2
𝑖=1

Taking into account the upper relations, Spearman's correlation coefficient is:

𝑛+1 𝑛+1
∑𝑛𝑖=1(𝑟𝑔(𝑥𝑖 ) − ) × (𝑟𝑔(𝑦𝑖 ) − )
𝑟𝑠 = 2 2 .
𝑛
√∑𝑖=1(𝑟𝑔(𝑥𝑖 ) − 𝑛 + 1 2 𝑛
√∑𝑖=1(𝑟𝑔(𝑦𝑖 ) − 𝑛 + 1 2
2 ) × 2 )

Spearman's correlation coefficient attains values in the range -1≤ rs ≤1, where for:
 rs > 0 it indicates a uniform monotonic relationship between the variables x and y (x is large,
then y is also large and vice versa),
 rs < 0 there is a mutual monotonic relationship between the variables x and y (x is large, then y
is small and vice versa),
 rs ≈ 0 there is no monotonic relationship between the variables x and y.
Example:
Calculate the correlation coefficient according to Spearman between the statistics grades and the shoe
sizes of the students surveyed (data from section 4.1.2).

Solution:
𝑛+1 5
Sample size: n = 4, such that 2
= 2 = 2.5

Spearman’s correlation coefficient is then calculated as

Student 1 Student 2 Student 3 Student 4
Variable x = Grade; 𝑟𝑔(𝑥𝑖 ) 4 3 1 2

Variable y = Shoe size; 𝑟𝑔(𝑦𝑖 ) 2.5 4 1 2.5

(4 − 2.5) × (3 − 2.5) × (1 − 2.5) × (2 − 2.5) ×
𝑛+1 𝑛+1
(𝑟𝑔(𝑥𝑖 ) − ) × (𝑟𝑔(𝑦𝑖 ) − ) (2.5 − 2.5) (4 − 2.5) (1 − 2.5) (2.5 − 2.5)
2 2 =0 = 0.75 = 2.25 =0
𝑛+1 2 (4 − 2.5)2 (3 − 2.5)2 (1 − 2.5)2 (2 − 2.5)2
(𝑟𝑔(𝑥𝑖 ) − ) = 2.25 = 0.25 = 2.25 = 0.25
2

33
 Student 1 Student 2 Student 3 Student 4
𝑛+1 2 (2.5 − 2.5)2 (4 − 2.5)2 (1 − 2.5)2 (2.5 − 2.5)2
(𝑟𝑔(𝑦𝑖 ) − ) =0 = 2.25 = 2.25 =0
2

0 + 0.75 + 2.25 + 0 3
𝑟𝑠 = = = 0.63 .
√(2.25 + 0.25 + 2.25 + 0.25) × √(0 + 2.25 + 2.25 + 0) 2.236 × 2.121

On the basis of this correlation, one would assume a medium-strong correlation between the shoe size
and the statistical score, which of course is not particularly meaningful.

A causal relationship may or may not exist between the examined characteristics. A known example of
a "pseudo-relationship" between two variables is a statistically significant correlation between the
number of breeding pairs of storks and the human birth rate (Matthews, 2000).

Figure 15: The relationship between the number of breeding pairs of storks and the human birth rate in
103 per year (the original data comes from Matthews, 2000).
The Bravais-Pearson correlation coefficient between the two variables is r = 0.61, indicating a
medium-strength correlation. From this correlation one can ask oneself, do the storks bring the babies?
Certainly not. The study shows a statistically significant relationship between the two variables, but
there is no causality.

The relationships between the variables can often be approximated by different mathematical
functions. The search for the appropriate function, which best describes the actual data, is called
regression and the corresponding mathematical model the regression model. As a rule, one begins
with the simplest type of regression, the so-called linear regression in which the data are
approximated by a linear function.

Assuming a linear relationship between two variables, the strength of a linear relationship can be
quantified using the Bravais-Pearson correlation coefficient, based on the shape of the point cloud in a
scatter plot. If at least one weak correlation is established (r ≠ 0), the relationship between the
variables can be approximated with a compensating straight line. That is, one tries to approximate the
point cloud by a straight line which is as close as possible to the actual observations. The process for

34
finding the linear equation that best describes the actual data is called a linear regression, and the
straight line with the best fit a linear regression model.
A linear regression model has the general form

𝑓(𝑥) = 𝛼 + 𝛽𝑥.
For the value pairs of the variables x and y (𝑥𝑖 , 𝑦𝑖 ), i = 1, .., n, the following equation holds for a linear
regression model:

𝑦𝑖 = 𝛼 + 𝛽𝑥𝑖 + 𝜀𝑖 = 𝑦̃𝑖 + 𝜀𝑖 , 𝑖 = 1, … , 𝑛,

where 𝜀𝑖 is the model error or the deviation of the observed pairs from the line of best fit and 𝑦̃𝑖 =
𝛼 + 𝛽𝑥𝑖 represent the regression model or the functional form for the line of best fit.

𝜀𝑖

𝑦̃𝑖 = 𝛼 + 𝛽𝑥𝑖 ; 𝑦𝑖 = 𝑦̃𝑖 + 𝜀𝑖

Figure 16: Linear regression model.

The estimators for the parameters  and  of the linear regression model are obtained most easily
from the so-called least squares method (see, for example, Fahrmeier, 2007). The method of the least
squares is based on the idea that the mathematically best line of fit is the one in which the sum of the
quadratic deviations between the observed values and the values determined by the regression is
minimal. The parameters  and  of the linear regression model are calculated as:
𝑆𝑥𝑦 ∑𝑛𝑖=1(𝑥𝑖 −𝑥̅ )(𝑦𝑖 − 𝑦̅)
𝛽= = ,
𝑆𝑥 2 ∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ )2

𝛼 = 𝑦̅ − 𝛽𝑥̅ ,

where 𝑆𝑥𝑦 is the empirical covariance, 𝑆𝑥 2 is the variance of the variable x, and 𝑥̅ and 𝑦̅ are the means
of the variables x and y.

Example:
Using the data from the example in section 4.1.1 (the resale value of a car), calculate the parameters of
the regression line.

Solution:
Number of observed pairs: n = 4;
0+1+2+3 50 + 45 + 33 + 30
𝑥̅ = = 1.5, 𝑦̅ = = 39.5.
4 4

35
x - year 0 1 2 3
y – car value 50 45 33 30
(0-1.5)×(50-39.5) (1-1.5)×(45-39.5) (2-1.5)×(33-39.5) (3-1.5)×(30-39.5)
(𝑥𝑖 −𝑥̅ )(𝑦𝑖 − 𝑦̅) = -15.75 = -2.75 = -3.25 = -14.25

(𝑥𝑖 − 𝑥̅ )2 (0-1.5)2 = 2.25 (1-1.5)2 = 0.25 (2-1.5)2 = 0.25 (3-1.5)2 = 2.25

∑𝑛𝑖=1(𝑥𝑖 −𝑥̅ )(𝑦𝑖 − 𝑦̅) −15.75 − 2.75 − 3.25 − 14.25 −36

𝛽= = = = −7.2
∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ )2 2.25 + 0.25 + 0.25 + 2.25 5

𝛼 = 𝑦̅ − 𝛽𝑥̅ = 39.5-(-7.2)×1.5=50.3

The linear regression model for the time value of the Volvo XC60 is: 𝑦̃𝑖 = 50.3 − 7.2 𝑥𝑖 .

The coefficient of determination R2 is a frequently used measure for the quality of the linear regression
model. It measures how well the linear model describes the data. In a simple regression (only one
independent variable), R2 corresponds to the square of the Bravais-Pearson correlation coefficient:

𝑅2 = 𝑟 2.
The coefficient of determination R2 attains values between zero and one. A value of 0 means that the
linear model is poorly describing the observed data. A R2 of 1 means that the model provides the
perfect adaptation to the data, or the data is already on a straight line.

The linear regression model determined in a regression analysis

𝑦̃𝑖 = 𝛼 + 𝛽𝑥𝑖
can be used to calculate a prediction for the variable y. For this, the model parameters α and β
calculated from the observed data are used, as well as the value of the variable x, for which a
prognosis is required. The procedure is illustrated by the following example
Example:
Using the data from the example in section 4.1.1 (the resale value of a car), make an estimate of the
car time value 4 years after the initial registration.

36
Solution:
According to the line of regression for the time value it follows:

𝑦𝑖 = 50.3 − 7.2 𝑥𝑖 = 50.3 − 7.2 × 4 𝑦𝑒𝑎𝑟𝑠 = 21 500€.

Now that the simple linear regression has been considered, where y depends on just one independent
variable x, the focus will be on the multiple linear regression, where y depends on more than one
predictor or independent variable x. Thus, the multiple linear regression is more general. Since this
type of regression is a generalization, most of what was observed in the simple regression transfers to
the multiple linear regression.

Let y depend on 𝑘 independent variables, such that y is explained by a linear combination of 𝑥𝑖 for
𝑖 = 1, … , 𝑘 (note that for the simple regression 𝑘 = 1 holds). Consequently, as seen above, the
multiple linear regression has the form

𝑦 = 𝛽0 + 𝛽1 𝑥1 + 𝛽2 𝑥2 + ⋯ + 𝛽𝑘 𝑥𝑘 + 𝜀,

where 𝜀 is the model error. In addition, a transformation of the predictive variables 𝑥i is possible (see
Section 4.2.6).

The model can be evaluated for observation pairs (𝑦𝑖 , 𝑥𝑖 ) for 𝑖 = 1, … , 𝑛 and 𝑥𝑖 = (𝑥𝑖1 , … , 𝑥𝑖𝑘 ) with
the errors 𝜀𝑖 . In detail, it follows

𝑦𝑖 = 𝛽0 + 𝛽1 𝑥𝑖1 + 𝛽2 𝑥𝑖2 + ⋯ + 𝛽𝑛 𝑥𝑖𝑘 + 𝜀𝑖 = 𝑦̃𝑖 + 𝜀𝑖 , for 𝑖 = 1, … , n.

Here the intercept or regression constant is denoted as 𝛽0 and not as 𝛼 as in the simple regression. In
order to get a better grasp of how this system of equations looks like, one uses the matrix notation:
𝑦1 1 𝑥11 𝑥12 ⋯ 𝑥1𝑛 𝛽0 𝜀1
𝑦2 1 𝑥21 𝑥22 ⋯ 𝑥2𝑛 𝛽1 𝜀2
[⋮ ]=[ ][ ] + [ ⋮ ] or 𝑦 = 𝑋𝛽 + 𝜀.
⋮ ⋮ ⋮ ⋯ ⋮ ⋮
𝑦𝑛 1 𝑥𝑛1 𝑥𝑛2 ⋯ 𝑥𝑛𝑘 𝛽𝑘 𝜀𝑛

The yet to be determined (𝑘 + 1) regression coefficients are often referred to as regression

parameters. For the determination of these (𝑘 + 1) regression coefficients one uses analogously to the
simple regression the least squares method. Usually the calculations, which contain matrix algebra, are
done with the help of statistical software. A simplification of these calculations is illustrated in the
following.

To obtain the vector 𝛽, one left-multiplies the above equation on both sides with the transposed matrix
𝑋 𝑇 and the inverse of 𝑋 𝑇 𝑋. This leads to

𝑋 𝑇 𝑦 = 𝑋 𝑇 𝑋𝛽
leading to

𝛽 = (𝑋 𝑇 𝑋)−1 𝑋 𝑇 𝑦
A necessary condition for this calculation process is the nonlinearity among the predictor variables,
which guarantees that the inverse of 𝑋 𝑇 𝑋 exists.

37
As before, it is important to know how good the approximations 𝑦̃𝑖 fit to the observed results and give
therefore a quantitative estimation. For this purpose one considers again the coefficient of
determination or 𝑅 2 for the multiple linear regression. Having more than one predictor, the coefficient
of determination is no longer equal to the Bravais-Pearson correlation coefficient, but is rather equal to

𝑅 2 = 𝑆𝑆𝑅/𝑆𝑆𝑇 = 1 − 𝑆𝑆𝐸/𝑆𝑆𝑇,
with SSR standing for regression sum of squares, i.e. the sum of the squared differences between the
regression approximations 𝑦̃𝑖 and the mean 𝑦̅ of the observations 𝑦𝑖 :
𝑚
𝑆𝑆𝑅 = ∑ (𝑦̃𝑖 − 𝑦̅ )²,
𝑖=1

SST as reference to total sum of squares, i.e. the sum of the squared distances of the observations 𝑦𝑖
from their mean 𝑦̅:
𝑚
𝑆𝑆𝑇 = ∑ (𝑦𝑖 − 𝑦̅ )²,
𝑖=1

and SSE as abbreviation for sum of squared errors, i.e. the sum of the squared distances of the
approximations 𝑦̃𝑖 and the observations 𝑦𝑖 :
𝑚
𝑆𝑆𝐸 = ∑ (𝑦̃𝑖 − 𝑦𝑖 )².
𝑖=1

The multiple linear regression enables the opportunity of considering an unlimited number of predictor
variables. Moreover, this can be accompanied by a nonlinear transformation of the predictor variables.
Nonlinearity is of high importance, since it is a relevant aspect for the computation via matrix algebra
and ensures that a calculation of the parameters 𝛽𝑖 is possible. Such a derivation of predictor variables
may be a very useful tool for improving the regression results. Examples for these transformations can
result from the observation of the scatter plot, i.e. from an empirical understanding, where it can
indicate that the transformation 𝑥2 = 𝑥12 is a possibility to enhance the regression or from physical
understanding. Empirically chosen transformations may also improve the physical understanding, but
this is less important for aims that pursue only a good forecast, since it is often not of interest to know
why the forecast preforms well.

Other possible transformations can be 𝑥2 = √𝑥1 , 𝑥2 = 1/𝑥1 . In some cases, a trigonometric,

exponential or logarithmic transformation can lead to an improvement of the regression. A further
transformation, which is commonly used, is the transformation into binary variables or so-called
dummy variables. These variables consist only of the values 1 and 0. The transformation depends on
whether the original value lies above or beneath a certain threshold value 𝑣:
1, 𝑖𝑓 𝑥1 > 𝑣
𝑥2 = { .
0, 𝑖𝑓 𝑥1 ≤ 𝑣
It is also possible to derivate more than just one binary variable from one predictor variable 𝑥1 , using
further threshold values. One may ask oneself now, why it is called a linear regression for one can use
nonlinear transformations. The answer is that the regression model is linear in the to be determined
parameters 𝛽𝑖 and it is not of concern how the transformation was executed.

38
The probability is a measure of the occurrence of an event. In general, the probability of occurrence
of an event A is represented by 𝑃(𝐴) (the symbol "P" simply denotes "probability"). For example, the
probability that a six-dice will roll a six is 𝑃(6) = 1/6.
A random experiment is an experiment that can be infinitely repeated and has a well-defined set of
(random) possible outcomes. Thus, e.g. in the random experiment of throwing a dice once, exactly 6
different outcomes are possible: the outcome can be one of the numbers 1, 2, 3, 4, 5 or 6. Although
one knows what the possible outcomes are, the outcome itself is random and cannot be predicted.

The possible outcomes of a random experiment are called elementary events . All elementary
events form the sample space .
Example:
a) When throwing a dice:
Elementary events i ϵ 1, 2, 3, 4, 5, 6 for i = 1,…,6.

Sample space = {1, 2, 3, 4, 5, 6}.

b) When throwing two dice, there are a total of 36 different combinations of the numbers obtained and
thus 36 elementary events:
Result of the second dice
Result of the first dice

The set of all elementary events create the sample space of this random experiment:

(1; 1), (1; 2),…, (6; 6)}.

If one is interested only in certain events of the sample space, e.g. "the result is an even number", they
are described as subsets of the sample set.

39
Examples:
Some subsets of the experiment “throwing a dice” are:

Subset A = {6}: The result is a “6“.

Subset B = {2, 4, 6}: The result is an even number.

Subset C = {1, 3, 5}: The result is an odd number.

Subset D = {1, 5}: The result is a “1” or “5”.

As described above, events can be described as subsets of the sample space. Thus the occurrence of
the events can be described by set theory operations. The most important set theory operations and
their meanings are shown in the following figure:


A ∪ B  A ∩ B 𝐴̅  

A B A B A B
A B

A ∪ B is the union of A ∩ B is the 𝐴̅ is the complement The events A and B are

the event A and B: intersection of the or complementary mutually exclusive
Either the result is A or events A and B: event of A: (“disjoint events”):
B or A and B A and B occur A does not occur. A ∩ B=∅.
simultaneously. simultaneously.

Figure 17: Set theory operations (linkages of events) and their meanings.

Examples:

a) When throwing a dice, the sample set is given by = {1, 2, 3, 4, 5, 6}. Consider
the subsets A = {1, 3, 5}, B = {2, 4, 6} und C = {1}, thus follows:

A ∪ B = {1, 2, 3, 4, 5, 6} =

A ∩ B = ∅, A ∩ C = {1},
̅ = {2, 4, 6} = B, B
A ̅ = {1, 3, 5} = A.

The Laplace experiment is a random experiment with a finite sample space and a uniform
distribution. For example, the two possible results (elementary events) "head" and "tail" are expected
with the same absolute or relative frequency. In an "experiment" with 1000 coin throws, one would
expect 500 times "head" and 500 times "tail" to be thrown and thus the relative frequency of the two
results is

In a Laplace experiment with the sample space nall n possible elementary events
n have the same probability:

40
 1
𝑃(𝜔𝑖 ) = , (i = 1, 2, … , n).
𝑛

The Laplace probability (the "classical probability") for the occurrence of an event A is
Number of cases favourable for event A
𝑃(𝐴) = .
𝑛
Examples:
When throwing a dice, all possible results occur with the same probability:

P(1) = P(2) = ⋯ = P(6) = 1/6.

a) What is the Laplace probability for the event A = {1, 3, 5} "odd number"?
Number of cases favourable for event A 3 1
𝑃(𝐴) = = = .
𝑛 6 2
b) What is the probability of the event B = {5, 6} "the number is 5 or 6"?
Number of cases favourable for event B 2 1
P(B) = = = .
𝑛 6 3
c) When throwing two dice, there are a total of 36 possible elementary events (combinations of the
two numbers). What is the probability of the event C = "The sum of the results is equal to 11"?

Result combinations Sum of the results

For the event C = "The sum of the outcomes is equal to 11", there are exactly 2 favourable cases, the
combination (the number of the first dice = 6, the number of the second dice = 5) and the combination
(the number of the first dice = 5, the number of the second dice = 6). Therefore,

P(C) = 2/36 = 1/18.

The Laplace probability can only be applied to a limited extent, namely to Laplace experiments. As
described in the previous section, Laplace experiments are random experiments with a finite sample
set and a uniform distribution of the events. For random experiments, which are not "Laplace
experiments", one can consider their relative frequencies to conclude on the probability of events.

41
For example, from an urn with 1 white, 2 grey, and 3 black balls, a ball is randomly taken and
returned, the relative frequencies ℎ𝑛 for the drawing of a white, grey or black ball in extensive
experiments (the number of individual experiments should be sufficiently large) are expected to be as
follows:

ℎ𝑛 (white ball) ≈ 1/6, ℎ𝑛 (grey ball) ≈ 1/3, ℎ𝑛 (black ball) ≈ 1/2.

With the increasing number of attempts, the actually observed values for relative frequencies fluctuate
less and less around the "expectation values". Accordingly, the values are stabilized around the above-
mentioned expectation values for large number of repetitions of the same experiment. For example, as
shown in the figure below, the relative frequency of occurrence of "6" stabilizes around the expected
value (1/6 = 16.7%) after throwing a dice more than a thousand times.

Figure 18: The number of throws and the relative frequency of the event "6".

The property that the relative frequencies of random events stabilize by a certain value with increasing
number of experiments n is the foundation for the so-called "statistical" or "empirical" definition of
probability:

The probability 𝑃(𝐴) of a random event A is replaced by the relative frequency ℎ𝑛 (𝐴) observed in
extensive test series:

𝑃(𝐴) ≈ ℎ𝑛 (𝐴) (Number of the individual repetitions of the same experiment should be sufficiently
large).

The so-called axioms (principles) of Kolmogoroff apply to probabilities, irrespective of whether they
are determined according to the "classical" or the "statistical" definition:
Axiom 1: The probability is always greater than or equal to 0 and can be at most equal to 1 (0 means
that an event is impossible):

0 ≤ P (A) ≤ 1.

42
Axiom 2: The probability that at least one of the elementary events in the entire sample space will
occur is 1

P(Ω) = 1.

Axiom 3: For pairwise mutually exclusive events 𝐴1 , 𝐴2 , . . , 𝐴𝑘 , it holds

𝑃(𝐴1 ∪ 𝐴2 ∪ … 𝐴𝑘 ) = 𝑃(𝐴1 ) + 𝑃(𝐴2 ) + ⋯ + 𝑃(𝐴𝑘 ).

Axiom 3 is also referred to as an "addition law for mutually exclusive events". From the axioms of
Kolmogoroff the following calculation rules can be derived for probabilities:
a) The probability for an event complementary to A is:
̅ ) = 1 − 𝑃(A).
𝑃( A
b) If two events are mutually exclusive, the probability of their union is equal to the sum of their
individual probabilities:

If 𝐴 ∩ 𝐵 = ∅, then 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵).

c) For the impossible event it holds

P (∅) = 0.
Example:
The probability of a delayed flight is 0.20. What is the probability that the flight is on time?

Solution: A denotes the event "flight is delayed" with 𝑃(𝐴) = 0.2. The event "flight is punctual" is a
complementary event to A with 𝑃(𝐴̅) = 1 − 𝑃(𝐴) = 1 – 0.20 = 0.80.

According to the third probabilistic axiom, the probability of their union (A ∪ B) is equal to the sum of
the individual probabilities for two mutually exclusive events ("disjoint events") A and B:

𝑃 (𝐴 ∪ 𝐵) = 𝑃 (𝐴) + 𝑃 (𝐵) 𝑖𝑓 𝐴 ∩ 𝐵 = ∅.

If A and B are not disjoint but two arbitrary events, then the probability of their union (𝐴 ∪ 𝐵) is
equal to the sum of both singular probabilities, reduced by the common probability 𝑃 (𝐴 ∩ 𝐵):

𝑃 (𝐴 ∪ 𝐵) = 𝑃 (𝐴) + 𝑃 (𝐵) − 𝑃 (𝐴 ∩ 𝐵).

The above is often referred to as the "addition law of probability".
Example:
What is the probability that the number “6” will at least occur once if the dice is thrown twice?
Solution: The event in which “6” occurs at least once in the double throw is the case if either a) in the
first throw or b) in the second throw or c) in either throw a “6” is obtained. If
Event A: Result of the first throw = 6, and
Event B: Result of the second throw = 6,

43
is the case, then the event "in both throws a “6” is obtained" corresponds to the event A ∩ B. The
probability that a “6” is obtained in the first throw is equal to the probability that a “6” is obtained in
the second throw, or P(A) = P(B) = 1/6.
From the example "throw with two dice" (section 5.1) we know that there are a total of 36 possible
combinations of the two numbers and that the combination (6; 6) only occurs once. Thus the
probability is P (A ∩ B) = 1/36.
The likelihood that the number "6" occurs at least once if the dice is thrown twice is

P (A ∪ B) = P (A) + P (B) – P (A ∩ B) = 1/6 + 1/6 - 1/36 = 11/36.

It is often desired to know how high the probability of the occurrence of a particular event B is
assuming that another event A has already occurred. For example, in practice, it is of great
importance to estimate the probability of a customer paying an invoice-bought but not yet paid item
after the first reminder.
The probability for the occurrence of the event B under the condition that the event A has already
occurred is called conditional probability and is characterized by the symbol 𝑃(𝐵|𝐴). The
conditional probability of B under condition A is defined by the following equation:
𝑃(𝐴 ∩ 𝐵)
𝑃(𝐵|𝐴) = , 𝑃(𝐴) ≠ 0.
𝑃(𝐴)
For the above example with the purchased but not paid article one can define the events A and B as A
= “first reminder” and B = “paid”. Thus, the conditional probability 𝑃(𝐵|𝐴) = 𝑃(“paid” | “first
reminder has been sent”) is sought.

Example:
In the case of a throw of two dice, only the events with the eye sum 10 are considered. What is the
probability that the numbers of the sum are both odd?
Solution: We consider the following events:
A: “The sum is 10”
B: “The numbers of both dice are odd”
We look for the probability that the numbers of both dice are odd under the condition that the sum is
10 - this is the conditional probability P (B | A).
In the case of a throw with two dice, there are a total of 36 different combinations of diced numbers
and the event A (“sum is 10”) is obtained for exactly 3 number combinations {(5;5), (4;6), (6;4)}.
Therefore, according to the classical probability definition

P (A) = 3/36.
The event (A ∩ B): The sum of the numbers is 10 and the number are odd is only realized by the
combination of numbers (5; 5). Therefore, among the 36 possible cases for the event (A ∩ B) there is
exactly one favourable case. Thus,
1
𝑃(𝐴 ∩ 𝐵) = .
36
44
The probability that the numbers of both dice are odd in the case of a sum of 10 is thus:
𝑃(𝐴 ∩ 𝐵) 1/36 1
𝑃(𝐵|𝐴) = = = .
𝑃(𝐴) 3/36 3

The probability for the simultaneous occurrence of two events A and B can be calculated as follows:

𝑃 (𝐴 ∩ 𝐵) = 𝑃 (𝐴) × 𝑃 (𝐵 | 𝐴).
It holds that

𝑃(𝐴 ∩ 𝐵) is the probability of the simultaneous occurrence of two events A and B or the probability
of the average of two events A and B,

𝑃(𝐴) is the probability of occurrence of event A (without any condition),

𝑃 (𝐵 | 𝐴) is the probability for the occurrence of the event B with the condition that the event A has
already occurred (conditional probability of B under condition A).

Since A ∩ B = B ∩ A, it follows

𝑃(𝐴) × 𝑃(𝐵 | 𝐴) = 𝑃(𝐵) × 𝑃(𝐴 | 𝐵).

Example:
In an urn are 5 balls, including 3 white and 2 black. In succession, two balls are removed randomly
and placed sideways. How big is the chance that you draw two white balls?

Solution: We consider the two random events

A: event “the first ball is white”,
B: event “the second ball is white”.
The corresponding probabilities are then:
1. Draw: Event A occurs with probability
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑎𝑠𝑒𝑠 𝑓𝑎𝑣𝑜𝑢𝑟𝑎𝑏𝑙𝑒 𝑓𝑜𝑟 𝑒𝑣𝑒𝑛𝑡 𝐴 3
𝑃(𝐴) = = .
𝑛 5
2. Draw: Event B occurs and event A has already occurred. So there are only 4 balls in the urn,
including 2 white and 2 black balls.
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑎𝑠𝑒𝑠 𝑓𝑎𝑣𝑜𝑢𝑟𝑎𝑏𝑙𝑒 𝑓𝑜𝑟 𝑒𝑣𝑒𝑛𝑡 𝐵 𝑢𝑛𝑑𝑒𝑟 𝑐𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝐴 2 1
𝑃(𝐵 | 𝐴) = = = .
𝑛 4 2
The probability of the simultaneous occurrence of the events A and B is then

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) × 𝑃(𝐵 | 𝐴) = 3/5 × 1/2 = 3/10 = 0.3.

Assuming that the experiment is repeated often enough, one can expect that two white balls are drawn
in 30% of all cases.

45
For two stochastically independent events A and B, the occurrence of the event A is completely
independent of whether the event B has occurred or not. This also applies to the reverse case. The
following relationships apply:

𝑃(𝐴 | 𝐵) = 𝑃(𝐴) and 𝑃(𝐵 | 𝐴) = 𝑃(𝐵).

The above means that the information about whether the event B has occurred or not does not provide
any additional information with respect to the probability 𝑃(𝐴) and vice versa.
From the multiplication law (see section5.8) follows

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) × 𝑃(𝐵 | 𝐴) = 𝑃(𝐴) × 𝑃(𝐵)

or two events A and B are called stochastically independent if

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) × 𝑃(𝐵).

Stochastically independent events are often referred to as statistically independent events. A stochastic
independence is not to be confused with causality.
Example:
A dice is thrown two times.
a) What is the probability of a “6” in both cases?
b) What is the likelihood that the first throw will score a “6” and the sum of the two diced numbers
will be “12”?
Solution: The result of a throw is completely independent of the result of the previous throw and vice
versa.
a) If A is the event "6 on the 1st throw" and B the event "6 on the 2nd throw" then we have
𝑃(𝐴) = 𝑃(𝐵) = 1/6.
Since the two events A and B are completely independent, the probability of a "6" being thrown twice
is

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) × 𝑃(𝐵) = 1/6 × 1/6 = 1/36.

Events A and B are both stochastically and causally independent.
b) The sum of the two diced numbers gives a 12 only if a "6" is obtained each time. Thus the
probability is equal to the probability under a):

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) × 𝑃(𝐵) = 1/6 × 1/6 = 1/36.

The difference to a) is that the events A and B are stochastically independent, but there is a causal
relationship between the two events since the throw of the first cube co-determines the sum of the
numbers.

46
An event tree is an analytical diagram in which a series of events is analysed in a chronological way
by means of a Boolean logic (i.e. “yes”/”no” or “true”, “false” or “1”, “0” etc.). It consists of an initial
event, paths or sequences, intermediate events (branching points) and outcomes (see
Figure 19).
The two-step random process shown in

Figure 19 has two possible results after the initial event (branch points 𝐴1 and 𝐴2 ). The branches
emanating from these branching points/ nodes lead to the possible results of the second stage. Since
this is a two-step random process, the outcomes after the second stage are also the final results.
The basic rules for the calculation of probabilities using event trees are:
a) The probabilities along a path are multiplied.
b) If several paths lead to the same final result, the probability of this final result is equal to the sum of
the probabilities along those paths.

Figure 19: Event tree for a multi-stage random process.

Example:
In an urn are 5 balls, including 3 white and 2 black. In succession, two balls are removed randomly
and placed sideways. How big is the chance that you draw two uniformly coloured balls?

Solution: We consider the following events:

W: event “a white ball is drawn”,
S: event “a black ball is drawn”.
1. Stage: In the urn there are 3 white and 2 black balls

𝑃(𝑊) = 3/5, 𝑃(𝑆) = 2/5.

2. Stage: In the urn there are either a) 2 white and 2 black balls when first a white ball has been drawn,
or b) 3 white and 1 black ball when a black ball has first been drawn

𝑃(𝑊|𝑊) = 2/4 = 1/2, 𝑃(𝑆|𝑊) = 2/4 = 1/2,

47
𝑃(𝑊|𝑆) = 3/4, 𝑃(𝑆|𝑆) = 1/4.

Two equal-coloured balls can be found on the paths oWW and oSS. Thus, the probability of drawing
two uniformly coloured spheres is:

𝑃 = 𝑃(𝑜𝑊𝑊) + 𝑃(𝑜𝑆𝑆) = 3/5 × 1/2 + 2/5 × 1/4 = 0.4.

The total probability for the occurrence of the event B is obtained by adding the probabilities along
all possible paths:

𝑃(𝐵) = 𝑃(𝐴1 ) × 𝑃(𝐵|𝐴1 ) + 𝑃(𝐴2 ) × 𝑃(𝐵|𝐴2 ) + 𝑃(𝐴3 ) × 𝑃(𝐵|𝐴3 ).

A1

A2 B

A3

Figure 20: Event B is accessible over several paths.

If the event B has already occurred, then the so-called Bayes’ Theorem applies to the probability that
this event has been reached via the intermediate station 𝐴𝑖 :
𝑃(𝐴𝑖 ) × 𝑃(𝐵|𝐴𝑖 )
𝑃(𝐴𝑖 |𝐵) = .
𝑃(𝐵)

A1

A2 B

A3

Figure 21: The probability that the event B has been reached just by a certain path (e.g., via the
intermediate event 𝐴1 ) can be determined according to the Bayes’ formula.

48
Example:
The Monty Hall problem (Whitaker, 1990): "Imagine you have to choose one of three doors as a
participant in a gameshow. Behind one of the doors is the win, a car, behind the other two there are
goats. You select Door #1, and the show master, who knows what is behind the doors, opens another
door, for example Door #3, behind which a goat appears. Now he asks you if you want to stay at Door
#1 or instead choose Door #2. Should you change your position? "
Solution: The car is located with a probability of 1/3 either behind the door 1, 2 or 3. As shown in the
lower picture, the possible constellations (or the sample space Ω) in this game are:

Door 1 Door 2 Door 3

Ω={Car, Goat, Goat;

Goat, Car, Goat;
Goat, Goat, Car}.

The participant can choose each of the three doors at the beginning. The solution scheme is basically
the same regardless of the decisions made by the player and the moderator at the beginning. We
assume the player chooses the number 1 and the moderator opens the door number 3. Under this
assumption, the door to which the player switches could be the door number 2. The problem is
reduced to the question:
What is the probability in case of a change to the door number 2 on condition that the moderator has
already opened the door number 3? This can be calculated using Bayes’ formula. If
W: the number of the door behind which the car is (win),
M: the number of the door opened by the moderator,
then the probability of winning in case of a change to door number 2 according to Bayes’ formula is:
𝑃(𝑊 = 2) × 𝑃(𝑀 = 3|𝑊 = 2)
𝑃(W = Door 2|M = Door 3) = .
𝑃(𝑀 = 3)
Here is

𝑃(W = 2) = 1/3 the probability that the car is behind the door number 2,

𝑃(M = 3|W = 2) = 1 the probability that the moderator will open the door number 3, under the
condition that the car is behind the door number 2 (If the player chooses the
door 1 and the car is behind the door 2, then the moderator has no other option
than to open the door number 3),

P (M = 3) the total probability that the moderator will open the door number 3.
The total probability that the moderator will open the door number 3 is

49
𝑃(𝑀 = 3) = 𝑃(𝑊 = 1) × 𝑃(𝑀 = 3|𝑊 = 1) + 𝑃(𝑊 = 2) × 𝑃(𝑀 = 3|𝑊 = 2) +

𝑃(𝑊 = 3) × 𝑃(𝑀 = 3|𝑊 = 3) = 1/3 × 1/2 + 1/3 × 1 + 1/3 × 0 = 1/2.

This means that the probability of winning in case of a change from the door number 1 to the door
number 2 according to Bayes’ formula is:
𝑃(W = 2) × 𝑃(M = 3|W = 2) 1/3 × 1
𝑃(W = Door 2|M = Door 3) = = = 2/3.
𝑃(M = 3) 1/2
In the "change door" strategy, the probability of winning is 66.7%, which is significantly higher than
the "first choice" strategy, with a probability of winning of 33.3%.

50
A variable X is called random variable if its value is the result of a random experiment. The value x,
which X attains in the execution of the random experiment, is called realization of X. As a rule, upper-
case letters are used for random variables, and lower-case letters are used for their realizations. For
example, if the selection of persons for a study occurs by chance, one can regard this as a random
experiment. If the questions of interest are, for example, the shoe size or the foot length, these can be
regarded as random variables. Their values, that is to say the concrete shoe sizes and foot lengths, are
then the realizations of the random variables.
There are two main types of random variables:
 Discrete random variables: A random variable X is called discrete if it can only attain
finitely or countably many realizations. For example, the sum of numbers shown on the
two dice is a discrete random variable that can only take a finite number of values, namely
the values 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12. For example, one can consider the shoe size
of a randomly selected person as a discrete random variable, as there is usually only a
limited number of shoe sizes.
 Continuous random variables: A random variable X is called continuous if it can take
any value within a range. This may be an interval (e.g., [0,1]), or any real number. For
example, one can consider the foot length of a randomly selected person as a continuous
random variable whose value can be any foot length from a finite interval.

Any possible result of a random experiment, i.e. every realization 𝑥𝑖 of a random variable 𝑋 =
{𝑥1 , 𝑥2 , … , 𝑥𝑛 } occurs with a certain frequency, provided the experiment is repeated very often. As
described in chapter 5.4, the probability 𝑝𝑖 of the result 𝑥𝑖 of the random experiment can be
approximated by the relative frequency ℎ𝑛 (𝑥𝑖 ) observed in extensive test series. Accordingly,
𝑝𝑖 ≈ ℎ𝑛 (𝑥𝑖 ) holds. The number 𝑝𝑖 indicates the probability with which the result of the random
experiment has exactly the value 𝑥𝑖 :

𝑃(𝑋 = 𝑥𝑖 ) = 𝑃(𝑥𝑖 ) = 𝑝𝑖 .
The probabilities 𝑝𝑖 always attain values in the interval [0, 1]:

0 ≤ 𝑝𝑖 ≤ 1.

Combining all realizations 𝑥𝑖 and the associated probabilities 𝑝𝑖 yields a distribution table:

Realization 𝑥𝑖 𝑥1 𝑥2 𝑥3 …
Probability 𝑝𝑖 𝑝1 𝑝2 𝑝3 …

The discrete function

𝑝𝑖 𝑥 = 𝑥𝑖,
𝑓(𝑥) = { 𝑓𝑜𝑟
0 𝑜𝑡ℎ𝑒𝑟 𝑥,

51
is called the probability mass function (pmf) of a discrete random variable. The corresponding
distribution function 𝐹(𝑥) is a step-function and indicates the probability that the random variable X
attains a value which is less than or equal to 𝑥:

𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥) = ∑ 𝑓(𝑥𝑖 )

𝑥𝑖 ≤𝑥

F(x)

1
f(x n)

f(x 3)

f(x 2)
f(x 1)

x1 x2 x3 xn x

Figure 22: Stair-shaped course of the distribution function 𝐹(𝑥) of a discrete random variable.

The pmf plot shows the probability 𝑝𝑖 for each 𝑥𝑖 . For example, there are exactly 11 possible
realizations for the sum of numbers shown on the two dice (𝑥1 =2, 𝑥2 = 3, … , 𝑥11 = 12 ) with the
probabilities

𝑝1 = 𝑃(𝑋 = 2) = 1/36, 𝑝2 = 𝑃(𝑋 = 3) = 2/36, … , 𝑝11 = 𝑃(𝑋 = 12) = 1/36 and ∑𝑖 𝑝𝑖 = 1.

Thus, the pmf plot for the sum of numbers shown on the two dice has the following form:
Probability

Sum of numbers
Figure 23: Pmf plot the sum of numbers shown on the two dice.
The expected value of X is given by:

𝑋̅ = ∑ 𝑥𝑖 × 𝑝𝑖 .
𝑖

Example:
A portfolio consists of three customer loans. The loans must be repaid after exactly one year. With a
certain probability, a deptor cannot paid a debt and “defaults”. Customers can default independently.

52
 The total loan amounts and default probabilities are shown in the following table:

Customer Loan Default Probability

A 1000 € 25%
B 1000 € 20%
C 2000 € 20%

It is assumed that the customer will either fully repay the loan or not at all, i.e. in the event of a default
of the borrower, a total loss arises. When e.g. the customer A defaults, the bank incurs a loss equal to
the full loan amount = 1000 €.
a) Determine all possible losses and their probabilities at the end of the year.
b) Visualise the pmf.
c) Determine the expected average loss.
Solution: There are eight possible default scenarios for the bank:
Defaults Loss
No default 0€
A 1000 €
B 1000 €
C 2000 €
A, B 2000 €
A, C 3000 €
B, C 3000 €
A, B, C 4000 €
It can be seen that there are five possible losses: 0€, 1000€, 2000€, 3000€, 4000€.
If P(A), P(B) and P(C) represent the default probability of the customers A, B and C then 1-P(A), 1-
P(B) and 1-P(C) is the probability that the customers A, B and C will repay their credit at the end of
the term.
a) From the given data, one can calculate the associated probability of occurrence for all five loss
amounts:

𝑃(0€) = (1 − 𝑃(A)) × (1 − 𝑃(B)) × (1 − 𝑃(C)) = 0.75 × 0.8 × 0.8 = 0.48.

𝑃(1000€) = 𝑃(A) × (1 − 𝑃(B)) × (1 − 𝑃(C)) + (1 − 𝑃(A)) × 𝑃(B) × (1 − 𝑃(C))

= 0.25 × 0.8 × 0.8 + 0.75 × 0.2 × 0.8 = 0.28.
Overall, the following table and the associated probability histogram are given:

Loss Default Calculation of the probability P

0€ No default (1 − 𝑃(𝐴)) × (1 − 𝑃(𝐵)) × (1 − 𝑃(𝐶)) 48%
1000€ A or B 𝑃(𝐴) × (1 − 𝑃(𝐵)) × (1 − 𝑃(𝐶)) + (1 − 𝑃(𝐴)) × 𝑃(𝐵) × (1 − 𝑃(𝐶)) 28%
2000€ A, B or C 𝑃(𝐴) × 𝑃(𝐵) × (1 − 𝑃(𝐶)) + (1 − 𝑃(𝐴)) × (1 − 𝑃(𝐵)) × 𝑃(𝐶) 16%
3000€ A, C or B, C 𝑃(𝐴) × (1 − 𝑃(𝐵)) × 𝑃(𝐶) + (1 − 𝑃(𝐴)) × 𝑃(𝐵) × 𝑃(𝐶) 7%
4000€ A, B, C 𝑃(𝐴) × 𝑃(𝐵) × 𝑃(𝐶) 1%

53
b)

Probability
Loss

As a "cross-check" we can add up the probabilities for the individual losses. The sum must be 100%,
since at least one of the five scenarios occurs:

48% + 28% + 16% + 7% + 1% = 100%.

c) The average loss 𝑉̅ is calculated from the sum of the products of loss and probability:

𝑉̅ = 0€ × 0.48 + 1000€ × 0.28 + 2000€ × 0.16 + 3000€ × 0.07 + 4000€ × 0.01 = 850€.

Random experiments with only two different possible outcomes that can occur with constant
probabilities are called Bernoulli experiments. In such an experiment, the event 𝐴 occurs with the
probability 𝑝 = 𝑃(A) and the complementary event A ̅ with the probability 𝑞 = 𝑃(A
̅ ) = 1 − 𝑝. Thus, in
a coin throw, there are exactly two different possible outcomes, namely 𝐴 =”tail” and A ̅ =”head”,
which have the probabilities 𝑝 = 𝑃(𝐴) = 1/2 and 𝑞 = 𝑃( A ̅ ) = 1 − 1/2 = 1/2. Thus, the coin toss
is a Bernoulli experiment.

Suppose a Bernoulli experiment is repeated n times. The probability that the event 𝐴 occurs is the
same for each repetition and the results of the individual experiments are independent of each other.
For example, a coin is thrown n = 3 times, and the event 𝐴 = “head” is noted with “yes” / “no” or
“success” / “failure”. The number of “yes” in the n-time repetition is a new random variable 𝑍, which
can attain the values 0, 1, … , 𝑛. For the above example “coin toss”, the total number of “head” results
obtained can be 0, 1, 2, or 3 in a three-time trial. The open question is - how big is the probability that
the total number 𝑍 of "yes" achieved with n repetitions is exactly k? The question is, what is the
probability for exactly 𝑘 = 0, 1, 2, or 3 “yes” within n = 3 repetitions?

The probability 𝑃(𝑍 = 𝑘) that 𝑍 attains the value 𝑘 (0 ≤ 𝑘 ≤ 𝑛) is obtained from the probability mass
function of the binomial distribution:
𝑛
𝑃(𝑍 = 𝑘) = ( ) × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 .
𝑘
Here is

𝑘 the total number of "yes" for n repetitions (0 ≤ 𝑘 ≤ 𝑛),

𝑛 the number of repetitions,

𝑝 the probability that "yes" occurs in each of the n repetitions or the sample proportion (𝑝 = 𝑐𝑜𝑛𝑠𝑡),
𝑛
and ( ) is the binomial coefficient:
𝑘
54
 𝑛 𝑛! 1 × 2 × …× 𝑛
( )= = ; (0! = 1).
𝑘 𝑘! × (𝑛 − 𝑘)! (1 × 2 × … × 𝑘) × (1 × 2 × … × (𝑛 − 𝑘))

The random variable 𝑍, which represents the number of “yes” when repeating the Bernoulli
experiment n times, is binomially distributed.
The distribution table for a binomially distributed random variable has the following form:

𝑘 0 1 2 … 𝑛
𝑛
( ) × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘
𝑘 𝑛 𝑛
𝑃(𝑍 = 𝑘) 𝑛 ( ) × 𝑝 × (1 − 𝑝)𝑛−1 ( ) × 𝑝2 × (1 − 𝑝)𝑛−2 ... 𝑝𝑛
= ( ) × 𝑝0 × (1 − 𝑝)𝑛−0 1 2
0
= (1 − 𝑝)𝑛

For the same number of repetitions of the Bernoulli experiment n, (for example 𝑛 = 5 repetitions), the
greater the probability 𝑝 is that "yes" occurs in each of the 𝑛 repetitions, the greater becomes 𝑃(𝑍 =
𝑛):

𝑛 = 5; 𝑝 = 0.1 𝑛 = 5; 𝑝 = 0.5 𝑛 = 5; 𝑝 = 0.9

Figure 24: Pmf for binomial distributed random variables with different probabilities that "yes" occurs
in each of the 𝑛 = 5 repetitions (𝑝 = 0.1, 𝑝 = 0.5 und 𝑝 = 0.9).

The binomial distribution is applied when there are two alternative outcomes. For example, it is
customary to characterize a product as “A-“ or “B-goods” or as “objection-free” or “defect” in product
quality controls. Since these are only two alternatives, such quality controls can be described as a
Bernoulli experiment. In the case of quality control, which divides the randomly tested 𝑛 products
into “A- and B-goods”, the random variable “the number of products identified as “B-goods” is an
approximately binomially distributed random variable with the parameters 𝑛 (the total number of
randomly chosen products) and 𝑝 (the probability that a product is a "B-good"). A binomially
distributed random variable is often characterized by the symbol 𝐵(𝑛, 𝑝).

Example:
Consider the flip of a coin three times. The result of a throw is in each case with a probability of ½
"head" or "tail" and the throws are independent of each other. The random variable X is given by the
number of "head". If the result of the three throws is e.g. "head", "head", "tail", X attains the value 2,
since head has occurred twice.
a) Which values can X attain?
b) What is the respective probability?
c) Specify the result in the form of a distribution table and a pmf plot.
55
d) Determine the expected value of X.

Solution:
a) The experiment "three times coin toss" has the following outcomes (1 = head, 0 = tail):
000, 001, 010, 011, 100, 101, 110, 111.
Thus X can attain the values 0, 1, 2 or 3.

b) Since in each throw "head" or "tail" is thrown with a probability of 𝑝 = 1/2, the probability for
1 1 1 1
each of the mentioned outcomes is given by 2 × 2 × 2 = 8 or

𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 = 𝑝𝑛 = 1/23 = 1/8.

c) The number of thrown heads is (in the same order):

0, 1, 1, 2, 1, 2, 2, 3.
X is binomially distributed and attains the values 0, 1, 2 and 3. The probability for the cases that the
number of thrown heads is 𝑘 = 0, 1, 2 or 3 may come, e.g. from the relative frequencies, or from the
probability function of the binomial distribution:
1 × 2 × …× 𝑛
𝑃(𝑋 = 𝑘) = × 𝑝𝑘 × (1 − 𝑝)𝑛−𝑘 .
(1 × 2 × … × 𝑘) × (1 × 2 × … × (𝑛 − 𝑘))

The distribution table has the following appearance:

Number of „Heads“ 𝑘 = 0 1 2 3
1×2×3
Probability 0! × (1 × 2 × 3) 1 3 1 3 1 1
1 1 1 3× = 3× = 1× =
𝑃(𝑋 = 𝑘) 8 8 8 8 8 8
× =1× =
8 8 8
The pmf plot is given by:

d) The expected value of X is given by:

𝑋̅ = ∑ 𝑥𝑖 × 𝑝𝑖 = 0 × 1/8 + 1 × 3/8 + 2 × 3/8 + 3 × 1/8 = 1.5.

𝑖

56
A variable is called continuous, if it can take any value from a real finite or infinite interval. Thus, e.g.
the body temperature of a randomly chosen person can be regarded as a continuous random variable
whose value can be any temperature from a finite interval. In contrast to the shoe size of a randomly
selected person, which can be regarded as a discrete random variable, the foot length is a continuous
random variable. Note the following: since there are only a limited number of shoe sizes, it is possible
to calculate the probability that a randomly selected person has the German shoe size 39; however, the
foot length can be any number from a finite interval, wherefore, it is considered impossible to
calculate the probability that someone will have the foot length of e.g. exactly 25,51 𝑐𝑚. Instead, one
would try to calculate the probability that someone has the foot length around 25,51 𝑐𝑚, e.g. between
25,5 𝑐𝑚 and 25,6 𝑐𝑚. This is done by using the so-called probability density function (pdf) or short
density function 𝑓(𝑥) and the associated distribution function 𝐹(𝑥) (see Figure 25). The distribution
function 𝐹(𝑥) indicates how large the probability is that the random variable 𝑋 attains a value less
than or equal to 𝑥:
𝑥
𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥) = ∫ 𝑓(𝑥)𝑑𝑥.
−∞

Figure 25: Probability density function 𝑓(𝑥) and the distribution function 𝐹(𝑥) of a continuous
random variable.

In contrast to a discrete variable, the probability is always considered in an interval for a continuous
random variable. The probability that the value of the random variable 𝑋 lies in the interval (𝑥; 𝑥 +
𝑑𝑥) corresponds to the area under the density function 𝑓(𝑥) between 𝑥 and 𝑥 + 𝑑𝑥:

a) b)
𝑃(𝑎 ≤ 𝑋 ≤ 𝑏)

x dx x+dx x a b

Figure 26: The marked area under the density function f (x) corresponds to the probability that the
value of a random variable 𝑋 a) lies in the interval (𝑥; 𝑥 + 𝑑𝑥); b) lies between 𝑎 and 𝑏.

The probability that a continuous random variable 𝑋 takes a value between 𝑎 and 𝑏, that is, 𝑃(𝑎 ≤
𝑋 ≤ 𝑏), e.g. 𝑃(25.5𝑐𝑚 ≤ 𝑓𝑜𝑜𝑡 𝑙𝑒𝑛𝑔𝑡ℎ ≤ 25.6𝑐𝑚), corresponds to the area under the density
function 𝑓(𝑥) between 𝑎 and 𝑏:

57
 𝑏
𝑃(𝑎 ≤ 𝑋 ≤ 𝑏) = 𝐹(𝑏) − 𝐹(𝑎) = ∫ 𝑓(𝑥)𝑑𝑥; 𝑎 < 𝑏.
𝑎

Since the value of a continuous random variable is between −∞ and ∞ with certainty, the
corresponding probability applies
∞
𝑃(−∞ ≤ 𝑋 ≤ ∞) = ∫ 𝑓(𝑥)𝑑𝑥 = 1.
−∞

The normal distribution is the most well-known distribution with numerous practical applications.
Thus, empirical distributions such as e.g. measurement errors in meteorological data sets, deviations
from the actual value in the serial production of devices, as well as measured values such as size,
weight, length, body temperature and the like can be approximated very well by the normal
distribution. In addition, many other distributions can be approximated by the normal distribution for
large data sets.

The distribution of a random variable 𝑋 with the density function

1 1 𝑥−𝜇 2
− ( )
𝑓(𝑥) = 𝑒 2 𝜎 ,
√2𝜋 𝜎
and the distribution function
𝑥 1 𝑡−𝜇 2
1 − ( )
𝐹(𝑥) = ∫ 𝑒 2 𝜎 𝑑𝑡,
√2𝜋 𝜎 −∞

is called Gaussian distribution. The parameters or ratios of the normal distribution are the mean (or
the expected value) 𝜇 of the basic population and the standard deviation 𝜎 of the population. A random
variable 𝑋 is called normally distributed if it has the density of the Gaussian normal distribution.
This is indicated by the symbol 𝑋~𝑁(𝜇, 𝜎).

The parameters 𝜇 and 𝜎 of the population are not known in most cases, since one cannot examine the
population, but only a sample. This is illustrated by the following example: Assuming you are
interested in knowing the probability of a healthy person having a body temperature below 36 ° C.
Since one cannot determine the body temperature of all healthy people in the world, one would first
draw a representative sample from the population (for example, the study by Mackowiak, Wasserman
and Levine, 1992 and Shoemaker, 1996). On the basis of the data from Shoemaker (1996), the body
temperature is a normally distributed random variable with an average value of 𝑥̅ = 36.8 °𝐶 and a
standard deviation 𝑠 of 𝑠 = 0.4 ° 𝐶. Starting from the assumption that the sample under investigation
represents the basic population, one would take the sample values 𝑥̅ and 𝑠 as estimated values for the
unknown parameters 𝜇 and 𝜎 of the total population in a next step. The probability that a healthy
person has a body temperature below 36 ° C is then obtained from the distribution function

𝑥 1 𝑡−𝑥̅ 2 𝑥 1 𝑡−36.8 2
1 − ( ) 1 − ( )
𝐹(𝑥) = 𝑃(𝑋 ≤ 36°𝐶) = ∫ 𝑒 2 𝑠 𝑑𝑡 = ∫ 𝑒 2 0.4 𝑑𝑡.
√2𝜋 𝑠 −∞ √2𝜋 0.4 −∞

58
 a) b)

f(x) s05

s
s5

m-smm+s

Figure 27: Gaussian normal distribution: a) Symmetry with respect to the line 𝑥 = µ; b) the standard
deviation 𝜎 determines the height and width of the bell curve.

The Gaussian density function 𝑓(𝑥) has a bell shape ( “Gaussian bell curve”) with the maximum at
𝑥 = µ and the inflection points at 𝜇 ± 𝜎. The density function is symmetric around the point 𝑥 = µ
∞
and is normalized, that is, the area under the density function has the value 1 (∫−∞ 𝑓(𝑥)𝑑𝑥 = 1).
Since f (x) is symmetric, the expectation value is 𝐸(𝑋) = 𝜇.

For a Gaussian bell curve, the mean value 𝜇 defines the position of the maximum and the standard
deviation 𝜎 defines both the height and the width of the bell curve (see

Figure 27). Namely, the larger the standard deviation 𝜎, the lower the maximum (the height of the bell
curve is smaller) and the flatter the curve decreases (the width of the bell curve is larger). For bell-
shaped distributions, the following empirical rule applies:
 about 68% of the values are within a standard deviation from the mean value,
 about 95% of the values are within two standard deviations from the mean value,
 about 99.7% of the values are within three standard deviations from the mean value.

The general Gaussian normal distribution can be transformed to the so-called standard normal
distribution. If 𝑋~𝑁(𝜇, 𝜎) is a normally distributed random variable with the parameters 𝜇 (mean
value) and 𝜎 (standard deviation), then the standardized random variable
𝑋−𝜇
𝑍=
𝜎
is standard normally distributed with the parameter values µ = 0 and 𝜎 = 1 or 𝑍~𝑁(0, 1).

𝑋~𝑁(𝜇, 𝜎) = 𝑁(5, 1.5) 𝑍~𝑁(𝜇, 𝜎) = 𝑁(0, 1)

59
x z
Figure 28: Transformation of a normal distributed 𝑁(5, 1.5) into a standard normal distributed
𝑁(0, 1) random variable. Thus, e.g. for 𝑥 = 6.5 the standardized variable 𝑍 = (𝑋 − 𝜇)/𝜎 = (6.5 −
5)/1.5 = 1.
Since the standard normal distribution is a special form of the normal distribution, it has a bell shape
as the Gaussian normal distribution, is axisymmetric and has the absolute maximum at µ = 0 and the
inflection points at ± 1. The corresponding cumulative distribution function is:
𝑧
1 1 2
𝛷(𝑧) = 𝑃(𝑍 ≤ 𝑧) = ∫ 𝑒 −2𝑡 𝑑𝑡.
−∞ √2𝜋

The following relationship exists between the distribution functions 𝛷(𝑧) (distribution function of the
standard normal distribution) and 𝐹(𝑥) (distribution function of the normal distribution)
𝑥−𝜇
𝐹(𝑥) = 𝛷(𝑧) = 𝛷 ( ) = 𝑃(𝑋 ≤ 𝑥).
𝜎
Thus, for a normally distributed random variable 𝑋~𝑁(𝜇, 𝜎), the probabilities can be calculated using
the distribution function 𝛷(𝑧) of the standard normal distribution. This was of great value, especially
in the times before computer-assisted data processing since the probabilities 𝛷(𝑧) = 𝑃(𝑍 ≤ 𝑧) are
given in tabular form (see Table 1 in the Appendix). For example, if one is interested in the probability
that a normally distributed random variable 𝑋~𝑁(𝜇, 𝜎) = 𝑁(5, 1.5) (see Figure 28) attains a value
smaller than or equal to 6.5, 𝑃(𝑋 ≤ 6.5), then this can very easily be answered by using the tabular
values 𝛷(𝑧) of the standard normal distribution:

𝑃(𝑋 ≤ 6.5) = 𝐹(𝑥) is wanted.

𝑥−𝜇 𝑥−𝜇 6.5−5
Since (𝑥) = 𝛷(𝑧) = 𝛷 ( ) , the value 𝑧 is calculated by 𝑧 = = = 1.
𝜎 𝜎 1.5

From Table 1 in the Appendix, the probability sought for 𝑧 = 1 follows

𝛷(1) = 0.8413 𝑜𝑟 𝑃(𝑋 ≤ 6.5) = 84.13%.

The basic rules for the calculation of the probabilities using the tabulated distribution function of the
standard normal distribution are presented in the following section.

The probabilities 𝑃(𝑋 ≤ 𝑥) for a normally distributed random variable 𝑋~𝑁(𝜇, 𝜎) can be calculated
using the table of the cumulative distribution function 𝛷(𝑧) of the standard normal distribution. As
mentioned in the previous section, this was of great value in the times before computer-assisted data
processing since neither the integral 𝐹(𝑥) nor 𝛷(𝑧) can be calculated analytically, but only by special
numerical methods. Nowadays, all statistical software programs can output the probabilities
automatically, which allows for quick analysis. If, however, a tabular calculation of the probability is
necessary, then the following scheme follows:

a) Standard score 𝑧 calculation, i.e., the standard transformation - the standard score is found by
dividing the corresponding deviation from the mean by the standard deviation:
𝑥−𝜇
𝑧= .
𝜎

60
b) If 𝑧 ≥ 0, the desired value 𝛷(𝑧) can be read immediately from Table 1 in the Appendix. If 𝑧 < 0,
the desired value can be calculated from the tabular value for 𝛷(𝑧) as 𝛷(−𝑧) = 1 − 𝛷(𝑧).
It holds:
𝑥−𝜇
(1) 𝑃 (𝑋 ≤ 𝑥) = 𝐹(𝑥) = 𝛷 ( ) = 𝛷(𝑧) (see Figure 29)
𝜎

𝑥−𝜇
(2) 𝑃 (𝑋 ≥ 𝑥) = 1 – 𝑃 (𝑋 ≤ 𝑥) = 1 − 𝐹(𝑥) = 1 – 𝛷 ( ) = 1 − 𝛷(𝑧) (see Figure 29)
𝜎

𝑃 (𝑋 ≥ 𝑥) = 1 − 𝐹(𝑥)

𝑃 (𝑋 ≤ 𝑥) = 𝐹(𝑥)

𝑥 x

Figure 29: The probability 𝑃 (𝑋 ≤ 𝑥) corresponds to the blue-marked area under the probability
density function, and the probability 𝑃 (𝑋 ≥ 𝑥) corresponds to the white-marked area.
𝑏−𝜇 𝑎−𝜇
(3) 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) = 𝐹(𝑏) − 𝐹(𝑎) = 𝛷 (
𝜎
)− 𝛷( 𝜎
) = 𝛷(𝑏 ∗ ) − 𝛷(𝑎∗ ) (see Figure 30)

𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏)
= 𝐹(𝑏) − 𝐹(𝑎)

𝑎 𝑏 x
Figure 30: The probability 𝑃 (𝑎 ≤ 𝑋 ≤ 𝑏) corresponds to the blue area under the probability
density function.

(4) 𝑃 (|𝑋 − µ| ≤ 𝑘𝜎) = 𝑃 (µ − 𝑘𝜎 ≤ 𝑋 ≤ µ + 𝑘 𝜎) = 2 × 𝛷(𝑘) − 1

The upper equation refers to the probability that the deviation from the mean value |𝑋 − µ| lies
within 𝑘 standard deviations (𝑘 is any number).

Example:
In a series production of gaming pieces, it is assumed that the height of the produced piece is normally
distributed in good approximation with a mean value of 5 cm and a standard deviation of 1 cm.
a) What is the probability of a height under 4.5 cm?
b) What is the probability of a height above 5.3 cm?
c) What is the probability of a height between 3.8 and 5.2 cm?

61
d) What is the minimum height of the 10% of the highest pieces?
e) At a maximum tolerance for a height of 2 cm, which proportion of the pieces is expected to be
sorted out?

Solution:
The height of the pieces produced is normally distributed with 𝑋~𝑁(5, 1).
𝑥−𝜇 4.5 − 5
𝑎) 𝑃 (𝑋 ≤ 4.5) = 𝛷 ( ) = 𝛷( ) = 𝛷(−0.5) = 1 − 𝛷(0.5) = 1 − 0.6915 = 0.3085
𝜎 1
= 30.8 %
𝑥−𝜇 5.3 − 5
𝑏) 𝑃 (𝑋 ≥ 5.3) = 1 – 𝛷 ( ) = 1−𝛷( ) = 1 − 𝛷(0.3) = 1 − 0.6179 = 0.3821
𝜎 1
= 38.2 %
5.2 − 5 3.8 − 5
𝑐) 𝑃 (3.8 ≤ 𝑋 ≤ 5.2) = 𝛷 ( )−𝛷( ) = 𝛷(0.2) − 𝛷(−1.2)
1 1
= 𝛷(0.2) − (1 − 𝛷(1.2)) = 𝛷(0.2) + 𝛷(1.2) − 1 = 0.5793 + 0.8849 − 1 = 0.4642 = 46.4 %

𝑥−𝜇 𝑥−5
𝑑) 𝑃 (𝑋 ≥ 𝑥) = 1 – 𝛷 ( ) = 0.1 → 𝛷 ( ) = 0.9 →
𝜎 1
From Table 1 it follows 𝛷(𝑧) ≈ 0.9 𝑓𝑜𝑟 𝑧 = 1.28 (since 𝛷(1.28) = 0.8997 ≈ 0.9)
𝑥−𝜇 𝑥−5
= = 1.28 → The minimum height of the 10% of the highest pieces 𝑥 ≈ 1.28 + 5
𝜎 1
= 6.28 𝑐𝑚
𝑒) 𝑃 (|𝑋 − µ| ≤ 𝑘𝜎) = 𝑃 (|𝑋 − 5| ≤ 𝑘 × 1);

𝑀𝑎𝑥𝑖𝑚𝑢𝑚 𝑡𝑜𝑙𝑒𝑟𝑎𝑛𝑐𝑒: 2𝑐𝑚 = 𝑘 × 1 → 𝑘 = 2

𝑃 (|𝑋 − µ| ≤ 𝑘𝜎) = 2 × 𝛷(𝑘) − 1 = 2 × 𝛷(2) − 1 = 2 × 0.9772 − 1 = 0.9544

Approximately 95.4% of all pieces are within the tolerance limits. Thus it can be expected that
approximately 4.6% of all produced pieces will be sorted out.

62
A sample represents a set of data collected from a population so that one can make inferences about
the population. As such, the sample itself serves to gain information about the population. For
example, if one is interested in the proportion of defective products in a production plant, the
estimation would be based on a randomly drawn sample from the population. How well this value
describes the population depends on many factors, e.g. the type of sampling, the quality of the data
obtained or the sample size. Moreover, the proportion of defective products obtained from the sample
provides only an estimator for the unknown true value. In general, when using sample to make
inferences about the population, the following questions arise:

 How can we estimate the unknown parameters of the total population?

 How accurate and how stable are such estimates?
The unknown parameters can be estimated by the so-called point estimation. However, the point
estimation does not allow any conclusions as to the accuracy and stability of the estimation. These
questions are addressed by the so-called interval estimation. The basics of point and interval
estimation are presented in the following sections.

The aim of the point estimation is to provide as accurate as possible estimator for the unknown
parameters of the population. The unknown parameters may be, e.g. the parameters of an assumed
distribution, or any distribution, or the statistical parameters, e.g. correlation or the p-quantile. For
example, assume that 𝑋 is a normally distributed random variable 𝑋~𝑁(𝜇, 𝜎), then the point
estimation problem is reduced to the determination of the parameters 𝜇 and 𝜎 of the normal
distribution. A statistic is called “unbiased” if it neither systematically overestimates nor
underestimates the correct value, but rather provides the correct value. The term “point estimation”
comes from the fact that the estimated value of a parameter corresponds to a point on the real line (see,
for example, Papula, 2011).

If an estimate has been found for an unknown parameter, the question is how exact is this estimate, or,
what is a plausible estimate? To check whether the estimated parameter values are plausible, the so-
called confidence intervals are formed. These indicate an estimated range of values which is likely to
include an unknown population parameter. Therefore, the term “interval estimate” is used for such
parameter estimates.

To determine a confidence interval, a certain confidence level (in practice, usually 𝛾 = 95 % or

𝛾 = 99 %) is selected. The significance level or the probability of error 𝛼 is then 𝛼 = 1 − 𝛾 (at
𝛾 = 95 % the probability of error is 𝛼 = 5 %). For a given significance level 𝛼, the upper and lower
bounds of the confidence intervals 𝐺𝑢 and 𝐺𝑙 formed from a sample provide a (1 − 𝛼) confidence
interval if

𝑃(𝐺𝑙 ≤ 𝐺𝑢 ) = 1,

𝑃(𝐺𝑙 ≤ 𝜃 ≤ 𝐺𝑢 ) = 1 − 𝛼.

63
 Confidence interval
boundaries

𝜃
𝐺𝑙 𝐺𝑢 Real line

Confidence interval

Figure 31: Confidence interval for the parameter 𝜃.

A confidence interval also provides information on the exact nature of a parameter estimate, namely,
the smaller the confidence interval, the more accurate the parameter estimate.

Estimate for the mean value

The mean value 𝑥̅ of the sample provides a suitable estimated value µ̂ for the unknown mean value µ
of the probability distribution of the random variable 𝑋
𝑛
1 1
µ ≈ µ̂ = 𝑥̅ = ∑ 𝑥𝑖 = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ).
𝑛 𝑛
𝑖=1

The estimate itself is a realization of the estimation function

𝑛
1
𝑋̅ = ∑ 𝑋𝑖 .
𝑛
𝑖=1

𝑋𝑖 , 𝑖 = 1, . . , 𝑛 are random variables, which have all the same distribution as the random variable 𝑋
under investigation. It holds

𝐸(𝑋̅ ) = 𝜇 and 𝑉𝑎𝑟(𝑋̅) = 𝜎 2 /𝑛.

The upper properties of the estimation function mean that the estimated value for the unknown mean
value µ is unbiased and that the variance of the estimation function decreases with an increase in
sample values.

Estimate for the variance

The variance of the sample 𝑠² provides a suitable estimate 𝜎̂ 2 for the unknown variance σ²:
𝑛
2 2
1 2
𝜎 ≈ 𝜎̂ = 𝑠 = ∑(𝑥𝑖 − 𝑥̅ )2 .
𝑛−1
𝑖=1

The corresponding estimation function is

𝑛
1
2
𝑆 = ∑(𝑋𝑖 − 𝑋̅)2 .
𝑛−1
𝑖=1
64
The estimated value for the unknown variance σ² is unbiased. Therefore,

𝐸(𝑆 2 ) = σ2 .

An estimate of the sample proportion p of a binomial distributed population

As described in Section 6.2.1, the binomial distribution is applied in practice when there are two
alternative decisions with constant probabilities (e.g., quality control with the outputs “objection free”
and “defect”, or quantification of a risk with the outputs “damage above the threshold” and “damage
below the threshold”, etc.). If one were interested in a quality control for the proportion of defective
products in a production plant, one would first draw a random sample from the population. A quality
control with two possible outputs (𝐴 and A ̅ ) with constant probabilities (for example “objection free”
and “defect”) can be regarded as a Bernoulli experiment. The random variable “the number of
defective products among 𝑛 randomly chosen products” can be assumed an approximately binomially
distributed variable. The event 𝐴 occurs in a Bernoulli experiment with the probability 𝑝 = 𝑃(𝐴) and
the complementary event A ̅ with the probability 𝑞 = 𝑃( A ̅ ) = 1 − 𝑝.

The relative frequency for the event 𝐴 in the case of an 𝑛 -fold execution of a Bernoulli experiment
provides a suitable estimated value 𝑝̂ for the unknown parametrical value 𝑝 of the binomially
distributed total population:
𝑘
𝑝 ≈ 𝑝̂ = ℎ(𝐴) = .
𝑛
Here are

𝑝 parameter value 𝑝 of the binomially distributed population (unknown, therefore estimated),

𝑝̂ estimated proportion value of the parameter value 𝑝,

𝑛 the sample size or number of repetitions of the Bernoulli experiment (e.g., 𝑛 = 10 products tested),

𝑘 the total number of “yes” among 𝑛 repetitions (if 𝐴 = “product defective” in quality control, then 𝑘
is the total number of defective products in 10 tested products).
If the parameter 𝑘 is defined as a random variable 𝑋 as follows

𝑋 = “the total number of “yes” among 𝑛 repetitions of the Bernoulli experiment“,

Then the corresponding estimation function for the parameter value 𝑝 of the binomially distributed
total population is

𝑃̂ = 𝑋/𝑛.

The expected value of 𝑋 is 𝐸(𝑋) = 𝑛𝑝 = 𝜇, and the variance 𝑉𝑎𝑟(𝑋) = 𝑛𝑝 (1 − 𝑝) = 𝜎 2 .

Confidence Interval for an Unknown Proportion Value p Using a Large Sample
In the case of quality control, the proportion of defective products in a production plant was
examined. Because most quality controls are designed to label a product as “A-good” or “B-good”, the
random variable “number of B-goods” among 𝑛 randomly selected products can be approximated by a
binomially distributed variable. As described in Section 6.2.1, a binomial distributed random variable
is often characterized by the symbol 𝐵(𝑛, 𝑝). For this example, 𝑛 is the total number of randomly
selected products, and 𝑝 is the proportion of “B-goods” in the total population. Since the parametrical
value 𝑝 of “B-goods” is generally unknown in the population, a confidence interval can also be formed
for this parameter, which contains the true value with a certain confidence.

65
The starting point for determining a (1 − 𝛼) confidence interval for the parameter 𝑝 (proportion value)
of a binomially distributed population is the assumption that the sample has a “large” sample scope.
For the large samples it can be assumed that the random variable 𝑋 = “the number “B-goods” among
𝑛 randomly selected products” is a nearly normally distributed random variable with a mean value
µ = 𝑛𝑝 and a standard deviation 𝜎 = √𝑛𝑝 (1 − 𝑝). The standardized random variable
𝑋−µ 𝑋 − 𝑛𝑝
𝑈 = =
𝜎 √𝑛𝑝 (1 − 𝑝)

is approximately standard normally distributed with a mean of µ = 0 and a standard deviation of σ = 1

(𝑈~𝑁(0,1)). The random variable 𝑋 = “the number “B-goods” among 𝑛 randomly chosen products”
can be written as 𝑋 = 𝑛𝑝̂ using a concrete sample, where 𝑝̂ is the sample proportion value. Thus the
random variable 𝑈 can be written as follows:
𝑛𝑝̂ − 𝑛𝑝 𝑛(𝑝̂ − 𝑝)
𝑈 = = .
√𝑛𝑝 (1 − 𝑝) √𝑛𝑝 (1 − 𝑝)

From the condition

𝑃(−𝑧1−𝛼/2 ≤ 𝑈 ≤ 𝑧1−𝛼/2 ) = 1 − 𝛼

a “provisional” (1 − 𝛼) confidence interval for the unknown parameter 𝑝 (proportion value) of a

binomially distributed population is obtained by elementary transformations:

√𝑛𝑝 (1 − 𝑝) √𝑛𝑝 (1 − 𝑝)
𝑝̂ − 𝑧1−𝛼/2 ∙ ≤ 𝑝 ≤ 𝑝̂ + 𝑧1−𝛼 ∙ .
𝑛 2 𝑛
The (1 − 𝛼) confidence interval for the unknown parameter 𝑝 is obtained by replacing the unknown
parameter 𝑝 in the interval limits by the estimated value 𝑝̂ determined from the concrete sample

𝑝̂ (1 − 𝑝̂ ) 𝑝̂ (1 − 𝑝̂ )
[𝑝̂ − 𝑧1−𝛼/2 ∙ √ , 𝑝̂ + 𝑧1−𝛼/2 ∙ √ ].
𝑛 𝑛

A summary of the most important:

Based on a data sample, a (1 − 𝛼) confidence interval for an unknown proportion value 𝑝 using a
large sample is determined step by step as follows:
Known parameters:

- Sample size 𝑛 (𝑛 > 30)

- Sample proportion value 𝑝̂ = 𝑘/𝑛, where 𝑘 represents the total number of positive results for the
sought event A. (For example, if 𝐴 = “B-good”, then 𝑘 is the total number of “B-goods” in the
sample.)

The (1 − 𝛼) confidence interval for the unknown parameter 𝑝 can be determined stepwise as follows:

1) Calculation of the parameter 𝑝 (proportion value) from the sample

66
 𝑘
𝑝̂ = ℎ(𝐴) = .
𝑛
2) Check the condition for a large sample:

𝑛𝑝̂ (1 − 𝑝̂ ) > 9.
If this is fulfilled, the further steps can be calculated, if not, then the sample is not long enough to
perform a confidence interval determination.

3) Determination of the quantiles of the standard normal distribution 𝑧1−𝛼/2 for the chosen confidence
level (usually 95 % and 1 − 𝛼 = 0.95 → 𝛼 = 0.05).

At a confidence level of 95%, the probability of error is 𝛼 = 5% = 0.05. Thus, for the probability
𝑝 = 0.975, the corresponding quantile 𝑧0.975 of the standard normal distribution can be read off
from Table 2 in the Appendix: 𝑧1−𝛼/2 = 𝑧1−0.05/2 = 𝑧0.975 = 1.960.

3) The (1 − 𝛼) confidence interval for the unknown parameter 𝑝 is then:

𝑝̂ (1 − 𝑝̂ ) 𝑝̂ (1 − 𝑝̂ )
[𝑝̂ − 𝑧1−𝛼/2 ∙ √ , 𝑝̂ + 𝑧1−𝛼/2 ∙ √ ].
𝑛 𝑛

Here are:

𝑝̂ – proportion value of the sample;

𝑛 – sample size;
𝑧1−𝛼/2 – the (1 − 𝛼/2) – quantile of the standard normal distribution.

Example:
In the case of quality control, the proportion of defective products in a production plant was examined.
A product is identified as “A-good” or “B-good” if it has certain characteristics determined by the test.
In the quality control, 500 products were randomly tested and 490 were marked as “A-“ and 10
products as “B-goods”. Assume that the random variable “the number of “B-goods” of 𝑛 randomly
chosen products” is an approximately binomially distributed random variable. In which interval, with
a 99% confidence, is the true value for the proportion of “B-goods”?
Solution:
𝑛 = 500 (sample size)

1) Parameter 𝑝 (proportion value) is calculated from the sample

𝑝̂ = ℎ(𝐴) = 𝑘/𝑛 = 10/500 = 0.02.

2) Verification of the condition for a large sample:

𝑛𝑝̂ (1 − 𝑝̂ ) > 9;500 ∙ 0.02 ∙ 0.98 = 9.8 > 9 → 𝑂𝐾.

3) For the confidence level of 99%, with 𝛼 = 1% = 0.01 the probability of the (1 − 𝛼/2) – quantile
is 1 − 𝛼/2 = 1 − 0.01/2 = 0.995 = 99.5%. From Table 2 in the Appendix, the corresponding
quantile 𝑧0.995 of the standard normal distribution can be read for the probability 𝑝 = 0.995: 𝑧1−𝛼/2 =
𝑧0.995 = 2.576.
67
3) The (1 − 𝛼) confidence interval for the unknown parameter 𝑝 is:

𝑝̂ (1 − 𝑝̂ ) 𝑝̂ (1 − 𝑝̂ )
[𝑝̂ − 𝑧1−𝛼 ∙ √ , 𝑝̂ + 𝑧1−𝛼 ∙ √ ]=
2 𝑛 2 𝑛

0.02 (1 − 0.02) 0.02 (1 − 0.02)

[0.02 − 2.576 ∙ √ ,0.02 + 2.576 ∙ √ ] = [0.0039 ,0.036 ].
500 500

The true but unknown fractional 𝑝 lies with a confidence of 99% in this interval.

Assume that the examined characteristic 𝑋 is a normally distributed random variable with the
unknown mean µ and a known variance σ². The starting point for the determination of a (1 − 𝛼)
confidence interval for µ of the normally distributed population is the point estimator 𝑋̅ for the
unknown mean value µ
𝑛
1
𝑋̅ = ∑ 𝑋𝑖 .
𝑛
𝑖=1

Since the random variable 𝑋 is a normally distributed random variable, the arithmetic mean is a
normally distributed random variable with the mean µ and a variance 𝜎 2 /𝑛 (see Section 6.3). The
standardized random variable 𝑈 = ( 𝑋̅ − µ)/(𝜎/√𝑛) is standard normally distributed with the mean
value µ = 0 and standard deviation σ = 1. From the condition

𝑃(−𝑧1−𝛼/2 ≤ 𝑈 ≤ 𝑧1−𝛼/2 ) = 1 − 𝛼,

a (1 − 𝛼) confidence interval for the unknown mean value µ of a normal distribution with known
variance σ² is obtained by elementary transformations:

𝑋̅ − 𝑧1−𝛼/2 ∙ 𝜎/√𝑛 ≤ µ ≤ 𝑋̅ + 𝑧1−𝛼/2 ∙ 𝜎/√𝑛 or [𝑋̅ − 𝑧1−𝛼/2 ∙ 𝜎/√𝑛 , 𝑋̅ + 𝑧1−𝛼/2 ∙ 𝜎/√𝑛].

Here are:

𝑋̅ the estimation function for the unknown mean value μ of the normally distributed population,
𝜎 the standard deviation of the population (here presumed to be known),

𝑛 the sample size,

𝑧1−𝛼/2 the (1 − 𝛼/2) – quantile of the standard normal distribution.

68
A summary of the most important:

A (1 − 𝛼) confidence interval for the mean value 𝜇 of a normal distribution with known variance 𝜎 2
is determined step by step as follows by means of a data sample with the realisations 𝑥1 , 𝑥2 , … , 𝑥𝑛 .

1) Calculation of the mean value 𝑥̅ of the given sample:

𝑛
1 1
𝑥̅ = ∑ 𝑥𝑖 = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ).
𝑛 𝑛
𝑖=1

2) Determination of the quantiles of the standard normal distribution 𝑧1−𝛼/2 for the chosen confidence
level (usually 95 % and 1 − 𝛼 = 0.95).

At a confidence level of 95%, the probability of error is 𝛼 = 5% = 0.05. Thus, for the probability
𝛼
𝑝 = 1 − 2 = 0.975, the corresponding quantile 𝑧0.975 of the standard normal distribution can be
read off from Table 2 in the Appendix: 𝑧1−𝛼/2 = 𝑧1−0.05/2 = 𝑧0.975 = 1.960.

3) The (1 − 𝛼) confidence interval for the unknown mean value 𝜇 of a normal distribution with known
variance 𝜎 2 is then:

[𝑥̅ − 𝑧1−𝛼/2 ∙ 𝜎/√𝑛 , 𝑥̅ + 𝑧1−𝛼/2 ∙ 𝜎/√𝑛].

Here are:

𝑥̅ – mean value of the sample;

𝜎 – standard deviation of the population (here presumed to be known);

𝑛 – sample size;

𝑧1−𝛼/2 – the (1 − 𝛼/2) – quantile of the standard normal distribution.

Example:
In a series production of gaming pieces, it is assumed that the height of the produced pieces is
normally distributed in good approximation with a standard deviation of 0.6 cm. In a sample of 50
pieces, a height of 4 cm was measured for 10 pieces, a height of 5 cm for 30 pieces and a height of 6
cm for 10 additional pieces. Determine the 95% and 99% confidence intervals for the unknown mean
𝜇.

Solution:
We follow the steps described in Section 0:
1) First, the sample mean is calculated:
𝑛
1 1 1
𝑥̅ = ∑ 𝑥𝑖 = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ) = (10 × 4 + 30 × 5 + 10 × 4) = 5𝑐𝑚.
𝑛 𝑛 50
𝑖=1

2) For the given confidence levels, the (1 − 𝛼/2) quantiles of the standard normal distribution are:

95% confidence interval: 𝛼 = 5% → 1 − 𝛼/2 = 0.975 → 𝑧1−𝛼/2 = 𝑧0.975 = 1.96,

69
99% confidence interval: 𝛼 = 1% → 1 − 𝛼/2 = 0.995 → 𝑧1−𝛼/2 = 𝑧0.995 = 2.576.

3) The 95% confidence interval for the unknown mean value µ is:

[𝑥̅ − 𝑧1−𝛼 ∙ 𝜎/√𝑛, 𝑥̅ + 𝑧1−𝛼 ∙ 𝜎/√𝑛] = [5 − 1.96 ∙ 0.6/√50, 5 + 1.96 ∙ 0.6/√50] =

2 2

[4.83 𝑐𝑚, 5.17 𝑐𝑚].

The 99% confidence interval for the unknown mean value µ is:

[5 − 2.576 ∙ 0.6/√50, 5 + 2.576 ∙ 0.6/√50] = [4.78 𝑐𝑚, 5.22 𝑐𝑚].

The true but unknown mean µ lies with a confidence of 99% in this interval.

Assume that the examined characteristic 𝑋 is a normally distributed random variable 𝑋~𝑁(𝜇, 𝜎) with
unknown mean µ and unknown variance σ². Since both the mean value and the variance of the total
population are unknown, the standardized random variable 𝑈 = ( 𝑋̅ − µ)/(𝜎/√𝑛) cannot be used for
the confidence interval determination. If in the equation for 𝑈 the unknown standard deviation 𝜎 is
replaced by the estimation function 𝑆 of the standard deviation (see Section 7.1), the following random
variable is obtained:

𝑇 = (𝑋̅ − µ)/(𝑆/√𝑛).

The random variable 𝑇 follows Student’s t-distribution with 𝑓 = 𝑛 − 1 degrees of freedom. From the
condition for the standardized random variable 𝑈 = (𝑋̅ − µ)/(𝜎/√𝑛) (𝑈~𝑁(0,1))

𝑃 (−𝑡1−𝛼 ≤ 𝑇 ≤ 𝑡1−𝛼 ) = 1 − 𝛼,
2 2

the (1 − 𝛼) confidence interval for the unknown mean value 𝜇 of a normal distribution with known
variance σ² is obtained by elementary transformations:

𝑋̅ − 𝑡1−𝛼/2 ∙ 𝑆/√𝑛 ≤ µ ≤ 𝑋̅ + 𝑡1−𝛼/2 ∙ 𝑆/√𝑛 or [𝑋̅ − 𝑡1−𝛼/2 ∙ 𝜎/√𝑛, 𝑋̅ + 𝑡1−𝛼/2 ∙ 𝑆/√𝑛].

Here are:

𝑋̅ the estimation function for the unknown mean value μ of the normally distributed population,
S the estimation function for the unknown standard deviation of the normally distributed population,

𝑛 the sample size,

𝑡1−𝛼/2 the (1 − 𝛼/2) – quantile of the t-distribution with 𝑓 = 𝑛 − 1 degrees of freedom.

In general, for large samples (rule of thumb 𝑛 > 30), the unknown variance 𝜎 2 can be estimated by
the variance of the sample 𝑠 2 . Thus the procedure described in Section 7.2 can be used for the
confidence interval determination!

70
A summary of the most important:

A (1 − 𝛼) confidence interval for the mean value 𝜇 of a normal distribution with unknown variance σ²
is determined stepwise as follows by means of a data sample with the realisations 𝑥1 , 𝑥2 , … , 𝑥𝑛 .

1) Calculation of the mean value 𝑥̅ of the given sample:

𝑛
1 1
𝑥̅ = ∑ 𝑥𝑖 = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ).
𝑛 𝑛
𝑖=1

2) Calculation of the standard deviation 𝑠 of the given sample:

𝑛
1
𝑠=√ ∑(𝑥𝑖 − 𝑥̅ )2 .
𝑛−1
𝑖=1

3) Determination of the quantiles of the t-distribution 𝑡1−𝛼/2 with 𝑓 = 𝑛 − 1 degrees of freedom for
the chosen confidence level (usually 95 % and 1 − 𝛼 = 0.95).

At a confidence level of 95% the probability of error is = 5% = 0.05. Thus, for the probability
𝛼
𝑝 = 1 − 2 = 0.975, the corresponding quantile 𝑡0.975 of the t-distribution at 𝑓 = 𝑛 − 1 degrees of
freedom can be read off from

Table 3 in the Appendix: 𝑡1−𝛼/2 = 𝑡1−0.05/2 = 𝑡0.975.

4) The (1 − 𝛼) confidence interval for the unknown mean value μ of a normal distribution with
unknown variance σ² is then:

[𝑥̅ − 𝑡1−𝛼/2 ∙ 𝑠/√𝑛 , 𝑥̅ + 𝑡1−𝛼/2 ∙ 𝑠/√𝑛].

Here are:

𝑥̅ – mean value of the sample;

𝑠 – standard deviation of the sample;

𝑛 – sample size;
𝑡1−𝛼/2 − the (1 − 𝛼/2) – quantile of the t-distribution with 𝑓 = 𝑛 − 1 degrees of freedom.

Note: For 𝑛 > 30, one can assume that 𝜎 ≈ 𝑠, and apply the method described in Section 7.2.

Example:
In the case of serial production of gaming pieces, it is assumed that the height of the produced pieces
is normally distributed in a good approximation. In a sample of 50 pieces, a height of 4 cm was
measured for 10 pieces, a height of 5 cm for 30 pieces and a height of 6 cm for 10 additional pieces.
Determine the 95% and 99% confidence intervals for the unknown mean µ.

71
Solution:
We follow the steps described in Section 7.3:

𝑛 = 50 (sample size).
1) First, the sample mean is calculated:
𝑛
1 1 1
𝑥̅ = ∑ 𝑥𝑖 = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ) = (10 × 4 + 30 × 5 + 10 × 4) = 5 𝑐𝑚.
𝑛 𝑛 50
𝑖=1

2) The standard deviation of the sample is

𝑛
1 1 20
𝑠=√ ∑(𝑥𝑖 − 𝑥̅ )2 = √ [10 × (4 − 5)2 + 30 × (5 − 5)2 + 10 × (6 − 5)2 ] = √ = 0.639.
𝑛−1 50 − 1 49
𝑖=1

3) For the given confidence levels the (1 − 𝛼/2) – quantiles of the t-distribution with 𝑓 = 𝑛 − 1 =
49 degrees of freedom are given by:

95% confidence interval: 𝛼 = 5% → 1 − 𝛼/2 = 0.975 → 𝑡1−𝛼/2 = 𝑡0.975 ≈ 2.009

99% confidence interval: 𝛼 = 1% → 1 − 𝛼/2 = 0.995 → 𝑡1−𝛼/2 = 𝑧0.995 ≈ 2.678

4) 95% and 99% confidence intervals:

The 95% confidence interval is

𝑠 𝑠 0.639 0.639
[𝑥̅ − 𝑡1−𝛼 ∙ , 𝑥̅ + 𝑡1−𝛼 ∙ ] = [5 − 2.009 ∙ , 5 + 2.009 ∙ ]
2 √𝑛 2 √𝑛 √50 √50

= [4.82 𝑐𝑚, 5.18 𝑐𝑚].

The true but unknown mean µ lies with a confidence of 95% in this interval.

The 99% confidence interval is

[5 − 2.678 ∙ 0.639/√50, 5 + 2.678 ∙ 0.639/√50] = [4.76 𝑐𝑚, 5.24 𝑐𝑚].

The true but unknown mean µ lies with a confidence of 99% in this interval.

In the case of an arbitrarily distributed characteristic, but with a “large” sample scope (rule of thumb
𝑛 > 30), one can start from a normally distributed population. If the variance σ² of the population is
known, the determination of a confidence interval for the unknown mean value μ is performed
according to the same scheme as in Section 0. If the variance σ² of the population is unknown, it can
be assumed to be “known” in this case and estimated by the variance of the sample σ² ≈ 𝑠 2 . The
confidence interval for the unknown mean value μ is then determined according to the same scheme as
in Section 7.3. In general, the larger the sample size, the smaller the difference between the confidence
intervals according to the two schemes.
A summary of the most important:

An approximate (𝟏 − 𝜶) confidence interval for the unknown mean value μ of any distribution with
𝑛 > 30 can be determined as follows:
72
a) If the variance σ² of the population is known, then by

[𝑥̅ − 𝑧1−𝛼/2 ∙ 𝜎/√𝑛 , 𝑥̅ + 𝑧1−𝛼/2 ∙ 𝜎/√𝑛].

b) If the variance σ² of the population is unknown, then by

[𝑥̅ − 𝑡1−𝛼/2 ∙ 𝑠/√𝑛 , 𝑥̅ + 𝑡1−𝛼/2 ∙ 𝑠/√𝑛].

Here are:

𝑥̅ – mean value of the sample;

𝜎 – standard deviation of the population;

𝑠 – standard deviation of the sample;

𝑛 – sample size;
and
𝑧1−𝛼/2 – the (1 − 𝛼/2) – quantile of the standard normal distribution;
𝑡1−𝛼/2 − the (1 − 𝛼/2) – quantile of the t-distribution with 𝑓 = 𝑛 − 1 degrees of freedom.

Example:

A sample consists of 50 pieces. A height of 4 cm was measured for 10 pieces, a height of 5 cm for 30
pieces and a height of 6 cm for 10 additional pieces.
Determine the 95% confidence interval for the unknown mean µ according to the assumption that
a) the height of the pieces produced generally has a standard deviation of 0.6 cm;
b) no information is available about the standard deviation of the population.

Solution:
a) Exactly as in the example in Section 0:
The 95% confidence interval is:

[𝑥̅ − 𝑧1−𝛼 ∙ 𝜎/√𝑛, 𝑥 + 𝑧1−𝛼 ∙ 𝜎/√𝑛] = [5 − 1.96 ∙ 0.6/√50, 5 + 1.96 ∙ 0.6/√50] =

2 2

[4.83 𝑐𝑚, 5.17 𝑐𝑚].

b) Exactly as in the example in Section 7.3:
The 95% confidence interval is:

[𝑥̅ − 𝑡1−𝛼 ∙ 𝑠/√𝑛, 𝑥̅ + 𝑡1−𝛼 ∙ 𝑠/√𝑛] = [5 − 2.009 ∙ 0.639/√50, 5 + 2.009 ∙ 0.639/√50] =

2 2
[4.82 𝑐𝑚, 5.18 𝑐𝑚].

The true but unknown mean μ lies with a confidence of 95% in this interval.

Suppose the examined characteristic 𝑋 is a normally distributed random variable 𝑋~𝑁(𝜇, 𝜎) with
unknown mean and unknown variance. Similar to the average of the population μ (see Sections 0-7.4),

73
one can also determine a confidence interval for the variance of the population σ². This is formed by
means of the random variable 𝑍 = ( 𝑛 − 1) ∙ 𝑆 2 /𝜎 2 . If the property 𝑋 is a normally distributed
random variable 𝑋~𝑁(𝜇, 𝜎), then the random variable 𝑍 is a chi-squared distributed random variable
with 𝑓 = 𝑛 − 1 degrees of freedom. From the condition

𝑃(𝑞𝛼/2 ≤ 𝑍 ≤ 𝑞1−𝛼/2 ) = 1 − 𝛼,

the (1 − 𝛼) confidence interval for the unknown variance σ² is obtained by elementary

transformations:

( 𝑛 − 1) ∙ 𝑆 2 /𝑞1−𝛼/2 ≤ σ² ≤ ( 𝑛 − 1) ∙ 𝑆 2 /𝑞𝛼/2 or [( 𝑛 − 1) ∙ 𝑆 2 /𝑞1−𝛼/2 , ( 𝑛 − 1) ∙ 𝑆 2 /𝑞𝛼/2 ].

Here are:
S the estimation function for the unknown standard deviation of the population,
n the sample size,
𝑞1−𝛼/2 or 𝑞𝛼/2 the quantiles of the der chi-squared distribution with 𝑓 = 𝑛 − 1 degrees of freedom.

A summary of the most important:

A (1 − 𝛼) confidence interval for the variance σ² will be determined step by step using a data sample
with the realisations 𝑥1 , 𝑥2 , … , 𝑥𝑛 as follows:

1) Calculation of the variance 𝑠 2 of the given sample:

𝑛
2
1
𝑠 = ∑(𝑥𝑖 − 𝑥̅ )2 .
𝑛−1
𝑖=1

2) Determination of the quantiles of the chi-squared distribution with 𝑓 = 𝑛 − 1 degrees of freedom

(see Table 4 in the Appendix) for the chosen confidence level (usually 95 % and 1 − 𝛼 = 0.95 →
𝛼 = 0.05).

3) The (1 − 𝛼) confidence interval for the unknown variance σ² is then

[( 𝑛 − 1) ∙ 𝑠 2 /𝑞1−𝛼/2 , ( 𝑛 − 1) ∙ 𝑠 2 /𝑞𝛼/2 ].

Here are:

𝑠 2 – the variance of the sample;

n – the sample size;
𝑞1−𝛼/2 and 𝑞𝛼/2 – the (1 − 𝛼/2) – and (𝛼/2) − quantile of the chi-squared distribution with 𝑓 = 𝑛 −
1 degrees of freedom.

Example:
In a series production of game pieces, it is assumed that the height of the produced pieces is normally
distributed in good approximation with a standard deviation of 0.6 cm. In a sample of 50 pieces, a
height of 4 cm was measured for 10 pieces, a height of 5 cm for 30 pieces and a height of 6 cm for 10
additional pieces. Determine the 95% confidence interval for the unknown variance σ².

74
Solution:
𝑛 = 50 (sample size).
1) The mean value and the standard deviation of the sample have already been calculated in Section
7.3:
𝑛 𝑛
1 1
𝑥̅ = ∑ 𝑥𝑖 = 5 𝑐𝑚 𝑠 = √ ∑(𝑥𝑖 − 𝑥̅)2 = 0.639 𝑐𝑚.
𝑛 𝑛−1
𝑖=1 𝑖=1

2) At a confidence level of 95%, with 𝛼 = 5% = 0.05 and thus (1 − 𝛼/2) = 1 − 0.05/2 = 0.975;
𝛼/2 = 0.05/2 = 0.025. From Table 4 in the Appendix, the corresponding quantiles of the chi-
squared distribution with 𝑓 = 𝑛 − 1 = 49 degrees of freedom are:

𝑞1−𝛼/2 = 𝑞0.975 ≈ 71.4; 𝑞𝛼/2 = 𝑞0.025 = 32.4.

3) The 1 − 𝛼 = 95% confidence interval for the unknown variance is:

[( 𝑛 − 1) ∙ 𝑠 2 /𝑞1−𝛼/2 , ( 𝑛 − 1) ∙ 𝑠 2 /𝑞𝛼/2 ] = [49 ∙ 0.6392 /71.4, 49 ∙ 0.6392 /32.4] = [0.28, 0.62].

The true but unknown value of variance σ² lies with a confidence of 95% in this interval.

75
If one has a presumption or assumption about the properties of a population, then one can test this by
juxtaposing two competing hypotheses. The two competing hypotheses are called the null hypothesis
and the alternative hypothesis. For example, the null hypothesis can be that a treatment has no effect,
that the manufacturer's specification of the filling weight of his packs is correct, or that the proportion
of defective products in a production plant is below a certain limit. Typically, in these cases, the
alternative hypothesis is that a treatment has an effect, that the manufacturer's specification is not
correct to the filling weight of his packs, or that the proportion of defective products in a production
plant is above a certain limit.

The main requirement for the null hypothesis is that it can be tested using a statistical test. This
means it must be possible to calculate the probability that the test rejects the null hypothesis if the null
hypothesis is true. This probability is called the significance level of the test (standard values for the
significance level are 5 % or 1 %). The main task of a statistical test is to make a decision as to
whether the null hypothesis can be accepted (or cannot be rejected) or whether it can be rejected in
favour of the alternative hypothesis. Since statistical tests are based on sample values, the decision can
be both correct and incorrect.

In the decisions about the null hypothesis and the alternative hypothesis, a distinction is made between
two types of errors or incorrect decisions: Type I error and Type II error. A Type I error is the case
when a true null hypothesis is rejected. A Type II error occurs when a false hypothesis is maintained
in reality.

Test decision
Null hypothesis Alternative hypothesis
Null hypothesis is true right wrong: Type I error
Reality Alternative hypothesis is
wrong: Type II error right
true

The two types of wrong decisions are illustrated by the following example: A pharmaceutical
company wants to check whether a new headache treatment is effective or not. The medication or a
placebo is used in a group of subjects (placebos are illicit drugs without real active ingredients). The
null hypothesis is that the treatment does not help with headaches, the alternative hypothesis is that the
treatment helps. The responses of the subjects who have either taken a “real” or a placebo medication
are evaluated and compared. If the effects in the two groups are not statistically significant, then the
new headache treatment cannot be expected to be significantly better for headaches than a placebo. In
the judgment on the new drug, two types of errors are possible: to infer that the drug helps against
headaches, if indeed this is not the case (Type I error because the true null hypothesis is rejected), and
otherwise, that an effective treatment does not help (Type II error because in reality the true alternative
hypothesis is rejected).

A statistical test is used to check two mutually exclusive statistical hypotheses. The two hypotheses
are the null hypothesis 𝐻0 and the alternative hypothesis 𝐻1 . The hypothesis 𝐻1 (i.e. our

76
assumption) can be that the weight of the cookie packs does not coincide with the “set-point” and the
hypothesis 𝐻0 thus is the contrary. Since a hypothesis can usually not be tested by means of the
population, but rather by means of a concrete sample (for example, by means of a certain number of
cookie packs), it is usually impossible to say with certainty whether a hypothesis is true or not. Thus,
e.g. the test size and/or test statistic, with which the competing hypotheses for our “biscuit” example
are examined, is formed on the basis of the average weight of all cookie packs from our sample. In
order to statistically verify whether the null hypothesis 𝐻0 is correct or whether the alternative
hypothesis 𝐻1 is to be rejected, the probability of a wrong decision is controlled by the significance
level  of the test (typically  is between 1% and 5%). The significance level , among other things,
serves to determine the so-called region of acceptance and region of rejection. If the test statistics
determined from the concrete sample fall into the region of acceptance, the null hypothesis 𝐻0 is
retained. This test decision does not mean that the null hypothesis 𝐻0 is actually correct, it means that
the sample used does not speak against the null hypothesis. If the test statistics determined from the
concrete sample fall into the region of rejection, then the null hypothesis 𝐻0 must be rejected in favour
of the alternative hypothesis 𝐻1 . It should also be noted that there is always a possibility of error, since
absolutely safe conclusions cannot be drawn from a random sample to a total population. Thus, the
planning and execution of a statistical test can be carried out by the following steps:

1. Formulating the null hypothesis 𝑯𝟎 and the alternative hypothesis 𝑯𝟏 or the test problem on
a concrete question.
Assume that the test statistic is an arbitrary parameter 𝜃 whose “set-point” is 𝜃0 . To return to the
above example, let us assume that the parameter 𝜃 is the weight of the cookie packs whose “set-
point” 𝜃0 is 500𝑔 according to the manufacturer's specifications.

The following test problems are common:

(a) Null hypothesis 𝑯𝟎 : 𝜃 = 𝜃𝟎 against alternative hypothesis 𝑯𝟏 : 𝜃 ≠ 𝜃𝟎

The alternative hypothesis is here “the value of the parameter µ is different from µ0 ”.
For example, one can test whether the manufacturer's indication of the weight of a pack of
biscuits (“set-point”) differs from the actual weight of the biscuit packs, or the test problem is
𝐻0 : 𝜃 = 500𝑔 against 𝐻1 : 𝜃 ≠ 500𝑔.

(b) Null hypothesis 𝑯𝟎 : 𝜃 ≥ 𝜃𝟎 against alternative hypothesis 𝑯𝟏 : 𝜃 < 𝜃𝟎

If one assumes that the actual weight of a pack of biscuits is usually less than the “set-point”,
then this assumption can be formulated as a test problem: 𝐻0 : 𝜃 ≥ 500𝑔 versus 𝐻1 : 𝜃 < 500𝑔.

(c) Null hypothesis 𝑯𝟎 : 𝜃 ≤ 𝜃𝟎 against alternative hypothesis 𝑯𝟏 : 𝜃 > 𝜃𝟎

If one assumes that the actual weight of a pack of biscuits is generally greater than the “set-
point”, then this assumption can be formulated as a test problem: 𝐻0 : 𝜃 ≤ 500𝑔 versus
𝐻1 : 𝜃 > 500𝑔.

77
 a)

/2 /2

𝑐𝑙 𝑐𝑢
Region of rejection Region of acceptance Region of rejection

b) c)

 

𝑐 𝑐
Region of acceptance Region of acceptance

Region of rejection Region of rejection

Figure 32: Assume that the test statistic is an arbitrary parameter 𝜃 whose “set-point” is 𝜃0 . The
following combinations of null hypothesis 𝐻0 and alternative hypothesis 𝐻1 are possible:

a) Two-tailed test with 𝐻0 : 𝜃 = 𝜃0 against 𝐻1 : 𝜃 ≠ 𝜃0;

b) One-tailed test with 𝐻0 : 𝜃 ≥ 𝜃0 against 𝐻1 : 𝜃 < 𝜃0 ;
c) One-tailed test with 𝐻0 : 𝜃 ≤ 𝜃0 against 𝐻1 : 𝜃 > 𝜃0 .
The white area is the likelihood of a Type I error and corresponds to the level of significance of the
test. If the test statistics determined from the concrete sample fall into the region of acceptance, then
the null hypothesis 𝐻0 is retained, otherwise rejected in favour of the alternative hypothesis 𝐻1 .

2. Select the significance level α. The significance level is the probability that the null hypothesis
will be rejected even though it is correct (so-called Type I error). A statistical test is tested at
significance level α

𝑃(𝐻1 𝑎𝑐𝑐𝑒𝑝𝑡𝑒𝑑|𝐻𝑜 𝑡𝑟𝑢𝑒) ≤ 𝛼 or 𝑃(𝑇𝑦𝑝𝑒 𝐼 𝑒𝑟𝑟𝑜𝑟) ≤ 𝛼.

Typical values for the significance level are 1 % and 5 %.

3. Determining the region of acceptance and rejection on the basis of the selected significance
level α and the test problem (null hypothesis 𝑯𝟎 and the alternative hypothesis 𝑯𝟏 ).
Depending on how the two competing hypotheses are defined, the following case distinctions are
made for the determination of region of acceptance and rejection of the hypothesis (see Figure 32):

a) For the test problem with

Null hypothesis 𝐻0 : 𝜃 = 𝜃0 against alternative hypothesis 𝐻1 : 𝜃 ≠ 𝜃0

78
 one speaks of a two-tailed test problem or a two-tailed test (see Figure 32 a). In the case of a two-
tailed test problem, from the alternative hypothesis, which “only” checks inequalities (𝜃 ≠ 𝜃0 ),
follows that both 𝜃 > 𝜃0 and 𝜃 < 𝜃0 are possible. The region of rejection is on both sides of the
region of acceptance and is therefore bilateral (see Figure 32 a).

b) For the test problem with

Null hypothesis 𝐻0 : 𝜃 ≥ 𝜃0 against alternative hypothesis 𝐻1 : 𝜃 < 𝜃0

one speaks of a one-tailed test problem or a one-tailed test (see Figure 32 b). Here the alternative
hypothesis 𝐻1 : 𝜃 < 𝜃0 only allows the values less than 𝜃0 , so the term is defined by a bounding
downwards.

c) For the test problem with

Null hypothesis 𝐻0 : 𝜃 ≤ 𝜃0 against alternative hypothesis 𝐻1 : 𝜃 > 𝜃0

one speaks of a one-tailed test problem or a one-tailed test (see Figure 32 c). Here the alternative
hypothesis 𝐻1 : 𝜃 > 𝜃0 allows only the values greater than 𝜃0 , which is why one speaks of a
bounding upwards.

From the significance level 𝛼 of the test and the probability distribution of the test statistics, the
region of acceptance and rejection can be defined as concrete interval. The interval limits
correspond to the quantiles of the probability distribution of the test statistics (𝛼/2 and 1 − 𝛼/2
with a two-tailed test, respectively (1 − 𝛼) with a one-tailed test).

For our “cookie example” assume that the random variable “weight of a biscuit” is approximately
normally distributed. If the standard deviation is known for the weight of the cookie packs (i.e., the
standard deviation of the population), then the corresponding quantiles of the standard normal
distribution are used to define the region of acceptance and rejection. If the standard deviation is
unknown, then the quantiles of the t-distribution by Student are used.

4. Calculate the value of the test statistics using a given concrete sample. Since statistical tests are
based on sample values, the statistical hypothesis, i.e. our assumption, is tested on the basis of a
concrete sample 𝑥1 , 𝑥2 , … 𝑥𝑛 of size 𝑛. Depending on the concrete question, the test statistic is
obtained by estimating corresponding parameters from the sample values. The distribution of the
random variable, which is of interest, must be considered (if it is a parameter test).

As mentioned above, one can assume for our “cookie sample” that the random variable “weight of
a cookie pack” is approximately normalized. If the variance 𝜎 2 of the random variable of interest
is known (i.e., the variance in the weight of the cookie pack), then the standardized random
variable 𝑧 = ( 𝑥̅ − µ0 )/(𝜎/√𝑛) can be used as test statistics. The standardized random variable 𝑧
is standard-normalized. Where 𝑥̅ is the average weight of the cookie packets determined from the
specific sample, µ0 is the "set-point" for pack weight (the manufacturer's specification, e.g.
µ0 = 500𝑔), 𝜎 is the standard deviation for the weight of the cookie packs, e.g. 𝜎 = 10𝑔) and 𝑛 is
the sample size (e.g. 𝑛 = 20 cookie packs to be tested).

5. Test decision: The null hypothesis is either accepted (not rejected) or rejected in favour of
alternative hypothesis:

79
 Null hypothesis 𝑯𝟎 accepted: The test statistic determined from the concrete sample falls into the
region of acceptance  the null hypothesis is accepted. This does not mean that the null hypothesis
is correct, but that on the basis of the sample used and the chosen significance level 𝛼, nothing
speaks against the null hypothesis.
Null hypothesis 𝑯𝟎 rejected: The test statistic determined from the concrete sample falls into the
region of rejection  the null hypothesis is rejected in favour of the alternative hypothesis. One
can therefore assume that the alternative hypothesis is correct, but one does not know this with
absolute certainty.

The so-called p-values are an alternative to the procedure described in the previous section for finding
a test decision. Instead of checking whether the test statistic determined from the specific sample falls
within the region of acceptance or rejection, one directly compares the p-value of the test statistic
with the chosen significance level 𝛼 of the test.

 p-value

Figure 33: Correlation between the p-value of the test statistic and the significance level 𝛼 of the test.
The p-value indicates the probability of obtaining the observed test statistic or an even more extreme
value in the direction of the alternative hypothesis under the null hypothesis (see Figure 33). Another
way to interpret the p-value is the description as the lowest level of significance, in which a test just
ignores the null hypothesis. Since p-values are probabilities, they always attain values in the interval
[0, 1], thus (0 ≤ 𝑝 ≤ 1).
Thus, the following decision scheme results:
Null hypothesis 𝑯𝟎 accepted: If the p-value is greater than the significance level 𝛼, (𝑝 > 𝛼).

Null hypothesis 𝑯𝟎 rejected: If the p-value is less than or equal to the significance level 𝛼, (𝑝 ≤ 𝛼).
The p-values are automatically given by the statistical software programs, whereby the fast execution
of a statistical test is possible. It is only to be considered when specifying the test properties whether
the concrete test problem is a one-tailed or a two-tailed test problem.

It should be tested whether the proportion value p of defective products in a production plant
corresponds to the manufacturer specification 𝑝0 or not. The starting point for an approximate
binomial test is the assumption that the sample has a “large” sample size. For large samples it can be
assumed that the random variable 𝑋 = “the number of defective products from n randomly selected
products” is a nearly normal distributed random variable. The test statistic is then based on the
standardized random variable 𝑧 = ( 𝑝̂ − 𝑝0 )/√𝑝0 (1 − 𝑝0 )/𝑛 and is approximately standard normally
distributed.

80
The test procedure for the approximate binomial test is summarized in the following:
1) Verification of the condition for large samples:

𝑛𝑝0 (1 − 𝑝0 ) > 9.
If this is fulfilled, the further steps can be calculated, if not, the sample is not sufficiently
comprehensive for an approximate binomial test.

2) Formulation of the test problem (null hypothesis 𝐻0 and alternative hypothesis 𝐻1 ) on the basis of
the specific question. The following test problems are common:

a) 𝐻0 : 𝑝 = 𝑝0 against 𝐻1 : 𝑝 ≠ 𝑝0 ,

b) 𝐻0 : 𝑝 ≥ 𝑝0 against 𝐻1 : 𝑝 < 𝑝0 ,

c) 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 .

3) Select significance level 𝛼: Typical values for the level of significance are 1 % and 5 %.
4) Determination of the region of acceptance and rejection: Since the test statistics z is standard
normally distributed, the interval limits are based on the corresponding quantiles of the standard
normal distribution:

a) For the test problem 𝐻0 : 𝑝 = 𝑝0 against 𝐻1 : 𝑝 ≠ 𝑝0 the region of acceptance is 𝑧𝛼/2 ≤ 𝑧 ≤ 𝑧1−𝛼/2 .

b) For the test problem 𝐻0 : 𝑝 ≥ 𝑝0 against 𝐻1 : 𝑝 < 𝑝0 the region of acceptance is 𝑧 ≥ 𝑧𝛼 .

c) For the test problem 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 the region of acceptance is 𝑧 ≤ 𝑧1−𝛼 .

For example, for the significance level 𝛼 = 1%, the (1 − 𝛼/2) quantile of the standard normal
distribution is 𝑧1−𝛼/2 = 𝑧1−0.01/2 = 𝑧0.995 = 2.576 (see Table 2 in the Appendix).

5) Calculate the value of the test statistic:

𝑝̂ − 𝑝0
𝑧= .
√𝑝0 (1 − 𝑝0 )
𝑛
Here are:

𝑝̂ – the proportion of the sample 𝑝̂ = 𝑘/𝑛;

𝑝0 – the value of the parameter p (e.g. the manufacturer’s value of the proportion);
n – sample size.
6) Test decision: If the test statistic determined from the concrete sample falls into the region of
acceptance, the null hypothesis 𝐻0 is retained. If the test statistic determined from the concrete sample
falls into the region of rejection, the null hypothesis must be rejected in favour of the alternative
hypothesis.

Example:
The chip manufacturer claims that the proportion of defective parts in its production is not more than
3%. Within the scope of a quality control, a random sample of n = 500 parts, a total of 20 defective
parts were found. Test the following statistical hypotheses on a level of significance 𝛼 = 5%:

81
a) 𝐻0 : 𝑝 = 𝑝0 against 𝐻1 : 𝑝 ≠ 𝑝0 .

b) 𝐻0 : 𝑝 ≥ 𝑝0 against 𝐻1 : 𝑝 < 𝑝0 .

c) 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 .

Solution: Since the sample has a large scale and the proportion value 𝑝 of defective products is to be
tested, the approximate binomial test is used:
1) Verification of the condition for large samples:

𝑛𝑝0 (1 − 𝑝0 ) > 9; 500 ∙ 0.03 ∙ (1 − 0.03) = 14.55 > 9 → 𝑇ℎ𝑒 𝑠𝑎𝑚𝑝𝑙𝑒 𝑖𝑠 𝑒𝑥𝑡𝑒𝑛𝑠𝑖𝑣𝑒.
2) The test problems are:

a) 𝐻0 : 𝑝 = 𝑝0 against 𝐻1 : 𝑝 ≠ 𝑝0 .

b) 𝐻0 : 𝑝 ≥ 𝑝0 against 𝐻1 : 𝑝 < 𝑝0 .

c) 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 .

3) Significance level is specified  𝛼 = 5%.

4) The region of acceptance and rejection is determined by the quantiles of the standard normal
distribution (see Table 2 in the Appendix). For the significance level 𝛼 = 5%, the quantiles are:

𝑧𝛼/2 = 𝑧0.05/2 = 𝑧0.025 = −1.96,

𝑧1−𝛼/2 = 𝑧1−0.05/2 = 𝑧0.975 = 1.96,

𝑧1−𝛼 = 𝑧1−0.05 = 𝑧0.95 = 1.645,

𝑧𝛼 = 𝑧0.05 = −1.645.

a) 𝐻0 : 𝑝 = 𝑝0 against 𝐻1 : 𝑝 ≠ 𝑝0 → the region of acceptance is 𝑧𝛼/2 ≤ 𝑧 ≤ 𝑧1−𝛼/2 or −1.96 ≤ 𝑧 ≤

1.96.

b) 𝐻0 : 𝑝 ≥ 𝑝0 against 𝐻1 : 𝑝 < 𝑝0 → the region of acceptance is 𝑧 ≥ 𝑧𝛼 or 𝑧 ≥ −1.645.

c) 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 → the region of acceptance is 𝑧 ≤ 𝑧1−𝛼 or 𝑧 ≤ 1.645.

5) Calculation of the test statistic:

𝑛 sample size: 𝑛 = 500,

𝑝̂ proportion of the sample: 𝑝̂ = 𝑘/𝑛 = 20/500 = 0.04 (k = Number of defective parts in the
sample),

𝑝0 value of the parameter p: 𝑝0 = 3% = 0.03.

𝑝̂−𝑝0 0.04−0.03
Thus, the test statistic is 𝑧 = = = 1.311.
𝑝 (1−𝑝0 ) 0.03(1−0.03)
√ 0 √
𝑛 500

6) Test decision: If the test statistics determined from the concrete sample fall into the region of
acceptance, the null hypothesis 𝐻0 is accepted, otherwise rejected in favour of the alternative
hypothesis.

82
a) 𝐻0 : 𝑝 = 𝑝0 against 𝐻1 : 𝑝 ≠ 𝑝0 → the value of the test statistic 𝑧 = 1.311 falls into the region of
acceptance −1.96 ≤ 𝑧 ≤ 1.96 → 𝐻0 accepted!

b) 𝐻0 : 𝑝 ≥ 𝑝0 against 𝐻1 : 𝑝 < 𝑝0 → the value of the test statistic 𝑧 = 1.311 falls into the region of
acceptance 𝑧 ≥ −1.645 → 𝐻0 accepted!

c) 𝐻0 : 𝑝 ≤ 𝑝0 against 𝐻1 : 𝑝 > 𝑝0 → the value of the test statistic 𝑧 = 1.311 falls into the region of
acceptance 𝑧 ≥ −1.645 → 𝐻0 accepted!

Suppose the random variable 𝑋 is normally distributed (or randomly distributed, but the sample has a
large scope – rule of thumb 𝑛 > 30) and the variance is known. In the hypotheses about the unknown
mean value µ (for example, 𝜇 ≠ 𝜇0 , 𝜇 > 𝜇0 , 𝜇 < 𝜇0 ), the corresponding test problems can be defined
on the basis of a concrete 𝑥1 , 𝑥2 , … 𝑥𝑛 of circumference 𝑛. The test statistic is based on the
standardized random variable 𝑧 = ( 𝑥̅ − µ0 )/(𝜎/√𝑛) which is standard normally distributed with the
expected value µ = 0 and the standard deviation σ = 1.
The test procedure for the Z-test is summarized as follows:

1) Formulation of the test problem (null hypothesis 𝐻0 and the alternative hypothesis 𝐻1 ) on the basis
of the specific question. The following test procedures are common:

a) 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 .
b) 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 .

c) 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 .

2) Select significance level 𝛼: Typical values for the level of significance are 1 % and 5 %.
3) Determination of the region of acceptance and rejection: Since the test statistic z is standard
normally distributed, the interval limits are based on the corresponding quantiles of the standard
normal distribution:

a) For the test problem 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 the region of acceptance is 𝑧𝛼/2 ≤ 𝑧 ≤

𝑧1−𝛼/2 .

b) For the test problem 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 the region of acceptance is 𝑧 ≥ 𝑧𝛼 .

c) For the test problem 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 the region of acceptance is 𝑧 ≤ 𝑧1−𝛼 .

For example, for the significance level 𝛼 = 1%, the (1 − 𝛼/2) quantile of the standard normal
distribution is 𝑧1−𝛼/2 = 𝑧1−0.01/2 = 𝑧0.995 = 2.576 (see Table 2 in the Appendix).

4) Calculation of the test statistic:

𝑥̅ − µ0
𝑧 = 𝜎 .
√𝑛
Here are:
1
x̅ – the mean value of the sample (calculated by 𝑥̅ = 𝑛 (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ));

83
µ0 – the expected value of the mean value (e.g. the “set-point” for the weight of a cookie pack);
𝜎 – the standard deviation of the population (here presumed to be known);
n – sample size.
5) Test decision: If the test statistics determined from the concrete sample fall into the region of
acceptance, the null hypothesis 𝐻0 is retained. If the test statistics determined from the concrete
sample fall into the region of rejection, the null hypothesis must be rejected in favour of the alternative
hypothesis.

Example:
A detergent manufacturer claims that the filling volume of its best-selling product is 500 ml with a
standard deviation σ of 10 ml. It is speculated that there has been significantly less detergent in a
package lately. To test this hypothesis, a sample of 100 packs was examined. The study of the sample
found that, on average, the packs contained 490 ml. The question is, of course, whether this is still
tolerable. To test this, test the following statistical hypotheses on a significance level of 𝛼 = 1 %.

a) 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 .

b) 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 .

c) 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 .

Solution:
It is assumed that the volume of a detergent pack is a normally distributed random variable. Therefore,
the Z-test can be used:
1) The test problems are:

a) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 ≠ 500 𝑚𝑙.

b) 𝐻0 : 𝜇 ≥ 500 𝑚𝑙 against 𝐻1 : 𝜇 < 500 𝑚𝑙.

c) 𝐻0 : 𝜇 ≤ 500 𝑚𝑙 against 𝐻1 : 𝜇 > 500 𝑚𝑙.

2) Significance level 𝛼 = 1 % = 0.01.

3) The region of acceptance and rejection is determined by the quantiles of the standard normal
distribution (see Table 2 in the Appendix). For the significance level 𝛼 = 1%, the quantiles are:

𝑧𝛼/2 = 𝑧0.01/2 = 𝑧0.005 = −2.576,

𝑧1−𝛼/2 = 𝑧1−0.01/2 = 𝑧0.995 = 2.576,

𝑧1−𝛼 = 𝑧1−0.01 = 𝑧0.99 = 2.326,

𝑧𝛼 = 𝑧0.01 = −2.326.

a) 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 → region of acceptance 𝑧𝛼/2 ≤ 𝑧 ≤ 𝑧1−𝛼/2 or −2.576 ≤ 𝑧 ≤

2.576.

b) 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 → region of acceptance 𝑧 ≥ 𝑧𝛼 or 𝑧 ≥ −2.326.

c) 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 → region of acceptance 𝑧 ≤ 𝑧1−𝛼 or 𝑧 ≤ 2.326.

4) Calculation of the test statistic:
84
 𝑥̅ − µ0 490 − 500
𝑧 = 𝜎 = = −10.
10
√𝑛 √100
Here are:

x̅ the mean value of the sample,

µ0 the expected value of the mean value (here 500 ml as “set-point”),

σ the standard deviation of the population (here given),
n the sample size.

5) Test decision: If the test statistics determined from the concrete sample fall into the region of
acceptance, the null hypothesis 𝐻0 is retained, otherwise rejected in favour of the alternative
hypothesis.

a) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 ≠ 500 𝑚𝑙 → the value of test statistic 𝑧 = −10 does not fall into the
region of acceptance −2.576 ≤ 𝑧 ≤ 2.576 → 𝐻0 rejected, since the manufacturer's data and the actual
amount of detergent in a package are different!

b) 𝐻0 : 𝜇 ≥ 500 𝑚𝑙 against 𝐻1 : 𝜇 < 500 𝑚𝑙 → the value of test statistic 𝑧 = −10 does not fall into the
region of acceptance 𝑧 ≥ −2.326 → 𝐻0 rejected, it must therefore be assumed that the actual amount
of detergent is less than the “set-point”!

c) 𝐻0 : 𝜇 ≤ 500 𝑚𝑙 against 𝐻1 : 𝜇 > 500 𝑚𝑙 → the value of test statistic 𝑧 = −10 falls into the region
of acceptance 𝑧 ≤ 2.326 → 𝐻0 accepted! One can assume that the amount of detergent is ≤ 500 𝑚𝑙.

Suppose the random variable 𝑋 of interest is normally distributed (or randomly distributed, but the
sample has a large scope – rule of thumb 𝑛 > 30) and the variance is unknown. In the hypotheses
about the unknown mean value µ (for example 𝜇 ≠ 𝜇0 , 𝜇 > 𝜇0 , 𝜇 < 𝜇0 ), the corresponding test
problems can be defined on the basis of a concrete sample 𝑥1 , 𝑥2 , … 𝑥𝑛 of circumference 𝑛. The test
statistics are based on the t-distributed random variable 𝑡 = ( 𝑥̅ − µ0 )/(𝑠/√𝑛) with 𝑓 = 𝑛 − 1
degrees of freedom.

In general, in the case of large samples (rule of thumb 𝑛 > 30), it can be assumed that the random
variable 𝑡 = ( 𝑥̅ − µ0 )/(𝑠/√𝑛) is standard normally distributed. If one estimates the unknown
variance 𝜎 2 by the variance of the sample 𝑠 2 , then the test procedure described in Section 8.5 can also
be used!

The test procedure for the t-test is summarized as follows:

1) Formulation of the test problem (null hypothesis 𝐻0 and the alternative hypothesis 𝐻1 ) on the basis
of the specific question. The following test procedures are common:

a) 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 .

b) 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 .
c) 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 .

85
2) Select significance level 𝛼: Typical values for the level of significance are 1 % and 5 %.
3) Determination of the region of acceptance and rejection: Since the test statistics are t-distributed,
the interval boundaries are based on the corresponding quantiles of the t-distribution with 𝑓 = 𝑛 − 1
degrees of freedom.

a) For the test problem 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 the region of acceptance is 𝑡𝛼/2,𝑓 ≤ 𝑡 ≤

𝑡1−𝛼,𝑓 .
2

b) For the test problem 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 the region of acceptance is 𝑡 ≥ 𝑡𝛼,𝑓 .

c) For the test problem 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 the region of acceptance is 𝑡 ≤ 𝑡1−𝛼,𝑓 .

For example, for a sample with a range of 𝑛 = 20 and a significance level 𝛼 = 1%, the (1 − 𝛼/2)
quantile of the t-distribution with 𝑓 = 𝑛 − 1 = 20 − 1 = 19 degrees of freedom is 𝑡1−𝛼/2,𝑓 =
𝑡1−0.01/2,19 = 𝑡0.995,19 = 2.861 (see

Table 3 in the Appendix).

4) Calculation of the test statistic:
𝑥̅ − µ0
𝑡 = 𝑠 .
√𝑛
Here are:
1
x̅ – the mean value of the sample (calculated by 𝑥̅ = 𝑛 (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 ));

µ0 – the expected value of the mean value (e.g. the “set-point” for the weight of a cookie pack);
1
s – the standard deviation of the sample (calculated as 𝑠 = √𝑛−1 ∑𝑛𝑖=1(𝑥𝑖 − 𝑥̅ )2 );

n – the sample size.

5) Test decision: If the test statistics determined from the concrete sample fall into the region of
acceptance, the null hypothesis 𝐻0 is retained, otherwise rejected in favour of the alternative
hypothesis.

Note: for large samples (rule of thumb 𝑛 > 30) one can assume that the random variable 𝑡 = ( 𝑥̅ −
µ0 )/(𝑠/√𝑛) is roughly standard normally distributed. Thus the test procedure described in Section 8.5
can also be used (𝜎 ≈ 𝑠)!

Example:
A detergent manufacturer claims that the filling volume of his best-selling product is 500 ml. It is
speculated that significantly less detergent has been contained in a package lately. To test this
hypothesis, a sample of 25 packs was examined. The study of the sample concluded that, on average,
the packs contained 490 ml with a standard deviation of 20 ml. The question is, of course, whether this
is still tolerable. To test this, test the following statistical hypotheses on a level of significance
𝛼 = 1 %.

a) 𝐻0 : 𝜇 = 𝜇0 against 𝐻1 : 𝜇 ≠ 𝜇0 .
86
b) 𝐻0 : 𝜇 ≥ 𝜇0 against 𝐻1 : 𝜇 < 𝜇0 .

c) 𝐻0 : 𝜇 ≤ 𝜇0 against 𝐻1 : 𝜇 > 𝜇0 .

Solution:
It is assumed that the volume of a detergent pack is a normally distributed random variable. Since the
standard deviation or variance of the population is not known, the t-test is applied:
1) The test problems are:

a) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 ≠ 500 𝑚𝑙.

b) 𝐻0 : 𝜇 ≥ 500 𝑚𝑙 against 𝐻1 : 𝜇 < 500 𝑚𝑙.

c) 𝐻0 : 𝜇 ≤ 500 𝑚𝑙 against 𝐻1 : 𝜇 > 500 𝑚𝑙.

2) Significance level 𝛼 = 1 % = 0.01.

3) The region of acceptance and rejection is defined by the quantiles of the t-distribution with 𝑓 = 𝑛 −
1 = 25 − 1 = 24 degrees of freedom (see Table 3 in the Appendix). For the significance level
𝛼 = 1%, the quantiles are:

𝑡𝛼/2,24 = 𝑡0.01/2,24 = 𝑡0.005,24 = −2.797,

𝑡1−𝛼/2,24 = 𝑡1−0.01/2,24 = 𝑡0.995,24 = 2.797,

𝑡1−𝛼,24 = 𝑡1−0.01,24 = 𝑡0.99,24 = 2.492,

𝑡𝛼,24 = 𝑡0.01,24 = −2.492.

a) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 ≠ 500 𝑚𝑙 → region of acceptance 𝑡𝛼/2,𝑓 ≤ 𝑡 ≤ 𝑡1−𝛼/2,𝑓 or

−2.797 ≤ 𝑧 ≤ 2.797.

b) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 < 500 𝑚𝑙 → region of acceptance 𝑡 ≥ 𝑡𝛼,𝑓 or 𝑡 ≥ −2.492.

c) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 > 500 𝑚𝑙 → region of acceptance 𝑡 ≤ 𝑡1−𝛼,𝑓 or 𝑡 ≤ 2.492.

4) Calculation of the test statistic:

𝑥̅ − µ0 490 − 500
𝑡 = 𝑠 = = −2.5.
20
√𝑛 √25
Here are:

x̅ the mean value of the sample (here x̅ = 490 𝑚𝑙),

µ0 the expected value of the mean value (here µ0 = 500 𝑚𝑙 as “set-point”),

𝑠 the standard deviation of the sample (here 𝑠 is given  𝑠 = 20 𝑚𝑙 ),

𝑛 the sample size (𝑛 = 25).

5) Test decision: If the test statistics determined from the concrete sample fall into the region of
acceptance, the null hypothesis 𝐻0 is retained, otherwise rejected in favour of the alternative
hypothesis.

87
a) 𝐻0 : 𝜇 = 500 𝑚𝑙 against 𝐻1 : 𝜇 ≠ 500 𝑚𝑙 → the value of the test statistic 𝑧 = −2.5 falls into the
region of acceptance −2.797 ≤ 𝑧 ≤ 2.797 → 𝐻0 accepted!

b) 𝐻0 : 𝜇 ≥ 500 𝑚𝑙 against 𝐻1 : 𝜇 < 500 𝑚𝑙 → the value of the test statistic 𝑧 = −2.5 does not fall
into the region of acceptance 𝑧 ≥ −2.492 → 𝐻0 rejected! It may be assumed that the actual amount of
detergent is less than the “set-point”!

c) 𝐻0 : 𝜇 ≤ 500 𝑚𝑙 against 𝐻1 : 𝜇 > 500 𝑚𝑙 → the value of the test statistic 𝑧 = −2.5 falls into the
region of acceptance 𝑧 ≤ 2.492 → 𝐻0 accepted! One can assume that the amount of detergent is
≤ 500 𝑚𝑙.

In many practical applications, the question is whether the mean values of two normally distributed
populations are identical or substantially different. In the hypotheses about the equality of the
unknown mean values 𝜇1 (mean value of the population 𝑋) and 𝜇2 (mean value of the population
population 𝑌), corresponding to the samples 𝑥1 , 𝑥2 , … 𝑥𝑛 and 𝑦1 , 𝑦2 , … 𝑦𝑛 one can define the respective
test problems. One distinguishes between the following cases:

a) the variances 𝜎12 and 𝜎22 are known: The test statistics are based on a standardized random variable
which is standard normally distributed with mean µ = 0 and standard deviation σ = 1.

b) the variances 𝜎12 and 𝜎22 are equal (𝜎12 = 𝜎22 ) but unknown: The test statistics are based on a t-
distributed random variable with 𝑓 = 𝑛 − 1 degrees of freedom.
The test procedure is summarized as follows:

𝑋 and 𝑌 are two normally distributed random variables with the unknown mean values 𝜇1 and 𝜇2 . The
test procedure is based on two concrete independent samples 𝑥1 , 𝑥2 , … 𝑥𝑛 of the circumference 𝑛 = 𝑛1
and 𝑦1 , 𝑦2 , … 𝑦𝑛 of the circumference 𝑛 = 𝑛2 :
1) Formulation of the test problem (null hypothesis 𝐻0 and the alternative hypothesis 𝐻1 ) on the basis
of the specific question. The following test procedures are common:
a) 𝐻0 : 𝜇1 = 𝜇2 against 𝐻1 : 𝜇1 ≠ 𝜇2 .
b) 𝐻0 : 𝜇1 ≥ 𝜇2 against 𝐻1 : 𝜇1 < 𝜇2 .
c) 𝐻0 : 𝜇1 ≤ 𝜇2 against 𝐻1 : 𝜇1 > 𝜇2 .

2) Select significance level 𝛼: Typical values for the level of significance are 1 % and 5 %.
3) Determination of the region of acceptance and rejection: The interval limits are either based on the
corresponding quantiles of the standard normal distribution (if the variance is known), or on the
quantiles of the t-distribution of Student with 𝑓 = 𝑛 − 1 degrees of freedom (if the variance is
unknown).
For the test problem 𝑎) the region of acceptance is ≤ 𝑧 ≤ 𝑧1−𝛼/2 or 𝑡𝛼/2 ≤ 𝑡 ≤ 𝑡1−𝛼/2 .
For the test problem 𝑏) the region of acceptance is 𝑧 ≥ 𝑧𝛼 or 𝑡 ≥ 𝑡𝛼 .
For the test problem 𝑐) the region of acceptance is 𝑧 ≤ 𝑧1−𝛼 or 𝑡 ≤ 𝑡1−𝛼 .
4) Calculation of the test statistic:
 If the variance is known:

88
 𝜎12 𝜎22
(
𝑧 = 𝑥̅ − 𝑦̅)/√ + .
𝑛1 𝑛2

For 𝑛1 > 30 and 𝑛2 > 30, instead of 𝜎12 and 𝜎22 , the variances 𝑠12 and 𝑠12 of the two samples can be
used:

𝑠12 𝑠22
𝑧 = ( 𝑥̅ − 𝑦̅)/√ + .
𝑛1 𝑛2

 If the variance is not known:

𝑛1 𝑛 2
𝑡 =√ ∙ ( 𝑥̅ − 𝑦̅)/𝑠 , 𝑠 2 = [(𝑛1 − 1)𝑠12 + (𝑛2 − 1)𝑠22 ]/(𝑛1 + 𝑛2 − 2).
𝑛1 +𝑛2

If 𝑛1 = 𝑛2 = 𝑛, then the formula is simplified as follows:

( 𝑥̅ − 𝑦̅)
𝑡 = .
2 2
√𝑠1 + 𝑠2
𝑛 𝑛
Here are:
1
x̅ – the mean value of the sample 𝑋 (calculated as 𝑥̅ = (𝑥1 + 𝑥2 + ⋯ + 𝑥𝑛 );
𝑛
1
y̅ – the mean value of the sample 𝑌 (calculated as 𝑦̅ = 𝑛 (𝑦1 + 𝑦2 + ⋯ + 𝑦𝑛 );

𝜎12 and 𝜎22 – the variances of 𝑋 and 𝑌;

𝑠12 and 𝑠12 – the variances of both samples;
𝑛1 and 𝑛2 – the sample sizes of the samples.
5) Test decision: If the test statistics determined from the concrete sample fall into region of
acceptance, the null hypothesis 𝐻0 is retained. If the test statistics determined from the concrete
sample fall into the region of rejection, the null hypothesis must be rejected in favour of the alternative
hypothesis.

89
[1] Fahrmeir L., Künstler R., Pigeot I., Tutz G.. Statistik : der Weg zur Datenanalyse. Springer-
Verlag, Berlin, 2007.
[2] Hartung J., Elpelt B., Klösener K-H. Statistik : Lehr- und Handbuch der angewandten Statistik
[mit zahlreichen durchgerechneten Beispielen], Olendbourg, München, 2009.
[3] Mackowiak P. A., Wasserman S. S., and Levine M. M."A Critical Appraisal of 98.6 Degrees F,
the Upper Limit of the Normal Body Temperature, and Other Legacies of Carl Reinhold August
Wunderlich," Journal of the American Medical Association, 268, 1578-1580, 1992.
[4] Matthews R. Storks deliver babies (p=0.008)Teaching Statistics, 2000 (2),36-38
[5] Papula L. Mathematik für Ingenieure und Naturwissenschaftler. Band 3.
Vieweg+Teubner,Wiesbaden, 2011.
[6] RKI, Robert Koch-Institut. Referenzperzentile für anthropometrische Maßzahlen und Blutdruck
aus der Studie zur Gesundheit von Kindern und Jugendlichen in Deutschland (KiGGS), 2013.
[7] Shoemaker A.L. What's Normal? -- Temperature, Gender, and Heart Rate, Journal of Statistics
Education v.4, n.2, 1996.
[8] Whitaker C. Ask Marilyn. Parade Magazine, 1990.

90
91
For z ≥ 4, Φ (z) ≈ 1 holds.

z 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09

0 0.5000 0.5040 0.5080 0.5120 0.5160 0.5199 0.5239 0.5279 0.5319 0.5359
0.1 0.5398 0.5438 0.5478 0.5517 0.5557 0.5596 0.5636 0.5675 0.5714 0.5753
0.2 0.5793 0.5832 0.5871 0.5910 0.5948 0.5987 0.6026 0.6064 0.6103 0.6141
0.3 0.6179 0.6217 0.6255 0.6293 0.6331 0.6368 0.6406 0.6443 0.6480 0.6517
0.4 0.6554 0.6591 0.6628 0.6664 0.6700 0.6736 0.6772 0.6808 0.6844 0.6879

0.5 0.6915 0.6950 0.6985 0.7019 0.7054 0.7088 0.7123 0.7157 0.7190 0.7224
0.6 0.7257 0.7291 0.7324 0.7357 0.7389 0.7422 0.7454 0.7486 0.7517 0.7549
0.7 0.7580 0.7611 0.7642 0.7673 0.7704 0.7734 0.7764 0.7794 0.7823 0.7852
0.8 0.7881 0.7910 0.7939 0.7967 0.7995 0.8023 0.8051 0.8078 0.8106 0.8133
0.9 0.8159 0.8186 0.8212 0.8238 0.8264 0.8289 0.8315 0.8340 0.8365 0.8389

1 0.8413 0.8438 0.8461 0.8485 0.8508 0.8531 0.8554 0.8577 0.8599 0.8621
1.1 0.8643 0.8665 0.8686 0.8708 0.8729 0.8749 0.8770 0.8790 0.8810 0.8830
1.2 0.8849 0.8869 0.8888 0.8907 0.8925 0.8944 0.8962 0.8980 0.8997 0.9015
1.3 0.9032 0.9049 0.9066 0.9082 0.9099 0.9115 0.9131 0.9147 0.9162 0.9177
1.4 0.9192 0.9207 0.9222 0.9236 0.9251 0.9265 0.9279 0.9292 0.9306 0.9319

1.5 0.9332 0.9345 0.9357 0.9370 0.9382 0.9394 0.9406 0.9418 0.9429 0.9441
1.6 0.9452 0.9463 0.9474 0.9484 0.9495 0.9505 0.9515 0.9525 0.9535 0.9545
1.7 0.9554 0.9564 0.9573 0.9582 0.9591 0.9599 0.9608 0.9616 0.9625 0.9633
1.8 0.9641 0.9649 0.9656 0.9664 0.9671 0.9678 0.9686 0.9693 0.9699 0.9706
1.9 0.9713 0.9719 0.9726 0.9732 0.9738 0.9744 0.9750 0.9756 0.9761 0.9767

2 0.9772 0.9778 0.9783 0.9788 0.9793 0.9798 0.9803 0.9808 0.9812 0.9817
2.1 0.9821 0.9826 0.9830 0.9834 0.9838 0.9842 0.9846 0.9850 0.9854 0.9857
2.2 0.9861 0.9864 0.9868 0.9871 0.9875 0.9878 0.9881 0.9884 0.9887 0.9890
2.3 0.9893 0.9896 0.9898 0.9901 0.9904 0.9906 0.9909 0.9911 0.9913 0.9916
2.4 0.9918 0.9920 0.9922 0.9925 0.9927 0.9929 0.9931 0.9932 0.9934 0.9936

2.5 0.9938 0.9940 0.9941 0.9943 0.9945 0.9946 0.9948 0.9949 0.9951 0.9952
2.6 0.9953 0.9955 0.9956 0.9957 0.9959 0.9960 0.9961 0.9962 0.9963 0.9964
2.7 0.9965 0.9966 0.9967 0.9968 0.9969 0.9970 0.9971 0.9972 0.9973 0.9974
2.8 0.9974 0.9975 0.9976 0.9977 0.9977 0.9978 0.9979 0.9979 0.9980 0.9981
2.9 0.9981 0.9982 0.9982 0.9983 0.9984 0.9984 0.9985 0.9985 0.9986 0.9986

3 0.9987 0.9987 0.9987 0.9988 0.9988 0.9989 0.9989 0.9989 0.9990 0.9990
3.1 0.9990 0.9991 0.9991 0.9991 0.9992 0.9992 0.9992 0.9992 0.9993 0.9993
3.2 0.9993 0.9993 0.9994 0.9994 0.9994 0.9994 0.9994 0.9995 0.9995 0.9995
3.3 0.9995 0.9995 0.9995 0.9996 0.9996 0.9996 0.9996 0.9996 0.9996 0.9997
3.4 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9997 0.9998

92
 p up p up p: given probability
0.900 1.282 0.100 -1.282
up: quantile referring to the probability p
0.950 1.645 0.050 -1.645
of the standard normal distribution
0.975 1.960 0.025 -1.960
0.990 2.326 0.010 -2.326
0.995 2.576 0.005 -2.576
0.999 3.090 0.001 -3.090

p
f 0.8 0.85 0.9 0.95 0.975 0.99 0.995
p: given probability
1 1.376 1.963 3.078 6.314 12.706 31.821 63.656
2 1.061 1.386 1.886 2.920 4.303 6.965 9.925 f: number of the degrees of
3 0.978 1.250 1.638 2.353 3.182 4.541 5.841 freedom
4 0.941 1.190 1.533 2.132 2.776 3.747 4.604
5 0.920 1.156 1.476 2.015 2.571 3.365 4.032 The table values are the
6 0.906 1.134 1.440 1.943 2.447 3.143 3.707 𝑡𝑝,𝑓 quantiles of Student’s t-
7 0.896 1.119 1.415 1.895 2.365 2.998 3.499 distribution.
8 0.889 1.108 1.397 1.860 2.306 2.896 3.355
9 0.883 1.100 1.383 1.833 2.262 2.821 3.250
10 0.879 1.093 1.372 1.812 2.228 2.764 3.169
11 0.876 1.088 1.363 1.796 2.201 2.718 3.106
Formulas:
12 0.873 1.083 1.356 1.782 2.179 2.681 3.055
𝑡(1−𝑝,𝑓) = −𝑡𝑝,𝑓
13 0.870 1.079 1.350 1.771 2.160 2.650 3.012
14 0.868 1.076 1.345 1.761 2.145 2.624 2.977 𝑡𝑝,𝑓 = −𝑡(1−𝑝,𝑓)
15 0.866 1.074 1.341 1.753 2.131 2.602 2.947
16 0.865 1.071 1.337 1.746 2.120 2.583 2.921
17 0.863 1.069 1.333 1.740 2.110 2.567 2.898
18 0.862 1.067 1.330 1.734 2.101 2.552 2.878
19 0.861 1.066 1.328 1.729 2.093 2.539 2.861
20 0.860 1.064 1.325 1.725 2.086 2.528 2.845
21 0.859 1.063 1.323 1.721 2.080 2.518 2.831
22 0.858 1.061 1.321 1.717 2.074 2.508 2.819
23 0.858 1.060 1.319 1.714 2.069 2.500 2.807
24 0.857 1.059 1.318 1.711 2.064 2.492 2.797
25 0.856 1.058 1.316 1.708 2.060 2.485 2.787
30 0.854 1.055 1.310 1.697 2.042 2.457 2.750
40 0.851 1.050 1.303 1.684 2.021 2.423 2.704
50 0.849 1.047 1.299 1.676 2.009 2.403 2.678
60 0.848 1.045 1.296 1.671 2.000 2.390 2.660
70 0.847 1.044 1.294 1.667 1.994 2.381 2.648
80 0.846 1.043 1.292 1.664 1.990 2.374 2.639
90 0.846 1.042 1.291 1.662 1.987 2.368 2.632
100 0.845 1.042 1.290 1.660 1.984 2.364 2.626

93
p: given probability
f: number of degrees of freedom
The table values are the zp,f quantiles of the chi-squared distribution.

p
f 0.005 0.010 0.025 0.050 0.100 0.500 0.900 0.950 0.975 0.990 0.995
1 0.000 0.000 0.001 0.004 0.016 0.455 2.706 3.841 5.024 6.635 7.879
2 0.010 0.020 0.051 0.103 0.211 1.386 4.605 5.991 7.378 9.210 10.597
3 0.072 0.115 0.216 0.352 0.584 2.366 6.251 7.815 9.348 11.345 12.838
4 0.207 0.297 0.484 0.711 1.064 3.357 7.779 9.488 11.143 13.277 14.860
5 0.412 0.554 0.831 1.145 1.610 4.351 9.236 11.070 12.832 15.086 16.750
6 0.676 0.872 1.237 1.635 2.204 5.348 10.645 12.592 14.449 16.812 18.548
7 0.989 1.239 1.690 2.167 2.833 6.346 12.017 14.067 16.013 18.475 20.278
8 1.344 1.647 2.180 2.733 3.490 7.344 13.362 15.507 17.535 20.090 21.955
9 1.735 2.088 2.700 3.325 4.168 8.343 14.684 16.919 19.023 21.666 23.589
10 2.156 2.558 3.247 3.940 4.865 9.342 15.987 18.307 20.483 23.209 25.188
11 2.603 3.053 3.816 4.575 5.578 10.341 17.275 19.675 21.920 24.725 26.757
12 3.074 3.571 4.404 5.226 6.304 11.340 18.549 21.026 23.337 26.217 28.299
13 3.565 4.107 5.009 5.892 7.042 12.340 19.812 22.362 24.736 27.688 29.819
14 4.075 4.660 5.629 6.571 7.790 13.339 21.064 23.685 26.119 29.141 31.319
15 4.601 5.229 6.262 7.261 8.547 14.339 22.307 24.996 27.488 30.578 32.801
16 5.142 5.812 6.908 7.962 9.312 15.338 23.542 26.296 28.845 32.000 34.267
17 5.697 6.408 7.564 8.672 10.085 16.338 24.769 27.587 30.191 33.409 35.718
18 6.265 7.015 8.231 9.390 10.865 17.338 25.989 28.869 31.526 34.805 37.156
19 6.844 7.633 8.907 10.117 11.651 18.338 27.204 30.144 32.852 36.191 38.582
20 7.434 8.260 9.591 10.851 12.443 19.337 28.412 31.410 34.170 37.566 39.997
21 8.034 8.897 10.283 11.591 13.240 20.337 29.615 32.671 35.479 38.932 41.401
22 8.643 9.543 10.982 12.338 14.041 21.337 30.813 33.924 36.781 40.289 42.796
23 9.261 10.196 11.689 13.091 14.848 22.337 32.007 35.172 38.076 41.638 44.181
24 9.886 10.856 12.401 13.848 15.659 23.337 33.196 36.415 39.364 42.980 45.559
25 10.520 11.524 13.120 14.611 16.473 24.337 34.382 37.652 40.646 44.314 46.928
26 11.160 12.198 13.844 15.379 17.292 25.336 35.563 38.885 41.923 45.642 48.290
27 11.808 12.879 14.573 16.151 18.114 26.336 36.741 40.113 43.195 46.963 49.645
28 12.461 13.565 15.308 16.928 18.939 27.336 37.916 41.337 44.461 48.278 50.994
29 13.121 14.257 16.047 17.708 19.768 28.336 39.087 42.557 45.722 49.588 52.336
30 13.787 14.953 16.791 18.493 20.599 29.336 40.256 43.773 46.979 50.892 53.672
31 14.458 15.655 17.539 19.281 21.434 30.336 41.422 44.985 48.232 52.191 55.003
32 15.134 16.362 18.291 20.072 22.271 31.336 42.585 46.194 49.480 53.486 56.328
33 15.815 17.074 19.047 20.867 23.110 32.336 43.745 47.400 50.725 54.776 57.648
34 16.501 17.789 19.806 21.664 23.952 33.336 44.903 48.602 51.966 56.061 58.964
35 17.192 18.509 20.569 22.465 24.797 34.336 46.059 49.802 53.203 57.342 60.275
40 20.707 22.164 24.433 26.509 29.051 39.335 51.805 55.758 59.342 63.691 66.766
50 27.991 29.707 32.357 34.764 37.689 49.335 63.167 67.505 71.420 76.154 79.490
60 35.534 37.485 40.482 43.188 46.459 59.335 74.397 79.082 83.298 88.379 91.952
70 43.275 45.442 48.758 51.739 55.329 69.334 85.527 90.531 95.023 100.425 104.215
80 51.172 53.540 57.153 60.391 64.278 79.334 96.578 101.879 106.629 112.329 116.321
90 59.196 61.754 65.647 69.126 73.291 89.334 107.565 113.145 118.136 124.116 128.299
100 67.328 70.065 74.222 77.929 82.358 99.334 118.498 124.342 129.561 135.807 140.169

94