Lecture #5

Statistical Learning

Dr. Debasis Samanta

Associate Professor
Department of Computer Science & Engineering

DSamanta@IIT Kharagpur Data Analytics (CS61061) 1

 Quote of the day..

Live as if you were to die tomorrow. Learn as if

you were to live forever.

— MAHATMA GANDHI, father of nation of India

DSamanta@IIT Kharagpur Data Analytics (CS61061) 2

In this presentation…
— Introduction to Statistical Learning

— Hypothesis Testing in Statistical Learning

— Statistical Learning Procedures

— Case Study 1: Coffee Sale (t-Test)

— Case Study 2: Machine Testing (z-test)

— Case Study 3: Perceptual Psychology (𝜒 ! -test)

— Discussion on Statistical Learning

DSamanta@IIT Kharagpur Data Analytics (CS61061) 3

 Introduction to Statistical Learning

DSamanta@IIT Kharagpur Data Analytics (CS61061) 4

Introduction

What do you think about this piece?

DSamanta@IIT Kharagpur Data Analytics (CS61061) 5

Introduction
The primary objective of statistical analysis is to use data from a sample to make
inferences about the population statistics from which the sample was drawn.

The mean and

µ, σ variance of
students in the
entire country?

This lecture aims to learn the basic

procedures for making such
inferences.

Sample

Mean and variance

𝑋! , S of GATE scores of
all students of IIT-
KGP

DSamanta@IIT Kharagpur Data Analytics (CS61061) 6

… Starting point
— Data from a population

DSamanta@IIT Kharagpur Data Analytics (CS61061) 7

… Starting point
— Distribution of population data (Normal distribution)

&
! %("#$) &&'&
𝑓 𝑥 = 𝑒 −∞ < 𝑥 < ∞
" #$

DSamanta@IIT Kharagpur Data Analytics (CS61061) 8

… Starting point
— Distribution of population data (Standard normal
distribution).

! '& %(' &

𝑓(𝑧: 0, 𝜎) = ∫ 𝑒 & 𝑑𝑧 −∞<𝑧 <∞
" #$ '(

𝑓(𝑧: 0, 𝜎)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 9

… Starting point
— Distribution of population data (Standard normal
distribution…..
&
! %("#$) &&'&
𝑓 𝑥 = 𝑒 −∞ < 𝑥 < ∞
" #$

234
The z-transformation 𝑧= 5
; P(Z=z) ≈ P(X=x)

! '& %(' &

𝑓(𝑧: 0, 𝜎) = " #$
∫' 𝑒 & 𝑑𝑧 −∞<𝑧 <∞
(

𝑓(𝑧: 0, 𝜎)
DSamanta@IIT Kharagpur Data Analytics (CS61061) 10
… Starting point
— Population to samples…..

Samples

DSamanta@IIT Kharagpur Data Analytics (CS61061) 11

… Starting point
— Distribution of a sample’s data

Samples

DSamanta@IIT Kharagpur Data Analytics (CS61061) 12

… Starting point
— Central Limit theorem…(Distribution of samples’ statistics).

Population’s statistics

𝑓(𝑧: 0, 𝜎) 634
z-estimation 𝑧=#
7 $

Samples’ statistics

𝑓(𝑧: 0, "7 ()
DSamanta@IIT Kharagpur Data Analytics (CS61061) 13
… Starting point
— The interpretation of z estimation

Samples
Sample statistics

DSamanta@IIT Kharagpur Data Analytics (CS61061) 14

 Hypothesis Testing

DSamanta@IIT Kharagpur Data Analytics (CS61061) 15

Hypothesis Testing
What is Hypothesis?
— “A hypothesis is an educated prediction that can be tested” ([Link]).

— “A hypothesis is a proposed explanation for a phenomenon” (Wikipedia).

— “A hypothesis is used to define the relationship between two variables” (Oxford

dictionary).
— “A supposition or proposed explanation made on the basis of limited evidence as a
starting point for further investigation” (Walpole).

— Example 5.1: Avogadro’s Hypothesis(1811)

“The volume of a gas is directly proportional to the number of molecules of the gas.”
𝑽 = 𝒂𝑵

DSamanta@IIT Kharagpur Data Analytics (CS61061) 16

Basic Approach: Hypothesis Testing
The approach:

— Conduct a test on hypothesis.

— Hypothesize that one (or more) parameter(s) has
(have) some specific value(s) or relationship.

— Make your decision about the parameter(s) based on one

(or more) sample statistic(s)

— Accuracy of the decision is expressed as the probability

that the decision is incorrect.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 17

Hypothesis Testing

Statistical inference

Sample

Null hypothesis Alternative hypothesis

DSamanta@IIT Kharagpur Data Analytics (CS61061) 18

A Simple Example
— There are two identically appearing boxes of balls. Box 1 contains 60 red and
40 blue balls, and Box 2 contains 40 red and 60 blue balls.

What is the probability that you pick a ball from Box 1 is read?

DSamanta@IIT Kharagpur Data Analytics (CS61061) 19

An Example of Hypothesis
— One of the box is sitting on the table, but you don’t know which one it is. You
have to guess which box it is: Box 1 or Box 2?

Hypothesis is that Box 2 is on the table. How to test the hypothesis?

DSamanta@IIT Kharagpur Data Analytics (CS61061) 20

Hypothesis: An Example
— To test the hypothesis, you collect a sample of five balls, say. The data from
this sample, specifically the number of red balls, is the sample data.

Such a sample can be used to test the hypothesis that Box 2 is on the table.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 21

Hypothesis: An Example

— Hypothesis is that

Box 2 is on the table.

H0: p = 0.4
Box 2 contains 40 red and 60 blue balls.

Such a hypothesis is called Null Hypothesis.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 22

 Hypothesis: An Example

— Alternate hypothesis

Box 1 is on the table.

Box 1 contains 60 red and 40 blue balls

H1: p = 0.6

Such two hypotheses are called Statistical Hypothesis.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 23

Statistical Hypothesis
— If the hypothesis is stated in terms of population parameters (such as mean
and variance), the hypothesis is called statistical hypothesis.

— Data from a sample (which may be an experiment) are used to test the
validity of the hypothesis.
— A procedure that enables us to agree (or disagree) with the statistical hypothesis is
called a test of the hypothesis.

Example 5.2:
1. To determine whether the wages of men and women are equal.

2. A product in the market is of standard quality.

3. Whether a particular medicine is effective to cure a disease.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 24

The Hypotheses
— The main purpose of statistical hypothesis testing is to choose between two
competing hypotheses.
Example 5.3:
One hypothesis might claim that wages of men and women are equal, while the
alternative might claim that men make more than women.

— Hypothesis testing start by making a set of two statements about the

parameter(s) in question.

— The hypothesis actually to be tested is usually given the symbol 𝐻! and is

commonly referred as the null hypothesis.

— The other hypothesis, which is assumed to be true when null hypothesis is

false, is referred as the alternate hypothesis and is often symbolized by 𝐻"

— The two hypotheses are exclusive and exhaustive.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 25

The Hypotheses
Example 5.4:
Ministry of Human Resource Development (MHRD), Government of India takes an
initiative to improve the country’s human resources and hence set up 23 IIT’s in the
country.

To measure the engineering aptitudes of graduates, MHRD conducts GATE

examination for a mark of 1000 in every year. A sample of 300 students who gave
GATE examination in 2021 were collected and the mean is observed as 220.

In this context, statistical hypothesis testing is to determine the mean mark of the all
GATE-2021 examinee.

The two hypotheses in this context are:

𝐻% : 𝜇 = 220
𝐻& : 𝜇 < 220

DSamanta@IIT Kharagpur Data Analytics (CS61061) 26

The Hypotheses

Note:
1. As null hypothesis, we could choose 𝐻! : 𝜇 ≤ 220 or 𝐻! : 𝜇 ≥ 220
2. It is customary to always have the null hypothesis with an equal sign.

3. As an alternative hypothesis there are many options available with us.

Examples 5.5:
I. 𝐻! : 𝜇 < 220
II. 𝐻! : 𝜇 > 220
III. 𝐻! : 𝜇 ≠ 220

4. The two hypothesis should be chosen in such a way that they are exclusive
and exhaustive.
— One or other must be true, but they cannot both be true.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 27

The Hypotheses
One-tailed test

— A statistical test in which the alternative hypothesis specifies that the

population parameter lies entirely above or below the value specified in 𝐻!
is called a one-sided (or one-tailed) test.

Example.
𝐻! : 𝜇 = 100 𝐻" : 𝜇 > 100

Two-tailed test
— An alternative hypothesis that specifies that the parameter can lie on their
sides of the value specified by 𝐻! is called a two-sided (or two-tailed) test.

Example.
𝐻! : 𝜇 = 100 𝐻" : 𝜇 <> 100
DSamanta@IIT Kharagpur Data Analytics (CS61061) 28
The Hypotheses
Note:
In fact, a 1-tailed test such as:

𝐻) : 𝜇 = 100
𝐻! : 𝜇 > 100
is same as

𝐻) : 𝜇 ≤ 100
𝐻! : 𝜇 > 100
In essence, 𝜇 > 100, it does not imply that 𝜇 > 80, 𝜇 > 90, etc.

What about two-tailed test?

𝐻) : 𝜇 = 100 𝐻! : 𝜇 <> 100

DSamanta@IIT Kharagpur Data Analytics (CS61061) 29

 Hypothesis Testing Procedure

DSamanta@IIT Kharagpur Data Analytics (CS61061) 30

Hypothesis Testing

DSamanta@IIT Kharagpur Data Analytics (CS61061) 31

Hypothesis Testing Procedures
The following five steps are followed when testing hypothesis

1. Specify 𝐻! and 𝐻" , the null and alternate hypothesis. Also, decide the
significance level of hypothesis test (it is denoted as 𝛼 and signifies the
rejection region for the specified 𝐻! ).

2. Determine an appropriate sampling distribution for testing.

3. Collect the sample data and calculate the test statistics.

4. Make a decision to either reject or fail to reject 𝐻! .

5. Interpret the result in common language suitable for practitioners.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 32

Note: Hypothesis Testing
— In summary, we have to choose between 𝐻! and 𝐻"

— The standard procedure is to assume 𝐻! is true.

(Just we presume innocent until proven guilty)

— Using statistical test, we try to determine whether there is sufficient

evidence to declare 𝐻! false.

— We reject 𝐻! only when the chance is small that 𝐻! is true.

— The procedure is based on probability theory, that is, there is a chance that
we can make errors.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 33

Significance Level of a Hypothesis Test

DSamanta@IIT Kharagpur Data Analytics (CS61061) 34

Errors in Hypothesis Testing
In hypothesis testing, there are two types of errors.
Type I error: A type I error occurs when we incorrectly reject 𝐻! (i.e., we
reject the null hypothesis, when 𝐻! is true).

Type II error: A type II error occurs when we incorrectly fail to reject 𝐻!

(i.e., we accept 𝐻! when it is not true).

DSamanta@IIT Kharagpur Data Analytics (CS61061) 35

Probabilities of Making Errors
Type I error calculation
𝜶: denotes the probability of making a Type I error

𝜶 = 𝐏 Rejecting 𝐻% 𝐻% is true)

Type II error calculation

𝜷: denotes the probability of making a Type II error

𝛃 = 𝐏 Accepting 𝐻) 𝐻) is false)
Note:
— 𝜶 and 𝛃 are not independent of each other as one increases, the other decreases.
— When the sample size increases, both to decrease since sampling error is reduced.
— In general, we focus on Type I error, but Type II error is also important,
particularly when sample size is small.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 36

Acceptable level of 𝛼
Assuming that we have the results of random sample. Hence, we use the
characteristics of sampling distribution to calculate the probabilities of making
either Type I or Type II error.

Example 5.6:
Suppose, two hypotheses in a statistical testing are:

𝐻! : 𝜇 = 𝑎
𝐻" : 𝜇 ≠ 𝑎

Also, assume that for a given sample, population obeys normal distribution. A
threshold limit say 𝑎 ± 𝛿 is used to say that they are significantly different from
a.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 37

Acceptable level of 𝛼

DSamanta@IIT Kharagpur Data Analytics (CS61061) 38

Calculating 𝛼

Here, shaded region implies the probability

that, X < 𝑎 − 𝛿 𝑜𝑟 X > 𝑎 + 𝛿
a - δ a a + δ

Thus the null hypothesis is to be rejected if the mean value is less than 𝑎 − 𝛿 or
greater than 𝑎 + 𝛿.

If X denotes the sample mean, then the Type I error is

𝛼 = 𝑃(X < 𝑎 − 𝛿 𝑜𝑟 X > 𝑎 + 𝛿, 𝑤ℎ𝑒𝑛 𝜇 = 𝑎 , i. e. , 𝐻) is true)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 39

The Rejection Region
The rejection region comprises of value of the test statistics for which
1. The probability when the null hypothesis is true is less than or equal to the specified 𝛼.
2. Probability when 𝐻! is true are greater than they are under 𝐻) .

a’ a a”
Rejection region for H0 for a
given value of α

Reject H0 Do not reject H0 Reject H0

! ≠a ! =a ! ≠a

DSamanta@IIT Kharagpur Data Analytics (CS61061) 40

The Significance Level 𝛼
— 𝛼 = 0%
— Always accepts the null hypothesis (Ultra liberal test)

— 𝛼 = 100%
— Always rejects the null hypothesis (Ultra conservative test)

— 𝛼 = 1% (Lesser probability to reject a 𝐻Z )

— 𝛼 = 5% (Higher probability to reject a 𝐻Z )

Note:
𝛼 is the maximum acceptable probability of
rejecting a true null hypothesis
a’ a a”
Rejection region for H0 for a
DSamanta@IIT Kharagpur Data Analytics (CS61061) given value of α 41
Two-Tailed Test
For two-tailed hypothesis test, hypotheses take the form
𝐻) : 𝜇 = 𝜇*)
𝐻! : 𝜇 ≠ 𝜇*)

In other words, to reject a null hypothesis, sample mean 𝜇 > 𝜇#! or 𝜇 < 𝜇#!
under a given 𝛼.

Thus, in a two-tailed test, there are two rejection regions (also known as critical
region), one on each tail of the sampling distribution curve.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 42

Two-Tailed Test
Acceptance region
Accept H0 ,if the sample
mean falls in this region

95 % of area

0.025 of area 0.025 of area

µH 0

Rejection region
Reject H0 ,if the sample mean falls
in either of these regions

Acceptance and rejection regions in case of a two-tailed test with 5% significance level.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 43

One-Tailed Test
A one-tailed test would be used when we are to test, say, whether the population mean is
either lower or higher than the hypothesis test value.

Symbolically,
𝐻) : 𝜇 = 𝜇*)
𝐻! : 𝜇 < 𝜇*) [𝑜𝑟 𝜇 > 𝜇*) ]

Wherein there is one rejection region only on the left-tail (or right-tail).
Acceptance region Acceptance region

.05 of area
.05 of area

Rejection region Rejection region

Left − tailed test Right − tailed test

DSamanta@IIT Kharagpur Data Analytics (CS61061) 44

Choosing the Rejection Boundary
When value of 𝛼 is known

𝛼 = Type I Error
= H0 is rejected when H0 is true
= P(Z falls in the rejection region when H0 is true)
= 𝑃(𝑍 < 𝑧! when 𝐻! = 𝜇)
or
= 𝑃(𝑍 > 𝑧!$ when 𝐻! = 𝜇)
or
= 𝑃(𝑍 > 𝑧!$ when 𝐻! = 𝜇) + 𝑃(𝑍 < 𝑧! when 𝐻! = 𝜇)

Here, 𝑧! and/or 𝑧!$ are the rejection region boundary and the value(s) can be
decided given a value of 𝛼 the level of significance.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 45

Choosing the Rejection Boundary
Example:
Consider the following for an application:
Population mean, 𝜇 = 8.0, population standard deviation, 𝜎 = 0.2, sample size, n = 16.
Decide the critical values for the rejection region given that 5% level of significance.
Here, 𝛼 = 0.05
Let, the critical values for the rejection region are C1 and C2 (say, two-tailed test).
Thus,
𝛼 = 𝑃(𝑍 > 𝑧)+ when 𝐻) = 8.0) + 𝑃(𝑍 < 𝑧) when 𝐻) = 8.0)
,(%-.) ,&%-.)
=𝑃 𝑍> +𝑃 𝑍 <
).#/ !0 ).#/ !0
Because of the symmetry of the normal distribution, exactly half of the rejection region is
in each tail. Therefore,
,(%-.) ,&%-.)
𝑃 𝑍> = 𝑃 𝑍< = 0.025
).#/ !0 ).#/ !0

DSamanta@IIT Kharagpur Data Analytics (CS61061) 46

Choosing the Rejection Boundary
Continued on…
, %-.)
( & , %-.)
𝑃 𝑍 > ).#/ !0
= 𝑃 𝑍 < ).#/ !0
= 0.025
From the statistical table of z-values, we get

%" &'.! %# &'.!

= 1.96 and = -1.96
!.)/ "+ !.)/ "+

Solving the above, we get

C1 = 8.098 and C2 = 7.902

DSamanta@IIT Kharagpur Data Analytics (CS61061) 47

Choosing the Value of 𝛼
When rejection boundary is given
Suppose, H0 is to be rejected when 𝑋F < 7.9 or 𝑋F > 8.1
Given population mean, 𝜇 = 8.0, population standard deviation, 𝜎 = 0.2, sample
size, n = 16.
Decide the value of 𝛼.
'."&'.! ,.-&'.!
𝛼=𝑃 𝑍> +𝑃 𝑍<
!.)/ "+ !.)/ "+
= 𝑃 𝑍 > 2.0 + 𝑃 𝑍 < − 2.0
= 0.0228 + 0.0228
= 0.0456

DSamanta@IIT Kharagpur Data Analytics (CS61061) 48

 Sampling Statistics for Testing

DSamanta@IIT Kharagpur Data Analytics (CS61061) 49

Parametric Tests and Sampling Distributions

The widely used sampling distribution for parametric tests are

— 𝑍 − test
— 𝑡 − test
— 𝜒 ! − test

Note:
All these tests are based on the assumption of normality (i.e., the source of data is
considered to be normally distributed).

DSamanta@IIT Kharagpur Data Analytics (CS61061) 50

Parametric Tests : Z-test
𝒁 − 𝐭𝐞𝐬𝐭: This is the simple most test in statistical learning.

— It is based on the normal probability distribution.

— Used for judging the significance of several statistical measures, particularly

the mean.

— Typically it is used for comparing the mean of a sample to some

hypothesized mean for the population in case of large sample, or
when population variance is known.

𝑋b − 𝜇
z=
𝜎/ 𝑛

DSamanta@IIT Kharagpur Data Analytics (CS61061) 51

Parametric Tests : t-test
𝒕 − 𝐭𝐞𝐬𝐭: It is based on the t-distribution.

— It is considered an appropriate test for judging the significance of a sample

mean or for judging the significance of difference between the means of two
samples in case of

— small sample(s)

— population variance is not known (in this case, we use the

variance of the sample as an estimate of the population
variance)
𝑋b − 𝜇
𝑡=
𝑆/ 𝑛

DSamanta@IIT Kharagpur Data Analytics (CS61061) 52

 𝟐
Parametric Tests : 𝝌 -test

𝝌𝟐 − 𝐭𝐞𝐬𝐭: It is based on Chi-squared distribution.

— It is used for comparing a sample variance to a theoretical

population variance.

𝑛 − 1 𝑆#
𝜒# =
𝜎#

DSamanta@IIT Kharagpur Data Analytics (CS61061) 53

Consultation of Statistical Tables
𝐇𝐨𝐰 𝐭𝐨 𝐟𝐢𝐧𝐝 𝐭−, 𝐳 − 𝐚𝐧𝐝 𝛘𝟐 𝐯𝐚𝐥𝐮𝐞𝐬?

𝐌𝐞𝐭𝐡𝐨𝐝 𝟏: 𝐁𝐚𝐬𝐞𝐝 𝐨𝐧 𝐏𝐃𝐅 𝐨𝐟 𝐚 𝐝𝐢𝐬𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧

! '& %(' &
𝑓(𝑧: 0, 𝜎) = ∫ 𝑒 & 𝑑𝑧
" #$ '(

𝐌𝐞𝐭𝐡𝐨𝐝 𝟐: 𝐓𝐨 𝐟𝐢𝐧𝐝 𝐚 𝐭−, 𝐳 − 𝐚𝐧𝐝 𝛘𝟐 𝐯𝐚𝐥𝐮𝐞 𝐮𝐬𝐢𝐧𝐠 𝐬𝐭𝐚𝐭𝐢𝐬𝐭𝐢𝐜𝐚𝐥 𝐭𝐚𝐛𝐥𝐞𝐬

DSamanta@IIT Kharagpur Data Analytics (CS61061) 54

 Case Study 1: t-Test

DSamanta@IIT Kharagpur Data Analytics (CS61061) 55

Case Study 1: Coffee Sale
A coffee vendor nearby Nalanda Academic complex has been having average
sales of 500 cups per day. Because of the development of another wing in the
complex, it expects to increase its sales. During the first 12 days, after the
inauguration of the new wing, the daily sales were as under:

550 570 490 615 505 580 570 460 600 580 530 526

On the basis of this sample information, can we conclude that the sales of coffee
have increased?

Consider 5% as the significance level of testing.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 56

Hypothesis Testing : 5 Steps
The following five steps are followed when testing hypothesis

1. Specify 𝐻! and 𝐻" , the null and alternate hypothesis, and significance level
of hypothesis test 𝛼.

2. Collect the sample data and calculate the test statistics.

3. Determine the critical value for parametric test.

4. Make a decision to either reject or fail to reject 𝐻! .

5. Interpret the result in common language suitable for practitioner.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 57

Case Study 1: Step 1
Step 1: Specification of hypotheses and level of significance 𝛂

Let us consider the hypotheses for the given problem as follows.

𝐻) : 𝜇 = 500 cups per day

The null hypothesis that sales average 500 cups per day and they have not
increased.

𝐻! : 𝜇 > 500
The alternative hypothesis is that the sales have increased.

Given the value of 𝛼 = 0.05 (𝑖. 𝑒. , 5% 𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑐𝑎𝑛𝑐𝑒 𝑙𝑒𝑣𝑒𝑙 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑒𝑠𝑡)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 58

Case Study 1: Step 2
Step 2: Sample-based test statistics and the rejection region for specified 𝐇𝟎

Given the sample as

550 570 490 615 505 580 570 460 580 530 526

Since the sample size is small and the population standard deviation is not known, we
shall use 𝑡 − 𝑡𝑒𝑠𝑡 assuming normal population. The test statistics 𝑡 is
𝑋b − 𝜇
𝑡=
𝑆/ 𝑛

To find 𝑋b and 𝑆, we make the following computations.

b ∑ 2* = 0340 = 548
𝑋=
( !#

DSamanta@IIT Kharagpur Data Analytics (CS61061) 59

Case Study 1: Step 2
𝑆𝑎𝑚𝑝𝑙𝑒 # 𝑿𝒊 7
𝑿𝒊 − 𝑿 7 )𝟐
(𝑿𝒊 − 𝑿
1 550 2 4
2 570 22 484
3 490 −58 3364
4 615 67 4489
5 505 −43 1849
6 580 32 1024
7 570 22 484
8 460 −88 7744
9 600 52 2704
10 580 32 1024
11 530 −18 324
12 526 −22 484

𝑛 = 12 F 𝑋& = 6576 H ' = 23978

F(𝑋& −𝑋)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 60

Case Study 1: Step 2

b #
∑(𝑋5 − 𝑋) 23978
𝑆= = = 46.68
𝑛−1 12 − 1

0
1&2 5' 5'
Hence, 𝑡 = = = = 3.558
3/ 4 5+.+'/ ") "6.5-

Note:
This gives a t-value given a sample of size n.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 61

Case Study 1: Step 3

Step 3: Decide the critical value for the hypothesis test

𝐷𝑒𝑔𝑟𝑒𝑒 𝑜𝑓 𝑓𝑟𝑒𝑒𝑑𝑜𝑚 = 𝑛 − 1 = 12 − 1 = 11

As 𝐻! is one-tailed, we shall determine the rejection region applying one-tailed in the

right tail because 𝐻! is more than type ) at 5% level of significance.

Using table of 𝑡 − 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 for 11 degrees of freedom and with 5% level of

significance,
Rejection region: 𝒕 > 𝟏. 𝟕𝟗𝟔

DSamanta@IIT Kharagpur Data Analytics (CS61061) 62

Case Study 1: Step 4
Step 4: Make a decision to either reject or fail to reject H0

Acceptance region 1.796

.05 of area

Rejection region 3.558

The observed value of 𝑡 = 3.558 which is in the rejection region and thus 𝐻) is rejected
at 5% level of significance.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 63

Case Study 1: Step 5
Step 5: Final comment and interpret the result

We can conclude that the sample data indicate that coffee sales have increased.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 64

Comments on Case Study 1
Step 1: Specification of hypotheses and significance level 𝛂

Let us consider the hypotheses for the given problem as follows.

𝐻) : 𝜇 = 500 cups per day

The null hypothesis that sales average 500 cups per day and they have not
increased.

𝐻! : 𝜇 > 500
The alternative hypothesis is that the sales have increased.

Given the value of 𝛼 = 0.01 (𝑖. 𝑒. , 1% 𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑐𝑎𝑛𝑐𝑒 𝑙𝑒𝑣𝑒𝑙 𝑜𝑓 𝑡ℎ𝑒 𝑡𝑒𝑠𝑡)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 65

 Case Study 2: z-Test

DSamanta@IIT Kharagpur Data Analytics (CS61061) 66

Case Study 2: Machine Testing
A medicine production company packages medicine in a tube of 8 ml. In
maintaining the control of the amount of medicine in tubes, they use a machine.
To monitor this control a sample of 16 tubes is taken from the production line at
random time interval and their contents are measured precisely. The mean amount
of medicine in these 16 tubes will be used to test the hypothesis that the machine
is indeed working properly. Maximum variance that can be allowed is 0.2.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 67

Case Study 2: Step 1

Step 1: Specification of hypothesis and level of significance 𝛂

The hypotheses are given in terms of the population mean of medicine per tube.

The null hypothesis is

𝐻! : 𝜇 = 8

The alternative hypothesis is

𝐻" : 𝜇 ≠ 8

We assume 𝛼, the significance level in our hypothesis testing ≈ 0.05.

(This signifies the probability that the machine needs to be adjusted less than 5%).

DSamanta@IIT Kharagpur Data Analytics (CS61061) 68

Case Study 2: Step 2

Step 2: Collect the sample data and calculate the test statistics

Sample results: 𝑛 = 16, x = 7.89, 𝜎 = 0.2

With the sample, the test statistics is

4.-6%-
𝑧= ).& = −2.20
& (,

Hence, 𝑍 = 2.20

DSamanta@IIT Kharagpur Data Analytics (CS61061) 69

Case Study 2: Step 3
Step 3: To decide the critical region for specified 𝐇𝟎

Rejection region: Given 𝛼 = 0.05, which gives (P 𝑍 > 1.96) (obtained from standard
normal calculation for 𝑛 𝑍: 0,1 = 0.025 for a rejection region with two-tailed test).
Acceptance region
Accept H0 ,if the sample
mean falls in this region

95 % of area

0.025 of area 0.025 of area

−1.96 µH 0 1.96
Rejection region
Reject H0 ,if the sample mean falls
in either of these regions

DSamanta@IIT Kharagpur Data Analytics (CS61061) 70

Case Study 2: Step 4

Step 4: Make a decision to either reject or fail to reject H0

-2.20 -1.96 0 1.96 2.20

Since 𝑍 > 1.96, we reject 𝐻)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 71

Case Study 2: Step 5
Step 5: Final comment and interpret the result

We conclude 𝜇 ≠ 8 and recommend that the machine be adjusted.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 72

 Comments with Case-2

DSamanta@IIT Kharagpur Data Analytics (CS61061) 73

Case Study 2: Comment 1 (𝛼 = 1%)
Suppose that in our initial setup of hypothesis test, if we choose 𝛼 = 0.01 instead of 0.05,
then the test can be summarized as:

1. 𝐻) : 𝜇 = 8 , 𝐻! : 𝜇 ≠ 8 𝛼 = 0.01

2. Reject 𝐻) if 𝑍 > 2.576

4.-6%-
3. Sample result n =16, 𝜎 = 0.2, 𝑋=7.89, Z = ).& = −2.20, 𝑍 = 2.20
& (,

4. 𝑍 < 2.20, we fail to reject 𝐻) = 8

5. We do not recommend that the machine be adjusted.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 74

 # = 7.91)
Case Study 2: Comment 1 (𝑿
Suppose that in our initial setup of hypothesis test, we choose 𝛼 = 0.05, and the collected
sample is 𝑋=7.91 of size 16 with 𝜎 = 0.2. In this case, the test can be summarized as:

1. 𝐻) : 𝜇 = 8 , 𝐻! : 𝜇 ≠ 8 𝛼 = 0.05

2. Reject 𝐻) if 𝑍 > 1.96

4.6!%-
3. Sample result n =16, 𝜎 = 0.2, 𝑋=7.91, Z = ).& = −1.80, 𝑍 = 1.80
& (,

4. 𝑍 < 1.96, we fail to reject 𝐻) = 8

5. We do not recommend that the machine be adjusted.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 75

Case Study 2: Observation
H0 is rejected with 𝑋n = 7.89
For = 0.05
H0 is not rejected with 𝑋n = 7.91

H0 is rejected with 𝛼 = 0.05

n = 7.89
For 𝑿
H0 is not rejected with 𝛼 = 0.01

What about with = 0.02, 0.03, … ?

DSamanta@IIT Kharagpur Data Analytics (CS61061) 76

 p Value Concept

DSamanta@IIT Kharagpur Data Analytics (CS61061) 77

p Value Concept
Issues
1. Many users do not have a fixed or definitive idea of what should
be an appropriate value for 𝛼 in hypothesis testing.
2. Using a specified level of significance, a decision differs even
for a minor change in sample statistics.
Need
— There should be a method of reporting the results of a hypothesis
without having to chose an exact value of level of significance.
— It can be left to the decision maker who will use the test result.

The method of reporting results is referred to as report the p-value of

a test.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 78

p Value: Definition
Definition
— p value is the probability of of committing Type I Error if the
actual sample value of the statistics is used as the boundary of the
rejection region.
— It is the smallest level of significance for which H0 is to be
rejected.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 79

p Value: Definition
Definition

𝑝 = 𝑃 𝑍 > 𝑧Z for one-tailed test

𝑝 = 2×𝑃 𝑍 > 𝑧Z for two-tailed test

DSamanta@IIT Kharagpur Data Analytics (CS61061) 80

p Value: Example
Example:
H0: 𝜇 = 8.0
H0: 𝜇 ≠ 8.0
n =16, 𝜎 = 0.2, 𝑋 = 7.89

Œ.•Ž3•.Z
𝑧Z= Z.!/ ••
= -2.20
𝑝 = 2×𝑃 𝑍 > 𝑧Z
= 2 × 0.0139
= 0.0278
≈ 3%

DSamanta@IIT Kharagpur Data Analytics (CS61061) 81

p Value: Interpretations
Manys interpretation:
— The example implies that the probability of Type-I error is ≈ 3%
with the considered sample.
— The null hypothesis is rejected with level of significance 0.0278
or higher.

— The inference of population mean 𝜇 = 8 is acceptable with ≈ 3%

error (or 97% test accuracy).

— Here, no need to specify significance level a priori.

— Reporting results with p-value is a better inform for decision

makers from data analyst.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 82

 "
Case Study 3: 𝜒 -Test

DSamanta@IIT Kharagpur Data Analytics (CS61061) 83

Case Study 3: Perceptual Psychology
In perceptual psychology, a person is asked to judge the relative areas of circles of
varying sizes. A person typically judges the areas on a perceptual scale that can be
approximated by
𝐽𝑢𝑑𝑔𝑒𝑑 𝑎𝑟𝑒𝑎 = 𝑎. 𝑇𝑟𝑢𝑒 𝑎𝑟𝑒𝑎 8
For most of the people, the exponent b is between 0.6 and 1. That is a person with an
exponent of 0.8, who sees two circles, one twice the area of the other, would judge the
larger one to be only 20.8 = 1.74. If the exponent is less than one then the person
underestimate the area; if larger than 1, he overestimate the area. Based on an experiment
with 24 people, a data on the perceptual psychology is observed.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 84

Case Study 3: Perceptual Psychology
Measured exponents in Perceptual Psychology
0.58 0.63 0.69 0.72 0.74 0.79
0.88 0.88 0.90 0.91 0.93 0.94
0.97 0.97 0.99 0.99 0.99 1.00
1.03 1.04 1.05 1.07 1.18 1.27

11
10
9
8 Suppose, in the study, that variability of subjects
8
7
is of concern. Researchers want to know
6 whether the variance of exponents differ from
5
0.02.
4
3
2 Consider the acceptable level of confidence is
1 5%.
0.5 0.7 0.9 1.1 1.3

DSamanta@IIT Kharagpur Data Analytics (CS61061) 85

Case Study 2: Step 1

Step 1: Specification of hypothesis and significance level 𝛂

The hypotheses of interest is given by

The null hypothesis is

𝐻! : 𝜎 ) = 0.02

The alternative hypothesis is

𝐻" : 𝜎 ) ¹ 0.02

We assume 𝛼, the significance level in our hypothesis testing ≈ 0.05.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 86

Case Study 2: Step 2
Step 2: Decide the rejection region for specified 𝐡𝐲𝐩𝐨𝐭𝐡𝐞𝐬𝐢𝐬 𝐭𝐞𝐬𝐭𝐢𝐧𝐠

Rejection region: Given 𝛼 = 0.05, and with degree of freedom = 24-1 = 23, the 𝜒 # value
of the critical region is 38.08.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 87

Case Study 2: Step 3

Step 3: Collect the sample data and calculate the test statistics

Sample results: 𝑛 = 24, 𝑆𝑆 = 0.628, 𝜎 # = 0.02

With the sample, the test statistics is

99 ).0#-
𝜒# = & = = 31.4
" ).)#

DSamanta@IIT Kharagpur Data Analytics (CS61061) 88

Case Study 2: Step 4

Step 4: Make a decision to either reject or fail to reject H0

Since,c# = 31.40 < 38.08, we cannot reject the null hypothesis.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 89

Case Study 2: Step 5
Step 5: Final comment and interpret the result

We conclude that the sample variance does not significantly differ from 0.02.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 90

Important notes on Hypothesis Testing

DSamanta@IIT Kharagpur Data Analytics (CS61061) 91

Considerations in Hypothesis Testing
Hypothesis testing is sensitive to…

1. Acceptable level of signi{icance, 𝛼 = 10%, 5%, 3%, 2%, 1%, etc.

2. z-test, t-test of c) -test?

3. Selection of a sample and hence observed values of 𝑋 and S.

4. Size of the sample and repetition of test with different samples
5. Reporting results with p values without specification of level of signi{icance.

6. Type-I Error of Type-II Error in testing?

7. ?????.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 92

 Calculation of Errors in SL

DSamanta@IIT Kharagpur Data Analytics (CS61061) 93

Example 5.7: Type-I Error of Calculating 𝛼

Consider the two hypotheses are

The null hypothesis is

𝐻! : 𝜇 = 8

The alternative hypothesis is

𝐻" : 𝜇 ≠ 8

Assume that given a sample of size 16 and standard deviation is 0.2 and sample
follows normal distribution.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 94

Example 5.7: Calculating 𝛼
We can decide the rejection region as follows.

Suppose, the null hypothesis is to be rejected if the mean value is less than 7.9 or greater
than 8.1. If X is the sample mean, then the probability of Type I error is

𝛼 = 𝑃(X < 7.9 𝑜𝑟 X > 8.1, when 𝜇 = 8)

Given 𝜎, the standard deviation of the sample is 0.2 and that the distribution follows normal
distribution.
Thus,
7.9 − 8
𝑃 X < 7.9 = 𝑃 𝑍 = = 𝑃 𝑍 < −2.0 = 0.0228
0.2N
16
and
8.1 − 8
𝑃 X > 8.1 = 𝑃 𝑍 = = 𝑃 𝑍 > 2.0 = 0.0228
0.2N
16

Hence, 𝛼 = 0.0228 + 0.0228 = 0.0456

DSamanta@IIT Kharagpur Data Analytics (CS61061) 95

Example 5.8: Calculating 𝛼 and 𝜷
There are two identically appearing boxes of chocolates. Box A contains 60 red and
40 black chocolates whereas box B contains 40 red and 60 black chocolates. There
is no label on the either box. One box is placed on the table. We are to test the
hypothesis that “Box B is on the table”.

To test the hypothesis an experiment is planned, which is as follows:

— Draw at random five chocolates from the box.
— We replace each chocolates before selecting a new one.
— The number of red chocolates in an experiment is considered as the sample
statistics.

Note: Since each draw is independent to each other, we can assume the sample distribution
follows binomial probability distribution.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 96

 Example 5.8: Calculating 𝛼
Let us express the population parameter as 𝑝 = the number of red chocolates in Box 𝐵.
The hypotheses of the problem can be stated as:
𝐻) : 𝑝 = 0.4 // Box B is on the table
𝐻! : 𝑝 = 0.6 // Box A is on the table
Calculating 𝜶:
In this example, the null hypothesis (𝐻) ) specifies that the probability of drawing a red
chocolate is 0.4 . This means that, lower proportion of red chocolates in observations
(𝑖. 𝑒. , 𝑠𝑎𝑚𝑝𝑙𝑒) favors the null hypothesis. In other words, drawing all red chocolates provides
sufficient evidence to reject the null hypothesis. Then, the probability of making a 𝑇𝑦𝑝𝑒 𝐼 error
is the probability of getting five red chocolates in a sample of five from Box B. That is,
𝛼=𝑃 𝑋=5 𝑤ℎ𝑒𝑛 𝑝 = 0.4
Using the binomial distribution
(! ;
= ;! (%; !
𝑝 (1 − 𝑝)(%; 𝑤ℎ𝑒𝑟𝑒 𝑛 = 5, 𝑥 = 5
= (0.4)3 = 0.01024
Thus, the probability of rejecting a true null hypothesis is ≈ 0.01. That is, there is
approximately 1 in 100 chance that the box B will be mislabeled as box A.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 97

 Example 5.8: Calculating 𝜷
The 𝑇𝑦𝑝𝑒 𝐼𝐼 error occurs if we fail to reject the null hypothesis when it is not true. For the
current illustration, such a situation occurs, if Box A is on the table but we did not get the five
red chocolates required to reject the hypothesis that Box B is on the table.
The probability of 𝑇𝑦𝑝𝑒 𝐼𝐼 error is then the probability of getting four or fewer red chocolates
in a sample of five from Box A.
That is,
𝛽=𝑃 𝑋≤4 when 𝑝 = 0.6
Using the probability rule:
𝑃 𝑋 ≤ 4 + 𝑃(𝑋 = 5) = 1
That is, 𝑃 𝑋 ≤ 4 = 1 − 𝑃(𝑋 = 5)
Now, 𝑃(𝑋 = 5) = (0.6)3
Hence, 𝛽 = 1 − (0.6)3
= 1 − 0.07776 = 0.92224

That is, the probability of making 𝑇𝑦𝑝𝑒 𝐼𝐼 error is over 92%. This means that, if Box A is on
the table, the probability that we will be unable to detect it is 0.92.
DSamanta@IIT Kharagpur Data Analytics (CS61061) 98
 Estimation with Confidence Interval

DSamanta@IIT Kharagpur Data Analytics (CS61061) 99

 Estimation
— The hypothesis testing makes a statement about the value of a
population parameter. [Subjective estimation]

— Instead it may be more interesting to know the value of a population

statistics (e.g., mean score in a quiz rather than if maen = 50 true or
false). [Quantitative estimation]

— Such a quantitative estimation in statistical learning is called

estimation (of a population parameter).

DSamanta@IIT Kharagpur Data Analytics (CS61061) 100

 Estimation
— There are two types of estimations:
— Single point estimate
— For example, sample mean is a single point estimate.
— This may vary from one sample to another.
— This is called zero probability of being correct.
— Not robust and reliable.

— Interval estimated
— Estimate with a range of values, for example, population mean is 20 ≤ 𝜇 ≤
22
— Reliable and robust with essentially non-zero probability of being correct.
— An alternative method to statistical learning.
— Popularly known as Confident Interval measurement.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 101

 Procedure Confidence Interval Measurement
0
1&2 7
𝑧= ⟹ 𝑋F - 𝜇 = 𝑧 ‚
7/ 4 4
7
⟹ 𝑋F − 𝑧 ‚ =𝜇
4
This implies that
7
𝑃 𝑍 > 𝑧! = 𝑃 𝑋F − 𝑧 ‚
4
Similarly,
7
𝑃 𝑍 < 𝑧! = 𝑃 𝑋F + 𝑧 ‚
4
Thus,
7 7
𝑃 𝑋F − 𝑧 ‚ < 𝜇 < 𝑃 𝑋F + 𝑧 ‚ with probability 1-𝛼
4 4
Therefore, the interval estimate of 𝜇 is
7 7
𝑋F − 𝑧8/) ‚ to 𝑋F + 𝑧8/) ‚
4 4

DSamanta@IIT Kharagpur Data Analytics (CS61061) 102

 Example: Confidence Interval Measurement
Suppose, a hypothesis testing for a population mean 𝜇 = 8.0 is as below.
𝑋F = 7.89, n = 16, = 0.2, 𝜎 = 0.2 and 𝛼 = 0.05

For this testing, we have

0
1&2
𝑧= ⟹ 1.96
7/ 4
Thus,
𝑧8/) = 0.98
Hence,
Confidence interval is 7.89 ± 0.98
— This is the interval estimate with 95% confidence (i.e., accuracy)
— We are 95% confident that the true mean is between 6.91 to 8.87
7
— Here, the term E = 𝑧8/) ‚ is called maximum error (also called error margin)
4
— Alternatively,
CI estimate is 𝑋F ± 𝐸
DSamanta@IIT Kharagpur Data Analytics (CS61061) 103
 Discussion

DSamanta@IIT Kharagpur Data Analytics (CS61061) 104

Hypothesis Testing Strategies
— The hypothesis testing determines the validity of an assumption
(technically described as null hypothesis), with a view to choose
between two conflicting hypothesis about the value of a
population parameter.

— There are two types of tests of hypotheses

Ø Parametric tests (also called standard test of hypotheses).

Ø Non-parametric tests (also called distribution-free test of hypotheses).

DSamanta@IIT Kharagpur Data Analytics (CS61061) 105

Parametric Tests : Applications
— Usually assume certain properties of the population from
which we draw samples.
• Observation come from a normal population

• Sample size is small

• Population parameters like mean, variance, etc. are hold good.

• Requires measurement equivalent to interval scaled data.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 106

Hypothesis Testing : Assumptions
Case 1: Normal population, population infinite, sample size may be large or small,
variance of the population is known.
𝑋b − 𝜇*)
𝑧=
𝜎/ 𝑛
Case 2: Population normal, population finite, sample size may large or
small………variance is known.
𝑋b − 𝜇*)
𝑧=
𝜎/ 𝑛[ (𝑁 − 𝑛)/(𝑁 − 1)]
Case 3: Population normal, population infinite, sample size is small and variance of
the population is unknown.
< -
2%= )
𝑡= 𝑤𝑖𝑡ℎ 𝑑𝑒𝑔𝑟𝑒𝑒 𝑜𝑓 𝑓𝑟𝑒𝑒𝑑𝑜𝑚 = (𝑛 − 1)
9/ (

< &
∑(2* %2)
and 𝑠= ((%!)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 107

Hypothesis Testing
Case 4: Population is normal, finite, variance is known and sample
with small size
0 (
1&2 !
𝑡= 𝑤𝑖𝑡ℎ 𝑑𝑒𝑔𝑟𝑒𝑒 𝑜𝑓 𝑓𝑟𝑒𝑒𝑑𝑜𝑚 = (𝑛 − 1)
7/ 4[ (;&4)/(;&")]

Note:
If variance of population 𝜎 is known, replace 𝑆 by 𝜎.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 108

Hypothesis Testing : Non-Parametric Test

— Non-Parametric tests
v Does not under any assumption

v Suitable for nominal or ordinal data

v Need entire population (or very large sample size)

DSamanta@IIT Kharagpur Data Analytics (CS61061) 109

 Reference

— The detail material related to this lecture can be found in

Probability and Statistics for Engineers and Scientists (8th Ed.) by

Ronald E. Walpole, Sharon L. Myers, Keying Ye (Pearson), 2013.

DSamanta@IIT Kharagpur Data Analytics (CS61061) 110

 Any question?

DSamanta@IIT Kharagpur Data Analytics (CS61061) 111

 Questions of the day…
1. In a hypothesis testing, suppose H0 is rejected. Does it mean
that H1 is accepted? Justify your answer.

2. Give the expressions for z, t and 𝝌𝟐 in terms of population and

sample parameters, whichever is applicable to each. Signifies
these values in terms of the respective distributions.

3. How can you obtain the value say P(z = a)? What this values
signifies?

4. On what occasion, you should consider z-distribution but not t-

distribution and vice-versa?

5. Give a situation when you should consider 𝝌𝟐 distribution but

neither z- nor t-distribution.
DSamanta@IIT Kharagpur Data Analytics (CS61061) 112