PCC-CSD601 Machine Learning for

Real World Application

Lecture Notes
Inferential Statistics
Exploratory data analysis helped you understand how to discover patterns in data using various
techniques and approaches. As you've learnt, EDA is one of the most important parts of the data analysis
process. It is also the part on which data analysts spend most of their time.

However, sometimes, you may require a very large amount of data for your analysis which may need too
much time and resources to acquire. In such situations, you are forced to work with a smaller sample of
the data, instead of having the entire data to work with.

Situations like these arise all the time at big companies like Amazon. For example, say the Amazon QC
department wants to know what proportion of the products in its warehouses are defective. Instead of
going through all of its products (which would be a lot!), the Amazon QC team can just check a small
sample of 1,000 products and then find, for this sample, the defect rate (i.e. the proportion of defective
products). Then, based on this sample's defect rate, the team can "infer" what the defect rate is for all the
products in the warehouses.

This process of “inferring” insights from sample data is called “Inferential Statistics”.

Random Variables

Before performing any kind of statistical analysis on a problem, it is advisable to quantify its outcomes by
using random variables.

So, the random variable X basically converts outcomes of experiments to something measurable.

For example, recall that we quantified the colours of the balls we would get after playing our game by
assigning a value of X to each outcome. We did so by defining X as the number of red balls we would get
after playing the game once.

Figure 1 - Quantifying Using Random Variables

Probability Distribution
A probability distribution for X, basically, is ANY form of representation that tells us the probability for all
possible values of X. It could be a table, a chart or an equation.

A probability distribution looks like the frequency distribution, just with a different scale. For example,
here are the probability distribution and the frequency distribution (histogram) for our UpGrad red ball
game –

Figure 2 – Frequency Distribution (Left) vs Probability Distribution (Right)

Expected Value
The expected value for a variable X is the value of X we would “expect” to get after performing the
experiment once. It is also called the expectation, average, and mean value. Mathematically speaking, for a
random variable X that can take values x1, x2, x3, …… xn, the expected value (EV) is given by:

EV(X) = x1*P(X = x1) + x2*P(X = x2) + x3*P(X = x3) + …………………. + xn*P(X = xn)

Where, P(X=xi) denotes the probability that the random variable will take the value xi.

For example, suppose you’re trying to find the expected value of the number of red balls in our UpGrad
game. The random variable X, which is the number of red balls the player gets after playing the game once,
can take values 0, 1, 2, 3 and 4. So, the expected value for the number of red balls would be –

EV(X) = 0*P(X = 0) + 1*P(X = 1) + 2*P(X = 2) + 3*P(X = 3) + 4*P(X = 4)

EV(X) = 0*(0.027) + 1*(0.160) + 2*(0.347) + 3*(0.333) + 4*(0.133) = 2.385
However, you can never get 2.385 red balls in one game, as the number of balls will be an integer, like 2 or
3. However, the expected value, the value you would “expect” to get after one experiment, does not have
to be a value that will turn up in the experiment/game.

So, the expected value, actually, is the average value of X that you will get after playing the game an
infinite number of times.

Probability Without Experiment

Using basic rules of probability, i.e., addition rule and multiplication rule, you saw how you could find the
probabilities for our UpGrad red ball game, without even playing the game once.

The probability distribution thus achieved (theoretical probability distribution), was very similar to the
distribution achieved earlier via experiment (observed probability distribution).

Figure 3 – Observed Probability Distribution (Left) vs Theoretical Probability Distribution (Right)

Notice that the values of P(X = 0) are very close in both graphs, as are the values of P(X = 1), P(X = 2), P(X =
3) and P(X = 4). If the number of experiments conducted would have been more than 75, the values would
have been even closer. In fact, for an infinite number of experiments, the values will be exactly same for
both graphs.

Binomial Distribution
The binomial distribution can be used to calculate the probability of an event, if it follows the following
conditions –

1. The total number of trials is fixed

2. Each trial is binary, i.e. has only two possible outcomes, success and failure
 3. The probability of success is the same for all the trials

Basically, it should be a series of yes or no questions, with the probability of yes remaining same for all
questions. Examples of such situations are –

1. Finding the probability of 5 out of the next 10 cars having an even numbered license plate
2. Finding the probability of 3 of the next 4 balls picked out from the bag, being red (UpGrad game
{balls are put back after drawing})
3. Finding the probability of 9 out of the next 20 coin tosses resulting in a heads.

For such a situation, the probability of r successes, is given by –

P(X = r) = 𝑛𝐶𝑟 (𝑝)𝑟 (1 − 𝑝)𝑛−𝑟

Where,
n is the total number of trials/questions
p is the probability of success in 1 trial
r is the number of successes after n trials

For example, in our UpGrad game –

Total number of trials, n = 4
Probability of getting a red ball in 1 trial, p = 0.6

So, the probability of getting r red balls is given by –

P(X = r) = 4𝐶𝑟 (0.6)𝑟 (0.4)4−𝑟

Using this, we get P(X = 0) = 4𝐶0 (0.6)0 (0.4)4 = 0.0256. Also, P(X = 1) = 4𝐶1 (0.6)1 (0.4)3 = 0.1536. Similarly,
we can find P(X = 2), P(X = 3) and P(X = 4).

Cumulative Probability
Cumulative probability of x, generally denoted by F(x), is the probability of the random variable X, taking a
value lesser than x. Mathematically speaking, we’d say –

F(x) = P(X < x)

For example, for our UpGrad game,

F(2) = P(X < 2) = P(X = 0) + P(X = 1) + P(X = 2) = 0.0256 + 0.1536 + 0.3456 = 0.5238.

Probability Density Functions

For a continuous random variable, the probability of getting an exact value is very low, almost zero.
Hence, when talking about the probability of continuous random variables, you can only talk in terms of
intervals.

For example, for a particular company, the probability of an employee’s commute time being exactly equal
to 35 minutes was zero, but the probability of an employee having a commute time between 35 and 40
minutes was 0.2.

Hence, for continuous random variables, probability density functions (PDFs) and cumulative distribution
functions (CDFs) are used, instead of the bar chart type of distribution used for the probability of discrete
random variables. These functions are preferred because they talk about probability in terms of intervals.

Figure 4 – PDFs vs. CDFs (X = commute time)

To find the cumulative probability using a CDF, you just have to check the value of the graph. For example,
F(28), i.e., the probability of an employee having a commute time less than or equal to 28 minutes, is given
by the value of the CDF at X = 28. In the PDF, it is given by the area under the graph, between X = 20, the
lowest value and X = 28.

Normal Distribution
A very commonly used probability density function is the normal distribution. It is a symmetric
distribution, and its mean, median and mode lie at the center.
 Figure 5 – Normal Distribution

Also, a variable that is normally distributed, follows the 1-2-3 rule, which states that there is a –

1. 68% probability of the variable lying within 1 standard deviation of the mean
2. 95% probability of the variable lying within 2 standard deviations of the mean
3. 99.7% probability of the variable lying within 3 standard deviations of the mean

Figure 6 – 1-2-3 Rule for Normal Distribution

Standard Normal Distribution

In order to find the probability for a normal variable, you actually do not need to know the value of the
mean or the standard deviation — it is enough to know the number of standard deviations away from the
mean your random variable is. That is given by:
𝑋−𝜇
Z=
𝜎

This is called the Z score, or the standard normal variable.

In fact, you can use the Z table to find the cumulative probability for various values of Z. For example, say,
you want to find the cumulative probability for Z = 0.68 using the Z table.
 Figure 7 – Z Table

The intersection of row “0.6” and column “0.08” is 0.7517, which is our answer.
Samples
Instead of finding the mean and standard deviation for the entire population, it is sometimes beneficial to
find the mean and standard deviation for only a small representative sample. You may have to do this
because of time and/or money constraints.
For example, for an office of 30,000 employees, we wanted to find the average commute time. So, instead
of asking all employees, we asked only 100 of them and found that for them, the mean was equal to 36.6
minutes and the standard deviation was equal to 10 minutes.
However, we said that it would not be fair to infer that the population mean is exactly equal to the sample
mean. This is because the flaws of the sampling process must have led to some error. Hence, the sample
mean’s value has to be reported with some error margin.
For example, the mean commute time for the office of 30,000 employees would be equal to 36.6 + 3
minutes, 36.6 + 1 minutes or 36.6 + 10 minutes or, for that matter, 36.6 minutes + some error margin
However, in order to find this margin, it would be necessary to understand what sampling distributions
are, as there properties help in finding this margin.

Sampling Distributions & Central Limit Theorem

The sampling distribution, which is basically the distribution of sample means of a population, has some
interesting properties which are collectively called the central limit theorem, which states that no matter
how the original population is distributed, the sampling distribution will follow these three properties –
1. Sampling Distribution’s Mean (𝜇𝑋̅ ) = Population Mean (𝜇)
𝜎
2. Sampling Distribution’s Standard Deviation (Standard Error) = , where σ is the population’s
√𝑛
standard deviation and n is the sample size
3. For n > 30, the sampling distribution becomes a normal distribution
To verify these properties, we performed sampling using data collected for our UpGrad game from Session
1. The values for the sampling distribution thus created (𝜇𝑋̅ = 2.348, S.E. = 0.4248) were pretty close to the
values predicted by theory (𝜇𝑋̅ = 2.385, S.E. = 0.44).
To summarise, the notation and formulae related to samples, populations and sampling distributions are –

Mean Estimation Using CLT

Using CLT, you can estimate the population mean from the sample mean and standard deviation.
For example, to estimate the mean commute time of 30,000 employees of an office, you took a sample of
100 employees and found their mean commute time. For this sample, the sample mean 𝑋̅ = 36.6 minutes,
sample standard deviation S = 10 minutes.
Using CLT, you can say that the sampling distribution for mean commute time will have -
1. Mean = μ {unknown}
𝜎 𝑆 10
2. Standard error = 𝑛 = = =1
√ √𝑛 √100
3. Since n(100) > 30, the sampling distribution is a normal distribution
Using these properties, you can claim that the probability that the population mean μ lies between 34.6
(36.6-2) mins and 38.6 (36.6+2) mins, is 95.4%.
Also, there is some terminology related to the claim -
1. Probability associated with the claim is called confidence level (Here it is 95.4%)
2. Maximum error made in sample mean is called margin of error (Here it is 2 minutes)
3. Final interval of values is called confidence interval {Here it is the range – (34.6, 38.6)}
In fact, you can generalise the entire process. Let’s say you have a sample with sample size n, mean 𝑋̅ and
standard deviation S. Now, the y% confidence interval (i.e., confidence interval corresponding to y%
confidence level) for μ will be given by the range –
∗
𝑍 𝑆 𝑍 𝑆∗
Confidence Interval = (𝑋̅ − 𝑛 , 𝑋̅ + 𝑛 )
√ √

Where, Z* is the Z-score associated with a y% confidence level.

For example, the 90% confidence interval for the mean commute time will be –
∗
𝑍 𝑆 𝑍 𝑆 ∗
μ = (𝑋̅ − 𝑛 , 𝑋̅ + 𝑛 )
√ √

Here,
𝑋̅ = 36.6 minutes
S = 10 minutes
n = 100
Z* = 1.65 (Z* corresponding to 90% confidence level)
So, the confidence interval is –
μ = (34.95 mins, 38.25 mins)
PCC-CSD601 Machine Learning for
Real World Application
PCC-CSD601

Hypothesis Testing – Summary

Concepts in Hypothesis Testing I

The statistical analyses learnt in Inferential Statistics enable you try to make inferences about population
mean and other population data from the sample data. However, you could not confirm the conclusions you
made about the population about the data. It is here that hypothesis testing comes into the picture.

Understanding Hypothesis Testing

What is a Hypothesis?
When we perform an analysis on a population sample — the analysis could be descriptive, inferential, or
exploratory in nature — we get certain information from which we can make claims about the entire
population. These are just the claims; we can’t be sure if they’re actually true. This kind of claim or
assumption is called a hypothesis.

Example: The average commute time of employees of a company to and fro office is 35 minutes

What is Hypothesis Testing?

There are ways to check if your hypothesis has any truth to it, and if the hypothesis is true then apply it to
the population parameters. This is called hypothesis testing. The goal is to determine whether there is
enough evidence to infer that the hypothesis about the population parameter is true. In hypothesis testing,
we confirm our assumptions about the population based on sample data.

Difference between Inferential Statistics & Hypothesis Testing

Inferential statistics is used to find the mean of a population parameter when you have no initial number
to start with. So, you start with the sampling activity and find out the sample mean. Then, you estimate the
population mean from the sample mean using the confidence interval.

Hypothesis testing is used to confirm your conclusion (or hypothesis) about the population mean (which
you know from EDA or your intuition). Through hypothesis testing, you can determine whether there is
enough evidence to conclude if the hypothesis about a population parameter is true or not.

Null & Alternate Hypotheses

Hypothesis Testing starts with the formulation of these two hypotheses:

• Null hypothesis (H₀): The status quo

• Alternate hypothesis (H₁): The challenge to the status quo

Example: Suppose a man has been charged with murder. In the criminal trial for this case, the jury has to
decide whether the defendant is innocent or guilty. Now, this can be turned into two hypotheses. You can
claim that the defendant is innocent, and you can claim that the defendant is not innocent, i.e. guilty.

Therefore, you have two opposing hypotheses about the defendant. These two opposing hypotheses are
called the null hypothesis and the alternate hypothesis.
• The null hypothesis is the prevailing belief about a population; it states that there is no change or no
difference in the situation. In our criminal trial example, the defendant was considered innocent. So,
the null hypothesis claims that he is innocent, just like he was before the murder charge. Null
Hypothesis is denoted by H0

• The alternate hypothesis, or research hypothesis as it is also called, is the claim that opposes the
null hypothesis. If you were the prosecutor in the trial, your claim would be that the defendant is
guilty, and you would try to prove this. So, the alternate hypothesis is an assumption that competes
with the null hypothesis. Alternate Hypothesis is denoted by H1

Outcome of Hypothesis Testing

Suppose, at the end of the trial, the jury reaches a decision. What would be the outcome in terms of
hypothesis testing?

If the defendant is found guilty, it means that the jury rejects the null hypothesis in favour of the alternate
hypothesis. The jury decides that there is enough evidence to support the alternate hypothesis, and to
conclude that the defendant is guilty.
On the other hand, if the jury acquits the defendant, it means that there is not enough evidence to support
the alternate hypothesis. Keep in mind that this does not mean that the defendant is innocent, it just means
that there is not enough evidence to conclude that he is guilty. In other words, we cannot accept the null
hypothesis; we can only fail to reject it.

Therefore, in hypothesis testing, if there is sufficient evidence to support the alternate hypothesis, you reject
the null hypothesis; and if there is not sufficient evidence to support the alternate hypothesis, you fail to
reject the null hypothesis. So, you should never say that you “accept” the null hypothesis.

You should never say that you “accept” the null hypothesis.
Formulating Null & Alternate Hypotheses
If your claim statement has words like “at least”, “at most”, “less than”, or “greater than”, you cannot
formulate the null hypothesis just from the claim statement (because it’s not necessary that the claim is
always about the status quo).
You can use the following rule to formulate the null and alternate hypotheses:

• The null hypothesis always has the following signs: = OR ≤ OR ≥

• The alternate hypothesis always has the following signs: ≠ OR > OR <

For example:

Situation 1: Flipkart claimed that its total valuation in December 2016 was at least $14 billion. Here, the
claim contains ≥ sign (i.e. the at least sign), so the null hypothesis is the original claim.

The hypothesis in this case can be formulated as:

Figure 1 - Hypotheses for Situation 1

Situation 2: Flipkart claimed that its total valuation in December 2016 was greater than $14 billion. Here,
the claim contains > sign (i.e. the ‘more than’ sign), so the null hypothesis is the complement of the
original claim.

The hypothesis in this case can be formulated as:

Figure 2 - Hypotheses for Situation 2

To summarize this, you cannot decide the status quo or formulate the null hypotheses from the claim
statement, you need to take care of signs in writing the null hypothesis. Null Hypothesis never contains ≠
or > or < signs. It always has to be formulated using ≠ or > or < signs.

Making a Decision
Once you have formulated the null and alternate hypotheses, the next most important step of hypothesis testing is
— making the decision to either reject or fail to reject the null hypothesis

Situation 1: If sample mean is greater than UCV or less than LCV, i.e. sample mean lies in the criticals region.

Figure 3 – Rejecting a Null Hypothesis

Situation 2: If sample mean is less than UCV or greater than LCV, i.e. sample mean lies in the acceptance region.

Figure 4 – Failing to reject a Null Hypothesis

The formulation of the null and alternate hypotheses determines the type of
them test and the position of the critical regions in the normal distribution.

You can tell the type of the test and the position of the critical region on the basis of the ‘sign’ in the
alternate hypothesis.

≠ in H₁ → Two-tailed test → Rejection region on both sides of distribution

< in H₁ → Lower-tailed test → Rejection region on left side of distribution

> in H₁ → Upper-tailed test → Rejection region on right side of distribution

Critical Value Method

After formulating the hypothesis, the steps you have to follow to make a decision using the critical value
method are as follows:

1. Calculate the value of Zċ from the given value of α (significance level). Take it a 5% if not specified in
the problem.
2. Calculate the critical values (UCV and LCV) from the value of Zċ.
3. Make the decision on the basis of the value of the sample mean x with respect to the critical values
(UCV AND LCV).

You can download the z-table from this link.

Critical Value Method – Complete Example

Assume that you are the owner of multiple AC stores. You want to know about the mean demand of AC units
per month per store during summer. Till now you have been ordering 350 AC units per store per month
based on the historic demand. But this time because of intense heat waves, you anticipate that the demand
might go up. So you want to check your assumption that the average units required in one month will be
different from 350 units per store.
In this case you are assuming that 350 units is the average number of units that are sold every month. When
you try visualising it, you use histogram and the mean comes out to be 350 approximately. This becomes the
mean / average of population. Following figure 5 shows the histogram for this.

Next you will define the null and the alternate hypothesis. You start with the null hypothesis, i.e. the
assumption about the status quo. So you assume that H0 is true, and this implies that your population mean
is still equal to 350. In this AC hypothesis problem, the assumption is that the average demand for AC units
per store in one month is 350 units. So your null hypothesis H0 states that the mean
demand of ACs is 350 units per store every month.

Figure 5 - Histogram showing the average number of ACs

Now that your null hypothesis is clear, the next step is to state the alternative or research hypothesis. H 1 is
the opposite of H0 — it challenges the status quo. This is the assumption that you try to prove, and you
should examine all evidence with respect to H1. In your AC sales problem, H1 is that the population mean µ
≠ 350.

You should always examine the evidences with respect to alternative hypothesis NOT with
respect to null hypothesis.

You know that the population standard deviation sigma (σ) is 90, i.e. the distribution obtained every year,
containing the sales numbers of every store, has a standard deviation of 90. This year after the sales are
over, you take a random sample of 25 stores and plot them. The mean sales turns out to be 370.16. This is
your evidence. You can clearly see that it differs from the assumed population mean of 350 units per store.
Following figure 6 shows the same.

You can see that it differs from the assumed population mean of 350 units per store. As you are working on
samples, you will compute the standard error. The standard error can be calculated as standard deviation /
sqrt(number of samples). You calculate the standard error, because you want to know is the value 370.16
has significant distance from mean 350, so that the null hypothesis can be rejected.
 Figure 6 - The mean of sample is 370.16

So sampling distribution of sample means can be drawn in the graph with the given information as:
 Figure 7 – Sampling distribution of sample means

Formulating Hypotheses:
The first step would be to formulate the hypotheses:
 Figure 8 – Formulating Hypotheses for AC sales problem

Making a Decision – Critical Value Method:

The first step of critical value method is to find Zc. To do that, you calculate the cumulative probability of
UCV from the value of α, which is further used to find the z-critical value (Zc) for UCV.
 Figure 9 – Formulating Hypotheses for AC sales problem
Then you find the z-score of cumulative probability of UCV (Zc in this case).
 Figure 10 – Find z-value for cumulative probability of UCV

Then you calculate the critical values (UCV and LCV) from the value of Zc.
 Figure 11 – Finding the UCV and LCV
As sample mean lies is less than UCV and greater than LCV, i.e. it lies in the acceptance region,
Decision: Fail to reject the null hypothesis

Hypothesis Testing – Summary

Concepts in Hypothesis Testing - II
There are various methods similar to the critical value method to statistically make
your decision about the hypothesis. One such method is the p-value method. This is an important method
and is used more frequently in the industry.

p-value Method
What is p-value?
A P-value measures the strength of evidence in support of a null hypothesis. Suppose the test statistic in a
hypothesis test is equal to K. The P-value is the probability of observing a test statistic as extreme as K,
assuming the null hypothesis is true. If the P-value is less than the significance level, we reject the null
hypothesis.

Figure 12 – Interpretation of p-value

After formulating the hypothesis, the steps you have to follow to make a decision using the p-value
method are as follows:

1. Calculate the value of z-score for the sample mean point on the distribution
 2. Calculate the p-value from the cumulative probability for the given z-
score using the z-table
3. Make a decision on the basis of the p-value (multiply it by 2 for a two-tailed test) with respect to
the given value of α (significance value).

To find the correct p-value from the z-score, first find the cumulative probability by simply looking at the
z-table, which gives you the area under the curve till that point.

Situation 1: The sample mean is on the right side of the distribution mean (the z-score is positive)

Example: z-score for sample point = + 3.02

Figure 13- z-table for positive z-scores

Cumulative probability of sample point = 0.9987

For one-tailed test → p = 1 - 0.9987 = 0.0013

For two-tailed test → p = 2 (1 - 0.9987) = 2 * 0.0013 = 0.0026

Situation 2: The sample mean is on the left side of the distribution mean (the z-score is negative)

Example: z-score for sample point = -3.02

Figure 14 - z-table for negative z-scores

Cumulative probability of sample point = 0.0013

For one-tailed test → p = 0.0013

For two-tailed test → p = 2 * 0.0013 = 0.0026
 p-value Method – Complete Example
Taking the same AC sales problem, hypotheses for the situation would remain the same.

Making a Decision
So you start by finding out the z-value for given sample mean.

Figure 15 – Finding z-value for sample mean

Then you find the z-value for the calculated z-value for sample mean.

Figure 16 – Finding cumulative probability for z-value

Then you find the p-value by the approach mentioned above.

 Figure 17 – Finding p-value from cumulative probability

As p-value (0.1802) is greater than the value of α (0.05),

Decision: Fail to reject the null hypothesis

Types of errors

There are two possible errors we can commit during hypothesis testing —
 • type I error
• type II error.

The type I error occurs when the null hypothesis is true but we reject it, i.e. reject H0 when it is true.
Example:
Just imagine, if the defendant is innocent of the murder, but is still convicted and given the death penalty, it
would be a gross miscarriage of justice. For a case like this, the type I error should have a 0.001 probability,
i.e. the jury should be convinced beyond reasonable doubt that the defendant is guilty, or an innocent man
might go to the gallows. On the other hand, for a civil trial, say, for damages in a car accident, the type I error
can have a larger margin like 0.49, i.e. upon a preponderance of the evidence.
The probability of type I error is denoted by alpha (α) and is usually 0.05 or 0.01, i.e. only a 5% or 1% chance.
The type I error is also called the level of significance of the hypothesis test.

The type II error occurs when the null hypothesis is false but we fail to reject it, i.e. fail to reject H0 when it
is false.
Example:
If the defendant is guilty, but the jury acquits him, it would be a type II error. In practical terms, this is the
most serious error you can make. If you let a murderer walk away, he might end up killing more people.
The probability of type II error is denoted by beta (β).

Figure 18 – Types of errors in Hypothesis Testing

 Hypothesis Testing – Summary
Industry Demonstration of Hypothesis Testing

T - Distribution

What is a T-distribution?
A T-distribution (or Student T distribution) is similar to the normal distribution in many cases; for example,
it is symmetrical about its central tendency. However, it is shorter than the normal distribution and has a
flatter tail, which would eventually mean that it has a larger standard deviation.
 Figure 19 - t-distribution vs standard normal distribution

At a sample size beyond 30, the t-distribution becomes approximately equal to the normal distribution.

Each t-distribution is distinguished by what statisticians call degrees of freedom, which are related to the
sample size of the data set. If your sample size is n, the degrees of freedom for the corresponding t-
distribution is n -1. For example, if your sample size is 10, you use a t-distribution with 10 -1 or 9 degrees of
freedom, denoted t9. Smaller sample sizes have flatter t-distributions than larger sample sizes. And as you
may expect, the larger the sample size is, and the larger the degree of freedom, the more the t-distribution
looks like a standard normal distribution or the Z-distribution.
When T-Distribution is used?

The most important use of the t-distribution is that you can approximate the value of the standard
deviation of the population (σ) from the sample standard deviation (s). However, as the sample size
increases more than 30, the t-value tends to be equal to the z-value. Thus, if you want to summarise the
decision-making in a flowchart given in the following figure 20, this is what you would get.

Figure 20 - Flowchart for deciding the test

If you look at how the method of making a decision changes if you are using the sample’s standard
deviation instead of the population’s. If you recall the critical value method, the first step is as follows:

1. Calculate the value of Zc from the given value of α (significance level). Take it as 5% if not specified
in the problem.

So, to find Zc, you would use the t-table instead of the z-table. The t-table contains values of Zc for a given
degree of freedom and value of α (significance level). Zc, in this case, can also be called as t-statistic
(critical).

Practically you would not need to refer to the z or the table when doing hypothesis testing in the industry.
Going forward when you need to do hypothesis testing in demonstrations of Excel or R, you would use the
term t-test since that is mostly performed in the industry. All calculations and results of a t-test are same
as the z-test whenever the sample size ≥ 30.

Two-sample Mean Test

Two-sample mean test - paired is used when your sample observations are from the same individual or
object. During this test, you are testing the same subject twice. For example, if you are testing a new drug,
you would need to compare the sample before and after the drug is taken to see if the results are
different.

Two-sample mean test - unpaired is used when your sample observations are independent. During this
test, you are not testing the same subject twice. For example, if you are testing a new drug, you would
compare its effectiveness to that of the standard available drug. So, you would take a sample of patients
who consumed the new drug and compare it with those who consumed the standard drug.

Two-sample Proportion Test

Two-sample proportion test is used when your sample observations are categorical, with two categories.
It could be True/False, 1/0, Yes/No, Male/Female, Success/Failure etc.

For example, if you are comparing the effectiveness of two drugs, you would define the desired outcome
of the drug as the success. So, you would take a sample of patients who consumed the new drug and
record the number of successes and compare it with successes in another sample who consumed the
standard drug.

A/B Testing
A/B testing is a direct industry application of the two-sample proportion test sample.

While developing an e-commerce website, there could be different opinions about the choices of various
elements, such as the shape of buttons, the text on the call-to-action buttons, the colour of various UI
elements, the copy on the website, or numerous other such things.

Often, the choice of these elements is very subjective and is difficult to predict which option would
perform better. To resolve such conflicts, you can use A/B testing. A/B testing provides a way for you to
test two different versions of the same element and see which one performs better.

A/B testing is entirely based on the two-sample proportion test, as the two-sample proportion test is used
when you want to compare the proportions of two different samples. You can use various tools to conduct
A/B testing (or two-sample proportion test) like R, Optimizely etc.