Business
Intelligence
Hypothesis Testing
LECTURE 6
MS. USHNA TASLEEM
What is a Hypothesis?
A hypothesis is an assumption about a population parameter. It is a
tentative statement that proposes a possible relationship between two or
more variables.
A hypothesis can be:
1. Null hypothesis (H0) – a statement that there is no difference or no effect.
2. Alternative hypothesis (H1) – a claim about the population that is
contradictory to H0.
Hypothesis testing evaluates two mutually exclusive statements (H0 and
H1) to determine which statement is best supported by the sample data.
Why Hypothesis Testing is
.Important in Business
Hypothesis testing allows business analysts to make statistical inferences
about a business problem. It is an objective data-driven approach to:
Evaluate business metrics against a target value. For example – is the
current customer satisfaction score significantly lower than our target of
85%?
Compare business metrics across time periods or categories. For example –
has website conversion rate increased this month compared to last month?
Quantify the impact of business initiatives. For example – did the email
marketing campaign result in a significant increase in sales?
Benefits
Supports data-driven decision making with statistical
evidence.
Helps save costs by making decisions backed by data
insights.
Enables measurement of success for business initiatives like
marketing campaigns, new product launches etc.
Provides a structured framework for business metric analysis.
Reduces the influence of individual biases in decision making.
By incorporating hypothesis testing in data analysis,
businesses can make sound decisions that are supported by
statistical evidence.
Steps
1. State the Hypotheses:
Define two opposite statements – one is the "null hypothesis"
(H0) that assumes nothing has changed or there’s no difference.
The other is the "alternative hypothesis" (H1) that suggests there
is a change or a difference.
Example:H0: The average customer rating this month is the same
as last month.H1: The average customer rating this month is
lower than last month.
Steps
2. Choose the Significance Level:
Decide how much risk you're willing to take of making a mistake
by rejecting the null hypothesis when it's actually true. This is
called the significance level (α).
Common choices are 0.10, 0.05, or 0.01. A lower number means
you're being more cautious. For example, α = 0.05 means you
accept a 5% chance of being wrong.
Steps
3. Select the Sample and Collect Data:
Choose a group that represents the whole population well and collect
the relevant data for testing the hypotheses – like customer ratings for
this month and last month.
Steps
5. Analyze the Sample Data:
Use a statistical test (like t-tests or chi-square) to analyze the
data. This test gives you a number (the test statistic) that you
compare against a set threshold to see if the result is significant.
6. Make a Decision:
Compare the test statistic to the threshold. If the statistic is in the
"rejection region," you reject the null hypothesis in favor of the
alternative. If not, you don’t reject the null hypothesis, meaning
there's not enough evidence to prove a difference.
Types of Hypothesis Tests
1. Parametric Tests
These tests make assumptions about the shape or parameters of the
population distribution.
Some examples are:
• Z-test – Tests a population mean when population standard deviation is known.
• T-test – Tests a population mean when standard deviation is unknown.
• F-test – Compares variances from two normal populations.
• ANOVA – Compares means of two or more populations.
Parametric tests are more powerful as they make use of the distribution
characteristics. But the assumptions need to hold true for valid results.
Types of Hypothesis Tests
2. Non-parametric Tests
These tests make no assumptions about the exact distribution of the
population. They are based on either ranks or frequencies.
Some examples are:
• Chi-square test – Tests if two categorical variables are related.
• Mann-Whitney U test – Compares medians from two independent groups.
• Wilcoxon signed-rank test – Compares paired observations or repeated
measurements.
• Kruskal Wallis test – Compares medians from two or more groups.
Non-parametric tests are distribution-free but less powerful than parametric
tests. They can be used when assumptions of parametric tests are violated.
The choice of statistical test depends on the hypotheses, data type and other
factors.
One-tailed and Two-tailed
Hypothesis Tests
Hypothesis tests can be one-tailed or two-tailed:
• One-tailed test – When H1 specifies a direction. For example: H0: μ =
10 H1: μ > 10 (or μ < 10)
• Two-tailed test – When H1 simply states ≠, not a specific direction.
For example: H0: μ = 10 H1: μ ≠ 10
One-tailed tests have greater power to detect an effect in the specified
direction. But we need prior knowledge on the direction of effect for
using them.
Two-tailed tests do not assume any direction and are more
conservative. They are used when we have no clear prior expectation
on the directionality.
Two-Tailed vs. One-Tailed Test
When a hypothesis test is set up to show that the sample mean
would be only higher than the population mean, this is referred to
as a one-tailed test.
A formulation of this hypothesis would be, for example, that "the
returns on an investment fund would be at least x%." One-tailed
tests could also be set up to show that the sample mean could be
only less than the population mean.
The key difference from a two-tailed test is that in a two-tailed
test, the sample mean could be different from the population
mean by being either higher or lower than it.
Two-Tailed vs. One-Tailed Test
If the sample being tested falls into the one-sided critical area,
the alternative hypothesis will be accepted instead of the null
hypothesis. A one-tailed test is also known as a directional
hypothesis or directional test.
A two-tailed test, on the other hand, is designed to examine both
sides of a specified data range to test whether a sample is
greater than or less than the range of values.
Critical Regions
In hypothesis testing, critical region is represented by set of values, where null
hypothesis is rejected. So it is also know as region of rejection. It takes different
boundary values for different level of significance. Below info graphics shows the
region of rejection that is critical region and region of acceptance with respect to the
level of significance 1%.
Question 1
A Telecom service provider claims that individual customers pay
on an average 400 rs. per month with standard deviation of 25
rs. A random sample of 50 customers bills during a given month
is taken with a mean of 250 and standard deviation of 15. What
to say with respect to the claim made by the service provider?
Solution
First thing first, Note down what is given in the question:
H0 (Null Hypothesis) : μ = 400
H1 (Alternate Hypothesis): μ ≠ 400 (Not equal means either μ > 400 or μ < 400 Hence it
will be validated with two tailed test )
σ = 25 (Population Standard Deviation)
LoS (α) = 5% (Take 5% if not given in question)
n = 50 (Sample size)
x̄ = 250 (Sample mean)
s = 15 (sample Standard deviation)
n > = 30 hence will go with z-test
Solution
First thing first, Note down what is given in the question:
H0 (Null Hypothesis) : μ = 400
H1 (Alternate Hypothesis): μ ≠ 400 (Not equal means either μ > 400 or μ < 400 Hence it
will be validated with two tailed test )
σ = 25 (Population Standard Deviation)
LoS (α) = 5% (Take 5% if not given in question)
n = 50 (Sample size)
x̄ = 250 (Sample mean)
s = 15 (sample Standard deviation)
n > = 30 hence will go with z-test
Step 1:
Calculate z using z-test formula as below:
z = (x̄ - μ)/ (σ/√n)
z = (250 - 400) / (25/√50)
z = -42.42
Step 2:
get z critical value from z table for α = 5%
z critical values = (-1.96, +1.96)
to accept the claim (significantly), calculated z should be in between
-1.96 < z < +1.96
but calculated z (-42.42) < -1.96 which mean reject the null
hypothesis
Interpreting Hypothesis Test
Results
Hypothesis testing results can be interpreted based on:
• p-value – Probability of obtaining sample results if H0 is true. Small p-
value (< α) indicates significant evidence against H0.
• Confidence intervals – Range of likely values for the population
parameter. If it does not contain the H0 value, we reject H0.
• Test statistic – Standardized value computed from sample data.
Compared against critical values to determine statistical significance.
• Effect size – Quantifies the magnitude or size of effect. Important for
interpreting practical significance.
Hypothesis testing indicates whether an effect exists or not. Measures like
effect size and confidence intervals provide additional insights on the
observed effect.
Common Errors in Hypothesis
Testing
Some common errors to watch out for:
• Having unclear, ambiguous hypotheses.
• Choosing an inappropriate significance level α.
• Using the wrong statistical test for data analysis.
• Interpreting a non-significant result as proof of no effect. Absence of evidence is not
evidence of absence.
• Concluding practical significance from statistical significance. Small p-values don’t
always imply practical business impact.
• Multiple testing without adjustment leading to elevated Type I errors.
• Stopping data collection prematurely when a significant result is obtained.
• Overlooking effect sizes, confidence intervals while focusing solely on p-values.
Proper application of hypothesis testing methodology minimizes such errors and
improves decision making.
Real-world Example of
Hypothesis Testing
A retailer wants to test if launching a new ecommerce website has resulted in increased
online sales.
The retailer gathers weekly sales data before and after the website launch:
H0: Launching the new website did not increase the average weekly online
sales
H1: Launching the new website increased the average weekly online sales
Significance level is chosen as 0.05. Appropriate parametric / non-parametric test is
selected based on data. Test results show that the p-value is 0.01, which is less than
0.05.
Therefore, we reject the null hypothesis and conclude that the new website launch has
resulted in significantly increased online sales at the 5% significance level.
The analyst also computes a 95% confidence interval for the difference in sales before
and after website launch. The retailer uses these insights to make data-backed
Summary
Hypothesis testing provides a formal process for making statistical
decisions using sample data. It helps assess business metrics against
benchmarks, quantify impact of initiatives and compare performance
across time periods or segments. By embedding hypothesis testing in
analytics, businesses can derive actionable insights for data-driven
decision making.