HYPOTHESIS
TESTING
Terms Used in Hypothesis Testing
Alpha Levels Usually we use α = .05. This corresponds to p =
.05.
The alpha level or the level of significance is a probability value
that is used to define the very unlikely sample outcomes if the null
hypothesis is true. In this case we would expect to obtain this
“outlier” sample in only 5% of the samples simply by chance. This
corresponds to p = .05. In other words, the probability of
obtaining this difference by chance is 5%.
Terms Used in Hypothesis Testing
Critical Region
The critical region is composed of extreme sample values that
are very unlikely to be obtained if the null hypothesis is true. The
boundaries for the critical region are determined by the alpha
level. If sample data fall in the critical region, the null hypothesis is
rejected.
Terms Used in Hypothesis Testing
What Is Meant by Significance?
A result is said to be significant or statistically significant if it is
very unlikely to occur when the null hypothesis is true. That is, the
result is sufficient to reject the null hypothesis.
What factors influence significance?
The size of the difference. The variability of the scores. The
number of scores in the sample. Is there a difference between
significance and meaningfulness?
What is Hypothesis Testing?
◦Hypothesis testing is a formal procedure for investigating our
ideas about the world using statistics. It is most often used by
scientists to test specific predictions, called hypotheses, that arise
from theories.
Fundamentals of Hypothesis Testing
Fundamentals of Hypothesis Testing
◦ It is clear that there can be two types of error in the judgment – Type 1
error, when the verdict is against the person while he was innocent, and
Type 2 error, when the verdict is in favor of the person while he was
guilty.
◦ According to the Presumption of Innocence, the person is considered
innocent until proven guilty. That means the judge must find the
evidence which convince him “beyond a reasonable doubt.” This
phenomenon of “Beyond a reasonable doubt” can be understood
as Probability (Judge Decided Guilty | Person is Innocent)
should be small.
Basic Concepts of Hypothesis Testing
◦We consider the Null Hypothesis to be true until we find strong
evidence against it. Then we accept the Alternate Hypothesis. We
also determine the Significance Level (⍺), which can be
understood as the probability of (Judge Decided Guilty | Person is
Innocent) in the previous example. Thus, if ⍺ is smaller, it will
require more evidence to reject the Null Hypothesis.
Basic Concepts of Hypothesis Testing
Δ is capital delta (lower case
is δ), a letter of the Greek
alphabet. It can be used in
equations just like normal
letters in a similar manner
to x. It is usually used to
represent difference.
Steps in Hypothesis Testing
There are 5 main steps in hypothesis testing:
1. State your research hypothesis as a null hypothesis and alternate
hypothesis (Ho) and (Ha or H1).
2. Collect data in a way designed to test the hypothesis.
3. Perform an appropriate statistical test.
4. Decide whether to reject or fail to reject your null hypothesis.
5. Present the findings in your results and discussion section.
Though the specific details might vary, the procedure you will use when
testing a hypothesis will always follow some version of these steps.
Step 1: State your null and alternate
hypothesis
After developing your initial research hypothesis (the prediction
that you want to investigate), it is important to restate it as a null
(Ho) and alternate (Ha) hypothesis so that you can test it
mathematically.
The alternate hypothesis is usually your initial hypothesis that
predicts a relationship between variables. The null hypothesis is a
prediction of no relationship between the variables you are
interested in.
Step 1: State your null and alternate
hypothesis
Hypothesis testing example:
You want to test whether there is a relationship between gender and
height. Based on your knowledge of human physiology, you
formulate a hypothesis that men are, on average, taller than women.
To test this hypothesis, you restate it as:
H0: Men are, on average, not taller than women.
Ha: Men are, on average, taller than women.
Step 2: Collect data
For a statistical test to be valid, it is important to
perform sampling and collect data in a way that is designed to test
your hypothesis. If your data are not representative, then you
cannot make statistical inferences about the population you are
interested in.
Step 2: Collect data
Hypothesis testing example
To test differences in average height between men and women,
your sample should have an equal proportion of men and women,
and cover a variety of socio-economic classes and any other control
variables that might influence average height.
You should also consider your scope (Nationwide? For one
country?) A potential data source in this case might be census data,
since it includes data from a variety of regions and social classes
and is available for many countries around the world.
Step 3: Perform a statistical test
There are a variety of statistical tests available, but they are all based
on the comparison of within-group variance (how spread out the
data is within a category) versus between-group variance (how
different the categories are from one another).
If the between-group variance is large enough that there is little or
no overlap between groups, then your statistical test will reflect that
by showing a low p-value. This means it is unlikely that the
differences between these groups came about by chance.
Step 3: Perform a statistical test
Alternatively, if there is high within-group variance and low
between-group variance, then your statistical test will reflect that
with a high p-value. This means it is likely that any difference you
measure between groups is due to chance.
Your choice of statistical test will be based on the type of
variables and the level of measurement of your collected data.
Step 3: Perform a statistical test
Hypothesis testing example
Based on the type of data you collected, you perform a one-tailed t-
test to test whether men are in fact taller than women. This test gives
you: a) an estimate of the difference in average height between the two
groups. b) a p-value showing how likely you are to see this difference if
the null hypothesis of no difference is true.
Your t-test shows an average height of 175.4 cm for men and an average
height of 161.7 cm for women, with an estimate of the true difference
ranging from 10.2 cm to infinity. The p-value is 0.002.
Step 3: Perform a Statistical Test
Hypothesis testing example
Based on the type of data you collected, you perform a one-tailed t-
test to test whether men are in fact taller than women. This test gives
you:
▪ an estimate of the difference in average height between the two
groups.
▪ a p-value showing how likely you are to see this difference if the null
hypothesis of no difference is true.
Your t-test shows an average height of 175.4 cm for men and an average
height of 161.7 cm for women, with an estimate of the true difference
ranging from 10.2 cm to infinity. The p-value is 0.002.
Step 4: Decide whether to reject or fail to
reject your null hypothesis
Based on the outcome of your statistical test, you will have to
decide whether to reject or fail to reject your null hypothesis.
In most cases you will use the p-value generated by your statistical
test to guide your decision. And in most cases, your
predetermined level of significance for rejecting the null hypothesis
will be 0.05 – that is, when there is a less than 5% chance that you
would see these results if the null hypothesis were true.
Step 4: Decide whether to reject or fail to
reject your null hypothesis
In some cases, researchers choose a more conservative level of
significance, such as 0.01 (1%). This minimizes the risk of
incorrectly rejecting the null hypothesis (Type I error).
Hypothesis testing example
In your analysis of the difference in average height between men
and women, you find that the p-value of 0.002 is below your cutoff
of 0.05, so you decide to reject your null hypothesis of no
difference.
Step 5: Present your findings
The results of hypothesis testing will be presented in the results
and discussion sections of your research
paper, dissertation or thesis.
In the results section you should give a brief summary of the data
and a summary of the results of your statistical test (for example,
the estimated difference between group means and associated p-
value). In the discussion, you can discuss whether your initial
hypothesis was supported by your results or not.
Step 5: Present your findings
In the formal language of hypothesis testing, we talk about
rejecting or failing to reject the null hypothesis. You will probably
be asked to do this in your statistics assignments.
Stating results in a statistics assignment
In our comparison of mean height between men and women we
found an average difference of 13.7 cm and a p-value of 0.002;
therefore, we can reject the null hypothesis that men are not taller
than women and conclude that there is likely a difference in height
between men and women.
Step 5: Present your findings
If your null hypothesis was rejected, this result is interpreted as
“supported the alternate hypothesis.”
Stating results in a research paper
We found a difference in average height between men and women
of 14.3cm, with a p-value of 0.002, consistent with our hypothesis
that there is a difference in height between men and women.
Step 5: Present your findings
These are superficial differences; you can see that they mean the same
thing.
You might notice that we don’t say that we reject or fail to reject the
alternate hypothesis. This is because hypothesis testing is not designed
to prove or disprove anything. It is only designed to test whether a pattern
we measure could have arisen spuriously, or by chance.
If we reject the null hypothesis based on our research (i.e., we find that it
is unlikely that the pattern arose by chance), then we can say our test lends
support to our hypothesis. But if the pattern does not pass our decision
rule, meaning that it could have arisen by chance, then we say the test
is inconsistent with our hypothesis.
Frequently asked questions about hypothesis
testing
What is hypothesis testing?
Hypothesis testing is a formal procedure for investigating our
ideas about the world using statistics. It is used by scientists to test
specific predictions, called hypotheses, by calculating how likely it is
that a pattern or relationship between variables could have arisen by
chance.
Frequently asked questions about hypothesis
testing
What is a hypothesis?
A hypothesis states your predictions about what your research will
find. It is a tentative answer to your research question that has not yet
been tested. For some research projects, you might have to write several
hypotheses that address different aspects of your research question.
A hypothesis is not just a guess — it should be based on existing
theories and knowledge. It also has to be testable, which means you can
support or refute it through scientific research methods (such as
experiments, observations and statistical analysis of data).
LET’S RECALL PREVIOUS
TERMINOLOGIES
Inferential Statistics
Inferential statistics is a way of making inferences about
populations based on samples. Given information about a subset
of examples, how do we draw conclusions about the full set
(including other specific examples in that full set)? Inferences based
on principles of evidence use sample statistics.
What is difference between inferential and
descriptive statistics?
Descriptive Statistics gives information about raw data regarding
its description or features. Inferential statistics, on the other hand,
draw inferences about the population by using data extracted from
the population.
What is difference between inferential and
descriptive statistics?
Descriptive Statistics gives information about raw data regarding
its description or features. Inferential statistics, on the other hand,
draw inferences about the population by using data extracted from
the population.
Reference
Bevans, R. (2022, December 07). Hypothesis Testing | A Step-by-Step
Guide with Easy Examples. Scribbr. Retrieved April 21, 2023, from
[Link]