Hypothesis Testing
Significant Differences
Hypothesis testing is designed to detect
significant differences: differences that did not
occur by random chance.
In the one sample case: we compare a random
sample (from a large group) to a population.
We compare a sample statistic to a population
parameter to see if there is a significant
difference.
Our Problem:
The education department at a university has been
accused of grade inflation so education majors
have much higher GPAs than students in general.
GPAs of all education majors should be compared
with the GPAs of all students.
There are 1000s of education majors, far too many to
interview.
How can this be investigated without interviewing all
education majors?
What we know:
The average GPA for
all students is 2.70.
This value is a
parameter.
To the right is the
statistical information
for a random sample
of education majors:
= 2.70
X =
3.00
s=
0.70
n=
117
Questions to ask:
Is there a difference between the parameter
(2.70) and the statistic (3.00)?
Could the observed difference have been
caused by random chance?
Is the difference real (significant)?
Two Possibilities:
1.
The sample mean (3.00) is the same as
the pop. mean (2.70).
The difference is trivial and caused by
random chance.
2.
The difference is real (significant).
Education majors are different from all
students.
1.
2.
The Null and Alternative
Null Hypothesis (H )
Hypotheses:
0
The difference is caused by random chance.
The H0 always states there is no significant difference. In
this case, we mean that there is no significant difference
between the population mean and the sample mean.
Alternative hypothesis (H1)
The difference is real.
(H1) always contradicts the H0.
One (and only one) of these explanations must be true.
Which one?
Test the Explanations
We always test the Null Hypothesis.
Assuming that the H0 is true:
What is the probability of getting the sample
mean (3.00) if the H0 is true and all education
majors really have a mean of 2.70? In other
words, the difference between the means is
due to random chance.
If the probability associated with this difference
is less than 0.05, reject the null hypothesis.
Test the Hypotheses
Use the .05 value as a guideline to identify differences
that would be rare or extremely unlikely if H0 is true.
This alpha value delineates the region of rejection.
Use the Z score formula for single samples
If the probability is less than .05, the calculated or
observed Z score will be beyond 1.96 (the critical
Z score).
Two-tailed Hypothesis Test:
When = .05, then .025 of the area is distributed on either
side of the curve in area (C )
The .95 in the middle section represents no significant
difference between the population and the sample mean.
The cut-off between the middle section and +/- .025 is
represented by a Z-value of +/- 1.96.
Testing Hypotheses:
Using The Five Step Model
1.
2.
3.
4.
5.
Make Assumptions and meet test
requirements.
State the null hypothesis.
Select the sampling distribution and
establish the critical region.
Compute the test statistic.
Make a decision and interpret results.
Step 1: Make Assumptions
and Meet Test Requirements
Random sampling
Level of Measurement is Interval-Ratio
Hypothesis testing assumes samples were selected using
random sampling.
In this case, the sample of 117 cases was randomly selected
from all education majors.
GPA is I-R so the mean is an appropriate statistic.
Sampling Distribution is normal in shape
This is a large sample (n100).
Step 2 State the Null
Hypothesis
H0: = 2.7 (in other words, H0:
= )
You can also state Ho: No difference between the sample
mean and the population parameter
(In other words, the sample mean of 3.0 really the same as
the population mean of 2.7 the difference is not real but
is due to chance.)
The sample of 117 comes from a population that has a
GPA of 2.7.
The difference between 2.7 and 3.0 is trivial and caused by
random chance.
Step 2 (cont.) State the Alternate
Hypothesis
H1: 2.7 (or, H0:
Or H1: There is a difference between the sample mean and
the population parameter
The sample of 117 comes a population that does not have
a GPA of 2.7. In reality, it comes from a different population.
The difference between 2.7 and 3.0 reflects an actual
difference between education majors and other students.
Note that we are testing whether the population the sample
comes from is from a different population or is the same as
the general student population.
Step 3 Select Sampling
Distribution and Establish the
Critical
Sampling Region
Distribution= Z
Alpha () = .05
is the indicator of rare events.
Any difference with a probability less than
is rare and will cause us to reject the H0.
Step 3 (cont.) Select Sampling
Distribution and Establish the
Critical
Region
Critical Region
begins at Z= 1.96
This is the critical Z score associated with
= .05, two-tailed test.
If the obtained Z score falls in the Critical
Region, or the region of rejection, then
we would reject the H0.
Step 4: Use Formula to Compute
the Test Statistic (Z for large
samples ( 100)
Z
N
When the Population is not
known,
use the following formula:
Z
s n 1
Test the Hypotheses
3.0 2.7
Z
4.62
.7
117 1
We can substitute the sample standard deviation
S for pop. s.d.) and correct for bias by
substituting N-1 in the denominator.
Substituting the values into the formula, we
calculate a Z score of 4.62.
Step 5 Make a Decision and
Interpret Results
The obtained Z score fell in the Critical Region, so we reject
the H0.
If the H0 were true, a sample outcome of 3.00 would be
unlikely.
Therefore, the H0 is false and must be rejected.
Education majors have a GPA that is significantly different
from the general student body (Z = 4.62, = .05).*
*Note: Always report significant statistics.
Looking at the curve:
(Area C = Critical Region when
=.05)
Summary:
The GPA of education majors is significantly different
from the GPA of the general student body.
In hypothesis testing, we try to identify statistically
significant differences that did not occur by random
chance.
In this example, the difference between the
parameter 2.70 and the statistic 3.00 was large and
unlikely (p < .05) to have occurred by random
chance.
Summary (cont.)
We rejected the H0 and concluded that the
difference was significant.
It is very likely that Education majors have
GPAs higher than the general student body
Rule of Thumb:
If the test statistic is in the Critical Region
( =.05, beyond 1.96):
Reject the H0. The difference is significant.
If the test statistic is not in the Critical Region
(at =.05, between +1.96 and -1.96):
Fail to reject the H0. The difference is not
significant.
Using the Students t
Distribution for Small Samples
(One
WhenSample
the sampleT-Test)
size is small
(approximately < 30) then the Students t
distribution should be used
The test statistic is known as t.
The curve of the t distribution is flatter than
that of the Z distribution but as the sample
size increases, the t-curve starts to resemble
the Z-curve
Degrees of Freedom
The curve of the t distribution varies with
sample size (the smaller the size, the flatter
the curve)
In using the t-table, we use degrees of
freedom based on the sample size.
For a one-sample test, df = n 1.
When looking at the table, find the t-value for
the appropriate df = n-1. This will be the
cutoff point for your critical region.
Formula for one sample t-test:
(Note that it is identical to z-test, but uses a
different distribution)
t
S
n 1
Example
A random sample of 26 sociology
graduates scored 458 on the GRE
advanced sociology test with a standard
deviation of 20. Is this significantly
different from the population average
( = 440)?
Solution (using five step
model)
Step 1: Make Assumptions and Meet Test
Requirements:
1. Random sample
2. Level of measurement is interval-ratio
3. The sample is small (<30)
Solution (cont.)
Step 2: State the null and alternate hypotheses.
H0: = 440 (or H0:
H1: 440 (H1:
= )
= )
Solution (cont.)
Step 3: Select Sampling Distribution and
Establish the Critical Region
1.
Small sample, I-R level, so use t
distribution.
Alpha () = .05
Degrees of Freedom = n-1 = 26-1 = 25
Critical t = 2.060
2.
3.
4.
Solution (cont.)
Step 4: Use Formula to Compute the Test Statistic
458 440
t
4.5
S
20
n 1
26 1
Looking at the curve for the t
distribution
Alpha () = .05
Step 5 Make a Decision and
Interpret Results
The obtained t score fell in the Critical Region, so
we reject the H0 (t (obtained) > t (critical)
If the H0 were true, a sample outcome of 458
would be unlikely.
Therefore, the H0 is false and must be rejected.
Sociology graduates have a GRE score that is
significantly different from the general student body
(t = 4.5, df = 25, = .05).
Testing Sample Proportions:
When your variable is at the nominal (or
ordinal) level the one sample z-test for
proportions should be used.
If the data are in % format, convert to a
proportion first.
The method is the same as the one sample
Z-test for means (see above)
Formula for Proportions:
Note: Ps is the sample proportion
Pu is the population proportion
Ps Pu
Pu (1 Pu ) / n
Example
In a recent statewide (or provincial)
election, 55% of voters rejected lotteries. A
random sample of 150 urban (or rural
communities) precincts showed that 49%
of voters rejected lotteries. Is the
difference significant?
Use the formula for proportions and 5 step
method to solve
Solution:
Step 1:
Random sample
L.O.M. is nominal
The sample is large
Step 2:
H0: Pu = .55 (convert % to proportion)
(Note you can also say H0: Ps = Pu )
H1: Pu .55 (H1: Ps Pu
Step 3:
The sample is large, use Z distribution
Alpha () = .05
Critical Z = 1.96
Solution (cont.)
Step 4
Ps Pu
.49 .55
1.48
Pu (1 Pu ) / n
.55(1 .55) / 150
Step 5
Z (obtained) < Z (critical)
Fail to reject Ho. There is no significant difference
between the state population and the precincts.
Main Considerations in
Hypothesis Testing:
Sample size
Use Z for large samples, t for small (<30)
There are two other choices to be made:
One-tailed or two-tailed test
Is there a difference? = 2-tailed test
Is the difference less than or greater than? = 1-tailed test
Alpha () level
.05, .01, or .001? (=.05 is most common)
Two-tailed vs. One-tailed
Tests
In a two-tailed test, the direction of the
difference is not predicted.
A two-tailed test splits the critical region
equally on both sides of the curve.
In a one-tailed test, the researcher predicts
the direction (i.e. greater or less than) of the
difference.
All of the critical region is placed on the side
of the curve in the direction of the prediction.
The Curve for Two- vs. One-tailed Tests at
= .05:
Two-tailed test:
is there a significant
difference?
One-tailed tests:
is the sample mean
greater than or Pu?
is the sample mean
less than or Pu?
Type I and Type II Errors
Type I, or alpha error:
Rejecting a true null hypothesis
Type II, or beta error:
Failing to reject a false null hypothesis
