Sampling Distributions
and Hypothesis Testing
Cal State Northridge
Ψ320
Ainsworth
Major Points
Sampling distribution – What are they?
Hypothesis testing
The null hypothesis
Test statistics and their distributions
The normal distribution and testing
Some other Important concepts
Psy 320 - Cal State Northridge 2
Hypothetical Study on
Intelligence
Can we create a pill that when taken
regularly (like a vitamin) increases
intelligence?
A group of 25 participants are given
30mg of IQPLUS everyday for ten days
At the end of 10 days the 25
participants are given the Stanford-
Binet intelligence test.
Psy 320 - Cal State Northridge 3
1
IQPLUS: Results
The mean IQ score of the 25
participants is 106 (the average score
for IQ in the population is 100)
Is this increase large enough to
conclude that IQPLUS was affective in
increasing the participants IQ?
Psy 320 - Cal State Northridge 4
What is the REAL Question?
Is the difference between 106 and 100
large enough to lead us to conclude that
it is a real difference?
Would we expect a similar kind of
difference with a repeat of this experiment?
Or...
Is the difference due to something called
“sampling error”?
Psy 320 - Cal State Northridge 5
Sampling Error
AKA sampling variation
The normal variability that we would
expect to find from one sample to
another, or one study to another
Random variability among observations
or statistics that is just due to chance
Psy 320 - Cal State Northridge 6
2
How Could we Assess
Sampling Error?
Take many groups of 25 participants
who did not get IQPLUS.
Record the average IQ score for each
group.
Plot the distribution and record its
mean and standard deviation.
This distribution is called a “Sampling
Distribution.”
Psy 320 - Cal State Northridge 7
Sampling Distribution
The distribution of a statistic over
repeated sampling from a specified
population.
Possible result for this example.
See next slide.
Shows the kinds of means we expect to
find when people don’t take IQPLUS.
Psy 320 - Cal State Northridge 8
Population Distribution
Psy 320 - Cal State Northridge 9
3
Population Distribution
Psy 320 - Cal State Northridge 10
What Do We Conclude?
When people don’t get IQPLUS the
majority of sample means fall between
96 and 104
Our IQPLUS group had an average IQ
score of 106
Our subjects’ responses were not like
normal.
Conclude that IQPLUS increased
intelligence Psy 320 - Cal State Northridge 11
Sampling Distribution
Central Tendency
“typical value”
Usually estimates the population parameter
The mean is the mean of the means
Dispersion - Standard Error
“variability”
The SD of a sampling distribution is called
the Standard Error (SE)
Shape - depends upon the statistic and
the assumptions 12
4
Hypothesis Testing
A formal way of doing what we just did
Start with hypothesis that subjects are
normal.
The null hypothesis
Find what normal subjects do.
Compare our subjects to that standard.
Psy 320 - Cal State Northridge 13
Steps in Hypothesis Testing
Define the null hypothesis.
Decide what you would expect to find if
the null hypothesis were true.
Look at what you actually found.
Reject the null if what you found is not
what you expected.
Psy 320 - Cal State Northridge 14
Hypothesis Testing Steps
1. State Null Hypothesis (H0)
2. Alternative Hypothesis (H1)
3. Decide on α (usually .05)
4. Decide on type of test
5. Find critical value & state decision rule
6. Calculate test
7. Apply decision rule
Psy 320 - Cal State Northridge 15
5
H0 and H1
Always stated in terms of population
parameters, never sample statistics.
Together, define the entire range of
possibilities.
H0: µ = 0
H1: µ ≠ 0
Psy 320 - Cal State Northridge 16
The Null Hypothesis
The hypothesis that our subjects came
from a population of normal responders.
The hypothesis that taking IQPLUS has
no affect on intelligence.
The hypothesis we usually want to
reject.
Psy 320 - Cal State Northridge 17
The Null Hypothesis
Specifies a single value (has the equals
sign in it somewhere)
Says:
“Nothing happened.”
“There’s no difference.”
“Chance is a good explanation for your
results.”
“Your sample statistic doesn’t differ from
your hypothesized population parameter.” 18
6
The Research Hypothesis
Also known as the “alternative hypothesis”
Always specifies a range of values (has an
inequality in it somewhere)
Says,
“Something happened.”
“There is a difference.”
“Chance is not a good explanation for your results.”
“Your sample statistic differs ‘significantly’ from
your hypothesized population parameter.”
Psy 320 - Cal State Northridge 19
Important Concepts
Concepts critical to hypothesis testing
It’s all probability
Decision
Type I error
Type II error
Critical values
One- and two-tailed tests
Psy 320 - Cal State Northridge 20
Probability Statement
The Unsettling Part to it all
YOU decide on the odds for casting your
decision in favor of H0 or H1.
Even after you decide, you’re still never
certain if you made the right decision.
All your choices are based upon probability
distributions (sampling distributions).
Nothing is ever proved!!
Psy 320 - Cal State Northridge 21
7
Decisions
When we test a hypothesis we draw a
conclusion; either correct or incorrect.
Type I error
Reject the null hypothesis when it is actually
correct.
Type II error
Retain the null hypothesis when it is actually
false.
Psy 320 - Cal State Northridge 22
Type I Errors
Assume IQPLUS really has no effect on
intelligence
Assume we conclude that IQPLUS does
work.
This is a Type I error
Probability set at alpha (α)
α usually at .05
Therefore, probability of Type I error = .05
Psy 320 - Cal State Northridge 23
Type II Errors
Assume IQPLUS makes a difference
Assume that we conclude it doesn’t
This is also an error (Type II)
Probability denoted beta (β)
We can’t set beta easily.
We’ll talk about this issue later.
Power = (1 - β) = probability of correctly
rejecting false null hypothesis.
Psy 320 - Cal State Northridge 24
8
Confusion Matrix
Reality Reality
H0 H1 H0 H1
“H0” 1-α β “H0” .95 .16
Decision
Decision
Your
Your
“H1” α 1-β “H1” .05 .84
1.00 1.00 1.00 1.00
Psy 320 - Cal State Northridge 25
Critical Values
These represent the point at which we
decide to reject null hypothesis.
e.g. We might decide to reject null when
(p|null) < .05.
Our test statistic has some value with p =
.05
We reject when we exceed that value.
That value is the critical value.
Psy 320 - Cal State Northridge 26
One- and Two-Tailed Tests
Two-tailed test rejects null when
obtained value too extreme in either
direction
Decide on this before collecting data.
One-tailed test rejects null if obtained
value is too low (or too high)
We only set aside one direction for
rejection.
Psy 320 - Cal State Northridge 27
9
One- & Two-Tailed Example
One-tailed test
Reject null if IQPLUS group shows an
increase in IQ
Probably wouldn’t expect a reduction and
therefore no point guarding against it.
Two-tailed test
Reject null if IQPLUS group has a mean
that is substantially higher or lower.
Psy 320 - Cal State Northridge 28
Hypothesis Testing and Z
29
1-tailed example (α
α = .05)
All 5% in one tail 103.28 − 106
= −1.36
2
Table E.10 − smaller area of − 1.36 = .087
β = .087
Table E.10 − smaller area of = .913
Power = 1 − β = .91
h0: µ = 100 h1: µ = 106
Psy 320 - Cal State Northridge 30
(1.64 * 2) + 100 = 103.28
10
2-tailed example (α
α = .05)
5% split into 2 tails 103.92 − 106
= −1.04
(2.5% in each tail) 2
Table E.10 − smaller area of − 1.04 = .149
β = .15
Table E.10 − smaller area of = .851
Power = 1 − β = .85
h0: µ = 100 h1: µ = 106
Psy 320 - Cal State Northridge 31
103.92
11