Causality and

experiments
The following need to send me their
clicker numbers:

Miquel Mijia
Isaiah Weiss

Review
• Diagramming causal relations
– Variables as nodes (boxes)
– Causal relations as arrows
• Not tracing the flow of activity, but causal relations
– If there are conditions under which changing one
variable will result in change of another variable,
include a arrow between the variables
• Sometimes there are important intermediate causes
such that a more ultimate cause only produces its
effect through a more proximate cause

Match struck Tip temperature Match lit

[yes, no] [>350°, <350°] [yes, no]

Review - 2
• Common cause
– A positive correlation between two variables may
be the result of a common cause for both
Pine needles Fish
[on tree, dropped] [alive, dead]

X
Toxic waste
[no, yes]

• If the causation is direct, there should be no way to

screen off E from C
 Review – 3
Common mistakes in causal reasoning
Ignoring a common cause
• You wake up with a fever. A few hours later you
find red spots on your skin. You conclude that
the fever must have caused the red spots.
Post hoc, ergo propter hoc
• You go to the doctor, the next day you feel
better, therefore the doctor caused you to feel
better
Confusing cause and effect
• Layoff of workers is cause of economy slowing
down
4

Clicker Question
Which causal fallacy does this example illustrate?
Whenever the power goes out, your Dad starts
beating on the wall. The power comes back on and
he takes credit for getting it on again.
A. Ignoring a common cause
B. Post hoc, ergo propter hoc
C. Confusing cause and effect
D. None of the above

Clicker Question
What causal fallacy is illustrated in this example: Mindy has a
car accident. When the police arrive, they find a lot of
empty beer cans in the passenger seat. They conclude that
the beers cans caused the accident.
A. Ignoring a common cause
B. Treating coincidence as a cause
C. Post hoc, ergo propter hoc
D. Confusing cause and effect
 The basic idea of an experiment
If the independent variable is the cause of the
dependent variable, then a manipulation of the
independent variable should produce a change in the
value of the dependent variable
And if it were not the cause, we would not
expect such a result from manipulation
Manipulation
?
Independent variable Dependent variable
[values] [values]

Experiments on regular
deterministic systems
When there is no variance in the population
being studied, statistical analysis is not necessary

The main danger is affirming the consequent

The key is to test a causal hypothesis in
which you would not expect the effect to
occur unless you were right about the
cause.
Manipulation
(vary the object dropped)
?
Object dropped Object falls
[no, yes]
[mass] [no, yes]
[ft/sec 2]

Clicker Question
To avoid affirming the consequent, which premise should one
use to confirm a hypothesis?
A. If X is the cause of Y, then Y will change as X
changes
B. If X is the cause of Y, then Y will not change as X
changes
C. If X is not the cause of Y, then Y will change as X
changes
D. If X is not the cause of Y, then Y will not change as X
changes
 Variability in non-deterministic
systems
Different systems of the same kind will vary in their
responses depending on
• their particular composition (genetics, etc.)
• their particular history, etc.

The same system may respond differently on different

occasions
Your reaction time will differ depending on
• how much sleep you have had
• what you have had to drink, etc.

Challenge: how to detect causal relations in the face of

background variability (noise)?
10

Experiments on non-
deterministic systems
Since complex systems
biological
cognitive
social
are not perfectly regular in their behavior

Researchers cannot simply do an experiment on one

instance and draw a conclusion about the whole
population
Rather they must work with samples and draw
conclusions based on statistical analysis
• Are the differences in the values of the dependent
variable greater than expected by chance?
11

Confounding Variables
The reason different individuals behave differently is that among
extraneous variables, some may be related to the effect of interest.

When such variables are or may be correlated with the putative cause
and may be responsible for the effect produced in the study, they are
called confounds

There are two kinds that are particularly important:

Subject variable confounds:
• Differences between subjects in the study
Procedural variable confounds:
• Differences in the way different groups are treated

If these variables are correlated with the independent variable and are
also causes of the dependent variable, the experiment is confounded
12
 Clicker Question
A confound is
A. The dependent variable in an experiment
B. An extraneous variable that may causally affect
the independent variable
C. An extraneous variable that may causally affect
the dependent variable
D. The independent variable in an experiment

Strategies for controlling

confounding variables
Locking
Most commonly used to control confounding procedural
variables

Randomization
Most commonly used to control confounding subject
variables

Matching subjects
A less preferred strategy for controlling confounding subject
variables
– Only works for known confounds

Making confounding variables into studied variables

14

Procedural variable confounds

When you conduct a manipulation, generally more than
one thing will be changed
– These variables will then be correlated with the
independent variable but with respect to the
independent variable being tested are
extraneous
– If one of the other variables is causally related to
the effect of interest, it rather than the variable
you are considering may be the cause—
confound
 Confounding Procedural
Variables
The president of the AGL corporation wanted to get his
workers to be more productive
She found that when each employee
was presented with a jar of jellybeans,
productivity increased

Was it the jellybeans that caused the

increased productivity? Or was it:
Novelty of the situation
Attention from the president
Desire to reciprocate

16

Controlling confounding
procedural variables
Correlation
or causation X Procedural variable

Manipulation
X
?
Independent variable Dependent variable
[values] [values]

Strategy: break the correlation—thereby breaking the

effect of the confounding variable
Commonly achieved via locking

Clicker Question
What method(s) could you use to eliminate the effects
of the attending from the president as a confounding
procedural variables in the jellybean case?
A. Randomly assign subject to get jellybeans
B. Lock the attention of the president for all
conditions
C. Match subjects on how much attention they
got from the president
D. Make the attention of the president a variable
studied in the experiment
 Demand characteristics can
create procedural confound
People may change their behavior when they are being studied
(recall: Reactivity Bias)
People want to be liked (or not!)
People want to be helpful (or not!)
People want you to think they’re a good/clever person
People want to be thought of as intelligent and normal (not
crazy, stupid or obsessed)
Problem if subjects figure out the point of an experiment
Solutions:
• Keep subjects blind as to the point
of the experiment or what is being
studied (single-blind experiment)
• Make sure procedure is locked so
all subjects are affected the same
19

Experimenter Bias Can Create

Procedural Confound
Danger that experimenters will see what they want to
see (a former of observer bias)
Mendel’s data is too perfect—there should be more
variability
• Most likely explanation is that he did not
deliberately cheat (remember, he was a monk!)
• but he reported the best cases and subjectively
biased his counting of plants

Keep the data-tabulator blind as to which group different

subjects are in
Double-blind study
20

Subject variable confounds

Subjects in an experiment may have different values on
other variables than the independent variable
People of different ages sleep different amounts
Women might be affected differently than men

If these aren’t the independent or dependent variable, these

variables are extraneous

If there is a correlation between these variables and the

independent variable,
they, rather than the variable you are focusing on, may
be what produce the change in the dependent variable
Such variables are confounds
21
 Controlling confounding subject
variables
Manipulation
?
Independent variable Dependent variable
[values] [values]

X
Correlation X
or causation Subject variable

Strategy: break the correlation—thereby breaking the

effect of the confounding variable
Random assignment of subjects is a strategy for
breaking the correlation

Controlling subject confounds:

Between subjects randomization

Manipulate independent variable

Task Task
IV
Key Key
Behavior Behavior

Statistically
Significant
DV Difference?
 Controlling subject confounds:
Within subjects design

Subjects serving as own control

Task Task

Key Key
Behavior Behavior

Statistically
Significant
DV Difference?

Between-subject design
GOOD NEWS:

– Participants are not “contaminated” in one

condition as a result of having participated in the
other

BAD NEWS:

– Requires a larger number of participants

– Runs the risk of non-equivalence of subject groups

 Within-subjects designs
GOOD NEWS:
• Requires a smaller number of participants
• Rules out any differences between subjects

BAD NEWS:
• Potential “contamination” of participants’ behavior
from previous trial: carryover effect
• Subjects might learn from one condition and that
could alter their behavior in the second condition
– Practice effect
– Fatigue effect

These are additional confounds that must be controlled

for.

Clicker Question
A within-subjects design
A. Uses participants as their own controls
B. Requires fewer participants than a between
subjects design
C. Runs the risk of a carryover effect
D. All of the above

Counterbalancing
Goal: eliminate confounds in within-subjects design

Within subject counterbalancing

Reversing order: ABBA
Across subject counterbalancing
–Complete: every possible sequence of conditions—requires n!
–Partial
• Random
• Latin Square:
– each condition appears once and only once in a given ordinal
position
– no two conditions are juxtaposed in the same order more than
once
Order 1: A B D C
Order 2: B C A D
Order 3: C D B A
Order 4: D A C B
 Pretest—Posttest Design
There is always a danger in an experiment that the members
of the two (or more) groups being studied already differ on
the dependent variable

Best control is to focus on change, not raw value of the

dependent variable
Pretest: measure the dependent variable before
the intervention
Posttest: measure the dependent variable after
the intervention
Change = Posttest – Pretest
Manipulation
?
Independent variable Dependent variable
[values] [change]

Limitations of pretest-posttest
design
Just measuring change in one group using a pretest
and a posttest allows for confounds
– Time has elapsed and subjects have gotten older
(maturation)
– Events occurring between the pretest and posttest
could affect the dependent variable (history)
– Experience with previous test may change
performance
– Pretest and posttest may vary in difficulty

Use of pretest-posttest does not obviate the need for a

control group

Example: Exercise and sleep

Is there a causal relation between exercise and sleep?
Manipulation
(instruct subjects to
exercise little or much)

?
Exercise Sleep
[little, much] [<8 hours, >8 hours]
X
X
Job stress
Correlation
 Example: alcohol and running
speed

? 100 yard dash speed

Alcohol
[no, yes] [seconds]

Between subjects or within

subjects
Between-subjects design
Different subjects would be used for the no-alcohol
and alcohol condition, and each would be tested
only once

Within-subjects design
Each subject would be tested both under the no-
alcohol and alcohol condition

35

Between subjects

Jim Roger

Angela Shane

Megan Sara

Peter Jessica
 Within subjects

Jim Jim

Angela Angela

Megan Megan

Peter Peter

Counterbalance
Alcohol No-alcohol
condition rest condition

Jim Jim
Angela Angela

No-alcohol Alcohol
condition rest condition

Megan Megan
Peter Peter

Internal validity
An experiment is internally valid if it was in fact the
manipulation of the independent variable that produced
the change in the dependent variable
– Are the effects on the dependent variable due
solely to the manipulation of the independent
variable?
– Was there a confounding subject variable that did
not get controlled?
– Was there a confounding procedural variable that
did not get controlled?
 Clicker Question
Which of the following is not a threat to internal validity
A. The independent variable is only a contributory
cause
B. The existence of a confounding procedural
variable
C. The existence of a confounding subject variable
D. All of the above

Planning an experiment
Say the color the following words are written in

Blue Pink
Brown Yellow
White Orange
Red Green
Does it seem harder to name the colors when the
words name a different color?

Planning an
experiment - 2
How might we test the claim that it is the meaning of the
word that makes it harder to say the color it is written in?

Operationalize the notion of being hard to read

– Slower reaction time when incongruent words
– More errors when incongruent words

Identify a sample population

– College undergraduates in psychology courses

Pick study design

– Between subject
– Within subject
 Controlling subject
variable confounds
What subject variables might you have to worry about
as confounds?

How to control for these confounds

If between subject
Randominze

If within subject
Counterbalance

43

Controlling for
procedural variables
What procedural variables should be controlled to avoid
confounds?
Context of presentation
Illumination of the stimuli
Length of words
Familiarity and frequency of words

Need to lock these variables so that they do not vary

across conditions

44

A miracle device?

Detect human beings hidden in building or behind

objects from almost two football fields away!
 The DKL LifeGuard
According to the company can detect human heartbeat
Range of up to 500 meters (540 yards)
No effective electronic or other countermeasures
No natural and synthetic material it cannot penetrate
No battery or any power sources required!
Repeatability of 99%
Can penetrate:
• 10-meter wide earthen barrier
• 10 feet of water
• concrete walls, steel bulkheads
Can distinguish a man, woman or child from any other animal,
even a gorilla or an orangutan
Requires only one day of operator training
Work as well in darkness as in daylight
Like no other technology on the market today.... literally in a
league of its own.

Double-blind test
• Five large plastic packing crates were set up in a line
a 30-foot intervals
• A DKL representative, using the DKL LifeGuard Model
2, tried to detect which of the five crates contained a
human being
• On preliminary tests in which the operator knew which
crate contained the person, the DKL LifeGuard found
the person 10 out of 10 times
• In the real, double-blind test, the operator found the
person 6 out of 25 times (and took much longer to
find the person)

External Validity
To what extent can you generalize the results of your
study?
Are they specific to a particular sample?
• College sophomores or the general population
Do they only apply in a particular (laboratory)
setting?
Do they generalize
beyond the details of
the manipulation
• Ecological validity

48
 Population Generalization
Will a study using one population generalize to another
population?

– Will a study of college sophomores generalize to

middle-aged adults?
– Will a study of chronically depressed patients
generalize to patients who are acutely
depressed?
– Will a study of captive raised dolphins generalize
to wild dolphins?
– Will a study on mice generalize to humans?

Setting Generalization
Will a study conducted in one laboratory or clinical
setting generalize to the setting of interest?

– Will results obtained in a flight simulator

generalize to an actual cockpit?

– Will results obtained in an outpatient setting

generalize to a psychiatric hospital?

– Will results obtained in a laboratory generalize to

customers in a store?

Manipulation Generalization
Will a result obtained with one task generalize to other
tasks or stimuli?

– Will studies of perceiving visual illusions

generalize to perception of ordinary objects?

– Will a survey of consumer attitudes generalize to

consumer behavior?
 Assessing External Validity
Don’t assume that the answer to external validity
questions is negative

Must make a plausibility judgment in assessing external

validity
– Is the target population different from the studied
population in ways that are likely to matter for the
causal claim?
– Is the target setting different from the studied setting
in ways that are likely to matter for the causal claim?
– Is the manipulation used in the experiment different
from the target process in nature in ways that are
likely to matter for the causal claim?

Example: Rats
and Saccharine

1977 Canadian study which fed pregnant rats up to 20% of

their body weight per day in saccharine showed an
increase in bladder tumors

Saccharine was banned in Canada and the FDA was

about to ban its use in the US
when Congress intervened
Assessing external validity:
– Are rats relevantly like humans
– Is eating in the laboratory like eating at home, etc.?
– Is feeding up to 20% of body weight like eating as
part of diet?