Evaluating Experiments
Internal and External Validity
Definitions
• Internal validity is the level to which we are confident that the
independent variable affected the dependent variable.
• External validity is the level at which we can generalize our
findings outside the experiment.
Internal Validity
The methodological quality of
the experiment
• To what extent is the DV influenced by
the IV and not some other variable?
• Are we justified in making a cause-
and-effect inference between the
conditions in the IV and the changes in
the DV?
Factors that can affect the internal
validity of an experiment
1. Construct validity – when an IV or DV is not fully operationalized and cannot
be properly manipulated/measured.
2. A confounding variable (e.g., participant variables, environmental variables,
materials used in the experiment).
3. Biases (demand characteristics, experimenter bias, social desirability effect,
reactivity, confirmation bias, order effects).
External validity
The generalizability of findings in
the experiment
• Can we generalize our findings to our target
population?
• Can we generalize our findings to the
situations in the world we are really
interested in?
External validity
The generalizability of findings in
the experiment
• Generalizability refers to the
extent to which the results of
the study can be applied
beyond the sample and
the setting used in the study
itself.
External validity
The generalizability of findings in the
experiment
• Population validity is high when the
sample is representative of the
target population. This depends on the
sampling technique and how it was
applied.
Culture - Ethnicity
Age
Gender
Socioeconomic status
Specific skills
External validity
The generalizability of findings in the
experiment
• Ecological validity is high when the
experiment is conducted in
a situation which resembles the real-
life setting in which the behavior would
normally occur.
• Mundane realism is high when
the task that participants are asked to
perform in an experiment is very similar
to something they would do in real life.
Internal vs. External Validity
• In psychological research, psychologists are “in a battle”
between internal and external validity.
Internal vs. External
Validity
• For a study to show causality, it
must be highly controlled. This is
high internal validity, but it
compromises external validity.
• For external validity to be high, the
situation should be naturalistic and
reflect real-life situations. This
means that the situation should not
be artificial or overly controlled.
This compromises internal validity.
Internal vs.
External Validity
• Therefore, we can never
draw conclusions from a
single study.
• Ideally, a theory should be
supported by a set of
experimental research that is
both internally and externally
valid.
• Internal validity: the methodological quality of
the experiment.
Did the experiment measure what it was
supposed to measure? Were the
extraneous variables controlled?
• External validity: the generalizability of the
findings.
Can we generalize the findings to the entire
target population? Is the sample representative?
Was the setting and/or the task similar to the
real world?
• Population validity => the sample is
representative
• Ecological validity => setting similar to the real
world
• Mundane realism => the task performed was
something participants would normally do.