Manipulation: The researchers create two versions of political

Introduction to Experiments, Causality, and Counterfactuals advertisement (positive and negative ads)

Experiment- a research method where the researcher manipulates Control: The researchers make sure both ads are equal in length, use
an independent variable and observes its effect on a dependent the same narrator’s voice and background music, etc.
variable, while holding other variables constant.
Randomization- key to ensuring that treatment and control groups
Confounding Variable- study that influences both the independent are statistically equivalent before the intervention.
and dependent variables, potentially leading to a spurious association
-It eliminates selection bias – differences in outcomes can be more
confidently attributed to the treatment.

Why randomization matters?

• Randomization creates two groups that are statistically equivalent

before the treatment, so the untreated group’s outcome approximates
the counterfactual for the treated group.

• It also reduces bias results.

Why use experiments in political science?

• The purpose is to identify causal relationships & cause and effects

between political phenomena, test theories about political
communication, policy impacts, voter mobilization, governance
reforms, etc.

• Experiments allow researchers to design interventions that mimic

real-world political processes but under controlled conditions.

Types of Experiments in Political Science

1. Laboratory Experiments – conducted in controlled environments

(e.g. a political decision-making simulation in a classroom).

2. Field Experiments – conducted in real-world settings (e.g. sending

varied political mailers to voters during election campaigns).

3. Survey Experiments – embedded within surveys, where

respondents are randomly assigned to different question framings or
scenarios.

Control vs. Manipulation

Control - ensuring other possible influencing factors remain constant

or balanced across groups.

Manipulation - changing one specific factor to see how it influences

outcomes.

Case Scenario 1: A political science research team wants to find out

Experimental Studies- Research method where the researcher
whether a positive campaign advertisement increases support for a
manipulates
candidate compared to a negative campaign advertisement.

Observational studies- researchers don’t control the assignment of • It requires ruling out alternative explanations for the observed
treatments and confirm observational findings. relationship.

Case Scenario 2: An observational study found that regions with • Answers “Did the treatment cause the observed change?”
higher voter education levels tend to have greater political
participation. Researchers might design an experiment to test Case Scenario 3: A researcher finds that implementing an
whether a voter education campaign directly increases turnout. anti-corruption law in a province led to a measurable decrease in
reported bribery cases, concluding that the law caused the reduction.
Causal inference- process of determining whether a change in one
variable directly influences d other.
Counterfactual reasoning - process of thinking about what would External validity- concerns the extent to which the study's results
have happened under different circumstances — imagining alternative can be generalized to other populations, settings, and situations.
reality where event is changed, and comparing it to what actually
happened. Threats to External Validity:

• Answers “"What would the outcome have been if the treatment or 1. Selection Bias: The sample studied may not be
intervention had not occurred?"
representative of the broader population, limiting
generalizability.
Case Scenario 4: A political scientist asks, “If the anti-corruption law
2. Testing Effect: As with internal validity, the act of
had not been implemented, would bribery cases have remained the
pre-testing can make participants behave differently,
same or increased?” and uses statistical models to estimate what
affecting how well the results apply outside the study
would have happened in that alternate scenario.
context.
3. Historical Context: Events that occur during the study
Internal validity- study establishes a causal relationship between the
might be unique to that specific time and not generalizable
independent and dependent variables, ensuring the results are not
to other historical periods.
due to other confounding factors.
4. Hawthorne Effect: Participants change their behavior
Threats to Internal Validity: because they know they are being observed, leading to
unnatural responses that don't reflect real-world behavior.
5. Stimulus Specificity: The specific nature of the stimulus
1. History: An external event occurring during the study that
(e.g., the experimental intervention) may not be the same in
affects the outcome. other contexts.
2. Maturation: Natural, age-related or developmental changes 6. Situation Specificity: The unique characteristics of the
in participants over time that might influence the results. research setting may not be found in other settings where
3. Testing: The effect of a pretest on a participant's the results are intended to apply.
performance on a subsequent test, due to familiarity or
practice.
4. Instrumentation: Changes in the way data is measured or
recorded over time, affecting the reliability of the results. Overlapping Threats- Some threats can affect both internal and
5. Selection Bias: When groups are inherently different from external validity, as they introduce alternative explanations for the
results or create unnatural conditions.
the start of the study, creating a systematic difference that
isn't due to the treatment.
6. Mortality/Attrition: Participants dropping out of the study in 1. Hawthorne Effect: It can reduce internal validity by
disproportionate numbers from certain groups, leading to providing an alternative explanation for improved
unequal groups at the end. outcomes and external validity because the behavior isn't
7. Regression to the Mean: The tendency for extreme scores natural.
to become less extreme on subsequent measurements. 2. Testing Effect: Repeated exposure to a measure can
8. Interaction Effects: When one threat interacts with another, improve scores (internal validity) and also make the
such as selection and maturation participant's experience atypical, reducing generalizability
(external validity).