An Introduction to
Experiment
Fengyan Cai
Shanghai Jiao Tong University
2025
Agenda
• Experiments and Other Research Method
• Source of Error in Experiment
• Hypotheses, IV and DV
Part I: Alternative Methods to
Controlled Experiments
• Observational Method
• Correlational Method
• Experimental Method
• 作者：主编 王永贵 副主编 项典典 马双 杜运周
• 书号：315621
•出版时间：2023-04-04
•ISBN：978-7-300-31562-1
？？高效 vs. 低效 ？？
《在家工作可以吗?来自中国实验的证据》通过在携程公司内
部做实验搜集数据，并最终发表在 The Quarterly
Journal of Economics上。
3月28日，我国的人口经济学家梁建章发布了题为《中国婚姻家庭报
告2022版》的报告，建议通过降低法定结婚年龄为18岁和生育三孩
的家庭享受房价打五折的方式来解决眼下问题。
[Link]
Why experiment?
The reason is that a true experiment is the best method for
finding out whether one thing really causes another.
When nonexperimental methods are
desirable?
One’s choice of method should be dictated by both the nature of
the research question and the phenomenon under investigation.
• Not interested in causation
• Some variables cannot be experimentally varied
• Ethical reasons
• Use alternative methods as a means of suggesting, clarifying,
refining, or extending experimental research findings, before or after
an experiment is run.
• Sometimes non-experimental information is collected during an
experiment (e.g. survey, interview)
1. Correlational studies
Studies in which the investigator is looking for a relationship
between two variables which can be measured – but which
cannot be controlled – are called correlational studies.
The degree of relationship between the two variables is
expressed as a correlation coefficient. The value of γ ranges
from +1.00 to -1.00.
• Cannot control the presumed antecedents or cannot assign subjects
at random.
• Instead of introducing treatments, the researcher makes observations
of events as they occur in nature.
• Can only tell the strength of the relationship but cannot tell the nature
• When one variable causes another, the two will inevitably be
correlated. However, the reverse is not true.
1. Correlational studies
Example 1:
• Children who watch hours of violent television shows every day are
more violent and aggressive than children who don’t.
• Suppose we find a high positive correlation, would we be able to
conclude that the children’s aggression was caused by the violent TV
shows?
NO!!
1. Correlational studies
Example 2:
• The type of birth control women used was correlated with whether
they had sexually transmitted diseases.
• The results showed that women whose partners used condoms had
more STDs than women who used diaphragms or contraceptive
sponges (Rosenberg et al, 1992)
1. Correlational studies
A. It is possible that the causal relationship is exactly the opposite of
the one we have hypothesized.
B. Or, it is possible that the two observed variables – X and Y – are
both highly correlated with (and maybe caused by) a third variable
– Z – which we don’t know about and haven’t measured.
1. Correlational studies
Covariation types
• True causation
• Reverse causation
• Double causation
• Mediation
• Spurious correlation - just happen to occur simultaneously
In-class Exercise: How to test the effect of
shopping on emotion
• 5 minutes group discussion
• Each group finds one student to write summary and report
• Randomly choose 3 groups to present the design
2. Experiments
I. Pre-experiment: treatment, but no good control
II. Quasi-experiment: treatment, better control group,
but no randomization
III. True experiment: treatment, and good control group,
randomization
2a. Pre-experimental designs
Suppose that you are interested in the effects of alcohol on
people’s feelings and behaviors.
How would you study this question?
i) Treatment -> Observation (no control group at all)
ii) Before -> Treatment -> After (not a real control)
2a. Pre-experimental designs
1. The one-shot case study:
X O
• Treatment -> Observation (no control group at all)
2. The one-group Pretest-posttest design
O X O
• Before -> Treatment -> After (not a real control)
2a. Pre-experimental designs
3. The Static-Group Comparison
X O1
O2
2b. Quasi-experimental designs
Refers to studies that have IVs and DVs but do not use
randomization to assign subjects to groups (Cook and Campbell
1979).
2b. Quasi-experimental designs
Example: effects of mood on altruism
• Demand effect in laboratory.
• Field setting (e.g., movie theatre) – more “natural” mood
induction.
Self-selection problem: People who choose to attend happy
versus sad movies are different to begin with.
2b. Quasi-experimental designs
“From the field to the laboratory to the field.”
The convergence of results across methods and across settings
will greatly enhance our confidence in both sets of findings.
2c. True Experiment Design
1. Pretest-Posttest control group design
R O1 X O2
R O3 O4
2. The Posttest-Only Control Group Design
R X O1
R O2
2c. True Experiment Design
3. The Solomon Four-Group Design
R O1 X O2
R O3 O4
R X O5
R O6
Possible variations
Number of treatment:
O1 X1 F X2 2 F
Number of measurements:
Types of measurement;
Types of treatment:
2c. The advantages of true experiments
• A great deal of control.
• Control errors via randomization.
• Systematic variation.
Given my goals, given my research question, given
methodological demands of the setting in which I choose to
answer that question, which is the tool that best suits the
requirements of the task?
Other typology of research
• Field vs. Laboratory Research
• Problem-Oriented vs. Process-Oriented Research
Other typology of research
• Field vs. Laboratory Research
Other typology of research
• Basic vs. Applied Research
• Basic: Try to find the best answer to the question of why people
behave the way they do, purely for reasons of intellectual curiosity.
• Applied: the goal is to solve specific problem
• Problem-Oriented vs. Process-Oriented Research
谢谢大家第一次课的积极配合与精彩表现！