Experimental and Quasi-Experimental Techniques in
Economics
Vijay Victor
January 15, 2026
Vijay Victor Causal Inference January 15, 2026 1 / 35
Course Outline
Topics Covered in This Course
Topic 1: Model misspecification issues, model selection criteria, and
their applications
Topic 2: Applications of limited dependent variable models in
microeconomics and macroeconomics
Topic 3: Estimation of panel data models and their applications
Topic 4: Estimation of quasi-experimental models for causal inference
Topic 5: Impact evaluation and its applications in public policy
Focus of This Module
This part of the course emphasizes causal inference, quasi-experimental
methods, and impact evaluation for evidence-based policy analysis.
Vijay Victor Causal Inference January 15, 2026 2 / 35
Reference Textbooks
Primary references for this module
Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., &
Vermeersch, C. M. J. (2016). Impact evaluation in practice (2nd ed.).
World Bank. [Link]
Angrist, J. D., & Pischke, J.-S. (2015). Mastering ’Metrics: The path
from cause to effect. Princeton University Press.
[Link]
How we will use these
Gertler et al. (2016): program evaluation logic, counterfactuals,
and impact evaluation designs
Angrist & Pischke (2015): intuition for identification and core
quasi-experimental tools
Vijay Victor Causal Inference January 15, 2026 3 / 35
Course Evaluation Scheme
Assessment Components
Assignment and Presentation – 20%
Quiz – 10%
Mid-Term Examination – 25%
End-Term Examination – 35%
Class Participation – 5%
Attendance – 5%
Note
Continuous assessment (assignments, quizzes, and participation) accounts
for 40% of the final grade.
Vijay Victor Causal Inference January 15, 2026 4 / 35
Introduction: The Critical Role of Causal Inference
Why Correlation ̸= Causation:
Observational data often reveals misleading patterns. For example:
Ice Cream and Drowning: Both rise in summer due to heat (a
confounding variable), but ice cream does not cause drowning.
Education and Earnings: Highly educated people earn more, but is
this due to schooling itself, family background, or innate talent?
Policy Stakes:
Misattributing causality can waste resources or harm communities.
Example: If a job training program correlates with higher wages, is it
due to the training or participants’ pre-existing motivation? Without
causal proof, scaling the program could be ineffective.
Core Objective of Causal Inference:
Isolate the “treatment effect” by answering: What would have happened
to the treated group if they had not received the intervention?
Vijay Victor Causal Inference January 15, 2026 5 / 35
The Credibility Revolution: A Paradigm Shift
Historical Context:
Pre-1990s Economics: Relied on theoretical models and simple
correlations (e.g., supply-demand curves) with limited empirical rigor.
Catalysts for Change:
High-profile policy failures (e.g., unintended consequences of welfare
reforms).
Advances in econometrics and computing power.
Pioneers like David Card, Esther Duflo, and Joshua Angrist
championing empirical methods.
Vijay Victor Causal Inference January 15, 2026 6 / 35
Key Tenets of the Revolution
Skepticism of “Theory Alone”: Demand for real-world validation.
Focus on Internal Validity: Ensuring findings reflect true causal
relationships.
Innovation in Methods: Development of RCTs and
quasi-experimental tools.
Impact:
Policy decisions increasingly rely on causal evidence (e.g., conditional
cash transfers in Mexico’s PROGRESA).
Nobel Prizes (e.g., 2021 Nobel to Card, Angrist, and Imbens)
cementing causal inference’s centrality.
Vijay Victor Causal Inference January 15, 2026 7 / 35
Impact Evaluation
From Credible Causality to Impact Evaluation
Why Impact Evaluation?
Credible causal inference tells us whether a program works.
Impact evaluation asks how much it works, for whom, and at
what cost.
Impact evaluation applies causal inference tools to:
assess policies and programs,
inform scale-up, redesign, or termination decisions,
improve resource allocation under constraints.
Key idea
Credibility is necessary, but not sufficient, for good policy evaluation.
Vijay Victor Causal Inference January 15, 2026 8 / 35
Impact Evaluation
What Is Impact Evaluation?
Definition:
Impact evaluation measures the causal effect of a program, policy, or
intervention on outcomes of interest, relative to a credible counterfactual.
Potential outcomes framework:
Impact = E[Y (1) − Y (0)]
In practice, impact evaluation also asks:
Is the effect economically meaningful (not just statistically
significant)?
Does it differ across groups (heterogeneity)?
Does it justify the cost?
Vijay Victor Causal Inference January 15, 2026 9 / 35
Impact Evaluation
Core Questions in Impact Evaluation
Impact evaluation goes beyond “Does it work?”
Key questions:
1 Effectiveness: Did the program cause a change in outcomes?
2 Magnitude: How large is the effect?
3 Heterogeneity: For whom does it work (gender, income, region)?
4 Mechanisms: Through what channels does the effect operate?
5 Cost-effectiveness: Is the impact worth the resources used?
Reminder
A statistically significant effect is not automatically a good policy.
Vijay Victor Causal Inference January 15, 2026 10 / 35
Impact Evaluation
The Impact Evaluation Cycle
A practical workflow:
1 Define the policy question
2 Specify outcomes (and when to measure them)
3 Construct a credible counterfactual
4 Estimate causal effects
5 Interpret magnitude and heterogeneity
6 Compare benefits to costs
7 Inform policy decisions (scale up / redesign / stop)
Key point
Impact evaluation is iterative, not a one-shot estimation exercise.
Vijay Victor Causal Inference January 15, 2026 11 / 35
Impact Evaluation
Impact Evaluation vs Program Monitoring
Important distinction:
Monitoring Impact Evaluation
Tracks inputs and outputs Measures causal effects on out-
comes
“How many were trained?” “Did training improve earnings?”
Descriptive Causal
Continuous (routine) Periodic (study-based)
Takeaway
Monitoring tells us what happened.
Impact evaluation tells us what would not have happened otherwise.
Vijay Victor Causal Inference January 15, 2026 12 / 35
Impact Evaluation
Why Credibility Comes First
Impact estimates are only meaningful if:
the comparison group is credible,
assumptions are transparent and defensible,
diagnostics support identification.
Otherwise, impact evaluation risks:
overstating benefits,
scaling ineffective programs,
misallocating scarce resources.
Link to the Credibility Revolution
Credibility is the foundation; impact evaluation is the application.
Vijay Victor Causal Inference January 15, 2026 13 / 35
Impact Evaluation
3. Experimental Methods: Randomized Controlled Trials
(RCTs)
The Gold Standard:
RCTs use random assignment to isolate treatment effects.
How RCTs Work:
Random Assignment: Split subjects into treatment (receives
intervention) and control (does not) groups.
Key Strength:
Balances both observed and unobserved traits (e.g., income, motivation)
across groups.
Example: Mexico’s PROGRESA program:
Randomly awarded cash transfers to families conditional on school/health
clinic attendance.
Result: Clear causal evidence that transfers improved education and health
outcomes.
Vijay Victor Causal Inference January 15, 2026 14 / 35
Impact Evaluation
Limitations of RCTs
Cost/Time: Large-scale trials are resource-intensive.
Ethics: Cannot randomize harmful conditions (e.g., poverty).
External Validity: Results may not generalize to other settings.
Vijay Victor Causal Inference January 15, 2026 15 / 35
Impact Evaluation
Quasi-Experimental Methods: Alternatives When RCTs
Aren’t Feasible
Core Idea:
Use observational data to mimic randomization through natural or
statistical experiments.
Vijay Victor Causal Inference January 15, 2026 16 / 35
Impact Evaluation
Propensity Score Matching (PSM)
Design:
Uses a statistical model (e.g., logistic regression) to assign a “propensity
score” reflecting each unit’s likelihood of receiving treatment, then
matches treated and untreated units with similar scores.
Example:
Comparing job training participants with non-participants who have nearly
identical profiles in terms of age, education, and past work experience.
Key Assumption – No Unobserved Differences:
All relevant confounders are included so that matching captures true
comparability.
Limitation:
Unmeasured traits (motivation, social networks) remain a potential source
of bias if they affect outcomes and differ across groups.
Vijay Victor Causal Inference January 15, 2026 17 / 35
Impact Evaluation
Difference-in-Differences (DiD)
Design:
Measures how outcomes change over time in a treated group compared to
a similar control group, capturing the “difference in differences.”
Example:
Card and Krueger (1994) analyzed employment changes in New Jersey
(treatment, with a minimum wage hike) versus Pennsylvania (control, no
change).
Key Assumption – Parallel Trends:
In the absence of treatment, both groups would have continued on
comparable paths. If this assumption fails (e.g., one region experiences a
unique economic shock), estimates become biased.
Limitation:
Large external events (like recessions) affecting only one group undermine
the validity of results.
Vijay Victor Causal Inference January 15, 2026 18 / 35
Impact Evaluation
Regression Discontinuity Design (RDD)
Design:
Compares units just above and just below a cutoff score or threshold that
determines treatment. Any jump in outcomes at the cutoff is attributed to
the treatment.
Example:
Evaluating scholarship effects by comparing students who barely qualify
(just above the cutoff) to those who barely miss out (just below it).
Key Assumption – No Manipulation Around the Cutoff:
Participants cannot precisely sort themselves to one side of the threshold
to gain or avoid treatment.
Limitation:
The estimated effect applies mainly to individuals near that cutoff (local
effect) and does not necessarily generalize broadly.
Vijay Victor Causal Inference January 15, 2026 19 / 35
Impact Evaluation
Choosing the Right Method
Feasibility:
RCTs may be too expensive, time-consuming, or impractical. In such
cases, quasi-experimental methods (DiD, IV, RDD, PSM) leverage existing
data.
Ethics:
Some interventions cannot be randomly withheld (e.g., critical health
treatments), so observational methods are necessary.
Trade-offs:
RCTs: Generally high internal validity but limited scope due to cost
or ethical constraints.
Quasi-Experiments: Broader applicability but rely on assumptions
(parallel trends, valid instruments, etc.).
Vijay Victor Causal Inference January 15, 2026 20 / 35
Impact Evaluation
Core Idea of Counterfactuals
When we ask a causal question, we’re trying to understand:
What would the outcome be with versus without a specific
intervention or treatment?
For any individual (or unit), we conceptualize two potential scenarios:
YiT : The outcome if the individual receives the treatment.
YiC : The outcome if the individual does not receive the treatment.
The difference between these two, YiT − YiC , would tell us the causal
effect of the treatment for that individual.
However, there’s a catch: in reality, we can only observe one of these
outcomes for any given person.
Vijay Victor Causal Inference January 15, 2026 21 / 35
Impact Evaluation
Example: Tutoring in Education
Consider a student, Alex, who struggles with studies. We want to know
how a one-on-one tutoring program affects his performance. Imagine two
scenarios:
With Tutoring (Treatment): Alex gets personalized tutoring
sessions alongside his regular classes.
Without Tutoring (Control): Alex continues with only his regular
classes, no tutoring.
Suppose:
If Alex receives tutoring, he scores 85 on his next class test
(YiT = 85).
If he doesn’t receive tutoring, he scores 70 on the same test
(YiC = 70).
Vijay Victor Causal Inference January 15, 2026 22 / 35
Impact Evaluation
Example: Tutoring in Education
The causal effect of tutoring for Alex would be 85 − 70 = 15 points. But
here’s the problem:
if Alex gets tutoring, we see his score of 85 but have no way of
knowing he’d have scored 70 without it.
If he doesn’t get tutoring, we see 70 but can’t observe the 85. We’re stuck
observing just one outcome, leaving the other as a counterfactual—a
”what if” scenario we can’t directly see.
Vijay Victor Causal Inference January 15, 2026 23 / 35
Impact Evaluation
The Fundamental Problem
Why It’s an Issue:
For any individual, we can’t observe both YiT and YiC at the same
time. This makes the individual-level treatment effect (YiT − YiC )
fundamentally unobservable.
We can imagine Alex’s score without tutoring when he gets it (or vice
versa), but we can’t measure it directly.
What This Means:
Since we can’t pinpoint the exact effect for Alex (or any single
person), researchers shift their focus to average effects across groups.
This brings us to the Average Treatment Effect (ATE), which
estimates the typical impact of a treatment across a population.
Vijay Victor Causal Inference January 15, 2026 24 / 35
Impact Evaluation
Average Treatment Effect (ATE)
The ATE is the expected difference in outcomes between the treatment
and control scenarios, averaged over all individuals:
ATE = E[YiT − YiC ]
In our tutoring example, the ATE would tell us, on average, how many
points tutoring boosts test scores across a group of students like Alex.
How We Might Estimate It:
A simple approach could be to:
Take a group of students who get tutoring and calculate their average
score (e.g., 85).
Take a group who don’t get tutoring and calculate their average score
(e.g., 70).
Subtract: 85 − 70 = 15 points.
Vijay Victor Causal Inference January 15, 2026 25 / 35
Impact Evaluation
The Need for a Valid Comparison Group
But this naive comparison has a flaw.
What if the students who get tutoring are already more motivated or
better at studies than those who don’t? The 15-point difference
might reflect their prior abilities, not just the tutoring’s effect.
To estimate the ATE accurately, we need a comparison group that
mirrors the treatment group in every way—except for receiving the
treatment.
If the tutored and non-tutored students are identical except for
tutoring, any difference in their average scores can confidently be
attributed to the program.
In real life, groups often differ systematically:
Self-Selection: Motivated students might seek out tutoring, while
less motivated ones don’t.
Other Factors: Differences in socioeconomic background, prior skills,
or teacher quality could skew results.
Vijay Victor Causal Inference January 15, 2026 26 / 35
Impact Evaluation
Selection Bias and Solution
Selection Bias:
If tutored students are naturally high-achievers, their higher scores
might stem from their starting point, not the tutoring.
This bias muddies the waters, making it hard to isolate the true effect
of the intervention.
Solution:
To overcome this, researchers use strategies like random assignment
(e.g., a randomized trial where students are assigned to tutoring or no
tutoring by chance).
This ensures the groups are comparable, minimizing bias and letting
us trust that the ATE reflects the treatment’s impact.
Vijay Victor Causal Inference January 15, 2026 27 / 35
Impact Evaluation
The Law of Large Numbers
Key Insight
The LLN says that when you have a large number of people in your trial,
the random assignment balances out the groups.
Think of it like flipping a coin many times:
With just 10 flips: might get 7 heads, 3 tails
With 1,000 flips: closer to 500 heads, 500 tails—pretty even!
In a trial, with a large sample:
Treatment and control groups will look similar
Similar in measured factors: age, gender, health conditions
Similar in unmeasured factors: stress levels, genetics
Vijay Victor Causal Inference January 15, 2026 28 / 35
Impact Evaluation
A Simple Example
Scenario
Suppose the drug lowers blood pressure by 5 mmHg for everyone.
Without Randomization:
Treatment group: sicker (150
mmHg) With Randomization:
Control group: healthier (140 Both groups start at 140 mmHg
mmHg) After drug: Treatment → 135
After drug: Treatment → 145 mmHg
mmHg Difference: 135 - 140 = -5
Difference: 145 - 140 = 5 mmHg
mmHg Correctly shows the drug’s effect
Result looks right, but starting
point skewed things!
Vijay Victor Causal Inference January 15, 2026 29 / 35
Impact Evaluation
Conditional Expectations
Let’s add some structure to this with the terms and math used in
statistics.
Treatment Indicator
Di = 1 if person i gets the treatment
Di = 0 if they’re in the control group
Observed Outcome
Yi is what we measure (e.g., blood pressure after the trial)
Conditional Expectation
E [Yi | Di = 1]: Average outcome for the treatment group
E [Yi | Di = 0]: Average outcome for the control group
Vijay Victor Causal Inference January 15, 2026 30 / 35
Impact Evaluation
Potential Outcomes
Each person has two potential outcomes:
Y1i : What their outcome would be if they got the treatment
Y0i : What their outcome would be if they didn’t
You only see one of these, depending on their group. Random assignment
ensures the groups are alike, so the average potential outcomes without
treatment are the same:
Key Equality from Randomization
E [Y0i | Di = 1] = E [Y0i | Di = 0]
This means the treatment group’s average blood pressure if they hadn’t
taken the drug would match the control group’s.
Vijay Victor Causal Inference January 15, 2026 31 / 35
Impact Evaluation
Deriving the Causal Effect
Let’s assume the treatment has a constant effect, adding κ to everyone’s
outcome (e.g., κ = −5 mmHg for our drug). So:
Y1i = Y0i + κ
Treatment Group: Control Group:
Yi = Y1i = Y0i + κ Yi = Y0i
Vijay Victor Causal Inference January 15, 2026 32 / 35
Impact Evaluation
Calculate the Averages
Treatment Group Average
E [Yi | Di = 1] = E [Y1i | Di = 1]
= E [Y0i + κ | Di = 1]
= E [Y0i | Di = 1] + κ
Control Group Average
E [Yi | Di = 0] = E [Y0i | Di = 0]
Vijay Victor Causal Inference January 15, 2026 33 / 35
Impact Evaluation
The Difference
The difference between treatment and control group averages is:
E [Yi | Di = 1] − E [Yi | Di = 0] = (E [Y0i | Di = 1] + κ) − E [Y0i | Di = 0]
Due to Randomization
E [Y0i | Di = 1] = E [Y0i | Di = 0]
Therefore, this simplifies to:
E [Yi | Di = 1] − E [Yi | Di = 0] = κ
Vijay Victor Causal Inference January 15, 2026 34 / 35
Impact Evaluation
Interpretation
The difference in averages directly gives us the causal effect of the
treatment!
In our example:
Treatment group averages: 135 mmHg
Control group averages: 140 mmHg
Difference: 135 - 140 = -5 mmHg
This is exactly the drug’s effect!
The Power of Randomization
Randomized controlled trials allow us to measure causal effects directly!
Vijay Victor Causal Inference January 15, 2026 35 / 35