Propensity Score Matching (PSM)

Dr. Muhammad Shahadat Hossain Siddiquee

Professor, Department of Economics
University of Dhaka
Email: [Link]@[Link]
Cell: +8801719397749
PSM AND REGRESSION-BASED METHODS
• Given that matching produces consistent estimates under weak
conditions, a practical advantage of PSM over OLS is that it reduces the
number of dimensions on which to match participants and comparison
units.
• Nevertheless, consistent OLS estimates of the ATE can be calculated
under the assumption of conditional exogeneity.
• One approach suggested by Hirano, Imbens, and Ridder (2003) is to
estimate a weighted least squares regression of the outcome on
treatment T and other observed covariates X unaffected by participation,
using the inverse of a nonparametric estimate of the propensity score.
• This approach leads to a fully efficient estimator, and the treatment
effect is estimated by
PSM AND REGRESSION-BASED METHODS…
with weights of 1 for participants and weights of for the control
observations.
• Ti1 is the treatment indicator, and the preceding specification attempts to
account for latent differences across treatment and comparison units that
would affect selection into the program as well as resulting outcomes. For an
estimate of the ATE for the population, the weights would be for the
participants and for the control units.
CASE STUDY 1: STEPS IN CREATING A MATCHED SAMPLE OF
NONPARTICIPANTS TO EVALUATE A FARMER-FIELD-SCHOOL
PROGRAM
• A farmer-field-school program was started in 1998 by scientists in
collaboration with CARE-Peru. In their study of the program, Godtland
and others (2004) applied three different steps for generating a common
support of propensity scores to match nonparticipants to the participant
sample.
• These steps combined methods that have been formally discussed in the
PSM literature and informal rules commonly applied in practice.
• First, a propensity score cutoff point was chosen, above which all
households were included in the comparison group. No formal rule exists
for choosing this cutoff point, and Godtland and others used as a
benchmark the average propensity score among participants of 0.6.
CASE STUDY 1…
• Second, the comparison group was chosen, using a nearest-neighbor
matching method, matching to each participant five nonparticipants
with the closest value of the propensity score (within a proposed 0.01
bound). Matches not in this range were removed from the sample.
• As a third approach, the full sample of nonparticipants (within the
common support) was used to construct a weighted match for each
participant, applying a nonparametric kernel regression method
proposed by Heckman, Ichimura, and Todd (1998).
• To evaluate the comparability of the participant and matched
nonparticipant samples across these three methods, Godtland and
others (2004) conducted balancing tests to see whether the means of
the observable variables for each group were significantly different.
CASE STUDY 1…
• For the first and second methods, the balancing test was performed by
dividing each comparison and treatment group into two strata,
ordered by probability propensity scores.
• Within each stratum, a t-test of equality of means across participants
and matched nonparticipants was conducted for each variable in the
farmer participation equation.
• Godtland and others found that the null was not rejected for all but a
few variables across the first two methods.
• For the third method, a test for the equality of means was conducted
across the samples of participants and their weighted matches. The
null was not rejected for all but two variables at the 10 percent level.
Overall, their results found no systematic differences in observed
characteristics across the participant and nonparticipant samples.
CASE STUDY 2: USE OF PSM AND TESTING FOR
SELECTION BIAS
• In their study of the Trabajar workfare program in Argentina, Jalan and
Ravallion (2003) conducted a post-intervention survey of both participants
and nonparticipants.
• The context made it more likely that both groups came from a similar
economic environment: 80 percent of Trabajar workers came from the
poorest 20 percent of the population, and the study used a sample of
about 2,800 Trabajar participants along with nonparticipants from a large
national survey.
• Kernel density estimation was used to match the sample of participants
and nonparticipants over common values of the propensity scores,
excluding nonparticipants for whom the estimated density was equal to
zero, as well as 2 percent of the nonparticipant sample from the top and
bottom of the distribution.
CASE STUDY 2…
• Estimates of the average treatment effect (ATE) based on the
nearest neighbor, the nearest five neighbors, and a kernel-weighted
matching were constructed, and average gains of about half the
maximum monthly Trabajar wage of US$200 were realized.
• Jalan and Ravallion (2003) also tested for potential remaining
selection bias on unobserved characteristics by applying the Sargan-
Wu-Hausman test.
• Specifically, on the sample of participants and matched
nonparticipants, they ran an ordinary least squares regression of
income on the propensity score, the residuals from the logit
participation equation, as well as a set of additional control
variables Z that exclude the instruments used to identify exogenous
variation in income gains.
CASE STUDY 2…
• In the study, the identifying instruments were provincial dummies,
because the allocations from the program varied substantially across
equally poor local areas but appeared to be correlated with the
province that the areas belonged to.
• This test was used to detect selection bias in the nearest-neighbor
estimates, where one participant was matched to one nonparticipant,
which lent itself to a comparable regression-based approach.
• If the coefficient on the residuals is significantly different from zero,
selection bias may continue to pose a problem in estimating the
program’s impact.
• In the analysis, this coefficient was not statistically significant under
the null hypothesis of no selection bias, and the coefficient on the
propensity score was similar to the average impact in the nearest-
neighbor matching estimate.