Propensity Score — Matching & Weighting

Dr. R. Düsing · Osnabrück University
People who opt into a program differ from non-participants — propensity scores make the two groups comparable.
Propensity Score Matching & Weighting — Help
The Problem: Selection Bias

In observational studies, people decide for themselves whether they receive a treatment T. This self-selection means the treatment and control groups differ in observable characteristics (covariates X) that also affect the outcome Y — this is called confounding. A naive comparison Y(T=1) − Y(T=0) does not estimate the causal effect; it mixes the effect with the baseline difference.

Example: first-year students with a lower high-school GPA and lower SES participate in a mentoring program more often. Compared naively, the program looks ineffective — not because it is, but because participants had worse starting conditions.

What Is a Propensity Score?

The propensity score (PS) is the conditional probability of treatment given all measured covariates:

e(X) = P(T = 1 | X₁, X₂, ..., Xₖ)

Rosenbaum & Rubin (1983): if treatment is conditionally independent of the potential outcomes given X (ignorability), it is also conditionally independent given the single scalar e(X). Instead of balancing k covariates simultaneously, it suffices to condition on the PS. In practice, the PS is estimated via logistic regression.

Assumptions — When Do PSM/IPW Work?

1. Ignorability (CIA / unconfoundedness): {Y(0), Y(1)} ⊥ T | X — conditional on the measured covariates, treatment is quasi-random. This assumption is not testable. PSM/IPW only eliminates bias from measured covariates; unmeasured confounders (omitted variable bias) remain a risk.

2. Common support (positivity): 0 < e(X) < 1 for all X. Every person must in principle be able to be treated or untreated. Violated if a characteristic perfectly predicts treatment.

3. SUTVA: no spillover between units; T is a clearly defined intervention.

Propensity Score Matching (PSM)

In 1:1 nearest-neighbor matching, every treated person is paired with the control person with the most similar PS. The caliper c limits the maximum distance |e_i − e_j| ≤ c. Recommendation: c = 0.2 · SD(logit(PS)) (Austin, 2011). Matching without replacement: every control person is used at most once.

Matching estimates the ATT (Average Treatment Effect on the Treated): the effect for those who were actually treated. Treated units without a match (outside common support) are excluded — this raises internal validity at the cost of sample size.

ATT (matching) = (1/M) · Σᵢ [ Yᵢ(T=1) − Y_match(i)(T=0) ]
Inverse Probability Weighting (IPW / IPTW)

Instead of matching people, every person gets a weight that reverses the selection mechanism. For the ATT:

Treated: w = 1 · Controls: w = e(X) / (1 − e(X))

Control persons with a high PS (who really should have been treated) get more weight — they are better counterfactual doubles. The result is a pseudo-randomized dataset. IPW is more efficient than matching (uses all data), but more sensitive to extreme PS values near 0 or 1, where weights can explode.

Checking Balance — Love Plot & SMD

The Love plot shows the Standardized Mean Difference (SMD) for each covariate before and after adjustment:

SMD = (μ_treated − μ_control) / √[(σ²_T + σ²_C) / 2]

|SMD| < 0.10 → good balance · |SMD| < 0.25 → acceptable · |SMD| ≥ 0.25 → substantial residual bias. A good Love plot shows all covariates moving close to 0 after adjustment. Balance ≠ no bias: only measured covariates are checked.

Which Method to Choose?

PSM: intuitive, transparent (individual pairs can be shown), conservative (common support only). Loss of control cases. Good when clear comparability of pairs matters.

IPW: more efficient (all data), flexible (ATE or ATT). Sensitive to extreme PS values. Good with a small sample or poor common support on the matching side.

Doubly robust (DR): combines outcome regression + IPW. Consistent if either the PS model or the outcome model is correct. State of the art in applied research (e.g. AIPW, TMLE).

Caution: all methods fail under unobserved confounders. Instrumental variables, regression discontinuity, or difference-in-differences are then more appropriate.

R Packages

MatchIt: flexible matching (nearest-neighbor, optimal, genetic, CEM). WeightIt: IPW and other weighting methods. cobalt: Love plot + balance statistics, compatible with MatchIt & WeightIt. twang: PS via gradient boosting (GBM). CBPS: covariate balancing PS — estimates the PS directly optimized for balance.

References

Rosenbaum, P. R. & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.

📋 Example — Mentoring Program
A university offers a voluntary mentoring program for first-year students. We want to know its effect on exam success. Problem: students with a lower high-school GPA (X₁) and lower SES (X₂) participate more often — the groups are not comparable.
①
The Problem — Selection Bias & Propensity Score
Treated and control units differ in baseline characteristics (X₁–X₃). The propensity score e(X) = P(T=1|X) condenses all covariates into a single scalar — and shows where the groups overlap (common support). Under IPW, the light-colored area shows the weighted pseudo-population: how the sample shifts due to the weights.
Computing…
②
The Correction — Covariate Balance
The Love plot shows the Standardized Mean Difference (SMD) for each covariate before and after adjustment. Goal: all |SMD| < 0.10 (green zone). A balanced Love plot means: the groups are comparable after adjustment.
Covariate SMD before SMD matching (ATT) SMD IPW
③
The Effect — Causal Estimation
The green line marks the true effect τ (known in the simulation model). How close do the estimators get? The naive comparison shows the bias — adjusted methods should approach the truth line.
Flashcards
What Is a Propensity Score?
e(X) = P(T=1|X) — the conditional treatment probability. Rosenbaum & Rubin (1983): if Y(0),Y(1) ⊥ T | X, then also Y(0),Y(1) ⊥ T | e(X). A powerful dimension reduction: k covariates → 1 scalar. Usually estimated via logistic regression.
Selection Bias & CIA
PSM/IPW requires ignorability (CIA): conditional on X, T is independent of Y(0),Y(1). This eliminates bias from measured covariates. Unmeasured confounders still cause bias — PSM/IPW is not a cure-all.
1:1 Nearest-Neighbor Matching
Every treated person is paired with the control person with the closest PS (without replacement). The caliper c limits the maximum distance. Recommendation: c = 0.2 · SD(logit(PS)). Without a caliper: guarantees a match, but possibly poor quality.
Common Support
Estimation is only valid in the overlap region of the PS distributions. Treated units with PS ≈ 1 have no similar controls — they get excluded during matching. Under IPW, these units get extreme weights (w → ∞) — clipping or stabilized weights are needed.
IPW Weights: ATE, ATT, ATU
Depending on whose perspective the target parameter takes, different groups are reweighted: ATT (wT=1, wC=e/(1−e)) holds the treated fixed and reshapes the controls into their counterpart. ATU does the reverse (wT=(1−e)/e, wC=1). ATE (wT=1/e, wC=1/(1−e)) reweights both groups to the overall population. Different weights → different pseudo-populations (see plot above). When ATE, ATT, and ATU also differ numerically (heterogeneous effects, HTEB) and how to interpret them is covered in detail in Causal Inference — Foundations.
Love Plot & SMD
SMD = (μ_T − μ_C) / SD_pooled. |SMD| < 0.10 → good balance · < 0.25 → acceptable. The Love plot shows balance for all covariates simultaneously — before and after adjustment. Balance only applies to measured variables.