Difference-in-Differences

Dr. R. Düsing · Osnabrück University
Difference-in-Differences — Help
Example

A school district introduces a support program (daily tutoring) starting in school year 2, in some of its schools — the treatment group. The remaining schools serve as the control group. Outcome Y: standardized achievement test. Goal: how large is the causal effect of the program?

A naive before-after comparison fails: students would have improved even without the program (maturation, general curriculum effects). DiD solves the problem with a control group — it shows how much students would have developed without the intervention.

Regression Model & Why β₃ = DiD

The DiD model is:

Y = α + β₁·Post + β₂·Treat + β₃·(Post × Treat) + ε

The four expected values from it:

E[Y | control, before] = α
E[Y | control, after] = α + β₁ → Δ_C = β₁
E[Y | treatment, before]= α + β₂
E[Y | treatment, after] = α + β₁ + β₂ + β₃ → Δ_T = β₁ + β₃

Difference of differences: DiD = Δ_T − Δ_C = (β₁ + β₃) − β₁ = β₃

β₃ is the coefficient of the interaction term (Post × Treat). It measures exactly the additional change in the treatment group beyond the general time trend β₁. The selection bias β₂ (time-constant group difference) cancels out completely in the double difference — that is the central strength of DiD.

What DiD Estimates — the ATT

The DiD estimator β̂₃ identifies the Average Treatment Effect on the Treated (ATT) — the causal effect of the program for exactly the units that were actually treated (here: the treatment schools).

ATT ≠ ATE (the average treatment effect across all units): if treatment schools differ systematically from control schools (selection bias, β₂ ≠ 0), the effect may not generalize to all schools — but it is still valid for the treated schools themselves. That is both the strength and the limitation of DiD.

Counterfactual Scenario — What Would Have Happened Otherwise?

A counterfactual scenario answers a hypothetical question: how would the treatment group have developed if it had not been treated? By definition this situation does not exist in the data — every school was either in the program or not.

In the DiD plot, the dashed green line is the counterfactual scenario: it shows how the treatment schools would presumably have developed — namely parallel to the control group. This line starts at the same point as the treatment group and implies where it would have ended up had it changed exactly like the control group. The DiD effect β̂₃ is exactly the gap between the actually observed after-value of the treatment group and this counterfactual value.

The parallel-trends argument makes the counterfactual assumption explicit: without intervention, both groups would have changed by the same amount. If that's plausible, the counterfactual is meaningfully reconstructed — and β̂₃ is causally interpretable.

→ The Causal Teaching tool shows counterfactual potential outcomes in the Rubin framework (Y(0), Y(1)) and explains why you can never observe both at once — the fundamental problem of causal inference.

Selection Bias — and Why DiD Is Robust

β₂ measures the time-constant difference between the treatment and control groups before the intervention. If, say, schools with more motivated students join the program (self-selection), their baseline values differ (β₂ ≠ 0).

DiD corrects this automatically: β₂ cancels out in the double difference — even with a large β₂, β̂₃ remains unbiased. The "selection bias" scenario and the combination scenario illustrate this.

Naive Comparisons — Why They Fail

In the combination scenario (β₁ = 1.5, β₂ = 1.5, β₃ = 3), you can see why two seemingly naive comparisons are nonetheless wrong:

① G1t1 − G0t1 (cross-section at the after time point): overestimates the ATT because the group difference β₂ (selection, time-constant) is included.

② G1t1 − G1t0 (before-after within the treatment group only): overestimates the ATT because the general time trend β₁ is included — the group would have improved even without the program.

DiD corrects both biases simultaneously: β₂ cancels out in the double difference, and so does β₁. Both orange arrows ① ② in the plot show how far off the naive comparisons are.

Parallel-Trends Assumption

DiD requires that, without intervention, both groups would have changed by the same amount. This assumption is not directly testable, but can be indirectly assessed via pre-period comparisons (placebo tests). The PT-violation slider allows manual exploration of this theoretical threat.

Limitations & Robustness

Anticipation effects: if the treatment group already changes its behavior before the program starts, the before-measurement is contaminated.
Spillover: if the control group indirectly benefits from the intervention, β₃ is underestimated.
Event study: in practice, DiD is estimated for several pre- and post-periods to check parallel trends and reveal dynamic effects.

References

Angrist, J. D. & Pischke, J.-S. (2009). Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press.

Running Example
PostTime point: 0 = school year before the program, 1 = school year after TreatGroup: 0 = control schools, 1 = treatment schools (support program) YOutcome: standardized achievement test (z-standardized) β₁General time trend — how much do students improve anyway? β₂Time-constant group difference (selection bias) — corrected by DiD β₃Interaction term Post×Treat — exactly the DiD estimator, causally interpretable
What causal effect does the support program have — and how does DiD separate this effect from general learning gains and selection differences?
DiD Plot — Before / After
Why is β₃ exactly the DiD estimator? Y = α + β₁·Post + β₂·Treat + β₃·(Post×Treat) + ε Δ_control = β₁  ·  Δ_treatment = β₁ + β₃  ·  DiD = Δ_T − Δ_C = β₃
β₁ (time trend) and β₂ (selection) cancel out in the double difference. β₃ = interaction term = DiD = ATT — the causal effect for the treated units.
2 × 2 — Cell Means
GroupBefore (Post=0)After (Post=1)Δ (After − Before)
Control group — — —
Treatment group — — —
DiD = Δ_T − Δ_C = β̂₃ — — —
Regression Output — OLS
CoefficientTrue valueβ̂SEtp
α · Intercept 0.00 — — — —
β₁ · Post — — — — —
β₂ · Treat — — — — —
β₃ · Post×Treat (DiD) — — — — —
Concepts
What Is Difference-in-Differences?
DiD compares the change (before → after) in the treatment group with the change in the control group. The double difference isolates the causal effect: Y = β₀ + β₁·Post + β₂·Treat + β₃·(Post×Treat). The interaction coefficient β₃ is the DiD estimator.
Why the Double Difference Works
In the double difference, the general time trend β₁ (both groups improve anyway) and the time-constant group difference β₂ (selection) cancel out. What's left is only β₃ — the part of the change that occurred additionally in the treatment group.
What DiD Estimates: the ATT
β₃ identifies the Average Treatment Effect on the Treated (ATT) — the causal effect for exactly the treated units. This is not necessarily the ATE (the effect across all units): if the treatment and control groups differ systematically (β₂ ≠ 0), the effect is valid for the treated, but not necessarily generalizable.
Parallel-Trends Assumption
The central identification assumption: without treatment, both groups would have changed by the same amount. It is counterfactual and not directly testable — but can be made plausible via several pre-periods (event study). If violated, β₃ contains a bias and is no longer causally interpretable.
Why Naive Comparisons Fail
① Cross-section, after only (treatment − control): overestimates, because the selection difference β₂ is included. ② Before-after within the treatment group only: overestimates, because the general time trend β₁ is included — the group would have improved even without the program. DiD corrects both biases at once.
Context & Related Tools
DiD is a quasi-experimental method for panel data: it uses a control group as a counterfactual, instead of adjusting for variables. Related: the confounding/backdoor thinking of causal inference and other designs without randomization. → Causal Inference · → Regression Discontinuity · → Propensity Score Matching
The Same Thing as a Moderation
Mathematically, DiD is nothing other than a moderation analysis: the model Y = β₀ + β₁·Post + β₂·Treat + β₃·(Post×Treat) is exactly an interaction model Y = b₀ + b₁·X + b₂·W + b₃·(X·W) with two binary predictors. Treat moderates the effect of time (and vice versa), and β₃ is the interaction = moderation coefficient. The two "differences" are the simple slopes of the two groups, and their difference is the DiD estimator. It is the same OLS regression — only the framing differs: econometrics calls it DiD, psychology calls it moderation. Once you see it, you never have to think of them separately again. → Moderation Analysis