6 Difference-in-Differences
Difference-in-differences (DiD) is one of the most widely used research designs in impact evaluation. It is useful when a policy affects one group of units but not another, and when we observe both groups before and after the treatment.
The basic idea is simple. We compare the change in outcomes for the treated group with the change in outcomes for a comparison group over the same period. The first difference removes time-invariant differences between treated and control units. The second difference removes shocks that are common to both groups.
6.1 The Evaluation Problem
Suppose there are two groups: a treated group and a control group. We observe both groups in two periods:
- a pre-treatment period, before the policy starts;
- a post-treatment period, after the policy starts.
Let \(D_i=1\) for units in the treated group and \(D_i=0\) for units in the control group. Let \(Post_t=1\) in the post-treatment period and \(Post_t=0\) in the pre-treatment period.
The target parameter is usually the ATT in the post-treatment period:
\[ ATT = E[Y_{i,post}(1)-Y_{i,post}(0)\mid D_i=1]. \]
The problem is familiar: for treated units after the policy, we observe \(Y_{i,post}(1)\), but we do not observe \(Y_{i,post}(0)\). We need to estimate what would have happened to the treated group in the absence of treatment.
DiD uses the change observed in the control group as an estimate of the counterfactual change that the treated group would have experienced without treatment.
DiD does not require treated and control units to have the same outcome levels before treatment. It requires them to have comparable trends in the absence of treatment.
6.2 Historical Intuition
An early example of DiD logic can be found in John Snow’s analysis of cholera in nineteenth-century London (Snow 1855). At the time, cholera was often explained by the dominant miasma theory, according to which disease spread through “bad air.” Snow instead argued that cholera was transmitted through contaminated water, but he needed empirical evidence to support his theory.
The empirical setting was unusually informative. In parts of London, households were served by different water companies. Before the relevant change, both the Lambeth Company and the Southwark & Vauxhall Company drew water from polluted sections of the River Thames. Later, Lambeth moved its water intake upstream,1 to a cleaner part of the river, while Southwark & Vauxhall continued to draw water from a more contaminated downstream source.
1 Lambeth had moved its water intake upstream in 1852 to improve water quality, relocating it beyond the main influence of London’s sewage. This change provided the crucial variation Snow exploited to test the waterborne-transmission hypothesis.
This created a comparison across two dimensions: before versus after the change in water supply, and households served by Lambeth versus households served by Southwark & Vauxhall. If cholera mortality declined more among households served by Lambeth than among households served by Southwark & Vauxhall, this difference could be interpreted as evidence that cleaner water reduced cholera mortality.
The logic is that of the traditional DiD design. The Southwark & Vauxhall households provide a comparison group, because they remained exposed to polluted water. The Lambeth households provide the treated group, because their water source changed. The key comparison is not simply whether cholera mortality was lower among Lambeth households after the change. Instead, the question is whether mortality declined more for Lambeth households than for households whose water supply did not change.
In economics, classic applications include Card’s study of the Mariel Boatlift and the Miami labor market (Card 1990), and Card and Krueger’s study of the employment effects of the 1992 minimum wage increase in New Jersey relative to Pennsylvania (Card and Krueger 1994).
6.3 The Basic DiD Estimator
With two groups and two periods, the DiD estimator is
\[ \widehat{\tau}_{\text{DiD}} = \left(\bar{Y}_{1,\text{post}}-\bar{Y}_{1,\text{pre}}\right) - \left(\bar{Y}_{0,\text{post}}-\bar{Y}_{0,\text{pre}}\right). \]
where \(\bar{Y}_{1,post}\) is the average post-treatment outcome for the treated group, \(\bar{Y}_{1,pre}\) is the average pre-treatment outcome for the treated group, and similarly for the control group.
This can also be written as
\[ \widehat{\tau}_{\text{DiD}} = \left(\bar{Y}_{1,post}-\bar{Y}_{0,post}\right) - \left(\bar{Y}_{1,pre}-\bar{Y}_{0,pre}\right). \]
These two expressions are algebraically equivalent. The first emphasizes changes over time within each group. The second emphasizes how the treated-control gap changes after treatment.
6.4 Parallel Trends
The central identifying assumption is the parallel-trends assumption. It states that, in the absence of treatment, the treated and control groups would have experienced the same average change in outcomes.
Formally,
\[ E[Y_{i,post}(0)-Y_{i,pre}(0)\mid D_i=1] = E[Y_{i,post}(0)-Y_{i,pre}(0)\mid D_i=0]. \]
This assumption allows the control-group trend to stand in for the missing untreated trend of the treated group.
Under parallel trends,
\[ E[Y_{i,post}(0)\mid D_i=1] = E[Y_{i,pre}(0)\mid D_i=1] + E[Y_{i,post}(0)-Y_{i,pre}(0)\mid D_i=0]. \]
In words, the counterfactual post-treatment outcome for the treated group is obtained by taking its pre-treatment outcome and adding the change observed in the control group.
The assumption is not that treated and control groups have the same outcome levels. The assumption is that, without treatment, their outcomes would have moved in parallel.
6.5 Regression Form
The two-period DiD estimator can also be obtained from the following regression:
\[ Y_{it} = \alpha + \beta Treat_i + \gamma Post_t + \tau_{\text{DiD}} (Treat_i \times Post_t) + \varepsilon_{it}. \]
The coefficient \(\tau_{\text{DiD}}\) provides the ATT estimate. In the simple two-group, two-period case, this regression coefficient is numerically equivalent to the traditional DiD formula:
\[ \widehat{\tau}_{DiD} = \left(\bar{Y}_{T,post}-\bar{Y}_{T,pre}\right) - \left(\bar{Y}_{C,post}-\bar{Y}_{C,pre}\right). \]
The regression terms have a simple interpretation:
- \(\alpha\) captures the average outcome of the control group before the policy;
- \(\beta\) captures baseline differences between treated and control units;
- \(\gamma\) captures the common change from the pre-treatment to the post-treatment period;
- \(\tau_{\text{DiD}}\) captures the additional change experienced by the treated group after the policy, namely the ATT.
Why use a regression if the coefficient is equivalent to a simple difference between four means? The main reason is that the regression framework also provides an estimate of uncertainty around the DiD estimate. It allows us to compute standard errors and confidence intervals for \(\widehat{\tau}_{\text{DiD}}\), and to test hypotheses about the underlying causal parameter \(\tau_{\text{DiD}}\).
Moreover, the regression formulation is easy to extend as shown in the following section.
6.6 Covariates and Conditional Parallel Trends
The DiD regression model can be estimated with additional covariates. Covariates may improve precision and help make the parallel-trends assumption more credible.
The relevant assumption then becomes conditional parallel trends: after conditioning on pre-treatment characteristics, the treated and control groups would have followed the same average trend in \(Y\) in the absence of treatment.
Covariates should be handled with care. They should typically be measured before treatment or otherwise be unaffected by treatment. Including post-treatment variables can introduce severe sources of bias.
Covariates in DiD are most convincing when they measure pre-treatment characteristics that help explain different outcome trends.
6.7 Functional Form Limitation
Parallel trends can depend on the scale of the outcome. Parallel trends in levels do not automatically imply parallel trends in logs, and parallel trends in logs do not automatically imply parallel trends in levels.
For example, suppose the outcome is wages. If wages grow by the same absolute amount in treated and control groups, trends may be parallel in levels. If wages grow by the same percentage amount, trends may be parallel in logs. These are different assumptions and imply different causal estimands.
This is why the choice between levels, logs, rates, and other transformations should be substantive rather than mechanical, and ideally made before examining the results to avoid cherry-picking the specification that produces the most favorable estimate.
6.8 Diagnostics and Validity Checks
The parallel-trends assumption is fundamentally about an unobserved counterfactual, so it cannot be tested directly. However, if at least two pre-treatment periods are available, its plausibility can be assessed indirectly by examining whether treated and control units followed similar trends before the treatment. If treated and control groups move very differently before treatment, the parallel-trends assumption becomes less credible.
With several pre-treatment periods, researchers can estimate placebo DiD effects using periods before the treatment actually occurred. If the method finds large pre-treatment placebo estimates, this suggests that the design may be capturing pre-existing differences in trends rather than the causal effect of the treatment.
Other useful checks include:
- plotting average outcomes for treated and control groups over time, rather than only the differences between them;
- checking sensitivity to alternative comparison groups;
- checking whether the estimated effect changes when adding relevant pre-treatment covariates;
- checking whether other policies or shocks occurred at the same time.
Similar pre-treatment trends support the credibility of a DiD design, but they do not prove that counterfactual trends would have remained parallel after treatment, since this assumption is not fully testable. Conversely, diverging pre-treatment trends should be viewed as a strong warning sign.
6.9 Worked Example
Suppose a public subsidy for workplace safety is introduced in one region but not in a neighboring region. The policy starts in 2024 and aims to reduce workplace accidents.
A DiD evaluation would compare:
- the change in accident rates in the treated region before and after 2024;
- the change in accident rates in the control region over the same period.
The estimator is
\[ \widehat{DiD} = \left(\bar{Y}_{treated,after}-\bar{Y}_{treated,before}\right) - \left(\bar{Y}_{control,after}-\bar{Y}_{control,before}\right). \]
If workplace accidents fall by 5 per 1,000 workers in the treated region and by 1 per 1,000 workers in the control region, the DiD estimate is
\[ -5 - (-1) = -4. \]
The interpretation is that the subsidy reduced accidents by 4 per 1,000 workers, provided that the treated and control regions would have followed parallel trends in the absence of the policy.
A note of caution. This interpretation would be less credible if accidents were already declining faster in the treated region before the policy, or if another safety regulation was introduced only in the treated region at the same time.
6.10 Ashenfelter’s Dip
A classic threat to DiD is Ashenfelter’s dip (Ashenfelter 1978). This occurs when units select into treatment after experiencing a temporary negative shock.
Suppose workers enter a training program after a sudden fall in earnings. Even without the program, their earnings might partially or fully recover. A DiD estimator comparing these workers with non-participants may mistakenly attribute the natural rebound to the training program.
The problem is that treatment timing is related to a temporary shock in the untreated potential outcome. A simple DiD design differences out time-invariant unobserved heterogeneity, not temporary shocks that affect treatment participation.
Longer panels can help diagnose the problem. If outcomes fall sharply just before treatment and then recover, the design may be capturing mean reversion rather than a causal effect.
6.11 A DiD extension: The Triple Differences Estimator
The triple difference estimator, or DDD, extends DiD by adding a third comparison dimension (Olden and Møen 2022).
Suppose a parental leave reform is introduced in some states but not others, and applies mainly to mothers rather than fathers. A DiD comparing mothers in treated and control states may be biased if mothers in treated states were already on a different trend. A second DiD for fathers can help capture state-specific changes that are not specific to mothers.
The triple difference is the difference between two DiD estimators:
\[ DDD = DiD_{mothers} - DiD_{fathers}. \]
The intuition is that if the bias is common to both comparisons, subtracting the second DiD removes it.
Triple differences do not eliminate the need for identifying assumptions. Instead, they require a parallel-trends assumption for the difference between two DiD comparisons: in the absence of the policy, the gap between these two DiD effects would have remained unchanged over time. Under this assumption, which is not fully testable, any post-treatment change in this gap can be attributed to the policy-specific effect of interest.
6.12 Matching Difference-in-Differences
Matching DiD combines the logic of matching with the logic of DiD. It is useful when treated and control units differ in observed characteristics, but panel or repeated cross-sectional data are available.2
2 Repeated cross-sectional data consist of separate samples drawn from the same population at different points in time. Unlike panel data, the same units do not need to be observed in every period.
The matching step makes treated and control units more comparable in terms of observed pre-treatment covariates. The DiD step then differences out time-invariant unobserved heterogeneity.
A key contribution to this literature is Heckman, Ichimura, and Todd (Heckman et al. 1997).
For the two-period case, a matching DiD estimator for the ATT can be written as
\[ \widehat{ATT}_{MDID} = \frac{1}{N_T} \sum_{i:D_i=1} \left[ (Y_{i,post}-Y_{i,pre}) - \sum_{j:D_j=0}w_{ij}(Y_{j,post}-Y_{j,pre}) \right], \]
where \(w_{ij}\) is the weight given to control unit \(j\) when constructing the counterfactual trend for treated unit \(i\).
Matching DiD relies on two main conditions:
- conditional parallel trends: after conditioning on covariates, treated and matched control units would have followed the same untreated trend;
- common support: for each treated unit, at least 1 comparable untreated unit must exist.
Importantly, matching is not automatically an improvement. If the matching variables are poorly chosen, if they are affected by treatment anticipation, or if matching discards many informative control units, the resulting DiD estimate may become less precise or even more biased. Matching can also worsen balance on pre-treatment outcome trends if it focuses only on baseline covariates. For this reason, matching DiD should be assessed empirically by checking common support, covariate balance, and especially the similarity of pre-treatment trends after matching.
Matching DiD controls for observed differences through matching and for time-invariant unobserved differences through differencing.
6.13 TWFE With Staggered Adoption
In the canonical DiD setting, there are two groups and two periods: one group becomes treated, the other remains untreated, and the DiD estimand compares the change in outcomes for the treated group with the change in outcomes for the control group. In such a simple setting, the DiD parameter has a transparent interpretation: it estimates the ATT under the parallel-trends assumption.
However, many empirical applications are more complex. Units may adopt the treatment at different times: some are treated early, others later, and some may never be treated. This setting is known as staggered adoption. In this case, the causal effects of interest are often group-time average treatment effects, usually denoted as \(ATT(g,t)\). These parameters measure the average effect of the treatment at time \(t\) for the group of units first treated in period \(g\).
For example, \(ATT(g,t)\) asks: among units first treated in period \(g\), what is the average difference at time \(t\) between their observed outcome under treatment and the outcome they would have experienced if they had not yet been treated?
For many years, the standard empirical approach in staggered-adoption settings was to estimate a two-way fixed effects (TWFE) model:
\[ Y_{it} = \alpha_i + \lambda_t + \tau D_{it} + X_{it}'\beta + u_{it}. \]
Here, \(\alpha_i\) are unit fixed effects, which absorb time-invariant differences across units. The terms \(\lambda_t\) are time fixed effects, which absorb shocks common to all units in a given period.3 The variable \(D_{it}\) indicates whether unit \(i\) is treated at time \(t\). The coefficient \(\tau\) is often interpreted as the ATT.
3 In practice, fixed effects can be thought of as a set of dummy variables. Unit fixed effects include a separate indicator for each unit, while time fixed effects include a separate indicator for each period. These dummies absorb persistent differences across units and common differences across periods.
Wolfers (2006) studies whether the staggered adoption of unilateral divorce laws raised divorce rates in the United States. Between 1968 and 1988, 29 states changed their divorce laws at different times. A natural static TWFE specification is
\[ DivorceRate_{st} = \alpha_s + \lambda_t + \tau Unilateral_{st} + u_{st}, \]
where state fixed effects \(\alpha_s\) absorb permanent differences across states, year fixed effects \(\lambda_t\) absorb common national shocks, and \(Unilateral_{st}\) indicates whether unilateral divorce was available in state \(s\) and year \(t\). The coefficient \(\tau\) compresses the effects observed at all post-reform horizons into a single estimate.
This restriction is consequential because the effect was not constant over time. In Wolfers’s dynamic specification, divorce rates increased by approximately 0.27 divorces per 1,000 people during the first two years after reform and by 0.22 during years three and four. The estimated increase then became smaller, and by years nine and ten the estimate was approximately -0.10, providing no evidence of a persistent positive effect.
However, in staggered-adoption designs, \(\tau\) does not generally estimate a single traditional DiD effect. It is instead a weighted average of many underlying comparisons across groups and time periods. Some of these comparisons use not-yet-treated units as controls, which is often natural. Others use already-treated units as controls for newly treated units. These comparisons can be problematic when treatment effects are heterogeneous, and especially when they evolve over time after treatment, because already-treated units are themselves experiencing treatment effects while serving as controls.
With staggered treatment timing and heterogeneous treatment effects, a TWFE coefficient may not estimate the policy-relevant ATT.
Therefore, the TWFE coefficient \(\tau\) should not be interpreted mechanically as the DiD estimand from the simple two-group, two-period design. When treatment effects are heterogeneous across groups or over time, \(\tau\) may combine different \(ATT(g,t)\) parameters with non-transparent weights, and in some cases even with negative weights. This can make the estimate difficult to interpret and potentially misleading (Chaisemartin and D’Haultfœuille 2020; Goodman-Bacon 2021; Callaway and Sant’Anna 2021; Sun and Abraham 2021; Imai and Kim 2021).
In staggered-adoption DiD designs, not all comparisons are equally credible. A particularly important problem arises when already-treated units are used as controls for units that are treated later.
Suppose one group is treated in period \(g\) and another group is treated in a later period \(g'\). Before period \(g'\), the later-treated group can still serve as a comparison group for the earlier-treated group, because it has not yet received the treatment. This is a not-yet-treated comparison and is often useful.
The problem appears after both groups have been treated. Once the earlier-treated group has already been exposed to the policy, its outcomes may include treatment effects. Using this group as a control for the later-treated group means comparing treated units with other treated units, rather than with units that remain untreated throughout the analyzed period. These comparisons are often called forbidden comparisons. They are problematic because they may subtract part of the treatment effect from the estimate, especially when treatment effects grow, fade, or otherwise evolve over time. In that case, the comparison no longer isolates the untreated potential outcome trend.
In recent years, several estimators have been proposed as alternatives to the standard TWFE model, especially in settings with staggered treatment adoption and heterogeneous treatment effects. These approaches are often discussed within the broader event-study literature, because they estimate how treatment effects evolve before and after treatment adoption while avoiding some of the problematic comparisons embedded in TWFE regressions. We will discuss these methods in (Chapter 7.)
6.14 Summary
DiD estimates the ATT by comparing changes over time in treated and control groups.
Its main strength is that it removes time-invariant differences between groups and common shocks affecting both groups over time. Its main weakness is that it relies on the parallel-trends assumption, which cannot be tested directly: in the absence of treatment, treated and control groups should have followed the same average trend.
In the simple two-group, two-period case, DiD is transparent and intuitive. In richer panel settings with staggered adoption and heterogeneous treatment effects, researchers must be more careful, as detailed in the next chapter.
Basic DiD can be implemented using standard regression tools. In the simplest two-group, two-period case, this usually means estimating a linear regression with a treatment indicator, a post-treatment indicator, and their interaction.
- R
- Python
statsmodels: a general statistical modeling library that can estimate DiD regressions using OLS and formula syntax.linearmodels: a package for panel-data models, including fixed-effects regressions useful for standard DiD applications.
- Stata