Lesson 11.3: Staggered Adoption: When TWFE Fails, and What to Use Instead

Written lesson

Staggered roll-outs

Policies rarely start everywhere at once. Minimum wage increases, castle-doctrine laws and broadband programmes arrive in different places in different years. TWFE with a single treatment dummy was the standard tool for such staggered designs until research around 2018–2021 (Goodman-Bacon; de Chaisemartin and D'Haultfœuille; Callaway and Sant'Anna; Sun and Abraham; Borusyak, Jaravel and Spiess) showed it can fail badly.

The problem: TWFE's δ is a weighted average of all possible 2×2 comparisons, including forbidden ones where units treated earlier serve as controls for units treated later. If effects grow over time, the early group's outcomes are still changing because of the treatment, which contaminates the comparison. Some weights can even be negative.

Python · Colab cell

!pip install -q pyfixest numpy pandas matplotlib causaldata
import numpy as np, pandas as pd
import pyfixest as pf
import matplotlib.pyplot as plt
rng = np.random.default_rng(3)

A simulation where TWFE gets the sign wrong

Three cohorts of municipalities adopt a wage subsidy in periods 2, 5 and 8 (and none are never-treated). The effect is positive and grows by +1 each period after adoption.

Python · Colab cell

def staggered(never_treated=0, units_per_cohort=30, T=12, cohorts=(2, 5, 8)):
    rows = []
    gs = [g for g in cohorts for _ in range(units_per_cohort)] + [0] * never_treated
    for i, g in enumerate(gs):
        a = rng.normal(0, 1)
        for t in range(T):
            treated = int(g > 0 and t >= g)
            tau = (t - g + 1) * 1.0 if treated else 0.0           # effect grows with time since adoption
            rows.append((i, t, g, treated, tau, a + 0.3 * t + tau + rng.normal(0, 0.5)))
    return pd.DataFrame(rows, columns=["unit", "t", "g", "D", "tau", "y"])

sim = staggered()
twfe = pf.feols("y ~ D | unit + t", data=sim, vcov={"CRV1": "unit"})
print(f"True average effect on the treated: {sim.tau[sim.D == 1].mean():.2f}")
print(f"TWFE estimate:                      {twfe.coef()['D']:.2f}")

Clean comparisons: cohort-by-cohort DiD

The fix is to only make clean comparisons: compare each adoption cohort with units that are not yet treated (or never treated), period by period. Callaway and Sant'Anna's group-time effects ATT(g, t) do exactly that and then aggregate. Here is the logic written out by hand with not-yet-treated controls:

Python · Colab cell

def att_gt(data):
    out = []
    for g in sorted(data.g[data.g > 0].unique()):
        for t in range(int(g), data.t.max() + 1):
            base = g - 1                                              # last pre-period of cohort g
            clean = data[(data.g == 0) | (data.g > t)]                # never or not-yet treated at t
            if clean.unit.nunique() == 0:
                continue
            ch_g = data[(data.g == g) & (data.t == t)].y.mean() - data[(data.g == g) & (data.t == base)].y.mean()
            ch_c = clean[clean.t == t].y.mean() - clean[clean.t == base].y.mean()
            out.append((g, t, t - g, ch_g - ch_c, (data.g == g).sum() / data.t.nunique()))
    return pd.DataFrame(out, columns=["g", "t", "e", "att", "n"])

gt = att_gt(sim)
overall = np.average(gt.att, weights=gt.n)
print(gt.round(2).head(8))
print(f"Aggregated ATT (clean comparisons): {overall:.2f} | truth over the same cells: "
      f"{np.mean([(t - g + 1) for g, t in zip(gt.g, gt.t)]):.2f}")

Only cells with a valid control group exist: once every cohort is treated, nothing is left to compare with.

Gardner's two-stage DiD (did2s, also Borusyak et al.'s imputation estimator) is another robust approach: estimate unit and time effects from untreated observations only, impute the counterfactual for treated ones, and average the gaps.

Python · Colab cell

sim_nt = staggered(never_treated=30)
two_stage = pf.did2s(sim_nt, yname="y", first_stage="~ 0 | unit + t", second_stage="~ D",
                     treatment="D", cluster="unit")
twfe_nt = pf.feols("y ~ D | unit + t", data=sim_nt, vcov={"CRV1": "unit"})
print(f"Truth {sim_nt.tau[sim_nt.D == 1].mean():.2f} | TWFE {twfe_nt.coef()['D']:.2f} | did2s {two_stage.coef()['D']:.2f}")

Real data: castle-doctrine laws and homicides

Between 2005 and 2010, 21 US states passed castle-doctrine ("stand your ground") laws expanding the right to use lethal force in self-defence. Cheng and Hoekstra (2013) asked whether they deterred crime or increased homicides. The castle data cover 50 states, 2000–2010; l_homicide is log homicides per capita.

Python · Colab cell

from causaldata import castle
c = castle.load_pandas().data[["sid", "year", "post", "l_homicide"]].copy()
c["year"] = c.year.astype("int64")
c["D"] = (c.post > 0).astype(int)
c["g"] = c.sid.map(c[c.D == 1].groupby("sid").year.min()).fillna(0).astype(int)
print(c.drop_duplicates("sid").g.value_counts().sort_index().rename("states by adoption year"))

twfe_c = pf.feols("l_homicide ~ D | sid + year", data=c, vcov={"CRV1": "sid"})
did2s_c = pf.did2s(c, yname="l_homicide", first_stage="~ 0 | sid + year", second_stage="~ D",
                   treatment="D", cluster="sid")
for name, m in [("TWFE", twfe_c), ("did2s", did2s_c)]:
    print(f"{name:6}: {m.coef()['D']:.3f} (SE {m.se()['D']:.3f})  ->  about {100 * (np.exp(m.coef()['D']) - 1):.0f}% more homicides")

Both estimators point to roughly 7% more homicides after adoption, similar in size to Cheng and Hoekstra's 8%, though less precise here because we do not weight by state population as they did. Here TWFE and the robust estimator agree, which is reassuring: many states never adopted, so clean comparisons dominate. The robust estimator is what a referee will ask for.

Exercise

Build a robust event study for the castle data: compute ATT(g, t) by hand with att_gt (using l_homicide as y, year as t), then average by event time e. Plot it. Are the effects stable after adoption?

Show a solution

Python · Colab cell

cc = c.rename(columns={"sid": "unit", "year": "t", "l_homicide": "y"})
cc["t"] = cc.t - cc.t.min(); cc["g"] = np.where(cc.g > 0, cc.g - c.year.min(), 0)
es = att_gt(cc).groupby("e").apply(lambda d: np.average(d.att, weights=d.n), include_groups=False)
plt.plot(es.index, es.values, marker="o"); plt.axhline(0, color="grey")
plt.xlabel("Years since adoption"); plt.ylabel("ATT (log homicides)"); plt.title("Castle doctrine: robust event study")
plt.show()
print(es.round(3))

Checklist and recap

I can explain why TWFE fails with staggered adoption and growing effects.
I can compute clean cohort-by-time DiD estimates and aggregate them.
I use a robust estimator (did2s, Callaway–Sant'Anna, Sun–Abraham) for staggered designs.