Two bad comparisons make one good one
When a province raises its minimum wage, two simple comparisons are tempting and both are wrong:
- Before vs after in the treated province mixes the policy with everything else that changed over time (the business cycle, Lesson 1.3).
- Treated vs control province after the policy mixes the policy with permanent differences between provinces.
Difference-in-differences subtracts one from the other: the change in the treated group minus the change in the control group. The control group's change stands in for what would have happened to the treated group without the policy.
!pip install -q numpy pandas statsmodels matplotlib seaborn
import numpy as np, pandas as pd
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt, seaborn as sns
rng = np.random.default_rng(42)
sns.set_theme(style="whitegrid")
A provincial minimum wage increase
50 municipalities observed over 10 months. The 25 in the treated province face a minimum wage increase from month 5. The two provinces have different employment levels, and there is a common downturn, but the trends would have been parallel without the policy. True effect on the youth employment rate: −0.05.
N_UNITS, N_TIME, T0 = 50, 10, 5
muni = np.repeat(np.arange(N_UNITS), N_TIME)
time = np.tile(np.arange(N_TIME), N_UNITS)
treated = (muni >= 25).astype(int)
post = (time >= T0).astype(int)
muni_level = rng.normal(0.60, 0.05, N_UNITS)[muni] + 0.05 * treated # treated province has higher employment
common_shock = (-0.01 * time) # a shared downturn
employment = muni_level + common_shock - 0.05 * treated * post + rng.normal(0, 0.02, len(muni))
df = pd.DataFrame({"muni": muni, "time": time, "treated": treated, "post": post, "employment": employment})
means = df.groupby(["treated", "post"]).employment.mean().unstack()
print(means.round(4))
before_after = means.loc[1, 1] - means.loc[1, 0]
treat_control = means.loc[1, 1] - means.loc[0, 1]
did = (means.loc[1, 1] - means.loc[1, 0]) - (means.loc[0, 1] - means.loc[0, 0])
print(f"Before/after (treated only): {before_after:+.4f}")
print(f"Treated vs control (post): {treat_control:+.4f}")
print(f"Difference-in-differences: {did:+.4f} (truth -0.05)")
DiD as a regression
The same number comes from a regression with the interaction of treated and post:
employment = β₀ + β₁·treated + β₂·post + δ·treated×post + ε
δ is the DiD estimate. The regression form lets us add covariates and compute standard errors. Because the policy varies at the province level and municipalities are observed repeatedly, cluster by municipality at least (Lesson 3.3), and be aware that with only two provinces true policy-level inference is fragile.
fit = smf.ols("employment ~ treated * post", data=df).fit(cov_type="cluster", cov_kwds={"groups": df["muni"]})
print(fit.summary().tables[1])
Parallel trends: what we assume and what we can check
The identifying assumption is parallel trends: without the policy, the treated group's outcome would have moved like the control group's. It is about counterfactual trends after treatment, so it cannot be tested directly. What we can check is whether trends were parallel before treatment.
trend = df.groupby(["time", "treated"]).employment.mean().unstack()
plt.figure(figsize=(10, 5))
plt.plot(trend.index, trend[0], marker="o", label="Control province")
plt.plot(trend.index, trend[1], marker="o", label="Treated province")
counterfactual = trend[1].iloc[:T0].tolist() + (trend[1].iloc[T0 - 1] + (trend[0].iloc[T0:] - trend[0].iloc[T0 - 1])).tolist()
plt.plot(trend.index, counterfactual, ls=":", color="grey", label="Treated counterfactual (parallel trend)")
plt.axvline(T0 - 0.5, color="red", ls="--", label="Policy")
plt.xlabel("Month"); plt.ylabel("Employment rate"); plt.legend(); plt.title("Difference-in-differences"); plt.show()
Parallel pre-trends are supporting evidence, not proof. Trends can diverge exactly when the policy starts for other reasons (Ashenfelter's dip: people enrol in training after a bad spell, so their earnings would have recovered anyway). Choose control groups that face the same shocks, and argue why nothing else changed for the treated group at the same time.
Exercise
Break parallel trends: give the treated province an extra upward trend of +0.005 per month from the start.
Recompute the DiD. Then add a group-specific linear trend (treated:time) to the regression. Does that fix
it here, and why is it risky in general?
Show a solution
df["emp_bad"] = df.employment + 0.005 * df.treated * df.time
bad = smf.ols("emp_bad ~ treated * post", data=df).fit()
fixed = smf.ols("emp_bad ~ treated * post + treated:time", data=df).fit()
print(f"DiD with diverging trends: {bad.params['treated:post']:+.4f}")
print(f"With group trend: {fixed.params['treated:post']:+.4f}")
# The trend term is estimated over the whole period, so it absorbs part of the policy's step change
# too: the estimate moves far from -0.05. Group trends are fragile; prefer better control groups,
# and use trends estimated on pre-periods only as a robustness check, never as a default.
Checklist and recap
- I can compute a 2×2 DiD from four means and from a regression.
- I can state parallel trends and explain what pre-trends can and cannot show.
- I cluster standard errors at the level the policy varies, and note when there are too few clusters.