Lesson 9.3: The Exclusion Restriction, Endogeneity Tests and Over-Identification

Written lesson

The Achilles heel: exclusion

The exclusion restriction says the instrument affects the outcome only through the treatment. With "distance to college" it fails if, for example:

Direct effect: places near colleges are urban, with higher wages regardless of schooling.
Alternative path: proximity brings networks, internships or part-time jobs that raise wages independently of the degree.

Unlike relevance, exclusion cannot be tested with the data at hand. It must be defended with institutional knowledge, and its failure can be large, because any direct effect is divided by the (often small) first stage.

Python · Colab cell

!pip install -q numpy pandas linearmodels statsmodels causaldata
import numpy as np, pandas as pd
from linearmodels.iv import IV2SLS
rng = np.random.default_rng(9)

Python · Colab cell

def iv_with_violation(direct_effect, n=20_000, first_stage=1.0):
    a = rng.normal(size=n); z = rng.binomial(1, 0.5, n)
    educ = 12 + first_stage * z + 2 * a + rng.normal(size=n)
    y = 25_000 + 3_000 * educ + 5_000 * a + direct_effect * z + rng.normal(0, 4_000, n)
    d = pd.DataFrame({"y": y, "educ": educ, "z": z, "const": 1.0})
    return IV2SLS(d.y, d.const, d.educ, d.z).fit().params["educ"]

for direct in [0, 250, 500, 1_000]:
    for fs in [1.0, 0.3]:
        print(f"direct effect ${direct:>5} | first stage {fs}: 2SLS = {iv_with_violation(direct, first_stage=fs):,.0f}")

The bias is approximately direct effect / first stage. A $500 direct effect barely matters with a strong first stage, but becomes huge when the instrument moves schooling only a little. Weak instruments and small exclusion violations are a dangerous combination.

Is the treatment endogenous at all? Wu–Hausman

If D is actually exogenous, OLS is consistent and more precise than IV. The Wu–Hausman test compares the two: a large difference suggests endogeneity. It assumes the instrument is valid, so it cannot rescue a bad instrument.

Python · Colab cell

from causaldata import close_college
card = close_college.load_pandas().data.dropna()
iv_card = IV2SLS.from_formula(
    "lwage ~ 1 + exper + I(exper**2) + black + south + smsa + married + [educ ~ nearc4]", card
).fit(cov_type="robust")
print(iv_card.wu_hausman())

With Card's data the test does not reject: IV is so imprecise that OLS and IV are statistically hard to tell apart. That is common, and is a reason to report both.

Two instruments and over-identification

With more instruments than endogenous variables, you can test whether the instruments agree with each other. If two valid instruments estimate the same effect, their 2SLS estimates should be similar; the Sargan (or Hansen J) test checks that.

Python · Colab cell

N = 10_000
a = rng.normal(size=N)
z1 = rng.binomial(1, 0.5, N)                    # near a college (valid)
z2 = rng.normal(size=N)                         # tuition subsidy shock (valid... or not)
for violation in [0, 1_500]:
    educ = 12 + 1.0 * z1 + 0.8 * z2 + 2 * a + rng.normal(size=N)
    y = 25_000 + 3_000 * educ + 5_000 * a + violation * z2 + rng.normal(0, 4_000, N)
    d = pd.DataFrame({"y": y, "educ": educ, "z1": z1, "z2": z2, "const": 1.0})
    r = IV2SLS(d.y, d.const, d.educ, d[["z1", "z2"]]).fit()
    print(f"z2 direct effect {violation:>5}: 2SLS = {r.params['educ']:,.0f} | Sargan p = {r.sargan.pval:.3f}")

The why

The Sargan test only detects instruments that disagree. If both instruments are invalid in the same direction, the test passes and the estimate is still wrong. Also, under heterogeneous effects, two valid instruments can legitimately estimate different LATEs (different complier groups), which makes the test reject for a reason that is not invalidity. Treat it as a diagnostic, not a certificate.

Workshop: defending an instrument

Exercise

Write a half-page defence of an instrument for your capstone or for one of these: rainfall shocks for agricultural income; judge leniency for incarceration; college proximity for schooling (Card). Cover:

Relevance: the first-stage mechanism and how you would show it.
Independence: why Z is as good as random (or random given which covariates).
Exclusion: list every other path from Z to Y you can think of and argue each away, or bound it.
Monotonicity: could anyone respond in the opposite direction?
Who are the compliers, and is their LATE the effect a policy-maker needs?

Then run the code below for Card's instrument: how large would a direct effect of growing up near a college on log wages need to be to change the conclusion?

Show a solution

Python · Colab cell

for direct in [0.0, 0.01, 0.02, 0.04]:
    # Remove an assumed direct effect of proximity on log wages, then re-estimate
    card["lwage_adj"] = card["lwage"] - direct * card["nearc4"]
    r = IV2SLS.from_formula(
        "lwage_adj ~ 1 + exper + I(exper**2) + black + south + smsa + married + [educ ~ nearc4]", card
    ).fit(cov_type="robust")
    print(f"assumed direct effect {100 * direct:.0f}% wages: return = {100 * r.params['educ']:.1f}% "
          f"(95% CI {100 * r.conf_int().loc['educ'].iloc[0]:.1f} to {100 * r.conf_int().loc['educ'].iloc[1]:.1f})")
# A direct effect of only 2-4% on wages wipes out most of the estimated return, because the first stage
# is about 0.32 years. A clear defence plus a 'how large would a violation need to be' table is more
# convincing than any single test statistic (Conley, Hansen and Rossi call this 'plausibly exogenous').

Checklist and recap

I can show how exclusion violations bias 2SLS, and why weak first stages make it worse.
I can run and interpret Wu–Hausman and Sargan tests, and state their limits.
I can write a structured defence of an instrument.