Lesson 3.2: Omitted Variable Bias and Partialling Out

Written lesson

Setup

Python · Colab cell

!pip install -q numpy polars statsmodels matplotlib seaborn causaldata
import numpy as np, polars as pl, pandas as pd
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt, seaborn as sns
rng = np.random.default_rng(2026)
sns.set_theme(style="darkgrid")

A world with ability bias

Now we add an unobserved factor, call it privilege / ability, that raises both the years of education a person completes and their wage. The true effect of a year of school is still exactly $2,500.

Python · Colab cell

N = 1_000
TRUE_RETURN = 2_500
ability = rng.normal(0, 1, N)
education = np.clip(np.round(12 + 1.5 * ability + rng.normal(0, 1.2, N)), 8, 20)
wage = 30_000 + TRUE_RETURN * education + 8_000 * ability + rng.normal(0, 5_000, N)
df = pd.DataFrame({"education": education, "ability": ability, "wage": wage})

naive = smf.ols("wage ~ education", data=df).fit()
controlled = smf.ols("wage ~ education + ability", data=df).fit()
print(f"True return:          ${TRUE_RETURN:,.0f}")
print(f"Naive estimate:       ${naive.params['education']:,.0f}")
print(f"Controlling ability:  ${controlled.params['education']:,.0f}")

The naive slope absorbs part of ability's effect, because ability travels together with education. When we include ability, OLS compares workers with the same ability and the bias disappears.

The why

Leaving out a variable that (1) affects the outcome and (2) is correlated with the treatment pushes its effect into the error term. The error is then correlated with the regressor, which breaks the key OLS assumption. This is omitted variable bias, the regression version of the confounding you saw in Lesson 2.3.

The OVB formula, checked by code

The bias has an exact form:

naive slope = true slope + (effect of ability on wage) × (slope of ability on education)

Python · Colab cell

gamma = controlled.params["ability"]                                  # effect of the omitted variable
delta = smf.ols("ability ~ education", data=df).fit().params["education"]  # its link to the treatment
print(f"Controlled + gamma*delta = {controlled.params['education'] + gamma * delta:,.1f}")
print(f"Naive slope              = {naive.params['education']:,.1f}")

The formula also tells you the sign of the bias without running anything: if the omitted factor raises the outcome and is positively related to treatment, the naive estimate is too high. You will use this reasoning in sensitivity analysis (Module 14).

Partialling out: the Frisch–Waugh–Lovell theorem

How does a multiple regression "hold ability constant"? The FWL theorem says the coefficient on education in wage ~ education + ability is identical to this three-step recipe:

Regress wage on ability, keep the residuals (the part of wage ability cannot explain).
Regress education on ability, keep the residuals (the part of education ability cannot explain).
Regress residualised wage on residualised education.

Python · Colab cell

df["wage_res"] = smf.ols("wage ~ ability", data=df).fit().resid
df["educ_res"] = smf.ols("education ~ ability", data=df).fit().resid
fwl = smf.ols("wage_res ~ educ_res - 1", data=df).fit()
print(f"FWL slope:        {fwl.params['educ_res']:,.2f}")
print(f"Multiple OLS:     {controlled.params['education']:,.2f}")

fig, axes = plt.subplots(1, 2, figsize=(12, 4))
sns.regplot(x="education", y="wage", data=df, ax=axes[0], scatter_kws={"alpha": .3}).set(title="Raw: confounded")
sns.regplot(x="educ_res", y="wage_res", data=df, ax=axes[1], scatter_kws={"alpha": .3}).set(title="Residualised: ability removed")
plt.tight_layout(); plt.show()

Key takeaway

Keep this picture in mind. Double Machine Learning (Module 7) is FWL with machine learning: it replaces the two first-step linear regressions with flexible models (random forests, boosting) so that confounders can act in complex, non-linear ways.

Real data: Card's returns to schooling

The close_college data (Card, 1995) contains 3,010 US men with years of education, experience and log wages. We do not observe ability here, so we cannot "fix" OVB with a control. We will come back to these data in Module 9 with an instrument.

Python · Colab cell

from causaldata import close_college
card = close_college.load_pandas().data
m1 = smf.ols("lwage ~ educ", data=card).fit(cov_type="HC1")
m2 = smf.ols("lwage ~ educ + exper + I(exper**2) + black + south + smsa + married", data=card).fit(cov_type="HC1")
print(f"Return to schooling, no controls:   {100*m1.params['educ']:.1f}% per year")
print(f"Return to schooling, with controls: {100*m2.params['educ']:.1f}% per year")

Adding controls raises the estimate here: controlling for experience matters because more schooling means fewer years of work at a given age. Controls do not always push an estimate down; the OVB formula tells you the direction once you know how the omitted variable relates to schooling and wages.

Exercise

In the simulated world, suppose ability is measured with error: you only observe ability_noisy = ability + noise with noise standard deviation 1. Control for the noisy measure instead. How much bias remains? What does this suggest about "controlling for" imperfect proxies such as test scores?

Show a solution

Python · Colab cell

df["ability_noisy"] = df["ability"] + rng.normal(0, 1, N)
proxy = smf.ols("wage ~ education + ability_noisy", data=df).fit()
print(f"Controlling a noisy proxy: ${proxy.params['education']:,.0f} (true ${TRUE_RETURN:,})")
# A noisy proxy removes only part of the bias. Imperfect controls leave residual confounding.

Checklist and recap

I can create OVB in a simulation and predict its sign.
I can verify the OVB formula numerically.
I can explain partialling out (FWL) and why it is the foundation of Double ML.