Lesson 5.3: Collider Bias, M-Bias and Selected Samples

Written lesson

The Hollywood paradox

Beauty and talent are independent in the population. To become a successful actor you need enough of a combination of both. Among actors, then, the less beautiful must be more talented, and vice versa. Conditioning on a collider manufactures a correlation.

Python · Colab cell

!pip install -q numpy pandas statsmodels networkx matplotlib seaborn
import numpy as np, pandas as pd, networkx as nx
import statsmodels.formula.api as smf
import matplotlib.pyplot as plt, seaborn as sns
rng = np.random.default_rng(88)
sns.set_theme(style="whitegrid")

Python · Colab cell

N = 10_000
beauty = rng.normal(size=N)
talent = rng.normal(size=N)
is_actor = (beauty + talent + rng.normal(0, 0.5, N) > 1.5).astype(int)
df = pd.DataFrame(dict(beauty=beauty, talent=talent, is_actor=is_actor))

whole = smf.ols("talent ~ beauty", data=df).fit().params["beauty"]
actors = smf.ols("talent ~ beauty", data=df[df.is_actor == 1]).fit().params["beauty"]
print(f"Whole population: {whole:+.3f} (truth: 0)")
print(f"Actors only:      {actors:+.3f} (spurious)")

sns.scatterplot(data=df.sample(3_000, random_state=1), x="beauty", y="talent", hue="is_actor", alpha=.5)
plt.title("Selecting on success creates a negative correlation"); plt.show()

Selected samples are colliders too

You do not need to put a collider in a regression to condition on it. Choosing your sample on it does the same. A classic labour-economics case: wages are only observed for people who work, and whether someone works depends on both their education and their (unobserved) outside options.

Python · Colab cell

N = 50_000
educ = rng.normal(12, 2, N)
outside_option = rng.normal(size=N)                     # e.g. spouse income, home production value
wage_potential = 10 + 1.0 * educ + 3 * rng.normal(size=N)
works = (0.6 * wage_potential - 2.5 * outside_option + rng.normal(size=N) > 13).astype(int)
lab = pd.DataFrame(dict(educ=educ, wage=wage_potential, works=works))

print(f"Share working: {lab.works.mean():.2f}")
print(f"Return to education, everyone:      {smf.ols('wage ~ educ', data=lab).fit().params['educ']:.3f} (truth 1.0)")
print(f"Return to education, workers only:  {smf.ols('wage ~ educ', data=lab[lab.works == 1]).fit().params['educ']:.3f}")

Low-educated people who still work tend to have unusually high wage draws, so the slope among workers is flattened. This is the selection problem Heckman's work addresses. In DAG terms, works is a collider of education and the wage noise.

M-bias: a pre-treatment bad control

"Control for everything measured before treatment" is a popular rule. It is not always safe. In the M-structure, a pre-treatment variable Z is a collider of two unobserved causes, one of the treatment and one of the outcome.

Python · Colab cell

edges = [("U1", "D"), ("U1", "Z"), ("U2", "Z"), ("U2", "Y"), ("D", "Y")]
pos = {"U1": (0, 1), "U2": (2, 1), "Z": (1, 0.5), "D": (0, 0), "Y": (2, 0)}
g = nx.DiGraph(edges)
plt.figure(figsize=(6, 3)); nx.draw_networkx(g, pos, node_size=1500, node_color="#bfdbfe", edgecolors="#334155")
plt.title("M-bias: Z is pre-treatment but a collider"); plt.axis("off"); plt.show()

N = 50_000
u1, u2 = rng.normal(size=N), rng.normal(size=N)
z = u1 + u2 + rng.normal(size=N)
d = 0.8 * u1 + rng.normal(size=N)
y = 1.0 * d + 0.8 * u2 + rng.normal(size=N)
m = pd.DataFrame(dict(d=d, y=y, z=z))
print(f"y ~ d     : {smf.ols('y ~ d', data=m).fit().params['d']:.3f} (unbiased, truth 1.0)")
print(f"y ~ d + z : {smf.ols('y ~ d + z', data=m).fit().params['d']:.3f} (M-bias)")

In practice M-bias is usually small compared with confounding bias, so do not let it paralyse you. The point is the method: decide controls from the graph, not from a rule of thumb.

Workshop: your capstone DAG

Exercise

Choose the policy question you will carry to the capstone (Module 15): for example, a digital-skills programme and wages, a wage subsidy and employment, or a carbon price and emissions.

List treatment, outcome, and 5–8 other variables.
Draw the DAG with networkx. Mark unobserved variables.
Use backdoor_sets from Lesson 5.2 to find a valid adjustment set.
Name one variable you will not control for, and say why (mediator, collider, descendant).
If no valid set exists, say which design you will use instead (IV, RDD, DiD, SCM).

Show a solution

Python · Colab cell

import itertools
d_sep = getattr(nx, "is_d_separator", None) or nx.d_separated

def backdoor_sets(g, treatment, outcome, observed):
    g_back = g.copy(); g_back.remove_edges_from(list(g.out_edges(treatment)))
    bad = nx.descendants(g, treatment)
    cand = [v for v in observed if v not in bad and v not in (treatment, outcome)]
    return [set(z) for k in range(len(cand) + 1) for z in itertools.combinations(cand, k)
            if d_sep(g_back, {treatment}, {outcome}, set(z))]

capstone = nx.DiGraph([
    ("Age", "DigitalTraining"), ("Age", "Wage"), ("PriorWage", "DigitalTraining"), ("PriorWage", "Wage"),
    ("Region", "DigitalTraining"), ("Region", "Wage"), ("DigitalTraining", "Certification"),
    ("Certification", "Wage"), ("DigitalTraining", "JobSwitch"), ("Wage", "JobSwitch"),
])
print(backdoor_sets(capstone, "DigitalTraining", "Wage",
                    ["Age", "PriorWage", "Region", "Certification", "JobSwitch"])[:3])
# Do not control for Certification (mediator) or JobSwitch (collider / descendant of treatment).

Checklist and recap

I can simulate and explain collider bias.
I recognise selected samples as conditioning on a collider.
I know M-bias exists and why "control for all pre-treatment variables" is a heuristic, not a law.
I have a first DAG for my capstone question.