{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "cells": [
  {
   "id": "m05-l3-0",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 5.3 · Collider Bias, M-Bias and Selected Samples\n",
    "\n",
    "*Accelerated Foundation in Applied Causal Inference*\n",
    "\n",
    "**By the end of this lesson you can:**\n",
    "\n",
    "- Simulate collider bias with the classic \"Hollywood actors\" example\n",
    "- Recognise sample selection (looking only at employed workers, surviving firms) as conditioning on a collider\n",
    "- Explain M-bias, where controlling for a pre-treatment variable still adds bias\n",
    "- Write a defensible DAG for your own capstone question\n",
    "\n",
    "> Read the full explanations on the course website, then run each cell here in order (**Runtime → Run all** works on a fresh runtime)."
   ]
  },
  {
   "id": "m05-l3-1",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Hollywood paradox"
   ]
  },
  {
   "id": "m05-l3-2",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "!pip install -q numpy pandas statsmodels networkx matplotlib seaborn\n",
    "import numpy as np, pandas as pd, networkx as nx\n",
    "import statsmodels.formula.api as smf\n",
    "import matplotlib.pyplot as plt, seaborn as sns\n",
    "rng = np.random.default_rng(88)\n",
    "sns.set_theme(style=\"whitegrid\")"
   ]
  },
  {
   "id": "m05-l3-3",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** Why do beautiful actors seem less talented, when beauty and talent are unrelated?"
   ]
  },
  {
   "id": "m05-l3-4",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "N = 10_000\n",
    "beauty = rng.normal(size=N)\n",
    "talent = rng.normal(size=N)\n",
    "is_actor = (beauty + talent + rng.normal(0, 0.5, N) > 1.5).astype(int)\n",
    "df = pd.DataFrame(dict(beauty=beauty, talent=talent, is_actor=is_actor))\n",
    "\n",
    "whole = smf.ols(\"talent ~ beauty\", data=df).fit().params[\"beauty\"]\n",
    "actors = smf.ols(\"talent ~ beauty\", data=df[df.is_actor == 1]).fit().params[\"beauty\"]\n",
    "print(f\"Whole population: {whole:+.3f} (truth: 0)\")\n",
    "print(f\"Actors only:      {actors:+.3f} (spurious)\")\n",
    "\n",
    "sns.scatterplot(data=df.sample(3_000, random_state=1), x=\"beauty\", y=\"talent\", hue=\"is_actor\", alpha=.5)\n",
    "plt.title(\"Selecting on success creates a negative correlation\"); plt.show()"
   ]
  },
  {
   "id": "m05-l3-5",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selected samples are colliders too"
   ]
  },
  {
   "id": "m05-l3-6",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** Is the return to education biased if I only use people with observed wages?"
   ]
  },
  {
   "id": "m05-l3-7",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "N = 50_000\n",
    "educ = rng.normal(12, 2, N)\n",
    "outside_option = rng.normal(size=N)                     # e.g. spouse income, home production value\n",
    "wage_potential = 10 + 1.0 * educ + 3 * rng.normal(size=N)\n",
    "works = (0.6 * wage_potential - 2.5 * outside_option + rng.normal(size=N) > 13).astype(int)\n",
    "lab = pd.DataFrame(dict(educ=educ, wage=wage_potential, works=works))\n",
    "\n",
    "print(f\"Share working: {lab.works.mean():.2f}\")\n",
    "print(f\"Return to education, everyone:      {smf.ols('wage ~ educ', data=lab).fit().params['educ']:.3f} (truth 1.0)\")\n",
    "print(f\"Return to education, workers only:  {smf.ols('wage ~ educ', data=lab[lab.works == 1]).fit().params['educ']:.3f}\")"
   ]
  },
  {
   "id": "m05-l3-8",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## M-bias: a pre-treatment bad control"
   ]
  },
  {
   "id": "m05-l3-9",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "edges = [(\"U1\", \"D\"), (\"U1\", \"Z\"), (\"U2\", \"Z\"), (\"U2\", \"Y\"), (\"D\", \"Y\")]\n",
    "pos = {\"U1\": (0, 1), \"U2\": (2, 1), \"Z\": (1, 0.5), \"D\": (0, 0), \"Y\": (2, 0)}\n",
    "g = nx.DiGraph(edges)\n",
    "plt.figure(figsize=(6, 3)); nx.draw_networkx(g, pos, node_size=1500, node_color=\"#bfdbfe\", edgecolors=\"#334155\")\n",
    "plt.title(\"M-bias: Z is pre-treatment but a collider\"); plt.axis(\"off\"); plt.show()\n",
    "\n",
    "N = 50_000\n",
    "u1, u2 = rng.normal(size=N), rng.normal(size=N)\n",
    "z = u1 + u2 + rng.normal(size=N)\n",
    "d = 0.8 * u1 + rng.normal(size=N)\n",
    "y = 1.0 * d + 0.8 * u2 + rng.normal(size=N)\n",
    "m = pd.DataFrame(dict(d=d, y=y, z=z))\n",
    "print(f\"y ~ d     : {smf.ols('y ~ d', data=m).fit().params['d']:.3f} (unbiased, truth 1.0)\")\n",
    "print(f\"y ~ d + z : {smf.ols('y ~ d + z', data=m).fit().params['d']:.3f} (M-bias)\")"
   ]
  },
  {
   "id": "m05-l3-10",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Workshop: your capstone DAG"
   ]
  },
  {
   "id": "m05-l3-11",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ✍️ Exercise\n",
    "\n",
    "Choose the policy question you will carry to the capstone (Module 15): for example, a digital-skills\n",
    "programme and wages, a wage subsidy and employment, or a carbon price and emissions.\n",
    "\n",
    "1. List treatment, outcome, and 5–8 other variables.\n",
    "2. Draw the DAG with `networkx`. Mark unobserved variables.\n",
    "3. Use `backdoor_sets` from Lesson 5.2 to find a valid adjustment set.\n",
    "4. Name one variable you will **not** control for, and say why (mediator, collider, descendant).\n",
    "5. If no valid set exists, say which design you will use instead (IV, RDD, DiD, SCM)."
   ]
  },
  {
   "id": "m05-l3-12",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Your answer here"
   ]
  },
  {
   "id": "m05-l3-13",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checklist and recap"
   ]
  }
 ]
}