{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "cells": [
  {
   "id": "m09-l1-0",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 9.1 · Instruments, the Wald Estimator and Compliers\n",
    "\n",
    "*Accelerated Foundation in Applied Causal Inference*\n",
    "\n",
    "**By the end of this lesson you can:**\n",
    "\n",
    "- State the three IV conditions: relevance, independence and the exclusion restriction\n",
    "- Compute the Wald estimator as reduced form divided by first stage\n",
    "- Classify units as compliers, always-takers, never-takers and defiers\n",
    "- Show by simulation that IV estimates the LATE for compliers, not the ATE\n",
    "\n",
    "> Read the full explanations on the course website, then run each cell here in order (**Runtime → Run all** works on a fresh runtime)."
   ]
  },
  {
   "id": "m09-l1-1",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## When nothing observable closes the backdoor"
   ]
  },
  {
   "id": "m09-l1-2",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "!pip install -q numpy pandas statsmodels linearmodels matplotlib\n",
    "import numpy as np, pandas as pd\n",
    "import statsmodels.formula.api as smf\n",
    "rng = np.random.default_rng(101)"
   ]
  },
  {
   "id": "m09-l1-3",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## An encouragement design"
   ]
  },
  {
   "id": "m09-l1-4",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** What is the earnings effect of an adult-education course when enrolment is self-selected but encouragement was random?"
   ]
  },
  {
   "id": "m09-l1-5",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "N = 20_000\n",
    "motivation = rng.normal(size=N)\n",
    "z = rng.binomial(1, 0.5, N)                                         # random encouragement letter\n",
    "\n",
    "# Each person's type, determined by motivation (never defiers here: monotonicity holds)\n",
    "u = rng.uniform(size=N)\n",
    "high = motivation > 0\n",
    "always = u < np.where(high, 0.30, 0.10)\n",
    "never = (~always) & (u > np.where(high, 0.80, 0.45))\n",
    "complier = ~(always | never)\n",
    "d = np.where(always, 1, np.where(never, 0, z))                       # compliers follow the letter\n",
    "\n",
    "effect = np.where(complier, 2_000, np.where(always, 4_000, 3_000))   # heterogeneous effects by type\n",
    "y = 30_000 + 5_000 * motivation + effect * d + rng.normal(0, 4_000, N)\n",
    "df = pd.DataFrame({\"z\": z, \"d\": d, \"y\": y, \"type\": np.select([always, never, complier], [\"always-taker\", \"never-taker\", \"complier\"], default=\"\")})\n",
    "\n",
    "print(df.type.value_counts(normalize=True).round(3))\n",
    "print(f\"Naive difference (enrolled vs not): {df.y[df.d == 1].mean() - df.y[df.d == 0].mean():,.0f}\")\n",
    "print(f\"True ATE: {effect.mean():,.0f} | true LATE (compliers): 2,000\")"
   ]
  },
  {
   "id": "m09-l1-6",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Wald estimator"
   ]
  },
  {
   "id": "m09-l1-7",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** How much did the letter raise enrolment, how much did it raise earnings, and what is the implied course effect?"
   ]
  },
  {
   "id": "m09-l1-8",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "first_stage = df.d[df.z == 1].mean() - df.d[df.z == 0].mean()\n",
    "reduced_form = df.y[df.z == 1].mean() - df.y[df.z == 0].mean()\n",
    "wald = reduced_form / first_stage\n",
    "print(f\"First stage (enrolment gap):   {first_stage:.3f}\")\n",
    "print(f\"Reduced form / ITT (earnings):  {reduced_form:,.0f}\")\n",
    "print(f\"Wald IV estimate:               {wald:,.0f}\")\n",
    "\n",
    "# The same number from regressions\n",
    "fs = smf.ols(\"d ~ z\", data=df).fit().params[\"z\"]\n",
    "rf = smf.ols(\"y ~ z\", data=df).fit().params[\"z\"]\n",
    "print(f\"Ratio of regression slopes:     {rf / fs:,.0f}\")"
   ]
  },
  {
   "id": "m09-l1-9",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compliers and the LATE"
   ]
  },
  {
   "id": "m09-l1-10",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "df[\"motivated\"] = (motivation > 0).astype(int)\n",
    "complier_share_motivated = (\n",
    "    (df.d * df.motivated)[df.z == 1].mean() - (df.d * df.motivated)[df.z == 0].mean()\n",
    ") / first_stage\n",
    "print(f\"Share motivated: compliers ≈ {complier_share_motivated:.2f} vs everyone {df.motivated.mean():.2f}\")"
   ]
  },
  {
   "id": "m09-l1-11",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ✍️ Exercise\n",
    "\n",
    "Introduce 5% **defiers** (people who enrol only if they do *not* get the letter) with an effect of −3,000.\n",
    "Recompute the Wald estimate. Why does it no longer equal the compliers' effect?"
   ]
  },
  {
   "id": "m09-l1-12",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Your answer here"
   ]
  },
  {
   "id": "m09-l1-13",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checklist and recap"
   ]
  }
 ]
}