{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "cells": [
  {
   "id": "m13-l3-0",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 13.3 · Social Networks and Insurance Take-up: Peer Effects in Real Data\n",
    "\n",
    "*Accelerated Foundation in Applied Causal Inference*\n",
    "\n",
    "**By the end of this lesson you can:**\n",
    "\n",
    "- Analyse a real field experiment on information diffusion (Cai, de Janvry and Sadoulet, 2015)\n",
    "- Estimate how friends' decisions relate to a farmer's own insurance take-up\n",
    "- Explain Manski's reflection problem and why peer correlations are not automatically causal\n",
    "- Use randomised variation in peers' exposure to identify social-network effects\n",
    "\n",
    "> Read the full explanations on the course website, then run each cell here in order (**Runtime → Run all** works on a fresh runtime)."
   ]
  },
  {
   "id": "m13-l3-1",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Weather insurance in rural China"
   ]
  },
  {
   "id": "m13-l3-2",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "!pip install -q numpy pandas statsmodels matplotlib seaborn causaldata\n",
    "import numpy as np, pandas as pd\n",
    "import statsmodels.formula.api as smf\n",
    "import matplotlib.pyplot as plt, seaborn as sns\n",
    "from causaldata import social_insure\n",
    "sns.set_theme(style=\"whitegrid\")\n",
    "\n",
    "cols = [\"takeup_survey\", \"intensive\", \"pre_takeup_rate\", \"village\", \"male\", \"age\", \"agpop\",\n",
    "        \"ricearea_2010\", \"literacy\", \"risk_averse\", \"disaster_prob\", \"default\"]\n",
    "ins = social_insure.load_pandas().data.dropna(subset=cols)\n",
    "print(f\"{len(ins):,} households in {ins.village.nunique()} villages | take-up {ins.takeup_survey.mean():.2f}\")"
   ]
  },
  {
   "id": "m13-l3-3",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The randomised information sessions"
   ]
  },
  {
   "id": "m13-l3-4",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** Did the intensive information session raise insurance take-up?"
   ]
  },
  {
   "id": "m13-l3-5",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "def cluster_fit(formula, data):\n",
    "    return smf.ols(formula, data=data).fit(cov_type=\"cluster\", cov_kwds={\"groups\": data[\"village\"]})\n",
    "\n",
    "controls = \"male + age + agpop + ricearea_2010 + literacy + risk_averse + disaster_prob\"\n",
    "m_int = cluster_fit(f\"takeup_survey ~ intensive + {controls} + C(village)\", ins)\n",
    "print(f\"Effect of attending an intensive session: {m_int.params['intensive']:+.3f} (SE {m_int.bse['intensive']:.3f})\")"
   ]
  },
  {
   "id": "m13-l3-6",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Friends' take-up and the reflection problem"
   ]
  },
  {
   "id": "m13-l3-7",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "m_peer = cluster_fit(f\"takeup_survey ~ pre_takeup_rate + intensive + {controls} + C(village)\", ins)\n",
    "print(f\"Take-up rises by {m_peer.params['pre_takeup_rate']:.3f} when all friends (vs none) took up \"\n",
    "      f\"(SE {m_peer.bse['pre_takeup_rate']:.3f})\")\n",
    "\n",
    "ins[\"friends_bin\"] = pd.cut(ins.pre_takeup_rate, [-0.01, 0.2, 0.4, 0.6, 1.0])\n",
    "ins.groupby(\"friends_bin\", observed=True).takeup_survey.mean().plot(marker=\"o\")\n",
    "plt.ylabel(\"Own take-up\"); plt.xlabel(\"Friends' take-up rate\"); plt.title(\"Peers and insurance take-up\"); plt.show()"
   ]
  },
  {
   "id": "m13-l3-8",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using randomised peer exposure"
   ]
  },
  {
   "id": "m13-l3-9",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** How do I identify a peer effect when friends are similar to each other?"
   ]
  },
  {
   "id": "m13-l3-10",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "rng = np.random.default_rng(15)\n",
    "n = 20_000\n",
    "trait = rng.normal(size=n)                          # e.g. risk attitude, shared with friends (homophily)\n",
    "friends_trait = 0.7 * trait + rng.normal(0, 0.7, n)\n",
    "share_friends_intensive = rng.binomial(10, 0.5, n) / 10          # RANDOMISED exposure of friends\n",
    "friends_takeup = np.clip(0.2 + 0.2 * friends_trait + 0.6 * share_friends_intensive + rng.normal(0, 0.1, n), 0, 1)\n",
    "# True social effect: +0.4 per unit of friends' take-up; own trait also matters\n",
    "own_takeup = rng.binomial(1, np.clip(0.2 + 0.4 * friends_takeup + 0.15 * trait, 0.01, 0.99))\n",
    "sim = pd.DataFrame({\"own\": own_takeup, \"friends\": friends_takeup, \"exposure\": share_friends_intensive})\n",
    "\n",
    "from linearmodels.iv import IV2SLS\n",
    "ols = smf.ols(\"own ~ friends\", data=sim).fit().params[\"friends\"]\n",
    "iv = IV2SLS.from_formula(\"own ~ 1 + [friends ~ exposure]\", sim).fit().params[\"friends\"]\n",
    "print(f\"OLS peer effect: {ols:.2f} (confounded by homophily) | IV using randomised exposure: {iv:.2f} (truth 0.40)\")"
   ]
  },
  {
   "id": "m13-l3-11",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ✍️ Exercise\n",
    "\n",
    "In the simulation, set the true social effect to zero (`0.0 * friends_takeup`) and re-run both estimators.\n",
    "What does OLS report? Explain why to a programme manager who sees a strong correlation between friends'\n",
    "choices and concludes \"word of mouth works\"."
   ]
  },
  {
   "id": "m13-l3-12",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Your answer here"
   ]
  },
  {
   "id": "m13-l3-13",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checklist and recap"
   ]
  }
 ]
}