{
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "colab": {
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "cells": [
  {
   "id": "m15-l3-0",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 15.3 · CI/CD, the Executive Memo and the Capstone Submission\n",
    "\n",
    "*Accelerated Foundation in Applied Causal Inference*\n",
    "\n",
    "**By the end of this lesson you can:**\n",
    "\n",
    "- Automate data-pipeline and API tests with GitHub Actions\n",
    "- Structure the capstone repository so anyone can reproduce it\n",
    "- Write a two-page executive memo that translates ATE, LATE and HTE into decisions\n",
    "- Assess your own capstone against the grading rubric\n",
    "\n",
    "> Read the full explanations on the course website, then run each cell here in order (**Runtime → Run all** works on a fresh runtime)."
   ]
  },
  {
   "id": "m15-l3-1",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The capstone repository"
   ]
  },
  {
   "id": "m15-l3-2",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Testing the data pipeline"
   ]
  },
  {
   "id": "m15-l3-3",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Question it answers:** How do I know my modelling table is still valid after the source data change?"
   ]
  },
  {
   "id": "m15-l3-4",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "%%writefile data/process_data.py\n",
    "\"\"\"Build the tidy modelling panel. Run: python data/process_data.py\"\"\"\n",
    "from pathlib import Path\n",
    "import duckdb\n",
    "import numpy as np\n",
    "import polars as pl\n",
    "\n",
    "OUT = Path(__file__).parent / \"clean\" / \"panel.parquet\"\n",
    "\n",
    "\n",
    "def raw_inputs(seed: int = 1) -> tuple[pl.DataFrame, pl.DataFrame]:\n",
    "    rng = np.random.default_rng(seed)\n",
    "    months = pl.date_range(pl.date(2019, 1, 1), pl.date(2023, 12, 1), \"1mo\", eager=True)\n",
    "    wide = pl.DataFrame({\"month\": months} | {p: rng.normal(6, 0.5, len(months)) for p in [\"ON\", \"QC\", \"AB\"]})\n",
    "    policy = pl.DataFrame({\"province\": [\"ON\", \"QC\", \"AB\"], \"policy_start\": [None, \"2021-01-01\", \"2022-07-01\"]})\n",
    "    return wide, policy\n",
    "\n",
    "\n",
    "def build_panel(wide: pl.DataFrame, policy: pl.DataFrame) -> pl.DataFrame:\n",
    "    long = wide.unpivot(index=\"month\", variable_name=\"province\", value_name=\"unemp_rate\")\n",
    "    return duckdb.sql(\"\"\"\n",
    "        SELECT l.month, l.province, l.unemp_rate,\n",
    "               CASE WHEN p.policy_start IS NOT NULL AND l.month >= CAST(p.policy_start AS DATE) THEN 1 ELSE 0 END AS treated\n",
    "        FROM long l LEFT JOIN policy p USING (province)\n",
    "        ORDER BY province, month\n",
    "    \"\"\").pl()\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    OUT.parent.mkdir(parents=True, exist_ok=True)\n",
    "    build_panel(*raw_inputs()).write_parquet(OUT)\n",
    "    print(f\"Wrote {OUT}\")"
   ]
  },
  {
   "id": "m15-l3-5",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "%%writefile tests/test_data_pipeline.py\n",
    "import polars as pl\n",
    "from data.process_data import build_panel, raw_inputs\n",
    "\n",
    "\n",
    "def test_panel_is_tidy_and_complete():\n",
    "    panel = build_panel(*raw_inputs())\n",
    "    assert set(panel.columns) == {\"month\", \"province\", \"unemp_rate\", \"treated\"}\n",
    "    assert panel.select(pl.struct(\"province\", \"month\").is_duplicated().any()).item() is False\n",
    "    assert panel.group_by(\"province\").len()[\"len\"].n_unique() == 1          # balanced panel\n",
    "\n",
    "\n",
    "def test_values_are_plausible():\n",
    "    panel = build_panel(*raw_inputs())\n",
    "    assert panel[\"unemp_rate\"].is_between(0, 40).all()\n",
    "    assert panel[\"treated\"].is_in([0, 1]).all()\n",
    "    assert panel.filter(pl.col(\"province\") == \"ON\")[\"treated\"].sum() == 0  # Ontario never treated"
   ]
  },
  {
   "id": "m15-l3-6",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "import pytest\n",
    "code = pytest.main([\"-q\", \"tests/test_data_pipeline.py\"])\n",
    "assert code == 0"
   ]
  },
  {
   "id": "m15-l3-7",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GitHub Actions"
   ]
  },
  {
   "id": "m15-l3-8",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The executive memo"
   ]
  },
  {
   "id": "m15-l3-9",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rubric self-assessment"
   ]
  },
  {
   "id": "m15-l3-10",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ✍️ Exercise\n",
    "\n",
    "Before submitting, ask a classmate who has not seen your project to read only the first paragraph of your\n",
    "memo and say what they would do. If their answer differs from your recommendation, rewrite the paragraph."
   ]
  },
  {
   "id": "m15-l3-11",
   "cell_type": "code",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# Your answer here"
   ]
  },
  {
   "id": "m15-l3-12",
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checklist and recap"
   ]
  }
 ]
}