{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# F Tests for Exclusion Restrictions\n",
        "\n",
        "Learning goal: run a real multiple-regression inference workflow and write a careful interpretation.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Dataset check\n",
        "This cell confirms that the dataset file is available. If the file is missing, the notebook gives a safe warning rather than fake results.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "DATASET = \"BWGHT.DTA\"\n",
        "VARIABLES = ['bwght', 'cigs', 'faminc', 'motheduc', 'fatheduc']\n",
        "if not os.path.exists(DATASET):\n",
        "    raise FileNotFoundError(\n",
        "        \"Dataset file not installed yet\\n\"\n",
        "        \"Dataset: BWGHT\\n\"\n",
        "        \"Variables needed: bwght, cigs, faminc, motheduc, fatheduc\\n\"\n",
        "        \"Course data folder: https://drive.google.com/drive/folders/1_STdcydIcst-opcbwOKRFzUXsgxQgBoS?usp=sharing\\n\"\n",
        "        \"Admin upload instruction: upload the dataset in Admin -> Datasets, publish it, and make the file available to the notebook runner.\"\n",
        "    )\n",
        "print(\"Ready:\", DATASET)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Estimate the model\n",
        "Estimate OLS and inspect the coefficient, standard error, t statistic, and p-value.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import statsmodels.api as sm\n",
        "from scipy import stats\n",
        "\n",
        "df = pd.read_stata(DATASET)[VARIABLES].dropna()\n",
        "y = df[\"bwght\"]\n",
        "X = sm.add_constant(df[['cigs', 'faminc', 'motheduc', 'fatheduc']])\n",
        "model = sm.OLS(y, X).fit()\n",
        "print(model.summary())\n",
        "print(\"Focus coefficient:\", \"motheduc\", model.params[\"motheduc\"])\n",
        "print(\"Standard error:\", model.bse[\"motheduc\"])\n",
        "print(\"t statistic:\", model.tvalues[\"motheduc\"])\n",
        "print(\"two-sided p-value:\", model.pvalues[\"motheduc\"])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Inference calculation\n",
        "Compute the lesson-specific inference object and connect it to the hypothesis.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "restricted = sm.OLS(y, sm.add_constant(df[[\"cigs\", \"faminc\"]])).fit()\n",
        "q = 2\n",
        "f_stat = ((restricted.ssr - model.ssr) / q) / (model.ssr / model.df_resid)\n",
        "p_value = 1 - stats.f.cdf(f_stat, q, model.df_resid)\n",
        "print({\"F\": f_stat, \"p_value\": p_value, \"df_num\": q, \"df_den\": model.df_resid})\n",
        "print(model.f_test(\"motheduc = 0, fatheduc = 0\"))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Check your understanding\n",
        "Write one sentence that distinguishes statistical significance from practical importance for this model.\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python (Pyodide)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "pygments_lexer": "ipython3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}