{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Histograms, Skewness, and Transformations\n\nLearning goal: Compare residual histograms for wage and log wage.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Dataset check\nThis cell confirms the dataset file is available. Simulation notebooks mount a harmless CSV so public file delivery is still validated.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import os\n",
        "DATASET = \"WAGE1.csv\"\n",
        "VARIABLES = [\"wage\",\"educ\",\"exper\",\"tenure\"]\n",
        "if not os.path.exists(DATASET):\n",
        "    raise FileNotFoundError(\n",
        "        \"Dataset file not installed yet\\n\"\n",
        "        \"Dataset: WAGE1\\n\"\n",
        "        \"Variables needed: wage, educ, exper, tenure\\n\"\n",
        "        \"Course data folder: https://drive.google.com/drive/folders/1_STdcydIcst-opcbwOKRFzUXsgxQgBoS?usp=sharing\\n\"\n",
        "        \"Admin upload instruction: upload the dataset in Admin -> Datasets, publish it, and make the file available to the notebook runner.\"\n",
        "    )\n",
        "print(\"Ready:\", DATASET)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Run the lab\nRun the code, inspect the table or graph, and connect the result to the formula in the lesson.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import statsmodels.api as sm\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "data = pd.read_csv(\"WAGE1.csv\")\n",
        "data = data.dropna(subset=[\"wage\", \"educ\", \"exper\", \"tenure\"])\n",
        "X = sm.add_constant(data[[\"educ\", \"exper\", \"tenure\"]])\n",
        "model_level = sm.OLS(data[\"wage\"], X).fit()\n",
        "data[\"log_wage\"] = np.log(data[\"wage\"])\n",
        "model_log = sm.OLS(data[\"log_wage\"], X).fit()\n",
        "fig, axes = plt.subplots(1, 2, figsize=(10, 4))\n",
        "axes[0].hist(model_level.resid, bins=30)\n",
        "axes[0].set_title(\"Residuals: wage model\")\n",
        "axes[0].set_xlabel(\"residual\")\n",
        "axes[1].hist(model_log.resid, bins=30)\n",
        "axes[1].set_title(\"Residuals: log(wage) model\")\n",
        "axes[1].set_xlabel(\"residual\")\n",
        "plt.tight_layout()\n",
        "print(\"Level residual skewness:\", pd.Series(model_level.resid).skew())\n",
        "print(\"Log residual skewness:\", pd.Series(model_log.resid).skew())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Formula explanation\nExplain the probability limit, asymptotic approximation, standard-error pattern, or LM statistic in words. Do not treat output as automatic causal evidence.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Short exercise\nChange one sample size, regressor, or restriction. Write two sentences: what changed mechanically, and what assumption still matters?\n\n## Check your understanding\nDoes increasing n fix nonnormality, endogeneity, heteroskedasticity, or omitted variables? Explain.\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}