{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Neural Networks and Deep Learning for Economic Data\n",
        "\n**Opening question:** How does a stack of simple differentiable units learn a nonlinear mapping, and how do we know it has not merely memorized the training sample?\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "rng = np.random.default_rng(49)\n",
        "X = np.linspace(-1, 1, 200).reshape(-1, 1)\n",
        "y = 2 * X[:, 0] + 0.3 * np.sin(6 * X[:, 0])\n",
        "hidden_weights = rng.normal(size=(1, 12))\n",
        "hidden_bias = rng.normal(scale=0.25, size=12)\n",
        "hidden = np.tanh(X @ hidden_weights + hidden_bias)\n",
        "design = np.column_stack([np.ones(len(X)), hidden])\n",
        "output_weights = np.linalg.lstsq(design, y, rcond=None)[0]\n",
        "fitted = design @ output_weights\n",
        "print(\"Training RMSE:\", round(float(np.sqrt(np.mean((y - fitted) ** 2))), 4))\n",
        "\n",
        "new_x = np.array([[-0.5], [0.0], [0.5]])\n",
        "new_hidden = np.tanh(new_x @ hidden_weights + hidden_bias)\n",
        "new_design = np.column_stack([np.ones(len(new_x)), new_hidden])\n",
        "print(\"Hidden-layer shape:\", hidden.shape)\n",
        "print(\"Predictions:\", np.round(new_design @ output_weights, 3).tolist())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Interpretation check:** Interpretation. The network learns a nonlinear approximation on a small CPU-friendly problem. The exact final loss may vary slightly by software and hardware.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "values = np.array([-2.0, -0.5, 0.0, 0.5, 2.0])\n",
        "relu = np.maximum(values, 0)\n",
        "sigmoid = 1 / (1 + np.exp(-values))\n",
        "print(\"ReLU:\", relu.tolist())\n",
        "print(\"Sigmoid:\", np.round(sigmoid, 3).tolist())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Interpretation check:** Interpretation. Evaluation mode and no_grad disable training-specific behaviour and gradient storage during prediction.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Verified source output\n",
        "\n",
        "```text\n0.01343\n```\n\n```text\n[-1.024999976158142, 0.0, 1.0119999647140503]\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3"
    },
    "ceteris_lab": {
      "course_slug": "fundamentals-python-econometrics",
      "source_derived": true,
      "course_title": "Fundamentals of Python for Financial Econometrics",
      "chapter": 49,
      "source_origin": "source-derived"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}