{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Manipulating, Grouping, Reshaping, and Joining Data\n",
        "\n**Opening question:** How can information be reorganized without accidentally changing the number or meaning of observations?\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "\n",
        "df = pd.DataFrame({\n",
        "    \"province\": [\"ON\", \"ON\", \"QC\", \"QC\"],\n",
        "    \"year\": [2025, 2026, 2025, 2026],\n",
        "    \"income\": [60, 63, 55, 58],\n",
        "})\n",
        "summary = df.groupby(\"province\", as_index=False).agg(mean_income=(\"income\", \"mean\"))\n",
        "print(summary)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Interpretation check:** Interpretation. The result has one row per province, so its unit differs from the original province-year table.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "meta = pd.DataFrame({\"province\": [\"ON\", \"QC\"], \"region\": [\"Central\", \"Central\"]})\n",
        "merged = df.merge(meta, on=\"province\", how=\"left\", validate=\"many_to_one\", indicator=True)\n",
        "print(merged.shape)\n",
        "print(merged[\"_merge\"].value_counts().to_dict())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Interpretation check:** Interpretation. Cardinality validation and the merge indicator confirm that metadata attached without multiplying rows.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Verified source output\n",
        "\n",
        "```text\nprovince mean_income 0 ON 61.5 1 QC 56.5\n```\n\n```text\n(4, 5) {'both': 4, 'left_only': 0, 'right_only': 0}\n```\n\n```text\nprovince  mean_income\n0       ON         61.5\n1       QC         56.5\n```\n\n```text\n(4, 5)\n{'both': 4, 'left_only': 0, 'right_only': 0}\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3"
    },
    "ceteris_lab": {
      "course_slug": "fundamentals-python-econometrics",
      "source_derived": true,
      "course_title": "Fundamentals of Python for Financial Econometrics",
      "chapter": 11,
      "source_origin": "source-derived"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}