{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Natural-Language Processing and Transformers for Economics\n",
        "\n**Opening question:** How can text be converted into numerical representations while preserving enough context to classify or retrieve meaning?\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.metrics.pairwise import cosine_similarity\n",
        "\n",
        "docs = [\"inflation rose after energy prices increased\", \"energy costs pushed inflation higher\", \"the football match ended in a draw\"]\n",
        "X = TfidfVectorizer().fit_transform(docs)\n",
        "print(cosine_similarity(X[0], X).round(3).tolist()[0])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Interpretation check:** Interpretation. The two inflation documents share weighted vocabulary, while the unrelated sports sentence has zero overlap in this tiny corpus.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "texts = [\"flood warning issued\", \"sunny picnic today\", \"earthquake reported\", \"great concert tonight\"]\n",
        "labels = [1, 0, 1, 0]\n",
        "pipe = Pipeline([(\"tfidf\", TfidfVectorizer()), (\"model\", LogisticRegression())]).fit(texts, labels)\n",
        "print(pipe.predict([\"storm warning tonight\"]).tolist())\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "**Interpretation check:** Interpretation. The example demonstrates workflow, not reliable disaster detection. Four training texts are far too few for deployment.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.metrics.pairwise import cosine_similarity\n",
        "X = TfidfVectorizer().fit_transform(docs)\n",
        "print(cosine_similarity(X[0], X).round(3).tolist()[0])\n",
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.linear_model import LogisticRegression\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Verified source output\n",
        "\n",
        "```text\n[1.0, 0.25, 0.0]\n```\n\n```text\n[0]\n```\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3"
    },
    "ceteris_lab": {
      "course_slug": "fundamentals-python-econometrics",
      "source_derived": true,
      "course_title": "Fundamentals of Python for Financial Econometrics",
      "chapter": 51,
      "source_origin": "source-derived"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}