churnkit 0.75.0a1__py3-none-any.whl

churnkit-0.75.0a1.data/data/share/churnkit/exploration_notebooks/01a_a_temporal_text_deep_dive.ipynb

@@ -0,0 +1,961 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "cell-0",
+   "metadata": {
+    "papermill": {
+     "duration": 0.007095,
+     "end_time": "2026-02-02T01:44:16.460374",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:16.453279",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "# Chapter 1a.a: Temporal Text Columns Deep Dive\n",
+    "\n",
+    "**Purpose:** Transform TEXT columns in event-level data into numeric features, then aggregate across time windows.\n",
+    "\n",
+    "**When to use this notebook:**\n",
+    "- Your dataset is EVENT_LEVEL (time series)\n",
+    "- You have TEXT columns (tickets, messages, emails, etc.)\n",
+    "- Run after 01a_temporal_deep_dive.ipynb\n",
+    "\n",
+    "**Processing Flow:**\n",
+    "```\n",
+    "Event TEXT → Embeddings → PCA → pc1, pc2, ... → Time Window Aggregation\n",
+    "```\n",
+    "\n",
+    "**What you'll learn:**\n",
+    "- How to embed text at the event level\n",
+    "- How to choose between fast vs high-quality embedding models\n",
+    "- How PCA features aggregate across time windows\n",
+    "- Creating features like `ticket_text_pc1_mean_30d`\n",
+    "\n",
+    "**Outputs:**\n",
+    "- PC features per event\n",
+    "- Aggregation plan for PC features\n",
+    "- Updated findings with text processing metadata\n",
+    "\n",
+    "---\n",
+    "\n",
+    "## Two Approaches to Text Feature Engineering\n",
+    "\n",
+    "| Approach | Method | When to Use |\n",
+    "|----------|--------|-------------|\n",
+    "| **1. Embeddings + PCA + Aggregation** (This notebook) | Per-event PCA → aggregate | Temporal patterns in text |\n",
+    "| **2. LLM Labeling** (Future) | LLM labels → categorical aggregation | Specific categories needed |\n",
+    "\n",
+    "### Embedding Model Options\n",
+    "\n",
+    "| Model | Size | Embedding Dim | Speed | Quality | Best For |\n",
+    "|-------|------|---------------|-------|---------|----------|\n",
+    "| **MiniLM** (default) | 90 MB | 384 | Fast | Good | CPU, quick iteration, small datasets |\n",
+    "| **Qwen3-0.6B** | 1.2 GB | 1024 | Medium | Better | GPU available, production quality |\n",
+    "| **Qwen3-4B** | 8 GB | 2560 | Slow | High | 16GB+ GPU, multilingual, high accuracy |\n",
+    "| **Qwen3-8B** | 16 GB | 4096 | Slowest | Highest | 32GB+ GPU, research, max quality |\n",
+    "\n",
+    "**Note:** Models are downloaded on first use. For event-level data with many rows, faster models (MiniLM) are recommended unless you have a powerful GPU.\n",
+    "\n",
+    "### Processing Flow\n",
+    "\n",
+    "```\n",
+    "Per Event:  TEXT → Embedding → [pc1, pc2, pc3]\n",
+    "Aggregate:  customer_id → ticket_text_pc1_mean_30d, ticket_text_pc2_std_7d, ...\n",
+    "```\n",
+    "\n",
+    "This captures how text semantics change over time windows."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-1",
+   "metadata": {
+    "papermill": {
+     "duration": 0.005664,
+     "end_time": "2026-02-02T01:44:16.472356",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:16.466692",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.1 Load Previous Findings"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-2",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:16.484376Z",
+     "iopub.status.busy": "2026-02-02T01:44:16.484246Z",
+     "iopub.status.idle": "2026-02-02T01:44:18.851167Z",
+     "shell.execute_reply": "2026-02-02T01:44:18.850589Z"
+    },
+    "papermill": {
+     "duration": 2.373857,
+     "end_time": "2026-02-02T01:44:18.852091",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:16.478234",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from customer_retention.analysis.notebook_progress import track_and_export_previous\n",
+    "track_and_export_previous(\"01a_a_temporal_text_deep_dive.ipynb\")\n",
+    "\n",
+    "from customer_retention.analysis.auto_explorer import ExplorationFindings, TextProcessingMetadata\n",
+    "from customer_retention.analysis.visualization import ChartBuilder, display_figure, display_table, console\n",
+    "from customer_retention.core.config.column_config import ColumnType, DatasetGranularity\n",
+    "from customer_retention.stages.profiling import (\n",
+    "    TextColumnProcessor, TextProcessingConfig, TextColumnResult,\n",
+    "    TimeWindowAggregator, AggregationPlan,\n",
+    "    EMBEDDING_MODELS, get_model_info, list_available_models\n",
+    ")\n",
+    "import pandas as pd\n",
+    "import numpy as np\n",
+    "import plotly.graph_objects as go\n",
+    "import plotly.express as px\n",
+    "from plotly.subplots import make_subplots\n",
+    "from customer_retention.core.config.experiments import FINDINGS_DIR, EXPERIMENTS_DIR, OUTPUT_DIR, setup_experiments_structure"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-3",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:18.863269Z",
+     "iopub.status.busy": "2026-02-02T01:44:18.863126Z",
+     "iopub.status.idle": "2026-02-02T01:44:18.888705Z",
+     "shell.execute_reply": "2026-02-02T01:44:18.888264Z"
+    },
+    "papermill": {
+     "duration": 0.032282,
+     "end_time": "2026-02-02T01:44:18.889776",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:18.857494",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# === CONFIGURATION ===\n",
+    "from pathlib import Path\n",
+    "\n",
+    "# FINDINGS_DIR imported from customer_retention.core.config.experiments\n",
+    "\n",
+    "findings_files = [f for f in FINDINGS_DIR.glob(\"*_findings.yaml\") if \"multi_dataset\" not in f.name]\n",
+    "if not findings_files:\n",
+    "    raise FileNotFoundError(f\"No findings files found in {FINDINGS_DIR}. Run notebook 01 first.\")\n",
+    "\n",
+    "findings_files.sort(key=lambda f: f.stat().st_mtime, reverse=True)\n",
+    "FINDINGS_PATH = str(findings_files[0])\n",
+    "\n",
+    "print(f\"Found {len(findings_files)} findings file(s)\")\n",
+    "print(f\"Using: {FINDINGS_PATH}\")\n",
+    "\n",
+    "findings = ExplorationFindings.load(FINDINGS_PATH)\n",
+    "print(f\"\\nLoaded findings for {findings.column_count} columns from {findings.source_path}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-4",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:18.900980Z",
+     "iopub.status.busy": "2026-02-02T01:44:18.900867Z",
+     "iopub.status.idle": "2026-02-02T01:44:18.904105Z",
+     "shell.execute_reply": "2026-02-02T01:44:18.903459Z"
+    },
+    "papermill": {
+     "duration": 0.009642,
+     "end_time": "2026-02-02T01:44:18.904756",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:18.895114",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# Verify this is a time series dataset\n",
+    "# This notebook is ONLY for event-level (time series) data with multiple rows per entity\n",
+    "\n",
+    "if not findings.is_time_series:\n",
+    "    print(\"=\" * 70)\n",
+    "    print(\"WRONG NOTEBOOK FOR THIS DATASET\")\n",
+    "    print(\"=\" * 70)\n",
+    "    print()\n",
+    "    print(\"This dataset is ENTITY-LEVEL (one row per entity), not event-level.\")\n",
+    "    print()\n",
+    "    print(\"For TEXT columns in entity-level data, use:\")\n",
+    "    print(\"   02a_text_columns_deep_dive.ipynb\")\n",
+    "    print()\n",
+    "    print(\"This notebook (01a_a) is for TEXT columns in EVENT-LEVEL data where:\")\n",
+    "    print(\"   - Multiple events per entity (e.g., support tickets, transactions)\")\n",
+    "    print(\"   - Text is embedded per-event, then aggregated across time windows\")\n",
+    "    print()\n",
+    "    raise SystemExit(\"Please use 02a_text_columns_deep_dive.ipynb for entity-level data.\")\n",
+    "\n",
+    "ts_meta = findings.time_series_metadata\n",
+    "temporal_pattern = (ts_meta.temporal_pattern or \"unknown\").upper()\n",
+    "print(f\"Dataset confirmed as {temporal_pattern} (event-level)\")\n",
+    "ENTITY_COLUMN = ts_meta.entity_column\n",
+    "TIME_COLUMN = ts_meta.time_column\n",
+    "print(f\"   Entity column: {ENTITY_COLUMN}\")\n",
+    "print(f\"   Time column: {TIME_COLUMN}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-5",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:18.917113Z",
+     "iopub.status.busy": "2026-02-02T01:44:18.916992Z",
+     "iopub.status.idle": "2026-02-02T01:44:18.919898Z",
+     "shell.execute_reply": "2026-02-02T01:44:18.919272Z"
+    },
+    "papermill": {
+     "duration": 0.009896,
+     "end_time": "2026-02-02T01:44:18.920517",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:18.910621",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# Identify TEXT columns\n",
+    "text_columns = [\n",
+    "    name for name, col in findings.columns.items()\n",
+    "    if col.inferred_type == ColumnType.TEXT\n",
+    "]\n",
+    "\n",
+    "if not text_columns:\n",
+    "    print(\"\\u26a0\\ufe0f No TEXT columns detected in this dataset.\")\n",
+    "    print(\"   This notebook is only needed when TEXT columns are present.\")\n",
+    "    print(\"   Continue to notebook 01b_temporal_quality.ipynb\")\n",
+    "else:\n",
+    "    print(f\"\\u2705 Found {len(text_columns)} TEXT column(s):\")\n",
+    "    for col in text_columns:\n",
+    "        col_info = findings.columns[col]\n",
+    "        print(f\"   - {col} (Confidence: {col_info.confidence:.0%})\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-6",
+   "metadata": {
+    "papermill": {
+     "duration": 0.005389,
+     "end_time": "2026-02-02T01:44:18.931448",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:18.926059",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.2 Load Source Data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-7",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:18.943724Z",
+     "iopub.status.busy": "2026-02-02T01:44:18.943564Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.057178Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.056794Z"
+    },
+    "papermill": {
+     "duration": 0.121646,
+     "end_time": "2026-02-02T01:44:19.058713",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:18.937067",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "from customer_retention.stages.temporal import load_data_with_snapshot_preference, TEMPORAL_METADATA_COLS\n",
+    "\n",
+    "df, data_source = load_data_with_snapshot_preference(findings, output_dir=str(FINDINGS_DIR))\n",
+    "charts = ChartBuilder()\n",
+    "\n",
+    "print(f\"Loaded {len(df):,} events x {len(df.columns)} columns\")\n",
+    "print(f\"Data source: {data_source}\")\n",
+    "print(f\"Unique entities: {df[ENTITY_COLUMN].nunique():,}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-8",
+   "metadata": {
+    "papermill": {
+     "duration": 0.005568,
+     "end_time": "2026-02-02T01:44:19.069912",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.064344",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.3 Configuration\n",
+    "\n",
+    "### Available Embedding Models"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-9",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.081720Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.081594Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.085178Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.084432Z"
+    },
+    "papermill": {
+     "duration": 0.011809,
+     "end_time": "2026-02-02T01:44:19.087109",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.075300",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# Display available embedding models\n",
+    "print(\"Available Embedding Models\")\n",
+    "print(\"=\" * 80)\n",
+    "print(f\"{'Preset':<15} {'Model':<35} {'Size':<10} {'Dim':<8} {'GPU?'}\")\n",
+    "print(\"-\" * 80)\n",
+    "\n",
+    "for preset in list_available_models():\n",
+    "    info = get_model_info(preset)\n",
+    "    size = f\"{info['size_mb']} MB\" if info['size_mb'] < 1000 else f\"{info['size_mb']/1000:.1f} GB\"\n",
+    "    gpu = \"Yes\" if info['gpu_recommended'] else \"No\"\n",
+    "    print(f\"{preset:<15} {info['model_name']:<35} {size:<10} {info['embedding_dim']:<8} {gpu}\")\n",
+    "\n",
+    "print(\"\\nFor event-level data with many rows, MiniLM is recommended for faster processing.\")\n",
+    "print(\"Qwen3 models produce higher quality embeddings but require GPU for reasonable speed.\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bc5ekeprioi",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.101537Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.101409Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.105129Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.104551Z"
+    },
+    "papermill": {
+     "duration": 0.01134,
+     "end_time": "2026-02-02T01:44:19.105873",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.094533",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "# === TEXT PROCESSING CONFIGURATION ===\n",
+    "# Choose your embedding model preset:\n",
+    "#   \"minilm\"     - Fast, CPU-friendly, recommended for event-level data (default)\n",
+    "#   \"qwen3-0.6b\" - Better quality, needs GPU\n",
+    "#   \"qwen3-4b\"   - High quality, needs 16GB+ GPU\n",
+    "#   \"qwen3-8b\"   - Highest quality, needs 32GB+ GPU\n",
+    "\n",
+    "EMBEDDING_PRESET = \"minilm\"  # Recommended for event-level data\n",
+    "\n",
+    "# PCA configuration (capped at 10 for manageability in aggregation)\n",
+    "VARIANCE_THRESHOLD = 0.95  # Keep components explaining 95% of variance\n",
+    "MIN_COMPONENTS = 2         # At least 2 features per text column\n",
+    "MAX_COMPONENTS = 10        # Cap at 10 to keep aggregation manageable\n",
+    "\n",
+    "# Aggregation configuration\n",
+    "AGGREGATION_WINDOWS = [\"7d\", \"30d\", \"90d\", \"all_time\"]\n",
+    "AGGREGATION_FUNCS = [\"mean\", \"std\", \"first\", \"last\"]\n",
+    "\n",
+    "# Create configuration\n",
+    "model_info = get_model_info(EMBEDDING_PRESET)\n",
+    "text_config = TextProcessingConfig(\n",
+    "    embedding_model=model_info[\"model_name\"],\n",
+    "    variance_threshold=VARIANCE_THRESHOLD,\n",
+    "    max_components=MAX_COMPONENTS,\n",
+    "    min_components=MIN_COMPONENTS,\n",
+    "    batch_size=32\n",
+    ")\n",
+    "\n",
+    "print(\"Text Processing Configuration\")\n",
+    "print(\"=\" * 50)\n",
+    "print(f\"  Preset: {EMBEDDING_PRESET}\")\n",
+    "print(f\"  Model: {text_config.embedding_model}\")\n",
+    "print(f\"  Model size: {model_info['size_mb']} MB\")\n",
+    "print(f\"  Embedding dimension: {model_info['embedding_dim']}\")\n",
+    "print(f\"  GPU recommended: {'Yes' if model_info['gpu_recommended'] else 'No'}\")\n",
+    "print()\n",
+    "print(f\"  Variance threshold: {text_config.variance_threshold:.0%}\")\n",
+    "print(f\"  Max components: {text_config.max_components}\")\n",
+    "print()\n",
+    "print(\"Aggregation Configuration\")\n",
+    "print(\"=\" * 50)\n",
+    "print(f\"  Windows: {AGGREGATION_WINDOWS}\")\n",
+    "print(f\"  Functions: {AGGREGATION_FUNCS}\")\n",
+    "\n",
+    "if model_info['gpu_recommended']:\n",
+    "    print()\n",
+    "    print(\"Warning: This model works best with GPU. Consider 'minilm' for faster processing.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-10",
+   "metadata": {
+    "papermill": {
+     "duration": 0.011381,
+     "end_time": "2026-02-02T01:44:19.123406",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.112025",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.4 Text Column Analysis"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-11",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.143004Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.142803Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.148785Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.148008Z"
+    },
+    "papermill": {
+     "duration": 0.014313,
+     "end_time": "2026-02-02T01:44:19.149575",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.135262",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "if text_columns:\n",
+    "    for col_name in text_columns:\n",
+    "        print(f\"\\n{'='*70}\")\n",
+    "        print(f\"Column: {col_name}\")\n",
+    "        print(f\"{'='*70}\")\n",
+    "        \n",
+    "        text_series = df[col_name].fillna(\"\")\n",
+    "        \n",
+    "        non_empty = (text_series.str.len() > 0).sum()\n",
+    "        avg_length = text_series.str.len().mean()\n",
+    "        \n",
+    "        print(f\"\\n\\U0001f4ca Statistics:\")\n",
+    "        print(f\"   Total events: {len(text_series):,}\")\n",
+    "        print(f\"   Non-empty: {non_empty:,} ({non_empty/len(text_series)*100:.1f}%)\")\n",
+    "        print(f\"   Avg length: {avg_length:.0f} characters\")\n",
+    "        \n",
+    "        # Texts per entity\n",
+    "        texts_per_entity = df.groupby(ENTITY_COLUMN)[col_name].apply(\n",
+    "            lambda x: (x.fillna(\"\").str.len() > 0).sum()\n",
+    "        )\n",
+    "        print(f\"\\n\\U0001f465 Text events per entity:\")\n",
+    "        print(f\"   Mean: {texts_per_entity.mean():.1f}\")\n",
+    "        print(f\"   Median: {texts_per_entity.median():.0f}\")\n",
+    "        print(f\"   Max: {texts_per_entity.max():,}\")\n",
+    "        \n",
+    "        # Sample texts\n",
+    "        print(f\"\\n\\U0001f4dd Sample texts:\")\n",
+    "        samples = text_series[text_series.str.len() > 10].head(3)\n",
+    "        for i, sample in enumerate(samples, 1):\n",
+    "            truncated = sample[:80] + \"...\" if len(sample) > 80 else sample\n",
+    "            print(f\"   {i}. {truncated}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-12",
+   "metadata": {
+    "papermill": {
+     "duration": 0.007406,
+     "end_time": "2026-02-02T01:44:19.164113",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.156707",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.5 Process Text Columns (Per-Event Embeddings)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-13",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.177537Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.177372Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.180490Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.179931Z"
+    },
+    "papermill": {
+     "duration": 0.011249,
+     "end_time": "2026-02-02T01:44:19.181334",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.170085",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "if text_columns and findings.is_time_series:\n",
+    "    processor = TextColumnProcessor(text_config)\n",
+    "    \n",
+    "    print(\"Processing TEXT columns...\")\n",
+    "    print(\"(This may take a moment for large datasets)\\n\")\n",
+    "    \n",
+    "    results = []\n",
+    "    df_with_pcs = df.copy()\n",
+    "    \n",
+    "    for col_name in text_columns:\n",
+    "        print(f\"\\n{'='*70}\")\n",
+    "        print(f\"Processing: {col_name}\")\n",
+    "        print(f\"{'='*70}\")\n",
+    "        \n",
+    "        df_with_pcs, result = processor.process_column(df_with_pcs, col_name)\n",
+    "        results.append(result)\n",
+    "        \n",
+    "        print(f\"\\n\\u2705 Per-event processing complete:\")\n",
+    "        print(f\"   Components: {result.n_components}\")\n",
+    "        print(f\"   Explained variance: {result.explained_variance:.1%}\")\n",
+    "        print(f\"   Features: {', '.join(result.component_columns)}\")\n",
+    "    \n",
+    "    print(f\"\\n\\nDataFrame now has {len(df_with_pcs.columns)} columns (added {len(df_with_pcs.columns) - len(df.columns)} PC columns)\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-14",
+   "metadata": {
+    "papermill": {
+     "duration": 0.006049,
+     "end_time": "2026-02-02T01:44:19.193144",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.187095",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.6 Plan Time Window Aggregation\n",
+    "\n",
+    "PC features will be aggregated across time windows to create entity-level features.\n",
+    "\n",
+    "**Example output features:**\n",
+    "- `ticket_text_pc1_mean_7d` - Average of PC1 over last 7 days\n",
+    "- `ticket_text_pc2_std_30d` - Standard deviation of PC2 over last 30 days"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-15",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.206713Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.206571Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.209944Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.209398Z"
+    },
+    "papermill": {
+     "duration": 0.011853,
+     "end_time": "2026-02-02T01:44:19.210725",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.198872",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "if text_columns and findings.is_time_series and results:\n",
+    "    # Collect all PC columns\n",
+    "    all_pc_columns = []\n",
+    "    for result in results:\n",
+    "        all_pc_columns.extend(result.component_columns)\n",
+    "    \n",
+    "    print(f\"\\n{'='*70}\")\n",
+    "    print(\"AGGREGATION PLAN\")\n",
+    "    print(f\"{'='*70}\")\n",
+    "    \n",
+    "    aggregator = TimeWindowAggregator(ENTITY_COLUMN, TIME_COLUMN)\n",
+    "    plan = aggregator.generate_plan(\n",
+    "        df_with_pcs,\n",
+    "        windows=AGGREGATION_WINDOWS,\n",
+    "        value_columns=all_pc_columns,\n",
+    "        agg_funcs=AGGREGATION_FUNCS,\n",
+    "        include_event_count=False,\n",
+    "        include_recency=False\n",
+    "    )\n",
+    "    \n",
+    "    print(f\"\\n\\U0001f4ca Plan Summary:\")\n",
+    "    print(f\"   Entity column: {plan.entity_column}\")\n",
+    "    print(f\"   Time column: {plan.time_column}\")\n",
+    "    print(f\"   Windows: {[w.name for w in plan.windows]}\")\n",
+    "    print(f\"   Value columns: {len(plan.value_columns)}\")\n",
+    "    print(f\"   Aggregation functions: {plan.agg_funcs}\")\n",
+    "    print(f\"   Total features to create: {len(plan.feature_columns)}\")\n",
+    "    \n",
+    "    print(f\"\\n\\U0001f4dd Sample feature names:\")\n",
+    "    for feat in plan.feature_columns[:10]:\n",
+    "        print(f\"   - {feat}\")\n",
+    "    if len(plan.feature_columns) > 10:\n",
+    "        print(f\"   ... and {len(plan.feature_columns) - 10} more\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-16",
+   "metadata": {
+    "papermill": {
+     "duration": 0.006214,
+     "end_time": "2026-02-02T01:44:19.223400",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.217186",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.7 Visualize PC Distributions"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-17",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.236409Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.236281Z",
+     "iopub.status.idle": "2026-02-02T01:44:19.239879Z",
+     "shell.execute_reply": "2026-02-02T01:44:19.239304Z"
+    },
+    "papermill": {
+     "duration": 0.011111,
+     "end_time": "2026-02-02T01:44:19.240565",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.229454",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "if text_columns and results:\n",
+    "    for result in results:\n",
+    "        print(f\"\\n{'='*70}\")\n",
+    "        print(f\"PC Feature Distributions: {result.column_name}\")\n",
+    "        print(f\"{'='*70}\")\n",
+    "        \n",
+    "        # Distribution of PC1 and PC2\n",
+    "        if len(result.component_columns) >= 2:\n",
+    "            fig = make_subplots(rows=1, cols=2,\n",
+    "                                subplot_titles=(result.component_columns[0], result.component_columns[1]))\n",
+    "            \n",
+    "            fig.add_trace(go.Histogram(\n",
+    "                x=df_with_pcs[result.component_columns[0]],\n",
+    "                nbinsx=50, marker_color='steelblue', opacity=0.7\n",
+    "            ), row=1, col=1)\n",
+    "            \n",
+    "            fig.add_trace(go.Histogram(\n",
+    "                x=df_with_pcs[result.component_columns[1]],\n",
+    "                nbinsx=50, marker_color='coral', opacity=0.7\n",
+    "            ), row=1, col=2)\n",
+    "            \n",
+    "            fig.update_layout(\n",
+    "                title=f\"PC Feature Distributions: {result.column_name}\",\n",
+    "                height=350, template=\"plotly_white\", showlegend=False\n",
+    "            )\n",
+    "            display_figure(fig)\n",
+    "        \n",
+    "        # Scatter plot of PC1 vs PC2\n",
+    "        if len(result.component_columns) >= 2:\n",
+    "            fig = px.scatter(\n",
+    "                df_with_pcs.sample(min(5000, len(df_with_pcs))),\n",
+    "                x=result.component_columns[0],\n",
+    "                y=result.component_columns[1],\n",
+    "                title=f\"PC1 vs PC2 (sample): {result.column_name}\",\n",
+    "                opacity=0.4\n",
+    "            )\n",
+    "            fig.update_layout(template=\"plotly_white\", height=400)\n",
+    "            display_figure(fig)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-18",
+   "metadata": {
+    "papermill": {
+     "duration": 0.006033,
+     "end_time": "2026-02-02T01:44:19.252472",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.246439",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.8 Update Findings"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-19",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:19.265553Z",
+     "iopub.status.busy": "2026-02-02T01:44:19.265406Z",
+     "iopub.status.idle": "2026-02-02T01:44:21.946608Z",
+     "shell.execute_reply": "2026-02-02T01:44:21.946004Z"
+    },
+    "papermill": {
+     "duration": 2.688722,
+     "end_time": "2026-02-02T01:44:21.947559",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:19.258837",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "if text_columns and results:\n",
+    "    for result in results:\n",
+    "        metadata = TextProcessingMetadata(\n",
+    "            column_name=result.column_name,\n",
+    "            embedding_model=text_config.embedding_model,\n",
+    "            embedding_dim=result.embeddings_shape[1],\n",
+    "            n_components=result.n_components,\n",
+    "            explained_variance=result.explained_variance,\n",
+    "            component_columns=result.component_columns,\n",
+    "            variance_threshold_used=text_config.variance_threshold,\n",
+    "            processing_approach=\"pca\"\n",
+    "        )\n",
+    "        findings.text_processing[result.column_name] = metadata\n",
+    "        \n",
+    "        print(f\"\\u2705 Added text processing metadata for {result.column_name}\")\n",
+    "    \n",
+    "    findings.save(FINDINGS_PATH)\n",
+    "    print(f\"\\nFindings saved to: {FINDINGS_PATH}\")\n",
+    "\n",
+    "from customer_retention.analysis.notebook_html_exporter import export_notebook_html\n",
+    "export_notebook_html(Path(\"01a_a_temporal_text_deep_dive.ipynb\"), EXPERIMENTS_DIR / \"docs\")\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-20",
+   "metadata": {
+    "papermill": {
+     "duration": 0.006498,
+     "end_time": "2026-02-02T01:44:21.960437",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:21.953939",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "## 1a.a.9 Production Recommendations"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "cell-21",
+   "metadata": {
+    "execution": {
+     "iopub.execute_input": "2026-02-02T01:44:21.975562Z",
+     "iopub.status.busy": "2026-02-02T01:44:21.975429Z",
+     "iopub.status.idle": "2026-02-02T01:44:21.978458Z",
+     "shell.execute_reply": "2026-02-02T01:44:21.977893Z"
+    },
+    "papermill": {
+     "duration": 0.011763,
+     "end_time": "2026-02-02T01:44:21.979243",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:21.967480",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "outputs": [],
+   "source": [
+    "if text_columns and results:\n",
+    "    print(\"\\n\" + \"=\"*70)\n",
+    "    print(\"PRODUCTION PIPELINE RECOMMENDATIONS\")\n",
+    "    print(\"=\"*70)\n",
+    "    \n",
+    "    print(\"\\n\\U0001f527 Bronze Layer (per-event processing):\")\n",
+    "    for result in results:\n",
+    "        print(f\"\\n   {result.column_name}:\")\n",
+    "        print(f\"     Action: embed_reduce\")\n",
+    "        print(f\"     Model: {text_config.embedding_model}\")\n",
+    "        print(f\"     Components: {result.n_components}\")\n",
+    "        print(f\"     Output: {', '.join(result.component_columns[:3])}...\")\n",
+    "    \n",
+    "    print(\"\\n\\U0001f527 Silver Layer (entity aggregation):\")\n",
+    "    print(f\"   Windows: {AGGREGATION_WINDOWS}\")\n",
+    "    print(f\"   Functions: {AGGREGATION_FUNCS}\")\n",
+    "    print(f\"   Example features:\")\n",
+    "    for result in results[:1]:\n",
+    "        pc1 = result.component_columns[0]\n",
+    "        for window in AGGREGATION_WINDOWS[:2]:\n",
+    "            for func in AGGREGATION_FUNCS[:2]:\n",
+    "                print(f\"     - {pc1}_{func}_{window}\")\n",
+    "    \n",
+    "    print(\"\\n\\U0001f4a1 The pipeline generator will create these transformations automatically.\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cell-22",
+   "metadata": {
+    "papermill": {
+     "duration": 0.006461,
+     "end_time": "2026-02-02T01:44:21.994543",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:21.988082",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "---\n",
+    "\n",
+    "## Summary\n",
+    "\n",
+    "In this notebook, we:\n",
+    "\n",
+    "1. **Analyzed** TEXT columns in event-level data\n",
+    "2. **Generated per-event embeddings** using sentence-transformers\n",
+    "3. **Applied PCA** to reduce dimensions\n",
+    "4. **Planned aggregation** across time windows\n",
+    "5. **Updated findings** with processing metadata\n",
+    "\n",
+    "## Processing Flow\n",
+    "\n",
+    "```\n",
+    "Event TEXT → Embeddings (384-dim) → PCA (N components) → Aggregate by entity+window\n",
+    "```\n",
+    "\n",
+    "## Example Output Features\n",
+    "\n",
+    "For a `ticket_text` column with 3 PC components and 4 time windows:\n",
+    "- `ticket_text_pc1_mean_7d`, `ticket_text_pc1_std_7d`, ...\n",
+    "- `ticket_text_pc2_mean_7d`, `ticket_text_pc2_std_7d`, ...\n",
+    "- Total: 3 PCs × 4 windows × 4 functions = 48 features\n",
+    "\n",
+    "---\n",
+    "\n",
+    "## Next Steps\n",
+    "\n",
+    "Continue with the **Event Bronze Track**:\n",
+    "\n",
+    "1. **01b_temporal_quality.ipynb** - Check for duplicate events, temporal gaps\n",
+    "2. **01c_temporal_patterns.ipynb** - Detect trends, seasonality\n",
+    "3. **01d_event_aggregation.ipynb** - Aggregate all features (including text PCs) to entity-level"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "48dncdcdryw",
+   "metadata": {
+    "papermill": {
+     "duration": 0.006469,
+     "end_time": "2026-02-02T01:44:22.007428",
+     "exception": false,
+     "start_time": "2026-02-02T01:44:22.000959",
+     "status": "completed"
+    },
+    "tags": []
+   },
+   "source": [
+    "> **Save Reminder:** Save this notebook (Ctrl+S / Cmd+S) before running the next one.\n",
+    "> The next notebook will automatically export this notebook's HTML documentation from the saved file."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.4"
+  },
+  "papermill": {
+   "default_parameters": {},
+   "duration": 8.956491,
+   "end_time": "2026-02-02T01:44:24.631716",
+   "environment_variables": {},
+   "exception": null,
+   "input_path": "/Users/Vital/python/CustomerRetention/exploration_notebooks/01a_a_temporal_text_deep_dive.ipynb",
+   "output_path": "/Users/Vital/python/CustomerRetention/exploration_notebooks/01a_a_temporal_text_deep_dive.ipynb",
+   "parameters": {},
+   "start_time": "2026-02-02T01:44:15.675225",
+   "version": "2.6.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}