toga-ai 1.0.475 → 1.0.476

package/knowledge/1.0/apps/tools/features/talos-kb-documents-admin.md

@@ -95,6 +95,20 @@ so they displayed **double-encoded** (e.g. "Grand & Toy"). `vocabulary/post.
 `html_entity_decode()`s values on save so the DB stores **raw** text; `vocabulary/get.php`
 already escapes once at display. Do not re-add encoding on save.
 
+**Per-model scoping is STRICT — each model owns its own copy (fixed 2026-07-29).**
+`vocabulary/get.php` reads **strictly** `WHERE c_trueAiModelId = <selectedModel>` with **no
+NULL fallback** — `c_trueAiModelId` is NOT "NULL = global". The per-model scoping (tools #5)
+shipped with **no data backfill**: `Team/2026-07-27a` added `c_trueAiModelId` to
+`TranscriptPromptTerms`/`TranscriptReplacements`/`TranscriptPromptTemplate` as `DEFAULT NULL`
+and nothing populated it, so every existing row was NULL and selecting **any** model showed an
+**empty page** (confirmed prod: 122 terms / 110 replacements / 1 template, all NULL). Fixed by
+`dbchanges2 Team/2026-07-29a` (PR dbchanges2 #444): the existing NULL base rows are assigned to
+**dev-core**, then that base is **replicated into every other active AiModel** (each an
+independent, editable copy with its own fresh UUID). New models added later are **NOT
+auto-seeded** — a known limitation, since a Team migration can't enumerate `Client_True`
+models dynamically (see the separate-clusters gotcha in the
+[worker2 pipeline doc](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md#gotchas--known-issues)).
+
 ### `/talos/knowledge-bases` — KB list + inline rename
 
 The list/admin page (`get.php`) for Talos knowledge bases. Unlike KB Documents (whose KB
@@ -383,6 +397,16 @@ Refactor from a single hard-coded `development-team` KB to per-AI-model, data-dr
 
 ## Change history
 
+- 2026-07-29 — **Fixed the empty Vocabulary page: the per-model scoping shipped with no data
+  backfill.** `vocabulary/get.php` reads strictly `WHERE c_trueAiModelId = <model>` (no NULL
+  fallback), but `Team/2026-07-27a` added the column `DEFAULT NULL` and nothing populated it —
+  so every model showed empty (prod: 122 terms / 110 replacements / 1 template, all NULL). New
+  migration `dbchanges2 Team/2026-07-29a - TranscriptVocabularyBackfillAllModels.sql` (PR
+  dbchanges2 #444) assigns the base NULL rows to **dev-core** then replicates that base into
+  **every active AiModel** as an independent editable copy (fresh UUID each). Decision: vocabulary
+  is **strict per-model, each model owns its own copy** (not NULL=global, not shared);
+  later-added models are **not auto-seeded** (Team migrations can't enumerate `Client_True`
+  models). (ajean)
 - 2026-07-29 — **Refactored KB Documents + Vocabulary to per-AI-model, data-driven scoping**
   (from the single hard-coded `development-team` KB). KB Documents is now an
   `AiModel → VectorIndex → documents` cascade; every document action is tenant-guarded

package/knowledge/2.0/apps/dbchanges2/architecture.md

@@ -6,8 +6,8 @@ project: Database Changes
 client: shared
 type: architecture
 status: active
-updated: 2026-07-21
-owners: [jcardinal, mhammontree, bala]
+updated: 2026-07-29
+owners: [jcardinal, mhammontree, bala, ajean]
 files:
   - Core/
   - Client/
@@ -293,6 +293,44 @@ tables with a `UNIQUE(parentId, childId)` (e.g. `ContactAddresses`) can't be re-
 null-then-delete those. Then delete the non-keeper parents via the materialized double-nested
 subquery. Used in `Client_Prudential/2026-07-07 - Contact Dedup Merge.sql`.
 
+## Self-referencing INSERT...SELECT guard — wrap the existing-set subquery in a derived table
+
+The same MySQL restriction (and a second, subtler hazard) applies when an
+`INSERT ... SELECT` into a table must be made **idempotent / guarded per-key against that
+SAME table** — e.g. "insert a base row for every model that doesn't already have one."
+
+Reading the target table directly in the guard subquery fails two ways:
+
+1. **Error 1093** — "You can't specify target table 'X' for update in FROM clause," exactly as
+   with the self-referencing DELETE above.
+2. **A guard that flips mid-statement.** Even where MySQL allows it, a correlated
+   `NOT IN (SELECT … FROM <target>)` guard can be re-evaluated **as rows are inserted**, so the
+   set it checks against grows during the statement and the guard's answer changes partway
+   through — producing partial or duplicated inserts.
+
+Wrap the existing-set subquery in a **derived table** (with `DISTINCT` or `LIMIT`) so MySQL
+**materializes it once** against the pre-INSERT state:
+
+```sql
+-- CORRECT — the derived table snapshots the "already present" set before any insert
+INSERT INTO TranscriptPromptTerms (uuid, category, term, isActive, c_trueAiModelId)
+SELECT UUID(), t.category, t.term, t.isActive, m.mid
+FROM <base rows> t
+CROSS JOIN <active model ids> m
+WHERE m.mid NOT IN (
+    SELECT mid FROM (
+        SELECT DISTINCT c_trueAiModelId AS mid
+        FROM TranscriptPromptTerms
+        WHERE c_trueAiModelId IN (<active model ids>)
+    ) existing
+);
+```
+
+The extra `SELECT … FROM ( … ) existing` layer both dodges error 1093 and pins the guard to a
+one-time snapshot, so a per-key idempotent backfill stays correct. Used in
+`Team/2026-07-29a - TranscriptVocabularyBackfillAllModels.sql` to replicate the dev-core base
+vocabulary into every other active AI model exactly once.
+
 ## Relationship to the rest of 2.0
 
 `dbchanges2` is registered as a **2.0 core repo** (`role: core` in `registry.json`) — it is
@@ -303,6 +341,11 @@ defined in `2.0/apps/_underscore/architecture.md`, and its change files create/a
 tables that `_Model_*` classes map to.
 
 ## Change history
+- 2026-07-29 — Added *Self-referencing INSERT...SELECT guard — wrap the existing-set subquery in
+  a derived table*: an `INSERT...SELECT` guarded per-key against its own target must wrap the
+  existing-set subquery in a derived table (DISTINCT/LIMIT) so MySQL materializes it once —
+  avoids error 1093 and stops the guard flipping mid-statement as rows are inserted. First used
+  in `Team/2026-07-29a - TranscriptVocabularyBackfillAllModels.sql` (PR #444). (ajean)
 - 2026-07-21 — Added rule #6 to *Adding a new change*: never seed a `uuid` column with MySQL
   `UUID()` (time/MAC-based v1, violates the 2.0 v4-UUID standard); pre-generate a v4 UUID
   literal and hardcode it in `VALUES`. `Core/2026-05-08 … CronJobs_WorkerCleanup_insert` is a

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -29,7 +29,7 @@
 | [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Receives inbound webhook events from Startech (Easeedesk) and creates or updates the corresponding ticket in TOGA 2.0. | worker2/Worker/Startech.php |
 | [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus programmatically. | .claude/skills/plan-ticket/scripts/talos.js |
 | [Talos Pricing Automation (worker2 Cron — AWS Actuals, Calibration, Monthly Report)](features/talos-pricing-automation.md) | The worker2 half of the **Talos Pricing Platform** (see the talos `pricing-cogs-model` and tools `talos-pricing-ui` docs for the other halves). | worker2/Worker/Talos/Pricing.php, worker2/Database/TalosPricingCrons.sql |
-| [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | > **DB-DRIVEN AI-MODEL ROUTING (2026-07-29).** Which knowledge base a transcript is cleaned > into is now decided by the **meeting organizer's "home" AI model** | worker2/Worker/Team/Transcripts.php, worker2/bin/sync-knowledge-bases.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Client_True/2026-07-27a - TranscriptAiModelRoutingColumns.sql, dbchanges2/Client_True/2026-07-27b - TranscriptAiModelRoutingData.sql, dbchanges2/Team/2026-07-27a - TranscriptVocabularyAiModelScope.sql, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a, dbchanges2/Core/2026-07-02a, dbchanges2/Team/2026-07-02a, dbchanges2/Team/2026-07-08a, dbchanges2/Team/2026-07-09a, dbchanges2/Team/2026-07-10a, dbchanges2/Team/2026-07-28a - TranscriptProcessingRetryAttempts.sql, dbchanges2/Team/2026-07-28b - TranscriptPromptTemplateConverseModel.sql, dbchanges2/Core/2026-07-28a - TeamsTranscriptRetryCron.sql |
+| [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | > **DB-DRIVEN AI-MODEL ROUTING (2026-07-29).** Which knowledge base a transcript is cleaned > into is now decided by the **meeting organizer's "home" AI model** | worker2/Worker/Team/Transcripts.php, worker2/bin/sync-knowledge-bases.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Client_True/2026-07-27a - TranscriptAiModelRoutingColumns.sql, dbchanges2/Client_True/2026-07-27b - TranscriptAiModelRoutingData.sql, dbchanges2/Team/2026-07-27a - TranscriptVocabularyAiModelScope.sql, dbchanges2/Team/2026-07-29a - TranscriptVocabularyBackfillAllModels.sql, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a, dbchanges2/Core/2026-07-02a, dbchanges2/Team/2026-07-02a, dbchanges2/Team/2026-07-08a, dbchanges2/Team/2026-07-09a, dbchanges2/Team/2026-07-10a, dbchanges2/Team/2026-07-28a - TranscriptProcessingRetryAttempts.sql, dbchanges2/Team/2026-07-28b - TranscriptPromptTemplateConverseModel.sql, dbchanges2/Core/2026-07-28a - TeamsTranscriptRetryCron.sql |
 | [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php, _underscore/Model/Team/Sprint.php, dbchanges2/Core/CronJobs (SprintLockScheduled seed) |
 | [Teams Meeting Transcript Export](features/teams-transcript-export.md) | > **SUPERSEDED (2026-07-09) — the S3-staging model below is history.** `Export` is now a thin > **GRAPH-DIRECT** cron poller: it no longer archives raw VTT to ` | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
 | [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | `_Worker_Vapi` ([worker2/Worker/Vapi.php](worker2/Worker/Vapi.php)) is the **PHP side of the AI-BDR call loop** — the webhook that receives VAPI's end-of-call r | worker2/Worker/Vapi.php, worker2/Worker/Ai/Bdr/Vapi.php |

package/knowledge/2.0/apps/worker2/features/talos-transcript-ingestion.md

@@ -16,6 +16,7 @@ files:
   - dbchanges2/Client_True/2026-07-27a - TranscriptAiModelRoutingColumns.sql
   - dbchanges2/Client_True/2026-07-27b - TranscriptAiModelRoutingData.sql
   - dbchanges2/Team/2026-07-27a - TranscriptVocabularyAiModelScope.sql
+  - dbchanges2/Team/2026-07-29a - TranscriptVocabularyBackfillAllModels.sql
   - dbchanges2/Team/2026-06-30a
   - dbchanges2/Team/2026-06-30b
   - dbchanges2/Team/2026-06-30c
@@ -297,9 +298,17 @@ spacing (Bedrock allows only one in-flight ingestion per KB, so parallel syncs r
   index selection in `Process`).
 - **`Team.TranscriptPromptTerms` / `TranscriptReplacements` / `TranscriptPromptTemplate`** gained
   **`c_trueAiModelId`** (soft ref → `Client_True.AiModels.id`, dbchanges2 `Team/2026-07-27a`) so
-  the transcript-cleanup vocabulary is scoped per AI model.
+  the transcript-cleanup vocabulary is scoped per AI model. **`c_trueAiModelId` is STRICT
+  per-model, NOT "NULL = global"** — each model owns its own independent copy. The `2026-07-27a`
+  add shipped `DEFAULT NULL` with no backfill, so `dbchanges2 Team/2026-07-29a` (PR #444) seeds
+  it: the base NULL rows → **dev-core**, then that base is replicated into every active AiModel
+  (fresh UUID per copy). Later-added models are **not** auto-seeded (a Team migration can't
+  enumerate `Client_True` models — see the separate-clusters gotcha).
 - The `2026-07-27b` data backfill maps existing dev emails → dev-core model, `ecastellucci` →
   operations model, and each VectorIndex's S3 target.
+- **Active `Client_True.AiModels` (confirmed prod 2026-07-29):** id 1 `hr`, 2 `dev-core`
+  ("Talos DevCore"), 3 `one`, 4 `sales`, 6 `legal`, 7 `contact-center`, 8 `operations`,
+  9 `executive`, 10 `sales-demo`; id 5 `tech-support` is **INACTIVE**.
 - **These `c_`-prefixed columns needed ZERO `_underscore`/model-layer changes** — see the
   framework note under Gotchas.
 
@@ -453,9 +462,37 @@ part of the ingestion loop** (raw reads removed). Credential values live only in
 - **Cross-ACCOUNT S3.** `togaiq` (us-east-1) approved/archive writes use the `[talos]` key;
   `CopyObject` across accounts is impossible, so archive is get(in-memory)+put. (The old
   toga-private cross-account read is no longer in the loop.)
+- **The `Team` DB and `Client_True` live on SEPARATE clusters** (Team on the core cluster,
+  `Client_True` on the client cluster). Therefore a **Team migration cannot resolve a
+  `Client_True.AiModels.id` via a cross-DB subquery** — the two are not queryable in one
+  statement. This is why `c_trueAiModelId` is a **soft cross-DB ref with no FK** (nothing to
+  validate in-DB) and why any Team-side migration that needs a model id must **hardcode the id
+  literal** and document the resolving query + date in the file header (as
+  `Team/2026-07-29a` does). It is also why later-added models can't be auto-seeded with
+  vocabulary from a Team migration.
+- **worker2 reads the cleanup vocabulary GLOBALLY — it is NOT yet scoped by model.**
+  `_Worker_Team_Transcripts::loadActiveTemplate()` / `loadPromptTerms()` /
+  `loadActiveReplacements()` filter only `WHERE isActive = 1`, with **no `c_trueAiModelId`
+  scope**. So today `c_trueAiModelId` is consumed **only by the tools `/talos/vocabulary` admin
+  UI**, not by transcript cleaning — the worker cleans every transcript with whatever the
+  **global** set returns. **FOLLOW-UP (not done):** scope these three loaders by the transcript's
+  `aiModelId`; once models have divergent vocabulary the global read will clean every transcript
+  with the merged/global set instead of the routed model's copy.
 
 ## Change history
 
+- 2026-07-29 — **Backfilled the per-model transcript vocabulary and recorded three durable
+  facts.** `c_trueAiModelId` (added `DEFAULT NULL` by `Team/2026-07-27a`) was never populated, so
+  the tools `/talos/vocabulary` UI (which reads it strictly, no NULL fallback) showed empty for
+  every model. `dbchanges2 Team/2026-07-29a` (PR #444) assigns the base NULL rows to **dev-core**
+  and replicates that base into every active AiModel (independent copy, fresh UUID). Decision:
+  vocabulary is **strict per-model** (not NULL=global, not shared); later-added models are not
+  auto-seeded. Recorded: (1) **Team and `Client_True` are on separate clusters** → Team migrations
+  can't cross-DB-resolve a `Client_True.AiModels.id` and must hardcode the id literal; (2)
+  **worker2 reads vocabulary globally** (`isActive=1` only, no `c_trueAiModelId` scope) — the
+  column is consumed only by the tools UI today, with a follow-up to scope the worker's three
+  vocabulary loaders by the transcript's `aiModelId`; (3) the full active `Client_True.AiModels`
+  set (id 5 `tech-support` inactive). (ajean)
 - 2026-07-29 — **DB-driven per-user AI-model routing.** `Export` now polls
   `Client_True.Users WHERE c_transcriptAiModelId IS NOT NULL AND isActive=1` (the `[teams]`
   organizer id/email lists were removed) and threads `aiModelId` into each `Process` job;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.475",
+  "version": "1.0.476",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",