toga-ai 1.0.470 → 1.0.472

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -3,12 +3,13 @@
3
3
  | Doc | Summary | Files |
4
4
  |-----|---------|-------|
5
5
  | [Tools (1.0 Internal-Tools App) Architecture](architecture.md) | **Tools** is a standalone 1.0 (`App_`) application that houses many small internal tools behind simple interfaces, gated by Client_True staff persona. | tools/index.php, tools/_/app/framework.php, tools/_/app/frameworkindex.php, tools/assets/img/favicon/favicon.ico, tools/assets/img/favicon/favicon-32x32.png, tools/assets/img/favicon/favicon-16x16.png, tools/assets/img/favicon/apple-touch-icon.png, tools/_/app/auth.php, tools/_/app/nav.php, tools/common/header.php, tools/common/footer.php, tools/mvc/get.php, tools/mvc/_TEMPLATE/get.php, tools/docs/ADDING_A_TOOL.md |
6
+ | [ClickUp Sprint Dashboard (tools /clickup/react)](features/clickup-sprint-dashboard.md) | A native **ClickUp sprint dashboard** in the 1.0 `tools` app at route `/clickup/react`, built for a wall-mounted **TV display** ("TOGa IQ" internal sprint metri | tools/_/app/clickup/sprint.php, tools/v2/sprints/tile/index.php, tools/v2/sprints/current/index.php, tools/v2/sprints/status-breakdown/index.php, tools/v2/sprints/worktype-breakdown/index.php, tools/v2/sprints/points-by-dev/index.php, tools/v2/sprints/burndown/index.php, tools/assets/clickup/sprint-dashboard.html, tools/mvc/clickup/react/get.php, tools/mvc/login/get.php, tools/_/app/nav.php |
6
7
  | [CloudFront Client Setup](features/cloudfront-client-setup.md) | An SSO-gated admin tool at **`/devops/cloudfront-clients`** in the Tools 1.0 app that onboards a client onto **CloudFront + Route 53 across multiple AWS account | tools/_/app/devops/cloudfront.php, tools/mvc/devops/cloudfront-clients/get.php, tools/mvc/devops/cloudfront-clients/post.php, tools/assets/js/cloudfront-clients.js, tools/assets/css/cloudfront-clients.css, tools/_/app/nav.php, tools/_/app/frameworkindex.php, tools/config.production.ini |
7
8
  | [Design Demo Admin](features/design-demo-admin.md) | A self-serve admin UI at **`/design`** in the SSO-protected **Tools** app that lets the design team publish self-contained "Claude Design" HTML exports as **ver | tools/_/app/design/github.php, tools/mvc/design/get.php, tools/mvc/design/post.php, tools/assets/css/design.css, tools/assets/js/design.js, tools/_/app/frameworkindex.php, tools/_/app/nav.php, tools/composer.json |
8
9
  | [Tools — Developers Folder (UUID & Password Generators)](features/developer-tools.md) | The first two tools shipped in the Tools app, both under the **Developers** folder and gated to personas **Development Team** / **TOGa Technology**. | tools/mvc/developers/uuid/get.php, tools/mvc/developers/password/get.php |
9
10
  | [Tools MVC — Routing, CSRF & App_Database Access Patterns](features/mvc-data-access-patterns.md) | The load-bearing 1.0 (`App_`) framework conventions a developer needs when adding a page to the Tools app — URL routing, CSRF, and DB access through `App_Databa | tools/_/app/nav.php, tools/mvc/get.php |
10
11
  | [Tools Persona-Gated Navigation (App_Nav)](features/persona-gated-navigation.md) | `App_Nav` is the Tools app's two-level, **persona-gated** navigation. | tools/_/app/nav.php, tools/mvc/get.php |
11
12
  | [Tools SAML SSO Consumer & Persona-Gated Auth (App_Auth)](features/saml-sso-auth.md) | `App_Auth` is the Tools app's authentication layer: it consumes the SAML gateway `?saml=` handoff (see the 2.0 SAML downstream integration contract), establishe | tools/_/app/auth.php, tools/mvc/sso/initiate/get.php, tools/mvc/sso/get.php, tools/mvc/login/get.php, tools/mvc/login/post.php, tools/mvc/logout/get.php, tools/mvc/get.php, tools/config.production.ini, tools/config.local.ini |
12
- | [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | A Tools (1.0) admin UI to browse/fix the Talos knowledge-base documents and manage the transcript-cleanup vocabulary — without a deploy. | tools/mvc/talos/kb-documents/get.php, tools/mvc/talos/kb-documents/post.php, tools/mvc/talos/knowledge-bases/get.php, tools/mvc/talos/knowledge-bases/post.php, tools/mvc/talos/vocabulary/get.php, tools/mvc/talos/vocabulary/post.php, tools/_/app/talos/s3.php, tools/_/app/talos/bedrock.php, tools/_/app/pg.php, tools/_/app/worker.php, tools/_/app/nav.php, tools/config.production.ini, tools/config.alpha.ini, tools/.platform/httpd/conf.d/timeouts.conf, tools/.platform/hooks/prebuild/01-install-php-pgsql.sh, tools/.platform/hooks/postdeploy/01-restart-php.sh |
13
+ | [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | > **PER-AI-MODEL, DATA-DRIVEN SCOPING (2026-07-29).** The KB-documents and Vocabulary admin > UIs were refactored from a single hard-coded **"development-team"* | tools/mvc/talos/kb-documents/get.php, tools/mvc/talos/kb-documents/post.php, tools/mvc/talos/knowledge-bases/get.php, tools/mvc/talos/knowledge-bases/post.php, tools/mvc/talos/vocabulary/get.php, tools/mvc/talos/vocabulary/post.php, tools/_/app/talos/s3.php, tools/_/app/talos/bedrock.php, tools/_/app/pg.php, tools/_/app/model/true/aimodels.php, tools/_/app/model/true/vectorindexes.php, tools/_/app/model/true/aimodels_vectorindexes.php, tools/_/app/worker.php, tools/_/app/nav.php, tools/config.production.ini, tools/config.alpha.ini, tools/.platform/httpd/conf.d/timeouts.conf, tools/.platform/hooks/prebuild/01-install-php-pgsql.sh, tools/.platform/hooks/postdeploy/01-restart-php.sh |
13
14
  | [Talos Pricing UI (Onboarding, Dashboard, Benchmarks, Cost Factors + Estimator)](features/talos-pricing-ui.md) | The 1.0 (tools app) face of the **Talos Pricing Platform** — a "Talos Pricing" nav folder with four pages plus a client-side estimate engine. | tools/_/app/nav.php, tools/_/app/talos/estimator.php, tools/mvc/talos/onboarding/get.php, tools/mvc/talos/onboarding/post.php, tools/mvc/talos/pricing/get.php, tools/mvc/talos/benchmarks/get.php, tools/mvc/talos/factors/get.php, tools/mvc/talos/factors/post.php, tools/assets/css/style.css |
14
15
  | [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the **Tools** 1.0 app boots on Elastic Beanstalk running `PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. | tools/.ebextensions/004_http_to_https.config, tools/.ebextensions/006_mount-s3fs.config, tools/.ebextensions/007_setup_export_cache_folders.config, tools/.ebextensions/008_setup_ldap.config, tools/.ebextensions/009_setup_phpini.config, tools/.ebextensions/020_setup_git_libraries.config, tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config, tools/ebs/git.json |
@@ -0,0 +1,142 @@
1
+ ---
2
+ title: ClickUp Sprint Dashboard (tools /clickup/react)
3
+ framework: "1.0"
4
+ repo: tools
5
+ project: Tools
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-07-29
10
+ owners: [kyalamarthi]
11
+ files:
12
+ - tools/_/app/clickup/sprint.php
13
+ - tools/v2/sprints/tile/index.php
14
+ - tools/v2/sprints/current/index.php
15
+ - tools/v2/sprints/status-breakdown/index.php
16
+ - tools/v2/sprints/worktype-breakdown/index.php
17
+ - tools/v2/sprints/points-by-dev/index.php
18
+ - tools/v2/sprints/burndown/index.php
19
+ - tools/assets/clickup/sprint-dashboard.html
20
+ - tools/mvc/clickup/react/get.php
21
+ - tools/mvc/login/get.php
22
+ - tools/_/app/nav.php
23
+ related:
24
+ - ./persona-gated-navigation.md
25
+ - ./saml-sso-auth.md
26
+ - ./mvc-data-access-patterns.md
27
+ - ../architecture.md
28
+ - ../../../2.0/apps/worker2/features/team-sprint-management.md
29
+ ---
30
+
31
+ ## Summary
32
+
33
+ A native **ClickUp sprint dashboard** in the 1.0 `tools` app at route `/clickup/react`,
34
+ built for a wall-mounted **TV display** ("TOGa IQ" internal sprint metrics). It replaces a
35
+ standalone Node/Express prototype from a prior session, folding the dashboard into `tools`
36
+ so it runs on the existing App\_ framework + SSO with no separate service.
37
+
38
+ It reads the **same Team DB** and reproduces the **same Power BI-parity metric definitions**
39
+ already documented for worker2 — see
40
+ [team-sprint-management](../../../2.0/apps/worker2/features/team-sprint-management.md). This
41
+ doc does **not** duplicate those DAX→SQL definitions; it documents the tools-side delivery
42
+ (data engine, routes, UI, embed, nav, TV decisions).
43
+
44
+ ## How it works
45
+
46
+ ### Data engine — `App_Clickup_Sprint` (`_/app/clickup/sprint.php`)
47
+
48
+ Static methods, one per metric: `tile`, `status-breakdown`, `worktype-breakdown`,
49
+ `points-by-dev`, `burndown`, `current`. Each queries the **Team DB** via the `db_team`
50
+ connection (registry string `'db_team'`, from the `[database_team]` config section) using
51
+ `App_Database::query($sql, 'db_team')`. Responses are wrapped in the **api2 `/v2` envelope**
52
+ (`{success, data, errors}`) so the tool is a faithful stand-in for api2.
53
+
54
+ **Auth is done by hand as JSON**, not the HTML auth card: `App_Auth::isLoggedIn()` +
55
+ `App_Nav::personasForRoute('/clickup/react')` + `hasAnyPersona`; on failure it returns a
56
+ JSON **401/403** so the fetch-based UI gets a clean error instead of an HTML login page.
57
+
58
+ Two 1.0-framework gotchas this engine had to work around (both reusable beyond this
59
+ feature — see [Gotchas](#gotchas)):
60
+
61
+ 1. **Query-logging preflight on a `Defaults`-less DB.** Call
62
+ `App_Registry::set('queryLogging', false)` before the first query on `db_team`, or
63
+ `App_Query`'s one-time preflight (`SELECT ... FROM Defaults WHERE setting='QUERY_LOGGING'`)
64
+ throws `#1146 Table 'Team.Defaults' doesn't exist` and every metric 500s.
65
+ 2. **`App_Database::buildArrayOfRows()` takes its result by reference.** Assign the query
66
+ result to a variable first; passing `App_Database::query(...)` inline is fatal under the
67
+ E_ALL handler.
68
+
69
+ ### Routes — `v2/sprints/{metric}/index.php`
70
+
71
+ Thin route handlers mapping `/v2/sprints/*` to `App_Clickup_Sprint::serve($metric)`. One
72
+ `index.php` per metric (`tile`, `current`, `status-breakdown`, `worktype-breakdown`,
73
+ `points-by-dev`, `burndown`).
74
+
75
+ ### UI — `assets/clickup/sprint-dashboard.html`
76
+
77
+ **React via CDN in a single HTML file** — React 18.3.1 + ReactDOM (production UMD) +
78
+ Babel-standalone from CDN, JSX inside a `<script type="text/babel">`. **No build step, no
79
+ npm, no bundler.** It fetches `/v2/sprints/*` **same-origin**, so the SSO session cookie
80
+ authenticates the calls automatically. Do not mistake this for a bundled React app.
81
+
82
+ ### Embed page — `mvc/clickup/react/get.php`
83
+
84
+ A PHP MVC view that `requireAuth`s, then embeds `sprint-dashboard.html` in an
85
+ `<iframe allowfullscreen>` (cache-busted by `filemtime`). Heading "Clickup Dashboard",
86
+ subtitle notes the data is "…refreshed every 5 min."
87
+
88
+ ### Nav — `_/app/nav.php`
89
+
90
+ Registered under the **"Dashboard"** nav folder as
91
+ `['label'=>'Clickup','route'=>'/clickup/react']`, persona-gated like every folder (see
92
+ [persona-gated-navigation](./persona-gated-navigation.md)). The old Power BI embed nav entry
93
+ (`/clickup/dashboard`) was **removed** — the page file stays on disk, just unlinked — because
94
+ the React dashboard supersedes it.
95
+
96
+ ## TV display decisions
97
+
98
+ - **Soft 5-min auto-refresh.** The `useApi` hook re-fetches in place every 5 min via
99
+ `setInterval`; it does **not** `location.reload()`. On a full-screen TV a soft refresh
100
+ keeps the page full-screen and flash-free, whereas a full reload exits full screen every
101
+ cycle. (JS timers only throttle in hidden/background tabs; a foreground full-screen TV tab
102
+ is not throttled.)
103
+ - **16:9 stage.** The iframe stage uses `aspect-ratio:16/9` with
104
+ `max-height: calc(100vh - 170px)` so it stays TV-shaped without forcing page scroll. Full
105
+ width comes from lifting the shell content cap for this page only —
106
+ `.app-content:has(.cur-dash){max-width:none}` (same pattern the design tool uses in
107
+ `assets/css/design.css`); the dashboard `.page` has no max-width so it fills the stage.
108
+
109
+ ## Local development
110
+
111
+ - **Dev auth bypass.** `mvc/login/get.php` has a block that, when
112
+ `!empty($internal['dev_mode']) && !empty($internal['dev_autologin'])`, calls
113
+ `App_Auth::establishSession([...synthetic 'Local Dev' user...], ['Development Team','TOGa Technology'])`
114
+ and `App_MVC::routeTo('/clickup/dashboard')`, skipping the SSO form. The `dev_autologin`
115
+ flag lives **only** in the git-ignored `config.dev-local.ini` (next to `dev_mode`); prod
116
+ has `dev_mode=0` and no such flag, so the bypass is inert in every deployed env. The config
117
+ filename is chosen by `getenv('ENVIRONMENT')` → `config.<env>.ini`
118
+ (`App_Framework::initialize` / `App_Config::getConfigFilename`).
119
+ - **Run on Windows** (the repo's `docs/LOCAL_DEV.md` is macOS-worded):
120
+ `cd tools && ENVIRONMENT=dev-local php -d "include_path=.;C:/www/library" -S localhost:8000 router.php`.
121
+ The `App_` framework + `_.php` bootstrap load from the separate `library` repo via
122
+ `include_path`; `router.php` reproduces the `.htaccess` rewrite for PHP's built-in server.
123
+
124
+ > Credentials note: DB creds and the `dev_mode` / `dev_autologin` flags live in the
125
+ > git-ignored `config.dev-local.ini` — never commit values.
126
+
127
+ ## Gotchas
128
+
129
+ - **`queryLogging` preflight fails on any non-core DB without a `Defaults` table.** `App_Query`
130
+ (`library/app/query.php`) runs a one-time-per-request check on the **first** query of the
131
+ request against the active connection: `SELECT value FROM Defaults WHERE setting='QUERY_LOGGING'`.
132
+ If that first query runs on `db_team` (the Team schema has no `Defaults` table) it throws
133
+ `#1146 Table 'Team.Defaults' doesn't exist`. Set `App_Registry::set('queryLogging', false)`
134
+ before any query on a `Defaults`-less connection. General to any 1.0 App\_ code hitting the
135
+ Team DB or any other non-core schema.
136
+ - **`App_Database::buildArrayOfRows()` result is by-reference.**
137
+ `buildArrayOfRows(App_Database::query($sql,$db), false)` raises "Only variables should be
138
+ passed by reference", which the E_ALL error handler promotes to a fatal 500. Assign first:
139
+ `$res = App_Database::query(...); buildArrayOfRows($res, false);`.
140
+
141
+ ## Change history
142
+ - 2026-07-29 — Built `/clickup/react` as a native tools feature (data engine, `/v2/sprints/*` routes, CDN-React TV UI, iframe embed, "Dashboard/Clickup" nav); removed the old `/clickup/dashboard` Power BI nav entry; documented two reusable 1.0 gotchas (queryLogging preflight on Defaults-less DBs, buildArrayOfRows by-ref). (kyalamarthi)
@@ -6,8 +6,8 @@ project: Tools
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-07-28
10
- owners: [jcardinal]
9
+ updated: 2026-07-29
10
+ owners: [jcardinal, ajean]
11
11
  files:
12
12
  - tools/mvc/talos/kb-documents/get.php
13
13
  - tools/mvc/talos/kb-documents/post.php
@@ -18,6 +18,9 @@ files:
18
18
  - tools/_/app/talos/s3.php
19
19
  - tools/_/app/talos/bedrock.php
20
20
  - tools/_/app/pg.php
21
+ - tools/_/app/model/true/aimodels.php
22
+ - tools/_/app/model/true/vectorindexes.php
23
+ - tools/_/app/model/true/aimodels_vectorindexes.php
21
24
  - tools/_/app/worker.php
22
25
  - tools/_/app/nav.php
23
26
  - tools/config.production.ini
@@ -34,6 +37,17 @@ related:
34
37
 
35
38
  ## Summary
36
39
 
40
+ > **PER-AI-MODEL, DATA-DRIVEN SCOPING (2026-07-29).** The KB-documents and Vocabulary admin
41
+ > UIs were refactored from a single hard-coded **"development-team"** knowledge base to
42
+ > **per-AI-model** scoping — the Tools-side counterpart of the worker2 DB-driven routing (see
43
+ > [worker2 routing](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md#db-driven-ai-model-routing-2026-07-29)).
44
+ > KB Documents is now an **AiModel → VectorIndex → documents** cascade; Vocabulary is scoped by
45
+ > an AiModel picker; both **FAIL CLOSED** (throw, never the dev-team bucket, when a target is
46
+ > missing) and are tenant-guarded per model. New 1.0 models `App_Model_True_Aimodels`,
47
+ > `App_Model_True_Vectorindexes`, `App_Model_True_Aimodels_Vectorindexes` (`db_true`). PR tools
48
+ > #5 — code-complete/reviewed; see the **unverified Postgres slug assumption** in Gotchas before
49
+ > merge.
50
+
37
51
  A Tools (1.0) admin UI to browse/fix the Talos knowledge-base documents and manage the
38
52
  transcript-cleanup vocabulary — without a deploy. It is the human front-end to the worker2
39
53
  [Talos Transcript Ingestion](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md)
@@ -206,6 +220,33 @@ orphaned KB).
206
220
  though the *identical call without tags* would have succeeded. Confirm this grant with devops
207
221
  **before the next real KB creation** (see the orphaned-vector-bucket gotcha).
208
222
 
223
+ ## Per-AI-model scoping (2026-07-29)
224
+
225
+ Refactor from a single hard-coded `development-team` KB to per-AI-model, data-driven scoping.
226
+
227
+ - **KB Documents cascade.** `AiModel → VectorIndex → documents`, DB-driven (replaces the old
228
+ S3-folder-enumeration KB list). Every document action is **tenant-guarded**: requires an
229
+ `aiModelId`, loads only `c_isActive = 1` VectorIndexes, and **rejects** unless the index is
230
+ bridged to that model via `AiModels_VectorIndexes`. The **move** action additionally refuses
231
+ cross-model / cross-bucket / cross-region / same-location moves.
232
+ - **Per-index S3 context.** `App_Talos_S3` gained a **request-scoped active context** —
233
+ `useIndex()` / `reset()` / `client(?index)` — so bucket/region/prefix come from the selected
234
+ VectorIndex's `c_s3Bucket` / `c_s3Region` / `c_s3Prefix`. **FAILS CLOSED**: throws if any is
235
+ blank — it never falls back to the dev-team bucket.
236
+ - **Vocabulary scoping.** An AiModel picker; **all reads AND the reconcile writes**
237
+ (insert/update/delete) are scoped by `c_trueAiModelId`, so a save for one model can never touch
238
+ another model's rows. **Concurrency-hardened:** `FOR UPDATE` on the existing-row/ownership
239
+ reads, plus a **content-aware revision token** — SHA-256 over each row's id + content,
240
+ byte-identical in `get.php` and `post.php` — that rejects a **stale full-set submission before
241
+ any write** when another admin added, removed, OR edited a row (prevents phantom-delete and
242
+ lost-update).
243
+ - **Create/provision flow.** New `VectorIndexes` rows populate `c_s3*` + `c_isActive` and an
244
+ `AiModels_VectorIndexes` bridge row for the chosen model. The previously hard-coded single
245
+ Postgres assistant id was replaced with a **per-AiModel lookup BY SLUG** (`App_Pg` gained a
246
+ read `query()` method — it was write-only). See the unverified slug assumption in Gotchas.
247
+ - **New models** (`db_true`): `App_Model_True_Aimodels`, `App_Model_True_Vectorindexes`,
248
+ `App_Model_True_Aimodels_Vectorindexes`.
249
+
209
250
  ## Helpers
210
251
 
211
252
  - **`App_Talos_S3`** (`_/app/talos/s3.php`) — S3 client; list KB slugs; list/count approved
@@ -225,6 +266,8 @@ orphaned KB).
225
266
  section** (shared with the MySQL DBs) — **no new secret introduced**. `execute()` uses real
226
267
  PDO prepared statements (bound params). If the `pdo_pgsql` extension is missing it returns
227
268
  `ok=false` with a generic message (detail goes only to `error_log`, never to the client).
269
+ **As of 2026-07-29 it also has a read `query()` method** (it was write-only) — used by the
270
+ create flow to look up the per-AiModel Postgres assistant id by slug.
228
271
  - **`App_Worker::enqueue()`** (`_/app/worker.php`) — inserts a `Core.WorkerJobs` row and sends
229
272
  the `{"workerJobId": id}` SQS message, so a 1.0 app can enqueue worker2 jobs. Requires the
230
273
  `[worker]` (`queue_url`/`queue_region`) and `[aws]` (main-account key for SQS `SendMessage`)
@@ -315,12 +358,46 @@ orphaned KB).
315
358
  nullable)`, sectionSlug, chunkSlug, description` — create sets `docSlug=sectionSlug=chunkSlug=
316
359
  bedrockKbId`.
317
360
  - **Verified working end-to-end in production (2026-07-16)** with the VECTOR + S3 Vectors design.
361
+ - **UNVERIFIED ASSUMPTION (must confirm before tools #5 merges): `Client_True.AiModels.slug`
362
+ == the Postgres `assistants` slug.** The per-AiModel assistant lookup assumes the Postgres
363
+ table is `public.assistants(id, slug)` and that its `slug` matches the MySQL
364
+ `Client_True.AiModels.slug`. This is flagged in the code + PR and **must be confirmed against
365
+ live Postgres** before merge — otherwise create will map to the wrong / no assistant.
366
+ - **Tenant isolation is enforced in the app layer, not the DB.** KB-document actions require an
367
+ `aiModelId`, load only `c_isActive=1` indexes, and reject any index not bridged to the model;
368
+ Vocabulary reads+writes are scoped by `c_trueAiModelId`. Do not add a code path that skips
369
+ these guards — there is no DB-level FK/row-security backstop.
370
+ - **Vocabulary saves are guarded by a content-aware SHA-256 revision token** that must stay
371
+ **byte-identical** between `get.php` and `post.php`. If you change how the token is computed in
372
+ one, change the other in lockstep — a mismatch makes every save fail as "stale".
373
+ - **Deploy order (hard dependency): dbchanges2 #442 (schema + backfill) MUST apply first.**
374
+ tools #5 depends on the `c_` columns but not on worker2 #129 (deploy worker2 first as the live
375
+ pipeline, then tools).
376
+ - **The per-AI-model refactor (tools #5) is not yet run end-to-end in production.** It is
377
+ code-complete, `php -l` clean, and reviewed, but the per-model scoping / create flow has not
378
+ been exercised in prod pending merge and the unverified Postgres slug assumption above. (The
379
+ earlier VECTOR + S3 Vectors KB-create design *was* verified in prod on 2026-07-16.)
318
380
  - No new secret literals introduced; credentials live in the config sections above (documented by
319
381
  section/account, never by value). AWS account id `654654170868`, bucket `togaiq`, and the KB
320
382
  execution-role ARN are non-secret resource identifiers.
321
383
 
322
384
  ## Change history
323
385
 
386
+ - 2026-07-29 — **Refactored KB Documents + Vocabulary to per-AI-model, data-driven scoping**
387
+ (from the single hard-coded `development-team` KB). KB Documents is now an
388
+ `AiModel → VectorIndex → documents` cascade; every document action is tenant-guarded
389
+ (requires `aiModelId`, only `c_isActive=1` indexes, rejects unbridged; move refuses
390
+ cross-model/bucket/region/same-location). `App_Talos_S3` gained a request-scoped active
391
+ context (`useIndex()`/`reset()`/`client(?index)`) driving bucket/region/prefix from the
392
+ index's `c_s3*` — **fails closed** (throws) if blank, never the dev-team bucket. Vocabulary
393
+ gained an AiModel picker; reads + reconcile writes scoped by `c_trueAiModelId`;
394
+ concurrency-hardened with `FOR UPDATE` + a content-aware SHA-256 revision token (byte-identical
395
+ in get/post) that rejects a stale full-set submission. Create flow populates `c_s3*` +
396
+ `c_isActive` + an `AiModels_VectorIndexes` bridge row, and replaced the hard-coded Postgres
397
+ assistant id with a per-AiModel lookup by slug (`App_Pg` gained a read `query()`). New models
398
+ `App_Model_True_{Aimodels,Vectorindexes,Aimodels_Vectorindexes}` (`db_true`). **Unverified:**
399
+ `Client_True.AiModels.slug` == Postgres `assistants` slug — confirm before merge. **Deploy
400
+ after dbchanges2 #442.** PR tools #5. (ajean)
324
401
  - 2026-07-28 — **Every new Bedrock KB is now tagged** `tenant=True` + `model_role=knowledge_base`
325
402
  via the new `App_Talos_Bedrock::KB_TAGS` constant, passed as the `tags` parameter of
326
403
  `CreateKnowledgeBase` (required for togaiq cost allocation / model-inventory reporting; values
@@ -29,7 +29,7 @@
29
29
  | [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Receives inbound webhook events from Startech (Easeedesk) and creates or updates the corresponding ticket in TOGA 2.0. | worker2/Worker/Startech.php |
30
30
  | [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus programmatically. | .claude/skills/plan-ticket/scripts/talos.js |
31
31
  | [Talos Pricing Automation (worker2 Cron — AWS Actuals, Calibration, Monthly Report)](features/talos-pricing-automation.md) | The worker2 half of the **Talos Pricing Platform** (see the talos `pricing-cogs-model` and tools `talos-pricing-ui` docs for the other halves). | worker2/Worker/Talos/Pricing.php, worker2/Database/TalosPricingCrons.sql |
32
- | [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` runs a fully automated, cron-driven pipeline that ingests raw Teams transcripts into the **Talos / TOGa IQ** AWS Bedrock knowledge ba | worker2/Worker/Team/Transcripts.php, worker2/bin/sync-knowledge-bases.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a, dbchanges2/Core/2026-07-02a, dbchanges2/Team/2026-07-02a, dbchanges2/Team/2026-07-08a, dbchanges2/Team/2026-07-09a, dbchanges2/Team/2026-07-10a, dbchanges2/Team/2026-07-28a - TranscriptProcessingRetryAttempts.sql, dbchanges2/Team/2026-07-28b - TranscriptPromptTemplateConverseModel.sql, dbchanges2/Core/2026-07-28a - TeamsTranscriptRetryCron.sql |
32
+ | [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | > **DB-DRIVEN AI-MODEL ROUTING (2026-07-29).** Which knowledge base a transcript is cleaned > into is now decided by the **meeting organizer's "home" AI model** | worker2/Worker/Team/Transcripts.php, worker2/bin/sync-knowledge-bases.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Client_True/2026-07-27a - TranscriptAiModelRoutingColumns.sql, dbchanges2/Client_True/2026-07-27b - TranscriptAiModelRoutingData.sql, dbchanges2/Team/2026-07-27a - TranscriptVocabularyAiModelScope.sql, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a, dbchanges2/Core/2026-07-02a, dbchanges2/Team/2026-07-02a, dbchanges2/Team/2026-07-08a, dbchanges2/Team/2026-07-09a, dbchanges2/Team/2026-07-10a, dbchanges2/Team/2026-07-28a - TranscriptProcessingRetryAttempts.sql, dbchanges2/Team/2026-07-28b - TranscriptPromptTemplateConverseModel.sql, dbchanges2/Core/2026-07-28a - TeamsTranscriptRetryCron.sql |
33
33
  | [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php, _underscore/Model/Team/Sprint.php, dbchanges2/Core/CronJobs (SprintLockScheduled seed) |
34
34
  | [Teams Meeting Transcript Export](features/teams-transcript-export.md) | > **SUPERSEDED (2026-07-09) — the S3-staging model below is history.** `Export` is now a thin > **GRAPH-DIRECT** cron poller: it no longer archives raw VTT to ` | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
35
35
  | [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | `_Worker_Vapi` ([worker2/Worker/Vapi.php](worker2/Worker/Vapi.php)) is the **PHP side of the AI-BDR call loop** — the webhook that receives VAPI's end-of-call r | worker2/Worker/Vapi.php, worker2/Worker/Ai/Bdr/Vapi.php |
@@ -6,13 +6,16 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-07-28
10
- owners: [jcardinal]
9
+ updated: 2026-07-29
10
+ owners: [jcardinal, ajean]
11
11
  files:
12
12
  - worker2/Worker/Team/Transcripts.php
13
13
  - worker2/bin/sync-knowledge-bases.php
14
14
  - worker2/Config/production.ini
15
15
  - worker2/Database/TeamsTranscriptExports.sql
16
+ - dbchanges2/Client_True/2026-07-27a - TranscriptAiModelRoutingColumns.sql
17
+ - dbchanges2/Client_True/2026-07-27b - TranscriptAiModelRoutingData.sql
18
+ - dbchanges2/Team/2026-07-27a - TranscriptVocabularyAiModelScope.sql
16
19
  - dbchanges2/Team/2026-06-30a
17
20
  - dbchanges2/Team/2026-06-30b
18
21
  - dbchanges2/Team/2026-06-30c
@@ -40,6 +43,20 @@ related:
40
43
 
41
44
  ## Summary
42
45
 
46
+ > **DB-DRIVEN AI-MODEL ROUTING (2026-07-29).** Which knowledge base a transcript is cleaned
47
+ > into is now decided by the **meeting organizer's "home" AI model**, read from the DB — not by
48
+ > config. The old `[teams]` organizer id/email lists in `Config/production.ini` are **removed**;
49
+ > `Export` polls `Client_True.Users WHERE c_transcriptAiModelId IS NOT NULL AND isActive = 1`
50
+ > and threads an `aiModelId` into each `Process` job. `Process` resolves the model, loads only
51
+ > that model's **ACTIVE `Client_True.VectorIndexes`** (via the `AiModels_VectorIndexes` bridge),
52
+ > classifies among **only those** indexes, and uploads to that index's own
53
+ > `c_s3Bucket`/`c_s3Region`/`c_s3Prefix` — **no longer the single hard-coded
54
+ > `togaiq/development-team/` target**. The pipeline **FAILS CLOSED**: a user/organizer with no
55
+ > home model is never routed to a default KB. This resolves the "two competing registries" open
56
+ > point noted below — `Client_True.VectorIndexes` is now authoritative for routing. See
57
+ > [DB-driven AI-model routing](#db-driven-ai-model-routing-2026-07-29). PRs worker2 #129 /
58
+ > dbchanges2 #442.
59
+
43
60
  `_Worker_Team_Transcripts` runs a fully automated, cron-driven pipeline that ingests raw
44
61
  Teams transcripts into the **Talos / TOGa IQ** AWS Bedrock knowledge bases and emails a
45
62
  recap to the meeting organizer. **As of 2026-07-09 the pipeline is GRAPH-DIRECT**: there is
@@ -161,7 +178,7 @@ UNIQUE index.
161
178
  AI endpoint in the same minute, and kept inside Export's business-hours window because a re-drive
162
179
  emails a recap to the meeting organizer.
163
180
 
164
- ### `Process(int $transcriptId, string $graphUserId, string $meetingId, string $organizerEmail, string $subject, string $meetingStartIso, array $cc = [], array $bcc = [])`
181
+ ### `Process(int $transcriptId, string $graphUserId, string $meetingId, string $organizerEmail, string $subject, string $meetingStartIso, array $cc = [], array $bcc = [], int $aiModelId = 0)`
165
182
  1. Download the VTT **straight from Graph** (no S3 round trip).
166
183
  2. `stripAndGroupVtt()` — strips WebVTT timestamps and **merges consecutive same-speaker cues
167
184
  into one block** (~40% token reduction before the AI call).
@@ -216,6 +233,34 @@ Three deliberate deviations from the usual worker2 `_ApiRequest` idiom — do no
216
233
  Timeouts/attempts are class constants: `AI_DEFAULT_TIMEOUT = 600`, `AI_CONNECT_TIMEOUT = 20`,
217
234
  `AI_MAX_ATTEMPTS = 2`, `AI_BASE_DELAY_MS`.
218
235
 
236
+ ### DB-driven AI-model routing (2026-07-29)
237
+ Replaces the old config-driven `[teams]` organizer lists **and** the single hard-coded
238
+ `development-team` KB target. **Source of truth = the DB** (`Client_True`), not
239
+ `Config/production.ini`.
240
+
241
+ - **Organizer source (`Export`).** `Export` polls
242
+ `Client_True.Users WHERE c_transcriptAiModelId IS NOT NULL AND isActive = 1` — a user with a
243
+ non-NULL `c_transcriptAiModelId` **is** a transcript organizer. Each user's `aiModelId` is
244
+ threaded into that organizer's `Process` jobs. The `[teams]` organizer id/email lists were
245
+ **removed** from `production.ini`.
246
+ - **Model resolution (`Process`).** Uses the passed `aiModelId`; if `0` (e.g. a `Retry()`
247
+ re-drive that only carries `organizerEmail`), it **falls back** to resolving the organizer's
248
+ `Client_True.Users.c_transcriptAiModelId` by email (`Users.email` == the organizer UPN — there
249
+ is no Entra object-id column, join is by email/UPN). **FAILS CLOSED**: if no model resolves,
250
+ the transcript is **not** routed to any default/`general` KB — it is rejected, never
251
+ mis-filed.
252
+ - **Index selection.** Loads the model's **ACTIVE** `Client_True.VectorIndexes`
253
+ (`c_isActive = 1`) via the `AiModels_VectorIndexes` bridge table, and classifies the transcript
254
+ among **only those** indexes. **One active index → route directly, no AI classify call.**
255
+ - **Upload target is per-index.** The approved doc uploads to the selected VectorIndex's own
256
+ `c_s3Bucket` / `c_s3Region` / `c_s3Prefix` (not the single `togaiq/development-team/` target),
257
+ then Bedrock ingests by the index's `docSlug`. **`VectorIndexes.docSlug` (== `sectionSlug` ==
258
+ `chunkSlug`) IS the Bedrock knowledge-base id** — there is no separate bedrock-id column.
259
+ - **Composes with the upstream `Retry()` re-drive.** The merged `_production` `Retry()`
260
+ subsystem re-enqueues `Process` without an `aiModelId` but passes `organizerEmail`, so the
261
+ email fallback above resolves the model. Verified the routing logic composes with `Retry()`,
262
+ the `_ApiRequest` Talos transport switch, and the `Logs_True` DB alias registration.
263
+
219
264
  ### Organizer email resolution
220
265
  - `resolveOrganizerEmail()` calls Graph `/users/{id}?$select=mail,userPrincipalName` — but
221
266
  this needs the **`User.Read.All`** app permission, which the app registration **lacks**.
@@ -242,6 +287,22 @@ spacing (Bedrock allows only one in-flight ingestion per KB, so parallel syncs r
242
287
  - **`Team.TranscriptPromptTemplate`** — `name, bodyText, instruction, model, temperature, …`.
243
288
  - **`Team.KnowledgeBases`** — `slug, name, isGeneral, bedrockKbId, isActive`; auto-kept-current
244
289
  from Bedrock by the daily `SyncKnowledgeBases`.
290
+
291
+ **Routing model (`Client_True.*` schema, Client cluster — dbchanges2 `Client_True/2026-07-27a`+`b`):**
292
+ - **`Client_True.Users.c_transcriptAiModelId`** — soft ref → `Client_True.AiModels.id`; NULL means
293
+ the user is not a transcript organizer. This is the organizer source list (replaces `[teams]`).
294
+ - **`Client_True.VectorIndexes.c_s3Bucket` / `c_s3Region` / `c_s3Prefix` / `c_isActive`** — the
295
+ S3 upload target and active flag for each Bedrock KB. Upload routes to these per-index values.
296
+ - **`AiModels_VectorIndexes`** — bridge mapping a model to its knowledge bases (active ones drive
297
+ index selection in `Process`).
298
+ - **`Team.TranscriptPromptTerms` / `TranscriptReplacements` / `TranscriptPromptTemplate`** gained
299
+ **`c_trueAiModelId`** (soft ref → `Client_True.AiModels.id`, dbchanges2 `Team/2026-07-27a`) so
300
+ the transcript-cleanup vocabulary is scoped per AI model.
301
+ - The `2026-07-27b` data backfill maps existing dev emails → dev-core model, `ecastellucci` →
302
+ operations model, and each VectorIndex's S3 target.
303
+ - **These `c_`-prefixed columns needed ZERO `_underscore`/model-layer changes** — see the
304
+ framework note under Gotchas.
305
+
245
306
  **The two tables are BOTH live and NOT redundant** — different owners, different jobs:
246
307
 
247
308
  - **`Team.TranscriptExports`** — **discovery/observation ledger** owned by `Export()`: drives
@@ -360,8 +421,10 @@ part of the ingestion loop** (raw reads removed). Credential values live only in
360
421
  silently no-ops — the worker still returns HTTP-200, so the UI falsely reports "sync
361
422
  initiated". **Fix:** run `Team/Transcripts/SyncKnowledgeBases` (or
362
423
  `bin/sync-knowledge-bases.php`) to discover `development-team-*` KBs from Bedrock and upsert
363
- them into `Team.KnowledgeBases`. **Open architectural point:** two competing registries —
364
- `Team.KnowledgeBases` (worker2) vs `Client_True.VectorIndexes` (Tools UI).
424
+ them into `Team.KnowledgeBases`. **Resolved (2026-07-29):** the routing rewrite makes
425
+ `Client_True.VectorIndexes` (with its `c_s3*` + `AiModels_VectorIndexes` bridge) the
426
+ **authoritative** routing registry the pipeline reads; `Team.KnowledgeBases` remains only the
427
+ worker2 KB-registry mirror. Prefer `Client_True.VectorIndexes` for routing decisions.
365
428
  - **`initialize()` MUST register the Logs DB under `DB_CLIENT_LOGS` or the Talos api-logging is
366
429
  a silent no-op.** `_ApiRequest`'s logging branch writes via `_Model_Client_Logs_Api`, whose
367
430
  `DATABASE` const is `_underscore::DB_CLIENT_LOGS`. `Transcripts::initialize()` previously
@@ -380,12 +443,32 @@ part of the ingestion loop** (raw reads removed). Credential values live only in
380
443
  belongs **inside `_ApiRequest::execute()`** (reconnect/re-register before the post-call save)
381
444
  and affects every framework caller, so it is deferred pending architecture review. Do **not**
382
445
  work around it in `Transcripts.php`.
446
+ - **`c_`-prefixed columns are FRAMEWORK-DYNAMIC in 2.0 — no model-class change needed.** The new
447
+ `c_transcriptAiModelId` / `c_s3*` / `c_trueAiModelId` columns are read off raw `_Query` rows and
448
+ written/filtered as `_Model` array keys **without** being declared in the model class and with
449
+ **no** `Core.RecordFields` / ACL entries (matches the existing `c_clickupUserIdentifier`
450
+ precedent). Consequence: the `_underscore`/model layer required **zero** changes for these
451
+ columns (Phase 2 verified NO-OP). See the framework standard
452
+ [`c_`-prefixed custom columns](../../standards/framework-rules.md).
383
453
  - **Cross-ACCOUNT S3.** `togaiq` (us-east-1) approved/archive writes use the `[talos]` key;
384
454
  `CopyObject` across accounts is impossible, so archive is get(in-memory)+put. (The old
385
455
  toga-private cross-account read is no longer in the loop.)
386
456
 
387
457
  ## Change history
388
458
 
459
+ - 2026-07-29 — **DB-driven per-user AI-model routing.** `Export` now polls
460
+ `Client_True.Users WHERE c_transcriptAiModelId IS NOT NULL AND isActive=1` (the `[teams]`
461
+ organizer id/email lists were removed) and threads `aiModelId` into each `Process` job;
462
+ `Process(…, int $aiModelId=0)` resolves the model (email fallback via
463
+ `Users.c_transcriptAiModelId`, **fails closed** — no default KB), loads only that model's
464
+ ACTIVE `Client_True.VectorIndexes` via the `AiModels_VectorIndexes` bridge, classifies among
465
+ only those (1 index → direct, no classify), and uploads to the index's own
466
+ `c_s3Bucket`/`c_s3Region`/`c_s3Prefix` — replacing the single `togaiq/development-team/`
467
+ target. `docSlug` is the Bedrock KB id. Composes with the upstream `_production` merge
468
+ (`Retry()` re-drive, `_ApiRequest` Talos transport, `Logs_True` alias). `c_` columns are
469
+ framework-dynamic → zero `_underscore`/model changes. **Deploy order: dbchanges2 #442 (schema
470
+ + backfill) MUST apply before worker2 #129.** dbchanges2: `Client_True/2026-07-27a`+`b`,
471
+ `Team/2026-07-27a`. PR worker2 #129. (ajean)
389
472
  - 2026-07-28 — **Added `Team/Transcripts/Retry`, a bounded automatic re-drive of failed
390
473
  transcripts, and fixed the five-day 500 outage.** Discovered a failed `Process()` was
391
474
  *permanently* terminal (HTTP-200 workers + the `dtEnqueued` one-way latch + nothing re-reading
@@ -5,8 +5,8 @@ project: _Underscore
5
5
  client: shared
6
6
  type: standard
7
7
  status: active
8
- updated: 2026-07-28
9
- owners: [jcardinal]
8
+ updated: 2026-07-29
9
+ owners: [jcardinal, ajean]
10
10
  files: []
11
11
  related:
12
12
  - ../apps/_underscore/architecture.md
@@ -163,5 +163,26 @@ returns nothing (or returns only when it succeeds) leaves a blind spot in the au
163
163
  only for genuinely retriable conditions, where the queue's retry is the intended behaviour; a
164
164
  handled non-retriable failure should return a result describing what happened rather than throwing.
165
165
 
166
+ ## `c_`-prefixed custom columns are framework-dynamic
167
+
168
+ Columns prefixed `c_` are **custom/dynamic** columns that the `_underscore` model layer handles
169
+ **without any model-class declaration**:
170
+
171
+ - They are read straight off raw `_Query` result rows and written/filtered as `_Model` array
172
+ keys **without** being declared as properties on the `_Model_*` subclass.
173
+ - They require **no `Core.RecordFields` entry and no ACL entry**.
174
+ - Precedent: `c_clickupUserIdentifier`; the 2026-07 transcript-routing columns
175
+ (`Client_True.Users.c_transcriptAiModelId`, `Client_True.VectorIndexes.c_s3Bucket/c_s3Region/
176
+ c_s3Prefix/c_isActive`, and `Team.Transcript*.c_trueAiModelId`) follow the same pattern.
177
+
178
+ **Consequence:** adding a `c_` column is a **schema-only** change (a `dbchanges2` migration) —
179
+ the model/`_underscore` layer needs **zero** code changes to read, write, or filter on it. Do
180
+ not add `RecordFields`/ACL rows or model-class properties for a `c_` column; that is redundant
181
+ and not how the convention works. Use a non-`c_` column when you *do* want the full declared,
182
+ ACL-governed field treatment.
183
+
166
184
  ## Change history
185
+ - 2026-07-29 — Documented the `c_`-prefixed framework-dynamic column convention (schema-only,
186
+ no model-class/RecordFields/ACL declaration), generalized from the transcript AI-model routing
187
+ work. (ajean)
167
188
  - 2026-07-28 — Corrected the **Worker contract** and **Worker dispatch** sections to match the actual codebase: worker actions are `abstract class _Worker_<Name>` with `public static` entry methods dispatched via `_Worker::runTask(string $action, array|object $parameters)` on the `Category/Sub/File/MethodName` path, with parameters spread as **named arguments** (`$className::$functionName(...$parameters)`, worker2 `Controller/Index.php:325,567`; `_underscore/Worker.php:8`). This replaces the previous `extends _Worker` / `run()` / `_Queue::dispatch()` guidance — no worker in worker2 follows it, `_Queue::dispatch()` does not exist in the codebase, and following the old text would produce a worker that never dispatches. (jcardinal)
@@ -13,7 +13,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
13
13
  - **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
14
14
  - **test** (Test) — 13 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
15
15
  - **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
16
- - **tools** (Tools) — 10 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
16
+ - **tools** (Tools) — 11 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
17
17
 
18
18
  ## 2.0 framework
19
19
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.470",
3
+ "version": "1.0.472",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",