toga-ai 1.0.259 → 1.0.261
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/tools/INDEX.md +1 -0
- package/knowledge/1.0/apps/tools/features/talos-kb-documents-admin.md +88 -0
- package/knowledge/2.0/apps/_underscore/INDEX.md +1 -1
- package/knowledge/2.0/apps/_underscore/features/surface-resolver.md +57 -2
- package/knowledge/2.0/apps/toga25-supply/features/client-configurable-fields.md +23 -1
- package/knowledge/2.0/apps/worker2/INDEX.md +2 -1
- package/knowledge/2.0/apps/worker2/features/talos-transcript-ingestion.md +158 -0
- package/knowledge/2.0/apps/worker2/features/teams-transcript-export.md +23 -8
- package/knowledge/INDEX.md +2 -2
- package/knowledge/clients/compass-canada/profile.md +8 -1
- package/knowledge/clients/quad/profile.md +9 -1
- package/package.json +1 -1
|
@@ -7,5 +7,6 @@
|
|
|
7
7
|
| [Tools MVC — Routing, CSRF & App_Database Access Patterns](features/mvc-data-access-patterns.md) | The load-bearing 1.0 (`App_`) framework conventions a developer needs when adding a page to the Tools app — URL routing, CSRF, and DB access through `App_Databa | tools/_/app/nav.php, tools/mvc/get.php |
|
|
8
8
|
| [Tools Persona-Gated Navigation (App_Nav)](features/persona-gated-navigation.md) | `App_Nav` is the Tools app's two-level, **persona-gated** navigation. | tools/_/app/nav.php, tools/mvc/get.php |
|
|
9
9
|
| [Tools SAML SSO Consumer & Persona-Gated Auth (App_Auth)](features/saml-sso-auth.md) | `App_Auth` is the Tools app's authentication layer: it consumes the SAML gateway `?saml=` handoff (see the 2.0 SAML downstream integration contract), establishe | tools/_/app/auth.php, tools/mvc/sso/initiate/get.php, tools/mvc/sso/get.php, tools/mvc/login/get.php, tools/mvc/login/post.php, tools/mvc/logout/get.php, tools/mvc/get.php, tools/config.production.ini, tools/config.local.ini |
|
|
10
|
+
| [Talos Knowledge Base Admin UI (KB Documents + Vocabulary)](features/talos-kb-documents-admin.md) | A Tools (1.0) admin UI to browse/fix the Talos knowledge-base documents and manage the transcript-cleanup vocabulary — without a deploy. | tools/mvc/talos/kb-documents/get.php, tools/mvc/talos/kb-documents/post.php, tools/mvc/talos/vocabulary/get.php, tools/mvc/talos/vocabulary/post.php, tools/_/app/talos/s3.php, tools/_/app/worker.php, tools/_/app/nav.php |
|
|
10
11
|
| [Talos Pricing UI (Onboarding, Dashboard, Benchmarks, Cost Factors + Estimator)](features/talos-pricing-ui.md) | The 1.0 (tools app) face of the **Talos Pricing Platform** — a "Talos Pricing" nav folder with four pages plus a client-side estimate engine. | tools/_/app/nav.php, tools/_/app/talos/estimator.php, tools/mvc/talos/onboarding/get.php, tools/mvc/talos/onboarding/post.php, tools/mvc/talos/pricing/get.php, tools/mvc/talos/benchmarks/get.php, tools/mvc/talos/factors/get.php, tools/mvc/talos/factors/post.php, tools/assets/css/style.css |
|
|
11
12
|
| [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the **Tools** 1.0 app boots on Elastic Beanstalk running `PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. | tools/.ebextensions/004_http_to_https.config, tools/.ebextensions/006_mount-s3fs.config, tools/.ebextensions/007_setup_export_cache_folders.config, tools/.ebextensions/008_setup_ldap.config, tools/.ebextensions/009_setup_phpini.config, tools/.ebextensions/020_setup_git_libraries.config, tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config, tools/ebs/git.json |
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Talos Knowledge Base Admin UI (KB Documents + Vocabulary)
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: tools
|
|
5
|
+
project: Tools
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-06-30
|
|
10
|
+
owners: [jcardinal]
|
|
11
|
+
files:
|
|
12
|
+
- tools/mvc/talos/kb-documents/get.php
|
|
13
|
+
- tools/mvc/talos/kb-documents/post.php
|
|
14
|
+
- tools/mvc/talos/vocabulary/get.php
|
|
15
|
+
- tools/mvc/talos/vocabulary/post.php
|
|
16
|
+
- tools/_/app/talos/s3.php
|
|
17
|
+
- tools/_/app/worker.php
|
|
18
|
+
- tools/_/app/nav.php
|
|
19
|
+
related:
|
|
20
|
+
- ./mvc-data-access-patterns.md
|
|
21
|
+
- ./persona-gated-navigation.md
|
|
22
|
+
- ../architecture.md
|
|
23
|
+
- ../../../2.0/apps/worker2/features/talos-transcript-ingestion.md
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## Summary
|
|
27
|
+
|
|
28
|
+
A Tools (1.0) admin UI to browse/fix the Talos knowledge-base documents and manage the
|
|
29
|
+
transcript-cleanup vocabulary — without a deploy. It is the human front-end to the worker2
|
|
30
|
+
[Talos Transcript Ingestion](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md)
|
|
31
|
+
pipeline. A new **"Talos Knowledge Base"** nav group (personas *Development Team* / *TOGa
|
|
32
|
+
Technology*) exposes two pages.
|
|
33
|
+
|
|
34
|
+
## Pages
|
|
35
|
+
|
|
36
|
+
### `/talos/kb-documents` — approved-doc browser
|
|
37
|
+
- KB list derived from the **actual S3 sub-folders** under `development-team/` (title-cased,
|
|
38
|
+
e.g. `office-depot` → "Office Depot") — not from `Team.KnowledgeBases`.
|
|
39
|
+
- Lists approved docs excluding `*.metadata.json`; columns Meeting Date (`n/j/y`) / Title /
|
|
40
|
+
Size / Last modified (`n/j/y g:i A`).
|
|
41
|
+
- Clickable rows open an overlay **modal** with a type-aware preview: text inline; PDF via a
|
|
42
|
+
presigned S3 URL in an iframe; Office files via the Google Docs viewer (presigned URL; MS
|
|
43
|
+
Office viewer noted as an alternative); unsupported → download/delete only.
|
|
44
|
+
- Download + delete are icon buttons in the modal. **Delete enqueues the worker `SyncKb`
|
|
45
|
+
re-sync** so Bedrock reflects the removal.
|
|
46
|
+
|
|
47
|
+
### `/talos/vocabulary` — cleanup vocabulary manager
|
|
48
|
+
Single tabbed page: Prompt Template, Applications, Clients, People, Business Terms, Examples,
|
|
49
|
+
Replacements. (The old standalone `/talos/replacements` page was folded in here and deleted.)
|
|
50
|
+
Classic view → Edit → Save per tab, with **full-set reconciliation** (transactional, rolls
|
|
51
|
+
back on error), trash-icon row removal, and an auto-dismissing toast. Backs the `Team.*`
|
|
52
|
+
prompt/replacement tables that the worker2 pipeline reads at runtime.
|
|
53
|
+
|
|
54
|
+
## Helpers
|
|
55
|
+
|
|
56
|
+
- **`App_Talos_S3`** (`_/app/talos/s3.php`) — S3 client; list KB slugs; list/count approved;
|
|
57
|
+
presigned URL; get/delete; slug sanitize+lowercase; approved-prefix path guard.
|
|
58
|
+
- **`App_Worker::enqueue()`** (`_/app/worker.php`) — inserts a `Core.WorkerJobs` row and sends
|
|
59
|
+
the `{"workerJobId": id}` SQS message, so a 1.0 app can enqueue worker2 jobs.
|
|
60
|
+
|
|
61
|
+
## Deploy prerequisites
|
|
62
|
+
|
|
63
|
+
- `aws/aws-sdk-php` installed in `tools/vendor` (S3 + SQS).
|
|
64
|
+
- New `tools/config.*.ini` sections (none exist yet): `[database_core2]` (writable 2.0 Core,
|
|
65
|
+
dbname `Core`), `[worker]` (`queue_url` / `queue_region`), `[aws]` (production).
|
|
66
|
+
- Apply the dbchanges2 `Team/2026-06-30a..e` + `Core/2026-06-30a` migrations in order.
|
|
67
|
+
- Verify the Bedrock KB region / data-source name against the live account (TODOs in code).
|
|
68
|
+
|
|
69
|
+
## Gotchas
|
|
70
|
+
|
|
71
|
+
- **KB list is derived from S3, not the DB.** The browse UI enumerates live S3 folders under
|
|
72
|
+
`development-team/`; `Team.KnowledgeBases` is not the source of truth for what's shown.
|
|
73
|
+
- **Delete is not just an S3 delete** — it also enqueues a worker `SyncKb` job to re-sync the
|
|
74
|
+
Bedrock data source.
|
|
75
|
+
- No new secret literals introduced; credentials come from the new config sections above.
|
|
76
|
+
|
|
77
|
+
## Change history
|
|
78
|
+
|
|
79
|
+
- 2026-06-30 — Built the Talos KB admin UI: `/talos/kb-documents` (S3-derived KB list, approved
|
|
80
|
+
doc browser with type-aware preview modal, download + delete-with-resync) and `/talos/vocabulary`
|
|
81
|
+
(tabbed prompt/vocabulary/replacements manager with transactional full-set reconciliation;
|
|
82
|
+
folded in and deleted the old `/talos/replacements`). Added `App_Talos_S3` and
|
|
83
|
+
`App_Worker::enqueue()` (1.0 → worker2 SQS) helpers and the "Talos Knowledge Base" nav group. (jcardinal)
|
|
84
|
+
|
|
85
|
+
## Related docs
|
|
86
|
+
|
|
87
|
+
- [Talos Transcript Ingestion Pipeline](../../../2.0/apps/worker2/features/talos-transcript-ingestion.md) — the worker2 backend.
|
|
88
|
+
- [Tools MVC — Routing, CSRF & App_Database Access Patterns](./mvc-data-access-patterns.md)
|
|
@@ -14,6 +14,6 @@
|
|
|
14
14
|
| [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
|
|
15
15
|
| [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |
|
|
16
16
|
| [Recursive Item Fulfillments (upstream mirroring)](features/recursive-item-fulfillments.md) | In a multi-tier supply chain a sales order (SO) spawns a purchase order (PO) that becomes another SO downstream, and so on. | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillmentItem.php, _underscore/Model/Client/ItemFulfillmentItemUnit.php, _underscore/Model/Client/ItemFulfillmentPackage.php, _underscore/Model/Compass/AdvanceShippingNotice.php, dbchanges2/Core/2026-02-13 - 75601 - RecursiveItemFulfillmentCreation.sql, dbchanges2/Core/2026-06-04 - RecursiveItemFulfillmentPut.sql |
|
|
17
|
-
| [Surface Resolver (_Model_Core_Surface::resolve — replaces Page::meta)](features/surface-resolver.md) | The runtime for the platform-wide **Surface** UI presentation layer: 9 `_underscore` models plus a cached resolver, `_Model_Core_Surface::resolve(&$api, string | _underscore/Model/Core/Surface.php, dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql, dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql, dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql, dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql, dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql, _underscore/Model/Core/SurfaceElement.php, _underscore/Model/Core/Action.php, _underscore/Model/Core/Vocabulary.php, _underscore/Model/Core/VocabularyTerm.php, _underscore/Model/Core/Message.php, _underscore/Model/Client/SurfaceOverride.php, _underscore/Model/Client/MessageTranslation.php, _underscore/Model/Client/ThemeToken.php, _underscore/Model/Core/Page.php |
|
|
17
|
+
| [Surface Resolver (_Model_Core_Surface::resolve — replaces Page::meta)](features/surface-resolver.md) | The runtime for the platform-wide **Surface** UI presentation layer: 9 `_underscore` models plus a cached resolver, `_Model_Core_Surface::resolve(&$api, string | _underscore/Model/Core/Surface.php, _underscore/Model/Client/AclRecordScript.php, _underscore/Model/Core/RecordScript.php, dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql, dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql, dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql, dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql, dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql, _underscore/Model/Core/SurfaceElement.php, _underscore/Model/Core/Action.php, _underscore/Model/Core/Vocabulary.php, _underscore/Model/Core/VocabularyTerm.php, _underscore/Model/Core/Message.php, _underscore/Model/Client/SurfaceOverride.php, _underscore/Model/Client/MessageTranslation.php, _underscore/Model/Client/ThemeToken.php, _underscore/Model/Core/Page.php |
|
|
18
18
|
| [Tracking-Number Bridge Migration (ASN / Item Fulfillment / Item Receipt)](features/tracking-number-bridges.md) | Shipment tracking numbers used to live as **scalar FK columns** (`trackingNumberId`, `returnTrackingNumberId`) directly on the lowest-level "unit"/"item" tables | api2/Component/Api/V2/V2.php, _underscore/Model/Client/AdvanceShippingNoticeItemUnit.php, _underscore/Model/Client/AdvanceShippingNoticeItemUnits/TrackingNumber.php, _underscore/Model/Client/ItemFulfillmentItemUnits/TrackingNumber.php, _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Prudential/AdvanceShippingNotice.php, _underscore/Model/Compass/AdvanceShippingNotice.php, _underscore/Trait/Netsuite/ItemFulfillment.php, api2/Component/Api/Cxml/Cxml.php, dbchanges2/Client/2026-06-10 - TrackingNumberBridges.sql, dbchanges2/Core/2026-06-10 - TrackingNumberBridges.sql, dbchanges2/Client_Prudential/2026-06-15 - ItemFulfillmentTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Quad/2026-06-18a - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Nychh/2026-06-19b - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Nychh/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql, dbchanges2/Client_Quad/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql |
|
|
19
19
|
| [Units for Items for Purchase Orders — Data Structure](features/units-for-items-for-purchase-orders.md) | Describes how unit (serialized inventory) data is linked to sales-order and purchase-order line items behind the `units-for-items-for-purchase-orders` TableView | |
|
|
@@ -6,12 +6,16 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [jcardinal]
|
|
9
|
+
updated: 2026-07-01
|
|
10
|
+
owners: [jcardinal, apeterson]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Core/Surface.php
|
|
13
|
+
- _underscore/Model/Client/AclRecordScript.php
|
|
14
|
+
- _underscore/Model/Core/RecordScript.php
|
|
13
15
|
- dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql
|
|
14
16
|
- dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql
|
|
17
|
+
- dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql
|
|
18
|
+
- dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql
|
|
15
19
|
- dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql
|
|
16
20
|
- dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql
|
|
17
21
|
- dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql
|
|
@@ -113,6 +117,32 @@ which surfaced latent resolution gaps in `_buildBundle`/`_resolveElement`:
|
|
|
113
117
|
`_Model_<Client>_*` overrides + `Client_ApiPayloadInterceptor` hooks — the Surface layer does
|
|
114
118
|
**not** absorb business logic.
|
|
115
119
|
|
|
120
|
+
## How a surfaces scripted-API call is authorized (route-level gate)
|
|
121
|
+
|
|
122
|
+
Authorizing a surfaces script (`meta`, `meta-group`, `debug`) is a **two-layer** check. Get the
|
|
123
|
+
first layer wrong and you get a **403 `EZ-1 AUTHORIZATION`** before any element ACL runs.
|
|
124
|
+
|
|
125
|
+
1. **Route-level dispatch gate — `Client_<tenant>.AclRecordScripts`.** The surfaces dispatcher
|
|
126
|
+
authorizes each API script (route + method + phpMethod) via the **CLIENT-tier**
|
|
127
|
+
`AclRecordScripts` table (cols `uuid`, `recordScriptId` → `Core.RecordScripts`, `roleId` →
|
|
128
|
+
`Client_<tenant>.Roles`). The check is keyed on the JWT's **`id.client.roles`** (client roles),
|
|
129
|
+
**NOT** the platform `core.roles` and **NOT** the Core `AclRecordPermissions` chain.
|
|
130
|
+
`Core.RecordScripts` maps route/method/phpMethod → an ACL "record" (`recordId`). The three
|
|
131
|
+
surfaces scripts: `meta` (GET, phpMethod `meta`), `meta-group` (GET, `metaGroup`), `debug` (GET,
|
|
132
|
+
`metaDebug`). `_Model_Core_Surface::resolve/metaGroup` read the caller's roles from
|
|
133
|
+
**`id.client.roles`** (Surface.php ~lines 56/87).
|
|
134
|
+
2. **Element-level ACL (runs *inside* `meta`/`metaGroup`, AFTER the route gate passes)** — dropping
|
|
135
|
+
individual surface elements uses a **separate** client-tier table `_Model_Client_AclActionPermission`,
|
|
136
|
+
gated by each `SurfaceElement.aclActionId`. This is the "composition, not absorption" step in
|
|
137
|
+
`resolve()` step 4; it never causes the route-level 403.
|
|
138
|
+
|
|
139
|
+
**Debugging a surfaces 403 EZ-1 (hard-won):** it is the route-level gate — a missing
|
|
140
|
+
`Client_<tenant>.AclRecordScripts` grant resolved by **client role**. Do **not** chase
|
|
141
|
+
`Core.AclRecordPermissions`, `appId` scoping, or `AclLogicGroups`/`AclRecordExpressions` — all were
|
|
142
|
+
investigated and ruled out as red herrings: `appId = NULL` is the normal working pattern, and the
|
|
143
|
+
Core record grants + their logic-group expressions all evaluate `all`/`"1"`. The gate is purely
|
|
144
|
+
"does this client role have an `AclRecordScripts` row for this `recordScriptId`?"
|
|
145
|
+
|
|
116
146
|
## Gotchas
|
|
117
147
|
|
|
118
148
|
- **A Core RecordScript needs `Client.AclRecordScripts` rows per role to be invokable — record-level
|
|
@@ -167,8 +197,33 @@ which surfaced latent resolution gaps in `_buildBundle`/`_resolveElement`:
|
|
|
167
197
|
per-role `Client.AclRecordScripts` dispatch grant. Verified via toga-db — `meta` script id 25 was
|
|
168
198
|
granted roles 1/3/4 but `meta-group` (id 27) had **none** until the grant ran. Any new surfaces
|
|
169
199
|
scripted-API method repeats this: record READ never authorizes dispatch.
|
|
200
|
+
- **The meta-group grant is seeded per-tenant, not globally — new tenants repeat the 403.** When
|
|
201
|
+
`meta-group` shipped, the `AclRecordScripts` grant was seeded **only** for `Client_Compass`
|
|
202
|
+
(recordScriptId → roles 1,3,4). `Client_Quad` and `Client_CompassCanada` had **no** grant for the
|
|
203
|
+
meta-group recordScriptId at all — their `meta` was granted to client role 1 (why `meta` worked but
|
|
204
|
+
`meta-group` 403'd for the same user/token). Same-user-different-script + same-script-different-tenant
|
|
205
|
+
is the fingerprint of this gap. Fix (2026-07-01): two `dbchanges2/Client_<tenant>/` migrations
|
|
206
|
+
granting meta-group to **client role 1** (parity with each tenant's existing `meta` grant),
|
|
207
|
+
id-agnostic (resolve `Core.RecordScripts` id by route/method/phpMethod, cross-DB read of `Core.*`
|
|
208
|
+
per the 2026-06-29c ACL pattern) and re-runnable via `NOT EXISTS`. Surface-layer DB changes ship to
|
|
209
|
+
the **beta** environment only. ⚠ **Root-cause not yet closed:** the meta-group grant should be
|
|
210
|
+
added to the `dbchanges2/Client/` **baseline (blank) seed** so newly-cut clients get it
|
|
211
|
+
automatically — otherwise the next migrated tenant hits the identical 403. Only Compass (already
|
|
212
|
+
correct), Quad, and CompassCanada are addressed. Also note **role breadth**: Compass exposes
|
|
213
|
+
surfaces to roles 1,3,4; Quad/CC got role 1 only (parity with their `meta`). If Quad/CC should
|
|
214
|
+
match Compass, a follow-up migration aligning both `meta` and `meta-group` to roles 1,3,4 is needed.
|
|
170
215
|
|
|
171
216
|
## Change history
|
|
217
|
+
- 2026-07-01 — Documented the full route-level authorization mechanism (new section): the surfaces
|
|
218
|
+
script gate is `Client_<tenant>.AclRecordScripts` keyed on JWT `id.client.roles` (client roles,
|
|
219
|
+
not `core.roles`, not `Core.AclRecordPermissions`); element-drop is a separate
|
|
220
|
+
`_Model_Client_AclActionPermission`/`aclActionId` layer running inside `resolve()` after the route
|
|
221
|
+
gate; captured the red-herring list (`appId=NULL` is normal, Core grants all evaluate `all`/`"1"`).
|
|
222
|
+
Fixed the `meta-group` 403 EZ-1 for **Quad** and **Compass Canada** — the meta-group
|
|
223
|
+
`AclRecordScripts` grant had been seeded for Compass only; added two id-agnostic, NOT-EXISTS-guarded
|
|
224
|
+
`Client_<tenant>/` migrations granting meta-group to client role 1 (beta env only). Flagged the
|
|
225
|
+
open root-cause: add the grant to the `Client/` baseline seed, and decide role breadth (1 vs 1,3,4)
|
|
226
|
+
for Quad/CC. (apeterson)
|
|
172
227
|
- 2026-06-30 — Added `metaGroup()` (`GET /v2/surfaces/meta-group?slugs=a,b,c`): batched composition
|
|
173
228
|
over the existing per-slug `resolve()` cache, capped at `META_GROUP_MAX_SLUGS=25`, unknown slug →
|
|
174
229
|
empty bundle (chosen over a `parentSurfaceId` GROUP row post-CTO-review). Fixed latent
|
|
@@ -6,7 +6,7 @@ project: TOGa 2.5 Supply
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-07-01
|
|
10
10
|
owners: [apeterson]
|
|
11
11
|
files:
|
|
12
12
|
- toga25-supply/src/fieldsConfig/index.ts
|
|
@@ -64,6 +64,22 @@ and the `useClientFields()` hook.
|
|
|
64
64
|
`yourKey: BY_CLIENT[clientSlug] ?? DEFAULT_*`. Avoids editing all ~13 role bundles.
|
|
65
65
|
`DEFAULT_CLIENT_FIELDS` must also carry the new key (ultimate fallback).
|
|
66
66
|
|
|
67
|
+
### De-roling a client×role bundle to client-only
|
|
68
|
+
|
|
69
|
+
Some bundles were role-keyed (`{ADMIN, MANAGER}`) only for historical reasons — the roles held
|
|
70
|
+
byte-identical config. Those are being flattened to **client-only** (part of the ongoing
|
|
71
|
+
backend-driven / de-role goal). `orderViewFields` is now client-only: `COMPASS` and `COMPASSCANADA`
|
|
72
|
+
read their flat client JSON via `mergeOrderView(<client>OrderViewFields)` in
|
|
73
|
+
`src/fieldsConfig/index.ts` instead of reading `.ADMIN`/`.MANAGER` off the file, and
|
|
74
|
+
`FIELDS/COMPASSCANADA/orderViewFields.json` was flattened from `{ADMIN, MANAGER}` to a flat object.
|
|
75
|
+
|
|
76
|
+
When de-roling, watch the read path: `COMPASS`'s JSON was **already flat** but was still being read
|
|
77
|
+
via `.ADMIN` → `undefined` → the client silently fell back to `DEFAULT` and lost its
|
|
78
|
+
`salesOrderDetailsConfig` joins. This is the flat-bundle trap from the gotcha below — the fix is to
|
|
79
|
+
read the bundle itself, not a role key. `approvalViewFields` and `salesOrdersPageFields` remain
|
|
80
|
+
role-keyed (their ADMIN/MANAGER genuinely differ); fully de-roling those and removing `resolveRole`
|
|
81
|
+
is a separate pending task.
|
|
82
|
+
|
|
67
83
|
### Hydration (non-serializable bits)
|
|
68
84
|
|
|
69
85
|
Keep JSON **serializable** — strings, enums, arrays. No JSX, no functions, no Tailwind classes.
|
|
@@ -143,6 +159,12 @@ modal's chrome from the same per-client JSON, not from the component:
|
|
|
143
159
|
`vendorItemRecordViewFields` are record-modal-layout examples living under `src/layout/.../viewModel/FIELDS/`.
|
|
144
160
|
|
|
145
161
|
## Change history
|
|
162
|
+
- 2026-07-01 — De-roled `orderViewFields` from client×role to **client-only**: COMPASS/COMPASSCANADA
|
|
163
|
+
now read a flat client JSON via `mergeOrderView(...)` in `fieldsConfig/index.ts` (COMPASSCANADA's
|
|
164
|
+
JSON flattened from byte-identical `{ADMIN,MANAGER}`). Fixed a latent bug where COMPASS's
|
|
165
|
+
already-flat JSON was read via `.ADMIN` → `undefined` → silent fallback to DEFAULT (lost its
|
|
166
|
+
`salesOrderDetailsConfig` joins). `approvalViewFields`/`salesOrdersPageFields` stay role-keyed;
|
|
167
|
+
full de-role + `resolveRole` removal remains pending. Type-check clean. (apeterson)
|
|
146
168
|
- 2026-06-25 — SalesOrder action buttons render by presentation intent: `kind` (icon/text/viewLog) picks the renderer, `variant` a closed color set, `valueKey` is pure click-dispatch; a `componentRegistry` keyed by `kind` hydrates dedicated components (e.g. ViewLogButton). Added the `recordActionFields`-shape gotcha (per-persona blocks + sibling `poNumber` sub-config; walk `actionOptions` only). (apeterson)
|
|
147
169
|
- 2026-06-25 — Extended to record-modal layouts: per-mode `*ViewFields`/`*EditFields` JSON under `src/layout/.../viewModel/FIELDS/`, JSON-driven modal `header` (modalTag/labelTemplate/statusBadge) + `editItem` button gated via the Flag/Rule engine, and the flat-bundle defensive-default gotcha. (apeterson)
|
|
148
170
|
- 2026-06-23 — Documented from the `add-client-fields` skill during initial knowledge seed. (apeterson)
|
|
@@ -21,8 +21,9 @@
|
|
|
21
21
|
| [Startech Webhook Handler (worker2)](features/startech-webhook-handler.md) | Receives inbound webhook events from Startech (Easeedesk) and creates or updates the corresponding ticket in TOGA 2.0. | worker2/Worker/Startech.php |
|
|
22
22
|
| [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus programmatically. | .claude/skills/plan-ticket/scripts/talos.js |
|
|
23
23
|
| [Talos Pricing Automation (worker2 Cron — AWS Actuals, Calibration, Monthly Report)](features/talos-pricing-automation.md) | The worker2 half of the **Talos Pricing Platform** (see the talos `pricing-cogs-model` and tools `talos-pricing-ui` docs for the other halves). | worker2/Worker/Talos/Pricing.php, worker2/Database/TalosPricingCrons.sql |
|
|
24
|
+
| [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` (in addition to the upstream `Export` action — see [Teams Meeting Transcript Export](./teams-transcript-export.md)) now runs a fully | worker2/Worker/Team/Transcripts.php, worker2/bin/reprocess-transcripts.php, worker2/Config/production.ini, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a |
|
|
24
25
|
| [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php |
|
|
25
|
-
| [Teams Meeting Transcript Export](features/teams-transcript-export.md) | `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for Teams meeting transcripts produced by a set of organizers
|
|
26
|
+
| [Teams Meeting Transcript Export](features/teams-transcript-export.md) | `_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for Teams meeting transcripts produced by a set of organizers and archives t | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
|
|
26
27
|
| [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | `_Worker_Vapi` ([worker2/Worker/Vapi.php](worker2/Worker/Vapi.php)) is the **PHP side of the AI-BDR call loop** — the webhook that receives VAPI's end-of-call r | worker2/Worker/Vapi.php, worker2/Worker/Ai/Bdr/Vapi.php |
|
|
27
28
|
| [WJE Freshservice Sync (worker2)](features/wje-freshservice-sync.md) | WJE ("WJE IT", helpdesk `wje.freshservice.com`) is a **Freshservice**-based help-desk client whose tickets, contacts, assets, groups, categories, and canned res | worker2/Worker/Wje.php, _underscore/Component/Api/Wje/Wje.php, _underscore/Model/Wje/Ticket.php, _underscore/Model/Wje/TicketNote.php, _underscore/Model/Wje/Contact.php, _underscore/Model/Wje/Unit.php, _underscore/Model/Wje/TicketTeam.php, _underscore/Model/Wje/TicketCategory.php, _underscore/Model/Wje/AssetType.php, _underscore/Model/Wje/PredefinedReply.php, library/app/api/wje.php, worker/crons/toga2/wje/import_supporting_records.php, worker/crons/toga2/wje/sync_togasupply_wje.php, worker/crons/notifications/reports/wje/wje_common.php, library/app/systemmonitor/wje.php, dbchanges2/Client_Wje/2024-10-04 - WjeOnboarding.sql |
|
|
28
29
|
| [Ticket → ClickUp Pseudocode Planning (Talos-grounded)](workflows/ticket-to-pseudocode-planning.md) | A repeatable procedure for turning a ClickUp ticket into a reviewed, formatted implementation plan posted back to the ticket's `📝 Pseudocode` custom field. | test/@dave/clickup_md2delta.js |
|
|
@@ -0,0 +1,158 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-06-30
|
|
10
|
+
owners: [jcardinal]
|
|
11
|
+
files:
|
|
12
|
+
- worker2/Worker/Team/Transcripts.php
|
|
13
|
+
- worker2/bin/reprocess-transcripts.php
|
|
14
|
+
- worker2/Config/production.ini
|
|
15
|
+
- dbchanges2/Team/2026-06-30a
|
|
16
|
+
- dbchanges2/Team/2026-06-30b
|
|
17
|
+
- dbchanges2/Team/2026-06-30c
|
|
18
|
+
- dbchanges2/Team/2026-06-30d
|
|
19
|
+
- dbchanges2/Team/2026-06-30e
|
|
20
|
+
- dbchanges2/Core/2026-06-30a
|
|
21
|
+
related:
|
|
22
|
+
- ./teams-transcript-export.md
|
|
23
|
+
- ./creating-worker-actions.md
|
|
24
|
+
- ../architecture.md
|
|
25
|
+
- ../../../1.0/apps/tools/features/talos-kb-documents-admin.md
|
|
26
|
+
- ../../../1.0/apps/test/features/talos-kb-pipeline.md
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## Summary
|
|
30
|
+
|
|
31
|
+
`_Worker_Team_Transcripts` (in addition to the upstream `Export` action —
|
|
32
|
+
see [Teams Meeting Transcript Export](./teams-transcript-export.md)) now runs a fully
|
|
33
|
+
automated, cron-driven pipeline that ingests raw Teams transcripts into the **Talos / TOGa IQ**
|
|
34
|
+
AWS Bedrock knowledge bases. It replaces the old **manual** two-page web tooling
|
|
35
|
+
(`test/team/talos/kb_processor.php` + `kb_processor.ini` — see
|
|
36
|
+
[Talos Knowledge Base Pipeline](../../../1.0/apps/test/features/talos-kb-pipeline.md)); the
|
|
37
|
+
cleanup/classify/approve/archive/sync logic is a port of that script, but driven by cron jobs
|
|
38
|
+
and by **DB-editable** config instead of an `.ini`.
|
|
39
|
+
|
|
40
|
+
Flow: raw VTT at `s3://toga-private/transcripts/{date}/…` → `Scan` enqueues one `Process` job
|
|
41
|
+
per unprocessed key → `Process` AI-cleans + AI-classifies → approved `.txt` + Bedrock metadata
|
|
42
|
+
sidecar written to `s3://togaiq/development-team/{kb_slug}/approved/` → raw archived → Bedrock
|
|
43
|
+
`startIngestionJob` on that KB's data source.
|
|
44
|
+
|
|
45
|
+
## Key files / entry points
|
|
46
|
+
|
|
47
|
+
- `Worker/Team/Transcripts.php` — actions `Scan`, `Process(string $sourceKey)`,
|
|
48
|
+
`SyncKb(string $kbSlug)` (all alongside the existing `Export`).
|
|
49
|
+
- `bin/reprocess-transcripts.php` — standalone CLI backlog reprocessor.
|
|
50
|
+
- `Config/production.ini` `[talos]` section — S3/Bedrock targets (see Configuration).
|
|
51
|
+
- DB config + ledger tables in the `Team.*` schema (see Data model), authored in dbchanges2
|
|
52
|
+
under `Team/2026-06-30a..e` + `Core/2026-06-30a`.
|
|
53
|
+
|
|
54
|
+
## How it works
|
|
55
|
+
|
|
56
|
+
### `Scan` (cron)
|
|
57
|
+
Lists unprocessed raw transcripts under the `toga-private` `transcripts/` prefix (skips any
|
|
58
|
+
`sourceKey` already in the `Team.TranscriptProcessing` ledger) and enqueues one `Process` job
|
|
59
|
+
each via `_Worker::runTask('Team/Transcripts/Process', ['sourceKey' => …])`. Scheduled by a
|
|
60
|
+
`Core.CronJobs` row (weekday business hours).
|
|
61
|
+
|
|
62
|
+
### `Process(string $sourceKey)`
|
|
63
|
+
Port of `kb_processor`, per transcript:
|
|
64
|
+
1. Fetch the raw VTT from S3 (`toga-private`, **us-west-2**).
|
|
65
|
+
2. Call the TOGa IQ AI endpoint `https://api.togaiq.com/api/ai/generate`, model
|
|
66
|
+
`bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0`, with an **extended** `output_schema`
|
|
67
|
+
returning **both** `knowledge_doc` **and** `kb_slug` — the AI picks the single best active KB
|
|
68
|
+
from the supplied list, or `general`. The system prompt is **assembled at runtime** from DB
|
|
69
|
+
tables: a template body with `{{APPLICATIONS}}`/`{{CLIENTS}}`/`{{PEOPLE}}`/`{{BUSINESS_TERMS}}`/`{{EXAMPLES}}`
|
|
70
|
+
placeholders filled from `Team.TranscriptPromptTerms`.
|
|
71
|
+
3. Apply DB-driven text replacements from `Team.TranscriptReplacements` (`:c` case-sensitive,
|
|
72
|
+
`:w` whole-word, `:cw`) via `preg_replace_callback` **so replacement backrefs stay literal**.
|
|
73
|
+
4. Build the approved name `"{YYYY-MM-DD} - {Title Case}.txt"`.
|
|
74
|
+
5. Upload to `s3://togaiq/development-team/{kb_slug}/approved/` (**us-east-1**) plus a
|
|
75
|
+
`{name}.txt.metadata.json` Bedrock sidecar.
|
|
76
|
+
6. Archive the raw VTT to `…/{kb_slug}/archive/{date}/`, then delete the source.
|
|
77
|
+
7. AWS Bedrock `startIngestionJob` on that KB's data source (retry 5×30s on
|
|
78
|
+
`ConflictException` / throttling). **If `kb_slug === 'general'`, sync ALL active KBs.**
|
|
79
|
+
|
|
80
|
+
Idempotency + status are tracked in `Team.TranscriptProcessing`
|
|
81
|
+
(`pending/cleaned/uploaded/archived/synced/failed`).
|
|
82
|
+
|
|
83
|
+
### `SyncKb(string $kbSlug)`
|
|
84
|
+
On-demand single-KB re-sync (used by the Tools delete flow — see
|
|
85
|
+
[Talos KB Documents Admin](../../../1.0/apps/tools/features/talos-kb-documents-admin.md)).
|
|
86
|
+
|
|
87
|
+
### `bin/reprocess-transcripts.php` (CLI backlog reprocessor)
|
|
88
|
+
Standalone CLI that bootstraps `_underscore`, lists existing raw transcripts under the
|
|
89
|
+
`toga-private` `transcripts/` prefix, and runs each through `Process()`. Flags
|
|
90
|
+
`--prefix` / `--limit` / `--dry-run`. Tolerates **both** the old `{slug}/{date}/` layout and the
|
|
91
|
+
new `{date}/` layout. Requires the `ENVIRONMENT` env var, e.g.
|
|
92
|
+
`ENVIRONMENT=production php bin/reprocess-transcripts.php --dry-run`.
|
|
93
|
+
|
|
94
|
+
### Graph download hardening
|
|
95
|
+
The Graph transcript download is hardened against SSRF / token leak: `https`-only, a
|
|
96
|
+
`*.microsoft.com` host allowlist, and `FOLLOWLOCATION` off.
|
|
97
|
+
|
|
98
|
+
## Data model (`Team.*` schema, DB alias core cluster)
|
|
99
|
+
|
|
100
|
+
Authored in dbchanges2 (`Team/2026-06-30a..e`, `Core/2026-06-30a`, all dated 2026-06-30). All
|
|
101
|
+
PKs `INT UNSIGNED` to match the existing Talos Team-table family.
|
|
102
|
+
|
|
103
|
+
- **`Team.TranscriptReplacements`** — `pattern, replacement, isCaseSensitive, isWholeWord, isActive`.
|
|
104
|
+
Seeded ~109 rules from the old `.ini`.
|
|
105
|
+
- **`Team.TranscriptPromptTerms`** — `category` ENUM (`application/client/person/businessTerm/example`),
|
|
106
|
+
`term, isActive, dtCreated, dtUpdated`. Seeded from the `.ini` CORRECTIONS lists. PEOPLE are
|
|
107
|
+
stored as full "First Last" names (only 2 confidently paired; the rest seeded as individual
|
|
108
|
+
tokens, intended to be reconciled via the Tools vocabulary UI).
|
|
109
|
+
- **`Team.TranscriptPromptTemplate`** — `name, bodyText` (w/ placeholders), `instruction, model,
|
|
110
|
+
temperature, maxTokens, timeoutSeconds, isActive`. One active row seeded from the `.ini` system prompt.
|
|
111
|
+
- **`Team.KnowledgeBases`** — `slug, name, isGeneral, bedrockKbId, isActive`. Seeded `general` +
|
|
112
|
+
`office-depot`. NOTE: the Tools browse UI derives the **live** KB list from S3 folders, not this table.
|
|
113
|
+
- **`Team.TranscriptProcessing`** — `sourceKey UNIQUE(255), kbSlug, approvedKey, archiveKey,
|
|
114
|
+
status` ENUM (`pending/cleaned/uploaded/archived/synced/failed`), `aiModel, failureReason,
|
|
115
|
+
dtProcessed`. Pipeline ledger + idempotency.
|
|
116
|
+
- **`Core.CronJobs`** INSERT for action `Team/Transcripts/Scan` (weekday business hours);
|
|
117
|
+
includes `dtCreated = NOW()` since the column has no default, and depends on the
|
|
118
|
+
`maxExecutionTime` column migration.
|
|
119
|
+
|
|
120
|
+
## Configuration
|
|
121
|
+
|
|
122
|
+
`worker2/Config/production.ini` `[talos]`: `s3_bucket=togaiq`, `s3_region=us-east-1`,
|
|
123
|
+
`bedrock_region=us-east-1`, `kb_prefix=development-team-`, `s3_root_path=development-team/`.
|
|
124
|
+
|
|
125
|
+
## Gotchas / known issues
|
|
126
|
+
|
|
127
|
+
- **Cross-region S3.** The raw bucket `toga-private` is **us-west-2** while `togaiq` is
|
|
128
|
+
**us-east-1**. `Process()` uses two S3 clients; the archive step is a **cross-region copy**.
|
|
129
|
+
- **`[talos]` keys were added to `production.ini` only** — the beta/development configs still
|
|
130
|
+
need them before the pipeline runs in those environments.
|
|
131
|
+
- **`general` KB fans out.** A `Process()` classified as `general` (or a `general` sync) triggers
|
|
132
|
+
a sync of **all** active KBs, not one.
|
|
133
|
+
- **Replacements must use `preg_replace_callback`** so replacement text with `$`/`\` backrefs is
|
|
134
|
+
treated literally (a plain `preg_replace` would interpret them).
|
|
135
|
+
- **worker2 had no prior CLI precedent** — the backlog reprocessor bootstraps by
|
|
136
|
+
`chdir($projectRoot)` then `require vendor/autoload.php` + `_underscore.php`, and requires the
|
|
137
|
+
`ENVIRONMENT` env var.
|
|
138
|
+
- **Deploy TODOs left in code:** verify the live Bedrock KB region and data-source name against
|
|
139
|
+
the actual account before first prod run.
|
|
140
|
+
- **Pre-existing plaintext secrets** in `worker2/Config/production.ini` and the reference
|
|
141
|
+
`kb_processor.ini` (AWS / aegra) should be rotated — not introduced by this work; no new secret
|
|
142
|
+
literals were added to any new file or migration.
|
|
143
|
+
|
|
144
|
+
## Change history
|
|
145
|
+
|
|
146
|
+
- 2026-06-30 — Built the automated cron-driven ingestion pipeline (`Scan`/`Process`/`SyncKb`) +
|
|
147
|
+
the `bin/reprocess-transcripts.php` backlog CLI, porting the manual `kb_processor.php`. Added
|
|
148
|
+
the `Team.*` config/ledger schema (DB-editable replacements, prompt terms, prompt template, KB
|
|
149
|
+
list, processing ledger) and the `Scan` cron. Classification moved from title-based to AI (single
|
|
150
|
+
best KB, or `general`). Config vocabulary/replacements are now DB-driven and editable via the
|
|
151
|
+
Tools UI. (jcardinal)
|
|
152
|
+
|
|
153
|
+
## Related docs
|
|
154
|
+
|
|
155
|
+
- [Teams Meeting Transcript Export](./teams-transcript-export.md) — the upstream S3 producer.
|
|
156
|
+
- [Talos KB Documents Admin](../../../1.0/apps/tools/features/talos-kb-documents-admin.md) — the Tools UI.
|
|
157
|
+
- [Talos Knowledge Base Pipeline](../../../1.0/apps/test/features/talos-kb-pipeline.md) — the manual script this ports.
|
|
158
|
+
- [Creating Worker Actions](./creating-worker-actions.md)
|
|
@@ -6,8 +6,8 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-06-
|
|
10
|
-
owners: ["ajean"]
|
|
9
|
+
updated: 2026-06-30
|
|
10
|
+
owners: ["ajean", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Team/Transcripts.php
|
|
13
13
|
- worker2/Config/production.ini
|
|
@@ -16,13 +16,18 @@ files:
|
|
|
16
16
|
related:
|
|
17
17
|
- ../architecture.md
|
|
18
18
|
- ./creating-worker-actions.md
|
|
19
|
+
- ./talos-transcript-ingestion.md
|
|
19
20
|
---
|
|
20
21
|
|
|
21
22
|
## Summary
|
|
22
23
|
|
|
23
24
|
`_Worker_Team_Transcripts` (action `Team/Transcripts/Export`) polls Microsoft Graph for
|
|
24
|
-
Teams meeting transcripts produced by a set of organizers
|
|
25
|
-
|
|
25
|
+
Teams meeting transcripts produced by a set of organizers and archives the raw WebVTT to S3.
|
|
26
|
+
As of 2026-06-30 it **no longer classifies by client**: raw transcripts land **flat** at
|
|
27
|
+
`s3://toga-private/transcripts/{YYYY-MM-DD}/{HHMM}_{title-slug}_{shortId}.vtt` (no client
|
|
28
|
+
folder). Classification now happens later, by AI, in the downstream ingestion pipeline — see
|
|
29
|
+
[Talos Transcript Ingestion](./talos-transcript-ingestion.md), which is the consumer of this
|
|
30
|
+
S3 output. Scheduled via a single
|
|
26
31
|
`Core.CronJobs` row firing weekdays at 9:30 / 11:30 / 13:30 / 14:30 / 15:30 / 16:30 / 17:30
|
|
27
32
|
Central (`30 9,11,13,14,15,16,17 * * 1-5`).
|
|
28
33
|
|
|
@@ -69,11 +74,15 @@ date. `createdDateTime` is stamped minutes after the actual occurrence, so it da
|
|
|
69
74
|
correctly. `getMeeting()` is therefore used only for the subject. `dtExported` uses
|
|
70
75
|
`UTC_TIMESTAMP()` (not `NOW()`) so all three timestamps are UTC.
|
|
71
76
|
|
|
72
|
-
**Classification
|
|
73
|
-
`Core.Clients`
|
|
77
|
+
**Classification: REMOVED (2026-06-30).** The old title-based classification (substring match
|
|
78
|
+
against `Core.Clients` + `[teamsClientAliases]`) was inaccurate and has been deleted
|
|
79
|
+
(`classify()`, `buildClientIndex()`, `loadClientNames()`, `loadClientAliases()`, along with the
|
|
80
|
+
old `Backfill()`/`moveObject()`). Classification is now done by the AI in the downstream
|
|
81
|
+
ingestion `Process()` step, which picks the single best knowledge base (or `general`).
|
|
74
82
|
|
|
75
|
-
**S3 layout:** `s3://
|
|
76
|
-
(date/time in Central).
|
|
83
|
+
**S3 layout (current):** `s3://toga-private/transcripts/{YYYY-MM-DD}/{HHMM}_{title-slug}_{shortId}.vtt`
|
|
84
|
+
— **flat, no client/general folder** (date/time in Central). The old
|
|
85
|
+
`…/{client-or-general}/{date}/…` layout is history; the downstream reprocessor CLI tolerates both.
|
|
77
86
|
|
|
78
87
|
## Data model
|
|
79
88
|
|
|
@@ -162,6 +171,12 @@ Policy gap; a `400` only on a UPN means the id was never resolved to a GUID. (A
|
|
|
162
171
|
|
|
163
172
|
## Change history
|
|
164
173
|
|
|
174
|
+
- 2026-06-30 — **Removed title-based client classification and the client-folder S3 layout.**
|
|
175
|
+
Raw transcripts now land flat at `transcripts/{date}/…` with no client subfolder; deleted
|
|
176
|
+
`classify()`/`buildClientIndex()`/`loadClientNames()`/`loadClientAliases()` and the old
|
|
177
|
+
`Backfill()`/`moveObject()`. Classification moved downstream to the AI in the new
|
|
178
|
+
[Talos Transcript Ingestion](./talos-transcript-ingestion.md) pipeline, which is now the
|
|
179
|
+
consumer of this S3 output. (jcardinal)
|
|
165
180
|
- 2026-06-18 — Fixed recurring-meeting date misfiling: archive date + `dtMeeting` now come from
|
|
166
181
|
the transcript `createdDateTime`, not the series-anchor `meeting->startDateTime` (worker2 PR
|
|
167
182
|
#82, merged + deployed). Added `Backfill()` and ran it in prod (61/66 objects re-filed, 0
|
package/knowledge/INDEX.md
CHANGED
|
@@ -12,12 +12,12 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
12
12
|
- **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
|
|
13
13
|
- **test** (Test) — 12 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
|
|
14
14
|
- **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
|
|
15
|
-
- **tools** (Tools) —
|
|
15
|
+
- **tools** (Tools) — 8 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
|
|
16
16
|
|
|
17
17
|
## 2.0 framework
|
|
18
18
|
|
|
19
19
|
- **_underscore** (_Underscore) _(framework core)_ — 21 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
20
|
-
- **worker2** (Worker) —
|
|
20
|
+
- **worker2** (Worker) — 25 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
21
21
|
- **api2** (API) — 7 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
22
22
|
- **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
|
23
23
|
- **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
|
|
@@ -15,12 +15,13 @@ client: compass-canada
|
|
|
15
15
|
type: profile
|
|
16
16
|
status: active
|
|
17
17
|
updated: 2026-06-30
|
|
18
|
-
owners: [jcardinal, bala, tcox]
|
|
18
|
+
owners: [jcardinal, bala, tcox, apeterson]
|
|
19
19
|
files: []
|
|
20
20
|
related:
|
|
21
21
|
- ../compass-usa/profile.md
|
|
22
22
|
- ../../2.0/apps/toga2-commerce/features/expedited-shipping-gating.md
|
|
23
23
|
- ../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
|
|
24
|
+
- ../../2.0/apps/_underscore/features/surface-resolver.md
|
|
24
25
|
---
|
|
25
26
|
|
|
26
27
|
## Summary
|
|
@@ -50,6 +51,12 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
|
|
|
50
51
|
see [Config-Driven Expedited Shipping Gating](../../2.0/apps/toga2-commerce/features/expedited-shipping-gating.md).
|
|
51
52
|
|
|
52
53
|
## Notes
|
|
54
|
+
- **Surface layer (beta):** like Quad, `Client_CompassCanada` had **no** `AclRecordScripts` dispatch
|
|
55
|
+
grant for the surfaces `meta-group` script (only `meta`, to client role 1) → `GET /v2/surfaces/meta-group`
|
|
56
|
+
403'd EZ-1 while `meta` worked. Fixed 2026-07-01 via
|
|
57
|
+
`dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql` (meta-group →
|
|
58
|
+
client role 1, id-agnostic + NOT-EXISTS-guarded, beta only). Exposes surfaces to role 1 only, vs
|
|
59
|
+
Compass USA's 1/3/4. See [Surface Resolver](../../2.0/apps/_underscore/features/surface-resolver.md).
|
|
53
60
|
- Customer language preference: `UserGlobalSettings.settingId = 2` (`en` / `fr-CA`); customer-
|
|
54
61
|
facing emails are sent in EN or FR accordingly.
|
|
55
62
|
- Assortment (product-grouping) names are served in fr-CA via the `AssortmentTranslations` sidecar
|
|
@@ -14,11 +14,12 @@ client: quad
|
|
|
14
14
|
type: profile
|
|
15
15
|
status: active
|
|
16
16
|
updated: 2026-06-30
|
|
17
|
-
owners: ["jcardinal", "bala"]
|
|
17
|
+
owners: ["jcardinal", "bala", "apeterson"]
|
|
18
18
|
files: []
|
|
19
19
|
related:
|
|
20
20
|
- ../../2.0/apps/_underscore/features/tracking-number-bridges.md
|
|
21
21
|
- ../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
|
|
22
|
+
- ../../2.0/apps/_underscore/features/surface-resolver.md
|
|
22
23
|
---
|
|
23
24
|
|
|
24
25
|
## Summary
|
|
@@ -48,6 +49,13 @@ Client-specific DB change-sets live in `dbchanges2/Client_Quad/`.
|
|
|
48
49
|
(`_underscore/Model/Quad/SalesOrder.php`): non-SA orders use the inline machine, `SA%` orders
|
|
49
50
|
delegate to the base `_Model_Client_SalesOrder`. Contrast Compass, which is shipped-only. See
|
|
50
51
|
[IF stage lifecycle & order status](../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md).
|
|
52
|
+
- **Surface layer (beta):** the surfaces `meta-group` script had **no** `AclRecordScripts` dispatch
|
|
53
|
+
grant for `Client_Quad` (only `meta` was granted, to client role 1), so `GET /v2/surfaces/meta-group`
|
|
54
|
+
403'd (EZ-1) while `meta` worked. Fixed 2026-07-01 via
|
|
55
|
+
`dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql` (grants meta-group to
|
|
56
|
+
client role 1, id-agnostic + NOT-EXISTS-guarded, beta only). Quad exposes surfaces to role 1 only,
|
|
57
|
+
vs Compass's 1/3/4 — revisit if broader roles are needed. See
|
|
58
|
+
[Surface Resolver](../../2.0/apps/_underscore/features/surface-resolver.md).
|
|
51
59
|
- Tracking data: record 318 (item-level) is currently empty for this client; their tracking
|
|
52
60
|
populates the IF/shipment level (record 317, ~98% coverage) and unit level (record 319). The
|
|
53
61
|
rebuilt views use 318 (per the Compass pattern) and will show tracking once item-level
|
package/package.json
CHANGED