toga-ai 1.0.910 → 1.0.912

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -4,6 +4,7 @@
4
4
  |-----|---------|
5
5
  | [Library (1.0 Framework) Architecture](architecture.md) | The 1.0 `App_` framework core: bootstrap/autoloader, the `app/` (`App_`) + `browser/` (`Browser_`) class trees, and vendored packages; open for how any 1.0 app |
6
6
  | [Address Validation Gateway (App_Api_OfficeDepot::validateAddress + USPS fallback)](features/address-validation-gateway.md) | The shared 1.0 address-validation gateway (`App_Api_OfficeDepot::validateAddress` → USPS fallback); open when standardizing/verifying a typed address before sav |
7
+ | [API Request Logging & Retention (App_ApiTransaction → Logs.API / Logs_<Tenant>)](features/api-request-logging-and-retention.md) | Where every outbound API call is logged (1.0 `Logs.API` vs the 2.0 `Logs_<Tenant>` cluster), how the routing is chosen, and the 30-day retention trim; open befo |
7
8
  | [Where a new App_ class goes — the app/ folder IS a behavioral contract](features/app-class-placement-base-contracts.md) | Where a new `App_` class goes — each `app/` folder's base class carries real inherited behavior; open before creating any class under `app/model|client|api`. |
8
9
  | [App_Sso — Reusable 1.0 SSO Initiation (SP-initiated SAML via saml.togahub.com)](features/app-sso-initiation.md) | Reusable 1.0 SP-initiated SAML SSO via `saml.togahub.com` (`App_Sso::initiate`); the canonical doc to follow for any "add SSO to `<1.0 app>`" request. |
9
10
  | [Cron Execution Monitoring (App_Framework check-in/out → CronJobExecutions)](features/cron-execution-monitoring.md) | 1.0 cron check-in/out lifecycle via `App_Framework::cronInitialization()`/`cronFinished()` → `Common.CronJobExecutions`; open to monitor a 1.0 cron or read its |
@@ -0,0 +1,89 @@
1
+ ---
2
+ title: "API Request Logging & Retention (App_ApiTransaction → Logs.API / Logs_<Tenant>)"
3
+ framework: "1.0"
4
+ repo: library
5
+ project: Library
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-09-29
10
+ owners: ["mhammontree"]
11
+ files:
12
+ - library/app/apitransaction.php
13
+ - library/app/model/logs/api.php
14
+ - library/app/model/toga2/logs/api.php
15
+ - worker/crons/maintenance/maintenance_daily.php
16
+ related:
17
+ - error-capture-1-0.md
18
+ - ../architecture.md
19
+ ---
20
+
21
+ Where every outbound API call is logged (1.0 `Logs.API` vs the 2.0 `Logs_<Tenant>` cluster), how the routing is chosen, and the 30-day retention trim; open before changing log routing, retention, or debugging past-window history.
22
+
23
+ ## Summary
24
+
25
+ `App_ApiTransaction` writes one row per outbound API call. It has **two** storage paths — the legacy 1.0 `Logs`.`API` table and the 2.0 per-tenant `Logs_<Tenant>` cluster — and picks between them per run. Retention is enforced only on the 1.0 side, by a daily worker cron, at **30 days**.
26
+
27
+ ## How it works
28
+
29
+ ### Which log DB a call lands in
30
+ `library/app/apitransaction.php:377`:
31
+ ```php
32
+ if (App_Registry::varIsSet('toga2LogsDatabase')) { /* App_Model_Toga2_Logs_Api */ }
33
+ else { /* App_Model_Logs_API → legacy Logs.API */ }
34
+ ```
35
+ - 2.0 routing is **opt-in per cron**. Only 10 crons set it, all Prudential, each hardcoding
36
+ `App_Registry::set('toga2LogsDatabase', 'Logs_Prudential')`.
37
+ - Every other `crons/toga2/*` cron falls through to the 1.0 table — **this is why 2.0 client traffic
38
+ lands in the 1.0 `Logs.API`**.
39
+
40
+ ### Retention (1.0 only)
41
+ `worker/crons/maintenance/maintenance_daily.php` trims the log tables nightly.
42
+ - `$daysToKeep_APILog` = **30** (was 90; cut 2026-09-28, TRUE-82049, commit `89ef44f35`).
43
+ - The `DELETE FROM Logs.API` is **chunked**: 25,000 rows per statement in a do/while loop with a
44
+ 1s sleep, still bounded overall by `$maxLimitForDeletes`. Every *other* table this cron trims is
45
+ a single statement — `Logs.API` is the exception because it is ~12x the next biggest table
46
+ (`Vision_Log.Log`, 45.85 GB), and a single 5,000,000-row DELETE holds locks for minutes and lags
47
+ read replicas.
48
+ - **The one-time backlog purge (~20M rows) is separate and not done** as of 2026-09-29.
49
+
50
+ ### Storage facts (measured 2026-09-28)
51
+ - 1.0 `Logs`.`API`: **565.27 GB / 34,464,186 rows**, growing **~6.3 GB/day**. `dtStamp` is the
52
+ indexed date column; `sourceJob` names the writing cron.
53
+ - Top writers by bytes (2-day sample): `sync_togasupply_compass_usa.php` 2.53 GB (128 KB/row),
54
+ `periodic_forecast_discrepancy_fix.php` 1.63 GB, `sync_togasupply_hh.php` 0.84 GB,
55
+ `sync_togasupply_elite.php` 0.53 GB, `compass/workflow/5_create_netsuite_sales_order.php`
56
+ 1,581 KB/row, `infrastructure/system_monitors.php` 3,040 KB/row.
57
+ - 2.0 logs cluster (`writer.logs.database.togahub.com`): **~1.67 TB** across 36 `Logs_<Tenant>`
58
+ DBs, oldest rows **2025-08-24** (~13 months — *not* the 90 days believed), and **no scheduled
59
+ DB events trimming it**. Largest: `Logs_Wje` 372.80 GB, `Logs_Elite` 228.99 GB,
60
+ `Logs_Compass` 185.88 GB, `Logs_Nychh` 129.71 GB, `Logs_Prudential` 125.32 GB.
61
+ `Logs_Northwell` is 43.59 GB from only 194,643 rows = 224 KB/row.
62
+
63
+ ### Decision
64
+ **`Logs.API` is forensics-only; 30 days is enough** (Jeff, 2026-09-17). Debugging that needs
65
+ older history is no longer possible — slow-burn bugs must be caught by **alerting**, not
66
+ reconstructed after the fact.
67
+
68
+ ## Gotchas
69
+
70
+ - **DELETE does not return space to disk on InnoDB.** After the backlog purge the file stays
71
+ ~565 GB with `DATA_FREE` growing. Reclaiming disk needs `OPTIMIZE TABLE` (~565 GB spare
72
+ required) or a copy-and-swap — a separate decision, not part of the retention change.
73
+ - **Do NOT move 2.0 client logging to the 2.0 cluster without solving retention there first.**
74
+ It would shift ~7 GB/day onto a cluster that is already bigger and has **no working retention**.
75
+ Deliberately descoped from TRUE-82049. Nobody has found the 2.0 archiving script — it is not in
76
+ `worker2`, `api2`, or `_underscore`.
77
+ - **`App_Model_Toga2_Logs_Api` has NO `save()` override.** The 1.0 `App_Model_Logs_API::save()`
78
+ deliberately wraps `parent::save()` in try/catch because a long-running cron can outlive its
79
+ `db_logs` connection (MySQL "server has gone away"), which once aborted whole imports mid-run.
80
+ Routing traffic to the 2.0 model without adding that guard reintroduces that failure.
81
+ - **The 2.0 model has a `source` column the 2.0 branch never sets** — moving a cron to 2.0 logging
82
+ loses the writing-cron name (`sourceJob`'s equivalent).
83
+ - **Cap payload size when adding new logging.** Team precedent: worker2's
84
+ `MAX_LOG_PAYLOAD_LENGTH = 8000` with a `...[truncated]` marker
85
+ (`Worker/Netsuite/Opportunity.php:148`, `Worker/Clickup/Opportunity.php:45`). Rows of
86
+ 1,500–3,000 KB are what built the 565 GB table.
87
+
88
+ ## Change history
89
+ - 2026-09-29 — Created: `Logs.API` retention cut 90→30 days and chunked DELETE (TRUE-82049); dual 1.0/2.0 routing and 2.0-cluster storage facts recorded. (mhammontree)
@@ -6,8 +6,8 @@ project: Library
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-16
10
- owners: ["dfranks", "jcardinal", "kyalamarthi", "ajean"]
9
+ updated: 2026-09-28
10
+ owners: ["dfranks", "jcardinal", "kyalamarthi", "ajean", "rgirish"]
11
11
  files:
12
12
  - library/app/api/netsuite/rest.php
13
13
  - library/ssl/netsuite_ec_key.pem
@@ -110,7 +110,9 @@ Status codes (`transaction.status` letter):
110
110
 
111
111
  Header columns present (✓ confirmed in SuiteQL): `id`, `tranid`, `trandate`, `lastmodifieddate`, `type`, `status`, `entity`, `employee`, `leadsource`, `terms`, `memo`, `otherrefnum`, `foreigntotal`/`total` (both return the same value), `foreignamountunpaid` (AR balance — invoices only; NULL for SO/credit memo/cash sale), `duedate`, `shipmethod`, `opportunity`, `shippingaddress` (numeric id → join `transactionShippingAddress`), and custom fields (`CUSTBODY_END_CUSTOMER`, `CUSTBODY_STOCKING_ORDER`, …).
112
112
 
113
- Header fields NOT in SuiteQL (✗ — require a per-id REST GET): `shippingCost`, `taxTotal`, `subTotal`, `createdFrom` (also derivable from `previousTransactionLineLink`, but non-deterministically — see shim doc), `trackingNumbers`, `linkedTrackingNumbers`, and the **structured** `shippingAddress` object (SuiteQL gives only the id).
113
+ Header fields NOT in SuiteQL (✗ — require a per-id REST GET): `shippingCost`, `taxTotal`, `subTotal`, `createdFrom` (also derivable from `previousTransactionLineLink`, but non-deterministically — see shim doc), and the **structured** `shippingAddress` object (SuiteQL gives only the id).
114
+
115
+ Tracking numbers in SuiteQL (verified 2026-09-28): `transaction.trackingnumberlist` (on the IF) / `linkedtrackingnumberlist` (on the SO) return **`trackingnumber` record ids**, not the numbers — resolve with `SELECT id, trackingnumber FROM trackingnumber WHERE id IN (...)`. `trackingnumbermap` returned nothing. The REST per-id GET below also has them.
114
116
 
115
117
  ### `transactionline` table
116
118
 
@@ -6,7 +6,7 @@ project: Database Changes
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-23
9
+ updated: 2026-09-28
10
10
  owners: [bala, kyalamarthi, apeterson, ajean, jcardinal, rgirish]
11
11
  files:
12
12
  - dbchanges2/Client_Compass/2026-08-26a - CompassCreativeStudioPersona.sql
@@ -182,6 +182,8 @@ the item-fulfillment bridges, which sit side by side and do **not** agree:
182
182
  | `ItemFulfillmentItems_TrackingNumbers` | yes — `(itemFulfillmentItemId, trackingNumberId)` |
183
183
  | `ItemFulfillmentItemUnits_TrackingNumbers` | yes — `(itemFulfillmentItemUnitId, trackingNumberId)` |
184
184
  | `ItemFulfillmentItemUnits` | **no** — and it already holds **254** duplicate `(itemFulfillmentItemId, unitId)` rows as a result |
185
+ | `Users_Personas` | yes — `(userId, personaId)` (`userId_personaId_UNIQUE`, `Client/2026-09-22-BLANK_CLIENT_DATABASE.sql:11410`) |
186
+ | `Users_Roles` | yes — `(userId, roleId)` |
185
187
 
186
188
  The last row is the whole point: the table with no unique key had already accumulated duplicates
187
189
  that nobody noticed, because nothing ever errored.
@@ -198,6 +200,13 @@ WHERE
198
200
  ur.id IS NULL;
199
201
  ```
200
202
 
203
+ **Not every table has a `dtCreated` to set.** The "always write `dtCreated = NOW()`" rule below
204
+ applies only where the column exists. The user bridges do **not** have one —
205
+ `_Model_Client_Users_Persona` (`_underscore/Model/Client/Users/Persona.php`) declares exactly
206
+ `(id, uuid, userId, personaId)`, and `Users_Roles` is the same shape — so a hand-written INSERT names
207
+ only the uuid and the two ids. Read the model (or the blank-client schema) before adding a timestamp
208
+ column that does not exist.
209
+
201
210
  **A guarded insert that inserts zero rows is a success, not a wasted file** — see
202
211
  [Client-DB grant migrations are re-runnable NO-OPs](../../_underscore/features/acl-permission-chain.md).
203
212
 
@@ -505,6 +514,7 @@ an INSERT that names only `uuid`/`tableViewId`/`recordFieldId`/`isVisible`/`inde
505
514
  [FIELD_SQL calculated fields](../../_underscore/features/calculated-sql-fields.md).
506
515
 
507
516
  ## Change history
517
+ - 2026-09-28 — `Users_Personas` / `Users_Roles` added to the composite-unique table (both unique on the id pair, so a guarded re-run is a clean no-op), plus the counterpoint to the dtCreated rule: these user bridges have no timestamp column at all. (apeterson)
508
518
  - 2026-09-23 — Added the NULL-unsafe flag-guard trap (`col = 0` matches no NULL row; use `COALESCE(col, 0) = 0`), found when a bundles TableViewFields migration updated 0 rows; flagged the 2026-08-26 `EnableTableViewFieldFilters` files as carrying the same hole. (apeterson)
509
519
  - 2026-09-22 — Added multi-tier chain backfill rules from the Compass tracking/serial backfills: re-run the same file until the verify count is 0 (each pass moves data up one tier; both files needed 3 passes); `GROUP BY` the target key pair instead of `SELECT DISTINCT` when many downstream rows roll up to one upstream row (bundles tripped a bridge UNIQUE 6-to-1); measure the safe/ambiguous/no-match split before writing bulk DML and skip the ambiguous rows (4,900 broken links = 1,982 safe + 0 ambiguous + 2,918 no-match). Sharpened the unique-key rule with measured composite keys across the four item-fulfillment tables — `ItemFulfillmentItemUnits` has none and already carries 254 duplicates. (rgirish)
510
520
  - 2026-09-21 — Added three lessons from the platform-wide `Users.contactId` NOT NULL backfill: a guarded
@@ -0,0 +1,73 @@
1
+ ---
2
+ title: Where Secrets Live (AWS Secrets Manager, Build Credentials)
3
+ framework: "2.0"
4
+ project: _Underscore
5
+ client: shared
6
+ type: standard
7
+ status: active
8
+ updated: 2026-09-28
9
+ owners: [jcardinal]
10
+ related:
11
+ - frontend-deploy.md
12
+ - ../apps/_underscore/features/cloud-secrets-manager.md
13
+ - ../apps/api2/architecture.md
14
+ - ../apps/toga25-blox/architecture.md
15
+ ---
16
+
17
+ Every secret a machine (build, CI, app) reads lives in one place, AWS Secrets Manager in the production account; open before adding, reading, or rotating any credential.
18
+
19
+ ## Rule
20
+
21
+ - One place: AWS Secrets Manager, **production account `654654170868`, `us-east-1`**. No secret values in config files, `.env`, source, CI YAML, or committed anywhere.
22
+ - Secrets are **JSON**. Value key is `token` (or named keys when a secret holds several related values).
23
+ - Never write a secret's actual value into a knowledge doc, commit, log, URL, or chat — location and name only.
24
+
25
+ ## Naming
26
+
27
+ - Folder-style prefix by **who reads it**: `build/` = anything a build/CI job may read (Amplify, GitHub Actions).
28
+ - Other existing prefixes: `toga/ssl/*` (SAML SP certs), `talos/<env>/*`, `sftp/<client>/*`.
29
+ - Current `build/` secrets:
30
+ - `build/github-read` — fine-grained GitHub token on the shared deployment user `agilantgit`, resource owner org `agilantsolutions`, Contents read-only on `toga25-blox` / `toga25-desk` / `toga25-iq` / `toga2-template`, 1-year expiry — renew before it expires.
31
+ - `build/fontawesome` — FontAwesome Pro package token.
32
+
33
+ ## Env var name for local dev / CI
34
+
35
+ Upper-case the secret name, `/` → `_`, append `_<KEY>` upper-cased.
36
+
37
+ - `build/fontawesome` + key `token` → `BUILD_FONTAWESOME_TOKEN`. Read by the app's `.npmrc`. Each developer sets it once: `setx BUILD_FONTAWESOME_TOKEN "<token>"`, then open a new terminal.
38
+ - `build/github-read` → `BUILD_GITHUB_READ_TOKEN`.
39
+
40
+ ## Cross-account access (prod-stored, non-prod reader)
41
+
42
+ Secret is encrypted with KMS key `alias/toga-secrets` (key policy lets non-prod account `975050298201` decrypt only via Secrets Manager) plus a resource policy on the secret naming the reader roles. Readers must use the **full ARN**: `arn:aws:secretsmanager:us-east-1:654654170868:secret:<name>`. Same pattern as `toga/ssl/*` (read by non-prod `aws-elasticbeanstalk-ec2-role`).
43
+
44
+ ### Build readers today (non-prod `975050298201`)
45
+
46
+ - `amplify-platform-build` — Amplify service role; inline policy `platform-build-secrets-read` grants `secretsmanager:GetSecretValue` on prod `secret:build/*` + `kms:Decrypt` on `toga-secrets` via Secrets Manager. Attached to Amplify app `toga25-desk` (`d311sdoq12ddne`).
47
+ - `github-actions-toga25-blox` — GitHub OIDC provider (`token.actions.githubusercontent.com`), scoped to repo `agilantsolutions/toga25-blox` only, same policy shape. Used by `.github/workflows/apps-build.yml`.
48
+ - **Open item:** production Amplify builds need their own `amplify-platform-build` role in the **prod** account (same-account read — no cross-account sharing needed there).
49
+
50
+ ## Adding a secret / a reader
51
+
52
+ - **New build secret:** create under `build/`, encrypt with `alias/toga-secrets`, add the same resource policy (the two non-prod roles above). No IAM change needed — both roles already allow `build/*`.
53
+ - **New non-prod reader role:** add its ARN to each shared secret's resource policy.
54
+
55
+ ## Rotation
56
+
57
+ - Rotate at the source (GitHub, FontAwesome, etc.), update the value in Secrets Manager. Builds pick it up next run; developers update their local env var.
58
+ - **Open item:** old FontAwesome + npmjs tokens are committed in `toga25-desk` / `toga25-iq` / `toga2-template` git history — rotate once every app reads `build/fontawesome`. Legacy apps `toga25-supply` / `toga2-commerce` still use the current (unrotated) token, so rotating now would break them first.
59
+ - Any credential that has ever been committed must be treated as compromised and rotated — deleting the line does not remove it from history. See [worker2 committed secrets debt register](../apps/worker2/features/committed-secrets-and-config-debt.md) and [api2 architecture Security note](../apps/api2/architecture.md#security-note-committed-secrets) for the existing backlog this standard is meant to stop growing.
60
+
61
+ ## Never
62
+
63
+ - Commit a token.
64
+ - Put a token in a URL, log, or chat.
65
+ - Give a build role access to all secrets — a build runs hundreds of npm packages; scope to `build/*` or narrower.
66
+
67
+ ## Related
68
+
69
+ - [2.0 Frontend Deployment (Vite + AWS Amplify)](frontend-deploy.md)
70
+ - [_Cloud Secrets Manager helper](../apps/_underscore/features/cloud-secrets-manager.md) — the app-side (PHP) reader; this standard covers the AWS-side layout and build/CI readers
71
+ - [API (api2) Architecture — Security note](../apps/api2/architecture.md#security-note-committed-secrets)
72
+ - [toga25-blox Architecture](../apps/toga25-blox/architecture.md)
73
+ </content>
@@ -4,7 +4,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
4
4
 
5
5
  ## 1.0 framework
6
6
 
7
- - **library** (Library) _(framework core)_ — 28 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
7
+ - **library** (Library) _(framework core)_ — 29 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
8
8
  - **worker** (Worker) — 40 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
9
9
  - **dbchanges** (Database Changes) _(framework core)_ — 1 doc(s) → [1.0/apps/dbchanges/INDEX.md](1.0/apps/dbchanges/INDEX.md)
10
10
  - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
@@ -22,7 +22,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
22
22
  - **_underscore** (_Underscore) _(framework core)_ — 90 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
23
23
  - **worker2** (Worker) — 80 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
24
24
  - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
25
- - **dbchanges2** (Database Changes) _(framework core)_ — 21 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
25
+ - **dbchanges2** (Database Changes) _(framework core)_ — 22 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
26
26
  - **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
27
27
  - **saml** (SAML SSO Gateway) — 9 doc(s) → [2.0/apps/saml/INDEX.md](2.0/apps/saml/INDEX.md)
28
28
  - **apiproxy** (API Proxy) — 1 doc(s) → [2.0/apps/apiproxy/INDEX.md](2.0/apps/apiproxy/INDEX.md)
@@ -2,4 +2,4 @@
2
2
 
3
3
  | Doc | Framework | Summary |
4
4
  |-----|-----------|---------|
5
- | [Adyen](profile.md) | 2.0 | Adyen live TOGa Supply client (`Client_Adyen`) on the 1.0 NetSuite→TOGa Supply sync, status draft; open for the `isFulfillable` prod/beta column gap, the beta=c |
5
+ | [Adyen](profile.md) | 2.0 | Adyen live TOGa Supply client (`Client_Adyen`) on the 1.0 NetSuite→TOGa Supply sync, status draft; open for the prod tracking-API test orders + SR↔NetSuite matc |
@@ -11,21 +11,27 @@ project: Worker
11
11
  client: adyen
12
12
  type: profile
13
13
  status: draft
14
- updated: 2026-09-16
14
+ updated: 2026-09-28
15
15
  owners: ["rgirish"]
16
16
  files:
17
17
  - worker/crons/toga2/netsuite/sync_togasupply_adyen.php
18
18
  - library/app/api/toga2.php
19
19
  - _underscore/Model/Client/Item.php
20
+ - _underscore/Trait/Netsuite/SalesOrder.php
21
+ - _underscore/Trait/Netsuite/ItemFulfillment.php
22
+ - dbchanges2/Client_Adyen/2026-09-22a - AdyenApiTestSeed.sql
23
+ - dbchanges2/Client_Adyen/2026-09-28a - LinkTestOrdersToNetsuite.sql
24
+ - dbchanges2/Client_Adyen/2026-09-28b - RemoveUnmatchedTestOrders.sql
20
25
  related:
21
26
  - ../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md
27
+ - ../../1.0/apps/library/features/netsuite-suiteql-api-reference.md
22
28
  - ../growrk/features/transfer-order-flow.md
23
29
  - ../../2.0/apps/_underscore/features/acl-permission-chain.md
24
30
  - ../../2.0/apps/dbchanges2/workflows/client-schema-drift-audit.md
25
31
  - ../../2.0/apps/dbchanges2/workflows/nonprod-metadata-drift-repair.md
26
32
  ---
27
33
 
28
- Adyen live TOGa Supply client (`Client_Adyen`) on the 1.0 NetSuite→TOGa Supply sync, status draft; open for the `isFulfillable` prod/beta column gap, the beta=client-sandbox env trap, and the ACL findings.
34
+ Adyen live TOGa Supply client (`Client_Adyen`) on the 1.0 NetSuite→TOGa Supply sync, status draft; open for the prod tracking-API test orders + SR↔NetSuite matching, the `isFulfillable` column gap, the beta=client-sandbox env trap, and the ACL findings.
29
35
 
30
36
  ## Summary
31
37
 
@@ -36,6 +42,7 @@ Adyen live TOGa Supply client (`Client_Adyen`) on the 1.0 NetSuite→TOGa Supply
36
42
 
37
43
  - **NetSuite → TOGa Supply sync.** Per-client thin wrapper over the shared engine — full mechanics on [per-client sync](../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md).
38
44
  - **Transfer orders ENABLED** (`IS_ENABLED_INTEGRATION_TRANSFER_ORDERS = true`). Adyen's NetSuite transfer orders arrive **as NetSuite sales orders**, classified by `TRANSFER_ORDER_DETECTION_MODE__ZERO_DOLLAR_HOLD` — `holdInvoice === true` **AND** `total === 0`, same rule as GroWrk (see [GroWrk transfer order flow](../growrk/features/transfer-order-flow.md)). Mode is now declared explicitly in Adyen's launcher; `App_Api_Toga2::isTransferOrder()` throws if it is missing, so **`library` and `worker` must deploy together** for Adyen.
45
+ - **IDs:** `Core.Clients` id **48**; NetSuite customer **31054** ("4950 Adyen").
39
46
  - **`Client_Adyen`** is one of the client DBs with surface-layer / ACL drift noted on the [ACL permission chain](../../2.0/apps/_underscore/features/acl-permission-chain.md) and [surface-layer schema](../../2.0/apps/dbchanges2/features/surface-layer-schema.md) docs.
40
47
 
41
48
  ## Gotchas
@@ -48,6 +55,32 @@ Adyen live TOGa Supply client (`Client_Adyen`) on the 1.0 NetSuite→TOGa Supply
48
55
  - **⚠ Adyen's working beta calls hit `client-sandbox`, NOT `dev-sandbox`.** Calls go to `api.beta.togahub.com` with a JWT carrying `"env":"beta"`, which maps to the MCP environment **`client-sandbox`**. `Core.Domains` in client-sandbox shows `api.beta.togahub.com` with `env = 'beta'` and **`clientId = NULL`** — a shared API domain, so the tenant comes from the **JWT's client uuid**, not the hostname.
49
56
  - Comparing **dev-sandbox against prod** compares two environments the developer is not using — it produced **three consecutive wrong answers** in the 2026-09-08 session. Decode the token's `env` + `id.client.uuid` before any environment diff — recipe in [non-prod metadata drift, step 1](../../2.0/apps/dbchanges2/workflows/nonprod-metadata-drift-repair.md#steps).
50
57
 
58
+ ### Prod tracking-API test orders — linked to real NetSuite (2026-09-28)
59
+
60
+ Adyen tests the tracking API on **prod**. `2026-09-22a - AdyenApiTestSeed.sql` seeded dummy SalesOrders / ItemFulfillments / TrackingNumbers (fake `1ZADYEN…` numbers). Before the NetSuite sync runs they had to become real, or the sync would create **duplicate SOs**.
61
+
62
+ - **`2026-09-28a - LinkTestOrdersToNetsuite.sql` (RAN on prod):** sets `SalesOrders.c_netsuiteInternalSalesOrderId` + `number` = NS `tranid` + `dateOrder`; `ItemFulfillments.c_netsuiteInternalItemFulfillmentId` + `number` + `dateItemFulfillment`; swaps fake tracking for real FedEx numbers, status `PENDING`. Verified: 22 SOs linked, 21 IFs + 42 tracking numbers real.
63
+ - **`2026-09-28b - RemoveUnmatchedTestOrders.sql` (RAN on prod):** deletes the dummy SO/PO/IF chain for SOs with no NS match. ServiceRequests kept.
64
+ - **Why this prevents duplicates:** the sync matches existing rows by `c_netsuiteInternalSalesOrderId` / `c_netsuiteInternalItemFulfillmentId` (`_underscore/Trait/Netsuite/SalesOrder.php`, `ItemFulfillment.php`). It will replace the dummy `SalesOrderItems` (itemId 6).
65
+ - **⚠ Leftovers:** dummy POs `PO-SR%` and vendor `V-TEST-001` still exist for the linked SOs.
66
+
67
+ **⚠ The Adyen NetSuite sync has never run.** `Parameters` cursors stuck at `2026-08-20 00:00:00|0`, the `Customers` row has no `c_netsuiteInternalCustomerId`, no `Locations`. Set these up before expecting sync data.
68
+
69
+ **How to match an Adyen SR to its NetSuite SO — there is NO shared key.**
70
+
71
+ - Adyen's reference number is `ServiceRequests.c_externalReferenceId` (e.g. 48753831); `SalesOrders.serviceRequestId` → `ServiceRequests.id`. NetSuite does not store it.
72
+ - Match SR contact name + ship-to address against NS SO `otherrefnum` (holds the employee name) / shipping attention + addr1.
73
+ - NS Adyen reclaim SOs: memo "Employee Leaving" / "Contractor Leaving", item `SVC-FS-OFFBOARD-ADN`, one IF each with **2 FedEx packages**. First number (87x/875/876) = outbound, 792x = return label — **this outbound/return split is an assumption, not confirmed.**
74
+ - NS tracking-number lookup in SuiteQL: see [SuiteQL reference → `transaction` table](../../1.0/apps/library/features/netsuite-suiteql-api-reference.md).
75
+
76
+ **Known data gaps (2026-09-28):**
77
+
78
+ 1. SR100024 duplicates SR100021 (same Adyen ref 48748808) — a lookup by that ref returns **2 SRs**.
79
+ 2. SR100026 → NS SO 290879, not fulfilled yet.
80
+ 3. SR100027–30 have no NS SO.
81
+ 4. NS SO 288261 has no SR in Toga.
82
+ 5. SR100000/01/02/25 are Adyen test SRs with no NS order.
83
+
51
84
  ### Open defect — `Client_Adyen.Items` missing `isFulfillable` (prod AND beta)
52
85
 
53
86
  - Root cause of the 2026-09-08 "works in sandbox, not in production" report; **NOT an ACL problem**. Adyen is the **only** tenant missing the column.
@@ -91,3 +124,4 @@ The `AclRecordPermissions` prod-vs-dev-sandbox route-name diff from that session
91
124
  - Formal/legal client name and account owner — recorded here simply as **Adyen**.
92
125
  - Which repos beyond the `apps:` list are genuinely in Adyen's scope.
93
126
  - Whether Adyen has client-specific models/features that deserve their own docs.
127
+ - Confirm with Adyen/ops: which FedEx package is outbound vs return; whether SR100024 should be closed; who sets up the NetSuite sync (customer id, Locations, cursors).
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: nycdoe
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-09-25
9
+ updated: 2026-09-29
10
10
  owners: [mhammontree, sking]
11
11
  files:
12
12
  - worker/crons/sync/nycdoe/send_ticket_updates.php
@@ -16,6 +16,7 @@ files:
16
16
  - library/app/model/common/nycdoeticket.php
17
17
  - dbchanges/Common/MH/2026-09-16.sql
18
18
  - library/app/api/nycdoev2.php
19
+ - worker/crons/sync/nycdoe/import_inc.php
19
20
  - togadesk/desk/includes/classes/class.repair.php
20
21
  related:
21
22
  - servicenow-integration.md
@@ -101,6 +102,60 @@ Current selection query:
101
102
  - The standalone assignment-group block was **deleted** — the main payload already diffs `assignment_group`, so it was a duplicate GET, a second code path, and the source of the lockout stamp.
102
103
  - Debug pin already in the file: the commented `#AND NYCDOETickets.ticketNumber = '...'` line scopes a pass to one ticket.
103
104
 
105
+ ### Assignment-group scoping — what counts as OUR ticket (rule, Mark 2026-09-28)
106
+ **We only care about an INC or RITM while it is assigned to one of OUR ServiceNow assignment
107
+ groups. If it is not, drop it and never re-check it. NEVER add another vendor's group to
108
+ `Common.NYCDOETicketsDepartments`** to "make it sync" — a group that is not ours means DOE
109
+ reassigned the incident away from ASI.
110
+
111
+ **⚠ `Common.NYCDOETickets.isValidDepartmentName` means the OPPOSITE of its name.** `1` =
112
+ **EXCLUDED from processing**; `0`/`NULL` = processed. Every write of `1` is on a skip path
113
+ (invalid site code, unmapped assignment group, or SD QA — deliberately skipped at
114
+ `process_tickets.php:392`), and all four DOE selections pick up only `= 0 OR IS NULL`. The
115
+ companion column `departmentName` carries the human reason (`INVALID_SITE_CODE`,
116
+ `Invalid Address Resolution`, or the actual ServiceNow group name). Reading the column name
117
+ literally gives you the exact opposite of the truth. Writers: `process_tickets.php` (348, 392,
118
+ 1520), `import_asn.php:104`, `import_inc.php:85`, `send_ticket_updates.php:235`,
119
+ `send_request_item_updates.php:56`. A pure rename across 13 sites would be safe, but must not be
120
+ bundled with a behavior change.
121
+
122
+ **The drop is self-healing — which is why no new column is needed.** `import_inc.php:82-86` loads
123
+ the existing row by `referenceSysId` and unconditionally resets
124
+ `dtProcessed = null; isValidDepartmentName = null; departmentName = null;` on every re-import
125
+ (same in `import_asn.php:148`). While a ticket sits in a non-ASI group it never matches the
126
+ importer filter, so it stays dropped at zero cost; if DOE reassigns it back to ASI its
127
+ `sys_updated_on` moves, it re-imports, and the flag clears automatically. A purpose-built new
128
+ column would NOT be reset by either import path and would make the drop irreversible — reusing
129
+ `isValidDepartmentName` is the correct choice, not a compromise.
130
+
131
+ **⚠ OPEN DEFECT (backfilled by hand, CODE FIX NOT DONE — needs its own ticket).**
132
+ `send_ticket_updates.php` (~L375) treats an unmapped group as "someone forgot a mapping" and
133
+ emails devteam@togatech.com hourly saying *"Add the group to that table to let them sync."* That
134
+ advice is wrong. Measured cost: 15 tickets across 9 other-vendor groups (NOC Field, ATLAS Tier III
135
+ Support, SD Nagarro Queens, OASIS L3 Support, NGSIS/GAM Support, Central User Support, Field Ops
136
+ Queens, Field Ops Bronx, IET Support) re-GET every 6 minutes = **~3,600 wasted ServiceNow calls/day
137
+ and ~236 emails since 2026-09-18**. Those 15 INCs were set by hand in production to
138
+ `isValidDepartmentName = 1` with the real group in `departmentName`, and confirmed excluded.
139
+ **Agreed approach for the fix ticket (CTO-reviewed) — an ordered predicate:**
140
+ 1. Group **is** in `NYCDOETicketsDepartments` → process normally (this is what keeps SD QA working).
141
+ 2. Group is not in the table but **starts with `ASI`** → keep the throttled email; this is the only
142
+ case the current advice is right for.
143
+ 3. Neither → **drop silently.**
144
+ Do **not** close the repair orders — write one `repair_order_notes` line and send a one-time digest
145
+ to DOE ops. `send_request_item_updates.php` needs the same treatment for RITMs.
146
+
147
+ **⚠ Three competing definitions of "our DOE assignment groups" — they disagree.**
148
+ - **(a) `Common.NYCDOETicketsDepartments`** — 14 rows: ASI Bronx, ASI Brooklyn, ASI Manhattan,
149
+ ASI Queens, ASI Staten Island, ASI General, ASI Calls, ASI Reconnect, ASI Research, ASI DOA,
150
+ ASI TM, SD QA (`isImportEnabled=0`), ASI Disposal, ASI PCS Delivery. **This one is correct.**
151
+ - **(b) `download_tickets_incidents.php:80`** — a hardcoded 12-group ServiceNow query list that
152
+ includes "ASI Enhanced", which exists nowhere else. The file is referenced by **no**
153
+ `schedules/*.json` → **dead code; delete it in its own commit.**
154
+ - **(c) `App_Api_NYCDOEV2::listIncItems()` (`library/app/api/nycdoev2.php:912`)** — the **live**
155
+ importer filter, a prefix match `assignment_groupSTARTSWITHASI^stateIN1,2,3,10`. This is what
156
+ `import_inc.php` actually uses.
157
+ **Risk:** a future group named "ASI Something" imports under (c) then is dropped under (a).
158
+
104
159
  ### Batch loop, run caps & per-ticket isolation (TRUE-82129 current shape)
105
160
  Before this the cron ran the selection query **once per ticket under `LIMIT 1`** with a `sleep(1)` after every ticket — one ticket per 6-minute run, ~10 days for a full pass over the queue.
106
161
  - **One `LIMIT 250` (`BATCH_SIZE`) read into a PHP array, consumed with `array_shift`, refilled only when empty.** The query is `type=ALL`, ~35,021 rows, `Using temporary; Using filesort` (it drives off `managed_service_orders`, so `Common.NYCDOETickets.idx_dtLastChecked` is unusable at this join order). A filesort already sorts every surviving row before returning the top N, **so a bigger LIMIT costs nothing extra**.
@@ -138,6 +193,19 @@ There is **no sandbox for the SNOW integration**; verification happens in prod.
138
193
  2. INC2087375's `u_eta` moving from 2026-09-07 to 2026-09-17.
139
194
  3. `SELECT COUNT(*) FROM Common.NYCDOETickets WHERE dtLastChecked IS NOT NULL` climbing.
140
195
 
196
+ **Answering "did the alert fire, and did the mail actually send?" — two tables, not an inbox.**
197
+ An inbox check alone is misleading; the DB is decisive.
198
+ - **Did the alert fire?** marker rows in `Core.AdvanceShippingNoticeQueue`, `dedupeKey`
199
+ `system|ticket_sync_patch_failure_alert` and `system|ticket_sync_invalid_group_alert`.
200
+ - **Did the mail send?** `Logs.Emails` — columns `dtSent`, `isSuccess`, `error`, `subject`, `to`,
201
+ `sourceJob`. ⚠ the table is **`Emails`**, not `Email`, and the timestamp is **`dtSent`**, not
202
+ `dtStamp`.
203
+
204
+ **TRUE-82027 verified healthy in production 2026-09-28:** `dtLastChecked` + `idx_dtLastChecked`
205
+ present; ~2,784 active tickets cycling in under an hour at `RECHECK_INTERVAL_MINUTES=6`; PATCH
206
+ volume ~44,000/day at 443,706 x HTTP 200 against only 6 non-200 over 10 days (3x 502, 1x 429);
207
+ reproduction case INC2087375 back in rotation and syncing.
208
+
141
209
  ### RITM cron — deliberately unchanged by TRUE-81044
142
210
  `send_request_item_updates.php` already pushes native `state:"8"` (On Hold) and **never wrote** `u_status_task`, so TRUE-81044's ask was already satisfied there. It carries a defensive `HOLD_*` guard on the scheduled-update path so it can't push `state:"-14"` while held; its reverse-hold logic still **reads** `u_status_task == 'On Hold'`.
143
211
  > **OPEN QUESTION:** ServiceNow's communication was scoped to **Incident** state. We did **not** extend `hold_reason` / `u_eta` / `u_other_reason_on_hold` to the RITM payload without confirmation. Whether the RITM On Hold transition carries the same mandatory-set requirement is unconfirmed — ask before changing the RITM payload.
@@ -175,7 +243,7 @@ The SNOW round-trip note reconciliation (delete-and-reinsert in `process_tickets
175
243
  - **Query shape that works (TRUE-82332):** `endpoint LIKE '%nycd3%...'` and broad `requestPayload LIKE` scans time out (>300s) on the MCP. Use exact matches: PATCH `endpoint = 'https://supportadmin.schools.nyc/api/nycd3/data/tasks/INCxxxx'`; GET `endpoint = 'https://supportadmin.schools.nyc/api/nycd3/v2/data/tasks?domain=diit%2Calm&filter=sys_class_name%3Dincident%5Enumber%3DINCxxxx'`. For sweeps: `FORCE INDEX (dtStamp)`, windows ≤2 days, plus `sourceJob` / `method` filters. The state:3↔state:2 oscillation in TRUE-79922 was proven from this table (INC2074899, INC2185183 showed `state:3` then `state:2` pushed seconds apart in one run).
176
244
  - **`NYCDOETickets` (`App_Model_Common_NYCDOETicket`, `db_common` / legacy `Common`)** stores the inbound SNOW record in `rawData`, a **`FIELDTYPE_BLOBSTORAGE`** field (S3 bucket `asifiles`, key `prod/Common/NYCDOETickets/rawData/{id}`) — **NOT a queryable DB column.**
177
245
  - **⚠ `repair_orders.dtUpdated` is skewed — never build a selection query on it.** It is written exactly **240 minutes behind** the clock used by `repair_order_notes.notesdate` and `NYCDOETickets.dtSynced`. **Do NOT compensate with `+240 MINUTE`** — 240 is almost certainly UTC-vs-Eastern and becomes 300 at the DST change (2026-11-01), so a hardcoded offset breaks twice a year. TRUE-82027 sidestepped it by removing `dtUpdated` from the query. Mechanism + the second (720-minute) skew: [Field-Service Dispatch](../../../1.0/apps/togadesk/features/field-service-dispatch.md).
178
- - **⚠ `Logs.API` history is capped at 30 days from 2026-09-17** (Jeff, sprint review) and existing logs are being purged. This investigation depended on history far older than that (9,914 PATCHes back to 2026-08-20; a hold last pushed 22 days earlier; the 2,392-call storm). **Slow-burn bugs like this can no longer be traced backwards — they must be caught by alerting.** NYCDOE stays 1.0 so its logs stay in legacy `Logs.API`; Compass/Elite/WJE move their API logging to 2.0. Agreed: hold the purge until TRUE-82027 is verified in production.
246
+ - **⚠ `Logs.API` history is capped at 30 days — SHIPPED 2026-09-28 (TRUE-82049).** This investigation depended on history far older than that (9,914 PATCHes back to 2026-08-20; a hold last pushed 22 days earlier; the 2,392-call storm). **Slow-burn bugs like this can no longer be traced backwards — they must be caught by alerting.** NYCDOE stays 1.0 so its logs stay in legacy `Logs.API`. Mechanics, storage facts and the not-yet-run backlog purge: [API Request Logging & Retention](../../../1.0/apps/library/features/api-request-logging-and-retention.md).
179
247
  - **Worker prod code cannot run on a Windows dev box.** The DB/S3 data layer fetches EC2 instance-metadata (IMDSv2) credentials and fails off-EC2. Investigate with the toga DB MCP and the `Logs.API` table instead of running the crons locally.
180
248
 
181
249
  ## Gotchas
@@ -202,6 +270,7 @@ The SNOW round-trip note reconciliation (delete-and-reinsert in `process_tickets
202
270
  - `qqStatus()` collapses all recognized hold sub-statuses to base `HOLD` in `repair_orders.status`; the specific hold **sub-reason persists only in `repair_order_notes.newstatus`**. Any status badge / report / filter reading `repair_orders.status` directly cannot distinguish the six hold variants after a recompute.
203
271
 
204
272
  ## Change history
273
+ - 2026-09-29 — Assignment-group scoping rule + inverted `isValidDepartmentName` + the three competing group definitions; TRUE-82027 verified in prod (marker rows + `Logs.Emails` method). (mhammontree)
205
274
  - 2026-09-25 — TRUE-82332: hold-release no longer overwrites the close `state`; close re-sent when SN not closed; stuck-ticket re-queue recipe; `Logs.API` query shapes. (mhammontree)
206
275
  - 2026-09-18 — TRUE-82129: fixed the by-reference `fetchOne()` notice that killed every run; batch loop + run caps replaced LIMIT 1 + per-ticket sleep; alert emails de-PII'd and escaped; unmapped-group email now grouped per run. (mhammontree)
207
276
  - 2026-09-17 — TRUE-82027: outbound push runs once per order per pass; `dtLastChecked` split from `dtSynced`; failure alerting instead of a throw. (mhammontree)
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: nycdoe
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-09-25
9
+ updated: 2026-09-29
10
10
  owners: [mhammontree, sking]
11
11
  files:
12
12
  - worker/crons/sync/nycdoe/import_asn.php
@@ -62,7 +62,7 @@ The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents +
62
62
  | Table | Purpose |
63
63
  |---|---|
64
64
  | `NYCDOETickets` | Local ticket mirror (keyed `referenceSysId`; `type` = INCIDENT / RITM) |
65
- | `NYCDOETicketsDepartments` | Allowlist of valid assignment groups — others are not processed |
65
+ | `NYCDOETicketsDepartments` | Allowlist of OUR assignment groups — others are dropped and never re-checked. ⚠ Two other places define this list differently, and `NYCDOETickets.isValidDepartmentName` is **inverted** (`1` = excluded): see [Hold-Status Sync](hold-status-sync.md) → *Assignment-group scoping* |
66
66
  | `NYCDOELocations` | Site-id → address mirror (backs ASN ship-to resolution) |
67
67
  | `AdvanceShippingNoticeQueue` | Stage-1 landing zone; **UNIQUE index `idx_dedupeKey`** |
68
68
  | `AdvanceShippingNotices` / `…Items` / `…Units` | ASN header (vendor+PO) → per-part items (`qtyOrder`) → individual units |
@@ -130,7 +130,7 @@ Vendor SFTP ───(legacy_import_asn.php, ser+non-ser)─┘ [UNIQUE ded
130
130
 
131
131
  ### Debugging & DB topology (DOE 1.0 sync)
132
132
  The 1.0 worker cannot run on a dev/Windows box; use the toga DB MCP + `Logs.API` instead of running prod code locally.
133
- - **⚠ `Logs.API` retention is 30 days from 2026-09-17** (Jeff, sprint review) and existing logs are being purged — so it is no longer a *historical* debugging avenue, only a recent-window one. Bugs that burn slowly (a hold not re-pushed for 22 days; 9,914 PATCHes traced back to 2026-08-20) can no longer be reconstructed after the fact and must be caught by **alerting** instead. NYCDOE stays 1.0, so its logs remain in legacy `Logs.API`; Compass/Elite/WJE move their API logging to 2.0.
133
+ - **⚠ `Logs.API` retention is 30 days — SHIPPED 2026-09-28 (TRUE-82049)**; mechanics, storage facts and the not-yet-run backlog purge in [API Request Logging & Retention](../../../1.0/apps/library/features/api-request-logging-and-retention.md) — so it is no longer a *historical* debugging avenue, only a recent-window one. Bugs that burn slowly (a hold not re-pushed for 22 days; 9,914 PATCHes traced back to 2026-08-20) can no longer be reconstructed after the fact and must be caught by **alerting** instead. NYCDOE stays 1.0, so its logs remain in legacy `Logs.API`; Compass/Elite/WJE move their API logging to 2.0.
134
134
  - **`repair_orders.dtUpdated` and `.lastupdateddate` are both time-skewed in `TOGaDeskSupport`** — never select or diff on them, and never compensate with a fixed offset. Detail: [Field-Service Dispatch](../../../1.0/apps/togadesk/features/field-service-dispatch.md).
135
135
  - **`dbchanges` does not describe the real 1.0 schema** — some live columns were created outside the repo. Verify indexes/columns against the database, not the repo: [Authoring & Shipping a 1.0 dbchanges SQL File](../../../1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md).
136
136
  - **`db_core` alias → legacy (V1) environment, schema `Core`** — holds the FLAT tables `AdvanceShippingNotices` / `AdvanceShippingNoticeItems` / `AdvanceShippingNoticeUnits` (columns incl. `togadeskRepairOrderId`, `serialNumber`, `netSuiteInternalSalesOrderId`, `netSuiteInternalPurchaseOrderId`, `customerPurchaseOrder`). The 1.0 cron reads **legacy/Core**.
@@ -262,5 +262,6 @@ All **live** DOE business logic is 1.0. The only other 2.0 code is the in-progre
262
262
  - Local deep-dive (full WR260236464 case study with record IDs): `doe_service_now_integration.md` in Mark's WWW workspace root; also the `service-now-integration` skill (`asn-import-flow.md`)
263
263
 
264
264
  ## Change history
265
+ - 2026-09-29 — Assignment-group scoping corrected (inverted `isValidDepartmentName`, three competing group lists) with detail in hold-status-sync; `Logs.API` 30-day retention shipped. (mhammontree)
265
266
  - 2026-09-25 — TRUE-82332: cross-link to the close-clobber fix; test-repo copy of the ServiceNow credential noted (mhammontree)
266
267
  - 2026-09-25 — Corrected "no 2.0 tenant" (TRUE-80519 2.0 ASN rewrite now exists); DOE-Open-Lenovo marked retired (mhammontree)
@@ -7,4 +7,5 @@
7
7
  | [Quad: PO + ASN email importer (import_po_and_asn.php)](features/po-asn-email-import.md) | 1.0 | Quad's inbox-driven PO/ASN CSV importer and its 2026-08 failure classes; open when a Quad PO/ASN email jams the mailbox, fails EV-12, or you touch `import_po_an |
8
8
  | [Quad Graphics](profile.md) | 2.0 | Quad Graphics is a TOGA 2.0 client on the `_underscore` platform, prod schema `Client_Quad`. |
9
9
  | [Quad Graphics — reversing the 30-item catalog fold (storefront-assortment recovery)](workflows/catalog-fold-reversal.md) | 2.0 | How the 2026-08-31 catalog-blind item merge damaged Quad's storefront and how the reversal was built; open when assessing/repairing catalog-fold damage on a two |
10
+ | [Giving a Quad user every persona plus the `Global Admin` role](workflows/granting-all-personas-and-global-admin.md) | 2.0 | The two-table grant for "give internal user X everything on Quad" — every `Personas` row + `Global Admin` — and why it does **not** produce a combined catalogue |
10
11
  | [Giving Quad's internal TOGA staff extra toga25-supply nav (the `Agilant - Administrators` role pattern)](workflows/granting-internal-admin-navigation.md) | 2.0 | How to surface extra 2.5 Supply pages to internal TOGA staff on a tenant without showing them to the client's own users — Quad's first internal-admin role; open |
@@ -23,6 +23,7 @@ related:
23
23
  - ../../2.0/apps/toga2-commerce/features/storefront-persona-gating-and-home-counts.md
24
24
  - ./workflows/catalog-fold-reversal.md
25
25
  - ./workflows/granting-internal-admin-navigation.md
26
+ - ./workflows/granting-all-personas-and-global-admin.md
26
27
  - ./features/po-asn-email-import.md
27
28
  - ./features/enter-po-details-vendor-selection-ev12.md
28
29
  - ../../2.0/apps/_underscore/features/sales-order-po-number-sourcing.md
@@ -63,7 +64,10 @@ Client-specific DB change-sets live in `dbchanges2/Client_Quad/`.
63
64
  - **Quad is the ONLY storefront tenant with the persona switcher**
64
65
  (`determineShouldShowPersonaSwitcher` returns true for QUAD alone). With it on, `getLoginSettings`
65
66
  sets `_personaIds` to a **single** persona (the saved default, else the first) — so a Quad user
66
- never sees a combined view of their personas. `Client_Quad` also has **no persona 24 at all** (not
67
+ never sees a combined view of their personas. Granting a user **all** personas therefore only makes
68
+ them **selectable one at a time** in the switcher; it does not widen what they see at once
69
+ ([all personas + Global Admin](./workflows/granting-all-personas-and-global-admin.md)).
70
+ `Client_Quad` also has **no persona 24 at all** (not
67
71
  on a user, not on a location, the persona does not exist), so the Compass "persona 24" storefront
68
72
  hack never touched Quad. See
69
73
  [Storefront persona gating](../../2.0/apps/toga2-commerce/features/storefront-persona-gating-and-home-counts.md).
@@ -0,0 +1,86 @@
1
+ ---
2
+ title: Giving a Quad user every persona plus the `Global Admin` role
3
+ framework: "2.0"
4
+ repo: dbchanges2
5
+ project: Database Changes
6
+ client: quad
7
+ type: workflow
8
+ status: active
9
+ updated: 2026-09-28
10
+ owners: ["apeterson"]
11
+ files:
12
+ - dbchanges2/Client_Quad/2026-09-28b - GrantAllPersonasToNSchenkel.sql
13
+ related:
14
+ - ../profile.md
15
+ - ./granting-internal-admin-navigation.md
16
+ - ../../compass-usa/workflows/granting-persona-bundle-access.md
17
+ - ../../../2.0/apps/_underscore/features/effective-persona-resolution.md
18
+ - ../../../2.0/apps/toga2-commerce/features/storefront-persona-gating-and-home-counts.md
19
+ - ../../../2.0/apps/dbchanges2/features/rerunnable-additive-inserts.md
20
+ ---
21
+
22
+ The two-table grant for "give internal user X everything on Quad" — every `Personas` row + `Global Admin` — and why it does **not** produce a combined catalogue view on this tenant.
23
+
24
+ ## Summary
25
+
26
+ A recurring request shape: *give internal user X everything on tenant Y*. On Quad it is **two
27
+ guarded, re-runnable INSERTs**, both entirely inside `Client_Quad` (no `Core` join — separate prod
28
+ clusters):
29
+
30
+ 1. **`Users_Personas`** — one row per `Client_Quad.Personas` row the user does not already hold
31
+ (`Users` resolved by `email`, `CROSS JOIN Personas`), uuid from the house inline `RANDOM_BYTES()`
32
+ v4 expression.
33
+ 2. **`Users_Roles`** — the tenant's `Global Admin` role.
34
+
35
+ Both are additive: the user keeps every persona and role they already hold (roles resolve as a
36
+ union — [ACL permission chain](../../../2.0/apps/_underscore/features/acl-permission-chain.md)).
37
+ Nothing is removed, so this is not a reversible-by-re-run change; removing access is a separate,
38
+ destructive job.
39
+
40
+ ## Steps
41
+
42
+ 1. **Guard `Users_Personas` on `(userId, personaId)`** — that *is* the real unique key
43
+ (`userId_personaId_UNIQUE`, verified against
44
+ `dbchanges2/Client/2026-09-22-BLANK_CLIENT_DATABASE.sql:11410`), so a guarded re-run is a true
45
+ no-op rather than a duplicate-key abort.
46
+ 2. **Resolve the role BY NAME, not by id** — `INNER JOIN Roles r ON r.name = 'Global Admin'`. On
47
+ Quad that is id **7** today, but ids drift per environment; the name keeps one file correct
48
+ everywhere.
49
+ 3. **Name only `uuid`, `userId`, `personaId` / `roleId`.** `Users_Personas` and `Users_Roles` carry
50
+ **no `dtCreated`** — see the gotcha below.
51
+ 4. **End with two verify SELECTs** — the user's persona list ordered by
52
+ `CAST(number AS UNSIGNED)` (`Personas.number` is a string column) and their role list.
53
+ 5. **Hand the file to the developer to run.** Verify in the app by logging out and back in.
54
+
55
+ ### The 2026-09-28 Quad migration
56
+ `dbchanges2/Client_Quad/2026-09-28b - GrantAllPersonasToNSchenkel.sql` — branch `TRUE-82247`,
57
+ **not committed and not yet run in any environment.** Grants `nschenkel@togatech.com` every
58
+ `Client_Quad` persona plus `Global Admin`.
59
+
60
+ Context: NSchenkel is one of the three `@togatech.com` Quad users deliberately left **off** the new
61
+ `Agilant - Administrators` role created the same day
62
+ ([internal-admin navigation](./granting-internal-admin-navigation.md) — that role went to
63
+ `devteam@togatech.com` only). This grant gives a different, wider reach: the tenant's own
64
+ `Global Admin`, not the nav-only internal role.
65
+
66
+ ## Gotchas
67
+
68
+ - **⚠ On Quad, "all personas" does NOT mean "sees everything at once".** Quad is the **only**
69
+ toga2-commerce tenant with the **persona switcher**
70
+ (`determineShouldShowPersonaSwitcher` returns true for `QUAD` alone), and with it on
71
+ `getLoginSettings` sets `_personaIds` to a **single** persona (the saved default, else the first).
72
+ So this grant makes every persona **selectable, one at a time** — it does not build the combined
73
+ catalogue view the same grant would give on Compass / Compass Canada. Expect the reporter to say
74
+ "the storefront did not change". See
75
+ [Storefront persona gating](../../../2.0/apps/toga2-commerce/features/storefront-persona-gating-and-home-counts.md).
76
+ - **This touches only source 1 of the three effective-persona sources.** `Locations_Personas` on the
77
+ user's own location and on the 5-hop sector are untouched — so a "who has persona N" check still
78
+ has to resolve all three. See
79
+ [Effective persona resolution](../../../2.0/apps/_underscore/features/effective-persona-resolution.md).
80
+ - **`Global Admin` is not an internal-only role on Quad.** Role 7 is held by 7 users, five of them
81
+ `@quad.com` — fine for granting *to* one internal user, never a place to hang an internal-only
82
+ override.
83
+
84
+ ## Change history
85
+ - 2026-09-28 — Created; documents the `Users_Personas` + `Users_Roles` all-personas grant written for
86
+ NSchenkel (not yet run) and the Quad persona-switcher caveat (apeterson)
@@ -16,6 +16,7 @@ related:
16
16
  - ../../../2.0/apps/toga25-supply/features/side-navigation-and-default-route.md
17
17
  - ../../../2.0/apps/_underscore/features/acl-permission-chain.md
18
18
  - ../../../2.0/apps/dbchanges2/features/rerunnable-additive-inserts.md
19
+ - ./granting-all-personas-and-global-admin.md
19
20
  ---
20
21
 
21
22
  How to surface extra 2.5 Supply pages to internal TOGA staff on a tenant without showing them to the client's own users — Quad's first internal-admin role; open for "internal user can't see page X" on any tenant.
@@ -45,6 +46,8 @@ staff. That is a tenant-wide behavior change dressed up as a per-user fix.
45
46
  3. **Create a dedicated `Agilant - Administrators` role** in the client DB if the tenant has none,
46
47
  and assign only the internal users the developer names. Quad got **`devteam@togatech.com` only**;
47
48
  the other three `@togatech.com` Quad users (JWise, NSchenkel, rgreen) were deliberately left off.
49
+ NSchenkel was instead granted the tenant's own `Global Admin` role (a different, wider reach) —
50
+ [all personas + Global Admin](./granting-all-personas-and-global-admin.md).
48
51
  4. **Add role-scoped `SurfaceOverrides` rows** (`surfaceId` 39, the element ids, `IS_VISIBLE` = `'1'`,
49
52
  `personaId`/`languageId` NULL) for that new role. Nothing else — no `AclRecordPermissions`, no
50
53
  `AclFieldPermissions`, so the migration cannot widen data access.
@@ -77,5 +80,7 @@ Quad's starting state for contrast: **one** nav override total (element 109 `sal
77
80
  migration there.
78
81
 
79
82
  ## Change history
83
+ - 2026-09-28 — NSchenkel (left off this role) granted `Global Admin` + all personas instead, in a
84
+ separate file; linked. (apeterson)
80
85
  - 2026-09-28 — Created; Quad's first `Agilant - Administrators` role + Items/Vendor Items nav
81
86
  overrides written (not yet run), scoped away from `Global Admin` role 7 (apeterson)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.910",
3
+ "version": "1.0.912",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",