toga-ai 1.0.1095 → 1.0.1097

package/knowledge/1.0/apps/worker/workflows/netsuite-duplicate-items-dedup.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: workflow
 status: active
-updated: 2026-09-16
-owners: [jcardinal]
+updated: 2026-10-21
+owners: [jcardinal, rgirish]
 files:
   - library/app/api/toga2.php
   - worker/crons/toga2/netsuite/common_sync_togasupply.php
@@ -16,6 +16,7 @@ related:
   - ../../library/features/netsuite-suiteql-rest-shim.md
   - ../../../clients/compass-usa/workflows/catalog-fold-reversal.md
   - ../../../clients/compass-usa/features/item-catalogs-and-duplicate-items.md
+  - ../../../2.0/apps/dbchanges2/features/netsuite-id-unique-keys.md
 ---
 
 Per-client cleanup collapsing duplicate `Items` rows (same `partNumber`) to one canonical via recursive MERGE; open before running it — and read the catalog-blind warning first, it broke Compass + Quad.
@@ -51,14 +52,20 @@ One-time, per-client data cleanup that collapses **duplicate `Items` rows sharin
 
 - **The sync item lookup is Agilant-catalog-only.** Any non-Agilant item is **invisible** to `common_sync_togasupply.php` and will be **re-created** — the whole mechanism behind the duplicates. Deploy the catalog-stamp fix before (and independently of) this cleanup.
 - **`Items` has NO unique constraint on `partNumber`.** Duplicates persist silently; nothing at the DB level prevents them, which is why this is a MERGE-and-clean rather than a rely-on-the-constraint fix.
+- **New duplicates after this cleanup are keyed by NetSuite item id, not part number (2026-10).** The item sync looks up by part number, so a renamed or `_`/`-`-suffixed part number gets a new item for the same `c_netsuiteInternalItemId`. The TRUE-82291 dedupe merges by NetSuite id and a UNIQUE key on it waits for the sync fix — see [NetSuite id UNIQUE keys](../../../2.0/apps/dbchanges2/features/netsuite-id-unique-keys.md).
 - **Client DBs are schema-similar but NOT identical.** Some lack optional tables (e.g. `EntitlementServiceItems`). Run the merge **continue-on-error** so a missing-table MySQL **1146** is a **benign skip**, not a failed migration.
 - **A plain `UPDATE IGNORE` + delete-leftover is insufficient.** Dupe-vs-dupe unique-key collisions leave orphaned child FKs; only the survivor-per-key re-pointing (step 5) is correct.
 - **⚠ The merge is CATALOG-BLIND — it does NOT handle multi-catalog clients.** It groups on `UPPER(partNumber)` only, with an Agilant-preferred canonical rule. For a client that keeps the same part number in two intentional catalogs (Compass: store vs Agilant), it folds legitimate per-catalog twins together and **destroys** the store items. Confirm a client is single-catalog before running; otherwise make the dedup catalog-scoped. The Compass fold deleted 199 store items and broke MITS — see the [catalog-fold reversal](../../../clients/compass-usa/workflows/catalog-fold-reversal.md).
 - **`UPDATE IGNORE ...; DELETE ... WHERE fk=dupe` permanently DELETES collision rows** and drops its helper maps, so prod cannot self-reverse. A reversal needs a **pre-merge snapshot** restored as a scratch DB — plan the backup before running any variant of this.
 - **Blast radius of the 2026-08-31 `duplicate_items_merge_ALL_CLIENTS.sql` (assessment complete 2026-09-01):** it ran on 17 clients (Masonite, Ncci, Erau, Shrss, Wmchealth, Trividiahealth, Prudential, Miamidade, Browardsheriff, Nychh, Canon, Quad, Compass, Elite, Growrk, Spglobal, Endeavorhealth). **Only 2 were damaged — Compass and Quad, both two-catalog TOGa Commerce clients.** The other 15 needed no reversal: 13 single-catalog "Agilant"-only clients were a legitimate within-catalog dedup, NYCHH mixed Etilize + NetSuite in one catalog (no fold possible), and Prudential was cleared (single populated catalog; NULL `assetType` is its normal state). Triage per client with `Catalogs` LEFT JOIN `Items` (empty catalogs still show) + the pipeline signal (`c_etilizeProductId` = store vs `c_netsuiteInternalItemId` = Agilant); a client is damaged only if it has a two-catalog store/procurement design AND catalog-1 was folded into catalog-2 AND storefront/transmit depends on catalog-1. See the Compass [catalog-fold reversal](../../../clients/compass-usa/workflows/catalog-fold-reversal.md) and [Quad catalog-fold reversal](../../../clients/quad/workflows/catalog-fold-reversal.md).
 
+## Change history
+- 2026-10-21 — Linked the TRUE-82291 NetSuite-id dedupe + UNIQUE keys; new duplicates come from the part-number lookup. (rgirish)
+- 2026-09-16 — Catalog-blind correction and blast-radius assessment (Compass + Quad damaged). (jcardinal)
+
 ## Related
 
+- [NetSuite id UNIQUE keys](../../../2.0/apps/dbchanges2/features/netsuite-id-unique-keys.md)
 - [NetSuite → TOGa Supply Per-Client Sync](../features/netsuite-togasupply-per-client-sync.md)
 - [NetSuite SuiteQL REST shim](../../library/features/netsuite-suiteql-rest-shim.md)
 - [Compass catalog-fold reversal](../../../clients/compass-usa/workflows/catalog-fold-reversal.md)

package/knowledge/2.0/apps/dbchanges2/INDEX.md

@@ -3,6 +3,7 @@
 | Doc | Summary |
 |-----|---------|
 | [Database Changes (dbchanges2) Repository Architecture](architecture.md) | The 2.0 SQL change-set repo: folder→database layout, the filename execution contract, and the migration-authoring rules; open for any dbchanges2 schema/data cha |
+| [NetSuite id UNIQUE keys on c_netsuite* columns (rollout, per-client dedupe, never-unique columns)](features/netsuite-id-unique-keys.md) | Which `c_netsuite*` id columns in NetSuite client DBs are UNIQUE, which never can be, the 3-file key rollout + per-client dedupe pattern (TRUE-82291), and where |
 | [Re-runnable additive migrations (uuid4 in SQL, INSERT guards, the DISTINCT trap, conditional ALTERs)](features/rerunnable-additive-inserts.md) | Most `dbchanges2` files are **additive data grants** run by hand against production, often more than once (once per environment, or twice because someone was no |
 | [Surface Layer Schema (UI presentation/config tables)](features/surface-layer-schema.md) | Persistent schema + seed rules for the platform Surface UI-config layer (Surfaces/SurfaceElements/Overrides/ThemeTokens etc.); open for any DB-driven UI config |
 | [2.0 New-Client Onboarding (manual process)](workflows/client-onboarding.md) | How to stand up a new 2.0 client (tenant) by hand; open when onboarding a tenant or checking what a step means. |

package/knowledge/2.0/apps/dbchanges2/features/netsuite-id-unique-keys.md

@@ -0,0 +1,81 @@
+---
+title: NetSuite id UNIQUE keys on c_netsuite* columns (rollout, per-client dedupe, never-unique columns)
+framework: "2.0"
+repo: dbchanges2
+project: Database Changes
+client: shared
+type: feature
+status: active
+updated: 2026-10-21
+owners: [rgirish]
+files:
+  - dbchanges2/_modules/netsuite/2026-09-24b - SalesOrderNetsuiteIdUnique.sql
+  - dbchanges2/_modules/netsuite/2026-10-20a - NetsuiteIdUniqueKeys.sql
+  - dbchanges2/_modules/netsuite/2026-10-20b - NetsuiteIdUniqueKeysAfterDedupe.sql
+  - dbchanges2/_modules/netsuite/2026-10-20c - NetsuiteIdUniqueKeysAfterSyncFix.sql
+  - dbchanges2/Client_Growrk/2026-10-20a - NetsuiteIdDedupe.sql
+  - dbchanges2/Client_Growrk/2026-09-25a - SalesOrderNetsuiteIdDedupe.sql
+  - library/app/api/toga2.php
+related:
+  - rerunnable-additive-inserts.md
+  - ../../../../1.0/apps/worker/workflows/netsuite-duplicate-items-dedup.md
+  - ../../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md
+  - ../../../../clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md
+  - ../../api2/features/v2-deadlock-retry.md
+---
+Which `c_netsuite*` id columns in NetSuite client DBs are UNIQUE, which never can be, the 3-file key rollout + per-client dedupe pattern (TRUE-82291), and where the duplicates come from; open before adding a key on a NetSuite id or deduping NetSuite-synced rows.
+
+## Summary
+- `c_netsuite*` id columns had no UNIQUE key, so syncs and the 1.0→2.0 migration made duplicate rows silently. TRUE-82291 adds keys in 3 module files + one dedupe file per dirty client (18 clients).
+- **A UNIQUE key turns a silent duplicate insert into MySQL 1062 / API 500.** Fix the sync's lookup (match by NetSuite id) **before** adding the key, or the sync breaks.
+- **Never UNIQUE** (decided, rgirish 2026-10-20):
+  - `Client_Compass.SalesOrders.c_netsuiteInternalSalesOrderId` — the Compass ODP workflow and the NetSuite SO sync write the same NetSuite SO id on purpose. See [ODP pipeline](../../../../clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md).
+  - Columns that point at a parent record: `Client_Aig.Entitlements.c_netsuiteInternalSalesOrderId`, `Entitlements.c_netsuiteLineNumber`, `Client_Growrk.ServiceRequests.c_netsuiteInternalSalesOrderId`, `Client_Prudential.PurchaseOrders.c_netsuiteInternalSalesOrderId`.
+
+## How it works
+
+### The files (run order)
+1. **`2026-09-24b`** (earlier) — UNIQUE on `SalesOrders.c_netsuiteInternalSalesOrderId`, after the per-client SO dedupe (e.g. `Client_Growrk/2026-09-25a`). Skips Adyen + Elite (already unique) and Compass.
+2. **`2026-10-20a`** — 11 columns clean in all 22 NetSuite clients (prod check 2026-10-19): `ItemFulfillments`, `ItemReceipts`, `PurchaseOrders`, `PaymentTerms`, `Subsidiaries`, `TransferOrders.c_netsuiteInternalSalesOrderId` (holds the NetSuite TO id; the UNIQUE key replaces plain `idx_to_netsuite_so_id`), `TransferOrderItems` line key, the 3 `*Stages` status columns, `Contacts.c_netsuiteInternalLeadId` (Adyen + Elite only). Plus `Locations.c_netsuiteInternalCustomerId` (only Prudential was missing it).
+3. **`Client_<Name>/2026-10-20a - NetsuiteIdDedupe.sql`** — per client, see below. Commit by hand.
+4. **`2026-10-20b`** — keys on columns that had duplicates: `Customers`, `Countries`, `Vendors`, `ShippingMethods`, `InventoryAdjustments`, `Invoices`, `Locations` (NetSuite location id), `SalesOrders` (never Compass), `SalesOrderItems`. Run after that client's dedupe is committed.
+5. **`2026-10-20c`** — `Items`, `Units`, `ItemFulfillmentItemUnits`. **Blocked until the sync fixes ship** (see Gotchas).
+
+### Guard shape (all module files, safe to re-run)
+- Each step checks `information_schema`: column exists, no UNIQUE key already leads with it. 20b/20c also count duplicates into `@duplicates` via a prepared statement and skip when > 0. DDL runs through `PREPARE`.
+- A skipped key is not an error: the verify SELECT at the end lists columns with no `NON_UNIQUE = 0` row. Re-run the file after more cleanup.
+- Generic conditional-ALTER pattern: [re-runnable additive migrations](rerunnable-additive-inserts.md#re-runnable-alters-mysql-8-has-no-add-column-if-not-exists).
+
+### Per-client dedupe file pattern
+- **PART 1 (no live change):** build work tables (maps old id → keeper id), back up every row that PART 2 touches, print a report.
+- **PART 2:** one transaction, checks that stop it on surprise counts, `COMMIT` typed by hand after reading the report.
+- **SalesOrders block** reuses `Client_Growrk/2026-09-25a` (ran in prod 2026-09-25).
+- **Items block:**
+  1. `ItemMap` — keeper = MIN `id` per NetSuite item id.
+  2. Unit map — group `Units` by assignment id first, then by serial on the merged item; resolve chains (A→B→C becomes A→C).
+  3. `ItemFulfillmentItemUnits` keeper — the only distinct row; else the only row whose unit's item matches the SO line item; else **MANUAL** (left for a human).
+  4. `VendorItems` grouped by the key **after** the item merge.
+- **Unique bridge tables: move only what the keeper lacks, delete the rest.** Never `UPDATE IGNORE` (same lesson as [Items dedup](../../../../1.0/apps/worker/workflows/netsuite-duplicate-items-dedup.md)).
+
+### Where the duplicates come from (prod, read-only, 2026-10-19)
+- **1.0→2.0 migration (2025-09-10):** same-second copies with the same order number in `SalesOrders` / `SalesOrderItems` — Browardsheriff, Canon, Endeavorhealth, Erau, Growrk, Masonite, Miamidade, Ncci, Shrss, Spglobal, Trividiahealth. Spglobal + Endeavorhealth also got later re-import copies from the SO sync.
+- **Items:** the 1.0 item sync (`library/app/api/toga2.php`) looks items up by **part number**, so the same NetSuite item id gets a new TOGa item when the part number changes or gains a `_` / `-` suffix. Still happening (latest 2026-10-13).
+- **`ItemFulfillmentItemUnits`:** exact copies, plus a fulfillment sync bug that attaches one unit to every line of a fulfillment (Nychh assignment 5919438 on 5 lines).
+- **Units:** Endeavorhealth 1,257 units duplicated 2026-09-15 under duplicate items.
+- **Small sets:** Nychh/Prudential `Customers`; `_united States` country rows holding `_unitedStates`; Nychh `Vendors` "Kiosk" x9; Prudential `ShippingMethods` 318627 on 2 rows; Spglobal `InventoryAdjustments` 233 x3; Growrk invoices copied as `<number>-<nsId>`; Compass `Locations` 26/156 x14.
+
+## Gotchas
+- **Do not run 20c before the sync fixes are deployed.** The item sync (part-number lookup) and the fulfillment sync (one unit on every line) would hit 1062 on every run instead of making a copy.
+- **Before adding a key on any other `c_netsuite*` column, ask: is it the record's own id, or a pointer to a parent?** Pointer columns repeat by design — never unique.
+- A dedupe with MANUAL rows leaves duplicates, so 20b/20c skip that column for that client. That is expected; finish the MANUAL rows, then re-run.
+- Client DBs differ (Contacts NetSuite column only on Adyen + Elite). The guards make a missing column a skip, not a failure.
+
+## Change history
+- 2026-10-21 — Created: TRUE-82291 rollout (20a/20b/20c + 18 client dedupes), duplicate sources from prod 2026-10-19, never-unique columns incl. Compass SalesOrders. (rgirish)
+
+## Related
+- [Re-runnable additive migrations](rerunnable-additive-inserts.md)
+- [1.0 Items dedup (recursive MERGE)](../../../../1.0/apps/worker/workflows/netsuite-duplicate-items-dedup.md)
+- [NetSuite → TOGa Supply per-client sync](../../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)
+- [api2 retry on 1062 race](../../api2/features/v2-deadlock-retry.md)
+- [Compass ODP pipeline to NetSuite](../../../../clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md)

package/knowledge/2.0/apps/toga2-commerce/features/view-as-session-tokens.md

@@ -6,7 +6,7 @@ project: TOGa Commerce
 client: shared
 type: feature
 status: active
-updated: 2026-10-07
+updated: 2026-10-21
 owners: [tcox, bala, apeterson]
 files:
   - src/api/sessionTokens.ts
@@ -27,6 +27,7 @@ related:
   - 2.0/apps/toga2-commerce/workflows/cypress-testing.md
   - 2.0/apps/api2/architecture.md
   - 2.0/apps/_underscore/features/order-for-delegation.md
+  - clients/compass-usa/features/people-file-user-lifecycle.md
 ---
 
 How an admin impersonates another user and gets back: the delegated token pair, the parked admin pair in localStorage, and the return path. Open before touching auth tokens, the axios refresh, or the view-as button.
@@ -90,6 +91,7 @@ Seeds a long-lived `ADMIN_ACCESS_JWT`; stubs `/auth/delegator` → `mock-delegat
 - **Never test "View As is on" with `Object.keys(returnOriginalUser).length > 0`.** Order For parks the same user; use `useIsViewAsActive`.
 - **Clear the delegated state before any await on return.** `setUser(owner)` / `clearOriginalUser` / `endDelegation` must land before the token refresh or any refetch, or an expired parked token's refresh opens a window where the "parked pair gone while delegated" guard logs the user out (and the banner and cart-clear flicker).
 - **⚠ Admin View As is now gated by the Order For delegation list.** `canUserOrderForOtherUser` guards `/users/encryptedUser` and `/auth/delegator` for everyone, so View As returns **403 EZ-1** for any target not on the signed-in user's `Users_DelegatorUsers` rows (beta and client-sandbox). The UI shows nothing — `onError` only logs to Sentry; no toast by decision (tcox), since prod carries the full list.
+- **⚠ View As on a target with no `Users_Roles` row strands the admin.** Every call as the target 403s EZ-1 (`/v2/users/encryptedUser`, `/v2/pages/meta`, `/v2/core-parameters`, bundles), the banner shows the **admin's own** name, return does not work, and clearing cookies does not help. Not a token bug. Diagnose: `Logs_<Tenant>.Api` rows with `responseCode = 403`, `responsePayload.audience.user` = target, app TOGa Commerce; then check `Client_<Tenant>.Users_Roles` for that user id. First case: Compass user 311024, created by the PEOPLE import a day earlier (TRUE-82976). Cause + fix: [PEOPLE-file lifecycle](../../../../clients/compass-usa/features/people-file-user-lifecycle.md#gotchas).
 - **A view-as test must stub `/v2/pages/meta?page=home-page&app=commerce`.** Unstubbed, the real call fails and triggers the enter-view-as rollback, so the spec sees a return-to-admin it did not ask for.
 
 ## Open items (out of ticket, not fixed)
@@ -102,6 +104,7 @@ Seeds a long-lived `ADMIN_ACCESS_JWT`; stubs `/auth/delegator` → `mock-delegat
 
 ## Change history
 
+- 2026-10-21 — Gotcha: View As on a role-less target 403s everything and strands the admin; how to diagnose (TRUE-82976). (bala)
 - 2026-10-07 — Open item: `/users/encryptedUser` permission gap found 2026-09-25 (prod grants to ordinary roles); confirm the delegation gate is live in prod. (jcardinal)
 - 2026-09-30 — Order For merged onto the TRUE-81991 session layer on `_sandbox-client`: parked pair cleared via `AUTH_STORAGE_KEYS`; return and switch use `clearPersistedQueryCache`; a refresh that lost the identity race now returns the current token instead of `null` (stopped a false logout on return); View As `onError` restores the admin's language and logs out if the parked pair is gone. (tcox)
 - 2026-09-22 — Refresh identity binding upgraded to an epoch bumped on the view-as swap; `Bearer undefined` swap bug fixed. TRUE-81991 (`e546db3d`). (apeterson)

package/knowledge/INDEX.md

@@ -24,7 +24,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 - **_underscore** (_Underscore) _(framework core)_ — 113 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 92 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 27 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
-- **dbchanges2** (Database Changes) _(framework core)_ — 22 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
+- **dbchanges2** (Database Changes) _(framework core)_ — 23 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
 - **saml** (SAML SSO Gateway) — 10 doc(s) → [2.0/apps/saml/INDEX.md](2.0/apps/saml/INDEX.md)
 - **apiproxy** (API Proxy) — 1 doc(s) → [2.0/apps/apiproxy/INDEX.md](2.0/apps/apiproxy/INDEX.md)

package/knowledge/clients/canon/profile.md

@@ -5,12 +5,13 @@ apps:
   - library
   - worker
   - vision
+  - dbchanges2
 project: Worker
 client: canon
 type: profile
 status: active
-updated: 2026-10-15
-owners: ["jcardinal", "mhammontree"]
+updated: 2026-10-21
+owners: ["jcardinal", "mhammontree", "rgirish"]
 files: []
 related:
   - ../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

package/knowledge/clients/compass-usa/features/people-file-user-lifecycle.md

@@ -6,7 +6,7 @@ project: Worker
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-10-19
+updated: 2026-10-21
 owners: [bala, ajean, mhammontree]
 files:
   - worker2/Worker/Client/Compass/PeopleFile.php
@@ -24,6 +24,7 @@ related:
   - ../../../2.0/apps/_underscore/features/record-change-audit-log.md
   - ../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md
   - ../../compass-canada/workflows/copying-a-us-user-into-the-canada-tenant.md
+  - ../../../2.0/apps/toga2-commerce/features/view-as-session-tokens.md
 ---
 The nightly Compass PEOPLE-file cron's Users-row lifecycle (insert/reactivate/deactivate), why it creates duplicate accounts, and its raw-SQL/index traps; open before touching PeopleFile.php or debugging stranded approvals.
 
@@ -68,6 +69,13 @@ Current order for a **new** user:
 
 A `_isNewUser` flag on each staged contact is what separates the two paths; existing users missing a contact keep the old create-then-`UPDATE` route. `insertContact()` is the shared helper both use.
 
+### Base roles for new users (same run, restored 2026-10-21)
+Every user gets **Base** + **Compass Base** in `Users_Roles`. Two staging paths, both in the per-row loop:
+- **Existing user** (~line 847-856): stages the missing role ids directly.
+- **New user** (~line 946-956): stages `['_needsUserId' => true, 'username' => ..., 'roleId' => ...]`, same pattern as personas. After the `Users` batch INSERT, the resolver (~line 1288-1302) maps `username` to the new id through the post-insert refresh (`$lookups['users']`).
+
+Role ids are looked up by uuid constants `ROLE_UUID_BASE` / `ROLE_UUID_COMPASS_BASE`, never hardcoded: Compass Base is id **7** in `Client_Compass` and **8** in `Client_CompassCanada`.
+
 ### Order of operations inside a single run
 Within one job the **inserts (and their id refresh) run before the inactivation batch**. That ordering is the hook point for anything that has to bridge an old row to a new one: at insert-refresh time the **new row already has its id** and the **old row is still findable**. A deactivation-time hook would be worse — by then the new row's relationship to the old one has to be rediscovered, and the deactivation path has no model hooks at all.
 
@@ -105,7 +113,9 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
 - **Widening the lookup window is not a free fix.** Removing or extending the grace window would stop new duplicates at source, but also changes which rows the `contactId`-unlinking logic considers and would begin reactivating long-dead accounts. Deliberate deferred item, not an oversight.
 - **`PeopleFile.php` contains a literal SFTP credential in class constants.** Do not paste class constants from this file into tickets, docs, or chat transcripts.
 - **⚠ `TEST_MODE` and `DRY_RUN` do different jobs and NEITHER alone is safe — set BOTH to test.** `TEST_MODE` reads a local file instead of SFTP and skips the SFTP delete, but does **not** stop a single database write. `DRY_RUN` skips the DB writes but does **not** stop the SFTP delete — a "safe" dry run still removes the PEOPLE file from `sftp.goagilant.com` and destroys the input you were inspecting. `DRY_RUN` also skips the whole contact/user insert path (it sits inside `if (!self::DRY_RUN)`), so a dry run **cannot** test that code and reports a misleading success. `TEST_FILE_PATH`'s committed value (`//var//www//html//...`) is not where the app lives on Elastic Beanstalk (`/var/app/current/...`).
-- **⚠ Read newly inserted user ids with `MAX(id)`, on the transaction connection.** A returning employee usually keeps the **same username** as their dead row, so a plain username lookup after the insert can return the **dead** id. And the new rows are invisible to any other connection until the tenant transaction commits, so the read has to happen on the transaction connection.
+- **⚠ Read newly inserted user ids with `MAX(id)`, on the transaction connection.** A returning employee usually keeps the **same username** as their dead row, so a plain username lookup after the insert can return the **dead** id. And the new rows are invisible to any other connection until the tenant transaction commits, so the read has to happen on the transaction connection. **Open (not fixed, TRUE-82976):** the post-insert refresh SELECT (~line 1150) that fills `$lookups['users']` has **no `ORDER BY`**, and the last row per username wins. New-user roles and personas resolve their `userId` through that map, so they can land on the old inactive row. Fix candidate: `ORDER BY id` so the newest row wins.
+- **⚠ A new user with no `Users_Roles` row 403s on every api2 call** (EZ-1 "You are not authorized to perform this action"). From commit `51cfcb9` (TRUE-76423, 2026-01-29) to 2026-10-21 the new-user role staging was missing, so new users got roles only on the **next** daily run (cron `Client/Compass/PeopleFile/ProcessPeopleFiles`, ~09:15 CT). Users created 2026-10-21 were backfilled by `dbchanges2/Client_Compass/2026-10-21a - NewUsersBaseRoles.sql` (76 users) and `Client_CompassCanada/2026-10-21a - NewUsersBaseRoles.sql` (12 users). Any backfill must be date-scoped: Canada dev account `Jugveer.Randhawa` (id 2) lacks Compass Base **on purpose**. Commerce symptom: [View As](../../../2.0/apps/toga2-commerce/features/view-as-session-tokens.md#gotchas).
+- **The `_needsUserId` resolver drops a staged row silently** when its username is not in the refreshed map. No error, no count. A missing role or persona after a "successful" run can be this.
 - **`_Model_Client_Logs_Record::addNote()` is a no-op in this cron.** Staged notes are only flushed by api2, so a worker must write its `Record` rows itself. See [Record Change Audit Log](../../../2.0/apps/_underscore/features/record-change-audit-log.md).
 - **`Client_Compass.Users` is ~176,000 rows with no index on `c_hrEmpPersonnelNbr`, `email`, `c_hrEmpUsername`, or `isActive`.** Any per-row lookup against those columns is a full scan — `EXPLAIN` returns `type=ALL`, 176,018 rows, no possible keys (re-confirmed prod 2026-09-02, same on `Client_CompassCanada`). An index migration is recommended and **still not done**. **The blank-client template is not a safe proxy for this schema**: `dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql` declares `UNIQUE KEY email (email)`, and Compass has diverged from it. Check the live tenant, never the template.
 - **⚠ There is NO UNIQUE KEY on `Users.email` in either tenant — the database will silently accept a duplicate email.** Prod-verified 2026-09-04. Complete index list on `Users` in **both** `Client_Compass` and `Client_CompassCanada`: `PRIMARY(id)`, `uuid`, `Users_UNIQUE(clientId, homeClientUserId)`, `contactId`, `Users_Users_FK(supervisorUserId)`, `Users_Addresses_FK(addressId)`, `Users_Images_FK(thumbnailImageId)`, `dtCreated`, `dtUpdated`, `Users_c_supportedByUserId`, `fk_users_defaultCostCenterId`. Also the proof of the missing-index gotcha above. **Consequence: any code that writes an email must do its own cross-row duplicate check** — and because email is a login/reset identity in 2.0, a duplicate is a security problem, not just messy data. See [Users.email is a login identity](../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md).
@@ -133,6 +143,7 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
 - [Users.email as login identity](../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md)
 
 ## Change history
+- 2026-10-21 — TRUE-82976: restored new-user Base + Compass Base role staging (`_needsUserId`), lost in `51cfcb9`; prod backfill 152 US / 24 CA `Users_Roles` rows; logged the open no-`ORDER BY` refresh risk. (bala)
 - 2026-10-19 — TRUE-82932: files for Oct 17/18/19 missed while the shared SFTP server banned worker2; no email = import never ran; linked the shared-server doc. No code changed. (bala)
 - 2026-10-06 — Noted cart Add New User as a second source of duplicate `Users` rows (now checks email too, TRUE-82423). (bala)
 - 2026-09-23 — Recorded that the import is **not** all-or-nothing (post-commit email + SFTP delete run outside the tenant transaction; job 1076884 failed yet kept 460 US / 26 CA users). TRUE-82230 confirmed in prod: 543 `Client_Compass` users created, **zero** null-or-0 `contactId`, two manual ACTION runs ~18s each — needed because the 09:15 cron row (job 1784367) was stranded. (mhammontree)

package/knowledge/clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md

@@ -6,7 +6,7 @@ project: Worker
 client: compass-usa
 type: workflow
 status: active
-updated: 2026-10-15
+updated: 2026-10-21
 owners: ["rgirish", "bala", "dfranks", "jcardinal", "mhammontree"]
 files:
   - worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
@@ -35,6 +35,7 @@ related:
   - 1.0/apps/library/features/netsuite-soap-toolkit-search.md
   - 1.0/apps/library/features/app-cloud-aws-credentials.md
   - clients/compass-usa/features/oneuptime-office-depot-inbound-files-monitor.md
+  - 2.0/apps/dbchanges2/features/netsuite-id-unique-keys.md
 ---
 The end-to-end Compass ODP order → NetSuite pipeline through the numbered 1.0 worker crons, plus the "this order never reached NetSuite" diagnostic method; open for any ODP-to-NetSuite tracing or cron-mechanics question.
 
@@ -193,6 +194,7 @@ The NetSuite writeback lands on the **downstream ODP SalesOrder (customerId=1)**
 - **To confirm a PO actually reached ODP:** check `PurchaseOrders.dtSubmitted` is set **AND** that downstream ODP SO(s) exist via the `PurchaseOrders_SalesOrders` join — their existence proves ODP received the PO and 850'd back.
 
 ## Gotchas
+- **⚠ `Client_Compass.SalesOrders.c_netsuiteInternalSalesOrderId` can never be UNIQUE.** The Compass order and its Office Depot order share one NetSuite SO id by design (this workflow's cron 5 stamp and the NetSuite SO sync write the same id). Every NetSuite-id key migration skips this column on Compass — see [NetSuite id UNIQUE keys](../../../2.0/apps/dbchanges2/features/netsuite-id-unique-keys.md). Do not dedupe Compass SOs by NetSuite id.
 - **⚠ Cron 3a linked every Compass PO line to the LAST ODP SO line (2026-04-28 → 2026-10-06; fixed `worker` `02cc2473`).** Commit `afb520c1` left a scalar `$officeDepotSalesOrderItemUuid` that the customer-PO link loop reused, so all `PurchaseOrderItems_SalesOrderItems` rows for a PO pointed at the last ODP SO line. The recursive IF walk then broke at the ODP→Compass tier and Compass orders got no fulfillment or tracking (proven on SA130861 / SA130762). Fix: link via `$lookupSalesOrderItemUuidFromPurchaseOrderLine[lineNumber]`; repeated vendor part numbers pair one-to-one (`array_shift` on the lookup). Zero bad links on ~440 orders since 2026-10-07. History is repaired by [retrofix2](../../../1.0/apps/test/features/compass-retrofix2-sql-generator.md) (bulk) or [fix_item_fulfillment_chain](../../../1.0/apps/test/features/compass-fix-item-fulfillment-chain.md) (one order). **Since 3a resolves 850 lines through ODP `VendorItems`, the ODP SO line is often the catalog-2 twin of the Compass item — match these tiers by part number, not `itemId`.**
 - **⚠ 2026-10-10 → 10-12: ODP POs stopped arriving because the Lambda stopped, not 3a.** An attacker used a leaked long-lived IAM key; AWS suspended services including Lambda, so `rename_edi_files` stopped at **07:29 CDT Sat 2026-10-10** and **52 POs** piled up in `processed/`. 67 `.net` files (850s + 997s) were moved by hand on 2026-10-12; 3a created all 52 POs (0 rejects) and cron 5 sent them to NetSuite. The leaked key was deleted the same day; 3a and the move cron now run on the `WorkloadsRuntime` role ([AWS credentials](../../../1.0/apps/library/features/app-cloud-aws-credentials.md)). All stuck POs imported. The `OfficeDepot/` backlog monitor stayed green throughout (its folder was empty); the [inbound files monitor](../features/oneuptime-office-depot-inbound-files-monitor.md) now covers that gap. Symptom of a dead mover: files pile up in `processed/`, `OfficeDepot/` stays empty, no new ODP SalesOrders.
 - **⚠ `edi/1_download_edi_s3_create_po_toga.php` is DEAD CODE (correction 2026-08-04).** It appears in **no** schedule file. The scheduled entry *named* "Download EDI From S3 Files & Create PO - EDI Office Depot" (`*/5 * * * *`, `worker/schedules/cron.worker.sync.json` `:254-259`) actually points at **`toga2/compass/workflow/3a_import_office_depot_purchase_orders.php`**. The two scripts have **different idempotency and different API call shapes** — `edi/1`'s create branch POSTs the SO with *nested* items and a `customer.name`, while **3a POSTs the header alone (uuids) and then POSTs the items in a separate request**. Reasoning from `edi/1` gives wrong answers. **General rule: never infer which script runs from its filename or folder** — verify against `worker/schedules/cron.*.json`, and note the schedule entry's **name can be actively misleading**. The folders `compass/edi/`, `compass/DOA_edi_odp_flow/`, and `compass/DOA_Scripts/` are **legacy / inactive**; `compass/edi/3_send_edi_855(reject).php` is an abandoned sketch using **non-standard** reject codes (`BAK01=A1` / `BAK02=RE`) — **do not** follow it.
@@ -231,6 +233,7 @@ The NetSuite writeback lands on the **downstream ODP SalesOrder (customerId=1)**
 - **Editing cron recipient lists — a grep hit is not a live hit.** Email recipients are hardcoded in the cron scripts, and **several dormant copies of the same script exist** under `crons/toga2/compass/edi/` and `crons/toga2/compass/DOA_edi_odp_flow/`. Before changing a recipient list, cross-check the file against `worker/schedules/*.json` for a referencing entry whose `active` is not `0`. Only the scheduled copies matter.
 
 ## Change history
+- 2026-10-21 — Gotcha: Compass SalesOrders NetSuite SO id is never UNIQUE (TRUE-82291 skips it). (rgirish)
 - 2026-10-15 — Cron 3a last-ODP-line link bug (since `afb520c1`) root-caused and fixed in `02cc2473`; re-drop does not repair a wrong link. (jcardinal)
 - 2026-10-14 — TRUE-82797 (pending deploy): cron 5 merged only one ODP SO per Compass SO, so siblings went out as a 2nd NetSuite SO (41 Compass SOs in 60 days). Fix merges all ODP SOs of a Compass SO, splits tablets by `TABLET_KIT_PART_NUMBERS`, externalId = lowest ODP SO id, duplicate-part guard emails devteam. Decided against category grouping. (rgirish)
 - 2026-10-13 — Stop-gap move cron live on the role (Lambda still suspended); all stuck POs imported; linked the new `OfficeDepotInboundFiles` monitor. (bala)

package/knowledge/clients/endeavor-health/profile.md

@@ -4,11 +4,12 @@ framework: "2.0"
 apps:
   - library
   - worker
+  - dbchanges2
 project: Worker
 client: endeavor-health
 type: profile
 status: active
-updated: 2026-09-18
+updated: 2026-10-21
 owners: ["jcardinal", "rgirish"]
 files: []
 related:

package/knowledge/clients/spglobal/profile.md

@@ -6,12 +6,13 @@ apps:
   - worker
   - vision
   - store
+  - dbchanges2
 project: Worker
 client: spglobal
 type: profile
 status: active
-updated: 2026-10-20
-owners: ["jcardinal", "mhammontree", "snaredla"]
+updated: 2026-10-21
+owners: ["jcardinal", "mhammontree", "snaredla", "rgirish"]
 files:
   - store/_/app/framework.php
   - store/common/spglobal/welcome/get.php

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.1095",
+  "version": "1.0.1097",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",