toga-ai 1.0.860 → 1.0.861

package/knowledge/1.0/apps/store/INDEX.md

@@ -3,4 +3,5 @@
 | Doc | Summary |
 |-----|---------|
 | [Store (1.0 Storefront / SAML ACS) Architecture](architecture.md) | 1.0 (App_) storefront that also hosts the SAML ACS for several `*.togahub.com` clients (inbound IdP assertion decrypted here, not in the 2.0 `saml` gateway); op |
+| [Prudential SAML/hash SSO login — user+contact provisioning (and the unsigned ?hash impersonation debt)](features/prudential-sso-login.md) | The Prudential storefront SSO block that provisions the user + contact on login, why it now links `Users.contactId`, and the pre-existing unsigned `?hash=` impe |
 | [Store SAML SP Certificate & Private Keys (togahub.com)](features/saml-sso-certificate-and-keys.md) | How the store SAML SP credential (`togahub.com` X.509 cert + RSA private key) is stored, loaded and rotated — incl. |

package/knowledge/1.0/apps/store/features/prudential-sso-login.md

@@ -0,0 +1,37 @@
+---
+title: "Prudential SAML/hash SSO login — user+contact provisioning (and the unsigned ?hash impersonation debt)"
+framework: "1.0"
+repo: store
+project: Store
+client: prudential
+type: feature
+status: active
+updated: 2026-09-21
+owners: [jcardinal]
+files:
+  - store/_/app/framework.php
+related:
+  - ../architecture.md
+  - ../../../../2.0/apps/_underscore/features/client-user-contactid-required.md
+  - ../../../../clients/prudential/features/device-information-import-and-contact-linking.md
+---
+The Prudential storefront SSO block that provisions the user + contact on login, why it now links `Users.contactId`, and the pre-existing unsigned `?hash=` impersonation debt. Open before touching store SSO login or Prudential user provisioning.
+
+## Summary
+
+On Prudential SAML SSO (`store/_/app/framework.php`), after the assertion is decrypted the block **finds/creates the user and the contact separately**. It historically **never linked them** — the root cause of ~14k null-`contactId` Prudential users. It now links them.
+
+## How it works
+
+- After find/create of the user and contact, the block issues a **non-fatal** `PUT /users/{uuid} {contact:{uuid}}` to link `Users.contactId`.
+- `Users.contactId` is a normal forward FK, so `{uuid}` links safely (not a singular-FK back-reference case).
+- The link checks `isSuccess` and **logs on failure without breaking login** — a failed link must never block a user's SSO.
+
+This is the Prudential storefront's slice of the platform-wide contactId initiative — see [client-user-contactid-required](../../../../2.0/apps/_underscore/features/client-user-contactid-required.md). Existing null-contactId Prudential users are backfilled by the `Client_Prudential/2026-09-21a` migration (XID-keeper link) — see [device-information-import-and-contact-linking](../../../../clients/prudential/features/device-information-import-and-contact-linking.md).
+
+## Gotchas
+
+- **⚠ SECURITY DEBT — the `?hash=` login trusts an UNSIGNED identity from the URL (impersonation).** After assertion, the Prudential branch base64-encodes a plain `xid:first:last:email` string and redirects to `<returnTo>?hash=<base64>` (`framework.php` ~L120-130). The hash is **not signed / no HMAC**, so any crafted `?hash=` can **impersonate any employee** — it creates the user+contact and reads their devices/ServiceNow assets — and PII rides in the URL query string (web-server logs). Pre-existing; a ticket is being filed separately. Do not extend this handoff without signing it. The store architecture also flags this handoff as unsigned/unencrypted.
+
+## Change history
+- 2026-09-21 — Initial. Added the non-fatal `PUT /users {contact:{uuid}}` link so SSO no longer leaves `contactId` null (root cause of ~14k null-contactId Prudential users). Recorded the pre-existing unsigned `?hash=` impersonation debt as a known security gotcha (ticket filed separately). (jcardinal)

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -15,6 +15,7 @@
 | [FIELD_SQL calculated fields — the underscore-prefix + same-name-method contract](features/calculated-sql-fields.md) | A `FIELD_SQL` (calculated) field on a `_Model` is bound by a **two-part contract that `_Model` enforces by throwing at model-construction time**, not by convent |
 | [Carrier Shipping Labels (UPS/FedEx) & NetSuite Item Fulfillment](features/carrier-shipping-labels.md) | Backend mechanics behind Fulfill & Ship: buying/persisting a carrier label (UPS/FedEx) and creating the NetSuite Item Fulfillment; open for label, reprint, retu |
 | [Running 2.0 code from a bare CLI script (bootstrap + transactions)](features/cli-script-bootstrap.md) | Minimum recipe + traps for running real 2.0 framework code (`_Model`/`_Query`/`_Database`) from a bare `php script.php`; open before writing a CLI harness/backf |
+| [Client Users require a contactId — the postPost backfill interceptor & every creation path](features/client-user-contactid-required.md) | Every `Client_<Tenant>.Users` row must have a non-null `contactId`; the shared `_Model_Client_User::postPost` interceptor guarantees it for api2-created users, |
 | [_Cloud S3 helpers (copy / get / delete / list)](features/cloud-s3-helpers.md) | `_Cloud` S3 helpers (copy/get/delete/list) with per-bucket region override and a 5 GB single-PUT limit; open before doing S3 access from 2.0 code. |
 | [_Cloud Secrets Manager helper (_Cloud::getSecret)](features/cloud-secrets-manager.md) | `_Cloud::getSecret` reads AWS Secrets Manager (own default-provider auth, never throws, per-process cache); open before pulling secret material at request time. |
 | [_Component_*/_Model_* project-namespace registration (autoloader) & backslash-qualify traps](features/component-model-namespace-registration.md) | Project-local `_Component_*`/`_Model_*` must declare the app namespace AND live in a `<Name>/<Name>.php` directory, or the autoloader fatals at runtime; open wh |

package/knowledge/2.0/apps/_underscore/features/client-user-contactid-required.md

@@ -0,0 +1,67 @@
+---
+title: "Client Users require a contactId — the postPost backfill interceptor & every creation path"
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: feature
+status: active
+updated: 2026-09-21
+owners: [jcardinal]
+files:
+  - _underscore/Model/Client/User.php
+  - worker2/Worker/Sync/ActiveDirectory.php
+  - worker2/Worker/Netsuite/Opportunity.php
+  - dbchanges2/Core/2026-09-21d - Register users postPost contact interceptor.sql
+related:
+  - ../../api2/features/api-payload-interceptors.md
+  - ../../api2/features/nested-relationship-writes.md
+  - ./note-record-authorship.md
+  - ./user-email-as-identity-and-password-reset.md
+  - ../../../1.0/apps/store/features/prudential-sso-login.md
+  - ../../../clients/prudential/features/device-information-import-and-contact-linking.md
+  - ../../dbchanges2/features/rerunnable-additive-inserts.md
+---
+Every `Client_<Tenant>.Users` row must have a non-null `contactId`; the shared `_Model_Client_User::postPost` interceptor guarantees it for api2-created users, and each other creation path was fixed to link one too. Open before touching user creation or `Users.contactId`.
+
+## Summary
+
+Decision (2026-09-21, CTO Option B): `Client_<Tenant>.Users.contactId` is made **NOT NULL per tenant** after backfilling a Contact for every user missing one. Null `contactId` was the root of the missing/duplicate-contact bugs, so the goal is to kill it and keep it dead — a **forward fix on every creation path** plus a per-client backfill+ALTER.
+
+**Rollout order that matters:** deploy the forward fixes first, then run the per-client backfill+`NOT NULL` migrations. A backfill run before the code fix regenerates null rows.
+
+## How it works
+
+### The shared forward fix — `_Model_Client_User::postPost` (CTO Option B)
+
+`postPost(&$api, &$payload)` on the **base client user model** creates a Contact + a primary **Work** email and links `Users.contactId` when an api2-created user has none. Registered as a **postPost interceptor** via `Core.ApiPayloadInterceptors` (`recordId 2` = users, `clientId` + `apiId` NULL = all clients, `prePostProcessing 'POST'`, `httpMethod 'POST'`, guarded insert) — see [api-payload-interceptors](../../api2/features/api-payload-interceptors.md) for how the row makes the hook live.
+
+- **No-op** when `contactId` is already set, or the user has no email. Runs **inside the create transaction**.
+- Chosen over a base-model `save()` override for blast radius / "app-level over core" — the interceptor is the row-gated switch, not a change every save inherits.
+- Placed on `_Model_Client_User` **on purpose** (the "row is the switch" pattern): the `apiId NULL / clientId NULL` row makes it fire for every tenant's api2 CRUD.
+
+**⚠ Set `Contacts.primaryContactEmailAddressId` SERVER-SIDE — a second model load+save — NOT via a nested payload.** The nested singular-FK path is unreliable: the CREATE path re-parents a value-matched child, and the UPDATE path echoes null. See [nested-relationship-writes](../../api2/features/nested-relationship-writes.md) (singular-FK back-reference defects).
+
+### Every 2.0 user-creation origin path (does it set contactId?)
+
+The interceptor covers the **generic api2 CRUD** for everyone. The dedicated creation paths each had to be fixed or already set it:
+
+| Path | Repo / code | Client | contactId |
+|---|---|---|---|
+| api2 CRUD (all clients) | `_Model_Client_User::postPost` | all | now linked (interceptor) |
+| AD sync | `worker2/Worker/Sync/ActiveDirectory.php` | True | **fixed** — now creates+links |
+| ClickUp user cache | `worker2/Worker/Netsuite/Opportunity.php` `cacheClickupUser()` (raw SQL) | True | **fixed** — `ensureTrueContactId` |
+| Store SAML/hash SSO | `store/_/app/framework.php` | Prudential | **fixed** — see [prudential-sso-login](../../../1.0/apps/store/features/prudential-sso-login.md) |
+| SAML gateway | `_Model_Rate_ClientAuthentication` (saml) | Rate | already sets it |
+| Compass PeopleFile | worker2 (raw SQL) | Compass | already sets it |
+
+Northwell ServiceNow users arrive via api2 eBonding (`_Model_Northwell_Ticket` interceptors), not a dedicated worker, so the interceptor covers them.
+
+## Gotchas
+
+- **⚠ A reconstructed `new _Model_Client_User($id)` does NOT carry `contactId`.** In `ActiveDirectory.php` the AD sync reads the existing `contactId` from the **bulk SELECT** and only creates a contact when it is truly missing. A naive `empty($user->contactId)` guard against a freshly reconstructed model reads null even when the row has a contact — and would **mint a duplicate contact every sync**. Trust the SELECT'd value, not the rehydrated model property.
+- **⚠ Set `primaryContactEmailAddressId` with a second server-side load+save, never a nested payload.** The nested singular-FK write re-parents (CREATE) or echoes null (UPDATE).
+- **Interceptor no-ops on a user with no email** — a no-email straggler gets no Contact from this path (the backfill migrations handle those name-only, per client). It is why the fan-out `NOT NULL` skips a client that still has a no-email residual (see [rerunnable-additive-inserts](../../dbchanges2/features/rerunnable-additive-inserts.md)).
+
+## Change history
+- 2026-09-21 — Initial. Decided `Client_*.Users.contactId` goes NOT NULL platform-wide (CTO Option B); built the shared `_Model_Client_User::postPost` interceptor (Core row: recordId 2, clientId+apiId NULL) to create+link a Contact for api2-created users; fixed the worker2 AD-sync and `cacheClickupUser` True paths; recorded the origin map and the reconstructed-model `contactId` gotcha. (jcardinal)

package/knowledge/2.0/apps/dbchanges2/features/rerunnable-additive-inserts.md

@@ -268,6 +268,50 @@ Worked example: `Client/2026-08-28b - TransferOrderItemsTimestamps.sql` (adds
 `Client/2026-08-18` `qtyCommitted` file uses a bare `ALTER` and is exactly the shape this avoids** —
 it is not the template to copy.
 
+### ⚠ Guarding a `NOT NULL` ALTER — `sql_mode = ''` SILENTLY coerces leftover NULLs
+
+`ALTER … MODIFY col <type> NOT NULL` after a backfill looks safe, but this cluster's `sql_mode` is
+`''` (empty), so MySQL does **not** reject rows that are still NULL — it **silently coerces them to
+`0`** (an invalid FK, e.g. `Users.contactId = 0`). Same `sql_mode` family as the zero-date `dtCreated`
+trap above. Guard the ALTER on a residual count and compile it to the ALTER only when clean:
+
+```sql
+SET @n = (SELECT COUNT(*) FROM Users WHERE contactId IS NULL);
+SET @sql = IF(@n = 0,
+	'ALTER TABLE Users MODIFY contactId INT NOT NULL',
+	'<abort-or-skip>'
+);
+PREPARE stmt FROM @sql; EXECUTE stmt; DEALLOCATE PREPARE stmt;
+```
+
+- **Single-tenant file → abort LOUD.** Make `<abort-or-skip>` a statement that errors under `sql_mode ''`
+  — `SELECT \`unknown col\` FROM DUAL` throws *Unknown column*, so the file stops instead of shipping a
+  broken column.
+- **Fan-out (`Client/`) file → skip with `DO 0`.** Make `<abort-or-skip>` = `'DO 0'` so one tenant's
+  no-email straggler does not abort the ALTER for every other tenant. Wrap the backfill in
+  `START TRANSACTION … COMMIT` and put the ALTER **after** the COMMIT.
+
+### ⚠ latin1 schema drift breaks a forced COLLATE in a fan-out file
+
+Some tenants' `uuid`/text columns are `latin1`, not utf8mb4 (verified Nychh, Spglobal). So a forced
+`Contacts.uuid COLLATE utf8mb4_0900_ai_ci` in a `Client/` fan-out dies on those tenants:
+`ERROR 1253 COLLATION 'utf8mb4_0900_ai_ci' is not valid for CHARACTER SET 'latin1'`. Join
+charset-agnostically instead:
+
+```sql
+CONVERT(Contacts.uuid USING utf8mb4) COLLATE utf8mb4_0900_ai_ci = <temp.uuid>
+```
+
+A single-tenant file targeting a **known-utf8mb4** client may keep the plain `COLLATE`.
+
+### Backfilling a user→contact link deduped by a client identifier key
+
+When users are keyed to a contact by a client identifier (e.g. `c_employeeXid`, `c_sysId`), pick
+**ONE keeper per key** with `ROW_NUMBER()` (prefer primary-email, then name, then oldest id), link
+**all** users in that key group to the keeper, and dedupe residual **creates by key too** — one Contact
+per key group, never one per user when they share a key. Create a Work email only when the user has one;
+a no-email user gets a name-only Contact from its own `firstName`/`lastName`.
+
 ### `LAST_INSERT_ID()` into a session variable ties the whole file to one session
 
 The common shape — `INSERT` a parent, `SET @parentId = LAST_INSERT_ID();`, then insert children
@@ -366,6 +410,12 @@ concluding a migration is production-safe.
   [FIELD_SQL calculated fields](../../_underscore/features/calculated-sql-fields.md).
 
 ## Change history
+- 2026-09-21 — Added three lessons from the platform-wide `Users.contactId` NOT NULL backfill: a guarded
+  `NOT NULL` ALTER (`sql_mode ''` silently coerces leftover NULLs to `0`, so gate on a residual count —
+  single-tenant aborts loud, fan-out `DO 0` skips); latin1 schema drift breaks a forced
+  `COLLATE utf8mb4_0900_ai_ci` in a `Client/` fan-out (Nychh/Spglobal are latin1) so join via
+  `CONVERT(col USING utf8mb4) COLLATE …`; and the deduped user→contact backfill (one keeper per client
+  key via ROW_NUMBER, dedupe residual creates by key, name-only Contact for a no-email user). (jcardinal)
 - 2026-09-21 — Narrowed the "reuse one uuid literal across tenants" rule: it holds only when the rows share a
   natural key. An env-specific twin file (guarded on `recordFieldId`, which differs per environment) reusing the
   literal dies on `Duplicate entry … aclfieldpermissions.uuid`. Also added the DELETE-first transaction rehearsal

package/knowledge/INDEX.md

@@ -14,12 +14,12 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 - **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
 - **test** (Test) — 16 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
 - **toga** (TOGa) — 3 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
-- **store** (Store) — 2 doc(s) → [1.0/apps/store/INDEX.md](1.0/apps/store/INDEX.md)
+- **store** (Store) — 3 doc(s) → [1.0/apps/store/INDEX.md](1.0/apps/store/INDEX.md)
 - **tools** (Tools) — 18 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 88 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 89 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 74 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
@@ -55,6 +55,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 - **Endeavor Health** (`endeavor-health`) → [clients/endeavor-health/INDEX.md](clients/endeavor-health/INDEX.md)
 - **Fordham University** (`fordham`) → [clients/fordham/INDEX.md](clients/fordham/INDEX.md)
 - **GroWrk** (`growrk`) → [clients/growrk/INDEX.md](clients/growrk/INDEX.md)
+- **Managelife** (`managelife`) → [clients/managelife/INDEX.md](clients/managelife/INDEX.md)
 - **Northwell Health** (`northwell`) → [clients/northwell/INDEX.md](clients/northwell/INDEX.md)
 - **New York City Department of Education** (`nycdoe`) → [clients/nycdoe/INDEX.md](clients/nycdoe/INDEX.md)
 - **NYC Health & Hospitals** (`nychh`) → [clients/nychh/INDEX.md](clients/nychh/INDEX.md)

package/knowledge/clients/managelife/INDEX.md

@@ -0,0 +1,5 @@
+# Client: Managelife `managelife`
+
+| Doc | Framework | Summary |
+|-----|-----------|---------|
+| [Managelife](profile.md) | 2.0 | Managelife 2.0 platform tenant (`Client_Managelife`). |

package/knowledge/clients/managelife/profile.md

@@ -0,0 +1,22 @@
+---
+title: "Managelife"
+framework: "2.0"
+apps:
+  - dbchanges2
+project: Database Changes
+client: managelife
+type: profile
+status: active
+updated: 2026-09-21
+owners: [jcardinal]
+files: []
+related:
+  - ../../2.0/apps/_underscore/features/client-user-contactid-required.md
+---
+
+Managelife 2.0 platform tenant (`Client_Managelife`). Stub profile — expand as work touches it. Formal client name not yet confirmed.
+
+## Summary
+
+- 2.0 tenant with a `Client_Managelife` database.
+- First recorded work: the platform-wide `Users.contactId` NOT NULL backfill — `Client_Managelife/2026-09-21a` gave its 1 user (id 5, no email) a name-only Contact from its own `firstName`/`lastName`, linked it, then set `contactId` NOT NULL. See [client-user-contactid-required](../../2.0/apps/_underscore/features/client-user-contactid-required.md).

package/knowledge/clients/prudential/features/device-information-import-and-contact-linking.md

@@ -57,6 +57,11 @@ Resumable, unit-driven. For each `Unit` with `contactId` NULL it queries Service
 - **No `continue`** in the helpers added to this cron (team standard); the new backfill file is tab-indented (the legacy cron body is space-indented — only the added helpers were converted).
 - API auth uses the `App_Api_Toga2::CLIENT_UUID_PRUDENTIAL` / `API_UUID_PRUDENTIAL` / `API_SECRET_PRUDENTIAL` constants (names only — values live in config, not here).
 
+- **⚠ `Client_Prudential.Users.contactId` is now NOT NULL — and do NOT blind email-create contacts for Prudential.** The `Client_Prudential/2026-09-21a` backfill linked ~14,298 null-contactId users to their `c_employeeXid` **keeper** contact (ROW_NUMBER: primary-email, then name, then oldest id), created a Contact+email for only ~20 residual, then set `contactId` NOT NULL. Prudential's Contacts were just deduped 893k→19.7k, so a blanket email-create backfill would **re-pollute** them — link by XID keeper, never create-per-user. Part of the platform-wide contactId initiative: [client-user-contactid-required](../../../2.0/apps/_underscore/features/client-user-contactid-required.md).
+
+## Change history
+- 2026-09-21 — `Client_Prudential.Users.contactId` made NOT NULL after the `2026-09-21a` backfill linked null-contactId users to their XID keeper contact. Noted the "never blind email-create for Prudential" rule (would re-pollute the just-deduped Contacts) and cross-linked the platform contactId initiative. (jcardinal)
+
 ## Related
 - [Prudential profile](../profile.md)
 - [2.0 api2 — Nested-relationship writes & child matching](../../../2.0/apps/api2/features/nested-relationship-writes.md) — the general api2 behavior this incident exposed.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.860",
+  "version": "1.0.861",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",