toga-ai 1.0.282 → 1.0.283

package/knowledge/2.0/apps/api2/INDEX.md

@@ -6,6 +6,7 @@
 | [Multi-Client (Cross-Client) Data Retrieval](features/cross-client-data-retrieval.md) | A single authenticated V2 GET listing can return records across **many** clients (designed for 1000+) that the caller is entitled to, honoring **each target cli | api2/Component/Api/CrossClient/CrossClient.php, api2/Component/Api/V2/V2.php, api2/Controller/Index.php, api2/_.php, _underscore/Model/Cache/Table.php, _underscore/Model/Cache/Tables/Client.php, dbchanges2/Core/2026-06-30b - CacheClusterRegistrationAndRecordTtl.sql, dbchanges2/Cache/2026-06-30a - MultiClientCacheTables.sql |
 | [Encrypted-User-UUID Auth Handoff (/auth/encrypted-user-uuid)](features/encrypted-user-uuid-auth-handoff.md) | `POST /auth/encrypted-user-uuid` is the intended **cross-client / SSO-handoff identity mechanism**: given an encrypted `{client, user}` UUID pair, it mints a fr | api2/Component/Api/CrossClient/CrossClient.php |
 | [Language Translation Layer (audience.language + sidecar tables)](features/language-translation-layer.md) | Serves the same TOGa data (Item title/description/longDescription, expanding later) in multiple languages without forking the schema or breaking English consume | api2/Component/Api/V2/V2.php, api2/Component/Api/V2/Response/Response.php, _underscore/Model/Core/Setting.php, _underscore/Model/Core/RecordField.php, _underscore/Model/Core/DefaultGlobalSetting.php, _underscore/Model/Client/ItemTranslation.php, dbchanges2/Client/2026-06-23a - ItemTranslations.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql, dbchanges2/Core/2026-06-23a - RecordFieldsTranslationColumn.sql, dbchanges2/Core/2026-06-23b - ItemTranslationsRecord.sql |
+| [Nested-relationship writes & child matching (link vs. create)](features/nested-relationship-writes.md) | When a 2.0 API write payload (`POST`/`PUT`) contains a **nested related object** (e.g. | api2/Component/Api/V2/V2.php |
 | [POST + JSON-body args for scripted APIs](features/scripted-api-post-body-args.md) | The V2 engine can run a Record Script (scripted API) for a **POST** request, and a scripted API can receive its arguments from the **JSON request body** instead | api2/Component/Api/V2/V2.php |
 | [Surface action-state via the surface=<slug> request option (M2M-safe)](features/surface-meta-option.md) | An opt-in V2 engine request option, `surface=<slug>`, that attaches per-record UI action state (`isVisible`/`isEnabled`) to a GET response **under `meta.surface | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Surface.php |
 | [Tickets API (/v2/tickets)](features/tickets-api.md) | The generic ticket endpoint of the 2.0 REST API. | Component/Api/V2/V2.php |

package/knowledge/2.0/apps/api2/architecture.md

@@ -6,8 +6,8 @@ project: API
 client: shared
 type: architecture
 status: active
-updated: 2026-06-08
-owners: [jcardinal]
+updated: 2026-07-07
+owners: [jcardinal, bala]
 files:
   - api2/Controller/Index.php
   - api2/Component/Api/V2/V2.php
@@ -34,6 +34,9 @@ framework is **pulled at deploy, not vendored**. Composer deps: `sentry/sentry`,
 **Critical rules:** every action method must return the exact envelope
 `['success' => bool, 'data' => mixed, 'errors' => array]` — never a raw string or bare array.
 On error: `success=false`, `data=null`, `errors=[...]`. Deviating breaks API consumers silently.
+A **nested related object** in a write payload links to an existing record **only** by an
+identifier (`uuid`); a non-unique custom field (e.g. `c_employeeXid`) is not a match key and
+forces a **new-record insert** every write — link with `contact: {uuid}`.
 
 ## Dependencies
 

package/knowledge/2.0/apps/api2/features/nested-relationship-writes.md

@@ -0,0 +1,80 @@
+---
+title: "Nested-relationship writes & child matching (link vs. create)"
+framework: "2.0"
+repo: api2
+project: API
+client: shared
+type: feature
+status: active
+updated: 2026-07-07
+owners: ["bala"]
+files:
+  - api2/Component/Api/V2/V2.php
+related:
+  - ../architecture.md
+---
+
+## Summary
+
+When a 2.0 API write payload (`POST`/`PUT`) contains a **nested related object** (e.g. a unit
+payload with an embedded `contact: {…}`), the api2 V2 engine decides per nested object whether to
+**link an existing** related record or **create a new one**. The rule that governs this is narrow
+and easy to get wrong:
+
+> A nested related object is matched to an existing record **only** by fields flagged as
+> **identifiers** (`uuid` is the identifier field). If the object contains an identifier, the
+> engine **loads and links** the existing record. If it contains no identifier field, the engine
+> **creates a new child record** — even if the object carries a value (like a custom field) that
+> *looks* unique to a human.
+
+Implemented in `Component/Api/V2/V2.php` (`getForeignKeyValue()` / the nested-write matching in
+`processRoutePairs()`).
+
+## How it works
+
+- **`uuid` is the identifier field.** A nested object of `{uuid: <existing-uuid>}` forces a
+  **MATCH**: the engine loads that existing record and links the parent to it — it never creates.
+  (The numeric internal `id` also links an existing record.)
+- **A non-unique custom field is NOT a match key.** Passing a nested object that contains only a
+  plain custom field — e.g. `contact: {c_employeeXid: 'X208000'}` — provides **no identifier**, so
+  the engine has nothing to match on and **inserts a new child record every time**. It does not
+  matter that `c_employeeXid` is conceptually the employee's unique key; it is not flagged as an
+  identifier, so the engine ignores it for matching.
+
+## Correct pattern
+
+To **link** an existing related record, send its identifier:
+
+```
+"contact": { "uuid": "<existing-contact-uuid>" }   // links; never creates
+```
+
+To **create** a new related record, send its data **without** an identifier (and accept that a new
+row is written). Never rely on a non-identifier field (a custom field, a name, an email) to
+de-duplicate — the engine will not use it to match, and you will accumulate duplicate empty child
+rows on every write.
+
+If you need to link by a business key (like an employee XID), **resolve that key to a uuid in your
+caller first** (e.g. build a `key → uuid` map), then send `{uuid}`.
+
+## Gotcha
+
+- **Nested write with only a non-identifier field silently creates duplicates.** This is a
+  duplicate-generating footgun: each write inserts a new empty child and the parent links to a
+  fresh shell (or is left with a NULL FK). It compounds on any cron/integration that writes the
+  same entity repeatedly. Root-caused in production on the Prudential device-sync cron, which sent
+  `contact: {c_employeeXid}` and generated ~893k empty Contact rows (~5,600/day). See
+  [Prudential device import + contact linking](../../../clients/prudential/features/device-information-import-and-contact-linking.md).
+
+## Change history
+
+- 2026-07-07 — Documented that api2 nested-relationship writes match an existing child **only** by
+  identifier (`uuid`); a non-unique custom field is not a match key and forces a new-record insert.
+  Correct pattern is `contact: {uuid}` (resolve business keys to uuid in the caller). Surfaced by
+  the Prudential "null null" contact-export incident. (bala)
+
+## Related docs
+
+- [api2 architecture](../architecture.md) — the V2 CRUD engine (`processRoutePairs`).
+- [Prudential device import + contact linking](../../../clients/prudential/features/device-information-import-and-contact-linking.md)
+  — the production incident that exposed this behavior.

package/knowledge/2.0/apps/dbchanges2/architecture.md

@@ -6,8 +6,8 @@ project: Database Changes
 client: shared
 type: architecture
 status: active
-updated: 2026-07-02
-owners: [jcardinal, mhammontree]
+updated: 2026-07-07
+owners: [jcardinal, mhammontree, bala]
 files:
   - Core/
   - Client/
@@ -233,6 +233,17 @@ query into a temporary derived table, so the outer `DELETE` no longer "sees" a l
 its own target. Used in `2026-07-02a - FixSA133377TrackingSerialAndDuplicateIF.sql` to prune
 duplicate `ItemFulfillments` rows.
 
+### FK back-reference cycle — re-point detail rows, don't delete them
+
+When deduping a parent (e.g. `Contacts`) that has a back-reference FK to its own detail rows
+(`primaryContactEmailAddressId` → `ContactEmailAddresses`, `primaryContactPhoneNumberId` →
+`ContactPhoneNumbers`), deleting the non-keeper detail rows breaks the cycle. Instead re-point
+`ContactEmailAddresses`/`ContactPhoneNumbers.contactId` to the keeper (safe because `contactId`
+is non-unique on those tables → no collision), which preserves every primary pointer. Detail
+tables with a `UNIQUE(parentId, childId)` (e.g. `ContactAddresses`) can't be re-pointed —
+null-then-delete those. Then delete the non-keeper parents via the materialized double-nested
+subquery. Used in `Client_Prudential/2026-07-07 - Contact Dedup Merge.sql`.
+
 ## Relationship to the rest of 2.0
 
 `dbchanges2` is registered as a **2.0 core repo** (`role: core` in `registry.json`) — it is

package/knowledge/INDEX.md

@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 1.0 framework
 
 - **library** (Library) _(framework core)_ — 11 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
-- **worker** (Worker) — 13 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
+- **worker** (Worker) — 14 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
 - **togadesk** (TOGa Desk) — 8 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)
 - **togaview** (TOGa View) — 6 doc(s) → [1.0/apps/togaview/INDEX.md](1.0/apps/togaview/INDEX.md)
@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 - **_underscore** (_Underscore) _(framework core)_ — 22 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 27 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
-- **api2** (API) — 9 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
+- **api2** (API) — 10 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
 - **saml** (SAML SSO Gateway) — 3 doc(s) → [2.0/apps/saml/INDEX.md](2.0/apps/saml/INDEX.md)

package/knowledge/clients/prudential/INDEX.md

@@ -4,6 +4,7 @@
 |-----|-----------|---------|-------|
 | [Prudential: Dell ASN units PRE/POST interceptor (legacy key + flat tracking)](features/dell-asn-units-interceptor.md) | 2.0 | After the tracking-number bridge migration, the ASN unit route was renamed (`advance-shipping-notice-units` → `advance-shipping-notice-item-units`), so the inhe | _underscore/Model/Prudential/AdvanceShippingNotice.php, dbchanges2/Client_Prudential/2026-06-10 - AsnUnitsInterceptor.sql |
 | [Prudential: Dell LCH IOP transmissions (LCHRequestV2)](features/dell-lch-iop-transmissions.md) | 1.0 | Outbound order transmissions from the 1.0 worker tier to Dell's Lifecycle Hub (LCH) ITSM integration. | library/app/api/delllch.php, worker/crons/toga2/prudential/transmissions_to_dell.php, worker/crons/toga2/prudential_beta/transmissions_to_dell_india.php, worker/crons/toga2/prudential_beta/transmissions_to_dell_ireland.php, worker/crons/toga2/prudential_beta/transmissions_to_dell_usa.php |
+| [Prudential: Device information import + unit→contact linking (import_device_information.php)](features/device-information-import-and-contact-linking.md) | 1.0 | Prudential's **device-sync** cron (`worker/crons/toga2/prudential/import_device_information.php`) pulls device/asset records (from ServiceNow / the Dell CMDB fe | worker/crons/toga2/prudential/import_device_information.php, worker/crons/toga2/prudential/backfill_unit_contacts.php, dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql |
 | [Prudential: Service Request Regional Address Validation](features/service-request-address-validation.md) | 2.0 | The `prePost` interceptor on `_Model_Prudential_ServiceRequest` validates `deliverToAddress` fields differently depending on which Prudential regional customer | _underscore/Model/Prudential/ServiceRequest.php |
 | [Prudential Order Shipped Email — transmit_ordershipped_updates_prudential.php](features/transmit-ordershipped-email.md) | 1.0 | Cron script that transmits "Order Shipped" updates to ServiceNow (RITM) and sends a shipped notification email to the end user. | worker/crons/toga2/prudential/transmit_ordershipped_updates_prudential.php, worker/crons/toga2/prudential/transmit_closecomplete_updates_prudential.php, worker/crons/toga2/prudential/generate_sales_and_purchase_orders_from_service_requests.php, worker/crons/toga2/prudential_beta/generate_sales_and_purchase_orders_from_service_requests.php |
 | [Prudential Financial](profile.md) | 2.0 | Prudential is a TOGA client whose device-fulfillment flow is driven by **Dell** via the Dell API (`Client_Prudential.Apis.id = 2`). |  |

package/knowledge/clients/prudential/features/device-information-import-and-contact-linking.md

@@ -0,0 +1,151 @@
+---
+title: "Prudential: Device information import + unit→contact linking (import_device_information.php)"
+framework: "1.0"
+repo: worker
+project: Worker
+client: prudential
+type: client-feature
+status: active
+updated: 2026-07-07
+owners: ["bala"]
+files:
+  - worker/crons/toga2/prudential/import_device_information.php
+  - worker/crons/toga2/prudential/backfill_unit_contacts.php
+  - dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql
+related:
+  - ../profile.md
+  - ../../../2.0/apps/api2/features/nested-relationship-writes.md
+---
+
+## Summary
+
+Prudential's **device-sync** cron (`worker/crons/toga2/prudential/import_device_information.php`)
+pulls device/asset records (from ServiceNow / the Dell CMDB feed) and PUTs them into the 2.0
+platform (`Client_Prudential` via the TOGa2 API), writing `Units` and their owning `Contact`.
+Each unit's contact is keyed to an employee by the custom field **`c_employeeXid`**.
+
+This doc records how the cron links a unit to its real employee contact, the production incident
+that came from getting that wrong, and the two remediation pieces (a dedup migration and a
+null-contact backfill cron).
+
+## How it works
+
+- The cron loads **all** existing Prudential contacts once into an in-memory
+  **`c_employeeXid` → keeper uuid** map (`loadPrudentialContactMap`).
+- `resolvePrudentialContactUuid(<XID>)` returns the keeper contact uuid for an employee XID —
+  the **oldest named contact** for that XID. On a genuine miss it creates **exactly one**
+  contact and caches it so the same run never creates a second.
+- All four unit-write payloads link the existing contact by sending **`contact: {uuid: <keeper-uuid>}`**
+  — an identifier the api2 write engine can match — **never** `contact: {c_employeeXid: …}`.
+
+The keeper uuid comes from the resolved map, so every unit for a given employee links to the one
+real contact and no duplicate empty contacts are created.
+
+## Why `contact: {uuid}` and not `contact: {c_employeeXid}`
+
+This is the crux. In the api2 V2 write engine a nested related object is matched to an existing
+record **only** by fields flagged as identifiers (`uuid`). A plain, non-unique custom field like
+`c_employeeXid` is **not** a match key, so a nested `contact: {c_employeeXid: …}` object never
+matches an existing contact — the engine **inserts a new (empty) Contact every time** and links
+the unit to that shell. See the general behavior doc:
+[2.0 api2 — Nested-relationship writes & child matching](../../../2.0/apps/api2/features/nested-relationship-writes.md).
+
+## The production incident (root cause)
+
+The Prudential TOGa Supply contact export showed **"null null"** contact names and blank
+phone/email/XID on ~11.6k of ~27.8k rows, and searching the export by employee name or XID
+returned almost nothing.
+
+**Root cause:** the device-sync cron sent the unit's contact as a nested object containing
+**only `c_employeeXid`**. Because that field is not an identifier, api2 could not match an
+existing Contact, so it **INSERTed a new empty Contact on every run** and either linked the unit
+to that empty shell or left `Units.contactId` NULL. Verified in prod logs: two `PUT /units` for
+the same XID seconds apart each created a **different** new contact (`dtCreated = now`,
+`firstName = null`).
+
+**Blast radius in prod:** ~**893k** Contact rows for only ~**19.5k** real employees (XID
+`X208000` alone had **178k** rows), and ~**23k** Units with `contactId` NULL. The cron was
+generating roughly **5,600 duplicate empty contacts/day**.
+
+## Remediation
+
+Two independent pieces, run **in order**:
+
+1. **Cron fix (deployed to prod, live)** — the in-memory map + `contact: {uuid}` payloads above.
+   This **stops new junk**. It must be deployed before anything cleans up existing junk, or the
+   cleanup regenerates (observed on sandbox).
+2. **Dedup migration** — `dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql`
+   (tested on a full prod copy; **not yet run on prod**). Removes **existing** junk: collapses
+   Contacts to one keeper per `c_employeeXid`. See *Dedup migration* below.
+3. **Null-contact backfill cron** — `worker/crons/toga2/prudential/backfill_unit_contacts.php`
+   (logic tested; **not yet run**). Links `Units` that still have `contactId` NULL, resolving the
+   owner from **ServiceNow by serial** (SQL cannot — a unit stores no XID of its own). See
+   *Null-contact backfill* below. Run **after** the dedup migration so the contact map is small.
+
+### Dedup migration (`2026-07-07 - Contact Dedup Merge.sql`)
+
+Keeper per `c_employeeXid` = **oldest NAMED contact with a primary email**, else oldest named,
+else oldest. Rows with blank/NULL XID are **excluded** so unrelated contacts never merge. Phases:
+
+1. Re-point the **16 columns** that reference `Contacts` to the keeper.
+2. **Re-point** `ContactEmailAddresses`/`ContactPhoneNumbers.contactId` to the keeper (instead of
+   deleting them). This resolves the `Contacts.primaryContactEmailAddressId` /
+   `primaryContactPhoneNumberId` back-reference **FK cycle** — `contactId` is non-unique on those
+   tables so no collision — and preserves every primary pointer.
+3. `ContactAddresses` has `UNIQUE(contactId, addressId)`, so it is **nulled then deleted**
+   (0 cross-links observed).
+4. Delete non-keeper bridge/detail rows that have no remaining back-reference.
+5. Delete non-keeper contacts via a **materialized double-nested subquery** (avoids MySQL
+   error 1093 "can't specify target table" on Contacts-targeting statements — the same idiom
+   documented in the dbchanges2 architecture doc).
+
+Uses inline keeper subqueries — **no work table, no backup table** (the DBA takes a DB backup
+separately). Sandbox result on a full prod copy: Contacts **893,379 → 19,754**; duplicate XIDs
+**3,820 → 0**; named contacts preserved; SalesOrders/ServiceRequests/PurchaseOrders/Units counts
+unchanged and zero now point to an empty contact; ContactEmailAddresses/PhoneNumbers preserved;
+zero dangling/orphaned references.
+
+### Null-contact backfill (`backfill_unit_contacts.php`)
+
+Resumable, unit-driven. For each `Unit` with `contactId` NULL it queries ServiceNow by serial
+(`/gbts/v1/sn_pcaas/asset?serial_number=`) for the owner XID, resolves the contact uuid (reusing
+the map logic), and PUTs `/units {serialNumber, contact:{uuid}}`. Resumable via a `Parameters`
+pointer key **`SERVICE_NOW_UNIT_CONTACT_BACKFILL`** (upserts; generates a uuid4 on insert),
+time-bounded per run.
+
+**Do NOT force NULL contactId to zero.** Null-contact units are frequently **legitimate**: Dell
+has not yet assigned the laptop to an employee, so the device is unassigned inventory (ServiceNow
+returns no `ownedBy`). Verified read-only on the prod copy: of **22,960** null-contact units, only
+**26** shipped on a Toga ASN (all 26 trace to a named order contact); the other ~22,934 never went
+through a Toga order and cluster in inventory dispositions/stages (only 15 "In Use"). **Caveat:**
+the device-import cron also pulls CMDB assets never ordered through Toga, so "not on an ASN" does
+not by itself prove unassigned — definitive ownership is **ServiceNow-by-serial only**. A residual
+of no-owner units after backfill is **expected and correct**.
+
+## Gotchas
+
+- **Never link a related record by a non-identifier custom field.** `c_employeeXid` is not a match
+  key; only `uuid` (or the numeric internal id) links an existing child. Passing the custom field
+  alone silently creates a new empty record on every write.
+- **Ordering is load-bearing:** deploy the cron fix → run the dedup migration → run the backfill.
+  Running dedup before the cron fix regenerates duplicates.
+- **No `continue`** in the helpers added to this cron (team standard); the new backfill file is
+  tab-indented (the legacy cron body is space-indented — only the added helpers were converted).
+- API auth uses the `App_Api_Toga2::CLIENT_UUID_PRUDENTIAL` / `API_UUID_PRUDENTIAL` /
+  `API_SECRET_PRUDENTIAL` constants (names only — values live in config, not here).
+
+## Change history
+
+- 2026-07-07 — Root-caused the "null null" export incident to nested `contact: {c_employeeXid}`
+  writes (~893k empty Contacts, ~23k NULL-contact Units, ~5,600 dupes/day). Fixed the cron to
+  build a `c_employeeXid → keeper uuid` map and send `contact: {uuid}` (deployed to prod). Built
+  the Contact dedup merge migration (tested on prod copy, not yet run) and a ServiceNow-by-serial
+  null-contact backfill cron (logic tested, not yet run). Confirmed NULL-contact units are often
+  legitimate unassigned inventory — do not force to zero. (bala)
+
+## Related docs
+
+- [Prudential profile](../profile.md)
+- [2.0 api2 — Nested-relationship writes & child matching](../../../2.0/apps/api2/features/nested-relationship-writes.md)
+  — the general api2 behavior this incident exposed.
+- dbchanges2 architecture — *Self-referencing DELETE* idiom used by the dedup migration.

package/knowledge/clients/prudential/profile.md

@@ -12,13 +12,14 @@ project: _Underscore
 client: prudential
 type: profile
 status: active
-updated: 2026-07-06
-owners: ["jcardinal", "rgirish"]
+updated: 2026-07-07
+owners: ["jcardinal", "rgirish", "bala"]
 files: []
 related:
   - features/dell-asn-units-interceptor.md
   - features/service-request-address-validation.md
   - features/dell-lch-iop-transmissions.md
+  - features/device-information-import-and-contact-linking.md
 ---
 
 ## Summary
@@ -60,3 +61,5 @@ order-status transmissions.
 - Dell LCH IOP transmissions (LCHRequestV2, outbound 1.0).
 - 2.0 _underscore: Tracking-Number Bridge Migration.
 - features/transmit-ordershipped-email.md
+- features/device-information-import-and-contact-linking.md — device-sync cron, unit→contact
+  linking, the "null null" contact-export incident + dedup/backfill remediation.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.282",
+  "version": "1.0.283",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",