toga-ai 1.0.281 → 1.0.283

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -57,6 +57,28 @@ the originating page. Status changes are audited through `RepairHistory` (`repai
57
57
  onsite/install started → review → closed; depot repairs add parts/tracking steps. Bulk actions
58
58
  exist for closing installs and editing onsite tickets.
59
59
 
60
+ ### Status is a denormalized cache (`repair_orders.status`)
61
+ This is **shared, client-agnostic** behavior — every repair-order client is subject to it.
62
+ - Status transitions are **recorded on `repair_order_notes.newstatus`**, not written to
63
+ `repair_orders.status` directly. `Repair::addNotes`, `Repair::changeStatus`, and
64
+ `Repair::reworkRepair` (`class.repair.php`) each only create a note — none writes the
65
+ status column itself.
66
+ - **`repair_orders.status` is a denormalized CACHE**, reconciled from the latest note **only
67
+ when** `App_Model_TogaDesk_RepairOrder::updateStatus()` runs (fired by the central actions
68
+ `scheduleDispatch`, `technicianOnsite`, `technicianEnRoute`, `awaitingDispatch`,
69
+ `reworkAwaitingScheduling`, and the part flows). `updateStatus()` writes via a raw `UPDATE`
70
+ with no history row (see gotchas).
71
+ - **The LIVE status the UI shows is DERIVED, not the cache:** the grid/detail SELECTs use
72
+ `qqStatus() AS status` (`repairorder.php` ~L356), recomputing status from the latest note on
73
+ read. So the UI can show a status the cache column does not currently hold.
74
+ - **Consequence:** any consumer that reads the cached `repair_orders.status` (e.g. the worker
75
+ sync crons) diverges from the derived UI status whenever a note changed status but no
76
+ `updateStatus()` ran afterward — the consumer then acts on a **stale** status. This is
77
+ exactly how a DOE hold set via a comment failed to sync (see
78
+ [NYCDOE Hold-Status Sync](../../../../clients/nycdoe/features/hold-status-sync.md)).
79
+ - `updateStatus()`'s notification block is **enqueue-only** — it inserts dedup'd
80
+ `repair_order_notifications` rows; it sends no inline email.
81
+
60
82
  ## UPS shipment / carrier-account billing (Prepare Shipment flow)
61
83
  The "PREPARE SHIPMENT" button opens modal `desk/template/modals/central/addTracking.php`,
62
84
  whose carrier/account dropdown is built from the constant
@@ -126,6 +148,12 @@ always confirm the client arm before changing shared central logic.
126
148
  account-location map noted under *UPS shipment / carrier-account billing*.
127
149
 
128
150
  ## Change history
151
+ - 2026-07-07 — TRUE-80060 (togadesk portion): documented the durable, client-agnostic
152
+ status-cache model — status transitions land on `repair_order_notes.newstatus`;
153
+ `repair_orders.status` is a denormalized cache reconciled only by `updateStatus()`; the UI
154
+ shows the derived `qqStatus() AS status`, so cache-reading consumers (e.g. worker crons) can
155
+ act on a stale status when a note changed status without a following `updateStatus()`. Noted
156
+ `updateStatus()`'s notification block is enqueue-only. (mhammontree)
129
157
  - 2026-07-07 — TRUE-80060: documented the shared `qqStatus()`/`updateStatus()` status-recompute
130
158
  hold gates (in `library/app/model/togadesk/repairorder.php`) — silent raw-`UPDATE` overwrite,
131
159
  hold states must be gated or dropped, the six-hold-variant collapse to base `STATUS_HOLD`, and
@@ -6,6 +6,7 @@
6
6
  | [Multi-Client (Cross-Client) Data Retrieval](features/cross-client-data-retrieval.md) | A single authenticated V2 GET listing can return records across **many** clients (designed for 1000+) that the caller is entitled to, honoring **each target cli | api2/Component/Api/CrossClient/CrossClient.php, api2/Component/Api/V2/V2.php, api2/Controller/Index.php, api2/_.php, _underscore/Model/Cache/Table.php, _underscore/Model/Cache/Tables/Client.php, dbchanges2/Core/2026-06-30b - CacheClusterRegistrationAndRecordTtl.sql, dbchanges2/Cache/2026-06-30a - MultiClientCacheTables.sql |
7
7
  | [Encrypted-User-UUID Auth Handoff (/auth/encrypted-user-uuid)](features/encrypted-user-uuid-auth-handoff.md) | `POST /auth/encrypted-user-uuid` is the intended **cross-client / SSO-handoff identity mechanism**: given an encrypted `{client, user}` UUID pair, it mints a fr | api2/Component/Api/CrossClient/CrossClient.php |
8
8
  | [Language Translation Layer (audience.language + sidecar tables)](features/language-translation-layer.md) | Serves the same TOGa data (Item title/description/longDescription, expanding later) in multiple languages without forking the schema or breaking English consume | api2/Component/Api/V2/V2.php, api2/Component/Api/V2/Response/Response.php, _underscore/Model/Core/Setting.php, _underscore/Model/Core/RecordField.php, _underscore/Model/Core/DefaultGlobalSetting.php, _underscore/Model/Client/ItemTranslation.php, dbchanges2/Client/2026-06-23a - ItemTranslations.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql, dbchanges2/Core/2026-06-23a - RecordFieldsTranslationColumn.sql, dbchanges2/Core/2026-06-23b - ItemTranslationsRecord.sql |
9
+ | [Nested-relationship writes & child matching (link vs. create)](features/nested-relationship-writes.md) | When a 2.0 API write payload (`POST`/`PUT`) contains a **nested related object** (e.g. | api2/Component/Api/V2/V2.php |
9
10
  | [POST + JSON-body args for scripted APIs](features/scripted-api-post-body-args.md) | The V2 engine can run a Record Script (scripted API) for a **POST** request, and a scripted API can receive its arguments from the **JSON request body** instead | api2/Component/Api/V2/V2.php |
10
11
  | [Surface action-state via the surface=<slug> request option (M2M-safe)](features/surface-meta-option.md) | An opt-in V2 engine request option, `surface=<slug>`, that attaches per-record UI action state (`isVisible`/`isEnabled`) to a GET response **under `meta.surface | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Surface.php |
11
12
  | [Tickets API (/v2/tickets)](features/tickets-api.md) | The generic ticket endpoint of the 2.0 REST API. | Component/Api/V2/V2.php |
@@ -6,8 +6,8 @@ project: API
6
6
  client: shared
7
7
  type: architecture
8
8
  status: active
9
- updated: 2026-06-08
10
- owners: [jcardinal]
9
+ updated: 2026-07-07
10
+ owners: [jcardinal, bala]
11
11
  files:
12
12
  - api2/Controller/Index.php
13
13
  - api2/Component/Api/V2/V2.php
@@ -34,6 +34,9 @@ framework is **pulled at deploy, not vendored**. Composer deps: `sentry/sentry`,
34
34
  **Critical rules:** every action method must return the exact envelope
35
35
  `['success' => bool, 'data' => mixed, 'errors' => array]` — never a raw string or bare array.
36
36
  On error: `success=false`, `data=null`, `errors=[...]`. Deviating breaks API consumers silently.
37
+ A **nested related object** in a write payload links to an existing record **only** by an
38
+ identifier (`uuid`); a non-unique custom field (e.g. `c_employeeXid`) is not a match key and
39
+ forces a **new-record insert** every write — link with `contact: {uuid}`.
37
40
 
38
41
  ## Dependencies
39
42
 
@@ -0,0 +1,80 @@
1
+ ---
2
+ title: "Nested-relationship writes & child matching (link vs. create)"
3
+ framework: "2.0"
4
+ repo: api2
5
+ project: API
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-07-07
10
+ owners: ["bala"]
11
+ files:
12
+ - api2/Component/Api/V2/V2.php
13
+ related:
14
+ - ../architecture.md
15
+ ---
16
+
17
+ ## Summary
18
+
19
+ When a 2.0 API write payload (`POST`/`PUT`) contains a **nested related object** (e.g. a unit
20
+ payload with an embedded `contact: {…}`), the api2 V2 engine decides per nested object whether to
21
+ **link an existing** related record or **create a new one**. The rule that governs this is narrow
22
+ and easy to get wrong:
23
+
24
+ > A nested related object is matched to an existing record **only** by fields flagged as
25
+ > **identifiers** (`uuid` is the identifier field). If the object contains an identifier, the
26
+ > engine **loads and links** the existing record. If it contains no identifier field, the engine
27
+ > **creates a new child record** — even if the object carries a value (like a custom field) that
28
+ > *looks* unique to a human.
29
+
30
+ Implemented in `Component/Api/V2/V2.php` (`getForeignKeyValue()` / the nested-write matching in
31
+ `processRoutePairs()`).
32
+
33
+ ## How it works
34
+
35
+ - **`uuid` is the identifier field.** A nested object of `{uuid: <existing-uuid>}` forces a
36
+ **MATCH**: the engine loads that existing record and links the parent to it — it never creates.
37
+ (The numeric internal `id` also links an existing record.)
38
+ - **A non-unique custom field is NOT a match key.** Passing a nested object that contains only a
39
+ plain custom field — e.g. `contact: {c_employeeXid: 'X208000'}` — provides **no identifier**, so
40
+ the engine has nothing to match on and **inserts a new child record every time**. It does not
41
+ matter that `c_employeeXid` is conceptually the employee's unique key; it is not flagged as an
42
+ identifier, so the engine ignores it for matching.
43
+
44
+ ## Correct pattern
45
+
46
+ To **link** an existing related record, send its identifier:
47
+
48
+ ```
49
+ "contact": { "uuid": "<existing-contact-uuid>" } // links; never creates
50
+ ```
51
+
52
+ To **create** a new related record, send its data **without** an identifier (and accept that a new
53
+ row is written). Never rely on a non-identifier field (a custom field, a name, an email) to
54
+ de-duplicate — the engine will not use it to match, and you will accumulate duplicate empty child
55
+ rows on every write.
56
+
57
+ If you need to link by a business key (like an employee XID), **resolve that key to a uuid in your
58
+ caller first** (e.g. build a `key → uuid` map), then send `{uuid}`.
59
+
60
+ ## Gotcha
61
+
62
+ - **Nested write with only a non-identifier field silently creates duplicates.** This is a
63
+ duplicate-generating footgun: each write inserts a new empty child and the parent links to a
64
+ fresh shell (or is left with a NULL FK). It compounds on any cron/integration that writes the
65
+ same entity repeatedly. Root-caused in production on the Prudential device-sync cron, which sent
66
+ `contact: {c_employeeXid}` and generated ~893k empty Contact rows (~5,600/day). See
67
+ [Prudential device import + contact linking](../../../clients/prudential/features/device-information-import-and-contact-linking.md).
68
+
69
+ ## Change history
70
+
71
+ - 2026-07-07 — Documented that api2 nested-relationship writes match an existing child **only** by
72
+ identifier (`uuid`); a non-unique custom field is not a match key and forces a new-record insert.
73
+ Correct pattern is `contact: {uuid}` (resolve business keys to uuid in the caller). Surfaced by
74
+ the Prudential "null null" contact-export incident. (bala)
75
+
76
+ ## Related docs
77
+
78
+ - [api2 architecture](../architecture.md) — the V2 CRUD engine (`processRoutePairs`).
79
+ - [Prudential device import + contact linking](../../../clients/prudential/features/device-information-import-and-contact-linking.md)
80
+ — the production incident that exposed this behavior.
@@ -6,8 +6,8 @@ project: Database Changes
6
6
  client: shared
7
7
  type: architecture
8
8
  status: active
9
- updated: 2026-07-02
10
- owners: [jcardinal, mhammontree]
9
+ updated: 2026-07-07
10
+ owners: [jcardinal, mhammontree, bala]
11
11
  files:
12
12
  - Core/
13
13
  - Client/
@@ -233,6 +233,17 @@ query into a temporary derived table, so the outer `DELETE` no longer "sees" a l
233
233
  its own target. Used in `2026-07-02a - FixSA133377TrackingSerialAndDuplicateIF.sql` to prune
234
234
  duplicate `ItemFulfillments` rows.
235
235
 
236
+ ### FK back-reference cycle — re-point detail rows, don't delete them
237
+
238
+ When deduping a parent (e.g. `Contacts`) that has a back-reference FK to its own detail rows
239
+ (`primaryContactEmailAddressId` → `ContactEmailAddresses`, `primaryContactPhoneNumberId` →
240
+ `ContactPhoneNumbers`), deleting the non-keeper detail rows breaks the cycle. Instead re-point
241
+ `ContactEmailAddresses`/`ContactPhoneNumbers.contactId` to the keeper (safe because `contactId`
242
+ is non-unique on those tables → no collision), which preserves every primary pointer. Detail
243
+ tables with a `UNIQUE(parentId, childId)` (e.g. `ContactAddresses`) can't be re-pointed —
244
+ null-then-delete those. Then delete the non-keeper parents via the materialized double-nested
245
+ subquery. Used in `Client_Prudential/2026-07-07 - Contact Dedup Merge.sql`.
246
+
236
247
  ## Relationship to the rest of 2.0
237
248
 
238
249
  `dbchanges2` is registered as a **2.0 core repo** (`role: core` in `registry.json`) — it is
@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
5
5
  ## 1.0 framework
6
6
 
7
7
  - **library** (Library) _(framework core)_ — 11 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
8
- - **worker** (Worker) — 13 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
8
+ - **worker** (Worker) — 14 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
9
9
  - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
10
10
  - **togadesk** (TOGa Desk) — 8 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)
11
11
  - **togaview** (TOGa View) — 6 doc(s) → [1.0/apps/togaview/INDEX.md](1.0/apps/togaview/INDEX.md)
@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
19
19
 
20
20
  - **_underscore** (_Underscore) _(framework core)_ — 22 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
21
21
  - **worker2** (Worker) — 27 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
22
- - **api2** (API) — 9 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
22
+ - **api2** (API) — 10 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
23
23
  - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
24
24
  - **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
25
25
  - **saml** (SAML SSO Gateway) — 3 doc(s) → [2.0/apps/saml/INDEX.md](2.0/apps/saml/INDEX.md)
@@ -2,6 +2,6 @@
2
2
 
3
3
  | Doc | Framework | Summary | Files |
4
4
  |-----|-----------|---------|-------|
5
- | [NYCDOE Ticket Hold-Status Sync (ServiceNow ⇄ TOGaDesk)](features/hold-status-sync.md) | 1.0 | DOE ticket **hold** status must round-trip between ServiceNow (SNOW) and TOGaDesk and **stay held** — holds are SLA-bearing in both systems. | worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/send_request_item_updates.php, library/app/model/togadesk/repairorder.php, library/app/api/nycdoev2.php |
5
+ | [NYCDOE Ticket Hold-Status Sync (ServiceNow ⇄ TOGaDesk)](features/hold-status-sync.md) | 1.0 | DOE ticket **hold** status must round-trip between ServiceNow (SNOW) and TOGaDesk and **stay held** — holds are SLA-bearing in both systems. | worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/send_request_item_updates.php, library/app/model/togadesk/repairorder.php, library/app/api/nycdoev2.php, togadesk/desk/includes/classes/class.repair.php |
6
6
  | [NYCDOE ServiceNow / ASN Integration](features/servicenow-integration.md) | 1.0 | The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents + RITMs) into local tables, turns vendor shipment notices into NetSuite Sales Orde | worker/crons/sync/nycdoe/import_asn.php, worker/crons/sync/nycdoe/import_inc.php, worker/crons/sync/nycdoe/legacy_import_asn.php, worker/crons/sync/nycdoe/legacy_process_asn_queue.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/1_send_asn_to_netsuite.php, worker/crons/sync/nycdoe/2_send_serials_to_netsuite.php, worker/crons/sync/nycdoe/3_create_installation_ticket.php, worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/send_request_item_updates.php, worker/crons/sync/nycdoe/send_nycdoe_proof_of_delivery.php, worker/crons/sync/nycdoe/sync_nycdoe_locations.php, worker/crons/sync/nycdoe/receive_edi_purchase_orders.php, worker/crons/sync/nycdoe/send_edi_open_invoices.php, worker/crons/notifications/nycdoe/, worker/schedules/cron.worker.sync.json, worker/schedules/cron.worker.notification.json, library/app/api/nycdoe.php, library/app/api/nycdoev2.php, library/app/asnprocessor/manufacturer.php, library/app/asnprocessor/apple.php, library/app/asnprocessor/lenovo.php, library/app/asnprocessor/lexmark.php, library/app/asnprocessor/acer.php, library/app/edi.php |
7
7
  | [New York City Department of Education](profile.md) | 1.0 | NYC DOE (New York City Department of Education) is a TOGA client whose entire integration runs in the **1.0 worker tier** (~30 cron scripts under `worker/crons/ | |
@@ -14,6 +14,7 @@ files:
14
14
  - worker/crons/sync/nycdoe/send_request_item_updates.php
15
15
  - library/app/model/togadesk/repairorder.php
16
16
  - library/app/api/nycdoev2.php
17
+ - togadesk/desk/includes/classes/class.repair.php
17
18
  related:
18
19
  - servicenow-integration.md
19
20
  - ../profile.md
@@ -25,14 +26,14 @@ related:
25
26
  DOE ticket **hold** status must round-trip between ServiceNow (SNOW) and TOGaDesk and
26
27
  **stay held** — holds are SLA-bearing in both systems. This doc covers the bidirectional
27
28
  hold-status sync for DOE Incidents (INC) and Request Items (RITM), the authoritative
28
- business rule that governs it, and the two independent defects that let holds silently
29
- revert. It is the status-sync companion to the broader
29
+ business rule that governs it, and the three independent defects that let holds silently
30
+ revert or never reach ServiceNow. It is the status-sync companion to the broader
30
31
  [ServiceNow / ASN integration](servicenow-integration.md) doc; the underlying crons and
31
32
  `App_Api_NYCDOEV2` plumbing are documented there.
32
33
 
33
- > **Two distinct hold-revert paths — different repos, mechanisms, and revert targets.**
34
- > A held DOE order could be knocked out of hold by *either* of two unrelated code paths;
35
- > both had to be closed:
34
+ > **Three distinct hold-failure paths — different repos, mechanisms, and symptoms.**
35
+ > A held DOE order could fail to persist/propagate via *any* of three unrelated code paths;
36
+ > all three had to be closed (see *Coordinated three-repo fix & deploy ordering* below):
36
37
  > - **TRUE-79922 (worker crons):** the SNOW-side `state` 3⇄2 oscillation — an outbound
37
38
  > self-clobber pushing `state:2` over a held `state:3`, plus an inbound wrong-field read
38
39
  > — reverted the order to **"In Progress"**. Fixed; see *Change history* + the sections
@@ -43,6 +44,14 @@ revert. It is the status-sync companion to the broader
43
44
  > *Local status recompute* below. This defect is in a **shared, client-agnostic** model
44
45
  > method — surfaced by DOE but affecting every TogaDesk repair-order client; the durable
45
46
  > mechanism lives in [Field-Service Dispatch](../../../1.0/apps/togadesk/features/field-service-dispatch.md).
47
+ > - **TRUE-80060 (`togadesk` desk app):** a hold set **via a note/comment**
48
+ > (`Repair::addNotes`) never reached ServiceNow at all. `addNotes` recorded the new status
49
+ > only on `repair_order_notes.newstatus` — never writing the `repair_orders.status` cache
50
+ > nor calling `updateStatus()` — and the TRUE-75199 real-time-sync optimization suppressed
51
+ > `repair_orders.dtUpdated`. The worker reads the **cached** `repair_orders.status` and
52
+ > selects orders by `dtUpdated >= dtSynced`, so an on-hold-with-comment order left a stale
53
+ > non-hold cache **and** an un-bumped `dtUpdated` — the worker never selected it and never
54
+ > pushed `state:3`. Fixed; see *Hold set via a note/comment* below.
46
55
 
47
56
  > DOE "tickets" are **`repair_orders`** rows (`App_Model_TogaDesk_RepairOrder`,
48
57
  > `db_togadesk` / legacy `TOGaDeskSupport`, **clientid = 16**) — NOT the generic `tickets`
@@ -117,6 +126,41 @@ revert. It is the status-sync companion to the broader
117
126
  - The revert is **silent** — `updateStatus()` writes via a raw `UPDATE` with **no**
118
127
  `repair_order_history` / `repair_order_notes` entry.
119
128
 
129
+ ### Hold set via a note/comment (`addNotes`) — TRUE-80060 (togadesk)
130
+ - In TOGaDesk a status change carried on a comment goes through `Repair::addNotes`
131
+ (`togadesk/desk/includes/classes/class.repair.php`), which records the new status on
132
+ `repair_order_notes.newstatus`. Because `repair_orders.status` is a denormalized cache
133
+ reconciled only by `updateStatus()` (see
134
+ [Field-Service Dispatch](../../../1.0/apps/togadesk/features/field-service-dispatch.md)),
135
+ a hold set this way did **not** update the cache the worker reads.
136
+ - Compounding it, the **TRUE-75199** real-time comment-sync optimization suppressed
137
+ `repair_orders.dtUpdated`, so the order also failed the worker's `dtUpdated >= dtSynced`
138
+ selection — the hold never even entered the outbound cron.
139
+ - **Fix (scoped to `clientid == 16`):** (a) suppress `dtUpdated` only for a **pure comment**
140
+ (`newstatus == oldStatus`); keep it bumped when the status changed, so the worker selects
141
+ the order — the worker's work-note query filters `referenceId IS NULL`, so a
142
+ real-time-reconciled comment is not re-sent; (b) after a **status-changing** note, call
143
+ `App_Model_TogaDesk_RepairOrder::updateStatus()` to reconcile `repair_orders.status` from
144
+ the latest note. This reconcile depends on the library TRUE-80060 `qqStatus()` fix — see
145
+ deploy ordering below.
146
+
147
+ ## Coordinated three-repo fix & deploy ordering (TRUE-79922 / TRUE-80060)
148
+ Making DOE holds persist took a change in **three repos**, with a **deploy-order dependency**:
149
+ - **worker — TRUE-79922:** stop the SNOW `state` 3⇄2 oscillation (outbound self-clobber +
150
+ inbound wrong-field read). *(failure path #1 above)*
151
+ - **library — TRUE-80060:** `qqStatus()` recognizes all six `HOLD_*` on recompute so
152
+ `updateStatus()` can't demote a hold. *(failure path #2 above)*
153
+ - **togadesk — TRUE-80060:** `addNotes` reconciles the cache + keeps `dtUpdated` so a
154
+ note-set hold reaches the worker. *(failure path #3 above)*
155
+
156
+ > **DEPLOY ORDER: `library` must ship WITH or BEFORE `togadesk`.** The togadesk `addNotes`
157
+ > change calls `updateStatus()`, which relies on the library `qqStatus()` fix. Ship togadesk
158
+ > first and `updateStatus()` recomputes a base hold **out of** hold — reintroducing failure
159
+ > path #2 on every note-set hold.
160
+
161
+ The SNOW round-trip note reconciliation (delete-and-reinsert in `process_tickets`) is
162
+ **intentionally kept** as a failover for SNOW API unreliability — not removed by this work.
163
+
120
164
  ## Reference facts (DOE sync debugging)
121
165
 
122
166
  - **SNOW request/response logs:** logged to the legacy `Logs` schema, **`API` table**
@@ -146,21 +190,42 @@ revert. It is the status-sync companion to the broader
146
190
  - **Diagnosing which writer dropped a hold:** `repair_order_history`
147
191
  (`repairid, userid, dtStamp, note`) records *user-driven* status changes. A hold reverting
148
192
  with **no** history row between the hold and the revert = a raw-`UPDATE` writer
149
- (`updateStatus`/`qqStatus`), **not** the note/reply path (`class.repair.php::addNotes` syncs
150
- replies to SNOW but never writes `repair_orders.status`) and **not** the worker inbound cron
151
- (`process_tickets.php` contains no reference to `ORDER_ASSIGNED_AWAITING_SCHEDULING`). This
152
- is how TRUE-80060 was pinned to `qqStatus()` rather than the SNOW round-trip.
193
+ (`updateStatus`/`qqStatus`), **not** the worker inbound cron (`process_tickets.php` contains
194
+ no reference to `ORDER_ASSIGNED_AWAITING_SCHEDULING`). Note that **since TRUE-80060** a
195
+ *status-changing* `class.repair.php::addNotes` now **calls** `updateStatus()` (so it too can
196
+ move the cache via that raw `UPDATE`); a **pure comment** still writes nothing to
197
+ `repair_orders.status`. This history-gap technique is how TRUE-80060 was pinned to
198
+ `qqStatus()` rather than the SNOW round-trip.
153
199
  - **KNOWN REMAINING GAP (needs a follow-up ticket):** both `qqStatus()` hold gates live inside
154
200
  the `type IN (ONSITE_REPAIR, ONSITE_SERVICE)` branch. The `DEPOT_REPAIR` branch
155
201
  (`repairorder.php` from ~L799) has **no hold gate at all**, so a `DEPOT_REPAIR` order set to
156
202
  any hold status is still silently recomputed out of hold by `updateStatus()` — the same
157
203
  class of bug, untouched by TRUE-80060.
204
+ - **KNOWN REMAINING GAP (needs a follow-up ticket) — inbound note reconciliation is paged too
205
+ small.** `process_tickets.php`'s inbound comment reconciliation fetches only `page_size=50,
206
+ page_number=1` of the SNOW journal, then **deletes any local note not in that set** (the
207
+ remainder-delete ~L1326–1330). An order with **>50** comments/work_notes can have local
208
+ notes silently deleted. It is also inconsistent with `addNotes`, which fetches
209
+ `page_size=100`.
210
+ - **KNOWN REMAINING GAP — `buildDoubleFieldArray` collapses unsynced notes.** In
211
+ `process_tickets`, `buildDoubleFieldArray` keys on `IFNULL(referenceId,0)`, so **all**
212
+ NULL-`referenceId` notes collapse to key `'0'`, making the remainder-delete handle multiple
213
+ unsynced notes inconsistently.
158
214
  - `qqStatus()` collapses all recognized hold sub-statuses to base `HOLD` in
159
215
  `repair_orders.status`; the specific hold **sub-reason persists only in
160
216
  `repair_order_notes.newstatus`**. Any status badge / report / filter reading
161
217
  `repair_orders.status` directly cannot distinguish the six hold variants after a recompute.
162
218
 
163
219
  ## Change history
220
+ - 2026-07-07 — TRUE-80060 (togadesk portion): closed the **third** DOE hold-failure path — a
221
+ hold set via a note/comment (`Repair::addNotes`) never reaching SNOW. `addNotes` recorded
222
+ status only on `repair_order_notes.newstatus` (never the `repair_orders.status` cache the
223
+ worker reads), and the TRUE-75199 optimization suppressed `dtUpdated`, so the worker's
224
+ `dtUpdated >= dtSynced` selection skipped it. Fix (clientid 16): suppress `dtUpdated` only
225
+ for a pure comment; call `updateStatus()` after a status-changing note. Documented the
226
+ coordinated three-repo fix + `library`-before-`togadesk` deploy order, and two new
227
+ `process_tickets` reconciliation gaps (page_size=50 remainder-delete; `IFNULL(referenceId,0)`
228
+ collapse). php -l clean; php-reviewer 0 blocking. (mhammontree)
164
229
  - 2026-07-07 — TRUE-80060: closed the **second** DOE hold-revert path — the local
165
230
  `qqStatus()` recompute (in the shared model `library/app/model/togadesk/repairorder.php`)
166
231
  silently demoting held orders to `ORDER_ASSIGNED_AWAITING_SCHEDULING` whenever
@@ -4,11 +4,12 @@ framework: "1.0"
4
4
  apps:
5
5
  - worker
6
6
  - library
7
+ - togadesk
7
8
  project: Worker
8
9
  client: nycdoe
9
10
  type: profile
10
11
  status: active
11
- updated: 2026-06-10
12
+ updated: 2026-07-07
12
13
  owners: [mhammontree]
13
14
  files: []
14
15
  related:
@@ -4,6 +4,7 @@
4
4
  |-----|-----------|---------|-------|
5
5
  | [Prudential: Dell ASN units PRE/POST interceptor (legacy key + flat tracking)](features/dell-asn-units-interceptor.md) | 2.0 | After the tracking-number bridge migration, the ASN unit route was renamed (`advance-shipping-notice-units` → `advance-shipping-notice-item-units`), so the inhe | _underscore/Model/Prudential/AdvanceShippingNotice.php, dbchanges2/Client_Prudential/2026-06-10 - AsnUnitsInterceptor.sql |
6
6
  | [Prudential: Dell LCH IOP transmissions (LCHRequestV2)](features/dell-lch-iop-transmissions.md) | 1.0 | Outbound order transmissions from the 1.0 worker tier to Dell's Lifecycle Hub (LCH) ITSM integration. | library/app/api/delllch.php, worker/crons/toga2/prudential/transmissions_to_dell.php, worker/crons/toga2/prudential_beta/transmissions_to_dell_india.php, worker/crons/toga2/prudential_beta/transmissions_to_dell_ireland.php, worker/crons/toga2/prudential_beta/transmissions_to_dell_usa.php |
7
+ | [Prudential: Device information import + unit→contact linking (import_device_information.php)](features/device-information-import-and-contact-linking.md) | 1.0 | Prudential's **device-sync** cron (`worker/crons/toga2/prudential/import_device_information.php`) pulls device/asset records (from ServiceNow / the Dell CMDB fe | worker/crons/toga2/prudential/import_device_information.php, worker/crons/toga2/prudential/backfill_unit_contacts.php, dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql |
7
8
  | [Prudential: Service Request Regional Address Validation](features/service-request-address-validation.md) | 2.0 | The `prePost` interceptor on `_Model_Prudential_ServiceRequest` validates `deliverToAddress` fields differently depending on which Prudential regional customer | _underscore/Model/Prudential/ServiceRequest.php |
8
9
  | [Prudential Order Shipped Email — transmit_ordershipped_updates_prudential.php](features/transmit-ordershipped-email.md) | 1.0 | Cron script that transmits "Order Shipped" updates to ServiceNow (RITM) and sends a shipped notification email to the end user. | worker/crons/toga2/prudential/transmit_ordershipped_updates_prudential.php, worker/crons/toga2/prudential/transmit_closecomplete_updates_prudential.php, worker/crons/toga2/prudential/generate_sales_and_purchase_orders_from_service_requests.php, worker/crons/toga2/prudential_beta/generate_sales_and_purchase_orders_from_service_requests.php |
9
10
  | [Prudential Financial](profile.md) | 2.0 | Prudential is a TOGA client whose device-fulfillment flow is driven by **Dell** via the Dell API (`Client_Prudential.Apis.id = 2`). | |
@@ -0,0 +1,151 @@
1
+ ---
2
+ title: "Prudential: Device information import + unit→contact linking (import_device_information.php)"
3
+ framework: "1.0"
4
+ repo: worker
5
+ project: Worker
6
+ client: prudential
7
+ type: client-feature
8
+ status: active
9
+ updated: 2026-07-07
10
+ owners: ["bala"]
11
+ files:
12
+ - worker/crons/toga2/prudential/import_device_information.php
13
+ - worker/crons/toga2/prudential/backfill_unit_contacts.php
14
+ - dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql
15
+ related:
16
+ - ../profile.md
17
+ - ../../../2.0/apps/api2/features/nested-relationship-writes.md
18
+ ---
19
+
20
+ ## Summary
21
+
22
+ Prudential's **device-sync** cron (`worker/crons/toga2/prudential/import_device_information.php`)
23
+ pulls device/asset records (from ServiceNow / the Dell CMDB feed) and PUTs them into the 2.0
24
+ platform (`Client_Prudential` via the TOGa2 API), writing `Units` and their owning `Contact`.
25
+ Each unit's contact is keyed to an employee by the custom field **`c_employeeXid`**.
26
+
27
+ This doc records how the cron links a unit to its real employee contact, the production incident
28
+ that came from getting that wrong, and the two remediation pieces (a dedup migration and a
29
+ null-contact backfill cron).
30
+
31
+ ## How it works
32
+
33
+ - The cron loads **all** existing Prudential contacts once into an in-memory
34
+ **`c_employeeXid` → keeper uuid** map (`loadPrudentialContactMap`).
35
+ - `resolvePrudentialContactUuid(<XID>)` returns the keeper contact uuid for an employee XID —
36
+ the **oldest named contact** for that XID. On a genuine miss it creates **exactly one**
37
+ contact and caches it so the same run never creates a second.
38
+ - All four unit-write payloads link the existing contact by sending **`contact: {uuid: <keeper-uuid>}`**
39
+ — an identifier the api2 write engine can match — **never** `contact: {c_employeeXid: …}`.
40
+
41
+ The keeper uuid comes from the resolved map, so every unit for a given employee links to the one
42
+ real contact and no duplicate empty contacts are created.
43
+
44
+ ## Why `contact: {uuid}` and not `contact: {c_employeeXid}`
45
+
46
+ This is the crux. In the api2 V2 write engine a nested related object is matched to an existing
47
+ record **only** by fields flagged as identifiers (`uuid`). A plain, non-unique custom field like
48
+ `c_employeeXid` is **not** a match key, so a nested `contact: {c_employeeXid: …}` object never
49
+ matches an existing contact — the engine **inserts a new (empty) Contact every time** and links
50
+ the unit to that shell. See the general behavior doc:
51
+ [2.0 api2 — Nested-relationship writes & child matching](../../../2.0/apps/api2/features/nested-relationship-writes.md).
52
+
53
+ ## The production incident (root cause)
54
+
55
+ The Prudential TOGa Supply contact export showed **"null null"** contact names and blank
56
+ phone/email/XID on ~11.6k of ~27.8k rows, and searching the export by employee name or XID
57
+ returned almost nothing.
58
+
59
+ **Root cause:** the device-sync cron sent the unit's contact as a nested object containing
60
+ **only `c_employeeXid`**. Because that field is not an identifier, api2 could not match an
61
+ existing Contact, so it **INSERTed a new empty Contact on every run** and either linked the unit
62
+ to that empty shell or left `Units.contactId` NULL. Verified in prod logs: two `PUT /units` for
63
+ the same XID seconds apart each created a **different** new contact (`dtCreated = now`,
64
+ `firstName = null`).
65
+
66
+ **Blast radius in prod:** ~**893k** Contact rows for only ~**19.5k** real employees (XID
67
+ `X208000` alone had **178k** rows), and ~**23k** Units with `contactId` NULL. The cron was
68
+ generating roughly **5,600 duplicate empty contacts/day**.
69
+
70
+ ## Remediation
71
+
72
+ Two independent pieces, run **in order**:
73
+
74
+ 1. **Cron fix (deployed to prod, live)** — the in-memory map + `contact: {uuid}` payloads above.
75
+ This **stops new junk**. It must be deployed before anything cleans up existing junk, or the
76
+ cleanup regenerates (observed on sandbox).
77
+ 2. **Dedup migration** — `dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql`
78
+ (tested on a full prod copy; **not yet run on prod**). Removes **existing** junk: collapses
79
+ Contacts to one keeper per `c_employeeXid`. See *Dedup migration* below.
80
+ 3. **Null-contact backfill cron** — `worker/crons/toga2/prudential/backfill_unit_contacts.php`
81
+ (logic tested; **not yet run**). Links `Units` that still have `contactId` NULL, resolving the
82
+ owner from **ServiceNow by serial** (SQL cannot — a unit stores no XID of its own). See
83
+ *Null-contact backfill* below. Run **after** the dedup migration so the contact map is small.
84
+
85
+ ### Dedup migration (`2026-07-07 - Contact Dedup Merge.sql`)
86
+
87
+ Keeper per `c_employeeXid` = **oldest NAMED contact with a primary email**, else oldest named,
88
+ else oldest. Rows with blank/NULL XID are **excluded** so unrelated contacts never merge. Phases:
89
+
90
+ 1. Re-point the **16 columns** that reference `Contacts` to the keeper.
91
+ 2. **Re-point** `ContactEmailAddresses`/`ContactPhoneNumbers.contactId` to the keeper (instead of
92
+ deleting them). This resolves the `Contacts.primaryContactEmailAddressId` /
93
+ `primaryContactPhoneNumberId` back-reference **FK cycle** — `contactId` is non-unique on those
94
+ tables so no collision — and preserves every primary pointer.
95
+ 3. `ContactAddresses` has `UNIQUE(contactId, addressId)`, so it is **nulled then deleted**
96
+ (0 cross-links observed).
97
+ 4. Delete non-keeper bridge/detail rows that have no remaining back-reference.
98
+ 5. Delete non-keeper contacts via a **materialized double-nested subquery** (avoids MySQL
99
+ error 1093 "can't specify target table" on Contacts-targeting statements — the same idiom
100
+ documented in the dbchanges2 architecture doc).
101
+
102
+ Uses inline keeper subqueries — **no work table, no backup table** (the DBA takes a DB backup
103
+ separately). Sandbox result on a full prod copy: Contacts **893,379 → 19,754**; duplicate XIDs
104
+ **3,820 → 0**; named contacts preserved; SalesOrders/ServiceRequests/PurchaseOrders/Units counts
105
+ unchanged and zero now point to an empty contact; ContactEmailAddresses/PhoneNumbers preserved;
106
+ zero dangling/orphaned references.
107
+
108
+ ### Null-contact backfill (`backfill_unit_contacts.php`)
109
+
110
+ Resumable, unit-driven. For each `Unit` with `contactId` NULL it queries ServiceNow by serial
111
+ (`/gbts/v1/sn_pcaas/asset?serial_number=`) for the owner XID, resolves the contact uuid (reusing
112
+ the map logic), and PUTs `/units {serialNumber, contact:{uuid}}`. Resumable via a `Parameters`
113
+ pointer key **`SERVICE_NOW_UNIT_CONTACT_BACKFILL`** (upserts; generates a uuid4 on insert),
114
+ time-bounded per run.
115
+
116
+ **Do NOT force NULL contactId to zero.** Null-contact units are frequently **legitimate**: Dell
117
+ has not yet assigned the laptop to an employee, so the device is unassigned inventory (ServiceNow
118
+ returns no `ownedBy`). Verified read-only on the prod copy: of **22,960** null-contact units, only
119
+ **26** shipped on a Toga ASN (all 26 trace to a named order contact); the other ~22,934 never went
120
+ through a Toga order and cluster in inventory dispositions/stages (only 15 "In Use"). **Caveat:**
121
+ the device-import cron also pulls CMDB assets never ordered through Toga, so "not on an ASN" does
122
+ not by itself prove unassigned — definitive ownership is **ServiceNow-by-serial only**. A residual
123
+ of no-owner units after backfill is **expected and correct**.
124
+
125
+ ## Gotchas
126
+
127
+ - **Never link a related record by a non-identifier custom field.** `c_employeeXid` is not a match
128
+ key; only `uuid` (or the numeric internal id) links an existing child. Passing the custom field
129
+ alone silently creates a new empty record on every write.
130
+ - **Ordering is load-bearing:** deploy the cron fix → run the dedup migration → run the backfill.
131
+ Running dedup before the cron fix regenerates duplicates.
132
+ - **No `continue`** in the helpers added to this cron (team standard); the new backfill file is
133
+ tab-indented (the legacy cron body is space-indented — only the added helpers were converted).
134
+ - API auth uses the `App_Api_Toga2::CLIENT_UUID_PRUDENTIAL` / `API_UUID_PRUDENTIAL` /
135
+ `API_SECRET_PRUDENTIAL` constants (names only — values live in config, not here).
136
+
137
+ ## Change history
138
+
139
+ - 2026-07-07 — Root-caused the "null null" export incident to nested `contact: {c_employeeXid}`
140
+ writes (~893k empty Contacts, ~23k NULL-contact Units, ~5,600 dupes/day). Fixed the cron to
141
+ build a `c_employeeXid → keeper uuid` map and send `contact: {uuid}` (deployed to prod). Built
142
+ the Contact dedup merge migration (tested on prod copy, not yet run) and a ServiceNow-by-serial
143
+ null-contact backfill cron (logic tested, not yet run). Confirmed NULL-contact units are often
144
+ legitimate unassigned inventory — do not force to zero. (bala)
145
+
146
+ ## Related docs
147
+
148
+ - [Prudential profile](../profile.md)
149
+ - [2.0 api2 — Nested-relationship writes & child matching](../../../2.0/apps/api2/features/nested-relationship-writes.md)
150
+ — the general api2 behavior this incident exposed.
151
+ - dbchanges2 architecture — *Self-referencing DELETE* idiom used by the dedup migration.
@@ -12,13 +12,14 @@ project: _Underscore
12
12
  client: prudential
13
13
  type: profile
14
14
  status: active
15
- updated: 2026-07-06
16
- owners: ["jcardinal", "rgirish"]
15
+ updated: 2026-07-07
16
+ owners: ["jcardinal", "rgirish", "bala"]
17
17
  files: []
18
18
  related:
19
19
  - features/dell-asn-units-interceptor.md
20
20
  - features/service-request-address-validation.md
21
21
  - features/dell-lch-iop-transmissions.md
22
+ - features/device-information-import-and-contact-linking.md
22
23
  ---
23
24
 
24
25
  ## Summary
@@ -60,3 +61,5 @@ order-status transmissions.
60
61
  - Dell LCH IOP transmissions (LCHRequestV2, outbound 1.0).
61
62
  - 2.0 _underscore: Tracking-Number Bridge Migration.
62
63
  - features/transmit-ordershipped-email.md
64
+ - features/device-information-import-and-contact-linking.md — device-sync cron, unit→contact
65
+ linking, the "null null" contact-export incident + dedup/backfill remediation.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.281",
3
+ "version": "1.0.283",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",