toga-ai 1.0.863 → 1.0.864

package/knowledge/2.0/apps/_underscore/features/recursive-item-fulfillments.md

@@ -6,13 +6,17 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
+updated: 2026-09-22
 owners: [jcardinal, rgirish]
 files:
   - _underscore/Model/Client/ItemFulfillment.php
   - _underscore/Model/Client/ItemFulfillmentItem.php
   - _underscore/Model/Client/ItemFulfillmentItemUnit.php
   - _underscore/Model/Client/ItemFulfillmentPackage.php
+  - _underscore/Model/Client/ItemFulfillments/TrackingNumber.php
+  - _underscore/Model/Client/ItemFulfillmentItems/TrackingNumber.php
+  - _underscore/Model/Client/ItemFulfillmentItemUnits/TrackingNumber.php
+  - dbchanges2/Core/2026-09-18a - ItemFulfillmentTrackingNumberInterceptors.sql
   - _underscore/Model/Compass/AdvanceShippingNotice.php
   - dbchanges2/Core/2026-02-13 - 75601 - RecursiveItemFulfillmentCreation.sql
   - dbchanges2/Core/2026-06-04 - RecursiveItemFulfillmentPut.sql
@@ -22,6 +26,7 @@ related:
   - ../architecture.md
   - ../../api2/architecture.md
   - ../../../../clients/compass-usa/features/asn-to-item-fulfillment.md
+  - ../../../../clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md
 ---
 
 Engine that mirrors a downstream ItemFulfillment up the SO→PO→SO chain onto every customer-facing SO; open when touching IF creation, bundle scaling, or upstream-duplicate/half-quantity bugs.
@@ -40,9 +45,14 @@ The whole engine is in `_Model_Client_ItemFulfillment`; the three child models a
 | `Model/Client/ItemFulfillmentItem.php` | `postPost`, `postPut` | resolve owning IF uuid → driver |
 | `Model/Client/ItemFulfillmentItemUnit.php` | `postPost`, `postPut` | resolve owning IF uuid → driver |
 | `Model/Client/ItemFulfillmentPackage.php` | `postPost`, `postPut` | resolve owning IF uuid → driver |
+| `Model/Client/ItemFulfillments/TrackingNumber.php` | `postPost`, `postPut` | record **317** header-tracking bridge → `triggerUpstreamSync()` → driver (added 2026-09-18) |
+| `Model/Client/ItemFulfillmentItems/TrackingNumber.php` | `postPost`, `postPut` | record **318** item-tracking bridge → driver (added 2026-09-18) |
+| `Model/Client/ItemFulfillmentItemUnits/TrackingNumber.php` | `postPost`, `postPut` | record **319** unit-tracking bridge → driver (added 2026-09-18) |
 
 Triggers live on every child (not just the IF) so the sync is robust to all paths: the ASN flow (POSTs the IF then each child individually), nested IF POSTs, and direct child edits.
 
+**⚠ The three bridge models live in SUBFOLDERS named after the parent table, not as top-level files.** The path is `Model/Client/ItemFulfillments/TrackingNumber.php` (class `_Model_Client_ItemFulfillments_TrackingNumber`) — **not** `Model/Client/ItemFulfillmentTrackingNumber.php`. Note the folder is the **plural table name** while the sibling top-level models are singular (`ItemFulfillmentItem.php`). Searching for the singular filename finds nothing and reads as "this model does not exist."
+
 ### Driver and level walk
 **Driver — `ensureUpstreamItemFulfillment($api, $ifUuid)`:** a static `$isReconciling` re-entrancy guard makes nested re-entries (from our own `internalApiRequest` writes, which fire those records' interceptors) no-op. A `while` loop walks UP one level at a time via `reconcileUpstreamLevel`, with a `$visited` set to prevent cycles.
 
@@ -78,7 +88,9 @@ Engine is shared `_Model_Client_*` logic. Compass uses **empty pass-through subc
 - **Transfer-order fulfillments are explicitly excluded from the upstream walk.** `reconcileUpstreamLevel()` returns `null` immediately when the downstream IF has `transferOrderId` set (no `salesOrderId`). Intentional for standalone TOs (GroWrk v1). When SO→TO→SO chain scenarios are needed, Phase 5 of the GroWrk transfer order plan extends this method to walk `TransferOrders_SalesOrders` bridge tables — see `clients/growrk/features/transfer-order-flow.md`.
 - **Broken SOI↔POI bridge → empty header-only duplicate IFs (root cause, guard added 2026-07-02).** The item walk maps a downstream IFI up via `IFI.salesOrderItemId → PurchaseOrderItems_SalesOrderItems → SalesOrderItems_PurchaseOrderItems`. When that upstream `SalesOrderItems_PurchaseOrderItems` bridge is **missing/broken** (the same integrity class as the ~116 off-by-one Compass orders — see the Compass `asn-to-item-fulfillment` doc), `$desiredItems` resolves to **zero** even though the downstream IF has items. Pre-guard, step 4's eager resolve-or-create then POSTed an **empty header-only upstream IF** (e.g. Compass SA133377 got duplicate empty customer-facing IFs 71724/71725 alongside the legit direct-ASN IF 70635). **Guard:** `reconcileUpstreamLevel` now, after building `$desiredItems`, if it is empty **AND** the downstream IF has >0 ItemFulfillmentItems, `error_log()`s a diagnostic and returns `null` (skips the upstream mirror) — distinguished from the legitimate header-first POST case (downstream IF genuinely has 0 items), which is unchanged.
 - **Duplicate customer-facing IF: the engine only adopts an upstream IF it created itself, never a pre-existing one (known gap, adoption DEFERRED).** `reconcileUpstreamLevel` finds an existing upstream IF only via `upstreamItemFulfillmentId`; it does **not** adopt a pre-existing IF that already sits on the upstream SO from another flow (e.g. a direct-ASN-created IF). So a broken bridge could yield two customer-facing IFs on the same SO (SA133377: direct-ASN IF 70635 + a mirror-created duplicate). **Naive adoption is dangerous and was intentionally not implemented:** step 5's delete-stale step deletes upstream IFIs with no matching downstream group, so adopting an IF while the item walk is empty/broken would **DELETE the adopted IF's legitimate line items**. Requires a CTO design review before implementing adoption semantics. The 2026-07-02 broken-bridge guard prevents the duplicate cascade for this failure mode without needing adoption.
-- **Source-side IF tracking population belongs in the NetSuite sync writer, not a 317 interceptor (design decision, CTO-reviewed 2026-08-26).** For NetSuite-integrated clients the fix that populates item/unit tracking (318/319) from a fulfillment's single header tracking number lives in the **source writer** `App_Api_Toga2::syncItemFulfillmentFromNetsuite` (see the [per-client sync doc](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)) — **not** an `_underscore` payload interceptor on `ItemFulfillments_TrackingNumbers` (317). An interceptor fires **per row** as tracking is attached, so it cannot know at insertion time whether the shipment is single- or multi-package, and it would **fight this engine**: `reconcileUpstreamUnits`/`reconcileUpstreamPackages` own upstream 319 with delete-stale logic. The source writer knows NetSuite's one tracking number and the full item/unit set up front, and this recursive engine then mirrors it upstream. A 317-interceptor approach was prototyped and **abandoned**.
+- **⚠ NARROWED 2026-09-18 — the 317/318/319 interceptors ARE now required, for a different job than the one rejected in August.** The August decision below still stands for **populating** item/unit tracking from a header number; that work belongs in the NetSuite source writer. But the bridges also need an interceptor to **trigger the upstream walk** when tracking is attached after the chain already exists, and that one was missing entirely. Records **317/318/319 had ZERO `ApiPayloadInterceptors` rows** while their parents 28/29/30 had them: header tracking used to live in `ItemFulfillmentPackages` (record 41) with its own interceptor, and when that table was consolidated into the plain bridge `ItemFulfillments_TrackingNumbers`, **the registration did not follow the data**. So `reconcileUpstreamPackages()` — correct code — never fired on a tracking POST. Cost for Compass: **1,819 stranded orders since 2026-08-01, 95% of them tracking added after the chain was built.** Fix = `postPost`/`postPut` + `triggerUpstreamSync()` on the three bridge models **plus** `dbchanges2/Core/2026-09-18a - ItemFulfillmentTrackingNumberInterceptors.sql`. The two jobs do not fight: the writer decides *what* tracking a row carries, the interceptor only re-runs the mirror. See [Compass multi-tier tracking & serial mirroring](../../../clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md). **Generalize the lesson: when a table is consolidated or replaced, audit `ApiPayloadInterceptors` for the old record id — a lost registration is silent, never an error.**
+- **Units mirror on `unitId` ONLY — never on part number.** A unit is one physical device, so the shared `unitId` is the only safe key. **32,585** `upstreamItemFulfillmentItemId` links legitimately join **different** part numbers (kit/bundle roll-ups, vendor suffixes like `C40QYUC#ABA` → `C40QYUC`, service lines). Those are correct data; a "repair" that aligns part numbers across tiers corrupts them.
+- **Source-side IF tracking population belongs in the NetSuite sync writer, not a 317 interceptor (design decision, CTO-reviewed 2026-08-26; narrowed 2026-09-18 — see above).** For NetSuite-integrated clients the fix that populates item/unit tracking (318/319) from a fulfillment's single header tracking number lives in the **source writer** `App_Api_Toga2::syncItemFulfillmentFromNetsuite` (see the [per-client sync doc](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)) — **not** an `_underscore` payload interceptor on `ItemFulfillments_TrackingNumbers` (317). An interceptor fires **per row** as tracking is attached, so it cannot know at insertion time whether the shipment is single- or multi-package, and it would **fight this engine**: `reconcileUpstreamUnits`/`reconcileUpstreamPackages` own upstream 319 with delete-stale logic. The source writer knows NetSuite's one tracking number and the full item/unit set up front, and this recursive engine then mirrors it upstream. A 317-interceptor approach was prototyped and **abandoned**.
 - **Out of scope:** NetSuite sync of upstream IFs; transfer-order fulfillments (`transferOrderId`) are skipped.
 - **Performance:** every child write re-runs a full upstream walk (read-heavy, but each upstream record is written at most once — idempotent). Fine for normal fulfillment sizes.
 
@@ -87,3 +99,8 @@ Engine is shared `_Model_Client_*` logic. Compass uses **empty pass-through subc
 - GroWrk transfer order flow plan: `clients/growrk/features/transfer-order-flow.md`.
 - `_underscore` architecture (interceptors, `internalApiRequest`, `_Model` layer).
 - `api2` architecture (V2 metadata engine that fires these interceptors; the commit-logs / rollback-data-on-failure transaction invariant the throws rely on).
+
+## Change history
+- 2026-09-22 — Narrowed the "no 317 interceptor" decision: the three tracking bridges (317/318/319) had **zero** `ApiPayloadInterceptors` rows — lost when `ItemFulfillmentPackages`/record 41 was consolidated — so the upstream walk never ran when tracking was attached after the chain existed (1,819 stranded Compass orders). Added `postPost`/`postPut` to the three bridge models + a Core registration migration. Recorded that units mirror on `unitId` only (32,585 legitimate cross-part links) and that the bridge models live in plural-named subfolders. (rgirish)
+- 2026-08-18 — Bundle scaling re-gated on distinct downstream **items** rather than downstream SOI lines, fixing same-item over-shipments mirroring as fractional quantities (Compass SA135471). (jcardinal)
+- 2026-07-02 — Added the broken-bridge guard: an empty `$desiredItems` on a downstream IF that has items now logs and skips instead of POSTing an empty header-only upstream IF (Compass SA133377 duplicates). (jcardinal)

package/knowledge/2.0/apps/_underscore/features/tracking-number-bridges.md

@@ -6,8 +6,8 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: ["jcardinal", "mhammontree", "dfranks", "apeterson", "bala", "tcox"]
+updated: 2026-09-22
+owners: ["jcardinal", "mhammontree", "dfranks", "apeterson", "bala", "tcox", "rgirish"]
 files:
   - api2/Component/Api/V2/V2.php
   - _underscore/Model/Client/AdvanceShippingNoticeItemUnit.php
@@ -27,10 +27,12 @@ files:
   - dbchanges2/Client_Quad/2026-08-26a - BackfillQuadUnitLevelTrackingSingleTn.sql
   - dbchanges2/Client_Quad/2026-08-26b - BackfillQuadItemLevelTrackingSingleTn.sql
   - dbchanges2/Client_CompassCanada/2026-08-27a - ItemFulfillmentTableViewsRebuild.sql
+  - dbchanges2/Core/2026-09-18a - ItemFulfillmentTrackingNumberInterceptors.sql
 related:
   - recursive-item-fulfillments.md
   - ../../api2/features/cxml-shipnotice-gateway.md
   - ../../../../clients/compass-canada/features/item-fulfillment-tableviews.md
+  - ../../../../clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md
 ---
 
 How tracking numbers moved from scalar FK columns to `*_TrackingNumbers` bridge tables across ASN/IF/IR; open before writing/reading tracking, choosing which bridge a client populates, or debugging 403/blank tracking columns.
@@ -87,6 +89,8 @@ All bridges are registered as **`InherentRecordChildren`** of their parent recor
 ## Gotchas
 
 - **⚠ The three ASN bridge `trackingNumberId` fields do NOT share a `childPolicy`, and a new client must be provisioned per-API.** Core defaults: **1437** (unit bridge) `MATCH_CREATE`, **1441** (item bridge) `MATCH`, **1445** (header bridge) `MATCH`. Only the unit bridge may mint a `TrackingNumbers` row on its own. A producer that posts tracking at the item or header level (the cXML ShipNotice translator does, since 2026-06-11) therefore needs `MATCH_CREATE` **overrides** in that tenant's `Apis_RecordFields` for the writing API, or the write fails **EV-12 on field `trackingNumber`**. Compass USA has all three overridden (`apiId 4`); Compass Canada had none, which rejected 100% of its ship notices for 67 days (2026-06-11 → 2026-08-20). **Add these three override rows to the onboarding checklist for any client whose ASNs arrive by cXML** — and ship them as a `dbchanges2/Client_<Tenant>/` migration, since a manual patch dies on the next reseed. Do **not** flip the Core defaults: Core ids are shared by every tenant (the same reason the IF-side 2184 flip was rejected). Mechanics: [nested-relationship writes](../../api2/features/nested-relationship-writes.md) and the [cXML ShipNotice gateway](../../api2/features/cxml-shipnotice-gateway.md).
+- **⚠ Consolidating record 41 into bridge 317 dropped its INTERCEPTOR registration, and nobody noticed for months (found 2026-09-18).** `ItemFulfillmentPackages` (record 41) had its own `Core.ApiPayloadInterceptors` rows. When it was folded into the plain bridge `ItemFulfillments_TrackingNumbers` (317), the rows did **not** follow — and 318/319 never had any either, while their parents 28/29/30 did. Interceptors are **DB-registered, never auto-discovered**, so the recursive upstream mirror simply stopped running whenever tracking was attached after the fulfillment chain existed. Compass cost: **1,819 stranded orders since 2026-08-01.** Fixed by `dbchanges2/Core/2026-09-18a - ItemFulfillmentTrackingNumberInterceptors.sql` + `postPost`/`postPut` on the three bridge models. **Rule for any future table consolidation: audit `ApiPayloadInterceptors` for the OLD record id before dropping it — a lost registration throws nothing and logs nothing.** See [recursive item fulfillments](recursive-item-fulfillments.md) and [Compass multi-tier tracking & serial mirroring](../../../clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md).
+- **The bridge models live in plural-named SUBFOLDERS.** `Model/Client/ItemFulfillments/TrackingNumber.php` → class `_Model_Client_ItemFulfillments_TrackingNumber`. There is no top-level `ItemFulfillmentTrackingNumber.php`; searching for one finds nothing and reads as "the model does not exist."
 - **Tracking is now an inherent child, not a scalar.** Any producer that sent a flat `trackingNumber` on a unit must nest it under the unit's `*TrackingNumbers` child array, or it is silently dropped.
 - **Pruning a unit/item means deleting its bridge rows FIRST — the bridge FKs are `RESTRICT`, not cascade.** The IF bridge FKs (`ItemFulfillments/Items/ItemUnits_TrackingNumbers` → parent) are `ON DELETE RESTRICT`, so deleting a parent unit/item that NetSuite dropped errors unless you delete its 317/318/319 bridge links first. The 1.0 `App_Api_Toga2::syncItemFulfillmentFromNetsuite` full reconcile does exactly this (delete bridges, then parent). Any code that prunes IF units/items must do the same ordering. See the [per-client sync doc](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md).
 - **Migration FK pitfalls (all handled in the SQL):** deleting `RecordFields` hits `DefaultRecordFieldSettings` FKs (repoint first); deleting `AclRecordPermissions` hits the `AclLogicGroups`→`AclLogicGroupExpressions` chain (record-41 teardown wraps it in `SET FOREIGN_KEY_CHECKS=0`); deleting `TableViewFields` hits `TableViews.sortPrimaryTableViewFieldId` (null it first, re-point after); self-referencing `INSERT … NOT EXISTS` on the target table errors 1093 (use a derived table); and orphaned/dangling `trackingNumberId` values fail the bridge FK 1452 (join `TrackingNumbers` so orphans become NULL / are skipped).
@@ -129,3 +133,6 @@ All bridges are registered as **`InherentRecordChildren`** of their parent recor
 - compass-usa: ASN→Item Fulfillment, and the Item Fulfillment tracking TableView.
 - prudential: Dell ASN units interceptor.
 - [Compass Canada item-fulfillment TableViews](../../../../clients/compass-canada/features/item-fulfillment-tableviews.md).
+
+## Change history
+- 2026-09-22 — Recorded that folding record 41 (`ItemFulfillmentPackages`) into bridge 317 silently dropped its `ApiPayloadInterceptors` registration, and that 318/319 never had one — breaking the recursive upstream mirror for any tracking attached after the chain was built (1,819 stranded Compass orders). Added the subfolder/class-name note for the bridge models. (rgirish)

package/knowledge/2.0/apps/dbchanges2/features/rerunnable-additive-inserts.md

@@ -6,7 +6,7 @@ project: Database Changes
 client: shared
 type: feature
 status: active
-updated: 2026-09-21
+updated: 2026-09-22
 owners: [bala, kyalamarthi, apeterson, ajean, jcardinal, rgirish]
 files:
   - dbchanges2/Client_Compass/2026-08-26a - CompassCreativeStudioPersona.sql
@@ -14,6 +14,8 @@ files:
   - dbchanges2/Client/2026-09-01a - PurchaseOrderItemQtyFieldsApiRoleRead.sql
   - dbchanges2/Core/2026-09-08e - Insert QcBatch CronJob.sql
   - dbchanges2/_modules/netsuite/2026-09-11c - ItemClassRecordPermission.sql
+  - dbchanges2/Client_Compass/2026-09-18a - BackfillUpstreamItemFulfillmentTracking.sql
+  - dbchanges2/Client_Compass/2026-09-21a - BackfillUpstreamItemFulfillmentUnits.sql
 related:
   - ../architecture.md
   - ../../worker2/features/netsuite-salesorder-open-orders-sync.md
@@ -22,6 +24,7 @@ related:
   - ../../_underscore/features/tracking-number-bridges.md
   - ../../../../clients/compass-usa/workflows/granting-persona-bundle-access.md
   - ../../../../clients/compass-usa/workflows/granting-navigation-access.md
+  - ../../../../clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md
 ---
 
 ## Summary
@@ -168,6 +171,20 @@ before you write, not after.
 - **No unique key** — a re-run **silently duplicates every row**, and nothing tells you. Always guard
   these.
 
+**Check `information_schema.STATISTICS` for a COMPOSITE unique key before every bulk INSERT** — not
+just for a single-column one, and not by reading the `CREATE TABLE` from memory. Measured 2026-09 on
+the item-fulfillment bridges, which sit side by side and do **not** agree:
+
+| Table | Composite UNIQUE? |
+|---|---|
+| `ItemFulfillments_TrackingNumbers` | yes — `(itemFulfillmentId, trackingNumberId)` |
+| `ItemFulfillmentItems_TrackingNumbers` | yes — `(itemFulfillmentItemId, trackingNumberId)` |
+| `ItemFulfillmentItemUnits_TrackingNumbers` | yes — `(itemFulfillmentItemUnitId, trackingNumberId)` |
+| `ItemFulfillmentItemUnits` | **no** — and it already holds **254** duplicate `(itemFulfillmentItemId, unitId)` rows as a result |
+
+The last row is the whole point: the table with no unique key had already accumulated duplicates
+that nobody noticed, because nothing ever errored.
+
 Both guard shapes are in use and equivalent:
 
 ```sql
@@ -320,6 +337,56 @@ phases leaves `@parentId` `NULL` and the child inserts either fail on the FK or
 in the file header, and make the unguarded parent insert the one phase you skip on a re-run (set
 `@parentId` to the existing row by hand instead).
 
+### Backfilling a multi-tier chain: run the SAME file repeatedly until the verify count is 0
+
+A backfill that walks a parent/child **chain** (Compass's Compass SO → Office Depot SO → Agilant SO
+fulfillment tiers) moves data up **exactly one tier per pass**. Section N+1 can only act on rows
+section N created — and it does not see them until the **next** run, because each section reads a
+snapshot taken before the file started. So the file is not "run once"; it is **run until the verify
+section reports 0**. Both 2026-09 Compass tracking/unit backfills needed **3 passes** each.
+
+Write it accordingly:
+
+1. Make every section a guarded no-op on re-run (see above) so extra passes are free.
+2. End the file with a **verify section** that counts the rows still needing work.
+3. Put "re-run until the verify count is 0" in the file header. Without that line, the executor runs
+   it once, sees rows inserted, and calls it done — leaving the top tier unfixed.
+
+### ⚠ `SELECT DISTINCT` is not enough when many downstream rows roll up to ONE upstream row
+
+Distinct from the uuid trap above: even with the uuid correctly generated in the outer select,
+`DISTINCT` over the **source** rows still emits the same **target** pair several times when the
+mapping is many-to-one. Compass bundles had **6** downstream items feeding **one** upstream item, so
+the same `(itemFulfillmentItemId, trackingNumberId)` pair was emitted 6 times and tripped
+`ItemFulfillmentItems_TrackingNumbers_UNIQUE_1`.
+
+**Fix: `GROUP BY` the target key pair and `MIN()` the passthrough columns.** `DISTINCT` dedupes the
+whole row; `GROUP BY` dedupes the thing the unique key actually covers.
+
+```sql
+SELECT
+	<uuid expression>,
+	src.targetId,
+	src.trackingNumberId,
+	MIN(src.somePassthrough)
+FROM ( … ) src
+GROUP BY src.targetId, src.trackingNumberId
+```
+
+### Measure the safe / ambiguous / no-match split BEFORE writing any bulk DML
+
+The methodology that worked on the 2026-09 Compass link repair, and the one to reuse: before writing
+a single `UPDATE`, run a counting query that buckets every candidate row into **safe** (exactly one
+match), **ambiguous** (more than one), and **no-match**. Then **skip the ambiguous rows rather than
+guessing** — a wrong link is far more expensive to unpick than a row left alone for a follow-up
+ticket.
+
+Worked example: 4,900 broken `upstreamItemFulfillmentItemId` links = **1,982 safe** (unique part
+match) + **0 ambiguous** + **2,918 no-match** (missing bridge rows — raised as its own ticket, not
+patched). A second pre-flight check confirmed **0** upstream rows were claimed by two downstream
+rows, so the repair could not fan out. See
+[Compass multi-tier tracking & serial mirroring](../../../../clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md).
+
 ## Dry-running a destructive migration before it goes near an environment
 
 A rebuild file (DELETE + re-INSERT, FK detaches, index renumbers) cannot be proved safe by reading
@@ -410,6 +477,7 @@ concluding a migration is production-safe.
   [FIELD_SQL calculated fields](../../_underscore/features/calculated-sql-fields.md).
 
 ## Change history
+- 2026-09-22 — Added multi-tier chain backfill rules from the Compass tracking/serial backfills: re-run the same file until the verify count is 0 (each pass moves data up one tier; both files needed 3 passes); `GROUP BY` the target key pair instead of `SELECT DISTINCT` when many downstream rows roll up to one upstream row (bundles tripped a bridge UNIQUE 6-to-1); measure the safe/ambiguous/no-match split before writing bulk DML and skip the ambiguous rows (4,900 broken links = 1,982 safe + 0 ambiguous + 2,918 no-match). Sharpened the unique-key rule with measured composite keys across the four item-fulfillment tables — `ItemFulfillmentItemUnits` has none and already carries 254 duplicates. (rgirish)
 - 2026-09-21 — Added three lessons from the platform-wide `Users.contactId` NOT NULL backfill: a guarded
   `NOT NULL` ALTER (`sql_mode ''` silently coerces leftover NULLs to `0`, so gate on a residual count —
   single-tenant aborts loud, fan-out `DO 0` skips); latin1 schema drift breaks a forced

package/knowledge/INDEX.md

@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 89 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 90 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 74 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)

package/knowledge/clients/compass-usa/INDEX.md

@@ -14,6 +14,7 @@
 | [Compass MITS PO rejection — zero line items (400 + Issue + PM alert email)](features/mits-po-zero-line-item-rejection.md) | 2.0 | Guard that rejects MITS Compass purchase orders carrying zero line items (HTTP 400 + Logs.Issue + PM alert email); open when working on the zero-item guard, its |
 | [Compass MITS Sales-Order Transmit Alerting (US Issue/Event vs Canada in-cron email)](features/mits-sales-order-transmission-alerting.md) | 2.0 | How a `1_transmit_..._sales_orders_to_mits.php` failure reaches a human — USA rejections route through Logs Issue/Event, Canada still emails in-cron; open for " |
 | [Compass MR/MA Order Auto-Approval & Status Gate](features/mr-ma-order-approval-and-status.md) | 2.0 | Compass MR/MA order auto-approval and how the `_status` approval gate strands orders; open when an order is stuck at "Pending Initial Approval" or you're tempte |
+| [Compass: multi-tier tracking & serial mirroring (3-tier drop-ship chain) and the `_trackingNumbers` field](features/multi-tier-tracking-and-serial-mirroring.md) | 2.0 | Why a Compass customer order can show no tracking or serial while the goods shipped: the 3-tier drop-ship chain, the missing 317/318/319 interceptor rows, and t |
 | [Compass ODP EDI 850 Line-Item Resolution (VA part number to IN SKU fallback)](features/odp-edi-850-item-resolution.md) | 1.0 | How each PO1 line on an inbound Office Depot EDI 850 resolves to a real Compass catalog item before cron 3a writes anything; open for ODP 850 import failures, u |
 | [Compass Office Depot EDI 855 Acknowledgement + Over-Quantity PO Guard](features/odp-edi-855-acknowledgement-and-overquantity-guard.md) | 1.0 | How Compass acknowledges ODP EDI 850 POs with an X12 855 and rejects a duplicate/over-quantity PO before import; open for 855 accept/reject behaviour, over-quan |
 | [Compass ODP EDI File Retention & Import Retry (cron 3a)](features/odp-edi-file-retention-and-retry.md) | 1.0 | When cron 3a keeps vs deletes an ODP EDI file in S3, and why an api2 outage used to silently lose orders; open for lost ODP imports, retry behaviour, or the sti |

package/knowledge/clients/compass-usa/features/item-fulfillment-tracking-tableview.md

@@ -5,8 +5,8 @@ project: _Underscore
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-09-16
-owners: ["jcardinal", "bala"]
+updated: 2026-09-22
+owners: ["jcardinal", "bala", "rgirish"]
 files:
   - dbchanges2/Client_Compass/2026-06-10 - ItemFulfillmentsForSalesOrderItemsTableView.sql
   - dbchanges2/Client_Compass/2026-06-11 - ItemFulfillmentsForSalesOrdersTableView.sql
@@ -15,6 +15,7 @@ files:
 related:
   - ../../../2.0/apps/_underscore/features/tracking-number-bridges.md
   - ../../../2.0/apps/api2/features/tableview-apiwhereclause-row-filtering.md
+  - ./multi-tier-tracking-and-serial-mirroring.md
 ---
 Two config-only Compass TableViews (ids 12 & 13) showing fulfilled items in toga2-supply, re-rooted off Units and tracking sourced at item level; open when a fulfilled-items view shows 0 rows or blank tracking, or when editing these views.
 
@@ -48,6 +49,8 @@ Stable Core ids used: records 29 IFI, 30 IFIU, 31 Units, 15 SalesOrderItems, 14
 Compass-only. The underlying bridge model is shared (see the _underscore feature doc).
 
 ## Gotchas
+
+- **⚠ A blank Tracking # here while the Sales Orders GRID shows one is a DATA problem, not a view problem (2026-09-18).** View 12 reads only `ItemFulfillmentItems_TrackingNumbers` (318) scoped to **this** order, and the Shipments panel (`toga25-supply/.../hooks/useShipments.ts`) scopes to `salesOrderId = this order` too. The grid's `_trackingNumbers()` computed field used to walk a 4-hop PO bridge out to **another** order's fulfillment, so it found tracking that genuinely was not on this order. Compass's 3-tier chain had stopped mirroring tracking upward — check that before touching the view definition. See [multi-tier tracking & serial mirroring](multi-tier-tracking-and-serial-mirroring.md).
 - **A Unit-rooted view returns 0 rows whenever the order has no unit-level breakdown.** Symptom on SA132740 (salesOrderId 105826): 13 SalesOrderItems, 7 ItemFulfillmentItems, but **0 ItemFulfillmentItemUnits / 0 Units** → view 12 showed "0 records" while re-rooted sibling view 13 showed the items fine. Fix is structural (re-root + OUTER chain), independent of the missing-unit data issue.
 - **Tracking must be sourced at the ITEM level** via `ItemFulfillmentItems_TrackingNumbers` (record 318), joined directly onto `ItemFulfillmentItems` (rec 29). The original re-root (2026-06-10/11) left tracking on the **unit-level** bridge `ItemFulfillmentItemUnits_TrackingNumbers` (record 319), which is empty for non-serialized lines (no `ItemFulfillmentItemUnits` rows) → the OUTER join yielded NULL and the tracking column rendered blank even though the data was correct. Symptom: SO **MR243437** (id 105730) showed tracking at order level but not item level; item 266006 had 0 unit rows but a valid `ItemFulfillmentItems_TrackingNumbers` row (→ TN 67248). Fixed 2026-06-15.
 - **Compass imports exactly one tracking number per item fulfillment** (NetSuite does not support multiple tracking numbers at the unit level; a different tracking number arrives as a separate item fulfillment). So the item-level bridge (318) is always the complete source and re-pointing away from the unit-level bridge (319) loses nothing.
@@ -61,3 +64,7 @@ Compass-only. The underlying bridge model is shared (see the _underscore feature
 ## Related
 - [2.0 _underscore: Tracking-Number Bridge Migration](../../../2.0/apps/_underscore/features/tracking-number-bridges.md).
 - [2.0 api2: TableView row-filtering via apiWhereClause](../../../2.0/apps/api2/features/tableview-apiwhereclause-row-filtering.md) — the mechanism used for the FEE exclusion.
+
+## Change history
+- 2026-09-22 — Added the "blank tracking here but populated in the grid" gotcha: the divergence was the upstream mirroring gap in Compass's 3-tier chain, not a view misconfiguration. (rgirish)
+- 2026-06-11 — View 12 re-rooted at `ItemFulfillmentItems` (29) with an OUTER unit/tracking chain, fixing the 0-records result for orders with no serialized units. View 13 re-rooted 2026-06-10. (jcardinal)

package/knowledge/clients/compass-usa/features/multi-tier-tracking-and-serial-mirroring.md

@@ -0,0 +1,151 @@
+---
+title: "Compass: multi-tier tracking & serial mirroring (3-tier drop-ship chain) and the `_trackingNumbers` field"
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: compass-usa
+type: client-feature
+status: active
+updated: 2026-09-22
+owners: ["rgirish"]
+files:
+  - _underscore/Model/Compass/SalesOrder.php
+  - _underscore/Model/Client/ItemFulfillments/TrackingNumber.php
+  - _underscore/Model/Client/ItemFulfillmentItems/TrackingNumber.php
+  - _underscore/Model/Client/ItemFulfillmentItemUnits/TrackingNumber.php
+  - dbchanges2/Core/2026-09-18a - ItemFulfillmentTrackingNumberInterceptors.sql
+  - dbchanges2/Client_Compass/2026-09-18a - BackfillUpstreamItemFulfillmentTracking.sql
+  - dbchanges2/Client_Compass/2026-09-21a - BackfillUpstreamItemFulfillmentUnits.sql
+related:
+  - ../../../2.0/apps/_underscore/features/recursive-item-fulfillments.md
+  - ../../../2.0/apps/_underscore/features/tracking-number-bridges.md
+  - ../../../2.0/apps/api2/features/api-payload-interceptors.md
+  - ./item-fulfillment-tracking-tableview.md
+  - ./tracking-number-origin-and-asn-link.md
+---
+Why a Compass customer order can show no tracking or serial while the goods shipped: the 3-tier drop-ship chain, the missing 317/318/319 interceptor rows, and the two backfills. Open when tracking/serials are blank on a Compass order page but the grid looks fine.
+
+## Summary
+
+Compass runs a **3-tier drop-ship chain**: **Compass SO → Office Depot SO → Agilant SO**. Agilant
+physically ships and records tracking numbers and unit serials against the *Agilant* fulfillment.
+Those must mirror **UP** to the customer-facing Compass order. Max chain depth observed in prod is
+**3** (zero rows at depth 4).
+
+The tiers are linked by `ItemFulfillments.upstreamItemFulfillmentId` and
+`ItemFulfillmentItems.upstreamItemFulfillmentItemId`. The shared
+[recursive item fulfillment engine](../../../2.0/apps/_underscore/features/recursive-item-fulfillments.md)
+does the mirroring — it is **not** Compass code.
+
+**The bug (found 2026-09-18):** the mirroring only ever ran when the fulfillment *chain* was built.
+Adding a tracking number afterwards triggered nothing, so the tracking and serials stayed on the
+Agilant tier. **1,819 stranded cases since 2026-08-01; 1,723 of them (95%) had tracking created
+AFTER the chain was built.** Compass-only — `Client_Elite` returns 0 stranded rows because it has
+no multi-tier chain.
+
+## How it works
+
+### The three tiers
+
+```
+Compass SO   (customer-facing, what the user opens)
+  ↑ upstreamItemFulfillmentId
+Office Depot SO   (intermediate)
+  ↑ upstreamItemFulfillmentId
+Agilant SO   (ships; owns the real tracking + serials)
+```
+
+Units are shared records: a unit mirrors up matched on **`unitId` only**. Tracking mirrors up via
+the three bridges — 317 `ItemFulfillments_TrackingNumbers` (header), 318
+`ItemFulfillmentItems_TrackingNumbers` (item), 319 `ItemFulfillmentItemUnits_TrackingNumbers`
+(unit). See [tracking-number bridges](../../../2.0/apps/_underscore/features/tracking-number-bridges.md).
+
+### Root cause — records 317/318/319 had ZERO interceptor rows
+
+Parent records **28/29/30** (IF / IF item / IF item unit) had `Core.ApiPayloadInterceptors` rows;
+the three tracking **bridge** records **317/318/319 had none**. Header tracking used to live in its
+own table `ItemFulfillmentPackages` (record 41) with its own interceptor; when that was consolidated
+into the plain bridge `ItemFulfillments_TrackingNumbers`, **the interceptor registration did not
+follow**. So `_Model_Client_ItemFulfillment::reconcileUpstreamPackages()` (~L691) — which is
+correct code — never fired when a tracking number was attached.
+
+### The fix (two halves — both required)
+
+1. **PHP** — `postPost` / `postPut` + a private `triggerUpstreamSync()` added to all three bridge
+   models, calling `_Model_Client_ItemFulfillment::ensureUpstreamItemFulfillment()`. Pattern copied
+   verbatim from the working sibling `Model/Client/ItemFulfillmentItem.php` (L18-46).
+2. **SQL** — `dbchanges2/Core/2026-09-18a - ItemFulfillmentTrackingNumberInterceptors.sql` registers
+   POST/POST + POST/PUT for records 317, 318, 319 (6 INSERTs). **The PHP does nothing without it**
+   — interceptors are DB-registered, never auto-discovered
+   ([api payload interceptors](../../../2.0/apps/api2/features/api-payload-interceptors.md)).
+
+### `_Model_Compass_SalesOrder::_trackingNumbers()` — the field that papered over the gap
+
+A `FIELD_SQL` computed field (`Client_Compass.CustomRecordFields` id **83**, recordId **14**). It is
+a UNION of two branches:
+
+- **ASN branch** — reads `AdvanceShippingNoticeItemUnits_TrackingNumbers`. **Load-bearing; do NOT
+  delete it.** 9,681 orders system-wide have ASN tracking that exists on **no** fulfillment.
+  Declining (658 in May 2026 → 7 in Sep 2026) but still live. Verified: SA137652 carries
+  `483164100002` only via the ASN.
+- **Fulfillment branch** — used to walk a hardcoded **4-hop PO bridge**
+  (`SalesOrders_PurchaseOrders` / `PurchaseOrders_SalesOrders`, with aliases literally named
+  `CompassSO_PO` / `CompassPO_OfficeDepotSO` / `OfficeDepotSO_PO` / `OfficeDepotPO_AgilantSO`) to
+  read tracking off **another order's** fulfillment. Simplified 2026-09-18 to a direct 3-table join
+  on **this order's own** `ItemFulfillments`.
+
+**This is why the Sales Orders grid looked right while the order page looked broken.** The grid used
+`_trackingNumbers()`, which reached across to the Agilant order; the **Shipments panel**
+(`toga25-supply/src/pages/SalesOrders/view/SalesOrderRecordModalLayout/hooks/useShipments.ts`) and
+the **Item Fulfillments modal** (TableView 12, reads only `ItemFulfillmentItems_TrackingNumbers`)
+both scope to `salesOrderId = this order`, so they showed nothing. Same order, two answers.
+
+The simplification was proven a **strict superset** across 8,336 orders: old-finds-new-misses = **0**;
+the new query finds tracking on **5,264** orders the old one missed; 1,046 differ, all supersets.
+
+## Gotchas
+
+- **⚠ The interceptor SQL is NOT run and `_underscore` is NOT deployed (as of 2026-09-22).** Until
+  both land, new shipments keep stranding at roughly **365 every 3 days**. The backfills only fix
+  history.
+- **⚠ No end-to-end test exists.** No sandbox POST has proven the new hooks actually fire. The
+  registration and the method names were verified by reading, not by running.
+- **Never match units on part number — only on `unitId`.** **32,585** `upstreamItemFulfillmentItemId`
+  links legitimately join **different** part numbers: real kitting/bundle roll-ups
+  (`PC-KIT-OPT2` → `C40QYUC`), vendor suffixes (`C40QYUC#ABA` → `C40QYUC`), and service lines
+  (`ODP-COMPASS-ASSETTAG1`). These are correct and **must not be "fixed"**. A unit is one physical
+  device, so `unitId` is the only safe key.
+- **Asset Tag is a capture gap, not a sync gap.** `Units.assetTag` is NULL on **every** tier
+  including the origin. There is nothing to mirror — chasing it through the chain is wasted work.
+  Needs its own ticket at the capture end.
+- **Open bug — broken PO↔SO item bridge (needs its own ticket).** **2,918** `ItemFulfillmentItems`
+  have no matching upstream part and cannot be linked, because their
+  `PurchaseOrderItems_SalesOrderItems` rows are **missing entirely**. Example: Office Depot SO lines
+  619922 / 619923 / 619924 have **0** bridge rows, while line 619925 wrongly maps to all 4 Compass
+  lines. Same integrity class as the off-by-one bridge breakage in
+  [asn-to-item-fulfillment](asn-to-item-fulfillment.md).
+- **Latent — `triggerUpstreamSync()` skips silently.** It returns quietly when no parent fulfillment
+  is found (`if ($itemFulfillmentUuid)`). This matches the existing sibling's behaviour but violates
+  the team's throw-don't-skip rule. Harden all four files together, not one at a time.
+
+## Backfill results (both run and complete)
+
+| Backfill | Compass customer orders fixed | Rows added |
+|---|---|---|
+| Tracking (`2026-09-18a`) | **1,651** (+4,293 Office Depot intermediate) | 7,436 header + 11,856 item |
+| Units/serials (`2026-09-21a`) | **1,096** (+1,530 Office Depot, 658 Agilant) | unit rows + their tracking bridges |
+
+Verify counts all **0**. Integrity clean: 0 duplicates, 0 orphans, 0 self-links, 0 serials leaking
+across unrelated orders. The multi-pass technique these backfills needed is written up in
+[re-runnable additive migrations](../../../2.0/apps/dbchanges2/features/rerunnable-additive-inserts.md).
+
+## Change history
+- 2026-09-22 — First write-up. Root-caused blank tracking/serials on Compass customer orders to
+  `Core.ApiPayloadInterceptors` having **zero** rows for bridge records 317/318/319 (lost when
+  `ItemFulfillmentPackages`/record 41 was consolidated into `ItemFulfillments_TrackingNumbers`), so
+  `reconcileUpstreamPackages()` never fired on tracking attach — 1,819 stranded cases since
+  2026-08-01, 95% with tracking added after the chain was built. Added `postPost`/`postPut` to the
+  three bridge models, simplified `_trackingNumbers()`'s fulfillment branch from a 4-hop PO-bridge
+  walk to a direct join on the order's own fulfillments (proven strict superset over 8,336 orders),
+  and kept the load-bearing ASN branch. Backfilled 1,651 orders' tracking and 1,096 orders' serials.
+  Core interceptor SQL still unrun and `_underscore` undeployed. (rgirish)

package/knowledge/clients/compass-usa/profile.md

@@ -20,8 +20,8 @@ project: _Underscore
 client: compass-usa
 type: profile
 status: active
-updated: 2026-09-17
-owners: [jcardinal, bala, tcox, apeterson, dfranks, akhokhani, ajean]
+updated: 2026-09-22
+owners: [jcardinal, bala, tcox, apeterson, dfranks, akhokhani, ajean, rgirish]
 files: []
 related:
   - ../../2.0/apps/_underscore/features/sales-order-po-number-sourcing.md
@@ -37,6 +37,7 @@ related:
   - features/oneuptime-ma-refresh-order-monitor.md
   - workflows/catalog-fold-reversal.md
   - features/asn-to-item-fulfillment.md
+  - features/multi-tier-tracking-and-serial-mirroring.md
   - features/order-fulfillment-status-per-line.md
   - features/cost-centers.md
   - features/isfulfillable-data-quality-and-type-rule.md
@@ -272,6 +273,7 @@ separate, related client (see its own profile).
   clean; a **live Agilant Drop Ship spike** started 2026-08-11.
   [MR/MA Order Approval & Status Gate](features/mr-ma-order-approval-and-status.md).
 - Built on the shared 2.0 Recursive Item Fulfillments engine (upstream mirroring).
+- **⚠ Compass is a THREE-tier drop-ship chain (Compass SO → Office Depot SO → Agilant SO), and tracking/serials were not mirroring up (2026-09-18, fix NOT deployed).** Agilant ships and owns the real tracking numbers and unit serials; they must mirror up to the customer-facing Compass order. They did not, because `Core.ApiPayloadInterceptors` had **zero** rows for the three tracking-bridge records 317/318/319 — lost when `ItemFulfillmentPackages` was consolidated. **1,819 stranded orders since 2026-08-01**; Elite is unaffected (no multi-tier chain). Backfills fixed 1,651 orders' tracking and 1,096 orders' serials, but the **Core interceptor SQL is still unrun and `_underscore` undeployed**, so new shipments keep stranding at ~365 every 3 days. Also explains why the Sales Orders **grid** showed tracking while the order page's Shipments panel and Item Fulfillments modal showed none. [Multi-tier tracking & serial mirroring](features/multi-tier-tracking-and-serial-mirroring.md).
 - **`Items.isFulfillable` gates the storefront Qty Fulfilled cell (2026-07).** Sourced from
   NetSuite onto the Agilant source item during the 1.0 item sync
   ([Phase 1](../../1.0/apps/library/features/netsuite-item-isfulfillable-sync.md)) and propagated

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.863",
+  "version": "1.0.864",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",