toga-ai 1.0.64 → 1.0.66

package/knowledge/1.0/apps/worker/INDEX.md

@@ -4,3 +4,4 @@
 |-----|---------|-------|
 | [Worker (1.0 Framework) Architecture](architecture.md) | `worker` is the legacy (**1.0** `App_` framework) **background-job tier**. | worker/index.php, worker/_/app/framework.php, worker/crons/, worker/schedules/, worker/ebs/cron.worker.php, worker/.ebextensions/035_cron.worker.config |
 | [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/reconcile_netsuite_totals.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
+| [Prudential: Send Shipments for the Day report (daily cron)](features/send-shipments-for-the-day.md) | Daily cron (9:00 PM) that emails Prudential and Dell stakeholders an Excel report of all devices shipped that day, including tracking number, serial number, emp | worker/crons/notifications/reports/send_shipments_for_the_day.php |

package/knowledge/1.0/apps/worker/features/send-shipments-for-the-day.md

@@ -0,0 +1,65 @@
+---
+title: "Prudential: Send Shipments for the Day report (daily cron)"
+framework: "1.0"
+repo: worker
+project: Worker
+client: prudential
+type: feature
+status: active
+updated: 2026-06-12
+owners: ["rgirish"]
+files:
+  - worker/crons/notifications/reports/send_shipments_for_the_day.php
+related:
+  - ../../../clients/prudential/features/dell-asn-units-interceptor.md
+---
+
+## Summary
+
+Daily cron (9:00 PM) that emails Prudential and Dell stakeholders an Excel report of all
+devices shipped that day, including tracking number, serial number, employee name/XID, item
+description, FedEx status, and vendor.
+
+## Key files / entry points
+
+- `crons/notifications/reports/send_shipments_for_the_day.php` — self-contained cron script.
+  Scheduled via `schedules/cron.worker.notification.json`.
+
+## How it works
+
+1. Queries `Client_Prudential` DB joining `AdvanceShippingNotices` → items → units → tracking
+   bridge → `TrackingNumbers` → `Units` → `Items` → `Contacts`, filtered to `dateShipped = CURDATE()`.
+2. For each unit, calls `App_Api_Carrier_FedExApi::getStatus()` to get live carrier status.
+3. Builds an `.xlsx` workbook (PhpSpreadsheet) with 8 columns: Name, Employee XID, Device Serial
+   Number, Description, Tracking Number, Tracking Number Status, Shipping Carrier Updates, Vendor Name.
+4. Emails the file to Prudential/Dell contacts + BCC to internal ops.
+5. If no shipments found, sends a "No Shipments for the Day" notification instead.
+
+## Data model (join chain)
+
+```
+AdvanceShippingNotices
+  → AdvanceShippingNoticeItems       (advanceShippingNoticeId)
+  → PurchaseOrders                   (AdvanceShippingNotices.purchaseOrderId)
+  → SalesOrders_PurchaseOrders       (purchaseOrderId)
+  → SalesOrders                      (salesOrderId)
+  → Vendors                          (PurchaseOrders.vendorId)
+  → AdvanceShippingNoticeItemUnits   (advanceShippingNoticeItemId)
+  → AdvanceShippingNoticeItemUnits_TrackingNumbers  (advanceShippingNoticeItemUnitId)
+  → TrackingNumbers                  (trackingNumberId)
+  → Units                            (AdvanceShippingNoticeItemUnits.unitId)
+  → Items                            (Units.itemId)
+  LEFT JOIN Contacts                 (SalesOrders.contactId)
+```
+
+## Gotchas
+- The join chain changed in Jun 2026 when `AdvanceShippingNoticeUnits` was renamed to
+  `AdvanceShippingNoticeItemUnits` and tracking was moved to the bridge table
+  `AdvanceShippingNoticeItemUnits_TrackingNumbers`. If the query returns no rows unexpectedly,
+  verify these two tables exist and have data — they are the migration point.
+- `GROUP BY AdvanceShippingNotices.id` is intentional — one row per ASN even if multi-unit.
+  The report shows one tracking number per ASN row (the first joined); this is by design.
+
+## Change history
+- 2026-06-12 — Updated join chain: `AdvanceShippingNoticeUnits` → `AdvanceShippingNoticeItemUnits`
+  + added `AdvanceShippingNoticeItemUnits_TrackingNumbers` bridge join after DB migration. (rgirish)

package/knowledge/INDEX.md

@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 1.0 framework
 
 - **library** (Library) _(framework core)_ — 4 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
-- **worker** (Worker) — 3 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
+- **worker** (Worker) — 4 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 
 ## 2.0 framework
 

package/knowledge/clients/compass-usa/INDEX.md

@@ -6,3 +6,4 @@
 | [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two sibling Compass TableViews in `Client_Compass` display fulfilled items in toga2-supply, both driven by `TableViews` / `TableViewJoins` / `TableViewFields` c | dbchanges2/Client_Compass/2026-06-10 - ItemFulfillmentsForSalesOrderItemsTableView.sql, dbchanges2/Client_Compass/2026-06-11 - ItemFulfillmentsForSalesOrdersTableView.sql |
 | [Compass MITS PO → SO Item Linking](features/mits-po-to-so-item-linking.md) | 2.0 | MITS sends Compass inbound Purchase Orders (`POST /v2/purchase-orders`) against a Sales Order (`mitsSalesOrder`). | _underscore/Model/Compass/PurchaseOrder.php |
 | [Compass USA](profile.md) | 2.0 | Compass USA is a TOGA client running a multi-tier supply-chain commerce operation. |  |
+| [Compass Order Lifecycle & Data-Integrity Invariants](workflows/order-lifecycle-and-data-integrity.md) | 2.0 | End-to-end map of how a Compass order flows through the `Client_Compass` (2.0) database and the **expected raw-data shape** at each link/ASN/IF level. |  |

package/knowledge/clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md

@@ -0,0 +1,117 @@
+---
+title: Compass Order Lifecycle & Data-Integrity Invariants
+framework: "2.0"
+project: _Underscore
+client: compass-usa
+type: workflow
+status: active
+updated: 2026-06-12
+owners: ["jcardinal"]
+files: []
+related:
+  - clients/compass-usa/profile.md
+  - clients/compass-usa/features/asn-to-item-fulfillment.md
+  - clients/compass-usa/features/mits-po-to-so-item-linking.md
+  - clients/compass-usa/features/item-fulfillment-tracking-tableview.md
+---
+
+## Summary
+
+End-to-end map of how a Compass order flows through the `Client_Compass` (2.0) database and the
+**expected raw-data shape** at each link/ASN/IF level. This is the reference for diagnosing and
+retroactively repairing the order data, where 50+ historical bugs have left links, ASNs, and item
+fulfillments out of the shape the workflow should produce. All facts below were verified against the
+live `prod` / `Client_Compass` schema.
+
+## The chain (supply direction, by customerId / vendorId)
+
+```
+Compass customer SO  (SalesOrders.customerId = 2, number SA/MA/MR)
+  --SalesOrders_PurchaseOrders-->     MITS PO   (PurchaseOrders.customerId = 2, vendorId IN (1,26,25))
+    --PurchaseOrders_SalesOrders-->   Office Depot SO (customerId = 1)
+      --SalesOrders_PurchaseOrders--> Office Depot PO (vendorId = 1)
+        --PurchaseOrders_SalesOrders--> Agilant/NetSuite SO (customerId = 3)
+```
+
+- **SA** orders are user-created in Compass then sent to MITS; **MR** orders are created
+  retroactively from the inbound MITS PO (they do not originate as Compass-entered SOs); MA exists too.
+  Structurally MR uses the same bridges as SA.
+- One Compass SO item can split into multiple PO items (bundles / `c_isConfiguration`,
+  `parentSalesOrderItemId`, `bundleId`).
+
+## Item-link bridges & the purity rule
+
+- `SalesOrderItems_PurchaseOrderItems(salesOrderItemId, purchaseOrderItemId)` — a SO item → its PO item.
+- `PurchaseOrderItems_SalesOrderItems(purchaseOrderItemId, salesOrderItemId)` — a PO item → its downstream SO item.
+- Item identity: `PurchaseOrderItems.vendorItemId → VendorItems.itemId`. **`PurchaseOrderItems` has NO
+  `salesOrderItemId` column** — POI↔SOI is resolvable ONLY through these two bridges.
+- **PURE link ⇔ `SalesOrderItems.itemId = linked POI's VendorItems.itemId`. SPURIOUS ⇔ they differ.**
+  Spurious links (the lineNumber-match bug, see mits-po-to-so-item-linking) cause phantom over-fulfillment.
+- **Many-to-one is legitimate**, not spurious — several SOIs may link to one POI
+  (e.g. `PurchaseOrderItems_SalesOrderItems.purchaseOrderItemId = 174808` has 6 SOIs). Never treat
+  multiplicity as corruption; the only spuriousness test is the itemId mismatch.
+
+## Two fulfillment flows (different expected IF shapes)
+
+- **Flow A — direct ASN→IF** (MITS PO vendor in {25,26}, e.g. Strategic Systems; no Agilant leg):
+  an ASN against the MITS PO produces an ItemFulfillment on the Compass SO. Serialized units come
+  from the ASN units.
+- **Flow B — Office Depot / NetSuite** (MITS PO vendor 1, with ODP SO/PO + Agilant SO): the
+  fulfillment **originates on the Agilant IF** (created by NetSuite sync; `c_netsuiteInternalItemFulfillmentId`
+  set) and **mirrors UP the chain**. Units come from NetSuite inventory assignment, NOT the ASN
+  (ODP ASN units are frequently tracking-only, `unitId` NULL).
+
+### Upstream mirror direction (verified from real rows)
+The **Compass-SO IF is the TOP** (`upstreamItemFulfillmentId = NULL`). The ODP IF's
+`upstreamItemFulfillmentId` points to the Compass IF; the Agilant IF's points to the ODP IF. The
+back-link chain climbs **Agilant → ODP → Compass**. Item back-links use
+`ItemFulfillmentItems.upstreamItemFulfillmentItemId` the same direction and may be many-downstream →
+one-upstream for bundles.
+
+## Expected raw-data invariants (for detect-and-repair)
+
+1. **Links:** every SOI that should be procured has exactly one PURE bridge to the matching POI on the
+   correct PO; spurious (itemId-mismatch) bridges should not exist; the full header chain
+   (SalesOrders_PurchaseOrders / PurchaseOrders_SalesOrders) should be intact at each present layer.
+2. **Never over-fulfill:** `SUM(ItemFulfillmentItems.quantity)` per SOI must be `≤ SalesOrderItems.quantity`.
+   A shipped POI fanning out to many linked SOIs must not produce IF items totaling more than ordered.
+3. **ASN/IF tracking is maintained at all three levels and rolls up:** a tracking number on a unit must
+   also be linked at the item level and the header level — on BOTH the ASN side
+   (`AdvanceShippingNoticeItemUnits_TrackingNumbers` → `AdvanceShippingNoticeItems_TrackingNumbers` →
+   `AdvanceShippingNotices_TrackingNumbers`) and the IF side
+   (`ItemFulfillmentItemUnits_TrackingNumbers` → `ItemFulfillmentItems_TrackingNumbers` →
+   `ItemFulfillments_TrackingNumbers`). Always link existing `TrackingNumbers`; never create them.
+4. **Legacy stage:** a shipped IF (has items) with `itemFulfillmentStageId IS NULL` should be set to 3
+   ("Shipped"). Stage 3 = Shipped; stages 1/2 are unused in prod.
+5. **Mirror back-links:** every IF on a non-source SO (ODP/Agilant) should have
+   `upstreamItemFulfillmentId` set, and each of its items `upstreamItemFulfillmentItemId` set.
+
+## Verified schema corrections (older code/scripts assume these wrongly)
+- **`ItemFulfillmentPackages` does NOT exist** — header IF tracking is `ItemFulfillments_TrackingNumbers`.
+- `ItemFulfillmentItemUnits` has **no `trackingNumberId`** — unit tracking is the bridge
+  `ItemFulfillmentItemUnits_TrackingNumbers`.
+- ASN unit table is `AdvanceShippingNoticeItemUnits` (`unitId` nullable = tracking-only).
+- `ItemFulfillments.number` is UNIQUE NOT NULL; `dateItemFulfillment` NOT NULL.
+
+## Systems involved
+`Client_Compass` (2.0) tables; the 1.0 worker crons under `worker/crons/toga2/compass/workflow/`
+(transmit-to-MITS, transmit-to-vendors, ODP EDI import, Strategic Systems ASN import, NetSuite sync);
+`api2` cXML ShipNotice handler; the `_underscore` recursive item-fulfillment engine.
+
+## Retrofix tooling
+A standalone repair script lives in the worker 1.0 test area (`test/@jeff/compass/retrofix.php`,
+machine-local path tracked in dev memory). It sweeps Compass SOs newest-first, detects deviations from
+the invariants above, and repairs via direct SQL on `db_prod2_compass`. Safety rules baked in: rehearsal
+mode (logs intended writes, rolls back), per-phase gating, a hard block on any write to the
+`TrackingNumbers` table, and the never-over-fulfill cap. It is a diagnostic/repair tool, not part of the
+runtime workflow.
+
+## Edge cases & escalation
+- Incomplete orders (no PO/ASN/Agilant leg yet) are normal — repair only the layers that exist.
+- Logs for confirming original payloads: `Logs_Compass.Api` (inbound MITS POs apiId=2; inbound cXML
+  ShipNotices), legacy `Logs.API` (outbound transmits), legacy `Logs.FileLog` (`ODP_EDI`, ~30-day retention).
+- High-multiplier over-fulfillment (5×–20×) does not fit the split-PO spurious-link pattern — separate cause.
+
+## Change history
+- 2026-06-12 — Documented full lifecycle, both fulfillment flows, verified upstream-mirror direction,
+  the expected-shape invariants, and the retrofix tool (jcardinal)

package/knowledge/clients/prudential/INDEX.md

@@ -4,3 +4,4 @@
 |-----|-----------|---------|-------|
 | [Prudential: Dell ASN units PRE/POST interceptor (legacy key + flat tracking)](features/dell-asn-units-interceptor.md) | 2.0 | After the tracking-number bridge migration, the ASN unit route was renamed (`advance-shipping-notice-units` → `advance-shipping-notice-item-units`), so the inhe | _underscore/Model/Prudential/AdvanceShippingNotice.php, dbchanges2/Client_Prudential/2026-06-10 - AsnUnitsInterceptor.sql |
 | [Prudential Financial](profile.md) | 2.0 | Prudential is a TOGA client whose device-fulfillment flow is driven by **Dell** via the Dell API (`Client_Prudential.Apis.id = 2`). |  |
+| [Prudential: Dell ASN failed POST backfill replay](workflows/dell-asn-backfill-replay.md) | 2.0 | When Dell ASN POSTs fail in bulk (e.g. |  |

package/knowledge/clients/prudential/features/dell-asn-units-interceptor.md

@@ -5,8 +5,8 @@ project: _Underscore
 client: prudential
 type: client-feature
 status: active
-updated: 2026-06-10
-owners: ["jcardinal"]
+updated: 2026-06-12
+owners: ["jcardinal", "rgirish"]
 files:
   - _underscore/Model/Prudential/AdvanceShippingNotice.php
   - dbchanges2/Client_Prudential/2026-06-10 - AsnUnitsInterceptor.sql
@@ -28,7 +28,9 @@ interceptor translates it on the way in.
 - **`Model/Prudential/AdvanceShippingNotice.php::prePost(&$api, &$payload)`** — for each
   `$payload->advanceShippingNoticeItems[]`: renames the legacy `advanceShippingNoticeUnits` key to
   `advanceShippingNoticeItemUnits`, and lifts each unit's flat `trackingNumber` / `returnTrackingNumber`
-  into a single `advanceShippingNoticeItemUnitTrackingNumbers` bridge-child row, removing the flat fields.
+  into a single `advanceShippingNoticeItemUnitTrackingNumbers` bridge-child row using the FK field names
+  `trackingNumberId` / `returnTrackingNumberId` (not the plain field names — the engine resolves FK
+  children by field name ending in `Id`).
 - **Registration** — `Client_Prudential.ApiPayloadInterceptors` row: `recordId = 55`
   (advance-shipping-notices), `prePostProcessing = 'PRE'`, `httpMethod = 'POST'`, `apiId = 2` (Dell),
   `isActive = 1`. The engine resolves `_Model_Client_AdvanceShippingNotice` → `_Model_Prudential_AdvanceShippingNotice`
@@ -40,12 +42,41 @@ advanceShippingNoticeItems: [ { quantity, advanceShippingNoticeUnits: [
     { trackingNumber:{number}, returnTrackingNumber:{number}, unit:{serialNumber} } ] } ]
 ```
 The interceptor rewrites each item to `advanceShippingNoticeItemUnits` with the tracking moved into
-`advanceShippingNoticeItemUnitTrackingNumbers`.
+`advanceShippingNoticeItemUnitTrackingNumbers` as `{ trackingNumberId: {number:...}, returnTrackingNumberId: {number:...} }`.
+
+## Required DB config (`Core.RecordFields`)
+
+The bridge record's FK fields must have `childPolicy = MATCH_CREATE` — otherwise the engine throws
+`EV-12` when the `TrackingNumbers` row doesn't exist yet (Dell always sends new tracking numbers):
+
+```sql
+-- trackingNumberId on advance-shipping-notice-item-unit-tracking-numbers (id=1437)
+UPDATE Core.RecordFields SET childPolicy = 'MATCH_CREATE' WHERE id = 1437;
+-- returnTrackingNumberId on same record (id=2198)
+UPDATE Core.RecordFields SET childPolicy = 'MATCH_CREATE' WHERE id = 2198;
+```
+
+`MATCH_CREATE` = look up existing `TrackingNumbers` row by `number` (isIdentifier=1), create if not
+found. `MATCH` (the default) errors when no row exists — that was the root cause of the Jun 2026
+`EV-12` outage.
 
 ## Gotchas
 - Scoped to `apiId = 2` so other (internal) Prudential callers — which already send the new shape —
   are not double-transformed.
+- The interceptor must use **`trackingNumberId` / `returnTrackingNumberId`** (FK field names) on the
+  bridge child row — NOT `trackingNumber` / `returnTrackingNumber`. Using the plain names causes `EV-12`
+  because the engine treats them as unknown field lookups against `TrackingNumbers.number` with `MATCH`
+  policy and errors when no row is found.
 - DB change lives in `dbchanges2/Client_Prudential/`.
+- If Dell ASN POSTs fail in bulk, replay them from `Logs_Prudential.Api` — see the Dell ASN backfill
+  replay workflow doc.
+
+## Change history
+- 2026-06-12 — Fixed interceptor bug: bridge row keys were `trackingNumber`/`returnTrackingNumber`
+  instead of `trackingNumberId`/`returnTrackingNumberId`, causing EV-12 on all Dell ASN POSTs. Also
+  set `Core.RecordFields childPolicy = MATCH_CREATE` on ids 1437 and 2198. Replayed 28 failed payloads
+  from `Logs_Prudential.Api`. (rgirish)
 
 ## Related docs
 - 2.0 _underscore: Tracking-Number Bridge Migration.
+- Prudential: Dell ASN backfill replay workflow.

package/knowledge/clients/prudential/workflows/dell-asn-backfill-replay.md

@@ -0,0 +1,70 @@
+---
+title: "Prudential: Dell ASN failed POST backfill replay"
+framework: "2.0"
+project: API
+client: prudential
+type: workflow
+status: active
+updated: 2026-06-12
+owners: ["rgirish"]
+files: []
+related:
+  - ../features/dell-asn-units-interceptor.md
+---
+
+## Summary
+
+When Dell ASN POSTs fail in bulk (e.g. after a DB migration or interceptor bug), this workflow
+replays the failed payloads from the `Logs_Prudential.Api` log table using a fresh auth token.
+Used Jun 2026 after the `EV-12` tracking-number bug to backfill 28 missed shipments.
+
+## Steps
+
+1. **Confirm the root cause is fixed** before replaying — replaying into a broken system just
+   adds more failed log rows. Verify the interceptor fix and any `Core.RecordFields` DB changes
+   are applied to production.
+
+2. **Pull failed payloads from logs:**
+```sql
+SELECT id, dtStamp, transactionId, responseCode, requestPayload
+FROM Logs_Prudential.Api
+WHERE route LIKE '%advance-shipping%'
+  AND method = 'POST'
+  AND responseCode != 200
+ORDER BY id DESC
+LIMIT 28;
+```
+Adjust `LIMIT` and add a `dtStamp >= '...'` filter to scope to the incident window.
+
+3. **Get a fresh Dell API auth token** (the original `transactionId`s are already consumed —
+   each replay needs a new unique `transactionId`). The Dell API credentials (`client`, `api`,
+   `secret` UUIDs) are stored in `Client_Prudential.Apis` (id=2) — do not paste values here.
+
+4. **Test with the first payload only** — confirm `isSuccess: true` and `status: 201` before
+   continuing with the rest.
+
+5. **Replay all remaining payloads** using a script (Python recommended — shell array handling
+   breaks on multi-unit payloads with special characters). Generate a fresh UUID per request.
+   Log each result as `[N/total] OK` or `FAIL` with the error.
+
+6. **Verify** by checking `Logs_Prudential.Api` — successful replays appear with `responseCode = 201`
+   and new `transactionId`s distinct from the original failed ones.
+
+## Systems involved
+
+- `Logs_Prudential.Api` — source of failed payloads
+- `https://api.togahub.com/v2/advance-shipping-notices` — replay target
+- `Core.RecordFields` — `childPolicy` config (must be `MATCH_CREATE` on ids 1437 and 2198)
+- `_underscore/Model/Prudential/AdvanceShippingNotice.php` — PRE interceptor
+
+## Edge cases & escalation
+
+- **Token expires mid-run** (access token valid 1 hour): re-auth and resume from the first
+  failed item — already-created records are not duplicated.
+- **`EV-5` duplicate transactionId**: payload was already successfully processed. Skip it.
+- **`EV-12` still firing after fix**: `Core.RecordFields childPolicy` change may not have
+  cleared the config cache — wait a minute, then retry.
+- **`Box` serial numbers**: valid — some multi-unit shipments use `serialNumber: "Box"`.
+
+## Change history
+- 2026-06-12 — Created after replaying 28 failed Dell ASN POSTs from Jun 11 2026 incident. (rgirish)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.64",
+  "version": "1.0.66",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",