toga-ai 1.0.156 → 1.0.158

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -8,3 +8,4 @@
 | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |
 | [Recursive Item Fulfillments (upstream mirroring)](features/recursive-item-fulfillments.md) | In a multi-tier supply chain a sales order (SO) spawns a purchase order (PO) that becomes another SO downstream, and so on. | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillmentItem.php, _underscore/Model/Client/ItemFulfillmentItemUnit.php, _underscore/Model/Client/ItemFulfillmentPackage.php, _underscore/Model/Compass/AdvanceShippingNotice.php, dbchanges2/Core/2026-02-13 - 75601 - RecursiveItemFulfillmentCreation.sql, dbchanges2/Core/2026-06-04 - RecursiveItemFulfillmentPut.sql |
 | [Tracking-Number Bridge Migration (ASN / Item Fulfillment / Item Receipt)](features/tracking-number-bridges.md) | Shipment tracking numbers used to live as **scalar FK columns** (`trackingNumberId`, `returnTrackingNumberId`) directly on the lowest-level "unit"/"item" tables | api2/Component/Api/V2/V2.php, _underscore/Model/Client/AdvanceShippingNoticeItemUnit.php, _underscore/Model/Client/AdvanceShippingNoticeItemUnits/TrackingNumber.php, _underscore/Model/Client/ItemFulfillmentItemUnits/TrackingNumber.php, _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Prudential/AdvanceShippingNotice.php, _underscore/Model/Compass/AdvanceShippingNotice.php, _underscore/Trait/Netsuite/ItemFulfillment.php, api2/Component/Api/Cxml/Cxml.php, dbchanges2/Client/2026-06-10 - TrackingNumberBridges.sql, dbchanges2/Core/2026-06-10 - TrackingNumberBridges.sql, dbchanges2/Client_Prudential/2026-06-15 - ItemFulfillmentTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Quad/2026-06-18a - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Nychh/2026-06-19b - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Nychh/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql, dbchanges2/Client_Quad/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql |
+| [Units for Items for Purchase Orders — Data Structure](features/units-for-items-for-purchase-orders.md) | Describes how unit (serialized inventory) data is linked to sales-order and purchase-order line items behind the `units-for-items-for-purchase-orders` TableView |  |

package/knowledge/2.0/apps/_underscore/features/units-for-items-for-purchase-orders.md

@@ -0,0 +1,92 @@
+---
+title: Units for Items for Purchase Orders — Data Structure
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: feature
+status: active
+updated: 2026-06-22
+owners: ["bala"]
+files: []
+related: []
+---
+
+## Summary
+Describes how unit (serialized inventory) data is linked to sales-order and purchase-order
+line items behind the `units-for-items-for-purchase-orders` TableView. A unit only becomes
+visible in this view if it can be walked from the sales-order item out to a unit record. There
+are **two independent linkage paths** in the schema, and the view traverses only one of them —
+which is the root cause of "units not showing" cases.
+
+## How it works
+A `Unit` can be tied to an order line through either of two chains:
+
+**1. Receipt path (what this view follows):**
+```
+SalesOrderItems
+  -> SalesOrderItems_PurchaseOrderItems   (item-level bridge: salesOrderItemId, purchaseOrderItemId)
+  -> PurchaseOrderItems                   (PurchaseOrderItems.id = bridge.purchaseOrderItemId)
+  -> ItemReceiptItems                     (ItemReceiptItems.purchaseOrderItemId = PurchaseOrderItems.id)
+  -> ItemReceiptItemUnits                 (ItemReceiptItemUnits.itemReceiptItemId = ItemReceiptItems.id)
+  -> Units                                (ItemReceiptItemUnits.unitId)
+```
+
+**2. Fulfillment path (direct; NOT followed by this view):**
+```
+SalesOrderItems
+  -> ItemFulfillmentItems                 (ItemFulfillmentItems.salesOrderItemId = SalesOrderItems.id)
+  -> ItemFulfillmentItemUnits             (ItemFulfillmentItemUnits.itemFulfillmentItemId = ItemFulfillmentItems.id)
+  -> Units                                (ItemFulfillmentItemUnits.unitId)
+```
+
+The receipt path requires both the item-level bridge **and** an item receipt against the
+bridged purchase-order line. The fulfillment path reaches `SalesOrderItems` directly by
+`salesOrderItemId` with no purchase order or bridge involved.
+
+## Data model
+Header-level vs item-level bridges, and the level each transaction is anchored at:
+
+- **`PurchaseOrders_SalesOrders`** — *header* bridge: links a customer PO to a sales order
+  (`purchaseOrderId`, `salesOrderId`).
+- **`SalesOrderItems_PurchaseOrderItems`** — *item* bridge: links a sales-order line to a
+  vendor purchase-order line (`salesOrderItemId`, `purchaseOrderItemId`). This is the bridge
+  the receipt path depends on.
+- **Item receipts are PO-anchored only**: `ItemReceipts.purchaseOrderId`,
+  `ItemReceiptItems.purchaseOrderItemId` — there is **no sales-order FK on a receipt**. The
+  only way a received unit reaches a sales order is through the item bridge.
+- **Item fulfillments are SO-anchored**: `ItemFulfillments.salesOrderId`,
+  `ItemFulfillmentItems.salesOrderItemId`.
+- Both customer POs and vendor POs live in the same `PurchaseOrders` table, distinguished by
+  which bridge references them (header bridge = customer PO; item bridge via
+  `PurchaseOrderItems` = vendor PO).
+- `Items.inventoryType` governs whether units exist at all (see Gotchas).
+
+## Client variations
+None — uniform. The linkage structure is framework-level and identical across clients.
+
+## Gotchas / known issues
+- **The view only walks the receipt path.** A line whose units exist solely via the
+  fulfillment path (shipped from stock, never received against the bridged PO line) shows
+  **0 units** even though units physically exist. No data backfill can surface these — only a
+  resolver/view change that also walks the fulfillment path will.
+- **Missing item bridge → no units**, even when units were received. Without a
+  `SalesOrderItems_PurchaseOrderItems` row, the receipt path has no starting link.
+- **A present bridge is not sufficient.** If the bridged PO line has no
+  `ItemReceiptItemUnits`, the receipt path still resolves to nothing.
+- **`inventoryType` determines unit existence**: `SERIALIZED` and `HYBRID` items can produce
+  unit records; `NON_SERIALIZED` (bulk/consumable) never do. `HYBRID` items fulfilled as a
+  plain quantity (no serial capture) also produce no units — legitimately blank.
+- **Receipt-path counts are PO-line-wide.** Units surfaced reflect everything received against
+  the bridged PO line, which may exceed or differ from what was actually fulfilled to a given
+  sales order (over/under-attribution), because attribution is by PO line, not by the
+  fulfillment that shipped the unit.
+
+## Change history
+- 2026-06-22 — Documented the two-path units linkage model behind the
+  units-for-items-for-purchase-orders view and why the receipt-only traversal hides
+  fulfillment-only units. (bala)
+
+## Related docs
+- [[recursive-item-fulfillments]]
+- [[tracking-number-bridges]]

package/knowledge/INDEX.md

@@ -14,7 +14,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 9 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 10 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 8 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 4 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 1 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)

package/knowledge/clients/compass-canada/features/grand-and-toy-asn-import.md

@@ -5,7 +5,7 @@ project: _Underscore
 client: compass-canada
 type: client-feature
 status: active
-updated: 2026-06-19
+updated: 2026-06-22
 owners: ["bala"]
 files:
   - worker/crons/toga2/compasscanada/workflow/4_import_grand_and_toy_advance_shipping_notices.php
@@ -28,8 +28,9 @@ flow, adapted for Canadian carriers and bilingual email.
 ## Key files / entry points
 - `worker/crons/toga2/compasscanada/workflow/4_import_grand_and_toy_advance_shipping_notices.php`
   — the scheduled cron (1.0 worker tier). Reads mailbox `compasscanada.status@togatech.com`
-  (OAuth2 creds in config `[compasscanada]`), parses the 24-column CSV, posts to api2
-  `/advance-shipping-notices`. Scheduled in `worker/schedules/cron.worker.sync.json` (every 4h).
+  (OAuth2 creds — currently hardcoded with a TODO; must move to `App_Registry::get('config')['compasscanada']['gt_asn_oauth2']`
+  before production), parses the 24-column CSV, posts to api2 `/advance-shipping-notices`.
+  Scheduled in `worker/schedules/cron.worker.sync.json` (every 4h).
 - `_underscore/Model/Compass/AdvanceShippingNotice.php` — `postPost` interceptor that builds the
   ItemFulfillment chain on each ASN POST. Compass Canada inherits it via the empty
   `_underscore/Model/Compass/Canada/AdvanceShippingNotice.php`.
@@ -38,21 +39,27 @@ flow, adapted for Canadian carriers and bilingual email.
   user emails/notifications, and stamps `c_dtInTransitEmailSent = NOW()`.
 
 ## How it works
-1. Cron gets an O365 OAuth2 token (creds from `App_Registry::get('config')['compasscanada']`),
+1. Cron gets an O365 OAuth2 token (creds currently hardcoded; see Gotchas for migration path),
    opens the INBOX.
 2. Per attachment: decode CSV, gate on exactly 24 columns (otherwise email the team the file +
-   skip).
-3. Per row: skip electronic-delivery carriers (`Digital` / `E-delivered` / `E-verified`);
-   validate PO / part / tracking; look up SO + contact user + CC addresses; resolve carrier →
-   Ground shipping method; upsert `TrackingNumbers`; resolve the item by
-   `VendorItems.vendorPartNumber`; find-or-create a `Units` row per serial.
-4. POST `/advance-shipping-notices` with the tracking number nested at header + item + unit
-   level (so the interceptor sees it during the POST).
-5. The Compass ASN `postPost` interceptor (when enabled) creates the `ItemFulfillment` (one per
+   skip). Uses `preg_split('/\r?\n/', ...)` on the first line so Windows CRLF attachments parse
+   correctly.
+3. **Before the row loop** — build three lookup maps once per file: `$purchaseOrderUuidByNumber`
+   (`SELECT number, uuid FROM PurchaseOrders`), `$shippingMethodByCarrierId`, `$carrierIdByName`.
+4. Per row: validate PO / part / tracking (early `continue` with error recording on missing
+   critical fields); look up SO + contact user + CC addresses; resolve carrier →
+   Ground shipping method (falls back to `DEFAULT_GROUND_SHIPPING_METHOD_ID = 3`); upsert
+   `TrackingNumbers`; resolve item by `Items.partNumber` joined through `VendorItems`; find-or-create
+   a `Units` row per serial.
+5. POST `/advance-shipping-notices` with ASNIU (unit) payload so the interceptor can act.
+6. After the POST: query each generated ASNIU UUID back from DB, insert
+   `AdvanceShippingNoticeItemUnits_TrackingNumbers`. This runs for EVERY newly created ASNIU,
+   regardless of whether the TrackingNumber record was new or pre-existing.
+7. The Compass ASN `postPost` interceptor (when enabled) creates the `ItemFulfillment` (one per
    SO number, reused if it exists), `ItemFulfillmentItems`, `ItemFulfillmentItemUnits`, and the
    IF/IFI/IFIU `_TrackingNumbers` bridges.
-6. Only when a tracking number is newly inserted: write Notifications (1.0 `db_store` + 2.0),
-   resolve the user's language, send the EN/FR in-transit email via POST
+8. Only when a tracking number is newly inserted: write Notifications (1.0 `db_store` + 2.0),
+   resolve the user's language, send the EN/FR in-transit email via
    `/email-templates/sendEmail`, then stamp `c_dtInTransitEmailSent = NOW()`.
 
 ## Data model
@@ -73,30 +80,47 @@ a bilingual EN/FR in-transit email (USA is English only) and uses Canadian carri
 
 ## Gotchas / known issues
 - **CSV "Part Number" is the VENDOR part number (`VendorItems.vendorPartNumber`), NOT
-  `Items.partNumber`.** Resolve the item through `VendorItems`. (e.g. vendor `IMFP3260` →
-  `Items.partNumber` `C2CY5UC#ABA`.)
+  `Items.partNumber` directly.** Resolve the item through the `VendorItems` join. (e.g. vendor
+  `IMFP3260` → `Items.partNumber` `C2CY5UC#ABA`.)
 - **G&T's ASN file inconsistently drops the `IM` prefix** (`FP3260` vs `IMFP3260`) on some lines
   → item-not-found → those lines are skipped and reported in the import-report email. This is a
   G&T-side export issue; toga transmits the correct `IMFP…` to G&T as the cXML `SupplierPartID`
   (see `2_transmit_mits_purchase_orders_to_vendors.php`), so it is NOT a MITS bug.
+- **Build lookup maps OUTSIDE the while loop.** `$purchaseOrderUuidByNumber`, `$shippingMethodByCarrierId`,
+  and `$carrierIdByName` must be built once before the row loop — they were originally inside,
+  causing a full-table scan of `PurchaseOrders`, `ShippingMethods`, and `ShippingCarriers` on
+  every CSV row.
+- **ASNIU→TrackingNumber link is NOT gated on `$isNewTrackingNumber`.** The ASNIU is always
+  freshly created on each import row; the `AdvanceShippingNoticeItemUnits_TrackingNumbers` insert
+  must run regardless of whether the TrackingNumber record itself was new or pre-existing.
+  Gating it on "new TN only" means re-sent / re-processed CSV rows create orphaned ASNIUs with
+  no tracking link.
+- **`$shippingMethodByCarrierId[$selectedCarrierId]` needs a null-coalesce fallback.**  On PHP 8,
+  accessing a missing key without `?? DEFAULT_GROUND_SHIPPING_METHOD_ID` produces a deprecation
+  warning that becomes an error. Always write `$shippingMethodByCarrierId[$selectedCarrierId] ?? DEFAULT_GROUND_SHIPPING_METHOD_ID`.
+- **CSV CRLF attachments:** use `preg_split('/\r?\n/', $decodedAttachment)[0]` (not
+  `explode("\n", ...)`) for the header-line column-count check. `explode("\n")` leaves a trailing
+  `\r` on the first line when the file uses Windows line endings, making `str_getcsv` miscount
+  columns and silently skip the file.
 - **The ASN→IF interceptor must be ENABLED per client.** `Core.ApiPayloadInterceptors` for
-  `recordId 55`, `prePostProcessing=POST`, `httpMethod=POST` must be `isActive=1` (and the client
-  `AclRecordScripts` must grant the caller's roles). It was OFF for Compass Canada; enabling it is
-  required or ASNs post but the IF/IFI/IFIU chain never builds.
-- **Tracking must be nested in the ASN POST payload** (header/item/unit), not linked via SQL
-  after the POST — the interceptor runs during the POST, so post-POST links are invisible to it.
-- **`/email-templates/sendEmail` must be POST** (not GET) so a large `orderItems` HTML payload
-  does not hit the URL-length limit (HTTP 414). POST requires the `sendEmail` record script
-  registered for the POST method (`Core.RecordScripts`), else the engine returns `EV-8`.
+  `recordId 55`, `prePostProcessing=POST`, `httpMethod=POST` must be `isActive=1`. It was OFF for
+  Compass Canada; enabling it is required or ASNs post but the IF/IFI/IFIU chain never builds.
+- **OAuth2 credentials are currently hardcoded** in the cron with a `// TODO` comment. Before
+  going to production, move them to `config.<env>.ini` under `[compasscanada]` and read via
+  `App_Registry::get('config')['compasscanada'][…]`. The `config.<env>.ini` must be deployed
+  with the code or the OAuth2 token request fails silently.
 - Each CSV line creates one ASN; the interceptor dedups to **one ItemFulfillment per SO**.
 - A new carrier needs its **own** Ground `ShippingMethod`, or the TrackingNumber ends up with the
   right carrier but another carrier's Ground method id.
 - `App_Database::fetchOne()` (1.0) throws on a 0-row result — guard item lookups with `numRows()`
   before `fetchOne()`.
-- The cron reads OAuth2 creds from config `[compasscanada]`; the updated `config.<env>.ini` must
-  be deployed with the code or the token request fails.
 
 ## Change history
+- 2026-06-22 — Full variable rename (no abbreviations); moved PO/carrier/shipping-method lookup maps
+  outside the CSV row loop (was causing N full-table queries per file); fixed ASNIU→TN link to run
+  unconditionally for new ASNIUs (not only when TN was new); added PHP 8 null-coalesce on carrier→
+  method lookup; fixed CSV CRLF detection with `preg_split`; added `DEFAULT_GROUND_SHIPPING_METHOD_ID`
+  named constant; function renamed `buildOrderItemsHtmlGrandAndToy` (camelCase). (bala)
 - 2026-06-19 — Built the G&T ASN import cron + one-time backfill script; matched items by
   `VendorItems.vendorPartNumber`; added Canadian carriers + per-carrier Ground methods; EN/FR
   in-transit email via POST; enabled the Compass ASN→IF interceptor for Compass Canada. (bala)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.156",
+  "version": "1.0.158",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",