toga-ai 1.0.145 → 1.0.147

package/knowledge/2.0/apps/dbchanges2/architecture.md

@@ -6,8 +6,8 @@ project: Database Changes
 client: shared
 type: architecture
 status: active
-updated: 2026-06-12
-owners: [jcardinal]
+updated: 2026-06-19
+owners: [jcardinal, mhammontree]
 files:
   - Core/
   - Client/
@@ -141,6 +141,36 @@ file inside `HISTORIC` running; **never** add new changes there.
 5. Never edit or re-date an already-applied file — add a new dated file instead. Never put new
    work in a `HISTORIC` folder.
 
+## Bulk data loads — one statement, not many
+
+When a change inserts many rows (reference-data / code-table loads), write it as a **single
+bulk `INSERT`**, not one `INSERT` statement per row. The database is billed per query and the
+external executor runs each statement as its own round-trip, so N separate inserts cost N times
+the round-trips (and query cost) of one batched statement. This is a standing team preference.
+
+- Use one `INSERT ... VALUES (...),(...),...` or `INSERT ... SELECT` over a derived
+  (`UNION ALL`) row set.
+- To stay **idempotent** without a unique key to `INSERT IGNORE` against, anti-join the source
+  set to the target and insert only the misses:
+
+```sql
+INSERT INTO TargetTable (uuid, keyColumn, otherColumn)
+SELECT
+	src.uuid,
+	src.keyColumn,
+	src.otherColumn
+FROM (
+	SELECT '<uuid>' AS uuid, '<key>' AS keyColumn, '<value>' AS otherColumn
+	UNION ALL SELECT ...
+) AS src
+LEFT JOIN TargetTable existing ON existing.keyColumn = src.keyColumn
+WHERE
+	existing.id IS NULL;
+```
+
+- Keep the single statement well under `max_allowed_packet` (64 MB default) — a few thousand
+  rows is comfortably fine.
+
 ## Relationship to the rest of 2.0
 
 `dbchanges2` is registered as a **2.0 core repo** (`role: core` in `registry.json`) — it is

package/knowledge/INDEX.md

@@ -34,6 +34,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## Clients
 
+- **AIG (Staples Protection Plan)** (`aig`) → [clients/aig/INDEX.md](clients/aig/INDEX.md)
 - **Compass Canada** (`compass-canada`) → [clients/compass-canada/INDEX.md](clients/compass-canada/INDEX.md)
 - **Compass USA** (`compass-usa`) → [clients/compass-usa/INDEX.md](clients/compass-usa/INDEX.md)
 - **Elite** (`elite`) → [clients/elite/INDEX.md](clients/elite/INDEX.md)

package/knowledge/clients/aig/INDEX.md

@@ -0,0 +1,6 @@
+# Client: AIG (Staples Protection Plan) `aig`
+
+| Doc | Framework | Summary | Files |
+|-----|-----------|---------|-------|
+| [AIG Entitlement Intake & SaleItem Code Resolution](features/entitlement-intake.md) | 2.0 | AIG protection-plan entitlements arrive as `api2` V2 JSON POSTs. | _underscore/Model/Aig/Entitlement.php, dbchanges2/Client_Aig/2026-06-18a - TRUE-79534 AIG SaleItem codes.sql |
+| [AIG (Staples Protection Plan)](profile.md) | 2.0 | AIG is the warranty underwriter behind the **Staples Protection Plan** retail program. |  |

package/knowledge/clients/aig/features/entitlement-intake.md

@@ -0,0 +1,131 @@
+---
+title: AIG Entitlement Intake & SaleItem Code Resolution
+framework: "2.0"
+project: API
+client: aig
+type: client-feature
+status: active
+updated: 2026-06-19
+owners: ["mhammontree"]
+files:
+  - _underscore/Model/Aig/Entitlement.php
+  - dbchanges2/Client_Aig/2026-06-18a - TRUE-79534 AIG SaleItem codes.sql
+related:
+  - 2.0/apps/api2/architecture.md
+  - 2.0/apps/dbchanges2/architecture.md
+  - 2.0/apps/_underscore/architecture.md
+---
+
+## Summary
+
+AIG protection-plan entitlements arrive as `api2` V2 JSON POSTs. Before the entitlement is
+written, the `_Model_Aig_Entitlement::prePost` interceptor resolves the **sold warranty SKU**
+(`saleItem.partNumber`) against `Client_Aig.Items` to set the `Entitlement.saleItemId` foreign
+key and to attach fulfillment types/methods. If that part number has no matching `Items` row,
+the FK can't resolve and intake fails with **"Missing AIG item ID."** Keeping
+`Client_Aig.Items` populated with AIG's current sale-item catalog is what keeps intake working.
+
+## Key files / entry points
+
+- **`_underscore/Model/Aig/Entitlement.php`** — `prePost(&$api, &$payload)` is the PRE-POST
+  API payload interceptor (named by the `prePost*` convention) that runs on entitlement intake.
+- **`Client_Aig.Items`** — the catalog the interceptor looks up against (`partNumber` column).
+- **`dbchanges2/Client_Aig/…`** — where new code batches are loaded (see *Uploading new codes*).
+
+## How it works
+
+1. AIG POSTs an entitlement payload (vendor `STS_001`) to the `api2` V2 API.
+2. `_Model_Aig_Entitlement::prePost` runs and:
+   - cleans a duplicated-firstName-in-lastName quirk on `contact`;
+   - if `payload->item` (the covered **device/unit**) has a `description` but no `itemCategory`,
+     sets `itemCategory.name = item.description`;
+   - reads `partNumber = payload->saleItem->partNumber` and runs
+     `SELECT id FROM Items WHERE partNumber = '{partNumber}' LIMIT 1` to get `$itemId`;
+   - uses `$itemId` to load fulfillment **types** (`Items_EntitlementFulfillmentTypes`) and
+     **methods** (`Items_EntitlementFulfillmentMethods`) and injects them into the payload.
+3. The V2 engine resolves `Entitlement.saleItemId` (FK → `_Model_Client_Item`) from that same
+   `saleItem.partNumber`. **No matching `Items` row → unresolvable FK → "Missing AIG item ID".**
+4. `postPost` then emails the contact a Staples Protection Plan registration link
+   (`staplesprotection.togaview.com`).
+
+## Data model
+
+`Client_Aig.Items` is **dual-purpose** — it holds two different kinds of row, distinguished by
+whether `itemCategoryId` is set:
+
+| Row kind | Example `partNumber` | `description` | `itemCategoryId` |
+|---|---|---|---|
+| **Sale item** (warranty SKU) | `ASI-2YG2`, `SP-2D-TAB3` | price-band plan, e.g. `2YR Product under $500: ($30-$49.99)` | **NULL** |
+| **Unit/device item** | `157C`, `1159`, `5022` | device type, e.g. `Chromebook`, `Printer Inkjet` | set → `ItemCategories` |
+
+- `Client_Aig.ItemCategories` is a **device-classification** table (`Laptop`, `Printer`,
+  `Tablet`) — it classifies the physical hardware (unit items), **not** the warranty sale
+  items. Sale items correctly carry `itemCategoryId = NULL`; do not try to assign them a
+  category (the price-band plan descriptions don't map to a device type).
+- `ItemCategories.c_premiumTechSupportCategory` (enum `PC`/`Tablet`) is read by the AIG
+  ClosedClaims → NetSuite invoicing sync.
+
+### SaleItem code schemes
+
+AIG migrated their sale-item coding. The catalog spans:
+- **Legacy `SP-*`** — the original scheme (only a handful seeded historically).
+- **New `ASI-*` / `SM-*`** — current STS_001 (Staples) scheme; the bulk of the catalog.
+- **Numeric `SA` codes** (8-digit, e.g. `24664377`) — a separate program sheet in AIG's
+  spreadsheet, distinct part-number space (no collisions with the STS codes).
+
+## Uploading new codes (the recurring task)
+
+When AIG sends a new "Active SaleItemID" spreadsheet (columns `SaleItemID`, `Description`):
+
+1. Load it into `Client_Aig.Items` via a **`dbchanges2/Client_Aig/`** migration (dated
+   `YYYY-MM-DD<letter>` per the dbchanges2 contract).
+2. Insert minimally `uuid`, `partNumber`, `description` — everything else (`isActive`,
+   `isVisible`, `inventoryType=HYBRID`, timestamps) takes table defaults; **leave
+   `itemCategoryId` NULL** for sale items.
+3. Generate **fully-random UUIDs** (e.g. `uuid4`), never the time-based MySQL `UUID()` —
+   per the 2.0 standard. Bake literal UUIDs into the SQL since a migration can't call PHP's
+   `_String::generateUuid()`.
+4. Use a **single bulk `INSERT`** (see the dbchanges2 architecture rule), made idempotent with
+   an anti-join because `Items` has **no unique key on `partNumber`** (so `INSERT IGNORE`
+   can't help):
+   ```sql
+   INSERT INTO Items (uuid, partNumber, description)
+   SELECT src.uuid, src.partNumber, src.description
+   FROM ( SELECT '<uuid>' AS uuid, '<part>' AS partNumber, '<desc>' AS description
+          UNION ALL SELECT ... ) AS src
+   LEFT JOIN Items existing ON existing.partNumber = src.partNumber
+   WHERE existing.id IS NULL;
+   ```
+
+## Client variations
+
+This is AIG-specific behavior (the `_Model_Aig_Entitlement` override); other clients do not run
+this interceptor or use this dual-purpose Items pattern.
+
+## Gotchas / known issues
+
+- **"Missing AIG item ID" = the sale-item code isn't in `Client_Aig.Items`.** This is the
+  classic symptom of AIG shipping new codes before the catalog is loaded. Fix = load the codes.
+- **The `partNumber` lookup is raw-interpolated** (`WHERE partNumber = '{$partNumber}'`) — a
+  latent SQL-injection point in `_Model_Aig_Entitlement::prePost`. Payload data is partner-
+  supplied; should be escaped with `_Database::escape()` or parameterized. Not yet fixed
+  (TRUE-79534 was data-load only).
+- **The lookup is not vendor-scoped** (`LIMIT 1` on `partNumber` alone). It relies on
+  `partNumber` being globally unique within `Items`; safe today because the code spaces don't
+  collide, but a future collision would silently resolve to the wrong item.
+- **Unit/device item codes are a separate source.** The entitlement payload also carries
+  `entitlementUnits[].unit.item.partNumber` (e.g. `1167` "Tablet Accessories"); those device
+  codes are **not** in the SaleItemID spreadsheet and must be sourced separately if missing.
+- **No unique key on `Items.partNumber`** — use an anti-join for idempotent loads, not
+  `INSERT IGNORE`.
+
+## Change history
+
+- 2026-06-19 — Documented intake flow; loaded 2,200 new SaleItem codes (1,700 STS `ASI-*`/`SM-*`
+  + 500 unique numeric `SA`) into `Client_Aig.Items` via TRUE-79534, fixing "Missing AIG item ID"
+  on the new `ASI-*` scheme. Follow-up to TRUE-79441. (mhammontree)
+
+## Related docs
+
+- [api2 architecture](../../../2.0/apps/api2/architecture.md) — the V2 intake engine + payload interceptors.
+- [dbchanges2 architecture](../../../2.0/apps/dbchanges2/architecture.md) — migration naming + the bulk-insert rule.

package/knowledge/clients/aig/profile.md

@@ -0,0 +1,40 @@
+---
+title: "AIG (Staples Protection Plan)"
+framework: "2.0"
+apps:
+  - _underscore
+  - api2
+  - dbchanges2
+project: API
+client: aig
+type: profile
+status: active
+updated: 2026-06-19
+owners: ["mhammontree"]
+files: []
+related:
+  - clients/aig/features/entitlement-intake.md
+---
+
+## Summary
+
+AIG is the warranty underwriter behind the **Staples Protection Plan** retail program. In the
+2.0 platform AIG is a database-level tenant (`Client_Aig` schema) with its own `_Model_Aig_*`
+class overrides in `_underscore` (most pulling in NetSuite traits). Protection-plan
+**entitlements** are pushed to TOGa via the `api2` V2 JSON API; on intake an interceptor
+resolves the sold warranty SKU against `Client_Aig.Items`, and closed claims are invoiced
+back to AIG through NetSuite.
+
+The intake vendor on entitlement payloads is `STS_001` ("STS" = Staples); registration emails
+go out under the Staples Protection Plan brand and link to `staplesprotection.togaview.com`.
+
+## Integration touchpoints
+
+- **`api2`** — entitlement intake (`/v2` JSON), where the AIG payload is POSTed.
+- **`_underscore`** — `_Model_Aig_*` overrides (Entitlement intake interceptor, Contact sync,
+  NetSuite item/SO/invoice traits).
+- **`dbchanges2`** — `Client_Aig/` schema + reference-data migrations (e.g. the SaleItem code
+  catalog in `Client_Aig.Items`).
+
+See [entitlement-intake.md](features/entitlement-intake.md) for how a payload becomes an
+entitlement and how to load new sale-item codes.

package/knowledge/clients/compass-canada/INDEX.md

@@ -2,4 +2,5 @@
 
 | Doc | Framework | Summary | Files |
 |-----|-----------|---------|-------|
+| [Grand & Toy ASN Import (Compass Canada)](features/grand-and-toy-asn-import.md) | 2.0 | Imports Grand & Toy (G&T) Advance Shipping Notices for Compass Canada. | worker/crons/toga2/compasscanada/workflow/4_import_grand_and_toy_advance_shipping_notices.php, worker/crons/toga2/compasscanada/workflow/import_grand_and_toy_asn_from_file.php, worker/schedules/cron.worker.sync.json, _underscore/Model/Compass/AdvanceShippingNotice.php, _underscore/Model/Compass/Canada/AdvanceShippingNotice.php, dbchanges2/Client_CompassCanada/ |
 | [Compass Canada](profile.md) | 2.0 | Compass Canada is the Canadian arm of the Compass account — a separate TOGA tenant, related to but distinct from Compass USA. |  |

package/knowledge/clients/compass-canada/features/grand-and-toy-asn-import.md

@@ -0,0 +1,105 @@
+---
+title: Grand & Toy ASN Import (Compass Canada)
+framework: "2.0"
+project: _Underscore
+client: compass-canada
+type: client-feature
+status: active
+updated: 2026-06-19
+owners: ["bala"]
+files:
+  - worker/crons/toga2/compasscanada/workflow/4_import_grand_and_toy_advance_shipping_notices.php
+  - worker/crons/toga2/compasscanada/workflow/import_grand_and_toy_asn_from_file.php
+  - worker/schedules/cron.worker.sync.json
+  - _underscore/Model/Compass/AdvanceShippingNotice.php
+  - _underscore/Model/Compass/Canada/AdvanceShippingNotice.php
+  - dbchanges2/Client_CompassCanada/
+related:
+  - ../profile.md
+---
+
+## Summary
+Imports Grand & Toy (G&T) Advance Shipping Notices for Compass Canada. A 1.0 worker cron reads
+G&T's ASN CSV from a mailbox, posts each shipped line to the 2.0 API as an AdvanceShippingNotice
+(ASN); a Compass interceptor then auto-creates the ItemFulfillment chain, and the customer gets
+an in-transit email in English or French. It mirrors the Compass USA ODP / Strategic-Systems ASN
+flow, adapted for Canadian carriers and bilingual email.
+
+## Key files / entry points
+- `worker/crons/toga2/compasscanada/workflow/4_import_grand_and_toy_advance_shipping_notices.php`
+  — the scheduled cron (1.0 worker tier). Reads mailbox `compasscanada.status@togatech.com`
+  (OAuth2 creds in config `[compasscanada]`), parses the 24-column CSV, posts to api2
+  `/advance-shipping-notices`. Scheduled in `worker/schedules/cron.worker.sync.json` (every 4h).
+- `_underscore/Model/Compass/AdvanceShippingNotice.php` — `postPost` interceptor that builds the
+  ItemFulfillment chain on each ASN POST. Compass Canada inherits it via the empty
+  `_underscore/Model/Compass/Canada/AdvanceShippingNotice.php`.
+- `worker/crons/toga2/compasscanada/workflow/import_grand_and_toy_asn_from_file.php` — one-time
+  backfill variant: reads a CSV from disk (no mailbox/OAuth), creates ASN data only, sends NO
+  user emails/notifications, and stamps `c_dtInTransitEmailSent = NOW()`.
+
+## How it works
+1. Cron gets an O365 OAuth2 token (creds from `App_Registry::get('config')['compasscanada']`),
+   opens the INBOX.
+2. Per attachment: decode CSV, gate on exactly 24 columns (otherwise email the team the file +
+   skip).
+3. Per row: skip electronic-delivery carriers (`Digital` / `E-delivered` / `E-verified`);
+   validate PO / part / tracking; look up SO + contact user + CC addresses; resolve carrier →
+   Ground shipping method; upsert `TrackingNumbers`; resolve the item by
+   `VendorItems.vendorPartNumber`; find-or-create a `Units` row per serial.
+4. POST `/advance-shipping-notices` with the tracking number nested at header + item + unit
+   level (so the interceptor sees it during the POST).
+5. The Compass ASN `postPost` interceptor (when enabled) creates the `ItemFulfillment` (one per
+   SO number, reused if it exists), `ItemFulfillmentItems`, `ItemFulfillmentItemUnits`, and the
+   IF/IFI/IFIU `_TrackingNumbers` bridges.
+6. Only when a tracking number is newly inserted: write Notifications (1.0 `db_store` + 2.0),
+   resolve the user's language, send the EN/FR in-transit email via POST
+   `/email-templates/sendEmail`, then stamp `c_dtInTransitEmailSent = NOW()`.
+
+## Data model
+- ASN: `AdvanceShippingNotices` / `…Items` / `…ItemUnits` + their `…_TrackingNumbers` bridges.
+- IF: `ItemFulfillments` / `…Items` / `…ItemUnits` + their `…_TrackingNumbers` bridges.
+- Vendor: `App_Client_CompassCanada::UUID_VENDOR__GRAND_TOY`.
+- Carriers + methods: `ShippingCarriers` Precision / Purolator / ATSL / Nationex, each with its
+  own Ground `ShippingMethods` row (added 2026-06-18 via dbchanges2; tracking-URL prefixes +
+  carrier logos set on the carrier).
+- Language: `UserGlobalSettings.settingId = 2`, values `en` / `fr-CA`. In-transit `EmailTemplates`
+  uuids — EN `26d2cb67-bdc5-4973-9e39-160c89b0af56`, FR `fb8be211-7c07-4939-9682-08b6c3245f66`.
+- Interceptor wiring: `Core.RecordScripts` + `Core.ApiPayloadInterceptors`, `recordId 55`
+  (AdvanceShippingNotice). The `sendEmail` POST record script is `Core.RecordScripts recordId 202`.
+
+## Client variations
+Compass-Canada-specific; parallels Compass USA's ODP / Strategic-Systems ASN import. Canada sends
+a bilingual EN/FR in-transit email (USA is English only) and uses Canadian carriers.
+
+## Gotchas / known issues
+- **CSV "Part Number" is the VENDOR part number (`VendorItems.vendorPartNumber`), NOT
+  `Items.partNumber`.** Resolve the item through `VendorItems`. (e.g. vendor `IMFP3260` →
+  `Items.partNumber` `C2CY5UC#ABA`.)
+- **G&T's ASN file inconsistently drops the `IM` prefix** (`FP3260` vs `IMFP3260`) on some lines
+  → item-not-found → those lines are skipped and reported in the import-report email. This is a
+  G&T-side export issue; toga transmits the correct `IMFP…` to G&T as the cXML `SupplierPartID`
+  (see `2_transmit_mits_purchase_orders_to_vendors.php`), so it is NOT a MITS bug.
+- **The ASN→IF interceptor must be ENABLED per client.** `Core.ApiPayloadInterceptors` for
+  `recordId 55`, `prePostProcessing=POST`, `httpMethod=POST` must be `isActive=1` (and the client
+  `AclRecordScripts` must grant the caller's roles). It was OFF for Compass Canada; enabling it is
+  required or ASNs post but the IF/IFI/IFIU chain never builds.
+- **Tracking must be nested in the ASN POST payload** (header/item/unit), not linked via SQL
+  after the POST — the interceptor runs during the POST, so post-POST links are invisible to it.
+- **`/email-templates/sendEmail` must be POST** (not GET) so a large `orderItems` HTML payload
+  does not hit the URL-length limit (HTTP 414). POST requires the `sendEmail` record script
+  registered for the POST method (`Core.RecordScripts`), else the engine returns `EV-8`.
+- Each CSV line creates one ASN; the interceptor dedups to **one ItemFulfillment per SO**.
+- A new carrier needs its **own** Ground `ShippingMethod`, or the TrackingNumber ends up with the
+  right carrier but another carrier's Ground method id.
+- `App_Database::fetchOne()` (1.0) throws on a 0-row result — guard item lookups with `numRows()`
+  before `fetchOne()`.
+- The cron reads OAuth2 creds from config `[compasscanada]`; the updated `config.<env>.ini` must
+  be deployed with the code or the token request fails.
+
+## Change history
+- 2026-06-19 — Built the G&T ASN import cron + one-time backfill script; matched items by
+  `VendorItems.vendorPartNumber`; added Canadian carriers + per-carrier Ground methods; EN/FR
+  in-transit email via POST; enabled the Compass ASN→IF interceptor for Compass Canada. (bala)
+
+## Related docs
+- [Compass Canada profile](../profile.md)

package/knowledge/clients/compass-canada/profile.md

@@ -12,7 +12,7 @@ project: _Underscore
 client: compass-canada
 type: profile
 status: active
-updated: 2026-06-18
+updated: 2026-06-19
 owners: [jcardinal, bala]
 files: []
 related:
@@ -31,10 +31,17 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
 - **1.0:** worker crons under `worker/crons/toga2/compasscanada/`.
 
 ## Vendors & integrations
-- Not yet captured. Confirm vendors / ASN ingestion paths before relying on them.
+- **Grand & Toy (G&T)** — primary hardware vendor. SOs flow toga → MITS → PO to G&T; G&T sends
+  back ASNs. ASN ingestion (email CSV + the auto-created ItemFulfillment chain + bilingual
+  in-transit email) is documented in [Grand & Toy ASN Import](features/grand-and-toy-asn-import.md).
+  Vendor uuid: `App_Client_CompassCanada::UUID_VENDOR__GRAND_TOY`. Canadian carriers in use:
+  UPS, FedEx, Purolator, Precision, Nationex, ATSL (each with a Ground `ShippingMethod`).
+- Worker crons for the G&T flow live under `worker/crons/toga2/compasscanada/workflow/`
+  (`1_…` transmit SOs to MITS, `2_…` transmit POs to vendors, `3_…` status from G&T cXML,
+  `4_…` import G&T ASNs).
 
 ## Notes
-- **Stub** — created alongside the Compass USA profile for disambiguation. The 2026-06-08 ASN
-  → ItemFulfillment work was for **Compass USA**, not Compass Canada. Expand this profile as
-  Compass Canada behavior is investigated.
+- Customer language preference: `UserGlobalSettings.settingId = 2` (`en` / `fr-CA`); customer-
+  facing emails are sent in EN or FR accordingly.
+- The 2026-06-08 ASN → ItemFulfillment work was for **Compass USA**, not Compass Canada.
 - Related: [Compass USA](../compass-usa/profile.md).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.145",
+  "version": "1.0.147",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",