toga-ai 1.0.255 → 1.0.257

package/knowledge/1.0/apps/library/features/netsuite-suiteql-api-reference.md

@@ -6,7 +6,7 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-06-25
+updated: 2026-06-30
 owners: ["dfranks"]
 files:
   - library/app/api/netsuite/rest.php
@@ -330,9 +330,23 @@ numbers. Budget hours for multi-year runs and launch them under `nohup`/`tmux`.
   (PHP 8.0) is for running probes on the laptop only, **not** for compat checking.
 - Field/relationship availability varies by NetSuite account **and** record type — probe the live
   account before assuming a column exists.
+- **`IS NOT NULL` on a custom `transactionline` column inside an aggregate query is a ~15× planner
+  trap — prefer `> 0` (or `NVL(col,0) <> 0`).** Filtering a custom employee-ref column
+  (`custcol_sales_rep_line`) with `AND tl.custcol_sales_rep_line IS NOT NULL` inside a
+  `GROUP BY` / power-sum aggregate (`MOD(SUM(MOD(id*id,P)),P)`) made the planner scan
+  pathologically: **~122s** vs **~8s** for the semantically identical `AND tl.custcol_sales_rep_line
+  > 0` (employee internalIds are always positive) — a ~15× regression purely from the predicate form.
+  Measured against the Forecast2 reconciliation checker (TRUE-79968). Use `> 0` / `NVL(col,0) <> 0`
+  for a custom ref column you only need to test for presence in an aggregate; reserve `IS NOT NULL`
+  for non-aggregate row filters.
 
 ## Change history
 
+- 2026-06-30 — **Recorded the `IS NOT NULL` vs `> 0` planner trap for custom `transactionline`
+  columns in aggregate queries** (TRUE-79968): a custom employee-ref column filtered with
+  `IS NOT NULL` inside a power-sum `GROUP BY` aggregate scanned ~15× slower (~122s) than the
+  identical `> 0` predicate (~8s). Prefer `> 0` / `NVL(col,0) <> 0` for presence tests in
+  aggregates. (dfranks)
 - 2026-06-25 — **Documented Journal Entry line structure + the JE→Forecast.Sales deferral.** JEs
   invert the `mainline` rule — **all** JE `transactionline` rows are `mainline='T'` (no `'F'` detail
   split), so a `mainline='F'` revenue query over a JE returns empty. Also: `tl.salesrep` does not

package/knowledge/1.0/apps/worker/features/forecast2-netsuite-reconciliation.md

@@ -233,6 +233,22 @@ by reconciling a chosen tranDate range directly against NetSuite.
   are correctly excluded **by design** (e.g. JE 7213407 = a Bank↔Equity reclass, 0 allowlist lines → never
   imported — not a miss). An accrual JE and its NetSuite auto-reversal are **separate transactions**, each
   reconciled independently by its own `internalId`.
+- **The rep-only-NS-side / unfiltered-FC-side asymmetry is DELIBERATE and load-bearing — do NOT add
+  a rep filter to any Forecast-side query (TRUE-79968, dfranks 2026-06-30).** The importer now keeps
+  **only JE lines that carry a raw `custcol_sales_rep_line`** (see the sale-import doc). To make the
+  fixer be the **cleanup mechanism** for any pre-existing no-rep JE rows (there is no separate delete
+  script), the **NS-side** JE SuiteQL in both `checker.php` (`journalEntryNsTotals` arm of the sales
+  `nsSub`) and `fixer.php` (`journalEntryNsTotals` FIND + `fixJournalEntries` FIX) is scoped to
+  **rep-bearing lines** (mirroring the importer), while the **Forecast-side** reads in both tools stay
+  **UNFILTERED** (still SUM all JE rows, incl. any `salesRepEmployeeId IS NULL`). That asymmetry is
+  what surfaces the stale rows: NS (rep-only) **<** FC (still holds no-rep rows) → FIND flags the JE →
+  the existing per-`'journalEntry'` orphan-delete in `fixJournalEntries` removes them. **If the FC side
+  were ALSO rep-filtered, both sides would match and the stale rows would be invisible / never purged.**
+  A blanket `DELETE WHERE salesRepEmployeeId IS NULL` is the wrong fix — locally-null also means
+  "rep present in NetSuite but unmapped to `Forecast.Employees`," which must survive; the fixer sweep
+  correctly re-derives "no raw rep" from NetSuite. One-time cleanup =
+  `fixer.php --commit --prod --category sales` over the JE window, then `checker.php` to confirm
+  (in current prod this was a no-op — 0 no-rep JE rows — so the importer filter is preventative).
 - **JE item dimension is NULL today; here is how to enable it later.** `itemId` is null on JE Sales rows
   because the JE import path reads item **only** via the configurable `JE_LINE_ITEM_FIELD` (currently null)
   — it does **not** read native `transactionline.item`, so even a populated native `tl.item` would NOT flow
@@ -468,11 +484,32 @@ None — Forecast2 is a single shared dataset.
   invariant, any field added to the open-order **importer** (TRUE-79162) must also be added to this
   OOI path (and ideally behind the same `forecastColumnExists` guard pattern) or `fixer.php`/`looper`
   will overwrite the importer's value to NULL on its next run.
+- **`IS NOT NULL` on a CUSTOM `transactionline` column inside a SuiteQL aggregate is a ~15× planner
+  trap — use `> 0` instead.** Filtering `custcol_sales_rep_line` (an employee-ref custom column) with
+  `AND tl.custcol_sales_rep_line IS NOT NULL` inside checker's UNION'd moment-fingerprint (the
+  `MOD(SUM(MOD(id*id,P)),P)` power-sum aggregates) made the planner scan pathologically: a full-window
+  checker run measured **~122s** with `IS NOT NULL` vs **~8s** with the semantically identical
+  `AND tl.custcol_sales_rep_line > 0` (employee internalIds are always positive). Applied to
+  `checker.php` (JE arm) and `fixer.php` (both JE queries) with inline comments warning against
+  reverting. See the SuiteQL API reference for the general rule. (Use `> 0`, or `NVL(col,0) <> 0`,
+  never `IS NOT NULL`, for a custom employee-ref `transactionline` column in a SuiteQL aggregate.)
 - These tools live in `test/@dave/` (developer tooling), but `trueup_open_orders` has been run
   against production. The `Defaults`/checkpoint mechanics of the scheduled sync are separate.
 
 ## Change history
 
+- 2026-06-30 — **Made the fixer the cleanup mechanism for no-rep JE rows + killed a ~15× SuiteQL
+  planner trap (TRUE-79968, dfranks).** The importer now keeps only rep-bearing JE lines (see the
+  sale-import doc); to let `fixer.php` purge any pre-existing no-rep `Forecast.Sales` JE rows with no
+  separate delete script, the **NS-side** JE SuiteQL in both `checker.php` and `fixer.php` was scoped
+  to rep-bearing lines while the **FC-side reads stay UNFILTERED** — a deliberate, load-bearing
+  asymmetry (NS rep-only < FC still-has-no-rep → FIND flags → existing per-`'journalEntry'` orphan
+  delete removes them; filtering the FC side too would hide them forever). A blanket
+  `DELETE WHERE salesRepEmployeeId IS NULL` is wrong (locally-null also = NS-present-but-unmapped rep,
+  which must survive). **Perf:** replaced `AND tl.custcol_sales_rep_line IS NOT NULL` with
+  `AND ... > 0` in checker (JE arm) + fixer (both JE queries) — a clean full-window checker run dropped
+  from **~122s to ~8s** purely from the predicate form, with inline anti-revert comments. Current prod
+  had 0 no-rep JE rows, so the cleanup sweep was a no-op (filter is preventative). (dfranks)
 - 2026-06-30 — **Folded JOURNAL ENTRIES into the SALES reconciliation in `fixer.php` + `checker.php`
   (TRUE-79862).** Architecture decision (dfranks): a JE is **not its own category** — JE rows live in the
   same `Forecast.Sales` table (`netsuiteTransactionType='journalEntry'`) as the four sale types, so the

package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-06-17
-owners: ["dfranks"]
+updated: 2026-06-30
+owners: ["dfranks", "bala"]
 files:
   - worker/crons/toga2/netsuite/common_sync_togasupply.php
   - worker/crons/toga2/netsuite/sync_togasupply_canon.php
@@ -17,6 +17,7 @@ related:
   - ../architecture.md
   - forecast2-netsuite-reconciliation.md
   - ../../library/features/toga2-api-client-and-bridge.md
+  - ../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
 ---
 
 ## Summary
@@ -150,6 +151,10 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
 
 ## Change history
 
+- 2026-06-30 — The IF section's `syncItemFulfillmentFromNetsuite` (`library/app/api/toga2.php`) now
+  imports **all** fulfillment statuses (`['_picked','_packed','_shipped']`, not just shipped),
+  stamping/advancing the IF stage from `c_netsuiteInternalItemFulfillmentStatus`. Full mechanics
+  live in the [IF stage lifecycle doc](../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md). (bala)
 - 2026-06-17 — Documented the full onboarding prerequisites (client→NetSuite customer link +
   per-client `c_netsuite*` custom fields) after AIG ran but imported nothing. Added AIG + Growrk
   thin wrappers, schedule entries, and dbchanges2 Parameters seeds; built AIG's

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -9,6 +9,7 @@
 | [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
 | [Record-Changed Event Publishing (_Event::publish to SQS)](features/event-publish-sqs.md) | `_Event::publish()` (in `_underscore/Event.php`) is the PHP side of the real-time event pipeline. | _underscore/Event.php |
 | [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/verify_je_classification.php, test/@dave/probe_je_accounts.php, test/@dave/probe_je_shape.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
+| [Item-Fulfillment Stage Lifecycle (picked/packed/shipped) & Order Status](features/item-fulfillment-stage-lifecycle-and-order-status.md) | Every ItemFulfillment (IF) now carries an explicit **stage** — picked → packed → shipped — resolved through `ItemFulfillmentStages → ItemFulfillmentStatuses` (m | _underscore/Model/Client/SalesOrder.php, _underscore/Model/Quad/SalesOrder.php, _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/SalesOrderStatus.php, _underscore/Model/Client/SalesOrderItem.php, _underscore/Model/Client/Item.php, _underscore/Model/Client/PurchaseOrderItem.php, library/app/api/toga2.php, dbchanges2/Client/2026-06-30a - BackfillNullStageItemFulfillmentsToShipped.sql, dbchanges2/Client/2026-06-30b - SalesOrderStatusesPickedPacked.sql, dbchanges2/Client/2026-06-30c - ItemFulfillmentStageIdNotNull.sql, dbchanges2/Client_CompassCanada/2026-06-30a - ItemFulfillmentLifecycleAndShippedBackfill.sql |
 | [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php |
 | [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
 | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |

package/knowledge/2.0/apps/_underscore/features/forecast-sale-import.md

@@ -6,7 +6,7 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-06-29
+updated: 2026-06-30
 owners: [dfranks]
 files:
   - _underscore/Component/Forecast/SaleImport/SaleImport.php
@@ -163,6 +163,19 @@ location, memo). Handler `_Worker_Netsuite_JournalEntry`: `post`/`put` → `sync
 item-line upsert/reconcile machinery (`syncLines`/`guardedInsert`/`deleteRows`).
 
 **Mapping model (durable design):**
+- **Only JE lines that carry a sales rep are imported (TRUE-79968, dfranks 2026-06-30).**
+  `buildJournalEntryRows` applies a **PER-LINE** filter **before** the (salesRep,item)
+  bucketing: a GL line whose **raw** `custcol_sales_rep_line` is empty is **skipped**
+  (`continue`). Consequences — a JE with no rep on any line yields **zero** `Forecast.Sales`
+  rows; a mixed JE keeps only its rep-bearing groups; offset/balancing lines (which omit the
+  custom column) drop out naturally. **The test is on the RAW NetSuite line field, NOT the
+  mapped local `salesRepEmployeeId`** — so a rep present in NetSuite but **not yet mapped** to
+  a local `Forecast.Employees` row **still qualifies** (its group keeps a null
+  `salesRepEmployeeId`). This is deliberate: it keeps no-rep JE noise out of Forecast revenue
+  while not silently dropping a real-but-unmapped rep's revenue. (A blanket
+  `DELETE WHERE salesRepEmployeeId IS NULL` would be **wrong** for the same reason — locally
+  null also means "rep present in NS but unmapped"; any cleanup must re-derive "no raw rep"
+  from NetSuite. See the reconciliation doc for how the fixer sweep does exactly that.)
 - **Sales rep is now LIVE on JE lines** as the custom column **`custcol_sales_rep_line`**
   (`JE_LINE_SALESREP_FIELD = 'custcol_sales_rep_line'`, wired 2026-06-29 after live Apr–May
   2026 probing). The value is a NetSuite **employee REFERENCE object** `{links, id, refName}`;
@@ -429,6 +442,20 @@ record is deleted in NetSuite.)
 - The cron's sign handling is not portable here — see Sign convention.
 
 ## Change history
+- 2026-06-30 — **JE import now keeps ONLY rep-bearing lines** (TRUE-79968, dfranks).
+  `buildJournalEntryRows` skips any GL line whose **raw** `custcol_sales_rep_line` is empty,
+  applied **per-line before** the (salesRep,item) bucketing: a no-rep JE → 0 Sales rows, a
+  mixed JE keeps only rep-bearing groups. The filter tests the **RAW NS field**, not the
+  mapped local `salesRepEmployeeId`, so a rep present in NetSuite but unmapped to
+  `Forecast.Employees` still qualifies (group stays null `salesRepEmployeeId`) — which is also
+  why a blanket `DELETE WHERE salesRepEmployeeId IS NULL` is wrong; cleanup must re-derive
+  "no raw rep" from NetSuite. No-rep cleanup is handled by the existing `fixer.php` orphan-
+  delete sweep (see the reconciliation doc), not a separate delete script — in current prod
+  there were 0 no-rep JE rows so the filter is preventative going forward.
+  **Confirmed the JE webhook handler `_Worker_Netsuite_JournalEntry`
+  (worker2/Worker/Netsuite/JournalEntry.php) holds no business logic** — post/put delegate to
+  `_Component_Forecast_SaleImport::syncJournalEntry`, delete to `removeAllJournalEntry`; editing
+  the importer in `_underscore` is the single correct entry point for JE import changes. (dfranks)
 - 2026-06-29 — **Wired JE sales-rep ingestion live** (TRUE-79862): set
   `JE_LINE_SALESREP_FIELD = 'custcol_sales_rep_line'` after live Apr–May 2026 probing
   confirmed the rep is now present on JE lines as that custom column — a NetSuite employee

package/knowledge/2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md

@@ -0,0 +1,174 @@
+---
+title: Item-Fulfillment Stage Lifecycle (picked/packed/shipped) & Order Status
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: feature
+status: active
+updated: 2026-06-30
+owners: [bala]
+files:
+  - _underscore/Model/Client/SalesOrder.php
+  - _underscore/Model/Quad/SalesOrder.php
+  - _underscore/Model/Compass/SalesOrder.php
+  - _underscore/Model/Compass/SalesOrderStatus.php
+  - _underscore/Model/Client/SalesOrderItem.php
+  - _underscore/Model/Client/Item.php
+  - _underscore/Model/Client/PurchaseOrderItem.php
+  - library/app/api/toga2.php
+  - dbchanges2/Client/2026-06-30a - BackfillNullStageItemFulfillmentsToShipped.sql
+  - dbchanges2/Client/2026-06-30b - SalesOrderStatusesPickedPacked.sql
+  - dbchanges2/Client/2026-06-30c - ItemFulfillmentStageIdNotNull.sql
+  - dbchanges2/Client_CompassCanada/2026-06-30a - ItemFulfillmentLifecycleAndShippedBackfill.sql
+related:
+  - recursive-item-fulfillments.md
+  - carrier-shipping-labels.md
+  - ../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md
+  - ../../../clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md
+  - ../../../clients/compass-usa/features/asn-to-item-fulfillment.md
+---
+
+## Summary
+
+Every ItemFulfillment (IF) now carries an explicit **stage** — picked → packed → shipped —
+resolved through `ItemFulfillmentStages → ItemFulfillmentStatuses` (matched by status `slug`).
+This makes the *picked/packed* portion of the fulfillment lifecycle visible (previously only
+*shipped* IFs were imported, so picked/packed were invisible) and drives a calculated order
+`_status` state machine. Two deliberately different status policies exist:
+
+- **Base + Quad:** order status walks the full machine — Pending → Picked → Packed →
+  Partially Fulfilled → Fulfilled — so non-Compass clients see the picked/packed lifecycle.
+- **Compass USA + Compass Canada:** all IF stages are *imported*, but order status counts
+  **shipped only** — picked/packed never advance a Compass order's status. Compass walks
+  Pending → Partially Fulfilled → Fulfilled by shipped quantity alone.
+
+Independently of order status, **inventory movement (`qtyFulfilled` / `qtyCommitted`) is
+shipped-only for ALL clients** — picked/packed quantities never count as inventory movement.
+
+## Key files / entry points
+
+- **`Model/Client/SalesOrder.php`** — base `_status` calculated field. With `salesOrderStageId`
+  NULL and a non-SA order: 0 fulfillments → Pending Fulfillment; nothing shipped → Picked (if
+  any line picked) else Packed; something shipped → Partially Fulfilled (shipped qty < ordered)
+  else Fulfilled. An explicit `salesOrderStageId` overrides everything.
+- **`Model/Quad/SalesOrder.php`** — Quad `_status`. `SA%` orders delegate to the parent; non-SA
+  orders use the inline picked/packed/partial/fulfilled machine.
+- **`Model/Compass/SalesOrder.php`** — Compass `_status`, shipped-only: both the existence
+  `COUNT` and the shipped-qty `SUM` subqueries `INNER JOIN ItemFulfillmentStages →
+  ItemFulfillmentStatuses` filtered to the shipped status slug. Shared by Compass USA + Canada.
+- **`Model/Compass/SalesOrderStatus.php`** — adds `const SLUG__PICKED = 'picked'` and
+  `SLUG__PACKED = 'packed'`.
+- **`Model/Client/{SalesOrderItem,Item,PurchaseOrderItem}.php`** — the `_qtyFulfilled` /
+  `_qtyCommitted` calculated SQL fields (5 sites) gate the IF-item sum to the shipped stage via
+  the `ItemFulfillmentStages → ItemFulfillmentStatuses` join.
+- **`library/app/api/toga2.php`** — `App_Api_Toga2::syncItemFulfillmentFromNetsuite` imports the
+  IF stage from NetSuite (see *How it works* / the per-client-sync doc).
+
+## How it works
+
+### IF stage resolution
+
+A stage is resolved by joining `ItemFulfillmentStages → ItemFulfillmentStatuses` and matching the
+status **`slug`** (`'picked'` / `'packed'` / `'shipped'`). The shipped checks in the status
+machines use the constant `_Model_Client_ItemFulfillmentStatus::SLUG__SHIPPED` (never a hardcoded
+`'shipped'`); the *picked* check stays a string literal because no `SLUG__PICKED` constant exists
+on `ItemFulfillmentStatus` (the picked/packed slug constants live on the **Compass**
+`SalesOrderStatus`, not the base IF status).
+
+> **Stage resolution must use a single-row INNER JOIN on the status `slug`, not a scalar
+> subquery on stage `name`** — a scalar subquery can throw "subquery returns more than 1 row".
+> Compass USA + Canada lack the `c_netsuite*` column on stages, so slug is the only portable key.
+
+### NetSuite import widening (change 2026-06-30)
+
+`syncItemFulfillmentFromNetsuite` previously imported only *shipped* IFs. It now imports **all**
+fulfillment statuses — filter widened to `['_picked','_packed','_shipped']`:
+- **On create** it stamps the IF stage from the NetSuite ship status
+  (`c_netsuiteInternalItemFulfillmentStatus`) instead of hardcoding shipped.
+- **On re-sync** it compares the current stage's `c_netsuiteInternalItemFulfillmentStatus` against
+  the NetSuite status and PUTs a new stage, so a fulfillment advances picked → packed → shipped.
+- Applies to **all** clients including Compass / Compass Canada (which import all stages but still
+  count shipped-only for order status — see below).
+
+### Order-status machines
+
+| Client | Statuses walked | Driven by |
+|---|---|---|
+| Base, Quad (non-SA) | Pending → Picked → Packed → Partially Fulfilled → Fulfilled | all IF stages |
+| Compass USA + Canada | Pending → Partially Fulfilled → Fulfilled | **shipped IF stage only** |
+
+Quad `SA%` orders delegate to the base machine; an explicit `salesOrderStageId` overrides the
+calculated status everywhere.
+
+### Inventory movement (all clients)
+
+`_qtyFulfilled` and `_qtyCommitted` sum only IF items whose stage is **shipped** — picked/packed
+quantities are never counted as inventory movement, regardless of the new picked/packed
+visibility in order status.
+
+## Data model / schema (dbchanges2, 2026-06-30)
+
+Fan-out (`Client/`, every client) + a Compass-Canada-specific set. Execution is alphabetical, so
+`30a` (seed + backfill) runs before `30c` (NOT NULL alter):
+
+- **`Client/2026-06-30a` — BackfillNullStageItemFulfillmentsToShipped:** idempotently seeds a
+  shipped `ItemFulfillmentStatus` + `ItemFulfillmentStage` (**unconditional** — only an
+  `INSERT…SELECT WHERE NOT EXISTS` slug guard, so fresh empty client DBs also get it), then
+  backfills NULL-stage `ItemFulfillments` to the shipped stage via `UPDATE…INNER JOIN` resolving
+  the stage through `ItemFulfillmentStatuses.slug = 'shipped' LIMIT 1` (single-row join, not a
+  scalar subquery).
+- **`Client/2026-06-30b` — SalesOrderStatusesPickedPacked:** idempotent `INSERT…SELECT WHERE NOT
+  EXISTS` of picked + packed `SalesOrderStatuses`, `colorHex` stored **without** the leading `#`
+  (picked `F5A623`, packed `4DBCC4`).
+- **`Client/2026-06-30c` — ItemFulfillmentStageIdNotNull:** `ALTER ItemFulfillments MODIFY
+  itemFulfillmentStageId INT UNSIGNED NOT NULL`. Safe because `30a` (alphabetically first)
+  guarantees every fulfillment already has a stage.
+- **`Client_CompassCanada/2026-06-30a` — ItemFulfillmentLifecycleAndShippedBackfill:** creates
+  Canada's 3 `ItemFulfillmentStatuses` (picked/packed/shipped) and 3 `ItemFulfillmentStages`, all
+  idempotent (`INSERT…SELECT WHERE NOT EXISTS` on slug for statuses; stages guarded by `NOT
+  EXISTS` of a stage for that status slug), then the same single-row-join shipped backfill. Status
+  slug is UNIQUE and stage name is UNIQUE, so the guards align with the unique keys.
+
+## Migration correctness rules (review-driven, baked into the above)
+
+- **Seed migrations must be retry-safe:** use `INSERT…SELECT WHERE NOT EXISTS`, and resolve the
+  shipped stage via a single-row `INNER JOIN` on `slug` — never a scalar subquery on `name`
+  (which can throw "subquery returns more than 1 row" when more than one stage shares a name).
+- **The shipped seed must be UNCONDITIONAL** (no `EXISTS`-fulfillments gate) so fresh client DBs
+  get a stage *before* the NOT NULL alter and *before* any future fulfillment insert.
+
+## Gotchas / known issues
+
+- **NULL-SUM correctness bug (fixed):** the base shipped-qty subquery returned NULL when no rows
+  matched (empty `SUM` = NULL), and `NULL < orderedQty` is not true, so empty orders read
+  **Fulfilled**. Wrap the shipped-qty subquery with `IFNULL(…, 0)` before comparing to the ordered
+  quantity.
+- **Compass imports all stages but counts shipped-only.** Do not assume importing picked/packed
+  IFs changes a Compass order's status — by design it does not (the Compass supply UI must show
+  shipped-only progression). This supersedes the old "stages 1/2 are unused in prod" assumption in
+  the Compass order-lifecycle workflow.
+- The base/Compass picked check is a string literal `'picked'` (no `SLUG__PICKED` on
+  `_Model_Client_ItemFulfillmentStatus`); only the shipped check uses a constant. Adding a base IF
+  picked/packed constant would let the literal be removed.
+
+## Change history
+- 2026-06-30 — Built the IF stage lifecycle end-to-end: NetSuite sync imports all
+  picked/packed/shipped statuses and advances stages on re-sync; base + Quad order `_status`
+  picked/packed/partial/fulfilled machine; Compass USA + Canada order status shipped-only;
+  `qtyFulfilled`/`qtyCommitted` shipped-only for all clients; dbchanges2 stage seed + backfill +
+  `itemFulfillmentStageId` NOT NULL (Client fan-out + Compass Canada). Fixed the NULL-SUM
+  Fulfilled bug (`IFNULL`). Baked in the retry-safe seed + single-row-join + unconditional-seed
+  migration rules. (bala)
+
+## Related docs
+- [Recursive Item Fulfillments](recursive-item-fulfillments.md) — the engine that mirrors IFs (and
+  now their stages) up the chain.
+- [Carrier Shipping Labels & NetSuite IF](carrier-shipping-labels.md) — IF creation on the
+  fulfill-and-ship path.
+- [NetSuite → TOGa Supply Per-Client Sync](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)
+  — the importer surface; `syncItemFulfillmentFromNetsuite` is the stage-stamping entry point.
+- [Compass Order Lifecycle & Data-Integrity Invariants](../../../clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md)
+  — the stage-3-Shipped invariant updated by this work.
+- [Compass ASN → ItemFulfillment Auto-Creation](../../../clients/compass-usa/features/asn-to-item-fulfillment.md)
+  — ASN-created IFs now resolve the shipped stage.

package/knowledge/INDEX.md

@@ -16,7 +16,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 20 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 21 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 24 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 7 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)

package/knowledge/clients/compass-canada/profile.md

@@ -20,6 +20,7 @@ files: []
 related:
   - ../compass-usa/profile.md
   - ../../2.0/apps/toga2-commerce/features/expedited-shipping-gating.md
+  - ../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
 ---
 
 ## Summary
@@ -56,4 +57,8 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
   values, which were then cleaned to English-only. See
   [Assortment Name Translation](../../2.0/apps/_underscore/features/assortment-name-translation.md).
 - The 2026-06-08 ASN → ItemFulfillment work was for **Compass USA**, not Compass Canada.
+- **Order status is shipped-only**, same as Compass USA (shared `_Model_Compass_SalesOrder`).
+  Compass Canada's own IF lifecycle stages (picked/packed/shipped) + shipped backfill are seeded by
+  `dbchanges2/Client_CompassCanada/2026-06-30a - ItemFulfillmentLifecycleAndShippedBackfill.sql`.
+  See [IF stage lifecycle & order status](../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md).
 - Related: [Compass USA](../compass-usa/profile.md).

package/knowledge/clients/compass-usa/features/asn-to-item-fulfillment.md

@@ -5,8 +5,8 @@ project: _Underscore
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-06-18
-owners: [jcardinal]
+updated: 2026-06-30
+owners: [jcardinal, bala]
 files:
   - _underscore/Model/Compass/AdvanceShippingNotice.php
   - _underscore/Model/Compass/PurchaseOrder.php
@@ -20,6 +20,7 @@ files:
   - dbchanges2/Client_Compass/2026-06-18b - CleanupSA132881DuplicateTracking.sql
 related:
   - ../../../2.0/apps/_underscore/features/recursive-item-fulfillments.md
+  - ../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
 ---
 
 ## Summary
@@ -149,6 +150,11 @@ Compass Canada (`Model/Compass/Canada/`) is a separate sub-client. The Compass c
 - Interceptor is **DB-driven**: `postPost` does nothing without the Compass
   `ApiPayloadInterceptors` rows in that env.
 - The IF is named after the SO number; an already-existing IF with that number is reused.
+- **ASN-created fulfillments land at the shipped stage (2026-06-30).** `resolveOrCreateItemFulfillment`
+  resolves the shipped `ItemFulfillmentStage` by status `slug` (portable across Compass USA + Canada,
+  which lack the `c_netsuite` column on stages) and sets `itemFulfillmentStage` on the payload — an
+  ASN means the goods shipped, so its IF must not be stage-less. See the
+  [IF stage lifecycle](../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md) doc.
 - **cXML SQL-injection hardening (2026-06-11):** the `ShipNoticeRequest` path interpolated the
   cXML `Sender` Identity / SharedSecret and the OrderReference `orderID` (purchase order
   number) straight into queries. These are now passed through `_Database::escape()`.
@@ -201,6 +207,10 @@ Compass Canada (`Model/Compass/Canada/`) is a separate sub-client. The Compass c
 
 ## Change history
 Dated one-liners, newest first.
+- 2026-06-30 — `resolveOrCreateItemFulfillment` now resolves the shipped `ItemFulfillmentStage` by
+  status slug and sets `itemFulfillmentStage` on the IF payload, so ASN-created fulfillments land
+  shipped (not stage-less) on both Compass USA + Canada. Part of the platform-wide IF stage
+  lifecycle work. (bala)
 - 2026-06-18 — Reconcile path now skips IFIs that already carry item-level tracking
   (`itemFulfillmentItemHasTracking()`), stopping duplicate/different tracking numbers from
   vendor ASN re-transmissions piling onto fully-fulfilled lines (qty-1 SA132898 / USB31CC1 had

package/knowledge/clients/compass-usa/profile.md

@@ -19,6 +19,7 @@ files: []
 related:
   - features/asn-to-item-fulfillment.md
   - ../../2.0/apps/toga2-commerce/features/expedited-shipping-gating.md
+  - ../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
 ---
 
 ## Summary
@@ -57,5 +58,10 @@ separate, related client (see its own profile).
   `_Model_Compass_AdvanceShippingNotice::postPost` handler.
 
 ## Notes
+- **Order status is shipped-only (2026-06-30).** Compass imports all IF stages
+  (picked/packed/shipped) but its order `_status` (`_underscore/Model/Compass/SalesOrder.php`,
+  shared by Compass USA + Canada) counts **shipped only** — picked/packed never advance a Compass
+  order. Contrast Quad, which gets the full picked/packed machine. See
+  [IF stage lifecycle & order status](../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md).
 - Built on the shared 2.0 Recursive Item Fulfillments engine (upstream mirroring).
 - This profile is a starting point; expand as more Compass-specific behavior is captured.

package/knowledge/clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md

@@ -5,8 +5,8 @@ project: _Underscore
 client: compass-usa
 type: workflow
 status: active
-updated: 2026-06-12
-owners: ["jcardinal"]
+updated: 2026-06-30
+owners: ["jcardinal", "bala"]
 files: []
 related:
   - clients/compass-usa/profile.md
@@ -14,6 +14,7 @@ related:
   - clients/compass-usa/features/mits-po-to-so-item-linking.md
   - clients/compass-usa/features/item-fulfillment-tracking-tableview.md
   - clients/compass-usa/features/mits-po-transmission-to-vendors.md
+  - 2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
 ---
 
 ## Summary
@@ -82,8 +83,12 @@ one-upstream for bundles.
    `AdvanceShippingNotices_TrackingNumbers`) and the IF side
    (`ItemFulfillmentItemUnits_TrackingNumbers` → `ItemFulfillmentItems_TrackingNumbers` →
    `ItemFulfillments_TrackingNumbers`). Always link existing `TrackingNumbers`; never create them.
-4. **Legacy stage:** a shipped IF (has items) with `itemFulfillmentStageId IS NULL` should be set to 3
-   ("Shipped"). Stage 3 = Shipped; stages 1/2 are unused in prod.
+4. **Stage is now mandatory:** as of 2026-06-30 `ItemFulfillments.itemFulfillmentStageId` is NOT NULL
+   and the picked/packed/shipped stages are all in use (resolved by status `slug`, not the legacy
+   numeric "stage 3 = Shipped"; stages 1/2 are no longer "unused"). The 2026-06-30a migration backfills
+   any NULL-stage IF to the shipped stage. Compass order `_status` still counts **shipped only** even
+   though all stages are imported — see the
+   [IF stage lifecycle doc](../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md).
 5. **Mirror back-links:** every IF on a non-source SO (ODP/Agilant) should have
    `upstreamItemFulfillmentId` set, and each of its items `upstreamItemFulfillmentItemId` set.
 
@@ -114,5 +119,8 @@ runtime workflow.
 - High-multiplier over-fulfillment (5×–20×) does not fit the split-PO spurious-link pattern — separate cause.
 
 ## Change history
+- 2026-06-30 — Updated the IF-stage invariant: `itemFulfillmentStageId` is now NOT NULL and
+  picked/packed/shipped stages are all in use (resolved by slug), superseding "stage 3 = Shipped;
+  stages 1/2 unused". Compass order status remains shipped-only. (bala)
 - 2026-06-12 — Documented full lifecycle, both fulfillment flows, verified upstream-mirror direction,
   the expected-shape invariants, and the retrofix tool (jcardinal)

package/knowledge/clients/quad/profile.md

@@ -18,6 +18,7 @@ owners: ["jcardinal", "bala"]
 files: []
 related:
   - ../../2.0/apps/_underscore/features/tracking-number-bridges.md
+  - ../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
 ---
 
 ## Summary
@@ -42,6 +43,11 @@ Client-specific DB change-sets live in `dbchanges2/Client_Quad/`.
   [Tracking-Number Bridge Migration](../../2.0/apps/_underscore/features/tracking-number-bridges.md).
 
 ## Notes
+- **Order status: full picked/packed lifecycle (2026-06-30).** Quad gets the full
+  picked → packed → partially-fulfilled → fulfilled order `_status` machine
+  (`_underscore/Model/Quad/SalesOrder.php`): non-SA orders use the inline machine, `SA%` orders
+  delegate to the base `_Model_Client_SalesOrder`. Contrast Compass, which is shipped-only. See
+  [IF stage lifecycle & order status](../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md).
 - Tracking data: record 318 (item-level) is currently empty for this client; their tracking
   populates the IF/shipment level (record 317, ~98% coverage) and unit level (record 319). The
   rebuilt views use 318 (per the Compass pattern) and will show tracking once item-level

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.255",
+  "version": "1.0.257",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",