toga-ai 1.0.919 → 1.0.920

package/knowledge/1.0/apps/library/INDEX.md

@@ -17,7 +17,7 @@
 | [Error Capture in 1.0 (App_Error_Capture → shared 2.0 Logs DB)](features/error-capture-1-0.md) | 1.0 error-reporting into the shared 2.0 Logs DB (`App_Error_Capture::captureException`), mirroring 2.0's Issue/Event; open when adding a 1.0 capture path or deb |
 | [HTTP 500 Error Monitor (App_SystemMonitor_500Error) — email-only count; diagnose via error.id](features/http-500-error-monitor.md) | 1.0 monitor `App_SystemMonitor_500Error`: counts `Logs.Api` responseCode=500 in the last 5 min and emails devteam; no ClickUp. |
 | [1.0 MVC Page Pattern & New-App Skeleton](features/mvc-page-pattern-and-app-skeleton.md) | Recipe for a new 1.0 (`App_`) app and for adding pages: folder-based MVC routing, page lifecycle, models, multi-DB config; open when standing up a 1.0 app or ad |
-| [NetSuite Closed sales orders — keep if related records, delete if none](features/netsuite-closed-order-rule.md) | How the NetSuite → TOGa Supply sync treats a Closed SO (all clients): keep + import when anything was created from it, delete when nothing was; plus the ACL the |
+| [NetSuite closed orders — drop only when every line is closed and nothing was created from it](features/netsuite-closed-order-rule.md) | When the NetSuite → TOGa Supply sync drops (skips + deletes) an order (all clients): only when EVERY item line is closed AND nothing was created from it; plus t |
 | [NetSuite File Cabinet Content Retrieval via RESTlet (fetchInvoiceFile)](features/netsuite-filecabinet-restlet.md) | How 1.0 pulls NetSuite File Cabinet binary content (invoice PDFs) via a SuiteScript RESTlet (`fetchInvoiceFile`); open when fetching NetSuite file bytes over RE |
 | [assetType from NetSuite itemtype during Item Sync (opt-in per client)](features/netsuite-item-assettype-sync.md) | Stamp `Items.assetTypeId` from NetSuite `item.itemtype` during item sync (opt-in per client, fill-NULL-only); open before enabling asset-type mapping for a clie |
 | [NetSuite classification → ItemClasses sync (opt-in per client)](features/netsuite-item-class-sync.md) | `App_Api_Toga2::getCreateItemClass()` mirrors NetSuite's **classification tree** into the client's `ItemClasses` table and stamps `Items.itemClassId`. |

package/knowledge/1.0/apps/library/features/netsuite-closed-order-rule.md

@@ -1,5 +1,5 @@
 ---
-title: "NetSuite Closed sales orders — keep if related records, delete if none"
+title: "NetSuite closed orders — drop only when every line is closed and nothing was created from it"
 framework: "1.0"
 repo: library
 project: Library
@@ -14,32 +14,45 @@ files:
   - worker/crons/toga2/netsuite/common_sync_togasupply.php
   - worker/crons/toga2/netsuite/process_netsuite_order.php
   - worker/crons/toga2/netsuite/delete_closed_netsuite_orders.php
+  - _underscore/Model/Client/SalesOrderItem.php
+  - _underscore/Model/Client/PurchaseOrderItem.php
+  - dbchanges2/Core/2026-09-30a - OrderItemIsOpenRecordFields.sql
+  - dbchanges2/Client/2026-09-30a - OrderItemIsOpen.sql
 related:
   - ../../worker/features/netsuite-togasupply-per-client-sync.md
   - ../../worker/workflows/reprocessing-one-netsuite-order.md
   - ../../../2.0/apps/_underscore/features/acl-permission-chain.md
+  - ../../../2.0/apps/dbchanges2/workflows/verifying-a-migration-ran.md
   - ../../../clients/compass-usa/profile.md
 ---
 
-How the NetSuite → TOGa Supply sync treats a Closed SO (all clients): keep + import when anything was created from it, delete when nothing was; plus the ACL the delete needs.
+When the NetSuite → TOGa Supply sync drops (skips + deletes) an order (all clients): only when EVERY item line is closed AND nothing was created from it; plus the line `isOpen` field and the ACL the delete needs.
 
 ## Summary
-- **Rule (2026-09-29, all NetSuite sync clients):** a NetSuite SO in status `Closed`:
-  - **with** related records (any transaction whose `createdfrom` is the SO — fulfillment, invoice, PO, return) = **completed**. Keep it and import it like any order, shipments included.
-  - **with none** = **duplicate**. Never import it; delete the TOGa order if it was imported before it closed.
-- **Replaces the 2026-09-23 decision "delete every Closed order".** That rule skipped closed orders that really shipped, so TOGa on-hand stayed above NetSuite.
+- **Rule v3 (2026-09-30, all NetSuite sync clients).** The SO header status `Closed` is **not used**. An order is dropped only when both are true:
+  1. every item line has `transactionline.isclosed = 'T'`, and
+  2. no transaction was created from it (no `transactionline` with `createdfrom` = the order).
+- Every other case imports: some lines open (e.g. 9 open + 1 closed), all closed but shipped/invoiced/has a PO, all open, zero item lines.
+- **Why the related-records part:** NetSuite had 132 SOs since 2025 with every line closed but real shipments (Staples 43, GroWrk 21, ODP Veyer 14, Smoke Stacks 11, NYCDOE 5, Endeavor 5, Elite 2 — e.g. Elite SO 269622 → shipment 304666). Deleting them drops real stock movements.
+- Replaces 2026-09-29 "Closed status + related records = keep" and 2026-09-23 "delete every Closed".
+- Status: reviewed clean, **not yet deployed**.
 
 ## How it works
-- `App_Api_Netsuite_Rest::hasRelatedTransactions(int $orderId)` — SuiteQL `COUNT(*)` on `transactionline` where `createdfrom = <id>` and `transaction <> <id>`.
-- `App_Api_Toga2::isNetsuiteOrderClosedWithoutRelatedRecords($nsOrder)` = status `Closed` AND no related transactions. Used by the SO loop in `common_sync_togasupply.php`, `syncTransferOrderFromNetsuite`, and `process_netsuite_order.php`.
-- `App_Api_Toga2::deleteClosedOrderFromNetsuite()` has its own **KEEP guard** (re-checks `hasRelatedTransactions`, logs `KEEP`, returns false). So every caller, including the one-time tool `delete_closed_netsuite_orders.php` (dry run unless `--execute`), is safe.
+- **Item line** = `mainline = 'F'`, `item > 0`, not ShipItem/TaxItem/TaxGroup/Discount/EndGroup/BeginGroup/Subtotal/Description/Markup/Payment. One home: `App_Api_Netsuite_Rest::SALES_ORDER_ITEM_LINE_FILTER`.
+- `rest.php`: SO shim and both PO shims read `tl.isclosed` → `$lineShim->isClosed`. `hasOpenLines(int $orderId)`, `hasRelatedTransactions(int $orderId)`. `listFullyClosedSalesOrderIdsWithoutRelatedRecords()` (replaces `listClosedSalesOrderIds`) feeds the bulk tool.
+- `App_Api_Toga2::isNetsuiteOrderFullyClosedWithoutRelatedRecords($nsOrder)` is the one check (replaces `isNetsuiteOrderClosed` + `isNetsuiteOrderClosedWithoutRelatedRecords`). Used at 4 places: SO sync, PO source order, $0-SO Transfer Orders, fulfillment cascade.
+- `deleteClosedOrderFromNetsuite()` **KEEP guard**: re-checks live and keeps (logs `KEEP`, returns false) if `hasOpenLines` OR `hasRelatedTransactions`. Live re-check is needed because the bulk tool `delete_closed_netsuite_orders.php` (dry run unless `--execute`) passes an order with no lines.
 - The delete removes the SalesOrder/TransferOrder with its lines, invoices, fulfillments (units, tracking bridges) and bridge rows. Records on the other side of a bridge (PO, other SO, TrackingNumber, Unit) are never deleted — only the link row. Fails loud; a half-done delete is finished by the next run.
-- Backfill for shipped closed orders that were skipped under the old rule: re-run their fulfillments with `process_netsuite_records.php` (see [re-processing workflow](../../worker/workflows/reprocessing-one-netsuite-order.md)).
+- **Line field `isOpen`** on `SalesOrderItems` (after `preferredVendorId`) and `PurchaseOrderItems` (after `vendorItemId`): `TINYINT UNSIGNED NOT NULL DEFAULT 1`; model `[self::FIELD_BOOLEAN, self::FIELDOPT_DEFAULT => true]`. Core.RecordFields 2634 (record 15) / 2635 (record 18), BOOLEAN. Client ACL: API role read+write, Base role read, roles by name. The sync writes 0/1 from the NS line (`property_exists` + `(int)` change check). Old rows stay 1 until their order re-syncs.
+- Backfill shipped orders skipped under old rules: re-run their fulfillments with `process_netsuite_records.php` ([re-processing workflow](../../worker/workflows/reprocessing-one-netsuite-order.md)).
 
 ## Gotchas
-- **The delete reads every route in `CLOSED_ORDER_DELETE_ROUTES` (toga2.php) — the client's API role (roleId 3) needs read on ALL of them:** record permission + an expression on each logic group + field permission (at least `uuid`). One gap throws, and because it runs in the SO loop, **SALES_ORDERS freezes** (and the sections after it). Find gaps per client with `App_Api_Toga2::checkClosedOrderDeleteAccess()` (run by `delete_closed_netsuite_orders.php`; it lists every gap in one run). ACL layers: [acl-permission-chain](../../../2.0/apps/_underscore/features/acl-permission-chain.md).
-  - Worked example (Compass, 2026-09-29): records 312/313 (transfer-orders) had logic groups with no expression; 257 (entitlement-sales-orders) and 254 (sales-order-payments) had no `AclRecordPermissions`, then no `AclFieldPermissions` (`EZ-2` on `uuid`). Fix: `dbchanges2/Client_Compass/2026-09-29a - TransferOrderAclExpressions.sql`. Then reset `NETSUITE_EXECUTION_MODE_SALES_ORDERS` to `864000-IDLE`.
+- **In NetSuite a shipped/fulfilled line usually stays `isclosed = 'F'` when the order is closed**; only unfulfilled lines flip to `'T'` (e.g. closed SO 279721 has 1 open line). So a Closed header does not mean "all lines closed".
+- **`Client/2026-09-30a` ALTER is not guarded** — a second run fails on duplicate column. The INSERTs are guarded. Deploy order + per-DB check: [verifying a migration ran](../../../2.0/apps/dbchanges2/workflows/verifying-a-migration-ran.md).
+- **The delete reads every route in `CLOSED_ORDER_DELETE_ROUTES` (toga2.php) — the client's API role (roleId 3) needs read on ALL of them:** record permission + an expression on each logic group + field permission (at least `uuid`). One gap throws in the SO loop and **SALES_ORDERS freezes**. Find gaps with `App_Api_Toga2::checkClosedOrderDeleteAccess()` (run by `delete_closed_netsuite_orders.php`; lists every gap in one run). ACL layers: [acl-permission-chain](../../../2.0/apps/_underscore/features/acl-permission-chain.md).
+  - Compass, 2026-09-29: records 312/313 had logic groups with no expression; 257 and 254 had no `AclRecordPermissions`, then no `AclFieldPermissions` (`EZ-2` on `uuid`). Fix: `dbchanges2/Client_Compass/2026-09-29a - TransferOrderAclExpressions.sql`, then reset `NETSUITE_EXECUTION_MODE_SALES_ORDERS` to `864000-IDLE`.
 - Adding a route to `CLOSED_ORDER_DELETE_ROUTES` = a new ACL need on every client. Run the access check fleet-wide before deploy.
 
 ## Change history
-- 2026-09-30 — Created. Closed SO with related records is now kept + imported; only Closed with none is deleted (replaces 2026-09-23 "delete every Closed"). Added KEEP guard in the delete and the ACL gotcha from the Compass SALES_ORDERS freeze. (rgirish)
+- 2026-09-30 — Created: Closed SO with related records kept; only Closed with none deleted. KEEP guard + Compass ACL gotcha. (rgirish)
+- 2026-09-30 — Rule v3: header status dropped; drop only when every item line `isclosed = 'T'` AND no related records. Added line `isOpen` field. (rgirish)

package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

@@ -511,13 +511,13 @@ record; every section behaves the way SALES_ORDERS always did (throw → freeze
 poison record re-tried each run → go read the error). This is an application of the universal
 coding-style rule *never catch-and-silently-ignore*, not a new engine-specific policy.
 
-### Closed NetSuite orders (rule 2026-09-29)
+### Closed NetSuite orders (rule v3, 2026-09-30)
 
-A Closed SO **with** related records (fulfillment, invoice, PO…) is completed — imported like any order.
-A Closed SO with **none** is a duplicate — never imported, deleted if already in TOGa. Replaces the
-2026-09-23 "delete every Closed" rule and the 2026-09-08 stage-only `updateTransferOrderStageToClosed`
-(removed). Full rule, KEEP guard and the ACL the delete needs (a gap freezes SALES_ORDERS):
-[netsuite-closed-order-rule](../../library/features/netsuite-closed-order-rule.md).
+An order is dropped (skipped, deleted if already in TOGa) only when **every item line** is closed in
+NetSuite (`isclosed = 'T'`) **and** nothing was created from it. The header `Closed` status is not
+used. Replaces the 2026-09-29 and 2026-09-23 rules and the 2026-09-08 `updateTransferOrderStageToClosed`
+(removed). Full rule, KEEP guard, line `isOpen` field and the ACL the delete needs (a gap freezes
+SALES_ORDERS): [netsuite-closed-order-rule](../../library/features/netsuite-closed-order-rule.md).
 
 ### ⚠ An OUT-OF-SCOPE transfer-order destination customer wedged INVENTORY_ADJUSTMENTS — the importer now SKIPS it (fixed + deployed 2026-09-14)
 
@@ -2044,7 +2044,7 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
   [NetSuite Sync Alert Monitor](../../library/features/netsuite-sync-alert-monitor.md).
 
 ## Change history
-- 2026-09-30 — Closed SO rule: keep if related records, delete only if none ([doc](../../library/features/netsuite-closed-order-rule.md)). Sync never auto-creates warehouse Locations (`throwIfUnknownWarehouse`); unit location from the line, not `custitemnumber_location`. Added `process_netsuite_records.php` and the stuck-sync diagnosis path. (rgirish)
+- 2026-09-30 — Closed-order rule v3: drop only when every line is closed and nothing was created from it ([doc](../../library/features/netsuite-closed-order-rule.md)). Sync never auto-creates warehouse Locations (`throwIfUnknownWarehouse`); unit location from the line, not `custitemnumber_location`. Added `process_netsuite_records.php` and the stuck-sync diagnosis path. (rgirish)
 - 2026-09-24 — TO↔PO links: two-direction reconcile (vendor POs no longer upstream; retireStale moves downstream SO links to 326/330); fixed SO downstream line-link deletes on PO re-sync; new `process_netsuite_order.php` + `NETSUITE_SYNC_LOOKUPS_ONLY`. (jcardinal)
 - 2026-09-22 — **The `Logs.Issue` 589 doom-loop's framework root cause is now fully explained and has
   tooling, but this sync is NOT yet patched.** Three independent reasons the 2006 reconnect was dead

package/knowledge/1.0/apps/worker/workflows/reprocessing-one-netsuite-order.md

@@ -41,7 +41,7 @@ Steps the script runs:
 2. Defines `NETSUITE_SYNC_LOOKUPS_ONLY` and requires the wrapper → `common_sync_togasupply.php` builds config + lookups, then returns before any section.
 3. `--tranid` → `App_Api_Netsuite_Rest::findSalesOrderIdByTranId()` (format-validated).
 4. Refuses an order whose customer / end-user customer does not map to this client.
-5. Closed with no related records → `deleteClosedOrderFromNetsuite` ([rule](../../library/features/netsuite-closed-order-rule.md)); Closed with related records syncs like any order; TO (detection mode) → `syncTransferOrderFromNetsuite` (includes the [two-direction PO link reconcile](../../library/features/transfer-order-purchase-order-link-reconcile.md)); else → `syncSalesOrderFromNetsuite`.
+5. Every line closed and no related records → `deleteClosedOrderFromNetsuite` ([rule](../../library/features/netsuite-closed-order-rule.md)); TO (detection mode) → `syncTransferOrderFromNetsuite` (includes the [two-direction PO link reconcile](../../library/features/transfer-order-purchase-order-link-reconcile.md)); else → `syncSalesOrderFromNetsuite`.
 6. `listPurchaseOrderIdsCreatedFromOrder()` → `syncPurchaseOrderFromNetsuite` for every vendor PO created from the order.
 
 Verify: grep the PHP error log for `reconcileTransferOrderPurchaseOrderLinks` and `syncPurchaseOrderFromNetsuite: DELETE`.
@@ -60,5 +60,5 @@ php crons/toga2/netsuite/process_netsuite_records.php <client> --fulfillment=<id
 - Needs `IS_ENABLED_INTEGRATION_TRANSFER_ORDERS` true in the wrapper for TO routing; otherwise the order syncs as a sales order.
 
 ## Change history
-- 2026-09-30 — Added `process_netsuite_records.php` (fulfillments/receipts by id); Closed routing follows the new keep/delete rule. (rgirish)
+- 2026-09-30 — Added `process_netsuite_records.php` (fulfillments/receipts by id); Closed routing follows the closed-order rule (v3: line-level). (rgirish)
 - 2026-09-24 — Created; prod-verified on NYCHH TO 11. (jcardinal)

package/knowledge/2.0/apps/dbchanges2/INDEX.md

@@ -12,4 +12,4 @@
 | [Local vs prod MySQL config parity — why “it passed locally” is not evidence](workflows/local-vs-prod-mysql-config-parity.md) | Per-machine MySQL server-config differences that make a migration pass locally and fail on deploy; open before trusting any local migration run. |
 | [Repairing non-prod metadata drift (works in prod, broken in beta/dev-sandbox)](workflows/nonprod-metadata-drift-repair.md) | Repair procedure when a feature works in prod but not beta/dev-sandbox because that environment's metadata rows drifted behind; open on a "works in prod, broken |
 | [Targeting the right database (tenant restores, and running a change file against the wrong DB)](workflows/targeting-the-right-database.md) | How to confirm you are pointed at the intended DB before a dump/restore or running a change file; open before any tenant restore or reset. |
-| [Verifying whether a dbchanges2 migration actually ran in an environment](workflows/verifying-a-migration-ran.md) | Prove whether a given dbchanges2 file has run in an environment; open when a "did this apply here?" question comes up. |
+| [Verifying whether a dbchanges2 migration actually ran in an environment](workflows/verifying-a-migration-ran.md) | Prove whether a given dbchanges2 file has run in an environment, and the deploy order for a new column a 2.0 model reads; open when a "did this apply here?" que |

package/knowledge/2.0/apps/dbchanges2/workflows/verifying-a-migration-ran.md

@@ -6,17 +6,19 @@ project: Database Changes
 client: shared
 type: workflow
 status: active
-updated: 2026-09-16
-owners: [apeterson]
+updated: 2026-09-30
+owners: [apeterson, rgirish]
 files:
   - Client/2026-08-11b - SalesOrderPurchaseOrdersField.sql
   - Client/2026-08-13 - UnitsForItemsPO_SalesOrderItemsJoin.sql
+  - Client/2026-09-30a - OrderItemIsOpen.sql
 related:
   - ../architecture.md
   - ../features/surface-layer-schema.md
+  - ./framework-branch-ahead-of-schema.md
 ---
 
-Prove whether a given dbchanges2 file has run in an environment; open when a "did this apply here?" question comes up.
+Prove whether a given dbchanges2 file has run in an environment, and the deploy order for a new column a 2.0 model reads; open when a "did this apply here?" question comes up or before deploying a model with a new column.
 
 ## Summary
 
@@ -29,11 +31,32 @@ dbchanges2 has no execution ledger you can query — "did this file run here?" m
 3. **For a guarded file, find its distinguishing side-effect.** Pick something a pre-existing row would *not* already have — typically the ACL grant pair it adds. Worked example: `Client/2026-08-11b - SalesOrderPurchaseOrdersField.sql` confirmed on `Client_Nychh` by the grant pair **exactly Base + API, `isWritable = 0`**; Compass already had `_purchaseOrders` as `CustomRecordFields` id 77 under a *different* uuid, so the field row alone proved nothing.
 4. **State the conclusion as "the effect is present in env X"**, not "the file ran", unless step 1 succeeded.
 
+### New column a 2.0 model reads — deploy order
+
+The model SELECTs every declared property, so one tenant DB without the column = 1054 on every request there.
+1. Run the Core file (RecordFields).
+2. Run the Client ALTER on **every** `Client_*` DB.
+3. Verify per DB — this must return 0 rows:
+   ```sql
+   SELECT t.TABLE_SCHEMA, t.TABLE_NAME
+   FROM information_schema.TABLES t
+   WHERE t.TABLE_SCHEMA LIKE 'Client\_%'
+     AND t.TABLE_NAME IN ('<Table1>', '<Table2>')
+     AND NOT EXISTS (
+       SELECT 1 FROM information_schema.COLUMNS c
+       WHERE c.TABLE_SCHEMA = t.TABLE_SCHEMA AND c.TABLE_NAME = t.TABLE_NAME AND c.COLUMN_NAME = '<column>'
+     );
+   ```
+   Backup copies (e.g. `Client_Compass_20260831_120000`) show up here; they are not live and can be left.
+4. Only then deploy `_underscore`.
+
 ## Gotchas
 
 - **A guarded `INSERT ... WHERE NOT EXISTS` is a NO-OP where the row already exists under a different uuid.** So *row absent* ≠ "the file did not run", and *row present* ≠ "this file created it". Both directions of the naive inference are unsound.
 - Seed rows created with `UUID()` differ per environment, so cross-environment uuid comparison is meaningless. Compare **ids and structure**, not uuids. (Same reason surface migrations key on the literal `id` — see [Surface Layer Schema](../features/surface-layer-schema.md).)
 - Fan-out folders (`Client/`) run per tenant, so a file can be present in one client schema and genuinely absent in another. Always verify against the specific schema, not "the environment".
+- **The migration runner can report success while skipping DBs.** 2026-09-30 `isOpen` rollout: `Client_Apiary`, `Client_Craftex` (both tables) and `Client_True` (`PurchaseOrderItems`) were missed; after `_underscore` deployed, prod raised Logs.Issue 895 `Unknown column 'isOpen' in 'field list'` 49 times until fixed with schema-qualified ALTERs. Always run the verify query above.
+- MySQL 8.0.39 (Dynamic row format): `ADD COLUMN ... AFTER` was instant even on 565k rows — size is not a reason to delay.
 
 ## Related
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.919",
+  "version": "1.0.920",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",