toga-ai 1.0.847 → 1.0.848

package/knowledge/1.0/apps/library/features/netsuite-suiteql-rest-shim.md

@@ -6,7 +6,7 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-09-17
+updated: 2026-09-21
 owners: [dfranks, jcardinal, kyalamarthi, rgirish]
 files:
   - library/app/api/netsuite/rest.php
@@ -57,6 +57,7 @@ Field semantics (verified on the live account):
 - **Line number = `transactionline.id`, NOT `linesequencenumber`.** `tl.id` equals the REST `item[].line` value (the stable SOAP-era line key); `linesequencenumber` is physical order and re-maps/collides when SOAP-era records are edited. Verified: sales orders 106 orders / 344 lines (2019–2029, incl. edited orders with line-id gaps), invoices + cash sales 62/62. `listSales` historically resolved line via `uniquekey → REST lineUniqueKey → line`; `tl.id` gives the same answer in bulk without the per-id GET. **Re-confirmed at scale 2026-08-27:** the two columns diverge on **2,061 of 4,387** lines (~47%), and `Forecast.OpenOrderItems.lineNumber` (which stores REST `$line->line`) matched `transactionline.id` on **99 of 99** randomly sampled prod rows plus all 7 multi-root orders, with zero join misses. So keying a NetSuite ↔ Forecast line join on `linesequencenumber` silently mismatches about half the book — always join on `transactionline.id`.
 - **ShipItem lines have `costestimate = NULL`** (100% of 42K+ lines, 2025–26). In any profit expression `SUM(-foreignamount + costestimate)`, a NULL cost makes the whole term NULL and SQL **drops the row from the SUM** — so shipping *revenue* still counts but shipping *profit* does not, silently understating NS profit. Always `NVL(tl.costestimate, 0)`. (Caused a fake $663K "gap" in the audit tools — see the reconciliation doc.)
 - **Cash refunds & credit memos carry extra COGS/inventory lines.** SuiteQL `transactionline` with `mainline='F' AND item>0` returns, per item, the customer line **plus** a `CUSTOMERRETURN` and an `ASSET` posting (all `item>0`), which REST's `item[].items` sublist omits. They are distinguished by **`tl.iscogs = 'T'`** (and `accountinglinetype` IN `CUSTOMERRETURN`/`ASSET`). Filter **`tl.iscogs = 'F'`** to get exactly the customer-facing lines. Invoices/cash sales do not have this (0 extra lines). Without the filter, bulk imports duplicate refund/memo lines.
+- **⚠ `listSales()` takes a `$recordTypes` allowlist — pass `RECORD_TYPE_*` CONSTANTS, never raw SuiteQL codes (2026-09-21).** `listSales(...)`'s trailing `array $recordTypes` **defaults to all four** `RECORD_TYPE_INVOICE`/`_CASH_SALE`/`_CASH_REFUND`/`_CREDIT_MEMO`, which the **forecast import** (`worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php`, `periodic_forecast_discrepancy_fix.php`) relies on. `listSales` maps each constant to its SuiteQL code internally (`RECORD_TYPE_INVOICE`→`CustInvc`, `_CASH_SALE`→`CashSale`, `_CASH_REFUND`→`CashRfnd`, `_CREDIT_MEMO`→`CustCred`) and **throws on an unknown/empty type**. The **togasupply sync passes `[RECORD_TYPE_INVOICE]` — INVOICES only** — because a credit memo's `createdFrom` is the invoice it credits (not a sales order) and dead-ended `syncInvoiceFromNetsuite`; see the [per-client-sync doc](../../worker/features/netsuite-togasupply-per-client-sync.md).
 - **Sign conventions are per-type and self-consistent.** SuiteQL returns `foreignamount`/`quantity`/`costestimate` **negative for invoices & cash sales, positive for credit memos & cash refunds**. A uniform `-foreignamount` therefore yields **positive** revenue for invoices and **negative** for reversals — the agreed Forecast2 convention. Do NOT also apply a per-type factor (it double-negates reversals back to positive). The shim flips `amount = -foreignamount`, `costEstimate = -costestimate`.
 - **⚠ `->holdInvoice` on a sales-order shim is NOT a native NetSuite field — it is normalised from the custom checkbox `custbody_asi_hold` (internal id 4082), and that field is SHARED by all clients.** NetSuite REST exposes no top-level `holdInvoice`. The `"asi"` prefix is **Agilant Solutions Inc** (the company), *not* a client prefix — do not read it as client-specific. The read lives **inside the shim**, in all three fetch paths: `buildSalesOrderShims()` (~L533, feeding both `listSalesOrders()` and `fetchSalesOrderByIdFull()`) and `fetchSalesOrderById()` (~L1537), so callers such as `App_Api_Toga2::isTransferOrder()` read an already-normalised boolean. (An earlier version read `$nsOrder->holdInvoice` in the caller with a `custbody_hold_invoice` fallback — the wrong field name; commit `07c31e24`, 2026-08-19, found the real one and moved the read down here.) `worker/crons/toga2/compass/edi/2_create_so_netsuite.php` **writes** the same field. **⚠ The SOAP client `library/app/netsuite.php` never sets `holdInvoice` at all**, so any detection that depends on it (e.g. the `zero_dollar_hold` transfer-order mode — see [per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md)) is **silently always-false** on a SOAP path. Verified not a live risk: the togasupply engine calls `App_Api_Netsuite_Rest::` on every sales-order/receipt/fulfillment path.
 - **`createdFrom` is not a usable SuiteQL column.** `transaction.createdfrom` is **always NULL** in SuiteQL (26/26). It is recoverable from `previoustransactionlinelink.previousdoc` excluding `previoustype = 'Opprtnty'` (an invoice links to both its SO and its opportunity; REST's `createdFrom` is the SO). **But this is non-deterministic for multi-link transactions** (credit memos return several non-Opprtnty `previousdoc` rows in arbitrary order) — two runs can pick different values. REST `createdFrom` (per-id GET) is the only deterministic source. Treat the link table as best-effort, not authoritative.
@@ -93,6 +94,9 @@ Field semantics (verified on the live account):
 - Field availability varies by NetSuite account **and** record type — probe the live account before assuming a column/relationship exists.
 - **`App_NetSuite::getCustomFieldValue()` (`library/app/netsuite.php`) must be warning-safe.** 1.0 runs on PHP 7.2 in an unguarded path, so any PHP warning → `exit()` (kills the cron). It was hardened 2026-08-27 with `?? []` / `?? null` guards on the custom-field-list traversal. Keep that discipline when touching it.
 
+## Change history
+- 2026-09-21 — `listSales()` gained a `$recordTypes` allowlist param (defaults to all 4 for the forecast import); callers pass `RECORD_TYPE_*` constants, not raw SuiteQL codes; throws on unknown/empty. Lets the togasupply sync request INVOICES only. (jcardinal)
+
 ## Related
 
 - `netsuite-suiteql-api-reference.md` — the broad API reference (auth, SuiteQL mechanics, table schemas, type/status codes, custom-field map, performance) these semantics sit on top of.

package/knowledge/1.0/apps/library/features/toga2-api-client-and-bridge.md

@@ -6,7 +6,7 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
+updated: 2026-09-21
 owners: [jcardinal, mhammontree, bala]
 files:
   - library/app/api/toga2.php
@@ -163,7 +163,7 @@ Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven i
 ## Gotchas
 
 - **A 2.0 GET silently TRUNCATES a nested child collection — never trust a single response for a count.** When you read a parent with a nested `hasMany` collection in one GET, api2 can return the collection **partial** (observed: a fulfillment's nested `itemFulfillmentItemUnits` came back **45 of 70**) with no error and no obvious signal — so any verification that counts off the nested array is silently wrong. **Read the collection flat and fully paginated** (query the child resource directly with the universal `meta->nextPage` loop) and take counts from **`meta.totalRecordCount`**, not from a single nested response. Surfaced building the NYCHH fulfillment asset-tag verifier (`worker/crons/toga2/netsuite/verify_fulfillment_asset_tag_sync_nychh.php`). **Confirmed again on the fulfillment reconcile (2026-09-13):** the `salesOrderItems` nested collection truncated at **25**, and a fulfillment's `itemFulfillmentItemUnits` re-created 25,766 links for 1,192 real units. The reconcile now reads both flat + fully paginated via two new private helpers, `getItemFulfillmentItemUnitsByItemFulfillmentUuid()` and `getSalesOrderItemsBySalesOrderUuid()` (`recordsPerPage` 1000, page-cap 500, `nextPage ?? null`) — detail on the [per-client sync doc](../../worker/features/netsuite-togasupply-per-client-sync.md).
-- **🔐 Hardcoded credentials.** Every client's `CLIENT_UUID_*`, `API_UUID_*`, and `API_SECRET_*` (plus the "True Workers" master keys) are committed as plaintext class constants at the top of `toga2.php`. Treat them as **compromised-if-leaked**; do not reproduce the values in the KB, tickets, or logs. Long-term they should move to config/SSM. (Observed 2026-06-23.)
+- **🔐 Hardcoded credentials.** Every client's `CLIENT_UUID_*`, `API_UUID_*`, and `API_SECRET_*` (plus the "True Workers" master keys) are committed as plaintext class constants at the top of `toga2.php`. Treat them as **compromised-if-leaked**; do not reproduce the values in the KB, tickets, or logs. Long-term they should move to config/SSM. (Observed 2026-06-23.) **⚠ Also (found 2026-09-21, NOT fixed): a committed JWT (a dev/NYCHH token) sits inline at `library/app/api/toga2.php:641`, inside `App_Api_Toga2::send()` — the live sync call path.** Pre-existing, not introduced this session. **Rotate the token and delete the line;** record its location only, never the value.
 - **Two unrelated `sync_togasupply_<client>.php` families.** Files under `worker/crons/toga2/netsuite/` are the **NetSuite supply importer** (`require` the common engine). Files under `worker/crons/toga2/<client>/` (e.g. `aig/`, `wje/`) are this **bridge** (`syncWithToga`/`syncWithTogadesk`). Same filename, different integration, different parameter keys — they coexist harmlessly. Confirm which folder before editing.
 - **`syncWithToga` and the bridge build SQL by hand** with `App_Database::sqlEscape()` rather than the `App_Model` layer; follow the surrounding escaping discipline when modifying.
 - **Per-record error isolation** in the TOGaDesk sync is via `try/catch` → `\Sentry\captureException`; a thrown exception elsewhere (transport, checkpoint read) still aborts the whole run.
@@ -179,6 +179,9 @@ Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven i
 - **This logic IS testable without a database — and there is a harness.** `App_Database::sqlEscape()` needs **no DB connection** (pure `str_replace`), so the email transform can be exercised statically. The TRUE-79401 transform is inlined in the DB-writing `private static syncContactToToga1Customer()`, so the harness **mirrors** it and guards against drift by asserting the production statements verbatim — see [static no-DB regression harness](../../test/features/static-no-db-regression-harness.md). **If you edit those 5 statements, the harness will refuse to report until it is updated.** The clean fix if you are already in there: extract the transform to a `public static` helper so the mirror disappears.
 - **Pre-existing (low severity, out of scope): a raw-vs-escaped comparison causes a redundant UPDATE every sync.** At the update path (~line 8584) `syncContactToToga1Customer` compares `$existingCustomer['emailAddress'] !== $emailAddress` — the DB value from `App_Database::fetchRow()` is **unescaped**, while `$emailAddress` has already been through `App_Database::sqlEscape()`. For any value containing `'`, `"` or `\` the two can **never** compare equal, so that column is rewritten on **every** run. Same pattern applies to `businessName` and `phoneNumber` just above. **Predates TRUE-79401**, is a redundant write (not data loss), and was deliberately left out of that ticket's scope — recorded here so a future reader knows it is known.
 
+## Change history
+- 2026-09-21 — Recorded a pre-existing committed JWT (dev/NYCHH token) at `toga2.php:641` inside `send()` (the sync call path); flagged to rotate + delete, location-only. (jcardinal)
+
 ## Related
 
 - [NetSuite → TOGa Supply per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md) — the importer that uses this class's transport and `*FromNetsuite()` methods.

package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

@@ -6,7 +6,7 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-09-18
+updated: 2026-09-21
 owners: ["dfranks", "bala", "jcardinal", "mhammontree", "snaredla", "rgirish"]
 files:
   - worker/crons/toga2/netsuite/common_sync_togasupply.php
@@ -834,10 +834,10 @@ then at header level:
 Prod check: 391 rows TO-side, 293 SO-side, **0 with both, 0 with neither**.
 
 **It throws — never skips — on:** more than one SalesOrder for the NetSuite id; a TO with no
-lines; an SO line with no matching TO line; a line carrying `InvoiceItems` (**`InvoiceItems` has no
-`transferOrderItemId`**, so they physically cannot be moved); or any bridge whose existence check
+lines; an SO line with no matching TO line; or any bridge whose existence check
 returns no `totalRecordCount` (see the ACL shape below). Consistent with the 2026-08-27 fail-loud
-policy.
+policy. **A stale transfer-order Invoice no longer freezes the run — it is now DELETED up front
+(2026-09-21, below);** the old per-line `InvoiceItems` throw is kept only as a safety net.
 
 **Bridge direction is PO-FIRST, confirmed from NYCHH data rather than guessed.** The naming rule is
 `<source>_<created from it>`. `Client_Nychh` populates `PurchaseOrderItems_TransferOrderItems` (98
@@ -922,6 +922,56 @@ actually have to write code for** — the schema alone would have sent us after
 | `PurchaseOrders_TransferOrders` | 17 → **104** |
 | `PurchaseOrderItems_TransferOrderItems` | 28 → **163** |
 
+#### ⚠ A stale transfer-order Invoice froze SALES_ORDERS — the retirement now DELETES it up front (fixed + deployed + prod-verified 2026-09-21, NYCHH)
+
+The retirement's old "throw on any `InvoiceItems`" rule froze NYCHH `SALES_ORDERS` at `1-RUNNING`
+(`Logs.Issue` **813**, `clientId 3`). NetSuite order **4680233** (number **172445**) was reclassified
+`SalesOrder`→`TransferOrder`; the stale `SalesOrder` **65** carried `Invoice` **297781** (id **1143**).
+`Invoices`/`InvoiceItems` have **no `transferOrderId`/`transferOrderItemId`**, so the retirement could
+not move them and threw *"resolve manually"* every run.
+
+**Team decision (jcardinal): a transfer-order's invoices are NOT imported, so a stale one is deleted,
+not preserved.** New `private static deleteStaleTransferOrderInvoicesForSalesOrder()` runs **first** in
+the retirement (before the line/header moves):
+
+- Deletes, in FK order, the stale TO invoice's **`InvoiceItems` → `InvoiceTrackingNumbers` → the
+  `Invoice` header** — both children are FK **RESTRICT**, so children first.
+- `Invoices` has **no soft-delete/void flag**, so it is a **hard delete**.
+- **Throws (refuses)** if the invoice also bills a **different** SalesOrder (a shared invoice) — never
+  deletes another order's billing.
+- The old per-line `InvoiceItems` throw is kept as a **safety net** for anything the up-front delete
+  didn't clear.
+- All GETs use `recordsPerPage` 1000 + `isset($meta->totalRecordCount)` guards; fail-loud preserved.
+
+Reviews: php-reviewer clean, cso SAFE TO SHIP, cto AGREE (cto caught the second FK child,
+`InvoiceTrackingNumbers`). **Verified prod:** SalesOrder 65 + Invoice 1143 deleted, TransferOrder 1929
+kept, `NETSUITE_EXECUTION_MODE_SALES_ORDERS` `1-RUNNING → 1-IDLE`, cursor advanced past `…|4680233`.
+
+#### ⚠ The togasupply sync imports INVOICES ONLY — a credit memo froze INVOICES (fixed + deployed + prod-verified 2026-09-21, NYCHH)
+
+NYCHH `INVOICES` froze at `1-RUNNING` (`Logs.Issue` **804**) because `syncInvoiceFromNetsuite` refused a
+**credit memo** (NetSuite **4446850**, type `CustCred`). A credit memo's `createdFrom` is the **INVOICE
+it credits, not a sales order** — chain: credit memo `4446850` → invoice `176645` (`CustInvc`) → sales
+order `2603333`. `syncInvoiceFromNetsuite` only resolves `createdFrom` as a sales/transfer order or
+RtnAuth, so a credit memo dead-ends and throws forever.
+
+**Root cause: the INVOICES header fetch was pulling four transaction types.** `App_Api_Netsuite_Rest::listSales()`
+returned `CustInvc`, `CashSale`, `CashRfnd`, `CustCred`; credit memos + cash refunds were never in scope
+for the togasupply sync. **Team decision (jcardinal): the togasupply sync imports INVOICES ONLY.**
+
+- **Fix (worker):** `common_sync_togasupply.php` (INVOICES section, ~L978) now calls `listSales()` with
+  `[App_Api_Netsuite_Rest::RECORD_TYPE_INVOICE]`.
+- **Fix (library):** `listSales()` gained a trailing `array $recordTypes` param defaulting to all four
+  `RECORD_TYPE_*` constants (so the **forecast import** —
+  `worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php` +
+  `periodic_forecast_discrepancy_fix.php` — is unaffected via the default). Callers pass **`RECORD_TYPE_*`
+  constants, never raw SuiteQL codes**; `listSales` maps constant→code internally and **throws on an
+  unknown/empty type**. Detail on the [SuiteQL/REST shim](../../library/features/netsuite-suiteql-rest-shim.md).
+
+php-reviewer clean (additive; forecast callers unaffected), sql-reviewer SAFE TO MERGE (allowlist
+mapping, no injection). **Verified prod:** `NETSUITE_EXECUTION_MODE_INVOICES` `1-RUNNING → 1-IDLE`
+(window still 1s; self-heals ×3 from 2024-02-22).
+
 ### ⚠ TWO code paths create `PurchaseOrderItems` — only one stamped `fulfillmentType` (fixed 2026-09-17)
 
 `fulfillmentType` was stamped by `syncPurchaseOrderFromNetsuite()` (from the order-level
@@ -1283,7 +1333,9 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
   **`lastmodifieddate`, NOT `trandate`** — `App_Api_Netsuite_Rest::listSales()` defaults
   `$dateField='lastmodifieddate'` and the engine calls it without the 4th arg. So a backdated
   `trandate` is irrelevant to whether an invoice imports, and a just-modified invoice still won't
-  import until it is >24h old (the upper-bound cap).
+  import until it is >24h old (the upper-bound cap). **The engine also now passes
+  `[RECORD_TYPE_INVOICE]` (INVOICES only — 2026-09-21); credit memos / cash refunds are out of scope
+  and a credit memo threw here — see the credit-memo freeze subsection above.**
 - **⚠ DEPLOY ORDER: the cursor-rename migration MUST run on EVERY client DB before the worker/library
   deploy — there is NO old-key fallback in code.** `Client/2026-09-02b - NetsuiteSyncCursorRename.sql`
   renames `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` → `NETSUITE_LAST_SYNC_CURSOR_<SECTION>`. If the code
@@ -1995,6 +2047,15 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
   [NetSuite Sync Alert Monitor](../../library/features/netsuite-sync-alert-monitor.md).
 
 ## Change history
+- 2026-09-21 — **Two NYCHH sync freezes fixed, deployed (library + worker together) and prod-verified.**
+  (1) SALES_ORDERS froze (`Logs.Issue` 813) on a stale transfer-order invoice the retirement couldn't
+  move (`Invoices` has no `transferOrderId`). New `deleteStaleTransferOrderInvoicesForSalesOrder()`
+  deletes the stale TO invoice up front — `InvoiceItems` → `InvoiceTrackingNumbers` → header (both FK
+  RESTRICT), hard delete (no void flag), throws only if the invoice also bills a different SalesOrder;
+  team decision that TO invoices aren't imported. (2) INVOICES froze (`Logs.Issue` 804) on a credit
+  memo whose `createdFrom` is the invoice it credits, not a sales order; the sync now imports INVOICES
+  only — `listSales()` gained a `$recordTypes` param (defaults to all 4 for the forecast import) and the
+  engine passes `[RECORD_TYPE_INVOICE]`. Both verified `1-RUNNING → 1-IDLE`. (jcardinal)
 - 2026-09-18 — **Three corrections and one deployed fix.** (1) The stale-SalesOrder retirement
   method shipped **missing four FK dependencies** and failed on each in turn: `ItemFulfillments`
   **header** `salesOrderId` (all 218 orders), the **upstream** `PurchaseOrders_SalesOrders` header

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.847",
+  "version": "1.0.848",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",