toga-ai 1.0.280 → 1.0.281

package/knowledge/1.0/apps/library/features/netsuite-suiteql-api-reference.md

@@ -262,6 +262,21 @@ work; the pk is `nkey`). Columns: `addr1`, `addr2`, `addressee`, `attention`, `c
 - `entityid` and `companyname` are split in SuiteQL; SOAP concatenated them as
   `"<entityid> <companyname>"`. When they're equal, return the value once (don't duplicate).
 
+## `systemnote` table — field-change forensics
+
+The audit trail of individual field changes — use it to see exactly **who / what / when / how** a
+field changed, **including sublist LINE edits that fire no User Event and therefore no webhook**
+(see the sale-import doc's AMQ line-field blind spot).
+
+- **Performance: filter by `field` + `"date"`, NEVER by `recordid` alone.** A `recordid`-only
+  filter is a pathological full scan (**~279s**); adding a `field` predicate + a `"date"` range
+  drops it to **~1s**.
+- Columns: `context` (`UIF` = user interface; also `CSV`, `WS`, …), `name` (the acting user),
+  `role`, `recordid`, `field`, `linetransactionid`, `lineid`, `oldvalue`, `newvalue`, and `"date"`
+  (a reserved word — quote it). A sublist line-field inline edit (e.g. `MCOSTESTIMATE`) shows up
+  here as a `context='UIF'` row with the acting `name` even though it bumped **no**
+  `lastmodifieddate` and fired no webhook.
+
 ## Custom fields
 
 | NS internalId | Script id | Description |
@@ -339,9 +354,24 @@ numbers. Budget hours for multi-year runs and launch them under `nohup`/`tmux`.
   Measured against the Forecast2 reconciliation checker (TRUE-79968). Use `> 0` / `NVL(col,0) <> 0`
   for a custom ref column you only need to test for presence in an aggregate; reserve `IS NOT NULL`
   for non-aggregate row filters.
+  - **⚠ `> 0` is NOT a universal fix — it is column-dependent.** A predicate on a *different*
+    custom column, `tl.custcol_item_sku > 0`, **never returned even as a STANDALONE aggregate**
+    (not just inside the UNION+MOD fingerprint), while `tl.custcol_sales_rep_line > 0` in the same
+    query stayed fast (~8s). Merely **SELECTing** the custom column is cheap either way. **Durable
+    rule when a custom-column predicate hangs:** keep the fast predicates in SQL, **SELECT** the
+    problem column, and apply that gate in **PHP** — do not assume `> 0` will rescue every custom
+    column. (TRUE-79973, `test/@dave/checker.php` / `fixer.php`.)
 
 ## Change history
 
+- 2026-07-07 — **Added the `systemnote` field-change forensics section and qualified the custom-column
+  perf rule** (Forecast2 SALES profit-drift forensics). `systemnote` filtered by `field`+`"date"` is
+  ~1s vs ~279s by `recordid` alone; it carries `context` (`UIF`), `name` (acting user), `role`,
+  `linetransactionid`, `lineid`, `oldvalue`/`newvalue` — and is the only NetSuite-side record of a
+  sublist line-field inline edit (which bumps no `lastmodifieddate` and fires no webhook). Also
+  qualified the `IS NOT NULL`→`> 0` note: `> 0` is **column-dependent, not universal** —
+  `custcol_item_sku > 0` hung even standalone while `custcol_sales_rep_line > 0` stayed fast; when a
+  custom-column predicate hangs, SELECT the column and gate in PHP (TRUE-79973). (dfranks)
 - 2026-06-30 — **Recorded the `IS NOT NULL` vs `> 0` planner trap for custom `transactionline`
   columns in aggregate queries** (TRUE-79968): a custom employee-ref column filtered with
   `IS NOT NULL` inside a power-sum `GROUP BY` aggregate scanned ~15× slower (~122s) than the

package/knowledge/1.0/apps/worker/features/forecast2-netsuite-reconciliation.md

@@ -493,11 +493,42 @@ None — Forecast2 is a single shared dataset.
   `checker.php` (JE arm) and `fixer.php` (both JE queries) with inline comments warning against
   reverting. See the SuiteQL API reference for the general rule. (Use `> 0`, or `NVL(col,0) <> 0`,
   never `IS NOT NULL`, for a custom employee-ref `transactionline` column in a SuiteQL aggregate.)
+  **⚠ But `> 0` is column-dependent, not a universal fix:** `tl.custcol_item_sku > 0` hung even as a
+  standalone aggregate while `tl.custcol_sales_rep_line > 0` stayed fast (~8s) — when a custom-column
+  predicate hangs, SELECT the column and gate in PHP instead. (TRUE-79973; see the SuiteQL API ref.)
+- **These reconcilers are the ONLY backstop for NetSuite sublist line-field inline edits — the
+  real-time webhook path is structurally blind to them.** Editing a line-level field inline in the
+  NetSuite UI (e.g. estimated cost `TRANLINE.MCOSTESTIMATE`) fires **no** User Event `afterSubmit`
+  (not create/edit/`xedit`) and **does not bump `lastmodifieddate`** — it writes only a `systemnote`
+  (context `UIF`). So the AMQ enqueuer never fires (no webhook) **and** any `lastmodifieddate`-windowed
+  `list*()`/cron sync misses it too — `Forecast.Sales` **cost/profit drifts while revenue reconciles**
+  (the $0-revenue cost-line case). This is the concrete mechanism behind the Summary's "older rows
+  whose NetSuite values change (e.g. cost re-valuation) are never re-pulled." `checker`/`fixer` catch
+  it because they reconcile a chosen `tranDate` range directly against NetSuite by internal id, not by
+  `lastmodifieddate`. Real case: invoice 6715127 (2026-02-20) drifted $3,392.70; recurring/multi-user
+  (~12 line-cost edits/day). See the sale-import doc's AMQ blind-spot section. To pinpoint *who/what/
+  when* changed a line, poll SuiteQL `systemnote` filtered by `field`+`"date"` (see the SuiteQL API ref).
+- **Inbound webhook forensics — `db_logs` `Logs.Webhook`.** Every inbound webhook TOGA receives is
+  logged here (columns: `dtStamp`, `sourceIp`, `method`, `route`, `requestHeaders`, `requestPayload`);
+  NetSuite events are `route='/netsuite'` with payload `{recordType,internalId,eventType,action,timestamp}`.
+  Logging began **~2026-06-18**, so its absence before then is not evidence a webhook didn't fire. Use it
+  to confirm whether a given transaction's edit actually produced an inbound POST (paired with the AMQ
+  enqueuer execution log on the NetSuite side).
 - These tools live in `test/@dave/` (developer tooling), but `trueup_open_orders` has been run
   against production. The `Defaults`/checkpoint mechanics of the scheduled sync are separate.
 
 ## Change history
 
+- 2026-07-07 — **Recorded that these reconcilers are the only backstop for NetSuite sublist
+  line-field inline edits + added `Logs.Webhook` forensics** (SALES profit-drift forensics; invoice
+  6715127, 2026-02-20, $3,392.70). A line-level inline edit (e.g. `TRANLINE.MCOSTESTIMATE`) fires no
+  User Event `afterSubmit` (not create/edit/`xedit`) and does **not** bump `lastmodifieddate` — only
+  a `systemnote` (context `UIF`) — so both the AMQ webhook path and any `lastmodifieddate`-windowed
+  sync miss it; profit drifts while revenue reconciles. `checker`/`fixer` catch it (they reconcile by
+  `tranDate` range + internal id, not `lastmodifieddate`). Documented `db_logs.Logs.Webhook` for
+  inbound-webhook forensics (route `/netsuite`, payload shape; logging began ~2026-06-18) and qualified
+  the custom-column perf rule (`> 0` is column-dependent — `custcol_item_sku > 0` hung even standalone;
+  SELECT + gate in PHP). (dfranks)
 - 2026-06-30 — **Made the fixer the cleanup mechanism for no-rep JE rows + killed a ~15× SuiteQL
   planner trap (TRUE-79968, dfranks).** The importer now keeps only rep-bearing JE lines (see the
   sale-import doc); to let `fixer.php` purge any pre-existing no-rep `Forecast.Sales` JE rows with no

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -9,7 +9,7 @@
 | [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
 | [Error Reporting — Issue/Event Aggregation (exceptionHandler)](features/error-reporting-issue-event.md) | `_underscore`'s global exception handler persists every uncaught exception into a two-table **Issue / Event** model in the **shared Core Logs DB** (`_underscore | _underscore/Error.php, _underscore/Model/Core/Logs/Issue.php, _underscore/Model/Core/Logs/Event.php, dbchanges2/Logs/2026-07-06 - Issue and Event tables.sql |
 | [Record-Changed Event Publishing (_Event::publish to SQS)](features/event-publish-sqs.md) | `_Event::publish()` (in `_underscore/Event.php`) is the PHP side of the real-time event pipeline. | _underscore/Event.php |
-| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/verify_je_classification.php, test/@dave/probe_je_accounts.php, test/@dave/probe_je_shape.php, test/@dave/fixer.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
+| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/verify_je_classification.php, test/@dave/probe_je_accounts.php, test/@dave/probe_je_shape.php, test/@dave/fixer.php, test/@dave/Junk Drawer/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/Junk Drawer/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
 | [Item-Fulfillment Stage Lifecycle (picked/packed/shipped) & Order Status](features/item-fulfillment-stage-lifecycle-and-order-status.md) | Every ItemFulfillment (IF) now carries an explicit **stage** — picked → packed → shipped — resolved through `ItemFulfillmentStages → ItemFulfillmentStatuses` (m | _underscore/Model/Client/SalesOrder.php, _underscore/Model/Quad/SalesOrder.php, _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/SalesOrderStatus.php, _underscore/Model/Client/SalesOrderItem.php, _underscore/Model/Client/Item.php, _underscore/Model/Client/PurchaseOrderItem.php, library/app/api/toga2.php, dbchanges2/Client/2026-06-30a - BackfillNullStageItemFulfillmentsToShipped.sql, dbchanges2/Client/2026-06-30b - SalesOrderStatusesPickedPacked.sql, dbchanges2/Client/2026-06-30c - ItemFulfillmentStageIdNotNull.sql, dbchanges2/Client_CompassCanada/2026-06-30a - ItemFulfillmentLifecycleAndShippedBackfill.sql |
 | [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php |
 | [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |

package/knowledge/2.0/apps/_underscore/features/forecast-sale-import.md

@@ -30,8 +30,8 @@ files:
   - test/@dave/probe_je_accounts.php
   - test/@dave/probe_je_shape.php
   - test/@dave/fixer.php
-  - test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js
-  - test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js
+  - test/@dave/Junk Drawer/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js
+  - test/@dave/Junk Drawer/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js
 related:
   - ../architecture.md
   - ../../worker2/architecture.md
@@ -295,6 +295,39 @@ events as specific **subtypes** (`inventoryItem`, `nonInventoryResaleItem`, `kit
 would be needed only for a future real-time item webhook, **not** for this Sales importer
 (which keeps items fresh via the hourly item pull cron plus the inline self-heal).
 
+### AMQ enqueuer trigger behavior + the line-field inline-edit blind spot (durable — read for any real-time NS→TOGA sync debugging)
+The enqueuer is `customscript_ue_amq_enqueue` (**script id 2618**), a UserEvent `afterSubmit`
+deployed **one-per-record-type** (invoice/cashSale/creditMemo/cashRefund/journalEntry/salesOrder/
+opportunity/items/…). The **invoice** deployment (primarykey **16019**) is **RELEASED**,
+`allroles=T`, `allemployees=T`, with **every execution context enabled** — so it is NOT
+restricted by user/role/context.
+
+**What DOES fire it:** record `create` / `edit` / `delete`, **and `xedit`** — a BODY-field inline
+edit (this is the source of the `eventType:"xedit"` webhooks, built by the script's own
+`buildPayload`). On fire it logs `AMQ enquerer triggered by "<context.type>" event` then
+`AMQ enqueued {recordType,internalId,action}`, enqueues to custom record
+`customrecordcustomrecord_api_msg_queue` (**internal id 1185**), and kicks the drainer
+`customscript_ss_amq_drain`.
+
+**What does NOT fire it — the structural blind spot:** editing a **sublist LINE field**
+(e.g. estimated cost `TRANLINE.MCOSTESTIMATE`) inline via the NetSuite UI raises **NO** record
+User Event `afterSubmit` — **not `create`, not `edit`, not `xedit`**. It also does **NOT** bump
+the transaction's `lastmodifieddate`; it writes only a **`systemnote`** (context `UIF`). So the
+enqueuer never fires → no AMQ row → no `webhook.togahub.com/netsuite` POST → `Forecast.Sales`
+cost/profit **silently drifts**. When the edited line is $0-revenue (a fee/cost line), **revenue
+reconciles while profit drifts** — the classic symptom. Proven with live enqueuer execution-log
+evidence: two invoices whose ONLY change that day was a cost line-edit had **zero** "triggered by"
+entries, while 8 other cost-edited invoices that also had normal create/edit activity **did** fire
+and synced. This is recurring/multi-user (~12/day; 35 `MCOSTESTIMATE` edits in 3 days by 9 users).
+
+**Corollary gotcha: `lastmodifieddate` is NOT a reliable change signal for line-cost edits** — a
+line-field inline edit leaves it untouched, so **both** the webhook pipeline **and** any
+`lastmodifieddate`-windowed `list*()`/cron sync miss it entirely. **Only reconciliation**
+(`checker`/`fixer` — see the reconciliation doc) **or a `systemnote` poll** catches this drift.
+Forensic tool for who/what/when/how a line changed: SuiteQL `systemnote` filtered by `field` +
+`"date"` (see the SuiteQL API-reference `systemnote` section — filtering by `recordid` alone is a
+~279s full scan; by `field`+`"date"` it's ~1s).
+
 ## E2E testing playbook + harness inventory (run this cold)
 This section is the reusable procedure for verifying the NetSuite → `Forecast.Sales` path
 end-to-end against a **local** Forecast mirror. It is written to be runnable in a future
@@ -452,6 +485,18 @@ record is deleted in NetSuite.)
 - The cron's sign handling is not portable here — see Sign convention.
 
 ## Change history
+- 2026-07-07 — **Documented the AMQ enqueuer's line-field inline-edit blind spot** (Forecast2 SALES
+  profit-drift forensics; invoice 6715127 / 2026-02-20 / $3,392.70). Editing a **sublist LINE
+  field** (e.g. cost `TRANLINE.MCOSTESTIMATE`) inline in the NetSuite UI raises **no** record UE
+  `afterSubmit` (not create/edit/**xedit**), does **not** bump `lastmodifieddate`, and writes only a
+  `systemnote` (context `UIF`) — so `customscript_ue_amq_enqueue` never fires and `Forecast.Sales`
+  cost/profit silently drifts (revenue reconciles when the line is $0-revenue). Proven via live
+  enqueuer execution logs (2 cost-line-only-edited invoices had zero "triggered by" entries vs 8
+  that had other activity and synced); recurring/multi-user (~12/day). Corollary:
+  `lastmodifieddate` is not a reliable change signal for line-cost edits — only reconciliation or a
+  `systemnote` poll catches them. Also recorded the enqueuer's exact trigger surface (script id
+  2618; invoice deployment pk 16019 RELEASED/allroles/allemployees; fires create/edit/delete +
+  `xedit` BODY edits; enqueues to custom record 1185; kicks `customscript_ss_amq_drain`). (dfranks)
 - 2026-07-07 — **CORRECTION: JE lines DO carry an item — in the custom column
   `custcol_item_sku`, not the native item field** (TRUE-79973, dfranks). Live probe
   2026-07-07 confirmed native `transactionline.item` / REST `line.item` is empty on **all

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.280",
+  "version": "1.0.281",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",