toga-ai 1.0.750 → 1.0.752

package/knowledge/1.0/apps/library/features/netsuite-sync-alert-monitor.md

@@ -6,8 +6,8 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-07-10
-owners: ["bala"]
+updated: 2026-09-02
+owners: ["bala", "jcardinal"]
 files:
   - library/app/systemmonitor/netsuiteintegration.php
   - worker/crons/infrastructure/system_monitors.php
@@ -104,8 +104,26 @@ that worker job does the syncing, this library class notices when it stops.
   checkpoints mean anything; and a client that is held / never transacts supply (AIG) must be
   left **off** this list or it produces permanent false-positive tickets.
 
+- **⚠ THIS MONITOR IS NOW BLIND — the cursor-rename shipped 2026-09-02 and this monitor was NOT updated.**
+  The sync's per-record resume cursor deployed to prod 2026-09-02 (worker `06e8f9ef` + library `3a70b59c`),
+  renaming each section's position key from `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` to
+  `NETSUITE_LAST_SYNC_CURSOR_<SECTION>` (value now `"<lastmodifieddate>|<netsuite id>"`). This monitor's
+  `NETSUITE_SYNC_PARAMETER_KEYS` still lists the old `DATETIME` keys, so its `where..in` `/parameters` read
+  now matches **zero rows** and it reports OK forever — **no stale-checkpoint alerts fire**. This was an
+  accepted gap at deploy time, but it is now live. **FIX OWED:** update `NETSUITE_SYNC_PARAMETER_KEYS` to the
+  `CURSOR` names **and** parse the `<date>|<id>` value (`strtotime` the part before `|`) before the 48h
+  compare. See the [per-client sync doc](../../worker/features/netsuite-togasupply-per-client-sync.md).
+
 ## Change history
 
+- 2026-09-02 — **The sync's `NETSUITE_LAST_SYNC_DATETIME_*` → `NETSUITE_LAST_SYNC_CURSOR_*` rename SHIPPED to
+  prod (worker `06e8f9ef` + library `3a70b59c`), so this monitor is now BLIND** — its `where..in` read matches
+  zero rows and it reports OK forever. Monitor was not updated with the rename (accepted gap, now live). Fix
+  owed: update `NETSUITE_SYNC_PARAMETER_KEYS` to the CURSOR names + `strtotime` the part before `|`. (jcardinal)
+- 2026-09-02 — Flagged that the sync's pending `NETSUITE_LAST_SYNC_DATETIME_*` →
+  `NETSUITE_LAST_SYNC_CURSOR_*` key rename (BUILT, not yet deployed) will make this monitor match zero rows
+  and stop alerting; it was intentionally left un-updated. Update the key list + parse the `<date>|<id>`
+  value when the rename ships. (jcardinal)
 - 2026-07-10 — **Fixed the monitor timing out and silently not running since 2026-06-29.**
   Rewrote `getClientStatus()` to read all six checkpoints per client in one TOGa2
   `where..in` `/parameters` call (was 84 serialized calls ≈ 230s, over the 300s

package/knowledge/1.0/apps/worker/INDEX.md

@@ -10,7 +10,7 @@
 | [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/checker.php, worker2/Component/Forecast/SaleImport/SaleImport.php, test/@dave/looper.php, test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, tools/bin/forecast/fixer.php, tools/bin/forecast/checker.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/reconcile_drift_2023plus.php, test/@dave/probe_invoice_gap_2026.php, test/@dave/probe_creditmemo_gap_detail.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
 | [Forecast2 Supporting-Records Nightly Import (1.0 cron) — Customers & ConsolidatedCustomers](features/forecast2-supporting-records-import.md) | The nightly **1.0** pull that keeps the Forecast2 lookup/dimension tables (`Forecast.Accounts`, `Classifications`, `Customers`, `ConsolidatedCustomers`, `Employ | worker/crons/toga2/forecast2/import_supporting_records.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/schedules/cron.worker.infrastructure.json, worker/.ebextensions/009_setup_phpini.config, worker/crons/sync/netsuite/netsuite_customers.php, library/app/api/netsuite/rest.php, library/app/model/forecast2/customer.php, library/app/model/forecast2/consolidatedcustomer.php |
 | [NetSuite Sales Order Sales Rep Sourcing (Staples & ODP EDI orders)](features/netsuite-sales-order-sales-rep-sourcing.md) | How the **sales rep** on a NetSuite Sales Order is determined for the two 1.0 `worker` EDI order-creation integrations (Staples cXML and Compass/ODP EDI). | worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, worker/crons/sync/staples/sync_staples_cxml.php, test/@Mark/NetSuite/TRUE_80451_customer_salesrep_diag.php |
-| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/crons/toga2/netsuite/sync_togasupply_elite.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, library/app/api/toga2.php, library/app/api/netsuite/rest.php, library/app/framework.php, library/app/systemmonitor/netsuiteintegration.php, test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php, test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php |
+| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/crons/toga2/netsuite/sync_togasupply_elite.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, library/app/api/toga2.php, library/app/api/netsuite/rest.php, library/app/query.php, library/app/database.php, library/app/framework.php, library/app/systemmonitor/netsuiteintegration.php, dbchanges2/Client/2026-09-02b - NetsuiteSyncCursorRename.sql, test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php, test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php |
 | [OneUptime Server monitor + disk/memory hygiene on the 1.0 worker EB host](features/oneuptime-server-monitor-host-hygiene.md) | The 1.0 `agilant-worker` EB environment runs on the **legacy Amazon Linux 1 PHP 7.2 platform** (Apache httpd/prefork, s3fs mounts, cron) and repeatedly went dow | worker/.ebextensions/040_disk_memory_hygiene.config, worker/.ebextensions/045_oneuptime_agent.config, worker/ebs/cron.worker.php, worker/ebs/mount-s3fs-folders.php, worker/ebs/apache_settings.php, worker/ebs/setup_phpini.php |
 | [OneUptime external uptime monitoring for 1.0 workers](features/oneuptime-worker-uptime-monitoring.md) | Every 1.0 worker box self-reports its liveness to an external OneUptime monitor once per minute by curl-POSTing to a per-worker "Incoming Request" heartbeat URL | library/app/worker.php, worker/crons/worker/worker_heartbeat.php, worker/ebs/cron.worker.php, worker/.ebextensions/045_oneuptime_agent.config |
 | [Prudential: Send Shipments for the Day report (daily cron)](features/send-shipments-for-the-day.md) | Daily cron (9:00 PM) that emails Prudential and Dell stakeholders an Excel report of all devices shipped that day, including tracking number, serial number, emp | worker/crons/notifications/reports/send_shipments_for_the_day.php |

package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

@@ -16,8 +16,11 @@ files:
   - dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql
   - library/app/api/toga2.php
   - library/app/api/netsuite/rest.php
+  - library/app/query.php
+  - library/app/database.php
   - library/app/framework.php
   - library/app/systemmonitor/netsuiteintegration.php
+  - dbchanges2/Client/2026-09-02b - NetsuiteSyncCursorRename.sql
   - test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php
   - test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php
 related:
@@ -89,6 +92,36 @@ its IDLE write. **There is no timeout, memory cap, or resource limit anywhere in
 never *tune the window size down*. (Corollary: a window that has grown back to the cap is positive
 evidence the section is completing.)
 
+### Per-record resume cursor — checkpoint after EACH record, not window-end (LIVE 2026-09-02)
+
+The historical weakness of window-end-only checkpointing: a mid-window crash (poison record, or the
+log-DB doom loop above) re-processes the **whole** window every run — the giant-fulfillment doom loop is a
+direct consequence. This change converts the checkpoint to **per-record keyset (seek) resume** so a
+transient crash resumes at the next record instead of redoing the window. **Deployed to prod
+2026-09-02** — worker commit `06e8f9ef` + library commit `3a70b59c` (both tips of `_production`), with the
+rename migration run on every client DB first (see the deploy-order gotcha below).
+
+- **Key rename per section:** `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` → `NETSUITE_LAST_SYNC_CURSOR_<SECTION>`,
+  value format **`"<lastmodifieddate>|<netsuite internal id>"`**. The id breaks same-second ties.
+- **Fetch change (`library/app/api/netsuite/rest.php`, all 6 header queries):** add
+  `ORDER BY lastmodifieddate, id` and change the lower bound to
+  `lastmodifieddate > :t OR (lastmodifieddate = :t AND id > :fromId)` (upper bound `<= :to` kept). A new
+  `$fromId` param was threaded into the 6 list funcs. Added `lastModifiedDate` to the fetched record object
+  for **PURCHASE_ORDERS, ITEM_RECEIPTS, ITEM_FULFILLMENTS, INVENTORY_ADJUSTMENTS** (SALES_ORDERS + INVOICES
+  already carried it).
+- **Save after EACH main record** (one PUT of `lastmod|id`); on a clean window with 0 records, PUT
+  `"<windowTop>|0"` so empty date ranges still advance. No new `try/catch` — stays fail-loud.
+- **Migration is MANDATORY, and runs BEFORE the code (see the deploy-order gotcha in Gotchas):**
+  `dbchanges2/Client/2026-09-02b - NetsuiteSyncCursorRename.sql` (Client fan-out) renames the keys **and**
+  appends `|0` to carry the value across; safe to re-run. It was run on all client DBs first, then the
+  code deployed — the required order.
+- **⚠ OPEN follow-up — new-client seed was edited in place (dbchanges2 rule violation).** The seed module
+  `dbchanges2/_modules/netsuite/2026-08-05 - CLEAN NETSUITE CLINET.SQL` had its 6 seed rows **renamed in
+  place** to CURSOR keys with `|0`. That breaks the dbchanges2 "never edit an already-applied file" rule and
+  the client-schema drift audit. **Revert that file and add a NEW dated `_modules/netsuite/` file** carrying
+  the same guarded `UPDATE ... WHERE key LIKE 'NETSUITE_LAST_SYNC_DATETIME_%'` so future new clients land on
+  CURSOR keys. Still uncommitted in the working tree.
+
 ### ⚠ Per-record isolation is SECTION-SPECIFIC — and SALES_ORDERS has NONE (corrected 2026-08-17)
 
 Whether a bad record is "stepped over" or freezes the section depends entirely on whether **that
@@ -305,6 +338,12 @@ Per-client `Parameters` table (2.0 client DB, e.g. `Client_Canon.Parameters`; co
 
 Parameters are stored **per client DB** but accessed **through the TOGa2 API**, not direct SQL.
 
+**Key rename (LIVE 2026-09-02):** the position keys are now
+`NETSUITE_LAST_SYNC_CURSOR_<SECTION>` with value `"<lastmodifieddate>|<netsuite internal id>"` for all 6
+sections — see the per-record resume cursor in *How it works*. The `NETSUITE_EXECUTION_MODE_*` keys are
+unchanged. Renamed on every client DB by `Client/2026-09-02b - NetsuiteSyncCursorRename.sql` (run before
+the code deploy).
+
 ## Client variations
 
 - **Transfer-order routing is opt-in per wrapper (default OFF).** A client whose NetSuite
@@ -459,6 +498,19 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
   `$dateField='lastmodifieddate'` and the engine calls it without the 4th arg. So a backdated
   `trandate` is irrelevant to whether an invoice imports, and a just-modified invoice still won't
   import until it is >24h old (the upper-bound cap).
+- **⚠ DEPLOY ORDER: the cursor-rename migration MUST run on EVERY client DB before the worker/library
+  deploy — there is NO old-key fallback in code.** `Client/2026-09-02b - NetsuiteSyncCursorRename.sql`
+  renames `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` → `NETSUITE_LAST_SYNC_CURSOR_<SECTION>`. If the code
+  ships first, `startModeIteration()` reads a CURSOR key that doesn't exist yet and falls back to
+  `MIN_DATETIME_TO_CHECK_FOR_NETSUITE_DATA` (= `'2018-01-01'` for 17 of 18 clients) — re-scanning ~8 years
+  across all sections and all clients at once, a thundering herd against NetSuite + api2. Both the php and
+  sql reviewers flagged this as critical. The 2026-09-02 deploy followed the correct order (migration on all
+  client DBs first, then code). Same rule for any future position-key rename: schema first, code second.
+- **Per-record checkpoint PUT multiplies api2 round-trips on high-count catch-up windows (tuning note, not
+  shipped).** The per-record resume cursor does one `App_Api_Toga2::send` PUT per record. On a big backfill
+  window (e.g. sales orders / POs) that is one extra api2 round-trip per record. Optional future tuning:
+  batch the checkpoint every N records while still writing it **before** the next record's dispatch (so
+  crash-resume stays correct). Not built.
 - **A parse error in the shared `library/app/api/netsuite/rest.php` takes down EVERY client at
   once (tier-wide outage).** All wrappers autoload `App_Api_Netsuite_Rest` in the pre-section
   LOOKUPS/setup phase of `common_sync_togasupply.php` (~line 433). Because PHP parses the whole
@@ -856,6 +908,15 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
   line). **Fix (`library`):** skip lines whose `adjustQtyBy` is null in **BOTH** passes over
   `inventoryList->inventory` (the item-build pass and the serial/unit pass). Shared fix; unblocked NYCHH
   inventory adjustments.
+  - **Prod-confirmed 2026-09-02** (commit `8bae8a2a`, 2026-08-29; the null-skip lands at
+    `toga2.php:5728` and `:5879`). Before it, NYCHH `INVENTORY_ADJUSTMENTS` was frozen `3-IDLE` (window
+    collapsed to ~3s), watermark stuck at **2022-12-28**, advancing only seconds/run — `Logs.Issue`
+    **542** (`clientId 3`), 35 occurrences 2026-08-28..09-01. The fix reached prod ~2026-09-01 afternoon
+    and **Issue 542 stopped 09-01 14:08**. This `3-IDLE`-crawl-with-frozen-watermark signature is the same
+    poison-record shrink as a `1-RUNNING` freeze — a section that throws on a record too old to fall
+    outside the collapsed window keeps re-throwing; once the window shrinks past that record's timestamp it
+    reports IDLE but crawls seconds/run and can never advance past it. Read it as "still throwing," not
+    "small/slow section" (see the item-fulfillment doom-loop gotcha below for the same signature).
 - **⚠ Item-receipt sync froze on a PARTIAL receipt's NULL-quantity PO lines — the item-receipt TWIN of the
   inventory-adjustment NULL-quantity fix above (fixed 2026-08-31).** For a partial item receipt NetSuite
   returns **all** the PO's linked lines but sets `quantity=null` on the lines **not received on this
@@ -877,14 +938,33 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
   `transactionId` (`Logs.Event` carries `clientId`, `Logs.Issue` does not) → `Logs_Nychh.Api` on that
   `transactionId` for the untruncated `EV-10`/1048 rejection and the payload showing the `quantity:null`
   lines.
-- **⚠ OPEN, not yet root-caused (found 2026-08-31): NYCHH ITEM_FULFILLMENTS watermark frozen at 2021-12-29.**
-  `NETSUITE_EXECUTION_MODE_ITEM_FULFILLMENTS='3-IDLE'` with the window collapsed to ~3s (it has been
-  throwing recently) but currently **IDLE** — so it crawls a few seconds per run rather than sitting in a
-  hard `1-RUNNING` freeze. Distinct from the fixed ITEM_RECEIPTS freeze above; needs its own investigation.
-  Context: the 2026-08-31 full 19-client togasupply prod sweep found all other clients healthy
-  (`864000-IDLE`) except Elite SALES_ORDERS `864000-RUNNING` (healthy mid-run, large window, not stuck) and
-  AIG SALES_ORDERS `8-RUNNING` (the known held account, already documented). Only NYCHH ITEM_RECEIPTS was in
-  a true `1-RUNNING` freeze.
+- **NYCHH ITEM_FULFILLMENTS watermark frozen at 2021-12-29 (found 2026-08-31; ROOT-CAUSED 2026-09-02 —
+  the log-DB doom loop below).** `NETSUITE_EXECUTION_MODE_ITEM_FULFILLMENTS='3-IDLE'` with the window
+  collapsed to ~3s (throwing recently) but currently **IDLE** — so it crawls a few seconds per run rather
+  than a hard `1-RUNNING` freeze. This is the exact signature of the doom loop in the next gotcha: a giant
+  fulfillment throws every run, the window shrinks ÷3 until it no longer spans that fulfillment, then goes
+  IDLE but can never advance past it. Context: the 2026-08-31 full 19-client togasupply prod sweep found
+  all other clients healthy (`864000-IDLE`) except Elite SALES_ORDERS `864000-RUNNING` (healthy mid-run,
+  large window, not stuck) and AIG SALES_ORDERS `8-RUNNING` (the known held account).
+- **⚠ Item-fulfillment DOOM LOOP from a dropped LOG-DB connection during a giant fulfillment (found
+  2026-09-02, NOT yet fixed — the per-record cursor below is the mitigation).** A single very large
+  fulfillment POSTs thousands of `/item-fulfillment-item-units` over api2 HTTP; that long unit-POST phase
+  lets the **worker's log-DB connection idle past MySQL `wait_timeout`**. The **next** fulfillment's refetch
+  — `fetchItemFulfillmentById` at `library/app/api/toga2.php:4440`, the **first line** of
+  `syncItemFulfillmentFromNetsuite` — tries to log that NetSuite REST call and dies at
+  `library/app/query.php:168` `mysqli_select_db` *"MySQL server has gone away / Connection timed out"*
+  (`Logs.Issue` **589**). The 1.0 framework escalates that warning to `exit`, so the run aborts mid-window;
+  the watermark only advances at window end (`finishModeIteration`), so the whole window — including the
+  giant fulfillment — **re-processes next run, times out at the same spot, and the window shrinks ÷3 each
+  run** (→ the `3-IDLE` crawl signature above once the window no longer spans the giant fulfillment).
+  - **Root of the crash:** the "gone away" recovery in `App_Query::execute` (`library/app/query.php`,
+    `database.php:431`) only guards the `mysqli_query` call (errno **2006** reconnect), **NOT** the
+    `mysqli_select_db` that runs before it — which is the first touch of the stale connection, so the
+    reconnect never gets a chance. A durable framework gap, not sync-specific.
+  - **Durable engine rule reconfirmed:** a shrinking window in this sync **ALWAYS** means a thrown
+    exception — there is **no** volume/time/memory backoff anywhere in the script. "Window not 864000" ⇒
+    something threw. The log-connection idle-out is a concrete cause to add to the poison-record and
+    shim-shape causes above.
 - **⚠ Item-fulfillment stale-item prune must delete tracking-number children before the unit AND the item
   (fixed 2026-08-30).** In `syncItemFulfillmentFromNetsuite`'s cleanup loop, deleting a stale
   item-fulfillment-item-**unit** failed api2 `EV-11` / MySQL **1451** (FK
@@ -967,8 +1047,59 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
   A "silent freeze" here is a serial/data collision, not necessarily a catalog/item-matching problem —
   read the untruncated Api-log rejection before assuming item matching.
 
+- **⚠ The position key defaults to the EARLIEST date when MISSING — a code-only cursor rename WITHOUT
+  the data migration resets every client to re-sync from scratch (found 2026-09-02).** When the section's
+  position key is absent, the engine falls back to `MIN_DATETIME_TO_CHECK_FOR_NETSUITE_DATA`
+  (`common_sync_togasupply.php` ~L573). So shipping the `NETSUITE_LAST_SYNC_DATETIME_*` →
+  `NETSUITE_LAST_SYNC_CURSOR_*` rename in **code only** makes every client's old key invisible → each
+  section restarts from the earliest date and re-syncs its entire history. The migration
+  (`dbchanges2/Client/2026-09-02b - NetsuiteSyncCursorRename.sql`, Client fan-out) **must carry the value
+  across** — it renames the key and appends `|0` in one `UPDATE ... REPLACE(...)`, re-runnable. **Ship the
+  migration on every client DB before (or with) the code.**
+- **⚠ The NetSuite Sync Alert monitor will SILENTLY STOP after the cursor rename (known, accepted
+  2026-09-02).** `App_SystemMonitor_NetSuiteIntegration` (`library/app/systemmonitor/netsuiteintegration.php`)
+  hardcodes the old `NETSUITE_LAST_SYNC_DATETIME_*` keys, so once the rename ships its `where..in`
+  `/parameters` read matches **zero rows** and it stops alerting on stale checkpoints. The developer
+  **intentionally left the monitor un-updated** this session — a documented, accepted gap. Update the
+  monitor's key list to `NETSUITE_LAST_SYNC_CURSOR_*` (and parse the `<date>|<id>` value) when the rename
+  deploys, or NetSuite sync stalls go unmonitored. Cross-ref:
+  [NetSuite Sync Alert Monitor](../../library/features/netsuite-sync-alert-monitor.md).
+
 ## Change history
 
+- 2026-09-02 — **Per-record resume cursor DEPLOYED to prod (shared: `worker` + `library` + `dbchanges2`).**
+  Now live: worker commit `06e8f9ef` + library commit `3a70b59c` (tips of `_production`); rename migration
+  `Client/2026-09-02b - NetsuiteSyncCursorRename.sql` run on all client DBs **before** the code. Recorded the
+  deploy-order rule (no old-key fallback — code-first would fall back to `MIN_DATETIME_TO_CHECK_FOR_NETSUITE_DATA`
+  = `'2018-01-01'` and thundering-herd re-scan ~8 years for all clients; php + sql reviewers flagged it
+  critical). Two open items: (1) the new-client seed `_modules/netsuite/2026-08-05 - CLEAN NETSUITE CLINET.SQL`
+  was edited in place (dbchanges2 rule violation) — revert + add a new dated file; (2) optional tuning — batch
+  the per-record checkpoint PUT every N records on high-count windows. Both still uncommitted. (jcardinal)
+- 2026-09-02 — **Per-record resume cursor BUILT (shared: `worker` + `library` + `dbchanges2`; shipped later
+  the same day — see entry above).** Converts the section checkpoint from window-end-only to per-record
+  keyset resume so a mid-window crash (poison record or the log-DB doom loop) resumes at the next record
+  instead of re-processing the whole window. Position key `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` →
+  `NETSUITE_LAST_SYNC_CURSOR_<SECTION>`, value `"<lastmodifieddate>|<netsuite id>"` (id breaks same-second
+  ties); `rest.php`'s 6 header queries gain `ORDER BY lastmodifieddate, id` + a keyset lower bound + a
+  `$fromId` param; cursor PUT after each record (and `"<windowTop>|0"` on an empty window); `lastModifiedDate`
+  added to the PO/IR/IF/InvAdj fetched objects. Migration `Client/2026-09-02b - NetsuiteSyncCursorRename.sql`
+  (fan-out) renames the keys and appends `|0` — **mandatory**, because a missing key defaults to the earliest
+  date and would re-sync every client's full history. Monitor `netsuiteintegration.php` intentionally left on
+  the old DATETIME keys (will stop alerting after the rename — accepted). (jcardinal)
+- 2026-09-02 — **Item-fulfillment doom loop root-caused (found, NOT fixed — the cursor above mitigates).** A
+  giant fulfillment's long unit-POST phase idles the worker's log-DB connection past MySQL `wait_timeout`; the
+  next fulfillment's `fetchItemFulfillmentById` (`toga2.php:4440`) dies logging its NetSuite call at
+  `query.php:168` `mysqli_select_db` "server has gone away" (`Logs.Issue 589`) → 1.0 warning→`exit` → window
+  re-processes and shrinks ÷3 each run (the `3-IDLE` crawl). Root cause: `App_Query::execute`'s gone-away
+  recovery (`query.php`/`database.php:431`) guards only the `mysqli_query` (2006 reconnect), not the
+  `mysqli_select_db` before it. Explains the NYCHH ITEM_FULFILLMENTS freeze at 2021-12-29. Reconfirmed the
+  engine rule: a shrinking window ALWAYS means a throw (no volume/time/memory backoff exists). (jcardinal)
+- 2026-09-02 — **Prod-confirmed the NYCHH inventory-adjustment NULL-`adjustQtyBy` freeze + its fix (commit
+  `8bae8a2a`, `toga2.php:5728`/`:5879`).** `INVENTORY_ADJUSTMENTS` had been frozen `3-IDLE`, watermark stuck
+  2022-12-28, crawling seconds/run — `Logs.Issue 542` (`clientId 3`), 35 occ 2026-08-28..09-01, EV-10/MySQL
+  1048 "quantityAdjustBy cannot be null" on NetSuite serialized-detail sub-lines. The null-skip reached prod
+  ~2026-09-01 afternoon; Issue 542 stopped 09-01 14:08. Recorded that `3-IDLE`-crawl = the same poison-record
+  shrink as `1-RUNNING`, not a slow section. (jcardinal)
 - 2026-09-02 — **PURCHASE_ORDERS cleanup DELETE freeze fixed (shared, `library` only — WRITTEN +
   REVIEWED, NOT yet committed/deployed).** `syncPurchaseOrderFromNetsuite`'s "remove dropped items"
   cleanup deleted a NetSuite-dropped PO line with a bare throwing DELETE; a leftover item receipt

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.750",
+  "version": "1.0.752",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",