toga-ai 1.0.925 → 1.0.926

package/knowledge/1.0/apps/library/features/cron-execution-monitoring.md

@@ -6,8 +6,8 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: [dfranks, bala]
+updated: 2026-10-01
+owners: [dfranks, bala, mhammontree]
 files:
   - library/app/framework.php
 related:
@@ -48,6 +48,7 @@ Two consequences:
 ## Gotchas
 
 - **⚠ A run skipped by the overlap guard leaves NO ROW — absence is ambiguous.** `cronInitialization()` calls `exitIfProcessRunning()` **before** the `CronJobExecutions` INSERT, and the `cronFinished(true)` that guard then calls does nothing because `cronLogId` was never set. So a blocked run is indistinguishable from a run that never launched: **a missing row for an expected slot means "skipped", not "hung"**, and a hung run looks different (it has `dtCheckIn` with no `dtCheckOut`). `isProcessRunning()` matches **any** `ps -ef` line containing the script path (it only excludes `/bin/sh` and its own pid), so a stray `tail -f` or editor on that path blocks the cron indefinitely.
+- **`CronJobExecutions` is huge (~120M rows) with no index on `job`.** Never query it per run (e.g. as a cron's watermark) — it scans the table. Use a small dedicated table instead.
 - **`db_log` has no `CronJobExecutions` table.** A `cronFinished()` UPDATE that runs against the `db_log` connection throws on **every** 1.0 cron job. Both the INSERT and the UPDATE must target `db_common`.
 - **A `git merge` can silently resurrect pre-refactor logic.** These functions were cleanly consolidated to a single `db_common`-only INSERT/UPDATE pair (commit `0a3a6f75`), but a later `git merge origin/_production` (`509614036605…`) reintroduced the old dual-path structure from `_production`'s still-unmigrated copy — leaving duplicate INSERT/UPDATE blocks and reverting one path's connection argument back to `db_log`. Automatic merge resolution updated the SQL text (renamed the table) but restored the wrong connection name, producing a platform-wide breaking bug disguised as a simple PR-feedback rework. **When a merge touches a function you already refactored on your branch and the base branch never got that refactor, diff against the merge-base — do not just eyeball the current file** — to catch resurrected logic. (Fixed 2026-07-06: consolidated back to one `db_common` INSERT/UPDATE pair.)
 
@@ -55,3 +56,6 @@ Two consequences:
 
 - [Worker architecture](../../worker/architecture.md), [App_Email queued sending & attachments](./email-queue-attachments.md).
 - [OneUptime worker2 monitoring](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md), [Tracing a worker cron run in production](../../worker/workflows/tracing-a-worker-cron-run-in-production.md).
+
+## Change history
+- 2026-10-01 — Added the no-index / ~120M-row gotcha (rejected as a watermark source in TRUE-82180). (mhammontree)

package/knowledge/1.0/apps/library/features/mvc-page-pattern-and-app-skeleton.md

@@ -6,8 +6,8 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: [jcardinal]
+updated: 2026-10-01
+owners: [jcardinal, mhammontree]
 files:
   - library/app/framework.php
   - library/app/frameworkindex.php
@@ -84,9 +84,12 @@ The app depends on the `library` core repo (bootstrapped via `_.php`).
 ## Gotchas
 
 - **Route folders must stay distinct.** Matching is longest-to-shortest, so overlapping route folders can produce surprise matches.
-- **An unset `ENVIRONMENT` loads the wrong config file.** `getenv('ENVIRONMENT')` defaults to `'worker'`, so a missing env var silently loads `config.worker.ini`.
+- **An unset `ENVIRONMENT` loads PRODUCTION config.** `getenv('ENVIRONMENT')` defaults to `'worker'`, so running a 1.0 cron locally without it loads `worker/config.worker.ini` — prod DB hosts, `test_mode=0`, `email_errors=1`. Set `ENVIRONMENT` to your dev config name (`config.dev-<name>.ini`) first. Symptom: errors mention `/var/www/cache`.
 - **Secrets are currently stored in plaintext ini files** — a known practice, not a recommended one. Document where a credential lives, never its value.
 
 ## Related
 
 - [Library architecture](../architecture.md), [togaview architecture](../../togaview/architecture.md).
+
+## Change history
+- 2026-10-01 — Unset-`ENVIRONMENT` gotcha: it loads production config locally; added symptom and fix (TRUE-82180). (mhammontree)

package/knowledge/1.0/apps/worker/INDEX.md

@@ -15,6 +15,7 @@
 | [OneUptime Server monitor + disk/memory hygiene on the 1.0 worker EB host](features/oneuptime-server-monitor-host-hygiene.md) | OneUptime Server monitor (per worker role) plus disk/memory hygiene for the legacy AL1 PHP 7.2 1.0 worker EB host; open for the host-metrics agent, monitor crit |
 | [OneUptime external uptime monitoring for 1.0 workers](features/oneuptime-worker-uptime-monitoring.md) | Per-minute external OneUptime "Incoming Request" heartbeat from every 1.0 worker box (Degraded 3 min / Offline 5 min); open for the heartbeat push, monitor prov |
 | [Prudential: Send Shipments for the Day report (daily cron)](features/send-shipments-for-the-day.md) | Daily 9:00 PM Prudential/Dell cron emailing an Excel of every device shipped today (tracking, serial, XID, FedEx status, vendor); open for the ASN join chain an |
+| [Staples 856 ASN Outbound (NetSuite fulfillments → SFTP, tracked in StaplesAdvanceShipNotices)](features/staples-asn-856-outbound.md) | Hourly cron sending Staples 856 ASNs for shipped NetSuite item fulfillments, one row per fulfillment in `Common.StaplesAdvanceShipNotices`; open for the queue/r |
 | [Staples cXML Order Import (SFTP → NetSuite)](features/staples-cxml-order-import.md) | Hourly cron importing Staples cXML POs from SFTP into NetSuite Sales Orders (writes 855 ack back); open for the per-file flow, saved-search-8412 Location resolu |
 | [UMA Student Roster Sync (SFTP → TOGaDeskSupport customers) + Ivanti Ticket Sync](features/uma-student-sync.md) | Daily SFTP full-roster file loads UMA students into TOGaDeskSupport.customers (clientId 10); open for the file format, insert/deactivate rules, how the Ivanti t |
 | [Walmart Daily Sales Report (send_toga_walmart_report.php)](features/walmart-daily-sales-report.md) | The two Walmart sales workbooks (summary + details) built from `TOGA_Walmart` (in-store) and `TOGA_WalmartEComm` (.COM); open for the pseudo-store 1472 channel |

package/knowledge/1.0/apps/worker/features/staples-asn-856-outbound.md

@@ -0,0 +1,67 @@
+---
+title: Staples 856 ASN Outbound (NetSuite fulfillments → SFTP, tracked in StaplesAdvanceShipNotices)
+framework: "1.0"
+repo: worker
+project: Worker
+client: staples
+type: client-feature
+status: active
+updated: 2026-10-01
+owners: [mhammontree]
+files:
+  - worker/crons/sync/staples/staples_asn_netsuite.php
+  - dbchanges/Common/MH/2026-09-30.sql
+  - worker/schedules/cron.worker.sync.json
+related:
+  - ../../../../clients/staples/profile.md
+  - ./staples-cxml-order-import.md
+  - ../../../../clients/staples/features/oneuptime-sftp-folder-monitors.md
+  - ../../library/features/cron-execution-monitoring.md
+  - ../../library/features/mvc-page-pattern-and-app-skeleton.md
+---
+
+Hourly cron sending Staples 856 ASNs for shipped NetSuite item fulfillments, one row per fulfillment in `Common.StaplesAdvanceShipNotices`; open for the queue/retry/dedupe design, the 855 gate, statuses, manual SKIPPED, alerts, and how to test it locally.
+
+## Summary
+
+`staples_asn_netsuite.php` (`50 * * * *`, `schedules/cron.worker.sync.json`) builds a cXML 856 ASN for each shipped item fulfillment of NetSuite customer **30057** and puts it in Staples SFTP `/out`. Since TRUE-82180 (2026-09-30) every fulfillment is tracked in **`Common.StaplesAdvanceShipNotices`** (`db_common`), so a shipment is retried until sent and never sent twice unless its shipping data changed.
+
+**Rule (decided 2026-09-30, Jeff via Mark):** never send Staples the same ASN twice unless the shipping data changed. The fix stays in 1.0 — 2.0 would need an importer rewrite, and NetSuite use will likely move to an internal replacement app later. Old backlog and POs with no 855 are handled by hand, not in code.
+
+## How it works
+
+Table: one row per item fulfillment, `UNIQUE itemFulfillmentInternalId`. Status `PENDING` (not sent; `lastError` says why it waits) / `SENT` / `FAILED` (gave up, needs a developer) / `SKIPPED` (manual only, never retried).
+
+Each run (`cronInitialization(14400, false)` — must **not** use the read cluster; ends with `cronFinished()` / `closeConnections()`):
+
+1. **Queue.** Search 30057 fulfillments modified in the last 7 days (`ASN_LOOKBACK_SECONDS`). New fulfillment → insert `PENDING`. Existing row whose `dtFulfillmentModified` changed → back to `PENDING` (unless `SKIPPED`).
+2. **Process** up to 200 (`ASN_MAX_ROWS_PER_RUN`) `PENDING` rows, oldest `dtUpdated` first.
+3. **855 gate.** A row waits (stays `PENDING`) until `855_<PO>_*` exists in SFTP `/archive`. Likely reason the gate exists: it limits ASNs to orders that came in through the Staples cXML feed (the 856 needs the original cXML in `/archive/archive_in` for Staples line numbers) and keeps 855 before 856. Kept as a wait, not a drop.
+4. **Dedupe by `shippingHash`** = sha1 of shipping data only (carrier, ship-to, boxes with tracking/weight/items/serials, services). It excludes the random Package identifier, timestamps, and `noticeDate`, so a rebuild of unchanged data gives the same hash. Same hash as last sent → no file. A changed hash re-sends with operation still **`new`** (decided 2026-09-30).
+5. **Pre-go-live shipments** (`tranDate` before `MIN(dtCreated)` of the table, no hash yet) are marked `SENT` without sending if an old `856_<PO>_*` file in `/archive` or `/out` contains `shipmentID= "<tranId>"` (the old job wrote a space before the quote).
+6. **Send.** File name `856_<PO>_<fulfillmentInternalId>_<time>`; raw file logged to FileLog job `STAPLES_ASN_OUT_ORDER`. In test mode (`App_Registry::inTestMode()`) the file is written only to the cache folder — never uploaded to the live SFTP.
+7. **Alerts.** One email (same 3 recipients as before) listing rows not sent after 24 h; each row alerts once (`dtAlerted`, cleared on requeue).
+
+**Errors are per row.** A local `set_error_handler` turns PHP warnings into a row failure (otherwise 1.0 `App_Error` exits the whole run). A row fails up to 5 times (`ASN_MAX_SEND_ATTEMPTS`), then `FAILED`. A `SoapFault` / `RuntimeException` (NetSuite or SFTP down) moves the row to the back and stops the run.
+
+**Stop checking a PO handled by hand:** `UPDATE StaplesAdvanceShipNotices SET status = 'SKIPPED' WHERE poNumber = '<PO>';`
+
+**Deploy order:** merge dbchanges first (`_production` applies on the next worker deploy), then deploy worker.
+
+## Testing locally
+
+1. Load prod `Common` locally and run `dbchanges/Common/MH/2026-09-30.sql`.
+2. Set `ENVIRONMENT` to your dev config name first (see the unset-`ENVIRONMENT` gotcha in [1.0 MVC page pattern](../../library/features/mvc-page-pattern-and-app-skeleton.md)) — otherwise the run uses production config.
+3. Run with PHP 7.2 (`xampp_7_2_33`). It reads prod NetSuite and the live SFTP read-only; test mode keeps files in cache.
+
+Expected (2026-10-01 run): first run 101 rows, 90 matched as already sent by the old job, 0 files, ~640 s; second run ~100 s, nothing sent; changed data or a new shipment = exactly one file; alert once per row.
+
+## Gotchas
+
+- **Old design (pre-TRUE-82180) — why ASNs were late and doubled.** It searched only fulfillments modified in the last 3600 s, so each shipment got one chance; a skipped one went out only when someone edited the fulfillment (days/weeks late, ~half of ASNs). Dedupe checked only `/archive` by prefix `856_<PO>_<n>` where `n` restarted every run, and every fulfillment on the SO was re-sent when one changed.
+- **Do not use `Common.CronJobExecutions` or `Logs.FileLog` as a watermark/dedupe source** — both are too big and unindexed for per-run lookups (see [Cron execution monitoring](../../library/features/cron-execution-monitoring.md) and the FileLog note below). That is why the dedicated table exists.
+- **`Logs.FileLog` has no index on `job` / `fileName`** (verified locally) and `Logs` is ~400 GB — never `LIKE`-scan its BLOB per record; it cannot be pulled locally in full.
+- **Hardcoded credentials (parked).** The SFTP password and the cXML SharedSecret are hardcoded in `staples_asn_netsuite.php`; they should move to config.
+
+## Change history
+- 2026-10-01 — First doc. TRUE-82180 rewrite: `StaplesAdvanceShipNotices` queue with 7-day lookback, shipping-hash dedupe, per-row retry/FAILED, 24 h alert, test mode never uploads, old-job match for pre-go-live shipments. Committed worker `cf5198a27`, dbchanges `8b1f7cb` on branch TRUE-82180; not deployed as of 2026-10-01. (mhammontree)

package/knowledge/INDEX.md

@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 1.0 framework
 
 - **library** (Library) _(framework core)_ — 30 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
-- **worker** (Worker) — 40 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
+- **worker** (Worker) — 41 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 - **dbchanges** (Database Changes) _(framework core)_ — 1 doc(s) → [1.0/apps/dbchanges/INDEX.md](1.0/apps/dbchanges/INDEX.md)
 - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
 - **togadesk** (TOGa Desk) — 13 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)

package/knowledge/clients/staples/profile.md

@@ -4,18 +4,20 @@ framework: "1.0"
 apps:
   - worker
   - worker2
+  - dbchanges
   - dbchanges2
 project: Worker
 client: staples
 type: profile
 status: active
-updated: 2026-09-18
+updated: 2026-10-01
 owners: [jcardinal, bala, mhammontree]
 files:
   - worker/crons/sync/staples/sync_staples_cxml.php
   - worker/crons/sync/staples/staples_asn_netsuite.php
 related:
   - ../../1.0/apps/worker/features/staples-cxml-order-import.md
+  - ../../1.0/apps/worker/features/staples-asn-856-outbound.md
   - ./features/oneuptime-sftp-folder-monitors.md
   - ../../1.0/apps/worker/features/netsuite-sales-order-sales-rep-sourcing.md
 ---
@@ -27,7 +29,7 @@ Staples — headless Worker-1.0 integration client (two SFTP crons); open for it
 Staples is a **headless** integration client — no UI, Worker 1.0 only, depending on the `library` (1.0 `App_`) core. Its entire footprint is two crons under `worker/crons/sync/staples/`, both scheduled in `schedules/cron.worker.sync.json`:
 
 1. **`sync_staples_cxml.php`** — `45 * * * *`, "Create Sales Orders in Netsuite From Staples". Inbound: pulls cXML purchase orders off SFTP, creates NetSuite Sales Orders, and writes 855 acknowledgements back. The documented feature — see **`1.0/apps/worker/features/staples-cxml-order-import.md`**.
-2. **`staples_asn_netsuite.php`** — `50 * * * *`, "Create ASN from net suite to staples". Outbound ASN (Advance Ship Notice) from NetSuite back to Staples. (Not yet documented in detail.)
+2. **`staples_asn_netsuite.php`** — `50 * * * *`, "Create ASN from net suite to staples". Outbound 856 ASN (Advance Ship Notice) per shipped NetSuite fulfillment, tracked in `Common.StaplesAdvanceShipNotices`. Rule: never send the same ASN twice unless shipping data changed; stays in 1.0 (decided 2026-09-30). See **`1.0/apps/worker/features/staples-asn-856-outbound.md`**.
 
 **NetSuite identity:** Staples is the NetSuite parent entity/customer **internalId 30057**. Individual Staples buying accounts are NetSuite **child customers** matched by their Staples `CustomerID` against **saved search 8412** (match key `custentity_external_customer_id`; location from `custentity_customer_location`).
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.925",
+  "version": "1.0.926",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",