toga-ai 1.0.602 → 1.0.604

package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

@@ -6,7 +6,7 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-08-17
+updated: 2026-08-18
 owners: ["dfranks", "bala", "jcardinal", "mhammontree", "snaredla"]
 files:
   - worker/crons/toga2/netsuite/common_sync_togasupply.php
@@ -27,6 +27,7 @@ related:
   - ../../library/features/toga2-api-client-and-bridge.md
   - ../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
   - ../../../clients/elite/features/netsuite-togasupply-sync.md
+  - ../../../clients/compass-usa/features/sales-order-line-renumbering.md
 ---
 
 ## Summary
@@ -402,9 +403,46 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
   `itemFulfillmentItems` or `invoiceItems` reverse-hasMany collections (only the PO-link bridges), so
   the fix keys off the recorded "could-not-delete" set rather than trusting those collections — see
   [App_Api_Toga2 client & bridge](../../library/features/toga2-api-client-and-bridge.md).
+- **⚠ A second, distinct poison-order variant froze SALES_ORDERS again on SO 14106 (fixed
+  2026-08-18) — this one was on the api2/2.0 side, not in `library`.** Fixing the 2026-08-17
+  renumber-DELETE bug let Compass USA's sync advance past SO 7447 (`06:13:26`) to SO **14106**
+  (`06:30:41`), which then threw for a completely different reason: the api2 `PUT /sales-orders`
+  fired `_Model_Compass_SalesOrder::postPut()` → `_Model_Compass_SalesOrderItem::renumberLineNumbers()`,
+  whose **single-UPDATE renumber raised MySQL 1062 duplicate-key** on any order whose `id`-order ≠
+  `lineNumber`-order (SO 14106: `id 86674` line 19→6 collided with `id 86675` still on 6). The 1062
+  fataled the api2 request, `App_Api_Toga2::send()` threw, and — same as before — the
+  `try/catch`-less SALES_ORDERS loop aborted the whole section:
+  `NETSUITE_EXECUTION_MODE_SALES_ORDERS='1-RUNNING'`, watermark frozen at `2026-08-12 06:30:41`
+  (~6 days), other five sections healthy at `432000-IDLE`. **Root-cause fix is in `_underscore`
+  (2.0), not this worker** — the renumber is now two-phase (park negative, flip positive); full
+  detail in [Compass sales-order line-number renumbering](../../../clients/compass-usa/features/sales-order-line-renumbering.md).
+  The worker/1.0 side is only the trigger path (unchanged). **Lesson: the missing SALES_ORDERS
+  per-record `try/catch` means every future api2-side write error on a single order re-freezes the
+  section** — the poison-order class is not closed by fixing individual orders.
+- **⚠ Diagnostic trap: a fatal api2 PUT leaves NO row in `Logs_<Client>.Api` — check `Logs.Issue`
+  instead.** When the api2 request fatals *before* the request-logging middleware (as the 1062
+  renumber collision does), it writes **nothing** to `Logs_Compass.Api`. Do **not** conclude "no
+  failing api2 call happened" from an empty per-client Api log. The error surfaces only in the shared
+  `Logs.Issue` tracker (here **Issue 246**, first seen 2026-08-17 22:16), with the full trace
+  pointing at `renumberLineNumbers ← postPut ← V2.php processRoutePairs`. Localize the stalled
+  section from `Client_<Client>.Parameters` (`NETSUITE_EXECUTION_MODE_*` RUNNING-vs-IDLE +
+  `NETSUITE_LAST_SYNC_DATETIME_*` frozen watermark), then read `Logs.Issue` — not `Logs_<Client>.Api`
+  — for the throwing order.
 
 ## Change history
 
+- 2026-08-18 — Recorded the **next poison-order variant** that re-froze Compass USA's SALES_ORDERS
+  section on SO **14106** right after the 2026-08-17 fix let the sync advance past SO 7447. This one
+  is 2.0-side: the api2 `PUT /sales-orders` → `postPut` → `renumberLineNumbers()` **single-UPDATE**
+  renumber threw MySQL **1062** on any order whose `id`-order ≠ `lineNumber`-order, fataling the api2
+  request; the `try/catch`-less SALES_ORDERS loop aborted the section (mode `1-RUNNING`, watermark
+  stuck `2026-08-12 06:30:41`, ~6 days). Root-cause fix is in **`_underscore`** (two-phase renumber —
+  see the [renumbering doc](../../../clients/compass-usa/features/sales-order-line-renumbering.md));
+  this worker is only the trigger and is unchanged. Also recorded the **diagnostic trap**: the fatal
+  PUT wrote **no** `Logs_Compass.Api` row (it died before the logging middleware), so the failure was
+  visible only in shared `Logs.Issue` (Issue 246) — do not read an empty per-client Api log as "no
+  failing call." Reinforced that the missing SALES_ORDERS per-record `try/catch` keeps the
+  poison-order class open. Diagnosis only on this repo (read-only). (jcardinal)
 - 2026-08-17 — Fixed a Compass USA ~5-day SALES_ORDERS freeze: an **invoiced** line that NetSuite
   renumbered could not be DELETEd (FK `InvoiceItems.salesOrderItemId` `RESTRICT` → MySQL 1451 →
   api2 EV-11), and because the SALES_ORDERS loop has **no per-record `try/catch`** the throw aborted

package/knowledge/2.0/apps/_underscore/features/email-template-sending.md

@@ -7,7 +7,7 @@ client: shared
 type: feature
 status: active
 updated: 2026-08-03
-owners: ["jcardinal", "bala", "mhammontree"]
+owners: ["jcardinal", "bala", "mhammontree", "apeterson"]
 files:
   - _underscore/Model/Client/EmailTemplate.php
   - _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php
@@ -98,6 +98,16 @@ can keep using `sendEmail($api, ...)`.
   2026-06-15 the only entry point was `sendEmail(&$api, ...)`, so callers with no API
   context faked one: `$api = (object)['client' => (object)['clientIdentifier' => …]]`.
   That hack is obsolete — pass the identifier to `send()` instead.
+- **⚠ Template vars MUST be spread into the variadic, never passed as a bare array.** Both
+  `sendEmail(&$api, $uuid, $to, $cc, $bcc, ...$args)` and `send(...)` collect the template
+  variables via `...$args`. Pass them **spread** so PHP 8.1+ named-argument collection yields
+  string keys/string values (`sendEmail($api, $uuid, $to, [], [], ...['userName'=>$name, ...])`
+  → `$args = ['userName'=>$name, ...]`). Passing the map as a single positional argument
+  (`sendEmail($api, $uuid, $to, [], [], $vars)`) makes `$args = [0 => [...]]` — a numeric key
+  whose value is an *array*. `replaceTemplateVariables()` then runs `str_replace('{0}', <array>, …)`
+  → `TypeError: str_replace(): Argument #2 must be of type string, array given`, surfacing as a
+  production 500 / **EO-1**. `Model/Compass/*` gets this right (spread); the four `Model/Quad/*`
+  call sites were copied from Compass but dropped the `...` — fixed 2026-08-18.
 - **`sendEmail()`'s signature is load-bearing for scripted APIs** — the Record Script engine
   (`api2/Component/Api/V2/V2.php`, ~line 3594) calls the method with `api` as a named
   argument, so the first param must stay `&$api`. Do not "clean it up" by removing it.
@@ -151,6 +161,12 @@ worker method) in-process instead.
 
 ## Change history
 
+- 2026-08-18 — Fixed the Quad order-approval/rejection emails' production 500
+  (`str_replace(): Argument #2 must be string`, EO-1): the four `Model/Quad/ApprovalDecision.php`
+  + `Model/Quad/SalesOrder.php` call sites passed the template-vars map as a bare positional
+  array, so `...$args` collected `[0 => [...]]` and `replaceTemplateVariables()` fed an array to
+  `str_replace`. Fix = spread the array (`...[...]`) to match the working `Model/Compass/*`
+  pattern. Added the **"template vars must be spread, never a bare array"** gotcha. (apeterson)
 - 2026-08-03 — Refined the `CharSet` gotcha: the **transmit** side doesn't set it either, so the
   effective wire charset is **iso-8859-1**, and a body that needs accents should escape with
   `htmlentities($v, ENT_QUOTES | ENT_SUBSTITUTE, 'UTF-8', false)` (details on

package/knowledge/2.0/apps/api2/INDEX.md

@@ -6,7 +6,7 @@
 | [API Payload Interceptors (metadata-registered prePost/postPut model hooks)](features/api-payload-interceptors.md) | A `_Model`'s `prePost()` / `postPost()` / `prePut()` / `postPut()` hooks are **not** called by the model. | api2/Component/Api/V2/V2.php, api2/Controller/Index.php, _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Compass/PurchaseOrder.php, _underscore/Model/Elite/SalesOrder.php, _underscore/Model/Elite/ServiceRequest.php, _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/SalesOrderItem.php, dbchanges2/Core/2026-06-30a - ItemFulfillmentStageDefaultInterceptor.sql, dbchanges2/Core/2026-08-06a - Service request and sales order payload interceptors.sql, dbchanges2/Client_Compass/2026-08-06 - RemoveBrokenSalesOrderItemPostPostInterceptor.sql, dbchanges2/Client_Compass/2026-08-14a - AddSalesOrderInactiveStandaloneItemPreInterceptors.sql, dbchanges2/Client_CompassCanada/2026-08-14a - AddSalesOrderInactiveStandaloneItemPreInterceptors.sql |
 | [Multi-Client (Cross-Client) Data Retrieval](features/cross-client-data-retrieval.md) | A single authenticated V2 GET listing can return records across **many** clients (designed for 1000+) that the caller is entitled to, honoring **each target cli | api2/Component/Api/CrossClient/CrossClient.php, api2/Component/Api/V2/V2.php, api2/Controller/Index.php, _underscore/Model/Cache/Table.php, _underscore/Model/Cache/Tables/Client.php, _underscore/Model/Core/Record.php, worker2/Worker/Platform/Cache.php, worker2/Controller/Index.php, worker2/_.php, dbchanges2/Cache/2026-06-30a - MultiClientCacheTables.sql, dbchanges2/Core/2026-06-30b - CacheClusterRegistrationAndRecordTtl.sql, dbchanges2/Core/2026-07-27a - PlatformCacheCleanCron.sql |
 | [Encrypted-User-UUID Auth Handoff (/auth/encrypted-user-uuid)](features/encrypted-user-uuid-auth-handoff.md) | `POST /auth/encrypted-user-uuid` is the intended **cross-client / SSO-handoff identity mechanism**: given an encrypted `{client, user}` UUID pair, it mints a fr | api2/Component/Api/CrossClient/CrossClient.php |
-| [ENVIRONMENT (not the EB environment name) decides the _underscore branch and Config file](features/environment-variable-drives-underscore-branch.md) | An api2 Elastic Beanstalk instance decides **which `_underscore` branch it clones** and **which `Config/<env>.ini` it loads** from the EB environment property * | api2/.ebextensions/git.php, api2/.ebextensions/php_include_underscore.config, api2/.ebextensions/git.sandbox-dev.json, api2/Config/beta.ini, api2/Config/sandbox-dev.ini, api2/Component/Api/V2/V2.php |
+| [ENVIRONMENT (not the EB environment name) decides the _underscore branch and Config file](features/environment-variable-drives-underscore-branch.md) | An api2 Elastic Beanstalk instance decides **which `_underscore` branch it clones** and **which `Config/<env>.ini` it loads** from the EB environment property * | api2/.ebextensions/git.php, api2/.ebextensions/php_include_underscore.config, api2/.ebextensions/git.sandbox-dev.json, api2/.ebextensions/git.sandbox-client.json, api2/Config/beta.ini, api2/Config/sandbox-dev.ini, api2/Component/Api/V2/V2.php |
 | [Health-check endpoint (/health liveness short-circuit)](features/health-check-endpoint.md) | `_Controller_Index::api()` short-circuits **liveness/health-probe** requests to an HTTP 200 **before** any routing, DB bootstrap, or V2 engine work runs. | api2/Controller/Index.php |
 | [Language Translation Layer (audience.language + sidecar tables)](features/language-translation-layer.md) | Serves the same TOGa data (Item title/description/longDescription, plus item **feature** text — `Features.name`, `ItemCategoryFeatureGroups.name`, `ItemFeatures | api2/Component/Api/V2/V2.php, api2/Component/Api/V2/Response/Response.php, _underscore/Model/Core/Setting.php, _underscore/Model/Core/RecordField.php, _underscore/Model/Core/DefaultGlobalSetting.php, _underscore/Model/Client/ItemTranslation.php, _underscore/Model/Client/FeatureTranslation.php, _underscore/Model/Client/ItemCategoryFeatureGroupTranslation.php, _underscore/Model/Client/ItemFeatureTranslation.php, dbchanges2/Client/2026-06-23a - ItemTranslations.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql, dbchanges2/Client/2026-07-13a - FeatureTranslations.sql, dbchanges2/Client/2026-07-13b - FeatureTranslationsAcl.sql, dbchanges2/Core/2026-06-23a - RecordFieldsTranslationColumn.sql, dbchanges2/Core/2026-06-23b - ItemTranslationsRecord.sql, dbchanges2/Core/2026-07-13 - FeatureTranslationsRecord.sql |
 | [/auth/login resolves the client from the email domain, not the Bearer token (cross-client user path)](features/login-cross-client-user-resolution.md) | `POST /v2/auth/login` (email/password user login) can silently swap the target client mid-request. | api2/Component/Api/V2/V2.php, _underscore/String.php |

package/knowledge/2.0/apps/api2/features/environment-variable-drives-underscore-branch.md

@@ -7,11 +7,12 @@ client: shared
 type: feature
 status: active
 updated: 2026-08-06
-owners: ["bala", "mhammontree"]
+owners: ["bala", "mhammontree", "apeterson"]
 files:
   - api2/.ebextensions/git.php
   - api2/.ebextensions/php_include_underscore.config
   - api2/.ebextensions/git.sandbox-dev.json
+  - api2/.ebextensions/git.sandbox-client.json
   - api2/Config/beta.ini
   - api2/Config/sandbox-dev.ini
   - api2/Component/Api/V2/V2.php
@@ -116,12 +117,36 @@ one.
   `_underscore`. (Contrast: the 1.0 **tools** app works cleanly because its helper *and* its config
   live in the same repo, and it clones `library` from a **fixed** branch, `_production`, rather
   than an `ENVIRONMENT`-derived one.)
+- **⚠ A `git.<env>.json` `"branch"` key can pin a tier to the WRONG `_underscore` branch, silently.**
+  When a code fix pushed to a branch does not appear after an api2 deploy, before debugging the
+  code verify which branch that EB environment's `.ebextensions/git.<env>.json` actually clones:
+  a present `"branch"` value overrides the `'_' . strtolower(ENVIRONMENT)` default. **Concrete
+  (fixed 2026-08-18):** `git.sandbox-client.json` had `"branch": "_production"`, so the
+  `sandbox-client` environment ran `_underscore` from `_production` — never `_sandbox-client` —
+  which is why fixes merged to `_sandbox-client` never showed up there. Confirmed two ways: the
+  config file value, and live runtime behavior matching `_production` (unfixed) rather than
+  `_sandbox-client` (fixed). Fix = remove the `"branch"` key so it falls back to `_sandbox-client`.
+- **Deploy path is build-time, so a git push alone does not update running instances.** The
+  `_underscore` clone happens in the prebuild hook at build time: commit the fix on the feature
+  branch, merge that branch into the deploy-channel branch (`_` + `ENVIRONMENT`, e.g.
+  `_sandbox-client`), push, **then redeploy the EB environment** to re-run the clone.
+- **⚠ SECURITY — every `.ebextensions/git.*.json` carries a plaintext GitHub PAT** committed in
+  the repo (used to clone `_underscore`). Pre-existing leak; treat the committed tokens as
+  compromised, rotate, and move them to EB environment properties / SSM. Do **not** copy the token
+  value anywhere — this note records only that the credential lives in these files.
 - **Duplicate integration code is the trap this creates.** When the same capability is needed in
   both tiers it must exist twice (once per framework) — keep the credential/config location
   documented in both places so a rotation updates both.
 
 ## Change history
 
+- 2026-08-18 — Recorded a second concrete `"branch"`-pin trap: `git.sandbox-client.json` pinned
+  `"branch": "_production"`, so the `sandbox-client` EB environment cloned `_underscore` from
+  `_production` and never received `_sandbox-client` merges (confirmed via both the config value and
+  live runtime matching the unfixed `_production` code); fix = drop the `"branch"` key. Added the
+  general diagnostic ("fix not appearing after deploy → check `git.<env>.json` branch pin first"),
+  the build-time deploy note (merge into `_`+ENVIRONMENT then redeploy — a push alone is inert),
+  and the ⚠ plaintext-GitHub-PAT-in-`git.*.json` security gotcha (location only). (apeterson)
 - 2026-08-06 — TRUE-80282 deploy: independently re-confirmed the mapping from a second tier
   (`api.beta.togahub.com` = `ENVIRONMENT=beta` → `Config/beta.ini` → branch `_beta`) at a cost of two
   deploy cycles, and recorded **why** the branch derives: there is deliberately **no

package/knowledge/clients/compass-usa/features/sales-order-line-renumbering.md

@@ -6,8 +6,8 @@ project: _Underscore
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-08-11
-owners: ["bala"]
+updated: 2026-08-18
+owners: ["bala", "jcardinal"]
 files:
   - _underscore/Model/Compass/SalesOrderItem.php
   - _underscore/Model/Compass/SalesOrder.php
@@ -43,6 +43,29 @@ Live production wiring (`ApiPayloadInterceptors`, Compass):
 Add/edit renumbering rides the **parent** (`sales-orders`) hooks; delete renumbering rides the
 **child**'s `preDelete`.
 
+### How the renumber SQL must run — two phases, never one UPDATE (2026-08-18)
+
+Lines are numbered `1..N` by `ROW_NUMBER() OVER (ORDER BY id)`, but the write **cannot** be a
+single `UPDATE … JOIN (SELECT id, ROW_NUMBER() … AS rowNumber) SET lineNumber = rowNumber`.
+`SalesOrderItems` carries `UNIQUE(salesOrderId, lineNumber)`, which MySQL enforces **per row as
+each row updates**, not deferred to statement end. Whenever `id`-order ≠ current `lineNumber`-order,
+a row that takes a number another **not-yet-updated** row still holds raises **MySQL 1062
+duplicate-key** mid-statement and the whole UPDATE (and its request) fatals.
+
+> Observed: SO **14106**, `id 86674` moving line 19→**6** collided with `id 86675` still sitting on
+> line 6 → `Duplicate entry '14106-6' for key 'salesOrderId'`.
+
+The collision-free form (same numbering outcome, `ROW_NUMBER() OVER (ORDER BY id)` unchanged) is a
+**two-phase** renumber:
+
+1. **Park negative** — `SET lineNumber = -Temporary.rowNumber` for every line. Negatives never
+   collide with the positive originals or finals, and each `-rowNumber` is distinct.
+2. **Flip positive** — `SET lineNumber = -lineNumber WHERE … AND lineNumber < 0`.
+
+Values only ever range `-N..N`, safe for the `SMALLINT` `lineNumber` column. Both UPDATEs run in
+the **same api2 request → same `DB_CLIENT` transaction**, so the intermediate negative state is
+never externally visible and the pair rolls back together on failure. Signature unchanged (PHP 8.1).
+
 ## Why delete-time renumbering CANNOT be a `postDelete`
 
 A `postDelete` interceptor can never fire at all. api2's post-processing interceptor block runs
@@ -87,9 +110,42 @@ fire reliably as the request's **first-resolved record**. Verified healthy after
   subclasses inherit it. Do not add per-region copies.
 - **These interceptor rows have no route and no audit trail.** They are direct SQL only; nothing
   records who added or removed one. Snapshot Compass's row set into the ticket before changing it.
+- **⚠ The renumber must never be a single collision-prone UPDATE.** `UNIQUE(salesOrderId,
+  lineNumber)` is enforced per-row mid-statement, so a one-shot `SET lineNumber = rowNumber` throws
+  MySQL 1062 the moment any row takes a number a not-yet-updated row still holds. Renumber in two
+  phases (park negative, then flip positive) — see *How it works*.
+- **Both Compass regions are affected.** `renumberLineNumbers()` lives on the shared
+  `_Model_Compass_SalesOrderItem` base, so **Compass Canada** (`compass-canada`) inherits the same
+  bug and the same fix; there is no per-region copy to touch.
+- **⚠ The `preDelete` (delete-a-middle-line) path is a pre-existing latent collision — unchanged by
+  this fix.** When a middle line is deleted, the row being removed keeps its positive `lineNumber`
+  during the renumber, so `preDelete` can still hit the same 1062 as the old add/edit code did. It
+  is healthy in practice only because deletes are almost always the **last** line. If middle-line
+  deletes become common, `preDelete` needs the same two-phase treatment.
+- **This was not a one-order anomaly.** As of 2026-08-18, **38,947 of 105,169** Compass orders (37%)
+  have `lineNumber`-order ≠ `id`-order (`ROW_NUMBER() OVER (PARTITION BY salesOrderId ORDER BY id)`)
+  — any of them ran the buggy renumber on its next PUT/edit and could collide. No mass data retrofix
+  is needed for the freeze: the two-phase renumber normalizes each order as the sync re-touches it.
+  A post-deploy sweep of that set could confirm whether any were left with gap'd numbering (which
+  MITS/PO linking depends on — see [MITS PO → SO item linking](./mits-po-to-so-item-linking.md))
+  from silently-rolled-back renumbers.
 
 ## Change history
 
+- 2026-08-18 — **Fixed** the `renumberLineNumbers()` single-UPDATE **1062 duplicate-key collision**.
+  The renumber ran as one `UPDATE … JOIN (SELECT id, ROW_NUMBER() OVER (ORDER BY id) AS rowNumber)
+  SET lineNumber = rowNumber`; because `UNIQUE(salesOrderId, lineNumber)` is enforced per-row
+  mid-statement, any order whose `id`-order ≠ `lineNumber`-order fataled with `Duplicate entry`
+  (observed SO 14106, `id 86674` line 19→6 vs `id 86675` still on line 6). Replaced with a
+  **two-phase** renumber (Phase 1 parks each line at `-rowNumber`, Phase 2 flips positive) that keeps
+  the exact `ROW_NUMBER() OVER (ORDER BY id)` outcome, never collides, stays within the `SMALLINT`
+  range, and runs both UPDATEs in the same api2 `DB_CLIENT` transaction (atomic, no visible negative
+  state). Shared base class → **Compass Canada inherits the fix**. Systemic scope: 38,947/105,169
+  (37%) Compass orders currently have out-of-order numbering and were all exposed. Noted the
+  pre-existing `preDelete` middle-line-delete path can still collide identically (unchanged, latent).
+  This collision was the root cause of the Compass USA SALES_ORDERS sync freeze on SO 14106 — cascade
+  + diagnosis recorded in the [NetSuite → TOGa Supply per-client sync](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md).
+  (Fix confirmed live in prod; local working tree still shows the pre-fix single UPDATE.) (jcardinal)
 - 2026-08-11 — Documented the as-built wiring and the reasoning behind it after a production
   post-mortem (investigation only, no code change): add/edit renumbering hangs off
   `_Model_Compass_SalesOrder::postPost`/`postPut` (record **14** `sales-orders`), delete-time

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.602",
+  "version": "1.0.604",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",