toga-ai 1.0.508 → 1.0.510

package/knowledge/1.0/apps/library/INDEX.md

@@ -8,7 +8,7 @@
 | [Diagnostic Dialog — View Recommended Services Routing](features/diagnostic-dialog-view-recommended-services.md) | Two "View Recommended Services" buttons exist in the TOGa Refresh 2026 SR view: 1. | library/app/model/toga/diagnostic.php, library/app/model/servicerequest.php |
 | [Elite Freshservice Sync (library)](features/elite-freshservice-sync.md) | `App_Api_Toga2` in `library/app/api/toga2.php` orchestrates bidirectional sync between TOGA 2 and TOGaDesk. | library/app/api/toga2.php |
 | [Branded HTML Email Templates (App_Email_Template)](features/email-templates.md) | `App_Email_Template` (`app/email/template.php`) is the base class for branded HTML emails in the 1.0 (`App_`) framework. | library/app/email/template.php, library/app/email/agilant.php |
-| [Error Capture in 1.0 (App_Error_Capture → shared 2.0 Logs DB)](features/error-capture-1-0.md) | The 1.0 side of the platform error-reporting pipeline (TRUE-78188). | library/app/error/capture.php, library/app/error.php, library/app/exception/business.php, library/app/cloud.php, worker/config.worker.ini, worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php |
+| [Error Capture in 1.0 (App_Error_Capture → shared 2.0 Logs DB)](features/error-capture-1-0.md) | The 1.0 side of the platform error-reporting pipeline (TRUE-78188). | library/app/error/capture.php, library/app/error.php, library/app/exception/business.php, library/app/api/toga2.php, library/app/cloud.php, worker/config.worker.ini, worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php |
 | [1.0 MVC Page Pattern & New-App Skeleton](features/mvc-page-pattern-and-app-skeleton.md) | This is the **reusable recipe for standing up a new 1.0 (`App_`) application** and for adding pages to one — the folder-based MVC routing, the page lifecycle, t | library/app/framework.php, library/app/frameworkindex.php, library/app/mvc.php, library/app/database.php, library/app/model.php, library/app/config.php |
 | [isFulfillable from NetSuite during Item Sync (Phase 1)](features/netsuite-item-isfulfillable-sync.md) | This is the **1.0 (Phase 1)** half of the `isFulfillable` feature: reading the NetSuite `isfulfillable` flag during item sync and stamping it onto the **Agilant | library/app/netsuite.php, library/app/api/toga2.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/backfill_isfulfillable_jul5.php |
 | [NetSuite SuiteQL/REST API Reference](features/netsuite-suiteql-api-reference.md) | General working reference for the Agilant NetSuite integration: how to authenticate, how SuiteQL behaves, and the confirmed schema of the tables/columns/codes w | library/app/api/netsuite/rest.php, library/ssl/netsuite_ec_key.pem, test/@dave/Junk Drawer/nsq.php |

package/knowledge/1.0/apps/library/features/error-capture-1-0.md

@@ -12,11 +12,13 @@ files:
   - library/app/error/capture.php
   - library/app/error.php
   - library/app/exception/business.php
+  - library/app/api/toga2.php
   - library/app/cloud.php
   - worker/config.worker.ini
   - worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
 related:
   - ../architecture.md
+  - ./toga2-api-client-and-bridge.md
   - ../../../2.0/apps/_underscore/features/error-reporting-issue-event.md
   - ../../../2.0/apps/worker2/features/error-escalation-cron.md
   - ../../tools/features/errors-curation-console.md
@@ -76,13 +78,42 @@ whichever occurrence happened to be seen first — i.e. one arbitrary order numb
 
 Compass USA's sales-order → MITS transmit rejection was the first business-exception use case.
 
-### Client attribution
+### Client attribution — resolved from the API client uuid at `App_Api_Toga2::authenticate()`
 
 `App_Error::setCurrentClientId()` exists in 1.0 for parity with 2.0's ambient current-client, but
-**1.0 has no central hook** equivalent to `_Database::registerClientDatabases()`. Client-scoped
-1.0 crons therefore **set it explicitly** (see
-`worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php`). Capture
-prefers an explicit `clientId` argument and falls back to the ambient value.
+**1.0 has no central hook** equivalent to `_Database::registerClientDatabases()`. Because of that,
+the explicit setter was only ever called by **one** hand-edited Compass cron, so
+**every other 1.0 cron recorded `clientId` NULL**. Evidence: `Logs.Event` held **2,093 rows with
+`clientId` NULL** against **107** with a client (all of them 2.0/api2 JWT traffic); issue `1Q`
+traced back to the 1.0 cron `crons/toga2/quad/import_po_and_asn.php`.
+
+**`App_Api_Toga2::authenticate()` is 1.0's true choke point.** `send()` authenticates before every
+call, and every 1.0→2.0 cron arrives there holding its `UUID_CLIENT`. It now calls
+`App_Error_Capture::setCurrentClientUuid($uuid)` — a **store only, no lookup**, because
+`authenticate()` is a happy-path function and must not take on a query.
+
+`App_Error_Capture` then resolves uuid → `Core.Clients.id` **lazily**, only when an error is
+actually captured, cached per process (`resolveClientIdFromUuid()`, private). It walks
+**`CORE_DATABASE_LINK_CANDIDATES = ['db_toga2core', 'db_prod_toga2core', 'db_core']`** because 1.0
+apps register the same Core connection under different link names. **Every failure path returns
+`null` and records no client rather than throwing.**
+
+Precedence is unchanged: an explicit `clientId` argument wins, then an explicit
+`setCurrentClientId()`, then the ambient uuid.
+
+> **⚠ The resolution must run INSIDE the existing
+> `App_Error::setThrowExceptionsEnabled(false)` guard.** An unconfigured DB link **warns** before
+> it fails, and in 1.0 a warning outside that guard terminates the cron (see the first gotcha
+> below). Do not hoist the lookup out of the guarded block.
+
+### The reference encoder is a byte-identical mirror of 2.0's
+
+`capture.php` carries its own copy of `encodeReference()` plus `REFERENCE_ALPHABET`
+(`'0123456789ACDEFGHJKMNPQRSTUVWXYZ'`) and `REFERENCE_OFFSET` (`128`), matching
+`_Model_Core_Logs_Issue` **byte for byte**. Both write the same **UNIQUE** `Issue.reference`
+column in the same shared table, so a divergence between them is a failed INSERT and a **lost**
+error. Change one, change both, deploy together. Rationale for the alphabet and the offset lives in
+the [2.0 doc](../../../2.0/apps/_underscore/features/error-reporting-issue-event.md#the-quotable-reference--base-32-over-an-unambiguous-alphabet-reworked-2026-08-04).
 
 ### Fingerprint fallback (no usable stack)
 
@@ -151,6 +182,17 @@ exists in `api2`/`worker2`. So 1.0 reads the AL1 container config at
 
 ## Change history
 
+- 2026-08-04 (later, **uncommitted/undeployed** at time of writing) — **Fixed: 1.0 errors recorded
+  `clientId` NULL on every cron but one** (2,093 NULL `Logs.Event` rows vs 107 with a client; issue
+  `1Q` from `crons/toga2/quad/import_po_and_asn.php`). `App_Api_Toga2::authenticate()` — 1.0's real
+  choke point, since `send()` authenticates before every call — now records the ambient client uuid
+  (store only, no lookup), and `App_Error_Capture` resolves uuid → `Core.Clients.id` lazily and
+  per-process on capture, walking `CORE_DATABASE_LINK_CANDIDATES`
+  (`db_toga2core`/`db_prod_toga2core`/`db_core`) because 1.0 apps name the Core link differently;
+  every failure path returns null rather than throwing, and the lookup deliberately runs inside the
+  `setThrowExceptionsEnabled(false)` guard because an unconfigured link *warns* before it fails.
+  Also mirrored 2.0's **base-32 reference encoder** (`REFERENCE_ALPHABET` + `REFERENCE_OFFSET = 128`)
+  byte-for-byte. (jcardinal)
 - 2026-08-04 — Built the 1.0 side of TRUE-78188: `App_Error_Capture` writing occurrences into the
   shared 2.0 `Logs` DB (opt-in via `[database_toga2logs]`, hand-written escaped SQL);
   `App_Exception_Business` with a fingerprint string byte-identical to 2.0's so one `issueKey`

package/knowledge/1.0/apps/library/features/toga2-api-client-and-bridge.md

@@ -6,7 +6,7 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-08-03
+updated: 2026-08-04
 owners: [jcardinal, mhammontree]
 files:
   - library/app/api/toga2.php
@@ -22,6 +22,7 @@ related:
   - netsuite-suiteql-rest-shim.md
   - ../../worker/features/netsuite-togasupply-per-client-sync.md
   - ../architecture.md
+  - ./error-capture-1-0.md
 ---
 
 ## Summary
@@ -77,6 +78,18 @@ belong to the NetSuite importer — documented in the per-client-sync doc, not h
 - Returns the decoded response object; callers read `->data->{resource}`, `->meta->nextPage`,
   `->isSuccess`, `->status`, `->messages[].code`.
 
+> **`authenticate()` is also 1.0's ambient-client choke point for error reporting** (added
+> 2026-08-04). Every 1.0→2.0 cron passes through it holding its `UUID_CLIENT`, so it calls
+> `App_Error_Capture::setCurrentClientUuid()` — **store only, no lookup**: this is a happy-path
+> function and must not acquire a query. The uuid → `Core.Clients.id` resolution happens lazily
+> inside error capture. Keep it that way; see
+> [error-capture-1-0](./error-capture-1-0.md).
+
+> **Latent, pre-existing, uninvestigated:** several `self::$togaClientUuid` references sit in
+> methods of this class (roughly lines 2880–4250) with **no such property declared** — an
+> undefined-property bug waiting to fire (and in 1.0, a warning inside an unguarded path
+> terminates the cron). Not introduced by, and not fixed by, the 2026-08-04 error-reporting work.
+
 ### Options DSL (how 1.0 expresses 2.0 queries)
 
 `$options` is an assoc array assembled into the 2.0 query string:
@@ -340,6 +353,11 @@ enable flags** and an optional `$monitorTogadeskDepartmentIds[]`:
 
 ## Change history
 
+- 2026-08-04 (**uncommitted/undeployed** at time of writing) — `authenticate()` now records the
+  ambient client uuid for error reporting (`App_Error_Capture::setCurrentClientUuid()`, store-only)
+  because it is the one place every 1.0→2.0 cron passes through holding its `UUID_CLIENT` — this is
+  what fixed 1.0 errors recording `clientId` NULL. Also noted the pre-existing undeclared
+  `self::$togaClientUuid` references (~L2880–4250) as a latent undefined-property bug. (jcardinal)
 - 2026-08-03 (later pass) — TRUE-79401 **static regression harness + one correction.** BUILT a
   no-DB PHP harness (`test/@Mark/AIG/test_multi_email.php`, 12 cases, PHP 7.2.33, `php -l` clean)
   that feeds mocked `/contacts` fragments through the email transform and asserts the delimited

package/knowledge/1.0/apps/tools/features/errors-curation-console.md

@@ -72,6 +72,48 @@ curated text, affected clients) right, with an always-present Type chip.
   time.
 - Hourly-occurrence chart; fingerprints table re-sized for the narrow column with an icon-only
   Move button.
+- Header chip order is **Issue / Environment / Type / Urgency** — Type was moved ahead of Urgency so
+  urgency finishes the row at the far right.
+
+### Table presentation rules (established 2026-08-04)
+
+**Stacked datetimes in every table on both pages** — listing *Last seen*; detail *Events → When*,
+*Affected clients → Last*, *ClickUp episodes → Opened/Resolved*, *Fingerprints → First seen*. A
+shared `$datetime` closure emits
+`<span class="dt"><span class="dt-date">…</span><span class="dt-time">…</span></span>`, each part
+`nowrap`, split on the single space MySQL puts between date and time; a blank value renders an em
+dash. Tabular figures keep digits aligned down the column.
+
+> **Why:** letting a datetime wrap naturally let the browser pick its own break point, and the point
+> it picked was **inside the day of the month** (`2026-08-` / `04`). This replaced the old
+> `.wrap-date` / `.wrap-anywhere` approach, whose now-dead CSS rule was removed.
+
+**The summary column has a 100px floor via table overflow, not a cell `min-width`.** The summary
+`<col>` is deliberately left **unwidthed** so it absorbs leftover space — which is also why *every
+other* column must declare a width, or `col-type` absorbs it instead. The floor comes from
+`.issue-listing { min-width: 1026px }` inside a new `.table-scroll { overflow-x: auto }` wrapper.
+
+> **⚠ Under `table-layout: fixed` a browser compresses ALL columns proportionally at narrow
+> viewports and ignores `min-width` on a cell.** A cell-level `min-width` therefore cannot work —
+> the table has to overflow and scroll. `col-when` narrowed 168px → 104px now that dates stack (the
+> 64px went to the summary), and a duplicate `.col-type` rule was removed in favour of the
+> pre-existing 104px one. **The pager sits outside the scroll wrapper.**
+
+**Whole-row click replaced the two per-cell links** (Issue ID and Summary). The `<tr>` carries
+`class="row-link"`, `data-href`, `tabindex="0"` and an `aria-label`; `.issue-id-link` was renamed
+`.issue-id` (keeping the accent colour as the affordance) and `tr.row-link` got cursor/hover/
+`focus-visible` rules.
+
+This page was deliberately **JavaScript-free**, and this is the one thing that cannot be done
+without it: no element but an anchor is clickable, and an anchor cannot wrap a `<tr>`. A small
+**delegated** click+keydown handler was added. Ctrl/Cmd/Shift-click still opens a new tab;
+Enter/Space work because removing the anchors removed the tab stop; clicks on `a`, `button`,
+`input`, `select`, `textarea`, `summary` or `label` are ignored so future in-row controls keep their
+own behaviour.
+
+> **Known trade-off:** a screen reader now announces these as **labelled rows, not links**.
+> `role="link"` was deliberately **not** applied, because it would stop the element being a table
+> row.
 
 **This page notifies nobody.** Escalation, ClickUp ticketing, and email are owned entirely by
 `_Worker_Infrastructure_Errors::Escalate`. Adding a notify action here would create a second,
@@ -128,6 +170,21 @@ They were renamed from plural on 2026-08-01, after the pipeline was already in p
 
 ## Change history
 
+- 2026-08-04 (later, **uncommitted/undeployed** at time of writing) — UI pass on both `/errors`
+  pages: **stacked datetimes** in every table via a shared `$datetime` closure emitting
+  `.dt`/`.dt-date`/`.dt-time` (a naturally wrapping datetime broke inside the day of the month), and
+  the dead `.wrap-date`/`.wrap-anywhere` rules removed; a **100px floor on the summary column**
+  delivered as `.issue-listing { min-width: 1026px }` inside a new `.table-scroll` overflow wrapper
+  (with the pager outside it), because `table-layout: fixed` ignores a cell `min-width` and
+  compresses all columns proportionally — `col-when` narrowed 168px → 104px and a duplicate
+  `.col-type` rule removed; the two per-cell links replaced by a **whole-row click**
+  (`tr.row-link` + `data-href` + `tabindex` + `aria-label` and one small delegated click/keydown
+  handler — the single unavoidable use of JS on this deliberately JS-free page, since an anchor
+  cannot wrap a `<tr>`), honouring Ctrl/Cmd/Shift-click and ignoring clicks on interactive
+  descendants, with the accepted trade-off that screen readers announce labelled rows rather than
+  links (`role="link"` would stop the element being a row); and issue-detail header order changed to
+  Issue / Environment / Type / Urgency. Note **`library` and `tools` must both deploy** for any of
+  this to be visible. (jcardinal)
 - 2026-08-04 — Console build-out: the listing gained real pagination (25/page with a filtered
   `COUNT`, replacing a silent `LIMIT 200`), an EB Environment column **and filter**, a
   BUSINESS/TECHNICAL Type column derived from `issueKey` (no new DB column), and a 3-column filter

package/knowledge/2.0/apps/_underscore/features/error-reporting-issue-event.md

@@ -190,15 +190,47 @@ A uniform shape is what lets a viewer render context **generically** (see the To
 card/JSON-tree renderer). Consumers must read **both** the old and new key positions during the
 7-day `Event` retention overlap.
 
-### The quotable reference
+### The quotable reference — base-32 over an unambiguous alphabet (reworked 2026-08-04)
 
-`Issues.reference` is a Sentry-style base-26 encoding of `Issues.id`: 1 → `1A`, 26 → `1Z`,
-27 → `2A`. Because `AUTO_INCREMENT` never reuses ids, gaps are expected and permanent — a
-reference can never be recycled onto a different problem, so it is safe to quote to a customer.
+`Issue.reference` encodes `Issue.id` in **base 32** over
+`REFERENCE_ALPHABET = '0123456789ACDEFGHJKMNPQRSTUVWXYZ'`, most-significant character first, no
+padding (`_Model_Core_Logs_Issue::encodeReference()`). **All ten digits are kept; `O`, `I`, `L`
+and `B` are removed** because in the contexts a reference actually gets used — read aloud on a
+call, typed from a screenshot, pasted into a ClickUp title — they impersonate `0`, `1`, `1` and
+`8`. A reference exists to be quoted, so ambiguity is the one defect it cannot have.
+
+It replaced a decimal-counter + letter, base-26 scheme (`1A`, `2Q`, `3O`).
+
+> **⚠ `REFERENCE_OFFSET = 128` exists to stop the new scheme from OVERWRITING the old one —
+> do not remove it.** `Issue.reference` is **UNIQUE**, and every legacy value begins `1`–`3`.
+> With no offset, new id 75 encodes to `2A`, which id 27 already holds — the INSERT fails and the
+> error is **lost instead of recorded**, precisely the failure mode this pipeline exists to
+> prevent. The offset burns the first 128 (= 4 × 32) code points so every new reference starts at
+> `4` or later and both generations coexist forever. Verified by script over 200,000 ids: no
+> collisions among themselves, none against the legacy values, no banned characters.
+
+The column is `varchar(16)`, so the growth to 3 characters (≈ id 897) and 4 (≈ id 29,000) cannot
+truncate. Because `AUTO_INCREMENT` never reuses ids, gaps are expected and permanent — a reference
+can never be recycled onto a different problem.
+
+**The 1.0 encoder in `library/app/error/capture.php` must stay byte-identical** — both write the
+same UNIQUE column in the same shared table. Change one, change the other, deploy together.
 
 The reference is written by PHP in a second statement (see MySQL constraints below), which is
 why the column is nullable even though it is always populated.
 
+#### Existing data is NOT being truncated (decided 2026-08-04)
+
+Nothing needs deleting to adopt the new scheme. References are **stored, never re-derived from
+the id at read time**: the listing search, the Tools Move-fingerprint prompt (which accepts a
+reference *or* a numeric id) and every display path read the stored string, so both generations
+work side by side. Accepted cost: 10 of the 68 existing issues keep ambiguous references (`1B`,
+`1I`, `2B`, `2I`, `2L`, `2O`, `3B`, `3I`, `3L`, `3O`) until they age out through the normal GC.
+
+> **If you ever do truncate `Logs.Issue`:** `TRUNCATE` **resets `AUTO_INCREMENT`**, and
+> `IssueFingerprint`, `IssueEmailAddress`, `IssueClickupTask` and `Event` all reference
+> `Issue.id` — they must be cleared in the **same** operation or they orphan onto reused ids.
+
 ### Shutdown handler
 
 `register_shutdown_function` is registered with a **pre-allocated memory reserve**. Before this,
@@ -249,6 +281,13 @@ DB, so the pipeline is no longer 2.0-only; see
 has to opt in by adding a `[database_toga2logs]` config section, so Sentry must remain for any 1.0
 app that has not been wired up yet.
 
+## Deploy order for a change that spans this pipeline
+
+`_underscore` **first** (api2 and worker2 clone it at prebuild), then api2, worker2, `library`,
+`worker`, `tools`. Anything touching the reference encoder or a shared `Logs` table name must land
+in `_underscore` and `library` in the **same** release, and the `/errors` console work needs
+**both** `library` and `tools` deployed before it is visible.
+
 ## Gotchas / known issues
 
 - **Read-your-writes: reads go to the READ host and cannot see an uncommitted write.** A
@@ -379,6 +418,16 @@ clientUserId). **Neither was built.** As built instead:
 
 ## Change history
 
+- 2026-08-04 (later, **uncommitted/undeployed** at time of writing) — **Reference encoding reworked
+  to base 32** over `'0123456789ACDEFGHJKMNPQRSTUVWXYZ'` (all digits kept; `O`/`I`/`L`/`B` dropped
+  as impersonators of `0`/`1`/`1`/`8`), replacing the base-26 decimal-counter+letter scheme, with
+  **`REFERENCE_OFFSET = 128`** so new values start at `4` and cannot collide with the legacy `1`–`3`
+  values on the UNIQUE column — a collision would fail the INSERT and *lose* the error. Verified
+  over 200,000 ids. The 1.0 encoder in `library/app/error/capture.php` mirrors it byte-for-byte.
+  **Decided: existing Issue/Event data is not truncated** — references are stored and never
+  re-derived, so both generations coexist; 10 of 68 issues keep ambiguous references until GC.
+  Recorded the TRUNCATE warning (it resets `AUTO_INCREMENT` and four child tables reference
+  `Issue.id`). Next production reference will be `68` (max id 72). (jcardinal)
 - 2026-08-04 — **Decided: `_Exception_Business` extends `_Exception_Validation`** so api2 answers
   4xx for a business condition while still recording it — with the ordering trap that every
   `catch (_Exception_Validation …)` must now test business **first** (api2's controller and the

package/knowledge/2.0/apps/api2/features/api-payload-interceptors.md

@@ -6,7 +6,7 @@ project: API
 client: shared
 type: feature
 status: active
-updated: 2026-08-03
+updated: 2026-08-04
 owners: ["mhammontree", "dfranks"]
 files:
   - api2/Component/Api/V2/V2.php
@@ -82,6 +82,50 @@ tenant's other APIs at all. See
 > result above is authoritative; the code path is misleading. Anyone tempted to reason it out
 > from V2.php will reach the wrong answer.
 
+## The dispatch is UNGUARDED — a row naming a method that does not exist hard-fatals the endpoint
+
+The derived name (step 2 above) is **pure convention with nothing validating it**, and there is
+**no `phpMethod` column in production** to override it. All **four** call sites in
+`api2/Component/Api/V2/V2.php` invoke the derived name **without a `method_exists()` check**:
+
+| Site | Phase |
+|------|-------|
+| `~L8655` | builds the interceptor lookup |
+| `~L3148` | pre-processing dispatch |
+| `~L5694` | post-processing dispatch |
+| `~L7411` / `~L7437` | alternate pre-processing path |
+
+So a **single `ApiPayloadInterceptors` row whose (recordId, phase, method) resolves to a method the
+model does not define takes the endpoint down** with a PHP fatal —
+`Call to undefined method _Model_<Slug>_<Model>::postPost()` — not a handled 4xx/5xx. This is the
+mirror image of the "missing row" failure mode above: a *missing* row is silent, an *extra/wrong*
+row is fatal.
+
+**Fix direction:** guard all four sites with `method_exists()` and raise a logged `Logs.Issue`
+instead of letting the request fatal.
+
+### The client-model resolution that decides which class must define the method
+
+At `~L5681-5690` the engine derives the concrete class by `str_replace`-ing `_Model_Client_` →
+`_Model_<jwt client slug>_` on `$record->model`, guarded by `class_exists`. For Compass USA that
+yields `_Model_Compass_Usa_*`.
+
+The chain is `_Model_Compass_Usa_X extends _Model_Compass_X extends _Model_Client_X`, so **a method
+defined on the shared Compass base is inherited by both the Usa and Canada subclasses** — a
+per-region subclass does *not* need its own copy. Do not add duplicate hooks per region.
+
+### `Core.Records` facts for interceptor debugging
+
+- **`Core.Records.model` is UNIQUE** — exactly one record row per model class, so an interceptor's
+  `recordId` maps to precisely one model. No ambiguity to resolve.
+- The column is **`route`** (not `routePlural` / `routeSingular`) in the current schema. Example:
+  `recordId 15` = route `sales-order-items` = `_Model_Client_SalesOrderItem`.
+- **Diagnostic:** join the client's active interceptor rows to `Core.Records` and check each derived
+  method actually exists in `_underscore/Model/<Slug>/`. Compass's legitimate rows (records **14**
+  sales-orders, **17** purchase-orders, **19** vendor-items, **21** items, **37** contacts, **55**
+  advance-shipping-notices, **179** approval-decisions) all map 1:1 to real methods — **the row whose
+  record has no corresponding method is the anomaly.**
+
 ## Worked example — the EV-10 that was not a code bug
 
 `_Model_Client_ItemFulfillment::prePost()` defaults `itemFulfillmentStageId` to the shipped stage on
@@ -99,6 +143,14 @@ and the failing environment**. It is a small table, and the drift is usually exa
 - **⚠ A hook that "isn't running" is a missing registration row before it is a code bug.** Check
   `ApiPayloadInterceptors` for `(recordId, phase, method)` **first** — never start by re-reading the
   PHP.
+- **⚠ An interceptor row for a method that does not exist is a production outage, not a no-op.**
+  The dispatch is unguarded at all four call sites — the endpoint fatals with *"Call to undefined
+  method"*. If a write endpoint suddenly 500s for one client only, list that client's
+  `ApiPayloadInterceptors` rows and confirm every derived method exists.
+- **⚠ A migration that enables an interceptor must insert `isActive = 0`.** Activation is a *data*
+  change that switches on a *code* path; if the PHP defining the method is not confirmed deployed to
+  that environment, the row takes the endpoint down. Insert inactive, verify the deploy, then flip
+  `isActive = 1` — and remember every environment activates independently.
 - **The method name is derived, so a typo'd enum silently misses.** A row with
   `prePostProcessing = 'PRE'`, `httpMethod = 'PUT'` resolves to `prePut`, not `prePost`; the engine
   will simply find no method and move on.
@@ -114,6 +166,17 @@ and the failing environment**. It is a small table, and the drift is usually exa
 
 ## Change history
 
+- 2026-08-04 — Documented the **inverse failure mode**: the derived-name dispatch is **unguarded at
+  all four `V2.php` call sites** (`~L8655` lookup, `~L3148` pre, `~L5694` post, `~L7411`/`~L7437`
+  alternate pre) and there is **no `phpMethod` column in production**, so one row naming a
+  nonexistent method hard-fatals the endpoint (*"Call to undefined method
+  `_Model_Compass_Usa_SalesOrderItem::postPost()`"*). Added the client-model resolution at
+  `~L5681-5690` (`_Model_Client_` → `_Model_<slug>_`, `class_exists`-guarded) and the
+  `Usa → Compass → Client` inheritance rule (define once on the Compass base; no per-region copy),
+  the `Core.Records` debugging facts (`model` is UNIQUE, column is `route`, record 15 =
+  `sales-order-items`, Compass's 1:1 legitimate row set), and the rule that a migration enabling an
+  interceptor must land `isActive = 0` first. Diagnosed from a production incident; no code change.
+  (dfranks)
 - 2026-08-03 — Added two verified behaviours from a live dev probe: (1) interceptor registration
   is **per-API** (`apiId` on the row, in both the Core and `Client_<X>` tables), so a guard in a
   hook is scoped by **DB config rather than code** and a new row widens its blast radius; (2) the

package/knowledge/2.0/apps/api2/features/request-logging.md

@@ -6,7 +6,7 @@ project: API
 client: shared
 type: feature
 status: active
-updated: 2026-08-03
+updated: 2026-08-04
 owners: ["mhammontree", "dfranks"]
 files:
   - api2/Component/Api/V2/V2.php
@@ -15,7 +15,9 @@ files:
   - _underscore/Model/Core/Logs/Api.php
 related:
   - ../architecture.md
+  - ./api-payload-interceptors.md
   - ../../../../clients/aig/features/entitlement-intake.md
+  - ../../../../clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md
 ---
 
 ## Summary
@@ -61,6 +63,28 @@ That is the deciding factor when choosing between `_Exception_Validation` (400,
 request, no Issue) and `_Exception_Business` (500, Sentry + a routed Issue). Do **not** reach for
 `_Exception_Business` merely to "make sure the rejection is recorded" — it already is.
 
+### ⚠ But an uncaught PHP **fatal** can leave NO log row anywhere
+
+The commit-the-logs / roll-back-the-data contract above holds for the **handled** paths — a
+validation rejection or an exception the controller catches. It did **not** hold for an **uncaught
+`Error`** observed in production on 2026-08-04 (a `Call to undefined method` raised from an
+[API payload interceptor](api-payload-interceptors.md) dispatch): the fatal was reported to the
+error tracker, but **`Logs_Compass.Api` contained zero 5xx rows for the entire day and the failing
+request appears nowhere in it at all.** Suspected regression in the recent `Error.php` /
+Issue-Event table normalization work.
+
+Practical consequences:
+
+- **Do not conclude "the request never happened" from an empty log table.** For a fatal, absence of
+  a log row is not evidence of absence of the request. Check the error tracker's per-event detail —
+  that may be the *only* record of the individual request.
+- **Diagnose from the DATA, not the log.** The data transaction rolls back while a *previous*
+  successful request already committed its own rows, so a fatal on a multi-request write sequence
+  leaves a detectable **orphan / zero-child parent**. Sweep for those instead of grepping logs — see
+  [Compass Order Lifecycle & Data-Integrity Invariants](../../../clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md#detecting-empty-shell-orders-left-by-a-fatal-mid-sequence).
+- **Orphan sweeps only find PARTIAL casualties.** An attempt that rolled back completely leaves no
+  row in any table and, with no API log row, no trail at all.
+
 ## Blind spots when auditing a client's API traffic
 
 Looking only at `Logs_<Client>.Api` misses four sources:
@@ -110,6 +134,12 @@ re-send.
 
 ## Change history
 
+- 2026-08-04 — Added the limit of the commit-logs contract: an **uncaught PHP fatal** (a
+  `Call to undefined method` from an interceptor dispatch) produced **zero rows in
+  `Logs_Compass.Api`** — no 5xx row for the whole day and no row for the failing request — while
+  still reaching the error tracker. Suspected regression in the `Error.php` / Issue-Event
+  normalization work. So log-based diagnosis fails for this class of 500; sweep the **data** for
+  zero-child parents instead, and accept that fully-rolled-back attempts leave no trail. (dfranks)
 - 2026-08-03 — Corrected a common wrong assumption: an **HTTP 400 leaves a full log row**.
   `Controller/Index.php` L369-377 commits `DB_LOGS`/`DB_CLIENT_LOGS` and rolls back only the
   business transaction, so rejections are queryable and attributable to the credential; what they

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -18,7 +18,7 @@
 | [Compass VIP Support Importer (worker2)](features/compass-vip-support-importer.md) | A worker2 action that ingests Compass's quarterly VIP spreadsheet and assigns each VIP user's support technician by setting `Users.c_supportedByUserId` in `Clie | worker2/Worker/Client/Compass/VipSupport.php |
 | [Creating Worker Actions](features/creating-worker-actions.md) | How to add a new callable Worker action — a PHP class whose `public static` methods are invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). | worker2/Worker/, worker2/Controller/Index.php, _underscore/Worker.php |
 | [Elite Freshservice Sync (worker2)](features/elite-freshservice-sync.md) | `_Worker_Elite` processes Freshservice webhook events and syncs them into TOGA 2. | worker2/Worker/Elite.php, worker2/Config/dev-kmaramreddy-laptop.ini |
-| [Error Escalation Cron (Errors::Escalate → ClickUp / email)](features/error-escalation-cron.md) | `_Worker_Infrastructure_Errors::Escalate` (renamed from `SyncWithClickup`) is the sole owner of **escalation, de-escalation, ClickUp ticketing, reminders, busin | worker2/Worker/Infrastructure/Errors.php, worker2/Worker/Clickup/ErrorTask.php, worker2/Worker/Clickup.php, worker2/Controller/Index.php, worker2/Config/production.ini, _underscore/Model/Core/Logs/Issue.php, dbchanges2/Core/2026-07-30a - Error escalation cron job.sql, dbchanges2/Logs/2026-08-03a - Issue clickupPriority.sql |
+| [Error Escalation Cron (Errors::Escalate → ClickUp / email)](features/error-escalation-cron.md) | `_Worker_Infrastructure_Errors::Escalate` (renamed from `SyncWithClickup`) is the sole owner of **escalation, de-escalation, ClickUp ticketing, reminders, busin | worker2/Worker/Infrastructure/Errors.php, worker2/Worker/Notification/Email.php, worker2/Worker/Notification/EmailTemplate.php, worker2/Worker/Client/True.php, worker2/Worker/Clickup/ErrorTask.php, worker2/Worker/Clickup.php, worker2/Controller/Index.php, worker2/Config/production.ini, _underscore/Model/Core/Logs/Issue.php, dbchanges2/Core/2026-07-30a - Error escalation cron job.sql, dbchanges2/Logs/2026-08-03a - Issue clickupPriority.sql |
 | [Etilize Catalog Item Import & Refresh](features/etilize-catalog-item-import.md) | Client-generic catalog onboarding from an S3 CSV plus an Etilize re-pull. | worker2/Worker/Etilize/Items.php |
 | [Etilize Item Translation Import](features/etilize-item-translation-import.md) | The abstract worker class `_Worker_Etilize_ItemTranslations` imports **non-English** item text from Etilize into the client's `ItemTranslations` table. | worker2/Worker/Etilize/ItemTranslations.php |
 | [Monitoring Framework (Orchestrator + Child Monitors)](features/monitoring-framework.md) | A unified, DB-driven monitoring framework for business-critical data flows (Compass POs, Prudential asset imports, AIG closed claims, …). | worker2/Worker/Monitor.php, worker2/Worker/Monitors/, worker2/Worker/Monitors/RateEntitlement.php, worker2/Worker/Notification/Email.php, worker2/Worker/Rate.php, dbchanges2/Core/2026-05-21 - Monitors.sql, dbchanges2/Core/2026-06-29a - Rate Entitlement Contract Monitor.sql |

package/knowledge/2.0/apps/worker2/features/error-escalation-cron.md

@@ -10,6 +10,9 @@ updated: 2026-08-04
 owners: ["jcardinal"]
 files:
   - worker2/Worker/Infrastructure/Errors.php
+  - worker2/Worker/Notification/Email.php
+  - worker2/Worker/Notification/EmailTemplate.php
+  - worker2/Worker/Client/True.php
   - worker2/Worker/Clickup/ErrorTask.php
   - worker2/Worker/Clickup.php
   - worker2/Controller/Index.php
@@ -23,6 +26,8 @@ related:
   - ../../../1.0/apps/library/features/error-capture-1-0.md
   - ./creating-worker-actions.md
   - ./clickup-project-routing.md
+  - ./notification-email.md
+  - ./notification-email-template.md
 ---
 
 ## Summary
@@ -107,10 +112,64 @@ which is safe only because the automation never sets status itself.
 > stamping `dtAcknowledged` ~1s after `dtCreated` and permanently disabling the neglect axis for
 > that issue. Status webhooks arriving within 60s of creation are ignored.
 
+### ⚠ Escalation email had never been sent once — two independent causes (fixed 2026-08-04)
+
+Evidence: `Logs_True.Email` contained **zero** error-escalation rows *ever*, while unrelated
+worker2 mail (merge-conflict notices, sprint reports) delivered within ~60s — so the mail pipeline
+itself was healthy. Every `Infrastructure/Errors/Escalate` run reported `"emailsSent":0` alongside
+`"failures":0` and therefore **read as healthy**. All 48 HIGH/URGENT issues sat at
+`notificationCount` 4 — the last rung of the ladder — having sent nothing.
+
+**Root cause A (structural — the important one).** `processTechnicalIssue()` returned early when
+the ClickUp task would read identically (priority already correct **and** count unchanged), and
+**that `return` sat before the `if ($shouldNotify)` block.** `persistIssueState()` still ran on the
+way out with `didNotify: $shouldNotify`, stamping `dtLastNotified` and incrementing
+`notificationCount`. So a run that sent nothing still **consumed a rung** of the
+30/60/240/1440-minute `REMINDER_INTERVAL_MINUTES` ladder — and since a live issue normally sits at
+a settled priority with an unchanged count, that was **nearly every run**.
+
+Fix: the ClickUp write is now skipped on its own merits, inside
+`if ($needsPriorityWrite || $hasCountChanged)`, and the notification decision is made
+**independently, below it**.
+
+> **Lesson worth generalizing: "does ClickUp need writing?" and "does anyone need telling?" are two
+> separate questions.** Conflating them into one early return silenced the entire reminder ladder
+> while every metric reported success.
+
+**Root cause B.** Both send paths built a `_Email` and called `->send()` **inline inside the cron's
+open `DB_LOGS` transaction**. `_Email::send()` resolves the client's log database, registers a
+second connection and toggles the read host from in there; anything it threw was caught into
+`error_log` only — invisible in the console *and* in the run summary. Both paths now **enqueue via
+`_Worker::runTask`**, so a failure is a visible failed `WorkerJobs` row.
+
+### `emailFailures` in the run summary
+
+`Escalate()`'s JSON gained **`emailFailures`**, threaded through `processIssue()` /
+`processTechnicalIssue()`'s outcome arrays and counting notifications that were *meant* to go out
+and did not. It is reported **separately from `failures`** (which counts issues that threw). It
+exists specifically so the failure mode above — `emailsSent:0` with `failures:0` reading as
+healthy — cannot recur silently.
+
 ### Notification and task presentation
 
 - The technical escalation email to **devteam@togatech.com** now **coexists** with the ClickUp
   update. It was an `elseif`, so only one of the two ever fired.
+- **The technical escalation email uses the TOGA Technology stored email template.**
+  `sendTechnicalEscalationEmail()` enqueues **`Notification/EmailTemplate/Send`** with
+  `clientIdentifier` `'True'` and
+  `_Worker_Client_True::EMAIL_TEMPLATE_UUID__TOGA_TECHNOLOGY`
+  (`232d4edb-c2fa-4a8b-b5b9-d5800c962e19`) — the same pattern `Team/Transcripts.php` uses. Verified
+  against `Client_True.EmailTemplates`: `isActive = 1`; `{subject}` appears in **both** the subject
+  line and the branded header banner; `{body}` sits inside a **600px-wide** content cell that
+  already sets Plus Jakarta Sans 16px; the row carries its own `sendFromEmailAddress`
+  (`noreply@togatech.com`) and a single priority column (`NORMAL`).
+  - **`buildTechnicalEmailBody()` is therefore a FRAGMENT, not a document.** Its wrapper was
+    changed from `max-width:680px` + its own `font-family` to `max-width:100%` and no font —
+    680px would overflow the template's 600px cell.
+  - **Per-send priority is not available on the template path**, so urgency rides in the subject
+    and body instead.
+- **Business-issue emails moved to `Notification/Email/Send`**, which *does* keep per-client log
+  attribution and per-send priority.
 - ClickUp titles carry the **reference only** — `[1E]`, not `[1E-1]`. Including the occurrence
   number made the title churn on every escalation.
 - **Occurrences** and **Last Occurrence** custom fields are synced, and task bodies open with a
@@ -172,6 +231,13 @@ this controller needs the same treatment.**
 
 ## Gotchas / known issues
 
+- **Never send email inline inside this cron's open transaction — enqueue it.** `_Email::send()`
+  registers a second connection and toggles the read host, and a throw from in there was only ever
+  `error_log`'d. Use `_Worker::runTask` so a failure becomes a visible failed `WorkerJobs` row.
+- **A "nothing changed, return early" shortcut must not sit above the notification block.** It
+  consumed reminder-ladder rungs without sending anything, for months, while reporting
+  `emailsSent:0 / failures:0` as success. Decide the outbound write and the notification
+  independently.
 - **Persist state before, or independently of, any outbound call.** A regression chain took the
   whole run down every minute: `syncCustomFields()` read `$issue->dtLastOccurred`, which
   `loadWorkingSet()` never selected; `_Error::errorHandler()` promotes the undefined-property
@@ -207,6 +273,25 @@ this controller needs the same treatment.**
 
 ## Change history
 
+- 2026-08-04 (later, **uncommitted/undeployed** at time of writing) — **Fixed: escalation email had
+  never been sent, ever**, for two independent reasons. (A) `processTechnicalIssue()` returned early
+  when the ClickUp task would read identically, and that return sat *above* the `if ($shouldNotify)`
+  block while `persistIssueState()` still stamped `dtLastNotified`/`notificationCount` on the way
+  out — so nearly every run silently consumed a rung of the 30/60/240/1440-minute ladder; all 48
+  HIGH/URGENT issues had reached rung 4 having sent zero mail. The ClickUp write is now gated on its
+  own merits (`$needsPriorityWrite || $hasCountChanged`) and the notification decision is made
+  independently below it. (B) Both send paths built `_Email` and called `->send()` **inline inside
+  the open `DB_LOGS` transaction**, where a throw was only `error_log`'d — both now enqueue via
+  `_Worker::runTask`. **Built:** the technical escalation email now goes through the stored TOGA
+  Technology template (`Notification/EmailTemplate/Send`, clientIdentifier `True`,
+  `_Worker_Client_True::EMAIL_TEMPLATE_UUID__TOGA_TECHNOLOGY`), which makes
+  `buildTechnicalEmailBody()` a fragment (`max-width:100%`, no font — 680px would overflow the
+  template's 600px cell) and removes per-send priority (urgency moves into the subject/body);
+  business emails moved to `Notification/Email/Send`, which keeps per-client log attribution and
+  priority. **Built:** an **`emailFailures`** counter in `Escalate()`'s run summary, separate from
+  `failures`, so `emailsSent:0 / failures:0` can never read as healthy again. Verified by reading
+  only — proof will be `emailsSent` going non-zero (or `emailFailures` naming the problem) on the
+  first Escalate run after the worker2 deploy. (jcardinal)
 - 2026-08-04 — **Fixed: worker job failures were never captured at all** — worker2's front
   controller catches `Throwable` and returns 200, so six catch sites in
   `worker2/Controller/Index.php` now call `_Error::captureException()` after rollback and before the

package/knowledge/2.0/apps/worker2/features/notification-email.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-06-24
-owners: ["mhammontree"]
+updated: 2026-08-04
+owners: ["mhammontree", "jcardinal"]
 files:
   - worker2/Worker/Notification/Email.php
   - _underscore/Model/Client/EmailTemplate.php
@@ -39,7 +39,7 @@ migration backfilled first).
 - `worker2/Worker/Notification/Email.php` — `abstract _Worker_Notification_Email`. One method:
   `Send(string $clientIdentifier, string $subject, string $body, string|array $to=[],
   string|array $cc=[], string|array $bcc=[], string $fromEmail='donotreply@togatech.com',
-  string $fromName='TOGA Technology')`. The entry point. Self-registers the client DB (no
+  string $fromName='TOGA Technology', ?int $priority=null)`. The entry point. Self-registers the client DB (no
   `initialize()` — see below), wraps the body, and sends.
 - `_underscore/Model/Client/EmailTemplate.php` — supplies `WRAPPER_UUID` and
   `renderWrappedBody($subject, $body)`. Documented in
@@ -69,6 +69,20 @@ migration backfilled first).
 5. **Send** via a plain `_Email`: `setClientIdentifier`, `addTo/Cc/Bcc`, `setFrom`,
    `setSubject`, `setBody($wrapped ?? $raw)`, `send()`.
 
+### Optional per-send priority (added 2026-08-04)
+
+The trailing **`?int $priority = null`** maps to `_Email::setPriority()`, which emits the
+`X-Priority` / `Importance` / `X-MSMail-Priority` headers. Omit it and nothing changes — the
+dispatcher spreads **named** parameters, so a new trailing optional parameter is backward
+compatible for every existing caller and queued job.
+
+It was added so that moving the error pipeline's **business** escalation email off an inline
+`_Email` and onto this worker did not silently drop the urgency headers (see
+[error-escalation-cron](./error-escalation-cron.md)). Note the sibling
+[`Notification/EmailTemplate/Send`](./notification-email-template.md) path has **no** per-send
+priority — a stored template carries its own single priority column — so urgency there has to live
+in the subject and body.
+
 ### Enqueuing it
 
 `_Worker::runTask('Notification/Email/Send', ['clientIdentifier'=>…, 'subject'=>…, 'body'=>…, …])`
@@ -130,6 +144,11 @@ working recipe — verified in classic + new Outlook:
   are replaced simultaneously, so a `{body}` literal inside the subject can't be re-expanded.
 
 ## Change history
+- 2026-08-04 (**uncommitted/undeployed** at time of writing) — Added an optional trailing
+  `?int $priority = null` to `Send()`, mapping to `_Email::setPriority()`
+  (`X-Priority`/`Importance`/`X-MSMail-Priority`). Backward compatible because the dispatcher
+  spreads named parameters. Added so the error pipeline's business escalation email kept its urgency
+  headers when it moved off an inline `_Email` onto this worker. (jcardinal)
 - 2026-06-24 — Built the DB-driven notification-email mechanism (TRUE-79240): new
   `_Worker_Notification_Email::Send` entry point; branded shell moved out of code (deleted
   `_underscore/Email/Template.php`, which also carried the 1.0 double-render bug) into a reserved

package/knowledge/INDEX.md

@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 1.0 framework
 
 - **library** (Library) _(framework core)_ — 15 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
-- **worker** (Worker) — 19 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
+- **worker** (Worker) — 20 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 - **dbchanges** (Database Changes) _(framework core)_ — 1 doc(s) → [1.0/apps/dbchanges/INDEX.md](1.0/apps/dbchanges/INDEX.md)
 - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
 - **togadesk** (TOGa Desk) — 11 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)

package/knowledge/clients/compass-usa/INDEX.md

@@ -12,5 +12,6 @@
 | [Compass USA](profile.md) | 2.0 | Compass USA is a TOGA client running a multi-tier supply-chain commerce operation. |  |
 | [Compass Cross-Kit Bundle Corruption — Detection & Repair](workflows/cross-kit-bundle-corruption.md) | 2.0 | A frontend regression in `toga2-commerce`'s edit-order bundle submission mis-attributed bundle (kit) line items and **fees/warranties** to the **wrong kit**, pe | src/api/syncSalesOrderItemsFromLocalStorageCartToApi.ts |
 | [Compass Office Depot Duplicate PO-Line Cleanup (dual-catalog SKU)](workflows/odp-duplicate-po-line-cleanup.md) | 1.0 | The NetSuite fulfillment-sales-order importer duplicated Office Depot purchase-order lines on Compass USA because it reconciled the fulfillment SO against the e | library/app/api/toga2.php, dbchanges2/Client_Compass/2026-07-22a - CleanupOfficeDepotDuplicatePurchaseOrderItems.sql |
-| [Compass ODP Order Pipeline to NetSuite (numbered worker crons)](workflows/odp-order-pipeline-to-netsuite.md) | 1.0 | The end-to-end **Compass Office Depot (ODP) order → NetSuite** pipeline as it actually runs through the 1.0 `worker` crons under `worker/crons/toga2/compass/`, | worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php, worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, worker/crons/toga2/compass/edi/1_download_edi_s3_create_po_toga.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php, worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, library/app/client/compass.php |
+| [Recovering a Lost Compass ODP EDI 850 Import (re-drop from Logs.FileLog)](workflows/odp-edi-import-recovery.md) | 1.0 | How to recover a Compass **Office Depot EDI 850** import that failed partway — the case where cron **3a** created the ODP SalesOrder header, the follow-up item | worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php, worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, worker/schedules/cron.worker.sync.json |
+| [Compass ODP Order Pipeline to NetSuite (numbered worker crons)](workflows/odp-order-pipeline-to-netsuite.md) | 1.0 | The end-to-end **Compass Office Depot (ODP) order → NetSuite** pipeline as it actually runs through the 1.0 `worker` crons under `worker/crons/toga2/compass/`, | worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php, worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, worker/crons/toga2/compass/edi/1_download_edi_s3_create_po_toga.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php, worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, worker/schedules/cron.worker.sync.json, library/app/client/compass.php |
 | [Compass Order Lifecycle & Data-Integrity Invariants](workflows/order-lifecycle-and-data-integrity.md) | 2.0 | End-to-end map of how a Compass order flows through the `Client_Compass` (2.0) database and the **expected raw-data shape** at each link/ASN/IF level. |  |

package/knowledge/clients/compass-usa/workflows/odp-edi-import-recovery.md

@@ -0,0 +1,111 @@
+---
+title: Recovering a Lost Compass ODP EDI 850 Import (re-drop from Logs.FileLog)
+framework: "1.0"
+repo: worker
+project: Worker
+client: compass-usa
+type: workflow
+status: active
+updated: 2026-08-04
+owners: ["dfranks"]
+files:
+  - worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php
+  - worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php
+  - worker/schedules/cron.worker.sync.json
+related:
+  - clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md
+  - clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md
+  - ../../../2.0/apps/api2/features/api-payload-interceptors.md
+  - ../../../2.0/apps/api2/features/request-logging.md
+  - clients/compass-usa/profile.md
+---
+
+## Summary
+
+How to recover a Compass **Office Depot EDI 850** import that failed partway — the case where cron
+**3a** created the ODP SalesOrder header, the follow-up item write died (e.g. an api2 fatal), and 3a
+then **deleted the S3 object**, leaving a zero-item order and nothing to retry.
+
+The recovery works because the raw x12 is archived in `Logs.FileLog` and **3a is fully idempotent**:
+re-uploading the file makes it repair the items, the ODP PO, and both bridges. Verified end-to-end
+in **production** on 2026-08-04 for ODP PO `41608261-1135` / ODP SO `475390815001`.
+
+## When to use this
+
+Symptoms: a zero-item ODP SalesOrder (see the orphan sweeps in
+[order lifecycle & data integrity](order-lifecycle-and-data-integrity.md)), and cron 5 re-attempting
+the NetSuite SO every 10 minutes because it selects on `c_dtTransmittedToNetsuite IS NULL`.
+
+## The runbook
+
+### Step 1 — retrieve the archived x12
+
+The raw EDI is on the **legacy-core cluster**, table **`Logs.FileLog`**:
+
+- `job = 'ODP_EDI'`
+- `fileName = '<odpPoNumber>.x12'` — note **`.x12`**, even though the inbound S3 filename is
+  `EDI_<odpPoNumber>.txt`.
+
+**Query it bounded by `job` + `fileTimestamp`.** An unbounded `fileName LIKE` scan times out on this
+table.
+
+Extract with `mysql --raw --batch --skip-column-names`, then **strip the trailing newline the client
+appends** so the byte count matches the stored `LENGTH(fileData)` exactly. A byte-count mismatch means
+you have a corrupted payload — do not upload it.
+
+### Step 2 — know that S3 has nothing left
+
+3a **deletes the S3 object after processing** (`:769`), so a failed import leaves no file to retry.
+Recovery is a **re-upload**, to:
+
+```
+s3://agilant-as2/OfficeDepot/EDI_<odpPoNumber>.txt
+```
+
+### Step 3 — clear the cause BEFORE re-uploading
+
+**Confirm no `ApiPayloadInterceptors` row exists for the `sales-order-items` record** (`recordId 15`)
+whose derived method the model does not define. If one is still active, the item write fatals again
+**and the file is deleted a second time**. See
+[API payload interceptors](../../../2.0/apps/api2/features/api-payload-interceptors.md).
+
+### Step 4 — let 3a repair it (≤ 5 minutes)
+
+3a runs `*/5`. Because every step is existence-guarded it **reuses** the existing SO and **backfills**
+what is missing: unmatched items (matched case-insensitively by `partNumber`), the ODP PO if absent,
+and both the `PurchaseOrders_SalesOrders` and `PurchaseOrderItems_SalesOrderItems` bridges. No
+duplicates are created.
+
+**Then set `customerPurchaseOrder` manually** — 3a never updates the SalesOrder *header* on an
+existing order, so it stays `NULL`. Prefer direct SQL over a `PUT`; the reasoning and its two
+tradeoffs are in the
+[lifecycle doc](order-lifecycle-and-data-integrity.md#repairing-a-header-only-scalar-field-direct-sql-vs-put).
+
+### Step 5 — let cron 5 transmit to NetSuite
+
+Cron 5 (`*/10`) picks the now-complete ODP SO up and stamps `c_dtTransmittedToNetsuite` +
+`c_netsuiteInternalSalesOrderId`. It excludes `number LIKE 'MA%'`. Its stamps landing is the
+confirmation the recovery finished.
+
+## Gotchas / known issues
+
+- **⚠ Re-uploading before fixing the root cause destroys the file again** (3a deletes on the way
+  through). Step 3 is not optional.
+- **⚠ The archive filename extension differs from the S3 one** — `.x12` in `Logs.FileLog`,
+  `EDI_<po>.txt` in S3. Searching for the wrong one finds nothing.
+- **Don't search for the ODP SalesOrder by the `BEG` number.** `BEG` is the ODP *PO*; the SO number
+  is `REF~QC`. Full segment map in the
+  [ODP pipeline doc](odp-order-pipeline-to-netsuite.md).
+- **`Logs.FileLog` has limited retention** (~30 days) — beyond that there is no archived x12 and the
+  recovery must come from ODP re-sending.
+- A **fully** rolled-back import leaves no row anywhere and no API log row, so it will not show up in
+  an orphan sweep at all; only the error tracker's per-event detail can enumerate those.
+
+## Change history
+
+- 2026-08-04 — Documented the runbook after recovering a production ODP order whose item write was
+  killed by an api2 fatal: `Logs.FileLog` (`job = 'ODP_EDI'`, `<po>.x12`, bounded by `fileTimestamp`)
+  as the x12 archive, the trailing-newline/byte-count check, the re-upload path to
+  `s3://agilant-as2/OfficeDepot/`, the mandatory "clear the interceptor first" step, 3a's idempotent
+  repair plus the `customerPurchaseOrder` header gap, and cron 5's `*/10` stamps as the completion
+  signal. (dfranks)

package/knowledge/clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md

@@ -6,16 +6,18 @@ project: Worker
 client: compass-usa
 type: workflow
 status: active
-updated: 2026-08-03
-owners: ["rgirish", "bala"]
+updated: 2026-08-04
+owners: ["rgirish", "bala", "dfranks"]
 files:
   - worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
   - worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php
   - worker/crons/toga2/compass/edi/1_download_edi_s3_create_po_toga.php
   - worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php
   - worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php
+  - worker/schedules/cron.worker.sync.json
   - library/app/client/compass.php
 related:
+  - clients/compass-usa/workflows/odp-edi-import-recovery.md
   - clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md
   - clients/compass-usa/features/mits-po-transmission-to-vendors.md
   - clients/compass-usa/features/mits-po-to-so-item-linking.md
@@ -40,7 +42,8 @@ Commerce → Compass SalesOrder            (SalesOrders.customerId = 2)
 MITS → PurchaseOrder                      (PurchaseOrders.vendorId = 1 = OFFICE DEPOT; POST /v2/purchase-orders; PO number e.g. 50305906-1)
   [cron 2_transmit_mits_purchase_orders_to_vendors] → ODP (cXML; email fallback per integration)   [sets PurchaseOrders.dtSubmitted]
 ODP → 850 EDI dropped to S3 (AS2)         (bucket agilant-as2, prefix OfficeDepot/)
-  [cron edi/1_download_edi_s3_create_po_toga]     → creates ODP SalesOrder (customerId = 1); deletes the S3 file after processing
+  [cron workflow/3a_import_office_depot_purchase_orders  — NOT edi/1, see correction below]
+                                                → creates ODP SalesOrder (customerId = 1); deletes the S3 file after processing
 ODP SalesOrder (customerId = 1)
   [cron 5_create_netsuite_sales_orders_from_office_depot_purchase_orders, hourly] → NetSuite SO
      writes back c_dtTransmittedToNetsuite + c_netsuiteInternalSalesOrderId ONTO the ODP SO (customerId=1)
@@ -56,12 +59,57 @@ ODP SalesOrder (customerId = 1)
    `PurchaseOrders.dtSubmitted` when transmitted (~2 min after PO creation). (See the dedicated
    [MITS PO Transmission to Vendors](../features/mits-po-transmission-to-vendors.md) feature for
    the item-less-PO gotcha and the CXML-vs-EMAIL routing detail.)
-3. **`edi/1_download_edi_s3_create_po_toga.php`** — pulls the ODP **850** from S3 bucket
-   `agilant-as2`, prefix `OfficeDepot/` (delivered via AS2), and creates a downstream **ODP
-   SalesOrder** (customerId=1 = Office Depot). **Deletes the file from S3 after processing** —
-   so an absent S3 object is expected once ingested, not evidence of a miss.
-4. **`workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php`** — runs
-   **hourly**; picks up ODP SalesOrders
+3. **`workflow/3a_import_office_depot_purchase_orders.php`** — pulls the ODP **850** from S3 bucket
+   `agilant-as2`, prefix `OfficeDepot/` (excluding `OUTBOX/` and `SENT/`; `:46` S3 read,
+   `:54-58` prefix filters) and creates the downstream **ODP SalesOrder** (customerId=1 = Office
+   Depot). **Deletes the S3 object after processing** (`:769`) — so an absent S3 object is expected
+   once ingested, not evidence of a miss, *and* a failed import leaves nothing to retry.
+
+   > **⚠ Correction (2026-08-04): `edi/1_download_edi_s3_create_po_toga.php` is DEAD CODE.** It
+   > appears in **no** schedule file. The scheduled entry *named* "Download EDI From S3 Files &
+   > Create PO - EDI Office Depot" (`*/5 * * * *`, `worker/schedules/cron.worker.sync.json`
+   > `:254-259`) actually points at **`toga2/compass/workflow/3a_import_office_depot_purchase_orders.php`**.
+   > The two scripts have **different idempotency and different API call shapes** — `edi/1`'s create
+   > branch POSTs the SO with *nested* items and a `customer.name`, while **3a POSTs the header alone
+   > (uuids) and then POSTs the items in a separate request**. Reasoning from `edi/1` gives wrong
+   > answers about what production does.
+   >
+   > **General rule: never infer which script runs from its filename or folder.** Verify against
+   > `worker/schedules/cron.*.json` — and note the schedule entry's **name can be actively
+   > misleading**.
+
+### 3a is idempotent — re-dropping an EDI file is safe
+
+Every step of 3a is gated by a `totalRecordCount == 0` existence check, so a re-drop produces **no
+duplicates and backfills whatever is missing**:
+
+| Step | Guard | Behavior on re-run |
+|------|-------|--------------------|
+| ODP SalesOrder | `:162-227` existence check | reuses the existing SO |
+| SO items | `:470-498` | matches existing items **case-insensitively by `partNumber`**; creates only unmatched ones |
+| ODP PurchaseOrder | `:356-436` | creates only if missing |
+| `PurchaseOrders_SalesOrders` + `PurchaseOrderItems_SalesOrderItems` bridges | `:632` | creates only if missing |
+
+**KNOWN GAP: 3a never updates the SalesOrder HEADER on an existing order.** So
+`customerPurchaseOrder` stays `NULL` on a repaired order and must be set manually. See
+[Recovering a lost ODP EDI import](odp-edi-import-recovery.md).
+
+### The EDI 850 → TOGa identifier mapping (not guessable — this is the key to tracing an ODP order)
+
+| 850 segment | Meaning | Where it lands |
+|-------------|---------|----------------|
+| **`BEG`** | the **Office Depot PO number** (e.g. `41608261-1135`) | `PurchaseOrders.number`, vendor "Agilant Solutions Inc" |
+| **`REF~QC`** | the **ODP SALES ORDER number** (e.g. `475390815001`) | `SalesOrders.number` with `customerId = 1` |
+| **`REF~EU` / `REF~PO`** | the **Compass MITS PO number** (e.g. `50310002-1`) | its `c_mitsSalesOrder` field holds the ODP SO number |
+| **`REF~LU`** | the **Compass sales order** (e.g. `SA135273`) | `SalesOrders` with `customerId = 2` |
+| **`PO1`** | the line | qty, UOM, unit price, vendor part, ODP item id |
+
+**⚠ `REF~QC` — not `BEG` — is the value to search on for the ODP SalesOrder.** Searching
+`PurchaseOrders` / `SalesOrders` for the `BEG` number finds nothing for the SO and wastes time.
+(This is the segment-level detail behind the "don't search by the MITS PO number" rule below.)
+4. **`workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php`** — scheduled
+   **every 10 minutes** (`*/10`, verified against `worker/schedules/` on 2026-08-04; this doc
+   previously said "hourly"); picks up ODP SalesOrders
    (`OfficeDepotSalesOrders.customerId = 1 AND c_dtTransmittedToNetsuite IS NULL`), creates the
    SO in NetSuite, and writes back `c_dtTransmittedToNetsuite` + `c_netsuiteInternalSalesOrderId`
    **onto the ODP SalesOrder (customerId=1), never the Compass SO (customerId=2)**.
@@ -70,6 +118,10 @@ ODP SalesOrder (customerId = 1)
    - **computer-kit orders** (Bundles 187–196) wait for a **2nd PO** before syncing.
    - a **30-minute delay** after the first ODP PO is created.
 
+   Because the selection is only `c_dtTransmittedToNetsuite IS NULL` (plus `number NOT LIKE 'MA%'`),
+   a **zero-item ODP SalesOrder sits in this queue retrying every 10 minutes** until its lines are
+   restored — the retry loop is the symptom, not the cause.
+
 ## Cron 5 NetSuite SO creation — failure modes & the two ODP error emails
 
 Two different Compass ODP error emails exist, from **two different pipeline stages**. They look
@@ -141,6 +193,15 @@ wrong record makes every order look "stuck." Use the **join**, never a number ma
   transmission feature.)
 
 ## Change history
+- 2026-08-04 — **Corrected the 850-import step**: `edi/1_download_edi_s3_create_po_toga.php` is
+  **dead code in no schedule**; the schedule entry named "Download EDI From S3 Files & Create PO -
+  EDI Office Depot" (`*/5`) actually runs `workflow/3a_import_office_depot_purchase_orders.php`,
+  which has different idempotency and posts the SO header and its items as **two separate API
+  requests**. Added 3a's full existence-check/idempotency map (re-dropping a file is safe; header is
+  never updated so `customerPurchaseOrder` stays NULL), the **EDI 850 → TOGa identifier mapping**
+  (`BEG` = ODP PO, `REF~QC` = ODP SO — the one to search on, `REF~EU`/`REF~PO` = MITS PO,
+  `REF~LU` = Compass SO), and corrected cron 5's schedule to `*/10`. Production incident
+  investigation; no code change. (dfranks)
 - 2026-08-03 — Documented cron 5's "Please enter a value for Ext Price" root cause (`rate` set
   only for NetSuite item groups, never plain items → NetSuite `add()` USER_ERROR, order re-emails
   hourly while stuck), the two distinct ODP error emails (3a PO-import vs 5 SO-creation) and cron

package/knowledge/clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md

@@ -5,8 +5,8 @@ project: _Underscore
 client: compass-usa
 type: workflow
 status: active
-updated: 2026-06-30
-owners: ["jcardinal", "bala"]
+updated: 2026-08-04
+owners: ["jcardinal", "bala", "dfranks"]
 files: []
 related:
   - clients/compass-usa/profile.md
@@ -93,6 +93,41 @@ one-upstream for bundles.
 5. **Mirror back-links:** every IF on a non-source SO (ODP/Agilant) should have
    `upstreamItemFulfillmentId` set, and each of its items `upstreamItemFulfillmentItemId` set.
 
+## Detecting "empty shell" orders left by a fatal mid-sequence
+
+The ODP EDI import creates a SalesOrder **header** and its **line items** in **separate API
+requests** (`POST /sales-orders`, then `POST /sales-order-items` ~4s later — see the
+[ODP pipeline doc](odp-order-pipeline-to-netsuite.md)). If the *second* request dies, the header is
+already committed and the items roll back, leaving a **zero-item sales order**. This is the
+signature to sweep for whenever an api2 write fatals — and, because a fatal can leave **no API log
+row at all** ([request logging](../../../2.0/apps/api2/features/request-logging.md)), the **data is
+the only trail**.
+
+Sweeps against `Client_Compass`:
+
+1. **Zero-item sales orders** — `SalesOrders LEFT JOIN SalesOrderItems … GROUP BY so.id HAVING
+   COUNT(soi.id) = 0`.
+2. **Zero-item purchase orders** — the same shape over `PurchaseOrders` / `PurchaseOrderItems`.
+3. **ODP SOs with no upstream PO** — ODP SOs (`customerId = 1`) `LEFT JOIN
+   PurchaseOrders_SalesOrders WHERE` the bridge row `IS NULL`.
+
+> **⚠ LIMITATION: this finds only PARTIAL casualties.** An attempt that rolled back *completely*
+> leaves no row in any table, and with no API log row there is no trail anywhere — only the error
+> tracker's **per-event detail** can enumerate those. Never report an orphan sweep as a complete
+> casualty list.
+
+### Repairing a header-only scalar field: direct SQL vs `PUT`
+
+When only a scalar needs setting on an existing order (e.g. `customerPurchaseOrder`, which the ODP
+importer never backfills), **direct SQL is the deliberate choice over `PUT /sales-orders/{uuid}`**: a
+`PUT` fires the record-14 `POST`/`PUT` [payload
+interceptor](../../../2.0/apps/api2/features/api-payload-interceptors.md) and drags the whole order
+back through MITS/NetSuite logic for one field.
+
+Accept the two tradeoffs: **no API log row** for the change, and **`dtUpdated` still moves** (the
+column is `ON UPDATE CURRENT_TIMESTAMP`), which can look like a fresh modification to anything doing
+`dtUpdated`-based change detection downstream.
+
 ## Verified schema corrections (older code/scripts assume these wrongly)
 - **`ItemFulfillmentPackages` does NOT exist** — header IF tracking is `ItemFulfillments_TrackingNumbers`.
 - `ItemFulfillmentItemUnits` has **no `trackingNumberId`** — unit tracking is the bridge
@@ -120,6 +155,13 @@ runtime workflow.
 - High-multiplier over-fulfillment (5×–20×) does not fit the split-PO spurious-link pattern — separate cause.
 
 ## Change history
+- 2026-08-04 — Added the **"empty shell" order signature** — the ODP importer writes the SO header
+  and its items in two separate API requests, so a fatal on the second leaves a committed,
+  zero-item order — plus the three orphan sweeps that detect it (zero-item SOs, zero-item POs, ODP
+  SOs with no `PurchaseOrders_SalesOrders` bridge) and the limitation that they only find *partial*
+  casualties. Also recorded why a header-only scalar repair uses **direct SQL over `PUT`** (a `PUT`
+  fires the record-14 interceptor and the whole MITS/NetSuite path) and its two tradeoffs (no API
+  log row; `dtUpdated` still moves via `ON UPDATE CURRENT_TIMESTAMP`). (dfranks)
 - 2026-06-30 — Updated the IF-stage invariant: `itemFulfillmentStageId` is now NOT NULL and
   picked/packed/shipped stages are all in use (resolved by slug), superseding "stage 3 = Shipped;
   stages 1/2 unused". Compass order status remains shipped-only. (bala)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.508",
+  "version": "1.0.510",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",