toga-ai 1.0.893 → 1.0.895

package/knowledge/1.0/apps/library/INDEX.md

@@ -14,7 +14,7 @@
 | [Branded HTML Email Templates (App_Email_Template)](features/email-templates.md) | Branded HTML email base class `App_Email_Template` (header/body/footer); open when building or fixing a 1.0 branded email or its Outlook rendering. |
 | [App_Email Side Effects & Test Mode (how to send a real email that writes nothing)](features/email-test-mode-and-write-free-sends.md) | 1.0 `App_Email::send()` side effects (log/queue writes, test-mode recipient rewrite) and how to send a genuinely write-free test; open before writing any 1.0 dr |
 | [Error Capture in 1.0 (App_Error_Capture → shared 2.0 Logs DB)](features/error-capture-1-0.md) | 1.0 error-reporting into the shared 2.0 Logs DB (`App_Error_Capture::captureException`), mirroring 2.0's Issue/Event; open when adding a 1.0 capture path or deb |
-| [HTTP 500 Error Monitor (App_SystemMonitor_500Error) — and why its \"Error Type\" is not a diagnosis](features/http-500-error-monitor.md) | 1.0 monitor `App_SystemMonitor_500Error` watching `Logs.Api` responseCode=500; open when an api2 500 alert arrives — its "Error Type" is noise, use `error.id`. |
+| [HTTP 500 Error Monitor (App_SystemMonitor_500Error) — email-only count; diagnose via error.id](features/http-500-error-monitor.md) | 1.0 monitor `App_SystemMonitor_500Error`: counts `Logs.Api` responseCode=500 in the last 5 min and emails devteam; no ClickUp. |
 | [1.0 MVC Page Pattern & New-App Skeleton](features/mvc-page-pattern-and-app-skeleton.md) | Recipe for a new 1.0 (`App_`) app and for adding pages: folder-based MVC routing, page lifecycle, models, multi-DB config; open when standing up a 1.0 app or ad |
 | [NetSuite File Cabinet Content Retrieval via RESTlet (fetchInvoiceFile)](features/netsuite-filecabinet-restlet.md) | How 1.0 pulls NetSuite File Cabinet binary content (invoice PDFs) via a SuiteScript RESTlet (`fetchInvoiceFile`); open when fetching NetSuite file bytes over RE |
 | [assetType from NetSuite itemtype during Item Sync (opt-in per client)](features/netsuite-item-assettype-sync.md) | Stamp `Items.assetTypeId` from NetSuite `item.itemtype` during item sync (opt-in per client, fill-NULL-only); open before enabling asset-type mapping for a clie |

package/knowledge/1.0/apps/library/features/http-500-error-monitor.md

@@ -1,13 +1,13 @@
 ---
-title: "HTTP 500 Error Monitor (App_SystemMonitor_500Error) — and why its \"Error Type\" is not a diagnosis"
+title: "HTTP 500 Error Monitor (App_SystemMonitor_500Error) — email-only count; diagnose via error.id"
 framework: "1.0"
 repo: library
 project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: ["bala"]
+updated: 2026-09-25
+owners: ["bala", "jcardinal"]
 files:
   - library/app/systemmonitor/500error.php
   - worker/crons/infrastructure/system_monitors.php
@@ -19,67 +19,56 @@ related:
   - ./cron-execution-monitoring.md
   - ../../../../2.0/apps/_underscore/features/error-reporting-issue-event.md
   - ../../../../2.0/apps/api2/features/v2-api-error-codes.md
+  - ../../../../2.0/apps/worker2/features/error-escalation-cron.md
 ---
 
-1.0 monitor `App_SystemMonitor_500Error` watching `Logs.Api` responseCode=500; open when an api2 500 alert arrives — its "Error Type" is noise, use `error.id`.
+1.0 monitor `App_SystemMonitor_500Error`: counts `Logs.Api` responseCode=500 in the last 5 min and emails devteam; no ClickUp. Open when a "HTTP 500 Error Alert" email arrives — diagnose via `error.id`.
 
 ## Summary
 
-`App_SystemMonitor_500Error` (`library/app/systemmonitor/500error.php`, title **"HTTP 500 Error Alert"**) is the 1.0 system monitor that watches **`Logs.Api` for `responseCode = 500`** and raises a ClickUp task + a `devteam@togatech.com` email when it finds any. It is the alert most api2 production incidents arrive through.
+`App_SystemMonitor_500Error` (`library/app/systemmonitor/500error.php`, title **"HTTP 500 Error Alert"**) counts **`Logs.Api` rows with `responseCode = 500`** in the last 5 minutes. Any row → returns not-OK → the `App_SystemMonitor` base class emails `devteam@togatech.com`. It carries no per-error detail.
 
-> **⚠ The "Error Type" line in the alert is almost always the literal string `No message found in response payload`, and that string carries ZERO diagnostic signal.** It is generated **by this monitor**, not by the API, and it is the *expected* value for every genuine production api2 500. Do not spend time on it, do not search for it, and do not treat two alerts that share it as the same bug. **The real diagnostic is `error.id` in the logged response payload** — see the lookup recipe below.
+**It creates NO ClickUp tasks (removed 2026-09-25, jcardinal decision).** 500s must not go to ClickUp from here; the 2.0 error pipeline (`Logs.Issue`/`Logs.Event` + [escalation cron](../../../../2.0/apps/worker2/features/error-escalation-cron.md)) owns error tracking. Do not re-add ClickUp task creation.
 
 ## How it works
 
-- **Schedule.** `minuteFrequency = 5`, `minHourToRun = 8`, `maxHourToRun = 17`, `alertRemindFrequency = 240`. Registered in the shared dispatcher `worker/crons/infrastructure/system_monitors.php` alongside the other `App_SystemMonitor` classes (same 300s shared budget — see [NetSuite Sync Alert Monitor](./netsuite-sync-alert-monitor.md) for how a monitor can silently die inside that budget).
-- **Query.** `getStatus()` selects `id, requestHeaders, instanceId, route, method, dtStamp, queryString, responsePayload` from **`Logs.Api`** where `responseCode = 500` and `dtStamp >= NOW() - INTERVAL 5 MINUTE`, on the **`db_toga2_logs`** connection. Zero rows = OK. **The window is 5 minutes and the monitor keeps no cursor**, so its view of an incident is only ever the last 5 minutes.
-- **Non-production rows are filtered by a referer heuristic** — a `referer:` header containing `beta`/`gamma`, or lacking `https://` or `.com`, marks the row non-prod and skips it.
-- **Grouping.** `groupErrors()` keys each group on `referer . ' HTTP 500 Error: ' . $errorMessage` and creates/updates one ClickUp task per key in list `901110669877` (or the current dev-sprint list when it resolves one). Two ClickUp custom fields are set from substring matches on the request headers: a **stakeholder** map (`compass`, `prudential`, `quad`, `spglobal`, `nychh`, `growrk`) and a **platform** map (`togasupply`, `togacommerce`, `togadesk`), each falling back to a default uuid.
+- **Schedule.** `minuteFrequency = 5`, `minHourToRun = 8`, `maxHourToRun = 17`, `alertRemindFrequency = 240`. Registered in the shared dispatcher `worker/crons/infrastructure/system_monitors.php` (same 300s shared budget — see [NetSuite Sync Alert Monitor](./netsuite-sync-alert-monitor.md)).
+- **Query.** `getStatus()` runs `SELECT id FROM Logs.Api WHERE responseCode = 500 AND dtStamp >= NOW() - INTERVAL 5 MINUTE` on `db_toga2_logs`. Zero rows = OK; else `[false, "<N> 500 Errors were found…"]`. No cursor, no grouping, no non-prod filter.
 
-### Where the "Error Type" string comes from
+### Diagnosing an alert: `error.id` (verified lookup recipe)
 
-`groupErrors()` reads `$respPayload->messages[0]->identifiers->message` out of the logged `responsePayload` and falls back to the literal `'No message found in response payload'` when that path is absent (`500error.php:93-99`).
-
-**api2 leaves those identifiers empty on every production 500 by design.** `api2/Controller/Index.php` (~L271-285) populates `message` and `trace` **only** when `_Environment::isDebugMode()` is true, because `getTrace()` returns frame **arguments** verbatim and any stack with `_Database::register()` on it would hand the database username and password to anyone who could force a 500. See the api2 [error-codes doc](../../../../2.0/apps/api2/features/v2-api-error-codes.md). So the fallback is not a failure — it is the **normal** state in production. Its only meaning is "this 500 came from production api2."
-
-### The real diagnostic: `error.id` (verified lookup recipe)
-
-The api2 envelope carries `error: { code, id }`. **`id` is the quotable Core Logs reference**, in the form `<Logs.Issue.reference>-<Logs.Event.eventNumber>` (e.g. `78-859`), produced by `_Error::captureException()` (`_underscore/Error.php:334+`), which resolves/creates a `Logs.Issue` by fingerprint, writes a `Logs.Event`, and returns the reference. Three queries get you from an alert to the exception:
+api2 leaves `message`/`trace` empty on production 500s by design (`api2/Controller/Index.php` ~L271-285 fills them only in debug mode, to keep DB credentials out of stack args — see [error codes](../../../../2.0/apps/api2/features/v2-api-error-codes.md)). The envelope carries `error: { code, id }`; **`id` = `<Logs.Issue.reference>-<Logs.Event.eventNumber>`** (e.g. `78-859`), from `_Error::captureException()` (`_underscore/Error.php:334+`).
 
 ```sql
-# 1. Read error.id out of the logged envelope (ids come from the alert body)
+# 1. Recent 500s and their envelopes (read error.id from responsePayload)
 SELECT id, route, method, dtStamp, responsePayload
 FROM Logs.Api
-WHERE id IN (<ids from the alert>);
+WHERE responseCode = 500 AND dtStamp >= NOW() - INTERVAL 1 HOUR;
 
-# 2. The exception itself + its full framework stack (reference = the part before the dash)
+# 2. The exception + full framework stack (reference = part before the dash)
 SELECT id, reference, status, errorMessage, trace, totalOccurrences, dtLastOccurred
 FROM Logs.Issue
 WHERE reference = '78';
 
-# 3. The specific occurrence — and WHICH TENANT it belongs to
+# 3. The occurrence — and WHICH TENANT
 SELECT clientId, dtCreated, errorMessage
 FROM Logs.Event
 WHERE issueId = <issue.id> AND eventNumber = <n>;
 ```
 
-- `Logs.Issue.trace` is the full framework stack — the thing the alert deliberately does not contain.
-- `Logs.Event.clientId` is how you find out **which client** was affected; the monitor's stakeholder guess is a header substring match and is not authoritative.
-- **api2's own catch block is the ONLY capture point for the inbound-API 500 surface.** api2 handles `Throwable` in its front controller, so `_Error::exceptionHandler` never fires for API traffic. If api2's catch did not capture it, nothing did.
+- `Logs.Event.clientId` is the authoritative tenant.
+- **api2's own catch block is the ONLY capture point for inbound-API 500s** (`_Error::exceptionHandler` never fires for API traffic). If api2's catch did not capture it, nothing did.
 
-Full mechanics of the Issue/Event model, reference encoding, and its failure modes: [error reporting](../../../../2.0/apps/_underscore/features/error-reporting-issue-event.md).
+Full Issue/Event mechanics: [error reporting](../../../../2.0/apps/_underscore/features/error-reporting-issue-event.md).
 
 ## Gotchas
 
-- **⚠ Known defect (2026-08-10, NOT fixed) — all unrelated 500s collapse into ONE ClickUp task.** Because the grouping key is `referer + errorMessage` and `$errorMessage` is the **constant** `No message found in response payload` for every production 500, the key degenerates to the referer alone. Consequences: distinct unrelated defects sharing a referer are **merged into one ClickUp task** (mixed routes, mixed incidents); the responder cannot tell one bug from five; the occurrence count is meaningless as impact. **Proposed fix (recorded, not implemented):** parse the `Logs.Issue` reference out of `error.id` and **group on that** — the platform's own deduplicated identity — and surface `error.id` in the task description so the responder can skip the manual lookup. Fall back to the current key only when `error.id` is null.
-- **A 500 whose request has NO `referer:` header passes the production filter and groups under an empty referer.** The `$isProd = false` assignment only happens *inside* the referer branch (`500error.php:55-74`), so a header set with no referer line keeps `$isProd = true` and `$referer = ''`. Server-to-server inbound API traffic has no referer, so all of it lands in one unnamed group — compounding the collapse defect above.
-- **Repeat count in the alert is NOT user impact.** apiproxy retries a 5xx across regions, so one user action routinely produces several near-identical `Logs.Api` 500 rows within seconds, on different `instanceId`s, **sharing one `transactionId`**. Check the transaction id before reading a burst as a burst of users.
-- **The 5-minute window means the alert is a sample, not the incident.** Always go to `Logs.Api` / `Logs.Issue` for the full extent and the first occurrence — `Issue.dtLastOccurred` + `totalOccurrences` are the real timeline.
-- **`Logs.Issue.status = RESOLVED` does not mean fixed** — it means *stopped firing* (the worker2 frequency-decay cron auto-resolves on consecutive quiet windows). A live defect on a low-traffic tenant reads as RESOLVED the moment that tenant goes quiet. See [error-issue auto-resolution](../../../../2.0/apps/worker2/features/error-issue-auto-resolution.md).
-- **Hardcoded ClickUp API token in source.** `CLICK_UP_API_ACCESS_TOKEN` is a literal class constant in this file at line 14 (pre-existing, same pattern as the NetSuite Sync Alert monitor). It belongs in config/secrets; treat the in-repo value as compromised. (Location only — the value is never copied into the KB.)
-- **Stakeholder/platform attribution is a substring guess.** `str_contains($headers, 'growrk')` style matching mis-attributes any request whose headers happen to mention another tenant, and silently defaults when nothing matches. Use `Logs.Event.clientId` for the real tenant.
+- **The count is not user impact.** apiproxy retries a 5xx across regions, so one user action makes several `Logs.Api` 500 rows within seconds, on different `instanceId`s, **sharing one `transactionId`**. Check the transaction id first.
+- **The 5-minute window is a sample, not the incident.** Use `Logs.Issue.dtLastOccurred` + `totalOccurrences` for the real timeline.
+- **No non-prod filter any more.** The old referer heuristic went with the ClickUp code; every 500 row in the window counts.
+- **`Logs.Issue.status = RESOLVED` means stopped firing, not fixed** — see [error-issue auto-resolution](../../../../2.0/apps/worker2/features/error-issue-auto-resolution.md).
 
 ## Related
 
 - [Error capture in 1.0](./error-capture-1-0.md), [NetSuite sync alert monitor](./netsuite-sync-alert-monitor.md), [Cron execution monitoring](./cron-execution-monitoring.md).
-- [2.0 error reporting Issue/Event](../../../../2.0/apps/_underscore/features/error-reporting-issue-event.md), [v2 API error codes](../../../../2.0/apps/api2/features/v2-api-error-codes.md).
+- [2.0 error reporting Issue/Event](../../../../2.0/apps/_underscore/features/error-reporting-issue-event.md), [v2 API error codes](../../../../2.0/apps/api2/features/v2-api-error-codes.md), [error escalation cron](../../../../2.0/apps/worker2/features/error-escalation-cron.md).

package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md

@@ -6,7 +6,7 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-09-24
+updated: 2026-09-25
 owners: ["dfranks", "bala", "jcardinal", "mhammontree", "snaredla", "rgirish"]
 files:
   - worker/crons/toga2/netsuite/common_sync_togasupply.php
@@ -1208,6 +1208,8 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
 
 ## Gotchas / known issues
 
+- **The 2026-09-17 transient-read retry covers NetSuite reads only** (`App_Api_Netsuite_Rest::send`), **not** api2 calls via `App_Api_Toga2::send`. An api2 blip (e.g. "cURL error with API transaction: Empty response body from API (HTTP 200)") still aborts the section for that run and drops the window to `288000-IDLE`; it self-heals next run. One such failure is not a collapse — check the window grows back before acting. Seen NYCHH ITEM_FULFILLMENTS 2026-09-25, issue 873 (`common_sync_togasupply.php:1287` → `toga2.php:6630`).
+
 - **⚠ Sales-order downstream LINE links were deleted on every PO re-sync (fixed 2026-09-24).** In
   `syncPurchaseOrderFromNetsuite` `$incomingSalesOrderItemUuid` was initialized null and never
   assigned, so the existing-line branch deleted every `SalesOrderItems_PurchaseOrderItems` link and

package/knowledge/1.0/apps/worker/workflows/tracing-a-worker-cron-run-in-production.md

@@ -116,7 +116,7 @@ Detail on what else `send()` writes (the `Common.EmailsQueued` branch, `setIsLog
   ```
   The script then does nothing on every tick — and per Steps 2 and 4 you see no output and no `CronJobExecutions` row, i.e. it looks exactly like "the cron never fired". Typed parameters and return types (`: void`, `: array`, `: bool`, `?array`, `?object`, `object $x`) are PHP 7.0/7.1 and **are** safe here; already used in `worker/crons/toga2/compass/backfill_items_isfulfillable.php` and `update_item_fulfillments_in_netsuite.php`.
   - `1.0/standards/backend-php.md` already forbids named arguments in 1.0 — but a workspace `CLAUDE.md` / `.github/copilot-instructions.md` that says *"use PHP 8 Named Parameters"* applies to **2.0 only**. The 1.0 standard wins in `library/`, `worker/` and every other 1.0 app.
-  - **Do not read a PHP 8 function call in existing code as proof the tier is PHP 8.** `library/app/systemmonitor/500error.php` uses `str_contains()`; that file is not a version signal, it is a latent bug.
+  - **Do not read a PHP 8 function call in existing code as proof the tier is PHP 8.** A 1.0 file using `str_contains()` (as `library/app/systemmonitor/500error.php` did until 2026-09-25) is not a version signal; it is a latent bug.
 - **Deploy-time trap: a `&` (or any shell metacharacter) in the cron FILENAME silently kills it.** `ebs/cron.worker.php:41` builds each crontab line by plain string concatenation (`'php /var/www/html/crons/' . $cron->cron`) with **no quoting and no escaping**. Cron parses that line as a shell command, so a `&` in the filename is read as *"run everything before this in the background and start a new command."*
   - Worked example (found 2026-09-03): a developer renamed the Compass Canada status cron to `crons/toga2/compasscanada/update_salesorder_status_from_g&t.php` and repointed `schedules/cron.worker.sync.json` at it. The crontab line becomes two commands — `php .../update_salesorder_status_from_g` (a file that does not exist) and a stray `t.php` — so the real script **never executes**, on every tick, with no error anywhere. Looks exactly like "the cron never fired".
   - **Rule: a cron filename must be plain `[a-z0-9_]` plus `.php`.** No `&`, no spaces, no `;`, `|`, `$`, `(`, `)`, `*`, `?`, or quotes. The *display name* in `schedules/*.json` is free text and may contain anything; only the `cron` path is shell-parsed.

package/knowledge/2.0/apps/_underscore/features/error-reporting-issue-event.md

@@ -506,9 +506,8 @@ WHERE issueId = <issue.id> AND eventNumber = <n>;
   [api2 request logging](../../api2/features/request-logging.md). Read occurrence counts through
   it or you will overstate user impact.
 - The **1.0 alert surface** that fires on these is
-  [App_SystemMonitor_500Error](../../../../1.0/apps/library/features/http-500-error-monitor.md) — its
-  "Error Type" is a fallback literal with **zero** diagnostic value, which is why this recipe
-  exists.
+  [App_SystemMonitor_500Error](../../../../1.0/apps/library/features/http-500-error-monitor.md) — an
+  email-only count with no per-error detail, which is why this recipe exists.
 
 #### Shutdown handler
 

package/knowledge/2.0/apps/toga25-supply/features/persisted-query-cache.md

@@ -6,8 +6,8 @@ project: TOGa 2.5 Supply
 client: shared
 type: feature
 status: active
-updated: 2026-09-24
-owners: ["apeterson"]
+updated: 2026-09-25
+owners: ["apeterson", "jcardinal"]
 files:
   - toga25-supply/src/App.tsx
   - toga25-supply/src/contexts/AuthContext.tsx
@@ -52,15 +52,16 @@ const localStoragePersister = createSyncStoragePersister({
 
 `PersistQueryClientProvider` (`src/App.tsx:33`) wraps the app. The `QueryClient` default is `staleTime: 0` (`src/App.tsx:18`).
 
-### What is persisted — everything except three namespaces
+### What is persisted — everything except four namespaces
 
-`shouldDehydrateQuery` (`src/App.tsx:63-70`) keeps TanStack's default (persist successful queries) **minus** three query-key namespaces:
+`shouldDehydrateQuery` (`src/App.tsx:63-70`) keeps TanStack's default (persist successful queries) **minus** four query-key namespaces:
 
 | Excluded namespace | Why |
 |---|---|
 | `talos` | chat titles / assistant ids are per-session user data; must not sit on disk |
 | `surface-meta` | the surface bundle carries **permission and feature-gate** state |
 | `surface-meta-group` | same bundle, grouped endpoint |
+| `table-meta` | TableView column/join config is schema-shaped; a stale copy sends dropped joins/columns (see gotcha) |
 
 The surface exclusion is the important one. `useFetchSurfaceMeta` runs at `staleTime: Infinity`, so a *persisted* copy would never refetch — a permission **grant**, or worse a **revoke**, would not reach a browser that already had one until the persister's 24h `maxAge` expired. Excluding it costs one request per page load; the in-memory cache still covers every route change. `staleTime: Infinity` is safe there **only because** it is excluded from persisting.
 
@@ -69,7 +70,7 @@ The surface exclusion is the important one. `useFetchSurfaceMeta` runs at `stale
 | Query key | Request |
 |---|---|
 | `["table-data", slug, …]` | `getDataTableData` (blox) |
-| `["table-meta", slug]` | `GET /table-views/meta` (blox `components/Table/hooks/useFetchTablePageMeta.js:8`) |
+| `["table-meta", slug]` | `GET /table-views/meta` (blox `useFetchTablePageMeta.ts:15`) — **not persisted** |
 | `["status-colors", route]` | `GET /{route}?fields=slug,colorHex&sort=sortOrder` |
 | `["currentUser"]` | `GET /users` |
 | `["clientFields", slug, role]` | client `FIELDS` config |
@@ -143,6 +144,9 @@ Type-check **both** `-p tsconfig.app.json` and `-p tsconfig.node.json` — `vite
 
 ## Gotchas
 
+- **⚠ Never persist a query that carries schema-shaped config (TableView meta, column/join lists).** Before 2026-09-25 `table-meta` was persisted: after any TableView column/join removal, each browser's first load restored the OLD meta and fired the grid with the dropped join → api2 **1054 "Unknown column … in on clause"** in bursts, then meta refetched (`staleTime: 0`) and the grid recovered. A page reload does NOT avoid it (cache is in localStorage). Proven: NYCHH TableView change `Client_Nychh/2026-09-24a` (dropped `TransferOrders.purchaseOrderId` join) → prod `Logs.Event` issue 872. `table-data` / `table-data-infinite` stay persisted — safe because gated `enabled: !!tableViewMeta` (only a brief stale-row flash).
+- **Diagnosis:** a 1054 on a column a migration dropped, while the prod TableView is already clean = stale persisted front-end config, not a DB fix. Confirm: the error burst ends seconds before the same user's first good query in `Logs_<Tenant>.Api`.
+
 - **⚠ A non-zero `staleTime` on a persisted query serves yesterday's data and never fetches.** Data hooks in this app use `staleTime: 0`. Hooks that currently break this rule **and are persisted**: `useClientFields`, `useCurrentUser`, `useStatusTypeValues`, `useStatusColors`, `useSalesOrderUuidForServiceRequest` (all `Infinity`), and the three VendorItem form hooks (`5 * 60_000`). Each can serve a previous session's answer.
 - `useTalosSession` and `useTalosThreads` also set a non-zero `staleTime`, but their keys start with `talos`, so they are **not** persisted — in-memory only, safe.
 - **⚠ An edit to a client `FIELDS` JSON looks like it does nothing — and it is NOT Vite HMR.** `useClientFields` is `staleTime: Infinity` **and** persisted, so the old config is frozen in `supply-chain-query-cache`. Clear that key to see the edit. Proper fix: exclude `clientFields` in `shouldDehydrateQuery` (`src/App.tsx:42`).
@@ -152,4 +156,5 @@ Type-check **both** `-p tsconfig.app.json` and `-p tsconfig.node.json` — `vite
 - The `maxAge` is one timestamp for the entire blob — you cannot expire a single query through it.
 
 ## Change history
+- 2026-09-25 — Excluded `table-meta` from persistence (stale TableView meta caused api2 1054 bursts after a join removal; NYCHH issue 872). Deployed to prod. (jcardinal)
 - 2026-09-24 — Noted the frozen-`FIELDS`-JSON symptom (persisted `useClientFields` at `staleTime: Infinity`, mistaken for broken Vite HMR) and the `shouldDehydrateQuery` fix. (apeterson)

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -10,7 +10,7 @@
 | [Callback Scheduling ("call me back") — worker2 AI-BDR](features/callback-scheduling.md) | The AI-BDR callback path (prospect says "call me back" → dialer redials) and its production defects; open when working on Vapi callback capture/parse/schedule o |
 | [ClickUp Connectivity Watchdog](features/clickup-connectivity-watchdog.md) | Cron watchdog that emails when the ClickUp integration goes silent in business hours; open when touching ClickUp webhook health or `_Worker_Clickup_Health`. |
 | [ClickUp Custom-Field Conventions (constant IDs, runtime option values, Task Type migration)](features/clickup-custom-field-conventions.md) | How worker2 code must reference ClickUp custom fields; open before reading/writing any ClickUp custom field or touching the Task Type migration. |
-| [ClickUp Design Sprint Automation (Final Design Outcome)](features/clickup-design-sprint-automation.md) | `_Worker_Clickup_Design` drives the design-sprint workflow in ClickUp (spawn iteration / complete task on Final Design Outcome, daily review reminders); open wh |
+| [ClickUp Design Sprint Automation (Final Design Outcome)](features/clickup-design-sprint-automation.md) | `_Worker_Clickup_Design` drives the design-sprint workflow in ClickUp (FDO default on create, spawn iteration / complete task on Final Design Outcome, daily rev |
 | [ClickUp General Automation (Sub-Environment reservation, Has UI / Has Code roll-ups)](features/clickup-general-automation.md) | `_Worker_Clickup_GeneralAutomation` reacts to ClickUp custom-field changes on a Task; open when working on sub-environment reservation or Has UI / Has Code roll |
 | [ClickUp GitHub-tab Auto-linking & Ticket-id Branch Naming](features/clickup-github-autolink.md) | How ClickUp surfaces branches/PRs/commits in a ticket's GitHub tab and the branch/PR-title naming that triggers it; open when naming branches for ticket work or |
 | [ClickUp Project & Opportunity Multi-List Routing](features/clickup-project-routing.md) | Routes ClickUp tasks into secondary multi-list memberships by custom-field values; open for opportunity/epic routing, the stage-is-a-list constraint, or webhook |
@@ -18,7 +18,7 @@
 | [ClickUp Subtask Activity → Parent Opportunity/Epic Comments](features/clickup-subtask-activity.md) | Mirrors subtask status/comment activity up to its top-level Opportunity/Epic as one comment; open for the subtask fan-out, its dedup ledger, or structural ances |
 | [ClickUp Task Description Fluffer (Talos KB Investigation)](features/clickup-task-fluffer-talos.md) | Async worker that "fluffs" a new ClickUp task by running its name+description through the Talos `dev-core` KB agent and writing the report back; open for the fl |
 | [ClickUp Webhook Health Monitor → OneUptime (Monitor/Clickup/CheckWebhook)](features/clickup-webhook-health-monitor.md) | worker2 cron that checks the team's ClickUp webhook subscriptions and pages via a OneUptime Incoming-Request monitor; open when tuning the ClickUp webhook alarm |
-| [ClickUp Work Type Automation (Committed / Conditional / Stretch)](features/clickup-work-type-automation.md) | Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date; open when work-type promotion/demotion |
+| [ClickUp Work Type Automation (Committed / Conditional / Stretch)](features/clickup-work-type-automation.md) | Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date, and sets new-task defaults on taskCrea |
 | [worker2 Codeception integration tests (2.0 test-framework pilot)](features/codeception-integration-tests.md) | First 2.0 app-level automated test suite: Codeception 5 / PHPUnit 11 in worker2 (pilot, Jeff's direction), real-DB integration tests with hard local-only safety |
 | [worker2 — Committed Secrets & Config Exposure (debt register)](features/committed-secrets-and-config-debt.md) | Standing list of credentials committed to the worker2 repo (locations only); open before adding a credential to worker2 or assuming a config value is private. |
 | [Compass VIP Support Importer (worker2)](features/compass-vip-support-importer.md) | worker2 action that ingests Compass's quarterly VIP spreadsheet and sets each VIP's support tech (`Users.c_supportedByUserId` in `Client_Compass`); open when wo |

package/knowledge/2.0/apps/worker2/features/clickup-design-sprint-automation.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: ["ajean"]
+updated: 2026-09-25
+owners: ["ajean", "jcardinal"]
 files:
   - worker2/Worker/Clickup/Design.php
   - worker2/Worker/Clickup.php
@@ -18,13 +18,13 @@ related:
   - ./team-sprint-management.md
 ---
 
-`_Worker_Clickup_Design` drives the design-sprint workflow in ClickUp (spawn iteration / complete task on Final Design Outcome, daily review reminders); open when working on design-sprint automation, FDO handling, or the spurious-email / no-spawn bugs.
+`_Worker_Clickup_Design` drives the design-sprint workflow in ClickUp (FDO default on create, spawn iteration / complete task on Final Design Outcome, daily review reminders); open when working on design-sprint automation, FDO handling, or the spurious-email / no-spawn bugs.
 
 ## Summary
 
 `_Worker_Clickup_Design` replaces a set of native ClickUp automations. When a task's **Final Design Outcome** (FDO) drop-down is set to an actionable value, the worker spawns an iteration task or completes the task; a daily cron reminds the Design Approver of pending reviews.
 
-**Current state (2026-07-15):** the silent no-op fix (worker2 PR #98, commit `7a36564`) and the extended iterated-automation review workflow are shipped and live. On top of that, an intermittent "setting FDO to *Iterated – New Task* does NOT spawn" failure was fixed end-to-end on **worker2 PR #113 + _underscore PR #669** (deploy together — see Data model). Two root causes fixed: (1) the spawn gate ignored design tasks in a **Backlog Design list** (only the True Design Sprints folder / hardcoded list IDs were admitted); (2) `handleFinalDesignOutcome` resolved the FDO from a stale per-process `getTaskDetails` snapshot taken before ClickUp reflected the drop-down write (read-after-write race) — now the webhook's authoritative `after` value is threaded in. Also hardened shared `_Component_Api_Clickup::send()` reliability and added observability breadcrumbs. 55 test assertions pass; `php -l` clean. The native ClickUp automations masking the original no-op must stay disabled — see Gotchas.
+**Current state (2026-09-25):** live on both design folders (`DESIGN_SPRINTS_FOLDER_IDS`: True Design Sprints `90115939842`, Design Sprints `90116585900`) + Backlog Design lists. Approve→complete and iterate→spawn verified working after the 9/21 multi-folder gate fix (commit `6dcebe0`); earlier misses on the new folder were the old single-folder gate. History of the 2026-07 no-op / no-spawn fixes (worker2 PR #98, #113 + _underscore PR #669) is in Gotchas. The native ClickUp automations masking the original no-op must stay disabled.
 
 ## How it works
 
@@ -32,15 +32,16 @@ related:
 
 - `Worker/Clickup/Design.php` → `_Worker_Clickup_Design` — the design-sprint engine.
   - `handleFinalDesignOutcome(string $taskId, ?string $webhookFdoValue = null)` — decides spawn vs. complete; branches on **all five** FDO options explicitly (no silent fall-through). Optional `$webhookFdoValue` is the authoritative new value from the webhook, preferred over the (possibly stale) snapshot — see Gotchas (read-after-write).
-  - `isDesignSprintTask(...)` — the spawn/handler gate. Admits: the True Design Sprints folder, **OR** (the Backlog folder `BACKLOG_FOLDER_ID` AND the list name matches `DESIGN_LIST_NAME_PATTERN`), **OR** the `DESIGN_LIST_IDS` allowlist (fallback). Fixes the "backlog Design list silently skipped" root cause — see Gotchas.
+  - `handleTaskCreated($taskId, $taskDetails)` — called from `_Worker_Clickup` `taskCreated` (wrapped in `setFieldSafely`). Sets FDO = **In Progress** when empty; design-scope gated. Replaces a native ClickUp automation that existed only on the old True Design Sprints folder (decision: code, not a ClickUp automation).
+  - `isDesignSprintTask(...)` — the spawn/handler gate. Admits: any `DESIGN_SPRINTS_FOLDER_IDS` folder, **OR** (the Backlog folder `BACKLOG_FOLDER_ID` AND the list name matches `DESIGN_LIST_NAME_PATTERN`), **OR** the `DESIGN_LIST_IDS` allowlist (fallback). Fixes the "backlog Design list silently skipped" root cause — see Gotchas.
   - `resolveFinalDesignOutcomeOptionId(object $taskDetails, ?string $overrideValue = null)` (Design.php:298-317) — maps the drop-down value to an option ID; **string-compares** value/orderindex/id (see Gotchas). When `$overrideValue` is given (the webhook `after` value) it wins over the snapshot's field value; the option *list* is still read from the snapshot.
   - `handleTaskEnteredIterated()` — **new**; fired when a task ENTERS "iterated" status. Sets FDO = "In Review" then increments the Iteration custom field by +1 (idempotent — see Gotchas). Folder-gated via `isInDesignSprintsFolder()`.
   - `isInDesignSprintsFolder(...)` — **new** private folder-gate helper (reused by the iterated-entry trigger).
-  - `spawnIterationTask(...)` / `buildIterationCreatePayload(...)` / `carryDesignApprover(...)` / `carryComments(...)` (**new**) — duplicate the task and copy description, Design Approver, and now all user comments; the duplicate no longer bumps Iteration (moved to the iterated-entry trigger) and is set to FDO = "In Progress" post-create.
+  - `spawnIterationTask(...)` / `buildIterationCreatePayload(...)` / `carryDesignApprover(...)` / `carryComments(...)` / `carryDropdownField(...)` — duplicate the task in status `STATUS_NEW_TASK` (`in progress`) and copy description, Design Approver, Task Knowledge (`CLICK_UP_CUSTOM_FIELD_ID__TASK_KNOWLEDGE`, const in Design.php), Work Effort (`CLICK_UP_CUSTOM_FIELD_ID__WORKEFFORT`), and all user comments; the duplicate no longer bumps Iteration (moved to the iterated-entry trigger) and is set to FDO = "In Progress" post-create.
   - `completeTask(...)` — closes an approved / cancelled task (also used on iterated → complete).
   - `SendReviewReminders(...)` — daily reminder cron action; now also fetches the Backlog folder's lists and includes those matching `DESIGN_LIST_NAME_PATTERN`, mirroring `isDesignSprintTask()` so newly-created backlog Design lists get reminders without a code change.
   - `findExistingSpawnedIteration(...)` — its previously-silent `catch` now logs.
-  - `TRUE_DESIGN_SPRINTS_FOLDER_ID`, `BACKLOG_FOLDER_ID` (`'90116192732'`), and `DESIGN_LIST_NAME_PATTERN` (`/(?<!non-)\bdesign\b/i`) constants.
+  - `DESIGN_SPRINTS_FOLDER_IDS`, `BACKLOG_FOLDER_ID` (`'90116192732'`), and `DESIGN_LIST_NAME_PATTERN` (`/(?<!non-)\bdesign\b/i`) constants.
 - `Worker/Clickup.php` `_Worker_Clickup::extractCustomFieldAfterValue(object $historyItem): ?string` (**new**) — normalizes a webhook `history_items[].after` custom-field value (scalar orderindex / object / JSON array with `id`|`orderindex`). Distinguishes a **missing** `after` (returns `null` → caller falls back to the snapshot) from an **explicit null** `after` (returns `''` sentinel → field was cleared → resolve to no outcome, do NOT act on the stale previous option).
 - `Worker/Clickup.php` `Webhook` — the `switch ($historyItem->after->status)` block (see Gotchas re: after vs. before) now has a `case _Worker_Clickup_Design::STATUS_ITERATED` dispatching to `handleTaskEnteredIterated()`.
 - `Worker/Clickup.php` — routes ClickUp webhooks.
@@ -57,8 +58,8 @@ related:
    - **"Cancelled"** → `completeTask` (terminal; newly handled — Cancelled now closes the task).
    - **"In Progress" / "In Review"** → explicit no-op.
    - unrecognized → logged (no more silent implicit-else fall-through).
-3. **Iterated-entry trigger (new):** when a True Design Sprints task ENTERS the "iterated" status, `handleTaskEnteredIterated()` sets FDO = **"In Review"** and increments the **Iteration** custom field by +1. This now drives the review state + round count on review submission; previously nothing fired on entering "iterated". Folder-gated.
-4. **Duplicate field carry-over:** the spawned iteration task copies description, Design Approver, and **all user comments** (`carryComments()` — GET `/task/{id}/comment`, re-posted oldest-first with `[Original by <user> on <date>]` attribution, non-fatal). It copies **Iteration unchanged** (omitted if the source has none — the +1 now lives only on the iterated-entry trigger, so a review round is never double-counted) and is set to FDO = **"In Progress"** post-create (POST `/task/{id}/field/{id}` drop_down pattern).
+3. **Iterated-entry trigger:** when a design-sprint task ENTERS the "iterated" status, `handleTaskEnteredIterated()` sets FDO = **"In Review"** and increments the **Iteration** custom field by +1. This now drives the review state + round count on review submission; previously nothing fired on entering "iterated". Folder-gated.
+4. **Duplicate field carry-over:** the spawned iteration task starts in `in progress` and copies description, Design Approver, Task Knowledge, Work Effort, and **all user comments** (`carryComments()` — GET `/task/{id}/comment`, re-posted oldest-first with `[Original by <user> on <date>]` attribution, non-fatal). It copies **Iteration unchanged** (omitted if the source has none — the +1 now lives only on the iterated-entry trigger, so a review round is never double-counted) and is set to FDO = **"In Progress"** post-create (POST `/task/{id}/field/{id}` drop_down pattern).
 5. Daily reminder cron `Core.CronJobs` id 21 ("Design Review Reminders", `0 9 * * 1-5`, action `Clickup/Design/SendReviewReminders`) nudges the approver about pending reviews.
 
 ### Data model / configuration
@@ -71,6 +72,8 @@ Configuration lives as ClickUp identifiers (folder IDs, custom-field IDs, drop-d
 
 ## Gotchas
 
+- **Webhooks triggered by the automation's own ClickUp user ARE delivered.** Do not blame self-suppression when a chained step (e.g. spawn → FDO write) seems not to fire; check the folder gate first.
+- **Status names must exist in every gated list.** `STATUS_NEW_TASK = 'in progress'` was verified present in the design lists via prod webhooks (the old `'to do'` is not used).
 - **The ClickUp API auth is a single hard-coded personal token — a dead token silently kills ALL ClickUp automation.** The integration authenticates via one committed class constant, `_Component_Api_Clickup::ACCESS_TOKEN` (in `_underscore/Component/Api/Clickup/Clickup.php`), identical on `_production` and feature branches. Known secret-in-committed-code risk. When the token is revoked/expired, `GET /user` and `GET /task` return **HTTP 401 "Oauth token not found" / error `OAUTH_019`**; `getTaskDetails` then throws and **every** Clickup/Webhook worker job fails — presenting as "the automation silently does nothing." This happened mid-session on 2026-07-07 (~09:49). **Fix:** regenerate the token in ClickUp, update the `ACCESS_TOKEN` constant, and deploy `_underscore`. (Do not record the token value anywhere.)
 - **Iterated-entry trigger is idempotent — order matters.** `handleTaskEnteredIterated()` writes **FDO = In Review FIRST** as the marker, THEN increments Iteration. A re-delivered / retried webhook sees FDO already "In Review" and skips, so **Iteration can never compound-double-increment**. Worst case is a single missed +1, visible as a failed WorkerJob. Reordering these two writes reintroduces the double-count risk.
 - **`taskStatusUpdated` dispatch structure (route-safety facts).** The `switch ($historyItem->after->status)` block sits **OUTSIDE** the `if (!is_null($sprintNumber))` guard, so entering-status handlers fire even for design tasks with no "sprint N" tag — **each case must self-guard on `sprintNumber`** (the individual cases do). No webhook loop: custom-field writes emit `taskUpdated` (not `taskStatusUpdated`) and land on the FDO no-op cases; the Iteration write's custom_field is named "Iteration" so it never re-enters the FDO dispatch gate.

package/knowledge/2.0/apps/worker2/features/clickup-work-type-automation.md

@@ -6,16 +6,17 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
+updated: 2026-09-25
 owners: ["jcardinal"]
 files:
   - worker2/Worker/Clickup.php
   - worker2/Tests/Worker/ClickupWorkTypeTest.php
 related:
   - ./clickup-project-routing.md
+  - ../workflows/clickup-credential-rotation.md
 ---
 
-Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date; open when work-type promotion/demotion misbehaves.
+Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date, and sets new-task defaults on taskCreated; open when work-type promotion/demotion or taskCreated field writes misbehave.
 
 ## Summary
 
@@ -41,6 +42,13 @@ Flow:
    - `Stretch` with a due date inside the current sprint → set Committed/Conditional as above.
 6. When a task is depended on by another (`$dependency->depends_on == $taskId`), it recurses into the blocking direction so completing a task re-evaluates the tasks it was blocking. The `$dependsOnTaskId` argument breaks two-node cycles.
 
+### New-task defaults (`taskCreated`)
+
+- Every optional field write in `taskCreated` goes through private `setFieldSafely($taskId, $fieldLabel, callable)`: try/catch → `error_log` + Sentry. One refused field never aborts the job (the later Clickup/Fluffer/Process dispatch still runs).
+- **Work Type = Unplanned** when empty, only if the list name starts with `Sprint` AND `isTaskInQaQcScope()` (Dev Sprints folder). Option id resolved by name at runtime (`WORK_TYPE_UNPLANNED`). Non-`Sprint` lists get no Work Type write (jcardinal decision).
+- **Previous Sprint Work Type** = the Work Type name (or `Unplanned` when empty), resolved by name via the field's own options.
+- Design FDO default is dispatched from here too — see [design sprint automation](./clickup-design-sprint-automation.md).
+
 Data model: reads/writes the `Team` database (`Tasks`, `Developers`, `Tasks_Developers`, `Sprints`) and the ClickUp REST API. The Work Type itself lives in ClickUp as a custom field (`CLICK_UP_CUSTOM_FIELD_ID__WORKTYPE`), not in the DB.
 
 ## Gotchas
@@ -50,6 +58,8 @@ Data model: reads/writes the `Team` database (`Tasks`, `Developers`, `Tasks_Deve
 - **No PHPUnit harness** in worker2. The regression test `Tests/Worker/ClickupWorkTypeTest.php` is a plain-PHP script (reflection on `isStatusComplete`) run with `php Tests/Worker/ClickupWorkTypeTest.php`.
 - **ClickUp dropdown custom-field `value` is the option ORDERINDEX, not its name.** The webhook payload for a dropdown/labels custom field delivers the selected option's integer `orderindex`, *not* the option's display name. Any name-keyed lookup (e.g. `$lookupPreviousSprintWorkTypeByName`) will silently never match if fed the raw `value`. Resolve the name first via the field's own `type_config->options` — build a map keyed by `orderindex` (`$workTypesByIndex[$opt->orderindex] = $opt`) and read `->name` from it. This is the proven pattern in `_Worker_Team_Sprint::SprintLock` (`Worker/Team/Sprint.php`, the Work Type resolve ~L3894 and the POST ~L3971). Guard the lookup with `isset($workTypesByIndex[$value])` so a stale/deleted option orderindex doesn't warn.
 
+- **2026-09-21..25 outage: every taskCreated failed (~113 jobs, 0 Fluffer jobs).** The automation user lacked edit rights on Previous Sprint Work Type → HTTP 401 `ACCESS_606`; the throw killed the rest of the job. Fixed by field permissions + `setFieldSafely`. Any new optional write must use `setFieldSafely`. Account-change check: [credential rotation](../workflows/clickup-credential-rotation.md).
+
 ## Related
 
 - [ClickUp Project & Opportunity Multi-List Routing](./clickup-project-routing.md)

package/knowledge/2.0/apps/worker2/features/qa-qc-review-pipeline.md

@@ -5,8 +5,8 @@ repo: worker2
 project: Worker
 client: shared
 type: feature
-status: draft
-updated: 2026-09-16
+status: active
+updated: 2026-09-25
 owners: [jcardinal]
 files:
   - worker2/Worker/Clickup/QaReview.php
@@ -26,14 +26,14 @@ related:
   - ./clickup-custom-field-conventions.md
 ---
 
-ClickUp-driven QA/QC review + promotion chain (Dev→QA→QC→Client Review→Stage→Prod) for worker2; open when touching a review gate, the QC/Stage crons, or the Feature/Hotfix flows. Not deployed yet — `feature-qa-qc` working tree.
+ClickUp-driven QA/QC review + promotion chain (Dev→QA→QC→Client Review→Stage→Prod) for worker2; open when touching a review gate, the QC/Stage crons, or the Feature/Hotfix flows. Live in prod on the Dev Sprints folder.
 
 ## Summary
 
 The ClickUp-driven review + promotion chain for the platform-wide QA/QC dev process (per the approved
 `QA-QC Plan.txt`), moving a task through **Development → QA → QC → (Client Review) → Stage →
-Production**. Several **new** workers plus entry points in the shared ClickUp webhook handler. **Not
-deployed yet** — on the `feature-qa-qc` branch working tree.
+Production**. Several workers plus entry points in the shared ClickUp webhook handler. **Live in
+production** (deployed on `_production`).
 
 - `_Worker_Clickup_QaReview` — the UI → FE → BE review chain (branches by Task Type; Hotfix BE-approve
   jumps straight to QC-Hotfix).
@@ -47,9 +47,10 @@ deployed yet** — on the `feature-qa-qc` branch working tree.
 - `_Worker_Clickup_BeReview` — **auto** BE-approve driven by GitHub PR reviews (removes the manual
   BE-review step).
 
-**Scope gate — only fires in one ClickUp folder.** The whole automation is gated to the dedicated
-**"(TEST) True Dev Sprints"** folder (id `90116585845`, list `901111612373`) via
-`_Worker_Clickup::QA_QC_FOLDER_ID` + `isTaskInQaQcScope`, so it never touches live sprints.
+**Scope gate — Dev Sprints folder only.** `_Worker_Clickup::QA_QC_FOLDER_IDS` = `['90116585845']`
+(**Dev Sprints**, the live sprint folder; formerly named "(TEST) True Dev Sprints"). The retired True
+Dev Sprints folder `90020432579` was removed 2026-09-25. `isTaskInQaQcScope` matches the home folder
+or any list the task is in (Tasks-in-Multiple-Lists).
 
 When a review is **denied**, the task is routed to **Rework** and its code changes are **reverted** from
 the QA/QC branch — see [QA/QC branch automation](./qa-qc-branch-automation.md) for how the revert
@@ -138,13 +139,17 @@ when a task *left* a status; the QA/QC entry must be on entering the status.
 - **The unit-test gate is a stub (`runUnitTests()` returns true).** It does not block anything yet.
 - **Custom-field option VALUES are resolved at runtime, IDs are constants** — see
   [ClickUp custom-field conventions](./clickup-custom-field-conventions.md).
-- **The automation only fires inside the "(TEST) True Dev Sprints" folder.** `isTaskInQaQcScope` gates
-  the fan-out, `handleDevelopmentFinished`, and `QcBatch`. A task outside that folder is ignored —
-  verify the folder before expecting the flow to run.
+- **The automation only fires inside `QA_QC_FOLDER_IDS` (Dev Sprints).** `isTaskInQaQcScope` gates
+  the fan-out, `handleDevelopmentFinished`, `QcBatch`, and the taskCreated Work Type default. A task
+  outside that folder is ignored — verify the folder before expecting the flow to run.
+- **Dev Finished with neither Has UI Changes nor Has Code Changes checked → `qa approved` (skips
+  review).** Intended (jcardinal confirmed); do not "fix".
 - **A Feature advances only when ALL subtasks clear a gate** — a single lagging subtask holds the whole
   feature. Subtasks carry no code; only the feature branch does.
-- **`_underscore CLICK_UP_CUSTOM_FIELD_ID__CLIENT_REVIEW` was repointed** to the "(TEST) True Dev
-  Sprints" folder's field (`2c4ab51a-…`); all field ids were verified against that folder.
+- **`_underscore CLICK_UP_CUSTOM_FIELD_ID__CLIENT_REVIEW` was repointed** to the Dev Sprints
+  folder's field (`2c4ab51a-…`); all field ids were verified against that folder.
+- **The automation user needs edit rights on every field it writes** — a missing field permission
+  returns HTTP 401 `ACCESS_606`. See [credential rotation](../workflows/clickup-credential-rotation.md).
 
 ### Known security gaps (discovered, not yet fixed — separate tickets)
 

package/knowledge/2.0/apps/worker2/features/team-sprint-management.md

@@ -6,7 +6,7 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-09-21
+updated: 2026-09-25
 owners: ["jcardinal", "kyalamarthi"]
 files:
   - worker2/Worker/Team/Sprint.php
@@ -355,8 +355,12 @@ see [TOGa IQ Sprint Dashboard API](../../api2/features/sprint-dashboard-api.md)
 - **`SprintLock` is NOT a safe way to repopulate empty tables.** It captures before it reads, so it
   looks like a bootstrap (its `getSprint()` call is gated behind `$generateSprintLockReport`), but
   its capture block (~L4016-4030) **writes back to ClickUp for every task** — sets Previous Sprint
-  Work Type, sets "Added at sprint launch" = true, sets Condition = "On Track". Run outside the real
-  launch ceremony it corrupts those fields across the whole sprint.
+  Work Type and sets "Added at sprint launch" = true. Run outside the real launch ceremony it
+  corrupts those fields across the whole sprint.
+- **The ClickUp "Condition" field (On Track) is retired (2026-09-25).** Not in the QA/QC plan, not in
+  the Dev Sprints folder, never read by code. The `SprintLock` write was removed (it would fail on the
+  new folder). `CLICK_UP_CUSTOM_FIELD_ID__CONDITION` / `CONDITION__ON_TRACK` still exist in
+  `_underscore` but are unused — do not add new writes.
 - **`SprintDaily` returns nothing on success** — a production run reads `Successfully Executed
   (4.255 seconds): <no return message>`, so a run that worked is indistinguishable from one that
   never fired. Breaks the return-on-both-paths rule in the

package/knowledge/2.0/apps/worker2/workflows/clickup-credential-rotation.md

@@ -6,13 +6,12 @@ project: Worker
 client: shared
 type: workflow
 status: active
-updated: 2026-09-16
-owners: ["ajean"]
+updated: 2026-09-25
+owners: ["ajean", "jcardinal"]
 files:
   - _underscore/Component/Api/Clickup/Clickup.php
   - worker2/Worker/Team/Sprint.php
   - worker2/Worker/Clickup/Project.php
-  - library/app/systemmonitor/500error.php
   - library/app/systemmonitor/netsuiteintegration.php
   - library/app/systemmonitor/emailspam.php
 related:
@@ -50,7 +49,6 @@ breaking the legacy 1.0 crons**. Two facts make this a procedure rather than a c
 | `worker2` | `Worker/Team/Sprint.php` |
 | `worker2` | `Config/production.ini` → `[clickup] token` |
 | `worker2` | a dev `.ini` (not committed) |
-| `library` | `app/systemmonitor/500error.php` |
 | `library` | `app/systemmonitor/netsuiteintegration.php` |
 
 **Token B — a second, OLDER token.** Easy to miss, and the reason a "complete" rotation still breaks
@@ -109,12 +107,20 @@ is a live correctness bug in the actor guard, independent of any rotation.
    `'12633140'` to `'87469444'`. See the correctness note above — this step is not cosmetic.
 7. Replace the token in every Token A location, and Token B in `library/app/systemmonitor/emailspam.php`
    plus the three `worker/crons/infrastructure/*` crons.
-8. Deploy worker2, and deploy/verify the 1.0 `library` + `worker` crons.
-9. Only now, with the **original** token, delete the old webhooks recorded in step 2.
-10. Revoke the old token(s).
+8. In ClickUp, give the new account **edit rights on every custom field the automation writes**
+   (Work Type, Previous Sprint Work Type, Environment, Final Design Outcome, the QA/QC review fields,
+   and so on). Space/list access is not enough — fields have their own permissions.
+9. Deploy worker2, and deploy/verify the 1.0 `library` + `worker` crons.
+10. Only now, with the **original** token, delete the old webhooks recorded in step 2.
+11. Revoke the old token(s).
 
 ## Gotchas
 
+- **Field-level edit permission is separate from space/list access.** After the automation moved to
+  user 87469444 (2026-09-21), it could not edit Previous Sprint Work Type / Environment → HTTP 401
+  `[ACCESS_606] You do not have permission to do this action` → every `taskCreated` job failed for 4
+  days. Check field permissions for every field the automation writes whenever the account changes.
+  Code side: [work type automation](../features/clickup-work-type-automation.md) (`setFieldSafely`).
 - **The hardcoded token in `_underscore/Component/Api/Clickup/Clickup.php` is in committed git
   history.** Rotating the value does not remove it from history — treat the old token as compromised and
   revoke it, do not merely stop using it.

package/knowledge/sessions/2026-09-25-TRUE_80519-mhammontree.md

@@ -0,0 +1,112 @@
+---
+type: session
+slug: TRUE_80519
+title: NYCDOE 2.0 ASN Phase 2 parsing + worker2 Codeception pilot
+author: mhammontree
+repos: [worker2, _underscore, dbchanges2]
+framework: "2.0"
+client: nycdoe
+status: active
+created: 2026-09-25
+updated: 2026-09-25
+---
+
+# Session: TRUE_80519
+**Date:** 2026-09-25
+**Project/Repo:** worker2, _underscore, dbchanges2 (2.0)
+**Task:** Build and verify Phase 2 (ASN file parsing → Case/ServiceRequest creation) of the NYCDOE ServiceNow/ASN/NetSuite 2.0 rewrite (TRUE-80519), then set up worker2 as 2.0's first Codeception/PHPUnit test pilot and codify the Phase 2 verification as regression tests.
+
+---
+
+## What WORKED
+
+- **Phase 2 parsing complete for all 4 manufacturers**, each verified end-to-end via manual Postman testing against real sample files, then codified as automated tests:
+  - Lenovo: real file → 66 Cases, 746 ServiceRequests; re-run confirmed dedupe (`serviceRequestsCreated: 0`).
+  - Lexmark: real file → 4 Cases, 14 ServiceRequests; dedupe confirmed.
+  - Apple: real file → 56 Cases, 402 ServiceRequests; dedupe confirmed.
+  - Acer: real file → 12 Cases, 1145 ServiceRequests; dedupe confirmed.
+  - `Worker/Sftp/Parse.php` implements one `_Worker_Sftp_Parse::<Manufacturer>()` static method per manufacturer, header-validated against a hardcoded expected-column-names constant (LENOVO_HEADER, LEXMARK_HEADER, ACER_HEADER, APPLE_HEADER), rejecting the whole file loudly on any mismatch.
+- **Case grain resolved**: one `Case` per customer PO (via new `c_customerPurchaseOrder` field), not one per file — confirmed from real sample data containing multiple POs per file.
+- **NetSuite linkage design corrected** per Rohan: SO/PO header stamp belongs on `SalesOrders`/`PurchaseOrders` (`SalesOrders.serviceRequestId`, `SalesOrders_PurchaseOrders`, `SalesOrderItems.quantity`/`.qtyCommitted`), not as `Case` custom fields as originally assumed. This narrowed Phase 2's own `c_` field list to RITM linkage + `filesId` provenance only.
+- **`_Model_Nycdoe_Case` / `_Model_Nycdoe_ServiceRequest`** (`_underscore/Model/Nycdoe/`) — client model overrides declaring the `c_` fields as real properties, required for direct `_Model` writes (same constraint as Phase 1's Files model).
+- **dbchanges2 migrations applied and confirmed working**: `ServiceRequestStages` seeded (New/Scheduled/On Hold/Delivered/Cancelled), `ServiceRequests` gained `c_partNumber`/`c_serialNumber`/`c_ritmNumber`/`c_filesId`, `Cases` gained `c_customerPurchaseOrder` and the missing `longDescription` column (schema/model drift fix).
+- **Codeception 5 / PHPUnit 11 set up in worker2** as 2.0's test pilot, CTO-reviewed before building (AGREE + 5 required changes, all applied) and php-reviewer-reviewed after building (2 real bugs found and fixed — see below). Dev-only via `composer require-dev`; confirmed `composer.lock`'s non-dev `packages` block is byte-identical before/after, and `--no-dev --dry-run` cleanly removes every test package (matches worker2's real deploy hook `composer install --no-dev`).
+- **`tests/Integration/ParseCest.php`** — 6 tests (one per manufacturer + header-mismatch rejection + Lenovo comma-repair regression), all passing, confirmed repeatable across multiple runs, confirmed clean fixture teardown via a separate verification script.
+- **Two hard safety gates** in `tests/Support/Helper/Integration.php`, both run before any write: refuses to run unless resolved `ENVIRONMENT` is `dev-*` (read from gitignored `tests/local.env.php`, never hardcoded); refuses to run against a non-local `Client_Nycdoe` database host. Verified both actually fire (temporarily pointed `local.env.php` at `'production'`, confirmed the suite refused to run, then restored it).
+- **Lenovo comma-in-description bug fixed and tested**: ported 1.0's exact fix (`library/app/asnprocessor/lenovo.php`) for Lenovo's `Product Long Description` column sometimes containing an unquoted, unescaped comma that shifts every later column. Added `repairLenovoRowColumnCount()`. Regression test (`lenovoRepairsAnUnescapedCommaInTheDescription`) — confirmed it actually fails without the fix (serial number lookup breaks once columns shift) before confirming it passes with the fix.
+- **php-reviewer caught 2 real bugs in the test harness itself, both fixed and re-verified**:
+  1. `commitDatabaseTransaction()` was a no-op — `_Database::transactionCommit(null)` only acts when the global `-1` flag is set, which is only set by the parameterless `_Database::transactionStart()` (called by `_Route` in production, never by our test). Fixed by calling `_Database::transactionStart()` in `_before()`. This also proved an earlier workaround (`_Database::setIsReadHostEnabled(false)`) was unnecessary once real commits happened — removed it after 3 clean re-runs confirmed tests still pass without it.
+  2. Unnecessary `ReflectionClass`/`setAccessible` into `_Database::$_registers`, which is actually a `public static` property — simplified to direct access.
+- **Everything captured and published to the team knowledge base** (commit `146c68d7` on `origin/_main`): 9 docs written (2 new: `clients/nycdoe/features/asn-sftp-ingestion-2-0.md`, `2.0/apps/worker2/features/codeception-integration-tests.md`; 5 updated: `servicenow-integration.md`, `cli-script-bootstrap.md`, `model-magic-field-access.md`, `item-classification-itemclasses.md`, nycdoe `profile.md`; 2 elevated standards updated with developer approval: `framework-rules.md`, `backend-testing.md`).
+
+## What did NOT work — DO NOT RETRY THESE
+
+- **Assuming Lenovo's file layout matched 1.0's stale `asnprocessor/lenovo.php` code (47 columns)** — wrong. The real, current Lenovo file is a genuinely different 30-column layout, confirmed against 3 real live sample files. The header-name validation correctly rejected the file when tested against the wrong assumed header — this was the design working as intended, not a bug, but the underlying `LENOVO_HEADER` constant itself had to be rewritten from scratch based on real file content, not from reading old code.
+- **A "stale FIELD_STORAGE file due to id reuse" theory** — raised mid-investigation of the Lenovo header mismatch (suspecting local folder-mode storage served old content for a reused row id). Proven wrong: a fresh row showed the same "wrong" header, which turned out to be the real, current file format all along. Retracted for the record so it isn't mistaken for a real finding later.
+- **Assuming Lexmark's `Order_Date` was `M/D/YYYY`** — wrong; it's `YYYY/MM/DD`. Caught by re-reading the real sample file before testing (direct lesson from the Lenovo mistake above), not by a failed test in production.
+- **Assuming Acer/Lexmark share identical header column names** (only positions) — wrong; 3 column names differ (`First_Name_Shipping`/`Last_Name_Shipping`/`SerialNo` for Acer vs `Contact_Name_Shipping`/`Email_Shipping`/`Svc_Tag_ID` for Lexmark). A single shared `LEXMARK_ACER_HEADER` constant would have rejected every real Acer file. Split into `LEXMARK_HEADER`/`ACER_HEADER`.
+- **Assuming a 2-digit year never appears in Apple's `order_date`** — wrong; a newer sample file had `10/23/25`, which the original `parseDate()` would have turned into year `0025`. Fixed with `expandYear()`.
+- **Running `vendor/bin/codecept bootstrap` fully non-interactively** — it stopped and waited on an interactive `(y/n)` prompt ("composer.json updated. Do you want to run 'composer update'?") that the Bash tool couldn't answer; had to `TaskStop` the backgrounded process and finish the setup manually (`generate:suite`, manual `composer update` with the lock-diff safety check) instead of relying on the bootstrap wizard end-to-end.
+- **The original `commitDatabaseTransaction()`/`setIsReadHostEnabled(false)` combination** — see "What WORKED" above for the real root cause and fix; the original diagnosis (separate read connection can't see an uncommitted write) was directionally right but incomplete — the actual missing piece was the global `transactionStart()` call, not just connection routing.
+
+## Not tried yet (candidates for next session)
+
+- **Acer/Lenovo/Lexmark date-format edge cases beyond what real samples showed** — Acer's `YYYY/MM/DD` format is now confirmed (not just guessed), but only against one sample file; worth reconfirming if a 2-digit-year Acer file ever surfaces (same class of bug already found for Apple).
+- **Re-confirm the Lenovo comma-in-description fix against a real occurrence** — no real sample file with this exact issue was available this session; the fix mirrors 1.0's proven logic exactly but has not been tested against real Lenovo data that actually has the bug.
+- **Deploy/rollout planning for Phase 1 + 2** (discussed conceptually, not started):
+  - Confirm whether Phase 1's SFTP fetch is read-only / non-destructive relative to what 1.0 already consumes from the same SFTP (must not interfere with 1.0's live pipeline).
+  - Decide a "shadow mode" plan: run 2.0 ingestion+parsing against the same real files 1.0 processes, writing into the isolated `Client_Nycdoe` 2.0 tables, purely for validation — low risk since Phase 3 (NetSuite/ServiceNow outbound calls) doesn't exist yet, so nothing can double-fire.
+  - Decide what "shadow mode is good enough to cut over" means concretely (e.g., compare 2.0 Case/ServiceRequest counts per manufacturer per day against 1.0's `AdvanceShippingNotices` counts) — no comparison method defined yet.
+  - No monitoring/alerting exists yet for Phase 1/2 failures (header rejections, comma-repair firing, etc.) — needs a decision on whether/how to alert (Logs.Issue entry? tie into Mark's separate Monitoring System Design project?).
+  - Branch/PR/deploy process not started: needs `php-reviewer` + `cso` on the diff, Rohan's deploy window for `_underscore`/`dbchanges2` changes, and Mark's own pseudocode/proposal to Jeff (Claude's output is reference material only — Mark authors the actual submission).
+- **Follow-up with Rohan**: whether other clients' `Cases` tables are also missing the `longDescription` column (found only for `Client_Nycdoe` this session).
+- **Low-priority housekeeping**: stale `DOE-OPEN-LENOVO-*.CSV` files still sit on the real Lenovo SFTP and aren't cleaned up by anything (per Skyler/Eric, safe to delete, retired process) — not blocking, not scheduled.
+- **INC (Incident) ticket flow** — a separate flow from the ASN/RITM pipeline this session covered, not yet redesigned for 2.0 (flagged in the main plan as "next session" from an earlier date, still open).
+- **Phase 3** (NetSuite Sales Order/PO dispatch, ServiceNow RITM linkage, outbound sync) — explicitly out of scope for Phase 2, not started.
+
+## Current file state
+
+| File | Status | Notes |
+|------|--------|-------|
+| `worker2/Worker/Sftp/Parse.php` | Modified, uncommitted | Phase 2 parsing for all 4 manufacturers + Lenovo comma-repair fix. Lint-clean, all tests pass. |
+| `worker2/tests/Support/Helper/Integration.php` | New, uncommitted | Codeception module: framework boot, dev/local safety gates, fixture + transaction helpers. |
+| `worker2/tests/Integration/ParseCest.php` | New, uncommitted | 6 integration tests, all passing. |
+| `worker2/tests/local.env.php.example` | New, uncommitted | Template for the gitignored per-developer env file. |
+| `worker2/tests/local.env.php` | New, gitignored (not committed by design) | This machine's value: `dev-markhammontree-laptop`. |
+| `worker2/codeception.yml`, `worker2/tests/Integration.suite.yml` | New, uncommitted | Codeception config. |
+| `worker2/.gitattributes` | New, uncommitted | `export-ignore` for `tests/` and `codeception.yml`. |
+| `worker2/.htaccess` | Modified, uncommitted | Added deny rules for `tests/` and `*.yml`. |
+| `worker2/.gitignore` | Modified, uncommitted | Added `/tests/_output/`, `/tests/Support/_generated/`, `/tests/local.env.php`. |
+| `worker2/composer.json`, `worker2/composer.lock` | Modified, uncommitted | Added `codeception/codeception:^5`, `phpunit/phpunit:^11`, `codeception/module-asserts` as `require-dev` only. Confirmed non-dev `packages` block unchanged. |
+| `_underscore/Model/Nycdoe/Case.php` | New, uncommitted | Declares `c_customerPurchaseOrder`. |
+| `_underscore/Model/Nycdoe/ServiceRequest.php` | New, uncommitted | Declares `c_partNumber`, `c_serialNumber`, `c_ritmNumber`, `c_filesId`. |
+| `dbchanges2/Client_Nycdoe/2026-09-22a - Seed ServiceRequestStages for DOE installs.sql` | New, uncommitted | Confirmed applied to local dev DB. |
+| `dbchanges2/Client_Nycdoe/2026-09-22b - Add ASN parsing fields to ServiceRequests.sql` | New, uncommitted | Confirmed applied. |
+| `dbchanges2/Client_Nycdoe/2026-09-22c - Add customerPurchaseOrder field to Cases.sql` | New, uncommitted | Confirmed applied. |
+| `dbchanges2/Client_Nycdoe/2026-09-25a - Add missing longDescription column to Cases.sql` | New, uncommitted | Confirmed applied; fixes real schema/model drift against shared `_Model_Client_Case`. |
+| `C:\Users\mhammontree\.claude\plans\phase-2-asn-parsing-plan.md` | Continuously updated throughout | Full change history of every finding this session, kept current. |
+| Team KB (`~/toga-tech/knowledge/`) | Published | Commit `146c68d7` on `origin/_main`. 9 docs (2 new, 7 updated), see "What WORKED". |
+
+**None of the code changes above have been committed or pushed to their respective repos yet** — draft commit titles/descriptions were prepared for worker2 (2 commits), `_underscore` (1 commit), and `dbchanges2` (1 commit), but no `git commit` has been run per the standing "Claude never commits without explicit ask" rule.
+
+## Decisions made
+
+- **One `Case` per customer PO, not per file.** Rationale: real sample data (Lexmark, Apple) showed multiple distinct customer POs in a single file, so "one Case per file" was never workable. No alternative seriously considered once real data was checked.
+- **NetSuite SO/PO stamp lives on `SalesOrders`/`PurchaseOrders`, not as `Case` custom fields.** Rationale: Rohan's direct correction, verified against real schema. Rejected: the original assumption from the main plan's earlier design session.
+- **Lenovo kept as its own header/parse path, never shared with Lexmark/Acer**, even though both `Lenovo()` and `runLexmarkAcerFamily()` looked similar early on. Rationale (Mark's explicit instruction): Lenovo is not actually a CDW supplier (only Acer/Lexmark are); sharing code risks a future Lenovo-only format change breaking two unrelated suppliers. Rejected: a single shared parsing function keyed by manufacturer name.
+- **`DOE-OPEN-LENOVO` files are retired and must never be parsed**, confirmed by Skyler/Eric. Rationale: an old, retired process Lenovo still sends to the SFTP; Phase 1's existing filename pattern already excludes them, so no code change was needed, just confirmation.
+- **Lenovo's comma-in-description bug is repaired, not rejected** (unlike a genuine header-shape mismatch, which stays fatal). Rationale: Mark's direct confirmation that, per 1.0's history, this is the ONLY field Lenovo ever breaks this way — a well-understood, narrow quirk worth auto-fixing rather than bouncing every affected file for manual resend.
+- **2.0 test framework: Codeception 5 + PHPUnit 11, `worker2` as the sole pilot repo.** Rationale: Jeff's direction; CTO-reviewed first given it's a framework-adoption decision. Explicitly rejected: copying 1.0's `api` repo pattern of running tests against production on every deploy via an `.ebextensions` postdeploy hook — tests stay a manual/local step here, never a deploy-time gate, unless explicitly changed later.
+- **`integration` suite naming, not `unit`/`functional`.** Rationale (from the CTO review): `Parse.php` needs a real DB, so it isn't a pure unit test; Codeception reserves "functional" for framework-module suites (Symfony/Laravel-style), which doesn't apply here — `integration` avoids both misnomers.
+- **Test-only workarounds must match production's real transaction/connection behavior, not just "make the test pass."** Concretely: once `php-reviewer` proved the transaction-commit workaround was a no-op, the correct fix was to make it a REAL commit (matching what `_Route` does), not to just silence the symptom differently. This also let an earlier workaround (`setIsReadHostEnabled(false)`) be removed as unneeded once the real fix was in — verified empirically (3 clean re-runs without it) rather than assumed.
+
+## Blockers
+
+None for continuing Phase 2 work itself. For **rollout** (the open question this session ended on): no blocker yet identified, but the next session needs to (1) confirm Phase 1's SFTP read is non-destructive to 1.0's own consumption, and (2) decide the shadow-mode comparison method, before recommending an actual rollout plan to Mark for his own submission to Jeff.
+
+## Exact next step
+
+> Start the next session by reading `C:\Users\mhammontree\.claude\plans\phase-2-asn-parsing-plan.md` in full (it has the complete, current design + change history), then investigate whether `worker2/Worker/Sftp/Import.php`'s SFTP fetch is read-only/non-destructive relative to 1.0's own consumption of the same DOE SFTP — this is the first concrete fact needed before proposing a "shadow mode" parallel-run plan to Mark.
+
+---
+_Saved by /session-save on 2026-09-25_

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.893",
+  "version": "1.0.895",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",