toga-ai 1.0.892 → 1.0.893

package/knowledge/2.0/apps/_underscore/features/cli-script-bootstrap.md

@@ -6,8 +6,8 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: ["bala"]
+updated: 2026-09-25
+owners: ["bala", "mhammontree"]
 files:
   - _underscore/Database.php
   - _underscore/Environment.php
@@ -53,6 +53,9 @@ For DB-**free** unit tests of interceptor logic, use the PHPUnit scaffold instea
   raised. This is the CLI face of the lazy-transaction behavior documented on
   `_underscore/architecture.md` — `_Database::register()` starts a lazy transaction that MySQL rolls
   back when the connection closes uncommitted.
+- **Keep the CWD at the app root for the whole run, not just the `require`.** `Loader::autoloader`
+  resolves project classes (e.g. `_Worker_Sftp_Parse`) against `./` lazily at first use, so a
+  harness that `chdir`s back after booting breaks autoloading on the first call into app code.
 - **`ENVIRONMENT` must be set before the framework loads**, not after — `_Environment::initialize`
   fatals on a missing value rather than defaulting.
 - **Local MySQL and PROD both run `sql_mode = ''`**, so `ONLY_FULL_GROUP_BY` is **not** enforced in
@@ -74,3 +77,6 @@ For DB-**free** unit tests of interceptor logic, use the PHPUnit scaffold instea
 - [Client Email Template Sending](./email-template-sending.md) — its local test-harness section.
 - [French (fr-CA) Order Email Localization (Compass Canada)](../../../clients/compass-canada/features/french-order-email-localization.md)
   — the work these traps surfaced in.
+
+## Change history
+- 2026-09-25 — Added lazy-CWD autoloader gotcha (found building the worker2 Codeception harness) (mhammontree)

package/knowledge/2.0/apps/_underscore/features/item-classification-itemclasses.md

@@ -6,8 +6,8 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
-owners: ["rgirish"]
+updated: 2026-09-25
+owners: ["rgirish", "mhammontree"]
 files:
   - _underscore/Model/Client/ItemClass.php
   - _underscore/Model/Client/Item.php
@@ -120,7 +120,9 @@ Queried live via SuiteQL (2026-09-11): **311 classifications**, columns
 - **A hierarchy path column must be `TEXT`, not `varchar(512)`.** NetSuite `fullname` values are
   long, and the 2.0 standard caps varchar at 255.
 - **The table is already `utf8mb4_0900_ai_ci`**, so new columns need no explicit `COLLATE`.
-- **`c_` custom fields are NOT declared as properties on the 2.0 Model class.** The framework
+- **`c_` custom fields are NOT declared as properties on the 2.0 Model class** when written
+  through the /v2 API (a direct `_Model` write does need a declaration — see
+  [_Model magic-field access](model-magic-field-access.md)). The API engine
   reads them from `CustomRecordFields` (`ItemClasses` is `Core.Records` id **105**). Confirmed:
   no `public $c_netsuite*` property exists on any `Model/Client` class. Copy the pattern in
   `dbchanges2/_modules/netsuite/2026-08-18 - TransferOrdersNetsuiteStatus.sql`.
@@ -136,3 +138,6 @@ Queried live via SuiteQL (2026-09-11): **311 classifications**, columns
 - [Prudential ItemClasses device-tier data](../../../../clients/prudential/features/itemclasses-device-tier-data.md)
 - [Client schema-drift audit](../../dbchanges2/workflows/client-schema-drift-audit.md)
 - [NetSuite item-class sync (1.0)](../../../../1.0/apps/library/features/netsuite-item-class-sync.md)
+
+## Change history
+- 2026-09-25 — Scoped the "c_ not declared" note to /v2 API writes (mhammontree)

package/knowledge/2.0/apps/_underscore/features/model-magic-field-access.md

@@ -6,7 +6,7 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-09-16
+updated: 2026-09-25
 owners: ["jcardinal", "mhammontree", "apeterson"]
 files:
   - _underscore/Model/Core/Model.php
@@ -79,9 +79,11 @@ class does not declare. Two things follow immediately, and they cut the search s
 
 1. **It is never a schema problem.** No SQL ran. Do not check `information_schema`, do not suspect a
    missing migration on *that* column — check the model class.
-2. **The column is never `c_`-prefixed.** Custom `c_` columns are framework-dynamic and need no
-   declaration, so this error can only ever fire for a **standard** column. The fix is binary:
-   declare the property on the model, or stop reading it.
+2. **It can be a `c_` column too.** The /v2 API engine handles `c_` columns dynamically, but
+   `_Model` itself does not: a direct model read/write (`__get`/`__set`) of an undeclared `c_`
+   field throws this same message. The fix is binary: declare the property on the model (a
+   client override such as `_Model_Nycdoe_ServiceRequest`, or a `_Trait_Netsuite_*`), or stop
+   using it.
 
 **The trap that produces it: two similarly-named models, one of which lacks the column.** Swapping
 one for another at a call site **lints clean and only dies at runtime** — nothing in PHP or CI
@@ -135,3 +137,6 @@ wrong: **1054** = the database is behind the model (see
 - [Surface FIELD ↔ RecordField binding](./surface-field-recordfield-binding.md)
 - [Framework branch ahead of schema](../../dbchanges2/workflows/framework-branch-ahead-of-schema.md)
 - [Rate subscription cancellation](../../../../clients/rate/features/subscription-cancellation.md)
+
+## Change history
+- 2026-09-25 — Corrected: undeclared `c_` fields also throw on direct `_Model` access; dynamic `c_` handling is the /v2 API engine only (mhammontree)

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -19,6 +19,7 @@
 | [ClickUp Task Description Fluffer (Talos KB Investigation)](features/clickup-task-fluffer-talos.md) | Async worker that "fluffs" a new ClickUp task by running its name+description through the Talos `dev-core` KB agent and writing the report back; open for the fl |
 | [ClickUp Webhook Health Monitor → OneUptime (Monitor/Clickup/CheckWebhook)](features/clickup-webhook-health-monitor.md) | worker2 cron that checks the team's ClickUp webhook subscriptions and pages via a OneUptime Incoming-Request monitor; open when tuning the ClickUp webhook alarm |
 | [ClickUp Work Type Automation (Committed / Conditional / Stretch)](features/clickup-work-type-automation.md) | Auto-maintains each ClickUp task's Work Type (Committed/Conditional/Stretch) from its dependencies, status, and due date; open when work-type promotion/demotion |
+| [worker2 Codeception integration tests (2.0 test-framework pilot)](features/codeception-integration-tests.md) | First 2.0 app-level automated test suite: Codeception 5 / PHPUnit 11 in worker2 (pilot, Jeff's direction), real-DB integration tests with hard local-only safety |
 | [worker2 — Committed Secrets & Config Exposure (debt register)](features/committed-secrets-and-config-debt.md) | Standing list of credentials committed to the worker2 repo (locations only); open before adding a credential to worker2 or assuming a config value is private. |
 | [Compass VIP Support Importer (worker2)](features/compass-vip-support-importer.md) | worker2 action that ingests Compass's quarterly VIP spreadsheet and sets each VIP's support tech (`Users.c_supportedByUserId` in `Client_Compass`); open when wo |
 | [Creating Worker Actions](features/creating-worker-actions.md) | How to add a new callable Worker action — a PHP class whose `public static` methods are invoked as background jobs (via webhook, cron, or `_Worker::runTask()`). |

package/knowledge/2.0/apps/worker2/features/codeception-integration-tests.md

@@ -0,0 +1,51 @@
+---
+title: worker2 Codeception integration tests (2.0 test-framework pilot)
+framework: "2.0"
+repo: worker2
+project: Worker
+client: shared
+type: feature
+status: active
+updated: 2026-09-25
+owners: [mhammontree]
+files:
+  - worker2/codeception.yml
+  - worker2/tests/Integration.suite.yml
+  - worker2/tests/Support/Helper/Integration.php
+  - worker2/tests/Integration/ParseCest.php
+  - worker2/tests/local.env.php.example
+  - worker2/composer.json
+  - worker2/.gitattributes
+  - worker2/.htaccess
+related:
+  - ../../../standards/backend-testing.md
+  - ../../_underscore/features/cli-script-bootstrap.md
+  - ../../_underscore/features/model-interceptor-unit-testing.md
+  - ../../../../clients/nycdoe/features/asn-sftp-ingestion-2-0.md
+---
+
+First 2.0 app-level automated test suite: Codeception 5 / PHPUnit 11 in worker2 (pilot, Jeff's direction), real-DB integration tests with hard local-only safety gates; open before adding or running worker2 tests.
+
+## Summary
+
+worker2 is the **pilot repo** for a 2.0 test framework. One suite, `Integration` (not `unit`/`functional` — the code under test needs a real DB), boots the real `_underscore` framework and calls worker actions in-process against a **local** client DB. First coverage: `Sftp/Parse` (6 tests). **Tests are a manual/local step — never a deploy-time gate.** 1.0's pattern (`codecept.phar` + `.ebextensions/040_run_tests.config` running tests against production on every deploy) was deliberately NOT copied.
+
+## How it works
+
+1. **Dev-only install.** Packages are `require-dev` only (`codeception/codeception ^5`, `phpunit/phpunit ^11`, `codeception/module-asserts`). The deploy hook `.platform/hooks/postdeploy/015_install_composer.sh` runs `composer install --no-dev`, so nothing ships; the non-dev block of `composer.lock` was byte-identical before/after.
+2. **Never deployed.** `.gitattributes` `export-ignore`s `/tests` + `/codeception.yml`; `.htaccess` denies `^tests/`.
+3. **Per-dev environment.** Copy `tests/local.env.php.example` → `tests/local.env.php` (gitignored) returning your Apache `SetEnv ENVIRONMENT` value (e.g. `dev-<name>-laptop`). Or export `ENVIRONMENT`.
+4. **Run** from the worker2 root with PHP 8: `vendor/bin/codecept run Integration`.
+5. **Safety gates (run before any write):** helper refuses unless `ENVIRONMENT` starts `dev-`; `assertUsingLocalClientDatabaseOnly($clientId)` refuses unless the registered `DB_CLIENT` `hostname`/`writeHost`/`readHost` are all `127.0.0.1`/`localhost`/`::1`. Call it before every action call.
+6. **Boot + transaction.** `_before()` boots once (same two lines as `index.php`) then calls parameterless `_Database::transactionStart()` — the call `_Route` makes for a real request. `commitDatabaseTransaction()` then really commits.
+7. **Fixtures** mirror production field-by-field (`createFileFixture` mirrors `Import.php` `processFile()`); each test cleans up its own rows.
+
+## Gotchas
+
+- **Calling a worker action directly bypasses `_Route`, so there is no open transaction** — without the explicit `_Database::transactionStart()` the commit is a silent no-op and reads miss writes. Do not work around the symptom (e.g. disabling read-host connections); start the transaction. See [CLI script bootstrap](../../_underscore/features/cli-script-bootstrap.md).
+- **Do not `chdir` back after booting.** `Loader::autoloader` resolves project classes against the CWD lazily at first use, so restoring the old CWD breaks autoloading of `_Worker_*` classes mid-test.
+- **`_Database::$_registers` is public static** — read it directly; no Reflection.
+- **Prove a regression test fails without the fix** before trusting it (done for the Lenovo comma repair: serial mismatch once columns shift).
+
+## Change history
+- 2026-09-25 — Created: Codeception/PHPUnit pilot, Integration suite for Sftp/Parse; cto-reviewed design, php-reviewer fixes to the harness (mhammontree)

package/knowledge/2.0/standards/backend-testing.md

@@ -1,11 +1,11 @@
 ---
-title: 2.0 Backend Testing (no PHPUnit — ad-hoc client-pinned scripts)
+title: 2.0 Backend Testing (ad-hoc client-pinned scripts; worker2 Codeception pilot)
 framework: "2.0"
 project: _Underscore
 client: shared
 type: standard
 status: active
-updated: 2026-09-16
+updated: 2026-09-25
 owners: [mhammontree]
 files:
   - _underscore/Query.php
@@ -14,17 +14,19 @@ files:
   - _underscore/Model.php
   - test/@Mark/Rate/verify_cancel_subscription.php
   - test/@Mark/Rate/verify_aig_contract_identifier_persistence.php
+  - worker2/tests/Support/Helper/Integration.php
 related:
   - ../apps/_underscore/architecture.md
   - ../apps/_underscore/features/per-client-database-connections.md
   - ../apps/test/architecture.md
+  - ../apps/worker2/features/codeception-integration-tests.md
 ---
 
-How to verify 2.0 backend behavior with no PHPUnit — ad-hoc `test`-repo scripts that pin a real client DB; open before writing any 2.0 backend verification script.
+How to verify 2.0 backend behavior — ad-hoc `test`-repo scripts that pin a real client DB (everywhere), or the worker2 Codeception pilot suite; open before writing any 2.0 backend verification.
 
 ## Summary
 
-The 2.0 backend (`_underscore` + its apps) has **no PHPUnit harness**. Verify backend behavior with **ad-hoc scripts in the `test` repo** (per-dev folders, e.g. `@Mark`) that bootstrap `_underscore` from a 2.0 app and pin a real client database explicitly. This is the accepted verification path until/unless a formal harness exists — do not assume a `phpunit` suite.
+The 2.0 backend has **no shared PHPUnit harness** — the only automated suite is the **worker2 Codeception/PHPUnit pilot** ([codeception-integration-tests](../apps/worker2/features/codeception-integration-tests.md)), local-only and never a deploy gate. Everywhere else, verify backend behavior with **ad-hoc scripts in the `test` repo** (per-dev folders, e.g. `@Mark`) that bootstrap `_underscore` from a 2.0 app and pin a real client database explicitly.
 
 ## Bootstrapping a verification script
 
@@ -131,7 +133,7 @@ Also design around what a direct-call script **cannot** see: calling a model met
 
 ## Rules
 
-- Do not add or assume a PHPUnit suite for 2.0 backend work; write a `test`-repo script instead.
+- Outside worker2, do not add or assume a PHPUnit suite; write a `test`-repo script. In worker2, add tests to the Codeception `Integration` suite (local-only; never wire it into deploy).
 - Always set `ENVIRONMENT` before bootstrap (PowerShell: `$env:ENVIRONMENT='...'`, or `putenv`).
 - Always pin the concrete `Client_<Id>` schema to `DB_CLIENT` (and logs/archive if the code logs).
 - **Commit seeded rows before you read them back** — the read connection cannot see uncommitted writes; commit cleanup too so it persists.

package/knowledge/2.0/standards/framework-rules.md

@@ -5,8 +5,8 @@ project: _Underscore
 client: shared
 type: standard
 status: active
-updated: 2026-09-16
-owners: [jcardinal, ajean, apeterson]
+updated: 2026-09-25
+owners: [jcardinal, ajean, apeterson, mhammontree]
 files: []
 related:
   - ../apps/_underscore/architecture.md
@@ -19,7 +19,7 @@ related:
   - framework-rules-cases.md
 ---
 
-Core 2.0 `_underscore` framework rules — class naming, the worker contract & dispatch, the api2 `{success,data,errors}` envelope, dbchanges2 schema/isolation rules, and `c_` dynamic columns; open when writing/reviewing any 2.0 code.
+Core 2.0 `_underscore` framework rules — class naming, the worker contract & dispatch, the api2 `{success,data,errors}` envelope, dbchanges2 schema/isolation rules, and `c_` columns (dynamic in /v2, declared for direct `_Model` use); open when writing/reviewing any 2.0 code.
 
 These rules apply to all 2.0 repos: `_underscore` (core), `worker2`, and `api2`. They are
 loaded on-demand by `/kickoff` when a 2.0 repo is in scope (not always-on) — see the
@@ -317,18 +317,24 @@ continue. Data accuracy outranks job "completion," even when the failure is not
 ACL error on one record) — a developer reconciling a stalled job is the intended outcome, not a
 job that reports success having quietly skipped records.
 
-## `c_`-prefixed custom columns are framework-dynamic
+## `c_`-prefixed custom columns are dynamic in the /v2 API — not in `_Model`
 
-Columns prefixed `c_` are **custom/dynamic** columns that the `_underscore` model layer handles
+Columns prefixed `c_` are **custom/dynamic** columns. The **/v2 API engine** handles them
 **without any model-class declaration**:
 
-- They are read straight off raw `_Query` result rows and written/filtered as `_Model` array
-  keys **without** being declared as properties on the `_Model_*` subclass.
+- Through /v2 they are read, written, and filtered **without** being declared as properties on
+  the `_Model_*` subclass. Raw `_Query` result rows also return them as-is.
 - They require **no `Core.RecordFields` entry and no ACL entry**.
 - Precedent examples (`c_clickupUserIdentifier`, the 2026-07 transcript-routing columns): see framework-rules-cases.md.
 
-**Consequence:** adding a `c_` column is a **schema-only** change (a `dbchanges2` migration) —
-the model/`_underscore` layer needs **zero** code changes to read, write, or filter on it. Do
-not add `RecordFields`/ACL rows or model-class properties for a `c_` column; that is redundant
-and not how the convention works. Use a non-`c_` column when you *do* want the full declared,
-ACL-governed field treatment.
+**Consequence:** for data that only flows through /v2 (or raw `_Query`), adding a `c_` column
+is a **schema-only** change (a `dbchanges2` migration). Do not add `RecordFields`/ACL rows for a
+`c_` column. Use a non-`c_` column when you *do* want the full declared, ACL-governed field
+treatment.
+
+**Exception — direct `_Model` access.** `_Model` itself is **not** dynamic: `__get`/`__set` of an
+undeclared field throws `There is no field called 'c_…'`. Code that reads or writes a `c_` column
+through a model instance (worker2 actions, crons, scripts) **must declare it as a property** on a
+client model override (e.g. `_Model_Nycdoe_ServiceRequest`) or a `_Trait_*`. Precedent:
+`_underscore/Model/Nycdoe/{File,Case,ServiceRequest}.php` (TRUE-80519). Symptom and fix:
+[model-magic-field-access](../apps/_underscore/features/model-magic-field-access.md).

package/knowledge/INDEX.md

@@ -20,7 +20,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 2.0 framework
 
 - **_underscore** (_Underscore) _(framework core)_ — 90 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
-- **worker2** (Worker) — 75 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
+- **worker2** (Worker) — 77 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)

package/knowledge/clients/nycdoe/INDEX.md

@@ -2,7 +2,8 @@
 
 | Doc | Framework | Summary |
 |-----|-----------|---------|
+| [NYCDOE 2.0 ASN SFTP Ingestion + Parsing (Sftp/Import → Sftp/Parse → Case/ServiceRequest)](features/asn-sftp-ingestion-2-0.md) | 2.0 | 2.0 rewrite of DOE's vendor-SFTP ASN intake (TRUE-80519 Phases 1–2): per-manufacturer fetch to `Files`, header-validated parse into one Case per customer PO + o |
 | [NYCDOE EDI 850 Receive & 997 Acknowledgement](features/edi-850-receive-and-997-ack.md) | 1.0 | DOE's EDI 850-in → NetSuite SO + inline 997-out ack; open for the 997 build, its "always accept" stub, or a DOE ack complaint. |
 | [NYCDOE Ticket Hold-Status Sync (ServiceNow ⇄ TOGaDesk)](features/hold-status-sync.md) | 1.0 | Bidirectional DOE ticket hold-status sync (ServiceNow ⇄ TOGaDesk), the four defects that let holds silently revert or never re-send, and the outbound cron's sel |
 | [NYCDOE ServiceNow / ASN Integration](features/servicenow-integration.md) | 1.0 | Full DOE ServiceNow/ASN integration map — ticket mirror, ASN → NetSuite SO/TOGa Desk pipeline, outbound status/POD, EDI; open for any DOE cron/pipeline work or |
-| [New York City Department of Education](profile.md) | 1.0 | NYC DOE client profile (1.0 worker tier, ServiceNow + EDI); open for DOE's platform scope, endpoints, API clients, and where its ticket/ASN/EDI logic lives. |
+| [New York City Department of Education](profile.md) | 1.0 | NYC DOE client profile (1.0 worker tier live; 2.0 ASN rewrite in progress), ServiceNow + EDI; open for DOE's platform scope, endpoints, API clients, and where i |

package/knowledge/clients/nycdoe/features/asn-sftp-ingestion-2-0.md

@@ -0,0 +1,63 @@
+---
+title: NYCDOE 2.0 ASN SFTP Ingestion + Parsing (Sftp/Import → Sftp/Parse → Case/ServiceRequest)
+framework: "2.0"
+repo: worker2
+project: Worker
+client: nycdoe
+type: client-feature
+status: active
+updated: 2026-09-25
+owners: [mhammontree]
+files:
+  - worker2/Worker/Sftp/Import.php
+  - worker2/Worker/Sftp/Parse.php
+  - _underscore/Model/Nycdoe/File.php
+  - _underscore/Model/Nycdoe/Case.php
+  - _underscore/Model/Nycdoe/ServiceRequest.php
+  - dbchanges2/Client_Nycdoe/2026-09-22b - Add ASN parsing fields to ServiceRequests.sql
+  - dbchanges2/Client_Nycdoe/2026-09-22c - Add customerPurchaseOrder field to Cases.sql
+  - worker2/tests/Integration/ParseCest.php
+related:
+  - servicenow-integration.md
+  - ../profile.md
+  - ../../../2.0/apps/worker2/features/codeception-integration-tests.md
+  - ../../../2.0/apps/_underscore/features/cli-script-bootstrap.md
+---
+
+2.0 rewrite of DOE's vendor-SFTP ASN intake (TRUE-80519 Phases 1–2): per-manufacturer fetch to `Files`, header-validated parse into one Case per customer PO + one ServiceRequest per line; open for any 2.0 DOE ASN/SFTP work.
+
+## Summary
+
+The 2.0 replacement for 1.0's `legacy_import_asn.php` + `asnprocessor/*` (see [ServiceNow / ASN Integration](servicenow-integration.md) for the 1.0 pipeline, still live). **ASN-first:** nothing is created in 2.0 except from an ingested ASN file. Phase 1 (fetch) and Phase 2 (parse) are built and verified locally against real sample files for all 4 manufacturers; Phase 3 (NetSuite SO, freeze/growth lineage) is not built. Not yet deployed/scheduled in prod.
+
+## How it works
+
+1. **`Sftp/Import/Run`** — one `Core.CronJobs` row per client+manufacturer (`dbchanges2/Core/2026-09-14a - Add SFTP Import cron - NYCDOE <Mfr>.sql`). Connects (secret by reference via `_Component_Secrets::resolve`), lists files matching `fileNamePattern`, dedupes (size/mtime, then checksum), saves each new file as a `Files` row (`Files.data` is `FIELD_STORAGE` → auto S3), `c_status='pending'`, then `_Worker::runTask('Sftp/Parse/<adapter>', …)`. Client-agnostic; the client's `Files` model class is passed per CronJobs row. `RunLocalTest()` swaps SFTP listing for a local folder (local SFTP is blocked: dev AWS SSO cannot assume WorkloadsRuntime).
+2. **`Sftp/Parse/{Lenovo,Lexmark,Acer,Apple}`** — loads the `Files` row, **validates the header by exact name match** (mismatch → `c_status='failed'` + throw; never a column-count guess — the direct fix for 1.0's "Lenovo Off-Layout ASN File"), parses rows, then:
+   - **Case = one per customer PO** (`Cases.c_customerPurchaseOrder`; `Cases.number` is `FIELD_AUTOINCREMENT`, not settable). One file can hold several POs.
+   - **ServiceRequest = one per line**, `serviceRequestTypeId` = `Onsite`, stage = `New` (both looked up by name, never hardcoded ids). Fields: `c_partNumber`, `c_serialNumber`, `c_ritmNumber`, `c_filesId`.
+   - **Dedupe on re-run:** serialized → match `c_serialNumber`; non-serialized → `(caseId, c_partNumber)` — one record per part per Case regardless of qty. Re-running a file creates nothing (verified all 4).
+   - Fail-loud: a bad row aborts the job; no catch-and-skip.
+
+| Mfr | Layout | Header const | Date column / format |
+|---|---|---|---|
+| Lenovo | own 30-col layout (not CDW) | `LENOVO_HEADER` | Ship Date `M/D/YYYY H:MM:SS AM/PM` (date part) |
+| Lexmark | CDW 47-col | `LEXMARK_HEADER` | `Order_Date` `YYYY/MM/DD` |
+| Acer | CDW 47-col, 3 names differ from Lexmark | `ACER_HEADER` | `YYYY/MM/DD` |
+| Apple | own | `APPLE_HEADER` | `order_date` `M/D/YY` or `M/D/YYYY` |
+
+Schema (`dbchanges2/Client_Nycdoe/`): `2026-09-14a` Files `c_status`/`c_manufacturer`/`c_dtProcessed`; `2026-09-22a` ServiceRequestStages seed (New/Scheduled/On Hold/Delivered/Cancelled); `2026-09-22b` ServiceRequests fields; `2026-09-22c` Cases `c_customerPurchaseOrder`; `2026-09-25a` Cases `longDescription`.
+
+Tests: `worker2/tests/Integration/ParseCest.php` (all 4 mfrs + header rejection + comma repair) — see [worker2 Codeception integration tests](../../../2.0/apps/worker2/features/codeception-integration-tests.md).
+
+## Gotchas
+
+- **Direct `_Model` writes need every `c_` field declared as a real property** — `_Model::__set()` throws `There is no field called 'c_…'` otherwise. The "c_ needs no declaration" behavior is the /v2 API engine's, not `_Model`'s. Hence `_Model_Nycdoe_File` / `_Model_Nycdoe_Case` / `_Model_Nycdoe_ServiceRequest`.
+- **1.0 `asnprocessor/lenovo.php` is not a source of truth for today's Lenovo layout.** Its 47-col mapping is stale; the real file is 30 cols (confirmed on 3 live samples). Verify every header and date format against a **real current sample file**, never the 1.0 code — wrong Lenovo header, Lexmark date, and Apple 2-digit year (`'25'` → year `0025`) were all caught this way.
+- **Acer and Lexmark share column positions, not header names** (Acer `First_Name_Shipping`/`Last_Name_Shipping`/`SerialNo` vs Lexmark `Contact_Name_Shipping`/`Email_Shipping`/`Svc_Tag_ID`). Keep separate header constants. Lenovo stays its own path (not a CDW supplier).
+- **Lenovo unescaped comma in `Product Long Description` (col 18) is repaired, not rejected** — `repairLenovoRowColumnCount()` merges the extra fragments back (ported from 1.0; the only field Lenovo breaks this way). A true header mismatch is still fatal. No real occurrence seen in 2.0 yet — re-confirm when Lenovo next sends one.
+- **`DOE-OPEN-LENOVO-*` files are a retired process — never parse; delete on sight** (Skyler/Eric). Lenovo's `fileNamePattern` `DOE-Lenovo-ASN-*.csv` already excludes them. SFTP cleanup of old ones is deferred.
+- **Schema drift: `_Model_Client_Case` declares `longDescription` but `Client_Nycdoe.Cases` lacked the column** → any direct-model Case load/save 1054s. Fixed for DOE (`2026-09-25a`). Other tenants' `Cases` may have the same gap — unchecked (ask Rohan).
+
+## Change history
+- 2026-09-25 — Phase 2 parsing built for all 4 mfrs; header validation, comma repair, real-sample header/date corrections; integration tests added (mhammontree)

package/knowledge/clients/nycdoe/features/servicenow-integration.md

@@ -6,7 +6,7 @@ project: Worker
 client: nycdoe
 type: client-feature
 status: active
-updated: 2026-09-18
+updated: 2026-09-25
 owners: [mhammontree, sking]
 files:
   - worker/crons/sync/nycdoe/import_asn.php
@@ -43,6 +43,7 @@ related:
   - edi-850-receive-and-997-ack.md
   - ../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md
   - hold-status-sync.md
+  - asn-sftp-ingestion-2-0.md
   - ../../../1.0/apps/worker/architecture.md
   - ../../../1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md
 ---
@@ -145,7 +146,7 @@ The five 1.0 `App_SystemMonitor_ServiceNow*` monitors scrape logs for **arrived-
 - **Any structured HTTP response below 500 is a PASS** — it proves DNS, TLS, the load balancer, routing, and the auth layer are alive. A `401` is a *healthy* answer, not a failure.
 - **Accepted coverage gap: token validity is NOT monitored.** The OAuth client `App_Api_NYCDOEV2` lives in the 1.0 `library` repo, which worker2 cannot reach. Re-implementing it in worker2 would duplicate DOE's credentials into a second repo and race the shared access token. An expired/revoked token will surface via the 1.0 failure monitors, not this probe — do not "fix" this by forking the client.
 
-This probe is the client's **only 2.0 footprint**. DOE is **1.0 by default** — `worker2` appears in the client's `apps:` scope solely so this monitor loads; not the start of a 2.0 migration (there is no `Client_Nycdoe` 2.0 tenant). All business logic stays in the 1.0 tier. See [OneUptime push-metric monitors for 2.0 workers](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md).
+All **live** DOE business logic is 1.0. The only other 2.0 code is the in-progress ASN rewrite on the `Client_Nycdoe` tenant — see [2.0 ASN SFTP Ingestion](asn-sftp-ingestion-2-0.md). See [OneUptime push-metric monitors for 2.0 workers](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md).
 
 ### Operating rules when changing this integration
 1. **Trace the full pipeline downstream before fixing** — the constraint that makes an edit unsafe usually lives downstream (the freeze point and the consumer/display query).
@@ -170,7 +171,8 @@ This probe is the client's **only 2.0 footprint**. DOE is **1.0 by default** —
   - **Asset-tag repair is a SEPARATE, TWO-STORE backfill** — see the derived-`assets.tag` gotcha below. Generate the SQL **mechanically** from the customer's serial→tag CSV (never hand-transcribe), match serials through `App_NYCDOE::serialKey()` (the leading-`S` gotcha below), scope the statements to the ASN/MSO, and fill **only still-empty** tags so the file is re-runnable. Done this way for ASN `26720` / PO `S202648137` / MSO `329924` (87 units) on 2026-08-13, PR dbchanges#265.
   - **Known remaining damage from this defect** (each needs its own customer serial→tag list): ASN `26641` / PO `S202668066` — **162 of 363** units still tagless; ASN `26706` / PO `WR270003276` — **1 of 7**.
   - **A fixture-based regression test is owed with the parser fix**: assert that serial, quantity and assetTag each land in the right field for **both** layouts (do not detect by column count alone). ⚠ Do not quote constants out of `asnprocessor/lenovo.php` into a doc or ticket — that file contains hardcoded FTP credentials.
-  - **Recommended future code fixes (proposed, NOT approved, NOT implemented):** map columns by **header name** and hard-reject an unknown layout with a loud alert; stop building the dedupe key from a field that can be a constant; widen `AdvanceShippingNoticeItems.partNumber`.
+  - **`DOE-Open-Lenovo` is a retired Lenovo process — it should never be parsed; delete on sight** (Skyler/Eric, 2026-09-25). The 1.0 parser still picks it up (unchanged).
+  - **Recommended 1.0 code fixes (proposed, NOT approved, NOT implemented):** map columns by **header name** and hard-reject an unknown layout with a loud alert; stop building the dedupe key from a field that can be a constant; widen `AdvanceShippingNoticeItems.partNumber`. The 2.0 rewrite already does exact header validation and excludes `DOE-Open-Lenovo` by filename — see [2.0 ASN SFTP Ingestion](asn-sftp-ingestion-2-0.md).
   - **UNRESOLVED.** On ASN `26720`, items `54672`/`54673` carried 31 and 25 units with **correct** serials in `serialNumber` and `assetTag` NULL — which the three collapsed queue rows cannot account for. The run that created those units was **not identified**. Still open; do not assume it is explained.
   - **Ruled out:** the recent **Quad** ASN import work is unrelated — Quad lives under `worker/crons/toga2/**` against the toga2 `AdvanceShippingNoticeItemUnits` schema, while the DOE path is `Core.AdvanceShippingNoticeUnits` via `asnprocessor/lenovo.php`.
 
@@ -257,3 +259,6 @@ This probe is the client's **only 2.0 footprint**. DOE is **1.0 by default** —
 - [NYCDOE EDI 850 Receive & 997 Acknowledgement](edi-850-receive-and-997-ack.md) — 850-in/997-out
 - [Worker (1.0) architecture](../../../1.0/apps/worker/architecture.md) — cron dispatch, role election, schedule files
 - Local deep-dive (full WR260236464 case study with record IDs): `doe_service_now_integration.md` in Mark's WWW workspace root; also the `service-now-integration` skill (`asn-import-flow.md`)
+
+## Change history
+- 2026-09-25 — Corrected "no 2.0 tenant" (TRUE-80519 2.0 ASN rewrite now exists); DOE-Open-Lenovo marked retired (mhammontree)

package/knowledge/clients/nycdoe/profile.md

@@ -7,26 +7,29 @@ apps:
   - togadesk
   - dbchanges
   - worker2
+  - _underscore
+  - dbchanges2
 project: Worker
 client: nycdoe
 type: profile
 status: active
-updated: 2026-09-16
+updated: 2026-09-25
 owners: [mhammontree, sking]
 files: []
 related:
   - features/servicenow-integration.md
   - features/edi-850-receive-and-997-ack.md
+  - features/asn-sftp-ingestion-2-0.md
 ---
 
-NYC DOE client profile (1.0 worker tier, ServiceNow + EDI); open for DOE's platform scope, endpoints, API clients, and where its ticket/ASN/EDI logic lives.
+NYC DOE client profile (1.0 worker tier live; 2.0 ASN rewrite in progress), ServiceNow + EDI; open for DOE's platform scope, endpoints, API clients, and where its ticket/ASN/EDI logic lives.
 
 ## Summary
-NYC DOE — TOGA client whose entire integration runs in the **1.0 worker tier** (~30 cron scripts under `worker/crons/sync/nycdoe/` and `worker/crons/notifications/nycdoe/`, on the `sync` and `notification` worker roles). ServiceNow is DOE's system of record for tickets (Incidents and RITMs); TOGA mirrors those locally, drives fulfillment through TOGa Desk and NetSuite, and pushes status / ETA / proof-of-delivery back. A parallel EDI relationship (850 POs in, invoices out) runs over DOE's SFTP.
+NYC DOE — TOGA client whose **live** integration runs in the **1.0 worker tier** (~30 cron scripts under `worker/crons/sync/nycdoe/` and `worker/crons/notifications/nycdoe/`, on the `sync` and `notification` worker roles). ServiceNow is DOE's system of record for tickets (Incidents and RITMs); TOGA mirrors those locally, drives fulfillment through TOGa Desk and NetSuite, and pushes status / ETA / proof-of-delivery back. A parallel EDI relationship (850 POs in, invoices out) runs over DOE's SFTP. A **2.0 rewrite (TRUE-80519)** of the ASN intake is being built against the `Client_Nycdoe` tenant — not yet live.
 
 ## Platforms & data
-- **1.0 worker tier** — all business logic. Local mirror tables in `db_common` (`NYCDOETickets`, `NYCDOELocations`, `AdvanceShippingNotice*`).
-- **`worker2` (2.0) — monitoring only.** DOE's *only* 2.0 code is the OneUptime liveness probe `Monitor/Nycdoe/ServiceNowEndpointHealth` (`worker2/Worker/Monitor/Nycdoe.php`, TRUE-80587). This client is **1.0 by default** — `worker2` is in `apps:` solely so that monitor loads. **Not** a 2.0 migration; there is no `Client_Nycdoe` 2.0 tenant.
+- **1.0 worker tier** — all live business logic. Local mirror tables in `db_common` (`NYCDOETickets`, `NYCDOELocations`, `AdvanceShippingNotice*`).
+- **2.0 (`worker2` + `_underscore` + `dbchanges2`, tenant `Client_Nycdoe`)** — (1) the OneUptime liveness probe `Monitor/Nycdoe/ServiceNowEndpointHealth` (`worker2/Worker/Monitor/Nycdoe.php`, TRUE-80587, live); (2) the ASN rewrite `Sftp/Import` → `Sftp/Parse` → Case/ServiceRequest (Phases 1–2 built, not deployed) — see [2.0 ASN SFTP Ingestion](features/asn-sftp-ingestion-2-0.md). Entity map: `Case` = 1.0 managed_service_order, `ServiceRequest` = 1.0 repair_order; DOE does not use `Ticket`. Client models: `_underscore/Model/Nycdoe/`.
 - **TOGa Desk:** DOE is client id **16** (`db_togadesk`) — repair orders, managed service orders, the "#received / #ordered" Receiving Summary UI.
 - **NetSuite:** Sales Orders, PO reconciliation, item receipts, Item Fulfillments, invoices (SuiteTalk toolkit in `library`).
 - **`dbchanges` (1.0):** DOE schema + **data-repair** changes ship here (`dbchanges/Core/` targets the legacy/V1 `Core` schema). Stranded-ASN recoveries are data-only repairs — see the [1.0 dbchanges authoring workflow](../../1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md).
@@ -46,3 +49,4 @@ NYC DOE — TOGA client whose entire integration runs in the **1.0 worker tier**
 
 ## Key features (this client)
 - [ServiceNow / ASN Integration](features/servicenow-integration.md) — full integration map: ticket sync, the ASN pipeline (core flow), outbound status sync, EDI, SME-confirmed business rules.
+- [2.0 ASN SFTP Ingestion + Parsing](features/asn-sftp-ingestion-2-0.md) — the TRUE-80519 2.0 rewrite (fetch → header-validated parse → Case/ServiceRequest).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.892",
+  "version": "1.0.893",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",