toga-ai 1.0.961 → 1.0.962

package/knowledge/clients/nycdoe/features/asn-sftp-ingestion-2-0.md

@@ -66,7 +66,7 @@ The 2.0 replacement for 1.0's `legacy_import_asn.php` + `asnprocessor/*` (see [S
 5. **`Run()`** gained `deleteSourceOnSuccess` and `retiredFileNamePatterns` params and skips files modified < 5 min ago. Validates `fileModelClass` (`is_a _Model_Client_File`) and `adapter` (a method on `_Worker_Sftp_Parse`).
 6. **Kill switch** is 1.0's `[nycdoe] asn_delete_mode` (missing = old immediate delete).
 
-**Manual release of a failed file:** set `Files.c_status='released'`, then queue `Sftp/Import/DeleteSource` for that `filesUuid`.
+**Manual release of a failed file:** see *Failed file* in the runbook below.
 
 **Before cutover (open):**
 - `Run()` has no retry for a lost Parse or lost `DeleteSource` job (only `RunForFile` re-drives). A row stuck `processing` turns `failed` only on the next 1.0 trigger after 30 min.
@@ -79,6 +79,29 @@ Core `2026-09-14a` Lenovo row carries `retiredFileNamePatterns`. Neither `2026-0
 
 Tests: `ParseCest.php` (all 4 mfrs, header rejection, comma repair, claim, kept `failed` status) + `ImportDecisionCest.php` (RunForFile decisions, path check, retired names) — see [worker2 Codeception integration tests](../../../2.0/apps/worker2/features/codeception-integration-tests.md).
 
+## Activation / deployment runbook (TRUE-80519)
+PRs open 2026-10-06: library, worker2, dbchanges2, _underscore (worker optional for Part A).
+
+**Part A — inert (no 1.0 behavior change):**
+1. dbchanges2 `Client_Nycdoe`, in order: `2026-09-14a`, `2026-09-22a`, `2026-09-22b`, `2026-09-22c`, `2026-09-25a`.
+2. dbchanges2 `Core` `2026-09-14a` ×4 (Acer/Apple/Lenovo/Lexmark CronJobs, `isActive=0`).
+3. Confirm `Files` has `c_status`/`c_manufacturer`/`c_dtProcessed` + checksum index, and 4 inactive `Sftp/Import/Run` rows.
+4. Deploy `_underscore` (Rohan's weekly window; `Model/Nycdoe` File/Case/ServiceRequest — commits `05a98743`, `1223e2a9`).
+5. Deploy worker2.
+6. Deploy library + worker with **no** ini change.
+7. Confirm 1.0 still imports to `AdvanceShippingNoticeQueue`, deletes from SFTP, one `Logs.FileLog` row per file.
+
+**Part B — shadow mode (gated delete on):**
+1. Confirm the 4 Secrets Manager secrets `sftp/nycdoe/<mfr>` exist and WorkloadsRuntime can read them.
+2. Set `[nycdoe] asn_delete_mode = gated` in `worker/config.worker.ini`; deploy worker.
+3. Verify: one FileLog row per file, no re-processing/re-email; `Core.WorkerJobs` `Sftp/Import/RunForFile` results `queued`/`in_progress`/`already_done_delete_queued`; `Files.c_status` → `succeeded`; `Sftp/Import/DeleteSource` result `deleted`; files leave SFTP within ~15–30 min; `DOE-OPEN-LENOVO-*` deleted; other unmatched names kept.
+
+**Rollback:** remove `asn_delete_mode` (or set `immediate`), deploy worker.
+
+**Failed file:** investigate (Tools → Errors / `Logs.Issue`), set `Files.c_status='released'`, queue `Sftp/Import/DeleteSource {clientId: 38, filesUuid}`.
+
+**Later (not this deploy):** cutover = turn the 4 crons on + remove NYCDOE processors from 1.0; decide manual ASN file handling; Phase 3 NetSuite; separate tickets for host-key check + FTP password rotation (see *Before cutover* above).
+
 ## Gotchas
 
 - **Direct `_Model` writes need every `c_` field declared as a real property** — `_Model::__set()` throws `There is no field called 'c_…'` otherwise. The "c_ needs no declaration" behavior is the /v2 API engine's, not `_Model`'s. Hence `_Model_Nycdoe_File` / `_Model_Nycdoe_Case` / `_Model_Nycdoe_ServiceRequest`.
@@ -93,6 +116,7 @@ Tests: `ParseCest.php` (all 4 mfrs, header rejection, comma repair, claim, kept
 - **Schema drift: `_Model_Client_Case` declares `longDescription` but `Client_Nycdoe.Cases` lacked the column** → any direct-model Case load/save 1054s. Fixed for DOE (`2026-09-25a`). Other tenants' `Cases` may have the same gap — unchecked (ask Rohan).
 
 ## Change history
+- 2026-10-06 — Activation/deployment runbook (Part A inert, Part B shadow mode, rollback, failed-file release) (mhammontree)
 - 2026-10-06 — Gated delete built (RunForFile/DeleteSource, Parse claim + status-outside-transaction fix, retired names, `released` status); open cutover items (mhammontree)
 - 2026-09-29 — Activation plan: gated-delete decision, proposed RunForFile/DeleteSource design, CronJobs rows set isActive=0 (mhammontree)
 - 2026-09-25 — Phase 2 parsing built for all 4 mfrs; header validation, comma repair, real-sample header/date corrections; integration tests added (mhammontree)

package/knowledge/clients/nycdoe/features/hold-status-sync.md

@@ -6,7 +6,7 @@ project: Worker
 client: nycdoe
 type: client-feature
 status: active
-updated: 2026-09-29
+updated: 2026-10-06
 owners: [mhammontree, sking]
 files:
   - worker/crons/sync/nycdoe/send_ticket_updates.php
@@ -178,7 +178,7 @@ Before this the cron ran the selection query **once per ticket under `LIMIT 1`**
 All three `error_log()` calls were removed (the per-ticket technician sys_id message, TRUE-82027's "alerted on N failures" line, and one added in TRUE-82129). **1.0 worker crons must produce no output** — they run unattended under CRON and `error_log()` goes to stderr on CLI. Failures surface only in `Logs.Issue` and the throttled alert email. General 1.0 worker rule, not DOE-specific (Mark, 2026-09-18).
 
 ### Missing technician `referenceId` is the NORMAL state, not an incident
-**265 of 389 (68%) TOGaDesk technicians have no `people.referenceId`** (ServiceNow sys_id), so `u_technician` is left **unchanged** on those pushes — deliberate: blanking it would wipe a technician ServiceNow may legitimately hold. State, hold, ETA and assignment_group still sync. This is why the per-ticket log line was removed. Reconcile with `worker/crons/sync/nycdoe/report_technician_identities.php` / `backfill_technician_identities.php` (needs its own ticket).
+**265 of 389 (68%) TOGaDesk technicians have no `people.referenceId`** (ServiceNow sys_id), so `u_technician` is left **unchanged** on those pushes — deliberate: blanking it would wipe a technician ServiceNow may legitimately hold. State, hold, ETA and assignment_group still sync. This is why the per-ticket log line was removed. Reconcile with `worker/crons/sync/nycdoe/report_technician_identities.php` (read-only classifier; needs its own ticket). `backfill_technician_identities.php` is **deleted on worker branch `TRUE-80519`** (commit `9e1dc9afa`; takes effect when that PR merges) — recover it from git history before that commit if a backfill is ever run. `send_ticket_updates.php:537` comment still points at the deleted file.
 
 ### Failure alerting instead of a throw (TRUE-82027)
 **DECISION (Mark, 2026-09-17): no circuit breaker** — 1.0 is transitioning to 2.0 under a separate task, so the cheap path was chosen.
@@ -270,6 +270,7 @@ The SNOW round-trip note reconciliation (delete-and-reinsert in `process_tickets
 - `qqStatus()` collapses all recognized hold sub-statuses to base `HOLD` in `repair_orders.status`; the specific hold **sub-reason persists only in `repair_order_notes.newstatus`**. Any status badge / report / filter reading `repair_orders.status` directly cannot distinguish the six hold variants after a recompute.
 
 ## Change history
+- 2026-10-06 — `backfill_technician_identities.php` deleted on worker branch TRUE-80519 (effective on merge); git-history pointer added. (mhammontree)
 - 2026-09-29 — Assignment-group scoping rule + inverted `isValidDepartmentName` + the three competing group definitions; TRUE-82027 verified in prod (marker rows + `Logs.Emails` method). (mhammontree)
 - 2026-09-25 — TRUE-82332: hold-release no longer overwrites the close `state`; close re-sent when SN not closed; stuck-ticket re-queue recipe; `Logs.API` query shapes. (mhammontree)
 - 2026-09-18 — TRUE-82129: fixed the by-reference `fetchOne()` notice that killed every run; batch loop + run caps replaced LIMIT 1 + per-ticket sleep; alert emails de-PII'd and escaped; unmapped-group email now grouped per run. (mhammontree)

package/knowledge/clients/nycdoe/features/servicenow-integration.md

@@ -81,7 +81,7 @@ The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents +
 | NetSuite SO create (Stage 3, **freeze**) | `1_send_asn_to_netsuite.php` | every 2 h at :07 |
 | NetSuite PO/serials (Stage 4) | `2_send_serials_to_netsuite.php` | every 2 h at :27 |
 | TOGa Desk repair orders (Stage 5) | `3_create_installation_ticket.php` | every 5 min |
-| NetSuite Item Fulfillment (Stage 6) | `4_create_ns_item_fulfillment.php` (**paused**, `active: 0`); `5_DOA_…` unscheduled | not running |
+| NetSuite Item Fulfillment (Stage 6) | `4_create_ns_item_fulfillment.php` (**paused**, `active: 0`); `5_DOA_…` unscheduled (moving to `crons/sync/DOA/`, see below) | not running |
 | ETA → ServiceNow | `send_ticket_updates.php` | every 6 min |
 | RITM state → ServiceNow | `send_request_item_updates.php` | every 5 min |
 | Proof of delivery → ServiceNow | `send_nycdoe_proof_of_delivery.php` | every 5 min |
@@ -91,7 +91,7 @@ The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents +
 | EDI invoices out (NetSuite saved search 2108) | `send_edi_open_invoices.php` | every 2 h at :13 |
 | Installation schedule reports | `notifications/nycdoe/send_installation_schedules_doe.php` (3 business days ahead, weekends skipped, **holidays NOT skipped**), daily + current-week variants, weekly part-orders | 5–8 AM daily/Mondays |
 
-**Unscheduled / dormant:** `download_tickets*.php` (superseded by `import_inc.php` / `import_asn.php`), `DOA_*` variants, `doe_install_fix.php`, `send_installation_schedules.php` (base variant). Note `sync/syncro/download_tickets.php` in the schedule is the **Syncro** integration, not NYCDOE. A second `crons/infrastructure/import_asn.php` also calls the asnprocessors but is **not scheduled** — only `legacy_import_asn.php` runs.
+**Unscheduled / dormant:** `download_tickets*.php` (superseded by `import_inc.php` / `import_asn.php`), `send_installation_schedules.php` (base variant). **Worker branch `TRUE-80519` (commit `9e1dc9afa`, takes effect when that PR merges):** the 5 DOA files (`5_DOA_set_item_fulfillment_as_shipped.php`, `DOA_3_B_create_installation_ticket.php`, `DOA_3_create_installation_ticket.php`, `DOA_3_create_installation_ticket_savedsearch.php`, `DOA_delete_query_for_doe.php`) move to `crons/sync/DOA/`; `doe_install_fix.php`, `backfill_technician_identities.php`, `test_assignment_revert_guard.php`, `test_multi_po_receipt_resolution.php` are deleted (none scheduled or called; recover from git history before `9e1dc9afa`). Note `sync/syncro/download_tickets.php` in the schedule is the **Syncro** integration, not NYCDOE. A second `crons/infrastructure/import_asn.php` also calls the asnprocessors but is **not scheduled** — only `legacy_import_asn.php` runs.
 
 ### The ASN pipeline
 ```
@@ -269,6 +269,7 @@ All **live** DOE business logic is 1.0. The only other 2.0 code is the in-progre
 - Local deep-dive (full WR260236464 case study with record IDs): `doe_service_now_integration.md` in Mark's WWW workspace root; also the `service-now-integration` skill (`asn-import-flow.md`)
 
 ## Change history
+- 2026-10-06 — Dormant DOA files moved to `crons/sync/DOA/` and 4 unused scripts deleted on worker branch TRUE-80519 (effective on merge) (mhammontree)
 - 2026-10-06 — Stage 1b gated delete mode (`asn_delete_mode`, `releaseSourceFile`, FileLog saved before release) built, not deployed (mhammontree)
 - 2026-09-29 — Stage 1b file-delete path (delete not gated on parse; SFTP password rides in SQS), Acer-as-Lexmark, FileLog index facts (mhammontree)
 - 2026-09-29 — Assignment-group scoping corrected (inverted `isValidDepartmentName`, three competing group lists) with detail in hold-status-sync; `Logs.API` 30-day retention shipped. (mhammontree)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.961",
+  "version": "1.0.962",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",