toga-ai 1.0.580 → 1.0.581

package/knowledge/1.0/apps/dbchanges/INDEX.md

@@ -2,4 +2,4 @@
 
 | Doc | Summary | Files |
 |-----|---------|-------|
-| [Authoring & Shipping a 1.0 dbchanges SQL File](workflows/authoring-and-shipping-sql-files.md) | `dbchanges` is the **1.0** (legacy/V1) schema-and-data change repository — the 1.0 sibling of 2.0's `dbchanges2`. | dbchanges/index.php, dbchanges/Core/, worker/crons/infrastructure/execute_dbchanges.php, worker/.ebextensions/030_dbchanges.config |
+| [Authoring & Shipping a 1.0 dbchanges SQL File](workflows/authoring-and-shipping-sql-files.md) | `dbchanges` is the **1.0** (legacy/V1) schema-and-data change repository — the 1.0 sibling of 2.0's `dbchanges2`. | dbchanges/index.php, dbchanges/Core/, dbchanges/TOGaDeskSupport/, worker/crons/infrastructure/execute_dbchanges.php, worker/.ebextensions/030_dbchanges.config |

package/knowledge/1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md

@@ -6,11 +6,12 @@ project: Database Changes
 client: shared
 type: workflow
 status: active
-updated: 2026-07-29
+updated: 2026-08-14
 owners: [sking]
 files:
   - dbchanges/index.php
   - dbchanges/Core/
+  - dbchanges/TOGaDeskSupport/
   - worker/crons/infrastructure/execute_dbchanges.php
   - worker/.ebextensions/030_dbchanges.config
 related:
@@ -97,6 +98,27 @@ WHERE t.id IN (<ids>)
       ) AS preflight) = <expected-row-count>;
 ```
 
+## One top-level folder = one DATABASE (and the silent-skip trap)
+
+The repo's top level is a list of **database names**, not modules: `Core`, `TOGaDeskSupport`,
+`Common`, `Logs`, `Bridge_NetSuite`, … The runner walks every top-level directory and treats the
+**folder name as the database to connect to** (`index.php` ~L55–60), then — for each folder — scans
+`config.<env>.ini` and uses the **first `[database*]` group it finds** (`~L76–80`; in
+`config.prod.ini` that is `[database_main]`) for hostname/username/password. So *all* folders are
+attempted with **one** set of credentials against one host, with only the DB name varying.
+
+Consequence: **a folder whose database is not reachable from that host is skipped silently.**
+The failure path is a bare `echo "Skipping <db> Database... unable to connect"` (`~L189`) — no
+`Logs.Errors` row, no `exit()`, no `_dbchanges` bookkeeping. Nothing retries it, and nothing tells
+you. A change that spans two databases on two different clusters (e.g. `Core` on **db_core** and
+`TOGaDeskSupport` on **db_togadesk**) therefore needs **two separate runs**, one per cluster, and
+an applier pointed at only one of them **half-applies the change and reports success**.
+
+**Rule: a multi-database change is not done until you have verified each database independently.**
+Verify **per-pair** (does *this* key have *this* value?), not by a `COUNT(*) ... IS NOT NULL` total —
+a per-count check passes on the cluster that applied and tells you nothing about the one that
+didn't.
+
 ## Which branches actually auto-apply
 
 | Path | Trigger | Timing |
@@ -121,7 +143,11 @@ The repo ships `config.alpha/beta/demo/hotfix/prod/stage/test/worker.ini`, there
    is a no-op rather than a wrong write.
 5. Open a PR into `_production` (or the target `_<env>` branch).
 6. After merge: `_<env>` branches apply within ~2 min; **`_production` waits for the next worker
-   deploy** — verify with a read-only query afterwards rather than assuming.
+   deploy** — verify with a read-only query afterwards rather than assuming. **Merging the PR
+   applies nothing by itself** — `dbchanges` has no CI, no workflow, no runner of its own; every
+   application is driven from the worker tier.
+7. If the change spans more than one top-level (database) folder, verify **each database
+   separately** — see the silent-skip trap above.
 
 ## Gotchas
 
@@ -130,6 +156,14 @@ The repo ships `config.alpha/beta/demo/hotfix/prod/stage/test/worker.ini`, there
   local one.
 - **No `DELIMITER`, no stored routines, no explicit `COMMIT`.** See above.
 - **Error 1093** is why the preflight subquery must be wrapped in a derived table.
+- **A merged PR is not an applied change.** Confirmed live 2026-08-13 (PR dbchanges#265): after the
+  merge into `_production`, both target tables were still 100% unchanged. Never tell a customer or a
+  ticket "applied" off a merge — always re-read the data.
+- **A cross-database change half-applies without warning.** Same incident: the `TOGaDeskSupport`
+  file landed and the `Core` file did not, leaving `assets.tag` 87/87 correct while
+  `Core.AdvanceShippingNoticeUnits.assetTag` was 87/87 still empty — a state that *looks* fixed in
+  the UI. Cause is the one-credential-set / folder-is-a-database model above. Verify every database
+  in the change, per-pair.
 - **Retry semantics are a feature**: because the data change and the `_dbchanges` row share a
   transaction, a failing file re-runs next cycle. Do not "help" it with a manual COMMIT.
 - **⚠ SECURITY — hardcoded GitHub token.** `worker/crons/infrastructure/execute_dbchanges.php`
@@ -140,6 +174,13 @@ The repo ships `config.alpha/beta/demo/hotfix/prod/stage/test/worker.ini`, there
   argv string. No credential value is recorded here by design.
 
 ## Change history
+- 2026-08-14 — Documented that each **top-level folder is a database name** and that the runner uses
+  the **first `[database*]` config group** for every folder, so a folder on another cluster is
+  **silently skipped** (bare echo, no `Logs.Errors`, no `exit()`, no retry). Confirmed live during
+  the DOE asset-tag backfill (PR dbchanges#265): merging the PR applied nothing, and then a
+  `TOGaDeskSupport`-only run left `Core` untouched — a half-fix that looked complete in the UI.
+  Added the rule that a multi-database change needs one run per cluster and **per-pair**
+  verification, not a non-null count. (sking)
 - 2026-07-29 — Repo onboarded to the KB (1.0 `dbchanges`, framework core sibling of `dbchanges2`).
   Documented the `index.php` `";\n"` split + `mysqli_query()` model and the comment-only-chunk /
   error-1065 queue abort, the per-file transaction & retry semantics, the derived-table preflight

package/knowledge/clients/nycdoe/INDEX.md

@@ -3,5 +3,5 @@
 | Doc | Framework | Summary | Files |
 |-----|-----------|---------|-------|
 | [NYCDOE Ticket Hold-Status Sync (ServiceNow ⇄ TOGaDesk)](features/hold-status-sync.md) | 1.0 | DOE ticket **hold** status must round-trip between ServiceNow (SNOW) and TOGaDesk and **stay held** — holds are SLA-bearing in both systems. | worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/send_request_item_updates.php, library/app/model/togadesk/repairorder.php, library/app/api/nycdoev2.php, togadesk/desk/includes/classes/class.repair.php |
-| [NYCDOE ServiceNow / ASN Integration](features/servicenow-integration.md) | 1.0 | The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents + RITMs) into local tables, turns vendor shipment notices into NetSuite Sales Orde | worker/crons/sync/nycdoe/import_asn.php, worker/crons/sync/nycdoe/import_inc.php, worker/crons/sync/nycdoe/legacy_import_asn.php, worker/crons/sync/nycdoe/legacy_process_asn_queue.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/1_send_asn_to_netsuite.php, worker/crons/sync/nycdoe/2_send_serials_to_netsuite.php, worker/crons/sync/nycdoe/3_create_installation_ticket.php, worker/crons/sync/nycdoe/test_multi_po_receipt_resolution.php, worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/send_request_item_updates.php, worker/crons/sync/nycdoe/send_nycdoe_proof_of_delivery.php, worker/crons/sync/nycdoe/sync_nycdoe_locations.php, worker/crons/sync/nycdoe/receive_edi_purchase_orders.php, worker/crons/sync/nycdoe/send_edi_open_invoices.php, worker/crons/notifications/nycdoe/, worker/schedules/cron.worker.sync.json, worker/schedules/cron.worker.notification.json, library/app/api/nycdoe.php, library/app/api/nycdoev2.php, library/app/asnprocessor/manufacturer.php, library/app/asnprocessor/apple.php, library/app/asnprocessor/lenovo.php, library/app/asnprocessor/lexmark.php, library/app/asnprocessor/acer.php, library/app/edi.php, library/app/netsuite.php, dbchanges/Core/SK/ |
+| [NYCDOE ServiceNow / ASN Integration](features/servicenow-integration.md) | 1.0 | The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents + RITMs) into local tables, turns vendor shipment notices into NetSuite Sales Orde | worker/crons/sync/nycdoe/import_asn.php, worker/crons/sync/nycdoe/import_inc.php, worker/crons/sync/nycdoe/legacy_import_asn.php, worker/crons/sync/nycdoe/legacy_process_asn_queue.php, worker/crons/sync/nycdoe/process_tickets.php, worker/crons/sync/nycdoe/1_send_asn_to_netsuite.php, worker/crons/sync/nycdoe/2_send_serials_to_netsuite.php, worker/crons/sync/nycdoe/3_create_installation_ticket.php, worker/crons/sync/nycdoe/test_multi_po_receipt_resolution.php, worker/crons/sync/nycdoe/send_ticket_updates.php, worker/crons/sync/nycdoe/send_request_item_updates.php, worker/crons/sync/nycdoe/send_nycdoe_proof_of_delivery.php, worker/crons/sync/nycdoe/sync_nycdoe_locations.php, worker/crons/sync/nycdoe/receive_edi_purchase_orders.php, worker/crons/sync/nycdoe/send_edi_open_invoices.php, worker/crons/notifications/nycdoe/, worker/schedules/cron.worker.sync.json, worker/schedules/cron.worker.notification.json, library/app/api/nycdoe.php, library/app/api/nycdoev2.php, library/app/nycdoe.php, library/app/asnprocessor/manufacturer.php, library/app/asnprocessor/apple.php, library/app/asnprocessor/lenovo.php, library/app/asnprocessor/lexmark.php, library/app/asnprocessor/acer.php, library/app/edi.php, library/app/netsuite.php, dbchanges/Core/SK/, dbchanges/TOGaDeskSupport/SK/ |
 | [New York City Department of Education](profile.md) | 1.0 | NYC DOE (New York City Department of Education) is a TOGA client whose entire integration runs in the **1.0 worker tier** (~30 cron scripts under `worker/crons/ |  |

package/knowledge/clients/nycdoe/features/servicenow-integration.md

@@ -6,7 +6,7 @@ project: Worker
 client: nycdoe
 type: client-feature
 status: active
-updated: 2026-08-11
+updated: 2026-08-14
 owners: [mhammontree, sking]
 files:
   - worker/crons/sync/nycdoe/import_asn.php
@@ -29,6 +29,7 @@ files:
   - worker/schedules/cron.worker.notification.json
   - library/app/api/nycdoe.php
   - library/app/api/nycdoev2.php
+  - library/app/nycdoe.php
   - library/app/asnprocessor/manufacturer.php
   - library/app/asnprocessor/apple.php
   - library/app/asnprocessor/lenovo.php
@@ -37,6 +38,7 @@ files:
   - library/app/edi.php
   - library/app/netsuite.php
   - dbchanges/Core/SK/
+  - dbchanges/TOGaDeskSupport/SK/
 related:
   - ../profile.md
   - hold-status-sync.md
@@ -250,10 +252,29 @@ Vendor SFTP ───(legacy_import_asn.php, ser+non-ser)─┘      [UNIQUE ded
     item/unit lookup keys (built from the truncated DB value, compared against the full
     in-memory string), and two long descriptions sharing their first 64 characters collapse
     into the same item.
+  - **It is also the root cause of the DOE missing-asset-tag class** (traced 2026-08-13). Because
+    the shift is +1, `assetTag ← $cols[23]` picks up the **serial**, and the **real asset-tag
+    column `$cols[24]` is never read at all** — so no unit from an off-layout file can ever get a
+    correct tag. Further collateral on the same rows: `workPhone` holds a date and `shipZipCode`
+    holds a person's name. Treat any Lenovo missing-tag report as this bug until proven otherwise,
+    and fix the parser rather than only backfilling.
   - **Remediation is MANUAL data repair — there is no automated recovery.** Correct
     `Core.AdvanceShippingNoticeItems.partNumber` on the affected items to the real NetSuite
     OEM SKU (replacing the description the malformed file supplied); the pipeline then
     proceeds normally. Done this way for ASN `26720` on 2026-08-11.
+  - **Asset-tag repair is a SEPARATE, TWO-STORE backfill** — see the derived-`assets.tag` gotcha
+    below. Generate the SQL **mechanically** from the customer's serial→tag CSV (never hand-
+    transcribe), match serials through `App_NYCDOE::serialKey()` (the leading-`S` gotcha below),
+    scope the statements to the ASN/MSO, and fill **only still-empty** tags so the file is
+    re-runnable. Done this way for ASN `26720` / PO `S202648137` / MSO `329924` (87 units) on
+    2026-08-13, PR dbchanges#265.
+  - **Known remaining damage from this defect** (each needs its own customer serial→tag list):
+    ASN `26641` / PO `S202668066` — **162 of 363** units still tagless; ASN `26706` /
+    PO `WR270003276` — **1 of 7**.
+  - **A fixture-based regression test is owed with the parser fix**: assert that serial, quantity
+    and assetTag each land in the right field for **both** layouts (do not detect by column count
+    alone). ⚠ Do not quote constants out of `asnprocessor/lenovo.php` into a doc or ticket — that
+    file contains hardcoded FTP credentials.
   - **Recommended future code fixes (proposed, NOT approved, NOT implemented):** map columns
     by **header name** and hard-reject an unknown layout with a loud alert; stop building the
     dedupe key from a field that can be a constant; widen
@@ -266,6 +287,30 @@ Vendor SFTP ───(legacy_import_asn.php, ser+non-ser)─┘      [UNIQUE ded
     `worker/crons/toga2/**` against the toga2 `AdvanceShippingNoticeItemUnits` schema, while
     the DOE path is `Core.AdvanceShippingNoticeUnits` via `asnprocessor/lenovo.php`.
 
+- **⚠ `TOGaDeskSupport.assets.tag` is DERIVED from `Core.AdvanceShippingNoticeUnits.assetTag` — an
+  asset-tag repair MUST touch BOTH stores.** The TOGa Desk UI renders `assets.tag`, but that column
+  is *populated from* the ASN unit's `assetTag` at install-ticket creation
+  (`3_create_installation_ticket.php` ~L150). Fixing only `assets.tag` makes the UI look correct
+  while the ASN record stays blank, so every ASN-driven consumer — reports, NetSuite / ServiceNow
+  transmissions, proof of delivery — still sees no tag. This is the trap that makes a half-fix look
+  like a full fix, and it compounds with the `dbchanges` **cross-database silent skip**: `Core/` and
+  `TOGaDeskSupport/` are different databases on different clusters, so the repair is **two separate
+  runs** and an applier that picks up one directory reports success on a 50% fix (confirmed live
+  2026-08-13: `assets.tag` 87/87 correct while `Core…Units.assetTag` was 87/87 empty). Always
+  verify **per-pair** (serial ↔ tag) in **both** stores, never a non-null count. See the
+  [1.0 dbchanges authoring workflow](../../../1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md).
+
+- **The leading `S` on Lenovo/Lexmark serials is stripped ON PURPOSE — do not "correct" it.**
+  Customer and vendor lists show ThinkPad serials as `SA00HZ20` while the DB stores `A00HZ20`: the
+  importer deliberately strips **one** leading capital `S` for Lenovo and Lexmark
+  (`legacy_process_asn_queue.php` ~L269-274). This is intended normalization, **not corruption** —
+  re-adding the `S` to stored serials would break the dedupe lookups that rely on the stripped form.
+  - Always compare via **`App_NYCDOE::serialKey()`** (`library/app/nycdoe.php` ~L51 =
+    `normalizeSerial()` then `stripLeadingSerialS()`), never a raw string compare.
+  - **Never** `REPLACE(serial,'S','')` — that strips *every* `S`, not just the leading one.
+  - Concretely: 25 of the 87 serials in the 2026-08-13 asset-tag backfill would have silently
+    failed to match without this.
+
 - **The "Outstanding Sales Orders - PO Reconciliation Required" email is FLEET-WIDE, not a
   per-ASN alert** (`2_send_serials_to_netsuite.php:270-357`). It runs at the end of every
   5-minute run and re-queries **all** `AdvanceShippingNoticeItems` with
@@ -518,6 +563,22 @@ use the toga DB MCP + `Logs.API` instead of running prod code locally.
    of the consumer query; `php -l` every touched file.
 
 ## Change history
+- 2026-08-14 — Tied the DOE **missing-asset-tag** class to the existing "Lenovo Off-Layout ASN File"
+  bug: the +1 shift puts the serial in `assetTag` and means the real asset-tag column (`$cols[24]`)
+  is **never read**, so no unit from an off-layout file can get a tag (still **open** — needs the
+  parser fix plus a fixture regression test covering both layouts). Backfilled 87 tags for ASN
+  `26720` / PO `S202648137` / MSO `329924` (client 16, P.S. 235 Lenox School) in **both** stores
+  (`dbchanges/Core/SK/2026-08-13-backfill-asn-asset-tags-S202648137.sql`,
+  `dbchanges/TOGaDeskSupport/SK/2026-08-13-backfill-asset-tags-S202648137.sql`, PR dbchanges#265,
+  branch `fix/backfill-asset-tags-s202648137` → `_production`); SQL generated mechanically from the
+  customer serial→tag CSV, 87/87 verified against the live order before generation and round-trip
+  verified byte-identical, scoped to the ASN/MSO and filling only still-empty tags, so re-runnable.
+  Documented that `TOGaDeskSupport.assets.tag` is **derived** from
+  `Core.AdvanceShippingNoticeUnits.assetTag` (both must be repaired; confirmed live half-fix where
+  the UI was 87/87 right and the ASN 87/87 empty) and that the leading `S` on Lenovo/Lexmark serials
+  is stripped **on purpose** (compare via `App_NYCDOE::serialKey()`; 25/87 depended on it).
+  Remaining damage from the same defect: ASN `26641`/PO `S202668066` (162/363) and ASN `26706`/PO
+  `WR270003276` (1/7). No code changes in `worker`/`library`/`togadesk`. (sking)
 - 2026-08-11 — Investigated (read-only, **no code change**) ASN `26720` / PO `S202648137` never completing: root-caused the **"Lenovo Off-Layout ASN File"** bug — `asnprocessor/lenovo.php` maps columns by COUNT with no header validation, so the 31-column `DOE-Open-Lenovo-*.csv` shifts every field +1, making `serialNumber` the constant `"1"`, which collapses 87 source lines into 3 `completed` queue rows and loses units with no error/alert. Documented the diagnostic fingerprint, the 13-row systemic blast radius, the `varchar(64)` partNumber truncation amplifier, and the **manual** partNumber data repair as the only remediation. Also documented that the "Outstanding Sales Orders - PO Reconciliation Required" email is fleet-wide (and its count inflated), ruled out the Quad ASN work, and left the ASN `26720` items `54672`/`54673` correct-serial units as UNRESOLVED. (mhammontree)
 - 2026-08-06 — Named and cross-referenced the umbrella failure pattern **"Pre-ASN Manual
   Receipt Mismatch"** (aka "Shipment Received Notice") tying together the existing

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.580",
+  "version": "1.0.581",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",