toga-ai 1.0.1202 → 1.0.1204

package/knowledge/1.0/apps/library/features/netsuite-suiteql-api-reference.md

@@ -6,8 +6,8 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-10-19
-owners: ["dfranks", "jcardinal", "kyalamarthi", "ajean", "rgirish", "bala"]
+updated: 2026-10-29
+owners: ["dfranks", "jcardinal", "kyalamarthi", "ajean", "rgirish", "bala", "mhammontree"]
 files:
   - library/app/api/netsuite/rest.php
   - library/ssl/netsuite_ec_key.pem
@@ -19,6 +19,7 @@ related:
   - ../../worker/features/forecast2-netsuite-reconciliation.md
   - ../../../clients/nychh/features/netsuite-inventory-adjustment-fulfillment-link.md
   - ../../worker/features/compass-ma-sales-order-exception-report.md
+  - ../../../clients/nycdoe/features/servicenow-integration.md
 ---
 
 Working reference for the Agilant NetSuite REST/SuiteQL integration (`App_Api_Netsuite_Rest`): auth, SuiteQL mechanics, table schemas, type/status codes, performance; open when writing or debugging a NetSuite query.
@@ -292,6 +293,16 @@ SOAP one). First caller: [Compass MA exception report](../../worker/features/com
 - Returns `[UPPERCASED_PO => [row, ...]]`, row = `id, tranid, otherrefnum, status` (status letter,
   see the codes above). One PO can map to several SOs; the caller picks.
 
+### Receipts by sales order: `listItemReceiptPartsBySalesOrderIds()`
+
+`listItemReceiptPartsBySalesOrderIds(array $salesOrderIds): array` — one SuiteQL per
+`RECEIPT_SALES_ORDER_ID_CHUNK_SIZE` (900) SO ids. Chain: PO lines with `createdfrom` = SO →
+`ItemRcpt` lines with `createdfrom` = PO (`mainline = 'F'`, `item > 0`, `quantity <> 0`). Returns
+rows `so, itemname (BUILTIN.DF(item)), firstreceipt (MIN trandate, YYYY-MM-DD)` per SO + item.
+REST replacement for the SOAP SO → PO → receipt fan-out; first caller is the DOE received flag
+([NYCDOE ServiceNow / ASN Integration](../../../clients/nycdoe/features/servicenow-integration.md) →
+Stage 5 *Received gate*). Checked 2026-10-29 on DOE data: finds every receipt the SOAP chain finds.
+
 ## Gotchas
 
 - **PHP 7.2 prod constraint.** `library`/`worker` run PHP 7.2 (Amazon Linux 1 EB). Banned syntax: arrow functions (`fn() =>`), typed properties (`public int $x`), `??=`, `match()`, numeric separators (`1_000`). Lint with `C:\xampp7\php\php.exe -l` before deploying — `C:\xampp8\php` (PHP 8.0) is for running probes on the laptop only, **not** for compat checking.
@@ -307,6 +318,8 @@ SOAP one). First caller: [Compass MA exception report](../../worker/features/com
 - `../../worker/features/forecast2-netsuite-reconciliation.md` — the audit/trueup tooling that consumes this API.
 
 ## Change history
+- 2026-10-29 — Added `listItemReceiptPartsBySalesOrderIds()` (first receipt date per SO + item,
+  900 SOs per query) for the DOE received flag (mhammontree, TRUE-83394)
 - 2026-10-16 — Added `listSalesOrdersByOtherRefNums()` (batched SO lookup by PO number, handles
   `A + B` / `A / B` combined `otherrefnum`). REST-only, survives the SOAP retirement. (bala, TRUE-82888)
 - 2026-10-01 — Added `nexttransactionlink` as the order → child-document join (`createdfrom` is a

package/knowledge/1.0/apps/worker/workflows/running-a-worker-cron-locally.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: workflow
 status: active
-updated: 2026-10-23
-owners: ["bala"]
+updated: 2026-10-29
+owners: ["bala", "mhammontree"]
 files:
   - worker/_/app/framework.php
   - library/app/framework.php
@@ -68,6 +68,15 @@ Use Laragon's PHP 7.2.33 build (old worker tier). worker1.5 prod runs PHP 8.5, s
 `phpexcel/` does not parse on PHP 8 ([PHP 8.5 compatibility](../../library/features/php-8-5-runtime-compatibility.md)).
 
 ## Gotchas
+- **worker1.5 scripts — four extra traps** (TRUE-83394):
+  1. xampp_8x `php.ini` `include_path` lists `_underscore` before `library`, so `_.php` loads the
+     2.0 bootstrap. Pass `-d include_path=".;C:/WWW/library"`.
+  2. library `_.php` finds the app root from `SCRIPT_FILENAME` — run the file by its real path
+     inside `worker1.5`.
+  3. worker1.5 has no local `vendor/` — run `composer install` first.
+  4. **No `ENVIRONMENT` = prod `config.worker.ini`.** An error then lands in prod V2
+     `Logs.Issue` and auto-creates a ClickUp task (happened 2026-10-29: issue 1A6). Set
+     `ENVIRONMENT` before every run.
 - **`App_Framework_Worker::initialize()` cannot run twice in one process** — the second
   `define('SENTRY_PROJECT', ...)` fatals. One cron run per PHP process; a wrapper script cannot
   require two crons.
@@ -85,6 +94,8 @@ Use Laragon's PHP 7.2.33 build (old worker tier). worker1.5 prod runs PHP 8.5, s
 - [App_Email side effects & test mode](../../library/features/email-test-mode-and-write-free-sends.md)
 
 ## Change history
+- 2026-10-29 — worker1.5 local-run traps: include_path order, SCRIPT_FILENAME, no vendor/, unset
+  ENVIRONMENT → prod Logs.Issue + ClickUp task (mhammontree, TRUE-83394)
 - 2026-10-23 — PhpSpreadsheet clean on PHP 8 (TRUE-83159); worker1.5 prod is 8.5, phpexcel does not parse there (bala)
 - 2026-10-23 — Deprecations no longer exit since library PR #905 (TRUE-83107) (bala)
 - 2026-10-16 — Created from the cron 7 local test harness (prod-pointing configs, Sentry, SMTP,

package/knowledge/2.0/apps/worker2/features/oneuptime-worker2-monitoring.md

@@ -125,6 +125,8 @@ Each measures one Compass/Office Depot (or Compass Canada) integration metric an
   count threshold — this catches the 1.0 cron's
   [poison-order starvation](../../../clients/compass-usa/features/mits-sales-order-transmission-alerting.md)
   **and** the cron not running at all (worker1.5 outage 2026-10-22/23, TRUE-83116 — check that first).
+  The incident title says "backlog is too high" for both rules, even when one order is stuck; triage
+  steps live in the [MITS alerting doc](../../../../clients/compass-usa/features/mits-sales-order-transmission-alerting.md#triage-a-compasssalesordertransmission-alarm).
   **Age is measured from the LATER of the two approval decisions — `GREATEST(stage-1, stage-2)
   dtDecision`** — because an order is eligible to transmit only once **both** stages are approved.
   Measuring from **stage-2 alone is wrong** (source of a false HIGH): Compass **VIP auto-approve stamps
@@ -650,6 +652,10 @@ production deploy.**
   11 pre-existing instances across `Monitor/Compass.php`, `Monitor/Aig.php`, `Monitor/Operations.php`);
   never log it and never write its value into a doc. Moving these to `Config/production.ini` is a
   recognized future improvement, not yet done.
+- **⚠ Never run a monitor method from a local worker2.** The push URL is hardcoded in the method, with
+  no environment check, so a local run (`ENVIRONMENT=dev-<name>` via Apache `SetEnv`,
+  `databaseClient=localhost`) pushes **local DB numbers into the real prod monitor** — it can clear or
+  fake a live incident. To check a monitor's numbers, run its SQL read-only against prod instead.
 - **Registering the cron before the deploy lands fails every run.** A `Core.CronJobs` row for a method
   that is not yet on the instance throws `Call to undefined method _Worker_Monitor_Compass::<Name>()`
   from `Controller/Index.php:375`, because the box still runs the old `/var/app/current/` copy. Register
@@ -687,6 +693,7 @@ production deploy.**
 - [OneUptime 1.0 worker uptime monitoring](../../../1.0/apps/worker/features/oneuptime-worker-uptime-monitoring.md) — the 1.0 push-heartbeat predecessor
 
 ## Change history
+- 2026-10-29 — Gotcha: never run a monitor from local worker2, the hardcoded push URL feeds local numbers into the prod monitor; `CompassSalesOrderTransmission` triage now in the MITS alerting doc (TRUE-83407). (bala)
 - 2026-10-29 — Every OneUptime push retries once (`setAutoRetry`, 1 retry, 2s; `PUSH_RETRY_*` consts on every monitor class); worker1.5 heartbeat too (TRUE-83398). (bala)
 - 2026-10-29 — Gotcha: "stopped reporting" incidents from failed OneUptime pushes (`OneUptime push FAILED.` + `alarm=OK` in `WorkerJobs.output`) and how to tell them from a real outage (TRUE-83380). (bala)
 - 2026-10-27 — `EmailedAsnImport` moved off php-imap2 to `_Component_Email_Inbound_Imap` (webklex), TLS now validated, no ext-imap (TRUE-82933). (jcardinal)

package/knowledge/clients/compass-usa/features/mits-sales-order-transmission-alerting.md

@@ -5,10 +5,11 @@ project: _Underscore
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-10-23
+updated: 2026-10-29
 owners: ["dfranks", "bala", "jcardinal"]
 files:
   - worker1.5/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
+  - worker2/Worker/Monitor/Compass.php
   - worker1.5/crons/toga2/compasscanada/workflow/1_transmit_compassCanada_sales_orders_to_mits.php
   - worker2/Worker/Infrastructure/Errors.php
   - tools/mvc/errors/post.php
@@ -27,7 +28,7 @@ related:
   - ../../../1.0/apps/tools/features/errors-curation-console.md
   - ../../../1.0/apps/worker1.5/workflows/diagnosing-crons-not-running.md
 ---
-How a `1_transmit_..._sales_orders_to_mits.php` failure reaches a human — USA rejections route through Logs Issue/Event, Canada still emails in-cron; open for "add someone to MITS error emails," a stuck transmit backlog, or the poison-order starvation.
+How a `1_transmit_..._sales_orders_to_mits.php` failure reaches a human — USA rejections route through Logs Issue/Event, Canada still emails in-cron; open for "add someone to MITS error emails," a `CompassSalesOrderTransmission` alarm / stuck transmit backlog, or a MITS rejection (AssetType, SalesOrder:Exists, ShippingEmail).
 
 ## Summary
 How a failure in the `1_transmit_..._sales_orders_to_mits.php` cron reaches a human — for **both** Compass tenants, because **they no longer work the same way**.
@@ -69,6 +70,13 @@ Both crons are live on the single worker1.5 server (since TRUE-83078, 2026-10-22
 
 Configuration is **never retroactive**: `shouldNotify()` fires on the first occurrence, on escalation, or when the backed-off reminder interval elapses, and returns false for a window with zero events. The first email lands on the **next live rejection**, not on existing history.
 
+### Triage a `CompassSalesOrderTransmission` alarm
+The OneUptime incident title says "Compass sales order transmission backlog is too high" for **both** alarm rules (`pendingCount > 10` OR oldest pending > 180 min). One order stuck past 180 min fires it with a count of 2 (2026-10-29, TRUE-83407). So:
+1. Run the monitor's SQL (`_Worker_Monitor_Compass::CompassSalesOrderTransmission`, `worker2/Worker/Monitor/Compass.php`) **read-only on prod** to get the stuck order numbers. Never run the monitor itself locally: it pushes local numbers into the real prod monitor ([gotcha](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md#gotchas)).
+2. Rule out a dead cron (next section).
+3. Read the per-order reason in `Logs.Event.errorMessage` for issueKey `COMPASS_SALES_ORDER_MITS_TRANSMIT_REJECTED`. `Logs.Issue.errorMessage` keeps only the **first** message ever raised, not the latest.
+4. Match the reason to a signature below.
+
 ### First check on a backlog alarm: is the cron running at all?
 Before reading rejections, rule out a dead cron. 2026-10-22/23 (TRUE-83116): the `CompassSalesOrderTransmission` alarm fired (16 pending, oldest 836 min) with **zero** new `Logs.Issue` 24 events and MITS returning 200 on every call — the last `Logs.API` call to MITS was 16:10 the day before. The transmit cron was not running at all: the worker1.5 crontab shell made the overlap guard exit every run. Signs: no new Issue 24 events + stale last `Logs.API` call + `c_dtTransmittedToCompass` not moving. Steps: [Diagnosing worker1.5 crons that stopped running](../../../1.0/apps/worker1.5/workflows/diagnosing-crons-not-running.md).
 
@@ -78,9 +86,13 @@ The alert almost never fires on **volume**. It fires because one bad order **abo
 - The driver query uses **`ORDER BY RAND()`**, so *which* orders get processed before the bad one is random each run. A stuck "poison" order therefore blocks a **different** set of good orders every run — good orders can wait days behind one bad one.
 - Throughput ceiling: the cron sleeps **30s per order** (86400s time limit), scheduled every 5 min but overlap-guarded, so it is effectively one long-running instance — ~110 orders/hr.
 
-**Two rejection signatures seen in prod** (Logs.Issue id 24, issueKey `COMPASS_SALES_ORDER_MITS_TRANSMIT_REJECTED`, ~2329 occurrences since 2026-08-03):
+**Rejection signatures seen in prod** (Logs.Issue id 24, issueKey `COMPASS_SALES_ORDER_MITS_TRANSMIT_REJECTED`, ~2329 occurrences since 2026-08-03):
 1. **HTTP 400 `The AssetType field is required`** — a line item's catalog Item has `assetTypeId = NULL`, so the payload `assetType` is null and MITS rejects the **whole order**. This was the active blocker (e.g. item `4U555A-S`; ~10 distinct NULL-assetType items blocked 9 of 11 pending orders).
 2. **HTTP 403 `SalesOrder:Exists`** — MITS already created the order on a prior attempt, but our side never recorded `c_dtTransmittedToCompass`, so the cron **retries it forever**. **Now handled (2026-09-02):** on a 403 `SalesOrder:Exists` the cron stamps `SalesOrders.c_dtTransmittedToCompass`, writes a `Record`, and **continues** to the next order instead of throwing — so an already-created order no longer starves the queue. Root cause of the original loop (why we never stamped the first time) was the `App_ApiTransaction` decode fatal — see the gotcha below.
+3. **HTTP 400 `Your request is not valid! ... The ShippingEmail ...`** — the order's contact has no email. The cron sends `shippingEmail` = the contact's primary email (`Contacts.primaryContactEmailAddressId` → `ContactEmailAddresses`), else the email of the first `Users` row with that `contactId`. Both empty → MITS rejects, the cron throws every 5 min for that order. Seen 2026-10-29: SA139813 and SA139936 (contact 334468). Usual source: the PEOPLE file had no email, so the import made a new user + contact with none while the person's old inactive user still holds it (see [PEOPLE-file user lifecycle](people-file-user-lifecycle.md)). **Fix is data, not code:**
+   1. Confirm the right address with the Compass team (Vanessa). An older inactive `Users` row of the same person often has it.
+   2. Add a `ContactEmailAddresses` row for the contact (`name = 'WORK'`), then set `Contacts.primaryContactEmailAddressId` to it. Example: `dbchanges2/Client_Compass/2026-10-29a - SA139936ContactEmail.sql`.
+   3. The next `*/5` run sends the order. The nightly PEOPLE import does not undo the fix while the file still has no email for that person.
 
 **Deeper root cause of the 2026-08-31 outage — a wrong catalog merge, not just missing data.** The NULL-assetType items were largely created by an ad-hoc cross-client dedup that folded Compass's **199 catalog-1 (store) items into their Agilant (catalog-2) canonical**, so Compass order lines resolved to Agilant items that have no store classification. The same merge fused their VendorItems, which triggered the SQL 1242 abort (see gotchas). Full root cause + reversal playbook: [catalog-fold reversal](../workflows/catalog-fold-reversal.md). Compass keeps two intentional catalogs and the customer→catalog rule is load-bearing — see [Compass item catalogs & duplicate items](./item-catalogs-and-duplicate-items.md).
 
@@ -106,6 +118,7 @@ The worker2 OneUptime monitor now catches this starvation directly: `CompassSale
 - As of 2026-08-04 prod `Logs.IssueEmailAddress` is empty, so this is the **first business-routed issue on the platform**: `processBusinessIssue()` will execute in production for the first time when this issue next fires. Watch that run rather than assuming it works.
 
 ## Change history
+- 2026-10-29 — Alarm triage steps (incident title says "backlog" even for one stuck order; latest reason is in `Logs.Event`); third rejection signature, HTTP 400 `ShippingEmail` from a contact with no email, fixed by data (TRUE-83407). (bala)
 - 2026-10-23 — Crons now on worker1.5 (`*/5` / `*/10`); backlog alarm can mean the cron is not running at all, check that first (TRUE-83116). (bala)
 - 2026-10-06 — Order query no longer joins `Users` on `contactId` (duplicate users caused double POST + false 403 `SalesOrder:Exists` alerts); noted `Logs.API` and the intentional kit config line. (bala)
 

package/knowledge/clients/compass-usa/features/people-file-user-lifecycle.md

@@ -120,7 +120,8 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
 - **`Client_Compass.Users` is ~176,000 rows with no index on `c_hrEmpPersonnelNbr`, `email`, `c_hrEmpUsername`, or `isActive`.** Any per-row lookup against those columns is a full scan — `EXPLAIN` returns `type=ALL`, 176,018 rows, no possible keys (re-confirmed prod 2026-09-02, same on `Client_CompassCanada`). An index migration is recommended and **still not done**. **The blank-client template is not a safe proxy for this schema**: `dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql` declares `UNIQUE KEY email (email)`, and Compass has diverged from it. Check the live tenant, never the template.
 - **⚠ There is NO UNIQUE KEY on `Users.email` in either tenant — the database will silently accept a duplicate email.** Prod-verified 2026-09-04. Complete index list on `Users` in **both** `Client_Compass` and `Client_CompassCanada`: `PRIMARY(id)`, `uuid`, `Users_UNIQUE(clientId, homeClientUserId)`, `contactId`, `Users_Users_FK(supervisorUserId)`, `Users_Addresses_FK(addressId)`, `Users_Images_FK(thumbnailImageId)`, `dtCreated`, `dtUpdated`, `Users_c_supportedByUserId`, `fk_users_defaultCostCenterId`. Also the proof of the missing-index gotcha above. **Consequence: any code that writes an email must do its own cross-row duplicate check** — and because email is a login/reset identity in 2.0, a duplicate is a security problem, not just messy data. See [Users.email is a login identity](../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md).
 - **⚠ The `Users` ↔ `Contacts` email/phone mirror is reached through the PRIMARY pointers, not through `name = 'WORK'`.** `PeopleFile.php:300-310` joins on `Contacts.primaryContactEmailAddressId` / `Contacts.primaryContactPhoneNumberId`. `name = 'WORK'` is only set at **insert**, and there is **no unique key on `(contactId, name)`** — so a contact can hold several `WORK` rows and "the WORK row" is **not deterministic**. Follow the primary pointers; never match on the name.
-- **⚠ The importer re-syncs the `Contacts` child rows UNCONDITIONALLY for every existing user.** `PeopleFile.php:630-632` runs **outside** the `hasDiff` gate, so a `Contacts` email/phone edit made by hand or by a tool is reverted on the next run **even when the `Users` row itself does not diff**. Any out-of-band contact edit is temporary in exactly the same way an out-of-band activation is.
+- **⚠ The importer re-syncs the `Contacts` child rows UNCONDITIONALLY for every existing user.** `PeopleFile.php:630-632` runs **outside** the `hasDiff` gate, so a `Contacts` email/phone edit made by hand or by a tool is reverted on the next run **even when the `Users` row itself does not diff** — **but only when the file carries a value**. `stageContactsForUpdate()` inserts/updates the email only for a non-empty file email, and the cleanup keeps contacts a `Users` row references, so a hand-added email on a person the file sends **without** an email survives.
+- **⚠ A re-hire with no email in the file gets a new user + contact with NO email.** The old inactive user (outside the 5-day window) still holds the address. 2026-10-27: new user 311624 (`southa02`) + contact 334468 with no email, old user 16333 (`southa01`) had it. Downstream break: the MITS transmit cron has no `shippingEmail` and MITS rejects the order — fix and steps in [MITS alerting](mits-sales-order-transmission-alerting.md) (signature 3).
 - **⚠ `executeForTenant()`'s inactivation sweep is the ONLY place in the file that writes `isActive = 0`.** Every other `isActive` path in the ~2,200-line class only forces it back to **1**. Two consequences: (a) **an activation made outside this cron is temporary** — the next run deactivates the person again unless they are in that day's PEOPLE file — while (b) **a deactivation is durable**, because nothing here ever reactivates anyone on its own. This makes the [Tools Compass admin pages](../../../1.0/apps/tools/features/compass-user-persona-admin.md) a one-way-durable tool.
 - **⚠ A user created by hand in the Compass app (by an Admin or Manager) is temporary.** It gets only the **Base** persona and **no `locationId`**, so it sees only Base content (no sector personas, no segment persona). If the person is not in the PEOPLE file, the next 09:16 run sets it inactive (3 cases seen 2026-10-26/27). Fix is to get them into the PEOPLE file, not to re-activate or hand-add personas.
 - **`ensureCrossRegionUsersActive()` runs `UPDATE Users SET isActive = 1 WHERE email = X` with no `isActive` filter**, so it reactivates **every** row carrying a protected address, dead duplicates included. Verified prod 2026-09-02 for the original four, and 2026-09-16 for the first two CA-only addresses: each resolves to exactly **one** row per database, so nothing is being resurrected today — but the query would happily do it if a duplicate appeared. That hardcoded list is also the **only** durable way to hold a cross-region user active, and changing it needs a code change and a deploy per request. **Recommended (not done): scope it to a `Users.id` list instead of matching on email** — matching on email means an email edit silently moves a user in or out of the protected set, which is why the [Tools Compass admin](../../../1.0/apps/tools/features/compass-user-persona-admin.md) blocks the protected addresses from being edited at all.
@@ -147,6 +148,7 @@ The insert-time hook designed 2026-08-12 was built and grew into a reusable engi
 - [Users.email as login identity](../../../2.0/apps/_underscore/features/user-email-as-identity-and-password-reset.md)
 
 ## Change history
+- 2026-10-29 — TRUE-83407: re-hire with no file email gets a new user + contact with no email (breaks MITS `shippingEmail`); a hand-added email survives while the file has none. (bala)
 - 2026-10-29 — TRUE-83392: 10/29 file failed on a split row (`EYERLC01`); fixed the line by hand, re-uploaded, queued one `ProcessPeopleFiles` run (job 2950480, success). Added the bad-row, queue-block and attachment-size gotchas. No code changed. Parked until the problem happens again (no backlog task, by developer choice): optional join-short-lines fix; the attachment size limit; `Client_Compass` user 93605 (`eyerlc01`) has `c_supportedByUserId` NULL after import although the file names supervisor `HILLB17` — not investigated. Also: tell Compass about the line break. (mhammontree)
 - 2026-10-27 — Hand-created app users get only Base + no location and are deactivated by the next run when not in the file (TRUE-83280). (bala)
 - 2026-10-21 — TRUE-82976: fixed the post-insert refresh (`AND isActive = 1 ORDER BY id`, worker2 `e22b3ec`) so new-user roles, personas and supervisor link always go to the newest active row. (bala)

package/knowledge/clients/nycdoe/features/servicenow-integration.md

@@ -10,7 +10,10 @@ updated: 2026-10-29
 owners: [mhammontree, sking, bala, jcardinal, rgirish]
 files:
   - worker1.5/crons/sync/nycdoe/3_create_installation_ticket.php
+  - worker1.5/crons/sync/nycdoe/3a_flag_received_asn_items.php
   - worker1.5/schedules/cron.worker.json
+  - library/app/api/netsuite/rest.php
+  - dbchanges/Core/MH/2026-10-29-asn-items-netsuite-received.sql
   - worker1.5/ebs/cron.worker.php
   - library/app/framework.php
   - worker/crons/sync/nycdoe/import_asn.php
@@ -56,6 +59,7 @@ related:
   - ../../../1.0/apps/worker/architecture.md
   - ../../../1.0/apps/worker1.5/architecture.md
   - ../../../1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md
+  - ../../../1.0/apps/library/features/netsuite-suiteql-api-reference.md
 ---
 
 Full DOE ServiceNow/ASN integration map — ticket mirror, ASN → NetSuite SO/TOGa Desk pipeline, outbound status/POD, EDI; open for any DOE cron/pipeline work or a stranded-ASN/duplicate-fulfillment investigation.
@@ -88,6 +92,7 @@ The NYCDOE/ServiceNow integration mirrors DOE's ServiceNow tickets (Incidents +
 | Queue → ASN records (Stage 2) | `legacy_process_asn_queue.php` | every 15 min |
 | NetSuite SO create (Stage 3, **freeze**) | `1_send_asn_to_netsuite.php` | every 2 h at :07 |
 | NetSuite PO/serials (Stage 4) | `2_send_serials_to_netsuite.php` | every 2 h at :27 |
+| Stage 5 received flag ("DOE/NS Integration - Step 3a Flag Received Lines") | `3a_flag_received_asn_items.php` — stamps `AdvanceShippingNoticeItems.dtNetSuiteReceived` (see Stage 5 → *Received gate*) | every 10 min |
 | TOGa Desk repair orders (Stage 5; schedule name "DOE/NS Integration - Step 3") | `3_create_installation_ticket.php` | `*/5`, but each run lasts ~590 s (own `$stopTime`) + overlap guard → effectively every ~10 min |
 | NetSuite Item Fulfillment (Stage 6) | `4_create_ns_item_fulfillment.php` (**paused**, `active: 0`); `5_DOA_…` unscheduled (moving to `crons/sync/DOA/`, see below) | not running |
 | ETA → ServiceNow | `send_ticket_updates.php` | every 6 min |
@@ -131,8 +136,15 @@ Vendor SFTP ───(legacy_import_asn.php, ser+non-ser)─┘      [UNIQUE ded
 - **Stage 4:** reconciles the NetSuite **PO** (`netSuiteInternalPurchaseOrderId` — not the SO id) and sends serial inventory assignments. Already multi-PO aware (`listPurchaseOrdersCreatedFromSalesOrder`, groups items by PO, stamps each item's `netSuiteInternalPurchaseOrderId`) **but only stamps items where the PO is NULL and never re-stamps** — so an item stamped to PO-A but later *received* on PO-B is never corrected. **TOGA does NOT create NetSuite POs anywhere in the DOE flow.** Stage 3 creates only the SO (and, when needed, a special-order inventory Item); POs are generated inside NetSuite by procurement/special-order. A single SO can therefore be fulfilled across **multiple POs** (back-orders, multi-vendor, re-cut lines) — a legitimate NetSuite outcome, so the durable fix lives downstream (resolve receipts SO-wide), not in preventing PO splits.
 - **Stage 5:** for units `WHERE togadeskRepairOrderId IS NULL`, reads serials from NetSuite item receipts (so TOGa Desk RO serials are **driven by NetSuite**, not the ASN unit), creates/reuses TOGa Desk manufacturers/models/assets/serial-numbers/MSO and creates **a new repair_orders row unconditionally per loop iteration** — one RO per Unit (a 50-cable line = 1 unit = 1 RO). Existing-serial lookup is scoped **per item row** (`$asnItemId`). Item receipts are resolved **Sales-Order-wide** via `App_NetSuite::listItemReceiptsCreatedFromSalesOrder($soId)` (fans across ALL POs from `listPurchaseOrdersCreatedFromSalesOrder`, dedupes receipts by `internalId`), cached **per sales-order internal id** (`$itemReceiptsBySo`, keyed by `netSuiteInternalSalesOrderId`) — a shared SO is still fetched once — with the `getItemReceipt` detail cache kept per (globally-unique) receipt `internalId`. Both the inner item query **and** the outer ASN-candidate query require `netSuiteInternalSalesOrderId <> 0` (a sentinel guard mirroring the PO guard) so a null/zero-SO row cannot poison the cache or match the wrong SO's receipts. Self-healing: units previously stuck under an unstamped PO clear on subsequent 5-min cron runs.
   - **Receipt resolution is SO-ONLY — the stamped PO id never finds a receipt** (traced 2026-07-29, `3_create_installation_ticket.php`: candidate query ~L248-278, per-item ~L311-341, receipt resolution ~L354-360, part-number match ~L418-422, per-unit gate ~L511-535). The chain is strictly `AdvanceShippingNoticeItems.netSuiteInternalSalesOrderId` → `listPurchaseOrdersCreatedFromSalesOrder` (all POs created from that SO) → `listItemReceiptsCreateFromPurchaseOrder` for each → dedupe (this is what `listItemReceiptsCreatedFromSalesOrder` wraps, `library/app/netsuite.php` ~L739 / ~L762 / ~L797, details via `getItemReceipt` ~L1697). **`netSuiteInternalPurchaseOrderId` is NEVER used to look up receipts** — it is only a non-null / non-zero **eligibility gate**. The schema is misleading here: the PO column *looks* authoritative and is not.
-  - **Item receipts are never persisted.** They are fetched **live from NetSuite on every run** (no table, no bridge row), and the receipt-line → ASN-line linkage is derived **at runtime by part-number string match** after stripping the `DOE-` prefix and `_` characters. So NetSuite is the only source of truth for what was received, and a part-number formatting drift breaks the match silently.
-  - **Candidate pick is `ORDER BY RAND() LIMIT 1` ON PURPOSE — do not change it to a fixed order** (rgirish, 2026-10-23). Newest-first was tried and the script stuck for days on one ASN with bad data. Cost: ~1,700–1,800 pending candidate ASNs (1,698 on 2026-10-23, 1,795 on 2026-10-29; oldest 2023-10-02, most will never complete) and only a handful finish per run, so a new, fully received ASN can wait hours or days by chance (2026-10-22: 4 ASNs completed in 7.5 h). Symptom "received in NetSuite, no repair order in Desk" with correct SO/PO stamps = this wait, not a stamp mismatch. A DOE report that lists POs by repair order will miss the PO until then.
+  - **Item receipts are never persisted.** They are fetched **live from NetSuite on every run** (no table, no bridge row), and the receipt-line → ASN-line linkage is derived **at runtime by part-number string match**. So NetSuite is the only source of truth for what was received, and a part-number formatting drift breaks the match silently.
+  - **One shared part-match rule (TRUE-83394):** `App_NYCDOE::receiptPartMatchesAsnPart()` (`library/app/nycdoe.php`) — `receiptPartKey()` (drop every `_`, then strip a **leading** `DOE-`) `===` `asnPartKey()` (strip a leading `DOE-`). Used by Stage 5 and the 3a flag cron; if the two ever disagree, a received line is never picked or never ticketed. Before TRUE-83394 Stage 5 removed `DOE-` anywhere and compared with loose `==`.
+  - **Received gate (TRUE-83394, built 2026-10-29; SQL applied to prod 1.0 Core 2026-10-29, library + worker1.5 deploy together).** `Core.AdvanceShippingNoticeItems.dtNetSuiteReceived` (datetime NULL, + `KEY idx_installTicketReady (installTicketStatus, dtNetSuiteReceived, advanceShippingNoticeId)`) = first NetSuite receipt date for the line's SO + part. NULL = no receipt seen yet.
+    - **Writer:** `3a_flag_received_asn_items.php` (every 10 min) loads pending unstamped lines (same gate as Stage 5), runs one SuiteQL per 900 SOs via `App_Api_Netsuite_Rest::listItemReceiptPartsBySalesOrderIds()` (SO → PO lines `createdfrom` SO → receipt lines `createdfrom` PO; `MIN(trandate)` per SO + item), matches with the shared part rule, stamps. **Never clears a stamp. No watermark** — it rechecks every unstamped line each run, so the first run backfilled and a missed run heals itself. Errors are not caught → `Logs.Issue` (alert on failure).
+    - **Reader:** Stage 5's driver and per-ASN item queries add `dtNetSuiteReceived IS NOT NULL`, **except** 1 pick in 10 (`UNFILTERED_PICK_ONE_IN`, a safety net if the flag is ever wrong) and manual single-ASN runs (no filter).
+    - **SuiteQL finds what SOAP finds** (checked 2026-10-29): all 106,083 DOE receipt lines since 2023-09 have `BUILTIN.DF(item) = item.itemid`, no sub-items; 24,099 POs from DOE SOs, none linked to the SO only at header level.
+    - **Stuck line = received but still pending.** A stamped line that never gets an RO (partial receipt, serial already on another line of the same PO) now gets picked over and over. Planned (Mark, 2026-10-29): after ~1 week live, count pending lines with `dtNetSuiteReceived` older than 30 days; only then open a backlog task. Pre-2025 purge skipped for now.
+  - **Candidate pick is `ORDER BY RAND() LIMIT 1` ON PURPOSE — do not change it to a fixed order** (rgirish, 2026-10-23; kept in TRUE-83394). Newest-first was tried and the script stuck for days on one ASN with bad data; oldest-first would keep hitting stuck lines. Before the received gate the pick ran over ~1,700–1,800 pending ASNs (oldest 2023-10-02), most never received, so a new fully received ASN waited hours or days by chance (2026-10-22: 4 ASNs completed in 7.5 h). Symptom "received in NetSuite, no repair order in Desk" with correct SO/PO stamps = this wait, not a stamp mismatch. After the gate deploys, check `dtNetSuiteReceived` first: NULL → 3a has not seen the receipt (part-key or SO-chain issue); stamped → normal wait.
+    - **Queue makeup 2026-10-29 (why the gate):** 1,764 pending ASNs / 4,598 pending lines / 1,959 SOs. Only 363 lines (923 units) have a NetSuite receipt for their SO + part, nearly all 2026, 200 received < 30 days ago. 4,235 lines have no receipt at all (e.g. ASN 14654 / SO 195834: 4 of 6 lines never got a vendor PO). "Old duplicates" is mostly false — `dbchanges/Core/SK/2026-07-30-dedupe-asn-install-units.sql` (run 2026-08-05) already stamped 31,716 dup units. Pre-2025: 536 ASNs / 1,871 lines; they got 904 tickets only in a burst Jul 9–16 2026, none since.
     - **Default response for ONE such ASN: no action** — no manual run, no code change; let the cron pick it (Brian Yard, 2026-10-29). Run by hand (below) only when asked, or for a batch/outage gap.
     - **Fast triage (SuiteQL, 2 queries):** `transaction` + `transactionline` `WHERE id IN (<SO>,<PO>) OR createdfrom IN (<SO>,<PO>)` shows the SO → PO → ItemRcpt chain at once; `inventoryassignment WHERE transaction = <receipt id>` shows the received serial. PO created from SO + receipt holds the ASN serial = clean chain → it is this wait, not [Pre-ASN Manual Receipt Mismatch](#gotchas).
     - Worked case (TRUE-83394): `WR270047873` = ASN 27376 (Apple, 1 unit, serial `CK7W74WQCG`), late from the TRUE-82940 Stage 1b outage; SO 7686363 → PO 7686364 → receipt 7693049 all clean on 10-28; still no RO on 10-29 → left to the cron.
@@ -161,7 +173,7 @@ Both crons were dead in prod for months before this ticket (see gotcha). Ship **
 - These crons stay on until 2.0 takes over sending — see [profile](../profile.md) → *Platforms & data*.
 
 ### Debugging & DB topology (DOE 1.0 sync)
-The 1.0 worker cannot run on a dev/Windows box; use the toga DB MCP + `Logs.API` instead of running prod code locally.
+Prefer the toga DB MCP + `Logs.API` over running a DOE cron locally — a local run talks to prod by default ([Running a 1.0 worker cron locally](../../../1.0/apps/worker/workflows/running-a-worker-cron-locally.md)).
 - **⚠ `Logs.API` retention is 30 days — SHIPPED 2026-09-28 (TRUE-82049)**; mechanics, storage facts and the not-yet-run backlog purge in [API Request Logging & Retention](../../../1.0/apps/library/features/api-request-logging-and-retention.md) — so it is no longer a *historical* debugging avenue, only a recent-window one. Bugs that burn slowly (a hold not re-pushed for 22 days; 9,914 PATCHes traced back to 2026-08-20) can no longer be reconstructed after the fact and must be caught by **alerting** instead. NYCDOE stays 1.0, so its logs remain in legacy `Logs.API`; Compass/Elite/WJE move their API logging to 2.0.
 - **`repair_orders.dtUpdated` and `.lastupdateddate` are both time-skewed in `TOGaDeskSupport`** — never select or diff on them, and never compensate with a fixed offset. Detail: [Field-Service Dispatch](../../../1.0/apps/togadesk/features/field-service-dispatch.md).
 - **`dbchanges` does not describe the real 1.0 schema** — some live columns were created outside the repo. Verify indexes/columns against the database, not the repo: [Authoring & Shipping a 1.0 dbchanges SQL File](../../../1.0/apps/dbchanges/workflows/authoring-and-shipping-sql-files.md).
@@ -268,7 +280,7 @@ All **live** DOE business logic is 1.0. The only other 2.0 code is the in-progre
         OR (AdvanceShippingNoticeItems.togadeskRepairOrderId IS NULL AND NOT EXISTS (SELECT 1 FROM AdvanceShippingNoticeUnits WHERE AdvanceShippingNoticeUnits.advanceShippingNoticeItemId = AdvanceShippingNoticeItems.id))
       )
     ```
-    Note this only confirms the row is an eligible *candidate* for the `*/5 min` Stage 5 cron (which then picks ONE ASN at random via `ORDER BY RAND() LIMIT 1` across all pending ASNs) — it does not by itself confirm a ticket was created; wait for the cron and check `togadeskRepairOrderId`/`dtInstallTicketStatusChanged` to confirm ticketing completed.
+    Note this is the pre-TRUE-83394 shape (the live query also needs `dtNetSuiteReceived IS NOT NULL` on 9 picks in 10) and only confirms the row is an eligible *candidate* for the Stage 5 cron (which then picks ONE ASN at random via `ORDER BY RAND() LIMIT 1`) — it does not by itself confirm a ticket was created; wait for the cron and check `togadeskRepairOrderId`/`dtInstallTicketStatusChanged` to confirm ticketing completed.
   - Unlike instances 1–2 below, no duplicate/replacement SO was involved here — the mismatch is a same-ASN sibling PO the warehouse had already been using for other lines on this ASN.
 
 - **⚠ OPEN BUG (TRUE-83064, 2026-10-22; data repair COMPLETE 2026-10-23, root-cause code fix NOT done) — one customer PO split into two ASNs (two vendors) → wrong manufacturer + stranded cross-received units.** Customer PO `WR270089579` came in as two ASNs, one under the Acer vendor and one under the Lexmark vendor, for the same Acer part `DOE-NX.JHWAA.004-NYCDOE` (likely the Acer-as-Lexmark SFTP path, Stage 1b above).
@@ -318,6 +330,7 @@ All **live** DOE business logic is 1.0. The only other 2.0 code is the in-progre
 - Local deep-dive (full WR260236464 case study with record IDs): `doe_service_now_integration.md` in Mark's WWW workspace root; also the `service-now-integration` skill (`asn-import-flow.md`)
 
 ## Change history
+- 2026-10-29 — TRUE-83394 (reopened): Stage 5 received gate — `dtNetSuiteReceived` column (SQL applied to prod), new `3a_flag_received_asn_items.php` cron, Step 3 filters on it except 1 in 10, shared part-match rule; queue makeup; SuiteQL = SOAP check; stuck-set count planned after ~1 week. CTO agreed. library + worker1.5 deploy together. (mhammontree)
 - 2026-10-29 — TRUE-83394: `WR270047873` received in NetSuite but no Desk RO = Stage 5 random-pick wait (1,795 candidates); default = no action for one ASN (Brian Yard); SuiteQL chain triage tip. No code change. (mhammontree)
 - 2026-10-27 — TRUE-82940: `apple.php:74` PHP 8.5 outage (10-22 → 27) fixed, not deployed; errors land in V2 `Logs.Issue`; Stage 1b cadence is 15 min (worker1.5); gated mode sends one `RunForFiles` per account per run. (mhammontree)
 - 2026-10-23 — Daily open/closed report row; manual re-run gotcha (duplicate email, 24 h window shift) (TRUE-83159) (bala)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.1202",
+  "version": "1.0.1204",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",