toga-ai 1.0.690 → 1.0.691

package/knowledge/INDEX.md

@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 ## 1.0 framework
 
 - **library** (Library) _(framework core)_ — 20 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
-- **worker** (Worker) — 30 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
+- **worker** (Worker) — 31 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
 - **dbchanges** (Database Changes) _(framework core)_ — 1 doc(s) → [1.0/apps/dbchanges/INDEX.md](1.0/apps/dbchanges/INDEX.md)
 - **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
 - **togadesk** (TOGa Desk) — 13 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)

package/knowledge/clients/compass-usa/INDEX.md

@@ -14,6 +14,7 @@
 | [Compass MR/MA Order Auto-Approval & Status Gate](features/mr-ma-order-approval-and-status.md) | 2.0 | Compass **MR** and **MA** sales orders are system-generated from the MITS / Office Depot EDI pipeline (they do not originate as user-entered SA orders) and must | _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/PurchaseOrder.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php |
 | [Compass ODP EDI 850 Line-Item Resolution (VA part number to IN SKU fallback)](features/odp-edi-850-item-resolution.md) | 1.0 | How each **PO1 line** on an inbound Office Depot (ODP) **EDI 850** is resolved to a real `Client_Compass` catalog item before cron `3a_import_office_depot_purch | worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php, library/app/Edi.php |
 | [Compass Office Depot EDI 855 Acknowledgement + Over-Quantity PO Guard](features/odp-edi-855-acknowledgement-and-overquantity-guard.md) | 1.0 | How Compass acknowledges Office Depot (ODP) inbound **EDI 850** purchase orders with an **X12 855**, and the **over-quantity guard** that rejects a duplicate PO | worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php, worker/crons/toga2/compass/workflow/4_transmit_office_depot_po_acknowledgements.php, library/app/edi.php |
+| [Compass ODP EDI File Retention & Import Retry (cron 3a)](features/odp-edi-file-retention-and-retry.md) | 1.0 | When the Compass Office Depot **850 importer** (cron `3a`, every 5 min) keeps an EDI file in S3 and when it deletes it. | worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php |
 | [Compass USA — Fulfilled vs Partially Fulfilled (order total kept; per-line rule built then rejected)](features/order-fulfillment-status-per-line.md) | 2.0 | TOGa Supply shows **Fulfilled** on Compass USA orders that are only partly shipped. | _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/Canada/SalesOrder.php |
 | [Compass PEOPLE-File User Lifecycle (duplicate accounts, reactivation grace window, raw-SQL deactivation)](features/people-file-user-lifecycle.md) | 2.0 | The nightly **PEOPLE** file cron (`_Worker_Client_Compass_PeopleFile`, `worker2/Worker/Client/Compass/PeopleFile.php`) owns the whole `Users` row lifecycle for | worker2/Worker/Client/Compass/PeopleFile.php |
 | [Persona Model & Levy-Sector Gating (worker2 PEOPLE cron)](features/persona-model-and-levy-gating.md) | 2.0 | Compass USA catalogue visibility is driven by **personas** in `Client_Compass`. | worker2/Worker/Client/Compass/PeopleFile.php |

package/knowledge/clients/compass-usa/features/odp-edi-855-acknowledgement-and-overquantity-guard.md

@@ -6,7 +6,7 @@ project: Worker
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-08-25
+updated: 2026-08-28
 owners: ["jcardinal", "bala"]
 files:
   - worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php
@@ -14,6 +14,7 @@ files:
   - library/app/edi.php
 related:
   - odp-edi-850-item-resolution.md
+  - odp-edi-file-retention-and-retry.md
   - ../workflows/odp-order-pipeline-to-netsuite.md
   - ../workflows/odp-duplicate-po-line-cleanup.md
   - mits-po-transmission-to-vendors.md
@@ -139,6 +140,29 @@ Throwing **after** `cronFinished()` is safe: the overlap guard
 cronFinished's `CronJobExecutions` checkout row, and on CLI an uncaught `App_Exception_Business`
 just echoes `ERROR [ref]:` and exits.
 
+### ⚠ The 855 storm got a second route in — and a second guard (2026-08-28)
+
+Deferring the throw was enough only while **every** file was deleted after one pass. Cron 3a now
+**keeps a failed EDI file in S3 and retries it every 5 minutes**
+([file retention & retry](odp-edi-file-retention-and-retry.md)), so the reject branch is reachable
+again on a later attempt — which would re-send the same reject 855 on every run, the exact storm the
+deferred throw was written to prevent.
+
+Two things close it:
+
+- **`wasPurchaseOrderRejectionAlreadyAcknowledged(string $purchaseOrderNumber): bool`** — the reject
+  855 is now built and sent **only the first time** the guard trips for a PO. A prior send is
+  detected by looking for a `Logs.FileLog` row with `job = 'ODP_EDI'` and
+  `fileName = '<purchaseOrderNumber>-ACK-REJECT.x12'`. **The acknowledgement log row is the idempotency
+  key** — there is no `dtAcknowledged` to lean on here, because a rejected PO is never persisted.
+- **An over-quantity rejection is deliberately NOT treated as an import failure**, so the file is
+  still **deleted** on a reject. The reasoning: a reject 855 has already gone back to Office Depot,
+  so the file is finished with and retrying it would achieve nothing. (The
+  `SKIP_IF_CONTAINS_PART_NUMBERS` skip is treated the same way — an intentional business skip.)
+
+So in practice the reject path still ends in a delete, and the helper is the belt-and-braces guard
+for the case where a **different** PO in the same multi-PO file fails and keeps the file alive.
+
 ## ⚠ Open item — reject codes vs ODP's 855 companion guide
 The reject codes used (`BAK02 = RD`, `ACK01 = IR`) are the **standards-correct** X12 values.
 They have **not** yet been confirmed against Office Depot's 855 companion guide. If ODP
@@ -147,6 +171,13 @@ A legacy, abandoned sketch (`compass/edi/3_send_edi_855(reject).php`) used **non
 codes `BAK01 = A1` / `BAK02 = RE` — that path was **deliberately NOT followed**.
 
 ## Change history
+- 2026-08-28 — Cron 3a now retries a failed EDI file instead of deleting it, which reopened the
+  **855-storm** risk on the reject branch. Added
+  `wasPurchaseOrderRejectionAlreadyAcknowledged()` — keyed on a `Logs.FileLog`
+  `<PO>-ACK-REJECT.x12` row, since a rejected PO is never persisted and has no `dtAcknowledged` — so
+  a reject 855 is sent only on the first trip. Recorded that an over-quantity rejection (and a
+  `SKIP_IF_CONTAINS_PART_NUMBERS` skip) is **deliberately not a failure**, so those files are still
+  deleted. Not deployed as of 2026-08-28. (bala)
 - 2026-08-25 — The over-quantity guard is now fed **resolved** part numbers (it ignores parts it
   cannot match, so a mistyped ODP part number had been bypassing duplicate protection), the
   **reject 855 echoes `ediPartNumber`** (what ODP sent) instead of our corrected value, and the

package/knowledge/clients/compass-usa/features/odp-edi-file-retention-and-retry.md

@@ -0,0 +1,137 @@
+---
+title: Compass ODP EDI File Retention & Import Retry (cron 3a)
+framework: "1.0"
+repo: worker
+project: Worker
+client: compass-usa
+type: client-feature
+status: active
+updated: 2026-08-28
+owners: ["bala"]
+files:
+  - worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php
+related:
+  - odp-edi-855-acknowledgement-and-overquantity-guard.md
+  - odp-edi-850-item-resolution.md
+  - ../workflows/odp-edi-import-recovery.md
+  - ../workflows/odp-order-pipeline-to-netsuite.md
+  - ../profile.md
+---
+
+## Summary
+When the Compass Office Depot **850 importer** (cron `3a`, every 5 min) keeps an EDI file in S3 and
+when it deletes it. Until 2026-08-28 the file was deleted **unconditionally** at the end of the 850
+branch, so **any** import failure silently destroyed the order: nothing written, no acknowledgement,
+cron 5 never built a NetSuite SO, and no file left to retry. 3a now deletes the object **only when
+every purchase order in the file imported end to end**; anything less and the file stays in
+`OfficeDepot/` and the next run (5 min later) tries again.
+
+> **⚠ Status 2026-08-28 — written in the working tree, NOT committed and NOT deployed.** Production
+> still has the unconditional-delete behavior. Treat the silent-data-loss section below as live
+> production behavior until this ships.
+
+## The failure this was built for (production, 2026-08-27)
+
+Office Depot POs `41832546-1214` and `41832551-5910` never reached NetSuite. Between **19:11 and
+19:16** api2 returned **HTTP 500** (envelope code `EO-1`, "Operation Failed: An internal server
+error occurred") to **every** lookup 3a makes: `GET /items`, `GET /vendor-items`,
+`GET /purchase-orders`. Nothing was written, no 855 was produced, and 3a then deleted the EDI file.
+The orders were only recoverable because the raw x12 was archived in `Logs.FileLog`.
+
+This is a **silent data-loss class of bug**, not a one-off: any api2 outage inside the 5-minute
+window destroyed whatever ODP had sent during it.
+
+### ⚠ Where the failure trail actually is (this cost real time)
+
+- **NOT in `Logs_Compass.Api`.** The failing api2 calls do **not** appear in the per-client Api log,
+  only the surrounding successful `POST /v2/auth/api` and `GET /v2/clients` calls do. Do not conclude
+  from an empty `Logs_Compass.Api` that 3a never ran or never called api2.
+- **The reliable trail is the legacy `Logs` schema:** `Logs.Emails` with subject
+  `Office Depot PO Import error: <PO>`, plus `Logs.FileLog` with `job = 'ODP_EDI'`.
+- **Diagnostic shape that identifies this exact failure:** a `Logs.FileLog` row for `<PO>.x12` with
+  **no** matching `<PO>-ACK.x12` row, **and** no `PurchaseOrders` row in `Client_Compass` for that PO
+  number.
+- **The only surviving copy of the EDI is `Logs.FileLog.fileData`.** Recovery is: pull `fileData`,
+  write it back byte for byte, re-upload to the **root** of `OfficeDepot/` (never `OUTBOX/` or
+  `SENT/`, which 3a skips). Full runbook:
+  [Recovering a lost ODP EDI import](../workflows/odp-edi-import-recovery.md).
+
+## How it works
+
+### One flag, one choke point
+Every failure path in 3a already funnelled through **`sendErrorNotification()`**, so that single
+function is the choke point. It now clears the file-scoped flag
+**`$isCurrentEdiFileFullyImported`**, and that flag gates the S3 delete: the object is removed only
+when the flag survives the file, otherwise the PO numbers are pushed onto
+`$retriedPurchaseOrderNumbers` and the file is left in place.
+
+**Why it matters:** there are roughly **25 separate `continue` sites** in the 850 branch. Gating on
+the notification function means no error branch can be missed, and a future error path inherits the
+retry for free as long as it reports through `sendErrorNotification()`. The flag is reset to `true`
+at the top of each file's 850 branch, so every file is judged on its own attempt.
+
+### What is deliberately NOT a failure (the file still deletes)
+- **The over-quantity guard rejection** — a reject EDI 855 has already been sent to Office Depot, so
+  the file is finished with. See
+  [855 Acknowledgement + Over-Quantity Guard](odp-edi-855-acknowledgement-and-overquantity-guard.md).
+- **The `SKIP_IF_CONTAINS_PART_NUMBERS` skip** — an intentional business skip, not an error.
+
+### Supporting guards (retrying breaks things that used to run exactly once)
+
+| Helper / constant | Why it exists |
+|---|---|
+| `wasPurchaseOrderRejectionAlreadyAcknowledged()` | A retried file must not re-send the over-quantity **reject 855** on every run. Detects a prior send by looking for a `Logs.FileLog` row named `<PO>-ACK-REJECT.x12`. This is the **855 storm** the original deferred-throw comment warned about, now reachable a second way. |
+| `wasImportErrorAlreadyEmailedRecently()` + `HOURS_BETWEEN_IMPORT_ERROR_EMAILS = 1` | A permanently failing file (e.g. a part missing from the Compass catalog) would email the team every 5 minutes: **288 times a day**. Checks `Logs.Emails` by subject inside the window and stays quiet. |
+| `isEdiFileAlreadyInFileLog()` + `DAYS_TO_SEARCH_FOR_LOGGED_EDI_FILE = 7` | Keeps exactly **one** `Logs.FileLog` row per file no matter how many attempts it takes. Matches on `job`, `fileName`, `fileData` and a bounded `dtStamp`. |
+
+The file-log name is the run's imported PO numbers joined by `;` plus `.x12`, so a multi-PO 850
+archives under a semicolon-joined name and searching for a single PO number needs a `LIKE`.
+
+### The new business exception
+**`COMPASS_OFFICE_DEPOT_PO_IMPORT_INCOMPLETE`** (`URGENCY_HIGH`) is thrown from
+`$retriedPurchaseOrderNumbers` **after** `App_Framework::cronFinished()` — the same deferred pattern
+the rejection exception uses, and for the same reason (an inline throw unwinds the loop before the
+per-file bookkeeping).
+
+**⚠ Only one throw can win.** It is raised **after** the existing
+`COMPASS_OFFICE_DEPOT_PO_IMPORT_REJECTED_OVER_QUANTITY` throw, deliberately: a rejection is the more
+serious event, so when both happen in one run the rejection is the one that surfaces as a Logs.Issue.
+
+## Why retrying is safe (verified against the code before shipping)
+- The existing **"do we already have this PO"** check (`GET /purchase-orders` by number **and**
+  `vendorId` = Office Depot) makes a re-import **idempotent**.
+- It also keeps the **over-quantity guard from firing on a PO a previous attempt already created**,
+  because the guard sits *inside* the `totalRecordCount == 0` branch — an already-imported PO never
+  reaches it.
+- Sales-order items are matched by **part number** against the existing SO, so a partial import
+  backfills rather than duplicates.
+
+This is the same idempotency map recorded in the
+[ODP pipeline doc](../workflows/odp-order-pipeline-to-netsuite.md); the retry simply automates the
+re-drop that used to be a manual recovery.
+
+## Gotchas / known issues
+- **⚠ A file that can never import now retries forever.** No cap and no park-after-N behavior was
+  added. Unimportable files accumulate in `OfficeDepot/`, lengthening every run against the cron's
+  **900-second ceiling** — and that folder is already the subject of the unfixed 138k-object listing
+  problem. Flagged to the developer; **not yet decided**.
+- **⚠ Do not deploy with `EMAIL_TO` narrowed.** As observed in the working tree on 2026-08-28,
+  `EMAIL_TO` in this cron had been reduced locally to a **single** address instead of the three team
+  addresses. That is a local debugging edit, not a change made by this work — restore the full list
+  before shipping.
+- Recovery timing: re-importing an old EDI file restarts cron 5's **30-minute** release clock,
+  because that clock runs from the Office Depot PO's `dtCreated`. A recovered order waits another
+  half hour. See the cron 5 section of the
+  [ODP pipeline doc](../workflows/odp-order-pipeline-to-netsuite.md).
+
+## Change history
+- 2026-08-28 — Built: 3a now deletes an EDI file **only when every PO in it imported end to end**,
+  gated by `$isCurrentEdiFileFullyImported` cleared inside `sendErrorNotification()` (the one choke
+  point all ~25 error paths already used). Over-quantity rejects and `SKIP_IF_CONTAINS_PART_NUMBERS`
+  skips still delete. Added `wasPurchaseOrderRejectionAlreadyAcknowledged()` (no repeat reject 855),
+  `wasImportErrorAlreadyEmailedRecently()` + `HOURS_BETWEEN_IMPORT_ERROR_EMAILS = 1` (no 288
+  emails/day), `isEdiFileAlreadyInFileLog()` + `DAYS_TO_SEARCH_FOR_LOGGED_EDI_FILE = 7` (one file-log
+  row per file), and the deferred `COMPASS_OFFICE_DEPOT_PO_IMPORT_INCOMPLETE` exception. Recorded the
+  2026-08-27 api2 500 (`EO-1`) incident that destroyed POs 41832546-1214 / 41832551-5910, and that
+  those api2 failures are **absent from `Logs_Compass.Api`** — the trail is `Logs.Emails` +
+  `Logs.FileLog`. **Not committed, not deployed.** (bala)

package/knowledge/clients/compass-usa/profile.md

@@ -41,6 +41,7 @@ related:
   - workflows/odp-duplicate-po-line-cleanup.md
   - features/odp-edi-850-item-resolution.md
   - features/odp-edi-855-acknowledgement-and-overquantity-guard.md
+  - features/odp-edi-file-retention-and-retry.md
   - ../../2.0/apps/worker2/features/compass-vip-support-importer.md
   - ../../2.0/apps/toga2-commerce/features/expedited-shipping-gating.md
   - ../../2.0/apps/toga2-commerce/features/category-tile-sort-order.md
@@ -113,6 +114,18 @@ separate, related client (see its own profile).
   [ODP EDI 850 Line-Item Resolution](features/odp-edi-850-item-resolution.md).
   **Never create an `Items` row so a mangled inbound part number "matches"** — that is what put
   duplicate junk items in the Agilant catalog (`ODTABKITTINGOPT3` vs `ODPTABKITTINGOPT3`).
+- **⚠ A failed 850 import silently destroys the order (2026-08-28, fix NOT deployed).** Cron 3a
+  deleted the EDI file from S3 **unconditionally**, so an api2 outage (HTTP 500 / `EO-1`, 2026-08-27)
+  meant ODP POs `41832546-1214` / `41832551-5910` were never imported, never acknowledged, never
+  reached NetSuite — and the only surviving copy was `Logs.FileLog`. The fix keeps the file and
+  retries it every 5 minutes until every PO in it imports. **These api2 failures do not appear in
+  `Logs_Compass.Api`** — use `Logs.Emails` + `Logs.FileLog`:
+  [ODP EDI File Retention & Import Retry](features/odp-edi-file-retention-and-retry.md).
+- **Office Depot only 850s back what they buy FROM us (2026-08-28).** A Compass SO can mix a PO ODP
+  buys from Toga (returns an 850) with one Toga buys from ODP (never returns one) — both carry
+  `vendorId` = Office Depot. Mixed orders can therefore only ever be released to NetSuite by cron 5's
+  **30-minute timeout**, so they are consistently half an hour late by design, not by fault. See the
+  release gate in [ODP order pipeline to NetSuite](workflows/odp-order-pipeline-to-netsuite.md).
 - ASN ingestion entry points: cXML to the V2 API (logged in `Logs_Compass.Api`) and
   `worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php`.
 

package/knowledge/clients/compass-usa/workflows/odp-edi-import-recovery.md

@@ -6,13 +6,14 @@ project: Worker
 client: compass-usa
 type: workflow
 status: active
-updated: 2026-08-04
-owners: ["dfranks"]
+updated: 2026-08-28
+owners: ["dfranks", "bala"]
 files:
   - worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php
   - worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php
   - worker/schedules/cron.worker.sync.json
 related:
+  - clients/compass-usa/features/odp-edi-file-retention-and-retry.md
   - clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md
   - clients/compass-usa/workflows/order-lifecycle-and-data-integrity.md
   - ../../../2.0/apps/api2/features/api-payload-interceptors.md
@@ -30,12 +31,43 @@ The recovery works because the raw x12 is archived in `Logs.FileLog` and **3a is
 re-uploading the file makes it repair the items, the ODP PO, and both bridges. Verified end-to-end
 in **production** on 2026-08-04 for ODP PO `41608261-1135` / ODP SO `475390815001`.
 
+The same runbook covers the **total** loss case, which is more common than the partial one: 3a fails
+**before writing anything** (e.g. api2 is down), deletes the file anyway, and the order simply never
+exists — no zero-item order, no ODP PO, nothing to sweep for. Production example 2026-08-27, ODP POs
+`41832546-1214` and `41832551-5910`, when api2 answered **HTTP 500 / `EO-1`** to every lookup between
+19:11 and 19:16.
+
+> **Once the conditional-delete change ships, most of this becomes unnecessary** — 3a keeps a failed
+> file in `OfficeDepot/` and retries it every 5 minutes on its own. See
+> [ODP EDI File Retention & Import Retry](../features/odp-edi-file-retention-and-retry.md). It was
+> **not deployed as of 2026-08-28**, so this runbook is still live, and it stays the recovery path
+> for anything lost before the deploy.
+
 ## When to use this
 
 Symptoms: a zero-item ODP SalesOrder (see the orphan sweeps in
 [order lifecycle & data integrity](order-lifecycle-and-data-integrity.md)), and cron 5 re-attempting
 the NetSuite SO every 10 minutes because it selects on `c_dtTransmittedToNetsuite IS NULL`.
 
+### The diagnostic shape that identifies a destroyed import
+
+For the total-loss case there is no order to notice, so match on the **file log** instead:
+
+- a `Logs.FileLog` row (`job = 'ODP_EDI'`) for **`<odpPoNumber>.x12`**, **and**
+- **no** matching **`<odpPoNumber>-ACK.x12`** row (nothing was ever acknowledged), **and**
+- **no** `PurchaseOrders` row in `Client_Compass` for that PO number.
+
+That combination means the 850 arrived, was archived, and then died — the file is already gone from
+S3 and the file-log copy is the only one left.
+
+### ⚠ `Logs_Compass.Api` is NOT where these failures show up
+
+The failing api2 calls 3a makes (`GET /items`, `GET /vendor-items`, `GET /purchase-orders`) were
+**absent** from the per-client `Logs_Compass.Api` log in the 2026-08-27 incident — only the
+surrounding successful `POST /v2/auth/api` and `GET /v2/clients` calls appear. **Do not** read an
+empty `Logs_Compass.Api` as "3a never ran." The reliable trail is **`Logs.Emails`** (subject
+`Office Depot PO Import error: <PO>`) plus **`Logs.FileLog`** (`job = 'ODP_EDI'`).
+
 ## The runbook
 
 ### Step 1 — retrieve the archived x12
@@ -55,13 +87,20 @@ you have a corrupted payload — do not upload it.
 
 ### Step 2 — know that S3 has nothing left
 
-3a **deletes the S3 object after processing** (`:769`), so a failed import leaves no file to retry.
-Recovery is a **re-upload**, to:
+3a **deletes the S3 object after processing**, so a failed import leaves no file to retry. (After
+the conditional-delete change ships this is only true of a *successful* import — see the note in the
+Summary.) Recovery is a **re-upload**, to the **root** of the prefix:
 
 ```
 s3://agilant-as2/OfficeDepot/EDI_<odpPoNumber>.txt
 ```
 
+**⚠ The root of `OfficeDepot/`, never `OfficeDepot/OUTBOX/` or `OfficeDepot/SENT/`.** 3a skips both
+of those prefixes when processing, so a file dropped there is archived forever and never imported.
+
+Write the bytes back **exactly** as stored: pull `fileData`, write it to disk unmodified, and confirm
+the byte count against `LENGTH(fileData)` before uploading.
+
 ### Step 3 — clear the cause BEFORE re-uploading
 
 **Confirm no `ApiPayloadInterceptors` row exists for the `sales-order-items` record** (`recordId 15`)
@@ -90,7 +129,11 @@ confirmation the recovery finished.
 ## Gotchas / known issues
 
 - **⚠ Re-uploading before fixing the root cause destroys the file again** (3a deletes on the way
-  through). Step 3 is not optional.
+  through). Step 3 is not optional. This is exactly what the conditional-delete change removes —
+  until it deploys, assume one attempt per recovered file.
+- **⚠ A recovered order waits another 30 minutes.** Cron 5's release timer runs from the Office
+  Depot PO's `dtCreated`, i.e. **when we imported the 850** — so re-importing restarts it. See the
+  release gate in the [ODP pipeline doc](odp-order-pipeline-to-netsuite.md).
 - **⚠ The archive filename extension differs from the S3 one** — `.x12` in `Logs.FileLog`,
   `EDI_<po>.txt` in S3. Searching for the wrong one finds nothing.
 - **Don't search for the ODP SalesOrder by the `BEG` number.** `BEG` is the ODP *PO*; the SO number
@@ -103,6 +146,14 @@ confirmation the recovery finished.
 
 ## Change history
 
+- 2026-08-28 — Added the **total-loss** case (3a fails before writing anything, so there is no
+  zero-item order to sweep for) with the 2026-08-27 api2 **500 / `EO-1`** incident that destroyed ODP
+  POs `41832546-1214` / `41832551-5910`; the **file-log diagnostic shape** (`<PO>.x12` present, no
+  `<PO>-ACK.x12`, no `PurchaseOrders` row); the warning that these api2 failures are **absent from
+  `Logs_Compass.Api`** so `Logs.Emails` + `Logs.FileLog` are the real trail; the **re-upload to the
+  root of `OfficeDepot/`, never `OUTBOX/` or `SENT/`** rule; and the note that cron 5's 30-minute
+  timer restarts on re-import. Flagged that 3a's **conditional delete + auto-retry** (written
+  2026-08-28, not deployed) largely supersedes this runbook once it ships. (bala)
 - 2026-08-04 — Documented the runbook after recovering a production ODP order whose item write was
   killed by an api2 fatal: `Logs.FileLog` (`job = 'ODP_EDI'`, `<po>.x12`, bounded by `fileTimestamp`)
   as the x12 archive, the trailing-newline/byte-count check, the re-upload path to

package/knowledge/clients/compass-usa/workflows/odp-order-pipeline-to-netsuite.md

@@ -6,7 +6,7 @@ project: Worker
 client: compass-usa
 type: workflow
 status: active
-updated: 2026-08-25
+updated: 2026-08-28
 owners: ["rgirish", "bala", "dfranks", "jcardinal", "mhammontree"]
 files:
   - worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
@@ -19,6 +19,7 @@ files:
   - library/app/client/compass.php
 related:
   - 1.0/apps/worker/features/netsuite-sales-order-sales-rep-sourcing.md
+  - clients/compass-usa/features/odp-edi-file-retention-and-retry.md
   - clients/compass-usa/features/odp-edi-850-item-resolution.md
   - clients/compass-usa/features/odp-edi-855-acknowledgement-and-overquantity-guard.md
   - 1.0/apps/worker/workflows/tracing-a-worker-cron-run-in-production.md
@@ -67,8 +68,18 @@ ODP SalesOrder (customerId = 1)
 3. **`workflow/3a_import_office_depot_purchase_orders.php`** — pulls the ODP **850** from S3 bucket
    `agilant-as2`, prefix `OfficeDepot/` (excluding `OUTBOX/` and `SENT/`; `:46` S3 read,
    `:54-58` prefix filters) and creates the downstream **ODP SalesOrder** (customerId=1 = Office
-   Depot). **Deletes the S3 object after processing** (`:769`) — so an absent S3 object is expected
-   once ingested, not evidence of a miss, *and* a failed import leaves nothing to retry.
+   Depot). **Deletes the S3 object after processing** — so an absent S3 object is expected once
+   ingested, not evidence of a miss.
+
+   > **⚠ Corrected 2026-08-28 — the delete is now CONDITIONAL.** 3a used to delete the object
+   > **unconditionally**, so a failed import left nothing to retry and **silently destroyed the
+   > order**. It now deletes only when **every PO in the file imported end to end**; otherwise the
+   > file stays in `OfficeDepot/` and the next `*/5` run retries it. Which failures count, the
+   > reject-855 / error-email / file-log guards that retrying made necessary, and the
+   > **retries-forever** limitation:
+   > [ODP EDI File Retention & Import Retry](../features/odp-edi-file-retention-and-retry.md).
+   > **Not deployed as of 2026-08-28** — production still deletes unconditionally, so treat any
+   > pre-deploy incident as a lost file.
 
    > **⚠ Correction (2026-08-04): `edi/1_download_edi_s3_create_po_toga.php` is DEAD CODE.** It
    > appears in **no** schedule file. The scheduled entry *named* "Download EDI From S3 Files &
@@ -148,13 +159,72 @@ Cron 3a resolves each line through that SKU when the `VA` text does not match th
    **onto the ODP SalesOrder (customerId=1), never the Compass SO (customerId=2)**.
    **Exclusion filters** (an order legitimately waiting is not a bug):
    - `CompassSalesOrders.number NOT LIKE 'MA%'` — MA orders are excluded.
-   - **computer-kit orders** (Bundles 187–196) wait for a **2nd PO** before syncing.
-   - a **30-minute delay** after the first ODP PO is created.
+   - the two-part **release gate** below (this is the "30-minute delay").
+   - **computer-kit orders** (Bundles 187–196) wait for a **2nd PO** before syncing. This one is
+     **not** in the `WHERE` clause — it is a per-row check run in PHP *after* the query
+     (`Bundles.number IN (192, 187, 188, 195, 193, 189, 194, 190, 191, 196)`, then a count of that
+     Compass SO's POs), so such an order is selected and then skipped.
 
    Because the selection is only `c_dtTransmittedToNetsuite IS NULL` (plus `number NOT LIKE 'MA%'`),
    a **zero-item ODP SalesOrder sits in this queue retrying every 10 minutes** until its lines are
    restored — the retry loop is the symptom, not the cause.
 
+### How cron 5 decides a Compass order is ready for NetSuite (the release gate)
+
+The query groups by **`CompassSalesOrders.id`** and releases the order when **either** condition
+holds (verified against the cron source 2026-08-28, the `WHERE` block around `:90-115`):
+
+- **(a) Everything ODP owes us has come back.** The count of Compass POs on that Compass SO with
+  `vendorId` = Office Depot that have **nothing chained back to them** is **zero** — the test in the
+  SQL is `OfficeDepotSO_PO.id IS NULL`.
+- **(b) OR the 30-minute timeout.** The **earliest** Office Depot PO chained to that Compass SO was
+  created at least **30 minutes** ago (`IFNULL(MIN(OfficeDepotPO.dtCreated), NOW()) <=
+  DATE_SUB(NOW(), INTERVAL 30 MINUTE)`), in which case it transmits with whatever has come back.
+
+**Cron 5 never opens an EDI file and never looks in S3.** It infers that the 850 arrived purely from
+the **link rows cron 3a writes**: Compass PO → `PurchaseOrders_SalesOrders` → Office Depot SO →
+`SalesOrders_PurchaseOrders` → Office Depot PO. If 3a did not write the bridges, cron 5 cannot tell
+the difference between "ODP has not answered yet" and "the import died."
+
+**⚠ The 30-minute clock runs from the Office Depot PO's `dtCreated`** — i.e. **the moment we
+imported the 850**, not when the Compass order was raised. So re-importing an old EDI file
+**restarts the timer**, and a recovered order waits another 30 minutes before it can transmit.
+
+**⚠ A late 850 produces a SECOND NetSuite sales order.** If a PO's 850 arrives *after* the Compass
+SO has already transmitted, the new ODP SO is picked up on its own and creates a **separate**
+NetSuite SO rather than joining the first.
+
+Fields on `Client_Compass.SalesOrders`: `c_dtTransmittedToNetsuite`, `c_netsuiteInternalSalesOrderId`.
+
+**⚠ Do not infer the schedule from transmission times.** Transmissions cluster at **:00 and :30**,
+which reads like a 30-minute cron — it is not. The schedule is **`*/10`** (re-verified in
+`worker/schedules/cron.worker.sync.json` on 2026-08-28); the clustering is the 30-minute **release
+gate** above, not the cadence.
+
+### ⚠ Office Depot only sends an 850 back for what they buy FROM us
+
+Confirmed by the developer with Office Depot (2026-08-28). A single Compass sales order can carry
+**two different kinds** of Compass purchase order, and **both** have `vendorId` = Office Depot:
+
+| Direction | Example | Does an 850 come back? |
+|---|---|---|
+| **ODP buys FROM Toga** (we supply the item) | the laptop | **Yes** — 3a turns it into an ODP SalesOrder + PurchaseOrder |
+| **Toga buys FROM ODP** (they supply the item) | accessories | **No** — nothing ever comes back |
+
+Worked example (2026-08-28): Compass SO **`SA136541`** carried `50311948-1` (laptop `C40QYUC`,
+returned as ODP PO `41832546-1214`) **and** `50311949-1` (accessories — HDMI adapter, cable, dock,
+mouse — which never returns).
+
+**Consequence:** on any Compass SO that **mixes** the two kinds, release condition **(a) can never be
+satisfied**, because the buy-from-ODP PO permanently has nothing chained to it. Those orders are
+**always** released by the 30-minute timeout instead, so they reach NetSuite **half an hour late,
+every single time**. Nothing is lost — but a "why is this order always 30 minutes behind?" ticket
+is this, not a fault.
+
+**Open improvement (not done):** narrow condition (a) to count only the POs Office Depot actually
+buys from us. **The distinguishing field in the data has not been identified yet** — `vendorId` is
+the same for both, so this needs a data investigation before any code change.
+
 ### Cron 5 sales rep — sourced from the end customer, not hardcoded (TRUE-80451)
 
 The SO `entity` is the **parent** customer **35581 "ODP Veyer (B2B)"**; the real buying account
@@ -272,6 +342,18 @@ wrong record makes every order look "stuck." Use the **join**, never a number ma
   transmission feature.)
 
 ## Change history
+- 2026-08-28 — **Cron 3a's S3 delete is no longer unconditional** (it destroyed the order on any
+  failure; corrected the step-3 note) — details in the new
+  [ODP EDI File Retention & Import Retry](../features/odp-edi-file-retention-and-retry.md), which
+  also records the 2026-08-27 api2 500 incident and that those failures are **absent from
+  `Logs_Compass.Api`**. Documented **cron 5's actual release gate** (`OfficeDepotSO_PO.id IS NULL`
+  count = 0 **OR** earliest ODP PO `dtCreated` older than 30 min), that cron 5 reads only the
+  bridge rows and never S3, that the 30-min clock starts at **our import** (so a recovery waits
+  again), that a late 850 makes a **second** NetSuite SO, and that the :00/:30 clustering is the
+  gate and **not** the schedule (still `*/10`). Recorded the ODP business rule that **an 850 only
+  comes back for what ODP buys FROM us**, so mixed orders (e.g. `SA136541`) can only ever release on
+  the timeout. Corrected the Bundles 187–196 filter as a post-query PHP check, not a `WHERE`
+  clause. (bala)
 - 2026-08-25 — Cron 3a now **resolves every 850 line to a catalog item before it writes anything**
   (a bad line used to leave a half-built ODP SO); documented the `PO1` two-identifier detail
   (`elements[7]` = drifting `VA` text, `elements[9]` = ODP's reliable `IN` SKU) behind the new

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.690",
+  "version": "1.0.691",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",