toga-ai 1.0.461 → 1.0.462

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -13,6 +13,7 @@
 | [_Cloud S3 helpers (copy / get / delete / list)](features/cloud-s3-helpers.md) | `_Cloud` centralizes AWS SDK S3 usage for the 2.0 stack so the `S3Client` never leaks into workers or app code. | _underscore/Cloud.php |
 | [_Component_*/_Model_* project-namespace registration (autoloader) & backslash-qualify traps](features/component-model-namespace-registration.md) | Every **project-local** `_Component_*` and `_Model_*` class in a 2.0 app **must declare the project namespace** at the top of the file: ```php namespace <NAMESP | _underscore/Loader.php, worker2/_.php, api2/_.php, worker2/Component/Forecast/Db/Db.php, worker2/Component/Forecast/SaleImport/SaleImport.php, api2/Component/Api/Netsuite/Netsuite.php |
 | [Re-pointing a DB alias mid-request (_Database::register park/restore)](features/database-alias-repointing.md) | `_Database` keys **all live per-database runtime state by the connection ALIAS** (`Client` / `_underscore::DB_CLIENT`, `ClientLogs`, `Archive`), **not** by the | _underscore/Database.php, _underscore/Query.php, api2/Component/Api/V2/V2.php, api2/Component/Api/CrossClient/CrossClient.php |
+| [2.0 Email Send Pipeline (queue + Send worker)](features/email-send-pipeline.md) | In 2.0, `_Email::send()` **does not transmit** — it queues the message. | _underscore/Email.php, worker2/Worker/Infrastructure/Email/Send.php |
 | [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
 | [Error Reporting — Issue/Event Aggregation (agreed POST-to-receiver design)](features/error-reporting-issue-event.md) | Platform-wide error-reporting infrastructure for TOGA 2.0, built around a two-table **Issue / Event** aggregation model in the shared **Core Logs DB**. | _underscore/Error.php, _underscore/Model/Core/Logs/Issue.php, _underscore/Model/Core/Logs/Event.php, dbchanges2/Logs/2026-07-06 - Issue and Event tables.sql |
 | [Record-Changed Event Publishing (_Event::publish to SQS)](features/event-publish-sqs.md) | `_Event::publish()` (in `_underscore/Event.php`) is the PHP side of the real-time event pipeline. | _underscore/Event.php |

package/knowledge/2.0/apps/_underscore/features/email-send-pipeline.md

@@ -0,0 +1,69 @@
+---
+title: 2.0 Email Send Pipeline (queue + Send worker)
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: feature
+status: active
+updated: 2026-07-28
+owners: ["bala"]
+files:
+  - _underscore/Email.php
+  - worker2/Worker/Infrastructure/Email/Send.php
+related:
+  - email-template-sending.md
+---
+
+## Summary
+
+In 2.0, `_Email::send()` **does not transmit** — it queues the message. It writes a `PENDING`
+row into `Logs_[Client].Email` (the `body` column is `mediumtext`) and stores any attachments as
+BLOBs in `Logs_[Client].EmailAttachment`, then returns. A separate cron worker action,
+`Infrastructure/Email/Send` (`_Worker_Infrastructure_Email_Send::Run`), scheduled in
+`Core.CronJobs` at `* * * * *` (every minute), actually sends the `PENDING` rows via **AWS SES
+SMTP** using PHPMailer, then flips each row to `SENT` or `FAILED`. So a 2.0 email lands within
+~1 minute of the `send()` call — **not instantly**.
+
+## Key files / entry points
+
+- `_underscore/Email.php` — `_Email::send()` builds/validates the PHPMailer message, writes the
+  `PENDING` `Logs_[Client].Email` row (+ attachment BLOBs), and enqueues the send. It **throws**
+  unless both `setClientIdentifier(...)` and a From address (`fromEmailAddress`) are set. The
+  `Logs_[Client]` databases live on a separate cluster (`production-logs-*`), so `send()` points
+  `DB_CLIENT_LOGS` at the correct `Logs_[Client]` schema before inserting.
+- `worker2/Worker/Infrastructure/Email/Send.php` — `Run()` fetches up to 100 `PENDING` rows for
+  the client, and `sendEmail()` transmits each via SES SMTP. Constants: `STATUS_PENDING`,
+  `STATUS_SENT`, `STATUS_FAILED`, `MAX_RETRIES`.
+
+## How it works
+
+1. A caller (model, interceptor, worker) builds `_Email`, sets client identifier + From, adds
+   recipients/subject/body, and calls `send()`.
+2. `send()` inserts a `PENDING` `Logs_[Client].Email` row (attachments → `EmailAttachment` BLOBs)
+   and returns — nothing is transmitted yet.
+3. Every minute, `Infrastructure/Email/Send::Run` selects up to 100 `PENDING` rows and sends each
+   through PHPMailer over SES SMTP.
+4. On success the row flips to `SENT`; on failure `retryCount` is incremented and the row is
+   retried on subsequent runs until `MAX_RETRIES` (3), after which it is marked `FAILED`.
+
+## Gotchas / known issues
+
+- **~1-minute latency, not instant.** Anything that assumes an email is sent synchronously at the
+  `send()` call is wrong — transmission happens on the next Send-worker tick.
+- **The Send worker forces HTML mode.** `sendEmail()` calls `$mailer->IsHTML(true)`
+  **unconditionally**, ignoring the sender's `setIsHtml(false)`. A caller that queued a
+  "plain-text" message is still transmitted HTML-mode. Author bodies accordingly.
+- **⚠ SECURITY — hardcoded AWS SES SMTP credentials in `_underscore/Email.php`.** The
+  `SMTP_USERNAME` / `SMTP_PASSWORD` class constants hold **real** SES SMTP credentials committed
+  in source, violating the no-secrets-in-code rule. Documenting the **location only** — do not
+  copy the values. Remediation: treat as compromised, rotate in AWS SES, and move them into
+  `Config/[environment].ini` (read via `_Config`). Also tracked on
+  [`email-template-sending.md`](email-template-sending.md).
+
+## Change history
+- 2026-07-28 — Created: documented that `_Email::send()` **queues** (writes a `PENDING`
+  `Logs_[Client].Email` row + attachment BLOBs) rather than transmitting, and that the worker2
+  `Infrastructure/Email/Send` cron (`Core.CronJobs` `* * * * *`) does the actual SES SMTP send
+  with `MAX_RETRIES` (3) → `SENT`/`FAILED`, giving ~1-minute delivery latency. Noted the Send
+  worker's unconditional `IsHTML(true)` override and the hardcoded SES SMTP creds location. (bala)

package/knowledge/2.0/apps/_underscore/features/email-template-sending.md

@@ -6,13 +6,14 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-07-16
+updated: 2026-07-28
 owners: ["jcardinal", "bala", "mhammontree"]
 files:
   - _underscore/Model/Client/EmailTemplate.php
   - _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php
   - _underscore/Email.php
 related:
+  - email-send-pipeline.md
   - ../../worker2/features/notification-email.md
   - ../../worker2/features/notification-email-template.md
 ---
@@ -86,6 +87,13 @@ can keep using `sendEmail($api, ...)`.
 
 ## Gotchas / known issues
 
+- **`_Email::send()` QUEUES, it does not directly transmit.** `send()` writes a `PENDING` row into
+  `Logs_[Client].Email` (+ attachment BLOBs) and returns; the actual PHPMailer/SES SMTP send,
+  retries, and the `SENT`/`FAILED` outcome happen ~1 minute later in the worker2
+  `Infrastructure/Email/Send` cron. So an email is **not** sent synchronously at the `send()` call,
+  and the "throws when `PHPMailer::Send()` returns false" behavior described under *Failure
+  surfacing* below reflects the pre-queue path — final delivery success/failure is now decided by
+  the Send worker. See [`email-send-pipeline.md`](email-send-pipeline.md) for the full pipeline.
 - **Use `send()` from any non-API context (workers, cron, internal code).** Before
   2026-06-15 the only entry point was `sendEmail(&$api, ...)`, so callers with no API
   context faked one: `$api = (object)['client' => (object)['clientIdentifier' => …]]`.
@@ -138,6 +146,10 @@ worker method) in-process instead.
 
 ## Change history
 
+- 2026-07-28 — Clarified that `_Email::send()` **queues** a `PENDING` `Logs_[Client].Email` row
+  rather than transmitting; the actual SES SMTP send/retry/`SENT`/`FAILED` happens in the worker2
+  `Infrastructure/Email/Send` cron (~1-min latency). Added the new
+  [`email-send-pipeline.md`](email-send-pipeline.md) feature doc and cross-linked it. (bala)
 - 2026-07-16 — Recorded the **hardcoded AWS SES SMTP credentials** security gotcha in
   `_Email` (`SMTP_USERNAME`/`SMTP_PASSWORD` constants; pre-existing — rotate + move to
   `Config`). Surfaced while fixing a production 500 (EO-1) whose root cause was a single

package/knowledge/2.0/apps/api2/architecture.md

@@ -107,6 +107,15 @@ One ~2,000-line `execute()` then `processRoutePairs()`:
 5. **Transaction logging** — every request logged (to client/core Logs DB, or as a JSONL
    line shipped by CloudWatch when `[api] log_filepath` is set).
 
+> **⚠ Auto-generated `Api.transactionId` collides under concurrency → 1062 → HTTP 500.**
+> Separate from the *client-supplied* `transactionId` uniqueness check (EV-5, above): the inbound
+> request-logger inserts its `Api` log row with a UNIQUE `transactionId` set to a
+> millisecond-precision timestamp (`Y-m-d H:i:s.v`). Concurrent nested writes generated within the
+> same millisecond collide on that UNIQUE key → MySQL **1062** → **HTTP 500**. This breaks ingestion
+> for high-volume senders (seen on the Compass/Veyer ASN feed, `sourceIp 34.232.23.158`) and is
+> platform-wide. Fix direction: make the logged `transactionId` unique-enough (uuid) or
+> retry-on-1062. Until fixed, a burst of concurrent posts can intermittently 500 with no app-level cause.
+
 ## CRUD engine — `processRoutePairs()`
 
 **Metadata-driven** — routes/models/fields/permissions come from Core/Client DB tables, not
@@ -222,5 +231,6 @@ they are the known sharp edges. Do not re-discover these from scratch.
 
 ## Change history
 - 2026-07-28 — Added a consolidated **Known issues / accepted risks** section (8 items), absorbing the previously free-floating deferred raw-exception-disclosure follow-up as item 1, so the tier's sharp edges (unrotated committed secrets, pre-execute phase still outside the main guard, local Logs DB name mismatch, permissive CORS, unpinned `_underscore` build clone, untested `V2.php` monolith, JWT rotation overlap window) are in one place instead of scattered. Recorded that `DB_CACHE` is resolved by name (`Databases.name = 'Cache'`), never by a hardcoded id, which differs per Core instance. (jcardinal)
+- 2026-07-28 — Added gotcha: the request-logger's auto-generated `Api.transactionId` (millisecond timestamp `Y-m-d H:i:s.v`, UNIQUE) collides under concurrent same-millisecond nested writes → MySQL 1062 → HTTP 500; platform-wide, observed on the Compass/Veyer ASN feed (`sourceIp 34.232.23.158`). Distinct from the client-supplied `transactionId`/EV-5 uniqueness contract. Fix direction: uuid the logged id or retry-on-1062. (bala)
 - 2026-07-27 — Sharpened the committed-secret note: the plaintext GitHub PAT lives in the per-env **`.ebextensions/git.*.json`** files (used by the `prebuild/git.sh` clone hook to pull `_underscore`), must be rotated and moved to SSM / EB env properties (location + remediation only, no value). (mhammontree)
 - 2026-07-23 — Documented the now-guarded Core/Logs DB bootstrap in the front controller: the pre-execute block runs before the `execute()` try/catch, the Core Logs schema name is resolved from a `Core.Database` row (`id = CORE_LOGS_DATABASE_ID`) so a name-mismatched local Logs DB reads as missing, and the failure is now wrapped in `try/catch (\Throwable)` returning `INVALID_CONFIGURATION` + Sentry instead of a fatal (guarded no-op rollback, `Database.php:219–226`). Added the deferred raw-getMessage/getTrace client-disclosure follow-up to the Security note. (jcardinal)

package/knowledge/INDEX.md

@@ -17,8 +17,8 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 39 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
-- **worker2** (Worker) — 32 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 40 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **worker2** (Worker) — 33 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 18 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)

package/knowledge/clients/prudential/INDEX.md

@@ -3,9 +3,10 @@
 | Doc | Framework | Summary | Files |
 |-----|-----------|---------|-------|
 | [Prudential: Dell ASN units PRE/POST interceptor (legacy key + flat tracking)](features/dell-asn-units-interceptor.md) | 2.0 | After the tracking-number bridge migration, the ASN unit route was renamed (`advance-shipping-notice-units` → `advance-shipping-notice-item-units`), so the inhe | _underscore/Model/Prudential/AdvanceShippingNotice.php, dbchanges2/Client_Prudential/2026-06-10 - AsnUnitsInterceptor.sql |
-| [Prudential: Dell LCH IOP transmissions (LCHRequestV2)](features/dell-lch-iop-transmissions.md) | 1.0 | Outbound order transmissions from the 1.0 worker tier to Dell's Lifecycle Hub (LCH) ITSM integration. | library/app/api/delllch.php, library/app/apitransaction.php, worker/crons/toga2/prudential/transmissions_to_dell_usa.php, worker/crons/toga2/prudential/transmissions_to_dell_ireland.php, worker/crons/toga2/prudential/transmissions_to_dell_india.php |
+| [Prudential: Dell LCH IOP transmissions (LCHRequestV2)](features/dell-lch-iop-transmissions.md) | 1.0 | Outbound order transmissions from the 1.0 worker tier to Dell's Lifecycle Hub (LCH) ITSM integration. | library/app/api/delllch.php, library/app/apitransaction.php, worker/crons/toga2/prudential/transmissions_to_dell_usa.php, worker/crons/toga2/prudential/transmissions_to_dell_ireland.php, worker/crons/toga2/prudential/transmissions_to_dell_india.php, worker/crons/toga2/prudential/generate_sales_and_purchase_orders_from_service_requests.php |
 | [Prudential: Device information import + unit→contact linking (import_device_information.php)](features/device-information-import-and-contact-linking.md) | 1.0 | Prudential's **device-sync** cron (`worker/crons/toga2/prudential/import_device_information.php`) pulls device/asset records (from ServiceNow / the Dell CMDB fe | worker/crons/toga2/prudential/import_device_information.php, worker/crons/toga2/prudential/backfill_unit_contacts.php, dbchanges2/Client_Prudential/2026-07-07 - Contact Dedup Merge.sql |
 | [Prudential: Service Request Regional Address Validation](features/service-request-address-validation.md) | 2.0 | The `prePost` interceptor on `_Model_Prudential_ServiceRequest` validates `deliverToAddress` fields differently depending on which Prudential regional customer | _underscore/Model/Prudential/ServiceRequest.php, _underscore/Test/Prudential/ServiceRequestTest.php |
+| [Prudential: Service Request rejection alert email](features/service-request-rejection-alert-email.md) | 2.0 | When a Prudential ServiceNow→TOGa service-request submission (`POST /v2/service-requests`) is **rejected by validation**, TOGa now sends a real-time internal al | worker2/Worker/Client/Prudential/reports/ReqRejectionEmail.php, _underscore/Model/Prudential/ServiceRequest.php |
 | [Prudential Order Shipped Email — transmit_ordershipped_updates_prudential.php](features/transmit-ordershipped-email.md) | 1.0 | Cron script that transmits "Order Shipped" updates to ServiceNow (RITM) and sends a shipped notification email to the end user. | worker/crons/toga2/prudential/transmit_ordershipped_updates_prudential.php, worker/crons/toga2/prudential/transmit_closecomplete_updates_prudential.php, worker/crons/toga2/prudential/transmit_rejected_cancelled_updates_prudential.php, worker/crons/toga2/prudential/generate_sales_and_purchase_orders_from_service_requests.php, worker/crons/toga2/prudential_beta/generate_sales_and_purchase_orders_from_service_requests.php, worker/crons/notifications/reports/prudential_exception_report.php |
 | [Prudential Financial](profile.md) | 2.0 | Prudential is a TOGA client whose device-fulfillment flow is driven by **Dell** via the Dell API (`Client_Prudential.Apis.id = 2`). |  |
 | [Prudential: Dell ASN failed POST backfill replay](workflows/dell-asn-backfill-replay.md) | 2.0 | When Dell ASN POSTs fail in bulk (e.g. |  |

package/knowledge/clients/prudential/features/dell-asn-units-interceptor.md

@@ -5,13 +5,14 @@ project: _Underscore
 client: prudential
 type: client-feature
 status: active
-updated: 2026-06-12
-owners: ["jcardinal", "rgirish"]
+updated: 2026-07-28
+owners: ["jcardinal", "rgirish", "bala"]
 files:
   - _underscore/Model/Prudential/AdvanceShippingNotice.php
   - dbchanges2/Client_Prudential/2026-06-10 - AsnUnitsInterceptor.sql
 related:
   - ../../../2.0/apps/_underscore/features/tracking-number-bridges.md
+  - transmit-ordershipped-email.md
 ---
 
 ## Summary
@@ -60,6 +61,24 @@ UPDATE Core.RecordFields SET childPolicy = 'MATCH_CREATE' WHERE id = 2198;
 found. `MATCH` (the default) errors when no row exists — that was the root cause of the Jun 2026
 `EV-12` outage.
 
+## Inbound endpoint & logging (where a Dell ASN lands)
+
+- **Endpoint:** Dell ASNs arrive via inbound `POST /v2/advance-shipping-notices`; **201** =
+  accepted/created.
+- **Success vs. failure log split.** Successful ASN POSTs are logged in `Logs_Prudential.Api`, but
+  **failures often land in the shared base `Logs` schema, not the client log** — so a "no trace in
+  `Logs_Prudential.Api`" ASN may have failed and been recorded in `Logs.Api` instead. Check both.
+- **Filter by `sourceIp` to avoid conflating clients.** Prudential's ASN traffic comes from
+  **`13.86.101.210`** (apiId 2). A *different* client's ASN feed comes from **`34.232.23.158`** —
+  when auditing the shared `Logs` schema, filter on `sourceIp` or you will mix the two clients'
+  ASNs together.
+- **`AdvanceShippingNotices.purchaseOrderId` links an ASN to its PO.** A NULL/unmatched
+  `purchaseOrderId` corresponds to a **400 `EV-12`** on the inbound POST (the ASN could not be
+  matched/created). A successful 201 with a matched PO is what then triggers the shipped-email
+  chain (`transmit_ordershipped_updates_prudential.php` stamps
+  `c_dtTransmittedOrderShippedUpdateToPrudential` + `c_dtEmailSentOrderShipped`) — see
+  `transmit-ordershipped-email.md`.
+
 ## Gotchas
 - Scoped to `apiId = 2` so other (internal) Prudential callers — which already send the new shape —
   are not double-transformed.
@@ -72,6 +91,11 @@ found. `MATCH` (the default) errors when no row exists — that was the root cau
   replay workflow doc.
 
 ## Change history
+- 2026-07-28 — Documented the inbound ASN endpoint (`POST /v2/advance-shipping-notices`, 201 =
+  success), the success-vs-failure log split (successes in `Logs_Prudential.Api`, failures often
+  in shared base `Logs`), the `sourceIp` filter to separate Prudential (`13.86.101.210`) from
+  another client's ASN feed (`34.232.23.158`), and that `AdvanceShippingNotices.purchaseOrderId`
+  links the ASN to its PO (unmatched = 400 `EV-12`) and gates the shipped-email chain. (bala)
 - 2026-06-12 — Fixed interceptor bug: bridge row keys were `trackingNumber`/`returnTrackingNumber`
   instead of `trackingNumberId`/`returnTrackingNumberId`, causing EV-12 on all Dell ASN POSTs. Also
   set `Core.RecordFields childPolicy = MATCH_CREATE` on ids 1437 and 2198. Replayed 28 failed payloads

package/knowledge/clients/prudential/features/dell-lch-iop-transmissions.md

@@ -6,7 +6,7 @@ project: Worker
 client: prudential
 type: client-feature
 status: active
-updated: 2026-07-24
+updated: 2026-07-28
 owners: ["rgirish", "bala"]
 files:
   - library/app/api/delllch.php
@@ -14,6 +14,7 @@ files:
   - worker/crons/toga2/prudential/transmissions_to_dell_usa.php
   - worker/crons/toga2/prudential/transmissions_to_dell_ireland.php
   - worker/crons/toga2/prudential/transmissions_to_dell_india.php
+  - worker/crons/toga2/prudential/generate_sales_and_purchase_orders_from_service_requests.php
 related:
   - ../profile.md
   - dell-asn-units-interceptor.md
@@ -57,9 +58,12 @@ WHERE PurchaseOrders.vendorId = 1        # Dell = Vendors.id 1
 ```
 
 - **Region lives on `ServiceRequests.customerId`**, resolved through the join to
-  `Client_Prudential.Customers`. Region rows: **USA = id 3, uuid
-  `b3f7a2c1-4e89-4d6a-9c3b-8f1e5d2a7b04`**; India and Ireland are separate `Customers` rows /
-  uuids. This is NOT the SalesOrder customer — the SO customer is always
+  `Client_Prudential.Customers`. Region rows:
+  - **USA — id 3, uuid `b3f7a2c1-4e89-4d6a-9c3b-8f1e5d2a7b04`**
+  - **Ireland — id 5, uuid `9a4e7d2b-5f13-48c6-b8e1-3d6a9c2f7e85`**
+  - **India — id 4, uuid `e6d1c8f4-2a75-4b3e-a9d7-1c4f6e8b3a52`**
+
+  This is NOT the SalesOrder customer — the SO customer is always
   **"Agilant - Tech Hub" (id 1, uuid `36da53f8-38d2-404f-becf-f58f33c215d3`)**.
 - A REQ is only ever transmitted, validated, or marked once it is selected here. Unit/bundle/SO/PO
   creation happens elsewhere and does **not** send anything to Dell.
@@ -172,6 +176,17 @@ This is the established pattern for all Prudential script changes.
   stuck this way, all with `customerId` NULL. The 2.0-side guard added in
   `service-request-address-validation.md` stops NEW customer-less REQs; already-stuck ones need a
   `customerId` backfill from ship-to country.
+  - **Root cause is a TOGa-side mapping gap, not missing SNOW data.** The inbound ServiceNow
+    payload carries **no customer field** at all — `ServiceRequests.customerId` is assigned
+    *inside TOGa* by `generate_sales_and_purchase_orders_from_service_requests.php`, so a NULL
+    means the generation cron failed to derive/assign a region, not that SNOW omitted anything.
+  - **The country-split cutover (~2026-07-17) is when this started biting.** Before the split,
+    `transmissions_to_dell_v1.php` had **no customer gate**, so every REQ transmitted regardless.
+    The regional split files added `INNER JOIN Customers` + `WHERE Customers.uuid = '<region>'`,
+    which is what drops NULL-`customerId` REQs. ~38 REQs stranded this way since the cutover.
+  - **Remediation:** backfill `ServiceRequests.customerId = 3` (USA) for the stuck US-ship-to
+    REQs; **durable fix** = have the generation cron derive country/`customerId` from
+    `deliverToAddress` so `customerId` is never NULL.
 - **`dtSubmitted` is stamped without inspecting the response (latent bug, not yet fixed).** Every
   regional cron sets `PurchaseOrders.dtSubmitted = NOW()` immediately after the send without
   checking Dell's HTTP status, and `App_ApiTransaction::execute()` (`library/app/apitransaction.php`)
@@ -190,6 +205,14 @@ This is the established pattern for all Prudential script changes.
   do not reintroduce them.
 
 ## Change history
+- 2026-07-28 — Recorded all three region rows (USA id 3, Ireland id 5, India id 4, with uuids) and
+  the root cause of NULL `ServiceRequests.customerId`: the inbound SNOW payload carries no customer
+  field, so `customerId` is assigned TOGa-side by
+  `generate_sales_and_purchase_orders_from_service_requests.php` — a NULL is a TOGa mapping gap.
+  The ~2026-07-17 regional-split cutover added the `INNER JOIN Customers` gate (the old
+  `transmissions_to_dell_v1.php` had none), stranding ~38 REQs. Durable fix = derive
+  country/`customerId` from `deliverToAddress` in the generation cron; interim = backfill
+  `customerId = 3` for stuck US-ship-to REQs. (bala)
 - 2026-07-24 — Documented the live-cron reality (production is the three regional split files under
   `crons/toga2/prudential/`; monolith `transmissions_to_dell.php` disabled, `_v1` dead), the
   selection query, region model on `ServiceRequests.customerId` (USA id 3), the `App_Api_Delllch`

package/knowledge/clients/prudential/features/service-request-address-validation.md

@@ -6,7 +6,7 @@ project: _Underscore
 client: prudential
 type: client-feature
 status: active
-updated: 2026-07-24
+updated: 2026-07-28
 owners: ["rgirish", "bala"]
 files:
   - _underscore/Model/Prudential/ServiceRequest.php
@@ -15,6 +15,7 @@ related:
   - clients/prudential/profile.md
   - clients/prudential/features/dell-asn-units-interceptor.md
   - clients/prudential/features/dell-lch-iop-transmissions.md
+  - clients/prudential/features/service-request-rejection-alert-email.md
   - ../../../2.0/apps/_underscore/features/model-interceptor-unit-testing.md
 ---
 
@@ -47,8 +48,11 @@ back to USA rules.
    `Client_Prudential.Customers` for the matching `name`. Returns `'India'`, `'Ireland'`, or
    `'USA'` (default for anything unrecognised or when uuid is null).
 3. A `match` expression dispatches to the correct validator.
-4. On any validation failure, an `Exception` is thrown with a semicolon-delimited list of all
-   errors — the API engine catches it and returns a 4xx to Dell.
+4. **Errors are collected, not fail-fast.** `prePost` runs all validators and accumulates every
+   error. On any failure it enqueues the internal rejection-alert worker task (see
+   `service-request-rejection-alert-email.md`) and then throws a single `_Exception_Validation`
+   carrying all errors — the API engine maps it to a 400 (behavior unchanged from the caller's
+   perspective; the change is that all errors are now reported at once and staff are alerted).
 
 ## Data model
 
@@ -109,8 +113,22 @@ with a space (e.g. `F92 FP83`). No numeric restriction.
   when Prudential starts including them.
 - **`line3` is India-only.** USA and Ireland validators do not check `line3` (it is not in
   their Dell spec). If Dell starts sending it for USA/Ireland it is silently ignored.
+- **⚠ OPEN (pre-existing HIGH) — unguarded unit access after the validation block throws a 500.**
+  Code that runs *after* the validation block reads
+  `serviceRequestUnits[0]->unit->serialNumber` / `->assetTag` without guarding for a missing
+  `serviceRequestUnits`. A payload with no units throws an **uncaught non-validation
+  `_Exception` → HTTP 500** (which dumps globals in debug mode) and, because it is not an
+  `_Exception_Validation`, it **skips the rejection-alert email** entirely. Fix: guard the unit
+  access (or validate `serviceRequestUnits` presence inside the collected-error block) so it
+  fails as a 400 with an alert instead of a silent 500. Tracked as a follow-up.
 
 ## Change history
+- 2026-07-28 — `prePost` now **collects all validation errors** (was fail-fast) and, on any
+  failure, enqueues the internal rejection-alert worker task before throwing one
+  `_Exception_Validation` with all errors (400 unchanged). See
+  `service-request-rejection-alert-email.md`. Also recorded the open HIGH: unguarded
+  `serviceRequestUnits[0]->unit` access after the validation block throws a 500 and skips the
+  alert. (bala)
 - 2026-07-24 — Added `validateCustomer()` as the first check in `prePost`: rejects a create when
   `customer.uuid` is missing/null/blank (`trim(... ?? '') === ''`) via `_Exception_Validation`
   (→ HTTP 400), stopping customer-less/unroutable REQs at the boundary. Previously a blank uuid

package/knowledge/clients/prudential/features/service-request-rejection-alert-email.md

@@ -0,0 +1,94 @@
+---
+title: "Prudential: Service Request rejection alert email"
+framework: "2.0"
+repo: worker2
+project: Worker
+client: prudential
+type: client-feature
+status: active
+updated: 2026-07-28
+owners: ["bala"]
+files:
+  - worker2/Worker/Client/Prudential/reports/ReqRejectionEmail.php
+  - _underscore/Model/Prudential/ServiceRequest.php
+related:
+  - service-request-address-validation.md
+  - ../../../2.0/apps/_underscore/features/email-send-pipeline.md
+  - ../profile.md
+---
+
+## Summary
+
+When a Prudential ServiceNow→TOGa service-request submission (`POST /v2/service-requests`)
+is **rejected by validation**, TOGa now sends a real-time internal alert email so staff know
+the instant a REQ fails and can tell Prudential exactly what to resubmit. The
+`_Model_Prudential_ServiceRequest::prePost()` interceptor collects **every** validation error
+(not fail-fast), enqueues a worker task carrying the REQ number + reason + full payload, then
+throws a single `_Exception_Validation` — the 400 returned to ServiceNow is unchanged. The
+worker action `_Worker_Client_Prudential_reports_ReqRejectionEmail::sendRejectionEmail` sends
+the alert.
+
+## Scope
+
+- **Validation rejections only.** The MySQL 1062 duplicate-key race on `ServiceRequests.number`
+  (a duplicate SR-number re-POST) is deliberately **out of scope** — it is an internal defect
+  routed to engineering, not a "resubmit this" message for Prudential.
+
+## Key files / entry points
+
+- `_underscore/Model/Prudential/ServiceRequest.php` — `prePost()` collects all validator errors
+  and, on any failure, enqueues the alert via
+  `_Worker::runTask('Client/Prudential/reports/ReqRejectionEmail/sendRejectionEmail', {reqNumber, reason, payload})`
+  before throwing one `_Exception_Validation` carrying all errors.
+- `worker2/Worker/Client/Prudential/reports/ReqRejectionEmail.php` —
+  `sendRejectionEmail(string $reqNumber, string $reason, object|array $payload): ...` builds and
+  sends a **plain-text** email (REQ# + reason + full JSON payload) to a class-constant recipient
+  list (currently `vburks@togatech.com`, `adiamond@togatech.com`).
+
+## How it works
+
+1. `prePost()` runs every validator, **accumulating** errors instead of throwing on the first.
+2. If the error list is non-empty, it enqueues one `ReqRejectionEmail/sendRejectionEmail` worker
+   task with the REQ number, the joined reason string, and the full inbound payload.
+3. It then throws a single `_Exception_Validation` holding all errors → `Controller/Index.php`
+   maps it to **HTTP 400** to ServiceNow (unchanged behavior).
+4. The worker action assembles the plain-text body and sends it via `_Email`. Because sends run
+   through the 2.0 email-send pipeline, the message lands within ~1 minute (see
+   `email-send-pipeline.md`).
+
+## Send guards (why a queue failure can't turn the 400 into a 500)
+
+- `setIsHtml(false)` — plain-text intent (see caveat below).
+- `setClientIdentifier('Prudential')` and a fixed From of `noreply@togatech.com`.
+- `json_encode(..., JSON_INVALID_UTF8_SUBSTITUTE)` with a false-fallback so a bad byte can't
+  fatal the encode; control characters stripped from `reqNumber`.
+- The **whole dispatch is wrapped in a `\Throwable` catch** that `error_log`s an
+  identifier-only message (never PII) — so if the queue/email dispatch fails, the original
+  validation 400 is still what ServiceNow receives; the alert failure never escalates to a 500.
+
+## Accepted residual risk (data-owner decision)
+
+The **full inbound payload (including PII) is emailed** and persisted to `Logs_Prudential.Email`.
+This was an explicit data-owner decision to maximize the staff alert's usefulness, accepted as a
+known residual risk.
+
+## Gotchas / known issues
+
+- **A non-validation exception skips the alert.** The alert only fires for
+  `_Exception_Validation`. A payload that throws a *different* uncaught `_Exception` before the
+  validation block completes (e.g. the unguarded `serviceRequestUnits[0]->unit` access — see
+  `service-request-address-validation.md`) produces a **500 and no alert**. Fix that guard to
+  keep the alert reliable.
+- **"Plain text" is transmitted HTML anyway.** `setIsHtml(false)` is honored by `_Email` when it
+  queues the row, but the worker2 `Infrastructure/Email/Send` action calls `IsHTML(true)`
+  unconditionally, so the message is sent HTML-mode regardless (see `email-send-pipeline.md`).
+
+## Change history
+- 2026-07-28 — Created: real-time internal alert on `POST /v2/service-requests` validation
+  rejection. `prePost()` now collects all validation errors and enqueues the
+  `ReqRejectionEmail/sendRejectionEmail` worker task (REQ# + reason + full payload) before
+  throwing one `_Exception_Validation` (400 unchanged). Recipients are a class constant
+  (`vburks@`, `adiamond@`). Whole dispatch wrapped in `\Throwable` catch so a queue failure can't
+  turn the 400 into a 500. Scope: validation rejections only (the 1062 SR-number duplicate race
+  is routed to engineering). Accepted residual risk: full payload PII is emailed + persisted to
+  `Logs_Prudential.Email`. (bala)

package/knowledge/clients/prudential/profile.md

@@ -7,17 +7,19 @@ apps:
   - dbchanges2
   - websocket
   - worker
+  - worker2
   - library
 project: _Underscore
 client: prudential
 type: profile
 status: active
-updated: 2026-07-07
+updated: 2026-07-28
 owners: ["jcardinal", "rgirish", "bala"]
 files: []
 related:
   - features/dell-asn-units-interceptor.md
   - features/service-request-address-validation.md
+  - features/service-request-rejection-alert-email.md
   - features/dell-lch-iop-transmissions.md
   - features/device-information-import-and-contact-linking.md
 ---
@@ -41,6 +43,23 @@ order-status transmissions.
   POST to Dell's LCH `LCHRequestV2` endpoint via the 1.0 `App_Api_Delllch` client. See the
   Dell LCH IOP transmissions feature doc.
 
+## Reference data (enums)
+
+`Client_Prudential.PurchaseOrders.purchaseOrderStageId` — PO lifecycle stage:
+
+| id | Stage         | id | Stage         |
+|----|---------------|----|---------------|
+| 3  | Open          | 4  | Configuration |
+| 9  | Accepted      | 5  | Ready To Ship |
+| 7  | In Progress   | 2  | Received      |
+| 6  | POD           | 1  | Rejected      |
+| 8  | Cancelled     |    |               |
+
+- PO lifecycle is tracked by `purchaseOrderStageId`. The `dtAcknowledged` column is **unused
+  (always NULL)** — do not key logic on it.
+
+`ServiceRequests.serviceRequestTypeId`: **1 = New Hire, 2 = Breakfix, 3 = Refresh, 5 = Reclaim**.
+
 ## Gotchas
 - Dell will not change their payload shape — see the Dell ASN units interceptor feature doc for the
   PRE/POST translation that keeps their feed working after the tracking-number bridge migration.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.461",
+  "version": "1.0.462",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",