toga-ai 1.0.1179 → 1.0.1180

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-10-15
9
+ updated: 2026-10-28
10
10
  owners: ["bala", "jcardinal"]
11
11
  files:
12
12
  - worker/crons/toga2/compass/update_salesorder_status_from_odp.php
@@ -21,6 +21,7 @@ files:
21
21
  - worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php
22
22
  - dbchanges2/Client_Compass/2026-09-03 - compass_intransit_email_backlog_backfill.sql
23
23
  related:
24
+ - ../../../clients/compass-usa/features/delivered-emails.md
24
25
  - ../../../clients/compass-usa/features/tracking-number-origin-and-asn-link.md
25
26
  - ../../../clients/compass-usa/features/contact-email-resolution.md
26
27
  - ../../../2.0/apps/worker2/features/tracking-status-refresh.md
@@ -42,6 +43,11 @@ cron builds three zones: **Items in this shipment** (just that package), **Alrea
42
43
  and earlier packages). The dynamic HTML is injected into a stored `EmailTemplates` body via the
43
44
  `sendEmail` scripted API. Canada renders EN or FR per the user's language setting.
44
45
 
46
+ **The delivered email is no longer 1.0 (TRUE-79683).** It is the worker2 job
47
+ [Compass delivered email](../../../../clients/compass-usa/features/delivered-emails.md); both
48
+ `send_delivered_email.php` crons are switched off in `worker1.5/schedules/cron.worker.json`. This doc
49
+ now owns the **in-transit** crons only; delivered-cron notes below are history.
50
+
45
51
  ## How it works
46
52
 
47
53
  ### Key files / entry points
@@ -178,12 +184,9 @@ IFNULL(Users.email, ContactEmailAddresses.emailAddress) AS contactEmail
178
184
  The same `IFNULL` resolves the display name, and the `WHERE` requires the resolved address to be
179
185
  non-empty.
180
186
 
181
- **Delivered crons (USA + Canada `send_delivered_email.php`, 2026-10-15, TRUE-82847)** use the same
182
- precedence, but pick **one** `Users` row: `LEFT JOIN Users ON Users.id = (SELECT MAX(id) FROM Users
183
- WHERE contactId = … AND isActive = 1 AND email <> '')` (newest active user with email), fallback
184
- `Contacts.primaryContactEmailAddressId` → `ContactEmailAddresses`. Rows with no usable address are
185
- excluded in SQL (no UPS call, no email, status stays non-`DELIVERED`, so it sends later if an
186
- address appears). `Notifications` rows are written only when `userId > 0`. Never go back to a plain
187
+ **Delivered emails pick ONE `Users` row** (TRUE-82847, kept in the worker2 job): newest active user
188
+ with an email (`Users.id = (SELECT MAX(id) … isActive = 1 AND email <> '')`), fallback
189
+ `Contacts.primaryContactEmailAddressId` → `ContactEmailAddresses`. Never go back to a plain
187
190
  `INNER JOIN Users ON contactId` — it yields one email per `Users` row (see gotcha).
188
191
 
189
192
  **The `ContactEmailAddresses.contactId = Contacts.id` predicate is not optional.** About 14% of
@@ -352,18 +355,12 @@ the tracking number as emailed (it retries next run).
352
355
  future change to the pendingFulfillment or canceled branches of
353
356
  `_Model_Compass_SalesOrder::_status()` must be mirrored into all three crons by hand.** See
354
357
  [Compass Fulfilled vs Partially Fulfilled](../../../clients/compass-usa/features/order-fulfillment-status-per-line.md).
355
- - **⚠ The USA delivered cron has NO date window.** `send_delivered_email.php` picks every ASN-unit-linked tracking row with `status <> 'DELIVERED'`, however old. Any bulk ASN load (replay, backfill) emails customers about old orders unless both crons are off and the loaded tracking ids are stamped first. Procedure: [ODP ASN replay backfill](../../../clients/compass-usa/workflows/odp-asn-replay-backfill.md).
356
- - **⚠ Conflict with the new 2.0 centralized tracking-status refresh.** The delivered-email cron
357
- (`worker/crons/toga2/compass/send_delivered_email.php`) polls UPS itself, sends the delivered
358
- template (uuid `fe961c7c-277b-4a89-be1d-3d318d8c7718`) via api2 `GET /email-templates/sendEmail`,
359
- and writes `TrackingNumbers.status` with a **direct DB UPDATE** guarded by
360
- `WHERE status <> 'DELIVERED'`. The new platform-wide
361
- [tracking-status refresh (worker2)](../../../2.0/apps/worker2/features/tracking-status-refresh.md)
362
- writes `status` via api2 PUT for **all** clients. If the refresh marks a Compass row `DELIVERED`
363
- **first**, this cron's guard **skips** it and the delivered email **never sends** — a real conflict
364
- to resolve before the refresh is enabled for Compass. There is **no `postPut` interceptor** on
365
- tracking-numbers today; the plan (not built) is to move this email logic into a Compass `postPut`
366
- interceptor and retire this 1.0 cron.
358
+ - **Why the 1.0 delivered crons were retired (TRUE-79683).** They polled UPS themselves and guarded
359
+ on `status <> 'DELIVERED'`, so every box the worker2
360
+ [tracking-status refresh](../../../../2.0/apps/worker2/features/tracking-status-refresh.md) marked
361
+ DELIVERED first was skipped and never emailed (12 Oct: 50 of 80 orders). They also had no date
362
+ window and ignored the send result. Replaced by the
363
+ [worker2 delivered email](../../../../clients/compass-usa/features/delivered-emails.md).
367
364
 
368
365
  - **A contact can have 2 active `Users` rows** (old one with NULL email + new one with a new HR
369
366
  username, e.g. `thompj53` → `thompj55`). Prod 2026-10-15: 2 such contacts (326758, 283892). Any
@@ -373,16 +370,10 @@ the tracking number as emailed (it retries next run).
373
370
  behavior](../../../clients/compass-usa/features/people-file-user-lifecycle.md); the new-username
374
371
  case is not proven in importer code.
375
372
  - **One tracking number on several sales orders sends one delivered email per SO.** Known, left
376
- as is (decided 2026-10-15, TRUE-82847).
377
- - Delivered template = prod `emailTemplateId` 26 in `Logs_Compass.Email`; baseline 2026-09-15..10-15:
378
- 959 SENT, 1 FAILED.
379
- - **⚠ The delivered cron never checks the send result.** It calls `App_Api_Toga2::send(..., false)`
380
- (`$throwExceptionOnApiError = false`) and ignores the response, so it marks
381
- `TrackingNumbers.status = 'DELIVERED'` and inserts notifications even when api2 rejects the email.
382
- Its "only mark DELIVERED after confirmed send" comment is wrong. "Sent" only means queued to
383
- `Logs_Compass.Email`; worker2 `Infrastructure/Email/Send` does the real send. Not changed.
373
+ as is (decided 2026-10-15, TRUE-82847; same in the worker2 job).
384
374
 
385
375
  ## Change history
376
+ - 2026-10-28 — Delivered email moved to worker2 (postPut queue + 10-minute cron), 1.0 delivered crons off in worker1.5; doc now covers in-transit only (TRUE-79683). (bala)
386
377
  - 2026-10-15 — Delivered crons (USA + Canada) pick one active `Users` row with email + Contacts fallback; documented the 2-active-users duplicate. (jcardinal)
387
378
  - 2026-10-15 — Recorded that the delivered cron ignores the send result (TRUE-82846). (bala)
388
379
  - 2026-10-13 — Recorded that the USA delivered cron has no date window, so bulk ASN loads must suppress first. (bala)
@@ -6,7 +6,7 @@ project: _Underscore
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-10-16
9
+ updated: 2026-10-28
10
10
  owners: ["bala", "tcox", "dfranks", "apeterson", "jcardinal", "rgirish"]
11
11
  files:
12
12
  - _underscore/Email.php
@@ -68,7 +68,9 @@ SMTP** using PHPMailer, then flips each row to `SENT` or `FAILED`. So a 2.0 emai
68
68
  `emailTemplateId` (after `dtCreated`), `reviewOfEmailId` (self FK, `ON DELETE SET NULL` — links a
69
69
  review copy to the held original), `dtReviewed` (after `dtSent`), `reviewedBy` (after
70
70
  `failureReason`). Migration: `dbchanges2/Logs_Client/2026-09-23a - EmailReview.sql` — must run
71
- **before** the `_underscore` deploy or every `send()` fails. Who writes `HELD`: see
71
+ **before** the `_underscore` deploy or every `send()` fails with 1054 `Unknown column
72
+ 'emailTemplateId'`. On that error in a non-prod env, apply this existing file to the `Logs_<Client>`
73
+ DB; do not write new SQL (beta `Logs_Compass` / `Logs_CompassCanada` lacked it until 2026-10-28). Who writes `HELD`: see
72
74
  [email-template-sending.md](email-template-sending.md) *Review hold*; who releases it: Tools
73
75
  [/emails](../../../../1.0/apps/tools/features/email-queue-review.md).
74
76
 
@@ -141,33 +143,25 @@ derive one from the other).
141
143
  - **The Send worker forces HTML mode.** `sendEmail()` calls `$mailer->IsHTML(true)`
142
144
  **unconditionally**, ignoring the sender's `setIsHtml(false)`. A caller that queued a
143
145
  "plain-text" message is still transmitted HTML-mode. Author bodies accordingly.
144
- - **⚠ The whole chain sends as iso-8859-1 — escape non-ASCII, or it mojibakes.** **Neither**
145
- `_Email::send()` (queue side) **nor** `_Worker_Infrastructure_Email_Send::Run()` (transmit side)
146
- sets `$mailer->CharSet`, so PHPMailer's default **iso-8859-1** applies end to end. Any raw UTF-8
147
- in the rendered body (accented French text, em dash, curly quotes) arrives garbled. Two
148
- consequences for anything building an email body in PHP:
149
- - **Escape dynamic text with `_String::encodeToAsciiHtmlEntities($value)`** (`_underscore/String.php`).
150
- It runs `htmlentities($value, ENT_QUOTES | ENT_SUBSTITUTE, 'UTF-8', false)` **and then numbers
151
- every remaining character above U+007F** — necessary because `htmlentities()` converts only
152
- characters that have a *named* entity, so e.g. the French narrow no-break space **U+202F**
153
- (used before a colon) survives as raw UTF-8 and mojibakes. See
154
- [`string-html-entity-helpers.md`](string-html-entity-helpers.md). Both non-default arguments to
155
- the underlying call are required:
156
- - `double_encode = false` — some stored data (e.g. `ItemTranslations.title` in one
157
- environment) already holds **literal HTML entities as text**; re-encoding turns `&eacute;`
158
- into `&amp;eacute;`, which the recipient reads as a literal `&eacute;`.
159
- - `ENT_SUBSTITUTE` — passing explicit flags **drops** PHP 8.1's default `ENT_SUBSTITUTE`, and
160
- `htmlentities()` returns an **empty string** on invalid UTF-8, so a single bad byte silently
161
- blanks the entire field.
162
- - **Stored `EmailTemplates` bodies should be pure ASCII.** A raw non-breaking space (U+00A0) or a
163
- stray `Â` (U+00C2) pasted into a template body renders as mojibake in the delivered mail — use
164
- `&nbsp;` / `&eacute;` entities instead. Real defect: 8 Compass Canada French templates carried a
165
- `Â`+NBSP sequence before the colon in `"Commande :"`.
166
-
167
- The root-cause fix is still to set `$mailer->CharSet = 'UTF-8'` — but that changes **every**
168
- client's mail, so it must be its own tested change, not a drive-by. See
169
- [`email-template-sending.md`](email-template-sending.md) and
170
- [Compass Canada French order emails](../../../clients/compass-canada/features/french-order-email-localization.md).
146
+ - **Mail is sent as UTF-8, and subjects are entity-decoded (fixed 2026-10-28, TRUE-79683).**
147
+ `_Worker_Infrastructure_Email_Send::sendEmail()` sets `$mailer->CharSet =
148
+ PHPMailer::CHARSET_UTF8` and sends the subject through `html_entity_decode($subject, ENT_QUOTES |
149
+ ENT_HTML401, 'UTF-8')`. Before, PHPMailer's default **iso-8859-1** label garbled every UTF-8
150
+ character in the stored `utf8mb4` text (em dash in True ClickUp alert subjects, UTF-8 NBSP in
151
+ Compass bodies), and subjects showed raw entities (all 7 Compass Canada French subjects read e.g.
152
+ `Livr&eacute;e`, since at least Aug 2026). The body is HTML, so entities always rendered there.
153
+ Rules now:
154
+ - **Template subjects and bodies may use entities or real UTF-8.** Both arrive correct. Never write
155
+ anything that relies on iso-8859-1.
156
+ - Only `;`-terminated entities decode, so a bare `&` in a subject (`CC & MS Scrum`, `A&M`) is
157
+ unchanged. Prod check, 90 days of all `Logs_*`: only 5 subjects held entities, all Compass Canada
158
+ French.
159
+ - `_Email::send()` (queue side) still sets no `CharSet`; harmless, it only writes the row.
160
+ - **Still escape dynamic text for HTML** with `_String::encodeToAsciiHtmlEntities($value)` — see
161
+ [`string-html-entity-helpers.md`](string-html-entity-helpers.md) for its required
162
+ `double_encode = false` + `ENT_SUBSTITUTE` flags.
163
+ - A stray `Â` (U+00C2) before an NBSP in a stored template is bad data (double-encoded paste), not
164
+ a charset issue. Fix the row. Real defect: 8 Compass Canada French templates, `"Commande :"`.
171
165
  - **⚠ SECURITY — hardcoded AWS SES SMTP credentials in `_underscore/Email.php`.** The
172
166
  `SMTP_USERNAME` / `SMTP_PASSWORD` class constants hold **real** SES SMTP credentials committed
173
167
  in source, violating the no-secrets-in-code rule. Documenting the **location only** — do not
@@ -214,6 +208,7 @@ derive one from the other).
214
208
  for a hotfix.
215
209
 
216
210
  ## Change history
211
+ - 2026-10-28 — Send worker now sends UTF-8 and entity-decodes subjects; replaced the iso-8859-1 gotcha; 1054 `emailTemplateId` = apply the existing EmailReview file (TRUE-79683). (bala)
217
212
  - 2026-10-16 — `SENT` ≠ delivered; linked the SES sender-identity/DMARC doc after Compass TechHub mail was DMARC-rejected (TRUE-82883) (bala)
218
213
  - 2026-10-14 — Gotcha: client-sandbox missing `Logs_<Client>.Email` tables and a Send cron that never drains `PENDING` rows (rgirish)
219
214
  - 2026-09-23 — Review hold: `HELD`/`CANCELLED` statuses, `emailTemplateId`/`reviewOfEmailId`/
@@ -6,7 +6,7 @@ project: _Underscore
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-10-20
9
+ updated: 2026-10-28
10
10
  owners: ["jcardinal", "bala", "mhammontree", "apeterson"]
11
11
  files:
12
12
  - _underscore/Model/Client/EmailTemplate.php
@@ -201,16 +201,10 @@ can keep using `sendEmail($api, ...)`.
201
201
  Any fire-and-forget caller now propagates that exception, which is intended (so an email is
202
202
  never silently marked as sent). Blast radius is every client, including the Compass/Quad
203
203
  `SalesOrder`/`ApprovalDecision` order-placed emails.
204
- - **`_Email::send()` does not set PHPMailer `CharSet=UTF-8`.** Non-ASCII subjects/body (em
205
- dash, curly quotes, accented names) **mojibake** (e.g. `—` → `—`). Scoped workaround:
206
- use ASCII-only content. Recommended **root-cause fix**: set `$mailer->CharSet = 'UTF-8'` in
207
- `_Email::send()` — but that affects **every** client, so it should be a separate, tested
208
- change rather than a drive-by edit.
209
- - **The transmit side doesn't set it either**, so the effective wire charset is **iso-8859-1**
210
- end to end. For a body that genuinely needs accents (e.g. French), the working escape is
211
- `htmlentities($v, ENT_QUOTES | ENT_SUBSTITUTE, 'UTF-8', false)` — both non-default arguments
212
- matter. See the encoding-chain gotcha on
213
- [`email-send-pipeline.md`](email-send-pipeline.md) for why.
204
+ - **Charset: fixed 2026-10-28 (TRUE-79683).** The worker2 Send action now transmits UTF-8 and
205
+ entity-decodes subjects, so subjects and bodies may hold real UTF-8 or entities; the old
206
+ ASCII-only workaround is no longer needed. Details on
207
+ [`email-send-pipeline.md`](email-send-pipeline.md).
214
208
  - **⚠ SECURITY — live AWS SES SMTP credentials are hardcoded in `_underscore/Email.php`.**
215
209
  The `SMTP_USERNAME` / `SMTP_PASSWORD` class constants (near the top of `_Email`) hold
216
210
  **real** AWS SES SMTP credentials committed in source, violating the team no-secrets-in-code
@@ -243,6 +237,7 @@ worker method) in-process instead.
243
237
 
244
238
  ## Change history
245
239
 
240
+ - 2026-10-28 — CharSet gotcha closed: Send worker sends UTF-8 and decodes subject entities (TRUE-79683). (bala)
246
241
  - 2026-10-20 — `EmailTemplates.kind` SYSTEM / NOTIFICATION; this doc = SYSTEM; linked the NOTIFICATION blocks doc (TRUE-82934). (jcardinal)
247
242
  - 2026-10-19 — Recorded the public, unverified togatech.com use of sendEmail and the lockdown follow-up. (akhokhani)
248
243
  - 2026-10-15 — Recorded the empty-string `to` + cc gap (queued, fails in worker2). (jcardinal)
@@ -6,7 +6,7 @@ project: _Underscore
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-16
9
+ updated: 2026-10-28
10
10
  owners: ["bala"]
11
11
  files:
12
12
  - _underscore/String.php
@@ -37,7 +37,7 @@ Entry points (`_underscore/String.php`):
37
37
  - `ENT_SUBSTITUTE` — passing explicit flags **drops** PHP 8.1's default, and `htmlentities()` returns an **empty string** on invalid UTF-8, silently blanking the whole field.
38
38
  2. `preg_replace_callback('/[\x{0080}-\x{10FFFF}]/u', …)` converting every remaining character above U+007F to a **numeric** entity (`&#NNNN;`) via `mb_ord()`.
39
39
 
40
- **Why pass 2 exists:** `htmlentities()` only converts characters that have a *named* entity. French text routinely contains the **narrow no-break space U+202F** before a colon (`Commande :`), which has no name and survives pass 1 as **raw UTF-8**. Since neither `_Email::send()` nor worker2's Send action sets PHPMailer's `CharSet`, mail goes out as the PHPMailer default **iso-8859-1** and that raw UTF-8 arrives corrupted. Pass 2 closes the gap. Empty/`null` input returns `''`.
40
+ **Why pass 2 exists:** `htmlentities()` only converts characters that have a *named* entity. French text routinely contains the **narrow no-break space U+202F** before a colon (`Commande :`), which has no name and survives pass 1 as **raw UTF-8**. That corrupted mail while the Send worker used iso-8859-1. Since 2026-10-28 (TRUE-79683) mail goes out as UTF-8, so raw UTF-8 is safe too; ASCII output stays correct under any charset, so keep using the helper. Empty/`null` input returns `''`.
41
41
 
42
42
  Use it for **every** dynamic string entering an email body — free-text notes, item titles, part numbers.
43
43
 
@@ -51,6 +51,9 @@ Use it for **every** dynamic string entering an email body — free-text notes,
51
51
  - Not fixed as of 2026-08-05 (found while working elsewhere in the file) — worth its own ticket. Fix is a one-word change to `haystack:`.
52
52
  - Secondary defect in the same method: `$indStart` has `strlen()` **added** before the `=== false` check, so the "not found" guard can never fire — `false + n` is `n`.
53
53
 
54
+ ## Change history
55
+ - 2026-10-28 — Pass 2 is no longer needed for correctness: the Send worker now sends UTF-8 (TRUE-79683). Helper still recommended. (bala)
56
+
54
57
  ## Related
55
58
 
56
59
  - [email-send-pipeline](email-send-pipeline.md)
@@ -6,9 +6,11 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-18
9
+ updated: 2026-10-28
10
10
  owners: ["bala"]
11
11
  files:
12
+ - worker2/LambdaFunctions/CronScheduler/lambda_function.py
13
+ - worker2/Controller/Index.php
12
14
  - worker2/.ebextensions/git.php
13
15
  - worker2/.ebextensions/git.json
14
16
  - worker2/.ebextensions/php_include_underscore.config
@@ -31,13 +33,19 @@ behaves the same wastes a deploy cycle and hides stale code.
31
33
 
32
34
  | EB environment | Tier | `ENVIRONMENT` | `_underscore` branch | Deployed by |
33
35
  |---|---|---|---|---|
34
- | `worker-invoke-sandbox-dev` | **Worker** (SQS) | `sandbox-dev` | `_sandbox-dev` | **by hand only** |
35
- | `worker-sandbox-dev` | **WebServer** | `beta` | `_beta` | CodePipeline |
36
+ | `worker-invoke-sandbox-dev` | **Worker** (SQS) | `sandbox-dev` (`Core.Environments` id 23) | `_sandbox-dev` | **by hand only** |
37
+ | `worker-sandbox-dev` | **WebServer** | `beta` (`Core.Environments` id 4) | `_beta` | CodePipeline |
36
38
 
37
39
  - `worker.beta.togahub.com` resolves to the same IPs as
38
40
  `worker-sandbox-dev.us-east-1.elasticbeanstalk.com`, so an HTTP invoke tests the **WebServer** box.
39
41
  - **The cron runs on `worker-invoke-sandbox-dev`.** CodePipeline never touches it, so it keeps
40
- running old code until someone deploys it manually.
42
+ running old code until someone deploys it manually. Cron path: Lambda
43
+ `WorkerCronScheduler-SandboxDev` (`worker2/LambdaFunctions/CronScheduler/lambda_function.py`)
44
+ inserts a `Core.WorkerJobs` row (`jobType = 'CRON'`) → SQS `WorkerSandboxDevQueue` →
45
+ `worker-invoke-sandbox-dev`. **Are crons running?** Check `Core.WorkerJobs.dtStarted` for recent
46
+ `jobType = 'CRON'` rows.
47
+ - Direct HTTP calls (Postman) hit `worker.beta.togahub.com` = `worker-sandbox-dev`
48
+ (`ENVIRONMENT=beta`). Non-prod sends go to `devteam@goagilant.com` (debug redirect).
41
49
  - `worker2/Config/beta.ini` and `Config/sandbox-dev.ini` both point at
42
50
  `sandbox-dev.cluster-cb860gg2qw2n.us-east-1.rds.amazonaws.com`, so **finding your row in the
43
51
  database proves nothing about which box wrote it.**
@@ -85,8 +93,17 @@ Before you go looking for a missing SQS message on a sandbox, confirm the produc
85
93
  - **Never infer the environment from the EB environment name.** `worker-sandbox-dev` is
86
94
  `ENVIRONMENT=beta`; the name and the value disagree on purpose-built tiers more often than not.
87
95
  - **The clone deletes `.git`**, so the instance carries no commit id. Verify by file presence.
96
+ - **Each `Core.Environments` id needs its own `Core.DatabaseHosts` rows, including Logs
97
+ (`databaseId` 11).** Env 23 (`sandbox-dev`) had none for Logs, so every job on
98
+ `worker-invoke-sandbox-dev` failed `No Core Logs Database host found`
99
+ (`worker2/Controller/Index.php` ~260) and no beta cron ran. Fixed 2026-10-28: beta Core
100
+ `DatabaseHosts` id 4736 (env 23, Logs, host `sandbox-dev.cluster-cb860gg2qw2n.us-east-1.rds.amazonaws.com`).
101
+ Check this first when every sandbox-dev cron fails at once.
102
+ - **Send queued beta emails by hand:** POST `{"action":"Infrastructure/Email/Send/Run"}` to
103
+ `worker.beta.togahub.com`.
88
104
 
89
105
  ## Change history
106
+ - 2026-10-28 — Cron path (Lambda → `Core.WorkerJobs` CRON → SQS `WorkerSandboxDevQueue`), Environments ids 23 / 4, missing Logs `DatabaseHosts` row for env 23 fixed (id 4736), manual Send/Run (TRUE-79683). (bala)
90
107
 
91
108
  - 2026-09-18 — Initial capture, from the NYCHH transfer-order-email work. Recorded that
92
109
  `worker-invoke-sandbox-dev` (Worker tier, `ENVIRONMENT=sandbox-dev`, branch `_sandbox-dev`) runs
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-28
9
+ updated: 2026-10-28
10
10
  owners: ["bala", "jcardinal"]
11
11
  files:
12
12
  - worker2/Worker/Sync/TrackingNumbers.php
@@ -140,8 +140,9 @@ bookkeeping columns.
140
140
 
141
141
  ### Decisions
142
142
  - **Write path = api2 PUT, not direct-DB (deliberate).** Direct-DB was **rejected** so that a future
143
- **`postPut` interceptor** on tracking-numbers can fire the in-transit / delivered emails on a status
144
- change. There is no such interceptor today (see the interceptor doc). This is a departure from other
143
+ **`postPut` interceptor** on tracking-numbers can fire emails on a status change. Used since
144
+ TRUE-79683: the Compass postPut queues the
145
+ [delivered email](../../../../clients/compass-usa/features/delivered-emails.md). This is a departure from other
145
146
  worker2 crons (e.g. NYCHH asset-tag backfill) that write direct-DB to dodge the PHP-8.5 crash — here
146
147
  the interceptor requirement wins, which is exactly why the crash below is a hard gate.
147
148
 
@@ -211,15 +212,14 @@ catch up first.** See
211
212
  Open question: nothing today re-tries or reports parked rows. A periodic unpark (or a monitor on
212
213
  parked-row count) is unbuilt work.
213
214
 
214
- ### ⚠ Conflict with the 1.0 Compass delivered-email cron
215
- The in-transit / delivered emails **today** come from a separate **1.0** Compass worker cron
216
- (`worker/crons/toga2/compass/send_delivered_email.php`) that polls UPS itself, sends via api2 `GET
217
- /email-templates/sendEmail`, and writes `TrackingNumbers.status` via a **direct DB UPDATE** guarded by
218
- `WHERE status <> 'DELIVERED'`. If this refresh marks a Compass row `DELIVERED` **first**, that cron's
219
- guard skips it and the delivered email **never sends**. This is a real conflict to resolve before
220
- enabling the refresh for Compass — see the
221
- [Compass in-transit/delivered emails doc](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md).
222
- Plan (not built): move the email logic into a Compass `postPut` interceptor and retire the 1.0 cron.
215
+ ### Compass delivered email is driven by this refresh (resolved, TRUE-79683)
216
+ The old 1.0 delivered crons skipped any box this refresh had already marked `DELIVERED`, so the email
217
+ never sent. Resolved: they are off, and `_Model_Compass_TrackingNumber::postPut` queues worker2
218
+ `Client/Compass/DeliveredEmails/SendForTrackingNumber` on each PUT that sets `DELIVERED` (10-minute
219
+ cron backup). See [Compass delivered email](../../../../clients/compass-usa/features/delivered-emails.md).
220
+
221
+ ## Change history
222
+ - 2026-10-28 — Compass delivered-email conflict resolved: postPut on this refresh's PUT queues the worker2 delivered email (TRUE-79683). (bala)
223
223
 
224
224
  ## Related
225
225
 
@@ -228,6 +228,6 @@ Plan (not built): move the email logic into a Compass `postPut` interceptor and
228
228
  - [Running worker2 locally](../workflows/running-worker2-locally.md)
229
229
  - [Carrier shipping labels](../../_underscore/features/carrier-shipping-labels.md) — carrier-library batch support.
230
230
  - [Tracking-number bridges](../../_underscore/features/tracking-number-bridges.md)
231
- - [API Payload Interceptors](../../api2/features/api-payload-interceptors.md) — the future `postPut` interceptor.
231
+ - [API Payload Interceptors](../../api2/features/api-payload-interceptors.md) — how the Compass `postPut` interceptor is registered.
232
232
  - [Compass in-transit/delivered emails (1.0)](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md)
233
233
  - [NYCHH asset-tag backfill](nychh-asset-tag-backfill.md) — same PHP-8.5 crash symptom, direct-DB workaround.
@@ -23,7 +23,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
23
23
  ## 2.0 framework
24
24
 
25
25
  - **_underscore** (_Underscore) _(framework core)_ — 117 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
26
- - **worker2** (Worker) — 100 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
26
+ - **worker2** (Worker) — 101 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
27
27
  - **api2** (API) — 29 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
28
28
  - **dbchanges2** (Database Changes) _(framework core)_ — 24 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
29
29
  - **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
@@ -3,7 +3,7 @@
3
3
  | Doc | Framework | Summary |
4
4
  |-----|-----------|---------|
5
5
  | [French (fr-CA) Item Feature Translations (Compass Canada)](features/french-item-feature-translations.md) | 2.0 | Renders item feature names/group headers/values in fr-CA on the Compass Canada storefront; open for Etilize feature translation, feature/group dedup, or the sid |
6
- | [French (fr-CA) Order Email Localization (Compass Canada)](features/french-order-email-localization.md) | 2.0 | How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation) under an iso-8859-1 send; open for any locali |
6
+ | [French (fr-CA) Order Email Localization (Compass Canada)](features/french-order-email-localization.md) | 2.0 | How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation) and their HTML escaping; open for any localiz |
7
7
  | [Grand & Toy ASN Import (Compass Canada)](features/grand-and-toy-asn-import.md) | 2.0 | Imports Grand & Toy ASNs for Compass Canada (CSV cron + cXML gateway) and auto-builds the ItemFulfillment chain + bilingual in-transit email; open for G&T ASN f |
8
8
  | [Compass Canada — Item-Fulfillment TableViews (dead-Core-id 500, rebuilt onto the USA shape)](features/item-fulfillment-tableviews.md) | 2.0 | Why Compass Canada's item-fulfillment TableViews 500'd on dead Core ids and how they were rebuilt onto the USA shape; open when a Canada TableView 500s on /meta |
9
9
  | [Compass Canada: incoming MITS purchase-order monitor (Monitor/Compass/CompassCanadaIncomingPurchaseOrders)](features/oneuptime-incoming-purchase-orders-monitor.md) | 2.0 | worker2 OneUptime push monitor for Compass Canada sales orders sent to MITS that have not got their full purchase order back, plus how MITS links a Canada PO to |
@@ -5,7 +5,7 @@ project: _Underscore
5
5
  client: compass-canada
6
6
  type: client-feature
7
7
  status: active
8
- updated: 2026-09-25
8
+ updated: 2026-10-28
9
9
  owners: ["bala"]
10
10
  files:
11
11
  - _underscore/Model/Compass/SalesOrder.php
@@ -13,7 +13,7 @@ files:
13
13
  - _underscore/Component/Api/Togaiq/Togaiq.php
14
14
  - _underscore/String.php
15
15
  - library/app/client/compasscanada.php
16
- - worker/crons/toga2/compasscanada/send_delivered_email.php
16
+ - worker2/Worker/Client/Compass/DeliveredEmails.php
17
17
  - worker/crons/toga2/compasscanada/compass_email_reminders.php
18
18
  - worker/crons/toga2/compasscanada/compass_cancel_pending_approval_orders.php
19
19
  - worker/crons/toga2/compasscanada/workflow/test_email_previews_prod.php
@@ -25,15 +25,16 @@ related:
25
25
  - ../../../2.0/apps/_underscore/features/togaiq-gateway-client.md
26
26
  - ../../../2.0/apps/_underscore/features/string-html-entity-helpers.md
27
27
  - ../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md
28
+ - ../../compass-usa/features/delivered-emails.md
28
29
  ---
29
- How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation) under an iso-8859-1 send; open for any localized-email work or encoding/mojibake bugs.
30
+ How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation) and their HTML escaping; open for any localized-email work or encoding/mojibake bugs.
30
31
 
31
32
  ## Summary
32
33
  Compass Canada order emails (order requested, manager-approval request, approved/rejected, in-transit, delivered, reminders) are sent in **each recipient's own language**. The French *wrapper* (subject + body template) was already selected correctly; what was broken was the **`{orderItems}` item block**, built from `Items.title`, which never consulted the `ItemTranslations` sidecar and had hardcoded English `P/N:` / `Qty:` labels. So a French user got a French email with English product names.
33
34
 
34
35
  Two load-bearing points, easy to get wrong again:
35
36
  1. **Language is per recipient, not per order.** One order's requester and manager can have different languages, so the item block must be rebuilt for each recipient.
36
- 2. **These emails go out as iso-8859-1**, so French text must be HTML-entity-escaped with an exact flag set (see Encoding) or the recipient sees mojibake or a blank product title.
37
+ 2. **Dynamic French text must be HTML-entity-escaped with an exact flag set** (see Encoding), or the recipient sees a double-encoded entity or a blank product title.
37
38
 
38
39
  Everything else in these emails has a pre-translated DB version (templates, `ItemTranslations`, label constants). **The one exception is the approver's free-text denial reason**, typed live in one language — machine-translated at send time through Talos.
39
40
 
@@ -80,11 +81,11 @@ Approvers in a bilingual org type in whichever language they think in, so a Fren
80
81
  The sender does **one** language lookup and derives both the template UUID and the note language from it, so they can never disagree. **Compass USA is excluded outright** — `resolveRecipientLanguageCode()` returns `'en'` for every non-Canada client, so treating `'en'` as a translatable target would add a ~1.2s synchronous external call to **every US rejection** for zero benefit; the client gate prevents that. **No machine-translation disclosure label (deliberate):** an earlier build appended `(traduction automatique) Texte original: <english>` under the translated note; the developer removed it, so the reader sees only the translated text. **Accepted risk:** a wrong machine translation now reaches the reader with no original to compare — which is why the gateway's system prompt is hardened against fabrication.
81
82
 
82
83
  ### Encoding — why those exact `htmlentities` flags
83
- Compass emails are transmitted as **iso-8859-1**: neither `_Email::send()` nor `_Worker_Infrastructure_Email_Send::Run()` sets `$mailer->CharSet`, so PHPMailer's default applies. Any raw UTF-8 accent in the rendered body arrives as mojibake. Entity-escaping the dynamic text sidesteps it — but both non-default arguments are required:
84
+ Until 2026-10-28 these emails went out as **iso-8859-1**, so raw UTF-8 accents arrived as mojibake and French subjects showed raw entities (`Livr&eacute;e`). The worker2 Send action now sends UTF-8 and decodes subject entities (TRUE-79683, see [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md)), so templates may hold entities or real UTF-8. Dynamic text is still entity-escaped, and both non-default arguments are required:
84
85
  - **`double_encode = false`** — in some environments `ItemTranslations.title` holds *literal HTML entities as text*. Re-encoding those produced `&amp;eacute;`, read as a literal `&eacute;`. (Prod and dev-sandbox data is clean raw-accent UTF-8; a stale local dataset had 71 of 215 rows entity-encoded. The flag makes the code correct under either data shape.)
85
86
  - **`ENT_SUBSTITUTE`** — passing explicit flags **drops** PHP 8.1's default `ENT_SUBSTITUTE`, and `htmlentities()` returns an **empty string** on invalid UTF-8. Omitting it silently blanks the whole product title.
86
87
 
87
- **`htmlentities()` alone is not enough** — it only converts characters with a **named** entity. French text carries the narrow no-break space **U+202F** before a colon, which has no name and survives as raw UTF-8 into the iso-8859-1 send. Use **`_String::encodeToAsciiHtmlEntities()`**, which runs the `htmlentities()` call above and then numbers every remaining character above U+007F. Applied to the rejection note, the item title and the part number. See [_String helpers](../../../2.0/apps/_underscore/features/string-html-entity-helpers.md).
88
+ **`htmlentities()` alone is not enough** — it only converts characters with a **named** entity. French text carries the narrow no-break space **U+202F** before a colon, which has no name and survives as raw UTF-8 (it mojibaked under the old iso-8859-1 send). Use **`_String::encodeToAsciiHtmlEntities()`**, which runs the `htmlentities()` call above and then numbers every remaining character above U+007F. Applied to the rejection note, the item title and the part number. See [_String helpers](../../../2.0/apps/_underscore/features/string-html-entity-helpers.md).
88
89
 
89
90
  ### Resilience — a language lookup must never break the caller
90
91
  `App_Query::execute()` (1.0, `library/app/query.php`) genuinely throws on any MySQL error, so this is a real failure mode. The guard is placed differently per tier, on purpose:
@@ -130,20 +131,22 @@ Tested through the **real private `sendRejectionEmailToUser()`** by reflection,
130
131
  - **Do not `COALESCE` the sidecar title with the source title in SQL.** `COALESCE(NULLIF(ItemTranslations.title, ''), Items.title)` yields a `utf8mb4_bin` result (MySQL cannot reconcile the two input collations). `SELECT`ing it works, but **any** comparison, `ORDER BY`, `GROUP BY` or `DISTINCT` on that expression throws **MySQL 1267 "Illegal mix of collations"** — reproduced against prod. Worse, the collations **differ per client database** (`Client_CompassCanada.Items.title` = `utf8mb4_0900_ai_ci`, `Client_Compass.Items.title` = `utf8mb4_unicode_ci`, `ItemTranslations.title` = `utf8mb4_unicode_ci` in both), so a hardcoded `COLLATE` clause is brittle. Select the two columns separately and fall back in PHP — which is also what the shared translation layer does (`_Model::load()`, deliberately not a JOIN/COALESCE).
131
132
  - **The `{role}` token appears TWICE in the French rejection template, with different grammar.** The second occurrence is `par un <strong>{role} TechHub</strong>`, which is already correct. So the missing elision must be fixed **in the template**, never by moving an article into the PHP role constant — that would break the second slot (`par un l'admin TechHub`). The template now reads `Refus&eacute;e par&nbsp;: {role}`, correct for both `admin` and `gestionnaire`.
132
133
  - **⚠ Do NOT "fix" the `approvalManagerId` subquery in `_Model_Compass_SalesOrder::postPut`.** It reads the step-2 `ApprovalDecision.decidedByUserId` (not `assignedToUserId`), which is `NULL` while the decision is pending — so `$managerEmail` stays empty and that arm never sends. Looks like a bug but is exactly what **prevents duplicate manager emails**: the canonical manager approval-request email is sent by `_Model_Compass_ApprovalDecision`, which correctly uses `ManagerDecision.assignedToUserId`. Switching the column would send a second manager email on every subsequent `SalesOrders` PUT. Evidence: prod `Logs_CompassCanada.Email` held 461 emails over ~3 weeks including 113 manager-approval emails, with **zero** duplicate `(to, subject)` pairs. Also on [Approval-Decision Flow](../../compass-usa/features/approval-decision-flow.md).
133
- - **French template bodies must be pure ASCII.** Under the iso-8859-1 send, a raw non-breaking space in a stored template body renders as mojibake. Eight Compass Canada French templates carried a stray `Â` (U+00C2) immediately followed by a raw NBSP (U+00A0) before the colon in `"Commande :"` (10 occurrences); replacing that 4-byte sequence with the `&nbsp;` entity makes those bodies ASCII. Applied to prod **and** dev-sandbox. Check for this whenever a French template is authored or pasted from Word/Docs.
134
+ - **Check pasted French templates for a stray `Â` (U+00C2) before an NBSP.** That is a double-encoded paste (Word/Docs) and shows as `Â` under any charset. Eight Compass Canada French templates had it before the colon in `"Commande :"` (10 occurrences); replaced with `&nbsp;` in prod **and** dev-sandbox.
134
135
  - **Compass US is provably unaffected.** `Client_Compass` has **0** `ItemTranslations` rows and only `en` in `Languages`, and every language branch is gated on `clientIdentifier === 'Compass_Canada'`, so the French path cannot alter a US email.
135
136
  - **"No language setting" and "chose English" are different things — keep them different.** `resolveRecipientLanguageCode()` returned `'en'` for both, so there was no way to tell a user who picked English from one never asked. Use `resolveExplicitRecipientLanguageCode()` (returns `''` for "no row") anywhere the answer decides whether to spend a **paid, synchronous external call**. Keep the old method for template selection, where an English default is correct.
136
137
  - **A translation must never be able to block an email.** Two independent guards: the note falls back to the original text on any failure, and the catch is on **`Throwable`** — a class missing from the deployed `_underscore` branch throws `Error`, not `Exception`. Not theoretical: the branch actually deployed to a tier is decided by the `ENVIRONMENT` variable, not the EB environment name — see [ENVIRONMENT drives the _underscore branch](../../../2.0/apps/api2/features/environment-variable-drives-underscore-branch.md).
137
138
  - **The 1.0 Compass Canada crons have no escaping gap — verified, no change made.** Prod `Client_CompassCanada.ItemTranslations` holds **220** French titles and **zero** contain any character above U+00FF, so the nine `worker/crons/toga2/compasscanada` crons' plain `htmlentities()` is sufficient for the data that exists. Re-check if a future import introduces en dashes, curly quotes or narrow no-break spaces — then those crons need the same ASCII-safe encoder (which lives in 2.0 and would have to be duplicated in `library`).
138
139
 
139
140
  ## Change history
141
+ - 2026-10-28 — Mail now sent UTF-8 with subject entities decoded (French subjects no longer show `&eacute;`); delivered email moved to worker2 (TRUE-79683). (bala)
140
142
  - 2026-09-25 — renamed product references from "TogaIQ" to "Talos" (rebrand; TOGa IQ is now the separate `toga25-iq` app) (jcardinal)
141
143
 
142
144
  ## Related
143
145
  - [Compass Canada profile](../profile.md)
144
146
  - [Compass Approval-Decision Flow](../../compass-usa/features/approval-decision-flow.md) — the shared parent that sends the approval/notification emails and resolves EN/FR templates.
145
147
  - [Language Translation Layer](../../../2.0/apps/api2/features/language-translation-layer.md) — the `ItemTranslations` sidecar and its PHP-side fallback rule.
146
- - [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md) — the queue + Send worker, and the missing `CharSet` that makes these emails iso-8859-1.
148
+ - [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md) — the queue + Send worker and its UTF-8 / subject-decode rule.
149
+ - [Compass delivered email (worker2)](../../compass-usa/features/delivered-emails.md) — per-box delivered email for US + Canada; French labels in its `LABELS`.
147
150
  - [Talos Gateway Client](../../../2.0/apps/_underscore/features/togaiq-gateway-client.md) — the AI gateway the denial-reason translation calls, and its anti-fabrication prompt.
148
151
  - [_String helpers](../../../2.0/apps/_underscore/features/string-html-entity-helpers.md) — the ASCII-safe entity encoder these emails require.
149
152
  - [Compass Partial In-Transit & Delivered Emails](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md) — the 1.0 crons whose item rows this localizes.
@@ -20,10 +20,11 @@ project: _Underscore
20
20
  client: compass-canada
21
21
  type: profile
22
22
  status: active
23
- updated: 2026-10-26
23
+ updated: 2026-10-28
24
24
  owners: [jcardinal, bala, tcox, apeterson, ajean, mhammontree, rgirish]
25
25
  files: []
26
26
  related:
27
+ - ../compass-usa/features/delivered-emails.md
27
28
  - ../compass-usa/profile.md
28
29
  - ../../2.0/apps/toga2-commerce/features/storefront-persona-gating-and-home-counts.md
29
30
  - ../../2.0/apps/toga2-commerce/features/kit-item-price-display.md
@@ -195,9 +196,10 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
195
196
  Compass USA's 1/3/4. See [Surface Resolver](../../2.0/apps/_underscore/features/surface-resolver.md).
196
197
  - Customer language preference: `UserGlobalSettings.settingId = 2` (`en` / `fr-CA`); customer-
197
198
  facing emails are sent in EN or FR accordingly — **per recipient**, including the `{orderItems}`
198
- block, and entity-escaped because the send is iso-8859-1. See
199
- [French Order Email Localization](features/french-order-email-localization.md). French
200
- `EmailTemplates` bodies must be pure ASCII.
199
+ block, with dynamic text entity-escaped. Mail is sent UTF-8 since 2026-10-28, so templates may
200
+ use entities or real UTF-8. See
201
+ [French Order Email Localization](features/french-order-email-localization.md). Delivered email:
202
+ [Compass delivered email (worker2)](../compass-usa/features/delivered-emails.md).
201
203
  - **⚠ Open security hole (2026-10-09, not fixed):** `Client_CompassCanada/2026-01-15 - UserGlobalSettingsAcl.sql`
202
204
  grants UserGlobalSettings writes with **no own-row check** — a user could write another user's row. See
203
205
  [own-row guards](../../2.0/apps/api2/features/api-payload-interceptors.md#own-row-guards-in-a-pre-interceptor).
@@ -7,6 +7,7 @@
7
7
  | [Compass Contact Email Resolution & Crossed `primaryContactEmailAddressId` Pointers](features/contact-email-resolution.md) | 2.0 | Any Compass code reading a contact's email must assert the email row belongs to that contact (~14% of pointers are crossed); open before reading Compass contact |
8
8
  | [Corporate vs Field vs Foodbuy laptop personas (Levy/Compass x Corporate/Field + Foodbuy, PEOPLE cron)](features/corporate-vs-field-laptop-personas.md) | 2.0 | `PeopleFile` gives every US user one laptop-segment persona per half (Levy: Corporate/Field; Compass: Corporate/Field/Foodbuy) by cost center / sector `J00000`. |
9
9
  | [Cost Centers — Unit Locations, numeric-only policy](features/cost-centers.md) | 2.0 | A Compass "cost center" is a Unit-level Location keyed by `c_erpSystemEntityId` (NOT a `CostCenters` row), numeric-only since 2026-07-06; open for cost-center d |
10
+ | [Compass USA + Canada per-package delivered email (worker2 DeliveredEmails, postPut queue + backup cron)](features/delivered-emails.md) | 2.0 | Compass USA + Canada "delivered" email, one per box (tracking number), sent by worker2 when a box turns DELIVERED; open for any delivered-email bug, resend, tem |
10
11
  | [Compass isFulfillable — Data Quality & the Type-Derived Rule](features/isfulfillable-data-quality-and-type-rule.md) | 2.0 | Why Compass storefront Qty-Fulfilled cells stay dead — `Items.isFulfillable` NULLs are structurally unreachable, some set values are wrong, and the type-derived |
11
12
  | [Compass Item Catalogs (Compass vs Agilant) & the Duplicate Items the NetSuite Sync Creates](features/item-catalogs-and-duplicate-items.md) | 1.0 | `Client_Compass.Items` is split across catalogs (Compass vs Agilant) and the NetSuite sync creates duplicate rows for one product; open first whenever a Compass |
12
13
  | [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two config-only Compass TableViews (ids 12 & 13) showing fulfilled items in toga2-supply, re-rooted off Units and tracking sourced at item level; open when a fu |
@@ -0,0 +1,77 @@
1
+ ---
2
+ title: Compass USA + Canada per-package delivered email (worker2 DeliveredEmails, postPut queue + backup cron)
3
+ framework: "2.0"
4
+ repo: worker2
5
+ project: Worker
6
+ client: compass-usa
7
+ type: client-feature
8
+ status: active
9
+ updated: 2026-10-28
10
+ owners: ["bala"]
11
+ files:
12
+ - worker2/Worker/Client/Compass/DeliveredEmails.php
13
+ - _underscore/Model/Compass/TrackingNumber.php
14
+ - dbchanges2/Client_Compass/2026-10-19a - DeliveredEmails.sql
15
+ - dbchanges2/Client_CompassCanada/2026-10-19a - DeliveredEmails.sql
16
+ - dbchanges2/Core/2026-10-16a - CompassDeliveredEmailsCron.sql
17
+ - worker1.5/schedules/cron.worker.json
18
+ related:
19
+ - ../../../2.0/apps/worker2/features/tracking-status-refresh.md
20
+ - ../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md
21
+ - ../../compass-canada/features/french-order-email-localization.md
22
+ - ../../../2.0/apps/_underscore/features/email-send-pipeline.md
23
+ - ../../../2.0/apps/api2/features/api-payload-interceptors.md
24
+ - contact-email-resolution.md
25
+ - ../workflows/odp-asn-replay-backfill.md
26
+ ---
27
+ Compass USA + Canada "delivered" email, one per box (tracking number), sent by worker2 when a box turns DELIVERED; open for any delivered-email bug, resend, template, link or tracking-button change.
28
+
29
+ ## Summary
30
+ Replaced the 1.0 `send_delivered_email.php` crons (TRUE-79683, switched off in `worker1.5/schedules/cron.worker.json`). Those asked UPS themselves and skipped boxes worker2's [tracking-status refresh](../../../2.0/apps/worker2/features/tracking-status-refresh.md) had already marked DELIVERED, so most delivered emails were lost. Now the refresh's api2 PUT is the trigger.
31
+
32
+ ## How it works
33
+ 1. worker2 `Sync/TrackingNumbers/Refresh` PUTs `TrackingNumbers.status` through api2.
34
+ 2. `_Model_Compass_TrackingNumber::postPut` (empty `Usa` / `Canada` subclasses; `ApiPayloadInterceptors` row record 62, POST-processing, PUT, in each client DB) **only queues** `_Worker::runTask('Client/Compass/DeliveredEmails/SendForTrackingNumber', {tenant: US|CA, trackingNumberUuid})` for each box the PUT set to DELIVERED. Status comes from the request body when the reply omits it.
35
+ 3. `SendForTrackingNumber` reads the saved row, sends, stamps.
36
+ 4. Backup: `Core.CronJobs` "Compass Delivered Emails", `Client/Compass/DeliveredEmails/SendPending`, `*/10 * * * *`, `{"dryRun": false}`, both tenants in one run, max 100 boxes per tenant.
37
+ 5. **Claim before send:** both paths run `UPDATE TrackingNumbers SET c_dtDeliveredEmailSent = NOW() ... WHERE id = X AND c_dtDeliveredEmailSent IS NULL`; only the caller that changes the row sends. One email per box per order. A failed send throws and rolls the claim back, so the next run retries.
38
+
39
+ Candidate box: `status = DELIVERED`, linked to an order through the ASN unit chain, has an `ItemFulfillmentItems_TrackingNumbers` row, `c_dtDeliveredEmailSent IS NULL`. Cron window: `GREATEST(TrackingNumbers.dtUpdated, AdvanceShippingNotices.dtCreated)` within `LOOKBACK_DAYS` (14). The instant job has no window.
40
+
41
+ Orders per box: every buyer SO on the ASN's PO that has a line in this box (one email per SO). Recipient: newest active `Users` row with an email, else `Contacts.primaryContactEmailAddressId` with the ownership guard ([contact email resolution](contact-email-resolution.md)). CC from the order. `Notifications` row only when `userId > 0`.
42
+
43
+ Template: full when this box is the whole order, else partial (sections: this box, already delivered, in transit, being prepared). UUIDs in `TENANT_SETTINGS`; Canada picks EN/FR per recipient (`UserGlobalSettings.settingId = 2`).
44
+
45
+ ### Template variables and links (designer review, 2026-10-28)
46
+ - `{userOrderUrl}` (all 6 templates, US + CA EN/FR, full + partial): order number links to `https://<commerceHost>/order-details?uuid=<SalesOrders.uuid>&type=orderHistory`. `commerceHost` per tenant: `compass.togacommerce.com` / `compasscanada.togacommerce.com`.
47
+ - `{trackingUrl}` (Track Shipment button), `buildTrackingUrl()` order:
48
+ 1. `TrackingNumbers.trackingUrl` (supplier URL, e.g. Convey), when it is a safe `http(s)` URL.
49
+ 2. `ShippingCarriers.urlLinkPrefix` + number, only when the prefix takes a number (ends in `=`, or a non-root path ending `/`).
50
+ 3. A home-page prefix as is (the number is not added).
51
+ 4. `_Model_Client_ShippingCarrier::generateTrackingNumberLink(number)` (by number pattern).
52
+ This is the reverse of the 1.0 in-transit cron, which puts the carrier prefix first.
53
+ - French labels (`LABELS`): part number `R&eacute;f :` in every delivered email (full and partial). Heading `Articles dans cette exp&eacute;dition`. The 2.0 order emails still use `N/P`.
54
+
55
+ ### Manual actions (worker2 POST)
56
+ - Dry run: `{"action":"Client/Compass/DeliveredEmails/SendPending","parameters":{"dryRun":true}}`.
57
+ - Test one box to yourself (nothing stamped): add `"dryRun":false,"tenant":"CA","trackingNumber":"...","testRecipientEmail":"<you>@togatech.com"`. `trackingNumber` and `testRecipientEmail` must go together; test address must be `@togatech.com` / `@goagilant.com`.
58
+ - Resend by order: `Client/Compass/DeliveredEmails/SendForSalesOrders` `{tenant, salesOrderNumbers[1..50], dryRun}`. Uses the order's latest delivered box. Without `testRecipientEmail` the customer gets it again and the box is stamped.
59
+
60
+ ## Gotchas
61
+ - **Never send inside the interceptor.** api2 commits the Logs DB even when the client DB rolls back (email for a PUT that never saved), it needs try/catch, and a bad interceptor breaks every tracking PUT. The postPut only queues.
62
+ - **Deploy `_underscore` before the interceptor row.** A row whose method does not exist breaks every tracking-number PUT.
63
+ - **The instant job can run before api2 commits.** It then finds no DELIVERED row and sends nothing. The cron sends it later.
64
+ - **The instant job never stamps a box with no order or recipient yet** (the ASN or fulfillment link can land after DELIVERED). The cron stamps it without an email, or it would hold a batch slot for 14 days.
65
+ - **Carriers the refresh cannot poll (freight) never reach DELIVERED**, so they never get this email.
66
+ - **Canada Precision and Grand & Toy prefixes are home pages only.** Their button opens the carrier site without the number (expected, step 3 above).
67
+ - **To stop emails during a bulk load, stamp `c_dtDeliveredEmailSent`** on the loaded tracking ids first. Both paths skip a stamped box. See [ODP ASN replay backfill](../workflows/odp-asn-replay-backfill.md).
68
+
69
+ ## Change history
70
+ - 2026-10-28 — Designer review: `{userOrderUrl}` Commerce order link in all templates, supplier `trackingUrl` first for the button, French `R&eacute;f :` label (TRUE-79683). (bala)
71
+ - 2026-10-26 — Created: worker2 per-box delivered email, Compass postPut only queues, 10-minute cron backup, `c_dtDeliveredEmailSent` claim; 1.0 delivered crons off (TRUE-79683). (bala)
72
+
73
+ ## Related
74
+ - [Tracking-status refresh](../../../2.0/apps/worker2/features/tracking-status-refresh.md) — the status writer that triggers this.
75
+ - [Compass in-transit emails (1.0)](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md)
76
+ - [Compass Canada French emails](../../compass-canada/features/french-order-email-localization.md)
77
+ - [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md)
@@ -22,10 +22,11 @@ project: _Underscore
22
22
  client: compass-usa
23
23
  type: profile
24
24
  status: active
25
- updated: 2026-10-27
25
+ updated: 2026-10-28
26
26
  owners: [jcardinal, bala, tcox, apeterson, dfranks, akhokhani, ajean, rgirish, amadda]
27
27
  files: []
28
28
  related:
29
+ - features/delivered-emails.md
29
30
  - features/talos-supply-assistant.md
30
31
  - features/saml-sso.md
31
32
  - ../../2.0/apps/_underscore/features/sales-order-po-number-sourcing.md
@@ -6,14 +6,14 @@ project: _Underscore
6
6
  client: compass-usa
7
7
  type: workflow
8
8
  status: active
9
- updated: 2026-10-13
9
+ updated: 2026-10-28
10
10
  owners: ["bala"]
11
11
  files:
12
12
  - _underscore/Model/Compass/SalesOrder.php
13
13
  - _underscore/Model/Compass/AdvanceShippingNotice.php
14
14
  - worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php
15
15
  - worker/crons/toga2/compass/update_salesorder_status_from_odp.php
16
- - worker/crons/toga2/compass/send_delivered_email.php
16
+ - worker2/Worker/Client/Compass/DeliveredEmails.php
17
17
  - worker/schedules/cron.worker.sync.json
18
18
  related:
19
19
  - ../features/asn-to-item-fulfillment.md
@@ -33,13 +33,13 @@ So the only correct way to mark an ODP order fulfilled is to **create an ASN**.
33
33
 
34
34
  ## Steps
35
35
  1. **Repair item bridges first.** `postPost` creates an IF, and supply shows tracking, only when each ASN item maps to a SOI through `SalesOrderItems_PurchaseOrderItems`. Sweep the target POs for (a) PO lines with no bridge row and (b) PO lines linked to placeholder SOI id 1. Link or repoint by `VendorItems.itemId = SalesOrderItems.itemId` AND equal quantity. See the [bridge-gap gotcha](../features/mits-po-to-so-item-linking.md#gotchas).
36
- 2. **Turn off customer emails.** In `worker/schedules/cron.worker.sync.json` set `"active": 0` on "Update Compass SalesOrder Status From ODP CXML" (`update_salesorder_status_from_odp.php`) and "Delievered Emails Compass" (`send_delivered_email.php`). Deploy worker. A replayed ASN looks new to the in-transit cron (window is `AdvanceShippingNotices.dtCreated`, 7 days), and the delivered cron has no date window at all.
36
+ 2. **Turn off customer emails.** In-transit: set `"active": 0` on "Update Compass SalesOrder Status From ODP CXML" (`update_salesorder_status_from_odp.php`) and deploy; a replayed ASN looks new to it (window is `AdvanceShippingNotices.dtCreated`, 7 days). Delivered (worker2 since TRUE-79683, see [delivered email](../features/delivered-emails.md)): it emails any ASN-linked `DELIVERED` box with `c_dtDeliveredEmailSent IS NULL`, and its window also counts `AdvanceShippingNotices.dtCreated`, so a replay looks new. Stamp `c_dtDeliveredEmailSent = NOW()` on every tracking id already in `TrackingNumbers` that you will load, before step 4.
37
37
  3. **Build one ASN per ODP PO.** Every PO line at full PO-line quantity, tracking on the header and the last item (same shape as the 3b cron), **plus `advanceShippingNoticeItemUnits` carrying `advanceShippingNoticeItemUnitTrackingNumbers`** (the ODP cXML shape). Without units, supply's tracking column shows nothing. **Dedupe tracking numbers inside each ASN** (a repeat fails the whole POST with 1062).
38
38
  4. **POST `/v2/advance-shipping-notices`** authenticated as `Client_Compass.Apis` id 4 "Office Depot (Cxml)". The secret lives in that table row; read it at run time, never copy it into a script or doc. 2026-10-13 tooling was one-off scripts outside any repo (`build_odp_tracking_asns.py`, `replay_compass_asns.sh`).
39
39
  5. **Check every PO in the DB, not the curl output.** A curl `FAIL 000` (no response) saved nothing. Confirm the ASN row and its IF exist before resending, or you post a duplicate ASN.
40
40
  6. **If ASNs went in without units, add them by SQL.** Insert tracking-only `AdvanceShippingNoticeItemUnits` (`unitId` NULL) and `AdvanceShippingNoticeItemUnits_TrackingNumbers` rows with literal uuid4 values (2026-10-13: 416 units, 647 bridge rows).
41
- 7. **Retire the loaded tracking numbers.** `UPDATE TrackingNumbers SET c_dtInTransitEmailSent = NOW(), status = 'DELIVERED'` scoped to `TrackingNumbers.id IN (<tracking ids on the loaded ASNs>)` only.
42
- 8. **Turn both crons back on** (`"active": 1`) and deploy worker.
41
+ 7. **Retire the loaded tracking numbers.** `UPDATE TrackingNumbers SET c_dtInTransitEmailSent = NOW(), c_dtDeliveredEmailSent = NOW(), status = 'DELIVERED'` scoped to `TrackingNumbers.id IN (<tracking ids on the loaded ASNs>)` only. Without `c_dtDeliveredEmailSent` the worker2 delivered cron emails every one of them.
42
+ 8. **Turn the in-transit cron back on** (`"active": 1`) and deploy.
43
43
 
44
44
  ## Gotchas
45
45
  - **⚠ Never scope step 7 by "tracking created today".** That also catches live ODP ASNs that arrived the same day and silences real customer emails. The suppression SQL printed by `replay_compass_asns.sh` is this broad version; do not run it as is.
@@ -49,6 +49,7 @@ So the only correct way to mark an ODP order fulfilled is to **create an ASN**.
49
49
  - Step 7 sets `status = 'DELIVERED'` without asking the carrier. Only safe for old shipments; never use it on a recent batch.
50
50
 
51
51
  ## Change history
52
+ - 2026-10-28 — Delivered email is now worker2 (TRUE-79683): suppress it by stamping `c_dtDeliveredEmailSent`, not by a 1.0 cron switch. (bala)
52
53
  - 2026-10-13 — Created from the first ODP historical tracking backfill (279 orders to Fulfilled). (bala)
53
54
 
54
55
  ## Related
@@ -6,8 +6,8 @@ project: _Underscore
6
6
  client: rate
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-09-18
10
- owners: [mhammontree]
9
+ updated: 2026-10-28
10
+ owners: [mhammontree, bala]
11
11
  files:
12
12
  - _underscore/Model/Rate/Entitlement.php
13
13
  - worker2/Worker/Notification/EmailTemplate.php
@@ -69,17 +69,19 @@ Figma fonts (TOGa View 2.0 email designs):
69
69
  - **AIG Terms & Conditions URL** — `WARRANTY_TERMS_URL` set **provisionally** to `https://hometechsupport.rate.togaview.com/terms/home-warranty`. **Provisional** — the PM expects it to change before/after go-live.
70
70
 
71
71
  Remaining items are **optional / separate, not go-live blockers**:
72
- - The `_Email.php` UTF-8 `CharSet` framework fix (root-cause of the ASCII-only-subjects workaround below — see [`email-template-sending.md`](../../../2.0/apps/_underscore/features/email-template-sending.md)).
73
72
  - An Axiforma web font for the footer "POWERED BY" label (currently rendered as the logo image).
74
73
 
75
74
  ## Gotchas
76
75
  - **Shared interceptor with TRUE-79533 (WH per-address guard) — same `prePost`/`postPost`.** This workflow's `postPost` trigger + helpers live in the **same** `_Model_Rate_Entitlement` interceptors that the [Whole Home Warranty per-address guard](whole-home-warranty-purchase-guard.md) edits, and TRUE-79251 also adds the passThrough `const PASSTHROUGH_KEY_FULFILLMENT_PREFERENCE` + the contractFulfillmentPreference (EV-8) fix inside `prePost`. A bad `_production`→`_beta` merge already mis-resolved this once and fatally broke every Rate purchase save. When merging either ticket across branches, hand-merge into ONE `prePost()` keeping both features' logic — see the **shared-interceptor merge hazard** gotcha on that doc for the correct resolution and why `php -l` is not a sufficient gate.
77
76
  - **`border-radius` on a `<td>` renders SQUARE in many clients (Outlook).** Use an `<img>` for circular badges/icons (e.g. the green circle-check) instead of a CSS-rounded table cell.
78
- - **ASCII-only subjects (mojibake workaround).** `_Email::send()` does not set PHPMailer `CharSet=UTF-8`, so non-ASCII subjects/body (em dash, curly quotes, accents) mojibake. This workflow uses ASCII-only subjects; the proper root-cause fix is tracked on [`email-template-sending.md`](../../../2.0/apps/_underscore/features/email-template-sending.md).
77
+ - **ASCII-only subjects are no longer required.** They were a mojibake workaround; since 2026-10-28 (TRUE-79683) the worker2 Send action sends UTF-8. See [`email-send-pipeline.md`](../../../2.0/apps/_underscore/features/email-send-pipeline.md).
79
78
  - **⚠ A non-prod tenant missing its `EmailTemplates` seed looks like a code error, not a data gap.** It surfaces as a FAILED `Notification/EmailTemplate/Send` worker job naming the template **uuid**: *"Could not load `_Model_Client_EmailTemplate` object based upon these criteria: `uuid` = '2c4f8a1e-…'"* (dev-sandbox `Core.WorkerJobs` 141941, 2026-09-18). dev-sandbox `Client_Rate.EmailTemplates` held only 2 rows (Forgot Password, Talos); prod has both Rate templates as ids 3 and 4. Fix: run the existing `dbchanges2/Client_Rate/2026-06-30a - Rate purchase email templates.sql` on that environment. Check the tenant's `EmailTemplates` rows before debugging the sender.
80
79
  - **⚠ OPEN — both templates render badly in email dark mode (from TRUE-79251; needs a design decision, not a code fix).** (1) The CTA button becomes an **empty dark bar** — background `#21292d` disappears into the client's dark theme and the white label is invisible. (2) The TOGA footer wordmark shows inside a **dark block**, because the image was rasterized onto the exact footer colour `#21292d`, which the client lightens in dark mode. Light mode is fine, which is why neither was caught — the *"flatten the asset onto the exact region background"* recipe above is precisely what breaks under dark mode.
81
80
  - **Worker DB registration** — see [`notification-email-template.md`](../../../2.0/apps/worker2/features/notification-email-template.md): worker actions do not auto-register `DB_CLIENT`.
82
81
 
82
+ ## Change history
83
+ - 2026-10-28 — ASCII-only subject workaround retired: mail now sent UTF-8 (TRUE-79683). (bala)
84
+
83
85
  ## Related
84
86
  - [Rate profile](clients/rate/profile.md)
85
87
  - [AIG contract creation](clients/rate/features/aig-contract-creation.md)
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.1179",
3
+ "version": "1.0.1180",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",