toga-ai 1.0.1179 → 1.0.1180
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md +18 -27
- package/knowledge/2.0/apps/_underscore/features/email-send-pipeline.md +24 -29
- package/knowledge/2.0/apps/_underscore/features/email-template-sending.md +6 -11
- package/knowledge/2.0/apps/_underscore/features/string-html-entity-helpers.md +5 -2
- package/knowledge/2.0/apps/worker2/features/sandbox-dev-worker-environments.md +21 -4
- package/knowledge/2.0/apps/worker2/features/tracking-status-refresh.md +13 -13
- package/knowledge/INDEX.md +1 -1
- package/knowledge/clients/compass-canada/INDEX.md +1 -1
- package/knowledge/clients/compass-canada/features/french-order-email-localization.md +11 -8
- package/knowledge/clients/compass-canada/profile.md +6 -4
- package/knowledge/clients/compass-usa/INDEX.md +1 -0
- package/knowledge/clients/compass-usa/features/delivered-emails.md +77 -0
- package/knowledge/clients/compass-usa/profile.md +2 -1
- package/knowledge/clients/compass-usa/workflows/odp-asn-replay-backfill.md +6 -5
- package/knowledge/clients/rate/features/service-purchase-emails.md +6 -4
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["bala", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/compass/update_salesorder_status_from_odp.php
|
|
@@ -21,6 +21,7 @@ files:
|
|
|
21
21
|
- worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php
|
|
22
22
|
- dbchanges2/Client_Compass/2026-09-03 - compass_intransit_email_backlog_backfill.sql
|
|
23
23
|
related:
|
|
24
|
+
- ../../../clients/compass-usa/features/delivered-emails.md
|
|
24
25
|
- ../../../clients/compass-usa/features/tracking-number-origin-and-asn-link.md
|
|
25
26
|
- ../../../clients/compass-usa/features/contact-email-resolution.md
|
|
26
27
|
- ../../../2.0/apps/worker2/features/tracking-status-refresh.md
|
|
@@ -42,6 +43,11 @@ cron builds three zones: **Items in this shipment** (just that package), **Alrea
|
|
|
42
43
|
and earlier packages). The dynamic HTML is injected into a stored `EmailTemplates` body via the
|
|
43
44
|
`sendEmail` scripted API. Canada renders EN or FR per the user's language setting.
|
|
44
45
|
|
|
46
|
+
**The delivered email is no longer 1.0 (TRUE-79683).** It is the worker2 job
|
|
47
|
+
[Compass delivered email](../../../../clients/compass-usa/features/delivered-emails.md); both
|
|
48
|
+
`send_delivered_email.php` crons are switched off in `worker1.5/schedules/cron.worker.json`. This doc
|
|
49
|
+
now owns the **in-transit** crons only; delivered-cron notes below are history.
|
|
50
|
+
|
|
45
51
|
## How it works
|
|
46
52
|
|
|
47
53
|
### Key files / entry points
|
|
@@ -178,12 +184,9 @@ IFNULL(Users.email, ContactEmailAddresses.emailAddress) AS contactEmail
|
|
|
178
184
|
The same `IFNULL` resolves the display name, and the `WHERE` requires the resolved address to be
|
|
179
185
|
non-empty.
|
|
180
186
|
|
|
181
|
-
**Delivered
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
`Contacts.primaryContactEmailAddressId` → `ContactEmailAddresses`. Rows with no usable address are
|
|
185
|
-
excluded in SQL (no UPS call, no email, status stays non-`DELIVERED`, so it sends later if an
|
|
186
|
-
address appears). `Notifications` rows are written only when `userId > 0`. Never go back to a plain
|
|
187
|
+
**Delivered emails pick ONE `Users` row** (TRUE-82847, kept in the worker2 job): newest active user
|
|
188
|
+
with an email (`Users.id = (SELECT MAX(id) … isActive = 1 AND email <> '')`), fallback
|
|
189
|
+
`Contacts.primaryContactEmailAddressId` → `ContactEmailAddresses`. Never go back to a plain
|
|
187
190
|
`INNER JOIN Users ON contactId` — it yields one email per `Users` row (see gotcha).
|
|
188
191
|
|
|
189
192
|
**The `ContactEmailAddresses.contactId = Contacts.id` predicate is not optional.** About 14% of
|
|
@@ -352,18 +355,12 @@ the tracking number as emailed (it retries next run).
|
|
|
352
355
|
future change to the pendingFulfillment or canceled branches of
|
|
353
356
|
`_Model_Compass_SalesOrder::_status()` must be mirrored into all three crons by hand.** See
|
|
354
357
|
[Compass Fulfilled vs Partially Fulfilled](../../../clients/compass-usa/features/order-fulfillment-status-per-line.md).
|
|
355
|
-
-
|
|
356
|
-
|
|
357
|
-
(
|
|
358
|
-
|
|
359
|
-
and
|
|
360
|
-
|
|
361
|
-
[tracking-status refresh (worker2)](../../../2.0/apps/worker2/features/tracking-status-refresh.md)
|
|
362
|
-
writes `status` via api2 PUT for **all** clients. If the refresh marks a Compass row `DELIVERED`
|
|
363
|
-
**first**, this cron's guard **skips** it and the delivered email **never sends** — a real conflict
|
|
364
|
-
to resolve before the refresh is enabled for Compass. There is **no `postPut` interceptor** on
|
|
365
|
-
tracking-numbers today; the plan (not built) is to move this email logic into a Compass `postPut`
|
|
366
|
-
interceptor and retire this 1.0 cron.
|
|
358
|
+
- **Why the 1.0 delivered crons were retired (TRUE-79683).** They polled UPS themselves and guarded
|
|
359
|
+
on `status <> 'DELIVERED'`, so every box the worker2
|
|
360
|
+
[tracking-status refresh](../../../../2.0/apps/worker2/features/tracking-status-refresh.md) marked
|
|
361
|
+
DELIVERED first was skipped and never emailed (12 Oct: 50 of 80 orders). They also had no date
|
|
362
|
+
window and ignored the send result. Replaced by the
|
|
363
|
+
[worker2 delivered email](../../../../clients/compass-usa/features/delivered-emails.md).
|
|
367
364
|
|
|
368
365
|
- **A contact can have 2 active `Users` rows** (old one with NULL email + new one with a new HR
|
|
369
366
|
username, e.g. `thompj53` → `thompj55`). Prod 2026-10-15: 2 such contacts (326758, 283892). Any
|
|
@@ -373,16 +370,10 @@ the tracking number as emailed (it retries next run).
|
|
|
373
370
|
behavior](../../../clients/compass-usa/features/people-file-user-lifecycle.md); the new-username
|
|
374
371
|
case is not proven in importer code.
|
|
375
372
|
- **One tracking number on several sales orders sends one delivered email per SO.** Known, left
|
|
376
|
-
as is (decided 2026-10-15, TRUE-82847).
|
|
377
|
-
- Delivered template = prod `emailTemplateId` 26 in `Logs_Compass.Email`; baseline 2026-09-15..10-15:
|
|
378
|
-
959 SENT, 1 FAILED.
|
|
379
|
-
- **⚠ The delivered cron never checks the send result.** It calls `App_Api_Toga2::send(..., false)`
|
|
380
|
-
(`$throwExceptionOnApiError = false`) and ignores the response, so it marks
|
|
381
|
-
`TrackingNumbers.status = 'DELIVERED'` and inserts notifications even when api2 rejects the email.
|
|
382
|
-
Its "only mark DELIVERED after confirmed send" comment is wrong. "Sent" only means queued to
|
|
383
|
-
`Logs_Compass.Email`; worker2 `Infrastructure/Email/Send` does the real send. Not changed.
|
|
373
|
+
as is (decided 2026-10-15, TRUE-82847; same in the worker2 job).
|
|
384
374
|
|
|
385
375
|
## Change history
|
|
376
|
+
- 2026-10-28 — Delivered email moved to worker2 (postPut queue + 10-minute cron), 1.0 delivered crons off in worker1.5; doc now covers in-transit only (TRUE-79683). (bala)
|
|
386
377
|
- 2026-10-15 — Delivered crons (USA + Canada) pick one active `Users` row with email + Contacts fallback; documented the 2-active-users duplicate. (jcardinal)
|
|
387
378
|
- 2026-10-15 — Recorded that the delivered cron ignores the send result (TRUE-82846). (bala)
|
|
388
379
|
- 2026-10-13 — Recorded that the USA delivered cron has no date window, so bulk ASN loads must suppress first. (bala)
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["bala", "tcox", "dfranks", "apeterson", "jcardinal", "rgirish"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Email.php
|
|
@@ -68,7 +68,9 @@ SMTP** using PHPMailer, then flips each row to `SENT` or `FAILED`. So a 2.0 emai
|
|
|
68
68
|
`emailTemplateId` (after `dtCreated`), `reviewOfEmailId` (self FK, `ON DELETE SET NULL` — links a
|
|
69
69
|
review copy to the held original), `dtReviewed` (after `dtSent`), `reviewedBy` (after
|
|
70
70
|
`failureReason`). Migration: `dbchanges2/Logs_Client/2026-09-23a - EmailReview.sql` — must run
|
|
71
|
-
**before** the `_underscore` deploy or every `send()` fails
|
|
71
|
+
**before** the `_underscore` deploy or every `send()` fails with 1054 `Unknown column
|
|
72
|
+
'emailTemplateId'`. On that error in a non-prod env, apply this existing file to the `Logs_<Client>`
|
|
73
|
+
DB; do not write new SQL (beta `Logs_Compass` / `Logs_CompassCanada` lacked it until 2026-10-28). Who writes `HELD`: see
|
|
72
74
|
[email-template-sending.md](email-template-sending.md) *Review hold*; who releases it: Tools
|
|
73
75
|
[/emails](../../../../1.0/apps/tools/features/email-queue-review.md).
|
|
74
76
|
|
|
@@ -141,33 +143,25 @@ derive one from the other).
|
|
|
141
143
|
- **The Send worker forces HTML mode.** `sendEmail()` calls `$mailer->IsHTML(true)`
|
|
142
144
|
**unconditionally**, ignoring the sender's `setIsHtml(false)`. A caller that queued a
|
|
143
145
|
"plain-text" message is still transmitted HTML-mode. Author bodies accordingly.
|
|
144
|
-
-
|
|
145
|
-
`
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
stray `Â` (U+00C2) pasted into a template body renders as mojibake in the delivered mail — use
|
|
164
|
-
` ` / `é` entities instead. Real defect: 8 Compass Canada French templates carried a
|
|
165
|
-
`Â`+NBSP sequence before the colon in `"Commande :"`.
|
|
166
|
-
|
|
167
|
-
The root-cause fix is still to set `$mailer->CharSet = 'UTF-8'` — but that changes **every**
|
|
168
|
-
client's mail, so it must be its own tested change, not a drive-by. See
|
|
169
|
-
[`email-template-sending.md`](email-template-sending.md) and
|
|
170
|
-
[Compass Canada French order emails](../../../clients/compass-canada/features/french-order-email-localization.md).
|
|
146
|
+
- **Mail is sent as UTF-8, and subjects are entity-decoded (fixed 2026-10-28, TRUE-79683).**
|
|
147
|
+
`_Worker_Infrastructure_Email_Send::sendEmail()` sets `$mailer->CharSet =
|
|
148
|
+
PHPMailer::CHARSET_UTF8` and sends the subject through `html_entity_decode($subject, ENT_QUOTES |
|
|
149
|
+
ENT_HTML401, 'UTF-8')`. Before, PHPMailer's default **iso-8859-1** label garbled every UTF-8
|
|
150
|
+
character in the stored `utf8mb4` text (em dash in True ClickUp alert subjects, UTF-8 NBSP in
|
|
151
|
+
Compass bodies), and subjects showed raw entities (all 7 Compass Canada French subjects read e.g.
|
|
152
|
+
`Livrée`, since at least Aug 2026). The body is HTML, so entities always rendered there.
|
|
153
|
+
Rules now:
|
|
154
|
+
- **Template subjects and bodies may use entities or real UTF-8.** Both arrive correct. Never write
|
|
155
|
+
anything that relies on iso-8859-1.
|
|
156
|
+
- Only `;`-terminated entities decode, so a bare `&` in a subject (`CC & MS Scrum`, `A&M`) is
|
|
157
|
+
unchanged. Prod check, 90 days of all `Logs_*`: only 5 subjects held entities, all Compass Canada
|
|
158
|
+
French.
|
|
159
|
+
- `_Email::send()` (queue side) still sets no `CharSet`; harmless, it only writes the row.
|
|
160
|
+
- **Still escape dynamic text for HTML** with `_String::encodeToAsciiHtmlEntities($value)` — see
|
|
161
|
+
[`string-html-entity-helpers.md`](string-html-entity-helpers.md) for its required
|
|
162
|
+
`double_encode = false` + `ENT_SUBSTITUTE` flags.
|
|
163
|
+
- A stray `Â` (U+00C2) before an NBSP in a stored template is bad data (double-encoded paste), not
|
|
164
|
+
a charset issue. Fix the row. Real defect: 8 Compass Canada French templates, `"Commande :"`.
|
|
171
165
|
- **⚠ SECURITY — hardcoded AWS SES SMTP credentials in `_underscore/Email.php`.** The
|
|
172
166
|
`SMTP_USERNAME` / `SMTP_PASSWORD` class constants hold **real** SES SMTP credentials committed
|
|
173
167
|
in source, violating the no-secrets-in-code rule. Documenting the **location only** — do not
|
|
@@ -214,6 +208,7 @@ derive one from the other).
|
|
|
214
208
|
for a hotfix.
|
|
215
209
|
|
|
216
210
|
## Change history
|
|
211
|
+
- 2026-10-28 — Send worker now sends UTF-8 and entity-decodes subjects; replaced the iso-8859-1 gotcha; 1054 `emailTemplateId` = apply the existing EmailReview file (TRUE-79683). (bala)
|
|
217
212
|
- 2026-10-16 — `SENT` ≠ delivered; linked the SES sender-identity/DMARC doc after Compass TechHub mail was DMARC-rejected (TRUE-82883) (bala)
|
|
218
213
|
- 2026-10-14 — Gotcha: client-sandbox missing `Logs_<Client>.Email` tables and a Send cron that never drains `PENDING` rows (rgirish)
|
|
219
214
|
- 2026-09-23 — Review hold: `HELD`/`CANCELLED` statuses, `emailTemplateId`/`reviewOfEmailId`/
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["jcardinal", "bala", "mhammontree", "apeterson"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Client/EmailTemplate.php
|
|
@@ -201,16 +201,10 @@ can keep using `sendEmail($api, ...)`.
|
|
|
201
201
|
Any fire-and-forget caller now propagates that exception, which is intended (so an email is
|
|
202
202
|
never silently marked as sent). Blast radius is every client, including the Compass/Quad
|
|
203
203
|
`SalesOrder`/`ApprovalDecision` order-placed emails.
|
|
204
|
-
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
`
|
|
208
|
-
change rather than a drive-by edit.
|
|
209
|
-
- **The transmit side doesn't set it either**, so the effective wire charset is **iso-8859-1**
|
|
210
|
-
end to end. For a body that genuinely needs accents (e.g. French), the working escape is
|
|
211
|
-
`htmlentities($v, ENT_QUOTES | ENT_SUBSTITUTE, 'UTF-8', false)` — both non-default arguments
|
|
212
|
-
matter. See the encoding-chain gotcha on
|
|
213
|
-
[`email-send-pipeline.md`](email-send-pipeline.md) for why.
|
|
204
|
+
- **Charset: fixed 2026-10-28 (TRUE-79683).** The worker2 Send action now transmits UTF-8 and
|
|
205
|
+
entity-decodes subjects, so subjects and bodies may hold real UTF-8 or entities; the old
|
|
206
|
+
ASCII-only workaround is no longer needed. Details on
|
|
207
|
+
[`email-send-pipeline.md`](email-send-pipeline.md).
|
|
214
208
|
- **⚠ SECURITY — live AWS SES SMTP credentials are hardcoded in `_underscore/Email.php`.**
|
|
215
209
|
The `SMTP_USERNAME` / `SMTP_PASSWORD` class constants (near the top of `_Email`) hold
|
|
216
210
|
**real** AWS SES SMTP credentials committed in source, violating the team no-secrets-in-code
|
|
@@ -243,6 +237,7 @@ worker method) in-process instead.
|
|
|
243
237
|
|
|
244
238
|
## Change history
|
|
245
239
|
|
|
240
|
+
- 2026-10-28 — CharSet gotcha closed: Send worker sends UTF-8 and decodes subject entities (TRUE-79683). (bala)
|
|
246
241
|
- 2026-10-20 — `EmailTemplates.kind` SYSTEM / NOTIFICATION; this doc = SYSTEM; linked the NOTIFICATION blocks doc (TRUE-82934). (jcardinal)
|
|
247
242
|
- 2026-10-19 — Recorded the public, unverified togatech.com use of sendEmail and the lockdown follow-up. (akhokhani)
|
|
248
243
|
- 2026-10-15 — Recorded the empty-string `to` + cc gap (queued, fails in worker2). (jcardinal)
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["bala"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/String.php
|
|
@@ -37,7 +37,7 @@ Entry points (`_underscore/String.php`):
|
|
|
37
37
|
- `ENT_SUBSTITUTE` — passing explicit flags **drops** PHP 8.1's default, and `htmlentities()` returns an **empty string** on invalid UTF-8, silently blanking the whole field.
|
|
38
38
|
2. `preg_replace_callback('/[\x{0080}-\x{10FFFF}]/u', …)` converting every remaining character above U+007F to a **numeric** entity (`&#NNNN;`) via `mb_ord()`.
|
|
39
39
|
|
|
40
|
-
**Why pass 2 exists:** `htmlentities()` only converts characters that have a *named* entity. French text routinely contains the **narrow no-break space U+202F** before a colon (`Commande :`), which has no name and survives pass 1 as **raw UTF-8**.
|
|
40
|
+
**Why pass 2 exists:** `htmlentities()` only converts characters that have a *named* entity. French text routinely contains the **narrow no-break space U+202F** before a colon (`Commande :`), which has no name and survives pass 1 as **raw UTF-8**. That corrupted mail while the Send worker used iso-8859-1. Since 2026-10-28 (TRUE-79683) mail goes out as UTF-8, so raw UTF-8 is safe too; ASCII output stays correct under any charset, so keep using the helper. Empty/`null` input returns `''`.
|
|
41
41
|
|
|
42
42
|
Use it for **every** dynamic string entering an email body — free-text notes, item titles, part numbers.
|
|
43
43
|
|
|
@@ -51,6 +51,9 @@ Use it for **every** dynamic string entering an email body — free-text notes,
|
|
|
51
51
|
- Not fixed as of 2026-08-05 (found while working elsewhere in the file) — worth its own ticket. Fix is a one-word change to `haystack:`.
|
|
52
52
|
- Secondary defect in the same method: `$indStart` has `strlen()` **added** before the `=== false` check, so the "not found" guard can never fire — `false + n` is `n`.
|
|
53
53
|
|
|
54
|
+
## Change history
|
|
55
|
+
- 2026-10-28 — Pass 2 is no longer needed for correctness: the Send worker now sends UTF-8 (TRUE-79683). Helper still recommended. (bala)
|
|
56
|
+
|
|
54
57
|
## Related
|
|
55
58
|
|
|
56
59
|
- [email-send-pipeline](email-send-pipeline.md)
|
|
@@ -6,9 +6,11 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["bala"]
|
|
11
11
|
files:
|
|
12
|
+
- worker2/LambdaFunctions/CronScheduler/lambda_function.py
|
|
13
|
+
- worker2/Controller/Index.php
|
|
12
14
|
- worker2/.ebextensions/git.php
|
|
13
15
|
- worker2/.ebextensions/git.json
|
|
14
16
|
- worker2/.ebextensions/php_include_underscore.config
|
|
@@ -31,13 +33,19 @@ behaves the same wastes a deploy cycle and hides stale code.
|
|
|
31
33
|
|
|
32
34
|
| EB environment | Tier | `ENVIRONMENT` | `_underscore` branch | Deployed by |
|
|
33
35
|
|---|---|---|---|---|
|
|
34
|
-
| `worker-invoke-sandbox-dev` | **Worker** (SQS) | `sandbox-dev` | `_sandbox-dev` | **by hand only** |
|
|
35
|
-
| `worker-sandbox-dev` | **WebServer** | `beta` | `_beta` | CodePipeline |
|
|
36
|
+
| `worker-invoke-sandbox-dev` | **Worker** (SQS) | `sandbox-dev` (`Core.Environments` id 23) | `_sandbox-dev` | **by hand only** |
|
|
37
|
+
| `worker-sandbox-dev` | **WebServer** | `beta` (`Core.Environments` id 4) | `_beta` | CodePipeline |
|
|
36
38
|
|
|
37
39
|
- `worker.beta.togahub.com` resolves to the same IPs as
|
|
38
40
|
`worker-sandbox-dev.us-east-1.elasticbeanstalk.com`, so an HTTP invoke tests the **WebServer** box.
|
|
39
41
|
- **The cron runs on `worker-invoke-sandbox-dev`.** CodePipeline never touches it, so it keeps
|
|
40
|
-
running old code until someone deploys it manually.
|
|
42
|
+
running old code until someone deploys it manually. Cron path: Lambda
|
|
43
|
+
`WorkerCronScheduler-SandboxDev` (`worker2/LambdaFunctions/CronScheduler/lambda_function.py`)
|
|
44
|
+
inserts a `Core.WorkerJobs` row (`jobType = 'CRON'`) → SQS `WorkerSandboxDevQueue` →
|
|
45
|
+
`worker-invoke-sandbox-dev`. **Are crons running?** Check `Core.WorkerJobs.dtStarted` for recent
|
|
46
|
+
`jobType = 'CRON'` rows.
|
|
47
|
+
- Direct HTTP calls (Postman) hit `worker.beta.togahub.com` = `worker-sandbox-dev`
|
|
48
|
+
(`ENVIRONMENT=beta`). Non-prod sends go to `devteam@goagilant.com` (debug redirect).
|
|
41
49
|
- `worker2/Config/beta.ini` and `Config/sandbox-dev.ini` both point at
|
|
42
50
|
`sandbox-dev.cluster-cb860gg2qw2n.us-east-1.rds.amazonaws.com`, so **finding your row in the
|
|
43
51
|
database proves nothing about which box wrote it.**
|
|
@@ -85,8 +93,17 @@ Before you go looking for a missing SQS message on a sandbox, confirm the produc
|
|
|
85
93
|
- **Never infer the environment from the EB environment name.** `worker-sandbox-dev` is
|
|
86
94
|
`ENVIRONMENT=beta`; the name and the value disagree on purpose-built tiers more often than not.
|
|
87
95
|
- **The clone deletes `.git`**, so the instance carries no commit id. Verify by file presence.
|
|
96
|
+
- **Each `Core.Environments` id needs its own `Core.DatabaseHosts` rows, including Logs
|
|
97
|
+
(`databaseId` 11).** Env 23 (`sandbox-dev`) had none for Logs, so every job on
|
|
98
|
+
`worker-invoke-sandbox-dev` failed `No Core Logs Database host found`
|
|
99
|
+
(`worker2/Controller/Index.php` ~260) and no beta cron ran. Fixed 2026-10-28: beta Core
|
|
100
|
+
`DatabaseHosts` id 4736 (env 23, Logs, host `sandbox-dev.cluster-cb860gg2qw2n.us-east-1.rds.amazonaws.com`).
|
|
101
|
+
Check this first when every sandbox-dev cron fails at once.
|
|
102
|
+
- **Send queued beta emails by hand:** POST `{"action":"Infrastructure/Email/Send/Run"}` to
|
|
103
|
+
`worker.beta.togahub.com`.
|
|
88
104
|
|
|
89
105
|
## Change history
|
|
106
|
+
- 2026-10-28 — Cron path (Lambda → `Core.WorkerJobs` CRON → SQS `WorkerSandboxDevQueue`), Environments ids 23 / 4, missing Logs `DatabaseHosts` row for env 23 fixed (id 4736), manual Send/Run (TRUE-79683). (bala)
|
|
90
107
|
|
|
91
108
|
- 2026-09-18 — Initial capture, from the NYCHH transfer-order-email work. Recorded that
|
|
92
109
|
`worker-invoke-sandbox-dev` (Worker tier, `ENVIRONMENT=sandbox-dev`, branch `_sandbox-dev`) runs
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["bala", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Sync/TrackingNumbers.php
|
|
@@ -140,8 +140,9 @@ bookkeeping columns.
|
|
|
140
140
|
|
|
141
141
|
### Decisions
|
|
142
142
|
- **Write path = api2 PUT, not direct-DB (deliberate).** Direct-DB was **rejected** so that a future
|
|
143
|
-
**`postPut` interceptor** on tracking-numbers can fire
|
|
144
|
-
|
|
143
|
+
**`postPut` interceptor** on tracking-numbers can fire emails on a status change. Used since
|
|
144
|
+
TRUE-79683: the Compass postPut queues the
|
|
145
|
+
[delivered email](../../../../clients/compass-usa/features/delivered-emails.md). This is a departure from other
|
|
145
146
|
worker2 crons (e.g. NYCHH asset-tag backfill) that write direct-DB to dodge the PHP-8.5 crash — here
|
|
146
147
|
the interceptor requirement wins, which is exactly why the crash below is a hard gate.
|
|
147
148
|
|
|
@@ -211,15 +212,14 @@ catch up first.** See
|
|
|
211
212
|
Open question: nothing today re-tries or reports parked rows. A periodic unpark (or a monitor on
|
|
212
213
|
parked-row count) is unbuilt work.
|
|
213
214
|
|
|
214
|
-
###
|
|
215
|
-
The
|
|
216
|
-
|
|
217
|
-
/
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
Plan (not built): move the email logic into a Compass `postPut` interceptor and retire the 1.0 cron.
|
|
215
|
+
### Compass delivered email is driven by this refresh (resolved, TRUE-79683)
|
|
216
|
+
The old 1.0 delivered crons skipped any box this refresh had already marked `DELIVERED`, so the email
|
|
217
|
+
never sent. Resolved: they are off, and `_Model_Compass_TrackingNumber::postPut` queues worker2
|
|
218
|
+
`Client/Compass/DeliveredEmails/SendForTrackingNumber` on each PUT that sets `DELIVERED` (10-minute
|
|
219
|
+
cron backup). See [Compass delivered email](../../../../clients/compass-usa/features/delivered-emails.md).
|
|
220
|
+
|
|
221
|
+
## Change history
|
|
222
|
+
- 2026-10-28 — Compass delivered-email conflict resolved: postPut on this refresh's PUT queues the worker2 delivered email (TRUE-79683). (bala)
|
|
223
223
|
|
|
224
224
|
## Related
|
|
225
225
|
|
|
@@ -228,6 +228,6 @@ Plan (not built): move the email logic into a Compass `postPut` interceptor and
|
|
|
228
228
|
- [Running worker2 locally](../workflows/running-worker2-locally.md)
|
|
229
229
|
- [Carrier shipping labels](../../_underscore/features/carrier-shipping-labels.md) — carrier-library batch support.
|
|
230
230
|
- [Tracking-number bridges](../../_underscore/features/tracking-number-bridges.md)
|
|
231
|
-
- [API Payload Interceptors](../../api2/features/api-payload-interceptors.md) — the
|
|
231
|
+
- [API Payload Interceptors](../../api2/features/api-payload-interceptors.md) — how the Compass `postPut` interceptor is registered.
|
|
232
232
|
- [Compass in-transit/delivered emails (1.0)](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md)
|
|
233
233
|
- [NYCHH asset-tag backfill](nychh-asset-tag-backfill.md) — same PHP-8.5 crash symptom, direct-DB workaround.
|
package/knowledge/INDEX.md
CHANGED
|
@@ -23,7 +23,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
23
23
|
## 2.0 framework
|
|
24
24
|
|
|
25
25
|
- **_underscore** (_Underscore) _(framework core)_ — 117 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
26
|
-
- **worker2** (Worker) —
|
|
26
|
+
- **worker2** (Worker) — 101 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
27
27
|
- **api2** (API) — 29 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
28
28
|
- **dbchanges2** (Database Changes) _(framework core)_ — 24 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
|
29
29
|
- **toga2-supply** (TOGa Supply) — 9 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
| Doc | Framework | Summary |
|
|
4
4
|
|-----|-----------|---------|
|
|
5
5
|
| [French (fr-CA) Item Feature Translations (Compass Canada)](features/french-item-feature-translations.md) | 2.0 | Renders item feature names/group headers/values in fr-CA on the Compass Canada storefront; open for Etilize feature translation, feature/group dedup, or the sid |
|
|
6
|
-
| [French (fr-CA) Order Email Localization (Compass Canada)](features/french-order-email-localization.md) | 2.0 | How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation)
|
|
6
|
+
| [French (fr-CA) Order Email Localization (Compass Canada)](features/french-order-email-localization.md) | 2.0 | How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation) and their HTML escaping; open for any localiz |
|
|
7
7
|
| [Grand & Toy ASN Import (Compass Canada)](features/grand-and-toy-asn-import.md) | 2.0 | Imports Grand & Toy ASNs for Compass Canada (CSV cron + cXML gateway) and auto-builds the ItemFulfillment chain + bilingual in-transit email; open for G&T ASN f |
|
|
8
8
|
| [Compass Canada — Item-Fulfillment TableViews (dead-Core-id 500, rebuilt onto the USA shape)](features/item-fulfillment-tableviews.md) | 2.0 | Why Compass Canada's item-fulfillment TableViews 500'd on dead Core ids and how they were rebuilt onto the USA shape; open when a Canada TableView 500s on /meta |
|
|
9
9
|
| [Compass Canada: incoming MITS purchase-order monitor (Monitor/Compass/CompassCanadaIncomingPurchaseOrders)](features/oneuptime-incoming-purchase-orders-monitor.md) | 2.0 | worker2 OneUptime push monitor for Compass Canada sales orders sent to MITS that have not got their full purchase order back, plus how MITS links a Canada PO to |
|
|
@@ -5,7 +5,7 @@ project: _Underscore
|
|
|
5
5
|
client: compass-canada
|
|
6
6
|
type: client-feature
|
|
7
7
|
status: active
|
|
8
|
-
updated: 2026-
|
|
8
|
+
updated: 2026-10-28
|
|
9
9
|
owners: ["bala"]
|
|
10
10
|
files:
|
|
11
11
|
- _underscore/Model/Compass/SalesOrder.php
|
|
@@ -13,7 +13,7 @@ files:
|
|
|
13
13
|
- _underscore/Component/Api/Togaiq/Togaiq.php
|
|
14
14
|
- _underscore/String.php
|
|
15
15
|
- library/app/client/compasscanada.php
|
|
16
|
-
-
|
|
16
|
+
- worker2/Worker/Client/Compass/DeliveredEmails.php
|
|
17
17
|
- worker/crons/toga2/compasscanada/compass_email_reminders.php
|
|
18
18
|
- worker/crons/toga2/compasscanada/compass_cancel_pending_approval_orders.php
|
|
19
19
|
- worker/crons/toga2/compasscanada/workflow/test_email_previews_prod.php
|
|
@@ -25,15 +25,16 @@ related:
|
|
|
25
25
|
- ../../../2.0/apps/_underscore/features/togaiq-gateway-client.md
|
|
26
26
|
- ../../../2.0/apps/_underscore/features/string-html-entity-helpers.md
|
|
27
27
|
- ../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md
|
|
28
|
+
- ../../compass-usa/features/delivered-emails.md
|
|
28
29
|
---
|
|
29
|
-
How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation)
|
|
30
|
+
How Compass Canada order emails render per-recipient EN/FR (item block, labels, denial-reason machine translation) and their HTML escaping; open for any localized-email work or encoding/mojibake bugs.
|
|
30
31
|
|
|
31
32
|
## Summary
|
|
32
33
|
Compass Canada order emails (order requested, manager-approval request, approved/rejected, in-transit, delivered, reminders) are sent in **each recipient's own language**. The French *wrapper* (subject + body template) was already selected correctly; what was broken was the **`{orderItems}` item block**, built from `Items.title`, which never consulted the `ItemTranslations` sidecar and had hardcoded English `P/N:` / `Qty:` labels. So a French user got a French email with English product names.
|
|
33
34
|
|
|
34
35
|
Two load-bearing points, easy to get wrong again:
|
|
35
36
|
1. **Language is per recipient, not per order.** One order's requester and manager can have different languages, so the item block must be rebuilt for each recipient.
|
|
36
|
-
2. **
|
|
37
|
+
2. **Dynamic French text must be HTML-entity-escaped with an exact flag set** (see Encoding), or the recipient sees a double-encoded entity or a blank product title.
|
|
37
38
|
|
|
38
39
|
Everything else in these emails has a pre-translated DB version (templates, `ItemTranslations`, label constants). **The one exception is the approver's free-text denial reason**, typed live in one language — machine-translated at send time through Talos.
|
|
39
40
|
|
|
@@ -80,11 +81,11 @@ Approvers in a bilingual org type in whichever language they think in, so a Fren
|
|
|
80
81
|
The sender does **one** language lookup and derives both the template UUID and the note language from it, so they can never disagree. **Compass USA is excluded outright** — `resolveRecipientLanguageCode()` returns `'en'` for every non-Canada client, so treating `'en'` as a translatable target would add a ~1.2s synchronous external call to **every US rejection** for zero benefit; the client gate prevents that. **No machine-translation disclosure label (deliberate):** an earlier build appended `(traduction automatique) Texte original: <english>` under the translated note; the developer removed it, so the reader sees only the translated text. **Accepted risk:** a wrong machine translation now reaches the reader with no original to compare — which is why the gateway's system prompt is hardened against fabrication.
|
|
81
82
|
|
|
82
83
|
### Encoding — why those exact `htmlentities` flags
|
|
83
|
-
|
|
84
|
+
Until 2026-10-28 these emails went out as **iso-8859-1**, so raw UTF-8 accents arrived as mojibake and French subjects showed raw entities (`Livrée`). The worker2 Send action now sends UTF-8 and decodes subject entities (TRUE-79683, see [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md)), so templates may hold entities or real UTF-8. Dynamic text is still entity-escaped, and both non-default arguments are required:
|
|
84
85
|
- **`double_encode = false`** — in some environments `ItemTranslations.title` holds *literal HTML entities as text*. Re-encoding those produced `&eacute;`, read as a literal `é`. (Prod and dev-sandbox data is clean raw-accent UTF-8; a stale local dataset had 71 of 215 rows entity-encoded. The flag makes the code correct under either data shape.)
|
|
85
86
|
- **`ENT_SUBSTITUTE`** — passing explicit flags **drops** PHP 8.1's default `ENT_SUBSTITUTE`, and `htmlentities()` returns an **empty string** on invalid UTF-8. Omitting it silently blanks the whole product title.
|
|
86
87
|
|
|
87
|
-
**`htmlentities()` alone is not enough** — it only converts characters with a **named** entity. French text carries the narrow no-break space **U+202F** before a colon, which has no name and survives as raw UTF-8
|
|
88
|
+
**`htmlentities()` alone is not enough** — it only converts characters with a **named** entity. French text carries the narrow no-break space **U+202F** before a colon, which has no name and survives as raw UTF-8 (it mojibaked under the old iso-8859-1 send). Use **`_String::encodeToAsciiHtmlEntities()`**, which runs the `htmlentities()` call above and then numbers every remaining character above U+007F. Applied to the rejection note, the item title and the part number. See [_String helpers](../../../2.0/apps/_underscore/features/string-html-entity-helpers.md).
|
|
88
89
|
|
|
89
90
|
### Resilience — a language lookup must never break the caller
|
|
90
91
|
`App_Query::execute()` (1.0, `library/app/query.php`) genuinely throws on any MySQL error, so this is a real failure mode. The guard is placed differently per tier, on purpose:
|
|
@@ -130,20 +131,22 @@ Tested through the **real private `sendRejectionEmailToUser()`** by reflection,
|
|
|
130
131
|
- **Do not `COALESCE` the sidecar title with the source title in SQL.** `COALESCE(NULLIF(ItemTranslations.title, ''), Items.title)` yields a `utf8mb4_bin` result (MySQL cannot reconcile the two input collations). `SELECT`ing it works, but **any** comparison, `ORDER BY`, `GROUP BY` or `DISTINCT` on that expression throws **MySQL 1267 "Illegal mix of collations"** — reproduced against prod. Worse, the collations **differ per client database** (`Client_CompassCanada.Items.title` = `utf8mb4_0900_ai_ci`, `Client_Compass.Items.title` = `utf8mb4_unicode_ci`, `ItemTranslations.title` = `utf8mb4_unicode_ci` in both), so a hardcoded `COLLATE` clause is brittle. Select the two columns separately and fall back in PHP — which is also what the shared translation layer does (`_Model::load()`, deliberately not a JOIN/COALESCE).
|
|
131
132
|
- **The `{role}` token appears TWICE in the French rejection template, with different grammar.** The second occurrence is `par un <strong>{role} TechHub</strong>`, which is already correct. So the missing elision must be fixed **in the template**, never by moving an article into the PHP role constant — that would break the second slot (`par un l'admin TechHub`). The template now reads `Refusée par : {role}`, correct for both `admin` and `gestionnaire`.
|
|
132
133
|
- **⚠ Do NOT "fix" the `approvalManagerId` subquery in `_Model_Compass_SalesOrder::postPut`.** It reads the step-2 `ApprovalDecision.decidedByUserId` (not `assignedToUserId`), which is `NULL` while the decision is pending — so `$managerEmail` stays empty and that arm never sends. Looks like a bug but is exactly what **prevents duplicate manager emails**: the canonical manager approval-request email is sent by `_Model_Compass_ApprovalDecision`, which correctly uses `ManagerDecision.assignedToUserId`. Switching the column would send a second manager email on every subsequent `SalesOrders` PUT. Evidence: prod `Logs_CompassCanada.Email` held 461 emails over ~3 weeks including 113 manager-approval emails, with **zero** duplicate `(to, subject)` pairs. Also on [Approval-Decision Flow](../../compass-usa/features/approval-decision-flow.md).
|
|
133
|
-
- **French
|
|
134
|
+
- **Check pasted French templates for a stray `Â` (U+00C2) before an NBSP.** That is a double-encoded paste (Word/Docs) and shows as `Â` under any charset. Eight Compass Canada French templates had it before the colon in `"Commande :"` (10 occurrences); replaced with ` ` in prod **and** dev-sandbox.
|
|
134
135
|
- **Compass US is provably unaffected.** `Client_Compass` has **0** `ItemTranslations` rows and only `en` in `Languages`, and every language branch is gated on `clientIdentifier === 'Compass_Canada'`, so the French path cannot alter a US email.
|
|
135
136
|
- **"No language setting" and "chose English" are different things — keep them different.** `resolveRecipientLanguageCode()` returned `'en'` for both, so there was no way to tell a user who picked English from one never asked. Use `resolveExplicitRecipientLanguageCode()` (returns `''` for "no row") anywhere the answer decides whether to spend a **paid, synchronous external call**. Keep the old method for template selection, where an English default is correct.
|
|
136
137
|
- **A translation must never be able to block an email.** Two independent guards: the note falls back to the original text on any failure, and the catch is on **`Throwable`** — a class missing from the deployed `_underscore` branch throws `Error`, not `Exception`. Not theoretical: the branch actually deployed to a tier is decided by the `ENVIRONMENT` variable, not the EB environment name — see [ENVIRONMENT drives the _underscore branch](../../../2.0/apps/api2/features/environment-variable-drives-underscore-branch.md).
|
|
137
138
|
- **The 1.0 Compass Canada crons have no escaping gap — verified, no change made.** Prod `Client_CompassCanada.ItemTranslations` holds **220** French titles and **zero** contain any character above U+00FF, so the nine `worker/crons/toga2/compasscanada` crons' plain `htmlentities()` is sufficient for the data that exists. Re-check if a future import introduces en dashes, curly quotes or narrow no-break spaces — then those crons need the same ASCII-safe encoder (which lives in 2.0 and would have to be duplicated in `library`).
|
|
138
139
|
|
|
139
140
|
## Change history
|
|
141
|
+
- 2026-10-28 — Mail now sent UTF-8 with subject entities decoded (French subjects no longer show `é`); delivered email moved to worker2 (TRUE-79683). (bala)
|
|
140
142
|
- 2026-09-25 — renamed product references from "TogaIQ" to "Talos" (rebrand; TOGa IQ is now the separate `toga25-iq` app) (jcardinal)
|
|
141
143
|
|
|
142
144
|
## Related
|
|
143
145
|
- [Compass Canada profile](../profile.md)
|
|
144
146
|
- [Compass Approval-Decision Flow](../../compass-usa/features/approval-decision-flow.md) — the shared parent that sends the approval/notification emails and resolves EN/FR templates.
|
|
145
147
|
- [Language Translation Layer](../../../2.0/apps/api2/features/language-translation-layer.md) — the `ItemTranslations` sidecar and its PHP-side fallback rule.
|
|
146
|
-
- [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md) — the queue + Send worker
|
|
148
|
+
- [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md) — the queue + Send worker and its UTF-8 / subject-decode rule.
|
|
149
|
+
- [Compass delivered email (worker2)](../../compass-usa/features/delivered-emails.md) — per-box delivered email for US + Canada; French labels in its `LABELS`.
|
|
147
150
|
- [Talos Gateway Client](../../../2.0/apps/_underscore/features/togaiq-gateway-client.md) — the AI gateway the denial-reason translation calls, and its anti-fabrication prompt.
|
|
148
151
|
- [_String helpers](../../../2.0/apps/_underscore/features/string-html-entity-helpers.md) — the ASCII-safe entity encoder these emails require.
|
|
149
152
|
- [Compass Partial In-Transit & Delivered Emails](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md) — the 1.0 crons whose item rows this localizes.
|
|
@@ -20,10 +20,11 @@ project: _Underscore
|
|
|
20
20
|
client: compass-canada
|
|
21
21
|
type: profile
|
|
22
22
|
status: active
|
|
23
|
-
updated: 2026-10-
|
|
23
|
+
updated: 2026-10-28
|
|
24
24
|
owners: [jcardinal, bala, tcox, apeterson, ajean, mhammontree, rgirish]
|
|
25
25
|
files: []
|
|
26
26
|
related:
|
|
27
|
+
- ../compass-usa/features/delivered-emails.md
|
|
27
28
|
- ../compass-usa/profile.md
|
|
28
29
|
- ../../2.0/apps/toga2-commerce/features/storefront-persona-gating-and-home-counts.md
|
|
29
30
|
- ../../2.0/apps/toga2-commerce/features/kit-item-price-display.md
|
|
@@ -195,9 +196,10 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
|
|
|
195
196
|
Compass USA's 1/3/4. See [Surface Resolver](../../2.0/apps/_underscore/features/surface-resolver.md).
|
|
196
197
|
- Customer language preference: `UserGlobalSettings.settingId = 2` (`en` / `fr-CA`); customer-
|
|
197
198
|
facing emails are sent in EN or FR accordingly — **per recipient**, including the `{orderItems}`
|
|
198
|
-
block,
|
|
199
|
-
|
|
200
|
-
|
|
199
|
+
block, with dynamic text entity-escaped. Mail is sent UTF-8 since 2026-10-28, so templates may
|
|
200
|
+
use entities or real UTF-8. See
|
|
201
|
+
[French Order Email Localization](features/french-order-email-localization.md). Delivered email:
|
|
202
|
+
[Compass delivered email (worker2)](../compass-usa/features/delivered-emails.md).
|
|
201
203
|
- **⚠ Open security hole (2026-10-09, not fixed):** `Client_CompassCanada/2026-01-15 - UserGlobalSettingsAcl.sql`
|
|
202
204
|
grants UserGlobalSettings writes with **no own-row check** — a user could write another user's row. See
|
|
203
205
|
[own-row guards](../../2.0/apps/api2/features/api-payload-interceptors.md#own-row-guards-in-a-pre-interceptor).
|
|
@@ -7,6 +7,7 @@
|
|
|
7
7
|
| [Compass Contact Email Resolution & Crossed `primaryContactEmailAddressId` Pointers](features/contact-email-resolution.md) | 2.0 | Any Compass code reading a contact's email must assert the email row belongs to that contact (~14% of pointers are crossed); open before reading Compass contact |
|
|
8
8
|
| [Corporate vs Field vs Foodbuy laptop personas (Levy/Compass x Corporate/Field + Foodbuy, PEOPLE cron)](features/corporate-vs-field-laptop-personas.md) | 2.0 | `PeopleFile` gives every US user one laptop-segment persona per half (Levy: Corporate/Field; Compass: Corporate/Field/Foodbuy) by cost center / sector `J00000`. |
|
|
9
9
|
| [Cost Centers — Unit Locations, numeric-only policy](features/cost-centers.md) | 2.0 | A Compass "cost center" is a Unit-level Location keyed by `c_erpSystemEntityId` (NOT a `CostCenters` row), numeric-only since 2026-07-06; open for cost-center d |
|
|
10
|
+
| [Compass USA + Canada per-package delivered email (worker2 DeliveredEmails, postPut queue + backup cron)](features/delivered-emails.md) | 2.0 | Compass USA + Canada "delivered" email, one per box (tracking number), sent by worker2 when a box turns DELIVERED; open for any delivered-email bug, resend, tem |
|
|
10
11
|
| [Compass isFulfillable — Data Quality & the Type-Derived Rule](features/isfulfillable-data-quality-and-type-rule.md) | 2.0 | Why Compass storefront Qty-Fulfilled cells stay dead — `Items.isFulfillable` NULLs are structurally unreachable, some set values are wrong, and the type-derived |
|
|
11
12
|
| [Compass Item Catalogs (Compass vs Agilant) & the Duplicate Items the NetSuite Sync Creates](features/item-catalogs-and-duplicate-items.md) | 1.0 | `Client_Compass.Items` is split across catalogs (Compass vs Agilant) and the NetSuite sync creates duplicate rows for one product; open first whenever a Compass |
|
|
12
13
|
| [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two config-only Compass TableViews (ids 12 & 13) showing fulfilled items in toga2-supply, re-rooted off Units and tracking sourced at item level; open when a fu |
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Compass USA + Canada per-package delivered email (worker2 DeliveredEmails, postPut queue + backup cron)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: worker2
|
|
5
|
+
project: Worker
|
|
6
|
+
client: compass-usa
|
|
7
|
+
type: client-feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-10-28
|
|
10
|
+
owners: ["bala"]
|
|
11
|
+
files:
|
|
12
|
+
- worker2/Worker/Client/Compass/DeliveredEmails.php
|
|
13
|
+
- _underscore/Model/Compass/TrackingNumber.php
|
|
14
|
+
- dbchanges2/Client_Compass/2026-10-19a - DeliveredEmails.sql
|
|
15
|
+
- dbchanges2/Client_CompassCanada/2026-10-19a - DeliveredEmails.sql
|
|
16
|
+
- dbchanges2/Core/2026-10-16a - CompassDeliveredEmailsCron.sql
|
|
17
|
+
- worker1.5/schedules/cron.worker.json
|
|
18
|
+
related:
|
|
19
|
+
- ../../../2.0/apps/worker2/features/tracking-status-refresh.md
|
|
20
|
+
- ../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md
|
|
21
|
+
- ../../compass-canada/features/french-order-email-localization.md
|
|
22
|
+
- ../../../2.0/apps/_underscore/features/email-send-pipeline.md
|
|
23
|
+
- ../../../2.0/apps/api2/features/api-payload-interceptors.md
|
|
24
|
+
- contact-email-resolution.md
|
|
25
|
+
- ../workflows/odp-asn-replay-backfill.md
|
|
26
|
+
---
|
|
27
|
+
Compass USA + Canada "delivered" email, one per box (tracking number), sent by worker2 when a box turns DELIVERED; open for any delivered-email bug, resend, template, link or tracking-button change.
|
|
28
|
+
|
|
29
|
+
## Summary
|
|
30
|
+
Replaced the 1.0 `send_delivered_email.php` crons (TRUE-79683, switched off in `worker1.5/schedules/cron.worker.json`). Those asked UPS themselves and skipped boxes worker2's [tracking-status refresh](../../../2.0/apps/worker2/features/tracking-status-refresh.md) had already marked DELIVERED, so most delivered emails were lost. Now the refresh's api2 PUT is the trigger.
|
|
31
|
+
|
|
32
|
+
## How it works
|
|
33
|
+
1. worker2 `Sync/TrackingNumbers/Refresh` PUTs `TrackingNumbers.status` through api2.
|
|
34
|
+
2. `_Model_Compass_TrackingNumber::postPut` (empty `Usa` / `Canada` subclasses; `ApiPayloadInterceptors` row record 62, POST-processing, PUT, in each client DB) **only queues** `_Worker::runTask('Client/Compass/DeliveredEmails/SendForTrackingNumber', {tenant: US|CA, trackingNumberUuid})` for each box the PUT set to DELIVERED. Status comes from the request body when the reply omits it.
|
|
35
|
+
3. `SendForTrackingNumber` reads the saved row, sends, stamps.
|
|
36
|
+
4. Backup: `Core.CronJobs` "Compass Delivered Emails", `Client/Compass/DeliveredEmails/SendPending`, `*/10 * * * *`, `{"dryRun": false}`, both tenants in one run, max 100 boxes per tenant.
|
|
37
|
+
5. **Claim before send:** both paths run `UPDATE TrackingNumbers SET c_dtDeliveredEmailSent = NOW() ... WHERE id = X AND c_dtDeliveredEmailSent IS NULL`; only the caller that changes the row sends. One email per box per order. A failed send throws and rolls the claim back, so the next run retries.
|
|
38
|
+
|
|
39
|
+
Candidate box: `status = DELIVERED`, linked to an order through the ASN unit chain, has an `ItemFulfillmentItems_TrackingNumbers` row, `c_dtDeliveredEmailSent IS NULL`. Cron window: `GREATEST(TrackingNumbers.dtUpdated, AdvanceShippingNotices.dtCreated)` within `LOOKBACK_DAYS` (14). The instant job has no window.
|
|
40
|
+
|
|
41
|
+
Orders per box: every buyer SO on the ASN's PO that has a line in this box (one email per SO). Recipient: newest active `Users` row with an email, else `Contacts.primaryContactEmailAddressId` with the ownership guard ([contact email resolution](contact-email-resolution.md)). CC from the order. `Notifications` row only when `userId > 0`.
|
|
42
|
+
|
|
43
|
+
Template: full when this box is the whole order, else partial (sections: this box, already delivered, in transit, being prepared). UUIDs in `TENANT_SETTINGS`; Canada picks EN/FR per recipient (`UserGlobalSettings.settingId = 2`).
|
|
44
|
+
|
|
45
|
+
### Template variables and links (designer review, 2026-10-28)
|
|
46
|
+
- `{userOrderUrl}` (all 6 templates, US + CA EN/FR, full + partial): order number links to `https://<commerceHost>/order-details?uuid=<SalesOrders.uuid>&type=orderHistory`. `commerceHost` per tenant: `compass.togacommerce.com` / `compasscanada.togacommerce.com`.
|
|
47
|
+
- `{trackingUrl}` (Track Shipment button), `buildTrackingUrl()` order:
|
|
48
|
+
1. `TrackingNumbers.trackingUrl` (supplier URL, e.g. Convey), when it is a safe `http(s)` URL.
|
|
49
|
+
2. `ShippingCarriers.urlLinkPrefix` + number, only when the prefix takes a number (ends in `=`, or a non-root path ending `/`).
|
|
50
|
+
3. A home-page prefix as is (the number is not added).
|
|
51
|
+
4. `_Model_Client_ShippingCarrier::generateTrackingNumberLink(number)` (by number pattern).
|
|
52
|
+
This is the reverse of the 1.0 in-transit cron, which puts the carrier prefix first.
|
|
53
|
+
- French labels (`LABELS`): part number `Réf :` in every delivered email (full and partial). Heading `Articles dans cette expédition`. The 2.0 order emails still use `N/P`.
|
|
54
|
+
|
|
55
|
+
### Manual actions (worker2 POST)
|
|
56
|
+
- Dry run: `{"action":"Client/Compass/DeliveredEmails/SendPending","parameters":{"dryRun":true}}`.
|
|
57
|
+
- Test one box to yourself (nothing stamped): add `"dryRun":false,"tenant":"CA","trackingNumber":"...","testRecipientEmail":"<you>@togatech.com"`. `trackingNumber` and `testRecipientEmail` must go together; test address must be `@togatech.com` / `@goagilant.com`.
|
|
58
|
+
- Resend by order: `Client/Compass/DeliveredEmails/SendForSalesOrders` `{tenant, salesOrderNumbers[1..50], dryRun}`. Uses the order's latest delivered box. Without `testRecipientEmail` the customer gets it again and the box is stamped.
|
|
59
|
+
|
|
60
|
+
## Gotchas
|
|
61
|
+
- **Never send inside the interceptor.** api2 commits the Logs DB even when the client DB rolls back (email for a PUT that never saved), it needs try/catch, and a bad interceptor breaks every tracking PUT. The postPut only queues.
|
|
62
|
+
- **Deploy `_underscore` before the interceptor row.** A row whose method does not exist breaks every tracking-number PUT.
|
|
63
|
+
- **The instant job can run before api2 commits.** It then finds no DELIVERED row and sends nothing. The cron sends it later.
|
|
64
|
+
- **The instant job never stamps a box with no order or recipient yet** (the ASN or fulfillment link can land after DELIVERED). The cron stamps it without an email, or it would hold a batch slot for 14 days.
|
|
65
|
+
- **Carriers the refresh cannot poll (freight) never reach DELIVERED**, so they never get this email.
|
|
66
|
+
- **Canada Precision and Grand & Toy prefixes are home pages only.** Their button opens the carrier site without the number (expected, step 3 above).
|
|
67
|
+
- **To stop emails during a bulk load, stamp `c_dtDeliveredEmailSent`** on the loaded tracking ids first. Both paths skip a stamped box. See [ODP ASN replay backfill](../workflows/odp-asn-replay-backfill.md).
|
|
68
|
+
|
|
69
|
+
## Change history
|
|
70
|
+
- 2026-10-28 — Designer review: `{userOrderUrl}` Commerce order link in all templates, supplier `trackingUrl` first for the button, French `Réf :` label (TRUE-79683). (bala)
|
|
71
|
+
- 2026-10-26 — Created: worker2 per-box delivered email, Compass postPut only queues, 10-minute cron backup, `c_dtDeliveredEmailSent` claim; 1.0 delivered crons off (TRUE-79683). (bala)
|
|
72
|
+
|
|
73
|
+
## Related
|
|
74
|
+
- [Tracking-status refresh](../../../2.0/apps/worker2/features/tracking-status-refresh.md) — the status writer that triggers this.
|
|
75
|
+
- [Compass in-transit emails (1.0)](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md)
|
|
76
|
+
- [Compass Canada French emails](../../compass-canada/features/french-order-email-localization.md)
|
|
77
|
+
- [2.0 Email Send Pipeline](../../../2.0/apps/_underscore/features/email-send-pipeline.md)
|
|
@@ -22,10 +22,11 @@ project: _Underscore
|
|
|
22
22
|
client: compass-usa
|
|
23
23
|
type: profile
|
|
24
24
|
status: active
|
|
25
|
-
updated: 2026-10-
|
|
25
|
+
updated: 2026-10-28
|
|
26
26
|
owners: [jcardinal, bala, tcox, apeterson, dfranks, akhokhani, ajean, rgirish, amadda]
|
|
27
27
|
files: []
|
|
28
28
|
related:
|
|
29
|
+
- features/delivered-emails.md
|
|
29
30
|
- features/talos-supply-assistant.md
|
|
30
31
|
- features/saml-sso.md
|
|
31
32
|
- ../../2.0/apps/_underscore/features/sales-order-po-number-sourcing.md
|
|
@@ -6,14 +6,14 @@ project: _Underscore
|
|
|
6
6
|
client: compass-usa
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-28
|
|
10
10
|
owners: ["bala"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Compass/SalesOrder.php
|
|
13
13
|
- _underscore/Model/Compass/AdvanceShippingNotice.php
|
|
14
14
|
- worker/crons/toga2/compass/workflow/3b_import_strategic_systems_advance_shipping_notices.php
|
|
15
15
|
- worker/crons/toga2/compass/update_salesorder_status_from_odp.php
|
|
16
|
-
-
|
|
16
|
+
- worker2/Worker/Client/Compass/DeliveredEmails.php
|
|
17
17
|
- worker/schedules/cron.worker.sync.json
|
|
18
18
|
related:
|
|
19
19
|
- ../features/asn-to-item-fulfillment.md
|
|
@@ -33,13 +33,13 @@ So the only correct way to mark an ODP order fulfilled is to **create an ASN**.
|
|
|
33
33
|
|
|
34
34
|
## Steps
|
|
35
35
|
1. **Repair item bridges first.** `postPost` creates an IF, and supply shows tracking, only when each ASN item maps to a SOI through `SalesOrderItems_PurchaseOrderItems`. Sweep the target POs for (a) PO lines with no bridge row and (b) PO lines linked to placeholder SOI id 1. Link or repoint by `VendorItems.itemId = SalesOrderItems.itemId` AND equal quantity. See the [bridge-gap gotcha](../features/mits-po-to-so-item-linking.md#gotchas).
|
|
36
|
-
2. **Turn off customer emails.** In
|
|
36
|
+
2. **Turn off customer emails.** In-transit: set `"active": 0` on "Update Compass SalesOrder Status From ODP CXML" (`update_salesorder_status_from_odp.php`) and deploy; a replayed ASN looks new to it (window is `AdvanceShippingNotices.dtCreated`, 7 days). Delivered (worker2 since TRUE-79683, see [delivered email](../features/delivered-emails.md)): it emails any ASN-linked `DELIVERED` box with `c_dtDeliveredEmailSent IS NULL`, and its window also counts `AdvanceShippingNotices.dtCreated`, so a replay looks new. Stamp `c_dtDeliveredEmailSent = NOW()` on every tracking id already in `TrackingNumbers` that you will load, before step 4.
|
|
37
37
|
3. **Build one ASN per ODP PO.** Every PO line at full PO-line quantity, tracking on the header and the last item (same shape as the 3b cron), **plus `advanceShippingNoticeItemUnits` carrying `advanceShippingNoticeItemUnitTrackingNumbers`** (the ODP cXML shape). Without units, supply's tracking column shows nothing. **Dedupe tracking numbers inside each ASN** (a repeat fails the whole POST with 1062).
|
|
38
38
|
4. **POST `/v2/advance-shipping-notices`** authenticated as `Client_Compass.Apis` id 4 "Office Depot (Cxml)". The secret lives in that table row; read it at run time, never copy it into a script or doc. 2026-10-13 tooling was one-off scripts outside any repo (`build_odp_tracking_asns.py`, `replay_compass_asns.sh`).
|
|
39
39
|
5. **Check every PO in the DB, not the curl output.** A curl `FAIL 000` (no response) saved nothing. Confirm the ASN row and its IF exist before resending, or you post a duplicate ASN.
|
|
40
40
|
6. **If ASNs went in without units, add them by SQL.** Insert tracking-only `AdvanceShippingNoticeItemUnits` (`unitId` NULL) and `AdvanceShippingNoticeItemUnits_TrackingNumbers` rows with literal uuid4 values (2026-10-13: 416 units, 647 bridge rows).
|
|
41
|
-
7. **Retire the loaded tracking numbers.** `UPDATE TrackingNumbers SET c_dtInTransitEmailSent = NOW(), status = 'DELIVERED'` scoped to `TrackingNumbers.id IN (<tracking ids on the loaded ASNs>)` only.
|
|
42
|
-
8. **Turn
|
|
41
|
+
7. **Retire the loaded tracking numbers.** `UPDATE TrackingNumbers SET c_dtInTransitEmailSent = NOW(), c_dtDeliveredEmailSent = NOW(), status = 'DELIVERED'` scoped to `TrackingNumbers.id IN (<tracking ids on the loaded ASNs>)` only. Without `c_dtDeliveredEmailSent` the worker2 delivered cron emails every one of them.
|
|
42
|
+
8. **Turn the in-transit cron back on** (`"active": 1`) and deploy.
|
|
43
43
|
|
|
44
44
|
## Gotchas
|
|
45
45
|
- **⚠ Never scope step 7 by "tracking created today".** That also catches live ODP ASNs that arrived the same day and silences real customer emails. The suppression SQL printed by `replay_compass_asns.sh` is this broad version; do not run it as is.
|
|
@@ -49,6 +49,7 @@ So the only correct way to mark an ODP order fulfilled is to **create an ASN**.
|
|
|
49
49
|
- Step 7 sets `status = 'DELIVERED'` without asking the carrier. Only safe for old shipments; never use it on a recent batch.
|
|
50
50
|
|
|
51
51
|
## Change history
|
|
52
|
+
- 2026-10-28 — Delivered email is now worker2 (TRUE-79683): suppress it by stamping `c_dtDeliveredEmailSent`, not by a 1.0 cron switch. (bala)
|
|
52
53
|
- 2026-10-13 — Created from the first ODP historical tracking backfill (279 orders to Fulfilled). (bala)
|
|
53
54
|
|
|
54
55
|
## Related
|
|
@@ -6,8 +6,8 @@ project: _Underscore
|
|
|
6
6
|
client: rate
|
|
7
7
|
type: client-feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: [mhammontree]
|
|
9
|
+
updated: 2026-10-28
|
|
10
|
+
owners: [mhammontree, bala]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Rate/Entitlement.php
|
|
13
13
|
- worker2/Worker/Notification/EmailTemplate.php
|
|
@@ -69,17 +69,19 @@ Figma fonts (TOGa View 2.0 email designs):
|
|
|
69
69
|
- **AIG Terms & Conditions URL** — `WARRANTY_TERMS_URL` set **provisionally** to `https://hometechsupport.rate.togaview.com/terms/home-warranty`. **Provisional** — the PM expects it to change before/after go-live.
|
|
70
70
|
|
|
71
71
|
Remaining items are **optional / separate, not go-live blockers**:
|
|
72
|
-
- The `_Email.php` UTF-8 `CharSet` framework fix (root-cause of the ASCII-only-subjects workaround below — see [`email-template-sending.md`](../../../2.0/apps/_underscore/features/email-template-sending.md)).
|
|
73
72
|
- An Axiforma web font for the footer "POWERED BY" label (currently rendered as the logo image).
|
|
74
73
|
|
|
75
74
|
## Gotchas
|
|
76
75
|
- **Shared interceptor with TRUE-79533 (WH per-address guard) — same `prePost`/`postPost`.** This workflow's `postPost` trigger + helpers live in the **same** `_Model_Rate_Entitlement` interceptors that the [Whole Home Warranty per-address guard](whole-home-warranty-purchase-guard.md) edits, and TRUE-79251 also adds the passThrough `const PASSTHROUGH_KEY_FULFILLMENT_PREFERENCE` + the contractFulfillmentPreference (EV-8) fix inside `prePost`. A bad `_production`→`_beta` merge already mis-resolved this once and fatally broke every Rate purchase save. When merging either ticket across branches, hand-merge into ONE `prePost()` keeping both features' logic — see the **shared-interceptor merge hazard** gotcha on that doc for the correct resolution and why `php -l` is not a sufficient gate.
|
|
77
76
|
- **`border-radius` on a `<td>` renders SQUARE in many clients (Outlook).** Use an `<img>` for circular badges/icons (e.g. the green circle-check) instead of a CSS-rounded table cell.
|
|
78
|
-
- **ASCII-only subjects
|
|
77
|
+
- **ASCII-only subjects are no longer required.** They were a mojibake workaround; since 2026-10-28 (TRUE-79683) the worker2 Send action sends UTF-8. See [`email-send-pipeline.md`](../../../2.0/apps/_underscore/features/email-send-pipeline.md).
|
|
79
78
|
- **⚠ A non-prod tenant missing its `EmailTemplates` seed looks like a code error, not a data gap.** It surfaces as a FAILED `Notification/EmailTemplate/Send` worker job naming the template **uuid**: *"Could not load `_Model_Client_EmailTemplate` object based upon these criteria: `uuid` = '2c4f8a1e-…'"* (dev-sandbox `Core.WorkerJobs` 141941, 2026-09-18). dev-sandbox `Client_Rate.EmailTemplates` held only 2 rows (Forgot Password, Talos); prod has both Rate templates as ids 3 and 4. Fix: run the existing `dbchanges2/Client_Rate/2026-06-30a - Rate purchase email templates.sql` on that environment. Check the tenant's `EmailTemplates` rows before debugging the sender.
|
|
80
79
|
- **⚠ OPEN — both templates render badly in email dark mode (from TRUE-79251; needs a design decision, not a code fix).** (1) The CTA button becomes an **empty dark bar** — background `#21292d` disappears into the client's dark theme and the white label is invisible. (2) The TOGA footer wordmark shows inside a **dark block**, because the image was rasterized onto the exact footer colour `#21292d`, which the client lightens in dark mode. Light mode is fine, which is why neither was caught — the *"flatten the asset onto the exact region background"* recipe above is precisely what breaks under dark mode.
|
|
81
80
|
- **Worker DB registration** — see [`notification-email-template.md`](../../../2.0/apps/worker2/features/notification-email-template.md): worker actions do not auto-register `DB_CLIENT`.
|
|
82
81
|
|
|
82
|
+
## Change history
|
|
83
|
+
- 2026-10-28 — ASCII-only subject workaround retired: mail now sent UTF-8 (TRUE-79683). (bala)
|
|
84
|
+
|
|
83
85
|
## Related
|
|
84
86
|
- [Rate profile](clients/rate/profile.md)
|
|
85
87
|
- [AIG contract creation](clients/rate/features/aig-contract-creation.md)
|
package/package.json
CHANGED