toga-ai 1.0.792 → 1.0.794

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-09-04
9
+ updated: 2026-09-10
10
10
  owners: ["bala"]
11
11
  files:
12
12
  - worker/crons/toga2/compass/update_salesorder_status_from_odp.php
@@ -209,6 +209,14 @@ the tracking number as emailed (it retries next run).
209
209
  [Compass Canada French order emails](../../../clients/compass-canada/features/french-order-email-localization.md).
210
210
 
211
211
  ## Gotchas / known issues
212
+ - **An ASN with zero line items is invisible to this cron — no partial in-transit and no delivered
213
+ email, and no error.** Candidates are selected through `AdvanceShippingNoticeItemUnits` +
214
+ `AdvanceShippingNoticeItemUnits_TrackingNumbers`, so an ASN whose `AdvanceShippingNoticeItems` is
215
+ empty can never reach it. Compass Canada's G&T importer produced exactly that class of ASN until
216
+ 2026-09-08 (validation guard sat inside the serial-number branch) — the shipment looked real and
217
+ the customer was never told. When a Compass order is "shipped but no email", check for an empty
218
+ ASN before looking at this cron. See
219
+ [G&T ASN Import](../../../clients/compass-canada/features/grand-and-toy-asn-import.md).
212
220
 
213
221
  - **⚠ A cron filename containing `&` cannot run at all.** `worker/ebs/cron.worker.php:41` builds
214
222
  the crontab command by plain unquoted concatenation
@@ -284,6 +292,9 @@ the tracking number as emailed (it retries next run).
284
292
  interceptor and retire this 1.0 cron.
285
293
 
286
294
  ## Change history
295
+ - 2026-09-10 — Recorded that a zero-line-item ASN is invisible to this cron (candidates come from
296
+ `AdvanceShippingNoticeItemUnits(_TrackingNumbers)`), the failure mode Compass Canada's G&T
297
+ importer produced until 2026-09-08. (bala)
287
298
 
288
299
  - 2026-09-04 — **Fixed the Compass USA in-transit email outage (2026-08-27 → 2026-09-03, 130–190
289
300
  emails/day → near zero).** Root cause: already-delivered rows were `continue`d without stamping
@@ -5,7 +5,7 @@
5
5
  | [TOGa Commerce (toga2-commerce / commerce2-react) Architecture](architecture.md) | `toga2-commerce` (npm package name **`commerce2-react`**, product name **TOGa Commerce**) is the customer-facing **B2B commerce storefront** of the 2.0 platform |
6
6
  | [Bundle item visibility & selectability (the three flags, and why they are enforced nowhere but the client)](features/bundle-item-visibility-and-selectability.md) | Which components of a kit a shopper can see and choose is decided **entirely in the browser**. |
7
7
  | [Cart Bundle Submission & the bundleUuid Identity Contract](features/cart-bundle-submission-and-identity.md) | How cart **bundles** (kits) are turned into `SalesOrderItems` when a cart is submitted or an existing order is edited, and the **identity-field contract** every |
8
- | [Cart Notification Emails — duplicate prevention](features/cart-notification-emails.md) | On the cart "Notifications" section a user can add CC email addresses to an order. |
8
+ | [Cart Notification Emails — auto-add/remove lifecycle + duplicate prevention](features/cart-notification-emails.md) | On the cart "Notifications" section a user can add CC email addresses to an order. |
9
9
  | [Cart Order-Total & Shipping Computation](features/cart-order-total-computation.md) | The Cart summary section (subtotal / shipping / tax / total) is **data-driven** from `cartData`. |
10
10
  | [Cart Page — config-driven form architecture (current state + planned refactor)](features/cart-page-config-architecture.md) | The Cart page (`src/pages/Cart/`) is the most config-heavy page in `toga2-commerce`. |
11
11
  | [Catalog cache freshness — the 24h persisted query cache, and how to opt a query out of it](features/catalog-cache-freshness.md) | TOGa Commerce runs a **single `QueryClient` with a 24-hour default `staleTime`**, and persists it to **`localStorage["commerce"]`** through `PersistQueryClientP |
@@ -1,16 +1,18 @@
1
1
  ---
2
- title: Cart Notification Emails — duplicate prevention
2
+ title: Cart Notification Emails — auto-add/remove lifecycle + duplicate prevention
3
3
  framework: "2.0"
4
4
  repo: toga2-commerce
5
5
  project: TOGa Commerce
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-07-27
9
+ updated: 2026-09-10
10
10
  owners: ["bala", "tcox"]
11
11
  files:
12
12
  - src/pages/Cart/CartPage.tsx
13
13
  - src/pages/Cart/view/cartForm/CartForm.tsx
14
+ - src/pages/Cart/view/cartForm/EditCart.tsx
15
+ - src/pages/Cart/view/cartForm/EditOrder.tsx
14
16
  - src/stores/useEmailOptionsStore.ts
15
17
  - src/stores/useCartSalesQuoteZu.ts
16
18
  - src/pages/Cart/viewModel/FIELDS/*/*/*/CARTPAGE.ts
@@ -21,12 +23,19 @@ related:
21
23
  ---
22
24
 
23
25
  ## Summary
24
- On the cart "Notifications" section a user can add CC email addresses to an order. Adding the
25
- same address twice (often with different casing) used to slip through to the API and blow up on
26
- submit with a MySQL 1062 duplicate-key error on the `SalesOrderEmailAddresses.salesOrderId_emailAddress`
27
- unique index (whose collation `utf8mb4_0900_ai_ci` is case-insensitive). This feature blocks
28
- duplicates case-insensitively in the frontend and shows a sapphire info message instead of letting
29
- the error reach the user.
26
+ On the cart "Notifications" section a user can add CC email addresses to an order. Two jobs live
27
+ here:
28
+
29
+ 1. **Duplicate prevention.** Adding the same address twice (often with different casing) used to
30
+ slip through to the API and blow up on submit with a MySQL 1062 duplicate-key error on the
31
+ `SalesOrderEmailAddresses.salesOrderId_emailAddress` unique index (collation
32
+ `utf8mb4_0900_ai_ci` is case-insensitive). The frontend blocks duplicates case-insensitively and
33
+ shows a sapphire info message instead of letting the error reach the user.
34
+ 2. **Auto-add/remove lifecycle.** Picking an **order-for user** or a **Delegate Manager**
35
+ auto-adds their notification emails. **Changing the pick must remove the previous person's
36
+ auto-added emails.** Until 2026-09-10 it did not: the old person and their manager stayed in the
37
+ list, still checked, and still in the submit payload — so notifications went to the wrong
38
+ people unless the buyer noticed and unchecked them.
30
39
 
31
40
  ## Key files / entry points
32
41
  - `CartPage.tsx` — `handleAddEmail`: validates the email regex, then on success calls
@@ -35,10 +44,23 @@ the error reach the user.
35
44
  - `CartForm.tsx` — owns the duplicate UX. `handleAddEmailWithDuplicateCheck` wraps the passed-in
36
45
  `handleAddEmail`: it case-insensitively checks the existing `emails` list and, on a match, shows
37
46
  the message instead of adding. The **Add Email** button calls this wrapper.
38
- - `useEmailOptionsStore.ts` — `addEmailOption` de-dupes the checkbox list.
39
- - `useCartSalesQuoteZu.ts` `addEmail` de-dupes the actual `salesOrderEmailAddresses` payload.
47
+ - `CartPage.tsx` — also owns the **Delegate Manager** lifecycle:
48
+ `advancedSelectConfig.delegateManager.onSelect` / `.onClearInput`, the
49
+ `lastDelegateManagerEmailRef` (useRef), the `removeDelegateManagerEmail` guard, the
50
+ `watch("hasDelegateManager")` and `watch("delegateManager")` effects, and `handleClearUser`.
51
+ - `EditCart.tsx` — cart-checkout mode. The order-for change effect (`onSuccess` of the user fetch)
52
+ owns add **and** remove of the order-for user's emails, plus the local `ensureEmailSelected`
53
+ helper and the `orderForUserChanged` flag.
54
+ - `EditOrder.tsx` — edit-order mode. Still has the **original add-only** order-for pattern.
55
+ - `useEmailOptionsStore.ts` — `addEmailOption` de-dupes the checkbox list; `removeEmailOption`
56
+ removes case-insensitively; `selectEmailOption(email)` marks an option checked (idempotent —
57
+ never toggles).
58
+ - `useCartSalesQuoteZu.ts` — `addEmail` de-dupes and `removeEmail` removes from the actual
59
+ `salesOrderEmailAddresses` payload, both case-insensitively.
40
60
 
41
61
  ## How it works
62
+
63
+ ### Duplicate prevention
42
64
  1. **Before (manual add):** clicking Add Email runs `handleAddEmailWithDuplicateCheck` in
43
65
  `CartForm`. It compares `(e.email ?? "").trim().toLowerCase()` against the normalized input. On a
44
66
  match it sets `duplicateEmailMessage` and returns (does not add).
@@ -53,6 +75,41 @@ the error reach the user.
53
75
  5. A `useEffect` clears the message when the input, selected user (`orderForUser?.uuid`), or email
54
76
  list (`emails.length`) changes.
55
77
 
78
+ ### Order-for user — add AND remove (`EditCart.tsx`, cart-checkout mode only)
79
+ Auto-added set per user: **their email**, `supervisorUser.email`, and
80
+ `c_supportedByUserId.email`.
81
+
82
+ 1. On success of the order-for user fetch, read the **previous** user from
83
+ `useSelectedUserZu.getState().selectedUser` — the store, **not** the render closure (the closure
84
+ is stale inside the query callback).
85
+ 2. If the previous uuid differs from the new one (`orderForUserChanged`), remove that person's three
86
+ auto-added addresses from **both** `useEmailOptionsStore` and
87
+ `useCartSalesQuoteZu.salesOrder.salesOrderEmailAddresses`. **Never remove the logged-in user's
88
+ own address** even if it matches one of the three.
89
+ 3. Re-seed the new user's addresses through the local **`ensureEmailSelected(email)`** helper:
90
+ `addEmailOption` → `selectEmailOption` → `addEmail`. Idempotent, so re-running it is safe.
91
+ 4. The seeding block runs when `salesOrderEmailAddresses` is empty **or** when
92
+ `orderForUserChanged`. Before the fix it ran only on empty, so the second pick's emails were
93
+ added **un-checked** and never reached the payload.
94
+
95
+ ### Delegate Manager email lifecycle (`CartPage.tsx`, cart-checkout mode)
96
+ "Associate" in the Compass UI = Delegate Manager. Its email is tracked by
97
+ **`lastDelegateManagerEmailRef`** (a `useRef`), because the form value cannot be used (see
98
+ Gotchas). The email is parsed out of the option label, which has the shape
99
+ `"First Last (email)"`.
100
+
101
+ | Action | What happens |
102
+ |---|---|
103
+ | Pick a manager | `onSelect` removes the *previous* ref email if different, adds the new one checked, sets the ref |
104
+ | Click the field's **X** | `onClearInput` removes the ref email and clears the ref |
105
+ | Un-check "has delegate manager" | the `watch("hasDelegateManager")` effect removes the ref email and empties the field |
106
+ | Clear the order-for user | `handleClearUser` also un-checks the box, empties the field, and clears the ref |
107
+ | Restore a saved cart | the `watch("delegateManager")` effect seeds the ref; a restored name has no `"(email)"`, so it falls back to `salesOrder.approvalUser.email` |
108
+
109
+ **`removeDelegateManagerEmail` never removes an address that is also** the order-for user's, their
110
+ supervisor's, their support tech's, or the logged-in user's — those are owned by the order-for
111
+ lifecycle above.
112
+
56
113
  ## Data model
57
114
  Frontend only. The payload list maps to the `SalesOrderEmailAddresses` table (api2/backend), which
58
115
  has a case-insensitive unique index on `(salesOrderId, emailAddress)`.
@@ -67,19 +124,57 @@ the message resolves to `undefined` and renders blank.
67
124
  - The real source of the 1062 was `useCartSalesQuoteZu.addEmail` comparing with `===`
68
125
  (case-sensitive). Both stores must compare case-insensitively; fixing only the UI list is not
69
126
  enough because `addEmail` builds the payload.
127
+ - **Removes must be case-insensitive too, not just adds.** Both stores' *adds* were already
128
+ case-insensitive while their *removes* used exact match, so a hand-typed `Bob@X.com` survived a
129
+ remove of `bob@x.com` and shipped in the payload. `removeEmailOption` and `removeEmail` are now
130
+ case-insensitive.
131
+ - **`handleEmailChange` TOGGLES — never use it to seed.** Re-running it on an already-checked
132
+ address **un-checks** it. Use the idempotent `ensureEmailSelected` /
133
+ `selectEmailOption` path for any auto-seeding.
134
+ - **`CartFormSection` overwrites the form value BEFORE your handler runs.** It calls
135
+ `field.onChange(value)` and *then* `config.onSelect(value, valueKey)` (same for
136
+ `field.onChange(null)` before `config.onClearInput`). So
137
+ `formMethods.getValues("delegateManager")` inside those handlers already holds the **new** value
138
+ — you **cannot** read the previous pick from the form. Keep the previous value in a `useRef`.
139
+ - **Read the previous order-for user from the store, not the closure.** Inside the user-fetch
140
+ `onSuccess` the render closure's `selectedUser` is stale; use
141
+ `useSelectedUserZu.getState().selectedUser`.
142
+ - **`delegateManager` defaults to a truthy object** (`{uuid:"",name:""}`) now that it is a typed
143
+ form field, so `if (formValues.delegateManager)` is always true. Check
144
+ `formValues.delegateManager?.uuid` instead (3 call sites: `EditCart.tsx` ×2, `EditOrder.tsx` ×1).
145
+ - **`EditOrder.tsx` (edit-order mode) still has the add-only order-for pattern** — the same
146
+ wrong-recipient bug very likely exists there. Left alone deliberately: edit-order has different
147
+ manager rules (`assignedTo`). Fix it as its own task.
70
148
  - Normalize with `(x ?? "").trim().toLowerCase()`, not `x?.trim().toLowerCase()` — both are
71
149
  nullish-safe (optional chaining short-circuits the whole chain, it does not throw), but `(x ?? "")`
72
150
  guarantees a string and avoids an `undefined === undefined` match edge.
73
151
  - Adding the label to only some `CARTPAGE.ts` files leaves other client/role/language users with a
74
152
  blank message.
75
153
  - Do not use a toaster or `setError` (red) for this — product wants the sapphire info style.
76
- - **Now pinned by e2e.** `cypress/e2e/cartPage/cartV2.cy.ts` covers both the auto-population
77
- (order-for user + supervisor emails, protected rows unremovable) and the case-insensitive
78
- duplicate block surfacing the *"This email has already been added"* banner. These are exactly
79
- the parity behaviors the config-cart refactor spike silently lost, so keep the spec green when
80
- touching this flow (see [cypress-testing](../workflows/cypress-testing.md)).
154
+ - **e2e coverage is branch-dependent check before you rely on it.** On `_production`,
155
+ `cypress/e2e/cartPage/cartV2.cy.ts` covers both the auto-population (order-for user + supervisor
156
+ emails, protected rows unremovable) and the case-insensitive duplicate block surfacing the
157
+ *"This email has already been added"* banner the exact parity behaviors the config-cart
158
+ refactor spike silently lost, so keep it green (see
159
+ [cypress-testing](../workflows/cypress-testing.md)). But on `#sprint86` that file **does not
160
+ exist** and every spec in `cypress/e2e/cartPage/cart.cy.ts` is **commented out**, so there is no
161
+ net for this flow there. Verify manually on such a branch.
162
+ - **ESLint cannot run in this repo checkout** (pre-existing): `.eslintrc.cjs` references
163
+ `eslint-plugin-react-compiler`, which is not installed. Type-check with
164
+ `npx tsc --noEmit -p tsconfig.app.json` instead.
81
165
 
82
166
  ## Change history
167
+ - 2026-09-10 — FIXED: changing the order-for user or the Delegate Manager left the previous
168
+ person's auto-added emails checked and in the submit payload, so notifications went to the wrong
169
+ people (reported by a Compass USA user on `compass.togacommerce.com`; fix is client-neutral).
170
+ Order-for path (`EditCart.tsx`) now removes the prior user's three auto-added addresses and
171
+ re-seeds via the idempotent `ensureEmailSelected` (replacing the toggling `handleEmailChange`),
172
+ and seeds on `orderForUserChanged` as well as on empty. Delegate Manager path (`CartPage.tsx`)
173
+ gained `lastDelegateManagerEmailRef` + `removeDelegateManagerEmail` covering pick / re-pick / X /
174
+ un-check / clear-user / restore. Stores: added `selectEmailOption`, made `removeEmailOption` and
175
+ `removeEmail` case-insensitive. `hasDelegateManager` / `delegateManager` are now typed form fields
176
+ with defaults, so delegate checks moved to `?.uuid`. Edit-order mode not fixed (different manager
177
+ rules). (tcox)
83
178
  - 2026-07-27 — No behavior change. This duplicate-prevention UX (email auto-population +
84
179
  case-insensitive duplicate banner) is now covered by cart e2e slice 1 (`cartV2.cy.ts`); linked
85
180
  the [cypress-testing](../workflows/cypress-testing.md) workflow doc. (tcox)
@@ -6,7 +6,7 @@ project: TOGa Commerce
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-07-27
9
+ updated: 2026-09-10
10
10
  owners: ["apeterson", "tcox"]
11
11
  files:
12
12
  - src/pages/Cart/CartPage.tsx
@@ -85,6 +85,17 @@ based on `inEditMode`. `CartFormRenderer.tsx` exists only on the `implement-conf
85
85
  Two mode-change `reset()` effects (~lines 267, 282) re-seed from that same union shape. This is
86
86
  wrong for per-client field sets: it registers fields a client never renders and bakes one client's
87
87
  default into all of them, and risks RHF controlled/uncontrolled warnings.
88
+ As of 2026-09-10 `hasDelegateManager` (`false`) and `delegateManager` (`{uuid:"",name:""}`) are
89
+ part of that union and are typed in `CartFormFieldNames` — previously they were undefined and
90
+ untyped. Because the default object is **truthy**, any check on the field must test
91
+ `delegateManager?.uuid`, not the object.
92
+ - **`CartFormSection` runs `field.onChange` BEFORE the config handler.** For an `advancedSelect` it
93
+ calls `field.onChange(value)` then `config?.onSelect(value, valueKey)` (and
94
+ `field.onChange(null)` then `config?.onClearInput(valueKey)`). So a config handler can never read
95
+ the field's **previous** value from the form — it is already overwritten. Handlers that need the
96
+ prior pick must keep it in a `useRef` (see
97
+ [cart-notification-emails](cart-notification-emails.md) for the Delegate Manager case). The
98
+ planned `hydrateCartConfig` (Phase 5) must preserve this ordering contract or document a change.
88
99
  - **Client differences are hardcoded, not config.** Examples observed: COMPASS renders
89
100
  `DuplicateKitGuardrail`, QUAD does not; COMPASS seeds the cart form (cost center / manager / emails)
90
101
  from the order-for user, QUAD does not. These are per-client conditionals in cart code today.
@@ -285,6 +296,11 @@ extension** of toga2.5's philosophy, not a literal copy.
285
296
  edit-order mode?
286
297
 
287
298
  ## Change history
299
+ - 2026-09-10 — Two gotchas added from cart-notification work: `hasDelegateManager` /
300
+ `delegateManager` are now typed form fields in the union `defaultValues` (truthy default → check
301
+ `?.uuid`), and documented that `CartFormSection` calls `field.onChange` **before** the
302
+ `advancedSelect` config's `onSelect`/`onClearInput`, so a handler cannot read the field's previous
303
+ value. No refactor progress. (tcox)
288
304
  - 2026-07-27 — Phase 0 oracle is now partly real: cart e2e slice 1 (`cartV2.cy.ts`, 12 tests)
289
305
  landed and pins the exact parity behaviors the prior spike lost (notification-email
290
306
  auto-population, duplicate-email surfacing) plus render/gating/clear/empty flows. Corrected the
@@ -5,11 +5,12 @@ project: _Underscore
5
5
  client: compass-canada
6
6
  type: client-feature
7
7
  status: active
8
- updated: 2026-08-20
8
+ updated: 2026-09-10
9
9
  owners: ["bala"]
10
10
  files:
11
11
  - api2/Component/Api/Cxml/Cxml.php
12
12
  - worker/crons/toga2/compasscanada/workflow/3_import_grand_and_toy_advance_shipping_notices.php
13
+ - worker/crons/toga2/compasscanada/workflow/check_grand_and_toy_inbox.php
13
14
  - worker/crons/toga2/compasscanada/workflow/import_grand_and_toy_asn_from_file.php
14
15
  - worker/crons/toga2/compasscanada/workflow_beta/import_grand_and_toy_asn_from_file.php
15
16
  - worker/schedules/cron.worker.sync.json
@@ -21,6 +22,7 @@ related:
21
22
  - ../workflows/grand-and-toy-asn-backfill.md
22
23
  - ../../../2.0/apps/api2/features/cxml-shipnotice-gateway.md
23
24
  - ../../compass-usa/features/asn-to-item-fulfillment.md
25
+ - ../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md
24
26
  ---
25
27
 
26
28
  ## Summary
@@ -30,10 +32,19 @@ G&T's ASN CSV from a mailbox, posts each shipped line to the 2.0 API as an Advan
30
32
  an in-transit email in English or French. It mirrors the Compass USA ODP / Strategic-Systems ASN
31
33
  flow, adapted for Canadian carriers and bilingual email.
32
34
 
33
- **The live path today is cXML, not the CSV cron.** G&T transmits ship notices to the api2 cXML
34
- gateway (`cxml-writer`), which translates them into the same `POST /v2/advance-shipping-notices`.
35
- The 24-column CSV cron below is **retired** it remains documented because it is the only source of
36
- serial numbers and the reference for the field mapping.
35
+ **Both transports are live the CSV cron is NOT retired (corrected 2026-09-10).** G&T transmits
36
+ ship notices to the api2 cXML gateway (`cxml-writer`), which translates them into the same
37
+ `POST /v2/advance-shipping-notices`. **In parallel**, G&T emails the 24-column CSV and the 1.0
38
+ worker cron `3_import_grand_and_toy_advance_shipping_notices.php` still runs on schedule
39
+ (`0 */4 * * *`, six runs a day) and was fixed and deployed to production on **2026-09-08**. The CSV
40
+ cron matters for two reasons beyond the field mapping: it is the **only source of serial numbers**,
41
+ and it is the **only thing that sends the in-transit email** (api2 does not — see *In-transit email
42
+ is owned by the cron*).
43
+
44
+ > **The importer was renamed** `4_import_grand_and_toy_advance_shipping_notices.php` →
45
+ > `3_import_grand_and_toy_advance_shipping_notices.php` (commit `d38d0c33`) and
46
+ > `worker/schedules/cron.worker.sync.json` was repointed. Any older note referring to the `4_` name
47
+ > is stale.
37
48
 
38
49
  > ### Outage 2026-03 to 2026-08-20 — two stacked defects, 518 shipments lost
39
50
  >
@@ -86,7 +97,10 @@ serial numbers and the reference for the field mapping.
86
97
  3. **Before the row loop** — build three lookup maps once per file: `$purchaseOrderUuidByNumber`
87
98
  (`SELECT number, uuid FROM PurchaseOrders`), `$shippingMethodByCarrierId`, `$carrierIdByName`.
88
99
  4. Per row: validate PO / part / tracking (early `continue` with error recording on missing
89
- critical fields); look up SO + contact user + CC addresses; resolve carrier
100
+ critical fields). **The item lookup (`PurchaseOrderItems` `VendorItems.vendorPartNumber`
101
+ `Items.id`, PO-scoped) runs for EVERY row, before anything is written** — it is not inside the
102
+ serial-number branch (see the 2026-09-08 zero-line-item gotcha). Then look up SO + contact user
103
+ + CC addresses; resolve carrier →
90
104
  Ground shipping method (falls back to `DEFAULT_GROUND_SHIPPING_METHOD_ID = 3`); upsert
91
105
  `TrackingNumbers`; resolve item by `Items.partNumber` joined through `VendorItems`; find-or-create
92
106
  a `Units` row per serial.
@@ -113,6 +127,67 @@ serial numbers and the reference for the field mapping.
113
127
  - Interceptor wiring: `Core.RecordScripts` + `Core.ApiPayloadInterceptors`, `recordId 55`
114
128
  (AdvanceShippingNotice). The `sendEmail` POST record script is `Core.RecordScripts recordId 202`.
115
129
 
130
+ ## CSV file shape — what silently rejects a whole file
131
+
132
+ The importer's only file-level gate is the **column count of the first line, which must be exactly
133
+ 24**. It does **not** check the file name, the extension, or the header names — every field is read
134
+ **by position**. Consequences seen live (three real files rejected in a row, 2026-09-03 → 2026-09-08):
135
+
136
+ - An Apple **`.numbers`** file is a zip (first bytes `PK`), so line 1 parses as **1 column** → whole
137
+ file rejected.
138
+ - A CSV **exported from Numbers** carries the sheet-name **title row on line 1**, which is also
139
+ **1 column** → whole file rejected, even though the real 24-column header sits on line 2.
140
+ - Rejection sends the *"CompassCanada G&T ASN: Unexpected column count, file not processed"* email to
141
+ `vburks@togatech.com` + `devTeam@togatech.com` and the file is skipped. Nothing is imported.
142
+
143
+ **The cron drains the mailbox on every run.** Each message is moved out of INBOX after processing —
144
+ to **ARCHIVE** when it carried an attachment, to **Junk Email** when it did not. So a file that
145
+ already ran is never in INBOX; look in ARCHIVE. Schedule is `0 */4 * * *` in
146
+ `worker/schedules/cron.worker.sync.json` (six runs a day, on the hour), and the worker server clock
147
+ is **America/Chicago** (set by `App_Framework::cronInitialization`). Run history for any 1.0 worker
148
+ cron is in the legacy **`Common.CronJobExecutions`** table (`job` = the script path, `dtCheckIn` /
149
+ `dtCheckOut`) — see [Cron Execution Monitoring](../../../1.0/apps/library/features/cron-execution-monitoring.md).
150
+
151
+ ## Reading the mailbox without touching it (`check_grand_and_toy_inbox.php`)
152
+
153
+ Because the cron drains INBOX, there was no way to answer *"did G&T actually mail a file?"*.
154
+ `worker/crons/toga2/compasscanada/workflow/check_grand_and_toy_inbox.php` is a **read-only
155
+ inspector** for `compasscanada.status@togatech.com`. It opens `INBOX`, `ARCHIVE` and `Junk Email`
156
+ with `OP_XOAUTH2 | OP_READONLY` and **never moves, deletes, imports or emails anything**, so it is
157
+ safe to run in production.
158
+
159
+ Per attachment it reports: byte size; the first line's column count against the 24-column contract;
160
+ the first 3 lines each with their own column count (a stray title row is the usual culprit); and a
161
+ **row-level dry run** mirroring the importer's per-row guards — importable / no tracking / no PO /
162
+ no part number / scientific-notation tracking / electronic delivery. `--save=<folder>` dumps each
163
+ attachment to disk for offline checking. It reads the Azure OAuth values **out of the importer file
164
+ at runtime** rather than duplicating the client secret.
165
+
166
+ Two 1.0 IMAP gotchas encoded in it (they cost real time to find):
167
+ - **`imap2_headerinfo()` fails on this setup** — it needs `ZBateson\MailMimeParser`, which is not
168
+ installed. Read raw headers with **`imap2_fetchheader()`** and parse `Date` / `From` / `Subject`
169
+ with a regex instead.
170
+ - **`App_Framework::registerPhpImap2()` does not load `php-imap2/src/Polyfill.php`**, so loading
171
+ that file does not give you `imap2_headerinfo()` either.
172
+
173
+ ## In-transit email is owned by the cron, not api2
174
+
175
+ The importer sends the Compass Canada in-transit email itself and stamps
176
+ `TrackingNumbers.c_dtInTransitEmailSent`, guarded by
177
+ `if (!empty($salesOrderNumber) && $isNewTrackingNumber)`. So **the email fires only when the cron
178
+ creates a brand-new `TrackingNumbers` row.** Two practical consequences:
179
+
180
+ 1. **Never delete a `TrackingNumbers` row during cleanup.** A replay then recreates it as new and
181
+ sends a **duplicate** in-transit email to that contact. Keep the row and its
182
+ `c_dtInTransitEmailSent` stamp instead.
183
+ 2. **Posting an ASN by hand through api2 emails nobody**, because the send lives in the cron. Useful
184
+ for a silent data repair, and the reason a hand-posted ASN looks "complete but nobody was told".
185
+
186
+ Downstream, `worker/crons/toga2/compasscanada/update_salesorder_status_from_g&t.php` selects its
187
+ candidates via `AdvanceShippingNoticeItemUnits` + `AdvanceShippingNoticeItemUnits_TrackingNumbers`,
188
+ so an ASN with **zero items** can never reach it — no partial in-transit or delivered email either.
189
+ See [Compass Partial In-Transit & Delivered Emails](../../../1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md).
190
+
116
191
  ## The cXML documents G&T actually sends
117
192
 
118
193
  Measured across **all 1,074** stored documents in `Logs_CompassCanada.Api` — the shape never varies:
@@ -161,6 +236,13 @@ created and linked to the ASNIU, and interceptor **55** then propagates it to
161
236
  `ItemFulfillmentItemUnits`. Verified in prod (Units row created + linked, IFIU populated).
162
237
 
163
238
  Constraints:
239
+ - **A bare `unit.uuid` will NOT create the Unit.** Posting
240
+ `advanceShippingNoticeItemUnits[].unit.uuid` for a uuid that has no `Units` row returns
241
+ **400 `EV-12`** *"unable to find a unique match based on the provided search criteria"* on field
242
+ `unit` — api2 only *resolves* an existing Unit by uuid. `MATCH_UPSERT` can only create when the
243
+ nested object carries enough to build the row (`itemId` + `serialNumber`). The CSV importer works
244
+ around this by `INSERT`ing the `Units` row itself (`uuid`, `itemId`, `dtUpdated`, `serialNumber`)
245
+ **before** it builds the payload, so **any manual replay must create the Unit first**.
164
246
  - `Units.itemId` is **NOT NULL**, so the item must be supplied.
165
247
  - The `Units` identifiers are `uuid`, `itemId`, `serialNumber`, `assetTag`, `vendorId`, `macAddress`
166
248
  — so **item + serialNumber** gives find-or-create on that pair.
@@ -174,6 +256,21 @@ Compass-Canada-specific; parallels Compass USA's ODP / Strategic-Systems ASN imp
174
256
  a bilingual EN/FR in-transit email (USA is English only) and uses Canadian carriers.
175
257
 
176
258
  ## Gotchas / known issues
259
+ - **⚠ A validation guard placed inside the serial-number branch produced ASNs with ZERO line
260
+ items, silently (fixed + deployed to prod 2026-09-08).** The PO-scoped part-number lookup
261
+ (`VendorItems.vendorPartNumber` on the PO) sat **inside** `if (!empty($serialNumbersForLine))`.
262
+ A CSV row with **no serial number** skipped the check entirely, so the cron wrote the ASN
263
+ **header**, then found no PO line to attach, and left the ASN with **zero
264
+ `AdvanceShippingNoticeItems`**. No error was raised — it looked like a real shipment but shipped
265
+ nothing. A row **with** a serial hit the check and was correctly rejected into the error report.
266
+ **That is why one bug produced two unrelated-looking symptoms** (empty ASN vs. rejected row).
267
+ Fix: the lookup moved out of the serial branch so it runs for every row before anything is
268
+ written, and a failing row returns a validation error naming the part number and the PO. An empty
269
+ ASN is also invisible downstream — the status cron selects via
270
+ `AdvanceShippingNoticeItemUnits(_TrackingNumbers)`, so it never reaches it.
271
+ - **⚠ The in-transit email is gated on `$isNewTrackingNumber`, so deleting a `TrackingNumbers`
272
+ row during cleanup re-emails the customer.** See
273
+ [In-transit email is owned by the cron](#in-transit-email-is-owned-by-the-cron-not-api2).
177
274
  - **⚠ `PurchaseOrderItems_SalesOrderItems` is EMPTY across ALL 865 Compass Canada purchase orders —
178
275
  this is NORMAL and must NOT be "repaired".** `_Model_Compass_AdvanceShippingNotice::postPost`
179
276
  resolves PO to SO without that bridge for this client. Inserting bridge rows to make one ASN
@@ -228,6 +325,20 @@ a bilingual EN/FR in-transit email (USA is English only) and uses Canadian carri
228
325
  before `fetchOne()`.
229
326
 
230
327
  ## Change history
328
+ - 2026-09-10 — **Corrected the "CSV cron is retired" claim: both transports are live.** The CSV
329
+ importer (renamed `4_…` → `3_import_grand_and_toy_advance_shipping_notices.php`, commit
330
+ `d38d0c33`) still runs `0 */4 * * *` and is the only source of serials and the only sender of the
331
+ in-transit email. **Fixed (prod 2026-09-08) the guard-placement bug that created ASNs with zero
332
+ line items** — the PO-scoped part-number lookup was inside the serial-number branch, so
333
+ serial-less rows bypassed it; now it runs for every row before any write and returns a named
334
+ validation error. Documented: the file-shape rules that reject a whole file (first line only,
335
+ exactly 24 columns, position-based — so `.numbers` zips and Numbers-exported title rows both
336
+ count as 1 column); the mailbox drain to ARCHIVE / Junk Email; `Common.CronJobExecutions` as the
337
+ run log and the America/Chicago worker clock; the `$isNewTrackingNumber` email gate (never delete
338
+ a `TrackingNumbers` row, and a hand-posted ASN emails nobody); and that a bare `unit.uuid` returns
339
+ 400 EV-12 rather than creating the Unit. Built the read-only inbox inspector
340
+ `check_grand_and_toy_inbox.php` with its two IMAP gotchas (`imap2_headerinfo()` needs
341
+ `ZBateson\MailMimeParser`; `registerPhpImap2()` skips `Polyfill.php`). (bala)
231
342
  - 2026-08-20 — **Root-caused and fixed a 67-day, 100%-rejection outage on the G&T cXML ASN feed and
232
343
  backfilled 518 shipments.** Root cause: nested tracking-number creation is governed by the
233
344
  per-API `Client_<x>.Apis_RecordFields.overrideChildPolicy`, and Canada's "Grand & Toy (Cxml)" API
@@ -17,7 +17,7 @@ project: _Underscore
17
17
  client: compass-canada
18
18
  type: profile
19
19
  status: active
20
- updated: 2026-09-04
20
+ updated: 2026-09-10
21
21
  owners: [jcardinal, bala, tcox, apeterson, ajean]
22
22
  files: []
23
23
  related:
@@ -65,8 +65,11 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
65
65
  - **Grand & Toy (G&T)** — primary hardware vendor. SOs flow toga → MITS → PO to G&T; G&T sends
66
66
  back ASNs. ASN ingestion (email CSV + the auto-created ItemFulfillment chain + bilingual
67
67
  in-transit email) is documented in [Grand & Toy ASN Import](features/grand-and-toy-asn-import.md).
68
- Vendor uuid: `App_Client_CompassCanada::UUID_VENDOR__GRAND_TOY`. **The live ASN path is cXML**
69
- (api2 `cxml-writer`), not the retired 24-column CSV cron; carrier codes G&T sends are `UPS`,
68
+ Vendor uuid: `App_Client_CompassCanada::UUID_VENDOR__GRAND_TOY`. **Two ASN paths are live at
69
+ once (corrected 2026-09-10):** the api2 `cxml-writer` cXML gateway **and** the 24-column CSV
70
+ email cron `workflow/3_import_grand_and_toy_advance_shipping_notices.php` (`0 */4 * * *`) — the
71
+ CSV cron is **not** retired, and it is the only source of serial numbers and the only sender of
72
+ the in-transit email. Carrier codes G&T sends are `UPS`,
70
73
  `FDXH`, `PUROL`, `PRECI`, `NATEX`, `GANDT`, `DHL`, `XXXX` (placeholder, TrackingID = `DIRECT`).
71
74
  The cXML path needs **no** Ground `ShippingMethod` per carrier (`TrackingNumbers.shippingMethodId`
72
75
  is nullable and cXML sends no method) — that requirement came from the CSV cron only.
@@ -81,8 +84,11 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
81
84
  client.** `postPost` resolves PO to SO without it; never insert bridge rows to "repair" an ASN
82
85
  failure here.
83
86
  - Worker crons for the G&T flow live under `worker/crons/toga2/compasscanada/workflow/`
84
- (`1_…` transmit SOs to MITS, `2_…` transmit POs to vendors, `3_…` status from G&T cXML,
85
- `4_…` import G&T ASNs).
87
+ (`1_…` transmit SOs to MITS, `2_…` transmit POs to vendors, `3_import_grand_and_toy_advance_shipping_notices.php`
88
+ import G&T ASNs). **The importer was renamed `4_…` → `3_…` (commit `d38d0c33`) and
89
+ `cron.worker.sync.json` repointed** — any note using the `4_` name is stale. SO-status updates
90
+ live one level up in `update_salesorder_status_from_g&t.php`, and the read-only mailbox inspector
91
+ is `workflow/check_grand_and_toy_inbox.php`.
86
92
  - **⚠ MITS transmit alerting has diverged from Compass USA.** Canada's `1_…transmit…to_mits`
87
93
  cron still raises all **three** of its failure alerts as in-cron `App_Email_Agilant` emails
88
94
  (No Items Found / more than 15 items / Invalid response from MITS API), so changing who is
@@ -5,10 +5,11 @@ project: _Underscore
5
5
  client: compass-canada
6
6
  type: workflow
7
7
  status: active
8
- updated: 2026-08-20
8
+ updated: 2026-09-10
9
9
  owners: ["bala"]
10
10
  files:
11
11
  - worker/crons/toga2/compasscanada/workflow_beta/import_grand_and_toy_asn_from_file.php
12
+ - worker/crons/toga2/compasscanada/workflow/3_import_grand_and_toy_advance_shipping_notices.php
12
13
  - api2/Component/Api/Cxml/Cxml.php
13
14
  - _underscore/Model/Compass/AdvanceShippingNotice.php
14
15
  related:
@@ -57,6 +58,41 @@ original EV-12 rejections — see
57
58
  **vendor** part number, and reference the item **by uuid only**. See the ASN-import doc's
58
59
  serial-number section for the constraints.
59
60
 
61
+ ## Route C — SQL backfill of a missing ASN line (a zero-line-item ASN)
62
+
63
+ Used 2026-09-08 to repair ASNs the importer had created with **zero
64
+ `AdvanceShippingNoticeItems`** (guard-placement bug — see the
65
+ [import doc](../features/grand-and-toy-asn-import.md)). This route is only tractable because of one
66
+ fact about how api2 builds the fulfillment side.
67
+
68
+ **`ItemFulfillments.number` IS the sales-order number, and the column is UNIQUE.** So there is at
69
+ most **one `ItemFulfillment` per sales order**, and when a second ASN arrives for the same SO api2
70
+ attaches its items to the **existing** fulfillment rather than creating another one. Verified in
71
+ prod: posting a new ASN for `SAC100689` attached to the existing `ItemFulfillment 671` instead of
72
+ failing on a duplicate number.
73
+
74
+ **Consequence — whenever the sales order already has a fulfillment, a hand backfill needs no new
75
+ `Addresses` row and no `ItemFulfillments.number` allocation.** That matters: allocating a `SAC`
76
+ number by hand in prod risks colliding with a live fulfillment.
77
+
78
+ Rows to insert per missing ASN line:
79
+
80
+ 1. `AdvanceShippingNoticeItems`
81
+ 2. `AdvanceShippingNoticeItems_TrackingNumbers`
82
+ 3. `ItemFulfillmentItems`
83
+ 4. `ItemFulfillmentItems_TrackingNumbers`
84
+ 5. `ItemFulfillments_TrackingNumbers` — **only** where the fulfillment header does not already
85
+ carry that tracking number.
86
+
87
+ **Do not delete the `TrackingNumbers` row and replay the cron instead.** The importer's email is
88
+ gated on `$isNewTrackingNumber`, so a recreated row counts as new and sends a **duplicate**
89
+ in-transit email to the customer. Keep the row and its `c_dtInTransitEmailSent` stamp, and repair
90
+ the missing children around it. (Posting the ASN through api2 by hand emails nobody, because the
91
+ send lives in the cron.)
92
+
93
+ **A `Units` row must exist first** if the line is serialized — a bare `unit.uuid` in an ASN POST
94
+ returns 400 `EV-12`, it does not create the Unit.
95
+
60
96
  ## Dedupe and safety guards
61
97
 
62
98
  - **`TrackingNumbers.number` carries a UNIQUE key**, so a replayed tracking number `MATCH`es rather
@@ -87,6 +123,13 @@ original EV-12 rejections — see
87
123
  - **One `TrackingID` even when `TotalPackages > 1`** — do not synthesize per-package numbers.
88
124
 
89
125
  ## Change history
126
+ - 2026-09-10 — Added **Route C — SQL backfill of a zero-line-item ASN**, after repairing the
127
+ Compass Canada ASNs the importer created with no `AdvanceShippingNoticeItems`. Recorded that
128
+ `ItemFulfillments.number` is the sales-order number **and UNIQUE**, so there is one
129
+ `ItemFulfillment` per SO and api2 attaches later ASN items to it (proven in prod: `SAC100689`
130
+ → existing `ItemFulfillment 671`) — which means a backfill needs **no** new `Addresses` row and
131
+ **no** `SAC` number allocation. Listed the five row types per missing line, and the rule to
132
+ **keep** (never delete) the `TrackingNumbers` row so a replay cannot re-email the customer. (bala)
90
133
  - 2026-08-20 — Documented the two-route backfill after recovering the 67-day G&T outage (518
91
134
  shipments / 536 ASNs / 512 TrackingNumbers / 406 ItemFulfillments / 332 Units). Recorded the cXML
92
135
  replay source and the ~20s `maxAuthsPerHour` pacing, the three-bridge JSON payload requirement,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.792",
3
+ "version": "1.0.794",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",