toga-ai 1.0.847 → 1.0.849
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/library/features/netsuite-suiteql-rest-shim.md +5 -1
- package/knowledge/1.0/apps/library/features/toga2-api-client-and-bridge.md +5 -2
- package/knowledge/1.0/apps/tools/features/compass-user-persona-admin.md +0 -37
- package/knowledge/1.0/apps/worker/INDEX.md +1 -1
- package/knowledge/1.0/apps/worker/features/compass-partial-in-transit-delivered-emails.md +9 -72
- package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md +66 -5
- package/knowledge/2.0/apps/api2/INDEX.md +1 -1
- package/knowledge/2.0/apps/api2/architecture.md +91 -461
- package/knowledge/2.0/apps/worker2/INDEX.md +1 -1
- package/knowledge/2.0/apps/worker2/features/oneuptime-worker2-monitoring.md +373 -540
- package/knowledge/clients/compass-usa/INDEX.md +2 -2
- package/knowledge/clients/compass-usa/features/people-file-user-lifecycle.md +53 -267
- package/knowledge/clients/compass-usa/features/tracking-number-origin-and-asn-link.md +32 -70
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@ project: Library
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
9
|
+
updated: 2026-09-21
|
|
10
10
|
owners: [dfranks, jcardinal, kyalamarthi, rgirish]
|
|
11
11
|
files:
|
|
12
12
|
- library/app/api/netsuite/rest.php
|
|
@@ -57,6 +57,7 @@ Field semantics (verified on the live account):
|
|
|
57
57
|
- **Line number = `transactionline.id`, NOT `linesequencenumber`.** `tl.id` equals the REST `item[].line` value (the stable SOAP-era line key); `linesequencenumber` is physical order and re-maps/collides when SOAP-era records are edited. Verified: sales orders 106 orders / 344 lines (2019–2029, incl. edited orders with line-id gaps), invoices + cash sales 62/62. `listSales` historically resolved line via `uniquekey → REST lineUniqueKey → line`; `tl.id` gives the same answer in bulk without the per-id GET. **Re-confirmed at scale 2026-08-27:** the two columns diverge on **2,061 of 4,387** lines (~47%), and `Forecast.OpenOrderItems.lineNumber` (which stores REST `$line->line`) matched `transactionline.id` on **99 of 99** randomly sampled prod rows plus all 7 multi-root orders, with zero join misses. So keying a NetSuite ↔ Forecast line join on `linesequencenumber` silently mismatches about half the book — always join on `transactionline.id`.
|
|
58
58
|
- **ShipItem lines have `costestimate = NULL`** (100% of 42K+ lines, 2025–26). In any profit expression `SUM(-foreignamount + costestimate)`, a NULL cost makes the whole term NULL and SQL **drops the row from the SUM** — so shipping *revenue* still counts but shipping *profit* does not, silently understating NS profit. Always `NVL(tl.costestimate, 0)`. (Caused a fake $663K "gap" in the audit tools — see the reconciliation doc.)
|
|
59
59
|
- **Cash refunds & credit memos carry extra COGS/inventory lines.** SuiteQL `transactionline` with `mainline='F' AND item>0` returns, per item, the customer line **plus** a `CUSTOMERRETURN` and an `ASSET` posting (all `item>0`), which REST's `item[].items` sublist omits. They are distinguished by **`tl.iscogs = 'T'`** (and `accountinglinetype` IN `CUSTOMERRETURN`/`ASSET`). Filter **`tl.iscogs = 'F'`** to get exactly the customer-facing lines. Invoices/cash sales do not have this (0 extra lines). Without the filter, bulk imports duplicate refund/memo lines.
|
|
60
|
+
- **⚠ `listSales()` takes a `$recordTypes` allowlist — pass `RECORD_TYPE_*` CONSTANTS, never raw SuiteQL codes (2026-09-21).** `listSales(...)`'s trailing `array $recordTypes` **defaults to all four** `RECORD_TYPE_INVOICE`/`_CASH_SALE`/`_CASH_REFUND`/`_CREDIT_MEMO`, which the **forecast import** (`worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php`, `periodic_forecast_discrepancy_fix.php`) relies on. `listSales` maps each constant to its SuiteQL code internally (`RECORD_TYPE_INVOICE`→`CustInvc`, `_CASH_SALE`→`CashSale`, `_CASH_REFUND`→`CashRfnd`, `_CREDIT_MEMO`→`CustCred`) and **throws on an unknown/empty type**. The **togasupply sync passes `[RECORD_TYPE_INVOICE]` — INVOICES only** — because a credit memo's `createdFrom` is the invoice it credits (not a sales order) and dead-ended `syncInvoiceFromNetsuite`; see the [per-client-sync doc](../../worker/features/netsuite-togasupply-per-client-sync.md).
|
|
60
61
|
- **Sign conventions are per-type and self-consistent.** SuiteQL returns `foreignamount`/`quantity`/`costestimate` **negative for invoices & cash sales, positive for credit memos & cash refunds**. A uniform `-foreignamount` therefore yields **positive** revenue for invoices and **negative** for reversals — the agreed Forecast2 convention. Do NOT also apply a per-type factor (it double-negates reversals back to positive). The shim flips `amount = -foreignamount`, `costEstimate = -costestimate`.
|
|
61
62
|
- **⚠ `->holdInvoice` on a sales-order shim is NOT a native NetSuite field — it is normalised from the custom checkbox `custbody_asi_hold` (internal id 4082), and that field is SHARED by all clients.** NetSuite REST exposes no top-level `holdInvoice`. The `"asi"` prefix is **Agilant Solutions Inc** (the company), *not* a client prefix — do not read it as client-specific. The read lives **inside the shim**, in all three fetch paths: `buildSalesOrderShims()` (~L533, feeding both `listSalesOrders()` and `fetchSalesOrderByIdFull()`) and `fetchSalesOrderById()` (~L1537), so callers such as `App_Api_Toga2::isTransferOrder()` read an already-normalised boolean. (An earlier version read `$nsOrder->holdInvoice` in the caller with a `custbody_hold_invoice` fallback — the wrong field name; commit `07c31e24`, 2026-08-19, found the real one and moved the read down here.) `worker/crons/toga2/compass/edi/2_create_so_netsuite.php` **writes** the same field. **⚠ The SOAP client `library/app/netsuite.php` never sets `holdInvoice` at all**, so any detection that depends on it (e.g. the `zero_dollar_hold` transfer-order mode — see [per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md)) is **silently always-false** on a SOAP path. Verified not a live risk: the togasupply engine calls `App_Api_Netsuite_Rest::` on every sales-order/receipt/fulfillment path.
|
|
62
63
|
- **`createdFrom` is not a usable SuiteQL column.** `transaction.createdfrom` is **always NULL** in SuiteQL (26/26). It is recoverable from `previoustransactionlinelink.previousdoc` excluding `previoustype = 'Opprtnty'` (an invoice links to both its SO and its opportunity; REST's `createdFrom` is the SO). **But this is non-deterministic for multi-link transactions** (credit memos return several non-Opprtnty `previousdoc` rows in arbitrary order) — two runs can pick different values. REST `createdFrom` (per-id GET) is the only deterministic source. Treat the link table as best-effort, not authoritative.
|
|
@@ -93,6 +94,9 @@ Field semantics (verified on the live account):
|
|
|
93
94
|
- Field availability varies by NetSuite account **and** record type — probe the live account before assuming a column/relationship exists.
|
|
94
95
|
- **`App_NetSuite::getCustomFieldValue()` (`library/app/netsuite.php`) must be warning-safe.** 1.0 runs on PHP 7.2 in an unguarded path, so any PHP warning → `exit()` (kills the cron). It was hardened 2026-08-27 with `?? []` / `?? null` guards on the custom-field-list traversal. Keep that discipline when touching it.
|
|
95
96
|
|
|
97
|
+
## Change history
|
|
98
|
+
- 2026-09-21 — `listSales()` gained a `$recordTypes` allowlist param (defaults to all 4 for the forecast import); callers pass `RECORD_TYPE_*` constants, not raw SuiteQL codes; throws on unknown/empty. Lets the togasupply sync request INVOICES only. (jcardinal)
|
|
99
|
+
|
|
96
100
|
## Related
|
|
97
101
|
|
|
98
102
|
- `netsuite-suiteql-api-reference.md` — the broad API reference (auth, SuiteQL mechanics, table schemas, type/status codes, custom-field map, performance) these semantics sit on top of.
|
|
@@ -6,7 +6,7 @@ project: Library
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
9
|
+
updated: 2026-09-21
|
|
10
10
|
owners: [jcardinal, mhammontree, bala]
|
|
11
11
|
files:
|
|
12
12
|
- library/app/api/toga2.php
|
|
@@ -163,7 +163,7 @@ Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven i
|
|
|
163
163
|
## Gotchas
|
|
164
164
|
|
|
165
165
|
- **A 2.0 GET silently TRUNCATES a nested child collection — never trust a single response for a count.** When you read a parent with a nested `hasMany` collection in one GET, api2 can return the collection **partial** (observed: a fulfillment's nested `itemFulfillmentItemUnits` came back **45 of 70**) with no error and no obvious signal — so any verification that counts off the nested array is silently wrong. **Read the collection flat and fully paginated** (query the child resource directly with the universal `meta->nextPage` loop) and take counts from **`meta.totalRecordCount`**, not from a single nested response. Surfaced building the NYCHH fulfillment asset-tag verifier (`worker/crons/toga2/netsuite/verify_fulfillment_asset_tag_sync_nychh.php`). **Confirmed again on the fulfillment reconcile (2026-09-13):** the `salesOrderItems` nested collection truncated at **25**, and a fulfillment's `itemFulfillmentItemUnits` re-created 25,766 links for 1,192 real units. The reconcile now reads both flat + fully paginated via two new private helpers, `getItemFulfillmentItemUnitsByItemFulfillmentUuid()` and `getSalesOrderItemsBySalesOrderUuid()` (`recordsPerPage` 1000, page-cap 500, `nextPage ?? null`) — detail on the [per-client sync doc](../../worker/features/netsuite-togasupply-per-client-sync.md).
|
|
166
|
-
- **🔐 Hardcoded credentials.** Every client's `CLIENT_UUID_*`, `API_UUID_*`, and `API_SECRET_*` (plus the "True Workers" master keys) are committed as plaintext class constants at the top of `toga2.php`. Treat them as **compromised-if-leaked**; do not reproduce the values in the KB, tickets, or logs. Long-term they should move to config/SSM. (Observed 2026-06-23.)
|
|
166
|
+
- **🔐 Hardcoded credentials.** Every client's `CLIENT_UUID_*`, `API_UUID_*`, and `API_SECRET_*` (plus the "True Workers" master keys) are committed as plaintext class constants at the top of `toga2.php`. Treat them as **compromised-if-leaked**; do not reproduce the values in the KB, tickets, or logs. Long-term they should move to config/SSM. (Observed 2026-06-23.) **⚠ Also (found 2026-09-21, NOT fixed): a committed JWT (a dev/NYCHH token) sits inline at `library/app/api/toga2.php:641`, inside `App_Api_Toga2::send()` — the live sync call path.** Pre-existing, not introduced this session. **Rotate the token and delete the line;** record its location only, never the value.
|
|
167
167
|
- **Two unrelated `sync_togasupply_<client>.php` families.** Files under `worker/crons/toga2/netsuite/` are the **NetSuite supply importer** (`require` the common engine). Files under `worker/crons/toga2/<client>/` (e.g. `aig/`, `wje/`) are this **bridge** (`syncWithToga`/`syncWithTogadesk`). Same filename, different integration, different parameter keys — they coexist harmlessly. Confirm which folder before editing.
|
|
168
168
|
- **`syncWithToga` and the bridge build SQL by hand** with `App_Database::sqlEscape()` rather than the `App_Model` layer; follow the surrounding escaping discipline when modifying.
|
|
169
169
|
- **Per-record error isolation** in the TOGaDesk sync is via `try/catch` → `\Sentry\captureException`; a thrown exception elsewhere (transport, checkpoint read) still aborts the whole run.
|
|
@@ -179,6 +179,9 @@ Signature takes `$integrationType` (`TICKETS` or `REPAIR_ORDERS`) plus **seven i
|
|
|
179
179
|
- **This logic IS testable without a database — and there is a harness.** `App_Database::sqlEscape()` needs **no DB connection** (pure `str_replace`), so the email transform can be exercised statically. The TRUE-79401 transform is inlined in the DB-writing `private static syncContactToToga1Customer()`, so the harness **mirrors** it and guards against drift by asserting the production statements verbatim — see [static no-DB regression harness](../../test/features/static-no-db-regression-harness.md). **If you edit those 5 statements, the harness will refuse to report until it is updated.** The clean fix if you are already in there: extract the transform to a `public static` helper so the mirror disappears.
|
|
180
180
|
- **Pre-existing (low severity, out of scope): a raw-vs-escaped comparison causes a redundant UPDATE every sync.** At the update path (~line 8584) `syncContactToToga1Customer` compares `$existingCustomer['emailAddress'] !== $emailAddress` — the DB value from `App_Database::fetchRow()` is **unescaped**, while `$emailAddress` has already been through `App_Database::sqlEscape()`. For any value containing `'`, `"` or `\` the two can **never** compare equal, so that column is rewritten on **every** run. Same pattern applies to `businessName` and `phoneNumber` just above. **Predates TRUE-79401**, is a redundant write (not data loss), and was deliberately left out of that ticket's scope — recorded here so a future reader knows it is known.
|
|
181
181
|
|
|
182
|
+
## Change history
|
|
183
|
+
- 2026-09-21 — Recorded a pre-existing committed JWT (dev/NYCHH token) at `toga2.php:641` inside `send()` (the sync call path); flagged to rotate + delete, location-only. (jcardinal)
|
|
184
|
+
|
|
182
185
|
## Related
|
|
183
186
|
|
|
184
187
|
- [NetSuite → TOGa Supply per-client sync](../../worker/features/netsuite-togasupply-per-client-sync.md) — the importer that uses this class's transport and `*FromNetsuite()` methods.
|
|
@@ -249,40 +249,3 @@ names, reads `Core.ClientEmailDomains` / `Core.Clients`, and inserts into `Logs_
|
|
|
249
249
|
added in worker2 on 2026-09-16, or the block silently under-covers the protected set.
|
|
250
250
|
- **ops** — rotate every secret in `tools/config.production.ini`; consider a least-privilege
|
|
251
251
|
Compass account.
|
|
252
|
-
|
|
253
|
-
## Change history
|
|
254
|
-
|
|
255
|
-
- 2026-09-16 — Recorded that worker2's cross-region protected list **grew from four addresses to
|
|
256
|
-
six** (`ALWAYS_ACTIVE_EMAILS_BOTH` + `ALWAYS_ACTIVE_EMAILS_CA`), so this tool's
|
|
257
|
-
`isCrossRegionProtectedEmail()` block now under-covers it: the two new `@compassdigital.io`
|
|
258
|
-
Canada-protected addresses can still be email-edited here and would silently drop out of the
|
|
259
|
-
protected set, because the worker2 match is on email. No `tools` code changed. Detail:
|
|
260
|
-
[PEOPLE-File User Lifecycle](../../../clients/compass-usa/features/people-file-user-lifecycle.md).
|
|
261
|
-
(bala)
|
|
262
|
-
|
|
263
|
-
- 2026-09-04 — Added **email, phone and cost-center editing** (PR tools#14,
|
|
264
|
-
`feature/compass-user-field-edits`, **open / not merged / not deployed**): new `App_Compass`
|
|
265
|
-
setters plus the `normaliseEmail` / `normalisePhone` / `emailDomainIsRegistered` /
|
|
266
|
-
`usersWithEmail` / `isCrossRegionProtectedEmail` validators, a cost-center picker with a mandatory
|
|
267
|
-
before/after persona diff, and a `Users` + primary `Contacts` write in one transaction. No schema
|
|
268
|
-
change. Recorded the decision to ship **all three** fields against independent
|
|
269
|
-
architecture/security advice to defer email, and that the six safety rails (typed twice,
|
|
270
|
-
`Core.ClientEmailDomains` domain check, cross-row duplicate refusal, duplicate-twin confirmation,
|
|
271
|
-
cross-region address block, audit-before-write) are the conditions of that decision. Recorded that
|
|
272
|
-
**cost-center edits are deliberately temporary** because `c_erpSystemEntityId` is not overwritten.
|
|
273
|
-
**Resolved the "no staff-actor audit trail" open item** — `Logs_<Tenant>.Record.userId` is
|
|
274
|
-
nullable, so `recordId = 2` (`Users`) + `primaryKeyId` + staff identity in `note` works with no
|
|
275
|
-
schema change (`note` is 255 **characters** — truncate with `mb_substr`); hence the new
|
|
276
|
-
`[database_compasslogs]` / `[database_compasscanadalogs]` ini sections, still on the existing
|
|
277
|
-
`[database_true]` account. Prod-verified that **neither tenant has a `UNIQUE KEY` on
|
|
278
|
-
`Users.email`**, so the duplicate check must live in code, and that `compass-canada.com` is
|
|
279
|
-
registered to Compass **USA** (client 2), so an email-domain allowlist must span both Compass
|
|
280
|
-
clients. Marked PR tools#13 **merged** and the doc `active`. (ajean)
|
|
281
|
-
- 2026-09-04 — Initial: built the two-page Compass user/persona admin (PR tools#13, unmerged) — the
|
|
282
|
-
first `tools` pages that write to a client tenant DB, gated to `Development Team` only, with a
|
|
283
|
-
US/Canada switcher and new `[database_compass]` / `[database_compasscanada]` ini sections reusing
|
|
284
|
-
the existing `[database_true]` account. Recorded Jeff Cardinal's scope reset (no schema changes;
|
|
285
|
-
`dbchanges2#478` and `worker2#160` closed unmerged) and its consequence — **activation is temporary
|
|
286
|
-
because the `c_isKeepActive` flag was dropped**, while deactivation is durable. Logged the accepted
|
|
287
|
-
credential-reuse risk, the 4-day session persona staleness, the missing staff-actor audit trail, and
|
|
288
|
-
that every user search is a full scan. (ajean)
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
| [Canon received-PO → NetSuite sales order sync](features/canon-received-po-netsuite-order-sync.md) | Canon-only worker cron that reads Vision customer POs and creates the matching NetSuite sales order (bundles exploded, prefixes stripped); open before touching |
|
|
7
7
|
| [Compass MA Sales Order Exception Report](features/compass-ma-sales-order-exception-report.md) | Worker cron emailing ops the "Compass Refresh Exception Report" of `MA%` Compass sales orders whose ODP NetSuite order isn't created yet; open for the two-hop b |
|
|
8
8
|
| [Compass Manager Approval Reminder Emails (1.0 worker crons)](features/compass-manager-approval-reminder-emails.md) | Two per-tenant Compass worker crons that email approvers about sales orders awaiting a decision; open before touching the approval `orderUrl` (both still send a |
|
|
9
|
-
| [Compass Partial In-Transit & Delivered Emails (per package)](features/compass-partial-in-transit-delivered-emails.md) | Compass USA
|
|
9
|
+
| [Compass Partial In-Transit & Delivered Emails (per package)](features/compass-partial-in-transit-delivered-emails.md) | Compass USA + Canada per-package in-transit and delivered emails; open for the two-pass drain, the ASN-vs-TrackingNumbers window clock, the recipient fallback, |
|
|
10
10
|
| [Elite TOGA 2.0 → TOGaDeskSupport Standalone Attachment Sync](features/elite-togadesk-attachment-sync.md) | Standalone 5-min Elite cron that downloads Freshservice-origin file attachments from TOGA 2.0 (`sourceUrl` S3) into TOGaDeskSupport; open for the uuid-resolutio |
|
|
11
11
|
| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools (`test/@dave/`, `tools/bin/forecast/`) that audit and repair drift between the prod `Forecast` DB and NetSuite; open when reconciling Sales / OpenOrde |
|
|
12
12
|
| [Forecast2 Supporting-Records Nightly Import (1.0 cron) — Customers & ConsolidatedCustomers](features/forecast2-supporting-records-import.md) | Nightly 1.0 cron that pulls Forecast2 lookup/dimension tables (Accounts, Classifications, Customers, ConsolidatedCustomers, Employees) from NetSuite; open when |
|
|
@@ -30,6 +30,8 @@ related:
|
|
|
30
30
|
- ../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
|
|
31
31
|
---
|
|
32
32
|
|
|
33
|
+
Compass USA + Canada per-package in-transit and delivered emails; open for the two-pass drain, the ASN-vs-TrackingNumbers window clock, the recipient fallback, and the many freeze/backlog gotchas.
|
|
34
|
+
|
|
33
35
|
## Summary
|
|
34
36
|
|
|
35
37
|
Compass USA and Compass Canada send a **per-package** in-transit email (and a matching
|
|
@@ -39,7 +41,9 @@ cron builds three zones: **Items in this shipment** (just that package), **Alrea
|
|
|
39
41
|
and earlier packages). The dynamic HTML is injected into a stored `EmailTemplates` body via the
|
|
40
42
|
`sendEmail` scripted API. Canada renders EN or FR per the user's language setting.
|
|
41
43
|
|
|
42
|
-
##
|
|
44
|
+
## How it works
|
|
45
|
+
|
|
46
|
+
### Key files / entry points
|
|
43
47
|
|
|
44
48
|
- `worker/crons/toga2/compass/update_salesorder_status_from_odp.php` — Compass USA prod cron,
|
|
45
49
|
registered in `worker/schedules/cron.worker.sync.json` as **"Update Compass SalesOrder Status
|
|
@@ -55,7 +59,7 @@ and earlier packages). The dynamic HTML is injected into a stored `EmailTemplate
|
|
|
55
59
|
- Shared helper set in each cron: `getOrderFulfillmentData`, `buildItemRowsHtml`,
|
|
56
60
|
`buildSectionHeaderHtml`, `buildPackageBlockHtml`, `buildTrackingUrl`, `buildOrderItemsHtml`.
|
|
57
61
|
|
|
58
|
-
|
|
62
|
+
### Mechanics
|
|
59
63
|
|
|
60
64
|
1. **Driving query** finds shipped sales orders and their tracking numbers (`computedStatusSlug
|
|
61
65
|
= 'shipped'`, `c_dtInTransitEmailSent IS NULL`, rolling date window on
|
|
@@ -214,7 +218,7 @@ the tracking number as emailed (it retries next run).
|
|
|
214
218
|
`#111928`; P/N `#666666`; Qty regular dark `#111928`.
|
|
215
219
|
- **20px** gap between consecutive "Already shipped" packages (16px spacer + 4px heading top).
|
|
216
220
|
|
|
217
|
-
|
|
221
|
+
### Data model
|
|
218
222
|
|
|
219
223
|
- `TrackingNumbers`: `number`, `trackingUrl`, `shippingCarrierId`, `dtCreated`, `status`,
|
|
220
224
|
`c_dtInTransitEmailSent` (the idempotency flag the cron sets).
|
|
@@ -223,7 +227,7 @@ the tracking number as emailed (it retries next run).
|
|
|
223
227
|
bridge). `ItemFulfillments.salesOrderId` ties a package to the buyer SO.
|
|
224
228
|
- `EmailTemplates` — the stored template body with `{placeholder}` tokens.
|
|
225
229
|
|
|
226
|
-
|
|
230
|
+
### Client variations
|
|
227
231
|
|
|
228
232
|
- **Compass USA** — English only; `update_salesorder_status_from_odp.php`. Has the full
|
|
229
233
|
2026-09-03 hardening: two-pass drain, 400-row cap, 14-day window, `isTrackingStatusFinal()`,
|
|
@@ -350,74 +354,7 @@ the tracking number as emailed (it retries next run).
|
|
|
350
354
|
tracking-numbers today; the plan (not built) is to move this email logic into a Compass `postPut`
|
|
351
355
|
interceptor and retire this 1.0 cron.
|
|
352
356
|
|
|
353
|
-
##
|
|
354
|
-
- 2026-09-16 — **Re-anchored the in-transit window on `AdvanceShippingNotices.dtCreated`** (was
|
|
355
|
-
`TrackingNumbers.dtCreated`, `update_salesorder_status_from_odp.php` line ~432);
|
|
356
|
-
`IN_TRANSIT_EMAIL_LOOKBACK_DAYS` stays 7 with a comment explaining why. Cause: commit `81824325`
|
|
357
|
-
(2026-09-15 11:35) cut the lookback 14 → 7 while the window still measured the **invoice** date,
|
|
358
|
-
so 22 shipments invoiced 2026-09-03 whose ASNs landed 2026-09-15 fell out of the window (only
|
|
359
|
-
MR245462 still owed a real email; the other 21 read DELIVERED). Verified on prod: 22 rows in,
|
|
360
|
-
0 rows lost, 1 catch-up email. **Corrected the old "invoice tracking numbers never touch the ASN
|
|
361
|
-
chain" gotcha** — the invoice sync is in fact the normal origin of Compass `TrackingNumbers` rows;
|
|
362
|
-
new doc
|
|
363
|
-
[tracking-number origin and the ASN link](../../../clients/compass-usa/features/tracking-number-origin-and-asn-link.md).
|
|
364
|
-
Recorded the **rejected** alternative of making 3b email for pre-existing tracking numbers (it
|
|
365
|
-
would downgrade split orders from the partial template to the plain one). Not yet committed or
|
|
366
|
-
deployed. (bala)
|
|
367
|
-
|
|
368
|
-
- 2026-09-10 — Recorded that a zero-line-item ASN is invisible to this cron (candidates come from
|
|
369
|
-
`AdvanceShippingNoticeItemUnits(_TrackingNumbers)`), the failure mode Compass Canada's G&T
|
|
370
|
-
importer produced until 2026-09-08. (bala)
|
|
371
|
-
|
|
372
|
-
- 2026-09-04 — **Fixed the Compass USA in-transit email outage (2026-08-27 → 2026-09-03, 130–190
|
|
373
|
-
emails/day → near zero).** Root cause: already-delivered rows were `continue`d without stamping
|
|
374
|
-
`c_dtInTransitEmailSent`, so the candidate set grew to 2,379 and every run burned its
|
|
375
|
-
`cronInitialization(3600)` hour inside rows dating back to a hardcoded `'2026-06-15'`, never
|
|
376
|
-
reaching new ASNs. Fix: two-pass drain (stamp finished rows, then send), newest-first
|
|
377
|
-
`ORDER BY MAX(dtCreated) DESC` with `IN_TRANSIT_EMAIL_BATCH_SIZE = 400`, rolling
|
|
378
|
-
`IN_TRANSIT_EMAIL_LOOKBACK_DAYS = 14` replacing the fixed date, and `isTrackingStatusFinal()`
|
|
379
|
-
short-circuiting the carrier call on `DELIVERED`/`RETURNED_TO_SENDER` using the status worker2
|
|
380
|
-
`Sync/TrackingNumbers` already maintains. Also **widened the recipient**: `LEFT OUTER JOIN Users`
|
|
381
|
-
+ `Contacts`/`ContactEmailAddresses` with an ownership guard and
|
|
382
|
-
`IFNULL(Users.email, ContactEmailAddresses.emailAddress)`, which recovered 12 EDI-created MR
|
|
383
|
-
orders that have no `Users` row (regression-checked: 4,416 of 4,419 SA rows selected by both old
|
|
384
|
-
and new queries, 0 gained, 0 lost, identical resolved address/name). Backlog cleared by
|
|
385
|
-
**stamping** 64,132 out-of-window + 421 final-status rows rather than emailing them. Recorded
|
|
386
|
-
that **invoice tracking numbers are not in scope** (bridged via `InvoiceTrackingNumbers`, never
|
|
387
|
-
the ASN chain) and that a **`&` in a cron filename stops the cron dead**. Canada got the batching
|
|
388
|
-
changes only (renamed to `update_salesorder_status_from_g&t.php`; duplicate
|
|
389
|
-
`workflow/3_update_salesorder_status_from_grand_and_toy.php` deleted); its recipient block is
|
|
390
|
-
**still `INNER JOIN Users`**. New OneUptime monitor `Monitor/Compass/InTransitEmailQueue` now
|
|
391
|
-
watches this queue — see
|
|
392
|
-
[OneUptime push-metric monitors](../../../2.0/apps/worker2/features/oneuptime-worker2-monitoring.md). (bala)
|
|
393
|
-
- 2026-08-26 — Flagged the **duplicated `_status` CASE** in the three crons as a standing divergence
|
|
394
|
-
risk after the 2.0 Compass fulfillment rule was rewritten. Verified the crons are unaffected: their
|
|
395
|
-
copies only emit canceled / pendingApprovalUnknown / pendingFulfillment / shipped and filter on
|
|
396
|
-
`shipped`, and the pendingFulfillment gate was left byte-identical. No code change here. (bala)
|
|
397
|
-
|
|
398
|
-
- 2026-08-24 — Recorded the **conflict with the new 2.0 platform-wide tracking-status refresh**: that
|
|
399
|
-
refresh writes `TrackingNumbers.status` via api2 PUT for all clients, and this cron's delivered
|
|
400
|
-
UPDATE is guarded by `WHERE status <> 'DELIVERED'` — so if the refresh marks a Compass row
|
|
401
|
-
`DELIVERED` first, the delivered email never sends. Noted there is no `postPut` interceptor on
|
|
402
|
-
tracking-numbers and the plan to move this email logic into a Compass `postPut` interceptor and
|
|
403
|
-
retire this cron. Added `send_delivered_email.php` to the file list. (bala)
|
|
404
|
-
- 2026-08-03 — Compass Canada **item rows are now localized**: titles resolved from the
|
|
405
|
-
`ItemTranslations` sidecar (two-column select + PHP fallback, never a mixed-collation SQL
|
|
406
|
-
`COALESCE`) and translated `N/P` / `Qté` labels; language resolution moved onto the shared 1.0
|
|
407
|
-
helper `App_Client_CompassCanada::resolveUserLanguageCode()` with `try/catch` → English at every
|
|
408
|
-
call site so a lookup failure can't abort the rest of the run. Item text is entity-escaped for the
|
|
409
|
-
iso-8859-1 send. Flagged the **`library` deployment dependency** (uncommitted on `_production`;
|
|
410
|
-
all 8 Canada crons fatal without it). TRUE-80543, not yet deployed. Detail lives on
|
|
411
|
-
[Compass Canada French order emails](../../../clients/compass-canada/features/french-order-email-localization.md). (bala)
|
|
412
|
-
- 2026-06-18 — Switched Canada `getOrderFulfillmentData` from the ASN chain to the
|
|
413
|
-
`ItemFulfillmentItems` line bridge (match Compass USA); item info all 14px. (bala)
|
|
414
|
-
- 2026-06-18 — Brought both prod crons to parity: per-package partial assembly, `buildTrackingUrl`
|
|
415
|
-
carrier-URL precedence, POST body transport + `throwExceptionOnApiError=true`, mark-sent only
|
|
416
|
-
on success; restored crons after a git stash/merge had duplicated helper functions. (bala)
|
|
417
|
-
- 2026-06-17 — Styling pass: 18px headers, medium dark item titles, regular dark Qty, 20px
|
|
418
|
-
between packages. (bala)
|
|
419
|
-
|
|
420
|
-
## Related docs
|
|
357
|
+
## Related
|
|
421
358
|
|
|
422
359
|
- `2.0/apps/_underscore/features/email-template-sending.md` — the `sendEmail` scripted API +
|
|
423
360
|
its silent-failure hardening.
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
9
|
+
updated: 2026-09-21
|
|
10
10
|
owners: ["dfranks", "bala", "jcardinal", "mhammontree", "snaredla", "rgirish"]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/netsuite/common_sync_togasupply.php
|
|
@@ -834,10 +834,10 @@ then at header level:
|
|
|
834
834
|
Prod check: 391 rows TO-side, 293 SO-side, **0 with both, 0 with neither**.
|
|
835
835
|
|
|
836
836
|
**It throws — never skips — on:** more than one SalesOrder for the NetSuite id; a TO with no
|
|
837
|
-
lines; an SO line with no matching TO line;
|
|
838
|
-
`transferOrderItemId`**, so they physically cannot be moved); or any bridge whose existence check
|
|
837
|
+
lines; an SO line with no matching TO line; or any bridge whose existence check
|
|
839
838
|
returns no `totalRecordCount` (see the ACL shape below). Consistent with the 2026-08-27 fail-loud
|
|
840
|
-
policy.
|
|
839
|
+
policy. **A stale transfer-order Invoice no longer freezes the run — it is now DELETED up front
|
|
840
|
+
(2026-09-21, below);** the old per-line `InvoiceItems` throw is kept only as a safety net.
|
|
841
841
|
|
|
842
842
|
**Bridge direction is PO-FIRST, confirmed from NYCHH data rather than guessed.** The naming rule is
|
|
843
843
|
`<source>_<created from it>`. `Client_Nychh` populates `PurchaseOrderItems_TransferOrderItems` (98
|
|
@@ -922,6 +922,56 @@ actually have to write code for** — the schema alone would have sent us after
|
|
|
922
922
|
| `PurchaseOrders_TransferOrders` | 17 → **104** |
|
|
923
923
|
| `PurchaseOrderItems_TransferOrderItems` | 28 → **163** |
|
|
924
924
|
|
|
925
|
+
#### ⚠ A stale transfer-order Invoice froze SALES_ORDERS — the retirement now DELETES it up front (fixed + deployed + prod-verified 2026-09-21, NYCHH)
|
|
926
|
+
|
|
927
|
+
The retirement's old "throw on any `InvoiceItems`" rule froze NYCHH `SALES_ORDERS` at `1-RUNNING`
|
|
928
|
+
(`Logs.Issue` **813**, `clientId 3`). NetSuite order **4680233** (number **172445**) was reclassified
|
|
929
|
+
`SalesOrder`→`TransferOrder`; the stale `SalesOrder` **65** carried `Invoice` **297781** (id **1143**).
|
|
930
|
+
`Invoices`/`InvoiceItems` have **no `transferOrderId`/`transferOrderItemId`**, so the retirement could
|
|
931
|
+
not move them and threw *"resolve manually"* every run.
|
|
932
|
+
|
|
933
|
+
**Team decision (jcardinal): a transfer-order's invoices are NOT imported, so a stale one is deleted,
|
|
934
|
+
not preserved.** New `private static deleteStaleTransferOrderInvoicesForSalesOrder()` runs **first** in
|
|
935
|
+
the retirement (before the line/header moves):
|
|
936
|
+
|
|
937
|
+
- Deletes, in FK order, the stale TO invoice's **`InvoiceItems` → `InvoiceTrackingNumbers` → the
|
|
938
|
+
`Invoice` header** — both children are FK **RESTRICT**, so children first.
|
|
939
|
+
- `Invoices` has **no soft-delete/void flag**, so it is a **hard delete**.
|
|
940
|
+
- **Throws (refuses)** if the invoice also bills a **different** SalesOrder (a shared invoice) — never
|
|
941
|
+
deletes another order's billing.
|
|
942
|
+
- The old per-line `InvoiceItems` throw is kept as a **safety net** for anything the up-front delete
|
|
943
|
+
didn't clear.
|
|
944
|
+
- All GETs use `recordsPerPage` 1000 + `isset($meta->totalRecordCount)` guards; fail-loud preserved.
|
|
945
|
+
|
|
946
|
+
Reviews: php-reviewer clean, cso SAFE TO SHIP, cto AGREE (cto caught the second FK child,
|
|
947
|
+
`InvoiceTrackingNumbers`). **Verified prod:** SalesOrder 65 + Invoice 1143 deleted, TransferOrder 1929
|
|
948
|
+
kept, `NETSUITE_EXECUTION_MODE_SALES_ORDERS` `1-RUNNING → 1-IDLE`, cursor advanced past `…|4680233`.
|
|
949
|
+
|
|
950
|
+
#### ⚠ The togasupply sync imports INVOICES ONLY — a credit memo froze INVOICES (fixed + deployed + prod-verified 2026-09-21, NYCHH)
|
|
951
|
+
|
|
952
|
+
NYCHH `INVOICES` froze at `1-RUNNING` (`Logs.Issue` **804**) because `syncInvoiceFromNetsuite` refused a
|
|
953
|
+
**credit memo** (NetSuite **4446850**, type `CustCred`). A credit memo's `createdFrom` is the **INVOICE
|
|
954
|
+
it credits, not a sales order** — chain: credit memo `4446850` → invoice `176645` (`CustInvc`) → sales
|
|
955
|
+
order `2603333`. `syncInvoiceFromNetsuite` only resolves `createdFrom` as a sales/transfer order or
|
|
956
|
+
RtnAuth, so a credit memo dead-ends and throws forever.
|
|
957
|
+
|
|
958
|
+
**Root cause: the INVOICES header fetch was pulling four transaction types.** `App_Api_Netsuite_Rest::listSales()`
|
|
959
|
+
returned `CustInvc`, `CashSale`, `CashRfnd`, `CustCred`; credit memos + cash refunds were never in scope
|
|
960
|
+
for the togasupply sync. **Team decision (jcardinal): the togasupply sync imports INVOICES ONLY.**
|
|
961
|
+
|
|
962
|
+
- **Fix (worker):** `common_sync_togasupply.php` (INVOICES section, ~L978) now calls `listSales()` with
|
|
963
|
+
`[App_Api_Netsuite_Rest::RECORD_TYPE_INVOICE]`.
|
|
964
|
+
- **Fix (library):** `listSales()` gained a trailing `array $recordTypes` param defaulting to all four
|
|
965
|
+
`RECORD_TYPE_*` constants (so the **forecast import** —
|
|
966
|
+
`worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php` +
|
|
967
|
+
`periodic_forecast_discrepancy_fix.php` — is unaffected via the default). Callers pass **`RECORD_TYPE_*`
|
|
968
|
+
constants, never raw SuiteQL codes**; `listSales` maps constant→code internally and **throws on an
|
|
969
|
+
unknown/empty type**. Detail on the [SuiteQL/REST shim](../../library/features/netsuite-suiteql-rest-shim.md).
|
|
970
|
+
|
|
971
|
+
php-reviewer clean (additive; forecast callers unaffected), sql-reviewer SAFE TO MERGE (allowlist
|
|
972
|
+
mapping, no injection). **Verified prod:** `NETSUITE_EXECUTION_MODE_INVOICES` `1-RUNNING → 1-IDLE`
|
|
973
|
+
(window still 1s; self-heals ×3 from 2024-02-22).
|
|
974
|
+
|
|
925
975
|
### ⚠ TWO code paths create `PurchaseOrderItems` — only one stamped `fulfillmentType` (fixed 2026-09-17)
|
|
926
976
|
|
|
927
977
|
`fulfillmentType` was stamped by `syncPurchaseOrderFromNetsuite()` (from the order-level
|
|
@@ -1283,7 +1333,9 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
|
|
|
1283
1333
|
**`lastmodifieddate`, NOT `trandate`** — `App_Api_Netsuite_Rest::listSales()` defaults
|
|
1284
1334
|
`$dateField='lastmodifieddate'` and the engine calls it without the 4th arg. So a backdated
|
|
1285
1335
|
`trandate` is irrelevant to whether an invoice imports, and a just-modified invoice still won't
|
|
1286
|
-
import until it is >24h old (the upper-bound cap).
|
|
1336
|
+
import until it is >24h old (the upper-bound cap). **The engine also now passes
|
|
1337
|
+
`[RECORD_TYPE_INVOICE]` (INVOICES only — 2026-09-21); credit memos / cash refunds are out of scope
|
|
1338
|
+
and a credit memo threw here — see the credit-memo freeze subsection above.**
|
|
1287
1339
|
- **⚠ DEPLOY ORDER: the cursor-rename migration MUST run on EVERY client DB before the worker/library
|
|
1288
1340
|
deploy — there is NO old-key fallback in code.** `Client/2026-09-02b - NetsuiteSyncCursorRename.sql`
|
|
1289
1341
|
renames `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` → `NETSUITE_LAST_SYNC_CURSOR_<SECTION>`. If the code
|
|
@@ -1995,6 +2047,15 @@ library (or vice versa) crashes GroWrk and Adyen on their next sync run.
|
|
|
1995
2047
|
[NetSuite Sync Alert Monitor](../../library/features/netsuite-sync-alert-monitor.md).
|
|
1996
2048
|
|
|
1997
2049
|
## Change history
|
|
2050
|
+
- 2026-09-21 — **Two NYCHH sync freezes fixed, deployed (library + worker together) and prod-verified.**
|
|
2051
|
+
(1) SALES_ORDERS froze (`Logs.Issue` 813) on a stale transfer-order invoice the retirement couldn't
|
|
2052
|
+
move (`Invoices` has no `transferOrderId`). New `deleteStaleTransferOrderInvoicesForSalesOrder()`
|
|
2053
|
+
deletes the stale TO invoice up front — `InvoiceItems` → `InvoiceTrackingNumbers` → header (both FK
|
|
2054
|
+
RESTRICT), hard delete (no void flag), throws only if the invoice also bills a different SalesOrder;
|
|
2055
|
+
team decision that TO invoices aren't imported. (2) INVOICES froze (`Logs.Issue` 804) on a credit
|
|
2056
|
+
memo whose `createdFrom` is the invoice it credits, not a sales order; the sync now imports INVOICES
|
|
2057
|
+
only — `listSales()` gained a `$recordTypes` param (defaults to all 4 for the forecast import) and the
|
|
2058
|
+
engine passes `[RECORD_TYPE_INVOICE]`. Both verified `1-RUNNING → 1-IDLE`. (jcardinal)
|
|
1998
2059
|
- 2026-09-18 — **Three corrections and one deployed fix.** (1) The stale-SalesOrder retirement
|
|
1999
2060
|
method shipped **missing four FK dependencies** and failed on each in turn: `ItemFulfillments`
|
|
2000
2061
|
**header** `salesOrderId` (all 218 orders), the **upstream** `PurchaseOrders_SalesOrders` header
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
| Doc | Summary |
|
|
4
4
|
|-----|---------|
|
|
5
|
-
| [API (api2 / TOGa API v2) Architecture](architecture.md) |
|
|
5
|
+
| [API (api2 / TOGa API v2) Architecture](architecture.md) | Backend for the public TOGa 2.0 API — metadata/ACL-driven REST JSON (`/v2`) + a cXML gateway, both over `_underscore` `_Model`; open for the response-envelope c |
|
|
6
6
|
| [API-Key Authentication (/v2/auth/api) and what a key can actually reach](features/api-key-authentication-and-scoping.md) | How a machine caller gets a bearer token from `POST /v2/auth/api` and exactly what that key can reach; open when scoping an API consumer or auditing what a key |
|
|
7
7
|
| [API Payload Interceptors (metadata-registered prePost/postPut model hooks)](features/api-payload-interceptors.md) | How `_Model` prePost/postPost/prePut/postPut hooks are dispatched by the api2 /v2 engine only when a metadata row exists — the missing-row silent no-op, the wro |
|
|
8
8
|
| [Multi-Client (Cross-Client) Data Retrieval](features/cross-client-data-retrieval.md) | The scatter-gather engine that returns one entitled user's records across many client DBs from a single GET (triggered by a `client` query option); open when wo |
|