toga-ai 1.0.563 → 1.0.565
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/library/features/netsuite-item-isfulfillable-sync.md +32 -4
- package/knowledge/1.0/apps/worker/INDEX.md +2 -2
- package/knowledge/1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md +140 -3
- package/knowledge/1.0/apps/worker/workflows/isfulfillable-multi-client-backfill.md +21 -1
- package/knowledge/1.0/apps/worker/workflows/onboarding-client-to-netsuite-togasupply-sync.md +95 -6
- package/knowledge/2.0/apps/_underscore/INDEX.md +1 -1
- package/knowledge/2.0/apps/_underscore/features/fulfillable-item-propagation.md +69 -6
- package/knowledge/INDEX.md +2 -2
- package/knowledge/clients/compass-canada/profile.md +13 -1
- package/knowledge/clients/compass-usa/INDEX.md +1 -0
- package/knowledge/clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md +187 -0
- package/knowledge/clients/compass-usa/profile.md +10 -1
- package/knowledge/clients/elite/INDEX.md +2 -1
- package/knowledge/clients/elite/features/netsuite-togasupply-sync.md +141 -0
- package/knowledge/clients/elite/features/togadesk-service-request-intake.md +16 -0
- package/knowledge/clients/elite/profile.md +24 -0
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@ project: Library
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-08-12
|
|
10
10
|
owners: [bala]
|
|
11
11
|
files:
|
|
12
12
|
- library/app/netsuite.php
|
|
@@ -18,6 +18,7 @@ related:
|
|
|
18
18
|
- ../../worker/features/netsuite-togasupply-per-client-sync.md
|
|
19
19
|
- ../../worker/workflows/isfulfillable-multi-client-backfill.md
|
|
20
20
|
- ../../../2.0/apps/_underscore/features/fulfillable-item-propagation.md
|
|
21
|
+
- ../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md
|
|
21
22
|
---
|
|
22
23
|
|
|
23
24
|
## Summary
|
|
@@ -85,9 +86,27 @@ physical goods from services — `itemtype` would, but the sync has never used `
|
|
|
85
86
|
`isSerialized`), so this feature intentionally reflects NetSuite's flag **as-is**. **Net effect:
|
|
86
87
|
nearly everything resolves to `fulfillable = 1`.**
|
|
87
88
|
|
|
88
|
-
- **Open product decision (
|
|
89
|
-
|
|
90
|
-
**
|
|
89
|
+
- **Open product decision (RESOLVED for Compass, 2026-08-12):** because services come back
|
|
90
|
+
fulfillable, the storefront makes everything clickable. A prod evidence review found that
|
|
91
|
+
`itemtype` **cannot confirm** fulfillability (physical types like DESKTOPS show 0% tracked) but a
|
|
92
|
+
**tracking ride-along test** cleanly disqualifies fee/software/miscellaneous lines. Compass adopted
|
|
93
|
+
a **type-derived override** of NetSuite's value — see
|
|
94
|
+
[Compass isFulfillable data quality & type rule](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md).
|
|
95
|
+
Read the write-time-guard gotcha below before implementing it anywhere.
|
|
96
|
+
|
|
97
|
+
## ⚠ CRITICAL — the existing-item refresh REVERTS any local override
|
|
98
|
+
`getCreateItem`'s refresh (added 2026-07-28) rewrites `isFulfillable` on **existing** items whenever
|
|
99
|
+
NetSuite's value differs from the stored value. NetSuite returns **`T` for service (`NonInvtPart`)
|
|
100
|
+
items**, so **any locally-derived `0` is flipped back to `1` on the next sync run**. A data migration
|
|
101
|
+
alone cannot hold an override — it silently reverts within a day.
|
|
102
|
+
|
|
103
|
+
Proven from the prod audit log: Compass items **2382 / 2384 / 2385** (`US-IOS-SSC-SHR`,
|
|
104
|
+
`US-IOS-VENUENEXT`, `US-IOS-VIRTUALMGR` — itemType `SERVICES`, assetType `CONSULTING`) were stamped
|
|
105
|
+
**NULL → 1 on 2026-08-04 13:56:00-01** by `apiId 1`, `userId NULL`, inside a 1–2 items/second cron
|
|
106
|
+
sweep that started 13:45 and set every item to `1`.
|
|
107
|
+
|
|
108
|
+
**Rule: any policy that overrides NetSuite's flag must be enforced at WRITE TIME** — a
|
|
109
|
+
`prePut`/`prePost` guard on Items in `_Model_Client_Item` — **in addition to** the backfill.
|
|
91
110
|
|
|
92
111
|
## Verification performed
|
|
93
112
|
- **Prod read-only dry-run:** 20 reachable Compass items; chain walk correct
|
|
@@ -106,6 +125,13 @@ nearly everything resolves to `fulfillable = 1`.**
|
|
|
106
125
|
in `Client_Compass.Apis` (name `Agilant`) — never reproduce the secret value.
|
|
107
126
|
|
|
108
127
|
## Change history
|
|
128
|
+
- 2026-08-12 — Prod investigation (no code change): recorded that the **existing-item refresh
|
|
129
|
+
reverts any local override** (NetSuite returns `T` for services; audit-log proof on Compass items
|
|
130
|
+
2382/2384/2385, stamped NULL→1 on 2026-08-04), so an override must be enforced by a write-time
|
|
131
|
+
`prePut`/`prePost` guard, not a migration. Closed the long-standing **open product decision** for
|
|
132
|
+
Compass — item type cannot *confirm* fulfillability, but the tracking ride-along test disqualifies
|
|
133
|
+
fee/software/miscellaneous lines; Compass adopted a type-derived override (see the client doc).
|
|
134
|
+
(bala)
|
|
109
135
|
- 2026-07-28 — Code-review hardening: (1) `getCreateItem` now **refreshes `isFulfillable` on
|
|
110
136
|
existing items** when NetSuite's value differs (not create-only), mirroring the serialized flag;
|
|
111
137
|
diff-only PUT keeps re-syncs no-op. (2) The two other `/items` creation paths
|
|
@@ -131,3 +157,5 @@ nearly everything resolves to `fulfillable = 1`.**
|
|
|
131
157
|
- [Phase 2 — isFulfillable propagation up the SO↔PO chain (2.0)](../../../2.0/apps/_underscore/features/fulfillable-item-propagation.md).
|
|
132
158
|
- [isFulfillable multi-client backfill (worker)](../../worker/workflows/isfulfillable-multi-client-backfill.md)
|
|
133
159
|
— the cross-client catch-up cron and its client-DB / catalog-matching gotchas.
|
|
160
|
+
- [Compass isFulfillable — Data Quality & the Type-Derived Rule](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md)
|
|
161
|
+
— measured prod state, the structural reach limit, and the Compass override + its write-time guard.
|
|
@@ -7,10 +7,10 @@
|
|
|
7
7
|
| [Compass Partial In-Transit & Delivered Emails (per package)](features/compass-partial-in-transit-delivered-emails.md) | Compass USA and Compass Canada send a **per-package** in-transit email (and a matching delivered email) instead of one email listing the whole order. | worker/crons/toga2/compass/update_salesorder_status_from_odp.php, worker/crons/toga2/compasscanada/update_salesorder_status_from_odp.php, worker/crons/toga2/compasscanada/workflow/3_update_salesorder_status_from_grand_and_toy.php, worker/crons/toga2/compasscanada/send_delivered_email.php, worker/crons/toga2/compasscanada/workflow/test_partial_in_transit_email.php, worker/crons/toga2/compasscanada/workflow/test_partial_delivered_email.php, library/app/client/compasscanada.php |
|
|
8
8
|
| [Elite TOGA 2.0 → TOGaDeskSupport Standalone Attachment Sync](features/elite-togadesk-attachment-sync.md) | `sync_togadesk_elite_attachments.php` is a standalone cron (every 5 minutes) that syncs file attachments from TOGA 2.0 into TOGaDeskSupport for Elite. | worker/crons/toga2/elite/sync_togadesk_elite_attachments.php, worker/crons/toga2/elite/test_sync_togadesk_elite_attachments.php |
|
|
9
9
|
| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/checker.php, worker2/Component/Forecast/SaleImport/SaleImport.php, test/@dave/looper.php, test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/reconcile_drift_2023plus.php, test/@dave/probe_invoice_gap_2026.php, test/@dave/probe_creditmemo_gap_detail.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
|
|
10
|
-
| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, library/app/api/netsuite/rest.php, library/app/systemmonitor/netsuiteintegration.php |
|
|
10
|
+
| [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/crons/toga2/netsuite/sync_togasupply_elite.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, library/app/api/toga2.php, library/app/api/netsuite/rest.php, library/app/framework.php, library/app/systemmonitor/netsuiteintegration.php, test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php, test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php |
|
|
11
11
|
| [OneUptime Server monitor + disk/memory hygiene on the 1.0 worker EB host](features/oneuptime-server-monitor-host-hygiene.md) | The 1.0 `agilant-worker` EB environment runs on the **legacy Amazon Linux 1 PHP 7.2 platform** (Apache httpd/prefork, s3fs mounts, cron) and repeatedly went dow | worker/.ebextensions/040_disk_memory_hygiene.config, worker/.ebextensions/045_oneuptime_agent.config, worker/ebs/cron.worker.php, worker/ebs/mount-s3fs-folders.php, worker/ebs/apache_settings.php, worker/ebs/setup_phpini.php |
|
|
12
12
|
| [OneUptime external uptime monitoring for 1.0 workers](features/oneuptime-worker-uptime-monitoring.md) | Every 1.0 worker box self-reports its liveness to an external OneUptime monitor once per minute by curl-POSTing to a per-worker "Incoming Request" heartbeat URL | library/app/worker.php, worker/crons/worker/worker_heartbeat.php |
|
|
13
13
|
| [Prudential: Send Shipments for the Day report (daily cron)](features/send-shipments-for-the-day.md) | Daily cron (9:00 PM) that emails Prudential and Dell stakeholders an Excel report of all devices shipped that day, including tracking number, serial number, emp | worker/crons/notifications/reports/send_shipments_for_the_day.php |
|
|
14
14
|
| [Diagnosing frozen 1.0 worker cron check-ins (Sentry "missed" flood)](workflows/diagnosing-frozen-cron-checkins.md) | When 1.0 worker cron timestamps freeze and Sentry project `worker1` fills with **`missed`** check-ins, the intuitive diagnosis — a wedged `App_Framework::isProc | worker/.ebextensions/cron.config, library/app/worker.php |
|
|
15
15
|
| [isFulfillable Multi-Client Backfill (all togasupply clients)](workflows/isfulfillable-multi-client-backfill.md) | One-time backfill that catches up `Items.isFulfillable` on **existing** items across **all 17 togasupply clients** (AIG, Broward Sheriff, Canon, Endeavor Health | worker/crons/toga2/netsuite/backfill_isfulfillable_all_clients.php, library/app/api/toga2.php |
|
|
16
|
-
| [Onboarding a Client to the NetSuite TOGa Supply Sync](workflows/onboarding-client-to-netsuite-togasupply-sync.md) | How to add a new TOGa 2 client to the per-client NetSuite → TOGa Supply importer (`worker/crons/toga2/netsuite/`). | worker/crons/toga2/netsuite/sync_togasupply.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql |
|
|
16
|
+
| [Onboarding a Client to the NetSuite TOGa Supply Sync](workflows/onboarding-client-to-netsuite-togasupply-sync.md) | How to add a new TOGa 2 client to the per-client NetSuite → TOGa Supply importer (`worker/crons/toga2/netsuite/`). | worker/crons/toga2/netsuite/sync_togasupply.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_elite.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, dbchanges2/_modules/netsuite/2026-08-05 - CLEAN NETSUITE CLINET.SQL |
|
|
@@ -6,20 +6,27 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-08-
|
|
10
|
-
owners: ["dfranks", "bala", "jcardinal", "mhammontree"]
|
|
9
|
+
updated: 2026-08-12
|
|
10
|
+
owners: ["dfranks", "bala", "jcardinal", "mhammontree", "snaredla"]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/netsuite/common_sync_togasupply.php
|
|
13
13
|
- worker/crons/toga2/netsuite/sync_togasupply_canon.php
|
|
14
|
+
- worker/crons/toga2/netsuite/sync_togasupply_elite.php
|
|
14
15
|
- worker/schedules/cron.worker.sync.json
|
|
15
16
|
- dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql
|
|
17
|
+
- library/app/api/toga2.php
|
|
16
18
|
- library/app/api/netsuite/rest.php
|
|
19
|
+
- library/app/framework.php
|
|
17
20
|
- library/app/systemmonitor/netsuiteintegration.php
|
|
21
|
+
- test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php
|
|
22
|
+
- test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php
|
|
18
23
|
related:
|
|
19
24
|
- ../architecture.md
|
|
20
25
|
- forecast2-netsuite-reconciliation.md
|
|
26
|
+
- ../workflows/onboarding-client-to-netsuite-togasupply-sync.md
|
|
21
27
|
- ../../library/features/toga2-api-client-and-bridge.md
|
|
22
28
|
- ../../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
|
|
29
|
+
- ../../../clients/elite/features/netsuite-togasupply-sync.md
|
|
23
30
|
---
|
|
24
31
|
|
|
25
32
|
## Summary
|
|
@@ -54,11 +61,61 @@ through the public TOGa2 API (a 1.0↔2.0 bridge).
|
|
|
54
61
|
Each section (sales orders, POs, invoices, item receipts, item fulfillments, inventory
|
|
55
62
|
adjustments) uses **adaptive time-windowing** via `startModeIteration()`/`finishModeIteration()`:
|
|
56
63
|
a window stored as `<seconds>-<STATE>` in the client's `Parameters` table shrinks ÷3 when the
|
|
57
|
-
prior run was left `RUNNING` (didn't finish) and grows ×3 (capped at
|
|
64
|
+
prior run was left `RUNNING` (didn't finish) and grows ×3 (capped at
|
|
65
|
+
`MAX_TIME_WINDOW_TO_FETCH_FROM_NETSUITE_SECONDS`, default 432000s = 5 days) when it
|
|
58
66
|
finished `IDLE`. The window state is written `RUNNING` at the **start** of each section, so a
|
|
59
67
|
crash leaves it visibly RUNNING and the next run backs off. `LAST_SYNC_DATETIME_*` advances to
|
|
60
68
|
the window's upper bound after each successful section.
|
|
61
69
|
|
|
70
|
+
### The window cap is per-client overridable (2026-08-12)
|
|
71
|
+
|
|
72
|
+
`common_sync_togasupply.php:3` is
|
|
73
|
+
`defined('MAX_TIME_WINDOW_TO_FETCH_FROM_NETSUITE_SECONDS') || define(…, 432000)` — **not** a
|
|
74
|
+
`const` — so a wrapper may `const MAX_TIME_WINDOW_TO_FETCH_FROM_NETSUITE_SECONDS = …;`
|
|
75
|
+
*before* its `require_once` to widen its own catch-up window. Only Elite does (2592000 = 30 days,
|
|
76
|
+
while backfilling from 2024); the other 17 wrappers inherit the 5-day default. Use this only for a
|
|
77
|
+
deep backfill and put the reason in a comment next to it — at 5-day windows on a 5-minute cron,
|
|
78
|
+
~858 days of history takes days of wall-clock to catch up; at 30 days it took ~90 minutes.
|
|
79
|
+
|
|
80
|
+
### ⚠ A shrinking window is a BUG SIGNAL, not a load signal
|
|
81
|
+
|
|
82
|
+
The single most expensive misreading of this engine. `startModeIteration()` writes
|
|
83
|
+
`<seconds>-RUNNING` at the **start** of every section and `finishModeIteration()` writes `-IDLE` at
|
|
84
|
+
the end; the ÷3 `REDUCE_INTERVAL_FACTOR` shrink fires only because the *previous* run never reached
|
|
85
|
+
its IDLE write. **There is no timeout, memory cap, or resource limit anywhere in the script** — so
|
|
86
|
+
"didn't finish" can only mean **it threw**. A collapsing window therefore means *go read Sentry*,
|
|
87
|
+
never *tune the window size down*. (Corollary: a window that has grown back to the cap is positive
|
|
88
|
+
evidence the section is completing.)
|
|
89
|
+
|
|
90
|
+
### ⚠ A per-record API failure does NOT stop the cursor — the record is skipped forever
|
|
91
|
+
|
|
92
|
+
The cursor write (`LAST_SYNC_DATETIME_*`) and `finishModeIteration()` both run **unconditionally
|
|
93
|
+
after** the record loop. A record that throws mid-loop is captured to Sentry and **stepped over**,
|
|
94
|
+
while the section still reports success and advances the cursor past its window. So
|
|
95
|
+
**"section IDLE + cursor advancing" is not proof of a complete import** — it only proves the run
|
|
96
|
+
didn't die.
|
|
97
|
+
|
|
98
|
+
Proven concretely on Elite: sales order **169441** (NetSuite internal id 4624062, trandate
|
|
99
|
+
2024-04-15, customer PO `BEGGS/04152024/…`) was simply absent from `Client_Elite.SalesOrders`
|
|
100
|
+
because its window ran ~15 minutes before the missing-vendor fix (below) landed.
|
|
101
|
+
|
|
102
|
+
**Recovery: rewind the section's cursor.** Set `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` in the
|
|
103
|
+
client's `Parameters` table back before the lost window. Re-import is **idempotent** — records match
|
|
104
|
+
on `c_netsuiteInternal*Id` and `SalesOrders.c_netsuiteInternalSalesOrderId` carries a unique index —
|
|
105
|
+
so a rewind re-PUTs what is already there and POSTs only what is missing.
|
|
106
|
+
|
|
107
|
+
### Sections are NOT order-dependent — one cursor can be rewound alone
|
|
108
|
+
|
|
109
|
+
Each section's `startModeIteration()` reads only **its own** `Parameters` key, and the PO section
|
|
110
|
+
resolves its own sales order: `syncPurchaseOrderFromNetsuite()` calls
|
|
111
|
+
`syncSalesOrderFromNetsuite()` itself (`library/app/api/toga2.php:1813`) when a PO has an
|
|
112
|
+
originating order, and the link is conditional (`if ($salesOrder)` at :2221) so a PO whose order
|
|
113
|
+
cannot be resolved still imports. Consequences worth knowing before you "fix" a skew:
|
|
114
|
+
|
|
115
|
+
- A purchase-orders cursor running 50 days **ahead** of sales-orders is safe, not a defect.
|
|
116
|
+
- Rewinding one section without touching the others is safe.
|
|
117
|
+
- Which is also why the per-section operator script below is a legitimate tool rather than a hack.
|
|
118
|
+
|
|
62
119
|
### Adding a new client (the full recipe)
|
|
63
120
|
|
|
64
121
|
A wrapper + schedule alone is **not enough** — the client must also be provisioned in three
|
|
@@ -127,6 +184,30 @@ them provisioned: `/customers`→`c_netsuiteInternalCustomerId`; `/countries`→
|
|
|
127
184
|
`/shipping-methods`→`c_netsuiteInternalShipMethodId`. The names are a **convention**, not
|
|
128
185
|
discovered per-client.
|
|
129
186
|
|
|
187
|
+
### Manual operator scripts (built for Elite, reusable pattern)
|
|
188
|
+
|
|
189
|
+
Two CLI scripts written for the Elite backfill live under
|
|
190
|
+
`test/@srija/Elite Testing/Service Requests/` (they are operator tools, **not** scheduled crons —
|
|
191
|
+
do not add them to `cron.worker.sync.json`):
|
|
192
|
+
|
|
193
|
+
- **`test_sync_togasupply_elite_section.php`** — runs **one** section by enabling only its
|
|
194
|
+
`IS_ENABLED_INTEGRATION_*` flag with `define()` instead of `const`, so a slow section can be
|
|
195
|
+
backfilled without the earlier sections eating the run. **It is NOT a dry run — it writes the real
|
|
196
|
+
cursors.**
|
|
197
|
+
- **`test_diagnose_togasupply_elite.php`** — **read-only** per-record IMPORT/SKIP match reporter for
|
|
198
|
+
a given section and date range. It deliberately does **not** call
|
|
199
|
+
`App_Framework::cronInitialization()` (that takes the process lock and exits silently, and writes a
|
|
200
|
+
`CronJobExecutions` row) and it drains/disables output buffering so `echo` actually reaches the
|
|
201
|
+
terminal.
|
|
202
|
+
|
|
203
|
+
**⚠ A manual script does not mutually exclude with the scheduled cron.**
|
|
204
|
+
`App_Framework::isProcessRunning()` (`library/app/framework.php:833-842`) defaults to
|
|
205
|
+
`$_SERVER['SCRIPT_FILENAME']` and greps `ps -ef` for **its own filename** — so a per-section script
|
|
206
|
+
with a different filename runs happily *alongside* `sync_togasupply_<client>.php`. Both then race the
|
|
207
|
+
same `Parameters` cursors: whoever writes last wins and the other's window is silently skipped
|
|
208
|
+
(and windows shrink). Any manual backfill must either pass the cron's filename to
|
|
209
|
+
`isProcessRunning()` explicitly or be run with the cron's schedule entry disabled.
|
|
210
|
+
|
|
130
211
|
## Data model
|
|
131
212
|
|
|
132
213
|
Per-client `Parameters` table (2.0 client DB, e.g. `Client_Canon.Parameters`; columns
|
|
@@ -149,6 +230,44 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
|
|
|
149
230
|
|
|
150
231
|
## Gotchas / known issues
|
|
151
232
|
|
|
233
|
+
- **A client with NEITHER a manufacturer NOR a catalog on its items used to crash the whole run
|
|
234
|
+
(fixed 2026-08-12).** `$lookupItemByClientUuidAndPartNumberUpper[$uuidClient]` and
|
|
235
|
+
`$lookupAllCatalogItemsByClientUuidAndPartNumberUpper[$uuidClient]` were only keyed **inside** the
|
|
236
|
+
items loop, so a sparse client never got the key created at all. Every `sync*FromNetsuite()` takes
|
|
237
|
+
them as `array &`, so the unset key passed as **null** →
|
|
238
|
+
`TypeError: Argument 8 passed to App_Api_Toga2::syncSalesOrderFromNetsuite() must be of the type
|
|
239
|
+
array, null given`. Both are now initialised to `[]` at `common_sync_togasupply.php:74-75`, before
|
|
240
|
+
the loop. This was **shared** code — the latent bug was reachable by any sparse client and Elite's
|
|
241
|
+
three initial items (both fields null) merely exposed it. Lesson for this engine: any per-client
|
|
242
|
+
lookup passed by reference must be initialised **outside** the loop that fills it.
|
|
243
|
+
- **Countries are auto-created from the NetSuite shipping address, with a malformed code and name.**
|
|
244
|
+
`library/app/api/toga2.php:696-730` (and a duplicate at :1868-1905): when
|
|
245
|
+
`$nsShippingAddress->country` is not in `$lookupCountryByNetsuiteCountry`, the sync **POSTs
|
|
246
|
+
`/countries`**, deriving the name by splitting on non-lowercase characters and the code as
|
|
247
|
+
`substr(word1,0,1) . substr(word2,1,1)`. NetSuite's **leading underscore is never stripped**, so
|
|
248
|
+
`_unitedKingdom` → name `_united Kingdom` / code `_i`; `_canada` → `_canada` / `_C`;
|
|
249
|
+
`_australia` → `_australia` / `_A`. **Codes can collide.** An address with an *empty* country sets
|
|
250
|
+
`$isInvalidAddress = true` instead. The rows then surface in any UI that shows a country.
|
|
251
|
+
**Remedy: pre-seed `Countries`** with the correct `code`/`name` **and** the exact
|
|
252
|
+
`c_netsuiteCountry` string before the first import — see the onboarding workflow.
|
|
253
|
+
Note the direction: **NetSuite → TOGa only.** Nothing in this cron ever writes to NetSuite (every
|
|
254
|
+
NetSuite call is a `list*`/`fetch*`), and NetSuite's country list is a fixed system list that
|
|
255
|
+
cannot be extended, so there is nothing to fix on the NetSuite side.
|
|
256
|
+
- **Item receipts are matched by the originating PO's customer, NOT by warehouse location.**
|
|
257
|
+
`common_sync_togasupply.php:844-905` uses the warehouse location ids only to **narrow** the SuiteQL
|
|
258
|
+
query; interest is decided from the receipt's `createdFrom` purchase order — its shipTo customer,
|
|
259
|
+
falling back to the custom field `NETSUITE_CUSTOM_FIELD_ID__END_USER_CUSTOMER`. So: a receipt from
|
|
260
|
+
a **stock PO** with no shipTo and no end-customer cannot be attributed to any client and is
|
|
261
|
+
skipped; and because the location filter is shared, receipts belonging to **other** clients at a
|
|
262
|
+
shared warehouse are returned and correctly skipped. Modelling receipts as "location-matched"
|
|
263
|
+
produces wrong diagnostics.
|
|
264
|
+
- **`Units.locationId` is never populated by this sync — transaction location lives on the item
|
|
265
|
+
rows.** All 229 `Client_Elite.Units` have `locationId` NULL. `locationId` exists only on `Units`
|
|
266
|
+
and `InventoryAdjustmentItems` (not on the `*ItemUnits` bridge tables), and
|
|
267
|
+
`InventoryAdjustmentItems.locationId` **is** correctly populated (Elite: 6 rows at NetSuite
|
|
268
|
+
location 196 "Elite"/Chicago, 1 at 5 "New York"). Aggregate `_qtyOnHand` derives from
|
|
269
|
+
`ItemReceiptItems` joined through `PurchaseOrderItems` → `VendorItems`, not from
|
|
270
|
+
`Units.locationId`. When building an inventory view, read location from the **transaction items**.
|
|
152
271
|
- **⚠ OPEN, UNTRACKED (found 2026-08-11): the GroWrk item-receipt sync has been failing every 5
|
|
153
272
|
minutes since 2026-08-06.** `Logs.Issue` **124** for `clientId` **33**, *"Invalid API Response"* from
|
|
154
273
|
`App_Api_Toga2::send()` (`library/app/api/toga2.php:3245`) via
|
|
@@ -245,6 +364,24 @@ Parameters are stored **per client DB** but accessed **through the TOGa2 API**,
|
|
|
245
364
|
|
|
246
365
|
## Change history
|
|
247
366
|
|
|
367
|
+
- 2026-08-12 — TRUE-80499 (Elite onboarding, shared-engine work). **Fixed** the sparse-client
|
|
368
|
+
`TypeError`: the two per-client item lookups are now initialised at
|
|
369
|
+
`common_sync_togasupply.php:74-75` instead of only inside the items loop, so a client whose items
|
|
370
|
+
have neither a manufacturer nor a catalog no longer passes `null` into an `array &` parameter.
|
|
371
|
+
**Built** the per-client window override (`MAX_TIME_WINDOW_TO_FETCH_FROM_NETSUITE_SECONDS` is now
|
|
372
|
+
`defined()||define()`, so a wrapper can widen its catch-up window; Elite uses 30 days, the other 17
|
|
373
|
+
stay at 5). Recorded four corrections that cost real diagnostic time: a **shrinking window is a bug
|
|
374
|
+
signal, not a load signal** (nothing in the script has a timeout — "didn't finish" means "threw");
|
|
375
|
+
a **per-record failure does not stop the cursor**, so the record is skipped permanently while the
|
|
376
|
+
section still reports IDLE and advances (recovery = rewind `NETSUITE_LAST_SYNC_DATETIME_<SECTION>`;
|
|
377
|
+
re-import is idempotent); **sections are independent** so one cursor can be rewound alone and a
|
|
378
|
+
skewed PO cursor is not a defect; and **item receipts are attributed via the `createdFrom` PO's
|
|
379
|
+
customer**, not by warehouse location. Also recorded the **malformed auto-created countries**
|
|
380
|
+
(`_unitedKingdom` → code `_i`), that `Units.locationId` is never populated by this sync, and the two
|
|
381
|
+
Elite operator scripts plus the `isProcessRunning()` own-filename trap that lets a manual script
|
|
382
|
+
race the scheduled cron. Client-specific results in
|
|
383
|
+
[Elite NetSuite → TOGa Supply sync](../../../clients/elite/features/netsuite-togasupply-sync.md).
|
|
384
|
+
(snaredla)
|
|
248
385
|
- 2026-08-11 — TRUE-80824 (side finding, no code change): recorded an **open, untracked** failure —
|
|
249
386
|
GroWrk (`clientId` 33) `syncItemReceiptFromNetsuite` has raised "Invalid API Response" from
|
|
250
387
|
`App_Api_Toga2::send()` every 5 minutes since 2026-08-06 (`Logs.Issue` 124, ~288 events/day).
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-08-12
|
|
10
10
|
owners: [bala]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/netsuite/backfill_isfulfillable_all_clients.php
|
|
@@ -16,6 +16,7 @@ related:
|
|
|
16
16
|
- ../../library/features/netsuite-item-isfulfillable-sync.md
|
|
17
17
|
- ../../../2.0/apps/_underscore/features/fulfillable-item-propagation.md
|
|
18
18
|
- ../../../clients/compass-usa/profile.md
|
|
19
|
+
- ../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md
|
|
19
20
|
---
|
|
20
21
|
|
|
21
22
|
## Summary
|
|
@@ -46,6 +47,20 @@ recursive up-chain propagation** — it never writes the DB directly and never w
|
|
|
46
47
|
beta/sandbox endpoint.
|
|
47
48
|
|
|
48
49
|
## Gotchas / known issues
|
|
50
|
+
- **⚠⚠ Re-running this backfill CANNOT close the remaining NULLs — they are structurally
|
|
51
|
+
unreachable (verified prod, 2026-08-12).** The cron only re-PUTs **Agilant source** items and
|
|
52
|
+
relies on the api2 interceptor to walk up the chain, which requires the client PO to have an
|
|
53
|
+
Agilant-tier SO behind it (a `PurchaseOrderItems_SalesOrderItems` row). **Vendor-direct POs have
|
|
54
|
+
none.** In prod `Client_Compass`, **zero** items with `isFulfillable IS NULL` reach the Agilant
|
|
55
|
+
tier (vs **52,990** line-level reaches for `isFulfillable = 1` items). Per vendor — items ordered /
|
|
56
|
+
reaching the Agilant tier: OFFICE DEPOT **497/124**; STRATEGIC SYSTEMS **66/0**; FREEDOM GROUPS LLC
|
|
57
|
+
**61/0**; PRESIDIO **45/0**; COMPASS GROUP **35/0**; ENCOMPASS SUPPLY CHAIN **2/0**. Only ODP
|
|
58
|
+
partially routes through Agilant NetSuite. **Do not schedule another run expecting these to fill
|
|
59
|
+
in** — closing the gap needs a different data source (see the
|
|
60
|
+
[Compass data-quality doc](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md)).
|
|
61
|
+
- **⚠ Compass Canada was never covered by this run.** `Client_CompassCanada` is **220 items, 100%
|
|
62
|
+
NULL** as of 2026-08-12 — the storefront Qty Fulfilled cell is dead across the entire Canada
|
|
63
|
+
tenant. Canada is not in the 17-client list above and has never been stamped at all.
|
|
49
64
|
- **Agilant catalog is matched by NAME, not id.** `Catalogs.name = 'Agilant'` is stable across every
|
|
50
65
|
client, but the `catalogId` **differs per client** — **Quad = 2, all other clients = 1** (verified
|
|
51
66
|
across all 17 client DBs). Hardcoding `catalogId = 2` would silently hit only Quad and miss the
|
|
@@ -73,6 +88,11 @@ recursive up-chain propagation** — it never writes the DB directly and never w
|
|
|
73
88
|
client backfilled from all its Agilant items.
|
|
74
89
|
|
|
75
90
|
## Change history
|
|
91
|
+
- 2026-08-12 — Prod audit of the post-run state: recorded that the remaining NULLs are
|
|
92
|
+
**structurally unreachable by this cron** (vendor-direct POs have no Agilant-tier SO for the
|
|
93
|
+
interceptor to walk; 0 NULL Compass items reach the Agilant tier, with per-vendor reach counts),
|
|
94
|
+
and that **Compass Canada was never covered** (220 items, 100% NULL). Post-run Compass state:
|
|
95
|
+
770 NULL / 25 zero / 697 one, with 12,318 order lines fulfilled but flagged NULL. (bala)
|
|
76
96
|
- 2026-07-28 — Built and ran the multi-client `backfill_isfulfillable_all_clients.php`: direct
|
|
77
97
|
client-DB scoping of Agilant-catalog items (matched by `Catalogs.name`, not id) keyed by
|
|
78
98
|
`c_netsuiteInternalItemId`, NetSuite reads by internal id (cached across clients), re-PUT to api2 so
|
package/knowledge/1.0/apps/worker/workflows/onboarding-client-to-netsuite-togasupply-sync.md
CHANGED
|
@@ -6,14 +6,19 @@ project: Worker
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: workflow
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
10
|
-
owners: ["dfranks"]
|
|
9
|
+
updated: 2026-08-12
|
|
10
|
+
owners: ["dfranks", "snaredla"]
|
|
11
11
|
files:
|
|
12
12
|
- worker/crons/toga2/netsuite/sync_togasupply.php
|
|
13
13
|
- worker/crons/toga2/netsuite/common_sync_togasupply.php
|
|
14
|
+
- worker/crons/toga2/netsuite/sync_togasupply_elite.php
|
|
14
15
|
- worker/schedules/cron.worker.sync.json
|
|
15
16
|
- dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql
|
|
16
|
-
|
|
17
|
+
- dbchanges2/_modules/netsuite/2026-08-05 - CLEAN NETSUITE CLINET.SQL
|
|
18
|
+
related:
|
|
19
|
+
- ../features/netsuite-togasupply-per-client-sync.md
|
|
20
|
+
- ../../../clients/elite/features/netsuite-togasupply-sync.md
|
|
21
|
+
- ../../../2.0/apps/dbchanges2/workflows/client-schema-drift-audit.md
|
|
17
22
|
---
|
|
18
23
|
|
|
19
24
|
## Summary
|
|
@@ -49,6 +54,71 @@ API (`App_Api_Toga2::send`), which lands in that client's `Client_<Name>` databa
|
|
|
49
54
|
5. **Deploy order matters:** the Parameters seed must hit prod `Client_<Name>` **before or
|
|
50
55
|
with** the cron going live, or the sync 404-aborts (and throws a Sentry error) every 5
|
|
51
56
|
minutes.
|
|
57
|
+
6. **Seed the shared synthetic "Agilant" vendor row** — see the dedicated section below. The
|
|
58
|
+
`netsuite` dbchanges2 module does **not** do it, and without it every order carrying a customer PO
|
|
59
|
+
number fails.
|
|
60
|
+
7. **Grant `AclCustomFieldPermissions` for the module's custom fields** — also below. The `netsuite`
|
|
61
|
+
module registers `CustomRecordFields` rows and adds the `c_` columns but grants **no field-level
|
|
62
|
+
ACL**, so api2 answers `403 EV-2`.
|
|
63
|
+
8. **Pre-seed `Countries`** for every country the client ships to, with the correct `code`/`name`
|
|
64
|
+
**and** the exact NetSuite `c_netsuiteCountry` string (`_unitedKingdom`, `_canada`, …). If you
|
|
65
|
+
skip this, the sync auto-creates the row from the NetSuite string and derives a **malformed**
|
|
66
|
+
name and a collision-prone 2-char code — mechanics and examples in the
|
|
67
|
+
[engine doc](../features/netsuite-togasupply-per-client-sync.md).
|
|
68
|
+
9. **Widen the window only if you are backfilling deep history.** A wrapper may
|
|
69
|
+
`const MAX_TIME_WINDOW_TO_FETCH_FROM_NETSUITE_SECONDS = …;` before the `require_once` (Elite: 30
|
|
70
|
+
days while backfilling from 2024). At the 5-day default on a 5-minute cron, ~2.4 years of history
|
|
71
|
+
takes days to catch up.
|
|
72
|
+
|
|
73
|
+
## The synthetic "Agilant" vendor must exist in the client's own DB (found 2026-08-06)
|
|
74
|
+
|
|
75
|
+
13 of the 18 wrappers pass the **same** vendor uuid
|
|
76
|
+
`10ed18ec-0999-53d9-f9db-5892d90f09d4` ("Agilant") as
|
|
77
|
+
`CLIENT_CONFIGURATION['importCustomerPurchaseOrdersOnVendor']` — but `Vendors` is a **per-client
|
|
78
|
+
table**, so that uuid has to exist as a row in **every** `Client_<Name>.Vendors`. **Nothing in
|
|
79
|
+
`dbchanges2/_modules/netsuite/` seeds it** (verified: the uuid appears in no file in the repo).
|
|
80
|
+
|
|
81
|
+
Symptom when it is missing: **every** sales order that carries a customer PO number (NetSuite
|
|
82
|
+
`otherRefNum`) fails `POST /purchase-orders` with **400 `EV-12` "unable to find a unique match"** on
|
|
83
|
+
the nested vendor. Orders without a customer PO import fine, so the failure looks partial and
|
|
84
|
+
data-dependent rather than structural. On Elite this produced **191 Sentry events in one day**
|
|
85
|
+
(`WORKER1-420`) and `PurchaseOrders` stuck at **0**; after seeding the row it went to **234**.
|
|
86
|
+
|
|
87
|
+
Seed it with a `dbchanges2/Client_<Name>/<YYYY-MM-DD><letter> - <Name>AgilantVendor.sql` inserting
|
|
88
|
+
the vendor with that **exact** uuid (reuse it — a fresh UUID defeats the whole shared-uuid pattern).
|
|
89
|
+
|
|
90
|
+
> **The undocumented pattern to internalise: a uuid shared across clients still needs a row per
|
|
91
|
+
> client database.** It reads like a global id and behaves like a per-tenant one.
|
|
92
|
+
|
|
93
|
+
## The `netsuite` module grants no field-level ACL (found 2026-08-06)
|
|
94
|
+
|
|
95
|
+
`dbchanges2/_modules/netsuite/2026-08-05 - CLEAN NETSUITE CLINET.SQL` inserts the
|
|
96
|
+
`CustomRecordFields` rows and adds the `c_` columns but contains **zero**
|
|
97
|
+
`AclCustomFieldPermissions` statements (`grep -c` = 0; the smaller
|
|
98
|
+
`2026-07-10a - UnitInventoryFields.sql` *does* grant, which is why the gap is easy to miss).
|
|
99
|
+
|
|
100
|
+
Symptom: api2 returns **403 `EV-2` "do not have the necessary permissions to read the specified
|
|
101
|
+
fields"** — a whole-request failure, not a silently dropped field.
|
|
102
|
+
|
|
103
|
+
Measured on Elite vs. a healthy client: **Elite 27 fields / 0 grants; Quad 26 / 25** grants to
|
|
104
|
+
**role 1 (Base), `isWritable = 1`**. Fixed with
|
|
105
|
+
`dbchanges2/Client_Elite/2026-08-06a - EliteNetsuiteCustomFieldAcl.sql`. Same failure class as the
|
|
106
|
+
GroWrk incidents — the general audit playbook is
|
|
107
|
+
[client schema-drift audit](../../../2.0/apps/dbchanges2/workflows/client-schema-drift-audit.md).
|
|
108
|
+
|
|
109
|
+
**Post-onboarding check, per client:** every `CustomRecordFields` row added by the module should have
|
|
110
|
+
exactly one grant. Count fields vs. grants and diff against a known-good client before declaring the
|
|
111
|
+
onboarding done.
|
|
112
|
+
|
|
113
|
+
## Recovering records the first run skipped
|
|
114
|
+
|
|
115
|
+
A record that throws mid-loop is logged to Sentry and **stepped over while the cursor still
|
|
116
|
+
advances** — so a clean-looking IDLE section can be missing records (see the engine doc). Recovery is
|
|
117
|
+
to rewind `NETSUITE_LAST_SYNC_DATETIME_<SECTION>` in the client's `Parameters` table to before the
|
|
118
|
+
bad window. Re-import is **idempotent** (matched on `c_netsuiteInternal*Id`;
|
|
119
|
+
`SalesOrders.c_netsuiteInternalSalesOrderId` has a unique index) and sections are independent, so one
|
|
120
|
+
cursor can be rewound alone. Expect to do this at least once on any onboarding where a provisioning
|
|
121
|
+
gap was found *after* the cron started running.
|
|
52
122
|
|
|
53
123
|
## Parameters seed (the required, easily-missed step)
|
|
54
124
|
The sync reads/writes per-client sync state via the TOGa2 API `/parameters` endpoint, which
|
|
@@ -103,11 +173,30 @@ the directory; no `USE`).
|
|
|
103
173
|
end-user matching. Always probe NetSuite first.
|
|
104
174
|
- **Parameters not seeded in prod** → sync throws every 5 min; surfaces as a `worker1` Sentry
|
|
105
175
|
error and zero imports.
|
|
106
|
-
-
|
|
107
|
-
the checkpoint does not advance past a throwing window, so it self-heals once the dependency
|
|
108
|
-
exists.
|
|
176
|
+
- **⚠ CORRECTED 2026-08-12 — a failing record does NOT self-heal.** This section previously said
|
|
177
|
+
"the checkpoint does not advance past a throwing window, so it self-heals once the dependency
|
|
178
|
+
exists." **That is wrong for a per-record failure.** The cursor write and `finishModeIteration()`
|
|
179
|
+
run unconditionally *after* the record loop, so a record that throws is Sentry-logged, stepped
|
|
180
|
+
over, and **permanently skipped** while the section reports success. Only a throw that escapes the
|
|
181
|
+
loop (setup/lookup phase, transport) leaves the checkpoint behind. Fix a provisioning gap and you
|
|
182
|
+
must **rewind the cursor** — see *Recovering records the first run skipped* above.
|
|
183
|
+
- **⚠ OPEN: `dbchanges2/_modules/netsuite/2026-08-05 - CLEAN NETSUITE CLINET.SQL` sits in the
|
|
184
|
+
SHARED module folder** and will run against **all 18 opted-in clients** on the next unscoped run.
|
|
185
|
+
It also carries a `CLINET` typo and **no letter suffix**, which the dbchanges2
|
|
186
|
+
`YYYY-MM-DD<letter>` contract requires. Know it is there before you trigger a module run.
|
|
109
187
|
|
|
110
188
|
## Change history
|
|
189
|
+
- 2026-08-12 — TRUE-80499 (Elite): added the **three provisioning steps this workflow was missing**,
|
|
190
|
+
each found by a production failure rather than by reading the module. (1) The shared synthetic
|
|
191
|
+
**"Agilant" vendor** uuid `10ed18ec-…f09d4` must be seeded as a **row in every**
|
|
192
|
+
`Client_<Name>.Vendors` — nothing in the `netsuite` module does it, and without it every order with
|
|
193
|
+
a customer PO number 400s `EV-12` on the nested vendor (Elite: 191 Sentry events/day, PurchaseOrders
|
|
194
|
+
0 → 234 after seeding). (2) The module registers `CustomRecordFields` and the `c_` columns but grants
|
|
195
|
+
**no `AclCustomFieldPermissions`** → 403 `EV-2` (Elite 27 fields / 0 grants vs. Quad 26 / 25 to role
|
|
196
|
+
1). (3) **Pre-seed `Countries`** or the sync auto-creates malformed rows from the NetSuite string.
|
|
197
|
+
Also added the per-client backfill-window override, a **cursor-rewind recovery** procedure, and
|
|
198
|
+
**corrected** this doc's claim that a throwing record self-heals — it does not, the cursor advances
|
|
199
|
+
past it. Flagged the unsuffixed shared `CLEAN NETSUITE CLINET.SQL` as an open hazard. (snaredla)
|
|
111
200
|
- 2026-06-16 — Documented the onboarding process after adding Quad (TRUE-79575): wrapper +
|
|
112
201
|
schedule + the required 12-key Parameters seed; captured the `isParentCustomer` NetSuite
|
|
113
202
|
probe and the local Client_/Logs_ schema requirement. (dfranks)
|
|
@@ -23,7 +23,7 @@
|
|
|
23
23
|
| [Record-Changed Event Publishing (_Event::publish to SQS)](features/event-publish-sqs.md) | `_Event::publish()` (in `_underscore/Event.php`) is the PHP side of the real-time event pipeline. | _underscore/Event.php |
|
|
24
24
|
| [FIELD_STORAGE fields — per-row lazy hydration and the platform-wide missing-column 500](features/field-storage-row-hydration.md) | `FIELD_STORAGE` is the 2.0 field type for blob-backed columns (S3 or local folder). | _underscore/Model.php, _underscore/Model/Client/TrackingNumber.php, _underscore/Model/Client/Invoice.php, dbchanges2/Core/HISTORIC/2024/2024-11b - item-fulfillments.sql |
|
|
25
25
|
| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | worker2/Component/Forecast/SaleImport/SaleImport.php, worker2/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/verify_je_classification.php, test/@dave/probe_je_accounts.php, test/@dave/probe_je_shape.php, test/@dave/fixer.php, test/@dave/Junk Drawer/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/Junk Drawer/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
|
|
26
|
-
| [isFulfillable Propagation Up the SO↔PO Chain](features/fulfillable-item-propagation.md) | `Items.isFulfillable` is a boolean that gates whether a storefront line's **Qty Fulfilled** cell is actionable. | _underscore/Model/Client/Item.php, _underscore/Model/Compass/Item.php, dbchanges2/Core/2026-07-17 - Items isFulfillable RecordField.sql, dbchanges2/Core/2026-07-17 - RegisterItemIsFulfillableInterceptors.sql, dbchanges2/Client/2026-07-17 - ItemsisFulfillable.sql |
|
|
26
|
+
| [isFulfillable Propagation Up the SO↔PO Chain](features/fulfillable-item-propagation.md) | `Items.isFulfillable` is a boolean that gates whether a storefront line's **Qty Fulfilled** cell is actionable. | _underscore/Model/Client/Item.php, _underscore/Model/Compass/Item.php, toga2-supply/src/pages/Orders/view/OrderView/components/sections/OrderItemsTableSection.tsx, dbchanges2/Core/2026-07-17 - Items isFulfillable RecordField.sql, dbchanges2/Core/2026-07-17 - RegisterItemIsFulfillableInterceptors.sql, dbchanges2/Client/2026-07-17 - ItemsisFulfillable.sql |
|
|
27
27
|
| [Item-Fulfillment Stage Lifecycle (picked/packed/shipped) & Order Status](features/item-fulfillment-stage-lifecycle-and-order-status.md) | Every ItemFulfillment (IF) now carries an explicit **stage** — picked → packed → shipped — resolved through `ItemFulfillmentStages → ItemFulfillmentStatuses` (m | _underscore/Model/Client/SalesOrder.php, _underscore/Model/Quad/SalesOrder.php, _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/SalesOrderStatus.php, _underscore/Model/Client/SalesOrderItem.php, _underscore/Model/Client/Item.php, _underscore/Model/Client/PurchaseOrderItem.php, library/app/api/toga2.php, dbchanges2/Client/2026-06-30a - BackfillNullStageItemFulfillmentsToShipped.sql, dbchanges2/Client/2026-06-30b - SalesOrderStatusesPickedPacked.sql, dbchanges2/Client/2026-06-30c - ItemFulfillmentStageIdNotNull.sql, dbchanges2/Client_CompassCanada/2026-06-30a - ItemFulfillmentLifecycleAndShippedBackfill.sql |
|
|
28
28
|
| [DB-free unit testing for _underscore model interceptors](features/model-interceptor-unit-testing.md) | `_underscore` shipped with **no** PHPUnit setup (no `composer.json`/`phpunit`; only vendored PhpOffice tests existed). | _underscore/Test/bootstrap.php, _underscore/Test/Prudential/ServiceRequestTest.php, test/@Bala/tests/netsuite_salesorder_payload_tests.php |
|
|
29
29
|
| [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php, _underscore/Model.php, _underscore/Model/Rate/Subscription.php |
|
|
@@ -6,11 +6,12 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-08-12
|
|
10
10
|
owners: [bala]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Client/Item.php
|
|
13
13
|
- _underscore/Model/Compass/Item.php
|
|
14
|
+
- toga2-supply/src/pages/Orders/view/OrderView/components/sections/OrderItemsTableSection.tsx
|
|
14
15
|
- dbchanges2/Core/2026-07-17 - Items isFulfillable RecordField.sql
|
|
15
16
|
- dbchanges2/Core/2026-07-17 - RegisterItemIsFulfillableInterceptors.sql
|
|
16
17
|
- dbchanges2/Client/2026-07-17 - ItemsisFulfillable.sql
|
|
@@ -18,6 +19,7 @@ related:
|
|
|
18
19
|
- recursive-item-fulfillments.md
|
|
19
20
|
- ../../../1.0/apps/library/features/netsuite-item-isfulfillable-sync.md
|
|
20
21
|
- ../../../clients/compass-usa/profile.md
|
|
22
|
+
- ../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md
|
|
21
23
|
---
|
|
22
24
|
|
|
23
25
|
## Summary
|
|
@@ -78,6 +80,35 @@ same bridge topology, walked for a different payload (a scalar flag rather than
|
|
|
78
80
|
too** — the same integrity class as the Compass off-by-one bridge bugs (see
|
|
79
81
|
[MITS PO → SO Item Linking](../../../clients/compass-usa/features/mits-po-to-so-item-linking.md)).
|
|
80
82
|
|
|
83
|
+
## Reach limit — vendor-fulfilled items can NEVER be reached (structural, verified 2026-08-12)
|
|
84
|
+
The walk only reaches a client item whose PO has an **Agilant-tier SO behind it** (a
|
|
85
|
+
`PurchaseOrderItems_SalesOrderItems` row). **Vendor-direct POs have none**, so an item that is only
|
|
86
|
+
ever fulfilled by an outside vendor **can never receive `isFulfillable` from this pipeline** — no
|
|
87
|
+
amount of re-running the backfill changes that. It is a **structural gap, not a catch-up gap.**
|
|
88
|
+
|
|
89
|
+
Prod `Client_Compass` (2026-08-12): **zero** items with `isFulfillable IS NULL` reach the Agilant
|
|
90
|
+
tier, versus **52,990** line-level reaches for `isFulfillable = 1` items. Per vendor (Compass items
|
|
91
|
+
ordered / reaching the Agilant tier): OFFICE DEPOT **497/124**; STRATEGIC SYSTEMS **66/0**; FREEDOM
|
|
92
|
+
GROUPS LLC **61/0**; PRESIDIO **45/0**; COMPASS GROUP **35/0**; ENCOMPASS SUPPLY CHAIN **2/0**. Only
|
|
93
|
+
ODP partially routes through Agilant NetSuite. Full analysis + the remediation options:
|
|
94
|
+
[Compass isFulfillable data quality & type rule](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md).
|
|
95
|
+
|
|
96
|
+
## Where the flag is consumed (frontend gate)
|
|
97
|
+
`toga2-supply/src/pages/Orders/view/OrderView/components/sections/OrderItemsTableSection.tsx` —
|
|
98
|
+
`isQtyFulfilledClickable` (~lines 21–30) gates the **Qty Fulfilled** cell on:
|
|
99
|
+
|
|
100
|
+
```
|
|
101
|
+
String(item?.Items?.isFulfillable) === "1" && Number(item?._qtyFulfilled) !== 0
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
- **`NULL` and `0` behave identically here** — both make the cell dead.
|
|
105
|
+
- The gate also carries a **hardcoded Compass exception**
|
|
106
|
+
(`hostName === COMPASS && AssetTypes.name === 'FEE' → false`) that duplicates the proposed
|
|
107
|
+
type-derived data rule; it should be **deleted once that rule lands in the DB** so the logic lives
|
|
108
|
+
in one place.
|
|
109
|
+
- Because the gate *also* requires `_qtyFulfilled !== 0`, an item that has never been fulfilled is
|
|
110
|
+
unclickable regardless of the flag's value.
|
|
111
|
+
|
|
81
112
|
## Data model / schema (deploy)
|
|
82
113
|
Three `dbchanges2` migrations register the field + hooks; **all must be present in the target
|
|
83
114
|
env or the flag write is rejected:**
|
|
@@ -90,9 +121,16 @@ env or the flag write is rejected:**
|
|
|
90
121
|
|
|
91
122
|
## Conflict rule
|
|
92
123
|
Multi-source kits (an item reachable from more than one source) use **LAST-WRITER-WINS**. A
|
|
93
|
-
deterministic `MIN`/`MAX` aggregate was **deliberately deferred**
|
|
124
|
+
deterministic `MIN`/`MAX` aggregate was **deliberately deferred** on the assumption that values are
|
|
94
125
|
uniformly `1` (see the Phase-1 gotcha: NetSuite flags nearly everything fulfillable).
|
|
95
126
|
|
|
127
|
+
**⚠ That assumption is no longer safe (2026-08-12).** Prod `Client_Compass` holds **25 items with
|
|
128
|
+
`isFulfillable = 0`** — all bundle headers, **14 with real fulfillments** (worst case `B4NY9UC`:
|
|
129
|
+
11,062 order lines / 2,864 fulfillment rows, its Qty Fulfilled cell disabled). NetSuite marks
|
|
130
|
+
group/kit **headers** `F`, but in TOGa **the header is the line that ships**. Mixed 0/1 values are
|
|
131
|
+
therefore real, and last-writer-wins can land on the wrong one. See the
|
|
132
|
+
[Compass data-quality doc](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md).
|
|
133
|
+
|
|
96
134
|
## Client variations
|
|
97
135
|
- **Compass** (USA/Canada, DB `Client_Compass`) is the built/verified client. Its `_Model_Compass_Item`
|
|
98
136
|
overrides `postPost`/`postPut` for a price override, so it **must** `parent::` up to the base or
|
|
@@ -123,12 +161,35 @@ uniformly `1` (see the Phase-1 gotcha: NetSuite flags nearly everything fulfilla
|
|
|
123
161
|
**resolved/substituted at deploy** — it cannot join across clusters at runtime.
|
|
124
162
|
- **⚠ Compass subclass shadowing.** If a future client subclass overrides Items `postPost`/`postPut`
|
|
125
163
|
without calling `parent::`, propagation silently stops for that client.
|
|
126
|
-
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
164
|
+
- **⚠ Chain-propagation writes are INVISIBLE to the audit log (debugging gotcha).**
|
|
165
|
+
`propagateFulfillableAcrossChain` issues a **raw `UPDATE Items … WHERE id IN (…)` through `_Query`**
|
|
166
|
+
(`Item.php` ~lines 562–571), bypassing the model layer and therefore the `Logs_<client>` audit
|
|
167
|
+
trail. A **sync PUT is logged; a chain propagation is silent** — so "no log row" does **not** mean
|
|
168
|
+
"never written". Use that as the discriminator between the two write paths: Compass item **2383**
|
|
169
|
+
(Agilant source 2720, NetSuite id 1536664) has **no** log row while 2382/2384/2385 do, which alone
|
|
170
|
+
identifies which path set each value.
|
|
171
|
+
**Query anchor:** prod `Core.RecordFields` **id 2434 = `Items.isFulfillable`** (`recordId 21`); join
|
|
172
|
+
`Logs_<client>.Record` (`recordId = 21`, `primaryKeyId = <itemId>`) to `Logs_<client>.RecordField`
|
|
173
|
+
`ON logRecordId` with `recordFieldId = 2434`. Same technique applies to any
|
|
174
|
+
interceptor-vs-propagation debugging.
|
|
175
|
+
- **Product decision — analyzed 2026-08-12, decision taken for Compass (was "unresolved").** Because
|
|
176
|
+
NetSuite's `isfulfillable` is `T` for services as well as physical goods (see Phase-1 doc),
|
|
177
|
+
propagation makes **everything** clickable. A prod evidence review showed item **type cannot
|
|
178
|
+
confirm** fulfillability (0%-tracked types like DESKTOPS are physical) but the **tracking
|
|
179
|
+
ride-along test can disqualify** fee/software/miscellaneous lines. Compass has since adopted a
|
|
180
|
+
**type-derived override** of NetSuite's flag, which **must be enforced at write time** because the
|
|
181
|
+
Phase-1 daily refresh reverts it. Rule, projected impact, caveats:
|
|
182
|
+
[Compass isFulfillable data quality & type rule](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md).
|
|
130
183
|
|
|
131
184
|
## Change history
|
|
185
|
+
- 2026-08-12 — Prod investigation (read-only, no code change): recorded the engine's **structural
|
|
186
|
+
reach limit** (vendor-direct POs have no Agilant-tier SO, so those items can never receive the
|
|
187
|
+
flag — 0 of the NULL Compass items reach the Agilant tier vs 52,990 reaches for `=1` items),
|
|
188
|
+
documented the **toga2-supply frontend gate** that consumes the flag (plus its hardcoded Compass
|
|
189
|
+
`FEE` exception to retire), added the **chain-propagation writes are invisible to the audit log**
|
|
190
|
+
gotcha with the `Core.RecordFields 2434` / `Logs_<client>` query anchor, and retired the
|
|
191
|
+
"values are uniformly 1" assumption behind the deferred conflict rule (25 Compass zeros, 14 with
|
|
192
|
+
real fulfillments). (bala)
|
|
132
193
|
- 2026-07-28 — Fixed a **read-replica routing** bug: `propagateFulfillableAcrossChain`'s source-item
|
|
133
194
|
read and the recursive chain-walk CTE now `setisReadHostEnabled(false)` so they run on the writer
|
|
134
195
|
and see the request's own open write transaction (read-your-writes) — a replica read could return
|
|
@@ -151,3 +212,5 @@ uniformly `1` (see the Phase-1 gotcha: NetSuite flags nearly everything fulfilla
|
|
|
151
212
|
bridge topology.
|
|
152
213
|
- [Compass MITS PO → SO Item Linking](../../../clients/compass-usa/features/mits-po-to-so-item-linking.md)
|
|
153
214
|
— how the SOI↔POI bridges get built (and the integrity bugs that break this walk).
|
|
215
|
+
- [Compass isFulfillable — Data Quality & the Type-Derived Rule](../../../clients/compass-usa/features/isfulfillable-data-quality-and-type-rule.md)
|
|
216
|
+
— measured prod state, why the NULLs are unreachable, and the Compass override.
|
package/knowledge/INDEX.md
CHANGED
|
@@ -5,7 +5,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
5
5
|
## 1.0 framework
|
|
6
6
|
|
|
7
7
|
- **library** (Library) _(framework core)_ — 17 doc(s) → [1.0/apps/library/INDEX.md](1.0/apps/library/INDEX.md)
|
|
8
|
-
- **worker** (Worker) —
|
|
8
|
+
- **worker** (Worker) — 22 doc(s) → [1.0/apps/worker/INDEX.md](1.0/apps/worker/INDEX.md)
|
|
9
9
|
- **dbchanges** (Database Changes) _(framework core)_ — 1 doc(s) → [1.0/apps/dbchanges/INDEX.md](1.0/apps/dbchanges/INDEX.md)
|
|
10
10
|
- **worker1.5** (Worker 1.5) — 0 doc(s) → [1.0/apps/worker1.5/INDEX.md](1.0/apps/worker1.5/INDEX.md)
|
|
11
11
|
- **togadesk** (TOGa Desk) — 12 doc(s) → [1.0/apps/togadesk/INDEX.md](1.0/apps/togadesk/INDEX.md)
|
|
@@ -18,7 +18,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
18
18
|
|
|
19
19
|
## 2.0 framework
|
|
20
20
|
|
|
21
|
-
- **_underscore** (_Underscore) _(framework core)_ —
|
|
21
|
+
- **_underscore** (_Underscore) _(framework core)_ — 55 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
22
22
|
- **worker2** (Worker) — 48 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
23
23
|
- **api2** (API) — 22 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
24
24
|
- **dbchanges2** (Database Changes) _(framework core)_ — 8 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
|
@@ -15,12 +15,13 @@ project: _Underscore
|
|
|
15
15
|
client: compass-canada
|
|
16
16
|
type: profile
|
|
17
17
|
status: active
|
|
18
|
-
updated: 2026-08-
|
|
18
|
+
updated: 2026-08-12
|
|
19
19
|
owners: [jcardinal, bala, tcox, apeterson]
|
|
20
20
|
files: []
|
|
21
21
|
related:
|
|
22
22
|
- ../compass-usa/profile.md
|
|
23
23
|
- ../compass-usa/features/approval-decision-flow.md
|
|
24
|
+
- ../compass-usa/features/isfulfillable-data-quality-and-type-rule.md
|
|
24
25
|
- features/french-order-email-localization.md
|
|
25
26
|
- ../../2.0/apps/toga2-commerce/features/expedited-shipping-gating.md
|
|
26
27
|
- ../../2.0/apps/_underscore/features/item-fulfillment-stage-lifecycle-and-order-status.md
|
|
@@ -104,6 +105,17 @@ to but distinct from Compass USA. Like Compass USA it spans the **2.0** commerce
|
|
|
104
105
|
impact: 34 personas, 7,017 users with personas, 1,461 with more than one. Fix `5313a9b1` exists on
|
|
105
106
|
`TRUE-80672`/`_beta` but is **not in `_production`**. See
|
|
106
107
|
[Client Fields → Login-settings branches](../../2.0/apps/toga2-commerce/features/client-fields.md).
|
|
108
|
+
- **⚠ `Items.isFulfillable` is 100% NULL in Canada — the Qty Fulfilled cell is dead storefront-wide.**
|
|
109
|
+
All **220** `Client_CompassCanada` items are NULL (prod, 2026-08-12): Canada was **never** included
|
|
110
|
+
in the multi-client backfill and has never been stamped at all. Note also that Canada uses a
|
|
111
|
+
**different type vocabulary** than Compass USA — title-case `assetTypes` (`Equipment`, `Hardware`,
|
|
112
|
+
`Services`, `Fee`, `Consulting`) and its own `itemTypes` (`LAPTOP`, `MACBOOK`, `MONITORS`,
|
|
113
|
+
`CABLES`, `SERVICE`, `Services`, `CONSULTING`, `SOFTWARE LICENSING`), with **no `FEE` or
|
|
114
|
+
`SERVICEFEES` itemType at all**. The US-derived rule list still matches, because the columns are
|
|
115
|
+
`utf8mb4_0900_ai_ci` (case-insensitive) — **do not split it into two hand-maintained lists**, but
|
|
116
|
+
do re-confirm the Canada vocabulary separately. Projected impact of the proposed rule here: 13 → 0,
|
|
117
|
+
207 → 1. See
|
|
118
|
+
[isFulfillable — Data Quality & the Type-Derived Rule](../compass-usa/features/isfulfillable-data-quality-and-type-rule.md).
|
|
107
119
|
- The 2026-06-08 ASN → ItemFulfillment work was for **Compass USA**, not Compass Canada.
|
|
108
120
|
- **Order status is shipped-only**, same as Compass USA (shared `_Model_Compass_SalesOrder`).
|
|
109
121
|
Compass Canada's own IF lifecycle stages (picked/packed/shipped) + shipped backfill are seeded by
|
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
| [Compass Approval-Decision Flow (Notifications & Manager Reassignment)](features/approval-decision-flow.md) | 2.0 | Compass's sales-order approval flow — approval/notification email lists, **manager reassignment**, VIP auto-approve, and EN/FR localization — lives **entirely i | _underscore/Model/Compass/ApprovalDecision.php, _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/Usa/ApprovalDecision.php, _underscore/Model/Compass/Canada/ApprovalDecision.php |
|
|
6
6
|
| [Compass ASN → ItemFulfillment Auto-Creation](features/asn-to-item-fulfillment.md) | 2.0 | For Compass USA, posting an AdvanceShippingNotice (ASN) auto-creates the ItemFulfillment (IF) on the upstream SalesOrder. | _underscore/Model/Compass/AdvanceShippingNotice.php, _underscore/Model/Compass/PurchaseOrder.php, api2/Component/Api/Cxml/Cxml.php, dbchanges2/Client_Compass/2026-06-11 - AsnItemTrackingNumberAcl.sql, dbchanges2/Client_Compass/2026-06-15b - BackfillSA132781ItemFulfillmentTracking.sql, dbchanges2/Client_Compass/2026-06-16 - CleanupSA132763CrossLineTracking.sql, dbchanges2/Client_Compass/2026-06-16b - CleanupSA132743CrossLineTracking.sql, dbchanges2/Client_Compass/2026-06-16c - BackfillSA132763C40QYUCTracking.sql, dbchanges2/Client_Compass/2026-06-18a - CleanupSA132898DuplicateTracking.sql, dbchanges2/Client_Compass/2026-06-18b - CleanupSA132881DuplicateTracking.sql, dbchanges2/Client_Compass/2026-07-02a - FixSA133377TrackingSerialAndDuplicateIF.sql |
|
|
7
7
|
| [Cost Centers — Unit Locations, numeric-only policy](features/cost-centers.md) | 2.0 | A Compass "cost center" — the value a user picks in commerce and that lands on an order — is **not** a `CostCenters` row. | toga2-commerce/src/pages/Cart/api/CartApi.ts, worker1.5/crons/toga2/compass/import_locations.php, _underscore/Model/Compass/SalesOrder.php, api2/Component/Api/V2/V2.php, dbchanges2/Client_Compass/2026-07-06 - RemoveNonNumericCostCenters.sql |
|
|
8
|
+
| [Compass isFulfillable — Data Quality & the Type-Derived Rule](features/isfulfillable-data-quality-and-type-rule.md) | 2.0 | A **prod-data investigation (2026-08-12, read-only)** into why so many Compass storefront lines still have a dead **Qty Fulfilled** cell. | _underscore/Model/Client/Item.php, library/app/api/toga2.php, worker/crons/toga2/netsuite/backfill_isfulfillable_all_clients.php, toga2-supply/src/pages/Orders/view/OrderView/components/sections/OrderItemsTableSection.tsx |
|
|
8
9
|
| [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two sibling Compass TableViews in `Client_Compass` display fulfilled items in toga2-supply, both driven by `TableViews` / `TableViewJoins` / `TableViewFields` c | dbchanges2/Client_Compass/2026-06-10 - ItemFulfillmentsForSalesOrderItemsTableView.sql, dbchanges2/Client_Compass/2026-06-11 - ItemFulfillmentsForSalesOrdersTableView.sql, dbchanges2/Client_Compass/2026-06-15a - FixItemFulfillmentTrackingNumberJoins.sql, dbchanges2/Client/2026-07-15a - ExcludeFeeItemsFromItemFulfillmentsForSalesOrdersView.sql |
|
|
9
10
|
| [Compass MITS PO → SO Item Linking](features/mits-po-to-so-item-linking.md) | 2.0 | MITS sends Compass inbound Purchase Orders (`POST /v2/purchase-orders`) against a Sales Order (`mitsSalesOrder`). | _underscore/Model/Compass/PurchaseOrder.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php |
|
|
10
11
|
| [Compass MITS PO Transmission to Vendors](features/mits-po-transmission-to-vendors.md) | 2.0 | The 1.0 worker cron `2_transmit_mits_purchase_orders_to_vendors.php` transmits Compass PurchaseOrders to their vendors (Office Depot, Strategic Systems, Compass | worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, worker/crons/toga2/compasscanada/workflow/2_transmit_mits_purchase_orders_to_vendors.php, library/app/client/compass.php |
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Compass isFulfillable — Data Quality & the Type-Derived Rule
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: _underscore
|
|
5
|
+
project: _Underscore
|
|
6
|
+
client: compass-usa
|
|
7
|
+
type: client-feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-08-12
|
|
10
|
+
owners: ["bala"]
|
|
11
|
+
files:
|
|
12
|
+
- _underscore/Model/Client/Item.php
|
|
13
|
+
- library/app/api/toga2.php
|
|
14
|
+
- worker/crons/toga2/netsuite/backfill_isfulfillable_all_clients.php
|
|
15
|
+
- toga2-supply/src/pages/Orders/view/OrderView/components/sections/OrderItemsTableSection.tsx
|
|
16
|
+
related:
|
|
17
|
+
- ../profile.md
|
|
18
|
+
- ../../compass-canada/profile.md
|
|
19
|
+
- ../../../2.0/apps/_underscore/features/fulfillable-item-propagation.md
|
|
20
|
+
- ../../../1.0/apps/library/features/netsuite-item-isfulfillable-sync.md
|
|
21
|
+
- ../../../1.0/apps/worker/workflows/isfulfillable-multi-client-backfill.md
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## Summary
|
|
25
|
+
A **prod-data investigation (2026-08-12, read-only)** into why so many Compass storefront lines still
|
|
26
|
+
have a dead **Qty Fulfilled** cell. Conclusion in one line: the remaining `Items.isFulfillable`
|
|
27
|
+
NULLs are **not a backfill backlog — they are structurally unreachable**, and the values that *are*
|
|
28
|
+
set are partly wrong. This doc records the measured state of `Client_Compass` /
|
|
29
|
+
`Client_CompassCanada`, the evidence for (and against) deriving the flag from item type, the
|
|
30
|
+
**decision** to override NetSuite with a type-derived rule for Compass, and the **write-time guard**
|
|
31
|
+
that decision requires in order to survive the daily NetSuite sync.
|
|
32
|
+
|
|
33
|
+
No source files were changed and **no migration has been written yet**. All figures below were read
|
|
34
|
+
from **production** on **2026-08-12**.
|
|
35
|
+
|
|
36
|
+
## Measured state (prod, 2026-08-12)
|
|
37
|
+
- **`Client_Compass`, catalog 1:** **770 NULL / 25 zero / 697 one.**
|
|
38
|
+
- **12,318 Compass order lines have real `ItemFulfillmentItems` but a NULL flag** — the Qty
|
|
39
|
+
Fulfilled cell is dead on all of them. A further **4,873** lines sit behind `isFulfillable = 0`.
|
|
40
|
+
- **`Client_CompassCanada`: 220 items, 100% NULL.** Canada has **never** been stamped at all, so the
|
|
41
|
+
Qty Fulfilled cell is dead across the **entire** Canada storefront.
|
|
42
|
+
|
|
43
|
+
## Why the NULLs cannot be backfilled (structural, not a catch-up gap)
|
|
44
|
+
The flag is born on an **Agilant-catalog source item** during the 1.0 NetSuite item sync
|
|
45
|
+
([Phase 1](../../../1.0/apps/library/features/netsuite-item-isfulfillable-sync.md)) and then
|
|
46
|
+
propagates **up** the `SalesOrderItems`↔`PurchaseOrderItems` chain
|
|
47
|
+
([Phase 2](../../../2.0/apps/_underscore/features/fulfillable-item-propagation.md)). That walk needs
|
|
48
|
+
the Compass PO to have an **Agilant-tier SO behind it** (a `PurchaseOrderItems_SalesOrderItems`
|
|
49
|
+
row). **Vendor-direct POs have none**, so a vendor-fulfilled item can never receive the flag from
|
|
50
|
+
the current pipeline.
|
|
51
|
+
|
|
52
|
+
Verified in prod `Client_Compass`: **zero** items with `isFulfillable IS NULL` reach the Agilant
|
|
53
|
+
tier, against **52,990** line-level reaches for `isFulfillable = 1` items.
|
|
54
|
+
|
|
55
|
+
Compass items ordered / reaching the Agilant tier, per vendor:
|
|
56
|
+
|
|
57
|
+
| Vendor | Ordered | Reaching Agilant tier |
|
|
58
|
+
|---|---|---|
|
|
59
|
+
| OFFICE DEPOT | 497 | 124 |
|
|
60
|
+
| STRATEGIC SYSTEMS | 66 | 0 |
|
|
61
|
+
| FREEDOM GROUPS LLC | 61 | 0 |
|
|
62
|
+
| PRESIDIO | 45 | 0 |
|
|
63
|
+
| COMPASS GROUP | 35 | 0 |
|
|
64
|
+
| ENCOMPASS SUPPLY CHAIN | 2 | 0 |
|
|
65
|
+
|
|
66
|
+
**Consequence: re-running the multi-client backfill cannot move a single one of these.** Only ODP
|
|
67
|
+
partially routes through Agilant NetSuite.
|
|
68
|
+
|
|
69
|
+
## The 25 `isFulfillable = 0` Compass items are WRONG — they should be 1
|
|
70
|
+
All 25 are **bundle items**; **14 of them have real fulfillments**. Worst case **`B4NY9UC`**:
|
|
71
|
+
**11,062 order lines and 2,864 fulfillment rows** — the most-shipped laptop in the catalog, with its
|
|
72
|
+
Qty Fulfilled cell **disabled**. NetSuite marks group/kit **headers** `F`, but in TOGa **the header
|
|
73
|
+
IS the line that ships**. This supersedes the earlier assumption (in the Phase 1/Phase 2 docs) that
|
|
74
|
+
the zeros were harmless because values were "uniformly 1".
|
|
75
|
+
|
|
76
|
+
## Can item type reproduce NetSuite's flag? (evidence)
|
|
77
|
+
**The naive test fails.** "Does this type ever fulfil?" does not separate anything:
|
|
78
|
+
`SERVICES` 119/120 known-fulfillable, `SOFTWARE` 7/7, `FEE` 3/3, while `HARDWARE` fulfils on only
|
|
79
|
+
**26.9%** of lines. `inventoryType = HYBRID` is the **column default** = noise.
|
|
80
|
+
`EntitlementFulfillmentTypes` / `EntitlementFulfillmentMethods` are **EMPTY** in prod.
|
|
81
|
+
|
|
82
|
+
**The decisive test is the tracking "ride-along":** does a fulfillment row carry **its own** tracking
|
|
83
|
+
number, or does it share one with another line on the same shipment?
|
|
84
|
+
|
|
85
|
+
| Type | Fulfillment rows | With tracking | Ride-along (shares tracking) |
|
|
86
|
+
|---|---|---|---|
|
|
87
|
+
| itemType `FEE` | 9,639 | 1 (0.0%) | — (0 units) |
|
|
88
|
+
| assetType `FEE` | 10,071 | 22 | **22/22 = 100%** |
|
|
89
|
+
| itemType `SOFTWARE` | — | 129 | **129/129 = 100%** |
|
|
90
|
+
| assetType `MISCELLANEOUS` | — | 385 | **385/385 = 100%** |
|
|
91
|
+
| itemType `SERVICES` | — | 687 | 274 = **39.9%** (413 rows carry their OWN tracking) |
|
|
92
|
+
|
|
93
|
+
A 100% ride-along rate means the type **never has its own shipment** — a fee/licence line billed
|
|
94
|
+
alongside hardware. `SERVICES` fails that test: 413 service rows genuinely ship on their own.
|
|
95
|
+
|
|
96
|
+
**⚠ Absence of tracking does NOT prove non-fulfillable.** `DESKTOPS`, `CANTEEN TECHNOLOGY` and
|
|
97
|
+
`VENDING AND OCS ORDERING` all show **0% tracked** yet are physical goods. **Type can only
|
|
98
|
+
DISQUALIFY an item, never confirm one.**
|
|
99
|
+
|
|
100
|
+
## Decision — the type-derived rule for Compass (2026-08-12)
|
|
101
|
+
Developer's call; it **overrides NetSuite's value**:
|
|
102
|
+
|
|
103
|
+
```
|
|
104
|
+
isFulfillable = 0 WHERE itemType OR assetType IN
|
|
105
|
+
('FEE','SERVICEFEES','SERVICES','SERVICE','CONSULTING')
|
|
106
|
+
isFulfillable = 1 otherwise
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Projected impact:
|
|
110
|
+
- **`Client_Compass`:** **186 → 0** (54 currently NULL + 131 currently **1** + 1 currently 0) and
|
|
111
|
+
**606 → 1**.
|
|
112
|
+
- **`Client_CompassCanada`:** **13 → 0**, **207 → 1**.
|
|
113
|
+
|
|
114
|
+
**⚠ Recorded caveat — the SERVICES/CONSULTING half is NOT supported by the evidence.** 413
|
|
115
|
+
`SERVICES` fulfillment rows carry their own tracking, and the rule flips **131 currently-clickable
|
|
116
|
+
items off**. That half **needs product sign-off**. `SOFTWARE` (100% ride-along) is the *stronger*
|
|
117
|
+
candidate on the evidence but was deliberately **left out of scope**.
|
|
118
|
+
|
|
119
|
+
## ⚠ CRITICAL — a backfill alone will be undone by the daily NetSuite sync
|
|
120
|
+
Phase 1's `getCreateItem` **refreshes `isFulfillable` on EXISTING items** whenever NetSuite's value
|
|
121
|
+
differs from the stored value. NetSuite returns **`T` for service (`NonInvtPart`) items**, so any
|
|
122
|
+
type-derived **0** is **flipped back to 1 on the next sync run**.
|
|
123
|
+
|
|
124
|
+
Proven from the audit log: Compass items **2382 / 2384 / 2385**
|
|
125
|
+
(`US-IOS-SSC-SHR`, `US-IOS-VENUENEXT`, `US-IOS-VIRTUALMGR` — itemType `SERVICES`, assetType
|
|
126
|
+
`CONSULTING`) were stamped **NULL → 1 on 2026-08-04 13:56:00-01** by `apiId 1`, `userId NULL`,
|
|
127
|
+
inside a **1–2 items/second cron sweep** that began 13:45 and set every item to 1.
|
|
128
|
+
|
|
129
|
+
**Therefore the rule MUST be enforced at write time** — a `prePut`/`prePost` guard on Items in
|
|
130
|
+
`_Model_Client_Item` — **in addition to** the backfill migration. A migration on its own silently
|
|
131
|
+
reverts within a day.
|
|
132
|
+
|
|
133
|
+
## Evidence-based alternative signals (if type is not wanted)
|
|
134
|
+
- **`ItemFulfillmentItems`** (via `SalesOrderItems.salesOrderItemId`) = **proven shipped**.
|
|
135
|
+
- **`VendorItems`** (`isActive = 1`) = **procurable from a vendor**.
|
|
136
|
+
|
|
137
|
+
Tiering the 770 NULL Compass items: **184** have fulfillment history · **286** have an active
|
|
138
|
+
`VendorItems` row · **300** have neither. The frontend gate already requires
|
|
139
|
+
`_qtyFulfilled !== 0`, so **tier-3 items cannot be clicked regardless of the flag**, and can be left
|
|
140
|
+
NULL.
|
|
141
|
+
|
|
142
|
+
## Where the storefront consumes the flag (+ a hardcode to retire)
|
|
143
|
+
`toga2-supply/src/pages/Orders/view/OrderView/components/sections/OrderItemsTableSection.tsx`,
|
|
144
|
+
`isQtyFulfilledClickable` (~lines 21–30):
|
|
145
|
+
|
|
146
|
+
```
|
|
147
|
+
String(item?.Items?.isFulfillable) === "1" && Number(item?._qtyFulfilled) !== 0
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
plus an extra **hardcoded** `hostName === COMPASS && AssetTypes.name === 'FEE' → false`.
|
|
151
|
+
|
|
152
|
+
- **NULL and 0 behave identically** at this gate.
|
|
153
|
+
- That **FEE hardcode duplicates the proposed data rule** — once the rule lands in the DB it should
|
|
154
|
+
be **deleted** so the logic lives in one place.
|
|
155
|
+
|
|
156
|
+
## Compass Canada uses a DIFFERENT type vocabulary
|
|
157
|
+
Canada's `assetTypes` are **title case** (`Equipment`, `Hardware`, `Services`, `Fee`, `Consulting`)
|
|
158
|
+
and its `itemTypes` are a **separate vocabulary** (`LAPTOP`, `MACBOOK`, `MONITORS`, `CABLES`,
|
|
159
|
+
`SERVICE`, `Services`, `CONSULTING`, `SOFTWARE LICENSING`) — it has **no `FEE` or `SERVICEFEES`
|
|
160
|
+
itemType at all**.
|
|
161
|
+
|
|
162
|
+
Because these tables are `utf8mb4_0900_ai_ci` (**case-insensitive**), the single US-derived
|
|
163
|
+
`IN ('FEE','SERVICEFEES',…)` list **matches Canada's title-case values** — verified against live
|
|
164
|
+
data. **Do not hand-maintain two string lists**, but **do** re-confirm the Canada vocabulary
|
|
165
|
+
separately, since the names genuinely differ and a US list *looks* wrong for Canada.
|
|
166
|
+
|
|
167
|
+
## Change history
|
|
168
|
+
- 2026-08-12 — Prod read-only investigation of `Items.isFulfillable` across `Client_Compass` /
|
|
169
|
+
`Client_CompassCanada`. Established that remaining NULLs are **structurally unreachable**
|
|
170
|
+
(vendor-direct POs have no Agilant-tier SO, so the chain walk has nothing to walk — 0 NULL items
|
|
171
|
+
reach the Agilant tier vs 52,990 reaches for `=1` items), that the **25 zeros are wrong** (all
|
|
172
|
+
bundle headers, 14 with real fulfillments, incl. `B4NY9UC` at 11,062 lines / 2,864 fulfillments),
|
|
173
|
+
and that **Canada is 100% NULL**. Proved item type cannot *confirm* fulfillability but the
|
|
174
|
+
**tracking ride-along test** cleanly disqualifies fee/software/miscellaneous lines (100%
|
|
175
|
+
ride-along) while `SERVICES` fails it (39.9%). **Decided** the type-derived rule
|
|
176
|
+
(`FEE/SERVICEFEES/SERVICES/SERVICE/CONSULTING → 0`, else 1) with the SERVICES/CONSULTING half
|
|
177
|
+
flagged for product sign-off, and established that it **must** be enforced by a `prePut`/`prePost`
|
|
178
|
+
guard because the daily sync's existing-item refresh reverts it (audit-log proof: items
|
|
179
|
+
2382/2384/2385 stamped NULL→1 on 2026-08-04). (bala)
|
|
180
|
+
|
|
181
|
+
## Related docs
|
|
182
|
+
- [Phase 1 — isFulfillable from NetSuite during item sync (1.0)](../../../1.0/apps/library/features/netsuite-item-isfulfillable-sync.md)
|
|
183
|
+
— where the value is born and the refresh that reverts overrides.
|
|
184
|
+
- [Phase 2 — isFulfillable propagation up the SO↔PO chain (2.0)](../../../2.0/apps/_underscore/features/fulfillable-item-propagation.md)
|
|
185
|
+
— the chain walk whose reach is the structural limit described here.
|
|
186
|
+
- [isFulfillable multi-client backfill (worker)](../../../1.0/apps/worker/workflows/isfulfillable-multi-client-backfill.md)
|
|
187
|
+
— the cron that **cannot** close this gap.
|
|
@@ -17,7 +17,7 @@ project: _Underscore
|
|
|
17
17
|
client: compass-usa
|
|
18
18
|
type: profile
|
|
19
19
|
status: active
|
|
20
|
-
updated: 2026-08-
|
|
20
|
+
updated: 2026-08-12
|
|
21
21
|
owners: [jcardinal, bala, tcox, apeterson, dfranks]
|
|
22
22
|
files: []
|
|
23
23
|
related:
|
|
@@ -27,6 +27,7 @@ related:
|
|
|
27
27
|
- features/mits-sales-order-transmission-alerting.md
|
|
28
28
|
- features/asn-to-item-fulfillment.md
|
|
29
29
|
- features/cost-centers.md
|
|
30
|
+
- features/isfulfillable-data-quality-and-type-rule.md
|
|
30
31
|
- features/approval-decision-flow.md
|
|
31
32
|
- workflows/cross-kit-bundle-corruption.md
|
|
32
33
|
- workflows/odp-duplicate-po-line-cleanup.md
|
|
@@ -134,4 +135,12 @@ separate, related client (see its own profile).
|
|
|
134
135
|
up the SO↔PO chain to the client-facing Compass item by an api2 interceptor
|
|
135
136
|
([Phase 2](../../2.0/apps/_underscore/features/fulfillable-item-propagation.md)). Verified on prod
|
|
136
137
|
Compass chains; a one-time July-5+ backfill re-PUTs source items through the same path.
|
|
138
|
+
**⚠ Prod audit 2026-08-12 — the flag is largely broken for Compass and NO backfill can fix it.**
|
|
139
|
+
770 NULL / 25 zero / 697 one; **12,318** order lines are really fulfilled but flagged NULL, so
|
|
140
|
+
their Qty Fulfilled cell is dead. The NULLs are **structurally unreachable** (vendor-direct POs
|
|
141
|
+
have no Agilant-tier SO to walk up from), and the 25 zeros are **wrong** (bundle headers that do
|
|
142
|
+
ship — incl. `B4NY9UC`, 11,062 lines). A **type-derived override** was decided and **must** be
|
|
143
|
+
enforced by a write-time guard, because the daily NetSuite sync reverts it. Evidence, rule,
|
|
144
|
+
projected impact and caveats:
|
|
145
|
+
[isFulfillable — Data Quality & the Type-Derived Rule](features/isfulfillable-data-quality-and-type-rule.md).
|
|
137
146
|
- This profile is a starting point; expand as more Compass-specific behavior is captured.
|
|
@@ -2,9 +2,10 @@
|
|
|
2
2
|
|
|
3
3
|
| Doc | Framework | Summary | Files |
|
|
4
4
|
|-----|-----------|---------|-------|
|
|
5
|
+
| [Elite — NetSuite → TOGa Supply inbound sync (TRUE-80499 onboarding)](features/netsuite-togasupply-sync.md) | 1.0 | Elite is the 18th client on the shared NetSuite → TOGa Supply importer ([engine](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)). | worker/crons/toga2/netsuite/sync_togasupply_elite.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/schedules/cron.worker.sync.json, _underscore/Model/Elite/SalesOrder.php, _underscore/Model/Elite/PurchaseOrder.php, _underscore/Model/Elite/ItemReceipt.php, dbchanges2/Client_Elite/_modules.txt, test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php, test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php |
|
|
5
6
|
| [Elite SalesOrder → NetSuite Push (postPost/postPut interceptors → worker2)](features/salesorder-netsuite-push.md) | 2.0 | Elite orders created in Toga are pushed into NetSuite **event-driven**, not on a cron. | _underscore/Model/Elite/SalesOrder.php, _underscore/Model/Elite/ServiceRequest.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Core/2026-08-06a - Service request and sales order payload interceptors.sql |
|
|
6
7
|
| [Elite — Sales Order stage change posts a reply on the TOGa Desk (1.0) ticket](features/salesorder-status-togadesk-reply.md) | 2.0 | When an Elite sales order's **stage** changes, a reply is posted on the originating **TOGa Desk (1.0)** ticket so the requester sees progress where they raised | worker2/Worker/Sync/SalesOrderStatus.php, _underscore/Model/Elite/SalesOrderStatus.php, _underscore/Model/Elite/SalesOrder.php |
|
|
7
8
|
| [Elite — supply2 frontend scope (Inventory + Service Requests, both built)](features/supply2-scope.md) | 2.0 | Scope for onboarding Elite to the `toga2-supply` frontend (host `ELITE`). | toga2-supply/ELITE-CLIENT-TASK-NOTES.md, toga2-supply/src/pages/Orders/view/OrderView/viewModel/FIELDS/ELITE/BASEFIELDS.json, toga2-supply/src/pages/Orders/viewModel/FIELDS/ELITE/BASE.json, toga2-supply/src/pages/Inventory/viewModel/FIELDS/DUMMYGROUPOPTIONS.ts, toga2-supply/src/hooks/useFetchData.tsx, toga2-supply/src/components/ui/Toaster.tsx, toga2-supply/src/pages/Inventory/viewModel/FIELDS/DUMMYGROUPOPTIONS.ts, toga2-supply/src/pages/Inventory/viewModel/FIELDS/INVENTORYPAGEFIELDS.ts, toga2-supply/src/pages/Inventory/viewModel/index.ts, toga2-supply/src/pages/Inventory/listing/InventoryPage.tsx, toga2-supply/src/pages/Inventory/listing/InventoryRouter.tsx, toga2-supply/src/pages/Inventory/listing/InventorySubTablePage.tsx, toga2-supply/src/utils/resolveClientHostName.ts, toga2-supply/src/utils/convertConstructorColumnTitles.ts, toga2-supply/src/pages/Orders/OrdersPage.tsx, toga2-supply/src/pages/Orders/viewModel/FIELDS/ELITE/BASE.json, toga2-supply/src/pages/Orders/viewModel/useOrdersPageViewModel.ts, toga2-supply/src/pages/Orders/api/OrdersApi.ts, toga2-supply/src/pages/Orders/view/OrderView/viewModel/useOrderDetailsViewModel.ts, toga2-supply/src/components/ui/Tables/PrimaryTable/helperFunctions/renderModalContent.tsx, toga2-supply/src/components/layout/SlideMenu/SlideMenu.tsx, toga2-supply/package.json |
|
|
8
9
|
| [Elite — stale TableView config (11 dead Core.RecordFields across 9 views)](features/supply2-tableview-config-drift.md) | 2.0 | `Client_Elite`'s `TableViewJoins` predate **two** platform bridge-table migrations and still reference **11 deleted `Core.RecordFields` ids (211, 321, 932, 358, | dbchanges2/Client_Elite/, dbchanges2/Client_Nychh/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql |
|
|
9
10
|
| [Elite — raising a Service Request from a TOGa Desk ticket (App_Api_ServiceRequest)](features/togadesk-service-request-intake.md) | 1.0 | An Elite agent raises a **Service Request** from a TOGa Desk (1.0) ticket via a modal. | library/app/api/servicerequest.php, library/app/model/togadesk/ticket.php, togadesk/desk/includes/classes/class.ticket.php, togadesk/desk/template/modals/tickets/serviceRequest.php, togadesk/desk/includes/controllers/modals/tickets/serviceRequest.php, togadesk/desk/includes/controllers/actions/tickets/serviceRequest.php, dbchanges2/Client_Elite/2026-08-11a - EliteServiceRequestTicketUnique.sql, test/@srija/Elite Testing/Service Requests/test_elite_desk_service_request.php |
|
|
10
|
-
| [Elite](profile.md) | 2.0 | Elite is a managed-services client that uses **Freshservice** as their helpdesk platform. | worker2/Worker/Elite.php, worker2/Worker/Sync/ServiceRequest.php, library/app/api/toga2.php, library/app/api/servicerequest.php, _underscore/Model/Elite/SalesOrder.php, _underscore/Model/Elite/ServiceRequest.php, togadesk/desk/includes/classes/class.ticket.php |
|
|
11
|
+
| [Elite](profile.md) | 2.0 | Elite is a managed-services client that uses **Freshservice** as their helpdesk platform. | worker2/Worker/Elite.php, worker2/Worker/Sync/ServiceRequest.php, library/app/api/toga2.php, library/app/api/servicerequest.php, _underscore/Model/Elite/SalesOrder.php, _underscore/Model/Elite/ServiceRequest.php, togadesk/desk/includes/classes/class.ticket.php, worker/crons/toga2/netsuite/sync_togasupply_elite.php |
|
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Elite — NetSuite → TOGa Supply inbound sync (TRUE-80499 onboarding)"
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: worker
|
|
5
|
+
project: Worker
|
|
6
|
+
client: elite
|
|
7
|
+
type: client-feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-08-12
|
|
10
|
+
owners: ["snaredla"]
|
|
11
|
+
files:
|
|
12
|
+
- worker/crons/toga2/netsuite/sync_togasupply_elite.php
|
|
13
|
+
- worker/crons/toga2/netsuite/common_sync_togasupply.php
|
|
14
|
+
- worker/schedules/cron.worker.sync.json
|
|
15
|
+
- _underscore/Model/Elite/SalesOrder.php
|
|
16
|
+
- _underscore/Model/Elite/PurchaseOrder.php
|
|
17
|
+
- _underscore/Model/Elite/ItemReceipt.php
|
|
18
|
+
- dbchanges2/Client_Elite/_modules.txt
|
|
19
|
+
- test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php
|
|
20
|
+
- test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php
|
|
21
|
+
related:
|
|
22
|
+
- ../profile.md
|
|
23
|
+
- ../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md
|
|
24
|
+
- ../../../1.0/apps/worker/workflows/onboarding-client-to-netsuite-togasupply-sync.md
|
|
25
|
+
- ./salesorder-netsuite-push.md
|
|
26
|
+
- ./supply2-scope.md
|
|
27
|
+
- ../../growrk/features/units-netsuite-custom-fields.md
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Summary
|
|
31
|
+
|
|
32
|
+
Elite is the 18th client on the shared NetSuite → TOGa Supply importer
|
|
33
|
+
([engine](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)). This doc records
|
|
34
|
+
what is **Elite-specific**; everything about how the engine works, and the two provisioning gaps this
|
|
35
|
+
onboarding discovered, lives in the engine and
|
|
36
|
+
[onboarding workflow](../../../1.0/apps/worker/workflows/onboarding-client-to-netsuite-togasupply-sync.md)
|
|
37
|
+
docs.
|
|
38
|
+
|
|
39
|
+
**Direction matters:** this is the **inbound** half (NetSuite → TOGa). The **outbound** push of Elite
|
|
40
|
+
orders *into* NetSuite is a separate, event-driven path — see
|
|
41
|
+
[salesorder-netsuite-push](./salesorder-netsuite-push.md). Nothing in this cron ever writes to
|
|
42
|
+
NetSuite.
|
|
43
|
+
|
|
44
|
+
**Backfill result (2026-08, April 2024 → Aug 2026, ~90 minutes):** 257 SalesOrders (0 missing a
|
|
45
|
+
NetSuite id), 234 PurchaseOrders, 201 ItemFulfillments, 29 ItemReceipts, 2 InventoryAdjustments,
|
|
46
|
+
153 Items, 229 Units, 1042 Contacts, 806 Addresses.
|
|
47
|
+
|
|
48
|
+
## What was built
|
|
49
|
+
|
|
50
|
+
| Piece | Detail |
|
|
51
|
+
|---|---|
|
|
52
|
+
| **Wrapper** | `worker/crons/toga2/netsuite/sync_togasupply_elite.php` — `isParentCustomer` **false**, NetSuite customer internal id **36443**, end-user-customer custom field **3149** |
|
|
53
|
+
| **Schedule** | `cron.worker.sync.json`, `*/5 * * * *`, `active: 1` |
|
|
54
|
+
| **Client models** | 18 new `_underscore/Model/Elite/*.php` trait models (see below) |
|
|
55
|
+
| **Schema** | `dbchanges2/Client_Elite/_modules.txt` opts into `_modules/netsuite` |
|
|
56
|
+
| **Warehouse locations** | `2026-08-05b` (NetSuite **196** "Elite"/Chicago) and `2026-08-06d` (NetSuite **5** "New York") — both confirmed to carry real Elite inventory-adjustment activity before seeding |
|
|
57
|
+
| **Vendor + ACL fixes** | `2026-08-06e - EliteAgilantVendor.sql`, `2026-08-06a - EliteNetsuiteCustomFieldAcl.sql` — the two module gaps, [written up in the workflow](../../../1.0/apps/worker/workflows/onboarding-client-to-netsuite-togasupply-sync.md) |
|
|
58
|
+
|
|
59
|
+
### The 18 trait models are pure convention — and fail SILENTLY
|
|
60
|
+
|
|
61
|
+
Each is only `class _Model_Elite_X extends _Model_Client_X { use _Trait_Netsuite_X; }`:
|
|
62
|
+
Country, Customer, InventoryAdjustment, Invoice, Item, ItemFulfillment, ItemFulfillmentItemUnit,
|
|
63
|
+
ItemFulfillmentStage, ItemReceipt, Location, PaymentTerm, PurchaseOrder, SalesOrder, SalesOrderItem,
|
|
64
|
+
SalesOrderStage, ShippingMethod, Subsidiary, Vendor.
|
|
65
|
+
|
|
66
|
+
Resolution is **pure class-name string substitution off the JWT client slug** —
|
|
67
|
+
`api2/Component/Api/V2/V2.php:2878-2886` reads `jwtPayload.id.client.slug` and does
|
|
68
|
+
`str_replace('_Model_Client_', '_Model_Elite_', …)` then `class_exists()`; the same block repeats at
|
|
69
|
+
:3148, :3744 and :4177 for nested/sort paths. **If `class_exists` fails, it falls back to the base
|
|
70
|
+
`_Model_Client_X` with no error, no log, and a 200 response** — the `c_netsuite*` fields simply are
|
|
71
|
+
not there. So a typo'd filename or class name presents as "the sync ran and imported nothing useful",
|
|
72
|
+
never as a failure.
|
|
73
|
+
|
|
74
|
+
The mirror-image trap is the one that bit GroWrk: once the model **does** resolve, every `c_` field
|
|
75
|
+
the trait declares must exist as a column, a `CustomRecordFields` row **and** an ACL grant, or reads
|
|
76
|
+
500/403 — see [GroWrk NetSuite-module custom fields](../../growrk/features/units-netsuite-custom-fields.md).
|
|
77
|
+
|
|
78
|
+
### Elite-only wrapper deviations
|
|
79
|
+
|
|
80
|
+
- **`MAX_TIME_WINDOW_TO_FETCH_FROM_NETSUITE_SECONDS = 2592000` (30 days)**, vs. the 5-day default the
|
|
81
|
+
other 17 wrappers inherit — Elite is backfilling from 2024 and 5-day windows on a 5-minute cron were
|
|
82
|
+
too slow. This is only possible because the shared engine now `define()`s the cap instead of
|
|
83
|
+
`const`-ing it. **Reduce it to the default once the backfill is caught up.**
|
|
84
|
+
- **`IS_ENABLED_INTEGRATION_INVOICES = false` — TEMPORARY.** The invoice section scans the whole
|
|
85
|
+
NetSuite transaction table per window, kept running out of time, and **starved the item-receipt and
|
|
86
|
+
item-fulfillment sections that follow it** — which are what `_qtyOnHand` derives from. Re-enable
|
|
87
|
+
once inventory has caught up. (Note the invoice section is also capped at `now − 86400s` by design,
|
|
88
|
+
so it always trails.)
|
|
89
|
+
- **`MIN_DATETIME_TO_CHECK_FOR_NETSUITE_DATA = '2018-01-01'`**, with the real start controlled by the
|
|
90
|
+
seeded `Parameters` cursors.
|
|
91
|
+
|
|
92
|
+
## Gotchas / known issues
|
|
93
|
+
|
|
94
|
+
- **⚠ Two files named `sync_togasupply_elite.php`, both scheduled.**
|
|
95
|
+
`worker/crons/toga2/netsuite/sync_togasupply_elite.php` (`*/5`) is **this** supply importer;
|
|
96
|
+
`worker/crons/toga2/elite/sync_togasupply_elite.php` (`*/2`) is the older
|
|
97
|
+
`syncWithToga`/`syncWithTogadesk` desk bridge with its own parameter keys. They coexist harmlessly
|
|
98
|
+
but **confirm the folder before editing** — this is the generic name collision the engine doc warns
|
|
99
|
+
about, now real for Elite.
|
|
100
|
+
- **⚠ `Client_Elite` has TWO active `Apis` rows** (Agilant `id 1`, Elite `id 2`). `getClientContext()`
|
|
101
|
+
does `LIMIT 1` with **no `ORDER BY`**, so a worker can authenticate as the wrong client. Not yet
|
|
102
|
+
fixed; be explicit about which api uuid/secret you pass.
|
|
103
|
+
- **⚠ `Client_Elite.Countries` needs a cleanup migration.** Three rows were auto-created by the sync
|
|
104
|
+
from the NetSuite strings before `Countries` was pre-seeded, with malformed codes/names:
|
|
105
|
+
`_unitedKingdom` → `_united Kingdom` / `_i`, `_canada` → `_canada` / `_C`, `_australia` →
|
|
106
|
+
`_australia` / `_A`. Fix `code`/`name` to `GB`/`CA`/`AU` **without touching `c_netsuiteCountry`** —
|
|
107
|
+
that string is the sync's match key and must keep NetSuite's exact value (leading underscore
|
|
108
|
+
included). Mechanics in the
|
|
109
|
+
[engine doc](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md).
|
|
110
|
+
This matters for Elite specifically because **Elite ships internationally** (see the shipping
|
|
111
|
+
decision on the [profile](../profile.md)).
|
|
112
|
+
- **⚠ The four migrations named above (`2026-08-05b`, `2026-08-06a`, `2026-08-06d`, `2026-08-06e`) are
|
|
113
|
+
NOT in the `dbchanges2` checkout** — absent from `_main`'s working tree and from
|
|
114
|
+
`origin/TRUE-80499` as of 2026-08-12. They may be uncommitted on another machine/branch. Exactly the
|
|
115
|
+
GroWrk situation ("production is ahead of the repo"): until they are committed, the next
|
|
116
|
+
environment or client provisioned from `dbchanges2` re-creates this drift. **Do not re-write them
|
|
117
|
+
from memory — find the branch.**
|
|
118
|
+
- **`Units.locationId` is NULL for all 229 Elite Units.** The sync never populates it; transaction
|
|
119
|
+
location lives on `InventoryAdjustmentItems.locationId` (populated: 6 rows at NetSuite 196, 1 at 5).
|
|
120
|
+
Any Elite inventory table view must read location from the transaction items.
|
|
121
|
+
- **Sales order 169441 was missing from the first backfill** (NetSuite id 4624062, 2024-04-15,
|
|
122
|
+
customer PO `BEGGS/04152024/…`) because its window ran ~15 minutes before the vendor fix. It is the
|
|
123
|
+
worked example for the engine's cursor-advances-past-a-failed-record behaviour — recovered by
|
|
124
|
+
rewinding `NETSUITE_LAST_SYNC_DATETIME_SALES_ORDERS`.
|
|
125
|
+
|
|
126
|
+
## Change history
|
|
127
|
+
|
|
128
|
+
- 2026-08-12 — TRUE-80499: **onboarded Elite to the NetSuite → TOGa Supply importer.** Built the
|
|
129
|
+
wrapper (customer 36443, `isParentCustomer` false, 30-day window, invoices temporarily disabled),
|
|
130
|
+
the schedule entry, 18 `_Model_Elite_*` netsuite trait models, the `_modules/netsuite` opt-in, and
|
|
131
|
+
the two warehouse Locations (NetSuite 196 Chicago, 5 New York). Fixed the two module gaps that
|
|
132
|
+
blocked it (missing shared "Agilant" `Vendors` row → 400 `EV-12` on every customer-PO order, 191
|
|
133
|
+
Sentry events/day, PurchaseOrders 0 → 234; and 27 custom fields with 0 ACL grants → 403 `EV-2`) —
|
|
134
|
+
both generalised into the
|
|
135
|
+
[onboarding workflow](../../../1.0/apps/worker/workflows/onboarding-client-to-netsuite-togasupply-sync.md).
|
|
136
|
+
Full backfill April 2024 → Aug 2026 completed in ~90 minutes (257 SalesOrders / 234 PurchaseOrders /
|
|
137
|
+
201 ItemFulfillments / 29 ItemReceipts / 153 Items / 229 Units / 1042 Contacts / 806 Addresses).
|
|
138
|
+
Recorded the silent `class_exists` fallback in client-model resolution, the duplicate
|
|
139
|
+
`sync_togasupply_elite.php` filename across two integrations, the two active `Apis` rows vs.
|
|
140
|
+
`getClientContext()`'s unordered `LIMIT 1`, the `Countries` cleanup still owed, and that the four
|
|
141
|
+
dbchanges2 migrations are not in the repo checkout. (snaredla)
|
|
@@ -154,6 +154,16 @@ wrong-country address.
|
|
|
154
154
|
select2 state list kept emptying the dropdown — the file's own comment records this. The
|
|
155
155
|
server-side pair check is the guard; the modal stays as-is.
|
|
156
156
|
|
|
157
|
+
### …and that is the ONLY address rule Elite wants (confirmed 2026-08-12)
|
|
158
|
+
|
|
159
|
+
Elite confirmed (via Bala): **no address validation — Elite ships internationally.** So the modal
|
|
160
|
+
must **not** enforce US state/postal formats and **must not** copy Prudential's `_unitedStates`
|
|
161
|
+
hardcoding. The country/state **pair** check above stays, because it is a correctness guard (a 2.0
|
|
162
|
+
`Addresses` row derives its country from its state), not a US-format rule — and the country must still
|
|
163
|
+
resolve to a real `Countries` row for the downstream NetSuite push. Elite also **dictates the shipping
|
|
164
|
+
method on the Sales Order: FedEx only.** Both rules and their reference orders are on the
|
|
165
|
+
[Elite profile](../profile.md#shipping-rules-confirmed-with-elite-by-bala-2026-08-12).
|
|
166
|
+
|
|
157
167
|
## Test harness
|
|
158
168
|
|
|
159
169
|
`test/@srija/Elite Testing/Service Requests/test_elite_desk_service_request.php` (PHP **7.2**)
|
|
@@ -204,6 +214,12 @@ them, so it proves everything *after* the permission check, not the check itself
|
|
|
204
214
|
|
|
205
215
|
## Change history
|
|
206
216
|
|
|
217
|
+
- 2026-08-12 (shipping rules) — Recorded Elite's confirmed shipping decisions (via Bala): the
|
|
218
|
+
**shipping method is dictated on the Sales Order and is FedEx only**, and there is deliberately
|
|
219
|
+
**no address validation** because Elite ships internationally — so the modal must not enforce US
|
|
220
|
+
state/postal formats and must not reuse Prudential's `_unitedStates` hardcoding. The country/state
|
|
221
|
+
pair check stays (correctness, not US-format), and the country must still resolve to a `Countries`
|
|
222
|
+
row for the NetSuite push. (snaredla)
|
|
207
223
|
- 2026-08-12 (later pass) — Recorded the **button gate** itself
|
|
208
224
|
(`isEliteServiceRequestTicket()`: client 163 + department 299 + custom field 88 = `Service
|
|
209
225
|
Request`; the button **replaces** "New Part Request" on those tickets, and the 2.0 uuid comes from
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
title: Elite
|
|
3
3
|
framework: "2.0"
|
|
4
4
|
apps:
|
|
5
|
+
- worker
|
|
5
6
|
- worker2
|
|
6
7
|
- library
|
|
7
8
|
- toga2-supply
|
|
@@ -24,7 +25,9 @@ files:
|
|
|
24
25
|
- _underscore/Model/Elite/SalesOrder.php
|
|
25
26
|
- _underscore/Model/Elite/ServiceRequest.php
|
|
26
27
|
- togadesk/desk/includes/classes/class.ticket.php
|
|
28
|
+
- worker/crons/toga2/netsuite/sync_togasupply_elite.php
|
|
27
29
|
related:
|
|
30
|
+
- features/netsuite-togasupply-sync.md
|
|
28
31
|
- 2.0/apps/worker2/features/elite-freshservice-sync.md
|
|
29
32
|
- 1.0/apps/library/features/elite-freshservice-sync.md
|
|
30
33
|
- features/supply2-scope.md
|
|
@@ -49,6 +52,27 @@ on beta as of 2026-08-10; see [supply2-scope](features/supply2-scope.md). The ba
|
|
|
49
52
|
dbchanges2 **PR #454** (TRUE-80499, snaredla24, 2026-08-05) added the **netsuite** module to
|
|
50
53
|
`Client_Elite/_modules.txt` — the supply-chain schema layer is arriving.
|
|
51
54
|
|
|
55
|
+
**Elite is live on the inbound NetSuite → TOGa Supply sync as of 2026-08-12 (TRUE-80499).** A
|
|
56
|
+
`*/5` worker cron backfilled April 2024 → Aug 2026 in ~90 minutes (257 sales orders, 234 purchase
|
|
57
|
+
orders, 201 item fulfillments, 153 items, 229 units) — this is what adds **`worker`** (1.0) to
|
|
58
|
+
Elite's app scope. Elite-specific config, the still-owed `Countries` cleanup, and the two active
|
|
59
|
+
`Apis` rows hazard are in [netsuite-togasupply-sync](features/netsuite-togasupply-sync.md).
|
|
60
|
+
|
|
61
|
+
## Shipping rules (confirmed with Elite by Bala, 2026-08-12)
|
|
62
|
+
|
|
63
|
+
Two client rules that constrain every order-creation surface (the desk Service Request modal, the
|
|
64
|
+
supply2 order forms, and the NetSuite push):
|
|
65
|
+
|
|
66
|
+
- **Elite dictates the shipping method on the Sales Order, and it is FedEx only** — the shipping-method
|
|
67
|
+
field must accept FedEx carrier values only.
|
|
68
|
+
- **No address validation.** Elite ships **internationally**, so **do not port Prudential's
|
|
69
|
+
`_unitedStates` hardcoding** and the SR modal must **not** enforce US state/postal formats.
|
|
70
|
+
Reference sales orders: **285264, 285379, 285346**.
|
|
71
|
+
|
|
72
|
+
The country still has to resolve to a real `Countries` row for the NetSuite push to succeed — "no
|
|
73
|
+
address validation" means no US-format enforcement, not "any string goes." Full reasoning is recorded
|
|
74
|
+
in `test/@srija/DOCS/ELITE/ELITE_SR_PLAN_BALA.md` and `ELITE_SR_PLAN_COMBINED.md`.
|
|
75
|
+
|
|
52
76
|
Elite's service-request data model: `ServiceRequests` is the record the `service-requests` table
|
|
53
77
|
view is built on (joining Tickets / Customers / ServiceRequestTypes / SalesOrders, with
|
|
54
78
|
`SalesOrders.serviceRequestId = ServiceRequests.id`). **That relationship is 1:N, not 1:1**
|
package/package.json
CHANGED