toga-ai 1.0.857 → 1.0.859
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/2.0/apps/dbchanges2/architecture.md +16 -2
- package/knowledge/clients/nychh/INDEX.md +1 -1
- package/knowledge/clients/nychh/features/netsuite-inventory-adjustment-fulfillment-link.md +50 -20
- package/knowledge/clients/nychh/features/netsuite-transfer-order-import.md +16 -4
- package/knowledge/sessions/2026-09-21-TRUE-82180-staples-asn-late-mhammontree.md +147 -0
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@ project: Database Changes
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: architecture
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
9
|
+
updated: 2026-09-21
|
|
10
10
|
owners: [jcardinal, mhammontree, bala, ajean]
|
|
11
11
|
files:
|
|
12
12
|
- Core/
|
|
@@ -193,6 +193,20 @@ Module folders use the **same `YYYY-MM-DD` naming + ordering rules** as client f
|
|
|
193
193
|
|
|
194
194
|
> **Module names should be lower-case**, and a module value may be a **nested path** that maps to a nested subfolder — e.g. a `_modules.txt` entry of `ai/bdr` would reference `_modules/ai/bdr/`. The repo currently contains an `_modules/Ai/Bdr/` folder whose casing is a **typo** (it should be `_modules/ai/bdr/`); no client references it yet, so there is no live example of a nested module in use. Treat lower-case as the convention going forward.
|
|
195
195
|
|
|
196
|
+
### When a change belongs in a module (authoring rule — pick the folder by WHO gets it)
|
|
197
|
+
|
|
198
|
+
**A change that belongs to a module/integration (NetSuite, Startech, …) MUST go in `_modules/<module>/` — never in `Client/` and never in a single `Client_<Name>/`.** Choose the folder by *which clients should receive the change*:
|
|
199
|
+
|
|
200
|
+
- **Belongs to a module** (a feature/integration) → **`_modules/<module>/`**. Every NetSuite schema/data change — a table, a column, a `CustomRecordFields` + `AclCustomFieldPermissions` registration, an interceptor — goes in **`_modules/netsuite/`**. It then reaches exactly the clients that list `netsuite` in their `_modules.txt` (every NetSuite client), and no others.
|
|
201
|
+
- **Truly platform-wide, every tenant** (not tied to any module) → `Client/`.
|
|
202
|
+
- **One tenant only** → `Client_<Name>/`.
|
|
203
|
+
|
|
204
|
+
**Common mistake (correct it):** putting a module change in `Client/` **over-applies** it to clients that don't use the module; putting it in one `Client_<Name>/` **under-applies** it to a single tenant when every module client needs it. If the change is about NetSuite (or Startech, etc.), the folder is `_modules/<module>/` — full stop. Worked example: `TransferOrderItems.c_netsuiteLineUniqueKey` (the column + its `CustomRecordFields`/ACL registration) belongs to **all** NetSuite clients, so both files live in `_modules/netsuite/`, not in `Client/` + `Client_Nychh/`.
|
|
205
|
+
|
|
206
|
+
**Core is the one exception.** `_modules/<module>/` fans out to **client** databases only, so the part of a module change that lives in the shared **`Core`** DB (`Core.Records` / `Core.RecordFields` registration, Core-level ACL) still goes in `Core/`. Only the **client-side** half (client tables, `CustomRecordFields`, `AclCustomFieldPermissions`, client ACL) goes in `_modules/<module>/`.
|
|
207
|
+
|
|
208
|
+
> **c_ custom-field type:** an integer custom field is registered in `CustomRecordFields` as `type = 'NUMBER'` with `precision = 0` — not `type = 'INTEGER'`. (`precision` is a MySQL reserved word — backtick the column list in the insert.)
|
|
209
|
+
|
|
196
210
|
## HISTORIC folders — always skipped
|
|
197
211
|
|
|
198
212
|
Any folder named **`HISTORIC`** is **ignored by the executor** — it is an archive of old, already-applied change files moved aside to keep the live list short. Present under `Client/HISTORIC/`, `Core/HISTORIC/`, `Logs_Client/HISTORIC/`, and `_modules/netsuite/HISTORIC/`, typically bucketed as `FILES BEFORE <date>` (or by year under `Core/`). **Never** rely on a file inside `HISTORIC` running; **never** add new changes there.
|
|
@@ -215,7 +229,7 @@ This lifecycle is automated by a local browser tool — see the [TOGa 2.0 Client
|
|
|
215
229
|
1. Pick the right folder for the target DB: `Core/`, `Logs/`, `Forecast/`, the shared `Client/` (every tenant), a specific `Client_<Name>/`, `Logs_Client/` (every tenant log), or a module under `_modules/<module>/`.
|
|
216
230
|
2. Name the file `YYYY-MM-DD<letter> - <ShortDescription>.sql` using **today's date** and a **mandatory** lower-case letter right after the date. If no file exists for today in that folder, start at `a` (e.g. `2026-06-12a - <ShortDescription>.sql`). If files already exist for today, use the next free letter after the highest one present (`b`, `c`, …) so yours runs after them.
|
|
217
231
|
3. Write **idempotent / forward-only** SQL that is safe to run in alphabetical order after all prior files in the folder. Follow `rules/common/security.md` — no credentials in the file.
|
|
218
|
-
4. To make a change reach only module-using clients, put it in `_modules/<module>/` and ensure the relevant clients list `<module>` in their `_modules.txt`.
|
|
232
|
+
4. To make a change reach only module-using clients, put it in `_modules/<module>/` and ensure the relevant clients list `<module>` in their `_modules.txt`. **A change that belongs to a module (NetSuite, Startech, …) MUST go here — not in `Client/` (over-applies to non-module clients) and not in a single `Client_<Name>/` (under-applies to one). Only its `Core`-DB half stays in `Core/`.** See *When a change belongs in a module*.
|
|
219
233
|
5. Never edit or re-date an already-applied file — add a new dated file instead. Never put new work in a `HISTORIC` folder.
|
|
220
234
|
6. **Never seed a `uuid` column with MySQL `UUID()` — not even in a bulk insert.** The 2.0 standard requires a fully-random **v4** UUID; `UUID()` is time/MAC-based (v1) and violates it. Since a migration can't call `_String::generateUuid()`:
|
|
221
235
|
- **Single-row insert:** **pre-generate a v4 UUID literal** (a real v4 generator) and hardcode the literal string in the `VALUES` clause.
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
| Doc | Framework | Summary |
|
|
4
4
|
|-----|-----------|---------|
|
|
5
5
|
| [NYCHH transfer-order destinations — location shipping addresses + delivery contacts](features/location-shipping-addresses.md) | 2.0 | NYCHH per-location shipping addresses + delivery contacts for transfer orders; open when a transfer's ship-to/target-location/contact looks wrong or shared. |
|
|
6
|
-
| [NYCHH inventory-adjustment → item-fulfillment linking (
|
|
6
|
+
| [NYCHH inventory-adjustment → item-fulfillment linking (many bridges, REST)](features/netsuite-inventory-adjustment-fulfillment-link.md) | 1.0 | NYCHH import of the NetSuite inventory-adjustment→item-fulfillment link (direct FK, REST); open when a NYCHH adjustment's FK link is missing or the INVENTORY_AD |
|
|
7
7
|
| [NYCHH NetSuite → TransferOrders import (stocking-flag routing, PO bridges, origin location)](features/netsuite-transfer-order-import.md) | 1.0 | The 1.0 NetSuite→`Client_Nychh.TransferOrders` importer (stocking-flag routing, PO bridge links, origin/destination resolution, status map); open for any NYCHH |
|
|
8
8
|
| [NYCHH PO links are UPSTREAM — the downstream SO→PO route returns empty](features/po-number-upstream-direction.md) | 2.0 | Why NYCHH PO Number is empty on the downstream SO→PO route/`_purchaseOrders` field (links are upstream); open when a NYCHH SO shows a PO in the list but not the |
|
|
9
9
|
| [NYCHH Talos AI Assistant Enablement (the only granted tenant)](features/talos-assistant-enablement.md) | 2.0 | How Talos AI assistant is enabled for NYCHH (the only granted tenant) via SurfaceOverrides; open when granting/changing NYCHH's chips or its assistant behaves w |
|
|
@@ -1,18 +1,20 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "NYCHH inventory-adjustment → item-fulfillment linking (
|
|
2
|
+
title: "NYCHH inventory-adjustment → item-fulfillment linking (many bridges, REST)"
|
|
3
3
|
framework: "1.0"
|
|
4
4
|
repo: library
|
|
5
5
|
project: Library
|
|
6
6
|
client: nychh
|
|
7
7
|
type: client-feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
9
|
+
updated: 2026-09-21
|
|
10
10
|
owners: [jcardinal, rgirish]
|
|
11
11
|
files:
|
|
12
12
|
- library/app/api/toga2.php
|
|
13
13
|
- library/app/api/netsuite/rest.php
|
|
14
|
-
-
|
|
14
|
+
- _underscore/Model/Client/ItemFulfillments/InventoryAdjustment.php
|
|
15
|
+
- _underscore/Model/Client/ItemFulfillmentItems/InventoryAdjustmentItem.php
|
|
15
16
|
- worker/crons/toga2/netsuite/sync_togasupply_hh.php
|
|
17
|
+
- worker/crons/toga2/netsuite/verify_inventory_adjustment_fulfillment_link_hh.php
|
|
16
18
|
- worker/crons/toga2/netsuite/diagnose_hh_import_inventory_adjustment.php
|
|
17
19
|
related:
|
|
18
20
|
- ./netsuite-transfer-order-import.md
|
|
@@ -33,29 +35,46 @@ NYCHH import of the NetSuite inventory-adjustment→item-fulfillment link (direc
|
|
|
33
35
|
|
|
34
36
|
For NYCHH stock adjustments, the NetSuite **inventory adjustment** carries a link to the **Item Fulfillment** it corrects, on the multi-select body custom field **`custbody_stock_adjustment`** (internalId **9735**). This feature imports that link into TOGa Supply for **adjustment → item fulfillment → sales order → purchase order** traceability. Per the **2026-08-27 TOGa Supply meeting**; the NetSuite custom field was implemented **<1 year ago**, so older adjustments carry **no** link.
|
|
35
37
|
|
|
36
|
-
**Current design (2026-
|
|
38
|
+
**Current design (2026-09-21): MANY bridges, over REST.** `custbody_stock_adjustment` is a **multi-select** body field — **one** inventory adjustment corrects **MANY** item fulfillments — so the link is **many-to-many**, stored in **two bridge tables**, replacing the single FK columns the prior design used:
|
|
37
39
|
|
|
38
|
-
|
|
40
|
+
| Level | Bridge table | `Core.Records` | `Core.RecordFields` |
|
|
41
|
+
|---|---|---|---|
|
|
42
|
+
| Header | `ItemFulfillments_InventoryAdjustments` | 363 | 2608-2611 |
|
|
43
|
+
| Line | `ItemFulfillmentItems_InventoryAdjustmentItems` | 364 | 2612-2615 |
|
|
44
|
+
|
|
45
|
+
The old single FK columns (`InventoryAdjustments.itemFulfillmentId` / `InventoryAdjustmentItems.itemFulfillmentItemId`, `RecordFields` 2517/2521) are **dropped/retired**. The whole inventory-adjustment sync — plus the on-demand fulfillment/sales-order import it triggers — runs over **REST** (SOAP was ~2 NetSuite round-trips per serial, ~1h for a large serialized adjustment).
|
|
46
|
+
|
|
47
|
+
Like [NYCHH transfer-order import](./netsuite-transfer-order-import.md), the bridge tables and models are **all-client infrastructure** but the **attach is gated NYCHH-only** by `IS_ENABLED_INVENTORY_ADJUSTMENT_FULFILLMENT_LINK` (defined `true` in `sync_togasupply_hh.php`).
|
|
39
48
|
|
|
40
49
|
**Renamed 2026-09-03.** This used to piggyback on the transfer-order routing boolean `IS_ENABLED_TRANSFER_ORDER_STOCKING_FLAG_ROUTING`, which was doing double duty. That constant is gone; the adjustment→fulfillment link now has its **own** flag and is fully independent of transfer-order detection (now a per-client *mode*, not a boolean — see [per-client sync](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)).
|
|
41
50
|
|
|
42
51
|
## How it works
|
|
43
52
|
|
|
44
|
-
### The two
|
|
53
|
+
### The two bridge tables (replaced the single FK columns)
|
|
45
54
|
|
|
46
|
-
| Level |
|
|
55
|
+
| Level | Bridge table | Bridge model (`_underscore`, NEW) | `Core.Records` |
|
|
47
56
|
|---|---|---|---|
|
|
48
|
-
| Header | `
|
|
49
|
-
|
|
|
57
|
+
| Header | `ItemFulfillments_InventoryAdjustments` | `Model/Client/ItemFulfillments/InventoryAdjustment.php` | 363 |
|
|
58
|
+
| Line | `ItemFulfillmentItems_InventoryAdjustmentItems` | `Model/Client/ItemFulfillmentItems/InventoryAdjustmentItem.php` | 364 |
|
|
59
|
+
|
|
60
|
+
Registered in `dbchanges2` this session: records **363/364** + `RecordFields` **2608-2615** + Super User ACL (`Core/2026-09-21a`); old fields **2517/2521** retired (`Core/2026-09-21b`); both bridge tables created all-client + existing 1-to-1 backfilled (`Client/2026-09-21a`); the Base+API four-table ACL chain on 363/364 (`Client/2026-09-21b`); old FK columns dropped, guarded/re-runnable (`Client/2026-09-21c`). The FK declarations were removed from `Model/Client/InventoryAdjustment.php` + `InventoryAdjustmentItem.php`.
|
|
50
61
|
|
|
51
|
-
|
|
62
|
+
### Line matching — by PRODUCT quantity, not a direct link
|
|
63
|
+
NetSuite has **no** direct item-fulfillment-item ↔ adjustment-item link. Match by **product**: the sum of a product's fulfillment-item quantity must equal that product's `quantityAdjustBy` on the adjustment (verified 16/16 on adj 1320). Link **product-to-product**, which handles a product that appears on more than one adjustment line. **FAIL LOUD** on any per-product quantity mismatch — never guess a pairing.
|
|
64
|
+
|
|
65
|
+
### Reconcile every run (self-healing, gated delete)
|
|
66
|
+
Each run **reconciles** both bridges: add missing links, and delete only links NetSuite no longer lists — so a cursor reset re-produces correct data. The delete is **gated**: if a bridge row's key reads back **null** (a read/ACL gap), **throw** instead of deleting — never wipe good links on a partial read.
|
|
67
|
+
|
|
68
|
+
### `_qtyReceived` rerouted through the line bridge
|
|
69
|
+
`Model/Nychh/Item.php` + `PurchaseOrderItem.php` `_qtyReceived` now route through the line bridge using `EXISTS` + an `InventoryAdjustmentItems.itemId` scope, so the many-to-many bridge cannot inflate `SUM(quantityAdjustBy)`.
|
|
52
70
|
|
|
53
71
|
### Attach code — `App_Api_Toga2::attachInventoryAdjustmentToItemFulfillment`
|
|
54
|
-
Lives in `library/app/api/toga2.php`, called from `syncInventoryAdjustmentFromNetsuite` **after the header upsert**, gated on `IS_ENABLED_INVENTORY_ADJUSTMENT_FULFILLMENT_LINK`. It:
|
|
55
|
-
1. reads `custbody_stock_adjustment`
|
|
56
|
-
2.
|
|
57
|
-
3.
|
|
58
|
-
4.
|
|
72
|
+
Lives in `library/app/api/toga2.php`, called from `syncInventoryAdjustmentFromNetsuite` **after the header upsert**, gated on `IS_ENABLED_INVENTORY_ADJUSTMENT_FULFILLMENT_LINK`. **Fully rewritten this session** for the many-to-many design. It:
|
|
73
|
+
1. reads **all** fulfillment ids from `custbody_stock_adjustment` (the multi-select body field — see the shim change below),
|
|
74
|
+
2. **loops every fulfillment**, importing each on demand when missing,
|
|
75
|
+
3. resolves each by `ItemFulfillments.c_netsuiteInternalItemFulfillmentId`,
|
|
76
|
+
4. builds + reconciles the **header** bridge (`ItemFulfillments_InventoryAdjustments`), then
|
|
77
|
+
5. builds + reconciles the **line** bridge (`ItemFulfillmentItems_InventoryAdjustmentItems`) by the product-quantity match above, with product-level fail-loud verify.
|
|
59
78
|
|
|
60
79
|
**Reading the product uuid needs `depth=3` (max-depth omission trap).** Step 4's product match reads `salesOrderItem->item->uuid` from `GET /item-fulfillment-items`. The V2 read engine **omits a non-null FK object that sits at the max requested `depth`** — and a null sibling FK still renders, so the omission is easy to miss. At `depth=2` the grandchild `item` (level 3: `itemFulfillmentItem → salesOrderItem → item`) was dropped → the product-match map was empty → **no `itemFulfillmentItemId` was written**. Fix: request **`depth=3`** *plus* explicit nested fields `salesOrderItem.item.uuid` / `transferOrderItem.item.uuid`. Framework detail: [V2 REST query contract — max-depth FK omission](../../../2.0/apps/api2/features/v2-rest-query-contract.md).
|
|
61
80
|
|
|
@@ -69,7 +88,8 @@ When the linked fulfillment has **not** been imported into TOGA yet, the attach
|
|
|
69
88
|
Threading the on-demand import required the **full item-fulfillment lookup set** — customer, country, state, location, manufacturer, item, all-catalog-items, vendor-item, shipping-carrier, shipping-method, warehouse — to be passed through `syncInventoryAdjustmentFromNetsuite` (signature **expanded 5 → 13 args**) and into `attachInventoryAdjustmentToItemFulfillment`, plus updating the `common_sync_togasupply.php` `INVENTORY_ADJUSTMENTS` call to pass them. **Design rule (developer instruction): all lookups are passed function-to-function**, like the other sync functions — not rebuilt or fetched inside the callee.
|
|
70
89
|
|
|
71
90
|
### The REST-shim fixes this migration required (`library/app/api/netsuite/rest.php`)
|
|
72
|
-
|
|
91
|
+
All framework-level NetSuite facts — full detail on [NetSuite SuiteQL/REST API Reference](../../../1.0/apps/library/features/netsuite-suiteql-api-reference.md):
|
|
92
|
+
- **`custbody_stock_adjustment` is multi-select — parse the whole scalar.** `fetchInventoryAdjustmentById` now splits the comma-separated SuiteQL scalar into `->stockAdjustmentItemFulfillmentInternalIds` (**all** ids, e.g. 17 on adj 1320), not just the first. The scalar already returns every id comma-joined; the old code read `custbody_stock_adjustment[0]` and linked only one.
|
|
73
93
|
- **Item-fulfillment REST shim returned zero line items.** `fetchItemFulfillmentById` fetched fulfillment lines from the REST **record sublist** `GET /record/v1/itemFulfillment/{id}/item?limit=1000&offset=N`. **REST record sublist endpoints reject `limit`/`offset`** (HTTP 400 `INVALID_PARAMETER`), so the shim silently got 0 lines and every fulfillment imported **header-only** — starving all downstream ItemFulfillmentItems and the line-level link. **Fix:** fetch lines via SuiteQL `transactionline` (the pattern every other shim in the file already uses).
|
|
74
94
|
- **Fulfillment SuiteQL returned ~3 rows per fulfilled item.** An Item Fulfillment (`ItemShip`) has three `transactionline` rows per fulfilled item: the order-linked line, a positive "wash" line, and an `isinventoryaffecting='T'` line — only the order-linked line maps to a source-order line. **Fix:** keep only lines with a `previoustransactionlinelink` of type `SalesOrd`/`TrnfrOrd` (via `EXISTS`), and derive `orderLine` from that link's `previousline` (correlated subquery). **`mainline='F'` is the WRONG filter** — NetSuite sometimes marks the first real fulfilled line `mainline='T'`.
|
|
75
95
|
|
|
@@ -87,23 +107,33 @@ php crons/toga2/netsuite/diagnose_hh_import_inventory_adjustment.php <togaClient
|
|
|
87
107
|
```
|
|
88
108
|
(creds = the `CLIENT_CONFIGURATION` client/api/secret from `sync_togasupply_hh.php`.)
|
|
89
109
|
|
|
110
|
+
### Verify harness — `verify_inventory_adjustment_fulfillment_link_hh.php`
|
|
111
|
+
A **manual operator tool** in `worker/crons/toga2/netsuite/` (NEW): imports one adjustment + verifies both bridges vs NetSuite, prints PASS/FAIL. Creds passed as CLI args.
|
|
112
|
+
|
|
90
113
|
### Go-live toggle
|
|
91
114
|
`IS_ENABLED_INTEGRATION_INVENTORY_ADJUSTMENTS` in `sync_togasupply_hh.php` is the go-live switch for this feature on the scheduled cron.
|
|
92
115
|
|
|
93
116
|
### Validation (end-to-end, prod)
|
|
94
|
-
- **
|
|
95
|
-
- **
|
|
117
|
+
- **Many-to-many (2026-09-21):** adj **1320** (NS id 5218734) → **17** header links + **16/16** line products matched, prod-verified end-to-end; INVENTORY_ADJUSTMENTS healthy (IDLE, cursor advancing).
|
|
118
|
+
- **Non-serialized (prior FK design):** linked adjustment **6994071** → fulfillment **6827251** — header + line link both populate.
|
|
119
|
+
- **Serialized (prior FK design):** linked adjustment **6871592** → fulfillment **6868763** (2 serials) — header, line, **and 2 serial units on BOTH** the adjustment item and the fulfillment item all populate correctly.
|
|
96
120
|
|
|
97
121
|
## Gotchas / known issues
|
|
98
122
|
|
|
99
|
-
-
|
|
100
|
-
- **
|
|
123
|
+
- **`custbody_stock_adjustment` is a MULTI-SELECT — one adjustment links MANY fulfillments.** The old code used `custbody_stock_adjustment[0]` and captured only the **first**: on adj **1320** (NS id 5218734) that linked **1 of 17** header + **1 of 16** lines. Always read every id from the scalar and loop.
|
|
124
|
+
- **Older adjustments have no link.** `custbody_stock_adjustment` was added in NetSuite <1 year ago; adjustments predating it will never populate a link — not a bug.
|
|
125
|
+
- **A missing bridge link on a NYCHH adjustment is usually CORRECT, not a bug.** Only records whose NetSuite multi-select body field `custbody_stock_adjustment` is populated get a link — and **0 of 27** NYCHH adjustments carry a value there (only **~1 in 40** recent adjustments account-wide do). So a null header/line FK almost always means NetSuite has **no** stock-adjustment link on that record, not a code fault. The REST shim's read of `custbody_stock_adjustment` as a **scalar SuiteQL column on `transaction`** works when the field IS populated (SuiteQL omits null columns from a row, so an empty field just looks "absent"); the shim only builds the link ref when a value exists. (A clarifying comment was added at the `customFieldList` build in `rest.php` `fetchInventoryAdjustmentById`.)
|
|
101
126
|
- **Reading the product uuid at `depth=2` silently returns nothing** — the level-3 `item` object is omitted at max depth. Use `depth=3` + explicit nested `fields` (see above).
|
|
102
127
|
- **The header upsert's WRITE read-back depth is bounded too — `depth=4`, not `5` (fixed 2026-09-08).** `syncInventoryAdjustmentFromNetsuite`'s `POST|PUT /inventory-adjustments` (`toga2.php` ~L5852) used to request `depth=5`; on a large serialized adjustment api2 spent ~830s building that read-back and the worker's cURL timed out at 900s (0 bytes) → the section froze at `1-RUNNING`. `depth=4` is the safe minimum: the deepest object read off the response is `inventoryAdjustmentItems[] → inventoryAdjustmentItemUnits[] → unit` (scalars only, 3 hops out), and api2 drops a non-null FK **at** the max depth, so `depth=3` would null `unit` and crash. **Do not revive the dead `unit->unitDisposition` block** (`toga2.php` ~6135-6138, ~6177-6184) — it reads `unit` one level deeper and would force `depth=5` again, reopening the timeout. Shared-engine detail: [per-client sync](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md).
|
|
103
128
|
- **A V2 PUT echo that omits `itemFulfillment` does NOT mean the write failed** — verify against the DB / `Logs_<Client>.Api`, not the echoed body.
|
|
104
129
|
- **Empty lookups drop or duplicate items.** The item loop filters to known warehouses and creates items on lookup-miss, so any manual/diagnostic invocation must seed the warehouse/item/manufacturer lookups (the harness does).
|
|
105
130
|
- **A frozen NYCHH `INVENTORY_ADJUSTMENTS` section is NOT necessarily an item-matching / two-catalog problem (verified 2026-09-01).** When the section stuck at `1-RUNNING`, item matching was suspected (wrong of two intentional catalogs). Verified in prod it was fine: item **560** (uuid `4c7b93d2-…`, partNumber **B11B242201**, HYBRID) is the Agilant-catalog item and the **only** item with that part number — no duplicate second-catalog item, `$uuidItem` resolved correctly, and the inventory-adjustment item-create already stamps catalog `Agilant`. The freeze was purely the serial-CASE bug (fix (c) above). **Diagnosis chain (no NetSuite access):** `Client_Nychh.Parameters` (localize the stuck section) → `Logs.Event` `clientId=3` (crash `toga2.php:6080` via `common_sync_togasupply.php:1180` `syncInventoryAdjustmentFromNetsuite`, + `transactionId`) → `Logs_Nychh.Api` on that `transactionId` (untruncated `EV-10`/1062 rejection + payload).
|
|
106
131
|
|
|
132
|
+
## Change history
|
|
133
|
+
- 2026-09-21 — Redesigned to **many-to-many**: `custbody_stock_adjustment` is multi-select (one adjustment → many fulfillments), so two bridge tables (`ItemFulfillments_InventoryAdjustments` rec 363, `ItemFulfillmentItems_InventoryAdjustmentItems` rec 364) replace the single FK columns; attach loops all fulfillments, line-matches by product quantity (fail-loud), reconciles each run with a gated delete. Prod-verified on adj 1320 (17 header + 16/16 line). (jcardinal)
|
|
134
|
+
- 2026-09-03 — Adjustment→fulfillment link moved to its own `IS_ENABLED_INVENTORY_ADJUSTMENT_FULFILLMENT_LINK` flag, independent of transfer-order detection. (jcardinal)
|
|
135
|
+
- 2026-08-29 — Migrated the sync + on-demand import from SOAP to REST. (jcardinal)
|
|
136
|
+
|
|
107
137
|
## Related
|
|
108
138
|
- [NYCHH NetSuite → TransferOrders import](./netsuite-transfer-order-import.md)
|
|
109
139
|
- [NYCHH transfer-order inventory & quantities](./transfer-order-inventory-quantities.md)
|
|
@@ -6,19 +6,18 @@ project: Library
|
|
|
6
6
|
client: nychh
|
|
7
7
|
type: client-feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-09-
|
|
9
|
+
updated: 2026-09-21
|
|
10
10
|
owners: [jcardinal, bala, rgirish]
|
|
11
11
|
files:
|
|
12
12
|
- library/app/api/toga2.php
|
|
13
13
|
- library/app/api/netsuite/rest.php
|
|
14
14
|
- library/app/netsuite.php
|
|
15
|
+
- _underscore/Trait/Netsuite/TransferOrderItem.php
|
|
16
|
+
- _underscore/Model/Nychh/TransferOrderItem.php
|
|
15
17
|
- worker/crons/toga2/netsuite/sync_togasupply_hh.php
|
|
16
18
|
- worker/crons/toga2/netsuite/common_sync_togasupply.php
|
|
17
|
-
- worker/crons/toga2/netsuite/diagnose_hh_stuck_transfer_order.php
|
|
18
|
-
- worker/crons/toga2/netsuite/probe_import_single_order_hh.php
|
|
19
19
|
- worker/crons/toga2/netsuite/relink_transfer_order_pos_hh.php
|
|
20
20
|
- dbchanges2/Core/2026-08-27a - TransferOrderUuidSearchableIdentifier.sql
|
|
21
|
-
- dbchanges2/Client/2026-08-28a - TransferOrderItemsTimestampColumns.sql
|
|
22
21
|
- dbchanges2/Client_Nychh/2026-09-16a - TransferOrderBridgeAcl.sql
|
|
23
22
|
related:
|
|
24
23
|
- ../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md
|
|
@@ -126,6 +125,16 @@ Verified in production 2026-09-02 (while building the **outbound** push): on a N
|
|
|
126
125
|
- null-guarded the dedup lookup (`?? null`),
|
|
127
126
|
- the header IF POST sets `transferOrder` when applicable.
|
|
128
127
|
|
|
128
|
+
#### Line match froze ITEM_FULFILLMENTS (issue 587) — matched by the wrong line number, fixed 2026-09-21
|
|
129
|
+
**Root cause** (proven on fulfillment 4437340 / NetSuite SalesOrd 4307193 = TransferOrder #157140): a transfer-order fulfillment line was matched by `lineNumber`, but `TransferOrderItems.lineNumber` stores the NetSuite **stable line** (`tl.id`) while the fulfillment shim's `orderLine` is the source **`linesequencenumber`** — they differ on **edited** orders, so the match missed → fail-loud throw at `toga2.php` ("could not match fulfillment line to a sales/transfer order item") → the `ITEM_FULFILLMENTS` window collapsed. Sales orders already dodge this by matching on the stable `c_netsuiteLineUniqueKey`; transfer orders had no such key.
|
|
130
|
+
|
|
131
|
+
**Fix (mirrors the sales-order pattern):**
|
|
132
|
+
- new trait `_underscore/Trait/Netsuite/TransferOrderItem.php` declaring `c_netsuiteLineUniqueKey` (mirrors `_Trait_Netsuite_SalesOrderItem`), consumed by a new `_underscore/Model/Nychh/TransferOrderItem.php` override (mirrors `Model/Nychh/SalesOrderItem.php`).
|
|
133
|
+
- `toga2.php` `syncItemFulfillmentFromNetsuite` matches the transfer-order fulfillment line by `c_netsuiteLineUniqueKey` **first**, falling back to `lineNumber`; `syncTransferOrderFromNetsuite` stamps `c_netsuiteLineUniqueKey` on `TransferOrderItems` POST/PUT + change-check so a re-import backfills existing rows.
|
|
134
|
+
- `dbchanges2/_modules/netsuite/` — the column add (`TransferOrderItems.c_netsuiteLineUniqueKey` INT UNSIGNED, all NetSuite clients) + `CustomRecordFields` registration on record **313** (`type='NUMBER'`, `precision=0`) + Base `AclCustomFieldPermissions` write grant. **These belong to ALL NetSuite clients, so they live in `_modules/netsuite/`** — not `Client/` or `Client_Nychh/` (see [dbchanges2 architecture — when a change belongs in a module](../../../2.0/apps/dbchanges2/architecture.md)).
|
|
135
|
+
|
|
136
|
+
**Deploy needs a SALES_ORDERS cursor rewind (Eastern)** to re-import transfer orders and stamp `c_netsuiteLineUniqueKey` on already-imported TOs (the fulfillment importer only imports a source order when missing). After rewind: issue 587 cleared, `ITEM_FULFILLMENTS` drained past 4437340, 0 errors, section IDLE.
|
|
137
|
+
|
|
129
138
|
### Transfer-order status map
|
|
130
139
|
`syncTransferOrderFromNetsuite`'s status switch was expanded to cover the real NetSuite statuses: **Billed**, **Pending Billing**, **Pending Billing-Partially Fulfilled** → `received` / `inTransit`. The **default** now **logs and defaults to `'pending'`** instead of throwing. An unmapped "Billed" status was previously throwing (and being swallowed), which is why order 128829 never imported.
|
|
131
140
|
|
|
@@ -207,6 +216,9 @@ Found by a fleet sweep of `NETSUITE_EXECUTION_MODE_*` across all 22 netsuite cli
|
|
|
207
216
|
- **The `custbody_stock_adjustment` inventory-adjustment link is on its OWN flag now.** `attachInventoryAdjustmentToItemFulfillment()` used to piggyback on the old routing boolean; it is now gated by `IS_ENABLED_INVENTORY_ADJUSTMENT_FULFILLMENT_LINK` (NYCHH-only) — see [inventory-adjustment → item-fulfillment linking](./netsuite-inventory-adjustment-fulfillment-link.md).
|
|
208
217
|
- **`custbody_stocking_order` must reach the shim.** `fetchSalesOrderById()` now carries the field via `customFieldList`; a missing custom field read returns null → the order routes as a transfer order (unchecked = transfer). Verify the field is populated in NetSuite before trusting the split.
|
|
209
218
|
|
|
219
|
+
## Change history
|
|
220
|
+
- 2026-09-21 — Fixed issue 587 (`ITEM_FULFILLMENTS` freeze): transfer-order fulfillment line was matched by `lineNumber` (= NetSuite stable `tl.id`) vs the shim's `orderLine` (= source `linesequencenumber`), which diverge on edited orders. Added `c_netsuiteLineUniqueKey` (new trait + Nychh model, `_modules/netsuite/` column/registration); match by it first, fall back to `lineNumber`. Needs a SALES_ORDERS cursor rewind on deploy. (jcardinal)
|
|
221
|
+
|
|
210
222
|
## Related
|
|
211
223
|
- [NetSuite → TOGa Supply per-client sync (engine)](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md)
|
|
212
224
|
- [toga2 API client and bridge](../../../1.0/apps/library/features/toga2-api-client-and-bridge.md)
|
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: session
|
|
3
|
+
slug: TRUE-82180-staples-asn-late
|
|
4
|
+
title: Staples ASNs sent days-to-weeks late — root cause investigation
|
|
5
|
+
author: mhammontree
|
|
6
|
+
repos: [worker, library, dbchanges]
|
|
7
|
+
framework: "1.0"
|
|
8
|
+
client: staples
|
|
9
|
+
status: active
|
|
10
|
+
created: 2026-09-21
|
|
11
|
+
updated: 2026-09-21
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Session: TRUE-82180-staples-asn-late
|
|
15
|
+
**Date:** 2026-09-21
|
|
16
|
+
**Project/Repo:** worker (1.0)
|
|
17
|
+
**Task:** Investigate why 856 ASNs are sent to Staples days-to-weeks after we ship and after we invoice — example PO 478KVU invoiced 9/8, ASN sent 9/18.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## What WORKED
|
|
22
|
+
|
|
23
|
+
- **Root cause identified and proven.** `worker/crons/sync/staples/staples_asn_netsuite.php` L48 sets
|
|
24
|
+
`$modifiedAfterEpochTime = $currentEpochTime - 3600` and the cron runs `50 * * * *`. The selection window
|
|
25
|
+
equals the run interval, so each shipment gets exactly ONE chance. No state, no backlog, no retry. A missed
|
|
26
|
+
shipment is only sent later when something edits the item fulfillment in NetSuite.
|
|
27
|
+
- **Confirmed the driver is a NetSuite `lastModifiedDate onOrAfter` search** — `App_NetSuite::listAllItemFulfillmentsForCustomer()`
|
|
28
|
+
in `library/app/netsuite.php` L1633-1679. It already accepts an after AND before timestamp, so no library
|
|
29
|
+
change is needed to change the window.
|
|
30
|
+
- **Proved it on the example order.** `Logs`.`FileLog` (env `legacy`), job `STAPLES_ASN_OUT_ORDER`:
|
|
31
|
+
- PO 478KVU: 850 imported 2026-09-01 12:45; ASN cXML says `shipmentDate="2026-09-02T21:00:00.000-07:00"`
|
|
32
|
+
(9/3 Central) and `noticeDate="2026-09-17T11:53:21.000-07:00"`.
|
|
33
|
+
- `noticeDate` is `$order->lastModifiedDate`. The fulfillment was edited 9/17 13:53 Central; the 14:52 run
|
|
34
|
+
sent it. 15 days after ship.
|
|
35
|
+
- **Sized the problem.** First-ASN vs `shipmentDate`, "late" = >1 day. By ship date:
|
|
36
|
+
09-01 49 on-time/17 late; 09-02 5/16; 09-03 4/13; 09-07 11/30; 09-08 15/11; 09-09 63/11; 09-10 9/9;
|
|
37
|
+
09-13 0/24; 09-14 0/27; 09-15 8/5. ~half late every day. 40+ orders shipped 8/18–9/07 all first-ASN'd in
|
|
38
|
+
one burst 9/15–9/18.
|
|
39
|
+
- **Found a second, separate bug: duplicate sends.** Dedupe at L453 checks SFTP `/archive` only, never `/out`.
|
|
40
|
+
PO 478KVU written 10 times in 10 hours. PO 755KVB written as `856_755KVB_1_*` repeatedly on 9/11 AND again
|
|
41
|
+
as `856_755KVB_2_1789602977` on a later day — same single item, same tracking `488192721813`. The
|
|
42
|
+
`<Package identifier="...">` is `mt_rand(1000,10000)` so it differs every send.
|
|
43
|
+
- **Proved `partialPackIdentifier` is unusable as a dedupe key.** It restarts at 1 every run and depends on
|
|
44
|
+
iteration order, so the `856_<PO>_<n>` prefix check against `/archive` cannot recognise an already-sent
|
|
45
|
+
shipment.
|
|
46
|
+
- **Ruled out the overlap guard.** `App_Framework::cronInitialization()` (`library/app/framework.php` L689-711)
|
|
47
|
+
calls `exitIfProcessRunning()` BEFORE inserting the `Common`.`CronJobExecutions` check-in row. Check-in rows
|
|
48
|
+
exist at `:50` for all 24 hours on 9/2, 9/3, 9/4. Every run started.
|
|
49
|
+
- **Ruled out reuse of existing tables.** `Common`.`AdvanceShipNotices` (163,145 rows) is an old INBOUND vendor
|
|
50
|
+
ASN feed (ACER/LENOVO/APPLE/LEXMARK), dead since 2022-05-17, wrong grain and wrong direction.
|
|
51
|
+
`TOGA_Staples` DB is the Staples retail-services app (Stores, Incidents, ServiceRequests) — unrelated.
|
|
52
|
+
- **CTO agent review completed** — verdict DISAGREE-WITH-ALTERNATIVE. Agreed on the state table in `Common`,
|
|
53
|
+
rejected widening the lookback, insisted on a different order of work.
|
|
54
|
+
- **Leadership write-up delivered** as a Claude doc:
|
|
55
|
+
https://claude.ai/code/artifact/95957a53-cd23-4ba6-a850-83b143b037b0
|
|
56
|
+
(private — Mark must share it from the Share menu before anyone else can open it).
|
|
57
|
+
|
|
58
|
+
## What did NOT work — DO NOT RETRY THESE
|
|
59
|
+
|
|
60
|
+
- **CTO claim that a stale `$itemCxmlConfirmation` is emitting package-less ASNs — FALSE, disproven.**
|
|
61
|
+
Query over all `STAPLES_ASN_OUT_ORDER` rows since 2026-08-20: `totalAsns 8831, noPackageTag 0, noItems 0`.
|
|
62
|
+
Every ASN sent contains both `<Package ` and `<ShipNoticeItem `. Do not re-open this theory. The leaked
|
|
63
|
+
variable is still a real code defect, it is just not firing in production.
|
|
64
|
+
- **CTO claim of cross-fulfillment item bleed — real in code, NOT demonstrated in data.** `$itemCXMLArray` is
|
|
65
|
+
initialised at L136, outside the `foreach ($partialItemFulFil as $order)` loop, so a second fulfillment would
|
|
66
|
+
carry the first's items. But only 2 POs since 9/1 have more than one pack (755KVB, 856KWB) and 755KVB's
|
|
67
|
+
`_1` and `_2` files are byte-identical apart from the random package identifier — one item each. Not proven.
|
|
68
|
+
- **Widening the lookback window to 7 days — rejected on numbers.** ~30 shipments/day x ~20 serials, one SOAP
|
|
69
|
+
`get` per serial (L326) plus `getSalesOrder` + `get itemFulfillment` + `listAllItemFulfillmentsForSalesOrder`
|
|
70
|
+
per fulfillment ≈ 4,600 SOAP calls per hourly run, 40–75 min runtime. It would collide with the next `:50`
|
|
71
|
+
run and `exitIfProcessRunning` would then start eating hours. Also re-writes the full cXML BLOB into
|
|
72
|
+
`FileLog` every pass (the FileLog write at L446 happens BEFORE the dedupe check at L453).
|
|
73
|
+
- **Using `CronJobExecutions.dtCheckOut` to detect runs that died mid-loop — does not work.** The script never
|
|
74
|
+
calls `App_Framework::cronFinished()` or `App_Database::closeConnections()`, so EVERY row has
|
|
75
|
+
`dtCheckOut = NULL`. A completed run and a crashed run are indistinguishable. Do not read NULL as "crashed".
|
|
76
|
+
- **Looking for "ASN Not Generated" emails to find failures — there are none, ever.** `Vision_Log`.`Email`
|
|
77
|
+
searched back to 2026-08-25 for `emailObject LIKE '%ASN Not Generated%'`: 0 rows. The else-branch at L469
|
|
78
|
+
has never fired. Late orders are dropped BEFORE reaching that check.
|
|
79
|
+
- **`toga_query` blocks the `REPLACE` keyword.** A `LENGTH(x)-LENGTH(REPLACE(x,...))` counting query is
|
|
80
|
+
rejected with "Forbidden keyword 'REPLACE' found in query". Use `LOCATE`/`SUBSTRING` or fetch and eyeball.
|
|
81
|
+
- **`Logs` schema is on the `legacy` environment, not `production`.** `toga_query` with
|
|
82
|
+
`environment='production'` gives "Table 'Logs.FileLog' doesn't exist". Also `FileLog` has `dtStamp`, NOT
|
|
83
|
+
`dtCreated`.
|
|
84
|
+
|
|
85
|
+
## Not tried yet (candidates for next session)
|
|
86
|
+
|
|
87
|
+
- Confirm the actual per-order silent skip. Strongest candidate is the `if ($matchedFile)` gate at L160 —
|
|
88
|
+
it requires OUR OWN 855 ack to already be in SFTP `/archive`, i.e. Staples must have collected it before we
|
|
89
|
+
will send the ASN. Customer-controlled, no log, no email. Needs SFTP pickup timing to confirm.
|
|
90
|
+
- Test the mid-run-death theory: per hour, compare the count of fulfillments the NetSuite search returned
|
|
91
|
+
against the `FileLog` row count for that run. A truncated tail proves a warning killed the cron.
|
|
92
|
+
- Size the item-bleed risk: find sales orders with multiple fulfillments AND multiple line items, and check
|
|
93
|
+
whether any delivered ASN carried another fulfillment's items.
|
|
94
|
+
- Nothing has been written. No code changed, no SQL written, no branch commits.
|
|
95
|
+
|
|
96
|
+
## Current file state
|
|
97
|
+
|
|
98
|
+
| File | Status | Notes |
|
|
99
|
+
|------|--------|-------|
|
|
100
|
+
| `worker/crons/sync/staples/staples_asn_netsuite.php` | **unchanged** | Read only. 496 lines. Branch `TRUE-82180` checked out, working tree clean. |
|
|
101
|
+
| `dbchanges/` | **unchanged** | Branch `TRUE-82180` checked out. No file created. Target would be `dbchanges/Common/MH/<date>.sql`. |
|
|
102
|
+
| `library/` | **unchanged** | On branch `TRUE-81955` (another ticket). Not needed for this work. |
|
|
103
|
+
| Claude doc (leadership findings) | created | https://claude.ai/code/artifact/95957a53-cd23-4ba6-a850-83b143b037b0 — private, not yet shared. |
|
|
104
|
+
|
|
105
|
+
## Decisions made
|
|
106
|
+
|
|
107
|
+
- **Ticket switched to pseudocode-only, for Jeff to review.** Mark changed the scope late in the session: the
|
|
108
|
+
ticket asks for an investigation, not a fix. Pseudocode + the findings breakdown go in the ClickUp task's
|
|
109
|
+
pseudocode field. **No code is to be written until Jeff reviews.**
|
|
110
|
+
- **This may be solved the 2.0 way instead of fixing 1.0.** Open possibility raised by Mark. Do not assume the
|
|
111
|
+
1.0 design below is the agreed path — it is the 1.0 option, not the decision.
|
|
112
|
+
- **If fixed in 1.0: a state table, not a wider window.** Rationale: the expensive work (per-serial SOAP calls)
|
|
113
|
+
must be driven by "not yet sent", not by "recently modified". Rejected: widening the lookback (see failures
|
|
114
|
+
above); rejected: keeping SFTP folder listings as the record of what was sent.
|
|
115
|
+
- **Home would be `Common` (`db_common`), table `StaplesAdvanceShipNotices`.** Rationale: `Common` already
|
|
116
|
+
holds cross-app NetSuite state (`App_Model_Common_NsSalesOrderItem`, `NsSerialNumber`, `NsTrackingNumber`)
|
|
117
|
+
and `CronJobExecutions`. Rejected `Logs` — it is append-only under retention pressure (API logs already cut
|
|
118
|
+
to 30 days), and mutable state must not sit where a purge job can delete it. Rejected `TOGA_Staples` —
|
|
119
|
+
wrong business domain. Name must NOT be `AdvanceShipNotices`, that is taken.
|
|
120
|
+
- **Key on `itemFulfillmentInternalId` with a UNIQUE constraint; rename the file to
|
|
121
|
+
`856_<PO>_<itemFulfillmentInternalId>`.** Rationale: stable NetSuite-assigned id, one ASN per fulfillment,
|
|
122
|
+
and the `/out` + `/archive` checks become exact matches instead of prefix guesses. This is the fix for the
|
|
123
|
+
duplicates.
|
|
124
|
+
- **Sequencing, if built (CTO's main correction).** 1) Fix the leaked per-iteration variables, the unchecked
|
|
125
|
+
`$conn->put()` return at L467, move the `FileLog` write after the dedupe, add `cronFinished()` +
|
|
126
|
+
`closeConnections()`, log every skip with a reason. 2) Add the table and switch the filename key. 3) Seed the
|
|
127
|
+
table as already-sent from `FileLog`, run one cycle log-only. 4) Only then enable retry, with an attempt cap
|
|
128
|
+
and an age alert. Rationale: enabling retry before step 1 would re-send bad ASNs at scale, and an empty table
|
|
129
|
+
on day one would dump months of history on Staples.
|
|
130
|
+
|
|
131
|
+
## Blockers
|
|
132
|
+
|
|
133
|
+
1. **Waiting on Jeff's review of the pseudocode.** Scope is now pseudocode only; no implementation.
|
|
134
|
+
2. **Unanswered, needed from Staples:** does a repeated `ShipNoticeRequest` with `operation="new"` and the same
|
|
135
|
+
`shipmentID` get ignored, or treated as a new shipment? We have been sending repeats for weeks, so this
|
|
136
|
+
also decides whether existing damage needs cleanup.
|
|
137
|
+
3. **Unanswered, business decision:** should the 855 gate at L160 exist at all? It hands our ASN timing to the
|
|
138
|
+
customer's file-pickup schedule.
|
|
139
|
+
|
|
140
|
+
## Exact next step
|
|
141
|
+
|
|
142
|
+
> Write the pseudocode into the ClickUp task TRUE-82180 pseudocode field (Mark has already put the findings
|
|
143
|
+
> breakdown there), covering the 4-step sequence in "Decisions made", and hold for Jeff's review. Do NOT write
|
|
144
|
+
> code in `worker` or `dbchanges` — both branches are clean and must stay that way until Jeff decides 1.0 vs 2.0.
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
_Saved by /session-save on 2026-09-21_
|
package/package.json
CHANGED