toga-ai 1.0.945 → 1.0.947
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/2.0/apps/_underscore/features/autoincrement-record-numbering.md +3 -2
- package/knowledge/2.0/apps/_underscore/features/units-for-items-for-purchase-orders.md +6 -2
- package/knowledge/2.0/apps/api2/features/environment-variable-drives-underscore-branch.md +4 -2
- package/knowledge/2.0/apps/api2/features/request-logging.md +12 -1
- package/knowledge/2.0/apps/worker2/features/nychh-asset-tag-backfill.md +29 -1
- package/knowledge/clients/compass-usa/INDEX.md +2 -0
- package/knowledge/clients/compass-usa/features/corporate-vs-field-laptop-personas.md +77 -0
- package/knowledge/clients/compass-usa/features/location-hierarchy-and-sectors.md +128 -0
- package/knowledge/clients/nychh/features/transfer-order-notification-emails.md +29 -5
- package/knowledge/clients/prudential/features/dell-lch-iop-transmissions.md +12 -0
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-05
|
|
10
10
|
owners: ["bala", "rgirish"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model.php
|
|
@@ -57,7 +57,7 @@ public $number = [
|
|
|
57
57
|
|
|
58
58
|
## Gotchas
|
|
59
59
|
|
|
60
|
-
- **⚠ Old path race (still live on prod until this ships):** `SELECT MAX+1` then a separate INSERT → concurrent creates pick one number → loser gets 1062 `Duplicate entry 'SR131021' for key 'ServiceRequests.number'` → HTTP 400 `EV-10`, record lost. The number in the error belongs to the **winner**.
|
|
60
|
+
- **⚠ Old path race (still live on prod until this ships):** `SELECT MAX+1` then a separate INSERT → concurrent creates pick one number → loser gets 1062 `Duplicate entry 'SR131021' for key 'ServiceRequests.number'` → HTTP 400 `EV-10`, record lost. The number in the error belongs to the **winner**. **Staggering or delaying callers does NOT fix it** — the unsafe window is wider than a second: on dev-sandbox 2026-09-30 a losing request read at `11:45:02` while the winner committed at `11:45:03`. Prod rate on the old path: **8 failed `POST /v2/service-requests` for Prudential between 2026-09-10 and 2026-09-21**. Only the counter removes it.
|
|
61
61
|
- **⚠ A prefix on a SHARED model is not unique across clients** — every tenant starts at `TA100000`. Never key a cross-client id on `number`; NetSuite account 1095849 is shared, so pushes key `externalId` on uuid (`toga-so-<uuid>` / `toga-to-<uuid>`).
|
|
62
62
|
- **SA / TA numbers are placeholders** — overwritten by NetSuite `tranId` on write-back (~12s). Never build a key or match on `number`. Keeping a Toga number permanently needs **two** changes: a guard in `writeBackNetSuiteId()` **and** dropping `number` from the PUT in `library/app/api/toga2.php` (~L4295) and `_underscore/Trait/Netsuite/TransferOrder.php` (tried 2026-09-02, reverted).
|
|
63
63
|
- **`FIELD_AUTOINCREMENT` is special-cased in `buildSqlFieldValue()`**, so a null value does not collapse a `load()` WHERE the way a null `FIELD_CHAR` does — see [_underscore architecture](../architecture.md).
|
|
@@ -79,6 +79,7 @@ Beta (`api.beta.togahub.com` → sandbox-dev), 2026-10-02: before fix 8/10 concu
|
|
|
79
79
|
- [NetSuite SalesOrder outbound push](../../worker2/features/netsuite-salesorder-outbound-push.md)
|
|
80
80
|
|
|
81
81
|
## Change history
|
|
82
|
+
- 2026-10-05 — Noted the old `MAX()+1` window exceeds one second, so staggering callers is not a workaround; added the prod failure rate still seen on `_production`/`_sandbox-client`. (bala)
|
|
82
83
|
- 2026-10-02 — Replaced unlocked `MAX()+1` with a per-tenant `Parameters` counter on its own committed connection (`DB_CLIENT_SEQUENCE`), self-seeding, 1062 resync retry; removed Prudential `usleep` stopgap. On `_beta`/`_sandbox-dev` only. (rgirish)
|
|
83
84
|
- 2026-09-22 — Recorded that the `MAX+1` generator is not concurrency-safe. (rgirish)
|
|
84
85
|
- 2026-09-02 — `TA` / 100000 added to the shared `_Model_Client_TransferOrder`. (bala)
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-05
|
|
10
10
|
owners: ["bala", "tcox", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Client/ServiceRequest.php
|
|
@@ -21,6 +21,7 @@ related:
|
|
|
21
21
|
- ./tracking-number-bridges.md
|
|
22
22
|
- ../../../clients/elite/features/supply2-tableview-config-drift.md
|
|
23
23
|
- ../../../clients/prudential/features/supply-orders-csv-export.md
|
|
24
|
+
- ../../../clients/nychh/features/transfer-order-notification-emails.md
|
|
24
25
|
---
|
|
25
26
|
|
|
26
27
|
How unit (serialized inventory) data links to sales-order/purchase-order line items behind the `units-for-items-for-purchase-orders` view; open when units don't show, a unit-to-order path is needed, or a view here 500s.
|
|
@@ -131,7 +132,7 @@ Header-level vs item-level bridges, and the level each transaction is anchored a
|
|
|
131
132
|
- **The view only walks the receipt path.** A line whose units exist solely via the fulfillment path (shipped from stock, never received against the bridged PO line) shows **0 units** even though units physically exist. No data backfill can surface these — only a resolver/view change that also walks the fulfillment path will.
|
|
132
133
|
- **Missing item bridge → no units**, even when units were received. Without a `SalesOrderItems_PurchaseOrderItems` row, the receipt path has no starting link.
|
|
133
134
|
- **A present bridge is not sufficient.** If the bridged PO line has no `ItemReceiptItemUnits`, the receipt path still resolves to nothing. The **fulfillment-chain** analog (NYCHH rebuild): a PO line with a bridge **and** a fulfillment but **zero `ItemFulfillmentItemUnits`** (no serialized units captured) correctly shows an **empty grid** — that is not a bug, just a line with no serials recorded.
|
|
134
|
-
- **`inventoryType` determines unit existence**: `SERIALIZED` and `HYBRID` items can produce unit records; `NON_SERIALIZED` (bulk/consumable) never do. `HYBRID` items fulfilled as a plain quantity (no serial capture) also produce no units — legitimately blank.
|
|
135
|
+
- **`inventoryType` determines unit existence**: `SERIALIZED` and `HYBRID` items can produce unit records; `NON_SERIALIZED` (bulk/consumable) never do. `HYBRID` items fulfilled as a plain quantity (no serial capture) also produce no units — legitimately blank. **Only `SERIALIZED` is reliable enough to *expect* units from.** `Items.inventoryType` is `enum('SERIALIZED','NON_SERIALIZED','HYBRID')` and the constants live on `_Model_Client_Item` (`INVENTORY_TYPE__SERIALIZED`) — compare against those, never a raw string. Measured on prod `Client_Nychh` fulfillment lines: NON_SERIALIZED **0 of 2,372** ever have units, HYBRID **510 of 5,743**, SERIALIZED **1,242 of 1,389**. So any "this line is missing its units/asset tags" alert must be gated on SERIALIZED or it fires on every accessory — see [NYCHH transfer-order emails](../../../clients/nychh/features/transfer-order-notification-emails.md).
|
|
135
136
|
- **Receipt-path counts are PO-line-wide.** Units surfaced reflect everything received against the bridged PO line, which may exceed or differ from what was actually fulfilled to a given sales order (over/under-attribution), because attribution is by PO line, not by the fulfillment that shipped the unit.
|
|
136
137
|
- **A 500 here is usually stale per-client view config, not missing data.** This view's joins were repointed onto the bridge records (319 tracking, 289 `SalesOrderItems_PurchaseOrderItems`) per client. A client that never got that migration 500s on `table-views/meta` (*"Model build for primary key id '1431' for `_Model_Core_RecordField` … 0 rows"*) and then on the data query (*"Unknown column `SalesOrderItems_PurchaseOrderItems.salesOrderItemId` in 'on clause'"*). Confirmed on `Client_Elite` (view 17) — see [Elite TableView config drift](../../../clients/elite/features/supply2-tableview-config-drift.md). Check the view config before investigating the linkage paths above.
|
|
137
138
|
- **A join can also be broken by a `parentRecordFieldId` that belongs to a record the view never joins.** Distinct from the dead-id case above: the id **exists**, it is just on the wrong record. Found 2026-08-17 - a join in this view carried `parentRecordFieldId` **92**, which belongs to record **289** (`SalesOrderItemPurchaseOrderItems`), a record the view does not join anywhere, and it **500'd** the view. **Proven by bisection:** all 23 other joins returned 200; adding this one returned 500. No fields or joins depended on it, so the fix was a targeted `DELETE`. The same row is present in **Elite, Compass and Prudential**. When a view 500s and the ids all resolve, check that every `parentRecordFieldId` sits on a record the view actually joins - and bisect the joins rather than reading the SQL.
|
|
@@ -144,3 +145,6 @@ Header-level vs item-level bridges, and the level each transaction is anchored a
|
|
|
144
145
|
- [tableview-joins](./tableview-joins.md)
|
|
145
146
|
- [Elite supply2 TableView config drift](../../../clients/elite/features/supply2-tableview-config-drift.md)
|
|
146
147
|
- [Prudential Sales Orders CSV export](../../../clients/prudential/features/supply-orders-csv-export.md)
|
|
148
|
+
|
|
149
|
+
## Change history
|
|
150
|
+
- 2026-10-05 — Tightened the `inventoryType` gotcha: named the `_Model_Client_Item::INVENTORY_TYPE__SERIALIZED` constant and added the prod NYCHH counts (NON_SERIALIZED 0 of 2,372 lines ever carry units, HYBRID 510 of 5,743, SERIALIZED 1,242 of 1,389), so a "missing units" alert is gated on SERIALIZED. (bala)
|
|
@@ -6,7 +6,7 @@ project: API
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-05
|
|
10
10
|
owners: ["bala", "mhammontree", "apeterson", "jcardinal", "tcox", "rgirish"]
|
|
11
11
|
files:
|
|
12
12
|
- api2/.ebextensions/git.php
|
|
@@ -89,6 +89,7 @@ So on `api.beta.togahub.com` and `api.dev.sandbox.togahub.com` the code fataled
|
|
|
89
89
|
- Before using a framework constant/method/property you (or a teammate) added recently, run **`git branch -a --contains <commit>`** in `_underscore` and confirm it is on the branch of **every** tier this code will run on — not just `_production`.
|
|
90
90
|
- When the symbol is not everywhere yet, either **wait for it to land on all tiers**, or **guard the reference** (`property_exists()` / `defined()` / a local constant) so an older framework falls back rather than fatals.
|
|
91
91
|
- **Never reference a not-yet-universal symbol from inside a `catch`/error path.** The worst place: the fatal masks the very error you were handling, and the incident is diagnosed against the wrong symptom.
|
|
92
|
+
- **A declared MODEL PROPERTY is such a symbol too, and it is the worst case: a whole-environment outage.** api2 assigning a field the deployed model does not declare throws from `_Model->__set()` before any response is built. Beta ran 639 x 500 `EO-1` on this for 19 hours (2026-09-21/22) — every client-scoped request, `/v2/auth/api` included. Applying the column migration does **not** help; only the branch does. Details: [V2 request logging](./request-logging.md).
|
|
92
93
|
|
|
93
94
|
## Gotchas
|
|
94
95
|
|
|
@@ -105,7 +106,7 @@ So on `api.beta.togahub.com` and `api.dev.sandbox.togahub.com` the code fataled
|
|
|
105
106
|
- **The clone is from a moving branch, not a pinned commit** (see [architecture.md](../architecture.md) critical rules) — two deploys of the same api2 commit can behave differently because `_underscore` moved underneath it.
|
|
106
107
|
- **⚠ The 1.0 `library` repo is NOT reachable from api2.** `include_path` is `".:/var/www/html/_underscore"` only; `library` is not deployed alongside api2 and api2 contains no reference to it. The two autoloaders are also incompatible — 1.0 lowercases the whole class path, 2.0 is case-sensitive PascalCase. **Consequence:** a 1.0 helper such as `App_Api_TogaIQ` cannot be called from any 2.0 API flow; integration code for 2.0 must be written in `_underscore`. (Contrast: the 1.0 **tools** app works cleanly because its helper *and* config live in the same repo, and it clones `library` from a **fixed** branch, `_production`, rather than an `ENVIRONMENT`-derived one.)
|
|
107
108
|
- **⚠ A `git.<env>.json` `"branch"` key can pin a tier to the WRONG `_underscore` branch, silently.** When a code fix pushed to a branch does not appear after an api2 deploy, before debugging the code verify which branch that EB environment's `.ebextensions/git.<env>.json` actually clones: a present `"branch"` value overrides the `'_' . strtolower(ENVIRONMENT)` default. **Concrete (fixed 2026-08-18):** `git.sandbox-client.json` had `"branch": "_production"`, so the `sandbox-client` environment ran `_underscore` from `_production` — never `_sandbox-client` — which is why fixes merged to `_sandbox-client` never showed up there. Confirmed two ways: the config value, and live runtime behavior matching `_production` (unfixed) rather than `_sandbox-client` (fixed). Fix = remove the `"branch"` key so it falls back to `_sandbox-client`.
|
|
108
|
-
- **✅ A MISSING `"branch"` key is CORRECT, not a misconfiguration — do not "fix" it by adding one back.** Re-confirmed 2026-08-24 in `.ebextensions/git.php`: the resolution is `property_exists($git, 'branch') ? $git->branch : ('_' . strtolower($env))`, with `$env` from `get-config environment -k ENVIRONMENT`. **Omitting `branch` is the intended design** — it derives `_<environment>` automatically, and both `git.sandbox-client.json` and `git.json` deliberately rely on that fallback. The failure mode is always the *presence* of a wrong pin, never an absence. A well-meaning "restore the missing branch key" re-creates the exact bug that removing it fixed.
|
|
109
|
+
- **✅ A MISSING `"branch"` key is CORRECT, not a misconfiguration — do not "fix" it by adding one back.** Re-confirmed 2026-08-24 in `.ebextensions/git.php`: the resolution is `property_exists($git, 'branch') ? $git->branch : ('_' . strtolower($env))`, with `$env` from `get-config environment -k ENVIRONMENT`. **Omitting `branch` is the intended design** — it derives `_<environment>` automatically, and both `git.sandbox-client.json` and `git.json` deliberately rely on that fallback. The failure mode is always the *presence* of a wrong pin, never an absence. A well-meaning "restore the missing branch key" re-creates the exact bug that removing it fixed. **The one exception is `beta`**, because it is the only tier whose `ENVIRONMENT` does not name its correct branch: the fallback resolves to `_beta` while api2 itself is built from `_sandbox-dev`, so beta needs an explicit pin (`git.beta.json`). `sandbox-client` keyless is fine — its fallback `_sandbox-client` **is** the right branch. Check the fallback *resolves to the branch you want* before trusting it.
|
|
109
110
|
- **⚠ "grep says this framework function doesn't exist" is meaningless without naming the checked-out branch.** Because `_underscore` is cloned at build from a **moving per-environment branch**, a local checkout is routinely behind the branch a tier runs. Concrete (2026-08-24): a session concluded — with **three** corroborating pieces of code evidence — that `_Model_Core_Surface::_resolveLanguageId()` had never been implemented and the KB was wrong. It was not; the local tree was on a branch predating it. The method is present on `_sandbox-client`, `_sandbox-dev`, `surface-layer` and `origin/_beta` (merged into `_sandbox-client` 2026-07-22, `6809c6b9`). **Always run `git branch --show-current` (and check sibling branches) before declaring framework code absent** — corroborating reads from the same stale tree multiply the error.
|
|
110
111
|
- **Deploy path is build-time, so a git push alone does not update running instances.** The `_underscore` clone happens in the prebuild hook at build time: commit the fix on the feature branch, merge into the deploy-channel branch (`_` + `ENVIRONMENT`, e.g. `_sandbox-client`), push, **then redeploy the EB environment** to re-run the clone.
|
|
111
112
|
- **⚠ SECURITY — every `.ebextensions/git.*.json` carries a plaintext GitHub PAT** committed in the repo (used to clone `_underscore`). Pre-existing leak; treat the committed tokens as compromised, rotate, move to EB environment properties / SSM. Do **not** copy the token value anywhere. **Escalated 2026-08-24 — still not rotated, and exposure has WIDENED.** The token is present in **all ~20 per-environment `git.*.json` files**, and an attempted in-place redaction *increased* exposure: BSD `sed` on macOS does not support `\|` alternation, so the command failed in a way that **printed the token value into the session transcript**. Two durable takeaways: (a) rotation is overdue — treat it as an open action item; (b) **do not attempt ad-hoc redaction of a secret with a one-liner** — a failed redaction commonly echoes the value it was meant to hide (GNU-vs-BSD `sed` regex differences make macOS one-liners especially prone). Rotate at the source instead.
|
|
@@ -113,6 +114,7 @@ So on `api.beta.togahub.com` and `api.dev.sandbox.togahub.com` the code fataled
|
|
|
113
114
|
- **Duplicate integration code is the trap this creates.** When the same capability is needed in both tiers it must exist twice (once per framework) — keep the credential/config location documented in both places so a rotation updates both.
|
|
114
115
|
|
|
115
116
|
## Change history
|
|
117
|
+
- 2026-10-05 — Clarified that beta is the only tier whose `_`+ENVIRONMENT fallback names the wrong branch (so it needs the pin, unlike sandbox-client), and extended the recently-added-symbol rule to declared model properties. (bala)
|
|
116
118
|
- 2026-10-02 — beta/dev-sandbox host → `api-sandbox-dev` mapping, `git.beta.json` pin to `_sandbox-dev` (uncommitted), deferred ENVIRONMENT rename and its blockers, sandbox-dev `max_connections` limit. (rgirish)
|
|
117
119
|
- 2026-09-25 — renamed product reference from "TogaIQ" to "Talos" (rebrand; TOGa IQ is now the separate `toga25-iq` app) (jcardinal)
|
|
118
120
|
|
|
@@ -6,13 +6,14 @@ project: API
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-05
|
|
10
10
|
owners: ["mhammontree", "dfranks", "bala", "jcardinal", "rgirish"]
|
|
11
11
|
files:
|
|
12
12
|
- api2/Component/Api/V2/V2.php
|
|
13
13
|
- api2/Controller/Index.php
|
|
14
14
|
- _underscore/Model/Client/Logs/Api.php
|
|
15
15
|
- _underscore/Model/Core/Logs/Api.php
|
|
16
|
+
- _underscore/Model.php
|
|
16
17
|
related:
|
|
17
18
|
- ../architecture.md
|
|
18
19
|
- ./api-key-authentication-and-scoping.md
|
|
@@ -69,6 +70,15 @@ The commit-the-logs / roll-back-the-data contract holds for the **handled** path
|
|
|
69
70
|
- **Orphan sweeps only find PARTIAL casualties.** An attempt that rolled back completely leaves no row in any table and, with no API log row, no trail at all.
|
|
70
71
|
- **It applies to plain GETs too, and the response body comes back EMPTY.** Re-confirmed 2026-08-27: a `GET /v2/table-views/meta` that throws inside `_Model_Client_TableView::meta()` (a `TableViewJoins` row pointing at a deleted `Core` record) returns **500 with no body at all** — no `error` code, no `messages` — and `Logs_CompassCanada.Api` holds no row for it. **An empty 500 body plus a missing Api log row is a usable fingerprint** for "a fatal was thrown before the response was assembled". See [tableview-joins](../../_underscore/features/tableview-joins.md).
|
|
71
72
|
|
|
73
|
+
### 🚨 The log-write line can take a WHOLE ENVIRONMENT down — including `/v2/auth/api`
|
|
74
|
+
|
|
75
|
+
`V2.php:2366` does `$log->originalUserId = $this->getOriginalUserId();` on the request-log model. If the **deployed** `_underscore/Model/Client/Logs/Api.php` does not **declare** `$originalUserId`, `_Model->__set()` (`_underscore/Model.php:498`) throws *"There is no field called 'originalUserId' in the '_Model_Client_Logs_Api' model"* **before the response is built** — so the request 500s `EO-1` no matter what it was doing. Live on beta 2026-09-21 14:28 → 2026-09-22 09:33: **639 x 500**, multiple tenants (`clientId` 5 Prudential, 39 Rate), including `POST /v2/auth/api`, so callers could not even obtain a token. Four durable rules:
|
|
76
|
+
|
|
77
|
+
- **It is a PHP model error, not a database error.** `_model_fields` is built from the model's **declared public properties**, so applying the column migration (`dbchanges2/Logs_Client/2026-09-18a - ApiOriginalUserId.sql`) does **not** fix it — that column already existed throughout the outage. The fix is deploying the `_underscore` branch that declares the property (merge to `_beta`, then redeploy). See [ENVIRONMENT decides the `_underscore` branch](./environment-variable-drives-underscore-branch.md).
|
|
78
|
+
- **Only CLIENT-logged requests fail.** `V2.php` guards the assignment with `if ($logsApiModelName === '\_Model_Client_Logs_Api')`, so a request with no resolved client logs to `_Model_Core_Logs_Api` and never reaches the broken line. **Therefore a quiet log does not prove a fix** — unauthenticated 401s keep returning 401 for the entire outage. Prove it with **one authenticated, client-scoped call**, never with "the errors stopped".
|
|
79
|
+
- **The prebuild `_underscore` clone runs near the START of a deploy.** A merge that lands *during* a deploy is missed — in this incident by **24 seconds** — and the deploy still reports success. Merge first, then start the deploy.
|
|
80
|
+
- **Generalizes beyond `originalUserId`:** any new client-log column written by `V2.php` is a whole-environment outage on a tier whose `_underscore` is behind. Land the model change on **every** tier's branch before the api2 side ships.
|
|
81
|
+
|
|
72
82
|
### ⚠ apiproxy's 5xx retry distorts BOTH logs — always read a 500 through `transactionId`
|
|
73
83
|
|
|
74
84
|
apiproxy retries a 5xx **across regions**, so **one user action produces up to six near-identical 500 rows** in core `Logs.Api`: same route, same payload, **one shared `transactionId`**, ~2 s apart, alternating `instanceId`s. Two consequences that have caused wrong conclusions:
|
|
@@ -219,6 +229,7 @@ Both a **fallback recovery path** and a **scope diagnostic**. In the TRUE-80179
|
|
|
219
229
|
- **`Logs_<Client>.Api` is pruned (once TRUE-81902 deploys)** — GET rows after 30 days, writes after 90. A missing older row is not proof the request never happened. See [Logs retention cleanup](../../worker2/features/logs-retention-cleanup.md).
|
|
220
230
|
|
|
221
231
|
## Change history
|
|
232
|
+
- 2026-10-05 — Recorded the beta `originalUserId` outage: an undeclared model property on the deployed `_Model_Client_Logs_Api` 500s every client-logged request (639, incl. `/v2/auth/api`); the column migration does not fix it and 401s stay healthy throughout. (bala)
|
|
222
233
|
- 2026-09-25 — Linked the `Logs_<Client>.Api` retention job (GET 30d / writes 90d). (rgirish)
|
|
223
234
|
- 2026-09-22 — Log-save now saves/restores `_Database` error state in `try/finally`; its own 1062s were disarming the unique-race retry on `/v2/service-requests`. Also recorded the `_production`-branch missing-method hazard and the rejected null-`transactionId` finding. (rgirish)
|
|
224
235
|
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: nychh
|
|
7
7
|
type: client-feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-
|
|
9
|
+
updated: 2026-10-05
|
|
10
10
|
owners: ["bala"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Client/Nychh.php
|
|
@@ -58,6 +58,31 @@ Two 1.0 `worker` scripts (run via PuTTY) back this up:
|
|
|
58
58
|
- **`worker/crons/toga2/netsuite/verify_fulfillment_asset_tag_sync_nychh.php`** — read-only, compares NetSuite vs TOGa per fulfillment to confirm the sync grabbed tags correctly.
|
|
59
59
|
- **`worker/crons/toga2/netsuite/backfill_all_asset_tags_from_netsuite_nychh.php`** — full-history SOAP per-serial backfill; `DRY_RUN` default, never-blank, resumable via a ledger file.
|
|
60
60
|
|
|
61
|
+
## ⚠ Open defects — the cron succeeds and fills 0 units
|
|
62
|
+
|
|
63
|
+
Cron **`Core.CronJobs` id 44**, `0 7,16 * * *`, `{"dryRun": false}`. It **reports success** every
|
|
64
|
+
run: 60,565 of 61,174 serials found, 24,978 carrying a NetSuite tag — and writes **0 units**.
|
|
65
|
+
Three faults, all **open as of 2026-10-05** (found, not fixed):
|
|
66
|
+
|
|
67
|
+
1. **`fetchTagsForSerials()` never paginates.** It sends 500 serials per chunk with `?limit=1000`
|
|
68
|
+
and reads `$page->items` **once**. Serials are **not unique** in NetSuite — 1,151,658
|
|
69
|
+
`inventorynumber` rows against 1,126,401 distinct values, and junk serials make it worse
|
|
70
|
+
(`'NA4'` alone returns 9 rows) — so a chunk can overflow one page and the tail is dropped
|
|
71
|
+
silently. That matches the 609 serials the run cannot find (inferred from the counts, not
|
|
72
|
+
reproduced).
|
|
73
|
+
2. **`loadUnitsBySerial()` keys units by serial** (`$unitsBySerial[$serial] = $row`). Prod has
|
|
74
|
+
**61,643 units with a serial but only 61,166 distinct**, so **477 units are overwritten** in the
|
|
75
|
+
map and can never be filled, whatever NetSuite holds.
|
|
76
|
+
3. **`fetchTagsForSerials()` keys the NetSuite side by serial too** — last row wins, so a row
|
|
77
|
+
with a NULL tag can **wipe a good value** read moments earlier.
|
|
78
|
+
|
|
79
|
+
Live case that exposed it: units **57144** and **59631** (serials `62C26G4`, `B2C26G4`) are blank
|
|
80
|
+
in `Client_Nychh.Units` since 12 Sep while NetSuite holds `ITD0081907` / `ITD0081919`.
|
|
81
|
+
|
|
82
|
+
**Do not over-correct — most blank tags are legitimately blank.** A sample of 25 blank-tag
|
|
83
|
+
serials had a NULL asset tag in NetSuite as well. Fixing the three faults recovers hundreds of
|
|
84
|
+
units, not tens of thousands.
|
|
85
|
+
|
|
61
86
|
## Gotchas
|
|
62
87
|
|
|
63
88
|
- **Do not filter on the custom columns.** SuiteQL will `SELECT` `custitemnumber_asset_tag` / `custitemnumber_mac_address` but a `WHERE custitemnumber_asset_tag IS NOT NULL` returns **0 rows with no error** — see the SuiteQL custom-field gotcha in [netsuite-rest-client](../../_underscore/features/netsuite-rest-client.md). Filter on the native `inventorynumber` column, select the custom columns, filter blanks in PHP.
|
|
@@ -70,3 +95,6 @@ Two 1.0 `worker` scripts (run via PuTTY) back this up:
|
|
|
70
95
|
- [NetSuite ↔ TOGa Supply per-client sync](../../../1.0/apps/worker/features/netsuite-togasupply-per-client-sync.md) — the fulfillment sync this backstops.
|
|
71
96
|
- [creating-worker-actions](creating-worker-actions.md) — dispatch + `DB_CLIENT` auto-registration.
|
|
72
97
|
- [NYCHH profile](../../../clients/nychh/profile.md).
|
|
98
|
+
|
|
99
|
+
## Change history
|
|
100
|
+
- 2026-10-05 — Recorded three open faults behind the "succeeds but fills 0 units" runs: no pagination in `fetchTagsForSerials()` (serials are not unique in NetSuite), and serial-keyed maps on both the TOGa side (477 of 61,643 units lost) and the NetSuite side (a NULL-tag row overwrites a good one). Also recorded the counter-evidence that most blank tags are genuinely blank in NetSuite. Not fixed. (bala)
|
|
@@ -5,10 +5,12 @@
|
|
|
5
5
|
| [Compass Approval-Decision Flow (Notifications & Manager Reassignment)](features/approval-decision-flow.md) | 2.0 | Compass sales-order approval-decision / notification flow — approval+CC email lists, manager reassignment, VIP auto-approve, EN/FR localization; open when touch |
|
|
6
6
|
| [Compass ASN → ItemFulfillment Auto-Creation](features/asn-to-item-fulfillment.md) | 2.0 | Compass USA ASN posting auto-creates the ItemFulfillment on the upstream SalesOrder and propagates tracking at every granularity; open when working ASN→IF creat |
|
|
7
7
|
| [Compass Contact Email Resolution & Crossed `primaryContactEmailAddressId` Pointers](features/contact-email-resolution.md) | 2.0 | Any Compass code reading a contact's email must assert the email row belongs to that contact (~14% of pointers are crossed); open before reading Compass contact |
|
|
8
|
+
| [Corporate vs Field laptop personas — the sector-52 split (designed, NOT built)](features/corporate-vs-field-laptop-personas.md) | 2.0 | Agreed design for splitting the Compass catalogue into a Corporate laptop and a field "laptop kit": attach Corporate at sector `J00000` via `Locations_Personas` |
|
|
8
9
|
| [Cost Centers — Unit Locations, numeric-only policy](features/cost-centers.md) | 2.0 | A Compass "cost center" is a Unit-level Location keyed by `c_erpSystemEntityId` (NOT a `CostCenters` row), numeric-only since 2026-07-06; open for cost-center d |
|
|
9
10
|
| [Compass isFulfillable — Data Quality & the Type-Derived Rule](features/isfulfillable-data-quality-and-type-rule.md) | 2.0 | Why Compass storefront Qty-Fulfilled cells stay dead — `Items.isFulfillable` NULLs are structurally unreachable, some set values are wrong, and the type-derived |
|
|
10
11
|
| [Compass Item Catalogs (Compass vs Agilant) & the Duplicate Items the NetSuite Sync Creates](features/item-catalogs-and-duplicate-items.md) | 1.0 | `Client_Compass.Items` is split across catalogs (Compass vs Agilant) and the NetSuite sync creates duplicate rows for one product; open first whenever a Compass |
|
|
11
12
|
| [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two config-only Compass TableViews (ids 12 & 13) showing fulfilled items in toga2-supply, re-rooted off Units and tracking sourced at item level; open when a fu |
|
|
13
|
+
| [Compass Location Hierarchy & Sectors (corporate detection, dedupe before counting)](features/location-hierarchy-and-sectors.md) | 2.0 | The 9-level `Client_Compass.Locations` tree, how "corporate" is identified (sector ERP-id prefix, no flag), and the junk duplicate rows that make every raw coun |
|
|
12
14
|
| [Compass MITS PO → SO Item Linking](features/mits-po-to-so-item-linking.md) | 2.0 | How Compass inbound MITS purchase orders link each PO item to the SalesOrderItem it fulfills (two paths, SA vs MR); open when a PO fails to link, cross-part lin |
|
|
13
15
|
| [Compass MITS PO Transmission to Vendors](features/mits-po-transmission-to-vendors.md) | 2.0 | The 1.0 worker cron that transmits Compass PurchaseOrders to vendors via cXML or emailed PDF; open for PO transmission channel/routing, the PDF rendering, or "d |
|
|
14
16
|
| [Compass MITS PO rejection — zero line items (400 + Issue + PM alert email)](features/mits-po-zero-line-item-rejection.md) | 2.0 | Guard that rejects MITS Compass purchase orders carrying zero line items (HTTP 400 + Logs.Issue + PM alert email); open when working on the zero-item guard, its |
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Corporate vs Field laptop personas — the sector-52 split (designed, NOT built)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
project: _Underscore
|
|
5
|
+
client: compass-usa
|
|
6
|
+
type: client-feature
|
|
7
|
+
status: deferred
|
|
8
|
+
updated: 2026-10-05
|
|
9
|
+
owners: [bala]
|
|
10
|
+
files:
|
|
11
|
+
- worker2/Worker/Client/Compass/PeopleFile.php
|
|
12
|
+
related:
|
|
13
|
+
- ./persona-model-and-levy-gating.md
|
|
14
|
+
- ./location-hierarchy-and-sectors.md
|
|
15
|
+
- ../workflows/persona-refactor-migration.md
|
|
16
|
+
- ../../../2.0/apps/_underscore/features/effective-persona-resolution.md
|
|
17
|
+
- ../profile.md
|
|
18
|
+
---
|
|
19
|
+
Agreed design for splitting the Compass catalogue into a Corporate laptop and a field "laptop kit": attach Corporate at sector `J00000` via `Locations_Personas`, assign Field in `PeopleFile`. Designed 2026-10-05, no code written. Open before building the field kit.
|
|
20
|
+
|
|
21
|
+
## Summary
|
|
22
|
+
|
|
23
|
+
Compass confirmed the new **field laptop kit REPLACES the standard laptop** for non-corporate users;
|
|
24
|
+
only admins see both. So the split is by **corporate vs not**, and corporate is a hierarchy question,
|
|
25
|
+
not a flag — see [Location Hierarchy & Sectors](./location-hierarchy-and-sectors.md).
|
|
26
|
+
|
|
27
|
+
Nothing is built. This doc records the agreed shape so the next session does not re-derive it.
|
|
28
|
+
|
|
29
|
+
## How it works (the agreed design)
|
|
30
|
+
|
|
31
|
+
1. **Corporate persona.** Move the standard laptop bundle(s) out of the shared persona into a new
|
|
32
|
+
**Corporate** persona, and attach it with a **single `Locations_Personas` row on sector `J00000`
|
|
33
|
+
(`Locations.id = 52`)**. Effective-persona resolution unions `Users_Personas`, the user's own
|
|
34
|
+
`Locations_Personas`, and their **sector's** (the 5-level walk), so every cost center under
|
|
35
|
+
`J00000` inherits it — including ones Compass adds on a later daily import.
|
|
36
|
+
2. **Field persona.** It **cannot** use `Locations_Personas`: "everyone except sector 52" is not a
|
|
37
|
+
single location. Assign it in `worker2/Worker/Client/Compass/PeopleFile.php` as the default in
|
|
38
|
+
`Users_Personas`, skipped for anyone under `J00000` — mirroring the existing Levy branch.
|
|
39
|
+
3. **Rejected alternative:** one `Locations_Personas` row per *other* sector (79 of them). It
|
|
40
|
+
silently misses any new sector Compass adds, which is the exact failure the feed is designed to
|
|
41
|
+
avoid.
|
|
42
|
+
4. **No `isActive` filter is needed.** No inactive cost center under `J00000` has a single active
|
|
43
|
+
user, so attaching at the sector only ever reaches real people.
|
|
44
|
+
|
|
45
|
+
## Prod state observed 2026-10-05 (⚠ conflicts with the refactor doc — re-verify before building)
|
|
46
|
+
|
|
47
|
+
A read of prod `Client_Compass` on 2026-10-05 found persona **1 still named "All"**, still holding
|
|
48
|
+
**all 23 bundles**, every US user on it, and **no `Locations_Personas` row for sector 52** (the table
|
|
49
|
+
is used at sector level for 16 other sectors, e.g. Levy sector 55 → persona 24). The standard laptop
|
|
50
|
+
was bundle **187** "Compass Standard HP ZBOOK 8 Laptop (Preferred)"; persona 1 also held **188** HP
|
|
51
|
+
Ebook Flip (inactive), **192** HP Pro Mini 400 G9, **195** Developer ZBOOK.
|
|
52
|
+
|
|
53
|
+
That contradicts [Persona Refactor](../workflows/persona-refactor-migration.md), which records the
|
|
54
|
+
three-way split (Base / Non-Levy id 40 / Levy 24) as applied and verified on prod 2026-09-17.
|
|
55
|
+
**One of the two is reading the wrong environment — resolve it before writing any migration.** Any
|
|
56
|
+
plan written against "persona 40 / Non-Levy" must be re-checked against prod first.
|
|
57
|
+
|
|
58
|
+
## Gotchas
|
|
59
|
+
|
|
60
|
+
- **Never hardcode a persona id.** Resolve by name, exactly as `PeopleFile` does for `Non-Levy` —
|
|
61
|
+
ids differ per environment. See [Persona Model & Levy-Sector Gating](./persona-model-and-levy-gating.md).
|
|
62
|
+
- **A persona grant is three tables** (`Personas_Bundles`, `Personas_Items`, `Personas_Assortments`),
|
|
63
|
+
not just bundles — see [granting a bundle](../workflows/granting-persona-bundle-access.md).
|
|
64
|
+
- **The DB does not restrict what a persona sees** — the storefront front end is the only gate. A new
|
|
65
|
+
persona alone changes nothing until `toga2-commerce` filters on it.
|
|
66
|
+
|
|
67
|
+
## Related
|
|
68
|
+
- [Persona Model & Levy-Sector Gating](./persona-model-and-levy-gating.md)
|
|
69
|
+
- [Location Hierarchy & Sectors](./location-hierarchy-and-sectors.md)
|
|
70
|
+
- [Persona Refactor migration](../workflows/persona-refactor-migration.md)
|
|
71
|
+
- [Effective persona resolution](../../../2.0/apps/_underscore/features/effective-persona-resolution.md)
|
|
72
|
+
|
|
73
|
+
## Change history
|
|
74
|
+
- 2026-10-05 — Created. Recorded the agreed corporate/field split (Corporate attached at sector
|
|
75
|
+
`J00000` via one `Locations_Personas` row; Field assigned in `PeopleFile` and skipped under
|
|
76
|
+
`J00000`; the 79-row per-sector alternative rejected) and the 2026-10-05 prod read that conflicts
|
|
77
|
+
with the persona-refactor doc. Nothing built. (bala)
|
|
@@ -0,0 +1,128 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Compass Location Hierarchy & Sectors (corporate detection, dedupe before counting)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
project: _Underscore
|
|
5
|
+
client: compass-usa
|
|
6
|
+
type: client-feature
|
|
7
|
+
status: active
|
|
8
|
+
updated: 2026-10-05
|
|
9
|
+
owners: [bala]
|
|
10
|
+
files:
|
|
11
|
+
- worker/crons/toga2/compass/DOA_Scripts/import_locations.php
|
|
12
|
+
- worker1.5/crons/toga2/compass/import_locations.php
|
|
13
|
+
- worker2/Worker/Client/Compass/PeopleFile.php
|
|
14
|
+
related:
|
|
15
|
+
- ./cost-centers.md
|
|
16
|
+
- ./persona-model-and-levy-gating.md
|
|
17
|
+
- ./corporate-vs-field-laptop-personas.md
|
|
18
|
+
- ../../../2.0/apps/_underscore/features/effective-persona-resolution.md
|
|
19
|
+
- ../profile.md
|
|
20
|
+
---
|
|
21
|
+
The 9-level `Client_Compass.Locations` tree, how "corporate" is identified (sector ERP-id prefix, no flag), and the junk duplicate rows that make every raw count wrong. Open before counting Compass locations/cost centers or targeting a group of users by location.
|
|
22
|
+
|
|
23
|
+
## Summary
|
|
24
|
+
|
|
25
|
+
Compass has no "corporate", "overhead" or "type" flag anywhere. Group membership is carried **only
|
|
26
|
+
by the location hierarchy and the ERP id on each node** (`Locations.c_erpEntityId`). Corporate =
|
|
27
|
+
everything under **sector `J00000` "Corporate Indirect OH Sector"** (`Locations.id = 52` in prod
|
|
28
|
+
`Client_Compass`, `locationTypeId = 6`). Levy's corporate office is **not** a sector — it is the
|
|
29
|
+
district `K9902` "Levy Home Office".
|
|
30
|
+
|
|
31
|
+
`Locations` also carries thousands of junk duplicate unit rows, so **no raw count of Compass
|
|
32
|
+
locations is correct**. Strip them first (recipe below).
|
|
33
|
+
|
|
34
|
+
## How it works
|
|
35
|
+
|
|
36
|
+
### The tree
|
|
37
|
+
|
|
38
|
+
| `locationTypeId` | Level |
|
|
39
|
+
|---|---|
|
|
40
|
+
| 1 | Unit (users sit here, via `Users.locationId`) |
|
|
41
|
+
| 2 | Complex |
|
|
42
|
+
| 3 | District |
|
|
43
|
+
| 4 | Region |
|
|
44
|
+
| 5 | Division |
|
|
45
|
+
| 6 | Sector |
|
|
46
|
+
| 7 | Sector Group |
|
|
47
|
+
| 8 | Business Group |
|
|
48
|
+
| 9 | Business Name (root) |
|
|
49
|
+
|
|
50
|
+
A user's **sector is exactly 5 parent hops above `Users.locationId`** — the same walk
|
|
51
|
+
`PeopleFile.php::isLocationUnderLevySector()` already does for Levy `K00000`. Reuse that walk; do
|
|
52
|
+
not re-derive it. Ids are set in `import_locations.php:37-45`.
|
|
53
|
+
|
|
54
|
+
### The daily feed
|
|
55
|
+
|
|
56
|
+
`worker/crons/toga2/compass/DOA_Scripts/import_locations.php` builds the whole tree from a 21-column
|
|
57
|
+
(0-20) SFTP file. **There is no "corporate" column** — the only corporate signal is the sector ERP id.
|
|
58
|
+
|
|
59
|
+
| Column | Meaning |
|
|
60
|
+
|---|---|
|
|
61
|
+
| 0 | unit id |
|
|
62
|
+
| 1-3 | business type / business group / sector group names |
|
|
63
|
+
| 4 / 10 | sector name / sector ERP id |
|
|
64
|
+
| 5 / 11 | division name / id |
|
|
65
|
+
| 6 / 12 | region name / id |
|
|
66
|
+
| 7 / 13 | district name / id |
|
|
67
|
+
| 8 / 14 | complex name / id |
|
|
68
|
+
| 9 | unit name |
|
|
69
|
+
| 16 | cost center entity id → `Locations.c_erpSystemEntityId` |
|
|
70
|
+
| 17-20 | street / city / state / zip |
|
|
71
|
+
|
|
72
|
+
Because new corporate cost centers arrive on this feed on their own, **anything scoped to "corporate"
|
|
73
|
+
must key off the hierarchy, never a client-maintained spreadsheet.**
|
|
74
|
+
|
|
75
|
+
### Reading the ERP id
|
|
76
|
+
|
|
77
|
+
The sector's **letter prefix carries down to every node under it**, so the prefix identifies the
|
|
78
|
+
sector without a walk: `J00000` Corporate Indirect OH → `JA0000` Accounting/Tax, `J90000` Indirect
|
|
79
|
+
G&A, `JC0000` Compass Digital Labs, `JS0000` Sales.
|
|
80
|
+
|
|
81
|
+
### Levy's corporate office is a DISTRICT
|
|
82
|
+
|
|
83
|
+
`K9902` "Levy Home Office" (`locationTypeId = 3`), under region `K99000` → division `K90000` →
|
|
84
|
+
sector `K00000`. It holds **35 active cost centers and all 239 active users** under Levy Overhead;
|
|
85
|
+
every other district under `K90000` has zero users. Its cost centers are named with an `LFS ` prefix
|
|
86
|
+
(LFS EXECUTIVE, LFS LEGAL, LFS PURCHASING…) and 14 of them share one complex, `C-33919` (LFS ADMIN).
|
|
87
|
+
The client's own corporate spreadsheet listed none of them.
|
|
88
|
+
|
|
89
|
+
### Counting locations correctly (prod `Client_Compass`, 2026-10-05)
|
|
90
|
+
|
|
91
|
+
Under sector 52 the raw row count was **3,472**; the real number is **1,113** cost centers (640
|
|
92
|
+
active, 473 inactive, 286 with at least one active user, 4,959 active users). The difference is junk:
|
|
93
|
+
|
|
94
|
+
- cost center `4085` "$$$Intst Merger Charges" repeated ~**872** times,
|
|
95
|
+
- cost center `5998` "$$$4 C's Misc Oper" repeated ~**868** times,
|
|
96
|
+
- **617** unit rows with a NULL/blank `c_erpSystemEntityId` under those same two names.
|
|
97
|
+
|
|
98
|
+
All of them are flagged Active and all have **zero users**. Drop them before any count, then group by
|
|
99
|
+
`c_erpSystemEntityId`.
|
|
100
|
+
|
|
101
|
+
## Gotchas
|
|
102
|
+
|
|
103
|
+
- **⚠ Never report a raw count of Compass locations.** A first report to the client said "2,342
|
|
104
|
+
active units" and was wrong; the client caught it. Dedupe first (above). The same duplication
|
|
105
|
+
exists tenant-wide, not just under sector 52 — see [Cost Centers](./cost-centers.md).
|
|
106
|
+
- **⚠ The `_9%` overhead division is NOT a corporate marker.** Every sector has an overhead division
|
|
107
|
+
whose ERP id matches `_9%` (`F90000` Eurest, `V90000` Canteen, `K90000` Levy, `J90000` IOH), but
|
|
108
|
+
Eurest's `F90000` alone holds **6,273 active units** — it is a catch-all, not a home office. Match
|
|
109
|
+
the sector (`J00000`), not the `9` pattern.
|
|
110
|
+
- **⚠ `Locations_Addresses` is one-to-many.** A plain `LEFT JOIN Addresses` fans out and duplicates
|
|
111
|
+
rows. Join through a `MIN(addressId)` sub-select.
|
|
112
|
+
- **An inactive ancestor does not make a user inactive.** Any hierarchy walk must run over an
|
|
113
|
+
**unfiltered** location set — see
|
|
114
|
+
[Persona Model & Levy-Sector Gating](./persona-model-and-levy-gating.md).
|
|
115
|
+
|
|
116
|
+
## Related
|
|
117
|
+
- [Cost Centers — Unit Locations, numeric-only policy](./cost-centers.md)
|
|
118
|
+
- [Persona Model & Levy-Sector Gating](./persona-model-and-levy-gating.md)
|
|
119
|
+
- [Corporate vs field laptop personas (deferred)](./corporate-vs-field-laptop-personas.md)
|
|
120
|
+
- [Effective persona resolution](../../../2.0/apps/_underscore/features/effective-persona-resolution.md)
|
|
121
|
+
|
|
122
|
+
## Change history
|
|
123
|
+
- 2026-10-05 — Created from the field-laptop research. Recorded that corporate is identified only by
|
|
124
|
+
sector `J00000` (`Locations.id = 52`) and the ERP-id letter prefix (no flag, no column in the
|
|
125
|
+
21-column feed), that Levy's corporate office is the **district** `K9902` not a sector, that the
|
|
126
|
+
`_9%` overhead-division pattern is a false marker (Eurest `F90000` = 6,273 units), and that raw
|
|
127
|
+
`Locations` counts are wrong — 3,472 raw rows under sector 52 are really 1,113 cost centers once
|
|
128
|
+
the ~872 + ~868 junk duplicates and 617 blank-code rows are stripped. (bala)
|
|
@@ -6,7 +6,7 @@ project: Worker
|
|
|
6
6
|
client: nychh
|
|
7
7
|
type: client-feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-10-
|
|
9
|
+
updated: 2026-10-05
|
|
10
10
|
owners: ["bala", "rgirish", "jcardinal"]
|
|
11
11
|
files:
|
|
12
12
|
- worker2/Worker/Client/Nychh/TransferOrderEmails.php
|
|
@@ -18,6 +18,7 @@ files:
|
|
|
18
18
|
- dbchanges2/Core/2026-09-16a - NychhTransferOrderEmailsCron.sql
|
|
19
19
|
related:
|
|
20
20
|
- ./transfer-order-netsuite-push.md
|
|
21
|
+
- ../../../2.0/apps/_underscore/features/units-for-items-for-purchase-orders.md
|
|
21
22
|
- ./netsuite-transfer-order-import.md
|
|
22
23
|
- ../../../2.0/apps/_underscore/features/email-template-sending.md
|
|
23
24
|
- ../../../2.0/apps/_underscore/features/email-send-pipeline.md
|
|
@@ -157,8 +158,21 @@ PUT follows.** Fulfillment **2704** in production: header row at 20:33:09, lines
|
|
|
157
158
|
Added 2026-09-30 (TRUE-82111). `SendFulfillment` runs `readLinesMissingAssetTags()` whenever
|
|
158
159
|
**packed** is one of the stages still owed. A fulfillment line counts as **short** when either:
|
|
159
160
|
|
|
160
|
-
- it has **fewer linked units than its quantity**
|
|
161
|
-
|
|
161
|
+
- the item is **`SERIALIZED`** and it has **fewer linked units than its quantity**
|
|
162
|
+
(`unitsLinked < ItemFulfillmentItems.quantity`), or
|
|
163
|
+
- a linked unit has a **blank `Units.assetTag`** — that half holds whatever the item type is.
|
|
164
|
+
|
|
165
|
+
**⚠ The unit-count half is SERIALIZED-only.** `Client_Nychh.Items.inventoryType` is
|
|
166
|
+
`enum('SERIALIZED','NON_SERIALIZED','HYBRID')`; only SERIALIZED items are expected to carry units,
|
|
167
|
+
so the query compares against `_Model_Client_Item::INVENTORY_TYPE__SERIALIZED` — never a raw
|
|
168
|
+
string. Type-blind, the rule reported every accessory as missing tags: the first live exception
|
|
169
|
+
(fulfillment **330197** on transfer order **291771**, 2026-10-05 09:42) fired on `AC785544`, a
|
|
170
|
+
wired desktop set. Measured on prod NYCHH fulfillment lines, NON_SERIALIZED lines have **never**
|
|
171
|
+
had units (0 of 2,372), HYBRID only 510 of 5,743, SERIALIZED 1,242 of 1,389 — the type-blind rule
|
|
172
|
+
would have warned on roughly **7,600 normal lines**. After the fix shipped the same fulfillment
|
|
173
|
+
passed at 10:10 and Order Packed went out with the accessory line correctly ignored. The
|
|
174
|
+
underlying rule lives in
|
|
175
|
+
[inventoryType determines unit existence](../../../2.0/apps/_underscore/features/units-for-items-for-purchase-orders.md).
|
|
162
176
|
|
|
163
177
|
When any line is short, `SendFulfillment` **returns before writing
|
|
164
178
|
`c_transferOrderEmailStageSent`** — so packed stays pending and the backstop cron re-sends it by
|
|
@@ -320,8 +334,10 @@ Cost of the split, stated so nobody is surprised later:
|
|
|
320
334
|
|
|
321
335
|
## Testing
|
|
322
336
|
|
|
323
|
-
**
|
|
324
|
-
|
|
337
|
+
**111 tests** as of 2026-10-05 — the last five cover the SERIALIZED-only unit-count rule, including
|
|
338
|
+
a mixed fulfillment whose exception email names only the serialized line. Each fix is proven by
|
|
339
|
+
**reverting it** and confirming the matching tests fail (reverting the inventory-type rule fails
|
|
340
|
+
three). The five scenarios were then run end to end on **dev sandbox** against fulfillment
|
|
325
341
|
**898** / transfer order **286536**.
|
|
326
342
|
|
|
327
343
|
A local end-to-end harness (scratchpad, **not committed**) runs the **real** worker class against a
|
|
@@ -332,6 +348,14 @@ loaded out of the dbchanges2 file so an unresolved `{placeholder}` fails a test.
|
|
|
332
348
|
unescaped values and both detail-box defects were all found there, not in production.
|
|
333
349
|
|
|
334
350
|
## Change history
|
|
351
|
+
- 2026-10-05 — **The asset-tag hold now only counts missing units on `SERIALIZED` items**
|
|
352
|
+
(TRUE-82111). `readLinesMissingAssetTags()` treated "fewer linked units than quantity" as missing
|
|
353
|
+
tags for every item, so accessories were reported short: first prod exception was `AC785544`, a
|
|
354
|
+
wired desktop set, on fulfillment 330197 / transfer order 291771. Now compared against
|
|
355
|
+
`_Model_Client_Item::INVENTORY_TYPE__SERIALIZED`; the blank-`assetTag` half is unchanged. Prod
|
|
356
|
+
shows NON_SERIALIZED lines never carry units (0 of 2,372), so the old rule would have warned on
|
|
357
|
+
~7,600 normal lines. Shipped and verified live: same fulfillment passed at 10:10 and Order Packed
|
|
358
|
+
sent. Harness now 111 tests. (bala)
|
|
335
359
|
- 2026-10-01 — **Packed/shipped are now HELD when a fulfillment line is short of asset tags**
|
|
336
360
|
(TRUE-82111). A line is short when it has fewer linked units than its quantity **or** a linked unit
|
|
337
361
|
has a blank `assetTag`; `SendFulfillment` then returns without writing
|
|
@@ -48,6 +48,8 @@ WHERE PurchaseOrders.vendorId = 1 # Dell = Vendors.id 1
|
|
|
48
48
|
This is NOT the SalesOrder customer — the SO customer is always **"Agilant - Tech Hub" (id 1, uuid `36da53f8-38d2-404f-becf-f58f33c215d3`)**.
|
|
49
49
|
- A REQ is only ever transmitted, validated, or marked once selected here. Unit/bundle/SO/PO creation happens elsewhere and does **not** send anything to Dell.
|
|
50
50
|
|
|
51
|
+
**⚠ The ship-to address Dell receives comes from `ServiceRequests.deliverToAddressId`, NOT the PO's `shipToAddressId`.** The cron joins `LEFT OUTER JOIN Addresses ON Addresses.id = ServiceRequests.deliverToAddressId` and sends `Addresses.line1` as `shipAddress.address1`. A separate `Addresses` row *is* created for the PurchaseOrder, and it is **not** the one transmitted. Correcting a bad Dell address means editing the **SR's deliverTo row**; fixing only the PO's address changes nothing and the same rejection repeats.
|
|
52
|
+
|
|
51
53
|
### Send mechanism
|
|
52
54
|
- `App_Api_Delllch::send` (`library/app/api/delllch.php`) POSTs to route `api/v2/request/LCHRequestV2`. Prod host `https://cms-iop.us.dell.com/iopv2/`; non-prod `.../iopv2np/`.
|
|
53
55
|
- Auth: OAuth2 client-credentials Bearer token obtained via `auth/token` (credentials in config, not in this doc).
|
|
@@ -56,6 +58,14 @@ WHERE PurchaseOrders.vendorId = 1 # Dell = Vendors.id 1
|
|
|
56
58
|
- **Event model (the key change):** the new schema adds a required **`requestEvent`** field — `'Create'` for new orders, `'Update'` for updates, `'Cancel'` for cancellations. Replaces the old overloaded `svcOrderType` that previously signalled both the SR type AND the event (`'CANCEL'`/`'UPDATE'`). In V2, `requestType` carries the SR type only (Refresh / Breakfix / Reclaim / Bulk).
|
|
57
59
|
- **Optional pre-submission validation:** `App_Api_Delllch::validateLCHRequestV2(array $payload, bool $isProduction = true): mixed` calls `api/v2/validate/LCHRequestV2` — returns 200 on a valid payload, `422` with a `problems[]` / `issues[]` array on schema violations.
|
|
58
60
|
|
|
61
|
+
### What a pre-send 422 does (and does not) do
|
|
62
|
+
Order of operations in each regional cron: **validate → send → stamp**.
|
|
63
|
+
1. `POST api/v2/validate/LCHRequestV2`. A rejection is `422` with e.g. `{"msg":{"problems":1,"issues":["Unsupported [shipAddress.address1] is a PO Box"]}}` — **Dell refuses PO Box ship-to addresses.**
|
|
64
|
+
2. On failure the cron emails `devteam@togatech.com` + `vburks@togatech.com`, throttled to **once an hour per PO** via `PurchaseOrders.c_dtLastValidationAlert`, then `continue`s. `dtSubmitted` is left NULL "so it retries once the data is corrected".
|
|
65
|
+
3. On success it POSTs `api/v2/request/LCHRequestV2` (Dell answers **202**) and only then runs `UPDATE PurchaseOrders SET dtSubmitted = NOW()` — which is what drops the PO out of the queue.
|
|
66
|
+
|
|
67
|
+
**Consequence: fixing the data is enough. No manual push, no re-queue** — the next 5-minute run picks the PO up and stamps it.
|
|
68
|
+
|
|
59
69
|
### Payload schema migration (old → new)
|
|
60
70
|
| Old (`LCHRequest`) | New (`LCHRequestV2`) |
|
|
61
71
|
| --- | --- |
|
|
@@ -108,11 +118,13 @@ Verification plan:
|
|
|
108
118
|
- **The country-split cutover (~2026-07-17) is when this started biting.** Before the split, `transmissions_to_dell_v1.php` had **no customer gate**, so every REQ transmitted regardless. The regional split files added `INNER JOIN Customers` + `WHERE Customers.uuid = '<region>'`, which drops NULL-`customerId` REQs. ~38 REQs stranded this way since the cutover.
|
|
109
119
|
- **Remediation:** backfill `ServiceRequests.customerId = 3` (USA) for the stuck US-ship-to REQs; **durable fix** = have the generation cron derive country/`customerId` from `deliverToAddress` so `customerId` is never NULL.
|
|
110
120
|
- **`dtSubmitted` is stamped without inspecting the response (latent bug, not yet fixed).** Every regional cron sets `PurchaseOrders.dtSubmitted = NOW()` immediately after the send without checking Dell's HTTP status, and `App_ApiTransaction::execute()` (`library/app/apitransaction.php`) only throws on curl transport failure / empty / non-JSON body — **not** on HTTP >= 400. So a Dell 4xx/5xx rejection is recorded as "sent", the PO drops out of the `dtSubmitted IS NULL` queue, and is never retried. `getAccessToken()` also does not check the token-call result (silent 401 path). Second way an order can silently fail to reach Dell; fix later by inspecting the response before stamping.
|
|
121
|
+
- **⚠ A 422 retries forever — a permanent data rejection is treated as transient.** Because validation failure `continue`s without stamping `dtSubmitted`, the same PO is re-selected **every 5 minutes, indefinitely**. One PO Box address cost **748 Dell asset lookups between 2026-09-18 and 2026-09-22** (~280 calls/day), each ending in the same 422. A 422 is a schema/data rejection that will never pass on retry: it should **park** the request (stamp a failure state) rather than re-queue. Until that is built, the hourly alert email is the only stop signal — and it stops nothing.
|
|
111
122
|
- `requestType` is the SR type only (Refresh/Breakfix/Reclaim/Bulk) — do **not** put the event there. Event signalling moved to the separate required `requestEvent` field (Create/Update/Cancel). Conflating them reproduces the old overloaded `svcOrderType` bug under the new schema.
|
|
112
123
|
- Dell deprecated `LCHRequest`; any script still POSTing to the old route will fail. All four transmission scripts must point at `LCHRequestV2`.
|
|
113
124
|
- The removed fields (e.g. `deviceType`, `lob`, `imageType`) will be rejected by V2 validation — do not reintroduce them.
|
|
114
125
|
|
|
115
126
|
## Change history
|
|
127
|
+
- 2026-10-05 — Recorded that the Dell ship-to comes from `ServiceRequests.deliverToAddressId` (not the PO address), Dell's PO Box 422, and the forever-retry loop a 422 creates (748 lookups for one bad address). Investigation only, no code change. (bala)
|
|
116
128
|
- 2026-09-29 — Noted the provision profile is read from the linked bundle; wrong bundle link sent India REQs the USA profile. (rgirish)
|
|
117
129
|
|
|
118
130
|
## Related
|
package/package.json
CHANGED