toga-ai 1.0.739 → 1.0.741

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -32,6 +32,7 @@
32
32
  | [_Model::save() vs raw _Query — no atomic conditional update](features/model-save-vs-query-atomic-update.md) | `_Model::save()` is a plain load-then-write ORM primitive and **cannot express an atomic conditional update** (an optimistic-concurrency / row-claim guard such | _underscore/Model.php, _underscore/Query.php, _underscore/Model/Rate/Subscription.php |
33
33
  | [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php, _underscore/Trait/Netsuite/SalesOrder.php, worker2/Worker/Netsuite/SalesOrder.php, worker2/Component/Forecast/SaleImport/SaleImport.php, worker2/Worker/Netsuite/Opportunity.php |
34
34
  | [NetSuite Sales Order sync — ship-to address, phone, and PO reference sourcing](features/netsuite-salesorder-address-phone-sync.md) | `_Trait_Netsuite_SalesOrder` is the **shared** sales-order importer composed into **22 client models** (every client on the dbchanges2 `netsuite` module). | _underscore/Trait/Netsuite/SalesOrder.php, _underscore/Model.php, dbchanges2/_modules/netsuite/2026-08-10a - AddressPhoneNumberApiRoleAcl.sql |
35
+ | [Outbound third-party API rate budget (_Component_Api_RateBudget) — designed + staged, NOT merged](features/outbound-api-rate-budget.md) | A **shared outbound rate budget for every third-party API** we call from 2.0 (ClickUp, NetSuite, FedEx, UPS, …): one small component plus one `Core` table, keye | _underscore/Component/Api/RateBudget/RateBudget.php, _underscore/Component/Api/Clickup/Clickup.php, _underscore/Route.php, _underscore/Query.php, dbchanges2/Core/2026-09-01a - Add ApiRateBudget table.sql, worker2/Worker/Clickup/Api.php |
35
36
  | [Legacy page meta (Page::meta) & context-scoped ClientRecordFieldSettings](features/page-meta-context-field-settings.md) | `_Model_Core_Page::meta()` is the **legacy** page-meta resolver behind `GET /pages/meta?slug=<slug>` — still the live path for `toga2-supply` and other pre-Surf | _underscore/Model/Core/Page.php, _underscore/Model/Client/TableView.php, toga2-supply/src/components/ui/Tables/PrimaryTable/PrimaryTable.tsx, toga2-supply/src/components/ui/Tables/PrimaryTable/helperFunctions/formatTableData.tsx |
36
37
  | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/Model.php, _underscore/Query.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php, api2/Controller/Index.php |
37
38
  | [Persona Name Translation (PersonaTranslations sidecar)](features/persona-name-translation.md) | Serves Persona **names** in multiple languages by adding a per-language **sidecar** table `PersonaTranslations`, reusing the platform's existing metadata-driven | _underscore/Model/Client/PersonaTranslation.php, dbchanges2/Client/2026-07-22b - PersonaTranslations.sql, dbchanges2/Core/2026-07-22a - PersonaTranslationsRecord.sql, dbchanges2/Client/2026-07-22c - PersonaTranslationsAcl.sql, dbchanges2/Client_CompassCanada/2026-07-22 - PersonaTranslationsFrench.sql, toga2-commerce/src/pages/Account/view/MySettingsView.tsx |
@@ -0,0 +1,190 @@
1
+ ---
2
+ title: Outbound third-party API rate budget (_Component_Api_RateBudget) — designed + staged, NOT merged
3
+ framework: "2.0"
4
+ repo: _underscore
5
+ project: _Underscore
6
+ client: shared
7
+ type: feature
8
+ status: draft
9
+ updated: 2026-09-02
10
+ owners: [ajean]
11
+ files:
12
+ - _underscore/Component/Api/RateBudget/RateBudget.php
13
+ - _underscore/Component/Api/Clickup/Clickup.php
14
+ - _underscore/Route.php
15
+ - _underscore/Query.php
16
+ - dbchanges2/Core/2026-09-01a - Add ApiRateBudget table.sql
17
+ - worker2/Worker/Clickup/Api.php
18
+ related:
19
+ - ../architecture.md
20
+ - ../../worker2/features/creating-worker-actions.md
21
+ - ../../worker2/features/clickup-project-routing.md
22
+ - ./async-query-execution.md
23
+ - ./model-save-vs-query-atomic-update.md
24
+ ---
25
+
26
+ ## Summary
27
+
28
+ A **shared outbound rate budget for every third-party API** we call from 2.0 (ClickUp,
29
+ NetSuite, FedEx, UPS, …): one small component plus one `Core` table, keyed by a
30
+ `bucketKey` string. It counts outbound calls per fixed 60-second window so we can *measure*
31
+ our real request rate before enforcing any cap.
32
+
33
+ **Status — nothing is shipped.** The component and its migration exist only as local staged
34
+ files; they are not committed to `_underscore` or `dbchanges2`, and no PR contains them. They
35
+ replace a rejected ClickUp-only design (533-line worker class + 106-line migration = 639 lines)
36
+ that lives in the still-open PRs worker2 **#137** and dbchanges2 **#449**. Read this doc as a
37
+ design record, not as live behavior.
38
+
39
+ ## Key files / entry points
40
+
41
+ - `_underscore/Component/Api/RateBudget/RateBudget.php` — `_Component_Api_RateBudget`
42
+ (staged; 222 lines, ~103 lines of code).
43
+ - `take(string $bucketKey, int $limit): ?int` — **the entire public surface.** Increments the
44
+ current window's counter and returns the post-increment count, or `null` if the store is
45
+ unreachable. It never blocks, never sleeps, never throws on saturation.
46
+ - `prune()` — deletes old windows; call from an existing cron, never on the hot path.
47
+ - `dbchanges2/Core/2026-09-01a - Add ApiRateBudget table.sql` — staged, 19 lines. Creates
48
+ `Core.ApiRateBudget`: `id`, `uuid`, `dtCreated`, `dtUpdated`, `bucketKey varchar(64)`,
49
+ `windowStart int UNSIGNED`, `requests int UNSIGNED`, with `UNIQUE KEY (bucketKey, windowStart)`.
50
+ - Rejected predecessor (in PR #137 / #449, ClickUp-only): `worker2/Worker/Clickup/Api.php`.
51
+
52
+ ## How it works
53
+
54
+ 1. A caller about to hit a provider calls `take('clickup', $limit)`.
55
+ 2. The component upserts `(bucketKey, windowStart)` incrementing `requests`, then reads the row
56
+ back by primary key, and returns the count.
57
+ 3. The caller decides what to do with the number. **Counting and enforcement are deliberately
58
+ separate.**
59
+
60
+ Design decisions worth keeping:
61
+
62
+ - **One table, one class, keyed by `bucketKey`** (`'clickup'`, `'netsuite'`, `'fedex'`, `'ups'`).
63
+ Never a second per-provider table or class. The table lives in `Core/`, not `Team/`, because it
64
+ is platform infrastructure, not team tooling.
65
+ - **Observe-only deployment is the point.** Run it for a week, read the real aggregate rate out of
66
+ `Core.ApiRateBudget`, then pick a threshold from measurement instead of guesswork. The
67
+ measurement *is* the test.
68
+ - **Fixed 60s window, boundary computed by MySQL** (`FLOOR(UNIX_TIMESTAMP()/N)*N`) and **never by
69
+ PHP** — instance clock drift across the fleet would split the budget. Accepted trade-off: a burst
70
+ straddling a boundary is bounded at 2x nominal. If that ever matters, move to a 10s window and
71
+ `SUM()` the last 6 rows.
72
+ - **Fail open.** If the store is unreachable: log once, return `null`, proceed with the call.
73
+ worker2 has **no DLQ and no redelivery**, so a fail-closed limiter outage would permanently
74
+ destroy production events — strictly worse than briefly exceeding a provider's cap.
75
+
76
+ ### Framework constraint: a cross-process counter needs its OWN DB alias, committed immediately
77
+
78
+ This is the one piece of essential complexity carried over from the rejected design, and it
79
+ applies to **any** shared counter or rollup in 2.0, not just this one.
80
+
81
+ `_underscore/Route.php` opens a transaction on the **default** connection and holds it open until
82
+ the job ends. So a counter written on the default alias is:
83
+
84
+ 1. **invisible to every other process** until the job ends — a fleet-wide total can never be read; and
85
+ 2. **a write lock held on one hot row** for up to the full 300s job budget — every worker that
86
+ touches the counter serialises behind it.
87
+
88
+ The only escape is to register a **second connection under its own alias** and call
89
+ `_Database::transactionCommit($alias)` immediately after the write. A "simplification" that drops
90
+ the alias silently breaks correctness *and* adds a fleet-wide lock.
91
+
92
+ ### Why not the one-statement `LAST_INSERT_ID(expr)` upsert-and-read
93
+
94
+ The tempting single statement —
95
+ `... VALUES (… LAST_INSERT_ID(1)) ON DUPLICATE KEY UPDATE requests = LAST_INSERT_ID(requests + 1)`
96
+ — is **not safe on a table with an `AUTO_INCREMENT id`**: `LAST_INSERT_ID()` returns the *new row
97
+ id* on the INSERT path but the *assigned expression* on the UPDATE path. The 2.0 table standard
98
+ mandates a leading `id AUTO_INCREMENT`, so the idiom would need empirical proof against the target
99
+ MySQL 8 before use.
100
+
101
+ **Decision:** upsert, then read the row back by primary key. One extra indexed lookup, obviously
102
+ correct. The resulting read-after-write skew is **upward-only** under concurrency (another process
103
+ may increment between our write and our read), so the number is conservative — it can cause an
104
+ early yield but never an overrun.
105
+
106
+ ### Enforcement policy (once there is data) — never sleep-wait
107
+
108
+ `MAX_EXECUTION_TIME_WORKER_ACTION` hard-caps any non-CronScheduler job at **300s**, and the
109
+ JobScheduler watchdog only *writes the failure row* — it does not stop the PHP process. Sleeping to
110
+ wait for budget therefore burns the cap and gets the job marked failed with **no retry and no DLQ**.
111
+ See [Creating Worker Actions](../../worker2/features/creating-worker-actions.md).
112
+
113
+ | Traffic class | Examples | Policy at the threshold |
114
+ |---|---|---|
115
+ | Bulk / sweep | sprint sync, stakeholder backfill, discovery | **Yield cleanly and reschedule the remainder.** Already chunked and resumable, so pausing costs nothing. This is where a cap gets real teeth. |
116
+ | Event-driven / escalation | Sentry→ClickUp, webhooks, design review | **Proceed over budget, counted and alarmed.** Low volume, irreplaceable. |
117
+
118
+ ## Verified findings about ClickUp metering today
119
+
120
+ - **The "500/min is worker2's deliberate half of ClickUp's 1000/min" premise is FALSE.** Both
121
+ `worker2/Worker/Clickup/Api.php` and the #449 migration assert that api2 and tools also call the
122
+ ClickUp API. Verified by grep across the `api2`, `tools` and `_underscore` checkouts: **zero**
123
+ references to `api.clickup.com` and **zero** `_Component_Api_Clickup::send()` calls in api2 or
124
+ tools. `tools` reads the `Team` database that worker2 populates; it does not call ClickUp.
125
+ worker2's realistic share is therefore **~900/min, not 500** — the limit must be re-derived with
126
+ the reviewer before any enforcement.
127
+ - **~24 of ~116 ClickUp call sites would be unmetered** (measured on the merged tree via
128
+ `git merge-tree --write-tree` + grep): 92 metered, 6 raw `_Component_Api_Clickup::send()`, and 18
129
+ hand-built `api.clickup.com` endpoints — `Worker/Team/Sprint.php` (6),
130
+ `Worker/Netsuite/Opportunity.php` (6), `Worker/Sentry.php` (4),
131
+ `Worker/Netsuite/OpportunityStakeholderBackfill.php` (2). The **burstiest** loops (paginated
132
+ sprint sweeps, the stakeholder backfill) are among the unmetered ones, so a limiter as built
133
+ misses exactly the traffic most likely to breach a cap. **Call-site consolidation is the real
134
+ work** — any limiter is only as good as that.
135
+ - **The rejected design's own enforcement test is blind to the biggest bypass.**
136
+ `TestNoDirectApiCalls()` greps only for `_Component_Api_Clickup::send(`, so it cannot see the 18
137
+ raw `api.clickup.com` sites, and it reports `RESULT: FAIL (2)` immediately post-merge because
138
+ `Worker/Monitor/Oneuptime.php` (5 unmetered calls) landed on `_production` after the branch was
139
+ cut. **A guard that passes while 18 bypasses exist manufactures false confidence.** This belongs
140
+ in CI as a ~10-line grep for `api.clickup.com` outside the component — not as a curl-invokable
141
+ worker action.
142
+
143
+ ## What was deliberately dropped from the rejected design
144
+
145
+ 45-vs-50 rolling-minute arithmetic, a 0.25s window-edge guard, batch reserve plus
146
+ `register_shutdown_function` slot refunds, `beginBulk()`, `isBudgetTight()`, and a `CHECK`
147
+ constraint hand-synced to a PHP constant. Rationale: that precision existed to guarantee
148
+ ≤495 requests per rolling minute — yet on saturation the limiter logged and **proceeded unmetered
149
+ anyway**, so the exact bound was unenforceable by design and the arithmetic bought nothing.
150
+
151
+ ## Gotchas / known issues
152
+
153
+ - `_Query::setQuery()` **executes non-SELECT queries immediately** — `new _Query($writeSql, $alias)`
154
+ runs the write eagerly, with no `execute()` call; `__destruct()` calls `execute()` as a backstop.
155
+ This is why any async flag must be set *before* `setQuery()`
156
+ (see [Asynchronous Query Execution](./async-query-execution.md)).
157
+ - `_Query::fetchRow()` returns an **object**, not an array — `$row->requests`, never
158
+ `$row['requests']`. The affected-rows accessor is **`getAffectedRows()`** (there is no
159
+ `affectedRows()`). `fetchOne()` returns the first column of the first row, or `false` — the
160
+ cleanest single-value read.
161
+ - **Security (location only):** `_underscore/Component/Api/Clickup/Clickup.php` declares a
162
+ hardcoded shared ClickUp workspace access token as a class constant next to `ENDPOINT`. Every
163
+ consumer authenticates as the same workspace identity, so per-consumer attribution and isolation
164
+ are impossible at the provider end — which also means a budget can never be attributed per caller
165
+ upstream. Needs rotation, a move to config/SSM, and a `/cso` pass. Already noted in
166
+ [ClickUp Design Sprint Automation](../../worker2/features/clickup-design-sprint-automation.md).
167
+
168
+ ## Ticket mapping (so the next reader does not conflate these)
169
+
170
+ - **TRUE-80662 "ClickUp - Fix Project Hub Routing"** (Sprint 83, back-end review; reviewer Jeff
171
+ Cardinal) — contains nothing about rate limiting. The limiter arrived *inside* its PRs (#137,
172
+ #449). The outbound budget work has **no ticket of its own yet**.
173
+ - **TRUE-80659** "formal rate limiting feature for the API … return a 429 with a Retry-Again
174
+ header" (filed by Jeff Cardinal; Backlog/On Bench, unassigned, unbuilt) — that is the **inbound**
175
+ 429 + `Retry-After` feature, a **different** concern from this outbound budget.
176
+ - **TRUE-4193** — an older per-IP **inbound** variant.
177
+
178
+ ## Change history
179
+ - 2026-09-02 — Created. Designed `_Component_Api_RateBudget` + `Core.ApiRateBudget` (staged, not
180
+ merged) as a provider-agnostic replacement for the rejected 639-line ClickUp-only limiter;
181
+ recorded the false 500/min premise, the ~24 unmetered ClickUp call sites, the blind enforcement
182
+ test, the dedicated-alias requirement for any cross-process counter, and the no-sleep-wait
183
+ enforcement tiering. (ajean)
184
+
185
+ ## Related docs
186
+ - [_underscore Framework Architecture](../architecture.md)
187
+ - [Creating Worker Actions](../../worker2/features/creating-worker-actions.md) — the 300s cap
188
+ - [ClickUp Project & Opportunity Multi-List Routing](../../worker2/features/clickup-project-routing.md) — PR #137's actual subject
189
+ - [Asynchronous Query Execution](./async-query-execution.md)
190
+ - [_Model::save() vs raw _Query](./model-save-vs-query-atomic-update.md)
@@ -3,7 +3,7 @@
3
3
  | Doc | Summary | Files |
4
4
  |-----|---------|-------|
5
5
  | [TOGa Commerce (toga2-commerce / commerce2-react) Architecture](architecture.md) | `toga2-commerce` (npm package name **`commerce2-react`**, product name **TOGa Commerce**) is the customer-facing **B2B commerce storefront** of the 2.0 platform | src/main.tsx, src/App.tsx, src/routes.tsx, src/contexts/AuthContext.tsx, src/contexts/helpers/getLoginSettings.ts, src/api/axiosInstance.ts, src/stores/, src/themeConfig/ThemeContext.tsx, src/fieldsConfig/index.ts, src/hooks/useAssignClientFields.ts, vite.config.ts, package.json |
6
- | [Bundle item visibility & selectability (the three flags, and why they are enforced nowhere but the client)](features/bundle-item-visibility-and-selectability.md) | Which components of a kit a shopper can see and choose is decided **entirely in the browser**. | toga2-commerce/src/pages/BundleView/api/BundleApi.tsx, toga2-commerce/src/pages/BundleView/helpers/formatOriginalBundleData.ts, toga2-commerce/src/pages/BundleView/helpers/formatBundleToAddToCart.ts, toga2-commerce/src/pages/BundleView/hooks/useBundleEditFromLocalStorage.ts, toga2-commerce/src/stores/useBundleBuilderZu.ts, _underscore/Model/Client/BundleItem.php, _underscore/Model/Client/SalesOrderItem.php, api2/Component/Api/V2/V2.php |
6
+ | [Bundle item visibility & selectability (the three flags, and why they are enforced nowhere but the client)](features/bundle-item-visibility-and-selectability.md) | Which components of a kit a shopper can see and choose is decided **entirely in the browser**. | toga2-commerce/src/pages/BundleView/api/BundleApi.tsx, toga2-commerce/src/pages/BundleView/helpers/formatOriginalBundleData.ts, toga2-commerce/src/pages/BundleView/helpers/formatBundleToAddToCart.ts, toga2-commerce/src/pages/BundleView/hooks/useBundleEditFromLocalStorage.ts, toga2-commerce/src/api/helpers/formatBundlesForSalesQuoteZuCartinEditCart.ts, toga2-commerce/src/api/genericApi.ts, toga2-commerce/src/api/FilterApi.ts, toga2-commerce/src/api/translatedSearchApi.ts, toga2-commerce/src/stores/useBundleBuilderZu.ts, _underscore/Model/Client/BundleItem.php, _underscore/Model/Client/SalesOrderItem.php, api2/Component/Api/V2/V2.php |
7
7
  | [Cart Bundle Submission & the bundleUuid Identity Contract](features/cart-bundle-submission-and-identity.md) | How cart **bundles** (kits) are turned into `SalesOrderItems` when a cart is submitted or an existing order is edited, and the **identity-field contract** every | src/api/syncSalesOrderItemsFromLocalStorageCartToApi.ts, src/utils/formatSalesOrderBundlesFromApi.ts, src/stores/useCartStoreZu.ts, src/pages/OrderDetails/helpers/formatSalesOrderDataFromLocalStorage.ts, src/pages/OrderDetails/view/components/OrderItems.tsx |
8
8
  | [Cart Notification Emails — duplicate prevention](features/cart-notification-emails.md) | On the cart "Notifications" section a user can add CC email addresses to an order. | src/pages/Cart/CartPage.tsx, src/pages/Cart/view/cartForm/CartForm.tsx, src/stores/useEmailOptionsStore.ts, src/stores/useCartSalesQuoteZu.ts, src/pages/Cart/viewModel/FIELDS/*/*/*/CARTPAGE.ts |
9
9
  | [Cart Order-Total & Shipping Computation](features/cart-order-total-computation.md) | The Cart summary section (subtotal / shipping / tax / total) is **data-driven** from `cartData`. | toga2-commerce/src/pages/Cart/viewModel/useCartViewModel.ts, toga2-commerce/src/pages/Cart/CartPage.tsx, toga2-commerce/src/pages/Cart/view/cartForm/CartForm.tsx, toga2-commerce/src/pages/Cart/helpers/shippingOptionGates.ts, toga2-commerce/src/pages/Cart/api/CartApi.ts |
@@ -6,13 +6,17 @@ project: TOGa Commerce
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-08-27
10
- owners: ["apeterson"]
9
+ updated: 2026-09-02
10
+ owners: ["apeterson", "jcardinal"]
11
11
  files:
12
12
  - toga2-commerce/src/pages/BundleView/api/BundleApi.tsx
13
13
  - toga2-commerce/src/pages/BundleView/helpers/formatOriginalBundleData.ts
14
14
  - toga2-commerce/src/pages/BundleView/helpers/formatBundleToAddToCart.ts
15
15
  - toga2-commerce/src/pages/BundleView/hooks/useBundleEditFromLocalStorage.ts
16
+ - toga2-commerce/src/api/helpers/formatBundlesForSalesQuoteZuCartinEditCart.ts
17
+ - toga2-commerce/src/api/genericApi.ts
18
+ - toga2-commerce/src/api/FilterApi.ts
19
+ - toga2-commerce/src/api/translatedSearchApi.ts
16
20
  - toga2-commerce/src/stores/useBundleBuilderZu.ts
17
21
  - _underscore/Model/Client/BundleItem.php
18
22
  - _underscore/Model/Client/SalesOrderItem.php
@@ -116,8 +120,28 @@ Also decided: **delete** the dead `where` clause in `BundleApi.tsx` rather than
116
120
  engine cannot prune nested children via `where` at all, and the clause could not express the
117
121
  fee / child-bundle exceptions even if it worked.
118
122
 
123
+ ## The bundle path never filters by catalog — it relays whatever item the API returns
124
+
125
+ The storefront applies a `catalogId = 1` filter **only** on standalone item search
126
+ (`src/api/genericApi.ts`, `FilterApi.ts`, `translatedSearchApi.ts`). The **bundle** path does not:
127
+ `GET /v2/bundles/{uuid}` returns each kit line's `bundleItems.item.uuid`, and the cart submit relays
128
+ that uuid verbatim (`src/api/helpers/formatBundlesForSalesQuoteZuCartinEditCart.ts`,
129
+ `BundleView/api/BundleApi.tsx`). So the item a kit checkout POSTs is **whatever `BundleItems.itemId`
130
+ resolves to on the server** — the browser never re-picks a catalog for it.
131
+
132
+ **Consequence:** if a kit's `BundleItems.itemId` points at the wrong-catalog item, **every checkout
133
+ of that kit silently POSTs the wrong item, and no frontend change fixes it — the data must be
134
+ corrected.** This was the ongoing source of *new* bad orders after Compass USA's 2026-08-31 catalog
135
+ fold: `BundleItems.itemId` for 434 active rows across 110 bundles still pointed at the Agilant
136
+ catalog-2 twins, so each new kit checkout POSTed the Agilant item. Root cause + the
137
+ restore-`BundleItems` fix:
138
+ [Compass catalog-fold reversal](../../../clients/compass-usa/workflows/catalog-fold-reversal.md).
139
+
119
140
  ## Gotchas / known issues
120
141
 
142
+ - **⚠ A folded/wrong `BundleItems.itemId` is invisible to the storefront's catalog filter.** The
143
+ catalog filter guards standalone search only; bundles relay the server's item uuid. Fix wrong-item
144
+ kits in the data (`BundleItems`), not in the frontend.
121
145
  - **⚠ Never read a flag on a `BundleItems` row without first knowing its row type.** Fee rows are
122
146
  invisible-but-mandatory; child-bundle rows are inactive-by-convention.
123
147
  - **⚠ Do not add a `where` to a uuid'd GET expecting nested filtering.** It is doubly inert (READ
@@ -145,6 +169,13 @@ direct API write (`Logs.Api` request payload for that order).
145
169
 
146
170
  ## Change history
147
171
 
172
+ - 2026-09-02 — Recorded that the storefront's `catalogId = 1` filter guards **standalone item
173
+ search only** (`genericApi.ts`, `FilterApi.ts`, `translatedSearchApi.ts`); the **bundle** path
174
+ relays whatever `bundleItems.item.uuid` `GET /v2/bundles/{uuid}` returns, so a kit checkout POSTs
175
+ whatever `BundleItems.itemId` resolves to server-side. A wrong-catalog `BundleItems.itemId`
176
+ therefore silently POSTs the wrong item and can only be fixed in data — this was the ongoing
177
+ source of new bad Compass orders after the 2026-08-31 catalog fold (434 rows / 110 bundles still
178
+ on the Agilant twins). (jcardinal)
148
179
  - 2026-08-27 — Created from an investigation-only session (no code changed) into how part number
149
180
  `S24D402GAN` reached Compass USA order `SA136441`. Recorded that the bundle fetch's
150
181
  `BundlesItems.isActive` `where` clause in `BundleApi.tsx` has **never executed** (uuid'd GET
@@ -6,7 +6,7 @@ project: Worker
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-08-31
9
+ updated: 2026-09-02
10
10
  owners: [jcardinal, ajean]
11
11
  files:
12
12
  - worker2/Worker/Clickup/Project.php
@@ -17,6 +17,7 @@ related:
17
17
  - ./netsuite-opportunity-sync.md
18
18
  - ./clickup-connectivity-watchdog.md
19
19
  - ./clickup-subtask-activity.md
20
+ - ../../_underscore/features/outbound-api-rate-budget.md
20
21
  ---
21
22
 
22
23
  ## Summary
@@ -166,6 +167,14 @@ None — uniform across all clients (shared internal automation for Agilant's Cl
166
167
  needs **`taskCreated` on LS + A&M** and a **workspace-wide `taskMoved`**, registered via
167
168
  `RegisterHubGuardWebhooks()` (which skips any already covered to avoid double-delivery).
168
169
 
170
+ - **PR #137 also carries a ClickUp rate limiter that is NOT part of this feature — and it was
171
+ rejected in review.** The open branch `fix/clickup-project-hub-routing` bundles a 533-line
172
+ `Worker/Clickup/Api.php` limiter (plus dbchanges2 #449) whose premise, coverage, and self-test are
173
+ all wrong: api2 and tools do **not** call ClickUp (verified by grep), ~24 of ~116 call sites stay
174
+ unmetered, and its `TestNoDirectApiCalls()` guard cannot see the 18 hand-built `api.clickup.com`
175
+ sites. Treat rate limiting as a separate subject with its own design —
176
+ [Outbound third-party API rate budget](../../_underscore/features/outbound-api-rate-budget.md).
177
+
169
178
  ## Diagnostics
170
179
 
171
180
  Read-only / setup actions, invoked via `curl -X POST https://worker.togahub.com/` with
@@ -176,6 +185,8 @@ Read-only / setup actions, invoked via `curl -X POST https://worker.togahub.com/
176
185
  `RESULT: PASS`), `RegisterOpportunityWebhook` and `RegisterHubGuardWebhooks` (one-time setup).
177
186
 
178
187
  ## Change history
188
+ - 2026-09-02 — Noted that PR #137 bundles a rejected ClickUp rate limiter unrelated to routing, and
189
+ split that subject out to its own doc. No routing behavior changed. (ajean)
179
190
  - 2026-08-31 — Recorded the triage consequence of the hybrid split (read-only investigation, no code
180
191
  changed): worker-created opportunity tasks were observed **homed in `901111987451`
181
192
  "Quote/Proposal/SOW/MSA"** rather than the `901111987449` list they were POSTed to, with the mover
@@ -17,6 +17,7 @@ related:
17
17
  - ../architecture.md
18
18
  - ./netsuite-opportunity-client-labels.md
19
19
  - ../../api2/features/api-payload-interceptors.md
20
+ - ../../_underscore/features/outbound-api-rate-budget.md
20
21
  ---
21
22
 
22
23
  ## Summary
@@ -232,6 +233,13 @@ production (2026-08-28, job `1075223`, a full `OpportunityStakeholderBackfill` d
232
233
  So an over-budget action loses its output **twice**: once to a false failure row, once to a dropped
233
234
  connection. Design for 300s.
234
235
 
236
+ **Corollary — never sleep-wait for an external rate budget inside an action.** Pausing to wait for
237
+ a third-party API slot burns the 300s cap, gets the job stamped failed by the watchdog, and there is
238
+ no retry and no DLQ to recover it. Instead: **bulk/sweep** actions yield at the threshold and
239
+ re-enqueue the remainder (they are already chunked and resumable, so pausing is free), while
240
+ **event-driven/escalation** actions proceed over budget, counted and alarmed. See
241
+ [Outbound third-party API rate budget](../../_underscore/features/outbound-api-rate-budget.md).
242
+
235
243
  ### Self-chunking: the pattern for work that cannot finish in one job
236
244
 
237
245
  An action that may exceed the budget works for a **sub-budget slice**, then **re-enqueues itself**
@@ -456,6 +464,9 @@ Rules: rename **both sides in one PR and one deploy**; grep every repo for the o
456
464
  commit-before-SQS transaction pattern that the worker relies on.
457
465
 
458
466
  ## Change history
467
+ - 2026-09-02 — Added the corollary that an action must **never sleep-wait for a third-party API
468
+ rate budget** (it burns the 300s cap and the failure is unrecoverable — no retry, no DLQ); bulk
469
+ work yields and re-enqueues, event-driven work proceeds over budget counted and alarmed. (ajean)
459
470
  - 2026-09-02 — Added two authoring rules learned on the End Customer label work (worker2 PR #158/#159):
460
471
  **gate any code that names a column from a hand-applied dbchanges2 migration** on an
461
472
  `information_schema.COLUMNS` check memoised in a `static` (an ungated write fails the whole query
@@ -5,8 +5,8 @@ project: _Underscore
5
5
  client: shared
6
6
  type: standard
7
7
  status: active
8
- updated: 2026-09-01
9
- owners: [jcardinal, mhammontree, dfranks, bala]
8
+ updated: 2026-09-02
9
+ owners: [jcardinal, mhammontree, dfranks, bala, ajean]
10
10
  files: []
11
11
  related:
12
12
  - ../apps/_underscore/architecture.md
@@ -521,6 +521,33 @@ unapproved for 6 days because the approval was stamped with a dead user id. Prod
521
521
  See: clients/compass-usa/features/approval-decision-flow.md and
522
522
  clients/compass-usa/features/stranded-approval-reassignment.md
523
523
 
524
+ ### A cross-process counter or shared rollup MUST use its own DB alias, committed immediately
525
+
526
+ `Route.php` opens a transaction on the **default** connection and holds it open until the request
527
+ or job ends. So anything you write on the default alias for other processes to read is broken twice
528
+ over:
529
+
530
+ 1. **Nobody else can see it** until your job ends — a fleet-wide total is unreadable by design.
531
+ 2. **You hold a write lock on one hot row** for up to the full 300s job budget, serialising every
532
+ other process that touches that counter.
533
+
534
+ So any shared counter, rate budget, sequence, or rollup must:
535
+
536
+ - register a **second connection under its own alias**, and
537
+ - call `_Database::transactionCommit($alias)` **immediately after the write** — not at end of job.
538
+
539
+ Removing the extra alias looks like a simplification. It is not: it silently breaks correctness and
540
+ adds a fleet-wide lock.
541
+
542
+ **Read the value back by primary key after the upsert.** Do not use the one-statement
543
+ `LAST_INSERT_ID(expr)` upsert-and-read idiom on a table with an `AUTO_INCREMENT id` (which our
544
+ table standard mandates): `LAST_INSERT_ID()` returns the new row id on the INSERT path but the
545
+ assigned expression on the UPDATE path, so the result is path- and version-dependent. One extra
546
+ indexed lookup is obviously correct. Under concurrency the read-after-write skew is upward-only,
547
+ so the number is conservative.
548
+
549
+ See [Outbound third-party API rate budget](../apps/_underscore/features/outbound-api-rate-budget.md).
550
+
524
551
  ## PHP
525
552
 
526
553
  ### PHP Tags
@@ -997,6 +1024,15 @@ See: 2.0/apps/worker2/features/cross-account-aws-access.md
997
1024
 
998
1025
  ## Change history
999
1026
 
1027
+ - 2026-09-02 — Database/SQL: any **cross-process counter or shared rollup needs its own DB alias,
1028
+ committed immediately**. `Route.php` holds a transaction open on the default connection until the
1029
+ request/job ends, so a counter written there is invisible to every other process (a fleet-wide
1030
+ total is unreadable) and holds a write lock on one hot row for up to the 300s job budget,
1031
+ serialising the fleet. Register a second connection under its own alias and
1032
+ `_Database::transactionCommit($alias)` right after the write. Also: read the value back by
1033
+ primary key instead of the one-statement `LAST_INSERT_ID(expr)` upsert-and-read idiom — with the
1034
+ mandated `AUTO_INCREMENT id` it returns the new row id on INSERT but the assigned expression on
1035
+ UPDATE, so it is path- and version-dependent. (ajean)
1000
1036
  - 2026-09-01 — TRUE-81576. Three additions: (1) Security — never ship a hardcoded-credential
1001
1037
  fallback; the FedEx `initialize()` "production fallback" holds a live `client_id`/`client_secret`
1002
1038
  byte-identical to `api2/Config/production.ini`, so it must throw on missing `[fedex]` config, the
@@ -18,7 +18,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
18
18
 
19
19
  ## 2.0 framework
20
20
 
21
- - **_underscore** (_Underscore) _(framework core)_ — 75 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
21
+ - **_underscore** (_Underscore) _(framework core)_ — 76 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
22
22
  - **worker2** (Worker) — 58 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
23
23
  - **api2** (API) — 25 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
24
24
  - **dbchanges2** (Database Changes) _(framework core)_ — 15 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
@@ -22,7 +22,7 @@
22
22
  | [Compass Sales-Order Line-Number Renumbering (and why it hangs off the SO hooks)](features/sales-order-line-renumbering.md) | 2.0 | Compass sales-order lines must stay numbered **1..N with no gaps** after any add, edit, or delete — downstream MITS/PO linking reads `lineNumber` as an identity | _underscore/Model/Compass/SalesOrderItem.php, _underscore/Model/Compass/SalesOrder.php |
23
23
  | [Stranded Approval Reassignment (repointing approvals off dead duplicate Compass Users rows)](features/stranded-approval-reassignment.md) | 2.0 | A Compass employee who leaves and comes back after more than `CONTACT_UNLINK_GRACE_DAYS` (5) is **inserted as a brand-new `Users` row** by the PEOPLE importer i | _underscore/Model/Compass/ApprovalDecision.php, worker2/Worker/Client/Compass/ApprovalReassignment.php, worker2/Worker/Client/Compass/PeopleFile.php |
24
24
  | [Compass USA](profile.md) | 2.0 | Compass USA is a TOGA client running a multi-tier supply-chain commerce operation. | |
25
- | [Compass USA — reversing the 199-item catalog fold (MITS outage recovery)](workflows/catalog-fold-reversal.md) | 2.0 | The **root cause** of the Compass MITS transmission outage (the `assetType`-NULL and duplicate-vendor symptoms) was an ad-hoc cross-client **duplicate-items mer | test/team/2.0 deployment/duplicate_items_merge_ALL_CLIENTS.sql, test/team/2.0 deployment/duplicate_items_merge_analysis.sql, worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php, library/app/api/toga2.php, worker/crons/toga2/netsuite/common_sync_togasupply.php |
25
+ | [Compass USA — reversing the 199-item catalog fold (MITS outage recovery)](workflows/catalog-fold-reversal.md) | 2.0 | The **root cause** of the Compass MITS transmission outage (the `assetType`-NULL and duplicate-vendor symptoms) was an ad-hoc cross-client **duplicate-items mer | test/team/2.0 deployment/duplicate_items_merge_ALL_CLIENTS.sql, test/team/2.0 deployment/duplicate_items_merge_analysis.sql, worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php, worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, library/app/api/toga2.php, worker/crons/toga2/netsuite/common_sync_togasupply.php |
26
26
  | [Compass Cross-Kit Bundle Corruption — Detection & Repair](workflows/cross-kit-bundle-corruption.md) | 2.0 | A frontend regression in `toga2-commerce`'s edit-order bundle submission mis-attributed bundle (kit) line items and **fees/warranties** to the **wrong kit**, pe | src/api/syncSalesOrderItemsFromLocalStorageCartToApi.ts |
27
27
  | [Granting a Compass user the same toga25-supply navigation as another user (role vs. SurfaceOverride)](workflows/granting-navigation-access.md) | 2.0 | "Give user X the same menu items user Y has" is the **navigation analog** of the recurring [bundle/persona grant](./granting-persona-bundle-access.md) request. | dbchanges2/Client_Compass/2026-08-12a - NavigationRoleGrants.sql, dbchanges2/Client_Compass/2026-02-25 - AgilantUsersViewAccess.sql, dbchanges2/Client_Compass/2026-08-11 - CompassODPViewVendorItemsAccess.sql, dbchanges2/Core/2026-08-12a - NavigationSurfaceSeed.sql |
28
28
  | [Granting a Compass user access to a bundle (persona grant) and the SuperUser role](workflows/granting-persona-bundle-access.md) | 2.0 | "Give user X sight of kit N" is a **recurring** Compass request, usually paired with "and make them a super user". | dbchanges2/Client_Compass/2026-08-26a - CompassCreativeStudioPersona.sql |
@@ -6,7 +6,7 @@ project: Library
6
6
  client: compass-usa
7
7
  type: client-feature
8
8
  status: active
9
- updated: 2026-09-01
9
+ updated: 2026-09-02
10
10
  owners: ["bala", "jcardinal"]
11
11
  files:
12
12
  - library/app/api/toga2.php
@@ -124,14 +124,18 @@ Plus `-1` / `-2` suffixed copies: items **2431, 2432, 2443, 2653**.
124
124
  span more than one catalog.** It matters only as a thing to re-check, because a part-number lookup
125
125
  with no catalog filter has **no ambiguity check on its exact-match branch**
126
126
  ([850 line-item resolution](odp-edi-850-item-resolution.md)).
127
- **⚠ Unresolved tension — reconcile before relying on "partNumber is unique."** The
128
- [2026-08-31 catalog-fold reversal](../workflows/catalog-fold-reversal.md) is recorded as having
129
- folded **199 catalog-1 items into their catalog-2 Agilant canonical by `UPPER(partNumber)`**,
130
- which requires those 199 to have shared a part number across the two catalogs pre-merge, and the
131
- reversal restored them. That does not square with "zero part numbers span two catalogs" verified
132
- the same day. Confirm on prod whether the restored catalog-1 twins carry **distinct** part numbers
133
- (e.g. the `#B1H` / `-S` variant forms above) or genuinely-shared ones before treating table-wide
134
- `partNumber` uniqueness as settled.
127
+ **Resolved: the twins carry DISTINCT part-number variants, so table-wide `partNumber` uniqueness
128
+ holds.** The Agilant `#ABA` form never equals the plain part number (e.g. `C40QYUC#ABA` vs
129
+ `C40QYUC`; `B7HJ2UC#ABA` vs `B7HJ2UC`), so any `UPPER(partNumber)` or partNumber-twin match
130
+ silently misses them. That is exactly why the
131
+ [catalog-fold reversal](../workflows/catalog-fold-reversal.md) ultimately mapped dupe→canon by
132
+ **primary key** from the pre-merge snapshot, never by part number, and why the `#ABA` PO-line
133
+ stragglers needed a manual VendorItem re-map. Only a history question remains (how the original
134
+ merge's `UPPER(partNumber)` partition folded pairs that do not share a part number); nothing
135
+ downstream depends on it. **⚠ Note:** the exact item ids ↔ catalog assignments in the
136
+ duplicate-pairs table above predate the 2026-08-31 fold and its reversal (which deleted then
137
+ restored catalog-1 items by their original snapshot ids), so re-read specific ids from current prod
138
+ before relying on them.
135
139
 
136
140
  ## Filtering by catalog: fine for *display*, never for *resolution*
137
141
 
@@ -196,6 +200,13 @@ children" rule:
196
200
  - **⚠ A NULL `catalogId` is a real state** (27 items in prod, 22 ODP `VendorItems` point at such
197
201
  items). Anything joining through `Catalogs` **drops those rows silently** — an inner join to
198
202
  `Catalogs` is itself a filter.
203
+ - **⚠ `BundleItems.itemId` is a silent catalog-resolution consumer — the storefront never
204
+ re-catalogs it.** The `toga2-commerce` storefront relays whatever item `GET /v2/bundles/{uuid}`
205
+ returns; its `catalogId = 1` filter guards standalone item search only, never bundles. So a kit
206
+ whose `BundleItems.itemId` points at the wrong-catalog twin makes **every checkout POST the wrong
207
+ item** (434 rows / 110 bundles after the 2026-08-31 fold). Fix in data, not the frontend. See
208
+ [catalog-fold reversal](../workflows/catalog-fold-reversal.md) and
209
+ [bundle item visibility & selectability](../../../2.0/apps/toga2-commerce/features/bundle-item-visibility-and-selectability.md).
199
210
  - **⚠ `isFulfillable` is born on the catalog-2 (Agilant) item** and propagates to the client-facing
200
211
  copy; the flag's home catalog is not an accident. See
201
212
  [isFulfillable data quality & the type rule](isfulfillable-data-quality-and-type-rule.md).
@@ -212,6 +223,15 @@ children" rule:
212
223
  [catalog-fold reversal](../workflows/catalog-fold-reversal.md).
213
224
 
214
225
  ## Change history
226
+ - 2026-09-02 — **Resolved the flagged "distinct vs shared partNumber" tension**: the Agilant twins
227
+ carry **distinct** part-number variants (the `#ABA` form ≠ the plain part number, e.g.
228
+ `C40QYUC#ABA` vs `C40QYUC`), so table-wide `partNumber` uniqueness holds and the catalog-fold
229
+ reversal maps dupe→canon by **primary key**, never by part number (part-number matching misses the
230
+ `#ABA` stragglers). Flagged that exact item ids ↔ catalog assignments in the duplicate-pairs table
231
+ predate the fold+reversal and should be re-read from prod. Added `BundleItems.itemId` as a silent
232
+ catalog-resolution consumer — the storefront relays the bundle item uuid with no catalog filter, so
233
+ a folded `BundleItems.itemId` (434 rows / 110 bundles) POSTs the wrong item on every kit checkout
234
+ until the data is fixed. (jcardinal)
215
235
  - 2026-09-01 — Folded in the **customer → catalog rule** (`customerId = 2` → catalog 1, everyone
216
236
  else → catalog 2; verified 182,787 vs 1 pre-merge) and the mandatory `customerId` scope on any
217
237
  catalog re-point (an unscoped one moved 687 Office-Depot lines). Added that the NetSuite sync
@@ -5,8 +5,8 @@ project: _Underscore
5
5
  client: compass-usa
6
6
  type: client-feature
7
7
  status: active
8
- updated: 2026-07-14
9
- owners: ["rgirish", "bala"]
8
+ updated: 2026-09-02
9
+ owners: ["rgirish", "bala", "jcardinal"]
10
10
  files:
11
11
  - worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php
12
12
  - worker/crons/toga2/compasscanada/workflow/2_transmit_mits_purchase_orders_to_vendors.php
@@ -14,6 +14,8 @@ files:
14
14
  related:
15
15
  - mits-po-to-so-item-linking.md
16
16
  - order-lifecycle-and-data-integrity.md
17
+ - item-catalogs-and-duplicate-items.md
18
+ - ../workflows/catalog-fold-reversal.md
17
19
  ---
18
20
 
19
21
  ## Summary
@@ -87,6 +89,19 @@ Compass Canada has a parallel `compasscanada/workflow/2_transmit_...` keyed on
87
89
  `CUSTOMER_ID__COMPASS_CANADA`.
88
90
 
89
91
  ## Gotchas / known issues
92
+ - **⚠ Everything transmitted is sourced from the `VendorItem` the PO line points at — a
93
+ wrong-catalog `vendorItemId` sends the wrong SKU, wrong cost, AND the wrong channel.** The vendor
94
+ SKU and the cXML-vs-EMAIL routing come entirely from the `VendorItem` that
95
+ `PurchaseOrderItems.vendorItemId` references (with `PurchaseOrderItems.cost` for price). So a PO
96
+ line pointing at an **Agilant** VendorItem instead of the Office-Depot one transmits Agilant's
97
+ **internal SKU** and **wrong cost**, and — because that VendorItem carries **no
98
+ `overridePoSubmissionIntegrationId`** — falls back to the vendor's default **EMAIL** integration
99
+ instead of **cXML**. Verified in the legacy `Logs.API`: good order **SA136666** went **cXML** to
100
+ `b2b.odpbusiness.com` with `SupplierPartID 5775600` / `UnitPrice 945.17`; bad order **SA136846**
101
+ went **email** with `C40QYUC#ABA` / `$61`. This was a downstream symptom of the 2026-08-31 catalog
102
+ fold (a folded PO line resolved to the Agilant twin) — root cause and reversal:
103
+ [catalog-fold reversal](../workflows/catalog-fold-reversal.md);
104
+ [two-catalog item model](item-catalogs-and-duplicate-items.md).
90
105
  - **Item-less PO is silently marked submitted without being sent (the "logs show not sent
91
106
  but dtSubmitted is set" symptom).** The `dtSubmitted = NOW()` UPDATE runs **unconditionally
92
107
  after** the per-integration `foreach`, not inside it. If a PO has **zero
@@ -109,6 +124,12 @@ Compass Canada has a parallel `compasscanada/workflow/2_transmit_...` keyed on
109
124
  `dtSubmitted` stamped without a log (separate, previously-observed failure mode).
110
125
 
111
126
  ## Change history
127
+ - 2026-09-02 — Recorded that cron 2 sources the vendor **SKU, cost, and cXML-vs-EMAIL routing**
128
+ entirely from the `VendorItem` at `PurchaseOrderItems.vendorItemId` (plus `PurchaseOrderItems.cost`),
129
+ so a PO line folded onto an **Agilant** VendorItem transmits Agilant's internal SKU, the wrong
130
+ cost, and — lacking `overridePoSubmissionIntegrationId` — drops from cXML to the default EMAIL
131
+ channel. Evidence: good `SA136666` cXML `SupplierPartID 5775600` / `945.17`; bad `SA136846` email
132
+ `C40QYUC#ABA` / `$61`. Downstream symptom of the 2026-08-31 catalog fold. (jcardinal)
112
133
  - 2026-07-14 — Fixed the emailed-PO PDF for both US Compass and Compass Canada: line-item
113
134
  rows now advance by their measured wrapped height (long descriptions no longer overwritten),
114
135
  and pagination is height-aware (a real 21-item PO went from a broken 17-page PDF to a clean
@@ -5,7 +5,7 @@ project: _Underscore
5
5
  client: compass-usa
6
6
  type: client-feature
7
7
  status: active
8
- updated: 2026-08-31
8
+ updated: 2026-09-02
9
9
  owners: ["dfranks", "bala", "jcardinal"]
10
10
  files:
11
11
  - worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
@@ -125,7 +125,12 @@ backlog (0 new, several >24h, one ~20 days old).
125
125
  order**. This was the active blocker (e.g. item `4U555A-S`; ~10 distinct NULL-assetType
126
126
  items blocked 9 of 11 pending orders).
127
127
  2. **HTTP 403 `SalesOrder:Exists`** — MITS already created the order on a prior attempt, but
128
- our side never recorded `c_dtTransmittedToCompass`, so the cron **retries it forever**.
128
+ our side never recorded `c_dtTransmittedToCompass`, so the cron **retries it forever**. **Now
129
+ handled (2026-09-02):** on a 403 `SalesOrder:Exists` the cron stamps
130
+ `SalesOrders.c_dtTransmittedToCompass`, writes a `Record`, and **continues** to the next order
131
+ instead of throwing — so an already-created order no longer starves the queue. Root cause of the
132
+ original loop (why we never stamped the first time) was the `App_ApiTransaction` decode fatal —
133
+ see the gotcha below.
129
134
 
130
135
  **Deeper root cause of the 2026-08-31 outage — a wrong catalog merge, not just missing data.**
131
136
  The NULL-assetType items were largely created by an ad-hoc cross-client dedup that folded
@@ -170,7 +175,19 @@ pages even when the count is under threshold. See
170
175
  console, pushed to ClickUp, and emailed — so the payload was deliberately removed. Alerts here
171
176
  name the order, not the person. (This is precisely what Canada still does wrong.)
172
177
  - **The MITS crons hold their API credentials inline in the file.** Treat both files as
173
- secret-bearing when copying snippets anywhere (tickets, PR bodies, knowledge docs).
178
+ secret-bearing when copying snippets anywhere (tickets, PR bodies, knowledge docs). The two
179
+ hardcoded MITS credentials live inline in
180
+ `1_transmit_compass_sales_orders_to_mits.php` — note their location, never paste the values.
181
+ - **⚠ 1.0 `App_ApiTransaction` built with `ENCODE_JSON` returns an ALREADY-DECODED object — do
182
+ NOT `json_decode()` it.** A transaction constructed with `App_ApiTransaction::ENCODE_JSON` hands
183
+ back a decoded object from `execute()`, not a JSON string. Calling `json_decode()` on it raises a
184
+ PHP 7.2 `"expects parameter 1 to be string, object given"` warning, which `App_Error` escalates to
185
+ a **FATAL**. In `1_transmit_compass_sales_orders_to_mits.php` that fatal fired **after** the order
186
+ transmitted to MITS but **before** stamping `c_dtTransmittedToCompass` — producing a ~42-order
187
+ backlog and the endless `SalesOrder:Exists` retry loop above (MITS kept the created order, we
188
+ never recorded it). Correct pattern:
189
+ `$decodedResp = is_string($resp) ? json_decode($resp) : $resp;`. This is a reusable 1.0 footgun,
190
+ not a Compass-only one — any `App_` code that `json_decode()`s an `ENCODE_JSON` response hits it.
174
191
  - **The transmit item query's `GROUP BY SalesOrderItems.id` hides false "duplicates."** In
175
192
  `1_transmit_compass_sales_orders_to_mits.php` (~L178-251, `GROUP BY` at L247) the
176
193
  `GROUP BY SalesOrderItems.id` collapses the `LEFT JOIN VendorItems` / `LEFT JOIN
@@ -193,6 +210,15 @@ pages even when the count is under threshold. See
193
210
  when this issue next fires. Watch that run rather than assuming it works.
194
211
 
195
212
  ## Change history
213
+ - 2026-09-02 — **Shipped HTTP 403 `SalesOrder:Exists` handling** in
214
+ `1_transmit_compass_sales_orders_to_mits.php`: on that response the cron now stamps
215
+ `c_dtTransmittedToCompass`, writes a `Record`, and continues instead of throwing forever (which
216
+ had starved the `ORDER BY RAND()` queue). Recorded the **root cause** of the loop — a 1.0
217
+ `App_ApiTransaction` built with `ENCODE_JSON` returns an **already-decoded object**, so a
218
+ `json_decode()` on it raises a PHP 7.2 warning that `App_Error` escalates to a **FATAL** *after*
219
+ transmit but *before* the stamp, producing a ~42-order backlog and the endless retry loop; the
220
+ fix is `is_string($resp) ? json_decode($resp) : $resp`. Flagged it as a reusable 1.0 footgun. Also
221
+ noted the two hardcoded MITS credentials' location. (jcardinal)
196
222
  - 2026-09-01 — Traced the 2026-08-31 outage to its **deeper root cause**: an ad-hoc
197
223
  cross-client dedup folded Compass's 199 catalog-1 store items into their Agilant canonical, so
198
224
  order lines resolved to unclassified Agilant items (HTTP 400) and their fused VendorItems
@@ -19,7 +19,7 @@ project: _Underscore
19
19
  client: compass-usa
20
20
  type: profile
21
21
  status: active
22
- updated: 2026-09-01
22
+ updated: 2026-09-02
23
23
  owners: [jcardinal, bala, tcox, apeterson, dfranks, akhokhani]
24
24
  files: []
25
25
  related:
@@ -92,6 +92,12 @@ separate, related client (see its own profile).
92
92
  corrupted in `Client_Compass.SalesOrderItems` (Canada and Quad: zero). The code fix is
93
93
  preventive; existing rows still need a DB data-fix. Detection query + remediation plan:
94
94
  [Cross-Kit Bundle Corruption](workflows/cross-kit-bundle-corruption.md).
95
+ - **⚠ A folded `BundleItems.itemId` makes the storefront POST the wrong-catalog item (2026-09-02).**
96
+ The storefront relays whatever item `GET /v2/bundles/{uuid}` returns and filters `catalogId = 1`
97
+ only on standalone search, never on bundles — so after the 2026-08-31 catalog fold, 434 kit-line
98
+ rows across 110 bundles still on the Agilant twins produced a fresh bad order on every kit
99
+ checkout. Fixed by restoring `BundleItems` in the complete catalog-fold reversal (fix is data, not
100
+ frontend): [catalog-fold reversal](workflows/catalog-fold-reversal.md).
95
101
 
96
102
  ## Vendors & integrations
97
103
  - **Office Depot (ODP)** — vendor id 1. ASNs arrive via **cXML** (direct V2 API) and via the
@@ -5,17 +5,20 @@ project: _Underscore
5
5
  client: compass-usa
6
6
  type: workflow
7
7
  status: active
8
- updated: 2026-09-01
8
+ updated: 2026-09-02
9
9
  owners: ["jcardinal", "bala"]
10
10
  files:
11
11
  - test/team/2.0 deployment/duplicate_items_merge_ALL_CLIENTS.sql
12
12
  - test/team/2.0 deployment/duplicate_items_merge_analysis.sql
13
13
  - worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php
14
+ - worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php
14
15
  - library/app/api/toga2.php
15
16
  - worker/crons/toga2/netsuite/common_sync_togasupply.php
16
17
  related:
17
18
  - ../features/item-catalogs-and-duplicate-items.md
18
19
  - ../features/mits-sales-order-transmission-alerting.md
20
+ - ../features/mits-po-transmission-to-vendors.md
21
+ - ../../../2.0/apps/toga2-commerce/features/bundle-item-visibility-and-selectability.md
19
22
  - ../../../1.0/apps/worker/workflows/netsuite-duplicate-items-dedup.md
20
23
  ---
21
24
 
@@ -59,48 +62,116 @@ NULL-catalog junk items** and **199 duplicate-Agilant items** (from the NetSuite
59
62
  inventory-adjustment item-create bug). **Only the 199 catalog-1 → catalog-2 folds were the
60
63
  damage.**
61
64
 
62
- **⚠ Open contradiction — not resolved, do not assume either side.** Folding by
63
- `UPPER(partNumber)` requires those 199 pairs to have **shared a part number across catalogs**
64
- pre-merge, and the reversal restored them. But a prod check of `Client_Compass` on **2026-09-01**
65
- found **zero part numbers duplicated at all, and zero spanning more than one catalog**. Both
66
- statements cannot be true of prod today. Re-check on prod whether the restored catalog-1 twins
67
- carry **distinct** part-number variants (the `#B1H` / `-S` forms) or genuinely-shared ones before
68
- relying on either the 199 figure or table-wide `partNumber` uniqueness. Same flag from the other
69
- side: [two-catalog item model](../features/item-catalogs-and-duplicate-items.md).
70
-
71
- ## The reversal playbook (durable)
72
-
73
- 1. **Restore a pre-merge snapshot as a scratch DB** — e.g. `Client_Compass_20260831_120000`
74
- (taken ~12:00, before the ~12:43 merge). **Validate it is pre-merge**: catalog-1 twins
75
- present, `customerId = 2` lines on catalog 1.
76
- 2. **Rebuild the merge's damage map from the snapshot** — the Agilant-preferred ranking above,
77
- filtered to `dupe.catalog = 1 AND canon.catalog = 2` = the 199.
78
- 3. **Write a set-based, single-transaction, FK-checks-ON, NO `IGNORE`, COMMIT-left-commented
79
- script** (`SNAP_DB.` / `TARGET_DB.` qualifiers) that:
80
- - Restores the 199 Compass `Items` + their `VendorItems` / `VendorItemCosts` **by id**
81
- (`NOT EXISTS` guard).
82
- - **Re-points current `customerId = 2` `SalesOrderItems`** from the Agilant canonical back
83
- to the Compass twin — **on CURRENT data, not a uuid-restore.** The sync re-created order
84
- lines with **new uuids**, so restoring snapshot lines by uuid **collided 1062** on
85
- `UNIQUE(salesOrderId, lineNumber)`.
86
- 4. **Rehearse on a same-day full prod clone** — this run: ~49,267 re-points, 199 items, 0
87
- orphans, held items 0-missing-assetType / 0-multi-vendor.
88
- 5. **Apply to prod with a pre-run backup.** Result: queue drained to 0 pending; all
89
- `customerId = 2` lines back on catalog 1.
90
-
91
- **CTO review gated this** (rehearse-on-clone, fail-loud, customer-scoped, tenant-locked).
65
+ **Part-number matching is unreliable here — the reversal maps dupe→canon by PRIMARY KEY, not
66
+ part number.** The Agilant twins carry **distinct** part-number variants, not shared ones: the
67
+ `#ABA` form does not equal the Compass part number (`C40QYUC#ABA` = item 2216 vs Compass
68
+ `C40QYUC` = 2157; `B7HJ2UC#ABA` = 2215 vs Compass `B7HJ2UC` = 2160). So any `UPPER(partNumber)`
69
+ or partNumber-twin re-point silently misses them, which is why the complete reversal keys off the
70
+ snapshot's own primary keys (below) and table-wide `partNumber` uniqueness still holds on prod.
71
+ (Left open only as history: how the original merge's `UPPER(partNumber)` partition folded pairs
72
+ that do not share a part number — the reversal does not depend on the answer, because it never
73
+ matches on part number.) Same note from the other side:
74
+ [two-catalog item model](../features/item-catalogs-and-duplicate-items.md).
75
+
76
+ ## The first reversal was INCOMPLETE — it left 10 tables folded
77
+
78
+ The earlier reversal (`compass_twocatalog_fix.sql`) restored `Items` / `VendorItems` /
79
+ `VendorItemCosts` and re-pointed **`SalesOrderItems`** — but **only the `customerId = 2` slice**
80
+ of those. It **left folded** every other table the merge touched: `customerId = 1` and
81
+ `customerId = 3` `SalesOrderItems` (~50k rows), `ItemFeatures`, `ItemImages`, **`BundleItems`**,
82
+ `Personas_Items`, `Units`, `ItemPrices`, `ItemMedia`, `AssortmentItems`, and `InvoiceItems` — 10
83
+ tables still pointing at the Agilant canonical.
84
+
85
+ **Durable rule: a fold reversal MUST cover EVERY table the merge enumerates.** Restoring the
86
+ parent `Items` and re-pointing one child table is not a reversal. The
87
+ `duplicate_items_merge_ALL_CLIENTS.sql` block lists **38** reference points — read it as the
88
+ checklist:
89
+
90
+ - **24 `itemId` tables**,
91
+ - **`Entitlements.saleItemId`**,
92
+ - **3 `vendorItemId` tables** — `PurchaseOrderItems`, `VendorItemCosts`, `Personas_VendorItems`,
93
+ - **10 `unitId` tables**.
94
+
95
+ Anything on that list still resolving to a canonical is unreversed. The left-folded `BundleItems`
96
+ in particular was the ongoing source of *new* bad orders (see below).
97
+
98
+ ## The complete reversal (durable method)
99
+
100
+ Restore **every** fold-affected row from the pre-merge snapshot
101
+ `Client_Compass_20260831_120000` (taken ~12:00, before the ~12:43 merge) **BY PRIMARY KEY** — not
102
+ by part-number twin matching (the `#ABA` variants make that unreliable, see above). The snapshot's
103
+ own primary keys are the only dependable dupe→canon mapping. **Validate the snapshot is pre-merge
104
+ first** (catalog-1 twins present, `customerId = 2` lines on catalog 1). Then, in a single
105
+ transaction, FK-checks-ON, **no `UPDATE IGNORE`**, COMMIT left commented (`SNAP_DB.` / `TARGET_DB.`
106
+ qualifiers):
107
+
108
+ 1. **Parents — full-row restore.** `Items`, `VendorItems`, `VendorItemCosts`, `Units` restored
109
+ whole from the snapshot by id (`NOT EXISTS` guard).
110
+ 2. **Children — FK-column-only restore.** For every child table on the merge's 38-point list,
111
+ restore **only the folded FK column** to its snapshot value, keyed by the child row's own PK.
112
+ (Verified zero non-FK drift, so nothing else is touched.)
113
+ 3. **Post-snapshot "broken-window" rows** — rows created **after 2026-08-31 12:00** that are not
114
+ in the snapshot are re-pointed canon→dupe on **current** data, **scoped `customerId = 2`** (the
115
+ uuid-collision and customer-scope rules below still apply — the sync re-created order lines with
116
+ **new uuids**, so a uuid-restore collided 1062 on `UNIQUE(salesOrderId, lineNumber)`).
117
+ 4. **Verify byte-for-byte.** This run: **0** rows still folded across all **38** tables and **0**
118
+ byte mismatches over **208,845** compared rows.
119
+
120
+ **Rehearse on a same-day full prod clone, then apply with a pre-run backup. CTO review gated the
121
+ approach** (rehearse-on-clone, fail-loud, customer-scoped, tenant-locked).
122
+
123
+ ### Stragglers: `#ABA`-variant PO lines need a manual VendorItem re-map
124
+ Because the `#ABA` twins share no part number, neither the merge's part-number map nor a
125
+ partNumber-twin re-point catches PurchaseOrder lines sitting on them. These were fixed by an
126
+ **explicit VendorItem mapping**, scoped to **pending `customerId = 2` PO lines**, then clearing
127
+ `PurchaseOrders.dtSubmitted` so the PO re-transmits: VendorItem **3845 → 3597** @ `945.17`,
128
+ VendorItem **3884 → 3600** @ `1627.38`. Why a mis-mapped PO line transmits the wrong SKU + cost
129
+ and drops to EMAIL rather than cXML:
130
+ [MITS PO transmission to vendors](../features/mits-po-transmission-to-vendors.md).
131
+
132
+ ## The folded `BundleItems` was the storefront's ongoing bad-order source
133
+ `BundleItems.itemId` (kit definitions) still pointed at the Agilant catalog-2 twins — **434 active
134
+ rows across 110 bundles**. The `toga2-commerce` storefront does **not** choose a catalog: it
135
+ relays whatever item uuid `GET /v2/bundles/{uuid}` returns (`bundleItems.item.uuid`). The
136
+ `catalogId = 1` filter runs **only** on standalone item search (`genericApi.ts`, `FilterApi.ts`,
137
+ `translatedSearchApi.ts`), **never** on bundles. So a folded `BundleItems.itemId` makes **every new
138
+ checkout of that kit POST the Agilant item** — a fresh wrong-catalog order each time. **The fix is
139
+ data (restore `BundleItems`); the frontend needs no change.** Storefront-side detail:
140
+ [bundle item visibility & selectability](../../../2.0/apps/toga2-commerce/features/bundle-item-visibility-and-selectability.md).
141
+
142
+ ## The NetSuite→TOGa sync is NOT the source of new cust-2 bad orders (suspect ruled out)
143
+ The 1.0 NetSuite→TOGa TOGaSupply sync (`worker/crons/toga2/netsuite/`,
144
+ `App_Api_Toga2::syncSalesOrderFromNetsuite`, `library/app/api/toga2.php` ~L752) writes **only the
145
+ Agilant `customerId = 3` sales order** for Compass — it **pins** `locationId = 615286` and
146
+ `customerId = 3` (~L807-809) and cannot write or re-point a `customerId = 2` (Compass storefront)
147
+ order. **Stopping it fixed nothing.** The real ongoing source of new bad orders was the folded
148
+ `BundleItems` (above), not the sync — this corrects the earlier "the sync re-points cust-2 lines
149
+ back to Agilant / stop it first" concern. (The sync still *creates items* in catalog 2 — that is
150
+ the original duplicate-item source, see the two-catalog model.)
151
+
152
+ Context — the Compass order chain is up to **four** SalesOrders, linked by the
153
+ `SalesOrders_PurchaseOrders` / `PurchaseOrders_SalesOrders` bridge tables: Compass storefront SO
154
+ (`cust 2`) → MITS PurchaseOrder → ODP SO (`cust 1`) → Agilant/NetSuite SO (`cust 3`). The catalog
155
+ rule still holds: `cust 2` → catalog 1; `cust 1` and `cust 3` → catalog 2.
92
156
 
93
157
  ## Critical rules
94
158
 
159
+ - **Cover EVERY table the merge lists — restore by PRIMARY KEY, not by part number.** The merge
160
+ enumerates 38 reference points (24 `itemId` + `Entitlements.saleItemId` + 3 `vendorItemId` +
161
+ 10 `unitId`); a reversal that touches only `Items` + one child table is incomplete (the first
162
+ attempt left 10 tables folded). `#ABA` twins share no part number, so partNumber matching misses
163
+ them — map dupe→canon from the snapshot's own PKs.
95
164
  - **Scope every catalog re-point by `customerId`.** An unscoped
96
165
  `UPDATE ... WHERE itemId = <agilantId>` wrongly moved **687 Office-Depot lines** onto the
97
166
  Compass item and had to be corrected.
98
- - **Re-point on CURRENT data, never restore order lines by uuid** — the sync minted new uuids,
99
- so a uuid-restore collides on `UNIQUE(salesOrderId, lineNumber)` (1062).
100
- - **Stop the Compass NetSuite sync first.** The sync re-creates Compass order lines on Agilant
101
- items, so the reversal is only durable if the sync is stopped; re-enabling risks re-pointing
102
- new/re-synced orders back to Agilant (2 lines seen after a test re-enable). See the
103
- [two-catalog item model](../features/item-catalogs-and-duplicate-items.md).
167
+ - **Re-point post-snapshot rows on CURRENT data, never restore order lines by uuid** — the sync
168
+ minted new uuids, so a uuid-restore collides on `UNIQUE(salesOrderId, lineNumber)` (1062).
169
+ - **The NetSuite→TOGa sync is NOT the ongoing source of new cust-2 bad orders — the folded
170
+ `BundleItems` is.** `syncSalesOrderFromNetsuite` pins `customerId = 3` for Compass and writes
171
+ only the Agilant-tier SO, so it cannot re-point a cust-2 storefront line; stopping it changes
172
+ nothing. Restore `BundleItems` to stop new wrong-catalog checkouts. (The sync still *creates
173
+ items* in catalog 2 — that is the original duplicate-item source, see the
174
+ [two-catalog item model](../features/item-catalogs-and-duplicate-items.md).)
104
175
  - **Never `UPDATE IGNORE` + delete-leftover for a catalog operation** — it permanently deletes
105
176
  collision rows and cannot be reversed. Use FK-checks-ON, fail-loud, single transaction.
106
177
 
@@ -136,6 +207,23 @@ catalog-1 items were folded into catalog-2 **AND** storefront/transmit depends o
136
207
  NULL-`assetType` order lines alone are **not** damage (Prudential and Quad show that as normal).
137
208
 
138
209
  ## Change history
210
+ - 2026-09-02 — **Completed the reversal and root-caused the rest of the incident.** Recorded that
211
+ the first reversal was **incomplete** — it restored `Items`/`VendorItems`/`VendorItemCosts` and
212
+ re-pointed only `customerId = 2` `SalesOrderItems`, leaving **10 tables** folded (cust-1/cust-3
213
+ `SalesOrderItems` ~50k, `ItemFeatures`, `ItemImages`, `BundleItems`, `Personas_Items`, `Units`,
214
+ `ItemPrices`, `ItemMedia`, `AssortmentItems`, `InvoiceItems`). Added the **cover-every-table**
215
+ rule (the merge lists 38 reference points: 24 `itemId` + `Entitlements.saleItemId` + 3
216
+ `vendorItemId` + 10 `unitId`) and the **complete method**: restore every fold-affected row from
217
+ the pre-merge snapshot **by PRIMARY KEY** (part-number matching fails because `#ABA` twins carry
218
+ distinct part numbers), parents full-row / children FK-column-only, post-snapshot rows re-pointed
219
+ cust-2 on current data; verified 0 folded across 38 tables / 0 byte mismatches over 208,845 rows.
220
+ Recorded the folded **`BundleItems`** (434 rows / 110 bundles) as the storefront's ongoing
221
+ bad-order source (toga2-commerce relays the bundle item uuid with no catalog filter — fix is
222
+ data), the **`#ABA` straggler** manual VendorItem re-map (3845→3597 @945.17, 3884→3600 @1627.38,
223
+ scoped cust-2 pending PO lines + clear `dtSubmitted`), and that the **NetSuite→TOGa sync is not**
224
+ the source of new cust-2 orders (it pins `customerId = 3` + `locationId = 615286`, writes only the
225
+ Agilant SO) — correcting the earlier "stop the sync first" rule. Resolved the part-number
226
+ contradiction for the reversal (maps by PK, never by partNumber). (jcardinal)
139
227
  - 2026-09-01 — Flagged (did **not** resolve) the contradiction between this doc's 199
140
228
  shared-`UPPER(partNumber)` catalog-1→catalog-2 folds and the same-day prod verification that zero
141
229
  Compass part numbers span two catalogs; mirrored from the item-catalogs doc so it is visible from
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.739",
3
+ "version": "1.0.741",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",