toga-ai 1.0.360 → 1.0.362
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/2.0/apps/_underscore/INDEX.md +1 -1
- package/knowledge/2.0/apps/_underscore/features/surface-resolver.md +23 -1
- package/knowledge/2.0/apps/dbchanges2/features/surface-layer-schema.md +77 -2
- package/knowledge/sessions/2026-07-16-bdr-hscontactid-rate-limit-tcox.md +81 -0
- package/package.json +1 -1
|
@@ -21,7 +21,7 @@
|
|
|
21
21
|
| [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
|
|
22
22
|
| [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |
|
|
23
23
|
| [Recursive Item Fulfillments (upstream mirroring)](features/recursive-item-fulfillments.md) | In a multi-tier supply chain a sales order (SO) spawns a purchase order (PO) that becomes another SO downstream, and so on. | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillmentItem.php, _underscore/Model/Client/ItemFulfillmentItemUnit.php, _underscore/Model/Client/ItemFulfillmentPackage.php, _underscore/Model/Compass/AdvanceShippingNotice.php, dbchanges2/Core/2026-02-13 - 75601 - RecursiveItemFulfillmentCreation.sql, dbchanges2/Core/2026-06-04 - RecursiveItemFulfillmentPut.sql, dbchanges2/Client_Compass/2026-07-02a - FixSA133377TrackingSerialAndDuplicateIF.sql |
|
|
24
|
-
| [Surface Resolver (_Model_Core_Surface::resolve — replaces Page::meta)](features/surface-resolver.md) | The runtime for the platform-wide **Surface** UI presentation layer: 9 `_underscore` models plus a cached resolver, `_Model_Core_Surface::resolve(&$api, string | _underscore/Model/Core/Surface.php, _underscore/Model/Client/AclRecordScript.php, _underscore/Model/Core/RecordScript.php, dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql, dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql, dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql, dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql, dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql, _underscore/Model/Core/SurfaceElement.php, _underscore/Model/Core/Action.php, _underscore/Model/Core/Vocabulary.php, _underscore/Model/Core/VocabularyTerm.php, _underscore/Model/Core/Message.php, _underscore/Model/Client/SurfaceOverride.php, _underscore/Model/Client/MessageTranslation.php, _underscore/Model/Client/ThemeToken.php, _underscore/Model/Core/Page.php |
|
|
24
|
+
| [Surface Resolver (_Model_Core_Surface::resolve — replaces Page::meta)](features/surface-resolver.md) | The runtime for the platform-wide **Surface** UI presentation layer: 9 `_underscore` models plus a cached resolver, `_Model_Core_Surface::resolve(&$api, string | _underscore/Model/Core/Surface.php, _underscore/Model/Client/AclRecordScript.php, _underscore/Model/Core/RecordScript.php, dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql, dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql, dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql, dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql, dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql, dbchanges2/Client_Compass/2026-07-15f - SalesOrderRecordActionsRemoveDeadConfigRuleOverrides.sql, _underscore/Model/Core/SurfaceElement.php, _underscore/Model/Core/Action.php, _underscore/Model/Core/Vocabulary.php, _underscore/Model/Core/VocabularyTerm.php, _underscore/Model/Core/Message.php, _underscore/Model/Client/SurfaceOverride.php, _underscore/Model/Client/MessageTranslation.php, _underscore/Model/Client/ThemeToken.php, _underscore/Model/Core/Page.php |
|
|
25
25
|
| [Tracking-Number Bridge Migration (ASN / Item Fulfillment / Item Receipt)](features/tracking-number-bridges.md) | Shipment tracking numbers used to live as **scalar FK columns** (`trackingNumberId`, `returnTrackingNumberId`) directly on the lowest-level "unit"/"item" tables | api2/Component/Api/V2/V2.php, _underscore/Model/Client/AdvanceShippingNoticeItemUnit.php, _underscore/Model/Client/AdvanceShippingNoticeItemUnits/TrackingNumber.php, _underscore/Model/Client/ItemFulfillmentItemUnits/TrackingNumber.php, _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Prudential/AdvanceShippingNotice.php, _underscore/Model/Compass/AdvanceShippingNotice.php, _underscore/Trait/Netsuite/ItemFulfillment.php, api2/Component/Api/Cxml/Cxml.php, dbchanges2/Client/2026-06-10 - TrackingNumberBridges.sql, dbchanges2/Core/2026-06-10 - TrackingNumberBridges.sql, dbchanges2/Client_Prudential/2026-06-15 - ItemFulfillmentTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Quad/2026-06-18a - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Nychh/2026-06-19b - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Growrk/2026-07-13a - ItemFulfillmentItemReceiptTrackingNumberAclLogicGroups.sql, dbchanges2/Client_Nychh/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql, dbchanges2/Client_Quad/2026-06-19a - FixItemFulfillmentTableViewTrackingAndRoot.sql |
|
|
26
26
|
| [Units for Items for Purchase Orders — Data Structure](features/units-for-items-for-purchase-orders.md) | Describes how unit (serialized inventory) data is linked to sales-order and purchase-order line items behind the `units-for-items-for-purchase-orders` TableView | |
|
|
27
27
|
| [Refreshing a Local Dev Database from Beta (dev-sandbox)](workflows/local-db-refresh-from-beta.md) | How to reset a local 2.0 dev database from the **beta / dev-sandbox** environment: dump each schema (`Core`, `Client_<Id>`, `Logs_<Id>`, …) from the beta host, | api2/Config/, _underscore/Loader.php, _underscore/Model/Client/BundleTranslation.php, api2/Component/Api/V2/V2.php, toga25-supply/sync_compasscanada_schema.sql |
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-07-
|
|
9
|
+
updated: 2026-07-16
|
|
10
10
|
owners: [jcardinal, apeterson]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Model/Core/Surface.php
|
|
@@ -19,6 +19,7 @@ files:
|
|
|
19
19
|
- dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql
|
|
20
20
|
- dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql
|
|
21
21
|
- dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql
|
|
22
|
+
- dbchanges2/Client_Compass/2026-07-15f - SalesOrderRecordActionsRemoveDeadConfigRuleOverrides.sql
|
|
22
23
|
- _underscore/Model/Core/SurfaceElement.php
|
|
23
24
|
- _underscore/Model/Core/Action.php
|
|
24
25
|
- _underscore/Model/Core/Vocabulary.php
|
|
@@ -229,6 +230,19 @@ Core record grants + their logic-group expressions all evaluate `all`/`"1"`. The
|
|
|
229
230
|
|
|
230
231
|
## Gotchas
|
|
231
232
|
|
|
233
|
+
- **A `SurfaceOverrides` row with `attribute='CONFIG'` carrying a RULE value is silently DEAD —
|
|
234
|
+
encode rules as `VISIBILITY_RULE`/`ENABLED_RULE`, never as `CONFIG`.** `_castOverride` routes the
|
|
235
|
+
`CONFIG` attribute into `$config`, **NOT** `$visibilityRule`/`$enabledRule`; a `CONFIG` row whose
|
|
236
|
+
`value` is a bare rule name (`'visibilityRule'`/`'enabledRule'`) is `error_log`'d and returns `null`,
|
|
237
|
+
so it never applies. Compounding it, `overridesByElementAndAttr` is keyed by **(element, attribute)**,
|
|
238
|
+
so multiple `CONFIG` rows for the **same** element collide and only one survives. `Client_Compass`
|
|
239
|
+
`2026-06-25d` encoded "Manager" visibility rules as `CONFIG` rows — an **entire batch** of client
|
|
240
|
+
overrides that therefore **never applied (silently)**. The modern, correct encoding uses the direct
|
|
241
|
+
`VISIBILITY_RULE`/`ENABLED_RULE` attributes (see the per-client rule-override section above). Cleanup
|
|
242
|
+
migration `Client_Compass/2026-07-15f` DELETEs exactly the dead rows (discriminator
|
|
243
|
+
`attribute='CONFIG' AND value IN ('visibilityRule','enabledRule')`) — **Compass only**;
|
|
244
|
+
`Client_CompassCanada` had no `25d` equivalent and was clean. Legitimate `CONFIG` rows (e.g. the
|
|
245
|
+
approvals-filter override in `15a`) must be **preserved**.
|
|
232
246
|
- **A Core RecordScript needs `Client.AclRecordScripts` rows per role to be invokable — record-level
|
|
233
247
|
`AclRecordPermissions` is necessary but NOT sufficient.** V2 authorizes a scripted-API call **not**
|
|
234
248
|
via `AclRecordPermissions` but via the Client-DB `AclRecordScripts` table
|
|
@@ -298,6 +312,14 @@ Core record grants + their logic-group expressions all evaluate `all`/`"1"`. The
|
|
|
298
312
|
match Compass, a follow-up migration aligning both `meta` and `meta-group` to roles 1,3,4 is needed.
|
|
299
313
|
|
|
300
314
|
## Change history
|
|
315
|
+
- 2026-07-16 — Recorded a resolver gotcha: a `SurfaceOverrides` row with `attribute='CONFIG'` carrying
|
|
316
|
+
a rule value is silently dead — `_castOverride` routes CONFIG→`$config`, not
|
|
317
|
+
`$visibilityRule`/`$enabledRule` (logs + returns null), and `overridesByElementAndAttr` keyed by
|
|
318
|
+
(element, attribute) collides multiple CONFIG rows for one element. `Client_Compass/2026-06-25d`
|
|
319
|
+
encoded Manager visibility rules as CONFIG → never applied; cleanup migration
|
|
320
|
+
`Client_Compass/2026-07-15f` deletes them (`attribute='CONFIG' AND value IN
|
|
321
|
+
('visibilityRule','enabledRule')`), CompassCanada had no 25d equivalent (clean). Rules must be
|
|
322
|
+
encoded only as `VISIBILITY_RULE`/`ENABLED_RULE`. (apeterson)
|
|
301
323
|
- 2026-07-13 — Made the Tier-1 gating RULES per-client overridable: added
|
|
302
324
|
`ATTRIBUTE__VISIBILITY_RULE`/`ATTRIBUTE__ENABLED_RULE` to `_Model_Client_SurfaceOverride`;
|
|
303
325
|
`_resolveElement` seeds locals from the Core element's rules then applies overrides;
|
|
@@ -6,7 +6,7 @@ project: Database Changes
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-07-
|
|
9
|
+
updated: 2026-07-16
|
|
10
10
|
owners: [jcardinal, apeterson]
|
|
11
11
|
files:
|
|
12
12
|
- dbchanges2/Client/2026-06-25a - SurfaceClientTables.sql
|
|
@@ -54,7 +54,7 @@ per-tenant deltas live in **Client** (mirrors `TableViews`/ACL/`ItemTranslations
|
|
|
54
54
|
| Table | DB | Purpose |
|
|
55
55
|
|---|---|---|
|
|
56
56
|
| `Surfaces` | Core | A configurable UI region (`type` ENUM TABLE/SECTION/BUTTON_BAR/ROW_ACTIONS/TAB_STRIP/FILTER_SET/MODAL/DRAWER) |
|
|
57
|
-
| `SurfaceElements` | Core | Ordered items: ~15 typed attr columns + one JSON `config` long-tail + `visibilityRule`/`enabledRule` JSON |
|
|
57
|
+
| `SurfaceElements` | Core | Ordered items: ~15 typed attr columns (two orthogonal axes `valueType` + `inputType`, plus edit-mode columns — see below) + one JSON `config` long-tail + `visibilityRule`/`enabledRule` JSON |
|
|
58
58
|
| `Actions` | Core | Behavior action catalog (FE registry keys + JSON payload) |
|
|
59
59
|
| `Vocabularies` | Core | A value-set (status/stage/priority/type/channel) per record |
|
|
60
60
|
| `VocabularyTerms` | Core | value → token/label/order |
|
|
@@ -68,6 +68,52 @@ per-tenant deltas live in **Client** (mirrors `TableViews`/ACL/`ItemTranslations
|
|
|
68
68
|
`value varchar(255)` + `c_longValue mediumtext`. Precedence **base < client < persona < role**
|
|
69
69
|
with language as an orthogonal overlay; the resolver applies it most-specific-last in one pass.
|
|
70
70
|
|
|
71
|
+
## SurfaceElements — two orthogonal axes + edit-mode columns (one tree serves view + edit)
|
|
72
|
+
|
|
73
|
+
A record modal is modeled as **ONE** set of `<record>-record-*` SECTION surfaces (a header +
|
|
74
|
+
section surfaces) that serves **both view and edit/create modes** — not two parallel view/edit
|
|
75
|
+
trees ("Option B"). This keeps one source of truth per record. Each `SurfaceElement` carries **two
|
|
76
|
+
orthogonal, independently-set axes** — neither derives from the other:
|
|
77
|
+
|
|
78
|
+
- **`valueType`** — data/display formatting (`STRING`/`CURRENCY`/`BOOLEAN`/`STATUS`/`DATE`/…), i.e.
|
|
79
|
+
how the value renders in **view** mode (mirrors `RecordFields.type`). `MULTISELECT` is deliberately
|
|
80
|
+
**not** in this ENUM — it is an input widget, not a display type.
|
|
81
|
+
- **`inputType`** — the **edit** widget (`TEXT`/`TEXTAREA`/`ADVANCED_SELECT`/`ADVANCED_MULTI_SELECT`/
|
|
82
|
+
`MULTISELECT`/`TOGGLE`/`CHECKBOX`/`RADIOBUTTON`/`DATE`/…); `NULL` for display-only elements.
|
|
83
|
+
|
|
84
|
+
View vs input **value bindings** live in `config.valueKey` / `config.editValueKey` — `editValueKey`
|
|
85
|
+
is present only when the edit binding differs from the view key. Section grouping (detail vs
|
|
86
|
+
`moreInfo`) is a `config.group` marker.
|
|
87
|
+
|
|
88
|
+
**Edit-mode columns** added to `SurfaceElements` (Core DDL, `Core/2026-06-25a`), so the schema can
|
|
89
|
+
express edit widgets, validation, required-ness, limits, searchability, and boolean captions:
|
|
90
|
+
|
|
91
|
+
| Column | Purpose |
|
|
92
|
+
|---|---|
|
|
93
|
+
| `inputType` ENUM | edit widget (see axis above) |
|
|
94
|
+
| `validationType` varchar(32) | name of the FE validation function invoked on input (e.g. `currency`) — **renamed from the earlier `hasTypeCheck`** |
|
|
95
|
+
| `showRequiredIndicator` tinyint | show the required-field asterisk |
|
|
96
|
+
| `isSearchable` tinyint | a select-type input is searchable |
|
|
97
|
+
| `characterLimit` smallint | max input length for text/textArea inputs |
|
|
98
|
+
| `trueMessageId` / `falseMessageId` int FK→`Messages` | **ONE unified boolean-label pair** serving both view display **and** toggle caption — replaces the earlier `truthyValue`/`falsyValue` **and** `activeLabel`/`inactiveLabel` |
|
|
99
|
+
|
|
100
|
+
Boolean labels resolve through the resolver's existing `Messages` join; shared
|
|
101
|
+
`common.yes`/`common.no`/`common.active`/`common.inactive` Messages are seeded **once** (in
|
|
102
|
+
`Core/2026-06-29a`) and reused. Applied to the **Item** record modal (`Core/2026-06-29a`) and the
|
|
103
|
+
**VendorItem** record modal (`Core/2026-06-29d`), each folded into that record's existing per-record
|
|
104
|
+
LIST seed file.
|
|
105
|
+
|
|
106
|
+
## Core surface seeds are a TRUE NEUTRAL DEFAULT (action buttons seed OFF)
|
|
107
|
+
|
|
108
|
+
A value belongs in a Core surface seed only if it is **(a) universal AND (b) safe-if-silently-
|
|
109
|
+
inherited**. An edit affordance visible to every role by default is **not** safe-if-inherited.
|
|
110
|
+
Therefore, when seeding a record-modal header **Edit button** (or a similar action button) into Core,
|
|
111
|
+
set **`isVisible=0, isEnabled=0`** — **even when** the `toga25-supply` `FIELDS/DEFAULT/*` bundle ships
|
|
112
|
+
it `isVisible: true`. Both `Core/2026-06-29a` (Item) and `Core/2026-06-29d` (VendorItem) seed the
|
|
113
|
+
`editItem` header element `0,0`, the latter noting "the DEFAULT bundle ships it on, but Core forces it
|
|
114
|
+
off." A client opts a role in via a `SurfaceOverride` (`IS_VISIBLE=1`), scoped by role/persona. This is
|
|
115
|
+
the seed-authoring corollary of the Core neutral-default re-baseline recorded in the resolver doc.
|
|
116
|
+
|
|
71
117
|
## ThemeTokens — per-tenant, physically isolated in each client DB
|
|
72
118
|
|
|
73
119
|
`ThemeTokens` lives **inside each tenant's own database** (`Client_Compass.ThemeTokens`,
|
|
@@ -142,6 +188,23 @@ SurfaceElements`). Core migrations run first; Client after. The session built/se
|
|
|
142
188
|
- `Client/2026-06-25b - SurfaceClientSeed.sql` — `ThemeTokens` + `MessageTranslations(en)`.
|
|
143
189
|
- `Client/2026-06-25c - SurfaceClientAcl.sql` — full ACL chain for the 3 CLIENT-aclDatabase records (recordId refs updated to the 333-block renumber).
|
|
144
190
|
|
|
191
|
+
## Migration maintenance while local-only (fold-back)
|
|
192
|
+
|
|
193
|
+
The entire Surface feature is **LOCAL-ONLY** right now — beta and prod are both clean per
|
|
194
|
+
`toga25-supply/db-migrations/PLAYBOOK.md`; no surface migration has applied to a shared env yet.
|
|
195
|
+
|
|
196
|
+
**While local-only**, record-modal seeds and session end-state fold **INTO the existing per-record
|
|
197
|
+
dated seed file** — never into a new dated patch file — so there is exactly **one canonical
|
|
198
|
+
final-state file per record**. Dated patch files drift and obscure the end state. This session folded
|
|
199
|
+
the Item record modal into `Core/2026-06-29a` and the VendorItem record modal into `Core/2026-06-29d`;
|
|
200
|
+
retired the `07-15a`/`07-15b` patches back into `Core/2026-06-25a`/`Core/2026-06-25c`; and deleted a
|
|
201
|
+
stray `Core/2026-07-16a` created in error.
|
|
202
|
+
|
|
203
|
+
This is the **deliberate exception** to the repo's "never edit an already-run migration — add a new
|
|
204
|
+
dated file instead" rule (see [architecture](../architecture.md)), and is safe **only** because
|
|
205
|
+
nothing has run on a shared env. Once these seeds promote to beta/prod, the normal add-a-new-dated-file
|
|
206
|
+
rule resumes.
|
|
207
|
+
|
|
145
208
|
## Gotchas
|
|
146
209
|
|
|
147
210
|
- **Cross-DB FKs (Core↔Client) are SOFT** — the ORM declares `FIELDOPT_FOREIGNKEY_MODEL` but **no
|
|
@@ -190,6 +253,18 @@ SurfaceElements`). Core migrations run first; Client after. The session built/se
|
|
|
190
253
|
because `TOOLTIP_MESSAGE` is numeric-only (cosmetic, deferred).
|
|
191
254
|
|
|
192
255
|
## Change history
|
|
256
|
+
- 2026-07-16 — Documented `SurfaceElements`' two orthogonal axes (`valueType` display vs `inputType`
|
|
257
|
+
edit widget — neither derived from the other) and the edit-mode columns added in `Core/2026-06-25a`
|
|
258
|
+
(`inputType`, `validationType` [renamed from `hasTypeCheck`], `showRequiredIndicator`, `isSearchable`,
|
|
259
|
+
`characterLimit`, a unified `trueMessageId`/`falseMessageId` boolean-label pair replacing
|
|
260
|
+
truthyValue/falsyValue + activeLabel/inactiveLabel; `MULTISELECT` removed from the `valueType` ENUM).
|
|
261
|
+
Recorded the record-modal "Option B" modeling (one shared `<record>-record-*` SECTION tree serves
|
|
262
|
+
view + edit; `config.valueKey`/`editValueKey`, `config.group`), applied to the Item (`Core/2026-06-29a`)
|
|
263
|
+
and VendorItem (`Core/2026-06-29d`) modals. Recorded the Core true-neutral-default seeding rule
|
|
264
|
+
(action/Edit buttons seed `isVisible=0/isEnabled=0` even when the DEFAULT bundle ships them on; clients
|
|
265
|
+
opt in via a SurfaceOverride) and the local-only fold-back migration practice (fold into the per-record
|
|
266
|
+
file, don't stack dated patches — the deliberate exception to the never-edit rule while nothing has run
|
|
267
|
+
on a shared env). (apeterson)
|
|
193
268
|
- 2026-07-01 — Documented `ThemeTokens` in detail (new section): it lives **inside each tenant's
|
|
194
269
|
own DB** (physical isolation, no confirmed Core default table), is a flat `slug→value` map with
|
|
195
270
|
the `category` ENUM(`COLOR`/`BACKGROUND`/`SPACING`/`RADIUS`/`TYPOGRAPHY`/`ICON`), and read by
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: session
|
|
3
|
+
slug: bdr-hscontactid-rate-limit
|
|
4
|
+
title: BDR ?hsContactId= enumeration rate-limit mitigation (security-reviewed SHIP)
|
|
5
|
+
author: tcox
|
|
6
|
+
repos: [bdr, ai-bdr]
|
|
7
|
+
framework: "2.0"
|
|
8
|
+
client: shared
|
|
9
|
+
status: active
|
|
10
|
+
created: 2026-07-16
|
|
11
|
+
updated: 2026-07-16
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Session: bdr-hscontactid-rate-limit
|
|
15
|
+
**Date:** 2026-07-16
|
|
16
|
+
**Project/Repo:** bdr (2.0) — local `C:\WWW\BDR`, GitHub `agilantsolutions/BDR`, branch `BDR-Phase-2`
|
|
17
|
+
**Task:** Build an interim app-layer rate-limit mitigation for the public funnel's `?hsContactId=` enumeration risk (PLAN §10, HIGH), driven to a security-reviewed SHIP; along the way, wire local dev env and diagnose why a live test call never fired.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## What WORKED
|
|
22
|
+
- **Interim rate-limit mitigation BUILT and SHIP-verified (two CSO review rounds).** New `src/proxy.ts` (Next 16 proxy), `src/lib/rateLimit.ts` (token bucket), `src/lib/clientIp.ts` (trusted IP resolution). Policy: entry pages with `?hsContactId=` **soft-degrade** over limit (strip id, `NextResponse.rewrite` to render anonymously — no PII, no Toga upsert; hsCampaignId preserved); `/api/contact`, `/api/call-now`, `/api/call-later` return a `429` `{isSuccess,status,error,messages,data}` envelope with `Retry-After`. Evidence: tsc clean, eslint clean, **104/104 vitest**, `next build` compiles it (logs `ƒ Proxy (Middleware)`).
|
|
23
|
+
- **C1 (Critical) FIXED — trusted client IP.** `resolveClientIp()` parses `X-Forwarded-For` **right-to-left** at `parts.length - 1 - RATE_LIMIT_TRUSTED_PROXY_HOPS` (default 0 => rightmost = CloudFront-appended viewer), so a forged/leftmost XFF can't shift the selected index. `x-real-ip` no longer trusted. `CloudFront-Viewer-Address` trusted ONLY when `RATE_LIMIT_TRUST_CF_VIEWER_HEADER=true` (env, default OFF). `normalizeIp` strips port, `[ipv6]:port`, `::ffff:` mapping. Verified by `test/clientIp.test.ts` incl. a 20-entry spoof-flood that still resolves to the trusted tail.
|
|
24
|
+
- **H1 (High) FIXED — limiter map hard-bounded.** `MAX_TRACKED_KEYS=10_000`; `check()` re-inserts existing keys (Map order = LRU); new-key insert at cap runs `makeRoom()` = throttled `sweepIdle` (≤ once/window) + O(1) LRU eviction loop. Verified: 50k unique keys → size ≤ 10k; a key touched every 50 iters survives a 40k flood and stays drained/denied.
|
|
25
|
+
- **M FIXED — `?hsCampaignId=` validated.** `page.tsx` now `isTogaUuid`-validates it before it flows into `togaLeadSink.upsertLead` as `togaCampaignUuid`; malformed drops to `undefined`.
|
|
26
|
+
- **CSO re-review verdict: SHIP** (SAFE TO SHIP) — all of C1/H1/M genuinely closed, no new defects.
|
|
27
|
+
- **Local dev env wired.** Created `C:\WWW\BDR\.env.local` (gitignored) by copying the 5 server-side vars (`HUBSPOT_ACCESS_TOKEN` + 4 `TOGA_*`) from `C:\WWW\info\.env.local` (PRODUCTION values, base `https://api.togahub.com/v2`). GA left unset locally (analytics.ts no-ops). Verified auth + contact writes hit prod.
|
|
28
|
+
- **Diagnosed the BDR call flow end to end.** A `200 POST /api/call-now` in the browser is success from BDR's side; Toga calls are server-side (never in the browser network tab). BDR only sets `dtNextContactRequested`/`isOkayToCall`/call-type on the contact; the worker2 dialer cron places the actual Vapi call. Confirmed the test contact `88800869-acc7-632b-2520-bbd601224ff5` got `dtNextContactRequested` written in prod.
|
|
29
|
+
- **Two `/capture` pushes to team KB (PUSHED).** `web-funnel-app.md` (rate-limit mitigation, server-side call seam, phone-less bug, `/api/contact` scrape vector, Next 16 proxy gotcha, dev-setup), `worker2/vapi-webhook-handler.md` (new "Outbound dialer eligibility" section), `ai-bdr/call-orchestration.md` (cross-ref). Second capture superseded the mid-state (test count → 104, clientIp.ts, SHIP verdict, 3 new security gotchas).
|
|
30
|
+
|
|
31
|
+
## What did NOT work — DO NOT RETRY THESE
|
|
32
|
+
- **Live test call never fired despite a correct 200 submit — TWO production-data blockers, not a code bug.**
|
|
33
|
+
1. The test contact's `Contacts.primaryContactPhoneNumberId` is **null**, so the dialer's eligibility SQL (`getEligibleContactsForVapi`, INNER JOIN `ContactPhoneNumbers ON Contacts.primaryContactPhoneNumberId`) can't see it. Root cause: the contact was created WITHOUT a phone (HubSpot record has none); the phone arrives via the later Call Now `PUT`, and **api2 creates the ContactPhoneNumber row but does not set the primary pointer on update**. Re-PUTting does NOT fix it (verified: primary stayed null while a phone row `(630) 803-7765` existed). This is a real product bug — any phone-less HubSpot contact can never be called; the funnel still shows success.
|
|
34
|
+
2. The active campaign `26.05 - AI BDR - Ryan Nitti` (`15d0c02f-501e-e081-2ead-61b78c854223`) reads `campaignAssistantId = null`; the dialer INNER JOINs `CampaignAssistants`, so a null-assistant campaign is never dialed. Could not confirm whether truly null or hidden from the API credential.
|
|
35
|
+
- **`node check-contact.mjs` with a `with=ContactCallType,PrimaryContactPhoneNumber,CampaignsContacts` clause → HTTP 404.** The `with=` relation names were wrong for this endpoint; the plain `join=…&where=(…)` query returns 200. Use the join form, not `with=`.
|
|
36
|
+
- **Toga `/contact-attempts` and `/campaigns-contacts` (some join variants) → HTTP 403/500 for this API credential.** Can't read ContactAttempts or all campaign-contact links directly; infer from contact + campaign state instead.
|
|
37
|
+
- **Running `npx tsc`/`npx vitest`/`npx eslint`/`npx next build` without `cd /c/WWW/BDR &&` first.** The Bash tool cwd resets to `c:\WWW`, so tsc resolves the wrong binary and vitest runs sibling repos (saw 189 failing toga-blox-npm tests). ALWAYS prefix `cd /c/WWW/BDR &&`.
|
|
38
|
+
- **`middleware.ts` on Next 16 is DEPRECATED.** Next 16.2.3 warns "use proxy instead". Correct convention is `src/proxy.ts` exporting a `proxy` function + `export const config: ProxyConfig`; import from `next/server`. (Confirmed from `node_modules/next/dist/lib/constants.js` + build source; did not ship a middleware.ts.)
|
|
39
|
+
- **`info/.env.beta` is a DEAD file** — empty URLs, obsolete var names (`TOGA_CLIENT_SECRET` vs the current `TOGA_CLIENT_API_UUID`/`SECRET` pair). Production (`info/.env.local`) is the only working credential set `info` has. Do not source dev env from `.env.beta`.
|
|
40
|
+
|
|
41
|
+
## Not tried yet (candidates for next session)
|
|
42
|
+
- **Commit the working-tree changes** on `BDR-Phase-2` (developer has not asked yet). Suggested split: `feat: rate-limit hsContactId enumeration surfaces` (proxy.ts, rateLimit.ts, clientIp.ts, tests, PLAN §10) + `fix: validate hsCampaignId before toga upsert` (page.tsx).
|
|
43
|
+
- **Unblock the live call test** (needs AI-BDR owner or a production write): set the test contact's `primaryContactPhoneNumberId`, and confirm/fix the campaign's `campaignAssistantId`.
|
|
44
|
+
- **Decide the phone-less-contact bug fix**: api2-side (set primary on phone-row create during update) vs. a BDR workaround (PUT linking the existing phone-row uuid as primary).
|
|
45
|
+
- **Non-gating security follow-ups (PLAN §10 / Phase 8):** lock the Amplify origin so nothing bypasses CloudFront; confirm the distribution injects+strips `CloudFront-Viewer-Address` before enabling `RATE_LIMIT_TRUST_CF_VIEWER_HEADER`; harden `readJsonObject` (body buffered before size check; measures UTF-16 length not bytes); re-verify no un-normalized path variant reaches a handler off the proxy `matcher`.
|
|
46
|
+
- **Phase 8 authoritative controls:** edge WAF per-IP rate rule (supersedes the interim in-app limiter); opaque/signed CRM link tokens instead of raw `hsContactId` (raise with HubSpot/CRM owner).
|
|
47
|
+
- Remaining PLAN §9 items (unchanged from prior sessions): §9.3 agent choice, §9.4 Landing service-picker scope, §9.5 campaign catalog, §9.6 togatech integration mechanism (gates deploy), §9.7 Tweaks panel in prod, §9.8 first-campaign copy, §9.12 Share button (still a mock), §9.13 connect-time copy. Phase 7 polish (pixel/motion/a11y; GA4 done).
|
|
48
|
+
|
|
49
|
+
## Current file state
|
|
50
|
+
| File | Status | Notes |
|
|
51
|
+
|------|--------|-------|
|
|
52
|
+
| `src/proxy.ts` | ADDED, uncommitted | Next 16 proxy; per-IP limits, soft-degrade entry + 429 API routes; env `RATE_LIMIT_TRUSTED_PROXY_HOPS`, `RATE_LIMIT_TRUST_CF_VIEWER_HEADER` |
|
|
53
|
+
| `src/lib/rateLimit.ts` | ADDED, uncommitted | Pure token-bucket, clock-injected, hard-bounded 10k-key LRU map |
|
|
54
|
+
| `src/lib/clientIp.ts` | ADDED, uncommitted | Pure trusted-IP resolution (right-to-left XFF; CF header opt-in; normalizeIp) |
|
|
55
|
+
| `src/app/page.tsx` | MODIFIED, uncommitted | `isTogaUuid`-validate `hsCampaignId` before Toga upsert |
|
|
56
|
+
| `test/rateLimit.test.ts` | ADDED, uncommitted | limiter unit tests incl. hard-bound-under-flood + LRU survival |
|
|
57
|
+
| `test/clientIp.test.ts` | ADDED, uncommitted | IP-resolution tests incl. spoofed-leftmost-XFF bypass attempt |
|
|
58
|
+
| `PLAN.md` | MODIFIED, uncommitted | §10 mitigation + SHIP + residuals; §11 Phase-8 checklist items |
|
|
59
|
+
| `.env.local` | ADDED, uncommitted (gitignored) | 5 server-side vars from info prod; NEVER committed |
|
|
60
|
+
| Repo otherwise | clean at `bb40b15` on `BDR-Phase-2` | all above are working-tree only; no commit/PR yet |
|
|
61
|
+
| Team KB (`toga-tech`) | PUSHED | web-funnel-app.md, worker2/vapi-webhook-handler.md, ai-bdr/call-orchestration.md updated via 2× /capture |
|
|
62
|
+
|
|
63
|
+
## Decisions made
|
|
64
|
+
- **App-layer rate limiting now, as defense-in-depth** — not the authoritative fix. Per-instance memory on Amplify SSR is honestly documented; the WAF per-IP rule + opaque/signed CRM tokens remain the real controls (Phase 8). Chosen because it's the only layer buildable pre-deploy.
|
|
65
|
+
- **Entry page soft-degrades; API routes hard-429.** Stripping `hsContactId` keeps a real (shared-NAT) user in the funnel minus prefill while denying a scraper any PII/upsert; the call/contact routes trigger real actions or leak full PII, so a hard stop is correct.
|
|
66
|
+
- **Trust IP from the rightmost XFF, not leftmost** (C1 fix) — the leftmost is client-prependable behind CloudFront; the rightmost is CloudFront-appended. `CloudFront-Viewer-Address` made **opt-in (default off)** so an unverified distribution can't be bypassed via a forged copy — safe-by-default, ops enables after verifying.
|
|
67
|
+
- **Hard-bound the limiter map with LRU eviction** (H1 fix) — a refill-only sweep is itself a unique-key-flood DoS; a hard ceiling + O(1) LRU + throttled reclaim guarantees bounded memory and O(1) amortized inserts.
|
|
68
|
+
- **Used Next 16 `proxy.ts`, not `middleware.ts`** — the latter is deprecated in 16.
|
|
69
|
+
- **Deferred `readJsonObject` + matcher-normalization fixes** as non-gating follow-ups — pre-existing, bounded by the Lambda payload cap + the call limiter; keeping the change focused and low-risk was the better call than widening scope into shared validated code.
|
|
70
|
+
- **Phone-less-contact bug recorded as a gotcha, not its own doc** — has an api2-side root cause but surfaces on the funnel + worker2 dialer docs per KB conventions.
|
|
71
|
+
|
|
72
|
+
## Blockers
|
|
73
|
+
- **Live call test blocked on production data** (owner: AI-BDR owner, e.g. Rohan/Akash): test contact `primaryContactPhoneNumberId` is null, and campaign `campaignAssistantId` appears null. Both need a production write or owner action; developer's standing rule is read-only DB access for Claude.
|
|
74
|
+
- **Committing to `BDR-Phase-2`** awaits developer go-ahead (standing "ask before git/remote" policy).
|
|
75
|
+
- Pre-existing PLAN blockers unchanged: §9.6 togatech integration (gates Phase 8 deploy), §9.8 first-campaign copy from leadership.
|
|
76
|
+
|
|
77
|
+
## Exact next step
|
|
78
|
+
> Decide whether to (a) commit the working-tree changes on `BDR-Phase-2` as the two commits noted above, or (b) tackle the phone-less-contact bug fix — whichever the developer wants first. The security work itself is complete (SHIP-verified) and fully captured; nothing there is pending.
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
_Saved by /session-save on 2026-07-16_
|
package/package.json
CHANGED