toga-ai 1.0.348 → 1.0.350

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -6,7 +6,7 @@ project: _Underscore
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-07-08
9
+ updated: 2026-07-15
10
10
  owners: ["dfranks"]
11
11
  files:
12
12
  - _underscore/Error.php
@@ -113,9 +113,30 @@ not extend the inline-persist branch.
113
113
  Logs DB, two dbchanges2 `Logs/` migrations that both `CREATE TABLE Issues`/`Events` will
114
114
  collide ("table already exists"). Only **one** migration may create the shared tables;
115
115
  later migrations only `ALTER`.
116
+ - **Cross-tenant secret exposure when captured context lands in the shared Core Logs DB.**
117
+ Moving the Issue/Event sink from a per-client Logs DB to the *shared* Core Logs DB widens
118
+ the blast radius of anything captured in `context`. A pre-migration `print_r($GLOBALS, true)`
119
+ dump (session tokens, auth headers, credentials from superglobals) that was acceptable while
120
+ per-tenant becomes a cross-tenant leak once every tenant can read the shared table.
121
+ `_underscore/Error.php::redactSensitiveContext()` recursively redacts superglobal keys
122
+ matching sensitive fragments (`password/secret/token/authorization/auth/apikey/credential/`
123
+ `cookie/session/jwt/private`) **before** persisting. Rule: sanitize captured context at the
124
+ point you move a log sink from per-client to shared.
125
+ - **Strict-mysqli varchar overflow silently drops the Issue/Event row.** `_Error::initialize()`
126
+ sets `mysqli_report(MYSQLI_REPORT_STRICT)`, so assigning an over-255-char value to a
127
+ `varchar(255)` column (`errorMessage`, `subject`) **throws** on `->save()`. Because the write
128
+ happens inside the error handler's `catch(Throwable)`, the throw is swallowed and the row is
129
+ silently dropped — the error pipeline fails precisely when a long/detailed message matters
130
+ most. Truncate any raw exception message / user string to the column width before `save()`
131
+ on any `_underscore` model write under strict mysqli.
116
132
 
117
133
  ## Change history
118
134
 
135
+ - 2026-07-15 — Added two framework gotchas from aligning the `_underscore` side of the
136
+ Issue/Event pipeline to the shared Core Logs DB: (1) cross-tenant secret exposure when
137
+ captured context lands in the shared DB, mitigated by `redactSensitiveContext()` in
138
+ Error.php; (2) strict-mysqli varchar overflow throwing inside the handler's `catch(Throwable)`
139
+ silently drops the Issue/Event row — truncate to column width before `save()`. (dfranks)
119
140
  - 2026-07-08 — Corrected KB drift: documented the approved 2026-05-21 error-monitoring
120
141
  architecture (handlers POST to a centralized `/errors` receiver; a worker2 cron builds
121
142
  issueHash, upserts Issues + inserts Events into the Core Logs DB, aggregates/escalates,
@@ -9,6 +9,7 @@
9
9
  | [Nested-relationship writes & child matching (link vs. create)](features/nested-relationship-writes.md) | When a 2.0 API write payload (`POST`/`PUT`) contains a **nested related object** (e.g. | api2/Component/Api/V2/V2.php |
10
10
  | [POST + JSON-body args for scripted APIs](features/scripted-api-post-body-args.md) | The V2 engine can run a Record Script (scripted API) for a **POST** request, and a scripted API can receive its arguments from the **JSON request body** instead | api2/Component/Api/V2/V2.php |
11
11
  | [Surface action-state via the surface=<slug> request option (M2M-safe)](features/surface-meta-option.md) | An opt-in V2 engine request option, `surface=<slug>`, that attaches per-record UI action state (`isVisible`/`isEnabled`) to a GET response **under `meta.surface | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Surface.php |
12
+ | [TableView row-filtering via apiWhereClause (options.where grammar, end to end)](features/tableview-apiwhereclause-row-filtering.md) | `TableViews.apiWhereClause` (TEXT, nullable) is the sanctioned, code-free way to restrict or exclude rows from a 2.0 table view. | api2/Component/Api/V2/V2.php, _underscore/Model/Client/TableView.php, toga2-supply/src/api/toga.ts |
12
13
  | [Tickets API (/v2/tickets)](features/tickets-api.md) | The generic ticket endpoint of the 2.0 REST API. | Component/Api/V2/V2.php |
13
14
  | [AWS CodePipeline Deployment via CodeConnections (GitHub → Elastic Beanstalk)](workflows/codepipeline-codeconnections-deploy.md) | 2.0 apps (`api2`, `_underscore`) are deployed through **AWS CodePipeline**. | |
14
15
  | [New Environment Configuration & Provisioning (api2)](workflows/environment-configuration-and-provisioning.md) | What it takes for a 2.0 API environment (e.g. | api2/Config/<environment>.ini, api2/Controller/Index.php, dbchanges2/Core/2026-06-16a - DatabaseHosts for new QA QC stage demo environments.sql, _underscore/Route.php |
@@ -0,0 +1,115 @@
1
+ ---
2
+ title: TableView row-filtering via apiWhereClause (options.where grammar, end to end)
3
+ framework: "2.0"
4
+ repo: api2
5
+ project: API
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-07-15
10
+ owners: ["bala"]
11
+ files:
12
+ - api2/Component/Api/V2/V2.php
13
+ - _underscore/Model/Client/TableView.php
14
+ - toga2-supply/src/api/toga.ts
15
+ related:
16
+ - ../../../clients/compass-usa/features/item-fulfillment-tracking-tableview.md
17
+ ---
18
+
19
+ ## Summary
20
+
21
+ `TableViews.apiWhereClause` (TEXT, nullable) is the sanctioned, code-free way to restrict or
22
+ exclude rows from a 2.0 table view. The clause is stored on the view, emitted verbatim in the
23
+ view meta by `_underscore`, pushed into the data request's `options.where.and` by the frontend,
24
+ and finally parsed into SQL by api2's `parseOptionsWhere`. Because api2 parses it for *every*
25
+ data request, the same `field:operator:value` grammar governs any `options.where` filter — table
26
+ views are just one producer of it. Use this to add a "restrict/exclude these rows" rule to a
27
+ view with a migration and no application code.
28
+
29
+ ## Key files / entry points
30
+
31
+ - **Emission** — `_underscore/Model/Client/TableView.php`, the `meta()` scripted API (~line 51):
32
+ emits `TableViews.apiWhereClause` verbatim in the view meta, and builds `join`/`ojoin` entries
33
+ from `TableViewJoins` (~line 631-641 of the consumer) so every table joined in the view is
34
+ present on the data request.
35
+ - **Consumption** — `toga2-supply/src/api/toga.ts` (~line 943-951): pushes the `apiWhereClause`
36
+ literal string into `options.where.and`, and (~line 631-641) builds `options.join` / `ojoin`
37
+ from the view's `TableViewJoins`.
38
+ - **Parsing** — `api2/Component/Api/V2/V2.php::parseOptionsWhere` (~line 7574-7773): turns the
39
+ clause string into SQL.
40
+
41
+ ## How it works
42
+
43
+ Clause syntax: `field:operator:value`. Conditions are comma-separated, grouped with parens, and
44
+ joined with `,AND,` / `,OR,` between conditions. The parser tracks parenthesis depth and only
45
+ splits conditions on commas at the **top** depth.
46
+
47
+ Operator tokens map to SQL as:
48
+
49
+ | token | SQL |
50
+ |-------|-----|
51
+ | `eq` | `=` |
52
+ | `ne` | `<>` |
53
+ | `gt` | `>` |
54
+ | `ge` | `>=` |
55
+ | `lt` | `<` |
56
+ | `le` | `<=` |
57
+ | `like` | `LIKE` |
58
+ | `contains` / `starts` / `ends` | wildcarded `LIKE` |
59
+ | `excludes` | `NOT LIKE` |
60
+ | `in` | `IN` |
61
+ | `notin` | `NOT IN` |
62
+ | `between` | `BETWEEN` |
63
+ | `not` | `NOT` |
64
+
65
+ The field slot accepts a **raw SQL expression**, not just a plain column — e.g.
66
+ `YEAR(SalesOrders.dateOrder)` or `IFNULL(Items.assetTypeId,0)`. Real-world example (the FEE
67
+ exclusion on the Compass fulfillment view): `(IFNULL(Items.assetTypeId,0):ne:<FEE id>)` — the
68
+ `IFNULL(...,0)` keeps rows with a NULL FK from being silently dropped.
69
+
70
+ ### Join alias rule (which table-qualified name to use)
71
+
72
+ In `_underscore/Model/Client/TableView.php` (~line 89-96), the **first** occurrence of a joined
73
+ table in a view gets the bare table name as its SQL alias (e.g. `Items`); each subsequent join to
74
+ the same table gets a `_B`, `_C`, … suffix (counter starts at ASCII 65). So an `apiWhereClause`
75
+ referencing `Items.<field>` only binds correctly when the view joins `Items` **exactly once**.
76
+ Before writing a clause, confirm the target table is joined once, or qualify against the correct
77
+ suffixed alias.
78
+
79
+ ## Data model
80
+
81
+ - `TableViews.apiWhereClause` (TEXT, nullable) — the view-level row filter.
82
+ - `TableViewJoins` — the view's joins; drive `options.join` / `ojoin` so any table referenced in
83
+ the clause is present on the data request.
84
+
85
+ ## Client variations
86
+
87
+ None — engine behavior. A specific client's view may carry its own `apiWhereClause` value.
88
+
89
+ ## Gotchas / known issues
90
+
91
+ - **Comma inside a function is safe.** The parser only splits conditions on top-depth commas, so
92
+ a comma inside `IFNULL(x,0)` or `IN(...)` does not break the clause.
93
+ - **An expression field must NOT start with `(`.** A condition that starts with `(` is treated by
94
+ the parser as a nested group and recursed into — so wrap an expression field in a function that
95
+ starts with a letter (e.g. `IFNULL(...)`), never a bare parenthesized expression.
96
+ - **NULL FKs drop silently.** A plain `field:ne:x` excludes rows where `field IS NULL` too. Wrap
97
+ nullable columns in `IFNULL(col,<sentinel>)` when the intent is "exclude only these values."
98
+ - **Operator allowlist.** `parseOptionsWhere` whitelists the operators above and rejects unknown
99
+ ones (e.g. `regexp` returns a 500) — see the Compass cost-centers doc. Numeric/regex filtering
100
+ that the grammar can't express must be done client-side or by adding an operator to api2.
101
+
102
+ ## Change history
103
+
104
+ - 2026-07-15 — Documented the end-to-end `apiWhereClause` row-filter mechanism (emission →
105
+ consumption → `parseOptionsWhere` grammar), the operator→SQL table, the raw-expression field
106
+ slot, the leading-paren / top-depth-comma parser behavior, and the `_underscore` join-alias
107
+ (`_B`/`_C`) rule. No code change — mechanism discovered while building the Compass FEE-exclusion
108
+ migration. (bala)
109
+
110
+ ## Related docs
111
+
112
+ - `clients/compass-usa/features/item-fulfillment-tracking-tableview.md` — the Compass fulfillment
113
+ view (id 12) that first used this to exclude FEE-asset-type items.
114
+ - `clients/compass-usa/features/cost-centers.md` — notes `parseOptionsWhere` rejecting `regexp`
115
+ with a 500 (operator allowlist).
@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
19
19
 
20
20
  - **_underscore** (_Underscore) _(framework core)_ — 32 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
21
21
  - **worker2** (Worker) — 30 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
22
- - **api2** (API) — 10 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
22
+ - **api2** (API) — 11 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
23
23
  - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
24
24
  - **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
25
25
  - **saml** (SAML SSO Gateway) — 3 doc(s) → [2.0/apps/saml/INDEX.md](2.0/apps/saml/INDEX.md)
@@ -4,7 +4,7 @@
4
4
  |-----|-----------|---------|-------|
5
5
  | [Compass ASN → ItemFulfillment Auto-Creation](features/asn-to-item-fulfillment.md) | 2.0 | For Compass USA, posting an AdvanceShippingNotice (ASN) auto-creates the ItemFulfillment (IF) on the upstream SalesOrder. | _underscore/Model/Compass/AdvanceShippingNotice.php, _underscore/Model/Compass/PurchaseOrder.php, api2/Component/Api/Cxml/Cxml.php, dbchanges2/Client_Compass/2026-06-11 - AsnItemTrackingNumberAcl.sql, dbchanges2/Client_Compass/2026-06-15b - BackfillSA132781ItemFulfillmentTracking.sql, dbchanges2/Client_Compass/2026-06-16 - CleanupSA132763CrossLineTracking.sql, dbchanges2/Client_Compass/2026-06-16b - CleanupSA132743CrossLineTracking.sql, dbchanges2/Client_Compass/2026-06-16c - BackfillSA132763C40QYUCTracking.sql, dbchanges2/Client_Compass/2026-06-18a - CleanupSA132898DuplicateTracking.sql, dbchanges2/Client_Compass/2026-06-18b - CleanupSA132881DuplicateTracking.sql, dbchanges2/Client_Compass/2026-07-02a - FixSA133377TrackingSerialAndDuplicateIF.sql |
6
6
  | [Cost Centers — Unit Locations, numeric-only policy](features/cost-centers.md) | 2.0 | A Compass "cost center" — the value a user picks in commerce and that lands on an order — is **not** a `CostCenters` row. | toga2-commerce/src/pages/Cart/api/CartApi.ts, worker1.5/crons/toga2/compass/import_locations.php, _underscore/Model/Compass/SalesOrder.php, api2/Component/Api/V2/V2.php, dbchanges2/Client_Compass/2026-07-06 - RemoveNonNumericCostCenters.sql |
7
- | [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two sibling Compass TableViews in `Client_Compass` display fulfilled items in toga2-supply, both driven by `TableViews` / `TableViewJoins` / `TableViewFields` c | dbchanges2/Client_Compass/2026-06-10 - ItemFulfillmentsForSalesOrderItemsTableView.sql, dbchanges2/Client_Compass/2026-06-11 - ItemFulfillmentsForSalesOrdersTableView.sql, dbchanges2/Client_Compass/2026-06-15a - FixItemFulfillmentTrackingNumberJoins.sql |
7
+ | [Compass: Item-Fulfillment TableViews (for-sales-order-items & for-sales-orders, tracking via bridge)](features/item-fulfillment-tracking-tableview.md) | 2.0 | Two sibling Compass TableViews in `Client_Compass` display fulfilled items in toga2-supply, both driven by `TableViews` / `TableViewJoins` / `TableViewFields` c | dbchanges2/Client_Compass/2026-06-10 - ItemFulfillmentsForSalesOrderItemsTableView.sql, dbchanges2/Client_Compass/2026-06-11 - ItemFulfillmentsForSalesOrdersTableView.sql, dbchanges2/Client_Compass/2026-06-15a - FixItemFulfillmentTrackingNumberJoins.sql, dbchanges2/Client/2026-07-15a - ExcludeFeeItemsFromItemFulfillmentsForSalesOrdersView.sql |
8
8
  | [Compass MITS PO → SO Item Linking](features/mits-po-to-so-item-linking.md) | 2.0 | MITS sends Compass inbound Purchase Orders (`POST /v2/purchase-orders`) against a Sales Order (`mitsSalesOrder`). | _underscore/Model/Compass/PurchaseOrder.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php |
9
9
  | [Compass MITS PO Transmission to Vendors](features/mits-po-transmission-to-vendors.md) | 2.0 | The 1.0 worker cron `2_transmit_mits_purchase_orders_to_vendors.php` transmits Compass PurchaseOrders to their vendors (Office Depot, Strategic Systems, Compass | worker/crons/toga2/compass/workflow/2_transmit_mits_purchase_orders_to_vendors.php, worker/crons/toga2/compasscanada/workflow/2_transmit_mits_purchase_orders_to_vendors.php, library/app/client/compass.php |
10
10
  | [Compass MR/MA Order Auto-Approval & Status Gate](features/mr-ma-order-approval-and-status.md) | 2.0 | Compass **MR** and **MA** sales orders are system-generated from the MITS / Office Depot EDI pipeline (they do not originate as user-entered SA orders) and must | _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/PurchaseOrder.php, worker/crons/toga2/compass/workflow/3a_import_office_depot_purchase_orders.php |
@@ -5,14 +5,16 @@ project: _Underscore
5
5
  client: compass-usa
6
6
  type: client-feature
7
7
  status: active
8
- updated: 2026-06-15
9
- owners: ["jcardinal"]
8
+ updated: 2026-07-15
9
+ owners: ["jcardinal", "bala"]
10
10
  files:
11
11
  - dbchanges2/Client_Compass/2026-06-10 - ItemFulfillmentsForSalesOrderItemsTableView.sql
12
12
  - dbchanges2/Client_Compass/2026-06-11 - ItemFulfillmentsForSalesOrdersTableView.sql
13
13
  - dbchanges2/Client_Compass/2026-06-15a - FixItemFulfillmentTrackingNumberJoins.sql
14
+ - dbchanges2/Client/2026-07-15a - ExcludeFeeItemsFromItemFulfillmentsForSalesOrdersView.sql
14
15
  related:
15
16
  - ../../../2.0/apps/_underscore/features/tracking-number-bridges.md
17
+ - ../../../2.0/apps/api2/features/tableview-apiwhereclause-row-filtering.md
16
18
  ---
17
19
 
18
20
  ## Summary
@@ -93,14 +95,30 @@ returnTrackingNumberId. Superseded unit-level keys: 2183 IFIU-bridge.itemFulfill
93
95
  - With null `joinOnTableViewJoinId`, the engine resolves a join's parent table by the `recordId` of
94
96
  `parentRecordFieldId` — this only works because each record appears once in the graph. Reusing a
95
97
  record twice in one view would require an explicit `joinOnTableViewJoinId`.
98
+ - **FEE-item exclusion (view 12) is filtered via `apiWhereClause`, and "asset type" is a nullable
99
+ FK — not a string.** `Items.assetTypeId` is a nullable FK to `AssetTypes(id, uuid, name)`; the
100
+ id for a given name (e.g. `FEE`) **differs per client** (Compass: `FEE` = id 3, 62 items). About
101
+ 30% of `Items` rows have a **NULL** `assetTypeId` (Compass: 227,685 of ~269,980 fulfilled-item
102
+ rows in this view), so any exclusion **must be NULL-safe** — a plain `assetTypeId:ne:<id>` would
103
+ silently drop every NULL row. The migration uses `(IFNULL(Items.assetTypeId,0):ne:<FEE id>)` with
104
+ the FEE id resolved per client DB via `(SELECT id FROM AssetTypes WHERE name='FEE' ...)`. See the
105
+ shared `apiWhereClause` mechanism doc. The `apiWhereClause` binds `Items.assetTypeId` correctly
106
+ only because view 12 joins `Items` exactly once (the join-alias rule).
107
+ - **The FEE-exclusion migration is team-wide, not Compass-only.** It lives in the `dbchanges2/Client/`
108
+ fan-out folder, so it runs against **every** client DB, guarded to fire only where the view exists,
109
+ a `FEE` asset type exists, and `Items` is joined exactly once. It composes idempotently onto any
110
+ existing `apiWhereClause` (a CASE skips if `%assetTypeId%` is already present). Verified read-only
111
+ on prod + dev-sandbox that it excludes only FEE rows and keeps all NULL-assetType rows.
96
112
 
97
113
  ## Client variations
98
114
  Compass-only. The underlying bridge model is shared (see the _underscore feature doc).
99
115
 
100
116
  ## Change history
117
+ - 2026-07-15 — Excluded FEE-asset-type items from view 12 (`item-fulfillments-for-sales-orders`) via a NULL-safe `apiWhereClause` `(IFNULL(Items.assetTypeId,0):ne:<FEE id>)`, FEE id resolved per client DB. Migration `2026-07-15a` in `dbchanges2/Client/` (fans out to all clients, idempotent, guarded). Verified read-only on prod + dev-sandbox. (bala)
101
118
  - 2026-06-15 — Re-pointed tracking from the unit-level bridge (319) to the item-level bridge `ItemFulfillmentItems_TrackingNumbers` (318) in views 12 & 13; fixes blank item-level tracking on non-serialized lines (e.g. SO MR243437). Migration `2026-06-15a`. (jcardinal)
102
119
  - 2026-06-11 — Re-rooted `item-fulfillments-for-sales-orders` (id 12) from Units to ItemFulfillmentItems with OUTER unit/tracking chain; fixes 0-records on orders lacking unit breakdown (e.g. SA132740). (jcardinal)
103
120
  - 2026-06-10 — Re-rooted `item-fulfillments-for-sales-order-items` (id 13) and moved tracking to the bridge join. (jcardinal)
104
121
 
105
122
  ## Related docs
106
123
  - 2.0 _underscore: Tracking-Number Bridge Migration.
124
+ - 2.0 api2: TableView row-filtering via apiWhereClause (the mechanism used for the FEE exclusion).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.348",
3
+ "version": "1.0.350",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",