toga-ai 1.0.676 → 1.0.677

package/knowledge/1.0/apps/library/INDEX.md

@@ -16,7 +16,7 @@
 | [1.0 MVC Page Pattern & New-App Skeleton](features/mvc-page-pattern-and-app-skeleton.md) | This is the **reusable recipe for standing up a new 1.0 (`App_`) application** and for adding pages to one — the folder-based MVC routing, the page lifecycle, t | library/app/framework.php, library/app/frameworkindex.php, library/app/mvc.php, library/app/database.php, library/app/model.php, library/app/config.php |
 | [isFulfillable from NetSuite during Item Sync (Phase 1)](features/netsuite-item-isfulfillable-sync.md) | This is the **1.0 (Phase 1)** half of the `isFulfillable` feature: reading the NetSuite `isfulfillable` flag during item sync and stamping it onto the **Agilant | library/app/netsuite.php, library/app/api/toga2.php, worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/backfill_isfulfillable_jul5.php |
 | [NetSuite SuiteQL/REST API Reference](features/netsuite-suiteql-api-reference.md) | General working reference for the Agilant NetSuite integration: how to authenticate, how SuiteQL behaves, and the confirmed schema of the tables/columns/codes w | library/app/api/netsuite/rest.php, library/ssl/netsuite_ec_key.pem, test/@dave/Junk Drawer/nsq.php |
-| [NetSuite SuiteQL/REST Shim — Field Semantics](features/netsuite-suiteql-rest-shim.md) | `App_Api_Netsuite_Rest` is the REST/SuiteQL replacement for the deprecated NetSuite SOAP toolkit. | library/app/api/netsuite/rest.php, library/app/netsuite.php |
+| [NetSuite SuiteQL/REST Shim — Field Semantics](features/netsuite-suiteql-rest-shim.md) | `App_Api_Netsuite_Rest` is the REST/SuiteQL replacement for the deprecated NetSuite SOAP toolkit. | library/app/api/netsuite/rest.php, library/app/netsuite.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php |
 | [NetSuite Sync Alert Monitor (App_SystemMonitor_NetSuiteIntegration)](features/netsuite-sync-alert-monitor.md) | `App_SystemMonitor_NetSuiteIntegration` (`library/app/systemmonitor/netsuiteintegration.php`, title **"NetSuite Sync Alert"**) is a 1.0 system monitor that watc | library/app/systemmonitor/netsuiteintegration.php, worker/crons/infrastructure/system_monitors.php |
 | [Startech PC Matic B2B Sync (library)](features/startech-pcmaticb2b-sync.md) | `library/app/api/toga2.php` handles bidirectional ticket sync for PC Matic B2B between TOGaDesk 1.0 and TOGA 2.0. | library/app/api/toga2.php, library/app/api/startechticket.php, worker/crons/toga2/startech/common_import_supporting_records.php |
 | [App_Api_Toga2 — TOGa2 API Client & 1.0↔2.0 Sync Bridge](features/toga2-api-client-and-bridge.md) | `App_Api_Toga2` (`library/app/api/toga2.php`, ~8400 lines) is the **1.0-side client for the TOGa 2 (`_underscore`/api2) public API** *and* the home of the cross | library/app/api/toga2.php, worker/crons/toga2/aig/sync_togasupply_aig.php, worker/crons/toga2/wje/sync_togasupply_wje.php, test/@Mark/AIG/test_multi_email.php |

package/knowledge/1.0/apps/library/features/netsuite-suiteql-rest-shim.md

@@ -6,16 +6,18 @@ project: Library
 client: shared
 type: feature
 status: active
-updated: 2026-08-25
-owners: [dfranks, jcardinal]
+updated: 2026-08-27
+owners: [dfranks, jcardinal, kyalamarthi]
 files:
   - library/app/api/netsuite/rest.php
   - library/app/netsuite.php
+  - worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php
 related:
   - netsuite-suiteql-api-reference.md
   - ../architecture.md
   - ../../worker/features/forecast2-netsuite-reconciliation.md
   - ../../../2.0/apps/_underscore/features/netsuite-rest-client.md
+  - ../../../../2.0/apps/worker2/features/netsuite-salesorder-open-orders-sync.md
 ---
 
 ## Summary
@@ -50,7 +52,11 @@ Forecast2 tables already store.
   re-maps/collides when SOAP-era records are edited. Verified: sales orders 106 orders / 344
   lines (2019–2029, incl. edited orders with line-id gaps), invoices + cash sales 62/62.
   `listSales` historically resolved line via `uniquekey → REST lineUniqueKey → line`; `tl.id`
-  gives the same answer in bulk without the per-id GET.
+  gives the same answer in bulk without the per-id GET. **Re-confirmed at scale 2026-08-27:** the two
+  columns actually diverge on **2,061 of 4,387** lines (~47%), and `Forecast.OpenOrderItems.lineNumber`
+  (which stores REST `$line->line`) matched `transactionline.id` on **99 of 99** randomly sampled prod
+  rows plus all 7 multi-root orders, with zero join misses. So keying a NetSuite ↔ Forecast line join on
+  `linesequencenumber` silently mismatches about half the book — always join on `transactionline.id`.
 
 - **ShipItem lines have `costestimate = NULL`** (100% of 42K+ lines, 2025–26). In any profit
   expression `SUM(-foreignamount + costestimate)`, a NULL cost makes the whole term NULL and SQL
@@ -146,6 +152,35 @@ None — uniform across clients (NetSuite is a single shared account).
 
 ## Gotchas / known issues
 
+- **`tl.location` was in this query, then wasn't, then was again — and the second removal was a MERGE,
+  not a decision.** History, because two separate tickets have now paid for it:
+  1. TRUE-79078 (commit **31fadd77**) added `tl.location` + `BUILTIN.DF(tl.location) AS location_name`
+     to the `listSalesOrders()` line query.
+  2. Merge **c3caa11a** (2026-06-24, *"Merge branch '_production' into TRUE-79078"*) resolved the
+     conflict in favour of `_production` and **silently dropped both columns** — leaving
+     `rest.php:521-523` still *reading* `$line->location`, so `$lineShim->location` became
+     unconditionally null. No syntax error, no test failure, just a permanently-null field.
+  3. The NYCHH transfer-order work re-added them on 2026-08-27 for an unrelated reason (origin-location
+     resolution / `EV-10`) — see
+     [NYCHH TransferOrders import](../../../clients/nychh/features/netsuite-transfer-order-import.md).
+     A second 2026-08-27 session (forecast2 open-orders, TRUE-79162) independently re-derived the same
+     defect, re-verified the restored query live (returns location `4` / `New York`), and **deferred its
+     copy of the fix to a future TRUE-79078 PR**.
+
+  **Consequence: two same-day records disagree about whether the SELECT is present, so verify the branch
+  you are on** — `git grep -n "tl.location" library/app/api/netsuite/rest.php` — rather than trusting
+  either doc. And note what restoring it *re-animates*: the leaf→root location rollup at
+  `worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php:1636-1934`, which
+  **auto-inserts and renames `Forecast.Locations` rows**, has been dead code the whole time
+  `$line->location` was null. It stays dormant only because `import_open_orders.php` is `"active": 0`
+  — re-enable that cron on a branch that selects `tl.location` and a dimension table starts being
+  written by a code path nobody has exercised since June. See
+  [open-orders sync](../../../../2.0/apps/worker2/features/netsuite-salesorder-open-orders-sync.md).
+- **A merge resolved "in favour of `_production`" is a silent feature revert — re-grep for your own
+  additions afterwards.** The c3caa11a case is the pattern: it removed two SELECT columns while leaving
+  their consumers in place. It also broke `listLocations()`'s closing brace, and *that* half **was**
+  repaired later (**5c1c194e**, "TRUE-80090: fixing syntax issue") — which is precisely what makes the
+  whole breakage read as already-fixed to the next person who looks.
 - The shim must stay PHP 7.2-compatible (library is 7.2 prod): no arrow functions, typed props,
   `??=`, or `match`. Lint with `C:\xampp7\php\php.exe -l` before deploying.
 - Field availability varies by NetSuite account **and** record type — probe the live account
@@ -157,6 +192,19 @@ None — uniform across clients (NetSuite is a single shared account).
 
 ## Change history
 
+- 2026-08-27 — **Traced the missing `tl.location` SELECT to its cause — a merge, not an omission — and
+  recorded what restoring it re-animates** (forecast2 open-orders, TRUE-79162). TRUE-79078 (31fadd77)
+  originally added `tl.location` + `BUILTIN.DF(tl.location)`; merge c3caa11a (2026-06-24) resolved in
+  favour of `_production` and dropped both while `rest.php:521-523` kept reading `$line->location`, so
+  the field was unconditionally null — and the leaf→root rollup at
+  `common_import_sales_from_netsuite.php:1636-1934`, which **auto-inserts/renames `Forecast.Locations`
+  rows**, has been dead code since (dormant only because `import_open_orders.php` is `active: 0`). Same
+  merge broke `listLocations()`'s brace and *that* half was repaired separately (5c1c194e), which is why
+  the breakage looks already-fixed. This session's own fix was verified live (location `4` /
+  `New York`) but **deferred to a future TRUE-79078 PR**, so it disagrees with the NYCHH entry below
+  about the current state — grep the branch before assuming. Also re-confirmed `tl.id` vs
+  `linesequencenumber` at scale: they diverge on 2,061 of 4,387 lines, and
+  `OpenOrderItems.lineNumber` == `tl.id` on 99/99 sampled prod rows. (kyalamarthi)
 - 2026-08-27 — Recorded three field-semantics facts from NYCHH transfer-order support:
   `listSalesOrders()` line SuiteQL now **selects `tl.location` + `BUILTIN.DF(tl.location)`** (the line
   shim read them but never selected them → always null, breaking TO origin resolution / V2 `EV-10`);

package/knowledge/1.0/apps/worker/INDEX.md

@@ -7,7 +7,7 @@
 | [Compass Manager Approval Reminder Emails (1.0 worker crons)](features/compass-manager-approval-reminder-emails.md) | Two 1.0 worker crons nag approvers about sales orders still waiting on a decision — one per Compass tenant. | worker/crons/toga2/compass/compass_email_reminders.php, worker/crons/toga2/compasscanada/compass_email_reminders.php, worker/schedules/cron.worker.sync.json, worker1.5/crons/toga2/compass/compass_email_reminders.php, worker1.5/schedules/cron.worker.json |
 | [Compass Partial In-Transit & Delivered Emails (per package)](features/compass-partial-in-transit-delivered-emails.md) | Compass USA and Compass Canada send a **per-package** in-transit email (and a matching delivered email) instead of one email listing the whole order. | worker/crons/toga2/compass/update_salesorder_status_from_odp.php, worker/crons/toga2/compass/send_delivered_email.php, worker/crons/toga2/compasscanada/update_salesorder_status_from_odp.php, worker/crons/toga2/compasscanada/workflow/3_update_salesorder_status_from_grand_and_toy.php, worker/crons/toga2/compasscanada/send_delivered_email.php, worker/crons/toga2/compasscanada/workflow/test_partial_in_transit_email.php, worker/crons/toga2/compasscanada/workflow/test_partial_delivered_email.php, library/app/client/compasscanada.php |
 | [Elite TOGA 2.0 → TOGaDeskSupport Standalone Attachment Sync](features/elite-togadesk-attachment-sync.md) | `sync_togadesk_elite_attachments.php` is a standalone cron (every 5 minutes) that syncs file attachments from TOGA 2.0 into TOGaDeskSupport for Elite. | worker/crons/toga2/elite/sync_togadesk_elite_attachments.php, worker/crons/toga2/elite/test_sync_togadesk_elite_attachments.php |
-| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/checker.php, worker2/Component/Forecast/SaleImport/SaleImport.php, test/@dave/looper.php, test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/reconcile_drift_2023plus.php, test/@dave/probe_invoice_gap_2026.php, test/@dave/probe_creditmemo_gap_detail.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
+| [Forecast2 ↔ NetSuite Reconciliation & Trueup Tooling](features/forecast2-netsuite-reconciliation.md) | CLI tools to **audit** and **repair** drift between the production `Forecast` DB (core2) and NetSuite. | test/@dave/checker.php, worker2/Component/Forecast/SaleImport/SaleImport.php, test/@dave/looper.php, test/@dave/reconcile_netsuite_totals.php, test/@dave/fixer.php, tools/bin/forecast/fixer.php, test/@dave/analyze_netsuite_forecast_diff.php, test/@dave/trueup_sales.php, test/@dave/reconcile_drift_2023plus.php, test/@dave/probe_invoice_gap_2026.php, test/@dave/probe_creditmemo_gap_detail.php, test/@dave/trueup_open_orders.php, test/@dave/loop_trueup_open_orders.php, test/@dave/trueup_opportunities.php, test/@dave/probe_sales_gap_direct.php, test/@dave/probe_missing_oo_timing.php, test/@dave/probe_missing_oo_createdby.php, test/@dave/probe_drift_so_dates.php, test/@dave/probe_profit_invoices.php, test/@dave/probe_profit_gap.php, worker/crons/toga2/forecast2/common_import_sales_from_netsuite.php, worker/crons/toga2/forecast2/periodic_forecast_discrepancy_fix_open_orders.php, worker/crons/toga2/forecast2/import_open_orders.php, worker/schedules/cron.worker.infrastructure.json |
 | [NetSuite Sales Order Sales Rep Sourcing (Staples & ODP EDI orders)](features/netsuite-sales-order-sales-rep-sourcing.md) | How the **sales rep** on a NetSuite Sales Order is determined for the two 1.0 `worker` EDI order-creation integrations (Staples cXML and Compass/ODP EDI). | worker/crons/toga2/compass/workflow/5_create_netsuite_sales_orders_from_office_depot_purchase_orders.php, worker/crons/sync/staples/sync_staples_cxml.php, test/@Mark/NetSuite/TRUE_80451_customer_salesrep_diag.php |
 | [NetSuite → TOGa Supply Per-Client Sync (thin wrappers)](features/netsuite-togasupply-per-client-sync.md) | Syncs NetSuite transactions (sales orders, purchase orders, invoices, item receipts, item fulfillments, inventory adjustments) into each TOGa Supply (2.0) clien | worker/crons/toga2/netsuite/common_sync_togasupply.php, worker/crons/toga2/netsuite/sync_togasupply_canon.php, worker/crons/toga2/netsuite/sync_togasupply_elite.php, worker/schedules/cron.worker.sync.json, dbchanges2/_modules/netsuite/2026-04-01 - Parameters.sql, library/app/api/toga2.php, library/app/api/netsuite/rest.php, library/app/framework.php, library/app/systemmonitor/netsuiteintegration.php, test/@srija/Elite Testing/Service Requests/test_sync_togasupply_elite_section.php, test/@srija/Elite Testing/Service Requests/test_diagnose_togasupply_elite.php |
 | [OneUptime Server monitor + disk/memory hygiene on the 1.0 worker EB host](features/oneuptime-server-monitor-host-hygiene.md) | The 1.0 `agilant-worker` EB environment runs on the **legacy Amazon Linux 1 PHP 7.2 platform** (Apache httpd/prefork, s3fs mounts, cron) and repeatedly went dow | worker/.ebextensions/040_disk_memory_hygiene.config, worker/.ebextensions/045_oneuptime_agent.config, worker/ebs/cron.worker.php, worker/ebs/mount-s3fs-folders.php, worker/ebs/apache_settings.php, worker/ebs/setup_phpini.php |

package/knowledge/1.0/apps/worker/features/forecast2-netsuite-reconciliation.md

@@ -6,14 +6,15 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-07-23
-owners: [dfranks, jcardinal]
+updated: 2026-08-27
+owners: [dfranks, jcardinal, kyalamarthi]
 files:
   - test/@dave/checker.php
   - worker2/Component/Forecast/SaleImport/SaleImport.php
   - test/@dave/looper.php
   - test/@dave/reconcile_netsuite_totals.php
   - test/@dave/fixer.php
+  - tools/bin/forecast/fixer.php
   - test/@dave/analyze_netsuite_forecast_diff.php
   - test/@dave/trueup_sales.php
   - test/@dave/reconcile_drift_2023plus.php
@@ -333,6 +334,44 @@ by reconciling a chosen tranDate range directly against NetSuite.
   is the load-bearing reason the reconciliation backstop exists, not an optimization. (Oracle docs:
   server scripting on CSV import `section_4676525683`; how UE scripts are executed `section_1512409310`.)
 
+### The open-orders discrepancy-fix cron now carries `locationId` (2026-08-27)
+
+`periodic_forecast_discrepancy_fix_open_orders.php` (`active:1`, `0 3 * * *`) contained **zero**
+occurrences of the string "location" in 775 lines: its line SuiteQL didn't select it,
+`buildInsertSql()` omitted `locationId` from both the SET list **and** the `ON DUPLICATE KEY UPDATE`
+list, `buildUpdateSql()` omitted it, and the `$compare` array had no `locationId` key. Because the cron
+**delete+reinserts** rows (three delete sites), every row it recreated came back with `locationId` NULL
+— which is why the 2.0 webhook writer alone could never hold the value. Root cause and census live in
+[NetSuite → Forecast Open-Orders Sync](../../../../2.0/apps/worker2/features/netsuite-salesorder-open-orders-sync.md).
+
+Five edits, in the order they matter:
+
+1. `location AS locationid` added to the line SuiteQL.
+2. A `Locations` lookup plus `$hasLocationCol` / `netsuiteParentLocationId` **column guards** built on
+   the file's existing `forecastColumnExists()` helper — dev-sandbox has no `locationId` column at all,
+   so an unguarded write raises **MySQL 1054** there.
+3. `resolveOoiLocationId()` copied from **`tools/bin/forecast/fixer.php:1328-1342`** (which already
+   mirrors `_Component_Forecast_Db::resolveLocationId`) rather than lifting the entangled tree walk out
+   of `common_import_sales_from_netsuite.php`. Pick the copy that is already a faithful mirror — that is
+   the cheapest way to honour the compute-identically invariant.
+4. `locationId` threaded through **both** SQL builders, which were also converted from **12 positional
+   parameters to a single array parameter** per the team 4+-parameter rule.
+5. A **null-safe `locationId` entry added to `$compare`.**
+
+**Edit 5 is the one that turns the cron into its own backfill — and it is the generalizable lesson.**
+Without a `$compare` entry, a row whose *only* difference from NetSuite is a missing `locationId` is
+classified **"unchanged"** and is never repaired, so the fix would have covered new rows and nothing
+else. With it, the cron repairs every row inside its `[today−2y, today+10y]` `tranDate` window (1,711
+rows / 750 orders) on its next run, and only the 1,133 rows / 608 orders that fall **before** the window
+(back to 2019-04-24, scanned by nothing at all) need a separate backfill — that is
+`Netsuite/OpenOrderLocationBackfill/Backfill` in worker2.
+
+**Verification gotcha worth keeping:** the first cut of the `ON DUPLICATE KEY UPDATE` clause rendered
+`classificationId = VALUES(classificationId),,` plus a missing separator before `itemId` — which would
+have broken **every insert the cron makes**. It was caught only by rendering the generated SQL through a
+stub harness before any run. **`php -l` cannot catch malformed SQL string assembly**; when you edit a
+query *builder*, print the built query and read it.
+
 ## Data model
 
 `Forecast.Sales`, `Forecast.OpenOrderItems` on the **core2** cluster
@@ -558,7 +597,12 @@ None — Forecast2 is a single shared dataset.
   **not** a resurrection risk against the webhook `removeAll` path; it is in fact the current backstop
   deleter for orders the webhook never sees billed (the invoice-transform gap).
 - **`fixer.php`'s OpenOrderItems path does NOT carry `locationId` / `quantityBackordered` /
-  `amountDue` — mirror any new open-order column here or the reconciler reverts it.** `fixer.php`
+  `amountDue` — mirror any new open-order column here or the reconciler reverts it.** (Still true as of
+  2026-08-27 for the write path, even though `tools/bin/forecast/fixer.php:1328-1342` already ships a
+  `resolveOoiLocationId()` helper mirroring `_Component_Forecast_Db::resolveLocationId` — a helper
+  existing is not the same as the OOI SELECT/compare/INSERT/UPDATE using it. The nightly cron **was**
+  fixed for `locationId` (above); `fixer.php` was **not**, so a `fixer`/`looper` pass can still revert
+  it.) `fixer.php`
   already handles anchor-line **`amountDue` on the Sales path**, guarded by a column-existence check
   `$hasAmountDueCol = forecastColumnExists('Sales','amountDue')` (so it's a no-op until the prod
   column lands). Its **OOI** path — the lookup SELECT, change-detection, the
@@ -602,6 +646,24 @@ None — Forecast2 is a single shared dataset.
 
 ## Change history
 
+- 2026-08-27 — **Taught the nightly open-orders discrepancy-fix cron about `locationId` — and turned it
+  into its own backfill (TRUE-79162).** The cron had **zero** occurrences of "location" in 775 lines
+  (line SuiteQL, both SQL builders, and `$compare` all omitted it) and it delete+reinserts rows, so it
+  was actively erasing the value the 2.0 webhook writer had set. Five edits: `location AS locationid` in
+  the line query; a `Locations` lookup behind `forecastColumnExists()` column guards (dev-sandbox has no
+  `locationId` column — unguarded writes raise MySQL 1054); `resolveOoiLocationId()` copied from
+  `tools/bin/forecast/fixer.php:1328-1342` (already a faithful mirror of
+  `_Component_Forecast_Db::resolveLocationId`) instead of the entangled walk in
+  `common_import_sales_from_netsuite.php`; `locationId` threaded through both SQL builders, which were
+  refactored from **12 positional params to one array param** per the 4+-param rule; and a null-safe
+  `locationId` entry in **`$compare`** — which is what makes the cron repair the 1,711 in-window rows
+  automatically (without it, a row whose only difference is a missing location classifies "unchanged"
+  and is never touched). The 1,133 pre-window rows go to worker2's
+  `Netsuite/OpenOrderLocationBackfill/Backfill`. Recorded the verification gotcha: the first cut of the
+  `ON DUPLICATE KEY UPDATE` clause rendered a double comma and a missing separator that would have broken
+  **every** insert — caught by rendering the generated SQL through a stub harness, which `php -l`
+  cannot do. Also corrected the `fixer.php`-OOI gotcha: `fixer.php` still does not write `locationId`
+  even though it already ships the `resolveOoiLocationId()` helper. (kyalamarthi)
 - 2026-07-23 — **Recorded WHY the reconciliation backstop is mandatory + the by-id re-fetch
   delete-safety mechanism** (TRUE-80262 planning; no code shipped, dfranks). Documented the broader
   NetSuite event-capture blind spot beyond the inline sublist edit: **bulk/mass updates and CSV

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -4,7 +4,7 @@
 |-----|---------|-------|
 | [Proposed — git-sourced base+overlay JSON authoring for the Surface layer](architecture/surface-authoring-proposal.md) | A **proposal / handoff recommendation** (not implemented) that the Surface layer's *authoring* model move off hand-authored SQL against the `SurfaceOverrides` E | _underscore/Model/Core/Surface.php, _underscore/Model/Client/SurfaceOverride.php |
 | [_underscore Framework Architecture](architecture.md) | `_underscore` is the shared PHP backend framework for **all 2.0 applications**. | _underscore/_underscore.php, _underscore/Loader.php, _underscore/Framework.php, _underscore/Model.php, _underscore/Database.php, _underscore/Query.php, _underscore/Route.php, _underscore/Component.php |
-| [ACL Permission Chain (Record & Field Authorization)](features/acl-permission-chain.md) | Authorization in the 2.0 API is **metadata-driven**: whether a role may Create/Read/Update/Delete a record is decided by rows across **four linked tables**, not | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Page.php, _underscore/Model/Client/TrackingNumber.php, dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql, dbchanges2/Client/2026-07-02c - TrackingNumberNeedsReturnLabelFieldPermission.sql, dbchanges2/Client/2026-07-22c - TrackingNumberMeasureIdsFieldPermission.sql, dbchanges2/Client_Quad/2026-08-24 - Quad Multi Currency Item Pricing.sql |
+| [ACL Permission Chain (Record & Field Authorization)](features/acl-permission-chain.md) | Authorization in the 2.0 API is **metadata-driven**: whether a role may Create/Read/Update/Delete a record is decided by rows across **four linked tables**, not | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Page.php, _underscore/Model/Core/Surface.php, _underscore/Model/Client/TrackingNumber.php, dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql, dbchanges2/Client/2026-07-02c - TrackingNumberNeedsReturnLabelFieldPermission.sql, dbchanges2/Client/2026-07-22c - TrackingNumberMeasureIdsFieldPermission.sql, dbchanges2/Client_Quad/2026-08-24 - Quad Multi Currency Item Pricing.sql |
 | [Address Uniqueness Normalization (unit identifier + 5-digit ZIP comparison)](features/address-uniqueness-normalization.md) | When a business rule says *"only one X per physical address"*, comparing address rows field-for-field does **not** work: the same dwelling is spelled many diffe | _underscore/Model/Rate/Entitlement.php |
 | [Address Validation (carrier waterfall + validateAddress scripted endpoint)](features/address-validation.md) | `_Model_Client_Address::validateAddress` verifies a US address against a **carrier waterfall (USPS → FedEx → UPS)** and returns a single canonical, carrier-norm | _underscore/Model/Client/Address.php, _underscore/Component/Library/Carriers/Usps/Usps.php |
 | [_ApiRequest — JSON encode/decode & api-logging behavior](features/apirequest-json-content-type.md) | `_ApiRequest` is the 2.0 outbound HTTP client. | _underscore/ApiRequest.php |
@@ -30,7 +30,7 @@
 | [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php, _underscore/Model.php, _underscore/Model/Rate/Subscription.php |
 | [_Model::save() parent FK cascade — stored-SQL-field recompute deadlocks](features/model-save-parent-cascade-stored-field-deadlock.md) | `_Model::save()` runs a **generic parent foreign-key cascade**: inserting (or saving) a child row that carries an FK to a parent causes `_Model` to **re-load an | _underscore/Model.php, _underscore/Model/Client/PurchaseOrder.php, _underscore/Model/Client/AdvanceShippingNotice.php |
 | [_Model::save() vs raw _Query — no atomic conditional update](features/model-save-vs-query-atomic-update.md) | `_Model::save()` is a plain load-then-write ORM primitive and **cannot express an atomic conditional update** (an optimistic-concurrency / row-claim guard such | _underscore/Model.php, _underscore/Query.php, _underscore/Model/Rate/Subscription.php |
-| [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php, _underscore/Trait/Netsuite/SalesOrder.php, worker2/Worker/Netsuite/SalesOrder.php, worker2/Component/Forecast/SaleImport/SaleImport.php, worker2/Worker/Netsuite/Opportunity.php |
+| [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php, _underscore/Trait/Netsuite/SalesOrder.php, worker2/Worker/Netsuite/SalesOrder.php, worker2/Component/Forecast/SaleImport/SaleImport.php, worker2/Worker/Netsuite/Opportunity.php |
 | [NetSuite Sales Order sync — ship-to address, phone, and PO reference sourcing](features/netsuite-salesorder-address-phone-sync.md) | `_Trait_Netsuite_SalesOrder` is the **shared** sales-order importer composed into **22 client models** (every client on the dbchanges2 `netsuite` module). | _underscore/Trait/Netsuite/SalesOrder.php, _underscore/Model.php, dbchanges2/_modules/netsuite/2026-08-10a - AddressPhoneNumberApiRoleAcl.sql |
 | [Legacy page meta (Page::meta) & context-scoped ClientRecordFieldSettings](features/page-meta-context-field-settings.md) | `_Model_Core_Page::meta()` is the **legacy** page-meta resolver behind `GET /pages/meta?slug=<slug>` — still the live path for `toga2-supply` and other pre-Surf | _underscore/Model/Core/Page.php, _underscore/Model/Client/TableView.php, toga2-supply/src/components/ui/Tables/PrimaryTable/PrimaryTable.tsx, toga2-supply/src/components/ui/Tables/PrimaryTable/helperFunctions/formatTableData.tsx |
 | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/Model.php, _underscore/Query.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php, api2/Controller/Index.php |
@@ -41,7 +41,7 @@
 | [SO↔PO bridge tables are TWO tables in OPPOSITE directions (upstream vs downstream)](features/sales-order-purchase-order-bridge-direction.md) | There are **two** bridge tables linking sales orders and purchase orders, and they mean **opposite things**. | _underscore/Model/Client/PurchaseOrders/SalesOrder.php, _underscore/Model/Client/SalesOrders/PurchaseOrder.php, _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/SalesOrder.php, _underscore/Trait/Netsuite/SalesOrder.php, library/app/api/toga2.php |
 | [Per-client sales-order status filter (Surface FILTER_SET → table meta `filterOptions`)](features/sales-order-status-filter-surface.md) | The status-filter dropdown on the sales-orders table is **per client**, driven by a Surface `FILTER_SET` rather than by the raw contents of the client's `SalesO | _underscore/Model/Client/TableView.php, _underscore/Model/Core/Surface.php, _underscore/Model/Client/SalesOrder.php, _underscore/Model/Quad/SalesOrder.php, _underscore/Model/Prudential/SalesOrder.php, toga-blox/src/components/Table/hooks/useFetchTablePageMeta.ts, toga-blox/src/api/types.ts, dbchanges2/Core/2026-08-24b - SalesOrderStatusFilterSurfaceSeed.sql, dbchanges2/Client_Compass/2026-08-24b - SalesOrderStatusFilterHides.sql, dbchanges2/Client_CompassCanada/2026-08-24b - SalesOrderStatusFilterHides.sql, dbchanges2/Client_Quad/2026-08-24b - SalesOrderStatusFilterHides.sql, dbchanges2/Client_Nychh/2026-08-24b - SalesOrderStatusFilterHides.sql |
 | [_String helpers — ASCII-safe HTML entity encoding (and the parseBetween trap)](features/string-html-entity-helpers.md) | `_String` is the 2.0 framework's static string utility class. | _underscore/String.php |
-| [Surface Resolver (_Model_Core_Surface::resolve — replaces Page::meta)](features/surface-resolver.md) | The runtime for the platform-wide **Surface** UI presentation layer: 9 `_underscore` models plus a cached resolver, `_Model_Core_Surface::resolve(&$api, string | _underscore/Model/Client/Language.php, dbchanges2/Core/2026-08-21 - SalesOrderDecisionSurfacesReseed.sql, dbchanges2/Core/2026-08-24 - RestoreApproveDenyRowActionsVisibility.sql, dbchanges2/Client_Quad/2026-08-24 - ProdPortApprovePoNumberEnabledRule.sql, dbchanges2/Client_Compass/2026-08-24 - ProdPortApprovalsGateAndApproveStepTwoRule.sql, dbchanges2/Client_CompassCanada/2026-08-24 - ProdPortApprovalsGateAndApproveStepTwoRule.sql, dbchanges2/Client_Nychh/2026-08-24 - FixInventoryGroupingsUnitsTopologyOverrideIds.sql, toga25-supply/src/App.tsx, toga25-supply/src/surface/useFetchSurfaceMeta.ts, toga25-supply/src/contexts/AuthContext.tsx, dbchanges2/Core/2026-07-21a - SalesOrderDecisionSummarySurfaceSeed.sql, dbchanges2/Core/2026-07-21b - SalesOrderDecisionActionSurfaceSeed.sql, dbchanges2/Core/2026-08-21a - SalesOrderDecisionSurfacesReseed.sql, dbchanges2/Client_Compass/2026-07-21c - SalesOrderDecisionSummaryTotalConcat.sql, dbchanges2/Client_Quad/2026-07-21c - SalesOrderDecisionSummaryOverride.sql, dbchanges2/Client_CompassCanada/2026-07-21c - SalesOrderDecisionSummaryTotalConcat.sql, dbchanges2/Client_Compass/2026-08-04a - ApprovalDetailsAssignedManagerPreferredStage.sql, _underscore/Model/Core/Surface.php, _underscore/Model/Client/AclRecordScript.php, _underscore/Model/Core/RecordScript.php, dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql, dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql, dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql, dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql, dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql, dbchanges2/Client_Compass/2026-07-15f - SalesOrderRecordActionsRemoveDeadConfigRuleOverrides.sql, dbchanges2/Core/2026-07-17h - Update - ClearApprovalsFilterButtonConfig.sql, dbchanges2/Client_Compass/2026-07-17a - SalesOrderApprovalActionsOverride.sql, dbchanges2/Client_Compass/2026-07-17b - SalesOrderApprovalsFilterButtonOverride.sql, dbchanges2/Client_CompassCanada/2026-07-17a - SalesOrderApprovalActionsOverride.sql, dbchanges2/Client_CompassCanada/2026-07-17b - SalesOrderApprovalsFilterButtonOverride.sql, dbchanges2/Client_Quad/2026-07-17a - SalesOrderApprovalActionsOverride.sql, dbchanges2/Client_Quad/2026-07-17b - SalesOrderApprovalsFilterButtonOverride.sql, _underscore/Model/Core/SurfaceElement.php, _underscore/Model/Core/Action.php, _underscore/Model/Core/Vocabulary.php, _underscore/Model/Core/VocabularyTerm.php, _underscore/Model/Core/Message.php, _underscore/Model/Client/SurfaceOverride.php, _underscore/Model/Client/MessageTranslation.php, _underscore/Model/Client/ThemeToken.php, _underscore/Model/Core/Page.php, api2/Component/Api/V2/V2.php |
+| [Surface Resolver (_Model_Core_Surface::resolve — replaces Page::meta)](features/surface-resolver.md) | The runtime for the platform-wide **Surface** UI presentation layer: 9 `_underscore` models plus a cached resolver, `_Model_Core_Surface::resolve(&$api, string | _underscore/Model/Client/Language.php, dbchanges2/Core/2026-08-21 - SalesOrderDecisionSurfacesReseed.sql, dbchanges2/Core/2026-08-24 - RestoreApproveDenyRowActionsVisibility.sql, dbchanges2/Client_Quad/2026-08-24 - ProdPortApprovePoNumberEnabledRule.sql, dbchanges2/Client_Compass/2026-08-24 - ProdPortApprovalsGateAndApproveStepTwoRule.sql, dbchanges2/Client_CompassCanada/2026-08-24 - ProdPortApprovalsGateAndApproveStepTwoRule.sql, dbchanges2/Client_Nychh/2026-08-24 - FixInventoryGroupingsUnitsTopologyOverrideIds.sql, toga25-supply/src/App.tsx, toga25-supply/src/surface/useFetchSurfaceMeta.ts, toga25-supply/src/contexts/AuthContext.tsx, dbchanges2/Core/2026-07-21a - SalesOrderDecisionSummarySurfaceSeed.sql, dbchanges2/Core/2026-07-21b - SalesOrderDecisionActionSurfaceSeed.sql, dbchanges2/Core/2026-08-21a - SalesOrderDecisionSurfacesReseed.sql, dbchanges2/Client_Compass/2026-07-21c - SalesOrderDecisionSummaryTotalConcat.sql, dbchanges2/Client_Quad/2026-07-21c - SalesOrderDecisionSummaryOverride.sql, dbchanges2/Client_CompassCanada/2026-07-21c - SalesOrderDecisionSummaryTotalConcat.sql, dbchanges2/Client_Compass/2026-08-04a - ApprovalDetailsAssignedManagerPreferredStage.sql, _underscore/Model/Core/Surface.php, _underscore/Model/Client/AclRecordScript.php, _underscore/Model/Core/RecordScript.php, dbchanges2/Core/2026-06-30a - SurfaceMetaGroupAndSalesOrderSections.sql, dbchanges2/Client/2026-06-30a - SurfaceMetaGroupAcl.sql, dbchanges2/Client_Quad/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Client_CompassCanada/2026-07-01a - GrantSurfacesMetaGroupScriptAcl.sql, dbchanges2/Core/2026-06-29b - SurfaceMetaPublicReadAcl.sql, dbchanges2/Client/2026-06-29c - SurfaceRecordScriptAcl.sql, dbchanges2/Core/2026-06-29c - SurfaceDebugPhpMethodFix.sql, dbchanges2/Client_Compass/2026-07-15f - SalesOrderRecordActionsRemoveDeadConfigRuleOverrides.sql, dbchanges2/Core/2026-07-17h - Update - ClearApprovalsFilterButtonConfig.sql, dbchanges2/Client_Compass/2026-07-17a - SalesOrderApprovalActionsOverride.sql, dbchanges2/Client_Compass/2026-07-17b - SalesOrderApprovalsFilterButtonOverride.sql, dbchanges2/Client_CompassCanada/2026-07-17a - SalesOrderApprovalActionsOverride.sql, dbchanges2/Client_CompassCanada/2026-07-17b - SalesOrderApprovalsFilterButtonOverride.sql, dbchanges2/Client_Quad/2026-07-17a - SalesOrderApprovalActionsOverride.sql, dbchanges2/Client_Quad/2026-07-17b - SalesOrderApprovalsFilterButtonOverride.sql, _underscore/Model/Core/SurfaceElement.php, _underscore/Model/Core/Action.php, _underscore/Model/Core/Vocabulary.php, _underscore/Model/Core/VocabularyTerm.php, _underscore/Model/Core/Message.php, _underscore/Model/Client/SurfaceOverride.php, _underscore/Model/Client/MessageTranslation.php, _underscore/Model/Client/ThemeToken.php, _underscore/Model/Core/Page.php, api2/Component/Api/V2/V2.php, dbchanges2/Core/2026-08-12a - NavigationSurfaceSeed.sql, dbchanges2/Client_Compass/2026-08-12a - NavigationRoleGrants.sql, toga25-supply/surface-layer-guide.md |
 | [Table-View Hyperlink Columns (meta → ACL → computed URL → render)](features/tableview-hyperlink-columns.md) | Any 2.0 table-view column can render its value as a clickable link instead of plain text. | _underscore/Model/Client/TableView.php, _underscore/Model/Client/TrackingNumber.php, api2/Component/Api/V2/V2.php, toga2-supply/src/api/toga.ts, toga2-supply/src/components/ui/Tables/PrimaryTable/helperFunctions/formatTableData.tsx, toga2-supply/src/components/ui/Tables/PrimaryTable/helperFunctions/convertData.tsx, toga2-supply/src/components/ui/Tables/hooks/useDataTableState.tsx, dbchanges2/Client/2026-07-20 - TrackingNumberHyperlinkAndFieldPermission.sql |
 | [TableView joins (TableViewJoins → SQL) — aliasing, chained multi-hop joins, ACL](features/tableview-joins.md) | `Client_*.TableViewJoins` rows are what let a table view show a column from a table other than its base record. | _underscore/Model/Client/TableView.php, dbchanges2/Client/2026-08-11a - ServiceRequestsInventoryUnitsTableViews.sql, dbchanges2/Client_Elite/2026-08-18 - InventoryUnitsItemColumns.sql, dbchanges2/Client_Nychh/2026-08-18 - InventoryUnitsItemColumns.sql |
 | [TogaIQ Gateway Client (_Component_Api_Togaiq) — AI generate/translate from 2.0](features/togaiq-gateway-client.md) | `_Component_Api_Togaiq` is the 2.0 framework's client for the **TogaIQ** (Talos) AI gateway. | _underscore/Component/Api/Togaiq/Togaiq.php, _underscore/ApiRequest.php |

package/knowledge/2.0/apps/_underscore/features/acl-permission-chain.md

@@ -6,11 +6,12 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-08-26
+updated: 2026-08-28
 owners: ["jcardinal", "mhammontree", "tcox", "bala"]
 files:
   - api2/Component/Api/V2/V2.php
   - _underscore/Model/Core/Page.php
+  - _underscore/Model/Core/Surface.php
   - _underscore/Model/Client/TrackingNumber.php
   - dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql
   - dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql
@@ -273,6 +274,31 @@ while the button that would invoke them is hidden, so every permission check you
 `AclActionPermissions` against a client where the control works, and grant the missing
 `(aclActionId, roleId)` in a `dbchanges2` migration.
 
+### ⚠ This gate applies ONLY to `Page::meta` apps — it is DEAD CONFIG in the surface-driven 2.5 app
+
+The flag mechanism above is `_Model_Core_Page::meta()` behavior, and `Page::meta` is being **replaced**
+by the Surface layer. In a surface-driven app (`toga25-supply`) the same-named `navigation-*` flags
+are **inert**:
+
+- `Core.AclActions` carries `navigation-approvals` (1), `navigation-edit-on-commerce` (2),
+  `navigation-sales-orders` (3), `navigation-items` (4), `navigation-vendor-items` (5),
+  `navigation-service-requests` (6), `navigation-edit-on-vendor-items` (7), and every tenant grants
+  some subset of them per role in `Client_<tenant>.AclActionPermissions`. It **looks** exactly like
+  the nav permission gate.
+- But `_Model_Core_Surface::resolve()` only drops an element when **the element declares** an
+  `aclActionId`:
+  `if ($element->aclActionId && !isset($permittedAclActionIds[(int)$element->aclActionId])) continue;`
+  (Surface.php ~L592). Verified prod-wide 2026-08-28: **no `SurfaceElement` on `appId = 1` sets
+  `aclActionId` at all** — all six `navigation` elements have it `NULL`.
+
+**So granting a role `navigation-sales-orders` has ZERO effect on the 2.5 side nav.** The only lever
+is a role-scoped `Client_<tenant>.SurfaceOverrides` row with `attribute = 'IS_VISIBLE'`, `value = '1'`.
+Both facts are true at once: the *same* `AclActionPermissions` row can be load-bearing in
+`toga2-supply` and completely inert in `toga25-supply`. Before you diff `AclActionPermissions`
+across clients, establish **which app** the missing control is in. Detail:
+[Surface Resolver](surface-resolver.md) and
+[toga25-supply side navigation](../../toga25-supply/features/side-navigation-and-default-route.md).
+
 Worked example (Quad, 2026-08-26): the vendor-items **Add and Edit buttons both** read
 `acl['vendor-items']['navigation-edit-on-vendor-items']` (AclActions id 7). `Client_Quad` had a row
 for id 5 only, so the page was reachable but button-less, even though record 19 already granted role
@@ -303,6 +329,21 @@ Worked example: the reprint API `GET /v2/tracking-numbers/reprint`
 Client-DB `AclRecordScripts` grant (migrations in `dbchanges2`) before it stopped 403ing — see the
 carrier-shipping-labels feature doc.
 
+**⚠ `AclRecordScripts` lives in the client DB, but its `roleId` is NOT always a client role id.**
+The role space is chosen by the **record**, exactly as for the CRUD chain above:
+`V2.php` (~L3090) reads
+`if ($record->aclDatabase == \_Model_Core_Record::ACL_DATABASE_CORE) { $roleIds = jwt id.core.roles; } else { $roleIds = jwt id.client.roles; }`
+and `getRecordScriptPhpMethod()` (~L6198) then runs
+`SELECT id FROM AclRecordScripts WHERE recordScriptId = <n> AND roleId IN (<$roleIds>)` against the
+**client** DB with whichever set it got. So for a `CORE` record such as `surfaces`, the client-DB
+`AclRecordScripts.roleId` values must be **`Core.Roles`** ids.
+
+api2 derives `id.core.roles` at token time (~L1611-1627) as
+`SELECT DISTINCT coreRoleId FROM Client_<tenant>.Roles WHERE id IN (<client roles>) AND coreRoleId IS NOT NULL`.
+Most tenant roles have `coreRoleId = NULL`, so this set is usually just `1` (Base) — which is why a
+single `AclRecordScripts` row for core role 1 lets an entire tenant through, and why this gate is
+seldom the cause of an "empty screen" report. Verified prod 2026-08-28 against `Client_Compass`.
+
 ## Add a writable field to a V2 record (FOUR requirements)
 
 Making a field persisted **and** API-writable on an existing V2 record takes **four** things —
@@ -436,6 +477,17 @@ hardcoded `Core.RecordFields` id literals instead of a subselect.
       and every repo is on the **same branch** so the generated model matches the DB.
 
 ## Change history
+- **2026-08-28** — Scoped the **third gate**: `Core.AclActions` + `AclActionPermissions` is
+  `_Model_Core_Page::meta()` behavior only, and is **dead config in the surface-driven `toga25-supply`
+  app** — `_Model_Core_Surface::resolve()` drops an element only when the element itself declares an
+  `aclActionId`, and no `SurfaceElement` on `appId = 1` does (verified prod-wide). Granting
+  `navigation-sales-orders` therefore does nothing for the 2.5 side nav; only a role-scoped
+  `SurfaceOverrides IS_VISIBLE` row works. The same `AclActionPermissions` row can be load-bearing in
+  `toga2-supply` and inert in `toga25-supply`, so establish the app before diffing grants. Also
+  corrected the `AclRecordScripts` gate: the table is client-DB but its `roleId` is matched against
+  the space `Records.aclDatabase` selects — **`Core.Roles`** ids for a `CORE` record such as
+  `surfaces` (`V2.php` ~L3090 / ~L6198), with `id.core.roles` derived from
+  `Roles.coreRoleId` (~L1611-1627) and therefore usually just Base. (bala)
 - 2026-08-27 — Recorded two facts from wiring NYCHH transfer-order custom fields: **there is NO
   ACL/metadata cache** (`buildLookups()` re-reads per request, so a new grant/metadata row applies on
   the next request — no cache-bust exists; a whole debugging pass was wasted assuming one did), and a

package/knowledge/2.0/apps/_underscore/features/netsuite-rest-client.md

@@ -6,10 +6,12 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-08-25
-owners: ["dfranks", "jcardinal", "bala"]
+updated: 2026-08-27
+owners: ["dfranks", "jcardinal", "bala", "kyalamarthi"]
 files:
   - _underscore/Component/Api/Netsuite/Netsuite.php
+  - _underscore/ApiRequest.php
+  - _underscore/Model/Client/Logs/Api.php
   - _underscore/Trait/Netsuite/SalesOrder.php
   - worker2/Worker/Netsuite/SalesOrder.php
   - worker2/Component/Forecast/SaleImport/SaleImport.php
@@ -208,10 +210,30 @@ consumer sources transaction status from a change-safe field:
 
 ## Logging
 
-`send()` calls `setLogging(false)`, so **NetSuite REST request/response bodies are NOT written to
-`Logs.Api`** — unlike platform/ClickUp/Freshservice/Sentry traffic. You cannot pull a historical
-NetSuite response for debugging. (Removing the default-off is under review; see the opportunity-sync
-doc.)
+> **CORRECTED 2026-08-27 — the opposite of what this section used to say.** It claimed
+> `send()` calls `setLogging(false)` and that NetSuite bodies are *not* logged. Verified false:
+> `_underscore/Component/Api/Netsuite/Netsuite.php` contains **zero** occurrences of `setLogging`, and
+> all three `_ApiRequest` constructions in it (`:100` authenticate, `:138` send, `:178`) pass only **4**
+> arguments — so the constructor's `$isLoggingEnabled = true` default applies.
+
+**Every** call through `_Component_Api_Netsuite` is logged — including a read-only SuiteQL `SELECT`.
+`_ApiRequest::execute()` writes a row to `DB_CLIENT_LOGS` via `_Model_Client_Logs_Api`
+(`_underscore/ApiRequest.php:253-270`) on each request.
+
+Two consequences, one good and one expensive:
+
+- **You *can* pull a historical NetSuite request/response for debugging** — the rows are in the
+  client-logs cluster, not missing. (Any older doc or comment telling you the bodies were never
+  captured is wrong; the payload-volume concern is real, but it is a retention/sampling question, not a
+  "logging is off" fact.)
+- **A machine with no `ClientLogs` schema cannot make even a read-only NetSuite query**, and the failure
+  presents as a **database credentials/unknown-database error rather than a logging error** — which is
+  why it reads as a NetSuite auth problem. See
+  [running worker2 locally](../../worker2/workflows/running-worker2-locally.md) and
+  [per-client database connections](./per-client-database-connections.md).
+
+Note that `Authorization` headers are persisted in plaintext by that same auto-logging path — see
+[`_ApiRequest` JSON/logging behavior](./apirequest-json-content-type.md).
 
 ## Gotchas / known issues
 
@@ -231,6 +253,15 @@ doc.)
 
 ## Change history
 
+- 2026-08-27 — **CORRECTED the Logging section: NetSuite REST calls ARE logged to `Logs.Api`, always.**
+  The previous claim (`send()` calls `setLogging(false)`, bodies not captured) is false — there are
+  **zero** `setLogging` occurrences in `_underscore/Component/Api/Netsuite/Netsuite.php`, and all three
+  `_ApiRequest` constructions (`:100`, `:138`, `:178`) pass only 4 args, so the constructor's
+  `$isLoggingEnabled = true` default applies. Every call — including a read-only SuiteQL SELECT —
+  writes a `_Model_Client_Logs_Api` row to `DB_CLIENT_LOGS` (`ApiRequest.php:253-270`). Durable
+  consequence: a laptop with no `ClientLogs` schema **cannot run even a read-only NetSuite query**, and
+  the failure surfaces as a database-credentials error, not a logging error. Surfaced while building the
+  open-orders location backfill (TRUE-79162). (kyalamarthi)
 - 2026-08-25 — Documented **2.0 NetSuite transaction-status sourcing** and the conclusion that
   the **NetSuite 2026.2 REST `status.id` text→letter standardization does not affect 2.0**
   (SuiteAnswers 89313). `fetchRecord()` returns the REST record raw (no central status

package/knowledge/2.0/apps/_underscore/features/surface-resolver.md

@@ -6,8 +6,8 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-08-27
-owners: [jcardinal, apeterson]
+updated: 2026-08-28
+owners: [jcardinal, apeterson, bala]
 files:
   - _underscore/Model/Client/Language.php
   - dbchanges2/Core/2026-08-21 - SalesOrderDecisionSurfacesReseed.sql
@@ -54,11 +54,15 @@ files:
   - _underscore/Model/Client/ThemeToken.php
   - _underscore/Model/Core/Page.php
   - api2/Component/Api/V2/V2.php
+  - dbchanges2/Core/2026-08-12a - NavigationSurfaceSeed.sql
+  - dbchanges2/Client_Compass/2026-08-12a - NavigationRoleGrants.sql
+  - toga25-supply/surface-layer-guide.md
 related:
   - ../../dbchanges2/features/surface-layer-schema.md
   - ../../api2/features/surface-meta-option.md
   - acl-permission-chain.md
   - sales-order-status-filter-surface.md
+  - ../../toga25-supply/features/side-navigation-and-default-route.md
 ---
 
 ## What it is
@@ -94,8 +98,11 @@ Inputs come from `$api` (the same accessor `_Model_Client_TableView::meta` uses)
    injection).
 2. One query: all `SurfaceElements` for `surfaceId`, ordered by `sortOrder`.
 3. One query: matching `SurfaceOverrides`; apply **most-specific-last** in a single in-memory pass
-   (`ORDER BY (roleId IS NULL) DESC, (personaId IS NULL) DESC`; language filtered to active or NULL).
-   Precedence **base < client < persona < role**, language an orthogonal overlay.
+   (`ORDER BY (roleId IS NULL) DESC, (personaId IS NULL) DESC, id ASC`; language filtered to active
+   or NULL). Rows are written into a `map[elementId][attribute]` as they are read, so **last write
+   wins** and the ORDER BY *is* the precedence rule: **base < client-wide < persona < role**, with
+   `id ASC` breaking ties inside a tier (later-inserted row wins). Language is an orthogonal
+   overlay. `_loadOverrides` (Surface.php ~L740).
 4. **ACL composition, not absorption** — batch-resolve permissions via
    `_Model_Client_AclActionPermission`; drop elements whose `aclActionId` isn't permitted (ACL stays
    exactly as-is, see [acl-permission-chain](acl-permission-chain.md)).
@@ -446,23 +453,90 @@ first layer wrong and you get a **403 `EZ-1 AUTHORIZATION`** before any element
 1. **Route-level dispatch gate — `Client_<tenant>.AclRecordScripts`.** The surfaces dispatcher
    authorizes each API script (route + method + phpMethod) via the **CLIENT-tier**
    `AclRecordScripts` table (cols `uuid`, `recordScriptId` → `Core.RecordScripts`, `roleId` →
-   `Client_<tenant>.Roles`). The check is keyed on the JWT's **`id.client.roles`** (client roles),
-   **NOT** the platform `core.roles` and **NOT** the Core `AclRecordPermissions` chain.
+   `Client_<tenant>.Roles`) — **NOT** the Core `AclRecordPermissions` chain.
    `Core.RecordScripts` maps route/method/phpMethod → an ACL "record" (`recordId`). The three
    surfaces scripts: `meta` (GET, phpMethod `meta`), `meta-group` (GET, `metaGroup`), `debug` (GET,
-   `metaDebug`). `_Model_Core_Surface::resolve/metaGroup` read the caller's roles from
-   **`id.client.roles`** (Surface.php ~lines 56/87).
+   `metaDebug`).
+   **⚠ Which role-id space that `roleId` is matched against is decided by the RECORD, not by the
+   table it lives in** — see the correction below. `surfaces` is an `aclDatabase = CORE` record, so
+   the dispatch gate matches `AclRecordScripts.roleId` against the JWT's **`id.core.roles`**.
+   (Earlier revisions of this doc said `id.client.roles` here. That is right for a
+   `aclDatabase = CLIENT` record and **wrong** for `surfaces`.) Separately,
+   `_Model_Core_Surface::resolve/metaGroup` read the caller's roles from **`id.client.roles`**
+   (Surface.php:53 and :108) — so the two layers genuinely disagree on purpose.
 2. **Element-level ACL (runs *inside* `meta`/`metaGroup`, AFTER the route gate passes)** — dropping
    individual surface elements uses a **separate** client-tier table `_Model_Client_AclActionPermission`,
    gated by each `SurfaceElement.aclActionId`. This is the "composition, not absorption" step in
    `resolve()` step 4; it never causes the route-level 403.
 
 **Debugging a surfaces 403 EZ-1 (hard-won):** it is the route-level gate — a missing
-`Client_<tenant>.AclRecordScripts` grant resolved by **client role**. Do **not** chase
+`Client_<tenant>.AclRecordScripts` grant. Do **not** chase
 `Core.AclRecordPermissions`, `appId` scoping, or `AclLogicGroups`/`AclRecordExpressions` — all were
 investigated and ruled out as red herrings: `appId = NULL` is the normal working pattern, and the
 Core record grants + their logic-group expressions all evaluate `all`/`"1"`. The gate is purely
-"does this client role have an `AclRecordScripts` row for this `recordScriptId`?"
+"does one of the caller's roles have an `AclRecordScripts` row for this `recordScriptId`?" —
+and for `surfaces` those are **CORE** role ids (corrected 2026-08-28; earlier wording said client
+role, see below).
+
+### ⚠ The two gates read DIFFERENT role-id spaces — the classic time-sink (verified prod 2026-08-28)
+
+The single fact that makes a surfaces authorization question tractable: **`AclRecordScripts.roleId`
+and `SurfaceOverrides.roleId` are ids from two different tables, even though both rows sit in
+`Client_<tenant>`.**
+
+| Gate | Table (all in `Client_<tenant>`) | `roleId` means | JWT source |
+|---|---|---|---|
+| Route dispatch (403 or not) | `AclRecordScripts` | a **`Core.Roles`** id, because `surfaces.aclDatabase = CORE` | `id.core.roles` |
+| Element visibility cascade | `SurfaceOverrides` | a **`Client_<tenant>.Roles`** id | `id.client.roles` |
+
+Mechanism, verified in source:
+
+- `V2.php` (~L3090) picks the role space off the record:
+  `if ($record->aclDatabase == \_Model_Core_Record::ACL_DATABASE_CORE) { $roleIds = ...['id','core','roles']; } else { ...['id','client','roles']; }`
+  and hands that `$roleIds` straight to `getRecordScriptPhpMethod()` (~L6198), which does
+  `SELECT id FROM AclRecordScripts WHERE recordScriptId = <n> AND roleId IN (<$roleIds>)`.
+- api2 derives `id.core.roles` at token time (~L1611-1627):
+  `SELECT DISTINCT coreRoleId FROM Client_<tenant>.Roles WHERE id IN (<client roles>) AND coreRoleId IS NOT NULL`.
+  Most tenant roles have `coreRoleId = NULL`, so the **core** role set is tiny — typically just
+  `1` (Base), inherited from whichever client role maps to it.
+- `_Model_Core_Surface::resolve` / `metaDebug` read `['id','client','roles']` (Surface.php:53, :108),
+  so the cascade never sees a core role id.
+
+**Practical consequence:** the dispatch gate is almost never the blocker for a normal user. Any user
+holding a client role that maps to `coreRoleId = 1` passes `meta`/`meta-group`, so an
+`AclRecordScripts` row for core role 1 covers the whole tenant. Worked example — `Client_Compass`
+has `AclRecordScripts` `(28,1) (28,3) (28,4) (29,3) (30,1) (30,3) (30,4)` and only roles 1 (Base)
+and 10 (Public) carry `coreRoleId = 1`, yet **every** Compass user passed the gate, because every
+Compass user holds Base. When a screen comes back empty, go to the `SurfaceOverrides` cascade
+(**client** role ids) — not to `AclRecordScripts`.
+
+### Prod ids for the `surfaces` record and its scripts — the toga25-supply guide lists DEV-SANDBOX ids
+
+`toga25-supply/surface-layer-guide.md` documents `Core.Records` **2300** and `Core.RecordScripts`
+**25 / 26 / 27**. Those are **dev-sandbox** values. **Do not reuse them in a prod migration or a
+prod query.** Verified against prod Core on 2026-08-28:
+
+| Object | Prod id | Detail |
+|---|---|---|
+| `Core.Records` `surfaces` | **333** | model `\_Model_Core_Surface`, route `surfaces`, `aclDatabase = CORE` |
+| `Core.RecordScripts` `meta` | **28** | GET → phpMethod `resolve` |
+| `Core.RecordScripts` `debug` | **29** | GET → phpMethod `metaDebug` |
+| `Core.RecordScripts` `meta-group` | **30** | GET → phpMethod `metaGroup` |
+
+Same discipline as the reserved-id blocks: **read the id out of the target environment before you
+write the file**; a surface-layer id that is correct on dev-sandbox is a silent mis-target on prod.
+
+### The element `aclActionId` gate is dead across the WHOLE app, not just navigation
+
+`resolve()` step 4 only ever drops an element that **declares** an `aclActionId`
+(`if ($element->aclActionId && !isset($permittedAclActionIds[(int)$element->aclActionId])) continue;`
+— Surface.php ~L592, mirrored in `metaDebug`'s `aclDropped` at ~L169). Verified prod-wide on
+2026-08-28: **no `SurfaceElement` on `appId = 1` sets `aclActionId` at all.** So the
+`AclActionPermissions` layer is currently inert for every surface in the 2.5 app — the nav surface
+is just the case people trip over. Element visibility in practice = the `SurfaceOverrides` cascade,
+full stop. See
+[acl-permission-chain → Navigation / action flags](acl-permission-chain.md) for why the
+`navigation-*` `AclActions` rows still exist and still matter to the *older* `Page::meta` apps.
 
 ## Gotchas
 
@@ -624,6 +698,22 @@ Core record grants + their logic-group expressions all evaluate `all`/`"1"`. The
   match Compass, a follow-up migration aligning both `meta` and `meta-group` to roles 1,3,4 is needed.
 
 ## Change history
+- 2026-08-28 — **Corrected the route-level gate's role-id space.** The dispatch check is keyed on
+  the role space that `Core.Records.aclDatabase` selects, not always `id.client.roles`: `surfaces`
+  is a **CORE** record, so `V2.php` (~L3090) takes `$roleIds` from **`id.core.roles`** and
+  `getRecordScriptPhpMethod` (~L6198) matches those **core** ids against
+  `Client_<tenant>.AclRecordScripts.roleId`, while the `SurfaceOverrides` cascade uses
+  **`id.client.roles`** (Surface.php:53/:108) — two different id spaces in two tables that both
+  live in the client DB. Recorded how api2 derives `id.core.roles`
+  (`SELECT DISTINCT coreRoleId FROM Roles WHERE id IN (<client roles>) AND coreRoleId IS NOT NULL`,
+  ~L1611-1627), which is why any Base-holding user passes the gate and the gate is rarely the
+  blocker. Added the **prod** ids for the `surfaces` record (`Core.Records` **333**) and its scripts
+  (**28** meta→resolve, **29** debug→metaDebug, **30** meta-group→metaGroup) and flagged that
+  `toga25-supply/surface-layer-guide.md` documents dev-sandbox values (record 2300, scripts 25/26/27)
+  that must not be reused on prod. Widened the `aclActionId` finding: **no** `SurfaceElement` on
+  `appId = 1` declares an `aclActionId`, so the element ACL gate is inert app-wide, not just for
+  navigation. Made the cascade precedence exact (`ORDER BY … , id ASC` + last-write-wins map).
+  All verified read-only against prod. (bala)
 - 2026-08-27 — Corrected the reserved-id framing: the blocks record the 2026-08-21 reseed only and are
   **not** an inventory of all surface ids. Pre-reseed surfaces kept AUTO_INCREMENT ids — the
   `navigation` TAB_STRIP is `SurfaceElements` **109–114** — so an id outside 125–146 is not evidence it