toga-ai 1.0.425 → 1.0.427

package/knowledge/1.0/apps/worker/features/forecast2-netsuite-reconciliation.md

@@ -6,7 +6,7 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-07-21
+updated: 2026-07-23
 owners: [dfranks, jcardinal]
 files:
   - test/@dave/checker.php
@@ -303,6 +303,36 @@ by reconciling a chosen tranDate range directly against NetSuite.
   the sale-import doc). The salesRep dimension itself is the JE line custom column `custcol_sales_rep_line`
   (wired under TRUE-79862).
 
+- **Delete-safety is a targeted BY-ID RE-FETCH from NetSuite, NOT a `deletedrecord` system-log
+  lookup — mirror this in any new reconciler.** The fixer never blind-deletes a Forecast row just
+  because a windowed pull didn't return it. Its delete path (`fixer.php` Sales path ~773-891, and
+  `checker.php`) works by **re-fetching each discrepant transaction by its internal id** from
+  NetSuite via the local `fetchSalesBulk()` shim (`WHERE transaction IN (...)` SuiteQL, same shape
+  as the legacy `listSales()`): it upserts the lines NetSuite returns for that id and **deletes only
+  the FC lines NetSuite no longer returns for that specific id**. If NS returns **nothing** for the
+  id, the txn is gone → its FC rows are deleted. This by-id re-fetch is the concrete implementation
+  of the team's **"no blind auto-delete"** rule — it guards against a transient window/read miss
+  masquerading as a deletion (a `lastmodifieddate` window or a paged full-window scan can transiently
+  omit a row; a per-id GET/IN-list re-confirms it). A NetSuite **`deletedrecord` system-log gate is
+  NOT existing behavior** and should not be invented as the delete confirmation — the authoritative
+  signal is "the by-id re-fetch returns no lines."
+- **Why an independent tranDate-range reconciliation is MANDATORY, not just a nicety — NetSuite event
+  capture cannot be guaranteed by any configuration.** Beyond the sublist line-field inline-edit blind
+  spot (below), a broader class of NetSuite changes fires **no** User Event SuiteScript at all, so the
+  AMQ enqueuer never runs, no webhook is posted, and `lastmodifieddate` is frequently **not** bumped:
+  - **Bulk / mass updates and CSV imports** — a CSV import only runs server SuiteScript (and workflows)
+    when **"Run Server SuiteScript and Trigger Workflows"** is enabled, which is **OFF by default**.
+  - **Changes made BY another script or workflow** — a UserEvent script **cannot be triggered by
+    another UserEvent script or a workflow**, so a server-side edit never re-fires the enqueuer UE.
+
+  Consequence: such edits drift `Forecast.Sales` (and any webhook-synced NetSuite data) **silently and
+  invisibly to BOTH** the real-time webhook path **and** a `lastmodifieddate`-windowed pull cron —
+  strictly broader than the already-known inline sublist line-field case. Durable implication: because
+  event capture is not guaranteeable, **correctness requires an independent reconciliation that reads
+  NetSuite by `tranDate` range and matches on internal id** (exactly what `checker`/`fixer` do) — this
+  is the load-bearing reason the reconciliation backstop exists, not an optimization. (Oracle docs:
+  server scripting on CSV import `section_4676525683`; how UE scripts are executed `section_1512409310`.)
+
 ## Data model
 
 `Forecast.Sales`, `Forecast.OpenOrderItems` on the **core2** cluster
@@ -572,6 +602,19 @@ None — Forecast2 is a single shared dataset.
 
 ## Change history
 
+- 2026-07-23 — **Recorded WHY the reconciliation backstop is mandatory + the by-id re-fetch
+  delete-safety mechanism** (TRUE-80262 planning; no code shipped, dfranks). Documented the broader
+  NetSuite event-capture blind spot beyond the inline sublist edit: **bulk/mass updates and CSV
+  imports fire no UE** (CSV runs server SuiteScript only when "Run Server SuiteScript and Trigger
+  Workflows" is on — OFF by default), and a **UE cannot be triggered by another UE/workflow**, so
+  script/workflow-driven edits drift `Forecast.Sales` invisibly to BOTH the webhook path and a
+  `lastmodifieddate`-windowed cron (and often don't bump `lastmodifieddate`). Durable implication:
+  event capture can't be guaranteed by any NetSuite config, so an independent `tranDate`-range
+  reconciliation matched on internal id is required for correctness (Oracle docs `section_4676525683`
+  / `section_1512409310`). Also documented the fixer/checker **delete-safety = targeted by-id re-fetch**
+  (`fetchSalesBulk` shim; delete FC lines NS no longer returns for that id) as the concrete "no blind
+  auto-delete" implementation — a `deletedrecord` system-log gate is NOT existing behavior and must not
+  be invented by a future reconciler. (dfranks)
 - 2026-07-21 — **Documented `reconcile_drift_2023plus.php` + the SOAP-era double-line bug (2023)**
   (folded in from a retired project-local CLAUDE.md). The SOAP-era importer ran twice for some
   invoice batches and inserted each invoice's lines twice under consecutive-but-different line

package/knowledge/2.0/apps/_underscore/features/forecast-sale-import.md

@@ -6,7 +6,7 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-07-14
+updated: 2026-07-23
 owners: [dfranks]
 files:
   - worker2/Component/Forecast/SaleImport/SaleImport.php
@@ -360,6 +360,21 @@ evidence: two invoices whose ONLY change that day was a cost line-edit had **zer
 entries, while 8 other cost-edited invoices that also had normal create/edit activity **did** fire
 and synced. This is recurring/multi-user (~12/day; 35 `MCOSTESTIMATE` edits in 3 days by 9 users).
 
+**Broader than the inline line-edit — a whole CLASS of NetSuite changes fires no UserEvent, so no
+webhook is posted (and `lastmodifieddate` is often not bumped):**
+- **Bulk imports / mass updates** and **CSV imports** — a CSV import runs server SuiteScript (and
+  workflows) **only** when **"Run Server SuiteScript and Trigger Workflows"** is enabled, which is
+  **OFF by default**; so the AMQ enqueuer UE never runs for a bulk/CSV load.
+- **Changes made BY another script or workflow** — a **UserEvent script cannot be triggered by
+  another UserEvent script or by a workflow**, so a server-side edit never re-fires the enqueuer.
+
+So event capture **cannot be guaranteed by any NetSuite configuration** — these edits drift
+`Forecast.Sales` (and any webhook-synced NetSuite data) **silently and invisibly to BOTH** the
+real-time webhook **and** a `lastmodifieddate`-windowed pull cron. This is the durable reason the
+independent tranDate-range reconciliation (`checker`/`fixer`, see the reconciliation doc) is
+**mandatory, not optional**. (Oracle docs: CSV server scripting `section_4676525683`; how UE scripts
+are executed `section_1512409310`.)
+
 **Corollary gotcha: `lastmodifieddate` is NOT a reliable change signal for line-cost edits** — a
 line-field inline edit leaves it untouched, so **both** the webhook pipeline **and** any
 `lastmodifieddate`-windowed `list*()`/cron sync miss it entirely. **Only reconciliation**
@@ -574,6 +589,14 @@ success from a `Forecast.Sales` row alone.
 - The cron's sign handling is not portable here — see Sign convention.
 
 ## Change history
+- 2026-07-23 — **Broadened the AMQ event-capture blind spot beyond inline line-edits** (TRUE-80262
+  planning; no code shipped, dfranks). Recorded that **bulk/mass updates and CSV imports** fire no UE
+  (CSV runs server SuiteScript only when "Run Server SuiteScript and Trigger Workflows" is on — OFF by
+  default) and that a **UE cannot be triggered by another UE or a workflow** — so script/workflow/bulk
+  edits post no webhook and often don't bump `lastmodifieddate`, drifting `Forecast.Sales` invisibly to
+  both the webhook path and a `lastmodifieddate`-windowed cron. Durable implication: NetSuite event
+  capture is not guaranteeable by any config, making the independent tranDate-range reconciliation
+  mandatory (Oracle docs `section_4676525683` / `section_1512409310`). (dfranks)
 - 2026-07-14 — **JE lines now resolve `customerId` (was hard-coded null) + corrected the
   engine's repo location to worker2** (TRUE-80129, dfranks). `buildJournalEntryRows` reads
   `JE_LINE_CUSTOMER_FIELD = 'entity'` — a **native** JE-line reference field (not a `custcol_*`,

package/knowledge/2.0/apps/_underscore/features/per-client-database-connections.md

@@ -7,7 +7,7 @@ client: shared
 type: feature
 status: active
 updated: 2026-07-23
-owners: ["dfranks", "jcardinal", "mhammontree", "apeterson"]
+owners: ["dfranks", "jcardinal", "mhammontree", "apeterson", "kyalamarthi"]
 files:
   - _underscore/Database.php
   - _underscore/ApiRequest.php
@@ -75,6 +75,22 @@ per-client), region-aware (1=us-east-1, 2=us-west-2, 3=eu-west-1) with per-regio
 registration is region-aware in `api2/Controller/Index.php`; the alias const is in `api2/_.php`.
 `Records.ttlCache` (TINYINT UNSIGNED, default 15) governs cache TTL.
 
+### The `Team` schema rides the Core cluster (`DB_TEAM`)
+
+The **`Team` schema** (tables `Tasks`, `Sprints`; models `_Model_Team_Task` / `_Model_Team_Sprint`
+with `const DATABASE = _underscore::DB_TEAM`) is **not a first-class api2 database** — it is not a
+value of `Core.Records.aclDatabase` (that enum is only `'CORE'` / `'CLIENT'`). It physically
+resolves to the **core cluster**: `DB_TEAM` **reads** route to `reader1.core.database.togahub.com`
+and **writes** to `writer.core.database.togahub.com`. So a `DB_TEAM` SELECT auto-routes to the core
+reader, and a Record backed by the `Team` schema must be registered with **`aclDatabase = 'CORE'`**
+(see [Record Scripts](../../api2/features/record-scripts.md)).
+
+**Where DB hosts come from — the split:** Core and other **non-client** cluster hosts (Core, the
+`Team`-schema-bearing core cluster, etc.) come from the api2 `Config/*.ini` **`[database]`
+section**, not from `DatabaseHosts`. **Per-client** hosts come from the `Core.DatabaseHosts` table,
+resolved by `_Database::registerClientDatabases()`. (Credential/host *values* are not reproduced
+here — they live in `Config/*.ini`.)
+
 ## Gotchas / known issues
 
 - **The "logs DB write trap" (local dev).** A laptop usually imports only `Client_<Id>`, not
@@ -108,6 +124,13 @@ registration is region-aware in `api2/Controller/Index.php`; the alias const is
 
 ## Change history
 
+- 2026-07-23 — Documented that the **`Team` schema** (`Tasks`/`Sprints`, `_Model_Team_*`,
+  `DB_TEAM`) is not a first-class api2 DB (not in `Records.aclDatabase`, which is CORE/CLIENT
+  only) and physically resolves to the **core cluster** (reads → `reader1.core…`, writes →
+  `writer.core…`), so `DB_TEAM` SELECTs auto-route to the core reader and a `Team`-backed Record
+  registers as `aclDatabase = 'CORE'`. Also recorded the host-resolution split: Core/non-client
+  hosts come from api2 `Config/*.ini` `[database]`; per-client hosts come from `DatabaseHosts`
+  via `registerClientDatabases()`. (kyalamarthi)
 - 2026-07-23 — Documented the shared **Core Logs** (`DB_LOGS`) analogue of the logs trap: its schema
   name is resolved from a `Core.Database` row (`id = CORE_LOGS_DATABASE_ID`) in
   `api2/Controller/Index.php::api()`, so a name-mismatched local Logs DB still reads as missing; and

package/knowledge/2.0/apps/api2/INDEX.md

@@ -7,6 +7,7 @@
 | [Encrypted-User-UUID Auth Handoff (/auth/encrypted-user-uuid)](features/encrypted-user-uuid-auth-handoff.md) | `POST /auth/encrypted-user-uuid` is the intended **cross-client / SSO-handoff identity mechanism**: given an encrypted `{client, user}` UUID pair, it mints a fr | api2/Component/Api/CrossClient/CrossClient.php |
 | [Language Translation Layer (audience.language + sidecar tables)](features/language-translation-layer.md) | Serves the same TOGa data (Item title/description/longDescription, plus item **feature** text — `Features.name`, `ItemCategoryFeatureGroups.name`, `ItemFeatures | api2/Component/Api/V2/V2.php, api2/Component/Api/V2/Response/Response.php, _underscore/Model/Core/Setting.php, _underscore/Model/Core/RecordField.php, _underscore/Model/Core/DefaultGlobalSetting.php, _underscore/Model/Client/ItemTranslation.php, _underscore/Model/Client/FeatureTranslation.php, _underscore/Model/Client/ItemCategoryFeatureGroupTranslation.php, _underscore/Model/Client/ItemFeatureTranslation.php, dbchanges2/Client/2026-06-23a - ItemTranslations.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql, dbchanges2/Client/2026-07-13a - FeatureTranslations.sql, dbchanges2/Client/2026-07-13b - FeatureTranslationsAcl.sql, dbchanges2/Core/2026-06-23a - RecordFieldsTranslationColumn.sql, dbchanges2/Core/2026-06-23b - ItemTranslationsRecord.sql, dbchanges2/Core/2026-07-13 - FeatureTranslationsRecord.sql |
 | [Nested-relationship writes & child matching (link vs. create)](features/nested-relationship-writes.md) | When a 2.0 API write payload (`POST`/`PUT`) contains a **nested related object** (e.g. | api2/Component/Api/V2/V2.php |
+| [Record Scripts (computed/aggregate /v2 endpoints — the authoring contract)](features/record-scripts.md) | In api2 you almost never write a controller. | api2/Component/Api/V2/V2.php, _underscore/Model/Team/Sprint.php |
 | [POST + JSON-body args for scripted APIs](features/scripted-api-post-body-args.md) | The V2 engine can run a Record Script (scripted API) for a **POST** request, and a scripted API can receive its arguments from the **JSON request body** instead | api2/Component/Api/V2/V2.php |
 | [Surface action-state via the surface=<slug> request option (M2M-safe)](features/surface-meta-option.md) | An opt-in V2 engine request option, `surface=<slug>`, that attaches per-record UI action state (`isVisible`/`isEnabled`) to a GET response **under `meta.surface | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Surface.php |
 | [TableView row-filtering via apiWhereClause (options.where grammar, end to end)](features/tableview-apiwhereclause-row-filtering.md) | `TableViews.apiWhereClause` (TEXT, nullable) is the sanctioned, code-free way to restrict or exclude rows from a 2.0 table view. | api2/Component/Api/V2/V2.php, _underscore/Model/Client/TableView.php, toga2-supply/src/api/toga.ts |

package/knowledge/2.0/apps/api2/features/record-scripts.md

@@ -0,0 +1,160 @@
+---
+title: Record Scripts (computed/aggregate /v2 endpoints — the authoring contract)
+framework: "2.0"
+repo: api2
+project: API
+client: shared
+type: feature
+status: active
+updated: 2026-07-23
+owners: ["kyalamarthi"]
+files:
+  - api2/Component/Api/V2/V2.php
+  - _underscore/Model/Team/Sprint.php
+related:
+  - ./scripted-api-post-body-args.md
+  - ./tableview-apiwhereclause-row-filtering.md
+  - ../architecture.md
+  - ../../_underscore/features/per-client-database-connections.md
+  - ../../worker2/features/team-sprint-management.md
+---
+
+## Summary
+
+In api2 you almost never write a controller. The generic **/v2 engine** serves models from
+metadata (Records / RecordFields / ACL rows), so ordinary CRUD is a *data* change, not code.
+For a payload the metadata CRUD path cannot express — a **computed value, an aggregate, or a
+purpose-built dashboard tile** — the mechanism is a **Record Script**: a `public static`
+method on the record's `_Model` whose return value becomes the `/v2` envelope `data`. This is
+the company-standard way to add a read-only, non-CRUD endpoint.
+
+This doc is the **authoring contract** for a Record Script (the general pattern and how to
+register one). For the POST + JSON-request-body variant (large argument values), see
+[POST + JSON-body args for scripted APIs](./scripted-api-post-body-args.md).
+
+Motivating example: `_Model_Team_Sprint::committedTile()` — a read-only dashboard tile over
+the internal TOGa IQ sprint (`Team` schema) data. **Decision recorded:** internal
+sprint/dashboard read endpoints are served from **api2 via a Record Script**, not by building
+a bespoke controller or standing the data up inside the talos / TOGa IQ app.
+
+## The method contract
+
+A Record Script is a static method on the record's model — for the example above, the
+signature is `public static function committedTile(&$api, int|string $sprint): object` on
+`_underscore/Model/Team/Sprint.php`. Its return value becomes the `/v2` envelope `data`.
+
+Rules the engine enforces (see `getRecordScriptPhpMethod()` + the dispatch in
+`processRoutePairs()` inside `api2/Component/Api/V2/V2.php`):
+
+- **First parameter is `&$api`** — the engine always sets `$args['api'] = $this` last, so the
+  method receives the live V2 engine instance (for auth context, `internalApiRequest()`, etc.).
+- **Remaining parameters are named** and are filled from the **query string**. Each query-string
+  key maps to the same-named PHP parameter; unmatched named args land in a `...$args` variadic
+  if the method declares one. (`transactionId` and `api` keys are reserved and never mapped.)
+- **The return value is the envelope `data`.** Return an object/array; do not echo, and do not
+  return the bare success/error envelope — the engine wraps it.
+- Invocation is effectively `$model::$phpMethod(...$args)`.
+
+## Running SQL inside a Record Script
+
+Record Scripts commonly run their own query (via `new \_Query($sql, \_underscore::DB_TEAM)`
+then `->fetchRow()`) rather than going through model CRUD. Two framework realities to respect:
+
+- **`_Query` has no bound-parameter support.** There is no `?`/`:name` binding. Every value
+  you place into the query text must be sanitized by hand **before** it is assembled into the
+  SQL: pass strings through `\_Database::escape()` and cast numerics with `(int)`. Record-script
+  arguments arrive straight from the query string (user input), so this sanitization is
+  mandatory — an unsanitized value in the query text is a SQL-injection hole. Treat
+  `\_Database::escape()` + `(int)` casting as the required substitute for the prepared
+  statements `_Query` does not offer.
+- **Query cache for "live" tiles.** A dashboard tile that must reflect current data should call
+  `\_Database::useQueryCache(false)` before the query and **restore the previous flag**
+  afterward (capture the value it returns and set it back), so it doesn't serve stale cached
+  rows and doesn't leave caching globally disabled for the rest of the request.
+- `DB_TEAM` SELECTs auto-route to the **core cluster reader** — see
+  [per-client database connections](../../_underscore/features/per-client-database-connections.md)
+  for why the `Team` schema resolves to core.
+
+## Registering / enabling an endpoint (metadata across two databases)
+
+Registering a Record Script is a **metadata change split across two databases plus a
+per-client ACL grant** — it is not a code deploy. The route is
+**`GET /v2/<recordRoute>/<scriptRoute>`**.
+
+**Data model:**
+
+- **`Core.Records`** (the route ↔ model row). Its **`aclDatabase` enum is only `'CORE'` or
+  `'CLIENT'`** — there is no `TEAM` value, so a `Team`-schema-backed record registers as
+  **`aclDatabase = 'CORE'`** (the `Team` schema physically lives on the core cluster).
+- **`Core.RecordScripts`** — `uuid`, `recordId` (FK → `Core.Records`), `method` (e.g. `GET`),
+  `route` (the script route), `phpMethod` (the static method name). **`UNIQUE(recordId, method,
+  route)`.**
+- **`Client.AclRecordScripts`** — model `_Model_Client_AclRecordScript`, on `DB_CLIENT`:
+  `recordScriptId` (FK → `Core.RecordScripts`) + `roleId` (FK → `Client.Roles`). This is the
+  ACL grant, and it lives **per client** — so **record-script authorization is per-client**:
+  granting a role in one client does not grant it in another.
+
+At dispatch the engine calls `getRecordScriptPhpMethod(record, httpMethod, route, roleIds)`,
+which joins `Core.RecordScripts` to the caller's `Client.AclRecordScripts` rows for their
+roles; no matching grant → the script does not run for that caller.
+
+**Where the rows go:** seed them in **dbchanges2** as dated migration files — the
+`Core.Records`/`Core.RecordScripts` rows under the `Core` folder, and the `AclRecordScripts`
+grant under the per-client `Client_<Name>` folder. See the
+[api2 architecture — metadata CRUD engine](../architecture.md) for the surrounding Records
+metadata model.
+
+## Data source note (core DB hosts/credentials)
+
+The core cluster hosts and credentials this endpoint reads through are **not** documented here;
+they live in **api2 `Config/*.ini`** (`[database]` section). Do not paste those values into
+knowledge or code.
+
+## Client variations
+
+None — this is engine behavior. Per-client access is entirely controlled by which roles have
+an `AclRecordScripts` grant in each client DB.
+
+## Gotchas / known issues
+
+- **No bound params in `_Query`.** Sanitize with `\_Database::escape()` (strings) and `(int)`
+  casts (numerics) for every value placed into the query text — query-string args are user
+  input.
+- **Restore the query-cache flag.** `useQueryCache(false)` is a global toggle; capture the
+  prior value and restore it, or you disable caching for the rest of the request.
+- **`aclDatabase` has no `TEAM`.** A `Team`-schema record registers as `'CORE'`; picking
+  `'CLIENT'` sends the engine looking in the wrong cluster.
+- **Registration is per-environment and split.** A missing `Core.RecordScripts` row → the route
+  isn't a script (falls through to normal CRUD / 404-shaped behavior); a missing
+  `AclRecordScripts` grant → the caller's roles can't run it. Both rows must exist in every
+  environment the API serves.
+- **The committed-tile definition is intentionally not the sprint-scoring definition.**
+  `committedTile()` uses the **Power BI / current-state** definition of done/committed
+  (`dtDone IS NOT NULL` + `workTypeNow`), which is *deliberately different* from TOGa IQ's
+  canonical sprint scoring (`statusNow IN (STATUS_IN__DONE)` + `workTypeAtLock`). See
+  [Team Sprint Management](../../worker2/features/team-sprint-management.md) before assuming a
+  tile agrees with sprint scores.
+
+## Change history
+
+- 2026-07-23 — Initial documentation of the Record Script authoring contract (static
+  `fn(&$api, ...namedParams)` on the record model; query-string → named args; return → `/v2`
+  envelope `data`), the SQL specifics (`_Query` has no bound params → `_Database::escape()` +
+  int-cast; `useQueryCache(false)` + restore for live tiles), and the registration metadata
+  (`Core.Records` `aclDatabase` CORE/CLIENT only, `Core.RecordScripts` UNIQUE(recordId,method,
+  route), per-client `Client.AclRecordScripts` grant). Motivated by
+  `_Model_Team_Sprint::committedTile()`, a read-only internal sprint dashboard tile — recording
+  the decision that internal sprint/dashboard reads are served from api2 via a Record Script
+  rather than the talos / TOGa IQ app. (kyalamarthi)
+
+## Related docs
+
+- [POST + JSON-body args for scripted APIs](./scripted-api-post-body-args.md) — the POST /
+  JSON-request-body variant for large argument values.
+- [api2 architecture](../architecture.md) — the metadata-driven /v2 CRUD engine that Record
+  Scripts extend.
+- [Per-Client Database Connections](../../_underscore/features/per-client-database-connections.md)
+  — why `DB_TEAM` resolves to the core cluster.
+- [Team Sprint Management & Reporting](../../worker2/features/team-sprint-management.md) — the
+  canonical sprint scoring definition the committed tile deliberately diverges from.
+</content>

package/knowledge/2.0/apps/worker2/INDEX.md

@@ -28,7 +28,7 @@
 | [Talos (TOGa IQ) Meeting-Notes Integration & Token Auto-Refresh (consumer)](features/talos-meeting-notes-integration.md) | How a **dev tool / agent consumes Talos (TOGa IQ)** to query the team meeting-notes corpus programmatically. | .claude/skills/plan-ticket/scripts/talos.js |
 | [Talos Pricing Automation (worker2 Cron — AWS Actuals, Calibration, Monthly Report)](features/talos-pricing-automation.md) | The worker2 half of the **Talos Pricing Platform** (see the talos `pricing-cogs-model` and tools `talos-pricing-ui` docs for the other halves). | worker2/Worker/Talos/Pricing.php, worker2/Database/TalosPricingCrons.sql |
 | [Talos Transcript Ingestion Pipeline (worker2 → AWS Bedrock KBs)](features/talos-transcript-ingestion.md) | `_Worker_Team_Transcripts` runs a fully automated, cron-driven pipeline that ingests raw Teams transcripts into the **Talos / TOGa IQ** AWS Bedrock knowledge ba | worker2/Worker/Team/Transcripts.php, worker2/bin/sync-knowledge-bases.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Team/2026-06-30a, dbchanges2/Team/2026-06-30b, dbchanges2/Team/2026-06-30c, dbchanges2/Team/2026-06-30d, dbchanges2/Team/2026-06-30e, dbchanges2/Core/2026-06-30a, dbchanges2/Core/2026-07-02a, dbchanges2/Team/2026-07-02a, dbchanges2/Team/2026-07-08a, dbchanges2/Team/2026-07-09a, dbchanges2/Team/2026-07-10a |
-| [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php, dbchanges2/Core/CronJobs (SprintLockScheduled seed) |
+| [Team Sprint Management & Reporting](features/team-sprint-management.md) | `_Worker_Team_Sprint` (file `Worker/Team/Sprint.php`) is the engine behind TOGA's internal **development-sprint process and reporting**. | worker2/Worker/Team/Sprint.php, _underscore/Model/Team/Sprint.php, dbchanges2/Core/CronJobs (SprintLockScheduled seed) |
 | [Teams Meeting Transcript Export](features/teams-transcript-export.md) | > **SUPERSEDED (2026-07-09) — the S3-staging model below is history.** `Export` is now a thin > **GRAPH-DIRECT** cron poller: it no longer archives raw VTT to ` | worker2/Worker/Team/Transcripts.php, worker2/Config/production.ini, worker2/Database/TeamsTranscriptExports.sql, dbchanges2/Core/2026-06-18a - Teams Transcript Export schedule.sql |
 | [VAPI Webhook Handler (worker2 — AI-BDR end-of-call processing)](features/vapi-webhook-handler.md) | `_Worker_Vapi` ([worker2/Worker/Vapi.php](worker2/Worker/Vapi.php)) is the **PHP side of the AI-BDR call loop** — the webhook that receives VAPI's end-of-call r | worker2/Worker/Vapi.php, worker2/Worker/Ai/Bdr/Vapi.php |
 | [WJE Freshservice Sync (worker2)](features/wje-freshservice-sync.md) | WJE ("WJE IT", helpdesk `wje.freshservice.com`) is a **Freshservice**-based help-desk client whose tickets, contacts, assets, groups, categories, and canned res | worker2/Worker/Wje.php, _underscore/Component/Api/Wje/Wje.php, _underscore/Model/Wje/Ticket.php, _underscore/Model/Wje/TicketNote.php, _underscore/Model/Wje/Contact.php, _underscore/Model/Wje/Unit.php, _underscore/Model/Wje/TicketTeam.php, _underscore/Model/Wje/TicketCategory.php, _underscore/Model/Wje/AssetType.php, _underscore/Model/Wje/PredefinedReply.php, library/app/api/wje.php, worker/crons/toga2/wje/import_supporting_records.php, worker/crons/toga2/wje/sync_togasupply_wje.php, worker/crons/notifications/reports/wje/wje_common.php, library/app/systemmonitor/wje.php, dbchanges2/Client_Wje/2024-10-04 - WjeOnboarding.sql |

package/knowledge/2.0/apps/worker2/features/monitoring-framework.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-06-29
-owners: [mhammontree]
+updated: 2026-07-23
+owners: [mhammontree, dfranks]
 files:
   - worker2/Worker/Monitor.php
   - worker2/Worker/Monitors/
@@ -54,6 +54,35 @@ orchestrator) · runtime config in the `Core.Monitors` table (no redeploy).
 Reused with **no changes**: `Core.CronJobs`, `Core.WorkerJobs`, the `WorkerCronScheduler`
 Lambda, the EB worker tier + SQS delivery (see [worker2 architecture](../architecture.md)).
 
+## Where monitor / data-quality results belong (design placement)
+
+worker2 has **two** monitoring patterns and neither writes to the central **`Logs` Issues/Events**
+tables. That is deliberate — **`Logs.Issues`/`Logs.Events` are strictly for application
+errors/exceptions** that escalate to ClickUp or email (business-routed by `IssueEmailAddresses`
+presence; see the escalation-cron work). **Periodic health-check, integration-health, and
+data-quality RESULTS do NOT go there** — they belong in the Monitor framework:
+
+- **Pattern A — internal DB-driven state machine** (this doc): a `Core.Monitors` row +
+  `_Worker_Monitor::Run(monitorId)` (`worker2/Worker/Monitor.php`) invokes a child `phpClass::Run()`
+  that returns `{isOk, message}`; the orchestrator manages ok/alert state, `consecutiveOkCount` /
+  `requiredConsecutiveOks` anti-flap, and sends alert/reminder/recovery mail via
+  `_Worker_Notification_Email::Send`, persisting state back to `Monitors`.
+- **Pattern B — "dumb reporter, smart monitor"** (see
+  [OneUptime push-metric monitors](./oneuptime-worker2-monitoring.md)): a child measures one metric
+  and POSTs a JSON body to a OneUptime incoming-request (heartbeat) monitor; the **threshold/alerting
+  lives in OneUptime** (tunable without a deploy). This is the right home for integration-health /
+  data-quality checks.
+
+So when deciding where a recurring check/reconciliation **result** goes, the answer is the Monitor
+framework (Pattern B / OneUptime for integration-health and data-quality), **not** Issues/Events.
+Key Pattern-B idioms (durable): the worker decides the threshold and emits a **string token**
+OneUptime matches (`"alarm":"HIGH"` → offline/incident, because OneUptime can only string-match a
+pushed body, not compare numbers); on a data-source failure POST an explicit **`status:error`** so
+OneUptime distinguishes "metric high" from "checker is blind"; the `_ApiRequest` push uses
+`setLogging(false)` + `setThrowExceptionsOnFailure(false)` so a failed ping never fails the job or
+depends on the Logs DB; the heartbeat URL is a **push credential — never log it**; and the
+human-readable result is the return string persisted in `WorkerJobs.output`.
+
 ## How it works
 
 One `Core.CronJobs` row per monitor (`action = 'Monitor/Run'`, `parameters =
@@ -237,6 +266,14 @@ clients' data flows (Compass, Prudential, AIG, Rate, …) but live as separate c
   HTML email + dashboard deep-links · anti-flap on the alarm side.
 
 ## Change history
+- 2026-07-23 — **Recorded the design-placement decision: monitor / data-quality RESULTS belong in the
+  Monitor framework (Pattern B / OneUptime for integration-health & data-quality), NOT in the central
+  `Logs` Issues/Events tables** (which are strictly for escalating application errors/exceptions).
+  Summarized both patterns side by side and the durable Pattern-B idioms (worker decides the threshold
+  and emits a string token OneUptime matches; explicit `status:error` on data-source failure; the push
+  is `setLogging(false)`+`setThrowExceptionsOnFailure(false)` so a failed ping never fails the job; the
+  heartbeat URL is a push credential — never logged; result string persisted in `WorkerJobs.output`).
+  TRUE-80262 planning; no code shipped. (dfranks)
 - 2026-06-29 — Built the first child monitor, `_Worker_Monitors_RateEntitlement` (TRUE-79129) + change-set `2026-06-29a`; added the log-scan worked example, the `_underscore` outbound-API-log detection pattern, and confirmed child details (orchestrator never calls `initialize()` so children self-register non-Core connections; no commit on read-only `Run()`; heredoc cannot interpolate `self::CONST`). Re-flagged that `Core.Monitors` is still local-only — the new change-set hard-fails where the table is absent. (mhammontree)
 - 2026-06-10 — Documented the v1.0 monitoring framework (orchestrator, `Core.Monitors` table, recovery-side anti-flap state machine, child contract). First child monitor + staging/prod migration still pending. (mhammontree)
 

package/knowledge/2.0/apps/worker2/features/team-sprint-management.md

@@ -6,10 +6,11 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-07-21
-owners: ["jcardinal"]
+updated: 2026-07-23
+owners: ["jcardinal", "kyalamarthi"]
 files:
   - worker2/Worker/Team/Sprint.php
+  - _underscore/Model/Team/Sprint.php
   - dbchanges2/Core/CronJobs (SprintLockScheduled seed)
 related:
   - ../architecture.md
@@ -228,8 +229,34 @@ backfills `workTypeAtLock` the first time a task is seen).
   Do **not** overload one map for both jobs. This pattern is replicated identically in
   `SprintLock()`, `CaptureSprintEnd()`, and `CaptureSprintDaily()` — fix all three together.
 
+## Two coexisting "committed / done" definitions (reconcile before comparing)
+
+There are **two intentionally-different definitions** of committed/done for a sprint task, and
+they do not agree:
+
+1. **Canonical sprint-scoring definition** — used by the scoring model above (and by the
+   reusable scoring helpers `_pointsByWorkType` / `_tasksByWorkType` on **`_Model_Team_Sprint`**
+   in `_underscore`, `Model/Team/Sprint.php`): a task is *done* when
+   `statusNow IN (STATUS_IN__DONE)` and its category is taken from **`workTypeAtLock`** (the
+   frozen at-lock baseline). This is what reliability/integrity scoring measures against.
+2. **Power BI / current-state definition** — used by dashboard tiles that must match the Power BI
+   numbers: *done* is **`dtDone IS NOT NULL`** and category is **`workTypeNow`** (current state,
+   not the at-lock snapshot). The api2 Record Script
+   [`_Model_Team_Sprint::committedTile()`](../../api2/features/record-scripts.md) deliberately
+   uses this definition.
+
+Neither is "wrong" — they answer different questions (scored performance vs. live current state).
+If a tile or report must **agree with TOGa IQ's canonical scoring**, reconcile it to definition
+(1); otherwise expect current-state tiles to diverge from sprint scores.
+
 ## Change history
 
+- 2026-07-23 — Recorded that two intentionally-different "committed/done" definitions coexist:
+  the canonical sprint-scoring definition (`statusNow IN (STATUS_IN__DONE)` + `workTypeAtLock`,
+  incl. the reusable `_pointsByWorkType`/`_tasksByWorkType` helpers on `_Model_Team_Sprint` in
+  `_underscore`) vs. the Power BI / current-state definition (`dtDone IS NOT NULL` +
+  `workTypeNow`) used by dashboard tiles such as the api2 `committedTile()` Record Script.
+  Added `_underscore/Model/Team/Sprint.php` to this doc's files. (kyalamarthi)
 - 2026-07-21 — Added `SprintLockScheduled()` action + a `Core.CronJobs` seed
   (`0 10 * * 3`, every Wednesday 10 AM Central, action `Team/Sprint/SprintLockScheduled`,
   maxExecutionTime 600) to run **Sprint Lock automatically at the start of each biweekly

package/knowledge/INDEX.md

@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 - **_underscore** (_Underscore) _(framework core)_ — 37 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 31 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
-- **api2** (API) — 13 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
+- **api2** (API) — 14 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
 - **toga2-supply** (TOGa Supply) — 3 doc(s) → [2.0/apps/toga2-supply/INDEX.md](2.0/apps/toga2-supply/INDEX.md)
 - **saml** (SAML SSO Gateway) — 3 doc(s) → [2.0/apps/saml/INDEX.md](2.0/apps/saml/INDEX.md)

package/knowledge/sessions/2026-07-23-compass-retrofix2-sql-generator-jcardinal.md

@@ -0,0 +1,69 @@
+---
+type: session
+slug: compass-retrofix2-sql-generator
+title: Compass retroactive data-fix v2 — SQL generator rewrite
+author: jcardinal
+repos: [test, worker, library, _underscore]
+framework: "2.0"
+client: compass-usa
+status: active
+created: 2026-07-23
+updated: 2026-07-23
+---
+
+# Session: compass-retrofix2-sql-generator
+**Date:** 2026-07-23
+**Project/Repo:** test/@jeff/compass (script), operating on Client_Compass (2.0)
+**Task:** Rewrite the Compass USA retroactive data-fix as a NEW standalone script that GENERATES SQL to a flat file (never writes the DB), fixing SO/PO item existence + the two item-link bridges bottom-up, then recursing all ItemFulfillments up to the Compass SO — replacing the old direct-write `retrofix.php`.
+
+---
+
+## What WORKED
+- **New script built & lint-clean:** `D:\WWW\test\@jeff\compass\retrofix2.php` (`php -l` passes). Standalone CLI: `require_once('_.php'); App_Framework_Sandbox::initialize();`, read-only DB alias `db_prod2_compass`, writes SQL to `retrofix2.phase1.sql` / `retrofix2.phase2.sql`.
+- **Two-pass PHASE model** (`const PHASE = 1|2`): Phase 1 generates item-link SQL → developer APPLIES it → re-run with PHASE=2 (now reads corrected bridges) → generates IF-recursion SQL. Chosen because Phase 2's up-mapping walks the item bridges Phase 1 fixes, and generate-only SQL can't see un-applied fixes.
+- **Chain resolution** (`getChainForSo`): developer's LEFT-JOIN header query, deduped into distinct id sets per level (compassPoIds/odpSoIds/odpPoIds/agilantSoIds + odpPo→odpSo, compassPo→odpSo pairings). Validated against prod: a Compass SO can have MULTIPLE Compass POs; recent orders legitimately have partial chains.
+- **Phase 1 L1 group logic** mirrors `library/app/api/toga2.php:778-897,1216-1352` in id-space: re-pulls Agilant SO from NetSuite (`App_NetSuite::getSalesOrder` by `c_netsuiteInternalSalesOrderId`), builds `memberItemId=>[groupItemId]` via `getItemDetails(...,'itemGroup',false)` on group-header lines (line has neither `rate` nor `amount`), matches Agilant SOI→ODP POI by (item OR group-item, quantity) non-consuming (1 POI → many SOIs). Memoizes getItemDetails per NS internal id.
+- **Phase 2** ports `_underscore/Model/Client/ItemFulfillment.php::reconcileUpstreamLevel` as SQL: header walk, item walk grouped-by-upstream-SOI with bundle scaling `min(rawSum, rawSum*upQtyOrdered/downOrderedSum)`, broken-bridge guard, create/link upstream IF+IFI, propagate units + 3 tracking bridges (shared unitId/trackingNumberId), carries planned rows forward in-memory across the Agilant→ODP→Compass climb. New rows use generated uuid + `(SELECT id … WHERE uuid=…)` refs.
+- **emit() guard** hard-blocks any write to `TrackingNumbers` or `Units` (only exact-table match; bridge tables like `ItemFulfillmentItemUnits_TrackingNumbers` correctly allowed).
+- **Bug fixes this session (all lint-verified):**
+  - L4 false inserts — root cause: existing-links query used `WHERE b.purchaseOrderItemId IN (<Compass PO HEADER ids>)`; fixed to join `PurchaseOrderItems … purchaseOrderId IN (compassPoIds)`. Confirmed against SA133672 (SO 108392) which no longer emits its two spurious inserts.
+  - Phase 2 `ifIdExpr` "Undefined array key realId" — IFI handles use `ifiRealId`/`ifiUuid`, IF handles use `realId`/`uuid`; helper now accepts both.
+  - MySQL 1093 (self-referential UPDATE) — `ifIdExpr` now wraps uuid lookups in a derived table `(SELECT id FROM (SELECT id FROM t WHERE uuid=…) AS _rf2_<hash>)`; inline upstream-IF-link expr routed through it too.
+  - MySQL 1451 (FK on stale deletes) — deletes now strictly bottom-up: IFIU_TrackingNumbers → IFIU → ItemFulfillmentItems_TrackingNumbers → ItemFulfillmentItems (and unit-tracking → unit for leftover units).
+  - Order header now prints date: `# SA123456 (2026-06-15)` (worklist selects dateOrder) for resume tracking.
+
+## What did NOT work — DO NOT RETRY THESE
+- **Relying on the plain `try/catch(Throwable)` around NetSuite calls to catch the SSL drop** ("SoapClient::__doRequest(): SSL: An existing connection was forcibly closed by the remote host"). The warning is intercepted by TOGA's GLOBAL `App_Error::handleError → handleException`, which (Sentry active) reports and can terminate the instance / not re-throw cleanly — so the retry loop never got a reliable shot. FIX APPLIED: `nsCall` now installs a LOCAL `set_error_handler` (throws ErrorException) around each attempt and `restore_error_handler()` after — so the warning is caught locally and retried. Do not go back to relying on the global handler.
+- **Old `retrofix.php` itemId-only "purity" test** for item-group links — would delete legitimate group links (1 ODP PO item ↔ many Agilant SO items whose itemIds differ). Superseded by the NetSuite group-map approach. Do not reintroduce pure itemId matching at L1.
+
+## Not tried yet (candidates for next session)
+- **End-to-end verification not yet done:** the idempotency re-run test (apply Phase 1 → re-run → expect empty) and the engine-parity check for a Flow-B order have NOT been run.
+- **L4 Compass-SO↔Compass-PO bundle links** are intentionally NOT auto-created — bundle/config lines (`c_isConfiguration`/`parentSalesOrderItemId`/`bundleId`) are only logged as SKIP. Needs the bundle-split logic from `worker/crons/toga2/compass/workflow/1_transmit_compass_sales_orders_to_mits.php` before enabling.
+- Confirm the fresh NetSuite retry fix (local error handler) actually survives a real SSL drop in a full batch run.
+- Phase 2 possible over-emission: already-linked IFIs may get redundant (harmless) upstream-link UPDATEs across multi-level climbs — validate on the parity test.
+- IF `number` generation (`F#####` via MAX+1) collision risk vs concurrent prod IF creation — apply Phase 2 SQL promptly.
+
+## Current file state
+| File | Status | Notes |
+|------|--------|-------|
+| D:\WWW\test\@jeff\compass\retrofix2.php | Created, lint-clean, working | Full 2-phase generator; all reported bugs fixed; NetSuite local-error-handler retry (5× backoff 3/6/9/12s) |
+| D:\WWW\test\@jeff\compass\retrofix2.phase1.sql | Generated (test window) | Output artifact; L4 spurious-insert bug fixed since last gen — regenerate |
+| D:\WWW\test\@jeff\compass\retrofix2.phase2.sql | Generated (test window) | Output artifact; 1093/1451 fixed since last gen — regenerate |
+| C:\Users\JCardinal\.claude\plans\i-need-to-continue-declarative-kahan.md | Created | Approved implementation plan (context, phases, open items, verification) |
+| D:\WWW\test\@jeff\compass\retrofix.php | Unchanged | Old direct-write version, kept for reference |
+
+## Decisions made
+- **Generate SQL, never write the DB** (developer applies manually) — user requirement; SELECTs + NetSuite GETs only.
+- **Two-pass PHASE constant** over single-pass in-memory bridge model — simpler/safer, matches rollout order (verify links first, then fulfillments). Rejected: single-pass modelling expected bridges in memory (too complex/error-prone to verify).
+- **NetSuite group resolution via live `getItemDetails`** (per user direction) rather than any persisted TOGA bundle table — re-pulls each Agilant SO from NetSuite.
+- **Flat SQL file, no transactions, `# SA###### (date)` comments** per user preference.
+- **L4 conservative** (insert pure links, log suspected bundles, delete only exact dups) — avoids destroying legit bundle links pending the script-1 bundle logic.
+
+## Blockers
+None blocking. Open dependency: Phase 2 requires the Phase 1 SQL to be APPLIED to the target DB first (two-pass), and full E2E verification is still pending.
+
+## Exact next step
+> Regenerate Phase 1 over a small test window (set `START_DATE`/`END_DATE` to ~1 day, `PHASE=1`) in `D:\WWW\test\@jeff\compass\retrofix2.php`, run it (live NetSuite + prod-read), then run the idempotency check: apply `retrofix2.phase1.sql` to a LOCAL Client_Compass copy, re-run PHASE=1, and confirm the second output is empty. Then repeat for PHASE=2.
+
+---
+_Saved by /session-save on 2026-07-23_

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.425",
+  "version": "1.0.427",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",