toga-ai 1.0.371 → 1.0.372

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -2,6 +2,7 @@
 
 | Doc | Summary | Files |
 |-----|---------|-------|
+| [Proposed — ExternalIdentifiers map table for integration id ↔ external-system linkage](architecture/external-identifiers-map-table.md) | A **proposal / counter-proposal (status: proposed — nothing here is built)** that cross-system identifier linkage (e.g. | _underscore/Model/Forecast/Opportunity.php, worker2/Worker/Netsuite/Opportunity.php |
 | [Proposed — git-sourced base+overlay JSON authoring for the Surface layer](architecture/surface-authoring-proposal.md) | A **proposal / handoff recommendation** (not implemented) that the Surface layer's *authoring* model move off hand-authored SQL against the `SurfaceOverrides` E | _underscore/Model/Core/Surface.php, _underscore/Model/Client/SurfaceOverride.php |
 | [_underscore Framework Architecture](architecture.md) | `_underscore` is the shared PHP backend framework for **all 2.0 applications**. | _underscore/_underscore.php, _underscore/Loader.php, _underscore/Framework.php, _underscore/Model.php, _underscore/Database.php, _underscore/Query.php, _underscore/Route.php, _underscore/Component.php |
 | [ACL Permission Chain (Record & Field Authorization)](features/acl-permission-chain.md) | Authorization in the 2.0 API is **metadata-driven**: whether a role may Create/Read/Update/Delete a record is decided by rows across **four linked tables**, not | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Page.php, dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql |
@@ -18,6 +19,7 @@
 | [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | worker2/Component/Forecast/SaleImport/SaleImport.php, worker2/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/verify_je_classification.php, test/@dave/probe_je_accounts.php, test/@dave/probe_je_shape.php, test/@dave/fixer.php, test/@dave/Junk Drawer/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/Junk Drawer/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
 | [Item-Fulfillment Stage Lifecycle (picked/packed/shipped) & Order Status](features/item-fulfillment-stage-lifecycle-and-order-status.md) | Every ItemFulfillment (IF) now carries an explicit **stage** — picked → packed → shipped — resolved through `ItemFulfillmentStages → ItemFulfillmentStatuses` (m | _underscore/Model/Client/SalesOrder.php, _underscore/Model/Quad/SalesOrder.php, _underscore/Model/Compass/SalesOrder.php, _underscore/Model/Compass/SalesOrderStatus.php, _underscore/Model/Client/SalesOrderItem.php, _underscore/Model/Client/Item.php, _underscore/Model/Client/PurchaseOrderItem.php, library/app/api/toga2.php, dbchanges2/Client/2026-06-30a - BackfillNullStageItemFulfillmentsToShipped.sql, dbchanges2/Client/2026-06-30b - SalesOrderStatusesPickedPacked.sql, dbchanges2/Client/2026-06-30c - ItemFulfillmentStageIdNotNull.sql, dbchanges2/Client_CompassCanada/2026-06-30a - ItemFulfillmentLifecycleAndShippedBackfill.sql |
 | [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php |
+| [_Model::save() vs raw _Query — no atomic conditional update](features/model-save-vs-query-atomic-update.md) | `_Model::save()` is a plain load-then-write ORM primitive and **cannot express an atomic conditional update** (an optimistic-concurrency / row-claim guard such | _underscore/Model.php, _underscore/Query.php |
 | [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
 | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |
 | [Recursive Item Fulfillments (upstream mirroring)](features/recursive-item-fulfillments.md) | In a multi-tier supply chain a sales order (SO) spawns a purchase order (PO) that becomes another SO downstream, and so on. | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillmentItem.php, _underscore/Model/Client/ItemFulfillmentItemUnit.php, _underscore/Model/Client/ItemFulfillmentPackage.php, _underscore/Model/Compass/AdvanceShippingNotice.php, dbchanges2/Core/2026-02-13 - 75601 - RecursiveItemFulfillmentCreation.sql, dbchanges2/Core/2026-06-04 - RecursiveItemFulfillmentPut.sql, dbchanges2/Client_Compass/2026-07-02a - FixSA133377TrackingSerialAndDuplicateIF.sql |

package/knowledge/2.0/apps/_underscore/architecture/external-identifiers-map-table.md

@@ -0,0 +1,77 @@
+---
+title: Proposed — ExternalIdentifiers map table for integration id ↔ external-system linkage
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: architecture
+status: proposed
+updated: 2026-07-17
+owners: ["dfranks"]
+files:
+  - _underscore/Model/Forecast/Opportunity.php
+  - worker2/Worker/Netsuite/Opportunity.php
+related:
+  - ../features/model-save-vs-query-atomic-update.md
+  - ../../worker2/features/netsuite-opportunity-sync.md
+---
+
+## Summary
+
+A **proposal / counter-proposal (status: proposed — nothing here is built)** that cross-system
+identifier linkage (e.g. a NetSuite opportunity ↔ its ClickUp task id) live in a dedicated
+**`ExternalIdentifiers` map table** rather than as ad-hoc columns bolted onto whichever business
+table happens to be nearby. The immediate driver is the TRUE-80044 atomic ClickUp-task claim, whose
+currently-mandated approach adds a `clickupTaskId` column to `Forecast.Opportunities`. This doc is
+`dfranks`'s architectural counter-proposal to that mandate, raised for open discussion — **not** a
+silent supersede of the reviewer decision. See the rebuttal section below.
+
+**Critical framing:** This is a **proposal under open discussion**, not the current system and not a
+settled reversal. The mandated approach on record (a `clickupTaskId` column on
+`Forecast.Opportunities`) remains the standing decision until the disagreement is resolved with the
+reviewer.
+
+## Design
+
+A single shared table keyed by (local record identity, external system, external id kind):
+
+- **Columns (sketch):** `id`, `entityType` (e.g. `opportunity`), `entityId` (the local business
+  key, e.g. `netsuiteOpportunityInternalId`), `externalSystem` (e.g. `clickup`), `externalIdKind`
+  (e.g. `task`), `externalId` (the value/sentinel), plus `dtCreated`/`dtUpdated`.
+- **Uniqueness:** a UNIQUE index on `(entityType, entityId, externalSystem, externalIdKind)` — this
+  is the single-row lock that makes the atomic claim (`UPDATE … SET externalId='<uuid sentinel>'
+  WHERE … AND externalId IS NULL`, commit, `affectedRows===1` wins) work exactly as the mandated
+  column approach would, but without touching the Forecast schema.
+- **Atomic claim mechanics are unchanged** from the mandated design — same raw `_Query` guarded
+  UPDATE (ORM `save()` cannot express it — see
+  [model-save-vs-query-atomic-update](../features/model-save-vs-query-atomic-update.md)), same
+  commit-before-external-side-effect, same sentinel-guarded compensation-delete on failure.
+
+## Rebuttal to the clickupTaskId-column mandate
+
+`2.0/apps/worker2/features/netsuite-opportunity-sync.md` records reviewer **Rohan Girish** mandating
+(2026-07-15) that the ClickUp-task dedup claim be implemented as a **`clickupTaskId` column on
+`Forecast.Opportunities`**, and rejecting the earlier plan's dedicated table. This doc is `dfranks`'s
+**counter-proposal**, offered as an open architectural disagreement to be resolved *with* the
+reviewer — the mandate stands until then. The three rebuttal arguments:
+
+1. **Separation of concerns.** A ClickUp task id is *integration-linkage* state, not a business
+   attribute of a Forecast opportunity. Storing it on `Forecast.Opportunities` conflates the
+   integration/mapping layer with the forecasting domain model.
+2. **Forecast-schema dumping-ground.** Every future external linkage (a second external system, a
+   different id kind) under the column approach means another one-off nullable column on a business
+   table, accreting integration cruft on the Forecast schema. A map table absorbs all of them with
+   no schema change per linkage.
+3. **Forecast-sync lifecycle decoupling.** The integration mapping must be able to **outlive a
+   possibly-discontinued Forecast sync.** Binding the ClickUp linkage to the Forecast opportunity row
+   ties its lifetime to the Forecast importer; if that sync is ever retired or reworked, the mapping
+   goes with it. A standalone map table keeps the NS↔ClickUp linkage durable independently of
+   Forecast.
+
+This is framed as a disagreement to resolve, not a reversal already taken. If the reviewer position
+holds, the mandated column approach is what ships.
+
+## Change history
+
+- 2026-07-17 — Proposed as a rebuttal to the reviewer-mandated clickupTaskId-column approach
+  (TRUE-80044). (dfranks)

package/knowledge/2.0/apps/_underscore/features/model-save-vs-query-atomic-update.md

@@ -0,0 +1,75 @@
+---
+title: _Model::save() vs raw _Query — no atomic conditional update
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: feature
+status: active
+updated: 2026-07-17
+owners: ["dfranks"]
+files:
+  - _underscore/Model.php
+  - _underscore/Query.php
+related:
+  - ./model-magic-field-access.md
+  - ../../worker2/features/netsuite-opportunity-sync.md
+---
+
+## Summary
+
+`_Model::save()` is a plain load-then-write ORM primitive and **cannot express an atomic
+conditional update** (an optimistic-concurrency / row-claim guard such as
+`... WHERE col IS NULL`). For any DB-level claim / optimistic-concurrency pattern you must drop
+to raw `_Query` — `_Query::getAffectedRows()` is the only surface that exposes an affected-rows
+count, which is what a claim needs to know whether *this* caller won. `save()` remains the correct
+idiom for ordinary by-primary-key writes.
+
+## How it works
+
+`_Model::save()`'s update branch always emits:
+
+```sql
+UPDATE <table> SET <changed cols> WHERE <primaryKey> = <id>
+```
+
+(`Model.php` ~line 285). Consequences that make it unusable for a conditional claim:
+
+- **No conditional guard.** The `WHERE` is hard-coded to the primary key; there is no way to attach
+  an extra predicate like `AND clickupTaskId IS NULL`. So two concurrent callers both pass any
+  in-PHP check and both write — the update is not a claim.
+- **Skips the write when nothing changed.** If no field differs from the loaded row, `save()` emits
+  no UPDATE at all. A claim that relies on "did my write land?" gets no signal.
+- **Returns no affected-rows count.** `save()` does not surface how many rows the UPDATE touched, so
+  even a guarded statement (if it could emit one) couldn't be adjudicated through the model.
+
+**For an atomic claim / conditional update, use `_Query` directly.** `_Query::getAffectedRows()`
+(`Query.php` line 106) is the only API that exposes the affected-rows count. The idiom is:
+
+```php
+// UPDATE ... SET claim='<sentinel>' WHERE <businessKey>=? AND claim IS NULL  (raw _Query)
+// then commit; affectedRows === 1 → this caller won the claim; === 0 → another already claimed it.
+```
+
+Back the guarded UPDATE with a UNIQUE index on the business key so it locks a single row, and commit
+before any external side effect so the row lock is released (and to dodge the `_Database` lazy-transaction
+write-drop). All **non-claim** writes stay on the ORM — `save()` is fine for plain
+load → set → save → `transactionCommit` by primary key (the idiom
+`_Worker_Netsuite_Opportunity::delete()` already uses).
+
+## Gotchas / known issues
+
+- Do **not** reach for `save()` to implement a row-claim / optimistic-concurrency guard — it silently
+  degrades to a non-atomic check-then-act because its `WHERE` is primary-key-only and it exposes no
+  affected-rows count. Use raw `_Query` + `getAffectedRows()`.
+- A no-diff `save()` emits **no** UPDATE, so "the write must have run because save() returned" is not a
+  safe assumption.
+
+## Change history
+
+- 2026-07-17 — Documented that `_Model::save()` cannot perform an atomic conditional update
+  (primary-key-only `WHERE`, skips no-diff writes, no affected-rows count) and that raw `_Query` +
+  `_Query::getAffectedRows()` is the required surface for any DB row-claim / optimistic-concurrency
+  pattern. Surfaced while designing the TRUE-80044 atomic ClickUp-task claim. (dfranks)
+</content>
+</invoke>

package/knowledge/2.0/apps/worker2/features/netsuite-opportunity-sync.md

@@ -31,6 +31,7 @@ files:
 related:
   - ./netsuite-salesorder-open-orders-sync.md
   - ../architecture.md
+  - ../../_underscore/architecture/external-identifiers-map-table.md
 ---
 
 ## Summary
@@ -177,6 +178,9 @@ tasks. The fix is a **race-safe DB claim** on the **existing `Forecast.Opportuni
 `NetsuiteOpportunityClickupTask` table in `Client_True`; reviewer Rohan Girish rejected that and mandated
 reusing `Forecast.Opportunities`.)
 
+A counter-proposal (ExternalIdentifiers map table) has been raised — see
+`_underscore/architecture/external-identifiers-map-table.md`; unresolved.
+
 How the claim is race-safe:
 - **Conditional UPDATE = the atomic claim.** A raw `_Query`:
   `UPDATE Opportunities SET clickupTaskId='<uuid sentinel>' WHERE netsuiteOpportunityInternalId=X AND

package/knowledge/INDEX.md

@@ -17,7 +17,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 33 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 35 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 30 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 12 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 3 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.371",
+  "version": "1.0.372",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",