toga-ai 1.0.217 → 1.0.219

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -8,6 +8,7 @@
8
8
  | [Create Elastic Beanstalk Environment (script)](features/create-elastic-beanstalk.md) | `team/aws/create_elastic_beanstalk.php` is a **standalone** (no `App_` framework) constants-driven PHP generator. | test/team/aws/create_elastic_beanstalk.php |
9
9
  | [Developer Generators (password, UUID)](features/dev-generators.md) | Two tiny **1.0 `App_` framework** convenience scripts for everyday developer needs. | test/team/generate_password.php, test/team/uuid.php |
10
10
  | [Forecast vs NetSuite Discrepancy Analysis](features/forecast-netsuite-discrepancy-analysis.md) | `team/forecast-netsuite/discrepancy_analysis.php` detects discrepancies between our **Forecast database** and **NetSuite** (the source of truth for all sales da | test/team/forecast-netsuite/discrepancy_analysis.php |
11
+ | [@goagilant.com → @togatech.com Email-Domain Migration (1.0 + 2.0)](features/goagilant-to-togatech-email-migration.md) | Reference + technique for migrating the company email domain `@goagilant.com` → `@togatech.com` across **both** platforms. | migrate_goagilant_to_togatech_2026-06-26.sql, migrate_goagilant_to_togatech_LEGACY_2026-06-26.sql |
11
12
  | [TableView Builder (2.0 TableViews SQL generator)](features/tableview-builder.md) | `team/tableViewBuilder/` generates SQL `INSERT` statements for the **2.0 `TableViews`**, `TableViewFields`, and `TableViewJoins` tables from a plain SQL `SELECT | test/team/tableViewBuilder/TableViewGenerator.php, test/team/tableViewBuilder/index.php, test/team/tableViewBuilder/Instructions.md |
12
13
  | [Talos Knowledge Base Pipeline (Uploader + Processor)](features/talos-kb-pipeline.md) | `team/talos/` holds the two-script web tooling that feeds the **TOGa Talos** (TOGa IQ) AI knowledge bases. | test/team/talos/kb_uploader.php, test/team/talos/kb_processor.php, test/team/talos/kb_processor.ini |
13
14
  | [TOGa 2.0 Client Onboarding SQL Generator](features/toga2-client-onboarding-sql.md) | `team/generate_toga2_onboarding_sql.php` generates the SQL needed to **onboard a new client** onto the 2.0 platform. | test/team/generate_toga2_onboarding_sql.php |
@@ -21,6 +21,7 @@ related:
21
21
  - ./features/url-domain-markdown-document.md
22
22
  - ./features/dev-generators.md
23
23
  - ./features/talos-kb-pipeline.md
24
+ - ./features/goagilant-to-togatech-email-migration.md
24
25
  ---
25
26
 
26
27
  ## Summary
@@ -83,6 +84,7 @@ manually against the target database; they do not execute changes themselves.
83
84
  | `generate_toga2_user_access_sql.php` | 1.0 | 2.0 client DBs | [toga2-user-cross-client-access-sql](./features/toga2-user-cross-client-access-sql.md) |
84
85
  | `generate_password.php`, `uuid.php` | 1.0 | dev utility | [dev-generators](./features/dev-generators.md) |
85
86
  | `talos/kb_uploader.php`, `talos/kb_processor.php` | standalone | S3 + AWS Bedrock KBs | [talos-kb-pipeline](./features/talos-kb-pipeline.md) |
87
+ | `migrate_goagilant_to_togatech_*.sql` | standalone | 2.0 `Client_` DBs + 1.0 schemas | [goagilant-to-togatech-email-migration](./features/goagilant-to-togatech-email-migration.md) |
86
88
 
87
89
  **Retired / not documented:** `team/DOA_gitbook/` is being retired.
88
90
 
@@ -91,3 +93,5 @@ manually against the target database; they do not execute changes themselves.
91
93
  - 2026-06-16 — Initial capture of the `team/` folder. Registered repo in `registry.json`.
92
94
  - 2026-06-22 — Un-retired `team/DOA_talos/` → `team/talos/`; documented its uploader +
93
95
  processor in [talos-kb-pipeline](./features/talos-kb-pipeline.md).
96
+ - 2026-06-26 — Documented the @goagilant.com → @togatech.com email-domain migration SQL
97
+ artifacts in [goagilant-to-togatech-email-migration](./features/goagilant-to-togatech-email-migration.md).
@@ -0,0 +1,134 @@
1
+ ---
2
+ title: "@goagilant.com → @togatech.com Email-Domain Migration (1.0 + 2.0)"
3
+ framework: "1.0"
4
+ repo: test
5
+ project: Test
6
+ client: shared
7
+ type: feature
8
+ status: active
9
+ updated: 2026-06-26
10
+ owners: [bala]
11
+ files:
12
+ - migrate_goagilant_to_togatech_2026-06-26.sql
13
+ - migrate_goagilant_to_togatech_LEGACY_2026-06-26.sql
14
+ related:
15
+ - ../architecture.md
16
+ - ./2-0-deployment-client-sql.md
17
+ - ../../../2.0/apps/_underscore/features/email-template-sending.md
18
+ - ../../../2.0/apps/_underscore/features/per-client-database-connections.md
19
+ ---
20
+
21
+ ## Summary
22
+
23
+ Reference + technique for migrating the company email domain `@goagilant.com` →
24
+ `@togatech.com` across **both** platforms. Two standalone SQL artifacts (run manually
25
+ against the right cluster, not framework code) were produced:
26
+
27
+ - `migrate_goagilant_to_togatech_2026-06-26.sql` — 2.0 prod, per-tenant `Client_` DBs.
28
+ - `migrate_goagilant_to_togatech_LEGACY_2026-06-26.sql` — 1.0 legacy.
29
+
30
+ The durable value is the **email-column blast-radius map** for each environment and the
31
+ **two collision gotchas** that make a blanket find/replace unsafe. Use this before any
32
+ future email-domain change.
33
+
34
+ ## Where @goagilant.com emails live
35
+
36
+ ### 2.0 prod (per-tenant `Client_` DBs, prod-client cluster)
37
+
38
+ Value columns that actually held `@goagilant.com` data:
39
+
40
+ | Column | Notes |
41
+ |---|---|
42
+ | `Users.email` | **UNIQUE** — identity logins. See Client_True gotcha. |
43
+ | `ContactEmailAddresses.emailAddress` | |
44
+ | `EmailTemplates.sendFromEmailAddress` | Present in **all 32 clients**; outbound `From:` — deliverability-sensitive (sender-verification gate below). |
45
+ | `IntegrationsEmail.fromEmailAddress` | Outbound `From:` — deliverability-sensitive. |
46
+ | `SalesOrderEmailAddresses.emailAddress` | Composite UNIQUE `(salesOrderId, emailAddress)`. |
47
+ | `EmailTemplateOutgoingEmailAddresses.emailAddress` | Composite UNIQUE. |
48
+
49
+ Columns that exist but held **zero** `@goagilant.com` data (skip):
50
+ `Campaigns.bookingEmailAddress`, `LocationEmailAddresses.emailAddress`,
51
+ `IntegrationsEmailOutgoingEmailAddresses.emailAddress`.
52
+
53
+ `ContactAttempts.c_contactEmail` exists **only** in `Client_True` (BDR).
54
+
55
+ Tenants with `Users.email` hits included Compass, CompassCanada, Managelife, Quad, Growrk,
56
+ Nychh, Prudential, Aig, Endeavorhealth, Erau, Masonite, Rate, Spglobal, Towfoundation, Wje,
57
+ Wmchealth, Ynhh, plus True (special-cased).
58
+
59
+ ### 1.0 legacy
60
+
61
+ | Table | Hits | Notes |
62
+ |---|---|---|
63
+ | `TOGA.Users.email` | 1117 goagilant | Main platform login. **NOT unique** (PRIMARY on `id` only). togatech migration already ~half done: 1077 togatech rows exist; 7 goagilant overlap a togatech local-part. |
64
+ | `TOGA_<tenant>.ClientUsers.email` | 0 goagilant | Client-portal logins — clean across all 14 tenants. |
65
+ | `TOGA_<tenant>.Contacts.emailAddress` | ~255 | CRM. `relayEmailAddress` = 0 everywhere. |
66
+ | `Vision.Users` | 343 | Sibling-app login, 0 togatech twins. |
67
+ | `RetailServices.DashboardUsers` | 6 | 0 togatech twins. |
68
+ | `ITADPortal.Users` | 3 | 0 togatech twins. |
69
+ | `TOGaDeskAdmin.users` | 2 | 0 togatech twins. |
70
+
71
+ **Not migrated** (high-volume ancillary, out of scope): helpdesk `TOGaDeskSupport.tickets`
72
+ (15725), `Common` directory (~3700), RetailServices customer data, Netsuite/Store. The
73
+ **Advisor** and **GoAgilant (WordPress)** schemas are not queryable via the toga-db MCP tool
74
+ and were not assessed here.
75
+
76
+ ## How it works (safe-rename technique)
77
+
78
+ Swap **only** the 14-char domain suffix, preserving the local-part exactly:
79
+
80
+ ```sql
81
+ UPDATE SomeTable
82
+ SET col = CONCAT(LEFT(col, CHAR_LENGTH(col) - 14), '@togatech.com')
83
+ WHERE col LIKE '%@goagilant.com';
84
+ ```
85
+
86
+ Why not `REPLACE()`: `REPLACE` is case-sensitive (misses `@GoAgilant.com`) and can touch a
87
+ mid-string match. `LEFT`/`CONCAT` on the trailing 14 chars touches only the domain and keeps
88
+ apostrophes/odd local-parts intact (e.g. `do'connell@`). The `LIKE` is the natural
89
+ case-insensitive guard for the match.
90
+
91
+ For the non-unique 1.0 `TOGA.Users`, a guarded variant uses `NOT EXISTS` against a
92
+ materialized derived table of existing `@togatech.com` addresses, so the 7 would-be
93
+ duplicate logins are skipped rather than creating colliding identities.
94
+
95
+ ## Scope (explicit developer instructions)
96
+
97
+ - **2.0:** exclude `Client_True.Users` (see gotcha); still migrate True's other email tables.
98
+ - **1.0:** scoped to **USERS + CONTACTS only** — explicitly exclude `Customers`,
99
+ `SMBContracts` (contracts/work orders), and send-from in 1.0.
100
+ - **Never** touch any `*Logs*` schemas.
101
+
102
+ ## Sender-address gate (deliverability)
103
+
104
+ `EmailTemplates.sendFromEmailAddress` (~118 rows across all 2.0 clients) and
105
+ `IntegrationsEmail.fromEmailAddress` change the outbound `From:` domain. These were gated
106
+ as a separate step pending `togatech.com` sender verification (SPF/DKIM/DMARC). The domain
107
+ was **confirmed verified 2026-06-26**, so the sender-address step is cleared to run. Do not
108
+ flip a `From:` domain ahead of sender verification — it breaks deliverability.
109
+
110
+ ## Gotchas
111
+
112
+ - **Client_True (2.0) UNIQUE twins.** `Users.email` is UNIQUE. Of 343 `@goagilant.com`
113
+ users, **325 already have a pre-staged `@togatech.com` twin** (twins mostly never logged
114
+ in, `dtLastLogin` NULL — a togatech cutover was partially staged). A blanket rename
115
+ collides on the UNIQUE constraint (fails, or with `IGNORE` **silently skips 325**). Of the
116
+ 18 no-twin rows, ~13 are service/SFTP/support accounts (`odpsftp@`, `staplessftp@`,
117
+ `support@`, …) whose rename could break integrations. **Decision:** exclude
118
+ `Client_True.Users.email` from the migration; migrate only True's 64 non-identity rows
119
+ (ContactEmailAddresses 56, Campaigns 1, ContactAttempts 1, EmailTemplates.sendFromEmailAddress
120
+ 6), which carry no UNIQUE-on-email risk.
121
+ - **Legacy `TOGA.Users` is non-unique and half-migrated.** PRIMARY is on `id` only, so a
122
+ rename can produce duplicate login emails. 1077 togatech rows already exist and 7 goagilant
123
+ rows overlap a togatech local-part — use the `NOT EXISTS`-guarded variant to skip those 7.
124
+ - **Run against the right cluster.** 2.0 lives in per-tenant `Client_` DBs on the prod-client
125
+ cluster; 1.0 in `TOGA` / `TOGA_<tenant>` / sibling-app schemas. Review the generated SQL
126
+ before executing — these are manual-run artifacts, nothing auto-executes.
127
+
128
+ ## Change history
129
+
130
+ - 2026-06-26 — Initial capture. Mapped 2.0 + 1.0 email columns, recorded the Client_True
131
+ UNIQUE-twin and legacy non-unique-login collision gotchas, the LEFT/CONCAT safe-rename
132
+ technique, scope exclusions, and the cleared sender-verification gate. (bala)
133
+ </content>
134
+ </invoke>
@@ -7,7 +7,7 @@
7
7
  | [Assortment Name Translation (AssortmentTranslations sidecar)](features/assortment-name-translation.md) | Serves Assortment (product-grouping) **names** in multiple languages by adding a per-language **sidecar** table `AssortmentTranslations`, reusing the platform's | _underscore/Model/Client/AssortmentTranslation.php, dbchanges2/Client/2026-06-26a - AssortmentTranslations.sql, dbchanges2/Core/2026-06-26a - AssortmentTranslationsRecord.sql, dbchanges2/Client/2026-06-26b - AssortmentTranslationsAcl.sql |
8
8
  | [Carrier Shipping Labels (UPS/FedEx) & NetSuite Item Fulfillment](features/carrier-shipping-labels.md) | Backend mechanics behind TOGa Supply's Fulfill & Ship: buying a carrier label (UPS/FedEx), persisting it, and creating the NetSuite Item Fulfillment with tracki | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillments/TrackingNumber.php, _underscore/Component/Library/LabelPdf/LabelPdf.php, _underscore/Component/Library/Carriers/Ups/Ups.php, _underscore/Component/Library/Carriers/Fedex/Fedex.php, _underscore/Trait/Netsuite/ItemFulfillment.php, _underscore/Trait/Netsuite/SalesOrder.php, _underscore/Component/Library/NetSuite/NetSuite.php, _underscore/Model/Client/TrackingNumber.php, _underscore/Model/Client/ShippingMethod.php, _underscore/Model.php, _underscore/Cloud.php |
9
9
  | [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
10
- | [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
10
+ | [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
11
11
  | [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php |
12
12
  | [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
13
13
  | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |
@@ -11,6 +11,7 @@ owners: [dfranks]
11
11
  files:
12
12
  - _underscore/Component/Forecast/SaleImport/SaleImport.php
13
13
  - _underscore/Component/Forecast/Db/Db.php
14
+ - _underscore/Component/Api/Netsuite/Netsuite.php
14
15
  - worker2/Worker/Netsuite/Invoice.php
15
16
  - worker2/Worker/Netsuite/CashSale.php
16
17
  - worker2/Worker/Netsuite/CreditMemo.php
@@ -24,6 +25,7 @@ files:
24
25
  - test/@dave/test_creditmemo_lifecycle.php
25
26
  - test/@dave/test_cashsale_lifecycle.php
26
27
  - test/@dave/test_cashrefund_lifecycle.php
28
+ - test/@dave/test_fetchrecord_routes.php
27
29
  - test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js
28
30
  - test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js
29
31
  related:
@@ -71,7 +73,7 @@ Engine public surface: `sync(int $internalId, string $type)` and
71
73
 
72
74
  ## How it works
73
75
  Per-record flow in `sync`:
74
- 1. REST GET the record (`expandSubResources=true`) via `_Component_Forecast_Db::fetchRecord`.
76
+ 1. REST GET the record (`expandSubResources=true`) via `_Component_Api_Netsuite::fetchRecord`.
75
77
  2. **Per-type status gate** → if excluded, call `removeAll` and stop. Exclusions: invoice
76
78
  `Voided`/`Rejected`; cashSale `Unapproved Payment`; creditMemo `Voided`; cashRefund has
77
79
  no gate. **A cashSale created via REST lands in status `Deposited`** (NOT the excluded
@@ -88,14 +90,32 @@ Per-record flow in `sync`:
88
90
  `removeAll(internalId)` deletes all rows for that transaction (used by delete events and by
89
91
  the status gate).
90
92
 
91
- ### Shared helpers on `_Component_Forecast_Db` (dedup)
92
- - `fetchRecord(route, label)` — NS REST GET + decode.
93
- - `collectItemLines(record)` + `nextPageRoute(links)` — `rel=next` sublist pagination.
93
+ ### Shared NS helpers — `fetchRecord` lives on `_Component_Api_Netsuite`, the rest on `_Component_Forecast_Db`
94
+ - **`_Component_Api_Netsuite::fetchRecord(route, label)`** — NS REST GET + decode. **Moved
95
+ here** (beside `createRecord`) from `_Component_Forecast_Db` — it is a generic REST read,
96
+ not Forecast-specific, so it belongs with the other REST primitives. **Verify the class
97
+ before relying on it** (it was relocated this session). The **7 callers** were all
98
+ reprefixed to `_Component_Api_Netsuite::fetchRecord`: `SaleImport.php` ×2, `Opportunity.php`
99
+ ×3, `SalesOrder.php` ×2 (static git-grep: zero stale `_Component_Forecast_Db::fetchRecord`
100
+ refs, all 7 resolve, lint clean; runtime-verified across all 5 SaleImport types +
101
+ opportunity/employee/salesOrder routes via `test_fetchrecord_routes.php`).
102
+ - **Still on `_Component_Forecast_Db`** (Forecast-specific): `collectItemLines(record)` +
103
+ `nextPageRoute(links)` (`rel=next` sublist pagination), `lookupId`, `toSqlDate`.
94
104
 
95
105
  `Opportunity.php` and `SalesOrder.php` were refactored onto these shared versions; their
96
106
  private `fetchRecord`/`collectAllItemLines`/`nextPageRoute` copies (and SalesOrder's
97
107
  orphaned `MAX_SUBLIST_PAGES` const) were deleted.
98
108
 
109
+ > **Process gotcha — enumerate callers of these shared NS helpers with `git grep`, not the
110
+ > fuzzy/agent search.** Any move/rename of `_Component_Api_Netsuite::createRecord` /
111
+ > `fetchRecord` (or the other shared NS helpers) must build its caller inventory with
112
+ > `git grep -nF` **per repo** — the fuzzy search tool silently **missed**
113
+ > `worker2/Worker/Netsuite/SalesOrder.php:925` (`createRecord`, the SalesOrder SOAP→REST
114
+ > outbound push — a real **production** caller) on a first pass, producing a wrong "zero
115
+ > callers / test-only" conclusion and a broken refactor that had to be reverted. Search
116
+ > case-insensitively for `::method`, `function method`, and string/callable forms; verify
117
+ > **zero stale refs** after editing.
118
+
99
119
  ## Sign convention (the load-bearing decision)
100
120
  The **raw REST record** returns each line `amount`/`costEstimate` **POSITIVE for all four
101
121
  types** (verified by live probe 2026-06-24 against NS account 1095849: creditMemo lines
@@ -218,28 +238,103 @@ events as specific **subtypes** (`inventoryItem`, `nonInventoryResaleItem`, `kit
218
238
  would be needed only for a future real-time item webhook, **not** for this Sales importer
219
239
  (which keeps items fresh via the hourly item pull cron plus the inline self-heal).
220
240
 
221
- ## Local testing harness (`test/@dave/test_invoice_lifecycle.php`)
222
- A CLI that boots worker2/_underscore (chdir to `worker2/` then `require index.php`) and drives a
223
- **full NetSuite invoice lifecycle** through the live `_underscore` REST client
224
- (`_Component_Api_Netsuite`) to exercise this importer end-to-end against a local Forecast mirror.
225
- Actions: `check | create | get | update | delete | sync | recent | amq | deployments | locations`.
226
-
227
- - **Fixture:** customer **58** ("8 Test Company") + item **103741** ("Test Other Charge for Sale",
228
- maps to local `Forecast.Items` id 25), **rate 0** → a **$0 invoice that still yields one Sales
229
- row** (the engine does **not** skip a $0 line — revenue `0.00` is written; same as the cron's
230
- Sales section). Confirms create→insert, edit→update (tracked-column change-detection), and
231
- delete→`removeAll` all land in local `Forecast.Sales`.
232
- - It carries its **own** `createInvoiceRecord()` that POSTs via `_ApiRequest` and parses the
233
- `Location` header with a **correct** regex (delimiter `~`) — a deliberate workaround for the live
234
- `_Component_Api_Netsuite::createRecord()` `#`-delimiter bug (see the
235
- [REST client doc](./netsuite-rest-client.md)); the framework was **not** edited this session.
236
- - `amq` action SuiteQLs the AMQ custom record; `deployments` SuiteQLs `scriptdeployment`+`script`
237
- to show which enqueuer fires per record type (used to find the dual-deployment trap below).
238
- - **Per-type siblings** `test_creditmemo_lifecycle.php`, `test_cashsale_lifecycle.php`,
239
- `test_cashrefund_lifecycle.php` (and `test_je_lifecycle.php`) mirror this harness, each with its
240
- own `create()` POSTing via `_ApiRequest` with the `~`-delimited Location regex workaround.
241
- **cashRefund and creditMemo require a header-level location** on create. The create webhook can
242
- lag **~20s** before the row appears (read-after-write/processing) — wait before asserting.
241
+ ## E2E testing playbook + harness inventory (run this cold)
242
+ This section is the reusable procedure for verifying the NetSuite → `Forecast.Sales` path
243
+ end-to-end against a **local** Forecast mirror. It is written to be runnable in a future
244
+ session with no prior context.
245
+
246
+ ### Harness inventory (`test/@dave/`, all PHP)
247
+ All harnesses boot worker2/_underscore by `chdir('C:/Users/dfranks/www/worker2'); require
248
+ 'index.php';` and drive the live `_underscore` REST client (`_Component_Api_Netsuite`).
249
+ **Run them with xampp8 PHP 8.0:** `C:\xampp8\php\php.exe test/@dave/<harness>.php <action> …`
250
+ (local `_underscore` floor is PHP 8.0.0).
251
+
252
+ | Harness | Type | Actions |
253
+ |---|---|---|
254
+ | `test_invoice_lifecycle.php` | invoice | `check`, `create`, `get <id>`, `sync <id>`, `update <id> <date>`, `delete <id>`, `recent` (plus `amq`/`deployments`/`locations` SuiteQL probes) |
255
+ | `test_creditmemo_lifecycle.php` | creditMemo | same per-type action set |
256
+ | `test_cashsale_lifecycle.php` | cashSale | same per-type action set |
257
+ | `test_cashrefund_lifecycle.php` | cashRefund | same per-type action set |
258
+ | `test_je_lifecycle.php` | journalEntry | per-type set **plus** `sync` (`syncJournalEntry`) and `remove` (`removeAllJournalEntry`, direct); `create` makes a **balanced reversing JE** |
259
+ | `test_fetchrecord_routes.php` | **read-only probe** | calls `_Component_Api_Netsuite::fetchRecord` against a real opportunity/employee/salesOrder id — proves that method per route, **no side effects** |
260
+
261
+ **Per-type actions explained:**
262
+ - `check` — boot + OAuth only, **no writes** (smoke test connectivity/creds).
263
+ - `create` — create a real record in NetSuite (tiny `$0`/`$0.01`).
264
+ - `get <id>` — REST GET the record.
265
+ - `sync <id>` — **DIRECT** `_Component_Forecast_SaleImport::sync()` (or `syncJournalEntry()`)
266
+ against NetSuite reads, **local, no webhook**. This is the local-only execution path.
267
+ - `update <id> <date>` — PATCH `tranDate` (a tracked field) to exercise edit→update.
268
+ - `delete <id>` — delete the record in NetSuite.
269
+ - `recent` — list recent records of that type.
270
+
271
+ Each harness carries its **own** local create helper that POSTs via `_ApiRequest` and parses
272
+ the `Location` header with a **correct `~`-delimited regex** — a deliberate workaround for the
273
+ live `_Component_Api_Netsuite::createRecord()` `#`-delimiter bug (still unfixed; see the
274
+ [REST client doc](./netsuite-rest-client.md)). The framework is **not** edited to test.
275
+
276
+ ### The 6-step lifecycle pattern (every type)
277
+ 1. **create** → a real NetSuite record.
278
+ 2. **verify** the local `forecast.Sales` row(s) appeared.
279
+ 3. **update** the `tranDate`.
280
+ 4. **verify** the update landed (tracked-column change-detection).
281
+ 5. **delete** the record.
282
+ 6. **verify** the local row is gone (`removeAll`).
283
+
284
+ Poll the **local** Forecast DB between steps:
285
+ `C:\xampp8\mysql\bin\mysql.exe -u root` → db **`forecast`**, table **`Sales`** (filter on the
286
+ test internalId, or customerId `2` / itemId `25`).
287
+
288
+ ### Two execution modes
289
+ - **(a) WEBHOOK path** — NS create → AMQ enqueuer → ngrok → local worker2 → engine. Requires:
290
+ the **dev** AMQ enqueuer **deployed on each record type under test** with its **AUDIENCE
291
+ including the REST/M2M integration user** (Execute-As-Role does **not** control triggering —
292
+ AUDIENCE does); the **prod** enqueuer left **UNDEPLOYED** for isolation; and **ngrok up** at
293
+ the enqueuer's `DEV_OVERRIDE` endpoint. Exercises the real inbound path.
294
+ - **(b) LOCAL-ONLY path** (use when deployments are off / you can't touch NetSuite scripts) —
295
+ skip the webhook entirely: call the harness **`sync`** action, which runs
296
+ `_Component_Forecast_SaleImport::sync()` / `syncJournalEntry()` **directly** against NetSuite
297
+ reads. Exercises the same import + `fetchRecord` with **no enqueuer/ngrok dependency**.
298
+
299
+ ### Fixtures
300
+ - Customer **58** ("8 Test Company") → local `forecast.Customers` id **2**.
301
+ - Item **103741** ("Test Other Charge for Sale") → local `forecast.Items` id **25**.
302
+ - Location **5** ("Main") — required as a **header-level** field on creditMemo/cashSale/cashRefund
303
+ and **line-level** on invoice.
304
+ - **JE fixture:** subsidiary **1**; accounts **245** (Product Revenue / Income), **248**
305
+ (COGS Product / COGS), **215** (Clearing - CSS — ignored, neither revenue nor cost); set a
306
+ `reversalDate` so NetSuite auto-creates the reversal.
307
+
308
+ ### Expected local revenue signs / row outcomes
309
+ - **invoice** → `+` (a **$0** invoice still writes **one** row, revenue `0.00` — the engine does
310
+ not skip a $0 line).
311
+ - **cashSale** → `+`; status must be **`Deposited`** (a REST-created cash sale lands Deposited,
312
+ **not** the excluded `Unapproved Payment`).
313
+ - **creditMemo** → `−`.
314
+ - **cashRefund** → `−`.
315
+ - **JE** → `revenue = Σ(credit − debit)` over revenue-acct lines, `cost = Σ(debit − credit)`
316
+ over cost-acct lines, `profit = revenue − cost`. The **reversal** lands as the negation keyed
317
+ by its **own** internalId with `createdFromNetsuiteTransactionInternalId = the original`.
318
+
319
+ ### Timing / verification notes
320
+ - The create webhook can lag **~20s** before the row appears (read-after-write / processing) —
321
+ wait before asserting.
322
+ - On the debug webhook path there are **no** local app logs — confirm inbound arrival at the
323
+ ngrok inspector `http://127.0.0.1:4040/api/requests/http` (body is **base64**; decode before
324
+ grepping; buffer is ephemeral). See Gotchas.
325
+
326
+ ### Prod hygiene (critical — tests write to the single prod NetSuite account)
327
+ All harnesses create **real** tiny (`$0`/`$0.01`) records in the **single prod NetSuite
328
+ account (1095849)**. The legacy 5-min pull cron
329
+ (`worker/crons/toga2/forecast2/import_sales.php`) can pull a briefly-alive test record into
330
+ **prod** `Forecast.Sales` **independent of the webhook code**. After each test batch:
331
+ 1. Query the **prod reader** (`reader1.core.database.togahub.com`, `Forecast.Sales`) for the
332
+ test internalIds.
333
+ 2. **Guarded-DELETE exact matches** (customerId `2` / itemId `25` / revenue `±0.01`) via the
334
+ **prod writer**.
335
+ Keep **create→delete windows short** so most records slip between cron runs; or pause
336
+ `import_sales` during a batch. (The daily discrepancy-fix also cleans stragglers since the
337
+ record is deleted in NetSuite.)
243
338
 
244
339
  ## Gotchas / known issues
245
340
  - **Testing locally pollutes PROD unless you UNDEPLOY the prod AMQ enqueuer first.** Each sale
@@ -287,6 +382,16 @@ Actions: `check | create | get | update | delete | sync | recent | amq | deploym
287
382
  - The cron's sign handling is not portable here — see Sign convention.
288
383
 
289
384
  ## Change history
385
+ - 2026-06-26 — **Documented the full e2e testing playbook + harness inventory to run cold**
386
+ (6-step create→verify→update→verify→delete→verify lifecycle; per-type harness action sets;
387
+ the two execution modes — WEBHOOK vs LOCAL-ONLY `sync` — fixtures, expected signs, prod
388
+ hygiene). Added the read-only `test/@dave/test_fetchrecord_routes.php` probe. Recorded that
389
+ **`fetchRecord` moved from `_Component_Forecast_Db` to `_Component_Api_Netsuite`** (beside
390
+ `createRecord`; 7 callers reprefixed — SaleImport ×2, Opportunity ×3, SalesOrder ×2;
391
+ verify-the-class-before-relying). Added the **`git grep` caller-enumeration gotcha**: build
392
+ caller inventories for the shared NS helpers with `git grep -nF` per repo, not the fuzzy
393
+ search, which silently missed the production `SalesOrder.php` `createRecord` caller and caused
394
+ a reverted refactor. (dfranks)
290
395
  - 2026-06-26 — **Hardened + verified the full webhook path for all four sale types end-to-end**
291
396
  (create→insert, edit→update, delete→removeAll) with sign conventions reconfirmed (invoice/cashSale
292
397
  +, creditMemo/cashRefund −). Recorded durable gotchas: a UE enqueuer fires for a REST/M2M user only
@@ -6,7 +6,7 @@ project: _Underscore
6
6
  client: shared
7
7
  type: feature
8
8
  status: active
9
- updated: 2026-06-25
9
+ updated: 2026-06-26
10
10
  owners: ["dfranks"]
11
11
  files:
12
12
  - _underscore/Component/Api/Netsuite/Netsuite.php
@@ -64,6 +64,24 @@ through `_ApiRequest` directly (mirroring `send()`'s auth/endpoint/header setup)
64
64
  > value carried in the thrown exception message (the lifecycle harness ships its own correct
65
65
  > `~`-delimited parse to sidestep this without editing the framework). Confirmed live 2026-06-26.
66
66
 
67
+ ### Read — `fetchRecord(string $route, string $label): array`
68
+
69
+ A generic NS REST **GET + decode** primitive. **Moved here from `_Component_Forecast_Db`**
70
+ (2026-06-26) so it sits beside `createRecord` with the other REST primitives — it is not
71
+ Forecast-specific. Its **7 callers** were reprefixed to `_Component_Api_Netsuite::fetchRecord`:
72
+ `SaleImport.php` ×2, `Opportunity.php` ×3, `SalesOrder.php` ×2 (static git-grep: zero stale
73
+ `_Component_Forecast_Db::fetchRecord` refs; runtime-verified across all 5 SaleImport types +
74
+ opportunity/employee/salesOrder routes via `test/@dave/test_fetchrecord_routes.php`).
75
+ **Verify the class before relying on it** — it was just relocated.
76
+
77
+ > **Enumerate callers of this class's shared helpers with `git grep`, not the fuzzy/agent
78
+ > search.** When moving/renaming `createRecord` / `fetchRecord`, build the caller inventory with
79
+ > `git grep -nF` **per repo** — the fuzzy search silently **missed**
80
+ > `worker2/Worker/Netsuite/SalesOrder.php:925` (`createRecord`, the SalesOrder SOAP→REST
81
+ > outbound push — a real **production** caller), producing a wrong "zero callers / test-only"
82
+ > conclusion and a broken refactor that had to be reverted. Search case-insensitively for
83
+ > `::method`, `function method`, and string/callable forms; confirm zero stale refs afterward.
84
+
67
85
  ### Update — reuse `send('PATCH', $route, $body)`
68
86
 
69
87
  Updates do **not** need a new helper. A NetSuite record PATCH returns 204 with no body, and
@@ -103,6 +121,12 @@ doc.)
103
121
 
104
122
  ## Change history
105
123
 
124
+ - 2026-06-26 — **`fetchRecord` (generic NS REST GET + decode) moved onto this class** from
125
+ `_Component_Forecast_Db`, beside `createRecord`; 7 callers reprefixed (SaleImport ×2,
126
+ Opportunity ×3, SalesOrder ×2), git-grep clean + runtime-verified. Recorded the
127
+ caller-enumeration lesson: use `git grep -nF` per repo (not the fuzzy search, which missed the
128
+ production `SalesOrder.php` `createRecord` caller and caused a reverted refactor) when
129
+ moving/renaming these shared NS helpers. (dfranks)
106
130
  - 2026-06-25 — **Recorded a live bug in `createRecord()`'s trailing-id regex.** The pattern
107
131
  `'#/(\d+)(?:[?#]|$)#'` uses `#` as both the PCRE delimiter and a class member, so it always throws
108
132
  `Unknown modifier ']'` — *after* the record is created, breaking the outbound create/push path.
@@ -10,7 +10,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
10
10
  - **togaview** (TOGa View) — 6 doc(s) → [1.0/apps/togaview/INDEX.md](1.0/apps/togaview/INDEX.md)
11
11
  - **webhook** (Webhook) — 1 doc(s) → [1.0/apps/webhook/INDEX.md](1.0/apps/webhook/INDEX.md)
12
12
  - **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
13
- - **test** (Test) — 11 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
13
+ - **test** (Test) — 12 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
14
14
  - **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
15
15
  - **tools** (Tools) — 5 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
16
16
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "toga-ai",
3
- "version": "1.0.217",
3
+ "version": "1.0.219",
4
4
  "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
5
5
  "keywords": [
6
6
  "claude",