toga-ai 1.0.217 → 1.0.219
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/knowledge/1.0/apps/test/INDEX.md +1 -0
- package/knowledge/1.0/apps/test/architecture.md +4 -0
- package/knowledge/1.0/apps/test/features/goagilant-to-togatech-email-migration.md +134 -0
- package/knowledge/2.0/apps/_underscore/INDEX.md +1 -1
- package/knowledge/2.0/apps/_underscore/features/forecast-sale-import.md +131 -26
- package/knowledge/2.0/apps/_underscore/features/netsuite-rest-client.md +25 -1
- package/knowledge/INDEX.md +1 -1
- package/package.json +1 -1
|
@@ -8,6 +8,7 @@
|
|
|
8
8
|
| [Create Elastic Beanstalk Environment (script)](features/create-elastic-beanstalk.md) | `team/aws/create_elastic_beanstalk.php` is a **standalone** (no `App_` framework) constants-driven PHP generator. | test/team/aws/create_elastic_beanstalk.php |
|
|
9
9
|
| [Developer Generators (password, UUID)](features/dev-generators.md) | Two tiny **1.0 `App_` framework** convenience scripts for everyday developer needs. | test/team/generate_password.php, test/team/uuid.php |
|
|
10
10
|
| [Forecast vs NetSuite Discrepancy Analysis](features/forecast-netsuite-discrepancy-analysis.md) | `team/forecast-netsuite/discrepancy_analysis.php` detects discrepancies between our **Forecast database** and **NetSuite** (the source of truth for all sales da | test/team/forecast-netsuite/discrepancy_analysis.php |
|
|
11
|
+
| [@goagilant.com → @togatech.com Email-Domain Migration (1.0 + 2.0)](features/goagilant-to-togatech-email-migration.md) | Reference + technique for migrating the company email domain `@goagilant.com` → `@togatech.com` across **both** platforms. | migrate_goagilant_to_togatech_2026-06-26.sql, migrate_goagilant_to_togatech_LEGACY_2026-06-26.sql |
|
|
11
12
|
| [TableView Builder (2.0 TableViews SQL generator)](features/tableview-builder.md) | `team/tableViewBuilder/` generates SQL `INSERT` statements for the **2.0 `TableViews`**, `TableViewFields`, and `TableViewJoins` tables from a plain SQL `SELECT | test/team/tableViewBuilder/TableViewGenerator.php, test/team/tableViewBuilder/index.php, test/team/tableViewBuilder/Instructions.md |
|
|
12
13
|
| [Talos Knowledge Base Pipeline (Uploader + Processor)](features/talos-kb-pipeline.md) | `team/talos/` holds the two-script web tooling that feeds the **TOGa Talos** (TOGa IQ) AI knowledge bases. | test/team/talos/kb_uploader.php, test/team/talos/kb_processor.php, test/team/talos/kb_processor.ini |
|
|
13
14
|
| [TOGa 2.0 Client Onboarding SQL Generator](features/toga2-client-onboarding-sql.md) | `team/generate_toga2_onboarding_sql.php` generates the SQL needed to **onboard a new client** onto the 2.0 platform. | test/team/generate_toga2_onboarding_sql.php |
|
|
@@ -21,6 +21,7 @@ related:
|
|
|
21
21
|
- ./features/url-domain-markdown-document.md
|
|
22
22
|
- ./features/dev-generators.md
|
|
23
23
|
- ./features/talos-kb-pipeline.md
|
|
24
|
+
- ./features/goagilant-to-togatech-email-migration.md
|
|
24
25
|
---
|
|
25
26
|
|
|
26
27
|
## Summary
|
|
@@ -83,6 +84,7 @@ manually against the target database; they do not execute changes themselves.
|
|
|
83
84
|
| `generate_toga2_user_access_sql.php` | 1.0 | 2.0 client DBs | [toga2-user-cross-client-access-sql](./features/toga2-user-cross-client-access-sql.md) |
|
|
84
85
|
| `generate_password.php`, `uuid.php` | 1.0 | dev utility | [dev-generators](./features/dev-generators.md) |
|
|
85
86
|
| `talos/kb_uploader.php`, `talos/kb_processor.php` | standalone | S3 + AWS Bedrock KBs | [talos-kb-pipeline](./features/talos-kb-pipeline.md) |
|
|
87
|
+
| `migrate_goagilant_to_togatech_*.sql` | standalone | 2.0 `Client_` DBs + 1.0 schemas | [goagilant-to-togatech-email-migration](./features/goagilant-to-togatech-email-migration.md) |
|
|
86
88
|
|
|
87
89
|
**Retired / not documented:** `team/DOA_gitbook/` is being retired.
|
|
88
90
|
|
|
@@ -91,3 +93,5 @@ manually against the target database; they do not execute changes themselves.
|
|
|
91
93
|
- 2026-06-16 — Initial capture of the `team/` folder. Registered repo in `registry.json`.
|
|
92
94
|
- 2026-06-22 — Un-retired `team/DOA_talos/` → `team/talos/`; documented its uploader +
|
|
93
95
|
processor in [talos-kb-pipeline](./features/talos-kb-pipeline.md).
|
|
96
|
+
- 2026-06-26 — Documented the @goagilant.com → @togatech.com email-domain migration SQL
|
|
97
|
+
artifacts in [goagilant-to-togatech-email-migration](./features/goagilant-to-togatech-email-migration.md).
|
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "@goagilant.com → @togatech.com Email-Domain Migration (1.0 + 2.0)"
|
|
3
|
+
framework: "1.0"
|
|
4
|
+
repo: test
|
|
5
|
+
project: Test
|
|
6
|
+
client: shared
|
|
7
|
+
type: feature
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-06-26
|
|
10
|
+
owners: [bala]
|
|
11
|
+
files:
|
|
12
|
+
- migrate_goagilant_to_togatech_2026-06-26.sql
|
|
13
|
+
- migrate_goagilant_to_togatech_LEGACY_2026-06-26.sql
|
|
14
|
+
related:
|
|
15
|
+
- ../architecture.md
|
|
16
|
+
- ./2-0-deployment-client-sql.md
|
|
17
|
+
- ../../../2.0/apps/_underscore/features/email-template-sending.md
|
|
18
|
+
- ../../../2.0/apps/_underscore/features/per-client-database-connections.md
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Summary
|
|
22
|
+
|
|
23
|
+
Reference + technique for migrating the company email domain `@goagilant.com` →
|
|
24
|
+
`@togatech.com` across **both** platforms. Two standalone SQL artifacts (run manually
|
|
25
|
+
against the right cluster, not framework code) were produced:
|
|
26
|
+
|
|
27
|
+
- `migrate_goagilant_to_togatech_2026-06-26.sql` — 2.0 prod, per-tenant `Client_` DBs.
|
|
28
|
+
- `migrate_goagilant_to_togatech_LEGACY_2026-06-26.sql` — 1.0 legacy.
|
|
29
|
+
|
|
30
|
+
The durable value is the **email-column blast-radius map** for each environment and the
|
|
31
|
+
**two collision gotchas** that make a blanket find/replace unsafe. Use this before any
|
|
32
|
+
future email-domain change.
|
|
33
|
+
|
|
34
|
+
## Where @goagilant.com emails live
|
|
35
|
+
|
|
36
|
+
### 2.0 prod (per-tenant `Client_` DBs, prod-client cluster)
|
|
37
|
+
|
|
38
|
+
Value columns that actually held `@goagilant.com` data:
|
|
39
|
+
|
|
40
|
+
| Column | Notes |
|
|
41
|
+
|---|---|
|
|
42
|
+
| `Users.email` | **UNIQUE** — identity logins. See Client_True gotcha. |
|
|
43
|
+
| `ContactEmailAddresses.emailAddress` | |
|
|
44
|
+
| `EmailTemplates.sendFromEmailAddress` | Present in **all 32 clients**; outbound `From:` — deliverability-sensitive (sender-verification gate below). |
|
|
45
|
+
| `IntegrationsEmail.fromEmailAddress` | Outbound `From:` — deliverability-sensitive. |
|
|
46
|
+
| `SalesOrderEmailAddresses.emailAddress` | Composite UNIQUE `(salesOrderId, emailAddress)`. |
|
|
47
|
+
| `EmailTemplateOutgoingEmailAddresses.emailAddress` | Composite UNIQUE. |
|
|
48
|
+
|
|
49
|
+
Columns that exist but held **zero** `@goagilant.com` data (skip):
|
|
50
|
+
`Campaigns.bookingEmailAddress`, `LocationEmailAddresses.emailAddress`,
|
|
51
|
+
`IntegrationsEmailOutgoingEmailAddresses.emailAddress`.
|
|
52
|
+
|
|
53
|
+
`ContactAttempts.c_contactEmail` exists **only** in `Client_True` (BDR).
|
|
54
|
+
|
|
55
|
+
Tenants with `Users.email` hits included Compass, CompassCanada, Managelife, Quad, Growrk,
|
|
56
|
+
Nychh, Prudential, Aig, Endeavorhealth, Erau, Masonite, Rate, Spglobal, Towfoundation, Wje,
|
|
57
|
+
Wmchealth, Ynhh, plus True (special-cased).
|
|
58
|
+
|
|
59
|
+
### 1.0 legacy
|
|
60
|
+
|
|
61
|
+
| Table | Hits | Notes |
|
|
62
|
+
|---|---|---|
|
|
63
|
+
| `TOGA.Users.email` | 1117 goagilant | Main platform login. **NOT unique** (PRIMARY on `id` only). togatech migration already ~half done: 1077 togatech rows exist; 7 goagilant overlap a togatech local-part. |
|
|
64
|
+
| `TOGA_<tenant>.ClientUsers.email` | 0 goagilant | Client-portal logins — clean across all 14 tenants. |
|
|
65
|
+
| `TOGA_<tenant>.Contacts.emailAddress` | ~255 | CRM. `relayEmailAddress` = 0 everywhere. |
|
|
66
|
+
| `Vision.Users` | 343 | Sibling-app login, 0 togatech twins. |
|
|
67
|
+
| `RetailServices.DashboardUsers` | 6 | 0 togatech twins. |
|
|
68
|
+
| `ITADPortal.Users` | 3 | 0 togatech twins. |
|
|
69
|
+
| `TOGaDeskAdmin.users` | 2 | 0 togatech twins. |
|
|
70
|
+
|
|
71
|
+
**Not migrated** (high-volume ancillary, out of scope): helpdesk `TOGaDeskSupport.tickets`
|
|
72
|
+
(15725), `Common` directory (~3700), RetailServices customer data, Netsuite/Store. The
|
|
73
|
+
**Advisor** and **GoAgilant (WordPress)** schemas are not queryable via the toga-db MCP tool
|
|
74
|
+
and were not assessed here.
|
|
75
|
+
|
|
76
|
+
## How it works (safe-rename technique)
|
|
77
|
+
|
|
78
|
+
Swap **only** the 14-char domain suffix, preserving the local-part exactly:
|
|
79
|
+
|
|
80
|
+
```sql
|
|
81
|
+
UPDATE SomeTable
|
|
82
|
+
SET col = CONCAT(LEFT(col, CHAR_LENGTH(col) - 14), '@togatech.com')
|
|
83
|
+
WHERE col LIKE '%@goagilant.com';
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Why not `REPLACE()`: `REPLACE` is case-sensitive (misses `@GoAgilant.com`) and can touch a
|
|
87
|
+
mid-string match. `LEFT`/`CONCAT` on the trailing 14 chars touches only the domain and keeps
|
|
88
|
+
apostrophes/odd local-parts intact (e.g. `do'connell@`). The `LIKE` is the natural
|
|
89
|
+
case-insensitive guard for the match.
|
|
90
|
+
|
|
91
|
+
For the non-unique 1.0 `TOGA.Users`, a guarded variant uses `NOT EXISTS` against a
|
|
92
|
+
materialized derived table of existing `@togatech.com` addresses, so the 7 would-be
|
|
93
|
+
duplicate logins are skipped rather than creating colliding identities.
|
|
94
|
+
|
|
95
|
+
## Scope (explicit developer instructions)
|
|
96
|
+
|
|
97
|
+
- **2.0:** exclude `Client_True.Users` (see gotcha); still migrate True's other email tables.
|
|
98
|
+
- **1.0:** scoped to **USERS + CONTACTS only** — explicitly exclude `Customers`,
|
|
99
|
+
`SMBContracts` (contracts/work orders), and send-from in 1.0.
|
|
100
|
+
- **Never** touch any `*Logs*` schemas.
|
|
101
|
+
|
|
102
|
+
## Sender-address gate (deliverability)
|
|
103
|
+
|
|
104
|
+
`EmailTemplates.sendFromEmailAddress` (~118 rows across all 2.0 clients) and
|
|
105
|
+
`IntegrationsEmail.fromEmailAddress` change the outbound `From:` domain. These were gated
|
|
106
|
+
as a separate step pending `togatech.com` sender verification (SPF/DKIM/DMARC). The domain
|
|
107
|
+
was **confirmed verified 2026-06-26**, so the sender-address step is cleared to run. Do not
|
|
108
|
+
flip a `From:` domain ahead of sender verification — it breaks deliverability.
|
|
109
|
+
|
|
110
|
+
## Gotchas
|
|
111
|
+
|
|
112
|
+
- **Client_True (2.0) UNIQUE twins.** `Users.email` is UNIQUE. Of 343 `@goagilant.com`
|
|
113
|
+
users, **325 already have a pre-staged `@togatech.com` twin** (twins mostly never logged
|
|
114
|
+
in, `dtLastLogin` NULL — a togatech cutover was partially staged). A blanket rename
|
|
115
|
+
collides on the UNIQUE constraint (fails, or with `IGNORE` **silently skips 325**). Of the
|
|
116
|
+
18 no-twin rows, ~13 are service/SFTP/support accounts (`odpsftp@`, `staplessftp@`,
|
|
117
|
+
`support@`, …) whose rename could break integrations. **Decision:** exclude
|
|
118
|
+
`Client_True.Users.email` from the migration; migrate only True's 64 non-identity rows
|
|
119
|
+
(ContactEmailAddresses 56, Campaigns 1, ContactAttempts 1, EmailTemplates.sendFromEmailAddress
|
|
120
|
+
6), which carry no UNIQUE-on-email risk.
|
|
121
|
+
- **Legacy `TOGA.Users` is non-unique and half-migrated.** PRIMARY is on `id` only, so a
|
|
122
|
+
rename can produce duplicate login emails. 1077 togatech rows already exist and 7 goagilant
|
|
123
|
+
rows overlap a togatech local-part — use the `NOT EXISTS`-guarded variant to skip those 7.
|
|
124
|
+
- **Run against the right cluster.** 2.0 lives in per-tenant `Client_` DBs on the prod-client
|
|
125
|
+
cluster; 1.0 in `TOGA` / `TOGA_<tenant>` / sibling-app schemas. Review the generated SQL
|
|
126
|
+
before executing — these are manual-run artifacts, nothing auto-executes.
|
|
127
|
+
|
|
128
|
+
## Change history
|
|
129
|
+
|
|
130
|
+
- 2026-06-26 — Initial capture. Mapped 2.0 + 1.0 email columns, recorded the Client_True
|
|
131
|
+
UNIQUE-twin and legacy non-unique-login collision gotchas, the LEFT/CONCAT safe-rename
|
|
132
|
+
technique, scope exclusions, and the cleared sender-verification gate. (bala)
|
|
133
|
+
</content>
|
|
134
|
+
</invoke>
|
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
| [Assortment Name Translation (AssortmentTranslations sidecar)](features/assortment-name-translation.md) | Serves Assortment (product-grouping) **names** in multiple languages by adding a per-language **sidecar** table `AssortmentTranslations`, reusing the platform's | _underscore/Model/Client/AssortmentTranslation.php, dbchanges2/Client/2026-06-26a - AssortmentTranslations.sql, dbchanges2/Core/2026-06-26a - AssortmentTranslationsRecord.sql, dbchanges2/Client/2026-06-26b - AssortmentTranslationsAcl.sql |
|
|
8
8
|
| [Carrier Shipping Labels (UPS/FedEx) & NetSuite Item Fulfillment](features/carrier-shipping-labels.md) | Backend mechanics behind TOGa Supply's Fulfill & Ship: buying a carrier label (UPS/FedEx), persisting it, and creating the NetSuite Item Fulfillment with tracki | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillments/TrackingNumber.php, _underscore/Component/Library/LabelPdf/LabelPdf.php, _underscore/Component/Library/Carriers/Ups/Ups.php, _underscore/Component/Library/Carriers/Fedex/Fedex.php, _underscore/Trait/Netsuite/ItemFulfillment.php, _underscore/Trait/Netsuite/SalesOrder.php, _underscore/Component/Library/NetSuite/NetSuite.php, _underscore/Model/Client/TrackingNumber.php, _underscore/Model/Client/ShippingMethod.php, _underscore/Model.php, _underscore/Cloud.php |
|
|
9
9
|
| [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
|
|
10
|
-
| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
|
|
10
|
+
| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, _underscore/Component/Api/Netsuite/Netsuite.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/JournalEntry.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, dbchanges2/Forecast/2026-06-26a - Add journalEntry to Sales transaction type enum.sql, test/@dave/test_invoice_lifecycle.php, test/@dave/test_je_lifecycle.php, test/@dave/test_creditmemo_lifecycle.php, test/@dave/test_cashsale_lifecycle.php, test/@dave/test_cashrefund_lifecycle.php, test/@dave/test_fetchrecord_routes.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
|
|
11
11
|
| [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php |
|
|
12
12
|
| [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
|
|
13
13
|
| [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |
|
|
@@ -11,6 +11,7 @@ owners: [dfranks]
|
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Component/Forecast/SaleImport/SaleImport.php
|
|
13
13
|
- _underscore/Component/Forecast/Db/Db.php
|
|
14
|
+
- _underscore/Component/Api/Netsuite/Netsuite.php
|
|
14
15
|
- worker2/Worker/Netsuite/Invoice.php
|
|
15
16
|
- worker2/Worker/Netsuite/CashSale.php
|
|
16
17
|
- worker2/Worker/Netsuite/CreditMemo.php
|
|
@@ -24,6 +25,7 @@ files:
|
|
|
24
25
|
- test/@dave/test_creditmemo_lifecycle.php
|
|
25
26
|
- test/@dave/test_cashsale_lifecycle.php
|
|
26
27
|
- test/@dave/test_cashrefund_lifecycle.php
|
|
28
|
+
- test/@dave/test_fetchrecord_routes.php
|
|
27
29
|
- test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js
|
|
28
30
|
- test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js
|
|
29
31
|
related:
|
|
@@ -71,7 +73,7 @@ Engine public surface: `sync(int $internalId, string $type)` and
|
|
|
71
73
|
|
|
72
74
|
## How it works
|
|
73
75
|
Per-record flow in `sync`:
|
|
74
|
-
1. REST GET the record (`expandSubResources=true`) via `
|
|
76
|
+
1. REST GET the record (`expandSubResources=true`) via `_Component_Api_Netsuite::fetchRecord`.
|
|
75
77
|
2. **Per-type status gate** → if excluded, call `removeAll` and stop. Exclusions: invoice
|
|
76
78
|
`Voided`/`Rejected`; cashSale `Unapproved Payment`; creditMemo `Voided`; cashRefund has
|
|
77
79
|
no gate. **A cashSale created via REST lands in status `Deposited`** (NOT the excluded
|
|
@@ -88,14 +90,32 @@ Per-record flow in `sync`:
|
|
|
88
90
|
`removeAll(internalId)` deletes all rows for that transaction (used by delete events and by
|
|
89
91
|
the status gate).
|
|
90
92
|
|
|
91
|
-
### Shared helpers on `_Component_Forecast_Db`
|
|
92
|
-
-
|
|
93
|
-
|
|
93
|
+
### Shared NS helpers — `fetchRecord` lives on `_Component_Api_Netsuite`, the rest on `_Component_Forecast_Db`
|
|
94
|
+
- **`_Component_Api_Netsuite::fetchRecord(route, label)`** — NS REST GET + decode. **Moved
|
|
95
|
+
here** (beside `createRecord`) from `_Component_Forecast_Db` — it is a generic REST read,
|
|
96
|
+
not Forecast-specific, so it belongs with the other REST primitives. **Verify the class
|
|
97
|
+
before relying on it** (it was relocated this session). The **7 callers** were all
|
|
98
|
+
reprefixed to `_Component_Api_Netsuite::fetchRecord`: `SaleImport.php` ×2, `Opportunity.php`
|
|
99
|
+
×3, `SalesOrder.php` ×2 (static git-grep: zero stale `_Component_Forecast_Db::fetchRecord`
|
|
100
|
+
refs, all 7 resolve, lint clean; runtime-verified across all 5 SaleImport types +
|
|
101
|
+
opportunity/employee/salesOrder routes via `test_fetchrecord_routes.php`).
|
|
102
|
+
- **Still on `_Component_Forecast_Db`** (Forecast-specific): `collectItemLines(record)` +
|
|
103
|
+
`nextPageRoute(links)` (`rel=next` sublist pagination), `lookupId`, `toSqlDate`.
|
|
94
104
|
|
|
95
105
|
`Opportunity.php` and `SalesOrder.php` were refactored onto these shared versions; their
|
|
96
106
|
private `fetchRecord`/`collectAllItemLines`/`nextPageRoute` copies (and SalesOrder's
|
|
97
107
|
orphaned `MAX_SUBLIST_PAGES` const) were deleted.
|
|
98
108
|
|
|
109
|
+
> **Process gotcha — enumerate callers of these shared NS helpers with `git grep`, not the
|
|
110
|
+
> fuzzy/agent search.** Any move/rename of `_Component_Api_Netsuite::createRecord` /
|
|
111
|
+
> `fetchRecord` (or the other shared NS helpers) must build its caller inventory with
|
|
112
|
+
> `git grep -nF` **per repo** — the fuzzy search tool silently **missed**
|
|
113
|
+
> `worker2/Worker/Netsuite/SalesOrder.php:925` (`createRecord`, the SalesOrder SOAP→REST
|
|
114
|
+
> outbound push — a real **production** caller) on a first pass, producing a wrong "zero
|
|
115
|
+
> callers / test-only" conclusion and a broken refactor that had to be reverted. Search
|
|
116
|
+
> case-insensitively for `::method`, `function method`, and string/callable forms; verify
|
|
117
|
+
> **zero stale refs** after editing.
|
|
118
|
+
|
|
99
119
|
## Sign convention (the load-bearing decision)
|
|
100
120
|
The **raw REST record** returns each line `amount`/`costEstimate` **POSITIVE for all four
|
|
101
121
|
types** (verified by live probe 2026-06-24 against NS account 1095849: creditMemo lines
|
|
@@ -218,28 +238,103 @@ events as specific **subtypes** (`inventoryItem`, `nonInventoryResaleItem`, `kit
|
|
|
218
238
|
would be needed only for a future real-time item webhook, **not** for this Sales importer
|
|
219
239
|
(which keeps items fresh via the hourly item pull cron plus the inline self-heal).
|
|
220
240
|
|
|
221
|
-
##
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
241
|
+
## E2E testing playbook + harness inventory (run this cold)
|
|
242
|
+
This section is the reusable procedure for verifying the NetSuite → `Forecast.Sales` path
|
|
243
|
+
end-to-end against a **local** Forecast mirror. It is written to be runnable in a future
|
|
244
|
+
session with no prior context.
|
|
245
|
+
|
|
246
|
+
### Harness inventory (`test/@dave/`, all PHP)
|
|
247
|
+
All harnesses boot worker2/_underscore by `chdir('C:/Users/dfranks/www/worker2'); require
|
|
248
|
+
'index.php';` and drive the live `_underscore` REST client (`_Component_Api_Netsuite`).
|
|
249
|
+
**Run them with xampp8 PHP 8.0:** `C:\xampp8\php\php.exe test/@dave/<harness>.php <action> …`
|
|
250
|
+
(local `_underscore` floor is PHP 8.0.0).
|
|
251
|
+
|
|
252
|
+
| Harness | Type | Actions |
|
|
253
|
+
|---|---|---|
|
|
254
|
+
| `test_invoice_lifecycle.php` | invoice | `check`, `create`, `get <id>`, `sync <id>`, `update <id> <date>`, `delete <id>`, `recent` (plus `amq`/`deployments`/`locations` SuiteQL probes) |
|
|
255
|
+
| `test_creditmemo_lifecycle.php` | creditMemo | same per-type action set |
|
|
256
|
+
| `test_cashsale_lifecycle.php` | cashSale | same per-type action set |
|
|
257
|
+
| `test_cashrefund_lifecycle.php` | cashRefund | same per-type action set |
|
|
258
|
+
| `test_je_lifecycle.php` | journalEntry | per-type set **plus** `sync` (`syncJournalEntry`) and `remove` (`removeAllJournalEntry`, direct); `create` makes a **balanced reversing JE** |
|
|
259
|
+
| `test_fetchrecord_routes.php` | **read-only probe** | calls `_Component_Api_Netsuite::fetchRecord` against a real opportunity/employee/salesOrder id — proves that method per route, **no side effects** |
|
|
260
|
+
|
|
261
|
+
**Per-type actions explained:**
|
|
262
|
+
- `check` — boot + OAuth only, **no writes** (smoke test connectivity/creds).
|
|
263
|
+
- `create` — create a real record in NetSuite (tiny `$0`/`$0.01`).
|
|
264
|
+
- `get <id>` — REST GET the record.
|
|
265
|
+
- `sync <id>` — **DIRECT** `_Component_Forecast_SaleImport::sync()` (or `syncJournalEntry()`)
|
|
266
|
+
against NetSuite reads, **local, no webhook**. This is the local-only execution path.
|
|
267
|
+
- `update <id> <date>` — PATCH `tranDate` (a tracked field) to exercise edit→update.
|
|
268
|
+
- `delete <id>` — delete the record in NetSuite.
|
|
269
|
+
- `recent` — list recent records of that type.
|
|
270
|
+
|
|
271
|
+
Each harness carries its **own** local create helper that POSTs via `_ApiRequest` and parses
|
|
272
|
+
the `Location` header with a **correct `~`-delimited regex** — a deliberate workaround for the
|
|
273
|
+
live `_Component_Api_Netsuite::createRecord()` `#`-delimiter bug (still unfixed; see the
|
|
274
|
+
[REST client doc](./netsuite-rest-client.md)). The framework is **not** edited to test.
|
|
275
|
+
|
|
276
|
+
### The 6-step lifecycle pattern (every type)
|
|
277
|
+
1. **create** → a real NetSuite record.
|
|
278
|
+
2. **verify** the local `forecast.Sales` row(s) appeared.
|
|
279
|
+
3. **update** the `tranDate`.
|
|
280
|
+
4. **verify** the update landed (tracked-column change-detection).
|
|
281
|
+
5. **delete** the record.
|
|
282
|
+
6. **verify** the local row is gone (`removeAll`).
|
|
283
|
+
|
|
284
|
+
Poll the **local** Forecast DB between steps:
|
|
285
|
+
`C:\xampp8\mysql\bin\mysql.exe -u root` → db **`forecast`**, table **`Sales`** (filter on the
|
|
286
|
+
test internalId, or customerId `2` / itemId `25`).
|
|
287
|
+
|
|
288
|
+
### Two execution modes
|
|
289
|
+
- **(a) WEBHOOK path** — NS create → AMQ enqueuer → ngrok → local worker2 → engine. Requires:
|
|
290
|
+
the **dev** AMQ enqueuer **deployed on each record type under test** with its **AUDIENCE
|
|
291
|
+
including the REST/M2M integration user** (Execute-As-Role does **not** control triggering —
|
|
292
|
+
AUDIENCE does); the **prod** enqueuer left **UNDEPLOYED** for isolation; and **ngrok up** at
|
|
293
|
+
the enqueuer's `DEV_OVERRIDE` endpoint. Exercises the real inbound path.
|
|
294
|
+
- **(b) LOCAL-ONLY path** (use when deployments are off / you can't touch NetSuite scripts) —
|
|
295
|
+
skip the webhook entirely: call the harness **`sync`** action, which runs
|
|
296
|
+
`_Component_Forecast_SaleImport::sync()` / `syncJournalEntry()` **directly** against NetSuite
|
|
297
|
+
reads. Exercises the same import + `fetchRecord` with **no enqueuer/ngrok dependency**.
|
|
298
|
+
|
|
299
|
+
### Fixtures
|
|
300
|
+
- Customer **58** ("8 Test Company") → local `forecast.Customers` id **2**.
|
|
301
|
+
- Item **103741** ("Test Other Charge for Sale") → local `forecast.Items` id **25**.
|
|
302
|
+
- Location **5** ("Main") — required as a **header-level** field on creditMemo/cashSale/cashRefund
|
|
303
|
+
and **line-level** on invoice.
|
|
304
|
+
- **JE fixture:** subsidiary **1**; accounts **245** (Product Revenue / Income), **248**
|
|
305
|
+
(COGS Product / COGS), **215** (Clearing - CSS — ignored, neither revenue nor cost); set a
|
|
306
|
+
`reversalDate` so NetSuite auto-creates the reversal.
|
|
307
|
+
|
|
308
|
+
### Expected local revenue signs / row outcomes
|
|
309
|
+
- **invoice** → `+` (a **$0** invoice still writes **one** row, revenue `0.00` — the engine does
|
|
310
|
+
not skip a $0 line).
|
|
311
|
+
- **cashSale** → `+`; status must be **`Deposited`** (a REST-created cash sale lands Deposited,
|
|
312
|
+
**not** the excluded `Unapproved Payment`).
|
|
313
|
+
- **creditMemo** → `−`.
|
|
314
|
+
- **cashRefund** → `−`.
|
|
315
|
+
- **JE** → `revenue = Σ(credit − debit)` over revenue-acct lines, `cost = Σ(debit − credit)`
|
|
316
|
+
over cost-acct lines, `profit = revenue − cost`. The **reversal** lands as the negation keyed
|
|
317
|
+
by its **own** internalId with `createdFromNetsuiteTransactionInternalId = the original`.
|
|
318
|
+
|
|
319
|
+
### Timing / verification notes
|
|
320
|
+
- The create webhook can lag **~20s** before the row appears (read-after-write / processing) —
|
|
321
|
+
wait before asserting.
|
|
322
|
+
- On the debug webhook path there are **no** local app logs — confirm inbound arrival at the
|
|
323
|
+
ngrok inspector `http://127.0.0.1:4040/api/requests/http` (body is **base64**; decode before
|
|
324
|
+
grepping; buffer is ephemeral). See Gotchas.
|
|
325
|
+
|
|
326
|
+
### Prod hygiene (critical — tests write to the single prod NetSuite account)
|
|
327
|
+
All harnesses create **real** tiny (`$0`/`$0.01`) records in the **single prod NetSuite
|
|
328
|
+
account (1095849)**. The legacy 5-min pull cron
|
|
329
|
+
(`worker/crons/toga2/forecast2/import_sales.php`) can pull a briefly-alive test record into
|
|
330
|
+
**prod** `Forecast.Sales` **independent of the webhook code**. After each test batch:
|
|
331
|
+
1. Query the **prod reader** (`reader1.core.database.togahub.com`, `Forecast.Sales`) for the
|
|
332
|
+
test internalIds.
|
|
333
|
+
2. **Guarded-DELETE exact matches** (customerId `2` / itemId `25` / revenue `±0.01`) via the
|
|
334
|
+
**prod writer**.
|
|
335
|
+
Keep **create→delete windows short** so most records slip between cron runs; or pause
|
|
336
|
+
`import_sales` during a batch. (The daily discrepancy-fix also cleans stragglers since the
|
|
337
|
+
record is deleted in NetSuite.)
|
|
243
338
|
|
|
244
339
|
## Gotchas / known issues
|
|
245
340
|
- **Testing locally pollutes PROD unless you UNDEPLOY the prod AMQ enqueuer first.** Each sale
|
|
@@ -287,6 +382,16 @@ Actions: `check | create | get | update | delete | sync | recent | amq | deploym
|
|
|
287
382
|
- The cron's sign handling is not portable here — see Sign convention.
|
|
288
383
|
|
|
289
384
|
## Change history
|
|
385
|
+
- 2026-06-26 — **Documented the full e2e testing playbook + harness inventory to run cold**
|
|
386
|
+
(6-step create→verify→update→verify→delete→verify lifecycle; per-type harness action sets;
|
|
387
|
+
the two execution modes — WEBHOOK vs LOCAL-ONLY `sync` — fixtures, expected signs, prod
|
|
388
|
+
hygiene). Added the read-only `test/@dave/test_fetchrecord_routes.php` probe. Recorded that
|
|
389
|
+
**`fetchRecord` moved from `_Component_Forecast_Db` to `_Component_Api_Netsuite`** (beside
|
|
390
|
+
`createRecord`; 7 callers reprefixed — SaleImport ×2, Opportunity ×3, SalesOrder ×2;
|
|
391
|
+
verify-the-class-before-relying). Added the **`git grep` caller-enumeration gotcha**: build
|
|
392
|
+
caller inventories for the shared NS helpers with `git grep -nF` per repo, not the fuzzy
|
|
393
|
+
search, which silently missed the production `SalesOrder.php` `createRecord` caller and caused
|
|
394
|
+
a reverted refactor. (dfranks)
|
|
290
395
|
- 2026-06-26 — **Hardened + verified the full webhook path for all four sale types end-to-end**
|
|
291
396
|
(create→insert, edit→update, delete→removeAll) with sign conventions reconfirmed (invoice/cashSale
|
|
292
397
|
+, creditMemo/cashRefund −). Recorded durable gotchas: a UE enqueuer fires for a REST/M2M user only
|
|
@@ -6,7 +6,7 @@ project: _Underscore
|
|
|
6
6
|
client: shared
|
|
7
7
|
type: feature
|
|
8
8
|
status: active
|
|
9
|
-
updated: 2026-06-
|
|
9
|
+
updated: 2026-06-26
|
|
10
10
|
owners: ["dfranks"]
|
|
11
11
|
files:
|
|
12
12
|
- _underscore/Component/Api/Netsuite/Netsuite.php
|
|
@@ -64,6 +64,24 @@ through `_ApiRequest` directly (mirroring `send()`'s auth/endpoint/header setup)
|
|
|
64
64
|
> value carried in the thrown exception message (the lifecycle harness ships its own correct
|
|
65
65
|
> `~`-delimited parse to sidestep this without editing the framework). Confirmed live 2026-06-26.
|
|
66
66
|
|
|
67
|
+
### Read — `fetchRecord(string $route, string $label): array`
|
|
68
|
+
|
|
69
|
+
A generic NS REST **GET + decode** primitive. **Moved here from `_Component_Forecast_Db`**
|
|
70
|
+
(2026-06-26) so it sits beside `createRecord` with the other REST primitives — it is not
|
|
71
|
+
Forecast-specific. Its **7 callers** were reprefixed to `_Component_Api_Netsuite::fetchRecord`:
|
|
72
|
+
`SaleImport.php` ×2, `Opportunity.php` ×3, `SalesOrder.php` ×2 (static git-grep: zero stale
|
|
73
|
+
`_Component_Forecast_Db::fetchRecord` refs; runtime-verified across all 5 SaleImport types +
|
|
74
|
+
opportunity/employee/salesOrder routes via `test/@dave/test_fetchrecord_routes.php`).
|
|
75
|
+
**Verify the class before relying on it** — it was just relocated.
|
|
76
|
+
|
|
77
|
+
> **Enumerate callers of this class's shared helpers with `git grep`, not the fuzzy/agent
|
|
78
|
+
> search.** When moving/renaming `createRecord` / `fetchRecord`, build the caller inventory with
|
|
79
|
+
> `git grep -nF` **per repo** — the fuzzy search silently **missed**
|
|
80
|
+
> `worker2/Worker/Netsuite/SalesOrder.php:925` (`createRecord`, the SalesOrder SOAP→REST
|
|
81
|
+
> outbound push — a real **production** caller), producing a wrong "zero callers / test-only"
|
|
82
|
+
> conclusion and a broken refactor that had to be reverted. Search case-insensitively for
|
|
83
|
+
> `::method`, `function method`, and string/callable forms; confirm zero stale refs afterward.
|
|
84
|
+
|
|
67
85
|
### Update — reuse `send('PATCH', $route, $body)`
|
|
68
86
|
|
|
69
87
|
Updates do **not** need a new helper. A NetSuite record PATCH returns 204 with no body, and
|
|
@@ -103,6 +121,12 @@ doc.)
|
|
|
103
121
|
|
|
104
122
|
## Change history
|
|
105
123
|
|
|
124
|
+
- 2026-06-26 — **`fetchRecord` (generic NS REST GET + decode) moved onto this class** from
|
|
125
|
+
`_Component_Forecast_Db`, beside `createRecord`; 7 callers reprefixed (SaleImport ×2,
|
|
126
|
+
Opportunity ×3, SalesOrder ×2), git-grep clean + runtime-verified. Recorded the
|
|
127
|
+
caller-enumeration lesson: use `git grep -nF` per repo (not the fuzzy search, which missed the
|
|
128
|
+
production `SalesOrder.php` `createRecord` caller and caused a reverted refactor) when
|
|
129
|
+
moving/renaming these shared NS helpers. (dfranks)
|
|
106
130
|
- 2026-06-25 — **Recorded a live bug in `createRecord()`'s trailing-id regex.** The pattern
|
|
107
131
|
`'#/(\d+)(?:[?#]|$)#'` uses `#` as both the PCRE delimiter and a class member, so it always throws
|
|
108
132
|
`Unknown modifier ']'` — *after* the record is created, breaking the outbound create/push path.
|
package/knowledge/INDEX.md
CHANGED
|
@@ -10,7 +10,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
10
10
|
- **togaview** (TOGa View) — 6 doc(s) → [1.0/apps/togaview/INDEX.md](1.0/apps/togaview/INDEX.md)
|
|
11
11
|
- **webhook** (Webhook) — 1 doc(s) → [1.0/apps/webhook/INDEX.md](1.0/apps/webhook/INDEX.md)
|
|
12
12
|
- **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
|
|
13
|
-
- **test** (Test) —
|
|
13
|
+
- **test** (Test) — 12 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
|
|
14
14
|
- **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
|
|
15
15
|
- **tools** (Tools) — 5 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
|
|
16
16
|
|
package/package.json
CHANGED