toga-ai 1.0.208 → 1.0.210

package/knowledge/1.0/apps/tools/INDEX.md

@@ -6,3 +6,4 @@
 | [Tools — Developers Folder (UUID & Password Generators)](features/developer-tools.md) | The first two tools shipped in the Tools app, both under the **Developers** folder and gated to personas **Development Team** / **TOGa Technology**. | tools/mvc/developers/uuid/get.php, tools/mvc/developers/password/get.php |
 | [Tools Persona-Gated Navigation (App_Nav)](features/persona-gated-navigation.md) | `App_Nav` is the Tools app's two-level, **persona-gated** navigation. | tools/_/app/nav.php |
 | [Tools SAML SSO Consumer & Persona-Gated Auth (App_Auth)](features/saml-sso-auth.md) | `App_Auth` is the Tools app's authentication layer: it consumes the SAML gateway `?saml=` handoff (see the 2.0 SAML downstream integration contract), establishe | tools/_/app/auth.php, tools/mvc/sso/get.php, tools/mvc/login/get.php, tools/mvc/login/post.php, tools/mvc/logout/get.php |
+| [Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)](workflows/deploy-to-elastic-beanstalk-al2023.md) | How the **Tools** 1.0 app boots on Elastic Beanstalk running `PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. | tools/.ebextensions/004_http_to_https.config, tools/.ebextensions/006_mount-s3fs.config, tools/.ebextensions/007_setup_export_cache_folders.config, tools/.ebextensions/008_setup_ldap.config, tools/.ebextensions/009_setup_phpini.config, tools/.ebextensions/020_setup_git_libraries.config, tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config, tools/ebs/git.json |

package/knowledge/1.0/apps/tools/workflows/deploy-to-elastic-beanstalk-al2023.md

@@ -0,0 +1,126 @@
+---
+title: Deploying Tools to Elastic Beanstalk (PHP 8.5 / Amazon Linux 2023)
+framework: "1.0"
+repo: tools
+project: Tools
+client: shared
+type: workflow
+status: active
+updated: 2026-06-26
+owners: [jcardinal]
+files:
+  - tools/.ebextensions/004_http_to_https.config
+  - tools/.ebextensions/006_mount-s3fs.config
+  - tools/.ebextensions/007_setup_export_cache_folders.config
+  - tools/.ebextensions/008_setup_ldap.config
+  - tools/.ebextensions/009_setup_phpini.config
+  - tools/.ebextensions/020_setup_git_libraries.config
+  - tools/.ebextensions/050_register_instance_to_shared_application_load_balancer.config
+  - tools/ebs/git.json
+related:
+  - ../architecture.md
+---
+
+## Summary
+
+How the **Tools** 1.0 app boots on Elastic Beanstalk running
+`PHP 8.5 on 64bit Amazon Linux 2023/4.13.1 (aarch64)`. The EB platform was upgraded from
+Amazon Linux 2 (AL2) to Amazon Linux 2023 (AL2023), which changes package names, the proxy
+(nginx instead of Apache), the PHP ini scan directory, and how credentials reach the
+instance. The `.ebextensions/*.config` files run alphabetically and are merged by EB; getting
+them wrong leaves the app booting without `/var/www/library` and serving 500s on every
+request. This doc is the migration checklist and the durable AL2023 gotchas learned doing it.
+
+## AL2023 package migration checklist
+
+When migrating any 1.0 app's `.ebextensions` from AL2 to AL2023, rename/remove these yum
+packages (AL2023 uses different names for the same libraries):
+
+- `libstdc++48` → `libstdc++`
+- `php73-ldap` → `php-ldap` (in `008_setup_ldap.config`)
+- **Remove `libcurl: []` entirely** — AL2023 ships `libcurl-minimal` pre-installed, and adding
+  `libcurl` to the yum packages list causes a package-conflict error that fails the deploy.
+
+## How it works
+
+### PHP ini — write the file directly, do not run a script
+
+`009_setup_phpini.config` writes `/etc/php.d/application.ini` with a cfn-init **`files:`**
+block, **not** a container_command that runs a PHP script. On AL2023, `/etc/php.d/` is the
+PHP ini scan directory, and cfn-init `files:` runs before container_commands and before
+PHP-FPM starts, so the file is guaranteed present at startup with no PHP execution needed.
+
+The old approach ran `ebs/setup_phpini.php` (now superseded) which called
+`php_ini_scanned_files()`, parsed it to find the scan dir, and wrote there. On AL2023 that
+failed silently (it was wrapped in `ignoreErrors: true`), so `include_path` never picked up
+`/var/www/library`, producing fatal `require_once('_.php')` errors at runtime.
+
+The `application.ini` sets `include_path = ".:/var/www/library"`, upload/post limits (16M),
+`memory_limit = 2G`, `display_errors`, the `redis.so` extension, and the rediscluster session
+handler (`session.save_path` points at the ElastiCache cluster cfg endpoint). Session GC
+maxlifetime is 4 days.
+
+### Cloning the libraries — bash, not the CodePipeline PHP script
+
+`020_setup_git_libraries.config` clones the framework libraries onto the instance. The old
+`setup_git_libraries.php` (downloaded from S3) requires a DB connection to the `Logs` database
+and a matching row in the `CodePipelineEnvironments` table; for a new app with **no**
+CodePipeline it exits silently without cloning anything, so the app boots without
+`/var/www/library` and 500s on every request.
+
+The replacement is pure bash via a cfn-init `files:` block that writes
+`/tmp/clone_git_repos.sh` (mode `000755`) plus a `container_commands` entry that runs
+`bash /tmp/clone_git_repos.sh`. It clones:
+
+- `agilantsolutions/library`   branch `_production` → `/var/www/library`
+- `agilantsolutions/resources` branch `_production` → `/var/app/ondeck/resources`
+
+`ebs/git.json` was corrected to match: branch `_production` for both repos and the resources
+path set to `/var/app/ondeck/resources`.
+
+### Duplicate config neutralization
+
+EB merges every `.config` alphabetically; two configs with **identical** `container_commands`
+keys silently overwrite each other, causing unpredictable failures (the redis build ran twice;
+an http→https collision obscured failure diagnosis). Two duplicate pairs were neutralized
+(replaced with a comment), keeping the canonical one of each:
+
+- `004_enforce_http_to_https.config` (dup of `004_http_to_https.config`) — neutralized
+- `010_setup_redis_sessions.config` (dup of `010_setup_redis.config`) — neutralized
+
+### ignoreErrors on non-fatal infrastructure commands
+
+Several `container_commands` write to system paths that do not exist on AL2023 (e.g.
+`/etc/httpd/conf.d/` — AL2023 EB uses **nginx**, not Apache, as the reverse proxy). These were
+marked `ignoreErrors: true` in `004`, `006`, `007`, `009`, and `050` so a non-fatal failure
+does not block the deploy. This also covers `006`'s `01_credentials` step: modern EB uses IAM
+roles, so `AWS_SECRET_KEY` / `AWS_ACCESS_KEY_ID` are not injected into `$_SERVER` and the
+s3fs credential write fails cleanly. Use `ignoreErrors` only on genuinely non-fatal infra
+setup — the git clone (`020`) deliberately does **not** use it (see gotchas).
+
+## Gotchas (durable AL2023 / EB rules)
+
+- **Write new-deploy files to `/var/app/ondeck/<path>`, never `/var/www/html/<path>` during
+  `container_commands`.** While container_commands run, `/var/www/html` still symlinks to the
+  **old** app. After the deploy swap, `/var/app/ondeck` becomes `/var/app/current`
+  (= `/var/www/html`). Writing to `/var/www/html` during container_commands writes into the
+  currently-running old app.
+- **Only `/var/log/eb-activity.log` (and `cfn-init.log`) are captured in EB bundle snapshots.**
+  Custom log files under `/var/log/` are invisible unless you SSH the instance. Deploy scripts
+  should log to `/var/log/eb-activity.log`.
+- **CodePipeline-dependent scripts are no-ops for apps without a pipeline.**
+  `setup_git_libraries.php` silently does nothing when the DB has no `CodePipelineEnvironments`
+  row — fatal for a brand-new app, since nothing gets cloned and the failure is silent.
+- **Make clone failures block the deploy.** The bash clone script does `exit 1` on clone
+  failure so the deploy fails visibly, rather than succeeding and serving 500s.
+- **PATH is unreliable in the cfn-init execution environment** — resolve binaries defensively,
+  e.g. `GIT=$(which git || echo /usr/bin/git)`.
+- **cfn-init `files:` runs before `container_commands`** — the correct pattern for writing a
+  script and then executing it in the same config.
+
+## Change history
+- 2026-06-26 — Documented the AL2 → AL2023 (PHP 8.5) EB migration for Tools: package renames
+  (`libstdc++48`→`libstdc++`, `php73-ldap`→`php-ldap`, drop `libcurl`), php ini via cfn-init
+  `files:` writing `/etc/php.d/application.ini`, bash-based library clone replacing the
+  CodePipeline PHP script, duplicate-config neutralization, `ignoreErrors` on non-fatal infra
+  commands, and the ondeck-path / eb-activity.log / pipeline-noop gotchas (jcardinal)

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -6,7 +6,7 @@
 | [ACL Permission Chain (Record & Field Authorization)](features/acl-permission-chain.md) | Authorization in the 2.0 API is **metadata-driven**: whether a role may Create/Read/Update/Delete a record is decided by rows across **four linked tables**, not | api2/Component/Api/V2/V2.php, _underscore/Model/Core/Page.php, dbchanges2/Client/2026-06-03- BLANK_CLIENT_DATABASE.sql, dbchanges2/Client/2026-06-23b - ItemTranslationsAcl.sql |
 | [Carrier Shipping Labels (UPS/FedEx) & NetSuite Item Fulfillment](features/carrier-shipping-labels.md) | Backend mechanics behind TOGa Supply's Fulfill & Ship: buying a carrier label (UPS/FedEx), persisting it, and creating the NetSuite Item Fulfillment with tracki | _underscore/Model/Client/ItemFulfillment.php, _underscore/Model/Client/ItemFulfillments/TrackingNumber.php, _underscore/Component/Library/LabelPdf/LabelPdf.php, _underscore/Component/Library/Carriers/Ups/Ups.php, _underscore/Component/Library/Carriers/Fedex/Fedex.php, _underscore/Trait/Netsuite/ItemFulfillment.php, _underscore/Trait/Netsuite/SalesOrder.php, _underscore/Component/Library/NetSuite/NetSuite.php, _underscore/Model/Client/TrackingNumber.php, _underscore/Model/Client/ShippingMethod.php, _underscore/Model.php, _underscore/Cloud.php |
 | [Client Email Template Sending](features/email-template-sending.md) | `_Model_Client_EmailTemplate` sends a stored, client-defined email template by UUID. | _underscore/Model/Client/EmailTemplate.php, _underscore/Model/Client/EmailTemplateOutgoingEmailAddress.php, _underscore/Email.php |
-| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php |
+| [Forecast.Sales NetSuite import engine (real-time webhook)](features/forecast-sale-import.md) | Real-time importer that takes a NetSuite **sale** record and writes its lines into `Forecast.Sales` (the Forecast2 revenue table). | _underscore/Component/Forecast/SaleImport/SaleImport.php, _underscore/Component/Forecast/Db/Db.php, worker2/Worker/Netsuite/Invoice.php, worker2/Worker/Netsuite/CashSale.php, worker2/Worker/Netsuite/CreditMemo.php, worker2/Worker/Netsuite/CashRefund.php, worker2/Worker/Netsuite/Opportunity.php, worker2/Worker/Netsuite/SalesOrder.php, test/@dave/test_invoice_lifecycle.php, test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js, test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js |
 | [_Model magic-field access (__get without __isset)](features/model-magic-field-access.md) | `_Model` exposes DB columns as "magic" properties via `__get()`, but it defines **no** `__isset()`. | _underscore/Model/Core/Model.php |
 | [NetSuite REST Client (_Component_Api_Netsuite) — record writes & SuiteQL](features/netsuite-rest-client.md) | `_Component_Api_Netsuite` is the **2.0 `_underscore` NetSuite REST client** — the shared primitive every worker2/api2 NetSuite caller uses for record GETs, Suit | _underscore/Component/Api/Netsuite/Netsuite.php |
 | [Per-Client Database Connections & the Local Logs Trap](features/per-client-database-connections.md) | When `_underscore` serves a request for a client it opens **three distinct per-client database connections**, not one. | _underscore/Database.php, _underscore/ApiRequest.php, _underscore/Model/Client/Logs/Api.php |

package/knowledge/2.0/apps/_underscore/features/forecast-sale-import.md

@@ -6,7 +6,7 @@ project: _Underscore
 client: shared
 type: feature
 status: active
-updated: 2026-06-25
+updated: 2026-06-26
 owners: [dfranks]
 files:
   - _underscore/Component/Forecast/SaleImport/SaleImport.php
@@ -17,9 +17,14 @@ files:
   - worker2/Worker/Netsuite/CashRefund.php
   - worker2/Worker/Netsuite/Opportunity.php
   - worker2/Worker/Netsuite/SalesOrder.php
+  - test/@dave/test_invoice_lifecycle.php
+  - test/@dave/NetSuite/api-message-queue/ue_api_msg_queue_enqueue.js
+  - test/@dave/NetSuite/api-message-queue/dev_ue_api_msg_queue_enqueue.js
 related:
   - ../architecture.md
   - ../../worker2/architecture.md
+  - ../../worker2/features/netsuite-salesorder-open-orders-sync.md
+  - ./netsuite-rest-client.md
 ---
 
 ## Summary
@@ -144,7 +149,51 @@ events as specific **subtypes** (`inventoryItem`, `nonInventoryResaleItem`, `kit
 would be needed only for a future real-time item webhook, **not** for this Sales importer
 (which keeps items fresh via the hourly item pull cron plus the inline self-heal).
 
+## Local testing harness (`test/@dave/test_invoice_lifecycle.php`)
+A CLI that boots worker2/_underscore (chdir to `worker2/` then `require index.php`) and drives a
+**full NetSuite invoice lifecycle** through the live `_underscore` REST client
+(`_Component_Api_Netsuite`) to exercise this importer end-to-end against a local Forecast mirror.
+Actions: `check | create | get | update | delete | sync | recent | amq | deployments | locations`.
+
+- **Fixture:** customer **58** ("8 Test Company") + item **103741** ("Test Other Charge for Sale",
+  maps to local `Forecast.Items` id 25), **rate 0** → a **$0 invoice that still yields one Sales
+  row** (the engine does **not** skip a $0 line — revenue `0.00` is written; same as the cron's
+  Sales section). Confirms create→insert, edit→update (tracked-column change-detection), and
+  delete→`removeAll` all land in local `Forecast.Sales`.
+- It carries its **own** `createInvoiceRecord()` that POSTs via `_ApiRequest` and parses the
+  `Location` header with a **correct** regex (delimiter `~`) — a deliberate workaround for the live
+  `_Component_Api_Netsuite::createRecord()` `#`-delimiter bug (see the
+  [REST client doc](./netsuite-rest-client.md)); the framework was **not** edited this session.
+- `amq` action SuiteQLs the AMQ custom record; `deployments` SuiteQLs `scriptdeployment`+`script`
+  to show which enqueuer fires per record type (used to find the dual-deployment trap below).
+
 ## Gotchas / known issues
+- **Testing locally pollutes PROD unless you UNDEPLOY the prod AMQ enqueuer first.** Each sale
+  transaction type has **TWO** NetSuite UE enqueuer deployments: **"AMQ — Enqueuer"**
+  (`customscript_ue_amq_enqueue`, **RELEASED** → posts to **prod** `webhook.togahub.com`) and
+  **"AMQ development script"** (`customscript2619` → ngrok via `DEV_OVERRIDE`/`debugWrap`). A
+  **RELEASED** deployment fires for **all** actors **including the REST/M2M integration user**, so a
+  REST-driven test invoice hits **prod too**. Observed first run: test-invoice edits flowed to prod
+  and a $0 row was written to **prod** `Forecast.Sales` — by the **legacy 5-min pull cron**, NOT the
+  webhook (prod worker2 has no invoice handler deployed yet, so prod `Netsuite/Invoice/*` jobs
+  watchdog-timeout at 300s, `isSuccess=0`). **To isolate a local test, undeploy the prod enqueuer**
+  for the record types under test (invoice/cashSale/cashRefund/creditMemo/journalEntry); the second
+  full create→update→delete run was then cleanly isolated (zero prod `Logs.Webhook`/`Forecast.Sales`
+  for the test id). The **dev** enqueuer's `debugWrap` envelope
+  `{action,parameters:{payload,headers}}` is required because a local worker2 has no
+  `WebhookIngestion` Lambda to wrap the raw body.
+- **No local app logs for debug-path webhooks — verify arrival via the ngrok inspector, and decode
+  the body.** The legacy `{action,parameters}` debug path writes **no** `WorkerJobs`/`Logs.Webhook`
+  locally. Confirm inbound arrival at the ngrok request inspector
+  `http://127.0.0.1:4040/api/requests/http` — but its **raw body is base64**, so decode the body
+  (after the `\r\n\r\n`) before grepping for `internalId`/`eventType`. The inspector buffer is
+  **ephemeral** (rotates/clears) — watch it live while triggering.
+- **Sales create webhook races NetSuite REST read-after-write lag** — the handler GETs the invoice
+  before its line sublist materializes → **0 lines → 0 rows**. Same gotcha as the SalesOrder
+  open-orders sync (see that doc's read-after-write-lag gotcha); it self-heals on the next edit
+  webhook or the daily discrepancy-fix.
+- **Local _underscore boots on PHP 8.0.30 (xampp8)** — the floor enforced in `_underscore.php` is
+  **8.0.0**, not 8.1.
 - **`_Component_*` classes must live under the `_underscore` framework root, never a project
   repo.** The autoloader rejects a `_Component_*`/`_Model_*` loaded from a project path
   (e.g. `./` under worker2) with "namespace … has not been defined" — at **class-load
@@ -154,6 +203,19 @@ would be needed only for a future real-time item webhook, **not** for this Sales
 - The cron's sign handling is not portable here — see Sign convention.
 
 ## Change history
+- 2026-06-26 — **Verified the full create→update→delete webhook lifecycle end-to-end on a local box**
+  (ngrok tunnel → local worker2 → local `Forecast.Sales`), driving a real **$0** NetSuite invoice
+  through create→insert, edit→update (tracked-column change-detection), delete→`removeAll`; confirmed
+  a $0 line is **not** skipped (one revenue-`0.00` row). Built the reusable harness
+  `test/@dave/test_invoice_lifecycle.php` (boots worker2/_underscore, drives the lifecycle via
+  `_Component_Api_Netsuite`; `amq`/`deployments` SuiteQL actions). Recorded durable testing gotchas:
+  the **dual AMQ enqueuer** RELEASED-fires-for-the-REST-user prod-pollution trap (undeploy the prod
+  enqueuer to isolate; prod's $0 row came from the legacy 5-min pull cron, not a prod webhook —
+  prod has no invoice handler so its `Netsuite/Invoice/*` jobs watchdog-timeout `isSuccess=0`); no
+  local app logs on the debug `{action,parameters}` path (verify via the ngrok inspector at
+  `:4040`, body is base64, buffer ephemeral); the Sales create webhook shares the SalesOrder
+  read-after-write line-sublist race; and local _underscore's PHP floor is 8.0.0. No production code
+  change. (dfranks)
 - 2026-06-25 — **Verified `sync()` end-to-end across all four sale types** (prod NS reads, local Forecast
   writes; 40+ records spanning invoice/cashSale/creditMemo/cashRefund): revenue reconciled **to the penny**
   against an independent SuiteQL `SUM(-foreignamount)` oracle on every record; the uniform sign factor held

package/knowledge/2.0/apps/_underscore/features/netsuite-rest-client.md

@@ -57,10 +57,12 @@ through `_ApiRequest` directly (mirroring `send()`'s auth/endpoint/header setup)
 > reads the class-`#` as a premature closing delimiter and the whole pattern **always throws
 > `preg_match(): Unknown modifier ']'`**. This fires *after* the record is already created, so the
 > create succeeds in NetSuite but `createRecord()` raises and the caller never gets the new id —
-> it breaks the entire outbound create/push path. **Fix direction:** change the delimiter so `#`
+> it breaks the entire outbound create/push path **for any record type** (observed on `journalentry`
+> and again on `invoice` create). **Fix direction:** change the delimiter so `#`
 > isn't both delimiter and class member, e.g. `~/(\d+)(?:[?#]|$)~`. Interim workaround used in
-> probes: recover the new id by regex-parsing the thrown exception message (the `Location` value
-> is in it). Confirmed live 2026-06-25.
+> probes and in `test/@dave/test_invoice_lifecycle.php`: parse the new internalId off the `Location`
+> value carried in the thrown exception message (the lifecycle harness ships its own correct
+> `~`-delimited parse to sidestep this without editing the framework). Confirmed live 2026-06-26.
 
 ### Update — reuse `send('PATCH', $route, $body)`
 

package/knowledge/INDEX.md

@@ -12,7 +12,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 - **walmarttechservices** (Walmart Tech Services) — 1 doc(s) → [1.0/apps/walmarttechservices/INDEX.md](1.0/apps/walmarttechservices/INDEX.md)
 - **test** (Test) — 11 doc(s) → [1.0/apps/test/INDEX.md](1.0/apps/test/INDEX.md)
 - **toga** (TOGa) — 2 doc(s) → [1.0/apps/toga/INDEX.md](1.0/apps/toga/INDEX.md)
-- **tools** (Tools) — 4 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
+- **tools** (Tools) — 5 doc(s) → [1.0/apps/tools/INDEX.md](1.0/apps/tools/INDEX.md)
 
 ## 2.0 framework
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.208",
+  "version": "1.0.210",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",