toga-ai 1.0.1182 → 1.0.1184

package/knowledge/1.0/apps/test/INDEX.md

@@ -19,6 +19,6 @@
 | [TOGa 2.0 Client Onboarding Wizard (local tool)](features/toga2-onboarding-wizard.md) | Local browser wizard automating 2.0 client onboarding end-to-end (generates all onboarding SQL + consolidates dbchanges2 loose SQL); local-only, renders secrets |
 | [TOGa 2.0 User Cross-Client Access SQL Generator](features/toga2-user-cross-client-access-sql.md) | 1.0 App_ script generating SQL to grant an existing 2.0 user cross-client access (Users_Roles grants); open when granting a home-client user into another client |
 | [URL & Domain Markdown Document Builder](features/url-domain-markdown-document.md) | 1.0 App_ script building a markdown doc of our URLs/domains from the 2.0 API (/environments, /domains); open to regenerate the URL/domain reference. |
-| [Legacy 1.0 web apps in toga-prod — naming, environment setup, DNS cutover](workflows/legacy-1-0-web-apps-in-toga-prod.md) | Where the 1.0 web apps (store, togadesk, togaview, toga, walmart-activate) run since 2026-10-22 (toga-prod 654654170868, us-east-1), their env/pipeline names an |
+| [Legacy 1.0 web apps in toga-prod — naming, environment setup, DNS cutover](workflows/legacy-1-0-web-apps-in-toga-prod.md) | Where the 1.0 web apps (store, togadesk, togaview, toga, walmart-activate) run since 2026-10-22 (toga-prod 654654170868; us-east-1, except TOGa Retail in us-wes |
 | [Responding to a PHP Security Release on Elastic Beanstalk (2.0 platform update, legacy stopgap, WAF, exposure check)](workflows/responding-to-a-php-security-release.md) | Playbook for a critical PHP CVE across both AWS accounts: patch 2.0 by EB platform update, shield legacy PHP <= 8.1 envs (stopgap .ebextension, allowlist WAF, r |
 | [Finding and Retiring Unused Elastic Beanstalk Environments (legacy AWS account 502614707982)](workflows/retiring-unused-legacy-elastic-beanstalk-environments.md) | Runbook to prove an EB env in legacy account 502614707982 (us-west-2) is unused (DNS in two accounts + external, shared-ALB rules, CodePipeline) and tear it dow |

package/knowledge/1.0/apps/test/workflows/legacy-1-0-web-apps-in-toga-prod.md

@@ -6,7 +6,7 @@ project: Test
 client: shared
 type: workflow
 status: active
-updated: 2026-10-27
+updated: 2026-10-28
 owners: ["rgirish"]
 files:
   - test/team/aws/create_elastic_beanstalk.php
@@ -16,6 +16,9 @@ files:
   - toga/_/app/framework.php
   - toga/.ebextensions/020_setup_git_libraries.config
   - toga/ebs/mount-s3fs-folders.php
+  - toga/.ebextensions/050_register_instance_to_shared_application_load_balancer.config
+  - toga/ebs/register_instance_to_shared_application_load_balancer.php
+  - toga/config.prod.ini
 related:
   - ../../dbchanges/workflows/authoring-and-shipping-sql-files.md
   - ../../../../clients/office-depot/profile.md
@@ -29,21 +32,32 @@ related:
   - ../../api/workflows/deploy-to-elastic-beanstalk-al2023.md
 ---
 
-Where the 1.0 web apps (store, togadesk, togaview, toga, walmart-activate) run since 2026-10-22 (toga-prod 654654170868, us-east-1), their env/pipeline names and settings, the TOGa Retail sandbox (nonprod), how a domain is cut over with "1" test names, and how each app maps host → client; open before any 1.0 web-app deploy, DNS, cert, or domain change.
+Where the 1.0 web apps (store, togadesk, togaview, toga, walmart-activate) run since 2026-10-22 (toga-prod 654654170868; us-east-1, except TOGa Retail in us-west-2 since 2026-10-28), their env/pipeline names and settings, the TOGa Retail sandbox (nonprod), how a domain is cut over with "1" test names, how an env moves region, and how each app maps host → client; open before any 1.0 web-app deploy, DNS, cert, region, or domain change.
 
 ## Summary
 
 On 2026-10-22 (TRUE-83055) the five legacy 1.0 web apps moved from legacy account 502614707982 (us-west-2) to toga-prod 654654170868 (us-east-1), each on **one** PHP 8.5 / AL2023 environment. Each new env replaced 2–3 legacy envs (store 3, togadesk 3, togaview 2, toga 3: agilant/odp/chat) — same code, same `ENVIRONMENT=prod`; traffic was small (store ~65k requests in 14 days). All legacy prod envs and pipelines for these apps are gone. The 1.0 `worker` still runs in the legacy account (`agilant-worker`).
 
-| EB application | EB environment | CodePipeline |
-|---|---|---|
-| `store` | `togatech-legacy-store-production` | `Legacy-Store-Production` |
-| `TOGa Desk` | `togatech-legacy-togadesk-production` | `Legacy-TOGaDesk-Production` |
-| `TOGa View` | `togatech-legacy-togaview-production` | `Legacy-TOGaView-Production` |
-| `TOGa Retail` (repo `toga`) | `togatech-legacy-togaretail-production` | `Legacy-TOGaRetail-Production` |
-| `Walmart` (repo `walmart-activate`) | `togatech-legacy-walmart-production` | `Legacy-Walmart-Production` |
+The 1.0 databases (`prod-cluster` / `prod-writer`, account 502614707982) stay in **us-west-2**, so a us-east-1 env pays cross-region latency on every query. On 2026-10-28 (TRUE-83361) TOGa Retail moved to toga-prod **us-west-2**, same names: pages ~0.3s vs 0.6–1.5s before. Its us-east-1 env, pipeline and EB app are deleted. The other four are still in us-east-1 (TOGa Desk move is TRUE-83360).
+
+| EB application | EB environment | CodePipeline | Region |
+|---|---|---|---|
+| `store` | `togatech-legacy-store-production` | `Legacy-Store-Production` | us-east-1 |
+| `TOGa Desk` | `togatech-legacy-togadesk-production` | `Legacy-TOGaDesk-Production` | us-east-1 |
+| `TOGa View` | `togatech-legacy-togaview-production` | `Legacy-TOGaView-Production` | us-east-1 |
+| `TOGa Retail` (repo `toga`) | `togatech-legacy-togaretail-production` | `Legacy-TOGaRetail-Production` | **us-west-2** |
+| `Walmart` (repo `walmart-activate`) | `togatech-legacy-walmart-production` | `Legacy-Walmart-Production` | us-east-1 |
+
+Pipelines: source GitHub `agilantsolutions/<repo>` branch `_production` → **Approval** → Deploy. A merge does not deploy until someone approves. Artifact bucket: `true-elasticbeanstalk-deployments-1` (us-east-1) / `-2` (us-west-2). The us-east-1 CodeConnection "Agilant Solutions GitHub" works from a us-west-2 pipeline.
+
+### TOGa Retail prod in us-west-2
 
-Pipelines: source GitHub `agilantsolutions/<repo>` branch `_production` → **Approval** → Deploy. A merge does not deploy until someone approves.
+- 2 x `t4g.large` arm64, PHP 8.5 AL2023 platform 4.13.13, `ENVIRONMENT=prod`.
+- VPC `true-production-vpc-2` (`vpc-0a447bf81a34396e1`, 10.202.0.0/16): ALB in `true-production-public-web-2a/2b/2c`, instances in `true-production-private-web-2a/2b/2c`. The VPC routes legacy DB network 172.30.0.0/16 through transit gateway `tgw-00a1af4e86e4b4c43`; legacy DB SG `sg-b900a8c3` already allows 3306.
+- HTTPS 443, policy `ELBSecurityPolicy-TLS-1-2-Ext-2018-06`. Default cert `agilantsolutions.com` (also `*.togatech.com`, `*.togatechnology.com`, `*.goagilant.com`, …); SNI: `togahub.com` / `*.togahub.com`, and `*.toga.agilantsolutions.com` / `*.toga.togatechnology.com`.
+- Sessions: `SESSION_REDIS_SAVE_PATH` → the **us-west-2** `togadesk-sessions` cluster (`togadesk-sessions.vniulb.0001.usw2.cache.amazonaws.com`, SG `togadesk-session-redis` `sg-05190dc12fc67c9d3`). The env's instance SG must be allowed on 6379 there. Session rules: [tools deploy § Redis sessions](../../tools/workflows/deploy-to-elastic-beanstalk-al2023.md).
+- DNS (EB alias zone us-west-2 `Z38NKT9BP95V3O`): toga-prod `toga.togatech.com`, `odp.toga.togatechnology.com` (A alias), `chat.togahub.com` (CNAME, TTL 300); legacy `agilantsolutions.com` zone: `odp-toga`, `odp.toga`, `staples.toga`, `synnex.toga` (A alias). ODP hosts 302 to the Azure SAML login.
+- `ebs/register_instance_to_shared_application_load_balancer.php` hardcodes us-west-2 but skips any env name containing `-prod` — it does nothing here.
 
 ### Non-prod: TOGa Retail sandbox (ODP, TRUE-83279)
 
@@ -64,7 +78,7 @@ ODP calls it "sandbox"; internally it **is beta** (replaces the old shared `agil
 1. Generate the command with the `production` profile of [create_elastic_beanstalk.php](../features/create-elastic-beanstalk.md): own public ALB, 2 x `t4g.large` arm64, PHP 8.5 AL2023 platform.
 2. Set env property `ENVIRONMENT=prod`. The 1.0 apps ship only `config.prod.ini` — `production` finds no config file.
 3. Name it `togatech-legacy-<app>-production`. EB env names are max 40 characters (why Walmart is `walmart`, not `walmart-activate`).
-4. HTTPS 443 listener: copy the SSL policy from the legacy listener; attach the main cert, then extra certs by SNI (`aws elbv2 add-listener-certificates`).
+4. HTTPS 443 listener: copy the SSL policy from the legacy listener; attach the main cert, then extra certs by SNI (`aws elbv2 add-listener-certificates` — one cert per call).
 5. If `/` redirects (Walmart), set process default `MatcherHTTPCode=200,301,302` or the env goes Red on health checks.
 6. Pipeline `Legacy-<App>-Production`: Source → Approval → Deploy.
 
@@ -72,10 +86,24 @@ ODP calls it "sandbox"; internally it **is beta** (replaces the old shared `agil
 
 1. Create a test record per real host with a trailing `1` (`prudential1.togacommerce.com`, `rumcsi1.togadesk.togatech.com`) pointing at the new env.
 2. Deploy a **temporary** code change that strips the trailing `1` in the app's host → client lookup (see table below). Test every client on its `1` host.
-3. UPSERT the real records as aliases to the new env (EB alias hosted zone for us-east-1: `Z117KPS5GTRQ2G`).
+3. UPSERT the real records as aliases to the new env (EB alias hosted zone: us-east-1 `Z117KPS5GTRQ2G`, us-west-2 `Z38NKT9BP95V3O`).
 4. Delete the `1` test records and revert the temporary code change.
 5. Retire the old envs per [retiring unused EB environments](retiring-unused-legacy-elastic-beanstalk-environments.md). Save each config first as an EB template (`backup-<env>-<yyyymmdd>`) — that is the rollback path.
 
+### Move an env to another region (same names)
+
+Done for TOGa Retail us-east-1 → us-west-2 (TRUE-83361). EB app/env/pipeline names are per region, so the new region reuses them.
+
+1. Create the env in the new region (steps above) on the target VPC; check it reaches the DB network and the DB SG allows 3306.
+2. Request ACM certs in the new region. If the DNS validation CNAMEs already exist in the same account, they issue in minutes (validation records are per account + domain, shared across regions).
+3. Point `SESSION_REDIS_SAVE_PATH` at the Redis cluster in the **same** region; add the env's instance SG to that Redis SG on 6379.
+4. Create the pipeline in the new region with that region's artifact bucket.
+5. Test each host against the new env CNAME (`curl --resolve` or Host header): same status and page size as the old env.
+6. UPSERT every DNS record (both accounts) to the new env's alias zone.
+7. Save the old env config to a file (EB saved configs die with the EB app — see Gotchas).
+8. Remove the old env's instance SG from the old-region Redis SG ingress (us-east-1 `sg-03e79bcb9c105e100`).
+9. Delete the old pipeline, terminate the old env, delete the old EB app.
+
 ### Where DNS lives
 
 - toga-prod 654654170868 Route 53: `togacommerce.com`, `togahub.com`, `togatech.com`, `togadesk.com`, `togaview.com`, `togatechnology.com`, `togaretail.com` (and the other `toga*` zones).
@@ -95,7 +123,9 @@ SAML ACS URLs are hard-coded (store: `prudential.togahub.com/?acs`, then redirec
 
 ## Gotchas
 
-- **ACM validation in toga-prod can take 30–60+ minutes** even with correct CNAMEs. Attach already-issued wildcard certs first; add the slow cert by SNI later.
+- **ACM validation in toga-prod can take 30–60+ minutes** for a new domain, even with correct CNAMEs. Attach already-issued wildcard certs first; add the slow cert by SNI later. (Fast when the same account already validated that domain in another region.)
+- **Deleting an EB application deletes its saved configurations.** Save the config elsewhere first. The us-east-1 `TOGa Retail` app is gone; its only backup is a local JSON.
+- **EB cannot delete an env's SG while another SG references it.** Remove the env's instance SG from the Redis SG ingress before terminating, or the terminate hangs/fails.
 - **`ENVIRONMENT` must be `prod`, not `production`** (see step 2). Wrong value = no config, every request fails.
 - **The Approval stage is the only brake on prod deploys.** The legacy `Worker` pipeline (still live, account 502) also fires on `library` pushes; it got an Approval stage on 2026-10-22 so a library merge no longer auto-deploys the worker. Every app clones library `_production` at deploy, so a library merge ships with the next approved deploy of any app.
 - **S3 mount fails the non-prod deploy.** Missing either S3 grant (sandbox section) → `mount-s3fs-folders.php` exits 1 → deploy fails at `02_mount_s3fs`.
@@ -104,5 +134,6 @@ SAML ACS URLs are hard-coded (store: `prudential.togahub.com/?acs`, then redirec
 - **Old env names in other docs are history.** `prudential-store-prod`, `true-togacommerce-prod`, `spglobal-store-prod`, `*-togadesk-prod` and pipeline `Store-Production` no longer exist.
 
 ## Change history
+- 2026-10-28 — TOGa Retail prod moved to us-west-2 (next to the 1.0 DBs), region-move steps, EB app/SG/cert gotchas (TRUE-83361) (rgirish)
 - 2026-10-27 — TOGa Retail sandbox for ODP (= beta internally), `-sandbox` host strip, nonprod S3 grants, `_beta` drift (TRUE-83279) (rgirish)
 - 2026-10-22 — Created: 5 legacy 1.0 web apps moved to toga-prod us-east-1, naming, env settings, "1"-host cutover, host → client lookup (TRUE-83055) (rgirish)

package/knowledge/1.0/apps/tools/workflows/deploy-to-elastic-beanstalk-al2023.md

@@ -75,9 +75,9 @@ Prod Tools runs multiple EB instances behind an ALB, so PHP sessions must live i
 
 **Shared 1.0 session store (decided 2026-10-24, jcardinal, TRUE-83179) — home of this rule for every legacy app:**
 1. `asi-session-cache` (us-west-2 cluster-mode) is **decommissioned**. Never point anything at it.
-2. Every 1.0/legacy app needing Redis sessions (tools, toga/togaretail, store, walmart-activate, togaview, vision, togadesk) uses ElastiCache `togadesk-sessions` (prod account 654654170868, us-east-1, non-cluster, redis 7.1).
+2. Every 1.0/legacy app needing Redis sessions (tools, toga/togaretail, store, walmart-activate, togaview, vision, togadesk) uses ElastiCache `togadesk-sessions` (prod account 654654170868, non-cluster, redis 7.1) **in the env's own region**: us-east-1, or the us-west-2 copy (SG `sg-05190dc12fc67c9d3`) for envs in us-west-2 (toga/TOGa Retail since 2026-10-28 — see [legacy 1.0 web apps](../../test/workflows/legacy-1-0-web-apps-in-toga-prod.md)).
 3. Endpoint comes only from EB env property `SESSION_REDIS_SAVE_PATH=tcp://<togadesk-sessions primary endpoint>:6379`. Never hardcode it in code or INI.
-4. Cache SG `togadesk-session-redis` (`sg-03e79bcb9c105e100`) must allow inbound 6379 from each app's EB instance SG. New app → add that rule. All legacy EB envs share the VPC.
+4. Cache SG `togadesk-session-redis` (us-east-1 `sg-03e79bcb9c105e100`, us-west-2 `sg-05190dc12fc67c9d3`) must allow inbound 6379 from each app's EB instance SG. New app → add that rule. Remove it before terminating an env, or EB cannot delete the env's SG.
 5. Not Redis users: api, saml, forward, worker1.5, library (only an error-filter string mentions `rediscluster`).
 
 `startSession()` in `<app>/_/app/framework.php` reads the property via `environmentValue()` (getenv → `$_SERVER` → `$_ENV`; php-fpm `clear_env` hides getenv). `seed[]=` value → `rediscluster` handler, else `redis`. Set with `ini_set()` (the INI-file parser drops `seed[]=` values — see the PHP ini step above) plus `session.gc_maxlifetime=345600` (4 days), guarded by `extension_loaded('redis')`, then `session_start()` itself. Empty property or Redis failure → `error_log` + file-session fallback (togaview is the exception: fails loud — see its deploy doc). The full outage story (sessions were previously on the file handler on local disk, so every deploy logged everyone out) and the graceful-degrade ordering vs. App_Error are in the [architecture doc](../architecture.md).
@@ -139,6 +139,7 @@ The deploy clones `agilantsolutions/library` and `resources` via `020_setup_git_
 - [TOGa Desk deploy (own Redis, offset/Medoo shims, syslog error visibility)](../../togadesk/workflows/deploy-to-elastic-beanstalk-al2023.md)
 
 ## Change history
+- 2026-10-28 — Redis sessions: use the cluster in the env's region (us-west-2 copy for TOGa Retail) (TRUE-83361) (rgirish)
 - 2026-10-23 — `REDIS_HOOK_OPTIONAL` hatch broken (hooks need `get-config`); double ini load warning (TRUE-83056) (jcardinal)
 - 2026-10-22 — Shared gotchas: `php --ini` scan folder on AL2023, dart-sass arm64 (TRUE-83055) (rgirish)
 - 2026-10-22 — Log-bundle gotcha corrected: `/var/log/messages` is included, custom logs are not; link Desk deploy (TRUE-83053) (rgirish)

package/knowledge/2.0/apps/worker2/features/committed-secrets-and-config-debt.md

@@ -6,8 +6,8 @@ project: Worker
 client: shared
 type: feature
 status: active
-updated: 2026-10-22
-owners: ["kyalamarthi", "ajean", "mhammontree", "jcardinal", "bala"]
+updated: 2026-10-28
+owners: ["kyalamarthi", "ajean", "mhammontree", "jcardinal", "bala", "amadda"]
 files:
   - worker2/.gitignore
   - worker2/Worker/Team/Github.php
@@ -16,6 +16,8 @@ files:
   - worker2/Config/netsuite_ec_key.pem
   - worker2/Worker/Team/Sprint.php
   - worker2/.platform/php/conf.d/00-log-errors.ini
+  - api2/.platform/php/conf.d/00-exception-ignore-args.ini
+  - worker2/Controller/Index.php
 related:
   - ../workflows/clickup-credential-rotation.md
   - ./automated-pr-merger-force-push-race.md
@@ -56,11 +58,19 @@ the publish on a credential literal.
 
 | Exposure | Location | Status |
 |---|---|---|
-| Stack traces include function arguments (first 15 chars of each string) | worker2 stores failed-job traces in `Core.WorkerJobs`; `.platform/php/conf.d/00-log-errors.ini` does **not** set `zend.exception_ignore_args = On` | **DevOps option, not done (Mark: devops decision).** Turning it on strips arguments from every trace. Known leak: phpseclib `login()` frames carry the password — NYCDOE `Sftp/Import` `connect()` works around it by not chaining the exception. |
 | NYCDOE vendor FTP passwords | `Core.WorkerJobs.parameters` of old 1.0 `Utilities/Sftp/DeleteFile/DeleteSftpFile` jobs (1.0 sent the password in the job; source = `FTP_*` constants in `library/app/asnprocessor/*.php`) | **Open — separate ticket: rotate.** See [NYCDOE ServiceNow / ASN Integration](../../../../clients/nycdoe/features/servicenow-integration.md). |
 
 ### Fixed / no longer true
 
+- **Argument values in stack traces** (2026-10-28, TRUE-83206) — `zend.exception_ignore_args = On`
+  is now pinned in repo config: worker2 `.platform/php/conf.d/00-log-errors.ini`, api2
+  `.platform/php/conf.d/00-exception-ignore-args.ini` (own file, one line, so api2 error logging is
+  unchanged). worker2 prod was already On via the server default; the pin stops drift. Needed
+  because worker2 `Controller/Index.php` writes `getTraceAsString()` into `Core.WorkerJobs.output`
+  and Sentry gets raw traces (`_Error::buildSafeTrace()` only covers `Logs.Issue`). No code in
+  api2, worker2 or `_underscore` reads trace `['args']`. Other PHP repos (tools, worker, togadesk,
+  toga2-commerce, …) are not covered — possible follow-up ticket.
+
 - **AWS `[cloud]` static access key** (2026-10-12, TRUE-82705) — key deleted in IAM and removed from
   every `Config/*.ini` (plus the dead `[teams]` `s3_*` keys). AWS access is the instance role assuming
   `WorkloadsRuntime` — see [cross-account AWS access](./cross-account-aws-access.md). Still in git
@@ -86,6 +96,11 @@ the publish on a credential literal.
 - **Any credential that has ever been committed must be rotated**, not just deleted. Deleting the
   line leaves it in history and in every branch that still carries it (166 of 167, for the GitHub
   token).
+- **`zend.exception_ignore_args` drops every argument value, nothing else.** Message, file, line,
+  and function/class names stay. It does not clean the exception **message** — never put a secret
+  in a message.
+- **`.platform/php/conf.d/*.ini` is read only by Elastic Beanstalk on deploy.** Local PHP uses its
+  own `php.ini` (setting stays off locally — fine for debugging). Do not test it locally.
 - **Keep this table current.** Add an entry whenever a committed credential is found, and move it to
   *Fixed* when it is rotated and relocated, so this stays the one place the team looks.
 
@@ -98,6 +113,7 @@ the publish on a credential literal.
 - [Rate PayPal Subscription Purchase & Webhook Pipeline](../../../../clients/rate/features/paypal-subscription-purchase-webhook.md)
 
 ## Change history
+- 2026-10-28 — Trace arguments moved to Fixed: `zend.exception_ignore_args = On` pinned in worker2 + api2 `.platform` ini (TRUE-83206). (amadda)
 - 2026-10-22 — Added the hardcoded Elite API secret in `Worker/Elite.php` (TRUE-82919). (jcardinal)
 - 2026-10-19 — Added the shared SFTP server passwords: Compass `PeopleFile.php` (worker2) and the 1.0 NYCDOE/Staples copies (TRUE-82898). (bala)
 - 2026-10-12 — Moved the `[cloud]` AWS key to Fixed (deleted, TRUE-82705); listed the `[talos]` AWS key as still open (jcardinal)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.1182",
+  "version": "1.0.1184",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",