toga-ai 1.0.822 → 1.0.824

package/knowledge/1.0/apps/tools/features/compass-user-persona-admin.md

@@ -6,8 +6,8 @@ project: Tools
 client: shared
 type: feature
 status: active
-updated: 2026-09-04
-owners: [ajean]
+updated: 2026-09-16
+owners: [ajean, bala]
 files:
   - tools/_/app/compass.php
   - tools/mvc/compass/users/get.php
@@ -101,6 +101,10 @@ conditions of the decision below — **do not remove one without re-opening it**
    (the same human on two `Users` rows).
 5. The four `ensureCrossRegionUsersActive()` addresses are **blocked as both source and target**
    (`isCrossRegionProtectedEmail()`), because that hardcoded worker2 list matches on email.
+   **⚠ Out of date as of 2026-09-16:** worker2 split that list into six addresses
+   (`ALWAYS_ACTIVE_EMAILS_BOTH` + `ALWAYS_ACTIVE_EMAILS_CA`), so the two new
+   `@compassdigital.io` Canada-protected addresses are **not** blocked here yet. See
+   [PEOPLE-File User Lifecycle](../../../clients/compass-usa/features/people-file-user-lifecycle.md).
 6. The **audit row is written before the write**, on its own connection (see the transaction
    gotcha below).
 
@@ -191,9 +195,14 @@ names, reads `Core.ClientEmailDomains` / `Core.Clients`, and inserts into `Logs_
   ~178,615 of 178,616 `Client_Compass` rows, so the client id cannot be derived from the user row.
   See [Compass Canada profile](../../../clients/compass-canada/profile.md).
 - **A cross-region user still needs a code change and a deploy.** The only durable way to hold one
-  active is the hardcoded four-address list in `ensureCrossRegionUsersActive()` in
+  active is the hardcoded list in `ensureCrossRegionUsersActive()` in
   `worker2/Worker/Client/Compass/PeopleFile.php`. This tool does not change that — and because that
-  list matches on **email**, those four addresses are blocked from this tool's email edit entirely.
+  list matches on **email**, those addresses are blocked from this tool's email edit entirely.
+  **As of 2026-09-16 that list is six addresses, not four**: `ALWAYS_ACTIVE_EMAILS_BOTH` (the
+  original four, held active in both tenants) plus `ALWAYS_ACTIVE_EMAILS_CA` (two
+  `@compassdigital.io` US staff held active in `Client_CompassCanada` only).
+  `isCrossRegionProtectedEmail()` here still only knows the original four, so the two new addresses
+  can be edited in this tool and silently lose their protection.
 - **Every user search is a full table scan.** `Client_Compass.Users` has no index on `email`,
   `c_hrEmpUsername`, `c_hrEmpPersonnelNbr` or `isActive` (176k rows; `EXPLAIN` reports `type=ALL`,
   no possible keys). Same on `Client_CompassCanada`. Keep result sets bounded and do not add
@@ -235,12 +244,22 @@ names, reads `Core.ClientEmailDomains` / `Core.Clients`, and inserts into `Logs_
   (`_underscore/Model/Compass/SalesOrder.php:287-305`): it joins
   `INNER JOIN Contacts ON Contacts.id = Users.id`, which should be `Users.contactId`.
 - **worker2** — scope `ensureCrossRegionUsersActive()` to a `Users.id` list instead of matching on
-  email; that would also lift this tool's block on those four addresses.
+  email; that would also lift this tool's block on those addresses.
+- **tools** — extend `isCrossRegionProtectedEmail()` to the two `ALWAYS_ACTIVE_EMAILS_CA` addresses
+  added in worker2 on 2026-09-16, or the block silently under-covers the protected set.
 - **ops** — rotate every secret in `tools/config.production.ini`; consider a least-privilege
   Compass account.
 
 ## Change history
 
+- 2026-09-16 — Recorded that worker2's cross-region protected list **grew from four addresses to
+  six** (`ALWAYS_ACTIVE_EMAILS_BOTH` + `ALWAYS_ACTIVE_EMAILS_CA`), so this tool's
+  `isCrossRegionProtectedEmail()` block now under-covers it: the two new `@compassdigital.io`
+  Canada-protected addresses can still be email-edited here and would silently drop out of the
+  protected set, because the worker2 match is on email. No `tools` code changed. Detail:
+  [PEOPLE-File User Lifecycle](../../../clients/compass-usa/features/people-file-user-lifecycle.md).
+  (bala)
+
 - 2026-09-04 — Added **email, phone and cost-center editing** (PR tools#14,
   `feature/compass-user-field-edits`, **open / not merged / not deployed**): new `App_Compass`
   setters plus the `normaliseEmail` / `normalisePhone` / `emailDomainIsRegistered` /

package/knowledge/2.0/apps/_underscore/INDEX.md

@@ -61,4 +61,5 @@
 | [USPS DPV Deliverability Verdict (is this address actually insurable/shippable?)](features/usps-dpv-deliverability.md) | **USPS returning HTTP 200 with a populated address is NOT evidence that the address is deliverable.** The authoritative signal is USPS's **DPV (Delivery Point V |
 | [Refreshing a Local Dev Database from Beta (dev-sandbox)](workflows/local-db-refresh-from-beta.md) | How to reset a local 2.0 dev database from the **beta / dev-sandbox** environment: dump each schema (`Core`, `Client_<Id>`, `Logs_<Id>`, …) from the beta host, |
 | [Deleting a shared branch does not remove bad commits — a stale local clone merges them back](workflows/recreated-shared-branch-stale-local-remerge.md) | **Deleting and recreating a shared environment branch removes only the *ref*.** Every teammate who still has that branch checked out locally keeps the full pre- |
+| [Rotating Public TLS Certificates (ACM, CloudFront, ALB, Elastic Beanstalk)](workflows/rotating-public-tls-certificates.md) | How to replace the public TLS certificates that terminate HTTPS for TOGa front-end domains (togahub, togacommerce, togadesk, togaretail, togasupply, togaview, t |
 | [Running a 2.0 App Locally (browser, end-to-end via api2)](workflows/running-a-2.0-app-locally.md) | The full dependency chain required to run a 2.0 client app **through the browser**, end-to-end, against a **local `api2`** (e.g. |

package/knowledge/2.0/apps/_underscore/workflows/rotating-public-tls-certificates.md

@@ -0,0 +1,178 @@
+---
+title: Rotating Public TLS Certificates (ACM, CloudFront, ALB, Elastic Beanstalk)
+framework: "2.0"
+repo: _underscore
+project: _Underscore
+client: shared
+type: workflow
+status: active
+updated: 2026-09-16
+owners: ["rgirish"]
+files: []
+related:
+  - 2.0/standards/ssl-certificate-trust-model.md
+  - 2.0/apps/saml/workflows/rotating-the-saml-sp-certificate.md
+  - 1.0/apps/tools/features/cloudfront-client-setup.md
+---
+
+## Summary
+
+How to replace the public TLS certificates that terminate HTTPS for TOGa front-end domains
+(togahub, togacommerce, togadesk, togaretail, togasupply, togaview, togaiq) before they
+expire. This is the browser-facing TLS cert on CloudFront and ALBs — **not** the SAML SP
+credential (that is a pinned bilateral cert, see
+[Rotating the SAML SP Certificate](../../saml/workflows/rotating-the-saml-sp-certificate.md)).
+
+This runbook was written from the 2026-09-16 rotation in AWS account **654654170868**, where
+one IMPORTED multi-domain cert (14 SANs, copied into us-east-1, us-west-2 and eu-west-1) served
+4 CloudFront distributions and 12 ALB listeners and expired the same day.
+
+**The three things that bite you:**
+1. **Elastic Beanstalk keeps its own saved cert setting**, separate from the ALB listener. Fixing
+   the listener is not enough — the next deploy puts the old cert straight back.
+2. **CloudFront holds exactly ONE cert per distribution.** There is no staging. The swap *is* the
+   cutover, and it takes 5-15 minutes.
+3. **An ACM wildcard matches exactly one label.** `*.togasupply.com` covers
+   `elite.togasupply.com` but **not** `compass.beta.togasupply.com`.
+
+**Root cause to avoid repeating:** an **IMPORTED** ACM cert never auto-renews. Always replace with
+**Amazon-issued, DNS-validated** ACM certs, which renew themselves.
+
+## When to run this
+
+- A cert is nearing expiry (ACM emails, or your own audit finds it).
+- You are moving off an imported cert onto Amazon-issued certs.
+- Any time you suspect a cert is expired — run the audit in step 1 regardless; it is cheap.
+
+## Step 1 — Audit EVERYTHING, not just the cert you know about
+
+Do **not** just follow the expiring cert's `InUseBy` list. Enumerate **all** CloudFront
+distributions and **all** ALB listeners in **every** region, resolve each attached cert's
+`NotAfter`, and flag anything expiring before your cutoff.
+
+This is how the 2026-09-16 rotation found a **second** cert that had already expired six months
+earlier (2026-03-24) and was still the only cert on 2 ALB listeners and 2 CloudFront
+distributions — 4 live togadesk hosts had been serving cert errors since March and nobody noticed.
+
+Also audit **every Elastic Beanstalk environment's saved config**, including suspended ones. A
+suspended environment with no live listener can still hold a stale cert ARN that comes back when
+it is rebuilt.
+
+**Check each CloudFront distribution's origins before you plan work on it.** One distribution in
+this rotation had a real alias and live DNS but all 3 origins pointed at load balancers that no
+longer existed — a dead leftover. Do not spend change-window time on it.
+
+## Step 2 — Decide the cert shape: one cert per domain
+
+Replace a bundled multi-domain cert with **one cert per domain** (root + wildcard, e.g.
+`togahub.com` + `*.togahub.com`). Reasons:
+
+- A 14-SAN request exceeds ACM's default 10-domain quota.
+- One domain's problem no longer takes down all seven.
+- Each distribution only needs the domain it actually serves.
+
+All new certs must be **Amazon-issued and DNS-validated** so they auto-renew. This is case 2 of
+the [SSL Certificate Trust-Model Strategy](../../../standards/ssl-certificate-trust-model.md):
+AWS terminates the TLS, so use ACM public, non-exportable, auto-renewing.
+
+## Step 3 — Request the certs in the right regions
+
+Region rules:
+
+- **CloudFront only reads certs from `us-east-1`.** No exceptions.
+- **An ALB reads certs from its own region.** A togahub ALB in us-west-2 needs a us-west-2 cert;
+  the same domain on an eu-west-1 ALB needs a separate eu-west-1 cert.
+
+Check ACM first — unused per-domain certs may already exist (3 did in this rotation). Then request
+only what is missing.
+
+## Step 4 — Validate by DNS in Route 53
+
+Add the validation CNAMEs to Route 53. Certs for the **same domain in different regions share ONE
+validation record** — the CNAME name and value are identical, so 6 certs across 3 regions needed
+only 5 CNAMEs.
+
+Validation completes in about 2 minutes once the records are live. Wait for `ISSUED` before
+touching anything.
+
+## Step 5 — Stage the ALB listeners first (safe, additive)
+
+`aws elbv2 add-listener-certificates` is **purely additive**. A listener holds many certs and picks
+by SNI, so adding the new cert alongside the old one changes nothing for live traffic. Do this
+ahead of the change window for every listener.
+
+## Step 6 — THE ONE PEOPLE MISS: update the Elastic Beanstalk saved config
+
+Adding a cert to an ALB listener does **not** update the EB environment's saved setting
+`aws:elbv2:listener:443 / SSLCertificateArns`. After all 12 listeners were fixed and verified
+healthy in this rotation, 12 EB environments still named the **old** cert in their saved config.
+
+Nothing breaks right away — the listener serves traffic. But the next **deploy, restart, rebuild or
+scaling event** re-applies the saved config and puts the expired cert back. A listener-level audit
+does not catch this.
+
+```
+aws elasticbeanstalk update-environment \
+  --environment-name <env> \
+  --option-settings "Namespace=aws:elbv2:listener:443,OptionName=SSLCertificateArns,Value=<new-arn>"
+```
+
+Observed behaviour: this cleanly **replaced** the old cert on the listener (leaving only the new
+one), and every environment stayed Green/Ok through the update — no traffic disruption.
+
+**Gotcha:** an EB environment whose CloudFormation stack is in `DELETE_FAILED` cannot be updated at
+all — `update-environment` is refused. That needs separate stack cleanup.
+
+## Step 7 — Check alias coverage before any CloudFront swap
+
+CloudFront **rejects** an update if any alias on the distribution is not covered by the new cert.
+Check every alias against the new cert's SANs **programmatically** — this is where the one-label
+wildcard rule bites: `*.togasupply.com` does not cover `compass.beta.togasupply.com`. Beta and
+gamma environments generally need their own certs for this reason.
+
+## Step 8 — Swap CloudFront, smallest blast radius first
+
+`ViewerCertificate.ACMCertificateArn` is a **single string**, not a list. There is no
+"add new alongside old" for CloudFront: the swap is the cutover, and it deploys in 5-15 minutes.
+
+Order used, lowest risk first:
+
+1. Distributions that are **already broken** (nothing to lose).
+2. Single-alias distributions.
+3. Mid-size (9 aliases).
+4. The largest last (the 21-alias distribution fronting 21 named enterprise clients).
+
+Back up each distribution's config before every swap — `aws cloudfront get-distribution-config`,
+and keep the **ETag**; you need it to update.
+
+**Hard edge:** once the old cert's expiry moment has passed there is no useful CloudFront rollback,
+because rolling back restores an expired cert. Only forward.
+
+## Step 9 — Verify on the live hosts
+
+```
+echo | openssl s_client -servername HOST -connect HOST:443 2>/dev/null \
+  | openssl x509 -noout -issuer -enddate
+```
+
+Also run it **without** `-servername` to check the no-SNI / default-cert path — that path can still
+be serving the old cert after everything else looks fine.
+
+Finish by re-running the step 1 audit: every CloudFront distribution, every ALB listener, and every
+EB saved config should name a cert with a future `NotAfter`.
+
+## Gotchas
+
+- **IMPORTED certs never auto-renew.** If the audit shows `Type: IMPORTED`, it is a future outage.
+- **A cert copied into 3 regions is 3 separate certs** — all 3 expire at once, and all 3 need
+  replacing.
+- **EB saved config is the silent landmine.** See step 6.
+- **CloudFront takes one cert; ALBs take many.** Two completely different risk profiles in the same
+  rotation — plan them separately.
+- **Wildcards match one label only.** Verify alias coverage with code, not by eye.
+- **Expired certs hide.** Sweep the whole account; do not trust `InUseBy`.
+
+## Change history
+- 2026-09-16 — Created from the account 654654170868 rotation: replaced an expiring 14-SAN IMPORTED
+  cert with 6 per-domain Amazon-issued DNS-validated certs, relinked 12 ALB listeners, 6 CloudFront
+  distributions and 11 EB environment configs; documented the EB saved-config trap. (rgirish)

package/knowledge/INDEX.md

@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
 
 ## 2.0 framework
 
-- **_underscore** (_Underscore) _(framework core)_ — 85 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
+- **_underscore** (_Underscore) _(framework core)_ — 86 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
 - **worker2** (Worker) — 69 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
 - **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
 - **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)

package/knowledge/clients/compass-usa/features/people-file-user-lifecycle.md

@@ -6,7 +6,7 @@ project: Worker
 client: compass-usa
 type: client-feature
 status: active
-updated: 2026-09-04
+updated: 2026-09-16
 owners: [bala, ajean]
 files:
   - worker2/Worker/Client/Compass/PeopleFile.php
@@ -85,6 +85,39 @@ insert-refresh time the **new row already has its id** and the **old row is stil
 deactivation-time hook would be worse — by then the new row's relationship to the old one has to be
 rediscovered, and the deactivation path has no model hooks at all (see below).
 
+### Always-active protection is per-tenant, and has two layers
+
+A small hardcoded set of people must stay active regardless of what HR sends. Since **2026-09-16**
+that set is split into two class constants on `_Worker_Client_Compass_PeopleFile`:
+
+- **`ALWAYS_ACTIVE_EMAILS_BOTH`** — the original four addresses (two `@compass-usa.com`, two
+  `@compass-canada.com`). Held active in **both** `Client_Compass` and `Client_CompassCanada`.
+- **`ALWAYS_ACTIVE_EMAILS_CA`** — two `@compassdigital.io` **Compass USA** staff (usernames
+  `shuttk01` / `racerk01` in `Client_Compass`). Held active in **`Client_CompassCanada` only**.
+  They are allowed to go inactive in the US tenant when the PEOPLE file drops them; that is the
+  requirement, not a bug.
+
+`isEmailAlwaysActive(string $email, string $tenantKey): bool` is the single decision point — a
+lowercased match that returns true for the BOTH list in either tenant, and for the CA list only when
+`$tenantKey === 'CA'`. Two layers consume it:
+
+1. **The inactivation loop** in `executeForTenant()` skips any user whose `Users.email` is
+   protected for that tenant, so the batch `UPDATE ... SET isActive = 0` never lists them.
+2. **`ensureCrossRegionUsersActive()`**, which runs after both tenants have executed and force-sets
+   `isActive = 1` — in both databases for the BOTH list, and in `Client_CompassCanada` only for the
+   CA list.
+
+**Today only layer 2 actually protects the CA-only pair.** In `Client_CompassCanada` both of those
+users have `c_hrEmpUsername = NULL`, and `loadLookups()` keys its map on username — so they never
+enter the map and can never reach the inactivation batch on the Canada side. The loop guard is the
+defence for the day HR gives them a Canada username, not what is holding them active now.
+(Read-only prod check, 2026-09-16.)
+
+**The email match works across a case difference.** `Client_Compass` stores these two addresses
+UPPERCASE while `Client_CompassCanada` stores them lowercase. `Users.email` uses
+`utf8mb4_0900_ai_ci`, a case-insensitive collation, so `WHERE email = '...'` still hits — each
+address resolves to exactly **one** row per database.
+
 ## Gotchas
 
 - **⚠ Deactivation is raw bulk SQL — model hooks never fire.** User inactivation is executed as
@@ -157,16 +190,26 @@ rediscovered, and the deactivation path has no model hooks at all (see below).
   makes the [Tools Compass admin
   pages](../../../1.0/apps/tools/features/compass-user-persona-admin.md) a one-way-durable tool.
 - **`ensureCrossRegionUsersActive()` runs `UPDATE Users SET isActive = 1 WHERE email = X` with no
-  `isActive` filter**, so it reactivates **every** row carrying one of its four hardcoded addresses,
-  dead duplicates included. Verified on prod 2026-09-02: each of the four addresses currently
-  resolves to exactly **one** active row in **both** tenants (zero inactive), so nothing is being
-  resurrected today — but the query would happily do it if a duplicate appeared. That hardcoded list
-  is also still the **only** durable way to hold a cross-region user active, and changing it needs a
-  code change and a deploy per request. **Recommended (not done): scope it to a `Users.id` list
-  instead of matching on email** — matching on email means an email edit silently moves a user in or
-  out of the protected set, which is why the [Tools Compass
-  admin](../../../1.0/apps/tools/features/compass-user-persona-admin.md) blocks those four addresses
-  from being edited at all.
+  `isActive` filter**, so it reactivates **every** row carrying a protected address, dead duplicates
+  included. Verified on prod 2026-09-02 for the original four, and 2026-09-16 for the two CA-only
+  addresses: each resolves to exactly **one** row per database, so nothing is being resurrected
+  today — but the query would happily do it if a duplicate appeared. That hardcoded list is also
+  still the **only** durable way to hold a cross-region user active, and changing it needs a code
+  change and a deploy per request. **Recommended (not done): scope it to a `Users.id` list instead
+  of matching on email** — matching on email means an email edit silently moves a user in or out of
+  the protected set, which is why the [Tools Compass
+  admin](../../../1.0/apps/tools/features/compass-user-persona-admin.md) blocks the protected
+  addresses from being edited at all.
+- **⚠ The Tools admin's edit block covers only the original four addresses, not the two CA-only
+  ones.** The protected set grew to six in worker2 on 2026-09-16; the `tools` block list was not
+  changed and was not re-checked. Until it is, someone can edit the email of a CA-protected user in
+  [Tools Compass admin](../../../1.0/apps/tools/features/compass-user-persona-admin.md) and silently
+  drop them out of the protected set — because the match is on email, not id.
+- **⚠ Adding a new "always keep this person active" rule means touching exactly two places.**
+  `executeForTenant()`'s inactivation loop is the only `isActive = 0` writer in the whole class, and
+  `ensureCrossRegionUsersActive()` is the post-run sweep. Guard both and the requirement is covered;
+  guard only the loop and the post-run sweep still decides the final state, guard only the sweep and
+  the user flickers inactive between the two steps.
 
 ## Remediation — BUILT 2026-08-20
 
@@ -197,6 +240,24 @@ question is now contained rather than blocking.
 
 ## Change history
 
+- 2026-09-16 — **Made the always-active protection tenant-aware.** Two Compass USA staff
+  (`@compassdigital.io`, usernames `shuttk01` / `racerk01`) exist in **both** tenants, but the
+  PEOPLE file lists them only as US rows (`c_erpSystemId` 1001), so they were routed to
+  `Client_Compass` alone and had no protection on the Canada side. Requirement: they may go inactive
+  in the USA per the file, but must **always** stay active in Canada. Split the previously
+  undocumented single hardcoded list into `ALWAYS_ACTIVE_EMAILS_BOTH` (the original four) and
+  `ALWAYS_ACTIVE_EMAILS_CA` (the two new ones), added
+  `isEmailAlwaysActive(string $email, string $tenantKey): bool` as the single lowercased decision
+  point, and wired it into **both** the `executeForTenant()` inactivation loop (skip protected
+  users) and `ensureCrossRegionUsersActive()` (both DBs for BOTH, `Client_CompassCanada` only for
+  CA). Prod read-only findings: in `Client_CompassCanada` both users have `c_hrEmpUsername = NULL`,
+  so they never enter `loadLookups()`'s username-keyed map and **cannot** reach the Canada
+  inactivation batch at all — the post-run sweep is what actually protects them today, and the loop
+  guard is defence for the day they get a Canada username. Also confirmed the two tenants store
+  these addresses in different case (US uppercase, CA lowercase) and the email match still works
+  because `Users.email` uses the case-insensitive `utf8mb4_0900_ai_ci` collation, one row per
+  address per database. Not committed or pushed — worker2 working tree only. (bala)
+
 - 2026-09-04 — Prod-verified the **full index list** on `Users` in both tenants and recorded the
   load-bearing consequence: **there is no `UNIQUE KEY` on `email`** (and no index on
   `c_hrEmpUsername` / `c_hrEmpPersonnelNbr` / `isActive`), so the database silently accepts duplicate

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "toga-ai",
-  "version": "1.0.822",
+  "version": "1.0.824",
   "description": "TOGA Technology Team Claude Knowledge System — shared AI coding harness with skills, knowledge base CLI, and project installer for Claude Code.",
   "keywords": [
     "claude",