toga-ai 1.0.822 → 1.0.823
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -61,4 +61,5 @@
|
|
|
61
61
|
| [USPS DPV Deliverability Verdict (is this address actually insurable/shippable?)](features/usps-dpv-deliverability.md) | **USPS returning HTTP 200 with a populated address is NOT evidence that the address is deliverable.** The authoritative signal is USPS's **DPV (Delivery Point V |
|
|
62
62
|
| [Refreshing a Local Dev Database from Beta (dev-sandbox)](workflows/local-db-refresh-from-beta.md) | How to reset a local 2.0 dev database from the **beta / dev-sandbox** environment: dump each schema (`Core`, `Client_<Id>`, `Logs_<Id>`, …) from the beta host, |
|
|
63
63
|
| [Deleting a shared branch does not remove bad commits — a stale local clone merges them back](workflows/recreated-shared-branch-stale-local-remerge.md) | **Deleting and recreating a shared environment branch removes only the *ref*.** Every teammate who still has that branch checked out locally keeps the full pre- |
|
|
64
|
+
| [Rotating Public TLS Certificates (ACM, CloudFront, ALB, Elastic Beanstalk)](workflows/rotating-public-tls-certificates.md) | How to replace the public TLS certificates that terminate HTTPS for TOGa front-end domains (togahub, togacommerce, togadesk, togaretail, togasupply, togaview, t |
|
|
64
65
|
| [Running a 2.0 App Locally (browser, end-to-end via api2)](workflows/running-a-2.0-app-locally.md) | The full dependency chain required to run a 2.0 client app **through the browser**, end-to-end, against a **local `api2`** (e.g. |
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Rotating Public TLS Certificates (ACM, CloudFront, ALB, Elastic Beanstalk)
|
|
3
|
+
framework: "2.0"
|
|
4
|
+
repo: _underscore
|
|
5
|
+
project: _Underscore
|
|
6
|
+
client: shared
|
|
7
|
+
type: workflow
|
|
8
|
+
status: active
|
|
9
|
+
updated: 2026-09-16
|
|
10
|
+
owners: ["rgirish"]
|
|
11
|
+
files: []
|
|
12
|
+
related:
|
|
13
|
+
- 2.0/standards/ssl-certificate-trust-model.md
|
|
14
|
+
- 2.0/apps/saml/workflows/rotating-the-saml-sp-certificate.md
|
|
15
|
+
- 1.0/apps/tools/features/cloudfront-client-setup.md
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Summary
|
|
19
|
+
|
|
20
|
+
How to replace the public TLS certificates that terminate HTTPS for TOGa front-end domains
|
|
21
|
+
(togahub, togacommerce, togadesk, togaretail, togasupply, togaview, togaiq) before they
|
|
22
|
+
expire. This is the browser-facing TLS cert on CloudFront and ALBs — **not** the SAML SP
|
|
23
|
+
credential (that is a pinned bilateral cert, see
|
|
24
|
+
[Rotating the SAML SP Certificate](../../saml/workflows/rotating-the-saml-sp-certificate.md)).
|
|
25
|
+
|
|
26
|
+
This runbook was written from the 2026-09-16 rotation in AWS account **654654170868**, where
|
|
27
|
+
one IMPORTED multi-domain cert (14 SANs, copied into us-east-1, us-west-2 and eu-west-1) served
|
|
28
|
+
4 CloudFront distributions and 12 ALB listeners and expired the same day.
|
|
29
|
+
|
|
30
|
+
**The three things that bite you:**
|
|
31
|
+
1. **Elastic Beanstalk keeps its own saved cert setting**, separate from the ALB listener. Fixing
|
|
32
|
+
the listener is not enough — the next deploy puts the old cert straight back.
|
|
33
|
+
2. **CloudFront holds exactly ONE cert per distribution.** There is no staging. The swap *is* the
|
|
34
|
+
cutover, and it takes 5-15 minutes.
|
|
35
|
+
3. **An ACM wildcard matches exactly one label.** `*.togasupply.com` covers
|
|
36
|
+
`elite.togasupply.com` but **not** `compass.beta.togasupply.com`.
|
|
37
|
+
|
|
38
|
+
**Root cause to avoid repeating:** an **IMPORTED** ACM cert never auto-renews. Always replace with
|
|
39
|
+
**Amazon-issued, DNS-validated** ACM certs, which renew themselves.
|
|
40
|
+
|
|
41
|
+
## When to run this
|
|
42
|
+
|
|
43
|
+
- A cert is nearing expiry (ACM emails, or your own audit finds it).
|
|
44
|
+
- You are moving off an imported cert onto Amazon-issued certs.
|
|
45
|
+
- Any time you suspect a cert is expired — run the audit in step 1 regardless; it is cheap.
|
|
46
|
+
|
|
47
|
+
## Step 1 — Audit EVERYTHING, not just the cert you know about
|
|
48
|
+
|
|
49
|
+
Do **not** just follow the expiring cert's `InUseBy` list. Enumerate **all** CloudFront
|
|
50
|
+
distributions and **all** ALB listeners in **every** region, resolve each attached cert's
|
|
51
|
+
`NotAfter`, and flag anything expiring before your cutoff.
|
|
52
|
+
|
|
53
|
+
This is how the 2026-09-16 rotation found a **second** cert that had already expired six months
|
|
54
|
+
earlier (2026-03-24) and was still the only cert on 2 ALB listeners and 2 CloudFront
|
|
55
|
+
distributions — 4 live togadesk hosts had been serving cert errors since March and nobody noticed.
|
|
56
|
+
|
|
57
|
+
Also audit **every Elastic Beanstalk environment's saved config**, including suspended ones. A
|
|
58
|
+
suspended environment with no live listener can still hold a stale cert ARN that comes back when
|
|
59
|
+
it is rebuilt.
|
|
60
|
+
|
|
61
|
+
**Check each CloudFront distribution's origins before you plan work on it.** One distribution in
|
|
62
|
+
this rotation had a real alias and live DNS but all 3 origins pointed at load balancers that no
|
|
63
|
+
longer existed — a dead leftover. Do not spend change-window time on it.
|
|
64
|
+
|
|
65
|
+
## Step 2 — Decide the cert shape: one cert per domain
|
|
66
|
+
|
|
67
|
+
Replace a bundled multi-domain cert with **one cert per domain** (root + wildcard, e.g.
|
|
68
|
+
`togahub.com` + `*.togahub.com`). Reasons:
|
|
69
|
+
|
|
70
|
+
- A 14-SAN request exceeds ACM's default 10-domain quota.
|
|
71
|
+
- One domain's problem no longer takes down all seven.
|
|
72
|
+
- Each distribution only needs the domain it actually serves.
|
|
73
|
+
|
|
74
|
+
All new certs must be **Amazon-issued and DNS-validated** so they auto-renew. This is case 2 of
|
|
75
|
+
the [SSL Certificate Trust-Model Strategy](../../../standards/ssl-certificate-trust-model.md):
|
|
76
|
+
AWS terminates the TLS, so use ACM public, non-exportable, auto-renewing.
|
|
77
|
+
|
|
78
|
+
## Step 3 — Request the certs in the right regions
|
|
79
|
+
|
|
80
|
+
Region rules:
|
|
81
|
+
|
|
82
|
+
- **CloudFront only reads certs from `us-east-1`.** No exceptions.
|
|
83
|
+
- **An ALB reads certs from its own region.** A togahub ALB in us-west-2 needs a us-west-2 cert;
|
|
84
|
+
the same domain on an eu-west-1 ALB needs a separate eu-west-1 cert.
|
|
85
|
+
|
|
86
|
+
Check ACM first — unused per-domain certs may already exist (3 did in this rotation). Then request
|
|
87
|
+
only what is missing.
|
|
88
|
+
|
|
89
|
+
## Step 4 — Validate by DNS in Route 53
|
|
90
|
+
|
|
91
|
+
Add the validation CNAMEs to Route 53. Certs for the **same domain in different regions share ONE
|
|
92
|
+
validation record** — the CNAME name and value are identical, so 6 certs across 3 regions needed
|
|
93
|
+
only 5 CNAMEs.
|
|
94
|
+
|
|
95
|
+
Validation completes in about 2 minutes once the records are live. Wait for `ISSUED` before
|
|
96
|
+
touching anything.
|
|
97
|
+
|
|
98
|
+
## Step 5 — Stage the ALB listeners first (safe, additive)
|
|
99
|
+
|
|
100
|
+
`aws elbv2 add-listener-certificates` is **purely additive**. A listener holds many certs and picks
|
|
101
|
+
by SNI, so adding the new cert alongside the old one changes nothing for live traffic. Do this
|
|
102
|
+
ahead of the change window for every listener.
|
|
103
|
+
|
|
104
|
+
## Step 6 — THE ONE PEOPLE MISS: update the Elastic Beanstalk saved config
|
|
105
|
+
|
|
106
|
+
Adding a cert to an ALB listener does **not** update the EB environment's saved setting
|
|
107
|
+
`aws:elbv2:listener:443 / SSLCertificateArns`. After all 12 listeners were fixed and verified
|
|
108
|
+
healthy in this rotation, 12 EB environments still named the **old** cert in their saved config.
|
|
109
|
+
|
|
110
|
+
Nothing breaks right away — the listener serves traffic. But the next **deploy, restart, rebuild or
|
|
111
|
+
scaling event** re-applies the saved config and puts the expired cert back. A listener-level audit
|
|
112
|
+
does not catch this.
|
|
113
|
+
|
|
114
|
+
```
|
|
115
|
+
aws elasticbeanstalk update-environment \
|
|
116
|
+
--environment-name <env> \
|
|
117
|
+
--option-settings "Namespace=aws:elbv2:listener:443,OptionName=SSLCertificateArns,Value=<new-arn>"
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
Observed behaviour: this cleanly **replaced** the old cert on the listener (leaving only the new
|
|
121
|
+
one), and every environment stayed Green/Ok through the update — no traffic disruption.
|
|
122
|
+
|
|
123
|
+
**Gotcha:** an EB environment whose CloudFormation stack is in `DELETE_FAILED` cannot be updated at
|
|
124
|
+
all — `update-environment` is refused. That needs separate stack cleanup.
|
|
125
|
+
|
|
126
|
+
## Step 7 — Check alias coverage before any CloudFront swap
|
|
127
|
+
|
|
128
|
+
CloudFront **rejects** an update if any alias on the distribution is not covered by the new cert.
|
|
129
|
+
Check every alias against the new cert's SANs **programmatically** — this is where the one-label
|
|
130
|
+
wildcard rule bites: `*.togasupply.com` does not cover `compass.beta.togasupply.com`. Beta and
|
|
131
|
+
gamma environments generally need their own certs for this reason.
|
|
132
|
+
|
|
133
|
+
## Step 8 — Swap CloudFront, smallest blast radius first
|
|
134
|
+
|
|
135
|
+
`ViewerCertificate.ACMCertificateArn` is a **single string**, not a list. There is no
|
|
136
|
+
"add new alongside old" for CloudFront: the swap is the cutover, and it deploys in 5-15 minutes.
|
|
137
|
+
|
|
138
|
+
Order used, lowest risk first:
|
|
139
|
+
|
|
140
|
+
1. Distributions that are **already broken** (nothing to lose).
|
|
141
|
+
2. Single-alias distributions.
|
|
142
|
+
3. Mid-size (9 aliases).
|
|
143
|
+
4. The largest last (the 21-alias distribution fronting 21 named enterprise clients).
|
|
144
|
+
|
|
145
|
+
Back up each distribution's config before every swap — `aws cloudfront get-distribution-config`,
|
|
146
|
+
and keep the **ETag**; you need it to update.
|
|
147
|
+
|
|
148
|
+
**Hard edge:** once the old cert's expiry moment has passed there is no useful CloudFront rollback,
|
|
149
|
+
because rolling back restores an expired cert. Only forward.
|
|
150
|
+
|
|
151
|
+
## Step 9 — Verify on the live hosts
|
|
152
|
+
|
|
153
|
+
```
|
|
154
|
+
echo | openssl s_client -servername HOST -connect HOST:443 2>/dev/null \
|
|
155
|
+
| openssl x509 -noout -issuer -enddate
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
Also run it **without** `-servername` to check the no-SNI / default-cert path — that path can still
|
|
159
|
+
be serving the old cert after everything else looks fine.
|
|
160
|
+
|
|
161
|
+
Finish by re-running the step 1 audit: every CloudFront distribution, every ALB listener, and every
|
|
162
|
+
EB saved config should name a cert with a future `NotAfter`.
|
|
163
|
+
|
|
164
|
+
## Gotchas
|
|
165
|
+
|
|
166
|
+
- **IMPORTED certs never auto-renew.** If the audit shows `Type: IMPORTED`, it is a future outage.
|
|
167
|
+
- **A cert copied into 3 regions is 3 separate certs** — all 3 expire at once, and all 3 need
|
|
168
|
+
replacing.
|
|
169
|
+
- **EB saved config is the silent landmine.** See step 6.
|
|
170
|
+
- **CloudFront takes one cert; ALBs take many.** Two completely different risk profiles in the same
|
|
171
|
+
rotation — plan them separately.
|
|
172
|
+
- **Wildcards match one label only.** Verify alias coverage with code, not by eye.
|
|
173
|
+
- **Expired certs hide.** Sweep the whole account; do not trust `InUseBy`.
|
|
174
|
+
|
|
175
|
+
## Change history
|
|
176
|
+
- 2026-09-16 — Created from the account 654654170868 rotation: replaced an expiring 14-SAN IMPORTED
|
|
177
|
+
cert with 6 per-domain Amazon-issued DNS-validated certs, relinked 12 ALB listeners, 6 CloudFront
|
|
178
|
+
distributions and 11 EB environment configs; documented the EB saved-config trap. (rgirish)
|
package/knowledge/INDEX.md
CHANGED
|
@@ -19,7 +19,7 @@ _Auto-generated by `knowledge.js index`. Do not hand-edit._
|
|
|
19
19
|
|
|
20
20
|
## 2.0 framework
|
|
21
21
|
|
|
22
|
-
- **_underscore** (_Underscore) _(framework core)_ —
|
|
22
|
+
- **_underscore** (_Underscore) _(framework core)_ — 86 doc(s) → [2.0/apps/_underscore/INDEX.md](2.0/apps/_underscore/INDEX.md)
|
|
23
23
|
- **worker2** (Worker) — 69 doc(s) → [2.0/apps/worker2/INDEX.md](2.0/apps/worker2/INDEX.md)
|
|
24
24
|
- **api2** (API) — 26 doc(s) → [2.0/apps/api2/INDEX.md](2.0/apps/api2/INDEX.md)
|
|
25
25
|
- **dbchanges2** (Database Changes) _(framework core)_ — 19 doc(s) → [2.0/apps/dbchanges2/INDEX.md](2.0/apps/dbchanges2/INDEX.md)
|
package/package.json
CHANGED