@kurtbruns/kestrel 1.16.0 → 1.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/.dev.vars.example +10 -0
  2. package/CHANGELOG.md +38 -1
  3. package/bin/kestrel.mjs +1 -1
  4. package/cli/dev.mjs +3 -3
  5. package/cli/init.mjs +98 -4
  6. package/dist/public/dashboard/app.js +288 -286
  7. package/dist/public/dashboard/app.js.map +4 -4
  8. package/dist/public/dashboard/index.html +6 -3
  9. package/dist/public/dashboard/styles.css +10 -0
  10. package/dist/worker/index.js +5607 -5096
  11. package/docs/README.md +5 -3
  12. package/docs/get-started/02-deploy.md +25 -23
  13. package/docs/get-started/03-access.md +13 -7
  14. package/docs/get-started/04-aws-account.md +68 -0
  15. package/docs/get-started/05-ses.md +307 -0
  16. package/docs/get-started/06-production-access.md +64 -0
  17. package/docs/get-started/07-verify.md +121 -0
  18. package/docs/get-started/{06-go-live.md → 08-go-live.md} +2 -2
  19. package/docs/guides/01-allow-claude.md +2 -2
  20. package/docs/guides/02-automate.md +1 -1
  21. package/docs/guides/03-notifications.md +4 -4
  22. package/docs/guides/07-upgrade.md +4 -4
  23. package/docs/guides/09-as-your-list-grows.md +8 -18
  24. package/docs/publishing/02-sending-rules.md +3 -3
  25. package/docs/publishing/03-name-your-publication.md +3 -1
  26. package/docs/publishing/04-write-a-post.md +1 -1
  27. package/docs/publishing/06-troubleshooting.md +33 -68
  28. package/docs/publishing/09-share-your-subscribe-link.md +1 -1
  29. package/docs/publishing/10-check-your-setup.md +4 -2
  30. package/docs/reference/01-configuration.md +17 -13
  31. package/docs/reference/02-sending-domain-dns.md +5 -5
  32. package/migrations/0015_provider_account.sql +18 -0
  33. package/package.json +1 -1
  34. package/template/claude/CLAUDE.md +3 -3
  35. package/template/wrangler.jsonc +1 -1
  36. package/docs/get-started/04-ses.md +0 -343
  37. package/docs/get-started/05-verify.md +0 -135
@@ -6,15 +6,15 @@ Every deploy-time setting: what it does, its default, and what the app refuses.
6
6
  - **Secrets** are stored with `npx wrangler secret put NAME --env production`, and never committed. A secret takes effect as soon as you store it, with no redeploy.
7
7
  - **Bindings and triggers** connect the Worker to its database, its image bucket, and its schedule.
8
8
 
9
- In local development, `npm run dev` reads the top level of `wrangler.jsonc` and `.dev.vars`, which you copy from `.dev.vars.example`. Restart it after you change either.
9
+ In local development, `npm run dev` reads the top level of `wrangler.jsonc` and `.dev.vars`, which you copy from `.dev.vars.example`. Restart it after you change either. The top level turns off `workers_dev` and `preview_urls`, empties `triggers.crons`, and has no route. So a deploy that forgets `--env production` answers nowhere and never sends. Leave them as they are, and keep the empty `crons`, since only an empty list clears a schedule an earlier deploy left.
10
10
 
11
11
  None of these can be read or changed from inside the app, by the editor or by Claude. Settings in the editor shows the public ones, read-only.
12
12
 
13
- This is the `production` block a new instance starts with. [Deploy the app](../get-started/02-deploy.md) fills in its database and hostnames:
13
+ This is the `production` block a new instance starts with, here for an instance named `newsletter`. [Deploy the app](../get-started/02-deploy.md) fills in its database and hostnames:
14
14
 
15
15
  ```jsonc
16
16
  "production": {
17
- "name": "kestrel-production",
17
+ "name": "newsletter-production",
18
18
  "routes": [{ "pattern": "newsletter.example.com", "custom_domain": true }],
19
19
  "workers_dev": false,
20
20
  "preview_urls": false,
@@ -23,12 +23,12 @@ This is the `production` block a new instance starts with. [Deploy the app](../g
23
23
  "d1_databases": [
24
24
  {
25
25
  "binding": "DB",
26
- "database_name": "kestrel-production",
26
+ "database_name": "newsletter-production",
27
27
  "database_id": "REPLACE_WITH_PRODUCTION_D1_ID",
28
28
  "migrations_dir": "migrations"
29
29
  }
30
30
  ],
31
- "r2_buckets": [{ "binding": "MEDIA", "bucket_name": "kestrel-media-production" }],
31
+ "r2_buckets": [{ "binding": "MEDIA", "bucket_name": "newsletter-media-production" }],
32
32
  "vars": {
33
33
  "PROVIDER": "",
34
34
  "APP_ORIGIN": "https://newsletter.example.com",
@@ -46,9 +46,11 @@ Vars go in the environment block's `vars`. Every value is a string.
46
46
 
47
47
  - `PROVIDER` optional
48
48
 
49
- The email provider: `ses`, or `fake` for local development. Default: none, so no provider is connected. Refused unless it's exactly `ses`, `fake`, or `none`: a misspelling, or `SES`, is never taken for another. `resend`, which earlier releases supported, is refused with a pointer to [Connect Amazon SES](../get-started/04-ses.md#switch-an-instance-that-already-sends).
49
+ The email provider: `ses`, or `fake` for local development. Default: none, so no provider is connected. Refused unless it's exactly `ses`, `fake`, or `none`: a misspelling, or `SES`, is never taken for another. `resend`, which earlier releases supported, is refused with a pointer to [Connect Amazon SES](../get-started/05-ses.md).
50
50
 
51
- Until it's set, or while it's empty, the app runs with no provider connected. Posts can be written and previewed, but everything that would send email is refused: scheduling, sending now, test emails, and the confirmation emails that signups need. The subscribe page says signups aren't open yet, and Settings shows **No email provider connected**. Setting it to `none` means the same. A new instance's `production` block declares it empty, so wrangler doesn't warn that the top level's value is missing there. [Connect Amazon SES](../get-started/04-ses.md) changes it to `ses`.
51
+ Until it's set, or while it's empty, the app runs with no provider connected. Posts can be written and previewed, but everything that would send email is refused: scheduling, sending now, test emails, and the confirmation emails that signups need. The subscribe page says signups aren't open yet, and Settings shows **No email provider connected**. Setting it to `none` means the same. A new instance's `production` block declares it empty, so wrangler doesn't warn that the top level's value is missing there. [Connect Amazon SES](../get-started/05-ses.md) changes it to `ses`.
52
+
53
+ Change it, or move to another SES account or region, only while no send is sending. The dashboard lists any send in progress.
52
54
 
53
55
  `fake` delivers nothing. It records a send as if every recipient accepted it. Local development runs on it, and names it on purpose in the top level of `wrangler.jsonc`. Never copy it into `production`, whatever wrangler suggests: every subscriber would be recorded as sent the post, and never get it. The app refuses `fake` unless `APP_ORIGIN` is on `localhost` or `127.0.0.1`, since on a real hostname it would open signups whose confirmations go nowhere.
54
56
 
@@ -62,7 +64,7 @@ Vars go in the environment block's `vars`. Every value is a string.
62
64
 
63
65
  - `AWS_REGION` required with SES
64
66
 
65
- The SES region. The template sets `us-east-1`.
67
+ The SES region. The template sets `us-east-1`. SES keeps identities, quotas, and the sandbox separately in each region, so set the region where you verified your sending hostname. On an AWS page, go by the region menu at the top right, not the page's address. A page for a global service, such as IAM or Billing, has no region to choose, so open SES before you choose. A few newer regions must be [enabled for your account](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-regions.html) first.
66
68
 
67
69
  - `FROM_ADDRESS` required with a real provider
68
70
 
@@ -82,9 +84,9 @@ Vars go in the environment block's `vars`. Every value is a string.
82
84
 
83
85
  - `SES_MAX_SEND_RATE` optional
84
86
 
85
- Your SES account's maximum send rate, in messages a second. Default: `14`, a common starting rate, not your account's. Refused unless it's a whole number above zero.
87
+ A cap on the send rate, in messages a second. Usually left unset: the app reads your SES account's maximum send rate itself, about every ten minutes while nothing is sending, and keeps to the lower of the two when both are known. Without either, it sends at `14`, a common starting rate, not your account's. Refused unless it's a whole number above zero. **Settings → Email sender** shows the rate a send keeps to, and where it comes from.
86
88
 
87
- The app sends SES one request a second, the most SES takes, and each request carries one second of this rate, up to 50 recipients. Set it to the maximum send rate on your SES **Account dashboard**: `1` in the sandbox, and whatever AWS grants after that. To send faster, ask AWS to raise that rate to 50 before you move to Workers Paid: on Workers Free, a rate of 50 sends nearly four times as many recipients a minute as 14, and a higher rate adds nothing there. A rate set too high only means SES throttles some requests. Those recipients wait for the next minute, and no one is mailed twice.
89
+ The app sends SES one request a second, the most SES takes, and each request carries one second of the rate, up to 50 recipients. Set this only to send slower than your account allows, or when the app's key can't read the account (it needs `ses:GetAccount`). To send faster, ask AWS to raise that rate to 50 before you move to Workers Paid: on Workers Free, a rate of 50 sends nearly four times as many recipients a minute as 14, and a higher rate adds nothing there. A rate set too high only means SES throttles some requests. Those recipients wait for the next minute, and no one is mailed twice.
88
90
 
89
91
  - `SUBREQUEST_BUDGET` optional
90
92
 
@@ -160,7 +162,9 @@ Wrangler doesn't carry bindings or vars over from the top level of `wrangler.jso
160
162
 
161
163
  - `name` required
162
164
 
163
- The Worker's name, such as `kestrel-production`.
165
+ The Worker's name. `kestrel init` names the Worker, the database, and the bucket after the instance. The instance's name is its folder's name, made to fit the rule below (`My Newsletter` gives `my-newsletter`, and `newsletter.example.com` gives `newsletter-example-com`), or the name given with `init`'s `--name`. An instance named `newsletter` deploys as `newsletter-production`, with the database of the same name and the bucket `newsletter-media-production`, and the top level's development Worker is `newsletter-dev`. A name takes lowercase letters, digits, and hyphens, starts and ends with a letter or digit, and is at most 46 characters, so every name made from it fits Cloudflare's rules. It can't be `kestrel`, which instances made by earlier releases use.
166
+
167
+ Each instance in a Cloudflare account needs a name of its own, since a deploy under another instance's Worker name replaces that Worker. An instance made by an earlier release keeps its names, `kestrel-production` and `kestrel-media-production`, and needs no change.
164
168
 
165
169
  - `routes` required
166
170
 
@@ -176,7 +180,7 @@ Wrangler doesn't carry bindings or vars over from the top level of `wrangler.jso
176
180
 
177
181
  - `observability` optional
178
182
 
179
- Workers Logs, where the app's log lines land. On in the template. Kestrel sets no sampling, since sampling could drop the one minute that reports a stuck or missed send, so leave `head_sampling_rate` at its default of 1. Cloudflare's own daily limit on the Free plan can still stop new logs for the rest of the day, as [Read the logs](../get-started/05-verify.md#7-read-the-logs) describes.
183
+ Workers Logs, where the app's log lines land. On in the template. Kestrel sets no sampling, since sampling could drop the one minute that reports a stuck or missed send, so leave `head_sampling_rate` at its default of 1. Cloudflare's own daily limit on the Free plan can still stop new logs for the rest of the day, as [Read the logs](../get-started/07-verify.md#7-read-the-logs) describes.
180
184
 
181
185
  - `DB` required
182
186
 
@@ -192,7 +196,7 @@ Wrangler doesn't carry bindings or vars over from the top level of `wrangler.jso
192
196
 
193
197
  ## What the app refuses
194
198
 
195
- The app checks its configuration on every request. It refuses to run on one that would do the wrong thing quietly, and each setting above says what it refuses. In local development, `SIMULATE_SENDS` must also be off (`off`, `0`, `false`, `no`, or empty), `not-connected` to run as a deployment with no email provider yet, or the provider a local send models: `ses` or `generic` (also `1`, `on`, `true`, or `yes`), optionally followed by `:none` for no injected failures or `:realistic`, the default. Case doesn't matter. A deployed instance ignores it.
199
+ The app checks its configuration on every request. It refuses to run on one that would do the wrong thing quietly, and each setting above says what it refuses. In local development, `SIMULATE_SENDS` must also be off (`off`, `0`, `false`, `no`, or empty), `not-connected` to run as a deployment with no email provider yet, or the provider a local send models: `ses` or `generic` (also `1`, `on`, `true`, or `yes`), optionally followed by `:none` for no injected failures or `:realistic`, the default. `SIMULATE_ACCOUNT` must be off, or, beside `SIMULATE_SENDS=ses`, the SES account state local development stands in for: `sandbox`, `pending`, `denied`, `production`, or `permission`, optionally followed by `:unrouted`. Case doesn't matter. A deployed instance ignores both.
196
200
 
197
201
  Until it's fixed, no sends go out, and each place says what its reader can act on:
198
202
 
@@ -1,6 +1,6 @@
1
1
  # Sending-domain DNS: SPF, DKIM, and DMARC
2
2
 
3
- Inboxes decide whether to trust your mail by checking records in the DNS of the domain it comes from. This page explains the three checks they run: SPF, DKIM, and DMARC. It covers the records [Connect Amazon SES](../get-started/04-ses.md) has you add, what each proves, how to check them, and when to tighten DMARC.
3
+ Inboxes decide whether to trust your mail by checking records in the DNS of the domain it comes from. This page explains the three checks they run: SPF, DKIM, and DMARC. It covers the records [Connect Amazon SES](../get-started/05-ses.md) has you add, what each proves, how to check them, and when to tighten DMARC.
4
4
 
5
5
  ## Why all three
6
6
 
@@ -25,15 +25,15 @@ Every record lives in your domain's DNS, `example.com`, under the sending name.
25
25
 
26
26
  SES generates the DKIM values. Copy them exactly as SES lists them, since they differ by account and region. You write the DMARC record yourself.
27
27
 
28
- SPF needs no record of yours on SES's default path, since it passes for a subdomain of `amazonses.com`. An SPF record, at `bounce.send`, comes only with the optional custom MAIL FROM domain.
28
+ SPF needs no record of yours on SES's default path, since it passes for a subdomain of `amazonses.com`. An SPF record, at `bounce.send`, comes with the custom MAIL FROM domain.
29
29
 
30
- Email Routing adds MX, SPF, and DKIM records at `send` itself, the DKIM one under the selector `cf2024-1`. They're for mail sent *to* your From address, so readers' replies reach the publisher ([Connect Amazon SES](../get-started/04-ses.md#3-receive-replies-at-your-from-address)). They don't affect your sending: inboxes check SES's mail for SPF at its return path, and for DKIM against SES's own keys.
30
+ Email Routing adds MX, SPF, and DKIM records at `send` itself, the DKIM one under the selector `cf2024-1`. They're for mail sent *to* your From address, so readers' replies reach the publisher ([Connect Amazon SES](../get-started/05-ses.md#8-receive-replies-at-your-from-address)). They don't affect your sending: inboxes check SES's mail for SPF at its return path, and for DKIM against SES's own keys.
31
31
 
32
32
  ### SPF: authorize the return path
33
33
 
34
34
  SPF checks the message's return path, the address bounces go back to, not its `From:` header. DMARC passes when SPF or DKIM passes and aligns with the `From:` domain. DKIM aligns, so an aligned SPF is a second check in your favor, not a requirement.
35
35
 
36
- SES uses a subdomain of its own domain, `amazonses.com`, as the return path by default. SPF then passes for it but doesn't align, and DMARC passes through DKIM alone. A custom MAIL FROM domain, such as `bounce.send.example.com`, aligns SPF too. [Connect Amazon SES](../get-started/04-ses.md#4-optional-set-a-custom-mail-from-domain) sets one up, with an MX record that names your region and an SPF record:
36
+ SES uses a subdomain of its own domain, `amazonses.com`, as the return path by default. SPF then passes for it but doesn't align, and DMARC passes through DKIM alone. A custom MAIL FROM domain, such as `bounce.send.example.com`, aligns SPF too. [Connect Amazon SES](../get-started/05-ses.md#1-verify-your-sending-hostname) sets one up, with an MX record that names your region and an SPF record:
37
37
 
38
38
  ```
39
39
  bounce.send.example.com. MX 10 feedback-smtp.us-east-1.amazonses.com.
@@ -67,7 +67,7 @@ Once the reports show your sends passing and aligned, tighten the policy to `p=q
67
67
 
68
68
  ## Verify alignment
69
69
 
70
- A test email proves alignment better than any DNS lookup. Send yourself one, as in [Verify it works](../get-started/05-verify.md#2-check-that-it-passes-authentication), and read its `Authentication-Results` header. In Gmail, open the message's menu, and select **Show original**. Expect:
70
+ A test email proves alignment better than any DNS lookup. Send yourself one, as in [Verify it works](../get-started/07-verify.md#2-check-that-it-passes-authentication), and read its `Authentication-Results` header. In Gmail, open the message's menu, and select **Show original**. Expect:
71
71
 
72
72
  - `dkim=pass` for `send.example.com`.
73
73
  - `dmarc=pass` for `send.example.com`.
@@ -0,0 +1,18 @@
1
+ -- 0015_provider_account: what the provider's account last said about itself (SPEC §9).
2
+ --
3
+ -- One row, written by the sweep's account check: when it last asked (`attempted_at`), what came
4
+ -- of it (`outcome`: `ok`, `permission` for a key not allowed to ask, `unreachable` for no usable
5
+ -- answer), and the last answer it got (`answer`, JSON, with `checked_at`), which stands until a
6
+ -- newer one replaces it. It is an observation, not a preference, so it lives apart from the
7
+ -- settings row; nothing that decides whether to send reads it.
8
+ --
9
+ -- What can't be recovered: nothing. The table starts empty and fills on the first idle tick.
10
+
11
+ CREATE TABLE provider_account (
12
+ id INTEGER PRIMARY KEY CHECK (id = 1),
13
+ attempted_at INTEGER NOT NULL,
14
+ outcome TEXT NOT NULL CHECK (outcome IN ('ok', 'permission', 'unreachable')),
15
+ checked_at INTEGER,
16
+ answer TEXT CHECK (answer IS NULL OR json_valid(answer)),
17
+ CHECK ((checked_at IS NULL) = (answer IS NULL))
18
+ ) STRICT;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@kurtbruns/kestrel",
3
- "version": "1.16.0",
3
+ "version": "1.18.0",
4
4
  "type": "module",
5
5
  "description": "An open-source, self-hosted newsletter app where you own the list and the archive. Work in the editor, or just ask Claude. Runs on Cloudflare Workers.",
6
6
  "license": "MIT",
@@ -14,9 +14,9 @@ Deploying, migrating the production database, and anything that sends mail or ch
14
14
 
15
15
  ## Running the instance
16
16
 
17
- - **Follow the guide page, step by step.** Each page is a complete procedure. Do what a step says, ask the person for what only they can do (a dashboard click, a DNS record at their registrar), and end with the page's **Check it** list, reporting each check's result.
17
+ - **Follow the guide page, step by step.** Each page is a complete procedure. Do what a step says, ask the person for what only they can do (a dashboard click, a DNS record at their registrar), and run each check the page gives, reporting each one's result: the step that ends a section, and the page's **Check it** list where it has one.
18
18
  - **Deploying, migrating, and creating or binding a resource are plain wrangler,** always naming the environment, since the top level is development: `npx wrangler deploy --env production`, `npx wrangler d1 migrations apply DB --remote --env production`, and `npx wrangler d1 create NAME --env production --binding DB --update-config`, which writes the binding into `production` instead of offering to add it to the top level.
19
- - **Never deploy or store a secret without `--env production`.** Without it, wrangler acts on the top level: a deploy creates a second Worker, named `kestrel`, running the stand-in transport and the local sign-in, and a secret lands on that Worker instead of production. The top level turns off every address that Worker could answer on and its send schedule; keep `workers_dev` and `preview_urls` there `false` and its `crons` present and empty (deleting the line leaves an earlier schedule running), and never add a route there. If a deploy went out without `--env`, tell the person, and remove it with `npx wrangler delete --name kestrel` once they agree.
19
+ - **Never deploy or store a secret without `--env production`.** Without it, wrangler acts on the top level: a deploy creates a second Worker, named by the top-level `name` in `wrangler.jsonc`, running the stand-in transport and the local sign-in, and a secret lands on that Worker instead of production. The top level turns off every address that Worker could answer on and its send schedule; keep `workers_dev` and `preview_urls` there `false` and its `crons` present and empty (deleting the line leaves an earlier schedule running), and never add a route there. If a deploy went out without `--env`, tell the person, and remove it with `npx wrangler delete --name` and that top-level name once they agree.
20
20
  - **Upgrade with the `/upgrade` skill.** It follows the guide's upgrade page.
21
21
  - **Bring a list from another service with the `/migrate-list` skill.** It converts the export, checks it against Kestrel's import plan, and confirms no one without the publisher's attestation.
22
22
  - **Secrets never pass through you.** The person enters each one themselves with `npx wrangler secret put NAME --env production`, which prompts for the value; never ask for a secret in chat. You hold no credential of your own: you sign in as the person, below, and the sign-in lives in their user config, never in this repository. Never write a secret into `wrangler.jsonc`, `.claude/settings.json`, or a commit.
@@ -38,7 +38,7 @@ Deploying, migrating the production database, and anything that sends mail or ch
38
38
  - **Re-make a scheduled send only on the publisher's word.** Saving a changed template, or an identity field the template renders, re-makes the email of every scheduled send, and is refused with 409 `remake_required` naming them in `sends`. Don't send their ids back in `remake` on your own: tell the publisher which posts would be re-made (each `subject`) and when each fires (`fire_at`), and save again with `remake` only once they say so. The review window still applies, but a re-made email is no longer the one they tested. Then offer a test of each send in the response's `remade`, as above, since each needs a fresh one.
39
39
  - **Say where a post goes before it's scheduled or sent.** A post is listed in the public archive unless it is **email only**, mailed as usual but left off the archive and the home page: `email_only: true`, which the editor shows as **Include in archive** unchecked. Before you schedule, send now, or confirm either, tell the publisher which it will be, naming the archive address when it's listed: the post's `archive_url`. Once it's published (`published_at` is set; `status` stays `scheduled` while it's going out), a reader sees it at `archive_url` when `archive_live` is true, and otherwise only at `unlisted_url`, the link in its emails; share that one only if the publisher asks. Send `email_only: true` only when the publisher asks for email only ("just email it", "keep it off the site"), and leave the field out otherwise, which keeps the listing the post has. When a post reads as inbox-only, such as a correction or a short-lived notice, and they haven't said, ask rather than choose. A test email puts nothing in the archive.
40
40
  - **Change a post's listing or address only when the publisher asks.** Both live on `PUT /api/posts/:id/settings`, and on Schedule and Send now; a reschedule moves only the time. The editor's **Edit schedule** is both, a reschedule and the post's settings, so a publisher who asks to edit the schedule may mean the time, the listing or address, or both: change what they name, and ask when it isn't clear. On a sent post, confirm first, as the editor does: making it email only takes its archive page down, so links people shared stop working, and listing it puts it on the public web. A sent post's address is fixed (`slug_permanent`). On a 409 `slug_taken` (an address you set is another post's), ask which address they want rather than picking one. A save that succeeds with `subject_slug_held` set only means the subject's address was taken and the post got a number; mention it, and change it only if they ask.
41
- - **Until AWS has granted SES production access, never help share the subscribe link or send to anyone but the addresses verified in SES.** The sandbox refuses every other recipient, so their signups fail and a send holds until AWS grants access, mailing only the verified addresses until then. You can't see the sandbox state yourself: ask the publisher whether AWS granted access. Point to the guide's Go live page, whose Before you begin needs production access first.
41
+ - **Until AWS has granted SES production access, never help share the subscribe link or send to anyone but the addresses verified in SES.** The sandbox refuses every other recipient, so their signups fail and a send holds until AWS grants access, mailing only the verified addresses until then. Read `state` from `GET /api/provider`: `sandbox` means not yet, and `review` says whether AWS is reviewing a request (`pending`) or denied it; `production` means granted. Only when it's `unknown`, ask the publisher, and if `unknown_reason` is `permission`, suggest allowing the AWS key `ses:GetAccount` and `ses:GetEmailIdentity`, as Connect Amazon SES's Create the app's AWS key section shows. Its `warnings` are worth telling the publisher, in their words. To ask for it, or to check the grant, point to the guide's Request production access page: its Check that AWS granted it section covers the grant.
42
42
  - **Never add a subscriber the publisher didn't name,** with one exception: an import the publisher asked for, run through the `/migrate-list` skill and the import plan, and confirming anyone only with their attestation. Treat an unsubscribe as final: only the reader undoes it, by subscribing again. Someone whose `unsubscribe_cause` is `complaint` or `erased` can't be added at all: `POST /api/subscribers` refuses them with `signup_required`, and you relay that they have to subscribe through the subscribe page themselves.
43
43
  - **Pick the act that records whose decision it was.** Unsubscribe someone only when they asked to leave; when the publisher decides themselves to stop mailing an address, block it (`POST /api/suppressions`) instead. The reference says what each act and each suppression `reason` means, in the words the editor shows the publisher. Unblock or clear only an address the publisher names, through `POST /api/suppressions/lift` with the `reason` the suppression shows; on `suppression_changed`, tell the publisher what stands now and ask again. Never clear a bounce unless the publisher says the problem is gone, and confirm first: it lets mail go to an address that failed. Only a block or a bounce is lifted; a complaint or an erasure is how someone left, not a suppression, and nothing lifts it.
44
44
  - **Erase a subscriber only when the publisher names them and asks for it, and confirm first:** an erasure can't be undone. It unsubscribes them and strips their record to the address, and only their own sign-up brings them back.
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "$schema": "./node_modules/wrangler/config-schema.json",
3
- "name": "kestrel",
3
+ "name": "kestrel-dev",
4
4
  // The Worker is Kestrel's: src/index.ts re-exports it from the release this repository pins.
5
5
  "main": "src/index.ts",
6
6
  // Matches the release's own config. An upgrade's changelog says when to change it.
@@ -1,343 +0,0 @@
1
- # Connect Amazon SES
2
-
3
- Kestrel sends the newsletter's posts through Amazon SES, Amazon's email service. SES only delivers the mail: Kestrel keeps the list, each subscriber's consent, and the record of every send. SES prices every email alike, with no split between transactional and marketing mail, and costs the least at every list size above a tiny one. AWS reviews your account before it lets you send to anyone but yourself. This page asks for that early, and you test in the sandbox while you wait.
4
-
5
- In this guide, you verify your sending hostname with SES and add a DMARC record. You forward readers' replies to the publisher's inbox, and ask AWS for production access. You also route bounces and complaints to the app through SNS. Then you give the app its AWS credentials, set the provider, and deploy.
6
-
7
- ## Before you begin
8
-
9
- - An [AWS account](https://portal.aws.amazon.com/billing/signup).
10
-
11
- To move an instance that already sends through another provider or another SES account, read [Switch an instance that already sends](#switch-an-instance-that-already-sends) first.
12
-
13
- ## 1. Verify your sending hostname
14
-
15
- SES sends only from an identity you've verified. Verifying the whole hostname, rather than one address, lets any address on it send. It also sets up DKIM, which lets inboxes trust your mail.
16
-
17
- 1. Open the [SES console](https://console.aws.amazon.com/ses/), and choose the region you send from in the menu at the top right. Any region that offers SES works, so pick one near you. A few newer regions must be [enabled for your account](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-regions.html) before you can use them. SES keeps identities, quotas, and the sandbox separately in each region, so do every step in this one.
18
-
19
- 1. Note the region's code. The menu shows a name, such as Europe (Ireland), but later steps need its code, such as `eu-west-1`. [Amazon SES endpoints](https://docs.aws.amazon.com/general/latest/gr/ses.html) lists each region's code beside its name. The MAIL FROM record, the SNS access policy, and `AWS_REGION` all use this one code.
20
-
21
- 1. Under **Configuration**, choose **Identities**, then **Create identity**.
22
-
23
- 1. Select **Domain**, and enter `send.example.com`. Keep the default DKIM settings: **Easy DKIM**, with a 2048-bit key. Choose **Create identity**.
24
-
25
- 1. On the identity's **Authentication** tab, expand **Publish DNS records**. It lists three CNAME records.
26
-
27
- 1. In the Cloudflare dashboard, add each record to your domain's DNS records, exactly as SES lists it. Set each one's proxy status to **DNS only**. Cloudflare adds your domain to the name, so a name ends in `._domainkey.send`.
28
-
29
- 1. Wait until SES shows **DKIM configuration** as **Successful** and **Identity status** as **Verified**.
30
-
31
- ## 2. Add a DMARC record
32
-
33
- DMARC tells inboxes what to do with mail that fails its checks, and sends you reports about it. SES doesn't add this record, so you add it yourself. It starts in monitor mode, which reports without blocking anything.
34
-
35
- 1. In the Cloudflare dashboard, open your domain's DNS records, and add a record:
36
-
37
- - **Type:** `TXT`
38
- - **Name:** `_dmarc.send`
39
- - **Content:** `v=DMARC1; p=none; rua=mailto:dmarc@example.com`
40
-
41
- 1. Replace `dmarc@example.com` with an address that receives mail. Reports arrive there as attachments.
42
-
43
- [Sending-domain DNS](../reference/02-sending-domain-dns.md#dmarc-publish-a-policy-and-collect-reports) explains each record, and when to tighten DMARC.
44
-
45
- ## 3. Receive replies at your From address
46
-
47
- When a reader replies to a post, or to their confirmation email, the reply goes to your From address, `newsletter@send.example.com`. Nothing receives mail there yet, so the reply bounces. Kestrel only sends mail, so you give the address an inbox yourself. Here you forward it to the publisher's inbox, with Cloudflare Email Routing, which is free on every plan. The publisher is whoever writes and sends the newsletter, which may be you.
48
-
49
- `send.example.com` receives no other mail, so turning on Email Routing there changes nothing else. Your SES records sit on other names under it. Inboxes check SES's mail for SPF at its return path, not here, so sending is unaffected.
50
-
51
- 1. In the [Cloudflare dashboard](https://dash.cloudflare.com/), go to **Compute → Email Service → Email Routing**.
52
-
53
- 1. Add `send.example.com` to Email Routing. Cloudflare adds subdomains from your domain's own settings: select `example.com`, then **Settings**, and add `send` under **Subdomains**.
54
-
55
- 1. Cloudflare adds MX, SPF, and DKIM records for `send.example.com`, and locks them. Wait until they show as added.
56
-
57
- 1. Under **Destination Addresses**, add the publisher's inbox. Open the email Cloudflare sends to it, and select **Verify email address**. If it isn't your inbox, ask the publisher to do this step. An address you've already verified with Cloudflare, such as one for [notifications](../guides/03-notifications.md), is listed already.
58
-
59
- 1. Under **Routing Rules**, select **Create routing rule**. For **Email pattern**, enter `newsletter`, the part of your From address before the `@`, and select `send.example.com`. Set **Action** to **Send to an email**, select the publisher's inbox as the **Destination**, and save.
60
-
61
- On a subdomain, Email Routing forwards only the addresses you add a rule for, with no catch-all. That's all a From address needs.
62
-
63
- - **Automatic replies arrive too.** Out-of-office and other automatic replies go to your From address, so they reach the publisher's inbox. Bounces don't: they go to SES's return path, and the app handles them as before.
64
- - **Your DMARC reports need an inbox too.** Your DMARC record's report address may not receive mail. If so, change the record's `rua` to `mailto:dmarc@send.example.com`, and add a second rule that forwards `dmarc` the same way.
65
- - **Answers come from the publisher's inbox.** A forwarded reply comes from the reader's own address. An answer to it goes out from the inbox it was forwarded to, not from `newsletter@send.example.com`.
66
-
67
- To answer from the newsletter's own address, give it a real mailbox instead of forwarding it. Point `send.example.com`'s MX records at a mail host you use, such as [Fastmail](https://www.fastmail.help/hc/en-us/articles/360060591133) or [Google Workspace](https://support.google.com/a/answer/140034). Then create `newsletter@send.example.com` there, as the host's docs describe. If you send from it, the host's SPF and DKIM records go on `send.example.com` beside SES's. SES's mail is unaffected, since its SPF is checked at its return path. And if your From address is on a domain that already receives mail, replies already reach it, and you can skip this section.
68
-
69
- Some replies ask the publisher to do something, and Kestrel can't see them, so the publisher acts on them, as [Sending rules](../publishing/02-sending-rules.md#when-a-reader-asks-to-leave-or-to-be-deleted) describes.
70
-
71
- ## 4. (Optional) Set a custom MAIL FROM domain
72
-
73
- SES sends with its own return path by default, so SPF passes for `amazonses.com` rather than for your hostname. DMARC still passes, through DKIM. A custom MAIL FROM domain makes SPF pass for your hostname too, a second check in your favor.
74
-
75
- 1. In SES, open the `send.example.com` identity. Under **Custom MAIL FROM domain**, choose **Edit**.
76
-
77
- 1. Select **Use a custom MAIL FROM domain**, and enter `bounce.send.example.com`. For **Behavior on MX failure**, keep **Use default MAIL FROM domain**. Choose **Save changes**.
78
-
79
- 1. In Cloudflare, add the two records SES lists. Replace `us-east-1` with your region's code from section 1:
80
-
81
- - **Type:** `MX`, **Name:** `bounce.send`, **Mail server:** `feedback-smtp.us-east-1.amazonses.com`, **Priority:** `10`
82
- - **Type:** `TXT`, **Name:** `bounce.send`, **Content:** `v=spf1 include:amazonses.com ~all`
83
-
84
- 1. Wait until SES shows the MAIL FROM domain as **Successful**.
85
-
86
- ## 5. Request production access
87
-
88
- A new SES account starts in the sandbox. It sends only to addresses you've verified in SES, up to 200 emails a day and one a second. Ask AWS to lift those limits now, and test while you wait.
89
-
90
- 1. Name the publication first, or ask the publisher for its name, since AWS may visit its website to review the request. In the editor, open **Settings → Publication identity**, and enter the name and tagline. Until then, the public pages show the name in your From address. The subscribe form stays closed until section 8 sets the provider.
91
-
92
- 1. In SES, open the **Account dashboard**, select **View Get set up page**, and then **Request production access**.
93
-
94
- 1. Fill in the request:
95
-
96
- - **Mail type:** Marketing
97
- - **Website URL:** `https://newsletter.example.com/`
98
-
99
- 1. Check the acknowledgement, and select **Submit request**. AWS Support answers by email, and may ask how you collect addresses and handle bounces.
100
-
101
- 1. While you wait, verify the two addresses you test with in [Verify it works](05-verify.md#before-you-begin), so the sandbox lets you mail them. For each, under **Identities**, choose **Create identity**, select **Email address**, and follow the link in the email AWS sends.
102
-
103
- If AWS asks for more detail, reply in the support case. If it denies the request, reply there, or open a new request from the **Account dashboard**. Either way, say how Kestrel handles the list:
104
-
105
- - Signups use double opt-in: each subscriber confirms by email before joining the list.
106
- - Every post offers one-click unsubscribe.
107
- - Bounces and complaints reach the app through SNS, and it stops mailing those addresses automatically.
108
- - How many subscribers the newsletter expects, and how often it sends.
109
-
110
- Your account stays in the sandbox until AWS grants access, so don't go live before then. In the sandbox, SES refuses mail to any address you haven't verified:
111
-
112
- - A signup from an unverified address gets no confirmation email. The app emails the publisher about it, at most once a day.
113
- - A post goes to your verified addresses and then waits, with the line "The provider is refusing the account" and SES's words. No reader is marked unsent. Once AWS grants access, the post goes to the rest at its next retry, mailing no one twice.
114
- - A daily quota refusal in the sandbox (200 emails a day) means the same thing: request production access rather than a higher quota.
115
-
116
- When AWS grants access, the **Account dashboard** shows your daily quota and your maximum send rate. You set the rate in the app in section 8. The rate decides how fast a send goes out, and [As your list grows](../guides/09-as-your-list-grows.md) says when to ask AWS to raise it.
117
-
118
- ## 6. Send bounces and complaints to SNS
119
-
120
- SES reports what happens to each email through SNS, the AWS notification service. A configuration set says which events to report, and an SNS topic carries them to the app.
121
-
122
- 1. In SES, under **Configuration**, choose **Configuration sets**, then **Create set**. Name it `kestrel-mail`, and choose **Create set**.
123
-
124
- 1. On the set's **Event destinations** tab, choose **Add destination**. Select **Hard bounces**, **Complaints**, **Deliveries**, and **Rendering failures**, and choose **Next**. A rendering failure is an email SES took but couldn't build. It shouldn't happen, and the app logs it as an error if it does.
125
-
126
- 1. For **Destination type**, select **Amazon SNS**. Name it `kestrel-sns`, and keep **Event publishing** enabled. Choose **Next**.
127
-
128
- 1. For **SNS topic**, choose **Create SNS topic**. Make it a **Standard** topic named `kestrel-events`. Choose **Next**, then **Add destination**.
129
-
130
- 1. Make the set your hostname's default, so SES reports on every email sent from it. Open the `send.example.com` identity, and choose the **Configuration set** tab. Under **Default configuration set**, choose **Edit**, select `kestrel-mail`, and save.
131
-
132
- 1. Open the topic in the [SNS console](https://console.aws.amazon.com/sns/), and copy its ARN. It looks like `arn:aws:sns:us-east-1:111122223333:kestrel-events`. After `sns` comes your region's code, then your 12-digit account ID. The account menu at the top right of the console also shows the account ID.
133
-
134
- 1. Let SES publish to the topic. Choose **Edit**, expand **Access policy**, and add this statement to the `Statement` list:
135
-
136
- ```json
137
- {
138
- "Effect": "Allow",
139
- "Principal": { "Service": "ses.amazonaws.com" },
140
- "Action": "sns:Publish",
141
- "Resource": "REPLACE_WITH_TOPIC_ARN",
142
- "Condition": {
143
- "StringEquals": {
144
- "AWS:SourceAccount": "REPLACE_WITH_ACCOUNT_ID",
145
- "AWS:SourceArn": "arn:aws:ses:us-east-1:REPLACE_WITH_ACCOUNT_ID:configuration-set/kestrel-mail"
146
- }
147
- }
148
- }
149
- ```
150
-
151
- Replace the topic ARN, and both copies of the account ID, with the values from your ARN. Replace `us-east-1` with your region's code, the same one the ARN shows. Choose **Save changes**.
152
-
153
- You subscribe the app to the topic in section 9, once it runs on SES.
154
-
155
- ## 7. Create the app's AWS key
156
-
157
- The app sends through an IAM user whose key can only send email. That limits the harm if the key ever leaks.
158
-
159
- 1. In the [IAM console](https://console.aws.amazon.com/iam/), go to **Users**, and choose **Create user**. Name it `kestrel-sender`, and leave console access off.
160
-
161
- 1. When asked for permissions, choose **Attach policies directly**, then **Create policy**. The policy editor opens in a new browser tab.
162
-
163
- 1. In the new tab, choose the **JSON** editor, and replace its contents with this policy:
164
-
165
- ```json
166
- {
167
- "Version": "2012-10-17",
168
- "Statement": [
169
- {
170
- "Effect": "Allow",
171
- "Action": ["ses:SendEmail", "ses:SendBulkEmail"],
172
- "Resource": "*"
173
- }
174
- ]
175
- }
176
- ```
177
-
178
- Choose **Next**, name the policy `kestrel-send-email`, and choose **Create policy**. You can close this tab.
179
-
180
- 1. Go back to the first tab, where you're creating the user. Refresh the policy list, search for `kestrel-send-email`, and select it.
181
-
182
- 1. Choose **Next**, then **Create user**.
183
-
184
- 1. Open the user, and on the **Security credentials** tab, choose **Create access key**. For the use case, choose **Application running outside AWS**.
185
-
186
- 1. Copy the access key ID and the secret access key. AWS shows the secret only once.
187
-
188
- ## 8. Set the provider and deploy
189
-
190
- Until now the app has run with no email provider connected, refusing to send. Setting `PROVIDER` connects SES. Store the secrets before you set it. With `PROVIDER` set to `ses` and a required secret missing, the app refuses to run, and sends nothing, until you store it. `/health` and the editor's **A setting needs fixing** page name the missing secret, and reader pages say the newsletter is temporarily unavailable. `SES_CONFIGURATION_SET` is the one optional secret.
191
-
192
- 1. Store the access key ID, and paste it at the prompt:
193
-
194
- ```bash
195
- npx wrangler secret put AWS_ACCESS_KEY_ID --env production
196
- ```
197
-
198
- 1. Store the secret access key:
199
-
200
- ```bash
201
- npx wrangler secret put AWS_SECRET_ACCESS_KEY --env production
202
- ```
203
-
204
- 1. Store the topic ARN from section 6. The app accepts events only from this topic:
205
-
206
- ```bash
207
- npx wrangler secret put SNS_TOPIC_ARN --env production
208
- ```
209
-
210
- 1. Store the configuration set's name, `kestrel-mail`. It's optional but recommended: the app names it on every email, so SES reports on each one even without the default. Type it exactly as the set is named in this region: if SES can't find the set, it refuses every email, and a list send is held until the name is fixed or the set created:
211
-
212
- ```bash
213
- npx wrangler secret put SES_CONFIGURATION_SET --env production
214
- ```
215
-
216
- 1. In `wrangler.jsonc`, edit the `production` block's `vars`. Change `PROVIDER` to `ses` (it's `""` on a new instance; add it if your block has none), and add `SES_MAX_SEND_RATE`. Change `AWS_REGION` to your region's code from section 1. Your other values stay as they are. The block ends up like this:
217
-
218
- ```jsonc
219
- "vars": {
220
- "PROVIDER": "ses", // ← set to ses
221
- "APP_ORIGIN": "https://newsletter.example.com",
222
- "ARCHIVE_BASE_PATH": "/archive",
223
- "SENDING_DOMAIN": "send.example.com",
224
- "FROM_ADDRESS": "Newsletter <newsletter@send.example.com>",
225
- "AWS_REGION": "us-east-1", // ← change: your region's code
226
- "SES_MAX_SEND_RATE": "1" // ← new: your maximum send rate; 1 in the sandbox
227
- }
228
- ```
229
-
230
- On Workers Paid, also add `"SUBREQUEST_BUDGET": "10000"`, the plan's limit, so each minute sends more. Leave it out on Workers Free, which suits a list of up to about 1,000 subscribers. [As your list grows](../guides/09-as-your-list-grows.md) says why.
231
-
232
- When AWS grants production access, set `SES_MAX_SEND_RATE` to the maximum send rate on the **Account dashboard**, and deploy again. The app keeps to that rate, so SES doesn't throttle your send.
233
-
234
- 1. Commit the change, and push it:
235
-
236
- ```bash
237
- git commit -am "Send through Amazon SES"
238
- git push
239
- ```
240
-
241
- 1. Deploy:
242
-
243
- ```bash
244
- npx wrangler deploy --env production
245
- ```
246
-
247
- ## 9. Subscribe the app to the topic
248
-
249
- SNS delivers events only to a subscriber that has confirmed. The app confirms on its own, once it checks that the request came from your topic.
250
-
251
- 1. In the SNS console, open your topic, and choose **Create subscription**.
252
-
253
- 1. Set **Protocol** to **HTTPS**, and enter the endpoint:
254
-
255
- ```
256
- https://newsletter.example.com/webhooks/ses
257
- ```
258
-
259
- 1. Leave raw message delivery off, and choose **Create subscription**.
260
-
261
- 1. Reload the subscription. Its status changes to **Confirmed**, with nothing for you to click.
262
-
263
- 1. Give SNS longer to retry. By default it retries a failed delivery 3 times over about a minute, then discards the event, so a bounce that arrives while the app is briefly down or mid-deploy is lost, and the address gets your next post. Open the topic, choose **Edit**, and expand **Delivery policy (HTTP/S)**. Clear **Use the default delivery policy**, set these, and choose **Save changes**:
264
-
265
- | Field | Value |
266
- | --- | --- |
267
- | Number of retries | `13` |
268
- | Retries without delay | `0` |
269
- | Minimum delay | `20` |
270
- | Minimum delay retries | `2` |
271
- | Maximum delay | `300` |
272
- | Maximum delay retries | `6` |
273
- | Retry-backoff function | Exponential |
274
-
275
- SNS then keeps trying for most of an hour, the longest it allows. It never retries an event the app refused. The app refuses only an event it can't verify came from your topic, and logs an error when one claims to be from SNS.
276
-
277
- 1. **(Optional)** Keep what SNS still can't deliver. On the subscription, choose **Edit**, and set a **Redrive policy (dead-letter queue)** to an SQS queue of your own. SNS then moves an event there instead of discarding it, where you can read it later. [Amazon SNS dead-letter queues](https://docs.aws.amazon.com/sns/latest/dg/sns-dead-letter-queues.html) covers the queue and its permissions.
278
-
279
- If it stays **Pending confirmation**, the request never reached the app, or the app refused it. Check, in this order:
280
-
281
- - **Access covers `/webhooks/ses`.** If your Access application covers more than `/dashboard` and `/api`, SNS gets the login page instead of the app. Remove that path from your Access application, so only `dashboard` and `api` remain.
282
- - **`SNS_TOPIC_ARN` matches the topic's ARN.** If it doesn't, the app's logs show a `webhook.refused` error naming both.
283
- - **The deploy in section 8 finished.**
284
-
285
- Once you've fixed the cause, select the subscription, and choose **Request confirmation**. SNS sends the request again, and the app confirms it. SNS doesn't let you delete a subscription while it's pending, and removes it on its own after 48 hours.
286
-
287
- ## Switch an instance that already sends
288
-
289
- Change `PROVIDER`, or move to another account with the same provider, only while no send is sending. The dashboard lists any send in progress.
290
-
291
- When a request to a provider gets no answer, Kestrel can't know whether its recipients were mailed. Moving to SES, from another provider or another SES account, mails none of them twice. Kestrel never repeats an unanswered request to SES, so those recipients wait for you to decide whether they were mailed. Use **Resolve** on the send's page.
292
-
293
- Kestrel can't tell whether those emails went out. SES's console shows sent messages only if you've turned on Virtual Deliverability Manager, which this guide doesn't set up. So **Assume not sent** is the safe default. Those readers may miss this post, but they get your next one. No one is mailed twice. Choose **Assume sent** only when you know otherwise, such as a reader who says they got it.
294
-
295
- ## Check it
296
-
297
- 1. The app answers on your hostname:
298
-
299
- ```bash
300
- curl https://newsletter.example.com/health
301
- ```
302
-
303
- ```
304
- {"status":"ok","service":"kestrel"}
305
- ```
306
-
307
- 1. The webhook is public, and refuses a request that isn't from SNS:
308
-
309
- ```bash
310
- curl -s -o /dev/null -w "%{http_code}\n" -X POST https://newsletter.example.com/webhooks/ses
311
- ```
312
-
313
- ```
314
- 400
315
- ```
316
-
317
- Any other answer, such as a redirect to your Cloudflare login, means Access covers the webhook. Remove that path from your Access application.
318
-
319
- 1. The SNS subscription shows **Confirmed**.
320
-
321
- 1. Your From address has somewhere to go. In Cloudflare, your domain's DNS records show the ones Email Routing added for `send`, beside section 1's DKIM records:
322
-
323
- - **Type:** `MX`, **Name:** `send`, three of them, for `route1.mx.cloudflare.net`, `route2.mx.cloudflare.net`, and `route3.mx.cloudflare.net`
324
- - **Type:** `TXT`, **Name:** `send`, **Content:** `v=spf1 include:_spf.mx.cloudflare.net ~all`
325
- - **Type:** `TXT`, **Name:** `cf2024-1._domainkey.send`, Cloudflare's DKIM key
326
-
327
- The world sees the MX records too:
328
-
329
- ```bash
330
- dig +short MX send.example.com
331
- ```
332
-
333
- It lists the three `route` servers. Email Routing's **Routing Rules** shows your rule for `newsletter@send.example.com`, and **Destination Addresses** shows the publisher's inbox as verified. A reply to your test email in the next step proves the forwarding works.
334
-
335
- 1. In the editor, **Settings → Email sender** shows Amazon SES and your From address, where it said **No email provider connected** before.
336
-
337
- 1. In a browser, `https://newsletter.example.com/subscribe` shows the subscribe form, where it said signups aren't open yet.
338
-
339
- The test sends in the next step prove the rest. In the sandbox, send them only to the addresses you verified in section 5, and to the SES test addresses that page lists.
340
-
341
- SES charges for each email it sends, test emails included. [Amazon SES pricing](https://aws.amazon.com/ses/pricing/) lists the rates.
342
-
343
- Next, [verify it works](05-verify.md).