@softure-ai/mailing 0.1.8 → 0.1.10

package/CHANGELOG.md

@@ -4,6 +4,38 @@ Newest first. Each version lists what changed for an app that uses `@softure-ai/
 production, the version gets a line `verified in: <app>@<commit>` ([docs/05](../../docs/05-adoption-playbook.md),
 "Definition of done"). Versions before the first one below are described in their GitHub Releases (`mailing@x.y.z`).
 
+## 0.1.10
+
+- `OutgoingMail.replyTo`: an optional reply-to address for one mail (exactly one address, checked like `to`, failure
+  field `replyTo`), sent instead of `mailing({ replyTo })`; without it the configured one is used. `ValidMail` carries
+  it as `replyTo: string | null`. The `Reply-To` header stays reserved.
+
+## 0.1.9
+
+- New `importDeliveries(ctx, rows)` (`/server`, and `importDeliveries(rows)` in `/next`) and `softure-mail import
+  <file|->` (JSON Lines): seed `mailing.deliveries` with the deliveries an app made before it adopted the module, so
+  the first `deliverOnce` or campaign run skips them. Rows `{ scope, address, status: "sent" | "rejected",
+  finishedAt, providerMessageId?, reason?, kind? }`; every row is checked before anything is written, rows already
+  in the ledger are left as they are, a re-run imports nothing twice. `checkImportedDeliveries` checks without
+  writing (`--dry-run`).
+- Migration `0004`: `mailing.deliveries.imported_at`; an imported `sent` row may have no `provider_message_id`.
+  Run `softure migrate`.
+- `maxAttempts` is a module option (default 5) and may be `null`, in the module or per `deliverOnce` call: then
+  `mailing.unavailable` always releases the claim and never closes the delivery.
+- `listCampaignRecipients(campaign, ctx)` module option: `softure-mail campaign` without `--recipients` takes the
+  recipients from it. New `listConfiguredCampaignRecipients` in `/server`.
+- `softure-mail` reads a file given as `-` from standard input (the campaign's content file or `--recipients`, the
+  import's history file), so it runs inside the app's container with `docker compose exec -T`. The README documents
+  that topology.
+- `legacyUnsubscribe.params` also takes `{ required, optional }`: a link is legacy when it carries every required
+  name, and the optional names it carries go to `verify` too, so two old link forms on one path both keep working.
+  The array form still means "all required".
+- `mailing({ oneClickInvalidLinkStatus: 200 })`: the one-click route answers 200 for a link that does not verify, so
+  the answer never tells whether a token is live. The default stays 400; a failure still answers 500.
+- `onUnsubscribed` receives `event.link`: `{ scheme: "signed" }` or `{ scheme: "legacy", values }` with the values
+  `verify` accepted, so the hook can find the row a legacy link named. New types `LegacyUnsubscribeParams` and
+  `VerifiedUnsubscribeLink`.
+
 ## 0.1.8
 
 - `checkSenderDns` takes `expectDmarc: { policy, subdomainPolicy, adkim, aspf }`: a DMARC record weaker than it (a

package/README.md

@@ -1,12 +1,10 @@
 # @softure-ai/mailing
 
 Sends one mail to one recipient through a provider adapter and answers with a typed result, never a
-throw. Ported from FIRE_TRACKER `src/lib/mail.ts` (a hand-written Resend `fetch`, plain text only),
-with the sender, reply-to, provider and timeout taken from configuration, an HTML body, and a fake
-provider for tests. List mail gets signed one-click unsubscribe (RFC 8058) and a suppression list,
-ported from FIRE_TRACKER `src/lib/unsubscribe-*.ts` and `/wypisz`. A delivery ledger sends a mail
-at most once per scope and recipient, and `softure-mail` sends campaigns from a content file where
-FIRE ran deployment-specific scripts.
+throw, with the sender, reply-to, provider and timeout taken from configuration, an HTML body, and a
+fake provider for tests. List mail gets signed one-click unsubscribe (RFC 8058) and a suppression
+list. A delivery ledger sends a mail at most once per scope and recipient, an app's earlier history
+can be imported into it, and `softure-mail` sends campaigns from a content file.
 
 ## 1. What it provides
 
@@ -42,7 +40,8 @@ FIRE ran deployment-specific scripts.
   the withdrawal, and **`legacyUnsubscribe`**, which keeps unsubscribe links the app sent before it
   adopted the module working (section 10).
 - **Sending once.** `deliverOnce(ctx, { scope, mail })` (`/server`, and `deliverOnce({ scope, mail })`
-  in `/next`) sends a mail at most once per scope and recipient through the delivery ledger.
+  in `/next`) sends a mail at most once per scope and recipient through the delivery ledger;
+  `importDeliveries` (and `softure-mail import`) seeds it with the app's earlier history.
 - **Campaigns.** `sendCampaign`, `planCampaign` and the content file parser in `/server`; the
   `softure-mail campaign` command sends a campaign from a content file.
 - **Sender DNS.** `checkSenderDns(domain)` and `softure-mail dns` report SPF, DKIM and DMARC, hold DMARC to a
@@ -70,6 +69,10 @@ the database; list mail without `db` throws, as a wiring bug).
 Rules that hold for every send:
 
 - **One recipient.** `to` is exactly one address; a list is a loop. No cc or bcc.
+- **Reply-To per mail.** `replyTo` (one address, checked like `to`) sends replies to that address
+  instead of `mailing({ replyTo })`, e.g. the buyer of a request mailed to the operator:
+  `sendMail({ to: operator, replyTo: request.account.email, subject, text })`. Without it the
+  configured one is used; the `Reply-To` header stays refused.
 - **Text always, HTML optionally.** `text` is required; `html` is a string the app rendered (from a
   template string or React email components rendered to a string: the module does not render).
 - **Headers cannot take over the mail.** `headers` is for headers like `List-Unsubscribe`;
@@ -90,8 +93,9 @@ address, and `t`, the base64url HMAC-SHA256 of a fixed prefix plus `r` under
 `MAILING_UNSUBSCRIBE_SECRET`. The address never appears in a URL, and the suppression table stores
 only `r`. The page (footer link) shows a button and changes nothing on open: mail scanners open
 links. The one-click route takes a mail client's POST without a session, verifies before touching
-the database and answers 200 (recorded, also again), 400 (link does not verify) or 500 (database
-failure, so the client retries); a GET redirects to the page. The whole link is a credential: it is
+the database and answers 200 (recorded, also again), 400 (link does not verify; 200 with
+`oneClickInvalidLinkStatus: 200`) or 500 (database failure, so the client retries); a GET redirects
+to the page. The whole link is a credential: it is
 never logged.
 
 ### Sending once: the delivery ledger
@@ -124,7 +128,10 @@ const outcome = await deliverOnce({
   Lowercase letters, digits and `._:-`, at most 128 characters. Campaigns use `campaign:<id>`.
 - **Idempotency key.** The ledger sends with `<scope>:<recipient key>`; callers do not pass one.
 - **Retries.** `unavailable` releases the claim (`pending`); the fifth attempt (`maxAttempts`) that
-  is still unavailable closes the row as `rejected`. A halt (`provider_refused`, `quota_exceeded`)
+  is still unavailable closes the row as `rejected`. Five rides out a short provider outage across
+  a few runs, while a provider that keeps failing on one mail does not keep it open forever. An app
+  that retries on its own schedule sets `mailing({ maxAttempts: null })` (or passes it per call):
+  `unavailable` then always releases the claim and never closes the row. A halt (`provider_refused`, `quota_exceeded`)
   never closes a row and never uses up an attempt, so a run with a bad key loses no recipient. The
   provider's HTTP status of the last failed answer is kept in `provider_status`.
 - **Interrupted sends.** A process that dies between the send and the outcome leaves a claim; after
@@ -138,6 +145,47 @@ const outcome = await deliverOnce({
   claim cannot overwrite the one that took over.
 - A malformed scope or kind throws (a bug); a database failure propagates.
 
+### Importing an existing history
+
+An app that already kept its own once-only records (which campaign went to which signup, which
+notice went to which user for which period) seeds the ledger before its first `deliverOnce` or
+campaign run; otherwise that run mails everyone again. Map each record to the scope the module
+will use for it and import:
+
+```ts
+import { importDeliveries } from "@softure-ai/mailing/server";
+
+const result = await importDeliveries(ctx, [
+  { scope: "campaign:2026-09-launch", address: "ada@example.org", status: "sent", finishedAt: "2026-09-20T10:15:00Z", kind: "newsletter" },
+  { scope: "account.trial-ending:user_7", address: "bob@example.org", status: "rejected", finishedAt: row.sentAt, reason: "mailing.rejected" },
+]);
+if (!result.ok) console.error(result.problems); // [{ index, problem }]: nothing was written
+else console.log(result.value); // { rows, imported, alreadyPresent, duplicates }
+```
+
+Or from a file, one JSON object per line with the same fields:
+
+```bash
+softure-mail import history.jsonl --dry-run   # checks the file, writes nothing
+softure-mail import history.jsonl
+docker compose exec -T app npx softure-mail import - < history.jsonl   # inside the app's container
+```
+
+- Rows: `scope`, `address` (only its key is stored), `status` (`sent` or `rejected`), `finishedAt`
+  (a date or ISO string, not in the future), optional `providerMessageId` (sent rows; history
+  rarely kept it, and an imported sent row may go without), `reason` (rejected rows: one of
+  `mailing.invalid_input`, `rejected`, `unavailable`, `suppressed`; default `mailing.rejected`) and
+  `kind` (default `transactional`).
+- **Every row is checked before anything is written**; problems name the row (the file's line) and
+  the field, never the address.
+- **Idempotent.** A row whose scope and recipient the ledger already has (imported earlier, or sent
+  by the module) is left as it is and counted as `alreadyPresent`; re-run the same import after a
+  failure. Two input rows for the same scope and recipient count once (`duplicates`).
+- Imported rows are closed (`sent` or `rejected`), with `imported_at` set and no `campaign_id`:
+  `deliverOnce`, `planCampaign` and `sendCampaign` match a delivery by scope, so they skip it.
+- Unsubscribes the app recorded go to `suppressRecipient`, not here: the suppression list is what
+  stops future campaigns.
+
 ### Campaigns: `softure-mail campaign`
 
 A campaign is one list mail to many recipients through the ledger. The content file is frontmatter
@@ -161,6 +209,25 @@ softure-mail campaign launch.md --recipients recipients.txt              # sends
 ```
 
 - The recipients file holds one address per line (`#` comments and blank lines skipped).
+- **Recipients from the database.** Without `--recipients` the command asks
+  `mailing({ listCampaignRecipients })`, which lists the addresses from the app's own data
+  (duplicates are fine; the filter and the suppression list still apply):
+
+  ```ts
+  listCampaignRecipients: async ({ kind }, ctx) => listOptedInAddresses(ctx.db, kind),
+  ```
+
+- **Where it runs.** The command needs the database (the ledger) and the provider key, so it runs
+  where both are: inside the app's container. When the database is not reachable from the
+  operator's machine, send the content file on standard input (`-`) and let the recipients come
+  from `listCampaignRecipients` (or pass `--recipients -` with the content as a file in the image):
+
+  ```bash
+  ssh app-host 'cd /srv/app && docker compose exec -T app npx softure-mail campaign - --dry-run' < launch.md
+  ssh app-host 'cd /srv/app && docker compose exec -T app npx softure-mail campaign -' < launch.md
+  ```
+
+  With the content on standard input, its `html:` file is looked up in the working directory.
 - The command loads `softure.config.*` like `softure migrate` (or `--config <file>`), opens the
   config's `database.handle` when set, otherwise its own connection on `database.url`, and needs `MAILING_UNSUBSCRIBE_SECRET` (campaigns are list
   mail). Run `softure migrate` first.
@@ -250,7 +317,10 @@ mailing({
 | `timeoutMs` | `number` | `10000` | 1000 to 60000. |
 | `onUnsubscribed` | `(event, ctx) => Promise<void>` | — | Runs on every verified unsubscribe, in its transaction (section 10). |
 | `legacyUnsubscribe` | `{ params, verify }` | — | Verifies unsubscribe links the app sent before it adopted the module (section 10). |
+| `oneClickInvalidLinkStatus` | `200 \| 400` | `400` | What the one-click route answers for a link that does not verify. `200` gives no oracle on whether a token is live; a failure still answers 500, and the page still shows its message. |
 | `filterCampaignRecipient` | `(recipient, ctx) => Promise<boolean>` | — | Decides per recipient whether a campaign goes to them (section 1, Campaigns). |
+| `listCampaignRecipients` | `(campaign, ctx) => Iterable<string> \| AsyncIterable<string> \| Promise<Iterable<string>>` | — | Lists a campaign's recipients from the app's data; `softure-mail campaign` uses it without `--recipients` (section 1, Campaigns). |
+| `maxAttempts` | `number \| null` | `5` | Claims before `mailing.unavailable` closes a delivery as rejected, 1 to 100; `null` never closes it (section 1, Retries). |
 | `staleClaimMs` | `number` | `900000` (15 min) | How long a delivery claim may stay open before another sender takes it over. 1 minute to 23 hours. |
 | `uncertainClaimMs` | `number` | `82800000` (23 h) | How old a claim may get before it waits for an operator (`uncertain`). More than `staleClaimMs`, at most 30 days; keep it under the provider's idempotency window. |
 | `routes` | `{ unsubscribe?, oneClick? }` | `/unsubscribe`, `/api/mailing/unsubscribe` | Where you mount the page and the route; links are built on `appOrigin` plus these paths. |
@@ -310,8 +380,9 @@ Schema `mailing`, applied by `softure migrate`:
 | --- | --- | --- |
 | `0001_create_suppressions.sql` | `mailing.suppressions` | `recipient_key` (primary key, 43-character base64url), `source` (`one-click`, `page`, `operator`), `created_at`. One row per address, the first opt-out kept. |
 | `0002_create_campaigns_and_deliveries.sql` | `mailing.campaigns` | `id` (kebab-case), `kind` (never `transactional`), `subject`, `content_hash` (sha256 of kind, subject and bodies), `created_at`. |
-| | `mailing.deliveries` | primary key (`scope`, `recipient_key`), `kind`, `campaign_id` (then `scope` is `campaign:<id>`), `status` (`pending`, `claimed`, `sent`, `rejected`), `attempts`, `claimed_at`, `finished_at`, `provider_message_id` (exactly when sent), `reason` (exactly when rejected). |
+| | `mailing.deliveries` | primary key (`scope`, `recipient_key`), `kind`, `campaign_id` (then `scope` is `campaign:<id>`), `status` (`pending`, `claimed`, `sent`, `rejected`), `attempts`, `claimed_at`, `finished_at`, `provider_message_id` (exactly when sent; see `0004`), `reason` (exactly when rejected). |
 | `0003_add_delivery_provider_status.sql` | `mailing.deliveries` | `provider_status` (the HTTP status of the last failed or released answer, 100 to 599, else null); `attempts` may be 0 on a `pending` row (a halt gave its attempt back). |
+| `0004_allow_imported_deliveries.sql` | `mailing.deliveries` | `imported_at` (set by `importDeliveries`, only on `sent`/`rejected` rows); `provider_message_id` only on `sent` rows, and required there unless the row was imported. |
 
 The health check (`checkMailingTables`) runs `select 1 from mailing.<table> limit 0` for the three
 tables.
@@ -351,8 +422,9 @@ The provider is the extension point (section 3).
 
 **`onUnsubscribed(event, ctx)`** runs on every verified unsubscribe (the page's button or a mail
 client's one-click POST), also a repeated one, in the transaction that stores the opt-out:
-`event` is `{ recipientKey, source }` (the key is privacy's email key of the same address, never the
-address), `ctx.db` is the transaction. A throw rolls the opt-out back: the route answers 500 and the
+`event` is `{ recipientKey, source, link }` (the key is privacy's email key of the same address, never the
+address; `link` is `{ scheme: "signed" }` or `{ scheme: "legacy", values }` with the values `verify`
+accepted, so a hook can find the row a legacy link named), `ctx.db` is the transaction. A throw rolls the opt-out back: the route answers 500 and the
 page offers a retry, so the opt-out and what the hook records never disagree. It does not run for
 `suppressRecipient` (a bounce or a script is not the person's choice). Wire the waitlist's handler,
 which withdraws its consents in privacy's ledger:
@@ -365,9 +437,12 @@ mailing({ from: "…", provider: resend(), onUnsubscribed: withdrawWaitlistConse
 
 **`legacyUnsubscribe: { params, verify }`** keeps unsubscribe links working that the app sent
 before it adopted the module, in its own scheme (an HMAC over a sign-up id, say, on its own path).
-`params` are the old link's query names (1 to 8; never `r` or `status`, which the module's links and
-page own; `t` may be shared). A link without `r` that carries every one of them (non-empty, at most
-512 characters) is a legacy link: the page shows the same button with the values in hidden fields,
+`params` are the old link's query names (never `r` or `status`, which the module's links and page
+own; `t` may be shared). An array names parameters the link must all carry; `{ required, optional }`
+also names ones it may carry, for an app that sent more than one form on the same path (at most 8
+names in all, at least one required). A link without `r` that carries every required one
+(non-empty, at most 512 characters) is a legacy link, and its values also hold the optional ones it
+has (an empty one counts as absent, one over 512 characters makes the link invalid): the page shows the same button with the values in hidden fields,
 and the action and the one-click POST call `verify(values, ctx)`. It returns the recipient's
 **address** when the link is genuine, else `null`; the module records the opt-out under that
 address's key and runs `onUnsubscribed`, exactly as for its own links. `verify` must check the
@@ -386,6 +461,25 @@ mailing({
 export { UnsubscribePage as default } from "@softure-ai/mailing/next";
 ```
 
+Two forms on one path (a signed `?u=<id>&t=<hmac>` in mail and a bare `?t=<token>` shown after
+sign-up), an opt-out recorded on the app's own row, and a one-click route that never tells a live
+token from a dead one:
+
+```ts
+mailing({
+  from: "…",
+  provider: resend(),
+  oneClickInvalidLinkStatus: 200,
+  legacyUnsubscribe: {
+    params: { required: ["t"], optional: ["u"] },
+    verify: ({ u, t }, ctx) => (u === undefined ? findAddressByToken(ctx.db, t) : verifyOldLink(ctx.db, u, t)),
+  },
+  onUnsubscribed: async (event, ctx) => {
+    if (event.link.scheme === "legacy") await markSignupUnsubscribed(ctx.db, event.link.values);
+  },
+}),
+```
+
 **`liftSuppression(ctx, address)`** is the other direction: a module that has just recorded a new
 explicit consent (the waitlist's sign-up) calls it in that consent's transaction. It deletes the
 row only when its source is `page` or `one-click`; an `operator` row stays. It returns whether it

package/dist/cli/run.d.ts

@@ -7,7 +7,7 @@ export interface CliOutput {
     readonly error: (line: string) => void;
 }
 export interface RunMailCliOptions {
-    /** The app's config. Needed by `campaign`, and by `dns` without `--domain`. */
+    /** The app's config. Needed by `campaign` and `import`, and by `dns` without `--domain`. */
     readonly config?: SoftureConfig;
     /** The arguments after the executable, e.g. `["campaign", "launch.md", "--recipients", "list.txt"]`. */
     readonly argv: readonly string[];
@@ -21,6 +21,8 @@ export interface RunMailCliOptions {
     readonly resolveCname?: ResolveCname;
     readonly sleep?: (ms: number) => Promise<void>;
     readonly env?: Env;
+    /** Reads all of standard input, for a file given as `-`. Default: `process.stdin`. */
+    readonly readStdin?: () => Promise<string>;
 }
 export declare const EXIT_OK = 0;
 export declare const EXIT_FAILED = 1;

package/dist/cli/run.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run.d.ts","sourceRoot":"","sources":["../../src/cli/run.ts"],"names":[],"mappings":"AAWA,OAAO,EAAe,KAAK,aAAa,EAA8B,MAAM,kBAAkB,CAAC;AAC/F,OAAO,EAA6C,KAAK,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAIhG,OAAO,EAAkI,KAAK,YAAY,EAAE,KAAK,SAAS,EAAE,KAAK,UAAU,EAAuB,MAAM,kBAAkB,CAAC;AAE3O,OAAO,EAAiF,KAAK,GAAG,EAAE,MAAM,+BAA+B,CAAC;AAExI,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,GAAG,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACrC,QAAQ,CAAC,KAAK,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;CACxC;AAED,MAAM,WAAW,iBAAiB;IAChC,+EAA+E;IAC/E,QAAQ,CAAC,MAAM,CAAC,EAAE,aAAa,CAAC;IAChC,wGAAwG;IACxG,QAAQ,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,CAAC;IACjC,mEAAmE;IACnE,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,MAAM,CAAC,EAAE,SAAS,CAAC;IAC5B,iIAAiI;IACjI,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,OAAO,CAAC,cAAc,CAAC,CAAC;IACjE,QAAQ,CAAC,UAAU,CAAC,EAAE,UAAU,CAAC;IACjC,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,CAAC;IAC/B,QAAQ,CAAC,YAAY,CAAC,EAAE,YAAY,CAAC;IACrC,QAAQ,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/C,QAAQ,CAAC,GAAG,CAAC,EAAE,GAAG,CAAC;CACpB;AAED,eAAO,MAAM,OAAO,IAAI,CAAC;AACzB,eAAO,MAAM,WAAW,IAAI,CAAC;AAC7B,eAAO,MAAM,UAAU,IAAI,CAAC;AAE5B,8EAA8E;AAC9E,eAAO,MAAM,gBAAgB,MAAM,CAAC;AAEpC,eAAO,MAAM,UAAU,QAgCa,CAAC;AAgCrC,yGAAyG;AACzG,wBAAsB,UAAU,CAAC,OAAO,EAAE,iBAAiB,GAAG,OAAO,CAAC,MAAM,CAAC,CAiB5E"}
+{"version":3,"file":"run.d.ts","sourceRoot":"","sources":["../../src/cli/run.ts"],"names":[],"mappings":"AAWA,OAAO,EAAe,KAAK,aAAa,EAA8B,MAAM,kBAAkB,CAAC;AAE/F,OAAO,EAA6C,KAAK,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAIhG,OAAO,EAAkI,KAAK,YAAY,EAAE,KAAK,SAAS,EAAE,KAAK,UAAU,EAAuB,MAAM,kBAAkB,CAAC;AAG3O,OAAO,EAAiF,KAAK,GAAG,EAAE,MAAM,+BAA+B,CAAC;AAExI,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,GAAG,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;IACrC,QAAQ,CAAC,KAAK,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,CAAC;CACxC;AAED,MAAM,WAAW,iBAAiB;IAChC,4FAA4F;IAC5F,QAAQ,CAAC,MAAM,CAAC,EAAE,aAAa,CAAC;IAChC,wGAAwG;IACxG,QAAQ,CAAC,IAAI,EAAE,SAAS,MAAM,EAAE,CAAC;IACjC,mEAAmE;IACnE,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,MAAM,CAAC,EAAE,SAAS,CAAC;IAC5B,iIAAiI;IACjI,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,OAAO,CAAC,cAAc,CAAC,CAAC;IACjE,QAAQ,CAAC,UAAU,CAAC,EAAE,UAAU,CAAC;IACjC,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,CAAC;IAC/B,QAAQ,CAAC,YAAY,CAAC,EAAE,YAAY,CAAC;IACrC,QAAQ,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/C,QAAQ,CAAC,GAAG,CAAC,EAAE,GAAG,CAAC;IACnB,sFAAsF;IACtF,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,OAAO,CAAC,MAAM,CAAC,CAAC;CAC5C;AAED,eAAO,MAAM,OAAO,IAAI,CAAC;AACzB,eAAO,MAAM,WAAW,IAAI,CAAC;AAC7B,eAAO,MAAM,UAAU,IAAI,CAAC;AAE5B,8EAA8E;AAC9E,eAAO,MAAM,gBAAgB,MAAM,CAAC;AAEpC,eAAO,MAAM,UAAU,QA6C8C,CAAC;AAqCtE,yGAAyG;AACzG,wBAAsB,UAAU,CAAC,OAAO,EAAE,iBAAiB,GAAG,OAAO,CAAC,MAAM,CAAC,CAmB5E"}

package/dist/cli/run.js

@@ -1,6 +1,6 @@
-// `softure-mail`: sends a campaign from a content file and checks the sender domain's DNS. The app
-// passes its validated config, so the same function serves the bin and an app script that a
-// bundler packs for a container:
+// `softure-mail`: sends a campaign from a content file, imports an app's delivery history into the
+// ledger and checks the sender domain's DNS. The app passes its validated config, so the same
+// function serves the bin and an app script that a bundler packs for a container:
 //
 //   // scripts/mail.ts
 //   import { runMailCli } from "@softure-ai/mailing/cli";
@@ -10,10 +10,12 @@ import { readFile } from "node:fs/promises";
 import { dirname, resolve } from "node:path";
 import { parseArgs } from "node:util";
 import { systemClock } from "@softure-ai/core";
+import { z } from "zod";
 import { openCommandDatabase } from "@softure-ai/db";
 import { parseCampaignFile, parseRecipientList } from "../server/campaign-file.js";
-import { planCampaign, sendCampaign } from "../server/campaigns.js";
+import { listConfiguredCampaignRecipients, planCampaign, sendCampaign } from "../server/campaigns.js";
 import { checkSenderDns, getSenderDomain, resendReturnPath } from "../server/dns.js";
+import { checkImportedDeliveries, importDeliveries } from "../server/import-deliveries.js";
 import { getMailingOptions } from "../server/options.js";
 import { MIN_UNSUBSCRIBE_SECRET_LENGTH, readUnsubscribeSecrets, UNSUBSCRIBE_SECRET_ENV } from "../server/unsubscribe-link.js";
 export const EXIT_OK = 0;
@@ -22,20 +24,30 @@ export const EXIT_USAGE = 2;
 /** Pause between sends: Resend's default limit is two requests per second. */
 export const DEFAULT_PAUSE_MS = 500;
 export const MAIL_USAGE = `Usage:
-  softure-mail campaign <content-file> --recipients <file> [--dry-run] [--pause-ms <ms>] [--resend-uncertain]
+  softure-mail campaign <content-file> [--recipients <file>] [--dry-run] [--pause-ms <ms>] [--resend-uncertain]
+  softure-mail import <history-file> [--dry-run]
   softure-mail dns [--domain <domain>] [--dkim-selector <name>]... [--spf-host <host>]...
                    [--dmarc-policy <p>] [--dmarc-sp <p>] [--dmarc-adkim <a>] [--dmarc-aspf <a>]
                    [--reply-to <address>] [--return-path <host>]... [--resend-return-path]
 
 campaign  Sends the campaign in <content-file> (frontmatter id, kind, subject, optional html,
           then the text body) to every address in the recipients file (one per line), at most
-          once each. Re-running sends only to recipients without an outcome yet. A refused
-          API key or a spent quota stops the run; the rest go out on the next one.
+          once each. Without --recipients, the recipients come from listCampaignRecipients in
+          the mailing options. Re-running sends only to recipients without an outcome yet. A
+          refused API key or a spent quota stops the run; the rest go out on the next one.
+  --recipients <file>  the recipients, one address per line
   --dry-run            count what would be sent and change nothing
   --pause-ms <ms>      pause after every send (default ${String(DEFAULT_PAUSE_MS)})
   --resend-uncertain   also send to recipients whose send was interrupted long ago
                        (it may have gone out: they may get the mail twice)
 
+import    Writes deliveries the app made before it adopted the module into the ledger, so
+          later sends skip them. <history-file> holds one JSON object per line: scope,
+          address, status (sent or rejected), finishedAt, and optional providerMessageId,
+          reason, kind. Rows already in the ledger are left as they are; any invalid row
+          stops the import before anything is written.
+  --dry-run            check the file and change nothing
+
 dns       Checks SPF, DKIM and DMARC for the sender domain (default: the domain of "from").
   --domain <domain>       check this domain instead
   --dkim-selector <name>  DKIM selector to look up (default resend; repeatable)
@@ -51,15 +63,21 @@ dns       Checks SPF, DKIM and DMARC for the sender domain (default: the domain
   --return-path <host>    a return-path host that must be a CNAME or have MX (repeatable)
   --resend-return-path    check Resend's send.<domain> and rsend.<domain> CNAMEs
 
-Options for both:
+Options for all:
   --config <file>    the app's softure.config file (bin only)
-  --help             show this help`;
+  --help             show this help
+
+A file given as - is read from standard input (one of them per command), e.g.
+  docker compose exec -T app npx softure-mail campaign - < launch.md`;
 const consoleOutput = {
     log: (line) => console.log(line),
     error: (line) => console.error(line),
 };
 const DMARC_POLICIES = ["quarantine", "reject"];
 const DMARC_ALIGNMENTS = ["r", "s"];
+/** A file name that means standard input. */
+const STDIN = "-";
+const CAMPAIGN_ONLY_OPTIONS = ["recipients", "pause-ms", "resend-uncertain"];
 const DNS_ONLY_OPTIONS = ["domain", "dkim-selector", "spf-host", "dmarc-policy", "dmarc-sp", "dmarc-adkim", "dmarc-aspf", "reply-to", "return-path", "resend-return-path"];
 /** Runs the command and returns the process exit code: 0 done, 1 failed or incomplete, 2 usage error. */
 export async function runMailCli(options) {
@@ -78,16 +96,18 @@ export async function runMailCli(options) {
             return runDns(command, options, output);
         case "campaign":
             return runCampaign(command, options, output);
+        case "import":
+            return runImport(command, options, output);
     }
 }
 function parseCommand(argv) {
     const [name, ...rest] = argv;
     if (name === undefined)
-        return "missing command; use campaign or dns";
+        return "missing command; use campaign, import or dns";
     if (name === "--help")
         return { kind: "help" };
-    if (name !== "campaign" && name !== "dns")
-        return `unknown command "${name}"; use campaign or dns`;
+    if (name !== "campaign" && name !== "import" && name !== "dns")
+        return `unknown command "${name}"; use campaign, import or dns`;
     let parsed;
     try {
         parsed = parseCommandArgs(rest);
@@ -99,7 +119,7 @@ function parseCommand(argv) {
     if (values.help === true)
         return { kind: "help" };
     if (name === "dns") {
-        const misplaced = ["recipients", "dry-run", "pause-ms", "resend-uncertain"].filter((option) => values[option] !== undefined);
+        const misplaced = [...CAMPAIGN_ONLY_OPTIONS, "dry-run"].filter((option) => values[option] !== undefined);
         if (misplaced.length > 0)
             return `dns does not take --${misplaced.join(", --")}`;
         if (positionals.length > 0)
@@ -118,6 +138,17 @@ function parseCommand(argv) {
             resendReturnPath: values["resend-return-path"] === true,
         };
     }
+    if (name === "import") {
+        const misplaced = [...CAMPAIGN_ONLY_OPTIONS, ...DNS_ONLY_OPTIONS].filter((option) => values[option] !== undefined);
+        if (misplaced.length > 0)
+            return `import does not take --${misplaced.join(", --")}`;
+        const [historyFile, ...extra] = positionals;
+        if (historyFile === undefined)
+            return "import needs a history file (- for standard input)";
+        if (extra.length > 0)
+            return `import takes one history file, got also "${extra.join(" ")}"`;
+        return { kind: "import", historyFile, dryRun: values["dry-run"] === true };
+    }
     const misplaced = DNS_ONLY_OPTIONS.filter((option) => values[option] !== undefined);
     if (misplaced.length > 0)
         return `campaign does not take --${misplaced.join(", --")}`;
@@ -126,12 +157,12 @@ function parseCommand(argv) {
         return "campaign needs a content file";
     if (extra.length > 0)
         return `campaign takes one content file, got also "${extra.join(" ")}"`;
-    if (values.recipients === undefined)
-        return "campaign needs --recipients <file>";
+    if (contentFile === STDIN && values.recipients === STDIN)
+        return "campaign can read only one of the content file and --recipients from standard input";
     const pauseMs = values["pause-ms"] === undefined ? DEFAULT_PAUSE_MS : Number(values["pause-ms"]);
     if (!Number.isInteger(pauseMs) || pauseMs < 0 || pauseMs > 60_000)
         return `--pause-ms expects whole milliseconds from 0 to 60000, got "${values["pause-ms"] ?? ""}"`;
-    return { kind: "campaign", contentFile, recipientsFile: values.recipients, dryRun: values["dry-run"] === true, pauseMs, resendUncertain: values["resend-uncertain"] === true };
+    return { kind: "campaign", contentFile, recipientsFile: values.recipients ?? null, dryRun: values["dry-run"] === true, pauseMs, resendUncertain: values["resend-uncertain"] === true };
 }
 function parseCommandArgs(args) {
     return parseArgs({
@@ -247,9 +278,19 @@ async function runCampaign(command, options, output) {
         output.error("softure-mail campaign: needs the app's config");
         return EXIT_FAILED;
     }
-    const loaded = await loadCampaign(resolve(cwd, command.contentFile), resolve(cwd, command.recipientsFile));
-    if (typeof loaded === "string") {
-        output.error(`softure-mail campaign: ${loaded}`);
+    const input = createInputReader(cwd, options.readStdin ?? readProcessStdin);
+    const content = await loadCampaignContent(command.contentFile, input);
+    if (typeof content === "string") {
+        output.error(`softure-mail campaign: ${content}`);
+        return EXIT_FAILED;
+    }
+    const listed = command.recipientsFile === null ? null : await loadRecipientFile(command.recipientsFile, input);
+    if (typeof listed === "string") {
+        output.error(`softure-mail campaign: ${listed}`);
+        return EXIT_FAILED;
+    }
+    if (listed === null && getMailingOptions(config).listCampaignRecipients === undefined) {
+        output.error("softure-mail campaign: pass --recipients <file>, or set listCampaignRecipients in the mailing options");
         return EXIT_FAILED;
     }
     if (!command.dryRun && readUnsubscribeSecrets(options.env ?? process.env).current === null) {
@@ -270,6 +311,7 @@ async function runCampaign(command, options, output) {
     }
     const ctx = { db: opened.handle.db, clock: systemClock, config };
     try {
+        const loaded = { campaign: content, recipients: listed ?? (await collectRecipients(ctx, content)) };
         return command.dryRun
             ? await reportPlan(ctx, loaded, command.resendUncertain, output)
             : await reportSend(ctx, loaded, { pauseMs: command.pauseMs, sleep: options.sleep, retakeUncertain: command.resendUncertain }, output);
@@ -283,27 +325,151 @@ async function runCampaign(command, options, output) {
         await opened.close();
     }
 }
-async function loadCampaign(contentPath, recipientsPath) {
-    const source = await readText(contentPath);
-    if (source === null)
-        return `cannot read the content file ${contentPath}`;
-    const parsed = parseCampaignFile(source);
+function createInputReader(cwd, readStdin) {
+    return {
+        read: async (file) => {
+            if (file === STDIN)
+                return { text: await readStdin(), name: "standard input", dir: cwd };
+            const path = resolve(cwd, file);
+            return { text: await readText(path), name: path, dir: dirname(path) };
+        },
+    };
+}
+async function loadCampaignContent(file, input) {
+    const source = await input.read(file);
+    if (source.text === null)
+        return `cannot read the content file ${source.name}`;
+    const parsed = parseCampaignFile(source.text);
     if (!parsed.ok)
-        return `${contentPath} is not a campaign:\n  ${parsed.problems.join("\n  ")}`;
+        return `${source.name} is not a campaign:\n  ${parsed.problems.join("\n  ")}`;
     const { htmlPath, ...content } = parsed.value;
     let html = null;
     if (htmlPath !== null) {
-        const htmlFile = resolve(dirname(contentPath), htmlPath);
+        // Next to the content file; for standard input, the working directory.
+        const htmlFile = resolve(source.dir, htmlPath);
         html = await readText(htmlFile);
         if (html === null)
             return `cannot read the HTML body ${htmlFile}`;
         if (html.trim() === "")
             return `the HTML body ${htmlFile} is empty`;
     }
-    const list = await readText(recipientsPath);
-    if (list === null)
-        return `cannot read the recipients file ${recipientsPath}`;
-    return { campaign: { ...content, html }, recipients: parseRecipientList(list) };
+    return { ...content, html };
+}
+async function loadRecipientFile(file, input) {
+    const list = await input.read(file);
+    if (list.text === null)
+        return `cannot read the recipients file ${list.name}`;
+    return parseRecipientList(list.text);
+}
+/** The recipients from `listCampaignRecipients`, which the caller checked is set. */
+async function collectRecipients(ctx, campaign) {
+    const recipients = [];
+    const source = await listConfiguredCampaignRecipients(ctx, campaign);
+    if (source === null)
+        return recipients;
+    for await (const address of source)
+        recipients.push(address);
+    return recipients;
+}
+/** One line of a history file. Unknown keys are refused, so a typo does not drop a field. */
+const historyRowSchema = z.strictObject({
+    scope: z.string(),
+    address: z.string(),
+    status: z.enum(["sent", "rejected"]),
+    finishedAt: z.string(),
+    providerMessageId: z.string().optional(),
+    reason: z.enum(["mailing.invalid_input", "mailing.rejected", "mailing.unavailable", "mailing.suppressed"]).optional(),
+    kind: z.string().optional(),
+});
+async function runImport(command, options, output) {
+    const config = options.config;
+    if (config === undefined) {
+        output.error("softure-mail import: needs the app's config");
+        return EXIT_FAILED;
+    }
+    const source = await createInputReader(options.cwd ?? process.cwd(), options.readStdin ?? readProcessStdin).read(command.historyFile);
+    if (source.text === null) {
+        output.error(`softure-mail import: cannot read the history file ${source.name}`);
+        return EXIT_FAILED;
+    }
+    const parsed = parseHistory(source.text);
+    if (parsed.problems.length > 0)
+        return reportImportProblems(parsed.problems, output);
+    const rows = parsed.rows.map((row) => row.delivery);
+    const lineOf = (problem) => parsed.rows[problem.index]?.line ?? 0;
+    // Checked before the database is opened, so a bad file costs no connection.
+    const checked = checkImportedDeliveries({ clock: systemClock }, rows);
+    if (!checked.ok)
+        return reportImportProblems(checked.problems.map((problem) => ({ line: lineOf(problem), problem: problem.problem })), output);
+    if (command.dryRun) {
+        output.log(`import, dry run: nothing written; rows ${String(rows.length)}, duplicates ${String(checked.value.duplicates)}`);
+        return EXIT_OK;
+    }
+    if (config.database === null) {
+        output.error("softure-mail import: the config has no database; set database.url in softure.config");
+        return EXIT_FAILED;
+    }
+    let opened;
+    try {
+        opened = await openDatabase(config.database, options.openDatabase);
+    }
+    catch (error) {
+        output.error(`softure-mail import: ${describeError(error)}`);
+        return EXIT_FAILED;
+    }
+    try {
+        const result = await importDeliveries({ db: opened.handle.db, clock: systemClock, config }, rows);
+        if (!result.ok)
+            return reportImportProblems(result.problems.map((problem) => ({ line: lineOf(problem), problem: problem.problem })), output);
+        const summary = result.value;
+        output.log(`import: rows ${String(summary.rows)}, imported ${String(summary.imported)}, already present ${String(summary.alreadyPresent)}, duplicates ${String(summary.duplicates)}`);
+        return EXIT_OK;
+    }
+    catch (error) {
+        output.error(`softure-mail import: ${describeError(error)} (did softure migrate run?)`);
+        return EXIT_FAILED;
+    }
+    finally {
+        await opened.close();
+    }
+}
+/** The history file's rows with their line numbers; problems name lines and fields, never values. */
+function parseHistory(text) {
+    const rows = [];
+    const problems = [];
+    text.split(/\r?\n/).forEach((source, index) => {
+        const line = index + 1;
+        if (source.trim() === "")
+            return;
+        let value;
+        try {
+            value = JSON.parse(source);
+        }
+        catch {
+            problems.push({ line, problem: "not a JSON object" });
+            return;
+        }
+        const parsed = historyRowSchema.safeParse(value);
+        if (!parsed.success) {
+            for (const issue of parsed.error.issues) {
+                const field = issue.path.length === 0 ? "row" : issue.path.join(".");
+                problems.push({ line, problem: issue.code === "unrecognized_keys" ? `unknown field(s) ${issue.keys.join(", ")}` : `${field}: ${issue.message}` });
+            }
+            return;
+        }
+        const { providerMessageId, reason, kind, ...required } = parsed.data;
+        rows.push({
+            line,
+            delivery: { ...required, ...(providerMessageId === undefined ? {} : { providerMessageId }), ...(reason === undefined ? {} : { reason }), ...(kind === undefined ? {} : { kind }) },
+        });
+    });
+    return { rows, problems };
+}
+function reportImportProblems(problems, output) {
+    output.error(`softure-mail import: nothing written, ${String(problems.length)} problem(s):`);
+    for (const { line, problem } of problems)
+        output.error(`  line ${String(line)}: ${problem}`);
+    return EXIT_FAILED;
 }
 async function reportPlan(ctx, loaded, retakeUncertain, output) {
     const plan = await planCampaign(ctx, loaded, { retakeUncertain });
@@ -383,6 +549,12 @@ async function openDatabase(database, open) {
     const handle = await open(database.url);
     return { handle, close: handle.close };
 }
+async function readProcessStdin() {
+    const chunks = [];
+    for await (const chunk of process.stdin)
+        chunks.push(typeof chunk === "string" ? Buffer.from(chunk) : chunk);
+    return Buffer.concat(chunks).toString("utf8");
+}
 async function readText(path) {
     try {
         return await readFile(path, "utf8");