@marktiderman/genesis-forms 0.7.0 → 0.8.0

package/README.md

@@ -83,7 +83,7 @@ const version = eventFeedback.hash();         // content fingerprint for publish
 | `isFormEvent(value)` | Guard for one well-formed event-log entry. |
 | `isSensitive(block)` | `sensitive: true` on a field, or the `props.sensitive` alias. Hidden inputs take `sensitive: true` too (`f.hidden.string({ source: "server", sensitive: true })`); devtools masks both. |
 | `/server` → `validateSubmission`, `publicDefinition` | Re-check a submission against the published definition: drops unknown and logic-hidden answers, ignores client values for server inputs, checks drafts by format only. Never throws on a malformed body. No IO. |
-| `/destinations` → `emailDestination` + `resendProvider`, `sheetsDestination`, `webhookDestination`, `responseRow` | Send a checked submission to an inbox, a Google Sheet or a signed webhook. fetch and WebCrypto only. See [Destinations](#destinations). |
+| `/destinations` → `emailDestination` + `resendProvider`, `sheetsDestination`, `webhookDestination`, `slackDestination`, `responseRow`, `createDeliveryRunner` | Send a checked submission to an inbox, a Google Sheet, Slack or a signed webhook, and run an app's outbox of deliveries. fetch and WebCrypto only. See [Destinations](#destinations). |
 | `explain(engine, blockId)` | "Why is this hidden?": the path, the jump that skipped a block, each show/hide rule with its result in plain English, and the field's validation. |
 | `/testing` → `simulate`, `coverage`, `runScenario`, `recordScenario`, `compactEvents` | Walk a form by key, enumerate its paths, and run JSON scenarios. See [Testing](#testing). |
 
@@ -542,7 +542,10 @@ store and log: never a secret, an answer, contact details or the webhook URL. Co
 has a timeout (`timeoutMs`, default 10 s) and honours `options.signal`. `deliverBatch(items,
 config)` (the sheet) sends many in one request; all succeed or fail together. Write your own by
 implementing `Destination` and throwing `DeliveryError` inside; `deliveryFailure(error)` turns any
-error into a safe result.
+error into a safe result. A destination that deliberately sends nothing (no address, a cap reached)
+resolves `{ ok: true, notSent: reason }`. `options.update` marks the delivery of an edit, and
+`options.destinationId` is handed to option functions (a secret, an account, a URL, a token) as
+their second argument, `{ destinationId }`, so they can look up what is stored for it.
 
 **Rows.** `responseRow(submission, definition)` flattens a submission: `id`, `submitted_at`, the
 app's `meta` columns, then one column per field key in form order. Numbers stay numbers, choices
@@ -553,8 +556,94 @@ submission. On an anonymous form identifying fields get no column. `sensitive` f
 | Destination | Config | Notes |
 | --- | --- | --- |
 | `emailDestination({ provider, from, replyTo?, render?, sensitive? })` | `{ to: string \| string[], subject? }` | HTML and plain text from the definition (`renderEmail`), every value HTML-escaped. Subject, From, To and Reply-To are stripped of line breaks or refused, so no answer can add a header. At most 50 recipients. Sensitive fields masked by default. The idempotency key is the delivery id plus a hash of the recipients. `resendProvider({ apiKey })` is built in; any `EmailProvider` works. `render` and `emailBody(lines)` let an app write its own content and keep the escaping. |
-| `sheetsDestination({ account, escapeFormulas?, sensitive? })` | `{ spreadsheetId, sheetName? }` | Service-account JWT signed with WebCrypto (RS256), token cached per process. Creates the tab, writes and freezes the header, adds a column for a new key without moving others, appends with `valueInputOption=RAW`, so Sheets stores text exactly as typed and never runs it as a formula (`@team thanks` stays `@team thanks`). No `'` is added by default, since RAW would store it literally; `escapeFormulas: true` adds one (headers included) for a sheet that is exported and opened elsewhere. `escapeFormula(cell)` is the same opt-in guard for CSV you write yourself: text starting with `=`, `@`, tab or CR, or with `+`/`-` followed by anything but number punctuation, gets a leading `'`. Keeps the tab, header and last row in `state`, so a normal delivery is one request. On `redelivery`, skips ids already in the last 500 rows. A 403/404 says to share the sheet with the service account. |
-| `webhookDestination({ secret, allowInsecureUrls?, allowPrivateHosts?, sensitive? })` | `{ url }` | [Standard Webhooks](https://www.standardwebhooks.com): `webhook-id`, `webhook-timestamp`, `webhook-signature: v1,<HMAC-SHA256>`. The secret is base64 (at least 16 bytes decoded), usually `whsec_`-prefixed; as in every Standard Webhooks library the prefix is optional and the rest is always base64-decoded, so receivers verify with the same string. A secret that is not padded standard base64 (a raw passphrase, say) fails permanently rather than signing deliveries no receiver could verify. Generate one with `"whsec_" + btoa(String.fromCharCode(...crypto.getRandomValues(new Uint8Array(32))))`. Body `{ type: "form.submitted", timestamp, data: { form, submission, row } }`. https only, no URL credentials, no localhost or private, loopback or link-local address literals, redirects not followed. A hostname that resolves to a private address is not caught: if untrusted users pick URLs, send from a network that cannot reach internal services. Receivers check with `verifyWebhook({ secret, headers, body })` (constant-time, five-minute tolerance). |
+| `sheetsDestination({ account \| accessToken, escapeFormulas?, sensitive? })` | `{ spreadsheetId, sheetName? }` | Signs in with a service account (`account`) or with an access token the app supplies (`accessToken(config, { destinationId })`, for a sheet a person connected with Google sign-in: see `googleOAuthAccessToken`). Service-account JWT signed with WebCrypto (RS256), token cached per process. Creates the tab, writes and freezes the header, adds a column for a new key without moving others, appends with `valueInputOption=RAW`, so Sheets stores text exactly as typed and never runs it as a formula (`@team thanks` stays `@team thanks`). No `'` is added by default, since RAW would store it literally; `escapeFormulas: true` adds one (headers included) for a sheet that is exported and opened elsewhere. `escapeFormula(cell)` is the same opt-in guard for CSV you write yourself: text starting with `=`, `@`, tab or CR, or with `+`/`-` followed by anything but number punctuation, gets a leading `'`. Keeps the tab, header and last row in `state`, so a normal delivery is one request. On `redelivery`, skips ids already in the last 500 rows. On `update` (an edit), reads the id column and overwrites the response's row where it is, appending only when the row is gone. A 403/404 says to share the sheet with the service account (or, with `accessToken`, that it is no longer connected). |
+| `webhookDestination({ secret, url?, resolveHost?, allowInsecureUrls?, allowPrivateHosts?, sensitive? })` | `{ url }` | [Standard Webhooks](https://www.standardwebhooks.com): `webhook-id`, `webhook-timestamp`, `webhook-signature: v1,<HMAC-SHA256>`. The secret is base64 (at least 16 bytes decoded), usually `whsec_`-prefixed; as in every Standard Webhooks library the prefix is optional and the rest is always base64-decoded, so receivers verify with the same string. A secret that is not padded standard base64 (a raw passphrase, say) fails permanently rather than signing deliveries no receiver could verify. Generate one with `newWebhookSecret()`. Body `{ type: "form.submitted", timestamp, data: { form, submission, row } }` (`form.updated` on `update`). `url(config, { destinationId })` looks the address up when it is kept out of config. https only, no URL credentials, no localhost or private, loopback or link-local address literals, redirects not followed. Pass `resolveHost` to also refuse a hostname that resolves to a non-public address (checked before every request; see below). Without it, such a hostname is not caught; either way, if untrusted users pick URLs, send from a network that cannot reach internal services. Receivers check with `verifyWebhook({ secret, headers, body })` (constant-time, five-minute tolerance). |
+| `slackDestination({ url?, resolveHost?, message? })` | `{ url }` | One message per submission to a Slack incoming webhook (`https://hooks.slack.com/services/…` only; anything else fails permanently). The heading (`New response: <title>`, `Updated response` on `update`), each answer as `label: value` in form order, then a link back or the response id. Every value is escaped (`escapeSlack`), so typed text can't mention `@channel` or make a link; answers over 1,500 characters are cut. Sensitive fields masked by default. `message(input)` sets the heading, extra lines, a link and fields to `omit` per delivery; `slackMessage(submission, definition, options)` builds the same text on its own. The URL is the credential, so `url(config, { destinationId })` can look it up. |
+
+### Sending by answer
+
+A destination's `config.when` is a form `Condition` (the same shape logic rules use). The runner
+sends a submission only when it matches, reading answers by field id and hidden inputs by name
+(trusted `context` wins over a hidden input of the same name). A submission that doesn't match is
+recorded as `filtered`. A `when` that is not a valid condition fails the delivery permanently
+rather than guessing. `passesCondition(submission, definition, config)`, `conditionOf(config)` and
+`isCondition(value)` are exported for apps that check it themselves.
+
+### Hook URLs and SSRF
+
+`parseHookUrl(raw, "webhook" | "slack")` checks an address when someone adds it: https on port 443,
+no credentials, at most 2,048 characters, and a public host (no localhost, single-label or
+`.local`/`.internal`/`.lan`-style names, no private or reserved IP literal); a Slack address must
+be a Slack incoming webhook. It throws a `HookUrlError` with a message safe to show. Before every
+request, `checkHookHost(url, resolveHost)` (what `resolveHost` turns on in the webhook and Slack
+destinations) resolves the name and refuses it unless every address is public by
+`isPublicAddress`, which knows the private, loopback, link-local, CGNAT, documentation,
+benchmarking, multicast, IPv4-mapped, NAT64, 6to4 and Teredo ranges. A lookup that fails is
+retryable; a private address is not. In Node, `resolveHost` can be
+`(host) => dns.promises.lookup(host, { all: true }).then((a) => a.map((x) => x.address))`. The
+name is checked and then fetched, so a name re-pointed in between (DNS rebinding) is not caught:
+where untrusted people choose URLs, also send from a network that cannot reach internal services.
+
+### Google sign-in for sheets
+
+`googleOAuthAccessToken({ clientId, clientSecret }, refreshToken)` trades a refresh token from the
+app's own consent flow (`DRIVE_FILE_SCOPE` reaches only files the person picked or created through
+the app) for an access token, cached per grant until a minute before it expires.
+`refreshGoogleAccessToken` is the uncached call. `invalid_grant` fails permanently with "Connect
+the sheet again"; 429 and 5xx are retryable. The app keeps the refresh token; this only trades it.
+
+```ts
+sheetsDestination({
+  accessToken: async (config, { destinationId }) =>
+    googleOAuthAccessToken(oauthClient, await refreshTokenFor(destinationId)),
+});
+```
+
+### Secrets at rest
+
+Webhook and Slack URLs, signing secrets and refresh tokens should not sit in config. The helpers
+seal them for whatever store the app uses: `deriveSecretKey(appKey, purpose)` derives an AES-256-GCM
+key from a 32-byte app secret (base64 or base64url, as `openssl rand -base64 32` prints) with
+HKDF-SHA-256, one key per purpose; `sealSecret(key, plaintext, aad)` gives `v1.<iv>.<ciphertext>`
+(base64url), bound to `aad` (the row it belongs to), and `openSecret(key, sealed, aad)` opens it,
+failing permanently for another row, another purpose or a rotated key.
+
+### The outbox runner
+
+`createDeliveryRunner` claims a batch of due deliveries from the app's store, sends each to its
+destination, and records the outcome. The store maps the three calls to its own tables:
+
+```ts
+import { createDeliveryRunner } from "@marktiderman/genesis-forms/destinations";
+
+const runner = createDeliveryRunner<{ responseId?: string }>({
+  // Lease up to `limit` due rows (a claim function in SQL, say) with their submission and definition.
+  claim: ({ limit, filter }) => store.claimDeliveries(limit, filter),
+  // Write the outcome; resolve false if the lease had run out.
+  record: (delivery, outcome) => store.finishDelivery(delivery.id, outcome),
+  saveState: (destinationId, state) => store.saveDestinationState(destinationId, state),
+  destinations: { email, sheet, slack, webhook },
+  log: (line) => console.warn(line), // ids, kinds and safe reasons only
+});
+
+await runner.run({ filter: { responseId } }); // right after a save, or from a job
+```
+
+Each `ClaimedDelivery` carries its `id` (stable across retries), `destinationId`, `kind`, `config`,
+`state`, `submission`, `definition`, its `attempt` (from 1), how many times it was `deferrals`
+(deferred), and `redelivery` and `update` flags. A response sent and edited in one sheet batch is
+written once, as its latest version.
+Deliveries to one destination go in one `deliverBatch` call where the destination has one (a sheet
+appends them in one request), else one by one with the state passed along; new state is saved
+once. Outcomes: `sent` (`alreadyThere` on a redelivery that found its row), `filtered` (didn't
+match `config.when`), `skipped` (the destination returned `notSent`, a deliberate no-send such as a
+cap; never retried), `retry` with `retryAt`, and `failed`. A retryable failure with the service's
+Retry-After is deferred without counting an attempt (`counted: false`), up to `maxDeferrals`
+(default 24, counted by the store in `ClaimedDelivery.deferrals`); any other retryable
+failure backs off by attempt (`defaultBackoff`: 1 minute doubling to 6 hours) until `maxAttempts`
+(default 8), then fails. Permanent failures and unknown kinds fail at once. A destination that
+throws is reduced to a safe message. `run` returns a `DeliveryReport` of counts, including `stale`
+(record resolved false) and `unrecorded` (record threw; the lease runs out and the row comes back).
 
 ## Testing