pipework 0.212.0 → 0.243.0

package/CHANGELOG.md

@@ -1,5 +1,145 @@
 # Changelog
 
+## 0.243.0
+
+- **Fixed: test setup failed with "No progress" while a long migration file was still applying** (#871). The template build reported progress only after each migration file committed, so a file whose statements together ran past `test.setupStallMs` on a loaded box (verity: 120+ s at `template-migrate`) failed the run. Setup now beats after every statement of a file (`template-migrate` detail `i/n <tag> stmt s/n`), before the first file (`start`, then `prepare <step>` for the tracker table, legacy tag recovery per file, the tag index and the applied read), and on each template-build step (`drop`, `create`, `owner`, `platform role`) and framework table statement. A single statement that hangs still trips the watchdog; its thresholds are unchanged. `applyMigrations` takes `onStatement(tag, statement, statements, index, total)` and `onPrepare(detail)`.
+
+## 0.242.0
+
+- **Fixed: `webhook.deliver` could reach the app's private network** (#866). Deliveries went through the global `fetch` to the tenant-registered URL, with no address check and redirects followed, and the response was kept on the delivery row. They now go through the new `untrustedFetch.post`: every resolved address checked at connect (loopback, private, link-local and cloud metadata, CGNAT, etc. refused), no redirect followed, the kept response body capped at 64 KiB. A refused attempt is a failed attempt, retried as before, and its reason (`address`, `scheme`, `redirects`, `network`, `timeout`) is recorded in a new nullable `error` column of the delivery table (`ensureTables` adds it to an existing table). **Changed:** an app whose webhooks go to its own internal services sets `createOutbound({ addresses: 'any' })` (plain fetch, still no redirects and the cap), which throws when `environment: 'production'` is passed. A non-2xx answer's body is no longer kept (`response_body` null; `status_code` recorded). `network: { resolve, connect }` is for tests.
+- **Added: `untrustedFetch.post(url, { body, headers, maxBytes, timeoutMs, maxRedirects, ... })`** (#866): `get`'s rules for a POST; a 307/308 resends the body, 301/302/303 become a GET without it. Trace step `pipework/fetch:post`.
+- **Fixed: SNS subscription and certificate fetches followed redirects** (#866): both now use `redirect: 'error'`.
+
+## 0.241.0
+
+- **Added: `crawl`, a polite, sandboxed site crawler that runs as a durable job** (#865). Turn it on with a `crawl: { requireApproval? }` Blueprint block (which also opts the `pipework_crawl*` tables into generated migrations); without it `crawl.create` throws `CrawlNotEnabledError`. `const site = crawl.create({ instance, name, queue, accept, userAgent, onPage?, store? })`; add `site.job` to a worker surface's `jobs`. `site.start({ seed, scope?, limits? })` records a crawl (one audit row) and queues it, or leaves it `awaiting_approval` until `site.approve(id)` when `requireApproval` is set. `accept({ url, contentType })` says what is listed; in-scope HTML is followed for links. robots.txt is read first per host and obeyed (RFC 9309: groups, longest match, `*`/`$`, `Crawl-delay`; 4xx allows all, 5xx/unreachable retried then stops with `robots_unreachable`). Per-host concurrency (1) and gap (max of `minGapMs` 1000 and `Crawl-delay`); limits on pages requested, depth, total bytes, bytes per response, wall time, request time, redirects and `onPage` time, and a crawl over one ends `stopped` with `stoppedBy` naming it. Every request goes through `untrustedFetch.get` (address rules at connect); each redirect is queued as its own URL and re-checked against scope and robots. Scope is the seed's host (a redirect of the seed itself to another host, e.g. `www.`, moves it) or `{ domain }`. Links come from a linear scanner that cannot execute anything. The frontier is in the database: a dead job resumes without refetching completed URLs (the one in flight may be fetched again); a later crawl of the same seed sends `If-None-Match`/`If-Modified-Since` and records a 304 as `unchanged`. `store` puts listed bytes into `storage` by sha256. `status`, `resources` and `refusals` read a crawl, only from its own tenant. One `pipework/fetch:get` trace step per request. `pipework/test` exports `crawlFixtureSite({ root, hosts, routes? })`, a directory served as a site under public-looking names, with redirects, delays, generated large bodies and a request log.
+- **Added: `untrustedFetch.get({ followRedirect })` and `etag` / `lastModified` on its ok result** (#865). `followRedirect(to, from)` returning false or a message refuses the hop as `rejected`.
+
+## 0.240.0
+
+- **Fixed: an exported `.appendOnlyVersions()` table fails declaration emit** (#863, TS2883). `AppendOnlyVersionsFields` is now exported from the package root, beside `EffectiveDatedFields`. Also newly root-exported, type-only, for the same reason: `FieldKind`, `FieldFlags`, `FieldRecord`, `ForeignKeyAction`, `CompositeForeignKey`, `PartitionSpec`, and `InsertProjection` / `SelectProjection` / `UpdateProjection` with zod's `ZodType` / `$ZodTypeInternals`, so an exported `insertShape()` / `selectShape()` / `updateShape()` result also emits.
+
+## 0.239.0
+
+- **Added: `alerts.raise({ resolveAfterQuietMs: null })` and `resendAfterMs: null`** (#861): `null` opts one alert out of the handle's quiet period (only `resolve` closes it) or its re-send period (never re-sent while open). A number still overrides the handle's value; omitted still falls back to it. The choice holds across raises while the alert stays open and clears when it reopens. A negative or non-integer value is refused. No schema change: `null` is stored as `-1` in the existing columns.
+- **Changed: a quiet resolve records itself** (#861): `resolvedBy` is `'quiet'` and `resolveNote` is `'quiet for <ms> ms'` (the period that applied); its audit row carries `by: 'quiet'`. Before, both were empty.
+
+## 0.238.0
+
+- **Fixed: a response schema with a lazy reference to a recursive schema stops the app booting** (#859). `toJsonSchema` now writes zod's `{ allOf: [{ $ref }] }` for a `z.lazy(() => node)` field as a bare `$ref` (other keys on the node, such as `description`, kept). Fastify's serializer looped forever on the `allOf` form when `node` was recursive (`Failed building the serialization schema ... Maximum call stack size exceeded`). The same output feeds the OpenAPI document. An `allOf` with more than one member, or a member that is more than a `$ref`, is unchanged.
+
+## 0.237.0
+
+- **Added: `support.requeueFailed` and `alerts.requeueFailed`** (#856): `({ by, sink?, failedSince? }) => { requeued }` puts the outbox's given-up deliveries (`failed_at` set, never delivered) back on the queue, across tenants, optionally only one sink's and/or those that failed at or after `failedSince`: `failed_at` cleared, `attempts` 0, due now; the next `tick` sends them. Delivery ids stay the same, so a receiver that dedupes on `delivery.id` takes each once. `by` (who asked) is required; each requeued delivery gets a `requeue` audit row with `by`, the attempts it had and its last error. It replaces hand SQL on `support_event_delivery` / `pipework_alert_deliveries` for answering `delivery-failed:<table>:<sink>`, which resolves after the resent deliveries go through. Like the other alerts audit rows, `alerts.requeueFailed`'s are written under a traced Flow.
+
+## 0.236.0
+
+- **Added: `instance.pool.connectionBudget(name)`** (#854): the connections a process may still open on a database's server without starving other pipework processes. Returns `{ maxConnections, reserved, held, othersUnopened, free }`: `reserved` is `superuser_reserved_connections` + `reserved_connections`, read from the server; `held` the client backends open now; `othersUnopened` each other live publishing process's reservation (the sum of its pools' `max`) minus what it holds; `free` what is left. Opt in per database with `pool: { publishReservation: true }`: each process then keeps one presence connection per server whose `application_name` is `pipework:<id>:<reservation>`, updated as pools are created and closed, and tags its pool connections `pipework:<id>` (a `pool.settings.application_name` of yours wins, and those connections are then not attributed). A process's reservation counts before its pools open a connection, and leaves `pg_stat_activity` with the process. No table, no superuser. **Limits:** a non-superuser cannot see other roles' `backend_type`, so `held` also counts their background workers, so `free` reads low rather than high. Each publishing process's presence connection is one more held connection.
+
+## 0.235.0
+
+- **Added: `mail.inbound({ attachmentRetentionDays })`** (#852). Each attachment stored through `storage` expires that many days after the message was received (from the inbound row's `received_at`, so a retried receive gives the same expiry), and the storage handle's `sweep` deletes the row and the object. A whole number of days, one or more (fractions round down; less is refused at create time). Unset, attachments have no expiry, as before.
+
+## 0.234.0
+
+- **Added: an http surface's OpenAPI document without an instance** (#848). `httpSurface.openapi()` builds the document from the surface's handlers, resources and `openapi` info, with no context and no I/O (`ConfigError` without `openapi` info). `pipework openapi [--surface <name>] [--out <file>]` loads the config, connects to nothing, and writes it (stdout without `--out`); without `--surface` exactly one http surface with `openapi` info must exist. The output is `JSON.stringify` of the document, byte-for-byte what `GET /openapi.json` serves: the server and `openapi()` register from one route list, so the spec can be a build output instead of a committed file.
+
+## 0.233.0
+
+- **Fixed: a received mail with an ambiguous author names none** (#847). With several `From:` fields, or several mailboxes in one `From:`, `ReceivedEmail.from` is the empty address and the new `fromProblem` says why (`'several-from-fields'` or `'several-mailboxes'`; `null` otherwise; type `FromProblem` on the root). Before, the first field's first mailbox was named, so a DMARC PASS for one domain could sit beside a `from` in another. DMARC vouches for `from` only when `fromProblem` is null and `from.address` is non-empty (RFC 7489 §6.6.1). A quoted display name with a comma is still one mailbox.
+
+## 0.232.0
+
+- **Changed: migrations are named by UTC timestamp and need no committed journal** (#846). `pipework generate` names a new migration `YYYYMMDDHHMMSS_<name>.sql` and writes `meta/<tag>.delta.json` beside it: the schema change that migration made. It no longer writes `meta/_journal.json` or `meta/NNNN_snapshot.json`, so two branches that each add a migration add only new files and merge without conflict. `migrate`, `migrate check` and `generate` take the order from the folder's `.sql` files: existing `NNNN_` migrations first, by index, then timestamped ones by time. The previous schema is the last `NNNN_snapshot.json` the old journal names, folded with each timestamped migration's delta. Existing `NNNN_` files, the journal and snapshots stay as they are (the journal still supplies the old migrations' times, and a tag it lists without a `.sql` is an error). A tracker that holds `0000`–`NNNN` applies only the new files. **Note:** every `.sql` in the migrations folder is now a migration (a file not named `NNNN_…` or `YYYYMMDDHHMMSS_…` is an error), and a `generate` that emits nothing writes nothing. Two branches that change the same column or index of one table still need the later one regenerated.
+
+## 0.231.0
+
+- **Added: `pipework serve --worker-only` and `manifold.start({ kinds })`** (#842). A worker-only process starts only the app's `worker` surfaces: it opens no port (`--port` with it is an error) and stops on SIGTERM like `serve`, so workers scale apart from the API against one queue. It does not run `script` surfaces (migrations, seeds), which belong to the main process. `start({ kinds: ['worker'] })` does the same in code; every surface is still validated, and a filter that matches none is a `ConfigError` naming the declared surfaces. The CLI passes the filter as `PIPEWORK_SURFACE_KINDS`.
+
+## 0.230.0
+
+- **Added: `support.create({ joinThread })`** (#840). `'contact'` (default) is unchanged: only the ticket's contact joins it by thread. `'conversation'`: a message threaded into a ticket's conversation (or sent to it with `ticketId`) joins it and reopens it whoever sent it; the message keeps its own sender, the ticket's contact and user stay, and only the contact's own message can fill a null `user_id`. The `receive` audit carries `joinedBy: 'conversation'` when someone else joined. A `separate`-screened message still opens its own ticket. Risk: anyone holding a reply's Message-ID can add to the ticket; pair it with a strict `screen`.
+
+## 0.229.0
+
+- **Fixed: `RECEIVE_JOB` and `SEND_JOB` are exported from the package root** (#841), so a worker's job map can name the email jobs (`'pipework.email.receive'`, `'pipework.email.send'`). `ReceivedEmail`, `Inbound`, `InboundSource` and the other inbound types have been on the root since 0.223.0.
+
+## 0.228.0
+
+- **Added: `support.http(handle, { path, credentials })`** (#835): an HTTP API for a support agent, shaped like `alerts.http`. Bearer keys, `name -> secret` (a drafting key) or `name -> { secret, approve: true }`; the name is the actor in audit rows, a draft's author and the approver. Routes: `GET /` (`state`, `category`, `tenantId`, `limit`, `cursor`), `GET /:id`, `PATCH /:id` (`category`), `POST /:id/transition`, `/:id/notes`, `/:id/links`, `/:id/drafts`, `PATCH` / `DELETE /drafts/:id`, `POST /drafts/:id/approve` (`{ version }`). No key 401, a key without `approve` 403 before anything runs, `canApprove` / `distinctApprover` 403, a stale or sent draft 409, a bad field 400. **Added:** `support.listAll({ state?, category?, tenantId?, limit?, cursor? })`, tickets across tenants. **Limit:** ids and lists are read across tenants on the system channel, which a database with `rls.platformRole` refuses.
+
+## 0.227.0
+
+- **Added: `support.create({ resolveContact })`** (#836): `({ address, name, tenantId }, tx) => Promise<{ userId } | null>` links a ticket to the sender's user. Called inside the receive's tenant transaction by `receiveEmail`, and by `receive` when no `contact.userId` is given: a new ticket gets the user in its insert (the `create` audit carries `userId`); a message joining a ticket whose `user_id` is null fills it (`userIdSet` on the `receive` audit). A non-null `user_id` is never changed. A throw rolls the receive back. Without it, behaviour is unchanged.
+
+## 0.226.0
+
+- **Added: composite foreign keys can clear only some columns on delete** (#834). `foreignKeys: [{ columns: ['tenantId', 'documentId'], references: { definition: Document, columns: ['tenantId', 'id'] }, onDelete: { setNull: ['documentId'] } }]` (or `{ setDefault: [...] }`) emits `ON DELETE SET NULL ("document_id")`, so deleting the document keeps the row and its tenant. Needs Postgres 15+. The list must be a non-empty subset of the key's columns, checked at define time. It is carried in the snapshot: switching an existing key to or from a column list drops and re-adds the constraint; keys without a list are unchanged and generate nothing.
+
+## 0.225.0
+
+- **Added: postpaid credit line per tenant** (#831). `creditLine.create({ warnShare?, timeZone?, onLineCrossed? })`, the arrears counterpart of prepaid credits, in whole cents. `openLine({ ceilingCents, caps?, warnShare? })` / `closeLine()`; `reserve` / `settle` / `release` by job id, as credits has them: a reserve fails past `ceiling - unpaid - held`, or past any cap (`{ name, cents, window: 'calendar_year' | 'calendar_month' | { from, to } }`, settled-in-window plus held), naming the limit; settle never fails on limits. `authorize({ reference, amountCents, from, to, replacesCaps? })` (a purchase order) replaces the ceiling, and the caps it names, while in force; one at a time; `authorizations()` lists what's left. `onLineCrossed({ limit, usedCents, limitCents, warnAtCents })` is awaited inside the write that takes a limit past its warn share (default 0.8), once per crossing. `recordPayment({ amountCents, reference })` clears unpaid entries oldest first, idempotent by reference, excess kept for later work. `statement({ from, to })` lists the period's entries and payments with totals. `line()` reads the state. Each call has a `...For(tenantId, ...)` form and is audited as `CreditLine`. **Migration:** set `creditLine: {}` and run `pipework generate`; it adds the `credit_line*` tables.
+
+## 0.224.0
+
+- **Added: join requests** (#830). `joinRequests.create({ isOwner, addMember, expiresInMs?, onRequested?, onSettled? })`: a user asks to join an existing org (`request(orgId, userId, { reason, evidence })`, one open request per user and org, a second returns the first) and an owner `approve`s or `refuse`s it; the joiner can `withdraw`. Approval calls the app's `addMember` in the same transaction, so a failed add leaves the request open. States `open -> approved | refused | withdrawn | expired`, with who settled it and when; expiry defaults to 30 days and `sweep()` marks lapsed ones (schedule it). `list({ status })` reads the Flow tenant's requests; `mine(userId)` reads one user's across orgs. Hooks are awaited inside the step's transaction; a throw rolls it back. Each step is a `JoinRequest` audit record on the org's chain (without `evidence`). **Migration:** set `joinRequests: {}` and run `pipework generate`; it adds `join_request`. **Limit:** `mine`, `withdraw` and `sweep` read across tenants on the system channel, which a database with `rls.platformRole` refuses.
+
+## 0.223.0
+
+- **Fixed: types missing from the package root** (#828). `LowBalanceState` and `LowBalanceCrossing` (credits) are now exported from `pipework`, as are the other public types a subsystem exported but the root left out: alerts `AlertOwnerChange` (`OwnerChange`), `ClientFaultReport`, `AlertRequestFaultsOptions`; mail inbound `Inbound`, `InboundConfig`, `InboundSource`, `InboundRecord`, `InboundScanResult`, `InboundSweepResult`, `ReceivedEmail`, `ReceivedAttachment`, `EmailAddress`, `EmailVerdicts`, `UnthreadedEmail`, `SesReceiptConfig`; access links `AccessLinkCodeConfig` (`CodeConfig`). Type-only; no runtime change.
+
+## 0.222.0
+
+- **Added: `pipe.filter.inList(expr, values, cast?)` and `notInList`** (#825): list membership on the filter surface, bound as one array parameter, `expr = ANY($1::<type>[])` / `expr <> ALL($1::<type>[])`, which Postgres hashes. Unlike `inArray` (one parameter per element, failing past 65,535) it takes any length, and unlike a join on `unnest` or `arrayAny` it plans no subquery. An empty list is `false` / `true`. Strings bind as `text`, integers as `bigint`, other numbers as `numeric`; pass `cast` for other columns, e.g. `inList(t.id, ids, 'uuid')`. Mixed types, null, NaN or a bad cast throw.
+
+## 0.221.0
+
+- **Added: acknowledging an alert can name its owner** (#824). `alerts.acknowledge(key, { by, note?, owner? })` and `POST /ops/alerts/:key/acknowledge` with `owner` replace the stored owner and append `{ from, to, by, at }` to the alert's new `ownerHistory` (nothing when it is already the owner). An acknowledged alert given a different `owner` changes the owner only, keeping who acknowledged it and when. An owner set this way survives later raises and reopens; the raising code's owner applies only until then. `alerts.list({ owner })` and `GET /ops/alerts?owner=a,b` filter on the current owner. An empty owner is refused (400 over HTTP). **Added:** `OwnerChange` type. **Migration:** run `pipework generate`; it adds `owner_history jsonb NOT NULL DEFAULT '[]'` to `pipework_alerts`.
+
+## 0.220.0
+
+- **Added: one-click unsubscribe per list for mail** (#821). `send` takes `list` (e.g. `'notice-digest'`); such a message carries RFC 8058 `List-Unsubscribe` and `List-Unsubscribe-Post: List-Unsubscribe=One-Click` headers with a signed, non-expiring link, and is skipped at delivery (status `unsubscribed`, no provider call, `Email:unsubscribed` audit) when its address has opted out of that list. Mail with no list is never affected. `MailConfig.unsubscribe: { url, secret }` (secret at least 32 characters; url https in production) is required for a send with a list. `mail.unsubscribeHttp()` serves the link's routes: GET a confirm page, POST the one-click opt-out (form-urlencoded, multipart or empty body, never read). `mail.unsubscribe` / `unsubscribed` / `resubscribe` back an app's settings page. Templates get `{ unsubscribeUrl }` as a second argument (null with no list). **Added:** `Builder.ignoreBody()` for a route that takes any body unread; `EmailRenderMeta` type. **Migration:** run `pipework generate`; it adds the `pipework_email_unsubscribes` table, the `list` column on `pipework_email_messages`, and `unsubscribed` to its status check.
+
+## 0.219.0
+
+- **Added: `/ready` and `/health` name the release** (#820). Both bodies carry `release: string | null`, which is `manifold.release`, the same value written on traces and job claims; null when no `release` block is configured or its variable is unset. A deploy can poll `/ready` until it answers with the release it stamped. `HealthStatus` gains `release`.
+
+## 0.218.0
+
+- **Added: `useTestClient(surface, context?, { before? })` in `pipework/test`** (#818). It makes the test client in its own `beforeAll` (after `before`, for app setup such as services and sinks) and closes it in its own `afterAll` only if it was made, so a file whose setup fails reports only that error, not a second `Cannot read properties of undefined (reading 'close')`. It replaces `let client; beforeAll(async () => { client = await api.test(useTestInstance()) }); afterAll(() => client.close())`. `context` defaults to `useTestInstance`. The returned `TestClient` forwards `inject` to the client made; `inject` rejects with a `[pipework]` error naming the case when the client was not made or is closed. `close()` may be called early, and the `afterAll` then does nothing.
+
+## 0.217.0
+
+- **Changed: the generated `tsconfig.base.json` turns on `noUnusedLocals`** (#816), so an unused import or local fails `pnpm type-check` and `pipework check` with TS6133, and `pipework sync` keeps it. `noUnusedParameters` stays off. A module turns either off or on through its `tsconfig` field in `pipework.config.ts`, which is merged over the base; for every module under a directory use a glob key, e.g. `modules: { 'packages/*': { tsconfig: { noUnusedLocals: false } } }`, and an explicit module key overrides the glob. **Migration:** run `pipework sync`; remove the unused locals it reports, or opt the module out as above.
+
+## 0.216.0
+
+- **Added: `.onRefused(hook)` on a route, called once for every 4xx it answers** (#814), including the refusals pipework makes before the handler runs: the input/params/query `VALIDATION_ERROR` (422), malformed JSON (400), auth 401, permission 403, rate-limit 429, body-limit 413 and upload 413/415, and the handler's own thrown 4xx. It is never called for a 5xx, the route's own `screen` refusal, or a URL no route matches. The hook gets `{ status, code, message, fields, auth, params, query, request, requestId }` (`auth` is null where auth refused). A plain object it returns is merged into the response's `error`, so a route can add `escalated: true`; `code`, `message`, `fields` and `details` keep pipework's values (a hook that sets one is warned and dropped). A throw is logged and the refusal goes out unchanged. It runs in the request's Flow as the trace step `pipework/http:on-refused` after `http.error`, once the request's transaction has rolled back. The upload `onRefuse` hook now also runs before the trace closes. **Added:** `RouteRefusal`, `RouteRefusedHook` types.
+
+## 0.215.2
+
+- **Fixed: a template build whose app role failed to create left a template every later run cloned** (#812). The marker that makes a template clonable was written before the app role, its grants and the test-system policies, so a refused `CREATE ROLE` left a marked template and each later run failed at `GRANT CONNECT … TO "<app role>"` (role does not exist) until the template was dropped by hand. The marker is now written last. A marked template is reused only while its app role and `rls.platformRole` exist on the cluster; otherwise it is rebuilt, which also repairs templates left by earlier versions. A `CREATE ROLE` refused for lack of privilege now throws `[pipework] The test login "<login>" needs CREATEROLE on the test cluster to make role "<role>"`.
+
+## 0.215.1
+
+- **Fixed: a forked test run left its workers' last queued clones in Postgres** (#810). The final drop ran on the fork's `disconnect`, which vitest 4 never sends (it SIGTERMs the fork after the stop handshake), so each fork left up to `dropBatchSize - 1` clones for the next run's reaper. The pipework vitest plugin now adds a `globalSetup` whose teardown runs in the parent after every worker has stopped and drops clones whose run lock is free (never templates, never a concurrent run's live clones), naming them on one `pipework:test-reap` stderr line. Skipped with `pipework({ database: false })`. Best effort. **Added:** `reapOrphanTestDatabases`'s `clonesOnly` option, and `reapConfiguredTestDatabases(instance, options)`.
+
+## 0.215.0
+
+- **Fixed: the orphan reap in test setup made no progress the stall watchdog could see** (#808). `run-prepare` reaps leftover test databases and stale templates one drop at a time; with dozens of orphans on a loaded host the run failed `SetupStalled` naming `run-prepare` while the reap was still working. A new phase `run-reap` beats before each candidate (detail `n/total name`, one total over clones and templates), so a hung drop is named. A clone's `CREATE DATABASE … WITH TEMPLATE` retry while the template is in use beats `clone-start` (detail `retry n`). **Added:** `reapOrphanTestDatabases(admin, liveTemplates, { onReap })` and `withTemplateRetry`'s `onRetry` option.
+
+## 0.214.0
+
+- **Fixed: a template build's work after the last migration made no progress the stall watchdog could see** (#806). Framework tables, the audit chain, framework policies, day partitions (hundreds with `audit.retentionDays: 365`), the template marker, role grants and test-system policies ran under the last `template-migrate` beat, so a loaded host tripped `SetupStalled` naming `template-migrate` while setup was still working. Each now advances setup: new phases `template-framework`, `template-partitions` (one beat per partition as its CREATE starts, detail `n/total name`) and `template-grants` (per routine and per policy), and each extension beats `template-build`. A stall now names the step actually stuck. **Added:** `syncTracePartitions(db, config, now, { onPartition })` and `provisionFrameworkTables`'s `onProgress` option.
+
+## 0.213.1
+
+- **Fixed: the test setup `beforeAll` was still bounded by vitest's 60 s `hookTimeout`** (#804). A template build that kept advancing on a loaded host was killed mid-build and restarted on the next run, so the stall watchdog (`test.setupStallMs`, #380) never got to judge it. The setup hook in `pipework/test/auto-setup` and in `setupPipeworkTests` now takes `SETUP_HOOK_TIMEOUT_MS` (setTimeout's ceiling), so setup fails only with `SetupStalled` when a phase stops advancing. Every other hook keeps the project `hookTimeout`.
+
+## 0.213.0
+
+- **Added: `uniqueIndex(...).nullsNotDistinct()`** (#802). The index is created `NULLS NOT DISTINCT` (PG 15+), so NULLs in its columns compare equal and `ON CONFLICT (cols)` fires on them. Turning it on or off for an existing index migrates as an index replace: the new index is built `CONCURRENTLY` beside the old one, the old one dropped `CONCURRENTLY`, the new one renamed into its place, with no ACCESS EXCLUSIVE lock held for the build. Calling it on a non-unique index throws. Snapshots record the flag only when set, so existing schemas produce no diff. **Fixed: a unique constraint or composite primary key added to an existing table was built under ACCESS EXCLUSIVE.** It is now built as a unique index `CONCURRENTLY`, then attached with `ADD CONSTRAINT … USING INDEX`. A changed one has its new index built first and is swapped in one `ALTER TABLE … DROP CONSTRAINT old, ADD CONSTRAINT … USING INDEX` statement, so the key is never absent (when the old key's columns are dropped or change type, the old constraint is dropped first, as before). Partitioned tables keep the plain `ADD CONSTRAINT`. Each pipework-named concurrent build first drops a leftover index of its name, so a migration that failed (for example on existing duplicate rows, which must be removed before a `NULLS NOT DISTINCT` or new unique index can build) can be rerun. A unique constraint or primary key that shares its name with a declared index on the table now fails at `pipework generate`. **Migration:** run `pipework generate`; tables that had NULL-duplicate rows under the old index need them removed before the migration applies.
+
 ## 0.212.0
 
 - **Added: `enqueue({ pendingKey })` coalesces with a pending job and takes the newer payload** (#800). While a job on the queue with the same key and the same tenant is pending (not yet claimed), `enqueue` replaces that job's payload with this call's and returns its id; every other option of the call is ignored and the job keeps its place in line (queue time, priority, attempts, flow). Once the job is claimed, the next `enqueue` inserts a new job, so a re-render enqueued after a render started still runs. Not combinable with `liveKey`. Atomic across concurrent enqueuers: a partial unique index `uq_<table>_pending_key` on `(pending_key, coalesce(flow_tenant, ''))` over pending jobs, `INSERT … ON CONFLICT DO UPDATE`, under a transaction-scoped advisory lock per key and tenant (held to the enclosing transaction's commit). A claimed job that would return to pending (a retry after `failAttempt`/`fail`, `reap`, `handBack`, `later`) while a newer pending job holds its key is cancelled instead, its `error_payload` `{ reason: 'superseded', by: <id> }`, and `waitFor` wakes; `FailedAttempt.outcome`, `ReapedJob.outcome` and the `reaped` job event's `outcome` gain `'superseded'`. `requeueDeadLetter` of a job whose key a pending job holds throws naming it. `ClaimedJob.pendingKey` carries the key. **Migration:** queue tables gain a `pending_key` column and the index; run `pipework generate`.

package/REFERENCE.md

@@ -29,7 +29,9 @@
 - [webhook — Inbound & Outbound Webhooks](#webhook-inbound-outbound-webhooks)
 - [mail — Transactional Email](#mail-transactional-email)
 - [accessLinks — Access Links](#accesslinks-access-links)
+- [joinRequests — Join Requests](#joinrequests-join-requests)
 - [credits — Prepaid Usage Credits](#credits-prepaid-usage-credits)
+- [creditLine — Postpaid Credit Line](#creditline-postpaid-credit-line)
 - [untrustedFetch — Fetch a URL a Stranger Gave](#untrustedfetch-fetch-a-url-a-stranger-gave)
 - [connectRetry — Retry a Call That Never Connected](#connectretry-retry-a-call-that-never-connected)
 - [storage — File Storage](#storage-file-storage)
@@ -724,6 +726,12 @@ Creates a worker surface — a long-running process that claims and executes job
 
 Creates a script surface — a one-shot process that runs and exits (migrations, seeds, CLI tools). The run is traced as a job-kind trace that names the surface.
 
+### Starting some surfaces: `manifold.start({ kinds })` and `pipework serve --worker-only`
+
+`manifold.start({ port?, kinds? })`: `kinds` (`'http' | 'worker' | 'script'`) starts only the surfaces of those kinds and logs the skipped ones once at info level. Unset starts every surface. The whole surface map is still validated, the start `port` is checked against the surfaces being started only, and `stop()` covers only those. A filter that matches no surface throws a `ConfigError` naming the declared surfaces.
+
+`pipework serve --worker-only` runs a job worker with no HTTP server, to scale workers apart from the API: it starts the `worker` surfaces only (the app's job handlers and the worker's own `concurrency`, classes and queue settings apply as in `serve`), opens no port, and stops cleanly on SIGTERM like `serve`. Several worker-only processes and one full `serve` share one database and one queue; a job held by a stopped worker is handed back and taken by another. `script` surfaces (migrations, seeds) are one-shot work that belongs to the main process, so a worker-only process does not run them and a scaled worker never repeats them. `--worker-only` with `--port` is an error. `dev` has no such flag. The flag sets `PIPEWORK_SURFACE_KINDS=worker` for the child; the variable takes a comma list of surface kinds, and an unknown kind fails at start.
+
 ## pipe — Database
 
 Database accessor — returns a Drizzle DB instance scoped to the current request/job/test context. Requires active AsyncLocalStorage context.
@@ -809,6 +817,9 @@ Expression helpers for `.set()`, `.values()` and `.where()` (a string or number
 **`filter.arrayAny(source, test)`** — true when some element of the array satisfies the test: `EXISTS (SELECT 1 FROM unnest(<source>) AS pw_el_<n>(v) WHERE <test>)`. False for an empty array.
 **`filter.arrayAll(source, test)`** — true when every element satisfies the test: `NOT EXISTS (SELECT 1 FROM unnest(<source>) AS pw_el_<n>(v) WHERE NOT (<test>))`. True for an empty array; a null element passes when the test on it is null (SQL three-valued logic).
 
+**`filter.inList(expr, values, cast?)`** — `<expr> = ANY($1::text[])`: a JS `string[]` / `number[]` bound as one array parameter whatever its length, hashed by Postgres, so it works past `inArray`'s 65,535-parameter limit. Empty list is `false` (no parameter). Strings bind as `text`, integers as `bigint`, other numbers as `numeric`; pass `cast` (`'uuid'`, `'int'`) for other columns: `filter.inList(t.id, ids, 'uuid')`. Mixed types, null, NaN or a bad cast throw.
+**`filter.notInList(expr, values, cast?)`** — `<expr> <> ALL($1::text[])`, same binding and `cast`; empty list is `true`.
+
 `source` is an array column, an `SQL` array expression, a JS `string[]` (bound as one parameter cast `::text[]`, also when empty) or `number[]` (`::numeric[]`), or `filter.jsonbElements(jsonbColumnOrSql)` for a jsonb array (`jsonb_array_elements_text`, elements as text). `test` receives the element as a typed `SQL` (column `text[]` gives `SQL<string>`). Aliases are unique per call, so calls nest.
 
 ```ts
@@ -926,6 +937,22 @@ const Unit = pipe.define('units', {
 await pipe.insert(Unit).values({ tenantId, key }).onConflictDoNothing({ target: ['tenantId', 'keyDigest'] })
 ```
 
+A composite foreign key is declared with the `foreignKeys` option. `onDelete` takes an action (`'cascade'`, `'set null'`, ...) that applies to every column of the key, or `{ setNull: [fields] }` / `{ setDefault: [fields] }` to clear only the listed fields (Postgres 15 or later: `ON DELETE SET NULL ("document_id")`). The list must be non-empty, without duplicates, and a subset of `columns`. Changing the action or the list drops and re-adds the constraint.
+
+```ts
+const Cost = pipe.define('costs', {
+  id: pipe.field.uuid().primaryKey().defaultRandom(),
+  tenantId: pipe.field.uuid(),
+  documentId: pipe.field.uuid().nullable(),
+}, {
+  foreignKeys: [{
+    columns: ['tenantId', 'documentId'],
+    references: { definition: Document, columns: ['tenantId', 'id'] },
+    onDelete: { setNull: ['documentId'] },   // deleting the document keeps tenantId, nulls documentId
+  }],
+})
+```
+
 ### `pipe.definitions`
 
 Returns the global registry of all domain definitions.
@@ -1359,7 +1386,7 @@ Signs a payload with HMAC for outbound delivery — returns body, signature, tim
 
 Mail namespace — email through a job queue, SES v2 or a dev stand-in, bounce and complaint suppression, and receiving.
 
-Opt in with blueprint `email: {}`; the generated migration adds `pipework_email_messages` (`key` unique, `to_address`, `template`, `data`, `status` `queued | sending | sent | suppressed | failed`, `provider_message_id`, `error`, `trace_id`, `created_at`, `sent_at`, `sending_at`, `failed_at`) and `pipework_email_suppressions` (`address` primary key, lowercased; `reason` `bounce | complaint`; `detail`; `created_at`), both `IF NOT EXISTS` and not tenant-scoped. `pipework_email_messages` also has `conversation_id` and `in_reply_to_inbound`. Receiving adds `pipework_email_inbound` (see `mail.inbound`).
+Opt in with blueprint `email: {}`; the generated migration adds `pipework_email_messages` (`key` unique, `to_address`, `template`, `data`, `status` `queued | sending | sent | suppressed | unsubscribed | failed`, `list`, `provider_message_id`, `error`, `trace_id`, `created_at`, `sent_at`, `sending_at`, `failed_at`) and `pipework_email_suppressions` (`address` primary key, lowercased; `reason` `bounce | complaint`; `detail`; `created_at`), both `IF NOT EXISTS` and not tenant-scoped. `pipework_email_messages` also has `conversation_id` and `in_reply_to_inbound`. Receiving adds `pipework_email_inbound` (see `mail.inbound`).
 
 ### `mail.create`
 
@@ -1369,6 +1396,10 @@ Creates the transactional email interface: `send(db, { to, template, data, key }
 
 `mail.sentTo(db, address, { template? })` (for tests and support tools) reads the messages to an address (any case), oldest first, optionally of one template: `{ id, template, status, data, createdAt, rendered }`, where `rendered` is `{ subject, text, html? }` as delivery would render the stored data (late data resolved; `__late` left out of `data`), or null when the template is gone or throws. `emailLinks.liveFor(db, address, purpose)` (same use) lists the address's links of that purpose that are not used, ended or expired, oldest first: `{ id, token, url, expiresAt }`, the token being the one `check` and `use` take.
 
+### Lists and one-click unsubscribe
+
+Set `unsubscribe: { url, secret }` on `mail.create` (`url` the absolute https URL of the route, http outside production; `secret` at least 32 characters) and send with `list: 'notice-digest'`. The message carries `List-Unsubscribe: <url?t=token>` and `List-Unsubscribe-Post: List-Unsubscribe=One-Click` (RFC 8058). The token is an HMAC-signed address and list, with no expiry. Mount `mail.unsubscribeHttp()` (public, no tenant) on the HTTP surface: GET shows a confirm page, POST (form-urlencoded, multipart or empty, never read) records the opt-out in `pipework_email_unsubscribes` (`address`, `list`, `created_at`; primary key `(address, list)`; not tenant-scoped; enabled by `email: {}`). A later send on that list to that address ends `unsubscribed` (`Email:unsubscribed` audit record) with no provider call; mail with no list, and other lists, are unaffected. `mail.unsubscribed(db, address, list)` reads the setting; `mail.unsubscribe(...)` and `mail.resubscribe(...)` write it from the app's own settings page (they need a Flow). A template is called `template(data, { unsubscribeUrl })`, with `unsubscribeUrl` the same link, or null without a list.
+
 ### `mail.ses`
 
 Amazon SES v2 provider, SigV4-signed without the AWS SDK.
@@ -1599,6 +1630,35 @@ await github.call({ key: `github:installation:${installationId}`, class: 'bulk'
 await conn.close() // at shutdown
 ```
 
+## joinRequests — Join Requests
+
+A signed-up user asks to join an org that already exists; one of the org's owners approves (the member is added in the same transaction) or refuses. For "your colleague's company already has an account": a domain match, a shared invoice id, an SSO tenant.
+
+Opt in with blueprint `joinRequests: {}`. The generated migration then includes one tenant-scoped table with forced RLS, `join_request`: `tenant_id` (the org), `user_id`, `status` (`open | approved | refused | withdrawn | expired`), `reason`, `evidence` jsonb, `requested_at`, `expires_at`, `settled_at`, `settled_by`, `role` (set on approve), `note` (set on refuse). A partial unique index allows one `open` row per (org, user).
+
+### `joinRequests.create`
+
+`joinRequests.create({ isOwner, addMember, expiresInMs?, onRequested?, onSettled? })` returns a handle. Roles are the app's, so the app passes them in:
+
+- `isOwner(tx, { orgId, userId })` — who may settle a request; called inside the org's transaction.
+- `addMember(tx, { orgId, userId, role, requestId, by })` — adds the member; called inside the approve transaction after the status update, so a throw rolls the approval back.
+- `expiresInMs` — how long a request stays open. Default 30 days.
+- `onRequested(tx, request)` / `onSettled(tx, request)` — awaited inside the transaction of the step (settled: approved, refused, withdrawn, expired), so the app can write its notice or enqueue its mail atomically; a throw rolls the step back.
+
+| method | meaning |
+|---|---|
+| `request(orgId, userId, { reason?, evidence? })` | The joiner asks; runs in the named org, so the Flow needs no tenant. Returns `{ request, created }`. While an unexpired open request exists it is returned with `created: false` (no audit, no hook). An open one past its expiry is marked expired first. |
+| `approve(requestId, { role? }, by)` | Flow tenant. `by` must pass `isOwner`, else `errors.Forbidden`. Returns `{ ok: true, request }` or `{ ok: false, reason: 'not-found' \| 'expired' \| 'already-settled' }`. |
+| `refuse(requestId, { note? }, by)` | As `approve`, without `addMember`. |
+| `withdraw(requestId, userId)` | The joiner withdraws their own open request; another user's gets `not-found`. |
+| `list({ status? })` | Flow tenant's requests, default `open`, newest first. |
+| `mine(userId)` | One user's requests across orgs, through the system channel, newest first. |
+| `sweep()` | Marks open requests past expiry as expired (`settled_by` null); schedule it, e.g. a cron job. Returns `{ expired }`. |
+
+Each step writes a `JoinRequest` audit record (verbs `request`, `approve`, `refuse`, `withdraw`, `expire`) on the org's tenant when the Flow has a trace with an open step. The record carries the request without `evidence`, which may be personal data.
+
+`mine`, `withdraw` and `sweep` read `join_request` across tenants on the system channel; a database in platform mode (`rls.platformRole`) refuses that read for a tenant table.
+
 ## credits — Prepaid Usage Credits
 
 Prepaid credits per tenant, like CI minutes: a subscription grants an allocation each period, packs are bought on top, and work draws them down. Every grant has its own expiry; spending draws the soonest-expiring grant first. A job reserves before it starts, settles the actual amount when it ends, and releases on failure. Units are whole numbers (safe integers, bigint in SQL); the app decides what one unit is.
@@ -1681,6 +1741,36 @@ export const stripeWebhook = webhook.defineInbound<StripeEvent>(
 
 `sourceRef: event.id` makes Stripe's redelivery of the same event credit once.
 
+## creditLine — Postpaid Credit Line
+
+A postpaid line per tenant, the counterpart of prepaid credits: the work runs first and is paid for later, within limits. A job reserves against the line before it starts, settles its actual cost as an unpaid entry, and releases on failure. Amounts are whole cents (safe integers, bigint in SQL). Who may have a line, the cap amounts and what happens to work past a limit stay the app's.
+
+Opt in with blueprint `creditLine: {}`. The generated migration then includes six tables, all tenant-scoped with forced RLS: `credit_line` (one per tenant: `ceiling_cents`, `warn_share`, `caps` jsonb, `opened_at`, `closed_at`), `credit_line_reservation` (`job_id` unique per tenant, `amount`, `status`, `settled_amount`, `authorization_id`), `credit_line_entry` (one per settled job: `amount_cents`, `paid_cents`, `authorization_id`, `at`), `credit_line_payment` (`reference` unique per tenant, `amount_cents`, `unapplied_cents`, `at`), `credit_line_payment_application` (which payment cleared how many cents of which entry) and `credit_line_authorization` (`reference` unique per tenant, `amount_cents`, `starts_at`, `ends_at`, `replaces_caps`, `revoked_at`).
+
+`creditLine.create({ warnShare?, timeZone?, onLineCrossed? })` returns a handle. `warnShare` (default 0.8) is the share of a limit at which it warns; a line or a cap can set its own. `timeZone` (default `'UTC'`) is the zone calendar windows are cut in. Every method takes its tenant from the Flow and has a `...For(tenantId, ...)` form, exactly as credits does; operations on one tenant's line run one after another.
+
+- `openLine({ ceilingCents, caps?, warnShare? })` — opens the line, or updates an open one (and reopens a closed one). A cap is `{ name, cents, window: 'calendar_year' | 'calendar_month' | { from, to }, warnShare? }`; names are unique. Audited as `CreditLine` `open`.
+- `closeLine()` — new reserves fail `'no-line'`; held reservations may still settle or release; unpaid work stays.
+- `reserve({ jobId, amountCents })` — `{ ok: true, reservation }`, `{ ok: false, reason: 'no-line' }` or `{ ok: false, reason: 'limit', limit: { kind, name }, limitCents, usedCents, requestedCents }`. With no authorization in force the ceiling's use is unpaid + held; then each cap's use (settled entries in its window + held) is checked in configured order, and the first one that would be exceeded is the refusal. A `jobId` that already has a reservation returns it unchanged.
+- `settle({ jobId, amountCents })` — writes the unpaid entry and never fails on limits. The same amount again is a no-op; a different amount, a released job or an unknown job throws. Payments with unapplied cents clear the new entry, oldest payment first.
+- `release({ jobId })` — `{ released, reservation }`; a settled job throws.
+- `authorize({ reference, amountCents, from, to, replacesCaps? })` — a purchase order, `{ authorization, created }`, idempotent by reference. While one is in force (`from <= now < to`, not revoked) its amount replaces the ceiling check and the caps named in `replacesCaps`; reservations made then, and their entries, count against it. Dates overlapping an unrevoked authorization throw `ValidationError`. `revokeAuthorization(reference)` ends it; `authorizations()` lists them with `usedCents`, `heldCents`, `remainingCents`, `inForce`.
+- `recordPayment({ amountCents, reference, at? })` — `{ payment, created }`, idempotent by reference. Clears unpaid entries oldest first and keeps any excess as `unappliedCents`.
+- `statement({ from, to })` — the entries and the payments (each with the entries it cleared) with `at` in `[from, to)`, and totals `settledCents`, `paidCents` and `unpaidCents` (everything unpaid as of now).
+- `line()` — `{ open, ceilingCents, unpaidCents, heldCents, availableCents, caps (with usedCents), authorization, ... }`; `availableCents` is the smallest headroom of the limits that apply now.
+
+`onLineCrossed({ tenantId, limit: { kind: 'ceiling' | 'cap' | 'authorization', name }, usedCents, limitCents, warnAtCents })` is awaited INSIDE the transaction of the write that took a limit's use from below `floor(limit * warnShare)` to at-or-above it (reserve, a settle above its reservation, openLine lowering a limit), in a Flow for that tenant where `pipe()` is that transaction. If it throws, the write rolls back. A release or a payment, or a raised limit, re-arms the warning; nothing is stored.
+
+Every step is audited on the tenant's chain as entity `CreditLine` with verbs `open`, `close`, `reserve`, `settle`, `release`, `authorize`, `revoke_authorization` and `payment`.
+
+```ts
+const line = pipework.creditLine.create({ onLineCrossed: ({ tenantId, limit }) => warnOwner(tenantId, limit.name) })
+await line.openLineFor(tenantId, { ceilingCents: 250_000, caps: [{ name: 'yearly', cents: 1_000_000, window: 'calendar_year' }] })
+const held = await line.reserve({ jobId: run.id, amountCents: 70_000 })
+if (!held.ok) return queue(run, held)
+await line.settle({ jobId: run.id, amountCents: used })
+```
+
 ## untrustedFetch — Fetch a URL a Stranger Gave
 
 No blueprint key. `untrustedFetch.get(url, options)` fetches one URL with GET and never lets it reach the app's private network. The rules cannot be switched off:
@@ -1948,13 +2038,15 @@ The first three are tenant-scoped with forced row-level security.
 
 ### `support.create`
 
-`support.create({ mail, sinks, queue, canApprove, distinctApprover?, replyTemplate?, screen?, jobType?, maxAttempts?, backoffMs? })` returns a handle.
+`support.create({ mail, sinks, queue, canApprove, distinctApprover?, replyTemplate?, screen?, resolveContact?, joinThread?, jobType?, maxAttempts?, backoffMs? })` returns a handle.
 
 - `mail` — the app's `mail.create(...)` result, whose `templates` include `...support.templates` (`'pipework.support.reply'`, rendered verbatim: the email is exactly the approved text and html).
 - `sinks` — `SupportSink`s (`Sink<SupportEventPayload>`): `alerts.webhook(...)` and `alerts.memory()` fit where one is expected (`alerts.memory<SupportEventPayload>()` as a variable), or your own `{ name, send(delivery) }`. At least one. A delivery's `kind` is the event kind, `key` the ticket id, `payload` `{ kind, ticketId, tenantId, subject, state, category, at }` (no message bodies).
 - `canApprove({ actorId, tenantId })` — who may approve; e.g. `rbac.can(actorId, 'support:approve', { tenantId })`.
 - `distinctApprover` — default `true`: the author of a draft (last editor) cannot approve it.
 - `screen(email)` — `'accept' | 'separate' | 'drop'`. `drop`: nothing is stored (audited). `separate`: always opens a new ticket (`conversation_id` null), never adds to an existing one, so a forged From address cannot put text into a ticket the agent is reading. Default: virus `FAIL` drops; DMARC `FAIL`, or SPF and DKIM both `FAIL`, separates; anything else, including no verdicts (the folder stand-in), is accepted.
+- `joinThread` — `'contact'` (default) or `'conversation'`; anything else throws at `create`. With `'contact'`, only the ticket's contact joins it by thread (a colleague's reply opens its own ticket) and `receive` with a `ticketId` refuses a sender who is not the contact. With `'conversation'`, a message threaded into the ticket's conversation, or sent with `ticketId`, joins it and reopens it whoever sent it: the message keeps its own sender in `from_address`, the ticket's contact does not change, a colleague never fills the ticket's null `user_id` (the resolver is not called for one), and the `receive` audit carries `joinedBy: 'conversation'`. A message `screen` separates still never joins. Risk: anyone who holds a reply's Message-ID can add to the ticket, so pair it with a strict `screen` (for example, require a DMARC pass).
+- `resolveContact({ address, name, tenantId }, tx)` — `Promise<{ userId } | null>`: the user a sender is (`address` lowercased). Called inside the receive's tenant transaction (`tx` is RLS-bound to that tenant; a throw rolls the receive back) by `receiveEmail` and by `receive` without `contact.userId`: for a new ticket (its `user_id` is set in the insert; the `create` audit carries `userId`), and for a message joining a ticket whose `user_id` is null, which it fills (the `receive` audit carries `userIdSet`). A non-null `user_id` is never changed and not resolved again. Not called for a dropped message. Without it, `receiveEmail` tickets have a null `user_id`.
 
 ### Receiving