@voltro/cli 0.33.0 → 0.35.0

package/templates/agent-docs/plugins/auth.md

@@ -11,7 +11,7 @@
 
 _Password + session-cookie auth — overview. Full docs in the Authentication section._
 
-`@voltro/plugin-auth` ships the full server-side auth suite — password hashing (with rehash-on-verify), HMAC session cookies (multi-key rotation + sliding-window auto-renewal), magic-link + password-reset flows, passkeys/WebAuthn (with atomic clone detection), CSRF, session enumeration + revocation, multi-tenant memberships + switch-tenant, TOTP/MFA with sign-in enforcement + recovery codes — plus React glue (`SubjectProvider`, `useSubject`, `RequireAuth`) and a browser passkey ceremony helper. Identity is a pluggable [strategy](/docs/authentication/strategies) protocol — the password flow is the default, and external IdPs stack on top.
+`@voltro/plugin-auth` ships the full server-side auth suite — password hashing (with rehash-on-verify), HMAC session cookies (multi-key rotation + sliding-window auto-renewal), magic-link + password-reset flows, email verification, tenant invitations, user impersonation, passkeys/WebAuthn (with atomic clone detection), CSRF, session enumeration + revocation, multi-tenant memberships + switch-tenant, TOTP/MFA with sign-in enforcement + recovery codes — plus React glue (`SubjectProvider`, `useSubject`, `RequireAuth`) and a browser passkey ceremony helper. Identity is a pluggable [strategy](/docs/authentication/strategies) protocol — the password flow is the default, and external IdPs stack on top.
 
 The single `authRoutesPlugin()` mounts **every** auth HTTP route under `/auth` — you don't hand-wire endpoints.
 
@@ -37,7 +37,7 @@ Add `authRoutesPlugin()` to your api's `plugins` array. It mounts the entire aut
 // app.config.ts
 import { authRoutesPlugin, postgresUserStore, mailSender } from '@voltro/plugin-auth'
 import { mailPlugin, MailService } from '@voltro/plugin-mail'
-import { bindConnectionSubject } from '@voltro/runtime'
+import { bindConnectionCredential } from '@voltro/runtime'
 import { Effect } from 'effect'
 
 // `mail` is the yielded MailService from the mail plugin's services layer.
@@ -49,7 +49,7 @@ const auth = (mail: MailService) =>
     cookieSecure:    process.env.NODE_ENV === 'production',
     appBaseUrl:      'https://app.example.com',
     sendEmail:       mailSender(mail),           // ← plugin-mail synergy (the default wiring)
-    rebind:          bindConnectionSubject,       // ← switch-tenant rebinds the live connection
+    rebind:          bindConnectionCredential,    // ← switch-tenant presents the new cookie on the live connection
     passkey: {
       rpId:   'example.com',
       rpName: 'Acme',
@@ -83,7 +83,7 @@ The plugin:
 - Mounts every auth route under `/auth` via `authRoutesPlugin()` (also re-exports the underlying Effect-typed handlers — `handleSignIn`, `handleSignUp`, `handleMagicLinkRequest`, `handleSwitchTenant`, … — if you'd rather mount a subset yourself).
 - Resolves `ctx.subject` via `AuthMiddleware` running the strategy chain — the built-in `voltroPasswordStrategy` reads the session cookie.
 - Exposes the typed user store via `postgresUserStore(sql)` (synchronous; the caller owns the `SqlClient`) or `memoryUserStore()` for dev/tests.
-- Contributes the auth tables (`usersTable`, `sessionsTable`, `membershipsTable`, `authTokensTable`, `passkeysTable`) via `authTables`.
+- Contributes the auth tables (`usersTable`, `sessionsTable`, `membershipsTable`, `authTokensTable`, `passkeysTable`, `invitationsTable`, `impersonationGrantsTable`, …) via `authTables`.
 
 ## Routes mounted under `/auth`
 
@@ -97,25 +97,150 @@ The plugin:
 | `POST /auth/password-reset` · `/password-reset/confirm` | — | reset flow (confirm revokes all sessions) |
 | `GET  /auth/sessions` · `POST /auth/sessions/revoke` · `/sessions/revoke-others` | cookie | device list + revocation |
 | `GET  /auth/memberships` · `POST /auth/switch-tenant` | cookie | multi-tenant membership + active-tenant switch |
+| `POST /auth/verify-email` · `/verify-email/callback` | — | request/resend a confirmation link (uniform `202`); redeem one (issues **no** session) |
+| `POST /auth/invitations` · `GET /auth/invitations` · `POST /auth/invitations/revoke` | cookie | create / list / withdraw tenant invitations (needs `invitations` config) |
+| `POST /auth/invitations/preview` · `/invitations/accept` · `/invitations/sign-up` | — / cookie | inspect a token, redeem it as the signed-in user, or redeem it by creating the invited account |
+| `POST /auth/impersonate/start` · `/impersonate/stop` | cookie | act as another user and end it (needs `impersonation` config) |
 | `POST /auth/mfa/enroll/start` · `/enroll/verify` · `/unenroll` · `/recovery-codes/regenerate` | cookie | TOTP enrolment (needs `mfa: { issuer }` config); `enroll/verify` returns one-time recovery codes |
 | `POST /auth/passkey/register/options` · `/register/verify` | cookie | passkey enrolment |
 | `POST /auth/passkey/assert/options` · `/assert/verify` | — | passkey sign-in |
 
 State-changing authenticated routes require a valid `x-csrf-token` header matching the `voltro:csrf` cookie.
 
+While a session is **impersonated**, the account-security routes are refused with `403 forbidden_while_impersonating` — see [User impersonation](#user-impersonation-log-in-as).
+
 ## Schema
 
 ```ts
 import { authTables } from '@voltro/plugin-auth/schema'
 // authTables = [usersTable, sessionsTable, membershipsTable, authTokensTable,
-//               passkeysTable, recoveryCodesTable, passkeyChallengesTable]
+//               passkeysTable, recoveryCodesTable, passkeyChallengesTable,
+//               loginAttemptsTable, invitationsTable, impersonationGrantsTable]
 ```
 
 Spread `authTables` into your `database/index.ts` handle + `voltro migrate` creates the tables.
 
 ## Multi-tenant memberships + switch-tenant
 
-A user belongs to MANY tenants. The Subject carries an active `tenantId` plus its memberships (in `metadata.memberships`, read via `subjectMemberships(subject)`). `POST /auth/switch-tenant` validates the user is a member of the target tenant, re-issues the session cookie with the new active tenant, and — when invoked over the live WebSocket with a `clientId` — rebinds the connection's Subject via `bindConnectionSubject` (the dispatcher re-scopes active subscriptions on rebind).
+A user belongs to MANY tenants. The Subject carries an active `tenantId` plus its memberships (in `metadata.memberships`, read via `subjectMemberships(subject)`). `POST /auth/switch-tenant` validates the user is a member of the target tenant, re-issues the session cookie with the new active tenant, and — when invoked over the live WebSocket with a `clientId` — **presents that same cookie on the connection** via `bindConnectionCredential`.
+
+It presents a CREDENTIAL, not a Subject, and that is the whole design. The rebound connection is resolved by the full auth chain on its next call — session revocation, `auth.resolveScopes`, the scope cache, the credential-expiry bound — exactly as a reconnecting browser would be. It used to hand the middleware a resolved Subject, which the middleware returned verbatim: the connection then kept that authority for its whole life, immune to a revocation performed anywhere else. (And it only worked under `voltro dev`; `voltro serve` had no such path at all, so the rebind silently did nothing in production.)
+
+What a rebind does NOT do: re-scope subscriptions already open on that connection. They were authorized under the previous cookie and run until the client re-subscribes. Calls made after the rebind see the new tenant.
+
+## Email verification
+
+`users` carries a nullable `emailVerifiedAt`, and `POST /auth/verify-email` + `/verify-email/callback` mint, send and redeem a confirmation link over the same single-use hashed-token table the magic-link and reset flows use.
+
+**What an unverified account may do is a product decision, so it is a config field with three values** rather than a behaviour the framework picks for every app:
+
+```ts
+authRoutesPlugin({
+  // …store, secret, defaultTenantId, sendEmail…
+  emailVerification: {
+    policy: 'strict',                                    // 'off' (default) · 'soft' · 'strict'
+    exemptAccountsCreatedBefore: new Date('2026-08-11'),  // your deploy instant
+  },
+})
+```
+
+| policy | login | the session |
+|---|---|---|
+| `'off'` (default) | proceeds | carries **no** mark |
+| `'soft'` | proceeds | marked — `isEmailVerified(subject)` is `false` |
+| `'strict'` | refused, `403 email_not_verified` | never issued |
+
+**Why `'off'` is the default.** The column arrives `NULL` on every existing row. Under `'strict'` as a default, the first boot after upgrading would refuse the next login of every user you already have — a total authentication outage caused by missing data rather than by anything a user did. `exemptAccountsCreatedBefore` is the adoption seam: accounts created before that instant count as verified, so you can turn `'strict'` on without a backfill.
+
+**`isEmailVerified(subject)` is tri-state, and `null` is not `false`.** A session minted while the policy was `'off'` says nothing about verification. Reading "no mark" as "unverified" would refuse every live session at the moment you switch the policy on — the same outage one layer up. Gate on `=== false`.
+
+**Three links prove an address, not one.** A magic link and a completed password reset are inbox round-trips exactly as a verification link is, so both stamp `emailVerifiedAt` (keeping the first timestamp — the column answers "since when", not "last clicked"). Without that, `'strict'` deadlocks a magic-link-only user: they can prove their address by signing in, and are refused the sign-in for not having proved it.
+
+**A verification link is not a credential.** Redeeming one marks the address and issues **no** session. Treating it as a sign-in would turn a link that sits 24 hours in a mailbox — and in every relay along the way — into a day-long credential.
+
+`'strict'` is enforced through the [subject-guard seam](#post-authentication-subject-guards), so it covers password, MFA verify, magic-link and passkey sign-in from one wiring. `authRoutesPlugin` installs the guard when you set the policy; if you mount the handlers yourself, add `emailVerificationGuard(config)` to `subjectGuards` — setting the policy alone changes only what the session is *marked* with.
+
+The request endpoint answers a uniform `202` (unknown address, already-verified address and cooled-down resend are indistinguishable) and sends at most one mail per `resendCooldownSeconds` (default 60). Without that bound, "resend" is a mail-bomb primitive aimed at any address known to have an account.
+
+## Tenant invitations
+
+`invitations` joins `authTables`, and six routes mount when you pass an `invitations` config:
+
+```ts
+authRoutesPlugin({
+  // …store, secret, defaultTenantId, sendEmail, appBaseUrl…
+  invitations: {
+    ttlSeconds: 7 * 24 * 60 * 60,          // default
+    inviterRoles: ['owner', 'admin'],      // default
+    roleRank: ['owner', 'admin', 'member', 'viewer'],  // default, most privileged first
+    maxPending: 500,                       // default
+  },
+})
+```
+
+**An invitation is a credential that grants access to someone else's data**, so it gets the token discipline of one — 32 CSPRNG bytes, only the SHA-256 stored, single-use via one conditional `UPDATE … RETURNING`, an expiry — plus three properties a login token has no need for:
+
+- **It is addressed.** The invited address is compared against the accepting user's; a mismatch is `403 invitation_email_mismatch`. A link forwarded, leaked into a channel or intercepted is refused rather than silently granting whoever opens it first.
+- **It carries its own authority, chosen by the inviter.** The accept request is `{ token }` and nothing else — there is no field an invitee could use to name their own role.
+- **It is revocable**, tenant-scoped in the SQL predicate, so an admin of one tenant cannot withdraw another's by id.
+
+**An inviter can never grant a role above their own.** That direction is not configurable: an `admin` who can mint an `owner` invitation and accept it from a second address has promoted themselves, which makes every role boundary in the product advisory. `canGrantRole` replaces the whole rule for a model that is not a line (a matrix, a per-tenant plan) — keep it a refusal by default. A role the ranking does not know is grantable only by the top role; treating an unfamiliar `superadmin` as probably-harmless is how it gets handed out by an `admin`.
+
+Both "the invitee already has an account" cases are covered. Signed in as the invited address, `POST /auth/invitations/accept` writes the membership. Signed out with no account, `POST /auth/invitations/sign-up` creates one — with the address taken from the **invitation**, never from the request, and marked already verified, since the invitation arrived in that mailbox and came back. An existing address is answered `409 account_exists` and the invitation is not burned.
+
+Re-inviting **supersedes**: every pending invitation for the same (tenant, address) is revoked before the new one is issued, so a resend cannot leave live tokens behind. `maxPending` bounds a tenant so a compromised admin account is not a mail cannon. The admin list never returns `tokenHash` — a hash verifies a guessed plaintext offline, and an admin list is not a place to publish a verifier.
+
+`invitations` is registered with the retention sweep at 90 days (`VOLTRO_INVITATIONS_TTL_HOURS`), armed only when the feature is configured.
+
+## User impersonation ("log in as")
+
+An impersonated session that is indistinguishable from a real one does not merely lack a feature — it retroactively destroys the audit trail of the whole product. Every row an agent touches is attributed to the user, so afterwards nobody can answer *"did the customer delete this, or did we?"* — including for the incident where it matters. Everything below follows from that.
+
+```ts
+import { authRoutesPlugin, impersonationAuditRedactor } from '@voltro/plugin-auth'
+import { auditPlugin } from '@voltro/plugin-audit'
+import { Effect } from 'effect'
+
+authRoutesPlugin({
+  // …store, secret, defaultTenantId…
+  impersonation: {
+    // REQUIRED. A function, never a role — see below.
+    authority: ({ actorUser }) => Effect.succeed(supportAgentIds.has(actorUser.id)),
+    // REQUIRED. Fires for 'started', 'stopped' AND 'refused'.
+    audit: (event) => { void recordSupportEvent(event) },
+    maxDurationSeconds: 900,   // default — 15 minutes
+    requireReason: true,       // default
+  },
+})
+
+// So the mark reaches the durable audit trail too — read the note below.
+auditPlugin({ sink: 'datastore', redactSubject: impersonationAuditRedactor() })
+```
+
+**`authority` is a function, never a role, and has no default.** Two separate reasons. "A role that happens to be admin-ish" is how this becomes a privilege-escalation path in a product where `admin` means "can edit the pricing page". And mechanically: the subject these routes resolve comes from the session cookie, which [carries identity only](/docs/authentication/sessions#the-cookie-carries-identity-never-authority) — it has no `scopes` at all, so a scope-based default would be unsatisfiable by every caller, i.e. a feature that refuses everyone.
+
+**`audit` is required** because impersonation's whole risk is an unrecorded action. A config that let you enable it while leaving the destination unset would make the dangerous half optional and the safe half opt-in.
+
+**The mark lives in two places with different failure modes.** `subject.metadata.impersonation` travels with the cookie and reaches the client, so a banner needs no extra endpoint (`impersonationOf(subject)` reads it). An `impersonationGrants` **row** is written *before* the session exists — so a session can never be reachable without the record naming who is behind it — and no redaction policy can drop it.
+
+> **Wire `impersonationAuditRedactor()` or your audit rows will not say who was behind an impersonated action.** `auditPlugin`'s default is `redactSubject: 'metadata'`, which replaces the whole metadata bag — correct in general (that bag is where a per-user provider credential lands) and it takes this mark with it. The redactor keeps the mark and redacts everything else: strictly safer than `'none'`, strictly more informative than the default.
+
+**Time-bounded means the cookie expires**, not that a row says it should. The grant duration is the cookie's `Max-Age`, the session row's `expiresAt` and the grant's `expiresAt`, minted from one number. A caller may request *less*; a request for more is clamped, never obeyed.
+
+**Stopping** closes the grant, deletes the impersonated session row and drops it from the revocation cache — so a copy of that cookie taken during the grant dies immediately, not at its own expiry — then re-issues the impersonator's own, never-revoked session for its **remaining** lifetime. Restoring does not extend their login. An actor whose own session died meanwhile is signed out rather than left as somebody else.
+
+**Four escalation refusals:**
+
+| refusal | what it stops |
+|---|---|
+| `self_impersonation` | acting as yourself — every action marked impersonated with no second identity behind it, i.e. noise in the field an investigator reads |
+| `nested_impersonation` | A→B, then as B→C. The mark carries one actor, so a chain attributes C's session to B — reaching any account with a **forged** attribution |
+| `target_may_impersonate` | acting as someone who can themselves impersonate. The probe is `authority` evaluated with the identities **swapped** ("could the target impersonate me?"), so there is no second policy to keep in step |
+| `forbidden_while_impersonating` | MFA enrolment/removal, recovery-code regeneration, passkey registration, switch-tenant, revoke-other-sessions, and starting another impersonation. Without this a 15-minute grant converts to permanent access in one request: enrol a passkey as the user and the time bound is decoration |
+
+The impersonator also gains no authority the target lacks, by construction: the cookie **is** the target's identity, carries no scopes, and authority is re-resolved per request from that identity. Being impersonated does not clear the target's brute-force lockout either — a support action must not undo the protection on the account someone is hammering.
+
+`impersonationGrants` is registered with the retention sweep at 365 days (`VOLTRO_IMPERSONATION_GRANTS_TTL_HOURS`) — matching the audit log, because it answers the same class of question — armed only when the feature is configured.
 
 ## MFA / TOTP sign-in enforcement
 

package/templates/agent-docs/plugins/billing.md

@@ -42,13 +42,14 @@ export default {
 
 What it provides:
 
-- The `_voltro_billing_*` tables (customers, subscriptions, invoices, usage, flush_claims) via `extendSchema.tables`.
+- The `_voltro_billing_*` tables (customers, subscriptions, invoices, usage, flush_claims, dunning_notices) via `extendSchema.tables`.
 - The `BillingService` Context.Tag — yield it in any handler.
 - The `requireEntitlement(ctx, key, cost)` in-handler quota guard + the declarative `enforce` map.
 - A webhook receiver at `POST /billing/webhook`.
-- The rpc routes `billing.startCheckout`, `billing.portalUrl`, `billing.subscription`, `billing.reportUsage`, `billing.changePlan`, `billing.changeSeats`.
+- The rpc routes `billing.startCheckout`, `billing.portalUrl`, `billing.subscription`, `billing.entitlementStatus`, `billing.reportUsage`, `billing.previewChange`, `billing.invoices`, `billing.changePlan`, `billing.changeSeats`. Every route resolves the tenant from the caller's subject, and each declares its access decision: `billing.subscription` + `billing.entitlementStatus` are `openAccess` (the tenant-scoped reads every member's account UI renders), while the routes that change what the tenant pays — `startCheckout`, `portalUrl`, `previewChange`, `changePlan`, `changeSeats` — and the invoice history (`invoices`) require the **`billing:manage`** scope, and `reportUsage` requires **`billing:report`** (a metering credential's scope — an unguarded usage report would let any session inflate its tenant's counters). Grant the scopes via an rbac role or your auth strategy's `resolveScopes`; `admin:full` passes, as always.
 - **Seat-based billing**; proration and failed-payment retries are Stripe's.
-- The typed `BillingError` + `EntitlementExceeded` errors, merged into every procedure's wire error union.
+- **Dunning** — a past-due notification sequence, a grace period and a lockout, composed on Stripe's outcomes.
+- The typed `BillingError`, `EntitlementExceeded` + `SubscriptionLocked` errors, merged into every procedure's wire error union.
 - A browser-safe `useStartCheckout()` hook on the `/web` subpath.
 
 ## The BillingService
@@ -68,7 +69,7 @@ export default (input: { tenantId: string }, _ctx) =>
   })
 ```
 
-The subscription row lives in your DB; the provider is the source of truth and webhooks keep the row in sync. `plan()` returns `'free'` when there is no active (or trialing) subscription.
+The subscription row lives in your DB; the provider is the source of truth and webhooks keep the row in sync. `plan()` returns the plan whose **limits apply right now**: `'free'` with no subscription, the paid plan while active or trialing, and — because a bounced card should not downgrade a customer on the same second — the paid plan for the whole [grace period](#failed-payments--stripe-retries-dunning-composes-on-the-outcome) after a failed payment, falling back only once the lockout is real.
 
 ## Entitlement checks
 
@@ -142,6 +143,8 @@ billingPlugin({
 
 `onEvent` keys are the normalized `BillingEvent` tags (`subscriptionUpserted`, `invoicePaid`, `invoicePaymentFailed`, `customerLinked`, `subscriptionCanceled`), not raw Stripe types.
 
+Every event carries an `occurredAt` (Stripe's `event.created`). Webhook delivery is at-least-once **and unordered**, so the subscription and invoice rows compare it against the timestamp of the state they already reflect and **drop anything older** — a redelivered `active` from before a decline cannot un-do the past-due, and a late `payment_failed` cannot flip a paid invoice back to `open`.
+
 ## Checkout + upgrade flows
 
 The `/web` subpath ships a browser-safe hook. It imports nothing from the server module — no secret, no node-only lib ever reaches the browser bundle. Pass it the generated `billing.startCheckout` rpc binding:
@@ -196,13 +199,14 @@ yield* billing.flushUsage()
 
 ## Tables
 
-All five are `_voltro_`-prefixed and built from the cross-dialect schema DSL (no raw SQL, no pg-only types, no `TEXT` defaults). Money is `integer` minor units + a `currency` text column:
+All six are `_voltro_`-prefixed and built from the cross-dialect schema DSL (no raw SQL, no pg-only types, no `TEXT` defaults). Money is `integer` minor units + a `currency` text column:
 
 - `_voltro_billing_customers` — tenant ↔ provider customer link.
 - `_voltro_billing_subscriptions` — one subscription per tenant (plan, status, seat `quantity`, period start + end, cancel-at).
 - `_voltro_billing_invoices` — invoice history (`amountMinor` integer + `currency`).
 - `_voltro_billing_usage` — per-tenant metered counters keyed by `(tenantId, entitlementKey, period)`.
 - `_voltro_billing_flush_claims` — INSERT-wins flush-window claims (multi-instance autopilot coordination); short-lived, retention defaults to 1 hour via `VOLTRO_BILLING_FLUSH_CLAIM_TTL_HOURS`.
+- `_voltro_billing_dunning_notices` — the sent-notice ledger, `UNIQUE (tenantId, episode, stepId)`. It is the **send gate**, not a report: a notice is claimed here before it goes out, so a duplicated webhook sends nothing. Retention defaults to ~400 days via `VOLTRO_BILLING_DUNNING_TTL_HOURS` — deliberately generous, because pruning a row belonging to a still-open episode would let its notice go out a second time.
 
 `_voltro_billing_usage` is append-only — one upserted counter row per `(tenant, key, period)` — so a closed period's row would otherwise live forever. The plugin registers a retention sweep on the row's `updatedAt`: a row is only touched while its window is current, so once a period closes it ages out, while the live period's row stays fresh and survives regardless. The bound defaults to ~400 days (a conservative window with headroom for end-of-period flush + back-dated reads) and is tunable via the `VOLTRO_BILLING_USAGE_TTL_HOURS` env var; the boot retention sweep drains rows past the TTL.
 
@@ -285,9 +289,9 @@ const quote = yield* billing.previewChange(tenantId, { quantity: 40 })
 
 Never quote a locally estimated number. The one Stripe previews is the one it charges.
 
-## Failed payments — Stripe retries, you read the status
+## Failed payments — Stripe retries, dunning composes on the outcome
 
-There is no dunning subsystem here. Stripe Smart Retries runs the retry schedule (configured in the Stripe Dashboard, where it can use Stripe's own timing models) and reports the outcome as a subscription status change:
+Stripe Smart Retries runs the retry schedule (configured in the Stripe Dashboard, where it can use Stripe's own timing models) and reports the outcome as a subscription status change:
 
 | From | On | To |
 | --- | --- | --- |
@@ -295,21 +299,134 @@ There is no dunning subsystem here. Stripe Smart Retries runs the retry schedule
 | `pastDue` | payment recovers | `active` |
 | `pastDue` | Stripe gives up | `canceled` |
 
-Those transitions arrive as `customer.subscription.updated` webhooks and land on the subscription row. **Key your UI on `subscription.status`**, not on a retry record:
+The framework does not reimplement that cadence and never will — a local retry schedule ran here once, on a fixed `[1,3,5,7]`-day rhythm, and drifted from Stripe's the moment the two disagreed. What the plugin adds is the part Stripe does not do for your app: a **past-due notification sequence**, a **grace period**, and a **lockout** your entitlement checks can read.
+
+### The grace clock is a column, not a timer
+
+When the provider confirms a subscription is past due, the plugin stamps `pastDueSince` on the subscription row. Everything else is derived from it at read time — so there is no job to miss a tick, fire twice, or run on two replicas at once, and nothing "expires" a grace period in the background.
+
+Two rules keep it honest, and both exist because sending an email and locking a customer out cannot be undone:
+
+- **`pastDueSince` is only written after the provider confirms it.** A failed-payment event triggers a *reconcile* that reads the subscription's current status from Stripe (`subscriptions.retrieve`) and writes the row from that — never from the event body. Webhook delivery is at-least-once **and unordered**, so a `payment_failed` genuinely can arrive after the retry that succeeded; reconciling against the object settles it.
+- **An unreachable provider never escalates.** If the reconcile cannot reach Stripe, `pastDueSince` stays unset — and without it there is no clock to expire, so the tenant stays in grace. Failing open is the only safe direction: the alternative is locking a paying customer out because your network was down.
+
+### Asking the truthful question
+
+`entitlementStatus()` is the one answer to "is this tenant entitled right now". It is a pure read of the local row — no provider call — so it is safe on a hot path:
 
 ```ts
-const sub = yield* billing.subscription(tenantId)
-if (sub?.status === 'pastDue') {
-  // Show a "update your payment method" banner + a link to the billing portal.
-  const { url } = yield* billing.portalUrl(tenantId, returnUrl)
-}
+import { Effect } from 'effect'
+import { BillingService } from '@voltro/plugin-billing'
+
+export default (input: { tenantId: string }, _ctx) =>
+  Effect.gen(function* () {
+    const billing = yield* BillingService
+    const status  = yield* billing.entitlementStatus(input.tenantId)
+    // { plan, billedPlan, status, entitled, inGrace, graceEndsAt, lockedSince, lockout }
+    return status
+  })
 ```
 
-`pastDue` deliberately keeps the customer's entitlements: a bounced card is a payment problem, and locking an organisation out of its own data over a bank decline is a support incident, not enforcement. Stripe cancels when it has genuinely given up, and `canceled` is what removes access.
+- `entitled: false` means **dunning has locked this tenant out**. A canceled subscription is not a lockout — it is simply the free tier.
+- `inGrace: true` with a `graceEndsAt` is the "your payment failed, you have until …" state. Show the banner and a billing-portal link.
+- `plan` is the plan whose **limits apply right now**; `billedPlan` is what they are subscribed to. Under a hard lockout the two differ.
+
+The same shape is available to the browser as the `billing.entitlementStatus` rpc query (timestamps as ISO strings).
+
+For a feature with no numeric quota to degrade — an export, an admin action — guard it directly:
+
+```ts
+import { Effect } from 'effect'
+import { requireEntitled } from '@voltro/plugin-billing'
+
+export default (input: { id: string }, ctx) =>
+  Effect.gen(function* () {
+    yield* requireEntitled(ctx)   // fails SubscriptionLocked once grace expired
+    return { ok: true }
+  })
+```
+
+`SubscriptionLocked` is a typed `Schema.TaggedError` carrying `{ tenantId, status, lockedSince, lockout }`, decoded on the client like every other framework error — so the UI can route to the billing portal instead of showing a generic failure.
+
+### The sequence
+
+Steps are declared with an `afterHours` measured from `pastDueSince`, so a step cannot be pulled forward by how often the provider happens to retry. Each provider event is the tick that evaluates whichever steps have come due:
+
+```ts
+import { billingPlugin, dunningMailNotifier } from '@voltro/plugin-billing'
+import { MailService } from '@voltro/plugin-mail'
+
+billingPlugin({
+  plans: { /* … */ },
+  dunning: {
+    graceHours: 168,                       // default: 7 days
+    steps: [                               // default: exactly these three
+      { id: 'payment-failed', afterHours: 0 },
+      { id: 'reminder',       afterHours: 72 },
+      { id: 'final-warning',  afterHours: 144 },
+    ],
+    lockout: 'hard',                       // default | 'soft'
+    portalReturnUrl: 'https://acme.com/billing',
+    notify: (notice) => Effect.sync(() => { /* your transport */ }),
+  },
+})
+```
+
+Each notice carries `{ stepId, tenantId, to, plan, status, pastDueSince, graceEndsAt, locked, lockout, portalUrl }` **plus** ready-to-send default `subject` / `html` / `text` — plain and unbranded, so the sequence works the moment `notify` is wired without inviting you to ship it unchanged.
+
+**With no `notify`, nothing is sent.** The sequence claims and logs. That is the deliberate default: the framework cannot address a customer on your behalf, and an email is irreversible.
+
+To send with `@voltro/plugin-mail`, bridge it — the plugin types the mail service structurally, so it takes on no dependency:
+
+```ts
+const mail = yield* MailService
+billingPlugin({ dunning: { notify: dunningMailNotifier(mail) } })
+```
+
+The recipient is resolved as: your `dunning.resolveRecipient(tenantId)` first, then the provider's customer email. A notice with no resolvable recipient still reaches `notify` (with `to: null`) so you can route it in-product.
+
+### Idempotency, and what happens when delivery lies
+
+Every send is claimed in `_voltro_billing_dunning_notices` under a `UNIQUE (tenantId, episode, stepId)` **before** it goes out. The episode key IS the clock — the epoch-ms of `pastDueSince` — so:
+
+- a **duplicated** webhook lands on a claimed key and sends nothing;
+- a **recovery** clears `pastDueSince`, which cancels every step still pending for that episode — a later failure opens a genuinely new episode and legitimately starts over;
+- a **stale** `customer.subscription.updated` cannot un-do a past-due, and a stale `invoice.payment_failed` cannot flip a paid invoice back to `open`: both rows carry the event's timestamp and drop anything older than the state they already reflect.
+
+Claim-then-send is on purpose. Its failure mode is one notice that never arrives; send-then-claim's is a customer receiving the same dunning email twice.
+
+### The lockout
+
+Once `graceEndsAt` passes, `entitled` turns false and one `locked` notice fires on the next reconcile.
+
+- **`lockout: 'hard'`** (default) — entitlement limits fall to the `'free'` plan's. Every existing `requireEntitlement` / `enforce` check starts answering with the free tier's numbers; you write no new code.
+- **`lockout: 'soft'`** — limits stay on the paid plan and only `entitlementStatus()` reports the lockout, so your app decides what to withhold.
+
+Recovery at any point — including after the lockout — restores the paid entitlements on the next reconcile.
+
+### Tunables
+
+Every number the framework picked on your behalf is a field with a default and an env override:
+
+| Option | Env | Default |
+| --- | --- | --- |
+| `dunning.enabled` | `VOLTRO_BILLING_DUNNING` (`on` / `off`) | `true` |
+| `dunning.graceHours` | `VOLTRO_BILLING_GRACE_HOURS` | `168` (7 days) |
+| `dunning.steps[].afterHours` | `VOLTRO_BILLING_DUNNING_STEP_HOURS` (positional, comma-separated) | `0,72,144` |
+| `dunning.lockout` | `VOLTRO_BILLING_LOCKOUT` (`hard` / `soft`) | `hard` |
+| notice-ledger retention | `VOLTRO_BILLING_DUNNING_TTL_HOURS` | `9600` (~400 days) |
+
+The env parsers **fail the boot** on a value they cannot read, and `VOLTRO_BILLING_DUNNING_STEP_HOURS` refuses a list whose length differs from the declared sequence — an operator re-timing a sequence they are not looking at is exactly the quiet wrong number this refuses to become.
+
+With `dunning.enabled: false` there is no grace clock and no reconcile: `pastDue` degrades entitlements at once, which is what this plugin did before dunning existed.
+
+### The optional sweep
+
+Provider events drive the sequence, and that covers the normal case — Stripe emits an event per retry attempt. `billing.dunningSweep()` reconciles every past-due tenant in one pass, for a step configured at an hour the provider happens not to emit an event at, or a reconcile missed during a provider outage. **Nothing schedules it for you**; wire it from your own `*.cron.tsx` if you want it. It is idempotent, it never charges anything, and it never asks the provider to retry.
 
 ## Provider portability
 
-The surface (`BillingService`, the entitlement engine, the DB rows) is provider-agnostic. A `BillingProvider` is a dumb adapter: checkout/portal URL minting, usage push, and a pure `normalizeEvent` mapping the provider's payload to a `BillingEvent`. Stripe and an in-memory mock ship in the box; a new provider is a new adapter against the same contract — pass it directly:
+The surface (`BillingService`, the entitlement engine, the DB rows) is provider-agnostic. A `BillingProvider` is a dumb adapter: checkout/portal URL minting, usage push, a `fetchSubscription` direct read (the current status dunning refuses to lock a customer out without), and a pure `normalizeEvent` mapping the provider's payload to a `BillingEvent`. Stripe and an in-memory mock ship in the box; a new provider is a new adapter against the same contract — pass it directly:
 
 ```ts
 import { billingPlugin } from '@voltro/plugin-billing'

package/templates/agent-docs/plugins/cdc-out.md

@@ -44,6 +44,11 @@ export default {
       maxAttempts: 5,              // per-record delivery attempts before dead-letter
       backoffBaseMs: 200,          // first retry delay; doubles per attempt, jittered
       deliveryTimeoutMs: 10_000,   // per-attempt timeout — aborts the sink call
+      leaseTtlMs: 15_000,          // leader lease; heartbeat renews at ttl/3
+      // Fleet-scope handoff (see "Delivery guarantees"):
+      dedupWindowMs: 60_000,       // how long an enqueue claim is kept — must exceed leaseTtlMs
+      handoffBufferMs: 60_000,     // how far back each replica buffers for a takeover to drain
+      handoffBufferSize: 10_000,   // hard ceiling on buffered changes per replica
     }),
   ],
 }
@@ -73,12 +78,44 @@ via `extendSchema`, migrated by `voltro dev`). The row's TypeID id **is** the
 record's `deliveryKey` — unique across replicas, stable across restarts and
 retries.
 
-- **Exactly-once ENQUEUE, fleet-wide.** On `changeScope: 'local'` stores each
-  replica enqueues only its OWN commits (injected cross-replica events are
-  skipped). On `'fleet'` stores (postgres `changeStrategy: 'cdc'`
-  LISTEN/NOTIFY, mysql binlog) only the holder of the leader lease
-  (`_voltro_cdcout_leases`, TTL-heartbeat) enqueues. Fail closed: no lease,
-  no enqueue.
+- **Enqueue is de-duplicated per observed change, fleet-wide, across leadership
+  handovers — with one hole: a replica that dies between winning a change's
+  claim and inserting its outbox row loses that change, because the claim
+  survives and nothing rescans orphan claims.** (Those are two statements with
+  no transaction around them, which is why this does not say "exactly-once".)
+
+  On `changeScope: 'local'` stores each replica enqueues only its OWN commits
+  (injected cross-replica events are skipped), so a change is observed by
+  exactly one process and needs nothing further.
+
+  On `'fleet'` stores (postgres `changeStrategy: 'cdc'` LISTEN/NOTIFY, mysql
+  binlog) EVERY replica sees the full stream, so every replica **buffers** it
+  in memory (`handoffBufferMs`, capped at `handoffBufferSize` entries, oldest
+  dropped first). The holder of the leader lease (`_voltro_cdcout_leases`,
+  TTL-heartbeat) enqueues as it goes; a replica that WINS the lease drains the
+  window its predecessor never got to. Enqueue is **idempotent per change
+  identity**: each change is keyed by a `changeKey` every replica computes
+  alike — a digest of `(pipe, op, row id, new image, old image)` plus an
+  occurrence counter that keeps two byte-identical changes to one row apart —
+  and claimed in `_voltro_cdcout_claims` under `unique(pipe, changeKey)`. So
+  the rows the dying leader already wrote collapse instead of duplicating, and
+  the ones it never reached are written by its successor.
+
+  A replica that boots into a fleet that is **already running** adopts the
+  fleet's occurrence counters from the claims already in the database before it
+  keys anything — otherwise its first sighting of an already-claimed change
+  would key occurrence 0, collide, and be dropped as a duplicate it is not.
+  `GET /_voltro/inspect/plugins/cdc-out/sinks` reports `handoff.seeded` and
+  `handoff.awaitingSeed` so you can see that happen rather than assume it.
+
+  What still bounds it, stated plainly: a change **no surviving replica
+  observed** is gone (the transport delivered it only to the dead process); a
+  handoff that takes longer than `handoffBufferMs` loses whatever aged out of
+  the buffer, and that count is reported as `handoff.dropped` on
+  `GET /_voltro/inspect/plugins/cdc-out/sinks` rather than dropped silently.
+  `dedupWindowMs` (default `max(60_000, 4 × leaseTtlMs)`) is how long a claim
+  is kept; it **must exceed `leaseTtlMs`** — a claim that expires mid-handoff
+  is a duplicate window, and the plugin refuses to boot with one.
 - **At-least-once FROM ENQUEUE.** The tap is post-commit — a crash in the
   narrow window between commit and the outbox insert loses that one event;
   the plugin does not claim better. From enqueue on, delivery survives
@@ -92,7 +129,9 @@ retries.
   `GET /_voltro/inspect/plugins/cdc-out/dead-letter` — and unblocks the pipe.
 - **Bounded storage.** Delivered/dead rows are purged by the framework
   retention sweep after `retentionHours` (default 72, env
-  `CDCOUT_RETENTION_HOURS`); pending rows are never purged.
+  `CDCOUT_RETENTION_HOURS`); pending rows are never purged. Enqueue claims are
+  short-lived by design — purged after `dedupWindowMs`, which only has to
+  outlive a leadership handoff.
 
 ## Multi-tenancy
 

package/templates/agent-docs/plugins/clickhouse.md

@@ -61,8 +61,39 @@ on graceful shutdown.
 | `password` | `string` | — | |
 | `database` | `string` | `'default'` | The plugin creates the events table inside it on first boot. |
 | `table` | `string` | `'events'` | Override the events-table name. |
-| `mirrorTables` | `ReadonlyArray<string>` | `[]` (events-only) | Reactive tables to CDC-mirror into `voltro_mirror_<table>` (a `ReplacingMergeTree`) so analytical queries JOIN events against live user data. |
+| `mirrorTables` | `ReadonlyArray<string>` | `[]` (events-only) | Reactive tables to CDC-mirror into `voltro_mirror_<table>` (a `ReplacingMergeTree(version)` over `{ id, data, version, is_deleted }`) so analytical queries JOIN events against live user data. The version is the framework's commit-order stamp, so a change that arrives late loses the collapse. |
 | `mirrorPrimaryKey` | `string` | `'id'` | Primary-key column on the mirrored source rows. |
+| `batch` | `{ maxSize?, flushIntervalMs? } \| false` | **on** — `{ maxSize: 20, flushIntervalMs: 5000 }` | Client-side batching of `track()` inserts. `false` opts out (one immediate, confirmed insert per event). |
+
+## Batching is on by default
+
+One HTTP insert per event is how a MergeTree gets **part-exploded** under load —
+every single-row insert creates a data part the server must merge away. So
+`track()` **batches by default**: rows buffer in the process and flush in ONE
+multi-row insert once 20 events are pending, 5 seconds after the first buffered
+event (whichever comes first), and on graceful shutdown (the `dispose` hook
+drains the buffer before the client closes).
+
+What changes observably with batching on:
+
+- A successful `track()` means **"buffered"**, not "ClickHouse accepted the
+  row" — the insert happens later, off the call path.
+- Events become visible to reads up to `flushIntervalMs` later than they were
+  tracked.
+- A flush failure is logged and that batch **dropped** (best-effort analytics —
+  re-queuing a partially-applied insert would risk duplicates). A hard crash
+  loses whatever is still buffered; a graceful shutdown loses nothing.
+
+If you need per-event delivery confirmation, opt out explicitly:
+
+```ts
+analytics: clickhouseAnalytics({
+  url: process.env.CLICKHOUSE_URL!,
+  batch: false,   // one immediate insert per track(); success = row accepted
+})
+```
+
+…or tune the window: `batch: { maxSize: 500, flushIntervalMs: 2000 }`.
 
 `database`, `table`, and each `mirrorTables` entry are validated as SQL
 identifiers at boot (they're interpolated into DDL) — a bad name fails loudly at

package/templates/agent-docs/plugins/duckdb.md

@@ -53,7 +53,7 @@ connection + instance are closed on graceful shutdown.
 | Option | Type | Default | Notes |
 |---|---|---|---|
 | `path` | `string` | in-memory | A file path (`.voltro/analytics.duckdb`) → durable across restarts, single-process. Omit or pass `:memory:` → events live in process memory, lost on restart (ephemeral dev / tests). |
-| `mirrorTables` | `ReadonlyArray<string>` | `[]` (events-only) | Reactive tables to CDC-mirror into `voltro_mirror_<table>` inside DuckDB so analytical queries JOIN events against live user data. |
+| `mirrorTables` | `ReadonlyArray<string>` | `[]` (events-only) | Reactive tables to CDC-mirror into `voltro_mirror_<table>` (`{ id, data, version, is_deleted }`) inside DuckDB so analytical queries JOIN events against live user data. Deletes write a tombstone — filter `is_deleted = false`. |
 | `mirrorPrimaryKey` | `string` | `'id'` | Primary-key column on the mirrored source rows. |
 
 ```ts