@adonis-agora/durable 0.49.1 → 0.50.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@adonis-agora/durable",
-  "version": "0.49.1",
+  "version": "0.50.0",
   "description": "Durable cross-app workflows for AdonisJS — deterministic replay engine, AdonisJS binding, ace commands, dashboard, OpenTelemetry, Telescope, testing harness and Redis admission, all in one package. Part of the Agora ecosystem.",
   "license": "MIT",
   "repository": {
@@ -125,8 +125,7 @@
   ],
   "sideEffects": false,
   "dependencies": {
-    "@adonis-agora/filter": "0.11.0",
-    "zod": "4.6.7"
+    "@adonis-agora/filter": "0.11.0"
   },
   "peerDependencies": {
     "@adonisjs/assembler": "^8.4.0",
@@ -170,7 +169,7 @@
     }
   },
   "devDependencies": {
-    "@adonis-agora/durable-dashboard": "0.8.0",
+    "@adonis-agora/durable-dashboard": "0.8.1",
     "@adonisjs/assembler": "8.6.0",
     "@adonisjs/core": "7.6.0",
     "@adonisjs/lucid": "22.4.2",
@@ -183,11 +182,14 @@
     "@tanstack/intent": "0.5.6",
     "@types/better-sqlite3": "9.6.0",
     "@types/node": "26.6.5",
+    "arktype": "2.2.9",
     "better-sqlite3": "13.0.3",
     "bullmq": "6.3.17",
     "cron-parser": "5.10.2",
     "ioredis": "6.0.0",
+    "pg": "8.23.1",
     "typescript": "7.0.2",
+    "valibot": "1.5.0",
     "vitest": "5.0.5",
     "zod": "4.6.7"
   },

package/skills/durable-cluster/SKILL.md

@@ -8,13 +8,14 @@ description: >-
   (compatible/degraded/incompatible) and capability routing (requires → blocked
   runs), layered tenant auth (signTenantToken/hmacTenantVerifier/verifyTenant),
   runGateway as the role-portable read/control surface, namespace vs partition,
-  and Python (durable-worker) / NestJS interop over the bullmq wire. Use when
+  tenant pods over bullmq, the @adonisjs/queue transport or the db transport, and Python
+  (durable-worker) / NestJS interop over the bullmq wire. Use when
   splitting web from workers, isolating tenants, or adding polyglot workers.
 license: MIT
 metadata:
   type: core
   library: "@adonis-agora/durable"
-  library_version: "0.25.0"
+  library_version: "0.51.0"
   framework: adonisjs
 sources:
   - 'DavideCarvalho/adonis-durable:docs/cluster/index.mdx'
@@ -44,11 +45,11 @@ execute changes.
 Split fleet: a pure coordinator plus store-less tenant workers.
 
 ```ts title="config/durable.ts (control plane)"
-import { defineConfig, stores, transports } from '@adonis-agora/durable'
+import { defineConfig, hmacTenantVerifier, stores, transports } from '@adonis-agora/durable'
 
 export default defineConfig({
   role: 'control-plane',
-  transport: 'bullmq', // cross-ecosystem wire; queue/db work for Adonis-only splits
+  transport: 'bullmq', // or 'queue' / 'db' for Adonis-only fleets
   transports: { bullmq: transports.bullmq({ connection: { host: '127.0.0.1', port: 6379 } }) },
   store: 'lucid',
   stores: { lucid: stores.lucid({ connection: 'pg' }) },
@@ -79,6 +80,41 @@ Isolation is three-layered: type (`store` is `never` on tenant), container (no
 store binding registered), object (`WorkerRuntime`/`ProxyRunGateway` have no
 store field).
 
+Tenant pods need a transport carrying the run-request protocol: `bullmq`,
+`queue` or `db` (the provider feature-detects the methods). Both sides use the
+same transport and backend. On `queue`, replies go to a per-pod
+`<prefix>:run-replies:<instanceId>` queue and live-tail events are pushed to
+each subscribed pod's `<prefix>:tenant-events:<instanceId>`; the pod announces
+its subscriptions and (with `workerKeys: false`) its worker advertisement to the
+control plane, which records them in its own key store — so the pod needs no
+database:
+
+```ts title="config/durable.ts (tenant pod, queue transport)"
+export default defineConfig({
+  role: 'tenant',
+  transport: 'queue',
+  transports: { queue: transports.queue({ connection: 'redis', workerKeys: false }) },
+  partition: 'acme-corp',
+  tenant: { token: process.env.DURABLE_TENANT_TOKEN },
+})
+```
+
+Queue-transport limits vs bullmq: replies/events are polled (not pushed), a new
+subscription can miss events for about a second, each event costs one job per
+subscribed pod, and live-tail needs a worker-key store on the control plane.
+Source: `docs/transports/queue.mdx` (tenant pods over the queue transport).
+
+On `db`, the pod points `transports.db()` at the control plane's database (it
+still owns no durable store). Requests, starts, replies, decisions and events
+are rows of `durable_transport_messages`; a reply or event is a row addressed to
+one instance, and the pod writes its own `-inbox:` / `-tenant-subscriber:` keys
+and worker advertisement to `durable_transport_worker_keys`. Remote workflow
+turns are task rows of kind `workflow`, and each decision is addressed to the
+engine replica that dispatched the turn (any replica could apply it: the engine
+applies a decision by run id). Needs the
+`create_durable_transport_messages` migration (or `autoCreate`). Source:
+`docs/transports/db.mdx` (remote workflows and tenant pods).
+
 ## Core patterns
 
 ### Pattern 1 — role-portable code via `runGateway`
@@ -107,16 +143,22 @@ export default class RunsController {
 Verbs: `getRun`, `listRuns`, `getCheckpoints`, `getRunChildren`,
 `getRunDetail` / `getFanMembers` (optional; summarize and page wide fans),
 `getSearchAttributes`, `start`, `signal`, `cancel`, `redispatchPending`,
-`workerHealth`, `subscribe`, `topology()`. Never branch on `role`.
+`workerHealth`, `subscribe`, `topology()`, and the NestJS gateway's `retry`
+(in-place requeue), `continue` (breakpoint), `retryWithInput`, `runFacets`,
+`runValueFacets`, `waitingFor` (optional on the interface; both built-in
+gateways implement them, tenant-scoped by the responder). Never branch on `role`.
 
 ```ts
 // Each fan of more than 50 members — or declared wider than 50 — comes back as one FanSummary in `fans`.
 const detail = await runGateway.getRunDetail?.(params.id, { summarizeFansOver: 50 })
 ```
 
-A tenant pod's dashboard reads run detail through the `getRunDetail` verb, and
-an older control plane rejects unknown verbs: upgrade the control plane first,
-or with its tenants.
+Tenants and the control plane upgrade in either order. A verb the control plane
+does not know comes back as `unsupported-request` (or, before 0.50, an
+`unknown run-request verb` error) and the proxy falls back at once: run detail
+is built from `getRun` + `getCheckpoints` and summarized on the tenant, and the
+tenant console degrades `retry` to a re-dispatch; it remembers the old control
+plane for a minute. Every request gets a reply, even when the gateway throws.
 Source: `docs/cluster/roles-and-config.mdx`.
 
 ### Pattern 2 — capability-aware routing and blocked runs
@@ -147,10 +189,11 @@ imports zero store code:
 
 ```ts title="worker.ts"
 import { RedisWorkerRegistry, WorkerRuntime } from '@adonis-agora/durable/worker'
-import { transports } from '@adonis-agora/durable'
+import { BullMQTransport, createBullMQDeps } from '@adonis-agora/durable'
 import { Redis } from 'ioredis'
 
-const transport = await transports.bullmq({ connection: process.env.REDIS_URL })({ app })
+// No Adonis app here, so build the transport directly instead of through `transports.bullmq()`.
+const transport = new BullMQTransport({ deps: await createBullMQDeps(process.env.REDIS_URL) })
 const runtime = new WorkerRuntime({
   transport,
   partition: 'acme-corp',
@@ -294,6 +337,28 @@ the queues the deployed fleet consumes — so the local process steals productio
 tasks; docs call this out under Tenancy pitfalls ("competes for, and steals, the
 deployed fleet's tasks"). Source: `docs/concepts/tenancy.mdx`.
 
+### HIGH a db-transport tenant pod on a database of its own
+
+Wrong:
+
+```ts
+// the pod's default Lucid connection is its own app database
+defineConfig({ role: 'tenant', transport: 'db', transports: { db: transports.db() }, partition: 'acme' })
+```
+
+Correct:
+
+```ts
+// a connection to the CONTROL PLANE's database (config/database.ts), used only for the transport tables
+defineConfig({ role: 'tenant', transport: 'db', transports: { db: transports.db({ connection: 'durable' }) }, partition: 'acme' })
+```
+
+Mechanism: on `db` the wire IS the transport tables, so a pod writing its
+requests to another database is never answered — every `runGateway` call times
+out after `requestTimeoutMs` and no step reaches it. Pods that cannot reach the
+control plane's database need `queue` or `bullmq`. Source:
+`docs/transports/db.mdx` (remote workflows and tenant pods).
+
 ### MEDIUM stamping runs into a namespace nothing polls
 
 Wrong:
@@ -311,7 +376,11 @@ await engine.start('checkout', order, id, { namespace: 'eu-west' })
 
 Mechanism: between namespaced engines there is no fallback owner; a namespace
 with neither an engine of its own nor an operator above it is a dead end where
-runs sit `pending` forever. Source: `docs/concepts/tenancy.mdx` (pitfalls).
+runs sit `pending` forever. Since 0.51 only a `control-plane` with no
+`namespace` is the operator; a `standalone` engine with no `namespace` is the
+`'default'` pool. Before 0.51 an unset namespace made ANY engine an operator,
+so a standalone app sharing a store drove every pool's runs. Source:
+`docs/concepts/tenancy.mdx` (pitfalls).
 
 See also: `durable-setup` — the standalone execution model this scales out of.
 See also: `durable-transports-stores` — bullmq/queue/db driver details,

package/skills/durable-determinism/SKILL.md

@@ -6,14 +6,15 @@ description: >-
   checkpointed ctx.now() and ctx.sideEffect() capture sources,
   NonDeterminismError at replay time, workflow versions for breaking shape
   changes (skew protection), ctx.patched for surgical in-place guards, and the
-  @adonis-agora/durable-eslint-plugin no-nondeterminism rule with its
-  recommended preset. Use when authoring or changing workflow code that has
+  @adonis-agora/durable-eslint-plugin rules (no-nondeterminism,
+  rethrow-control-flow-signals, no-io-in-workflow-body) and its recommended
+  preset. Use when authoring or changing workflow code that has
   runs in flight, fixing NonDeterminismError, or wiring determinism linting.
 license: MIT
 metadata:
   type: core
   library: "@adonis-agora/durable"
-  library_version: "0.25.0"
+  library_version: "0.51.0"
   framework: adonisjs
 sources:
   - 'DavideCarvalho/adonis-durable:docs/concepts/durability.mdx'
@@ -75,15 +76,20 @@ export default [
     plugins: { '@adonis-agora/durable': durable },
     rules: {
       '@adonis-agora/durable/no-nondeterminism': 'error',
+      '@adonis-agora/durable/rethrow-control-flow-signals': 'error',
+      '@adonis-agora/durable/no-io-in-workflow-body': 'error',
     },
   },
 ]
 
-// or simply:
+// or simply (enables all three):
 // export default [durable.configs.recommended]
 ```
 
-Each report names its replacement ("use `ctx.now()`"). The plugin is the cheap
+`rethrow-control-flow-signals` flags a `catch` around an awaited ctx op that
+does not rethrow `isWorkflowControlFlowSignal(error)`; `no-io-in-workflow-body`
+flags a global `fetch()` and raw `engine.*` calls in the body. Each report names
+its replacement ("use `ctx.now()`"). The plugin is the cheap
 early layer; `NonDeterminismError` is the replay-time backstop for calls it
 cannot see (transitive helpers). Source: `docs/tooling/linting.mdx`.
 

package/skills/durable-reliability/SKILL.md

@@ -5,23 +5,25 @@ description: >-
   backoff and jitter, FatalError and worker retryable:false, timeoutMs liveness
   vs the durable suspend path, pickupTimeoutMs, log.heartbeat, saga
   compensation (compensate callbacks/refs, UndoOf, compensationRetries,
-  cancel({compensate:true})), flow-control queues (registerQueue, concurrency,
+  cancel({compensate:true})), flow-control queues (config queues / registerQueue, concurrency,
   rateLimit, fairnessKey) with Redis-backed global admission (admission-redis),
-  the dead-letter queue (maxRecoveryAttempts, onDead, retryWithInput), the
+  the dead-letter queue (maxRecoveryAttempts, deadLetterWorkflow, deadLetter()
+  method, onDead, retryWithInput), retention (maxAge/maxCount policies), the
   stuck-run playbook (stranded signature, redispatchPending,
-  remoteRedispatchMs), ace ops commands (durable:work/runs/retry --stale), and
+  remoteRedispatchMs, remoteAdvanceSilenceMs for remote workflow turns), ace ops commands (durable:work/runs/retry --stale), and
   the testing harness (createTestEngine, tick, failOnce, conformance kits).
 license: MIT
 metadata:
   type: core
   library: "@adonis-agora/durable"
-  library_version: "0.25.0"
+  library_version: "0.51.0"
   framework: adonisjs
 sources:
   - 'DavideCarvalho/adonis-durable:docs/reliability/retries.mdx'
   - 'DavideCarvalho/adonis-durable:docs/reliability/sagas.mdx'
   - 'DavideCarvalho/adonis-durable:docs/reliability/dead-letter.mdx'
   - 'DavideCarvalho/adonis-durable:docs/reliability/flow-control.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/reliability/retention.mdx'
   - 'DavideCarvalho/adonis-durable:docs/reliability/failure-modes.mdx'
   - 'DavideCarvalho/adonis-durable:docs/cli.mdx'
   - 'DavideCarvalho/adonis-durable:docs/testing.mdx'
@@ -40,7 +42,7 @@ Retries are declared per step; `FatalError` opts a deterministic business
 failure out of the retry path:
 
 ```ts title="app/steps/charge.ts"
-import { defineStep } from '@adonis-agora/durable'
+import { FatalError, defineStep } from '@adonis-agora/durable'
 
 export const chargeCard = defineStep(
   'payments:charge-card',
@@ -48,7 +50,7 @@ export const chargeCard = defineStep(
     const res = await stripe.charge(input)
     if (res.declined) {
       // deterministic verdict — don't make the engine retry it
-      throw Object.assign(new Error('card declined'), { code: 'declined', retryable: false })
+      throw new FatalError('card declined', 'declined')
     }
     return { chargeId: res.id }
   },
@@ -56,7 +58,9 @@ export const chargeCard = defineStep(
 )
 ```
 
-A dispatched step WITHOUT `timeoutMs` takes the durable path: failure → persist
+`FatalError` carries `retryable: false`, which the step result builders read; a
+worker in another runtime gets the same effect by throwing any error with
+`retryable: false` (and an optional `code`). A dispatched step WITHOUT `timeoutMs` takes the durable path: failure → persist
 `wakeAt = now + backoff(attempt)` on the checkpoint → suspend → the timer poller
 re-dispatches. Crash-safe, never held in memory.
 
@@ -90,16 +94,20 @@ envelope (`{ input, output }`). Engine-level `compensationRetries` (default 1)
 bounds transient undo failures; failing compensations are skipped so they can't
 mask the original error. Trigger deliberately with
 `await engine.cancel(runId, { compensate: true })` or
-`engine.cancelWhere(filter, { compensate: true })`. Every undo surfaces as a
-visible `compensate:<step>` event. Source: `docs/reliability/sagas.mdx`.
+`engine.cancelWhere(filter, { compensate: true })`. The cancel is durable (a
+`cancel:<runId>` marker) and honored on every exit of the turn running when it
+lands: return, throw, continue-as-new or suspend all compensate and settle
+`cancelled`. Every undo surfaces as a visible `compensate:<step>` event.
+Source: `docs/reliability/sagas.mdx`.
 
 ### Pattern 2 — flow-control queues + fleet-wide admission
 
 Cap concurrency/rate durably — a blocked call re-suspends with a persisted
 `retryAt`, driven by the timer poller (nothing waits in memory):
 
-```ts title="start/durable.ts"
-engine.registerQueue({ name: 'emails', concurrency: 10, rateLimit: { limit: 1000, periodMs: 3_600_000 } })
+```ts title="config/durable.ts"
+queues: [{ name: 'emails', concurrency: 10, rateLimit: { limit: 1000, periodMs: 3_600_000 } }],
+// or at runtime: engine.registerQueue({ name: 'emails', … }) — same shape, replaces a same-named one
 ```
 
 ```ts
@@ -130,7 +138,12 @@ Source: `docs/reliability/flow-control.mdx`.
 
 Crash-recovery reclaims orphaned runs within ~`leaseMs`; a run that crashes the
 process every time would loop forever. Cap it and route the terminal `dead`
-status into a handler workflow (idempotent by the `dlq:<runId>` start id):
+status into a handler workflow — the engine starts it as `dlq:<runId>`
+(idempotent) with a `DeadLetter` input `{ deadRunId, workflow, input, error }`
+in the dead run's namespace. A run
+whose remote step is still in flight is parked `suspended`, never counted, and a
+retry (`requeue` / `durable:retry`) of a `dead`/`failed` run resets
+`recoveryAttempts` to 0:
 
 ```ts title="config/durable.ts"
 export default defineConfig({
@@ -139,20 +152,40 @@ export default defineConfig({
 })
 ```
 
-```ts title="start/durable.ts"
-engine.onDead((run) => {
-  void engine.start(
-    'dlq',
-    { deadRunId: run.id, workflow: run.workflow, input: run.input, error: run.error },
-    `dlq:${run.id}`,
-  )
-})
+```ts title="app/workflows/pipeline_workflow.ts"
+export default class PipelineWorkflow extends BaseWorkflow {
+  static workflow = { name: 'pipeline', version: '1' }
+  async run(ctx: WorkflowCtx, input: PipelineInput) { /* … */ }
+  // registered as `pipeline.dlq`; shares this instance's injected services
+  async deadLetter(ctx: WorkflowCtx, dl: DeadLetter<PipelineInput>) { /* alert, ticket */ }
+}
 ```
 
+Precedence: inline `deadLetter()` > per-workflow `deadLetterWorkflow`
+(`static workflow` / `register` / `registerRemote` option) > config
+`deadLetterWorkflow` (fleet default). Declaring both a method and the option
+throws. A `dlq:` handler run that dies is not routed again; a failed start
+(handler not registered) is `console.warn`-ed. `engine.onDead(run => …)` still
+fires first and is the manual escape hatch (keep the `dlq:<runId>` id).
+
 A dead run stays inspectable and retriable (`durable:retry <runId>`, dashboard
 retry, or `engine.retryWithInput(runId, fixedInput)` for bad data).
 Source: `docs/reliability/dead-letter.mdx`.
 
+Retention bounds terminal history (swept by `durable:work`, once a minute,
+`onEvict` archives first). Scoped policies take `maxAge`, `maxCount` (keep the N
+newest by start time within the scope; `0` deletes all matching), or both (most
+restrictive wins); a policy with neither fails boot:
+
+```ts title="config/durable.ts"
+retention: {
+  completed: '30d',
+  policies: [{ statuses: ['completed', 'failed'], maxCount: 200, scope: { namespace: 'acme' } }],
+},
+```
+
+Source: `docs/reliability/retention.mdx`.
+
 ### Pattern 4 — the stuck-run playbook
 
 A lost remote dispatch does **not** self-heal by default: the run cycles
@@ -166,22 +199,14 @@ node ace durable:retry checkout:1234
 ```
 
 ```ts title="app/controllers/ops_controller.ts"
-import { inject } from '@adonisjs/core'
-import {
-  DEFAULT_STALE_MS,
-  WorkflowEngine,
-  attachLiveness,
-  filterStale,
-  listRuns,
-} from '@adonis-agora/durable'
+import type { HttpContext } from '@adonisjs/core/http'
+import { DEFAULT_STALE_MS, attachLiveness, filterStale, listRuns } from '@adonis-agora/durable'
+import engine from '@adonis-agora/durable/services/main'
 
-@inject()
 export default class OpsController {
-  constructor(private engine: WorkflowEngine) {}
-
   async strandedRuns({ response }: HttpContext) {
-    const runs = await listRuns(this.engine, { statuses: ['running', 'suspended'], limit: 200 })
-    const stale = filterStale(await attachLiveness(this.engine, runs), DEFAULT_STALE_MS)
+    const runs = await listRuns(engine, { statuses: ['running', 'suspended'], limit: 200 })
+    const stale = filterStale(await attachLiveness(engine, runs), DEFAULT_STALE_MS)
     return response.ok({ stranded: stale.map(({ run }) => run.id) })
   }
 }
@@ -198,6 +223,20 @@ Three nets, in preference order:
 3. `engine.redispatchPending(runId)` — the operator escape hatch, safe on a
    healthy run (only touches already-pending checkpoints).
 
+Remote (polyglot) workflow TURNS have their own net: `remoteAdvanceSilenceMs`
+(config; `RemoteWorkflowExecutor`'s `timeoutMs` per workflow, shorter wins). The
+engine records the turn's random `taskId` on the run (`awaitingDecisionTaskId`),
+releases the lease, then enqueues; whichever replica consumes the decision
+applies it by run id (stale/duplicate decisions dropped), so a point-to-point
+`<P>-decisions` queue across replicas is safe. That is 0.51+: before 0.51 a
+replica that had not dispatched the turn consumed the decision, found no waiter
+and dropped it, leaving the run leased forever — upgrade any multi-replica fleet
+running remote (Python/NestJS) workflows. A turn with no decision and no
+run-scoped heartbeat (`{ runId, seq: 0, group }`, sent every 5s by the Python
+and TS workers) for the window is re-dispatched by the timer poller — never
+failed. Unset = wait for the decision forever. Source:
+`docs/reliability/multi-instance.mdx` (remote workflow turns).
+
 Source: `docs/reliability/failure-modes.mdx`, `docs/cli.mdx`.
 
 ### Pattern 5 — test workflows without infrastructure
@@ -282,7 +321,7 @@ Correct:
 
 ```ts
 const inFlight = counts['run.started'] - (counts['run.completed'] + counts['run.failed'])
-// or read the store: listRuns({ status: ['running', 'suspended'] })
+// or read the store: listRuns(engine, { statuses: ['running', 'suspended'] })
 ```
 
 Mechanism: `run.suspended` fires on EVERY park — each sleep, signal wait, child
@@ -296,7 +335,7 @@ Source: `docs/observability/events-and-interceptors.mdx`.
 Wrong:
 
 ```ts
-const stalled = (await engine.workerHealth()).some((g) => g.stalled) // always false on queue/db
+const stalled = (await engine.workerHealth()).some((g) => g.stalled) // always false where no group health is reported (eventEmitter, queue with workerKeys: false)
 ```
 
 Correct:
@@ -308,9 +347,10 @@ const stale = filterStale(
 )
 ```
 
-Mechanism: group health is BullMQ-only — on `queue`, `db` or `eventEmitter`,
+Mechanism: only `bullmq`, `queue` (with a worker-key store) and `db` report group
+health — on `eventEmitter`, or `queue` with `workerKeys: false`,
 `workerHealth()` returns `[]`, not an error, so "no stalled groups reported" says
-nothing. Use the stranded-signature query instead. Source:
+nothing. On `queue`, `depth` counts pending jobs only; on `db`, pending + claimed rows. Use the stranded-signature query instead. Source:
 `docs/reliability/failure-modes.mdx`.
 
 ### MEDIUM writing non-idempotent saga compensations

package/skills/durable-setup/SKILL.md

@@ -6,7 +6,7 @@ description: >-
   (memory, event-emitter, queue, db, bullmq; lucid store), the WorkflowEngine
   container singleton from @adonis-agora/durable/services/main,
   Workflow.dispatch vs .start, where runs execute (in-process dispatcher,
-  NOOP_RUN_DISPATCHER, durable:work), consumers, leaseMs. Use when setting up
+  drive: false, durable:work), consumers, leaseMs, shutdown drain. Use when setting up
   durable workflows for the first time, moving from dev defaults to production
   infrastructure, or diagnosing "runs vanish on restart" / "workflows compete
   with HTTP traffic".
@@ -14,7 +14,7 @@ license: MIT
 metadata:
   type: core
   library: "@adonis-agora/durable"
-  library_version: "0.25.0"
+  library_version: "0.51.0"
   framework: adonisjs
 sources:
   - 'DavideCarvalho/adonis-durable:docs/getting-started.mdx'
@@ -34,15 +34,20 @@ default is for development only.
 
 ## Setup
 
-Install via ace (registers `@adonis-agora/durable/durable_provider` in
-`adonisrc.ts` and publishes `config/durable.ts`):
+Install via ace. The `configure` hook registers the durable and dashboard
+providers, the Ace commands and the `app/workflows` / `app/steps` codegen hooks
+in `adonisrc.ts`, adds the `#workflows/*` / `#steps/*` subpath imports to
+`package.json`, and publishes `config/durable.ts`, `config/durable_dashboard.ts`
+and the Lucid migrations:
 
 ```bash
 node ace add @adonis-agora/durable
 ```
 
 The provider binds a singleton `WorkflowEngine` built from the config. Import it
-anywhere with `import engine from '@adonis-agora/durable/services/main'`.
+anywhere with `import engine from '@adonis-agora/durable/services/main'` — never
+`app.container.make(WorkflowEngine)` in app code. On a store-less `tenant` pod
+the engine is `undefined`; use the same module's `runGateway` export instead.
 
 ```ts title="config/durable.ts"
 import { defineConfig, stores, transports } from '@adonis-agora/durable'
@@ -120,49 +125,69 @@ export default defineConfig({
 })
 ```
 
-Run `node ace durable:work` alongside (or rely on the embedded worker) so
-recovery, timers and schedules are driven even if you later split processes.
+Run `node ace durable:work` alongside (or set `worker: { embedded: true }` to
+run the same loop inside the web process) so recovery, timers and schedules are
+driven even if you later split processes.
 Source: `docs/transports/index.mdx`.
 
 ### Pattern 2 — split web from workers
 
 `engine.start()` never blocks the caller, but by default the body executes on a
 microtask **on the calling instance**. On an API/dashboard pod that means heavy
-runs compete with HTTP traffic and the DB pool. Pass `NOOP_RUN_DISPATCHER` so
-the API pod only creates runs, and let `durable:work` execute them:
+runs compete with HTTP traffic and the DB pool. Set `drive: false` (with
+`consumers: 'never'`) so the API pod only creates runs, and let `durable:work`
+execute them:
 
 ```ts title="config/durable.ts (API/dashboard pod)"
-import { NOOP_RUN_DISPATCHER, defineConfig, stores, transports } from '@adonis-agora/durable'
+import { defineConfig, stores, transports } from '@adonis-agora/durable'
 
 export default defineConfig({
   transport: 'queue',
   transports: { queue: transports.queue({ connection: 'redis' }) },
   store: 'lucid',
   stores: { lucid: stores.lucid() },
-  runDispatcher: NOOP_RUN_DISPATCHER, // create runs; never execute bodies here
+  drive: false, // create runs; never execute bodies here
+  consumers: 'never', // claim no broker jobs either
 })
 ```
 
+`durable:work` drives through its poll whatever `drive` says, so one config file
+can serve both pods (runs a worker pod starts itself, e.g. children, are picked
+up on its next tick rather than inline). A resume handled on the API pod (an
+`engine.signal` or webhook) still executes there under the run's lease. An
+explicit `runDispatcher` wins over `drive`; `drive: false` plus
+`worker: { embedded: true }` throws at boot. Every store-backed process drains
+in-flight runs on shutdown (`worker.drainTimeoutMs`, else `drainTimeoutMs`,
+default 10s) before the transport closes; no `app.terminating` hook
+needed.
+
 ```bash title="a separate worker process"
-node ace durable:work --interval=500 --drainTimeout=15000
+node ace durable:work --interval=500 --drain-timeout=15000
 ```
 
 Each `durable:work` tick runs `engine.runPending()` → `recoverIncomplete()` →
 `resumeDueTimers()` → `sweepTimeouts()` → fires due schedules, then drains on
-`SIGINT`/`SIGTERM`. Source: `docs/concepts/durability.mdx`,
+`SIGINT`/`SIGTERM`. `--drain-timeout` bounds the whole shutdown: the wait for
+the tick in flight is deducted from the drain. The recovery fallbacks are
+`config/durable.ts` fields: `reconcileMs` (lost-wake re-drive, default 300000,
+`0` off) and `blockedPollMs` (blocked-run re-check, default 5000). Source: `docs/concepts/durability.mdx`,
 `docs/reliability/failure-modes.mdx`, `docs/cli.mdx`.
 
 ### Pattern 3 — dispatch vs start vs waitForRun
 
 One rule: `.dispatch(input)` = fire-and-forget (`{ runId }`); `.start(input)` =
-block until the run settles (terminal state or suspended) and get the result;
-`engine.waitForRun(runId)` resolves a previously started run.
+block until the run is **terminal** (it waits through sleeps/signals) and get its
+output; `engine.waitForRun(runId)` resolves a previously started run once it
+**settles** (terminal or suspended — except a `suspended` run still awaiting a
+remote workflow turn's decision, which keeps waiting until the decision lands) —
+pass `{ terminal: true }` to wait for the end.
 
 ```ts
 const { runId } = await OrderWorkflow.dispatch(order) // hot path — respond now
-const result = await OrderWorkflow.start(order) // scripts / short runs only
+const output = await OrderWorkflow.start(order) // scripts / short runs only
 await engine.start('order', order, runId)
-const settled = await engine.waitForRun(runId) // resolves on settle
+const settled = await engine.waitForRun(runId) // RunResult, resolves on settle
+const done = await engine.waitForRun(runId, { terminal: true, timeoutMs: 60_000 })
 ```
 
 `.start` blocks the calling context — avoid it on a hot HTTP path for
@@ -249,12 +274,13 @@ export default defineConfig({ transport: 'queue', transports: { /* … */ } })
 Correct:
 
 ```ts
-import { NOOP_RUN_DISPATCHER, defineConfig } from '@adonis-agora/durable'
+import { defineConfig } from '@adonis-agora/durable'
 
 export default defineConfig({
   transport: 'queue',
   transports: { /* … */ },
-  runDispatcher: NOOP_RUN_DISPATCHER,
+  drive: false,
+  consumers: 'never',
 })
 ```
 
@@ -284,9 +310,12 @@ export default defineConfig({
 ```
 
 —or don't round-trip inline: start the run, exit, let the worker fleet finish
-it. Mechanism: console/repl processes deliberately do not start broker consumer
-loops (they would claim jobs and die holding them), so nothing receives results
-in that process unless `consumers: 'always'`.
+it. Mechanism: with `consumers: 'auto'` (the default) console/repl processes
+deliberately do not start broker consumer loops (they would claim jobs and die
+holding them), so nothing receives results in that process unless
+`consumers: 'always'`. Before 0.50 the bullmq and db transports ignored this and
+a one-off `node ace` command consumed jobs; upgrade if you see that.
+`consumers: 'never'` makes a web fleet a pure producer.
 Source: `docs/transports/queue.mdx` ("consumers").
 
 See also: `durable-transports-stores` — picking drivers, namespaces, migrations.