@adonis-agora/durable 0.24.3 → 0.25.1

package/skills/durable-transports-stores/SKILL.md

@@ -0,0 +1,296 @@
+---
+name: durable-transports-stores
+description: >-
+  Wire @adonis-agora/durable drivers in config/durable.ts: transports
+  (memory/eventEmitter/queue over @adonisjs/queue, broker-less db over Lucid,
+  bullmq for cross-runtime), per-step transport pinning and failover pools,
+  controlPlanes.redis broadcast channel, the lucid StateStore (published
+  migration delegating to createDurableTables, autoSchema, repair warning,
+  updateRunIf compare-and-set), CodecStateStore payload encryption with a
+  PayloadCodec, and namespace vs partition isolation. Use when moving steps
+  across processes, persisting runs durably, sharing one database/broker
+  between pools, or encrypting payloads at rest.
+license: MIT
+metadata:
+  type: core
+  library: "@adonis-agora/durable"
+  library_version: "0.25.0"
+  framework: adonisjs
+sources:
+  - 'DavideCarvalho/adonis-durable:docs/transports/index.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/transports/queue.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/transports/db.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/transports/control-plane.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/stores/index.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/stores/lucid.mdx'
+---
+
+# Transports & state stores
+
+A transport carries a dispatched `ctx.step` to its handler and the result back;
+a state store persists runs and checkpoints. They are independent pluggable,
+config-selected drivers — mix any store with any transport. Drivers are named
+from the `transports`/`stores` maps and lazily import their optional peer only
+when selected.
+
+## Setup
+
+```bash
+node ace configure @adonis-agora/durable   # publishes config + migrations
+node ace migration:run                     # durable tables (+ db-transport tables)
+```
+
+```ts title="config/durable.ts"
+import { defineConfig, stores, transports } from '@adonis-agora/durable'
+
+export default defineConfig({
+  transport: 'queue',
+  transports: {
+    memory: transports.memory(),
+    'event-emitter': transports.eventEmitter(),
+    queue: transports.queue({ connection: 'redis' }), // names a config/queue.ts adapter
+    db: transports.db(), // pass { connection } for a non-default Lucid connection
+  },
+  store: 'lucid',
+  stores: { lucid: stores.lucid() },
+})
+```
+
+Switching transports is a one-line change; workflow code never changes.
+
+## Core patterns
+
+### Pattern 1 — cross-process steps without new infrastructure (`db`)
+
+The `db` transport makes dispatched steps rows in the database you already run —
+no Redis, no broker. Workers claim rows with an atomic portable lease (no
+`FOR UPDATE SKIP LOCKED` needed):
+
+```ts title="start/worker.ts (a separate worker process)"
+import db from '@adonisjs/lucid/services/db'
+import { DbTransport } from '@adonis-agora/durable'
+
+const transport = new DbTransport({ db }) // { partition } isolates this pool
+transport.handle('pipeline:extract', async (input) => extract(input))
+```
+
+Trade-off: throughput is bounded by polling + row contention — right for
+workflow-scale modest rates, not high-fanout firehoses. Delivery is
+at-least-once (crash between insert-result and delete-task), so handlers should
+be idempotent on `step_id`. Source: `docs/transports/db.mdx`.
+
+### Pattern 2 — real brokers (`queue`, `bullmq`) and stalled-claim tuning
+
+The `queue` transport rides any `@adonisjs/queue` adapter defined once in
+`config/queue.ts`; `bullmq` is the cross-ecosystem wire shared byte-for-byte
+with NestJS engines and Python workers:
+
+```ts title="config/durable.ts"
+export default defineConfig({
+  transports: {
+    queue: transports.queue({
+      connection: 'redis',
+      // A claim is NEVER renewed while a worker processes it — its age is the
+      // step's elapsed time. Raise the threshold above your longest step:
+      stalledThresholdMs: 2 * 60 * 60 * 1000, // default 30min
+      maxStalledCount: 3, // bounds a poison job
+      onError: (err) => logger.error({ err }, 'durable queue transport'),
+    }),
+  },
+})
+```
+
+One instance consumes engine-side results; run one or more `node ace
+durable:work` workers serving the step names. Source:
+`docs/transports/queue.mdx`.
+
+### Pattern 3 — multi-replica fan-out: the control plane
+
+The task transport is point-to-point; lifecycle events and cancellation need a
+broadcast channel across every replica. Omit it and the engine is local-only
+(correct for single-instance):
+
+```ts title="config/durable.ts"
+import { controlPlanes, defineConfig } from '@adonis-agora/durable'
+
+export default defineConfig({
+  // …
+  controlPlane: controlPlanes.redis({ connection: 'main' }), // config/redis.ts connection
+})
+```
+
+Channel is `${prefix}-control` (default prefix `durable`) — wire-compatible with
+the NestJS fleet's channel, so an AdonisJS and NestJS deployment sharing one
+Redis fan out across both runtimes. Source: `docs/transports/control-plane.mdx`.
+
+### Pattern 4 — schema ownership and encryption at rest
+
+The migration **delegates** to the library instead of snapshotting DDL, so the
+schema cannot drift from what the installed version writes:
+
+```ts title="database/migrations/…_create_durable_tables.ts"
+import { BaseSchema } from '@adonisjs/lucid/schema'
+import db from '@adonisjs/lucid/services/db'
+import { createDurableTables, dropDurableTables } from '@adonis-agora/durable'
+
+export default class extends BaseSchema {
+  static disableTransactions = true
+
+  async up() {
+    await createDurableTables(db, this.db.connectionName)
+  }
+
+  async down() {
+    await dropDurableTables(db, this.db.connectionName)
+  }
+}
+```
+
+By default the provider also provisions idempotently at boot (`ensureSchema`);
+turn that off to own every change through reviewed migrations:
+
+```ts
+export default defineConfig({ /* … */ autoSchema: false })
+// then `node ace migration:run` after EVERY package upgrade
+```
+
+Encrypt run/step/signal/event payloads by decorating any store with a codec
+(searchable metadata and structured error stay plaintext):
+
+```ts title="config/durable.ts"
+import { CodecStateStore, LucidStateStore, defineConfig } from '@adonis-agora/durable'
+import type { PayloadCodec } from '@adonis-agora/durable'
+import db from '@adonisjs/lucid/services/db'
+
+const aesCodec: PayloadCodec = {
+  encode(value) {
+    const body = Buffer.from(JSON.stringify(value), 'utf8').toString('base64')
+    return { body }
+  },
+  decode(value) {
+    return JSON.parse(Buffer.from((value as { body: string }).body, 'base64').toString('utf8'))
+  },
+}
+
+export default defineConfig({
+  store: 'encrypted',
+  stores: { encrypted: async () => new CodecStateStore(new LucidStateStore(db), aesCodec) },
+})
+```
+
+Source: `docs/stores/lucid.mdx`, `docs/stores/index.mdx`.
+
+## Common mistakes
+
+### HIGH setting `stalledThresholdMs` below the longest legitimate step
+
+Wrong:
+
+```ts
+transports.queue({ connection: 'redis', stalledThresholdMs: 60_000 }) // imports take ~90min
+```
+
+Correct:
+
+```ts
+transports.queue({ connection: 'redis', stalledThresholdMs: 2 * 60 * 60 * 1000 })
+```
+
+Mechanism: a claim's age equals the step's elapsed runtime because nothing
+renews it mid-processing; a threshold below the longest step redelivers jobs
+from merely-slow workers and double-runs the step. For tighter detection use
+per-step `timeoutMs`, which IS heartbeat-aware.
+Source: `docs/transports/queue.mdx` ("Stalled-job reclaim").
+
+### HIGH hand-copying the DDL into the migration
+
+Wrong:
+
+```ts
+export default class extends BaseSchema {
+  async up() {
+    this.schema.createTable('durable_workflow_runs', (t) => {
+      /* copied column list */
+    })
+  }
+}
+```
+
+Correct:
+
+```ts
+export default class extends BaseSchema {
+  static disableTransactions = true
+  async up() {
+    await createDurableTables(db, this.db.connectionName)
+  }
+}
+```
+
+Mechanism: the snapshot is only right until the next release drifts past it, and
+the two details are load-bearing — without `disableTransactions = true` a
+`pool.max: 1` setup hangs until acquire timeout (createDurableTables checks out
+its own connection), and `this.db` has no `.connection()` method, only
+`.connectionName`. Source: `docs/stores/lucid.mdx`.
+
+### MEDIUM leaving `autoSchema: false` with no upgrade discipline
+
+Wrong:
+
+```ts
+export default defineConfig({ store: 'lucid', stores: { lucid: stores.lucid() }, autoSchema: false })
+// …package upgraded twice, migrations never re-run
+```
+
+Correct:
+
+```bash
+node ace migration:run   # after every @adonis-agora/durable upgrade
+```
+
+Mechanism: with boot provisioning off, nothing repairs the schema; when the boot
+path DOES repair (autoSchema on), it logs a one-time "repaired the durable
+schema in place" warning meaning exactly this — the database was behind and the
+fix was never recorded in migration history. Source: `docs/stores/lucid.mdx`
+("The repair warning").
+
+### MEDIUM using the deprecated `group` option to isolate worker pools
+
+Wrong:
+
+```ts
+transports.db({ group: 'tenant-a' }) // silently accepted and ignored
+```
+
+Correct:
+
+```ts
+transports.db({ partition: 'tenant-a' })
+```
+
+Mechanism: routing is by handler name — an instance serves whatever names it
+registers, so there is no group to declare. Isolation comes from `partition`
+(suffixes routing tokens `<name>@<partition>`) plus `namespace` (scopes which
+runs this engine polls). Source: `docs/transports/db.mdx` options table,
+`docs/concepts/tenancy.mdx`.
+
+### MEDIUM wrapping a store in CodecStateStore and expecting heartbeats
+
+Wrong:
+
+```ts
+stores: { encrypted: async () => new CodecStateStore(new LucidStateStore(db), aesCodec) }
+// then relying on persisted heartbeat progress for long remote steps
+```
+
+Correct: accept the documented trade-off explicitly — the decorator does not
+implement the optional `recordStepHeartbeat`, so persisted liveness progress is
+traded for encryption at rest (the live `step.started` event still fires).
+
+Mechanism: the optional member list defines what decorators can silently drop;
+heartbeat persistence is one of them. Source: `docs/stores/index.mdx`
+(optional members table + warning).
+
+See also: `durable-reliability` — retries/timeouts interact with transport
+claim semantics; the stuck-run playbook.
+See also: `durable-cluster` — namespace/partition across topologies and tenants.

package/skills/durable-workflows/SKILL.md

@@ -0,0 +1,399 @@
+---
+name: durable-workflows
+description: >-
+  Author durable workflows and steps with @adonis-agora/durable: BaseWorkflow
+  classes under app/workflows (static workflow, constructor @inject DI,
+  dispatch/start statics, make:workflow), step handlers via @Step decorators,
+  defineStep, string names or transport.handle, ctx.localStep vs always-
+  dispatched ctx.step, durable sleeps (ctx.sleep/sleepUntil), signals
+  (waitForSignal/signal/signalWithStart), named events (waitForEvent/
+  publishEvent/eventBatch/onEvent/idempotent opts.id), webhooks (ctx.webhook),
+  external tasks (ctx.task/completeTask/failTask), transactional steps
+  (ctx.transaction), breakpoints, child workflows (child/startChild/all/
+  continueAsNew/GatherError), singleton workflows, entities (registerEntity/
+  callEntity), scheduling (ScheduledWorkflow/static schedule/@Scheduled),
+  tags and search attributes. Use when writing or composing workflow logic.
+license: MIT
+metadata:
+  type: core
+  library: "@adonis-agora/durable"
+  library_version: "0.25.0"
+  framework: adonisjs
+sources:
+  - 'DavideCarvalho/adonis-durable:docs/authoring/app-workflows.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/concepts/workflows-and-steps.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/concepts/sleep-and-signals.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/authoring/child-workflows.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/authoring/webhooks.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/authoring/entities.mdx'
+  - 'DavideCarvalho/adonis-durable:docs/authoring/scheduling.mdx'
+---
+
+# Authoring workflows & steps
+
+A workflow is a class under `app/workflows/` extending `BaseWorkflow` with a
+`static workflow = { name, version }`. The provider scans the directory at boot
+and registers every exported class — you never call `engine.register` by hand
+(it remains the low-level escape hatch). The convention mirrors `@adonisjs/queue`
+jobs under `app/jobs`; scaffold with `node ace make:workflow order`.
+
+## Setup
+
+```ts title="app/workflows/checkout_workflow.ts"
+import { inject } from '@adonisjs/core'
+import { BaseWorkflow } from '@adonis-agora/durable'
+import type { WorkflowCtx } from '@adonis-agora/durable'
+
+export default class CheckoutWorkflow extends BaseWorkflow {
+  static workflow = { name: 'checkout', version: '1' }
+
+  async run(ctx: WorkflowCtx, order: { id: number; total: number }) {
+    await ctx.localStep('reserveStock', async () => ({ reserved: true }))
+    const charge = await ctx.step('payments:charge-card', {
+      orderId: order.id,
+      amountCents: order.total,
+    })
+    const approval = await ctx.waitForSignal<{ approved: boolean }>(`approve:${order.id}`)
+    if (!approval.approved) return { status: 'rejected', chargeId: charge.chargeId }
+    await ctx.localStep('ship', async () => ({ shipped: true }))
+    return { status: 'shipped', chargeId: charge.chargeId }
+  }
+}
+```
+
+Run it from anywhere: `CheckoutWorkflow.dispatch(input, { runId })` enqueues;
+`CheckoutWorkflow.start(input)` awaits the settled result. Both statics are
+context-aware — inside a running body they become a linked child /
+fire-and-forget child instead of a top-level run.
+
+## Core patterns
+
+### Pattern 1 — typed steps with `@Step`, `defineStep`, and DI
+
+Steps are discovered from `app/steps` and served by name automatically. A
+`@Step` method ref passed to `ctx.step` gives full input/output typing:
+
+```ts title="app/steps/payment_steps.ts"
+import { Step } from '@adonis-agora/durable'
+import { z } from 'zod'
+
+export default class PaymentSteps {
+  @Step({
+    name: 'payments:charge-card',
+    input: z.object({ orderId: z.number().int(), amountCents: z.number().int() }),
+    output: z.object({ chargeId: z.string() }),
+    retries: 3,
+  })
+  async chargeCard(input: { orderId: number; amountCents: number }) {
+    return { chargeId: await stripe.charge(input) }
+  }
+}
+```
+
+```ts title="app/workflows/order_workflow.ts"
+import { inject } from '@adonisjs/core'
+import { BaseWorkflow } from '@adonis-agora/durable'
+import type { WorkflowCtx } from '@adonis-agora/durable'
+import PaymentSteps from '#steps/payment_steps'
+
+@inject()
+export default class OrderWorkflow extends BaseWorkflow {
+  static workflow = { name: 'order', version: '1' }
+
+  constructor(private payments: PaymentSteps) {
+    super()
+  }
+
+  async run(ctx: WorkflowCtx, order: Order) {
+    // Typed both ways — renaming chargeCard is a compile error here:
+    const charge = await ctx.step(this.payments.chargeCard, {
+      orderId: order.id,
+      amountCents: order.total,
+    })
+    return { chargeId: charge.chargeId }
+  }
+}
+```
+
+The instance is built once at registration and reused for every run — treat the
+constructor as wiring; keep per-run state in input and step outputs, never on
+`this`. Injected services are for **composing**, not doing work: reach them
+through step refs so the work is checkpointed.
+
+### Pattern 2 — pausing durably: sleeps, signals, events
+
+All three suspend the run with zero compute and survive restarts.
+
+```ts
+// Time-based wait — duration string or ms:
+await ctx.sleep('7 days')
+await ctx.sleepUntil(new Date('2026-12-31T00:00:00Z'))
+
+// Human-in-the-loop / webhook callback — point-to-point by token:
+const decision = await ctx.waitForSignal<{ approved: boolean }>(`approve:${order.id}`, {
+  timeoutMs: 24 * 60 * 60 * 1000, // throws SignalTimeoutError when the deadline passes
+})
+
+// Named pub/sub — one publish fans out to every matching waiter:
+const settled = await ctx.waitForEvent<{ orderId: string }>('payment.settled', {
+  match: { orderId: order.id },
+})
+```
+
+Deliver from controllers: `engine.signal(token, payload)` resumes one waiter;
+signals sent before the waiter registers are buffered, not lost;
+`engine.signalWithStart(workflow, input, runId, { token, payload })` starts the
+run if missing and delivers in one race-free call. Publish events with
+`engine.publishEvent(name, payload, { id: event.id })` — the upstream event id
+makes redelivery a no-op for `onEvent` starts. Register a workflow to start on
+events with `{ onEvent: ['payment.settled'] }`; coalesce chatty sources with
+`eventBatch: { mode: 'debounce', windowMs: 5_000 }` (last payload wins) or
+`{ mode: 'batch', maxSize: 50, windowMs: 60_000 }` (receives `{ events: [...] }`).
+
+```ts title="app/controllers/approvals_controller.ts"
+import { inject } from '@adonisjs/core'
+import { WorkflowEngine } from '@adonis-agora/durable'
+
+@inject()
+export default class ApprovalsController {
+  constructor(private engine: WorkflowEngine) {}
+
+  async approve({ params, request, response }: HttpContext) {
+    await this.engine.signal(`approve:${params.id}`, request.body())
+    return response.noContent()
+  }
+}
+```
+
+### Pattern 3 — children, fan-out, and singletons
+
+```ts
+const shipment = await ctx.child(ShippingWorkflow, { orderId: order.id }) // parent suspends
+await ctx.startChild('reindex-search', { postId: post.id }) // fire-and-forget
+const results = await ctx.all(ProcessItemWorkflow, batch.items) // outputs in input order
+if (input.n < 3) await ctx.continueAsNew({ n: input.n + 1 }) // clean history per generation
+```
+
+`ctx.all` default mode is `waitAll`: every sibling finishes, then the call throws
+a `GatherError` carrying `.failures`; `{ mode: 'failFast' }` cancels remaining
+siblings (plain cancel — no saga). Child ids derive from the call position and
+are replay-stable; never pass a random one.
+
+Serialize runs per subject with a singleton — runs sharing a key queue FIFO and
+are race-free across instances:
+
+```ts
+export default class SyncShopWorkflow extends BaseWorkflow {
+  static workflow = {
+    name: 'sync-shop',
+    version: '1',
+    singleton: { key: (input) => (input as { shopId: string }).shopId, limit: 1, maxQueueDepth: 20 },
+  }
+  // …
+}
+```
+
+Overflow beyond `limit + maxQueueDepth` rejects `dispatch` with
+`SingletonQueueFullError` before anything is created.
+
+### Pattern 4 — long-lived state, callbacks, cadences
+
+Entities serialize operations per key without DB locks (register from a
+preload):
+
+```ts title="start/entities.ts"
+import engine from '@adonis-agora/durable/services/main'
+
+engine.registerEntity<{ balanceCents: number }>('account', {
+  initialState: () => ({ balanceCents: 0 }),
+  handlers: {
+    deposit: (state, amountCents: number) => {
+      state.balanceCents += amountCents
+      return state.balanceCents
+    },
+    balance: (state) => state.balanceCents,
+  },
+})
+// drive: engine.signalEntity('account', key, 'deposit', 500)
+// read:  engine.getEntityState('account', key)
+// from a workflow: await ctx.callEntity('account', key, 'deposit', 500)
+```
+
+Wait on a foreign system's own callback by id (`engine.completeTask(runId, name,
+result)` / `failTask` resume the run):
+
+```ts
+const result = await ctx.task<TranscodeResult>(job.id, async () => {
+  await sqs.send({ queueUrl, body: JSON.stringify({ jobId: job.id, src: job.src }) })
+})
+```
+
+Recurring cadences are colocated on the class (or set in `config/durable.ts`
+`schedules`, which overrides by `key`); cron needs `cron-parser` installed where
+the worker runs, and only the `durable:work` loop fires schedules:
+
+```ts
+export default class DailyReportWorkflow extends BaseWorkflow {
+  static workflow = { name: 'daily-report', version: '1' }
+  static schedule = { cron: '0 2 * * *', timezone: 'America/Sao_Paulo' }
+  // or: @Scheduled({ everyMs: 60_000 }) above the class
+}
+```
+
+Each schedule window starts exactly once across a fleet via an idempotent
+time-bucket run id.
+
+Source: `docs/authoring/entities.mdx`, `docs/authoring/external-tasks.mdx`,
+`docs/authoring/scheduling.mdx`.
+
+## Common mistakes
+
+### HIGH calling an injected service directly inside `run`
+
+Wrong:
+
+```ts
+async run(ctx, input) {
+  const receipt = await this.billing.charge(input) // I/O outside a step
+  return { receipt }
+}
+```
+
+Correct:
+
+```ts
+async run(ctx, input) {
+  const receipt = await ctx.step(this.billing.charge, input)
+  return { receipt }
+}
+```
+
+Mechanism: work done outside `ctx.step`/`ctx.localStep` is never checkpointed,
+so it re-executes on every replay and its result is not durable. The docs warn
+explicitly that injected services exist "for composing the workflow, not for
+doing its work". Source: `docs/authoring/app-workflows.mdx`.
+
+### HIGH swallowing control-flow signals in a hand-rolled try/catch
+
+Wrong:
+
+```ts
+try {
+  await ctx.step('chargeCard', input)
+} catch (error) {
+  await ctx.step('refund', input) // also catches suspend/continue-as-new signals!
+  throw error
+}
+```
+
+Correct:
+
+```ts
+import { isWorkflowControlFlowSignal } from '@adonis-agora/durable'
+
+try {
+  await ctx.step('chargeCard', input)
+} catch (error) {
+  if (isWorkflowControlFlowSignal(error)) throw error
+  await ctx.step('refund', input)
+  throw error
+}
+```
+
+Mechanism: `ctx.sleep`, `ctx.waitForSignal` and `ctx.continueAsNew` unwind the
+turn by throwing; running your failure path on one records extra history a later
+replay never produces → `NonDeterminismError` on resume. The predicate checks a
+stamped marker, deliberately `false` for cancellations and real failures.
+Source: `docs/concepts/workflows-and-steps.mdx` ("Catching errors").
+
+### HIGH passing a random child id inside a workflow
+
+Wrong:
+
+```ts
+await ctx.startChild('reindex-search', item, crypto.randomUUID())
+```
+
+Correct:
+
+```ts
+await ctx.startChild('reindex-search', item) // deterministic positional id
+```
+
+Mechanism: inside a body, omitted child ids are derived deterministically from
+the call position so replay re-attaches to the same child; a fresh random id on
+each replay breaks that dedup and re-fires the child. Source:
+`docs/authoring/child-workflows.mdx` ("opts.runId — outside vs inside").
+
+### MEDIUM blocking a hot HTTP path on `.start`
+
+Wrong:
+
+```ts
+const result = await CheckoutWorkflow.start(order) // request held open until settle
+```
+
+Correct:
+
+```ts
+const { runId } = await CheckoutWorkflow.dispatch(order, { runId: `checkout:${order.id}` })
+return response.accepted({ runId })
+```
+
+Mechanism: `.start` resolves only when the run reaches a terminal state or
+`suspends` — hours later for a long workflow. Source: `docs/getting-started.mdx`
+(warning callout).
+
+### MEDIUM awaiting a webhook with no timeout
+
+Wrong:
+
+```ts
+const result = await hook.wait() // suspends indefinitely if the provider never calls back
+```
+
+Correct:
+
+```ts
+import { SignalTimeoutError } from '@adonis-agora/durable'
+
+try {
+  const result = await hook.wait({ timeoutMs: 30 * 60 * 1000 })
+} catch (error) {
+  if (!(error instanceof SignalTimeoutError)) throw error
+  // reconcile: poll the provider instead of hanging forever
+}
+```
+
+Mechanism: without `timeoutMs` there is no wake timer — a dropped callback
+leaves the run parked invisibly to the timer poller and crash recovery.
+Source: `docs/authoring/webhooks.mdx` ("Not waiting forever").
+
+### MEDIUM throwing business rejections out of entity handlers
+
+Wrong:
+
+```ts
+withdraw: (state, amountCents) => {
+  if (amountCents > state.balanceCents) throw new Error('insufficient funds')
+  state.balanceCents -= amountCents
+  return state.balanceCents
+},
+```
+
+Correct:
+
+```ts
+withdraw: (state, amountCents) =>
+  amountCents > state.balanceCents
+    ? { ok: false as const, reason: 'insufficient_funds' as const }
+    : ((state.balanceCents -= amountCents), { ok: true as const }),
+```
+
+Mechanism: a throwing handler fails the entity's immortal run, stranding every
+caller suspended on `ctx.callEntity`; model expected rejections as returned
+discriminated results and reserve throws for genuine programmer errors.
+Source: `docs/authoring/entities.mdx` (warning callout).
+
+See also: `durable-determinism` — why the body must stay deterministic and how
+to change it safely once live.