@voltro/cli 0.1.0

package/templates/agent-docs/workflows.md

@@ -0,0 +1,1351 @@
+# Workflows
+
+> Durable Effect workflows in Voltro — what they are, when to use them, and the current runtime boundaries.
+
+
+
+---
+
+<!-- source: en/workflows/overview.md -->
+## Overview
+
+_Durable Effect workflows in Voltro — what they are, when to use them, and the current runtime boundaries._
+
+A **workflow** in Voltro is a long-running Effect program that can survive process restarts. Durable work is expressed with `workflow({...})`, checkpointed `step({...})` calls, durable `sleep({...})`, optional external `awaitSignal(...)` waits, and synchronous `awaitUpdate(...)` messages.
+
+The implementation is `@effect/workflow` + `@effect/cluster`. No Restate, Temporal, Inngest, BullMQ, or separate worker runtime is required.
+
+Live — start a 3-step durable workflow and watch its timeline stream as each step
+completes (durable-execution UI no other framework ships):
+
+```tsx
+const run = useAction('app', 'demo.runPipeline')
+const { runId } = await run.run({})
+<WorkflowProgress api="app" runId={runId} />   {/* live step timeline */}
+```
+
+## What's in this section
+
+- [Defining workflows](/docs/workflows/definition) — `workflow`, `payload`, `success`, default execute factories, and checkpointed steps
+- [Retries & failure handling](/docs/workflows/retries) — per-step retries, failed runs, retry controls, and compensation
+- [Sleep & wakeups](/docs/workflows/sleep) — `sleep`, schedules, and `awaitSignal`
+- [Clustering](/docs/workflows/cluster) — multi-instance workflow distribution and schedule coordination
+- [Flow control](/docs/workflows/flow-control) — durable queues, bounded concurrency, and rate limits
+- [Versioning](/docs/workflows/versioning) — definition versions, compatibility metadata, and patch markers
+- [Debugging & inspection](/docs/workflows/debugging) — dashboard, inspect endpoints, CLI, tables, and tests
+
+## When to use a workflow
+
+| You have... | Use |
+|---|---|
+| Multi-step job that must survive a deploy | Workflow |
+| External side effects that need retry/resume semantics | Workflow |
+| Human approval or webhook callback in the middle of work | Workflow + `awaitSignal` |
+| External command that must validate and return a result | Workflow + `awaitUpdate` |
+| Domain event that should fan out to durable work | `triggerWorkflow(...)` + `ctx.events.emit(...)` |
+| One-off durable delay inside an operation | Workflow + `sleep` |
+| Recurring job such as "daily at 3am" | [Schedule](/docs/scheduling/overview) (`*.cron.tsx`) |
+| Atomic database write | Mutation |
+| Request-scoped HTTP/email/payment call | Action |
+| Progressive server-to-client output | Stream |
+| Request-scoped LLM chat | Agent or stream |
+
+The wedge: if the operation must survive the process, use a workflow. If it is just a short request, keep it in a mutation, action, query, or stream.
+
+## A minimal workflow
+
+A workflow is split into two files paired by basename — a **descriptor**
+(`*.workflow.tsx`, browser-safe, imports `@voltro/workflow/define`) that the
+codegen pulls into the client rpcGroup, and a **server executor**
+(`*.workflow.server.tsx`) that holds the database/AI/cluster imports:
+
+```tsx
+// apps/api/workflows/notes.summarise.workflow.tsx — descriptor (browser-safe)
+import { workflow } from '@voltro/workflow/define'
+import { Schema } from 'effect'
+
+export const SummariseNote = workflow({
+  name: 'notes.summarise',
+  payload: { noteId: Schema.String },
+  success: Schema.Struct({ summary: Schema.String }),
+  idempotencyKey: ({ noteId }) => `notes.summarise:${noteId}`,
+})
+```
+
+```tsx
+// apps/api/workflows/notes.summarise.workflow.server.tsx — executor (server-only)
+import { step } from '@voltro/workflow'
+import { Effect, Schema } from 'effect'
+import type { AppContext } from '@voltro/runtime'
+
+const buildExecute = (ctx: AppContext) =>
+  ({ noteId }: { noteId: string }) =>
+    Effect.gen(function* () {
+      const note = yield* step({
+        name: 'load-note',
+        input: { noteId },
+        success: Schema.Struct({ id: Schema.String, body: Schema.String }),
+        execute: Effect.tryPromise(() => ctx.store.query({
+          table: 'notes',
+          predicate: { field: 'id', op: 'eq', value: noteId },
+        } as never).then((rows) => rows[0] as never)),
+      })
+
+      const summary = yield* step({
+        name: 'summarise-with-llm',
+        input: { noteId },
+        success: Schema.String,
+        execute: summariseWithLlm(note.body),
+      })
+
+      yield* step({
+        name: 'save-summary',
+        input: { noteId },
+        success: Schema.Void,
+        execute: Effect.tryPromise(() =>
+          ctx.store.update('notes', noteId, { summary } as never).then(() => undefined),
+        ),
+      })
+
+      return { summary }
+    })
+
+export default buildExecute
+```
+
+`step({...})` is the checkpoint boundary. A plain `yield* someEffect` composes Effect logic, but it is not automatically recorded as a workflow step. Put external I/O, database writes, and expensive work inside `step`.
+
+## Starting a workflow
+
+Every discovered `*.workflow.tsx` is emitted into `rpcGroup.generated.ts` as a unary RPC. The payload and error schemas come from the workflow, but the RPC success type is always a `WorkflowRunHandle`. Calling it starts durable work and returns that handle immediately:
+
+```tsx
+import {
+  useWorkflow,
+  useWorkflowRun,
+  useWorkflowRunEvents,
+  useWorkflowRuns,
+  useWorkflowRunSteps,
+  useWorkflowUpdate,
+} from '@voltro/client'
+
+const summarise = useWorkflow<{ noteId: string }>(
+  'app',
+  'notes.summarise',
+)
+
+const run = await summarise.start({ noteId })
+const { run: live } = useWorkflowRun('app', run.id)
+```
+
+`run.id` is the durable execution id and can be passed to `useWorkflowRun(...)`. The hook subscribes to Voltro's built-in reactive workflow-run query, so the UI updates when `_voltro_workflow_runs` changes. Waiting for the success payload is explicit: use `ctx.workflows.wait(...)` on the server when another server-side operation really must block; otherwise render the live run row and let the workflow finish in the background.
+
+For app-level job centers and detail pages, use the rest of the workflow hook family:
+
+```tsx
+const { runs } = useWorkflowRuns('app', { tag: 'notes.summarise', limit: 25 })
+const { steps } = useWorkflowRunSteps('app', live?.id)
+const { events } = useWorkflowRunEvents('app', live?.id)
+const approve = useWorkflowUpdate('app')
+await approve.update({ id: live!.id }, 'approve', { decision: true })
+```
+
+Those hooks are backed by reactive framework tables too, so status, checkpointed steps, timers, signals, and update events flow through Voltro subscriptions instead of polling.
+
+From server code, use `ctx.workflows`:
+
+```ts
+export default async ({ noteId }, ctx) => {
+  await ctx.store.update('notes', noteId, { summaryStatus: 'queued' })
+  const run = await ctx.workflows.start('notes.summarise', { noteId })
+  return { runId: run.id }
+}
+```
+
+Inside a mutation, `ctx.workflows.start(...)` is transaction-aware: Voltro computes the deterministic run handle immediately, queues the actual start, and only launches the workflow after the mutation commits. If the mutation rolls back, the workflow never starts. Actions and incoming webhooks start immediately after their own verification/idempotency work has succeeded.
+
+Workflow controls use the same facade:
+
+```ts
+await ctx.workflows.signal({ id: runId }, 'approval', { approved: true })
+const approved = await ctx.workflows.update({ id: runId }, 'approve', { decision: true })
+await ctx.workflows.cancel('notes.summarise', executionId)
+await ctx.workflows.resume('notes.summarise', executionId)
+const snapshot = await ctx.workflows.query('notes.summarise', executionId)
+```
+
+Inside a workflow body, use `ctx.workflows.child(...)` for hierarchical work. Voltro records the parent execution id in `_voltro_workflow_start_contexts` before submitting the child run, so a different cluster runner still sees the same lineage when it starts executing the child.
+
+```ts
+const child = yield* Effect.promise(() =>
+  ctx.workflows.child('orders.shipOne', { orderId }, {
+    parentClosePolicy: 'cancel',
+  }),
+)
+
+const childSnapshot = yield* Effect.promise(() => ctx.workflows.wait(child))
+```
+
+`parentClosePolicy` controls what happens when the parent closes. `cancel` is the default and interrupts an open child. `terminate` also interrupts the child and records the decision as a hard parent-close action. `abandon` leaves the child running. The dashboard shows child runs, their parent execution id, and the chosen policy.
+
+Inside the workflow body, `awaitUpdate(...)` receives the tracked message, validates the payload, and records either `update-completed` with the result or `update-failed` with the validation/handler error:
+
+```tsx
+import { awaitUpdate } from '@voltro/workflow'
+import { Effect, Schema } from 'effect'
+
+const result = yield* awaitUpdate(ctx, {
+  name: 'approve',
+  schema: Schema.Struct({ decision: Schema.Boolean }),
+  success: Schema.Struct({ accepted: Schema.Boolean }),
+  handle: ({ decision }) => Effect.succeed({ accepted: decision }),
+})
+```
+
+## Event-triggered workflows
+
+Use `*.trigger.ts` files when durable work should start from domain events instead of a direct RPC or action call.
+
+```ts
+// apps/api/triggers/user.signup.trigger.ts
+import { triggerWorkflow } from '@voltro/runtime'
+
+export default triggerWorkflow<{ userId: string; plan: string }, { userId: string }>({
+  event: 'user.signup',
+  workflow: 'onboarding.start',
+  filter: (event) => event.data.plan !== 'free',
+  payload: (event) => ({ userId: event.data.userId }),
+  idempotencyKey: (event) => event.id,
+})
+```
+
+Then emit the event from a mutation, action, incoming webhook, schedule, or workflow:
+
+```ts
+await ctx.events.emit('user.signup', { userId, plan })
+```
+
+Voltro records the event in `_voltro_workflow_events`, records each trigger delivery in `_voltro_workflow_event_deliveries`, and starts matching workflows with `source: event:<name>`. Emits inside mutations are post-commit safe: if the mutation rolls back, the workflow fan-out is not launched.
+
+Use `event: '*'` for audit-style wildcard triggers that should see every emitted domain event. The local devtools and Voltro Cloud workflow dashboards mirror domain events and delivery rows live in the Events tab, including filtered/skipped deliveries, failed fan-out, and the spawned execution id.
+
+## Runtime boundaries
+
+- The workflow executor receives an `AppContext` built for the run. Starts from RPC handlers, mutations, actions, schedules, verified incoming handlers, and inspect tooling record a `source` such as `workflow-rpc`, `app-context`, `schedule:<name>`, `incoming:<id>`, or `inspect`. Request starts also carry the starter trace; authenticated request starts carry the resolved subject into the run record and executor context.
+- `ctx.events.emit(...)` is available when the app declares workflow event triggers. Triggered workflow starts carry the starter subject/trace and a `source` of `event:<name>`.
+- Put tenant/user ids that the workflow must enforce into the payload. That keeps business authorization deterministic across retries, resumes, and future cross-replica handoff.
+- Child workflows should be started with `ctx.workflows.child(...)`. Parent lineage and parent-close policy are persisted before the child start, so they survive cross-runner execution.
+- External incoming handlers created with `defineIncomingWebhook(...)` receive `context.workflows`, so a verified webhook can start, signal, or update a workflow after signature and idempotency checks.
+- Operational controls such as retry, cancel, suspend, resume, signal injection, and tracked updates also live on the dashboard, `voltro workflows ...`, and the inspect endpoints.
+
+## What is intentionally different from Temporal
+
+- **Polyglot workers.** Voltro workflows are TypeScript/Effect.
+- **A separate workflow service.** This is intentional: the engine is mounted by the framework alongside your API.
+- **Automatic code-version isolation.** A long-lived run resumed after a deploy executes the current workflow code. Use workflow `version`, `compatibleWith`, and `patches` metadata for deploy safety.
+
+What you gain: workflows live beside queries, mutations, actions, streams, schedules, schema, and plugins. The same codegen and inspect surfaces see them.
+
+## Where to go next
+
+- [Defining workflows](/docs/workflows/definition)
+- [Retries & failure handling](/docs/workflows/retries)
+- [Sleep & wakeups](/docs/workflows/sleep)
+- [Flow control](/docs/workflows/flow-control)
+- [Versioning](/docs/workflows/versioning)
+- [Debugging & inspection](/docs/workflows/debugging)
+
+
+
+---
+
+<!-- source: en/workflows/definition.md -->
+## Defining workflows
+
+_The *.workflow.tsx convention, workflow schemas, execute factories, checkpointed steps, child workflows, and replay rules._
+
+A workflow file is discovered by suffix: `*.workflow.tsx`. It exports one `workflow({...})` definition and a default factory that receives `AppContext` and returns the workflow executor.
+
+## Anatomy
+
+```tsx
+// apps/api/workflows/notes.summarise.workflow.tsx
+import { workflow, step } from '@voltro/workflow'
+import { Effect, Schema } from 'effect'
+import type { AppContext } from '@voltro/runtime'
+
+export const SummariseNote = workflow({
+  name: 'notes.summarise',
+  payload: { noteId: Schema.String },
+  success: Schema.Struct({ summary: Schema.String }),
+  idempotencyKey: ({ noteId }) => `notes.summarise:${noteId}`,
+})
+
+const buildExecute = (ctx: AppContext) =>
+  ({ noteId }: { noteId: string }, _executionId: string) =>
+    Effect.gen(function* () {
+      const note = yield* step({
+        name: 'load-note',
+        input: { noteId },
+        success: Schema.Struct({ id: Schema.String, body: Schema.String }),
+        execute: Effect.tryPromise(() => loadNote(ctx.store, noteId)),
+      })
+
+      const summary = yield* step({
+        name: 'summarise-with-llm',
+        input: { noteId },
+        success: Schema.String,
+        execute: summariseWithLlm(note.body),
+      })
+
+      yield* step({
+        name: 'save-summary',
+        input: { noteId },
+        success: Schema.Void,
+        execute: Effect.tryPromise(() => saveSummary(ctx.store, noteId, summary)),
+      })
+
+      return { summary }
+    })
+
+export default buildExecute
+```
+
+Two exports matter:
+
+- `workflow({...})` — the durable definition. `payload`, `success`, optional `error`, and optional `idempotencyKey` are read by codegen and the runtime.
+- `default` — a build-execute factory, `(ctx: AppContext) => (payload, executionId) => Effect`. The CLI calls `definition.toLayer(execute)` during registration.
+
+If the default export is not a function, discovery skips the file with a "missing default-export build-execute factory" warning.
+
+## Schema fields
+
+```ts
+export const ImportCustomers = workflow({
+  name: 'customers.import',
+  payload: {
+    uploadId: Schema.String,
+    dryRun: Schema.optional(Schema.Boolean),
+  },
+  success: Schema.Struct({
+    imported: Schema.Number,
+    skipped: Schema.Number,
+  }),
+  error: Schema.Union(InvalidCsv, ImportProviderDown),
+  idempotencyKey: ({ uploadId }) => `customers.import:${uploadId}`,
+  messages: {
+    signals: {
+      approval: Schema.Struct({ approved: Schema.Boolean }),
+    },
+    updates: {
+      approve: {
+        payload: Schema.Struct({ decision: Schema.Boolean }),
+        success: Schema.Struct({ accepted: Schema.Boolean }),
+      },
+    },
+  },
+})
+```
+
+`payload` is the start input. `success` is the resolved output. `error` is the typed failure channel. `idempotencyKey` deduplicates concurrent or repeated starts with the same logical input. `messages` is optional codegen metadata; it emits `WorkflowSignals`, `WorkflowUpdates`, `WorkflowQueries`, and `WorkflowMessages` type maps, while runtime validation still happens at `awaitSignal(...)` / `awaitUpdate(...)`.
+
+## Step boundaries
+
+`step({...})` wraps `@effect/workflow`'s `Activity.make`. It is the checkpointed unit that is journaled by the workflow engine and recorded into `_voltro_workflow_run_steps`.
+
+```ts
+const customer = yield* step({
+  name: 'fetch-customer',
+  input: { customerId },
+  success: Customer,
+  execute: Effect.tryPromise(() => crm.fetchCustomer(customerId)),
+})
+```
+
+The optional `input` field is not passed to the executor. It is persisted for inspection, truncated to a dashboard-safe size, and shown beside output/errors in the run timeline.
+
+Plain Effect composition is still useful:
+
+```ts
+const normalised = normaliseCustomer(customer)
+const enriched = yield* Effect.succeed(addDerivedFields(normalised))
+```
+
+But it is not a durable activity boundary by itself. If it does external I/O, writes to storage, charges a card, sends an email, or calls an LLM, put that work inside `step`.
+
+## Deterministic replay
+
+The workflow body can be replayed by the engine. The structure of checkpointed steps must be stable for the same payload.
+
+OK:
+
+```ts
+const note = yield* step({ name: 'load-note', success: Note, execute: loadNote(noteId) })
+
+if (note.archived) {
+  yield* step({ name: 'notify-archived', execute: notifyArchived(note.id) })
+} else {
+  yield* step({ name: 'summarise', execute: summarise(note.body) })
+}
+```
+
+The branch depends on a checkpointed value.
+
+Not OK:
+
+```ts
+if (Math.random() > 0.5) {
+  yield* step({ name: 'a', execute: doA })
+} else {
+  yield* step({ name: 'b', execute: doB })
+}
+```
+
+If time, randomness, or external state influences structure, capture it in a step first:
+
+```ts
+const choice = yield* step({
+  name: 'choose-branch',
+  success: Schema.Boolean,
+  execute: Effect.sync(() => Math.random() > 0.5),
+})
+```
+
+## Starting from the client
+
+Codegen synthesises an RPC for each workflow. To start it from React, use `useWorkflow(...)`; the call returns a run handle immediately and the durable work continues in the workflow engine:
+
+```tsx
+import { useWorkflow, useWorkflowRun } from '@voltro/client'
+
+const startImport = useWorkflow<{ uploadId: string }>('app', 'customers.import')
+
+const run = await startImport.start({ uploadId })
+const { run: liveRun } = useWorkflowRun('app', run.id)
+```
+
+Use `liveRun.status` and `liveRun.output` to render progress/result state. Waiting for the success payload is explicit on the server with `ctx.workflows.wait(...)`; UI code should usually subscribe to the run row instead of blocking the interaction.
+
+## Child workflows
+
+Inside a workflow body, start child runs through `ctx.workflows.child(...)`. The child gets its own durable run, step history, events, cancellation controls, and dashboard detail page. Voltro persists the parent execution id and parent-close policy before submitting the child start, so the relationship survives cross-runner execution in a cluster.
+
+```ts
+const child = yield* Effect.promise(() =>
+  ctx.workflows.child('documents.embed', { documentId }, {
+    parentClosePolicy: 'cancel',
+  }),
+)
+
+const result = yield* Effect.promise(() => ctx.workflows.wait(child))
+```
+
+Fan out with normal Effect concurrency:
+
+```ts
+const children = yield* Effect.all(
+  documents.map((doc) =>
+    Effect.promise(() =>
+      ctx.workflows.child('documents.embed', { documentId: doc.id }, {
+        parentClosePolicy: 'abandon',
+      }),
+    ),
+  ),
+  { concurrency: 8 },
+)
+```
+
+Parent-close policies:
+
+| Policy | Behavior |
+|---|---|
+| `cancel` | Default. Interrupt open children when the parent closes. |
+| `terminate` | Interrupt open children and record a hard parent-close action. |
+| `abandon` | Leave the child running when the parent closes. |
+
+The dashboard shows child runs, their parent execution id, and the selected policy. `ctx.workflows.wait(child)` accepts the run handle directly when the parent needs the child's success/failure snapshot.
+
+## Tenant and subject
+
+Workflow starts persist the starter subject, trace id, source, parent execution id, and parent-close policy in `_voltro_workflow_start_contexts`. Whichever runner first executes the workflow loads that context before building the executor `AppContext`.
+
+Still include tenant/user ids that the business process must enforce in `payload`, validate them in the first step, and scope store reads/writes deliberately. Payload data is replay-safe and makes authorization decisions auditable across retries and deploys.
+
+## Anti-patterns
+
+- **Using `input` in `workflow({...})`.** The current API is `payload`.
+- **Assuming every `yield*` is a persisted step.** Use `step({...})` for checkpoints.
+- **Starting workflows through `useMutation(...)`.** Use `useWorkflow(...)`; workflows are durable runs, not optimistic writes.
+- **Branching step structure on randomness or live external state.** Capture the value in a step first.
+- **Long-lived workflows without payload versioning.** A run resumed after a deploy uses the current workflow code.
+
+
+
+---
+
+<!-- source: en/workflows/retries.md -->
+## Retries & failure handling
+
+_Per-step retries, failed workflow runs, retry controls, compensation, and timeouts._
+
+Workflow durability and retry are related, but not identical. The engine checkpoints completed `step({...})` activities so an interrupted run can resume without re-running successful steps. Transient failures still need an explicit retry policy around the step that can fail.
+
+## Per-step retries
+
+Use `stepModule.retry(...)` when you want retry attempts to appear as separate step attempts in the dashboard:
+
+```tsx
+import { workflow, step, stepModule } from '@voltro/workflow'
+import { Effect, Schedule, Schema } from 'effect'
+
+class ProviderDown extends Schema.TaggedError<ProviderDown>()('ProviderDown', {
+  message: Schema.String,
+}) {}
+
+const summary = yield* stepModule.retry(
+  step({
+    name: 'summarise-with-llm',
+    input: { noteId },
+    success: Schema.String,
+    error: ProviderDown,
+    retry: {
+      strategy: 'exponential',
+      maxAttempts: 5,
+      baseDelay: '500 millis',
+    },
+    execute: callLlm(note.body),
+  }),
+  Schedule.exponential('500 millis').pipe(Schedule.recurs(4)),
+)
+```
+
+The `retry` field on `step({...})` is metadata for inspection. The actual retry behaviour comes from `stepModule.retry(...)`, `Effect.retry(...)`, or another Effect retry combinator.
+
+## What to retry
+
+| Failure kind | Recommended handling |
+|---|---|
+| HTTP 5xx, connection reset, provider 429 | Retry the step with backoff and jitter |
+| Validation error, malformed payload | Fail the workflow |
+| Permission error | Fail the workflow; fix caller or payload |
+| Timeout | Convert to a typed transient error, then retry if safe |
+| Cancel/suspend by operator | Do not catch unless you are deliberately cleaning up |
+
+Keep retry windows close to the side effect. A flaky LLM call should retry inside the LLM step, not by restarting the whole workflow body.
+
+## Failed runs
+
+When a workflow ultimately fails, Voltro records the run as `failed`. Query failed runs from any handler, action, or mutation with the typed SDK — no raw SQL, no knowledge of internal tables:
+
+```ts
+const failed = await ctx.workflows.listRuns({
+  status: 'failed',
+  workflowName: 'notes.summarise',
+})
+// failed: WorkflowRunSummary[] — id, workflowName, status, payload,
+// errorTag, errorMessage, startedAt, completedAt, durationMs, …
+```
+
+`listRuns(filter?)` accepts `{ workflowName / tag, status, limit, offset }` (all optional) and returns the most-recent runs first. The inspect dashboard surfaces the same data live, but the SDK is the primary path — it's typed and reactive-friendly.
+
+There is no separate `voltro_workflow_dlq` table in the current runtime. Failed runs are the triage queue.
+
+## Retrying a failed run
+
+Re-run a failed run by its id from a handler, action, or mutation:
+
+```ts
+for (const run of failed) {
+  // Re-run against the original payload …
+  await ctx.workflows.retry(run.id)
+  // … or replay against a corrected input after fixing bad data:
+  await ctx.workflows.retry(run.id, { payloadOverride: { ...run.payload, retries: 1 } })
+}
+```
+
+`retry(runId, options?)` resolves the workflow by its tag, re-executes it, and returns `{ executionId }` — the engine-assigned id of the fresh run. By default it uses the original run's recorded payload; pass `payloadOverride` to replay against a different input.
+
+For ad-hoc ops, the same operation is available manually — the CLI (`voltro workflows retry <runId>`) and the dashboard's "Retry" button both delegate to the same runtime code:
+
+```sh
+voltro workflows retry wfrun_01H...
+```
+
+## Compensation
+
+For saga-style workflows, model compensation explicitly with `Effect.catchAll` around the step that can fail after an earlier side effect:
+
+```ts
+yield* step({
+  name: 'reserve-inventory',
+  success: Schema.Void,
+  execute: reserveInventory(orderId),
+})
+
+yield* step({
+  name: 'charge-card',
+  success: Schema.Void,
+  execute: chargeCard(orderId).pipe(
+    Effect.catchAll((error) =>
+      releaseInventory(orderId).pipe(
+        Effect.zipRight(Effect.fail(error)),
+      ),
+    ),
+  ),
+})
+```
+
+There is no `defineSaga(...)` helper today. Compensation is ordinary Effect code, checkpointed when you put it inside a step.
+
+## Timeouts
+
+A hung external call is not a failure until it errors. Wrap it:
+
+```ts
+const response = yield* step({
+  name: 'fetch-provider',
+  success: ProviderResponse,
+  error: ProviderDown,
+  execute: fetchProvider(input).pipe(
+    Effect.timeoutFail({
+      duration: '30 seconds',
+      onTimeout: () => new ProviderDown({ message: 'provider timed out' }),
+    }),
+  ),
+})
+```
+
+If the operation is not idempotent, pass an idempotency key to the provider and also enforce a unique key in your own database.
+
+## Failure observability
+
+Every run has three inspectable layers:
+
+- `_voltro_workflow_runs` — run status, payload, output, top-level error, subject, start source, timing, trace id, parent execution id, parent-close policy.
+- `_voltro_workflow_run_steps` — each step attempt, recorded input/output/error, retry metadata, duration.
+- `_voltro_workflow_run_events` — run lifecycle, timers, signals, suspend/resume/cancel events.
+
+The dashboard and `voltro workflows show <runId>` read these same tables through inspect endpoints.
+
+## Anti-patterns
+
+- **Catching errors and returning success.** Operators lose the failed run.
+- **Retrying non-idempotent side effects.** Double charges and duplicate emails are workflow bugs, not retry bugs.
+- **Putting retry metadata on a step without `Effect.retry` or `stepModule.retry`.** Metadata only explains intent; it does not execute a policy.
+- **Expecting a DLQ table.** The current triage surface is failed runs plus retry controls.
+
+
+
+---
+
+<!-- source: en/workflows/sleep.md -->
+## Sleep & wakeups
+
+_Durable sleep, cron schedules, external signals, and how parked workflows show up in inspection._
+
+Use `sleep({...})` from `@voltro/workflow` for durable waits inside a workflow. It wraps `@effect/workflow`'s `DurableClock.sleep` and records timer events for the dashboard.
+
+## Basic sleep
+
+```tsx
+import { workflow, step, sleep } from '@voltro/workflow'
+import { Effect, Schema } from 'effect'
+
+const buildExecute = (ctx: AppContext) =>
+  ({ userId }: { userId: string }) =>
+    Effect.gen(function* () {
+      yield* step({
+        name: 'send-welcome',
+        success: Schema.Void,
+        execute: sendEmail(ctx, userId, 'welcome'),
+      })
+
+      yield* sleep({ name: 'retention-delay', duration: '7 days' })
+
+      yield* step({
+        name: 'send-retention-check-in',
+        success: Schema.Void,
+        execute: sendEmail(ctx, userId, 'retention-check-in'),
+      })
+    })
+```
+
+The workflow is not meant to hold a worker thread for the whole delay. The durable clock parks the workflow in the engine; Voltro records `timer-set` and `timer-fired` events in `_voltro_workflow_run_events`.
+
+## Sleep until a wall-clock time
+
+Compute a duration, then pass it to `sleep`:
+
+```ts
+const millisUntilNextThreeUtc = () => {
+  const now = new Date()
+  const target = new Date(now)
+  target.setUTCHours(3, 0, 0, 0)
+  if (target <= now) target.setUTCDate(target.getUTCDate() + 1)
+  return target.getTime() - now.getTime()
+}
+
+yield* sleep({
+  name: 'wait-until-3am-utc',
+  duration: `${millisUntilNextThreeUtc()} millis`,
+})
+```
+
+Use this for one-off waits that are part of a larger workflow. For recurring cadence, use schedules.
+
+## Recurring work belongs in schedules
+
+Do not build a forever loop with `sleep` for "run every day" jobs. Use `*.cron.tsx`:
+
+```tsx
+// apps/api/schedules/daily-rollup.cron.tsx
+import { defineSchedule } from '@voltro/runtime'
+
+export default defineSchedule({
+  name: 'rollup.daily',
+  cron: '0 3 * * *',
+  timezone: 'UTC',
+  onOverlap: 'skip',
+  handler: async ({ app, scheduledAt }) => {
+    await runRollup(app.store, scheduledAt)
+  },
+})
+```
+
+Schedules give you timezone parsing, overlap policy, backfill policy, cluster-wide coordination, run history, and manual "Run now" controls.
+
+## External wakeups with `awaitSignal`
+
+Use `awaitSignal(ctx, ...)` when a workflow must wait for an outside decision, such as a human approval or webhook callback.
+
+```tsx
+import { awaitSignal, step } from '@voltro/workflow'
+import { Effect, Schema } from 'effect'
+
+const decision = yield* awaitSignal(ctx, {
+  name: 'approval',
+  schema: Schema.Struct({
+    approved: Schema.Boolean,
+    reviewerId: Schema.String,
+  }),
+  timeoutMs: 24 * 60 * 60_000,
+})
+
+if (!decision.approved) {
+  yield* step({
+    name: 'cancel-order',
+    success: Schema.Void,
+    execute: cancelOrder(orderId, decision.reviewerId),
+  })
+}
+```
+
+Signals are matched by name inside one workflow run. The payload is decoded with the supplied schema.
+
+## Sending signals
+
+From the dashboard, open a running workflow and use "Send signal".
+
+From the CLI:
+
+```sh
+voltro workflows signal wfrun_01H... --name approval --payload '{"approved":true,"reviewerId":"usr_123"}'
+```
+
+From HTTP:
+
+```http
+POST /_voltro/inspect/workflows/runs/:id/signal
+Content-Type: application/json
+
+{ "signalName": "approval", "payload": { "approved": true, "reviewerId": "usr_123" } }
+```
+
+The runtime records `signal-awaited`, `signal-sent`, and `signal-received` events in `_voltro_workflow_run_events`. The polling activity inside `awaitSignal` is checkpointed; once it receives a payload, replay returns the journaled value.
+
+## Cancellation and suspend/resume
+
+Use operational controls, not sentinel rows:
+
+```sh
+voltro workflows cancel wfrun_01H...
+voltro workflows suspend wfrun_01H...
+voltro workflows resume wfrun_01H...
+```
+
+The inspect API exposes the same actions at:
+
+```http
+POST /_voltro/inspect/workflows/runs/:id/cancel
+POST /_voltro/inspect/workflows/runs/:id/suspend
+POST /_voltro/inspect/workflows/runs/:id/resume
+```
+
+## Anti-patterns
+
+- **Using `Effect.sleep` for durable workflow waits.** Use `sleep({...})` so timer events are recorded and the durable clock is used.
+- **Forever-loop cron workflows.** Use `*.cron.tsx` schedules for recurring cadence.
+- **Polling an external system with long sleeps.** Prefer webhooks or `awaitSignal`.
+- **Sleeping inside a database transaction.** Finish the write boundary before the wait.
+
+
+
+---
+
+<!-- source: en/workflows/cluster.md -->
+## Clustering
+
+_How Voltro scales workflows, crons, and reactivity across multiple instances — the built-in @effect/cluster runtime, schedule claims, and per-dialect CDC._
+
+Clustering is **built in**. There is no plugin to install and no config flag to flip: run more than one instance of your app against the same database and they form a cluster automatically. Durable workflows resume on any surviving instance, each cron fires exactly once across the whole fleet, and writes on one instance become reactive subscriptions on every other.
+
+`@effect/cluster` is a core dependency of `@voltro/workflow` — the CLI wires it for you when `voltro dev` or `voltro start` boots an app on a SQL store. You scale **out** by running N instances; the coordination machinery is already there.
+
+## How it's enabled
+
+There is nothing to enable in code. The wiring is automatic and driven by the store dialect:
+
+| Store dialect | Runner storage | Horizontal scale-out |
+|---|---|---|
+| `postgres`, `mysql`, `mariadb`, `mssql` | `sql` | yes — workflows resume on any instance |
+| `sqlite` | `memory` | no — durable replay within one process |
+| `memory` (dev default) | — | single process |
+
+On a SQL store, the runtime composes the dialect's `SqlClient` → `@effect/cluster` `SingleRunner` → `@effect/workflow` `ClusterWorkflowEngine` into one layer and provides it to your program. `sqlite` falls back to in-process durable replay (no cross-instance scale-out) because `@effect/cluster`'s runner-lock acquisition has no SQLite branch.
+
+You do not call the cluster layer yourself — the CLI does. The boot log tells you which mode resolved:
+
+```text
+[voltro:dev] workflow engine: cluster-sql, dialect=mariadb
+```
+
+### The deployment contract
+
+A real multi-instance deployment has to give each pod a routable identity. This is the entire operational surface of clustering, and it is what the framework's Helm baseline injects:
+
+- **Routable runner host (`POD_IP`).** When a workflow that started on a now-dead pod has to resume elsewhere, the surviving pod addresses the dead pod's runner by the address it advertised. The default is `localhost`, which is unreachable cross-pod. Inject the Kubernetes downward-API `status.podIP` as `POD_IP` (or set `VOLTRO_WORKFLOW_RUNNER_HOST`) so each pod advertises its real address. The runtime warns at boot if a SQL-backed runner is still on `localhost`.
+- **Fixed runner port.** `SingleRunner` otherwise picks a random port, which breaks restart-resume because the runner table still references the previous process's port. Pin it via `VOLTRO_WORKFLOW_RUNNER_PORT` (default `34000`) so a restarted pod inherits the same cluster identity.
+- **Unique, stable `server_id` per pod (MariaDB CDC).** The binlog reader needs a `server_id` that is unique across the fleet and stable across reschedules — duplicate ids silently break binlog streams. The runtime derives it by hashing a per-pod replica id (`POD_NAME` → FNV-1a). With a `StatefulSet` the pod name is ordinal and deterministic.
+- **Replication user (MariaDB).** The binlog CDC reader connects with a dedicated user holding `REPLICATION SLAVE, REPLICATION CLIENT`, against a server configured `binlog_format=ROW`, `binlog_row_image=FULL`, `gtid_strict_mode=ON`.
+
+The relevant downward-API wiring, from the framework's Helm baseline (the AWB 4-replica MariaDB deployment is the worked example):
+
+```yaml
+env:
+  - name: POD_NAME
+    valueFrom: { fieldRef: { fieldPath: metadata.name } }   # → stable per-pod CDC server_id
+  - name: POD_IP
+    valueFrom: { fieldRef: { fieldPath: status.podIP } }    # → routable workflow-runner host
+  - name: VOLTRO_WORKFLOW_RUNNER_HOST
+    value: "$(POD_IP)"
+  - name: VOLTRO_WORKFLOW_RUNNER_PORT
+    value: "34000"
+  - name: DB_DIALECT
+    value: "mariadb"
+  - name: CDC
+    value: "1"
+```
+
+That is the whole contract: stable per-pod identity (`POD_NAME` → `server_id`), routable runner host (`POD_IP`), a pinned runner port, the shared `DB_URL`, and CDC on. No application code changes between one instance and four.
+
+## Durable workflows resume on any instance
+
+Workflow durability lives in `@effect/cluster`'s engine, not in any Voltro table. Each workflow's journal (steps, idempotency keys, signals) is persisted to the cluster's storage on the shared database. When a pod dies mid-step:
+
+1. Its runner lease stops being renewed.
+2. Another runner reclaims the dead runner's shards.
+3. The reclaiming runner reads the workflow journal, fast-forwards past completed steps, and resumes from the unfinished step.
+
+Because steps are journaled with idempotency keys, a re-executed step replays its cached result rather than running its side effects twice. This is why the `POD_IP` runner host matters: the reclaiming pod must be able to reach the workflow's runner address to take it over.
+
+`@effect/cluster` is dialect-agnostic here — its `SqlRunnerStorage` and `SqlMessageStorage` dispatch internally via `sql.onDialectOrElse({ mssql, mysql, sqlite, orElse: postgres })`. MariaDB rides the `mysql` branch (`GET_LOCK`, `ON DUPLICATE KEY UPDATE`, native `RETURNING` since 10.5).
+
+> **mssql only:** `@effect/cluster`'s mssql storage has two driver-level bugs the framework fixes with a patch that a plain install can't carry. Run **`voltro add mssql`** once, then `pnpm install` — see [SQL Server → Workflow cluster](/docs/database/dialects/mssql#workflow-cluster). No-op on every other dialect.
+
+## Crons fire once cluster-wide
+
+Schedule coordination is app-wide. On a postgres store the default is `advisoryLock`; you can opt into `cluster`, or force `single` for a one-instance deployment with `app.config.ts` (`scheduling: { coordination }`). Individual `*.cron.tsx` files stay topology-agnostic.
+
+| `coordination` | Exactly-once mechanism | When |
+|---|---|---|
+| `single` | none — always fires | dev, single pod |
+| `advisoryLock` | claims row in `_voltro_schedule_claims` | multi-instance, default on a SQL store |
+| `cluster` | `@effect/cluster` shard ownership (`ClusterCron`) | when you already run the workflow engine and want one ownership model |
+
+**`advisoryLock`** is the lightweight tier. Every replica computes the same deterministic `scheduledAt` from the cron expression (not its own `Date.now()`, so clock skew is irrelevant), derives a claim key `${scheduleName}@${secondBucket}`, and races to `INSERT` a row into `_voltro_schedule_claims`. The primary-key conflict makes exactly one replica win; the rest lose cleanly and skip. The second-precision bucket makes claims self-expiring — a crashed winner doesn't block the next firing, because the next firing is a new bucket and therefore a new key. This is deliberately a claims **row**, not a session-level lock, because a connection pool can hand a `GET_LOCK`/`pg_advisory_lock` connection to another query before the unlock.
+
+**`cluster`** rides the same sharding machinery as durable workflows: `ClusterCron` fires on exactly one runner (the shard owner) and migrates ownership on runner death. The in-app timer is disarmed in this mode (`@effect/cluster` owns the clock); the firing is recorded with `coordinationOutcome: 'cluster'`. It needs a SQL store with cluster storage (everything except `sqlite`/`memory`), and degrades to `single` with a warning when that's unavailable.
+
+Overlap policy (`onOverlap: 'skip' | 'queue' | 'parallel'`) is also cluster-aware: `skip` checks for a `status='running'` row from **any** pod, not just the local one, so a long run on pod A suppresses a new firing on pod B.
+
+## Cross-instance reactivity (per dialect)
+
+A write on instance A has to become a reactive subscription delta on instance B. The transport is change-data-capture, and it is **not** Postgres-only — each dialect uses its native change feed:
+
+| Dialect | Transport | Latency |
+|---|---|---|
+| `postgres` | `LISTEN`/`NOTIFY` | fast |
+| `mariadb` | binlog CDC (ROW image, GTID) | push-based, per-replica reader |
+| `mysql` | inline emit only (no binlog CDC) | single-process |
+| `sqlite` | in-process bus | single-process |
+
+On MariaDB the binlog **is** the message bus — each replica tails it itself, resuming from its own `_voltro_cdc_offsets` row keyed by replica id. There is no Redis, NATS, or external broker. CDC is on by default for any SQL dialect (`CDC=1`); set `CDC=0` to fall back to single-process inline emit. The boot log reports the resolved transport:
+
+```text
+[voltro:dev] sql dialect resolved: mariadb — CDC: binlog CDC (ROW), RETURNING: native (INSERT/DELETE); UPDATE then SELECT
+```
+
+## Observability
+
+- **Dashboard.** Workflow runs, steps, and schedule runs stream live to the devtools/cloud dashboard.
+- **Tables.** Introspection rows live on the shared database — query them directly:
+  - `_voltro_workflow_start_contexts` — durable starter subject/trace/source plus parent execution id and parent-close policy, loaded by whichever runner first executes the workflow or child workflow.
+  - `_voltro_workflow_runs`, `_voltro_workflow_run_steps`, `_voltro_workflow_run_events` — workflow execution history (these are fire-and-forget introspection; durability itself lives in the cluster engine's own tables).
+  - `_voltro_schedule_runs` — every firing, with `replicaId` (which instance fired it) and `coordinationOutcome` (`single` / `wonLock` / `cluster` / …).
+  - `_voltro_schedule_claims` — the live exactly-once claim rows for `advisoryLock` schedules.
+- **Logs and traces.** `voltro logs --tail 100` shows the resolved dialect, CDC flavor, coordinator, and runner identity at boot, plus every firing and resume. `voltro traces` correlates a workflow's steps across pods.
+
+### `voltro cluster status`
+
+`voltro cluster status` renders the clustering snapshot of every running api — discovered the same way as `voltro logs` / `voltro traces` (the runtime registry), then merged with the live coordination state each instance reads from the shared store:
+
+```sh
+voltro cluster status                 # pretty table of every instance
+voltro cluster status --process myApi # restrict to one registered api
+voltro cluster status --format json   # machine-readable, pipe to jq
+```
+
+For each instance it shows the `replicaId`, the runner address (`host:port`), the resolved dialect and clustering tier (full / inline-only / single-process / none), the CDC flavor, the coordination kind, and — on the MariaDB CDC path — the binlog `server_id`. A SQL-backed runner advertising `localhost` / `127.0.0.1` is flagged loudly: other pods can't reach it, so cross-pod workflow resume silently breaks (inject `POD_IP` via the K8s downward API or set `VOLTRO_WORKFLOW_RUNNER_HOST`). A footer summarises the shared coordination state — active replicas seen in recent runs, current schedule claims, and in-flight schedule + workflow runs. The same view is on the dashboard's **Cluster** panel.
+
+Quick "which instance is running what" query:
+
+```sql
+SELECT "replicaId", count(*)
+FROM "_voltro_schedule_runs"
+WHERE status = 'running'
+GROUP BY "replicaId";
+```
+
+## The multi-dialect story
+
+Clustering works on every SQL dialect, with one capability gradient:
+
+- **`postgres` / `mariadb` / `mssql`** — full clustering: durable workflow resume across instances, `advisoryLock` or `cluster` crons, native cross-instance reactivity (`LISTEN`/`NOTIFY` on postgres, binlog CDC on mariadb).
+- **`mysql`** — durable workflow resume and schedule coordination work; cross-instance reactivity is inline-only (no binlog CDC path), so subscriptions don't fan out across instances.
+- **`sqlite`** — durable workflow **replay** within a single process (`runnerStorage: 'memory'`); no horizontal scale-out.
+- **`memory`** — single-process dev store; no clustering.
+
+The same app code runs unchanged across all of them — the runtime resolves the right runner storage, coordinator, and CDC transport from the dialect at boot.
+
+> **`clusterPlugin()` — deliberately not built.** A convenience marker plugin (declaring an app "expects multi-instance", optionally pinning the runner host/port) was considered and rejected: clustering is already automatic for any SQL store with N instances, so there is no behaviour for such a plugin to toggle. Its only knobs — runner host/port — are env-driven (`VOLTRO_WORKFLOW_RUNNER_HOST` / `VOLTRO_WORKFLOW_RUNNER_PORT`) and injected by the deployment (K8s downward API), which is where pod identity belongs. The idea is on record but will not ship.
+
+## Anti-patterns
+
+- **Assuming clustering is Postgres-only.** It isn't. MariaDB is a first-class clustering target (binlog CDC), and the AWB 4-replica MariaDB deployment is the production reference. `wal_level=logical` is a Postgres detail, not a clustering requirement.
+- **Leaving the runner host on `localhost` in multi-pod deployments.** Resume-on-another-pod silently breaks — pods can't reach a `localhost` runner. Inject `POD_IP`. The runtime warns about this at boot.
+- **Reusing a `server_id` across MariaDB pods.** Duplicate `server_id` silently breaks binlog streams. Derive it from a stable per-pod `POD_NAME` (the framework does this for you).
+- **Session-level locks for cron coordination.** `advisoryLock` uses a claims **row**, not `pg_advisory_lock`/`GET_LOCK`, precisely because pooled connections make session locks unreliable. Don't reach for session locks.
+
+
+
+---
+
+<!-- source: en/workflows/flow-control.md -->
+## Flow control
+
+_Durable queues, bounded concurrency, and rate limits inside workflow code._
+
+Use flow control when the workflow is durable, but the resource it touches is limited: a third-party API, a tenant import lane, a GPU job, or a webhook fan-out.
+
+Voltro exposes the workflow engine's durable queue and rate limiter through `@voltro/workflow`:
+
+```ts
+import {
+  durableQueue,
+  processQueue,
+  queueWorker,
+  rateLimit,
+  step,
+  workflow,
+} from '@voltro/workflow'
+import { Effect, Schema } from 'effect'
+```
+
+## Bounded concurrency
+
+Define a queue once, then process items from workflow code. A worker layer controls concurrency and is backed by the same SQL/cluster workflow engine, so another runner can resume after a crash.
+
+```ts
+export const ThumbnailQueue = durableQueue({
+  name: 'thumbnails',
+  payload: {
+    imageId: Schema.String,
+    sourceUrl: Schema.String,
+  },
+  success: Schema.Struct({ thumbnailUrl: Schema.String }),
+  idempotencyKey: ({ imageId }) => imageId,
+})
+
+export const thumbnailWorker = queueWorker(
+  ThumbnailQueue,
+  ({ imageId, sourceUrl }) =>
+    step({
+      name: 'render-thumbnail',
+      input: { imageId },
+      success: Schema.Struct({ thumbnailUrl: Schema.String }),
+      execute: renderThumbnail(sourceUrl),
+    }),
+  { concurrency: 4 },
+)
+```
+
+Export the worker from any discovered `*.workflow.tsx` module. `voltro dev` and `voltro serve` auto-mount branded `queueWorker(...)` layers at boot; no `app.config.ts` layer plumbing is needed.
+
+Inside a workflow:
+
+```ts
+const result = yield* processQueue(ThumbnailQueue, {
+  imageId,
+  sourceUrl,
+})
+```
+
+`processQueue(...)` is a workflow activity. The item is persisted, the workflow parks while a worker processes it, and replay returns the recorded worker result instead of re-enqueueing.
+
+## Rate limits
+
+Use `rateLimit(...)` before a step that talks to a constrained service:
+
+```ts
+yield* rateLimit({
+  name: 'stripe-write',
+  window: '1 minute',
+  limit: 100,
+  key: tenantId,
+})
+
+const invoice = yield* step({
+  name: 'create-invoice',
+  input: { tenantId, orderId },
+  execute: createStripeInvoice(orderId),
+})
+```
+
+The wait uses the durable workflow clock. A deploy or worker crash does not lose the delay.
+
+## Dashboard behavior
+
+Queue workers and rate-limited steps still record normal workflow steps, timers, and run events. The Workflows dashboard shows the parked run, the step that is waiting, and the later continuation through the same live run/step/event subscriptions. The Flow tab groups visible runs by workflow lane, queued/running/waiting status, and start source so pressure is visible in both local devtools and Voltro Cloud.
+
+## Rules of thumb
+
+- Use a durable queue for **bounded concurrency** and per-resource lanes.
+- Use `rateLimit(...)` for **API budgets** where excess work should wait, not fail.
+- Put the limiting key in your payload or workflow state so replay is deterministic.
+- Keep external I/O inside `step(...)` or the queue worker body.
+
+
+
+---
+
+<!-- source: en/workflows/versioning.md -->
+## Versioning
+
+_Workflow definition versions, compatibility metadata, and patch markers for long-running runs._
+
+Long-running workflows can outlive a deploy. Voltro does not run old JavaScript forever; a resumed run executes the current code. Make that explicit by versioning the workflow definition.
+
+```ts
+import { workflow } from '@voltro/workflow'
+import { Schema } from 'effect'
+
+export const ImportCustomers = workflow({
+  name: 'customers.import',
+  payload: { uploadId: Schema.String },
+  success: Schema.Struct({ imported: Schema.Number }),
+  idempotencyKey: ({ uploadId }) => uploadId,
+  version: 3,
+  compatibleWith: [2, 3],
+  patches: ['split-validate-and-write'],
+})
+```
+
+Voltro stores `workflowVersion` and `workflowPatches` on every `_voltro_workflow_runs` row when the run starts. The local devtools and Voltro Cloud dashboard show the version chip on run rows, so operators can spot old or incompatible runs during a deploy.
+
+## Compatibility
+
+`compatibleWith` is operator metadata. Use it to document which run versions the current code can still resume safely. If a change cannot resume old payloads or step layout, bump `version` and leave the old version out of `compatibleWith`.
+
+## Patch Markers
+
+`patches` are named change markers. Keep them short and stable:
+
+```ts
+patches: [
+  'split-validate-and-write',
+  'rename-export-step',
+]
+```
+
+Patch markers are recorded with the run. They make deploy review and dashboard triage concrete: you can answer which branching changes were active when the run started.
+
+## Rules
+
+- Bump `version` for payload shape changes, step-order changes that affect replay, or changed external side-effect semantics.
+- Keep old payload decoders inside the workflow body only while their version remains compatible.
+- Prefer additive payload changes with defaults over breaking changes.
+- Use the dashboard version chip during deploys to find runs that started on an older contract.
+
+
+
+---
+
+<!-- source: en/workflows/debugging.md -->
+## Debugging & inspection
+
+_The dashboard's Workflows panel, inspect endpoints, CLI controls, workflow tables, and testing helpers._
+
+Voltro records workflow runs, step attempts, and lifecycle events into framework tables. The dashboard, CLI, and inspect endpoints all read the same data.
+
+## What is recorded
+
+| Table | Contents |
+|---|---|
+| `_voltro_workflow_start_contexts` | One row per started execution id: starter subject, trace id, source, parent execution id, parent-close policy, and creation time. Used for cross-runner context handoff. |
+| `_voltro_workflow_runs` | One row per run: `id`, `tag`, `executionId`, `status`, `payload`, `workflowVersion`, `workflowPatches`, `output`, error fields, subject, start source, timing, trace id, parent execution id, parent-close policy. |
+| `_voltro_workflow_run_steps` | One row per step attempt: step name, attempt number, recorded input, retry metadata, output or error, duration. |
+| `_voltro_workflow_run_events` | Lifecycle events: `run-started`, `run-succeeded`, `run-failed`, `run-suspended`, `run-resumed`, `run-cancelled`, `timer-set`, `timer-fired`, `signal-awaited`, `signal-sent`, `signal-received`, `update-requested`, `update-received`, `update-completed`, `update-failed`. |
+| `_voltro_workflow_events` | Domain events emitted through `ctx.events.emit(...)`: event id, name, payload, source, subject, trace id, occurred time. |
+| `_voltro_workflow_event_deliveries` | One row per workflow trigger delivery: event id, trigger id, workflow name, execution id, idempotency key, status, error. |
+
+Run status is `running`, `succeeded`, `failed`, `cancelled`, or `suspended`.
+
+## Dashboard
+
+When `voltro dev` is running, the Workflows panel lists recent runs, their status, start source, timing, payload, output/error, step attempts, and events. The run detail view is the fastest way to answer:
+
+- Which step is currently running?
+- Was the run started by an RPC, `ctx.workflows`, a schedule, an incoming webhook, or inspect tooling?
+- Is this run a child, and will it be cancelled, terminated, or abandoned when the parent closes?
+- Which attempt failed?
+- What input did the step receive?
+- What output or typed error did it produce?
+- Did a timer or signal fire?
+
+The panel also exposes run controls for users with the right capability: cancel, retry, suspend, resume, send signal, and send a tracked update.
+
+Run filters can be saved as named views. The selected view and ad-hoc
+filters are mirrored into the URL query string, so a teammate can open the
+same filtered run list. The Workflows panel also includes dedicated
+Incoming and Flow tabs: Incoming groups runs whose source is
+`incoming:<id>` beside incoming-sourced domain events, while Flow groups
+queued/running/waiting work by workflow lane and start source.
+
+## CLI
+
+```sh
+voltro workflows list --status running --tail 50
+voltro workflows list --tag notes.summarise --format json
+voltro workflows start notes.summarise --payload '{"noteId":"note_123"}'
+voltro workflows show wfrun_01H...
+voltro workflows retry wfrun_01H...
+voltro workflows cancel wfrun_01H...
+voltro workflows suspend wfrun_01H...
+voltro workflows resume wfrun_01H...
+voltro workflows signal wfrun_01H... --name approval --payload '{"approved":true}'
+voltro workflows update wfrun_01H... --name approve --payload '{"decision":true}'
+voltro workflows children exec_01H...
+```
+
+The CLI discovers live API processes the same way as `voltro logs` and `voltro traces`, then calls inspect endpoints.
+
+## Inspect endpoints
+
+```http
+GET  /_voltro/inspect/workflows/runs
+GET  /_voltro/inspect/workflows/runs?tag=notes.summarise&status=failed
+GET  /_voltro/inspect/workflows/runs/:id/steps
+GET  /_voltro/inspect/workflows/runs/:id/events
+GET  /_voltro/inspect/workflows/events
+GET  /_voltro/inspect/workflows/events/:id/deliveries
+GET  /_voltro/inspect/workflows/children?parentExecutionId=exec_01H...
+
+POST /_voltro/inspect/workflows/runs/:id/cancel
+POST /_voltro/inspect/workflows/runs/:id/retry
+POST /_voltro/inspect/workflows/runs/:id/suspend
+POST /_voltro/inspect/workflows/runs/:id/resume
+POST /_voltro/inspect/workflows/runs/:id/signal
+POST /_voltro/inspect/workflows/runs/:id/update
+```
+
+Signal body:
+
+```json
+{
+  "signalName": "approval",
+  "payload": { "approved": true }
+}
+```
+
+Update body:
+
+```json
+{
+  "updateName": "approve",
+  "payload": { "decision": true },
+  "timeoutMs": 30000
+}
+```
+
+Retry normally uses the original payload. The inspect handler also supports payload override for operator tooling.
+
+## SQL inspection
+
+Recent failed runs:
+
+```sql
+SELECT id, tag, status, source, "errorTag", "errorMessage", "startedAt", "completedAt"
+FROM "_voltro_workflow_runs"
+WHERE status = 'failed'
+ORDER BY "startedAt" DESC
+LIMIT 50;
+```
+
+Slow steps:
+
+```sql
+SELECT "stepName", percentile_cont(0.95) WITHIN GROUP (ORDER BY "durationMs")
+FROM "_voltro_workflow_run_steps"
+WHERE status = 'succeeded'
+  AND "startedAt" > now() - interval '24 hours'
+GROUP BY "stepName"
+ORDER BY 2 DESC
+LIMIT 10;
+```
+
+Events for one run:
+
+```sql
+SELECT "eventType", payload, "occurredAt"
+FROM "_voltro_workflow_run_events"
+WHERE "runId" = 'wfrun_01H...'
+ORDER BY "occurredAt" ASC;
+```
+
+## Inspecting from code
+
+`inspectWorkflow(idOrExecutionId, store)` assembles one run into the same shape used by the dashboard detail view:
+
+```ts
+import { inspectWorkflow } from '@voltro/workflow'
+
+const state = await inspectWorkflow(runId, ctx.store)
+```
+
+It groups step attempts by step name, so a step that failed twice and succeeded on attempt three reports `attempts: 3` with the latest output/error surfaced.
+
+## Testing workflows
+
+Use `makeWorkflowRunner` from `@voltro/testing` to run a workflow in-process over the in-memory workflow engine and an in-memory recorder:
+
+```ts
+import { makeTestContext, makeWorkflowRunner } from '@voltro/testing'
+import { SummariseNote } from '../workflows/notes.summarise.workflow'
+import buildSummariseNote from '../workflows/notes.summarise.workflow'
+
+test('notes.summarise retries the LLM step', async () => {
+  const ctx = makeTestContext()
+  const runner = makeWorkflowRunner({
+    ctx,
+    workflows: [{
+      workflow: SummariseNote as never,
+      execute: buildSummariseNote(ctx) as never,
+    }],
+  })
+
+  const result = await runner.start('notes.summarise', { noteId: 'n_1' })
+  expect(result.status).toBe('succeeded')
+  expect(result.steps.find((s) => s.name === 'summarise-with-llm')?.attempts).toBe(3)
+})
+```
+
+The result includes `{ status, output, error, steps, runId }`. `runner.inspect(runId)` returns the assembled run later without re-running it.
+
+## Anti-patterns
+
+- **Querying old table names.** Use `_voltro_workflow_runs`, `_voltro_workflow_run_steps`, and `_voltro_workflow_run_events`.
+- **Filtering for `status=dead`.** Current failed runs use `status=failed`.
+- **Expecting replay-from-step APIs.** Current operator retry starts a new execution from the workflow payload; completed steps are not selectively replayed through a public API.
+- **Logging giant step inputs/outputs.** Step input/output is persisted for inspection. Store references to huge blobs instead of the blob itself.
+
+
+
+---
+
+<!-- source: en/workflows/event-triggers.md -->
+## Event triggers
+
+_defineEventTrigger — bind a domain event name to a workflow. ctx.events.emit(name, data) fans the event out to every matching trigger with filter, payload-mapping, and idempotency._
+
+A workflow is usually kicked off directly (`ctx.workflows.start(...)`). An **event trigger** decouples that: a `*.trigger.tsx` file binds a domain **event name** to a **workflow name**, and any handler that calls `ctx.events.emit('event.name', data)` fans the event out to every trigger listening for it. The emitter never names the workflow — "something happened" is separated from "run this workflow", so you add reactions without touching the code that emits.
+
+```tsx
+// triggers/orderPaid.trigger.tsx
+import { defineEventTrigger } from '@voltro/runtime'
+
+export default defineEventTrigger({
+  event:    'order.paid',                    // the event name to listen for
+  workflow: 'fulfilment.run',                // the workflow to start
+  // Optional: skip when the predicate returns false.
+  filter:  (e) => e.data.total > 0,
+  // Optional: map the event envelope → the workflow's payload (default: e.data).
+  payload: (e) => ({ orderId: e.data.orderId, tenantId: e.data.tenantId }),
+  // Optional: dedup key — two emits with the same key start the workflow once.
+  idempotencyKey: (e) => `fulfil:${e.data.orderId}`,
+})
+```
+
+Discovery walks every `*.trigger.tsx`; the `default` export must be a `defineEventTrigger(...)` descriptor (one per file). The workflow runtime is active whenever the app has any workflow OR any trigger.
+
+## Emitting an event
+
+From any handler, emit through `ctx.events`:
+
+```ts
+const execute = async (input: { orderId: string }, ctx: AppContext) => {
+  // ... mark the order paid ...
+  const result = await ctx.events.emit('order.paid', {
+    orderId:  input.orderId,
+    tenantId: ctx.request.subject.tenantId,
+    total:    order.total,
+  })
+  // result.eventId — the persisted event id
+  // result.triggered — one entry per matched trigger:
+  //   { triggerId, workflowName, status: 'started' | 'skipped' | 'failed', run?, reason? }
+}
+```
+
+`emit(name, data, options?)` accepts `{ id?, source? }` (override the event id / source label). It returns `{ eventId, triggered }` as soon as the workflows are kicked off — the workflows themselves run durably in the background, exactly like `ctx.workflows.start`.
+
+The event envelope every `filter` / `payload` / `idempotencyKey` receives is:
+
+```ts
+{ id, name, data, occurredAt, source, subject?, traceId? }
+```
+
+`subject` + `traceId` are inherited from the emitting request, so a triggered workflow continues the same trace and carries the same actor.
+
+## Matching + delivery
+
+- **Exact + wildcard.** A trigger matches its exact `event` name; a trigger registered for `'*'` matches every event (and runs in addition to exact matches).
+- **Idempotency.** Before starting, the runtime checks for an existing delivery with the same `idempotencyKey` (default `<eventId>:<triggerId>`) — a duplicate is recorded as `status: 'skipped', reason: 'duplicate'` and the workflow is not started twice.
+- **Every emit is audited.** The event itself lands in `_voltro_workflow_events`; one row per trigger lands in `_voltro_workflow_event_deliveries` with `status` (`starting → started`, or `skipped` / `failed`), the started `executionId`, and any error message. Both tables are reactive — the dashboard's Workflows view surfaces deliveries live.
+
+## When to use it
+
+Reach for an event trigger when ONE thing happening should fan out to several independent reactions, or when you want the emitter to stay ignorant of the consumers:
+
+- `order.paid` → start fulfilment AND a receipt-email workflow AND an analytics rollup — three triggers, one `emit`.
+- A mutation emits `user.signedUp`; an onboarding workflow trigger starts the drip sequence. The signup mutation never imports the onboarding workflow.
+
+If the handler already knows exactly which workflow to run and there's only one, call [`ctx.workflows.start(...)`](/docs/workflows/definition) directly — the trigger indirection only pays off when you're decoupling emit from react, or fanning one event out to many.