workflow 5.0.0-beta.2 → 5.0.0-beta.21

package/docs/deploying/world/vercel-world.mdx

@@ -139,6 +139,8 @@ On Vercel, workflow runs are pegged to the deployment that started them. This me
 
 This ensures long-running workflows complete reliably without being affected by subsequent deployments.
 
+For the full model, including rerunning on latest and explicit upgrade boundaries, see [Versioning](/docs/foundations/versioning).
+
 ## Security
 
 ### Consumer function security

package/docs/errors/abort-signal-timeout-in-workflow.mdx

@@ -0,0 +1,80 @@
+---
+title: abort-signal-timeout-in-workflow
+description: AbortSignal.timeout() cannot be used inside workflow functions because it relies on real timers which break deterministic replay.
+type: troubleshooting
+summary: Use sleep() with AbortController instead of AbortSignal.timeout() in workflow functions.
+prerequisites:
+  - /docs/foundations/workflows-and-steps
+related:
+  - /docs/foundations/cancellation
+  - /docs/api-reference/workflow/sleep
+  - /docs/errors/timeout-in-workflow
+---
+
+## Error
+
+```
+AbortSignal.timeout() is not supported in workflow functions.
+Use sleep() with an AbortController instead.
+```
+
+## Why This Happens
+
+`AbortSignal.timeout()` creates a signal that aborts after a real-time delay using an internal timer. Workflow functions must be [deterministic](/docs/foundations/workflows-and-steps) to support replay — they run the same code multiple times during the workflow's lifecycle, using the [event log](/docs/how-it-works/event-sourcing) to resume execution to the correct point.
+
+Real-time timers break this determinism because:
+- On the first execution, the timer might fire after 10 seconds
+- On replay, the timer would fire again, but the event log may have already advanced past that point
+- The timer's behavior depends on wall-clock time, which varies between executions
+
+## How to Fix
+
+Use [`sleep()`](/docs/api-reference/workflow/sleep) with an `AbortController` to create a deterministic timeout that cancels in-flight work:
+
+**Before (incorrect):**
+
+{/* @skip-typecheck: intentionally incorrect example */}
+```typescript lineNumbers
+export async function workflow() {
+  "use workflow";
+
+  // This will throw an error
+  const signal = AbortSignal.timeout(10_000); // [!code highlight]
+  const result = await fetchData(signal);
+  return result;
+}
+```
+
+**After (correct):**
+
+```typescript lineNumbers
+import { sleep } from "workflow";
+
+export async function workflow() {
+  "use workflow";
+
+  const controller = new AbortController(); // [!code highlight]
+  void sleep("10s").then(() => controller.abort()); // [!code highlight]
+
+  return await fetchData(controller.signal);
+}
+
+async function fetchData(signal: AbortSignal) {
+  "use step";
+  const response = await fetch("https://api.example.com/data", { signal });
+  return response.json();
+}
+```
+
+The `sleep()` + `AbortController` pattern is the durable equivalent of `AbortSignal.timeout()`. The sleep is recorded in the event log, so it replays deterministically. If `fetchData` finishes within 10 seconds you get the response; if not, the timer fires `controller.abort()`, `fetch` rejects with an `AbortError`, and the step's failure propagates to the workflow as a `FatalError` (no retries — abort is intentional cancellation).
+
+<Callout type="info">
+`AbortSignal.timeout()` works normally inside step functions, since steps have full Node.js runtime access and are not replayed.
+</Callout>
+
+## Related
+
+- [Cancellation](/docs/foundations/cancellation) — Patterns for cancelling in-flight work
+- [`sleep()` API Reference](/docs/api-reference/workflow/sleep) — Durable sleep primitive
+- [Workflows and Steps](/docs/foundations/workflows-and-steps) — Why workflow functions must be deterministic
+- [`setTimeout` in Workflow](/docs/errors/timeout-in-workflow) — Similar restriction on `setTimeout`

package/docs/errors/corrupted-event-log.mdx

@@ -9,21 +9,21 @@ related:
   - /docs/foundations/errors-and-retries
 ---
 
-This error occurs when the Workflow runtime encounters an event in the event log that no registered consumer can process. This means the event log is in an invalid state — typically due to duplicate or orphaned events.
+This error occurs when the Workflow runtime repeatedly cannot replay events in the event log. This usually means the event log is in an invalid state, such as duplicate or orphaned events, or that a runtime determinism bug persists across retry attempts.
 
-This is a **workflow-level fatal error**. It cannot be caught or handled inside your workflow code. A corrupted event log immediately fails the entire run without executing any more user code. The run must be retried from outside the workflow.
+This is a **workflow-level fatal error**. It cannot be caught or handled inside your workflow code. The runtime first retries transient replay divergence automatically; it marks the run as failed with this error only after replay still cannot recover.
 
 ## Error Message
 
 ```
-Unconsumed event in event log: eventType=<type>, correlationId=<id>, eventId=<id>. This indicates a corrupted or invalid event log.
+Workflow replay diverged <divergenceCount> times after <maxRecoveryReplays> recovery replays; latest divergent event was <eventId>. Last divergence: <details>
 ```
 
 ## Why This Happens
 
 Workflows persist their progress as an ordered event log. During replay, the runtime processes each event in sequence — every event must be consumed by a matching callback (e.g., a step or sleep waiting for its result). When an event has no matching consumer, the runtime cannot advance past it, which would block all subsequent events and hang the workflow indefinitely.
 
-Instead of silently hanging, the runtime raises a `WorkflowRuntimeError` to fail the workflow fast and surface the problem.
+Instead of silently hanging, the runtime retries a divergent replay before failing the workflow and surfacing this terminal error.
 
 Common scenarios that produce this error:
 
@@ -45,7 +45,7 @@ npm install workflow@latest
 
 ### 2. Retry the failed run
 
-Since this is a fatal error, the run is automatically marked as `failed`. You can re-run it using the **Re-run** button in the Workflow Dashboard.
+If this error is displayed, automatic replay recovery has already been exhausted and the run has been marked as `failed`. You can re-run it using the **Re-run** button in the Workflow Dashboard.
 
 ### 3. Report the issue
 

package/docs/errors/hook-conflict.mdx

@@ -73,9 +73,9 @@ export async function processPayment() {
 }
 ```
 
-## Handling Hook Conflicts in Your Workflow
+## Handling Hook Conflicts
 
-When a hook conflict occurs, awaiting the hook will throw a `HookConflictError`. You can catch this error to handle the conflict gracefully:
+When a hook conflict occurs, awaiting the hook will throw a `HookConflictError`. The error exposes the token that conflicted and, for current worlds, the run ID that currently owns it. `conflictingRunId` remains optional for compatibility with older persisted events and world implementations, so guard it before delegating:
 
 ```typescript lineNumbers
 import { createHook } from "workflow";
@@ -93,14 +93,64 @@ export async function processPayment(orderId: string) {
     if (HookConflictError.is(error)) { // [!code highlight]
       // Another workflow is already processing this order
       console.log(`Conflicting token: ${error.token}`);
-      return { success: false, reason: "duplicate-processing" };
+      if (error.conflictingRunId) {
+        console.log(`Active run: ${error.conflictingRunId}`);
+      }
+      return {
+        success: false,
+        reason: "duplicate-processing",
+        token: error.token,
+        runId: error.conflictingRunId
+      };
     }
     throw error; // Re-throw other errors
   }
 }
 ```
 
-This pattern is useful when you want to detect and handle duplicate processing attempts instead of letting the workflow fail.
+This pattern is useful when you want to detect duplicate processing inside the workflow. Runtime APIs such as `resumeHook()` and `getRun()` must be called outside workflow functions, for example from an API route or in a step.
+
+### Delegate to the Active Run
+
+In idempotency flows, a conflict means another active run already owns the hook token. You can return the duplicate-processing payload from the workflow, resume the active hook to deliver the payload to the existing run, then use `getRun(result.runId)` to wait for, stream, or cancel the active run:
+
+```typescript lineNumbers
+import { getRun, resumeHook, start } from "workflow/api";
+import { processPayment } from "@/workflows/process-payment";
+
+type ProcessPaymentResult =
+  | { success: true; payment: unknown }
+  | {
+      success: false;
+      reason: "duplicate-processing";
+      token: string;
+      runId?: string;
+    };
+
+export async function POST(request: Request) {
+  const { orderId, payment } = await request.json();
+  const run = await start(processPayment, [orderId]);
+  const result = (await run.returnValue) as ProcessPaymentResult;
+
+  if (
+    result.success === false &&
+    result.reason === "duplicate-processing" &&
+    result.runId
+  ) {
+    await resumeHook(result.token, payment); // [!code highlight]
+    const activeRun = getRun(result.runId); // [!code highlight]
+
+    return Response.json({
+      delegatedToRunId: activeRun.runId,
+      result: await activeRun.returnValue
+    });
+  }
+
+  return Response.json(result);
+}
+```
+
+If the caller needs live output instead of the final result, return `activeRun.getReadable()` from the same branch. If the duplicate request should replace the active work, call `await activeRun.cancel()` after inspecting the run.
 
 ## When Hook Tokens Are Released
 
@@ -122,4 +172,6 @@ After a workflow completes, its hook tokens become available for reuse by other
 ## Related
 
 - [Hooks](/docs/foundations/hooks) - Learn more about using hooks in workflows
+- [getRun](/docs/api-reference/workflow-api/get-run) - Retrieve or control the active run
+- [resumeHook](/docs/api-reference/workflow-api/resume-hook) - Deliver data to the active hook
 - [createWebhook](/docs/api-reference/workflow/create-webhook) - Alternative for fixed webhook URLs

package/docs/errors/index.mdx

@@ -37,12 +37,21 @@ Fix common mistakes when creating and executing workflows in the **Workflow SDK*
     <Card href="/docs/errors/corrupted-event-log" title="corrupted-event-log">
         Learn how to handle corrupted or invalid event logs.
     </Card>
+    <Card href="/docs/errors/replay-divergence" title="replay-divergence">
+        Learn how workflow replay divergence is recovered automatically.
+    </Card>
     <Card href="/docs/errors/step-not-registered" title="step-not-registered">
         Resolve step not registered errors caused by deployment mismatches.
     </Card>
+    <Card href="/docs/errors/step-executed-multiple-times" title="Step executed multiple times">
+        Diagnose duplicate step_started events from function crashes, timeouts, or OOMs.
+    </Card>
     <Card href="/docs/errors/workflow-not-registered" title="workflow-not-registered">
         Resolve workflow not registered errors caused by deployment mismatches.
     </Card>
+    <Card href="/docs/errors/runtime-decryption-failed" title="runtime-decryption-failed">
+        Resolve runtime decryption failures from the SDK's encryption layer.
+    </Card>
 </Cards>
 
 ## Learn More

package/docs/errors/replay-divergence.mdx

@@ -0,0 +1,27 @@
+---
+title: replay-divergence
+description: A workflow replay temporarily followed a path that did not match its recorded events.
+type: troubleshooting
+summary: Understand automatic recovery when a workflow replay diverges from its event history.
+prerequisites:
+  - /docs/foundations/workflows-and-steps
+related:
+  - /docs/errors/corrupted-event-log
+  - /docs/foundations/errors-and-retries
+---
+
+A replay divergence occurs when one invocation of a workflow cannot consume the durable event history using the promises, hooks, sleeps, or steps it created during replay.
+
+This is an SDK/runtime signal, not an error thrown by your workflow code. It is not catchable inside a workflow function.
+
+## Automatic Recovery
+
+A single divergent replay does not prove that persisted history is corrupted. For example, asynchronous delivery ordering may cause one invocation to follow the wrong side of a race while another replay can follow the recorded history correctly.
+
+The runtime automatically queues another replay when an invocation reports `REPLAY_DIVERGENCE`. No terminal `run_failed` event is written during these recovery attempts.
+
+If recovery replays continue to diverge after the retry budget is exhausted, the runtime marks the run as failed with `CORRUPTED_EVENT_LOG` and records the latest divergent event for diagnosis.
+
+## What To Do
+
+Most replay divergence signals recover without action. If a run ultimately fails with `CORRUPTED_EVENT_LOG`, update to the latest `workflow` package and report the run ID and error details if the failure persists.

package/docs/errors/runtime-decryption-failed.mdx

@@ -0,0 +1,77 @@
+---
+title: runtime-decryption-failed
+description: The SDK's built-in AES-GCM encryption layer failed to encrypt or decrypt a workflow payload.
+type: troubleshooting
+summary: Resolve runtime decryption failures caused by ciphertext corruption, key mismatch, or malformed envelopes.
+prerequisites:
+  - /docs/foundations/workflows-and-steps
+related:
+  - /docs/foundations/errors-and-retries
+---
+
+This error occurs when the Workflow SDK's built-in AES-GCM encryption layer fails while encrypting or decrypting a workflow payload. The SDK encrypts step inputs, step outputs, hook payloads, and other event-log data with a per-run AES-256 key whenever encryption is configured for the deployment.
+
+This is an **internal SDK failure** — your workflow code never invokes the encryption primitives directly. When this surfaces, it means the ciphertext, nonce, or auth tag the SDK tried to verify is not the bytes that were originally produced. The run is failed with the `RUNTIME_ERROR` classification.
+
+## Error Message
+
+```
+AES-256-GCM decryption failed: The operation failed for an operation-specific reason
+```
+
+The underlying cause is a native Web Crypto [`OperationError`](https://developer.mozilla.org/en-US/docs/Web/API/DOMException#operationerror) — most commonly raised by `AESCipherJob.onDone` in Node's `node:internal/crypto/util` module when the GCM authentication tag does not verify.
+
+The thrown `RuntimeDecryptionError` carries a small `context` object with diagnostic fields to help triangulate the source:
+
+- `operation` — `'encrypt'` or `'decrypt'`
+- `byteLength` — total byte length of the payload at the failure site
+- `formatPrefix` — the first 4 bytes of the input (`'encr'` for a well-formed encrypted envelope, otherwise a hex dump)
+
+## Why This Happens
+
+Common causes, in rough order of likelihood:
+
+1. **Ciphertext mutation or truncation in transit.** The encrypted payload reached the SDK with bytes that differ from what storage holds. Possible sources include a truncated HTTP response from a workflow-server ref endpoint, an edge-cache miss returning a partial 200, or a proxy drop during streaming. A truncated body whose first 4 bytes happen to still spell `encr` produces the exact "auth tag mismatch" symptom.
+2. **Key resolution mismatch.** The key used to decrypt is not the key that was used to encrypt — e.g. the run's `deploymentId` was not threaded through key resolution and the SDK fell back to the wrong deployment's key material.
+3. **Malformed encrypted envelope.** The envelope is too short to contain the GCM nonce (12 bytes) and auth tag (16 bytes), so decryption is rejected before it begins.
+
+## What To Do
+
+This error indicates an SDK or infrastructure problem — not a bug in your workflow code. Your workflow code does not need to change.
+
+### 1. Upgrade to the latest `workflow` package
+
+The underlying issue may have already been identified and fixed:
+
+```bash
+npm install workflow@latest
+```
+
+### 2. Retry the failed run
+
+Since this is a fatal error, the run is automatically marked as `failed`. You can re-run it using the **Re-run** button in the Workflow Dashboard.
+
+### 3. Report the issue
+
+If the error persists after upgrading, please [open an issue on GitHub](https://github.com/vercel/workflow/issues/new) so we can investigate. Include:
+
+- The version of the `workflow` package you are using
+- The run ID(s) of the affected workflow run(s)
+- The full error message, including the `context` fields (`operation`, `byteLength`, `formatPrefix`)
+- Whether the affected workflows make heavy use of large step inputs/outputs (which may indicate the failure is on the lazy-loaded ref read path)
+
+## This Error Cannot Be Caught
+
+Like other `WorkflowRuntimeError` subclasses, a runtime decryption failure is **not catchable** inside your workflow function. The runtime cannot safely continue executing user code when an event-log payload can't be verified, so the entire run fails immediately and is marked as `failed`.
+
+To handle this programmatically from outside the workflow, check the run status:
+
+```typescript lineNumbers
+import { getRun } from "workflow/api";
+
+const run = getRun("wrun_abc123");
+const status = await run.status;
+if (status === "failed") {
+  console.error("Run failed");
+}
+```

package/docs/errors/step-executed-multiple-times.mdx

@@ -0,0 +1,23 @@
+---
+title: Step executed multiple times
+description: A step ran more than once because its function invocation crashed before it could report a result.
+type: troubleshooting
+summary: Diagnose duplicate step_started events caused by function timeouts, OOMs, or network issues.
+prerequisites:
+  - /docs/foundations/workflows-and-steps
+related:
+  - /docs/observability
+  - /docs/foundations/errors-and-retries
+---
+
+There may be cases where you see multiple `step_started` events for the same step in a workflow run. This happens if the function invocation executing the step crashes unexpectedly, and the step can not report the error. The step will be re-tried according to your retry policy in this case, but no error will be visible in the [Observability UI](/docs/observability).
+
+## Common Causes
+
+- **Function timeouts**: if your step code runs longer than the configured maximum function duration, it will be killed. Compare the gap between the `step_started` events to your configured function duration to be sure.
+- **Out of memory (OOM)**: if your step code loads enough data into memory, especially if the step is invoked concurrently, the function invocation might run out of memory. You can see your function's peak memory use by going to the [Observability Query page](https://vercel.com/docs/observability) and showing the **Function Invocation Peak Memory** metric, then filtering down the **Route** to `/.well-known/workflow` endpoints.
+- **Network issues**: persistent firewall, network stability, and related issues might prevent your function from reporting results or errors. This should be temporary.
+
+## Getting Help
+
+If you consistently see multiple `step_started` events and have ruled out function timeouts, OOMs, and firewall issues, please [contact support](https://vercel.com/help).

package/docs/errors/step-not-registered.mdx

@@ -4,7 +4,7 @@ description: A step function is not registered in the current deployment.
 type: troubleshooting
 summary: Resolve step not registered errors caused by build issues.
 prerequisites:
-  - /docs/foundations/steps
+  - /docs/foundations/workflows-and-steps
 related:
   - /docs/errors/workflow-not-registered
   - /docs/api-reference/workflow-errors/step-not-registered-error