workflow 5.0.0-beta.41 → 5.0.0-beta.43

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -22,6 +22,10 @@ Where a standard chat implementation would require the user to resend their mess
22
22
 
23
23
  Resumable streams come out of the box with Workflow SDK, however, the client needs to recognize that a stream exists, and needs to know which stream to reconnect to, and needs to know where to start from. For this, Workflow SDK provides the [`WorkflowChatTransport`](/docs/api-reference/workflow-ai/workflow-chat-transport) helper, a drop-in transport for the AI SDK that handles client-side resumption logic for you.
24
24
 
25
+ <Callout type="info">
26
+ When deploying a streaming route to Vercel, enable request cancellation so a browser disconnect terminates that route's abandoned stream reader instead of letting the function run until `FUNCTION_INVOCATION_TIMEOUT`. See [Avoiding Function Timeouts After Client Disconnects](/docs/foundations/streaming#avoiding-function-timeouts-after-client-disconnects).
27
+ </Callout>
28
+
25
29
  ## Implementing stream resumption
26
30
 
27
31
  Let's add stream resumption to our Flight Booking Agent that we build in the [Building Durable AI Agents](/docs/ai) guide.
@@ -61,7 +61,7 @@ Throws [`HookNotFoundError`](/docs/api-reference/workflow-errors/hook-not-found-
61
61
  ## Usage Note
62
62
 
63
63
  <Callout type="warn">
64
- In most cases, you should not need to call `resumeWebhook()` directly. When you use `createWebhook()`, the framework automatically generates a random webhook token and provides a public URL at `/.well-known/workflow/v1/webhook/:token`. External systems can send HTTP requests directly to that URL.
64
+ In most cases, you should not need to call `resumeWebhook()` directly. When you use `createWebhook()`, the framework automatically generates a webhook token and provides a public URL at `/.well-known/workflow/v1/webhook/:token`. External systems can send HTTP requests directly to that URL.
65
65
 
66
66
  For server-side hook resumption with deterministic tokens, use [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook) with [`createHook()`](/docs/api-reference/workflow/create-hook) instead.
67
67
  </Callout>
@@ -8,9 +8,9 @@ related:
8
8
  - /docs/api-reference/workflow-errors/entity-conflict-error
9
9
  ---
10
10
 
11
- `PreconditionFailedError` is thrown by world implementations when an event creation is rejected because the client's event-log snapshot is stale either a newer out-of-band event (such as a received hook or a completed step) was recorded after the snapshot the client replayed from, or the snapshot is missing an event recorded at or before it. It corresponds to HTTP 412 Precondition Failed semantics.
11
+ `PreconditionFailedError` is thrown by world implementations when an event creation is rejected because the client's event-log snapshot is stale: the log already held more events than the position the creation named. It corresponds to HTTP 412 Precondition Failed semantics.
12
12
 
13
- This only occurs while the optimistic-concurrency guard is enabled (`WORKFLOW_PRECONDITION_GUARD`, on by default see [Runtime Tuning](/docs/configuration/runtime-tuning)); event creations that carry no snapshot are never rejected with this error.
13
+ No world in this repository throws it. A stale replay does not need to be refused: its log is a prefix rather than a prefix with a hole in it, replay is deterministic on a prefix, and the write it makes next comes back carrying the events it was pushed past (see [Stale reads](/docs/configuration/runtime-tuning#stale-reads-and-why-nothing-has-to-be-rejected)). The error and the runtime's handling of it remain for a world that would rather refuse than report — one that allocates positions somewhere other than the commit, and so cannot report a gap reliably. Event creations that carry no position are never rejected with it.
14
14
 
15
15
  A world rejects only on evidence and accepts the creation whenever it cannot decide, so this error always means the snapshot really was stale — but not receiving it does not prove the snapshot was current.
16
16
 
@@ -22,6 +22,10 @@ keywords:
22
22
 
23
23
  `world.analytics` is an optional, read-only namespace for observability surfaces — dashboards, CLIs, and admin tools that list large numbers of runs without touching payload data.
24
24
 
25
+ For observability and inspection listings, prefer this namespace over
26
+ [`world.runs.list()`](/docs/api-reference/workflow-runtime/world/storage#runslist).
27
+ The storage API remains available for operational and payload-bearing reads.
28
+
25
29
  It differs from [Storage](/docs/api-reference/workflow-runtime/world/storage) in two ways:
26
30
 
27
31
  - **Metadata only.** Results never include run input/output, step data, or hook tokens. There is no `resolveData` option.
@@ -133,7 +133,10 @@ const result = await world.events.listByCorrelationId({ // [!code highlight]
133
133
 
134
134
  ## world.runs
135
135
 
136
- Materialized from run events. Use it to list and inspect workflow runs.
136
+ Materialized from run events. Use it for canonical operational reads, including
137
+ reads that require workflow input or output data. For observability dashboards,
138
+ inspection tools, and historical listings, use
139
+ [`world.analytics.runs.list()`](/docs/api-reference/workflow-runtime/world/analytics#runslist).
137
140
 
138
141
  ### runs.get()
139
142
 
@@ -163,6 +166,14 @@ const result = await world.runs.list({ // [!code highlight]
163
166
 
164
167
  **Returns:** `{ data: WorkflowRun[], cursor?: string }`
165
168
 
169
+ <Callout type="warn">
170
+ Observability and inspection usage of `world.runs.list()` is deprecated. Use
171
+ [`world.analytics.runs.list()`](/docs/api-reference/workflow-runtime/world/analytics#runslist)
172
+ for metadata-only, plan-aware queries backed by the observability pipeline.
173
+ `world.runs.list()` remains supported for operational and payload-bearing
174
+ reads.
175
+ </Callout>
176
+
166
177
  ### Cancelling Runs
167
178
 
168
179
  To cancel a run, create a `run_cancelled` event via `world.events.create()` (see [world.events](#worldevents) above), or use the CLI or Web UI helpers.
@@ -0,0 +1,79 @@
1
+ ---
2
+ title: Batched event writes
3
+ description: An optional World API (events.createBatch) that appends an ordered set of events in one durable write with per-event outcomes, and a suspension fan-out fold that uses it.
4
+ ---
5
+
6
+ # Batched event writes (`events.createBatch`)
7
+
8
+ ## Motivation
9
+
10
+ A workflow suspension that schedules several steps and waits previously wrote one event per entity — one `world.events.create` call per `step_created` and `wait_created`. Against a remote World each write is its own network round trip and its own crash boundary. Batching folds a suspension's schedule into **one durable write** with per-event outcomes, cutting request count and making the whole fan-out land atomically per attempt.
11
+
12
+ ## The World spec addition
13
+
14
+ `Storage['events']` gains one **optional** method:
15
+
16
+ ```ts
17
+ import type {
18
+ BatchEventRequest,
19
+ CreateEventBatchParams,
20
+ EventBatchResult,
21
+ } from '@workflow/world';
22
+
23
+ interface BatchCapableEvents {
24
+ createBatch?(
25
+ runId: string,
26
+ events: BatchEventRequest[],
27
+ params?: CreateEventBatchParams
28
+ ): Promise<EventBatchResult>;
29
+ }
30
+ ```
31
+
32
+ The supporting types, excerpted (canonical definitions live in `@workflow/world`):
33
+
34
+ {/* @skip-typecheck illustrative excerpts of the canonical @workflow/world types */}
35
+ ```ts
36
+ interface BatchEventRequest {
37
+ /** The event — the same discriminated union the single `create` takes. */
38
+ event: CreateEventRequest;
39
+ /** Client event time; under slot identity, the source of the durable createdAt. */
40
+ occurredAt?: Date;
41
+ /** Per-event compute attribution, same as the single create's CreateEventParams. */
42
+ computeInstanceId?: string;
43
+ }
44
+
45
+ type BatchEventItemResult =
46
+ | { status: 200; event: Event; run?: WorkflowRun; step?: Step; wait?: Wait }
47
+ | { status: number; error: string; message: string };
48
+
49
+ interface EventBatchResult {
50
+ /** One entry per submitted event, in request order. */
51
+ results: BatchEventItemResult[];
52
+ }
53
+ ```
54
+
55
+ The contract:
56
+
57
+ - **Ordered**: events land in the run's log in request order at consecutive slots. A concurrent writer may push the whole batch to slots above the caller's view; no skipped-event report accompanies the batch result, so a position-tracking caller compares committed slots against its expectation and reloads to observe what interleaved (its local view stays a strict prefix of the log — never a hole).
58
+ - **Per-event outcomes**: the batch is processed as a whole, and each event reports what its own single `create` would have returned — `200` plus the materialized entity, or the single-path status/code (`409`/`conflict` for an event an earlier delivery already applied). Callers reuse their single-path conflict handling per event.
59
+ - **Idempotent on retry — for entity-conditioned shapes**: creates, terminal transitions, and the born-running pair are each guarded by their own entity condition, so retrying a batch of them that (partially) committed converges to per-event `409`s with nothing written twice. A standalone bare `step_started` or a `step_retrying` re-patches its step instead of converging, so `world-vercel` only auto-retries batches whose every event is retry-convergent (everything the runtime folds today is), and rejects `hook_received` in a batch outright.
60
+ - **Method presence is the capability declaration.** A World that doesn't implement it keeps the single-event path; a World that implements it must make each attempt atomic (a lost race leaves nothing behind). `world-vercel` implements it against `POST /v4/runs/:runId/events/batch` (slot-identity runs only, i.e. specVersion ≥ 6). `world-local` and `world-postgres` deliberately do not — batching buys nothing for a local write.
61
+ - **Not batchable** (Worlds reject the request): `run_created`, `run_started`, `run_cancelled`, `hook_created`, `hook_disposed`, `attr_set`, and multiple events targeting one entity — except `step_created` followed by `step_started` for the same step, which creates the step born-running.
62
+
63
+ ## The runtime integration (suspension fan-out fold)
64
+
65
+ **On by default.** The suspension handler folds a **clean fan-out** — the suspension's eager `step_created` and `wait_created` writes — into `createBatch` calls of at most 32 events (mirroring the server's transaction budgets). Chunks of a larger fan-out commit **concurrently**: slot assignment is the World's, so parallel chunks race for slot ranges exactly like the pre-fold path's parallel single writes did, and per-entity conditions — not commit order — carry correctness. The fold only engages when the World implements `createBatch`, the run is on slot identity, and the suspension carries no attribute writes, no hook writes, and no resilient step dispatch; everything else keeps the single-event path byte-for-byte.
66
+
67
+ **Per-chunk continuation.** Each chunk's follow-on work starts the moment **that chunk** commits, not when the whole fold does: a chunk's step-execution queue messages publish right off its own commit (publish-after-create holds per step), and only the chunk carrying the inline pairs gates the replay's continuation — trailing chunks' commits and publishes are joined before the invocation can acknowledge its message, so the durability contract ("every create durable before ack") is unchanged.
68
+
69
+ **Pre-claimed inline pairs.** When the fold engages and has company for them (at least two inline steps, or one plus other batchable events), the steps the runtime is about to execute inline join the batch as adjacent `[step_created, step_started]` pairs — the created row carrying the input, the started row a bare ownership-stamped claim the World folds into a born-running create. The inline bodies start straight off the pair chunk's commit (in parallel with the queue publishes and any trailing chunks) with no per-step claim POST at all, and a pair that loses its atomic create-claim to a concurrent delivery skips its body exactly as a lost lazy claim does. A lone inline step with nothing else to batch keeps the optimistic lazy-start path, whose claim overlaps the body.
70
+
71
+ Per-event `409`s are tolerated the same way the single path tolerates `EntityConflictError` (a concurrent delivery already created the entity); any other per-event failure fails the suspension write the way a single-path rejection would. A batch carrying a `step_started` (that is, any batch with inline pairs) is **not** retried in-process on a transport blip: a pair's `409` cannot be told apart from the caller's own earlier attempt having committed it, so recovery goes through queue redelivery instead, where the step's ownership stamp routes it back to the same invocation.
72
+
73
+ `createBatch` is optional, so today only the Vercel World folds at all: every other World keeps the single-event path and never sends a pair.
74
+
75
+ **Escape hatch:** set `WORKFLOW_BATCH_TRANSITIONS=0` (or `false`) to disable batching and restore the exact prior one-write-per-event path — see [`WORKFLOW_BATCH_TRANSITIONS`](/docs/configuration/worlds#workflow_batch_transitions).
76
+
77
+ ## Follow-up
78
+
79
+ The deferred sequential transition — holding `step_completed(N)` across the replay turn and committing `[step_completed(N), step_created(N+1), step_started(N+1)]` as one batch at the next lazy start — builds on this contract and ships separately.
@@ -7,7 +7,8 @@
7
7
  "resilient-start",
8
8
  "lazy-event-creation",
9
9
  "turbo-mode",
10
- "step-message-ownership"
10
+ "step-message-ownership",
11
+ "batched-event-writes"
11
12
  ],
12
13
  "defaultOpen": false
13
14
  }
@@ -71,17 +71,23 @@ For example, a workflow can run a 10-minute inline step even with `WORKFLOW_REPL
71
71
  - Only applies to Worlds with atomic, immutable deployments (the Vercel World). A run whose pinned deployment cannot be reached at all fails immediately regardless of this value.
72
72
  - Transient or unknown queue publishing failures use normal queue redelivery and do not consume this budget.
73
73
 
74
- ### `WORKFLOW_PRECONDITION_GUARD`
74
+ ### `WORKFLOW_RESILIENT_STEP_DISPATCH`
75
75
 
76
- - Default: enabled
77
- - An optimistic-concurrency guard for event creation: replay-context event creations describe the snapshot they replayed from its latest event timestamp (`stateUpdatedAt`), the number of events it contains (`stateEventCount`), and its event-log cursor (`stateCursor`) and a backend that supports the guard rejects a creation with 412 ([`PreconditionFailedError`](/docs/api-reference/workflow-errors/precondition-failed-error)) when a newer out-of-band event (a received hook or a completed step) was recorded after that snapshot, or when the snapshot is missing an event recorded at or before it.
78
- - On rejection the runtime restarts the replay in the same invocation from a corrected event log, and falls back to a re-invocation with a fresh replay once the restart budget is spent. The rejected write is never retried as-is: a replay working from a corrected log derives different events, so only a fresh replay may write again.
79
- - When enabled — and the World declares that it enforces the guard (`capabilities.preconditionGuard`; the Vercel World does) — the runtime also keeps the per-step event-log delta optimization (consuming the delta returned by a step's terminal write instead of issuing an extra `events.list` per step) active while the run has an open hook. Without an enforced guard, an open hook disables it.
80
- - While a hook is open on a guard-enforcing deployment, inline steps take the await-then-run path even when optimistic inline start is enabled: the step's `step_started` claim carries the snapshot and is awaited before the body runs, so a claim the backend rejects as stale never executes user code.
81
- - Backends that do not support the guard ignore the snapshot; they must not declare the capability, so guard-dependent optimizations stay off against them even when the flag is set.
82
- - The guard only ever rejects on evidence, and it fails open in every other case: a backend that cannot decide — because its record of recent events is incomplete, has expired, or covers only part of the run's history — must accept the write. A rejection therefore always means the snapshot really was incomplete, but the absence of one does not prove it was complete. Busy runs (wide step fan-outs, high hook volume) are the most likely to skip the check.
83
- - As a result, 412 volume describes a workload rather than the health of a deployment, and a run that never sees one is not evidence the guard is inactive.
84
- - Set `0` to disable.
76
+ - Default: disabled
77
+ - When a suspension hands newly created steps to the queue, the runtime publishes each step's execution message in parallel with its `step_created` event write instead of sequencing them, cutting a round trip per dispatched step. The message also carries the serialized step input (`stepInput`), so a transient `step_created` write failure (429 / 5xx / transport) still executes the step — the queue consumer idempotently re-ensures the event before running it, converging with the producer's write on the step's correlation ID. This mirrors resilient start (`runInput`) and the lazy hook resume (`hookInput`).
78
+ - It is off by default because the publish races the create's verdict, and a create can come back refused: as a duplicate this replay should stop pursuing, or as a [stale write](#stale-reads-and-why-nothing-has-to-be-rejected) on a World that refuses rather than reports. Either way the message carrying the payload is already out, so the consumer can materialize a step whose create was refused, and nothing orders the verdict before the consumer's redelivery re-ensure. The sequential path is the only one that gives the message a happens-after edge over it.
79
+ - Even when enabled, the runtime falls back to the sequential create-then-publish dispatch when the step input is too large to inline on the queue message, or when the run's queue transport cannot carry binary payloads (pre-CBOR spec versions).
80
+ - Producer-side recoveries are reported on the suspension span as `workflow.step.resilient_dispatch_recovered`; a consumer that materialized the event reports `workflow.step.resilient_dispatch_materialized`.
81
+ - Set `1` to enable it.
82
+
83
+ ### Stale reads, and why nothing has to be rejected
84
+
85
+ - Not a variable: this is how a replay working from an out-of-date event log stays correct, and why no World needs a precondition guard to make it so.
86
+ - Three properties do it together. A reader's log is always a **prefix** of the run's log, never a prefix with a hole in it — positions are allocated by the World at commit, so nothing lands behind a position a reader has already passed. Replay is **deterministic on a prefix**: the same prefix always yields the same decisions, so a shorter log does not mean a different run, only a run that has not caught up. And every write **reports what it missed**: a creation names the position it replayed from (`eventCount`), and the World returns the events occupying the positions it was pushed past. The replay merges those and continues, correcting itself on the write rather than on a read.
87
+ - So a stale replay costs a merge, not a rejection. None of the shipped Worlds refuses a write for being stale.
88
+ - A World *may* refuse instead, with 412 ([`PreconditionFailedError`](/docs/api-reference/workflow-errors/precondition-failed-error)) — appropriate when it allocates positions somewhere other than the commit and cannot report a gap reliably. The runtime handles that: it restarts the replay in the same invocation from a corrected event log, and falls back to a re-invocation with a fresh replay once the restart budget is spent. The rejected write is never retried as-is, because a replay working from a corrected log derives different events.
89
+ - A World that does refuse should only ever do so on evidence, and accept the write in every other case. A rejection then always means the position really was stale, while the absence of one proves nothing about currency.
90
+ - Two runtime behaviors follow from the properties above rather than from any fence. The per-step event-log delta optimization (consuming the delta returned by a step's terminal write instead of issuing an extra `events.list` per step) stays active while the run has an open hook: a `hook_received` missed by the delta window is observed one iteration later, and the next write brings it back. And while a hook is open, inline steps take the await-then-run path even when optimistic inline start is enabled — several invocations race for one step's claim there, and awaiting it means the body runs only for the writer that won.
85
91
 
86
92
  ### `WORKFLOW_SLOT_GAP_CHECK`
87
93
 
@@ -94,7 +100,7 @@ For example, a workflow can run a 10-minute inline step even with `WORKFLOW_REPL
94
100
  ### `WORKFLOW_PRECONDITION_MAX_INPROCESS_RESTARTS`
95
101
 
96
102
  - Default: `3`
97
- - How many times a single invocation restarts its replay in-process after a rejected event creation before it falls back to a re-invocation.
103
+ - How many times a single invocation restarts its replay in-process after an event creation is [rejected as stale](#stale-reads-and-why-nothing-has-to-be-rejected) before it falls back to a re-invocation. No shipped World rejects one, so this budget is reserved for a World that chooses to.
98
104
  - A restart reloads the event log and rebuilds the workflow from scratch, so it costs a replay but no queue round trip. A World may attach the missing events to its rejection, in which case the first restart needs no event-log request at all.
99
105
 
100
106
  ### `WORKFLOW_PRECONDITION_MAX_REINVOCATIONS`
@@ -208,6 +214,24 @@ For example, a workflow can run a 10-minute inline step even with `WORKFLOW_REPL
208
214
  - Default: enabled
209
215
  - On the Vercel World, lets concurrent event-log requests share one HTTP/2 connection instead of one connection per in-flight request.
210
216
  - Set `0` to take the event-log requests off HTTP/2 entirely, back to one request per HTTP/1.1 connection. Use this as the kill switch if HTTP/2 turns out to be at fault for event delivery problems.
217
+ - Has no effect when `WORKFLOW_NODE_HTTP` is enabled, which takes the whole HTTP/2 path away.
218
+
219
+ ### `WORKFLOW_NODE_HTTP`
220
+
221
+ - Default: disabled
222
+ - Makes the Vercel and Local Worlds issue their HTTP requests through Node's built-in `node:http` and `node:https` modules, instead of the HTTP client library those Worlds normally use.
223
+ - Set `1` to switch to Node's modules. Read the trade-offs below first: they cost throughput on every deployment, which is why this is opt-in.
224
+ - Use it when that library is not an option: a bundler that mangles it, a runtime that does not ship a working copy of it, or a transport-level fault you want to rule out. It is not reached by way of `fetch()` either, so a runtime whose `fetch()` is built on the same library is still covered.
225
+
226
+ Node's own modules do less than the client they replace, so enabling this drops the per-call-site tuning the Worlds configure:
227
+
228
+ - Event-log requests lose HTTP/2, so concurrent reads and writes no longer share one connection, and the enlarged HTTP/2 receive windows no longer apply. This is the largest difference, and it slows down replays that read a big event log. It does not apply to event writes on [`WORKFLOW_EVENTS_TRANSPORT=ws`](/docs/configuration/worlds#workflow_events_transport), which take neither transport.
229
+ - Requests lose their transport-level retry. Failures still surface to the layers above, which retry event writes and redeliver queue messages, so nothing is silently dropped, but a failure that a same-connection retry would have hidden now costs a full redelivery.
230
+ - Stream close loses its retry of retriable server errors. A transient failure at close can leave a stream marked closing until the run expires, where it would previously have resolved on the retry.
231
+
232
+ Connection pooling, keep-alive, and the request, header, and body deadlines are preserved: pooling and keep-alive are configured on Node's agents, and the deadlines are passed per request — from the Local World's two queue timeouts, and on the Vercel World from the same defaults its HTTP client applies today. Queue sends are the exception in the other direction: that client takes no transport override, so its requests keep using the library either way.
233
+
234
+ A `dispatcher` passed to `createVercelWorld()` still wins over this variable. The variable chooses which transport the World builds when you have not supplied one.
211
235
 
212
236
  ## Queue namespace
213
237
 
@@ -274,6 +274,14 @@ Platform-provided values such as `VERCEL_DEPLOYMENT_ID`, `VERCEL_PROJECT_ID`, an
274
274
  - Default: `1000`
275
275
  - Maximum stream chunks written in one Vercel World request. Larger batches are split.
276
276
 
277
+ ### `WORKFLOW_BATCH_TRANSITIONS`
278
+
279
+ - Surface: environment variable
280
+ - Default: on
281
+ - Set to `0` (or `false`) to **disable** batched event writes — the escape hatch that restores the exact prior one-write-per-event path.
282
+
283
+ When enabled (the default), a suspension's eager `step_created` and `wait_created` writes fold into batched `events.createBatch` calls (one durable write with per-event outcomes) on Worlds that implement the optional batch API. The fold only engages when the World implements `events.createBatch` (the Vercel World does; Local and Postgres do not), the run's spec version supports slot identity (≥ 6), and the suspension carries no attribute writes, hook writes, or resilient step dispatch — everything else keeps the single-event path unchanged, so disabling is only needed as an operational escape hatch. Batches are capped at 32 events; larger fan-outs commit in successive batches. See the [batched event writes changelog](/docs/changelog/batched-event-writes) for the World API contract.
284
+
277
285
  ### `WORKFLOW_EVENTS_TRANSPORT`
278
286
 
279
287
  - Factory option: none
@@ -2,14 +2,14 @@
2
2
  title: corrupted-event-log
3
3
  description: The workflow's event log contains an event that no consumer can process, indicating corruption or invalid state.
4
4
  type: troubleshooting
5
- summary: Resolve corrupted event log errors caused by duplicate or orphaned events.
5
+ summary: Resolve corrupted event log errors caused by orphaned or unattributable events.
6
6
  prerequisites:
7
7
  - /docs/foundations/workflows-and-steps
8
8
  related:
9
9
  - /docs/foundations/errors-and-retries
10
10
  ---
11
11
 
12
- This error occurs when the Workflow runtime repeatedly cannot replay events in the event log. This usually means the event log is in an invalid state, such as duplicate or orphaned events, or that a runtime determinism bug persists across retry attempts.
12
+ This error occurs when the Workflow runtime repeatedly cannot replay events in the event log. This usually means the event log is in an invalid state, such as an orphaned event or one no consumer can attribute to anything the workflow did, or that a runtime determinism bug persists across retry attempts.
13
13
 
14
14
  This is a **workflow-level fatal error**. It cannot be caught or handled inside your workflow code. The runtime first retries transient replay divergence automatically; it marks the run as failed with this error only after replay still cannot recover.
15
15
 
@@ -29,10 +29,9 @@ Before failing, the runtime retries a divergent replay and surfaces this termina
29
29
 
30
30
  Common scenarios that produce this error:
31
31
 
32
- 1. **Duplicate completion events** — Two `wait_completed` events for a single `wait_created`, or two `step_completed` events for the same step. The first is consumed normally, and the second resolves something already resolved, so no later consumer can claim it.
32
+ 1. **An unclaimed event that repeats nothing** — A duplicate of a kind the log already records for that entity is read past rather than failing the run, so a second `step_completed` or `wait_completed` is not this error (see [Duplicate Events](/docs/how-it-works/event-sourcing#duplicate-events)). What fails is an unclaimed event with no earlier counterpart to defer to: a `step_started` behind a `step_completed` on a log that never recorded a `step_started`, for instance. No consumer remains for the step, and there is no earlier event of that kind the replay could be reading instead.
33
33
  2. **Orphaned events** — A `step_completed` or `wait_completed` event whose `correlationId` doesn't match any step or sleep in the workflow code, so the replay reaches its end still holding it.
34
- 3. **Events after terminal state** — An event that arrives after its corresponding step or wait has already reached a terminal state (e.g., `step_retrying` after `step_completed`).
35
- 4. **A hole in the log** — Events are numbered by their position in the run's log, and those positions are dense, so a position below the log's highest that holds no event means the log the replay loaded is incomplete. The runtime cannot tell a position no write ever occupied from one whose event it failed to read, so it refuses to replay rather than produce a result that may be silently wrong. See [`WORKFLOW_SLOT_GAP_CHECK`](/docs/configuration/runtime-tuning#workflow_slot_gap_check).
34
+ 3. **A hole in the log** — Events are numbered by their position in the run's log, and those positions are dense, so a position below the log's highest that holds no event means the log the replay loaded is incomplete. The runtime cannot tell a position no write ever occupied from one whose event it failed to read, so it refuses to replay rather than produce a result that may be silently wrong. See [`WORKFLOW_SLOT_GAP_CHECK`](/docs/configuration/runtime-tuning#workflow_slot_gap_check).
36
35
 
37
36
  ## What To Do
38
37
 
@@ -20,7 +20,7 @@ A single divergent replay does not prove that persisted history is corrupted. Fo
20
20
 
21
21
  The runtime automatically queues another replay when an invocation reports `REPLAY_DIVERGENCE`. No terminal `run_failed` event is written during these recovery attempts.
22
22
 
23
- If recovery replays continue to diverge after the retry budget is exhausted, the runtime marks the run as failed with `CORRUPTED_EVENT_LOG` and records the latest divergent event for diagnosis.
23
+ If recovery replays continue to diverge after the recovery budget is exhausted, the runtime marks the run as failed with `CORRUPTED_EVENT_LOG` and records the latest divergent event for diagnosis.
24
24
 
25
25
  ## What To Do
26
26
 
@@ -116,7 +116,7 @@ Calling `createHook()` on its own does not register the hook — registration is
116
116
 
117
117
  ### Custom Tokens for Deterministic Hooks
118
118
 
119
- By default, hooks generate a random token. However, you often want to use a **custom token** that external systems can reconstruct. This is especially useful for long-running workflows where the same workflow instance should handle multiple events.
119
+ By default, hooks generate their own token. However, you often want to use a **custom token** that external systems can reconstruct. This is especially useful for long-running workflows where the same workflow instance should handle multiple events.
120
120
 
121
121
  For example, imagine a Slack bot where each channel should have its own workflow instance:
122
122
 
@@ -480,7 +480,7 @@ This pattern is especially valuable in larger applications where the workflow an
480
480
 
481
481
  ### Token Design
482
482
 
483
- Custom tokens are available for `createHook()` with server-side `resumeHook()` only. Webhooks (`createWebhook()`) always use randomly generated tokens to prevent unauthorized access to public webhook endpoints.
483
+ Custom tokens are available for `createHook()` with server-side `resumeHook()` only. Webhooks (`createWebhook()`) always generate their own unique tokens. A generated token is not trivial to guess, but it is not a strong security contract either, so anyone who obtains the URL can invoke an unintended webhook resumption. To prevent unauthenticated run resumptions entirely, prefer a **hook** over the **webhook** convenience and implement your own authentication on the route that calls `resumeHook()`.
484
484
 
485
485
  When using custom tokens with `createHook()`:
486
486
 
@@ -58,6 +58,30 @@ export async function POST() {
58
58
 
59
59
  When a client makes a request to this endpoint, they'll receive each message as it's written, without waiting for the workflow to complete.
60
60
 
61
+ ### Avoiding Function Timeouts After Client Disconnects
62
+
63
+ On Vercel, `run.readable` and `run.getReadable()` reconnect to Workflow's stream storage while the workflow is still running. By default, a client disconnect does not terminate the Vercel Function serving the stream. If a user closes the page or stops the request, the function can therefore keep reconnecting until it reaches its maximum duration and fails with `FUNCTION_INVOCATION_TIMEOUT`.
64
+
65
+ For streaming routes using the Node.js runtime, opt in to [request cancellation](https://vercel.com/docs/functions/functions-api-reference#enable-cancellation) in `vercel.json`:
66
+
67
+ ```json filename="vercel.json"
68
+ {
69
+ "functions": {
70
+ "app/api/stream/route.ts": {
71
+ "supportsCancellation": true
72
+ }
73
+ }
74
+ }
75
+ ```
76
+
77
+ Replace the function path with the path or glob for your streaming route. When the downstream client disconnects, Vercel terminates the matching function invocation instead of leaving its stream reader running. The workflow run and its durable stream continue independently, so the client can reconnect through another route invocation later.
78
+
79
+ <Callout type="warn">
80
+ Cancellation applies to every function matching the configured path or glob, even if the route does not listen to `request.signal`. Any other work in that invocation which is not wrapped in [`waitUntil`](https://vercel.com/docs/functions/functions-api-reference/vercel-functions-package#waituntil) or [`after`](https://nextjs.org/docs/app/api-reference/functions/after) can be lost. Only enable it for routes that are safe to terminate when their client disconnects.
81
+ </Callout>
82
+
83
+ This setting prevents abandoned stream readers from consuming the rest of a function invocation. It does not extend the function's maximum duration: an actively connected streaming response can still reach the configured limit, at which point the client should reconnect to the durable stream.
84
+
61
85
  ### Resuming Streams from a Specific Point
62
86
 
63
87
  Use `run.getReadable({ startIndex })` to resume a stream from a specific position. This is useful for reconnecting after timeouts or network interruptions:
@@ -94,7 +94,7 @@ flowchart TD
94
94
  - `cancelled`: Reserved for future use (not currently emitted)
95
95
 
96
96
  <Callout type="info">
97
- The `step_retrying` event is optional. Steps can retry without it - the retry mechanism works regardless of whether this event is emitted. You may see back-to-back `step_started` events in logs when a step retries after a timeout or when the error is not explicitly captured. See [Errors and Retries](/docs/foundations/errors-and-retries) for more on how retries work.
97
+ The `step_retrying` event is optional. Steps can retry without it - the retry mechanism works regardless of whether this event is emitted. You may see back-to-back `step_started` events in logs when a step retries after a timeout or when the error is not explicitly captured, and also when concurrent replays each commit one (see [Duplicate Events](#duplicate-events)). See [Errors and Retries](/docs/foundations/errors-and-retries) for more on how retries work.
98
98
  </Callout>
99
99
 
100
100
  When present, the `step_retrying` event moves a step back to `pending` state and records the error that caused the retry. This provides two benefits:
@@ -225,6 +225,44 @@ Terminal states represent the end of an entity's lifecycle. Once an entity reach
225
225
 
226
226
  Attempting to create an event that would transition an entity out of a terminal state will result in an error. This prevents inconsistent state and ensures the integrity of the event log.
227
227
 
228
+ That guard sits on the write path. A duplicate the write path does permit — a second `step_created` for a step that is not yet terminal, for example — is handled during replay instead, described next.
229
+
230
+ ## Duplicate Events
231
+
232
+ Concurrent invocations replaying the same run share one event log. An invocation working from a stale prefix — one that predates another invocation's write — can commit its own `step_created`, `step_started`, or `wait_created` for an entity the log already records one of. These writes pass the terminal-state guard above, so the write path commits them even when a backend validates transitions atomically with the insert.
233
+
234
+ Those duplicates are committed but inert. The outcome was decided by the first event of its kind at a lower position in the log, and every replay reads that same event at that same position, so a later copy cannot change what the workflow observes.
235
+
236
+ To keep an inert copy from failing an otherwise healthy run, the runtime groups event types into **classes** and tracks, per entity, which classes the current replay has already consumed. When an event is offered to every registered consumer and none wants it, and its class is already recorded for that entity, the replay steps over it instead of reporting a [replay divergence](/docs/errors/replay-divergence) — which, once the recovery budget is exhausted, ends the run with [`CORRUPTED_EVENT_LOG`](/docs/errors/corrupted-event-log).
237
+
238
+ | Class | Event types |
239
+ |-------|-------------|
240
+ | `run_started` | `run_started` |
241
+ | `step_created` | `step_created` |
242
+ | `step_started` | `step_started` |
243
+ | `step_retrying` | `step_retrying` |
244
+ | `step_terminal` | `step_completed`, `step_failed` |
245
+ | `wait_created` | `wait_created` |
246
+ | `wait_completed` | `wait_completed` |
247
+ | `hook_created` | `hook_created` |
248
+ | `hook_disposed` | `hook_disposed` |
249
+
250
+ Types that share a class are the mutually exclusive outcomes of one decision: a step either completes or fails, and the first outcome recorded is the one that counts. Classes are independent of one another, so passing over one does not suppress another. A step whose result is already in the log has still recorded exactly one `step_created`, which is what makes a second one ignorable on its own terms.
251
+
252
+ The two hook classes cover the same shape of duplicate, and replay reaches them less often because the write path resolves most hook duplicates before they reach the log: a run re-creating a hook it already owns converges on the existing `hook_created` rather than appending a second one, and a second `hook_disposed` for the same hook is refused as an idempotent no-op. A log that holds either anyway is read past like any other repeat.
253
+
254
+ The remaining event types belong to no class and are never skipped:
255
+
256
+ - `hook_received`: a hook legitimately receives many payloads under one ID, so a second `hook_received` is not a repeat of anything.
257
+ - `hook_conflict`: records a failed acquisition of a hook's token, which the same run can hit repeatedly over its lifetime as other runs take and release that token. The hook's own consumer stays registered and claims every copy it is offered, so a repeat is consumed rather than reaching the class check.
258
+ - `attr_set`: written on every [`setAttributes()`](/docs/api-reference/workflow/set-attributes) call, so a second write of the same key is a new fact rather than a repeat.
259
+ - `run_created` precedes every replay and is always consumed.
260
+ - `run_completed`, `run_failed`, and `run_cancelled` never reach the check. The runtime exits before replaying the workflow body once the log holds one of them, so no consumer ever takes one and no class is ever recorded for them.
261
+
262
+ Both kinds of skip are logged at `debug`, so neither reaches the console unless you run with `DEBUG=workflow:runtime:*`. A duplicate is a permanent feature of the log: every later replay re-reads it and lands on the same check, so anything printed unconditionally would print once per replay for the life of the run, and there is nothing to act on either way. A repeat that decides a class differently — a `step_failed` behind a `step_completed`, or the reverse — gets its own message, because unlike a re-commit of the same outcome there is no reading in which both writers were right.
263
+
264
+ The observability UI greys out the events it can identify this way, with the reason on hover. Its set is narrower than the runtime's: it reads the log without consumer state, and a consumer for an entity that is still open legitimately claims a repeat — each retry of a step writes another `step_started`. So it marks a repeat only once no consumer can remain for it: past a terminal event for the same entity, or a second `run_started`, of which the log records one per run. On a partial view of the log — one page of a paginated list, or search results — it marks nothing, since which copy came first is a property of the whole log.
265
+
228
266
  ## Event Correlation
229
267
 
230
268
  Events use a `correlationId` to link related events together. For step, hook, and wait events, the correlation ID identifies the specific entity instance:
@@ -256,7 +256,7 @@ import { createWebhook } from "workflow";
256
256
  export async function ingestWorkflow(endpointId: string) {
257
257
  "use workflow";
258
258
 
259
- // Webhook tokens are always randomly generated
259
+ // Webhook tokens are always generated for you
260
260
  using webhook = createWebhook(); // [!code highlight]
261
261
 
262
262
  const request = await webhook; // [!code highlight]
@@ -282,7 +282,7 @@ describe("ingestWorkflow", () => {
282
282
  it("should process webhook data", async () => {
283
283
  const run = await start(ingestWorkflow, ["ep-1"]);
284
284
 
285
- // Discover the randomly generated webhook token
285
+ // Discover the generated webhook token
286
286
  const hook = await waitForHook(run); // [!code highlight]
287
287
 
288
288
  // Resume the webhook with a Request object
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "workflow",
3
- "version": "5.0.0-beta.41",
3
+ "version": "5.0.0-beta.43",
4
4
  "description": "Workflow SDK - Build durable, resilient, and observable workflows",
5
5
  "main": "dist/typescript-plugin.cjs",
6
6
  "type": "module",
@@ -59,18 +59,18 @@
59
59
  },
60
60
  "dependencies": {
61
61
  "ms": "2.1.3",
62
- "@workflow/astro": "5.0.0-beta.41",
63
- "@workflow/cli": "5.0.0-beta.41",
64
- "@workflow/core": "5.0.0-beta.41",
65
- "@workflow/errors": "5.0.0-beta.16",
62
+ "@workflow/astro": "5.0.0-beta.43",
63
+ "@workflow/cli": "5.0.0-beta.43",
64
+ "@workflow/core": "5.0.0-beta.43",
65
+ "@workflow/errors": "5.0.0-beta.17",
66
66
  "@workflow/typescript-plugin": "5.0.0-beta.5",
67
67
  "@workflow/utils": "5.0.0-beta.8",
68
- "@workflow/next": "5.0.0-beta.41",
69
- "@workflow/nest": "5.0.0-beta.41",
70
- "@workflow/nitro": "5.0.0-beta.41",
71
- "@workflow/nuxt": "5.0.0-beta.41",
72
- "@workflow/sveltekit": "5.0.0-beta.41",
73
- "@workflow/rollup": "5.0.0-beta.41"
68
+ "@workflow/next": "5.0.0-beta.43",
69
+ "@workflow/nest": "5.0.0-beta.43",
70
+ "@workflow/nitro": "5.0.0-beta.43",
71
+ "@workflow/nuxt": "5.0.0-beta.43",
72
+ "@workflow/sveltekit": "5.0.0-beta.43",
73
+ "@workflow/rollup": "5.0.0-beta.43"
74
74
  },
75
75
  "devDependencies": {
76
76
  "@types/ms": "2.1.0",