workflow 5.0.0-beta.11 → 5.0.0-beta.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -7,321 +7,69 @@ description: Overhaul run start logic to tolerate world storage unavailability,
7
7
 
8
8
  ## Motivation
9
9
 
10
- When `world` storage is unavailable but the queue is up,
11
- `start()` previously failed entirely because `world.events.create(run_created)`
12
- is called before `world.queue()`. This change decouples run creation from queue
13
- dispatch so that runs can still be accepted when storage is degraded.
10
+ When `world` storage is unavailable but the queue is up, `start()` previously failed entirely because `world.events.create(run_created)` is called before `world.queue()`. This change decouples run creation from queue dispatch so that runs can still be accepted when storage is degraded.
14
11
 
15
- Additionally, the runtime previously called `world.runs.get(runId)` before
16
- `run_started`, adding an extra round-trip. By always calling `run_started`
17
- directly, we save that round-trip and can return pre-loaded events in the
18
- response to skip the initial `events.list` call, reducing TTFB.
12
+ Additionally, the runtime previously called `world.runs.get(runId)` before `run_started`, adding an extra round-trip. By always calling `run_started` directly, we save that round-trip and can return pre-loaded events in the response to skip the initial `events.list` call, reducing TTFB.
19
13
 
20
14
  ## Design
21
15
 
22
- ### `start()` changes (packages/core)
16
+ ### `start()` changes
23
17
 
24
- - `world.events.create` (run_created) and `world.queue` are now called **in parallel**
25
- via `Promise.allSettled`.
26
- - If `events.create` errors with **429 or 5xx**, we log a warning saying that run
27
- creation failed but the run was accepted — creation will be re-tried async by the
28
- runtime when it processes the queue message. The returned `Run` instance is marked
29
- with `resilientStart = true`.
30
- - If `events.create` errors with **409** (EntityConflictError), the run already exists
31
- (e.g., the queue handler's resilient start path created it first due to a cold-start
32
- race). This is treated as success.
18
+ - `world.events.create` (run_created) and `world.queue` are now called **in parallel** via `Promise.allSettled`.
19
+ - If `events.create` errors with **429 or 5xx**, we log a warning saying that run creation failed but the run was accepted — creation will be re-tried async by the runtime when it processes the queue message. The returned `Run` instance is marked with `resilientStart = true`.
20
+ - If `events.create` errors with **409** (EntityConflictError), the run already exists (e.g., the queue handler's resilient start path created it first due to a cold-start race). This is treated as success.
33
21
  - If `world.queue` fails, we still throw — the run truly failed and was not enqueued.
34
- - The queue invocation now receives all the run inputs (`input`, `deploymentId`,
35
- `workflowName`, `specVersion`, `executionContext`) via `runInput` so the runtime can
36
- create the run later if needed.
37
- - When the runtime re-enqueues itself, it does **not** pass these inputs — only the
38
- first queue cycle carries them.
22
+ - The queue invocation now receives all the run inputs (`input`, `deploymentId`, `workflowName`, `specVersion`, `executionContext`) via `runInput` so the runtime can create the run later if needed.
23
+ - When the runtime re-enqueues itself, it does **not** pass these inputs — only the first queue cycle carries them.
39
24
 
40
- ### `workflowEntrypoint` changes (packages/core)
25
+ ### `workflowEntrypoint` changes
41
26
 
42
- - When calling `world.events.create` with `run_started`, we now also always pass the
43
- run input that was sent through the queue, if available. The response will still be on off:
44
- - **200 with event (now running)**: As usual, but the server could have used the run input to create the run if it didn't exist yet. The response will be opaque to the runtime.
45
- - **200 without event (already running)**: As usual
46
- - **409 or 410 (already finished)**: As usual
27
+ - When calling `world.events.create` with `run_started`, we now also always pass the run input that was sent through the queue, if available. The world is responsible for creating the run if it doesn't already exist.
47
28
 
48
- ### `Run.returnValue` polling (packages/core)
29
+ ### `Run.returnValue` polling
49
30
 
50
- - When `resilientStart` is true on the Run instance (run_created failed), the
51
- `pollReturnValue` loop retries on `WorkflowRunNotFoundError` up to 3 times
52
- (1s + 3s + 6s = 10s total) to give the queue time to deliver and the runtime
53
- to create the run via `run_started`.
54
- - When `resilientStart` is false (normal path), 404 fails immediately — no delay
55
- for the common case of a wrong run ID.
31
+ - When `resilientStart` is true on the Run instance (run_created failed), the `pollReturnValue` loop retries on `WorkflowRunNotFoundError` up to 3 times (1s + 3s + 6s = 10s total) to give the queue time to deliver and the runtime to create the run via `run_started`.
32
+ - When `resilientStart` is false (normal path), 404 fails immediately — no delay for the common case of a wrong run ID.
56
33
 
57
- ### World / workflow-server changes
34
+ ### World contract changes
58
35
 
59
- - Posting `run_started` to a **non-existent** run is now allowed when the run input is
60
- sent along with the payload. The server:
61
- 1. Creates a `run_created` event first (so the event log is consistent).
62
- 2. Strips the input from the `run_started` event data (it lives on `run_created`).
63
- 3. Then creates the `run_started` event normally.
64
- 4. Emits a log and a Datadog metric (`workflow_server.resilient_start.run_created_via_run_started`)
65
- to track when this fallback path is hit.
66
- - When `run_started` encounters an **already-running** run, all worlds return `{ run }`
67
- with `event: undefined` instead of throwing. No duplicate event is created.
36
+ - Posting `run_started` to a **non-existent** run is now allowed when the run input is sent along with the payload. The world creates a `run_created` event first (so the event log is consistent), then creates the `run_started` event normally.
37
+ - When `run_started` encounters an **already-running** run, all worlds return `{ run }` with `event: undefined` instead of throwing. No duplicate event is created.
68
38
 
69
39
  ### Queue transport changes
70
40
 
71
- `Uint8Array` values (the serialized workflow input in `runInput`) don't survive plain
72
- JSON serialization. Each world uses a transport that preserves binary data:
41
+ `Uint8Array` values (the serialized workflow input in `runInput`) don't survive plain JSON serialization. Each world uses a transport that preserves binary data:
73
42
 
74
- - **world-vercel**: CBOR transport — CBOR-encodes the entire queue payload into a
75
- `Buffer` and uses `BufferTransport` from `@vercel/queue`. Uint8Array survives natively.
76
- - **world-local**: `TypedJsonTransport`uses the existing `jsonReplacer`/`jsonReviver`
77
- from `fs.ts` that encode Uint8Array as `{ __type: 'Uint8Array', data: '<base64>' }`.
78
- - **world-postgres**: Inline typed JSON transport — same tagged-envelope approach as
79
- world-local, inlined since world-postgres doesn't import from world-local.
43
+ - **world-vercel**: CBOR transport — CBOR-encodes the entire queue payload into a `Buffer` and uses `BufferTransport` from `@vercel/queue`. Uint8Array survives natively.
44
+ - **world-local**: `TypedJsonTransport` encodes Uint8Array as `{ __type: 'Uint8Array', data: '<base64>' }`.
45
+ - **world-postgres**: Inline typed JSON transport same tagged-envelope approach as world-local.
80
46
 
81
47
  ## Decisions
82
48
 
83
- 1. **Parallel not sequential**: We chose `Promise.allSettled` over sequential calls to
84
- minimize latency in the happy path.
49
+ 1. **Parallel not sequential**: We chose `Promise.allSettled` over sequential calls to minimize latency in the happy path.
85
50
 
86
- 2. **Already-running returns run without event**: When `run_started` encounters an
87
- already-running run, all worlds return `{ run }` with `event: undefined` (no
88
- `events` array) instead of throwing. The runtime detects this by checking for
89
- `result.event === undefined`. This avoids an extra `world.runs.get` round-trip.
51
+ 2. **Already-running returns run without event**: When `run_started` encounters an already-running run, all worlds return `{ run }` with `event: undefined` (no `events` array) instead of throwing. The runtime detects this by checking for `result.event === undefined`. This avoids an extra `world.runs.get` round-trip.
90
52
 
91
- 3. **Events in 200 response**: We only return events on the 200 path (first caller).
92
- On the already-running path, we fall back to the normal `events.list` call. This is
93
- correct because only on 200 can we be certain we know the full event history.
53
+ 3. **Events in 200 response**: We only return events on the 200 path (first caller). On the already-running path, we fall back to the normal `events.list` call. This is correct because only on 200 can we be certain we know the full event history.
94
54
 
95
- 4. **Conditional 404 retry on Run.returnValue**: Only when `resilientStart = true`
96
- (run_created failed). Normal runs fail fast on 404.
55
+ 4. **Conditional 404 retry on Run.returnValue**: Only when `resilientStart = true` (run_created failed). Normal runs fail fast on 404.
97
56
 
98
57
  ## Known concerns
99
58
 
100
- ### Cold-start race on Vercel (observed in CI)
59
+ ### Cold-start race on Vercel
101
60
 
102
- On Vercel, the parallel dispatch can cause the queue message to be processed before
103
- `run_created` completes, if `run_created` hits a cold-start lambda. Confirmed via
104
- Datadog: the `run_started` request hit a warm lambda (23ms) while `run_created` hit
105
- a cold lambda (727ms), even though `run_created` arrived at the edge 116ms earlier.
106
- When this happens:
61
+ On Vercel, the parallel dispatch can cause the queue message to be processed before `run_created` completes, if `run_created` hits a cold-start lambda. When this happens:
107
62
 
108
63
  1. The runtime's resilient start path creates the run from `run_started`.
109
64
  2. The original `run_created` arrives and gets 409 (EntityConflictError).
110
65
  3. `start()` treats the 409 as success (the run exists).
111
66
 
112
- This is handled correctly. The `resilientStart` flag is NOT set on the Run instance
113
- in this case (409 is not a retryable error), so `returnValue` fails fast on 404.
67
+ The `resilientStart` flag is NOT set on the Run instance in this case (409 is not a retryable error), so `returnValue` fails fast on 404.
114
68
 
115
- ### Local Prod test flakiness (resolved)
69
+ ### Atomicity of run entity creation
116
70
 
117
- On world-local, the queue's async IIFE can deliver the message before
118
- `events.create(run_created)` finishes writing to the shared filesystem. The
119
- resilient start path should handle this, but Local Prod tests showed occasional
120
- runs stuck at `pending` (no `run_started` event), and Windows CI showed
121
- "Unconsumed event in event log" errors from duplicate `run_created` events.
71
+ The normal `run_created` path and the resilient start path can race on creating the run entity. In `world-local`, both paths use `writeExclusive` (O_CREAT|O_EXCL) — atomic at the OS level, so exactly one writer wins and the other gets EEXIST. The normal path throws `EntityConflictError` on conflict (handled by `start()` as 409); the resilient start path re-reads the run from disk on conflict.
122
72
 
123
- **Root cause:** A TOCTOU race between the normal `run_created` path and the
124
- resilient start path. Both used `writeJSON` which checks existence with
125
- `fs.access()` (non-atomic), so both could pass the check and write separate
126
- `run_created` events with different event IDs. Fixed by switching both paths to
127
- `writeExclusive` (O_CREAT|O_EXCL) — see retrospective items 12 and 16.
73
+ In `world-postgres`, the resilient start path uses `onConflictDoNothing` plus a re-read on conflict for the same effect, with the same outcome on either side of the race.
128
74
 
129
- ## Follow-up work
130
-
131
- - [x] ~~Investigate Local Prod test flakiness~~ — resolved via `writeExclusive`
132
- for run entity creation (retrospective items 12, 16).
133
- - [ ] Monitor the Datadog metric in production to understand how often the fallback is hit.
134
- - [x] ~~Events optimization for re-enqueue cycles~~ — decided against. The
135
- already-running path returns early without writing an event, so preloading
136
- events there would require an extra filesystem/DB query on every re-enqueue.
137
- More importantly, on Vercel with at-least-once delivery, multiple lambdas can
138
- process the same run concurrently — the event snapshot could be stale or
139
- incomplete. The runtime's fallback to `events.list` is the correct behavior
140
- for re-enqueue cycles.
141
- - [x] ~~CborTransport pass-through~~ — refactored. `encode()`/`decode()` now
142
- live inside `CborTransport.serialize()`/`deserialize()`, matching the pattern
143
- used by TypedJsonTransport (world-local) and the inline transport
144
- (world-postgres). Call sites pass plain objects instead of pre-encoded buffers.
145
-
146
- ## Development retrospective
147
-
148
- Chronological log of mistakes, misunderstandings, and reverted approaches during
149
- development. Included for future reference when working on similar cross-cutting
150
- runtime changes.
151
-
152
- ### 1. Uint8Array corruption through JSON queue transport
153
-
154
- The initial implementation passed `runInput.input` (a `Uint8Array`) directly through
155
- the queue payload. `Uint8Array` doesn't survive `JSON.stringify` — it becomes
156
- `{"0":72,"1":101,...}`. This corrupted the workflow input when the resilient start
157
- path tried to recreate the run from the queue-delivered data.
158
-
159
- Caught by the `spawnWorkflowFromStepWorkflow` e2e test and the `world-testing`
160
- embedded tests, which failed with "Invalid input" from devalue's `unflatten()`.
161
-
162
- Three approaches were tried before landing on the final solution:
163
-
164
- 1. **Base64 encoding** (`btoa`/`atob`) — worked but fragile. The decode side used
165
- `typeof runInput.input === 'string'` as a discriminant, which was flagged as
166
- dangerous since non-binary inputs could also be strings.
167
- 2. **`Array.from()`/`new Uint8Array()`** — replaced base64 with a plain number array.
168
- Two problems: (a) 3x JSON size regression vs base64, and (b) `Array.isArray()`
169
- false-positives on v1Compat runs where `dehydrateWorkflowArguments` returns
170
- devalue's flat Array format.
171
- 3. **CBOR + BufferTransport** (final) — world-vercel CBOR-encodes the queue payload;
172
- world-local and world-postgres use a `TypedJsonTransport` with a tagged envelope.
173
-
174
- ### 2. Forgot to commit world-postgres transport fix (twice)
175
-
176
- After fixing world-local and world-vercel queue transports, the same `JsonTransport`
177
- corruption bug existed in world-postgres. The fix was written during a session but
178
- never committed — lost when the working directory was reset via stash/checkout. This
179
- happened twice. The fix only landed on the third attempt when it was committed and
180
- pushed immediately. All 14 Postgres e2e jobs failed each time.
181
-
182
- ### 3. Incorrect diagnosis of Vercel Prod 409 errors
183
-
184
- Multiple Vercel Prod e2e tests failed with `EntityConflictError: Workflow run with
185
- ID wrun_... already exists` on `run_created`. The initial assumption was that VQS
186
- couldn't deliver the queue message fast enough to beat the `run_created` call.
187
-
188
- Datadog logs showed otherwise: the `run_created` request arrived at Vercel's edge
189
- 116ms before `run_started`, but `run_created` hit a cold-start lambda (727ms) while
190
- `run_started` hit a warm one (23ms). Cold starts can invert expected execution order.
191
-
192
- ### 4. Removed EntityConflictError catch, then had to restore it
193
-
194
- The `workflowEntrypoint` error handler originally caught both `EntityConflictError`
195
- and `RunExpiredError`. When adding the "already-running returns run without event"
196
- behavior, `EntityConflictError` was removed from the catch since the new worlds
197
- wouldn't throw it. Reviewer flagged this: old worlds or world-vercel hitting an
198
- older workflow-server could still throw it. The catch was restored.
199
-
200
- ### 5. Duplicate `startedAt` check
201
-
202
- After refactoring the `run_started` flow, a `workflowRun.startedAt` null check
203
- existed both inside the `try` block and after the `catch` block. The second was
204
- unreachable. Removed after review.
205
-
206
- ### 6. WORKFLOW_SERVER_URL_OVERRIDE left set
207
-
208
- During development, `WORKFLOW_SERVER_URL_OVERRIDE` was set to a test URL pointing
209
- at the workflow-server preview deployment and accidentally committed. The Vercel
210
- bot flagged this. Reset to empty string.
211
-
212
- ### 7. e2e test assertion was wrong
213
-
214
- The resilient start e2e test stubbed `world.events.create` and asserted
215
- `createCallCount >= 2`. But the stub only intercepts calls from the test runner
216
- process — the server uses its own world. `createCallCount` was always 1. Changed
217
- to `expect(createCallCount).toBe(1)`.
218
-
219
- ### 8. Misattributed Local Prod timeouts as "pre-existing"
220
-
221
- Local Prod tests showed 60-second timeouts across various tests. Initially dismissed
222
- as CI flakes. Checking main's CI showed all Local Prod tests pass on main — the
223
- timeouts are caused by our changes. Should have compared against main immediately.
224
-
225
- ### 9. Attempted to revert parallel dispatch
226
-
227
- After identifying Local Prod timeouts, `start()` was partially reverted back to
228
- sequential dispatch. The user pointed out that parallel dispatch is the core value
229
- proposition of the PR. The revert was undone.
230
-
231
- ### 10. WorkflowRunNotFoundError retry was unconditional
232
-
233
- The initial `pollReturnValue` retry on `WorkflowRunNotFoundError` applied to all
234
- `Run` instances. A user calling `getRun()` with a wrong ID would wait 10 seconds
235
- before getting a 404. Fixed by adding a `resilientStart` flag: only retries when
236
- `run_created` actually failed.
237
-
238
- ### 11. Changeset `minor` vs `patch`
239
-
240
- The changeset was created with `"@workflow/core": minor`. Reviewer flagged this as
241
- violating repo rules ("all changes should be patch"). Changed after discussion.
242
-
243
- ### 12. world-local TOCTOU race causing duplicate `run_created` events (Windows CI)
244
-
245
- The resilient start path AND the normal `run_created` path in `world-local/events-storage.ts`
246
- both used `writeJSON` to create the run entity. `writeJSON` checks file existence with
247
- `fs.access()` then writes via temp+rename — a classic TOCTOU race. On the local world,
248
- the queue delivers via an async IIFE in the same event loop, so `events.create(run_created)`
249
- and `events.create(run_started)` (with resilient start) run concurrently:
250
-
251
- 1. Both paths call `fs.access(runPath)` → ENOENT (file doesn't exist yet)
252
- 2. Both proceed to write → the last `fs.rename` wins
253
- 3. Both succeed → both write their own `run_created` event with different event IDs
254
- 4. During replay, the consumer sees two `run_created` events → "Unconsumed event" error
255
-
256
- This caused consistent failures in `world-testing` embedded tests on Windows CI (`hooks`,
257
- `supports null bytes in step results`, `retriable and fatal errors` — all timing out at
258
- 60s with "Unconsumed event in event log" errors). Linux CI was not affected because the
259
- timing was different enough that the race window was rarely hit.
260
-
261
- Fixed by switching BOTH paths to `writeExclusive` (O_CREAT|O_EXCL), which is atomic at
262
- the OS level — exactly one writer wins, the other gets EEXIST. The normal `run_created`
263
- path throws `EntityConflictError` on conflict (handled by `start()` as 409). The resilient
264
- start path re-reads the run from disk on conflict. Either way, only one `run_created`
265
- event is written.
266
-
267
- ### 13. Non-atomic run + run_created event in world-postgres resilient path
268
-
269
- The resilient start path in `world-postgres/storage.ts` did two separate writes (run
270
- insert, then event insert) without a transaction. If the process crashed between them,
271
- the run would exist without a `run_created` event — an inconsistent event log.
272
-
273
- A `drizzle.transaction()` wrapper was attempted but dropped due to TypeScript inference
274
- issues with drizzle's transaction callback and the insert builder's overloads. The current
275
- fix keeps the two writes sequential but adds the same conflict-aware re-read pattern as
276
- world-local: when `onConflictDoNothing` produces no result (run already existed), the run
277
- is re-read so downstream logic sees the real state. The narrow crash window between the
278
- two writes is acceptable — if the run insert succeeds but the event insert crashes, the
279
- run exists and `run_started` will still proceed normally (the event log will be missing a
280
- `run_created` entry, but the run itself is functional).
281
-
282
- ### 14. Missing `WorkflowRunStatus` span attribute after parallel refactor
283
-
284
- The `start()` span previously set `Attribute.WorkflowRunStatus(result.run.status)`, but
285
- this was dropped in the parallel refactor because `result.run` is only available when
286
- `runCreatedResult` fulfilled. The attribute is now conditionally set when the result is
287
- available. In the resilient start case (run_created failed), the attribute is omitted
288
- rather than erroring.
289
-
290
- ### 15. `run_started` eventData leak in world-postgres result
291
-
292
- The `...data` spread in the result construction leaked `eventData` from `run_started`
293
- into the returned event object. Storage was already correct (`storedEventData` is
294
- `undefined` for `run_started`), but the returned result carried the input data. While
295
- harmless (the runtime doesn't use `result.event.eventData`), it was restored to match
296
- the pre-refactor behavior where eventData was explicitly stripped from the result.
297
-
298
- ### 16. Normal `run_created` path also needed `writeExclusive` (Windows CI)
299
-
300
- The initial TOCTOU fix (item 12) only changed the resilient start path to use
301
- `writeExclusive`. The normal `run_created` entity write still used `writeJSON` which
302
- checks existence with `fs.access()` then writes via temp+rename — not atomic. On
303
- Windows CI, the local queue's async IIFE delivered fast enough for both paths to pass
304
- their existence checks simultaneously, producing two `run_created` events with different
305
- event IDs. The events consumer saw the duplicate as "Unconsumed event in event log,"
306
- causing `hooks`, `supports null bytes in step results`, and `retriable and fatal errors`
307
- tests to time out at 60s. Fixed by also switching the normal `run_created` entity write to
308
- `writeExclusive`, making both paths use the same atomic gate.
309
-
310
- ### 17. CborTransport was a pass-through wrapper
311
-
312
- `world-vercel/queue.ts` had `CborTransport` implementing `Transport<Buffer>` with a
313
- no-op `serialize` (identity function) and a `deserialize` that reassembled chunks into
314
- a Buffer without decoding. The actual CBOR `encode()`/`decode()` calls happened at the
315
- call sites — `queue()` pre-encoded before calling `client.send()`, and the handler
316
- post-decoded after receiving from `client.handleCallback()`. This violated the transport
317
- abstraction (every other transport does its encoding inside serialize/deserialize) and
318
- meant the call site had to remember to pre-encode. Refactored to move `encode()`/`decode()`
319
- into the transport methods and changed the type from `Transport<Buffer>` to
320
- `Transport<unknown>`.
321
-
322
- ## Follow-up work (additional)
323
-
324
- - [x] ~~**CborTransport is a pass-through**~~ — Resolved. Moved `encode()`/`decode()`
325
- into `CborTransport.serialize()`/`CborTransport.deserialize()`. The transport is now
326
- self-contained: call sites pass plain objects, and the handler receives decoded objects.
327
- See retrospective item 17.
75
+ The narrow crash window in `world-postgres` between the run insert and the event insert is acceptable — if the run insert succeeds but the event insert crashes, the run exists and `run_started` will still proceed normally (the event log will be missing a `run_created` entry, but the run itself is functional).
@@ -1,154 +1,18 @@
1
1
  ---
2
- title: Durable Agent
3
- description: Replace a stateless AI agent with a durable one that survives crashes, retries tool calls, and streams output.
2
+ title: DurableAgent is now WorkflowAgent
3
+ description: Use AI SDK v7's WorkflowAgent for durable, resumable AI agents.
4
4
  type: guide
5
- summary: Convert an AI SDK Agent into a DurableAgent backed by a workflow, with tools as retryable steps.
5
+ summary: Build durable, resumable AI agents with AI SDK v7's WorkflowAgent.
6
6
  ---
7
7
 
8
- Use this pattern to make any AI SDK agent durable. The agent becomes a workflow, tools become steps, and the framework handles retries, streaming, and state persistence automatically.
8
+ ## WorkflowAgent from AI SDK v7
9
9
 
10
- ## When to use this
10
+ Use AI SDK v7's [`WorkflowAgent`](https://ai-sdk.dev/v7/docs/agents/workflow-agent#workflowagent) for new durable agent work. It replaces `DurableAgent` in v5 and keeps the current agent pattern in the AI SDK package.
11
11
 
12
- - Any AI agent with tool calls that should survive crashes and restarts
13
- - Agents where tool calls hit external APIs that need automatic retries
14
- - Long-running agent sessions where losing progress is unacceptable
15
- - Agents that need per-step observability in the workflow event log
12
+ - Import `WorkflowAgent` from `@ai-sdk/workflow` and run it inside a `"use workflow"` function.
13
+ - Stream `ModelCallStreamPart` chunks with `getWritable()`, then convert the run stream to UI message chunks with `createModelCallToUIChunkTransform()` in your route.
14
+ - Mark tool `execute` functions with `"use step"` when they should run as durable workflow steps with retry and observability behavior.
16
15
 
17
- <Callout type="info">
18
- A durable agent run stays on the deployment that started it. For multi-turn agents that should pick up newer code between turns, see [Versioning](/docs/foundations/versioning) for patterns that start the next turn or next session run with `deploymentId: "latest"`.
16
+ <Callout type="warn">
17
+ `DurableAgent` is deprecated in v5 and remains documented for existing code only. Existing `DurableAgent` users can refer to the [`DurableAgent` API reference](/docs/api-reference/workflow-ai/durable-agent) while migrating.
19
18
  </Callout>
20
-
21
- ## Pattern
22
-
23
- Replace `Agent` with `DurableAgent`, wrap the function in `"use workflow"`, mark each tool with `"use step"`, and stream output through `getWritable()`.
24
-
25
- ### Workflow
26
-
27
- ```typescript
28
- import { DurableAgent } from "@workflow/ai/agent";
29
- import { getWritable } from "workflow";
30
- import { z } from "zod";
31
- import type { ModelMessage, UIMessageChunk } from "ai";
32
-
33
- async function searchFlights({ from, to, date }: {
34
- from: string;
35
- to: string;
36
- date: string;
37
- }) {
38
- "use step"; // [!code highlight]
39
- const res = await fetch(
40
- `https://api.example.com/flights?from=${from}&to=${to}&date=${date}`
41
- );
42
- if (!res.ok) throw new Error(`Search failed: ${res.status}`);
43
- return res.json();
44
- }
45
-
46
- async function bookFlight({ flightId, passenger }: {
47
- flightId: string;
48
- passenger: string;
49
- }) {
50
- "use step"; // [!code highlight]
51
- const res = await fetch("https://api.example.com/bookings", {
52
- method: "POST",
53
- headers: { "Content-Type": "application/json" },
54
- body: JSON.stringify({ flightId, passenger }),
55
- });
56
- if (!res.ok) throw new Error(`Booking failed: ${res.status}`);
57
- return res.json();
58
- }
59
-
60
- async function checkWeather({ city }: { city: string }) {
61
- "use step"; // [!code highlight]
62
- const res = await fetch(`https://api.weather.com/forecast?city=${city}`);
63
- return res.json();
64
- }
65
-
66
- export async function flightAgent(messages: ModelMessage[]) {
67
- "use workflow";
68
-
69
- const agent = new DurableAgent({ // [!code highlight]
70
- model: "anthropic/claude-haiku-4.5",
71
- instructions: "You are a helpful flight booking assistant.",
72
- tools: {
73
- searchFlights: {
74
- description: "Search for available flights between two airports",
75
- inputSchema: z.object({
76
- from: z.string().describe("Departure airport code"),
77
- to: z.string().describe("Arrival airport code"),
78
- date: z.string().describe("Travel date (YYYY-MM-DD)"),
79
- }),
80
- execute: searchFlights,
81
- },
82
- bookFlight: {
83
- description: "Book a specific flight for a passenger",
84
- inputSchema: z.object({
85
- flightId: z.string().describe("Flight ID from search results"),
86
- passenger: z.string().describe("Passenger full name"),
87
- }),
88
- execute: bookFlight,
89
- },
90
- checkWeather: {
91
- description: "Check the weather forecast for a city",
92
- inputSchema: z.object({
93
- city: z.string().describe("City name"),
94
- }),
95
- execute: checkWeather,
96
- },
97
- },
98
- });
99
-
100
- const result = await agent.stream({ // [!code highlight]
101
- messages,
102
- writable: getWritable<UIMessageChunk>(), // [!code highlight]
103
- maxSteps: 10,
104
- });
105
-
106
- return { messages: result.messages };
107
- }
108
- ```
109
-
110
- ### API route
111
-
112
- ```typescript
113
- import type { UIMessage } from "ai";
114
- import { convertToModelMessages, createUIMessageStreamResponse } from "ai";
115
- import { start } from "workflow/api";
116
- import { flightAgent } from "@/app/workflows/flight-agent";
117
-
118
- export async function POST(req: Request) {
119
- const { messages }: { messages: UIMessage[] } = await req.json();
120
- const modelMessages = await convertToModelMessages(messages); // [!code highlight]
121
-
122
- const run = await start(flightAgent, [modelMessages]); // [!code highlight]
123
-
124
- return createUIMessageStreamResponse({ // [!code highlight]
125
- stream: run.readable,
126
- headers: {
127
- "x-workflow-run-id": run.runId,
128
- },
129
- });
130
- }
131
- ```
132
-
133
- ## How it works
134
-
135
- 1. **DurableAgent wraps Agent** — same API as AI SDK's `Agent`, but backed by a workflow. If the process crashes, the agent resumes from the last completed step on replay.
136
- 2. **Tools as steps** — each tool's `execute` function uses `"use step"`, giving it automatic retries, full Node.js access, and an entry in the workflow event log.
137
- 3. **Streaming** — `getWritable<UIMessageChunk>()` streams the agent's output (text chunks, tool calls, tool results) to the client in real time via `createUIMessageStreamResponse`.
138
- 4. **maxSteps** — limits the total number of LLM calls the agent can make, preventing runaway tool loops.
139
-
140
- ## Adapting to your use case
141
-
142
- - **Change the model** — replace `"anthropic/claude-haiku-4.5"` with any AI Gateway model string (e.g. `"openai/gpt-4o"`, `"anthropic/claude-sonnet-4-5"`).
143
- - **Add tools** — define a new `"use step"` function with a Zod schema. Each tool automatically gets retries and persistence.
144
- - **Workflow-level tools** — if a tool needs workflow primitives like `sleep()` or `createHook()`, omit `"use step"` so it runs in the workflow context instead.
145
- - **Multi-turn** — pass `result.messages` plus new user messages to subsequent `agent.stream()` calls for multi-turn conversations.
146
- - **Client integration** — use `useChat()` from `@ai-sdk/react` with `WorkflowChatTransport` from `@workflow/ai` for a full chat UI with reconnection support.
147
-
148
- ## Key APIs
149
-
150
- - [`"use workflow"`](/docs/api-reference/workflow/use-workflow) — declares the orchestrator function
151
- - [`"use step"`](/docs/api-reference/workflow/use-step) — declares step functions with retries and full Node.js access
152
- - [`DurableAgent`](/docs/api-reference/workflow-ai/durable-agent) — durable wrapper around AI SDK's Agent
153
- - [`getWritable()`](/docs/api-reference/workflow/get-writable) — streams agent output to the client
154
- - [`start()`](/docs/api-reference/workflow-api/start) — starts a workflow run from an API route
@@ -8,7 +8,7 @@ A curated collection of workflow patterns with clean, copy-paste code examples f
8
8
 
9
9
  ## Agent Patterns
10
10
 
11
- - [**Durable Agent**](/cookbook/agent-patterns/durable-agent) — Replace a stateless AI agent with one that survives crashes and retries tool calls
11
+ - [**WorkflowAgent**](/cookbook/agent-patterns/durable-agent) — Build durable, resumable AI agents with AI SDK's WorkflowAgent
12
12
  - [**Human-in-the-Loop**](/cookbook/agent-patterns/human-in-the-loop) — Pause an agent for human approval, then resume based on the decision
13
13
  - [**Agent Cancellation**](/cookbook/agent-patterns/agent-cancellation) — Stop a running agent immediately via `run.cancel()` or gracefully via a hook + `Promise.race`
14
14
 
@@ -9,21 +9,21 @@ related:
9
9
  - /docs/foundations/errors-and-retries
10
10
  ---
11
11
 
12
- This error occurs when the Workflow runtime encounters an event in the event log that no registered consumer can process. This means the event log is in an invalid state typically due to duplicate or orphaned events.
12
+ This error occurs when the Workflow runtime repeatedly cannot replay events in the event log. This usually means the event log is in an invalid state, such as duplicate or orphaned events, or that a runtime determinism bug persists across retry attempts.
13
13
 
14
- This is a **workflow-level fatal error**. It cannot be caught or handled inside your workflow code. A corrupted event log immediately fails the entire run without executing any more user code. The run must be retried from outside the workflow.
14
+ This is a **workflow-level fatal error**. It cannot be caught or handled inside your workflow code. The runtime first retries transient replay divergence automatically; it marks the run as failed with this error only after replay still cannot recover.
15
15
 
16
16
  ## Error Message
17
17
 
18
18
  ```
19
- Unconsumed event in event log: eventType=<type>, correlationId=<id>, eventId=<id>. This indicates a corrupted or invalid event log.
19
+ Workflow replay diverged <divergenceCount> times after <maxRecoveryReplays> recovery replays; latest divergent event was <eventId>. Last divergence: <details>
20
20
  ```
21
21
 
22
22
  ## Why This Happens
23
23
 
24
24
  Workflows persist their progress as an ordered event log. During replay, the runtime processes each event in sequence — every event must be consumed by a matching callback (e.g., a step or sleep waiting for its result). When an event has no matching consumer, the runtime cannot advance past it, which would block all subsequent events and hang the workflow indefinitely.
25
25
 
26
- Instead of silently hanging, the runtime raises a `WorkflowRuntimeError` to fail the workflow fast and surface the problem.
26
+ Instead of silently hanging, the runtime retries a divergent replay before failing the workflow and surfacing this terminal error.
27
27
 
28
28
  Common scenarios that produce this error:
29
29
 
@@ -45,7 +45,7 @@ npm install workflow@latest
45
45
 
46
46
  ### 2. Retry the failed run
47
47
 
48
- Since this is a fatal error, the run is automatically marked as `failed`. You can re-run it using the **Re-run** button in the Workflow Dashboard.
48
+ If this error is displayed, automatic replay recovery has already been exhausted and the run has been marked as `failed`. You can re-run it using the **Re-run** button in the Workflow Dashboard.
49
49
 
50
50
  ### 3. Report the issue
51
51
 
@@ -37,6 +37,9 @@ Fix common mistakes when creating and executing workflows in the **Workflow SDK*
37
37
  <Card href="/docs/errors/corrupted-event-log" title="corrupted-event-log">
38
38
  Learn how to handle corrupted or invalid event logs.
39
39
  </Card>
40
+ <Card href="/docs/errors/replay-divergence" title="replay-divergence">
41
+ Learn how workflow replay divergence is recovered automatically.
42
+ </Card>
40
43
  <Card href="/docs/errors/step-not-registered" title="step-not-registered">
41
44
  Resolve step not registered errors caused by deployment mismatches.
42
45
  </Card>
@@ -0,0 +1,27 @@
1
+ ---
2
+ title: replay-divergence
3
+ description: A workflow replay temporarily followed a path that did not match its recorded events.
4
+ type: troubleshooting
5
+ summary: Understand automatic recovery when a workflow replay diverges from its event history.
6
+ prerequisites:
7
+ - /docs/foundations/workflows-and-steps
8
+ related:
9
+ - /docs/errors/corrupted-event-log
10
+ - /docs/foundations/errors-and-retries
11
+ ---
12
+
13
+ A replay divergence occurs when one invocation of a workflow cannot consume the durable event history using the promises, hooks, sleeps, or steps it created during replay.
14
+
15
+ This is an SDK/runtime signal, not an error thrown by your workflow code. It is not catchable inside a workflow function.
16
+
17
+ ## Automatic Recovery
18
+
19
+ A single divergent replay does not prove that persisted history is corrupted. For example, asynchronous delivery ordering may cause one invocation to follow the wrong side of a race while another replay can follow the recorded history correctly.
20
+
21
+ The runtime automatically queues another replay when an invocation reports `REPLAY_DIVERGENCE`. No terminal `run_failed` event is written during these recovery attempts.
22
+
23
+ If recovery replays continue to diverge after the retry budget is exhausted, the runtime marks the run as failed with `CORRUPTED_EVENT_LOG` and records the latest divergent event for diagnosis.
24
+
25
+ ## What To Do
26
+
27
+ Most replay divergence signals recover without action. If a run ultimately fails with `CORRUPTED_EVENT_LOG`, update to the latest `workflow` package and report the run ID and error details if the failure persists.