workflow 5.0.0-beta.1 → 5.0.0-beta.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -4
- package/dist/api-workflow.d.ts +1 -1
- package/dist/api-workflow.d.ts.map +1 -1
- package/dist/api-workflow.js +2 -2
- package/dist/api.js +1 -1
- package/dist/astro.js +1 -1
- package/dist/index.js +1 -1
- package/dist/internal/builtins.js +1 -1
- package/dist/internal/class-serialization.js +1 -1
- package/dist/internal/errors.js +1 -1
- package/dist/nest.js +1 -1
- package/dist/next.cjs +1 -1
- package/dist/nitro.js +1 -1
- package/dist/nuxt.js +1 -1
- package/dist/observability.js +1 -1
- package/dist/runtime.js +1 -1
- package/dist/stdlib.js +1 -1
- package/dist/sveltekit.js +1 -1
- package/dist/typescript-plugin.cjs +1 -1
- package/dist/vite.js +1 -1
- package/dist/workflow.js +1 -1
- package/docs/ai/resumable-streams.mdx +1 -1
- package/docs/api-reference/workflow/create-webhook.mdx +37 -18
- package/docs/api-reference/workflow/get-workflow-metadata.mdx +34 -0
- package/docs/api-reference/workflow-ai/durable-agent.mdx +0 -4
- package/docs/api-reference/workflow-ai/index.mdx +0 -5
- package/docs/api-reference/workflow-ai/workflow-chat-transport.mdx +0 -4
- package/docs/cookbook/advanced/child-workflows.mdx +372 -0
- package/docs/cookbook/advanced/distributed-abort-controller.mdx +318 -0
- package/docs/cookbook/advanced/meta.json +9 -0
- package/docs/cookbook/advanced/publishing-libraries.mdx +336 -0
- package/docs/cookbook/advanced/serializable-steps.mdx +147 -0
- package/docs/cookbook/agent-patterns/agent-cancellation.mdx +205 -0
- package/docs/cookbook/agent-patterns/durable-agent.mdx +150 -0
- package/docs/cookbook/agent-patterns/human-in-the-loop.mdx +255 -0
- package/docs/cookbook/agent-patterns/meta.json +4 -0
- package/docs/cookbook/common-patterns/batching.mdx +105 -0
- package/docs/cookbook/common-patterns/idempotency.mdx +107 -0
- package/docs/cookbook/common-patterns/meta.json +15 -0
- package/docs/cookbook/common-patterns/rate-limiting.mdx +228 -0
- package/docs/cookbook/common-patterns/saga.mdx +247 -0
- package/docs/cookbook/common-patterns/scheduling.mdx +125 -0
- package/docs/cookbook/common-patterns/sequential-and-parallel.mdx +155 -0
- package/docs/cookbook/common-patterns/timeouts.mdx +99 -0
- package/docs/cookbook/common-patterns/webhooks.mdx +185 -0
- package/docs/cookbook/common-patterns/workflow-composition.mdx +118 -0
- package/docs/cookbook/index.mdx +38 -0
- package/docs/cookbook/integrations/ai-sdk.mdx +360 -0
- package/docs/cookbook/integrations/chat-sdk.mdx +303 -0
- package/docs/cookbook/integrations/meta.json +4 -0
- package/docs/cookbook/integrations/sandbox.mdx +516 -0
- package/docs/cookbook/meta.json +5 -0
- package/docs/deploying/world/local-world.mdx +1 -1
- package/docs/deploying/world/postgres-world.mdx +1 -1
- package/docs/deploying/world/vercel-world.mdx +1 -1
- package/docs/errors/start-invalid-workflow-function.mdx +1 -1
- package/docs/foundations/index.mdx +0 -3
- package/docs/foundations/meta.json +0 -1
- package/docs/foundations/serialization.mdx +1 -1
- package/docs/foundations/starting-workflows.mdx +1 -1
- package/docs/getting-started/index.mdx +8 -1
- package/docs/getting-started/meta.json +2 -1
- package/docs/getting-started/python.mdx +165 -0
- package/docs/meta.json +1 -0
- package/docs/migration-guides/index.mdx +34 -0
- package/docs/migration-guides/meta.json +9 -0
- package/docs/migration-guides/migrating-from-aws-step-functions.mdx +363 -0
- package/docs/migration-guides/migrating-from-inngest.mdx +314 -0
- package/docs/migration-guides/migrating-from-temporal.mdx +318 -0
- package/docs/migration-guides/migrating-from-trigger-dev.mdx +337 -0
- package/package.json +13 -13
- package/docs/foundations/common-patterns.mdx +0 -265
|
@@ -0,0 +1,360 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: AI SDK
|
|
3
|
+
description: Use AI SDK's streamText directly inside durable workflows for lower-level control over model calls and tool execution.
|
|
4
|
+
type: guide
|
|
5
|
+
summary: Use streamText() inside a workflow for full control over model options, stop conditions, and output schemas — while tools remain durable steps.
|
|
6
|
+
related:
|
|
7
|
+
- /docs/ai
|
|
8
|
+
- /docs/ai/chat-session-modeling
|
|
9
|
+
- /docs/ai/defining-tools
|
|
10
|
+
- /docs/ai/resumable-streams
|
|
11
|
+
- /docs/api-reference/workflow-ai/durable-agent
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
[AI SDK](https://ai-sdk.dev/) is Vercel's framework-agnostic TypeScript toolkit for building AI-powered apps and agents — unified provider access, streaming, tool calling, structured output, and UI hooks. Workflow SDK complements it by making those calls durable: the model request, the tool loop, and the multi-turn conversation all survive restarts and timeouts.
|
|
15
|
+
|
|
16
|
+
For the full AI SDK reference (providers, `streamText`, `generateObject`, `useChat`, tool calling, etc.) see the [AI SDK docs](https://ai-sdk.dev/docs). This page covers the Workflow-specific integration points.
|
|
17
|
+
|
|
18
|
+
<Callout type="info">
|
|
19
|
+
For most agent use cases, prefer [`DurableAgent`](/cookbook/agent-patterns/durable-agent) which wraps [`streamText`](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) and manages the tool loop automatically. This page covers using `streamText()` directly when you need lower-level control.
|
|
20
|
+
</Callout>
|
|
21
|
+
|
|
22
|
+
## When to use streamText directly
|
|
23
|
+
|
|
24
|
+
Use [`streamText()`](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) instead of `DurableAgent` when you need:
|
|
25
|
+
|
|
26
|
+
* **Custom stop conditions** — [`stopWhen`](https://ai-sdk.dev/docs/ai-sdk-core/agents#stop-conditions), [`prepareStep`](https://ai-sdk.dev/docs/ai-sdk-core/agents#prepare-step), or [`onStepFinish`](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text#on-step-finish) callbacks
|
|
27
|
+
* **Structured output** — [`Output.object()`](https://ai-sdk.dev/docs/ai-sdk-core/generating-structured-data) or `Output.array()` alongside tool calling
|
|
28
|
+
* **Step-level callbacks** — `onStepFinish` for logging, metrics, or branching logic
|
|
29
|
+
* **Provider options** — per-step model switching, reasoning budgets, or custom [provider options](https://ai-sdk.dev/docs/ai-sdk-core/provider-options)
|
|
30
|
+
|
|
31
|
+
## Multi-turn pattern
|
|
32
|
+
|
|
33
|
+
One workflow run = one full conversation. The workflow suspends between turns on a hook and resumes when the next user message arrives. Conversation state, tool history, and intermediate computation all live inside the run.
|
|
34
|
+
|
|
35
|
+
<Tabs items={['Workflow', 'API Route', 'Client']}>
|
|
36
|
+
|
|
37
|
+
<Tab value="Workflow">
|
|
38
|
+
|
|
39
|
+
```typescript title="workflows/support.ts" lineNumbers
|
|
40
|
+
import { streamText, stepCountIs } from "ai";
|
|
41
|
+
import { defineHook, getWritable, getWorkflowMetadata } from "workflow";
|
|
42
|
+
import type { ModelMessage, UIMessageChunk } from "ai";
|
|
43
|
+
import { z } from "zod";
|
|
44
|
+
|
|
45
|
+
const MAX_TURNS = 20;
|
|
46
|
+
|
|
47
|
+
export const turnHook = defineHook({ // [!code highlight]
|
|
48
|
+
schema: z.object({ message: z.string() }),
|
|
49
|
+
});
|
|
50
|
+
|
|
51
|
+
async function lookupOrder({ orderId }: { orderId: string }) {
|
|
52
|
+
"use step";
|
|
53
|
+
const res = await fetch(`https://api.store.com/orders/${orderId}`);
|
|
54
|
+
return res.json();
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
async function processRefund({ orderId, reason }: { orderId: string; reason: string }) {
|
|
58
|
+
"use step";
|
|
59
|
+
const res = await fetch("https://api.store.com/refunds", {
|
|
60
|
+
method: "POST",
|
|
61
|
+
body: JSON.stringify({ orderId, reason }),
|
|
62
|
+
});
|
|
63
|
+
return res.json();
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
const TOOLS = {
|
|
67
|
+
lookupOrder: {
|
|
68
|
+
description: "Look up an order by ID",
|
|
69
|
+
inputSchema: z.object({ orderId: z.string() }),
|
|
70
|
+
execute: lookupOrder,
|
|
71
|
+
},
|
|
72
|
+
processRefund: {
|
|
73
|
+
description: "Process a refund",
|
|
74
|
+
inputSchema: z.object({ orderId: z.string(), reason: z.string() }),
|
|
75
|
+
execute: processRefund,
|
|
76
|
+
},
|
|
77
|
+
};
|
|
78
|
+
|
|
79
|
+
// Per-turn step — streams one agent response to the durable writable // [!code highlight]
|
|
80
|
+
async function runTurn(messages: ModelMessage[]) {
|
|
81
|
+
"use step";
|
|
82
|
+
|
|
83
|
+
const result = streamText({
|
|
84
|
+
model: "anthropic/claude-haiku-4.5",
|
|
85
|
+
system: "You are a customer support agent.",
|
|
86
|
+
messages,
|
|
87
|
+
tools: TOOLS,
|
|
88
|
+
stopWhen: stepCountIs(8),
|
|
89
|
+
});
|
|
90
|
+
|
|
91
|
+
const writable = getWritable<UIMessageChunk>();
|
|
92
|
+
// preventClose keeps the durable writable open so the next turn can // write to it. Each turn still emits its own start + finish chunks.
|
|
93
|
+
await result.toUIMessageStream().pipeTo(writable, { preventClose: true }); // [!code highlight]
|
|
94
|
+
|
|
95
|
+
const response = await result.response;
|
|
96
|
+
return { responseMessages: response.messages };
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
export async function supportWorkflow(initialMessages: ModelMessage[]) {
|
|
100
|
+
"use workflow";
|
|
101
|
+
|
|
102
|
+
const { workflowRunId } = getWorkflowMetadata();
|
|
103
|
+
// Create the hook once, outside the loop — same token = HookConflictError // [!code highlight]
|
|
104
|
+
const hook = turnHook.create({ token: workflowRunId }); // [!code highlight]
|
|
105
|
+
let allMessages = initialMessages;
|
|
106
|
+
|
|
107
|
+
for (let turn = 0; turn < MAX_TURNS; turn++) {
|
|
108
|
+
const { responseMessages } = await runTurn(allMessages);
|
|
109
|
+
allMessages = [...allMessages, ...responseMessages];
|
|
110
|
+
|
|
111
|
+
const { message } = await hook; // [!code highlight] suspend until next user message
|
|
112
|
+
if (message === "/done") break;
|
|
113
|
+
|
|
114
|
+
allMessages = [...allMessages, { role: "user", content: message }];
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
return { turns: MAX_TURNS };
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
</Tab>
|
|
122
|
+
|
|
123
|
+
<Tab value="API Route">
|
|
124
|
+
|
|
125
|
+
One endpoint handles first turn, follow-ups, and the `/done` exit. The client sends `runId` in the body to distinguish first vs follow-up.
|
|
126
|
+
|
|
127
|
+
```typescript title="app/api/support/route.ts" lineNumbers
|
|
128
|
+
import type { UIMessage, UIMessageChunk } from "ai";
|
|
129
|
+
import { convertToModelMessages, createUIMessageStreamResponse } from "ai";
|
|
130
|
+
import { start, getRun } from "workflow/api";
|
|
131
|
+
import { supportWorkflow, turnHook } from "@/workflows/support";
|
|
132
|
+
|
|
133
|
+
// Pump the durable stream until this turn's `finish` chunk, then close // the HTTP response. The source reader is released (not cancelled) so the
|
|
134
|
+
// workflow's durable stream keeps flowing for the next turn.
|
|
135
|
+
function sliceUntilFinish( // [!code highlight]
|
|
136
|
+
source: ReadableStream<UIMessageChunk>
|
|
137
|
+
): ReadableStream<UIMessageChunk> {
|
|
138
|
+
return new ReadableStream<UIMessageChunk>({
|
|
139
|
+
async start(controller) {
|
|
140
|
+
const reader = source.getReader();
|
|
141
|
+
try {
|
|
142
|
+
while (true) {
|
|
143
|
+
const { done, value } = await reader.read();
|
|
144
|
+
if (done) break;
|
|
145
|
+
controller.enqueue(value);
|
|
146
|
+
if (value.type === "finish") break; // [!code highlight]
|
|
147
|
+
}
|
|
148
|
+
controller.close();
|
|
149
|
+
} catch (e) {
|
|
150
|
+
controller.error(e);
|
|
151
|
+
} finally {
|
|
152
|
+
reader.releaseLock();
|
|
153
|
+
}
|
|
154
|
+
},
|
|
155
|
+
});
|
|
156
|
+
}
|
|
157
|
+
|
|
158
|
+
// `/done` exits the workflow without emitting chunks. Return a synthetic
|
|
159
|
+
// start+finish so useChat's lifecycle terminates cleanly.
|
|
160
|
+
function emptyTurnStream(): ReadableStream<UIMessageChunk> {
|
|
161
|
+
return new ReadableStream<UIMessageChunk>({
|
|
162
|
+
start(controller) {
|
|
163
|
+
controller.enqueue({ type: "start", messageId: crypto.randomUUID() });
|
|
164
|
+
controller.enqueue({ type: "finish" });
|
|
165
|
+
controller.close();
|
|
166
|
+
},
|
|
167
|
+
});
|
|
168
|
+
}
|
|
169
|
+
|
|
170
|
+
export async function POST(req: Request) {
|
|
171
|
+
const { messages, runId }: { messages: UIMessage[]; runId?: string } =
|
|
172
|
+
await req.json();
|
|
173
|
+
const modelMessages = await convertToModelMessages(messages);
|
|
174
|
+
|
|
175
|
+
// Follow-up turn: resume hook, return stream starting AFTER the last turn // [!code highlight]
|
|
176
|
+
if (runId) {
|
|
177
|
+
try {
|
|
178
|
+
const run = getRun(runId);
|
|
179
|
+
|
|
180
|
+
// Snapshot tail before resuming so our slice only contains this turn // [!code highlight]
|
|
181
|
+
const probe = run.getReadable();
|
|
182
|
+
const tailIndex = await probe.getTailIndex();
|
|
183
|
+
await probe.cancel();
|
|
184
|
+
|
|
185
|
+
const lastUser = modelMessages.filter((m) => m.role === "user").at(-1);
|
|
186
|
+
const text =
|
|
187
|
+
typeof lastUser?.content === "string"
|
|
188
|
+
? lastUser.content
|
|
189
|
+
: Array.isArray(lastUser?.content)
|
|
190
|
+
? lastUser.content
|
|
191
|
+
.filter((p): p is { type: "text"; text: string } =>
|
|
192
|
+
"type" in p && p.type === "text"
|
|
193
|
+
)
|
|
194
|
+
.map((p) => p.text)
|
|
195
|
+
.join("")
|
|
196
|
+
: "";
|
|
197
|
+
|
|
198
|
+
await turnHook.resume(runId, { message: text }); // [!code highlight]
|
|
199
|
+
|
|
200
|
+
if (text === "/done") {
|
|
201
|
+
return createUIMessageStreamResponse({
|
|
202
|
+
stream: emptyTurnStream(),
|
|
203
|
+
headers: { "x-workflow-run-id": runId },
|
|
204
|
+
});
|
|
205
|
+
}
|
|
206
|
+
|
|
207
|
+
const stream = sliceUntilFinish(
|
|
208
|
+
run.getReadable({ startIndex: tailIndex + 1 }) // [!code highlight]
|
|
209
|
+
);
|
|
210
|
+
|
|
211
|
+
return createUIMessageStreamResponse({
|
|
212
|
+
stream,
|
|
213
|
+
headers: { "x-workflow-run-id": runId },
|
|
214
|
+
});
|
|
215
|
+
} catch (e: unknown) {
|
|
216
|
+
const msg = e instanceof Error ? e.message.toLowerCase() : "";
|
|
217
|
+
if (!msg.includes("not found") && !msg.includes("expired")) throw e;
|
|
218
|
+
// Stale runId — fall through to start fresh
|
|
219
|
+
}
|
|
220
|
+
}
|
|
221
|
+
|
|
222
|
+
// First turn: start a new workflow // [!code highlight]
|
|
223
|
+
const run = await start(supportWorkflow, [modelMessages]);
|
|
224
|
+
const stream = sliceUntilFinish(run.readable);
|
|
225
|
+
|
|
226
|
+
return createUIMessageStreamResponse({
|
|
227
|
+
stream,
|
|
228
|
+
headers: { "x-workflow-run-id": run.runId },
|
|
229
|
+
});
|
|
230
|
+
}
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
</Tab>
|
|
234
|
+
|
|
235
|
+
<Tab value="Client">
|
|
236
|
+
|
|
237
|
+
Store the `runId` in a ref and pass it in the body of every follow-up. `WorkflowChatTransport` forwards it for you.
|
|
238
|
+
|
|
239
|
+
```tsx title="components/support-chat.tsx" lineNumbers
|
|
240
|
+
"use client";
|
|
241
|
+
|
|
242
|
+
import { useChat } from "@ai-sdk/react";
|
|
243
|
+
import { WorkflowChatTransport } from "@workflow/ai";
|
|
244
|
+
import { useMemo, useRef, useState } from "react";
|
|
245
|
+
|
|
246
|
+
export function SupportChat() {
|
|
247
|
+
const [input, setInput] = useState("");
|
|
248
|
+
const runIdRef = useRef<string | null>(null); // [!code highlight]
|
|
249
|
+
|
|
250
|
+
const transport = useMemo(
|
|
251
|
+
() =>
|
|
252
|
+
new WorkflowChatTransport({
|
|
253
|
+
api: "/api/support",
|
|
254
|
+
prepareSendMessagesRequest: ({ messages, body }) => ({
|
|
255
|
+
body: { ...body, messages, runId: runIdRef.current }, // [!code highlight]
|
|
256
|
+
}),
|
|
257
|
+
onChatSendMessage: (response) => {
|
|
258
|
+
const id = response.headers.get("x-workflow-run-id");
|
|
259
|
+
if (id) runIdRef.current = id; // [!code highlight]
|
|
260
|
+
},
|
|
261
|
+
}),
|
|
262
|
+
[]
|
|
263
|
+
);
|
|
264
|
+
|
|
265
|
+
const { messages, sendMessage, status } = useChat({ transport });
|
|
266
|
+
const busy = status === "streaming" || status === "submitted";
|
|
267
|
+
|
|
268
|
+
return (
|
|
269
|
+
<form
|
|
270
|
+
onSubmit={(e) => {
|
|
271
|
+
e.preventDefault();
|
|
272
|
+
if (busy || !input.trim()) return;
|
|
273
|
+
sendMessage({ text: input });
|
|
274
|
+
setInput("");
|
|
275
|
+
}}
|
|
276
|
+
>
|
|
277
|
+
{messages.map((m) => (
|
|
278
|
+
<div key={m.id}>{m.role}: {m.parts.map((p) => p.type === "text" ? p.text : "").join("")}</div>
|
|
279
|
+
))}
|
|
280
|
+
<input value={input} onChange={(e) => setInput(e.target.value)} disabled={busy} />
|
|
281
|
+
</form>
|
|
282
|
+
);
|
|
283
|
+
}
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
</Tab>
|
|
287
|
+
|
|
288
|
+
</Tabs>
|
|
289
|
+
|
|
290
|
+
## How it works
|
|
291
|
+
|
|
292
|
+
1. **One workflow = one conversation.** The workflow loops on a hook, keeping `allMessages`, tool history, and state alive across turns.
|
|
293
|
+
2. **Hook is created once.** `turnHook.create({ token: workflowRunId })` outside the loop — calling it twice with the same token throws `HookConflictError`.
|
|
294
|
+
3. **`preventClose: true`** on `pipeTo` keeps the durable writable open so the next turn can write to it.
|
|
295
|
+
4. **`sliceUntilFinish`** in the API reads chunks until `type === "finish"`, then closes the HTTP response. The source reader is released — not cancelled — so the workflow stream keeps flowing.
|
|
296
|
+
5. **`startIndex: tailIndex + 1`** gives each follow-up response only the new chunks, avoiding replay of previous turns.
|
|
297
|
+
6. **`/done`** resumes the hook so the workflow exits cleanly, then returns a synthetic `start` + `finish` so `useChat` transitions out of "streaming".
|
|
298
|
+
|
|
299
|
+
## Pitfalls
|
|
300
|
+
|
|
301
|
+
Non-obvious correctness details worth knowing before adapting this pattern.
|
|
302
|
+
|
|
303
|
+
### Snapshot `tailIndex` *before* resuming the hook
|
|
304
|
+
|
|
305
|
+
{/* @skip-typecheck - fragment referencing variables from the surrounding multi-turn pattern */}
|
|
306
|
+
```typescript
|
|
307
|
+
const tailIndex = await probe.getTailIndex(); // [!code highlight] FIRST
|
|
308
|
+
await probe.cancel();
|
|
309
|
+
await turnHook.resume(runId, { message: text }); // [!code highlight] THEN
|
|
310
|
+
const stream = run.getReadable({ startIndex: tailIndex + 1 });
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
Reversing the order races the workflow: by the time you read `tailIndex`, the next turn has already written its `start` chunk, and your `startIndex + 1` skips past it.
|
|
314
|
+
|
|
315
|
+
### Don't call `writable.close()` inside a workflow function
|
|
316
|
+
|
|
317
|
+
I/O operations like closing streams must happen inside a `"use step"` function. Calling `writable.close()` directly in the workflow body throws `Not supported in workflow functions`. When the workflow returns, the runtime closes the underlying writable for you.
|
|
318
|
+
|
|
319
|
+
### Don't use `TransformStream.terminate()` to slice the stream
|
|
320
|
+
|
|
321
|
+
A `TransformStream` with `controller.terminate()` on the `finish` chunk seems like the obvious fit for `sliceUntilFinish`, but throws `Invalid state: TransformStream has been terminated` when late-arriving chunks hit the transform callback. Manual pumping through a custom `ReadableStream` (as shown above) sidesteps the problem entirely.
|
|
322
|
+
|
|
323
|
+
### Release the source reader, don't cancel it
|
|
324
|
+
|
|
325
|
+
In `sliceUntilFinish`, use `reader.releaseLock()` in the `finally` block rather than `source.cancel()`. Cancelling propagates upstream and closes the durable writable, breaking the next turn. Releasing the lock just detaches our reader; the durable stream keeps flowing.
|
|
326
|
+
|
|
327
|
+
### Handle stale `runId` gracefully
|
|
328
|
+
|
|
329
|
+
Clients can send a `runId` from a long-gone workflow (localStorage, back button, server restart). Wrap the follow-up path in a try/catch for `not found` / `expired` and fall through to the first-turn code path to start a fresh workflow.
|
|
330
|
+
|
|
331
|
+
## streamText vs DurableAgent
|
|
332
|
+
|
|
333
|
+
| | `streamText()` | `DurableAgent` |
|
|
334
|
+
|---|---|---|
|
|
335
|
+
| **Tool loop** | AI SDK handles via `stopWhen` | DurableAgent handles internally |
|
|
336
|
+
| **LLM call durability** | Re-executes on replay | Each LLM call is a durable step |
|
|
337
|
+
| **Stop conditions** | `stopWhen`, `prepareStep` | `prepareStep` only |
|
|
338
|
+
| **Structured output** | `Output.object()`, `Output.array()` | Not available |
|
|
339
|
+
| **Step callbacks** | `onStepFinish`, `onChunk` | Not available |
|
|
340
|
+
| **Setup** | Manual stream piping | Automatic |
|
|
341
|
+
|
|
342
|
+
Use `DurableAgent` for most agent use cases. Use `streamText` when you need the additional control.
|
|
343
|
+
|
|
344
|
+
## Key APIs
|
|
345
|
+
|
|
346
|
+
**AI SDK** ([docs](https://ai-sdk.dev/docs))
|
|
347
|
+
|
|
348
|
+
* [`streamText()`](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text) — core streaming function; `toUIMessageStream()` pipes into the durable writable
|
|
349
|
+
* [`tool()` / tool calling](https://ai-sdk.dev/docs/ai-sdk-core/tools-and-tool-calling) — tools wrap `"use step"` functions so each tool call is replayed from the log, not re-executed
|
|
350
|
+
* [`stepCountIs()` / `stopWhen`](https://ai-sdk.dev/docs/ai-sdk-core/agents#stop-conditions) — bound the agent loop inside each turn
|
|
351
|
+
* [`convertToModelMessages()`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/convert-to-model-messages) / [`createUIMessageStreamResponse()`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/create-ui-message-stream-response) — UI ↔ model message conversion at the API boundary
|
|
352
|
+
* [`useChat()`](https://ai-sdk.dev/docs/reference/ai-sdk-ui/use-chat) — React hook that consumes the UI message stream on the client
|
|
353
|
+
|
|
354
|
+
**Workflow SDK**
|
|
355
|
+
|
|
356
|
+
* [`"use step"`](/docs/api-reference/workflow/use-step) — makes tool executions durable
|
|
357
|
+
* [`defineHook()`](/docs/api-reference/workflow/define-hook) — suspension point for follow-up messages
|
|
358
|
+
* [`getWritable()`](/docs/api-reference/workflow/get-writable) — resumable stream output
|
|
359
|
+
* [`getRun()`](/docs/api-reference/workflow-api/get-run) — `run.getReadable({ startIndex })` for slicing per-turn streams
|
|
360
|
+
* [`WorkflowChatTransport`](/docs/api-reference/workflow-ai/workflow-chat-transport) — passes `runId` between turns
|
|
@@ -0,0 +1,303 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Chat SDK
|
|
3
|
+
description: Make Chat SDK bot sessions durable — one workflow run per conversation thread, with hooks bridging inbound platform events into long-running agent logic.
|
|
4
|
+
type: guide
|
|
5
|
+
summary: Chat SDK normalizes Slack, Teams, Discord, Telegram and friends into one thread/message model. Workflow SDK gives each thread a durable run that owns multi-turn state, can sleep for hours, and survives restarts.
|
|
6
|
+
related:
|
|
7
|
+
- /docs/cookbook/integrations/ai-sdk
|
|
8
|
+
- /docs/cookbook/integrations/sandbox
|
|
9
|
+
- /docs/api-reference/workflow/define-hook
|
|
10
|
+
- /docs/api-reference/workflow-api/start
|
|
11
|
+
- /docs/api-reference/workflow-api/get-run
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
[Chat SDK](https://chat-sdk.dev/) is a unified TypeScript SDK for building bots across Slack, Microsoft Teams, Google Chat, Discord, Telegram, GitHub, Linear, and WhatsApp. Write the bot once, deploy to every platform. It handles webhook verification, event normalization, subscriptions, and cross-platform features like cards and modals.
|
|
15
|
+
|
|
16
|
+
Workflow SDK complements it by making bot **sessions** durable. Each conversation thread maps to a long-running workflow run that:
|
|
17
|
+
|
|
18
|
+
- Owns multi-turn state in the durable event log instead of Redis-by-hand bookkeeping
|
|
19
|
+
- Can `sleep()` for hours or days waiting for a user reply, an approval, or a scheduled follow-up
|
|
20
|
+
- Survives deploys, cold starts, and crashes — the session picks up from the last step on replay
|
|
21
|
+
- Receives follow-up messages via hooks, so the bot stays responsive while the workflow is still running
|
|
22
|
+
|
|
23
|
+
The rest of this page covers the integration pattern. For a full Slack + Next.js + Redis walkthrough, see the [Durable chat sessions guide](https://chat-sdk.dev/docs/guides/durable-chat-sessions-nextjs) on chat-sdk.dev.
|
|
24
|
+
|
|
25
|
+
## How It Fits Together
|
|
26
|
+
|
|
27
|
+
Chat SDK owns the edge — webhook verification, event routing, `thread.post()` / `thread.stream()`. Workflow owns the session — state, loops, sleeps, retries. They meet at exactly two points:
|
|
28
|
+
|
|
29
|
+
```mermaid
|
|
30
|
+
flowchart TD
|
|
31
|
+
A["Platform webhook"] --> B["Chat SDK event handler<br/>(onNewMention, onSubscribedMessage, …)"]
|
|
32
|
+
B -->|"no runId in thread state"| C["start(durableChatSession, …)"]
|
|
33
|
+
B -->|"runId in thread state"| D["resumeHook(runId, { message })"]
|
|
34
|
+
C --> E["Workflow run (durable)<br/>one per thread; suspends between turns"]
|
|
35
|
+
D --> E
|
|
36
|
+
E --> F[""use step" helpers<br/>thread.post(), thread.subscribe(), thread.setState(), …"]
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
- **Inbound** — Chat SDK handlers decide whether to `start(workflow, [thread, message])` or `resumeHook(runId, { message })`. The `runId` lives in Chat SDK's thread state (Redis, Postgres, or any state adapter).
|
|
40
|
+
- **Outbound** — the workflow calls Chat SDK APIs (`thread.post()`, `thread.subscribe()`, `thread.setState()`) from inside step functions. Never from the top level of a workflow file — adapter packages use Node-only modules that aren't available in the workflow sandbox.
|
|
41
|
+
|
|
42
|
+
## Why Workflow + Chat SDK
|
|
43
|
+
|
|
44
|
+
Without Workflow, a long-running bot session usually means one of:
|
|
45
|
+
- Holding a webhook request open while the agent runs (doesn't survive restarts, blows past platform timeouts)
|
|
46
|
+
- Writing session state to Redis manually, plus a scheduler for timeouts and retries, plus custom reconnection logic
|
|
47
|
+
|
|
48
|
+
Workflow replaces all of that with a single durable function. The bot can:
|
|
49
|
+
|
|
50
|
+
- Run a tool loop for minutes while the user watches typing indicators
|
|
51
|
+
- Wait for a human approval in another thread before continuing
|
|
52
|
+
- Schedule a follow-up message 24 hours later via `sleep("24h")`
|
|
53
|
+
- Pause on sandbox snapshot, resume when the user sends the next command (see the [Sandbox integration](/docs/cookbook/integrations/sandbox))
|
|
54
|
+
|
|
55
|
+
Because the session *is* a workflow run, its history is recoverable from the event log — no separate message store to keep in sync.
|
|
56
|
+
|
|
57
|
+
## The Pattern: One Thread = One Workflow Run
|
|
58
|
+
|
|
59
|
+
Three files. The bot definition is separate from the workflow so adapter packages stay out of the workflow sandbox.
|
|
60
|
+
|
|
61
|
+
<Tabs items={['Bot Setup', 'Workflow', 'Event Handlers']}>
|
|
62
|
+
|
|
63
|
+
<Tab value="Bot Setup">
|
|
64
|
+
|
|
65
|
+
Register the `Chat` instance as a singleton so step functions can dynamically import it and resolve adapters + state:
|
|
66
|
+
|
|
67
|
+
```typescript title="lib/bot.ts" lineNumbers
|
|
68
|
+
import { Chat } from "chat";
|
|
69
|
+
import { createSlackAdapter } from "@chat-adapter/slack";
|
|
70
|
+
import { createRedisState } from "@chat-adapter/state-redis";
|
|
71
|
+
|
|
72
|
+
const adapters = {
|
|
73
|
+
slack: createSlackAdapter(),
|
|
74
|
+
};
|
|
75
|
+
|
|
76
|
+
export interface ThreadState {
|
|
77
|
+
runId?: string; // [!code highlight]
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
export const bot = new Chat<typeof adapters, ThreadState>({
|
|
81
|
+
userName: "durable-bot",
|
|
82
|
+
adapters,
|
|
83
|
+
state: createRedisState(),
|
|
84
|
+
dedupeTtlMs: 600_000,
|
|
85
|
+
}).registerSingleton(); // [!code highlight]
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
`registerSingleton()` is important: Chat SDK re-hydrates `Thread` objects inside step functions, and it needs a registered singleton to resolve adapters and state for those rehydrated instances.
|
|
89
|
+
|
|
90
|
+
</Tab>
|
|
91
|
+
|
|
92
|
+
<Tab value="Workflow">
|
|
93
|
+
|
|
94
|
+
The workflow is a plain loop over a hook. It receives the serialized thread + first message from the handler, revives them via Chat SDK's standalone `reviver`, and every platform-side effect goes inside a `"use step"` helper:
|
|
95
|
+
|
|
96
|
+
```typescript title="workflows/durable-chat-session.ts" lineNumbers
|
|
97
|
+
import { Message, reviver, type Thread } from "chat";
|
|
98
|
+
import { defineHook, getWorkflowMetadata } from "workflow";
|
|
99
|
+
import type { ThreadState } from "@/lib/bot";
|
|
100
|
+
|
|
101
|
+
// Hook payload lives in its own file so the webhook side can import it without
|
|
102
|
+
// pulling in the workflow module.
|
|
103
|
+
import type { ChatTurnPayload } from "@/workflows/chat-turn-hook";
|
|
104
|
+
|
|
105
|
+
const chatTurnHook = defineHook<ChatTurnPayload>(); // [!code highlight]
|
|
106
|
+
|
|
107
|
+
async function postAssistantMessage(
|
|
108
|
+
thread: Thread<ThreadState>,
|
|
109
|
+
text: string
|
|
110
|
+
) {
|
|
111
|
+
"use step";
|
|
112
|
+
// Dynamic import keeps adapter packages out of the workflow sandbox.
|
|
113
|
+
const { bot } = await import("@/lib/bot"); // [!code highlight]
|
|
114
|
+
await bot.initialize();
|
|
115
|
+
await thread.post(text);
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
async function runTurn(text: string) {
|
|
119
|
+
"use step";
|
|
120
|
+
// Your AI SDK call, database lookup, tool loop, etc.
|
|
121
|
+
return `You said: ${text}`;
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
async function handleMessage(
|
|
125
|
+
thread: Thread<ThreadState>,
|
|
126
|
+
message: Message
|
|
127
|
+
) {
|
|
128
|
+
const text = message.text.trim();
|
|
129
|
+
if (text.toLowerCase() === "done") return false;
|
|
130
|
+
|
|
131
|
+
const reply = await runTurn(text);
|
|
132
|
+
await postAssistantMessage(thread, reply);
|
|
133
|
+
return true;
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
export async function durableChatSession(payload: string) {
|
|
137
|
+
"use workflow";
|
|
138
|
+
|
|
139
|
+
const { workflowRunId } = getWorkflowMetadata();
|
|
140
|
+
const { thread, message } = JSON.parse(payload, reviver) as { // [!code highlight]
|
|
141
|
+
thread: Thread<ThreadState>;
|
|
142
|
+
message: Message;
|
|
143
|
+
};
|
|
144
|
+
|
|
145
|
+
const hook = chatTurnHook.create({ token: workflowRunId });
|
|
146
|
+
|
|
147
|
+
await postAssistantMessage(thread, "Session started. Reply here; send `done` to stop.");
|
|
148
|
+
|
|
149
|
+
if (!(await handleMessage(thread, message))) return;
|
|
150
|
+
|
|
151
|
+
// Each hook resumption is one turn. The workflow stays suspended between
|
|
152
|
+
// messages — zero compute cost while idle.
|
|
153
|
+
while (true) {
|
|
154
|
+
const { message: nextRaw } = await hook; // [!code highlight]
|
|
155
|
+
const next = Message.fromJSON(nextRaw);
|
|
156
|
+
if (!(await handleMessage(thread, next))) return;
|
|
157
|
+
}
|
|
158
|
+
}
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
```typescript title="workflows/chat-turn-hook.ts" lineNumbers
|
|
162
|
+
import type { SerializedMessage } from "chat";
|
|
163
|
+
|
|
164
|
+
export type ChatTurnPayload = {
|
|
165
|
+
message: SerializedMessage;
|
|
166
|
+
};
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
</Tab>
|
|
170
|
+
|
|
171
|
+
<Tab value="Event Handlers">
|
|
172
|
+
|
|
173
|
+
Handlers live outside the workflow file so adapter dependencies don't leak in. They decide whether to start a new workflow or resume an existing one, then store the `runId` in thread state:
|
|
174
|
+
|
|
175
|
+
```typescript title="lib/chat-session-handlers.ts" lineNumbers
|
|
176
|
+
import type { Message, Thread } from "chat";
|
|
177
|
+
import { getRun, resumeHook, start } from "workflow/api";
|
|
178
|
+
import { bot, type ThreadState } from "@/lib/bot";
|
|
179
|
+
import { durableChatSession } from "@/workflows/durable-chat-session";
|
|
180
|
+
import type { ChatTurnPayload } from "@/workflows/chat-turn-hook";
|
|
181
|
+
|
|
182
|
+
async function startSession(thread: Thread<ThreadState>, message: Message) {
|
|
183
|
+
const run = await start(durableChatSession, [ // [!code highlight]
|
|
184
|
+
JSON.stringify({
|
|
185
|
+
thread: thread.toJSON(),
|
|
186
|
+
message: message.toJSON(),
|
|
187
|
+
}),
|
|
188
|
+
]);
|
|
189
|
+
await thread.setState({ runId: run.runId });
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
async function routeTurn(thread: Thread<ThreadState>, message: Message) {
|
|
193
|
+
const state = await thread.state;
|
|
194
|
+
|
|
195
|
+
// No run yet, or the previous run finished — start fresh.
|
|
196
|
+
if (!state?.runId || !(await getRun(state.runId).exists)) {
|
|
197
|
+
await startSession(thread, message);
|
|
198
|
+
return;
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
try {
|
|
202
|
+
await resumeHook<ChatTurnPayload>(state.runId, { // [!code highlight]
|
|
203
|
+
message: message.toJSON(),
|
|
204
|
+
});
|
|
205
|
+
} catch (err) {
|
|
206
|
+
const msg = err instanceof Error ? err.message.toLowerCase() : "";
|
|
207
|
+
if (msg.includes("not found") || msg.includes("expired")) {
|
|
208
|
+
// Stale runId — start a new session rather than dropping the message.
|
|
209
|
+
await startSession(thread, message);
|
|
210
|
+
return;
|
|
211
|
+
}
|
|
212
|
+
throw err;
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
bot.onNewMention(async (thread, message) => {
|
|
217
|
+
await thread.subscribe();
|
|
218
|
+
await routeTurn(thread, message);
|
|
219
|
+
});
|
|
220
|
+
|
|
221
|
+
bot.onSubscribedMessage(async (thread, message) => {
|
|
222
|
+
await routeTurn(thread, message);
|
|
223
|
+
});
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
Wire Chat SDK's webhook handler into a catch-all route. Importing `chat-session-handlers` for side effects registers the event handlers before the first webhook arrives:
|
|
227
|
+
|
|
228
|
+
```typescript title="app/api/webhooks/[platform]/route.ts" lineNumbers
|
|
229
|
+
import "@/lib/chat-session-handlers";
|
|
230
|
+
import { after } from "next/server";
|
|
231
|
+
import { bot } from "@/lib/bot";
|
|
232
|
+
|
|
233
|
+
type Platform = keyof typeof bot.webhooks;
|
|
234
|
+
|
|
235
|
+
export async function POST(
|
|
236
|
+
req: Request,
|
|
237
|
+
{ params }: { params: Promise<{ platform: string }> }
|
|
238
|
+
) {
|
|
239
|
+
const { platform } = await params;
|
|
240
|
+
const handler = bot.webhooks[platform as Platform];
|
|
241
|
+
if (!handler) return new Response(`Unknown platform: ${platform}`, { status: 404 });
|
|
242
|
+
|
|
243
|
+
return handler(req, { waitUntil: (task) => after(() => task) }); // [!code highlight]
|
|
244
|
+
}
|
|
245
|
+
```
|
|
246
|
+
|
|
247
|
+
</Tab>
|
|
248
|
+
|
|
249
|
+
</Tabs>
|
|
250
|
+
|
|
251
|
+
## How It Works
|
|
252
|
+
|
|
253
|
+
1. **Thread state stores the `runId`.** Chat SDK's state adapter (Redis, Postgres, memory) holds `{ runId }` per thread. That's the only piece of glue between the two SDKs.
|
|
254
|
+
2. **First mention → `start()`.** Handler serializes `thread` + `message` with `toJSON()`, passes them through `start(durableChatSession, [payload])`, stashes the returned `runId` in thread state.
|
|
255
|
+
3. **Subsequent messages → `resumeHook()`.** Handler looks up the `runId`, serializes the new message, and resumes the workflow's hook. The workflow picks up on the next `await hook` iteration.
|
|
256
|
+
4. **Workflow posts back via steps.** All Chat SDK side effects (`thread.post`, `thread.subscribe`, `thread.setState`) happen inside `"use step"` helpers that dynamically import the bot. This keeps adapter packages outside the workflow sandbox.
|
|
257
|
+
5. **Session ends — two ways.** The workflow returns normally (user said `done`, approval granted, etc.), or the workflow throws. Either way the run completes; the next inbound message with the stale `runId` falls through to `startSession()`.
|
|
258
|
+
|
|
259
|
+
The workflow is fully durable between turns: `await hook` suspends with zero compute cost, and platform webhooks can fire from anywhere without concern for which server instance handled the previous turn.
|
|
260
|
+
|
|
261
|
+
## Extending the Pattern
|
|
262
|
+
|
|
263
|
+
Because the session is just a workflow, everything else from the cookbook composes naturally:
|
|
264
|
+
|
|
265
|
+
- **Stream AI SDK responses into the thread.** Use the [AI SDK integration](/docs/cookbook/integrations/ai-sdk) pattern inside a step, then pass `result.fullStream` to `thread.post()` — Chat SDK handles platform-specific streaming (Slack edit-in-place, Telegram message-per-chunk, etc.).
|
|
266
|
+
- **Give the bot a sandbox.** Combine with the [Sandbox integration](/docs/cookbook/integrations/sandbox): each thread gets its own persistent sandbox session, snapshots on idle, resumes on the next message. That's effectively a coding-agent bot.
|
|
267
|
+
- **Human-in-the-loop approvals.** `Promise.race([hook, approvalHook])` inside the workflow, post buttons in the thread via [cards](https://chat-sdk.dev/docs/cards), resume `approvalHook` from `bot.onAction(...)`.
|
|
268
|
+
- **Scheduled follow-ups.** `sleep("24h")` before a proactive check-in. Surviving restarts is free.
|
|
269
|
+
|
|
270
|
+
## Pitfalls
|
|
271
|
+
|
|
272
|
+
### Don't import the bot at the top of workflow files
|
|
273
|
+
|
|
274
|
+
Adapter packages (`@chat-adapter/slack`, `@chat-adapter/telegram`, etc.) depend on Node-only modules that aren't available in the workflow bundler's sandbox. Keep `import { bot } from "@/lib/bot"` inside `"use step"` functions with `await import(...)`. Use `reviver` from `chat` for deserialization inside the workflow — it's standalone and has no adapter dependencies.
|
|
275
|
+
|
|
276
|
+
### Register the bot as a singleton
|
|
277
|
+
|
|
278
|
+
`new Chat({...}).registerSingleton()`. Chat SDK rehydrates `Thread` objects inside step functions via `reviver`, and it looks up adapters + state from the registered singleton. Without it, thread methods throw when called from step contexts.
|
|
279
|
+
|
|
280
|
+
### Hook payloads must be JSON-serializable
|
|
281
|
+
|
|
282
|
+
`Message` and `Thread` have methods, so pass them through `.toJSON()` / `Message.fromJSON()` across the hook boundary. Define a `ChatTurnPayload` type in its own file so both the webhook handler (in the Node bundle) and the workflow (in the workflow sandbox) can share it without dragging in adapter code.
|
|
283
|
+
|
|
284
|
+
### Handle stale `runId`s
|
|
285
|
+
|
|
286
|
+
A workflow run ends but its `runId` is still cached in thread state. The next message calls `resumeHook` on a dead run and throws `not found` / `expired`. Gate on `getRun(runId).exists` before resuming, or catch the error and fall through to `startSession`. Either way the user's message must not be dropped.
|
|
287
|
+
|
|
288
|
+
### Keep the hook outside the loop
|
|
289
|
+
|
|
290
|
+
One `chatTurnHook.create({ token: workflowRunId })` per workflow run, reused every iteration. Creating a new hook with the same token throws `HookConflictError`. This is the same rule as the [AI SDK](/docs/cookbook/integrations/ai-sdk) and [Sandbox](/docs/cookbook/integrations/sandbox) session patterns.
|
|
291
|
+
|
|
292
|
+
### Platform timeouts are separate from workflow timeouts
|
|
293
|
+
|
|
294
|
+
Slack wants a 200 within 3 seconds. The webhook handler returns immediately after `resumeHook` (which is fast) — the workflow then runs in the background and posts back via `thread.post`. Don't try to `await` the whole turn inside the webhook handler; that's what breaks in the naive integration.
|
|
295
|
+
|
|
296
|
+
## Key APIs
|
|
297
|
+
|
|
298
|
+
- [`Chat`](https://chat-sdk.dev/docs/api/chat) / [`Thread`](https://chat-sdk.dev/docs/api/thread) / [`Message`](https://chat-sdk.dev/docs/api/message) — Chat SDK primitives. `toJSON()` / `fromJSON()` / `reviver` are the serialization layer.
|
|
299
|
+
- [`start()`](/docs/api-reference/workflow-api/start) — start a new session workflow. Store the returned `runId` in thread state.
|
|
300
|
+
- [`resumeHook()`](/docs/api-reference/workflow-api/resume-hook) — forward a new platform message to the running workflow.
|
|
301
|
+
- [`getRun()`](/docs/api-reference/workflow-api/get-run) — `run.exists` before resuming, to detect stale `runId`s.
|
|
302
|
+
- [`defineHook()`](/docs/api-reference/workflow/define-hook) — per-turn suspension point inside the workflow.
|
|
303
|
+
- [`registerSingleton()`](https://chat-sdk.dev/docs/api/chat) — makes the bot resolvable from inside step functions.
|