ai 7.0.94 → 7.0.95
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/dist/index.js +7 -1
- package/dist/index.js.map +1 -1
- package/dist/internal/index.js +1 -1
- package/docs/03-ai-sdk-core/35-image-generation.mdx +2 -0
- package/docs/03-ai-sdk-core/42-batch.mdx +272 -0
- package/docs/03-ai-sdk-core/index.mdx +5 -0
- package/docs/03-ai-sdk-harnesses/05-harness-adapters.mdx +13 -11
- package/docs/06-advanced/04-caching.mdx +7 -0
- package/docs/07-reference/01-ai-sdk-core/20-start-batch.mdx +155 -0
- package/docs/07-reference/01-ai-sdk-core/21-get-batch-status.mdx +133 -0
- package/docs/07-reference/01-ai-sdk-core/22-get-batch-results.mdx +109 -0
- package/docs/07-reference/01-ai-sdk-core/index.mdx +15 -0
- package/package.json +3 -3
- package/src/embed/embed.ts +8 -0
package/dist/internal/index.js
CHANGED
|
@@ -314,6 +314,8 @@ for (const file of result.files) {
|
|
|
314
314
|
| Provider | Model | Support sizes (`width x height`) or aspect ratios (`width : height`) |
|
|
315
315
|
| ------------------------------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
316
316
|
| [xAI Grok](/providers/ai-sdk-providers/xai#image-models) | `grok-imagine-image` | `1:1`, `16:9`, `9:16`, `4:3`, `3:4`, `3:2`, `2:3`, `2:1`, `1:2`, `19.5:9`, `9:19.5`, `20:9`, `9:20`, `auto` |
|
|
317
|
+
| [OpenAI](/providers/ai-sdk-providers/openai#image-models) | `gpt-image-2.5-flare` | 1024x1024, 1536x1024, 1024x1536, custom |
|
|
318
|
+
| [OpenAI](/providers/ai-sdk-providers/openai#image-models) | `gpt-image-2.5-sunburst` | 1024x1024, 1536x1024, 1024x1536, custom |
|
|
317
319
|
| [OpenAI](/providers/ai-sdk-providers/openai#image-models) | `gpt-image-2` | 1024x1024, 1536x1024, 1024x1536 |
|
|
318
320
|
| [OpenAI](/providers/ai-sdk-providers/openai#image-models) | `dall-e-3` | 1024x1024, 1792x1024, 1024x1792 |
|
|
319
321
|
| [OpenAI](/providers/ai-sdk-providers/openai#image-models) | `dall-e-2` | 256x256, 512x512, 1024x1024 |
|
|
@@ -0,0 +1,272 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Batch
|
|
3
|
+
description: Learn how to process asynchronous batches with the AI SDK.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Batch
|
|
7
|
+
|
|
8
|
+
<Note type="warning">
|
|
9
|
+
Batch support is experimental and the API may change in patch releases.
|
|
10
|
+
</Note>
|
|
11
|
+
|
|
12
|
+
Batches let you submit multiple independent requests for asynchronous
|
|
13
|
+
processing. The provider processes the batch in the background, so your
|
|
14
|
+
application does not need to keep the request open while the model generates
|
|
15
|
+
the results. This is useful for workloads such as classification,
|
|
16
|
+
summarization, and content generation that do not need an immediate response.
|
|
17
|
+
|
|
18
|
+
The batch API is designed to support multiple request types. Currently, only
|
|
19
|
+
`type: 'text'` is supported.
|
|
20
|
+
|
|
21
|
+
The AI SDK provides three functions for the batch lifecycle:
|
|
22
|
+
|
|
23
|
+
- [`experimental_startBatch`](/docs/reference/ai-sdk-core/start-batch)
|
|
24
|
+
submits a batch and returns its initial status and a serializable reference.
|
|
25
|
+
- [`experimental_getBatchStatus`](/docs/reference/ai-sdk-core/get-batch-status)
|
|
26
|
+
retrieves the latest status and request counts.
|
|
27
|
+
- [`experimental_getBatchResults`](/docs/reference/ai-sdk-core/get-batch-results)
|
|
28
|
+
asynchronously iterates over the terminal result for each request.
|
|
29
|
+
|
|
30
|
+
All three functions are exported from `ai`. The examples below use aliases so
|
|
31
|
+
the shorter names can be used in application code:
|
|
32
|
+
|
|
33
|
+
```ts
|
|
34
|
+
import {
|
|
35
|
+
experimental_getBatchResults as getBatchResults,
|
|
36
|
+
experimental_getBatchStatus as getBatchStatus,
|
|
37
|
+
experimental_startBatch as startBatch,
|
|
38
|
+
} from 'ai';
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Supported providers
|
|
42
|
+
|
|
43
|
+
Batch processing requires a provider that implements the batch interface.
|
|
44
|
+
Support is provider- and model-specific. The first-party providers with
|
|
45
|
+
support are:
|
|
46
|
+
|
|
47
|
+
| Provider | Provider value | Model example | Provider API |
|
|
48
|
+
| ---------------------------------------------------- | -------------- | ------------------------------ | --------------------------------------------------------------------------------------------- |
|
|
49
|
+
| [Anthropic](/providers/ai-sdk-providers/anthropic) | `anthropic` | `'claude-haiku-4-5'` | [Message Batches API](https://platform.claude.com/docs/en/build-with-claude/batch-processing) |
|
|
50
|
+
| [Google](/providers/ai-sdk-providers/google) | `google` | `'gemini-3.6-flash'` | [Gemini Batch API](https://ai.google.dev/gemini-api/docs/batch-api) |
|
|
51
|
+
| [OpenAI](/providers/ai-sdk-providers/openai) | `openai` | `'gpt-4.1-nano'` | [Batch API](https://developers.openai.com/api/docs/guides/batch) |
|
|
52
|
+
| [xAI](/providers/ai-sdk-providers/xai) | `xai` | `'grok-4.3'` | [Batch API](https://docs.x.ai/developers/advanced-api-usage/batch-api) |
|
|
53
|
+
| [AI Gateway](/providers/ai-sdk-providers/ai-gateway) | global default | `'anthropic/claude-haiku-4.5'` | [Batch processing](https://vercel.com/docs/ai-gateway/models-and-providers/batch-processing) |
|
|
54
|
+
|
|
55
|
+
See the provider documentation for the supported models, limits, and native
|
|
56
|
+
batch behavior. For example, OpenAI batch support is available through the
|
|
57
|
+
Responses API, not `openai.chat()`, and xAI batch support is available through
|
|
58
|
+
the Responses API, not `xai.chat()`.
|
|
59
|
+
|
|
60
|
+
## Starting a batch
|
|
61
|
+
|
|
62
|
+
Pass a provider and one or more uniquely identified requests to `startBatch`.
|
|
63
|
+
Each request must specify `type: 'text'`, a model ID, and either a text `prompt`
|
|
64
|
+
or a `messages` array. Requests can use different model IDs when supported by
|
|
65
|
+
the provider. Each request can also use the usual text-generation settings such
|
|
66
|
+
as `instructions`, `maxOutputTokens`, `temperature`, `topP`, `topK`,
|
|
67
|
+
`presencePenalty`, `frequencyPenalty`, `stopSequences`, `seed`, and `reasoning`.
|
|
68
|
+
|
|
69
|
+
Tool definitions and tool settings are provided on individual requests. A tool
|
|
70
|
+
with the same name must have the same definition in every request that uses it.
|
|
71
|
+
|
|
72
|
+
```ts
|
|
73
|
+
import { anthropic } from '@ai-sdk/anthropic';
|
|
74
|
+
import { experimental_startBatch as startBatch } from 'ai';
|
|
75
|
+
|
|
76
|
+
const provider = anthropic;
|
|
77
|
+
|
|
78
|
+
const batch = await startBatch({
|
|
79
|
+
provider,
|
|
80
|
+
requests: [
|
|
81
|
+
{
|
|
82
|
+
id: 'capital-france',
|
|
83
|
+
type: 'text',
|
|
84
|
+
model: 'claude-haiku-4-5',
|
|
85
|
+
prompt: 'What is the capital of France?',
|
|
86
|
+
},
|
|
87
|
+
{
|
|
88
|
+
id: 'capital-germany',
|
|
89
|
+
type: 'text',
|
|
90
|
+
model: 'claude-haiku-4-5',
|
|
91
|
+
prompt: 'What is the capital of Germany?',
|
|
92
|
+
},
|
|
93
|
+
],
|
|
94
|
+
});
|
|
95
|
+
|
|
96
|
+
console.log(batch.id, batch.status);
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Request IDs must be non-empty and unique within the batch. They are the link
|
|
100
|
+
between an input request and its result. Results are not guaranteed to arrive
|
|
101
|
+
in input order, so use the ID rather than an array position when associating a
|
|
102
|
+
result with application data.
|
|
103
|
+
|
|
104
|
+
You can pass provider-specific settings at the batch level with
|
|
105
|
+
`providerOptions`, or on an individual request with that request's
|
|
106
|
+
`providerOptions`. Settings supported by the provider can differ between the
|
|
107
|
+
two levels. The `warnings` property on the start result contains warnings about
|
|
108
|
+
unsupported settings.
|
|
109
|
+
|
|
110
|
+
The batch reference returned by `startBatch` is serializable. Persist it
|
|
111
|
+
before the process exits if the batch will be completed by another process or
|
|
112
|
+
at a later time. When retrieving a batch, pass the same provider. The AI SDK
|
|
113
|
+
uses the reference to ensure that a batch is not read through an incompatible
|
|
114
|
+
provider.
|
|
115
|
+
|
|
116
|
+
## Batch tools
|
|
117
|
+
|
|
118
|
+
Batch text requests support both client-defined tools and provider-defined
|
|
119
|
+
tools. Tool definitions are sent with each request that includes them, so the
|
|
120
|
+
model can call them independently for each request.
|
|
121
|
+
|
|
122
|
+
Client-defined tools are definition-only in a batch. Their `execute` functions
|
|
123
|
+
are never invoked, and the AI SDK does not submit tool results or start a
|
|
124
|
+
follow-up generation after retrieving a tool call. Pass the same tool set to
|
|
125
|
+
`getBatchResults` to validate and normalize returned tool calls:
|
|
126
|
+
|
|
127
|
+
```ts
|
|
128
|
+
import { anthropic } from '@ai-sdk/anthropic';
|
|
129
|
+
import {
|
|
130
|
+
experimental_getBatchResults as getBatchResults,
|
|
131
|
+
experimental_startBatch as startBatch,
|
|
132
|
+
tool,
|
|
133
|
+
} from 'ai';
|
|
134
|
+
import { z } from 'zod';
|
|
135
|
+
|
|
136
|
+
const tools = {
|
|
137
|
+
get_weather: tool({
|
|
138
|
+
description: 'Get the current weather for a location.',
|
|
139
|
+
inputSchema: z.object({ location: z.string() }),
|
|
140
|
+
execute: async ({ location }) => {
|
|
141
|
+
// This function is not called by batch processing.
|
|
142
|
+
return { location, temperature: 21, condition: 'sunny' };
|
|
143
|
+
},
|
|
144
|
+
}),
|
|
145
|
+
};
|
|
146
|
+
|
|
147
|
+
const batch = await startBatch({
|
|
148
|
+
provider: anthropic,
|
|
149
|
+
requests: [
|
|
150
|
+
{
|
|
151
|
+
id: 'weather-san-francisco',
|
|
152
|
+
type: 'text',
|
|
153
|
+
model: 'claude-haiku-4-5',
|
|
154
|
+
prompt: 'Call get_weather for San Francisco, California.',
|
|
155
|
+
tools,
|
|
156
|
+
toolChoice: { type: 'tool', toolName: 'get_weather' },
|
|
157
|
+
},
|
|
158
|
+
],
|
|
159
|
+
});
|
|
160
|
+
|
|
161
|
+
for await (const item of getBatchResults({
|
|
162
|
+
provider: anthropic,
|
|
163
|
+
batch,
|
|
164
|
+
tools,
|
|
165
|
+
})) {
|
|
166
|
+
if (item.status === 'succeeded') {
|
|
167
|
+
console.log(item.id, item.content);
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
Provider-defined tools, such as web search or code execution, can execute on
|
|
173
|
+
the provider when supported by that provider's batch API. Their tool calls and
|
|
174
|
+
results are returned as normalized `content` parts. See the provider
|
|
175
|
+
documentation for the tools and models available in batches.
|
|
176
|
+
|
|
177
|
+
## Checking batch status
|
|
178
|
+
|
|
179
|
+
The start result includes the initial status. Use `getBatchStatus` to retrieve
|
|
180
|
+
the latest status while the provider is processing the batch:
|
|
181
|
+
|
|
182
|
+
```ts
|
|
183
|
+
import { setTimeout } from 'node:timers/promises';
|
|
184
|
+
|
|
185
|
+
let status = batch.status;
|
|
186
|
+
let error = batch.error;
|
|
187
|
+
|
|
188
|
+
while (status === 'pending') {
|
|
189
|
+
await setTimeout(10_000);
|
|
190
|
+
const latestStatus = await getBatchStatus({
|
|
191
|
+
provider: anthropic,
|
|
192
|
+
batch,
|
|
193
|
+
});
|
|
194
|
+
status = latestStatus.status;
|
|
195
|
+
error = latestStatus.error;
|
|
196
|
+
}
|
|
197
|
+
|
|
198
|
+
if (status === 'failed') {
|
|
199
|
+
throw new Error(error?.message ?? 'The batch failed.');
|
|
200
|
+
}
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
The normalized batch status is one of:
|
|
204
|
+
|
|
205
|
+
- `pending` — the provider is still processing the batch.
|
|
206
|
+
- `completed` — the batch reached a terminal state and results can be
|
|
207
|
+
retrieved.
|
|
208
|
+
- `failed` — the batch could not be completed. The `error` property may
|
|
209
|
+
contain additional details.
|
|
210
|
+
|
|
211
|
+
Status responses can also include `requestCounts`, `createdAt`, `expiresAt`,
|
|
212
|
+
`rawStatus`, and provider metadata. `requestCounts` reports the total,
|
|
213
|
+
pending, completed, and failed requests known by the provider.
|
|
214
|
+
|
|
215
|
+
For webhook-capable providers, pass `webhookUrl` to `startBatch` to
|
|
216
|
+
receive a notification when the batch reaches a terminal state. Webhook
|
|
217
|
+
support and payloads are provider-specific. The provider pages linked above
|
|
218
|
+
describe their webhook behavior; providers that do not support webhooks return
|
|
219
|
+
an unsupported warning and continue without one.
|
|
220
|
+
|
|
221
|
+
## Retrieving results
|
|
222
|
+
|
|
223
|
+
After the batch is complete, `getBatchResults` returns an async iterable. Each
|
|
224
|
+
item contains the ID and terminal status for one input request:
|
|
225
|
+
|
|
226
|
+
```ts
|
|
227
|
+
for await (const item of getBatchResults({ provider: anthropic, batch })) {
|
|
228
|
+
if (item.status === 'succeeded') {
|
|
229
|
+
console.log(item.id, item.text);
|
|
230
|
+
} else {
|
|
231
|
+
console.error(item.id, item.status, item.error);
|
|
232
|
+
}
|
|
233
|
+
}
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
Successful items include:
|
|
237
|
+
|
|
238
|
+
- `text` — the concatenated text content. This can be an empty string when the
|
|
239
|
+
result contains no text parts.
|
|
240
|
+
- `content` — ordered, normalized content parts, including text, reasoning,
|
|
241
|
+
files, sources, tool calls, tool results, and provider content where
|
|
242
|
+
supported.
|
|
243
|
+
- `finishReason` and optional `rawFinishReason`.
|
|
244
|
+
- `usage` and optional `response` metadata.
|
|
245
|
+
- Optional `providerMetadata`.
|
|
246
|
+
|
|
247
|
+
Failed, cancelled, and expired items include their `id` and status. Failed
|
|
248
|
+
items include an `error`; cancelled and expired items may also include one.
|
|
249
|
+
One failed request does not necessarily mean that every request in the batch
|
|
250
|
+
failed, so handle each item independently.
|
|
251
|
+
|
|
252
|
+
Batch retrieval does not run an AI SDK tool loop or invoke client-defined
|
|
253
|
+
`execute` functions. Provider-defined tools may execute on the provider as part
|
|
254
|
+
of batch processing. Treat result content and provider metadata as untrusted
|
|
255
|
+
model output, and avoid logging it indiscriminately because it can contain
|
|
256
|
+
sensitive data.
|
|
257
|
+
|
|
258
|
+
## Request controls
|
|
259
|
+
|
|
260
|
+
`getBatchStatus` and `getBatchResults` accept `providerOptions`, `headers`,
|
|
261
|
+
`timeout`, `abortSignal`, and `maxRetries` for the current status or result
|
|
262
|
+
retrieval operation:
|
|
263
|
+
|
|
264
|
+
- `maxRetries` controls retries for status and result retrieval. It does not
|
|
265
|
+
retry batch creation, which could create a duplicate batch. It defaults to 2;
|
|
266
|
+
set it to 0 to disable retries.
|
|
267
|
+
- `abortSignal` cancels the current status or result request.
|
|
268
|
+
- `timeout` limits the current HTTP operation.
|
|
269
|
+
|
|
270
|
+
These controls affect communication with the provider. They do not change the
|
|
271
|
+
provider's processing deadline or cancel a batch that has already been
|
|
272
|
+
submitted.
|
|
@@ -98,6 +98,11 @@ description: Learn about AI SDK Core.
|
|
|
98
98
|
description: 'Learn how to generate speech with AI SDK Core.',
|
|
99
99
|
href: '/docs/ai-sdk-core/speech',
|
|
100
100
|
},
|
|
101
|
+
{
|
|
102
|
+
title: 'Batch',
|
|
103
|
+
description: 'Submit requests for asynchronous batch processing.',
|
|
104
|
+
href: '/docs/ai-sdk-core/batch',
|
|
105
|
+
},
|
|
101
106
|
{
|
|
102
107
|
title: 'File Uploads',
|
|
103
108
|
description: 'Learn how to upload files to providers with the AI SDK.',
|
|
@@ -20,6 +20,7 @@ The AI SDK includes the following harness adapters:
|
|
|
20
20
|
- [Cursor](/providers/ai-sdk-harnesses/cursor) (`@ai-sdk/harness-cursor`)
|
|
21
21
|
- [Deep Agents](/providers/ai-sdk-harnesses/deepagents) (`@ai-sdk/harness-deepagents`)
|
|
22
22
|
- [fx](/providers/ai-sdk-harnesses/fx) (`@ai-sdk/harness-fx`)
|
|
23
|
+
- [GitHub Copilot](/providers/ai-sdk-harnesses/github-copilot) (`@ai-sdk/harness-github-copilot`)
|
|
23
24
|
- [Grok Build](/providers/ai-sdk-harnesses/grok-build) (`@ai-sdk/harness-grok-build`)
|
|
24
25
|
- [OpenCode](/providers/ai-sdk-harnesses/opencode) (`@ai-sdk/harness-opencode`)
|
|
25
26
|
- [Pi](/providers/ai-sdk-harnesses/pi) (`@ai-sdk/harness-pi`)
|
|
@@ -32,14 +33,15 @@ The AI SDK includes the following harness adapters:
|
|
|
32
33
|
|
|
33
34
|
## Adapter Capabilities
|
|
34
35
|
|
|
35
|
-
| Adapter
|
|
36
|
-
|
|
|
37
|
-
| [Claude Code](/providers/ai-sdk-harnesses/claude-code)
|
|
38
|
-
| [Cline](/providers/ai-sdk-harnesses/cline)
|
|
39
|
-
| [Codex](/providers/ai-sdk-harnesses/codex)
|
|
40
|
-
| [Cursor](/providers/ai-sdk-harnesses/cursor)
|
|
41
|
-
| [Deep Agents](/providers/ai-sdk-harnesses/deepagents)
|
|
42
|
-
| [fx](/providers/ai-sdk-harnesses/fx)
|
|
43
|
-
| [
|
|
44
|
-
| [
|
|
45
|
-
| [
|
|
36
|
+
| Adapter | Runtime location | Custom tools | Custom skills | Structured output | Built-in tool approval | Built-in tool filtering |
|
|
37
|
+
| ------------------------------------------------------------ | ---------------- | ------------ | ------------- | ----------------- | ---------------------- | ---------------------------- |
|
|
38
|
+
| [Claude Code](/providers/ai-sdk-harnesses/claude-code) | Sandbox bridge | <Check /> | <Check /> | <Check /> | <Check /> | <Check /> |
|
|
39
|
+
| [Cline](/providers/ai-sdk-harnesses/cline) | Host process | <Check /> | <Check /> | <Check /> | <Check /> | <Check /> |
|
|
40
|
+
| [Codex](/providers/ai-sdk-harnesses/codex) | Sandbox bridge | <Check /> | <Check /> | <Check /> | <Cross /> | <Cross /> |
|
|
41
|
+
| [Cursor](/providers/ai-sdk-harnesses/cursor) | Sandbox via ACP | <Check /> | <Check /> | <Cross /> | <Check /> | <Cross /> |
|
|
42
|
+
| [Deep Agents](/providers/ai-sdk-harnesses/deepagents) | Sandbox bridge | <Check /> | <Check /> | <Check /> | <Check /> | <Check /> via auto-rejection |
|
|
43
|
+
| [fx](/providers/ai-sdk-harnesses/fx) | Sandbox via ACP | <Check /> | <Check /> | <Cross /> | <Check /> | <Cross /> |
|
|
44
|
+
| [GitHub Copilot](/providers/ai-sdk-harnesses/github-copilot) | Sandbox via ACP | <Check /> | <Check /> | <Cross /> | <Check /> | <Cross /> |
|
|
45
|
+
| [Grok Build](/providers/ai-sdk-harnesses/grok-build) | Sandbox via ACP | <Check /> | <Check /> | <Check /> | <Check /> | <Cross /> |
|
|
46
|
+
| [OpenCode](/providers/ai-sdk-harnesses/opencode) | Sandbox bridge | <Check /> | <Check /> | <Check /> | <Check /> | <Check /> via auto-rejection |
|
|
47
|
+
| [Pi](/providers/ai-sdk-harnesses/pi) | Host process | <Check /> | <Check /> | <Cross /> | <Check /> | <Check /> |
|
|
@@ -110,6 +110,13 @@ export const cacheMiddleware: LanguageModelV4Middleware = {
|
|
|
110
110
|
responses but you can use any KV storage provider you would like.
|
|
111
111
|
</Note>
|
|
112
112
|
|
|
113
|
+
<Note>
|
|
114
|
+
This middleware caches the raw model response before AI SDK validates
|
|
115
|
+
structured output. When using structured output, cache only a response that
|
|
116
|
+
has passed your schema validation; otherwise, an invalid response can be
|
|
117
|
+
replayed from the cache on later requests.
|
|
118
|
+
</Note>
|
|
119
|
+
|
|
113
120
|
`LanguageModelV4Middleware` has two methods: `wrapGenerate` and `wrapStream`. `wrapGenerate` is called when using [`generateText`](/docs/reference/ai-sdk-core/generate-text), while `wrapStream` is called when using [`streamText`](/docs/reference/ai-sdk-core/stream-text).
|
|
114
121
|
|
|
115
122
|
For `wrapGenerate`, you can cache the response directly. Instead, for `wrapStream`, you cache an array of the stream parts, which can then be used with [`simulateReadableStream`](/docs/ai-sdk-core/testing#simulate-ui-message-stream-responses) function to create a simulated `ReadableStream` that returns the cached response. In this way, the cached response is returned chunk-by-chunk as if it were being generated by the model. You can control the initial delay and delay between chunks by adjusting the `initialDelayInMs` and `chunkDelayInMs` parameters of `simulateReadableStream`.
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: experimental_startBatch
|
|
3
|
+
description: API Reference for experimental_startBatch.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# `experimental_startBatch()`
|
|
7
|
+
|
|
8
|
+
<Note type="warning">
|
|
9
|
+
Batch support is experimental and the API may change in patch releases.
|
|
10
|
+
</Note>
|
|
11
|
+
|
|
12
|
+
Starts an asynchronous batch. Currently, only `type: 'text'` requests are
|
|
13
|
+
supported. For a complete guide to the batch lifecycle, see
|
|
14
|
+
[Batch](/docs/ai-sdk-core/batch).
|
|
15
|
+
|
|
16
|
+
```ts
|
|
17
|
+
import { experimental_startBatch as startBatch } from 'ai';
|
|
18
|
+
|
|
19
|
+
const batch = await startBatch({
|
|
20
|
+
requests: [
|
|
21
|
+
{
|
|
22
|
+
id: 'france',
|
|
23
|
+
type: 'text',
|
|
24
|
+
model: 'gpt-4.1-nano',
|
|
25
|
+
prompt: 'What is the capital of France?',
|
|
26
|
+
},
|
|
27
|
+
],
|
|
28
|
+
});
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## Import
|
|
32
|
+
|
|
33
|
+
<Snippet text={`import { experimental_startBatch } from "ai"`} prompt={false} />
|
|
34
|
+
|
|
35
|
+
## API Signature
|
|
36
|
+
|
|
37
|
+
### Parameters
|
|
38
|
+
|
|
39
|
+
<PropertiesTable
|
|
40
|
+
content={[
|
|
41
|
+
{
|
|
42
|
+
name: 'provider',
|
|
43
|
+
type: 'Experimental_BatchProvider',
|
|
44
|
+
isOptional: true,
|
|
45
|
+
description:
|
|
46
|
+
'The provider to use. Defaults to the global provider, or the AI Gateway when no global provider is configured.',
|
|
47
|
+
},
|
|
48
|
+
{
|
|
49
|
+
name: 'requests',
|
|
50
|
+
type: 'Array<Experimental_BatchRequest>',
|
|
51
|
+
description:
|
|
52
|
+
'The independent requests to process. Each request must have a unique, non-empty id, a supported type, and type-specific properties such as model and prompt or messages for text requests.',
|
|
53
|
+
},
|
|
54
|
+
{
|
|
55
|
+
name: 'providerOptions',
|
|
56
|
+
type: 'ProviderOptions',
|
|
57
|
+
isOptional: true,
|
|
58
|
+
description: 'Additional provider-specific options for the batch.',
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
name: 'webhookUrl',
|
|
62
|
+
type: 'string',
|
|
63
|
+
isOptional: true,
|
|
64
|
+
description:
|
|
65
|
+
'URL to notify when the batch reaches a terminal state. Support and payloads are provider-specific.',
|
|
66
|
+
},
|
|
67
|
+
{
|
|
68
|
+
name: 'abortSignal',
|
|
69
|
+
type: 'AbortSignal',
|
|
70
|
+
isOptional: true,
|
|
71
|
+
description:
|
|
72
|
+
'An optional abort signal to cancel the request that starts the batch.',
|
|
73
|
+
},
|
|
74
|
+
{
|
|
75
|
+
name: 'timeout',
|
|
76
|
+
type: 'number | { totalMs?: number }',
|
|
77
|
+
isOptional: true,
|
|
78
|
+
description: 'Maximum time allowed for the batch creation request.',
|
|
79
|
+
},
|
|
80
|
+
{
|
|
81
|
+
name: 'headers',
|
|
82
|
+
type: 'Record<string, string | undefined>',
|
|
83
|
+
isOptional: true,
|
|
84
|
+
description: 'Additional HTTP headers for the request.',
|
|
85
|
+
},
|
|
86
|
+
]}
|
|
87
|
+
/>
|
|
88
|
+
|
|
89
|
+
### Returns
|
|
90
|
+
|
|
91
|
+
<PropertiesTable
|
|
92
|
+
content={[
|
|
93
|
+
{
|
|
94
|
+
name: 'version',
|
|
95
|
+
type: '2',
|
|
96
|
+
description: 'Version of the serializable batch reference.',
|
|
97
|
+
},
|
|
98
|
+
{
|
|
99
|
+
name: 'id',
|
|
100
|
+
type: 'string',
|
|
101
|
+
description: 'Provider batch identifier.',
|
|
102
|
+
},
|
|
103
|
+
{
|
|
104
|
+
name: 'provider',
|
|
105
|
+
type: 'string',
|
|
106
|
+
description: 'Provider identifier for the batch.',
|
|
107
|
+
},
|
|
108
|
+
{
|
|
109
|
+
name: 'status',
|
|
110
|
+
type: "'pending' | 'completed' | 'failed'",
|
|
111
|
+
description: 'Initial normalized batch status.',
|
|
112
|
+
},
|
|
113
|
+
{
|
|
114
|
+
name: 'rawStatus',
|
|
115
|
+
type: 'string',
|
|
116
|
+
isOptional: true,
|
|
117
|
+
description: 'The provider-specific batch status, when available.',
|
|
118
|
+
},
|
|
119
|
+
{
|
|
120
|
+
name: 'requestCounts',
|
|
121
|
+
type: '{ total: number; pending: number; completed: number; failed: number }',
|
|
122
|
+
isOptional: true,
|
|
123
|
+
description: 'Provider-reported request counts.',
|
|
124
|
+
},
|
|
125
|
+
{
|
|
126
|
+
name: 'error',
|
|
127
|
+
type: 'Experimental_BatchError',
|
|
128
|
+
isOptional: true,
|
|
129
|
+
description: 'Error details when the batch fails.',
|
|
130
|
+
},
|
|
131
|
+
{
|
|
132
|
+
name: 'createdAt',
|
|
133
|
+
type: 'string',
|
|
134
|
+
isOptional: true,
|
|
135
|
+
description: 'Creation timestamp, when provided by the provider.',
|
|
136
|
+
},
|
|
137
|
+
{
|
|
138
|
+
name: 'expiresAt',
|
|
139
|
+
type: 'string',
|
|
140
|
+
isOptional: true,
|
|
141
|
+
description: 'Expiration timestamp, when provided by the provider.',
|
|
142
|
+
},
|
|
143
|
+
{
|
|
144
|
+
name: 'providerMetadata',
|
|
145
|
+
type: 'ProviderMetadata',
|
|
146
|
+
isOptional: true,
|
|
147
|
+
description: 'Provider-specific metadata for the batch.',
|
|
148
|
+
},
|
|
149
|
+
{
|
|
150
|
+
name: 'warnings',
|
|
151
|
+
type: 'Warning[]',
|
|
152
|
+
description: 'Warnings returned while starting the batch.',
|
|
153
|
+
},
|
|
154
|
+
]}
|
|
155
|
+
/>
|
|
@@ -0,0 +1,133 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: experimental_getBatchStatus
|
|
3
|
+
description: API Reference for experimental_getBatchStatus.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# `experimental_getBatchStatus()`
|
|
7
|
+
|
|
8
|
+
<Note type="warning">
|
|
9
|
+
Batch support is experimental and the API may change in patch releases.
|
|
10
|
+
</Note>
|
|
11
|
+
|
|
12
|
+
Retrieves the latest status of an asynchronous batch. For a complete guide to
|
|
13
|
+
the batch lifecycle, see [Batch](/docs/ai-sdk-core/batch).
|
|
14
|
+
|
|
15
|
+
```ts
|
|
16
|
+
import { anthropic } from '@ai-sdk/anthropic';
|
|
17
|
+
import { experimental_getBatchStatus as getBatchStatus } from 'ai';
|
|
18
|
+
|
|
19
|
+
const status = await getBatchStatus({
|
|
20
|
+
provider: anthropic,
|
|
21
|
+
batch,
|
|
22
|
+
});
|
|
23
|
+
|
|
24
|
+
console.log(status.status, status.requestCounts);
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## Import
|
|
28
|
+
|
|
29
|
+
<Snippet
|
|
30
|
+
text={`import { experimental_getBatchStatus } from "ai"`}
|
|
31
|
+
prompt={false}
|
|
32
|
+
/>
|
|
33
|
+
|
|
34
|
+
## API Signature
|
|
35
|
+
|
|
36
|
+
### Parameters
|
|
37
|
+
|
|
38
|
+
<PropertiesTable
|
|
39
|
+
content={[
|
|
40
|
+
{
|
|
41
|
+
name: 'provider',
|
|
42
|
+
type: 'Experimental_BatchProvider',
|
|
43
|
+
isOptional: true,
|
|
44
|
+
description:
|
|
45
|
+
'The provider used to access the batch. Defaults to the global provider, or the AI Gateway when no global provider is configured.',
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
name: 'batch',
|
|
49
|
+
type: 'Experimental_BatchReference',
|
|
50
|
+
description:
|
|
51
|
+
'The serializable reference returned by experimental_startBatch.',
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
name: 'providerOptions',
|
|
55
|
+
type: 'ProviderOptions',
|
|
56
|
+
isOptional: true,
|
|
57
|
+
description: 'Additional provider-specific options for status retrieval.',
|
|
58
|
+
},
|
|
59
|
+
{
|
|
60
|
+
name: 'maxRetries',
|
|
61
|
+
type: 'number',
|
|
62
|
+
isOptional: true,
|
|
63
|
+
description:
|
|
64
|
+
'Maximum number of retries for status retrieval. Set to 0 to disable retries. Default: 2.',
|
|
65
|
+
},
|
|
66
|
+
{
|
|
67
|
+
name: 'abortSignal',
|
|
68
|
+
type: 'AbortSignal',
|
|
69
|
+
isOptional: true,
|
|
70
|
+
description: 'An optional abort signal to cancel the request.',
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
name: 'timeout',
|
|
74
|
+
type: 'number | { totalMs?: number }',
|
|
75
|
+
isOptional: true,
|
|
76
|
+
description: 'Maximum time allowed for the status request.',
|
|
77
|
+
},
|
|
78
|
+
{
|
|
79
|
+
name: 'headers',
|
|
80
|
+
type: 'Record<string, string | undefined>',
|
|
81
|
+
isOptional: true,
|
|
82
|
+
description: 'Additional HTTP headers for the request.',
|
|
83
|
+
},
|
|
84
|
+
]}
|
|
85
|
+
/>
|
|
86
|
+
|
|
87
|
+
### Returns
|
|
88
|
+
|
|
89
|
+
<PropertiesTable
|
|
90
|
+
content={[
|
|
91
|
+
{
|
|
92
|
+
name: 'status',
|
|
93
|
+
type: "'pending' | 'completed' | 'failed'",
|
|
94
|
+
description: 'The latest normalized batch status.',
|
|
95
|
+
},
|
|
96
|
+
{
|
|
97
|
+
name: 'rawStatus',
|
|
98
|
+
type: 'string',
|
|
99
|
+
isOptional: true,
|
|
100
|
+
description: 'The provider-specific batch status, when available.',
|
|
101
|
+
},
|
|
102
|
+
{
|
|
103
|
+
name: 'requestCounts',
|
|
104
|
+
type: '{ total: number; pending: number; completed: number; failed: number }',
|
|
105
|
+
isOptional: true,
|
|
106
|
+
description: 'Provider-reported request counts.',
|
|
107
|
+
},
|
|
108
|
+
{
|
|
109
|
+
name: 'error',
|
|
110
|
+
type: 'Experimental_BatchError',
|
|
111
|
+
isOptional: true,
|
|
112
|
+
description: 'Error details when the batch fails.',
|
|
113
|
+
},
|
|
114
|
+
{
|
|
115
|
+
name: 'createdAt',
|
|
116
|
+
type: 'string',
|
|
117
|
+
isOptional: true,
|
|
118
|
+
description: 'Creation timestamp, when provided by the provider.',
|
|
119
|
+
},
|
|
120
|
+
{
|
|
121
|
+
name: 'expiresAt',
|
|
122
|
+
type: 'string',
|
|
123
|
+
isOptional: true,
|
|
124
|
+
description: 'Expiration timestamp, when provided by the provider.',
|
|
125
|
+
},
|
|
126
|
+
{
|
|
127
|
+
name: 'providerMetadata',
|
|
128
|
+
type: 'ProviderMetadata',
|
|
129
|
+
isOptional: true,
|
|
130
|
+
description: 'Provider-specific metadata for the batch.',
|
|
131
|
+
},
|
|
132
|
+
]}
|
|
133
|
+
/>
|