ai 7.0.126 → 7.0.128
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +29 -0
- package/dist/index.d.ts +234 -154
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +307 -199
- package/dist/index.js.map +1 -1
- package/dist/internal/index.d.ts +278 -68
- package/dist/internal/index.d.ts.map +1 -1
- package/dist/internal/index.js +371 -72
- package/dist/internal/index.js.map +1 -1
- package/dist/test/index.d.ts +25 -17
- package/dist/test/index.d.ts.map +1 -1
- package/dist/test/index.js +27 -21
- package/dist/test/index.js.map +1 -1
- package/docs/02-getting-started/03-nextjs-pages-router.mdx +1 -1
- package/docs/02-getting-started/07-expo.mdx +1 -1
- package/docs/03-agents/07-workflow-agent.mdx +80 -12
- package/docs/03-ai-sdk-core/16-mcp-tools.mdx +44 -0
- package/docs/03-ai-sdk-core/19-tool-search.mdx +40 -4
- package/docs/03-ai-sdk-core/{32-evaluation.mdx → 32-decisions.mdx} +49 -49
- package/docs/03-ai-sdk-core/35-image-generation.mdx +6 -1
- package/docs/03-ai-sdk-core/36-transcription.mdx +42 -39
- package/docs/03-ai-sdk-core/37-speech.mdx +2 -0
- package/docs/03-ai-sdk-core/45-provider-management.mdx +8 -8
- package/docs/03-ai-sdk-core/60-telemetry.mdx +40 -31
- package/docs/03-ai-sdk-core/65-lifecycle-callbacks.mdx +21 -22
- package/docs/03-ai-sdk-harnesses/08-terminal-ui.mdx +2 -2
- package/docs/04-ai-sdk-ui/02-chatbot.mdx +2 -1
- package/docs/06-advanced/09-multistep-interfaces.mdx +1 -1
- package/docs/07-reference/01-ai-sdk-core/14-decide.mdx +74 -0
- package/docs/07-reference/01-ai-sdk-core/23-create-mcp-client.mdx +58 -4
- package/docs/07-reference/01-ai-sdk-core/23-tool-search.mdx +62 -6
- package/docs/07-reference/01-ai-sdk-core/40-provider-registry.mdx +11 -11
- package/docs/07-reference/01-ai-sdk-core/42-custom-provider.mdx +9 -9
- package/docs/07-reference/01-ai-sdk-core/66-extract-reasoning-middleware.mdx +24 -8
- package/docs/07-reference/02-ai-sdk-ui/01-use-chat.mdx +13 -0
- package/docs/07-reference/02-ai-sdk-ui/32-prune-messages.mdx +1 -1
- package/docs/07-reference/03-ai-sdk-rsc/01-stream-ui.mdx +1 -0
- package/docs/07-reference/04-ai-sdk-workflow/01-workflow-agent.mdx +39 -2
- package/docs/07-reference/05-ai-sdk-errors/ai-decision-unsupported-question-type-error.mdx +31 -0
- package/docs/07-reference/05-ai-sdk-errors/ai-no-such-model-error.mdx +3 -3
- package/docs/07-reference/05-ai-sdk-errors/ai-no-such-provider-error.mdx +3 -3
- package/docs/08-migration-guides/23-migration-guide-7-0.mdx +1 -1
- package/docs/08-migration-guides/29-migration-guide-4-0.mdx +1 -1
- package/docs/09-troubleshooting/05-tool-invocation-missing-result.mdx +2 -2
- package/package.json +12 -12
- package/src/decide/decide-events.ts +129 -0
- package/src/{evaluate/evaluate.ts → decide/decide.ts} +36 -38
- package/src/decide/decision-provider.ts +11 -0
- package/src/decide/decision-result.ts +39 -0
- package/src/decide/index.ts +64 -0
- package/src/{evaluate → decide}/restricted-telemetry-dispatcher.ts +7 -7
- package/src/{evaluate/validate-evaluation.ts → decide/validate-decision.ts} +9 -9
- package/src/error/index.ts +10 -1
- package/src/generate-speech/generated-audio-file.ts +5 -4
- package/src/generate-text/default-generate-text-result.ts +127 -0
- package/src/generate-text/generate-text.ts +2 -122
- package/src/generate-text/validate-tool-approvals.ts +5 -1
- package/src/global.ts +1 -1
- package/src/index.ts +1 -1
- package/src/middleware/extract-reasoning-middleware.ts +25 -6
- package/src/model/resolve-model.ts +45 -14
- package/src/registry/custom-provider.ts +38 -26
- package/src/registry/index.ts +13 -1
- package/src/registry/no-such-provider-error.ts +2 -0
- package/src/registry/provider-registry.ts +56 -29
- package/src/telemetry/create-telemetry-dispatcher.ts +17 -5
- package/src/telemetry/telemetry.ts +35 -19
- package/src/telemetry/tracing-channel.ts +1 -1
- package/src/test/decision-mock-model-v4.ts +35 -0
- package/src/tool-search/prepare-tool-search.ts +49 -29
- package/src/tool-search/tool-search.ts +58 -4
- package/src/transcribe/transcribe.ts +1 -1
- package/src/ui/chat.ts +9 -1
- package/src/ui/process-ui-message-stream.ts +21 -0
- package/src/ui-message-stream/create-ui-message-stream.ts +29 -4
- package/docs/07-reference/01-ai-sdk-core/14-evaluate.mdx +0 -74
- package/docs/07-reference/05-ai-sdk-errors/ai-evaluation-unsupported-question-type-error.mdx +0 -31
- package/src/evaluate/evaluate-events.ts +0 -131
- package/src/evaluate/evaluation-provider.ts +0 -6
- package/src/evaluate/evaluation-result.ts +0 -39
- package/src/evaluate/index.ts +0 -13
- package/src/test/evaluation-mock-model-v4.ts +0 -27
|
@@ -1,24 +1,24 @@
|
|
|
1
1
|
---
|
|
2
|
-
title:
|
|
3
|
-
description:
|
|
2
|
+
title: Decisions
|
|
3
|
+
description: Decide answers to Choice, Score, and Boolean questions against shared state.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
#
|
|
6
|
+
# Decisions
|
|
7
7
|
|
|
8
|
-
`
|
|
9
|
-
|
|
8
|
+
`experimental_decide` decides answers to named questions against one shared state using
|
|
9
|
+
a decision model. State can be a string, JSON object, or JSON array. An array
|
|
10
10
|
is one state, not a batch of unrelated inputs.
|
|
11
11
|
|
|
12
|
-
This API and the
|
|
13
|
-
in patch releases. Pass
|
|
12
|
+
This API and the decision model specification are experimental and may change
|
|
13
|
+
in patch releases. Pass a decision model instance, resolve a model through a
|
|
14
14
|
provider registry, or use a string ID. Strings resolve through Vercel AI Gateway
|
|
15
|
-
unless you configure
|
|
15
|
+
unless you configure a decision-capable default provider.
|
|
16
16
|
|
|
17
17
|
```ts
|
|
18
|
-
import {
|
|
18
|
+
import { experimental_decide, type Experimental_DecisionModel } from 'ai';
|
|
19
19
|
|
|
20
|
-
async function triage(model:
|
|
21
|
-
return
|
|
20
|
+
async function triage(model: Experimental_DecisionModel, message: string) {
|
|
21
|
+
return experimental_decide({
|
|
22
22
|
model,
|
|
23
23
|
state: { message },
|
|
24
24
|
questions: {
|
|
@@ -46,16 +46,16 @@ async function triage(model: Experimental_EvaluationModel, message: string) {
|
|
|
46
46
|
|
|
47
47
|
## Provider models
|
|
48
48
|
|
|
49
|
-
Use a provider's `
|
|
49
|
+
Use a provider's `decisionModel` factory:
|
|
50
50
|
|
|
51
|
-
| Provider | Example model
|
|
52
|
-
| ----------- |
|
|
53
|
-
| TypeSafe AI | `typeSafeAi.
|
|
54
|
-
| OpenAI | `openai.
|
|
55
|
-
| Anthropic | `anthropic.
|
|
56
|
-
| Google | `google.
|
|
51
|
+
| Provider | Example model |
|
|
52
|
+
| ----------- | ------------------------------------------------------ |
|
|
53
|
+
| TypeSafe AI | `typeSafeAi.decisionModel('jev-latest')` |
|
|
54
|
+
| OpenAI | `openai.decisionModel('gpt-5.6-luna')` |
|
|
55
|
+
| Anthropic | `anthropic.decisionModel('claude-haiku-4-5-20251001')` |
|
|
56
|
+
| Google | `google.decisionModel('gemini-3.5-flash-lite')` |
|
|
57
57
|
|
|
58
|
-
TypeSafe AI's [Jev](https://vercel.com/i/what-is-jev) supplies native Choice, Score, and Boolean
|
|
58
|
+
TypeSafe AI's [Jev](https://vercel.com/i/what-is-jev) supplies native Choice, Score, and Boolean decisions. OpenAI,
|
|
59
59
|
Anthropic, and Google adapt structured language-model output for all three types.
|
|
60
60
|
Boolean answers contain prompted estimates of P(true), validated to be finite
|
|
61
61
|
and in `[0, 1]`. These estimates are not guaranteed to be calibrated. Choice and
|
|
@@ -63,14 +63,14 @@ Score answers do not include probability distributions. Select a model that
|
|
|
63
63
|
supports the provider's structured-output API; judge quality against your own
|
|
64
64
|
labeled examples before choosing a model for a task.
|
|
65
65
|
|
|
66
|
-
The language-model adapters
|
|
66
|
+
The language-model adapters decide all answers in one prompt. They do not
|
|
67
67
|
provide TypeSafe's native independent-question execution semantics. The example
|
|
68
68
|
model IDs demonstrate API compatibility; they are not benchmark-selected defaults.
|
|
69
69
|
|
|
70
70
|
Language-model adapters request `reasoning: 'none'` by default. Each provider
|
|
71
71
|
maps this setting to its model's reasoning controls; it does not guarantee that
|
|
72
72
|
every model runs without thinking. To enable reasoning for more demanding
|
|
73
|
-
|
|
73
|
+
decisions, pass the model's supported reasoning settings through
|
|
74
74
|
`providerOptions`, which take precedence over this default. For example, use
|
|
75
75
|
`providerOptions: { openai: { reasoningEffort: 'high' } }` with an OpenAI model
|
|
76
76
|
that supports that effort.
|
|
@@ -86,22 +86,22 @@ import { openai } from '@ai-sdk/openai';
|
|
|
86
86
|
import {
|
|
87
87
|
customProvider,
|
|
88
88
|
createProviderRegistry,
|
|
89
|
-
|
|
89
|
+
experimental_decide,
|
|
90
90
|
} from 'ai';
|
|
91
91
|
|
|
92
92
|
const registry = createProviderRegistry({
|
|
93
93
|
triage: customProvider({
|
|
94
|
-
|
|
95
|
-
native: typeSafeAi.
|
|
96
|
-
compact: openai.
|
|
94
|
+
decisionModels: {
|
|
95
|
+
native: typeSafeAi.decisionModel('jev-latest'),
|
|
96
|
+
compact: openai.decisionModel('gpt-6-luna'),
|
|
97
97
|
},
|
|
98
98
|
fallbackProvider: typeSafeAi,
|
|
99
99
|
}),
|
|
100
100
|
openai,
|
|
101
101
|
});
|
|
102
102
|
|
|
103
|
-
const result = await
|
|
104
|
-
model: registry.
|
|
103
|
+
const result = await experimental_decide({
|
|
104
|
+
model: registry.decisionModel('triage:native'),
|
|
105
105
|
state: 'I was charged twice.',
|
|
106
106
|
questions: {
|
|
107
107
|
department: {
|
|
@@ -118,10 +118,10 @@ result.answers.department.choice; // 'billing' | 'support'
|
|
|
118
118
|
Registry IDs use `providerId:modelId`; the `separator` option changes the
|
|
119
119
|
separator. Only the first separator is used, so model IDs can contain it.
|
|
120
120
|
Custom aliases take precedence over a fallback provider. A fallback only resolves
|
|
121
|
-
unknown model IDs; it does not retry failed
|
|
121
|
+
unknown model IDs; it does not retry failed decisions or substitute a model
|
|
122
122
|
when a question type is unsupported. Registered providers keep their own
|
|
123
123
|
credentials and settings. Registry language/image middleware does not wrap
|
|
124
|
-
|
|
124
|
+
decision models.
|
|
125
125
|
|
|
126
126
|
### Default-provider strings
|
|
127
127
|
|
|
@@ -129,9 +129,9 @@ String IDs use Vercel AI Gateway by default. Configure Gateway authentication
|
|
|
129
129
|
with `AI_GATEWAY_API_KEY` or Vercel OIDC, then pass a Gateway model ID:
|
|
130
130
|
|
|
131
131
|
```ts
|
|
132
|
-
import {
|
|
132
|
+
import { experimental_decide } from 'ai';
|
|
133
133
|
|
|
134
|
-
const result = await
|
|
134
|
+
const result = await experimental_decide({
|
|
135
135
|
model: 'typesafe-ai/jev',
|
|
136
136
|
state: 'I was charged twice. Please refund the extra charge.',
|
|
137
137
|
questions: {
|
|
@@ -149,10 +149,10 @@ provider once at application startup:
|
|
|
149
149
|
```ts
|
|
150
150
|
// Using the registry above, expose an alias through a custom provider:
|
|
151
151
|
globalThis.AI_SDK_DEFAULT_PROVIDER = customProvider({
|
|
152
|
-
|
|
152
|
+
decisionModels: { native: registry.decisionModel('triage:native') },
|
|
153
153
|
});
|
|
154
154
|
|
|
155
|
-
const result = await
|
|
155
|
+
const result = await experimental_decide({
|
|
156
156
|
model: 'native',
|
|
157
157
|
state: 'I was charged twice.',
|
|
158
158
|
questions: {
|
|
@@ -165,20 +165,20 @@ const result = await experimental_evaluate({
|
|
|
165
165
|
```
|
|
166
166
|
|
|
167
167
|
A direct provider such as `typeSafeAi` can also be the default; then use its
|
|
168
|
-
unprefixed model ID, such as `'jev-latest'`. String values in `
|
|
168
|
+
unprefixed model ID, such as `'jev-latest'`. String values in `decisionModels`
|
|
169
169
|
also resolve through this global default. Prefer model instances in aliases to
|
|
170
170
|
avoid resolution cycles. Global configuration affects other AI SDK functions
|
|
171
171
|
too; avoid changing it per request in a shared process.
|
|
172
172
|
|
|
173
|
-
An explicitly configured default provider must expose
|
|
173
|
+
An explicitly configured default provider must expose a `decisionModel`
|
|
174
174
|
method; Gateway is used only when no default provider is configured. Missing
|
|
175
|
-
registry providers throw `NoSuchProviderError`; missing models or
|
|
175
|
+
registry providers throw `NoSuchProviderError`; missing models or decision
|
|
176
176
|
capabilities throw
|
|
177
|
-
`NoSuchModelError` with `modelType: '
|
|
177
|
+
`NoSuchModelError` with `modelType: 'decisionModel'`. Unsupported model versions
|
|
178
178
|
throw `UnsupportedModelVersionError`, including models returned by registries.
|
|
179
179
|
The stable `ProviderV4` and `ProviderRegistryProvider` interfaces are unchanged;
|
|
180
|
-
keep the inferred registry type, or use `
|
|
181
|
-
to retain its experimental `
|
|
180
|
+
keep the inferred registry type, or use `Experimental_DecisionProviderRegistry`,
|
|
181
|
+
to retain its experimental `decisionModel` method.
|
|
182
182
|
|
|
183
183
|
## Question types
|
|
184
184
|
|
|
@@ -214,7 +214,7 @@ are preserved, never silently normalized.
|
|
|
214
214
|
Choice and Score distributions are optional. Boolean probability is required:
|
|
215
215
|
`0.98` means a strong yes and `0.02` means a strong no. It is not confidence in
|
|
216
216
|
either outcome. The SDK does not promise calibration across providers. Structured
|
|
217
|
-
language-model adapters prompt the model to estimate P(true); native
|
|
217
|
+
language-model adapters prompt the model to estimate P(true); native decision providers return
|
|
218
218
|
their API probabilities. Provider-specific confidence statistics belong
|
|
219
219
|
in `providerMetadata`. TypeSafe exposes its separate Choice/Score confidence
|
|
220
220
|
statistic at `result.providerMetadata?.typesafe?.confidence`, keyed by question
|
|
@@ -245,34 +245,34 @@ if (result.answers.requestsRefund.probability >= 0.8) {
|
|
|
245
245
|
|
|
246
246
|
The model's `supportedQuestionTypes` are checked before calling the provider.
|
|
247
247
|
Any unsupported question fails the entire call with
|
|
248
|
-
`
|
|
248
|
+
`Experimental_DecisionUnsupportedQuestionTypeError`. Successful calls return
|
|
249
249
|
an answer for every question; there is no partial success, and Core does not
|
|
250
250
|
substitute models automatically.
|
|
251
251
|
|
|
252
252
|
Invalid inputs throw `InvalidArgumentError`. Missing answers, mismatched answer
|
|
253
253
|
types, invalid options, scores, or probabilities throw `InvalidResponseDataError`.
|
|
254
254
|
Transient provider failures use the normal retry policy (`maxRetries: 2` by
|
|
255
|
-
default). Use `abortSignal` to cancel
|
|
255
|
+
default). Use `abortSignal` to cancel the decisions request, `headers` for request headers,
|
|
256
256
|
and `providerOptions` for provider-specific settings.
|
|
257
257
|
|
|
258
258
|
The result includes `usage`, `warnings`, `providerMetadata`, and `response`.
|
|
259
259
|
Unknown token counts stay `undefined`; `totalTokens` is available only when both
|
|
260
260
|
input and output counts are known.
|
|
261
261
|
|
|
262
|
-
For tests, use `
|
|
262
|
+
For tests, use `Experimental_DecisionMockModelV4` from `ai/test`.
|
|
263
263
|
|
|
264
264
|
## Scope and examples
|
|
265
265
|
|
|
266
|
-
|
|
266
|
+
`experimental_decide` currently returns one complete result for one shared state. It does
|
|
267
267
|
not stream answers, perform multilabel classification, or batch unrelated
|
|
268
268
|
states. Run separate calls for separate states. Provider support and judgment
|
|
269
269
|
quality depend on the chosen model. Core does not choose another model, but a
|
|
270
270
|
provider can implement routing through its provider options. For example, AI
|
|
271
|
-
Gateway supports conditional
|
|
271
|
+
Gateway supports conditional decision fallbacks under
|
|
272
272
|
`providerOptions.gateway.models`.
|
|
273
273
|
|
|
274
274
|
Runnable examples are in
|
|
275
|
-
[`examples/ai-functions/src/
|
|
275
|
+
[`examples/ai-functions/src/decide`](https://github.com/vercel/ai/tree/main/examples/ai-functions/src/decide),
|
|
276
276
|
including basic examples for TypeSafe, OpenAI, Anthropic, and Google, model
|
|
277
277
|
registries, custom aliases, default-provider strings, and probability-based routing.
|
|
278
278
|
|
|
@@ -281,7 +281,7 @@ registries, custom aliases, default-provider strings, and probability-based rout
|
|
|
281
281
|
Explore use cases, implementation guides, and a deployable template for using
|
|
282
282
|
Jev from TypeSafe AI with the AI SDK:
|
|
283
283
|
|
|
284
|
-
- [Jev use cases](https://vercel.com/i/jev-use-cases): Explore ideas for applying
|
|
285
|
-
- [Classify, route, and score with Jev and AI SDK](https://vercel.com/kb/guide/typesafe-jev-and-ai-sdk): Learn how to ask typed questions, route answers using probabilities, and test routing logic with a mock
|
|
284
|
+
- [Jev use cases](https://vercel.com/i/jev-use-cases): Explore ideas for applying decisions in your application.
|
|
285
|
+
- [Classify, route, and score with Jev and AI SDK](https://vercel.com/kb/guide/typesafe-jev-and-ai-sdk): Learn how to ask typed questions, route answers using probabilities, and test routing logic with a mock decision model.
|
|
286
286
|
- [Jev and AI SDK form router guide](https://vercel.com/kb/guide/jev-ai-sdk-form-router): Explore form routing with Jev and the AI SDK.
|
|
287
|
-
- [Jev and AI SDK form router template](https://vercel.com/templates/next.js/jev-and-ai-sdk): Deploy a Next.js application that routes form submissions with Jev and uses a language model fallback for uncertain or failed
|
|
287
|
+
- [Jev and AI SDK form router template](https://vercel.com/templates/next.js/jev-and-ai-sdk): Deploy a Next.js application that routes form submissions with Jev and uses a language model fallback for uncertain or failed decisions.
|
|
@@ -308,7 +308,7 @@ __PROVIDER_IMPORT__;
|
|
|
308
308
|
const model = wrapImageModel({
|
|
309
309
|
model: __IMAGE_MODEL__,
|
|
310
310
|
middleware: {
|
|
311
|
-
specificationVersion: '
|
|
311
|
+
specificationVersion: 'v4',
|
|
312
312
|
transformParams: async ({ params }) => ({
|
|
313
313
|
...params,
|
|
314
314
|
size: params.size ?? '1024x1024',
|
|
@@ -400,6 +400,11 @@ for (const file of result.files) {
|
|
|
400
400
|
| [Together.ai](/providers/ai-sdk-providers/togetherai#image-models) | `black-forest-labs/FLUX.1.1-pro` | 512x512, 768x768, 1024x1024 |
|
|
401
401
|
| [Together.ai](/providers/ai-sdk-providers/togetherai#image-models) | `black-forest-labs/FLUX.1-pro` | 512x512, 768x768, 1024x1024 |
|
|
402
402
|
| [Together.ai](/providers/ai-sdk-providers/togetherai#image-models) | `black-forest-labs/FLUX.1-schnell-Free` | 512x512, 768x768, 1024x1024 |
|
|
403
|
+
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-2-pro` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
404
|
+
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-2-max` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
405
|
+
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-2-flex` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
406
|
+
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-2-klein-9b` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
407
|
+
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-2-klein-4b` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
403
408
|
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-kontext-pro` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
404
409
|
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-kontext-max` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
405
410
|
| [Black Forest Labs](/providers/ai-sdk-providers/black-forest-labs#image-models) | `flux-pro-1.1-ultra` | From 3:7 (portrait) to 7:3 (landscape) |
|
|
@@ -138,7 +138,9 @@ The `audio` stream must contain raw audio chunks. `Uint8Array` chunks are raw by
|
|
|
138
138
|
OpenAI streaming transcription uses `openai.transcription('gpt-realtime-whisper')`.
|
|
139
139
|
Cartesia uses `cartesia.transcription('ink-2')` for streaming-only Ink 2
|
|
140
140
|
transcription. ElevenLabs uses
|
|
141
|
-
`elevenLabs.transcription('scribe_v2_realtime')` for Scribe v2 Realtime.
|
|
141
|
+
`elevenLabs.transcription('scribe_v2_realtime')` for Scribe v2 Realtime. Azure
|
|
142
|
+
uses `azure.transcription('mai-transcribe-2-streaming')` for streaming-only
|
|
143
|
+
MAI-Transcribe-2-Streaming. xAI
|
|
142
144
|
uses the same `xai.transcription()` model for request/response and streaming
|
|
143
145
|
transcription; `experimental_streamTranscribe` selects the provider's
|
|
144
146
|
WebSocket STT transport.
|
|
@@ -329,43 +331,44 @@ try {
|
|
|
329
331
|
|
|
330
332
|
## Transcription Models
|
|
331
333
|
|
|
332
|
-
| Provider | Model
|
|
333
|
-
| ----------------------------------------------------------------------------------- |
|
|
334
|
-
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `whisper-1`
|
|
335
|
-
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-transcribe`
|
|
336
|
-
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-mini-transcribe`
|
|
337
|
-
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-transcribe-diarize`
|
|
338
|
-
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v1`
|
|
339
|
-
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v1_experimental`
|
|
340
|
-
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v2`
|
|
341
|
-
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#streaming-transcription-models) | `scribe_v2_realtime`
|
|
342
|
-
| [Groq](/providers/ai-sdk-providers/groq#transcription-models) | `whisper-large-v3-turbo`
|
|
343
|
-
| [Groq](/providers/ai-sdk-providers/groq#transcription-models) | `whisper-large-v3`
|
|
344
|
-
| [Mistral](/providers/ai-sdk-providers/mistral#transcription-models) | `voxtral-mini-latest`
|
|
345
|
-
| [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `whisper-1`
|
|
346
|
-
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-transcribe) | `mai-transcribe-2`
|
|
347
|
-
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-transcribe) | `mai-transcribe-1.5`
|
|
348
|
-
| [Azure
|
|
349
|
-
| [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `gpt-4o-
|
|
350
|
-
| [
|
|
351
|
-
| [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `
|
|
352
|
-
| [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `
|
|
353
|
-
| [
|
|
354
|
-
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `
|
|
355
|
-
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `
|
|
356
|
-
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova
|
|
357
|
-
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova-
|
|
358
|
-
| [
|
|
359
|
-
| [
|
|
360
|
-
| [AssemblyAI](/providers/ai-sdk-providers/assemblyai#transcription-models) | `universal-3-pro`
|
|
361
|
-
| [
|
|
362
|
-
| [Fal](/providers/ai-sdk-providers/fal#transcription-models) | `
|
|
363
|
-
| [
|
|
364
|
-
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `
|
|
365
|
-
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `
|
|
366
|
-
| [
|
|
367
|
-
| [
|
|
368
|
-
| [Cartesia](/providers/ai-sdk-providers/cartesia#
|
|
369
|
-
| [
|
|
334
|
+
| Provider | Model |
|
|
335
|
+
| ----------------------------------------------------------------------------------- | ---------------------------- |
|
|
336
|
+
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `whisper-1` |
|
|
337
|
+
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-transcribe` |
|
|
338
|
+
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-mini-transcribe` |
|
|
339
|
+
| [OpenAI](/providers/ai-sdk-providers/openai#transcription-models) | `gpt-4o-transcribe-diarize` |
|
|
340
|
+
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v1` |
|
|
341
|
+
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v1_experimental` |
|
|
342
|
+
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#transcription-models) | `scribe_v2` |
|
|
343
|
+
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#streaming-transcription-models) | `scribe_v2_realtime` |
|
|
344
|
+
| [Groq](/providers/ai-sdk-providers/groq#transcription-models) | `whisper-large-v3-turbo` |
|
|
345
|
+
| [Groq](/providers/ai-sdk-providers/groq#transcription-models) | `whisper-large-v3` |
|
|
346
|
+
| [Mistral](/providers/ai-sdk-providers/mistral#transcription-models) | `voxtral-mini-latest` |
|
|
347
|
+
| [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `whisper-1` |
|
|
348
|
+
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-transcribe) | `mai-transcribe-2` |
|
|
349
|
+
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-transcribe) | `mai-transcribe-1.5` |
|
|
350
|
+
| [Azure MAI](/providers/ai-sdk-providers/azure#mai-transcribe-2-streaming) | `mai-transcribe-2-streaming` |
|
|
351
|
+
| [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `gpt-4o-transcribe` |
|
|
352
|
+
| [Azure OpenAI](/providers/ai-sdk-providers/azure#transcription-models) | `gpt-4o-mini-transcribe` |
|
|
353
|
+
| [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `machine` |
|
|
354
|
+
| [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `low_cost` |
|
|
355
|
+
| [Rev.ai](/providers/ai-sdk-providers/revai#transcription-models) | `fusion` |
|
|
356
|
+
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `base` (+ variants) |
|
|
357
|
+
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `enhanced` (+ variants) |
|
|
358
|
+
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova` (+ variants) |
|
|
359
|
+
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova-2` (+ variants) |
|
|
360
|
+
| [Deepgram](/providers/ai-sdk-providers/deepgram#transcription-models) | `nova-3` (+ variants) |
|
|
361
|
+
| [Gladia](/providers/ai-sdk-providers/gladia#transcription-models) | `default` |
|
|
362
|
+
| [AssemblyAI](/providers/ai-sdk-providers/assemblyai#transcription-models) | `universal-3-5-pro` |
|
|
363
|
+
| [AssemblyAI](/providers/ai-sdk-providers/assemblyai#transcription-models) | `universal-3-pro` |
|
|
364
|
+
| [Fal](/providers/ai-sdk-providers/fal#transcription-models) | `whisper` |
|
|
365
|
+
| [Fal](/providers/ai-sdk-providers/fal#transcription-models) | `wizper` |
|
|
366
|
+
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `chirp_2` |
|
|
367
|
+
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `chirp_3` |
|
|
368
|
+
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#transcription-models) | `telephony` |
|
|
369
|
+
| [xAI](/providers/ai-sdk-providers/xai#transcription-models) | `default` |
|
|
370
|
+
| [Cartesia](/providers/ai-sdk-providers/cartesia#transcription-models) | `ink-whisper` |
|
|
371
|
+
| [Cartesia](/providers/ai-sdk-providers/cartesia#streaming-transcription-models) | `ink-2` |
|
|
372
|
+
| [Fish Audio](/providers/ai-sdk-providers/fish-audio#transcription-models) | `transcribe-1` |
|
|
370
373
|
|
|
371
374
|
Above are a small subset of the transcription models supported by the AI SDK providers. For more, see the respective provider documentation.
|
|
@@ -158,6 +158,8 @@ try {
|
|
|
158
158
|
| [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `tts-1-hd` |
|
|
159
159
|
| [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `gpt-4o-mini-tts` |
|
|
160
160
|
| [Mistral](/providers/ai-sdk-providers/mistral#speech-models) | `voxtral-mini-tts-2603` |
|
|
161
|
+
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-voice) | `mai-voice-2.1-flash` |
|
|
162
|
+
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-voice) | `mai-voice-2.1` |
|
|
161
163
|
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-voice) | `mai-voice-2-flash` |
|
|
162
164
|
| [Azure Speech](/providers/ai-sdk-providers/azure#mai-voice) | `mai-voice-2` |
|
|
163
165
|
| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_v3` |
|
|
@@ -469,12 +469,12 @@ const result = await streamText({
|
|
|
469
469
|
|
|
470
470
|
This simplifies provider usage and makes it easier to switch between providers without changing your model references throughout your codebase.
|
|
471
471
|
|
|
472
|
-
## Experimental
|
|
473
|
-
|
|
474
|
-
Custom providers accept `
|
|
475
|
-
`
|
|
476
|
-
`
|
|
477
|
-
explicitly configured default provider with
|
|
478
|
-
middleware for language and image models does not apply to
|
|
479
|
-
See [
|
|
472
|
+
## Experimental decision models
|
|
473
|
+
|
|
474
|
+
Custom providers accept `decisionModels` aliases, and registries expose
|
|
475
|
+
`decisionModel('provider:model')`. These methods return model instances for
|
|
476
|
+
`experimental_decide`. Direct string IDs use Gateway by default, or an
|
|
477
|
+
explicitly configured default provider with a `decisionModel` method. Registry
|
|
478
|
+
middleware for language and image models does not apply to decision.
|
|
479
|
+
See [Decisions](/docs/ai-sdk-core/decisions#model-aliases-and-registries)
|
|
480
480
|
for aliases, default-provider configuration, and capability differences.
|
|
@@ -114,7 +114,7 @@ const result = await generateText({
|
|
|
114
114
|
});
|
|
115
115
|
```
|
|
116
116
|
|
|
117
|
-
|
|
117
|
+
Not registering integrations with `registerTelemetry()` prevents delivery to globally registered integrations, but does not disable per-call integrations or Node.js tracing-channel subscribers. To disable telemetry events for a call, including tracing-channel events, pass `telemetry: { isEnabled: false }`. Apply this setting to every call if you want to opt out throughout your application.
|
|
118
118
|
|
|
119
119
|
## Telemetry Metadata
|
|
120
120
|
|
|
@@ -157,7 +157,7 @@ const result = await generateText({
|
|
|
157
157
|
});
|
|
158
158
|
```
|
|
159
159
|
|
|
160
|
-
In this example, telemetry integrations receive `runtimeContext` as `{ requestId: 'req_abc' }`. Properties set to `false` or omitted are excluded. If `telemetry.includeRuntimeContext` is omitted, no runtime context properties are included. `telemetry.includeRuntimeContext` is supported by `generateText`, `streamText`, `ToolLoopAgent`, `embed`, `embedMany`, `rerank`, and `
|
|
160
|
+
In this example, telemetry integrations receive `runtimeContext` as `{ requestId: 'req_abc' }`. Properties set to `false` or omitted are excluded. If `telemetry.includeRuntimeContext` is omitted, no runtime context properties are included. `telemetry.includeRuntimeContext` is supported by `generateText`, `streamText`, `ToolLoopAgent`, `embed`, `embedMany`, `rerank`, and `experimental_decide`.
|
|
161
161
|
|
|
162
162
|
<Note>
|
|
163
163
|
`telemetry.includeRuntimeContext` only filters telemetry integrations,
|
|
@@ -354,6 +354,10 @@ class MyIntegration implements Telemetry {
|
|
|
354
354
|
async onAbort(event) {
|
|
355
355
|
console.log('Stream aborted after', event.steps.length, 'finished steps');
|
|
356
356
|
}
|
|
357
|
+
|
|
358
|
+
async onError(event) {
|
|
359
|
+
console.error('Generation failed:', event.callId, event.error);
|
|
360
|
+
}
|
|
357
361
|
}
|
|
358
362
|
|
|
359
363
|
export function myIntegration(): Telemetry {
|
|
@@ -415,27 +419,26 @@ export function myIntegration(): Telemetry {
|
|
|
415
419
|
description: 'Called when an individual reranking model call completes.',
|
|
416
420
|
},
|
|
417
421
|
{
|
|
418
|
-
name: '
|
|
419
|
-
type: '(event:
|
|
420
|
-
description: 'Called when an experimental
|
|
422
|
+
name: 'experimental_onDecideStart',
|
|
423
|
+
type: '(event: Experimental_DecideStartEvent) => void | PromiseLike<void>',
|
|
424
|
+
description: 'Called when an experimental decision operation begins.',
|
|
421
425
|
},
|
|
422
426
|
{
|
|
423
|
-
name: '
|
|
424
|
-
type: '(event:
|
|
427
|
+
name: 'experimental_onDecisionModelCallStart',
|
|
428
|
+
type: '(event: Experimental_DecisionModelCallStartEvent) => void | PromiseLike<void>',
|
|
425
429
|
description:
|
|
426
|
-
'Called when the logical
|
|
430
|
+
'Called when the logical decision model call begins. The call includes any provider retries.',
|
|
427
431
|
},
|
|
428
432
|
{
|
|
429
|
-
name: '
|
|
430
|
-
type: '(event:
|
|
433
|
+
name: 'experimental_onDecisionModelCallEnd',
|
|
434
|
+
type: '(event: Experimental_DecisionModelCallEndEvent) => void | PromiseLike<void>',
|
|
431
435
|
description:
|
|
432
|
-
'Called when the logical
|
|
436
|
+
'Called when the logical decision model call, including any retries, completes.',
|
|
433
437
|
},
|
|
434
438
|
{
|
|
435
|
-
name: '
|
|
436
|
-
type: '(event:
|
|
437
|
-
description:
|
|
438
|
-
'Called when an experimental evaluation operation completes.',
|
|
439
|
+
name: 'experimental_onDecideEnd',
|
|
440
|
+
type: '(event: Experimental_DecideEndEvent) => void | PromiseLike<void>',
|
|
441
|
+
description: 'Called when an experimental decision operation completes.',
|
|
439
442
|
},
|
|
440
443
|
{
|
|
441
444
|
name: 'onEnd',
|
|
@@ -449,10 +452,16 @@ export function myIntegration(): Telemetry {
|
|
|
449
452
|
description:
|
|
450
453
|
'Called when a streaming text generation operation is aborted before it completes.',
|
|
451
454
|
},
|
|
455
|
+
{
|
|
456
|
+
name: 'onError',
|
|
457
|
+
type: '(event: { callId: string; error: unknown }) => void | PromiseLike<void>',
|
|
458
|
+
description:
|
|
459
|
+
'Called when an unrecoverable error occurs. The event identifies the operation and contains the thrown value.',
|
|
460
|
+
},
|
|
452
461
|
]}
|
|
453
462
|
/>
|
|
454
463
|
|
|
455
|
-
Experimental
|
|
464
|
+
Experimental decision events use the four `experimental_*` callbacks above.
|
|
456
465
|
They are not included in the stable `onStart` and `onEnd` event unions.
|
|
457
466
|
|
|
458
467
|
The event types for each method are the same as the corresponding [lifecycle callbacks](/docs/ai-sdk-core/lifecycle-callbacks). See the lifecycle callbacks documentation for the full property reference of each event.
|
|
@@ -598,30 +607,30 @@ corresponding input/output recording controls. Raw audio is not captured, so
|
|
|
598
607
|
transcription input and speech output do not populate message attributes with
|
|
599
608
|
audio content or placeholder audio parts.
|
|
600
609
|
|
|
601
|
-
####
|
|
610
|
+
#### experimental_decide
|
|
602
611
|
|
|
603
|
-
For `
|
|
604
|
-
kind:
|
|
605
|
-
**`
|
|
612
|
+
For `experimental_decide`, the integration records two spans with `CLIENT`
|
|
613
|
+
kind: a **`decide {modelId}`** root span for the whole operation and one
|
|
614
|
+
**`decide {modelId}`** child span for decision model work, including
|
|
606
615
|
retries. Both include the provider and requested model with
|
|
607
|
-
`gen_ai.operation.name: "
|
|
608
|
-
usage is recorded on the child span. When
|
|
616
|
+
`gen_ai.operation.name: "decide"`. Token
|
|
617
|
+
usage is recorded on the child span. When decision attributes are enabled,
|
|
609
618
|
answers are recorded on both spans so the operation span contains its output.
|
|
610
619
|
|
|
611
|
-
`
|
|
620
|
+
`decide` is an AI SDK-defined custom operation name, not a well-known
|
|
612
621
|
OpenTelemetry GenAI operation. OpenTelemetry permits custom operation names
|
|
613
622
|
when no well-known value describes the operation.
|
|
614
623
|
|
|
615
624
|
OpenTelemetry's `gen_ai.evaluation.result` event describes the result of
|
|
616
625
|
evaluating a GenAI output for quality or other characteristics. In contrast,
|
|
617
|
-
`
|
|
626
|
+
`experimental_decide` can use arbitrary JSON state for classification,
|
|
618
627
|
routing, scoring, and similar tasks. The integration therefore does not emit a
|
|
619
628
|
`gen_ai.evaluation.result` event.
|
|
620
629
|
|
|
621
|
-
|
|
622
|
-
semantic-convention attributes. Enable the `
|
|
630
|
+
Decision state, questions, and answers also do not have applicable GenAI
|
|
631
|
+
semantic-convention attributes. Enable the `experimental_decision`
|
|
623
632
|
supplemental option to record them under AI SDK-specific
|
|
624
|
-
`ai.
|
|
633
|
+
`ai.decision.*` attributes.
|
|
625
634
|
|
|
626
635
|
#### GenAI span details
|
|
627
636
|
|
|
@@ -678,10 +687,10 @@ registerTelemetry(
|
|
|
678
687
|
|
|
679
688
|
The callback runs when each span is created and receives:
|
|
680
689
|
|
|
681
|
-
- `spanType`: the type of span being created (`operation`, `step`, `languageModel`, `tool`, `embedding`, `reranking`, or `
|
|
690
|
+
- `spanType`: the type of span being created (`operation`, `step`, `languageModel`, `tool`, `embedding`, `reranking`, or `experimental_decision`).
|
|
682
691
|
- `operationId`: the AI SDK operation ID for the current call, such as `ai.generateText` or `ai.streamText`.
|
|
683
692
|
- `callId`: the unique ID for the current AI SDK call.
|
|
684
|
-
- `runtimeContext`: the telemetry-filtered runtime context for text generation, embedding, reranking, and
|
|
693
|
+
- `runtimeContext`: the telemetry-filtered runtime context for text generation, embedding, reranking, and decision spans. Text generation spans also reflect updates from `prepareStep`.
|
|
685
694
|
|
|
686
695
|
Custom attributes are merged with AI SDK attributes on the span. AI SDK-owned
|
|
687
696
|
attributes take precedence when a custom attribute uses the same key.
|
|
@@ -741,7 +750,7 @@ registerTelemetry(
|
|
|
741
750
|
providerMetadata: true,
|
|
742
751
|
embedding: true,
|
|
743
752
|
reranking: true,
|
|
744
|
-
|
|
753
|
+
experimental_decision: true,
|
|
745
754
|
runtimeContext: true,
|
|
746
755
|
headers: true,
|
|
747
756
|
toolChoice: true,
|
|
@@ -756,7 +765,7 @@ The available options are:
|
|
|
756
765
|
- `providerMetadata`: `ai.response.providerMetadata` on operation, step, and model-call spans.
|
|
757
766
|
- `embedding`: embedding inputs and outputs.
|
|
758
767
|
- `reranking`: rerank input documents and ranking output.
|
|
759
|
-
- `
|
|
768
|
+
- `experimental_decision`: experimental decision state, questions, and answers.
|
|
760
769
|
- `runtimeContext`: `ai.settings.context.*`.
|
|
761
770
|
- `headers`: `ai.request.headers.*`.
|
|
762
771
|
- `toolChoice`: `ai.prompt.toolChoice`.
|