ai 6.0.30 → 6.0.32

package/docs/03-ai-sdk-core/37-speech.mdx

@@ -0,0 +1,167 @@
+---
+title: Speech
+description: Learn how to generate speech from text with the AI SDK.
+---
+
+# Speech
+
+<Note type="warning">Speech is an experimental feature.</Note>
+
+The AI SDK provides the [`generateSpeech`](/docs/reference/ai-sdk-core/generate-speech)
+function to generate speech from text using a speech model.
+
+```ts
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import { openai } from '@ai-sdk/openai';
+
+const audio = await generateSpeech({
+  model: openai.speech('tts-1'),
+  text: 'Hello, world!',
+  voice: 'alloy',
+});
+```
+
+### Language Setting
+
+You can specify the language for speech generation (provider support varies):
+
+```ts
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import { lmnt } from '@ai-sdk/lmnt';
+
+const audio = await generateSpeech({
+  model: lmnt.speech('aurora'),
+  text: 'Hola, mundo!',
+  language: 'es', // Spanish
+});
+```
+
+To access the generated audio:
+
+```ts
+const audio = audio.audioData; // audio data e.g. Uint8Array
+```
+
+## Settings
+
+### Provider-Specific settings
+
+You can set model-specific settings with the `providerOptions` parameter.
+
+```ts highlight="7-11"
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import { openai } from '@ai-sdk/openai';
+
+const audio = await generateSpeech({
+  model: openai.speech('tts-1'),
+  text: 'Hello, world!',
+  providerOptions: {
+    openai: {
+      // ...
+    },
+  },
+});
+```
+
+### Abort Signals and Timeouts
+
+`generateSpeech` accepts an optional `abortSignal` parameter of
+type [`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)
+that you can use to abort the speech generation process or set a timeout.
+
+```ts highlight="7"
+import { openai } from '@ai-sdk/openai';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+
+const audio = await generateSpeech({
+  model: openai.speech('tts-1'),
+  text: 'Hello, world!',
+  abortSignal: AbortSignal.timeout(1000), // Abort after 1 second
+});
+```
+
+### Custom Headers
+
+`generateSpeech` accepts an optional `headers` parameter of type `Record<string, string>`
+that you can use to add custom headers to the speech generation request.
+
+```ts highlight="7"
+import { openai } from '@ai-sdk/openai';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+
+const audio = await generateSpeech({
+  model: openai.speech('tts-1'),
+  text: 'Hello, world!',
+  headers: { 'X-Custom-Header': 'custom-value' },
+});
+```
+
+### Warnings
+
+Warnings (e.g. unsupported parameters) are available on the `warnings` property.
+
+```ts
+import { openai } from '@ai-sdk/openai';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+
+const audio = await generateSpeech({
+  model: openai.speech('tts-1'),
+  text: 'Hello, world!',
+});
+
+const warnings = audio.warnings;
+```
+
+### Error Handling
+
+When `generateSpeech` cannot generate a valid audio, it throws a [`AI_NoSpeechGeneratedError`](/docs/reference/ai-sdk-errors/ai-no-speech-generated-error).
+
+This error can arise for any the following reasons:
+
+- The model failed to generate a response
+- The model generated a response that could not be parsed
+
+The error preserves the following information to help you log the issue:
+
+- `responses`: Metadata about the speech model responses, including timestamp, model, and headers.
+- `cause`: The cause of the error. You can use this for more detailed error handling.
+
+```ts
+import {
+  experimental_generateSpeech as generateSpeech,
+  NoSpeechGeneratedError,
+} from 'ai';
+import { openai } from '@ai-sdk/openai';
+
+try {
+  await generateSpeech({
+    model: openai.speech('tts-1'),
+    text: 'Hello, world!',
+  });
+} catch (error) {
+  if (NoSpeechGeneratedError.isInstance(error)) {
+    console.log('AI_NoSpeechGeneratedError');
+    console.log('Cause:', error.cause);
+    console.log('Responses:', error.responses);
+  }
+}
+```
+
+## Speech Models
+
+| Provider                                                           | Model                    |
+| ------------------------------------------------------------------ | ------------------------ |
+| [OpenAI](/providers/ai-sdk-providers/openai#speech-models)         | `tts-1`                  |
+| [OpenAI](/providers/ai-sdk-providers/openai#speech-models)         | `tts-1-hd`               |
+| [OpenAI](/providers/ai-sdk-providers/openai#speech-models)         | `gpt-4o-mini-tts`        |
+| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_v3`              |
+| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_multilingual_v2` |
+| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_flash_v2_5`      |
+| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_flash_v2`        |
+| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_turbo_v2_5`      |
+| [ElevenLabs](/providers/ai-sdk-providers/elevenlabs#speech-models) | `eleven_turbo_v2`        |
+| [LMNT](/providers/ai-sdk-providers/lmnt#speech-models)             | `aurora`                 |
+| [LMNT](/providers/ai-sdk-providers/lmnt#speech-models)             | `blizzard`               |
+| [Hume](/providers/ai-sdk-providers/hume#speech-models)             | `default`                |
+
+Above are a small subset of the speech models supported by the AI SDK providers. For more, see the respective provider documentation.

package/docs/03-ai-sdk-core/40-middleware.mdx

@@ -0,0 +1,480 @@
+---
+title: Language Model Middleware
+description: Learn how to use middleware to enhance the behavior of language models
+---
+
+# Language Model Middleware
+
+Language model middleware is a way to enhance the behavior of language models
+by intercepting and modifying the calls to the language model.
+
+It can be used to add features like guardrails, RAG, caching, and logging
+in a language model agnostic way. Such middleware can be developed and
+distributed independently from the language models that they are applied to.
+
+## Using Language Model Middleware
+
+You can use language model middleware with the `wrapLanguageModel` function.
+It takes a language model and a language model middleware and returns a new
+language model that incorporates the middleware.
+
+```ts
+import { wrapLanguageModel } from 'ai';
+
+const wrappedLanguageModel = wrapLanguageModel({
+  model: yourModel,
+  middleware: yourLanguageModelMiddleware,
+});
+```
+
+The wrapped language model can be used just like any other language model, e.g. in `streamText`:
+
+```ts highlight="2"
+const result = streamText({
+  model: wrappedLanguageModel,
+  prompt: 'What cities are in the United States?',
+});
+```
+
+## Multiple middlewares
+
+You can provide multiple middlewares to the `wrapLanguageModel` function.
+The middlewares will be applied in the order they are provided.
+
+```ts
+const wrappedLanguageModel = wrapLanguageModel({
+  model: yourModel,
+  middleware: [firstMiddleware, secondMiddleware],
+});
+
+// applied as: firstMiddleware(secondMiddleware(yourModel))
+```
+
+## Built-in Middleware
+
+The AI SDK comes with several built-in middlewares that you can use to configure language models:
+
+- `extractReasoningMiddleware`: Extracts reasoning information from the generated text and exposes it as a `reasoning` property on the result.
+- `extractJsonMiddleware`: Extracts JSON from text content by stripping markdown code fences. Useful when using `Output.object()` with models that wrap JSON responses in code blocks.
+- `simulateStreamingMiddleware`: Simulates streaming behavior with responses from non-streaming language models.
+- `defaultSettingsMiddleware`: Applies default settings to a language model.
+- `addToolInputExamplesMiddleware`: Adds tool input examples to tool descriptions for providers that don't natively support the `inputExamples` property.
+
+### Extract Reasoning
+
+Some providers and models expose reasoning information in the generated text using special tags,
+e.g. <think> and </think>.
+
+The `extractReasoningMiddleware` function can be used to extract this reasoning information and expose it as a `reasoning` property on the result.
+
+```ts
+import { wrapLanguageModel, extractReasoningMiddleware } from 'ai';
+
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: extractReasoningMiddleware({ tagName: 'think' }),
+});
+```
+
+You can then use that enhanced model in functions like `generateText` and `streamText`.
+
+The `extractReasoningMiddleware` function also includes a `startWithReasoning` option.
+When set to `true`, the reasoning tag will be prepended to the generated text.
+This is useful for models that do not include the reasoning tag at the beginning of the response.
+For more details, see the [DeepSeek R1 guide](/docs/guides/r1#deepseek-r1-middleware).
+
+### Extract JSON
+
+Some models wrap JSON responses in markdown code fences (e.g., ` ```json ... ``` `) even when you request structured output.
+
+The `extractJsonMiddleware` function strips these code fences from the response, making it compatible with `Output.object()`.
+
+```ts
+import { wrapLanguageModel, extractJsonMiddleware, Output } from 'ai';
+import { z } from 'zod';
+
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: extractJsonMiddleware(),
+});
+
+const result = await generateText({
+  model,
+  output: Output.object({
+    schema: z.object({
+      name: z.string(),
+      ingredients: z.array(z.string()),
+    }),
+  }),
+  prompt: 'Generate a recipe.',
+});
+```
+
+You can also provide a custom transform function for models that use different formatting:
+
+```ts
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: extractJsonMiddleware({
+    transform: text => text.replace(/^PREFIX/, '').replace(/SUFFIX$/, ''),
+  }),
+});
+```
+
+### Simulate Streaming
+
+The `simulateStreamingMiddleware` function can be used to simulate streaming behavior with responses from non-streaming language models.
+This is useful when you want to maintain a consistent streaming interface even when using models that only provide complete responses.
+
+```ts
+import { wrapLanguageModel, simulateStreamingMiddleware } from 'ai';
+
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: simulateStreamingMiddleware(),
+});
+```
+
+### Default Settings
+
+The `defaultSettingsMiddleware` function can be used to apply default settings to a language model.
+
+```ts
+import { wrapLanguageModel, defaultSettingsMiddleware } from 'ai';
+
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: defaultSettingsMiddleware({
+    settings: {
+      temperature: 0.5,
+      maxOutputTokens: 800,
+      providerOptions: { openai: { store: false } },
+    },
+  }),
+});
+```
+
+### Add Tool Input Examples
+
+The `addToolInputExamplesMiddleware` function adds tool input examples to tool descriptions.
+This is useful for providers that don't natively support the `inputExamples` property on tools.
+The middleware serializes the examples into the tool's description text so models can still benefit from seeing example inputs.
+
+```ts
+import { wrapLanguageModel, addToolInputExamplesMiddleware } from 'ai';
+
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: addToolInputExamplesMiddleware({
+    examplesPrefix: 'Input Examples:',
+  }),
+});
+```
+
+When you define a tool with `inputExamples`, the middleware will append them to the tool's description:
+
+```ts
+import { generateText, tool } from 'ai';
+import { z } from 'zod';
+
+const result = await generateText({
+  model, // wrapped model from above
+  tools: {
+    weather: tool({
+      description: 'Get the weather in a location',
+      inputSchema: z.object({
+        location: z.string(),
+      }),
+      inputExamples: [
+        { input: { location: 'San Francisco' } },
+        { input: { location: 'London' } },
+      ],
+    }),
+  },
+  prompt: 'What is the weather in Tokyo?',
+});
+```
+
+The tool description will be transformed to:
+
+```
+Get the weather in a location
+
+Input Examples:
+{"location":"San Francisco"}
+{"location":"London"}
+```
+
+#### Options
+
+- `examplesPrefix` (required): A prefix text to prepend before the examples.
+- `formatExample` (optional): A custom formatter function for each example. Receives the example object and its index. Default: `JSON.stringify(example.input)`.
+- `removeInputExamples` (optional): Whether to remove the `inputExamples` property from the tool after adding them to the description. Default: `true`.
+
+```ts
+const model = wrapLanguageModel({
+  model: yourModel,
+  middleware: addToolInputExamplesMiddleware({
+    examplesPrefix: 'Input Examples:',
+    formatExample: (example, index) =>
+      `${index + 1}. ${JSON.stringify(example.input)}`,
+    removeInputExamples: true,
+  }),
+});
+```
+
+## Community Middleware
+
+The AI SDK provides a Language Model Middleware specification. Community members can develop middleware that adheres to this specification, making it compatible with the AI SDK ecosystem.
+
+Here are some community middlewares that you can explore:
+
+### Custom tool call parser
+
+The [Custom tool call parser](https://github.com/minpeter/ai-sdk-tool-call-middleware) middleware extends tool call capabilities to models that don't natively support the OpenAI-style `tools` parameter. This includes many self-hosted and third-party models that lack native function calling features.
+
+<Note>
+  Using this middleware on models that support native function calls may result
+  in unintended performance degradation, so check whether your model supports
+  native function calls before deciding to use it.
+</Note>
+
+This middleware enables function calling capabilities by converting function schemas into prompt instructions and parsing the model's responses into structured function calls. It works by transforming the JSON function definitions into natural language instructions the model can understand, then analyzing the generated text to extract function call attempts. This approach allows developers to use the same function calling API across different model providers, even with models that don't natively support the OpenAI-style function calling format, providing a consistent function calling experience regardless of the underlying model implementation.
+
+The `@ai-sdk-tool/parser` package offers three middleware variants:
+
+- `createToolMiddleware`: A flexible function for creating custom tool call middleware tailored to specific models
+- `hermesToolMiddleware`: Ready-to-use middleware for Hermes & Qwen format function calls
+- `gemmaToolMiddleware`: Pre-configured middleware for Gemma 3 model series function call format
+
+Here's how you can enable function calls with Gemma models that don't support them natively:
+
+```ts
+import { wrapLanguageModel } from 'ai';
+import { gemmaToolMiddleware } from '@ai-sdk-tool/parser';
+
+const model = wrapLanguageModel({
+  model: openrouter('google/gemma-3-27b-it'),
+  middleware: gemmaToolMiddleware,
+});
+```
+
+Find more examples at this [link](https://github.com/minpeter/ai-sdk-tool-call-middleware/tree/main/examples/core/src).
+
+## Implementing Language Model Middleware
+
+<Note>
+  Implementing language model middleware is advanced functionality and requires
+  a solid understanding of the [language model
+  specification](https://github.com/vercel/ai/blob/v5/packages/provider/src/language-model/v2/language-model-v2.ts).
+</Note>
+
+You can implement any of the following three function to modify the behavior of the language model:
+
+1. `transformParams`: Transforms the parameters before they are passed to the language model, for both `doGenerate` and `doStream`.
+2. `wrapGenerate`: Wraps the `doGenerate` method of the [language model](https://github.com/vercel/ai/blob/v5/packages/provider/src/language-model/v2/language-model-v2.ts).
+   You can modify the parameters, call the language model, and modify the result.
+3. `wrapStream`: Wraps the `doStream` method of the [language model](https://github.com/vercel/ai/blob/v5/packages/provider/src/language-model/v2/language-model-v2.ts).
+   You can modify the parameters, call the language model, and modify the result.
+
+Here are some examples of how to implement language model middleware:
+
+## Examples
+
+<Note>
+  These examples are not meant to be used in production. They are just to show
+  how you can use middleware to enhance the behavior of language models.
+</Note>
+
+### Logging
+
+This example shows how to log the parameters and generated text of a language model call.
+
+```ts
+import type {
+  LanguageModelV3Middleware,
+  LanguageModelV3StreamPart,
+} from '@ai-sdk/provider';
+
+export const yourLogMiddleware: LanguageModelV3Middleware = {
+  wrapGenerate: async ({ doGenerate, params }) => {
+    console.log('doGenerate called');
+    console.log(`params: ${JSON.stringify(params, null, 2)}`);
+
+    const result = await doGenerate();
+
+    console.log('doGenerate finished');
+    console.log(`generated text: ${result.text}`);
+
+    return result;
+  },
+
+  wrapStream: async ({ doStream, params }) => {
+    console.log('doStream called');
+    console.log(`params: ${JSON.stringify(params, null, 2)}`);
+
+    const { stream, ...rest } = await doStream();
+
+    let generatedText = '';
+    const textBlocks = new Map<string, string>();
+
+    const transformStream = new TransformStream<
+      LanguageModelV3StreamPart,
+      LanguageModelV3StreamPart
+    >({
+      transform(chunk, controller) {
+        switch (chunk.type) {
+          case 'text-start': {
+            textBlocks.set(chunk.id, '');
+            break;
+          }
+          case 'text-delta': {
+            const existing = textBlocks.get(chunk.id) || '';
+            textBlocks.set(chunk.id, existing + chunk.delta);
+            generatedText += chunk.delta;
+            break;
+          }
+          case 'text-end': {
+            console.log(
+              `Text block ${chunk.id} completed:`,
+              textBlocks.get(chunk.id),
+            );
+            break;
+          }
+        }
+
+        controller.enqueue(chunk);
+      },
+
+      flush() {
+        console.log('doStream finished');
+        console.log(`generated text: ${generatedText}`);
+      },
+    });
+
+    return {
+      stream: stream.pipeThrough(transformStream),
+      ...rest,
+    };
+  },
+};
+```
+
+### Caching
+
+This example shows how to build a simple cache for the generated text of a language model call.
+
+```ts
+import type { LanguageModelV3Middleware } from '@ai-sdk/provider';
+
+const cache = new Map<string, any>();
+
+export const yourCacheMiddleware: LanguageModelV3Middleware = {
+  wrapGenerate: async ({ doGenerate, params }) => {
+    const cacheKey = JSON.stringify(params);
+
+    if (cache.has(cacheKey)) {
+      return cache.get(cacheKey);
+    }
+
+    const result = await doGenerate();
+
+    cache.set(cacheKey, result);
+
+    return result;
+  },
+
+  // here you would implement the caching logic for streaming
+};
+```
+
+### Retrieval Augmented Generation (RAG)
+
+This example shows how to use RAG as middleware.
+
+<Note>
+  Helper functions like `getLastUserMessageText` and `findSources` are not part
+  of the AI SDK. They are just used in this example to illustrate the concept of
+  RAG.
+</Note>
+
+```ts
+import type { LanguageModelV3Middleware } from '@ai-sdk/provider';
+
+export const yourRagMiddleware: LanguageModelV3Middleware = {
+  transformParams: async ({ params }) => {
+    const lastUserMessageText = getLastUserMessageText({
+      prompt: params.prompt,
+    });
+
+    if (lastUserMessageText == null) {
+      return params; // do not use RAG (send unmodified parameters)
+    }
+
+    const instruction =
+      'Use the following information to answer the question:\n' +
+      findSources({ text: lastUserMessageText })
+        .map(chunk => JSON.stringify(chunk))
+        .join('\n');
+
+    return addToLastUserMessage({ params, text: instruction });
+  },
+};
+```
+
+### Guardrails
+
+Guard rails are a way to ensure that the generated text of a language model call
+is safe and appropriate. This example shows how to use guardrails as middleware.
+
+```ts
+import type { LanguageModelV3Middleware } from '@ai-sdk/provider';
+
+export const yourGuardrailMiddleware: LanguageModelV3Middleware = {
+  wrapGenerate: async ({ doGenerate }) => {
+    const { text, ...rest } = await doGenerate();
+
+    // filtering approach, e.g. for PII or other sensitive information:
+    const cleanedText = text?.replace(/badword/g, '<REDACTED>');
+
+    return { text: cleanedText, ...rest };
+  },
+
+  // here you would implement the guardrail logic for streaming
+  // Note: streaming guardrails are difficult to implement, because
+  // you do not know the full content of the stream until it's finished.
+};
+```
+
+## Configuring Per Request Custom Metadata
+
+To send and access custom metadata in Middleware, you can use `providerOptions`. This is useful when building logging middleware where you want to pass additional context like user IDs, timestamps, or other contextual data that can help with tracking and debugging.
+
+```ts
+import { generateText, wrapLanguageModel } from 'ai';
+__PROVIDER_IMPORT__;
+import type { LanguageModelV3Middleware } from '@ai-sdk/provider';
+
+export const yourLogMiddleware: LanguageModelV3Middleware = {
+  wrapGenerate: async ({ doGenerate, params }) => {
+    console.log('METADATA', params?.providerMetadata?.yourLogMiddleware);
+    const result = await doGenerate();
+    return result;
+  },
+};
+
+const { text } = await generateText({
+  model: wrapLanguageModel({
+    model: __MODEL__,
+    middleware: yourLogMiddleware,
+  }),
+  prompt: 'Invent a new holiday and describe its traditions.',
+  providerOptions: {
+    yourLogMiddleware: {
+      hello: 'world',
+    },
+  },
+});
+
+console.log(text);
+```