@mastra/mcp-docs-server 1.2.15-alpha.19 → 1.2.15-alpha.21

package/.docs/{reference/voice/inworld-realtime.md → integrations/voice/inworld.md}

@@ -1,14 +1,152 @@
 > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
 
-# Inworld Realtime voice
+# Inworld
+
+## Voice
+
+The Inworld voice implementation in Mastra provides streaming text-to-speech (TTS) and batch speech-to-text (STT) capabilities using Inworld AI's API. It supports multiple TTS and STT models, configurable audio encodings, and progressive audio streaming.
+
+For real-time, full-duplex speech-to-speech, the same package exports [`InworldRealtimeVoice`](#realtime).
+
+### Usage example
+
+```typescript
+import { InworldVoice } from '@mastra/voice-inworld'
+
+// Initialize with default configuration (uses INWORLD_API_KEY environment variable)
+const voice = new InworldVoice()
+
+// Initialize with custom configuration
+const voice = new InworldVoice({
+  speechModel: {
+    name: 'inworld-tts-2',
+    apiKey: 'your-api-key',
+  },
+  listeningModel: {
+    name: 'groq/whisper-large-v3',
+    apiKey: 'your-api-key',
+  },
+  speaker: 'Dennis',
+})
+
+// Text-to-Speech (streaming)
+const audioStream = await voice.speak('Hello, world!')
+
+// Speech-to-Text
+const transcript = await voice.listen(audioStream)
+```
+
+### Constructor parameters
+
+**speechModel** (`InworldVoiceConfig`): Configuration for text-to-speech functionality. (Default: `{ name: 'inworld-tts-2' }`)
+
+**speechModel.name** (`'inworld-tts-2' | 'inworld-tts-1.5-max' | 'inworld-tts-1.5-mini'`): The Inworld TTS model to use.
+
+**speechModel.apiKey** (`string`): Inworld API key. Falls back to INWORLD\_API\_KEY environment variable.
+
+**listeningModel** (`InworldListeningConfig`): Configuration for speech-to-text functionality. (Default: `{ name: 'groq/whisper-large-v3' }`)
+
+**listeningModel.name** (`'groq/whisper-large-v3'`): The Inworld STT model to use.
+
+**listeningModel.apiKey** (`string`): Inworld API key. Falls back to INWORLD\_API\_KEY environment variable.
+
+**speaker** (`string`): Default voice ID to use for text-to-speech. (Default: `'Dennis'`)
+
+**audioEncoding** (`'LINEAR16' | 'MP3' | 'OGG_OPUS' | 'ALAW' | 'MULAW' | 'FLAC' | 'PCM' | 'WAV'`): Default audio encoding for TTS output. (Default: `'MP3'`)
+
+**sampleRateHertz** (`number`): Default sample rate for TTS output. (Default: `48000`)
+
+**language** (`string`): Default BCP-47 language code for STT. (Default: `'en-US'`)
+
+### Methods
+
+#### `speak(input, options?)`
+
+Converts text to speech using Inworld's streaming TTS endpoint. Returns a readable stream that emits audio chunks progressively as they arrive.
+
+```typescript
+const audioStream = await voice.speak('Hello, world!', {
+  speaker: 'Olivia',
+  audioEncoding: 'WAV',
+  sampleRateHertz: 24000,
+  speakingRate: 1.2,
+  temperature: 0.8,
+})
+```
+
+**input** (`string | NodeJS.ReadableStream`): Text to convert to speech. If a stream is provided, it will be converted to text first.
+
+**options** (`InworldSpeakOptions`): Additional options for speech synthesis.
+
+**options.speaker** (`string`): Override the default speaker for this request.
+
+**options.audioEncoding** (`AudioEncoding`): Override the default audio encoding.
+
+**options.sampleRateHertz** (`number`): Override the default sample rate.
+
+**options.speakingRate** (`number`): Adjust the speaking rate.
+
+**options.temperature** (`number`): Controls voice variability. Honored on inworld-tts-1.5-\* models; ignored by inworld-tts-2.
+
+**options.deliveryMode** (`'STABLE' | 'BALANCED' | 'CREATIVE'`): Steering control for delivery style. Only honored by inworld-tts-2.
+
+**options.language** (`string`): BCP-47 language code for this request. Auto-detected when omitted.
+
+**Returns:** `Promise<NodeJS.ReadableStream>`
+
+#### `listen(input, options?)`
+
+Converts speech to text using Inworld's batch STT endpoint.
+
+```typescript
+const transcript = await voice.listen(audioStream, {
+  audioEncoding: 'MP3',
+  sampleRateHertz: 44100,
+  language: 'ja-JP',
+})
+```
+
+**input** (`NodeJS.ReadableStream`): Audio stream to transcribe.
+
+**options** (`InworldListenOptions`): Additional options for transcription.
+
+**options.audioEncoding** (`'LINEAR16' | 'MP3' | 'OGG_OPUS' | 'FLAC' | 'AUTO_DETECT'`): Audio encoding of the input stream.
+
+**options.sampleRateHertz** (`number`): Sample rate of the input audio.
+
+**options.language** (`string`): BCP-47 language code for transcription.
+
+**options.numberOfChannels** (`number`): Number of audio channels in the input.
+
+**Returns:** `Promise<string>`
+
+#### `getSpeakers()`
+
+Returns a list of available voices from the Inworld API.
+
+```typescript
+const speakers = await voice.getSpeakers()
+// [{ voiceId: 'Dennis', name: 'Dennis', language: 'en', description: '...', tags: ['friendly'], source: 'SYSTEM' }, ...]
+```
+
+**Returns:** `Promise<Array<{ voiceId: string; name: string; language: string; description: string; tags: string[]; source: string }>>`
+
+### Notes
+
+- The TTS endpoint uses progressive NDJSON streaming, so audio playback can begin before the full response is received.
+- An API key can be provided via the `speechModel` or `listeningModel` config, or the `INWORLD_API_KEY` environment variable. TTS and STT keys are resolved independently: passing distinct `speechModel.apiKey` and `listeningModel.apiKey` values lets each service use its own credential. If only one is provided, it's reused for both services as a fallback before the env var.
+- `inworld-tts-2` is the default flagship model. Use `deliveryMode` (`STABLE` | `BALANCED` | `CREATIVE`) to steer delivery style on this model. The `temperature` option is ignored on `inworld-tts-2`.
+- The `inworld-tts-1.5-mini` model offers lower latency at the cost of reduced voice quality compared to `inworld-tts-1.5-max`.
+
+## Realtime
 
 The `InworldRealtimeVoice` class provides real-time, full-duplex voice interaction using [Inworld AI's Realtime API](https://docs.inworld.ai/realtime/quickstart-websocket) over WebSockets. It supports speech-to-speech, tool calling, and Inworld-specific session knobs such as semantic voice activity detection, MCP tool routing, and playback speed.
 
 Inworld's wire protocol is the OpenAI Realtime GA spec, so client and server event names match `@mastra/voice-openai-realtime`. The provider-level differences are the endpoint (which uses a client-generated session key in the URL), the `Authorization: Basic <key>` header, the typed `session` constructor field for Inworld-specific knobs, and a typed `providerData` object for Inworld extensions (STT, TTS, memory, back-channel, responsiveness) sent under `session.providerData`.
 
-For batch text-to-speech and speech-to-text, see [`@mastra/voice-inworld`](https://mastra.ai/reference/voice/inworld).
+For batch text-to-speech and speech-to-text, see [`@mastra/voice-inworld`](#voice).
 
-## Usage example
+### Usage example
 
 ```typescript
 import { InworldRealtimeVoice } from '@mastra/voice-inworld'
@@ -58,7 +196,7 @@ voice.close()
 
 > Inworld API keys are pre-Basic-encoded. Paste them verbatim into `INWORLD_API_KEY`. The package doesn't re-encode them.
 
-## Constructor parameters
+### Constructor parameters
 
 **apiKey** (`string`): Inworld API key. Falls back to the INWORLD\_API\_KEY environment variable. Keys are Basic-encoded and passed verbatim in the Authorization header.
 
@@ -80,7 +218,7 @@ voice.close()
 
 **connectTimeoutMs** (`number`): Max time connect() will wait for both the WebSocket handshake and the initial session.updated round-trip. A pre-open error or close on the WebSocket — or this timeout expiring — surfaces as a rejected promise instead of an uncaught socket error. (Default: `15000`)
 
-### `session` (typed knobs)
+#### `session` (typed knobs)
 
 Use the typed `session` field for documented Inworld realtime options. Fields compose with the connect-time defaults (e.g. `audio.output.voice` set from `speaker`):
 
@@ -116,7 +254,7 @@ Use the typed `session` field for documented Inworld realtime options. Fields co
 
 **prompt** (`string | null`): Reference to a server-side prompt template. Pass null to clear it.
 
-### `providerData` (Inworld extensions)
+#### `providerData` (Inworld extensions)
 
 `providerData` is a typed object for Inworld-specific realtime extensions. It's sent under `session.providerData` on every `session.update`, and composes with any `session.providerData` you set via the `session` field: the constructor `providerData` wins on key collisions.
 
@@ -141,9 +279,9 @@ const voice = new InworldRealtimeVoice({
 })
 ```
 
-## Methods
+### Methods
 
-### `connect()`
+#### `connect()`
 
 Opens the WebSocket connection, sends the initial `session.update`, and resolves once the server acknowledges with `session.updated`. Must be called before `speak()`, `listen()`, or `send()`.
 
@@ -155,7 +293,7 @@ await voice.connect()
 
 Returns: `Promise<void>`
 
-### `speak()`
+#### `speak()`
 
 Sends a text message to the model and triggers an audio response. The returned promise resolves only after the full response lifecycle completes (`response.done` for the response this call triggered), and rejects if the response is interrupted by user speech or if a transport error occurs.
 
@@ -169,7 +307,7 @@ Serial `speak()` calls are the supported pattern. Concurrent calls share the sam
 
 Returns: `Promise<void>`
 
-### `listen()`
+#### `listen()`
 
 Sends a single audio buffer as a user turn and asks the model to respond with text only.
 
@@ -177,7 +315,7 @@ Sends a single audio buffer as a user turn and asks the model to respond with te
 
 Returns: `Promise<void>`
 
-### `send()`
+#### `send()`
 
 Streams audio data to the server in real time. Useful for continuous microphone input.
 
@@ -187,7 +325,7 @@ Streams audio data to the server in real time. Useful for continuous microphone
 
 Returns: `Promise<void>`
 
-### `updateConfig()`
+#### `updateConfig()`
 
 Sends a `session.update` to the server. The typed `session` field is deep-merged into the payload, and any constructor `providerData` is nested under `session.providerData`.
 
@@ -195,7 +333,7 @@ Sends a `session.update` to the server. The typed `session` field is deep-merged
 
 Returns: `void`
 
-### `addInstructions()`
+#### `addInstructions()`
 
 Sets the system instructions used on the next `connect()` or `updateConfig()` call.
 
@@ -203,7 +341,7 @@ Sets the system instructions used on the next `connect()` or `updateConfig()` ca
 
 Returns: `void`
 
-### `addTools()`
+#### `addTools()`
 
 Registers tools that the model can call during the session. When `InworldRealtimeVoice` is attached to an Agent, tools configured for the Agent are made available automatically.
 
@@ -211,7 +349,7 @@ Registers tools that the model can call during the session. When `InworldRealtim
 
 Returns: `void`
 
-### `answer()`
+#### `answer()`
 
 Sends a `response.create` event to trigger a model response, optionally with per-response options.
 
@@ -219,9 +357,9 @@ Sends a `response.create` event to trigger a model response, optionally with per
 
 Returns: `Promise<void>`
 
-### Turn-taking
+#### Turn-taking
 
-#### `commitInput()`
+##### `commitInput()`
 
 Manually commits buffered input audio as a user turn. Use this for push-to-talk or manual turn-taking when `turn_detection` is set to `null`.
 
@@ -231,7 +369,7 @@ voice.commitInput()
 
 Returns: `void`
 
-#### `clearInput()`
+##### `clearInput()`
 
 Discards buffered input audio without committing it as a user turn.
 
@@ -241,7 +379,7 @@ voice.clearInput()
 
 Returns: `void`
 
-#### `clearOutput()`
+##### `clearOutput()`
 
 Clears the server's entire output audio buffer, stopping playback. This also stops any in-flight back-channel audio. The default barge-in path (`response.cancel` on `interrupted`) is back-channel-safe. Prefer it. Use `clearOutput()` only when you want to flush everything.
 
@@ -251,23 +389,23 @@ voice.clearOutput()
 
 Returns: `void`
 
-### `close()` and `disconnect()`
+#### `close()` and `disconnect()`
 
 Both methods close the WebSocket and mark the instance as disconnected.
 
 Returns: `void`
 
-### `getSpeakers()`
+#### `getSpeakers()`
 
 Returns the curated voice list bundled with the package. Inworld's catalog is larger than this list; any voice ID can be passed to `speaker` at runtime.
 
 Returns: `Promise<Array<{ voiceId: string }>>`
 
-### `on()` and `off()`
+#### `on()` and `off()`
 
 Register and remove event listeners. See [Events](#events) below.
 
-## Events
+### Events
 
 The `InworldRealtimeVoice` class emits the following events:
 
@@ -325,7 +463,7 @@ The `InworldRealtimeVoice` class emits the following events:
 
 **error** (`event`): Emitted on transport or server errors.
 
-## Voices
+### Voices
 
 The package includes a curated set of voice IDs returned from `getSpeakers()`:
 
@@ -342,7 +480,7 @@ The package includes a curated set of voice IDs returned from `getSpeakers()`:
 
 Any voice ID from [Inworld's voice catalog](https://docs.inworld.ai/quickstart-tts) can be passed to `speaker` at runtime.
 
-## Notes
+### Notes
 
 - API keys can be provided via constructor options or the `INWORLD_API_KEY` environment variable. Keys are pre-Basic-encoded. Don't re-encode them.
 - The WebSocket URL appends `?key=<sessionId>&protocol=realtime`. The model is configured via the initial `session.update`, not the URL.