@mastra/voice-openai 0.12.0 → 0.12.1-alpha.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/dist/docs/SKILL.md +21 -21
- package/dist/docs/{SOURCE_MAP.json → assets/SOURCE_MAP.json} +1 -1
- package/dist/docs/{agents/01-adding-voice.md → references/docs-agents-adding-voice.md} +18 -17
- package/dist/docs/{voice/01-overview.md → references/docs-voice-overview.md} +79 -119
- package/dist/docs/{voice/03-speech-to-text.md → references/docs-voice-speech-to-text.md} +10 -13
- package/dist/docs/{voice/02-text-to-speech.md → references/docs-voice-text-to-speech.md} +14 -17
- package/dist/docs/references/reference-voice-composite-voice.md +121 -0
- package/dist/docs/references/reference-voice-openai.md +88 -0
- package/dist/docs/references/reference-voice-voice.getSpeakers.md +129 -0
- package/dist/docs/references/reference-voice-voice.listen.md +164 -0
- package/dist/docs/references/reference-voice-voice.speak.md +157 -0
- package/package.json +7 -8
- package/dist/docs/README.md +0 -32
- package/dist/docs/voice/04-reference.md +0 -568
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# @mastra/voice-openai
|
|
2
2
|
|
|
3
|
+
## 0.12.1-alpha.0
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- dependencies updates: ([#13189](https://github.com/mastra-ai/mastra/pull/13189))
|
|
8
|
+
- Updated dependency [`openai@^5.23.2` ↗︎](https://www.npmjs.com/package/openai/v/5.23.2) (from `^5.11.0`, in `dependencies`)
|
|
9
|
+
- Updated dependencies:
|
|
10
|
+
- @mastra/core@1.5.0-alpha.1
|
|
11
|
+
|
|
3
12
|
## 0.12.0
|
|
4
13
|
|
|
5
14
|
### Minor Changes
|
package/dist/docs/SKILL.md
CHANGED
|
@@ -1,33 +1,33 @@
|
|
|
1
1
|
---
|
|
2
|
-
name: mastra-voice-openai
|
|
3
|
-
description: Documentation for @mastra/voice-openai.
|
|
2
|
+
name: mastra-voice-openai
|
|
3
|
+
description: Documentation for @mastra/voice-openai. Use when working with @mastra/voice-openai APIs, configuration, or implementation.
|
|
4
|
+
metadata:
|
|
5
|
+
package: "@mastra/voice-openai"
|
|
6
|
+
version: "0.12.1-alpha.0"
|
|
4
7
|
---
|
|
5
8
|
|
|
6
|
-
|
|
9
|
+
## When to use
|
|
7
10
|
|
|
8
|
-
|
|
9
|
-
> **Package**: @mastra/voice-openai
|
|
11
|
+
Use this skill whenever you are working with @mastra/voice-openai to obtain the domain-specific knowledge.
|
|
10
12
|
|
|
11
|
-
##
|
|
13
|
+
## How to use
|
|
12
14
|
|
|
13
|
-
|
|
15
|
+
Read the individual reference documents for detailed explanations and code examples.
|
|
14
16
|
|
|
15
|
-
|
|
16
|
-
cat docs/SOURCE_MAP.json
|
|
17
|
-
```
|
|
17
|
+
### Docs
|
|
18
18
|
|
|
19
|
-
|
|
20
|
-
-
|
|
21
|
-
-
|
|
22
|
-
-
|
|
19
|
+
- [Voice](references/docs-agents-adding-voice.md)
|
|
20
|
+
- [Voice in Mastra](references/docs-voice-overview.md) - Overview of voice capabilities in Mastra, including text-to-speech, speech-to-text, and real-time speech-to-speech interactions.
|
|
21
|
+
- [Speech-to-Text (STT)](references/docs-voice-speech-to-text.md) - Overview of Speech-to-Text capabilities in Mastra, including configuration, usage, and integration with voice providers.
|
|
22
|
+
- [Text-to-Speech (TTS)](references/docs-voice-text-to-speech.md) - Overview of Text-to-Speech capabilities in Mastra, including configuration, usage, and integration with voice providers.
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
### Reference
|
|
25
25
|
|
|
26
|
+
- [Reference: CompositeVoice](references/reference-voice-composite-voice.md) - Documentation for the CompositeVoice class, which enables combining multiple voice providers for flexible text-to-speech and speech-to-text operations.
|
|
27
|
+
- [Reference: OpenAI](references/reference-voice-openai.md) - Documentation for the OpenAIVoice class, providing text-to-speech and speech-to-text capabilities.
|
|
28
|
+
- [Reference: voice.getSpeakers()](references/reference-voice-voice.getSpeakers.md) - Documentation for the getSpeakers() method available in voice providers, which retrieves available voice options.
|
|
29
|
+
- [Reference: voice.listen()](references/reference-voice-voice.listen.md) - Documentation for the listen() method available in all Mastra voice providers, which converts speech to text.
|
|
30
|
+
- [Reference: voice.speak()](references/reference-voice-voice.speak.md) - Documentation for the speak() method available in all Mastra voice providers, which converts text to speech.
|
|
26
31
|
|
|
27
32
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
## Available Topics
|
|
31
|
-
|
|
32
|
-
- [Agents](agents/) - 1 file(s)
|
|
33
|
-
- [Voice](voice/) - 8 file(s)
|
|
33
|
+
Read [assets/SOURCE_MAP.json](assets/SOURCE_MAP.json) for source code references.
|
|
@@ -164,7 +164,7 @@ The following files are created:
|
|
|
164
164
|
- **hybrid-question.mp3** – Hybrid agent's spoken question.
|
|
165
165
|
- **unified-response.mp3** – Unified agent's spoken response.
|
|
166
166
|
|
|
167
|
-
```typescript
|
|
167
|
+
```typescript
|
|
168
168
|
import "dotenv/config";
|
|
169
169
|
|
|
170
170
|
import path from "path";
|
|
@@ -323,30 +323,31 @@ const voice = new CompositeVoice({
|
|
|
323
323
|
```
|
|
324
324
|
|
|
325
325
|
For the complete list of supported AI SDK providers and their capabilities:
|
|
326
|
-
|
|
327
|
-
|
|
326
|
+
|
|
327
|
+
- [Transcription](https://ai-sdk.dev/docs/providers/openai/transcription)
|
|
328
|
+
- [Speech](https://ai-sdk.dev/docs/providers/elevenlabs/speech)
|
|
328
329
|
|
|
329
330
|
## Supported Voice Providers
|
|
330
331
|
|
|
331
332
|
Mastra supports multiple voice providers for text-to-speech (TTS) and speech-to-text (STT) capabilities:
|
|
332
333
|
|
|
333
|
-
| Provider | Package | Features | Reference
|
|
334
|
-
| --------------- | ------------------------------- | ------------------------- |
|
|
335
|
-
| OpenAI | `@mastra/voice-openai` | TTS, STT | [Documentation](https://mastra.ai/reference/
|
|
336
|
-
| OpenAI Realtime | `@mastra/voice-openai-realtime` | Realtime speech-to-speech | [Documentation](https://mastra.ai/reference/
|
|
337
|
-
| ElevenLabs | `@mastra/voice-elevenlabs` | High-quality TTS | [Documentation](https://mastra.ai/reference/
|
|
338
|
-
| PlayAI | `@mastra/voice-playai` | TTS | [Documentation](https://mastra.ai/reference/
|
|
339
|
-
| Google | `@mastra/voice-google` | TTS, STT | [Documentation](https://mastra.ai/reference/
|
|
340
|
-
| Deepgram | `@mastra/voice-deepgram` | STT | [Documentation](https://mastra.ai/reference/
|
|
341
|
-
| Murf | `@mastra/voice-murf` | TTS | [Documentation](https://mastra.ai/reference/
|
|
342
|
-
| Speechify | `@mastra/voice-speechify` | TTS | [Documentation](https://mastra.ai/reference/
|
|
343
|
-
| Sarvam | `@mastra/voice-sarvam` | TTS, STT | [Documentation](https://mastra.ai/reference/
|
|
344
|
-
| Azure | `@mastra/voice-azure` | TTS, STT | [Documentation](https://mastra.ai/reference/
|
|
345
|
-
| Cloudflare | `@mastra/voice-cloudflare` | TTS | [Documentation](https://mastra.ai/reference/
|
|
334
|
+
| Provider | Package | Features | Reference |
|
|
335
|
+
| --------------- | ------------------------------- | ------------------------- | ------------------------------------------------------------------ |
|
|
336
|
+
| OpenAI | `@mastra/voice-openai` | TTS, STT | [Documentation](https://mastra.ai/reference/voice/openai) |
|
|
337
|
+
| OpenAI Realtime | `@mastra/voice-openai-realtime` | Realtime speech-to-speech | [Documentation](https://mastra.ai/reference/voice/openai-realtime) |
|
|
338
|
+
| ElevenLabs | `@mastra/voice-elevenlabs` | High-quality TTS | [Documentation](https://mastra.ai/reference/voice/elevenlabs) |
|
|
339
|
+
| PlayAI | `@mastra/voice-playai` | TTS | [Documentation](https://mastra.ai/reference/voice/playai) |
|
|
340
|
+
| Google | `@mastra/voice-google` | TTS, STT | [Documentation](https://mastra.ai/reference/voice/google) |
|
|
341
|
+
| Deepgram | `@mastra/voice-deepgram` | STT | [Documentation](https://mastra.ai/reference/voice/deepgram) |
|
|
342
|
+
| Murf | `@mastra/voice-murf` | TTS | [Documentation](https://mastra.ai/reference/voice/murf) |
|
|
343
|
+
| Speechify | `@mastra/voice-speechify` | TTS | [Documentation](https://mastra.ai/reference/voice/speechify) |
|
|
344
|
+
| Sarvam | `@mastra/voice-sarvam` | TTS, STT | [Documentation](https://mastra.ai/reference/voice/sarvam) |
|
|
345
|
+
| Azure | `@mastra/voice-azure` | TTS, STT | [Documentation](https://mastra.ai/reference/voice/mastra-voice) |
|
|
346
|
+
| Cloudflare | `@mastra/voice-cloudflare` | TTS | [Documentation](https://mastra.ai/reference/voice/mastra-voice) |
|
|
346
347
|
|
|
347
348
|
## Next Steps
|
|
348
349
|
|
|
349
|
-
- [Voice API Reference](https://mastra.ai/reference/
|
|
350
|
+
- [Voice API Reference](https://mastra.ai/reference/voice/mastra-voice) - Detailed API documentation for voice capabilities
|
|
350
351
|
- [Text to Speech Examples](https://github.com/mastra-ai/voice-examples/tree/main/text-to-speech) - Interactive story generator and other TTS implementations
|
|
351
352
|
- [Speech to Text Examples](https://github.com/mastra-ai/voice-examples/tree/main/speech-to-text) - Voice memo app and other STT implementations
|
|
352
353
|
- [Speech to Speech Examples](https://github.com/mastra-ai/voice-examples/tree/main/speech-to-speech) - Real-time voice conversation with call analysis
|
|
@@ -1,12 +1,10 @@
|
|
|
1
|
-
> Overview of voice capabilities in Mastra, including text-to-speech, speech-to-text, and real-time speech-to-speech interactions.
|
|
2
|
-
|
|
3
1
|
# Voice in Mastra
|
|
4
2
|
|
|
5
3
|
Mastra's Voice system provides a unified interface for voice interactions, enabling text-to-speech (TTS), speech-to-text (STT), and real-time speech-to-speech (STS) capabilities in your applications.
|
|
6
4
|
|
|
7
5
|
## Adding Voice to Agents
|
|
8
6
|
|
|
9
|
-
To learn how to integrate voice capabilities into your agents, check out the [Adding Voice to Agents](https://mastra.ai/docs/
|
|
7
|
+
To learn how to integrate voice capabilities into your agents, check out the [Adding Voice to Agents](https://mastra.ai/docs/agents/adding-voice) documentation. This section covers how to use both single and multiple voice providers, as well as real-time interactions.
|
|
10
8
|
|
|
11
9
|
```typescript
|
|
12
10
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -28,12 +26,11 @@ You can then use the following voice capabilities:
|
|
|
28
26
|
|
|
29
27
|
### Text to Speech (TTS)
|
|
30
28
|
|
|
31
|
-
Turn your agent's responses into natural-sounding speech using Mastra's TTS capabilities.
|
|
32
|
-
Choose from multiple providers like OpenAI, ElevenLabs, and more.
|
|
29
|
+
Turn your agent's responses into natural-sounding speech using Mastra's TTS capabilities. Choose from multiple providers like OpenAI, ElevenLabs, and more.
|
|
33
30
|
|
|
34
|
-
For detailed configuration options and advanced features, check out our [Text-to-Speech guide](
|
|
31
|
+
For detailed configuration options and advanced features, check out our [Text-to-Speech guide](https://mastra.ai/docs/voice/text-to-speech).
|
|
35
32
|
|
|
36
|
-
|
|
33
|
+
**OpenAI**:
|
|
37
34
|
|
|
38
35
|
```typescript
|
|
39
36
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -60,10 +57,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
60
57
|
playAudio(audioStream);
|
|
61
58
|
```
|
|
62
59
|
|
|
63
|
-
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/
|
|
60
|
+
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/voice/openai) for more information on the OpenAI voice provider.
|
|
64
61
|
|
|
65
|
-
|
|
66
|
-
**azure:**
|
|
62
|
+
**Azure**:
|
|
67
63
|
|
|
68
64
|
```typescript
|
|
69
65
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -89,10 +85,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
89
85
|
playAudio(audioStream);
|
|
90
86
|
```
|
|
91
87
|
|
|
92
|
-
Visit the [Azure Voice Reference](https://mastra.ai/reference/
|
|
88
|
+
Visit the [Azure Voice Reference](https://mastra.ai/reference/voice/azure) for more information on the Azure voice provider.
|
|
93
89
|
|
|
94
|
-
|
|
95
|
-
**elevenlabs:**
|
|
90
|
+
**ElevenLabs**:
|
|
96
91
|
|
|
97
92
|
```typescript
|
|
98
93
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -118,10 +113,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
118
113
|
playAudio(audioStream);
|
|
119
114
|
```
|
|
120
115
|
|
|
121
|
-
Visit the [ElevenLabs Voice Reference](https://mastra.ai/reference/
|
|
116
|
+
Visit the [ElevenLabs Voice Reference](https://mastra.ai/reference/voice/elevenlabs) for more information on the ElevenLabs voice provider.
|
|
122
117
|
|
|
123
|
-
|
|
124
|
-
**playai:**
|
|
118
|
+
**PlayAI**:
|
|
125
119
|
|
|
126
120
|
```typescript
|
|
127
121
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -147,10 +141,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
147
141
|
playAudio(audioStream);
|
|
148
142
|
```
|
|
149
143
|
|
|
150
|
-
Visit the [PlayAI Voice Reference](https://mastra.ai/reference/
|
|
144
|
+
Visit the [PlayAI Voice Reference](https://mastra.ai/reference/voice/playai) for more information on the PlayAI voice provider.
|
|
151
145
|
|
|
152
|
-
|
|
153
|
-
**google:**
|
|
146
|
+
**Google**:
|
|
154
147
|
|
|
155
148
|
```typescript
|
|
156
149
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -176,10 +169,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
176
169
|
playAudio(audioStream);
|
|
177
170
|
```
|
|
178
171
|
|
|
179
|
-
Visit the [Google Voice Reference](https://mastra.ai/reference/
|
|
172
|
+
Visit the [Google Voice Reference](https://mastra.ai/reference/voice/google) for more information on the Google voice provider.
|
|
180
173
|
|
|
181
|
-
|
|
182
|
-
**cloudflare:**
|
|
174
|
+
**Cloudflare**:
|
|
183
175
|
|
|
184
176
|
```typescript
|
|
185
177
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -205,10 +197,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
205
197
|
playAudio(audioStream);
|
|
206
198
|
```
|
|
207
199
|
|
|
208
|
-
Visit the [Cloudflare Voice Reference](https://mastra.ai/reference/
|
|
200
|
+
Visit the [Cloudflare Voice Reference](https://mastra.ai/reference/voice/cloudflare) for more information on the Cloudflare voice provider.
|
|
209
201
|
|
|
210
|
-
|
|
211
|
-
**deepgram:**
|
|
202
|
+
**Deepgram**:
|
|
212
203
|
|
|
213
204
|
```typescript
|
|
214
205
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -234,10 +225,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
234
225
|
playAudio(audioStream);
|
|
235
226
|
```
|
|
236
227
|
|
|
237
|
-
Visit the [Deepgram Voice Reference](https://mastra.ai/reference/
|
|
228
|
+
Visit the [Deepgram Voice Reference](https://mastra.ai/reference/voice/deepgram) for more information on the Deepgram voice provider.
|
|
238
229
|
|
|
239
|
-
|
|
240
|
-
**speechify:**
|
|
230
|
+
**Speechify**:
|
|
241
231
|
|
|
242
232
|
```typescript
|
|
243
233
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -263,10 +253,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
263
253
|
playAudio(audioStream);
|
|
264
254
|
```
|
|
265
255
|
|
|
266
|
-
Visit the [Speechify Voice Reference](https://mastra.ai/reference/
|
|
256
|
+
Visit the [Speechify Voice Reference](https://mastra.ai/reference/voice/speechify) for more information on the Speechify voice provider.
|
|
267
257
|
|
|
268
|
-
|
|
269
|
-
**sarvam:**
|
|
258
|
+
**Sarvam**:
|
|
270
259
|
|
|
271
260
|
```typescript
|
|
272
261
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -292,10 +281,9 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
292
281
|
playAudio(audioStream);
|
|
293
282
|
```
|
|
294
283
|
|
|
295
|
-
Visit the [Sarvam Voice Reference](https://mastra.ai/reference/
|
|
284
|
+
Visit the [Sarvam Voice Reference](https://mastra.ai/reference/voice/sarvam) for more information on the Sarvam voice provider.
|
|
296
285
|
|
|
297
|
-
|
|
298
|
-
**murf:**
|
|
286
|
+
**Murf**:
|
|
299
287
|
|
|
300
288
|
```typescript
|
|
301
289
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -321,20 +309,17 @@ const audioStream = await voiceAgent.voice.speak(text, {
|
|
|
321
309
|
playAudio(audioStream);
|
|
322
310
|
```
|
|
323
311
|
|
|
324
|
-
Visit the [Murf Voice Reference](https://mastra.ai/reference/
|
|
325
|
-
|
|
326
|
-
|
|
312
|
+
Visit the [Murf Voice Reference](https://mastra.ai/reference/voice/murf) for more information on the Murf voice provider.
|
|
327
313
|
|
|
328
314
|
### Speech to Text (STT)
|
|
329
315
|
|
|
330
|
-
Transcribe spoken content using various providers like OpenAI, ElevenLabs, and more. For detailed configuration options and more, check out [Speech to Text](
|
|
316
|
+
Transcribe spoken content using various providers like OpenAI, ElevenLabs, and more. For detailed configuration options and more, check out [Speech to Text](https://mastra.ai/docs/voice/speech-to-text).
|
|
331
317
|
|
|
332
318
|
You can download a sample audio file from [here](https://github.com/mastra-ai/realtime-voice-demo/raw/refs/heads/main/how_can_i_help_you.mp3).
|
|
333
319
|
|
|
334
|
-
|
|
335
|
-
<AudioPlayback audio="https://github.com/mastra-ai/realtime-voice-demo/raw/refs/heads/main/how_can_i_help_you.mp3" />
|
|
320
|
+
[](https://github.com/mastra-ai/realtime-voice-demo/raw/refs/heads/main/how_can_i_help_you.mp3)
|
|
336
321
|
|
|
337
|
-
|
|
322
|
+
**OpenAI**:
|
|
338
323
|
|
|
339
324
|
```typescript
|
|
340
325
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -361,10 +346,9 @@ console.log(`User said: ${transcript}`);
|
|
|
361
346
|
const { text } = await voiceAgent.generate(transcript);
|
|
362
347
|
```
|
|
363
348
|
|
|
364
|
-
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/
|
|
349
|
+
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/voice/openai) for more information on the OpenAI voice provider.
|
|
365
350
|
|
|
366
|
-
|
|
367
|
-
**azure:**
|
|
351
|
+
**Azure**:
|
|
368
352
|
|
|
369
353
|
```typescript
|
|
370
354
|
import { createReadStream } from "fs";
|
|
@@ -392,10 +376,9 @@ console.log(`User said: ${transcript}`);
|
|
|
392
376
|
const { text } = await voiceAgent.generate(transcript);
|
|
393
377
|
```
|
|
394
378
|
|
|
395
|
-
Visit the [Azure Voice Reference](https://mastra.ai/reference/
|
|
379
|
+
Visit the [Azure Voice Reference](https://mastra.ai/reference/voice/azure) for more information on the Azure voice provider.
|
|
396
380
|
|
|
397
|
-
|
|
398
|
-
**elevenlabs:**
|
|
381
|
+
**ElevenLabs**:
|
|
399
382
|
|
|
400
383
|
```typescript
|
|
401
384
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -422,10 +405,9 @@ console.log(`User said: ${transcript}`);
|
|
|
422
405
|
const { text } = await voiceAgent.generate(transcript);
|
|
423
406
|
```
|
|
424
407
|
|
|
425
|
-
Visit the [ElevenLabs Voice Reference](https://mastra.ai/reference/
|
|
408
|
+
Visit the [ElevenLabs Voice Reference](https://mastra.ai/reference/voice/elevenlabs) for more information on the ElevenLabs voice provider.
|
|
426
409
|
|
|
427
|
-
|
|
428
|
-
**google:**
|
|
410
|
+
**Google**:
|
|
429
411
|
|
|
430
412
|
```typescript
|
|
431
413
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -452,10 +434,9 @@ console.log(`User said: ${transcript}`);
|
|
|
452
434
|
const { text } = await voiceAgent.generate(transcript);
|
|
453
435
|
```
|
|
454
436
|
|
|
455
|
-
Visit the [Google Voice Reference](https://mastra.ai/reference/
|
|
437
|
+
Visit the [Google Voice Reference](https://mastra.ai/reference/voice/google) for more information on the Google voice provider.
|
|
456
438
|
|
|
457
|
-
|
|
458
|
-
**cloudflare:**
|
|
439
|
+
**Cloudflare**:
|
|
459
440
|
|
|
460
441
|
```typescript
|
|
461
442
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -482,10 +463,9 @@ console.log(`User said: ${transcript}`);
|
|
|
482
463
|
const { text } = await voiceAgent.generate(transcript);
|
|
483
464
|
```
|
|
484
465
|
|
|
485
|
-
Visit the [Cloudflare Voice Reference](https://mastra.ai/reference/
|
|
466
|
+
Visit the [Cloudflare Voice Reference](https://mastra.ai/reference/voice/cloudflare) for more information on the Cloudflare voice provider.
|
|
486
467
|
|
|
487
|
-
|
|
488
|
-
**deepgram:**
|
|
468
|
+
**Deepgram**:
|
|
489
469
|
|
|
490
470
|
```typescript
|
|
491
471
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -512,10 +492,9 @@ console.log(`User said: ${transcript}`);
|
|
|
512
492
|
const { text } = await voiceAgent.generate(transcript);
|
|
513
493
|
```
|
|
514
494
|
|
|
515
|
-
Visit the [Deepgram Voice Reference](https://mastra.ai/reference/
|
|
495
|
+
Visit the [Deepgram Voice Reference](https://mastra.ai/reference/voice/deepgram) for more information on the Deepgram voice provider.
|
|
516
496
|
|
|
517
|
-
|
|
518
|
-
**sarvam:**
|
|
497
|
+
**Sarvam**:
|
|
519
498
|
|
|
520
499
|
```typescript
|
|
521
500
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -542,16 +521,13 @@ console.log(`User said: ${transcript}`);
|
|
|
542
521
|
const { text } = await voiceAgent.generate(transcript);
|
|
543
522
|
```
|
|
544
523
|
|
|
545
|
-
Visit the [Sarvam Voice Reference](https://mastra.ai/reference/
|
|
546
|
-
|
|
547
|
-
|
|
524
|
+
Visit the [Sarvam Voice Reference](https://mastra.ai/reference/voice/sarvam) for more information on the Sarvam voice provider.
|
|
548
525
|
|
|
549
526
|
### Speech to Speech (STS)
|
|
550
527
|
|
|
551
|
-
Create conversational experiences with speech-to-speech capabilities. The unified API enables real-time voice interactions between users and AI agents.
|
|
552
|
-
For detailed configuration options and advanced features, check out [Speech to Speech](./speech-to-speech).
|
|
528
|
+
Create conversational experiences with speech-to-speech capabilities. The unified API enables real-time voice interactions between users and AI agents. For detailed configuration options and advanced features, check out [Speech to Speech](https://mastra.ai/docs/voice/speech-to-speech).
|
|
553
529
|
|
|
554
|
-
|
|
530
|
+
**OpenAI**:
|
|
555
531
|
|
|
556
532
|
```typescript
|
|
557
533
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -580,10 +556,9 @@ const micStream = getMicrophoneStream();
|
|
|
580
556
|
await voiceAgent.voice.send(micStream);
|
|
581
557
|
```
|
|
582
558
|
|
|
583
|
-
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/
|
|
559
|
+
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/voice/openai-realtime) for more information on the OpenAI voice provider.
|
|
584
560
|
|
|
585
|
-
|
|
586
|
-
**google:**
|
|
561
|
+
**Google**:
|
|
587
562
|
|
|
588
563
|
```typescript
|
|
589
564
|
import { Agent } from "@mastra/core/agent";
|
|
@@ -631,15 +606,13 @@ const micStream = getMicrophoneStream();
|
|
|
631
606
|
await voiceAgent.voice.send(micStream);
|
|
632
607
|
```
|
|
633
608
|
|
|
634
|
-
Visit the [Google Gemini Live Reference](https://mastra.ai/reference/
|
|
635
|
-
|
|
636
|
-
|
|
609
|
+
Visit the [Google Gemini Live Reference](https://mastra.ai/reference/voice/google-gemini-live) for more information on the Google Gemini Live voice provider.
|
|
637
610
|
|
|
638
611
|
## Voice Configuration
|
|
639
612
|
|
|
640
613
|
Each voice provider can be configured with different models and options. Below are the detailed configuration options for all supported providers:
|
|
641
614
|
|
|
642
|
-
|
|
615
|
+
**OpenAI**:
|
|
643
616
|
|
|
644
617
|
```typescript
|
|
645
618
|
// OpenAI Voice Configuration
|
|
@@ -660,10 +633,9 @@ const voice = new OpenAIVoice({
|
|
|
660
633
|
});
|
|
661
634
|
```
|
|
662
635
|
|
|
663
|
-
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/
|
|
636
|
+
Visit the [OpenAI Voice Reference](https://mastra.ai/reference/voice/openai) for more information on the OpenAI voice provider.
|
|
664
637
|
|
|
665
|
-
|
|
666
|
-
**azure:**
|
|
638
|
+
**Azure**:
|
|
667
639
|
|
|
668
640
|
```typescript
|
|
669
641
|
// Azure Voice Configuration
|
|
@@ -686,10 +658,9 @@ const voice = new AzureVoice({
|
|
|
686
658
|
});
|
|
687
659
|
```
|
|
688
660
|
|
|
689
|
-
Visit the [Azure Voice Reference](https://mastra.ai/reference/
|
|
661
|
+
Visit the [Azure Voice Reference](https://mastra.ai/reference/voice/azure) for more information on the Azure voice provider.
|
|
690
662
|
|
|
691
|
-
|
|
692
|
-
**elevenlabs:**
|
|
663
|
+
**ElevenLabs**:
|
|
693
664
|
|
|
694
665
|
```typescript
|
|
695
666
|
// ElevenLabs Voice Configuration
|
|
@@ -705,10 +676,9 @@ const voice = new ElevenLabsVoice({
|
|
|
705
676
|
});
|
|
706
677
|
```
|
|
707
678
|
|
|
708
|
-
Visit the [ElevenLabs Voice Reference](https://mastra.ai/reference/
|
|
679
|
+
Visit the [ElevenLabs Voice Reference](https://mastra.ai/reference/voice/elevenlabs) for more information on the ElevenLabs voice provider.
|
|
709
680
|
|
|
710
|
-
|
|
711
|
-
**playai:**
|
|
681
|
+
**PlayAI**:
|
|
712
682
|
|
|
713
683
|
```typescript
|
|
714
684
|
// PlayAI Voice Configuration
|
|
@@ -724,10 +694,9 @@ const voice = new PlayAIVoice({
|
|
|
724
694
|
});
|
|
725
695
|
```
|
|
726
696
|
|
|
727
|
-
Visit the [PlayAI Voice Reference](https://mastra.ai/reference/
|
|
697
|
+
Visit the [PlayAI Voice Reference](https://mastra.ai/reference/voice/playai) for more information on the PlayAI voice provider.
|
|
728
698
|
|
|
729
|
-
|
|
730
|
-
**google:**
|
|
699
|
+
**Google**:
|
|
731
700
|
|
|
732
701
|
```typescript
|
|
733
702
|
// Google Voice Configuration
|
|
@@ -746,10 +715,9 @@ const voice = new GoogleVoice({
|
|
|
746
715
|
});
|
|
747
716
|
```
|
|
748
717
|
|
|
749
|
-
Visit the [Google Voice Reference](https://mastra.ai/reference/
|
|
718
|
+
Visit the [Google Voice Reference](https://mastra.ai/reference/voice/google) for more information on the Google voice provider.
|
|
750
719
|
|
|
751
|
-
|
|
752
|
-
**cloudflare:**
|
|
720
|
+
**Cloudflare**:
|
|
753
721
|
|
|
754
722
|
```typescript
|
|
755
723
|
// Cloudflare Voice Configuration
|
|
@@ -765,10 +733,9 @@ const voice = new CloudflareVoice({
|
|
|
765
733
|
});
|
|
766
734
|
```
|
|
767
735
|
|
|
768
|
-
Visit the [Cloudflare Voice Reference](https://mastra.ai/reference/
|
|
736
|
+
Visit the [Cloudflare Voice Reference](https://mastra.ai/reference/voice/cloudflare) for more information on the Cloudflare voice provider.
|
|
769
737
|
|
|
770
|
-
|
|
771
|
-
**deepgram:**
|
|
738
|
+
**Deepgram**:
|
|
772
739
|
|
|
773
740
|
```typescript
|
|
774
741
|
// Deepgram Voice Configuration
|
|
@@ -787,10 +754,9 @@ const voice = new DeepgramVoice({
|
|
|
787
754
|
});
|
|
788
755
|
```
|
|
789
756
|
|
|
790
|
-
Visit the [Deepgram Voice Reference](https://mastra.ai/reference/
|
|
757
|
+
Visit the [Deepgram Voice Reference](https://mastra.ai/reference/voice/deepgram) for more information on the Deepgram voice provider.
|
|
791
758
|
|
|
792
|
-
|
|
793
|
-
**speechify:**
|
|
759
|
+
**Speechify**:
|
|
794
760
|
|
|
795
761
|
```typescript
|
|
796
762
|
// Speechify Voice Configuration
|
|
@@ -806,10 +772,9 @@ const voice = new SpeechifyVoice({
|
|
|
806
772
|
});
|
|
807
773
|
```
|
|
808
774
|
|
|
809
|
-
Visit the [Speechify Voice Reference](https://mastra.ai/reference/
|
|
775
|
+
Visit the [Speechify Voice Reference](https://mastra.ai/reference/voice/speechify) for more information on the Speechify voice provider.
|
|
810
776
|
|
|
811
|
-
|
|
812
|
-
**sarvam:**
|
|
777
|
+
**Sarvam**:
|
|
813
778
|
|
|
814
779
|
```typescript
|
|
815
780
|
// Sarvam Voice Configuration
|
|
@@ -824,10 +789,9 @@ const voice = new SarvamVoice({
|
|
|
824
789
|
});
|
|
825
790
|
```
|
|
826
791
|
|
|
827
|
-
Visit the [Sarvam Voice Reference](https://mastra.ai/reference/
|
|
792
|
+
Visit the [Sarvam Voice Reference](https://mastra.ai/reference/voice/sarvam) for more information on the Sarvam voice provider.
|
|
828
793
|
|
|
829
|
-
|
|
830
|
-
**murf:**
|
|
794
|
+
**Murf**:
|
|
831
795
|
|
|
832
796
|
```typescript
|
|
833
797
|
// Murf Voice Configuration
|
|
@@ -842,10 +806,9 @@ const voice = new MurfVoice({
|
|
|
842
806
|
});
|
|
843
807
|
```
|
|
844
808
|
|
|
845
|
-
Visit the [Murf Voice Reference](https://mastra.ai/reference/
|
|
809
|
+
Visit the [Murf Voice Reference](https://mastra.ai/reference/voice/murf) for more information on the Murf voice provider.
|
|
846
810
|
|
|
847
|
-
|
|
848
|
-
**openai-realtime:**
|
|
811
|
+
**OpenAI Realtime**:
|
|
849
812
|
|
|
850
813
|
```typescript
|
|
851
814
|
// OpenAI Realtime Voice Configuration
|
|
@@ -864,10 +827,9 @@ const voice = new OpenAIRealtimeVoice({
|
|
|
864
827
|
});
|
|
865
828
|
```
|
|
866
829
|
|
|
867
|
-
For more information on the OpenAI Realtime voice provider, refer to the [OpenAI Realtime Voice Reference](https://mastra.ai/reference/
|
|
830
|
+
For more information on the OpenAI Realtime voice provider, refer to the [OpenAI Realtime Voice Reference](https://mastra.ai/reference/voice/openai-realtime).
|
|
868
831
|
|
|
869
|
-
|
|
870
|
-
**google-gemini-live:**
|
|
832
|
+
**Google Gemini Live**:
|
|
871
833
|
|
|
872
834
|
```typescript
|
|
873
835
|
// Google Gemini Live Voice Configuration
|
|
@@ -881,10 +843,9 @@ const voice = new GeminiLiveVoice({
|
|
|
881
843
|
});
|
|
882
844
|
```
|
|
883
845
|
|
|
884
|
-
Visit the [Google Gemini Live Reference](https://mastra.ai/reference/
|
|
846
|
+
Visit the [Google Gemini Live Reference](https://mastra.ai/reference/voice/google-gemini-live) for more information on the Google Gemini Live voice provider.
|
|
885
847
|
|
|
886
|
-
|
|
887
|
-
**aisdk:**
|
|
848
|
+
**AI SDK**:
|
|
888
849
|
|
|
889
850
|
```typescript
|
|
890
851
|
// AI SDK Voice Configuration
|
|
@@ -907,7 +868,6 @@ const voiceAgent = new Agent({
|
|
|
907
868
|
voice,
|
|
908
869
|
});
|
|
909
870
|
```
|
|
910
|
-
|
|
911
871
|
|
|
912
872
|
### Using Multiple Voice Providers
|
|
913
873
|
|
|
@@ -1003,17 +963,17 @@ const voice = new CompositeVoice({
|
|
|
1003
963
|
});
|
|
1004
964
|
```
|
|
1005
965
|
|
|
1006
|
-
For more information on the CompositeVoice, refer to the [CompositeVoice Reference](https://mastra.ai/reference/
|
|
966
|
+
For more information on the CompositeVoice, refer to the [CompositeVoice Reference](https://mastra.ai/reference/voice/composite-voice).
|
|
1007
967
|
|
|
1008
968
|
## More Resources
|
|
1009
969
|
|
|
1010
|
-
- [CompositeVoice](https://mastra.ai/reference/
|
|
1011
|
-
- [MastraVoice](https://mastra.ai/reference/
|
|
1012
|
-
- [OpenAI Voice](https://mastra.ai/reference/
|
|
1013
|
-
- [OpenAI Realtime Voice](https://mastra.ai/reference/
|
|
1014
|
-
- [Azure Voice](https://mastra.ai/reference/
|
|
1015
|
-
- [Google Voice](https://mastra.ai/reference/
|
|
1016
|
-
- [Google Gemini Live Voice](https://mastra.ai/reference/
|
|
1017
|
-
- [Deepgram Voice](https://mastra.ai/reference/
|
|
1018
|
-
- [PlayAI Voice](https://mastra.ai/reference/
|
|
970
|
+
- [CompositeVoice](https://mastra.ai/reference/voice/composite-voice)
|
|
971
|
+
- [MastraVoice](https://mastra.ai/reference/voice/mastra-voice)
|
|
972
|
+
- [OpenAI Voice](https://mastra.ai/reference/voice/openai)
|
|
973
|
+
- [OpenAI Realtime Voice](https://mastra.ai/reference/voice/openai-realtime)
|
|
974
|
+
- [Azure Voice](https://mastra.ai/reference/voice/azure)
|
|
975
|
+
- [Google Voice](https://mastra.ai/reference/voice/google)
|
|
976
|
+
- [Google Gemini Live Voice](https://mastra.ai/reference/voice/google-gemini-live)
|
|
977
|
+
- [Deepgram Voice](https://mastra.ai/reference/voice/deepgram)
|
|
978
|
+
- [PlayAI Voice](https://mastra.ai/reference/voice/playai)
|
|
1019
979
|
- [Voice Examples](https://github.com/mastra-ai/voice-examples)
|