@mastra/mcp-docs-server 1.1.5 → 1.1.6-alpha.0

package/.docs/reference/voice/google.md

@@ -0,0 +1,228 @@
+# Google
+
+The Google Voice implementation in Mastra provides both text-to-speech (TTS) and speech-to-text (STT) capabilities using Google Cloud services. It supports multiple voices, languages, advanced audio configuration options, and both standard API key authentication and Vertex AI mode for enterprise deployments.
+
+## Usage Example
+
+```typescript
+import { GoogleVoice } from '@mastra/voice-google'
+
+// Initialize with default configuration (uses GOOGLE_API_KEY environment variable)
+const voice = new GoogleVoice()
+
+// Text-to-Speech
+const audioStream = await voice.speak('Hello, world!', {
+  languageCode: 'en-US',
+  audioConfig: {
+    audioEncoding: 'LINEAR16',
+  },
+})
+
+// Speech-to-Text
+const transcript = await voice.listen(audioStream, {
+  config: {
+    encoding: 'LINEAR16',
+    languageCode: 'en-US',
+  },
+})
+
+// Get available voices for a specific language
+const voices = await voice.getSpeakers({ languageCode: 'en-US' })
+```
+
+## Constructor Parameters
+
+**speechModel?:** (`GoogleModelConfig`): Configuration for text-to-speech functionality (Default: `{ apiKey: process.env.GOOGLE_API_KEY }`)
+
+**listeningModel?:** (`GoogleModelConfig`): Configuration for speech-to-text functionality (Default: `{ apiKey: process.env.GOOGLE_API_KEY }`)
+
+**speaker?:** (`string`): Default voice ID to use for text-to-speech (Default: `'en-US-Casual-K'`)
+
+**vertexAI?:** (`boolean`): Enable Vertex AI mode for enterprise deployments. Uses project-based authentication instead of API keys. Requires 'project' to be set. (Default: `false`)
+
+**project?:** (`string`): Google Cloud project ID (required when vertexAI is true). Falls back to GOOGLE\_CLOUD\_PROJECT environment variable.
+
+**location?:** (`string`): Google Cloud region for Vertex AI. Falls back to GOOGLE\_CLOUD\_LOCATION environment variable. (Default: `'us-central1'`)
+
+### GoogleModelConfig
+
+**apiKey?:** (`string`): Google Cloud API key. Falls back to GOOGLE\_API\_KEY environment variable. Not used when vertexAI is true.
+
+**keyFilename?:** (`string`): Path to service account JSON key file. Falls back to GOOGLE\_APPLICATION\_CREDENTIALS environment variable.
+
+**credentials?:** (`object`): In-memory service account credentials object with client\_email and private\_key properties.
+
+## Methods
+
+### speak()
+
+Converts text to speech using Google Cloud Text-to-Speech service.
+
+**input:** (`string | NodeJS.ReadableStream`): Text to convert to speech. If a stream is provided, it will be converted to text first.
+
+**options?:** (`object`): Speech synthesis options
+
+**options.speaker?:** (`string`): Voice ID to use for this request
+
+**options.languageCode?:** (`string`): Language code for the voice (e.g., 'en-US'). Defaults to the language code from the speaker ID or 'en-US'
+
+**options.audioConfig?:** (`ISynthesizeSpeechRequest['audioConfig']`): Audio configuration options from Google Cloud Text-to-Speech API (Default: `{ audioEncoding: 'LINEAR16' }`)
+
+Returns: `Promise<NodeJS.ReadableStream>`
+
+### listen()
+
+Converts speech to text using Google Cloud Speech-to-Text service.
+
+**audioStream:** (`NodeJS.ReadableStream`): Audio stream to transcribe
+
+**options?:** (`object`): Recognition options
+
+**options.stream?:** (`boolean`): Whether to use streaming recognition
+
+**options.config?:** (`IRecognitionConfig`): Recognition configuration from Google Cloud Speech-to-Text API (Default: `{ encoding: 'LINEAR16', languageCode: 'en-US' }`)
+
+Returns: `Promise<string>`
+
+### getSpeakers()
+
+Returns an array of available voice options, where each node contains:
+
+**voiceId:** (`string`): Unique identifier for the voice
+
+**languageCodes:** (`string[]`): List of language codes supported by this voice
+
+### isUsingVertexAI()
+
+Checks if Vertex AI mode is enabled.
+
+Returns: `boolean` - `true` if using Vertex AI, `false` otherwise
+
+### getProject()
+
+Gets the configured Google Cloud project ID.
+
+Returns: `string | undefined` - The project ID or `undefined` if not set
+
+### getLocation()
+
+Gets the configured Google Cloud location/region.
+
+Returns: `string` - The location (default: `'us-central1'`)
+
+## Authentication
+
+The Google Voice provider supports two authentication methods:
+
+### Standard Mode (API Key)
+
+Uses a Google Cloud API key for authentication. Suitable for development and simple use cases.
+
+```typescript
+// Using environment variable (GOOGLE_API_KEY)
+const voice = new GoogleVoice()
+
+// Using explicit API key
+const voice = new GoogleVoice({
+  speechModel: { apiKey: 'your-api-key' },
+  listeningModel: { apiKey: 'your-api-key' },
+  speaker: 'en-US-Casual-K',
+})
+```
+
+### Vertex AI Mode (Service Account)
+
+Uses Google Cloud project-based authentication with service accounts. Recommended for production and enterprise deployments.
+
+**Benefits:**
+
+- Better security (no API keys in code)
+- IAM-based access control
+- Project-level billing and quotas
+- Audit logging
+- Enterprise features
+
+**Configuration Options:**
+
+```typescript
+// Using Application Default Credentials (ADC)
+// Set GOOGLE_APPLICATION_CREDENTIALS and GOOGLE_CLOUD_PROJECT env vars
+const voice = new GoogleVoice({
+  vertexAI: true,
+  project: 'your-gcp-project',
+  location: 'us-central1', // Optional, defaults to 'us-central1'
+})
+
+// Using service account key file
+const voice = new GoogleVoice({
+  vertexAI: true,
+  project: 'your-gcp-project',
+  speechModel: {
+    keyFilename: '/path/to/service-account.json',
+  },
+  listeningModel: {
+    keyFilename: '/path/to/service-account.json',
+  },
+})
+
+// Using in-memory credentials
+const voice = new GoogleVoice({
+  vertexAI: true,
+  project: 'your-gcp-project',
+  speechModel: {
+    credentials: {
+      client_email: 'service-account@project.iam.gserviceaccount.com',
+      private_key: '-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----',
+    },
+  },
+})
+```
+
+#### Required Permissions
+
+#### IAM Roles
+
+For Text-to-Speech:
+
+- `roles/texttospeech.admin` - Text-to-Speech Admin (full access)
+- `roles/texttospeech.editor` - Text-to-Speech Editor (create and manage)
+- `roles/texttospeech.viewer` - Text-to-Speech Viewer (read-only)
+
+For Speech-to-Text:
+
+- `roles/speech.client` - Speech-to-Text Client
+
+#### OAuth Scopes
+
+For synchronous Text-to-Speech synthesis:
+
+- `https://www.googleapis.com/auth/cloud-platform` - Full access to Google Cloud Platform services
+
+For long-audio Text-to-Speech operations:
+
+- `locations.longAudioSynthesize` - Create long-audio synthesis operations
+- `operations.get` - Get operation status
+- `operations.list` - List operations
+
+## Important Notes
+
+1. **Authentication**: Either a Google Cloud API key (standard mode) or service account credentials (Vertex AI mode) is required.
+
+2. **Environment Variables**:
+
+   - `GOOGLE_API_KEY` - API key for standard mode
+   - `GOOGLE_CLOUD_PROJECT` - Project ID for Vertex AI mode
+   - `GOOGLE_CLOUD_LOCATION` - Location for Vertex AI mode (defaults to 'us-central1')
+   - `GOOGLE_APPLICATION_CREDENTIALS` - Path to service account key file
+
+3. The default voice is set to `'en-US-Casual-K'`.
+
+4. Both text-to-speech and speech-to-text services use LINEAR16 as the default audio encoding.
+
+5. The `speak()` method supports advanced audio configuration through the Google Cloud Text-to-Speech API.
+
+6. The `listen()` method supports various recognition configurations through the Google Cloud Speech-to-Text API.
+
+7. Available voices can be filtered by language code using the `getSpeakers()` method.
+
+8. Vertex AI mode provides enterprise features including IAM control, audit logs, and project-level billing.

package/.docs/reference/voice/mastra-voice.md

@@ -0,0 +1,311 @@
+# MastraVoice
+
+The MastraVoice class is an abstract base class that defines the core interface for voice services in Mastra. All voice provider implementations (like OpenAI, Deepgram, PlayAI, Speechify) extend this class to provide their specific functionality. The class now includes support for real-time speech-to-speech capabilities through WebSocket connections.
+
+## Usage Example
+
+```typescript
+import { MastraVoice } from '@mastra/core/voice'
+
+// Create a voice provider implementation
+class MyVoiceProvider extends MastraVoice {
+  constructor(config: {
+    speechModel?: BuiltInModelConfig
+    listeningModel?: BuiltInModelConfig
+    speaker?: string
+    realtimeConfig?: {
+      model?: string
+      apiKey?: string
+      options?: unknown
+    }
+  }) {
+    super({
+      speechModel: config.speechModel,
+      listeningModel: config.listeningModel,
+      speaker: config.speaker,
+      realtimeConfig: config.realtimeConfig,
+    })
+  }
+
+  // Implement required abstract methods
+  async speak(
+    input: string | NodeJS.ReadableStream,
+    options?: { speaker?: string },
+  ): Promise<NodeJS.ReadableStream | void> {
+    // Implement text-to-speech conversion
+  }
+
+  async listen(
+    audioStream: NodeJS.ReadableStream,
+    options?: unknown,
+  ): Promise<string | NodeJS.ReadableStream | void> {
+    // Implement speech-to-text conversion
+  }
+
+  async getSpeakers(): Promise<Array<{ voiceId: string; [key: string]: unknown }>> {
+    // Return list of available voices
+  }
+
+  // Optional speech-to-speech methods
+  async connect(): Promise<void> {
+    // Establish WebSocket connection for speech-to-speech communication
+  }
+
+  async send(audioData: NodeJS.ReadableStream | Int16Array): Promise<void> {
+    // Stream audio data in speech-to-speech
+  }
+
+  async answer(): Promise<void> {
+    // Trigger voice provider to respond
+  }
+
+  addTools(tools: Array<unknown>): void {
+    // Add tools for the voice provider to use
+  }
+
+  close(): void {
+    // Close WebSocket connection
+  }
+
+  on(event: string, callback: (data: unknown) => void): void {
+    // Register event listener
+  }
+
+  off(event: string, callback: (data: unknown) => void): void {
+    // Remove event listener
+  }
+}
+```
+
+## Constructor Parameters
+
+**config?:** (`VoiceConfig`): Configuration object for the voice service
+
+**config.speechModel?:** (`BuiltInModelConfig`): Configuration for the text-to-speech model
+
+**config.listeningModel?:** (`BuiltInModelConfig`): Configuration for the speech-to-text model
+
+**config.speaker?:** (`string`): Default speaker/voice ID to use
+
+**config.name?:** (`string`): Name for the voice provider instance
+
+**config.realtimeConfig?:** (`object`): Configuration for real-time speech-to-speech capabilities
+
+### BuiltInModelConfig
+
+**name:** (`string`): Name of the model to use
+
+**apiKey?:** (`string`): API key for the model service
+
+### RealtimeConfig
+
+**model?:** (`string`): Model to use for real-time speech-to-speech capabilities
+
+**apiKey?:** (`string`): API key for the real-time service
+
+**options?:** (`unknown`): Provider-specific options for real-time capabilities
+
+## Abstract Methods
+
+These methods must be implemented by unknown class extending MastraVoice.
+
+### speak()
+
+Converts text to speech using the configured speech model.
+
+```typescript
+abstract speak(
+  input: string | NodeJS.ReadableStream,
+  options?: {
+    speaker?: string;
+    [key: string]: unknown;
+  }
+): Promise<NodeJS.ReadableStream | void>
+```
+
+Purpose:
+
+- Takes text input and converts it to speech using the provider's text-to-speech service
+- Supports both string and stream input for flexibility
+- Allows overriding the default speaker/voice through options
+- Returns a stream of audio data that can be played or saved
+- May return void if the audio is handled by emitting 'speaking' event
+
+### listen()
+
+Converts speech to text using the configured listening model.
+
+```typescript
+abstract listen(
+  audioStream: NodeJS.ReadableStream,
+  options?: {
+    [key: string]: unknown;
+  }
+): Promise<string | NodeJS.ReadableStream | void>
+```
+
+Purpose:
+
+- Takes an audio stream and converts it to text using the provider's speech-to-text service
+- Supports provider-specific options for transcription configuration
+- Can return either a complete text transcription or a stream of transcribed text
+- Not all providers support this functionality (e.g., PlayAI, Speechify)
+- May return void if the transcription is handled by emitting 'writing' event
+
+### getSpeakers()
+
+Returns a list of available voices supported by the provider.
+
+```typescript
+abstract getSpeakers(): Promise<Array<{ voiceId: string; [key: string]: unknown }>>
+```
+
+Purpose:
+
+- Retrieves the list of available voices/speakers from the provider
+- Each voice must have at least a voiceId property
+- Providers can include additional metadata about each voice
+- Used to discover available voices for text-to-speech conversion
+
+## Optional Methods
+
+These methods have default implementations but can be overridden by voice providers that support speech-to-speech capabilities.
+
+### connect()
+
+Establishes a WebSocket or WebRTC connection for communication.
+
+```typescript
+connect(config?: unknown): Promise<void>
+```
+
+Purpose:
+
+- Initializes a connection to the voice service for communication
+- Must be called before using features like send() or answer()
+- Returns a Promise that resolves when the connection is established
+- Configuration is provider-specific
+
+### send()
+
+Streams audio data in real-time to the voice provider.
+
+```typescript
+send(audioData: NodeJS.ReadableStream | Int16Array): Promise<void>
+```
+
+Purpose:
+
+- Sends audio data to the voice provider for real-time processing
+- Useful for continuous audio streaming scenarios like live microphone input
+- Supports both ReadableStream and Int16Array audio formats
+- Must be in connected state before calling this method
+
+### answer()
+
+Triggers the voice provider to generate a response.
+
+```typescript
+answer(): Promise<void>
+```
+
+Purpose:
+
+- Sends a signal to the voice provider to generate a response
+- Used in real-time conversations to prompt the AI to respond
+- Response will be emitted through the event system (e.g., 'speaking' event)
+
+### addTools()
+
+Equips the voice provider with tools that can be used during conversations.
+
+```typescript
+addTools(tools: Array<Tool>): void
+```
+
+Purpose:
+
+- Adds tools that the voice provider can use during conversations
+- Tools can extend the capabilities of the voice provider
+- Implementation is provider-specific
+
+### close()
+
+Disconnects from the WebSocket or WebRTC connection.
+
+```typescript
+close(): void
+```
+
+Purpose:
+
+- Closes the connection to the voice service
+- Cleans up resources and stops any ongoing real-time processing
+- Should be called when you're done with the voice instance
+
+### on()
+
+Registers an event listener for voice events.
+
+```typescript
+on<E extends VoiceEventType>(
+  event: E,
+  callback: (data: E extends keyof VoiceEventMap ? VoiceEventMap[E] : unknown) => void,
+): void
+```
+
+Purpose:
+
+- Registers a callback function to be called when the specified event occurs
+- Standard events include 'speaking', 'writing', and 'error'
+- Providers can emit custom events as well
+- Event data structure depends on the event type
+
+### off()
+
+Removes an event listener.
+
+```typescript
+off<E extends VoiceEventType>(
+  event: E,
+  callback: (data: E extends keyof VoiceEventMap ? VoiceEventMap[E] : unknown) => void,
+): void
+```
+
+Purpose:
+
+- Removes a previously registered event listener
+- Used to clean up event handlers when they're no longer needed
+
+## Event System
+
+The MastraVoice class includes an event system for real-time communication. Standard event types include:
+
+**speaking:** (`{ text: string; audioStream?: NodeJS.ReadableStream; audio?: Int16Array }`): Emitted when the voice provider is speaking, contains audio data
+
+**writing:** (`{ text: string, role: string }`): Emitted when text is transcribed from speech
+
+**error:** (`{ message: string; code?: string; details?: unknown }`): Emitted when an error occurs
+
+## Protected Properties
+
+**listeningModel?:** (`BuiltInModelConfig | undefined`): Configuration for the speech-to-text model
+
+**speechModel?:** (`BuiltInModelConfig | undefined`): Configuration for the text-to-speech model
+
+**speaker?:** (`string | undefined`): Default speaker/voice ID
+
+**realtimeConfig?:** (`{ model?: string; apiKey?: string; options?: unknown } | undefined`): Configuration for real-time speech-to-speech capabilities
+
+## Telemetry Support
+
+MastraVoice includes built-in telemetry support through the `traced` method, which wraps method calls with performance tracking and error monitoring.
+
+## Notes
+
+- MastraVoice is an abstract class and cannot be instantiated directly
+- Implementations must provide concrete implementations for all abstract methods
+- The class provides a consistent interface across different voice service providers
+- Speech-to-speech capabilities are optional and provider-specific
+- The event system enables asynchronous communication for real-time interactions
+- Telemetry is automatically handled for all method calls

package/.docs/reference/voice/murf.md

@@ -0,0 +1,122 @@
+# Murf
+
+The Murf voice implementation in Mastra provides text-to-speech (TTS) capabilities using Murf's AI voice service. It supports multiple voices across different languages.
+
+## Usage Example
+
+```typescript
+import { MurfVoice } from '@mastra/voice-murf'
+
+// Initialize with default configuration (uses MURF_API_KEY environment variable)
+const voice = new MurfVoice()
+
+// Initialize with custom configuration
+const voice = new MurfVoice({
+  speechModel: {
+    name: 'GEN2',
+    apiKey: 'your-api-key',
+    properties: {
+      format: 'MP3',
+      rate: 1.0,
+      pitch: 1.0,
+      sampleRate: 48000,
+      channelType: 'STEREO',
+    },
+  },
+  speaker: 'en-US-cooper',
+})
+
+// Text-to-Speech with default settings
+const audioStream = await voice.speak('Hello, world!')
+
+// Text-to-Speech with custom properties
+const audioStream = await voice.speak('Hello, world!', {
+  speaker: 'en-UK-hazel',
+  properties: {
+    format: 'WAV',
+    rate: 1.2,
+    style: 'casual',
+  },
+})
+
+// Get available voices
+const voices = await voice.getSpeakers()
+```
+
+## Constructor Parameters
+
+**speechModel?:** (`MurfConfig`): Configuration for text-to-speech functionality (Default: `{ name: 'GEN2' }`)
+
+**speaker?:** (`string`): Default voice ID to use for text-to-speech (Default: `'en-UK-hazel'`)
+
+### MurfConfig
+
+**name:** (`'GEN1' | 'GEN2'`): The Murf model generation to use (Default: `'GEN2'`)
+
+**apiKey?:** (`string`): Murf API key. Falls back to MURF\_API\_KEY environment variable
+
+**properties?:** (`object`): Default properties for all speech synthesis requests
+
+### Speech Properties
+
+**style?:** (`string`): Speaking style for the voice
+
+**rate?:** (`number`): Speech rate multiplier
+
+**pitch?:** (`number`): Voice pitch adjustment
+
+**sampleRate?:** (`8000 | 24000 | 44100 | 48000`): Audio sample rate in Hz
+
+**format?:** (`'MP3' | 'WAV' | 'FLAC' | 'ALAW' | 'ULAW'`): Output audio format
+
+**channelType?:** (`'STEREO' | 'MONO'`): Audio channel configuration
+
+**pronunciationDictionary?:** (`Record<string, string>`): Custom pronunciation mappings
+
+**encodeAsBase64?:** (`boolean`): Whether to encode the audio as base64
+
+**variation?:** (`number`): Voice variation parameter
+
+**audioDuration?:** (`number`): Target audio duration in seconds
+
+**multiNativeLocale?:** (`string`): Locale for multilingual support
+
+## Methods
+
+### speak()
+
+Converts text to speech using Murf's API.
+
+**input:** (`string | NodeJS.ReadableStream`): Text to convert to speech. If a stream is provided, it will be converted to text first.
+
+**options?:** (`object`): Speech synthesis options
+
+**options.speaker?:** (`string`): Override the default speaker for this request
+
+**options.properties?:** (`object`): Override default speech properties for this request
+
+Returns: `Promise<NodeJS.ReadableStream>`
+
+### getSpeakers()
+
+Returns an array of available voice options, where each node contains:
+
+**voiceId:** (`string`): Unique identifier for the voice
+
+**name:** (`string`): Display name of the voice
+
+**language:** (`string`): Language code for the voice
+
+**gender:** (`string`): Gender of the voice
+
+### listen()
+
+This method is not supported by Murf and will throw an error. Murf does not provide speech-to-text functionality.
+
+## Important Notes
+
+1. A Murf API key is required. Set it via the `MURF_API_KEY` environment variable or pass it in the constructor.
+2. The service uses GEN2 as the default model version.
+3. Speech properties can be set at the constructor level and overridden per request.
+4. The service supports extensive audio customization through properties like format, sample rate, and channel type.
+5. Speech-to-text functionality is not supported.