@mastra/voice-aws-nova-sonic 0.0.0-studio-cli-20260504022012

package/CHANGELOG.md

@@ -0,0 +1,51 @@
+# @mastra/voice-aws-nova-sonic
+
+## 0.0.0-studio-cli-20260504022012
+
+### Patch Changes
+
+- Updated dependencies [[`6dcd65f`](https://github.com/mastra-ai/mastra/commit/6dcd65f2a34069e6dc43ba35f1d11119b9b40bef), [`c05c9a1`](https://github.com/mastra-ai/mastra/commit/c05c9a13230988cef6d438a62f37760f31927bc7), [`e24aacb`](https://github.com/mastra-ai/mastra/commit/e24aacba07bd66f5d95b636dc24016fca26b52cf), [`1c2dda8`](https://github.com/mastra-ai/mastra/commit/1c2dda805fbfccc0abf55d4cb20cc34402dc3f0c), [`c721164`](https://github.com/mastra-ai/mastra/commit/c7211643f7ac861f83b19a3757cc921487fc9d75), [`1b55954`](https://github.com/mastra-ai/mastra/commit/1b559541c1e08a10e49d01ffc51a634dfc37a286), [`5adc55e`](https://github.com/mastra-ai/mastra/commit/5adc55e63407be8ee977914957d68bcc2a075ceb), [`70017d7`](https://github.com/mastra-ai/mastra/commit/70017d72ab741b5d7040e2a15c251a317782e39e), [`e4942bc`](https://github.com/mastra-ai/mastra/commit/e4942bc7fdc903572f7d84f26d5e15f9d39c763d)]:
+  - @mastra/core@0.0.0-studio-cli-20260504022012
+
+## 0.1.0
+
+### Minor Changes
+
+- Add new `@mastra/voice-aws-nova-sonic` voice provider for AWS Bedrock Nova 2 Sonic. ([#13232](https://github.com/mastra-ai/mastra/pull/13232))
+
+  The provider exposes a real-time bidirectional voice interface backed by the
+  `InvokeModelWithBidirectionalStreamCommand` API on AWS Bedrock, including:
+  - Live microphone streaming (`send` / `listen`) and assistant audio playback
+    via `speaking` events
+  - Live transcription via `writing` events with `SPECULATIVE` / `FINAL`
+    generation stages
+  - Barge-in / interrupt detection
+  - Speaker selection across all 18 Nova Sonic voices and configurable
+    endpointing sensitivity
+  - Tool calling with per-session `RequestContext`
+  - Configurable AWS region, model id, credentials (or default credential
+    provider chain), and inference / turn-detection parameters
+
+### Patch Changes
+
+- Updated dependencies [[`1723e09`](https://github.com/mastra-ai/mastra/commit/1723e099829892419ddbfe49287acfeac2522724), [`629f9e9`](https://github.com/mastra-ai/mastra/commit/629f9e9a7e56aa8f129515a3923c5813298790c7), [`25168fb`](https://github.com/mastra-ai/mastra/commit/25168fb9c1de9db7f8171df4f58ceb842c53aa29), [`ab34b5a`](https://github.com/mastra-ai/mastra/commit/ab34b5a2191b8e4353df1dbf7b9155e7d6628d79), [`5fb6c2a`](https://github.com/mastra-ai/mastra/commit/5fb6c2a95c1843cc231704b91354311fc1f34a71), [`2b0f355`](https://github.com/mastra-ai/mastra/commit/2b0f3553be3e9e5524da539a66e5cf82668440a4), [`394f0cf`](https://github.com/mastra-ai/mastra/commit/394f0cfc31e6b4d801219fdef2e9cc69e5bc8682), [`b2deb29`](https://github.com/mastra-ai/mastra/commit/b2deb29412b300c868655b5840463614fbb7962d), [`66644be`](https://github.com/mastra-ai/mastra/commit/66644beac1aa560f0e417956ff007c89341dc382), [`e109607`](https://github.com/mastra-ai/mastra/commit/e10960749251e34d46b480a20648c490fd30381b), [`310b953`](https://github.com/mastra-ai/mastra/commit/310b95345f302dcd5ba3ed862bdc96f059d44122), [`3d7f709`](https://github.com/mastra-ai/mastra/commit/3d7f709b615e588050bb6283c4ee5cfe2978cbde), [`48a42f1`](https://github.com/mastra-ai/mastra/commit/48a42f114a4006a95e0b7a1b5ad1a24815a175c2), [`8091c7c`](https://github.com/mastra-ai/mastra/commit/8091c7c944d15e13fef6d61b6cfd903f158d4006), [`2c83efc`](https://github.com/mastra-ai/mastra/commit/2c83efc4482b3efe50830e3b8b4ba9a8d219edff), [`43f0e1d`](https://github.com/mastra-ai/mastra/commit/43f0e1d5d5a74ba6fc746f2ad89ebe0c64777a7d), [`da0b9e2`](https://github.com/mastra-ai/mastra/commit/da0b9e2ba7ecc560213b426d6c097fe63946086e), [`282a10c`](https://github.com/mastra-ai/mastra/commit/282a10c9446e9922afe80e10e3770481c8ac8a28), [`04151c7`](https://github.com/mastra-ai/mastra/commit/04151c7dcea934b4fe9076708a23fac161195414), [`8091c7c`](https://github.com/mastra-ai/mastra/commit/8091c7c944d15e13fef6d61b6cfd903f158d4006)]:
+  - @mastra/core@1.31.0
+
+## 0.1.0-alpha.0
+
+### Minor Changes
+
+- Add new `@mastra/voice-aws-nova-sonic` voice provider for AWS Bedrock Nova 2 Sonic. ([#13232](https://github.com/mastra-ai/mastra/pull/13232))
+
+  The provider exposes a real-time bidirectional voice interface backed by the
+  `InvokeModelWithBidirectionalStreamCommand` API on AWS Bedrock, including:
+  - Live microphone streaming (`send` / `listen`) and assistant audio playback
+    via `speaking` events
+  - Live transcription via `writing` events with `SPECULATIVE` / `FINAL`
+    generation stages
+  - Barge-in / interrupt detection
+  - Speaker selection across all 18 Nova Sonic voices and configurable
+    endpointing sensitivity
+  - Tool calling with per-session `RequestContext`
+  - Configurable AWS region, model id, credentials (or default credential
+    provider chain), and inference / turn-detection parameters

package/LICENSE.md

@@ -0,0 +1,30 @@
+Portions of this software are licensed as follows:
+
+- All content that resides under any directory named "ee/" within this
+  repository, including but not limited to:
+  - `packages/core/src/auth/ee/`
+  - `packages/server/src/server/auth/ee/`
+    is licensed under the license defined in `ee/LICENSE`.
+
+- All third-party components incorporated into the Mastra Software are
+  licensed under the original license provided by the owner of the
+  applicable component.
+
+- Content outside of the above-mentioned directories or restrictions is
+  available under the "Apache License 2.0" as defined below.
+
+# Apache License 2.0
+
+Copyright (c) 2025 Kepler Software, Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

package/README.md

@@ -0,0 +1,384 @@
+# @mastra/voice-aws-nova-sonic
+
+Mastra integration for AWS Nova 2 Sonic, providing real-time bidirectional speech-to-speech capabilities using Amazon Bedrock's bidirectional streaming API.
+
+## Features
+
+- **Real-time bidirectional streaming**: Continuous audio streaming in both directions
+- **Multilingual support**: Supports English, French, Italian, German, Spanish, Portuguese, and Hindi
+- **Polyglot voices**: Voices that can speak multiple languages within the same session
+- **Barge-in support**: Users can interrupt the assistant mid-speech; handled server-side by Nova Sonic
+- **Tool/function calling**: Support for agentic workflows and async tool execution
+- **Cross-modal input**: Support for both audio and text inputs in the same conversation
+- **Natural turn-taking**: Intelligent voice activity detection and turn management
+- **Robust error handling**: Comprehensive error handling with detailed error codes
+
+## Installation
+
+```bash
+npm install @mastra/voice-aws-nova-sonic
+# or
+pnpm add @mastra/voice-aws-nova-sonic
+# or
+yarn add @mastra/voice-aws-nova-sonic
+```
+
+## Prerequisites
+
+- Node.js >= 22.13.0
+- AWS account with access to Amazon Bedrock
+- AWS credentials configured (see [AWS Setup](#aws-setup))
+- Access to Nova 2 Sonic model in your AWS region
+
+## AWS Setup
+
+### 1. Enable Nova 2 Sonic in Amazon Bedrock
+
+1. Go to the [Amazon Bedrock Console](https://console.aws.amazon.com/bedrock/)
+2. Navigate to "Model access" in the left sidebar
+3. Request access to "Amazon Nova 2 Sonic" model
+4. Wait for approval (usually instant)
+
+### 2. Configure AWS Credentials
+
+You can configure AWS credentials in several ways:
+
+**Option 1: Environment Variables**
+
+```bash
+export AWS_ACCESS_KEY_ID=your-access-key-id
+export AWS_SECRET_ACCESS_KEY=your-secret-access-key
+export AWS_REGION=us-east-1
+```
+
+**Option 2: AWS Credentials File**
+
+```ini
+# ~/.aws/credentials
+[default]
+aws_access_key_id = your-access-key-id
+aws_secret_access_key = your-secret-access-key
+```
+
+**Option 3: IAM Role** (for EC2/Lambda)
+
+- Attach an IAM role with Bedrock permissions to your EC2 instance or Lambda function
+
+**Option 4: Explicit Credentials in Code**
+
+```typescript
+import { NovaSonicVoice } from '@mastra/voice-aws-nova-sonic';
+
+const voice = new NovaSonicVoice({
+  region: 'us-east-1',
+  credentials: {
+    accessKeyId: 'your-access-key-id',
+    secretAccessKey: 'your-secret-access-key',
+  },
+});
+```
+
+### 3. IAM Permissions
+
+Your AWS credentials need the following IAM permissions:
+
+```json
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithBidirectionalStream"],
+      "Resource": "arn:aws:bedrock:*::foundation-model/amazon.nova-2-sonic-v1:0"
+    }
+  ]
+}
+```
+
+## Usage
+
+### Basic Example
+
+```typescript
+import { Agent } from '@mastra/core/agent';
+import { NovaSonicVoice } from '@mastra/voice-aws-nova-sonic';
+
+const agent = new Agent({
+  name: 'Nova Sonic Agent',
+  instructions: 'You are a helpful assistant with real-time voice capabilities.',
+  model: 'openai/gpt-4o',
+  voice: new NovaSonicVoice({
+    region: 'us-east-1',
+    speaker: 'tiffany',
+  }),
+});
+
+// Connect to the voice service
+await agent.voice.connect();
+
+// Listen for agent audio responses (stream of audio data)
+agent.voice.on('speaker', audioStream => {
+  // Pipe to your audio output (e.g., speaker, WebSocket, file)
+  audioStream.pipe(yourAudioOutput);
+});
+
+// Listen for text transcriptions
+agent.voice.on('writing', ({ text, role, generationStage }) => {
+  // generationStage is 'SPECULATIVE' (preview) or 'FINAL' (actual transcript)
+  console.log(`[${role}] ${text}`);
+});
+
+// Send continuous audio from the microphone (NodeJS.ReadableStream of PCM16 audio)
+await agent.voice.send(microphoneStream);
+```
+
+### Advanced Configuration
+
+```typescript
+import { NovaSonicVoice } from '@mastra/voice-aws-nova-sonic';
+
+const voice = new NovaSonicVoice({
+  region: 'us-east-1', // or 'us-west-2', 'ap-northeast-1'
+  model: 'amazon.nova-2-sonic-v1:0',
+  speaker: 'matthew', // or 'tiffany', 'amy', etc.
+  languageCode: 'en-US',
+  instructions: 'You are a helpful assistant.',
+  sessionConfig: {
+    tools: [
+      {
+        name: 'search',
+        description: 'Search the web',
+        inputSchema: {
+          type: 'object',
+          properties: {
+            query: { type: 'string' },
+          },
+          required: ['query'],
+        },
+      },
+    ],
+    turnDetectionConfiguration: {
+      // HIGH = fastest (1.5s pause), MEDIUM = balanced (1.75s), LOW = slowest (2s)
+      endpointingSensitivity: 'MEDIUM',
+    },
+  },
+  debug: true,
+});
+
+await voice.connect();
+```
+
+### With Tools
+
+```typescript
+import { Agent } from '@mastra/core/agent';
+import { NovaSonicVoice } from '@mastra/voice-aws-nova-sonic';
+import { createTool } from '@mastra/core/tools';
+import { z } from 'zod';
+
+const weatherTool = createTool({
+  id: 'weather',
+  description: 'Get weather information',
+  inputSchema: z.object({
+    location: z.string(),
+  }),
+  execute: async ({ context }) => {
+    // Fetch weather data
+    return { temperature: 72, condition: 'sunny' };
+  },
+});
+
+const agent = new Agent({
+  name: 'Weather Agent',
+  instructions: 'You help users get weather information.',
+  model: 'openai/gpt-4o',
+  tools: {
+    weather: weatherTool,
+  },
+  voice: new NovaSonicVoice({
+    region: 'us-east-1',
+  }),
+});
+
+await agent.voice.connect();
+// Tools are automatically available to the voice model
+```
+
+### Cross-Modal Text Input
+
+Send text messages during an active voice session:
+
+```typescript
+// After connecting and starting audio streaming
+await agent.voice.speak('What is the weather in New York?');
+```
+
+## API Reference
+
+### Constructor
+
+```typescript
+new NovaSonicVoice(config?: NovaSonicVoiceConfig)
+```
+
+**Configuration Options:**
+
+- `region` (string, optional): AWS region. Default: `'us-east-1'`. Supported: `'us-east-1'`, `'us-west-2'`, `'ap-northeast-1'`
+- `model` (string, optional): Model ID. Default: `'amazon.nova-2-sonic-v1:0'`
+- `credentials` (Credentials, optional): AWS credentials. If not provided, uses default credential chain
+- `speaker` (string, optional): Voice name/identifier (e.g., `'matthew'`, `'tiffany'`, `'amy'`)
+- `languageCode` (string, optional): Language code (e.g., `'en-US'`, `'fr-FR'`)
+- `instructions` (string, optional): System instructions for the model
+- `tools` (array, optional): Tool definitions
+- `sessionConfig` (object, optional): Session configuration including `turnDetectionConfiguration`, `tools`, `inferenceConfiguration`
+- `debug` (boolean, optional): Enable debug logging. Default: `false`
+
+### Methods
+
+#### `connect(options?)`
+
+Establishes connection to AWS Bedrock. Must be called before using other methods.
+
+```typescript
+await voice.connect();
+```
+
+#### `speak(input, options?)`
+
+Send cross-modal text input during an active voice session. Nova Sonic processes it and responds with audio.
+
+```typescript
+await voice.speak('Hello, world!');
+```
+
+#### `listen(audioStream, options?)`
+
+Stream audio input for transcription. For Nova Sonic, this is equivalent to `send()`.
+
+```typescript
+await voice.listen(audioStream);
+```
+
+#### `send(audioData)`
+
+Stream audio data in real-time. Accepts a `NodeJS.ReadableStream` (PCM16 audio) or an `Int16Array`.
+
+```typescript
+// Stream from a ReadableStream
+await voice.send(audioStream);
+
+// Or with Int16Array
+const audioArray = new Int16Array([...]);
+await voice.send(audioArray);
+```
+
+#### `close()`
+
+Disconnect and cleanup resources.
+
+```typescript
+voice.close();
+```
+
+#### `on(event, callback)`
+
+Register an event listener.
+
+```typescript
+voice.on('speaking', ({ audio }) => {
+  // audio is a base64-encoded string of PCM audio
+});
+
+voice.on('writing', ({ text, role, generationStage }) => {
+  // generationStage: 'SPECULATIVE' (preview) or 'FINAL' (actual transcript)
+  console.log(`${role}: ${text}`);
+});
+
+voice.on('error', ({ message, code }) => {
+  console.error(`Error: ${message} (${code})`);
+});
+```
+
+#### `off(event, callback)`
+
+Remove an event listener.
+
+```typescript
+voice.off('speaking', callback);
+```
+
+### Events
+
+- **`speaker`**: Audio stream (`NodeJS.ReadableStream`) for the full response
+- **`speaking`**: Audio chunk `{ audio: string, audioData: Buffer, response_id?: string }`
+- **`writing`**: Text transcription `{ text: string, role: 'assistant' | 'user', generationStage?: 'SPECULATIVE' | 'FINAL' }`
+- **`error`**: Error event `{ message: string, code?: string, details?: unknown }`
+- **`toolCall`**: Tool invocation `{ name: string, args: Record<string, any>, id: string }`
+- **`turnComplete`**: Turn completion `{ timestamp: number }`
+- **`interrupt`**: Barge-in detected `{ type: string, timestamp: number }`
+- **`contentStart`**: Content block started (raw Nova Sonic event)
+- **`contentEnd`**: Content block ended (raw Nova Sonic event)
+- **`usage`**: Token usage `{ inputTokens: number, outputTokens: number, totalTokens: number }`
+
+## Supported Regions
+
+- `us-east-1` (US East - N. Virginia)
+- `us-west-2` (US West - Oregon)
+- `ap-northeast-1` (Asia Pacific - Tokyo)
+
+## Supported Languages
+
+- English (US, UK, India, Australia)
+- French
+- Italian
+- German
+- Spanish
+- Portuguese
+- Hindi
+
+## Error Handling
+
+The package provides error handling with specific error codes:
+
+```typescript
+import { NovaSonicError, NovaSonicErrorCode } from '@mastra/voice-aws-nova-sonic';
+
+voice.on('error', ({ message, code, details }) => {
+  if (code === NovaSonicErrorCode.CONNECTION_FAILED) {
+    // Handle connection error
+  } else if (code === NovaSonicErrorCode.CREDENTIALS_MISSING) {
+    // Handle credentials error
+  }
+});
+```
+
+## Troubleshooting
+
+### Connection Issues
+
+- Verify AWS credentials are configured correctly
+- Check that Nova 2 Sonic is enabled in your AWS Bedrock console
+- Ensure your IAM role/user has the required permissions
+- Verify the region supports Nova 2 Sonic
+
+### Audio Issues
+
+- Ensure audio format is compatible (PCM, 16-bit, 16kHz)
+- Check sample rate matches expected format
+- Verify audio stream is not empty
+
+### Authentication Issues
+
+- Check AWS credentials are valid
+- Verify IAM permissions include Bedrock access
+- Ensure region is correct
+
+## License
+
+Apache-2.0
+
+## Links
+
+- [Mastra Documentation](https://mastra.ai)
+- [AWS Nova 2 Sonic Documentation](https://docs.aws.amazon.com/nova/latest/nova2-userguide/using-conversational-speech.html)
+- [Amazon Bedrock Documentation](https://docs.aws.amazon.com/bedrock/)

package/dist/docs/SKILL.md

@@ -0,0 +1,27 @@
+---
+name: mastra-voice-aws-nova-sonic
+description: Documentation for @mastra/voice-aws-nova-sonic. Use when working with @mastra/voice-aws-nova-sonic APIs, configuration, or implementation.
+metadata:
+  package: "@mastra/voice-aws-nova-sonic"
+  version: "0.0.0-studio-cli-20260504022012"
+---
+
+## When to use
+
+Use this skill whenever you are working with @mastra/voice-aws-nova-sonic to obtain the domain-specific knowledge.
+
+## How to use
+
+Read the individual reference documents for detailed explanations and code examples.
+
+### Docs
+
+- [Voice in Mastra](references/docs-voice-overview.md) - Overview of voice capabilities in Mastra, including text-to-speech, speech-to-text, and real-time speech-to-speech interactions.
+- [Speech-to-Speech capabilities in Mastra](references/docs-voice-speech-to-speech.md) - Overview of speech-to-speech capabilities in Mastra, including real-time interactions and event-driven architecture.
+
+### Reference
+
+- [Reference: AWS Nova Sonic voice](references/reference-voice-aws-nova-sonic.md) - Documentation for the NovaSonicVoice class, providing real-time speech-to-speech capabilities via AWS Bedrock Nova 2 Sonic.
+
+
+Read [assets/SOURCE_MAP.json](assets/SOURCE_MAP.json) for source code references.

package/dist/docs/assets/SOURCE_MAP.json

@@ -0,0 +1,6 @@
+{
+  "version": "0.0.0-studio-cli-20260504022012",
+  "package": "@mastra/voice-aws-nova-sonic",
+  "exports": {},
+  "modules": {}
+}