@mastra/mcp-docs-server 1.2.15-alpha.19 → 1.2.15-alpha.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/code-mode.md +1 -1
- package/.docs/docs/agents/networks.md +2 -2
- package/.docs/docs/agents/overview.md +2 -2
- package/.docs/docs/agents/using-tools.md +1 -1
- package/.docs/docs/browser/overview.md +8 -8
- package/.docs/docs/browser/recording.md +2 -2
- package/.docs/docs/capabilities/{channels/overview.md → channels.md} +70 -7
- package/.docs/docs/capabilities/subagents.md +2 -2
- package/.docs/docs/deployment/cloud-providers.md +9 -9
- package/.docs/docs/deployment/overview.md +9 -9
- package/.docs/docs/deployment/sandbox.md +3 -3
- package/.docs/docs/deployment/web-framework.md +6 -6
- package/.docs/docs/deployment/workflow-runners.md +2 -2
- package/.docs/docs/harness/agent-controller.md +2 -2
- package/.docs/docs/index.md +8 -8
- package/.docs/docs/long-running-agents/durable-agents.md +2 -2
- package/.docs/docs/long-running-agents/signal-providers.md +2 -15
- package/.docs/docs/long-running-agents/signals.md +1 -1
- package/.docs/docs/mastra-platform/database.md +2 -2
- package/.docs/docs/mastra-platform/deploy.md +27 -0
- package/.docs/docs/mastra-platform/server.md +1 -1
- package/.docs/docs/mastra-platform/trace-intelligence.md +9 -13
- package/.docs/docs/mcp/overview.md +42 -0
- package/.docs/docs/memory/memory-processors.md +5 -5
- package/.docs/docs/memory/message-history.md +1 -1
- package/.docs/docs/memory/observational-memory.md +1 -1
- package/.docs/docs/observability/integrations/exporters/mastra-storage.md +16 -16
- package/.docs/docs/observability/integrations/overview.md +3 -3
- package/.docs/docs/server/auth/fga.md +1 -1
- package/.docs/docs/server/auth/simple-auth.md +1 -1
- package/.docs/docs/server/auth.md +8 -8
- package/.docs/docs/server/custom-api-routes.md +35 -0
- package/.docs/docs/storage/overview.md +14 -13
- package/.docs/docs/studio/observability.md +1 -1
- package/.docs/docs/workflows/overview.md +1 -1
- package/.docs/docs/workflows/snapshots.md +11 -11
- package/.docs/docs/workflows/time-travel.md +2 -0
- package/.docs/docs/workspace/filesystem.md +15 -15
- package/.docs/docs/workspace/sandbox.md +17 -17
- package/.docs/docs/workspace/search.md +1 -1
- package/.docs/guides/agent-frameworks/ai-sdk.md +2 -2
- package/.docs/guides/deployment/mastra-workers.md +1 -1
- package/.docs/guides/getting-started/quickstart.md +3 -3
- package/.docs/guides/index.md +8 -8
- package/.docs/guides/voice/overview.md +55 -106
- package/.docs/guides/voice/speech-to-text.md +7 -7
- package/.docs/guides/voice/text-to-speech.md +9 -10
- package/.docs/{guides/build-your-ui → integrations/agentic-ui}/ai-sdk-ui.md +2 -2
- package/.docs/{guides/build-your-ui/copilotkit/overview.md → integrations/agentic-ui/copilotkit.md} +256 -3
- package/.docs/{docs/browser → integrations/browsers}/agent-browser.md +2 -2
- package/.docs/{docs/browser → integrations/browsers}/browser-viewer.md +3 -3
- package/.docs/{docs/browser → integrations/browsers}/firecrawl.md +2 -2
- package/.docs/{docs/browser → integrations/browsers}/stagehand.md +1 -1
- package/.docs/{docs/capabilities → integrations}/channels/discord.md +2 -2
- package/.docs/integrations/channels/github.md +103 -0
- package/.docs/{docs/capabilities → integrations}/channels/imessage.md +7 -5
- package/.docs/{docs/capabilities → integrations}/channels/slack.md +4 -4
- package/.docs/{docs/capabilities → integrations}/channels/teams.md +2 -2
- package/.docs/{docs/capabilities → integrations}/channels/telegram.md +2 -2
- package/.docs/{docs/capabilities → integrations}/channels/whatsapp.md +2 -2
- package/.docs/{reference/storage/dsql.md → integrations/databases/aurora-dsql.md} +1 -1
- package/.docs/{reference/storage → integrations/databases}/clickhouse.md +2 -2
- package/.docs/{reference/storage → integrations/databases}/cloudflare-d1.md +1 -1
- package/.docs/{reference/storage/cloudflare.md → integrations/databases/cloudflare-kv.md} +1 -1
- package/.docs/{reference/storage → integrations/databases}/convex.md +1 -1
- package/.docs/{reference/storage → integrations/databases}/duckdb.md +5 -5
- package/.docs/{reference/storage → integrations/databases}/dynamodb.md +1 -1
- package/.docs/{reference/storage/lance.md → integrations/databases/lancedb.md} +1 -1
- package/.docs/{reference/storage → integrations/databases}/libsql.md +2 -2
- package/.docs/{reference/storage → integrations/databases}/mongodb.md +1 -1
- package/.docs/{reference/storage → integrations/databases}/mssql.md +1 -1
- package/.docs/integrations/databases/neon.md +220 -0
- package/.docs/{reference/storage → integrations/databases}/oracledb.md +1 -1
- package/.docs/{reference/storage → integrations/databases}/postgresql.md +1 -1
- package/.docs/{reference/storage → integrations/databases}/redis.md +1 -1
- package/.docs/{reference/storage → integrations/databases}/spanner.md +1 -1
- package/.docs/{reference/storage → integrations/databases}/upstash.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/aws-bedrock-agentcore.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/aws-lambda.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/azure-app-services.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/cloudflare.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/digital-ocean.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/netlify.md +1 -1
- package/.docs/{guides/deployment → integrations/deploy}/vercel.md +2 -2
- package/.docs/{reference/workspace/s3-filesystem.md → integrations/file-storage/amazon-s3.md} +5 -5
- package/.docs/{reference/workspace/archil-filesystem.md → integrations/file-storage/archil.md} +3 -3
- package/.docs/{reference/workspace/azure-blob-filesystem.md → integrations/file-storage/azure-blob.md} +2 -2
- package/.docs/{reference/workspace/gcs-filesystem.md → integrations/file-storage/google-cloud-storage.md} +5 -5
- package/.docs/{reference/workspace/mesa-filesystem.md → integrations/file-storage/mesa.md} +1 -1
- package/.docs/{reference/workspace/files-sdk-filesystem.md → integrations/file-storage/vercel-files.md} +5 -5
- package/.docs/{guides/getting-started → integrations/frameworks}/astro.md +1 -1
- package/.docs/{guides/getting-started → integrations/frameworks}/electron.md +1 -1
- package/.docs/{guides/getting-started → integrations/frameworks}/next-js.md +2 -2
- package/.docs/{guides/getting-started → integrations/frameworks}/nuxt.md +1 -1
- package/.docs/{guides/getting-started → integrations/frameworks}/sveltekit.md +1 -1
- package/.docs/{guides/getting-started → integrations/frameworks}/vite-react.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/arize.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/arthur.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/braintrust.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/confident-ai.md +1 -1
- package/.docs/integrations/observability/datadog.md +538 -0
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/laminar.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/langfuse.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/langsmith.md +1 -1
- package/.docs/{docs/observability/integrations/exporters/otel.md → integrations/observability/opentelemetry.md} +276 -44
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/posthog.md +1 -1
- package/.docs/{docs/observability/integrations/exporters → integrations/observability}/sentry.md +1 -1
- package/.docs/{reference/workspace/apple-container-sandbox.md → integrations/sandboxes/apple-container.md} +1 -1
- package/.docs/{reference/workspace/daytona-sandbox.md → integrations/sandboxes/daytona.md} +2 -2
- package/.docs/{reference/workspace/docker-sandbox.md → integrations/sandboxes/docker.md} +1 -1
- package/.docs/{reference/workspace/e2b-sandbox.md → integrations/sandboxes/e2b.md} +3 -3
- package/.docs/{reference/workspace/modal-sandbox.md → integrations/sandboxes/modal.md} +2 -2
- package/.docs/{reference/workspace/railway-sandbox.md → integrations/sandboxes/railway.md} +8 -0
- package/.docs/{reference/workspace/vercel-sandbox.md → integrations/sandboxes/vercel.md} +141 -12
- package/.docs/{reference → integrations}/tools/brightdata.md +1 -1
- package/.docs/{reference → integrations}/tools/perplexity.md +1 -1
- package/.docs/{reference → integrations}/tools/tavily.md +1 -1
- package/.docs/{reference → integrations}/voice/aws-nova-sonic.md +1 -1
- package/.docs/{reference → integrations}/voice/cloudflare.md +3 -3
- package/.docs/{reference/voice/google-gemini-live.md → integrations/voice/google.md} +305 -30
- package/.docs/{reference/voice/inworld-realtime.md → integrations/voice/inworld.md} +163 -25
- package/.docs/{reference → integrations}/voice/livekit.md +437 -34
- package/.docs/{reference/voice/openai-realtime.md → integrations/voice/openai.md} +117 -20
- package/.docs/integrations.md +147 -0
- package/.docs/models/embeddings.md +2 -2
- package/.docs/models/environment-variables.md +1 -0
- package/.docs/models/gateways/openrouter.md +4 -2
- package/.docs/models/gateways/vercel.md +2 -1
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/coralbricks.md +75 -0
- package/.docs/models/providers/evroc.md +2 -3
- package/.docs/models/providers/hyper.md +6 -6
- package/.docs/models/providers/kilo.md +7 -5
- package/.docs/models/providers/llmgateway.md +1 -2
- package/.docs/models/providers/nano-gpt.md +1 -3
- package/.docs/models/providers/opencode.md +0 -1
- package/.docs/models/providers/privatemode-ai.md +5 -4
- package/.docs/models/providers/snowflake-cortex.md +5 -1
- package/.docs/models/providers/wandb.md +2 -1
- package/.docs/models/providers/zenmux.md +0 -15
- package/.docs/models/providers.md +1 -0
- package/.docs/reference/agent-controller/agent-controller-class.md +32 -1
- package/.docs/reference/agents/channels.md +2 -2
- package/.docs/reference/agents/generateLegacy.md +2 -2
- package/.docs/reference/agents/getDefaultOptions.md +1 -1
- package/.docs/reference/agents/getDefaultStreamOptions.md +1 -1
- package/.docs/reference/agents/inngest-agent.md +1 -1
- package/.docs/reference/agents/network.md +1 -1
- package/.docs/reference/ai-sdk/handle-network-stream.md +1 -1
- package/.docs/reference/ai-sdk/network-route.md +1 -1
- package/.docs/reference/auth/auth0.md +1 -1
- package/.docs/reference/auth/better-auth.md +1 -1
- package/.docs/reference/auth/clerk.md +1 -1
- package/.docs/reference/auth/firebase.md +1 -1
- package/.docs/reference/auth/google.md +1 -1
- package/.docs/reference/auth/supabase.md +1 -1
- package/.docs/reference/auth/workos.md +1 -1
- package/.docs/reference/browser/agent-browser.md +1 -1
- package/.docs/reference/browser/browser-viewer.md +1 -1
- package/.docs/reference/browser/firecrawl-browser.md +1 -1
- package/.docs/reference/browser/stagehand-browser.md +1 -1
- package/.docs/reference/channels/channel-provider.md +1 -1
- package/.docs/reference/channels/slack-provider.md +2 -2
- package/.docs/reference/cli/mastra.md +2 -2
- package/.docs/reference/configuration.md +1 -26
- package/.docs/reference/core/getStorage.md +1 -1
- package/.docs/reference/core/getVector.md +2 -2
- package/.docs/reference/core/listVectors.md +2 -2
- package/.docs/reference/core/setStorage.md +1 -1
- package/.docs/reference/file-based-agents/storage.md +1 -1
- package/.docs/reference/file-based-agents/studio.md +1 -1
- package/.docs/reference/index.md +27 -55
- package/.docs/{guides → reference}/migrations/ai-sdk-v4-to-v5.md +1 -1
- package/.docs/{guides → reference}/migrations/mastra-cloud.md +1 -1
- package/.docs/{guides → reference}/migrations/upgrade-to-v1/cli.md +1 -1
- package/.docs/{guides → reference}/migrations/upgrade-to-v1/memory.md +1 -1
- package/.docs/{guides → reference}/migrations/upgrade-to-v1/overview.md +41 -41
- package/.docs/{guides → reference}/migrations/upgrade-to-v1/tools.md +1 -1
- package/.docs/{guides → reference}/migrations/upgrade-to-v1/tracing.md +10 -10
- package/.docs/reference/observability/tracing/bridges/datadog.md +2 -2
- package/.docs/reference/observability/tracing/bridges/otel.md +2 -2
- package/.docs/reference/observability/tracing/exporters/arize.md +1 -1
- package/.docs/reference/observability/tracing/exporters/arthur.md +1 -1
- package/.docs/reference/observability/tracing/exporters/confident-ai.md +1 -1
- package/.docs/reference/observability/tracing/exporters/mastra-platform-exporter.md +12 -0
- package/.docs/reference/observability/tracing/exporters/otel.md +2 -2
- package/.docs/reference/pubsub/lease-provider.md +1 -1
- package/.docs/{guides → reference}/rag/chunking-and-embedding.md +16 -19
- package/.docs/reference/rag/database-config.md +1 -1
- package/.docs/{guides/rag/graph-rag.md → reference/rag/graph-rag-guide.md} +1 -1
- package/.docs/{guides → reference}/rag/overview.md +2 -2
- package/.docs/{guides → reference}/rag/retrieval.md +1 -1
- package/.docs/{guides → reference}/rag/vector-databases.md +1 -1
- package/.docs/reference/server/create-route.md +27 -1
- package/.docs/reference/storage/composite.md +3 -3
- package/.docs/reference/storage/retention.md +4 -4
- package/.docs/reference/streaming/ChunkType.md +3 -3
- package/.docs/reference/streaming/agents/streamLegacy.md +3 -3
- package/.docs/reference/tools/mcp-server.md +1 -1
- package/.docs/reference/vectors/convex.md +2 -2
- package/.docs/reference/vectors/oracledb.md +2 -2
- package/.docs/reference/workers/overview.md +8 -8
- package/.docs/reference/workflows/run-methods/timeTravel.md +1 -0
- package/.docs/reference/workspace/platform-filesystem.md +2 -2
- package/.docs/reference/workspace/platform-sandbox.md +10 -2
- package/.docs/reference/workspace/process-manager.md +3 -3
- package/.docs/reference/workspace/sandbox.md +10 -0
- package/CHANGELOG.md +16 -0
- package/dist/index.js +1 -1
- package/dist/{src-BZcgzbk9.js → src-D-W-bx5t.js} +2 -2
- package/dist/{src-BZcgzbk9.js.map → src-D-W-bx5t.js.map} +1 -1
- package/dist/stdio.js +1 -1
- package/package.json +5 -5
- package/.docs/docs/capabilities/channels/other-adapters.md +0 -68
- package/.docs/docs/observability/integrations/bridges/datadog.md +0 -219
- package/.docs/docs/observability/integrations/bridges/otel.md +0 -234
- package/.docs/docs/observability/integrations/exporters/datadog.md +0 -321
- package/.docs/guides/build-your-ui/copilotkit/channels.md +0 -86
- package/.docs/guides/build-your-ui/copilotkit/generative-ui.md +0 -174
- package/.docs/guides/guide/chef-michel.md +0 -211
- package/.docs/guides/guide/publishing-mcp-server.md +0 -137
- package/.docs/guides/guide/slack-assistant.md +0 -193
- package/.docs/guides/guide/stock-agent.md +0 -132
- package/.docs/guides/guide/web-search.md +0 -322
- package/.docs/guides/guide/whatsapp-chat-bot.md +0 -407
- package/.docs/guides/voice/realtime-voice.md +0 -430
- package/.docs/reference/voice/google.md +0 -290
- package/.docs/reference/voice/inworld.md +0 -137
- package/.docs/reference/voice/openai.md +0 -96
- package/.docs/reference/voice/playai.md +0 -82
- package/.docs/reference/workspace/vercel-serverless.md +0 -128
- /package/.docs/{guides/concepts → docs/guides}/multi-agent-systems.md +0 -0
- /package/.docs/{guides/concepts → docs/guides}/streaming.md +0 -0
- /package/.docs/{guides/build-your-ui → integrations/agentic-ui}/assistant-ui.md +0 -0
- /package/.docs/{guides/build-your-ui → integrations/agentic-ui}/openui.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/auth0.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/better-auth.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/clerk.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/firebase.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/google.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/okta.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/supabase.md +0 -0
- /package/.docs/{docs/server → integrations}/auth/workos.md +0 -0
- /package/.docs/{guides/deployment → integrations/deploy}/amazon-ec2.md +0 -0
- /package/.docs/{guides/deployment → integrations/deploy}/inngest.md +0 -0
- /package/.docs/{guides/deployment → integrations/deploy}/kubernetes.md +0 -0
- /package/.docs/{guides/deployment → integrations/deploy}/temporal.md +0 -0
- /package/.docs/{reference/workspace/agentfs-filesystem.md → integrations/file-storage/agentfs.md} +0 -0
- /package/.docs/{reference/workspace/google-drive-filesystem.md → integrations/file-storage/google-drive.md} +0 -0
- /package/.docs/{guides/getting-started → integrations/frameworks}/express.md +0 -0
- /package/.docs/{guides/getting-started → integrations/frameworks}/hono.md +0 -0
- /package/.docs/{guides/getting-started → integrations/frameworks}/nestjs.md +0 -0
- /package/.docs/{reference/workspace/agentcore-runtime-sandbox.md → integrations/sandboxes/agentcore.md} +0 -0
- /package/.docs/{reference/workspace/blaxel-sandbox.md → integrations/sandboxes/blaxel.md} +0 -0
- /package/.docs/{guides/guide → integrations/tools}/firecrawl.md +0 -0
- /package/.docs/{reference → integrations}/voice/azure.md +0 -0
- /package/.docs/{reference → integrations}/voice/deepgram.md +0 -0
- /package/.docs/{reference → integrations}/voice/elevenlabs.md +0 -0
- /package/.docs/{reference → integrations}/voice/mistral.md +0 -0
- /package/.docs/{reference → integrations}/voice/murf.md +0 -0
- /package/.docs/{reference → integrations}/voice/sarvam.md +0 -0
- /package/.docs/{reference → integrations}/voice/speechify.md +0 -0
- /package/.docs/{reference/voice/xai-realtime.md → integrations/voice/xai.md} +0 -0
- /package/.docs/{guides/getting-started → reference}/manual-install.md +0 -0
- /package/.docs/{guides → reference}/migrations/agentnetwork.md +0 -0
- /package/.docs/{guides → reference}/migrations/network-to-supervisor.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/agent.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/client.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/deployment.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/evals.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/mastra.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/mcp.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/processors.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/rag.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/storage.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/vectors.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/voice.md +0 -0
- /package/.docs/{guides → reference}/migrations/upgrade-to-v1/workflows.md +0 -0
- /package/.docs/{guides → reference}/migrations/vnext-to-standard-apis.md +0 -0
|
@@ -1,10 +1,413 @@
|
|
|
1
1
|
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
2
|
|
|
3
|
-
#
|
|
3
|
+
# Realtime voice
|
|
4
|
+
|
|
5
|
+
## Quickstart
|
|
6
|
+
|
|
7
|
+
Realtime voice turns a Mastra agent into a live call a user can talk over, in the browser or over the phone. Mastra builds it on [LiveKit](https://livekit.io), an open source WebRTC platform for realtime audio and video.
|
|
8
|
+
|
|
9
|
+
The [`@mastra/livekit`](#api-reference) package connects Mastra agents to the [LiveKit Agents framework](https://docs.livekit.io/agents/): LiveKit owns the audio loop like voice activity detection, streaming speech-to-text, semantic turn detection, barge-in, and text-to-speech. Your Mastra agent generates every reply with its own model, tools, and memory.
|
|
10
|
+
|
|
11
|
+
Use realtime voice when you need low-latency, interruptible voice conversations. For provider-based speech-to-speech without LiveKit, see [Speech to Speech](https://mastra.ai/guides/voice/speech-to-speech).
|
|
12
|
+
|
|
13
|
+
These steps take you from an empty project to a voice agent you can talk to. A voice session has two moving parts you set up here: an API route on your Mastra server that hands out access tokens, and a separate worker process that runs the audio pipeline and calls your agent each turn.
|
|
14
|
+
|
|
15
|
+
1. Install the integration package along with the LiveKit plugins for voice activity detection and turn detection:
|
|
16
|
+
|
|
17
|
+
**npm**:
|
|
18
|
+
|
|
19
|
+
```bash
|
|
20
|
+
npm install @mastra/livekit @livekit/agents @livekit/agents-plugin-silero @livekit/agents-plugin-livekit
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
**pnpm**:
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
pnpm add @mastra/livekit @livekit/agents @livekit/agents-plugin-silero @livekit/agents-plugin-livekit
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
**Yarn**:
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
yarn add @mastra/livekit @livekit/agents @livekit/agents-plugin-silero @livekit/agents-plugin-livekit
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
**Bun**:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
bun add @mastra/livekit @livekit/agents @livekit/agents-plugin-silero @livekit/agents-plugin-livekit
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
2. Set your LiveKit credentials inside an `.env` file. Create a free project on [LiveKit Cloud](https://cloud.livekit.io), or run a local server with [`livekit-server --dev`](https://docs.livekit.io/home/self-hosting/local/):
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
LIVEKIT_URL=wss://your-project.livekit.cloud
|
|
45
|
+
LIVEKIT_API_KEY=your-api-key
|
|
46
|
+
LIVEKIT_API_SECRET=your-api-secret
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
3. Add a voice agent to your Mastra instance and expose a connection route. The `liveKitConnectionRoute()` helper adds a `POST /voice/livekit/connection-details` endpoint that mints a LiveKit token and dispatches your agent into a room:
|
|
50
|
+
|
|
51
|
+
```typescript
|
|
52
|
+
import { Mastra } from '@mastra/core/mastra'
|
|
53
|
+
import { Agent } from '@mastra/core/agent'
|
|
54
|
+
import { liveKitConnectionRoute } from '@mastra/livekit'
|
|
55
|
+
|
|
56
|
+
const supportAgent = new Agent({
|
|
57
|
+
id: 'support',
|
|
58
|
+
name: 'Support',
|
|
59
|
+
instructions: 'You are a friendly phone support agent. Keep replies short and conversational.',
|
|
60
|
+
model: 'openai/gpt-5-mini',
|
|
61
|
+
})
|
|
62
|
+
|
|
63
|
+
export const mastra = new Mastra({
|
|
64
|
+
agents: { support: supportAgent },
|
|
65
|
+
server: {
|
|
66
|
+
apiRoutes: [liveKitConnectionRoute({ agentName: 'mastra-voice' })],
|
|
67
|
+
},
|
|
68
|
+
})
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
4. Create the worker. It runs as a separate process, answers LiveKit sessions, and calls your agent each turn. Worker APIs live on the `@mastra/livekit/worker` entry point, so the Mastra server never loads the LiveKit agents runtime. This example uses LiveKit Inference model strings for speech-to-text and text-to-speech, so no provider plugins are required:
|
|
72
|
+
|
|
73
|
+
```typescript
|
|
74
|
+
import { fileURLToPath } from 'node:url'
|
|
75
|
+
import { createLiveKitWorker, runLiveKitWorker } from '@mastra/livekit/worker'
|
|
76
|
+
import { mastra } from './index'
|
|
77
|
+
|
|
78
|
+
export default createLiveKitWorker({
|
|
79
|
+
mastra,
|
|
80
|
+
agent: 'support',
|
|
81
|
+
stt: 'deepgram/nova-3',
|
|
82
|
+
tts: 'cartesia/sonic-3',
|
|
83
|
+
turnDetection: 'multilingual',
|
|
84
|
+
greeting: 'Hi! How can I help you today?',
|
|
85
|
+
})
|
|
86
|
+
|
|
87
|
+
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
|
88
|
+
runLiveKitWorker({ entry: import.meta.url, agentName: 'mastra-voice' })
|
|
89
|
+
}
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
The `agent` option selects which Mastra agent answers each session. Pass a fixed key as shown, or omit it to use the `agentId` from the dispatch metadata, so one worker can serve every agent on your Mastra instance.
|
|
93
|
+
|
|
94
|
+
5. Download the turn detection and voice activity detection models once. Then run the worker in one terminal and your Mastra server in another:
|
|
95
|
+
|
|
96
|
+
```bash
|
|
97
|
+
npx livekit-agents download-files
|
|
98
|
+
npx tsx src/mastra/voice-worker.ts dev
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
**npm**:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
npm run dev
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**pnpm**:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
pnpm run dev
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
**Yarn**:
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
yarn dev
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
**Bun**:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
bun run dev
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
The worker registers with your LiveKit server and waits for sessions, while `mastra dev` serves the connection route.
|
|
126
|
+
|
|
127
|
+
6. Talk to your agent. Open the hosted [LiveKit Agents Playground](https://agents-playground.livekit.io) and connect it to your project to start a call without building a frontend.
|
|
128
|
+
|
|
129
|
+
To wire up your own app instead, call the connection route for a token. `POST /voice/livekit/connection-details` accepts optional `agentId`, `threadId`, and `resourceId` fields in the request body and returns:
|
|
130
|
+
|
|
131
|
+
```json
|
|
132
|
+
{
|
|
133
|
+
"serverUrl": "wss://your-project.livekit.cloud",
|
|
134
|
+
"roomName": "mastra-voice-a1b2c3d4",
|
|
135
|
+
"participantName": "user-1",
|
|
136
|
+
"participantToken": "eyJhbGci..."
|
|
137
|
+
}
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
This response matches the contract used by LiveKit's frontend starters, so apps built from [agent-starter-react](https://github.com/livekit-examples/agent-starter-react) or the [LiveKit React components](https://docs.livekit.io/reference/components/react/) work without changes.
|
|
141
|
+
|
|
142
|
+
### Turn detection and interruptions
|
|
143
|
+
|
|
144
|
+
LiveKit decides when the user finished speaking and when the agent was interrupted. The defaults work well; tune them with `turnHandling`:
|
|
145
|
+
|
|
146
|
+
```typescript
|
|
147
|
+
export default createLiveKitWorker({
|
|
148
|
+
mastra,
|
|
149
|
+
agent: 'support',
|
|
150
|
+
stt: 'deepgram/nova-3',
|
|
151
|
+
tts: 'cartesia/sonic-3',
|
|
152
|
+
turnDetection: 'multilingual',
|
|
153
|
+
turnHandling: {
|
|
154
|
+
endpointing: { mode: 'dynamic', minDelay: 300, maxDelay: 3000 },
|
|
155
|
+
interruption: { minDuration: 500, resumeFalseInterruption: true },
|
|
156
|
+
},
|
|
157
|
+
})
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
- `turnDetection: 'multilingual'`: Runs LiveKit's semantic end-of-turn model locally on CPU. It reads the live transcript to avoid cutting users off mid-thought. Use `'vad'` or `'stt'` for silence-based endpointing instead.
|
|
161
|
+
- `endpointing`: Bounds how long the agent waits after the user stops speaking.
|
|
162
|
+
- `interruption`: Controls barge-in. When the user speaks over the agent, LiveKit stops playback and cancels the in-flight Mastra stream, so token generation stops too.
|
|
163
|
+
- `preemptiveGeneration`: Starts the Mastra agent's reply while the user is still finishing, hiding time-to-first-token. The worker disables it by default: each preemptive attempt runs the Mastra agent on an interim transcript, and every run persists the user message, which duplicates messages in the thread. Re-enable it with `preemptiveGeneration: { enabled: true }` if latency matters more than exact thread history.
|
|
164
|
+
|
|
165
|
+
See the [LiveKit turn detection docs](https://docs.livekit.io/agents/logic/turns/) for all options.
|
|
166
|
+
|
|
167
|
+
### Per-call voices and transcription
|
|
168
|
+
|
|
169
|
+
The top-level `stt` and `tts` options apply to every call. To pick them per call, one voice or language per tenant, set the `configuration.stt` and `configuration.tts` resolvers instead. Each resolver runs once per call with the dispatch metadata, request context, room name, and job context, and returns a value accepted by the matching top-level option. The value is either a plugin instance or an inference model string. Return `undefined` to fall back to the top-level option.
|
|
170
|
+
|
|
171
|
+
The following example gives each tenant its own text-to-speech voice, keyed off the `tenant` entry in the dispatch metadata:
|
|
172
|
+
|
|
173
|
+
```typescript
|
|
174
|
+
import * as cartesia from '@livekit/agents-plugin-cartesia'
|
|
175
|
+
|
|
176
|
+
// One voice id per tenant, resolved from the dispatch metadata on each call.
|
|
177
|
+
const tenantVoices: Record<string, string> = {
|
|
178
|
+
meridian: 'your-cartesia-voice-id-1',
|
|
179
|
+
coastal: 'your-cartesia-voice-id-2',
|
|
180
|
+
}
|
|
181
|
+
// The resolver runs during call setup, so cache plugin instances across calls.
|
|
182
|
+
const ttsByVoice = new Map<string, cartesia.TTS>()
|
|
183
|
+
|
|
184
|
+
export default createLiveKitWorker({
|
|
185
|
+
mastra,
|
|
186
|
+
agent: 'support',
|
|
187
|
+
stt: 'deepgram/nova-3',
|
|
188
|
+
tts: 'cartesia/sonic-3',
|
|
189
|
+
configuration: {
|
|
190
|
+
tts: ({ requestContext }) => {
|
|
191
|
+
const voice = tenantVoices[requestContext?.tenant as string]
|
|
192
|
+
if (!voice) return undefined // fall back to the top-level `tts`
|
|
193
|
+
let tts = ttsByVoice.get(voice)
|
|
194
|
+
if (!tts) {
|
|
195
|
+
tts = new cartesia.TTS({ voice })
|
|
196
|
+
ttsByVoice.set(voice, tts)
|
|
197
|
+
}
|
|
198
|
+
return tts
|
|
199
|
+
},
|
|
200
|
+
},
|
|
201
|
+
})
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
`configuration.stt` works the same way for per-call transcription, for example a different transcription model or language per tenant. The greeting has a matching per-call form: `configuration.greeting.text` accepts a resolver with the same call context, so one worker can open with each tenant's own phrasing.
|
|
205
|
+
|
|
206
|
+
### Memory and threads
|
|
207
|
+
|
|
208
|
+
When the resolved Mastra agent has memory configured, each call becomes one memory thread:
|
|
209
|
+
|
|
210
|
+
- `thread` defaults to the `threadId` from dispatch metadata, then to the room name.
|
|
211
|
+
- `resource` defaults to the `resourceId` from dispatch metadata, then to the thread. Send your end user's id here so calls group under the right user. Mastra Studio sends the agent id, matching how its sidebar lists threads.
|
|
212
|
+
- When the thread doesn't exist yet, the worker creates it titled "Voice call" with metadata `{ source: 'livekit' }`, and the spoken greeting is saved as the first assistant message so the thread reads as a full call transcript (disable with `persistGreeting: false`).
|
|
213
|
+
|
|
214
|
+
Each turn sends only the new user input; Mastra Memory supplies history, semantic recall, and working memory. Pin a session to an existing thread by passing `threadId` in the connection request body, which is useful for continuing a text conversation by voice. In Studio, starting a call from an open chat binds the call to that thread, and the transcript fills into the chat after each exchange.
|
|
215
|
+
|
|
216
|
+
When a user interrupts the agent, the in-flight generation aborts and nothing from that turn is persisted at that moment. LiveKit keeps the part the user actually heard in its transcript, and on the next turn the worker re-sends that heard-only fragment so the thread backfills to match the call. A user who hangs up right after interrupting leaves that final fragment unrecorded. See [interrupted turns](#interrupted-turns) for the details and a reconciliation recipe.
|
|
217
|
+
|
|
218
|
+
### Speak while tools run
|
|
219
|
+
|
|
220
|
+
Voice conversations can't go silent while a slow tool runs. Use `toolFeedback` to speak a short phrase when the Mastra agent starts a tool call:
|
|
221
|
+
|
|
222
|
+
```typescript
|
|
223
|
+
export default createLiveKitWorker({
|
|
224
|
+
mastra,
|
|
225
|
+
agent: 'support',
|
|
226
|
+
stt: 'deepgram/nova-3',
|
|
227
|
+
tts: 'cartesia/sonic-3',
|
|
228
|
+
toolFeedback: ({ toolName }) =>
|
|
229
|
+
toolName === 'searchOrders' ? 'Let me look that up.' : undefined,
|
|
230
|
+
})
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
The phrase is spoken as part of the reply and recorded in the transcript.
|
|
234
|
+
|
|
235
|
+
### Generate replies with a workflow
|
|
236
|
+
|
|
237
|
+
By default the worker generates each reply with a Mastra agent. To run multi-step logic per turn (for example classify intent, route, call tools in sequence, then compose a reply), generate replies with a Mastra [workflow](https://mastra.ai/docs/workflows/overview) instead. Set `workflow` in place of `agent`.
|
|
238
|
+
|
|
239
|
+
LiveKit still owns the audio loop and calls into Mastra once per turn, so the workflow runs to completion each turn. The workflow can't suspend or resume, and no conversation state carries between turns. Pass the transcript in through `workflowInput` so the workflow stays stateless:
|
|
240
|
+
|
|
241
|
+
```typescript
|
|
242
|
+
import { createLiveKitWorker, chatContextToMessages } from '@mastra/livekit/worker'
|
|
243
|
+
import { mastra } from './index'
|
|
244
|
+
|
|
245
|
+
export default createLiveKitWorker({
|
|
246
|
+
mastra,
|
|
247
|
+
workflow: 'phoneConversation',
|
|
248
|
+
workflowInput: ({ chatCtx }) => ({ history: chatContextToMessages(chatCtx) }),
|
|
249
|
+
replyStep: 'generateResponse',
|
|
250
|
+
stt: 'deepgram/nova-3',
|
|
251
|
+
tts: 'cartesia/sonic-3',
|
|
252
|
+
turnDetection: 'multilingual',
|
|
253
|
+
})
|
|
254
|
+
```
|
|
255
|
+
|
|
256
|
+
A workflow streams structured step events, not text. To speak tokens as they generate, the reply step pipes its agent's text into the step `writer`:
|
|
257
|
+
|
|
258
|
+
```typescript
|
|
259
|
+
const generateResponse = createStep({
|
|
260
|
+
id: 'generateResponse',
|
|
261
|
+
// input and output schemas omitted
|
|
262
|
+
execute: async ({ inputData, mastra, writer, abortSignal }) => {
|
|
263
|
+
const stream = await mastra.getAgent('voice').stream(inputData.history, { abortSignal })
|
|
264
|
+
await stream.textStream.pipeTo(writer)
|
|
265
|
+
return { assistantMessage: await stream.text }
|
|
266
|
+
},
|
|
267
|
+
})
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
- `replyStep`: Restricts spoken output to one step. Omit it to speak every step that writes to its `writer`.
|
|
271
|
+
- `resultText`: A fallback that derives the reply from the final run result when no step streams text. Streaming through `writer` gives lower time-to-first-token, so prefer it.
|
|
272
|
+
- `abortSignal`: Forward the step's `abortSignal` into `agent.stream()` so barge-in stops generation promptly. When the user interrupts, the worker cancels the run.
|
|
273
|
+
- `generate`: For full control, pass a `generate` function instead. It can be any reply generator that turns a turn into a text stream.
|
|
274
|
+
|
|
275
|
+
With a workflow, the worker doesn't persist turns automatically the way an agent's `stream()` does. Persist conversation history inside the workflow, or keep the LiveKit transcript as the source of truth and pass it in each turn.
|
|
276
|
+
|
|
277
|
+
### Use Mastra as the LLM component
|
|
278
|
+
|
|
279
|
+
`createLiveKitWorker()` owns the LiveKit session for you. To own the session yourself, use [`MastraLLM`](#mastrallm) instead: a standard LiveKit LLM plugin that puts a Mastra agent in the `llm` slot of your own `voice.AgentSession`. The Mastra app, agent loop, tools, memory, observability, runs on your Mastra server, and the worker reaches it over HTTP. The worker process needs no Mastra app, database, or model provider keys.
|
|
280
|
+
|
|
281
|
+
```typescript
|
|
282
|
+
import { fileURLToPath } from 'node:url'
|
|
283
|
+
import { defineAgent, voice } from '@livekit/agents'
|
|
284
|
+
import * as silero from '@livekit/agents-plugin-silero'
|
|
285
|
+
import { MastraLLM } from '@mastra/livekit/plugin'
|
|
286
|
+
import { runLiveKitWorker } from '@mastra/livekit/worker'
|
|
287
|
+
|
|
288
|
+
export default defineAgent({
|
|
289
|
+
entry: async ctx => {
|
|
290
|
+
await ctx.connect()
|
|
291
|
+
|
|
292
|
+
const session = new voice.AgentSession({
|
|
293
|
+
llm: new MastraLLM({
|
|
294
|
+
remote: { baseUrl: process.env.MASTRA_URL!, agentId: 'support' },
|
|
295
|
+
memory: { thread: ctx.room.name!, resource: 'user-7' },
|
|
296
|
+
}),
|
|
297
|
+
stt: 'deepgram/nova-3',
|
|
298
|
+
tts: 'cartesia/sonic-3',
|
|
299
|
+
vad: await silero.VAD.load(),
|
|
300
|
+
// Required with `memory`: LiveKit enables preemptive generation by default.
|
|
301
|
+
turnHandling: { preemptiveGeneration: { enabled: false } },
|
|
302
|
+
})
|
|
303
|
+
|
|
304
|
+
await session.start({
|
|
305
|
+
// These instructions never reach the Mastra agent; its own instructions apply.
|
|
306
|
+
agent: new voice.Agent({ instructions: 'Replies come from the Mastra agent.' }),
|
|
307
|
+
room: ctx.room,
|
|
308
|
+
})
|
|
309
|
+
|
|
310
|
+
session.say('Hi! How can I help you today?')
|
|
311
|
+
},
|
|
312
|
+
})
|
|
313
|
+
|
|
314
|
+
if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
|
315
|
+
runLiveKitWorker({ entry: import.meta.url, agentName: 'mastra-voice' })
|
|
316
|
+
}
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
Both paths share the same reply pipeline underneath; choose by who should own the session:
|
|
320
|
+
|
|
321
|
+
| | `createLiveKitWorker()` | `MastraLLM` |
|
|
322
|
+
| ------------------------- | ------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- |
|
|
323
|
+
| Session ownership | The worker helper builds and manages the `AgentSession` | Your code builds the session; every LiveKit option and hook is yours |
|
|
324
|
+
| Where the Mastra app runs | In the worker process | On your Mastra server, reached over HTTP (or in-process via `agent`) |
|
|
325
|
+
| Worker process needs | Your Mastra app, storage, and model provider keys | Only the LiveKit SDK and network access to your server |
|
|
326
|
+
| Built-in conveniences | Greeting, consent gating, agent-initiated hang-up, thread bootstrap, observability roll-up | Rebuild what you need with the [session helpers](#runendcall) |
|
|
327
|
+
| Best for | Fastest path to a working voice agent; Studio voice mode | Existing LiveKit apps and full control over the session |
|
|
328
|
+
|
|
329
|
+
Tools stay on the Mastra agent and execute on the server. LiveKit-side tools passed to the session are ignored. Tool activity reaches the worker through `toolFeedback` (spoken filler), `onToolCall` (fires as each tool call starts), and `onTurnComplete` (fires after each reply with the text, tool calls, and token usage). Agent-initiated hang-up takes a few lines: pair `onToolCall` with [`runEndCall()`](#runendcall).
|
|
330
|
+
|
|
331
|
+
> **Warning:** Don't combine the `memory` option with LiveKit's `preemptiveGeneration`, which LiveKit enables by default in sessions you build yourself. A speculative turn that completes before LiveKit discards it persists a user message and a never-spoken reply to the thread. Set `turnHandling: { preemptiveGeneration: { enabled: false } }`, or run without `memory` and pass the full transcript each turn.
|
|
332
|
+
|
|
333
|
+
`MastraLLM` also accepts an in-process Mastra `agent` instance, session ownership without a second deployment, or a custom `generate` function. The remote transport is available standalone as [`createRemoteAgentReplyGenerator()`](#createremoteagentreplygenerator), which also plugs into `createLiveKitWorker`'s `generate` option to run the batteries-included worker against a remote server.
|
|
334
|
+
|
|
335
|
+
### Server-initiated sessions
|
|
336
|
+
|
|
337
|
+
Use `dispatchVoiceSession()` to add a voice agent to a room from your own code, for example to join an existing room or to drive an outbound [SIP call](https://docs.livekit.io/sip/):
|
|
338
|
+
|
|
339
|
+
```typescript
|
|
340
|
+
import { dispatchVoiceSession } from '@mastra/livekit'
|
|
341
|
+
|
|
342
|
+
await dispatchVoiceSession({
|
|
343
|
+
roomName: 'support-call-42',
|
|
344
|
+
agentName: 'mastra-voice',
|
|
345
|
+
metadata: { agentId: 'support', threadId: 'thread-42', resourceId: 'user-7' },
|
|
346
|
+
})
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
### Observability
|
|
350
|
+
|
|
351
|
+
When the Mastra instance has [observability](https://mastra.ai/docs/observability/overview) configured, the worker traces each call. It opens one `voice call` span per session and nests everything under it:
|
|
352
|
+
|
|
353
|
+
- Every turn's Mastra agent run, with model generation, tool calls, and memory operations, exactly as a text chat records them.
|
|
354
|
+
- A child span for each LiveKit pipeline metric: speech-to-text, text-to-speech, end-of-utterance (turn detection), voice activity detection, and the model's time-to-first-token. These carry the latency and audio measurements that text traces can't show.
|
|
355
|
+
- A per-model usage roll-up (token, character, and audio totals for the whole call) written to the span when the session ends.
|
|
356
|
+
|
|
357
|
+
The worker is a separate process, so point storage at a backend that accepts concurrent writes from both the server and the worker. SQLite-backed [LibSQL](https://mastra.ai/integrations/databases/libsql) works. Single-writer stores don't. Traces, memory, and threads can share one store:
|
|
358
|
+
|
|
359
|
+
```typescript
|
|
360
|
+
import { Mastra } from '@mastra/core/mastra'
|
|
361
|
+
import { LibSQLStore } from '@mastra/libsql'
|
|
362
|
+
import { Observability, MastraStorageExporter } from '@mastra/observability'
|
|
363
|
+
|
|
364
|
+
export const mastra = new Mastra({
|
|
365
|
+
storage: new LibSQLStore({ id: 'voice-agent-storage', url: 'file:./voice-agent.db' }),
|
|
366
|
+
observability: new Observability({
|
|
367
|
+
configs: {
|
|
368
|
+
default: {
|
|
369
|
+
serviceName: 'voice-agent',
|
|
370
|
+
exporters: [new MastraStorageExporter()],
|
|
371
|
+
},
|
|
372
|
+
},
|
|
373
|
+
}),
|
|
374
|
+
})
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
Tracing is on by default. Pass `observability: false` to `createLiveKitWorker` to turn it off.
|
|
378
|
+
|
|
379
|
+
### Deployment
|
|
380
|
+
|
|
381
|
+
The worker is a separate process from your Mastra server. Deploy it as a long-running Node service with the production command:
|
|
382
|
+
|
|
383
|
+
```bash
|
|
384
|
+
node dist/voice-worker.js start
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
LiveKit's guidance on sizing, graceful shutdown, and hosting applies unchanged. See [Deploying agents](https://docs.livekit.io/agents/ops/deployment/). Workers connect outbound to LiveKit, so they don't need inbound ports.
|
|
388
|
+
|
|
389
|
+
### How it works
|
|
390
|
+
|
|
391
|
+
A LiveKit voice session involves three pieces:
|
|
392
|
+
|
|
393
|
+
1. Your Mastra server mints a LiveKit access token and dispatches your agent into a room. The dispatch carries metadata such as the Mastra agent id, memory thread, and resource.
|
|
394
|
+
2. A LiveKit agent worker (a separate long-running process) receives the job and runs the audio pipeline. Audio flows between the browser and the worker over WebRTC and never passes through your Mastra HTTP server.
|
|
395
|
+
3. Each time the user finishes a turn, the worker calls the Mastra agent's `stream()` with the new input and speaks the streamed text. When the user interrupts, LiveKit cancels the stream and Mastra stops generating.
|
|
396
|
+
|
|
397
|
+
Conversation history lives in Mastra Memory, so voice sessions and text chat can share one thread.
|
|
398
|
+
|
|
399
|
+
### Related
|
|
400
|
+
|
|
401
|
+
- [`@mastra/livekit` reference](#api-reference)
|
|
402
|
+
- [Speech to Speech](https://mastra.ai/guides/voice/speech-to-speech)
|
|
403
|
+
- [Agent Memory](https://mastra.ai/docs/memory/overview)
|
|
404
|
+
- [LiveKit Agents docs](https://docs.livekit.io/agents/)
|
|
405
|
+
|
|
406
|
+
## API reference
|
|
4
407
|
|
|
5
408
|
The `@mastra/livekit` package connects Mastra agents to the LiveKit Agents framework. LiveKit runs the audio pipeline (voice activity detection, speech-to-text, turn detection, text-to-speech, barge-in) and the package bridges reply generation to a Mastra agent's `stream()` call.
|
|
6
409
|
|
|
7
|
-
See [Realtime voice](
|
|
410
|
+
See [Realtime voice](#quickstart) for setup and concepts.
|
|
8
411
|
|
|
9
412
|
The package has three entry points:
|
|
10
413
|
|
|
@@ -12,7 +415,7 @@ The package has three entry points:
|
|
|
12
415
|
- `@mastra/livekit/worker`: the worker runtime, [`createLiveKitWorker()`](#createlivekitworker), [`runLiveKitWorker()`](#runlivekitworker), [`chatContextToMessages()`](#chatcontexttomessages), and the session helpers [`speakGreeting()`](#speakgreeting), [`waitForAgentDoneSpeaking()`](#waitforagentdonespeaking), and [`runEndCall()`](#runendcall). Import it only from the worker entry file.
|
|
13
416
|
- `@mastra/livekit/plugin`: the LLM-component plugin, [`MastraLLM`](#mastrallm) and [`createRemoteAgentReplyGenerator()`](#createremoteagentreplygenerator). Import it in workers that build their own `voice.AgentSession`. `createRemoteAgentReplyGenerator()` is also exported from `@mastra/livekit/worker` because it plugs into `createLiveKitWorker()`'s `generate` option. `MastraLLM` is plugin-only.
|
|
14
417
|
|
|
15
|
-
|
|
418
|
+
### `createLiveKitWorker()`
|
|
16
419
|
|
|
17
420
|
Builds a LiveKit agent definition that answers voice sessions with Mastra agents. Use it as the default export of your worker entry file.
|
|
18
421
|
|
|
@@ -34,7 +437,7 @@ if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
|
|
34
437
|
}
|
|
35
438
|
```
|
|
36
439
|
|
|
37
|
-
|
|
440
|
+
#### Options
|
|
38
441
|
|
|
39
442
|
**mastra** (`Mastra`): The Mastra instance whose agents handle voice sessions.
|
|
40
443
|
|
|
@@ -92,11 +495,11 @@ if (process.argv[1] === fileURLToPath(import.meta.url)) {
|
|
|
92
495
|
|
|
93
496
|
**onSessionStart** (`(args: { session, ctx, agent, metadata }) => void | Promise<void>`): Called after the session starts. Attach event listeners or trigger replies here.
|
|
94
497
|
|
|
95
|
-
|
|
498
|
+
### `runLiveKitWorker()`
|
|
96
499
|
|
|
97
500
|
Starts the LiveKit worker CLI (`dev`, `start`, and `connect` subcommands) for a worker entry file. Call it from the file that default-exports the worker definition, guarded so it only runs when executed directly (the worker spawns a child process per session that re-imports the same file). Using this helper instead of `cli.runApp` from `@livekit/agents` guarantees the worker runtime and the bridge share one copy of the LiveKit SDK.
|
|
98
501
|
|
|
99
|
-
|
|
502
|
+
#### Options
|
|
100
503
|
|
|
101
504
|
**entry** (`string | URL`): The worker entry module whose default export is the agent definition. Pass import.meta.url.
|
|
102
505
|
|
|
@@ -104,7 +507,7 @@ Starts the LiveKit worker CLI (`dev`, `start`, and `connect` subcommands) for a
|
|
|
104
507
|
|
|
105
508
|
**serverOptions** (`Partial<ServerOptions>`): Extra LiveKit ServerOptions merged over what this helper builds.
|
|
106
509
|
|
|
107
|
-
|
|
510
|
+
### `pipeAgentReplyToWriter()`
|
|
108
511
|
|
|
109
512
|
Streams a Mastra agent's reply into a workflow step's `writer` on the workflow reply path. It forwards the agent's text deltas, so text-to-speech starts before the full reply is ready, and its tool-call chunks, so `toolFeedback` fires and `onTurnComplete` sees the tool list. Piping only `stream.textStream` silently drops tool calls. Pass the step's `abortSignal` to `agent.stream()` so barge-in stops generation promptly.
|
|
110
513
|
|
|
@@ -124,13 +527,13 @@ const generateResponse = createStep({
|
|
|
124
527
|
|
|
125
528
|
Returns: `Promise<string>`, the accumulated reply text.
|
|
126
529
|
|
|
127
|
-
|
|
530
|
+
#### Parameters
|
|
128
531
|
|
|
129
532
|
**agentStream** (`AgentReplyStreamLike`): The stream returned by agent.stream() — anything exposing a fullStream async iterable.
|
|
130
533
|
|
|
131
534
|
**writer** (`WritableStream<unknown>`): The workflow step's writer.
|
|
132
535
|
|
|
133
|
-
|
|
536
|
+
### `chatContextToMessages()`
|
|
134
537
|
|
|
135
538
|
Converts a LiveKit chat context into plain messages accepted by `agent.stream()`, excluding instructions and function calls. Use it in `workflowInput` to pass the full transcript into a stateless workflow.
|
|
136
539
|
|
|
@@ -146,9 +549,9 @@ export default createLiveKitWorker({
|
|
|
146
549
|
|
|
147
550
|
Returns: `VoiceTurnMessage[]`, where each entry is `{ role: 'system' | 'user' | 'assistant'; content: string; id?: string }`.
|
|
148
551
|
|
|
149
|
-
|
|
552
|
+
### `MastraLLM`
|
|
150
553
|
|
|
151
|
-
A standard LiveKit LLM plugin (`llm.LLM`) backed by a Mastra agent. Use it when you build the `voice.AgentSession` yourself and want Mastra in the `llm` slot. [`createLiveKitWorker()`](#createlivekitworker) is the managed alternative. See [Use Mastra as the LLM component](
|
|
554
|
+
A standard LiveKit LLM plugin (`llm.LLM`) backed by a Mastra agent. Use it when you build the `voice.AgentSession` yourself and want Mastra in the `llm` slot. [`createLiveKitWorker()`](#createlivekitworker) is the managed alternative. See [Use Mastra as the LLM component](#use-mastra-as-the-llm-component) for how to choose.
|
|
152
555
|
|
|
153
556
|
With `remote`, the plugin streams each turn from your Mastra server over HTTP using Server-Sent Events (SSE). The agent loop, tools, and memory run server-side, and interrupting the agent aborts the server-side generation.
|
|
154
557
|
|
|
@@ -170,7 +573,7 @@ const session = new voice.AgentSession({
|
|
|
170
573
|
|
|
171
574
|
The plugin reports `provider` as `mastra` and `model` as the agent id, so LiveKit metrics and fallback adapters identify it like any other LLM.
|
|
172
575
|
|
|
173
|
-
|
|
576
|
+
#### Constructor options
|
|
174
577
|
|
|
175
578
|
Provide exactly one reply source: `remote`, `agent`, or `generate`.
|
|
176
579
|
|
|
@@ -192,17 +595,17 @@ Provide exactly one reply source: `remote`, `agent`, or `generate`.
|
|
|
192
595
|
|
|
193
596
|
> **Warning:** Don't combine `memory` with the session's `preemptiveGeneration` option, which LiveKit enables by default in sessions you build yourself. A speculative turn that completes before LiveKit discards it persists a user message and a never-spoken reply to the thread. Set `turnHandling: { preemptiveGeneration: { enabled: false } }` on the session. Stateless mode (no `memory`) works with preemptive generation.
|
|
194
597
|
|
|
195
|
-
|
|
598
|
+
#### Tools run on the Mastra agent
|
|
196
599
|
|
|
197
600
|
Tools are defined and executed server-side on the Mastra agent. The plugin never forwards LiveKit tool definitions: if the session passes a non-empty `toolCtx`, it logs a one-time warning naming the ignored tools. Every tool must complete server-side: a tool that requires approval or client-side execution fails the turn with a descriptive error instead of hanging the call.
|
|
198
601
|
|
|
199
602
|
Tool activity reaches the worker through `toolFeedback`, `onToolCall`, and `onTurnComplete`.
|
|
200
603
|
|
|
201
|
-
|
|
604
|
+
#### Instructions
|
|
202
605
|
|
|
203
606
|
LiveKit injects your `voice.Agent`'s `instructions` into the chat context of every request. The plugin drops them because the server-side Mastra agent's own instructions are authoritative. To change the prompt, change the Mastra agent.
|
|
204
607
|
|
|
205
|
-
|
|
608
|
+
#### Interrupted turns
|
|
206
609
|
|
|
207
610
|
When the user interrupts a reply:
|
|
208
611
|
|
|
@@ -237,11 +640,11 @@ session.on(voice.AgentSessionEventTypes.ConversationItemAdded, ({ item }) => {
|
|
|
237
640
|
})
|
|
238
641
|
```
|
|
239
642
|
|
|
240
|
-
|
|
643
|
+
#### Usage metrics
|
|
241
644
|
|
|
242
645
|
When the server reports token usage for a turn, the plugin feeds it to LiveKit, so the session's `metrics_collected` events carry time-to-first-token, duration, and token counts like any LLM plugin. The same usage object (`promptTokens`, `completionTokens`, `promptCachedTokens`, `totalTokens`) arrives on `onTurnComplete` as `result.usage`.
|
|
243
646
|
|
|
244
|
-
|
|
647
|
+
#### Errors and timeouts
|
|
245
648
|
|
|
246
649
|
The transport throws LiveKit's `APIError` types (`APIStatusError`, `APIConnectionError`, `APITimeoutError`), so the session's retry policy (`connOptions.maxRetry`) and `FallbackAdapter` failover work unchanged. A turn is never retried after its first token: a voice reply is better failed fast than replayed half-heard.
|
|
247
650
|
|
|
@@ -249,11 +652,11 @@ A connect and first-token watchdog uses the session's `connOptions.timeoutMs` (1
|
|
|
249
652
|
|
|
250
653
|
If the Mastra server goes down mid-call, each reply attempt fails with a typed error after its retries, and LiveKit closes the session after several consecutive failed replies. Restore the server before that budget runs out and the call recovers on the next turn.
|
|
251
654
|
|
|
252
|
-
|
|
655
|
+
#### Message content
|
|
253
656
|
|
|
254
657
|
Message extraction is text-only: image content is dropped, and audio content is included only through its transcript. Voice pipelines aren't affected, but items you inject into the chat context yourself must carry text.
|
|
255
658
|
|
|
256
|
-
|
|
659
|
+
### `createRemoteAgentReplyGenerator()`
|
|
257
660
|
|
|
258
661
|
Builds a reply generator that runs the agent loop on a **remote** Mastra server over HTTP/SSE. `MastraLLM`'s `remote` mode uses it internally. Use it directly through `createLiveKitWorker`'s `generate` option to run the batteries-included worker against a remote server:
|
|
259
662
|
|
|
@@ -279,7 +682,7 @@ Cancelling a turn (barge-in) tears down the HTTP request, which aborts generatio
|
|
|
279
682
|
|
|
280
683
|
Returns: `VoiceReplyGenerator`.
|
|
281
684
|
|
|
282
|
-
|
|
685
|
+
#### Options
|
|
283
686
|
|
|
284
687
|
**baseUrl** (`string`): Base URL of the remote Mastra server, for example https\://my-app.example.com.
|
|
285
688
|
|
|
@@ -303,7 +706,7 @@ Returns: `VoiceReplyGenerator`.
|
|
|
303
706
|
|
|
304
707
|
**onTurnComplete** (`(ctx: VoiceTurnCompleteContext) => void | Promise<void>`): Called once per turn after the reply finished streaming, off the audio path.
|
|
305
708
|
|
|
306
|
-
|
|
709
|
+
### `speakGreeting()`
|
|
307
710
|
|
|
308
711
|
Speaks an opening greeting on a session you own, honoring interruption and playout options. Returns the LiveKit `SpeechHandle`, or `undefined` when there's no greeting text. `createLiveKitWorker()` uses it internally for its `greeting` configuration.
|
|
309
712
|
|
|
@@ -317,13 +720,13 @@ await speakGreeting(session, {
|
|
|
317
720
|
})
|
|
318
721
|
```
|
|
319
722
|
|
|
320
|
-
|
|
723
|
+
#### Parameters
|
|
321
724
|
|
|
322
725
|
**session** (`voice.AgentSession`): The session to speak on.
|
|
323
726
|
|
|
324
727
|
**greeting** (`{ text?: string; allowInterruptions?: boolean; awaitPlayout?: boolean }`): The greeting text and playout options. When awaitPlayout is true, the returned promise resolves after the greeting finished playing (or was interrupted).
|
|
325
728
|
|
|
326
|
-
|
|
729
|
+
### `waitForAgentDoneSpeaking()`
|
|
327
730
|
|
|
328
731
|
Resolves once the agent is no longer producing or playing a reply: its state has left `thinking` and `speaking`. Resolves immediately when the agent is already idle, and always resolves within `maxWaitMs` (30 seconds by default) as a safety cap. Use it before tearing a session down so closing words play out instead of being cut off.
|
|
329
732
|
|
|
@@ -333,7 +736,7 @@ import { waitForAgentDoneSpeaking } from '@mastra/livekit/worker'
|
|
|
333
736
|
await waitForAgentDoneSpeaking(session)
|
|
334
737
|
```
|
|
335
738
|
|
|
336
|
-
|
|
739
|
+
### `runEndCall()`
|
|
337
740
|
|
|
338
741
|
Ends the call after the agent asks to hang up. It waits for the agent's closing words and speaks an optional final `message` without interruption. It then deletes the room and hangs up the caller, including SIP callers. The job shuts down with its registered callbacks.
|
|
339
742
|
|
|
@@ -357,7 +760,7 @@ const llm = new MastraLLM({
|
|
|
357
760
|
|
|
358
761
|
The exported constants `DEFAULT_END_CALL_TOOL` (`'endCall'`), `DEFAULT_END_CALL_REASON`, and `DEFAULT_END_CALL_MAX_WAIT_MS` (30000) hold the defaults.
|
|
359
762
|
|
|
360
|
-
|
|
763
|
+
#### Parameters
|
|
361
764
|
|
|
362
765
|
**session** (`voice.AgentSession`): The session whose agent is finishing its closing words.
|
|
363
766
|
|
|
@@ -367,7 +770,7 @@ The exported constants `DEFAULT_END_CALL_TOOL` (`'endCall'`), `DEFAULT_END_CALL_
|
|
|
367
770
|
|
|
368
771
|
**logger** (`{ warn: (message: string, ...args: unknown[]) => void }`): Receives warnings when teardown steps fail. Pass your logger or console.
|
|
369
772
|
|
|
370
|
-
|
|
773
|
+
### `createEndCallTool()`
|
|
371
774
|
|
|
372
775
|
Builds the Mastra tool an agent calls when it wants to end the call. The tool signals intent and can run optional bookkeeping. The worker performs the actual hang-up. The tool lives on the server-safe root entry. Add it to agents defined in server code.
|
|
373
776
|
|
|
@@ -387,7 +790,7 @@ const supportAgent = new Agent({
|
|
|
387
790
|
|
|
388
791
|
With `createLiveKitWorker()`, set `configuration: { endCall: {} }` and the worker watches for the tool and hangs up. On a session you own, rebuild the hang-up with [`runEndCall()`](#runendcall).
|
|
389
792
|
|
|
390
|
-
|
|
793
|
+
#### Options
|
|
391
794
|
|
|
392
795
|
**id** (`string`): Tool id the agent calls to end the call. Must match the name the worker watches for (the worker's configuration.endCall.tool, or your own onToolCall check). (Default: `'endCall'`)
|
|
393
796
|
|
|
@@ -395,7 +798,7 @@ With `createLiveKitWorker()`, set `configuration: { endCall: {} }` and the worke
|
|
|
395
798
|
|
|
396
799
|
**onEndCall** (`(request: { reason?: string; resourceId?: string; threadId?: string }) => void | Promise<void>`): Bookkeeping hook called when the agent invokes the tool — record the reason or mark the call resolved. Runs inside the turn; keep it quick. It does not hang up the call.
|
|
397
800
|
|
|
398
|
-
|
|
801
|
+
### `liveKitConnectionRoute()`
|
|
399
802
|
|
|
400
803
|
Returns an [API route](https://mastra.ai/docs/server/custom-api-routes) that mints a LiveKit access token with the voice agent dispatched into the room. Frontends call it to join a session.
|
|
401
804
|
|
|
@@ -412,7 +815,7 @@ export const mastra = new Mastra({
|
|
|
412
815
|
|
|
413
816
|
The route accepts a JSON body with optional `agentId`, `threadId`, and `resourceId` fields and responds with `{ serverUrl, roomName, participantName, participantToken }`. The `threadId` defaults to the generated room name.
|
|
414
817
|
|
|
415
|
-
|
|
818
|
+
#### Options
|
|
416
819
|
|
|
417
820
|
**path** (`string`): Route path. (Default: `'/voice/livekit/connection-details'`)
|
|
418
821
|
|
|
@@ -434,7 +837,7 @@ The route accepts a JSON body with optional `agentId`, `threadId`, and `resource
|
|
|
434
837
|
|
|
435
838
|
**metadata** (`(args) => LiveKitSessionMetadata | Promise<LiveKitSessionMetadata>`): Builds the session metadata delivered to the worker. Defaults to passing through agentId, threadId, and resourceId from the request body.
|
|
436
839
|
|
|
437
|
-
|
|
840
|
+
### `dispatchVoiceSession()`
|
|
438
841
|
|
|
439
842
|
Dispatches a Mastra voice agent into a LiveKit room programmatically: for server-initiated sessions such as outbound calls.
|
|
440
843
|
|
|
@@ -448,7 +851,7 @@ await dispatchVoiceSession({
|
|
|
448
851
|
})
|
|
449
852
|
```
|
|
450
853
|
|
|
451
|
-
|
|
854
|
+
#### Options
|
|
452
855
|
|
|
453
856
|
**roomName** (`string`): Room to dispatch the agent into. Created on demand.
|
|
454
857
|
|
|
@@ -462,7 +865,7 @@ await dispatchVoiceSession({
|
|
|
462
865
|
|
|
463
866
|
**apiSecret** (`string`): LiveKit API secret. (Default: `process.env.LIVEKIT_API_SECRET`)
|
|
464
867
|
|
|
465
|
-
|
|
868
|
+
### `LiveKitSessionMetadata`
|
|
466
869
|
|
|
467
870
|
The metadata passed from the Mastra server to the worker through LiveKit job dispatch.
|
|
468
871
|
|
|
@@ -476,7 +879,7 @@ The metadata passed from the Mastra server to the worker through LiveKit job dis
|
|
|
476
879
|
|
|
477
880
|
The metadata travels as a JSON string. `liveKitConnectionRoute()` and `dispatchVoiceSession()` serialize it for you; use `serializeSessionMetadata(metadata)` when dispatching through your own code, or write the JSON directly in LiveKit-side configuration such as a SIP dispatch rule. Entries in `requestContext` reach the agent's runtime-defined instructions, tools, and input processors on every turn of the call.
|
|
478
881
|
|
|
479
|
-
|
|
882
|
+
### Related
|
|
480
883
|
|
|
481
|
-
- [Realtime voice](
|
|
884
|
+
- [Realtime voice](#quickstart)
|
|
482
885
|
- [LiveKit Agents docs](https://docs.livekit.io/agents/)
|