@mastra/mcp-docs-server 1.1.8 → 1.1.9-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/adding-voice.md +4 -4
- package/.docs/docs/agents/agent-approval.md +3 -3
- package/.docs/docs/agents/agent-memory.md +3 -3
- package/.docs/docs/agents/guardrails.md +3 -3
- package/.docs/docs/agents/network-approval.md +5 -2
- package/.docs/docs/agents/networks.md +2 -2
- package/.docs/docs/agents/overview.md +2 -2
- package/.docs/docs/agents/processors.md +42 -24
- package/.docs/docs/agents/structured-output.md +2 -2
- package/.docs/docs/agents/supervisor-agents.md +3 -3
- package/.docs/docs/agents/using-tools.md +3 -3
- package/.docs/docs/build-with-ai/mcp-docs-server.md +5 -5
- package/.docs/docs/build-with-ai/skills.md +2 -2
- package/.docs/docs/community/contributing-templates.md +1 -1
- package/.docs/docs/community/discord.md +3 -3
- package/.docs/docs/community/licensing.md +2 -2
- package/.docs/docs/deployment/cloud-providers.md +2 -2
- package/.docs/docs/deployment/mastra-server.md +2 -2
- package/.docs/docs/deployment/monorepo.md +1 -1
- package/.docs/docs/deployment/overview.md +3 -3
- package/.docs/docs/deployment/studio.md +3 -3
- package/.docs/docs/deployment/web-framework.md +2 -2
- package/.docs/docs/deployment/workflow-runners.md +1 -1
- package/.docs/docs/evals/built-in-scorers.md +1 -1
- package/.docs/docs/evals/custom-scorers.md +6 -6
- package/.docs/docs/evals/overview.md +2 -2
- package/.docs/docs/evals/running-in-ci.md +6 -6
- package/.docs/docs/getting-started/build-with-ai.md +3 -3
- package/.docs/docs/getting-started/manual-install.md +1 -1
- package/.docs/docs/getting-started/project-structure.md +2 -2
- package/.docs/docs/index.md +63 -17
- package/.docs/docs/mastra-cloud/deployment.md +1 -1
- package/.docs/docs/mastra-cloud/studio.md +1 -1
- package/.docs/docs/mcp/overview.md +1 -1
- package/.docs/docs/mcp/publishing-mcp-server.md +4 -4
- package/.docs/docs/memory/memory-processors.md +9 -9
- package/.docs/docs/memory/message-history.md +4 -4
- package/.docs/docs/memory/observational-memory.md +11 -7
- package/.docs/docs/memory/semantic-recall.md +9 -9
- package/.docs/docs/memory/storage.md +1 -1
- package/.docs/docs/memory/working-memory.md +20 -20
- package/.docs/docs/observability/datasets/overview.md +1 -1
- package/.docs/docs/observability/datasets/running-experiments.md +1 -1
- package/.docs/docs/observability/logging.md +1 -1
- package/.docs/docs/observability/overview.md +5 -5
- package/.docs/docs/observability/tracing/bridges/otel.md +9 -9
- package/.docs/docs/observability/tracing/exporters/arize.md +3 -3
- package/.docs/docs/observability/tracing/exporters/braintrust.md +1 -1
- package/.docs/docs/observability/tracing/exporters/cloud.md +2 -2
- package/.docs/docs/observability/tracing/exporters/datadog.md +3 -3
- package/.docs/docs/observability/tracing/exporters/default.md +8 -8
- package/.docs/docs/observability/tracing/exporters/laminar.md +1 -1
- package/.docs/docs/observability/tracing/exporters/langfuse.md +3 -3
- package/.docs/docs/observability/tracing/exporters/langsmith.md +4 -4
- package/.docs/docs/observability/tracing/exporters/otel.md +8 -8
- package/.docs/docs/observability/tracing/exporters/posthog.md +2 -2
- package/.docs/docs/observability/tracing/exporters/sentry.md +4 -4
- package/.docs/docs/observability/tracing/overview.md +24 -24
- package/.docs/docs/observability/tracing/processors/sensitive-data-filter.md +13 -13
- package/.docs/docs/rag/chunking-and-embedding.md +5 -5
- package/.docs/docs/rag/overview.md +2 -2
- package/.docs/docs/rag/retrieval.md +4 -4
- package/.docs/docs/rag/vector-databases.md +13 -13
- package/.docs/docs/server/auth/auth0.md +2 -2
- package/.docs/docs/server/auth/clerk.md +1 -1
- package/.docs/docs/server/auth/composite-auth.md +9 -9
- package/.docs/docs/server/auth/custom-auth-provider.md +12 -12
- package/.docs/docs/server/auth/firebase.md +3 -3
- package/.docs/docs/server/auth/jwt.md +1 -1
- package/.docs/docs/server/auth/simple-auth.md +9 -9
- package/.docs/docs/server/auth/supabase.md +1 -1
- package/.docs/docs/server/auth/workos.md +1 -1
- package/.docs/docs/server/auth.md +2 -2
- package/.docs/docs/server/custom-adapters.md +7 -7
- package/.docs/docs/server/custom-api-routes.md +2 -2
- package/.docs/docs/server/mastra-client.md +2 -2
- package/.docs/docs/server/mastra-server.md +2 -2
- package/.docs/docs/server/request-context.md +2 -2
- package/.docs/docs/server/server-adapters.md +3 -3
- package/.docs/docs/streaming/events.md +2 -2
- package/.docs/docs/streaming/overview.md +2 -2
- package/.docs/docs/streaming/tool-streaming.md +46 -32
- package/.docs/docs/streaming/workflow-streaming.md +1 -1
- package/.docs/docs/voice/overview.md +3 -3
- package/.docs/docs/voice/speech-to-speech.md +1 -1
- package/.docs/docs/voice/speech-to-text.md +2 -2
- package/.docs/docs/voice/text-to-speech.md +2 -2
- package/.docs/docs/workflows/agents-and-tools.md +1 -1
- package/.docs/docs/workflows/control-flow.md +45 -3
- package/.docs/docs/workflows/error-handling.md +4 -4
- package/.docs/docs/workflows/overview.md +3 -3
- package/.docs/docs/workflows/snapshots.md +1 -1
- package/.docs/docs/workflows/suspend-and-resume.md +1 -1
- package/.docs/docs/workflows/time-travel.md +3 -3
- package/.docs/docs/workflows/workflow-state.md +1 -1
- package/.docs/docs/workspace/filesystem.md +3 -3
- package/.docs/docs/workspace/overview.md +53 -8
- package/.docs/docs/workspace/sandbox.md +72 -13
- package/.docs/docs/workspace/search.md +1 -1
- package/.docs/docs/workspace/skills.md +4 -4
- package/.docs/guides/build-your-ui/ai-sdk-ui.md +2 -2
- package/.docs/guides/build-your-ui/assistant-ui.md +1 -1
- package/.docs/guides/build-your-ui/copilotkit.md +2 -2
- package/.docs/guides/deployment/digital-ocean.md +1 -1
- package/.docs/guides/deployment/inngest.md +4 -4
- package/.docs/guides/getting-started/astro.md +1 -1
- package/.docs/guides/getting-started/electron.md +1 -1
- package/.docs/guides/getting-started/next-js.md +1 -1
- package/.docs/guides/getting-started/vite-react.md +1 -1
- package/.docs/guides/guide/ai-recruiter.md +4 -4
- package/.docs/guides/guide/chef-michel.md +4 -4
- package/.docs/guides/guide/code-review-bot.md +3 -3
- package/.docs/guides/guide/dev-assistant.md +5 -5
- package/.docs/guides/guide/docs-manager.md +3 -3
- package/.docs/guides/guide/github-actions-pr-description.md +4 -4
- package/.docs/guides/guide/notes-mcp-server.md +4 -4
- package/.docs/guides/guide/research-assistant.md +4 -4
- package/.docs/guides/guide/research-coordinator.md +1 -1
- package/.docs/guides/guide/stock-agent.md +6 -6
- package/.docs/guides/guide/web-search.md +2 -2
- package/.docs/guides/guide/whatsapp-chat-bot.md +1 -1
- package/.docs/guides/migrations/agentnetwork.md +1 -1
- package/.docs/guides/migrations/ai-sdk-v4-to-v5.md +3 -3
- package/.docs/guides/migrations/network-to-supervisor.md +1 -1
- package/.docs/guides/migrations/upgrade-to-v1/agent.md +1 -1
- package/.docs/guides/migrations/upgrade-to-v1/client.md +2 -2
- package/.docs/guides/migrations/upgrade-to-v1/deployment.md +2 -2
- package/.docs/guides/migrations/upgrade-to-v1/evals.md +1 -1
- package/.docs/guides/migrations/upgrade-to-v1/mastra.md +1 -1
- package/.docs/guides/migrations/upgrade-to-v1/memory.md +2 -2
- package/.docs/guides/migrations/upgrade-to-v1/overview.md +3 -3
- package/.docs/guides/migrations/upgrade-to-v1/storage.md +4 -4
- package/.docs/guides/migrations/upgrade-to-v1/tools.md +2 -2
- package/.docs/guides/migrations/upgrade-to-v1/tracing.md +2 -2
- package/.docs/guides/migrations/upgrade-to-v1/vectors.md +3 -3
- package/.docs/guides/migrations/upgrade-to-v1/voice.md +1 -1
- package/.docs/guides/migrations/upgrade-to-v1/workflows.md +6 -6
- package/.docs/guides/migrations/vnext-to-standard-apis.md +3 -3
- package/.docs/models/embeddings.md +4 -4
- package/.docs/models/gateways/custom-gateways.md +4 -4
- package/.docs/models/gateways/netlify.md +2 -3
- package/.docs/models/gateways/openrouter.md +9 -2
- package/.docs/models/gateways/vercel.md +11 -2
- package/.docs/models/gateways.md +2 -2
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/302ai.md +3 -3
- package/.docs/models/providers/abacus.md +24 -14
- package/.docs/models/providers/aihubmix.md +10 -5
- package/.docs/models/providers/alibaba-cn.md +83 -74
- package/.docs/models/providers/alibaba-coding-plan-cn.md +78 -0
- package/.docs/models/providers/alibaba-coding-plan.md +78 -0
- package/.docs/models/providers/alibaba.md +3 -3
- package/.docs/models/providers/anthropic.md +4 -4
- package/.docs/models/providers/bailing.md +3 -3
- package/.docs/models/providers/baseten.md +3 -3
- package/.docs/models/providers/berget.md +3 -3
- package/.docs/models/providers/cerebras.md +4 -4
- package/.docs/models/providers/chutes.md +7 -6
- package/.docs/models/providers/clarifai.md +81 -0
- package/.docs/models/providers/cloudferro-sherlock.md +8 -7
- package/.docs/models/providers/cloudflare-workers-ai.md +6 -5
- package/.docs/models/providers/cortecs.md +10 -8
- package/.docs/models/providers/deepinfra.md +11 -6
- package/.docs/models/providers/deepseek.md +4 -4
- package/.docs/models/providers/drun.md +73 -0
- package/.docs/models/providers/evroc.md +3 -3
- package/.docs/models/providers/fastrouter.md +3 -3
- package/.docs/models/providers/fireworks-ai.md +3 -3
- package/.docs/models/providers/firmware.md +31 -23
- package/.docs/models/providers/friendli.md +3 -3
- package/.docs/models/providers/github-models.md +3 -3
- package/.docs/models/providers/google.md +7 -5
- package/.docs/models/providers/groq.md +4 -4
- package/.docs/models/providers/helicone.md +3 -3
- package/.docs/models/providers/huggingface.md +3 -3
- package/.docs/models/providers/iflowcn.md +3 -3
- package/.docs/models/providers/inception.md +7 -5
- package/.docs/models/providers/inference.md +3 -3
- package/.docs/models/providers/io-net.md +3 -3
- package/.docs/models/providers/jiekou.md +3 -3
- package/.docs/models/providers/kilo.md +6 -4
- package/.docs/models/providers/kimi-for-coding.md +4 -4
- package/.docs/models/providers/kuae-cloud-coding-plan.md +3 -3
- package/.docs/models/providers/llama.md +3 -3
- package/.docs/models/providers/lmstudio.md +3 -3
- package/.docs/models/providers/lucidquery.md +3 -3
- package/.docs/models/providers/meganova.md +3 -3
- package/.docs/models/providers/minimax-cn-coding-plan.md +4 -4
- package/.docs/models/providers/minimax-cn.md +4 -4
- package/.docs/models/providers/minimax-coding-plan.md +4 -4
- package/.docs/models/providers/minimax.md +4 -4
- package/.docs/models/providers/mistral.md +4 -4
- package/.docs/models/providers/moark.md +3 -3
- package/.docs/models/providers/modelscope.md +3 -3
- package/.docs/models/providers/moonshotai-cn.md +3 -3
- package/.docs/models/providers/moonshotai.md +3 -3
- package/.docs/models/providers/morph.md +3 -3
- package/.docs/models/providers/nano-gpt.md +523 -42
- package/.docs/models/providers/nebius.md +37 -37
- package/.docs/models/providers/nova.md +3 -3
- package/.docs/models/providers/novita-ai.md +3 -3
- package/.docs/models/providers/nvidia.md +7 -5
- package/.docs/models/providers/ollama-cloud.md +4 -5
- package/.docs/models/providers/openai.md +7 -5
- package/.docs/models/providers/opencode-go.md +3 -3
- package/.docs/models/providers/opencode.md +39 -36
- package/.docs/models/providers/ovhcloud.md +3 -3
- package/.docs/models/providers/perplexity-agent.md +4 -4
- package/.docs/models/providers/perplexity.md +4 -4
- package/.docs/models/providers/poe.md +11 -5
- package/.docs/models/providers/privatemode-ai.md +3 -3
- package/.docs/models/providers/qihang-ai.md +3 -3
- package/.docs/models/providers/qiniu-ai.md +23 -8
- package/.docs/models/providers/requesty.md +20 -4
- package/.docs/models/providers/scaleway.md +3 -3
- package/.docs/models/providers/siliconflow-cn.md +10 -4
- package/.docs/models/providers/siliconflow.md +3 -3
- package/.docs/models/providers/stackit.md +3 -3
- package/.docs/models/providers/stepfun.md +3 -3
- package/.docs/models/providers/submodel.md +3 -3
- package/.docs/models/providers/synthetic.md +3 -3
- package/.docs/models/providers/togetherai.md +5 -7
- package/.docs/models/providers/upstage.md +3 -3
- package/.docs/models/providers/vivgrid.md +4 -4
- package/.docs/models/providers/vultr.md +3 -3
- package/.docs/models/providers/wandb.md +3 -3
- package/.docs/models/providers/xai.md +34 -31
- package/.docs/models/providers/xiaomi.md +4 -4
- package/.docs/models/providers/zai-coding-plan.md +3 -3
- package/.docs/models/providers/zai.md +3 -3
- package/.docs/models/providers/zenmux.md +7 -5
- package/.docs/models/providers/zhipuai-coding-plan.md +3 -3
- package/.docs/models/providers/zhipuai.md +3 -3
- package/.docs/models/providers.md +4 -0
- package/.docs/reference/agents/agent.md +3 -3
- package/.docs/reference/agents/generateLegacy.md +1 -1
- package/.docs/reference/agents/getDefaultGenerateOptions.md +1 -1
- package/.docs/reference/agents/getDefaultOptions.md +1 -1
- package/.docs/reference/agents/getDefaultStreamOptions.md +1 -1
- package/.docs/reference/agents/getDescription.md +1 -1
- package/.docs/reference/agents/network.md +5 -3
- package/.docs/reference/ai-sdk/handle-chat-stream.md +2 -0
- package/.docs/reference/ai-sdk/handle-network-stream.md +2 -0
- package/.docs/reference/ai-sdk/network-route.md +2 -0
- package/.docs/reference/ai-sdk/to-ai-sdk-stream.md +1 -1
- package/.docs/reference/ai-sdk/to-ai-sdk-v4-messages.md +1 -1
- package/.docs/reference/ai-sdk/to-ai-sdk-v5-messages.md +1 -1
- package/.docs/reference/auth/auth0.md +7 -7
- package/.docs/reference/auth/better-auth.md +2 -2
- package/.docs/reference/auth/clerk.md +1 -1
- package/.docs/reference/auth/firebase.md +5 -5
- package/.docs/reference/auth/jwt.md +1 -1
- package/.docs/reference/auth/supabase.md +1 -1
- package/.docs/reference/auth/workos.md +6 -6
- package/.docs/reference/cli/mastra.md +5 -5
- package/.docs/reference/client-js/agents.md +22 -22
- package/.docs/reference/client-js/error-handling.md +2 -2
- package/.docs/reference/client-js/logs.md +2 -2
- package/.docs/reference/client-js/mastra-client.md +2 -2
- package/.docs/reference/client-js/memory.md +6 -6
- package/.docs/reference/client-js/observability.md +4 -4
- package/.docs/reference/client-js/telemetry.md +1 -1
- package/.docs/reference/client-js/tools.md +3 -3
- package/.docs/reference/client-js/vectors.md +2 -2
- package/.docs/reference/client-js/workflows.md +12 -12
- package/.docs/reference/configuration.md +62 -6
- package/.docs/reference/core/getDeployer.md +1 -1
- package/.docs/reference/core/getGatewayById.md +1 -1
- package/.docs/reference/core/getLogger.md +1 -1
- package/.docs/reference/core/getMCPServer.md +2 -2
- package/.docs/reference/core/getMCPServerById.md +2 -2
- package/.docs/reference/core/getMemory.md +1 -1
- package/.docs/reference/core/getScorer.md +4 -4
- package/.docs/reference/core/getScorerById.md +2 -2
- package/.docs/reference/core/getServer.md +1 -1
- package/.docs/reference/core/getStorage.md +1 -1
- package/.docs/reference/core/getStoredAgentById.md +3 -3
- package/.docs/reference/core/getTelemetry.md +1 -1
- package/.docs/reference/core/getWorkflow.md +1 -1
- package/.docs/reference/core/listAgents.md +1 -1
- package/.docs/reference/core/listMCPServers.md +3 -3
- package/.docs/reference/core/listMemory.md +1 -1
- package/.docs/reference/core/listScorers.md +1 -1
- package/.docs/reference/core/listStoredAgents.md +3 -3
- package/.docs/reference/core/listVectors.md +1 -1
- package/.docs/reference/core/mastra-class.md +2 -2
- package/.docs/reference/core/mastra-model-gateway.md +11 -11
- package/.docs/reference/core/setLogger.md +1 -1
- package/.docs/reference/core/setStorage.md +1 -1
- package/.docs/reference/datasets/dataset.md +2 -2
- package/.docs/reference/datasets/datasets-manager.md +1 -1
- package/.docs/reference/datasets/get.md +2 -2
- package/.docs/reference/datasets/getDetails.md +1 -1
- package/.docs/reference/datasets/listItems.md +1 -1
- package/.docs/reference/deployer/vercel.md +1 -1
- package/.docs/reference/deployer.md +4 -4
- package/.docs/reference/evals/answer-relevancy.md +4 -4
- package/.docs/reference/evals/answer-similarity.md +3 -3
- package/.docs/reference/evals/bias.md +4 -4
- package/.docs/reference/evals/completeness.md +6 -6
- package/.docs/reference/evals/content-similarity.md +3 -3
- package/.docs/reference/evals/context-precision.md +9 -9
- package/.docs/reference/evals/context-relevance.md +7 -7
- package/.docs/reference/evals/create-scorer.md +7 -7
- package/.docs/reference/evals/faithfulness.md +3 -3
- package/.docs/reference/evals/hallucination.md +8 -14
- package/.docs/reference/evals/keyword-coverage.md +4 -4
- package/.docs/reference/evals/mastra-scorer.md +7 -7
- package/.docs/reference/evals/noise-sensitivity.md +11 -11
- package/.docs/reference/evals/prompt-alignment.md +5 -5
- package/.docs/reference/evals/run-evals.md +5 -5
- package/.docs/reference/evals/scorer-utils.md +17 -17
- package/.docs/reference/evals/textual-difference.md +4 -4
- package/.docs/reference/evals/tone-consistency.md +5 -5
- package/.docs/reference/evals/tool-call-accuracy.md +10 -10
- package/.docs/reference/evals/toxicity.md +3 -3
- package/.docs/reference/harness/harness-class.md +5 -3
- package/.docs/reference/index.md +2 -0
- package/.docs/reference/memory/clone-utilities.md +7 -7
- package/.docs/reference/memory/cloneThread.md +5 -5
- package/.docs/reference/memory/createThread.md +1 -1
- package/.docs/reference/memory/deleteMessages.md +1 -1
- package/.docs/reference/memory/getThreadById.md +1 -1
- package/.docs/reference/memory/listThreads.md +3 -3
- package/.docs/reference/memory/memory-class.md +1 -1
- package/.docs/reference/memory/observational-memory.md +8 -6
- package/.docs/reference/memory/recall.md +1 -1
- package/.docs/reference/observability/tracing/bridges/otel.md +6 -6
- package/.docs/reference/observability/tracing/configuration.md +17 -17
- package/.docs/reference/observability/tracing/exporters/arize.md +4 -4
- package/.docs/reference/observability/tracing/exporters/braintrust.md +3 -3
- package/.docs/reference/observability/tracing/exporters/cloud-exporter.md +6 -6
- package/.docs/reference/observability/tracing/exporters/console-exporter.md +4 -4
- package/.docs/reference/observability/tracing/exporters/datadog.md +4 -4
- package/.docs/reference/observability/tracing/exporters/default-exporter.md +6 -6
- package/.docs/reference/observability/tracing/exporters/laminar.md +2 -2
- package/.docs/reference/observability/tracing/exporters/langfuse.md +4 -4
- package/.docs/reference/observability/tracing/exporters/langsmith.md +6 -6
- package/.docs/reference/observability/tracing/exporters/otel.md +12 -12
- package/.docs/reference/observability/tracing/exporters/posthog.md +3 -3
- package/.docs/reference/observability/tracing/exporters/sentry.md +5 -5
- package/.docs/reference/observability/tracing/instances.md +9 -9
- package/.docs/reference/observability/tracing/interfaces.md +39 -39
- package/.docs/reference/observability/tracing/processors/sensitive-data-filter.md +6 -6
- package/.docs/reference/observability/tracing/spans.md +15 -13
- package/.docs/reference/processors/message-history-processor.md +1 -1
- package/.docs/reference/processors/processor-interface.md +21 -17
- package/.docs/reference/processors/token-limiter-processor.md +2 -2
- package/.docs/reference/rag/chunk.md +2 -2
- package/.docs/reference/rag/database-config.md +8 -8
- package/.docs/reference/rag/document.md +11 -11
- package/.docs/reference/rag/embeddings.md +5 -5
- package/.docs/reference/rag/extract-params.md +8 -8
- package/.docs/reference/rag/graph-rag.md +4 -4
- package/.docs/reference/rag/metadata-filters.md +15 -15
- package/.docs/reference/rag/rerank.md +2 -2
- package/.docs/reference/rag/rerankWithScorer.md +2 -2
- package/.docs/reference/server/create-route.md +2 -0
- package/.docs/reference/server/express-adapter.md +1 -1
- package/.docs/reference/server/fastify-adapter.md +1 -1
- package/.docs/reference/server/hono-adapter.md +1 -1
- package/.docs/reference/server/koa-adapter.md +2 -2
- package/.docs/reference/server/mastra-server.md +16 -16
- package/.docs/reference/server/register-api-route.md +7 -7
- package/.docs/reference/server/routes.md +1 -1
- package/.docs/reference/storage/cloudflare-d1.md +5 -5
- package/.docs/reference/storage/cloudflare.md +3 -3
- package/.docs/reference/storage/composite.md +1 -1
- package/.docs/reference/storage/convex.md +6 -6
- package/.docs/reference/storage/dynamodb.md +7 -7
- package/.docs/reference/storage/lance.md +5 -5
- package/.docs/reference/storage/libsql.md +1 -1
- package/.docs/reference/storage/mongodb.md +6 -6
- package/.docs/reference/storage/mssql.md +4 -4
- package/.docs/reference/storage/overview.md +2 -2
- package/.docs/reference/storage/postgresql.md +7 -7
- package/.docs/reference/storage/upstash.md +4 -4
- package/.docs/reference/streaming/ChunkType.md +13 -13
- package/.docs/reference/streaming/agents/MastraModelOutput.md +6 -6
- package/.docs/reference/streaming/agents/stream.md +2 -2
- package/.docs/reference/streaming/agents/streamLegacy.md +1 -1
- package/.docs/reference/streaming/workflows/observeStream.md +2 -2
- package/.docs/reference/streaming/workflows/resumeStream.md +1 -1
- package/.docs/reference/streaming/workflows/stream.md +1 -1
- package/.docs/reference/templates/overview.md +4 -4
- package/.docs/reference/tools/create-tool.md +10 -10
- package/.docs/reference/tools/document-chunker-tool.md +4 -4
- package/.docs/reference/tools/graph-rag-tool.md +7 -7
- package/.docs/reference/tools/mcp-client.md +13 -13
- package/.docs/reference/tools/mcp-server.md +27 -27
- package/.docs/reference/tools/vector-query-tool.md +12 -12
- package/.docs/reference/vectors/astra.md +13 -13
- package/.docs/reference/vectors/chroma.md +18 -18
- package/.docs/reference/vectors/convex.md +15 -15
- package/.docs/reference/vectors/couchbase.md +21 -21
- package/.docs/reference/vectors/duckdb.md +17 -17
- package/.docs/reference/vectors/elasticsearch.md +14 -14
- package/.docs/reference/vectors/lance.md +22 -22
- package/.docs/reference/vectors/libsql.md +15 -15
- package/.docs/reference/vectors/mongodb.md +18 -18
- package/.docs/reference/vectors/opensearch.md +11 -11
- package/.docs/reference/vectors/pg.md +23 -21
- package/.docs/reference/vectors/pinecone.md +15 -15
- package/.docs/reference/vectors/qdrant.md +15 -15
- package/.docs/reference/vectors/s3vectors.md +22 -22
- package/.docs/reference/vectors/turbopuffer.md +14 -14
- package/.docs/reference/vectors/upstash.md +15 -15
- package/.docs/reference/vectors/vectorize.md +16 -16
- package/.docs/reference/voice/azure.md +12 -10
- package/.docs/reference/voice/cloudflare.md +9 -7
- package/.docs/reference/voice/composite-voice.md +5 -5
- package/.docs/reference/voice/deepgram.md +5 -5
- package/.docs/reference/voice/elevenlabs.md +7 -7
- package/.docs/reference/voice/google-gemini-live.md +22 -22
- package/.docs/reference/voice/google.md +12 -12
- package/.docs/reference/voice/mastra-voice.md +18 -18
- package/.docs/reference/voice/murf.md +8 -8
- package/.docs/reference/voice/openai-realtime.md +19 -17
- package/.docs/reference/voice/openai.md +12 -8
- package/.docs/reference/voice/playai.md +9 -7
- package/.docs/reference/voice/sarvam.md +8 -6
- package/.docs/reference/voice/speechify.md +11 -9
- package/.docs/reference/voice/voice.addInstructions.md +4 -4
- package/.docs/reference/voice/voice.addTools.md +3 -3
- package/.docs/reference/voice/voice.answer.md +2 -2
- package/.docs/reference/voice/voice.close.md +4 -4
- package/.docs/reference/voice/voice.connect.md +9 -7
- package/.docs/reference/voice/voice.events.md +4 -4
- package/.docs/reference/voice/voice.getSpeakers.md +4 -4
- package/.docs/reference/voice/voice.listen.md +17 -11
- package/.docs/reference/voice/voice.off.md +4 -4
- package/.docs/reference/voice/voice.on.md +5 -5
- package/.docs/reference/voice/voice.send.md +2 -2
- package/.docs/reference/voice/voice.speak.md +19 -9
- package/.docs/reference/voice/voice.updateConfig.md +4 -4
- package/.docs/reference/workflows/run-methods/startAsync.md +1 -1
- package/.docs/reference/workflows/run-methods/timeTravel.md +1 -1
- package/.docs/reference/workflows/run.md +3 -3
- package/.docs/reference/workflows/step.md +2 -2
- package/.docs/reference/workflows/workflow-methods/create-run.md +1 -1
- package/.docs/reference/workflows/workflow.md +1 -1
- package/.docs/reference/workspace/blaxel-sandbox.md +164 -0
- package/.docs/reference/workspace/daytona-sandbox.md +50 -141
- package/.docs/reference/workspace/e2b-sandbox.md +41 -77
- package/.docs/reference/workspace/filesystem.md +25 -11
- package/.docs/reference/workspace/gcs-filesystem.md +21 -1
- package/.docs/reference/workspace/local-filesystem.md +24 -10
- package/.docs/reference/workspace/local-sandbox.md +27 -102
- package/.docs/reference/workspace/process-manager.md +296 -0
- package/.docs/reference/workspace/s3-filesystem.md +21 -1
- package/.docs/reference/workspace/sandbox.md +9 -1
- package/.docs/reference/workspace/workspace-class.md +95 -27
- package/CHANGELOG.md +15 -0
- package/dist/tools/course.d.ts +7 -27
- package/dist/tools/course.d.ts.map +1 -1
- package/dist/tools/docs.d.ts +6 -18
- package/dist/tools/docs.d.ts.map +1 -1
- package/dist/tools/embedded-docs.d.ts +12 -112
- package/dist/tools/embedded-docs.d.ts.map +1 -1
- package/dist/tools/migration.d.ts +6 -26
- package/dist/tools/migration.d.ts.map +1 -1
- package/package.json +6 -6
|
@@ -36,7 +36,7 @@ if (storedConfig) {
|
|
|
36
36
|
|
|
37
37
|
**result** (`Agent | StorageAgentType | null`): Returns an \`Agent\` instance by default, or \`StorageAgentType\` when \`raw: true\`. Returns \`null\` if no agent with the given ID exists.
|
|
38
38
|
|
|
39
|
-
## Primitive
|
|
39
|
+
## Primitive resolution
|
|
40
40
|
|
|
41
41
|
When creating an `Agent` instance from stored configuration, the method resolves references to registered primitives:
|
|
42
42
|
|
|
@@ -46,9 +46,9 @@ When creating an `Agent` instance from stored configuration, the method resolves
|
|
|
46
46
|
- **Memory**: Resolved from `memory` registered in Mastra config
|
|
47
47
|
- **Scorers**: Resolved from `scorers` registered in Mastra config, including sampling configuration
|
|
48
48
|
|
|
49
|
-
If a referenced primitive
|
|
49
|
+
If a referenced primitive isn't found in the registry, a warning is logged but the agent is still created.
|
|
50
50
|
|
|
51
|
-
## StorageAgentType
|
|
51
|
+
## `StorageAgentType`
|
|
52
52
|
|
|
53
53
|
When using `raw: true`, the returned object has the following structure:
|
|
54
54
|
|
|
@@ -25,7 +25,7 @@ For best TypeScript support, use `getWorkflow()` with the workflow's **registrat
|
|
|
25
25
|
- Output data schemas
|
|
26
26
|
- Step results
|
|
27
27
|
|
|
28
|
-
If you need to retrieve a workflow by its `id` property instead of its registration key, you can use `getWorkflowById()`, but note that type inference
|
|
28
|
+
If you need to retrieve a workflow by its `id` property instead of its registration key, you can use `getWorkflowById()`, but note that type inference won't be as precise.
|
|
29
29
|
|
|
30
30
|
## Parameters
|
|
31
31
|
|
|
@@ -38,18 +38,18 @@ const servers = mastra.listMCPServers()
|
|
|
38
38
|
|
|
39
39
|
## Parameters
|
|
40
40
|
|
|
41
|
-
This method
|
|
41
|
+
This method doesn't accept any parameters.
|
|
42
42
|
|
|
43
43
|
## Returns
|
|
44
44
|
|
|
45
45
|
**servers** (`Record<string, MCPServerBase> | undefined`): A record of all registered MCP server instances, where keys are the registry keys (from mcpServers configuration) and values are MCPServerBase instances, or undefined if no servers are registered.
|
|
46
46
|
|
|
47
|
-
## Related
|
|
47
|
+
## Related methods
|
|
48
48
|
|
|
49
49
|
- [Mastra.getMCPServer()](https://mastra.ai/reference/core/getMCPServer) - Retrieve an MCP server by its registry key
|
|
50
50
|
- [Mastra.getMCPServerById()](https://mastra.ai/reference/core/getMCPServerById) - Retrieve an MCP server by its intrinsic `id` property
|
|
51
51
|
|
|
52
|
-
## See
|
|
52
|
+
## See also
|
|
53
53
|
|
|
54
54
|
- [MCP overview](https://mastra.ai/docs/mcp/overview)
|
|
55
55
|
- [MCP server reference](https://mastra.ai/reference/tools/mcp-server)
|
|
@@ -20,7 +20,7 @@ This method takes no parameters.
|
|
|
20
20
|
|
|
21
21
|
**memory** (`Record<string, MastraMemory>`): An object containing all registered memory instances, keyed by their registry keys.
|
|
22
22
|
|
|
23
|
-
## Example: Checking
|
|
23
|
+
## Example: Checking registered memory
|
|
24
24
|
|
|
25
25
|
```typescript
|
|
26
26
|
import { Mastra } from '@mastra/core'
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
The `listScorers()` method returns all scorers that have been registered with the Mastra instance. Scorers are used for evaluating AI outputs and can override default scorers during agent generation or workflow execution.
|
|
4
4
|
|
|
5
|
-
## Usage
|
|
5
|
+
## Usage example
|
|
6
6
|
|
|
7
7
|
```typescript
|
|
8
8
|
import { mastra } from './mastra'
|
|
@@ -53,7 +53,7 @@ for (const config of result.agents) {
|
|
|
53
53
|
|
|
54
54
|
**hasMore** (`boolean`): Whether there are more pages available.
|
|
55
55
|
|
|
56
|
-
## Primitive
|
|
56
|
+
## Primitive resolution
|
|
57
57
|
|
|
58
58
|
When creating `Agent` instances (default behavior), each stored agent's configuration is resolved against registered primitives:
|
|
59
59
|
|
|
@@ -63,9 +63,9 @@ When creating `Agent` instances (default behavior), each stored agent's configur
|
|
|
63
63
|
- **Memory**: Resolved from `memory` registered in Mastra config
|
|
64
64
|
- **Scorers**: Resolved from `scorers` registered in Mastra config
|
|
65
65
|
|
|
66
|
-
If a referenced primitive
|
|
66
|
+
If a referenced primitive isn't found, a warning is logged but the agent is still created.
|
|
67
67
|
|
|
68
|
-
## Example: Iterating
|
|
68
|
+
## Example: Iterating through all stored agents
|
|
69
69
|
|
|
70
70
|
```typescript
|
|
71
71
|
async function getAllStoredAgents(mastra: Mastra) {
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Mastra
|
|
1
|
+
# Mastra class
|
|
2
2
|
|
|
3
3
|
The `Mastra` class is the central orchestrator in any Mastra application, managing agents, workflows, storage, logging, observability, and more. Typically, you create a single instance of `Mastra` to coordinate your application.
|
|
4
4
|
|
|
@@ -41,7 +41,7 @@ Visit the [Configuration reference](https://mastra.ai/reference/configuration) f
|
|
|
41
41
|
|
|
42
42
|
**logger** (`Logger`): Logger instance created with new PinoLogger() (Default: `Console logger with INFO level`)
|
|
43
43
|
|
|
44
|
-
**idGenerator** (`() => string`): Custom ID generator function. Used by agents, workflows, memory, and other components to generate unique identifiers.
|
|
44
|
+
**idGenerator** (`(context?: IdGeneratorContext) => string`): Custom ID generator function. Used by agents, workflows, memory, and other components to generate unique identifiers. Receives optional context such as idType, source, entityId, and threadId to support context-aware ID formats.
|
|
45
45
|
|
|
46
46
|
**workflows** (`Record<string, Workflow>`): Workflows to register. Structured as a key-value pair, with keys being the workflow name and values being the workflow instance. (Default: `{}`)
|
|
47
47
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Abstract base class for implementing custom model gateways. Gateways handle provider-specific logic for accessing language models, including provider configuration, authentication, URL construction, and model instantiation.
|
|
4
4
|
|
|
5
|
-
## Class
|
|
5
|
+
## Class overview
|
|
6
6
|
|
|
7
7
|
```typescript
|
|
8
8
|
import { MastraModelGateway, type ProviderConfig } from '@mastra/core/llm'
|
|
@@ -53,15 +53,15 @@ class MyCustomGateway extends MastraModelGateway {
|
|
|
53
53
|
}
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
-
## Required
|
|
56
|
+
## Required properties
|
|
57
57
|
|
|
58
58
|
**id** (`string`): Unique identifier for the gateway. This ID is used as the prefix for all providers from this gateway (e.g., "netlify/anthropic"). Exception: models.dev is a provider registry and doesn't use a prefix.
|
|
59
59
|
|
|
60
60
|
**name** (`string`): Human-readable name for the gateway.
|
|
61
61
|
|
|
62
|
-
## Required
|
|
62
|
+
## Required methods
|
|
63
63
|
|
|
64
|
-
### fetchProviders()
|
|
64
|
+
### `fetchProviders()`
|
|
65
65
|
|
|
66
66
|
Fetches provider configurations from the gateway.
|
|
67
67
|
|
|
@@ -83,7 +83,7 @@ Fetches provider configurations from the gateway.
|
|
|
83
83
|
|
|
84
84
|
**docUrl** (`string`): Optional documentation URL
|
|
85
85
|
|
|
86
|
-
### buildUrl()
|
|
86
|
+
### `buildUrl()`
|
|
87
87
|
|
|
88
88
|
Builds the API URL for a specific model/provider combination.
|
|
89
89
|
|
|
@@ -95,7 +95,7 @@ Builds the API URL for a specific model/provider combination.
|
|
|
95
95
|
|
|
96
96
|
**Returns:** `string | undefined | Promise<string | undefined>`
|
|
97
97
|
|
|
98
|
-
### getApiKey()
|
|
98
|
+
### `getApiKey()`
|
|
99
99
|
|
|
100
100
|
Retrieves the API key for authentication.
|
|
101
101
|
|
|
@@ -105,7 +105,7 @@ Retrieves the API key for authentication.
|
|
|
105
105
|
|
|
106
106
|
**Returns:** `Promise<string>`
|
|
107
107
|
|
|
108
|
-
### resolveLanguageModel()
|
|
108
|
+
### `resolveLanguageModel()`
|
|
109
109
|
|
|
110
110
|
Creates a language model instance.
|
|
111
111
|
|
|
@@ -119,15 +119,15 @@ Creates a language model instance.
|
|
|
119
119
|
|
|
120
120
|
**Returns:** `Promise<LanguageModelV2> | LanguageModelV2`
|
|
121
121
|
|
|
122
|
-
## Instance
|
|
122
|
+
## Instance methods
|
|
123
123
|
|
|
124
|
-
### getId()
|
|
124
|
+
### `getId()`
|
|
125
125
|
|
|
126
126
|
Returns the gateway's unique identifier.
|
|
127
127
|
|
|
128
128
|
**Returns:** `string` - The gateway's `id` property
|
|
129
129
|
|
|
130
|
-
## Model ID
|
|
130
|
+
## Model ID format
|
|
131
131
|
|
|
132
132
|
For true gateways, the gateway ID is used as a prefix and models are accessed using this format:
|
|
133
133
|
|
|
@@ -139,7 +139,7 @@ Examples:
|
|
|
139
139
|
|
|
140
140
|
- Gateway with `id = 'custom'`: `'custom/my-provider/model-1'`
|
|
141
141
|
|
|
142
|
-
## Built-in
|
|
142
|
+
## Built-in implementations
|
|
143
143
|
|
|
144
144
|
- **NetlifyGateway** - Netlify AI Gateway integration
|
|
145
145
|
- **ModelsDevGateway** - Registry of OpenAI-compatible providers
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Dataset
|
|
1
|
+
# Dataset class
|
|
2
2
|
|
|
3
3
|
**Added in:** `@mastra/core@1.4.0`
|
|
4
4
|
|
|
@@ -60,7 +60,7 @@ const history = await dataset.getItemHistory({ itemId: 'item-id' })
|
|
|
60
60
|
|
|
61
61
|
## Access
|
|
62
62
|
|
|
63
|
-
`Dataset`
|
|
63
|
+
`Dataset` isn't instantiated directly. Obtain one via `DatasetsManager`:
|
|
64
64
|
|
|
65
65
|
```typescript
|
|
66
66
|
const dataset = await mastra.datasets.get({ id: 'dataset-id' })
|
|
@@ -78,7 +78,7 @@ const comparison = await mastra.datasets.compareExperiments({
|
|
|
78
78
|
|
|
79
79
|
## Access
|
|
80
80
|
|
|
81
|
-
`DatasetsManager`
|
|
81
|
+
`DatasetsManager` isn't instantiated directly. Access it from a `Mastra` instance:
|
|
82
82
|
|
|
83
83
|
```typescript
|
|
84
84
|
const mastra = new Mastra({
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
**Added in:** `@mastra/core@1.4.0`
|
|
4
4
|
|
|
5
|
-
Retrieves an existing dataset by ID. Throws a `MastraError` if the dataset
|
|
5
|
+
Retrieves an existing dataset by ID. Throws a `MastraError` if the dataset doesn't exist.
|
|
6
6
|
|
|
7
7
|
## Usage example
|
|
8
8
|
|
|
@@ -26,6 +26,6 @@ console.log(details.name)
|
|
|
26
26
|
|
|
27
27
|
## Returns
|
|
28
28
|
|
|
29
|
-
Throws `MastraError` if the dataset
|
|
29
|
+
Throws `MastraError` if the dataset isn't found.
|
|
30
30
|
|
|
31
31
|
**result** (`Promise<Dataset>`): A Dataset instance.
|
|
@@ -26,7 +26,7 @@ This method takes no parameters.
|
|
|
26
26
|
|
|
27
27
|
## Returns
|
|
28
28
|
|
|
29
|
-
Returns a `Promise<DatasetRecord>`. Throws `MastraError` if the dataset
|
|
29
|
+
Returns a `Promise<DatasetRecord>`. Throws `MastraError` if the dataset isn't found.
|
|
30
30
|
|
|
31
31
|
**id** (`string`): Unique identifier of the dataset.
|
|
32
32
|
|
|
@@ -41,7 +41,7 @@ When `version` is specified:
|
|
|
41
41
|
|
|
42
42
|
**result** (`Promise<DatasetItem[]>`): Array of all items at the specified dataset version.
|
|
43
43
|
|
|
44
|
-
When `version`
|
|
44
|
+
When `version` isn't specified:
|
|
45
45
|
|
|
46
46
|
**result** (`Promise<object>`): Paginated response.
|
|
47
47
|
|
|
@@ -113,4 +113,4 @@ With `studio: true`:
|
|
|
113
113
|
|
|
114
114
|
When studio is enabled, the routing is configured so that `/api/*` requests are handled by the serverless function, static assets (JS, CSS) are served from Vercel's Edge CDN with no function invocations, and all other paths fall back to `index.html` for client-side routing.
|
|
115
115
|
|
|
116
|
-
This folder is generated during build and
|
|
116
|
+
This folder is generated during build and shouldn't be committed to version control.
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
The Deployer handles the deployment of standalone Mastra applications by packaging code, managing environment files, and serving applications using the Hono framework. Concrete implementations must define the deploy method for specific deployment targets.
|
|
4
4
|
|
|
5
|
-
## Usage
|
|
5
|
+
## Usage example
|
|
6
6
|
|
|
7
7
|
If you want to create a custom deployer, you can extend the abstract `Deployer` class and implement the `deploy` method with your specific deployment logic. Here's an example:
|
|
8
8
|
|
|
@@ -28,7 +28,7 @@ class CustomDeployer extends Deployer {
|
|
|
28
28
|
|
|
29
29
|
## Parameters
|
|
30
30
|
|
|
31
|
-
### Constructor
|
|
31
|
+
### Constructor parameters
|
|
32
32
|
|
|
33
33
|
**args** (`object`): Configuration options for the Deployer.
|
|
34
34
|
|
|
@@ -44,7 +44,7 @@ class CustomDeployer extends Deployer {
|
|
|
44
44
|
|
|
45
45
|
**deploy** (`(outputDirectory: string) => Promise<void>`): Abstract method that must be implemented by subclasses. Handles the deployment process to the specified output directory.
|
|
46
46
|
|
|
47
|
-
## Inherited
|
|
47
|
+
## Inherited methods from bundler
|
|
48
48
|
|
|
49
49
|
The Deployer class inherits the following key methods from the Bundler class:
|
|
50
50
|
|
|
@@ -54,7 +54,7 @@ The Deployer class inherits the following key methods from the Bundler class:
|
|
|
54
54
|
|
|
55
55
|
**\_bundle** (`(serverFile: string, mastraEntryFile: string, outputDirectory: string, bundleLocation?: string) => Promise<void>`): Bundles the application using the specified server and Mastra entry files.
|
|
56
56
|
|
|
57
|
-
## Core
|
|
57
|
+
## Core concepts
|
|
58
58
|
|
|
59
59
|
### Deployment Lifecycle
|
|
60
60
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Answer
|
|
1
|
+
# Answer relevancy scorer
|
|
2
2
|
|
|
3
3
|
The `createAnswerRelevancyScorer()` function accepts a single options object with the following properties:
|
|
4
4
|
|
|
@@ -12,7 +12,7 @@ The `createAnswerRelevancyScorer()` function accepts a single options object wit
|
|
|
12
12
|
|
|
13
13
|
This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
|
|
14
14
|
|
|
15
|
-
##
|
|
15
|
+
## `.run()` returns
|
|
16
16
|
|
|
17
17
|
**runId** (`string`): The id of the run (optional).
|
|
18
18
|
|
|
@@ -30,7 +30,7 @@ This function returns an instance of the MastraScorer class. The `.run()` method
|
|
|
30
30
|
|
|
31
31
|
**reason** (`string`): Explanation of the score.
|
|
32
32
|
|
|
33
|
-
## Scoring
|
|
33
|
+
## Scoring details
|
|
34
34
|
|
|
35
35
|
The scorer evaluates relevancy through query-answer alignment, considering completeness and detail level, but not factual correctness.
|
|
36
36
|
|
|
@@ -58,7 +58,7 @@ A relevancy score between 0 and 1:
|
|
|
58
58
|
- **0.7–0.9**: The response mostly answers the query but may include minor unrelated content.
|
|
59
59
|
- **0.4–0.6**: The response partially answers the query, mixing relevant and unrelated information.
|
|
60
60
|
- **0.1–0.3**: The response includes minimal relevant content and largely misses the intent of the query.
|
|
61
|
-
- **0.0**: The response is entirely unrelated and
|
|
61
|
+
- **0.0**: The response is entirely unrelated and doesn't answer the query.
|
|
62
62
|
|
|
63
63
|
## Example
|
|
64
64
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Answer
|
|
1
|
+
# Answer similarity scorer
|
|
2
2
|
|
|
3
3
|
The `createAnswerSimilarityScorer()` function creates a scorer that evaluates how similar an agent's output is to a ground truth answer. This scorer is specifically designed for CI/CD testing scenarios where you have expected answers and want to ensure consistency over time.
|
|
4
4
|
|
|
@@ -24,7 +24,7 @@ The `createAnswerSimilarityScorer()` function creates a scorer that evaluates ho
|
|
|
24
24
|
|
|
25
25
|
This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but **requires ground truth** to be provided in the run object.
|
|
26
26
|
|
|
27
|
-
##
|
|
27
|
+
## `.run()` returns
|
|
28
28
|
|
|
29
29
|
**runId** (`string`): The id of the run (optional).
|
|
30
30
|
|
|
@@ -42,7 +42,7 @@ This function returns an instance of the MastraScorer class. The `.run()` method
|
|
|
42
42
|
|
|
43
43
|
**generateReasonPrompt** (`string`): The prompt used for generating the explanation.
|
|
44
44
|
|
|
45
|
-
## Scoring
|
|
45
|
+
## Scoring details
|
|
46
46
|
|
|
47
47
|
The scorer uses a multi-step process:
|
|
48
48
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Bias
|
|
1
|
+
# Bias scorer
|
|
2
2
|
|
|
3
3
|
The `createBiasScorer()` function accepts a single options object with the following properties:
|
|
4
4
|
|
|
@@ -10,7 +10,7 @@ The `createBiasScorer()` function accepts a single options object with the follo
|
|
|
10
10
|
|
|
11
11
|
This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
|
|
12
12
|
|
|
13
|
-
##
|
|
13
|
+
## `.run()` returns
|
|
14
14
|
|
|
15
15
|
**runId** (`string`): The id of the run (optional).
|
|
16
16
|
|
|
@@ -28,7 +28,7 @@ This function returns an instance of the MastraScorer class. The `.run()` method
|
|
|
28
28
|
|
|
29
29
|
**generateReasonPrompt** (`string`): The prompt sent to the LLM for the generateReason step (optional).
|
|
30
30
|
|
|
31
|
-
## Bias
|
|
31
|
+
## Bias categories
|
|
32
32
|
|
|
33
33
|
The scorer evaluates several types of bias:
|
|
34
34
|
|
|
@@ -37,7 +37,7 @@ The scorer evaluates several types of bias:
|
|
|
37
37
|
3. **Racial/Ethnic Bias**: Discrimination based on race, ethnicity, or national origin
|
|
38
38
|
4. **Geographical Bias**: Prejudice based on location or regional stereotypes
|
|
39
39
|
|
|
40
|
-
## Scoring
|
|
40
|
+
## Scoring details
|
|
41
41
|
|
|
42
42
|
The scorer evaluates bias through opinion analysis based on:
|
|
43
43
|
|
|
@@ -1,14 +1,14 @@
|
|
|
1
|
-
# Completeness
|
|
1
|
+
# Completeness scorer
|
|
2
2
|
|
|
3
3
|
The `createCompletenessScorer()` function evaluates how thoroughly an LLM's output covers the key elements present in the input. It analyzes nouns, verbs, topics, and terms to determine coverage and provides a detailed completeness score.
|
|
4
4
|
|
|
5
5
|
## Parameters
|
|
6
6
|
|
|
7
|
-
The `createCompletenessScorer()` function
|
|
7
|
+
The `createCompletenessScorer()` function doesn't take any options.
|
|
8
8
|
|
|
9
9
|
This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
|
|
10
10
|
|
|
11
|
-
##
|
|
11
|
+
## `.run()` returns
|
|
12
12
|
|
|
13
13
|
**runId** (`string`): The id of the run (optional).
|
|
14
14
|
|
|
@@ -31,7 +31,7 @@ The `.run()` method returns a result in the following shape:
|
|
|
31
31
|
}
|
|
32
32
|
```
|
|
33
33
|
|
|
34
|
-
## Element
|
|
34
|
+
## Element extraction details
|
|
35
35
|
|
|
36
36
|
The scorer extracts and analyzes several types of elements:
|
|
37
37
|
|
|
@@ -48,7 +48,7 @@ The extraction process includes:
|
|
|
48
48
|
- Special handling of short words (3 characters or less)
|
|
49
49
|
- Deduplication of elements
|
|
50
50
|
|
|
51
|
-
### extractStepResult
|
|
51
|
+
### `extractStepResult`
|
|
52
52
|
|
|
53
53
|
From the `.run()` method, you can get the `extractStepResult` object with the following properties:
|
|
54
54
|
|
|
@@ -57,7 +57,7 @@ From the `.run()` method, you can get the `extractStepResult` object with the fo
|
|
|
57
57
|
- **missingElements**: Input elements not found in the output.
|
|
58
58
|
- **elementCounts**: The number of elements in the input and output.
|
|
59
59
|
|
|
60
|
-
## Scoring
|
|
60
|
+
## Scoring details
|
|
61
61
|
|
|
62
62
|
The scorer evaluates completeness through linguistic element coverage analysis.
|
|
63
63
|
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Content
|
|
1
|
+
# Content similarity scorer
|
|
2
2
|
|
|
3
3
|
The `createContentSimilarityScorer()` function measures the textual similarity between two strings, providing a score that indicates how closely they match. It supports configurable options for case sensitivity and whitespace handling.
|
|
4
4
|
|
|
@@ -12,7 +12,7 @@ The `createContentSimilarityScorer()` function accepts a single options object w
|
|
|
12
12
|
|
|
13
13
|
This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
|
|
14
14
|
|
|
15
|
-
##
|
|
15
|
+
## `.run()` returns
|
|
16
16
|
|
|
17
17
|
**runId** (`string`): The id of the run (optional).
|
|
18
18
|
|
|
@@ -22,7 +22,7 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
|
|
|
22
22
|
|
|
23
23
|
**score** (`number`): Similarity score (0-1) where 1 indicates perfect similarity.
|
|
24
24
|
|
|
25
|
-
## Scoring
|
|
25
|
+
## Scoring details
|
|
26
26
|
|
|
27
27
|
The scorer evaluates textual similarity through character-level matching and configurable text normalization.
|
|
28
28
|
|
|
@@ -1,18 +1,18 @@
|
|
|
1
|
-
# Context
|
|
1
|
+
# Context precision scorer
|
|
2
2
|
|
|
3
3
|
The `createContextPrecisionScorer()` function creates a scorer that evaluates how relevant and well-positioned retrieved context pieces are for generating expected outputs. It uses **Mean Average Precision (MAP)** to reward systems that place relevant context earlier in the sequence.
|
|
4
4
|
|
|
5
|
-
It
|
|
5
|
+
It's especially useful for these use cases:
|
|
6
6
|
|
|
7
|
-
## RAG
|
|
7
|
+
## RAG system evaluation
|
|
8
8
|
|
|
9
9
|
Ideal for evaluating retrieved context in RAG pipelines where:
|
|
10
10
|
|
|
11
11
|
- Context ordering matters for model performance
|
|
12
|
-
- You need to measure retrieval quality beyond
|
|
12
|
+
- You need to measure retrieval quality beyond basic relevance
|
|
13
13
|
- Early relevant context is more valuable than later relevant context
|
|
14
14
|
|
|
15
|
-
## Context
|
|
15
|
+
## Context window optimization
|
|
16
16
|
|
|
17
17
|
Use when optimizing context selection for:
|
|
18
18
|
|
|
@@ -28,13 +28,13 @@ Use when optimizing context selection for:
|
|
|
28
28
|
|
|
29
29
|
**Note**: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.
|
|
30
30
|
|
|
31
|
-
##
|
|
31
|
+
## `.run()` returns
|
|
32
32
|
|
|
33
33
|
**score** (`number`): Mean Average Precision score between 0 and scale (default 0-1)
|
|
34
34
|
|
|
35
35
|
**reason** (`string`): Human-readable explanation of the context precision evaluation
|
|
36
36
|
|
|
37
|
-
## Scoring
|
|
37
|
+
## Scoring details
|
|
38
38
|
|
|
39
39
|
### Mean Average Precision (MAP)
|
|
40
40
|
|
|
@@ -77,7 +77,7 @@ The reason field explains:
|
|
|
77
77
|
Use results to:
|
|
78
78
|
|
|
79
79
|
- **Improve retrieval**: Filter out irrelevant context before ranking
|
|
80
|
-
- **Optimize ranking**: Ensure relevant context
|
|
80
|
+
- **Optimize ranking**: Ensure relevant context surfaces early
|
|
81
81
|
- **Tune chunk size**: Balance context detail vs. relevance precision
|
|
82
82
|
- **Evaluate embeddings**: Test different embedding models for better retrieval
|
|
83
83
|
|
|
@@ -177,7 +177,7 @@ For more details on `runEvals`, see the [runEvals reference](https://mastra.ai/r
|
|
|
177
177
|
|
|
178
178
|
To add this scorer to an agent, see the [Scorers overview](https://mastra.ai/docs/evals/overview) guide.
|
|
179
179
|
|
|
180
|
-
## Comparison with
|
|
180
|
+
## Comparison with context relevance
|
|
181
181
|
|
|
182
182
|
Choose the right scorer for your needs:
|
|
183
183
|
|
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# Context
|
|
1
|
+
# Context relevance scorer
|
|
2
2
|
|
|
3
3
|
The `createContextRelevanceScorerLLM()` function creates a scorer that evaluates how relevant and useful provided context was for generating agent responses. It uses weighted relevance levels and applies penalties for unused high-relevance context and missing information.
|
|
4
4
|
|
|
5
|
-
It
|
|
5
|
+
It's especially useful for these use cases:
|
|
6
6
|
|
|
7
|
-
## Content
|
|
7
|
+
## Content generation evaluation
|
|
8
8
|
|
|
9
9
|
Best for evaluating context quality in:
|
|
10
10
|
|
|
@@ -12,7 +12,7 @@ Best for evaluating context quality in:
|
|
|
12
12
|
- RAG pipelines needing nuanced relevance assessment
|
|
13
13
|
- Systems where missing context affects quality
|
|
14
14
|
|
|
15
|
-
## Context
|
|
15
|
+
## Context selection optimization
|
|
16
16
|
|
|
17
17
|
Use when optimizing for:
|
|
18
18
|
|
|
@@ -28,13 +28,13 @@ Use when optimizing for:
|
|
|
28
28
|
|
|
29
29
|
Note: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.
|
|
30
30
|
|
|
31
|
-
##
|
|
31
|
+
## `.run()` returns
|
|
32
32
|
|
|
33
33
|
**score** (`number`): Weighted relevance score between 0 and scale (default 0-1)
|
|
34
34
|
|
|
35
35
|
**reason** (`string`): Human-readable explanation of the context relevance evaluation
|
|
36
36
|
|
|
37
|
-
## Scoring
|
|
37
|
+
## Scoring details
|
|
38
38
|
|
|
39
39
|
### Weighted Relevance Scoring
|
|
40
40
|
|
|
@@ -512,7 +512,7 @@ const evaluateRAG = async testCases => {
|
|
|
512
512
|
}
|
|
513
513
|
```
|
|
514
514
|
|
|
515
|
-
## Comparison with
|
|
515
|
+
## Comparison with context precision
|
|
516
516
|
|
|
517
517
|
Choose the right scorer for your needs:
|
|
518
518
|
|