@mastra/mcp-docs-server 1.2.13-alpha.0 → 1.2.13-alpha.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (131) hide show
  1. package/.docs/docs/agents/agent-approval.md +72 -0
  2. package/.docs/docs/agents/overview.md +1 -1
  3. package/.docs/docs/agents/supervisor-agents.md +2 -2
  4. package/.docs/docs/agents/using-tools.md +61 -33
  5. package/.docs/docs/{evals/datasets → datasets}/overview.md +3 -3
  6. package/.docs/docs/{evals/datasets → datasets}/running-experiments.md +57 -9
  7. package/.docs/docs/deployment/workers.md +14 -14
  8. package/.docs/docs/editor/overview.md +201 -206
  9. package/.docs/docs/evals/evals-with-memory.md +1 -1
  10. package/.docs/docs/getting-started/develop.md +196 -0
  11. package/.docs/docs/index.md +78 -1
  12. package/.docs/docs/long-running-agents/durable-agents.md +2 -2
  13. package/.docs/docs/mastra-platform/overview.md +1 -1
  14. package/.docs/docs/mastra-platform/workspaces.md +152 -0
  15. package/.docs/docs/memory/observational-memory.md +53 -13
  16. package/.docs/docs/memory/overview.md +14 -0
  17. package/.docs/docs/server/auth/workers.md +7 -5
  18. package/.docs/docs/server/mastra-client.md +60 -0
  19. package/.docs/docs/server/pubsub.md +2 -2
  20. package/.docs/docs/studio/observability.md +4 -0
  21. package/.docs/docs/studio/overview.md +2 -2
  22. package/.docs/docs/workflows/overview.md +1 -1
  23. package/.docs/docs/workflows/scheduled-workflows.md +1 -0
  24. package/.docs/docs/workspace/search.md +1 -1
  25. package/.docs/guides/deployment/kubernetes.md +2 -0
  26. package/.docs/guides/deployment/mastra-workers.md +350 -6
  27. package/.docs/guides/deployment/vercel.md +2 -0
  28. package/.docs/guides/getting-started/quickstart.md +3 -3
  29. package/.docs/{docs → guides}/rag/chunking-and-embedding.md +1 -1
  30. package/.docs/{docs → guides}/rag/graph-rag.md +1 -1
  31. package/.docs/{docs → guides}/rag/overview.md +2 -2
  32. package/.docs/{docs → guides}/rag/retrieval.md +1 -1
  33. package/.docs/{docs → guides}/voice/overview.md +4 -4
  34. package/.docs/{docs → guides}/voice/realtime-voice.md +2 -2
  35. package/.docs/{docs → guides}/voice/speech-to-speech.md +1 -1
  36. package/.docs/{docs → guides}/voice/speech-to-text.md +1 -1
  37. package/.docs/{docs → guides}/voice/text-to-speech.md +2 -2
  38. package/.docs/models/embeddings.md +63 -65
  39. package/.docs/models/environment-variables.md +2 -1
  40. package/.docs/models/gateways/openrouter.md +3 -6
  41. package/.docs/models/gateways/vercel.md +4 -1
  42. package/.docs/models/index.md +1 -1
  43. package/.docs/models/providers/cortecs.md +2 -1
  44. package/.docs/models/providers/crof.md +2 -1
  45. package/.docs/models/providers/crossmodel.md +4 -3
  46. package/.docs/models/providers/empiriolabs.md +2 -1
  47. package/.docs/models/providers/greenpt.md +98 -0
  48. package/.docs/models/providers/huggingface.md +3 -1
  49. package/.docs/models/providers/hyper.md +10 -7
  50. package/.docs/models/providers/llmgateway.md +5 -4
  51. package/.docs/models/providers/minimax.md +1 -1
  52. package/.docs/models/providers/neuralwatt.md +3 -1
  53. package/.docs/models/providers/openai.md +2 -2
  54. package/.docs/models/providers/opencode-go.md +2 -1
  55. package/.docs/models/providers/opencode.md +1 -1
  56. package/.docs/models/providers/perplexity-agent.md +3 -1
  57. package/.docs/models/providers/scaleway.md +1 -3
  58. package/.docs/models/providers/the-grid-ai.md +3 -3
  59. package/.docs/models/providers/zai-coding-plan.md +11 -13
  60. package/.docs/models/providers/zhipuai-coding-plan.md +12 -12
  61. package/.docs/models/providers.md +1 -0
  62. package/.docs/reference/agents/agent.md +1 -1
  63. package/.docs/reference/agents/durable-agent.md +12 -1
  64. package/.docs/reference/agents/generate.md +1 -1
  65. package/.docs/reference/agents/getVoice.md +1 -1
  66. package/.docs/{docs/getting-started → reference}/build-with-ai.md +5 -110
  67. package/.docs/reference/cli/create-mastra.md +2 -0
  68. package/.docs/reference/cli/mastra.md +36 -16
  69. package/.docs/reference/client-js/agents.md +1 -1
  70. package/.docs/reference/configuration.md +2 -2
  71. package/.docs/reference/core/getEditor.md +1 -1
  72. package/.docs/reference/core/getVector.md +2 -2
  73. package/.docs/reference/core/listVectors.md +2 -2
  74. package/.docs/reference/core/mastra-class.md +2 -2
  75. package/.docs/reference/datasets/startExperiment.md +8 -0
  76. package/.docs/reference/datasets/startExperimentAsync.md +4 -0
  77. package/.docs/reference/editor/browser-provider.md +2 -2
  78. package/.docs/reference/editor/filesystem-provider.md +1 -1
  79. package/.docs/reference/editor/mastra-editor.md +31 -18
  80. package/.docs/reference/editor/prompt-blocks.md +137 -0
  81. package/.docs/reference/editor/sandbox-provider.md +1 -1
  82. package/.docs/reference/editor/storage-browser-ref.md +3 -3
  83. package/.docs/reference/editor/storage-workspace-ref.md +3 -3
  84. package/.docs/reference/editor/tool-provider.md +44 -10
  85. package/.docs/reference/editor/tools.md +100 -0
  86. package/.docs/reference/editor/versioning.md +130 -0
  87. package/.docs/reference/evals/mastra-scorer.md +68 -8
  88. package/.docs/reference/evals/summarization.md +203 -0
  89. package/.docs/reference/file-based-agents/config.md +39 -1
  90. package/.docs/reference/index.md +4 -4
  91. package/.docs/reference/memory/observational-memory.md +76 -24
  92. package/.docs/reference/memory/serialized-memory-config.md +3 -3
  93. package/.docs/reference/observability/tracing/interfaces.md +6 -1
  94. package/.docs/reference/processors/processor-interface.md +14 -2
  95. package/.docs/reference/processors/regex-filter-processor.md +81 -9
  96. package/.docs/reference/processors/token-limiter-processor.md +15 -1
  97. package/.docs/reference/project-structure.md +1 -1
  98. package/.docs/reference/rag/database-config.md +1 -1
  99. package/.docs/reference/storage/composite.md +2 -0
  100. package/.docs/reference/streaming/agents/stream.md +1 -1
  101. package/.docs/reference/tools/isolated-vm-transport.md +1 -1
  102. package/.docs/reference/vectors/mongodb.md +13 -13
  103. package/.docs/reference/voice/livekit.md +3 -3
  104. package/.docs/reference/workers/overview.md +10 -8
  105. package/.docs/reference/workspace/platform-filesystem.md +8 -5
  106. package/.docs/reference/workspace/platform-sandbox.md +83 -7
  107. package/CHANGELOG.md +51 -0
  108. package/README.md +1 -1
  109. package/package.json +5 -5
  110. package/.docs/docs/agent-builder/access-control.md +0 -99
  111. package/.docs/docs/agent-builder/browser.md +0 -63
  112. package/.docs/docs/agent-builder/channels.md +0 -78
  113. package/.docs/docs/agent-builder/configuration.md +0 -132
  114. package/.docs/docs/agent-builder/deploying.md +0 -124
  115. package/.docs/docs/agent-builder/integrations.md +0 -115
  116. package/.docs/docs/agent-builder/memory.md +0 -69
  117. package/.docs/docs/agent-builder/model-policy.md +0 -50
  118. package/.docs/docs/agent-builder/overview.md +0 -114
  119. package/.docs/docs/agent-builder/skill-registries.md +0 -31
  120. package/.docs/docs/agent-builder/workspace.md +0 -62
  121. package/.docs/docs/editor/prompts.md +0 -140
  122. package/.docs/docs/editor/tools.md +0 -148
  123. package/.docs/docs/getting-started/file-based-agents.md +0 -121
  124. package/.docs/docs/mastra-platform/workspace.md +0 -111
  125. package/.docs/docs/what-is-mastra.md +0 -161
  126. package/.docs/reference/client-js/agent-builder.md +0 -163
  127. package/.docs/reference/editor/agent-builder/agent-builder-options.md +0 -76
  128. package/.docs/reference/editor/agent-builder/builder-agent-defaults.md +0 -79
  129. package/.docs/reference/editor/agent-builder/builder-models.md +0 -66
  130. /package/.docs/{docs → guides}/getting-started/manual-install.md +0 -0
  131. /package/.docs/{docs → guides}/rag/vector-databases.md +0 -0
@@ -104,6 +104,78 @@ A tool's own `requireApproval` setting takes precedence over the function above.
104
104
 
105
105
  > **Note:** Function-based `requireToolApproval` is only available on regular `stream()` / `generate()` calls. Durable agents and stored agents persist their options, and a function can't be serialized, so they accept only a boolean. If you pass a function in those contexts it falls back to requiring approval for every tool call.
106
106
 
107
+ #### Bind approval to the exact tool arguments
108
+
109
+ For sensitive tools, bind the approval to the exact tool name and arguments that were shown to the reviewer. If those arguments drift before execution, the tool shouldn't run under the old approval.
110
+
111
+ The `tool-call-approval` chunk already includes `toolName`, `toolCallId`, and `args`. You can fingerprint those fields when the approval request is shown. The example below uses a JSON string as the fingerprint, but in production you should use a stable hash of the tool name and arguments:
112
+
113
+ ```typescript
114
+ import { Agent } from '@mastra/core/agent'
115
+
116
+ // For your production usecase, build a stable hash of the tool name and args
117
+ function actionFingerprint(toolName: string, args: unknown) {
118
+ const payload = JSON.stringify({ toolName, args })
119
+ return `fingerprint-${payload}`
120
+ }
121
+
122
+ const sensitiveTools = new Set(['issue_refund', 'delete_record'])
123
+ const approvedFingerprints = new Set<string>()
124
+
125
+ export const approvalBoundAgent = new Agent({
126
+ id: 'approval-bound-agent',
127
+ name: 'Approval Bound Agent',
128
+ model: 'openai/gpt-5.6-sol',
129
+ tools: { issueRefundTool, deleteRecordTool },
130
+ hooks: {
131
+ beforeToolCall: ({ toolName, input }) => {
132
+ if (!sensitiveTools.has(toolName)) return
133
+
134
+ const fingerprint = actionFingerprint(toolName, input)
135
+ if (!approvedFingerprints.delete(fingerprint)) {
136
+ return {
137
+ proceed: false,
138
+ output: `Tool call blocked: approval did not match ${toolName} arguments.`,
139
+ }
140
+ }
141
+ },
142
+ },
143
+ })
144
+ ```
145
+
146
+ ```typescript
147
+ const stream = await approvalBoundAgent.stream('Refund order ord-1042', {
148
+ requireToolApproval: ({ toolName }) => sensitiveTools.has(toolName),
149
+ })
150
+
151
+ async function consumeApprovalStream(currentStream: typeof stream) {
152
+ for await (const chunk of currentStream.fullStream) {
153
+ if (chunk.type === 'tool-call-approval') {
154
+ const { toolName, toolCallId, args } = chunk.payload
155
+ const fingerprint = actionFingerprint(toolName, args)
156
+
157
+ // Present toolName, args, and fingerprint to your approval UI.
158
+ const approved = await showApprovalDialog({ toolName, args, fingerprint })
159
+
160
+ const nextStream = approved
161
+ ? await approveReviewedToolCall(currentStream.runId, toolCallId, fingerprint)
162
+ : await approvalBoundAgent.declineToolCall({ runId: currentStream.runId, toolCallId })
163
+
164
+ await consumeApprovalStream(nextStream)
165
+ }
166
+ }
167
+ }
168
+
169
+ async function approveReviewedToolCall(runId: string, toolCallId: string, fingerprint: string) {
170
+ approvedFingerprints.add(fingerprint)
171
+ return approvalBoundAgent.approveToolCall({ runId, toolCallId })
172
+ }
173
+
174
+ await consumeApprovalStream(stream)
175
+ ```
176
+
177
+ In production, store the approved fingerprint in durable storage scoped to the user, run, tool call, and policy version. The `Set` above is intentionally small so the boundary is clear: the approval is consumed once, and only for the same canonical tool arguments that were reviewed.
178
+
107
179
  ### Runtime suspension with `suspend()`
108
180
 
109
181
  A tool can also pause _during_ its `execute` function by calling `suspend()`. This is useful when the tool starts running and then discovers it needs additional user input or confirmation before it can finish.
@@ -210,7 +210,7 @@ Once your agent is running, use this table to find the right page for what you w
210
210
  | Keep your agent safe | [Guardrails](https://mastra.ai/docs/agents/guardrails) |
211
211
  | Build agents that correct their work | [Rubric scorer](https://mastra.ai/docs/agents/supervisor-agents) |
212
212
  | Swap instructions or models based on request context | [Dynamic configuration](https://mastra.ai/docs/server/request-context) |
213
- | Add speech-to-text or text-to-speech | [Voice](https://mastra.ai/docs/voice/overview) |
213
+ | Add speech-to-text or text-to-speech | [Voice](https://mastra.ai/guides/voice/overview) |
214
214
  | Connect to Slack, Discord, or Telegram | [Channels](https://mastra.ai/docs/capabilities/channels/overview) |
215
215
 
216
216
  ## Multi-agent systems
@@ -420,12 +420,12 @@ const result = await supervisor.generate('Research and write about AI safety', {
420
420
  })
421
421
  ```
422
422
 
423
- Version overrides propagate automatically through delegation. See [Subagent versioning](https://mastra.ai/docs/editor/overview) for details on resolution order and server API usage.
423
+ Version overrides propagate automatically through delegation. See [Subagent versioning](https://mastra.ai/reference/editor/versioning) for details on resolution order and server API usage.
424
424
 
425
425
  ## Related
426
426
 
427
427
  - [Background tasks](https://mastra.ai/docs/long-running-agents/background-tasks)
428
- - [Subagent versioning](https://mastra.ai/docs/editor/overview)
428
+ - [Subagent versioning](https://mastra.ai/reference/editor/versioning)
429
429
  - [Guide: Research coordinator](https://mastra.ai/guides/guide/research-coordinator)
430
430
  - [Agent.stream() reference](https://mastra.ai/reference/streaming/agents/stream)
431
431
  - [Agent.streamUntilIdle() reference](https://mastra.ai/reference/streaming/agents/streamUntilIdle)
@@ -65,31 +65,6 @@ export const weatherAgent = new Agent({
65
65
  })
66
66
  ```
67
67
 
68
- ## Use provider web search
69
-
70
- Import `webSearchTool` from `@mastra/core/tools` when you want the model provider to run its native web search tool. The tool has no local `execute` function. Mastra resolves it at run time from the active model, then passes the provider-managed tool to the model.
71
-
72
- ```typescript
73
- import { Agent } from '@mastra/core/agent'
74
- import { webSearchTool } from '@mastra/core/tools'
75
-
76
- export const researchAgent = new Agent({
77
- id: 'research-agent',
78
- name: 'Research Agent',
79
- instructions: `
80
- You are a helpful research assistant.
81
- Use web search when you need current information.`,
82
- model: 'openai/gpt-5.6-sol',
83
- tools: {
84
- webSearch: webSearchTool,
85
- },
86
- })
87
- ```
88
-
89
- `webSearchTool` supports OpenAI, Anthropic, Google Gemini, and xAI models. If Mastra can't infer one of those providers from the active model, the agent run fails with a `MastraError`.
90
-
91
- Only the `webSearchTool` value triggers provider web search. Custom tools with the names `webSearch` or `web_search` stay unchanged.
92
-
93
68
  ## Define schemas
94
69
 
95
70
  You can define the tool's `inputSchema` and `outputSchema` with any library that supports [Standard JSON Schema](https://standardschema.dev/json-schema). This includes libraries like [Zod](https://zod.dev/), [Valibot](https://valibot.dev/), and [ArkType](https://arktype.io/).
@@ -494,14 +469,67 @@ Note that for subagents, you'll see two different identifiers in stream response
494
469
 
495
470
  Mastra includes agent-agnostic built-in tools in `@mastra/core/tools` that add interactive and organizational capabilities to any agent.
496
471
 
497
- | Tool | Purpose |
498
- | --------------- | ------------------------------------------------- |
499
- | `ask_user` | Ask the user a question and wait for their answer |
500
- | `submit_plan` | Submit a plan file for user approval |
501
- | `task_write` | Create or replace a structured task list |
502
- | `task_update` | Update one tracked task by ID |
503
- | `task_complete` | Mark one tracked task completed |
504
- | `task_check` | Check task list completion status |
472
+ | Tool | Purpose |
473
+ | --------------- | ---------------------------------------------------- |
474
+ | `ask_user` | Ask the user a question and wait for their answer |
475
+ | `submit_plan` | Submit a plan file for user approval |
476
+ | `task_write` | Create or replace a structured task list |
477
+ | `task_update` | Update one tracked task by ID |
478
+ | `task_complete` | Mark one tracked task completed |
479
+ | `task_check` | Check task list completion status |
480
+ | `webSearchTool` | Run provider-native web search with the active model |
481
+ | `webFetchTool` | Fetch a web page by URL and return its text content |
482
+
483
+ ### Use provider web search
484
+
485
+ Import `webSearchTool` from `@mastra/core/tools` when you want the model provider to run its native web search tool. Mastra resolves it at run time from the active model, then passes the provider-managed tool to the model.
486
+
487
+ ```typescript
488
+ import { Agent } from '@mastra/core/agent'
489
+ import { webSearchTool } from '@mastra/core/tools'
490
+
491
+ export const researchAgent = new Agent({
492
+ id: 'research-agent',
493
+ name: 'Research Agent',
494
+ instructions: 'Use web search when you need current information.',
495
+ model: 'openai/gpt-5.6-sol',
496
+ tools: {
497
+ search: webSearchTool,
498
+ },
499
+ })
500
+ ```
501
+
502
+ `webSearchTool` supports OpenAI, Anthropic, Google Gemini, and xAI models. If Mastra can't infer one of those providers from the active model, the agent run fails with a `MastraError`.
503
+
504
+ The `search` key is only the agent-local tool name. Use any key. The `webSearchTool` value tells Mastra to use provider web search.
505
+
506
+ ### Fetch a web page
507
+
508
+ Import `webFetchTool` from `@mastra/core/tools` when the agent needs to read a specific URL. The tool requests the page over HTTP or HTTPS and returns its text content plus response metadata.
509
+
510
+ ```typescript
511
+ import { Agent } from '@mastra/core/agent'
512
+ import { webFetchTool } from '@mastra/core/tools'
513
+
514
+ export const readerAgent = new Agent({
515
+ id: 'reader-agent',
516
+ name: 'Reader Agent',
517
+ instructions: 'Fetch the page the user links to before answering.',
518
+ model: 'openai/gpt-5.6-sol',
519
+ tools: {
520
+ fetch: webFetchTool,
521
+ },
522
+ })
523
+ ```
524
+
525
+ The tool takes a single `url` input and returns `content`, `truncated`, `status`, `statusText`, `contentType`, `url`, and `ok`. It applies these limits:
526
+
527
+ - Only `http:` and `https:` URLs are allowed.
528
+ - Requests to `localhost` and to private or reserved IP addresses are blocked, including addresses returned by DNS resolution.
529
+ - Responses are truncated at 100,000 characters, with `truncated: true` in the result.
530
+ - Requests follow at most 5 redirects and time out after 15 seconds.
531
+
532
+ Failures don't throw. The tool returns `isError: true` with the reason in `content`, so the agent can retry or explain the problem.
505
533
 
506
534
  ### Ask the user a question
507
535
 
@@ -49,7 +49,7 @@ Visit the [`DatasetsManager` reference](https://mastra.ai/reference/datasets/dat
49
49
 
50
50
  You can also manage datasets in [Studio](https://mastra.ai/docs/studio/overview). After opening Studio, select **Datasets** from the sidebar to see all your available datasets or create a new one.
51
51
 
52
- To get started, select **Create Dataset** and set a name, description, and optional schemas. After confirming, you'll see the dataset details page with two tabs: **Items** and [**Experiments**](https://mastra.ai/docs/evals/datasets/running-experiments).
52
+ To get started, select **Create Dataset** and set a name, description, and optional schemas. After confirming, you'll see the dataset details page with two tabs: **Items** and [**Experiments**](https://mastra.ai/docs/datasets/running-experiments).
53
53
 
54
54
  In the **Items** view you can add, update, and delete items, and view version history. Select **Add Item** to insert a new item with JSON editors for input and ground truth. From this view you can also import items in bulk from a CSV or JSON file. When importing, map each column to the corresponding dataset field.
55
55
 
@@ -198,11 +198,11 @@ Fetch the exact items that existed at a past version:
198
198
  const items = await dataset.listItems({ version: 2 })
199
199
  ```
200
200
 
201
- You can also pin experiments to a version, see [running experiments](https://mastra.ai/docs/evals/datasets/running-experiments). Visit the [`Dataset` reference](https://mastra.ai/reference/datasets/dataset) for the full list of methods and parameters.
201
+ You can also pin experiments to a version, see [running experiments](https://mastra.ai/docs/datasets/running-experiments). Visit the [`Dataset` reference](https://mastra.ai/reference/datasets/dataset) for the full list of methods and parameters.
202
202
 
203
203
  ## Related
204
204
 
205
- - [Running experiments](https://mastra.ai/docs/evals/datasets/running-experiments)
205
+ - [Running experiments](https://mastra.ai/docs/datasets/running-experiments)
206
206
  - [Scorers overview](https://mastra.ai/docs/evals/overview)
207
207
  - [DatasetsManager reference](https://mastra.ai/reference/datasets/datasets-manager)
208
208
  - [Dataset reference](https://mastra.ai/reference/datasets/dataset)
@@ -4,7 +4,7 @@
4
4
 
5
5
  **Added in:** `@mastra/core@1.4.0`
6
6
 
7
- An experiment runs every item in a dataset through a target (an agent, a workflow, or a scorer) and then optionally scores the outputs. Use a scorer as the target when you want to evaluate an LLM judge itself. Results are persisted to storage so you can compare runs across different prompts, models, or code changes.
7
+ An experiment runs every item in a dataset through a target (an agent, a workflow, or a scorer) and then optionally scores the outputs. Use a scorer as the target when you want to evaluate an LLM judge itself. By default, results are persisted to storage so you can compare runs across different prompts, models, or code changes.
8
8
 
9
9
  **For AI agents:** Run `npx mastra api experiment run dataset_123 '{"name":"translation-baseline"}'` to start an experiment directly instead of opening Studio or writing a temporary script. Use a dataset ID returned by `npx mastra api dataset list` in place of the sample ID. The command requires a running Mastra server with dataset storage and registered experiment targets; start the local server with `npx mastra dev`, or pass the reachable server's base URL with `--url`. Run `npx mastra api experiment run --schema` before constructing different input, and get user approval before starting an experiment because it can make model calls. Install Mastra's skill with `npx skills add mastra-ai/skills --skill mastra` for complete API CLI discovery, targeting, schema, authentication, and error-handling guidance.
10
10
 
@@ -131,9 +131,33 @@ for (const item of summary.results) {
131
131
 
132
132
  Visit the [Scorers overview](https://mastra.ai/docs/evals/overview) for details on available and custom scorers.
133
133
 
134
+ ## Control persistence per run
135
+
136
+ Use `persistence` to skip storage writes for a specific run. Experiment records and score records can be disabled independently:
137
+
138
+ ```typescript
139
+ const summary = await dataset.startExperiment({
140
+ targetType: 'agent',
141
+ targetId: 'translation-agent',
142
+ scorers: ['accuracy'],
143
+ persistence: {
144
+ experiments: 'none',
145
+ scores: 'none',
146
+ },
147
+ })
148
+ ```
149
+
150
+ The target and scorers still run, and `startExperiment()` still returns the item results and scores in `summary`. The settings are independent. For example, set only `scores: 'none'` to persist the experiment and its item results without creating score records.
151
+
152
+ Omitted settings default to `'default'`, which preserves the standard storage behavior. This policy only controls experiment and score records created by the run. It doesn't disable storage used by the target, such as agent memory, vectors, observability, or custom tool storage.
153
+
154
+ When `startExperimentAsync()` runs with `experiments: 'none'`, it doesn't persist an experiment record, progress updates, or item results. Score persistence remains controlled separately by `persistence.scores`. Without an experiment event observer, the run is fire-and-forget, and the experiment API can't report whether it completed or failed.
155
+
156
+ Use synchronous `startExperiment()` when the caller needs the returned summary. An experiment event observer can receive lifecycle events and the terminal summary.
157
+
134
158
  ## Tool mocks
135
159
 
136
- When an experiment runs an agent that calls side-effecting tools, you can make the run deterministic by attaching static tool mocks to individual dataset items. During the experiment, a mocked tool returns its declared output instead of executing. Tools that have no mock on the item run live.
160
+ When an experiment runs an agent that calls side-effecting tools, attach static tool mocks to individual dataset items to make the run deterministic. During the experiment, a mocked tool returns its declared output instead of executing. Tools without a mock on the item run live by default.
137
161
 
138
162
  Mocks live on the dataset item, so they version with the row and travel with the test case. Each mock declares a tool name, the arguments it expects, and the output to return:
139
163
 
@@ -152,6 +176,29 @@ await dataset.addItem({
152
176
 
153
177
  Tool mocks are supported for `agent` targets only.
154
178
 
179
+ ### Block undeclared tools
180
+
181
+ Set `unmockedToolPolicy: 'deny'` on an experiment to block every tool call that doesn't have a mock. This is useful when a live call could cause side effects:
182
+
183
+ ```typescript
184
+ const summary = await dataset.startExperiment({
185
+ targetType: 'agent',
186
+ targetId: 'weather-agent',
187
+ unmockedToolPolicy: 'deny',
188
+ })
189
+ ```
190
+
191
+ The default policy is `'allow'`. You can override the experiment policy on an individual stored or inline item:
192
+
193
+ ```typescript
194
+ await dataset.addItem({
195
+ input: 'What is the weather in Seattle?',
196
+ unmockedToolPolicy: 'allow',
197
+ })
198
+ ```
199
+
200
+ The item value takes precedence over the experiment value. A denied call fails with `TOOL_MOCK_NOT_DECLARED` before the tool executes. The failure isn't retried or added to `liveCalls`.
201
+
155
202
  ### Matching and consumption
156
203
 
157
204
  Arguments are matched strictly: object key order is ignored and array order is substantial, plus there is no type coercion. A mock is served only when the agent calls the tool with arguments that deep-equal the mock's `args`.
@@ -175,14 +222,15 @@ This is useful when a tool's arguments are noisy or generated by the model. The
175
222
 
176
223
  ### Failures
177
224
 
178
- A mocked tool call fails the item when the arguments don't match or all matching mocks have been consumed:
225
+ A tool call fails the item when it violates the mock configuration:
179
226
 
180
227
  - `TOOL_MOCK_MISMATCH`: the tool was called with arguments that no mock matches.
181
228
  - `TOOL_MOCK_EXHAUSTED`: every matching mock has already been consumed.
229
+ - `TOOL_MOCK_NOT_DECLARED`: the tool has no mock and the effective `unmockedToolPolicy` is `'deny'`.
182
230
 
183
- When a mocked tool is mis-called, the agent run is aborted immediately, so the model can't go on to call any further tools, including unmocked, side-effecting tools that would otherwise run live. These failures are deterministic, so they're not retried. Mocks that are declared but never used don't fail the item, they're reported as unconsumed.
231
+ On any of these failures, the agent run is aborted immediately, so the model can't go on to call any further tools, including unmocked, side-effecting tools that would otherwise run live. These failures are deterministic, so they're not retried. Mocks that are declared but never used don't fail the item, they're reported as unconsumed.
184
232
 
185
- While an item has mocks, the agent's tools execute sequentially so repeated `(toolName, args)` mocks are consumed in the provider's call order. This serialization applies only to items that declare mocks.
233
+ While mock interception is active, the agent's tools execute sequentially so repeated `(toolName, args)` mocks are consumed in the provider's call order. Interception is active when the item declares mocks or its effective `unmockedToolPolicy` is `'deny'`.
186
234
 
187
235
  ### Diagnostics
188
236
 
@@ -195,8 +243,8 @@ for (const item of summary.results) {
195
243
 
196
244
  console.log(report.served) // mocks matched and returned
197
245
  console.log(report.unconsumed) // mocks declared but never used
198
- console.log(report.liveCalls) // unmocked tools that ran live
199
- console.log(report.failure) // the mismatch/exhausted failure, if any
246
+ console.log(report.liveCalls) // undeclared tools allowed to run live
247
+ console.log(report.failure) // the first deterministic mock failure, if any
200
248
  }
201
249
  ```
202
250
 
@@ -205,7 +253,7 @@ In [Studio](https://mastra.ai/docs/studio/overview), edit a dataset item to auth
205
253
  ### Limitations
206
254
 
207
255
  - **No tool span for mocked calls.** A mocked call returns its output before the tool executes, so it doesn't create a tool span. Trajectory scorers backed by stored traces may therefore not see mocked tool calls. Trajectory extraction that falls back to the agent's message output still sees them, so trajectory scoring can differ depending on your observability configuration.
208
- - **Storage support.** Tool mocks and tool mock reports are persisted by the LibSQL, PostgreSQL, MongoDB, and Spanner adapters. The MySQL adapter doesn't support them and rejects writes that carry tool mocks or a tool mock report so the feature never silently runs tools live.
256
+ - **Storage support.** Tool mocks and tool mock reports are persisted by the LibSQL, PostgreSQL, MongoDB, and Spanner adapters. The MySQL adapter doesn't support them and rejects writes that carry tool mocks or a tool mock report. All dataset storage adapters persist `unmockedToolPolicy`.
209
257
 
210
258
  ## Async experiments
211
259
 
@@ -350,7 +398,7 @@ Visit the [`startExperiment` reference](https://mastra.ai/reference/datasets/sta
350
398
 
351
399
  ## Related
352
400
 
353
- - [Datasets overview](https://mastra.ai/docs/evals/datasets/overview)
401
+ - [Datasets overview](https://mastra.ai/docs/datasets/overview)
354
402
  - [Scorers overview](https://mastra.ai/docs/evals/overview)
355
403
  - [`startExperiment` reference](https://mastra.ai/reference/datasets/startExperiment)
356
404
  - [`listExperimentResults` reference](https://mastra.ai/reference/datasets/listExperimentResults)
@@ -17,7 +17,7 @@ Workers matter when any of these apply:
17
17
  - Different parts of the system need to scale independently (e.g., more orchestration capacity without more API instances)
18
18
  - Background tool calls should run on dedicated compute
19
19
 
20
- If your application handles light traffic and workflows complete quickly, the default in-process setup works fine. Skip the worker infrastructure until you need it.
20
+ If your application handles light traffic and workflows complete fast, the default in-process setup works fine. Skip the worker infrastructure until you need it.
21
21
 
22
22
  ## Worker types
23
23
 
@@ -33,11 +33,11 @@ The orchestration worker requires a PubSub backend that supports pull mode (e.g.
33
33
 
34
34
  ### Scheduler worker
35
35
 
36
- Polls storage for due cron schedules and publishes `workflow.start` events. It is a producer only, meaning it creates work for the orchestration worker to pick up.
36
+ Polls storage for due cron schedules and publishes `workflow.start` events. It's a producer only, meaning it creates work for the orchestration worker to pick up.
37
37
 
38
38
  The scheduler reads declarative `schedule` fields from your workflow definitions automatically. See [Scheduled workflows](https://mastra.ai/docs/workflows/scheduled-workflows) for how to declare schedules.
39
39
 
40
- **Do not run more than one scheduler instance.** Multiple schedulers polling the same storage would fire duplicate events for the same schedule.
40
+ **Don't run more than one scheduler instance.** Multiple schedulers polling the same storage would fire duplicate events for the same schedule.
41
41
 
42
42
  ### Background task worker
43
43
 
@@ -47,7 +47,7 @@ The background task worker manages concurrency limits, task lifecycle, and resul
47
47
 
48
48
  ## How workers run
49
49
 
50
- ### In-process (default)
50
+ ### In-process mode (default)
51
51
 
52
52
  With no configuration, Mastra creates and starts workers inside the API process. Events flow through an in-memory PubSub, and everything shares a single Node.js runtime.
53
53
 
@@ -64,7 +64,7 @@ This setup needs no external infrastructure beyond your storage adapter. It does
64
64
 
65
65
  ### Split processes
66
66
 
67
- To run workers separately, configure a distributed [PubSub](https://mastra.ai/docs/server/pubsub) backend and use the `MASTRA_WORKERS` environment variable to control which workers start in each process.
67
+ To run workers in their own processes, configure a distributed [PubSub](https://mastra.ai/docs/server/pubsub) backend and use the `MASTRA_WORKERS` environment variable to control which workers start in each process.
68
68
 
69
69
  **Redis Streams + PostgreSQL**:
70
70
 
@@ -100,38 +100,38 @@ export const mastra = new Mastra({
100
100
  })
101
101
  ```
102
102
 
103
- Any [supported storage backend](https://mastra.ai/reference/workers/overview) works swap the storage adapter for your preferred database.
103
+ Any [supported storage backend](https://mastra.ai/reference/workers/overview) works. Swap the storage adapter for your preferred database.
104
104
 
105
105
  Run the same build artifact in multiple containers, each with a different [`MASTRA_WORKERS`](https://mastra.ai/reference/workers/overview) value to control which worker starts in each process.
106
106
 
107
107
  Split deployments require a distributed PubSub backend ([`RedisStreamsPubSub`](https://mastra.ai/reference/pubsub/redis-streams) or [`GoogleCloudPubSub`](https://mastra.ai/reference/pubsub/google-cloud-pubsub)), a shared [storage backend](https://mastra.ai/reference/workers/overview), and network connectivity between the orchestration worker and the API.
108
108
 
109
- The [worker deployment guide](https://mastra.ai/guides/deployment/mastra-workers) walks through this setup with a Docker Compose example.
109
+ The [worker deployment guide](https://mastra.ai/guides/deployment/mastra-workers) walks through this setup with Docker Compose and Kubernetes examples.
110
110
 
111
111
  ## Network architecture
112
112
 
113
- Workers are internal infrastructure. They are not exposed to end users and do not need their own subdomain, public URL, or inbound HTTP route.
113
+ Workers are internal infrastructure. They're not exposed to end users and don't need their own subdomain, public URL, or inbound HTTP route.
114
114
 
115
115
  In a split deployment:
116
116
 
117
- - **The API server is the only public-facing process.** It serves all client HTTP requests REST endpoints, agent interactions, workflow triggers, and any custom routes.
118
- - **Workers connect outbound only.** They pull events from the distributed PubSub backend and read/write to the shared storage database. They do not accept inbound traffic from clients.
119
- - **The orchestration worker calls the API internally.** It sends step execution requests to the API over the container network using `MASTRA_STEP_EXECUTION_URL`. This is internal service-to-service communication, not a public endpoint.
117
+ - **The API server is the only public-facing process**: It serves all client HTTP requests, including REST endpoints, agent interactions, workflow triggers, and any custom routes.
118
+ - **Workers connect outbound only**: They pull events from the distributed PubSub backend and read/write to the shared storage database. They don't accept inbound traffic from clients.
119
+ - **The orchestration worker calls the API internally**: It sends step execution requests to the API over the container network using `MASTRA_STEP_EXECUTION_URL`. This is internal service-to-service communication, not a public endpoint.
120
120
 
121
121
  All three worker types (orchestration, scheduler, background task) sit behind the API on a private network. They share access to the PubSub backend and storage database but never receive traffic directly from clients. If a worker-related feature needs an HTTP route (for example, token minting for a voice integration), that route runs on the API server, not on the worker process.
122
122
 
123
123
  ## Known limitations
124
124
 
125
- - **No dead-letter queue**: Failed events are nacked and retried, but there is no DLQ for events that repeatedly fail.
125
+ - **No dead-letter queue**: Failed events are nacked and retried, but there's no DLQ for events that fail after all retries.
126
126
  - **No built-in health endpoint**: Workers don't expose an HTTP health check. Use container-level liveness probes or process monitoring.
127
127
  - **Scheduler is single-instance**: Running multiple scheduler processes causes duplicate schedule fires.
128
128
  - **Runs stuck in "running" after API crash**: If the API process crashes while executing a workflow step, the run remains in `running` status with no automatic retry. For [durable agents](https://mastra.ai/docs/long-running-agents/durable-agents), set `recovery.durableAgents` to `'auto'` in the Mastra config to automatically re-drive orphaned runs on server restart. See [Crash recovery](https://mastra.ai/docs/long-running-agents/durable-agents) for details.
129
129
 
130
130
  ## Related
131
131
 
132
- - [Worker deployment guide](https://mastra.ai/guides/deployment/mastra-workers): Docker Compose example and topology options
132
+ - [Worker deployment guide](https://mastra.ai/guides/deployment/mastra-workers): Docker Compose and Kubernetes examples
133
133
  - [Worker authentication](https://mastra.ai/docs/server/auth/workers): Secure worker-to-API communication
134
- - [Workers reference](https://mastra.ai/reference/workers/overview): Environment variables, worker types, and storage backends
134
+ - [Workers reference](https://mastra.ai/reference/workers/overview): Details about worker environment variables and types, with a list of supported storage backends
135
135
  - [CLI reference](https://mastra.ai/reference/cli/mastra): `mastra worker build` and `mastra worker start`
136
136
  - [PubSub](https://mastra.ai/docs/server/pubsub): Event delivery backends
137
137
  - [Scheduled workflows](https://mastra.ai/docs/workflows/scheduled-workflows): Declare cron schedules on workflows