@mastra/mcp-docs-server 1.2.8-alpha.2 → 1.2.8-alpha.25

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (154) hide show
  1. package/.docs/docs/agent-builder/integrations.md +1 -3
  2. package/.docs/docs/agent-builder/overview.md +2 -0
  3. package/.docs/docs/agents/overview.md +17 -13
  4. package/.docs/docs/agents/skills.md +1 -3
  5. package/.docs/docs/agents/structured-output.md +14 -7
  6. package/.docs/docs/agents/using-tools.md +66 -49
  7. package/.docs/docs/browser/agent-browser.md +1 -0
  8. package/.docs/docs/browser/firecrawl.md +129 -0
  9. package/.docs/docs/browser/overview.md +16 -2
  10. package/.docs/docs/capabilities/channels/slack.md +111 -1
  11. package/.docs/docs/deployment/mastra-server.md +35 -0
  12. package/.docs/docs/deployment/monorepo.md +2 -0
  13. package/.docs/docs/deployment/overview.md +6 -0
  14. package/.docs/docs/deployment/sandbox.md +277 -0
  15. package/.docs/docs/editor/overview.md +2 -6
  16. package/.docs/docs/editor/tools.md +2 -6
  17. package/.docs/docs/evals/multi-turn.md +178 -0
  18. package/.docs/docs/evals/overview.md +1 -0
  19. package/.docs/docs/evals/running-in-ci.md +3 -9
  20. package/.docs/docs/getting-started/build-with-ai.md +2 -6
  21. package/.docs/docs/getting-started/manual-install.md +1 -1
  22. package/.docs/docs/index.md +17 -13
  23. package/.docs/docs/long-running-agents/background-tasks.md +2 -2
  24. package/.docs/docs/long-running-agents/durable-agents.md +2 -1
  25. package/.docs/docs/long-running-agents/goals.md +3 -1
  26. package/.docs/docs/mastra-platform/database.md +12 -6
  27. package/.docs/docs/mastra-platform/deploy.md +14 -7
  28. package/.docs/docs/mastra-platform/environments.md +5 -4
  29. package/.docs/docs/mastra-platform/observability.md +4 -12
  30. package/.docs/docs/mastra-platform/overview.md +1 -1
  31. package/.docs/docs/mastra-platform/regions.md +73 -0
  32. package/.docs/docs/mastra-platform/workspace.md +111 -0
  33. package/.docs/docs/mcp/overview.md +1 -1
  34. package/.docs/docs/memory/observational-memory.md +77 -2
  35. package/.docs/docs/memory/overview.md +4 -0
  36. package/.docs/docs/observability/feedback.md +4 -0
  37. package/.docs/docs/observability/integrations/bridges/otel.md +1 -3
  38. package/.docs/docs/observability/integrations/exporters/otel.md +1 -3
  39. package/.docs/docs/server/auth/fga.md +40 -0
  40. package/.docs/docs/server/auth/simple-auth.md +1 -3
  41. package/.docs/docs/server/request-context.md +2 -0
  42. package/.docs/docs/studio/overview.md +2 -0
  43. package/.docs/docs/workflows/control-flow.md +1 -3
  44. package/.docs/docs/workspace/filesystem.md +1 -0
  45. package/.docs/docs/workspace/sandbox.md +1 -0
  46. package/.docs/guides/getting-started/quickstart.md +40 -25
  47. package/.docs/guides/index.md +1 -1
  48. package/.docs/guides/migrations/vnext-to-standard-apis.md +1 -3
  49. package/.docs/models/environment-variables.md +3 -0
  50. package/.docs/models/gateways/netlify.md +3 -1
  51. package/.docs/models/gateways/openrouter.md +343 -345
  52. package/.docs/models/gateways/vercel.md +8 -5
  53. package/.docs/models/index.md +1 -1
  54. package/.docs/models/providers/aki-io.md +78 -0
  55. package/.docs/models/providers/alibaba-token-plan-cn.md +22 -21
  56. package/.docs/models/providers/alibaba-token-plan.md +22 -21
  57. package/.docs/models/providers/ambient.md +12 -10
  58. package/.docs/models/providers/baseten.md +3 -2
  59. package/.docs/models/providers/cline-pass.md +82 -0
  60. package/.docs/models/providers/crossmodel.md +2 -1
  61. package/.docs/models/providers/deepinfra.md +4 -7
  62. package/.docs/models/providers/deepseek.md +3 -1
  63. package/.docs/models/providers/empiriolabs.md +2 -1
  64. package/.docs/models/providers/evroc.md +3 -2
  65. package/.docs/models/providers/google.md +6 -4
  66. package/.docs/models/providers/inferx.md +3 -3
  67. package/.docs/models/providers/kilo.md +1 -1
  68. package/.docs/models/providers/kimi-for-coding.md +10 -11
  69. package/.docs/models/providers/llmgateway.md +5 -8
  70. package/.docs/models/providers/moonshotai-cn.md +3 -2
  71. package/.docs/models/providers/moonshotai.md +3 -2
  72. package/.docs/models/providers/nebius.md +3 -1
  73. package/.docs/models/providers/novita-ai.md +3 -1
  74. package/.docs/models/providers/ollama-cloud.md +25 -42
  75. package/.docs/models/providers/opencode-go.md +5 -3
  76. package/.docs/models/providers/opencode.md +4 -2
  77. package/.docs/models/providers/orcarouter.md +1 -1
  78. package/.docs/models/providers/privatemode-ai.md +10 -10
  79. package/.docs/models/providers/thinkingmachines.md +73 -0
  80. package/.docs/models/providers/togetherai.md +2 -6
  81. package/.docs/models/providers/wandb.md +2 -1
  82. package/.docs/models/providers/zenmux.md +3 -1
  83. package/.docs/models/providers.md +3 -0
  84. package/.docs/reference/acp/acp-agent.md +2 -0
  85. package/.docs/reference/acp/create-acp-tool.md +28 -0
  86. package/.docs/reference/agent-controller/session.md +15 -0
  87. package/.docs/reference/agents/agent.md +13 -1
  88. package/.docs/reference/agents/durable-agent.md +23 -0
  89. package/.docs/reference/agents/generate.md +19 -1
  90. package/.docs/reference/browser/firecrawl-browser.md +71 -0
  91. package/.docs/reference/cli/create-mastra.md +118 -43
  92. package/.docs/reference/cli/mastra.md +59 -5
  93. package/.docs/reference/code-sdk/mount-agent-controller.md +4 -4
  94. package/.docs/reference/core/getMCPServer.md +1 -3
  95. package/.docs/reference/core/getMCPServerById.md +1 -3
  96. package/.docs/reference/core/getWorkflow.md +1 -3
  97. package/.docs/reference/core/listMCPServers.md +2 -6
  98. package/.docs/reference/datasets/addItem.md +1 -3
  99. package/.docs/reference/datasets/addItems.md +1 -3
  100. package/.docs/reference/datasets/compareExperiments.md +1 -3
  101. package/.docs/reference/datasets/create.md +1 -3
  102. package/.docs/reference/datasets/dataset.md +2 -6
  103. package/.docs/reference/datasets/datasets-manager.md +5 -15
  104. package/.docs/reference/datasets/delete.md +1 -3
  105. package/.docs/reference/datasets/deleteExperiment.md +1 -3
  106. package/.docs/reference/datasets/deleteItem.md +1 -3
  107. package/.docs/reference/datasets/deleteItems.md +1 -3
  108. package/.docs/reference/datasets/get.md +1 -3
  109. package/.docs/reference/datasets/getDetails.md +1 -3
  110. package/.docs/reference/datasets/getExperiment.md +1 -3
  111. package/.docs/reference/datasets/getItem.md +1 -3
  112. package/.docs/reference/datasets/getItemHistory.md +1 -3
  113. package/.docs/reference/datasets/list.md +1 -3
  114. package/.docs/reference/datasets/listExperimentResults.md +1 -3
  115. package/.docs/reference/datasets/listExperiments.md +1 -3
  116. package/.docs/reference/datasets/listItems.md +1 -3
  117. package/.docs/reference/datasets/listVersions.md +1 -3
  118. package/.docs/reference/datasets/startExperiment.md +1 -3
  119. package/.docs/reference/datasets/startExperimentAsync.md +1 -3
  120. package/.docs/reference/datasets/update.md +1 -3
  121. package/.docs/reference/datasets/updateItem.md +1 -3
  122. package/.docs/reference/editor/mastra-editor.md +1 -3
  123. package/.docs/reference/evals/context-recall.md +205 -0
  124. package/.docs/reference/evals/create-scorer.md +4 -10
  125. package/.docs/reference/evals/mastra-scorer.md +1 -3
  126. package/.docs/reference/evals/run-evals.md +94 -2
  127. package/.docs/reference/file-based-agents/tools.md +7 -6
  128. package/.docs/reference/index.md +5 -0
  129. package/.docs/reference/memory/observational-memory.md +21 -0
  130. package/.docs/reference/observability/tracing/exporters/posthog.md +39 -1
  131. package/.docs/reference/observability/tracing/processors/sensitive-data-filter.md +1 -3
  132. package/.docs/reference/processors/token-limiter-processor.md +1 -3
  133. package/.docs/reference/rag/database-config.md +12 -0
  134. package/.docs/reference/streaming/agents/stream.md +3 -1
  135. package/.docs/reference/templates/overview.md +6 -32
  136. package/.docs/reference/tools/create-tool.md +78 -52
  137. package/.docs/reference/tools/mcp-client.md +104 -24
  138. package/.docs/reference/tools/mcp-server.md +143 -44
  139. package/.docs/reference/tools/tavily.md +1 -1
  140. package/.docs/reference/tools/vector-query-tool.md +22 -0
  141. package/.docs/reference/vectors/mongodb.md +2 -0
  142. package/.docs/reference/vectors/turbopuffer.md +4 -0
  143. package/.docs/reference/voice/mistral.md +127 -0
  144. package/.docs/reference/workspace/daytona-sandbox.md +2 -6
  145. package/.docs/reference/workspace/platform-filesystem.md +182 -0
  146. package/.docs/reference/workspace/platform-sandbox.md +200 -0
  147. package/.docs/reference/workspace/railway-sandbox.md +37 -1
  148. package/CHANGELOG.md +85 -0
  149. package/dist/{chunk-GLPCVXXO.js → chunk-HGADBLKG.js} +2 -2
  150. package/dist/{chunk-GLPCVXXO.js.map → chunk-HGADBLKG.js.map} +1 -1
  151. package/dist/index.js +1 -1
  152. package/dist/stdio.js +1 -1
  153. package/dist/tools/docs.d.ts.map +1 -1
  154. package/package.json +7 -7
@@ -61,6 +61,41 @@ Read the [`mastra start`](https://mastra.ai/reference/cli/mastra) reference for
61
61
 
62
62
  ## Build configuration
63
63
 
64
+ ### Build-time configuration
65
+
66
+ Mastra reads the `bundler`, `deployer`, and `server` options while it builds your application. Keep those options as direct properties of the object passed to `new Mastra()` so the build can extract them.
67
+
68
+ The following entry-file shape works:
69
+
70
+ ```typescript
71
+ import { Mastra } from '@mastra/core/mastra'
72
+
73
+ export const mastra = new Mastra({
74
+ bundler: {
75
+ externals: ['sharp'],
76
+ },
77
+ server: {
78
+ port: 4111,
79
+ },
80
+ })
81
+ ```
82
+
83
+ The value of each option can come from a variable, import, or function call. The option itself must remain a direct property. Do not hide build-time options behind a factory call or an object spread:
84
+
85
+ ```typescript
86
+ const options = {
87
+ bundler: {
88
+ externals: ['sharp'],
89
+ },
90
+ }
91
+
92
+ // These patterns prevent Mastra from extracting `bundler` during the build.
93
+ export const mastra = new Mastra(createMastraOptions())
94
+ export const otherMastra = new Mastra({ ...options })
95
+ ```
96
+
97
+ When Mastra can't extract an option, it uses the default build behavior for that option. See the [configuration reference](https://mastra.ai/reference/configuration) for the available `bundler`, `deployer`, and `server` settings.
98
+
64
99
  ### Public folder
65
100
 
66
101
  If a `public` folder exists in your Mastra directory (`src/mastra/public`), its contents are copied to the output directory during build. These files are served as static assets by the server.
@@ -97,6 +97,8 @@ export const mastra = new Mastra({
97
97
  })
98
98
  ```
99
99
 
100
+ Keep `bundler` as a direct property in the object passed to `new Mastra()`. See [build-time configuration](https://mastra.ai/docs/deployment/mastra-server) for the supported entry-file shape.
101
+
100
102
  ## Environment variables
101
103
 
102
104
  Store `.env` files in the Mastra application directory (e.g., `apps/api/.env`), not the monorepo root.
@@ -51,6 +51,12 @@ Use this option for auto-scaling, minimal infrastructure management, or when you
51
51
  - [Netlify](https://mastra.ai/guides/deployment/netlify)
52
52
  - [Vercel](https://mastra.ai/guides/deployment/vercel)
53
53
 
54
+ ### Sandbox
55
+
56
+ Deploy a Mastra server into an ephemeral workspace sandbox (such as Vercel Sandbox or E2B) and get a live public URL in seconds.
57
+
58
+ Use this option for instant previews, deploys from your CI pipeline, verifying agent-built apps, or running untrusted per-user instances. Read the [sandbox deployment guide](https://mastra.ai/docs/deployment/sandbox).
59
+
54
60
  ### Web Framework
55
61
 
56
62
  When Mastra is integrated with a web framework, it deploys alongside your application using the framework's standard deployment process. The guides below cover framework-specific configuration requirements for deployment.
@@ -0,0 +1,277 @@
1
+ > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
2
+
3
+ # Deploy to a sandbox
4
+
5
+ `@mastra/deployer-sandbox` deploys a full Mastra server, including Studio, into an ephemeral workspace sandbox and returns a live public URL. Repeat deployments can finish faster because the deployer skips dependency installation.
6
+
7
+ Use sandbox deployments for:
8
+
9
+ - Agent-built apps: An agent generates a Mastra project and deploys it to verify the result.
10
+ - Continuous integration (CI): Spin up a real server for checks, then tear it down.
11
+ - Instant previews: Share a working agent with your team before merging.
12
+ - Multi-tenant untrusted code: Run per-user Mastra instances isolated from your infrastructure.
13
+
14
+ Sandboxes have provider-enforced runtime caps and expire. For production hosting, see the [deployment overview](https://mastra.ai/docs/deployment/overview).
15
+
16
+ ## Supported sandboxes
17
+
18
+ The deployer works with any workspace sandbox that supports networking (public port URLs):
19
+
20
+ - [Vercel Sandbox](https://mastra.ai/reference/workspace/vercel-sandbox) (`@mastra/vercel`)
21
+ - [E2B](https://mastra.ai/reference/workspace/e2b-sandbox) (`@mastra/e2b`)
22
+ - [Daytona](https://mastra.ai/reference/workspace/daytona-sandbox) (`@mastra/daytona`)
23
+
24
+ Provider authors can add support by implementing the optional `networking` capability on [`WorkspaceSandbox`](https://mastra.ai/reference/workspace/sandbox).
25
+
26
+ ## Quickstart
27
+
28
+ Install the deployer and a sandbox provider of your choice. This example uses Vercel Sandbox:
29
+
30
+ **npm**:
31
+
32
+ ```bash
33
+ npm install @mastra/deployer-sandbox @mastra/vercel
34
+ ```
35
+
36
+ **pnpm**:
37
+
38
+ ```bash
39
+ pnpm add @mastra/deployer-sandbox @mastra/vercel
40
+ ```
41
+
42
+ **Yarn**:
43
+
44
+ ```bash
45
+ yarn add @mastra/deployer-sandbox @mastra/vercel
46
+ ```
47
+
48
+ **Bun**:
49
+
50
+ ```bash
51
+ bun add @mastra/deployer-sandbox @mastra/vercel
52
+ ```
53
+
54
+ Configure the deployer in your `src/mastra/index.ts` file. The `sandboxName` identifies the deployment, so subsequent deployments with the same name reuse the existing sandbox.
55
+
56
+ ```typescript
57
+ import { Mastra } from '@mastra/core/mastra'
58
+ import { SandboxDeployer } from '@mastra/deployer-sandbox'
59
+ import { VercelSandbox } from '@mastra/vercel'
60
+
61
+ export const mastra = new Mastra({
62
+ deployer: new SandboxDeployer({
63
+ sandbox: new VercelSandbox({
64
+ sandboxName: 'my-preview',
65
+ timeout: 3_600_000, // 1 hour
66
+ ports: [4111],
67
+ }),
68
+ }),
69
+ })
70
+ ```
71
+
72
+ Build and deploy in one command:
73
+
74
+ ```bash
75
+ mastra build
76
+ ```
77
+
78
+ When a `SandboxDeployer()` is configured, `mastra build` bundles your project and deploys it into the sandbox. The deploy prints the API and Studio URLs and writes a `sandbox-deployment.json` manifest into `.mastra/output`:
79
+
80
+ ```text
81
+ API: https://<sandbox-id>-4111.vercel.run/api
82
+ Studio: https://<sandbox-id>-4111.vercel.run
83
+ ```
84
+
85
+ The manifest includes `expiresAt` when the sandbox provider reports an expiration time. Redeploys to the same sandbox skip the dependency install when the install inputs (`package.json`, bundled lockfiles, and the install command) are unchanged.
86
+
87
+ ### Using E2B
88
+
89
+ For E2B, the `id` identifies the deployment, so subsequent deployments with the same value reconnect to the existing sandbox, whether it's running or paused.
90
+
91
+ ```typescript
92
+ import { SandboxDeployer } from '@mastra/deployer-sandbox'
93
+ import { E2BSandbox } from '@mastra/e2b'
94
+
95
+ const deployer = new SandboxDeployer({
96
+ sandbox: new E2BSandbox({
97
+ id: 'my-preview',
98
+ template: 'base',
99
+ timeout: 3_600_000, // 1 hour
100
+ }),
101
+ })
102
+ ```
103
+
104
+ Pass `template: 'base'` unless you need filesystem mounts; without it, the provider builds a custom Filesystem in Userspace (FUSE) template on first use. E2B pauses instead of stopping: `stop()` snapshots the whole virtual machine (VM), including memory and running processes. When a paused sandbox wakes, the Mastra server resumes where it left off, and there's no relaunch step like on Vercel.
105
+
106
+ ### Using Daytona
107
+
108
+ For Daytona, the `id` identifies the deployment, so subsequent deployments with the same value reconnect to the existing sandbox. Set `public: true` to make the preview URL accessible without a token.
109
+
110
+ ```typescript
111
+ import { SandboxDeployer } from '@mastra/deployer-sandbox'
112
+ import { DaytonaSandbox } from '@mastra/daytona'
113
+
114
+ const deployer = new SandboxDeployer({
115
+ sandbox: new DaytonaSandbox({
116
+ id: 'my-preview',
117
+ public: true,
118
+ autoStopInterval: 30, // minutes
119
+ }),
120
+ })
121
+ ```
122
+
123
+ Stopping a Daytona sandbox persists its filesystem but not running processes, so waking works like Vercel: the resolver relaunches the server on `wake: true`.
124
+
125
+ ## Deploying programmatically
126
+
127
+ `deployToSandbox()` deploys a prebuilt output directory without the bundler. Unlike `SandboxDeployer()`, it doesn't include Studio unless you pass `studio: true`. Use it in CI or from agent code:
128
+
129
+ ```typescript
130
+ import { deployToSandbox } from '@mastra/deployer-sandbox'
131
+ import { VercelSandbox } from '@mastra/vercel'
132
+
133
+ const deployment = await deployToSandbox({
134
+ sandbox: new VercelSandbox({ sandboxName: 'ci-smoke', ports: [4111] }),
135
+ dir: '.mastra/output',
136
+ })
137
+
138
+ console.info(deployment.url) // https://<sandbox-id>-4111.vercel.run
139
+ await deployment.logs() // tail the server log
140
+ await deployment.stop() // stop the sandbox (resumable)
141
+ await deployment.destroy() // permanently delete the sandbox
142
+ ```
143
+
144
+ ## Lifecycle
145
+
146
+ ### Managing a deployment
147
+
148
+ Use the server-only `getDeployment()` export from `@mastra/deployer-sandbox/client` to retrieve an existing deployment. It identifies the sandbox through provider-specific configuration, such as a Vercel `sandboxName` or an E2B or Daytona `id`. The lookup isn't tied to the process that created the deployment, so you can use it from another server-side service or in CI.
149
+
150
+ ```typescript
151
+ import { getDeployment } from '@mastra/deployer-sandbox/client'
152
+ import { VercelSandbox } from '@mastra/vercel'
153
+
154
+ const deployment = await getDeployment({
155
+ sandbox: new VercelSandbox({ sandboxName: 'my-preview', ports: [4111] }),
156
+ port: 4111,
157
+ })
158
+
159
+ console.info(deployment.status, deployment.url)
160
+ await deployment.logs() // tail the server log
161
+ await deployment.stop() // stop the sandbox (resumable)
162
+ await deployment.destroy() // permanently delete the sandbox
163
+ ```
164
+
165
+ Pass `wake: true` to resume a stopped sandbox before returning. The server is relaunched only if it isn't healthy after the resume. Provider tooling works too, for example `vercel sandbox ls`, `vercel sandbox stop`, and `vercel sandbox rm`.
166
+
167
+ ### Expiry and URLs
168
+
169
+ Sandboxes expire according to the provider's runtime limits. When the provider reports an expiration time, the deploy logs it and `deployment.expiresAt` exposes it programmatically.
170
+
171
+ Sandbox URLs can change when a sandbox stops and resumes. Treat the URL as plumbing and the sandbox identity (for example, the `sandboxName`) as the stable handle. The routing tiers below deal with URL rotation.
172
+
173
+ ## Routing tiers
174
+
175
+ ### Tier 1: Direct URL
176
+
177
+ Use the printed URL directly for development, demos, and CI where a fresh URL per deploy is acceptable.
178
+
179
+ ### Tier 2: Resolve at runtime
180
+
181
+ `getDeployment()` resolves the current URL at runtime, so consumers never hold a stale URL. Any server that knows the sandbox name can resolve it, including one in a different codebase than the Mastra project:
182
+
183
+ ```typescript
184
+ import { getDeployment } from '@mastra/deployer-sandbox/client'
185
+ import { VercelSandbox } from '@mastra/vercel'
186
+
187
+ const deployment = await getDeployment({
188
+ sandbox: new VercelSandbox({ sandboxName: 'my-preview', ports: [4111] }),
189
+ wake: true,
190
+ })
191
+
192
+ console.info(deployment.url, deployment.status)
193
+ ```
194
+
195
+ With `wake: false` (the default) the sandbox isn't started and you get `{ url, status }` to act on. Resolving the URL, `stop()`, and `destroy()` attach to the existing sandbox by name without resuming it, so lifecycle operations on a stopped sandbox never wake it or start billing. With `wake: true` the sandbox resumes and the server is relaunched if it isn't answering. Whether a relaunch is needed depends on the provider: Vercel and Daytona restore the filesystem but not running processes, while E2B resumes the whole VM including the server process.
196
+
197
+ > **Warning:** `@mastra/deployer-sandbox/client` is server-only. Resolving a sandbox uses provider credentials that must never reach the browser. The module throws if imported in a browser context.
198
+
199
+ ### Tier 3: Stable URL for end users
200
+
201
+ Give end users a stable URL on your own domain and forward to the sandbox server-side, using either a route handler proxy or an Edge Config alias.
202
+
203
+ The example below shows a Vercel & Next.js setup, but the concept applies to any server-side framework or provider that can forward requests.
204
+
205
+ - **Route handler proxy.** `createSandboxHandler()` caches the sandbox URL and re-resolves after a connection-level failure, which covers URL rotation and cold wakes:
206
+
207
+ ```typescript
208
+ import { createSandboxHandler, getDeployment } from '@mastra/deployer-sandbox/client'
209
+ import { VercelSandbox } from '@mastra/vercel'
210
+
211
+ const handler = createSandboxHandler({
212
+ resolve: async () => {
213
+ const deployment = await getDeployment({
214
+ sandbox: new VercelSandbox({ sandboxName: 'my-preview', ports: [4111] }),
215
+ wake: true,
216
+ })
217
+ return deployment.url!
218
+ },
219
+ })
220
+
221
+ export { handler as GET, handler as POST }
222
+ ```
223
+
224
+ - **Edge Config alias.** Set the `alias` option on the deployer to keep a [Vercel Edge Config](https://vercel.com/docs/edge-config) item pointed at the current URL on every deploy:
225
+
226
+ ```typescript
227
+ const deployer = new SandboxDeployer({
228
+ sandbox: new VercelSandbox({ sandboxName: 'my-preview', ports: [4111] }),
229
+ alias: { edgeConfigId: 'ecfg_...', key: 'my-preview', token: process.env.VERCEL_TOKEN! },
230
+ })
231
+ ```
232
+
233
+ Then rewrite requests in Next.js middleware with `createSandboxProxy()`:
234
+
235
+ ```typescript
236
+ import { createSandboxProxy } from '@mastra/deployer-sandbox/client'
237
+
238
+ export const middleware = createSandboxProxy({ key: 'my-preview' })
239
+ export const config = { matcher: '/api/:path*' }
240
+ ```
241
+
242
+ ## CI example
243
+
244
+ Deploy a preview on every pull request:
245
+
246
+ ```yaml
247
+ name: Sandbox preview
248
+ on: pull_request
249
+
250
+ jobs:
251
+ preview:
252
+ runs-on: ubuntu-latest
253
+ env:
254
+ VERCEL_TOKEN: ${{ secrets.VERCEL_TOKEN }}
255
+ VERCEL_TEAM_ID: ${{ secrets.VERCEL_TEAM_ID }}
256
+ VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
257
+ steps:
258
+ - uses: actions/checkout@v4
259
+ - uses: actions/setup-node@v4
260
+ with:
261
+ node-version: 22
262
+ - run: npm ci
263
+ - run: npx mastra build
264
+ - run: curl --fail "$(jq -r .url .mastra/output/sandbox-deployment.json)/api"
265
+ ```
266
+
267
+ ## Security
268
+
269
+ - The sandbox URL is public. Anyone with the URL can reach your Mastra server, including Studio. Enable [server auth](https://mastra.ai/docs/server/auth) for anything beyond throwaway previews.
270
+ - Environment variables from your `.env` files are injected into the remote sandbox VM so the server can run. The deploy logs a warning when this happens. Don't deploy secrets you wouldn't put on a shared preview server.
271
+ - To restrict access to Tier 3 traffic, pass a `secret` to `createSandboxHandler()` or `createSandboxProxy()`. The helpers attach it as the `x-mastra-sandbox-secret` header on forwarded requests. Configure [server auth](https://mastra.ai/docs/server/auth) to require that header, and direct hits to the sandbox URL get rejected while traffic through your domain works.
272
+
273
+ ## Related
274
+
275
+ - [Deployment overview](https://mastra.ai/docs/deployment/overview)
276
+ - [Server authentication](https://mastra.ai/docs/server/auth)
277
+ - [`WorkspaceSandbox` reference](https://mastra.ai/reference/workspace/sandbox)
@@ -57,9 +57,7 @@ import { Mastra } from '@mastra/core'
57
57
  import { MastraEditor } from '@mastra/editor'
58
58
 
59
59
  export const mastra = new Mastra({
60
- agents: {
61
- /* your existing agents */
62
- },
60
+ agents: {/* your existing agents */},
63
61
  editor: new MastraEditor(),
64
62
  })
65
63
  ```
@@ -84,9 +82,7 @@ import { Mastra } from '@mastra/core'
84
82
  import { MastraEditor } from '@mastra/editor'
85
83
 
86
84
  export const mastra = new Mastra({
87
- agents: {
88
- /* your existing agents */
89
- },
85
+ agents: {/* your existing agents */},
90
86
  editor: new MastraEditor({
91
87
  source: 'code',
92
88
  }),
@@ -62,9 +62,7 @@ Integration providers connect external tool platforms to the editor. Once regist
62
62
  import { ComposioToolProvider } from '@mastra/editor/composio'
63
63
 
64
64
  export const mastra = new Mastra({
65
- agents: {
66
- /* your agents */
67
- },
65
+ agents: {/* your agents */},
68
66
  editor: new MastraEditor({
69
67
  toolProviders: {
70
68
  composio: new ComposioToolProvider({
@@ -91,9 +89,7 @@ Integration providers connect external tool platforms to the editor. Once regist
91
89
  import { ArcadeToolProvider } from '@mastra/editor/arcade'
92
90
 
93
91
  export const mastra = new Mastra({
94
- agents: {
95
- /* your agents */
96
- },
92
+ agents: {/* your agents */},
97
93
  editor: new MastraEditor({
98
94
  toolProviders: {
99
95
  arcade: new ArcadeToolProvider({
@@ -0,0 +1,178 @@
1
+ > Discover all available pages from the documentation index: https://mastra.ai/llms.txt
2
+
3
+ # Multi-turn evals
4
+
5
+ Multi-turn evals test agent behavior across a conversation. Instead of a single `input`, you provide an `inputs` array. Each entry is sent sequentially to the agent on the same thread, and scorers see the accumulated output from all turns.
6
+
7
+ ## When to use multi-turn evals
8
+
9
+ - The agent uses memory and must recall context from earlier turns
10
+ - The agent handles follow-up questions that depend on prior responses
11
+ - You need to verify tool-call sequences across a conversation
12
+ - The agent runs a multi-step workflow (search, confirm, execute)
13
+
14
+ ## Quickstart
15
+
16
+ ```typescript
17
+ import { runEvals } from '@mastra/core/evals'
18
+ import { checks } from '@mastra/evals/checks'
19
+ import { weatherAgent } from '../agents'
20
+
21
+ const result = await runEvals({
22
+ data: [
23
+ {
24
+ inputs: [
25
+ 'What is the weather in Brooklyn?',
26
+ 'What about tomorrow?',
27
+ 'Compare the two forecasts.',
28
+ ],
29
+ },
30
+ ],
31
+ target: weatherAgent,
32
+ scorers: [checks.calledTool('get_weather', { times: 2 }), checks.includes('Brooklyn')],
33
+ })
34
+ ```
35
+
36
+ Each turn runs `agent.generate()` with the same thread ID, so the agent sees the full conversation history. Scorers receive the accumulated output messages from all turns.
37
+
38
+ ## Memory is required for cross-turn recall
39
+
40
+ Multi-turn recall depends on the agent having a **memory store configured**. The shared thread ID is what lets each turn see the earlier ones — but a thread only persists history when the agent has memory. If the agent has no memory configured, the turns still run sequentially and their outputs still accumulate for scoring, but the agent will not recall earlier turns (each input runs in isolation). `runEvals` logs a warning when you use `inputs` on an agent without memory.
41
+
42
+ `runEvals` manages the conversation identity for you: it generates the shared `threadId` and injects a `resourceId` (Mastra memory scopes messages by resource + thread, so both are required for recall to work). By default the resource is derived from the generated thread so each conversation is isolated. To pin a specific resource — for example to reuse an existing user's memory — pass `targetOptions.memory.resource`; `runEvals` still owns the thread, so you don't provide one:
43
+
44
+ ```typescript
45
+ await runEvals({
46
+ target: weatherAgent,
47
+ data: [{ inputs: ['What is the weather in Brooklyn?', 'What about tomorrow?'] }],
48
+ scorers: [checks.similarity('weather forecast')],
49
+ targetOptions: { memory: { resource: 'user-42' } },
50
+ })
51
+ ```
52
+
53
+ See [Memory](https://mastra.ai/docs/memory/overview) for how to configure a memory store.
54
+
55
+ ## How it works
56
+
57
+ When a data item has an `inputs` array, `runEvals`:
58
+
59
+ 1. Creates a fresh thread (unique `threadId`) and resource for the conversation (a caller-provided `targetOptions.memory.resource` is preserved)
60
+ 2. Sends each input sequentially via `agent.generate()` on that thread
61
+ 3. Accumulates all output messages across turns
62
+ 4. Passes the full accumulated output to scorers for evaluation
63
+
64
+ Scorers see the complete conversation output, not just the last turn.
65
+
66
+ ## Scoring semantics
67
+
68
+ Two details matter when writing scorers for multi-turn items:
69
+
70
+ - **`run.output` is the accumulated output from every turn.** Output-based scorers — `checks.includes`, `checks.calledTool`, `checks.similarity`, and similar — evaluate the whole conversation. For example, `checks.calledTool('get_weather', { times: 2 })` counts calls across all turns.
71
+ - **`run.input` is only the first turn's input.** Scorers that compare input against output (faithfulness, answer relevancy, and other input-relative LLM scorers) only see the first user message, not the full conversation. Prefer output-based checks for multi-turn, or build scorers that read the accumulated `run.output` directly.
72
+
73
+ ## Per-turn assertions with `turns`
74
+
75
+ The `inputs` form scores the **accumulated** output holistically — a single score over every turn's output. That can hide per-turn failures: an output-based check like `checks.includes('Brooklyn')` passes if _any_ turn mentions Brooklyn, even when the follow-up turn is broken.
76
+
77
+ When you need to assert that a **specific turn** behaved correctly, use `turns` instead. Each turn is an object with its own `input` and optional `gates`/`scorers` that evaluate **only that turn's** input and output:
78
+
79
+ ```typescript
80
+ import { runEvals } from '@mastra/core/evals'
81
+ import { checks } from '@mastra/evals/checks'
82
+ import { weatherAgent } from '../agents'
83
+
84
+ const result = await runEvals({
85
+ data: [
86
+ {
87
+ turns: [
88
+ {
89
+ input: 'What is the weather in Brooklyn?',
90
+ gates: [checks.calledTool('get_weather')],
91
+ },
92
+ {
93
+ // The follow-up must call the tool again — it can't be satisfied
94
+ // by the first turn's tool call.
95
+ input: 'What about tomorrow?',
96
+ gates: [checks.calledTool('get_weather')],
97
+ scorers: [{ scorer: checks.similarity('tomorrow forecast'), threshold: 0.5 }],
98
+ },
99
+ ],
100
+ },
101
+ ],
102
+ target: weatherAgent,
103
+ })
104
+
105
+ result.verdict // 'passed' | 'scored' | 'failed'
106
+ result.turnResults // per-turn gate/threshold/scorer outcomes
107
+ ```
108
+
109
+ Semantics:
110
+
111
+ - A per-turn gate or scorer sees **only that turn's** `run.input` and `run.output` — never the accumulated conversation. This fixes both blind spots of `inputs`: the wrong turn can't satisfy a check, and `run.input` is correct for each turn.
112
+ - Per-turn outcomes fold into the overall [verdict](https://mastra.ai/docs/evals/gates-and-verdicts): a failing turn gate makes the verdict `failed`; a missed turn threshold (with gates passing) makes it `scored`.
113
+ - `result.turnResults[i]` reports each turn's `gateResults`, `thresholdResults`, and `scores`, so a failure points at the exact turn. Across multiple conversations, turn results are averaged by turn index.
114
+ - A turn with no `gates`/`scorers` just advances the conversation.
115
+ - Top-level `scorers`/`gates` still run holistically over the accumulated output, so you can combine "this turn must call the tool" with "the overall answer mentions Brooklyn."
116
+ - When the agent has storage configured, each per-turn scorer/gate result is persisted like top-level scores, so per-turn outcomes appear in your scores store. Each stored per-turn score is labeled with its turn index (`metadata.turnIndex`), shares the conversation's `threadId`, and links to that turn's own trace span.
117
+
118
+ Use `inputs` when a single holistic score over the whole conversation is enough; use `turns` when correctness depends on individual turns. `turns` cannot be combined with `input` or `inputs` in the same data item.
119
+
120
+ ## Combining with gates and thresholds
121
+
122
+ Multi-turn data items work with [gates and verdicts](https://mastra.ai/docs/evals/gates-and-verdicts). Use output-based scorers so gating reflects the full conversation:
123
+
124
+ ```typescript
125
+ import { runEvals } from '@mastra/core/evals'
126
+ import { checks } from '@mastra/evals/checks'
127
+
128
+ const result = await runEvals({
129
+ data: [
130
+ {
131
+ inputs: ['My favorite city is Brooklyn.', 'What is the weather in my favorite city?'],
132
+ },
133
+ ],
134
+ target: weatherAgent,
135
+ gates: [checks.calledTool('get_weather')],
136
+ scorers: [{ scorer: checks.similarity('Brooklyn weather forecast'), threshold: 0.5 }],
137
+ })
138
+
139
+ result.verdict // 'passed' | 'scored' | 'failed'
140
+ ```
141
+
142
+ ## Mixing single-turn and multi-turn
143
+
144
+ A single `runEvals` call can include both single-turn and multi-turn data items:
145
+
146
+ ```typescript
147
+ const result = await runEvals({
148
+ data: [
149
+ { input: 'What is the weather in Brooklyn?' },
150
+ {
151
+ inputs: ['My favorite city is Brooklyn.', 'What is the weather in my favorite city?'],
152
+ },
153
+ ],
154
+ target: weatherAgent,
155
+ scorers: [checks.includes('Brooklyn')],
156
+ })
157
+ ```
158
+
159
+ Single-turn items use `input` as usual. Multi-turn items use `inputs` — `input` can be omitted entirely.
160
+
161
+ ## Validation
162
+
163
+ `runEvals` throws a `MastraError` if `inputs` is present but empty:
164
+
165
+ ```typescript
166
+ // Throws: 'inputs' must be a non-empty array
167
+ await runEvals({
168
+ data: [{ inputs: [] }],
169
+ target: myAgent,
170
+ scorers: [myScorer],
171
+ })
172
+ ```
173
+
174
+ ## Related
175
+
176
+ - [`runEvals()` reference](https://mastra.ai/reference/evals/run-evals): Full API for `runEvals` parameters and returns
177
+ - [Gates and verdicts](https://mastra.ai/docs/evals/gates-and-verdicts): Enforce hard requirements and quality thresholds
178
+ - [Quick Checks](https://mastra.ai/docs/evals/quick-checks): Zero-LLM composable micro-scorers
@@ -142,6 +142,7 @@ Once registered, you can score traces interactively within Studio under the **Ob
142
142
 
143
143
  - Use [Quick Checks](https://mastra.ai/docs/evals/quick-checks) for fast, deterministic assertions on text and tool usage
144
144
  - Add [gates and verdicts](https://mastra.ai/docs/evals/gates-and-verdicts) to enforce hard requirements and quality thresholds
145
+ - Test [multi-turn conversations](https://mastra.ai/docs/evals/multi-turn) where behavior emerges across sequential turns
145
146
  - Learn how to create your own scorers in the [Creating Custom Scorers](https://mastra.ai/docs/evals/custom-scorers) guide
146
147
  - Explore built-in scorers in the [Built-in Scorers](https://mastra.ai/docs/evals/built-in-scorers) section
147
148
  - Test scorers with [Studio](https://mastra.ai/docs/studio/overview)
@@ -71,24 +71,18 @@ Create separate test cases for different evaluation scenarios:
71
71
 
72
72
  ```typescript
73
73
  describe('Weather Agent Tests', () => {
74
- const locationScorer = createScorer({
75
- /* ... */
76
- })
74
+ const locationScorer = createScorer({/* ... */})
77
75
 
78
76
  it('should handle location disambiguation', async () => {
79
77
  const result = await runEvals({
80
78
  data: [
81
79
  {
82
80
  input: 'weather in Berlin',
83
- groundTruth: {
84
- /* ... */
85
- },
81
+ groundTruth: {/* ... */},
86
82
  },
87
83
  {
88
84
  input: 'weather in Berlin, Maryland',
89
- groundTruth: {
90
- /* ... */
91
- },
85
+ groundTruth: {/* ... */},
92
86
  },
93
87
  ],
94
88
  target: weatherAgent,
@@ -142,13 +142,9 @@ If you're unable to use a local MCP server and need to connect to a remote serve
142
142
 
143
143
  ### Installation
144
144
 
145
- #### create-mastra
145
+ The [`create-mastra`](https://mastra.ai/reference/cli/create-mastra) command installs Mastra skills automatically. It doesn't configure the MCP docs server. Add the server manually when you need MCP-specific tools such as the migration tool.
146
146
 
147
- During the interactive [create-mastra](https://mastra.ai/reference/cli/create-mastra) wizard, choose one of your tools in the MCP step.
148
-
149
- #### Manual setup
150
-
151
- If there are no specific instructions for your tool below, you may be able to add the MCP server with this common JSON configuration anyways.
147
+ If there are no specific instructions for your tool below, you may be able to add the MCP server with this common JSON configuration.
152
148
 
153
149
  ```json
154
150
  {
@@ -4,7 +4,7 @@
4
4
 
5
5
  **For AI agents:** Use this guide when tasked to create a runnable Mastra project from scratch without a CLI/boilerplate. It provides code examples for agents, tools, model configuration and how to install dependencies. Do not navigate to the quickstart guide, which is for humans. The model string is Mastra's model router format ('provider/model'; use / and not : to separate provider and model). Do not install any ai-sdk packages.
6
6
 
7
- Use this guide to manually build a standalone Mastra server step by step. In most cases, it's quicker to follow the [quickstart guide](https://mastra.ai/guides/getting-started/quickstart), which achieves the same result using the [`mastra create`](https://mastra.ai/reference/cli/create-mastra) command. For existing projects, you can also use [`mastra init`](https://mastra.ai/reference/cli/mastra).
7
+ Use this guide to manually build a standalone Mastra server step by step. In most cases, it's quicker to follow the [quickstart guide](https://mastra.ai/guides/getting-started/quickstart), which achieves the same result using the [`create-mastra`](https://mastra.ai/reference/cli/create-mastra) command. For existing projects, you can also use [`mastra init`](https://mastra.ai/reference/cli/mastra).
8
8
 
9
9
  ## Instructions
10
10