@mastra/mcp-docs-server 1.2.27-alpha.1 → 1.2.27-alpha.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (86) hide show
  1. package/.docs/docs/agents/structured-output.md +17 -0
  2. package/.docs/docs/connections/a2a.md +4 -3
  3. package/.docs/docs/deployment/monorepo.md +2 -2
  4. package/.docs/docs/evals/datasets.md +53 -0
  5. package/.docs/docs/guides/build-an-eval-loop.md +395 -0
  6. package/.docs/docs/mastra-platform/alerts.md +83 -0
  7. package/.docs/docs/mastra-platform/observability.md +184 -0
  8. package/.docs/docs/mastra-platform/overview.md +2 -0
  9. package/.docs/docs/memory/message-history.md +21 -0
  10. package/.docs/docs/memory/observational-memory.md +33 -0
  11. package/.docs/docs/observability/feedback.md +1 -1
  12. package/.docs/docs/observability/tracing/overview.md +2 -0
  13. package/.docs/docs/server/custom-adapters.md +43 -0
  14. package/.docs/docs/subagents.md +38 -7
  15. package/.docs/integrations/channels/github.md +6 -2
  16. package/.docs/integrations/databases/clickhouse.md +1 -1
  17. package/.docs/integrations/observability/confident-ai.md +67 -43
  18. package/.docs/integrations/observability/langfuse.md +4 -0
  19. package/.docs/integrations/sandboxes/cloudflare-sandbox.md +36 -4
  20. package/.docs/models/environment-variables.md +5 -1
  21. package/.docs/models/gateways/netlify.md +8 -4
  22. package/.docs/models/gateways/openrouter.md +5 -2
  23. package/.docs/models/gateways/vercel.md +378 -379
  24. package/.docs/models/index.md +22 -1
  25. package/.docs/models/providers/ai21.md +78 -0
  26. package/.docs/models/providers/ainetcafe.md +77 -0
  27. package/.docs/models/providers/alibaba-cn.md +8 -6
  28. package/.docs/models/providers/alibaba-token-plan-cn.md +2 -1
  29. package/.docs/models/providers/alibaba-token-plan.md +3 -1
  30. package/.docs/models/providers/alibaba.md +2 -1
  31. package/.docs/models/providers/chutes.md +2 -2
  32. package/.docs/models/providers/cortecs.md +6 -7
  33. package/.docs/models/providers/digitalocean.md +1 -1
  34. package/.docs/models/providers/edenai.md +4 -7
  35. package/.docs/models/providers/empiriolabs.md +2 -1
  36. package/.docs/models/providers/fireworks-ai.md +11 -10
  37. package/.docs/models/providers/hyper.md +26 -37
  38. package/.docs/models/providers/inception.md +3 -3
  39. package/.docs/models/providers/inco.md +83 -0
  40. package/.docs/models/providers/iteracompute.md +14 -7
  41. package/.docs/models/providers/kilo.md +12 -9
  42. package/.docs/models/providers/llmgateway-providers.md +4 -2
  43. package/.docs/models/providers/llmgateway.md +1 -1
  44. package/.docs/models/providers/mistral.md +3 -2
  45. package/.docs/models/providers/nano-gpt.md +10 -18
  46. package/.docs/models/providers/nvidia.md +2 -1
  47. package/.docs/models/providers/oci.md +85 -0
  48. package/.docs/models/providers/ofox.md +24 -23
  49. package/.docs/models/providers/opencode.md +2 -1
  50. package/.docs/models/providers/ovhcloud.md +1 -1
  51. package/.docs/models/providers/privatemode-ai.md +3 -3
  52. package/.docs/models/providers/scnet-token-plan.md +2 -1
  53. package/.docs/models/providers/synthetic.md +2 -1
  54. package/.docs/models/providers/tensorx.md +2 -1
  55. package/.docs/models/providers/tinfoil.md +1 -1
  56. package/.docs/models/providers/umans-ai-coding-plan.md +3 -4
  57. package/.docs/models/providers/umans-ai.md +3 -4
  58. package/.docs/models/providers/vancine.md +10 -10
  59. package/.docs/models/providers/volcengine.md +3 -2
  60. package/.docs/models/providers/wandb.md +4 -4
  61. package/.docs/models/providers/xai.md +1 -3
  62. package/.docs/models/providers/zhipuai-coding-plan.md +2 -8
  63. package/.docs/models/providers.md +5 -1
  64. package/.docs/reference/agents/generate.md +1 -1
  65. package/.docs/reference/auth/clerk.md +25 -1
  66. package/.docs/reference/cli/mastra.md +84 -0
  67. package/.docs/reference/client-js/agents.md +25 -0
  68. package/.docs/reference/client-js/mastra-client.md +1 -1
  69. package/.docs/reference/client-js/observability.md +101 -4
  70. package/.docs/reference/code-sdk/mount-agent-controller.md +23 -0
  71. package/.docs/reference/core/getMCPServer.md +47 -0
  72. package/.docs/reference/index.md +2 -0
  73. package/.docs/reference/memory/memory-class.md +2 -0
  74. package/.docs/reference/memory/observational-memory.md +34 -4
  75. package/.docs/reference/observability/tracing/interfaces.md +3 -1
  76. package/.docs/reference/observability/tracing/trace-query.md +219 -46
  77. package/.docs/reference/pubsub/redis-streams.md +34 -0
  78. package/.docs/reference/rag/vector-databases.md +73 -0
  79. package/.docs/reference/storage/retention.md +56 -4
  80. package/.docs/reference/streaming/agents/stream.md +1 -1
  81. package/.docs/reference/tools/mcp-server.md +0 -28
  82. package/.docs/reference/vectors/azure-ai-search.md +150 -0
  83. package/.docs/reference/vectors/weaviate.md +128 -0
  84. package/.docs/reference/workspace/workspace-class.md +10 -2
  85. package/package.json +9 -11
  86. package/.docs/docs/connections/connect-mcp-client.md +0 -211
@@ -168,6 +168,190 @@ The CLI can infer platform credentials from your project environment. See the [`
168
168
 
169
169
  You can query exported feedback over HTTP. See the [observability feedback query API](https://mastra.ai/docs/mastra-platform/api) for its current status, regional endpoints, authentication, and project scoping.
170
170
 
171
+ ## Import existing traces
172
+
173
+ Use `mastra traces import` to move existing trace history from a supported observability provider into an existing Mastra Platform project. The command reads and validates complete traces at the source, converts them into Mastra spans, then uses a shared workflow for preparation, upload, resume, and verification.
174
+
175
+ The import command currently supports Langfuse. Additional providers can use the same command workflow when their adapters are added.
176
+
177
+ | Provider | Command argument | Source requirement |
178
+ | -------- | ---------------- | -------------------------------------------------- |
179
+ | Langfuse | `langfuse` | Langfuse Cloud or self-hosted Langfuse v4 or later |
180
+
181
+ ### Prerequisites
182
+
183
+ Before starting an import, you need:
184
+
185
+ - A Mastra Platform project.
186
+ - Mastra Platform authentication through `mastra auth login`, or a `MASTRA_API_TOKEN` and `MASTRA_ORG_ID` for a headless environment.
187
+ - `MASTRA_PLATFORM_ACCESS_TOKEN` for upload and read-back when using an interactive login. A dry run doesn't require this token.
188
+ - Credentials for a supported source provider.
189
+ - Enough local disk space to temporarily store the prepared traces.
190
+
191
+ ### Configure the destination
192
+
193
+ For an interactive import, sign in and select the organization that owns the destination project:
194
+
195
+ ```bash
196
+ npx mastra auth login
197
+ npx mastra auth orgs switch
198
+ ```
199
+
200
+ Interactive login authorizes project discovery. Set the Platform access token used by the destination project before uploading or verifying traces:
201
+
202
+ ```bash
203
+ MASTRA_PLATFORM_ACCESS_TOKEN=<mastra-platform-access-token>
204
+ ```
205
+
206
+ If `mastra init` configured observability for the project, use the token it wrote to `.env`. Otherwise, create an access token in [Mastra Platform](https://projects.mastra.ai).
207
+
208
+ You can run `--dry-run` without this token because a dry run doesn't contact the collector or query API.
209
+
210
+ Specify the destination with `--project`, or set `MASTRA_PROJECT_ID`. You can use a project name, slug, or ID with `--project`:
211
+
212
+ ```bash
213
+ npx mastra traces import langfuse --project my-project --dry-run
214
+ ```
215
+
216
+ If `MASTRA_PROJECT_ID` is set, it takes precedence over `--project`. Without either value, the CLI uses the project linked in `.mastra-project.json`.
217
+
218
+ For a headless environment, set the API token and organization ID. The command uses `MASTRA_API_TOKEN` for project discovery, upload, and read-back. Set the project ID or pass `--project`:
219
+
220
+ ```bash
221
+ MASTRA_API_TOKEN=<mastra-api-token>
222
+ MASTRA_ORG_ID=<organization-id>
223
+ MASTRA_PROJECT_ID=<project-id>
224
+ ```
225
+
226
+ ### Configure Langfuse
227
+
228
+ Create [project API keys in Langfuse](https://langfuse.com/docs/api-and-data-platform/features/public-api), then add them to `.env` or `.env.local` in the directory where you run the command:
229
+
230
+ ```bash
231
+ LANGFUSE_PUBLIC_KEY=<langfuse-public-key>
232
+ LANGFUSE_SECRET_KEY=<langfuse-secret-key>
233
+ ```
234
+
235
+ Langfuse Cloud defaults to the EU region at `https://cloud.langfuse.com`. Set `LANGFUSE_BASE_URL` to the [regional host](https://langfuse.com/security/data-regions) for US, Japan, or HIPAA Cloud, or to the origin of a self-hosted instance:
236
+
237
+ ```bash
238
+ LANGFUSE_BASE_URL=https://us.cloud.langfuse.com
239
+ ```
240
+
241
+ ### Preview the import
242
+
243
+ Run a dry run before uploading:
244
+
245
+ ```bash
246
+ npx mastra traces import langfuse --project my-project --dry-run
247
+ ```
248
+
249
+ The dry run reads the source, prepares valid Mastra traces locally, and reports:
250
+
251
+ - The source and destination projects.
252
+ - The fixed import window.
253
+ - Prepared and skipped trace and span counts.
254
+ - The size of the prepared local data.
255
+ - Provider warnings and skip reasons in `report.json`.
256
+
257
+ It doesn't upload traces. The command prints a `--resume` command that uploads the prepared data later without reading the source again.
258
+
259
+ ### Choose the import window
260
+
261
+ By default, the command selects the last 30 days, ending when the import starts. Use `--from` and `--to` to choose a smaller window:
262
+
263
+ ```bash
264
+ npx mastra traces import langfuse \
265
+ --project my-project \
266
+ --from "$FROM" \
267
+ --to "$TO" \
268
+ --dry-run
269
+ ```
270
+
271
+ Set `FROM` and `TO` to ISO 8601 dates or timestamps. The following rules apply:
272
+
273
+ - `--to` can't be in the future or outside the current 30-day Platform retention period.
274
+ - `--from` must be earlier than `--to` and inside the current retention period.
275
+ - The selected window can't exceed 30 days.
276
+ - If only `--to` is set, the default start is the later of 30 days before `--to` and the current retention boundary.
277
+
278
+ Trace eligibility is based on the root observation's start time. The Langfuse adapter discovers roots in the selected window and then reads all currently available observations for each selected trace. It skips the complete trace if the available parent-child tree or timestamps are invalid.
279
+
280
+ ### Upload and verify
281
+
282
+ Run the command without `--dry-run` to upload the prepared traces:
283
+
284
+ ```bash
285
+ npx mastra traces import langfuse --project my-project
286
+ ```
287
+
288
+ The CLI displays the preparation summary and asks for confirmation before upload. Pass `--yes` to skip this prompt in an automated environment:
289
+
290
+ ```bash
291
+ npx mastra traces import langfuse --project my-project --yes
292
+ ```
293
+
294
+ The importer keeps every trace in one upload request and checkpoints progress only after Mastra Platform acknowledges the batch. Temporary source and destination failures use bounded retries.
295
+
296
+ After every prepared trace is acknowledged, the importer reads back a deterministic sample of up to 10 traces. It verifies span IDs, parent links, names, span types, event flags, timestamps, and whether an error is present. It doesn't read back or compare input, output, attributes, metadata, or tags.
297
+
298
+ If read-back isn't available yet or a sampled trace differs, the import pauses and keeps its prepared data. Resume the import to retry verification without uploading acknowledged traces again.
299
+
300
+ ### Resume an import
301
+
302
+ Use the exact command printed by the CLI when a dry run, cancellation, interruption, upload failure, or paused verification leaves an import unfinished:
303
+
304
+ ```bash
305
+ npx mastra traces import langfuse \
306
+ --resume 00000000-0000-0000-0000-000000000000 \
307
+ --project my-project
308
+ ```
309
+
310
+ A resumed import must use the original provider and destination project. Its saved date window can't be changed, so `--from`, `--to`, and `--dry-run` can't be combined with `--resume`.
311
+
312
+ Resume behavior depends on where the command stopped:
313
+
314
+ - An interrupted preparation reads and prepares the source again.
315
+ - An interrupted upload starts with the first unacknowledged trace.
316
+ - Paused verification retries read-back without re-uploading acknowledged traces.
317
+ - A completed import reports that it's already complete and retries any remaining local cleanup.
318
+
319
+ ### Local files and cleanup
320
+
321
+ Import state is stored under:
322
+
323
+ ```text
324
+ ~/.mastra/imports/traces/<target-project-id>/<import-id>/
325
+ ```
326
+
327
+ | File | Purpose | Lifecycle |
328
+ | --------------- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
329
+ | `manifest.json` | Stores the source identity, date window, counts, and acknowledged progress. | Retained after completion. |
330
+ | `traces.jsonl` | Stores one complete normalized trace per line. | Retained for dry runs and unfinished imports, then removed after successful verification. |
331
+ | `report.json` | Stores counts, warnings, skip samples, and verification results. | Retained after it's written. |
332
+
333
+ These files can contain trace content. Mastra creates the import directory and files with permissions restricted to the local user. Protect the machine and account that run the import.
334
+
335
+ ### Imported data and limits
336
+
337
+ For Langfuse imports:
338
+
339
+ - Observations become Mastra spans with their parent-child relationships and timestamps.
340
+ - Supported fields such as names, input, output, errors, model details, usage, cost, tags, and source context are mapped when present.
341
+ - Destination trace and span IDs are deterministic. Original Langfuse IDs remain in span metadata for correlation.
342
+ - Unknown Langfuse observation types become generic spans and produce a warning instead of being silently discarded.
343
+ - Only trace observations are imported. Scores, feedback, datasets, attachments, and logs aren't imported.
344
+
345
+ The shared importer also applies these limits:
346
+
347
+ - Prepared local trace data is limited to 5 GiB per import. Use a smaller date window if an import reaches this limit.
348
+ - A complete trace must fit within the importer's 4 MiB upload payload limit. A larger trace is skipped and recorded with the `trace_too_large` reason.
349
+ - A trace is never split between upload requests.
350
+
351
+ Review `report.json` after a dry run or completed import to see exactly what was prepared, skipped, acknowledged, and verified.
352
+
353
+ See the [`mastra traces import` CLI reference](https://mastra.ai/reference/cli/mastra) for the complete option and environment-variable reference.
354
+
171
355
  ## Next steps
172
356
 
173
357
  - 📹 [Mastra observability and Studio workshop](https://www.youtube.com/watch?v=dKO_a3RPra0)
@@ -14,6 +14,8 @@ Deploy with a single command, [`mastra deploy`](https://mastra.ai/docs/mastra-pl
14
14
 
15
15
  Each project can run multiple [**Environments**](https://mastra.ai/docs/mastra-platform/environments) (for example `production` and `staging`) and provision [**Hosted databases**](https://mastra.ai/docs/mastra-platform/database) from the CLI or project settings to persist application data. Each environment also gets a managed [**Workspace**](https://mastra.ai/docs/mastra-platform/workspaces) that gives agents a filesystem and sandbox with no manual configuration.
16
16
 
17
+ Organization-level [**Alerts**](https://mastra.ai/docs/mastra-platform/alerts) notify your team when deploys fail or running services stop. Alerts can cover every project or selected projects and environments, with notifications sent to Slack, email, or webhooks.
18
+
17
19
  [**Trace Intelligence**](https://mastra.ai/docs/mastra-platform/trace-intelligence) finds recurring goals, outcomes, behaviors, and sentiment across your agent traces. Trace Intelligence is available in private beta for selected projects.
18
20
 
19
21
  ## Get started
@@ -124,6 +124,27 @@ You can use this history in two ways:
124
124
 
125
125
  > **Note:** `lastMessages` counts every stored message, including tool calls, tool results, and [signals](https://mastra.ai/docs/harness/signals) of any kind, so a single turn can add several messages to the count. The window also slides forward on every request: once a thread grows past the limit, the oldest message leaves context on each turn, which changes the start of the prompt and invalidates the provider prompt cache. For long-running conversations, use [Observational Memory](https://mastra.ai/docs/memory/observational-memory), which keeps the prompt prefix stable.
126
126
 
127
+ ### Limit history by tokens
128
+
129
+ Message count is a poor proxy for context size: a tool result can be a few tokens or thousands. Use `messageHistory` to keep recent history within a token budget instead:
130
+
131
+ ```typescript
132
+ export const agent = new Agent({
133
+ id: 'test-agent',
134
+ memory: new Memory({
135
+ options: {
136
+ messageHistory: { maxTokens: 8_000, atMaxRemoveTokens: 2_000 },
137
+ },
138
+ }),
139
+ })
140
+ ```
141
+
142
+ Mastra counts the complete prompt against `maxTokens`, including remembered history, system instructions, context, and the current turn. When the prompt exceeds the budget, Mastra removes the oldest remembered messages until the prompt is at most `maxTokens - atMaxRemoveTokens`. `atMaxRemoveTokens` defaults to 25% of `maxTokens`. Removing history in chunks keeps the prompt prefix stable across several turns, which helps provider prompt caches stay warm.
143
+
144
+ System messages, context, the current turn's input, and the agent's responses are never trimmed. Linked tool calls and results are removed together. If protected content alone exceeds `maxTokens`, Mastra removes all remembered history but keeps the protected content.
145
+
146
+ Trimmed messages stay in storage. During agent runs, Mastra persists a per-thread boundary so they're excluded from later turns. Setting `messageHistory` without `lastMessages` disables the default 10-message cap. Set both to combine a count cap with a token budget. Set `maxTokens` to `0` to disable message history.
147
+
127
148
  > **Tip:** When memory is enabled, [Studio](https://mastra.ai/docs/studio/overview) uses message history to display past conversations in the chat sidebar.
128
149
 
129
150
  ## Thread title generation
@@ -849,6 +849,39 @@ Transform hooks are always awaited, on every path (manual `observe()`/`reflect()
849
849
 
850
850
  Because hooks receive `threadId` and `resourceId`, you can also use them to update [working memory](https://mastra.ai/docs/memory/working-memory) via `memory.updateWorkingMemory()` during a cycle. These external updates aren't atomic with the OM text commit.
851
851
 
852
+ ### Redact skill results
853
+
854
+ Agent skills are injected into the agent as tools (`skill`, `skill_search`, `skill_read`). Their results contain the skill's full instructions or file contents, so without redaction the Observer re-observes that text every time a skill is used. `skillResultRedactor()` is a ready-made `beforeObservation` hook that replaces those results with a placeholder and leaves everything else in place. The tool call survives, so the Observer still records which skill was used and what it was called with.
855
+
856
+ ```typescript
857
+ import { Memory } from '@mastra/memory'
858
+ import { skillResultRedactor } from '@mastra/memory/hooks'
859
+
860
+ const memory = new Memory({
861
+ options: {
862
+ observationalMemory: {
863
+ model: 'google/gemini-2.5-flash',
864
+ hooks: {
865
+ beforeObservation: skillResultRedactor(),
866
+ },
867
+ },
868
+ },
869
+ })
870
+ ```
871
+
872
+ Pass `toolNames` to redact results from a different set of tools. Because a hook is a function over the messages, it composes with your own transforms by chaining the outputs. Await each chained hook so an async one isn't discarded:
873
+
874
+ ```typescript
875
+ const dropSkillResults = skillResultRedactor()
876
+
877
+ hooks: {
878
+ beforeObservation: async input => {
879
+ const messages = (await dropSkillResults(input))?.messages ?? input.messages
880
+ return { messages: messages.filter(m => m.role !== 'signal') }
881
+ },
882
+ }
883
+ ```
884
+
852
885
  ## Migrating existing threads
853
886
 
854
887
  No manual migration needed. OM reads existing messages and observes them lazily when thresholds are exceeded.
@@ -130,7 +130,7 @@ await observability!.deleteFeedback({
130
130
  })
131
131
  ```
132
132
 
133
- Deleted records also disappear from feedback analytics. ClickHouse uses a lightweight delete to hide rows without guaranteeing immediate physical removal, so open-source deployments must configure an [observability retention period](https://mastra.ai/reference/storage/retention) to physically purge them. ClickHouse doesn't configure a retention TTL for deletion requests in open-source deployments. Delete APIs intentionally leave cursor-only delta rows untouched. These rows contain identifiers rather than feedback payloads and expire within two days.
133
+ Deleted records also disappear from feedback analytics. ClickHouse uses a lightweight delete to hide rows without guaranteeing immediate physical removal, so open-source deployments must configure an [observability retention period](https://mastra.ai/reference/storage/retention) to physically purge them. Configure retention for every observability signal to expire deletion requests after the signal rows they protect. If any signal is unbounded, deletion requests also remain unbounded to prevent deleted data from being reintroduced. Delete APIs intentionally leave cursor-only delta rows untouched. These rows contain identifiers rather than feedback payloads and expire within two days.
134
134
 
135
135
  ## Query feedback analytics
136
136
 
@@ -459,6 +459,8 @@ export const mastra = new Mastra({
459
459
  })
460
460
  ```
461
461
 
462
+ `process()` must mutate the span it receives and return the same instance, or return `undefined` to drop the span. Don't return a copy (for example `{ ...span, input: '...' }`): `exportSpan()` and `isValid` are instance members of the live span, so a copy can't be exported. Mastra logs a processor error and drops the span in that case.
463
+
462
464
  Processors are executed in the order they're defined, allowing you to chain multiple transformations. Common use cases include:
463
465
 
464
466
  - Redacting sensitive data (passwords, tokens, API keys)
@@ -371,6 +371,49 @@ await server.init()
371
371
  app.listen(4111)
372
372
  ```
373
373
 
374
+ ## Test adapter compatibility
375
+
376
+ Use `@mastra/server-adapters-test-suite` to run the same conformance tests as Mastra's official adapters. Install it with its peer dependencies in your adapter project:
377
+
378
+ **npm**:
379
+
380
+ ```bash
381
+ npm install --save-dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
382
+ ```
383
+
384
+ **pnpm**:
385
+
386
+ ```bash
387
+ pnpm add --save-dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
388
+ ```
389
+
390
+ **Yarn**:
391
+
392
+ ```bash
393
+ yarn add --dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
394
+ ```
395
+
396
+ **Bun**:
397
+
398
+ ```bash
399
+ bun add --dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
400
+ ```
401
+
402
+ Provide framework-specific setup and request execution functions to the route suite:
403
+
404
+ ```typescript
405
+ import { createRouteAdapterTestSuite } from '@mastra/server-adapters-test-suite'
406
+ import { executeHttpRequest, setupAdapter } from './adapter-test-helpers'
407
+
408
+ createRouteAdapterTestSuite({
409
+ suiteName: 'My framework adapter',
410
+ setupAdapter,
411
+ executeHttpRequest,
412
+ })
413
+ ```
414
+
415
+ The package also provides root exports for Model Context Protocol (MCP), multipart request, HTTP logging, and body limit tests. See the [package README](https://github.com/mastra-ai/mastra/tree/main/server-adapters/server-adapters-test-suite) for the supported peer dependency versions.
416
+
374
417
  > **Tip:** The existing [@mastra/hono](https://github.com/mastra-ai/mastra/blob/main/server-adapters/hono/src/index.ts) and [@mastra/express](https://github.com/mastra-ai/mastra/blob/main/server-adapters/express/src/index.ts) implementations are good references when building your custom adapter. They show how to handle framework-specific patterns for context storage and middleware registration, plus response handling.
375
418
  >
376
419
  > If you want to use [Studio](https://mastra.ai/docs/studio/overview) with your server adapter, use [`mastra studio`](https://mastra.ai/reference/cli/mastra) to only launch the Studio UI.
@@ -135,11 +135,13 @@ The subagent reads these entries in its tools and dynamic configuration, such as
135
135
  Called after a delegation finishes. Use it to inspect results or provide feedback, or alternatively stop execution:
136
136
 
137
137
  - `context.bail()`: Stop the parent agent's loop immediately
138
- - Return `{ feedback: '...' }`: Add feedback that gets saved to the parent agent's memory and is visible to subsequent iterations
138
+ - Return `{ feedback: '...' }`: Add feedback that gets saved to the parent agent's memory and is available on the next turn
139
139
  - Return `{ resultText: '...' }`: Replace the tool result text the parent model sees for this delegation, within the current run
140
140
 
141
141
  Set `resultText` when the subagent's own result would give the parent a misleading signal. For example, empty text from a subagent stopped on a tool-calls step looks like a successful but empty delegation to the parent model. You can also replace the error text from a failed delegation with a more useful message. This doesn't recover the delegation. The parent still receives a failed tool result. Unlike `feedback` on the next turn, `resultText` affects the parent's reasoning immediately.
142
142
 
143
+ To stop the parent agent on failure and save feedback for a later turn, call `bail()` and return `feedback`:
144
+
143
145
  ```typescript
144
146
  const stream = await parentAgent.stream('Research AI trends', {
145
147
  maxSteps: 10,
@@ -161,12 +163,41 @@ const stream = await parentAgent.stream('Research AI trends', {
161
163
 
162
164
  The `context` object includes:
163
165
 
164
- | Property | Description |
165
- | ------------- | --------------------------------------------------------------------------------------------- |
166
- | `primitiveId` | The ID of the subagent that ran |
167
- | `result` | The subagent's response, including `text`, `usage`, `finishReason`, and `subAgentToolResults` |
168
- | `error` | Error if the delegation failed |
169
- | `bail()` | Function to stop the parent agent's loop |
166
+ | Property | Description |
167
+ | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
168
+ | `primitiveId` | The ID of the subagent that ran |
169
+ | `result` | The subagent's response, including `text`, `usage`, `finishReason`, `subAgentToolResults`, `subAgentThreadId`, and `subAgentResourceId` when available |
170
+ | `success` | Whether the delegation succeeded |
171
+ | `error` | The original error if the delegation failed |
172
+ | `messages` | The collected subagent transcript, including partial messages when available |
173
+ | `bail()` | Function to stop the parent agent's loop |
174
+
175
+ #### Failed delegations
176
+
177
+ When a subagent run fails, the parent model receives a failed tool result with a generic error message. This doesn't automatically stop the parent agent, which can explain the failure or choose another approach. Call `context.bail()` to stop its loop instead.
178
+
179
+ The hook receives `success: false`, the original `error`, and any collected partial results and messages. Returning `resultText` replaces the error text the parent model sees without changing the delegation's failed status or discarding its original cause. An empty string is also a valid replacement.
180
+
181
+ To let the parent continue with a curated failure message instead of stopping its loop, return `resultText` without calling `bail()`:
182
+
183
+ ```typescript
184
+ const stream = await parentAgent.stream('Research AI trends', {
185
+ maxSteps: 10,
186
+ delegation: {
187
+ onDelegationComplete: ({ success }) => {
188
+ if (!success) {
189
+ return {
190
+ resultText: 'The delegated task failed. Try another approach.',
191
+ }
192
+ }
193
+ },
194
+ },
195
+ })
196
+ ```
197
+
198
+ Prefer a curated message, as in this example. Returning `{ resultText: error.message }` explicitly forwards the underlying message to the parent model's provider, and the model may repeat it to the end user. Provider errors can contain sensitive information. A generic `resultText` isn't a redaction policy for logs, traces, or streamed error payloads.
199
+
200
+ For delegations run as [background tasks](https://mastra.ai/docs/harness/background-tasks), configured retries can invoke `onDelegationComplete` once per attempt. Account for repeated calls if your hook has side effects.
170
201
 
171
202
  ### Hook errors
172
203
 
@@ -41,9 +41,13 @@ GitHub Signals requires:
41
41
  - A Mastra storage adapter with memory and notification support. The provider stores each subscription in the thread's metadata, so the thread must already exist.
42
42
  - The `gitcrawl` command on `PATH`, configured to access the repositories you want to monitor. The provider runs `gitcrawl sync` and reads its SQLite database.
43
43
  - The `sqlite3` command on `PATH`.
44
- - The [GitHub CLI](https://cli.github.com/) installed and authenticated. The provider uses `gh api` to check whether comment authors have access to the repository before notifying the agent.
44
+ - The [GitHub CLI](https://cli.github.com/) installed and authenticated. The provider uses real `gh api` responses to check repository permissions and GitHub App ownership before notifying the agent.
45
45
 
46
- By default, comments from users with `admin`, `maintain`, or `write` access can trigger notifications. CodeRabbit and Devin bot comments are also allowed. Configure `authorizedPermissions`, `authorizedBots`, or `ignoredBots` when you need different rules.
46
+ By default, comments from users with `admin`, `maintain`, or `write` access can trigger notifications. CodeRabbit and Devin bot comments are explicitly allowed.
47
+
48
+ Bot authorization checks `ignoredBots` first, using an exact case-insensitive login match. Other bots can trigger notifications when they appear in `authorizedBots`, when the GitHub App is owned by a user whose repository permission appears in `authorizedPermissions`, or when the app is owned by the organization that owns the repository. The default authorized permissions are `admin`, `maintain`, and `write`.
49
+
50
+ Failed, inaccessible, malformed, and unsupported app-owner lookups deny the bot comment. Successful app-owner lookups are cached for 24 hours; failed lookups aren't cached.
47
51
 
48
52
  ## Agent and subscription
49
53
 
@@ -91,7 +91,7 @@ Trace deletion cascades to spans, trace roots and branches, metrics, logs, score
91
91
 
92
92
  Lightweight deletion is a hide-only operation that marks rows with ClickHouse's `_row_exists` mask. Physical removal depends on merges and deployment-configured retention TTLs. `ObservabilityStorageClickhouseVNext` applies retention only when you provide a `RetentionConfig`; Mastra OSS doesn't configure a default retention TTL.
93
93
 
94
- Deletion requests aren't purged automatically in Mastra OSS. Automatic retirement will be introduced with future database-agnostic retention configuration.
94
+ When all five observability signals have finite retention, Mastra also applies a TTL to deletion requests so they outlive the signal rows they protect. If any signal is unbounded, deletion requests remain unbounded. See [storage retention](https://mastra.ai/reference/storage/retention) for how the deletion-request TTL is calculated.
95
95
 
96
96
  ### Observability with the legacy domain
97
97
 
@@ -4,49 +4,49 @@
4
4
 
5
5
  # Confident AI
6
6
 
7
- [Confident AI](https://www.confident-ai.com/) is an LLM observability and evaluation platform for teams to build reliable AI applications in both development and production.
7
+ [Confident AI](https://www.confident-ai.com/) is an LLM observability and evaluation platform for teams building reliable AI applications in both development and production.
8
8
 
9
- The `@mastra/deepeval` package sends your Mastra traces to Confident AI, where you can run metrics against them and track quality over time. It builds on [DeepEval](https://www.confident-ai.com/docs), the open-source evaluation SDK behind the platform.
9
+ The `ConfidentMastraExporter` from [`confident-trace`](https://github.com/confident-ai/confident-trace) sends your Mastra traces to Confident AI, where agent runs show up with the full agent, model, and tool hierarchy, and can be scored automatically as they arrive.
10
10
 
11
11
  ## Installation
12
12
 
13
13
  **npm**:
14
14
 
15
15
  ```bash
16
- npm install @mastra/deepeval@latest
16
+ npm install confident-trace@latest
17
17
  ```
18
18
 
19
19
  **pnpm**:
20
20
 
21
21
  ```bash
22
- pnpm add @mastra/deepeval@latest
22
+ pnpm add confident-trace@latest
23
23
  ```
24
24
 
25
25
  **Yarn**:
26
26
 
27
27
  ```bash
28
- yarn add @mastra/deepeval@latest
28
+ yarn add confident-trace@latest
29
29
  ```
30
30
 
31
31
  **Bun**:
32
32
 
33
33
  ```bash
34
- bun add @mastra/deepeval@latest
34
+ bun add confident-trace@latest
35
35
  ```
36
36
 
37
37
  ## Configuration
38
38
 
39
39
  ### Prerequisites
40
40
 
41
- 1. **Confident AI account**: Sign up at [confident-ai.com](https://www.confident-ai.com/)
41
+ 1. **Confident AI account**: Sign up at [confident-ai.com](https://app.confident-ai.com)
42
42
  2. **API key**: Generate one in your Confident AI project settings
43
43
  3. **Environment variables**: Set your credentials:
44
44
 
45
45
  ```bash
46
46
  CONFIDENT_API_KEY=confident_proj_xxxxxxxxxxxxx
47
47
 
48
- # Optional
49
- CONFIDENT_TRACE_ENVIRONMENT=production # Defaults to "development"
48
+ # Optional. EU region, defaults to the US servers
49
+ CONFIDENT_OTEL_ENDPOINT=https://eu.otel.confident-ai.com/v1/traces
50
50
  ```
51
51
 
52
52
  ### Zero-Config Setup
@@ -56,14 +56,14 @@ With environment variables set, use the exporter with no configuration:
56
56
  ```typescript
57
57
  import { Mastra } from '@mastra/core'
58
58
  import { Observability } from '@mastra/observability'
59
- import { DeepEvalExporter } from '@mastra/deepeval'
59
+ import { ConfidentMastraExporter } from 'confident-trace/mastra'
60
60
 
61
61
  export const mastra = new Mastra({
62
62
  observability: new Observability({
63
63
  configs: {
64
- deepeval: {
64
+ confidentAI: {
65
65
  serviceName: 'my-service',
66
- exporters: [new DeepEvalExporter()],
66
+ exporters: [new ConfidentMastraExporter()],
67
67
  },
68
68
  },
69
69
  }),
@@ -77,17 +77,17 @@ You can also pass credentials directly (takes precedence over environment variab
77
77
  ```typescript
78
78
  import { Mastra } from '@mastra/core'
79
79
  import { Observability } from '@mastra/observability'
80
- import { DeepEvalExporter } from '@mastra/deepeval'
80
+ import { ConfidentMastraExporter } from 'confident-trace/mastra'
81
81
 
82
82
  export const mastra = new Mastra({
83
83
  observability: new Observability({
84
84
  configs: {
85
- deepeval: {
85
+ confidentAI: {
86
86
  serviceName: 'my-service',
87
87
  exporters: [
88
- new DeepEvalExporter({
88
+ new ConfidentMastraExporter({
89
89
  apiKey: process.env.CONFIDENT_API_KEY,
90
- environment: 'production',
90
+ endpoint: process.env.CONFIDENT_OTEL_ENDPOINT,
91
91
  }),
92
92
  ],
93
93
  },
@@ -96,47 +96,71 @@ export const mastra = new Mastra({
96
96
  })
97
97
  ```
98
98
 
99
- ### Metric collections
99
+ These options are available to you:
100
100
 
101
- Confident AI evaluates incoming traces against metric collections defined in your project. Attach them at the trace level or per span type. Trace-level metrics run against the whole trace, and per-type metrics run against matching spans.
101
+ ```typescript
102
+ new ConfidentMastraExporter({
103
+ apiKey: process.env.CONFIDENT_API_KEY, // Default: CONFIDENT_API_KEY
104
+ endpoint: process.env.CONFIDENT_OTEL_ENDPOINT, // Default: US servers
105
+ captureContent: false, // Omit inputs, outputs, and messages
106
+ maxContentBytes: 512, // Truncate captured content
107
+ timeoutMillis: 30000, // Flush and shutdown budget
108
+ resourceAttributes: { 'service.name': 'my-agent' }, // Overrides serviceName
109
+ })
110
+ ```
111
+
112
+ Traces carry Mastra's configured `serviceName` unless `resourceAttributes['service.name']` overrides it. Mastra's own sampling, filters, and span processors run before export, so whatever your pipeline lets through is what Confident AI receives.
113
+
114
+ ## Supported signals
115
+
116
+ The exporter converts ended Mastra spans into OTLP spans, preserving trace IDs, span IDs, parent IDs, external parent IDs, and original timestamps. It needs no start-event cache, so resumed workflow spans and out-of-order completion export correctly.
117
+
118
+ - **Agent and workflow execution**: Operation names, timing, status, and the parent-child relationships between steps.
119
+ - **Model calls**: Model details, messages, finish reasons, and [token usage](https://www.confident-ai.com/docs/llm-tracing/features/token-usage-cost).
120
+ - **Tool calls**: Tool names and their [input/output](https://www.confident-ai.com/docs/llm-tracing/features/input-output).
121
+ - **Trace details**: The root operation's name, tags, metadata, and input/output.
122
+ - **Errors**: Failed operations retain their error status to make failures visible in the trace.
123
+
124
+ This integration exports tracing data. Mastra logs, metrics, scores, and feedback aren't included.
125
+
126
+ ## Custom trace metadata
127
+
128
+ Metadata and tags passed through Mastra's `tracingOptions` are forwarded to the trace in Confident AI, where you can filter and group by them:
102
129
 
103
130
  ```typescript
104
- new DeepEvalExporter({
105
- metricCollection: 'trace-metrics', // trace-level
106
- llmMetricCollection: 'llm-metrics', // applied to LLM spans
107
- agentMetricCollection: 'agent-metrics', // applied to agent spans
108
- toolMetricCollectionMap: {
109
- search: 'search-tool-metrics', // applied to the "search" tool
131
+ const result = await mastra.getAgent('assistant').generate('How do you make the best coffee?', {
132
+ tracingOptions: {
133
+ metadata: { release: '2026-09', tier: 'enterprise' },
134
+ tags: ['production', 'support'],
110
135
  },
111
136
  })
112
137
  ```
113
138
 
114
- ### Complete Configuration
139
+ Your metadata is merged with the fields Mastra sets itself, such as `runId`. Tags apply to the root span, so they land on the trace rather than on individual spans.
140
+
141
+ ## Lifecycle
142
+
143
+ Mastra owns this exporter. It creates its own OTLP processor and never registers, replaces, or shuts down the global OpenTelemetry provider, so there is no `init()` call to make and no preload to add to your start command.
144
+
145
+ Spans are batched and sent when the exporter is shut down. Shut it down once, after every agent run and stream has finished, or a short-lived script exits before anything is sent:
115
146
 
116
147
  ```typescript
117
- new DeepEvalExporter({
118
- apiKey: process.env.CONFIDENT_API_KEY,
119
- environment: 'production', // Default: "development"
120
- name: 'my-trace', // Default: the Mastra serviceName
121
- tags: ['production'],
122
- metadata: { team: 'growth' },
123
- })
148
+ await observability.shutdown()
124
149
  ```
125
150
 
126
- ## Span type mapping
151
+ In a long-running server, shut down during graceful shutdown rather than per request. Traces can take up to 30 seconds to appear in the Observatory after being sent.
152
+
153
+ ## Threads and trace properties
154
+
155
+ This exporter maps Mastra's own tracing surface, so trace metadata and tags come from `tracingOptions` as shown above.
127
156
 
128
- Mastra spans map to the span types shown in Confident AI:
157
+ Grouping turns into [threads](https://www.confident-ai.com/docs/llm-tracing/features/threads) and setting trace-level properties such as user IDs use `confident-trace`'s SDK instrumentation, which attaches to Mastra directly rather than through an exporter. See the [Mastra integration guide](https://www.confident-ai.com/docs/integrations/third-party/mastra) for that setup.
129
158
 
130
- | Mastra span type | Confident AI span type |
131
- | ---------------------------------------------------------------------- | ---------------------- |
132
- | `AGENT_RUN`, `WORKFLOW_RUN` | `AGENT` |
133
- | `MODEL_GENERATION` | `LLM` |
134
- | `TOOL_CALL`, `MCP_TOOL_CALL`, `PROVIDER_TOOL_CALL`, `CLIENT_TOOL_CALL` | `TOOL` |
135
- | `RAG_EMBEDDING`, `RAG_VECTOR_OPERATION` | `RETRIEVER` |
136
- | All other exported span types | `CUSTOM` |
159
+ Choose one or the other. Do not use both at once.
137
160
 
138
161
  ## Related
139
162
 
140
163
  - [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
141
- - [DeepEvalExporter reference](https://mastra.ai/reference/observability/tracing/exporters/confident-ai)
142
- - [Confident AI documentation](https://www.confident-ai.com/docs)
164
+ - [Mastra integration guide](https://www.confident-ai.com/docs/integrations/third-party/mastra)
165
+ - [Online evals](https://www.confident-ai.com/docs/llm-tracing/online-evals)
166
+ - [confident-trace on GitHub](https://github.com/confident-ai/confident-trace)