@mastra/mcp-docs-server 1.2.27-alpha.1 → 1.2.27-alpha.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/agents/structured-output.md +17 -0
- package/.docs/docs/connections/a2a.md +4 -3
- package/.docs/docs/deployment/monorepo.md +2 -2
- package/.docs/docs/evals/datasets.md +53 -0
- package/.docs/docs/guides/build-an-eval-loop.md +395 -0
- package/.docs/docs/mastra-platform/alerts.md +83 -0
- package/.docs/docs/mastra-platform/observability.md +184 -0
- package/.docs/docs/mastra-platform/overview.md +2 -0
- package/.docs/docs/memory/message-history.md +21 -0
- package/.docs/docs/memory/observational-memory.md +33 -0
- package/.docs/docs/observability/feedback.md +1 -1
- package/.docs/docs/observability/tracing/overview.md +2 -0
- package/.docs/docs/server/custom-adapters.md +43 -0
- package/.docs/docs/subagents.md +38 -7
- package/.docs/integrations/channels/github.md +6 -2
- package/.docs/integrations/databases/clickhouse.md +1 -1
- package/.docs/integrations/observability/confident-ai.md +67 -43
- package/.docs/integrations/observability/langfuse.md +4 -0
- package/.docs/integrations/sandboxes/cloudflare-sandbox.md +36 -4
- package/.docs/models/environment-variables.md +5 -1
- package/.docs/models/gateways/netlify.md +8 -4
- package/.docs/models/gateways/openrouter.md +5 -2
- package/.docs/models/gateways/vercel.md +378 -379
- package/.docs/models/index.md +22 -1
- package/.docs/models/providers/ai21.md +78 -0
- package/.docs/models/providers/ainetcafe.md +77 -0
- package/.docs/models/providers/alibaba-cn.md +8 -6
- package/.docs/models/providers/alibaba-token-plan-cn.md +2 -1
- package/.docs/models/providers/alibaba-token-plan.md +3 -1
- package/.docs/models/providers/alibaba.md +2 -1
- package/.docs/models/providers/chutes.md +2 -2
- package/.docs/models/providers/cortecs.md +6 -7
- package/.docs/models/providers/digitalocean.md +1 -1
- package/.docs/models/providers/edenai.md +4 -7
- package/.docs/models/providers/empiriolabs.md +2 -1
- package/.docs/models/providers/fireworks-ai.md +11 -10
- package/.docs/models/providers/hyper.md +26 -37
- package/.docs/models/providers/inception.md +3 -3
- package/.docs/models/providers/inco.md +83 -0
- package/.docs/models/providers/iteracompute.md +14 -7
- package/.docs/models/providers/kilo.md +12 -9
- package/.docs/models/providers/llmgateway-providers.md +4 -2
- package/.docs/models/providers/llmgateway.md +1 -1
- package/.docs/models/providers/mistral.md +3 -2
- package/.docs/models/providers/nano-gpt.md +10 -18
- package/.docs/models/providers/nvidia.md +2 -1
- package/.docs/models/providers/oci.md +85 -0
- package/.docs/models/providers/ofox.md +24 -23
- package/.docs/models/providers/opencode.md +2 -1
- package/.docs/models/providers/ovhcloud.md +1 -1
- package/.docs/models/providers/privatemode-ai.md +3 -3
- package/.docs/models/providers/scnet-token-plan.md +2 -1
- package/.docs/models/providers/synthetic.md +2 -1
- package/.docs/models/providers/tensorx.md +2 -1
- package/.docs/models/providers/tinfoil.md +1 -1
- package/.docs/models/providers/umans-ai-coding-plan.md +3 -4
- package/.docs/models/providers/umans-ai.md +3 -4
- package/.docs/models/providers/vancine.md +10 -10
- package/.docs/models/providers/volcengine.md +3 -2
- package/.docs/models/providers/wandb.md +4 -4
- package/.docs/models/providers/xai.md +1 -3
- package/.docs/models/providers/zhipuai-coding-plan.md +2 -8
- package/.docs/models/providers.md +5 -1
- package/.docs/reference/agents/generate.md +1 -1
- package/.docs/reference/auth/clerk.md +25 -1
- package/.docs/reference/cli/mastra.md +84 -0
- package/.docs/reference/client-js/agents.md +25 -0
- package/.docs/reference/client-js/mastra-client.md +1 -1
- package/.docs/reference/client-js/observability.md +101 -4
- package/.docs/reference/code-sdk/mount-agent-controller.md +23 -0
- package/.docs/reference/core/getMCPServer.md +47 -0
- package/.docs/reference/index.md +2 -0
- package/.docs/reference/memory/memory-class.md +2 -0
- package/.docs/reference/memory/observational-memory.md +34 -4
- package/.docs/reference/observability/tracing/interfaces.md +3 -1
- package/.docs/reference/observability/tracing/trace-query.md +219 -46
- package/.docs/reference/pubsub/redis-streams.md +34 -0
- package/.docs/reference/rag/vector-databases.md +73 -0
- package/.docs/reference/storage/retention.md +56 -4
- package/.docs/reference/streaming/agents/stream.md +1 -1
- package/.docs/reference/tools/mcp-server.md +0 -28
- package/.docs/reference/vectors/azure-ai-search.md +150 -0
- package/.docs/reference/vectors/weaviate.md +128 -0
- package/.docs/reference/workspace/workspace-class.md +10 -2
- package/package.json +9 -11
- package/.docs/docs/connections/connect-mcp-client.md +0 -211
|
@@ -168,6 +168,190 @@ The CLI can infer platform credentials from your project environment. See the [`
|
|
|
168
168
|
|
|
169
169
|
You can query exported feedback over HTTP. See the [observability feedback query API](https://mastra.ai/docs/mastra-platform/api) for its current status, regional endpoints, authentication, and project scoping.
|
|
170
170
|
|
|
171
|
+
## Import existing traces
|
|
172
|
+
|
|
173
|
+
Use `mastra traces import` to move existing trace history from a supported observability provider into an existing Mastra Platform project. The command reads and validates complete traces at the source, converts them into Mastra spans, then uses a shared workflow for preparation, upload, resume, and verification.
|
|
174
|
+
|
|
175
|
+
The import command currently supports Langfuse. Additional providers can use the same command workflow when their adapters are added.
|
|
176
|
+
|
|
177
|
+
| Provider | Command argument | Source requirement |
|
|
178
|
+
| -------- | ---------------- | -------------------------------------------------- |
|
|
179
|
+
| Langfuse | `langfuse` | Langfuse Cloud or self-hosted Langfuse v4 or later |
|
|
180
|
+
|
|
181
|
+
### Prerequisites
|
|
182
|
+
|
|
183
|
+
Before starting an import, you need:
|
|
184
|
+
|
|
185
|
+
- A Mastra Platform project.
|
|
186
|
+
- Mastra Platform authentication through `mastra auth login`, or a `MASTRA_API_TOKEN` and `MASTRA_ORG_ID` for a headless environment.
|
|
187
|
+
- `MASTRA_PLATFORM_ACCESS_TOKEN` for upload and read-back when using an interactive login. A dry run doesn't require this token.
|
|
188
|
+
- Credentials for a supported source provider.
|
|
189
|
+
- Enough local disk space to temporarily store the prepared traces.
|
|
190
|
+
|
|
191
|
+
### Configure the destination
|
|
192
|
+
|
|
193
|
+
For an interactive import, sign in and select the organization that owns the destination project:
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
npx mastra auth login
|
|
197
|
+
npx mastra auth orgs switch
|
|
198
|
+
```
|
|
199
|
+
|
|
200
|
+
Interactive login authorizes project discovery. Set the Platform access token used by the destination project before uploading or verifying traces:
|
|
201
|
+
|
|
202
|
+
```bash
|
|
203
|
+
MASTRA_PLATFORM_ACCESS_TOKEN=<mastra-platform-access-token>
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
If `mastra init` configured observability for the project, use the token it wrote to `.env`. Otherwise, create an access token in [Mastra Platform](https://projects.mastra.ai).
|
|
207
|
+
|
|
208
|
+
You can run `--dry-run` without this token because a dry run doesn't contact the collector or query API.
|
|
209
|
+
|
|
210
|
+
Specify the destination with `--project`, or set `MASTRA_PROJECT_ID`. You can use a project name, slug, or ID with `--project`:
|
|
211
|
+
|
|
212
|
+
```bash
|
|
213
|
+
npx mastra traces import langfuse --project my-project --dry-run
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
If `MASTRA_PROJECT_ID` is set, it takes precedence over `--project`. Without either value, the CLI uses the project linked in `.mastra-project.json`.
|
|
217
|
+
|
|
218
|
+
For a headless environment, set the API token and organization ID. The command uses `MASTRA_API_TOKEN` for project discovery, upload, and read-back. Set the project ID or pass `--project`:
|
|
219
|
+
|
|
220
|
+
```bash
|
|
221
|
+
MASTRA_API_TOKEN=<mastra-api-token>
|
|
222
|
+
MASTRA_ORG_ID=<organization-id>
|
|
223
|
+
MASTRA_PROJECT_ID=<project-id>
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
### Configure Langfuse
|
|
227
|
+
|
|
228
|
+
Create [project API keys in Langfuse](https://langfuse.com/docs/api-and-data-platform/features/public-api), then add them to `.env` or `.env.local` in the directory where you run the command:
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
LANGFUSE_PUBLIC_KEY=<langfuse-public-key>
|
|
232
|
+
LANGFUSE_SECRET_KEY=<langfuse-secret-key>
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Langfuse Cloud defaults to the EU region at `https://cloud.langfuse.com`. Set `LANGFUSE_BASE_URL` to the [regional host](https://langfuse.com/security/data-regions) for US, Japan, or HIPAA Cloud, or to the origin of a self-hosted instance:
|
|
236
|
+
|
|
237
|
+
```bash
|
|
238
|
+
LANGFUSE_BASE_URL=https://us.cloud.langfuse.com
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
### Preview the import
|
|
242
|
+
|
|
243
|
+
Run a dry run before uploading:
|
|
244
|
+
|
|
245
|
+
```bash
|
|
246
|
+
npx mastra traces import langfuse --project my-project --dry-run
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
The dry run reads the source, prepares valid Mastra traces locally, and reports:
|
|
250
|
+
|
|
251
|
+
- The source and destination projects.
|
|
252
|
+
- The fixed import window.
|
|
253
|
+
- Prepared and skipped trace and span counts.
|
|
254
|
+
- The size of the prepared local data.
|
|
255
|
+
- Provider warnings and skip reasons in `report.json`.
|
|
256
|
+
|
|
257
|
+
It doesn't upload traces. The command prints a `--resume` command that uploads the prepared data later without reading the source again.
|
|
258
|
+
|
|
259
|
+
### Choose the import window
|
|
260
|
+
|
|
261
|
+
By default, the command selects the last 30 days, ending when the import starts. Use `--from` and `--to` to choose a smaller window:
|
|
262
|
+
|
|
263
|
+
```bash
|
|
264
|
+
npx mastra traces import langfuse \
|
|
265
|
+
--project my-project \
|
|
266
|
+
--from "$FROM" \
|
|
267
|
+
--to "$TO" \
|
|
268
|
+
--dry-run
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
Set `FROM` and `TO` to ISO 8601 dates or timestamps. The following rules apply:
|
|
272
|
+
|
|
273
|
+
- `--to` can't be in the future or outside the current 30-day Platform retention period.
|
|
274
|
+
- `--from` must be earlier than `--to` and inside the current retention period.
|
|
275
|
+
- The selected window can't exceed 30 days.
|
|
276
|
+
- If only `--to` is set, the default start is the later of 30 days before `--to` and the current retention boundary.
|
|
277
|
+
|
|
278
|
+
Trace eligibility is based on the root observation's start time. The Langfuse adapter discovers roots in the selected window and then reads all currently available observations for each selected trace. It skips the complete trace if the available parent-child tree or timestamps are invalid.
|
|
279
|
+
|
|
280
|
+
### Upload and verify
|
|
281
|
+
|
|
282
|
+
Run the command without `--dry-run` to upload the prepared traces:
|
|
283
|
+
|
|
284
|
+
```bash
|
|
285
|
+
npx mastra traces import langfuse --project my-project
|
|
286
|
+
```
|
|
287
|
+
|
|
288
|
+
The CLI displays the preparation summary and asks for confirmation before upload. Pass `--yes` to skip this prompt in an automated environment:
|
|
289
|
+
|
|
290
|
+
```bash
|
|
291
|
+
npx mastra traces import langfuse --project my-project --yes
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
The importer keeps every trace in one upload request and checkpoints progress only after Mastra Platform acknowledges the batch. Temporary source and destination failures use bounded retries.
|
|
295
|
+
|
|
296
|
+
After every prepared trace is acknowledged, the importer reads back a deterministic sample of up to 10 traces. It verifies span IDs, parent links, names, span types, event flags, timestamps, and whether an error is present. It doesn't read back or compare input, output, attributes, metadata, or tags.
|
|
297
|
+
|
|
298
|
+
If read-back isn't available yet or a sampled trace differs, the import pauses and keeps its prepared data. Resume the import to retry verification without uploading acknowledged traces again.
|
|
299
|
+
|
|
300
|
+
### Resume an import
|
|
301
|
+
|
|
302
|
+
Use the exact command printed by the CLI when a dry run, cancellation, interruption, upload failure, or paused verification leaves an import unfinished:
|
|
303
|
+
|
|
304
|
+
```bash
|
|
305
|
+
npx mastra traces import langfuse \
|
|
306
|
+
--resume 00000000-0000-0000-0000-000000000000 \
|
|
307
|
+
--project my-project
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
A resumed import must use the original provider and destination project. Its saved date window can't be changed, so `--from`, `--to`, and `--dry-run` can't be combined with `--resume`.
|
|
311
|
+
|
|
312
|
+
Resume behavior depends on where the command stopped:
|
|
313
|
+
|
|
314
|
+
- An interrupted preparation reads and prepares the source again.
|
|
315
|
+
- An interrupted upload starts with the first unacknowledged trace.
|
|
316
|
+
- Paused verification retries read-back without re-uploading acknowledged traces.
|
|
317
|
+
- A completed import reports that it's already complete and retries any remaining local cleanup.
|
|
318
|
+
|
|
319
|
+
### Local files and cleanup
|
|
320
|
+
|
|
321
|
+
Import state is stored under:
|
|
322
|
+
|
|
323
|
+
```text
|
|
324
|
+
~/.mastra/imports/traces/<target-project-id>/<import-id>/
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
| File | Purpose | Lifecycle |
|
|
328
|
+
| --------------- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
329
|
+
| `manifest.json` | Stores the source identity, date window, counts, and acknowledged progress. | Retained after completion. |
|
|
330
|
+
| `traces.jsonl` | Stores one complete normalized trace per line. | Retained for dry runs and unfinished imports, then removed after successful verification. |
|
|
331
|
+
| `report.json` | Stores counts, warnings, skip samples, and verification results. | Retained after it's written. |
|
|
332
|
+
|
|
333
|
+
These files can contain trace content. Mastra creates the import directory and files with permissions restricted to the local user. Protect the machine and account that run the import.
|
|
334
|
+
|
|
335
|
+
### Imported data and limits
|
|
336
|
+
|
|
337
|
+
For Langfuse imports:
|
|
338
|
+
|
|
339
|
+
- Observations become Mastra spans with their parent-child relationships and timestamps.
|
|
340
|
+
- Supported fields such as names, input, output, errors, model details, usage, cost, tags, and source context are mapped when present.
|
|
341
|
+
- Destination trace and span IDs are deterministic. Original Langfuse IDs remain in span metadata for correlation.
|
|
342
|
+
- Unknown Langfuse observation types become generic spans and produce a warning instead of being silently discarded.
|
|
343
|
+
- Only trace observations are imported. Scores, feedback, datasets, attachments, and logs aren't imported.
|
|
344
|
+
|
|
345
|
+
The shared importer also applies these limits:
|
|
346
|
+
|
|
347
|
+
- Prepared local trace data is limited to 5 GiB per import. Use a smaller date window if an import reaches this limit.
|
|
348
|
+
- A complete trace must fit within the importer's 4 MiB upload payload limit. A larger trace is skipped and recorded with the `trace_too_large` reason.
|
|
349
|
+
- A trace is never split between upload requests.
|
|
350
|
+
|
|
351
|
+
Review `report.json` after a dry run or completed import to see exactly what was prepared, skipped, acknowledged, and verified.
|
|
352
|
+
|
|
353
|
+
See the [`mastra traces import` CLI reference](https://mastra.ai/reference/cli/mastra) for the complete option and environment-variable reference.
|
|
354
|
+
|
|
171
355
|
## Next steps
|
|
172
356
|
|
|
173
357
|
- 📹 [Mastra observability and Studio workshop](https://www.youtube.com/watch?v=dKO_a3RPra0)
|
|
@@ -14,6 +14,8 @@ Deploy with a single command, [`mastra deploy`](https://mastra.ai/docs/mastra-pl
|
|
|
14
14
|
|
|
15
15
|
Each project can run multiple [**Environments**](https://mastra.ai/docs/mastra-platform/environments) (for example `production` and `staging`) and provision [**Hosted databases**](https://mastra.ai/docs/mastra-platform/database) from the CLI or project settings to persist application data. Each environment also gets a managed [**Workspace**](https://mastra.ai/docs/mastra-platform/workspaces) that gives agents a filesystem and sandbox with no manual configuration.
|
|
16
16
|
|
|
17
|
+
Organization-level [**Alerts**](https://mastra.ai/docs/mastra-platform/alerts) notify your team when deploys fail or running services stop. Alerts can cover every project or selected projects and environments, with notifications sent to Slack, email, or webhooks.
|
|
18
|
+
|
|
17
19
|
[**Trace Intelligence**](https://mastra.ai/docs/mastra-platform/trace-intelligence) finds recurring goals, outcomes, behaviors, and sentiment across your agent traces. Trace Intelligence is available in private beta for selected projects.
|
|
18
20
|
|
|
19
21
|
## Get started
|
|
@@ -124,6 +124,27 @@ You can use this history in two ways:
|
|
|
124
124
|
|
|
125
125
|
> **Note:** `lastMessages` counts every stored message, including tool calls, tool results, and [signals](https://mastra.ai/docs/harness/signals) of any kind, so a single turn can add several messages to the count. The window also slides forward on every request: once a thread grows past the limit, the oldest message leaves context on each turn, which changes the start of the prompt and invalidates the provider prompt cache. For long-running conversations, use [Observational Memory](https://mastra.ai/docs/memory/observational-memory), which keeps the prompt prefix stable.
|
|
126
126
|
|
|
127
|
+
### Limit history by tokens
|
|
128
|
+
|
|
129
|
+
Message count is a poor proxy for context size: a tool result can be a few tokens or thousands. Use `messageHistory` to keep recent history within a token budget instead:
|
|
130
|
+
|
|
131
|
+
```typescript
|
|
132
|
+
export const agent = new Agent({
|
|
133
|
+
id: 'test-agent',
|
|
134
|
+
memory: new Memory({
|
|
135
|
+
options: {
|
|
136
|
+
messageHistory: { maxTokens: 8_000, atMaxRemoveTokens: 2_000 },
|
|
137
|
+
},
|
|
138
|
+
}),
|
|
139
|
+
})
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Mastra counts the complete prompt against `maxTokens`, including remembered history, system instructions, context, and the current turn. When the prompt exceeds the budget, Mastra removes the oldest remembered messages until the prompt is at most `maxTokens - atMaxRemoveTokens`. `atMaxRemoveTokens` defaults to 25% of `maxTokens`. Removing history in chunks keeps the prompt prefix stable across several turns, which helps provider prompt caches stay warm.
|
|
143
|
+
|
|
144
|
+
System messages, context, the current turn's input, and the agent's responses are never trimmed. Linked tool calls and results are removed together. If protected content alone exceeds `maxTokens`, Mastra removes all remembered history but keeps the protected content.
|
|
145
|
+
|
|
146
|
+
Trimmed messages stay in storage. During agent runs, Mastra persists a per-thread boundary so they're excluded from later turns. Setting `messageHistory` without `lastMessages` disables the default 10-message cap. Set both to combine a count cap with a token budget. Set `maxTokens` to `0` to disable message history.
|
|
147
|
+
|
|
127
148
|
> **Tip:** When memory is enabled, [Studio](https://mastra.ai/docs/studio/overview) uses message history to display past conversations in the chat sidebar.
|
|
128
149
|
|
|
129
150
|
## Thread title generation
|
|
@@ -849,6 +849,39 @@ Transform hooks are always awaited, on every path (manual `observe()`/`reflect()
|
|
|
849
849
|
|
|
850
850
|
Because hooks receive `threadId` and `resourceId`, you can also use them to update [working memory](https://mastra.ai/docs/memory/working-memory) via `memory.updateWorkingMemory()` during a cycle. These external updates aren't atomic with the OM text commit.
|
|
851
851
|
|
|
852
|
+
### Redact skill results
|
|
853
|
+
|
|
854
|
+
Agent skills are injected into the agent as tools (`skill`, `skill_search`, `skill_read`). Their results contain the skill's full instructions or file contents, so without redaction the Observer re-observes that text every time a skill is used. `skillResultRedactor()` is a ready-made `beforeObservation` hook that replaces those results with a placeholder and leaves everything else in place. The tool call survives, so the Observer still records which skill was used and what it was called with.
|
|
855
|
+
|
|
856
|
+
```typescript
|
|
857
|
+
import { Memory } from '@mastra/memory'
|
|
858
|
+
import { skillResultRedactor } from '@mastra/memory/hooks'
|
|
859
|
+
|
|
860
|
+
const memory = new Memory({
|
|
861
|
+
options: {
|
|
862
|
+
observationalMemory: {
|
|
863
|
+
model: 'google/gemini-2.5-flash',
|
|
864
|
+
hooks: {
|
|
865
|
+
beforeObservation: skillResultRedactor(),
|
|
866
|
+
},
|
|
867
|
+
},
|
|
868
|
+
},
|
|
869
|
+
})
|
|
870
|
+
```
|
|
871
|
+
|
|
872
|
+
Pass `toolNames` to redact results from a different set of tools. Because a hook is a function over the messages, it composes with your own transforms by chaining the outputs. Await each chained hook so an async one isn't discarded:
|
|
873
|
+
|
|
874
|
+
```typescript
|
|
875
|
+
const dropSkillResults = skillResultRedactor()
|
|
876
|
+
|
|
877
|
+
hooks: {
|
|
878
|
+
beforeObservation: async input => {
|
|
879
|
+
const messages = (await dropSkillResults(input))?.messages ?? input.messages
|
|
880
|
+
return { messages: messages.filter(m => m.role !== 'signal') }
|
|
881
|
+
},
|
|
882
|
+
}
|
|
883
|
+
```
|
|
884
|
+
|
|
852
885
|
## Migrating existing threads
|
|
853
886
|
|
|
854
887
|
No manual migration needed. OM reads existing messages and observes them lazily when thresholds are exceeded.
|
|
@@ -130,7 +130,7 @@ await observability!.deleteFeedback({
|
|
|
130
130
|
})
|
|
131
131
|
```
|
|
132
132
|
|
|
133
|
-
Deleted records also disappear from feedback analytics. ClickHouse uses a lightweight delete to hide rows without guaranteeing immediate physical removal, so open-source deployments must configure an [observability retention period](https://mastra.ai/reference/storage/retention) to physically purge them.
|
|
133
|
+
Deleted records also disappear from feedback analytics. ClickHouse uses a lightweight delete to hide rows without guaranteeing immediate physical removal, so open-source deployments must configure an [observability retention period](https://mastra.ai/reference/storage/retention) to physically purge them. Configure retention for every observability signal to expire deletion requests after the signal rows they protect. If any signal is unbounded, deletion requests also remain unbounded to prevent deleted data from being reintroduced. Delete APIs intentionally leave cursor-only delta rows untouched. These rows contain identifiers rather than feedback payloads and expire within two days.
|
|
134
134
|
|
|
135
135
|
## Query feedback analytics
|
|
136
136
|
|
|
@@ -459,6 +459,8 @@ export const mastra = new Mastra({
|
|
|
459
459
|
})
|
|
460
460
|
```
|
|
461
461
|
|
|
462
|
+
`process()` must mutate the span it receives and return the same instance, or return `undefined` to drop the span. Don't return a copy (for example `{ ...span, input: '...' }`): `exportSpan()` and `isValid` are instance members of the live span, so a copy can't be exported. Mastra logs a processor error and drops the span in that case.
|
|
463
|
+
|
|
462
464
|
Processors are executed in the order they're defined, allowing you to chain multiple transformations. Common use cases include:
|
|
463
465
|
|
|
464
466
|
- Redacting sensitive data (passwords, tokens, API keys)
|
|
@@ -371,6 +371,49 @@ await server.init()
|
|
|
371
371
|
app.listen(4111)
|
|
372
372
|
```
|
|
373
373
|
|
|
374
|
+
## Test adapter compatibility
|
|
375
|
+
|
|
376
|
+
Use `@mastra/server-adapters-test-suite` to run the same conformance tests as Mastra's official adapters. Install it with its peer dependencies in your adapter project:
|
|
377
|
+
|
|
378
|
+
**npm**:
|
|
379
|
+
|
|
380
|
+
```bash
|
|
381
|
+
npm install --save-dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
|
|
382
|
+
```
|
|
383
|
+
|
|
384
|
+
**pnpm**:
|
|
385
|
+
|
|
386
|
+
```bash
|
|
387
|
+
pnpm add --save-dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
**Yarn**:
|
|
391
|
+
|
|
392
|
+
```bash
|
|
393
|
+
yarn add --dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
|
|
394
|
+
```
|
|
395
|
+
|
|
396
|
+
**Bun**:
|
|
397
|
+
|
|
398
|
+
```bash
|
|
399
|
+
bun add --dev @mastra/server-adapters-test-suite @mastra/core @mastra/mcp @mastra/server vitest zod
|
|
400
|
+
```
|
|
401
|
+
|
|
402
|
+
Provide framework-specific setup and request execution functions to the route suite:
|
|
403
|
+
|
|
404
|
+
```typescript
|
|
405
|
+
import { createRouteAdapterTestSuite } from '@mastra/server-adapters-test-suite'
|
|
406
|
+
import { executeHttpRequest, setupAdapter } from './adapter-test-helpers'
|
|
407
|
+
|
|
408
|
+
createRouteAdapterTestSuite({
|
|
409
|
+
suiteName: 'My framework adapter',
|
|
410
|
+
setupAdapter,
|
|
411
|
+
executeHttpRequest,
|
|
412
|
+
})
|
|
413
|
+
```
|
|
414
|
+
|
|
415
|
+
The package also provides root exports for Model Context Protocol (MCP), multipart request, HTTP logging, and body limit tests. See the [package README](https://github.com/mastra-ai/mastra/tree/main/server-adapters/server-adapters-test-suite) for the supported peer dependency versions.
|
|
416
|
+
|
|
374
417
|
> **Tip:** The existing [@mastra/hono](https://github.com/mastra-ai/mastra/blob/main/server-adapters/hono/src/index.ts) and [@mastra/express](https://github.com/mastra-ai/mastra/blob/main/server-adapters/express/src/index.ts) implementations are good references when building your custom adapter. They show how to handle framework-specific patterns for context storage and middleware registration, plus response handling.
|
|
375
418
|
>
|
|
376
419
|
> If you want to use [Studio](https://mastra.ai/docs/studio/overview) with your server adapter, use [`mastra studio`](https://mastra.ai/reference/cli/mastra) to only launch the Studio UI.
|
package/.docs/docs/subagents.md
CHANGED
|
@@ -135,11 +135,13 @@ The subagent reads these entries in its tools and dynamic configuration, such as
|
|
|
135
135
|
Called after a delegation finishes. Use it to inspect results or provide feedback, or alternatively stop execution:
|
|
136
136
|
|
|
137
137
|
- `context.bail()`: Stop the parent agent's loop immediately
|
|
138
|
-
- Return `{ feedback: '...' }`: Add feedback that gets saved to the parent agent's memory and is
|
|
138
|
+
- Return `{ feedback: '...' }`: Add feedback that gets saved to the parent agent's memory and is available on the next turn
|
|
139
139
|
- Return `{ resultText: '...' }`: Replace the tool result text the parent model sees for this delegation, within the current run
|
|
140
140
|
|
|
141
141
|
Set `resultText` when the subagent's own result would give the parent a misleading signal. For example, empty text from a subagent stopped on a tool-calls step looks like a successful but empty delegation to the parent model. You can also replace the error text from a failed delegation with a more useful message. This doesn't recover the delegation. The parent still receives a failed tool result. Unlike `feedback` on the next turn, `resultText` affects the parent's reasoning immediately.
|
|
142
142
|
|
|
143
|
+
To stop the parent agent on failure and save feedback for a later turn, call `bail()` and return `feedback`:
|
|
144
|
+
|
|
143
145
|
```typescript
|
|
144
146
|
const stream = await parentAgent.stream('Research AI trends', {
|
|
145
147
|
maxSteps: 10,
|
|
@@ -161,12 +163,41 @@ const stream = await parentAgent.stream('Research AI trends', {
|
|
|
161
163
|
|
|
162
164
|
The `context` object includes:
|
|
163
165
|
|
|
164
|
-
| Property | Description
|
|
165
|
-
| ------------- |
|
|
166
|
-
| `primitiveId` | The ID of the subagent that ran
|
|
167
|
-
| `result` | The subagent's response, including `text`, `usage`, `finishReason`, and `
|
|
168
|
-
| `
|
|
169
|
-
| `
|
|
166
|
+
| Property | Description |
|
|
167
|
+
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
168
|
+
| `primitiveId` | The ID of the subagent that ran |
|
|
169
|
+
| `result` | The subagent's response, including `text`, `usage`, `finishReason`, `subAgentToolResults`, `subAgentThreadId`, and `subAgentResourceId` when available |
|
|
170
|
+
| `success` | Whether the delegation succeeded |
|
|
171
|
+
| `error` | The original error if the delegation failed |
|
|
172
|
+
| `messages` | The collected subagent transcript, including partial messages when available |
|
|
173
|
+
| `bail()` | Function to stop the parent agent's loop |
|
|
174
|
+
|
|
175
|
+
#### Failed delegations
|
|
176
|
+
|
|
177
|
+
When a subagent run fails, the parent model receives a failed tool result with a generic error message. This doesn't automatically stop the parent agent, which can explain the failure or choose another approach. Call `context.bail()` to stop its loop instead.
|
|
178
|
+
|
|
179
|
+
The hook receives `success: false`, the original `error`, and any collected partial results and messages. Returning `resultText` replaces the error text the parent model sees without changing the delegation's failed status or discarding its original cause. An empty string is also a valid replacement.
|
|
180
|
+
|
|
181
|
+
To let the parent continue with a curated failure message instead of stopping its loop, return `resultText` without calling `bail()`:
|
|
182
|
+
|
|
183
|
+
```typescript
|
|
184
|
+
const stream = await parentAgent.stream('Research AI trends', {
|
|
185
|
+
maxSteps: 10,
|
|
186
|
+
delegation: {
|
|
187
|
+
onDelegationComplete: ({ success }) => {
|
|
188
|
+
if (!success) {
|
|
189
|
+
return {
|
|
190
|
+
resultText: 'The delegated task failed. Try another approach.',
|
|
191
|
+
}
|
|
192
|
+
}
|
|
193
|
+
},
|
|
194
|
+
},
|
|
195
|
+
})
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Prefer a curated message, as in this example. Returning `{ resultText: error.message }` explicitly forwards the underlying message to the parent model's provider, and the model may repeat it to the end user. Provider errors can contain sensitive information. A generic `resultText` isn't a redaction policy for logs, traces, or streamed error payloads.
|
|
199
|
+
|
|
200
|
+
For delegations run as [background tasks](https://mastra.ai/docs/harness/background-tasks), configured retries can invoke `onDelegationComplete` once per attempt. Account for repeated calls if your hook has side effects.
|
|
170
201
|
|
|
171
202
|
### Hook errors
|
|
172
203
|
|
|
@@ -41,9 +41,13 @@ GitHub Signals requires:
|
|
|
41
41
|
- A Mastra storage adapter with memory and notification support. The provider stores each subscription in the thread's metadata, so the thread must already exist.
|
|
42
42
|
- The `gitcrawl` command on `PATH`, configured to access the repositories you want to monitor. The provider runs `gitcrawl sync` and reads its SQLite database.
|
|
43
43
|
- The `sqlite3` command on `PATH`.
|
|
44
|
-
- The [GitHub CLI](https://cli.github.com/) installed and authenticated. The provider uses `gh api` to check
|
|
44
|
+
- The [GitHub CLI](https://cli.github.com/) installed and authenticated. The provider uses real `gh api` responses to check repository permissions and GitHub App ownership before notifying the agent.
|
|
45
45
|
|
|
46
|
-
By default, comments from users with `admin`, `maintain`, or `write` access can trigger notifications. CodeRabbit and Devin bot comments are
|
|
46
|
+
By default, comments from users with `admin`, `maintain`, or `write` access can trigger notifications. CodeRabbit and Devin bot comments are explicitly allowed.
|
|
47
|
+
|
|
48
|
+
Bot authorization checks `ignoredBots` first, using an exact case-insensitive login match. Other bots can trigger notifications when they appear in `authorizedBots`, when the GitHub App is owned by a user whose repository permission appears in `authorizedPermissions`, or when the app is owned by the organization that owns the repository. The default authorized permissions are `admin`, `maintain`, and `write`.
|
|
49
|
+
|
|
50
|
+
Failed, inaccessible, malformed, and unsupported app-owner lookups deny the bot comment. Successful app-owner lookups are cached for 24 hours; failed lookups aren't cached.
|
|
47
51
|
|
|
48
52
|
## Agent and subscription
|
|
49
53
|
|
|
@@ -91,7 +91,7 @@ Trace deletion cascades to spans, trace roots and branches, metrics, logs, score
|
|
|
91
91
|
|
|
92
92
|
Lightweight deletion is a hide-only operation that marks rows with ClickHouse's `_row_exists` mask. Physical removal depends on merges and deployment-configured retention TTLs. `ObservabilityStorageClickhouseVNext` applies retention only when you provide a `RetentionConfig`; Mastra OSS doesn't configure a default retention TTL.
|
|
93
93
|
|
|
94
|
-
|
|
94
|
+
When all five observability signals have finite retention, Mastra also applies a TTL to deletion requests so they outlive the signal rows they protect. If any signal is unbounded, deletion requests remain unbounded. See [storage retention](https://mastra.ai/reference/storage/retention) for how the deletion-request TTL is calculated.
|
|
95
95
|
|
|
96
96
|
### Observability with the legacy domain
|
|
97
97
|
|
|
@@ -4,49 +4,49 @@
|
|
|
4
4
|
|
|
5
5
|
# Confident AI
|
|
6
6
|
|
|
7
|
-
[Confident AI](https://www.confident-ai.com/) is an LLM observability and evaluation platform for teams
|
|
7
|
+
[Confident AI](https://www.confident-ai.com/) is an LLM observability and evaluation platform for teams building reliable AI applications in both development and production.
|
|
8
8
|
|
|
9
|
-
The
|
|
9
|
+
The `ConfidentMastraExporter` from [`confident-trace`](https://github.com/confident-ai/confident-trace) sends your Mastra traces to Confident AI, where agent runs show up with the full agent, model, and tool hierarchy, and can be scored automatically as they arrive.
|
|
10
10
|
|
|
11
11
|
## Installation
|
|
12
12
|
|
|
13
13
|
**npm**:
|
|
14
14
|
|
|
15
15
|
```bash
|
|
16
|
-
npm install @
|
|
16
|
+
npm install confident-trace@latest
|
|
17
17
|
```
|
|
18
18
|
|
|
19
19
|
**pnpm**:
|
|
20
20
|
|
|
21
21
|
```bash
|
|
22
|
-
pnpm add @
|
|
22
|
+
pnpm add confident-trace@latest
|
|
23
23
|
```
|
|
24
24
|
|
|
25
25
|
**Yarn**:
|
|
26
26
|
|
|
27
27
|
```bash
|
|
28
|
-
yarn add @
|
|
28
|
+
yarn add confident-trace@latest
|
|
29
29
|
```
|
|
30
30
|
|
|
31
31
|
**Bun**:
|
|
32
32
|
|
|
33
33
|
```bash
|
|
34
|
-
bun add @
|
|
34
|
+
bun add confident-trace@latest
|
|
35
35
|
```
|
|
36
36
|
|
|
37
37
|
## Configuration
|
|
38
38
|
|
|
39
39
|
### Prerequisites
|
|
40
40
|
|
|
41
|
-
1. **Confident AI account**: Sign up at [confident-ai.com](https://
|
|
41
|
+
1. **Confident AI account**: Sign up at [confident-ai.com](https://app.confident-ai.com)
|
|
42
42
|
2. **API key**: Generate one in your Confident AI project settings
|
|
43
43
|
3. **Environment variables**: Set your credentials:
|
|
44
44
|
|
|
45
45
|
```bash
|
|
46
46
|
CONFIDENT_API_KEY=confident_proj_xxxxxxxxxxxxx
|
|
47
47
|
|
|
48
|
-
# Optional
|
|
49
|
-
|
|
48
|
+
# Optional. EU region, defaults to the US servers
|
|
49
|
+
CONFIDENT_OTEL_ENDPOINT=https://eu.otel.confident-ai.com/v1/traces
|
|
50
50
|
```
|
|
51
51
|
|
|
52
52
|
### Zero-Config Setup
|
|
@@ -56,14 +56,14 @@ With environment variables set, use the exporter with no configuration:
|
|
|
56
56
|
```typescript
|
|
57
57
|
import { Mastra } from '@mastra/core'
|
|
58
58
|
import { Observability } from '@mastra/observability'
|
|
59
|
-
import {
|
|
59
|
+
import { ConfidentMastraExporter } from 'confident-trace/mastra'
|
|
60
60
|
|
|
61
61
|
export const mastra = new Mastra({
|
|
62
62
|
observability: new Observability({
|
|
63
63
|
configs: {
|
|
64
|
-
|
|
64
|
+
confidentAI: {
|
|
65
65
|
serviceName: 'my-service',
|
|
66
|
-
exporters: [new
|
|
66
|
+
exporters: [new ConfidentMastraExporter()],
|
|
67
67
|
},
|
|
68
68
|
},
|
|
69
69
|
}),
|
|
@@ -77,17 +77,17 @@ You can also pass credentials directly (takes precedence over environment variab
|
|
|
77
77
|
```typescript
|
|
78
78
|
import { Mastra } from '@mastra/core'
|
|
79
79
|
import { Observability } from '@mastra/observability'
|
|
80
|
-
import {
|
|
80
|
+
import { ConfidentMastraExporter } from 'confident-trace/mastra'
|
|
81
81
|
|
|
82
82
|
export const mastra = new Mastra({
|
|
83
83
|
observability: new Observability({
|
|
84
84
|
configs: {
|
|
85
|
-
|
|
85
|
+
confidentAI: {
|
|
86
86
|
serviceName: 'my-service',
|
|
87
87
|
exporters: [
|
|
88
|
-
new
|
|
88
|
+
new ConfidentMastraExporter({
|
|
89
89
|
apiKey: process.env.CONFIDENT_API_KEY,
|
|
90
|
-
|
|
90
|
+
endpoint: process.env.CONFIDENT_OTEL_ENDPOINT,
|
|
91
91
|
}),
|
|
92
92
|
],
|
|
93
93
|
},
|
|
@@ -96,47 +96,71 @@ export const mastra = new Mastra({
|
|
|
96
96
|
})
|
|
97
97
|
```
|
|
98
98
|
|
|
99
|
-
|
|
99
|
+
These options are available to you:
|
|
100
100
|
|
|
101
|
-
|
|
101
|
+
```typescript
|
|
102
|
+
new ConfidentMastraExporter({
|
|
103
|
+
apiKey: process.env.CONFIDENT_API_KEY, // Default: CONFIDENT_API_KEY
|
|
104
|
+
endpoint: process.env.CONFIDENT_OTEL_ENDPOINT, // Default: US servers
|
|
105
|
+
captureContent: false, // Omit inputs, outputs, and messages
|
|
106
|
+
maxContentBytes: 512, // Truncate captured content
|
|
107
|
+
timeoutMillis: 30000, // Flush and shutdown budget
|
|
108
|
+
resourceAttributes: { 'service.name': 'my-agent' }, // Overrides serviceName
|
|
109
|
+
})
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Traces carry Mastra's configured `serviceName` unless `resourceAttributes['service.name']` overrides it. Mastra's own sampling, filters, and span processors run before export, so whatever your pipeline lets through is what Confident AI receives.
|
|
113
|
+
|
|
114
|
+
## Supported signals
|
|
115
|
+
|
|
116
|
+
The exporter converts ended Mastra spans into OTLP spans, preserving trace IDs, span IDs, parent IDs, external parent IDs, and original timestamps. It needs no start-event cache, so resumed workflow spans and out-of-order completion export correctly.
|
|
117
|
+
|
|
118
|
+
- **Agent and workflow execution**: Operation names, timing, status, and the parent-child relationships between steps.
|
|
119
|
+
- **Model calls**: Model details, messages, finish reasons, and [token usage](https://www.confident-ai.com/docs/llm-tracing/features/token-usage-cost).
|
|
120
|
+
- **Tool calls**: Tool names and their [input/output](https://www.confident-ai.com/docs/llm-tracing/features/input-output).
|
|
121
|
+
- **Trace details**: The root operation's name, tags, metadata, and input/output.
|
|
122
|
+
- **Errors**: Failed operations retain their error status to make failures visible in the trace.
|
|
123
|
+
|
|
124
|
+
This integration exports tracing data. Mastra logs, metrics, scores, and feedback aren't included.
|
|
125
|
+
|
|
126
|
+
## Custom trace metadata
|
|
127
|
+
|
|
128
|
+
Metadata and tags passed through Mastra's `tracingOptions` are forwarded to the trace in Confident AI, where you can filter and group by them:
|
|
102
129
|
|
|
103
130
|
```typescript
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
toolMetricCollectionMap: {
|
|
109
|
-
search: 'search-tool-metrics', // applied to the "search" tool
|
|
131
|
+
const result = await mastra.getAgent('assistant').generate('How do you make the best coffee?', {
|
|
132
|
+
tracingOptions: {
|
|
133
|
+
metadata: { release: '2026-09', tier: 'enterprise' },
|
|
134
|
+
tags: ['production', 'support'],
|
|
110
135
|
},
|
|
111
136
|
})
|
|
112
137
|
```
|
|
113
138
|
|
|
114
|
-
|
|
139
|
+
Your metadata is merged with the fields Mastra sets itself, such as `runId`. Tags apply to the root span, so they land on the trace rather than on individual spans.
|
|
140
|
+
|
|
141
|
+
## Lifecycle
|
|
142
|
+
|
|
143
|
+
Mastra owns this exporter. It creates its own OTLP processor and never registers, replaces, or shuts down the global OpenTelemetry provider, so there is no `init()` call to make and no preload to add to your start command.
|
|
144
|
+
|
|
145
|
+
Spans are batched and sent when the exporter is shut down. Shut it down once, after every agent run and stream has finished, or a short-lived script exits before anything is sent:
|
|
115
146
|
|
|
116
147
|
```typescript
|
|
117
|
-
|
|
118
|
-
apiKey: process.env.CONFIDENT_API_KEY,
|
|
119
|
-
environment: 'production', // Default: "development"
|
|
120
|
-
name: 'my-trace', // Default: the Mastra serviceName
|
|
121
|
-
tags: ['production'],
|
|
122
|
-
metadata: { team: 'growth' },
|
|
123
|
-
})
|
|
148
|
+
await observability.shutdown()
|
|
124
149
|
```
|
|
125
150
|
|
|
126
|
-
|
|
151
|
+
In a long-running server, shut down during graceful shutdown rather than per request. Traces can take up to 30 seconds to appear in the Observatory after being sent.
|
|
152
|
+
|
|
153
|
+
## Threads and trace properties
|
|
154
|
+
|
|
155
|
+
This exporter maps Mastra's own tracing surface, so trace metadata and tags come from `tracingOptions` as shown above.
|
|
127
156
|
|
|
128
|
-
|
|
157
|
+
Grouping turns into [threads](https://www.confident-ai.com/docs/llm-tracing/features/threads) and setting trace-level properties such as user IDs use `confident-trace`'s SDK instrumentation, which attaches to Mastra directly rather than through an exporter. See the [Mastra integration guide](https://www.confident-ai.com/docs/integrations/third-party/mastra) for that setup.
|
|
129
158
|
|
|
130
|
-
|
|
131
|
-
| ---------------------------------------------------------------------- | ---------------------- |
|
|
132
|
-
| `AGENT_RUN`, `WORKFLOW_RUN` | `AGENT` |
|
|
133
|
-
| `MODEL_GENERATION` | `LLM` |
|
|
134
|
-
| `TOOL_CALL`, `MCP_TOOL_CALL`, `PROVIDER_TOOL_CALL`, `CLIENT_TOOL_CALL` | `TOOL` |
|
|
135
|
-
| `RAG_EMBEDDING`, `RAG_VECTOR_OPERATION` | `RETRIEVER` |
|
|
136
|
-
| All other exported span types | `CUSTOM` |
|
|
159
|
+
Choose one or the other. Do not use both at once.
|
|
137
160
|
|
|
138
161
|
## Related
|
|
139
162
|
|
|
140
163
|
- [Tracing Overview](https://mastra.ai/docs/observability/tracing/overview)
|
|
141
|
-
- [
|
|
142
|
-
- [
|
|
164
|
+
- [Mastra integration guide](https://www.confident-ai.com/docs/integrations/third-party/mastra)
|
|
165
|
+
- [Online evals](https://www.confident-ai.com/docs/llm-tracing/online-evals)
|
|
166
|
+
- [confident-trace on GitHub](https://github.com/confident-ai/confident-trace)
|