@mastra/mcp-docs-server 1.2.13-alpha.4 → 1.2.13-alpha.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.docs/docs/memory/observational-memory.md +30 -13
- package/.docs/docs/memory/overview.md +14 -0
- package/.docs/models/index.md +1 -1
- package/.docs/models/providers/minimax.md +1 -1
- package/.docs/models/providers/perplexity-agent.md +3 -1
- package/.docs/reference/agents/durable-agent.md +11 -0
- package/.docs/reference/evals/summarization.md +203 -0
- package/.docs/reference/index.md +1 -0
- package/.docs/reference/memory/observational-memory.md +74 -24
- package/.docs/reference/observability/tracing/interfaces.md +3 -0
- package/.docs/reference/workspace/platform-sandbox.md +22 -0
- package/CHANGELOG.md +7 -0
- package/package.json +4 -4
|
@@ -94,7 +94,7 @@ See [configuration options](https://mastra.ai/reference/memory/observational-mem
|
|
|
94
94
|
>
|
|
95
95
|
> For an AI SDK example, see [Using Mastra Memory](https://mastra.ai/guides/build-your-ui/ai-sdk-ui).
|
|
96
96
|
|
|
97
|
-
> **Note:** OM currently only supports `@mastra/pg`, `@mastra/libsql`, `@mastra/mongodb`, and `@mastra/convex` storage adapters. It uses background agents for managing memory. When no model is set, the default model is `google/gemini-2.5-flash`.
|
|
97
|
+
> **Note:** OM currently only supports `@mastra/pg`, `@mastra/libsql`, `@mastra/mysql`, `@mastra/mongodb`, and `@mastra/convex` storage adapters. It uses background agents for managing memory. When no model is set, the default model is `google/gemini-2.5-flash`.
|
|
98
98
|
|
|
99
99
|
## Temporal gap markers
|
|
100
100
|
|
|
@@ -378,7 +378,7 @@ new Agent({
|
|
|
378
378
|
})
|
|
379
379
|
```
|
|
380
380
|
|
|
381
|
-
You can also pass an allowlist of mimeType globs (for example `['image/*']`) to forward only the kinds the Observer can handle.
|
|
381
|
+
You can also pass an allowlist of mimeType globs (for example `['image/*']`) to forward only the kinds the Observer can handle. Alternatively, set `observeAttachments: 'auto'` to let Mastra decide from the provider capabilities registry: attachments are forwarded when the Observer model supports multimodal input and dropped otherwise, falling back to `true` when no capability data is available for the model.
|
|
382
382
|
|
|
383
383
|
```md
|
|
384
384
|
Date: 2026-01-15
|
|
@@ -399,12 +399,29 @@ Example: An agent using Playwright MCP might see 50,000+ tokens per page snapsho
|
|
|
399
399
|
|
|
400
400
|
When observations exceed their threshold (default: 40,000 tokens), the Reflector condenses them and combines related items, plus reflects on patterns.
|
|
401
401
|
|
|
402
|
+
Reflections don't accumulate as a separate, ever-growing layer. Each reflection rewrites the entire observation log: the Reflector's output becomes the new log, and new observations append after it. When the log next hits the threshold, the Reflector re-processes everything — including earlier reflections — condensing older information more aggressively while keeping recent detail. Memory stays bounded around the reflection threshold no matter how long the conversation runs.
|
|
403
|
+
|
|
402
404
|
The result is a three-tier system:
|
|
403
405
|
|
|
404
406
|
1. **Recent messages**: Exact conversation history for the current task
|
|
405
407
|
2. **Observations**: A log of what the Observer has seen
|
|
406
408
|
3. **Reflections**: Condensed observations when memory becomes too long
|
|
407
409
|
|
|
410
|
+
### Context over time
|
|
411
|
+
|
|
412
|
+
With default settings, the context window doesn't grow unbounded. It oscillates through an observe-and-shrink cycle:
|
|
413
|
+
|
|
414
|
+

|
|
415
|
+
|
|
416
|
+
1. **0 → 30k tokens**: Message history grows normally. In the background, the Observer buffers observations every \~6k tokens (`bufferTokens: 0.2`).
|
|
417
|
+
2. **30k reached**: Buffered observations activate instantly. Observed messages are removed from the context window and only \~6k tokens of recent history remain (`bufferActivation: 0.8` retains 20% of the threshold). The \~24k tokens of removed messages become roughly 1-5k tokens of observations at typical 5-40x compression.
|
|
418
|
+
3. **Repeat**: History grows from \~6k back toward 30k and shrinks again. Each cycle appends to the observation log, which grows much more slowly than raw history.
|
|
419
|
+
4. **Observations reach 40k**: The Reflector condenses the observation log — including any earlier reflections — into a new, smaller log.
|
|
420
|
+
|
|
421
|
+
The result: in the normal buffered cycle, raw history oscillates between roughly 6k and 30k tokens and the observation log stays around 40k tokens, however long the conversation runs. These are activation thresholds rather than hard caps — if background buffering falls behind, history can grow past the threshold until `blockAfter` (default `1.2`) forces a synchronous observation at \~36k tokens (\~48k for reflection) as a safety ceiling.
|
|
422
|
+
|
|
423
|
+
With [`shareTokenBudget`](https://mastra.ai/reference/memory/observational-memory) enabled, the two budgets pool together: while the observation log is small, message history can expand into the unused observation space (up to \~70k tokens with the defaults) before observation triggers, then shrinks back as observations accumulate.
|
|
424
|
+
|
|
408
425
|
### Retrieval mode
|
|
409
426
|
|
|
410
427
|
Normal OM compresses messages into observations, which is great for staying on task, but the original wording is gone. Retrieval mode fixes this by keeping each observation group linked to the raw messages that produced it. When the agent needs exact wording, tool output, or chronology that the summary compressed away, it can call a `recall` tool to page through the source messages.
|
|
@@ -684,7 +701,7 @@ Reflection works similarly, the Reflector runs in the background when observatio
|
|
|
684
701
|
| ------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
685
702
|
| `observation.bufferTokens` | `0.2` | How often to buffer. `0.2` means every 20% of `messageTokens`. With the default 30k threshold, that's roughly every 6k tokens. Can also be an absolute token count (e.g. `5000`). |
|
|
686
703
|
| `observation.bufferActivation` | `0.8` | How aggressively to clear the message window on activation. `0.8` means remove enough messages to keep only 20% of `messageTokens` remaining. Lower values keep more message history. |
|
|
687
|
-
| `observation.blockAfter` | `1.2` | Safety
|
|
704
|
+
| `observation.blockAfter` | `1.2` | Safety net if buffering can't keep up. Values from 1 up to (but not including) 100 multiply `messageTokens`: at `1.2`, synchronous observation is forced at 36k tokens (1.2 × 30k). Values of 100 or more are absolute token counts (e.g. `50_000`). |
|
|
688
705
|
| `activateAfterIdle` | none | Forces buffered observations to activate after a period of inactivity, even before `observation.messageTokens` is reached. Accepts a numeric millisecond value such as `300_000`, duration strings like `"5m"` or `"1hr"`, or `"auto"` for a provider-aware prompt cache TTL. |
|
|
689
706
|
| `activateOnProviderChange` | `false` | Forces buffered observations to activate when the next step uses a different `provider/model` than the one that produced the latest assistant step. Use this when switching providers or models would invalidate prompt cache reuse. |
|
|
690
707
|
| `reflection.bufferActivation` | `0.5` | When to start background reflection. `0.5` means reflection begins when observations reach 50% of the `observationTokens` threshold. |
|
|
@@ -696,16 +713,16 @@ If you're relying on prompt caching, set `activateAfterIdle` to `"auto"` or to a
|
|
|
696
713
|
|
|
697
714
|
With `"auto"`, Mastra chooses an idle activation TTL from the active model provider:
|
|
698
715
|
|
|
699
|
-
| Provider
|
|
700
|
-
|
|
|
701
|
-
| Anthropic, OpenRouter, unknown providers, xAI
|
|
702
|
-
| DeepSeek
|
|
703
|
-
| Google Gemini
|
|
704
|
-
| Groq
|
|
705
|
-
| OpenAI with `providerOptions.openai.promptCacheRetention: "24h"`
|
|
706
|
-
| OpenAI with `providerOptions.openai.promptCacheRetention: "in_memory"`
|
|
707
|
-
| OpenAI `gpt-4*`, `gpt-5`, `gpt-5-*`, `gpt-5.1
|
|
708
|
-
| Other OpenAI models
|
|
716
|
+
| Provider | Auto TTL |
|
|
717
|
+
| ------------------------------------------------------------------------------------------------------ | --------- |
|
|
718
|
+
| Anthropic, OpenRouter, unknown providers, xAI | 5 minutes |
|
|
719
|
+
| DeepSeek | 1 hour |
|
|
720
|
+
| Google Gemini | 24 hours |
|
|
721
|
+
| Groq | 2 hours |
|
|
722
|
+
| OpenAI with `providerOptions.openai.promptCacheRetention: "24h"` | 1 hour |
|
|
723
|
+
| OpenAI with `providerOptions.openai.promptCacheRetention: "in_memory"` | 5 minutes |
|
|
724
|
+
| OpenAI `gpt-4*`, `gpt-5`, `gpt-5-*`, and `gpt-5.1` through `gpt-5.4` (including `-` suffixed variants) | 5 minutes |
|
|
725
|
+
| Other OpenAI models | 1 hour |
|
|
709
726
|
|
|
710
727
|
```typescript
|
|
711
728
|
const memory = new Memory({
|
|
@@ -172,6 +172,20 @@ export const memoryAgent = new Agent({
|
|
|
172
172
|
|
|
173
173
|
See [Observational Memory](https://mastra.ai/docs/memory/observational-memory) for details on how observations and reflections work, and [the reference](https://mastra.ai/reference/memory/observational-memory) for all configuration options.
|
|
174
174
|
|
|
175
|
+
## What the model sees
|
|
176
|
+
|
|
177
|
+
Each memory feature lands in one of two places in the request sent to the model: the system messages or the conversation messages. Which layers are present depends on which features you've enabled — working memory, semantic recall, and Observational Memory only appear when configured, while message history is on by default. The diagram shows where each enabled layer is placed in the request; the list below describes what each layer contributes:
|
|
178
|
+
|
|
179
|
+

|
|
180
|
+
|
|
181
|
+
- [Working memory](https://mastra.ai/docs/memory/working-memory) is injected as a system message containing the template and the stored data. With `useStateSignals`, it's delivered as a state signal instead.
|
|
182
|
+
- [Semantic recall](https://mastra.ai/docs/memory/semantic-recall) matches from the current thread are inserted as regular messages and interleave with message history by timestamp. Matches from other threads are formatted into a system message instead.
|
|
183
|
+
- [Message history](https://mastra.ai/docs/memory/message-history) adds the last N messages in chronological order. Your new message always comes last.
|
|
184
|
+
- [Observational Memory](https://mastra.ai/docs/memory/observational-memory) replaces old raw history: reflections and observations live in a system message, and only messages that haven't been observed yet remain in the conversation. A short continuation reminder is placed at the start of the conversation messages.
|
|
185
|
+
- Context messages are the optional `context` array passed on a call, for example `agent.generate(msg, { context: [...] })`. Use them for one-off background such as app state or your own RAG results. They appear as regular conversation messages for that request only and are never saved to memory.
|
|
186
|
+
|
|
187
|
+
Conversation messages are ordered by timestamp and deduplicated by message ID, so recalled older messages appear before recent history. Context messages passed at call time are stamped with the current time, which places them after history and recall but before your new message. To inspect the exact context for a real request, use [Tracing](https://mastra.ai/docs/observability/tracing/overview) and open the LLM call spans, see [Observability](#observability) below.
|
|
188
|
+
|
|
175
189
|
## Memory in multi-agent systems
|
|
176
190
|
|
|
177
191
|
When a [supervisor agent](https://mastra.ai/docs/agents/supervisor-agents) delegates to a subagent, Mastra isolates subagent memory automatically. No flag enables this as it happens on every delegation. Understanding how this scoping works lets you decide what stays private and what to share intentionally.
|
package/.docs/models/index.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Model Providers
|
|
4
4
|
|
|
5
|
-
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to
|
|
5
|
+
Mastra provides a unified interface for working with LLMs across multiple providers, giving you access to 5153 models from 164 providers through a single API.
|
|
6
6
|
|
|
7
7
|
## Features
|
|
8
8
|
|
|
@@ -106,4 +106,4 @@ yarn add @ai-sdk/anthropic
|
|
|
106
106
|
bun add @ai-sdk/anthropic
|
|
107
107
|
```
|
|
108
108
|
|
|
109
|
-
For detailed provider-specific documentation, see the [AI SDK MiniMax (minimax.io) provider docs](https://ai-sdk.dev/providers/
|
|
109
|
+
For detailed provider-specific documentation, see the [AI SDK MiniMax (minimax.io) provider docs](https://ai-sdk.dev/providers/ai-sdk-providers/minimax).
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
# Perplexity Agent
|
|
4
4
|
|
|
5
|
-
Access
|
|
5
|
+
Access 20 Perplexity Agent models through Mastra's model router. Authentication is handled automatically using the `PERPLEXITY_API_KEY` environment variable.
|
|
6
6
|
|
|
7
7
|
Learn more in the [Perplexity Agent documentation](https://docs.perplexity.ai/docs/agent-api/models).
|
|
8
8
|
|
|
@@ -46,6 +46,8 @@ for await (const chunk of stream) {
|
|
|
46
46
|
| `perplexity-agent/google/gemini-2.5-pro` | 1.0M | | | | | | $1 | $10 |
|
|
47
47
|
| `perplexity-agent/google/gemini-3-flash-preview` | 1.0M | | | | | | $0.50 | $3 |
|
|
48
48
|
| `perplexity-agent/google/gemini-3.1-pro-preview` | 1.0M | | | | | | $2 | $12 |
|
|
49
|
+
| `perplexity-agent/moonshot-ai/kimi-k2.7-code` | 262K | | | | | | $0.95 | $4 |
|
|
50
|
+
| `perplexity-agent/moonshot-ai/kimi-k3` | 1.0M | | | | | | $3 | $15 |
|
|
49
51
|
| `perplexity-agent/nvidia/nemotron-3-super-120b-a12b` | 1.0M | | | | | | $0.25 | $3 |
|
|
50
52
|
| `perplexity-agent/openai/gpt-5-mini` | 400K | | | | | | $0.25 | $2 |
|
|
51
53
|
| `perplexity-agent/openai/gpt-5.1` | 400K | | | | | | $1 | $10 |
|
|
@@ -174,6 +174,17 @@ const { output, cleanup } = await durableAgent.observe(runId, {
|
|
|
174
174
|
await output.text
|
|
175
175
|
```
|
|
176
176
|
|
|
177
|
+
By default `observe()` waits indefinitely for events. If the process running the run stops unexpectedly, the run stops producing events but never emits a completion event, so the observed stream would wait forever. Pass `idleTimeoutMs` to bound that wait: after that many milliseconds of silence the stream ends. An optional `isAlive` check is consulted first — return `true` while the run is still being worked on (for example a long-running tool call, or a run paused waiting for human input) to keep waiting; returning `false`, or omitting `isAlive`, ends the stream with an error. A transient throw from `isAlive` is treated as "still alive", so a momentary check failure never ends a live stream.
|
|
178
|
+
|
|
179
|
+
```typescript
|
|
180
|
+
const { output } = await durableAgent.observe(runId, {
|
|
181
|
+
idleTimeoutMs: 30_000,
|
|
182
|
+
isAlive: () => runHeartbeat.isFresh(runId),
|
|
183
|
+
})
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
Ending a run on idle timeout runs the same cleanup as a run that errors (see the warning below), so its cached state is released rather than retained. Both options are opt-in; omit them for the previous wait-indefinitely behavior.
|
|
187
|
+
|
|
177
188
|
Returns: `Promise<DurableAgentStreamResult>`
|
|
178
189
|
|
|
179
190
|
> **Warning:** The `cleanup()` returned by `observe()` destroys the run's registry entries and cached events. Only call it when you are done with the run. If the run is suspended and you intend to resume later, don't call `cleanup()`. Let the auto-cleanup timer handle it after the run finishes or errors. Auto-cleanup doesn't fire on suspended events.
|
|
@@ -0,0 +1,203 @@
|
|
|
1
|
+
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
|
|
2
|
+
|
|
3
|
+
# Summarization scorer
|
|
4
|
+
|
|
5
|
+
The `createSummarizationScorer()` function creates a scorer that evaluates a summary on two axes: whether every claim it makes is supported by the source text, and whether it preserves the information the source states. The final score is the lower of the two, so a summary cannot pass by being faithful but empty, or thorough but wrong.
|
|
6
|
+
|
|
7
|
+
The summary is the agent's last message that carries text, and the source text defaults to the first user message of the run input. Pass `source` or `sourceExtractor` when the text being summarized lives somewhere else, such as a tool result.
|
|
8
|
+
|
|
9
|
+
## Usage example
|
|
10
|
+
|
|
11
|
+
Score a summary against the document it condenses.
|
|
12
|
+
|
|
13
|
+
```typescript
|
|
14
|
+
import { createSummarizationScorer } from '@mastra/evals/scorers/prebuilt'
|
|
15
|
+
|
|
16
|
+
const scorer = createSummarizationScorer({
|
|
17
|
+
model: 'openai/gpt-5.6-sol',
|
|
18
|
+
})
|
|
19
|
+
|
|
20
|
+
const result = await scorer.run({
|
|
21
|
+
input: {
|
|
22
|
+
inputMessages: [{ id: '1', role: 'user', content: sourceDocument }],
|
|
23
|
+
},
|
|
24
|
+
output: [{ id: '2', role: 'assistant', content: summary }],
|
|
25
|
+
})
|
|
26
|
+
|
|
27
|
+
console.log(result.score)
|
|
28
|
+
console.log(result.reason)
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## Summarization evaluation
|
|
32
|
+
|
|
33
|
+
Use this scorer when an agent condenses text:
|
|
34
|
+
|
|
35
|
+
- Document and transcript summarization
|
|
36
|
+
- Support thread and email digests
|
|
37
|
+
- Any step that compresses a long input into a short output
|
|
38
|
+
|
|
39
|
+
## Parameters
|
|
40
|
+
|
|
41
|
+
**model** (`MastraModelConfig`): The language model to use for judging claims and coverage questions
|
|
42
|
+
|
|
43
|
+
**options** (`SummarizationMetricOptions`): Configuration options for the scorer
|
|
44
|
+
|
|
45
|
+
**options.source** (`string`): Text the summary is judged against. Defaults to the user message of the run input
|
|
46
|
+
|
|
47
|
+
**options.sourceExtractor** (`(input, output) => string`): Function to derive the source text from the run input and output. Takes precedence over source
|
|
48
|
+
|
|
49
|
+
**options.maxQuestions** (`number`): Upper bound on the coverage questions drawn from the source (default: 10)
|
|
50
|
+
|
|
51
|
+
**options.scale** (`number`): Scale factor to multiply the final score (default: 1)
|
|
52
|
+
|
|
53
|
+
## `.run()` returns
|
|
54
|
+
|
|
55
|
+
**score** (`number`): Summarization score between 0 and scale (default 0-1), the lower of the alignment and coverage scores
|
|
56
|
+
|
|
57
|
+
**reason** (`string`): Human-readable explanation naming the axis that produced the score and the claims or questions behind it. Both axis scores appear in the text
|
|
58
|
+
|
|
59
|
+
**preprocessStepResult** (`object`): The alignment verdicts and the questions drawn from the source
|
|
60
|
+
|
|
61
|
+
**preprocessStepResult.alignment** (`{ claim: string; supported: boolean; reason: string }[]`): One verdict per claim the summary makes
|
|
62
|
+
|
|
63
|
+
**preprocessStepResult.questions** (`string[]`): The coverage questions drawn from the source text
|
|
64
|
+
|
|
65
|
+
**analyzeStepResult** (`object`): The coverage verdicts
|
|
66
|
+
|
|
67
|
+
**analyzeStepResult.coverage** (`{ question: string; answered: boolean; reason: string }[]`): One verdict per question, answered from the summary alone
|
|
68
|
+
|
|
69
|
+
The axis scores are derived from these verdicts rather than stored: alignment is the share of `alignment` entries with `supported: true`, and coverage is the share of `questions` whose `coverage` entry has `answered: true`.
|
|
70
|
+
|
|
71
|
+
## Scoring details
|
|
72
|
+
|
|
73
|
+
### Two-axis evaluation
|
|
74
|
+
|
|
75
|
+
The scorer runs a three-step pipeline:
|
|
76
|
+
|
|
77
|
+
1. **Source judgement**: the claims the summary makes are extracted and checked against the source, and closed-ended questions are drawn from the source. Every question is written so the source answers it "yes".
|
|
78
|
+
2. **Coverage**: each question is answered using the summary alone.
|
|
79
|
+
3. **Scoring**: the two ratios are computed and the lower one becomes the score.
|
|
80
|
+
|
|
81
|
+
The coverage step runs as a separate model call that never receives the source text. A judge that could see the source would answer questions from it rather than from the summary, which would hide the omissions this axis exists to measure.
|
|
82
|
+
|
|
83
|
+
### Scoring formula
|
|
84
|
+
|
|
85
|
+
```text
|
|
86
|
+
Alignment = supported_claims / total_claims
|
|
87
|
+
Coverage = answered_questions / total_questions
|
|
88
|
+
Summarization = min(Alignment, Coverage) × scale
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
The score is 0 when the summary yields no claims or the source yields no questions.
|
|
92
|
+
|
|
93
|
+
### Score interpretation
|
|
94
|
+
|
|
95
|
+
These ranges assume the default `scale` of 1. When using a custom scale, multiply accordingly.
|
|
96
|
+
|
|
97
|
+
- **0.9-1.0**: Excellent summary, faithful to the source and covering its main points
|
|
98
|
+
- **0.7-0.8**: Good summary with a small omission or an unsupported detail
|
|
99
|
+
- **0.4-0.6**: Moderate summary, either missing significant information or drifting from the source
|
|
100
|
+
- **0.1-0.3**: Poor summary, most of the source is lost or contradicted
|
|
101
|
+
- **0.0**: The summary supports no claims, answers no questions, or produced nothing to judge
|
|
102
|
+
|
|
103
|
+
### Reading the two axes
|
|
104
|
+
|
|
105
|
+
Both axes leave their verdicts on the run result: the alignment verdicts on the preprocess step, and the coverage verdicts on the analyze step. Each verdict carries the claim or question it belongs to and the reason behind it. The two failure modes look different:
|
|
106
|
+
|
|
107
|
+
- A low alignment score with high coverage means the summary invents or distorts detail
|
|
108
|
+
- A low coverage score with high alignment means the summary is accurate but leaves too much out
|
|
109
|
+
|
|
110
|
+
The reason field names whichever axis produced the score.
|
|
111
|
+
|
|
112
|
+
### What the score leaves out
|
|
113
|
+
|
|
114
|
+
Length plays no part in the score. A summary that repeats the source word for word supports every claim and answers every question, so it scores 1. Add a length check of your own when compression is part of what you're testing.
|
|
115
|
+
|
|
116
|
+
### Cost
|
|
117
|
+
|
|
118
|
+
Each evaluation makes three model calls. `maxQuestions` bounds the coverage half of the work, which otherwise grows with source length. Raise it for long documents where ten questions cannot represent the content.
|
|
119
|
+
|
|
120
|
+
## Scorer configuration
|
|
121
|
+
|
|
122
|
+
### Summarizing the run input
|
|
123
|
+
|
|
124
|
+
```typescript
|
|
125
|
+
const scorer = createSummarizationScorer({
|
|
126
|
+
model: 'openai/gpt-5.6-sol',
|
|
127
|
+
})
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
### Summarizing a document from elsewhere
|
|
131
|
+
|
|
132
|
+
```typescript
|
|
133
|
+
import { extractToolResults } from '@mastra/evals/scorers/utils'
|
|
134
|
+
|
|
135
|
+
const scorer = createSummarizationScorer({
|
|
136
|
+
model: 'openai/gpt-5.6-sol',
|
|
137
|
+
options: {
|
|
138
|
+
sourceExtractor: (input, output) => {
|
|
139
|
+
return extractToolResults(output)
|
|
140
|
+
.filter(({ toolName }) => toolName === 'fetchDocument')
|
|
141
|
+
.map(({ result }) => String(result))
|
|
142
|
+
.join('\n\n')
|
|
143
|
+
},
|
|
144
|
+
maxQuestions: 20,
|
|
145
|
+
},
|
|
146
|
+
})
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
## Example
|
|
150
|
+
|
|
151
|
+
Evaluate a summarization agent against a set of documents:
|
|
152
|
+
|
|
153
|
+
```typescript
|
|
154
|
+
import { runEvals } from '@mastra/core/evals'
|
|
155
|
+
import { createSummarizationScorer } from '@mastra/evals/scorers/prebuilt'
|
|
156
|
+
import { summarizerAgent } from './agent'
|
|
157
|
+
|
|
158
|
+
const scorer = createSummarizationScorer({
|
|
159
|
+
model: 'openai/gpt-5.6-sol',
|
|
160
|
+
options: { maxQuestions: 10 },
|
|
161
|
+
})
|
|
162
|
+
|
|
163
|
+
const result = await runEvals({
|
|
164
|
+
target: summarizerAgent,
|
|
165
|
+
scorers: [scorer],
|
|
166
|
+
data: [
|
|
167
|
+
{
|
|
168
|
+
input:
|
|
169
|
+
'The company was founded in 1995 by John Smith. It started with 10 employees and grew to 500 by 2020. The company is based in Seattle.',
|
|
170
|
+
},
|
|
171
|
+
],
|
|
172
|
+
onItemComplete: ({ scorerResults }) => {
|
|
173
|
+
console.log({
|
|
174
|
+
score: scorerResults[scorer.id].score,
|
|
175
|
+
reason: scorerResults[scorer.id].reason,
|
|
176
|
+
})
|
|
177
|
+
},
|
|
178
|
+
})
|
|
179
|
+
|
|
180
|
+
console.log(result.scores)
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
For more details on `runEvals`, see the [runEvals reference](https://mastra.ai/reference/evals/run-evals).
|
|
184
|
+
|
|
185
|
+
To add this scorer to an agent, see the [Scorers overview](https://mastra.ai/docs/evals/overview) guide.
|
|
186
|
+
|
|
187
|
+
## Comparison with faithfulness
|
|
188
|
+
|
|
189
|
+
| Use case | Summarization | Faithfulness |
|
|
190
|
+
| ------------------------- | ------------------------------- | --------------------------------- |
|
|
191
|
+
| **What it measures** | Support and coverage together | Support only |
|
|
192
|
+
| **Judged against** | The source text being condensed | Retrieved context or tool results |
|
|
193
|
+
| **Catches omission** | Yes | No |
|
|
194
|
+
| **Needs the full source** | Yes | No, context alone is enough |
|
|
195
|
+
|
|
196
|
+
Use `faithfulness` when the question is whether an answer stays grounded in retrieved context. Use `summarization` when the output is meant to stand in for a longer text.
|
|
197
|
+
|
|
198
|
+
## Related
|
|
199
|
+
|
|
200
|
+
- [Faithfulness Scorer](https://mastra.ai/reference/evals/faithfulness): Measures answer groundedness in context
|
|
201
|
+
- [Completeness Scorer](https://mastra.ai/reference/evals/completeness): Compares element coverage without a model
|
|
202
|
+
- [Content Similarity Scorer](https://mastra.ai/reference/evals/content-similarity): Compares text similarity without a model
|
|
203
|
+
- [Custom Scorers](https://mastra.ai/docs/evals/custom-scorers): Creating your own evaluation metrics
|
package/.docs/reference/index.md
CHANGED
|
@@ -152,6 +152,7 @@ The Reference section provides documentation of Mastra's API, including paramete
|
|
|
152
152
|
- [Noise Sensitivity Scorer](https://mastra.ai/reference/evals/noise-sensitivity)
|
|
153
153
|
- [Prompt Alignment Scorer](https://mastra.ai/reference/evals/prompt-alignment)
|
|
154
154
|
- [Rubric Scorer](https://mastra.ai/reference/evals/rubric)
|
|
155
|
+
- [Summarization Scorer](https://mastra.ai/reference/evals/summarization)
|
|
155
156
|
- [Textual Difference Scorer](https://mastra.ai/reference/evals/textual-difference)
|
|
156
157
|
- [Tone Consistency Scorer](https://mastra.ai/reference/evals/tone-consistency)
|
|
157
158
|
- [Tool Call Accuracy Scorers](https://mastra.ai/reference/evals/tool-call-accuracy)
|
|
@@ -27,7 +27,7 @@ export const agent = new Agent({
|
|
|
27
27
|
|
|
28
28
|
## Configuration
|
|
29
29
|
|
|
30
|
-
The `observationalMemory` option accepts `true`, a configuration object, or `false`. Setting `true` enables OM with `google/gemini-2.5-flash` as the default model. When passing a config object,
|
|
30
|
+
The `observationalMemory` option accepts `true`, a configuration object, or `false`. Setting `true` enables OM with `google/gemini-2.5-flash` as the default model. When passing a config object, set `model` at the top level or on `observation.model` and/or `reflection.model`; when all model fields are omitted, OM falls back to `google/gemini-2.5-flash`.
|
|
31
31
|
|
|
32
32
|
Observer input is multimodal-aware. OM keeps text placeholders like `[Image #1: screenshot.png]` in the transcript it builds for the Observer, and also sends the underlying image parts when possible. This applies to both single-thread observation and batched multi-thread observation. Non-image files appear as placeholders only.
|
|
33
33
|
|
|
@@ -35,7 +35,7 @@ OM performs thresholding with fast local token estimation. Text uses `tokenx`, a
|
|
|
35
35
|
|
|
36
36
|
**enabled** (`boolean`): Enable or disable Observational Memory. When omitted from a config object, defaults to true. Only enabled: false explicitly disables it. (Default: `true`)
|
|
37
37
|
|
|
38
|
-
**model** (`string | LanguageModel | DynamicModel | ModelByInputTokens | ModelWithRetries[]`): Model for both the Observer and Reflector agents. Sets the model for both at once. Cannot be used together with observation.model or reflection.model — an error will be thrown if both are set. When
|
|
38
|
+
**model** (`string | LanguageModel | DynamicModel | ModelByInputTokens | ModelWithRetries[]`): Model for both the Observer and Reflector agents. Sets the model for both at once. Cannot be used together with observation.model or reflection.model — an error will be thrown if both are set. When this and observation.model/reflection.model are all omitted, OM falls back to google/gemini-2.5-flash. Use "default" to explicitly use the default model (google/gemini-2.5-flash). (Default: `'google/gemini-2.5-flash'`)
|
|
39
39
|
|
|
40
40
|
**scope** (`'resource' | 'thread'`): Memory scope for observations. 'thread' keeps observations per-thread. 'resource' (experimental) shares observations across all threads for a resource, enabling cross-conversation memory. (Default: `'thread'`)
|
|
41
41
|
|
|
@@ -47,7 +47,9 @@ OM performs thresholding with fast local token estimation. Text uses `tokenx`, a
|
|
|
47
47
|
|
|
48
48
|
**temporalMarkers** (`boolean`): Insert temporal-gap reminder markers before new user messages when the previous message in the thread is at least 10 minutes older. The marker is persisted in memory, emitted as an inline reminder event so clients can render it specially, and shown to the observer so it can anchor observations to when events occurred. (Default: `false`)
|
|
49
49
|
|
|
50
|
-
**retrieval** (`boolean | { vector?: boolean; scope?: 'thread' | 'resource' }`):
|
|
50
|
+
**retrieval** (`boolean | { vector?: boolean; scope?: 'thread' | 'resource' }`): Let the agent look up the raw message history behind its observations. Observation groups keep durable pointers to the original messages, and a recall tool is registered so the agent can browse them. true enables cross-thread browsing by default. { vector: true } also enables semantic search using Memory's vector store and embedder. { scope: 'thread' } restricts the recall tool to the current thread only. Default scope is 'resource'. (Default: `false`)
|
|
51
|
+
|
|
52
|
+
**hooks** (`ObserveHooks`): Lifecycle hooks fired for every observation/reflection cycle — the manual observe()/reflect() APIs, turn-driven synchronous observation, and fire-and-forget async buffering. Callbacks receive threadId/resourceId/trigger call context ('manual' | 'turn-sync' | 'async-buffer'), and the end hooks (onObservationEnd/onReflectionEnd) additionally receive the OM model call's token usage and providerMetadata (where providers such as the AI Gateway report per-call cost), so apps can account for OM model spend without wrapping the observer/reflector models in middleware. Failed async-buffered cycles never throw; they report through the end hook's error field. Errors thrown by these hooks are caught and logged — they never fail the cycle.
|
|
51
53
|
|
|
52
54
|
**observation** (`ObservationalMemoryObservationConfig`): Configuration for the observation step. Controls when the Observer agent runs and how it behaves.
|
|
53
55
|
|
|
@@ -59,29 +61,33 @@ OM performs thresholding with fast local token estimation. Text uses `tokenx`, a
|
|
|
59
61
|
|
|
60
62
|
**observation.extract** (`Extractor[]`): Custom values to extract after observation. Schema-less extractors are requested inline in the Observer output. Schema-backed extractors run as a follow-up structured output call and are stored in thread OM metadata.
|
|
61
63
|
|
|
62
|
-
**observation.
|
|
64
|
+
**observation.manageWorkingMemory** (`boolean`): Let the Observer manage working memory through OM extraction. Adds WorkingMemoryExtractor, defaults workingMemory.agentManaged to false, and defaults workingMemory.useStateSignals to true. See Working memory updates.
|
|
65
|
+
|
|
66
|
+
**observation.observeAttachments** (`'auto' | boolean | string[]`): Controls which image/file attachments are forwarded to the Observer model alongside their placeholder text lines. true (default) forwards all attachments. false drops all attachments while keeping placeholders visible. 'auto' uses the provider capabilities registry to decide: attachments are forwarded when the Observer model supports multimodal input, dropped otherwise, and forwarded when no capability data is available for the model. An array is a case-insensitive mimeType allowlist supporting exact matches ('application/pdf'), wildcard subtypes ('image/\*'), and bare '\*' for everything. Useful when the Observer model is text-only (e.g. some DeepSeek endpoints) while the main agent uses a multimodal model. Tool-result attachments are filtered using the same rule.
|
|
63
67
|
|
|
64
68
|
**observation.messageTokens** (`number`): Token count of unobserved messages that triggers observation. When unobserved message tokens exceed this threshold, the Observer agent is called. Text is estimated locally with tokenx. Image parts are included with model-aware heuristics when possible, with deterministic fallbacks when image metadata is incomplete. Image-like file parts are counted the same way when uploads are normalized as files.
|
|
65
69
|
|
|
66
70
|
**observation.maxTokensPerBatch** (`number`): Maximum tokens per batch when observing multiple threads in resource scope. Threads are chunked into batches of this size and processed in parallel. Lower values mean more parallelism but more API calls.
|
|
67
71
|
|
|
68
|
-
**observation.modelSettings** (`ObservationalMemoryModelSettings`): Model settings for the Observer agent.
|
|
72
|
+
**observation.modelSettings** (`ObservationalMemoryModelSettings`): Model settings for the Observer agent. The maxOutputTokens: 100\_000 default is only applied with default model selection (no model set, "default", or a ModelByInputTokens selector). Custom models get no maxOutputTokens default.
|
|
69
73
|
|
|
70
74
|
**observation.modelSettings.temperature** (`number`): Temperature for generation. Lower values produce more consistent output.
|
|
71
75
|
|
|
72
|
-
**observation.modelSettings.maxOutputTokens** (`number`): Maximum output tokens. Set high to prevent truncation of observations.
|
|
76
|
+
**observation.modelSettings.maxOutputTokens** (`number`): Maximum output tokens. Set high to prevent truncation of observations. The 100000 default is only applied with default model selection; custom models get no default.
|
|
73
77
|
|
|
74
|
-
**observation.
|
|
78
|
+
**observation.providerOptions** (`ProviderOptions`): Provider-specific options passed to the Observer agent, such as Google thinking configuration.
|
|
79
|
+
|
|
80
|
+
**observation.bufferTokens** (`number | false`): How often background observation buffering runs. Values between 0 and 1 are fractions of messageTokens: 0.25 buffers every 25% of the threshold (7.5k tokens with the default 30k). Values of 1 or more are absolute token counts: 5000 buffers every 5k tokens. Buffered observations are stored until the messageTokens threshold is reached, then activate instantly without a blocking LLM call. Must resolve to less than messageTokens. Set to false to disable all async buffering (both observation and reflection).
|
|
75
81
|
|
|
76
82
|
**observation.bufferOnIdle** (`boolean`): Run background observation buffering when an agent turn ends and the agent becomes idle. This is separate from bufferTokens, which controls step-time async buffering. Set this to true to buffer short idle turns without waiting for the next turn or the messageTokens threshold.
|
|
77
83
|
|
|
78
|
-
**observation.bufferActivation** (`number`):
|
|
84
|
+
**observation.bufferActivation** (`number`): How much of the message window to clear when buffered observations activate. Values between 0 and 1 are the fraction of messageTokens to remove: 0.8 removes \~80% of the message history and keeps \~20% (6k tokens with the default 30k). Values of 1000 or more are the token count to keep: 4000 keeps \~4k message tokens after activation. Note the direction flips: a higher ratio removes more history, while a higher token count keeps more.
|
|
79
85
|
|
|
80
|
-
**observation.activateAfterIdle** (`number | string | false | "auto"`): Time before buffered observations are forced to activate after inactivity. Accepts milliseconds, a duration string, "auto" for a provider-aware prompt cache TTL, or false. If unset, the top-level activateAfterIdle value is used for observations. Set false to disable the top-level idle setting for observations.
|
|
86
|
+
**observation.activateAfterIdle** (`number | string | false | "auto"`): Time before buffered observations are forced to activate after inactivity. Accepts milliseconds, a duration string, "auto" for a provider-aware prompt cache TTL, or false. If unset, the top-level activateAfterIdle value is used for observations. Set false to disable the top-level idle setting for observations. Currently only applied when using the standalone ObservationalMemory class; new Memory(...) applies the top-level activateAfterIdle only.
|
|
81
87
|
|
|
82
|
-
**observation.activateOnProviderChange** (`boolean`): Force buffered observations to activate when the actor provider or model changes. If unset, the top-level activateOnProviderChange value is used for observations.
|
|
88
|
+
**observation.activateOnProviderChange** (`boolean`): Force buffered observations to activate when the actor provider or model changes. If unset, the top-level activateOnProviderChange value is used for observations. Currently only applied when using the standalone ObservationalMemory class; new Memory(...) applies the top-level activateOnProviderChange only.
|
|
83
89
|
|
|
84
|
-
**observation.blockAfter** (`number`):
|
|
90
|
+
**observation.blockAfter** (`number`): Safety net that forces a synchronous (blocking) observation when background buffering can't keep up. Values from 1 up to (but not including) 100 are multipliers of messageTokens: 1.2 forces a blocking observation at 120% of the threshold (36k tokens with the default 30k). Values of 100 or more are absolute token counts and must be greater than messageTokens. Between messageTokens and blockAfter, only async buffering and activation run; buffered activation still preserves a minimum remaining context (the smaller of 1000 tokens or the retention floor). Only relevant when bufferTokens is set. Defaults to 1.2 when async buffering is enabled.
|
|
85
91
|
|
|
86
92
|
**observation.previousObserverTokens** (`number | false`): Optional token budget for the observer's previous-observations context. When set to a number, the observations passed to the Observer agent are tail-truncated to fit within this budget while keeping the newest observations and preserving highlighted 🔴 items when possible. When a buffered reflection is pending, the already-reflected observation lines are automatically replaced with the reflection summary before truncation. Set to 0 to omit previous observations entirely, or false to disable truncation explicitly.
|
|
87
93
|
|
|
@@ -95,19 +101,21 @@ OM performs thresholding with fast local token estimation. Text uses `tokenx`, a
|
|
|
95
101
|
|
|
96
102
|
**reflection.observationTokens** (`number`): Token count of observations that triggers reflection. When observation tokens exceed this threshold, the Reflector agent is called to condense them.
|
|
97
103
|
|
|
98
|
-
**reflection.modelSettings** (`ObservationalMemoryModelSettings`): Model settings for the Reflector agent.
|
|
104
|
+
**reflection.modelSettings** (`ObservationalMemoryModelSettings`): Model settings for the Reflector agent. The maxOutputTokens: 100\_000 default is only applied with default model selection (no model set, "default", or a ModelByInputTokens selector). Custom models get no maxOutputTokens default.
|
|
99
105
|
|
|
100
106
|
**reflection.modelSettings.temperature** (`number`): Temperature for generation. Lower values produce more consistent output.
|
|
101
107
|
|
|
102
|
-
**reflection.modelSettings.maxOutputTokens** (`number`): Maximum output tokens. Set high to prevent truncation of observations.
|
|
108
|
+
**reflection.modelSettings.maxOutputTokens** (`number`): Maximum output tokens. Set high to prevent truncation of observations. The 100000 default is only applied with default model selection; custom models get no default.
|
|
109
|
+
|
|
110
|
+
**reflection.providerOptions** (`ProviderOptions`): Provider-specific options passed to the Reflector agent, such as Google thinking configuration.
|
|
103
111
|
|
|
104
|
-
**reflection.bufferActivation** (`number`):
|
|
112
|
+
**reflection.bufferActivation** (`number`): When background reflection starts, as a ratio (0-1) of observationTokens: 0.5 starts reflecting in the background once observations reach 50% of the threshold (20k tokens with the default 40k). When the full threshold is reached, the buffered reflection replaces the observations it covers, preserving any new observations appended after that range.
|
|
105
113
|
|
|
106
|
-
**reflection.activateAfterIdle** (`number | string | false | "auto"`): Time before buffered reflections are forced to activate after inactivity. Accepts milliseconds, a duration string, "auto" for a provider-aware prompt cache TTL, or false. Reflections do not inherit top-level activateAfterIdle; set this explicitly to opt reflections into idle activation.
|
|
114
|
+
**reflection.activateAfterIdle** (`number | string | false | "auto"`): Time before buffered reflections are forced to activate after inactivity. Accepts milliseconds, a duration string, "auto" for a provider-aware prompt cache TTL, or false. Reflections do not inherit top-level activateAfterIdle; set this explicitly to opt reflections into idle activation. Currently only applied when using the standalone ObservationalMemory class; this setting has no effect through new Memory(...).
|
|
107
115
|
|
|
108
|
-
**reflection.activateOnProviderChange** (`boolean`): Force buffered reflections to activate when the actor provider or model changes. Reflections do not inherit top-level activateOnProviderChange; set this explicitly to opt reflections into provider-change activation.
|
|
116
|
+
**reflection.activateOnProviderChange** (`boolean`): Force buffered reflections to activate when the actor provider or model changes. Reflections do not inherit top-level activateOnProviderChange; set this explicitly to opt reflections into provider-change activation. Currently only applied when using the standalone ObservationalMemory class; this setting has no effect through new Memory(...).
|
|
109
117
|
|
|
110
|
-
**reflection.blockAfter** (`number`):
|
|
118
|
+
**reflection.blockAfter** (`number`): Safety net that forces a synchronous (blocking) reflection when background reflection can't keep up. Values from 1 up to (but not including) 100 are multipliers of observationTokens: 1.2 forces a blocking reflection at 120% of the threshold (48k tokens with the default 40k). Values of 100 or more are absolute token counts and must be greater than observationTokens. Between observationTokens and blockAfter, only async buffering and activation run. Only relevant when bufferActivation is set. Defaults to 1.2 when async reflection is enabled.
|
|
111
119
|
|
|
112
120
|
### Token estimate metadata cache
|
|
113
121
|
|
|
@@ -150,7 +158,7 @@ const memory = new Memory({
|
|
|
150
158
|
|
|
151
159
|
**name** (`string`): Human-readable extractor name. OM slugifies this value into the extractor slug. Names must be unique after slug generation.
|
|
152
160
|
|
|
153
|
-
**slug** (`string`): Generated stable identifier for persisted values and XML tags. Slugs use lowercase letters, numbers, and hyphens. Built-in slugs and reserved XML tags cannot be used by custom extractors.
|
|
161
|
+
**slug** (`string`): Read-only property derived from name — not a constructor option. Generated stable identifier for persisted values and XML tags. Slugs use lowercase letters, numbers, and hyphens. Built-in slugs and reserved XML tags cannot be used by custom extractors.
|
|
154
162
|
|
|
155
163
|
**instructions** (`string | (context) => string`): Instructions for what to extract and when to update the value. Use a function to derive instructions from runtime context.
|
|
156
164
|
|
|
@@ -158,6 +166,8 @@ const memory = new Memory({
|
|
|
158
166
|
|
|
159
167
|
**includePreviousExtraction** (`boolean`): Controls whether the previous extraction is shown to the extractor on future OM runs. Set to false for values that should only come from the current OM run. (Default: `true`)
|
|
160
168
|
|
|
169
|
+
**metadataKeyPath** (`string | false`): Dot-separated OM metadata path used to persist the extracted value. Set to false to skip OM metadata persistence entirely. (Default: `'extracted.<slug>'`)
|
|
170
|
+
|
|
161
171
|
**onExtracted** (`(context) => T | void | Promise<T | void>`): Optional hook called after a custom extractor returns a value and before metadata is persisted. Returning a value replaces the extracted value. Throwing records an extraction failure.
|
|
162
172
|
|
|
163
173
|
### Extraction behavior
|
|
@@ -663,18 +673,45 @@ Emitted when buffered observations or reflections are activated (moved into the
|
|
|
663
673
|
|
|
664
674
|
**observations** (`string`): The activated observations text.
|
|
665
675
|
|
|
676
|
+
**triggeredBy** (`'threshold' | 'ttl' | 'provider_change'`): Whether activation was triggered by threshold crossing, activateAfterIdle expiry, or a model/provider change.
|
|
677
|
+
|
|
678
|
+
**lastActivityAt** (`number`): Unix-ms timestamp of the last assistant message part used for TTL checks.
|
|
679
|
+
|
|
680
|
+
**ttlExpiredMs** (`number`): How long activateAfterIdle had been exceeded when activation fired.
|
|
681
|
+
|
|
682
|
+
**previousModel** (`string`): Previous assistant model identifier that triggered activation (e.g. openai/gpt-4o).
|
|
683
|
+
|
|
684
|
+
**currentModel** (`string`): Current actor model identifier that triggered activation.
|
|
685
|
+
|
|
666
686
|
**recordId** (`string`): The OM record ID.
|
|
667
687
|
|
|
668
688
|
**threadId** (`string`): This thread's ID.
|
|
669
689
|
|
|
670
690
|
**config** (`ObservationMarkerConfig`): Snapshot of config at activation time.
|
|
671
691
|
|
|
692
|
+
### `data-om-thread-update`
|
|
693
|
+
|
|
694
|
+
Emitted when the Observer updates the thread title. Only emitted when `observation.threadTitle` is enabled.
|
|
695
|
+
|
|
696
|
+
**cycleId** (`string`): Unique ID for this observation cycle — shared with observation markers.
|
|
697
|
+
|
|
698
|
+
**threadId** (`string`): The thread ID that was updated.
|
|
699
|
+
|
|
700
|
+
**oldTitle** (`string`): The previous thread title. Undefined if the thread had no title.
|
|
701
|
+
|
|
702
|
+
**newTitle** (`string`): The new thread title.
|
|
703
|
+
|
|
704
|
+
**timestamp** (`string`): When this update occurred.
|
|
705
|
+
|
|
672
706
|
## Standalone usage
|
|
673
707
|
|
|
674
708
|
Most users should use the `Memory` class above. Using `ObservationalMemory` directly is mainly useful for benchmarking, experimentation, or when you need to control processor ordering with other processors (like [guardrails](https://mastra.ai/docs/agents/guardrails)).
|
|
675
709
|
|
|
710
|
+
The `ObservationalMemory` class is the engine; to attach it to an agent, wrap it in an `ObservationalMemoryProcessor`, which needs a `Memory` instance for loading and persisting messages. Note that `stores.memory` is typed as optional on storage adapters, so a non-null assertion (or a runtime check) is needed:
|
|
711
|
+
|
|
676
712
|
```typescript
|
|
677
|
-
import { ObservationalMemory } from '@mastra/memory/processors'
|
|
713
|
+
import { ObservationalMemory, ObservationalMemoryProcessor } from '@mastra/memory/processors'
|
|
714
|
+
import { Memory } from '@mastra/memory'
|
|
678
715
|
import { Agent } from '@mastra/core/agent'
|
|
679
716
|
import { LibSQLStore } from '@mastra/libsql'
|
|
680
717
|
|
|
@@ -683,8 +720,11 @@ const storage = new LibSQLStore({
|
|
|
683
720
|
url: 'file:./memory.db',
|
|
684
721
|
})
|
|
685
722
|
|
|
723
|
+
const memory = new Memory({ storage })
|
|
724
|
+
|
|
686
725
|
const om = new ObservationalMemory({
|
|
687
|
-
storage: storage.stores.memory
|
|
726
|
+
storage: storage.stores.memory!,
|
|
727
|
+
memory,
|
|
688
728
|
model: 'google/gemini-2.5-flash',
|
|
689
729
|
scope: 'resource',
|
|
690
730
|
observation: {
|
|
@@ -695,13 +735,15 @@ const om = new ObservationalMemory({
|
|
|
695
735
|
},
|
|
696
736
|
})
|
|
697
737
|
|
|
738
|
+
const omProcessor = new ObservationalMemoryProcessor(om, memory)
|
|
739
|
+
|
|
698
740
|
export const agent = new Agent({
|
|
699
741
|
id: 'my-agent',
|
|
700
742
|
name: 'my-agent',
|
|
701
743
|
instructions: 'You are a helpful assistant.',
|
|
702
744
|
model: 'openai/gpt-5-mini',
|
|
703
|
-
inputProcessors: [
|
|
704
|
-
outputProcessors: [
|
|
745
|
+
inputProcessors: [omProcessor],
|
|
746
|
+
outputProcessors: [omProcessor],
|
|
705
747
|
})
|
|
706
748
|
```
|
|
707
749
|
|
|
@@ -725,9 +767,11 @@ When `retrieval` is set (any truthy value), a `recall` tool is registered so the
|
|
|
725
767
|
|
|
726
768
|
**query** (`string`): Search query for mode: "search". Finds messages semantically similar to this text across all threads for the current user.
|
|
727
769
|
|
|
728
|
-
**cursor** (`string`): A message ID to anchor the recall query.
|
|
770
|
+
**cursor** (`string`): A message ID to anchor the recall query. Extract the start or end ID from an observation group range (e.g. from \_range: \startId:endId\\\_, use either startId or endId). If a range string is passed directly, the tool returns a hint explaining how to extract the correct ID. When both cursor and threadId are omitted for mode: "messages", the tool browses the current thread from the position set by anchor.
|
|
771
|
+
|
|
772
|
+
**threadId** (`string`): Browse a different thread by its ID, or pass "current" for the active thread. Use mode: "threads" first to discover thread IDs. When provided without a cursor, reading starts from the beginning of the thread.
|
|
729
773
|
|
|
730
|
-
**
|
|
774
|
+
**anchor** (`'start' | 'end'`): For mode: "messages" without a cursor, page from the start (oldest-first) or end (newest-first) of the thread. (Default: `'start'`)
|
|
731
775
|
|
|
732
776
|
**page** (`number`): Pagination offset. For messages: positive values page forward from cursor, negative values page backward. For threads: page number (0-indexed). 0 is treated as 1 for messages. (Default: `1`)
|
|
733
777
|
|
|
@@ -735,6 +779,10 @@ When `retrieval` is set (any truthy value), a `recall` tool is registered so the
|
|
|
735
779
|
|
|
736
780
|
**detail** (`'low' | 'high'`): Controls how much content is shown per message part. 'low' shows truncated text and tool names with positional indices (\[p0], \[p1]). 'high' shows full content including tool arguments and results, clamped to one part per call with continuation hints. (Default: `'low'`)
|
|
737
781
|
|
|
782
|
+
**partType** (`'text' | 'tool-call' | 'tool-result' | 'reasoning' | 'image' | 'file'`): Filter results to only include message parts of this type. Only applies to mode: "messages".
|
|
783
|
+
|
|
784
|
+
**toolName** (`string`): Filter results to only include tool-call and tool-result parts matching this tool name. Only applies to mode: "messages".
|
|
785
|
+
|
|
738
786
|
**partIndex** (`number`): Fetch a single message part at full detail by its positional index. Use this when a low-detail recall shows an interesting part at \[p1] — call again with partIndex: 1 to see the full content without loading every part.
|
|
739
787
|
|
|
740
788
|
**before** (`string`): For mode: "threads" only. Filter to threads created before this date. Accepts ISO 8601 format (e.g. "2026-03-15", "2026-03-10T00:00:00Z").
|
|
@@ -753,6 +801,8 @@ When `retrieval` is set (any truthy value), a `recall` tool is registered so the
|
|
|
753
801
|
|
|
754
802
|
**limit** (`number`): The limit used for this query.
|
|
755
803
|
|
|
804
|
+
**detail** (`'low' | 'high'`): The detail level used for this query.
|
|
805
|
+
|
|
756
806
|
**hasNextPage** (`boolean`): Whether more messages exist after this page.
|
|
757
807
|
|
|
758
808
|
**hasPrevPage** (`boolean`): Whether more messages exist before this page.
|
|
@@ -473,6 +473,7 @@ interface ToolCallAttributes {
|
|
|
473
473
|
toolId?: string
|
|
474
474
|
toolType?: string
|
|
475
475
|
toolDescription?: string
|
|
476
|
+
toolCallId?: string
|
|
476
477
|
success?: boolean
|
|
477
478
|
}
|
|
478
479
|
```
|
|
@@ -495,6 +496,8 @@ interface MCPToolCallAttributes {
|
|
|
495
496
|
/** Tool description */
|
|
496
497
|
toolDescription?: string
|
|
497
498
|
|
|
499
|
+
toolCallId?: string
|
|
500
|
+
|
|
498
501
|
/** Whether tool execution was successful */
|
|
499
502
|
success?: boolean
|
|
500
503
|
}
|
|
@@ -192,6 +192,28 @@ try {
|
|
|
192
192
|
|
|
193
193
|
`code` and `proxyMessage` are `undefined` when the response body isn't JSON, for example an HTML 502 from a load balancer.
|
|
194
194
|
|
|
195
|
+
`executeCommand` runs over the direct-exec data plane (a WebSocket to the Railway tcp-proxy) and can also throw two typed sandbox errors on unrecoverable failure:
|
|
196
|
+
|
|
197
|
+
```typescript
|
|
198
|
+
import { SandboxDestroyedError, SandboxExecTransportError } from '@mastra/platform-workspace'
|
|
199
|
+
|
|
200
|
+
try {
|
|
201
|
+
await sandbox.executeCommand('pytest')
|
|
202
|
+
} catch (err) {
|
|
203
|
+
if (err instanceof SandboxDestroyedError) {
|
|
204
|
+
// /exec-lease returned 410; the sandbox has been destroyed.
|
|
205
|
+
// The cached sandbox id and lease have already been cleared,
|
|
206
|
+
// so reusing the instance will reprovision on the next call.
|
|
207
|
+
} else if (err instanceof SandboxExecTransportError) {
|
|
208
|
+
// Both the initial WebSocket attempt and the built-in retry
|
|
209
|
+
// closed without an exit frame against a live sandbox.
|
|
210
|
+
console.error(err.closeCode, err.closeReason, err.wsEndpoint)
|
|
211
|
+
}
|
|
212
|
+
}
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
`SandboxExecTransportError` carries diagnostic fields (`opened`, `closeCode`, `closeReason`, `wsEndpoint`, plus `sandboxId`, `command`, and `attempts`) so operators can distinguish a broken Railway data plane from a failed command.
|
|
216
|
+
|
|
195
217
|
## Related
|
|
196
218
|
|
|
197
219
|
- [PlatformFilesystem reference](https://mastra.ai/reference/workspace/platform-filesystem)
|
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,12 @@
|
|
|
1
1
|
# @mastra/mcp-docs-server
|
|
2
2
|
|
|
3
|
+
## 1.2.13-alpha.5
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- Updated dependencies [[`594f7b2`](https://github.com/mastra-ai/mastra/commit/594f7b28f5263fb9982fd50d95c471fb971ea984), [`311f943`](https://github.com/mastra-ai/mastra/commit/311f943bee60e8fdf5c84499ea50e884276c936c), [`0c89896`](https://github.com/mastra-ai/mastra/commit/0c8989673fb7d106837098398131e570c6023b68), [`23b4238`](https://github.com/mastra-ai/mastra/commit/23b423844ad0bcf2a502a68dd62866d6160f9f6d), [`e320a76`](https://github.com/mastra-ai/mastra/commit/e320a763feaf65c6be3cebecf746defcbde161b3), [`03b4918`](https://github.com/mastra-ai/mastra/commit/03b4918c80d188ce375334c393e131c6e94bd7eb), [`14ef73a`](https://github.com/mastra-ai/mastra/commit/14ef73a4bbd73e7808414816eb0628ce1d80b5d7), [`1d677d5`](https://github.com/mastra-ai/mastra/commit/1d677d5f99d7db403f7828585e8c25f299f72628), [`93e28ec`](https://github.com/mastra-ai/mastra/commit/93e28ecce9031c02397e0ae8406593e5c7a95883), [`729dab4`](https://github.com/mastra-ai/mastra/commit/729dab408faccfaef0cbb048e5a4338f9172847e), [`484003d`](https://github.com/mastra-ai/mastra/commit/484003d33ff59330c86b19863e4a38732d7e4155), [`933d291`](https://github.com/mastra-ai/mastra/commit/933d291146b789c19442ad206f94da3e4be90c64)]:
|
|
8
|
+
- @mastra/core@1.56.0-alpha.3
|
|
9
|
+
|
|
3
10
|
## 1.2.13-alpha.4
|
|
4
11
|
|
|
5
12
|
### Patch Changes
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@mastra/mcp-docs-server",
|
|
3
|
-
"version": "1.2.13-alpha.
|
|
3
|
+
"version": "1.2.13-alpha.6",
|
|
4
4
|
"description": "MCP server for accessing Mastra.ai documentation, changelogs, and news.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
"jsdom": "^26.1.0",
|
|
29
29
|
"local-pkg": "^1.1.2",
|
|
30
30
|
"zod": "^4.4.3",
|
|
31
|
-
"@mastra/core": "1.56.0-alpha.
|
|
31
|
+
"@mastra/core": "1.56.0-alpha.3",
|
|
32
32
|
"@mastra/mcp": "^1.15.0"
|
|
33
33
|
},
|
|
34
34
|
"devDependencies": {
|
|
@@ -46,8 +46,8 @@
|
|
|
46
46
|
"typescript": "^6.0.3",
|
|
47
47
|
"vitest": "4.1.10",
|
|
48
48
|
"@internal/lint": "0.0.119",
|
|
49
|
-
"@
|
|
50
|
-
"@
|
|
49
|
+
"@mastra/core": "1.56.0-alpha.3",
|
|
50
|
+
"@internal/types-builder": "0.0.94"
|
|
51
51
|
},
|
|
52
52
|
"homepage": "https://mastra.ai",
|
|
53
53
|
"repository": {
|