@mastra/mcp-docs-server 1.2.13-alpha.1 → 1.2.13-alpha.11

package/.docs/reference/editor/versioning.md

@@ -0,0 +1,130 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Versioning
+
+Editor versions stored agents and prompt blocks. Database-backed resources use draft and publish operations. Code-backed agent overrides use deterministic files and Git history.
+
+See [Editor versioning](https://mastra.ai/docs/editor/overview) for release and experimentation patterns.
+
+## Database lifecycle
+
+The resource record stores an `activeVersionId`. Individual snapshots don't store a lifecycle status.
+
+| Term       | Meaning                                                                                      |
+| ---------- | -------------------------------------------------------------------------------------------- |
+| Latest     | The most recently created configuration snapshot                                             |
+| Published  | The snapshot selected by `activeVersionId`                                                   |
+| Draft      | The latest snapshot when it differs from `activeVersionId`, or when no active version exists |
+| Historical | Any other retained snapshot                                                                  |
+
+Saving changed snapshot fields creates a new latest version. Saving identical snapshot fields or changing metadata alone doesn't create a version.
+
+If an active version exists, creating a draft doesn't change the version handling published requests. Publishing updates `activeVersionId`. Restoring a historical version copies its configuration into a new inactive draft.
+
+The direct namespace methods and REST APIs differ in one important way. `editor.prompt.update()` creates an inactive draft. `editor.agent.update()` creates a version and immediately assigns it to `activeVersionId`. The stored-agent REST `PATCH` route creates an inactive draft unless `autoPublish` is enabled.
+
+When a generic stored resource has no active version, published resolution can fall back to the latest snapshot. For a code-defined agent override, requesting `status: 'published'` without an active override returns the original code agent.
+
+## Code source
+
+With `source: 'code'`, active agent overrides are serialized as deterministic JSON under `<codePath>/agents/<encodedAgentId>.json`. The default `codePath` is `./mastra/editor`.
+
+The initial agent is published to create the file. Later server updates create a draft by default and write the file when that version becomes active. Git versions are read-only in Studio and use the commit message as their change message. The default history scan reads up to 50 recent commits and skips consecutive commits whose parsed JSON snapshot is unchanged.
+
+See [`MastraEditor`](https://mastra.ai/reference/editor/mastra-editor) for source options.
+
+## Select an agent version
+
+Calling [`mastra.getAgentById()`](https://mastra.ai/reference/core/getAgentById) without a selector returns the registered code-defined agent. Pass `status` or `versionId` to apply a stored override. See [Select a version](https://mastra.ai/docs/editor/overview) for a TypeScript example.
+
+With the default server prefix, pass selectors as query parameters under `/api`:
+
+```bash
+# Published version
+curl http://localhost:4111/api/agents/support-agent
+
+# Latest draft
+curl http://localhost:4111/api/agents/support-agent?status=draft
+
+# Exact version
+curl http://localhost:4111/api/agents/support-agent?versionId=abc-123
+```
+
+See the [Client SDK agents reference](https://mastra.ai/reference/client-js/agents) for Client SDK and React SDK selectors.
+
+## Sub-agent versioning
+
+Version overrides propagate through [supervisor-agent delegation](https://mastra.ai/docs/agents/supervisor-agents) in request context. Define selectors at three levels:
+
+1. `Mastra` instance `versions`: Defaults for every invocation
+2. Server request-body `versions`: Per-request values added to request context
+3. Direct `generate()` or `stream()` `versions`: Per-invocation values
+
+Entries merge by agent ID. For the same ID, precedence is **direct invocation > request body or existing request context > Mastra instance default**.
+
+Set defaults on the Mastra instance:
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { MastraEditor } from '@mastra/editor'
+
+export const mastra = new Mastra({
+  agents: { supervisor, researchAgent, writerAgent },
+  editor: new MastraEditor(),
+  versions: {
+    agents: {
+      'research-agent': { status: 'published' },
+      'writer-agent': { versionId: 'abc-123' },
+    },
+  },
+})
+```
+
+Override one sub-agent for a direct invocation:
+
+```typescript
+const result = await supervisor.generate('Research and write about AI safety', {
+  versions: {
+    agents: {
+      'research-agent': { status: 'draft' },
+    },
+  },
+})
+```
+
+Or pass selectors in the server request body:
+
+```bash
+curl -X POST http://localhost:4111/api/agents/supervisor/generate \
+  -H "Content-Type: application/json" \
+  -d '{
+    "messages": [{ "role": "user", "content": "Research AI safety" }],
+    "versions": {
+      "agents": {
+        "research-agent": { "status": "draft" }
+      }
+    }
+  }'
+```
+
+If Editor isn't configured or a version can't be resolved, Mastra logs a warning and uses the code-defined sub-agent.
+
+## Stored-agent REST API
+
+The default Mastra server prefix is `/api`. A custom server prefix changes the paths below.
+
+| Method   | Path                                           | Description                                                          |
+| -------- | ---------------------------------------------- | -------------------------------------------------------------------- |
+| `GET`    | `/api/stored/agents`                           | List stored agents                                                   |
+| `POST`   | `/api/stored/agents`                           | Create a stored agent                                                |
+| `GET`    | `/api/stored/agents/:storedAgentId`            | Get a stored agent                                                   |
+| `PATCH`  | `/api/stored/agents/:storedAgentId`            | Update a stored agent                                                |
+| `DELETE` | `/api/stored/agents/:storedAgentId`            | Delete a stored agent                                                |
+| `GET`    | `/api/stored/agents/:storedAgentId/dependents` | List readable dependents and count hidden cross-workspace references |
+| `POST`   | `/api/stored/agents/:storedAgentId/export`     | Export the allowed override fields as deterministic JSON             |
+
+The dependents response lists caller-readable agents by `id` and `name`. Its `hiddenCount` field counts cross-workspace references that the caller can't read, but only when the target agent is public.
+
+Version-management routes are nested under `/api/stored/agents/:storedAgentId/versions`. See [version management](https://mastra.ai/reference/client-js/agents) for operations and Client SDK methods.
+
+The Client SDK exposes `listStoredAgents()`, `createStoredAgent()`, and `getStoredAgent()`. The resource returned by `getStoredAgent(id)` includes methods for updates, deletion, dependents, export, and version management.

package/.docs/reference/evals/mastra-scorer.md

@@ -76,16 +76,19 @@ const result = await scorer.run({
 The optional `judge` record contains details about the judge model calls made by prompt-based scorer steps. Its known keys are `preprocess`, `analyze`, `generateScore`, and `generateReason`. Each key contains an ordered `executions` array.
 
 ```typescript
-interface ScorerJudgeExecutionSuccess {
-  status: 'success'
+interface ScorerJudgeExecutionBase {
   prompt: string
-  output: JSONValue
   judgeModelId: string
   judgeProvider?: string
-  usage: ScorerJudgeUsage
   attemptCount: number
   modelCallCount: number
   durationMs: number
+}
+
+interface ScorerJudgeExecutionSuccess extends ScorerJudgeExecutionBase {
+  status: 'success'
+  output: JSONValue
+  usage: ScorerJudgeUsage
   cost?: {
     amount: number
     unit: string
@@ -93,7 +96,20 @@ interface ScorerJudgeExecutionSuccess {
   }
 }
 
-type ScorerJudgeExecution = ScorerJudgeExecutionSuccess
+interface ScorerJudgeExecutionFailure extends ScorerJudgeExecutionBase {
+  status: 'failed'
+  output?: JSONValue
+  rawOutput?: string
+  usage?: ScorerJudgeUsage
+  finishReason?: string
+  error: {
+    name: string
+    message: string
+    code?: string
+  }
+}
+
+type ScorerJudgeExecution = ScorerJudgeExecutionSuccess | ScorerJudgeExecutionFailure
 
 interface ScorerJudgeUsage {
   inputTokens?: number
@@ -119,18 +135,62 @@ const execution = result.judge?.generateScore?.executions[0]
 
 console.log(execution?.status)
 console.log(execution?.judgeModelId)
-console.log(execution?.usage.totalTokens)
+console.log(execution?.usage?.totalTokens)
 console.log(execution?.durationMs)
 ```
 
-The `status` value describes the outcome of the logical prompt-step execution, not the quality of the evaluated response. A structured-output fallback that eventually succeeds creates one `success` execution with an `attemptCount` greater than one.
+The `status` value describes the outcome of the logical prompt-step execution, not the quality of the evaluated response. A structured-output fallback that eventually succeeds creates one `success` execution with an `attemptCount` greater than one. Exhausted attempts create one `failed` execution.
+
+Successful executions require validated `output` and normalized `usage`. Failed executions require an `error` summary and include only the evidence the runtime received. A failed execution includes `output` only when the output was validated before a later callback or orchestration failure. Mastra doesn't parse `rawOutput` to create `output`.
 
 `attemptCount` counts judge invocations, including a structured-output fallback. `modelCallCount` counts the completed model steps across those attempts. `durationMs` covers the full prompt-step execution.
 
-Function steps don't create `judge` entries. Usage in this record belongs to the scorer's judge model, not the agent or workflow being evaluated. The optional `cost` field is present only when the execution directly reports an authoritative cost, source, and unit.
+Function steps don't create `judge` entries. Usage in this record belongs to the scorer's judge model, not the agent or workflow being evaluated. Filter by `status` when aggregating successful executions. Include both statuses when aggregating all completed provider usage. The optional `cost` field is present only on successful executions that directly report an authoritative cost, source, and unit.
 
 Use Mastra metrics to query aggregate usage, latency, and estimated cost across scorer runs. The `judge` record describes one scorer run and doesn't query metrics or traces.
 
+### Failed runs
+
+A failed scorer stage still rejects the `.run()` promise. Catch `ScorerRunError` to inspect completed stages and any results they produced:
+
+```typescript
+import { ScorerRunError } from '@mastra/core/evals'
+
+try {
+  const result = await scorer.run({ input, output })
+  console.log(result.score)
+} catch (error) {
+  if (error instanceof ScorerRunError) {
+    console.log(error.failedStep)
+    console.log(error.completedSteps)
+    console.log(error.result?.score)
+
+    const failedExecution = error.result?.judge?.[error.failedStep]?.executions.find(
+      execution => execution.status === 'failed',
+    )
+    console.log(failedExecution?.error)
+  }
+
+  throw error
+}
+```
+
+`ScorerRunError` exposes these properties:
+
+**failedStep** (`ScorerStepName`): The scorer stage that failed.
+
+**completedSteps** (`ScorerStepName[]`): The scorer stages that completed before the failure, in execution order.
+
+**result** (`ScorerRunResultSnapshot | undefined`): Outputs from completed scorer stages and judge execution evidence from attempted prompt stages. This property is omitted when neither is available.
+
+The `result` snapshot contains completed stage outputs and judge execution evidence. For example, if `generateReason` fails after `generateScore` returns `0`, `error.result.score` is `0`, the `generateScore` execution has `status: 'success'`, and the `generateReason` execution has `status: 'failed'`. The run remains failed.
+
+A prompt failure can create `error.result` with only run identity, input, and a failed `judge` entry. A function stage that fails before producing a scorer field doesn't create a result.
+
+`JSON.stringify(error)` uses the standard `MastraError` serialization and omits `result`, including successful and failed judge evidence. Read `result` explicitly when you need scorer artifacts or raw failed output.
+
+An in-memory experiment result can keep a completed score or reason from a failed scorer along with `error`, `failedStep`, and `completedSteps`. The scorer is still treated as failed, and a recovered score isn't written to legacy successful-score storage.
+
 ## Step execution flow
 
 When you call `.run()`, the MastraScorer executes the defined steps in this order:

package/.docs/reference/evals/summarization.md

@@ -0,0 +1,203 @@
+> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
+
+# Summarization scorer
+
+The `createSummarizationScorer()` function creates a scorer that evaluates a summary on two axes: whether every claim it makes is supported by the source text, and whether it preserves the information the source states. The final score is the lower of the two, so a summary can't pass by being faithful but empty, or thorough but wrong.
+
+The summary is the agent's last message that carries text, and the source text defaults to the first user message of the run input. Pass `source` or `sourceExtractor` when the text being summarized lives somewhere else, such as a tool result.
+
+## Usage example
+
+Score a summary against the document it condenses.
+
+```typescript
+import { createSummarizationScorer } from '@mastra/evals/scorers/prebuilt'
+
+const scorer = createSummarizationScorer({
+  model: 'openai/gpt-5.6-sol',
+})
+
+const result = await scorer.run({
+  input: {
+    inputMessages: [{ id: '1', role: 'user', content: sourceDocument }],
+  },
+  output: [{ id: '2', role: 'assistant', content: summary }],
+})
+
+console.log(result.score)
+console.log(result.reason)
+```
+
+## Summarization evaluation
+
+Use this scorer when an agent condenses text:
+
+- Document and transcript summarization
+- Support thread and email digests
+- Any step that compresses a long input into a short output
+
+## Parameters
+
+**model** (`MastraModelConfig`): The language model to use for judging claims and coverage questions
+
+**options** (`SummarizationMetricOptions`): Configuration options for the scorer
+
+**options.source** (`string`): Text the summary is judged against. Defaults to the user message of the run input
+
+**options.sourceExtractor** (`(input, output) => string`): Function to derive the source text from the run input and output. Takes precedence over source
+
+**options.maxQuestions** (`number`): Upper bound on the coverage questions drawn from the source (default: 10)
+
+**options.scale** (`number`): Scale factor to multiply the final score (default: 1)
+
+## `.run()` returns
+
+**score** (`number`): Summarization score between 0 and scale (default 0-1), the lower of the alignment and coverage scores
+
+**reason** (`string`): Human-readable explanation naming the axis that produced the score and the claims or questions behind it. Both axis scores appear in the text
+
+**preprocessStepResult** (`object`): The alignment verdicts and the questions drawn from the source
+
+**preprocessStepResult.alignment** (`{ claim: string; supported: boolean; reason: string }[]`): One verdict per claim the summary makes
+
+**preprocessStepResult.questions** (`string[]`): The coverage questions drawn from the source text
+
+**analyzeStepResult** (`object`): The coverage verdicts
+
+**analyzeStepResult.coverage** (`{ question: string; answered: boolean; reason: string }[]`): One verdict per question, answered from the summary alone
+
+The axis scores are derived from these verdicts rather than stored: alignment is the share of `alignment` entries with `supported: true`, and coverage is the share of `questions` whose `coverage` entry has `answered: true`.
+
+## Scoring details
+
+### Two-axis evaluation
+
+The scorer runs a three-step pipeline:
+
+1. **Source judgement**: the claims the summary makes are extracted and checked against the source, and closed-ended questions are drawn from the source. Every question is written so the source answers it "yes".
+2. **Coverage**: each question is answered using the summary alone.
+3. **Scoring**: the two ratios are computed and the lower one becomes the score.
+
+The coverage step runs as a separate model call that never receives the source text. A judge that could see the source would answer questions from it rather than from the summary, which would hide the omissions this axis exists to measure.
+
+### Scoring formula
+
+```text
+Alignment    = supported_claims / total_claims
+Coverage     = answered_questions / total_questions
+Summarization = min(Alignment, Coverage) × scale
+```
+
+The score is 0 when the summary yields no claims or the source yields no questions.
+
+### Score interpretation
+
+These ranges assume the default `scale` of 1. When using a custom scale, multiply accordingly.
+
+- **0.9-1.0**: Excellent summary, faithful to the source and covering its main points
+- **0.7-0.8**: Good summary with a small omission or an unsupported detail
+- **0.4-0.6**: Moderate summary, either missing important information or drifting from the source
+- **0.1-0.3**: Poor summary, most of the source is lost or contradicted
+- **0.0**: The summary produced nothing to judge, or it failed to support any claims. A summary that answers no questions also receives this score
+
+### Reading the two axes
+
+Both axes leave their verdicts on the run result: the alignment verdicts on the preprocess step, and the coverage verdicts on the analyze step. Each verdict carries the claim or question it belongs to and the reason behind it. A low alignment score has a different meaning from a low coverage score:
+
+- A low alignment score with high coverage means the summary invents or distorts detail
+- A low coverage score with high alignment means the summary is accurate but leaves too much out
+
+The reason field names whichever axis produced the score.
+
+### What the score leaves out
+
+Length plays no part in the score. A summary that repeats the source word for word supports every claim and answers every question, so it scores 1. Add a length check of your own when compression is part of what you're testing.
+
+### Cost
+
+Each evaluation makes three model calls. `maxQuestions` bounds the coverage half of the work, which otherwise grows with source length. Raise it for long documents where ten questions can't represent the content.
+
+## Scorer configuration
+
+### Summarizing the run input
+
+```typescript
+const scorer = createSummarizationScorer({
+  model: 'openai/gpt-5.6-sol',
+})
+```
+
+### Summarizing a document from elsewhere
+
+```typescript
+import { extractToolResults } from '@mastra/evals/scorers/utils'
+
+const scorer = createSummarizationScorer({
+  model: 'openai/gpt-5.6-sol',
+  options: {
+    sourceExtractor: (input, output) => {
+      return extractToolResults(output)
+        .filter(({ toolName }) => toolName === 'fetchDocument')
+        .map(({ result }) => String(result))
+        .join('\n\n')
+    },
+    maxQuestions: 20,
+  },
+})
+```
+
+## Example
+
+Evaluate a summarization agent against a set of documents:
+
+```typescript
+import { runEvals } from '@mastra/core/evals'
+import { createSummarizationScorer } from '@mastra/evals/scorers/prebuilt'
+import { summarizerAgent } from './agent'
+
+const scorer = createSummarizationScorer({
+  model: 'openai/gpt-5.6-sol',
+  options: { maxQuestions: 10 },
+})
+
+const result = await runEvals({
+  target: summarizerAgent,
+  scorers: [scorer],
+  data: [
+    {
+      input:
+        'The company was founded in 1995 by John Smith. It started with 10 employees and grew to 500 by 2020. The company is based in Seattle.',
+    },
+  ],
+  onItemComplete: ({ scorerResults }) => {
+    console.log({
+      score: scorerResults[scorer.id].score,
+      reason: scorerResults[scorer.id].reason,
+    })
+  },
+})
+
+console.log(result.scores)
+```
+
+For more details on `runEvals`, see the [runEvals reference](https://mastra.ai/reference/evals/run-evals).
+
+To add this scorer to an agent, see the [Scorers overview](https://mastra.ai/docs/evals/overview) guide.
+
+## Comparison with faithfulness
+
+| Use case                  | Summarization                   | Faithfulness                      |
+| ------------------------- | ------------------------------- | --------------------------------- |
+| **What it measures**      | Support and coverage together   | Support only                      |
+| **Judged against**        | The source text being condensed | Retrieved context or tool results |
+| **Catches omission**      | Yes                             | No                                |
+| **Needs the full source** | Yes                             | No, context alone is enough       |
+
+Use `faithfulness` when the question is whether an answer stays grounded in retrieved context. Use `summarization` when the output is meant to stand in for a longer text.
+
+## Related
+
+- [Faithfulness Scorer](https://mastra.ai/reference/evals/faithfulness): Measures answer groundedness in context
+- [Completeness Scorer](https://mastra.ai/reference/evals/completeness): Compares element coverage without a model
+- [Content Similarity Scorer](https://mastra.ai/reference/evals/content-similarity): Compares text similarity without a model
+- [Custom Scorers](https://mastra.ai/docs/evals/custom-scorers): Creating your own evaluation metrics

package/.docs/reference/file-based-agents/config.md

@@ -94,4 +94,42 @@ Keep `config.ts` focused on runtime options. Use sibling files for concerns that
 | Memory       | `config.memory`               | [`memory.ts`](https://mastra.ai/reference/file-based-agents/memory)             | `config.memory`                                         |
 | Workspace    | `config.workspace`            | [`workspace.ts`](https://mastra.ai/reference/file-based-agents/workspace)       | `config.workspace`                                      |
 
-Missing both `instructions.md` and `config.instructions` fails the build. Missing both `config.memory` and `memory.ts` leaves the agent without memory.
+Missing both `instructions.md` and `config.instructions` fails the build. Missing both `config.memory` and `memory.ts` leaves the agent without memory.
+
+## Discovery lifecycle
+
+File-based primitives are discovered by the Mastra bundler under `mastra dev` and `mastra build`. During discovery, Mastra reads supported files under `src/mastra/`, imports TypeScript and JavaScript modules, reads markdown instructions and skills, copies workspace seed files, and registers the assembled primitives with your Mastra app.
+
+After discovery, a file-based agent runs as a normal [`Agent`](https://mastra.ai/reference/agents/agent). Calling it from the Agent API, Studio, workflows, or your application code uses the same runtime as a code-defined agent.
+
+Discovery is source-based and conservative.
+
+It skips symlinks, test files, and directories that aren't agent directories. Workflows and singleton project files are file-routed only when they have a default export.
+
+Start your app through the Mastra CLI so discovery runs:
+
+**npm**:
+
+```bash
+npx mastra dev
+```
+
+**pnpm**:
+
+```bash
+pnpm dlx mastra dev
+```
+
+**Yarn**:
+
+```bash
+yarn dlx mastra dev
+```
+
+**Bun**:
+
+```bash
+bun x mastra dev
+```
+
+If you import your `mastra` instance directly, `agents/<name>/` directories and the other conventions aren't discovered. When you consume Mastra as a library, register those primitives in code instead.

package/.docs/reference/index.md

@@ -65,7 +65,6 @@ The Reference section provides documentation of Mastra's API, including paramete
 - [SlackProvider](https://mastra.ai/reference/channels/slack-provider)
 - [create-mastra](https://mastra.ai/reference/cli/create-mastra)
 - [mastra](https://mastra.ai/reference/cli/mastra)
-- [Agent Builder API](https://mastra.ai/reference/client-js/agent-builder)
 - [Agents API](https://mastra.ai/reference/client-js/agents)
 - [Conversations API](https://mastra.ai/reference/client-js/conversations)
 - [Error Handling](https://mastra.ai/reference/client-js/error-handling)
@@ -123,10 +122,10 @@ The Reference section provides documentation of Mastra's API, including paramete
 - [BlobStoreProvider](https://mastra.ai/reference/editor/blob-store-provider)
 - [MastraEditor Class](https://mastra.ai/reference/editor/mastra-editor)
 - [ProcessorProvider](https://mastra.ai/reference/editor/processor-provider)
+- [Prompt blocks](https://mastra.ai/reference/editor/prompt-blocks)
+- [Tool configuration](https://mastra.ai/reference/editor/tools)
 - [ToolProvider](https://mastra.ai/reference/editor/tool-provider)
-- [AgentBuilderOptions](https://mastra.ai/reference/editor/agent-builder/agent-builder-options)
-- [BuilderAgentDefaults](https://mastra.ai/reference/editor/agent-builder/builder-agent-defaults)
-- [Models default](https://mastra.ai/reference/editor/agent-builder/builder-models)
+- [Versioning](https://mastra.ai/reference/editor/versioning)
 - [BrowserProvider](https://mastra.ai/reference/editor/browser-provider)
 - [StorageBrowserRef](https://mastra.ai/reference/editor/storage-browser-ref)
 - [FilesystemProvider](https://mastra.ai/reference/editor/filesystem-provider)
@@ -152,6 +151,7 @@ The Reference section provides documentation of Mastra's API, including paramete
 - [Noise Sensitivity Scorer](https://mastra.ai/reference/evals/noise-sensitivity)
 - [Prompt Alignment Scorer](https://mastra.ai/reference/evals/prompt-alignment)
 - [Rubric Scorer](https://mastra.ai/reference/evals/rubric)
+- [Summarization Scorer](https://mastra.ai/reference/evals/summarization)
 - [Textual Difference Scorer](https://mastra.ai/reference/evals/textual-difference)
 - [Tone Consistency Scorer](https://mastra.ai/reference/evals/tone-consistency)
 - [Tool Call Accuracy Scorers](https://mastra.ai/reference/evals/tool-call-accuracy)