@mastra/mcp-docs-server 1.1.9-alpha.0 → 1.1.9-alpha.2

package/.docs/reference/evals/create-scorer.md

@@ -2,7 +2,7 @@
 
 Mastra provides a unified `createScorer` factory that allows you to define custom scorers for evaluating input/output pairs. You can use either native JavaScript functions or LLM-based prompt objects for each evaluation step. Custom scorers can be added to Agents and Workflow steps.
 
-## How to Create a Custom Scorer
+## How to create a custom scorer
 
 Use the `createScorer` factory to define your scorer with a name, description, and optional judge configuration. Then chain step methods to build your evaluation pipeline. You must provide at least a `generateScore` step.
 
@@ -35,7 +35,7 @@ const scorer = createScorer({
   })
 ```
 
-## createScorer Options
+## `createScorer` options
 
 **id** (`string`): Unique identifier for the scorer. Used as the name if \`name\` is not provided.
 
@@ -57,7 +57,7 @@ The judge only runs for steps defined as **prompt objects** (`preprocess`, `anal
 
 When a prompt-object step runs, its structured LLM output is stored in the corresponding result field (`preprocessStepResult`, `analyzeStepResult`, or the value consumed by `calculateScore` in `generateScore`).
 
-## Type Safety
+## Type safety
 
 You can specify input/output types when creating scorers for better type inference and IntelliSense support:
 
@@ -113,7 +113,7 @@ const customScorer = createScorer<CustomInput, CustomOutput>({
 
 Using these types provides autocomplete, compile-time validation, and better documentation for your scoring logic.
 
-## Trace Scoring with Agent Types
+## Trace scoring with agent types
 
 When you use `type: 'agent'`, your scorer is compatible for both adding directly to agents and scoring traces from agent interactions. The scorer automatically transforms trace data into the proper agent input/output format:
 
@@ -139,7 +139,7 @@ const mastra = new Mastra({
 })
 ```
 
-## Step Method Signatures
+## Step method signatures
 
 ### preprocess
 
@@ -199,7 +199,7 @@ The method can return any value. The returned value will be available to subsequ
 
 **judge** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
 
-### generateScore
+### `generateScore`
 
 **Required** step that computes the final numerical score.
 
@@ -234,7 +234,7 @@ When using prompt object mode, you must also provide a `calculateScore` function
 
 **calculateScore** (`function`): Function: ({ run, results, analyzeStepResult }) => number. Converts the LLM's structured output into a numerical score.
 
-### generateReason
+### `generateReason`
 
 Optional step that provides an explanation for the score.
 

package/.docs/reference/evals/faithfulness.md

@@ -1,4 +1,4 @@
-# Faithfulness Scorer
+# Faithfulness scorer
 
 The `createFaithfulnessScorer()` function evaluates how factually accurate an LLM's output is compared to the provided context. It extracts claims from the output and verifies them against the context, making it essential to measure RAG pipeline responses' reliability.
 
@@ -14,7 +14,7 @@ The `createFaithfulnessScorer()` function accepts a single options object with t
 
 This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The id of the run (optional).
 
@@ -32,7 +32,7 @@ This function returns an instance of the MastraScorer class. The `.run()` method
 
 **generateReasonPrompt** (`string`): The prompt sent to the LLM for the generateReason step (optional).
 
-## Scoring Details
+## Scoring details
 
 The scorer evaluates faithfulness through claim verification against provided context.
 

package/.docs/reference/evals/hallucination.md

@@ -1,4 +1,4 @@
-# Hallucination Scorer
+# Hallucination scorer
 
 The `createHallucinationScorer()` function evaluates whether an LLM generates factually correct information by comparing its output against the provided context. This scorer measures hallucination by identifying direct contradictions between the context and the output.
 
@@ -18,7 +18,7 @@ The `createHallucinationScorer()` function accepts a single options object with
 
 This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The id of the run (optional).
 
@@ -36,7 +36,7 @@ This function returns an instance of the MastraScorer class. The `.run()` method
 
 **generateReasonPrompt** (`string`): The prompt sent to the LLM for the generateReason step (optional).
 
-## Scoring Details
+## Scoring details
 
 The scorer evaluates hallucination through contradiction detection and unsupported claim analysis.
 
@@ -115,7 +115,7 @@ const scorer = createHallucinationScorer({
 })
 ```
 
-### Dynamic Context with getContext
+### Dynamic Context with `getContext`
 
 Use `getContext` for live scoring scenarios where context comes from tool results:
 
@@ -164,7 +164,7 @@ const agent = new Agent({
 })
 ```
 
-### Batch Evaluation with runEvals
+### Batch Evaluation with `runEvals`
 
 ```typescript
 import { runEvals } from '@mastra/core/evals'

package/.docs/reference/evals/keyword-coverage.md

@@ -1,4 +1,4 @@
-# Keyword Coverage Scorer
+# Keyword coverage scorer
 
 The `createKeywordCoverageScorer()` function evaluates how well an LLM's output covers the important keywords from the input. It analyzes keyword presence and matches while ignoring common words and stop words.
 
@@ -8,7 +8,7 @@ The `createKeywordCoverageScorer()` function doesn't take any options.
 
 This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The id of the run (optional).
 
@@ -35,7 +35,7 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 }
 ```
 
-## Scoring Details
+## Scoring details
 
 The scorer evaluates keyword coverage by matching keywords with the following features:
 

package/.docs/reference/evals/mastra-scorer.md

@@ -4,7 +4,7 @@ The `MastraScorer` class is the base class for all scorers in Mastra. It provide
 
 **Note:** Most users should use [`createScorer`](https://mastra.ai/reference/evals/create-scorer) to create scorer instances. Direct instantiation of `MastraScorer` isn't recommended.
 
-## How to Get a MastraScorer Instance
+## How to get a `MastraScorer` instance
 
 Use the `createScorer` factory function, which returns a `MastraScorer` instance:
 
@@ -22,7 +22,7 @@ const scorer = createScorer({
 // scorer is now a MastraScorer instance
 ```
 
-## .run() Method
+## `.run()` method
 
 The `.run()` method is the primary way to execute your scorer and evaluate input/output pairs. It processes the data through your defined steps (preprocess → analyze → generateScore → generateReason) and returns a comprehensive result object with the score, reasoning, and intermediate results.
 
@@ -37,7 +37,7 @@ const result = await scorer.run({
 })
 ```
 
-## .run() Input
+## `.run()` input
 
 **input** (`any`): Input data to be evaluated. Can be any type depending on your scorer's requirements.
 
@@ -49,7 +49,7 @@ const result = await scorer.run({
 
 **groundTruth** (`any`): Optional expected or reference output for comparison during scoring. Automatically passed when using runEvals.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The unique identifier for this scoring run.
 
@@ -69,7 +69,7 @@ const result = await scorer.run({
 
 **generateReasonPrompt** (`string`): Generate reason prompt, if defined (optional).
 
-## Step Execution Flow
+## Step execution flow
 
 When you call `.run()`, the MastraScorer executes the defined steps in this order:
 
@@ -80,7 +80,7 @@ When you call `.run()`, the MastraScorer executes the defined steps in this orde
 
 Each step receives the results from previous steps, allowing you to build complex evaluation pipelines.
 
-## Usage Example
+## Usage example
 
 ```typescript
 const scorer = createScorer({

package/.docs/reference/evals/noise-sensitivity.md

@@ -1,4 +1,4 @@
-# Noise Sensitivity Scorer
+# Noise sensitivity scorer
 
 The `createNoiseSensitivityScorerLLM()` function creates a **CI/testing scorer** that evaluates how robust an agent is when exposed to irrelevant, distracting, or misleading information. Unlike live scorers that evaluate single production runs, this scorer requires predetermined test data including both baseline responses and noisy variations.
 
@@ -17,7 +17,7 @@ Before using the noise sensitivity scorer, prepare your test data:
 
 **options** (`NoiseSensitivityOptions`): Configuration options for the scorer
 
-## CI/Testing Requirements
+## CI/testing requirements
 
 This scorer is designed exclusively for CI/testing environments and has specific requirements:
 
@@ -80,13 +80,13 @@ describe('Agent Noise Resistance Tests', () => {
 })
 ```
 
-## .run() Returns
+## `.run()` returns
 
 **score** (`number`): Robustness score between 0 and 1 (1.0 = completely robust, 0.0 = severely compromised)
 
 **reason** (`string`): Human-readable explanation of how noise affected the agent's response
 
-## Evaluation Dimensions
+## Evaluation dimensions
 
 The Noise Sensitivity scorer analyzes five key dimensions:
 
@@ -110,7 +110,7 @@ Compares how similar the responses are in their core message and conclusions. Ev
 
 Checks if noise causes the agent to generate false or fabricated information that wasn't present in either the query or the noise.
 
-## Scoring Algorithm
+## Scoring algorithm
 
 ### Formula
 
@@ -138,7 +138,7 @@ Each dimension receives an impact level with corresponding weights:
 
 When the LLM's direct score and the calculated score diverge by more than the discrepancy threshold, the scorer uses the lower (more conservative) score to ensure reliable evaluation.
 
-## Noise Types
+## Noise types
 
 ### Misinformation
 
@@ -158,7 +158,7 @@ Deliberately conflicting instructions designed to confuse.
 
 Example: "Write a summary of this article. Actually, ignore that and tell me about dogs instead."
 
-## CI/Testing Usage Patterns
+## CI/testing usage patterns
 
 ### Integration Testing
 
@@ -468,7 +468,7 @@ const strictScorer = createNoiseSensitivityScorerLLM({
 })
 ```
 
-## CI Test Suite: Testing different noise types
+## CI test suite: Testing different noise types
 
 Create comprehensive test suites to evaluate agent performance across various noise categories in your CI pipeline:
 
@@ -537,7 +537,7 @@ async function evaluateNoiseResistance(testCases) {
 }
 ```
 
-## CI Pipeline: Batch evaluation for model comparison
+## CI pipeline: Batch evaluation for model comparison
 
 Use in your CI pipeline to compare noise resistance across different models before deployment:
 

package/.docs/reference/evals/prompt-alignment.md

@@ -1,4 +1,4 @@
-# Prompt Alignment Scorer
+# Prompt alignment scorer
 
 The `createPromptAlignmentScorerLLM()` function creates a scorer that evaluates how well agent responses align with user prompts across multiple dimensions: intent understanding, requirement fulfillment, response completeness, and format appropriateness.
 
@@ -8,7 +8,7 @@ The `createPromptAlignmentScorerLLM()` function creates a scorer that evaluates
 
 **options** (`PromptAlignmentOptions`): Configuration options for the scorer
 
-## .run() Returns
+## `.run()` returns
 
 **score** (`number`): Multi-dimensional alignment score between 0 and scale (default 0-1)
 
@@ -52,7 +52,7 @@ The `createPromptAlignmentScorerLLM()` function creates a scorer that evaluates
 }
 ```
 
-## Scoring Details
+## Scoring details
 
 ### Scorer configuration
 
@@ -163,7 +163,7 @@ Final Score = Weighted Score × scale
 - Production monitoring where both user and system requirements matter
 - Holistic assessment of prompt-response alignment
 
-## Common Use Cases
+## Common use cases
 
 ### Code Generation Evaluation
 
@@ -597,7 +597,7 @@ const result = await scorer.run({
 // High score for both addressing the user's request AND following system guidelines
 ```
 
-## Comparison with Other Scorers
+## Comparison with other scorers
 
 | Aspect         | Prompt Alignment                           | Answer Relevancy             | Faithfulness                     |
 | -------------- | ------------------------------------------ | ---------------------------- | -------------------------------- |

package/.docs/reference/evals/run-evals.md

@@ -2,7 +2,7 @@
 
 The `runEvals` function enables batch evaluation of agents and workflows by running multiple test cases against scorers concurrently. This is essential for systematic testing, performance analysis, and validation of AI systems.
 
-## Usage Example
+## Usage example
 
 ```typescript
 import { runEvals } from '@mastra/core/evals'
@@ -43,7 +43,7 @@ console.log(`Processed ${result.summary.totalItems} items`)
 
 **onItemComplete** (`function`): Callback function called after each test case completes. Receives item, target result, and scorer results.
 
-## Data Item Structure
+## Data item structure
 
 **input** (`string | string[] | CoreMessage[] | any`): Input data for the target. For agents: messages or strings. For workflows: workflow input data.
 
@@ -55,7 +55,7 @@ console.log(`Processed ${result.summary.totalItems} items`)
 
 **startOptions** (`WorkflowRunOptions`): Per-item workflow run options (e.g. initialState, perStep, outputOptions). Merged on top of targetOptions, so per-item values take precedence. Only applicable when the target is a workflow.
 
-## Workflow Scorer Configuration
+## Workflow scorer configuration
 
 For workflows, you can specify scorers at different levels using `WorkflowScorerConfig`:
 
@@ -105,7 +105,7 @@ const result = await runEvals({
 })
 ```
 
-### Agent with targetOptions
+### Agent with `targetOptions`
 
 Pass execution options like `maxSteps` or `modelSettings` to customize agent behavior during evaluation:
 
@@ -149,7 +149,7 @@ const workflowResult = await runEvals({
 })
 ```
 
-### Workflow with per-item startOptions
+### Workflow with per-item `startOptions`
 
 Use `startOptions` on individual data items to customize each workflow run. Per-item values take precedence over `targetOptions`:
 

package/.docs/reference/evals/scorer-utils.md

@@ -1,4 +1,4 @@
-# Scorer Utils
+# Scorer utils
 
 Mastra provides utility functions to help extract and process data from scorer run inputs and outputs. These utilities are particularly useful in the `preprocess` step of custom scorers.
 
@@ -17,9 +17,9 @@ import {
 } from '@mastra/evals/scorers/utils'
 ```
 
-## Message Extraction
+## Message extraction
 
-### getAssistantMessageFromRunOutput
+### `getAssistantMessageFromRunOutput`
 
 Extracts the text content from the first assistant message in the run output.
 
@@ -42,7 +42,7 @@ const scorer = createScorer({
 
 **Returns:** `string | undefined` - The assistant message text, or undefined if no assistant message is found.
 
-### getUserMessageFromRunInput
+### `getUserMessageFromRunInput`
 
 Extracts the text content from the first user message in the run input.
 
@@ -57,7 +57,7 @@ Extracts the text content from the first user message in the run input.
 
 **Returns:** `string | undefined` - The user message text, or undefined if no user message is found.
 
-### extractInputMessages
+### `extractInputMessages`
 
 Extracts text content from all input messages as an array.
 
@@ -70,7 +70,7 @@ Extracts text content from all input messages as an array.
 
 **Returns:** `string[]` - Array of text strings from each input message.
 
-### extractAgentResponseMessages
+### `extractAgentResponseMessages`
 
 Extracts text content from all assistant response messages as an array.
 
@@ -83,9 +83,9 @@ Extracts text content from all assistant response messages as an array.
 
 **Returns:** `string[]` - Array of text strings from each assistant message.
 
-## Reasoning Extraction
+## Reasoning extraction
 
-### getReasoningFromRunOutput
+### `getReasoningFromRunOutput`
 
 Extracts reasoning text from the run output. This is particularly useful when evaluating responses from reasoning models like `deepseek-reasoner` that produce chain-of-thought reasoning.
 
@@ -138,9 +138,9 @@ const reasoningQualityScorer = createScorer({
 
 **Returns:** `string | undefined` - The reasoning text, or undefined if no reasoning is present.
 
-## System Message Extraction
+## System message extraction
 
-### getSystemMessagesFromRunInput
+### `getSystemMessagesFromRunInput`
 
 Extracts all system messages from the run input, including both standard system messages and tagged system messages (specialized prompts like memory instructions).
 
@@ -156,7 +156,7 @@ Extracts all system messages from the run input, including both standard system
 
 **Returns:** `string[]` - Array of system message strings.
 
-### getCombinedSystemPrompt
+### `getCombinedSystemPrompt`
 
 Combines all system messages into a single prompt string, joined with double newlines.
 
@@ -169,9 +169,9 @@ Combines all system messages into a single prompt string, joined with double new
 
 **Returns:** `string` - Combined system prompt string.
 
-## Tool Call Extraction
+## Tool call extraction
 
-### extractToolCalls
+### `extractToolCalls`
 
 Extracts information about all tool calls from the run output, including tool names, call IDs, and their positions in the message array.
 
@@ -216,11 +216,11 @@ type ToolCallInfo = {
 }
 ```
 
-## Test Utilities
+## Test utilities
 
 These utilities help create test data for scorer development.
 
-### createTestMessage
+### `createTestMessage`
 
 Creates a `MastraDBMessage` object for testing purposes.
 
@@ -247,7 +247,7 @@ const assistantMessage = createTestMessage({
 })
 ```
 
-### createAgentTestRun
+### `createAgentTestRun`
 
 Creates a complete test run object for testing scorers.
 
@@ -266,7 +266,7 @@ const result = await myScorer.run({
 })
 ```
 
-## Complete Example
+## Complete example
 
 Here's a complete example showing how to use multiple utilities together:
 

package/.docs/reference/evals/textual-difference.md

@@ -1,4 +1,4 @@
-# Textual Difference Scorer
+# Textual difference scorer
 
 The `createTextualDifferenceScorer()` function uses sequence matching to measure the textual differences between two strings. It provides detailed information about changes, including the number of operations needed to transform one text into another.
 
@@ -8,7 +8,7 @@ The `createTextualDifferenceScorer()` function doesn't take any options.
 
 This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The id of the run (optional).
 
@@ -31,7 +31,7 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 }
 ```
 
-## Scoring Details
+## Scoring details
 
 The scorer calculates several measures:
 

package/.docs/reference/evals/tone-consistency.md

@@ -1,4 +1,4 @@
-# Tone Consistency Scorer
+# Tone consistency scorer
 
 The `createToneScorer()` function evaluates the text's emotional tone and sentiment consistency. It can operate in two modes: comparing tone between input/output pairs or analyzing tone stability within a single text.
 
@@ -8,7 +8,7 @@ The `createToneScorer()` function doesn't take any options.
 
 This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The id of the run (optional).
 
@@ -32,7 +32,7 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 }
 ```
 
-## Scoring Details
+## Scoring details
 
 The scorer evaluates sentiment consistency through tone pattern analysis and mode-specific scoring.
 
@@ -65,7 +65,7 @@ Final score: `mode_specific_score * scale`
 - 0.1-0.3: Poor consistency with major tone changes
 - 0.0: No consistency - completely different tones
 
-### analyzeStepResult
+### `analyzeStepResult`
 
 Object with tone metrics:
 

package/.docs/reference/evals/tool-call-accuracy.md

@@ -1,11 +1,11 @@
-# Tool Call Accuracy Scorers
+# Tool call accuracy scorers
 
 Mastra provides two tool call accuracy scorers for evaluating whether an LLM selects the correct tools from available options:
 
 1. **Code-based scorer** - Deterministic evaluation using exact tool matching
 2. **LLM-based scorer** - Semantic evaluation using AI to assess appropriateness
 
-## Choosing Between Scorers
+## Choosing between scorers
 
 ### Use the Code-Based Scorer When:
 
@@ -23,7 +23,7 @@ Mastra provides two tool call accuracy scorers for evaluating whether an LLM sel
 - You need **explanations** for scoring decisions
 - You're evaluating **production agent behavior**
 
-## Code-Based Tool Call Accuracy Scorer
+## Code-based tool call accuracy scorer
 
 The `createToolCallAccuracyScorerCode()` function from `@mastra/evals/scorers/prebuilt` provides deterministic binary scoring based on exact tool matching and supports both strict and lenient evaluation modes, as well as tool calling order validation.
 
@@ -55,7 +55,7 @@ When `expectedToolOrder` is provided, the scorer validates tool calling sequence
 - **Strict Order (strictMode: true)**: Tools must be called in exactly the specified order with no extra tools
 - **Flexible Order (strictMode: false)**: Expected tools must appear in correct relative order (extra tools allowed)
 
-## Code-Based Scoring Details
+## Code-based scoring details
 
 - **Binary scores**: Always returns 0 or 1
 - **Deterministic**: Same input always produces same output
@@ -103,7 +103,7 @@ const strictOrderScorer = createCodeScorer({
 }
 ```
 
-## Code-Based Scorer Examples
+## Code-based scorer examples
 
 The code-based scorer provides deterministic, binary scoring (0 or 1) based on exact tool matching.
 
@@ -271,7 +271,7 @@ const result = await flexibleOrderScorer.run(run)
 console.log(result.score) // 1 - auth-tool comes before fetch-tool
 ```
 
-## LLM-Based Tool Call Accuracy Scorer
+## LLM-based tool call accuracy scorer
 
 The `createToolCallAccuracyScorerLLM()` function from `@mastra/evals/scorers/prebuilt` uses an LLM to evaluate whether the tools called by an agent are appropriate for the given user request, providing semantic evaluation rather than exact matching.
 
@@ -298,7 +298,7 @@ The LLM-based scorer provides:
 3. **Generate Score**: Calculates score based on appropriate vs total tool calls
 4. **Generate Reasoning**: Provides human-readable explanation
 
-## LLM-Based Scoring Details
+## LLM-based scoring details
 
 - **Fractional scores**: Returns values between 0.0 and 1.0
 - **Context-aware**: Considers user intent and appropriateness
@@ -341,7 +341,7 @@ const customModelScorer = createLLMScorer({
 }
 ```
 
-## LLM-Based Scorer Examples
+## LLM-based scorer examples
 
 The LLM-based scorer uses AI to evaluate whether tool selections are appropriate for the user's request.
 
@@ -465,7 +465,7 @@ console.log(result.score); // 1.0 - appropriate to ask for clarification
 console.log(result.reason); // "The agent appropriately asked for clarification rather than calling tools with insufficient information."
 ```
 
-## Comparing Both Scorers
+## Comparing both scorers
 
 Here's an example using both scorers on the same data:
 

package/.docs/reference/evals/toxicity.md

@@ -1,4 +1,4 @@
-# Toxicity Scorer
+# Toxicity scorer
 
 The `createToxicityScorer()` function evaluates whether an LLM's output contains racist, biased, or toxic elements. It uses a judge-based system to analyze responses for various forms of toxicity including personal attacks, mockery, hate speech, dismissive statements, and threats.
 
@@ -12,7 +12,7 @@ The `createToxicityScorer()` function accepts a single options object with the f
 
 This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
 
-## .run() Returns
+## `.run()` returns
 
 **runId** (`string`): The id of the run (optional).
 
@@ -41,7 +41,7 @@ This function returns an instance of the MastraScorer class. The `.run()` method
 }
 ```
 
-## Scoring Details
+## Scoring details
 
 The scorer evaluates toxicity through multiple aspects:
 

package/.docs/reference/harness/harness-class.md

@@ -1,4 +1,4 @@
-# Harness Class
+# Harness class
 
 **Added in:** `@mastra/core@1.5.0`