@mastra/mcp-docs-server 0.13.6 → 0.13.7-alpha.1

package/.docs/raw/reference/scorers/mastra-scorer.mdx

@@ -0,0 +1,218 @@
+---
+title: "Reference: MastraScorer | Scorers | Mastra Docs"
+description: Documentation for the MastraScorer base class in Mastra, which provides the foundation for all custom and built-in scorers.
+---
+
+# MastraScorer
+
+The `MastraScorer` class is the base class for all scorers in Mastra. It provides a standard `.run()` method for evaluating input/output pairs and supports multi-step scoring workflows. Most users will use `createScorer` or `createLLMScorer`, but advanced users can subclass or instantiate `MastraScorer` directly for full control.
+
+## Constructor Options
+
+<PropertiesTable
+  content={[
+    {
+      name: "name",
+      type: "string",
+      required: true,
+      description: "Name of the scorer.",
+    },
+    {
+      name: "description",
+      type: "string",
+      required: true,
+      description: "Description of what the scorer does.",
+    },
+    {
+      name: "extract",
+      type: "function",
+      required: false,
+      description: "Optional extraction step. See extract step signature below.",
+    },
+    {
+      name: "analyze",
+      type: "function",
+      required: true,
+      description: "Main scoring logic. See analyze step signature below.",
+    },
+    {
+      name: "reason",
+      type: "function",
+      required: false,
+      description: "Optional reason/explanation step. See reason step signature below.",
+    },
+    {
+      name: "metadata",
+      type: "object",
+      required: false,
+      description: "Optional metadata for the scorer.",
+    },
+    {
+      name: "isLLMScorer",
+      type: "boolean",
+      required: false,
+      description: "(Internal) Used to distinguish LLM scorers.",
+    },
+  ]}
+/>
+
+## Step Function Signatures
+
+### extract
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "Record<string, any>[]",
+      required: false,
+      description: "Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. `[{ role: 'user', content: 'hello world' }]`. If the scorer is used in a workflow, this will be the input of the workflow.",
+    },
+    {
+      name: "output",
+      type: "Record<string, any>",
+      required: true,
+      description: "Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.",
+    },
+  ]}
+/>
+Returns: `{ results: any }`  
+The method must return an object with a `results` property. The value of `results` will be passed to the analyze function as `extractStepResult`.
+
+### analyze
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "Record<string, any>[]",
+      required: true,
+      description: "Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. `[{ role: 'user', content: 'hello world' }]`. If the scorer is used in a workflow, this will be the input of the workflow.",
+    },
+    {
+      name: "output",
+      type: "Record<string, any>",
+      required: true,
+      description: "Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.",
+    },
+    {
+      name: "extractStepResult",
+      type: "object",
+      required: false,
+      description: "Result of the extract step, if defined (optional).",
+    },
+  ]}
+/>
+Returns: `{ score: number, results?: any }`  
+The method must return an object with a `score` property (required). Optionally, it may return a `results` property. The value of `results` will be passed to the reason function as `analyzeStepResult`.
+
+### reason
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "Record<string, any>[]",
+      required: true,
+      description: "Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. `[{ role: 'user', content: 'hello world' }]`. If the scorer is used in a workflow, this will be the input of the workflow.",
+    },
+    {
+      name: "output",
+      type: "Record<string, any>",
+      required: true,
+      description: "Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.",
+    },
+    {
+      name: "score",
+      type: "number",
+      required: true,
+      description: "Score computed by the analyze step.",
+    },
+    {
+      name: "analyzeStepResult",
+      type: "object",
+      required: true,
+      description: "Result of the analyze step.",
+    },
+    {
+      name: "extractStepResult",
+      type: "object",
+      required: false,
+      description: "Result of the extract step, if defined (optional).",  
+    },
+  ]}
+/>
+Returns: `{ reason: string }`  
+The method must return an object with a `reason` property, which should be a string explaining the score.
+
+All step functions can be async.
+
+## .run() Input
+
+<PropertiesTable
+  content={[
+    {
+      name: "runId",
+      type: "string",
+      required: false,
+      description: "The id of the run (optional).",
+    },
+    {
+      name: "input",
+      type: "Record<string, any>[]",
+      required: true,
+      description: "An array of records. This should contain user messages or the data to be evaluated.",
+    },
+    {
+      name: "output",
+      type: "Record<string, any>",
+      required: true,
+      description: "A record. This should contain the output to be evaluated.",
+    },
+    {
+      name: "additionalContext",
+      type: "Record<string, any>",
+      required: false,
+      description: "Additional context for the run (optional).",
+    },
+    {
+      name: "runtimeContext",
+      type: "Record<string, any>",
+      required: false,
+      description: "Runtime context for the run (optional).",
+    },
+  ]}
+/>
+
+## .run() Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "runId",
+      type: "string",
+      description: "The id of the run (optional).",
+    },
+    {
+      name: "extractStepResult",
+      type: "object",
+      description: "Result of the extract step, if defined (optional).",
+    },
+    {
+      name: "analyzeStepResult",
+      type: "object",
+      description: "Result of the analyze step (custom structure defined by your scorer).",
+    },
+    {
+      name: "score",
+      type: "number",
+      description: "Score computed by your analyze function.",
+    },
+    {
+      name: "reason",
+      type: "string",
+      description: "Reason/explanation for the score, if defined (optional).",
+    },
+  ]}
+/>
+
+## Integration
+
+MastraScorer instances can be used for agents and workflow steps

package/.docs/raw/reference/scorers/textual-difference.mdx

@@ -0,0 +1,76 @@
+---
+title: "Reference: Textual Difference | Scorers | Mastra Docs"
+description: Documentation for the Textual Difference Scorer in Mastra, which measures textual differences between strings using sequence matching.
+---
+
+# Textual Difference Scorer
+
+The `createTextualDifferenceScorer()` function uses sequence matching to measure the textual differences between two strings. It provides detailed information about changes, including the number of operations needed to transform one text into another.
+
+For a usage example, see the [Textual Difference Examples](/examples/scorers/textual-difference).
+
+## Parameters
+
+The `createTextualDifferenceScorer()` function does not take any options.
+
+This function returns an instance of the MastraScorer class. See the [MastraScorer reference](./mastra-scorer) for details on the `.run()` method and its input/output.
+
+## .run() Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "runId",
+      type: "string",
+      description: "The id of the run (optional).",
+    },
+    {
+      name: "analyzeStepResult",
+      type: "object",
+      description: "Object with difference metrics: { confidence: number, changes: number, lengthDiff: number }",
+    },
+    {
+      name: "score",
+      type: "number",
+      description: "Similarity ratio (0-1) where 1 indicates identical texts.",
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The scorer calculates several measures:
+
+- **Similarity Ratio**: Based on sequence matching between texts (0-1)
+- **Changes**: Count of non-matching operations needed
+- **Length Difference**: Normalized difference in text lengths
+- **Confidence**: Inversely proportional to length difference
+
+### Scoring Process
+
+1. Analyzes textual differences:
+   - Performs sequence matching between input and output
+   - Counts the number of change operations required
+   - Measures length differences
+2. Calculates metrics:
+   - Computes similarity ratio
+   - Determines confidence score
+   - Combines into weighted score
+
+Final score: `(similarity_ratio * confidence) * scale`
+
+### Score interpretation
+
+(0 to scale, default 0-1)
+
+- 1.0: Identical texts - no differences
+- 0.7-0.9: Minor differences - few changes needed
+- 0.4-0.6: Moderate differences - significant changes
+- 0.1-0.3: Major differences - extensive changes
+- 0.0: Completely different texts
+
+## Related
+
+- [Content Similarity Scorer](./content-similarity)
+- [Completeness Scorer](./completeness)
+- [Keyword Coverage Scorer](./keyword-coverage)

package/.docs/raw/reference/scorers/tone-consistency.mdx

@@ -0,0 +1,75 @@
+---
+title: "Reference: Tone Consistency | Scorers | Mastra Docs"
+description: Documentation for the Tone Consistency Scorer in Mastra, which evaluates emotional tone and sentiment consistency in text.
+---
+
+# Tone Consistency Scorer
+
+The `createToneScorer()` function evaluates the text's emotional tone and sentiment consistency. It can operate in two modes: comparing tone between input/output pairs or analyzing tone stability within a single text.
+
+For a usage example, see the [Tone Consistency Examples](/examples/scorers/tone-consistency).
+
+## Parameters
+
+The `createToneScorer()` function does not take any options.
+
+This function returns an instance of the MastraScorer class. See the [MastraScorer reference](./mastra-scorer) for details on the `.run()` method and its input/output.
+
+## .run() Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "runId",
+      type: "string",
+      description: "The id of the run (optional).",
+    },
+    {
+      name: "analyzeStepResult",
+      type: "object",
+      description: "Object with tone metrics: { responseSentiment: number, referenceSentiment: number, difference: number } (for comparison mode) OR { avgSentiment: number, sentimentVariance: number } (for stability mode)",
+    },
+    {
+      name: "score",
+      type: "number",
+      description: "Tone consistency/stability score (0-1).",
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The scorer evaluates sentiment consistency through tone pattern analysis and mode-specific scoring.
+
+### Scoring Process
+
+1. Analyzes tone patterns:
+   - Extracts sentiment features
+   - Computes sentiment scores
+   - Measures tone variations
+2. Calculates mode-specific score:
+   **Tone Consistency** (input and output):
+   - Compares sentiment between texts
+   - Calculates sentiment difference
+   - Score = 1 - (sentiment_difference / max_difference)
+   **Tone Stability** (single input):
+   - Analyzes sentiment across sentences
+   - Calculates sentiment variance
+   - Score = 1 - (sentiment_variance / max_variance)
+
+Final score: `mode_specific_score * scale`
+
+### Score interpretation
+
+(0 to scale, default 0-1)
+
+- 1.0: Perfect tone consistency/stability
+- 0.7-0.9: Strong consistency with minor variations
+- 0.4-0.6: Moderate consistency with noticeable shifts
+- 0.1-0.3: Poor consistency with major tone changes
+- 0.0: No consistency - completely different tones
+
+## Related
+
+- [Content Similarity Scorer](./content-similarity)
+- [Toxicity Scorer](./toxicity)

package/.docs/raw/reference/scorers/toxicity.mdx

@@ -0,0 +1,109 @@
+---
+title: "Reference: Toxicity | Scorers | Mastra Docs"
+description: Documentation for the Toxicity Scorer in Mastra, which evaluates LLM outputs for racist, biased, or toxic elements.
+---
+
+# Toxicity Scorer
+
+The `createToxicityScorer()` function evaluates whether an LLM's output contains racist, biased, or toxic elements. It uses a judge-based system to analyze responses for various forms of toxicity including personal attacks, mockery, hate speech, dismissive statements, and threats.
+
+For a usage example, see the [Toxicity Examples](/examples/scorers/toxicity).
+
+## Parameters
+
+The `createToxicityScorer()` function accepts a single options object with the following properties:
+
+<PropertiesTable
+  content={[
+    {
+      name: "model",
+      type: "LanguageModel",
+      required: true,
+      description: "Configuration for the model used to evaluate toxicity.",
+    },
+    {
+      name: "scale",
+      type: "number",
+      required: false,
+      defaultValue: "1",
+      description: "Maximum score value (default is 1).",
+    },
+  ]}
+/>
+
+This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](./mastra-scorer)), but the return value includes LLM-specific fields as documented below.
+
+## .run() Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "runId",
+      type: "string",
+      description: "The id of the run (optional).",
+    },
+    {
+      name: "analyzeStepResult",
+      type: "object",
+      description: "Object with verdicts: { verdicts: Array<{ verdict: 'yes' | 'no', reason: string }> }",
+    },
+    {
+      name: "analyzePrompt",
+      type: "string",
+      description: "The prompt sent to the LLM for the analyze step (optional).",
+    },
+    {
+      name: "score",
+      type: "number",
+      description: "Toxicity score (0 to scale, default 0-1).",
+    },
+    {
+      name: "reason",
+      type: "string",
+      description: "Detailed explanation of the toxicity assessment.",
+    },
+    {
+      name: "reasonPrompt",
+      type: "string",
+      description: "The prompt sent to the LLM for the reason step (optional).",
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The scorer evaluates toxicity through multiple aspects:
+
+- Personal attacks
+- Mockery or sarcasm
+- Hate speech
+- Dismissive statements
+- Threats or intimidation
+
+### Scoring Process
+
+1. Analyzes toxic elements:
+   - Identifies personal attacks and mockery
+   - Detects hate speech and threats
+   - Evaluates dismissive statements
+   - Assesses severity levels
+2. Calculates toxicity score:
+   - Weighs detected elements
+   - Combines severity ratings
+   - Normalizes to scale
+
+Final score: `(toxicity_weighted_sum / max_toxicity) * scale`
+
+### Score interpretation
+
+(0 to scale, default 0-1)
+
+- 0.8-1.0: Severe toxicity
+- 0.4-0.7: Moderate toxicity
+- 0.1-0.3: Mild toxicity
+- 0.0: No toxic elements detected
+
+## Related
+
+- [Tone Consistency Scorer](./tone-consistency)
+- [Bias Scorer](./bias)

package/.docs/raw/reference/templates.mdx

@@ -0,0 +1,222 @@
+---
+title: "Templates Reference"
+description: "Complete guide to creating, using, and contributing Mastra templates"
+---
+
+import { FileTree, Tabs, Callout } from 'nextra/components'
+
+This reference provides comprehensive information about Mastra templates, including how to use existing templates, create your own, and contribute to the community ecosystem.
+
+## Overview
+
+Mastra templates are pre-built project structures that demonstrate specific use cases and patterns. They provide:
+
+- **Working examples** - Complete, functional Mastra applications
+- **Best practices** - Proper project structure and coding conventions
+- **Educational resources** - Learn Mastra patterns through real implementations
+- **Quick starts** - Bootstrap projects faster than building from scratch
+
+## Using Templates
+
+### Installation
+
+Install a template using the `create-mastra` command:
+
+```bash copy
+npx create-mastra@latest --template template-name
+```
+
+This creates a complete project with all necessary code and configuration.
+
+### Setup Process
+
+After installation:
+
+1. **Navigate to project directory**:
+
+   ```bash copy
+   cd your-project-name
+   ```
+
+2. **Configure environment variables**:
+
+   ```bash copy
+   cp .env.example .env
+   ```
+
+   Edit `.env` with required API keys as documented in the template's README.
+
+3. **Install dependencies** (if not done automatically):
+
+   ```bash copy
+   npm install
+   ```
+
+4. **Start development server**:
+
+   ```bash copy
+   npm run dev
+   ```
+
+### Template Structure
+
+All templates follow this standardized structure:
+
+<FileTree>
+  <FileTree.Folder name="template-name" defaultOpen>
+    <FileTree.Folder name="src" defaultOpen>
+      <FileTree.Folder name="mastra" defaultOpen>
+        <FileTree.Folder name="agents">
+          <FileTree.File name="example-agent.ts" />
+        </FileTree.Folder>
+        <FileTree.Folder name="tools">
+          <FileTree.File name="custom-tool.ts" />
+        </FileTree.Folder>
+        <FileTree.Folder name="workflows">
+          <FileTree.File name="example-workflow.ts" />
+        </FileTree.Folder>
+        <FileTree.File name="index.ts" />
+      </FileTree.Folder>
+    </FileTree.Folder>
+    <FileTree.File name=".env.example" />
+    <FileTree.File name="package.json" />
+    <FileTree.File name="README.md" />
+    <FileTree.File name="tsconfig.json" />
+  </FileTree.Folder>
+</FileTree>
+
+## Creating Templates
+
+### Requirements
+
+Templates must meet these technical requirements:
+
+#### Project Structure
+
+- **Mastra code location**: All Mastra code must be in `src/mastra/` directory
+- **Component organization**:
+  - Agents: `src/mastra/agents/`
+  - Tools: `src/mastra/tools/`
+  - Workflows: `src/mastra/workflows/`
+  - Main config: `src/mastra/index.ts`
+
+#### TypeScript Configuration
+
+Use the standard Mastra TypeScript configuration:
+
+```json filename="tsconfig.json"
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "module": "ES2022",
+    "moduleResolution": "bundler",
+    "esModuleInterop": true,
+    "forceConsistentCasingInFileNames": true,
+    "strict": true,
+    "skipLibCheck": true,
+    "noEmit": true,
+    "outDir": "dist"
+  },
+  "include": ["src/**/*"]
+}
+```
+
+#### Environment Configuration
+
+Include a `.env.example` file with all required environment variables:
+
+```bash filename=".env.example"
+# OpenAI API key for LLM operations
+OPENAI_API_KEY=your_openai_api_key_here
+
+# Other service API keys as needed
+OTHER_SERVICE_API_KEY=your_api_key_here
+```
+
+### Code Standards
+
+#### LLM Provider
+
+Use OpenAI as the default provider unless demonstrating specific integrations:
+
+```typescript filename="src/mastra/agents/example-agent.ts"
+import { Agent } from '@mastra/core';
+import { openai } from '@ai-sdk/openai';
+
+const agent = new Agent({
+  name: 'example-agent',
+  model: openai('gpt-4'),
+  instructions: 'Your agent instructions here',
+  // ... other configuration
+});
+```
+
+#### Compatibility Requirements
+
+Templates must be:
+
+- **Single projects** - Not monorepos with multiple applications
+- **Framework-free** - No Next.js, Express, or other web framework boilerplate
+- **Mastra-focused** - Demonstrate Mastra functionality without additional layers
+- **Mergeable** - Structure code for easy integration into existing projects
+- **Node.js compatible** - Support Node.js 18 and higher
+- **ESM modules** - Use ES modules (`"type": "module"` in package.json)
+
+### Documentation Requirements
+
+#### README Structure
+
+Every template must include a comprehensive README:
+
+```markdown filename="README.md"
+# Template Name
+
+Brief description of what the template demonstrates.
+
+## Overview
+
+Detailed explanation of the template's functionality and use case.
+
+## Setup
+
+1. Copy `.env.example` to `.env` and fill in your API keys
+2. Install dependencies: `npm install`  
+3. Run the project: `npm run dev`
+
+## Environment Variables
+
+- `OPENAI_API_KEY`: Your OpenAI API key. Get one at [OpenAI Platform](https://platform.openai.com/api-keys)
+- `OTHER_API_KEY`: Description of what this key is for
+
+## Usage
+
+Instructions on how to use the template and examples of expected behavior.
+
+## Customization
+
+Guidelines for modifying the template for different use cases.
+```
+
+#### Code Comments
+
+Include clear comments explaining:
+
+- Complex logic or algorithms
+- API integrations and their purpose
+- Configuration options and their effects
+- Example usage patterns
+
+### Quality Standards
+
+Templates must demonstrate:
+
+- **Code quality** - Clean, well-commented, maintainable code
+- **Error handling** - Proper handling for external APIs and user inputs
+- **Type safety** - Full TypeScript typing with Zod validation
+- **Testing** - Verified functionality with fresh installations
+
+For information on contributing your own templates to the Mastra ecosystem, see the [Contributing Templates](/docs/community/contributing-templates) guide in the community section.
+
+<Callout type="info">
+  Templates provide an excellent way to learn Mastra patterns and accelerate development. Contributing templates helps the entire community build better AI applications.
+</Callout>