@mastra/mcp-docs-server 0.13.37 → 0.13.38

package/.docs/raw/reference/scorers/completeness.mdx

@@ -1,5 +1,5 @@
 ---
-title: "Reference: Completeness | Scorers | Mastra Docs"
+title: "Reference: Completeness Scorer | Scorers | Mastra Docs"
 description: Documentation for the Completeness Scorer in Mastra, which evaluates how thoroughly LLM outputs cover key elements present in the input.
 ---
 
@@ -25,12 +25,14 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
     {
       name: "preprocessStepResult",
       type: "object",
-      description: "Object with extracted elements and coverage details: { inputElements: string[], outputElements: string[], missingElements: string[], elementCounts: { input: number, output: number } }",
+      description:
+        "Object with extracted elements and coverage details: { inputElements: string[], outputElements: string[], missingElements: string[], elementCounts: { input: number, output: number } }",
     },
     {
       name: "score",
       type: "number",
-      description: "Completeness score (0-1) representing the proportion of input elements covered in the output.",
+      description:
+        "Completeness score (0-1) representing the proportion of input elements covered in the output.",
     },
   ]}
 />
@@ -109,17 +111,18 @@ A completeness score between 0 and 1:
 
 In this example, the response comprehensively addresses all aspects of the query with detailed information covering multiple dimensions.
 
-```typescript filename="src/example-high-completeness.ts" showLineNumbers copy
+```typescript title="src/example-high-completeness.ts" showLineNumbers copy
 import { createCompletenessScorer } from "@mastra/evals/scorers/llm";
 
-const scorer = createCompletenessScorer({ model: 'openai/gpt-4o-mini' });
+const scorer = createCompletenessScorer({ model: "openai/gpt-4o-mini" });
 
-const query = "Explain the process of photosynthesis, including the inputs, outputs, and stages involved.";
+const query =
+  "Explain the process of photosynthesis, including the inputs, outputs, and stages involved.";
 const response =
   "Photosynthesis is the process by which plants convert sunlight into chemical energy. Inputs: Carbon dioxide (CO2) from the air enters through stomata, water (H2O) is absorbed by roots, and sunlight provides energy captured by chlorophyll. The process occurs in two main stages: 1) Light-dependent reactions in the thylakoids convert light energy to ATP and NADPH while splitting water and releasing oxygen. 2) Light-independent reactions (Calvin cycle) in the stroma use ATP, NADPH, and CO2 to produce glucose. Outputs: Glucose (C6H12O6) serves as food for the plant, and oxygen (O2) is released as a byproduct. The overall equation is: 6CO2 + 6H2O + light energy → C6H12O6 + 6O2.";
 
 const result = await scorer.run({
-  input: [{ role: 'user', content: query }],
+  input: [{ role: "user", content: query }],
   output: { text: response },
 });
 
@@ -141,17 +144,18 @@ The output receives a high score because it addresses all requested aspects: inp
 
 In this example, the response addresses some key points but misses important aspects or lacks sufficient detail.
 
-```typescript filename="src/example-partial-completeness.ts" showLineNumbers copy
+```typescript title="src/example-partial-completeness.ts" showLineNumbers copy
 import { createCompletenessScorer } from "@mastra/evals/scorers/llm";
 
-const scorer = createCompletenessScorer({ model: 'openai/gpt-4o-mini' });
+const scorer = createCompletenessScorer({ model: "openai/gpt-4o-mini" });
 
-const query = "What are the benefits and drawbacks of remote work for both employees and employers?";
+const query =
+  "What are the benefits and drawbacks of remote work for both employees and employers?";
 const response =
   "Remote work offers several benefits for employees including flexible schedules, no commuting time, and better work-life balance. It also reduces costs for office space and utilities for employers. However, remote work can lead to isolation and communication challenges for employees.";
 
 const result = await scorer.run({
-  input: [{ role: 'user', content: query }],
+  input: [{ role: "user", content: query }],
   output: { text: response },
 });
 
@@ -173,17 +177,18 @@ The output receives a moderate score because it covers employee benefits and som
 
 In this example, the response only partially addresses the query and misses several important aspects.
 
-```typescript filename="src/example-low-completeness.ts" showLineNumbers copy
+```typescript title="src/example-low-completeness.ts" showLineNumbers copy
 import { createCompletenessScorer } from "@mastra/evals/scorers/llm";
 
-const scorer = createCompletenessScorer({ model: 'openai/gpt-4o-mini' });
+const scorer = createCompletenessScorer({ model: "openai/gpt-4o-mini" });
 
-const query = "Compare renewable and non-renewable energy sources in terms of cost, environmental impact, and sustainability.";
+const query =
+  "Compare renewable and non-renewable energy sources in terms of cost, environmental impact, and sustainability.";
 const response =
   "Renewable energy sources like solar and wind are becoming cheaper. They're better for the environment than fossil fuels.";
 
 const result = await scorer.run({
-  input: [{ role: 'user', content: query }],
+  input: [{ role: "user", content: query }],
   output: { text: response },
 });
 
@@ -206,4 +211,4 @@ The output receives a low score because it only briefly mentions cost and enviro
 - [Answer Relevancy Scorer](./answer-relevancy)
 - [Content Similarity Scorer](./content-similarity)
 - [Textual Difference Scorer](./textual-difference)
-- [Keyword Coverage Scorer](./keyword-coverage)
+- [Keyword Coverage Scorer](./keyword-coverage)

package/.docs/raw/reference/scorers/content-similarity.mdx

@@ -1,5 +1,5 @@
 ---
-title: "Reference: Content Similarity | Scorers | Mastra Docs"
+title: "Reference: Content Similarity Scorer | Scorers | Mastra Docs"
 description: Documentation for the Content Similarity Scorer in Mastra, which measures textual similarity between strings and provides a matching score.
 ---
 
@@ -44,7 +44,8 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
     {
       name: "preprocessStepResult",
       type: "object",
-      description: "Object with processed input and output: { processedInput: string, processedOutput: string }",
+      description:
+        "Object with processed input and output: { processedInput: string, processedOutput: string }",
     },
     {
       name: "analyzeStepResult",
@@ -54,7 +55,8 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
     {
       name: "score",
       type: "number",
-      description: "Similarity score (0-1) where 1 indicates perfect similarity.",
+      description:
+        "Similarity score (0-1) where 1 indicates perfect similarity.",
     },
   ]}
 />
@@ -82,7 +84,7 @@ Final score: `similarity_value * scale`
 
 In this example, the response closely resembles the query in both structure and meaning. Minor differences in tense and phrasing do not significantly affect the overall similarity.
 
-```typescript filename="src/example-high-similarity.ts" showLineNumbers copy
+```typescript title="src/example-high-similarity.ts" showLineNumbers copy
 import { createContentSimilarityScorer } from "@mastra/evals/scorers/llm";
 
 const scorer = createContentSimilarityScorer();
@@ -91,7 +93,7 @@ const query = "The quick brown fox jumps over the lazy dog.";
 const response = "A quick brown fox jumped over a lazy dog.";
 
 const result = await scorer.run({
-  input: [{ role: 'user', content: query }],
+  input: [{ role: "user", content: query }],
   output: { text: response },
 });
 
@@ -115,7 +117,7 @@ The output receives a high score because the response preserves the intent and c
 
 In this example, the response shares some conceptual overlap with the query but diverges in structure and wording. Key elements remain present, but the phrasing introduces moderate variation.
 
-```typescript filename="src/example-moderate-similarity.ts" showLineNumbers copy
+```typescript title="src/example-moderate-similarity.ts" showLineNumbers copy
 import { createContentSimilarityScorer } from "@mastra/evals/scorers/llm";
 
 const scorer = createContentSimilarityScorer();
@@ -124,7 +126,7 @@ const query = "A brown fox quickly leaps across a sleeping dog.";
 const response = "The quick brown fox jumps over the lazy dog.";
 
 const result = await scorer.run({
-  input: [{ role: 'user', content: query }],
+  input: [{ role: "user", content: query }],
   output: { text: response },
 });
 
@@ -148,7 +150,7 @@ The output receives a mid-range score because the response captures the general
 
 In this example, the response and query are unrelated in meaning, despite having a similar grammatical structure. There is little to no shared content overlap.
 
-```typescript filename="src/example-low-similarity.ts" showLineNumbers copy
+```typescript title="src/example-low-similarity.ts" showLineNumbers copy
 import { createContentSimilarityScorer } from "@mastra/evals/scorers/llm";
 
 const scorer = createContentSimilarityScorer();
@@ -157,7 +159,7 @@ const query = "The cat sleeps on the windowsill.";
 const response = "The quick brown fox jumps over the lazy dog.";
 
 const result = await scorer.run({
-  input: [{ role: 'user', content: query }],
+  input: [{ role: "user", content: query }],
   output: { text: response },
 });
 
@@ -192,4 +194,4 @@ A similarity score between 0 and 1:
 - [Completeness Scorer](./completeness)
 - [Textual Difference Scorer](./textual-difference)
 - [Answer Relevancy Scorer](./answer-relevancy)
-- [Keyword Coverage Scorer](./keyword-coverage)
+- [Keyword Coverage Scorer](./keyword-coverage)

package/.docs/raw/reference/scorers/context-precision.mdx

@@ -3,7 +3,7 @@ title: "Reference: Context Precision Scorer | Scorers | Mastra Docs"
 description: Documentation for the Context Precision Scorer in Mastra. Evaluates the relevance and precision of retrieved context for generating expected outputs using Mean Average Precision.
 ---
 
-import { PropertiesTable } from "@/components/properties-table";
+import PropertiesTable from "@site/src/components/PropertiesTable";
 
 # Context Precision Scorer
 
@@ -14,6 +14,7 @@ It is especially useful for these use cases:
 **RAG System Evaluation**
 
 Ideal for evaluating retrieved context in RAG pipelines where:
+
 - Context ordering matters for model performance
 - You need to measure retrieval quality beyond simple relevance
 - Early relevant context is more valuable than later relevant context
@@ -21,8 +22,9 @@ Ideal for evaluating retrieved context in RAG pipelines where:
 **Context Window Optimization**
 
 Use when optimizing context selection for:
+
 - Limited context windows
-- Token budget constraints  
+- Token budget constraints
 - Multi-step reasoning tasks
 
 ## Parameters
@@ -50,7 +52,8 @@ Use when optimizing context selection for:
         {
           name: "contextExtractor",
           type: "(input, output) => string[]",
-          description: "Function to dynamically extract context from the run input and output",
+          description:
+            "Function to dynamically extract context from the run input and output",
           required: false,
         },
         {
@@ -64,7 +67,6 @@ Use when optimizing context selection for:
   ]}
 />
 
-
 **Note**: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.
 
 ## .run() Returns
@@ -74,12 +76,14 @@ Use when optimizing context selection for:
     {
       name: "score",
       type: "number",
-      description: "Mean Average Precision score between 0 and scale (default 0-1)",
+      description:
+        "Mean Average Precision score between 0 and scale (default 0-1)",
     },
     {
       name: "reason",
       type: "string",
-      description: "Human-readable explanation of the context precision evaluation",
+      description:
+        "Human-readable explanation of the context precision evaluation",
     },
   ]}
 />
@@ -109,7 +113,7 @@ Where:
 ### Score Interpretation
 
 - **0.9-1.0**: Excellent precision - all relevant context early in sequence
-- **0.7-0.8**: Good precision - most relevant context well-positioned  
+- **0.7-0.8**: Good precision - most relevant context well-positioned
 - **0.4-0.6**: Moderate precision - relevant context mixed with irrelevant
 - **0.1-0.3**: Poor precision - little relevant context or poorly positioned
 - **0.0**: No relevant context found
@@ -117,6 +121,7 @@ Where:
 ### Reason analysis
 
 The reason field explains:
+
 - Which context pieces were deemed relevant/irrelevant
 - How positioning affected the MAP calculation
 - Specific relevance criteria used in evaluation
@@ -124,6 +129,7 @@ The reason field explains:
 ### Optimization insights
 
 Use results to:
+
 - **Improve retrieval**: Filter out irrelevant context before ranking
 - **Optimize ranking**: Ensure relevant context appears early
 - **Tune chunk size**: Balance context detail vs. relevance precision
@@ -134,7 +140,7 @@ Use results to:
 Given context: `[relevant, irrelevant, relevant, irrelevant]`
 
 - Position 0: Relevant → Precision = 1/1 = 1.0
-- Position 1: Skip (irrelevant)  
+- Position 1: Skip (irrelevant)
 - Position 2: Relevant → Precision = 2/3 = 0.67
 - Position 3: Skip (irrelevant)
 
@@ -146,15 +152,15 @@ MAP = (1.0 + 0.67) / 2 = 0.835 ≈ **0.83**
 
 ```typescript
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini',
+  model: "openai/gpt-4o-mini",
   options: {
     contextExtractor: (input, output) => {
       // Extract context dynamically based on the query
-      const query = input?.inputMessages?.[0]?.content || '';
-      
+      const query = input?.inputMessages?.[0]?.content || "";
+
       // Example: Retrieve from a vector database
       const searchResults = vectorDB.search(query, { limit: 10 });
-      return searchResults.map(result => result.content);
+      return searchResults.map((result) => result.content);
     },
     scale: 1,
   },
@@ -165,15 +171,15 @@ const scorer = createContextPrecisionScorer({
 
 ```typescript
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini', 
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
       // Simulate retrieved documents from vector database
-      'Document 1: Highly relevant content...',
-      'Document 2: Somewhat related content...',
-      'Document 3: Tangentially related...',
-      'Document 4: Not relevant...',
-      'Document 5: Highly relevant content...',
+      "Document 1: Highly relevant content...",
+      "Document 2: Somewhat related content...",
+      "Document 3: Tangentially related...",
+      "Document 4: Not relevant...",
+      "Document 5: Highly relevant content...",
       // ... up to dozens of context pieces
     ],
   },
@@ -187,15 +193,15 @@ const scorer = createContextPrecisionScorer({
 This example shows perfect context precision where all relevant context appears early:
 
 ```typescript
-import { createContextPrecisionScorer } from '@mastra/evals';
+import { createContextPrecisionScorer } from "@mastra/evals";
 
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini',
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
-      'Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen.',
-      'The process occurs in the chloroplasts of plant cells, specifically in the thylakoids.',
-      'Light-dependent reactions happen in the thylakoid membranes, while the Calvin cycle occurs in the stroma.',
+      "Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen.",
+      "The process occurs in the chloroplasts of plant cells, specifically in the thylakoids.",
+      "Light-dependent reactions happen in the thylakoid membranes, while the Calvin cycle occurs in the stroma.",
     ],
     scale: 1,
   },
@@ -205,17 +211,18 @@ const result = await scorer.run({
   input: {
     inputMessages: [
       {
-        id: '1',
-        role: 'user',
-        content: 'How does photosynthesis work in plants?',
+        id: "1",
+        role: "user",
+        content: "How does photosynthesis work in plants?",
       },
     ],
   },
   output: [
     {
-      id: '2', 
-      role: 'assistant',
-      content: 'Photosynthesis is the process where plants convert sunlight, CO2, and water into glucose and oxygen using chloroplasts.',
+      id: "2",
+      role: "assistant",
+      content:
+        "Photosynthesis is the process where plants convert sunlight, CO2, and water into glucose and oxygen using chloroplasts.",
     },
   ],
 });
@@ -233,17 +240,17 @@ console.log(result);
 This example shows moderate precision with both relevant and irrelevant context:
 
 ```typescript
-import { createContextPrecisionScorer } from '@mastra/evals';
+import { createContextPrecisionScorer } from "@mastra/evals";
 
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini', 
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
-      'Regular exercise improves cardiovascular health by strengthening the heart muscle.',
-      'A balanced diet should include fruits, vegetables, and whole grains.',
-      'Physical activity releases endorphins which improve mood and reduce stress.',
-      'The average person should drink 8 glasses of water per day.',
-      'Exercise also helps maintain healthy body weight and muscle mass.',
+      "Regular exercise improves cardiovascular health by strengthening the heart muscle.",
+      "A balanced diet should include fruits, vegetables, and whole grains.",
+      "Physical activity releases endorphins which improve mood and reduce stress.",
+      "The average person should drink 8 glasses of water per day.",
+      "Exercise also helps maintain healthy body weight and muscle mass.",
     ],
     scale: 1,
   },
@@ -253,17 +260,18 @@ const result = await scorer.run({
   input: {
     inputMessages: [
       {
-        id: '1',
-        role: 'user', 
-        content: 'What are the mental and physical benefits of exercise?',
+        id: "1",
+        role: "user",
+        content: "What are the mental and physical benefits of exercise?",
       },
     ],
   },
   output: [
     {
-      id: '2',
-      role: 'assistant', 
-      content: 'Exercise provides cardiovascular benefits, improves mood through endorphin release, and helps maintain healthy body composition.',
+      id: "2",
+      role: "assistant",
+      content:
+        "Exercise provides cardiovascular benefits, improves mood through endorphin release, and helps maintain healthy body composition.",
     },
   ],
 });
@@ -276,22 +284,22 @@ console.log(result);
 // }
 ```
 
-### Low precision example  
+### Low precision example
 
 This example shows poor context precision with mostly irrelevant context:
 
 ```typescript
-import { createContextPrecisionScorer } from '@mastra/evals';
+import { createContextPrecisionScorer } from "@mastra/evals";
 
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini',
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
-      'The weather forecast shows sunny skies this weekend.',
-      'Coffee is one of the world\'s most popular beverages.',
-      'Machine learning requires large amounts of training data.',
-      'Cats typically sleep 12-16 hours per day.',
-      'The capital of France is Paris.',
+      "The weather forecast shows sunny skies this weekend.",
+      "Coffee is one of the world's most popular beverages.",
+      "Machine learning requires large amounts of training data.",
+      "Cats typically sleep 12-16 hours per day.",
+      "The capital of France is Paris.",
     ],
     scale: 1,
   },
@@ -301,17 +309,18 @@ const result = await scorer.run({
   input: {
     inputMessages: [
       {
-        id: '1',
-        role: 'user',
-        content: 'How does photosynthesis work?',
+        id: "1",
+        role: "user",
+        content: "How does photosynthesis work?",
       },
     ],
   },
   output: [
     {
-      id: '2',
-      role: 'assistant',
-      content: 'Photosynthesis is the process by which plants convert sunlight into energy using chlorophyll.',
+      id: "2",
+      role: "assistant",
+      content:
+        "Photosynthesis is the process by which plants convert sunlight into energy using chlorophyll.",
     },
   ],
 });
@@ -328,16 +337,16 @@ console.log(result);
 
 Choose the right scorer for your needs:
 
-| Use Case | Context Relevance | Context Precision |
-|----------|-------------------|-------------------|
-| **RAG evaluation** | When usage matters | When ranking matters |
-| **Context quality** | Nuanced levels | Binary relevance |
-| **Missing detection** | ✓ Identifies gaps | ✗ Not evaluated |
-| **Usage tracking** | ✓ Tracks utilization | ✗ Not considered |
-| **Position sensitivity** | ✗ Position agnostic | ✓ Rewards early placement |
+| Use Case                 | Context Relevance    | Context Precision         |
+| ------------------------ | -------------------- | ------------------------- |
+| **RAG evaluation**       | When usage matters   | When ranking matters      |
+| **Context quality**      | Nuanced levels       | Binary relevance          |
+| **Missing detection**    | ✓ Identifies gaps    | ✗ Not evaluated           |
+| **Usage tracking**       | ✓ Tracks utilization | ✗ Not considered          |
+| **Position sensitivity** | ✗ Position agnostic  | ✓ Rewards early placement |
 
 ## Related
 
 - [Answer Relevancy Scorer](/reference/scorers/answer-relevancy) - Evaluates if answers address the question
 - [Faithfulness Scorer](/reference/scorers/faithfulness) - Measures answer groundedness in context
-- [Custom Scorers](/docs/scorers/custom-scorers) - Creating your own evaluation metrics
+- [Custom Scorers](/docs/scorers/custom-scorers) - Creating your own evaluation metrics