@mastra/mcp-docs-server 0.13.37 → 0.13.39-alpha.0

package/.docs/raw/reference/scorers/context-precision.mdx

@@ -3,7 +3,7 @@ title: "Reference: Context Precision Scorer | Scorers | Mastra Docs"
 description: Documentation for the Context Precision Scorer in Mastra. Evaluates the relevance and precision of retrieved context for generating expected outputs using Mean Average Precision.
 ---
 
-import { PropertiesTable } from "@/components/properties-table";
+import PropertiesTable from "@site/src/components/PropertiesTable";
 
 # Context Precision Scorer
 
@@ -14,6 +14,7 @@ It is especially useful for these use cases:
 **RAG System Evaluation**
 
 Ideal for evaluating retrieved context in RAG pipelines where:
+
 - Context ordering matters for model performance
 - You need to measure retrieval quality beyond simple relevance
 - Early relevant context is more valuable than later relevant context
@@ -21,8 +22,9 @@ Ideal for evaluating retrieved context in RAG pipelines where:
 **Context Window Optimization**
 
 Use when optimizing context selection for:
+
 - Limited context windows
-- Token budget constraints  
+- Token budget constraints
 - Multi-step reasoning tasks
 
 ## Parameters
@@ -50,7 +52,8 @@ Use when optimizing context selection for:
         {
           name: "contextExtractor",
           type: "(input, output) => string[]",
-          description: "Function to dynamically extract context from the run input and output",
+          description:
+            "Function to dynamically extract context from the run input and output",
           required: false,
         },
         {
@@ -64,7 +67,6 @@ Use when optimizing context selection for:
   ]}
 />
 
-
 **Note**: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.
 
 ## .run() Returns
@@ -74,12 +76,14 @@ Use when optimizing context selection for:
     {
       name: "score",
       type: "number",
-      description: "Mean Average Precision score between 0 and scale (default 0-1)",
+      description:
+        "Mean Average Precision score between 0 and scale (default 0-1)",
     },
     {
       name: "reason",
       type: "string",
-      description: "Human-readable explanation of the context precision evaluation",
+      description:
+        "Human-readable explanation of the context precision evaluation",
     },
   ]}
 />
@@ -109,7 +113,7 @@ Where:
 ### Score Interpretation
 
 - **0.9-1.0**: Excellent precision - all relevant context early in sequence
-- **0.7-0.8**: Good precision - most relevant context well-positioned  
+- **0.7-0.8**: Good precision - most relevant context well-positioned
 - **0.4-0.6**: Moderate precision - relevant context mixed with irrelevant
 - **0.1-0.3**: Poor precision - little relevant context or poorly positioned
 - **0.0**: No relevant context found
@@ -117,6 +121,7 @@ Where:
 ### Reason analysis
 
 The reason field explains:
+
 - Which context pieces were deemed relevant/irrelevant
 - How positioning affected the MAP calculation
 - Specific relevance criteria used in evaluation
@@ -124,6 +129,7 @@ The reason field explains:
 ### Optimization insights
 
 Use results to:
+
 - **Improve retrieval**: Filter out irrelevant context before ranking
 - **Optimize ranking**: Ensure relevant context appears early
 - **Tune chunk size**: Balance context detail vs. relevance precision
@@ -134,7 +140,7 @@ Use results to:
 Given context: `[relevant, irrelevant, relevant, irrelevant]`
 
 - Position 0: Relevant → Precision = 1/1 = 1.0
-- Position 1: Skip (irrelevant)  
+- Position 1: Skip (irrelevant)
 - Position 2: Relevant → Precision = 2/3 = 0.67
 - Position 3: Skip (irrelevant)
 
@@ -146,15 +152,15 @@ MAP = (1.0 + 0.67) / 2 = 0.835 ≈ **0.83**
 
 ```typescript
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini',
+  model: "openai/gpt-4o-mini",
   options: {
     contextExtractor: (input, output) => {
       // Extract context dynamically based on the query
-      const query = input?.inputMessages?.[0]?.content || '';
-      
+      const query = input?.inputMessages?.[0]?.content || "";
+
       // Example: Retrieve from a vector database
       const searchResults = vectorDB.search(query, { limit: 10 });
-      return searchResults.map(result => result.content);
+      return searchResults.map((result) => result.content);
     },
     scale: 1,
   },
@@ -165,15 +171,15 @@ const scorer = createContextPrecisionScorer({
 
 ```typescript
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini', 
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
       // Simulate retrieved documents from vector database
-      'Document 1: Highly relevant content...',
-      'Document 2: Somewhat related content...',
-      'Document 3: Tangentially related...',
-      'Document 4: Not relevant...',
-      'Document 5: Highly relevant content...',
+      "Document 1: Highly relevant content...",
+      "Document 2: Somewhat related content...",
+      "Document 3: Tangentially related...",
+      "Document 4: Not relevant...",
+      "Document 5: Highly relevant content...",
       // ... up to dozens of context pieces
     ],
   },
@@ -187,15 +193,15 @@ const scorer = createContextPrecisionScorer({
 This example shows perfect context precision where all relevant context appears early:
 
 ```typescript
-import { createContextPrecisionScorer } from '@mastra/evals';
+import { createContextPrecisionScorer } from "@mastra/evals";
 
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini',
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
-      'Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen.',
-      'The process occurs in the chloroplasts of plant cells, specifically in the thylakoids.',
-      'Light-dependent reactions happen in the thylakoid membranes, while the Calvin cycle occurs in the stroma.',
+      "Photosynthesis is the process by which plants convert sunlight, carbon dioxide, and water into glucose and oxygen.",
+      "The process occurs in the chloroplasts of plant cells, specifically in the thylakoids.",
+      "Light-dependent reactions happen in the thylakoid membranes, while the Calvin cycle occurs in the stroma.",
     ],
     scale: 1,
   },
@@ -205,17 +211,18 @@ const result = await scorer.run({
   input: {
     inputMessages: [
       {
-        id: '1',
-        role: 'user',
-        content: 'How does photosynthesis work in plants?',
+        id: "1",
+        role: "user",
+        content: "How does photosynthesis work in plants?",
       },
     ],
   },
   output: [
     {
-      id: '2', 
-      role: 'assistant',
-      content: 'Photosynthesis is the process where plants convert sunlight, CO2, and water into glucose and oxygen using chloroplasts.',
+      id: "2",
+      role: "assistant",
+      content:
+        "Photosynthesis is the process where plants convert sunlight, CO2, and water into glucose and oxygen using chloroplasts.",
     },
   ],
 });
@@ -233,17 +240,17 @@ console.log(result);
 This example shows moderate precision with both relevant and irrelevant context:
 
 ```typescript
-import { createContextPrecisionScorer } from '@mastra/evals';
+import { createContextPrecisionScorer } from "@mastra/evals";
 
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini', 
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
-      'Regular exercise improves cardiovascular health by strengthening the heart muscle.',
-      'A balanced diet should include fruits, vegetables, and whole grains.',
-      'Physical activity releases endorphins which improve mood and reduce stress.',
-      'The average person should drink 8 glasses of water per day.',
-      'Exercise also helps maintain healthy body weight and muscle mass.',
+      "Regular exercise improves cardiovascular health by strengthening the heart muscle.",
+      "A balanced diet should include fruits, vegetables, and whole grains.",
+      "Physical activity releases endorphins which improve mood and reduce stress.",
+      "The average person should drink 8 glasses of water per day.",
+      "Exercise also helps maintain healthy body weight and muscle mass.",
     ],
     scale: 1,
   },
@@ -253,17 +260,18 @@ const result = await scorer.run({
   input: {
     inputMessages: [
       {
-        id: '1',
-        role: 'user', 
-        content: 'What are the mental and physical benefits of exercise?',
+        id: "1",
+        role: "user",
+        content: "What are the mental and physical benefits of exercise?",
       },
     ],
   },
   output: [
     {
-      id: '2',
-      role: 'assistant', 
-      content: 'Exercise provides cardiovascular benefits, improves mood through endorphin release, and helps maintain healthy body composition.',
+      id: "2",
+      role: "assistant",
+      content:
+        "Exercise provides cardiovascular benefits, improves mood through endorphin release, and helps maintain healthy body composition.",
     },
   ],
 });
@@ -276,22 +284,22 @@ console.log(result);
 // }
 ```
 
-### Low precision example  
+### Low precision example
 
 This example shows poor context precision with mostly irrelevant context:
 
 ```typescript
-import { createContextPrecisionScorer } from '@mastra/evals';
+import { createContextPrecisionScorer } from "@mastra/evals";
 
 const scorer = createContextPrecisionScorer({
-  model: 'openai/gpt-4o-mini',
+  model: "openai/gpt-4o-mini",
   options: {
     context: [
-      'The weather forecast shows sunny skies this weekend.',
-      'Coffee is one of the world\'s most popular beverages.',
-      'Machine learning requires large amounts of training data.',
-      'Cats typically sleep 12-16 hours per day.',
-      'The capital of France is Paris.',
+      "The weather forecast shows sunny skies this weekend.",
+      "Coffee is one of the world's most popular beverages.",
+      "Machine learning requires large amounts of training data.",
+      "Cats typically sleep 12-16 hours per day.",
+      "The capital of France is Paris.",
     ],
     scale: 1,
   },
@@ -301,17 +309,18 @@ const result = await scorer.run({
   input: {
     inputMessages: [
       {
-        id: '1',
-        role: 'user',
-        content: 'How does photosynthesis work?',
+        id: "1",
+        role: "user",
+        content: "How does photosynthesis work?",
       },
     ],
   },
   output: [
     {
-      id: '2',
-      role: 'assistant',
-      content: 'Photosynthesis is the process by which plants convert sunlight into energy using chlorophyll.',
+      id: "2",
+      role: "assistant",
+      content:
+        "Photosynthesis is the process by which plants convert sunlight into energy using chlorophyll.",
     },
   ],
 });
@@ -328,16 +337,16 @@ console.log(result);
 
 Choose the right scorer for your needs:
 
-| Use Case | Context Relevance | Context Precision |
-|----------|-------------------|-------------------|
-| **RAG evaluation** | When usage matters | When ranking matters |
-| **Context quality** | Nuanced levels | Binary relevance |
-| **Missing detection** | ✓ Identifies gaps | ✗ Not evaluated |
-| **Usage tracking** | ✓ Tracks utilization | ✗ Not considered |
-| **Position sensitivity** | ✗ Position agnostic | ✓ Rewards early placement |
+| Use Case                 | Context Relevance    | Context Precision         |
+| ------------------------ | -------------------- | ------------------------- |
+| **RAG evaluation**       | When usage matters   | When ranking matters      |
+| **Context quality**      | Nuanced levels       | Binary relevance          |
+| **Missing detection**    | ✓ Identifies gaps    | ✗ Not evaluated           |
+| **Usage tracking**       | ✓ Tracks utilization | ✗ Not considered          |
+| **Position sensitivity** | ✗ Position agnostic  | ✓ Rewards early placement |
 
 ## Related
 
 - [Answer Relevancy Scorer](/reference/scorers/answer-relevancy) - Evaluates if answers address the question
 - [Faithfulness Scorer](/reference/scorers/faithfulness) - Measures answer groundedness in context
-- [Custom Scorers](/docs/scorers/custom-scorers) - Creating your own evaluation metrics
+- [Custom Scorers](/docs/scorers/custom-scorers) - Creating your own evaluation metrics