@mastra/mcp-docs-server 0.0.0-commonjs-20250414101718

package/.docs/raw/reference/evals/content-similarity.mdx

@@ -0,0 +1,183 @@
+---
+title: "Reference: Content Similarity | Evals | Mastra Docs"
+description: Documentation for the Content Similarity Metric in Mastra, which measures textual similarity between strings and provides a matching score.
+---
+
+# ContentSimilarityMetric
+
+The `ContentSimilarityMetric` class measures the textual similarity between two strings, providing a score that indicates how closely they match. It supports configurable options for case sensitivity and whitespace handling.
+
+## Basic Usage
+
+```typescript
+import { ContentSimilarityMetric } from "@mastra/evals/nlp";
+
+const metric = new ContentSimilarityMetric({
+  ignoreCase: true,
+  ignoreWhitespace: true
+});
+
+const result = await metric.measure(
+  "Hello, world!",
+  "hello world"
+);
+
+console.log(result.score); // Similarity score from 0-1
+console.log(result.info); // Detailed similarity metrics
+```
+
+## Constructor Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "options",
+      type: "ContentSimilarityOptions",
+      description: "Configuration options for similarity comparison",
+      isOptional: true,
+      defaultValue: "{ ignoreCase: true, ignoreWhitespace: true }",
+    }
+  ]}
+/>
+
+### ContentSimilarityOptions
+
+<PropertiesTable
+  content={[
+    {
+      name: "ignoreCase",
+      type: "boolean",
+      description: "Whether to ignore case differences when comparing strings",
+      isOptional: true,
+      defaultValue: "true",
+    },
+    {
+      name: "ignoreWhitespace",
+      type: "boolean",
+      description: "Whether to normalize whitespace when comparing strings",
+      isOptional: true,
+      defaultValue: "true",
+    }
+  ]}
+/>
+
+## measure() Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "string",
+      description: "The reference text to compare against",
+      isOptional: false,
+    },
+    {
+      name: "output",
+      type: "string",
+      description: "The text to evaluate for similarity",
+      isOptional: false,
+    }
+  ]}
+/>
+
+## Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "score",
+      type: "number",
+      description: "Similarity score (0-1) where 1 indicates perfect similarity",
+    },
+    {
+      name: "info",
+      type: "object",
+      description: "Detailed similarity metrics",
+      properties: [
+        {
+          type: "number",
+          parameters: [
+            {
+              name: "similarity",
+              type: "number",
+              description: "Raw similarity score between the two texts",
+            }
+          ]
+        }
+      ]
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The metric evaluates textual similarity through character-level matching and configurable text normalization.
+
+### Scoring Process
+
+1. Normalizes text:
+   - Case normalization (if ignoreCase: true)
+   - Whitespace normalization (if ignoreWhitespace: true)
+
+2. Compares processed strings using string-similarity algorithm:
+   - Analyzes character sequences
+   - Aligns word boundaries
+   - Considers relative positions
+   - Accounts for length differences
+
+Final score: `similarity_value * scale`
+
+### Score interpretation
+(0 to scale, default 0-1)
+- 1.0: Perfect match - identical texts
+- 0.7-0.9: High similarity - mostly matching content
+- 0.4-0.6: Moderate similarity - partial matches
+- 0.1-0.3: Low similarity - few matching patterns
+- 0.0: No similarity - completely different texts
+
+## Example with Different Options
+
+```typescript
+import { ContentSimilarityMetric } from "@mastra/evals/nlp";
+
+// Case-sensitive comparison
+const caseSensitiveMetric = new ContentSimilarityMetric({
+  ignoreCase: false,
+  ignoreWhitespace: true
+});
+
+const result1 = await caseSensitiveMetric.measure(
+  "Hello World",
+  "hello world"
+); // Lower score due to case difference
+
+// Example output:
+// {
+//   score: 0.75,
+//   info: { similarity: 0.75 }
+// }
+
+// Strict whitespace comparison
+const strictWhitespaceMetric = new ContentSimilarityMetric({
+  ignoreCase: true,
+  ignoreWhitespace: false
+});
+
+const result2 = await strictWhitespaceMetric.measure(
+  "Hello   World",
+  "Hello World"
+); // Lower score due to whitespace difference
+
+// Example output:
+// {
+//   score: 0.85,
+//   info: { similarity: 0.85 }
+// }
+```
+
+## Related
+
+- [Completeness Metric](./completeness)
+- [Textual Difference Metric](./textual-difference) 
+- [Answer Relevancy Metric](./answer-relevancy)
+- [Keyword Coverage Metric](./keyword-coverage)

package/.docs/raw/reference/evals/context-position.mdx

@@ -0,0 +1,190 @@
+---
+title: "Reference: Context Position | Metrics | Evals | Mastra Docs"
+description: Documentation for the Context Position Metric in Mastra, which evaluates the ordering of context nodes based on their relevance to the query and output.
+---
+
+# ContextPositionMetric
+
+The `ContextPositionMetric` class evaluates how well context nodes are ordered based on their relevance to the query and output. It uses position-weighted scoring to emphasize the importance of having the most relevant context pieces appear earlier in the sequence.
+
+## Basic Usage
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { ContextPositionMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new ContextPositionMetric(model, {
+  context: [
+    "Photosynthesis is a biological process used by plants to create energy from sunlight.",
+    "The process of photosynthesis produces oxygen as a byproduct.",
+    "Plants need water and nutrients from the soil to grow.",
+  ],
+});
+
+const result = await metric.measure(
+  "What is photosynthesis?",
+  "Photosynthesis is the process by which plants convert sunlight into energy.",
+);
+
+console.log(result.score); // Position score from 0-1
+console.log(result.info.reason); // Explanation of the score
+```
+
+## Constructor Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "model",
+      type: "ModelConfig",
+      description:
+        "Configuration for the model used to evaluate context positioning",
+      isOptional: false,
+    },
+    {
+      name: "options",
+      type: "ContextPositionMetricOptions",
+      description: "Configuration options for the metric",
+      isOptional: false,
+    },
+  ]}
+/>
+
+### ContextPositionMetricOptions
+
+<PropertiesTable
+  content={[
+    {
+      name: "scale",
+      type: "number",
+      description: "Maximum score value",
+      isOptional: true,
+      defaultValue: "1",
+    },
+    {
+      name: "context",
+      type: "string[]",
+      description: "Array of context pieces in their retrieval order",
+      isOptional: false,
+    },
+  ]}
+/>
+
+## measure() Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "string",
+      description: "The original query or prompt",
+      isOptional: false,
+    },
+    {
+      name: "output",
+      type: "string",
+      description: "The generated response to evaluate",
+      isOptional: false,
+    },
+  ]}
+/>
+
+## Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "score",
+      type: "number",
+      description: "Position score (0 to scale, default 0-1)",
+    },
+    {
+      name: "info",
+      type: "object",
+      description: "Object containing the reason for the score",
+      properties: [
+        {
+          type: "string",
+          parameters: [
+            {
+              name: "reason",
+              type: "string",
+              description: "Detailed explanation of the score",
+            },
+          ],
+        },
+      ],
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The metric evaluates context positioning through binary relevance assessment and position-based weighting.
+
+### Scoring Process
+
+1. Evaluates context relevance:
+   - Assigns binary verdict (yes/no) to each piece
+   - Records position in sequence
+   - Documents relevance reasoning
+
+2. Applies position weights:
+   - Earlier positions weighted more heavily (weight = 1/(position + 1))
+   - Sums weights of relevant pieces
+   - Normalizes by maximum possible score
+
+Final score: `(weighted_sum / max_possible_sum) * scale`
+
+### Score interpretation
+(0 to scale, default 0-1)
+- 1.0: Optimal - most relevant context first
+- 0.7-0.9: Good - relevant context mostly early
+- 0.4-0.6: Mixed - relevant context scattered
+- 0.1-0.3: Suboptimal - relevant context mostly later
+- 0.0: Poor ordering - relevant context at end or missing
+
+## Example with Analysis
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { ContextPositionMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new ContextPositionMetric(model, {
+  context: [
+    "A balanced diet is important for health.",
+    "Exercise strengthens the heart and improves blood circulation.",
+    "Regular physical activity reduces stress and anxiety.",
+    "Exercise equipment can be expensive.",
+  ],
+});
+
+const result = await metric.measure(
+  "What are the benefits of exercise?",
+  "Regular exercise improves cardiovascular health and mental wellbeing.",
+);
+
+// Example output:
+// {
+//   score: 0.5,
+//   info: {
+//     reason: "The score is 0.5 because while the second and third contexts are highly
+//           relevant to the benefits of exercise, they are not optimally positioned at
+//           the beginning of the sequence. The first and last contexts are not relevant
+//           to the query, which impacts the position-weighted scoring."
+//   }
+// }
+```
+
+## Related
+
+- [Context Precision Metric](./context-precision)
+- [Answer Relevancy Metric](./answer-relevancy)
+- [Completeness Metric](./completeness)
++ [Context Relevancy Metric](./context-relevancy)

package/.docs/raw/reference/evals/context-precision.mdx

@@ -0,0 +1,189 @@
+---
+title: "Reference: Context Precision | Metrics | Evals | Mastra Docs"
+description: Documentation for the Context Precision Metric in Mastra, which evaluates the relevance and precision of retrieved context nodes for generating expected outputs.
+---
+
+# ContextPrecisionMetric
+
+The `ContextPrecisionMetric` class evaluates how relevant and precise the retrieved context nodes are for generating the expected output. It uses a judge-based system to analyze each context piece's contribution and provides weighted scoring based on position.
+
+## Basic Usage
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { ContextPrecisionMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new ContextPrecisionMetric(model, {
+  context: [
+    "Photosynthesis is a biological process used by plants to create energy from sunlight.",
+    "Plants need water and nutrients from the soil to grow.",
+    "The process of photosynthesis produces oxygen as a byproduct.",
+  ],
+});
+
+const result = await metric.measure(
+  "What is photosynthesis?",
+  "Photosynthesis is the process by which plants convert sunlight into energy.",
+);
+
+console.log(result.score); // Precision score from 0-1
+console.log(result.info.reason); // Explanation of the score
+```
+
+## Constructor Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "model",
+      type: "LanguageModel",
+      description:
+        "Configuration for the model used to evaluate context relevance",
+      isOptional: false,
+    },
+    {
+      name: "options",
+      type: "ContextPrecisionMetricOptions",
+      description: "Configuration options for the metric",
+      isOptional: false,
+    },
+  ]}
+/>
+
+### ContextPrecisionMetricOptions
+
+<PropertiesTable
+  content={[
+    {
+      name: "scale",
+      type: "number",
+      description: "Maximum score value",
+      isOptional: true,
+      defaultValue: "1",
+    },
+    {
+      name: "context",
+      type: "string[]",
+      description: "Array of context pieces in their retrieval order",
+      isOptional: false,
+    },
+  ]}
+/>
+
+## measure() Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "string",
+      description: "The original query or prompt",
+      isOptional: false,
+    },
+    {
+      name: "output",
+      type: "string",
+      description: "The generated response to evaluate",
+      isOptional: false,
+    },
+  ]}
+/>
+
+## Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "score",
+      type: "number",
+      description: "Precision score (0 to scale, default 0-1)",
+    },
+    {
+      name: "info",
+      type: "object",
+      description: "Object containing the reason for the score",
+      properties: [
+        {
+          type: "string",
+          parameters: [
+            {
+              name: "reason",
+              type: "string",
+              description: "Detailed explanation of the score",
+            },
+          ],
+        },
+      ],
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The metric evaluates context precision through binary relevance assessment and Mean Average Precision (MAP) scoring.
+
+### Scoring Process
+
+1. Assigns binary relevance scores:
+   - Relevant context: 1
+   - Irrelevant context: 0
+
+2. Calculates Mean Average Precision:
+   - Computes precision at each position
+   - Weights earlier positions more heavily
+   - Normalizes to configured scale
+
+Final score: `Mean Average Precision * scale`
+
+### Score interpretation
+(0 to scale, default 0-1)
+- 1.0: All relevant context in optimal order
+- 0.7-0.9: Mostly relevant context with good ordering
+- 0.4-0.6: Mixed relevance or suboptimal ordering
+- 0.1-0.3: Limited relevance or poor ordering
+- 0.0: No relevant context
+
+## Example with Analysis
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { ContextPrecisionMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new ContextPrecisionMetric(model, {
+  context: [
+    "Exercise strengthens the heart and improves blood circulation.",
+    "A balanced diet is important for health.",
+    "Regular physical activity reduces stress and anxiety.",
+    "Exercise equipment can be expensive.",
+  ],
+});
+
+const result = await metric.measure(
+  "What are the benefits of exercise?",
+  "Regular exercise improves cardiovascular health and mental wellbeing.",
+);
+
+// Example output:
+// {
+//   score: 0.75,
+//   info: {
+//     reason: "The score is 0.75 because the first and third contexts are highly relevant
+//           to the benefits mentioned in the output, while the second and fourth contexts
+//           are not directly related to exercise benefits. The relevant contexts are well-positioned
+//           at the beginning and middle of the sequence."
+//   }
+// }
+```
+
+## Related
+
+- [Answer Relevancy Metric](./answer-relevancy)
+- [Context Position Metric](./context-position)
+- [Completeness Metric](./completeness)
+- [Context Relevancy Metric](./context-relevancy)