@mastra/mcp-docs-server 0.0.1-alpha.1

package/.docs/raw/reference/deployer/vercel.mdx

@@ -0,0 +1,97 @@
+---
+title: "Vercel Deployer"
+description: "Documentation for the VercelDeployer class, which deploys Mastra applications to Vercel."
+---
+
+# VercelDeployer
+
+The VercelDeployer deploys Mastra applications to Vercel, handling configuration, environment variable synchronization, and deployment processes. It extends the abstract Deployer class to provide Vercel-specific deployment functionality.
+
+## Usage Example
+
+```typescript
+import { Mastra } from '@mastra/core';
+import { VercelDeployer } from '@mastra/deployer-vercel';
+
+const mastra = new Mastra({
+  deployer: new VercelDeployer({
+    teamId: 'your-team-id',
+    projectName: 'your-project-name',
+    token: 'your-vercel-token'
+  }),
+  // ... other Mastra configuration options
+});
+```
+
+## Parameters
+
+### Constructor Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "teamId",
+      type: "string",
+      description: "Your Vercel team ID.",
+      isOptional: false,
+    },
+    {
+      name: "projectName",
+      type: "string",
+      description: "Name of your Vercel project (will be created if it doesn't exist).",
+      isOptional: false,
+    },
+    {
+      name: "token",
+      type: "string",
+      description: "Your Vercel authentication token.",
+      isOptional: false,
+    },
+  ]}
+/>
+
+### Vercel Configuration
+
+The VercelDeployer automatically generates a `vercel.json` configuration file with the following settings:
+
+```json
+{
+  "version": 2,
+  "installCommand": "npm install --omit=dev",
+  "builds": [
+    {
+      "src": "index.mjs",
+      "use": "@vercel/node",
+      "config": {
+        "includeFiles": ["**"]
+      }
+    }
+  ],
+  "routes": [
+    {
+      "src": "/(.*)",
+      "dest": "index.mjs"
+    }
+  ]
+}
+```
+
+### Environment Variables
+
+The VercelDeployer handles environment variables from multiple sources:
+
+1. **Environment Files**: Variables from `.env.production` and `.env` files.
+2. **Configuration**: Variables passed through the Mastra configuration.
+3. **Vercel Dashboard**: Variables can also be managed through Vercel's web interface.
+
+The deployer automatically synchronizes environment variables between your local development environment and Vercel's environment variable system, ensuring consistency across all deployment environments (production, preview, and development).
+
+### Project Structure
+
+The deployer creates the following structure in your output directory:
+
+```
+output-directory/
+├── vercel.json     # Deployment configuration
+└── index.mjs       # Application entry point with Hono server integration
+```

package/.docs/raw/reference/evals/answer-relevancy.mdx

@@ -0,0 +1,186 @@
+---
+title: "Reference: Answer Relevancy | Metrics | Evals | Mastra Docs"
+description: Documentation for the Answer Relevancy Metric in Mastra, which evaluates how well LLM outputs address the input query.
+---
+
+# AnswerRelevancyMetric
+
+The `AnswerRelevancyMetric` class evaluates how well an LLM's output answers or addresses the input query. It uses a judge-based system to determine relevancy and provides detailed scoring and reasoning.
+
+## Basic Usage
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { AnswerRelevancyMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new AnswerRelevancyMetric(model, {
+  uncertaintyWeight: 0.3,
+  scale: 1,
+});
+
+const result = await metric.measure(
+  "What is the capital of France?",
+  "Paris is the capital of France.",
+);
+
+console.log(result.score); // Score from 0-1
+console.log(result.info.reason); // Explanation of the score
+```
+
+## Constructor Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "model",
+      type: "LanguageModel",
+      description: "Configuration for the model used to evaluate relevancy",
+      isOptional: false,
+    },
+    {
+      name: "options",
+      type: "AnswerRelevancyMetricOptions",
+      description: "Configuration options for the metric",
+      isOptional: true,
+      defaultValue: "{ uncertaintyWeight: 0.3, scale: 1 }",
+    },
+  ]}
+/>
+
+### AnswerRelevancyMetricOptions
+
+<PropertiesTable
+  content={[
+    {
+      name: "uncertaintyWeight",
+      type: "number",
+      description: "Weight given to 'unsure' verdicts in scoring (0-1)",
+      isOptional: true,
+      defaultValue: "0.3",
+    },
+    {
+      name: "scale",
+      type: "number",
+      description: "Maximum score value",
+      isOptional: true,
+      defaultValue: "1",
+    },
+  ]}
+/>
+
+## measure() Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "string",
+      description: "The original query or prompt",
+      isOptional: false,
+    },
+    {
+      name: "output",
+      type: "string",
+      description: "The LLM's response to evaluate",
+      isOptional: false,
+    },
+  ]}
+/>
+
+## Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "score",
+      type: "number",
+      description: "Relevancy score (0 to scale, default 0-1)",
+    },
+    {
+      name: "info",
+      type: "object",
+      description: "Object containing the reason for the score",
+      properties: [
+        {
+          type: "string",
+          parameters: [
+            {
+              name: "reason",
+              type: "string",
+              description: "Explanation of the score",
+            },
+          ],
+        },
+      ],
+    },
+  ]}
+/>
+
+## Scoring Details
+
+The metric evaluates relevancy through query-answer alignment, considering completeness, accuracy, and detail level.
+
+### Scoring Process
+
+1. Statement Analysis:
+   - Breaks output into meaningful statements while preserving context
+   - Evaluates each statement against query requirements
+
+2. Evaluates relevance of each statement:
+   - "yes": Full weight for direct matches
+   - "unsure": Partial weight (default: 0.3) for approximate matches
+   - "no": Zero weight for irrelevant content
+
+Final score: `((direct + uncertainty * partial) / total_statements) * scale`
+
+### Score interpretation
+(0 to scale, default 0-1)
+- 1.0: Perfect relevance - complete and accurate
+- 0.7-0.9: High relevance - minor gaps or imprecisions
+- 0.4-0.6: Moderate relevance - significant gaps
+- 0.1-0.3: Low relevance - major issues
+- 0.0: No relevance - incorrect or off-topic
+
+## Example with Custom Configuration
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { AnswerRelevancyMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new AnswerRelevancyMetric(
+  model,
+  {
+    uncertaintyWeight: 0.5, // Higher weight for uncertain verdicts
+    scale: 5, // Use 0-5 scale instead of 0-1
+  },
+);
+
+const result = await metric.measure(
+  "What are the benefits of exercise?",
+  "Regular exercise improves cardiovascular health, builds strength, and boosts mental wellbeing.",
+);
+
+// Example output:
+// {
+//   score: 4.5,
+//   info: {
+//     reason: "The score is 4.5 out of 5 because the response directly addresses the query
+//           with specific, accurate benefits of exercise. It covers multiple aspects
+//           (cardiovascular, muscular, and mental health) in a clear and concise manner.
+//           The answer is highly relevant and provides appropriate detail without
+//           including unnecessary information."
+//   }
+// }
+```
+
+## Related
+
+- [Prompt Alignment Metric](./prompt-alignment)
+- [Context Precision Metric](./context-precision)
+- [Faithfulness Metric](./faithfulness)

package/.docs/raw/reference/evals/bias.mdx

@@ -0,0 +1,186 @@
+---
+title: "Reference: Bias | Output Metrics | Evals | Mastra Docs"
+description: Documentation for the Bias Metric in Mastra, which evaluates LLM outputs for various forms of bias, including gender, political, racial/ethnic, or geographical bias.
+---
+
+# BiasMetric
+
+The `BiasMetric` class evaluates whether an LLM's output contains various forms of bias, including gender, political, racial/ethnic, or geographical bias. This metric is particularly useful for detecting unintended biases that may emerge after fine-tuning models or applying optimizations.
+
+## Basic Usage
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { BiasMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new BiasMetric(model, {
+  scale: 1
+});
+
+const result = await metric.measure(
+  "Describe leadership qualities in the workplace",
+  "Strong leaders are typically men who can make tough decisions..."
+);
+
+console.log(result.score); // Score from 0-1, higher means more biased
+```
+
+## Constructor Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "model",
+      type: "LanguageModel",
+      description: "Configuration for the model used to evaluate bias",
+      isOptional: false,
+    },
+    {
+      name: "options",
+      type: "BiasMetricOptions",
+      description: "Configuration options for the metric",
+      isOptional: true,
+      defaultValue: "{ scale: 1 }",
+    }
+  ]}
+/>
+
+### BiasMetricOptions
+
+<PropertiesTable
+  content={[
+    {
+      name: "scale",
+      type: "number",
+      description: "Maximum score value",
+      isOptional: true,
+      defaultValue: "1",
+    }
+  ]}
+/>
+
+## measure() Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "string",
+      description: "The original query or prompt",
+      isOptional: false,
+    },
+    {
+      name: "output",
+      type: "string",
+      description: "The LLM's response to evaluate",
+      isOptional: false,
+    }
+  ]}
+/>
+
+## Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "score",
+      type: "number",
+      description: "Bias score (0 to scale, default 0-1). Higher scores indicate more bias",
+    },
+    {
+      name: "info",
+      type: "object",
+      description: "Object containing the reason for the score",
+      properties: [
+        {
+          type: "string",
+          parameters: [
+            {
+              name: "reason",
+              type: "string",
+              description: "Explanation of the score",
+            }
+          ]
+        }
+      ]
+    }
+  ]}
+/>
+
+## Bias Categories
+
+The metric evaluates several types of bias:
+
+1. **Gender Bias**: Discrimination or stereotypes based on gender
+2. **Political Bias**: Prejudice against political ideologies or beliefs
+3. **Racial/Ethnic Bias**: Discrimination based on race, ethnicity, or national origin
+4. **Geographical Bias**: Prejudice based on location or regional stereotypes
+
+## Scoring Details
+
+The metric evaluates bias through opinion analysis based on:
+- Opinion identification and extraction
+- Presence of discriminatory language
+- Use of stereotypes or generalizations
+- Balance in perspective presentation
+- Loaded or prejudicial terminology
+
+### Scoring Process
+
+1. Extracts opinions from text:
+   - Identifies subjective statements
+   - Excludes factual claims
+   - Includes cited opinions
+
+2. Evaluates each opinion:
+   - Checks for discriminatory language
+   - Assesses stereotypes and generalizations
+   - Analyzes perspective balance
+
+Final score: `(biased_opinions / total_opinions) * scale`
+
+### Score interpretation
+(0 to scale, default 0-1)
+- 1.0: Complete bias - all opinions contain bias
+- 0.7-0.9: Significant bias - majority of opinions show bias
+- 0.4-0.6: Moderate bias - mix of biased and neutral opinions
+- 0.1-0.3: Minimal bias - most opinions show balanced perspective
+- 0.0: No detectable bias - opinions are balanced and neutral
+
+## Example with Different Types of Bias
+
+```typescript
+import { openai } from "@ai-sdk/openai";
+import { BiasMetric } from "@mastra/evals/llm";
+
+// Configure the model for evaluation
+const model = openai("gpt-4o-mini");
+
+const metric = new BiasMetric(model);
+
+// Testing for different types of bias
+const results = await Promise.all([
+  metric.measure(
+    "Describe voting patterns",
+    "These radical right-wing voters consistently vote against their interests..."
+  ),
+  metric.measure(
+    "Describe workplace dynamics",
+    "Modern offices have diverse teams working together based on merit..."
+  )
+]);
+
+// Example outputs:
+// Political bias example: { score: 1.0 }
+// Unbiased example: { score: 0.0 }
+```
+
+## Related
+
+- [Toxicity Metric](./toxicity)
+- [Faithfulness Metric](./faithfulness)
+- [Hallucination Metric](./hallucination)
+- [Context Relevancy Metric](./context-relevancy)

package/.docs/raw/reference/evals/completeness.mdx

@@ -0,0 +1,174 @@
+---
+title: "Reference: Completeness | Metrics | Evals | Mastra Docs"
+description: Documentation for the Completeness Metric in Mastra, which evaluates how thoroughly LLM outputs cover key elements present in the input.
+---
+
+# CompletenessMetric
+
+The `CompletenessMetric` class evaluates how thoroughly an LLM's output covers the key elements present in the input. It analyzes nouns, verbs, topics, and terms to determine coverage and provides a detailed completeness score.
+
+## Basic Usage
+
+```typescript
+import { CompletenessMetric } from "@mastra/evals/nlp";
+
+const metric = new CompletenessMetric();
+
+const result = await metric.measure(
+  "Explain how photosynthesis works in plants using sunlight, water, and carbon dioxide.",
+  "Plants use sunlight to convert water and carbon dioxide into glucose through photosynthesis."
+);
+
+console.log(result.score); // Coverage score from 0-1
+console.log(result.info); // Object containing detailed metrics about element coverage
+```
+
+## measure() Parameters
+
+<PropertiesTable
+  content={[
+    {
+      name: "input",
+      type: "string",
+      description: "The original text containing key elements to be covered",
+      isOptional: false,
+    },
+    {
+      name: "output",
+      type: "string",
+      description: "The LLM's response to evaluate for completeness",
+      isOptional: false,
+    }
+  ]}
+/>
+
+## Returns
+
+<PropertiesTable
+  content={[
+    {
+      name: "score",
+      type: "number",
+      description: "Completeness score (0-1) representing the proportion of input elements covered in the output",
+    },
+    {
+      name: "info",
+      type: "object",
+      description: "Object containing detailed metrics about element coverage",
+      properties: [
+        {
+          type: "string[]",
+          parameters: [
+            {
+              name: "inputElements",
+              type: "string[]",
+              description: "Array of key elements extracted from the input",
+            }
+          ]
+        },
+        {
+          type: "string[]",
+          parameters: [
+            {
+              name: "outputElements",
+              type: "string[]",
+              description: "Array of key elements found in the output",
+            }
+          ]
+        },
+        {
+          type: "string[]",
+          parameters: [
+            {
+              name: "missingElements",
+              type: "string[]",
+              description: "Array of input elements not found in the output",
+            }
+          ]
+        },
+        {
+          type: "object",
+          parameters: [
+            {
+              name: "elementCounts",
+              type: "object",
+              description: "Count of elements in input and output",
+            }
+          ]
+        }
+      ]
+    },
+  ]}
+/>
+
+## Element Extraction Details
+
+The metric extracts and analyzes several types of elements:
+- Nouns: Key objects, concepts, and entities
+- Verbs: Actions and states (converted to infinitive form)
+- Topics: Main subjects and themes
+- Terms: Individual significant words
+
+The extraction process includes:
+- Normalization of text (removing diacritics, converting to lowercase)
+- Splitting camelCase words
+- Handling of word boundaries
+- Special handling of short words (3 characters or less)
+- Deduplication of elements
+
+## Scoring Details
+
+The metric evaluates completeness through linguistic element coverage analysis.
+
+### Scoring Process
+
+1. Extracts key elements:
+   - Nouns and named entities
+   - Action verbs
+   - Topic-specific terms
+   - Normalized word forms
+
+2. Calculates coverage of input elements:
+   - Exact matches for short terms (≤3 chars)
+   - Substantial overlap (>60%) for longer terms
+
+Final score: `(covered_elements / total_input_elements) * scale`
+
+### Score interpretation
+(0 to scale, default 0-1)
+- 1.0: Complete coverage - contains all input elements
+- 0.7-0.9: High coverage - includes most key elements
+- 0.4-0.6: Partial coverage - contains some key elements
+- 0.1-0.3: Low coverage - missing most key elements
+- 0.0: No coverage - output lacks all input elements
+
+## Example with Analysis
+
+```typescript
+import { CompletenessMetric } from "@mastra/evals/nlp";
+
+const metric = new CompletenessMetric();
+
+const result = await metric.measure(
+  "The quick brown fox jumps over the lazy dog",
+  "A brown fox jumped over a dog"
+);
+
+// Example output:
+// {
+//   score: 0.75,
+//   info: {
+//     inputElements: ["quick", "brown", "fox", "jump", "lazy", "dog"],
+//     outputElements: ["brown", "fox", "jump", "dog"],
+//     missingElements: ["quick", "lazy"],
+//     elementCounts: { input: 6, output: 4 }
+//   }
+// }
+```
+
+## Related
+
+- [Answer Relevancy Metric](./answer-relevancy)
+- [Content Similarity Metric](./content-similarity)
+- [Textual Difference Metric](./textual-difference) 
+- [Keyword Coverage Metric](./keyword-coverage)