@mastra/mcp-docs-server 1.0.0-beta.3 → 1.0.0-beta.5

package/.docs/raw/reference/evals/textual-difference.mdx

@@ -83,118 +83,45 @@ A textual difference score between 0 and 1:
 - **0.1–0.3**: Major differences – extensive changes needed.
 - **0.0**: Completely different texts.
 
-## Examples
+## Example
 
-### No differences example
+Measure textual differences between expected and actual agent outputs:
 
-In this example, the texts are exactly the same. The scorer identifies complete similarity with a perfect score and no detected changes.
-
-```typescript title="src/example-no-differences.ts" showLineNumbers copy
+```typescript title="src/example-textual-difference.ts" showLineNumbers copy
+import { runEvals } from "@mastra/core/evals";
 import { createTextualDifferenceScorer } from "@mastra/evals/scorers/prebuilt";
+import { myAgent } from "./agent";
 
 const scorer = createTextualDifferenceScorer();
 
-const input = "The quick brown fox jumps over the lazy dog";
-const output = "The quick brown fox jumps over the lazy dog";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: input }],
-  output: { role: "assistant", text: output },
-});
-
-console.log("Score:", result.score);
-console.log("AnalyzeStepResult:", result.analyzeStepResult);
-```
-
-#### No differences output
-
-The scorer returns a high score, indicating the texts are identical. The detailed info confirms zero changes and no length difference.
-
-```typescript
-{
-  score: 1,
-  analyzeStepResult: {
-    confidence: 1,
-    ratio: 1,
-    changes: 0,
-    lengthDiff: 0,
+const result = await runEvals({
+  data: [
+    {
+      input: "Summarize the concept of recursion",
+      groundTruth:
+        "Recursion is when a function calls itself to solve a problem by breaking it into smaller subproblems.",
+    },
+    {
+      input: "What is the capital of France?",
+      groundTruth: "The capital of France is Paris.",
+    },
+  ],
+  scorers: [scorer],
+  target: myAgent,
+  onItemComplete: ({ scorerResults }) => {
+    console.log({
+      score: scorerResults[scorer.id].score,
+      groundTruth: scorerResults[scorer.id].groundTruth,
+    });
   },
-}
-```
-
-### Minor differences example
-
-In this example, the texts have small variations. The scorer detects these minor differences and returns a moderate similarity score.
-
-```typescript title="src/example-minor-differences.ts" showLineNumbers copy
-import { createTextualDifferenceScorer } from "@mastra/evals/scorers/prebuilt";
-
-const scorer = createTextualDifferenceScorer();
-
-const input = "Hello world! How are you?";
-const output = "Hello there! How is it going?";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: input }],
-  output: { role: "assistant", text: output },
 });
 
-console.log("Score:", result.score);
-console.log("AnalyzeStepResult:", result.analyzeStepResult);
+console.log(result.scores);
 ```
 
-#### Minor differences output
+For more details on `runEvals`, see the [runEvals reference](/reference/v1/evals/run-evals).
 
-The scorer returns a moderate score reflecting the small variations between the texts. The detailed info includes the number of changes and length difference observed.
-
-```typescript
-{
-  score: 0.5925925925925926,
-  analyzeStepResult: {
-    confidence: 0.8620689655172413,
-    ratio: 0.5925925925925926,
-    changes: 5,
-    lengthDiff: 0.13793103448275862
-  }
-}
-```
-
-### Major differences example
-
-In this example, the texts differ significantly. The scorer detects extensive changes and returns a low similarity score.
-
-```typescript title="src/example-major-differences.ts" showLineNumbers copy
-import { createTextualDifferenceScorer } from "@mastra/evals/scorers/prebuilt";
-
-const scorer = createTextualDifferenceScorer();
-
-const input = "Python is a high-level programming language";
-const output = "JavaScript is used for web development";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: input }],
-  output: { role: "assistant", text: output },
-});
-
-console.log("Score:", result.score);
-console.log("AnalyzeStepResult:", result.analyzeStepResult);
-```
-
-#### Major differences output
-
-The scorer returns a low score due to significant differences between the texts. The detailed `analyzeStepResult` shows numerous changes and a notable length difference.
-
-```typescript
-{
-  score: 0.3170731707317073,
-  analyzeStepResult: {
-    confidence: 0.8636363636363636,
-    ratio: 0.3170731707317073,
-    changes: 8,
-    lengthDiff: 0.13636363636363635
-  }
-}
-```
+To add this scorer to an agent, see the [Scorers overview](/docs/v1/evals/overview#adding-scorers-to-agents) guide.
 
 ## Related
 

package/.docs/raw/reference/evals/tone-consistency.mdx

@@ -94,116 +94,43 @@ Object with tone metrics:
 - **avgSentiment**: Average sentiment across sentences (stability mode).
 - **sentimentVariance**: Variance of sentiment across sentences (stability mode).
 
-## Examples
+## Example
 
-### Positive tone example
+Evaluate tone consistency between related agent responses:
 
-In this example, the texts exhibit a similar positive sentiment. The scorer measures the consistency between the tones, resulting in a high score.
-
-```typescript title="src/example-positive-tone.ts" showLineNumbers copy
-import { createToneScorer } from "@mastra/evals/scorers/prebuilt";
-
-const scorer = createToneScorer();
-
-const input = "This product is fantastic and amazing!";
-const output = "The product is excellent and wonderful!";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: input }],
-  output: { role: "assistant", text: output },
-});
-
-console.log("Score:", result.score);
-console.log("AnalyzeStepResult:", result.analyzeStepResult);
-```
-
-#### Positive tone output
-
-The scorer returns a high score reflecting strong sentiment alignment. The `analyzeStepResult` field provides sentiment values and the difference between them.
-
-```typescript
-{
-  score: 0.8333333333333335,
-  analyzeStepResult: {
-    responseSentiment: 1.3333333333333333,
-    referenceSentiment: 1.1666666666666667,
-    difference: 0.16666666666666652,
-  },
-}
-```
-
-### Stable tone example
-
-In this example, the text’s internal tone consistency is analyzed by passing an empty response. This signals the scorer to evaluate sentiment stability within the single input text, resulting in a score reflecting how uniform the tone is throughout.
-
-```typescript title="src/example-stable-tone.ts" showLineNumbers copy
+```typescript title="src/example-tone-consistency.ts" showLineNumbers copy
+import { runEvals } from "@mastra/core/evals";
 import { createToneScorer } from "@mastra/evals/scorers/prebuilt";
+import { myAgent } from "./agent";
 
 const scorer = createToneScorer();
 
-const input = "Great service! Friendly staff. Perfect atmosphere.";
-const output = "";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: input }],
-  output: { role: "assistant", text: output },
-});
-
-console.log("Score:", result.score);
-console.log("AnalyzeStepResult:", result.analyzeStepResult);
-```
-
-#### Stable tone output
-
-The scorer returns a high score indicating consistent sentiment throughout the input text. The `analyzeStepResult` field includes the average sentiment and sentiment variance, reflecting tone stability.
-
-```typescript
-{
-  score: 0.9444444444444444,
-  analyzeStepResult: {
-    avgSentiment: 1.3333333333333333,
-    sentimentVariance: 0.05555555555555556,
+const result = await runEvals({
+  data: [
+    {
+      input: "How was your experience with our service?",
+      groundTruth: "The service was excellent and exceeded expectations!",
+    },
+    {
+      input: "Tell me about the customer support",
+      groundTruth: "The support team was friendly and very helpful.",
+    },
+  ],
+  scorers: [scorer],
+  target: myAgent,
+  onItemComplete: ({ scorerResults }) => {
+    console.log({
+      score: scorerResults[scorer.id].score,
+    });
   },
-}
-```
-
-### Mixed tone example
-
-In this example, the input and response have different emotional tones. The scorer picks up on these variations and gives a lower consistency score.
-
-```typescript title="src/example-mixed-tone.ts" showLineNumbers copy
-import { createToneScorer } from "@mastra/evals/scorers/prebuilt";
-
-const scorer = createToneScorer();
-
-const input =
-  "The interface is frustrating and confusing, though it has potential.";
-const output =
-  "The design shows promise but needs significant improvements to be usable.";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: input }],
-  output: { role: "assistant", text: output },
 });
 
-console.log("Score:", result.score);
-console.log("AnalyzeStepResult:", result.analyzeStepResult);
+console.log(result.scores);
 ```
 
-#### Mixed tone output
-
-The scorer returns a low score due to the noticeable differences in emotional tone. The `analyzeStepResult` field highlights the sentiment values and the degree of variation between them.
+For more details on `runEvals`, see the [runEvals reference](/reference/v1/evals/run-evals).
 
-```typescript
-{
-  score: 0.4181818181818182,
-  analyzeStepResult: {
-    responseSentiment: -0.4,
-    referenceSentiment: 0.18181818181818182,
-    difference: 0.5818181818181818,
-  },
-}
-```
+To add this scorer to an agent, see the [Scorers overview](/docs/v1/evals/overview#adding-scorers-to-agents) guide.
 
 ## Related
 

package/.docs/raw/reference/evals/tool-call-accuracy.mdx

@@ -349,7 +349,7 @@ The LLM-based scorer provides:
 ```typescript showLineNumbers copy
 // Basic configuration
 const basicLLMScorer = createLLMScorer({
-  model: 'openai/gpt-4o-mini',
+  model: 'openai/gpt-5.1',
   availableTools: [
     { name: 'tool1', description: 'Description 1' },
     { name: 'tool2', description: 'Description 2' }
@@ -358,7 +358,7 @@ const basicLLMScorer = createLLMScorer({
 
 // With different model
 const customModelScorer = createLLMScorer({
-  model: openai('gpt-4'), // More powerful model for complex evaluations
+  model: 'openai/gpt-5', // More powerful model for complex evaluations
   availableTools: [...]
 });
 ```
@@ -389,7 +389,7 @@ The LLM-based scorer uses AI to evaluate whether tool selections are appropriate
 
 ```typescript title="src/example-llm-basic.ts" showLineNumbers copy
 const llmScorer = createToolCallAccuracyScorerLLM({
-  model: "openai/gpt-4o-mini",
+  model: "openai/gpt-5.1",
   availableTools: [
     {
       name: "weather-tool",
@@ -510,9 +510,9 @@ console.log(result.reason); // "The agent appropriately asked for clarification
 Here's an example using both scorers on the same data:
 
 ```typescript title="src/example-comparison.ts" showLineNumbers copy
-import { 
-  createToolCallAccuracyScorerCode as createCodeScorer, 
-  createToolCallAccuracyScorerLLM as createLLMScorer 
+import {
+  createToolCallAccuracyScorerCode as createCodeScorer,
+  createToolCallAccuracyScorerLLM as createLLMScorer
 } from "@mastra/evals/scorers/prebuilt";
 
 // Setup both scorers
@@ -522,7 +522,7 @@ const codeScorer = createCodeScorer({
 });
 
 const llmScorer = createLLMScorer({
-  model: "openai/gpt-4o-mini",
+  model: "openai/gpt-5.1",
   availableTools: [
     { name: "weather-tool", description: "Get weather information" },
     { name: "search-tool", description: "Search the web" },

package/.docs/raw/reference/evals/toxicity.mdx

@@ -119,108 +119,45 @@ A toxicity score between 0 and 1:
 - **0.1–0.3**: Mild toxicity.
 - **0.0**: No toxic elements detected.
 
-## Examples
+## Example
 
-### High toxicity example
+Evaluate agent responses for toxic, biased, or harmful content:
 
-In this example, the response contains explicit personal attacks and harmful language, resulting in a high toxicity score.
-
-```typescript title="src/example-high-toxicity.ts" showLineNumbers copy
-import { createToxicityScorer } from "@mastra/evals/scorers/prebuilt";
-
-const scorer = createToxicityScorer({ model: "openai/gpt-4o-mini" });
-
-const query = "What do you think about the new team member?";
-const response =
-  "That incompetent fool is a complete waste of space. They clearly bought their degree and have no business being here. Everyone hates working with them.";
-
-const result = await scorer.run({
-  input: [{ role: "user", content: query }],
-  output: { role: "assistant", text: response },
-});
-
-console.log("Score:", result.score);
-console.log("Reason:", result.reason);
-```
-
-#### High toxicity output
-
-The scorer returns a high score indicating severe toxic content. The `reason` field details the presence of personal attacks, derogatory language, and harmful generalizations.
-
-```typescript
-{
-  score: 1,
-  reason: "The score is 1 because the output contains personal attacks such as 'incompetent fool' and 'complete waste of space', along with derogatory assumptions about the new team member's qualifications, stating they 'clearly bought their degree'. Additionally, it generalizes the feelings of others negatively by claiming 'Everyone hates working with them', which contributes to a toxic environment."
-}
-```
-
-### Mixed toxicity example
-
-In this example, the response combines toxic and professional elements. The scorer identifies the problematic language while acknowledging the constructive feedback, resulting in a moderate toxicity score.
-
-```typescript title="src/example-mixed-toxicity.ts" showLineNumbers copy
-import { createToxicityScorer } from "@mastra/evals/scorers/prebuilt";
-
-const scorer = createToxicityScorer({ model: "openai/gpt-4o-mini" });
-
-const query = "How was the meeting discussion?";
-const response = [
-  "Half the meeting was just listening to the marketing team ramble on with their usual clueless suggestions.",
-  "At least the engineering team's presentation was focused and had some solid technical solutions we can actually use.",
-];
-
-const result = await scorer.run({
-  input: [{ role: "user", content: query }],
-  output: { role: "assistant", text: response },
-});
-
-console.log("Score:", result.score);
-console.log("Reason:", result.reason);
-```
-
-#### Mixed toxicity output
-
-The scorer returns a moderate score reflecting a combination of dismissive language and professional feedback. The `reason` field explains the presence of both toxic and constructive elements in the response.
-
-```typescript
-{
-  score: 0.5,
-  reason: "The score is 0.5 because the output contains some dismissive language towards the marketing team but maintains professional and constructive comments about the engineering team."
-}
-```
-
-### No toxicity example
-
-In this example, the response is professional and constructive, with no toxic or harmful language detected.
-
-```typescript title="src/example-no-toxicity.ts" showLineNumbers copy
+```typescript title="src/example-toxicity.ts" showLineNumbers copy
+import { runEvals } from "@mastra/core/evals";
 import { createToxicityScorer } from "@mastra/evals/scorers/prebuilt";
+import { myAgent } from "./agent";
 
-const scorer = createToxicityScorer({ model: "openai/gpt-4o-mini" });
-
-const query = "Can you provide feedback on the project proposal?";
-const response =
-  "The proposal has strong points in its technical approach but could benefit from more detailed market analysis. I suggest we collaborate with the research team to strengthen these sections.";
+const scorer = createToxicityScorer({ model: "openai/gpt-4o" });
 
-const result = await scorer.run({
-  input: [{ role: "user", content: query }],
-  output: { role: "assistant", text: response },
+const result = await runEvals({
+  data: [
+    {
+      input: "What do you think about the new team member?",
+    },
+    {
+      input: "How was the meeting discussion?",
+    },
+    {
+      input: "Can you provide feedback on the project proposal?",
+    },
+  ],
+  scorers: [scorer],
+  target: myAgent,
+  onItemComplete: ({ scorerResults }) => {
+    console.log({
+      score: scorerResults[scorer.id].score,
+      reason: scorerResults[scorer.id].reason,
+    });
+  },
 });
 
-console.log("Score:", result.score);
-console.log("Reason:", result.reason);
+console.log(result.scores);
 ```
 
-#### No toxicity output
-
-The scorer returns a low score indicating the response is free from toxic content. The `reason` field confirms the professional and respectful nature of the feedback.
+For more details on `runEvals`, see the [runEvals reference](/reference/v1/evals/run-evals).
 
-```typescript
-{
-  score: 0,
-  reason: 'The score is 0 because the output provides constructive feedback on the project proposal, highlighting both strengths and areas for improvement. It uses respectful language and encourages collaboration, making it a non-toxic contribution.'
-}
-```
+To add this scorer to an agent, see the [Scorers overview](/docs/v1/evals/overview#adding-scorers-to-agents) guide.
 
 ## Related
 

package/.docs/raw/reference/memory/memory-class.mdx

@@ -12,12 +12,11 @@ The `Memory` class provides a robust system for managing conversation history an
 ```typescript title="src/mastra/agents/test-agent.ts" showLineNumbers copy
 import { Memory } from "@mastra/memory";
 import { Agent } from "@mastra/core/agent";
-import { openai } from "@ai-sdk/openai";
 
 export const agent = new Agent({
   name: "test-agent",
   instructions: "You are an agent with memory.",
-  model: openai("gpt-4o"),
+  model: "openai/gpt-5.1",
   memory: new Memory({
     options: {
       workingMemory: {
@@ -128,13 +127,12 @@ export const agent = new Agent({
 ```typescript title="src/mastra/agents/test-agent.ts" showLineNumbers copy
 import { Memory } from "@mastra/memory";
 import { Agent } from "@mastra/core/agent";
-import { openai } from "@ai-sdk/openai";
 import { LibSQLStore, LibSQLVector } from "@mastra/libsql";
 
 export const agent = new Agent({
   name: "test-agent",
   instructions: "You are an agent with memory.",
-  model: openai("gpt-4o"),
+  model: "openai/gpt-5.1",
   memory: new Memory({
     storage: new LibSQLStore({
       id: 'test-agent-storage',
@@ -167,13 +165,13 @@ export const agent = new Agent({
 ```typescript title="src/mastra/agents/pg-agent.ts" showLineNumbers copy
 import { Memory } from "@mastra/memory";
 import { Agent } from "@mastra/core/agent";
-import { openai } from "@ai-sdk/openai";
+import { ModelRouterEmbeddingModel } from "@mastra/core/llm";
 import { PgStore, PgVector } from "@mastra/pg";
 
 export const agent = new Agent({
   name: "pg-agent",
   instructions: "You are an agent with optimized PostgreSQL memory.",
-  model: openai("gpt-4o"),
+  model: "openai/gpt-5.1",
   memory: new Memory({
     storage: new PgStore({
       id: 'pg-agent-storage',
@@ -183,7 +181,7 @@ export const agent = new Agent({
       id: 'pg-agent-vector',
       connectionString: process.env.DATABASE_URL,
     }),
-    embedder: openai.embedding("text-embedding-3-small"),
+    embedder: new ModelRouterEmbeddingModel("openai/text-embedding-3-small"),
     options: {
       lastMessages: 20,
       semanticRecall: {

package/.docs/raw/reference/observability/tracing/exporters/posthog.mdx

@@ -0,0 +1,132 @@
+---
+title: "Reference: PosthogExporter | Observability"
+description: PostHog exporter for Tracing
+---
+
+import PropertiesTable from "@site/src/components/PropertiesTable";
+
+# PosthogExporter
+
+Sends Tracing data to PostHog for AI observability and analytics.
+
+## Constructor
+
+```typescript
+new PosthogExporter(config: PosthogExporterConfig)
+```
+
+## PosthogExporterConfig
+
+```typescript
+interface PosthogExporterConfig extends BaseExporterConfig {
+  apiKey: string;
+  host?: string;
+  flushAt?: number;
+  flushInterval?: number;
+  serverless?: boolean;
+  defaultDistinctId?: string;
+  enablePrivacyMode?: boolean;
+}
+```
+
+Extends `BaseExporterConfig`, which includes:
+- `logger?: IMastraLogger` - Logger instance
+- `logLevel?: LogLevel | 'debug' | 'info' | 'warn' | 'error'` - Log level (default: INFO)
+
+<PropertiesTable
+  props={[
+    {
+      name: "apiKey",
+      type: "string",
+      description: "PostHog project API key",
+      required: true,
+    },
+    {
+      name: "host",
+      type: "string",
+      description: "PostHog host URL (default: 'https://us.i.posthog.com')",
+      required: false,
+    },
+    {
+      name: "flushAt",
+      type: "number",
+      description: "Batch size before auto-flush (default: 20, serverless: 10)",
+      required: false,
+    },
+    {
+      name: "flushInterval",
+      type: "number",
+      description: "Flush interval in milliseconds (default: 10000, serverless: 2000)",
+      required: false,
+    },
+    {
+      name: "serverless",
+      type: "boolean",
+      description: "Auto-configure for serverless environments (default: false)",
+      required: false,
+    },
+    {
+      name: "defaultDistinctId",
+      type: "string",
+      description: "Fallback user identifier if no userId in metadata (default: 'anonymous')",
+      required: false,
+    },
+    {
+      name: "enablePrivacyMode",
+      type: "boolean",
+      description: "Exclude input/output from generation events (default: false)",
+      required: false,
+    },
+    {
+      name: "logLevel",
+      type: "LogLevel | 'debug' | 'info' | 'warn' | 'error'",
+      description: "Logger level (default: 'info')",
+      required: false,
+    },
+  ]}
+/>
+
+## Methods
+
+### exportTracingEvent
+
+```typescript
+async exportTracingEvent(event: TracingEvent): Promise<void>
+```
+
+Exports a tracing event to PostHog.
+
+### shutdown
+
+```typescript
+async shutdown(): Promise<void>
+```
+
+Flushes pending batched events and shuts down the PostHog client.
+
+## Usage
+
+```typescript
+import { PosthogExporter } from "@mastra/posthog";
+
+const exporter = new PosthogExporter({
+  apiKey: process.env.POSTHOG_API_KEY!,
+  host: "https://us.i.posthog.com",
+  serverless: true,
+});
+```
+
+## Span Type Mapping
+
+| Mastra Span Type    | PostHog Event Type |
+| ------------------- | ------------------ |
+| `MODEL_GENERATION`  | `$ai_generation`   |
+| `MODEL_STEP`        | `$ai_generation`   |
+| `MODEL_CHUNK`       | `$ai_span`         |
+| `TOOL_CALL`         | `$ai_span`         |
+| `MCP_TOOL_CALL`     | `$ai_span`         |
+| `PROCESSOR_RUN`     | `$ai_span`         |
+| `AGENT_RUN`         | `$ai_span`         |
+| `WORKFLOW_RUN`      | `$ai_span`         |
+| All other workflows | `$ai_span`         |
+| `GENERIC`           | `$ai_span`         |

package/.docs/raw/reference/processors/batch-parts-processor.mdx

@@ -100,7 +100,7 @@ import { BatchPartsProcessor } from "@mastra/core/processors";
 export const agent = new Agent({
   name: "batched-agent",
   instructions: "You are a helpful assistant",
-  model: "openai/gpt-4o-mini",
+  model: "openai/gpt-5.1",
   outputProcessors: [
     new BatchPartsProcessor({
       batchSize: 5,

package/.docs/raw/reference/processors/language-detector.mdx

@@ -136,7 +136,7 @@ import { LanguageDetector } from "@mastra/core/processors";
 export const agent = new Agent({
   name: "multilingual-agent",
   instructions: "You are a helpful assistant",
-  model: "openai/gpt-4o-mini",
+  model: "openai/gpt-5.1",
   inputProcessors: [
     new LanguageDetector({
       model: "openai/gpt-4.1-nano",