@mastra/mcp-docs-server 0.0.1 → 0.0.2-alpha.0

package/.docs/organized/changelogs/mastra.md

@@ -1,5 +1,56 @@
 # mastra
 
+## 0.4.1-alpha.1
+
+### Patch Changes
+
+- 2f6a8b6: Update port handling in dev command to ensure CLI port takes precedence over environment variables and add warning when overriding PORT environment variable.
+- Updated dependencies [beaf1c2]
+- Updated dependencies [3084e13]
+  - @mastra/core@0.6.2-alpha.0
+  - @mastra/deployer@0.2.2-alpha.1
+
+## 0.4.1-alpha.0
+
+### Patch Changes
+
+- aede1ea: Add non english support to weather example
+- Updated dependencies [4e6732b]
+  - @mastra/deployer@0.2.2-alpha.0
+
+## 0.4.0
+
+### Minor Changes
+
+- f9b6ab5: add Cerebras as a llm provider to create-mastra@latest
+
+### Patch Changes
+
+- 5052613: Added a new `mastra create --project-name <string>` flag so coder agents can create new Mastra projects with a one line command.
+- 1291e89: Add resizable-panel to playground-ui and use in agent and workflow sidebars
+- 1405e46: update the Groq model the create-mastra@latest sets
+- da8d9bb: Enable public dir copying if it exists
+- 9ba1e97: update playground ui for mastra and create-mastra
+- 5baf1ec: animate new traces
+- 65f2a4c: Add Mastra Docs MCP to the pnpm create mastra TUI with the option to install in Cursor or Windsurf
+- 9116d70: Handle the different workflow methods in workflow graph
+- 0709d99: add prop for dynamic empty text
+- Updated dependencies [cc7f392]
+- Updated dependencies [fc2f89c]
+- Updated dependencies [dfbb131]
+- Updated dependencies [f4854ee]
+- Updated dependencies [afaf73f]
+- Updated dependencies [0850b4c]
+- Updated dependencies [7bcfaee]
+- Updated dependencies [da8d9bb]
+- Updated dependencies [44631b1]
+- Updated dependencies [9116d70]
+- Updated dependencies [6e559a0]
+- Updated dependencies [5f43505]
+- Updated dependencies [61ad5a4]
+  - @mastra/deployer@0.2.1
+  - @mastra/core@0.6.1
+
 ## 0.4.0-alpha.2
 
 ### Patch Changes
@@ -247,56 +298,5 @@
 - Updated dependencies [c151ae6]
 - Updated dependencies [52e0418]
 - Updated dependencies [03236ec]
-- Updated dependencies [3764e71]
-- Updated dependencies [df982db]
-- Updated dependencies [0461849]
-- Updated dependencies [2259379]
-- Updated dependencies [358f069]
-  - @mastra/core@0.5.0-alpha.5
-  - @mastra/deployer@0.1.8-alpha.5
-
-## 0.2.9-alpha.4
-
-### Patch Changes
-
-- 144b3d5: Update traces table UI, agent Chat UI
-  Fix get workflows breaking
-- Updated dependencies [d79aedf]
-- Updated dependencies [144b3d5]
-  - @mastra/core@0.5.0-alpha.4
-  - @mastra/deployer@0.1.8-alpha.4
-
-## 0.2.9-alpha.3
-
-### Patch Changes
-
-- Updated dependencies [3d0e290]
-  - @mastra/core@0.5.0-alpha.3
-  - @mastra/deployer@0.1.8-alpha.3
-
-## 0.2.9-alpha.2
-
-### Patch Changes
-
-- Updated dependencies [02ffb7b]
-  - @mastra/core@0.5.0-alpha.2
-  - @mastra/deployer@0.1.8-alpha.2
-
-## 0.2.9-alpha.1
-
-### Patch Changes
-
-- e5149bb: Fix playground-ui agent-evals tab-content
-- Updated dependencies [dab255b]
-  - @mastra/core@0.5.0-alpha.1
-  - @mastra/deployer@0.1.8-alpha.1
-
-## 0.2.9-alpha.0
-
-### Patch Changes
-
-- 5fae49e: Configurable timeout on npm create mastra
-- 960690d: Improve client-js workflow watch dx
-- 62565c1: --no-timeout npm create mastra flag
 
-... 1959 more lines hidden. See full changelog in package directory.
+... 2010 more lines hidden. See full changelog in package directory.

package/.docs/organized/code-examples/agent.md

@@ -375,9 +375,14 @@ import { z } from 'zod';
 export const cookingTool = createTool({
   id: 'cooking-tool',
   description: 'My tool description',
-  inputSchema: z.object({}),
-  execute: async () => {
-    console.log('My tool is running!');
+  inputSchema: z.object({
+    ingredient: z.string(),
+  }),
+  execute: async ({ context }, options) => {
+    console.log('My tool is running!', context.ingredient);
+    if (options?.toolCallId) {
+      console.log('Tool call ID:', options.toolCallId);
+    }
     return 'My tool result';
   },
 });

package/.docs/organized/code-examples/ai-sdk-useChat.md

@@ -48,6 +48,7 @@ export const weatherAgent = new Agent({
 
       Your primary function is to help users get weather details for specific locations. When responding:
       - Always ask for a location if none is provided
+      - If the location name isn’t in English, please translate it
       - If giving a location with multiple parts (e.g. "New York, NY"), use the most relevant part (e.g. "New York")
       - Include relevant details like humidity, wind conditions, and precipitation
       - Keep responses concise but informative

package/.docs/organized/code-examples/weather-agent.md

@@ -35,6 +35,7 @@ export const weatherAgent = new Agent({
 
 Your primary function is to help users get weather details for specific locations. When responding:
 - Always ask for a location if none is provided
+- If the location name isn’t in English, please translate it
 - If giving a location with multiple parts (e.g. "New York, NY"), use the most relevant part (e.g. "New York")
 - Include relevant details like humidity, wind conditions, and precipitation
 - Keep responses concise but informative

package/.docs/raw/deployment/client.mdx

@@ -0,0 +1,120 @@
+---
+title: "MastraClient"
+description: "Learn how to set up and use the Mastra Client SDK"
+---
+
+# Mastra Client SDK
+
+The Mastra Client SDK provides a simple and type-safe interface for interacting with your [Mastra Server](/docs/deployment/server) from your client environment.
+
+## Development Requirements
+
+To ensure smooth local development, make sure you have:
+
+- Node.js 18.x or later installed
+- TypeScript 4.7+ (if using TypeScript)
+- A modern browser environment with Fetch API support
+- Your local Mastra server running (typically on port 4111)
+
+## Installation
+
+import { Tabs } from "nextra/components";
+
+<Tabs items={["npm", "yarn", "pnpm"]}>
+  <Tabs.Tab>
+```bash copy
+npm install @mastra/client-js
+```
+  </Tabs.Tab>
+  <Tabs.Tab>
+```bash copy
+yarn add @mastra/client-js
+```
+  </Tabs.Tab>
+  <Tabs.Tab>
+```bash copy
+pnpm add @mastra/client-js
+```
+  </Tabs.Tab>
+</Tabs>
+
+## Initialize Mastra Client
+
+To get started you'll need to initialize your MastraClient with necessary parameters:
+
+```typescript
+import { MastraClient } from "@mastra/client-js";
+
+const client = new MastraClient({
+  baseUrl: "http://localhost:4111", // Default Mastra development server port
+});
+```
+
+### Configuration Options
+
+You can customize the client with various options:
+
+```typescript
+const client = new MastraClient({
+  // Required
+  baseUrl: "http://localhost:4111",
+
+  // Optional configurations for development
+  retries: 3,           // Number of retry attempts
+  backoffMs: 300,       // Initial retry backoff time
+  maxBackoffMs: 5000,   // Maximum retry backoff time
+  headers: {            // Custom headers for development
+    "X-Development": "true"
+  }
+});
+```
+
+## Example
+
+Once your MastraClient is initialized you can start making client calls via the type-safe
+interface
+
+```typescript
+// Get a reference to your local agent
+const agent = client.getAgent("dev-agent-id");
+
+// Generate responses
+const response = await agent.generate({
+  messages: [
+    {
+      role: "user",
+      content: "Hello, I'm testing the local development setup!"
+    }
+  ]
+});
+```
+
+## Available Features
+
+Mastra client exposes all resources served by the Mastra Server
+
+- [**Agents**](/docs/reference/client-js/agents): Create and manage AI agents, generate responses, and handle streaming interactions
+- [**Memory**](/docs/reference/client-js/memory): Manage conversation threads and message history
+- [**Tools**](/docs/reference/client-js/tools): Access and execute tools available to agents
+- [**Workflows**](/docs/reference/client-js/workflows): Create and manage automated workflows
+- [**Vectors**](/docs/reference/client-js/vectors): Handle vector operations for semantic search and similarity matching
+
+
+## Best Practices
+1. **Error Handling**: Implement proper error handling for development scenarios
+2. **Environment Variables**: Use environment variables for configuration
+3. **Debugging**: Enable detailed logging when needed
+
+```typescript
+// Example with error handling and logging
+try {
+  const agent = client.getAgent("dev-agent-id");
+  const response = await agent.generate({
+    messages: [{ role: "user", content: "Test message" }]
+  });
+  console.log("Response:", response);
+} catch (error) {
+  console.error("Development error:", error);
+}
+```
+

package/.docs/raw/deployment/server.mdx

@@ -5,7 +5,7 @@ description: "Configure and customize the Mastra server with middleware and othe
 
 # Mastra Server
 
-When you deploy a Mastra application, it runs as an HTTP server that exposes your agents, workflows, and other functionality as API endpoints. This page explains how to configure and customize the server behavior.
+While developing or when you deploy a Mastra application, it runs as an HTTP server that exposes your agents, workflows, and other functionality as API endpoints. This page explains how to configure and customize the server behavior.
 
 ## Server Architecture
 

package/.docs/raw/evals/00-overview.mdx

@@ -1,106 +1,89 @@
 ---
 title: "Overview"
-description: "Mastra evals help you measure LLM output quality with metrics for relevance, bias, hallucination, and more."
+description: "Understanding how to evaluate and measure AI agent quality using Mastra evals."
 ---
 
 # Testing your agents with evals
 
+While traditional software tests have clear pass/fail conditions, AI outputs are non-deterministic — they can vary with the same input. Evals help bridge this gap by providing quantifiable metrics for measuring agent quality.
+
 Evals are automated tests that evaluate Agents outputs using model-graded, rule-based, and statistical methods. Each eval returns a normalized score between 0-1 that can be logged and compared. Evals can be customized with your own prompts and scoring functions.
 
 Evals can be run in the cloud, capturing real-time results. But evals can also be part of your CI/CD pipeline, allowing you to test and monitor your agents over time.
 
-## How to use evals
+## Types of Evals
+
+There are different kinds of evals, each serving a specific purpose. Here are some common types:
+
+1. **Textual Evals**: Evaluate accuracy, reliability, and context understanding of agent responses
+2. **Classification Evals**: Measure accuracy in categorizing data based on predefined categories
+3. **Tool Usage Evals**: Assess how effectively an agent uses external tools or APIs
+4. **Prompt Engineering Evals**: Explore impact of different instructions and input formats
 
-Evals need to be added to an agent. To use any of [the default metrics](/docs/evals/01-supported-evals), you can do the following:
+## Getting Started
+
+Evals need to be added to an agent. Here's an example using the faithfulness, content similarity, and hallucination metrics:
 
 ```typescript copy showLineNumbers filename="src/mastra/agents/index.ts"
 import { Agent } from "@mastra/core/agent";
 import { openai } from "@ai-sdk/openai";
-import { ToneConsistencyMetric } from "@mastra/evals/nlp";
+import { 
+  FaithfulnessMetric,
+  ContentSimilarityMetric,
+  HallucinationMetric 
+} from "@mastra/evals/nlp";
  
 export const myAgent = new Agent({
-  name: "My Agent",
-  instructions: "You are a helpful assistant.",
-  model: openai("gpt-4o-mini"),
-  evals: {
-    tone: new ToneConsistencyMetric()
-  },
-});
-```
-
-You can now view the evals in the Mastra dashboard, when using `mastra dev`.
-
-### Executing evals in your CI/CD pipeline
-
-We support any testing framework that supports ESM modules. For example, you can use [Vitest](https://vitest.dev/), [Jest](https://jestjs.io/) or [Mocha](https://mochajs.org/) to run evals in your CI/CD pipeline.
-
-```typescript copy showLineNumbers filename="src/mastra/agents/index.test.ts"
-import { describe, it, expect } from 'vitest';
-import { evaluate } from '@mastra/core/eval';
-import { myAgent } from './index';
-
-describe('My Agent', () => {
-  it('should be able to validate tone consistency', async () => {
-    const metric = new ToneConsistencyMetric();
-    const result = await evaluate(myAgent, 'Hello, world!', metric)
-
-    expect(result.score).toBe(1);
-  });
+  name: "ContentWriter",
+  instructions: "You are a content writer that creates accurate summaries",
+  model: openai("gpt-4o"),
+  evals: [
+    new FaithfulnessMetric(),     // Checks if output matches source material
+    new ContentSimilarityMetric({
+      threshold: 0.8              // Require 80% similarity with expected output
+    }),
+    new HallucinationMetric()
+  ]
 });
-
 ```
 
-You will need to configure a testSetup and globalSetup script for your testing framework to capture the eval results. It allows us to show these results in your mastra dashboard.
-
-
-#### Vitest
+You can view eval results in the Mastra dashboard when using `mastra dev`.
 
-These are the files you need to add to your project to run evals in your CI/CD pipeline and allow us to capture the results.
-Without these files, the evals will still run and fail when necessary but you won't be able to see the results in the Mastra dashboard.
+## Beyond Automated Testing
 
-```typescript copy showLineNumbers filename="globalSetup.ts"
-import { globalSetup } from '@mastra/evals';
+While automated evals are valuable, high-performing AI teams often combine them with:
 
-export default function setup() {
-  globalSetup()
-}
-```
+1. **A/B Testing**: Compare different versions with real users
+2. **Human Review**: Regular review of production data and traces
+3. **Continuous Monitoring**: Track eval metrics over time to detect regressions
 
-```typescript copy showLineNumbers filename="testSetup.ts"
-import { beforeAll } from 'vitest';
-import { attachListeners } from '@mastra/evals';
+## Understanding Eval Results
 
-beforeAll(async () => {
-  await attachListeners();
-});
-```
+Each eval metric measures a specific aspect of your agent's output. Here's how to interpret and improve your results:
 
-<details>
-  <summary>Store evals in Mastra Storage</summary>
+### Understanding Scores
+For any metric:
+1. Check the metric documentation to understand the scoring process
+2. Look for patterns in when scores change
+3. Compare scores across different inputs and contexts
+4. Track changes over time to spot trends
 
-  Pass your Mastra instance to store evals in the configured storage:
+### Improving Results
+When scores aren't meeting your targets:
+1. Check your instructions - Are they clear? Try making them more specific
+2. Look at your context - Is it giving the agent what it needs?
+3. Simplify your prompts - Break complex tasks into smaller steps
+4. Add guardrails - Include specific rules for tricky cases
 
-  ```typescript
-  import { mastra } from './your-mastra-setup';
+### Maintaining Quality
+Once you're hitting your targets:
+1. Monitor stability - Do scores remain consistent?
+2. Document what works - Keep notes on successful approaches
+3. Test edge cases - Add examples that cover unusual scenarios
+4. Fine-tune - Look for ways to improve efficiency
 
-  beforeAll(async () => {
-    // Store evals in Mastra Storage (requires storage to be enabled)
-    await attachListeners(mastra);
-  });
-  ```
+See [Textual Evals](/docs/evals/textual-evals) for more info on what evals can do.
 
-  This allows you to save evals in Mastra Storage. 
-  With file storage, evals persist and can be queried later. 
-  With memory storage, evals are isolated to the test process.
-</details>
+For more info on how to create your own evals, see the [Custom Evals](/docs/evals/02-custom-eval) guide.
 
-```typescript copy showLineNumbers filename="vitest.config.ts"
-import { defineConfig } from 'vitest/config'
-
-export default defineConfig({
-  test: {
-    globalSetup: './globalSetup.ts',
-    setupFiles: ['./testSetup.ts'],
-  },
-})
-```
+For running evals in your CI pipeline, see the [Running in CI](/docs/evals/running-in-ci) guide.

package/.docs/raw/evals/01-textual-evals.mdx

@@ -0,0 +1,53 @@
+---
+title: "Textual Evals"
+description: "Understand how Mastra uses LLM-as-judge methodology to evaluate text quality."
+---
+
+# Textual Evals
+
+Textual evals use an LLM-as-judge methodology to evaluate agent outputs. This approach leverages language models to assess various aspects of text quality, similar to how a teaching assistant might grade assignments using a rubric.
+
+Each eval focuses on specific quality aspects and returns a score between 0 and 1, providing quantifiable metrics for non-deterministic AI outputs.
+
+Mastra provides several eval metrics for assessing Agent outputs. Mastra is not limited to these metrics, and you can also [define your own evals](/docs/evals/02-custom-eval).
+
+## Why Use Textual Evals?
+
+Textual evals help ensure your agent:
+- Produces accurate and reliable responses
+- Uses context effectively
+- Follows output requirements
+- Maintains consistent quality over time
+
+## Available Metrics
+
+### Accuracy and Reliability
+
+These metrics evaluate how correct, truthful, and complete your agent's answers are:
+
+- [`hallucination`](/docs/reference/evals/hallucination): Detects facts or claims not present in provided context
+- [`faithfulness`](/docs/reference/evals/faithfulness): Measures how accurately responses represent provided context
+- [`content-similarity`](/docs/reference/evals/content-similarity): Evaluates consistency of information across different phrasings
+- [`completeness`](/docs/reference/evals/completeness): Checks if responses include all necessary information
+- [`answer-relevancy`](/docs/reference/evals/answer-relevancy): Assesses how well responses address the original query
+- [`textual-difference`](/docs/reference/evals/textual-difference): Measures textual differences between strings
+
+### Understanding Context
+
+These metrics evaluate how well your agent uses provided context:
+
+- [`context-position`](/docs/reference/evals/context-position): Analyzes where context appears in responses
+- [`context-precision`](/docs/reference/evals/context-precision): Evaluates whether context chunks are grouped logically
+- [`context-relevancy`](/docs/reference/evals/context-relevancy): Measures use of appropriate context pieces
+- [`contextual-recall`](/docs/reference/evals/contextual-recall): Assesses completeness of context usage
+
+### Output Quality
+
+These metrics evaluate adherence to format and style requirements:
+
+- [`tone`](/docs/reference/evals/tone-consistency): Measures consistency in formality, complexity, and style
+- [`toxicity`](/docs/reference/evals/toxicity): Detects harmful or inappropriate content
+- [`bias`](/docs/reference/evals/bias): Detects potential biases in the output
+- [`prompt-alignment`](/docs/reference/evals/prompt-alignment): Checks adherence to explicit instructions like length restrictions, formatting requirements, or other constraints
+- [`summarization`](/docs/reference/evals/summarization): Evaluates information retention and conciseness
+- [`keyword-coverage`](/docs/reference/evals/keyword-coverage): Assesses technical terminology usage