@mastra/mcp-docs-server 1.1.2 → 1.1.3-alpha.1

package/.docs/docs/agents/processors.md

@@ -342,6 +342,58 @@ export class CustomOutputProcessor implements Processor {
 }
 ```
 
+#### Emitting custom stream events with writer
+
+Output processors receive a `writer` object that lets you emit custom data chunks back to the client during streaming. This is useful for use cases like streaming moderation results or sending UI update signals without blocking the original stream.
+
+```typescript
+import type { Processor, ChunkType, MastraDBMessage } from "@mastra/core";
+
+export class ModerationProcessor implements Processor {
+  id = "moderation";
+
+  async processOutputResult({ messages, writer }) {
+    // Run moderation on the final output
+    const text = messages
+      .filter((m) => m.role === "assistant")
+      .flatMap((m) => m.content.parts?.filter((p) => p.type === "text"))
+      .map((p) => p.text)
+      .join(" ");
+
+    const result = await runModeration(text);
+
+    if (result.requiresChange) {
+      // Emit a custom event to the client with the moderated text
+      await writer?.custom({
+        type: "data-moderation-update",
+        data: {
+          originalText: text,
+          moderatedText: result.moderatedText,
+          reason: result.reason,
+        },
+      });
+    }
+
+    return messages;
+  }
+}
+```
+
+On the client, listen for the custom chunk type in the stream:
+
+```typescript
+const stream = await agent.stream("Hello");
+
+for await (const chunk of stream.fullStream) {
+  if (chunk.type === "data-moderation-update") {
+    // Update the UI with moderated text
+    updateDisplayedMessage(chunk.data.moderatedText);
+  }
+}
+```
+
+Custom chunk types must use the `data-` prefix (e.g., `data-moderation-update`, `data-status`).
+
 #### Adding metadata in output processors
 
 You can add custom metadata to messages in `processOutputResult`. This metadata is accessible via the response object:

package/.docs/docs/observability/datasets/overview.md

@@ -0,0 +1,188 @@
+# Datasets Overview
+
+**Added in:** `@mastra/core@1.4.0`
+
+Datasets are collections of test cases that you run experiments against to measure how well your agents and workflows perform. Each mutation creates a new version, so you can reproduce past experiments exactly. Pair datasets with [scorers](https://mastra.ai/docs/evals/overview) to track quality across prompts, models, or code changes.
+
+## Usage
+
+### Configure storage
+
+Configure storage in your Mastra instance. Datasets require a storage adapter that provides the `datasets` domain:
+
+```typescript
+import { Mastra } from "@mastra/core";
+import { LibSQLStore } from "@mastra/libsql";
+
+export const mastra = new Mastra({
+  storage: new LibSQLStore({
+    id: "my-store",
+    url: "file:./mastra.db",
+  }),
+});
+```
+
+### Accessing the datasets API
+
+All dataset operations are available through `mastra.datasets`:
+
+```typescript
+const datasets = mastra.datasets;
+
+// Create a dataset
+const dataset = await datasets.create({ name: "my-dataset" });
+
+// Retrieve an existing dataset
+const existing = await datasets.get({ id: "dataset-id" });
+
+// List all datasets
+const { datasets: all } = await datasets.list();
+```
+
+> **Info:** Visit the [`DatasetsManager` reference](https://mastra.ai/reference/datasets/datasets-manager) for the full list of methods.
+
+## Creating a dataset
+
+Call [`create()`](https://mastra.ai/reference/datasets/create) with a name and optional description:
+
+```typescript
+import { mastra } from "../index";
+
+const dataset = await mastra.datasets.create({
+  name: "translation-pairs",
+  description: "English to Spanish translation test cases",
+});
+
+console.log(dataset.id); // auto-generated UUID
+```
+
+### Defining schemas
+
+You can enforce the shape of `input` and `groundTruth` by passing Zod schemas. Mastra converts them to JSON Schema at creation time:
+
+```typescript
+import { z } from "zod";
+import { mastra } from "../index";
+
+const dataset = await mastra.datasets.create({
+  name: "translation-pairs",
+  inputSchema: z.object({
+    text: z.string(),
+    sourceLang: z.string(),
+    targetLang: z.string(),
+  }),
+  groundTruthSchema: z.object({
+    translation: z.string(),
+  }),
+});
+```
+
+Items that don't match the schema are rejected at insert time.
+
+## Adding items
+
+Use [`addItem()`](https://mastra.ai/reference/datasets/addItem) for a single item or [`addItems()`](https://mastra.ai/reference/datasets/addItems) to insert in bulk:
+
+```typescript
+// Single item
+await dataset.addItem({
+  input: { text: "Hello", sourceLang: "en", targetLang: "es" },
+  groundTruth: { translation: "Hola" },
+});
+
+// Bulk insert
+await dataset.addItems({
+  items: [
+    {
+      input: { text: "Goodbye", sourceLang: "en", targetLang: "es" },
+      groundTruth: { translation: "Adiós" },
+    },
+    {
+      input: { text: "Thank you", sourceLang: "en", targetLang: "es" },
+      groundTruth: { translation: "Gracias" },
+    },
+  ],
+});
+```
+
+## Updating and deleting items
+
+[`updateItem()`](https://mastra.ai/reference/datasets/updateItem), [`deleteItem()`](https://mastra.ai/reference/datasets/deleteItem), and [`deleteItems()`](https://mastra.ai/reference/datasets/deleteItems) let you modify or remove existing items by `itemId`:
+
+```typescript
+await dataset.updateItem({
+  itemId: "item-abc-123",
+  groundTruth: { translation: "¡Hola!" },
+});
+
+await dataset.deleteItem({ itemId: "item-abc-123" });
+
+await dataset.deleteItems({ itemIds: ["item-1", "item-2"] });
+```
+
+## Listing and searching items
+
+[`listItems()`](https://mastra.ai/reference/datasets/listItems) supports pagination and full-text search:
+
+```typescript
+// Paginated list
+const { items, pagination } = await dataset.listItems({
+  page: 0,
+  perPage: 50,
+});
+
+// Full-text search
+const { items: matches } = await dataset.listItems({
+  search: "Hello",
+});
+
+// List items at a specific version
+const v2Items = await dataset.listItems({ version: 2 });
+```
+
+## Versioning
+
+Every mutation to a dataset's items (add, update, or delete) bumps the dataset version. This lets you pin experiments to a specific snapshot of the data.
+
+### Listing versions
+
+Use [`listVersions()`](https://mastra.ai/reference/datasets/listVersions) to see the paginated history of versions:
+
+```typescript
+const { versions, pagination } = await dataset.listVersions();
+
+for (const v of versions) {
+  console.log(`Version ${v.version} — created ${v.createdAt}`);
+}
+```
+
+### Viewing item history
+
+See how a specific item changed across versions by calling [`getItemHistory()`](https://mastra.ai/reference/datasets/getItemHistory) with the `itemId`:
+
+```typescript
+const history = await dataset.getItemHistory({ itemId: "item-abc-123" });
+
+for (const row of history) {
+  console.log(`Version ${row.datasetVersion}`, row.input, row.groundTruth);
+}
+```
+
+### Pinning to a version
+
+Fetch the exact items that existed at a past version:
+
+```typescript
+const items = await dataset.listItems({ version: 2 });
+```
+
+You can also pin experiments to a version, see [running experiments](https://mastra.ai/docs/observability/datasets/running-experiments).
+
+> **Info:** Visit the [`Dataset` reference](https://mastra.ai/reference/datasets/dataset) for the full list of methods and parameters.
+
+## Related
+
+- [Running experiments](https://mastra.ai/docs/observability/datasets/running-experiments)
+- [Scorers overview](https://mastra.ai/docs/evals/overview)
+- [DatasetsManager reference](https://mastra.ai/reference/datasets/datasets-manager)
+- [Dataset reference](https://mastra.ai/reference/datasets/dataset)

package/.docs/docs/observability/datasets/running-experiments.md

@@ -0,0 +1,266 @@
+# Running Experiments
+
+**Added in:** `@mastra/core@1.4.0`
+
+An experiment runs every item in a dataset through a target (an agent, a workflow, or a scorer) and then optionally scores the outputs. Use a scorer as the target when you want to evaluate an LLM judge itself. Results are persisted to storage so you can compare runs across different prompts, models, or code changes.
+
+## Basic experiment
+
+Call [`startExperiment()`](https://mastra.ai/reference/datasets/startExperiment) with a target and scorers:
+
+```typescript
+import { mastra } from "../index";
+
+const dataset = await mastra.datasets.get({ id: "translation-dataset-id" });
+
+const summary = await dataset.startExperiment({
+  name: "gpt-5.1-baseline",
+  targetType: "agent",
+  targetId: "translation-agent",
+  scorers: ["accuracy", "fluency"],
+});
+
+console.log(summary.status);          // 'completed' | 'failed'
+console.log(summary.succeededCount);  // number of items that ran successfully
+console.log(summary.failedCount);     // number of items that failed
+```
+
+`startExperiment()` blocks until all items finish. For fire-and-forget execution, see [async experiments](#async-experiments).
+
+## Experiment targets
+
+You can point an experiment at a registered agent, workflow, or scorer.
+
+### Registered agent
+
+Point to an agent registered on your Mastra instance:
+
+```typescript
+const summary = await dataset.startExperiment({
+  name: "agent-v2-eval",
+  targetType: "agent",
+  targetId: "translation-agent",
+  scorers: ["accuracy"],
+});
+```
+
+Each item's `input` is passed directly to `agent.generate()`, so it must be a `string`, `string[]`, or `CoreMessage[]`.
+
+### Registered workflow
+
+Point to a workflow registered on your Mastra instance:
+
+```typescript
+const summary = await dataset.startExperiment({
+  name: "workflow-eval",
+  targetType: "workflow",
+  targetId: "translation-workflow",
+  scorers: ["accuracy"],
+});
+```
+
+The workflow receives each item's `input` as its trigger data.
+
+### Registered scorer
+
+Point to a scorer to evaluate an LLM judge against ground truth:
+
+```typescript
+const summary = await dataset.startExperiment({
+  name: "judge-accuracy-eval",
+  targetType: "scorer",
+  targetId: "accuracy",
+});
+```
+
+The scorer receives each item's `input` and `groundTruth`. LLM-based judges can drift over time as underlying models change, so it's important to periodically realign them against known-good labels. A dataset gives you a stable benchmark to detect that drift.
+
+## Scoring results
+
+Scorers automatically run after each item's target execution. Pass scorer instances or registered scorer IDs:
+
+**Scorer IDs**:
+
+```typescript
+// Reference scorers registered on the Mastra instance
+const summary = await dataset.startExperiment({
+  name: "with-registered-scorers",
+  targetType: "agent",
+  targetId: "translation-agent",
+  scorers: ["accuracy", "fluency"],
+});
+```
+
+**Scorer instances**:
+
+```typescript
+import { createAnswerRelevancyScorer } from "@mastra/evals/scorers/prebuilt";
+
+const relevancy = createAnswerRelevancyScorer({ model: "openai/gpt-4.1-nano" });
+
+const summary = await dataset.startExperiment({
+  name: "with-scorer-instances",
+  targetType: "agent",
+  targetId: "translation-agent",
+  scorers: [relevancy],
+});
+```
+
+Each item's results include per-scorer scores:
+
+```typescript
+for (const item of summary.results) {
+  console.log(item.itemId, item.output);
+  for (const score of item.scores) {
+    console.log(`  ${score.scorerName}: ${score.score} — ${score.reason}`);
+  }
+}
+```
+
+> **Info:** Visit the [Scorers overview](https://mastra.ai/docs/evals/overview) for details on available and custom scorers.
+
+## Async experiments
+
+`startExperiment()` blocks until every item completes. For long-running datasets, use [`startExperimentAsync()`](https://mastra.ai/reference/datasets/startExperimentAsync) to start the experiment in the background:
+
+```typescript
+const { experimentId, status } = await dataset.startExperimentAsync({
+  name: "large-dataset-run",
+  targetType: "agent",
+  targetId: "translation-agent",
+  scorers: ["accuracy"],
+});
+
+console.log(experimentId); // UUID
+console.log(status);       // 'pending'
+```
+
+Poll for completion using [`getExperiment()`](https://mastra.ai/reference/datasets/getExperiment):
+
+```typescript
+let experiment = await dataset.getExperiment({ experimentId });
+
+while (experiment.status === "pending" || experiment.status === "running") {
+  await new Promise(resolve => setTimeout(resolve, 5000));
+  experiment = await dataset.getExperiment({ experimentId });
+}
+
+console.log(experiment.status); // 'completed' | 'failed'
+```
+
+## Configuration options
+
+### Concurrency
+
+Control how many items run in parallel (default: 5):
+
+```typescript
+const summary = await dataset.startExperiment({
+  targetType: "agent",
+  targetId: "translation-agent",
+  maxConcurrency: 10,
+});
+```
+
+### Timeouts and retries
+
+Set a per-item timeout (in milliseconds) and retry count:
+
+```typescript
+const summary = await dataset.startExperiment({
+  targetType: "agent",
+  targetId: "translation-agent",
+  itemTimeout: 30_000,  // 30 seconds per item
+  maxRetries: 2,        // retry failed items up to 2 times
+});
+```
+
+Retries use exponential backoff. Abort errors are never retried.
+
+### Aborting an experiment
+
+Pass an `AbortSignal` to cancel a running experiment:
+
+```typescript
+const controller = new AbortController();
+
+// Cancel after 60 seconds
+setTimeout(() => controller.abort(), 60_000);
+
+const summary = await dataset.startExperiment({
+  targetType: "agent",
+  targetId: "translation-agent",
+  signal: controller.signal,
+});
+```
+
+Remaining items are marked as skipped in the summary.
+
+### Pinning a dataset version
+
+Run against a specific snapshot of the dataset:
+
+```typescript
+const summary = await dataset.startExperiment({
+  targetType: "agent",
+  targetId: "translation-agent",
+  version: 3, // use items from dataset version 3
+});
+```
+
+## Viewing results
+
+### Listing experiments
+
+```typescript
+const { experiments, pagination } = await dataset.listExperiments({
+  page: 0,
+  perPage: 10,
+});
+
+for (const exp of experiments) {
+  console.log(`${exp.name} — ${exp.status} (${exp.succeededCount}/${exp.totalItems})`);
+}
+```
+
+### Experiment details
+
+```typescript
+const experiment = await dataset.getExperiment({
+  experimentId: "exp-abc-123",
+});
+
+console.log(experiment.status);
+console.log(experiment.startedAt);
+console.log(experiment.completedAt);
+```
+
+### Item-level results
+
+```typescript
+const { results, pagination } = await dataset.listExperimentResults({
+  experimentId: "exp-abc-123",
+  page: 0,
+  perPage: 50,
+});
+
+for (const result of results) {
+  console.log(result.itemId, result.output, result.error);
+}
+```
+
+## Understanding the summary
+
+`startExperiment()` returns an `ExperimentSummary` with counts and per-item results:
+
+- `completedWithErrors` is `true` when the experiment finished but some items failed.
+- Items cancelled via `signal` appear in `skippedCount`.
+
+> **Info:** Visit the [`startExperiment` reference](https://mastra.ai/reference/datasets/startExperiment) for the full parameter and return type documentation.
+
+## Related
+
+- [Datasets overview](https://mastra.ai/docs/observability/datasets/overview)
+- [Scorers overview](https://mastra.ai/docs/evals/overview)
+- [`startExperiment` reference](https://mastra.ai/reference/datasets/startExperiment)
+- [`listExperimentResults` reference](https://mastra.ai/reference/datasets/listExperimentResults)

package/.docs/docs/observability/tracing/exporters/cloud.md

@@ -7,8 +7,9 @@ The `CloudExporter` sends traces to Mastra Cloud for centralized monitoring and
 ### Prerequisites
 
 1. **Mastra Cloud Account**: Sign up at [cloud.mastra.ai](https://cloud.mastra.ai)
-2. **Access Token**: Generate in Mastra Cloud → Settings → API Tokens
-3. **Environment Variables**: Set your credentials:
+2. **Mastra Cloud Project**: Create a project in Mastra Cloud. Traces are scoped per project, so even if you only want observability, you need a project to act as the destination for your traces.
+3. **Access Token**: Generate in your project's sidebar under **Project Settings → Access Tokens**
+4. **Environment Variables**: Set your credentials:
 
 ```bash
 MASTRA_CLOUD_ACCESS_TOKEN=mst_xxxxxxxxxxxxxxxx
@@ -90,9 +91,9 @@ new CloudExporter({
 
 1. Navigate to [cloud.mastra.ai](https://cloud.mastra.ai)
 
-2. Select your project
+2. Select the project associated with your access token
 
-3. Go to Observability → Traces
+3. Go to **Observability → Traces**
 
 4. Use filters to find specific traces:
 
@@ -101,6 +102,8 @@ new CloudExporter({
    - Trace ID
    - Error status
 
+> **Note:** Traces are scoped to the project that issued the access token. To view traces, make sure you're viewing the same project you generated the token from.
+
 ### Features
 
 - **Trace Timeline** - Visual execution flow

package/.docs/docs/server/server-adapters.md

@@ -479,10 +479,11 @@ When using server adapters, configuration comes from two places: the Mastra `ser
 
 The adapter reads these settings from `mastra.getServer()`:
 
-| Option          | Description                                                                             |
-| --------------- | --------------------------------------------------------------------------------------- |
-| `auth`          | Authentication config, used by `registerAuthMiddleware()`.                              |
-| `bodySizeLimit` | Default body size limit in bytes. Can be overridden per-adapter via `bodyLimitOptions`. |
+| Option          | Description                                                                                                                                     |
+| --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
+| `auth`          | Authentication config, used by `registerAuthMiddleware()`.                                                                                      |
+| `bodySizeLimit` | Default body size limit in bytes. Can be overridden per-adapter via `bodyLimitOptions`.                                                         |
+| `onError`       | Custom error handler called when an unhandled error occurs in a route handler. See [server.onError](https://mastra.ai/reference/configuration). |
 
 ### Adapter constructor only
 

package/.docs/guides/deployment/amazon-ec2.md

@@ -1,19 +1,23 @@
-# Amazon EC2
+# Deploy Mastra to Amazon EC2
 
-Deploy your Mastra applications to Amazon EC2 (Elastic Cloud Compute).
+Deploy your Mastra server to Amazon EC2. This gives you full control over your server environment and supports long-running agents and workflows.
 
-> **Note:** This guide assumes your Mastra application has been created using the default `npx create-mastra@latest` command. For more information on how to create a new Mastra application, refer to our [getting started guide](https://mastra.ai/guides/getting-started/quickstart)
+> **Info:** This guide covers deploying the [Mastra server](https://mastra.ai/docs/server/mastra-server). If you're using a [server adapter](https://mastra.ai/docs/server/server-adapters) or [web framework](https://mastra.ai/docs/deployment/web-framework), deploy the way you normally would for that framework.
 
-## Prerequisites
+## Before you begin
 
-- An AWS account with [EC2](https://aws.amazon.com/ec2/) access
-- An EC2 instance running Ubuntu 24+ or Amazon Linux
-- A domain name with an A record pointing to your instance
-- A reverse proxy configured (e.g., using [nginx](https://nginx.org/))
-- SSL certificate configured (e.g., using [Let's Encrypt](https://letsencrypt.org/))
-- Node.js 22.13.0 or later installed on your instance
+You'll need:
 
-## Deployment Steps
+- A [Mastra application](https://mastra.ai/guides/getting-started/quickstart)
+- An [EC2](https://aws.amazon.com/ec2/) instance (Ubuntu or Amazon Linux) with Git and Node.js 22.13.0+ installed
+
+For production, you'll also need:
+
+- A domain name pointing to your instance (required for SSL certificates)
+- An SSL certificate for your domain (e.g., using [Certbot](https://certbot.eff.org/) with Let's Encrypt)
+- A reverse proxy (e.g., [nginx](https://nginx.org/)) to forward traffic to your application
+
+## Deploy
 
 1. Connect to your EC2 instance and clone your repository:
 
@@ -26,7 +30,7 @@ Deploy your Mastra applications to Amazon EC2 (Elastic Cloud Compute).
    **Private Repository**:
 
    ```bash
-   git clone https://<your-username>:<your-personal-access-token>@github.com/<your-username>/<your-repository>.git
+   git clone git@github.com:<your-username>/<your-repository>.git
    ```
 
    Navigate to the repository directory:
@@ -67,7 +71,7 @@ Deploy your Mastra applications to Amazon EC2 (Elastic Cloud Compute).
    touch .env
    ```
 
-   Edit the `.env` file and add your environment variables:
+   Remember to set your environment variables needed to run your application (e.g. your model provider API key):
 
    ```bash
    OPENAI_API_KEY=<your-openai-api-key>
@@ -76,32 +80,51 @@ Deploy your Mastra applications to Amazon EC2 (Elastic Cloud Compute).
 
 4. Build the application:
 
+   **npm**:
+
    ```bash
    npm run build
    ```
 
+   **pnpm**:
+
+   ```bash
+   pnpm run build
+   ```
+
+   **Yarn**:
+
+   ```bash
+   yarn build
+   ```
+
+   **Bun**:
+
+   ```bash
+   bun run build
+   ```
+
 5. Run the application:
 
    ```bash
    node --env-file=".env" .mastra/output/index.mjs
    ```
 
-   > **Note:** Your Mastra application will run on port 4111 by default. Ensure your reverse proxy is configured to forward requests to this port.
+   This is a basic example. In production, use a process manager like [PM2](https://pm2.keymetrics.io/) or [systemd](https://systemd.io/) to keep your application running and handle restarts.
+
+   > **Warning:** Set up [authentication](https://mastra.ai/docs/server/auth) before exposing your endpoints publicly.
 
-## Connect to your Mastra server
+6. Your Mastra server is now running on port 4111, but it's only accessible locally.
 
-You can now connect to your Mastra server from your client application using a `MastraClient` from the `@mastra/client-js` package.
+   You can open port 4111 in your EC2 security group for direct access, or configure a reverse proxy (such as nginx) to listen on ports 80 and 443 and forward requests to `http://localhost:4111`.
 
-Refer to the [`MastraClient` documentation](https://mastra.ai/reference/client-js/mastra-client) for more information.
+   In production, you should use a reverse proxy so you can configure HTTPS. HTTPS encrypts traffic and is required for most webhook integrations and external services your agents interact with.
 
-```typescript
-import { MastraClient } from "@mastra/client-js";
+7. Verify your deployment at `https://<your-ec2-address>/api/agents`, which should return a JSON list of your agents.
 
-const mastraClient = new MastraClient({
-  baseUrl: "https://<your-domain-name>",
-});
-```
+8. You can now call your Mastra endpoints over HTTP.
 
 ## Next steps
 
-- [Mastra Client SDK](https://mastra.ai/reference/client-js/mastra-client)
+- [Connect from a Mastra Client](https://mastra.ai/docs/server/mastra-client)
+- [Deployment overview](https://mastra.ai/docs/deployment/overview)