@mastra/mcp-docs-server 1.1.4 → 1.1.5-alpha.0

package/.docs/docs/observability/datasets/overview.md

@@ -1,188 +0,0 @@
-# Datasets Overview
-
-**Added in:** `@mastra/core@1.4.0`
-
-Datasets are collections of test cases that you run experiments against to measure how well your agents and workflows perform. Each mutation creates a new version, so you can reproduce past experiments exactly. Pair datasets with [scorers](https://mastra.ai/docs/evals/overview) to track quality across prompts, models, or code changes.
-
-## Usage
-
-### Configure storage
-
-Configure storage in your Mastra instance. Datasets require a storage adapter that provides the `datasets` domain:
-
-```typescript
-import { Mastra } from "@mastra/core";
-import { LibSQLStore } from "@mastra/libsql";
-
-export const mastra = new Mastra({
-  storage: new LibSQLStore({
-    id: "my-store",
-    url: "file:./mastra.db",
-  }),
-});
-```
-
-### Accessing the datasets API
-
-All dataset operations are available through `mastra.datasets`:
-
-```typescript
-const datasets = mastra.datasets;
-
-// Create a dataset
-const dataset = await datasets.create({ name: "my-dataset" });
-
-// Retrieve an existing dataset
-const existing = await datasets.get({ id: "dataset-id" });
-
-// List all datasets
-const { datasets: all } = await datasets.list();
-```
-
-> **Info:** Visit the [`DatasetsManager` reference](https://mastra.ai/reference/datasets/datasets-manager) for the full list of methods.
-
-## Creating a dataset
-
-Call [`create()`](https://mastra.ai/reference/datasets/create) with a name and optional description:
-
-```typescript
-import { mastra } from "../index";
-
-const dataset = await mastra.datasets.create({
-  name: "translation-pairs",
-  description: "English to Spanish translation test cases",
-});
-
-console.log(dataset.id); // auto-generated UUID
-```
-
-### Defining schemas
-
-You can enforce the shape of `input` and `groundTruth` by passing Zod schemas. Mastra converts them to JSON Schema at creation time:
-
-```typescript
-import { z } from "zod";
-import { mastra } from "../index";
-
-const dataset = await mastra.datasets.create({
-  name: "translation-pairs",
-  inputSchema: z.object({
-    text: z.string(),
-    sourceLang: z.string(),
-    targetLang: z.string(),
-  }),
-  groundTruthSchema: z.object({
-    translation: z.string(),
-  }),
-});
-```
-
-Items that don't match the schema are rejected at insert time.
-
-## Adding items
-
-Use [`addItem()`](https://mastra.ai/reference/datasets/addItem) for a single item or [`addItems()`](https://mastra.ai/reference/datasets/addItems) to insert in bulk:
-
-```typescript
-// Single item
-await dataset.addItem({
-  input: { text: "Hello", sourceLang: "en", targetLang: "es" },
-  groundTruth: { translation: "Hola" },
-});
-
-// Bulk insert
-await dataset.addItems({
-  items: [
-    {
-      input: { text: "Goodbye", sourceLang: "en", targetLang: "es" },
-      groundTruth: { translation: "Adiós" },
-    },
-    {
-      input: { text: "Thank you", sourceLang: "en", targetLang: "es" },
-      groundTruth: { translation: "Gracias" },
-    },
-  ],
-});
-```
-
-## Updating and deleting items
-
-[`updateItem()`](https://mastra.ai/reference/datasets/updateItem), [`deleteItem()`](https://mastra.ai/reference/datasets/deleteItem), and [`deleteItems()`](https://mastra.ai/reference/datasets/deleteItems) let you modify or remove existing items by `itemId`:
-
-```typescript
-await dataset.updateItem({
-  itemId: "item-abc-123",
-  groundTruth: { translation: "¡Hola!" },
-});
-
-await dataset.deleteItem({ itemId: "item-abc-123" });
-
-await dataset.deleteItems({ itemIds: ["item-1", "item-2"] });
-```
-
-## Listing and searching items
-
-[`listItems()`](https://mastra.ai/reference/datasets/listItems) supports pagination and full-text search:
-
-```typescript
-// Paginated list
-const { items, pagination } = await dataset.listItems({
-  page: 0,
-  perPage: 50,
-});
-
-// Full-text search
-const { items: matches } = await dataset.listItems({
-  search: "Hello",
-});
-
-// List items at a specific version
-const v2Items = await dataset.listItems({ version: 2 });
-```
-
-## Versioning
-
-Every mutation to a dataset's items (add, update, or delete) bumps the dataset version. This lets you pin experiments to a specific snapshot of the data.
-
-### Listing versions
-
-Use [`listVersions()`](https://mastra.ai/reference/datasets/listVersions) to see the paginated history of versions:
-
-```typescript
-const { versions, pagination } = await dataset.listVersions();
-
-for (const v of versions) {
-  console.log(`Version ${v.version} — created ${v.createdAt}`);
-}
-```
-
-### Viewing item history
-
-See how a specific item changed across versions by calling [`getItemHistory()`](https://mastra.ai/reference/datasets/getItemHistory) with the `itemId`:
-
-```typescript
-const history = await dataset.getItemHistory({ itemId: "item-abc-123" });
-
-for (const row of history) {
-  console.log(`Version ${row.datasetVersion}`, row.input, row.groundTruth);
-}
-```
-
-### Pinning to a version
-
-Fetch the exact items that existed at a past version:
-
-```typescript
-const items = await dataset.listItems({ version: 2 });
-```
-
-You can also pin experiments to a version, see [running experiments](https://mastra.ai/docs/observability/datasets/running-experiments).
-
-> **Info:** Visit the [`Dataset` reference](https://mastra.ai/reference/datasets/dataset) for the full list of methods and parameters.
-
-## Related
-
-- [Running experiments](https://mastra.ai/docs/observability/datasets/running-experiments)
-- [Scorers overview](https://mastra.ai/docs/evals/overview)
-- [DatasetsManager reference](https://mastra.ai/reference/datasets/datasets-manager)
-- [Dataset reference](https://mastra.ai/reference/datasets/dataset)

package/.docs/docs/observability/datasets/running-experiments.md

@@ -1,266 +0,0 @@
-# Running Experiments
-
-**Added in:** `@mastra/core@1.4.0`
-
-An experiment runs every item in a dataset through a target (an agent, a workflow, or a scorer) and then optionally scores the outputs. Use a scorer as the target when you want to evaluate an LLM judge itself. Results are persisted to storage so you can compare runs across different prompts, models, or code changes.
-
-## Basic experiment
-
-Call [`startExperiment()`](https://mastra.ai/reference/datasets/startExperiment) with a target and scorers:
-
-```typescript
-import { mastra } from "../index";
-
-const dataset = await mastra.datasets.get({ id: "translation-dataset-id" });
-
-const summary = await dataset.startExperiment({
-  name: "gpt-5.1-baseline",
-  targetType: "agent",
-  targetId: "translation-agent",
-  scorers: ["accuracy", "fluency"],
-});
-
-console.log(summary.status);          // 'completed' | 'failed'
-console.log(summary.succeededCount);  // number of items that ran successfully
-console.log(summary.failedCount);     // number of items that failed
-```
-
-`startExperiment()` blocks until all items finish. For fire-and-forget execution, see [async experiments](#async-experiments).
-
-## Experiment targets
-
-You can point an experiment at a registered agent, workflow, or scorer.
-
-### Registered agent
-
-Point to an agent registered on your Mastra instance:
-
-```typescript
-const summary = await dataset.startExperiment({
-  name: "agent-v2-eval",
-  targetType: "agent",
-  targetId: "translation-agent",
-  scorers: ["accuracy"],
-});
-```
-
-Each item's `input` is passed directly to `agent.generate()`, so it must be a `string`, `string[]`, or `CoreMessage[]`.
-
-### Registered workflow
-
-Point to a workflow registered on your Mastra instance:
-
-```typescript
-const summary = await dataset.startExperiment({
-  name: "workflow-eval",
-  targetType: "workflow",
-  targetId: "translation-workflow",
-  scorers: ["accuracy"],
-});
-```
-
-The workflow receives each item's `input` as its trigger data.
-
-### Registered scorer
-
-Point to a scorer to evaluate an LLM judge against ground truth:
-
-```typescript
-const summary = await dataset.startExperiment({
-  name: "judge-accuracy-eval",
-  targetType: "scorer",
-  targetId: "accuracy",
-});
-```
-
-The scorer receives each item's `input` and `groundTruth`. LLM-based judges can drift over time as underlying models change, so it's important to periodically realign them against known-good labels. A dataset gives you a stable benchmark to detect that drift.
-
-## Scoring results
-
-Scorers automatically run after each item's target execution. Pass scorer instances or registered scorer IDs:
-
-**Scorer IDs**:
-
-```typescript
-// Reference scorers registered on the Mastra instance
-const summary = await dataset.startExperiment({
-  name: "with-registered-scorers",
-  targetType: "agent",
-  targetId: "translation-agent",
-  scorers: ["accuracy", "fluency"],
-});
-```
-
-**Scorer instances**:
-
-```typescript
-import { createAnswerRelevancyScorer } from "@mastra/evals/scorers/prebuilt";
-
-const relevancy = createAnswerRelevancyScorer({ model: "openai/gpt-4.1-nano" });
-
-const summary = await dataset.startExperiment({
-  name: "with-scorer-instances",
-  targetType: "agent",
-  targetId: "translation-agent",
-  scorers: [relevancy],
-});
-```
-
-Each item's results include per-scorer scores:
-
-```typescript
-for (const item of summary.results) {
-  console.log(item.itemId, item.output);
-  for (const score of item.scores) {
-    console.log(`  ${score.scorerName}: ${score.score} — ${score.reason}`);
-  }
-}
-```
-
-> **Info:** Visit the [Scorers overview](https://mastra.ai/docs/evals/overview) for details on available and custom scorers.
-
-## Async experiments
-
-`startExperiment()` blocks until every item completes. For long-running datasets, use [`startExperimentAsync()`](https://mastra.ai/reference/datasets/startExperimentAsync) to start the experiment in the background:
-
-```typescript
-const { experimentId, status } = await dataset.startExperimentAsync({
-  name: "large-dataset-run",
-  targetType: "agent",
-  targetId: "translation-agent",
-  scorers: ["accuracy"],
-});
-
-console.log(experimentId); // UUID
-console.log(status);       // 'pending'
-```
-
-Poll for completion using [`getExperiment()`](https://mastra.ai/reference/datasets/getExperiment):
-
-```typescript
-let experiment = await dataset.getExperiment({ experimentId });
-
-while (experiment.status === "pending" || experiment.status === "running") {
-  await new Promise(resolve => setTimeout(resolve, 5000));
-  experiment = await dataset.getExperiment({ experimentId });
-}
-
-console.log(experiment.status); // 'completed' | 'failed'
-```
-
-## Configuration options
-
-### Concurrency
-
-Control how many items run in parallel (default: 5):
-
-```typescript
-const summary = await dataset.startExperiment({
-  targetType: "agent",
-  targetId: "translation-agent",
-  maxConcurrency: 10,
-});
-```
-
-### Timeouts and retries
-
-Set a per-item timeout (in milliseconds) and retry count:
-
-```typescript
-const summary = await dataset.startExperiment({
-  targetType: "agent",
-  targetId: "translation-agent",
-  itemTimeout: 30_000,  // 30 seconds per item
-  maxRetries: 2,        // retry failed items up to 2 times
-});
-```
-
-Retries use exponential backoff. Abort errors are never retried.
-
-### Aborting an experiment
-
-Pass an `AbortSignal` to cancel a running experiment:
-
-```typescript
-const controller = new AbortController();
-
-// Cancel after 60 seconds
-setTimeout(() => controller.abort(), 60_000);
-
-const summary = await dataset.startExperiment({
-  targetType: "agent",
-  targetId: "translation-agent",
-  signal: controller.signal,
-});
-```
-
-Remaining items are marked as skipped in the summary.
-
-### Pinning a dataset version
-
-Run against a specific snapshot of the dataset:
-
-```typescript
-const summary = await dataset.startExperiment({
-  targetType: "agent",
-  targetId: "translation-agent",
-  version: 3, // use items from dataset version 3
-});
-```
-
-## Viewing results
-
-### Listing experiments
-
-```typescript
-const { experiments, pagination } = await dataset.listExperiments({
-  page: 0,
-  perPage: 10,
-});
-
-for (const exp of experiments) {
-  console.log(`${exp.name} — ${exp.status} (${exp.succeededCount}/${exp.totalItems})`);
-}
-```
-
-### Experiment details
-
-```typescript
-const experiment = await dataset.getExperiment({
-  experimentId: "exp-abc-123",
-});
-
-console.log(experiment.status);
-console.log(experiment.startedAt);
-console.log(experiment.completedAt);
-```
-
-### Item-level results
-
-```typescript
-const { results, pagination } = await dataset.listExperimentResults({
-  experimentId: "exp-abc-123",
-  page: 0,
-  perPage: 50,
-});
-
-for (const result of results) {
-  console.log(result.itemId, result.output, result.error);
-}
-```
-
-## Understanding the summary
-
-`startExperiment()` returns an `ExperimentSummary` with counts and per-item results:
-
-- `completedWithErrors` is `true` when the experiment finished but some items failed.
-- Items cancelled via `signal` appear in `skippedCount`.
-
-> **Info:** Visit the [`startExperiment` reference](https://mastra.ai/reference/datasets/startExperiment) for the full parameter and return type documentation.
-
-## Related
-
-- [Datasets overview](https://mastra.ai/docs/observability/datasets/overview)
-- [Scorers overview](https://mastra.ai/docs/evals/overview)
-- [`startExperiment` reference](https://mastra.ai/reference/datasets/startExperiment)
-- [`listExperimentResults` reference](https://mastra.ai/reference/datasets/listExperimentResults)

package/.docs/docs/observability/logging.md

@@ -1,99 +0,0 @@
-# Logging
-
-Mastra's logging system captures function execution, input data, and output responses in a structured format.
-
-When deploying to Mastra Cloud, logs are shown on the [Logs](https://mastra.ai/docs/mastra-cloud/observability) page. In self-hosted or custom environments, logs can be directed to files or external services depending on the configured transports.
-
-## Configuring logs with PinoLogger
-
-When [initializing a new Mastra project](https://mastra.ai/guides/getting-started/quickstart) using the CLI, `PinoLogger` is included by default.
-
-```typescript
-import { Mastra } from "@mastra/core/mastra";
-import { PinoLogger } from "@mastra/loggers";
-
-export const mastra = new Mastra({
-  logger: new PinoLogger({
-    name: "Mastra",
-    level: "info",
-  }),
-});
-```
-
-> **Info:** Visit [PinoLogger](https://mastra.ai/reference/logging/pino-logger) for all available configuration options.
-
-## Customizing logs
-
-Mastra provides access to a logger instance via the `mastra.getLogger()` method, available inside both workflow steps and tools. The logger supports standard severity levels: `debug`, `info`, `warn`, and `error`.
-
-### Logging from workflow steps
-
-Within a workflow step, access the logger via the `mastra` parameter inside the `execute` function. This allows you to log messages relevant to the step’s execution.
-
-```typescript
-import { createWorkflow, createStep } from "@mastra/core/workflows";
-import { z } from "zod";
-
-const step1 = createStep({
-  execute: async ({ mastra }) => {
-    const logger = mastra.getLogger();
-    logger.info("workflow info log");
-
-    return {
-      output: ""
-    };
-  }
-});
-
-export const testWorkflow = createWorkflow({...})
-  .then(step1)
-  .commit();
-```
-
-### Logging from tools
-
-Similarly, tools have access to the logger instance via the `mastra` parameter. Use this to log tool specific activity during execution.
-
-```typescript
-import { createTool } from "@mastra/core/tools";
-import { z } from "zod";
-
-export const testTool = createTool({
-  execute: async (inputData, context) => {
-    const logger = context?.mastra.getLogger();
-    logger?.info("tool info log");
-
-    return {
-      output: "",
-    };
-  },
-});
-```
-
-### Logging with additional data
-
-Logger methods accept an optional second argument for additional data. This can be any value, such as an object, string, or number.
-
-In this example, the log message includes an object with a key of `agent` and a value of the `testAgent` instance.
-
-```typescript
-import { createWorkflow, createStep } from "@mastra/core/workflows";
-import { z } from "zod";
-
-const step1 = createStep({
-  execute: async ({ mastra }) => {
-    const testAgent = mastra.getAgent("testAgent");
-    const logger = mastra.getLogger();
-
-    logger.info("workflow info log", { agent: testAgent });
-
-    return {
-      output: ""
-    };
-  }
-});
-
-export const testWorkflow = createWorkflow({...})
-  .then(step1)
-  .commit();
-```

package/.docs/docs/observability/overview.md

@@ -1,70 +0,0 @@
-# Observability Overview
-
-Mastra provides observability features for AI applications. Monitor LLM operations, trace agent decisions, and debug complex workflows with tools that understand AI-specific patterns.
-
-## Key Features
-
-### Tracing
-
-Specialized tracing for AI operations that captures:
-
-- **Model interactions**: Token usage, latency, prompts, and completions
-- **Agent execution**: Decision paths, tool calls, and memory operations
-- **Workflow steps**: Branching logic, parallel execution, and step outputs
-- **Automatic instrumentation**: Tracing with decorators
-
-## Storage Requirements
-
-The `DefaultExporter` persists traces to your configured storage backend. Not all storage providers support observability—for the full list, see [Storage Provider Support](https://mastra.ai/docs/observability/tracing/exporters/default).
-
-For production environments with high traffic, we recommend using **ClickHouse** for the observability domain via [composite storage](https://mastra.ai/reference/storage/composite). See [Production Recommendations](https://mastra.ai/docs/observability/tracing/exporters/default) for details.
-
-## Quick Start
-
-Configure Observability in your Mastra instance:
-
-```typescript
-import { Mastra } from "@mastra/core";
-import { PinoLogger } from "@mastra/loggers";
-import { LibSQLStore } from "@mastra/libsql";
-import {
-  Observability,
-  DefaultExporter,
-  CloudExporter,
-  SensitiveDataFilter,
-} from "@mastra/observability";
-
-export const mastra = new Mastra({
-  logger: new PinoLogger(),
-  storage: new LibSQLStore({
-    id: 'mastra-storage',
-    url: "file:./mastra.db", // Storage is required for tracing
-  }),
-  observability: new Observability({
-    configs: {
-      default: {
-        serviceName: "mastra",
-        exporters: [
-          new DefaultExporter(), // Persists traces to storage for Mastra Studio
-          new CloudExporter(), // Sends traces to Mastra Cloud (if MASTRA_CLOUD_ACCESS_TOKEN is set)
-        ],
-        spanOutputProcessors: [
-          new SensitiveDataFilter(), // Redacts sensitive data like passwords, tokens, keys
-        ],
-      },
-    },
-  }),
-});
-```
-
-> **Serverless environments:** The `file:./mastra.db` storage URL uses the local filesystem, which doesn't work in serverless environments like Vercel, AWS Lambda, or Cloudflare Workers. For serverless deployments, use external storage. See the [Vercel deployment guide](https://mastra.ai/guides/deployment/vercel) for a complete example.
-
-With this basic setup, you will see Traces and Logs in both Studio and in Mastra Cloud.
-
-We also support various external tracing providers like MLflow, Langfuse, Braintrust, and any OpenTelemetry-compatible platform (Datadog, New Relic, SigNoz, etc.). See more about this in the [Tracing](https://mastra.ai/docs/observability/tracing/overview) documentation.
-
-## What's Next?
-
-- **[Set up Tracing](https://mastra.ai/docs/observability/tracing/overview)**: Configure tracing for your application
-- **[Configure Logging](https://mastra.ai/docs/observability/logging)**: Add structured logging
-- **[API Reference](https://mastra.ai/reference/observability/tracing/instances)**: Detailed configuration options