@mastra/mcp-docs-server 1.1.2-alpha.0 → 1.1.3-alpha.0

package/.docs/reference/datasets/compareExperiments.md

@@ -0,0 +1,48 @@
+# DatasetsManager.compareExperiments()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Compares two or more experiments, producing per-item and per-scorer comparisons. Requires at least two experiment IDs.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const comparison = await mastra.datasets.compareExperiments({
+  experimentIds: ["exp-baseline", "exp-new"],
+  baselineId: "exp-baseline",
+});
+
+console.log(`Baseline: ${comparison.baselineId}`);
+
+for (const item of comparison.items) {
+  console.log(`Item ${item.itemId}:`);
+  console.log(`  Input: ${JSON.stringify(item.input)}`);
+
+  for (const [expId, result] of Object.entries(item.results)) {
+    if (result) {
+      console.log(`  ${expId}: output=${JSON.stringify(result.output)}, scores=${JSON.stringify(result.scores)}`);
+    }
+  }
+}
+```
+
+## Parameters
+
+**experimentIds:** (`string[]`): Array of experiment IDs to compare. Must contain at least 2.
+
+**baselineId?:** (`string`): ID of the baseline experiment. Defaults to the first ID in \`experimentIds\`.
+
+## Returns
+
+Throws `MastraError` if fewer than 2 experiment IDs are provided.
+
+**result:** (`Promise<object>`): objectbaselineId:stringID of the baseline experiment used for comparison.items:Array\<object>Per-item comparison data.objectitemId:stringID of the dataset item.input:unknown | nullInput data for the item.groundTruth:unknown | nullGround truth for the item.results:Record\<string, { output: unknown; scores: Record\<string, number | null> } | null>Results keyed by experiment ID. Each entry contains the output and scorer results for that experiment.
+
+## Related
+
+- [dataset.startExperiment()](https://mastra.ai/reference/datasets/startExperiment)
+- [dataset.listExperiments()](https://mastra.ai/reference/datasets/listExperiments)

package/.docs/reference/datasets/create.md

@@ -0,0 +1,49 @@
+# DatasetsManager.create()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Creates a new dataset. Zod schemas passed as `inputSchema` or `groundTruthSchema` are automatically converted to JSON Schema.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+import { z } from "zod";
+
+const mastra = new Mastra({ /* storage config */ });
+
+// Create with metadata
+const dataset = await mastra.datasets.create({
+  name: "QA evaluation set",
+  description: "Question-answer pairs for testing",
+  metadata: { team: "ml" },
+});
+
+// Create with Zod schemas
+const typedDataset = await mastra.datasets.create({
+  name: "Typed QA set",
+  inputSchema: z.object({
+    question: z.string(),
+    context: z.string().optional(),
+  }),
+  groundTruthSchema: z.object({
+    answer: z.string(),
+  }),
+});
+```
+
+## Parameters
+
+**name:** (`string`): Display name for the dataset.
+
+**description?:** (`string`): Description of the dataset.
+
+**inputSchema?:** (`unknown`): JSON Schema or Zod schema for item inputs. Zod schemas are auto-converted.
+
+**groundTruthSchema?:** (`unknown`): JSON Schema or Zod schema for item ground truths. Zod schemas are auto-converted.
+
+**metadata?:** (`Record<string, unknown>`): Arbitrary metadata.
+
+## Returns
+
+**result:** (`Promise<Dataset>`): A Dataset instance for the newly created dataset.

package/.docs/reference/datasets/dataset.md

@@ -0,0 +1,78 @@
+# Dataset Class
+
+**Added in:** `@mastra/core@1.4.0`
+
+The `Dataset` class provides methods for item CRUD, versioning, and experiment management on a single dataset. Obtained via `mastra.datasets.get()` or `mastra.datasets.create()`.
+
+## Usage examples
+
+### Add items and run an experiment
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.create({
+  name: "QA pairs",
+});
+
+// Add items
+await dataset.addItems({
+  items: [
+    { input: { question: "What is AI?" }, groundTruth: { answer: "Artificial Intelligence" } },
+    { input: { question: "What is ML?" }, groundTruth: { answer: "Machine Learning" } },
+  ],
+});
+
+// Run an experiment
+const summary = await dataset.startExperiment({
+  targetType: "agent",
+  targetId: "my-agent",
+  scorers: ["accuracy"],
+});
+
+console.log(`${summary.succeededCount}/${summary.totalItems} succeeded`);
+```
+
+### Manage versions and history
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+// List versions
+const { versions } = await dataset.listVersions();
+
+// List items at a specific version
+const items = await dataset.listItems({ version: 2 });
+
+// Get item change history
+const history = await dataset.getItemHistory({ itemId: "item-id" });
+```
+
+## Access
+
+`Dataset` is not instantiated directly. Obtain one via `DatasetsManager`:
+
+```typescript
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+// or
+const dataset = await mastra.datasets.create({ name: "My dataset" });
+```
+
+## Properties
+
+For the full dataset record (name, description, schemas, version, timestamps), call [`dataset.getDetails()`](https://mastra.ai/reference/datasets/getDetails).
+
+**id:** (`string`): The unique identifier of the dataset. Read-only.
+
+## Related
+
+- [DatasetsManager class](https://mastra.ai/reference/datasets/datasets-manager)
+- [dataset.startExperiment()](https://mastra.ai/reference/datasets/startExperiment)
+- [dataset.addItems()](https://mastra.ai/reference/datasets/addItems)
+- [dataset.listVersions()](https://mastra.ai/reference/datasets/listVersions)

package/.docs/reference/datasets/datasets-manager.md

@@ -0,0 +1,84 @@
+# DatasetsManager
+
+**Added in:** `@mastra/core@1.4.0`
+
+The `DatasetsManager` class provides the public API for managing datasets, including CRUD operations and cross-dataset experiment comparisons. Access it via `mastra.datasets`.
+
+## Usage examples
+
+### Create and list datasets
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+// Create a dataset
+const dataset = await mastra.datasets.create({
+  name: "QA pairs",
+  description: "Question-answer evaluation set",
+});
+
+// List all datasets
+const { datasets } = await mastra.datasets.list();
+```
+
+### Get and delete a dataset
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+// Get by ID
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+// Delete by ID
+await mastra.datasets.delete({ id: "dataset-id" });
+```
+
+### Get experiment
+
+Retrieves a specific experiment (run) by ID. Unlike `dataset.getExperiment()`, this works across all datasets without needing a dataset reference first.
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+// Get experiment directly without knowing the dataset
+const experiment = await mastra.datasets.getExperiment({
+  experimentId: "exp-id",
+});
+
+console.log(`Dataset: ${experiment.datasetId}`);
+console.log(`Status: ${experiment.status}`);
+```
+
+### Compare experiments
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const comparison = await mastra.datasets.compareExperiments({
+  experimentIds: ["exp-1", "exp-2"],
+  baselineId: "exp-1",
+});
+```
+
+## Access
+
+`DatasetsManager` is not instantiated directly. Access it from a `Mastra` instance:
+
+```typescript
+const mastra = new Mastra({ /* storage config */ });
+const datasetsManager = mastra.datasets;
+```
+
+## Related
+
+- [Dataset class](https://mastra.ai/reference/datasets/dataset)
+- [DatasetsManager.create()](https://mastra.ai/reference/datasets/create)
+- [DatasetsManager.compareExperiments()](https://mastra.ai/reference/datasets/compareExperiments)

package/.docs/reference/datasets/delete.md

@@ -0,0 +1,23 @@
+# DatasetsManager.delete()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Deletes a dataset by ID, including all items, versions, and experiments.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+await mastra.datasets.delete({ id: "dataset-id" });
+```
+
+## Parameters
+
+**id:** (`string`): ID of the dataset to delete.
+
+## Returns
+
+**result:** (`Promise<void>`): Resolves when the dataset is deleted.

package/.docs/reference/datasets/deleteExperiment.md

@@ -0,0 +1,25 @@
+# dataset.deleteExperiment()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Deletes an experiment (run) by ID, including all associated results.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+await dataset.deleteExperiment({ experimentId: "exp-id" });
+```
+
+## Parameters
+
+**experimentId:** (`string`): ID of the experiment to delete.
+
+## Returns
+
+**result:** (`Promise<void>`): Resolves when the experiment and its results are deleted.

package/.docs/reference/datasets/deleteItem.md

@@ -0,0 +1,25 @@
+# dataset.deleteItem()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Deletes a single item from the dataset by ID.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+await dataset.deleteItem({ itemId: "item-id" });
+```
+
+## Parameters
+
+**itemId:** (`string`): ID of the item to delete.
+
+## Returns
+
+**result:** (`Promise<void>`): Resolves when the item is deleted.

package/.docs/reference/datasets/deleteItems.md

@@ -0,0 +1,27 @@
+# dataset.deleteItems()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Deletes multiple items from the dataset in a single bulk operation.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+await dataset.deleteItems({
+  itemIds: ["item-1", "item-2", "item-3"],
+});
+```
+
+## Parameters
+
+**itemIds:** (`string[]`): Array of item IDs to delete.
+
+## Returns
+
+**result:** (`Promise<void>`): Resolves when all items are deleted.

package/.docs/reference/datasets/get.md

@@ -0,0 +1,29 @@
+# DatasetsManager.get()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Retrieves an existing dataset by ID. Throws a `MastraError` if the dataset does not exist.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+// Now use the dataset
+const details = await dataset.getDetails();
+console.log(details.name);
+```
+
+## Parameters
+
+**id:** (`string`): Unique identifier of the dataset.
+
+## Returns
+
+Throws `MastraError` if the dataset is not found.
+
+**result:** (`Promise<Dataset>`): A Dataset instance.

package/.docs/reference/datasets/getDetails.md

@@ -0,0 +1,45 @@
+# dataset.getDetails()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Retrieves the full dataset record from storage, including metadata, schemas, and version information.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+// Get dataset details
+const details = await dataset.getDetails();
+console.log(details.name, details.version);
+```
+
+## Parameters
+
+This method takes no parameters.
+
+## Returns
+
+Returns a `Promise<DatasetRecord>`. Throws `MastraError` if the dataset is not found.
+
+**id:** (`string`): Unique identifier of the dataset.
+
+**name:** (`string`): Display name of the dataset.
+
+**description?:** (`string`): Description of the dataset.
+
+**metadata?:** (`Record<string, unknown>`): Arbitrary metadata associated with the dataset.
+
+**inputSchema?:** (`Record<string, unknown>`): JSON Schema for item inputs.
+
+**groundTruthSchema?:** (`Record<string, unknown>`): JSON Schema for item ground truths.
+
+**version:** (`number`): Current version number of the dataset.
+
+**createdAt:** (`Date`): When the dataset was created.
+
+**updatedAt:** (`Date`): When the dataset was last updated.

package/.docs/reference/datasets/getExperiment.md

@@ -0,0 +1,28 @@
+# dataset.getExperiment()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Retrieves a specific experiment (run) by its ID.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+const experiment = await dataset.getExperiment({ experimentId: "exp-id" });
+
+console.log(`Status: ${experiment.status}`);
+console.log(`${experiment.succeededCount}/${experiment.totalItems} succeeded`);
+```
+
+## Parameters
+
+**experimentId:** (`string`): ID of the experiment to retrieve.
+
+## Returns
+
+**result:** (`Promise<Experiment>`): The experiment record. See dataset.listExperiments() for the full shape.

package/.docs/reference/datasets/getItem.md

@@ -0,0 +1,31 @@
+# dataset.getItem()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Retrieves a single dataset item by ID, optionally at a specific dataset version.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+// Get latest version of an item
+const item = await dataset.getItem({ itemId: "item-id" });
+
+// Get item at a specific dataset version
+const versionedItem = await dataset.getItem({ itemId: "item-id", version: 2 });
+```
+
+## Parameters
+
+**itemId:** (`string`): The unique identifier of the item.
+
+**version?:** (`number`): Dataset version to retrieve the item at. Defaults to the latest version.
+
+## Returns
+
+**result:** (`Promise<DatasetItem | null>`): The dataset item, or null if not found. See dataset.addItem() for the item shape.

package/.docs/reference/datasets/getItemHistory.md

@@ -0,0 +1,29 @@
+# dataset.getItemHistory()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Retrieves the full SCD-2 (Slowly Changing Dimension Type 2) history of a specific item across all dataset versions.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+const history = await dataset.getItemHistory({ itemId: "item-id" });
+
+for (const row of history) {
+  console.log(`Version ${row.datasetVersion}: ${row.isDeleted ? "deleted" : "active"}`);
+}
+```
+
+## Parameters
+
+**itemId:** (`string`): ID of the item to retrieve history for.
+
+## Returns
+
+**result:** (`Promise<DatasetItemRow[]>`): DatasetItemRowid:stringUnique row identifier.datasetId:stringID of the parent dataset.datasetVersion:numberDataset version for this row\.validTo:number | nullVersion at which this row was superseded, or \`null\` if still current.isDeleted:booleanWhether this row represents a deletion.input:unknownInput data at this version.groundTruth?:unknownGround truth at this version.metadata?:Record\<string, unknown>Metadata at this version.createdAt:DateWhen this row was created.updatedAt:DateWhen this row was last updated.

package/.docs/reference/datasets/list.md

@@ -0,0 +1,29 @@
+# DatasetsManager.list()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists all datasets with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const { datasets, pagination } = await mastra.datasets.list({ page: 0, perPage: 10 });
+
+for (const ds of datasets) {
+  console.log(`${ds.id}: ${ds.name} (v${ds.version})`);
+}
+```
+
+## Parameters
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of datasets per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectdatasets:DatasetRecord\[]Array of dataset records. See dataset.getDetails() for the record shape.pagination:PaginationInfoPagination metadata with \`total\`, \`page\`, \`perPage\`, and \`hasMore\`.

package/.docs/reference/datasets/listExperimentResults.md

@@ -0,0 +1,37 @@
+# dataset.listExperimentResults()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists individual item results for a specific experiment with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+const { results, pagination } = await dataset.listExperimentResults({
+  experimentId: "exp-id",
+  page: 0,
+  perPage: 50,
+});
+
+for (const result of results) {
+  console.log(`Item ${result.itemId}: ${result.error ? "FAILED" : "OK"}`);
+}
+```
+
+## Parameters
+
+**experimentId:** (`string`): ID of the experiment to list results for.
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of results per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectresults:ExperimentResult\[]Array of item-level results.ExperimentResultid:stringUnique result ID.experimentId:stringID of the parent experiment.itemId:stringID of the dataset item.itemDatasetVersion:number | nullDataset version of the item when executed.input:unknownInput data passed to the target.output:unknown | nullOutput from the target.groundTruth:unknown | nullExpected output.error:{ message: string; stack?: string; code?: string } | nullStructured error if execution failed.startedAt:DateWhen execution started.completedAt:DateWhen execution completed.retryCount:numberNumber of retry attempts.traceId:string | nullTrace ID for observability.createdAt:DateWhen the result record was created.pagination:PaginationInfoPagination metadata with \`total\`, \`page\`, \`perPage\`, and \`hasMore\`.

package/.docs/reference/datasets/listExperiments.md

@@ -0,0 +1,31 @@
+# dataset.listExperiments()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists all experiments (runs) for this dataset with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+const { experiments, pagination } = await dataset.listExperiments({ page: 0, perPage: 10 });
+
+for (const exp of experiments) {
+  console.log(`${exp.id}: ${exp.status} (${exp.succeededCount}/${exp.totalItems})`);
+}
+```
+
+## Parameters
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of experiments per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectexperiments:Experiment\[]Array of experiment records.Experimentid:stringUnique experiment ID.name?:stringDisplay name.description?:stringDescription.metadata?:Record\<string, unknown>Arbitrary metadata.datasetId:stringID of the parent dataset.datasetVersion:number | nullDataset version used for the experiment.targetType:'agent' | 'workflow' | 'scorer' | 'processor'Type of target used.targetId:stringID of the target used.status:'pending' | 'running' | 'completed' | 'failed'Current status of the experiment.totalItems:numberTotal number of items.succeededCount:numberNumber of successful items.failedCount:numberNumber of failed items.skippedCount:numberNumber of skipped items.startedAt:Date | nullWhen the experiment started.completedAt:Date | nullWhen the experiment completed.createdAt:DateWhen the experiment record was created.updatedAt:DateWhen the experiment record was last updated.pagination:PaginationInfoPagination metadata with \`total\`, \`page\`, \`perPage\`, and \`hasMore\`.

package/.docs/reference/datasets/listItems.md

@@ -0,0 +1,44 @@
+# dataset.listItems()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists items in the dataset. When a `version` is specified, returns all items at that version. Otherwise, returns paginated items from the latest version.
+
+## Usage example
+
+```typescript
+import { Mastra } from "@mastra/core";
+
+const mastra = new Mastra({ /* storage config */ });
+
+const dataset = await mastra.datasets.get({ id: "dataset-id" });
+
+// Paginated list (default: page 0, 20 per page)
+const result = await dataset.listItems();
+
+// List with search
+const filtered = await dataset.listItems({ search: "TypeScript", page: 0, perPage: 10 });
+
+// List all items at a specific version
+const versionedItems = await dataset.listItems({ version: 2 });
+```
+
+## Parameters
+
+**version?:** (`number`): Dataset version to list items at. When set, returns all items at that version (no pagination).
+
+**page?:** (`number`): Page number for pagination. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of items per page. Defaults to \`20\`.
+
+**search?:** (`string`): Search string to filter items.
+
+## Returns
+
+When `version` is specified:
+
+**result:** (`Promise<DatasetItem[]>`): Array of all items at the specified dataset version.
+
+When `version` is not specified:
+
+**result:** (`Promise<object>`): objectitems:DatasetItem\[]Array of items for the current page.pagination:objectPagination metadata.objecttotal:numberTotal number of items.page:numberCurrent page number.perPage:number | falseItems per page, or \`false\` if unpaginated.hasMore:booleanWhether more pages are available.