@mastra/core 1.7.0 → 1.8.0

package/dist/docs/references/reference-datasets-getItemHistory.md

@@ -0,0 +1,31 @@
+# dataset.getItemHistory()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Retrieves the full SCD-2 (Slowly Changing Dimension Type 2) history of a specific item across all dataset versions.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+const history = await dataset.getItemHistory({ itemId: 'item-id' })
+
+for (const row of history) {
+  console.log(`Version ${row.datasetVersion}: ${row.isDeleted ? 'deleted' : 'active'}`)
+}
+```
+
+## Parameters
+
+**itemId:** (`string`): ID of the item to retrieve history for.
+
+## Returns
+
+**result:** (`Promise<DatasetItemRow[]>`): DatasetItemRowid:stringUnique row identifier.datasetId:stringID of the parent dataset.datasetVersion:numberDataset version for this row\.validTo:number | nullVersion at which this row was superseded, or \`null\` if still current.isDeleted:booleanWhether this row represents a deletion.input:unknownInput data at this version.groundTruth?:unknownGround truth at this version.metadata?:Record\<string, unknown>Metadata at this version.createdAt:DateWhen this row was created.updatedAt:DateWhen this row was last updated.

package/dist/docs/references/reference-datasets-list.md

@@ -0,0 +1,31 @@
+# DatasetsManager.list()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists all datasets with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const { datasets, pagination } = await mastra.datasets.list({ page: 0, perPage: 10 })
+
+for (const ds of datasets) {
+  console.log(`${ds.id}: ${ds.name} (v${ds.version})`)
+}
+```
+
+## Parameters
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of datasets per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectdatasets:DatasetRecord\[]Array of dataset records. See dataset.getDetails() for the record shape.pagination:PaginationInfoPagination metadata with \`total\`, \`page\`, \`perPage\`, and \`hasMore\`.

package/dist/docs/references/reference-datasets-listExperimentResults.md

@@ -0,0 +1,39 @@
+# dataset.listExperimentResults()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists individual item results for a specific experiment with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+const { results, pagination } = await dataset.listExperimentResults({
+  experimentId: 'exp-id',
+  page: 0,
+  perPage: 50,
+})
+
+for (const result of results) {
+  console.log(`Item ${result.itemId}: ${result.error ? 'FAILED' : 'OK'}`)
+}
+```
+
+## Parameters
+
+**experimentId:** (`string`): ID of the experiment to list results for.
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of results per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectresults:ExperimentResult\[]Array of item-level results.ExperimentResultid:stringUnique result ID.experimentId:stringID of the parent experiment.itemId:stringID of the dataset item.itemDatasetVersion:number | nullDataset version of the item when executed.input:unknownInput data passed to the target.output:unknown | nullOutput from the target.groundTruth:unknown | nullExpected output.error:{ message: string; stack?: string; code?: string } | nullStructured error if execution failed.startedAt:DateWhen execution started.completedAt:DateWhen execution completed.retryCount:numberNumber of retry attempts.traceId:string | nullTrace ID for observability.createdAt:DateWhen the result record was created.pagination:PaginationInfoPagination metadata with \`total\`, \`page\`, \`perPage\`, and \`hasMore\`.

package/dist/docs/references/reference-datasets-listExperiments.md

@@ -0,0 +1,33 @@
+# dataset.listExperiments()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists all experiments (runs) for this dataset with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+const { experiments, pagination } = await dataset.listExperiments({ page: 0, perPage: 10 })
+
+for (const exp of experiments) {
+  console.log(`${exp.id}: ${exp.status} (${exp.succeededCount}/${exp.totalItems})`)
+}
+```
+
+## Parameters
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of experiments per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectexperiments:Experiment\[]Array of experiment records.Experimentid:stringUnique experiment ID.name?:stringDisplay name.description?:stringDescription.metadata?:Record\<string, unknown>Arbitrary metadata.datasetId:stringID of the parent dataset.datasetVersion:number | nullDataset version used for the experiment.targetType:'agent' | 'workflow' | 'scorer' | 'processor'Type of target used.targetId:stringID of the target used.status:'pending' | 'running' | 'completed' | 'failed'Current status of the experiment.totalItems:numberTotal number of items.succeededCount:numberNumber of successful items.failedCount:numberNumber of failed items.skippedCount:numberNumber of skipped items.startedAt:Date | nullWhen the experiment started.completedAt:Date | nullWhen the experiment completed.createdAt:DateWhen the experiment record was created.updatedAt:DateWhen the experiment record was last updated.pagination:PaginationInfoPagination metadata with \`total\`, \`page\`, \`perPage\`, and \`hasMore\`.

package/dist/docs/references/reference-datasets-listItems.md

@@ -0,0 +1,46 @@
+# dataset.listItems()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists items in the dataset. When a `version` is specified, returns all items at that version. Otherwise, returns paginated items from the latest version.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+// Paginated list (default: page 0, 20 per page)
+const result = await dataset.listItems()
+
+// List with search
+const filtered = await dataset.listItems({ search: 'TypeScript', page: 0, perPage: 10 })
+
+// List all items at a specific version
+const versionedItems = await dataset.listItems({ version: 2 })
+```
+
+## Parameters
+
+**version?:** (`number`): Dataset version to list items at. When set, returns all items at that version (no pagination).
+
+**page?:** (`number`): Page number for pagination. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of items per page. Defaults to \`20\`.
+
+**search?:** (`string`): Search string to filter items.
+
+## Returns
+
+When `version` is specified:
+
+**result:** (`Promise<DatasetItem[]>`): Array of all items at the specified dataset version.
+
+When `version` is not specified:
+
+**result:** (`Promise<object>`): objectitems:DatasetItem\[]Array of items for the current page.pagination:objectPagination metadata.objecttotal:numberTotal number of items.page:numberCurrent page number.perPage:number | falseItems per page, or \`false\` if unpaginated.hasMore:booleanWhether more pages are available.

package/dist/docs/references/reference-datasets-listVersions.md

@@ -0,0 +1,33 @@
+# dataset.listVersions()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Lists all versions of the dataset with pagination.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+const { versions, pagination } = await dataset.listVersions({ page: 0, perPage: 10 })
+
+for (const version of versions) {
+  console.log(`Version ${version.version} created at ${version.createdAt}`)
+}
+```
+
+## Parameters
+
+**page?:** (`number`): Page number. Defaults to \`0\`.
+
+**perPage?:** (`number`): Number of versions per page. Defaults to \`20\`.
+
+## Returns
+
+**result:** (`Promise<object>`): objectversions:DatasetVersion\[]Array of version records.DatasetVersionid:stringUnique identifier of the version record.datasetId:stringID of the parent dataset.version:numberVersion number.createdAt:DateWhen this version was created.pagination:objectPagination metadata.objecttotal:numberTotal number of versions.page:numberCurrent page number.perPage:number | falseVersions per page, or \`false\` if unpaginated.hasMore:booleanWhether more pages are available.

package/dist/docs/references/reference-datasets-startExperiment.md

@@ -0,0 +1,62 @@
+# dataset.startExperiment()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Runs an experiment on the dataset and waits for completion. Executes all items against a target (agent, workflow, or scorer) with optional scoring.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+// Run against a registered agent with scorers
+const summary = await dataset.startExperiment({
+  targetType: 'agent',
+  targetId: 'my-agent',
+  scorers: ['accuracy', 'relevancy'],
+  maxConcurrency: 10,
+})
+
+console.log(`${summary.succeededCount}/${summary.totalItems} succeeded`)
+console.log(`Status: ${summary.status}`)
+```
+
+## Parameters
+
+**targetType?:** (`'agent' | 'workflow' | 'scorer'`): Type of registered target to run items against. Use with \`targetId\`.
+
+**targetId?:** (`string`): ID of the registered target. Use with \`targetType\`.
+
+**scorers?:** (`(MastraScorer | string)[]`): Scorers to evaluate each result. Pass \`MastraScorer\` instances or registered scorer IDs.
+
+**name?:** (`string`): Display name for the experiment.
+
+**description?:** (`string`): Description of the experiment.
+
+**metadata?:** (`Record<string, unknown>`): Arbitrary metadata for the experiment.
+
+**version?:** (`number`): Pin to a specific dataset version. Defaults to the latest version.
+
+**maxConcurrency?:** (`number`): Maximum concurrent item executions. Defaults to \`5\`.
+
+**signal?:** (`AbortSignal`): AbortSignal for cancelling the experiment.
+
+**itemTimeout?:** (`number`): Per-item execution timeout in milliseconds.
+
+**maxRetries?:** (`number`): Maximum retries per item on failure. Defaults to \`0\` (no retries). Abort errors are never retried.
+
+## Returns
+
+**result:** (`Promise<ExperimentSummary>`): ExperimentSummaryexperimentId:stringUnique ID of the experiment.status:'pending' | 'running' | 'completed' | 'failed'Final status of the experiment.totalItems:numberTotal number of items in the dataset.succeededCount:numberNumber of items that succeeded.failedCount:numberNumber of items that failed.skippedCount:numberNumber of items skipped (e.g., due to abort).completedWithErrors:boolean\`true\` if the run completed but some items failed.startedAt:DateWhen the experiment started.completedAt:DateWhen the experiment completed.results:ItemWithScores\[]All item results with their scores.ItemWithScoresitemId:stringID of the dataset item.itemVersion:numberDataset version of the item when executed.input:unknownInput data passed to the target.output:unknown | nullOutput from the target, or \`null\` if failed.groundTruth:unknown | nullExpected output from the dataset item.error:{ message: string; stack?: string; code?: string } | nullStructured error if execution failed.startedAt:DateWhen item execution started.completedAt:DateWhen item execution completed.retryCount:numberNumber of retry attempts.scores:ScorerResult\[]Results from all scorers for this item.ScorerResultscorerId:stringID of the scorer.scorerName:stringDisplay name of the scorer.score:number | nullComputed score, or \`null\` if the scorer failed.reason:string | nullReason/explanation for the score.error:string | nullError message if the scorer failed.
+
+## Related
+
+- [dataset.startExperimentAsync()](https://mastra.ai/reference/datasets/startExperimentAsync)
+- [dataset.listExperiments()](https://mastra.ai/reference/datasets/listExperiments)
+- [DatasetsManager.compareExperiments()](https://mastra.ai/reference/datasets/compareExperiments)

package/dist/docs/references/reference-datasets-startExperimentAsync.md

@@ -0,0 +1,43 @@
+# dataset.startExperimentAsync()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Starts an experiment asynchronously (fire-and-forget). Returns immediately with the experiment ID and a `'pending'` status. The experiment runs in the background.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+// Start experiment without waiting
+const { experimentId, status } = await dataset.startExperimentAsync({
+  targetType: 'agent',
+  targetId: 'my-agent',
+  scorers: ['accuracy'],
+})
+
+console.log(`Experiment ${experimentId} started with status: ${status}`)
+
+// Check progress later
+const experiment = await dataset.getExperiment({ experimentId })
+console.log(`Current status: ${experiment.status}`)
+```
+
+## Parameters
+
+Takes the same `StartExperimentConfig` as [`dataset.startExperiment()`](https://mastra.ai/reference/datasets/startExperiment).
+
+## Returns
+
+**result:** (`Promise<object>`): objectexperimentId:stringUnique ID of the created experiment.status:'pending'Always \`'pending'\` since the experiment hasn't started executing yet.
+
+## Related
+
+- [dataset.startExperiment()](https://mastra.ai/reference/datasets/startExperiment)
+- [dataset.getExperiment()](https://mastra.ai/reference/datasets/getExperiment)

package/dist/docs/references/reference-datasets-update.md

@@ -0,0 +1,48 @@
+# dataset.update()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Updates dataset metadata, name, description, and/or schemas. Zod schemas are automatically converted to JSON Schema.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+import { z } from 'zod'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+// Update with plain metadata
+const updated = await dataset.update({
+  name: 'Updated QA pairs',
+  description: 'Revised evaluation set',
+})
+
+// Update with Zod schema (auto-converted to JSON Schema)
+const updated2 = await dataset.update({
+  inputSchema: z.object({
+    question: z.string(),
+    context: z.string().optional(),
+  }),
+})
+```
+
+## Parameters
+
+**name?:** (`string`): New display name.
+
+**description?:** (`string`): New description.
+
+**metadata?:** (`Record<string, unknown>`): Updated metadata.
+
+**inputSchema?:** (`unknown`): JSON Schema or Zod schema for item inputs.
+
+**groundTruthSchema?:** (`unknown`): JSON Schema or Zod schema for item ground truths.
+
+## Returns
+
+**result:** (`Promise<DatasetRecord>`): The updated dataset record. See dataset.getDetails() for the full shape.

package/dist/docs/references/reference-datasets-updateItem.md

@@ -0,0 +1,38 @@
+# dataset.updateItem()
+
+**Added in:** `@mastra/core@1.4.0`
+
+Updates an existing item in the dataset. Only the provided fields are updated. Updating an item creates a new version.
+
+## Usage example
+
+```typescript
+import { Mastra } from '@mastra/core'
+
+const mastra = new Mastra({
+  /* storage config */
+})
+
+const dataset = await mastra.datasets.get({ id: 'dataset-id' })
+
+const updated = await dataset.updateItem({
+  itemId: 'item-id',
+  input: { question: 'What is TypeScript?' },
+  groundTruth: { answer: 'A typed superset of JavaScript' },
+  metadata: { reviewed: true },
+})
+```
+
+## Parameters
+
+**itemId:** (`string`): ID of the item to update.
+
+**input?:** (`unknown`): Updated input data.
+
+**groundTruth?:** (`unknown`): Updated ground truth.
+
+**metadata?:** (`Record<string, unknown>`): Updated metadata.
+
+## Returns
+
+**result:** (`Promise<DatasetItem>`): The updated dataset item. See dataset.addItem() for the item shape.

package/dist/docs/references/reference-evals-answer-relevancy.md

@@ -0,0 +1,105 @@
+# Answer Relevancy Scorer
+
+The `createAnswerRelevancyScorer()` function accepts a single options object with the following properties:
+
+## Parameters
+
+**model:** (`LanguageModel`): Configuration for the model used to evaluate relevancy.
+
+**uncertaintyWeight:** (`number`): Weight given to 'unsure' verdicts in scoring (0-1). (Default: `0.3`)
+
+**scale:** (`number`): Maximum score value. (Default: `1`)
+
+This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
+
+## .run() Returns
+
+**runId:** (`string`): The id of the run (optional).
+
+**score:** (`number`): Relevancy score (0 to scale, default 0-1)
+
+**preprocessPrompt:** (`string`): The prompt sent to the LLM for the preprocess step (optional).
+
+**preprocessStepResult:** (`object`): Object with extracted statements: { statements: string\[] }
+
+**analyzePrompt:** (`string`): The prompt sent to the LLM for the analyze step (optional).
+
+**analyzeStepResult:** (`object`): Object with results: { results: Array<{ result: 'yes' | 'unsure' | 'no', reason: string }> }
+
+**generateReasonPrompt:** (`string`): The prompt sent to the LLM for the reason step (optional).
+
+**reason:** (`string`): Explanation of the score.
+
+## Scoring Details
+
+The scorer evaluates relevancy through query-answer alignment, considering completeness and detail level, but not factual correctness.
+
+### Scoring Process
+
+1. **Statement Preprocess:**
+   - Breaks output into meaningful statements while preserving context.
+
+2. **Relevance Analysis:**
+
+   - Each statement is evaluated as:
+
+     - "yes": Full weight for direct matches
+     - "unsure": Partial weight (default: 0.3) for approximate matches
+     - "no": Zero weight for irrelevant content
+
+3. **Score Calculation:**
+   - `((direct + uncertainty * partial) / total_statements) * scale`
+
+### Score Interpretation
+
+A relevancy score between 0 and 1:
+
+- **1.0**: The response fully answers the query with relevant and focused information.
+- **0.7–0.9**: The response mostly answers the query but may include minor unrelated content.
+- **0.4–0.6**: The response partially answers the query, mixing relevant and unrelated information.
+- **0.1–0.3**: The response includes minimal relevant content and largely misses the intent of the query.
+- **0.0**: The response is entirely unrelated and does not answer the query.
+
+## Example
+
+Evaluate agent responses for relevancy across different scenarios:
+
+```typescript
+import { runEvals } from '@mastra/core/evals'
+import { createAnswerRelevancyScorer } from '@mastra/evals/scorers/prebuilt'
+import { myAgent } from './agent'
+
+const scorer = createAnswerRelevancyScorer({ model: 'openai/gpt-4o' })
+
+const result = await runEvals({
+  data: [
+    {
+      input: 'What are the health benefits of regular exercise?',
+    },
+    {
+      input: 'What should a healthy breakfast include?',
+    },
+    {
+      input: 'What are the benefits of meditation?',
+    },
+  ],
+  scorers: [scorer],
+  target: myAgent,
+  onItemComplete: ({ scorerResults }) => {
+    console.log({
+      score: scorerResults[scorer.id].score,
+      reason: scorerResults[scorer.id].reason,
+    })
+  },
+})
+
+console.log(result.scores)
+```
+
+For more details on `runEvals`, see the [runEvals reference](https://mastra.ai/reference/evals/run-evals).
+
+To add this scorer to an agent, see the [Scorers overview](https://mastra.ai/docs/evals/overview) guide.
+
+## Related
+
+- [Faithfulness Scorer](https://mastra.ai/reference/evals/faithfulness)

package/dist/docs/references/reference-evals-answer-similarity.md

@@ -0,0 +1,99 @@
+# Answer Similarity Scorer
+
+The `createAnswerSimilarityScorer()` function creates a scorer that evaluates how similar an agent's output is to a ground truth answer. This scorer is specifically designed for CI/CD testing scenarios where you have expected answers and want to ensure consistency over time.
+
+## Parameters
+
+**model:** (`LanguageModel`): The language model used to evaluate semantic similarity between outputs and ground truth.
+
+**options:** (`AnswerSimilarityOptions`): Configuration options for the scorer.
+
+### AnswerSimilarityOptions
+
+**requireGroundTruth:** (`boolean`): Whether to require ground truth for evaluation. If false, missing ground truth returns score 0. (Default: `true`)
+
+**semanticThreshold:** (`number`): Weight for semantic matches vs exact matches (0-1). (Default: `0.8`)
+
+**exactMatchBonus:** (`number`): Additional score bonus for exact matches (0-1). (Default: `0.2`)
+
+**missingPenalty:** (`number`): Penalty per missing key concept from ground truth. (Default: `0.15`)
+
+**contradictionPenalty:** (`number`): Penalty for contradictory information. High value ensures wrong answers score near 0. (Default: `1.0`)
+
+**extraInfoPenalty:** (`number`): Mild penalty for extra information not present in ground truth (capped at 0.2). (Default: `0.05`)
+
+**scale:** (`number`): Score scaling factor. (Default: `1`)
+
+This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but **requires ground truth** to be provided in the run object.
+
+## .run() Returns
+
+**runId:** (`string`): The id of the run (optional).
+
+**score:** (`number`): Similarity score between 0-1 (or 0-scale if custom scale used). Higher scores indicate better similarity to ground truth.
+
+**reason:** (`string`): Human-readable explanation of the score with actionable feedback.
+
+**preprocessStepResult:** (`object`): Extracted semantic units from output and ground truth.
+
+**analyzeStepResult:** (`object`): Detailed analysis of matches, contradictions, and extra information.
+
+**preprocessPrompt:** (`string`): The prompt used for semantic unit extraction.
+
+**analyzePrompt:** (`string`): The prompt used for similarity analysis.
+
+**generateReasonPrompt:** (`string`): The prompt used for generating the explanation.
+
+## Scoring Details
+
+The scorer uses a multi-step process:
+
+1. **Extract**: Breaks down output and ground truth into semantic units
+2. **Analyze**: Compares units and identifies matches, contradictions, and gaps
+3. **Score**: Calculates weighted similarity with penalties for contradictions
+4. **Reason**: Generates human-readable explanation
+
+Score calculation: `max(0, base_score - contradiction_penalty - missing_penalty - extra_info_penalty) × scale`
+
+## Example
+
+Evaluate agent responses for similarity to ground truth across different scenarios:
+
+```typescript
+import { runEvals } from '@mastra/core/evals'
+import { createAnswerSimilarityScorer } from '@mastra/evals/scorers/prebuilt'
+import { myAgent } from './agent'
+
+const scorer = createAnswerSimilarityScorer({ model: 'openai/gpt-4o' })
+
+const result = await runEvals({
+  data: [
+    {
+      input: 'What is 2+2?',
+      groundTruth: '4',
+    },
+    {
+      input: 'What is the capital of France?',
+      groundTruth: 'The capital of France is Paris',
+    },
+    {
+      input: 'What are the primary colors?',
+      groundTruth: 'The primary colors are red, blue, and yellow',
+    },
+  ],
+  scorers: [scorer],
+  target: myAgent,
+  onItemComplete: ({ scorerResults }) => {
+    console.log({
+      score: scorerResults[scorer.id].score,
+      reason: scorerResults[scorer.id].reason,
+    })
+  },
+})
+
+console.log(result.scores)
+```
+
+For more details on `runEvals`, see the [runEvals reference](https://mastra.ai/reference/evals/run-evals).
+
+To add this scorer to an agent, see the [Scorers overview](https://mastra.ai/docs/evals/overview) guide.