@mastra/evals 1.10.5 → 1.10.6-alpha.0

package/dist/docs/SKILL.md

@@ -3,7 +3,7 @@ name: mastra-evals
 description: Documentation for @mastra/evals. Use when working with @mastra/evals APIs, configuration, or implementation.
 metadata:
   package: "@mastra/evals"
-  version: "1.10.5"
+  version: "1.10.6-alpha.0"
 ---
 
 ## When to use

package/dist/docs/assets/SOURCE_MAP.json

@@ -1,5 +1,5 @@
 {
-  "version": "1.10.5",
+  "version": "1.10.6-alpha.0",
   "package": "@mastra/evals",
   "exports": {},
   "modules": {}

package/dist/docs/references/docs-evals-built-in-scorers.md

@@ -24,7 +24,7 @@ These scorers evaluate how correct, truthful, and complete your agent's answers
 - [`tool-call-accuracy`](https://mastra.ai/reference/evals/tool-call-accuracy): Evaluates whether the LLM selects the correct tool from available options (`0-1`, higher is better)
 - [`trajectory-accuracy`](https://mastra.ai/reference/evals/trajectory-accuracy): Evaluates the expected action sequence for all span types. Covered spans include tool and model activity plus workflow steps (`0-1`, higher is better)
 - [`prompt-alignment`](https://mastra.ai/reference/evals/prompt-alignment): Measures how well agent responses align with user prompt intent, requirements, completeness, and format (`0-1`, higher is better)
-- [`multi-turn-judge`](https://mastra.ai/reference/evals/multi-turn-judge): Grades every assistant turn of a [multi-turn conversation](https://mastra.ai/docs/evals/multi-turn) against a plain-English criterion (`0` or `1`)
+- [`multi-turn-judge`](https://mastra.ai/reference/evals/multi-turn-judge): Grades every assistant turn of a [multi-turn conversation](https://mastra.ai/docs/evals/multi-turn) against a plain-English criterion (`0` or `scale`, default `1`)
 
 ### Context quality
 
@@ -58,7 +58,7 @@ Zero-LLM micro-scorers for fast, deterministic assertions. See the [Quick Checks
 
 - [`checks.includes`](https://mastra.ai/reference/evals/checks): Scores 1 if output contains a substring (`0` or `1`)
 - [`checks.excludes`](https://mastra.ai/reference/evals/checks): Scores 1 if output doesn't contain a substring (`0` or `1`)
-- [`checks.equals`](https://mastra.ai/reference/evals/checks): Scores 1 if output exactly matches a string (`0` or `1`)
+- [`checks.equals`](https://mastra.ai/reference/evals/checks): Scores 1 if output matches a string, ignoring case by default (`0` or `1`)
 - [`checks.matches`](https://mastra.ai/reference/evals/checks): Scores 1 if output matches a regex (`0` or `1`)
 - [`checks.similarity`](https://mastra.ai/reference/evals/checks): String similarity via Dice coefficient (`0-1`, or binary with threshold)
 - [`checks.calledTool`](https://mastra.ai/reference/evals/checks): Scores 1 if a tool was called at least N times (`0` or `1`)

package/dist/docs/references/reference-evals-answer-relevancy.md

@@ -10,9 +10,11 @@ The `createAnswerRelevancyScorer()` function accepts a single options object wit
 
 **model** (`LanguageModel`): Configuration for the model used to evaluate relevancy.
 
-**uncertaintyWeight** (`number`): Weight given to 'unsure' verdicts in scoring (0-1). (Default: `0.3`)
+**options** (`Record<'uncertaintyWeight' | 'scale', number>`): Configuration options for the scorer. (Default: `{ uncertaintyWeight: 0.3, scale: 1 }`)
 
-**scale** (`number`): Maximum score value. (Default: `1`)
+**options.uncertaintyWeight** (`number`): Weight given to 'unsure' verdicts in scoring (0-1).
+
+**options.scale** (`number`): Maximum score value.
 
 This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
 

package/dist/docs/references/reference-evals-answer-similarity.md

@@ -26,7 +26,7 @@ The `createAnswerSimilarityScorer()` function creates a scorer that evaluates ho
 
 **options.scale** (`number`): Score scaling factor.
 
-This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but **requires ground truth** to be provided in the run object.
+This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but expects ground truth to be provided in the run object. When `options.requireGroundTruth` is `true` (the default), a run without ground truth throws an error. Set it to `false` to score runs without ground truth as 0.
 
 ## `.run()` returns
 

package/dist/docs/references/reference-evals-bias.md

@@ -10,7 +10,9 @@ The `createBiasScorer()` function accepts a single options object with the follo
 
 **model** (`LanguageModel`): Configuration for the model used to evaluate bias.
 
-**scale** (`number`): Maximum score value. (Default: `1`)
+**options** (`BiasMetricOptions`): Configuration options for the scorer.
+
+**options.scale** (`number`): Maximum score value.
 
 This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
 

package/dist/docs/references/reference-evals-content-similarity.md

@@ -44,7 +44,7 @@ The scorer evaluates textual similarity through character-level matching and con
    - Considers relative positions
    - Accounts for length differences
 
-Final score: `similarity_value * scale`
+Final score: `similarity_value` (0-1, returned directly from the string-similarity comparison)
 
 ## Example
 

package/dist/docs/references/reference-evals-context-recall.md

@@ -30,7 +30,7 @@ Use when running evaluations against curated test sets:
 
 **options** (`ContextRecallMetricOptions`): Configuration options for the scorer
 
-Either `context` or `contextExtractor` must be provided. When both are provided, `contextExtractor` is used only if the run input and output are agent-shaped (`MastraDBMessage[]`); otherwise the scorer falls back to `context`.
+Either `context` or `contextExtractor` must be provided. When both are provided, `contextExtractor` is used only if the run input and output are agent-shaped: the input must be an agent run-input object (`{ inputMessages, rememberedMessages, systemMessages, taggedSystemMessages }`) and the output must be a `MastraDBMessage[]`. Otherwise the scorer falls back to `context`.
 
 ## `.run()` returns
 

package/dist/docs/references/reference-evals-context-relevance.md

@@ -119,7 +119,7 @@ Use results to improve your system:
 Control how penalties are applied for unused and missing context:
 
 ```typescript
-import { createContextRelevanceScorerLLM } from '@mastra/evals'
+import { createContextRelevanceScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 // Stricter penalty configuration
 const strictScorer = createContextRelevanceScorerLLM({
@@ -249,7 +249,7 @@ const scorer = createContextRelevanceScorerLLM({
 This example shows excellent context relevance where all context directly supports the response:
 
 ```typescript
-import { createContextRelevanceScorerLLM } from '@mastra/evals'
+import { createContextRelevanceScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createContextRelevanceScorerLLM({
   model: 'openai/gpt-5.6-sol',
@@ -296,7 +296,7 @@ console.log(result)
 This example shows moderate relevance with some context being irrelevant or unused:
 
 ```typescript
-import { createContextRelevanceScorerLLM } from '@mastra/evals'
+import { createContextRelevanceScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createContextRelevanceScorerLLM({
   model: 'openai/gpt-5.6-sol',
@@ -385,7 +385,7 @@ console.log(customResult)
 This example shows poor context relevance with mostly irrelevant information:
 
 ```typescript
-import { createContextRelevanceScorerLLM } from '@mastra/evals'
+import { createContextRelevanceScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createContextRelevanceScorerLLM({
   model: 'openai/gpt-5.6-sol',
@@ -433,7 +433,7 @@ console.log(result)
 Extract context at runtime based on the run input:
 
 ```typescript
-import { createContextRelevanceScorerLLM } from '@mastra/evals'
+import { createContextRelevanceScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createContextRelevanceScorerLLM({
   model: 'openai/gpt-5.6-sol',
@@ -476,7 +476,7 @@ const scorer = createContextRelevanceScorerLLM({
 Integrate with RAG pipelines to evaluate retrieved context:
 
 ```typescript
-import { createContextRelevanceScorerLLM } from '@mastra/evals'
+import { createContextRelevanceScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createContextRelevanceScorerLLM({
   model: 'openai/gpt-5.6-sol',

package/dist/docs/references/reference-evals-hallucination.md

@@ -46,15 +46,14 @@ The scorer evaluates hallucination through contradiction detection and unsupport
 
 ### Scoring Process
 
-1. Analyzes factual content:
+1. Extracts claims from the output:
 
-   - Extracts statements from context
-   - Identifies numerical values and dates
-   - Maps statement relationships
+   - Breaks the agent's response into factual claims
+   - Includes numerical values and dates
 
-2. Analyzes output for hallucinations:
+2. Checks each claim against the provided context:
 
-   - Compares against context statements
+   - Compares the claim with the context
    - Marks direct conflicts as hallucinations
    - Identifies unsupported claims as hallucinations
    - Evaluates numerical accuracy
@@ -125,7 +124,7 @@ Use `getContext` for live scoring scenarios where context comes from tool result
 
 ```typescript
 import { createHallucinationScorer } from '@mastra/evals/scorers/prebuilt'
-import { extractToolResults } from '@mastra/evals/scorers'
+import { extractToolResults } from '@mastra/evals/scorers/utils'
 
 const scorer = createHallucinationScorer({
   model: 'openai/gpt-5.6-sol',
@@ -146,7 +145,7 @@ Attach the scorer to an agent for live evaluation:
 ```typescript
 import { Agent } from '@mastra/core/agent'
 import { createHallucinationScorer } from '@mastra/evals/scorers/prebuilt'
-import { extractToolResults } from '@mastra/evals/scorers'
+import { extractToolResults } from '@mastra/evals/scorers/utils'
 
 const hallucinationScorer = createHallucinationScorer({
   model: 'openai/gpt-5.6-sol',

package/dist/docs/references/reference-evals-keyword-coverage.md

@@ -18,7 +18,7 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 
 **preprocessStepResult** (`object`): Object with extracted keywords: { referenceKeywords: Set\<string>, responseKeywords: Set\<string> }
 
-**analyzeStepResult** (`object`): Object with keyword coverage: { totalKeywords: number, matchedKeywords: number }
+**analyzeStepResult** (`object`): Object with keyword coverage: { totalKeywordsLength: number, matchedKeywordsLength: number }
 
 **score** (`number`): Coverage score (0-1) representing the proportion of matched keywords.
 
@@ -27,13 +27,13 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 ```typescript
 {
   runId: string,
-  extractStepResult: {
+  preprocessStepResult: {
     referenceKeywords: Set<string>,
     responseKeywords: Set<string>
   },
   analyzeStepResult: {
-    totalKeywords: number,
-    matchedKeywords: number
+    totalKeywordsLength: number,
+    matchedKeywordsLength: number
   },
   score: number
 }

package/dist/docs/references/reference-evals-noise-sensitivity.md

@@ -292,7 +292,7 @@ describe('Agent Noise Resistance CI Tests', () => {
 This example shows an agent that completely resists misinformation in a test scenario:
 
 ```typescript
-import { createNoiseSensitivityScorerLLM } from '@mastra/evals'
+import { createNoiseSensitivityScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createNoiseSensitivityScorerLLM({
   model: 'openai/gpt-5.6-sol',
@@ -383,7 +383,7 @@ console.log(result)
 This example shows an agent that incorporates misinformation:
 
 ```typescript
-import { createNoiseSensitivityScorerLLM } from '@mastra/evals'
+import { createNoiseSensitivityScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createNoiseSensitivityScorerLLM({
   model: 'openai/gpt-5.6-sol',
@@ -428,7 +428,7 @@ console.log(result)
 Adjust scoring sensitivity for your specific use case:
 
 ```typescript
-import { createNoiseSensitivityScorerLLM } from '@mastra/evals'
+import { createNoiseSensitivityScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 // Lenient scoring - more forgiving of minor issues
 const lenientScorer = createNoiseSensitivityScorerLLM({
@@ -477,7 +477,7 @@ const strictScorer = createNoiseSensitivityScorerLLM({
 Create detailed test suites to evaluate agent performance across noise categories in your CI pipeline:
 
 ```typescript
-import { createNoiseSensitivityScorerLLM } from '@mastra/evals'
+import { createNoiseSensitivityScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const noiseTestCases = [
   {
@@ -546,7 +546,7 @@ async function evaluateNoiseResistance(testCases) {
 Use in your CI pipeline to compare noise resistance across different models before deployment:
 
 ```typescript
-import { createNoiseSensitivityScorerLLM } from '@mastra/evals'
+import { createNoiseSensitivityScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 async function compareModelRobustness() {
   const models = [
@@ -599,7 +599,7 @@ function getGrade(score) {
 Include noise sensitivity tests in your security test suite to validate prompt injection resistance:
 
 ```typescript
-import { createNoiseSensitivityScorerLLM } from '@mastra/evals'
+import { createNoiseSensitivityScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createNoiseSensitivityScorerLLM({
   model: 'openai/gpt-5.6-sol',

package/dist/docs/references/reference-evals-prompt-alignment.md

@@ -308,7 +308,7 @@ for (const agent of agents) {
 ### Basic Configuration
 
 ```typescript
-import { createPromptAlignmentScorerLLM } from '@mastra/evals'
+import { createPromptAlignmentScorerLLM } from '@mastra/evals/scorers/prebuilt'
 
 const scorer = createPromptAlignmentScorerLLM({
   model: 'openai/gpt-5.6-sol',

package/dist/docs/references/reference-evals-scorer-utils.md

@@ -37,7 +37,7 @@ import {
 
 ### `getAssistantMessageFromRunOutput`
 
-Extracts the text content from the first assistant message in the run output.
+Extracts the text content from the last assistant message that contains text in the run output. Trailing assistant messages without text (for example, tool-call-only messages) are skipped.
 
 ```typescript
 const scorer = createScorer({
@@ -298,7 +298,7 @@ const trajectory = extractTrajectory(agentOutputMessages)
 // trajectory.rawOutput — the original MastraDBMessage[] array
 ```
 
-**Returns:** `Trajectory`: Contains `steps: TrajectoryStep[]`, `totalDurationMs`, and `rawOutput`.
+**Returns:** `Trajectory`: Contains `steps: TrajectoryStep[]` and `rawOutput`.
 
 ### `extractWorkflowTrajectory`
 
@@ -352,12 +352,12 @@ const trajectory = extractTrajectoryFromTrace(traceData.spans, rootSpanId)
 | `AGENT_RUN`            | `agent_run`            | `agentId` (from entity ID)                                    |
 | `WORKFLOW_RUN`         | `workflow_run`         | `workflowId` (from entity ID)                                 |
 | `WORKFLOW_STEP`        | `workflow_step`        | `output`                                                      |
-| `WORKFLOW_CONDITIONAL` | `workflow_conditional` | `conditionCount`, `selectedSteps`                             |
-| `WORKFLOW_PARALLEL`    | `workflow_parallel`    | `branchCount`, `parallelSteps`                                |
-| `WORKFLOW_LOOP`        | `workflow_loop`        | `loopType`, `totalIterations`                                 |
-| `WORKFLOW_SLEEP`       | `workflow_sleep`       | `sleepDurationMs`, `sleepType`                                |
-| `WORKFLOW_WAIT_EVENT`  | `workflow_wait_event`  | `eventName`, `eventReceived`                                  |
-| `PROCESSOR_RUN`        | `processor_run`        | `processorId`                                                 |
+| `WORKFLOW_CONDITIONAL` | `workflow_conditional` | Base fields only (`name`, `durationMs`, `metadata`)           |
+| `WORKFLOW_PARALLEL`    | `workflow_parallel`    | Base fields only (`name`, `durationMs`, `metadata`)           |
+| `WORKFLOW_LOOP`        | `workflow_loop`        | Base fields only (`name`, `durationMs`, `metadata`)           |
+| `WORKFLOW_SLEEP`       | `workflow_sleep`       | Base fields only (`name`, `durationMs`, `metadata`)           |
+| `WORKFLOW_WAIT_EVENT`  | `workflow_wait_event`  | Base fields only (`name`, `durationMs`, `metadata`)           |
+| `PROCESSOR_RUN`        | `processor_run`        | Base fields only (`name`, `durationMs`, `metadata`)           |
 
 Spans with types `GENERIC`, `MODEL_STEP`, `MODEL_CHUNK`, and `WORKFLOW_CONDITIONAL_EVAL` are skipped as noise.
 

package/dist/docs/references/reference-evals-tone-consistency.md

@@ -4,11 +4,13 @@
 
 # Tone consistency scorer
 
-The `createToneScorer()` function evaluates the text's emotional tone and sentiment consistency. It can operate in two modes: comparing tone between input/output pairs or analyzing tone stability within a single text.
+The `createToneScorer()` function evaluates the text's emotional tone and sentiment consistency. It can operate in two modes: comparing the agent output's sentiment against a configured `referenceTone`, or analyzing tone stability across the sentences of the agent output.
 
 ## Parameters
 
-The `createToneScorer()` function doesn't take any options.
+The `createToneScorer()` function accepts an optional config object with the following properties:
+
+**referenceTone** (`string`): Reference text whose sentiment the agent output is compared against. When omitted, the scorer measures sentiment stability across the output's sentences instead.
 
 This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
 
@@ -16,7 +18,7 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 
 **runId** (`string`): The id of the run (optional).
 
-**analyzeStepResult** (`object`): Object with tone metrics: { responseSentiment: number, referenceSentiment: number, difference: number } (for comparison mode) OR { avgSentiment: number, sentimentVariance: number } (for stability mode)
+**preprocessStepResult** (`object`): Object with tone metrics: { score: number, responseSentiment: number, referenceSentiment: number, difference: number } (for comparison mode) OR { score: number, avgSentiment: number, sentimentVariance: number } (for stability mode)
 
 **score** (`number`): Tone consistency/stability score (0-1).
 
@@ -25,7 +27,8 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
 ```typescript
 {
   runId: string,
-  analyzeStepResult: {
+  preprocessStepResult: {
+    score: number,
     responseSentiment?: number,
     referenceSentiment?: number,
     difference?: number,
@@ -48,20 +51,20 @@ The scorer evaluates sentiment consistency through tone pattern analysis and mod
    - Computes sentiment scores
    - Measures tone variations
 
-2. Calculates mode-specific score: **Tone Consistency** (input and output):
+2. Calculates mode-specific score: **Tone Consistency** (`referenceTone` set):
 
-   - Compares sentiment between texts
-   - Calculates sentiment difference
-   - Score = 1 - (sentiment\_difference / max\_difference) **Tone Stability** (single input):
-   - Analyzes sentiment across sentences
+   - Compares the output's sentiment with the `referenceTone` sentiment
+   - Calculates the absolute sentiment difference
+   - Score = max(0, 1 - sentiment\_difference) **Tone Stability** (no `referenceTone`):
+   - Analyzes sentiment across the output's sentences
    - Calculates sentiment variance
-   - Score = 1 - (sentiment\_variance / max\_variance)
+   - Score = max(0, 1 - sentiment\_variance)
 
-Final score: `mode_specific_score * scale`
+Final score: `mode_specific_score`
 
 ### Score interpretation
 
-(0 to scale, default 0-1)
+(0-1)
 
 - 1.0: Perfect tone consistency/stability
 - 0.7-0.9: Strong consistency with minor variations
@@ -69,37 +72,33 @@ Final score: `mode_specific_score * scale`
 - 0.1-0.3: Poor consistency with major tone changes
 - 0.0: No consistency - completely different tones
 
-### `analyzeStepResult`
+### `preprocessStepResult`
 
 Object with tone metrics:
 
 - **responseSentiment**: Sentiment score for the response (comparison mode).
-- **referenceSentiment**: Sentiment score for the input/reference (comparison mode).
+- **referenceSentiment**: Sentiment score for the `referenceTone` (comparison mode).
 - **difference**: Absolute difference between sentiment scores (comparison mode).
 - **avgSentiment**: Average sentiment across sentences (stability mode).
 - **sentimentVariance**: Variance of sentiment across sentences (stability mode).
 
 ## Example
 
-Evaluate tone consistency between related agent responses:
+Evaluate whether agent responses match a positive reference tone:
 
 ```typescript
 import { runEvals } from '@mastra/core/evals'
 import { createToneScorer } from '@mastra/evals/scorers/prebuilt'
 import { myAgent } from './agent'
 
-const scorer = createToneScorer()
+const scorer = createToneScorer({
+  referenceTone: 'We are happy to help and glad you had a great experience!',
+})
 
 const result = await runEvals({
   data: [
-    {
-      input: 'How was your experience with our service?',
-      groundTruth: 'The service was excellent and exceeded expectations!',
-    },
-    {
-      input: 'Tell me about the customer support',
-      groundTruth: 'The support team was friendly and very helpful.',
-    },
+    { input: 'How was your experience with our service?' },
+    { input: 'Tell me about the customer support' },
   ],
   scorers: [scorer],
   target: myAgent,

package/dist/docs/references/reference-evals-tool-call-accuracy.md

@@ -94,7 +94,7 @@ const strictOrderScorer = createCodeScorer({
 {
   runId: string,
   preprocessStepResult: {
-    expectedTool: string,
+    expectedTool?: string, // undefined when only expectedToolOrder is configured
     actualTools: string[],
     strictMode: boolean,
     expectedToolOrder?: string[],
@@ -283,7 +283,7 @@ The `createToolCallAccuracyScorerLLM()` function from `@mastra/evals/scorers/pre
 
 **model** (`MastraModelConfig`): The LLM model to use for evaluating tool appropriateness
 
-**availableTools** (`Array<{name: string, description: string}>`): List of available tools with their descriptions for context
+**availableTools** (`Tool[]`): Tools available to the agent, for example created with createTool(). Each tool's id and description are passed to the judge as context.
 
 ### Features
 
@@ -315,8 +315,8 @@ The LLM-based scorer provides:
 const basicLLMScorer = createLLMScorer({
   model: 'openai/gpt-5.6-sol',
   availableTools: [
-    { name: 'tool1', description: 'Description 1' },
-    { name: 'tool2', description: 'Description 2' }
+    createTool({ id: 'tool1', description: 'Description 1' }),
+    createTool({ id: 'tool2', description: 'Description 2' })
   ]
 });
 
@@ -352,21 +352,24 @@ The LLM-based scorer uses AI to evaluate whether tool selections are appropriate
 ### Basic LLM evaluation
 
 ```typescript
+import { createTool } from '@mastra/core/tools'
+import { createToolCallAccuracyScorerLLM } from '@mastra/evals/scorers/prebuilt'
+
 const llmScorer = createToolCallAccuracyScorerLLM({
   model: 'openai/gpt-5.6-sol',
   availableTools: [
-    {
-      name: 'weather-tool',
+    createTool({
+      id: 'weather-tool',
       description: 'Get current weather information for any location',
-    },
-    {
-      name: 'calendar-tool',
+    }),
+    createTool({
+      id: 'calendar-tool',
       description: 'Check calendar events and scheduling',
-    },
-    {
-      name: 'search-tool',
+    }),
+    createTool({
+      id: 'search-tool',
       description: 'Search the web for general information',
-    },
+    }),
   ],
 })
 
@@ -474,6 +477,7 @@ console.log(result.reason); // "The agent appropriately asked for clarification
 Here's an example using both scorers on the same data:
 
 ```typescript
+import { createTool } from '@mastra/core/tools'
 import {
   createToolCallAccuracyScorerCode as createCodeScorer,
   createToolCallAccuracyScorerLLM as createLLMScorer,
@@ -488,8 +492,8 @@ const codeScorer = createCodeScorer({
 const llmScorer = createLLMScorer({
   model: 'openai/gpt-5.6-sol',
   availableTools: [
-    { name: 'weather-tool', description: 'Get weather information' },
-    { name: 'search-tool', description: 'Search the web' },
+    createTool({ id: 'weather-tool', description: 'Get weather information' }),
+    createTool({ id: 'search-tool', description: 'Search the web' }),
   ],
 })
 

package/dist/docs/references/reference-evals-toxicity.md

@@ -12,7 +12,9 @@ The `createToxicityScorer()` function accepts a single options object with the f
 
 **model** (`LanguageModel`): Configuration for the model used to evaluate toxicity.
 
-**scale** (`number`): Maximum score value (default is 1). (Default: `1`)
+**options** (`ToxicityMetricOptions`): Configuration options.
+
+**options.scale** (`number`): Maximum score value.
 
 This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
 

package/dist/scorers/code/trajectory/index.d.ts

@@ -23,7 +23,7 @@ interface TrajectoryAccuracyScorerCodeOptions {
  *
  * @example
  * ```ts
- * import { createTrajectoryAccuracyScorerCode } from '@mastra/evals/scorers';
+ * import { createTrajectoryAccuracyScorerCode } from '@mastra/evals/scorers/prebuilt';
  *
  * const scorer = createTrajectoryAccuracyScorerCode({
  *   expectedTrajectory: {
@@ -135,7 +135,7 @@ export interface TrajectoryScorerCodeOptions {
  *
  * @example
  * ```ts
- * import { createTrajectoryScorerCode } from '@mastra/evals/scorers';
+ * import { createTrajectoryScorerCode } from '@mastra/evals/scorers/prebuilt';
  *
  * const scorer = createTrajectoryScorerCode({
  *   defaults: {

package/dist/scorers/llm/trajectory/index.d.ts

@@ -20,7 +20,7 @@ export interface TrajectoryAccuracyLLMOptions {
  *
  * @example
  * ```ts
- * import { createTrajectoryAccuracyScorerLLM } from '@mastra/evals/scorers';
+ * import { createTrajectoryAccuracyScorerLLM } from '@mastra/evals/scorers/prebuilt';
  *
  * // Without expected trajectory (evaluates quality based on task)
  * const scorer = createTrajectoryAccuracyScorerLLM({

package/dist/scorers/prebuilt/index.cjs

@@ -3852,7 +3852,7 @@ function formatExpectedSteps(steps, indent = 0) {
 *
 * @example
 * ```ts
-* import { createTrajectoryAccuracyScorerLLM } from '@mastra/evals/scorers';
+* import { createTrajectoryAccuracyScorerLLM } from '@mastra/evals/scorers/prebuilt';
 *
 * // Without expected trajectory (evaluates quality based on task)
 * const scorer = createTrajectoryAccuracyScorerLLM({
@@ -4312,7 +4312,7 @@ function expectationToExpectedSteps(expectation) {
 *
 * @example
 * ```ts
-* import { createTrajectoryAccuracyScorerCode } from '@mastra/evals/scorers';
+* import { createTrajectoryAccuracyScorerCode } from '@mastra/evals/scorers/prebuilt';
 *
 * const scorer = createTrajectoryAccuracyScorerCode({
 *   expectedTrajectory: {
@@ -4521,7 +4521,7 @@ function evaluateNestedExpectations(expectedSteps, actualSteps, weights = {
 *
 * @example
 * ```ts
-* import { createTrajectoryScorerCode } from '@mastra/evals/scorers';
+* import { createTrajectoryScorerCode } from '@mastra/evals/scorers/prebuilt';
 *
 * const scorer = createTrajectoryScorerCode({
 *   defaults: {