@mastra/mcp-docs-server 1.1.7 → 1.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (276) hide show
  1. package/.docs/docs/agents/agent-approval.md +61 -31
  2. package/.docs/docs/agents/supervisor-agents.md +1 -1
  3. package/.docs/docs/getting-started/manual-install.md +1 -1
  4. package/.docs/docs/index.md +68 -24
  5. package/.docs/docs/mastra-cloud/setup.md +1 -1
  6. package/.docs/docs/memory/observational-memory.md +9 -0
  7. package/.docs/docs/memory/semantic-recall.md +17 -1
  8. package/.docs/docs/workspace/skills.md +7 -5
  9. package/.docs/guides/deployment/aws-lambda.md +76 -165
  10. package/.docs/guides/deployment/azure-app-services.md +38 -61
  11. package/.docs/guides/deployment/cloudflare.md +1 -1
  12. package/.docs/guides/deployment/netlify.md +1 -1
  13. package/.docs/guides/deployment/vercel.md +1 -1
  14. package/.docs/models/gateways/netlify.md +5 -1
  15. package/.docs/models/index.md +1 -1
  16. package/.docs/models/providers/anthropic.md +9 -9
  17. package/.docs/models/providers/deepseek.md +1 -1
  18. package/.docs/models/providers/google.md +11 -11
  19. package/.docs/models/providers/openai.md +19 -19
  20. package/.docs/models/providers/opencode.md +33 -33
  21. package/.docs/models/providers/xai.md +3 -3
  22. package/.docs/reference/agents/agent.md +20 -20
  23. package/.docs/reference/agents/generate.md +200 -66
  24. package/.docs/reference/agents/generateLegacy.md +77 -35
  25. package/.docs/reference/agents/getDefaultGenerateOptions.md +4 -6
  26. package/.docs/reference/agents/getDefaultOptions.md +4 -6
  27. package/.docs/reference/agents/getDefaultStreamOptions.md +4 -6
  28. package/.docs/reference/agents/getDescription.md +1 -1
  29. package/.docs/reference/agents/getInstructions.md +4 -6
  30. package/.docs/reference/agents/getLLM.md +6 -8
  31. package/.docs/reference/agents/getMemory.md +4 -6
  32. package/.docs/reference/agents/getModel.md +4 -6
  33. package/.docs/reference/agents/getTools.md +5 -7
  34. package/.docs/reference/agents/getVoice.md +4 -6
  35. package/.docs/reference/agents/listAgents.md +4 -6
  36. package/.docs/reference/agents/listScorers.md +4 -6
  37. package/.docs/reference/agents/listTools.md +4 -6
  38. package/.docs/reference/agents/listWorkflows.md +4 -6
  39. package/.docs/reference/agents/network.md +69 -23
  40. package/.docs/reference/ai-sdk/chat-route.md +7 -7
  41. package/.docs/reference/ai-sdk/handle-chat-stream.md +12 -12
  42. package/.docs/reference/ai-sdk/handle-network-stream.md +4 -4
  43. package/.docs/reference/ai-sdk/handle-workflow-stream.md +11 -11
  44. package/.docs/reference/ai-sdk/network-route.md +3 -3
  45. package/.docs/reference/ai-sdk/to-ai-sdk-stream.md +9 -9
  46. package/.docs/reference/ai-sdk/to-ai-sdk-v4-messages.md +9 -9
  47. package/.docs/reference/ai-sdk/to-ai-sdk-v5-messages.md +5 -5
  48. package/.docs/reference/ai-sdk/with-mastra.md +12 -12
  49. package/.docs/reference/ai-sdk/workflow-route.md +3 -3
  50. package/.docs/reference/auth/auth0.md +6 -6
  51. package/.docs/reference/auth/better-auth.md +5 -5
  52. package/.docs/reference/auth/clerk.md +5 -5
  53. package/.docs/reference/auth/firebase.md +7 -7
  54. package/.docs/reference/auth/jwt.md +1 -1
  55. package/.docs/reference/auth/supabase.md +4 -4
  56. package/.docs/reference/auth/workos.md +6 -6
  57. package/.docs/reference/cli/create-mastra.md +14 -14
  58. package/.docs/reference/client-js/mastra-client.md +23 -23
  59. package/.docs/reference/client-js/workflows.md +3 -3
  60. package/.docs/reference/core/addGateway.md +2 -2
  61. package/.docs/reference/core/getAgent.md +2 -2
  62. package/.docs/reference/core/getAgentById.md +2 -2
  63. package/.docs/reference/core/getDeployer.md +1 -1
  64. package/.docs/reference/core/getGateway.md +2 -2
  65. package/.docs/reference/core/getGatewayById.md +2 -2
  66. package/.docs/reference/core/getLogger.md +1 -1
  67. package/.docs/reference/core/getMCPServer.md +2 -2
  68. package/.docs/reference/core/getMCPServerById.md +3 -3
  69. package/.docs/reference/core/getMemory.md +2 -2
  70. package/.docs/reference/core/getScorer.md +2 -2
  71. package/.docs/reference/core/getScorerById.md +2 -2
  72. package/.docs/reference/core/getServer.md +1 -1
  73. package/.docs/reference/core/getStorage.md +1 -1
  74. package/.docs/reference/core/getStoredAgentById.md +18 -20
  75. package/.docs/reference/core/getTelemetry.md +1 -1
  76. package/.docs/reference/core/getVector.md +2 -2
  77. package/.docs/reference/core/getWorkflow.md +3 -3
  78. package/.docs/reference/core/listAgents.md +1 -1
  79. package/.docs/reference/core/listGateways.md +1 -1
  80. package/.docs/reference/core/listLogs.md +9 -11
  81. package/.docs/reference/core/listLogsByRunId.md +9 -9
  82. package/.docs/reference/core/listMCPServers.md +1 -1
  83. package/.docs/reference/core/listMemory.md +1 -1
  84. package/.docs/reference/core/listScorers.md +1 -1
  85. package/.docs/reference/core/listStoredAgents.md +9 -11
  86. package/.docs/reference/core/listVectors.md +1 -1
  87. package/.docs/reference/core/listWorkflows.md +2 -2
  88. package/.docs/reference/core/mastra-class.md +17 -17
  89. package/.docs/reference/core/mastra-model-gateway.md +15 -15
  90. package/.docs/reference/core/setLogger.md +2 -4
  91. package/.docs/reference/core/setStorage.md +1 -1
  92. package/.docs/reference/datasets/addItem.md +20 -4
  93. package/.docs/reference/datasets/addItems.md +8 -2
  94. package/.docs/reference/datasets/compareExperiments.md +15 -3
  95. package/.docs/reference/datasets/create.md +6 -6
  96. package/.docs/reference/datasets/dataset.md +1 -1
  97. package/.docs/reference/datasets/delete.md +2 -2
  98. package/.docs/reference/datasets/deleteExperiment.md +2 -2
  99. package/.docs/reference/datasets/deleteItem.md +2 -2
  100. package/.docs/reference/datasets/deleteItems.md +2 -2
  101. package/.docs/reference/datasets/get.md +2 -2
  102. package/.docs/reference/datasets/getDetails.md +9 -9
  103. package/.docs/reference/datasets/getExperiment.md +2 -2
  104. package/.docs/reference/datasets/getItem.md +3 -3
  105. package/.docs/reference/datasets/getItemHistory.md +22 -2
  106. package/.docs/reference/datasets/list.md +7 -3
  107. package/.docs/reference/datasets/listExperimentResults.md +34 -4
  108. package/.docs/reference/datasets/listExperiments.md +41 -3
  109. package/.docs/reference/datasets/listItems.md +18 -6
  110. package/.docs/reference/datasets/listVersions.md +23 -3
  111. package/.docs/reference/datasets/startExperiment.md +62 -12
  112. package/.docs/reference/datasets/startExperimentAsync.md +5 -1
  113. package/.docs/reference/datasets/update.md +6 -6
  114. package/.docs/reference/datasets/updateItem.md +5 -5
  115. package/.docs/reference/deployer.md +8 -8
  116. package/.docs/reference/evals/answer-relevancy.md +11 -11
  117. package/.docs/reference/evals/answer-similarity.md +17 -19
  118. package/.docs/reference/evals/bias.md +10 -10
  119. package/.docs/reference/evals/completeness.md +3 -3
  120. package/.docs/reference/evals/content-similarity.md +6 -6
  121. package/.docs/reference/evals/context-precision.md +4 -4
  122. package/.docs/reference/evals/context-relevance.md +4 -4
  123. package/.docs/reference/evals/create-scorer.md +47 -49
  124. package/.docs/reference/evals/faithfulness.md +11 -11
  125. package/.docs/reference/evals/hallucination.md +17 -21
  126. package/.docs/reference/evals/keyword-coverage.md +4 -4
  127. package/.docs/reference/evals/mastra-scorer.md +14 -14
  128. package/.docs/reference/evals/noise-sensitivity.md +4 -4
  129. package/.docs/reference/evals/prompt-alignment.md +4 -4
  130. package/.docs/reference/evals/run-evals.md +16 -16
  131. package/.docs/reference/evals/scorer-utils.md +3 -3
  132. package/.docs/reference/evals/textual-difference.md +3 -3
  133. package/.docs/reference/evals/tone-consistency.md +3 -3
  134. package/.docs/reference/evals/tool-call-accuracy.md +5 -5
  135. package/.docs/reference/evals/toxicity.md +8 -8
  136. package/.docs/reference/harness/harness-class.md +34 -42
  137. package/.docs/reference/logging/pino-logger.md +5 -5
  138. package/.docs/reference/memory/clone-utilities.md +5 -5
  139. package/.docs/reference/memory/cloneThread.md +17 -21
  140. package/.docs/reference/memory/createThread.md +10 -10
  141. package/.docs/reference/memory/deleteMessages.md +2 -2
  142. package/.docs/reference/memory/getThreadById.md +2 -2
  143. package/.docs/reference/memory/listThreads.md +5 -5
  144. package/.docs/reference/memory/memory-class.md +12 -14
  145. package/.docs/reference/memory/observational-memory.md +102 -94
  146. package/.docs/reference/memory/recall.md +14 -16
  147. package/.docs/reference/observability/tracing/configuration.md +27 -10
  148. package/.docs/reference/observability/tracing/exporters/console-exporter.md +4 -7
  149. package/.docs/reference/processors/batch-parts-processor.md +8 -10
  150. package/.docs/reference/processors/language-detector.md +14 -16
  151. package/.docs/reference/processors/message-history-processor.md +7 -9
  152. package/.docs/reference/processors/moderation-processor.md +13 -15
  153. package/.docs/reference/processors/pii-detector.md +14 -16
  154. package/.docs/reference/processors/processor-interface.md +62 -62
  155. package/.docs/reference/processors/prompt-injection-detector.md +11 -13
  156. package/.docs/reference/processors/semantic-recall-processor.md +14 -16
  157. package/.docs/reference/processors/system-prompt-scrubber.md +12 -14
  158. package/.docs/reference/processors/token-limiter-processor.md +11 -13
  159. package/.docs/reference/processors/tool-call-filter.md +5 -7
  160. package/.docs/reference/processors/tool-search-processor.md +9 -11
  161. package/.docs/reference/processors/unicode-normalizer.md +8 -10
  162. package/.docs/reference/processors/working-memory-processor.md +14 -18
  163. package/.docs/reference/rag/chunk.md +38 -38
  164. package/.docs/reference/rag/database-config.md +11 -7
  165. package/.docs/reference/rag/document.md +2 -2
  166. package/.docs/reference/rag/embeddings.md +12 -12
  167. package/.docs/reference/rag/extract-params.md +23 -23
  168. package/.docs/reference/rag/graph-rag.md +12 -12
  169. package/.docs/reference/rag/rerank.md +25 -17
  170. package/.docs/reference/rag/rerankWithScorer.md +25 -17
  171. package/.docs/reference/server/create-route.md +14 -14
  172. package/.docs/reference/server/express-adapter.md +10 -10
  173. package/.docs/reference/server/fastify-adapter.md +10 -10
  174. package/.docs/reference/server/hono-adapter.md +10 -10
  175. package/.docs/reference/server/koa-adapter.md +10 -10
  176. package/.docs/reference/server/mastra-server.md +10 -10
  177. package/.docs/reference/server/register-api-route.md +13 -13
  178. package/.docs/reference/storage/cloudflare-d1.md +5 -5
  179. package/.docs/reference/storage/cloudflare.md +6 -6
  180. package/.docs/reference/storage/composite.md +9 -9
  181. package/.docs/reference/storage/convex.md +3 -3
  182. package/.docs/reference/storage/dynamodb.md +9 -9
  183. package/.docs/reference/storage/lance.md +3 -3
  184. package/.docs/reference/storage/libsql.md +2 -2
  185. package/.docs/reference/storage/mongodb.md +5 -5
  186. package/.docs/reference/storage/mssql.md +2 -2
  187. package/.docs/reference/storage/postgresql.md +25 -25
  188. package/.docs/reference/storage/upstash.md +3 -3
  189. package/.docs/reference/streaming/ChunkType.md +251 -59
  190. package/.docs/reference/streaming/agents/MastraModelOutput.md +86 -16
  191. package/.docs/reference/streaming/agents/stream.md +155 -43
  192. package/.docs/reference/streaming/agents/streamLegacy.md +79 -39
  193. package/.docs/reference/streaming/workflows/resumeStream.md +18 -8
  194. package/.docs/reference/streaming/workflows/stream.md +21 -9
  195. package/.docs/reference/streaming/workflows/timeTravelStream.md +4 -4
  196. package/.docs/reference/tools/create-tool.md +25 -21
  197. package/.docs/reference/tools/document-chunker-tool.md +7 -7
  198. package/.docs/reference/tools/graph-rag-tool.md +16 -18
  199. package/.docs/reference/tools/mcp-client.md +38 -27
  200. package/.docs/reference/tools/mcp-server.md +45 -45
  201. package/.docs/reference/tools/vector-query-tool.md +34 -22
  202. package/.docs/reference/vectors/astra.md +22 -22
  203. package/.docs/reference/vectors/chroma.md +44 -44
  204. package/.docs/reference/vectors/convex.md +26 -26
  205. package/.docs/reference/vectors/couchbase.md +30 -30
  206. package/.docs/reference/vectors/duckdb.md +29 -29
  207. package/.docs/reference/vectors/elasticsearch.md +27 -27
  208. package/.docs/reference/vectors/lance.md +46 -38
  209. package/.docs/reference/vectors/libsql.md +31 -31
  210. package/.docs/reference/vectors/mongodb.md +32 -32
  211. package/.docs/reference/vectors/opensearch.md +27 -27
  212. package/.docs/reference/vectors/pg.md +60 -44
  213. package/.docs/reference/vectors/pinecone.md +38 -38
  214. package/.docs/reference/vectors/qdrant.md +36 -36
  215. package/.docs/reference/vectors/s3vectors.md +24 -24
  216. package/.docs/reference/vectors/turbopuffer.md +29 -29
  217. package/.docs/reference/vectors/upstash.md +25 -25
  218. package/.docs/reference/vectors/vectorize.md +27 -27
  219. package/.docs/reference/voice/azure.md +19 -15
  220. package/.docs/reference/voice/cloudflare.md +10 -12
  221. package/.docs/reference/voice/composite-voice.md +10 -10
  222. package/.docs/reference/voice/deepgram.md +20 -14
  223. package/.docs/reference/voice/elevenlabs.md +20 -22
  224. package/.docs/reference/voice/google-gemini-live.md +42 -44
  225. package/.docs/reference/voice/google.md +25 -21
  226. package/.docs/reference/voice/mastra-voice.md +20 -20
  227. package/.docs/reference/voice/murf.md +24 -28
  228. package/.docs/reference/voice/openai-realtime.md +26 -26
  229. package/.docs/reference/voice/openai.md +14 -12
  230. package/.docs/reference/voice/playai.md +13 -15
  231. package/.docs/reference/voice/sarvam.md +21 -25
  232. package/.docs/reference/voice/speechify.md +11 -13
  233. package/.docs/reference/voice/voice.addInstructions.md +1 -1
  234. package/.docs/reference/voice/voice.addTools.md +1 -1
  235. package/.docs/reference/voice/voice.answer.md +1 -1
  236. package/.docs/reference/voice/voice.connect.md +3 -3
  237. package/.docs/reference/voice/voice.events.md +11 -11
  238. package/.docs/reference/voice/voice.getSpeakers.md +30 -30
  239. package/.docs/reference/voice/voice.listen.md +9 -9
  240. package/.docs/reference/voice/voice.off.md +2 -2
  241. package/.docs/reference/voice/voice.on.md +2 -2
  242. package/.docs/reference/voice/voice.send.md +1 -1
  243. package/.docs/reference/voice/voice.speak.md +11 -11
  244. package/.docs/reference/voice/voice.updateConfig.md +3 -3
  245. package/.docs/reference/workflows/run-methods/cancel.md +2 -2
  246. package/.docs/reference/workflows/run-methods/restart.md +17 -5
  247. package/.docs/reference/workflows/run-methods/resume.md +23 -9
  248. package/.docs/reference/workflows/run-methods/start.md +22 -8
  249. package/.docs/reference/workflows/run-methods/startAsync.md +12 -6
  250. package/.docs/reference/workflows/run-methods/timeTravel.md +29 -13
  251. package/.docs/reference/workflows/run.md +12 -12
  252. package/.docs/reference/workflows/step.md +24 -26
  253. package/.docs/reference/workflows/workflow-methods/branch.md +2 -2
  254. package/.docs/reference/workflows/workflow-methods/commit.md +1 -1
  255. package/.docs/reference/workflows/workflow-methods/create-run.md +4 -4
  256. package/.docs/reference/workflows/workflow-methods/dountil.md +3 -3
  257. package/.docs/reference/workflows/workflow-methods/dowhile.md +3 -3
  258. package/.docs/reference/workflows/workflow-methods/foreach.md +9 -9
  259. package/.docs/reference/workflows/workflow-methods/map.md +2 -2
  260. package/.docs/reference/workflows/workflow-methods/parallel.md +2 -2
  261. package/.docs/reference/workflows/workflow-methods/sleep.md +2 -2
  262. package/.docs/reference/workflows/workflow-methods/sleepUntil.md +2 -2
  263. package/.docs/reference/workflows/workflow-methods/then.md +2 -2
  264. package/.docs/reference/workflows/workflow.md +40 -50
  265. package/.docs/reference/workspace/daytona-sandbox.md +33 -33
  266. package/.docs/reference/workspace/e2b-sandbox.md +20 -20
  267. package/.docs/reference/workspace/filesystem.md +22 -22
  268. package/.docs/reference/workspace/gcs-filesystem.md +15 -15
  269. package/.docs/reference/workspace/local-filesystem.md +35 -35
  270. package/.docs/reference/workspace/local-sandbox.md +26 -26
  271. package/.docs/reference/workspace/s3-filesystem.md +18 -18
  272. package/.docs/reference/workspace/sandbox.md +8 -8
  273. package/.docs/reference/workspace/workspace-class.md +30 -34
  274. package/CHANGELOG.md +16 -0
  275. package/package.json +6 -6
  276. package/.docs/docs/getting-started/start.md +0 -28
@@ -4,45 +4,43 @@ The `createAnswerSimilarityScorer()` function creates a scorer that evaluates ho
4
4
 
5
5
  ## Parameters
6
6
 
7
- **model:** (`LanguageModel`): The language model used to evaluate semantic similarity between outputs and ground truth.
7
+ **model** (`LanguageModel`): The language model used to evaluate semantic similarity between outputs and ground truth.
8
8
 
9
- **options:** (`AnswerSimilarityOptions`): Configuration options for the scorer.
9
+ **options** (`AnswerSimilarityOptions`): Configuration options for the scorer.
10
10
 
11
- ### AnswerSimilarityOptions
11
+ **options.requireGroundTruth** (`boolean`): Whether to require ground truth for evaluation. If false, missing ground truth returns score 0.
12
12
 
13
- **requireGroundTruth:** (`boolean`): Whether to require ground truth for evaluation. If false, missing ground truth returns score 0. (Default: `true`)
13
+ **options.semanticThreshold** (`number`): Weight for semantic matches vs exact matches (0-1).
14
14
 
15
- **semanticThreshold:** (`number`): Weight for semantic matches vs exact matches (0-1). (Default: `0.8`)
15
+ **options.exactMatchBonus** (`number`): Additional score bonus for exact matches (0-1).
16
16
 
17
- **exactMatchBonus:** (`number`): Additional score bonus for exact matches (0-1). (Default: `0.2`)
17
+ **options.missingPenalty** (`number`): Penalty per missing key concept from ground truth.
18
18
 
19
- **missingPenalty:** (`number`): Penalty per missing key concept from ground truth. (Default: `0.15`)
19
+ **options.contradictionPenalty** (`number`): Penalty for contradictory information. High value ensures wrong answers score near 0.
20
20
 
21
- **contradictionPenalty:** (`number`): Penalty for contradictory information. High value ensures wrong answers score near 0. (Default: `1.0`)
21
+ **options.extraInfoPenalty** (`number`): Mild penalty for extra information not present in ground truth (capped at 0.2).
22
22
 
23
- **extraInfoPenalty:** (`number`): Mild penalty for extra information not present in ground truth (capped at 0.2). (Default: `0.05`)
24
-
25
- **scale:** (`number`): Score scaling factor. (Default: `1`)
23
+ **options.scale** (`number`): Score scaling factor.
26
24
 
27
25
  This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but **requires ground truth** to be provided in the run object.
28
26
 
29
27
  ## .run() Returns
30
28
 
31
- **runId:** (`string`): The id of the run (optional).
29
+ **runId** (`string`): The id of the run (optional).
32
30
 
33
- **score:** (`number`): Similarity score between 0-1 (or 0-scale if custom scale used). Higher scores indicate better similarity to ground truth.
31
+ **score** (`number`): Similarity score between 0-1 (or 0-scale if custom scale used). Higher scores indicate better similarity to ground truth.
34
32
 
35
- **reason:** (`string`): Human-readable explanation of the score with actionable feedback.
33
+ **reason** (`string`): Human-readable explanation of the score with actionable feedback.
36
34
 
37
- **preprocessStepResult:** (`object`): Extracted semantic units from output and ground truth.
35
+ **preprocessStepResult** (`object`): Extracted semantic units from output and ground truth.
38
36
 
39
- **analyzeStepResult:** (`object`): Detailed analysis of matches, contradictions, and extra information.
37
+ **analyzeStepResult** (`object`): Detailed analysis of matches, contradictions, and extra information.
40
38
 
41
- **preprocessPrompt:** (`string`): The prompt used for semantic unit extraction.
39
+ **preprocessPrompt** (`string`): The prompt used for semantic unit extraction.
42
40
 
43
- **analyzePrompt:** (`string`): The prompt used for similarity analysis.
41
+ **analyzePrompt** (`string`): The prompt used for similarity analysis.
44
42
 
45
- **generateReasonPrompt:** (`string`): The prompt used for generating the explanation.
43
+ **generateReasonPrompt** (`string`): The prompt used for generating the explanation.
46
44
 
47
45
  ## Scoring Details
48
46
 
@@ -4,29 +4,29 @@ The `createBiasScorer()` function accepts a single options object with the follo
4
4
 
5
5
  ## Parameters
6
6
 
7
- **model:** (`LanguageModel`): Configuration for the model used to evaluate bias.
7
+ **model** (`LanguageModel`): Configuration for the model used to evaluate bias.
8
8
 
9
- **scale:** (`number`): Maximum score value. (Default: `1`)
9
+ **scale** (`number`): Maximum score value. (Default: `1`)
10
10
 
11
11
  This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
12
12
 
13
13
  ## .run() Returns
14
14
 
15
- **runId:** (`string`): The id of the run (optional).
15
+ **runId** (`string`): The id of the run (optional).
16
16
 
17
- **preprocessStepResult:** (`object`): Object with extracted opinions: { opinions: string\[] }
17
+ **preprocessStepResult** (`object`): Object with extracted opinions: { opinions: string\[] }
18
18
 
19
- **preprocessPrompt:** (`string`): The prompt sent to the LLM for the preprocess step (optional).
19
+ **preprocessPrompt** (`string`): The prompt sent to the LLM for the preprocess step (optional).
20
20
 
21
- **analyzeStepResult:** (`object`): Object with results: { results: Array<{ result: 'yes' | 'no', reason: string }> }
21
+ **analyzeStepResult** (`object`): Object with results: { results: Array<{ result: 'yes' | 'no', reason: string }> }
22
22
 
23
- **analyzePrompt:** (`string`): The prompt sent to the LLM for the analyze step (optional).
23
+ **analyzePrompt** (`string`): The prompt sent to the LLM for the analyze step (optional).
24
24
 
25
- **score:** (`number`): Bias score (0 to scale, default 0-1). Higher scores indicate more bias.
25
+ **score** (`number`): Bias score (0 to scale, default 0-1). Higher scores indicate more bias.
26
26
 
27
- **reason:** (`string`): Explanation of the score.
27
+ **reason** (`string`): Explanation of the score.
28
28
 
29
- **generateReasonPrompt:** (`string`): The prompt sent to the LLM for the generateReason step (optional).
29
+ **generateReasonPrompt** (`string`): The prompt sent to the LLM for the generateReason step (optional).
30
30
 
31
31
  ## Bias Categories
32
32
 
@@ -10,11 +10,11 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
10
10
 
11
11
  ## .run() Returns
12
12
 
13
- **runId:** (`string`): The id of the run (optional).
13
+ **runId** (`string`): The id of the run (optional).
14
14
 
15
- **preprocessStepResult:** (`object`): Object with extracted elements and coverage details: { inputElements: string\[], outputElements: string\[], missingElements: string\[], elementCounts: { input: number, output: number } }
15
+ **preprocessStepResult** (`object`): Object with extracted elements and coverage details: { inputElements: string\[], outputElements: string\[], missingElements: string\[], elementCounts: { input: number, output: number } }
16
16
 
17
- **score:** (`number`): Completeness score (0-1) representing the proportion of input elements covered in the output.
17
+ **score** (`number`): Completeness score (0-1) representing the proportion of input elements covered in the output.
18
18
 
19
19
  The `.run()` method returns a result in the following shape:
20
20
 
@@ -6,21 +6,21 @@ The `createContentSimilarityScorer()` function measures the textual similarity b
6
6
 
7
7
  The `createContentSimilarityScorer()` function accepts a single options object with the following properties:
8
8
 
9
- **ignoreCase:** (`boolean`): Whether to ignore case differences when comparing strings. (Default: `true`)
9
+ **ignoreCase** (`boolean`): Whether to ignore case differences when comparing strings. (Default: `true`)
10
10
 
11
- **ignoreWhitespace:** (`boolean`): Whether to normalize whitespace when comparing strings. (Default: `true`)
11
+ **ignoreWhitespace** (`boolean`): Whether to normalize whitespace when comparing strings. (Default: `true`)
12
12
 
13
13
  This function returns an instance of the MastraScorer class. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
14
14
 
15
15
  ## .run() Returns
16
16
 
17
- **runId:** (`string`): The id of the run (optional).
17
+ **runId** (`string`): The id of the run (optional).
18
18
 
19
- **preprocessStepResult:** (`object`): Object with processed input and output: { processedInput: string, processedOutput: string }
19
+ **preprocessStepResult** (`object`): Object with processed input and output: { processedInput: string, processedOutput: string }
20
20
 
21
- **analyzeStepResult:** (`object`): Object with similarity: { similarity: number }
21
+ **analyzeStepResult** (`object`): Object with similarity: { similarity: number }
22
22
 
23
- **score:** (`number`): Similarity score (0-1) where 1 indicates perfect similarity.
23
+ **score** (`number`): Similarity score (0-1) where 1 indicates perfect similarity.
24
24
 
25
25
  ## Scoring Details
26
26
 
@@ -22,17 +22,17 @@ Use when optimizing context selection for:
22
22
 
23
23
  ## Parameters
24
24
 
25
- **model:** (`MastraModelConfig`): The language model to use for evaluating context relevance
25
+ **model** (`MastraModelConfig`): The language model to use for evaluating context relevance
26
26
 
27
- **options:** (`ContextPrecisionMetricOptions`): Configuration options for the scorer
27
+ **options** (`ContextPrecisionMetricOptions`): Configuration options for the scorer
28
28
 
29
29
  **Note**: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.
30
30
 
31
31
  ## .run() Returns
32
32
 
33
- **score:** (`number`): Mean Average Precision score between 0 and scale (default 0-1)
33
+ **score** (`number`): Mean Average Precision score between 0 and scale (default 0-1)
34
34
 
35
- **reason:** (`string`): Human-readable explanation of the context precision evaluation
35
+ **reason** (`string`): Human-readable explanation of the context precision evaluation
36
36
 
37
37
  ## Scoring Details
38
38
 
@@ -22,17 +22,17 @@ Use when optimizing for:
22
22
 
23
23
  ## Parameters
24
24
 
25
- **model:** (`MastraModelConfig`): The language model to use for evaluating context relevance
25
+ **model** (`MastraModelConfig`): The language model to use for evaluating context relevance
26
26
 
27
- **options:** (`ContextRelevanceOptions`): Configuration options for the scorer
27
+ **options** (`ContextRelevanceOptions`): Configuration options for the scorer
28
28
 
29
29
  Note: Either `context` or `contextExtractor` must be provided. If both are provided, `contextExtractor` takes precedence.
30
30
 
31
31
  ## .run() Returns
32
32
 
33
- **score:** (`number`): Weighted relevance score between 0 and scale (default 0-1)
33
+ **score** (`number`): Weighted relevance score between 0 and scale (default 0-1)
34
34
 
35
- **reason:** (`string`): Human-readable explanation of the context relevance evaluation
35
+ **reason** (`string`): Human-readable explanation of the context relevance evaluation
36
36
 
37
37
  ## Scoring Details
38
38
 
@@ -37,23 +37,21 @@ const scorer = createScorer({
37
37
 
38
38
  ## createScorer Options
39
39
 
40
- **id:** (`string`): Unique identifier for the scorer. Used as the name if \`name\` is not provided.
40
+ **id** (`string`): Unique identifier for the scorer. Used as the name if \`name\` is not provided.
41
41
 
42
- **name?:** (`string`): Name of the scorer. Defaults to \`id\` if not provided.
42
+ **name** (`string`): Name of the scorer. Defaults to \`id\` if not provided.
43
43
 
44
- **description:** (`string`): Description of what the scorer does.
44
+ **description** (`string`): Description of what the scorer does.
45
45
 
46
- **judge?:** (`object`): Optional judge configuration for LLM-based steps. See Judge Object section below.
46
+ **judge** (`object`): Optional judge configuration for LLM-based steps.
47
47
 
48
- **type?:** (`string`): Type specification for input/output. Use 'agent' for automatic agent types. For custom types, use the generic approach instead.
48
+ **judge.model** (`LanguageModel`): The LLM model instance to use for evaluation.
49
49
 
50
- This function returns a scorer builder that you can chain step methods onto. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
51
-
52
- ## Judge Object
50
+ **judge.instructions** (`string`): System prompt/instructions for the LLM.
53
51
 
54
- **model:** (`LanguageModel`): The LLM model instance to use for evaluation.
52
+ **type** (`string`): Type specification for input/output. Use 'agent' for automatic agent types. For custom types, use the generic approach instead.
55
53
 
56
- **instructions:** (`string`): System prompt/instructions for the LLM.
54
+ This function returns a scorer builder that you can chain step methods onto. See the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer) for details on the `.run()` method and its input/output.
57
55
 
58
56
  The judge only runs for steps defined as **prompt objects** (`preprocess`, `analyze`, `generateScore`, `generateReason` in prompt mode). If you use function steps only, the judge is never called and there is no LLM output to inspect. In that case, any score/reason must be produced by your functions.
59
57
 
@@ -149,28 +147,28 @@ Optional preprocessing step that can extract or transform data before analysis.
149
147
 
150
148
  **Function Mode:** Function: `({ run, results }) => any`
151
149
 
152
- **run.input:** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
150
+ **run.input** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
153
151
 
154
- **run.output:** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
152
+ **run.output** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
155
153
 
156
- **run.runId:** (`string`): Unique identifier for this scoring run.
154
+ **run.runId** (`string`): Unique identifier for this scoring run.
157
155
 
158
- **run.requestContext?:** (`object`): Request Context from the agent or workflow step being evaluated (optional).
156
+ **run.requestContext** (`object`): Request Context from the agent or workflow step being evaluated (optional).
159
157
 
160
- **results:** (`object`): Empty object (no previous steps).
158
+ **results** (`object`): Empty object (no previous steps).
161
159
 
162
160
  Returns: `any`\
163
161
  The method can return any value. The returned value will be available to subsequent steps as `preprocessStepResult`.
164
162
 
165
163
  **Prompt Object Mode:**
166
164
 
167
- **description:** (`string`): Description of what this preprocessing step does.
165
+ **description** (`string`): Description of what this preprocessing step does.
168
166
 
169
- **outputSchema:** (`ZodSchema`): Zod schema for the expected output of the preprocess step.
167
+ **outputSchema** (`ZodSchema`): Zod schema for the expected output of the preprocess step.
170
168
 
171
- **createPrompt:** (`function`): Function: ({ run, results }) => string. Returns the prompt for the LLM.
169
+ **createPrompt** (`function`): Function: ({ run, results }) => string. Returns the prompt for the LLM.
172
170
 
173
- **judge?:** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
171
+ **judge** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
174
172
 
175
173
  ### analyze
176
174
 
@@ -178,28 +176,28 @@ Optional analysis step that processes the input/output and any preprocessed data
178
176
 
179
177
  **Function Mode:** Function: `({ run, results }) => any`
180
178
 
181
- **run.input:** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
179
+ **run.input** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
182
180
 
183
- **run.output:** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
181
+ **run.output** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
184
182
 
185
- **run.runId:** (`string`): Unique identifier for this scoring run.
183
+ **run.runId** (`string`): Unique identifier for this scoring run.
186
184
 
187
- **run.requestContext?:** (`object`): Request Context from the agent or workflow step being evaluated (optional).
185
+ **run.requestContext** (`object`): Request Context from the agent or workflow step being evaluated (optional).
188
186
 
189
- **results.preprocessStepResult?:** (`any`): Result from preprocess step, if defined (optional).
187
+ **results.preprocessStepResult** (`any`): Result from preprocess step, if defined (optional).
190
188
 
191
189
  Returns: `any`\
192
190
  The method can return any value. The returned value will be available to subsequent steps as `analyzeStepResult`.
193
191
 
194
192
  **Prompt Object Mode:**
195
193
 
196
- **description:** (`string`): Description of what this analysis step does.
194
+ **description** (`string`): Description of what this analysis step does.
197
195
 
198
- **outputSchema:** (`ZodSchema`): Zod schema for the expected output of the analyze step.
196
+ **outputSchema** (`ZodSchema`): Zod schema for the expected output of the analyze step.
199
197
 
200
- **createPrompt:** (`function`): Function: ({ run, results }) => string. Returns the prompt for the LLM.
198
+ **createPrompt** (`function`): Function: ({ run, results }) => string. Returns the prompt for the LLM.
201
199
 
202
- **judge?:** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
200
+ **judge** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
203
201
 
204
202
  ### generateScore
205
203
 
@@ -207,34 +205,34 @@ The method can return any value. The returned value will be available to subsequ
207
205
 
208
206
  **Function Mode:** Function: `({ run, results }) => number`
209
207
 
210
- **run.input:** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
208
+ **run.input** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
211
209
 
212
- **run.output:** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
210
+ **run.output** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
213
211
 
214
- **run.runId:** (`string`): Unique identifier for this scoring run.
212
+ **run.runId** (`string`): Unique identifier for this scoring run.
215
213
 
216
- **run.requestContext?:** (`object`): Request Context from the agent or workflow step being evaluated (optional).
214
+ **run.requestContext** (`object`): Request Context from the agent or workflow step being evaluated (optional).
217
215
 
218
- **results.preprocessStepResult?:** (`any`): Result from preprocess step, if defined (optional).
216
+ **results.preprocessStepResult** (`any`): Result from preprocess step, if defined (optional).
219
217
 
220
- **results.analyzeStepResult?:** (`any`): Result from analyze step, if defined (optional).
218
+ **results.analyzeStepResult** (`any`): Result from analyze step, if defined (optional).
221
219
 
222
220
  Returns: `number`\
223
221
  The method must return a numerical score.
224
222
 
225
223
  **Prompt Object Mode:**
226
224
 
227
- **description:** (`string`): Description of what this scoring step does.
225
+ **description** (`string`): Description of what this scoring step does.
228
226
 
229
- **outputSchema:** (`ZodSchema`): Zod schema for the expected output of the generateScore step.
227
+ **outputSchema** (`ZodSchema`): Zod schema for the expected output of the generateScore step.
230
228
 
231
- **createPrompt:** (`function`): Function: ({ run, results }) => string. Returns the prompt for the LLM.
229
+ **createPrompt** (`function`): Function: ({ run, results }) => string. Returns the prompt for the LLM.
232
230
 
233
- **judge?:** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
231
+ **judge** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
234
232
 
235
233
  When using prompt object mode, you must also provide a `calculateScore` function to convert the LLM output to a numerical score:
236
234
 
237
- **calculateScore:** (`function`): Function: ({ run, results, analyzeStepResult }) => number. Converts the LLM's structured output into a numerical score.
235
+ **calculateScore** (`function`): Function: ({ run, results, analyzeStepResult }) => number. Converts the LLM's structured output into a numerical score.
238
236
 
239
237
  ### generateReason
240
238
 
@@ -242,29 +240,29 @@ Optional step that provides an explanation for the score.
242
240
 
243
241
  **Function Mode:** Function: `({ run, results, score }) => string`
244
242
 
245
- **run.input:** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
243
+ **run.input** (`any`): Input records provided to the scorer. If the scorer is added to an agent, this will be an array of user messages, e.g. \`\[{ role: 'user', content: 'hello world' }]\`. If the scorer is used in a workflow, this will be the input of the workflow.
246
244
 
247
- **run.output:** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
245
+ **run.output** (`any`): Output record provided to the scorer. For agents, this is usually the agent's response. For workflows, this is the workflow's output.
248
246
 
249
- **run.runId:** (`string`): Unique identifier for this scoring run.
247
+ **run.runId** (`string`): Unique identifier for this scoring run.
250
248
 
251
- **run.requestContext?:** (`object`): Request Context from the agent or workflow step being evaluated (optional).
249
+ **run.requestContext** (`object`): Request Context from the agent or workflow step being evaluated (optional).
252
250
 
253
- **results.preprocessStepResult?:** (`any`): Result from preprocess step, if defined (optional).
251
+ **results.preprocessStepResult** (`any`): Result from preprocess step, if defined (optional).
254
252
 
255
- **results.analyzeStepResult?:** (`any`): Result from analyze step, if defined (optional).
253
+ **results.analyzeStepResult** (`any`): Result from analyze step, if defined (optional).
256
254
 
257
- **score:** (`number`): Score computed by the generateScore step.
255
+ **score** (`number`): Score computed by the generateScore step.
258
256
 
259
257
  Returns: `string`\
260
258
  The method must return a string explaining the score.
261
259
 
262
260
  **Prompt Object Mode:**
263
261
 
264
- **description:** (`string`): Description of what this reasoning step does.
262
+ **description** (`string`): Description of what this reasoning step does.
265
263
 
266
- **createPrompt:** (`function`): Function: ({ run, results, score }) => string. Returns the prompt for the LLM.
264
+ **createPrompt** (`function`): Function: ({ run, results, score }) => string. Returns the prompt for the LLM.
267
265
 
268
- **judge?:** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
266
+ **judge** (`object`): (Optional) LLM judge for this step (can override main judge). See Judge Object section.
269
267
 
270
268
  All step functions can be async.
@@ -6,31 +6,31 @@ The `createFaithfulnessScorer()` function evaluates how factually accurate an LL
6
6
 
7
7
  The `createFaithfulnessScorer()` function accepts a single options object with the following properties:
8
8
 
9
- **model:** (`LanguageModel`): Configuration for the model used to evaluate faithfulness.
9
+ **model** (`LanguageModel`): Configuration for the model used to evaluate faithfulness.
10
10
 
11
- **context:** (`string[]`): Array of context chunks against which the output's claims will be verified.
11
+ **context** (`string[]`): Array of context chunks against which the output's claims will be verified.
12
12
 
13
- **scale:** (`number`): The maximum score value. The final score will be normalized to this scale. (Default: `1`)
13
+ **scale** (`number`): The maximum score value. The final score will be normalized to this scale. (Default: `1`)
14
14
 
15
15
  This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
16
16
 
17
17
  ## .run() Returns
18
18
 
19
- **runId:** (`string`): The id of the run (optional).
19
+ **runId** (`string`): The id of the run (optional).
20
20
 
21
- **preprocessStepResult:** (`string[]`): Array of extracted claims from the output.
21
+ **preprocessStepResult** (`string[]`): Array of extracted claims from the output.
22
22
 
23
- **preprocessPrompt:** (`string`): The prompt sent to the LLM for the preprocess step (optional).
23
+ **preprocessPrompt** (`string`): The prompt sent to the LLM for the preprocess step (optional).
24
24
 
25
- **analyzeStepResult:** (`object`): Object with verdicts: { verdicts: Array<{ verdict: 'yes' | 'no' | 'unsure', reason: string }> }
25
+ **analyzeStepResult** (`object`): Object with verdicts: { verdicts: Array<{ verdict: 'yes' | 'no' | 'unsure', reason: string }> }
26
26
 
27
- **analyzePrompt:** (`string`): The prompt sent to the LLM for the analyze step (optional).
27
+ **analyzePrompt** (`string`): The prompt sent to the LLM for the analyze step (optional).
28
28
 
29
- **score:** (`number`): A score between 0 and the configured scale, representing the proportion of claims that are supported by the context.
29
+ **score** (`number`): A score between 0 and the configured scale, representing the proportion of claims that are supported by the context.
30
30
 
31
- **reason:** (`string`): A detailed explanation of the score, including which claims were supported, contradicted, or marked as unsure.
31
+ **reason** (`string`): A detailed explanation of the score, including which claims were supported, contradicted, or marked as unsure.
32
32
 
33
- **generateReasonPrompt:** (`string`): The prompt sent to the LLM for the generateReason step (optional).
33
+ **generateReasonPrompt** (`string`): The prompt sent to the LLM for the generateReason step (optional).
34
34
 
35
35
  ## Scoring Details
36
36
 
@@ -6,45 +6,41 @@ The `createHallucinationScorer()` function evaluates whether an LLM generates fa
6
6
 
7
7
  The `createHallucinationScorer()` function accepts a single options object with the following properties:
8
8
 
9
- **model:** (`LanguageModel`): Configuration for the model used to evaluate hallucination.
9
+ **model** (`LanguageModel`): Configuration for the model used to evaluate hallucination.
10
10
 
11
- **options.scale:** (`number`): Maximum score value. (Default: `1`)
11
+ **options.scale** (`number`): Maximum score value. (Default: `1`)
12
12
 
13
- **options.context:** (`string[]`): Static context strings to use as ground truth for hallucination detection.
13
+ **options.context** (`string[]`): Static context strings to use as ground truth for hallucination detection.
14
14
 
15
- **options.getContext:** (`(params: GetContextParams) => string[] | Promise<string[]>`): A hook to dynamically resolve context at runtime. Takes priority over static context. Useful for live scoring where context (like tool results) is only available when the scorer runs.
15
+ **options.getContext** (`(params: GetContextParams) => string[] | Promise<string[]>`): A hook to dynamically resolve context at runtime. Takes priority over static context. Useful for live scoring where context (like tool results) is only available when the scorer runs.
16
16
 
17
- This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
18
-
19
- ### GetContextParams
17
+ **options.getContext.run** (`GetContextRun`): The scorer run containing input, output, runId, requestContext, and tracingContext.
20
18
 
21
- The `getContext` hook receives the following parameters:
19
+ **options.getContext.results** (`Record<string, any>`): Accumulated results from previous steps (e.g., preprocessStepResult with extracted claims).
22
20
 
23
- **run:** (`GetContextRun`): The scorer run containing input, output, runId, requestContext, and tracingContext.
21
+ **options.getContext.score** (`number`): The computed score. Only present when called from the generateReason step.
24
22
 
25
- **results:** (`Record<string, any>`): Accumulated results from previous steps (e.g., preprocessStepResult with extracted claims).
23
+ **options.getContext.step** (`'analyze' | 'generateReason'`): Which step is calling the hook. Useful for caching context between calls.
26
24
 
27
- **score:** (`number`): The computed score. Only present when called from the generateReason step.
28
-
29
- **step:** (`'analyze' | 'generateReason'`): Which step is calling the hook. Useful for caching context between calls.
25
+ This function returns an instance of the MastraScorer class. The `.run()` method accepts the same input as other scorers (see the [MastraScorer reference](https://mastra.ai/reference/evals/mastra-scorer)), but the return value includes LLM-specific fields as documented below.
30
26
 
31
27
  ## .run() Returns
32
28
 
33
- **runId:** (`string`): The id of the run (optional).
29
+ **runId** (`string`): The id of the run (optional).
34
30
 
35
- **preprocessStepResult:** (`object`): Object with extracted claims: { claims: string\[] }
31
+ **preprocessStepResult** (`object`): Object with extracted claims: { claims: string\[] }
36
32
 
37
- **preprocessPrompt:** (`string`): The prompt sent to the LLM for the preprocess step (optional).
33
+ **preprocessPrompt** (`string`): The prompt sent to the LLM for the preprocess step (optional).
38
34
 
39
- **analyzeStepResult:** (`object`): Object with verdicts: { verdicts: Array<{ statement: string, verdict: 'yes' | 'no', reason: string }> }
35
+ **analyzeStepResult** (`object`): Object with verdicts: { verdicts: Array<{ statement: string, verdict: 'yes' | 'no', reason: string }> }
40
36
 
41
- **analyzePrompt:** (`string`): The prompt sent to the LLM for the analyze step (optional).
37
+ **analyzePrompt** (`string`): The prompt sent to the LLM for the analyze step (optional).
42
38
 
43
- **score:** (`number`): Hallucination score (0 to scale, default 0-1).
39
+ **score** (`number`): Hallucination score (0 to scale, default 0-1).
44
40
 
45
- **reason:** (`string`): Detailed explanation of the score and identified contradictions.
41
+ **reason** (`string`): Detailed explanation of the score and identified contradictions.
46
42
 
47
- **generateReasonPrompt:** (`string`): The prompt sent to the LLM for the generateReason step (optional).
43
+ **generateReasonPrompt** (`string`): The prompt sent to the LLM for the generateReason step (optional).
48
44
 
49
45
  ## Scoring Details
50
46
 
@@ -10,13 +10,13 @@ This function returns an instance of the MastraScorer class. See the [MastraScor
10
10
 
11
11
  ## .run() Returns
12
12
 
13
- **runId:** (`string`): The id of the run (optional).
13
+ **runId** (`string`): The id of the run (optional).
14
14
 
15
- **preprocessStepResult:** (`object`): Object with extracted keywords: { referenceKeywords: Set\<string>, responseKeywords: Set\<string> }
15
+ **preprocessStepResult** (`object`): Object with extracted keywords: { referenceKeywords: Set\<string>, responseKeywords: Set\<string> }
16
16
 
17
- **analyzeStepResult:** (`object`): Object with keyword coverage: { totalKeywords: number, matchedKeywords: number }
17
+ **analyzeStepResult** (`object`): Object with keyword coverage: { totalKeywords: number, matchedKeywords: number }
18
18
 
19
- **score:** (`number`): Coverage score (0-1) representing the proportion of matched keywords.
19
+ **score** (`number`): Coverage score (0-1) representing the proportion of matched keywords.
20
20
 
21
21
  `.run()` returns a result in the following shape:
22
22
 
@@ -39,35 +39,35 @@ const result = await scorer.run({
39
39
 
40
40
  ## .run() Input
41
41
 
42
- **input:** (`any`): Input data to be evaluated. Can be any type depending on your scorer's requirements.
42
+ **input** (`any`): Input data to be evaluated. Can be any type depending on your scorer's requirements.
43
43
 
44
- **output:** (`any`): Output data to be evaluated. Can be any type depending on your scorer's requirements.
44
+ **output** (`any`): Output data to be evaluated. Can be any type depending on your scorer's requirements.
45
45
 
46
- **runId:** (`string`): Optional unique identifier for this scoring run.
46
+ **runId** (`string`): Optional unique identifier for this scoring run.
47
47
 
48
- **requestContext:** (`any`): Optional request context from the agent or workflow step being evaluated.
48
+ **requestContext** (`any`): Optional request context from the agent or workflow step being evaluated.
49
49
 
50
- **groundTruth:** (`any`): Optional expected or reference output for comparison during scoring. Automatically passed when using runEvals.
50
+ **groundTruth** (`any`): Optional expected or reference output for comparison during scoring. Automatically passed when using runEvals.
51
51
 
52
52
  ## .run() Returns
53
53
 
54
- **runId:** (`string`): The unique identifier for this scoring run.
54
+ **runId** (`string`): The unique identifier for this scoring run.
55
55
 
56
- **score:** (`number`): Numerical score computed by the generateScore step.
56
+ **score** (`number`): Numerical score computed by the generateScore step.
57
57
 
58
- **reason:** (`string`): Explanation for the score, if generateReason step was defined (optional).
58
+ **reason** (`string`): Explanation for the score, if generateReason step was defined (optional).
59
59
 
60
- **preprocessStepResult:** (`any`): Result of the preprocess step, if defined (optional).
60
+ **preprocessStepResult** (`any`): Result of the preprocess step, if defined (optional).
61
61
 
62
- **analyzeStepResult:** (`any`): Result of the analyze step, if defined (optional).
62
+ **analyzeStepResult** (`any`): Result of the analyze step, if defined (optional).
63
63
 
64
- **preprocessPrompt:** (`string`): Preprocess prompt, if defined (optional).
64
+ **preprocessPrompt** (`string`): Preprocess prompt, if defined (optional).
65
65
 
66
- **analyzePrompt:** (`string`): Analyze prompt, if defined (optional).
66
+ **analyzePrompt** (`string`): Analyze prompt, if defined (optional).
67
67
 
68
- **generateScorePrompt:** (`string`): Generate score prompt, if defined (optional).
68
+ **generateScorePrompt** (`string`): Generate score prompt, if defined (optional).
69
69
 
70
- **generateReasonPrompt:** (`string`): Generate reason prompt, if defined (optional).
70
+ **generateReasonPrompt** (`string`): Generate reason prompt, if defined (optional).
71
71
 
72
72
  ## Step Execution Flow
73
73
 
@@ -13,9 +13,9 @@ Before using the noise sensitivity scorer, prepare your test data:
13
13
 
14
14
  ## Parameters
15
15
 
16
- **model:** (`MastraModelConfig`): The language model to use for evaluating noise sensitivity
16
+ **model** (`MastraModelConfig`): The language model to use for evaluating noise sensitivity
17
17
 
18
- **options:** (`NoiseSensitivityOptions`): Configuration options for the scorer
18
+ **options** (`NoiseSensitivityOptions`): Configuration options for the scorer
19
19
 
20
20
  ## CI/Testing Requirements
21
21
 
@@ -82,9 +82,9 @@ describe('Agent Noise Resistance Tests', () => {
82
82
 
83
83
  ## .run() Returns
84
84
 
85
- **score:** (`number`): Robustness score between 0 and 1 (1.0 = completely robust, 0.0 = severely compromised)
85
+ **score** (`number`): Robustness score between 0 and 1 (1.0 = completely robust, 0.0 = severely compromised)
86
86
 
87
- **reason:** (`string`): Human-readable explanation of how noise affected the agent's response
87
+ **reason** (`string`): Human-readable explanation of how noise affected the agent's response
88
88
 
89
89
  ## Evaluation Dimensions
90
90
 
@@ -4,15 +4,15 @@ The `createPromptAlignmentScorerLLM()` function creates a scorer that evaluates
4
4
 
5
5
  ## Parameters
6
6
 
7
- **model:** (`MastraModelConfig`): The language model to use for evaluating prompt-response alignment
7
+ **model** (`MastraModelConfig`): The language model to use for evaluating prompt-response alignment
8
8
 
9
- **options:** (`PromptAlignmentOptions`): Configuration options for the scorer
9
+ **options** (`PromptAlignmentOptions`): Configuration options for the scorer
10
10
 
11
11
  ## .run() Returns
12
12
 
13
- **score:** (`number`): Multi-dimensional alignment score between 0 and scale (default 0-1)
13
+ **score** (`number`): Multi-dimensional alignment score between 0 and scale (default 0-1)
14
14
 
15
- **reason:** (`string`): Human-readable explanation of the prompt alignment evaluation with detailed breakdown
15
+ **reason** (`string`): Human-readable explanation of the prompt alignment evaluation with detailed breakdown
16
16
 
17
17
  `.run()` returns a result in the following shape:
18
18