@mastra/core 1.16.0-alpha.5 → 1.16.0

package/CHANGELOG.md

@@ -1,5 +1,134 @@
 # @mastra/core
 
+## 1.16.0
+
+### Minor Changes
+
+- Added dataset-agent association and experiment status tracking for the Evaluate workflow. ([#14470](https://github.com/mastra-ai/mastra/pull/14470))
+  - **Dataset targeting**: Added `targetType` and `targetIds` fields to datasets, enabling association with agents, scorers, or workflows. Datasets can now be linked to multiple entities.
+  - **Experiment status**: Added `status` field to experiment results (`'needs-review'`, `'reviewed'`, `'complete'`) for review queue workflow.
+  - **Dataset experiment routes**: Added API endpoints for triggering experiments from a dataset with configurable target type and target ID.
+  - **LLM data generation**: Added endpoint for generating dataset items using an LLM with configurable count and prompt.
+  - **Failure analysis**: Added endpoint for clustering experiment failures and proposing tags using LLM analysis.
+
+- Added agent version support for experiments. When triggering an experiment, you can now pass an `agentVersion` parameter to pin which agent version to use. The agent version is stored with the experiment and returned in experiment responses. ([#14562](https://github.com/mastra-ai/mastra/pull/14562))
+
+  ```ts
+  const client = new MastraClient();
+
+  await client.triggerDatasetExperiment({
+    datasetId: 'my-dataset',
+    targetType: 'agent',
+    targetId: 'my-agent',
+    version: 3, // pin to dataset version 3
+    agentVersion: 'ver_abc123', // pin to a specific agent version
+  });
+  ```
+
+- Added tool suspension handling to the Harness. ([#14611](https://github.com/mastra-ai/mastra/pull/14611))
+
+  When a tool calls `suspend()` during execution, the harness now emits a `tool_suspended` event, reports `agent_end` with reason `'suspended'`, and exposes `respondToToolSuspension()` to resume execution with user-provided data.
+
+  ```ts
+  harness.subscribe(event => {
+    if (event.type === 'tool_suspended') {
+      // event.toolName, event.suspendPayload, event.resumeSchema
+    }
+  });
+
+  // Resume after collecting user input
+  await harness.respondToToolSuspension({ resumeData: { confirmed: true } });
+  ```
+
+- Added `agentId` to the agent tool execution context. Tools executed by an agent can now access `context.agent.agentId` to identify which agent is calling them. This enables tools to look up agent metadata, share workspace configuration with sub-agents, or customize behavior per agent. ([#14502](https://github.com/mastra-ai/mastra/pull/14502))
+
+- Improved observability metrics and logs storage support. ([#14607](https://github.com/mastra-ai/mastra/pull/14607))
+  - Added typed observability storage fields for shared correlation context and cost data.
+  - Added storage-layer metric listing and richer metric aggregations that can return estimated cost alongside values.
+  - Improved observability filter parity across log and metric storage APIs.
+
+- Add optional `?path=` query param to workspace skill routes for disambiguating same-named skills. ([#14430](https://github.com/mastra-ai/mastra/pull/14430))
+
+  Skill routes continue to use `:skillName` in the URL path (no breaking change). When two skills share the same name (e.g. from different directories), pass the optional `?path=` query parameter to select the exact skill:
+
+  ```
+  GET /workspaces/:workspaceId/skills/:skillName?path=skills/brand-guidelines
+  ```
+
+  `SkillMetadata` now includes a `path` field, and the `list()` method returns all same-named skills for disambiguation. The client SDK's `getSkill()` accepts an optional `skillPath` parameter for disambiguation.
+
+- Added `ModelByInputTokens` in `@mastra/memory` for token-threshold-based model selection in Observational Memory. ([#14614](https://github.com/mastra-ai/mastra/pull/14614))
+
+  When configured, OM automatically selects different observer or reflector models based on the actual input token count at the time the OM call runs.
+
+  Example usage:
+
+  ```ts
+  import { Memory, ModelByInputTokens } from '@mastra/memory';
+
+  const memory = new Memory({
+    options: {
+      observationalMemory: {
+        model: new ModelByInputTokens({
+          upTo: {
+            10_000: 'google/gemini-2.5-flash',
+            40_000: 'openai/gpt-4o',
+            1_000_000: 'openai/gpt-4.5',
+          },
+        }),
+      },
+    },
+  });
+  ```
+
+  The `upTo` keys are inclusive upper bounds. OM resolves the matching tier directly at the observer or reflector call site. If the input exceeds the largest configured threshold, OM throws an error.
+
+  Improved Observational Memory tracing so traces show the observer and reflector spans and make it easier to see which resolved model was used at runtime.
+
+### Patch Changes
+
+- Update provider registry and model documentation with latest models and providers ([`68ed4e9`](https://github.com/mastra-ai/mastra/commit/68ed4e9f118e8646b60a6112dabe854d0ef53902))
+
+- Fixed `Harness.destroy()` to properly clean up heartbeats and workspace on teardown. ([#14568](https://github.com/mastra-ai/mastra/pull/14568))
+
+- Fixed null detection in tool input validation to check actual values at failing paths instead of relying on error message string matching. This ensures null values from LLMs are correctly handled even when validators produce error messages that don't contain the word "null" (e.g., "must be string"). Fixes #14476. ([#14496](https://github.com/mastra-ai/mastra/pull/14496))
+
+- Fixed missing tool lists in agent traces for streaming runs. Exporters like Datadog LLM Observability now receive the tools available to the agent. ([#14550](https://github.com/mastra-ai/mastra/pull/14550))
+
+- Fix consecutive tool-only loop iterations being merged into a single assistant message block. When the agentic loop runs multiple iterations that each produce only tool calls, the LLM would misinterpret them as parallel calls from a single turn. A `step-start` boundary is now inserted between iterations to ensure they are treated as sequential steps. ([#14652](https://github.com/mastra-ai/mastra/pull/14652))
+
+- Improved custom OpenAI-compatible model configuration guidance in the models docs. ([#14594](https://github.com/mastra-ai/mastra/pull/14594))
+
+- Added client/server body schemas for feedback and scores that omit the timestamp field, allowing it to be set server-side ([#14470](https://github.com/mastra-ai/mastra/pull/14470))
+
+- **Workspace skills now surface all same-named skills for disambiguation.** ([#14430](https://github.com/mastra-ai/mastra/pull/14430))
+
+  When multiple skills share the same name (e.g., a local `brand-guidelines` skill and one from `node_modules`), `list()` now returns all of them instead of only the tie-break winner. This lets agents and UIs see every available skill, along with its path and source type.
+
+  **Tie-breaking behavior:**
+  - `get(name)` still returns a single skill using source-type priority: local > managed > external
+  - If two skills share the same name _and_ source type, `get(name)` throws an error — rename one or move it to a different source type
+  - `get(path)` bypasses tie-breaking entirely and returns the exact skill
+
+  Agents and UIs now receive all same-named skills with their paths, which improves disambiguation in prompts and tool calls.
+
+  ```ts
+  const skills = await workspace.skills.list();
+  // Returns both local and external "brand-guidelines" skills
+
+  const exact = await workspace.skills.get('node_modules/@myorg/skills/brand-guidelines');
+  // Fetches the external copy directly by path
+  ```
+
+- Fixed Anthropic 'tool_use ids were found without tool_result blocks immediately after' error. When client tools (e.g. execute_command) and provider tools (e.g. web_search) are called in parallel, the tool ordering in message history could cause Anthropic to reject subsequent requests, making the thread unrecoverable. Tool blocks are now correctly split to satisfy Anthropic's ordering requirements. ([#14648](https://github.com/mastra-ai/mastra/pull/14648))
+
+- Fix Zod v3 and Zod v4 compatibility across public structured-output APIs. ([#14464](https://github.com/mastra-ai/mastra/pull/14464))
+
+  Mastra agent and client APIs accept schemas from either `zod/v3` or `zod/v4`, matching the documented peer dependency range and preserving TypeScript compatibility for both Zod versions.
+
+- Updated dependencies [[`47358d9`](https://github.com/mastra-ai/mastra/commit/47358d960bb2b931321de7e798f341ab0df81f44), [`d3930ea`](https://github.com/mastra-ai/mastra/commit/d3930eac51c30b0ecf7eaa54bb9430758b399777)]:
+  - @mastra/schema-compat@1.2.7
+
 ## 1.16.0-alpha.5
 
 ### Patch Changes

package/dist/agent/index.cjs

@@ -1,37 +1,37 @@
 'use strict';
 
-var chunkALCZ3AX7_cjs = require('../chunk-ALCZ3AX7.cjs');
+var chunkTP4WRFPO_cjs = require('../chunk-TP4WRFPO.cjs');
 var chunkANLO4VM2_cjs = require('../chunk-ANLO4VM2.cjs');
 
 
 
 Object.defineProperty(exports, "Agent", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.Agent; }
+  get: function () { return chunkTP4WRFPO_cjs.Agent; }
 });
 Object.defineProperty(exports, "TripWire", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.TripWire; }
+  get: function () { return chunkTP4WRFPO_cjs.TripWire; }
 });
 Object.defineProperty(exports, "isSupportedLanguageModel", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.isSupportedLanguageModel; }
+  get: function () { return chunkTP4WRFPO_cjs.isSupportedLanguageModel; }
 });
 Object.defineProperty(exports, "resolveThreadIdFromArgs", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.resolveThreadIdFromArgs; }
+  get: function () { return chunkTP4WRFPO_cjs.resolveThreadIdFromArgs; }
 });
 Object.defineProperty(exports, "supportedLanguageModelSpecifications", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.supportedLanguageModelSpecifications; }
+  get: function () { return chunkTP4WRFPO_cjs.supportedLanguageModelSpecifications; }
 });
 Object.defineProperty(exports, "tryGenerateWithJsonFallback", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.tryGenerateWithJsonFallback; }
+  get: function () { return chunkTP4WRFPO_cjs.tryGenerateWithJsonFallback; }
 });
 Object.defineProperty(exports, "tryStreamWithJsonFallback", {
   enumerable: true,
-  get: function () { return chunkALCZ3AX7_cjs.tryStreamWithJsonFallback; }
+  get: function () { return chunkTP4WRFPO_cjs.tryStreamWithJsonFallback; }
 });
 Object.defineProperty(exports, "MessageList", {
   enumerable: true,

package/dist/agent/index.js

@@ -1,4 +1,4 @@
-export { Agent, TripWire, isSupportedLanguageModel, resolveThreadIdFromArgs, supportedLanguageModelSpecifications, tryGenerateWithJsonFallback, tryStreamWithJsonFallback } from '../chunk-KXRAP65E.js';
+export { Agent, TripWire, isSupportedLanguageModel, resolveThreadIdFromArgs, supportedLanguageModelSpecifications, tryGenerateWithJsonFallback, tryStreamWithJsonFallback } from '../chunk-43Z3JRGQ.js';
 export { MessageList, TypeDetector, aiV5ModelMessageToV2PromptMessage, convertMessages } from '../chunk-SLZ3WO42.js';
 //# sourceMappingURL=index.js.map
 //# sourceMappingURL=index.js.map

package/dist/{chunk-7BZCZH6B.js → chunk-23RPY53R.js}

@@ -1,4 +1,4 @@
-import { isSupportedLanguageModel, Workflow } from './chunk-KXRAP65E.js';
+import { isSupportedLanguageModel, Workflow } from './chunk-43Z3JRGQ.js';
 import { saveScorePayloadSchema } from './chunk-SNASZNHI.js';
 import { resolveObservabilityContext } from './chunk-CT4YYQI3.js';
 import { MastraError } from './chunk-FJEVLHJT.js';
@@ -625,5 +625,5 @@ async function saveSingleScore({
 }
 
 export { createOnScorerHook, runEvals, validateAndSaveScore };
-//# sourceMappingURL=chunk-7BZCZH6B.js.map
-//# sourceMappingURL=chunk-7BZCZH6B.js.map
+//# sourceMappingURL=chunk-23RPY53R.js.map
+//# sourceMappingURL=chunk-23RPY53R.js.map

package/dist/{chunk-7BZCZH6B.js.map → chunk-23RPY53R.js.map}

@@ -1 +1 @@
-{"version":3,"sources":["../src/mastra/hooks.ts","../src/evals/run/scorerAccumulator.ts","../src/evals/run/index.ts"],"names":["pMap"],"mappings":";;;;;;AAOO,SAAS,mBAAmB,MAAA,EAAgB;AACjD,EAAA,OAAO,OAAO,QAAA,KAA+B;AAC3C,IAAA,MAAM,OAAA,GAAU,OAAO,UAAA,EAAW;AAElC,IAAA,IAAI,CAAC,OAAA,EAAS;AACZ,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,IAAA,CAAK,yDAAyD,CAAA;AAClF,MAAA;AAAA,IACF;AAEA,IAAA,MAAM,QAAA,GAAW,SAAS,MAAA,CAAO,EAAA;AACjC,IAAA,MAAM,aAAa,QAAA,CAAS,UAAA;AAC5B,IAAA,MAAM,SAAS,QAAA,CAAS,MAAA;AACxB,IAAA,MAAM,WAAW,MAAA,CAAO,EAAA;AAExB,IAAA,IAAI,CAAC,QAAA,EAAU;AACb,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,IAAA,CAAK,2DAA2D,CAAA;AACpF,MAAA;AAAA,IACF;AAEA,IAAA,IAAI;AACF,MAAA,MAAM,cAAc,MAAM,UAAA,CAAW,MAAA,EAAQ,QAAA,EAAU,YAAY,QAAQ,CAAA;AAE3E,MAAA,IAAI,CAAC,WAAA,EAAa;AAChB,QAAA,MAAM,IAAI,WAAA,CAAY;AAAA,UACpB,EAAA,EAAI,yBAAA;AAAA,UACJ,MAAA,EAAA,QAAA;AAAA,UACA,QAAA,EAAA,MAAA;AAAA,UACA,IAAA,EAAM,kBAAkB,QAAQ,CAAA,UAAA;AAAA,SACjC,CAAA;AAAA,MACH;AAEA,MAAA,IAAI,QAAQ,QAAA,CAAS,KAAA;AACrB,MAAA,IAAI,SAAS,QAAA,CAAS,MAAA;AAEtB,MAAA,MAAM,EAAE,gBAAA,EAAkB,GAAG,IAAA,EAAK,GAAI,QAAA;AAEtC,MAAA,MAAM,SAAA,GAAY,MAAM,WAAA,CAAY,MAAA,CAAO,GAAA,CAAI;AAAA,QAC7C,GAAG,IAAA;AAAA,QACH,KAAA;AAAA,QACA;AAAA,OACD,CAAA;AAED,MAAA,IAAI,MAAA;AACJ,MAAA,IAAI,OAAA;AACJ,MAAA,MAAM,WAAA,GAAc,SAAS,cAAA,EAAgB,WAAA;AAC7C,MAAA,IAAI,WAAA,IAAe,YAAY,OAAA,EAAS;AACtC,QAAA,MAAA,GAAS,WAAA,CAAY,EAAA;AACrB,QAAA,OAAA,GAAU,WAAA,CAAY,OAAA;AAAA,MACxB;AAEA,MAAA,MAAM,OAAA,GAAU;AAAA,QACd,GAAG,IAAA;AAAA,QACH,GAAG,SAAA;AAAA,QACH,QAAA;AAAA,QACA,QAAA;AAAA,QACA,MAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA,EAAQ;AAAA,UACN,GAAG,IAAA,CAAK,MAAA;AAAA,UACR,QAAA,EAAU,CAAC,CAAC,WAAA,CAAY,MAAA,CAAO;AAAA,SACjC;AAAA,QACA,QAAA,EAAU;AAAA,UACR,gBAAA,EAAkB,CAAC,CAAC;AAAA;AACtB,OACF;AACA,MAAA,MAAM,oBAAA,CAAqB,SAAS,OAAO,CAAA;AAE3C,MAAA,IAAI,WAAA,IAAe,UAAU,OAAA,EAAS;AACpC,QAAA,MAAM,IAAA;AAAA,UACJ,WAAA,CAAY,sBAAsB,YAAA,EAAa;AAAA,UAC/C,OAAM,QAAA,KAAY;AAChB,YAAA,IAAI,SAAS,eAAA,EAAiB;AAC5B,cAAA,IAAI;AACF,gBAAA,MAAM,SAAS,eAAA,CAAgB;AAAA,kBAC7B,OAAA;AAAA,kBACA,MAAA;AAAA,kBACA,OAAO,SAAA,CAAU,KAAA;AAAA,kBACjB,QAAQ,SAAA,CAAU,MAAA;AAAA,kBAClB,UAAA,EAAY,YAAY,MAAA,CAAO,EAAA;AAAA,kBAC/B,QAAA,EAAU;AAAA,oBACR,GAAI,WAAA,CAAY,QAAA,IAAY;AAAC;AAC/B,iBACD,CAAA;AAAA,cACH,SAAS,KAAA,EAAO;AAEd,gBAAA,MAAA,CAAO,SAAA,EAAU,EAAG,KAAA,CAAM,CAAA,2CAAA,EAA8C,KAAK,CAAA,CAAE,CAAA;AAAA,cACjF;AAAA,YACF;AAAA,UACF,CAAA;AAAA,UACA,EAAE,aAAa,CAAA;AAAE,SACnB;AAAA,MACF;AAAA,IACF,SAAS,KAAA,EAAO;AACd,MAAA,MAAM,cAAc,IAAI,WAAA;AAAA,QACtB;AAAA,UACE,EAAA,EAAI,kCAAA;AAAA,UACJ,MAAA,EAAA,QAAA;AAAA,UACA,QAAA,EAAA,MAAA;AAAA,UACA,OAAA,EAAS;AAAA,YACP,QAAA;AAAA,YACA,QAAA;AAAA,YACA;AAAA;AACF,SACF;AAAA,QACA;AAAA,OACF;AAEA,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,cAAA,CAAe,WAAW,CAAA;AAC9C,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,KAAA,CAAM,WAAA,CAAY,UAAU,CAAA;AAAA,IAClD;AAAA,EACF,CAAA;AACF;AAEA,eAAsB,oBAAA,CAAqB,SAAwB,OAAA,EAAkB;AACnF,EAAA,MAAM,WAAA,GAAc,MAAM,OAAA,CAAQ,QAAA,CAAS,QAAQ,CAAA;AACnD,EAAA,IAAI,CAAC,WAAA,EAAa;AAChB,IAAA,MAAM,IAAI,WAAA,CAAY;AAAA,MACpB,EAAA,EAAI,qCAAA;AAAA,MACJ,MAAA,EAAA,SAAA;AAAA,MACA,QAAA,EAAA,QAAA;AAAA,MACA,IAAA,EAAM;AAAA,KACP,CAAA;AAAA,EACH;AACA,EAAA,MAAM,aAAA,GAAgB,sBAAA,CAAuB,KAAA,CAAM,OAAO,CAAA;AAC1D,EAAA,MAAM,WAAA,CAAY,UAAU,aAAa,CAAA;AAC3C;AAEA,eAAe,UAAA,CAAW,MAAA,EAAgB,QAAA,EAAkB,UAAA,EAAoB,QAAA,EAAkB;AAChG,EAAA,IAAI,WAAA;AACJ,EAAA,IAAI,eAAe,OAAA,EAAS;AAE1B,IAAA,IAAI;AACF,MAAA,MAAM,KAAA,GAAQ,MAAA,CAAO,YAAA,CAAa,QAAQ,CAAA;AAC1C,MAAA,MAAM,OAAA,GAAU,MAAM,KAAA,CAAM,WAAA,EAAY;AACxC,MAAA,KAAA,MAAW,CAAC,CAAA,EAAG,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAO,CAAA,EAAG;AACjD,QAAA,IAAI,MAAA,CAAO,MAAA,CAAO,EAAA,KAAO,QAAA,EAAU;AACjC,UAAA,WAAA,GAAc,MAAA;AACd,UAAA;AAAA,QACF;AAAA,MACF;AAAA,IACF,CAAA,CAAA,MAAQ;AAEN,MAAA,IAAI;AACF,QAAA,MAAM,WAAA,GAAe,MAAM,MAAA,CAAO,SAAA,IAAa,KAAA,CAAM,OAAA,CAAQ,QAAQ,CAAA,IAAM,IAAA;AAC3E,QAAA,IAAI,WAAA,EAAa;AACf,UAAA,MAAM,OAAA,GAAU,MAAM,WAAA,CAAY,WAAA,EAAY;AAC9C,UAAA,KAAA,MAAW,CAAC,CAAA,EAAG,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAO,CAAA,EAAsB;AACpE,YAAA,IAAI,MAAA,CAAO,MAAA,CAAO,EAAA,KAAO,QAAA,EAAU;AACjC,cAAA,WAAA,GAAc,MAAA;AACd,cAAA;AAAA,YACF;AAAA,UACF;AAAA,QACF;AAAA,MACF,CAAA,CAAA,MAAQ;AAAA,MAER;AAAA,IACF;AAAA,EACF,CAAA,MAAA,IAAW,eAAe,UAAA,EAAY;AACpC,IAAA,MAAM,UAAU,MAAM,MAAA,CAAO,eAAA,CAAgB,QAAQ,EAAE,WAAA,EAAY;AACnE,IAAA,KAAA,MAAW,CAAC,CAAA,EAAG,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAO,CAAA,EAAG;AACjD,MAAA,IAAI,MAAA,CAAO,MAAA,CAAO,EAAA,KAAO,QAAA,EAAU;AACjC,QAAA,WAAA,GAAc,MAAA;AACd,QAAA;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAGA,EAAA,IAAI,CAAC,WAAA,EAAa;AAChB,IAAA,MAAM,sBAAA,GAAyB,MAAA,CAAO,aAAA,CAAc,QAAQ,CAAA;AAC5D,IAAA,WAAA,GAAc,sBAAA,GAAyB,EAAE,MAAA,EAAQ,sBAAA,EAAuB,GAAI,MAAA;AAAA,EAC9E;AAEA,EAAA,OAAO,WAAA;AACT;;;ACrLO,IAAM,mBAAN,MAAuB;AAAA,EACpB,aAAuC,EAAC;AAAA,EACxC,iBAA2C,EAAC;AAAA,EAC5C,aAAuD,EAAC;AAAA,EAEhE,UAAU,aAAA,EAAoC;AAC5C,IAAA,MAAM,mCAAmC,OAAA,IAAW,aAAA;AACpD,IAAA,IAAI,gCAAA,EAAkC;AACpC,MAAA,IAAA,CAAK,gBAAgB,aAAa,CAAA;AAAA,IACpC,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,cAAc,aAAa,CAAA;AAAA,IAClC;AAAA,EACF;AAAA,EAEQ,cAAc,aAAA,EAAoC;AACxD,IAAA,KAAA,MAAW,CAAC,UAAA,EAAY,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,aAAa,CAAA,EAAG;AAChE,MAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,UAAU,CAAA,EAAG;AAChC,QAAA,IAAA,CAAK,UAAA,CAAW,UAAU,CAAA,GAAI,EAAC;AAAA,MACjC;AACA,MAAA,IAAA,CAAK,UAAA,CAAW,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,IACtE;AAAA,EACF;AAAA,EAEQ,gBAAgB,aAAA,EAAoC;AAC1D,IAAA,IAAI,UAAA,IAAc,aAAA,IAAiB,aAAA,CAAc,QAAA,EAAU;AACzD,MAAA,KAAA,MAAW,CAAC,YAAY,MAAM,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,QAAQ,CAAA,EAAG;AACzE,QAAA,IAAI,CAAC,IAAA,CAAK,cAAA,CAAe,UAAU,CAAA,EAAG;AACpC,UAAA,IAAA,CAAK,cAAA,CAAe,UAAU,CAAA,GAAI,EAAC;AAAA,QACrC;AACA,QAAA,IAAA,CAAK,cAAA,CAAe,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,MAC1E;AAAA,IACF;AAEA,IAAA,IAAI,OAAA,IAAW,aAAA,IAAiB,aAAA,CAAc,KAAA,EAAO;AACnD,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,KAAK,CAAA,EAAG;AACvE,QAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,EAAG;AAC5B,UAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,GAAI,EAAC;AAAA,QAC7B;AACA,QAAA,KAAA,MAAW,CAAC,UAAA,EAAY,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAkC,CAAA,EAAG;AACrF,UAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,CAAA,EAAG;AACxC,YAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,IAAI,EAAC;AAAA,UACzC;AACA,UAAA,IAAA,CAAK,WAAW,MAAM,CAAA,CAAE,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,QAC9E;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAAA,EAEA,cAAc,iBAAA,EAAwD;AACpE,IAAA,KAAA,MAAW,CAAC,MAAA,EAAQ,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,iBAAiB,CAAA,EAAG;AACrE,MAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,EAAG;AAC5B,QAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,GAAI,EAAC;AAAA,MAC7B;AACA,MAAA,KAAA,MAAW,CAAC,UAAA,EAAY,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAW,CAAA,EAAG;AAC9D,QAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,CAAA,EAAG;AACxC,UAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,IAAI,EAAC;AAAA,QACzC;AACA,QAAA,IAAA,CAAK,WAAW,MAAM,CAAA,CAAE,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,MAC9E;AAAA,IACF;AAAA,EACF;AAAA,EAEA,gBAAA,GAAwC;AACtC,IAAA,MAAM,SAA8B,EAAC;AAErC,IAAA,KAAA,MAAW,CAAC,YAAY,UAAU,CAAA,IAAK,OAAO,OAAA,CAAQ,IAAA,CAAK,UAAU,CAAA,EAAG;AACtE,MAAA,MAAA,CAAO,UAAU,CAAA,GAAI,IAAA,CAAK,eAAA,CAAgB,UAAU,CAAA;AAAA,IACtD;AAGA,IAAA,IAAI,OAAO,IAAA,CAAK,IAAA,CAAK,cAAc,CAAA,CAAE,SAAS,CAAA,EAAG;AAC/C,MAAA,MAAA,CAAO,WAAW,EAAC;AACnB,MAAA,KAAA,MAAW,CAAC,YAAY,UAAU,CAAA,IAAK,OAAO,OAAA,CAAQ,IAAA,CAAK,cAAc,CAAA,EAAG;AAC1E,QAAA,MAAA,CAAO,QAAA,CAAS,UAAU,CAAA,GAAI,IAAA,CAAK,gBAAgB,UAAU,CAAA;AAAA,MAC/D;AAAA,IACF;AAEA,IAAA,IAAI,OAAO,IAAA,CAAK,IAAA,CAAK,UAAU,CAAA,CAAE,SAAS,CAAA,EAAG;AAC3C,MAAA,MAAA,CAAO,QAAQ,EAAC;AAChB,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,IAAA,CAAK,UAAU,CAAA,EAAG;AACnE,QAAA,MAAA,CAAO,KAAA,CAAM,MAAM,CAAA,GAAI,EAAC;AACxB,QAAA,KAAA,MAAW,CAAC,UAAA,EAAY,UAAU,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAW,CAAA,EAAG;AAClE,UAAA,MAAA,CAAO,MAAM,MAAM,CAAA,CAAE,UAAU,CAAA,GAAI,IAAA,CAAK,gBAAgB,UAAU,CAAA;AAAA,QACpE;AAAA,MACF;AAAA,IACF;AAEA,IAAA,OAAO,MAAA;AAAA,EACT;AAAA,EAEQ,gBAAgB,UAAA,EAA8B;AACpD,IAAA,IAAI,UAAA,CAAW,SAAS,CAAA,EAAG;AACzB,MAAA,OAAO,UAAA,CAAW,OAAO,CAAC,CAAA,EAAG,MAAM,CAAA,GAAI,CAAA,EAAG,CAAC,CAAA,GAAI,UAAA,CAAW,MAAA;AAAA,IAC5D,CAAA,MAAO;AACL,MAAA,OAAO,CAAA;AAAA,IACT;AAAA,EACF;AACF,CAAA;;;ACbA,eAAsB,SAAS,MAAA,EAaH;AAC1B,EAAA,MAAM,EAAE,MAAM,OAAA,EAAS,MAAA,EAAQ,eAAe,cAAA,EAAgB,WAAA,GAAc,GAAE,GAAI,MAAA;AAElF,EAAA,mBAAA,CAAoB,IAAA,EAAM,SAAS,MAAM,CAAA;AAEzC,EAAA,IAAI,UAAA,GAAa,CAAA;AACjB,EAAA,MAAM,gBAAA,GAAmB,IAAI,gBAAA,EAAiB;AAI9C,EAAA,MAAM,MAAA,GAAU,MAAA,CAAe,iBAAA,IAAoB,IAAM,MAAA,CAAe,MAAA;AACxE,EAAA,MAAM,OAAA,GAAU,QAAQ,UAAA,EAAW;AAEnC,EAAA,MAAMA,KAAAA,GAAAA,CAAQ,MAAM,OAAO,OAAO,CAAA,EAAG,OAAA;AACrC,EAAA,MAAMA,KAAAA;AAAA,IACJ,IAAA;AAAA,IACA,OAAO,IAAA,KAAgC;AACrC,MAAA,MAAM,YAAA,GAAe,MAAM,aAAA,CAAc,MAAA,EAAQ,MAAM,aAAa,CAAA;AACpE,MAAA,MAAM,aAAA,GAAgB,MAAM,UAAA,CAAW,OAAA,EAAS,cAAc,IAAI,CAAA;AAClE,MAAA,gBAAA,CAAiB,UAAU,aAAa,CAAA;AAGxC,MAAA,IAAI,OAAA,EAAS;AACX,QAAA,MAAM,mBAAA,CAAoB;AAAA,UACxB,OAAA;AAAA,UACA,aAAA;AAAA,UACA,MAAA;AAAA,UACA,IAAA;AAAA,UACA;AAAA,SACD,CAAA;AAAA,MACH;AAEA,MAAA,IAAI,cAAA,EAAgB;AAClB,QAAA,MAAM,cAAA,CAAe;AAAA,UACnB,IAAA;AAAA,UACA,YAAA;AAAA,UACA;AAAA,SACD,CAAA;AAAA,MACH;AAEA,MAAA,UAAA,EAAA;AAAA,IACF,CAAA;AAAA,IACA,EAAE,WAAA;AAAY,GAChB;AAEA,EAAA,OAAO;AAAA,IACL,MAAA,EAAQ,iBAAiB,gBAAA,EAAiB;AAAA,IAC1C,OAAA,EAAS;AAAA,MACP;AAAA;AACF,GACF;AACF;AAEA,SAAS,WAAW,MAAA,EAA8C;AAChE,EAAA,OAAO,MAAA,YAAkB,QAAA;AAC3B;AAEA,SAAS,uBAAuB,OAAA,EAA+C;AAC7E,EAAA,OAAO,OAAO,OAAA,KAAY,QAAA,IAAY,CAAC,KAAA,CAAM,QAAQ,OAAO,CAAA,KAAM,UAAA,IAAc,OAAA,IAAW,OAAA,IAAW,OAAA,CAAA;AACxG;AAEA,SAAS,mBAAA,CACP,IAAA,EACA,OAAA,EACA,MAAA,EACM;AACN,EAAA,IAAI,IAAA,CAAK,WAAW,CAAA,EAAG;AACrB,IAAA,MAAM,IAAI,WAAA,CAAY;AAAA,MACpB,MAAA,EAAQ,QAAA;AAAA,MACR,EAAA,EAAI,wCAAA;AAAA,MACJ,QAAA,EAAU,MAAA;AAAA,MACV,IAAA,EAAM;AAAA,KACP,CAAA;AAAA,EACH;AAEA,EAAA,KAAA,IAAS,CAAA,GAAI,CAAA,EAAG,CAAA,GAAI,IAAA,CAAK,QAAQ,CAAA,EAAA,EAAK;AACpC,IAAA,MAAM,IAAA,GAAO,KAAK,CAAC,CAAA;AACnB,IAAA,IAAI,CAAC,IAAA,IAAQ,OAAO,SAAS,QAAA,IAAY,EAAE,WAAW,IAAA,CAAA,EAAO;AAC3D,MAAA,MAAM,IAAI,WAAA,CAAY;AAAA,QACpB,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,mBAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM,8BAA8B,CAAC,CAAA,8BAAA;AAAA,OACtC,CAAA;AAAA,IACH;AAAA,EACF;AAGA,EAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AAC1B,IAAA,IAAI,OAAA,CAAQ,WAAW,CAAA,EAAG;AACxB,MAAA,MAAM,IAAI,WAAA,CAAY;AAAA,QACpB,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,qBAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM;AAAA,OACP,CAAA;AAAA,IACH;AAAA,EACF,WAAW,UAAA,CAAW,MAAM,CAAA,IAAK,sBAAA,CAAuB,OAAO,CAAA,EAAG;AAChE,IAAA,MAAM,UAAA,GACH,OAAA,CAAQ,QAAA,IAAY,OAAA,CAAQ,SAAS,MAAA,GAAS,CAAA,IAAO,OAAA,CAAQ,KAAA,IAAS,MAAA,CAAO,IAAA,CAAK,OAAA,CAAQ,KAAK,EAAE,MAAA,GAAS,CAAA;AAE7G,IAAA,IAAI,CAAC,UAAA,EAAY;AACf,MAAA,MAAM,IAAI,WAAA,CAAY;AAAA,QACpB,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,qBAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM;AAAA,OACP,CAAA;AAAA,IACH;AAAA,EACF,CAAA,MAAA,IAAW,CAAC,UAAA,CAAW,MAAM,KAAK,CAAC,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AACzD,IAAA,MAAM,IAAI,WAAA,CAAY;AAAA,MACpB,MAAA,EAAQ,QAAA;AAAA,MACR,EAAA,EAAI,uBAAA;AAAA,MACJ,QAAA,EAAU,MAAA;AAAA,MACV,IAAA,EAAM;AAAA,KACP,CAAA;AAAA,EACH;AACF;AAEA,eAAe,aAAA,CACb,MAAA,EACA,IAAA,EACA,aAAA,EAGA;AACA,EAAA,IAAI;AACF,IAAA,IAAI,UAAA,CAAW,MAAM,CAAA,EAAG;AACtB,MAAA,OAAO,MAAM,eAAA,CAAgB,MAAA,EAAQ,IAAA,EAAM,aAAmC,CAAA;AAAA,IAChF,CAAA,MAAO;AACL,MAAA,OAAO,MAAM,YAAA;AAAA,QACX,MAAA;AAAA,QACA,IAAA;AAAA,QACA;AAAA,OACF;AAAA,IACF;AAAA,EACF,SAAS,KAAA,EAAO;AACd,IAAA,MAAM,IAAI,WAAA;AAAA,MACR;AAAA,QACE,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,iDAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM,+DAAA;AAAA,QACN,OAAA,EAAS;AAAA,UACP,IAAA,EAAM,IAAA,CAAK,SAAA,CAAU,IAAI;AAAA;AAC3B,OACF;AAAA,MACA;AAAA,KACF;AAAA,EACF;AACF;AAEA,eAAe,eAAA,CAAgB,MAAA,EAAkB,IAAA,EAA6B,aAAA,EAAoC;AAChH,EAAA,MAAM,oBAAA,GAAuB,4BAA4B,IAAI,CAAA;AAC7D,EAAA,MAAM,MAAM,MAAM,MAAA,CAAO,UAAU,EAAE,cAAA,EAAgB,MAAM,CAAA;AAC3D,EAAA,MAAM,cAAA,GAAiB,MAAM,GAAA,CAAI,KAAA,CAAM;AAAA,IACrC,GAAG,aAAA;AAAA,IACH,GAAG,IAAA,CAAK,YAAA;AAAA,IACR,WAAW,IAAA,CAAK,KAAA;AAAA,IAChB,gBAAgB,IAAA,CAAK,cAAA;AAAA,IACrB,GAAG;AAAA,GACJ,CAAA;AAED,EAAA,OAAO;AAAA,IACL,WAAA,EAAa;AAAA,MACX,OAAO,IAAA,CAAK,KAAA;AAAA,MACZ,MAAA,EAAQ,cAAA,CAAe,MAAA,KAAW,SAAA,GAAY,eAAe,MAAA,GAAS,MAAA;AAAA,MACtE,aAAa,cAAA,CAAe;AAAA;AAC9B,GACF;AACF;AAEA,eAAe,YAAA,CACb,KAAA,EACA,IAAA,EACA,aAAA,EACA;AACA,EAAA,MAAM,oBAAA,GAAuB,4BAA4B,IAAI,CAAA;AAC7D,EAAA,MAAM,KAAA,GAAQ,MAAM,KAAA,CAAM,QAAA,EAAS;AACnC,EAAA,IAAI,wBAAA,CAAyB,KAAK,CAAA,EAAG;AACnC,IAAA,MAAM,EAAE,gBAAA,EAAkB,GAAG,WAAA,EAAY,GAAI,iBAAiB,EAAC;AAC/D,IAAA,MAAM,WAAA,GAAc;AAAA,MAClB,GAAG,WAAA;AAAA,MACH,GAAG,oBAAA;AAAA,MACH,SAAS,EAAC;AAAA,MACV,gBAAA,EAAkB,IAAA;AAAA,MAClB,gBAAgB,IAAA,CAAK;AAAA,KACvB;AACA,IAAA,OAAO,mBACH,MAAM,KAAA,CAAM,QAAA,CAAS,IAAA,CAAK,OAAO,EAAE,GAAG,WAAA,EAAa,gBAAA,EAAkB,CAAA,GACrE,MAAM,MAAM,QAAA,CAAS,IAAA,CAAK,OAAO,WAAW,CAAA;AAAA,EAClD,CAAA,MAAO;AACL,IAAA,OAAO,MAAM,KAAA,CAAM,cAAA,CAAe,IAAA,CAAK,KAAA,EAAO;AAAA,MAC5C,SAAS,EAAC;AAAA,MACV,gBAAA,EAAkB,IAAA;AAAA,MAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,MACrB,GAAG;AAAA,KACJ,CAAA;AAAA,EACH;AACF;AAEA,eAAe,UAAA,CACb,OAAA,EACA,YAAA,EACA,IAAA,EAC8B;AAC9B,EAAA,MAAM,gBAAqC,EAAC;AAE5C,EAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AAC1B,IAAA,KAAA,MAAW,UAAU,OAAA,EAAS;AAC5B,MAAA,IAAI;AACF,QAAA,MAAM,KAAA,GAAQ,MAAM,MAAA,CAAO,GAAA,CAAI;AAAA,UAC7B,KAAA,EAAO,aAAa,WAAA,EAAa,KAAA;AAAA,UACjC,MAAA,EAAQ,aAAa,WAAA,EAAa,MAAA;AAAA,UAClC,aAAa,IAAA,CAAK,WAAA;AAAA,UAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,UACrB,GAAG,4BAA4B,IAAI;AAAA,SACpC,CAAA;AAED,QAAA,aAAA,CAAc,MAAA,CAAO,EAAE,CAAA,GAAI,KAAA;AAAA,MAC7B,SAAS,KAAA,EAAO;AACd,QAAA,MAAM,IAAI,WAAA;AAAA,UACR;AAAA,YACE,MAAA,EAAQ,QAAA;AAAA,YACR,EAAA,EAAI,8CAAA;AAAA,YACJ,QAAA,EAAU,MAAA;AAAA,YACV,IAAA,EAAM,CAAA,+CAAA,EAAkD,MAAA,CAAO,EAAE,CAAA,CAAA;AAAA,YACjE,OAAA,EAAS;AAAA,cACP,UAAU,MAAA,CAAO,EAAA;AAAA,cACjB,IAAA,EAAM,IAAA,CAAK,SAAA,CAAU,IAAI;AAAA;AAC3B,WACF;AAAA,UACA;AAAA,SACF;AAAA,MACF;AAAA,IACF;AAAA,EACF,CAAA,MAAO;AAEL,IAAA,IAAI,QAAQ,QAAA,EAAU;AACpB,MAAA,MAAM,wBAA6C,EAAC;AACpD,MAAA,KAAA,MAAW,MAAA,IAAU,QAAQ,QAAA,EAAU;AACrC,QAAA,MAAM,KAAA,GAAQ,MAAM,MAAA,CAAO,GAAA,CAAI;AAAA,UAC7B,KAAA,EAAO,aAAa,WAAA,CAAY,KAAA;AAAA,UAChC,MAAA,EAAQ,aAAa,WAAA,CAAY,MAAA;AAAA,UACjC,aAAa,IAAA,CAAK,WAAA;AAAA,UAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,UACrB,GAAG,4BAA4B,IAAI;AAAA,SACpC,CAAA;AACD,QAAA,qBAAA,CAAsB,MAAA,CAAO,EAAE,CAAA,GAAI,KAAA;AAAA,MACrC;AACA,MAAA,IAAI,MAAA,CAAO,IAAA,CAAK,qBAAqB,CAAA,CAAE,SAAS,CAAA,EAAG;AACjD,QAAA,aAAA,CAAc,QAAA,GAAW,qBAAA;AAAA,MAC3B;AAAA,IACF;AAEA,IAAA,IAAI,QAAQ,KAAA,EAAO;AACjB,MAAA,MAAM,oBAAyC,EAAC;AAChD,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,OAAA,CAAQ,KAAK,CAAA,EAAG;AACjE,QAAA,MAAM,UAAA,GAAa,YAAA,CAAa,WAAA,CAAY,WAAA,GAAc,MAAM,CAAA;AAChE,QAAA,IAAI,UAAA,EAAY,MAAA,KAAW,SAAA,IAAa,UAAA,CAAW,WAAW,MAAA,EAAW;AACvE,UAAA,MAAM,cAAmC,EAAC;AAC1C,UAAA,KAAA,MAAW,UAAU,WAAA,EAAa;AAChC,YAAA,IAAI;AACF,cAAA,MAAM,KAAA,GAAQ,MAAM,MAAA,CAAO,GAAA,CAAI;AAAA,gBAC7B,OAAO,UAAA,CAAW,OAAA,KAAY,SAAY,UAAA,CAAW,OAAA,GAAU,aAAa,WAAA,CAAY,KAAA;AAAA,gBACxF,QAAQ,UAAA,CAAW,MAAA;AAAA,gBACnB,aAAa,IAAA,CAAK,WAAA;AAAA,gBAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,gBACrB,GAAG,4BAA4B,IAAI;AAAA,eACpC,CAAA;AACD,cAAA,WAAA,CAAY,MAAA,CAAO,EAAE,CAAA,GAAI,KAAA;AAAA,YAC3B,SAAS,KAAA,EAAO;AACd,cAAA,MAAM,IAAI,WAAA;AAAA,gBACR;AAAA,kBACE,MAAA,EAAQ,QAAA;AAAA,kBACR,EAAA,EAAI,mDAAA;AAAA,kBACJ,QAAA,EAAU,MAAA;AAAA,kBACV,IAAA,EAAM,CAAA,+CAAA,EAAkD,MAAA,CAAO,EAAE,YAAY,MAAM,CAAA,CAAA;AAAA,kBACnF,OAAA,EAAS;AAAA,oBACP,UAAU,MAAA,CAAO,EAAA;AAAA,oBACjB;AAAA;AACF,iBACF;AAAA,gBACA;AAAA,eACF;AAAA,YACF;AAAA,UACF;AACA,UAAA,IAAI,MAAA,CAAO,IAAA,CAAK,WAAW,CAAA,CAAE,SAAS,CAAA,EAAG;AACvC,YAAA,iBAAA,CAAkB,MAAM,CAAA,GAAI,WAAA;AAAA,UAC9B;AAAA,QACF;AAAA,MACF;AACA,MAAA,IAAI,MAAA,CAAO,IAAA,CAAK,iBAAiB,CAAA,CAAE,SAAS,CAAA,EAAG;AAC7C,QAAA,aAAA,CAAc,KAAA,GAAQ,iBAAA;AAAA,MACxB;AAAA,IACF;AAAA,EACF;AAEA,EAAA,OAAO,aAAA;AACT;AAMA,eAAe,mBAAA,CAAoB;AAAA,EACjC,OAAA;AAAA,EACA,aAAA;AAAA,EACA,MAAA;AAAA,EACA,IAAA;AAAA,EACA;AACF,CAAA,EAMkB;AAChB,EAAA,MAAM,WAAW,MAAA,CAAO,EAAA;AACxB,EAAA,MAAM,UAAA,GAAa,UAAA,CAAW,MAAM,CAAA,GAAI,UAAA,GAAa,OAAA;AAGrD,EAAA,IAAI,KAAA,CAAM,QAAQ,aAAa,CAAA,IAAK,EAAE,UAAA,IAAc,aAAA,IAAiB,WAAW,aAAA,CAAA,EAAgB;AAC9F,IAAA,KAAA,MAAW,CAAC,QAAA,EAAU,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,aAAa,CAAA,EAAG;AACnE,MAAA,IAAI,WAAA,IAAe,OAAO,WAAA,KAAgB,QAAA,IAAY,WAAW,WAAA,EAAa;AAC5E,QAAA,MAAM,eAAA,CAAgB;AAAA,UACpB,OAAA;AAAA,UACA,WAAA;AAAA,UACA,QAAA;AAAA,UACA,QAAA;AAAA,UACA,UAAA;AAAA,UACA,MAAA;AAAA,UACA,MAAA;AAAA,UACA;AAAA,SACD,CAAA;AAAA,MACH;AAAA,IACF;AAAA,EACF,CAAA,MAAO;AAEL,IAAA,IAAI,cAAc,QAAA,EAAU;AAC1B,MAAA,KAAA,MAAW,CAAC,UAAU,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,QAAQ,CAAA,EAAG;AAC5E,QAAA,IAAI,WAAA,IAAe,OAAO,WAAA,KAAgB,QAAA,IAAY,WAAW,WAAA,EAAa;AAC5E,UAAA,MAAM,eAAA,CAAgB;AAAA,YACpB,OAAA;AAAA,YACA,WAAA;AAAA,YACA,QAAA;AAAA,YACA,QAAA;AAAA,YACA,UAAA,EAAY,UAAA;AAAA,YACZ,MAAA;AAAA,YACA,MAAA;AAAA,YACA;AAAA,WACD,CAAA;AAAA,QACH;AAAA,MACF;AAAA,IACF;AAEA,IAAA,IAAI,cAAc,KAAA,EAAO;AACvB,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,KAAK,CAAA,EAAG;AACvE,QAAA,KAAA,MAAW,CAAC,QAAA,EAAU,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAkC,CAAA,EAAG;AACxF,UAAA,IAAI,WAAA,IAAe,OAAO,WAAA,KAAgB,QAAA,IAAY,WAAW,WAAA,EAAa;AAC5E,YAAA,MAAM,eAAA,CAAgB;AAAA,cACpB,OAAA;AAAA,cACA,WAAA;AAAA,cACA,QAAA;AAAA,cACA,QAAA,EAAU,MAAA;AAAA,cACV,UAAA,EAAY,MAAA;AAAA,cACZ,MAAA;AAAA,cACA,MAAA;AAAA,cACA;AAAA,aACD,CAAA;AAAA,UACH;AAAA,QACF;AAAA,MACF;AAAA,IACF;AAAA,EACF;AACF;AAKA,eAAe,eAAA,CAAgB;AAAA,EAC7B,OAAA;AAAA,EACA,WAAA;AAAA,EACA,QAAA;AAAA,EACA,QAAA;AAAA,EACA,UAAA;AAAA,EACA,MAAA;AAAA,EACA,MAAA;AAAA,EACA;AACF,CAAA,EASkB;AAChB,EAAA,IAAI;AAEF,IAAA,IAAI,MAAA,GAAS,MAAA,EAAQ,aAAA,GAAgB,QAAQ,CAAA;AAE7C,IAAA,IAAI,CAAC,MAAA,EAAQ;AAEX,MAAA,MAAM,aAAA,GAAgB,MAAO,MAAA,CAAe,WAAA,IAAc;AAC1D,MAAA,IAAI,aAAA,EAAe;AACjB,QAAA,KAAA,MAAW,CAAC,CAAA,EAAG,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,aAAa,CAAA,EAAG;AAC5D,UAAA,IAAK,WAAA,CAAoB,MAAA,EAAQ,EAAA,KAAO,QAAA,EAAU;AAChD,YAAA,MAAA,GAAU,WAAA,CAAoB,MAAA;AAC9B,YAAA;AAAA,UACF;AAAA,QACF;AAAA,MACF;AAAA,IACF;AAGA,IAAA,IAAI,OAAA;AACJ,IAAA,IAAI,MAAA;AACJ,IAAA,IAAI,KAAK,cAAA,EAAgB,WAAA,IAAe,IAAA,CAAK,cAAA,CAAe,YAAY,OAAA,EAAS;AAC/E,MAAA,MAAA,GAAS,IAAA,CAAK,eAAe,WAAA,CAAY,EAAA;AACzC,MAAA,OAAA,GAAU,IAAA,CAAK,eAAe,WAAA,CAAY,OAAA;AAAA,IAC5C;AAGA,IAAA,MAAM,oBAAyC,EAAC;AAChD,IAAA,IAAI,IAAA,CAAK,gBAAgB,MAAA,EAAW;AAClC,MAAA,iBAAA,CAAkB,cAAc,IAAA,CAAK,WAAA;AAAA,IACvC;AAEA,IAAA,MAAM,OAAA,GAAU;AAAA,MACd,GAAG,WAAA;AAAA,MACH,QAAA;AAAA,MACA,QAAA;AAAA,MACA,UAAA;AAAA,MACA,MAAA,EAAQ,MAAA;AAAA,MACR,MAAA,EAAQ;AAAA,QACN,EAAA,EAAI,QAAQ,EAAA,IAAM,QAAA;AAAA,QAClB,IAAA,EAAM,QAAQ,IAAA,IAAQ,QAAA;AAAA,QACtB,WAAA,EAAa,QAAQ,WAAA,IAAe,EAAA;AAAA,QACpC,IAAA,EAAM,QAAQ,IAAA,IAAQ,SAAA;AAAA,QACtB,GAAI,SAAS,EAAE,QAAA,EAAU,CAAC,CAAC,MAAA,CAAO,KAAA,EAAM,GAAI;AAAC,OAC/C;AAAA,MACA,MAAA,EAAQ;AAAA,QACN,IAAI,MAAA,CAAO,EAAA;AAAA,QACX,IAAA,EAAO,MAAA,CAAe,IAAA,IAAQ,MAAA,CAAO;AAAA,OACvC;AAAA;AAAA,MAEA,cAAA,EAAgB,KAAK,cAAA,GAAiB,MAAA,CAAO,YAAY,IAAA,CAAK,cAAA,CAAe,OAAA,EAAS,CAAA,GAAI,MAAA;AAAA;AAAA,MAE1F,mBAAmB,MAAA,CAAO,IAAA,CAAK,iBAAiB,CAAA,CAAE,MAAA,GAAS,IAAI,iBAAA,GAAoB,MAAA;AAAA;AAAA,MAEnF,OAAA;AAAA,MACA;AAAA,KACF;AAEA,IAAA,MAAM,oBAAA,CAAqB,SAAS,OAAO,CAAA;AAAA,EAC7C,SAAS,KAAA,EAAO;AAEd,IAAA,MAAA,EAAQ,aAAY,EAAG,IAAA,GAAO,CAAA,gCAAA,EAAmC,QAAQ,KAAK,KAAK,CAAA;AAAA,EACrF;AACF","file":"chunk-7BZCZH6B.js","sourcesContent":["import pMap from 'p-map';\nimport { ErrorCategory, ErrorDomain, MastraError } from '../error';\nimport { saveScorePayloadSchema } from '../evals';\nimport type { ScoringHookInput } from '../evals/types';\nimport type { Mastra } from '../mastra';\nimport type { MastraStorage } from '../storage';\n\nexport function createOnScorerHook(mastra: Mastra) {\n  return async (hookData: ScoringHookInput) => {\n    const storage = mastra.getStorage();\n\n    if (!storage) {\n      mastra.getLogger()?.warn('Storage not found, skipping score validation and saving');\n      return;\n    }\n\n    const entityId = hookData.entity.id as string;\n    const entityType = hookData.entityType;\n    const scorer = hookData.scorer;\n    const scorerId = scorer.id as string;\n\n    if (!scorerId) {\n      mastra.getLogger()?.warn('Scorer ID not found, skipping score validation and saving');\n      return;\n    }\n\n    try {\n      const scorerToUse = await findScorer(mastra, entityId, entityType, scorerId);\n\n      if (!scorerToUse) {\n        throw new MastraError({\n          id: 'MASTRA_SCORER_NOT_FOUND',\n          domain: ErrorDomain.MASTRA,\n          category: ErrorCategory.USER,\n          text: `Scorer with ID ${scorerId} not found`,\n        });\n      }\n\n      let input = hookData.input;\n      let output = hookData.output;\n\n      const { structuredOutput, ...rest } = hookData;\n\n      const runResult = await scorerToUse.scorer.run({\n        ...rest,\n        input,\n        output,\n      });\n\n      let spanId;\n      let traceId;\n      const currentSpan = hookData.tracingContext?.currentSpan;\n      if (currentSpan && currentSpan.isValid) {\n        spanId = currentSpan.id;\n        traceId = currentSpan.traceId;\n      }\n\n      const payload = {\n        ...rest,\n        ...runResult,\n        entityId,\n        scorerId: scorerId,\n        spanId,\n        traceId,\n        scorer: {\n          ...rest.scorer,\n          hasJudge: !!scorerToUse.scorer.judge,\n        },\n        metadata: {\n          structuredOutput: !!structuredOutput,\n        },\n      };\n      await validateAndSaveScore(storage, payload);\n\n      if (currentSpan && spanId && traceId) {\n        await pMap(\n          currentSpan.observabilityInstance.getExporters(),\n          async exporter => {\n            if (exporter.addScoreToTrace) {\n              try {\n                await exporter.addScoreToTrace({\n                  traceId: traceId,\n                  spanId: spanId,\n                  score: runResult.score as number,\n                  reason: runResult.reason as string,\n                  scorerName: scorerToUse.scorer.id,\n                  metadata: {\n                    ...(currentSpan.metadata ?? {}),\n                  },\n                });\n              } catch (error) {\n                // Log error but don't fail the hook if exporter fails\n                mastra.getLogger()?.error(`Failed to add score to trace via exporter: ${error}`);\n              }\n            }\n          },\n          { concurrency: 3 },\n        );\n      }\n    } catch (error) {\n      const mastraError = new MastraError(\n        {\n          id: 'MASTRA_SCORER_FAILED_TO_RUN_HOOK',\n          domain: ErrorDomain.SCORER,\n          category: ErrorCategory.USER,\n          details: {\n            scorerId,\n            entityId,\n            entityType,\n          },\n        },\n        error,\n      );\n\n      mastra.getLogger()?.trackException(mastraError);\n      mastra.getLogger()?.error(mastraError.toString());\n    }\n  };\n}\n\nexport async function validateAndSaveScore(storage: MastraStorage, payload: unknown) {\n  const scoresStore = await storage.getStore('scores');\n  if (!scoresStore) {\n    throw new MastraError({\n      id: 'MASTRA_SCORES_STORAGE_NOT_AVAILABLE',\n      domain: ErrorDomain.STORAGE,\n      category: ErrorCategory.SYSTEM,\n      text: 'Scores storage domain is not available',\n    });\n  }\n  const payloadToSave = saveScorePayloadSchema.parse(payload);\n  await scoresStore.saveScore(payloadToSave);\n}\n\nasync function findScorer(mastra: Mastra, entityId: string, entityType: string, scorerId: string) {\n  let scorerToUse;\n  if (entityType === 'AGENT') {\n    // Try code-defined agents first\n    try {\n      const agent = mastra.getAgentById(entityId);\n      const scorers = await agent.listScorers();\n      for (const [_, scorer] of Object.entries(scorers)) {\n        if (scorer.scorer.id === scorerId) {\n          scorerToUse = scorer;\n          break;\n        }\n      }\n    } catch {\n      // Agent not found in code-defined agents, try stored agents via editor\n      try {\n        const storedAgent = (await mastra.getEditor()?.agent.getById(entityId)) ?? null;\n        if (storedAgent) {\n          const scorers = await storedAgent.listScorers();\n          for (const [_, scorer] of Object.entries(scorers) as [string, any][]) {\n            if (scorer.scorer.id === scorerId) {\n              scorerToUse = scorer;\n              break;\n            }\n          }\n        }\n      } catch {\n        // Stored agent also not found, will fall back to mastra-registered scorer\n      }\n    }\n  } else if (entityType === 'WORKFLOW') {\n    const scorers = await mastra.getWorkflowById(entityId).listScorers();\n    for (const [_, scorer] of Object.entries(scorers)) {\n      if (scorer.scorer.id === scorerId) {\n        scorerToUse = scorer;\n        break;\n      }\n    }\n  }\n\n  // Fallback to mastra-registered scorer\n  if (!scorerToUse) {\n    const mastraRegisteredScorer = mastra.getScorerById(scorerId);\n    scorerToUse = mastraRegisteredScorer ? { scorer: mastraRegisteredScorer } : undefined;\n  }\n\n  return scorerToUse;\n}\n","export class ScoreAccumulator {\n  private flatScores: Record<string, number[]> = {};\n  private workflowScores: Record<string, number[]> = {};\n  private stepScores: Record<string, Record<string, number[]>> = {};\n\n  addScores(scorerResults: Record<string, any>) {\n    const isTargetWorkflowAndHasStepScores = 'steps' in scorerResults;\n    if (isTargetWorkflowAndHasStepScores) {\n      this.addNestedScores(scorerResults);\n    } else {\n      this.addFlatScores(scorerResults);\n    }\n  }\n\n  private addFlatScores(scorerResults: Record<string, any>) {\n    for (const [scorerName, result] of Object.entries(scorerResults)) {\n      if (!this.flatScores[scorerName]) {\n        this.flatScores[scorerName] = [];\n      }\n      this.flatScores[scorerName].push((result as { score: number }).score);\n    }\n  }\n\n  private addNestedScores(scorerResults: Record<string, any>) {\n    if ('workflow' in scorerResults && scorerResults.workflow) {\n      for (const [scorerName, result] of Object.entries(scorerResults.workflow)) {\n        if (!this.workflowScores[scorerName]) {\n          this.workflowScores[scorerName] = [];\n        }\n        this.workflowScores[scorerName].push((result as { score: number }).score);\n      }\n    }\n\n    if ('steps' in scorerResults && scorerResults.steps) {\n      for (const [stepId, stepResults] of Object.entries(scorerResults.steps)) {\n        if (!this.stepScores[stepId]) {\n          this.stepScores[stepId] = {};\n        }\n        for (const [scorerName, result] of Object.entries(stepResults as Record<string, any>)) {\n          if (!this.stepScores[stepId][scorerName]) {\n            this.stepScores[stepId][scorerName] = [];\n          }\n          this.stepScores[stepId][scorerName].push((result as { score: number }).score);\n        }\n      }\n    }\n  }\n\n  addStepScores(stepScorerResults: Record<string, Record<string, any>>) {\n    for (const [stepId, stepResults] of Object.entries(stepScorerResults)) {\n      if (!this.stepScores[stepId]) {\n        this.stepScores[stepId] = {};\n      }\n      for (const [scorerName, result] of Object.entries(stepResults)) {\n        if (!this.stepScores[stepId][scorerName]) {\n          this.stepScores[stepId][scorerName] = [];\n        }\n        this.stepScores[stepId][scorerName].push((result as { score: number }).score);\n      }\n    }\n  }\n\n  getAverageScores(): Record<string, any> {\n    const result: Record<string, any> = {};\n\n    for (const [scorerName, scoreArray] of Object.entries(this.flatScores)) {\n      result[scorerName] = this.getAverageScore(scoreArray);\n    }\n\n    // Add workflow scores\n    if (Object.keys(this.workflowScores).length > 0) {\n      result.workflow = {};\n      for (const [scorerName, scoreArray] of Object.entries(this.workflowScores)) {\n        result.workflow[scorerName] = this.getAverageScore(scoreArray);\n      }\n    }\n\n    if (Object.keys(this.stepScores).length > 0) {\n      result.steps = {};\n      for (const [stepId, stepScorers] of Object.entries(this.stepScores)) {\n        result.steps[stepId] = {};\n        for (const [scorerName, scoreArray] of Object.entries(stepScorers)) {\n          result.steps[stepId][scorerName] = this.getAverageScore(scoreArray);\n        }\n      }\n    }\n\n    return result;\n  }\n\n  private getAverageScore(scoreArray: number[]): number {\n    if (scoreArray.length > 0) {\n      return scoreArray.reduce((a, b) => a + b, 0) / scoreArray.length;\n    } else {\n      return 0;\n    }\n  }\n}\n","import type { CoreMessage } from '@internal/ai-sdk-v4';\nimport type { Agent, AgentExecutionOptions, AiMessageType, UIMessageWithMetadata } from '../../agent';\nimport { isSupportedLanguageModel } from '../../agent';\nimport { MastraError } from '../../error';\nimport { validateAndSaveScore } from '../../mastra/hooks';\nimport type { ObservabilityContext } from '../../observability';\nimport { resolveObservabilityContext } from '../../observability';\nimport type { RequestContext } from '../../request-context';\nimport { Workflow } from '../../workflows';\nimport type { AnyWorkflow, WorkflowResult, WorkflowRunStartOptions, StepResult } from '../../workflows';\nimport type { MastraScorer } from '../base';\nimport { ScoreAccumulator } from './scorerAccumulator';\n\ntype WorkflowRunOptions = WorkflowRunStartOptions & {\n  initialState?: any;\n};\n\ntype RunEvalsDataItem<TTarget = unknown> = {\n  input: TTarget extends Workflow<any, any>\n    ? any\n    : TTarget extends Agent\n      ? string | string[] | CoreMessage[] | AiMessageType[] | UIMessageWithMetadata[]\n      : unknown;\n  groundTruth?: any;\n  requestContext?: RequestContext;\n  startOptions?: WorkflowRunOptions;\n} & Partial<ObservabilityContext>;\n\ntype WorkflowScorerConfig = {\n  workflow?: MastraScorer<any, any, any, any>[];\n  steps?: Record<string, MastraScorer<any, any, any, any>[]>;\n};\n\ntype RunEvalsResult = {\n  scores: Record<string, any>;\n  summary: {\n    totalItems: number;\n  };\n};\n\n// Agent with scorers array\nexport function runEvals<TAgent extends Agent>(config: {\n  data: RunEvalsDataItem<TAgent>[];\n  scorers: MastraScorer<any, any, any, any>[];\n  target: TAgent;\n  targetOptions?: Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<TAgent>;\n    targetResult: Awaited<ReturnType<Agent['generate']>>;\n    scorerResults: Record<string, any>; // Flat structure: { scorerName: result }\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult>;\n\n// Workflow with scorers array\nexport function runEvals<TWorkflow extends AnyWorkflow>(config: {\n  data: RunEvalsDataItem<TWorkflow>[];\n  scorers: MastraScorer<any, any, any, any>[];\n  target: TWorkflow;\n  targetOptions?: WorkflowRunOptions;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<TWorkflow>;\n    targetResult: WorkflowResult<any, any, any, any>;\n    scorerResults: Record<string, any>; // Flat structure: { scorerName: result }\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult>;\n\n// Workflow with workflow configuration\nexport function runEvals<TWorkflow extends AnyWorkflow>(config: {\n  data: RunEvalsDataItem<TWorkflow>[];\n  scorers: WorkflowScorerConfig;\n  target: TWorkflow;\n  targetOptions?: WorkflowRunOptions;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<TWorkflow>;\n    targetResult: WorkflowResult<any, any, any, any>;\n    scorerResults: {\n      workflow?: Record<string, any>;\n      steps?: Record<string, Record<string, any>>;\n    };\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult>;\nexport async function runEvals(config: {\n  data: RunEvalsDataItem<any>[];\n  scorers: MastraScorer<any, any, any, any>[] | WorkflowScorerConfig;\n  target: Agent | Workflow;\n  targetOptions?:\n    | Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>\n    | WorkflowRunOptions;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<any>;\n    targetResult: any;\n    scorerResults: any;\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult> {\n  const { data, scorers, target, targetOptions, onItemComplete, concurrency = 1 } = config;\n\n  validateEvalsInputs(data, scorers, target);\n\n  let totalItems = 0;\n  const scoreAccumulator = new ScoreAccumulator();\n\n  // Get storage from target's Mastra instance if available\n  // Agent uses getMastraInstance(), Workflow uses .mastra getter\n  const mastra = (target as any).getMastraInstance?.() || (target as any).mastra;\n  const storage = mastra?.getStorage();\n\n  const pMap = (await import('p-map')).default;\n  await pMap(\n    data,\n    async (item: RunEvalsDataItem<any>) => {\n      const targetResult = await executeTarget(target, item, targetOptions);\n      const scorerResults = await runScorers(scorers, targetResult, item);\n      scoreAccumulator.addScores(scorerResults);\n\n      // Save scores to storage if available\n      if (storage) {\n        await saveScoresToStorage({\n          storage,\n          scorerResults,\n          target,\n          item,\n          mastra,\n        });\n      }\n\n      if (onItemComplete) {\n        await onItemComplete({\n          item,\n          targetResult: targetResult as any,\n          scorerResults: scorerResults as any,\n        });\n      }\n\n      totalItems++;\n    },\n    { concurrency },\n  );\n\n  return {\n    scores: scoreAccumulator.getAverageScores(),\n    summary: {\n      totalItems,\n    },\n  };\n}\n\nfunction isWorkflow(target: Agent | Workflow): target is Workflow {\n  return target instanceof Workflow;\n}\n\nfunction isWorkflowScorerConfig(scorers: any): scorers is WorkflowScorerConfig {\n  return typeof scorers === 'object' && !Array.isArray(scorers) && ('workflow' in scorers || 'steps' in scorers);\n}\n\nfunction validateEvalsInputs(\n  data: RunEvalsDataItem<any>[],\n  scorers: MastraScorer<any, any, any, any>[] | WorkflowScorerConfig,\n  target: Agent | Workflow,\n): void {\n  if (data.length === 0) {\n    throw new MastraError({\n      domain: 'SCORER',\n      id: 'RUN_EXPERIMENT_FAILED_NO_DATA_PROVIDED',\n      category: 'USER',\n      text: 'Failed to run experiment: Data array is empty',\n    });\n  }\n\n  for (let i = 0; i < data.length; i++) {\n    const item = data[i];\n    if (!item || typeof item !== 'object' || !('input' in item)) {\n      throw new MastraError({\n        domain: 'SCORER',\n        id: 'INVALID_DATA_ITEM',\n        category: 'USER',\n        text: `Invalid data item at index ${i}: must have 'input' properties`,\n      });\n    }\n  }\n\n  // Validate scorers\n  if (Array.isArray(scorers)) {\n    if (scorers.length === 0) {\n      throw new MastraError({\n        domain: 'SCORER',\n        id: 'NO_SCORERS_PROVIDED',\n        category: 'USER',\n        text: 'At least one scorer must be provided',\n      });\n    }\n  } else if (isWorkflow(target) && isWorkflowScorerConfig(scorers)) {\n    const hasScorers =\n      (scorers.workflow && scorers.workflow.length > 0) || (scorers.steps && Object.keys(scorers.steps).length > 0);\n\n    if (!hasScorers) {\n      throw new MastraError({\n        domain: 'SCORER',\n        id: 'NO_SCORERS_PROVIDED',\n        category: 'USER',\n        text: 'At least one workflow or step scorer must be provided',\n      });\n    }\n  } else if (!isWorkflow(target) && !Array.isArray(scorers)) {\n    throw new MastraError({\n      domain: 'SCORER',\n      id: 'INVALID_AGENT_SCORERS',\n      category: 'USER',\n      text: 'Agent scorers must be an array of scorers',\n    });\n  }\n}\n\nasync function executeTarget(\n  target: Agent | Workflow,\n  item: RunEvalsDataItem<any>,\n  targetOptions?:\n    | Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>\n    | WorkflowRunOptions,\n) {\n  try {\n    if (isWorkflow(target)) {\n      return await executeWorkflow(target, item, targetOptions as WorkflowRunOptions);\n    } else {\n      return await executeAgent(\n        target,\n        item,\n        targetOptions as Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>,\n      );\n    }\n  } catch (error) {\n    throw new MastraError(\n      {\n        domain: 'SCORER',\n        id: 'RUN_EXPERIMENT_TARGET_FAILED_TO_GENERATE_RESULT',\n        category: 'USER',\n        text: 'Failed to run experiment: Error generating result from target',\n        details: {\n          item: JSON.stringify(item),\n        },\n      },\n      error,\n    );\n  }\n}\n\nasync function executeWorkflow(target: Workflow, item: RunEvalsDataItem<any>, targetOptions?: WorkflowRunOptions) {\n  const observabilityContext = resolveObservabilityContext(item);\n  const run = await target.createRun({ disableScorers: true });\n  const workflowResult = await run.start({\n    ...targetOptions,\n    ...item.startOptions,\n    inputData: item.input,\n    requestContext: item.requestContext,\n    ...observabilityContext,\n  });\n\n  return {\n    scoringData: {\n      input: item.input,\n      output: workflowResult.status === 'success' ? workflowResult.result : undefined,\n      stepResults: workflowResult.steps as Record<string, StepResult<any, any, any, any>>,\n    },\n  };\n}\n\nasync function executeAgent(\n  agent: Agent,\n  item: RunEvalsDataItem<any>,\n  targetOptions?: Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>,\n) {\n  const observabilityContext = resolveObservabilityContext(item);\n  const model = await agent.getModel();\n  if (isSupportedLanguageModel(model)) {\n    const { structuredOutput, ...restOptions } = targetOptions ?? {};\n    const baseOptions = {\n      ...restOptions,\n      ...observabilityContext,\n      scorers: {},\n      returnScorerData: true,\n      requestContext: item.requestContext,\n    };\n    return structuredOutput\n      ? await agent.generate(item.input, { ...baseOptions, structuredOutput })\n      : await agent.generate(item.input, baseOptions);\n  } else {\n    return await agent.generateLegacy(item.input, {\n      scorers: {},\n      returnScorerData: true,\n      requestContext: item.requestContext,\n      ...observabilityContext,\n    });\n  }\n}\n\nasync function runScorers(\n  scorers: MastraScorer<any, any, any, any>[] | WorkflowScorerConfig,\n  targetResult: any,\n  item: RunEvalsDataItem<any>,\n): Promise<Record<string, any>> {\n  const scorerResults: Record<string, any> = {};\n\n  if (Array.isArray(scorers)) {\n    for (const scorer of scorers) {\n      try {\n        const score = await scorer.run({\n          input: targetResult.scoringData?.input,\n          output: targetResult.scoringData?.output,\n          groundTruth: item.groundTruth,\n          requestContext: item.requestContext,\n          ...resolveObservabilityContext(item),\n        });\n\n        scorerResults[scorer.id] = score;\n      } catch (error) {\n        throw new MastraError(\n          {\n            domain: 'SCORER',\n            id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_RESULT',\n            category: 'USER',\n            text: `Failed to run experiment: Error running scorer ${scorer.id}`,\n            details: {\n              scorerId: scorer.id,\n              item: JSON.stringify(item),\n            },\n          },\n          error,\n        );\n      }\n    }\n  } else {\n    // Handle workflow scorer config\n    if (scorers.workflow) {\n      const workflowScorerResults: Record<string, any> = {};\n      for (const scorer of scorers.workflow) {\n        const score = await scorer.run({\n          input: targetResult.scoringData.input,\n          output: targetResult.scoringData.output,\n          groundTruth: item.groundTruth,\n          requestContext: item.requestContext,\n          ...resolveObservabilityContext(item),\n        });\n        workflowScorerResults[scorer.id] = score;\n      }\n      if (Object.keys(workflowScorerResults).length > 0) {\n        scorerResults.workflow = workflowScorerResults;\n      }\n    }\n\n    if (scorers.steps) {\n      const stepScorerResults: Record<string, any> = {};\n      for (const [stepId, stepScorers] of Object.entries(scorers.steps)) {\n        const stepResult = targetResult.scoringData.stepResults?.[stepId];\n        if (stepResult?.status === 'success' && stepResult.output !== undefined) {\n          const stepResults: Record<string, any> = {};\n          for (const scorer of stepScorers) {\n            try {\n              const score = await scorer.run({\n                input: stepResult.payload !== undefined ? stepResult.payload : targetResult.scoringData.input,\n                output: stepResult.output,\n                groundTruth: item.groundTruth,\n                requestContext: item.requestContext,\n                ...resolveObservabilityContext(item),\n              });\n              stepResults[scorer.id] = score;\n            } catch (error) {\n              throw new MastraError(\n                {\n                  domain: 'SCORER',\n                  id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT',\n                  category: 'USER',\n                  text: `Failed to run experiment: Error running scorer ${scorer.id} on step ${stepId}`,\n                  details: {\n                    scorerId: scorer.id,\n                    stepId,\n                  },\n                },\n                error,\n              );\n            }\n          }\n          if (Object.keys(stepResults).length > 0) {\n            stepScorerResults[stepId] = stepResults;\n          }\n        }\n      }\n      if (Object.keys(stepScorerResults).length > 0) {\n        scorerResults.steps = stepScorerResults;\n      }\n    }\n  }\n\n  return scorerResults;\n}\n\n/**\n * Saves scorer results to storage when running evaluations.\n * This makes scores visible in Studio's observability section.\n */\nasync function saveScoresToStorage({\n  storage,\n  scorerResults,\n  target,\n  item,\n  mastra,\n}: {\n  storage: any;\n  scorerResults: Record<string, any>;\n  target: Agent | Workflow;\n  item: RunEvalsDataItem<any>;\n  mastra: any;\n}): Promise<void> {\n  const entityId = target.id;\n  const entityType = isWorkflow(target) ? 'WORKFLOW' : 'AGENT';\n\n  // Handle flat scorer results (for agents or workflow-level scorers)\n  if (Array.isArray(scorerResults) || !('workflow' in scorerResults && 'steps' in scorerResults)) {\n    for (const [scorerId, scoreResult] of Object.entries(scorerResults)) {\n      if (scoreResult && typeof scoreResult === 'object' && 'score' in scoreResult) {\n        await saveSingleScore({\n          storage,\n          scoreResult,\n          scorerId,\n          entityId,\n          entityType,\n          mastra,\n          target,\n          item,\n        });\n      }\n    }\n  } else {\n    // Handle workflow scorer config with workflow and step scorers\n    if (scorerResults.workflow) {\n      for (const [scorerId, scoreResult] of Object.entries(scorerResults.workflow)) {\n        if (scoreResult && typeof scoreResult === 'object' && 'score' in scoreResult) {\n          await saveSingleScore({\n            storage,\n            scoreResult,\n            scorerId,\n            entityId,\n            entityType: 'WORKFLOW',\n            mastra,\n            target,\n            item,\n          });\n        }\n      }\n    }\n\n    if (scorerResults.steps) {\n      for (const [stepId, stepScorers] of Object.entries(scorerResults.steps)) {\n        for (const [scorerId, scoreResult] of Object.entries(stepScorers as Record<string, any>)) {\n          if (scoreResult && typeof scoreResult === 'object' && 'score' in scoreResult) {\n            await saveSingleScore({\n              storage,\n              scoreResult,\n              scorerId,\n              entityId: stepId,\n              entityType: 'STEP',\n              mastra,\n              target,\n              item,\n            });\n          }\n        }\n      }\n    }\n  }\n}\n\n/**\n * Saves a single scorer result to storage\n */\nasync function saveSingleScore({\n  storage,\n  scoreResult,\n  scorerId,\n  entityId,\n  entityType,\n  mastra,\n  target,\n  item,\n}: {\n  storage: any;\n  scoreResult: any;\n  scorerId: string;\n  entityId: string;\n  entityType: string;\n  mastra: any;\n  target: Agent | Workflow;\n  item: RunEvalsDataItem<any>;\n}): Promise<void> {\n  try {\n    // Get scorer information\n    let scorer = mastra?.getScorerById?.(scorerId);\n\n    if (!scorer) {\n      // Try to get from target's scorers\n      const targetScorers = await (target as any).listScorers?.();\n      if (targetScorers) {\n        for (const [_, scorerEntry] of Object.entries(targetScorers)) {\n          if ((scorerEntry as any).scorer?.id === scorerId) {\n            scorer = (scorerEntry as any).scorer;\n            break;\n          }\n        }\n      }\n    }\n\n    // Extract tracing context if available\n    let traceId: string | undefined;\n    let spanId: string | undefined;\n    if (item.tracingContext?.currentSpan && item.tracingContext.currentSpan.isValid) {\n      spanId = item.tracingContext.currentSpan.id;\n      traceId = item.tracingContext.currentSpan.traceId;\n    }\n\n    // Build additional context with groundTruth if available\n    const additionalContext: Record<string, any> = {};\n    if (item.groundTruth !== undefined) {\n      additionalContext.groundTruth = item.groundTruth;\n    }\n\n    const payload = {\n      ...scoreResult,\n      scorerId,\n      entityId,\n      entityType,\n      source: 'TEST' as const,\n      scorer: {\n        id: scorer?.id || scorerId,\n        name: scorer?.name || scorerId,\n        description: scorer?.description || '',\n        type: scorer?.type || 'unknown',\n        ...(scorer ? { hasJudge: !!scorer.judge } : {}),\n      },\n      entity: {\n        id: target.id,\n        name: (target as any).name || target.id,\n      },\n      // Include requestContext from item\n      requestContext: item.requestContext ? Object.fromEntries(item.requestContext.entries()) : undefined,\n      // Include additionalContext with groundTruth\n      additionalContext: Object.keys(additionalContext).length > 0 ? additionalContext : undefined,\n      // Include tracing information\n      traceId,\n      spanId,\n    };\n\n    await validateAndSaveScore(storage, payload);\n  } catch (error) {\n    // Log error but don't fail the evaluation\n    mastra?.getLogger?.()?.warn?.(`Failed to save score for scorer ${scorerId}:`, error);\n  }\n}\n"]}
+{"version":3,"sources":["../src/mastra/hooks.ts","../src/evals/run/scorerAccumulator.ts","../src/evals/run/index.ts"],"names":["pMap"],"mappings":";;;;;;AAOO,SAAS,mBAAmB,MAAA,EAAgB;AACjD,EAAA,OAAO,OAAO,QAAA,KAA+B;AAC3C,IAAA,MAAM,OAAA,GAAU,OAAO,UAAA,EAAW;AAElC,IAAA,IAAI,CAAC,OAAA,EAAS;AACZ,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,IAAA,CAAK,yDAAyD,CAAA;AAClF,MAAA;AAAA,IACF;AAEA,IAAA,MAAM,QAAA,GAAW,SAAS,MAAA,CAAO,EAAA;AACjC,IAAA,MAAM,aAAa,QAAA,CAAS,UAAA;AAC5B,IAAA,MAAM,SAAS,QAAA,CAAS,MAAA;AACxB,IAAA,MAAM,WAAW,MAAA,CAAO,EAAA;AAExB,IAAA,IAAI,CAAC,QAAA,EAAU;AACb,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,IAAA,CAAK,2DAA2D,CAAA;AACpF,MAAA;AAAA,IACF;AAEA,IAAA,IAAI;AACF,MAAA,MAAM,cAAc,MAAM,UAAA,CAAW,MAAA,EAAQ,QAAA,EAAU,YAAY,QAAQ,CAAA;AAE3E,MAAA,IAAI,CAAC,WAAA,EAAa;AAChB,QAAA,MAAM,IAAI,WAAA,CAAY;AAAA,UACpB,EAAA,EAAI,yBAAA;AAAA,UACJ,MAAA,EAAA,QAAA;AAAA,UACA,QAAA,EAAA,MAAA;AAAA,UACA,IAAA,EAAM,kBAAkB,QAAQ,CAAA,UAAA;AAAA,SACjC,CAAA;AAAA,MACH;AAEA,MAAA,IAAI,QAAQ,QAAA,CAAS,KAAA;AACrB,MAAA,IAAI,SAAS,QAAA,CAAS,MAAA;AAEtB,MAAA,MAAM,EAAE,gBAAA,EAAkB,GAAG,IAAA,EAAK,GAAI,QAAA;AAEtC,MAAA,MAAM,SAAA,GAAY,MAAM,WAAA,CAAY,MAAA,CAAO,GAAA,CAAI;AAAA,QAC7C,GAAG,IAAA;AAAA,QACH,KAAA;AAAA,QACA;AAAA,OACD,CAAA;AAED,MAAA,IAAI,MAAA;AACJ,MAAA,IAAI,OAAA;AACJ,MAAA,MAAM,WAAA,GAAc,SAAS,cAAA,EAAgB,WAAA;AAC7C,MAAA,IAAI,WAAA,IAAe,YAAY,OAAA,EAAS;AACtC,QAAA,MAAA,GAAS,WAAA,CAAY,EAAA;AACrB,QAAA,OAAA,GAAU,WAAA,CAAY,OAAA;AAAA,MACxB;AAEA,MAAA,MAAM,OAAA,GAAU;AAAA,QACd,GAAG,IAAA;AAAA,QACH,GAAG,SAAA;AAAA,QACH,QAAA;AAAA,QACA,QAAA;AAAA,QACA,MAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA,EAAQ;AAAA,UACN,GAAG,IAAA,CAAK,MAAA;AAAA,UACR,QAAA,EAAU,CAAC,CAAC,WAAA,CAAY,MAAA,CAAO;AAAA,SACjC;AAAA,QACA,QAAA,EAAU;AAAA,UACR,gBAAA,EAAkB,CAAC,CAAC;AAAA;AACtB,OACF;AACA,MAAA,MAAM,oBAAA,CAAqB,SAAS,OAAO,CAAA;AAE3C,MAAA,IAAI,WAAA,IAAe,UAAU,OAAA,EAAS;AACpC,QAAA,MAAM,IAAA;AAAA,UACJ,WAAA,CAAY,sBAAsB,YAAA,EAAa;AAAA,UAC/C,OAAM,QAAA,KAAY;AAChB,YAAA,IAAI,SAAS,eAAA,EAAiB;AAC5B,cAAA,IAAI;AACF,gBAAA,MAAM,SAAS,eAAA,CAAgB;AAAA,kBAC7B,OAAA;AAAA,kBACA,MAAA;AAAA,kBACA,OAAO,SAAA,CAAU,KAAA;AAAA,kBACjB,QAAQ,SAAA,CAAU,MAAA;AAAA,kBAClB,UAAA,EAAY,YAAY,MAAA,CAAO,EAAA;AAAA,kBAC/B,QAAA,EAAU;AAAA,oBACR,GAAI,WAAA,CAAY,QAAA,IAAY;AAAC;AAC/B,iBACD,CAAA;AAAA,cACH,SAAS,KAAA,EAAO;AAEd,gBAAA,MAAA,CAAO,SAAA,EAAU,EAAG,KAAA,CAAM,CAAA,2CAAA,EAA8C,KAAK,CAAA,CAAE,CAAA;AAAA,cACjF;AAAA,YACF;AAAA,UACF,CAAA;AAAA,UACA,EAAE,aAAa,CAAA;AAAE,SACnB;AAAA,MACF;AAAA,IACF,SAAS,KAAA,EAAO;AACd,MAAA,MAAM,cAAc,IAAI,WAAA;AAAA,QACtB;AAAA,UACE,EAAA,EAAI,kCAAA;AAAA,UACJ,MAAA,EAAA,QAAA;AAAA,UACA,QAAA,EAAA,MAAA;AAAA,UACA,OAAA,EAAS;AAAA,YACP,QAAA;AAAA,YACA,QAAA;AAAA,YACA;AAAA;AACF,SACF;AAAA,QACA;AAAA,OACF;AAEA,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,cAAA,CAAe,WAAW,CAAA;AAC9C,MAAA,MAAA,CAAO,SAAA,EAAU,EAAG,KAAA,CAAM,WAAA,CAAY,UAAU,CAAA;AAAA,IAClD;AAAA,EACF,CAAA;AACF;AAEA,eAAsB,oBAAA,CAAqB,SAAwB,OAAA,EAAkB;AACnF,EAAA,MAAM,WAAA,GAAc,MAAM,OAAA,CAAQ,QAAA,CAAS,QAAQ,CAAA;AACnD,EAAA,IAAI,CAAC,WAAA,EAAa;AAChB,IAAA,MAAM,IAAI,WAAA,CAAY;AAAA,MACpB,EAAA,EAAI,qCAAA;AAAA,MACJ,MAAA,EAAA,SAAA;AAAA,MACA,QAAA,EAAA,QAAA;AAAA,MACA,IAAA,EAAM;AAAA,KACP,CAAA;AAAA,EACH;AACA,EAAA,MAAM,aAAA,GAAgB,sBAAA,CAAuB,KAAA,CAAM,OAAO,CAAA;AAC1D,EAAA,MAAM,WAAA,CAAY,UAAU,aAAa,CAAA;AAC3C;AAEA,eAAe,UAAA,CAAW,MAAA,EAAgB,QAAA,EAAkB,UAAA,EAAoB,QAAA,EAAkB;AAChG,EAAA,IAAI,WAAA;AACJ,EAAA,IAAI,eAAe,OAAA,EAAS;AAE1B,IAAA,IAAI;AACF,MAAA,MAAM,KAAA,GAAQ,MAAA,CAAO,YAAA,CAAa,QAAQ,CAAA;AAC1C,MAAA,MAAM,OAAA,GAAU,MAAM,KAAA,CAAM,WAAA,EAAY;AACxC,MAAA,KAAA,MAAW,CAAC,CAAA,EAAG,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAO,CAAA,EAAG;AACjD,QAAA,IAAI,MAAA,CAAO,MAAA,CAAO,EAAA,KAAO,QAAA,EAAU;AACjC,UAAA,WAAA,GAAc,MAAA;AACd,UAAA;AAAA,QACF;AAAA,MACF;AAAA,IACF,CAAA,CAAA,MAAQ;AAEN,MAAA,IAAI;AACF,QAAA,MAAM,WAAA,GAAe,MAAM,MAAA,CAAO,SAAA,IAAa,KAAA,CAAM,OAAA,CAAQ,QAAQ,CAAA,IAAM,IAAA;AAC3E,QAAA,IAAI,WAAA,EAAa;AACf,UAAA,MAAM,OAAA,GAAU,MAAM,WAAA,CAAY,WAAA,EAAY;AAC9C,UAAA,KAAA,MAAW,CAAC,CAAA,EAAG,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAO,CAAA,EAAsB;AACpE,YAAA,IAAI,MAAA,CAAO,MAAA,CAAO,EAAA,KAAO,QAAA,EAAU;AACjC,cAAA,WAAA,GAAc,MAAA;AACd,cAAA;AAAA,YACF;AAAA,UACF;AAAA,QACF;AAAA,MACF,CAAA,CAAA,MAAQ;AAAA,MAER;AAAA,IACF;AAAA,EACF,CAAA,MAAA,IAAW,eAAe,UAAA,EAAY;AACpC,IAAA,MAAM,UAAU,MAAM,MAAA,CAAO,eAAA,CAAgB,QAAQ,EAAE,WAAA,EAAY;AACnE,IAAA,KAAA,MAAW,CAAC,CAAA,EAAG,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,OAAO,CAAA,EAAG;AACjD,MAAA,IAAI,MAAA,CAAO,MAAA,CAAO,EAAA,KAAO,QAAA,EAAU;AACjC,QAAA,WAAA,GAAc,MAAA;AACd,QAAA;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAGA,EAAA,IAAI,CAAC,WAAA,EAAa;AAChB,IAAA,MAAM,sBAAA,GAAyB,MAAA,CAAO,aAAA,CAAc,QAAQ,CAAA;AAC5D,IAAA,WAAA,GAAc,sBAAA,GAAyB,EAAE,MAAA,EAAQ,sBAAA,EAAuB,GAAI,MAAA;AAAA,EAC9E;AAEA,EAAA,OAAO,WAAA;AACT;;;ACrLO,IAAM,mBAAN,MAAuB;AAAA,EACpB,aAAuC,EAAC;AAAA,EACxC,iBAA2C,EAAC;AAAA,EAC5C,aAAuD,EAAC;AAAA,EAEhE,UAAU,aAAA,EAAoC;AAC5C,IAAA,MAAM,mCAAmC,OAAA,IAAW,aAAA;AACpD,IAAA,IAAI,gCAAA,EAAkC;AACpC,MAAA,IAAA,CAAK,gBAAgB,aAAa,CAAA;AAAA,IACpC,CAAA,MAAO;AACL,MAAA,IAAA,CAAK,cAAc,aAAa,CAAA;AAAA,IAClC;AAAA,EACF;AAAA,EAEQ,cAAc,aAAA,EAAoC;AACxD,IAAA,KAAA,MAAW,CAAC,UAAA,EAAY,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,aAAa,CAAA,EAAG;AAChE,MAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,UAAU,CAAA,EAAG;AAChC,QAAA,IAAA,CAAK,UAAA,CAAW,UAAU,CAAA,GAAI,EAAC;AAAA,MACjC;AACA,MAAA,IAAA,CAAK,UAAA,CAAW,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,IACtE;AAAA,EACF;AAAA,EAEQ,gBAAgB,aAAA,EAAoC;AAC1D,IAAA,IAAI,UAAA,IAAc,aAAA,IAAiB,aAAA,CAAc,QAAA,EAAU;AACzD,MAAA,KAAA,MAAW,CAAC,YAAY,MAAM,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,QAAQ,CAAA,EAAG;AACzE,QAAA,IAAI,CAAC,IAAA,CAAK,cAAA,CAAe,UAAU,CAAA,EAAG;AACpC,UAAA,IAAA,CAAK,cAAA,CAAe,UAAU,CAAA,GAAI,EAAC;AAAA,QACrC;AACA,QAAA,IAAA,CAAK,cAAA,CAAe,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,MAC1E;AAAA,IACF;AAEA,IAAA,IAAI,OAAA,IAAW,aAAA,IAAiB,aAAA,CAAc,KAAA,EAAO;AACnD,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,KAAK,CAAA,EAAG;AACvE,QAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,EAAG;AAC5B,UAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,GAAI,EAAC;AAAA,QAC7B;AACA,QAAA,KAAA,MAAW,CAAC,UAAA,EAAY,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAkC,CAAA,EAAG;AACrF,UAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,CAAA,EAAG;AACxC,YAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,IAAI,EAAC;AAAA,UACzC;AACA,UAAA,IAAA,CAAK,WAAW,MAAM,CAAA,CAAE,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,QAC9E;AAAA,MACF;AAAA,IACF;AAAA,EACF;AAAA,EAEA,cAAc,iBAAA,EAAwD;AACpE,IAAA,KAAA,MAAW,CAAC,MAAA,EAAQ,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,iBAAiB,CAAA,EAAG;AACrE,MAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,EAAG;AAC5B,QAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,GAAI,EAAC;AAAA,MAC7B;AACA,MAAA,KAAA,MAAW,CAAC,UAAA,EAAY,MAAM,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAW,CAAA,EAAG;AAC9D,QAAA,IAAI,CAAC,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,CAAA,EAAG;AACxC,UAAA,IAAA,CAAK,UAAA,CAAW,MAAM,CAAA,CAAE,UAAU,IAAI,EAAC;AAAA,QACzC;AACA,QAAA,IAAA,CAAK,WAAW,MAAM,CAAA,CAAE,UAAU,CAAA,CAAE,IAAA,CAAM,OAA6B,KAAK,CAAA;AAAA,MAC9E;AAAA,IACF;AAAA,EACF;AAAA,EAEA,gBAAA,GAAwC;AACtC,IAAA,MAAM,SAA8B,EAAC;AAErC,IAAA,KAAA,MAAW,CAAC,YAAY,UAAU,CAAA,IAAK,OAAO,OAAA,CAAQ,IAAA,CAAK,UAAU,CAAA,EAAG;AACtE,MAAA,MAAA,CAAO,UAAU,CAAA,GAAI,IAAA,CAAK,eAAA,CAAgB,UAAU,CAAA;AAAA,IACtD;AAGA,IAAA,IAAI,OAAO,IAAA,CAAK,IAAA,CAAK,cAAc,CAAA,CAAE,SAAS,CAAA,EAAG;AAC/C,MAAA,MAAA,CAAO,WAAW,EAAC;AACnB,MAAA,KAAA,MAAW,CAAC,YAAY,UAAU,CAAA,IAAK,OAAO,OAAA,CAAQ,IAAA,CAAK,cAAc,CAAA,EAAG;AAC1E,QAAA,MAAA,CAAO,QAAA,CAAS,UAAU,CAAA,GAAI,IAAA,CAAK,gBAAgB,UAAU,CAAA;AAAA,MAC/D;AAAA,IACF;AAEA,IAAA,IAAI,OAAO,IAAA,CAAK,IAAA,CAAK,UAAU,CAAA,CAAE,SAAS,CAAA,EAAG;AAC3C,MAAA,MAAA,CAAO,QAAQ,EAAC;AAChB,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,IAAA,CAAK,UAAU,CAAA,EAAG;AACnE,QAAA,MAAA,CAAO,KAAA,CAAM,MAAM,CAAA,GAAI,EAAC;AACxB,QAAA,KAAA,MAAW,CAAC,UAAA,EAAY,UAAU,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAW,CAAA,EAAG;AAClE,UAAA,MAAA,CAAO,MAAM,MAAM,CAAA,CAAE,UAAU,CAAA,GAAI,IAAA,CAAK,gBAAgB,UAAU,CAAA;AAAA,QACpE;AAAA,MACF;AAAA,IACF;AAEA,IAAA,OAAO,MAAA;AAAA,EACT;AAAA,EAEQ,gBAAgB,UAAA,EAA8B;AACpD,IAAA,IAAI,UAAA,CAAW,SAAS,CAAA,EAAG;AACzB,MAAA,OAAO,UAAA,CAAW,OAAO,CAAC,CAAA,EAAG,MAAM,CAAA,GAAI,CAAA,EAAG,CAAC,CAAA,GAAI,UAAA,CAAW,MAAA;AAAA,IAC5D,CAAA,MAAO;AACL,MAAA,OAAO,CAAA;AAAA,IACT;AAAA,EACF;AACF,CAAA;;;ACbA,eAAsB,SAAS,MAAA,EAaH;AAC1B,EAAA,MAAM,EAAE,MAAM,OAAA,EAAS,MAAA,EAAQ,eAAe,cAAA,EAAgB,WAAA,GAAc,GAAE,GAAI,MAAA;AAElF,EAAA,mBAAA,CAAoB,IAAA,EAAM,SAAS,MAAM,CAAA;AAEzC,EAAA,IAAI,UAAA,GAAa,CAAA;AACjB,EAAA,MAAM,gBAAA,GAAmB,IAAI,gBAAA,EAAiB;AAI9C,EAAA,MAAM,MAAA,GAAU,MAAA,CAAe,iBAAA,IAAoB,IAAM,MAAA,CAAe,MAAA;AACxE,EAAA,MAAM,OAAA,GAAU,QAAQ,UAAA,EAAW;AAEnC,EAAA,MAAMA,KAAAA,GAAAA,CAAQ,MAAM,OAAO,OAAO,CAAA,EAAG,OAAA;AACrC,EAAA,MAAMA,KAAAA;AAAA,IACJ,IAAA;AAAA,IACA,OAAO,IAAA,KAAgC;AACrC,MAAA,MAAM,YAAA,GAAe,MAAM,aAAA,CAAc,MAAA,EAAQ,MAAM,aAAa,CAAA;AACpE,MAAA,MAAM,aAAA,GAAgB,MAAM,UAAA,CAAW,OAAA,EAAS,cAAc,IAAI,CAAA;AAClE,MAAA,gBAAA,CAAiB,UAAU,aAAa,CAAA;AAGxC,MAAA,IAAI,OAAA,EAAS;AACX,QAAA,MAAM,mBAAA,CAAoB;AAAA,UACxB,OAAA;AAAA,UACA,aAAA;AAAA,UACA,MAAA;AAAA,UACA,IAAA;AAAA,UACA;AAAA,SACD,CAAA;AAAA,MACH;AAEA,MAAA,IAAI,cAAA,EAAgB;AAClB,QAAA,MAAM,cAAA,CAAe;AAAA,UACnB,IAAA;AAAA,UACA,YAAA;AAAA,UACA;AAAA,SACD,CAAA;AAAA,MACH;AAEA,MAAA,UAAA,EAAA;AAAA,IACF,CAAA;AAAA,IACA,EAAE,WAAA;AAAY,GAChB;AAEA,EAAA,OAAO;AAAA,IACL,MAAA,EAAQ,iBAAiB,gBAAA,EAAiB;AAAA,IAC1C,OAAA,EAAS;AAAA,MACP;AAAA;AACF,GACF;AACF;AAEA,SAAS,WAAW,MAAA,EAA8C;AAChE,EAAA,OAAO,MAAA,YAAkB,QAAA;AAC3B;AAEA,SAAS,uBAAuB,OAAA,EAA+C;AAC7E,EAAA,OAAO,OAAO,OAAA,KAAY,QAAA,IAAY,CAAC,KAAA,CAAM,QAAQ,OAAO,CAAA,KAAM,UAAA,IAAc,OAAA,IAAW,OAAA,IAAW,OAAA,CAAA;AACxG;AAEA,SAAS,mBAAA,CACP,IAAA,EACA,OAAA,EACA,MAAA,EACM;AACN,EAAA,IAAI,IAAA,CAAK,WAAW,CAAA,EAAG;AACrB,IAAA,MAAM,IAAI,WAAA,CAAY;AAAA,MACpB,MAAA,EAAQ,QAAA;AAAA,MACR,EAAA,EAAI,wCAAA;AAAA,MACJ,QAAA,EAAU,MAAA;AAAA,MACV,IAAA,EAAM;AAAA,KACP,CAAA;AAAA,EACH;AAEA,EAAA,KAAA,IAAS,CAAA,GAAI,CAAA,EAAG,CAAA,GAAI,IAAA,CAAK,QAAQ,CAAA,EAAA,EAAK;AACpC,IAAA,MAAM,IAAA,GAAO,KAAK,CAAC,CAAA;AACnB,IAAA,IAAI,CAAC,IAAA,IAAQ,OAAO,SAAS,QAAA,IAAY,EAAE,WAAW,IAAA,CAAA,EAAO;AAC3D,MAAA,MAAM,IAAI,WAAA,CAAY;AAAA,QACpB,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,mBAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM,8BAA8B,CAAC,CAAA,8BAAA;AAAA,OACtC,CAAA;AAAA,IACH;AAAA,EACF;AAGA,EAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AAC1B,IAAA,IAAI,OAAA,CAAQ,WAAW,CAAA,EAAG;AACxB,MAAA,MAAM,IAAI,WAAA,CAAY;AAAA,QACpB,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,qBAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM;AAAA,OACP,CAAA;AAAA,IACH;AAAA,EACF,WAAW,UAAA,CAAW,MAAM,CAAA,IAAK,sBAAA,CAAuB,OAAO,CAAA,EAAG;AAChE,IAAA,MAAM,UAAA,GACH,OAAA,CAAQ,QAAA,IAAY,OAAA,CAAQ,SAAS,MAAA,GAAS,CAAA,IAAO,OAAA,CAAQ,KAAA,IAAS,MAAA,CAAO,IAAA,CAAK,OAAA,CAAQ,KAAK,EAAE,MAAA,GAAS,CAAA;AAE7G,IAAA,IAAI,CAAC,UAAA,EAAY;AACf,MAAA,MAAM,IAAI,WAAA,CAAY;AAAA,QACpB,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,qBAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM;AAAA,OACP,CAAA;AAAA,IACH;AAAA,EACF,CAAA,MAAA,IAAW,CAAC,UAAA,CAAW,MAAM,KAAK,CAAC,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AACzD,IAAA,MAAM,IAAI,WAAA,CAAY;AAAA,MACpB,MAAA,EAAQ,QAAA;AAAA,MACR,EAAA,EAAI,uBAAA;AAAA,MACJ,QAAA,EAAU,MAAA;AAAA,MACV,IAAA,EAAM;AAAA,KACP,CAAA;AAAA,EACH;AACF;AAEA,eAAe,aAAA,CACb,MAAA,EACA,IAAA,EACA,aAAA,EAGA;AACA,EAAA,IAAI;AACF,IAAA,IAAI,UAAA,CAAW,MAAM,CAAA,EAAG;AACtB,MAAA,OAAO,MAAM,eAAA,CAAgB,MAAA,EAAQ,IAAA,EAAM,aAAmC,CAAA;AAAA,IAChF,CAAA,MAAO;AACL,MAAA,OAAO,MAAM,YAAA;AAAA,QACX,MAAA;AAAA,QACA,IAAA;AAAA,QACA;AAAA,OACF;AAAA,IACF;AAAA,EACF,SAAS,KAAA,EAAO;AACd,IAAA,MAAM,IAAI,WAAA;AAAA,MACR;AAAA,QACE,MAAA,EAAQ,QAAA;AAAA,QACR,EAAA,EAAI,iDAAA;AAAA,QACJ,QAAA,EAAU,MAAA;AAAA,QACV,IAAA,EAAM,+DAAA;AAAA,QACN,OAAA,EAAS;AAAA,UACP,IAAA,EAAM,IAAA,CAAK,SAAA,CAAU,IAAI;AAAA;AAC3B,OACF;AAAA,MACA;AAAA,KACF;AAAA,EACF;AACF;AAEA,eAAe,eAAA,CAAgB,MAAA,EAAkB,IAAA,EAA6B,aAAA,EAAoC;AAChH,EAAA,MAAM,oBAAA,GAAuB,4BAA4B,IAAI,CAAA;AAC7D,EAAA,MAAM,MAAM,MAAM,MAAA,CAAO,UAAU,EAAE,cAAA,EAAgB,MAAM,CAAA;AAC3D,EAAA,MAAM,cAAA,GAAiB,MAAM,GAAA,CAAI,KAAA,CAAM;AAAA,IACrC,GAAG,aAAA;AAAA,IACH,GAAG,IAAA,CAAK,YAAA;AAAA,IACR,WAAW,IAAA,CAAK,KAAA;AAAA,IAChB,gBAAgB,IAAA,CAAK,cAAA;AAAA,IACrB,GAAG;AAAA,GACJ,CAAA;AAED,EAAA,OAAO;AAAA,IACL,WAAA,EAAa;AAAA,MACX,OAAO,IAAA,CAAK,KAAA;AAAA,MACZ,MAAA,EAAQ,cAAA,CAAe,MAAA,KAAW,SAAA,GAAY,eAAe,MAAA,GAAS,MAAA;AAAA,MACtE,aAAa,cAAA,CAAe;AAAA;AAC9B,GACF;AACF;AAEA,eAAe,YAAA,CACb,KAAA,EACA,IAAA,EACA,aAAA,EACA;AACA,EAAA,MAAM,oBAAA,GAAuB,4BAA4B,IAAI,CAAA;AAC7D,EAAA,MAAM,KAAA,GAAQ,MAAM,KAAA,CAAM,QAAA,EAAS;AACnC,EAAA,IAAI,wBAAA,CAAyB,KAAK,CAAA,EAAG;AACnC,IAAA,MAAM,EAAE,gBAAA,EAAkB,GAAG,WAAA,EAAY,GAAI,iBAAiB,EAAC;AAC/D,IAAA,MAAM,WAAA,GAAc;AAAA,MAClB,GAAG,WAAA;AAAA,MACH,GAAG,oBAAA;AAAA,MACH,SAAS,EAAC;AAAA,MACV,gBAAA,EAAkB,IAAA;AAAA,MAClB,gBAAgB,IAAA,CAAK;AAAA,KACvB;AACA,IAAA,OAAO,mBACH,MAAM,KAAA,CAAM,QAAA,CAAS,IAAA,CAAK,OAAO,EAAE,GAAG,WAAA,EAAa,gBAAA,EAAkB,CAAA,GACrE,MAAM,MAAM,QAAA,CAAS,IAAA,CAAK,OAAO,WAAW,CAAA;AAAA,EAClD,CAAA,MAAO;AACL,IAAA,OAAO,MAAM,KAAA,CAAM,cAAA,CAAe,IAAA,CAAK,KAAA,EAAO;AAAA,MAC5C,SAAS,EAAC;AAAA,MACV,gBAAA,EAAkB,IAAA;AAAA,MAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,MACrB,GAAG;AAAA,KACJ,CAAA;AAAA,EACH;AACF;AAEA,eAAe,UAAA,CACb,OAAA,EACA,YAAA,EACA,IAAA,EAC8B;AAC9B,EAAA,MAAM,gBAAqC,EAAC;AAE5C,EAAA,IAAI,KAAA,CAAM,OAAA,CAAQ,OAAO,CAAA,EAAG;AAC1B,IAAA,KAAA,MAAW,UAAU,OAAA,EAAS;AAC5B,MAAA,IAAI;AACF,QAAA,MAAM,KAAA,GAAQ,MAAM,MAAA,CAAO,GAAA,CAAI;AAAA,UAC7B,KAAA,EAAO,aAAa,WAAA,EAAa,KAAA;AAAA,UACjC,MAAA,EAAQ,aAAa,WAAA,EAAa,MAAA;AAAA,UAClC,aAAa,IAAA,CAAK,WAAA;AAAA,UAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,UACrB,GAAG,4BAA4B,IAAI;AAAA,SACpC,CAAA;AAED,QAAA,aAAA,CAAc,MAAA,CAAO,EAAE,CAAA,GAAI,KAAA;AAAA,MAC7B,SAAS,KAAA,EAAO;AACd,QAAA,MAAM,IAAI,WAAA;AAAA,UACR;AAAA,YACE,MAAA,EAAQ,QAAA;AAAA,YACR,EAAA,EAAI,8CAAA;AAAA,YACJ,QAAA,EAAU,MAAA;AAAA,YACV,IAAA,EAAM,CAAA,+CAAA,EAAkD,MAAA,CAAO,EAAE,CAAA,CAAA;AAAA,YACjE,OAAA,EAAS;AAAA,cACP,UAAU,MAAA,CAAO,EAAA;AAAA,cACjB,IAAA,EAAM,IAAA,CAAK,SAAA,CAAU,IAAI;AAAA;AAC3B,WACF;AAAA,UACA;AAAA,SACF;AAAA,MACF;AAAA,IACF;AAAA,EACF,CAAA,MAAO;AAEL,IAAA,IAAI,QAAQ,QAAA,EAAU;AACpB,MAAA,MAAM,wBAA6C,EAAC;AACpD,MAAA,KAAA,MAAW,MAAA,IAAU,QAAQ,QAAA,EAAU;AACrC,QAAA,MAAM,KAAA,GAAQ,MAAM,MAAA,CAAO,GAAA,CAAI;AAAA,UAC7B,KAAA,EAAO,aAAa,WAAA,CAAY,KAAA;AAAA,UAChC,MAAA,EAAQ,aAAa,WAAA,CAAY,MAAA;AAAA,UACjC,aAAa,IAAA,CAAK,WAAA;AAAA,UAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,UACrB,GAAG,4BAA4B,IAAI;AAAA,SACpC,CAAA;AACD,QAAA,qBAAA,CAAsB,MAAA,CAAO,EAAE,CAAA,GAAI,KAAA;AAAA,MACrC;AACA,MAAA,IAAI,MAAA,CAAO,IAAA,CAAK,qBAAqB,CAAA,CAAE,SAAS,CAAA,EAAG;AACjD,QAAA,aAAA,CAAc,QAAA,GAAW,qBAAA;AAAA,MAC3B;AAAA,IACF;AAEA,IAAA,IAAI,QAAQ,KAAA,EAAO;AACjB,MAAA,MAAM,oBAAyC,EAAC;AAChD,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,OAAA,CAAQ,KAAK,CAAA,EAAG;AACjE,QAAA,MAAM,UAAA,GAAa,YAAA,CAAa,WAAA,CAAY,WAAA,GAAc,MAAM,CAAA;AAChE,QAAA,IAAI,UAAA,EAAY,MAAA,KAAW,SAAA,IAAa,UAAA,CAAW,WAAW,MAAA,EAAW;AACvE,UAAA,MAAM,cAAmC,EAAC;AAC1C,UAAA,KAAA,MAAW,UAAU,WAAA,EAAa;AAChC,YAAA,IAAI;AACF,cAAA,MAAM,KAAA,GAAQ,MAAM,MAAA,CAAO,GAAA,CAAI;AAAA,gBAC7B,OAAO,UAAA,CAAW,OAAA,KAAY,SAAY,UAAA,CAAW,OAAA,GAAU,aAAa,WAAA,CAAY,KAAA;AAAA,gBACxF,QAAQ,UAAA,CAAW,MAAA;AAAA,gBACnB,aAAa,IAAA,CAAK,WAAA;AAAA,gBAClB,gBAAgB,IAAA,CAAK,cAAA;AAAA,gBACrB,GAAG,4BAA4B,IAAI;AAAA,eACpC,CAAA;AACD,cAAA,WAAA,CAAY,MAAA,CAAO,EAAE,CAAA,GAAI,KAAA;AAAA,YAC3B,SAAS,KAAA,EAAO;AACd,cAAA,MAAM,IAAI,WAAA;AAAA,gBACR;AAAA,kBACE,MAAA,EAAQ,QAAA;AAAA,kBACR,EAAA,EAAI,mDAAA;AAAA,kBACJ,QAAA,EAAU,MAAA;AAAA,kBACV,IAAA,EAAM,CAAA,+CAAA,EAAkD,MAAA,CAAO,EAAE,YAAY,MAAM,CAAA,CAAA;AAAA,kBACnF,OAAA,EAAS;AAAA,oBACP,UAAU,MAAA,CAAO,EAAA;AAAA,oBACjB;AAAA;AACF,iBACF;AAAA,gBACA;AAAA,eACF;AAAA,YACF;AAAA,UACF;AACA,UAAA,IAAI,MAAA,CAAO,IAAA,CAAK,WAAW,CAAA,CAAE,SAAS,CAAA,EAAG;AACvC,YAAA,iBAAA,CAAkB,MAAM,CAAA,GAAI,WAAA;AAAA,UAC9B;AAAA,QACF;AAAA,MACF;AACA,MAAA,IAAI,MAAA,CAAO,IAAA,CAAK,iBAAiB,CAAA,CAAE,SAAS,CAAA,EAAG;AAC7C,QAAA,aAAA,CAAc,KAAA,GAAQ,iBAAA;AAAA,MACxB;AAAA,IACF;AAAA,EACF;AAEA,EAAA,OAAO,aAAA;AACT;AAMA,eAAe,mBAAA,CAAoB;AAAA,EACjC,OAAA;AAAA,EACA,aAAA;AAAA,EACA,MAAA;AAAA,EACA,IAAA;AAAA,EACA;AACF,CAAA,EAMkB;AAChB,EAAA,MAAM,WAAW,MAAA,CAAO,EAAA;AACxB,EAAA,MAAM,UAAA,GAAa,UAAA,CAAW,MAAM,CAAA,GAAI,UAAA,GAAa,OAAA;AAGrD,EAAA,IAAI,KAAA,CAAM,QAAQ,aAAa,CAAA,IAAK,EAAE,UAAA,IAAc,aAAA,IAAiB,WAAW,aAAA,CAAA,EAAgB;AAC9F,IAAA,KAAA,MAAW,CAAC,QAAA,EAAU,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,aAAa,CAAA,EAAG;AACnE,MAAA,IAAI,WAAA,IAAe,OAAO,WAAA,KAAgB,QAAA,IAAY,WAAW,WAAA,EAAa;AAC5E,QAAA,MAAM,eAAA,CAAgB;AAAA,UACpB,OAAA;AAAA,UACA,WAAA;AAAA,UACA,QAAA;AAAA,UACA,QAAA;AAAA,UACA,UAAA;AAAA,UACA,MAAA;AAAA,UACA,MAAA;AAAA,UACA;AAAA,SACD,CAAA;AAAA,MACH;AAAA,IACF;AAAA,EACF,CAAA,MAAO;AAEL,IAAA,IAAI,cAAc,QAAA,EAAU;AAC1B,MAAA,KAAA,MAAW,CAAC,UAAU,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,QAAQ,CAAA,EAAG;AAC5E,QAAA,IAAI,WAAA,IAAe,OAAO,WAAA,KAAgB,QAAA,IAAY,WAAW,WAAA,EAAa;AAC5E,UAAA,MAAM,eAAA,CAAgB;AAAA,YACpB,OAAA;AAAA,YACA,WAAA;AAAA,YACA,QAAA;AAAA,YACA,QAAA;AAAA,YACA,UAAA,EAAY,UAAA;AAAA,YACZ,MAAA;AAAA,YACA,MAAA;AAAA,YACA;AAAA,WACD,CAAA;AAAA,QACH;AAAA,MACF;AAAA,IACF;AAEA,IAAA,IAAI,cAAc,KAAA,EAAO;AACvB,MAAA,KAAA,MAAW,CAAC,QAAQ,WAAW,CAAA,IAAK,OAAO,OAAA,CAAQ,aAAA,CAAc,KAAK,CAAA,EAAG;AACvE,QAAA,KAAA,MAAW,CAAC,QAAA,EAAU,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,WAAkC,CAAA,EAAG;AACxF,UAAA,IAAI,WAAA,IAAe,OAAO,WAAA,KAAgB,QAAA,IAAY,WAAW,WAAA,EAAa;AAC5E,YAAA,MAAM,eAAA,CAAgB;AAAA,cACpB,OAAA;AAAA,cACA,WAAA;AAAA,cACA,QAAA;AAAA,cACA,QAAA,EAAU,MAAA;AAAA,cACV,UAAA,EAAY,MAAA;AAAA,cACZ,MAAA;AAAA,cACA,MAAA;AAAA,cACA;AAAA,aACD,CAAA;AAAA,UACH;AAAA,QACF;AAAA,MACF;AAAA,IACF;AAAA,EACF;AACF;AAKA,eAAe,eAAA,CAAgB;AAAA,EAC7B,OAAA;AAAA,EACA,WAAA;AAAA,EACA,QAAA;AAAA,EACA,QAAA;AAAA,EACA,UAAA;AAAA,EACA,MAAA;AAAA,EACA,MAAA;AAAA,EACA;AACF,CAAA,EASkB;AAChB,EAAA,IAAI;AAEF,IAAA,IAAI,MAAA,GAAS,MAAA,EAAQ,aAAA,GAAgB,QAAQ,CAAA;AAE7C,IAAA,IAAI,CAAC,MAAA,EAAQ;AAEX,MAAA,MAAM,aAAA,GAAgB,MAAO,MAAA,CAAe,WAAA,IAAc;AAC1D,MAAA,IAAI,aAAA,EAAe;AACjB,QAAA,KAAA,MAAW,CAAC,CAAA,EAAG,WAAW,KAAK,MAAA,CAAO,OAAA,CAAQ,aAAa,CAAA,EAAG;AAC5D,UAAA,IAAK,WAAA,CAAoB,MAAA,EAAQ,EAAA,KAAO,QAAA,EAAU;AAChD,YAAA,MAAA,GAAU,WAAA,CAAoB,MAAA;AAC9B,YAAA;AAAA,UACF;AAAA,QACF;AAAA,MACF;AAAA,IACF;AAGA,IAAA,IAAI,OAAA;AACJ,IAAA,IAAI,MAAA;AACJ,IAAA,IAAI,KAAK,cAAA,EAAgB,WAAA,IAAe,IAAA,CAAK,cAAA,CAAe,YAAY,OAAA,EAAS;AAC/E,MAAA,MAAA,GAAS,IAAA,CAAK,eAAe,WAAA,CAAY,EAAA;AACzC,MAAA,OAAA,GAAU,IAAA,CAAK,eAAe,WAAA,CAAY,OAAA;AAAA,IAC5C;AAGA,IAAA,MAAM,oBAAyC,EAAC;AAChD,IAAA,IAAI,IAAA,CAAK,gBAAgB,MAAA,EAAW;AAClC,MAAA,iBAAA,CAAkB,cAAc,IAAA,CAAK,WAAA;AAAA,IACvC;AAEA,IAAA,MAAM,OAAA,GAAU;AAAA,MACd,GAAG,WAAA;AAAA,MACH,QAAA;AAAA,MACA,QAAA;AAAA,MACA,UAAA;AAAA,MACA,MAAA,EAAQ,MAAA;AAAA,MACR,MAAA,EAAQ;AAAA,QACN,EAAA,EAAI,QAAQ,EAAA,IAAM,QAAA;AAAA,QAClB,IAAA,EAAM,QAAQ,IAAA,IAAQ,QAAA;AAAA,QACtB,WAAA,EAAa,QAAQ,WAAA,IAAe,EAAA;AAAA,QACpC,IAAA,EAAM,QAAQ,IAAA,IAAQ,SAAA;AAAA,QACtB,GAAI,SAAS,EAAE,QAAA,EAAU,CAAC,CAAC,MAAA,CAAO,KAAA,EAAM,GAAI;AAAC,OAC/C;AAAA,MACA,MAAA,EAAQ;AAAA,QACN,IAAI,MAAA,CAAO,EAAA;AAAA,QACX,IAAA,EAAO,MAAA,CAAe,IAAA,IAAQ,MAAA,CAAO;AAAA,OACvC;AAAA;AAAA,MAEA,cAAA,EAAgB,KAAK,cAAA,GAAiB,MAAA,CAAO,YAAY,IAAA,CAAK,cAAA,CAAe,OAAA,EAAS,CAAA,GAAI,MAAA;AAAA;AAAA,MAE1F,mBAAmB,MAAA,CAAO,IAAA,CAAK,iBAAiB,CAAA,CAAE,MAAA,GAAS,IAAI,iBAAA,GAAoB,MAAA;AAAA;AAAA,MAEnF,OAAA;AAAA,MACA;AAAA,KACF;AAEA,IAAA,MAAM,oBAAA,CAAqB,SAAS,OAAO,CAAA;AAAA,EAC7C,SAAS,KAAA,EAAO;AAEd,IAAA,MAAA,EAAQ,aAAY,EAAG,IAAA,GAAO,CAAA,gCAAA,EAAmC,QAAQ,KAAK,KAAK,CAAA;AAAA,EACrF;AACF","file":"chunk-23RPY53R.js","sourcesContent":["import pMap from 'p-map';\nimport { ErrorCategory, ErrorDomain, MastraError } from '../error';\nimport { saveScorePayloadSchema } from '../evals';\nimport type { ScoringHookInput } from '../evals/types';\nimport type { Mastra } from '../mastra';\nimport type { MastraStorage } from '../storage';\n\nexport function createOnScorerHook(mastra: Mastra) {\n  return async (hookData: ScoringHookInput) => {\n    const storage = mastra.getStorage();\n\n    if (!storage) {\n      mastra.getLogger()?.warn('Storage not found, skipping score validation and saving');\n      return;\n    }\n\n    const entityId = hookData.entity.id as string;\n    const entityType = hookData.entityType;\n    const scorer = hookData.scorer;\n    const scorerId = scorer.id as string;\n\n    if (!scorerId) {\n      mastra.getLogger()?.warn('Scorer ID not found, skipping score validation and saving');\n      return;\n    }\n\n    try {\n      const scorerToUse = await findScorer(mastra, entityId, entityType, scorerId);\n\n      if (!scorerToUse) {\n        throw new MastraError({\n          id: 'MASTRA_SCORER_NOT_FOUND',\n          domain: ErrorDomain.MASTRA,\n          category: ErrorCategory.USER,\n          text: `Scorer with ID ${scorerId} not found`,\n        });\n      }\n\n      let input = hookData.input;\n      let output = hookData.output;\n\n      const { structuredOutput, ...rest } = hookData;\n\n      const runResult = await scorerToUse.scorer.run({\n        ...rest,\n        input,\n        output,\n      });\n\n      let spanId;\n      let traceId;\n      const currentSpan = hookData.tracingContext?.currentSpan;\n      if (currentSpan && currentSpan.isValid) {\n        spanId = currentSpan.id;\n        traceId = currentSpan.traceId;\n      }\n\n      const payload = {\n        ...rest,\n        ...runResult,\n        entityId,\n        scorerId: scorerId,\n        spanId,\n        traceId,\n        scorer: {\n          ...rest.scorer,\n          hasJudge: !!scorerToUse.scorer.judge,\n        },\n        metadata: {\n          structuredOutput: !!structuredOutput,\n        },\n      };\n      await validateAndSaveScore(storage, payload);\n\n      if (currentSpan && spanId && traceId) {\n        await pMap(\n          currentSpan.observabilityInstance.getExporters(),\n          async exporter => {\n            if (exporter.addScoreToTrace) {\n              try {\n                await exporter.addScoreToTrace({\n                  traceId: traceId,\n                  spanId: spanId,\n                  score: runResult.score as number,\n                  reason: runResult.reason as string,\n                  scorerName: scorerToUse.scorer.id,\n                  metadata: {\n                    ...(currentSpan.metadata ?? {}),\n                  },\n                });\n              } catch (error) {\n                // Log error but don't fail the hook if exporter fails\n                mastra.getLogger()?.error(`Failed to add score to trace via exporter: ${error}`);\n              }\n            }\n          },\n          { concurrency: 3 },\n        );\n      }\n    } catch (error) {\n      const mastraError = new MastraError(\n        {\n          id: 'MASTRA_SCORER_FAILED_TO_RUN_HOOK',\n          domain: ErrorDomain.SCORER,\n          category: ErrorCategory.USER,\n          details: {\n            scorerId,\n            entityId,\n            entityType,\n          },\n        },\n        error,\n      );\n\n      mastra.getLogger()?.trackException(mastraError);\n      mastra.getLogger()?.error(mastraError.toString());\n    }\n  };\n}\n\nexport async function validateAndSaveScore(storage: MastraStorage, payload: unknown) {\n  const scoresStore = await storage.getStore('scores');\n  if (!scoresStore) {\n    throw new MastraError({\n      id: 'MASTRA_SCORES_STORAGE_NOT_AVAILABLE',\n      domain: ErrorDomain.STORAGE,\n      category: ErrorCategory.SYSTEM,\n      text: 'Scores storage domain is not available',\n    });\n  }\n  const payloadToSave = saveScorePayloadSchema.parse(payload);\n  await scoresStore.saveScore(payloadToSave);\n}\n\nasync function findScorer(mastra: Mastra, entityId: string, entityType: string, scorerId: string) {\n  let scorerToUse;\n  if (entityType === 'AGENT') {\n    // Try code-defined agents first\n    try {\n      const agent = mastra.getAgentById(entityId);\n      const scorers = await agent.listScorers();\n      for (const [_, scorer] of Object.entries(scorers)) {\n        if (scorer.scorer.id === scorerId) {\n          scorerToUse = scorer;\n          break;\n        }\n      }\n    } catch {\n      // Agent not found in code-defined agents, try stored agents via editor\n      try {\n        const storedAgent = (await mastra.getEditor()?.agent.getById(entityId)) ?? null;\n        if (storedAgent) {\n          const scorers = await storedAgent.listScorers();\n          for (const [_, scorer] of Object.entries(scorers) as [string, any][]) {\n            if (scorer.scorer.id === scorerId) {\n              scorerToUse = scorer;\n              break;\n            }\n          }\n        }\n      } catch {\n        // Stored agent also not found, will fall back to mastra-registered scorer\n      }\n    }\n  } else if (entityType === 'WORKFLOW') {\n    const scorers = await mastra.getWorkflowById(entityId).listScorers();\n    for (const [_, scorer] of Object.entries(scorers)) {\n      if (scorer.scorer.id === scorerId) {\n        scorerToUse = scorer;\n        break;\n      }\n    }\n  }\n\n  // Fallback to mastra-registered scorer\n  if (!scorerToUse) {\n    const mastraRegisteredScorer = mastra.getScorerById(scorerId);\n    scorerToUse = mastraRegisteredScorer ? { scorer: mastraRegisteredScorer } : undefined;\n  }\n\n  return scorerToUse;\n}\n","export class ScoreAccumulator {\n  private flatScores: Record<string, number[]> = {};\n  private workflowScores: Record<string, number[]> = {};\n  private stepScores: Record<string, Record<string, number[]>> = {};\n\n  addScores(scorerResults: Record<string, any>) {\n    const isTargetWorkflowAndHasStepScores = 'steps' in scorerResults;\n    if (isTargetWorkflowAndHasStepScores) {\n      this.addNestedScores(scorerResults);\n    } else {\n      this.addFlatScores(scorerResults);\n    }\n  }\n\n  private addFlatScores(scorerResults: Record<string, any>) {\n    for (const [scorerName, result] of Object.entries(scorerResults)) {\n      if (!this.flatScores[scorerName]) {\n        this.flatScores[scorerName] = [];\n      }\n      this.flatScores[scorerName].push((result as { score: number }).score);\n    }\n  }\n\n  private addNestedScores(scorerResults: Record<string, any>) {\n    if ('workflow' in scorerResults && scorerResults.workflow) {\n      for (const [scorerName, result] of Object.entries(scorerResults.workflow)) {\n        if (!this.workflowScores[scorerName]) {\n          this.workflowScores[scorerName] = [];\n        }\n        this.workflowScores[scorerName].push((result as { score: number }).score);\n      }\n    }\n\n    if ('steps' in scorerResults && scorerResults.steps) {\n      for (const [stepId, stepResults] of Object.entries(scorerResults.steps)) {\n        if (!this.stepScores[stepId]) {\n          this.stepScores[stepId] = {};\n        }\n        for (const [scorerName, result] of Object.entries(stepResults as Record<string, any>)) {\n          if (!this.stepScores[stepId][scorerName]) {\n            this.stepScores[stepId][scorerName] = [];\n          }\n          this.stepScores[stepId][scorerName].push((result as { score: number }).score);\n        }\n      }\n    }\n  }\n\n  addStepScores(stepScorerResults: Record<string, Record<string, any>>) {\n    for (const [stepId, stepResults] of Object.entries(stepScorerResults)) {\n      if (!this.stepScores[stepId]) {\n        this.stepScores[stepId] = {};\n      }\n      for (const [scorerName, result] of Object.entries(stepResults)) {\n        if (!this.stepScores[stepId][scorerName]) {\n          this.stepScores[stepId][scorerName] = [];\n        }\n        this.stepScores[stepId][scorerName].push((result as { score: number }).score);\n      }\n    }\n  }\n\n  getAverageScores(): Record<string, any> {\n    const result: Record<string, any> = {};\n\n    for (const [scorerName, scoreArray] of Object.entries(this.flatScores)) {\n      result[scorerName] = this.getAverageScore(scoreArray);\n    }\n\n    // Add workflow scores\n    if (Object.keys(this.workflowScores).length > 0) {\n      result.workflow = {};\n      for (const [scorerName, scoreArray] of Object.entries(this.workflowScores)) {\n        result.workflow[scorerName] = this.getAverageScore(scoreArray);\n      }\n    }\n\n    if (Object.keys(this.stepScores).length > 0) {\n      result.steps = {};\n      for (const [stepId, stepScorers] of Object.entries(this.stepScores)) {\n        result.steps[stepId] = {};\n        for (const [scorerName, scoreArray] of Object.entries(stepScorers)) {\n          result.steps[stepId][scorerName] = this.getAverageScore(scoreArray);\n        }\n      }\n    }\n\n    return result;\n  }\n\n  private getAverageScore(scoreArray: number[]): number {\n    if (scoreArray.length > 0) {\n      return scoreArray.reduce((a, b) => a + b, 0) / scoreArray.length;\n    } else {\n      return 0;\n    }\n  }\n}\n","import type { CoreMessage } from '@internal/ai-sdk-v4';\nimport type { Agent, AgentExecutionOptions, AiMessageType, UIMessageWithMetadata } from '../../agent';\nimport { isSupportedLanguageModel } from '../../agent';\nimport { MastraError } from '../../error';\nimport { validateAndSaveScore } from '../../mastra/hooks';\nimport type { ObservabilityContext } from '../../observability';\nimport { resolveObservabilityContext } from '../../observability';\nimport type { RequestContext } from '../../request-context';\nimport { Workflow } from '../../workflows';\nimport type { AnyWorkflow, WorkflowResult, WorkflowRunStartOptions, StepResult } from '../../workflows';\nimport type { MastraScorer } from '../base';\nimport { ScoreAccumulator } from './scorerAccumulator';\n\ntype WorkflowRunOptions = WorkflowRunStartOptions & {\n  initialState?: any;\n};\n\ntype RunEvalsDataItem<TTarget = unknown> = {\n  input: TTarget extends Workflow<any, any>\n    ? any\n    : TTarget extends Agent\n      ? string | string[] | CoreMessage[] | AiMessageType[] | UIMessageWithMetadata[]\n      : unknown;\n  groundTruth?: any;\n  requestContext?: RequestContext;\n  startOptions?: WorkflowRunOptions;\n} & Partial<ObservabilityContext>;\n\ntype WorkflowScorerConfig = {\n  workflow?: MastraScorer<any, any, any, any>[];\n  steps?: Record<string, MastraScorer<any, any, any, any>[]>;\n};\n\ntype RunEvalsResult = {\n  scores: Record<string, any>;\n  summary: {\n    totalItems: number;\n  };\n};\n\n// Agent with scorers array\nexport function runEvals<TAgent extends Agent>(config: {\n  data: RunEvalsDataItem<TAgent>[];\n  scorers: MastraScorer<any, any, any, any>[];\n  target: TAgent;\n  targetOptions?: Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<TAgent>;\n    targetResult: Awaited<ReturnType<Agent['generate']>>;\n    scorerResults: Record<string, any>; // Flat structure: { scorerName: result }\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult>;\n\n// Workflow with scorers array\nexport function runEvals<TWorkflow extends AnyWorkflow>(config: {\n  data: RunEvalsDataItem<TWorkflow>[];\n  scorers: MastraScorer<any, any, any, any>[];\n  target: TWorkflow;\n  targetOptions?: WorkflowRunOptions;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<TWorkflow>;\n    targetResult: WorkflowResult<any, any, any, any>;\n    scorerResults: Record<string, any>; // Flat structure: { scorerName: result }\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult>;\n\n// Workflow with workflow configuration\nexport function runEvals<TWorkflow extends AnyWorkflow>(config: {\n  data: RunEvalsDataItem<TWorkflow>[];\n  scorers: WorkflowScorerConfig;\n  target: TWorkflow;\n  targetOptions?: WorkflowRunOptions;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<TWorkflow>;\n    targetResult: WorkflowResult<any, any, any, any>;\n    scorerResults: {\n      workflow?: Record<string, any>;\n      steps?: Record<string, Record<string, any>>;\n    };\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult>;\nexport async function runEvals(config: {\n  data: RunEvalsDataItem<any>[];\n  scorers: MastraScorer<any, any, any, any>[] | WorkflowScorerConfig;\n  target: Agent | Workflow;\n  targetOptions?:\n    | Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>\n    | WorkflowRunOptions;\n  onItemComplete?: (params: {\n    item: RunEvalsDataItem<any>;\n    targetResult: any;\n    scorerResults: any;\n  }) => void | Promise<void>;\n  concurrency?: number;\n}): Promise<RunEvalsResult> {\n  const { data, scorers, target, targetOptions, onItemComplete, concurrency = 1 } = config;\n\n  validateEvalsInputs(data, scorers, target);\n\n  let totalItems = 0;\n  const scoreAccumulator = new ScoreAccumulator();\n\n  // Get storage from target's Mastra instance if available\n  // Agent uses getMastraInstance(), Workflow uses .mastra getter\n  const mastra = (target as any).getMastraInstance?.() || (target as any).mastra;\n  const storage = mastra?.getStorage();\n\n  const pMap = (await import('p-map')).default;\n  await pMap(\n    data,\n    async (item: RunEvalsDataItem<any>) => {\n      const targetResult = await executeTarget(target, item, targetOptions);\n      const scorerResults = await runScorers(scorers, targetResult, item);\n      scoreAccumulator.addScores(scorerResults);\n\n      // Save scores to storage if available\n      if (storage) {\n        await saveScoresToStorage({\n          storage,\n          scorerResults,\n          target,\n          item,\n          mastra,\n        });\n      }\n\n      if (onItemComplete) {\n        await onItemComplete({\n          item,\n          targetResult: targetResult as any,\n          scorerResults: scorerResults as any,\n        });\n      }\n\n      totalItems++;\n    },\n    { concurrency },\n  );\n\n  return {\n    scores: scoreAccumulator.getAverageScores(),\n    summary: {\n      totalItems,\n    },\n  };\n}\n\nfunction isWorkflow(target: Agent | Workflow): target is Workflow {\n  return target instanceof Workflow;\n}\n\nfunction isWorkflowScorerConfig(scorers: any): scorers is WorkflowScorerConfig {\n  return typeof scorers === 'object' && !Array.isArray(scorers) && ('workflow' in scorers || 'steps' in scorers);\n}\n\nfunction validateEvalsInputs(\n  data: RunEvalsDataItem<any>[],\n  scorers: MastraScorer<any, any, any, any>[] | WorkflowScorerConfig,\n  target: Agent | Workflow,\n): void {\n  if (data.length === 0) {\n    throw new MastraError({\n      domain: 'SCORER',\n      id: 'RUN_EXPERIMENT_FAILED_NO_DATA_PROVIDED',\n      category: 'USER',\n      text: 'Failed to run experiment: Data array is empty',\n    });\n  }\n\n  for (let i = 0; i < data.length; i++) {\n    const item = data[i];\n    if (!item || typeof item !== 'object' || !('input' in item)) {\n      throw new MastraError({\n        domain: 'SCORER',\n        id: 'INVALID_DATA_ITEM',\n        category: 'USER',\n        text: `Invalid data item at index ${i}: must have 'input' properties`,\n      });\n    }\n  }\n\n  // Validate scorers\n  if (Array.isArray(scorers)) {\n    if (scorers.length === 0) {\n      throw new MastraError({\n        domain: 'SCORER',\n        id: 'NO_SCORERS_PROVIDED',\n        category: 'USER',\n        text: 'At least one scorer must be provided',\n      });\n    }\n  } else if (isWorkflow(target) && isWorkflowScorerConfig(scorers)) {\n    const hasScorers =\n      (scorers.workflow && scorers.workflow.length > 0) || (scorers.steps && Object.keys(scorers.steps).length > 0);\n\n    if (!hasScorers) {\n      throw new MastraError({\n        domain: 'SCORER',\n        id: 'NO_SCORERS_PROVIDED',\n        category: 'USER',\n        text: 'At least one workflow or step scorer must be provided',\n      });\n    }\n  } else if (!isWorkflow(target) && !Array.isArray(scorers)) {\n    throw new MastraError({\n      domain: 'SCORER',\n      id: 'INVALID_AGENT_SCORERS',\n      category: 'USER',\n      text: 'Agent scorers must be an array of scorers',\n    });\n  }\n}\n\nasync function executeTarget(\n  target: Agent | Workflow,\n  item: RunEvalsDataItem<any>,\n  targetOptions?:\n    | Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>\n    | WorkflowRunOptions,\n) {\n  try {\n    if (isWorkflow(target)) {\n      return await executeWorkflow(target, item, targetOptions as WorkflowRunOptions);\n    } else {\n      return await executeAgent(\n        target,\n        item,\n        targetOptions as Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>,\n      );\n    }\n  } catch (error) {\n    throw new MastraError(\n      {\n        domain: 'SCORER',\n        id: 'RUN_EXPERIMENT_TARGET_FAILED_TO_GENERATE_RESULT',\n        category: 'USER',\n        text: 'Failed to run experiment: Error generating result from target',\n        details: {\n          item: JSON.stringify(item),\n        },\n      },\n      error,\n    );\n  }\n}\n\nasync function executeWorkflow(target: Workflow, item: RunEvalsDataItem<any>, targetOptions?: WorkflowRunOptions) {\n  const observabilityContext = resolveObservabilityContext(item);\n  const run = await target.createRun({ disableScorers: true });\n  const workflowResult = await run.start({\n    ...targetOptions,\n    ...item.startOptions,\n    inputData: item.input,\n    requestContext: item.requestContext,\n    ...observabilityContext,\n  });\n\n  return {\n    scoringData: {\n      input: item.input,\n      output: workflowResult.status === 'success' ? workflowResult.result : undefined,\n      stepResults: workflowResult.steps as Record<string, StepResult<any, any, any, any>>,\n    },\n  };\n}\n\nasync function executeAgent(\n  agent: Agent,\n  item: RunEvalsDataItem<any>,\n  targetOptions?: Omit<AgentExecutionOptions<any>, 'scorers' | 'returnScorerData' | 'requestContext'>,\n) {\n  const observabilityContext = resolveObservabilityContext(item);\n  const model = await agent.getModel();\n  if (isSupportedLanguageModel(model)) {\n    const { structuredOutput, ...restOptions } = targetOptions ?? {};\n    const baseOptions = {\n      ...restOptions,\n      ...observabilityContext,\n      scorers: {},\n      returnScorerData: true,\n      requestContext: item.requestContext,\n    };\n    return structuredOutput\n      ? await agent.generate(item.input, { ...baseOptions, structuredOutput })\n      : await agent.generate(item.input, baseOptions);\n  } else {\n    return await agent.generateLegacy(item.input, {\n      scorers: {},\n      returnScorerData: true,\n      requestContext: item.requestContext,\n      ...observabilityContext,\n    });\n  }\n}\n\nasync function runScorers(\n  scorers: MastraScorer<any, any, any, any>[] | WorkflowScorerConfig,\n  targetResult: any,\n  item: RunEvalsDataItem<any>,\n): Promise<Record<string, any>> {\n  const scorerResults: Record<string, any> = {};\n\n  if (Array.isArray(scorers)) {\n    for (const scorer of scorers) {\n      try {\n        const score = await scorer.run({\n          input: targetResult.scoringData?.input,\n          output: targetResult.scoringData?.output,\n          groundTruth: item.groundTruth,\n          requestContext: item.requestContext,\n          ...resolveObservabilityContext(item),\n        });\n\n        scorerResults[scorer.id] = score;\n      } catch (error) {\n        throw new MastraError(\n          {\n            domain: 'SCORER',\n            id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_RESULT',\n            category: 'USER',\n            text: `Failed to run experiment: Error running scorer ${scorer.id}`,\n            details: {\n              scorerId: scorer.id,\n              item: JSON.stringify(item),\n            },\n          },\n          error,\n        );\n      }\n    }\n  } else {\n    // Handle workflow scorer config\n    if (scorers.workflow) {\n      const workflowScorerResults: Record<string, any> = {};\n      for (const scorer of scorers.workflow) {\n        const score = await scorer.run({\n          input: targetResult.scoringData.input,\n          output: targetResult.scoringData.output,\n          groundTruth: item.groundTruth,\n          requestContext: item.requestContext,\n          ...resolveObservabilityContext(item),\n        });\n        workflowScorerResults[scorer.id] = score;\n      }\n      if (Object.keys(workflowScorerResults).length > 0) {\n        scorerResults.workflow = workflowScorerResults;\n      }\n    }\n\n    if (scorers.steps) {\n      const stepScorerResults: Record<string, any> = {};\n      for (const [stepId, stepScorers] of Object.entries(scorers.steps)) {\n        const stepResult = targetResult.scoringData.stepResults?.[stepId];\n        if (stepResult?.status === 'success' && stepResult.output !== undefined) {\n          const stepResults: Record<string, any> = {};\n          for (const scorer of stepScorers) {\n            try {\n              const score = await scorer.run({\n                input: stepResult.payload !== undefined ? stepResult.payload : targetResult.scoringData.input,\n                output: stepResult.output,\n                groundTruth: item.groundTruth,\n                requestContext: item.requestContext,\n                ...resolveObservabilityContext(item),\n              });\n              stepResults[scorer.id] = score;\n            } catch (error) {\n              throw new MastraError(\n                {\n                  domain: 'SCORER',\n                  id: 'RUN_EXPERIMENT_SCORER_FAILED_TO_SCORE_STEP_RESULT',\n                  category: 'USER',\n                  text: `Failed to run experiment: Error running scorer ${scorer.id} on step ${stepId}`,\n                  details: {\n                    scorerId: scorer.id,\n                    stepId,\n                  },\n                },\n                error,\n              );\n            }\n          }\n          if (Object.keys(stepResults).length > 0) {\n            stepScorerResults[stepId] = stepResults;\n          }\n        }\n      }\n      if (Object.keys(stepScorerResults).length > 0) {\n        scorerResults.steps = stepScorerResults;\n      }\n    }\n  }\n\n  return scorerResults;\n}\n\n/**\n * Saves scorer results to storage when running evaluations.\n * This makes scores visible in Studio's observability section.\n */\nasync function saveScoresToStorage({\n  storage,\n  scorerResults,\n  target,\n  item,\n  mastra,\n}: {\n  storage: any;\n  scorerResults: Record<string, any>;\n  target: Agent | Workflow;\n  item: RunEvalsDataItem<any>;\n  mastra: any;\n}): Promise<void> {\n  const entityId = target.id;\n  const entityType = isWorkflow(target) ? 'WORKFLOW' : 'AGENT';\n\n  // Handle flat scorer results (for agents or workflow-level scorers)\n  if (Array.isArray(scorerResults) || !('workflow' in scorerResults && 'steps' in scorerResults)) {\n    for (const [scorerId, scoreResult] of Object.entries(scorerResults)) {\n      if (scoreResult && typeof scoreResult === 'object' && 'score' in scoreResult) {\n        await saveSingleScore({\n          storage,\n          scoreResult,\n          scorerId,\n          entityId,\n          entityType,\n          mastra,\n          target,\n          item,\n        });\n      }\n    }\n  } else {\n    // Handle workflow scorer config with workflow and step scorers\n    if (scorerResults.workflow) {\n      for (const [scorerId, scoreResult] of Object.entries(scorerResults.workflow)) {\n        if (scoreResult && typeof scoreResult === 'object' && 'score' in scoreResult) {\n          await saveSingleScore({\n            storage,\n            scoreResult,\n            scorerId,\n            entityId,\n            entityType: 'WORKFLOW',\n            mastra,\n            target,\n            item,\n          });\n        }\n      }\n    }\n\n    if (scorerResults.steps) {\n      for (const [stepId, stepScorers] of Object.entries(scorerResults.steps)) {\n        for (const [scorerId, scoreResult] of Object.entries(stepScorers as Record<string, any>)) {\n          if (scoreResult && typeof scoreResult === 'object' && 'score' in scoreResult) {\n            await saveSingleScore({\n              storage,\n              scoreResult,\n              scorerId,\n              entityId: stepId,\n              entityType: 'STEP',\n              mastra,\n              target,\n              item,\n            });\n          }\n        }\n      }\n    }\n  }\n}\n\n/**\n * Saves a single scorer result to storage\n */\nasync function saveSingleScore({\n  storage,\n  scoreResult,\n  scorerId,\n  entityId,\n  entityType,\n  mastra,\n  target,\n  item,\n}: {\n  storage: any;\n  scoreResult: any;\n  scorerId: string;\n  entityId: string;\n  entityType: string;\n  mastra: any;\n  target: Agent | Workflow;\n  item: RunEvalsDataItem<any>;\n}): Promise<void> {\n  try {\n    // Get scorer information\n    let scorer = mastra?.getScorerById?.(scorerId);\n\n    if (!scorer) {\n      // Try to get from target's scorers\n      const targetScorers = await (target as any).listScorers?.();\n      if (targetScorers) {\n        for (const [_, scorerEntry] of Object.entries(targetScorers)) {\n          if ((scorerEntry as any).scorer?.id === scorerId) {\n            scorer = (scorerEntry as any).scorer;\n            break;\n          }\n        }\n      }\n    }\n\n    // Extract tracing context if available\n    let traceId: string | undefined;\n    let spanId: string | undefined;\n    if (item.tracingContext?.currentSpan && item.tracingContext.currentSpan.isValid) {\n      spanId = item.tracingContext.currentSpan.id;\n      traceId = item.tracingContext.currentSpan.traceId;\n    }\n\n    // Build additional context with groundTruth if available\n    const additionalContext: Record<string, any> = {};\n    if (item.groundTruth !== undefined) {\n      additionalContext.groundTruth = item.groundTruth;\n    }\n\n    const payload = {\n      ...scoreResult,\n      scorerId,\n      entityId,\n      entityType,\n      source: 'TEST' as const,\n      scorer: {\n        id: scorer?.id || scorerId,\n        name: scorer?.name || scorerId,\n        description: scorer?.description || '',\n        type: scorer?.type || 'unknown',\n        ...(scorer ? { hasJudge: !!scorer.judge } : {}),\n      },\n      entity: {\n        id: target.id,\n        name: (target as any).name || target.id,\n      },\n      // Include requestContext from item\n      requestContext: item.requestContext ? Object.fromEntries(item.requestContext.entries()) : undefined,\n      // Include additionalContext with groundTruth\n      additionalContext: Object.keys(additionalContext).length > 0 ? additionalContext : undefined,\n      // Include tracing information\n      traceId,\n      spanId,\n    };\n\n    await validateAndSaveScore(storage, payload);\n  } catch (error) {\n    // Log error but don't fail the evaluation\n    mastra?.getLogger?.()?.warn?.(`Failed to save score for scorer ${scorerId}:`, error);\n  }\n}\n"]}