@axiastudio/aioc 0.2.4 → 0.2.6

package/README.md

@@ -23,7 +23,7 @@ Its core job is to make agent execution boundaries explicit, testable, auditable
 
 ## Release Status
 
-Current stable release: `0.2.4`.
+Current stable release: `0.2.5`.
 The stable line started with `0.1.0`.
 The core runtime surface is compatibility-managed. Breaking changes to the stable surface should ship only with explicit migration guidance and release notes.
 
@@ -38,6 +38,8 @@ AIOC `0.2.1` realigns the `0.2.x` package line with the current `main` history a
 AIOC `0.2.2` completes the descriptor instruction-composition surface with reusable `instruction_parts`, ordered `instructions_sequence`, and boolean `where` gates.
 AIOC `0.2.3` adds RFC-0010 policy composition helpers for exact-name tool and handoff policy dispatch without changing runtime enforcement semantics.
 AIOC `0.2.4` adds experimental governance-event packages and an OpenTelemetry Logs exporter for reduced, operational observability events derived from `RunRecord` values.
+AIOC `0.2.5` makes replay history-faithful by recording the initial input scope in `RunRecord` and replaying from it by default.
+AIOC `0.2.6` adds descriptor-level conditional agent handoffs with boolean `where` gates that filter handoff tools before provider requests.
 
 ### Experimental Packages
 
@@ -145,7 +147,7 @@ console.log(result.finalOutput);
 - `loadAgentHarnessDescriptorFromFile(...)`
 - `AgentHarnessDescriptor` and related descriptor types
 
-The Agent Harness Descriptor is included in the supported `0.2.x` API surface as the `aioc.agent_graph.v0` descriptor contract. Use it for controlled configuration, examples, and evaluation harnesses; keep executable tools, policies, providers, persistence, approvals, and deployment configuration in application code.
+The Agent Harness Descriptor is included in the supported `0.2.x` API surface as the `aioc.agent_graph.v0` descriptor contract. Use it for controlled configuration, examples, and evaluation harnesses; keep executable tools, policies, providers, persistence, approvals, and deployment configuration in application code. Descriptor handoffs may use boolean `where` gates to hide unavailable handoff tools before the provider call; `HandoffPolicy` still owns allow/deny/approval decisions for exposed handoffs.
 
 ## Policy Gate (Minimal)
 
@@ -262,6 +264,11 @@ This repository also contains `aioc-inspect`, a private reference example UI for
 
 ## Examples
 
+Suggested reading path: start with `example:hello`, then policy, approval,
+tool-policy, and policy-composition. Use run-record and harness examples after
+the basics; `example:non-regression` and `example:run-regression` are advanced
+workflows.
+
 | Command | Purpose | Needs API key |
 |---|---|---|
 | `npm run example:hello` | Minimal single-agent run | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
@@ -270,16 +277,20 @@ This repository also contains `aioc-inspect`, a private reference example UI for
 | `npm run example:approval-evidence` | Approval evidence passed through context and reevaluated by policy | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
 | `npm run example:tool-policy` | Straight tool + policy flow with allowed execution | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
 | `npm run example:policy-composition` | Exact-name tool policy dispatch with fallback deny | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
+| `npm run example:harness-rerun` | Replay against a modified harness with a mocked new tool output | Yes (`OPENAI_API_KEY`) |
 | `npm run example:run-record` | Run-record persistence with redaction + audit | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
 | `npm run example:rru:01-extract` | Minimal `extractToolCalls(...)` | No |
 | `npm run example:rru:02-compare` | Minimal `compareRunRecords(...)` | No |
 | `npm run example:rru:03-replay-strict` | Minimal strict replay | No |
 | `npm run example:rru:04-replay-hybrid` | Minimal hybrid replay | No |
 | `npm run example:non-regression` | Advanced v1/v2 run-record diff | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
+| `npm run example:run-regression` | Local `runRegressionSuite(...)` over a modified harness | Yes (`OPENAI_API_KEY`) |
 
 Notes:
 
 - for live-provider examples, set `AIOC_EXAMPLE_PROVIDER` to `openai` or `mistral`; the matching API key must also be available
+- `example:harness-rerun` and `example:run-regression` configure OpenAI directly from `OPENAI_API_KEY`; the model is declared in the YAML descriptor
+- run-record utility examples are deterministic and do not need a provider
 - `example:non-regression` is educational and can be non-deterministic because it uses a live provider.
 - canonical examples guide: `docs/CANONICAL-EXAMPLES.md`.
 

package/dist/agent.d.ts

@@ -3,6 +3,12 @@ import type { OutputGuardrail } from "./guardrails";
 import type { Tool } from "./tool";
 import type { ModelSettings } from "./types";
 export type AgentInstructions<TContext = unknown> = string | ((runContext: RunContext<TContext>) => string | Promise<string>);
+export type AgentHandoffCondition<TContext = unknown> = (runContext: RunContext<TContext>) => boolean;
+export interface AgentHandoff<TContext = unknown> {
+    agent: Agent<TContext>;
+    enabled?: AgentHandoffCondition<TContext>;
+}
+export type AgentHandoffEntry<TContext = unknown> = Agent<TContext> | AgentHandoff<TContext>;
 export interface AgentConfiguration<TContext = unknown> {
     name: string;
     handoffDescription?: string;
@@ -11,7 +17,7 @@ export interface AgentConfiguration<TContext = unknown> {
     model?: string;
     modelSettings?: ModelSettings;
     tools?: Tool<TContext>[];
-    handoffs?: Agent<TContext>[];
+    handoffs?: AgentHandoffEntry<TContext>[];
     outputGuardrails?: OutputGuardrail<TContext>[];
 }
 export declare class Agent<TContext = unknown> {
@@ -23,6 +29,7 @@ export declare class Agent<TContext = unknown> {
     modelSettings?: ModelSettings;
     tools: Tool<TContext>[];
     handoffs: Agent<TContext>[];
+    handoffRules: AgentHandoff<TContext>[];
     outputGuardrails: OutputGuardrail<TContext>[];
     constructor(config: AgentConfiguration<TContext>);
     resolveInstructions(runContext: RunContext<TContext>): Promise<string | undefined>;

package/dist/agent.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,eAAe,CAAC;AAC3C,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAC;AACpD,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AAE7C,MAAM,MAAM,iBAAiB,CAAC,QAAQ,GAAG,OAAO,IAC5C,MAAM,GACN,CAAC,CAAC,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;AAErE,MAAM,WAAW,kBAAkB,CAAC,QAAQ,GAAG,OAAO;IACpD,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,CAAC,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACzB,QAAQ,CAAC,EAAE,KAAK,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC7B,gBAAgB,CAAC,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;CAChD;AAED,qBAAa,KAAK,CAAC,QAAQ,GAAG,OAAO;IACnC,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,EAAE,MAAM,CAAC;IAC3B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACxB,QAAQ,EAAE,KAAK,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC5B,gBAAgB,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAElC,MAAM,EAAE,kBAAkB,CAAC,QAAQ,CAAC;IAY1C,mBAAmB,CACvB,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,GAC/B,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;CAS/B"}
+{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,eAAe,CAAC;AAC3C,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAC;AACpD,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AAE7C,MAAM,MAAM,iBAAiB,CAAC,QAAQ,GAAG,OAAO,IAC5C,MAAM,GACN,CAAC,CAAC,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;AAErE,MAAM,MAAM,qBAAqB,CAAC,QAAQ,GAAG,OAAO,IAAI,CACtD,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,KAC7B,OAAO,CAAC;AAEb,MAAM,WAAW,YAAY,CAAC,QAAQ,GAAG,OAAO;IAC9C,KAAK,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IACvB,OAAO,CAAC,EAAE,qBAAqB,CAAC,QAAQ,CAAC,CAAC;CAC3C;AAED,MAAM,MAAM,iBAAiB,CAAC,QAAQ,GAAG,OAAO,IAC5C,KAAK,CAAC,QAAQ,CAAC,GACf,YAAY,CAAC,QAAQ,CAAC,CAAC;AAE3B,MAAM,WAAW,kBAAkB,CAAC,QAAQ,GAAG,OAAO;IACpD,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,CAAC,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACzB,QAAQ,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,EAAE,CAAC;IACzC,gBAAgB,CAAC,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;CAChD;AAED,qBAAa,KAAK,CAAC,QAAQ,GAAG,OAAO;IACnC,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,EAAE,MAAM,CAAC;IAC3B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACxB,QAAQ,EAAE,KAAK,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC5B,YAAY,EAAE,YAAY,CAAC,QAAQ,CAAC,EAAE,CAAC;IACvC,gBAAgB,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAElC,MAAM,EAAE,kBAAkB,CAAC,QAAQ,CAAC;IAe1C,mBAAmB,CACvB,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,GAC/B,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;CAS/B"}

package/dist/agent.js

@@ -10,6 +10,7 @@ class Agent {
     modelSettings;
     tools;
     handoffs;
+    handoffRules;
     outputGuardrails;
     constructor(config) {
         this.name = config.name;
@@ -19,7 +20,8 @@ class Agent {
         this.model = config.model;
         this.modelSettings = config.modelSettings;
         this.tools = config.tools ?? [];
-        this.handoffs = config.handoffs ?? [];
+        this.handoffRules = (config.handoffs ?? []).map((handoff) => handoff instanceof Agent ? { agent: handoff } : handoff);
+        this.handoffs = this.handoffRules.map((rule) => rule.agent);
         this.outputGuardrails = config.outputGuardrails ?? [];
     }
     async resolveInstructions(runContext) {

package/dist/harness-descriptor.d.ts

@@ -20,11 +20,17 @@ export interface HarnessAgentDescriptor {
     model?: string;
     modelSettings?: Record<string, unknown>;
     tools?: string[];
-    handoffs?: string[];
+    handoffs?: HarnessHandoffEntryDescriptor[];
 }
-export interface HarnessInstructionWhereDescriptor {
+export interface HarnessWhereDescriptor {
     context: string;
 }
+export type HarnessInstructionWhereDescriptor = HarnessWhereDescriptor;
+export interface HarnessHandoffDescriptor {
+    agent: string;
+    where?: HarnessWhereDescriptor;
+}
+export type HarnessHandoffEntryDescriptor = string | HarnessHandoffDescriptor;
 export interface HarnessInstructionPartDescriptor {
     text: string;
     where?: HarnessInstructionWhereDescriptor;

package/dist/harness-descriptor.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"harness-descriptor.d.ts","sourceRoot":"","sources":["../src/harness-descriptor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAA0B,MAAM,SAAS,CAAC;AAExD,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,SAAS,CAAC;AAEnD,MAAM,WAAW,yBAAyB;IACxC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,qBAAqB;IACpC,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,sBAAsB;IACrC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,6BAA6B,CAAC;IAC7C,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACxC,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,QAAQ,CAAC,EAAE,MAAM,EAAE,CAAC;CACrB;AAED,MAAM,WAAW,iCAAiC;IAChD,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,gCAAgC;IAC/C,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,iCAAiC,CAAC;CAC3C;AAED,MAAM,MAAM,6BAA6B,GACrC,MAAM,GACN,gCAAgC,EAAE,CAAC;AAEvC,MAAM,WAAW,iCAAiC;IAChD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,MAAM,4BAA4B,GACpC,OAAO,GACP,iCAAiC,CAAC;AAEtC,MAAM,WAAW,6BAA6B;IAC5C,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,6BAA6B,CAAC,CAAC;IACvD,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,4BAA4B,CAAC,CAAC;CAC3D;AAED,MAAM,WAAW,sBAAsB;IACrC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,EAAE,yBAAyB,CAAC;IACrC,OAAO,EAAE,wBAAwB,CAAC;IAClC,OAAO,CAAC,EAAE,wBAAwB,CAAC;IACnC,iBAAiB,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC3C,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,qBAAqB,CAAC,CAAC;IAC9C,cAAc,CAAC,EAAE,IAAI,CACnB,sBAAsB,EACtB,OAAO,GAAG,eAAe,GAAG,cAAc,CAC3C,CAAC;IACF,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC;CAChD;AAED,MAAM,WAAW,oBAAoB,CAAC,QAAQ,GAAG,OAAO;IACtD,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC;IACvC,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,yBAAyB;IACxC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,GAAG,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,SAAS,CAAC,EAAE,OAAO,CAAC;CACrB;AAED,MAAM,WAAW,oBAAoB;IACnC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,cAAc,EAAE,MAAM,CAAC;IACvB,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,YAAY,CAAC,QAAQ,GAAG,OAAO;IAC9C,UAAU,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IAC5B,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;IACrC,cAAc,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,oBAAoB,CAAC;IAC/B,UAAU,EAAE,IAAI,CAAC,mBAAmB,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC1D,aAAa,CAAC,KAAK,CAAC,EAAE,yBAAyB,GAAG,QAAQ,CAAC;CAC5D;AAsiBD,wBAAgB,0BAA0B,CACxC,UAAU,EAAE,sBAAsB,GACjC,MAAM,CAER;AAED,wBAAgB,iBAAiB,CAAC,QAAQ,GAAG,OAAO,EAClD,UAAU,EAAE,sBAAsB,EAClC,QAAQ,GAAE,oBAAoB,CAAC,QAAQ,CAAM,GAC5C,YAAY,CAAC,QAAQ,CAAC,CAqExB"}
+{"version":3,"file":"harness-descriptor.d.ts","sourceRoot":"","sources":["../src/harness-descriptor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAA6C,MAAM,SAAS,CAAC;AAG3E,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,SAAS,CAAC;AAEnD,MAAM,WAAW,yBAAyB;IACxC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,qBAAqB;IACpC,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,sBAAsB;IACrC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,6BAA6B,CAAC;IAC7C,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACxC,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,QAAQ,CAAC,EAAE,6BAA6B,EAAE,CAAC;CAC5C;AAED,MAAM,WAAW,sBAAsB;IACrC,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,MAAM,iCAAiC,GAAG,sBAAsB,CAAC;AAEvE,MAAM,WAAW,wBAAwB;IACvC,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,sBAAsB,CAAC;CAChC;AAED,MAAM,MAAM,6BAA6B,GAAG,MAAM,GAAG,wBAAwB,CAAC;AAE9E,MAAM,WAAW,gCAAgC;IAC/C,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,iCAAiC,CAAC;CAC3C;AAED,MAAM,MAAM,6BAA6B,GACrC,MAAM,GACN,gCAAgC,EAAE,CAAC;AAEvC,MAAM,WAAW,iCAAiC;IAChD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,MAAM,4BAA4B,GACpC,OAAO,GACP,iCAAiC,CAAC;AAEtC,MAAM,WAAW,6BAA6B;IAC5C,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,6BAA6B,CAAC,CAAC;IACvD,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,4BAA4B,CAAC,CAAC;CAC3D;AAED,MAAM,WAAW,sBAAsB;IACrC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,EAAE,yBAAyB,CAAC;IACrC,OAAO,EAAE,wBAAwB,CAAC;IAClC,OAAO,CAAC,EAAE,wBAAwB,CAAC;IACnC,iBAAiB,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC3C,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,qBAAqB,CAAC,CAAC;IAC9C,cAAc,CAAC,EAAE,IAAI,CACnB,sBAAsB,EACtB,OAAO,GAAG,eAAe,GAAG,cAAc,CAC3C,CAAC;IACF,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC;CAChD;AAED,MAAM,WAAW,oBAAoB,CAAC,QAAQ,GAAG,OAAO;IACtD,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC;IACvC,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,yBAAyB;IACxC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,GAAG,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,SAAS,CAAC,EAAE,OAAO,CAAC;CACrB;AAED,MAAM,WAAW,oBAAoB;IACnC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,cAAc,EAAE,MAAM,CAAC;IACvB,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,YAAY,CAAC,QAAQ,GAAG,OAAO;IAC9C,UAAU,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IAC5B,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;IACrC,cAAc,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,oBAAoB,CAAC;IAC/B,UAAU,EAAE,IAAI,CAAC,mBAAmB,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC1D,aAAa,CAAC,KAAK,CAAC,EAAE,yBAAyB,GAAG,QAAQ,CAAC;CAC5D;AAqoBD,wBAAgB,0BAA0B,CACxC,UAAU,EAAE,sBAAsB,GACjC,MAAM,CAER;AAED,wBAAgB,iBAAiB,CAAC,QAAQ,GAAG,OAAO,EAClD,UAAU,EAAE,sBAAsB,EAClC,QAAQ,GAAE,oBAAoB,CAAC,QAAQ,CAAM,GAC5C,YAAY,CAAC,QAAQ,CAAC,CAiFxB"}

package/dist/harness-descriptor.js

@@ -30,6 +30,28 @@ function assertNonEmptyString(value, label) {
         throw new Error(`${label} must be a non-empty string.`);
     }
 }
+function assertHandoffEntry(value, label) {
+    if (typeof value === "string") {
+        assertNonEmptyString(value, label);
+        return;
+    }
+    if (!isPlainObject(value)) {
+        throw new Error(`${label} must be a non-empty string or an object.`);
+    }
+    assertNonEmptyString(value.agent, `${label}.agent`);
+    if (typeof value.where !== "undefined") {
+        assertInstructionWhere(value.where, label);
+    }
+}
+function assertHandoffEntries(value, label) {
+    if (typeof value === "undefined") {
+        return;
+    }
+    if (!Array.isArray(value)) {
+        throw new Error(`${label} must be an array.`);
+    }
+    value.forEach((entry, index) => assertHandoffEntry(entry, `${label}[${index}]`));
+}
 function validateDescriptorShape(descriptor) {
     assertPlainObject(descriptor, "Harness descriptor");
     assertPlainObject(descriptor.runtime, "Harness descriptor runtime");
@@ -49,6 +71,7 @@ function validateDescriptorShape(descriptor) {
     for (const [agentId, agentDescriptor] of Object.entries(descriptor.agents)) {
         assertPlainObject(agentDescriptor, `Harness descriptor agent "${agentId}"`);
         assertNoUnresolvedInstructionFile(agentDescriptor, `Harness descriptor agent "${agentId}"`);
+        assertHandoffEntries(agentDescriptor.handoffs, `Harness descriptor agent "${agentId}".handoffs`);
     }
     for (const [toolId, toolDescriptor] of Object.entries(descriptor.tools ?? {})) {
         assertPlainObject(toolDescriptor, `Harness descriptor tool "${toolId}"`);
@@ -202,31 +225,37 @@ function assertDeclaredPromptPaths(agentId, paths, promptAccessRules) {
         }
     }
 }
+function assertDeclaredBooleanContextPath(label, path, promptAccessRules) {
+    const rule = promptAccessRules.get(path);
+    if (!rule) {
+        throw new Error(`${label} references undeclared context path "${path}".`);
+    }
+    if (rule.type !== "boolean") {
+        throw new Error(`${label} context path "${path}" must be declared with type "boolean".`);
+    }
+}
 function assertDeclaredBooleanWherePaths(agentId, paths, promptAccessRules) {
     for (const path of paths) {
-        const rule = promptAccessRules.get(path);
-        if (!rule) {
-            throw new Error(`Agent "${agentId}" instruction where references undeclared context path "${path}".`);
-        }
-        if (rule.type !== "boolean") {
-            throw new Error(`Agent "${agentId}" instruction where context path "${path}" must be declared with type "boolean".`);
-        }
+        assertDeclaredBooleanContextPath(`Agent "${agentId}" instruction where`, path, promptAccessRules);
     }
 }
-function shouldIncludeInstructionPart(part, context) {
-    if (!part.where) {
+function shouldIncludeWhere(where, context, label) {
+    if (!where) {
         return true;
     }
-    const path = normalizePromptPath(part.where.context, "Instruction where.context path");
+    const path = normalizePromptPath(where.context, "Instruction where.context path");
     const resolved = readWhereContextValue(context, path);
     if (!resolved.exists || typeof resolved.value === "undefined") {
-        throw new Error(`Harness descriptor instruction where could not resolve context path "${path}".`);
+        throw new Error(`${label} could not resolve context path "${path}".`);
     }
     if (typeof resolved.value !== "boolean") {
-        throw new Error(`Harness descriptor instruction where context path "${path}" must resolve to a boolean.`);
+        throw new Error(`${label} context path "${path}" must resolve to a boolean.`);
     }
     return resolved.value === true;
 }
+function shouldIncludeInstructionPart(part, context) {
+    return shouldIncludeWhere(part.where, context, "Harness descriptor instruction where");
+}
 function renderInstructionParts(instructions, context, promptAccessRules) {
     const parts = [];
     for (const part of instructions) {
@@ -323,6 +352,22 @@ function resolveTool(toolId, descriptor, registry) {
     }
     return toolDefinition;
 }
+function readHandoffEntry(entry) {
+    if (typeof entry === "string") {
+        return {
+            agent: entry,
+        };
+    }
+    return entry;
+}
+function compileHandoffCondition(fromAgentId, toAgentId, where, promptAccessRules) {
+    if (!where) {
+        return undefined;
+    }
+    const path = normalizePromptPath(where.context, `Agent "${fromAgentId}" handoff "${toAgentId}" where.context path`);
+    assertDeclaredBooleanContextPath(`Agent "${fromAgentId}" handoff "${toAgentId}" where`, path, promptAccessRules);
+    return (runContext) => shouldIncludeWhere(where, runContext.context, `Harness descriptor handoff "${fromAgentId}" to "${toAgentId}" where`);
+}
 function hashAgentHarnessDescriptor(descriptor) {
     return `sha256:${(0, canonical_json_1.hashCanonicalJsonValue)(descriptor)}`;
 }
@@ -350,13 +395,20 @@ function buildAgentHarness(descriptor, registry = {}) {
         if (!agent) {
             continue;
         }
-        agent.handoffs = (agentDescriptor.handoffs ?? []).map((handoffId) => {
+        const handoffRules = (agentDescriptor.handoffs ?? []).map((entry) => {
+            const handoff = readHandoffEntry(entry);
+            const handoffId = handoff.agent;
             const handoffAgent = agents.get(handoffId);
             if (!handoffAgent) {
                 throw new Error(`Harness descriptor references unknown handoff agent "${handoffId}" from "${agentId}".`);
             }
-            return handoffAgent;
+            return {
+                agent: handoffAgent,
+                enabled: compileHandoffCondition(agentId, handoffId, handoff.where, promptAccessRules),
+            };
         });
+        agent.handoffRules = handoffRules;
+        agent.handoffs = handoffRules.map((handoff) => handoff.agent);
     }
     const entryAgent = agents.get(descriptor.runtime.entry_agent);
     if (!entryAgent) {

package/dist/index.d.ts

@@ -18,6 +18,8 @@ export * from "./providers/mistral";
 export * from "./providers/openai";
 export * from "./run";
 export * from "./run-output-events";
+export * from "./run-regression";
+export * from "./run-regression-summary";
 export * from "./run-record";
 export * from "./run-record-utils";
 export * from "./run-context";

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,SAAS,CAAC;AACxB,cAAc,oBAAoB,CAAC;AACnC,cAAc,UAAU,CAAC;AACzB,cAAc,UAAU,CAAC;AACzB,cAAc,cAAc,CAAC;AAC7B,cAAc,sBAAsB,CAAC;AACrC,cAAc,6BAA6B,CAAC;AAC5C,cAAc,QAAQ,CAAC;AACvB,cAAc,UAAU,CAAC;AACzB,cAAc,YAAY,CAAC;AAC3B,cAAc,UAAU,CAAC;AACzB,cAAc,sBAAsB,CAAC;AACrC,cAAc,oBAAoB,CAAC;AACnC,cAAc,kBAAkB,CAAC;AACjC,cAAc,kBAAkB,CAAC;AACjC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,qBAAqB,CAAC;AACpC,cAAc,oBAAoB,CAAC;AACnC,cAAc,OAAO,CAAC;AACtB,cAAc,qBAAqB,CAAC;AACpC,cAAc,cAAc,CAAC;AAC7B,cAAc,oBAAoB,CAAC;AACnC,cAAc,eAAe,CAAC;AAC9B,cAAc,kBAAkB,CAAC;AACjC,cAAc,QAAQ,CAAC;AACvB,cAAc,SAAS,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,SAAS,CAAC;AACxB,cAAc,oBAAoB,CAAC;AACnC,cAAc,UAAU,CAAC;AACzB,cAAc,UAAU,CAAC;AACzB,cAAc,cAAc,CAAC;AAC7B,cAAc,sBAAsB,CAAC;AACrC,cAAc,6BAA6B,CAAC;AAC5C,cAAc,QAAQ,CAAC;AACvB,cAAc,UAAU,CAAC;AACzB,cAAc,YAAY,CAAC;AAC3B,cAAc,UAAU,CAAC;AACzB,cAAc,sBAAsB,CAAC;AACrC,cAAc,oBAAoB,CAAC;AACnC,cAAc,kBAAkB,CAAC;AACjC,cAAc,kBAAkB,CAAC;AACjC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,qBAAqB,CAAC;AACpC,cAAc,oBAAoB,CAAC;AACnC,cAAc,OAAO,CAAC;AACtB,cAAc,qBAAqB,CAAC;AACpC,cAAc,kBAAkB,CAAC;AACjC,cAAc,0BAA0B,CAAC;AACzC,cAAc,cAAc,CAAC;AAC7B,cAAc,oBAAoB,CAAC;AACnC,cAAc,eAAe,CAAC;AAC9B,cAAc,kBAAkB,CAAC;AACjC,cAAc,QAAQ,CAAC;AACvB,cAAc,SAAS,CAAC"}

package/dist/index.js

@@ -34,6 +34,8 @@ __exportStar(require("./providers/mistral"), exports);
 __exportStar(require("./providers/openai"), exports);
 __exportStar(require("./run"), exports);
 __exportStar(require("./run-output-events"), exports);
+__exportStar(require("./run-regression"), exports);
+__exportStar(require("./run-regression-summary"), exports);
 __exportStar(require("./run-record"), exports);
 __exportStar(require("./run-record-utils"), exports);
 __exportStar(require("./run-context"), exports);

package/dist/run-record-utils.d.ts

@@ -48,6 +48,8 @@ export interface RunRecordComparison {
 }
 export declare function compareRunRecords<TContextA, TContextB>(left: RunRecord<TContextA>, right: RunRecord<TContextB>, options?: CompareRunRecordsOptions): RunRecordComparison;
 export type ReplayMode = "live" | "strict" | "hybrid";
+export type ReplayInputMode = "recorded" | "question";
+export type ReplayInputSource = "inputItemCount" | "requestFingerprint" | "questionFallback" | "question";
 export interface MissingToolCallResolution {
     action: "throw" | "use_live" | "use_output";
     output?: unknown;
@@ -65,6 +67,7 @@ export interface ReplayStats {
     replayedFromRecord: number;
     missingToolCalls: number;
     liveFallbackCalls: number;
+    inputSource: ReplayInputSource;
 }
 export interface ReplayFromRunRecordInput<TContext = unknown> {
     sourceRunRecord: RunRecord<TContext>;
@@ -72,6 +75,7 @@ export interface ReplayFromRunRecordInput<TContext = unknown> {
     agentFactory?: () => Agent<TContext> | Promise<Agent<TContext>>;
     mode: ReplayMode;
     runOptions?: Omit<NonStreamRunOptions<TContext>, "stream">;
+    inputMode?: ReplayInputMode;
     metadataOverrides?: Record<string, unknown>;
     onMissingToolCall?: ReplayMissingToolCallHandler;
 }

package/dist/run-record-utils.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run-record-utils.d.ts","sourceRoot":"","sources":["../src/run-record-utils.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAGhC,OAAO,KAAK,EAAE,SAAS,EAAmC,MAAM,cAAc,CAAC;AAE/E,OAAO,KAAK,EAAE,cAAc,EAAE,mBAAmB,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AAE9E,MAAM,WAAW,iBAAiB;IAChC,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,OAAO,CAAC;IACnB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,SAAS,EAAE,OAAO,CAAC;IACnB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,iBAAiB,EAAE,MAAM,CAAC;IAC1B,QAAQ,EAAE,MAAM,CAAC;CAClB;AAsBD,wBAAgB,gBAAgB,CAAC,QAAQ,EACvC,KAAK,EAAE,SAAS,CAAC,QAAQ,CAAC,GACzB,iBAAiB,EAAE,CAAC;AAEvB,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,cAAc,EAAE,GAAG,iBAAiB,EAAE,CAAC;AA+C/E,MAAM,MAAM,0BAA0B,GAClC,UAAU,GACV,WAAW,GACX,QAAQ,GACR,YAAY,GACZ,UAAU,CAAC;AAEf,MAAM,WAAW,wBAAwB;IACvC,eAAe,CAAC,EAAE,0BAA0B,EAAE,CAAC;IAC/C,eAAe,CAAC,EAAE,0BAA0B,EAAE,CAAC;IAC/C,iBAAiB,CAAC,EAAE,OAAO,CAAC;CAC7B;AAED,MAAM,WAAW,mBAAmB;IAClC,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,UAAU,GAAG,cAAc,GAAG,eAAe,CAAC;IACpD,IAAI,CAAC,EAAE,OAAO,CAAC;IACf,KAAK,CAAC,EAAE,OAAO,CAAC;CACjB;AAED,MAAM,WAAW,0BAA0B;IACzC,iBAAiB,EAAE,OAAO,CAAC;IAC3B,iBAAiB,EAAE,OAAO,CAAC;IAC3B,mBAAmB,EAAE,OAAO,CAAC;IAC7B,sBAAsB,EAAE,OAAO,CAAC;CACjC;AAED,MAAM,WAAW,0BAA0B;IACzC,eAAe,EAAE,MAAM,CAAC;IACxB,eAAe,EAAE,MAAM,CAAC;IACxB,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,EAAE,MAAM,CAAC;IACnB,gBAAgB,EAAE,MAAM,CAAC;IACzB,gBAAgB,EAAE,MAAM,CAAC;IACzB,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,MAAM,WAAW,mBAAmB;IAClC,KAAK,EAAE,OAAO,CAAC;IACf,OAAO,EAAE,0BAA0B,CAAC;IACpC,OAAO,EAAE,0BAA0B,CAAC;IACpC,WAAW,EAAE,mBAAmB,EAAE,CAAC;CACpC;AA4FD,wBAAgB,iBAAiB,CAAC,SAAS,EAAE,SAAS,EACpD,IAAI,EAAE,SAAS,CAAC,SAAS,CAAC,EAC1B,KAAK,EAAE,SAAS,CAAC,SAAS,CAAC,EAC3B,OAAO,CAAC,EAAE,wBAAwB,GACjC,mBAAmB,CAwKrB;AAED,MAAM,MAAM,UAAU,GAAG,MAAM,GAAG,QAAQ,GAAG,QAAQ,CAAC;AAEtD,MAAM,WAAW,yBAAyB;IACxC,MAAM,EAAE,OAAO,GAAG,UAAU,GAAG,YAAY,CAAC;IAC5C,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB;AAED,MAAM,WAAW,0BAA0B;IACzC,IAAI,EAAE,UAAU,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,OAAO,CAAC;IACnB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,MAAM,MAAM,4BAA4B,GAAG,CACzC,KAAK,EAAE,0BAA0B,KAC9B,yBAAyB,GAAG,OAAO,CAAC,yBAAyB,CAAC,CAAC;AAEpE,MAAM,WAAW,WAAW;IAC1B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,kBAAkB,EAAE,MAAM,CAAC;IAC3B,gBAAgB,EAAE,MAAM,CAAC;IACzB,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,wBAAwB,CAAC,QAAQ,GAAG,OAAO;IAC1D,eAAe,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IACrC,KAAK,CAAC,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IACxB,YAAY,CAAC,EAAE,MAAM,KAAK,CAAC,QAAQ,CAAC,GAAG,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;IAChE,IAAI,EAAE,UAAU,CAAC;IACjB,UAAU,CAAC,EAAE,IAAI,CAAC,mBAAmB,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC3D,iBAAiB,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC5C,iBAAiB,CAAC,EAAE,4BAA4B,CAAC;CAClD;AAED,MAAM,WAAW,yBAAyB,CAAC,QAAQ,GAAG,OAAO;IAC3D,MAAM,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC5B,eAAe,CAAC,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IACtC,WAAW,EAAE,WAAW,CAAC;CAC1B;AAiND,wBAAsB,mBAAmB,CAAC,QAAQ,GAAG,OAAO,EAC1D,KAAK,EAAE,wBAAwB,CAAC,QAAQ,CAAC,GACxC,OAAO,CAAC,yBAAyB,CAAC,QAAQ,CAAC,CAAC,CA8D9C"}
+{"version":3,"file":"run-record-utils.d.ts","sourceRoot":"","sources":["../src/run-record-utils.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAGhC,OAAO,KAAK,EAAE,SAAS,EAAmC,MAAM,cAAc,CAAC;AAE/E,OAAO,KAAK,EAAE,cAAc,EAAE,mBAAmB,EAAE,SAAS,EAAE,MAAM,SAAS,CAAC;AAE9E,MAAM,WAAW,iBAAiB;IAChC,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,OAAO,CAAC;IACnB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,SAAS,EAAE,OAAO,CAAC;IACnB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,iBAAiB,EAAE,MAAM,CAAC;IAC1B,QAAQ,EAAE,MAAM,CAAC;CAClB;AAsBD,wBAAgB,gBAAgB,CAAC,QAAQ,EACvC,KAAK,EAAE,SAAS,CAAC,QAAQ,CAAC,GACzB,iBAAiB,EAAE,CAAC;AAEvB,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,cAAc,EAAE,GAAG,iBAAiB,EAAE,CAAC;AA+C/E,MAAM,MAAM,0BAA0B,GAClC,UAAU,GACV,WAAW,GACX,QAAQ,GACR,YAAY,GACZ,UAAU,CAAC;AAEf,MAAM,WAAW,wBAAwB;IACvC,eAAe,CAAC,EAAE,0BAA0B,EAAE,CAAC;IAC/C,eAAe,CAAC,EAAE,0BAA0B,EAAE,CAAC;IAC/C,iBAAiB,CAAC,EAAE,OAAO,CAAC;CAC7B;AAED,MAAM,WAAW,mBAAmB;IAClC,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,UAAU,GAAG,cAAc,GAAG,eAAe,CAAC;IACpD,IAAI,CAAC,EAAE,OAAO,CAAC;IACf,KAAK,CAAC,EAAE,OAAO,CAAC;CACjB;AAED,MAAM,WAAW,0BAA0B;IACzC,iBAAiB,EAAE,OAAO,CAAC;IAC3B,iBAAiB,EAAE,OAAO,CAAC;IAC3B,mBAAmB,EAAE,OAAO,CAAC;IAC7B,sBAAsB,EAAE,OAAO,CAAC;CACjC;AAED,MAAM,WAAW,0BAA0B;IACzC,eAAe,EAAE,MAAM,CAAC;IACxB,eAAe,EAAE,MAAM,CAAC;IACxB,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,EAAE,MAAM,CAAC;IACnB,gBAAgB,EAAE,MAAM,CAAC;IACzB,gBAAgB,EAAE,MAAM,CAAC;IACzB,cAAc,EAAE,MAAM,CAAC;CACxB;AAED,MAAM,WAAW,mBAAmB;IAClC,KAAK,EAAE,OAAO,CAAC;IACf,OAAO,EAAE,0BAA0B,CAAC;IACpC,OAAO,EAAE,0BAA0B,CAAC;IACpC,WAAW,EAAE,mBAAmB,EAAE,CAAC;CACpC;AA4FD,wBAAgB,iBAAiB,CAAC,SAAS,EAAE,SAAS,EACpD,IAAI,EAAE,SAAS,CAAC,SAAS,CAAC,EAC1B,KAAK,EAAE,SAAS,CAAC,SAAS,CAAC,EAC3B,OAAO,CAAC,EAAE,wBAAwB,GACjC,mBAAmB,CAwKrB;AAED,MAAM,MAAM,UAAU,GAAG,MAAM,GAAG,QAAQ,GAAG,QAAQ,CAAC;AACtD,MAAM,MAAM,eAAe,GAAG,UAAU,GAAG,UAAU,CAAC;AACtD,MAAM,MAAM,iBAAiB,GACzB,gBAAgB,GAChB,oBAAoB,GACpB,kBAAkB,GAClB,UAAU,CAAC;AAEf,MAAM,WAAW,yBAAyB;IACxC,MAAM,EAAE,OAAO,GAAG,UAAU,GAAG,YAAY,CAAC;IAC5C,MAAM,CAAC,EAAE,OAAO,CAAC;CAClB;AAED,MAAM,WAAW,0BAA0B;IACzC,IAAI,EAAE,UAAU,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,OAAO,CAAC;IACnB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,MAAM,MAAM,4BAA4B,GAAG,CACzC,KAAK,EAAE,0BAA0B,KAC9B,yBAAyB,GAAG,OAAO,CAAC,yBAAyB,CAAC,CAAC;AAEpE,MAAM,WAAW,WAAW;IAC1B,iBAAiB,EAAE,MAAM,CAAC;IAC1B,kBAAkB,EAAE,MAAM,CAAC;IAC3B,gBAAgB,EAAE,MAAM,CAAC;IACzB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,WAAW,EAAE,iBAAiB,CAAC;CAChC;AAED,MAAM,WAAW,wBAAwB,CAAC,QAAQ,GAAG,OAAO;IAC1D,eAAe,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IACrC,KAAK,CAAC,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IACxB,YAAY,CAAC,EAAE,MAAM,KAAK,CAAC,QAAQ,CAAC,GAAG,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;IAChE,IAAI,EAAE,UAAU,CAAC;IACjB,UAAU,CAAC,EAAE,IAAI,CAAC,mBAAmB,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC3D,SAAS,CAAC,EAAE,eAAe,CAAC;IAC5B,iBAAiB,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IAC5C,iBAAiB,CAAC,EAAE,4BAA4B,CAAC;CAClD;AAED,MAAM,WAAW,yBAAyB,CAAC,QAAQ,GAAG,OAAO;IAC3D,MAAM,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC5B,eAAe,CAAC,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IACtC,WAAW,EAAE,WAAW,CAAC;CAC1B;AAwUD,wBAAsB,mBAAmB,CAAC,QAAQ,GAAG,OAAO,EAC1D,KAAK,EAAE,wBAAwB,CAAC,QAAQ,CAAC,GACxC,OAAO,CAAC,yBAAyB,CAAC,QAAQ,CAAC,CAAC,CAmE9C"}

package/dist/run-record-utils.js

@@ -278,16 +278,61 @@ function buildRecordedToolQueues(source) {
     const calls = extractToolCalls(source).filter((call) => call.hasOutput && call.name.length > 0);
     const queues = new Map();
     for (const call of calls) {
+        const output = toReplayableRecordedToolOutput(call.output);
+        if (output.status === "not_replayable") {
+            continue;
+        }
         const key = toToolCallKey(call);
         const queue = queues.get(key);
+        const replayableCall = {
+            ...call,
+            output: output.value,
+        };
         if (queue) {
-            queue.push(call);
+            queue.push(replayableCall);
             continue;
         }
-        queues.set(key, [call]);
+        queues.set(key, [replayableCall]);
     }
     return queues;
 }
+function isRecordObject(value) {
+    return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function isToolResultEnvelopeLike(value) {
+    if (!isRecordObject(value)) {
+        return false;
+    }
+    return ((value.status === "ok" ||
+        value.status === "denied" ||
+        value.status === "approval_required") &&
+        "code" in value &&
+        "publicReason" in value &&
+        "data" in value);
+}
+function toReplayableRecordedToolOutput(value) {
+    if (!isToolResultEnvelopeLike(value)) {
+        return {
+            status: "replayable",
+            value,
+        };
+    }
+    if (value.status === "ok") {
+        return {
+            status: "replayable",
+            value: value.data,
+        };
+    }
+    if (value.status === "denied" || value.status === "approval_required") {
+        return {
+            status: "not_replayable",
+        };
+    }
+    return {
+        status: "replayable",
+        value,
+    };
+}
 function findRecordedToolOutput(queues, toolName, args, mode) {
     const argsCanonicalJson = (0, canonical_json_1.toCanonicalJson)(args);
     const argsHash = (0, canonical_json_1.hashCanonicalJson)(argsCanonicalJson);
@@ -396,15 +441,48 @@ async function resolveReplayAgent(input) {
     }
     throw new Error("Replay configuration is missing an agent or agentFactory.");
 }
+function isValidInputItemCount(value, items) {
+    return (typeof value === "number" &&
+        Number.isInteger(value) &&
+        value >= 0 &&
+        value <= items.length);
+}
+function resolveReplayInput(sourceRunRecord, inputMode) {
+    if (inputMode === "question") {
+        return {
+            input: sourceRunRecord.question,
+            inputSource: "question",
+        };
+    }
+    if (isValidInputItemCount(sourceRunRecord.inputItemCount, sourceRunRecord.items)) {
+        return {
+            input: sourceRunRecord.items.slice(0, sourceRunRecord.inputItemCount),
+            inputSource: "inputItemCount",
+        };
+    }
+    const firstRequestMessageCount = sourceRunRecord.requestFingerprints[0]?.messageCount;
+    if (isValidInputItemCount(firstRequestMessageCount, sourceRunRecord.items)) {
+        return {
+            input: sourceRunRecord.items.slice(0, firstRequestMessageCount),
+            inputSource: "requestFingerprint",
+        };
+    }
+    return {
+        input: sourceRunRecord.question,
+        inputSource: "questionFallback",
+    };
+}
 async function replayFromRunRecord(input) {
     const baseAgent = await resolveReplayAgent(input);
     const extractedSourceCalls = extractToolCalls(input.sourceRunRecord);
+    const replayInput = resolveReplayInput(input.sourceRunRecord, input.inputMode ?? "recorded");
     const replayStats = {
         recordedToolCalls: extractedSourceCalls.filter((call) => call.hasOutput)
             .length,
         replayedFromRecord: 0,
         missingToolCalls: 0,
         liveFallbackCalls: 0,
+        inputSource: replayInput.inputSource,
     };
     const mode = input.mode;
     const replayAgent = mode === "live"
@@ -432,7 +510,7 @@ async function replayFromRunRecord(input) {
             },
         };
     }
-    const result = await (0, run_1.run)(replayAgent, input.sourceRunRecord.question, {
+    const result = await (0, run_1.run)(replayAgent, replayInput.input, {
         ...runOptions,
         context: replayContext,
         policies: replayPolicies,

package/dist/run-record.d.ts

@@ -93,6 +93,7 @@ export interface RunRecord<TContext = unknown> {
     contextSnapshot: TContext;
     contextRedacted?: boolean;
     items: AgentInputItem[];
+    inputItemCount?: number;
     promptSnapshots: PromptSnapshotRecord[];
     requestFingerprints: RequestFingerprintRecord[];
     policyDecisions: PolicyDecisionRecord[];

package/dist/run-record.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run-record.d.ts","sourceRoot":"","sources":["../src/run-record.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,SAAS,CAAC;AAE9C,MAAM,MAAM,eAAe,GAAG,WAAW,GAAG,QAAQ,CAAC;AAErD,MAAM,WAAW,sBAAsB;IACrC,IAAI,EAAE,MAAM,GAAG,SAAS,CAAC;IACzB,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,oBAAoB;IACnC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,OAAO,GAAG,MAAM,GAAG,kBAAkB,CAAC;IAChD,MAAM,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,CAAC,EAAE,OAAO,GAAG,aAAa,CAAC;IACrC,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,sBAAsB,CAAC;IACjC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,uBAAuB;IACtC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,aAAa,EAAE,MAAM,CAAC;IACtB,QAAQ,EAAE,MAAM,GAAG,WAAW,CAAC;IAC/B,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,oBAAoB;IACnC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,wBAAwB;IACvC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,EAAE,MAAM,CAAC;IACd,cAAc,EAAE,MAAM,CAAC;IACvB,wBAAwB,EAAE,MAAM,CAAC;IACjC,WAAW,EAAE,MAAM,CAAC;IACpB,gBAAgB,EAAE,MAAM,CAAC;IACzB,YAAY,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,MAAM,CAAC;IAClB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,YAAY,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,qBAAqB;IACpC,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,MAAM,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,qBAAsB,SAAQ,qBAAqB;IAClE,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,YAAY,EAAE,MAAM,CAAC;IACrB,eAAe,EAAE,OAAO,CAAC;IACzB,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,wBAAyB,SAAQ,qBAAqB;IACrE,IAAI,EAAE,SAAS,CAAC;IAChB,aAAa,EAAE,MAAM,CAAC;IACtB,WAAW,EAAE,MAAM,CAAC;IACpB,cAAc,EAAE,OAAO,CAAC;IACxB,oBAAoB,EAAE,MAAM,CAAC;CAC9B;AAED,MAAM,MAAM,iBAAiB,GACzB,qBAAqB,GACrB,wBAAwB,CAAC;AAE7B,MAAM,WAAW,SAAS,CAAC,QAAQ,GAAG,OAAO;IAC3C,KAAK,EAAE,MAAM,CAAC;IACd,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,eAAe,CAAC;IACxB,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,eAAe,EAAE,QAAQ,CAAC;IAC1B,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B,KAAK,EAAE,cAAc,EAAE,CAAC;IACxB,eAAe,EAAE,oBAAoB,EAAE,CAAC;IACxC,mBAAmB,EAAE,wBAAwB,EAAE,CAAC;IAChD,eAAe,EAAE,oBAAoB,EAAE,CAAC;IACxC,kBAAkB,CAAC,EAAE,iBAAiB,EAAE,CAAC;IACzC,kBAAkB,CAAC,EAAE,uBAAuB,EAAE,CAAC;IAC/C,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,+BAA+B,CAAC,QAAQ,GAAG,OAAO;IACjE,eAAe,EAAE,QAAQ,CAAC;IAC1B,eAAe,EAAE,OAAO,CAAC;CAC1B;AAED,MAAM,MAAM,wBAAwB,CAAC,QAAQ,GAAG,OAAO,IAAI,CACzD,OAAO,EAAE,QAAQ,KAEf,+BAA+B,CAAC,QAAQ,CAAC,GACzC,OAAO,CAAC,+BAA+B,CAAC,QAAQ,CAAC,CAAC,CAAC;AAEvD,MAAM,MAAM,eAAe,CAAC,QAAQ,GAAG,OAAO,IAAI,CAChD,MAAM,EAAE,SAAS,CAAC,QAAQ,CAAC,KACxB,OAAO,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC;AAE1B,MAAM,WAAW,aAAa,CAAC,QAAQ,GAAG,OAAO;IAC/C,KAAK,EAAE,eAAe,CAAC,QAAQ,CAAC,CAAC;CAClC;AAED,MAAM,WAAW,gBAAgB,CAAC,QAAQ,GAAG,OAAO;IAClD,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACnC,eAAe,CAAC,EAAE,wBAAwB,CAAC,QAAQ,CAAC,CAAC;IACrD,iBAAiB,CAAC,EAAE,OAAO,CAAC;IAC5B,IAAI,CAAC,EAAE,aAAa,CAAC,QAAQ,CAAC,GAAG,eAAe,CAAC,QAAQ,CAAC,CAAC;CAC5D"}
+{"version":3,"file":"run-record.d.ts","sourceRoot":"","sources":["../src/run-record.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,SAAS,CAAC;AAE9C,MAAM,MAAM,eAAe,GAAG,WAAW,GAAG,QAAQ,CAAC;AAErD,MAAM,WAAW,sBAAsB;IACrC,IAAI,EAAE,MAAM,GAAG,SAAS,CAAC;IACzB,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,oBAAoB;IACnC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,OAAO,GAAG,MAAM,GAAG,kBAAkB,CAAC;IAChD,MAAM,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,CAAC,EAAE,OAAO,GAAG,aAAa,CAAC;IACrC,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,sBAAsB,CAAC;IACjC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,uBAAuB;IACtC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,aAAa,EAAE,MAAM,CAAC;IACtB,QAAQ,EAAE,MAAM,GAAG,WAAW,CAAC;IAC/B,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,oBAAoB;IACnC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB;AAED,MAAM,WAAW,wBAAwB;IACvC,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,EAAE,MAAM,CAAC;IACb,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,KAAK,EAAE,MAAM,CAAC;IACd,cAAc,EAAE,MAAM,CAAC;IACvB,wBAAwB,EAAE,MAAM,CAAC;IACjC,WAAW,EAAE,MAAM,CAAC;IACpB,gBAAgB,EAAE,MAAM,CAAC;IACzB,YAAY,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,MAAM,CAAC;IAClB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,YAAY,EAAE,MAAM,CAAC;IACrB,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,qBAAqB;IACpC,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,MAAM,CAAC;IACf,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,MAAM,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,qBAAsB,SAAQ,qBAAqB;IAClE,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,YAAY,EAAE,MAAM,CAAC;IACrB,eAAe,EAAE,OAAO,CAAC;IACzB,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,wBAAyB,SAAQ,qBAAqB;IACrE,IAAI,EAAE,SAAS,CAAC;IAChB,aAAa,EAAE,MAAM,CAAC;IACtB,WAAW,EAAE,MAAM,CAAC;IACpB,cAAc,EAAE,OAAO,CAAC;IACxB,oBAAoB,EAAE,MAAM,CAAC;CAC9B;AAED,MAAM,MAAM,iBAAiB,GACzB,qBAAqB,GACrB,wBAAwB,CAAC;AAE7B,MAAM,WAAW,SAAS,CAAC,QAAQ,GAAG,OAAO;IAC3C,KAAK,EAAE,MAAM,CAAC;IACd,SAAS,EAAE,MAAM,CAAC;IAClB,WAAW,EAAE,MAAM,CAAC;IACpB,MAAM,EAAE,eAAe,CAAC;IACxB,SAAS,EAAE,MAAM,CAAC;IAClB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,MAAM,CAAC;IACjB,eAAe,EAAE,QAAQ,CAAC;IAC1B,eAAe,CAAC,EAAE,OAAO,CAAC;IAC1B,KAAK,EAAE,cAAc,EAAE,CAAC;IACxB,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,eAAe,EAAE,oBAAoB,EAAE,CAAC;IACxC,mBAAmB,EAAE,wBAAwB,EAAE,CAAC;IAChD,eAAe,EAAE,oBAAoB,EAAE,CAAC;IACxC,kBAAkB,CAAC,EAAE,iBAAiB,EAAE,CAAC;IACzC,kBAAkB,CAAC,EAAE,uBAAuB,EAAE,CAAC;IAC/C,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACpC;AAED,MAAM,WAAW,+BAA+B,CAAC,QAAQ,GAAG,OAAO;IACjE,eAAe,EAAE,QAAQ,CAAC;IAC1B,eAAe,EAAE,OAAO,CAAC;CAC1B;AAED,MAAM,MAAM,wBAAwB,CAAC,QAAQ,GAAG,OAAO,IAAI,CACzD,OAAO,EAAE,QAAQ,KAEf,+BAA+B,CAAC,QAAQ,CAAC,GACzC,OAAO,CAAC,+BAA+B,CAAC,QAAQ,CAAC,CAAC,CAAC;AAEvD,MAAM,MAAM,eAAe,CAAC,QAAQ,GAAG,OAAO,IAAI,CAChD,MAAM,EAAE,SAAS,CAAC,QAAQ,CAAC,KACxB,OAAO,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC;AAE1B,MAAM,WAAW,aAAa,CAAC,QAAQ,GAAG,OAAO;IAC/C,KAAK,EAAE,eAAe,CAAC,QAAQ,CAAC,CAAC;CAClC;AAED,MAAM,WAAW,gBAAgB,CAAC,QAAQ,GAAG,OAAO;IAClD,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACnC,eAAe,CAAC,EAAE,wBAAwB,CAAC,QAAQ,CAAC,CAAC;IACrD,iBAAiB,CAAC,EAAE,OAAO,CAAC;IAC5B,IAAI,CAAC,EAAE,aAAa,CAAC,QAAQ,CAAC,GAAG,eAAe,CAAC,QAAQ,CAAC,CAAC;CAC5D"}

package/dist/run-recorder-runtime.d.ts

@@ -17,6 +17,7 @@ export type PendingRequestFingerprintRecord = Omit<RequestFingerprintRecord, "ti
 };
 interface RunRecorderCreateOptions<TContext> {
     input: string | AgentInputItem[];
+    inputItemCount: number;
     context: TContext;
     providerName: string;
     recordOptions?: RunRecordOptions<TContext>;
@@ -31,6 +32,7 @@ export declare class RunRecorder<TContext = unknown> {
     private readonly runRecordId;
     private readonly startedAt;
     private readonly question;
+    private readonly inputItemCount;
     private readonly providerName;
     private readonly metadata?;
     private readonly contextSnapshot;

package/dist/run-recorder-runtime.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run-recorder-runtime.d.ts","sourceRoot":"","sources":["../src/run-recorder-runtime.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EACV,uBAAuB,EACvB,oBAAoB,EACpB,oBAAoB,EACpB,wBAAwB,EAGxB,gBAAgB,EAEhB,iBAAiB,EAClB,MAAM,cAAc,CAAC;AACtB,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,SAAS,CAAC;AAE9C,MAAM,MAAM,2BAA2B,GAAG,IAAI,CAC5C,oBAAoB,EACpB,WAAW,CACZ,CAAC;AACF,MAAM,MAAM,8BAA8B,GAAG,IAAI,CAC/C,uBAAuB,EACvB,WAAW,CACZ,CAAC;AACF,MAAM,MAAM,2BAA2B,GAAG,IAAI,CAC5C,oBAAoB,EACpB,WAAW,GAAG,YAAY,GAAG,YAAY,CAC1C,GAAG;IACF,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB,CAAC;AACF,MAAM,MAAM,+BAA+B,GAAG,IAAI,CAChD,wBAAwB,EACtB,WAAW,GACX,gBAAgB,GAChB,0BAA0B,GAC1B,aAAa,GACb,kBAAkB,GAClB,cAAc,GACd,WAAW,GACX,mBAAmB,GACnB,cAAc,GACd,WAAW,CACd,GAAG;IACF,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,EAAE,cAAc,EAAE,CAAC;IAC3B,KAAK,EAAE,KAAK,CAAC;QACX,IAAI,EAAE,MAAM,CAAC;QACb,WAAW,EAAE,MAAM,CAAC;QACpB,UAAU,EAAE,OAAO,CAAC;KACrB,CAAC,CAAC;IACH,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACzC,CAAC;AA+BF,UAAU,wBAAwB,CAAC,QAAQ;IACzC,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,CAAC;IACjC,OAAO,EAAE,QAAQ,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,aAAa,CAAC,EAAE,gBAAgB,CAAC,QAAQ,CAAC,CAAC;CAC5C;AAED,UAAU,0BAA0B;IAClC,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,cAAc,EAAE,CAAC;CACzB;AAyFD,qBAAa,WAAW,CAAC,QAAQ,GAAG,OAAO;IACzC,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAmC;IACnE,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAS;IACrC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAS;IACtC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA0B;IACpD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA4C;IAC5E,OAAO,CAAC,QAAQ,CAAC,iBAAiB,CAAU;IAE5C,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA8B;IAC9D,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAA2B;IAC9D,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAiC;IACpE,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA8B;IAC9D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAkC;IACtE,OAAO,CAAC,mBAAmB,CAAM;IACjC,OAAO,CAAC,gBAAgB,CAAS;IAEjC,OAAO;WAoBM,MAAM,CAAC,QAAQ,GAAG,OAAO,EACpC,OAAO,EAAE,wBAAwB,CAAC,QAAQ,CAAC,GAC1C,OAAO,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAC;IAsBjC,IAAI,KAAK,IAAI,MAAM,CAElB;IAED,gBAAgB,GAAI,UAAU,2BAA2B,KAAG,IAAI,CAK9D;IAEF,mBAAmB,GAAI,UAAU,iBAAiB,KAAG,IAAI,CAEvD;IAEF,mBAAmB,GAAI,UAAU,8BAA8B,KAAG,IAAI,CAKpE;IAEF,gBAAgB,GAAI,UAAU,2BAA2B,KAAG,IAAI,CAW9D;IAEF,oBAAoB,GAClB,aAAa,+BAA+B,KAC3C,IAAI,CAgCL;IAEF,eAAe,GAAI,SAAS,MAAM,KAAG,IAAI,CAEvC;IAEI,aAAa,CAAC,OAAO,EAAE,0BAA0B,GAAG,OAAO,CAAC,IAAI,CAAC;IAIjE,UAAU,CACd,OAAO,EAAE,0BAA0B,EACnC,KAAK,EAAE,OAAO,GACb,OAAO,CAAC,IAAI,CAAC;YAIF,IAAI;CA8CnB"}
+{"version":3,"file":"run-recorder-runtime.d.ts","sourceRoot":"","sources":["../src/run-recorder-runtime.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EACV,uBAAuB,EACvB,oBAAoB,EACpB,oBAAoB,EACpB,wBAAwB,EAGxB,gBAAgB,EAEhB,iBAAiB,EAClB,MAAM,cAAc,CAAC;AACtB,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,SAAS,CAAC;AAE9C,MAAM,MAAM,2BAA2B,GAAG,IAAI,CAC5C,oBAAoB,EACpB,WAAW,CACZ,CAAC;AACF,MAAM,MAAM,8BAA8B,GAAG,IAAI,CAC/C,uBAAuB,EACvB,WAAW,CACZ,CAAC;AACF,MAAM,MAAM,2BAA2B,GAAG,IAAI,CAC5C,oBAAoB,EACpB,WAAW,GAAG,YAAY,GAAG,YAAY,CAC1C,GAAG;IACF,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB,CAAC;AACF,MAAM,MAAM,+BAA+B,GAAG,IAAI,CAChD,wBAAwB,EACtB,WAAW,GACX,gBAAgB,GAChB,0BAA0B,GAC1B,aAAa,GACb,kBAAkB,GAClB,cAAc,GACd,WAAW,GACX,mBAAmB,GACnB,cAAc,GACd,WAAW,CACd,GAAG;IACF,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,QAAQ,EAAE,cAAc,EAAE,CAAC;IAC3B,KAAK,EAAE,KAAK,CAAC;QACX,IAAI,EAAE,MAAM,CAAC;QACb,WAAW,EAAE,MAAM,CAAC;QACpB,UAAU,EAAE,OAAO,CAAC;KACrB,CAAC,CAAC;IACH,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;CACzC,CAAC;AA+BF,UAAU,wBAAwB,CAAC,QAAQ;IACzC,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,CAAC;IACjC,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,EAAE,QAAQ,CAAC;IAClB,YAAY,EAAE,MAAM,CAAC;IACrB,aAAa,CAAC,EAAE,gBAAgB,CAAC,QAAQ,CAAC,CAAC;CAC5C;AAED,UAAU,0BAA0B;IAClC,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,KAAK,EAAE,cAAc,EAAE,CAAC;CACzB;AAyFD,qBAAa,WAAW,CAAC,QAAQ,GAAG,OAAO;IACzC,OAAO,CAAC,QAAQ,CAAC,eAAe,CAAmC;IACnE,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAS;IACrC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAS;IAClC,OAAO,CAAC,QAAQ,CAAC,cAAc,CAAS;IACxC,OAAO,CAAC,QAAQ,CAAC,YAAY,CAAS;IACtC,OAAO,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAA0B;IACpD,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA4C;IAC5E,OAAO,CAAC,QAAQ,CAAC,iBAAiB,CAAU;IAE5C,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA8B;IAC9D,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAA2B;IAC9D,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAiC;IACpE,OAAO,CAAC,QAAQ,CAAC,eAAe,CAA8B;IAC9D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAkC;IACtE,OAAO,CAAC,mBAAmB,CAAM;IACjC,OAAO,CAAC,gBAAgB,CAAS;IAEjC,OAAO;WAsBM,MAAM,CAAC,QAAQ,GAAG,OAAO,EACpC,OAAO,EAAE,wBAAwB,CAAC,QAAQ,CAAC,GAC1C,OAAO,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAC;IAuBjC,IAAI,KAAK,IAAI,MAAM,CAElB;IAED,gBAAgB,GAAI,UAAU,2BAA2B,KAAG,IAAI,CAK9D;IAEF,mBAAmB,GAAI,UAAU,iBAAiB,KAAG,IAAI,CAEvD;IAEF,mBAAmB,GAAI,UAAU,8BAA8B,KAAG,IAAI,CAKpE;IAEF,gBAAgB,GAAI,UAAU,2BAA2B,KAAG,IAAI,CAW9D;IAEF,oBAAoB,GAClB,aAAa,+BAA+B,KAC3C,IAAI,CAgCL;IAEF,eAAe,GAAI,SAAS,MAAM,KAAG,IAAI,CAEvC;IAEI,aAAa,CAAC,OAAO,EAAE,0BAA0B,GAAG,OAAO,CAAC,IAAI,CAAC;IAIjE,UAAU,CACd,OAAO,EAAE,0BAA0B,EACnC,KAAK,EAAE,OAAO,GACb,OAAO,CAAC,IAAI,CAAC;YAIF,IAAI;CA+CnB"}

package/dist/run-recorder-runtime.js

@@ -94,6 +94,7 @@ class RunRecorder {
     runRecordId;
     startedAt;
     question;
+    inputItemCount;
     providerName;
     metadata;
     contextSnapshot;
@@ -105,11 +106,12 @@ class RunRecorder {
     requestFingerprints = [];
     observedFinalOutput = "";
     runRecordWritten = false;
-    constructor(runRecordWriter, runRecordId, startedAt, question, providerName, metadata, contextSnapshot, includePromptText) {
+    constructor(runRecordWriter, runRecordId, startedAt, question, inputItemCount, providerName, metadata, contextSnapshot, includePromptText) {
         this.runRecordWriter = runRecordWriter;
         this.runRecordId = runRecordId;
         this.startedAt = startedAt;
         this.question = question;
+        this.inputItemCount = inputItemCount;
         this.providerName = providerName;
         this.metadata = metadata;
         this.contextSnapshot = contextSnapshot;
@@ -121,7 +123,7 @@ class RunRecorder {
         const startedAt = new Date().toISOString();
         const question = extractQuestion(options.input);
         const contextSnapshot = await resolveContextSnapshot(options.context, options.recordOptions);
-        return new RunRecorder(runRecordWriter, runRecordId, startedAt, question, options.providerName, options.recordOptions?.metadata, contextSnapshot, options.recordOptions?.includePromptText ?? false);
+        return new RunRecorder(runRecordWriter, runRecordId, startedAt, question, options.inputItemCount, options.providerName, options.recordOptions?.metadata, contextSnapshot, options.recordOptions?.includePromptText ?? false);
     }
     get runId() {
         return this.runRecordId;
@@ -213,6 +215,7 @@ class RunRecorder {
             contextSnapshot: this.contextSnapshot.contextSnapshot,
             contextRedacted: this.contextSnapshot.contextRedacted,
             items: [...options.items],
+            inputItemCount: this.inputItemCount,
             promptSnapshots: [...this.promptSnapshots],
             requestFingerprints: [...this.requestFingerprints],
             policyDecisions: [...this.policyDecisions],

package/dist/run-regression-summary.d.ts

@@ -0,0 +1,34 @@
+import type { RunRegressionResult, RunRegressionStatus } from "./run-regression";
+export interface RunRegressionCaseSummary {
+    name: string;
+    status: RunRegressionStatus;
+    baselineRunId: string;
+    candidateRunId: string;
+    signals: {
+        statusChanged: boolean;
+        toolsChanged: boolean;
+        policyChanged: boolean;
+        finalOutputChanged: boolean;
+    };
+    judge?: {
+        verdict: RunRegressionStatus;
+        summary: string;
+    };
+}
+export interface RunRegressionSummary {
+    suite?: string;
+    status: RunRegressionStatus;
+    totals: {
+        cases: number;
+        passed: number;
+        warned: number;
+        failed: number;
+    };
+    cases: RunRegressionCaseSummary[];
+}
+export interface SummarizeRunRegressionResultsOptions<TContext = unknown> {
+    suite?: string;
+    classifyCase?: (result: RunRegressionResult<TContext>) => RunRegressionStatus;
+}
+export declare function summarizeRunRegressionResults<TContext = unknown>(results: Array<RunRegressionResult<TContext>>, options?: SummarizeRunRegressionResultsOptions<TContext>): RunRegressionSummary;
+//# sourceMappingURL=run-regression-summary.d.ts.map

package/dist/run-regression-summary.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"run-regression-summary.d.ts","sourceRoot":"","sources":["../src/run-regression-summary.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,mBAAmB,EACnB,mBAAmB,EACpB,MAAM,kBAAkB,CAAC;AAE1B,MAAM,WAAW,wBAAwB;IACvC,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,mBAAmB,CAAC;IAC5B,aAAa,EAAE,MAAM,CAAC;IACtB,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,EAAE;QACP,aAAa,EAAE,OAAO,CAAC;QACvB,YAAY,EAAE,OAAO,CAAC;QACtB,aAAa,EAAE,OAAO,CAAC;QACvB,kBAAkB,EAAE,OAAO,CAAC;KAC7B,CAAC;IACF,KAAK,CAAC,EAAE;QACN,OAAO,EAAE,mBAAmB,CAAC;QAC7B,OAAO,EAAE,MAAM,CAAC;KACjB,CAAC;CACH;AAED,MAAM,WAAW,oBAAoB;IACnC,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,mBAAmB,CAAC;IAC5B,MAAM,EAAE;QACN,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;QACf,MAAM,EAAE,MAAM,CAAC;QACf,MAAM,EAAE,MAAM,CAAC;KAChB,CAAC;IACF,KAAK,EAAE,wBAAwB,EAAE,CAAC;CACnC;AAED,MAAM,WAAW,oCAAoC,CAAC,QAAQ,GAAG,OAAO;IACtE,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,CAAC,MAAM,EAAE,mBAAmB,CAAC,QAAQ,CAAC,KAAK,mBAAmB,CAAC;CAC/E;AA2DD,wBAAgB,6BAA6B,CAAC,QAAQ,GAAG,OAAO,EAC9D,OAAO,EAAE,KAAK,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC,EAC7C,OAAO,GAAE,oCAAoC,CAAC,QAAQ,CAAM,GAC3D,oBAAoB,CA2BtB"}

package/dist/run-regression-summary.js

@@ -0,0 +1,71 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.summarizeRunRegressionResults = summarizeRunRegressionResults;
+function defaultClassifyRunRegressionCase(result) {
+    if (result.candidate.status === "failed" ||
+        result.judge?.verdict === "fail") {
+        return "fail";
+    }
+    if (!result.comparison.equal || result.judge?.verdict === "warn") {
+        return "warn";
+    }
+    return "pass";
+}
+function summarizeCase(result, status) {
+    const judge = result.judge
+        ? {
+            verdict: result.judge.verdict,
+            summary: result.judge.summary,
+        }
+        : undefined;
+    return {
+        name: result.name,
+        status,
+        baselineRunId: result.baseline.runId,
+        candidateRunId: result.candidate.runId,
+        signals: {
+            statusChanged: result.baseline.status !== result.candidate.status,
+            toolsChanged: !result.comparison.summary.sameToolCallShape,
+            policyChanged: !result.comparison.summary.samePolicyDecisions,
+            finalOutputChanged: !result.comparison.summary.sameFinalResponse,
+        },
+        ...(judge ? { judge } : {}),
+    };
+}
+function toSuiteStatus(totals) {
+    if (totals.failed > 0) {
+        return "fail";
+    }
+    if (totals.warned > 0) {
+        return "warn";
+    }
+    return "pass";
+}
+function summarizeRunRegressionResults(results, options = {}) {
+    const classifyCase = options.classifyCase ?? (defaultClassifyRunRegressionCase);
+    const totals = {
+        cases: results.length,
+        passed: 0,
+        warned: 0,
+        failed: 0,
+    };
+    const cases = results.map((result) => {
+        const status = classifyCase(result);
+        if (status === "pass") {
+            totals.passed += 1;
+        }
+        else if (status === "warn") {
+            totals.warned += 1;
+        }
+        else {
+            totals.failed += 1;
+        }
+        return summarizeCase(result, status);
+    });
+    return {
+        ...(typeof options.suite === "undefined" ? {} : { suite: options.suite }),
+        status: toSuiteStatus(totals),
+        totals,
+        cases,
+    };
+}

package/dist/run-regression.d.ts

@@ -0,0 +1,71 @@
+import type { RunRecord } from "./run-record";
+import { type RunRegressionSummary, type SummarizeRunRegressionResultsOptions } from "./run-regression-summary";
+import { type CompareRunRecordsOptions, type ReplayFromRunRecordInput, type RunRecordComparison } from "./run-record-utils";
+export type RunRegressionStatus = "pass" | "warn" | "fail";
+export interface RunRegressionExpectation {
+    intent?: string;
+    shouldUseTools?: string[];
+    shouldAvoidTools?: string[];
+    shouldPreserve?: string[];
+    shouldImprove?: string[];
+    notes?: string;
+}
+export interface RunJudgeFinding {
+    severity: "info" | "warn" | "error";
+    reason: string;
+    evidence?: string;
+}
+export interface RunJudgeResult {
+    verdict: RunRegressionStatus;
+    summary: string;
+    findings: RunJudgeFinding[];
+    score?: number;
+    judgeModel?: string;
+    judgePromptVersion?: string;
+}
+export interface RunJudgeInput<TContext = unknown, TDescriptor = unknown> {
+    baseline: RunRecord<TContext>;
+    candidate: RunRecord<TContext>;
+    comparison: RunRecordComparison;
+    expectation?: RunRegressionExpectation;
+    baselineDescriptor?: TDescriptor;
+    candidateDescriptor?: TDescriptor;
+}
+export type RunJudge<TContext = unknown, TDescriptor = unknown> = (input: RunJudgeInput<TContext, TDescriptor>) => Promise<RunJudgeResult> | RunJudgeResult;
+export interface RunRegressionResult<TContext = unknown> {
+    name: string;
+    baseline: RunRecord<TContext>;
+    candidate: RunRecord<TContext>;
+    comparison: RunRecordComparison;
+    judge?: RunJudgeResult;
+}
+export interface RunRegressionSuiteCase<TContext = unknown> {
+    name?: string;
+    baseline: RunRecord<TContext>;
+}
+export interface RunRegressionSuite<TContext = unknown> {
+    name?: string;
+    expectation?: RunRegressionExpectation;
+    cases: Array<RunRegressionSuiteCase<TContext>>;
+}
+export interface RunRegressionCaseInput<TContext = unknown> extends Omit<ReplayFromRunRecordInput<TContext>, "sourceRunRecord"> {
+    name?: string;
+    baseline: RunRecord<TContext>;
+    comparisonOptions?: CompareRunRecordsOptions;
+}
+export interface RunRegressionSuiteInput<TContext = unknown, TDescriptor = unknown> extends Omit<RunRegressionCaseInput<TContext>, "baseline" | "name"> {
+    suite: RunRegressionSuite<TContext>;
+    judge?: RunJudge<TContext, TDescriptor>;
+    baselineDescriptor?: TDescriptor;
+    candidateDescriptor?: TDescriptor;
+    summaryOptions?: Omit<SummarizeRunRegressionResultsOptions<TContext>, "suite">;
+}
+export interface RunRegressionSuiteResult<TContext = unknown> {
+    name?: string;
+    expectation?: RunRegressionExpectation;
+    results: Array<RunRegressionResult<TContext>>;
+    summary: RunRegressionSummary;
+}
+export declare function runRegressionCase<TContext = unknown>(input: RunRegressionCaseInput<TContext>): Promise<RunRegressionResult<TContext>>;
+export declare function runRegressionSuite<TContext = unknown, TDescriptor = unknown>(input: RunRegressionSuiteInput<TContext, TDescriptor>): Promise<RunRegressionSuiteResult<TContext>>;
+//# sourceMappingURL=run-regression.d.ts.map

package/dist/run-regression.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"run-regression.d.ts","sourceRoot":"","sources":["../src/run-regression.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,cAAc,CAAC;AAC9C,OAAO,EAEL,KAAK,oBAAoB,EACzB,KAAK,oCAAoC,EAC1C,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAGL,KAAK,wBAAwB,EAC7B,KAAK,wBAAwB,EAC7B,KAAK,mBAAmB,EACzB,MAAM,oBAAoB,CAAC;AAE5B,MAAM,MAAM,mBAAmB,GAAG,MAAM,GAAG,MAAM,GAAG,MAAM,CAAC;AAE3D,MAAM,WAAW,wBAAwB;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,cAAc,CAAC,EAAE,MAAM,EAAE,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,EAAE,CAAC;IAC5B,cAAc,CAAC,EAAE,MAAM,EAAE,CAAC;IAC1B,aAAa,CAAC,EAAE,MAAM,EAAE,CAAC;IACzB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,eAAe;IAC9B,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC;IACpC,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,mBAAmB,CAAC;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE,eAAe,EAAE,CAAC;IAC5B,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,kBAAkB,CAAC,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,aAAa,CAAC,QAAQ,GAAG,OAAO,EAAE,WAAW,GAAG,OAAO;IACtE,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC9B,SAAS,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC/B,UAAU,EAAE,mBAAmB,CAAC;IAChC,WAAW,CAAC,EAAE,wBAAwB,CAAC;IACvC,kBAAkB,CAAC,EAAE,WAAW,CAAC;IACjC,mBAAmB,CAAC,EAAE,WAAW,CAAC;CACnC;AAED,MAAM,MAAM,QAAQ,CAAC,QAAQ,GAAG,OAAO,EAAE,WAAW,GAAG,OAAO,IAAI,CAChE,KAAK,EAAE,aAAa,CAAC,QAAQ,EAAE,WAAW,CAAC,KACxC,OAAO,CAAC,cAAc,CAAC,GAAG,cAAc,CAAC;AAE9C,MAAM,WAAW,mBAAmB,CAAC,QAAQ,GAAG,OAAO;IACrD,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC9B,SAAS,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC/B,UAAU,EAAE,mBAAmB,CAAC;IAChC,KAAK,CAAC,EAAE,cAAc,CAAC;CACxB;AAED,MAAM,WAAW,sBAAsB,CAAC,QAAQ,GAAG,OAAO;IACxD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;CAC/B;AAED,MAAM,WAAW,kBAAkB,CAAC,QAAQ,GAAG,OAAO;IACpD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,wBAAwB,CAAC;IACvC,KAAK,EAAE,KAAK,CAAC,sBAAsB,CAAC,QAAQ,CAAC,CAAC,CAAC;CAChD;AAED,MAAM,WAAW,sBAAsB,CAAC,QAAQ,GAAG,OAAO,CAAE,SAAQ,IAAI,CACtE,wBAAwB,CAAC,QAAQ,CAAC,EAClC,iBAAiB,CAClB;IACC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC9B,iBAAiB,CAAC,EAAE,wBAAwB,CAAC;CAC9C;AAED,MAAM,WAAW,uBAAuB,CACtC,QAAQ,GAAG,OAAO,EAClB,WAAW,GAAG,OAAO,CACrB,SAAQ,IAAI,CAAC,sBAAsB,CAAC,QAAQ,CAAC,EAAE,UAAU,GAAG,MAAM,CAAC;IACnE,KAAK,EAAE,kBAAkB,CAAC,QAAQ,CAAC,CAAC;IACpC,KAAK,CAAC,EAAE,QAAQ,CAAC,QAAQ,EAAE,WAAW,CAAC,CAAC;IACxC,kBAAkB,CAAC,EAAE,WAAW,CAAC;IACjC,mBAAmB,CAAC,EAAE,WAAW,CAAC;IAClC,cAAc,CAAC,EAAE,IAAI,CACnB,oCAAoC,CAAC,QAAQ,CAAC,EAC9C,OAAO,CACR,CAAC;CACH;AAED,MAAM,WAAW,wBAAwB,CAAC,QAAQ,GAAG,OAAO;IAC1D,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,wBAAwB,CAAC;IACvC,OAAO,EAAE,KAAK,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAC9C,OAAO,EAAE,oBAAoB,CAAC;CAC/B;AA0BD,wBAAsB,iBAAiB,CAAC,QAAQ,GAAG,OAAO,EACxD,KAAK,EAAE,sBAAsB,CAAC,QAAQ,CAAC,GACtC,OAAO,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC,CA6BxC;AAiBD,wBAAsB,kBAAkB,CACtC,QAAQ,GAAG,OAAO,EAClB,WAAW,GAAG,OAAO,EAErB,KAAK,EAAE,uBAAuB,CAAC,QAAQ,EAAE,WAAW,CAAC,GACpD,OAAO,CAAC,wBAAwB,CAAC,QAAQ,CAAC,CAAC,CAmD7C"}

package/dist/run-regression.js

@@ -0,0 +1,91 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.runRegressionCase = runRegressionCase;
+exports.runRegressionSuite = runRegressionSuite;
+const run_regression_summary_1 = require("./run-regression-summary");
+const run_record_utils_1 = require("./run-record-utils");
+function toRegressionCaseName(input) {
+    return input.name ?? input.baseline.runId;
+}
+function toReplayRunOptions(input) {
+    return {
+        ...(input.runOptions ?? {}),
+        record: input.runOptions?.record ?? {},
+    };
+}
+function toReplayAgentFields(input) {
+    return {
+        ...(input.agent ? { agent: input.agent } : {}),
+        ...(input.agentFactory ? { agentFactory: input.agentFactory } : {}),
+    };
+}
+async function runRegressionCase(input) {
+    const replay = await (0, run_record_utils_1.replayFromRunRecord)({
+        sourceRunRecord: input.baseline,
+        ...toReplayAgentFields(input),
+        mode: input.mode,
+        runOptions: toReplayRunOptions(input),
+        inputMode: input.inputMode,
+        metadataOverrides: input.metadataOverrides,
+        onMissingToolCall: input.onMissingToolCall,
+    });
+    if (!replay.replayRunRecord) {
+        throw new Error("runRegressionCase expected replayFromRunRecord to produce a candidate RunRecord.");
+    }
+    const comparison = (0, run_record_utils_1.compareRunRecords)(input.baseline, replay.replayRunRecord, input.comparisonOptions);
+    return {
+        name: toRegressionCaseName(input),
+        baseline: input.baseline,
+        candidate: replay.replayRunRecord,
+        comparison,
+    };
+}
+function resolveSuiteCase(caseDefinition) {
+    const baseline = caseDefinition.baseline;
+    const name = caseDefinition.name ?? baseline.runId;
+    return {
+        name,
+        baseline,
+    };
+}
+async function runRegressionSuite(input) {
+    const results = [];
+    const { suite, judge, baselineDescriptor, candidateDescriptor, summaryOptions, ...caseOptions } = input;
+    for (const caseDefinition of suite.cases) {
+        const regressionCase = resolveSuiteCase(caseDefinition);
+        const result = await runRegressionCase({
+            ...caseOptions,
+            name: regressionCase.name,
+            baseline: regressionCase.baseline,
+        });
+        const judgeResult = judge
+            ? await judge({
+                baseline: result.baseline,
+                candidate: result.candidate,
+                comparison: result.comparison,
+                ...(typeof suite.expectation === "undefined"
+                    ? {}
+                    : { expectation: suite.expectation }),
+                ...(typeof baselineDescriptor === "undefined"
+                    ? {}
+                    : { baselineDescriptor }),
+                ...(typeof candidateDescriptor === "undefined"
+                    ? {}
+                    : { candidateDescriptor }),
+            })
+            : undefined;
+        results.push(judgeResult ? { ...result, judge: judgeResult } : result);
+    }
+    const summary = (0, run_regression_summary_1.summarizeRunRegressionResults)(results, {
+        suite: suite.name,
+        ...(summaryOptions ?? {}),
+    });
+    return {
+        ...(typeof suite.name === "undefined" ? {} : { name: suite.name }),
+        ...(typeof suite.expectation === "undefined"
+            ? {}
+            : { expectation: suite.expectation }),
+        results,
+        summary,
+    };
+}

package/dist/run.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run.d.ts","sourceRoot":"","sources":["../src/run.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAsChC,OAAO,EACL,cAAc,EAEd,mBAAmB,EAGnB,SAAS,EACT,cAAc,EACd,gBAAgB,EACjB,MAAM,SAAS,CAAC;AAUjB,KAAK,eAAe,CAAC,QAAQ,IAAI;IAC/B,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,SAAS,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;CAC5B,CAAC;AAGF,YAAY,EACV,kBAAkB,EAClB,wBAAwB,GACzB,MAAM,mBAAmB,CAAC;AAuwB3B,qBAAa,iBAAiB,CAAC,QAAQ,GAAG,OAAO;IAC/C,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,MAAM,CAA0C;IACxD,OAAO,CAAC,KAAK,CAA4B;gBAGvC,MAAM,EAAE,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC,EAC/C,KAAK,EAAE,eAAe,CAAC,QAAQ,CAAC;IAMlC,QAAQ,IAAI,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC;IAQnD,IAAI,OAAO,IAAI,cAAc,EAAE,CAE9B;IAED,IAAI,SAAS,IAAI,KAAK,CAAC,QAAQ,CAAC,CAE/B;CACF;AAED,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,EAAE,gBAAgB,CAAC,QAAQ,CAAC,GAClC,OAAO,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;AAExC,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,CAAC,EAAE,mBAAmB,CAAC,QAAQ,CAAC,GACtC,OAAO,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC"}
+{"version":3,"file":"run.d.ts","sourceRoot":"","sources":["../src/run.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAqB,MAAM,SAAS,CAAC;AAsCnD,OAAO,EACL,cAAc,EAEd,mBAAmB,EAGnB,SAAS,EACT,cAAc,EACd,gBAAgB,EACjB,MAAM,SAAS,CAAC;AAUjB,KAAK,eAAe,CAAC,QAAQ,IAAI;IAC/B,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,SAAS,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;CAC5B,CAAC;AAGF,YAAY,EACV,kBAAkB,EAClB,wBAAwB,GACzB,MAAM,mBAAmB,CAAC;AAmxB3B,qBAAa,iBAAiB,CAAC,QAAQ,GAAG,OAAO;IAC/C,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,MAAM,CAA0C;IACxD,OAAO,CAAC,KAAK,CAA4B;gBAGvC,MAAM,EAAE,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC,EAC/C,KAAK,EAAE,eAAe,CAAC,QAAQ,CAAC;IAMlC,QAAQ,IAAI,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC;IAQnD,IAAI,OAAO,IAAI,cAAc,EAAE,CAE9B;IAED,IAAI,SAAS,IAAI,KAAK,CAAC,QAAQ,CAAC,CAE/B;CACF;AAED,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,EAAE,gBAAgB,CAAC,QAAQ,CAAC,GAClC,OAAO,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;AAExC,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,CAAC,EAAE,mBAAmB,CAAC,QAAQ,CAAC,GACtC,OAAO,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC"}

package/dist/run.js

@@ -62,11 +62,16 @@ function sanitizeToolSegment(input) {
         .replace(/^_+|_+$/g, "");
     return sanitized || "agent";
 }
-function buildTurnTools(agent) {
+function buildTurnTools(agent, runContext) {
     const handoffRegistry = new Map();
     const handoffTools = [];
     const reservedNames = new Set(agent.tools.map((tool) => tool.name));
-    for (const handoffAgent of agent.handoffs) {
+    const handoffRules = agent.handoffRules;
+    for (const handoffRule of handoffRules) {
+        if (handoffRule.enabled && !handoffRule.enabled(runContext)) {
+            continue;
+        }
+        const handoffAgent = handoffRule.agent;
         const baseName = `handoff_to_${sanitizeToolSegment(handoffAgent.name)}`;
         let toolName = baseName;
         let suffix = 2;
@@ -331,7 +336,7 @@ async function* runLoop(state, runId, provider, runContext, maxTurns, logger, po
             activeTurn = turn + 1;
             const currentAgent = state.lastAgent;
             await logEmitter.turnStarted(currentAgent.name, activeTurn);
-            const { providerTools, handoffRegistry } = buildTurnTools(currentAgent);
+            const { providerTools, handoffRegistry } = buildTurnTools(currentAgent, runContext);
             const model = resolveAgentModel(currentAgent);
             const systemPrompt = await currentAgent.resolveInstructions(runContext);
             onPromptSnapshot?.({
@@ -564,6 +569,7 @@ async function run(startingAgent, input, options = {}) {
     const maxTurns = options.maxTurns ?? 10;
     const runRecorder = await run_recorder_runtime_1.RunRecorder.create({
         input,
+        inputItemCount: state.history.length,
         context: runContext.context,
         providerName: provider.constructor.name,
         recordOptions: options.record,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@axiastudio/aioc",
-  "version": "0.2.4",
+  "version": "0.2.6",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
   "files": [
@@ -27,12 +27,14 @@
     "example:tool-policy": "tsx src/examples/basic/tools.ts",
     "example:policy-composition": "tsx src/examples/basic/policy-composition.ts",
     "example:harness": "tsx src/examples/harness-descriptor/customer-support.ts",
+    "example:harness-rerun": "tsx src/examples/harness-descriptor/rerun-modified-harness.ts",
     "example:run-record": "tsx src/examples/basic/run-record-sink.ts",
     "example:rru:01-extract": "tsx src/examples/run-record-utils-minimal/01-extract-tool-calls.ts",
     "example:rru:02-compare": "tsx src/examples/run-record-utils-minimal/02-compare-run-records.ts",
     "example:rru:03-replay-strict": "tsx src/examples/run-record-utils-minimal/03-replay-strict.ts",
     "example:rru:04-replay-hybrid": "tsx src/examples/run-record-utils-minimal/04-replay-hybrid.ts",
     "example:non-regression": "tsx src/examples/non-regression/v1-v2-runrecord-diff.ts",
+    "example:run-regression": "tsx src/examples/run-regression/age-adapted-suite.ts",
     "inspect:samples": "tsx scripts/generate-aioc-inspect-samples.ts",
     "docs:sync": "node scripts/sync-aioc-docs.mjs",
     "docs:dev": "npm run docs:sync && cd apps/aioc-docs && zsh -lc 'npm run dev'",