@axiastudio/aioc 0.2.5 → 0.2.7

package/README.md

@@ -39,9 +39,15 @@ AIOC `0.2.2` completes the descriptor instruction-composition surface with reusa
 AIOC `0.2.3` adds RFC-0010 policy composition helpers for exact-name tool and handoff policy dispatch without changing runtime enforcement semantics.
 AIOC `0.2.4` adds experimental governance-event packages and an OpenTelemetry Logs exporter for reduced, operational observability events derived from `RunRecord` values.
 AIOC `0.2.5` makes replay history-faithful by recording the initial input scope in `RunRecord` and replaying from it by default.
+AIOC `0.2.6` adds descriptor-level conditional agent handoffs with boolean `where` gates that filter handoff tools before provider requests.
 
 ### Experimental Packages
 
+`@axiastudio/aioc-regression-judge` implements the RFC-0012 companion judge
+helpers outside the core runtime package. It builds bounded judge inputs for
+run-regression suites and parses structured `RunJudgeResult` outputs while the
+host application owns the model invocation.
+
 `@axiastudio/aioc-governance-events` implements the RFC-0009 governance-event
 mapper outside the core runtime package. It derives reduced, redacted,
 event-shaped records from `RunRecord` values.
@@ -146,7 +152,7 @@ console.log(result.finalOutput);
 - `loadAgentHarnessDescriptorFromFile(...)`
 - `AgentHarnessDescriptor` and related descriptor types
 
-The Agent Harness Descriptor is included in the supported `0.2.x` API surface as the `aioc.agent_graph.v0` descriptor contract. Use it for controlled configuration, examples, and evaluation harnesses; keep executable tools, policies, providers, persistence, approvals, and deployment configuration in application code.
+The Agent Harness Descriptor is included in the supported `0.2.x` API surface as the `aioc.agent_graph.v0` descriptor contract. Use it for controlled configuration, examples, and evaluation harnesses; keep executable tools, policies, providers, persistence, approvals, and deployment configuration in application code. Descriptor handoffs may use boolean `where` gates to hide unavailable handoff tools before the provider call; `HandoffPolicy` still owns allow/deny/approval decisions for exposed handoffs.
 
 ## Policy Gate (Minimal)
 
@@ -263,6 +269,11 @@ This repository also contains `aioc-inspect`, a private reference example UI for
 
 ## Examples
 
+Suggested reading path: start with `example:hello`, then policy, approval,
+tool-policy, and policy-composition. Use run-record and harness examples after
+the basics; `example:non-regression`, `example:run-regression`, and
+`example:run-regression-judge` are advanced workflows.
+
 | Command | Purpose | Needs API key |
 |---|---|---|
 | `npm run example:hello` | Minimal single-agent run | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
@@ -271,19 +282,38 @@ This repository also contains `aioc-inspect`, a private reference example UI for
 | `npm run example:approval-evidence` | Approval evidence passed through context and reevaluated by policy | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
 | `npm run example:tool-policy` | Straight tool + policy flow with allowed execution | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
 | `npm run example:policy-composition` | Exact-name tool policy dispatch with fallback deny | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
+| `npm run example:harness-rerun` | Replay against a modified harness with a mocked new tool output | Yes (`OPENAI_API_KEY`) |
 | `npm run example:run-record` | Run-record persistence with redaction + audit | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
 | `npm run example:rru:01-extract` | Minimal `extractToolCalls(...)` | No |
 | `npm run example:rru:02-compare` | Minimal `compareRunRecords(...)` | No |
 | `npm run example:rru:03-replay-strict` | Minimal strict replay | No |
 | `npm run example:rru:04-replay-hybrid` | Minimal hybrid replay | No |
 | `npm run example:non-regression` | Advanced v1/v2 run-record diff | Yes (`AIOC_EXAMPLE_PROVIDER` + matching provider API key) |
+| `npm run example:run-regression` | Local `runRegressionSuite(...)` over a modified harness | Yes (`OPENAI_API_KEY`) |
+| `npm run example:run-regression-judge` | Local `runRegressionSuite(...)` with an LLM judge | Yes (`OPENAI_API_KEY`) |
 
 Notes:
 
 - for live-provider examples, set `AIOC_EXAMPLE_PROVIDER` to `openai` or `mistral`; the matching API key must also be available
+- `example:harness-rerun`, `example:run-regression`, and `example:run-regression-judge` configure OpenAI directly from `OPENAI_API_KEY`; harness models are declared in YAML descriptors
+- run-record utility examples are deterministic and do not need a provider
 - `example:non-regression` is educational and can be non-deterministic because it uses a live provider.
 - canonical examples guide: `docs/CANONICAL-EXAMPLES.md`.
 
+### Optional LangChain Examples
+
+Optional LangChain interoperability examples live in `examples/langchain`.
+They use their own `package.json` so LangChain dependencies do not become
+dependencies of the core `@axiastudio/aioc` runtime package.
+
+They demonstrate two composition patterns:
+
+- **aioc-first, LangChain-extended**: aioc owns the governed agent run while
+  LangChain provides OSS components behind aioc tools, such as retrieval.
+- **LangGraph-orchestrated, aioc-governed**: LangGraph owns workflow
+  orchestration while selected graph nodes call aioc for policy-gated execution
+  and portable audit evidence.
+
 ## Test Commands
 
 - `npm run test:unit`

package/dist/agent.d.ts

@@ -3,6 +3,12 @@ import type { OutputGuardrail } from "./guardrails";
 import type { Tool } from "./tool";
 import type { ModelSettings } from "./types";
 export type AgentInstructions<TContext = unknown> = string | ((runContext: RunContext<TContext>) => string | Promise<string>);
+export type AgentHandoffCondition<TContext = unknown> = (runContext: RunContext<TContext>) => boolean;
+export interface AgentHandoff<TContext = unknown> {
+    agent: Agent<TContext>;
+    enabled?: AgentHandoffCondition<TContext>;
+}
+export type AgentHandoffEntry<TContext = unknown> = Agent<TContext> | AgentHandoff<TContext>;
 export interface AgentConfiguration<TContext = unknown> {
     name: string;
     handoffDescription?: string;
@@ -11,7 +17,7 @@ export interface AgentConfiguration<TContext = unknown> {
     model?: string;
     modelSettings?: ModelSettings;
     tools?: Tool<TContext>[];
-    handoffs?: Agent<TContext>[];
+    handoffs?: AgentHandoffEntry<TContext>[];
     outputGuardrails?: OutputGuardrail<TContext>[];
 }
 export declare class Agent<TContext = unknown> {
@@ -23,6 +29,7 @@ export declare class Agent<TContext = unknown> {
     modelSettings?: ModelSettings;
     tools: Tool<TContext>[];
     handoffs: Agent<TContext>[];
+    handoffRules: AgentHandoff<TContext>[];
     outputGuardrails: OutputGuardrail<TContext>[];
     constructor(config: AgentConfiguration<TContext>);
     resolveInstructions(runContext: RunContext<TContext>): Promise<string | undefined>;

package/dist/agent.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,eAAe,CAAC;AAC3C,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAC;AACpD,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AAE7C,MAAM,MAAM,iBAAiB,CAAC,QAAQ,GAAG,OAAO,IAC5C,MAAM,GACN,CAAC,CAAC,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;AAErE,MAAM,WAAW,kBAAkB,CAAC,QAAQ,GAAG,OAAO;IACpD,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,CAAC,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACzB,QAAQ,CAAC,EAAE,KAAK,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC7B,gBAAgB,CAAC,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;CAChD;AAED,qBAAa,KAAK,CAAC,QAAQ,GAAG,OAAO;IACnC,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,EAAE,MAAM,CAAC;IAC3B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACxB,QAAQ,EAAE,KAAK,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC5B,gBAAgB,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAElC,MAAM,EAAE,kBAAkB,CAAC,QAAQ,CAAC;IAY1C,mBAAmB,CACvB,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,GAC/B,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;CAS/B"}
+{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["../src/agent.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,MAAM,eAAe,CAAC;AAC3C,OAAO,KAAK,EAAE,eAAe,EAAE,MAAM,cAAc,CAAC;AACpD,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AAE7C,MAAM,MAAM,iBAAiB,CAAC,QAAQ,GAAG,OAAO,IAC5C,MAAM,GACN,CAAC,CAAC,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,KAAK,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;AAErE,MAAM,MAAM,qBAAqB,CAAC,QAAQ,GAAG,OAAO,IAAI,CACtD,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,KAC7B,OAAO,CAAC;AAEb,MAAM,WAAW,YAAY,CAAC,QAAQ,GAAG,OAAO;IAC9C,KAAK,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IACvB,OAAO,CAAC,EAAE,qBAAqB,CAAC,QAAQ,CAAC,CAAC;CAC3C;AAED,MAAM,MAAM,iBAAiB,CAAC,QAAQ,GAAG,OAAO,IAC5C,KAAK,CAAC,QAAQ,CAAC,GACf,YAAY,CAAC,QAAQ,CAAC,CAAC;AAE3B,MAAM,WAAW,kBAAkB,CAAC,QAAQ,GAAG,OAAO;IACpD,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,CAAC,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACzB,QAAQ,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,EAAE,CAAC;IACzC,gBAAgB,CAAC,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;CAChD;AAED,qBAAa,KAAK,CAAC,QAAQ,GAAG,OAAO;IACnC,IAAI,EAAE,MAAM,CAAC;IACb,kBAAkB,EAAE,MAAM,CAAC;IAC3B,YAAY,CAAC,EAAE,iBAAiB,CAAC,QAAQ,CAAC,CAAC;IAC3C,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,KAAK,EAAE,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;IACxB,QAAQ,EAAE,KAAK,CAAC,QAAQ,CAAC,EAAE,CAAC;IAC5B,YAAY,EAAE,YAAY,CAAC,QAAQ,CAAC,EAAE,CAAC;IACvC,gBAAgB,EAAE,eAAe,CAAC,QAAQ,CAAC,EAAE,CAAC;gBAElC,MAAM,EAAE,kBAAkB,CAAC,QAAQ,CAAC;IAe1C,mBAAmB,CACvB,UAAU,EAAE,UAAU,CAAC,QAAQ,CAAC,GAC/B,OAAO,CAAC,MAAM,GAAG,SAAS,CAAC;CAS/B"}

package/dist/agent.js

@@ -10,6 +10,7 @@ class Agent {
     modelSettings;
     tools;
     handoffs;
+    handoffRules;
     outputGuardrails;
     constructor(config) {
         this.name = config.name;
@@ -19,7 +20,8 @@ class Agent {
         this.model = config.model;
         this.modelSettings = config.modelSettings;
         this.tools = config.tools ?? [];
-        this.handoffs = config.handoffs ?? [];
+        this.handoffRules = (config.handoffs ?? []).map((handoff) => handoff instanceof Agent ? { agent: handoff } : handoff);
+        this.handoffs = this.handoffRules.map((rule) => rule.agent);
         this.outputGuardrails = config.outputGuardrails ?? [];
     }
     async resolveInstructions(runContext) {

package/dist/harness-descriptor.d.ts

@@ -20,11 +20,17 @@ export interface HarnessAgentDescriptor {
     model?: string;
     modelSettings?: Record<string, unknown>;
     tools?: string[];
-    handoffs?: string[];
+    handoffs?: HarnessHandoffEntryDescriptor[];
 }
-export interface HarnessInstructionWhereDescriptor {
+export interface HarnessWhereDescriptor {
     context: string;
 }
+export type HarnessInstructionWhereDescriptor = HarnessWhereDescriptor;
+export interface HarnessHandoffDescriptor {
+    agent: string;
+    where?: HarnessWhereDescriptor;
+}
+export type HarnessHandoffEntryDescriptor = string | HarnessHandoffDescriptor;
 export interface HarnessInstructionPartDescriptor {
     text: string;
     where?: HarnessInstructionWhereDescriptor;

package/dist/harness-descriptor.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"harness-descriptor.d.ts","sourceRoot":"","sources":["../src/harness-descriptor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAA0B,MAAM,SAAS,CAAC;AAExD,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,SAAS,CAAC;AAEnD,MAAM,WAAW,yBAAyB;IACxC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,qBAAqB;IACpC,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,sBAAsB;IACrC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,6BAA6B,CAAC;IAC7C,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACxC,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,QAAQ,CAAC,EAAE,MAAM,EAAE,CAAC;CACrB;AAED,MAAM,WAAW,iCAAiC;IAChD,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,WAAW,gCAAgC;IAC/C,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,iCAAiC,CAAC;CAC3C;AAED,MAAM,MAAM,6BAA6B,GACrC,MAAM,GACN,gCAAgC,EAAE,CAAC;AAEvC,MAAM,WAAW,iCAAiC;IAChD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,MAAM,4BAA4B,GACpC,OAAO,GACP,iCAAiC,CAAC;AAEtC,MAAM,WAAW,6BAA6B;IAC5C,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,6BAA6B,CAAC,CAAC;IACvD,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,4BAA4B,CAAC,CAAC;CAC3D;AAED,MAAM,WAAW,sBAAsB;IACrC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,EAAE,yBAAyB,CAAC;IACrC,OAAO,EAAE,wBAAwB,CAAC;IAClC,OAAO,CAAC,EAAE,wBAAwB,CAAC;IACnC,iBAAiB,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC3C,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,qBAAqB,CAAC,CAAC;IAC9C,cAAc,CAAC,EAAE,IAAI,CACnB,sBAAsB,EACtB,OAAO,GAAG,eAAe,GAAG,cAAc,CAC3C,CAAC;IACF,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC;CAChD;AAED,MAAM,WAAW,oBAAoB,CAAC,QAAQ,GAAG,OAAO;IACtD,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC;IACvC,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,yBAAyB;IACxC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,GAAG,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,SAAS,CAAC,EAAE,OAAO,CAAC;CACrB;AAED,MAAM,WAAW,oBAAoB;IACnC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,cAAc,EAAE,MAAM,CAAC;IACvB,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,YAAY,CAAC,QAAQ,GAAG,OAAO;IAC9C,UAAU,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IAC5B,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;IACrC,cAAc,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,oBAAoB,CAAC;IAC/B,UAAU,EAAE,IAAI,CAAC,mBAAmB,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC1D,aAAa,CAAC,KAAK,CAAC,EAAE,yBAAyB,GAAG,QAAQ,CAAC;CAC5D;AAsiBD,wBAAgB,0BAA0B,CACxC,UAAU,EAAE,sBAAsB,GACjC,MAAM,CAER;AAED,wBAAgB,iBAAiB,CAAC,QAAQ,GAAG,OAAO,EAClD,UAAU,EAAE,sBAAsB,EAClC,QAAQ,GAAE,oBAAoB,CAAC,QAAQ,CAAM,GAC5C,YAAY,CAAC,QAAQ,CAAC,CAqExB"}
+{"version":3,"file":"harness-descriptor.d.ts","sourceRoot":"","sources":["../src/harness-descriptor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAA6C,MAAM,SAAS,CAAC;AAG3E,OAAO,KAAK,EAAE,IAAI,EAAE,MAAM,QAAQ,CAAC;AACnC,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,SAAS,CAAC;AAEnD,MAAM,WAAW,yBAAyB;IACxC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,CAAC,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,qBAAqB;IACpC,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,sBAAsB;IACrC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,YAAY,CAAC,EAAE,6BAA6B,CAAC;IAC7C,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,aAAa,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACxC,KAAK,CAAC,EAAE,MAAM,EAAE,CAAC;IACjB,QAAQ,CAAC,EAAE,6BAA6B,EAAE,CAAC;CAC5C;AAED,MAAM,WAAW,sBAAsB;IACrC,OAAO,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,MAAM,iCAAiC,GAAG,sBAAsB,CAAC;AAEvE,MAAM,WAAW,wBAAwB;IACvC,KAAK,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,sBAAsB,CAAC;CAChC;AAED,MAAM,MAAM,6BAA6B,GAAG,MAAM,GAAG,wBAAwB,CAAC;AAE9E,MAAM,WAAW,gCAAgC;IAC/C,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,CAAC,EAAE,iCAAiC,CAAC;CAC3C;AAED,MAAM,MAAM,6BAA6B,GACrC,MAAM,GACN,gCAAgC,EAAE,CAAC;AAEvC,MAAM,WAAW,iCAAiC;IAChD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,MAAM,4BAA4B,GACpC,OAAO,GACP,iCAAiC,CAAC;AAEtC,MAAM,WAAW,6BAA6B;IAC5C,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,OAAO,CAAC,EAAE,OAAO,CAAC;IAClB,MAAM,CAAC,EAAE,OAAO,CAAC;IACjB,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;CACxB;AAED,MAAM,WAAW,wBAAwB;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,6BAA6B,CAAC,CAAC;IACvD,UAAU,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,4BAA4B,CAAC,CAAC;CAC3D;AAED,MAAM,WAAW,sBAAsB;IACrC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,EAAE,yBAAyB,CAAC;IACrC,OAAO,EAAE,wBAAwB,CAAC;IAClC,OAAO,CAAC,EAAE,wBAAwB,CAAC;IACnC,iBAAiB,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IAC3C,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,qBAAqB,CAAC,CAAC;IAC9C,cAAc,CAAC,EAAE,IAAI,CACnB,sBAAsB,EACtB,OAAO,GAAG,eAAe,GAAG,cAAc,CAC3C,CAAC;IACF,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,sBAAsB,CAAC,CAAC;CAChD;AAED,MAAM,WAAW,oBAAoB,CAAC,QAAQ,GAAG,OAAO;IACtD,KAAK,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC;IACvC,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,yBAAyB;IACxC,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,GAAG,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACpB,SAAS,CAAC,EAAE,OAAO,CAAC;CACrB;AAED,MAAM,WAAW,oBAAoB;IACnC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,cAAc,EAAE,MAAM,CAAC;IACvB,eAAe,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,WAAW,YAAY,CAAC,QAAQ,GAAG,OAAO;IAC9C,UAAU,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;IAC5B,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC;IACrC,cAAc,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,oBAAoB,CAAC;IAC/B,UAAU,EAAE,IAAI,CAAC,mBAAmB,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC1D,aAAa,CAAC,KAAK,CAAC,EAAE,yBAAyB,GAAG,QAAQ,CAAC;CAC5D;AAqoBD,wBAAgB,0BAA0B,CACxC,UAAU,EAAE,sBAAsB,GACjC,MAAM,CAER;AAED,wBAAgB,iBAAiB,CAAC,QAAQ,GAAG,OAAO,EAClD,UAAU,EAAE,sBAAsB,EAClC,QAAQ,GAAE,oBAAoB,CAAC,QAAQ,CAAM,GAC5C,YAAY,CAAC,QAAQ,CAAC,CAiFxB"}

package/dist/harness-descriptor.js

@@ -30,6 +30,28 @@ function assertNonEmptyString(value, label) {
         throw new Error(`${label} must be a non-empty string.`);
     }
 }
+function assertHandoffEntry(value, label) {
+    if (typeof value === "string") {
+        assertNonEmptyString(value, label);
+        return;
+    }
+    if (!isPlainObject(value)) {
+        throw new Error(`${label} must be a non-empty string or an object.`);
+    }
+    assertNonEmptyString(value.agent, `${label}.agent`);
+    if (typeof value.where !== "undefined") {
+        assertInstructionWhere(value.where, label);
+    }
+}
+function assertHandoffEntries(value, label) {
+    if (typeof value === "undefined") {
+        return;
+    }
+    if (!Array.isArray(value)) {
+        throw new Error(`${label} must be an array.`);
+    }
+    value.forEach((entry, index) => assertHandoffEntry(entry, `${label}[${index}]`));
+}
 function validateDescriptorShape(descriptor) {
     assertPlainObject(descriptor, "Harness descriptor");
     assertPlainObject(descriptor.runtime, "Harness descriptor runtime");
@@ -49,6 +71,7 @@ function validateDescriptorShape(descriptor) {
     for (const [agentId, agentDescriptor] of Object.entries(descriptor.agents)) {
         assertPlainObject(agentDescriptor, `Harness descriptor agent "${agentId}"`);
         assertNoUnresolvedInstructionFile(agentDescriptor, `Harness descriptor agent "${agentId}"`);
+        assertHandoffEntries(agentDescriptor.handoffs, `Harness descriptor agent "${agentId}".handoffs`);
     }
     for (const [toolId, toolDescriptor] of Object.entries(descriptor.tools ?? {})) {
         assertPlainObject(toolDescriptor, `Harness descriptor tool "${toolId}"`);
@@ -202,31 +225,37 @@ function assertDeclaredPromptPaths(agentId, paths, promptAccessRules) {
         }
     }
 }
+function assertDeclaredBooleanContextPath(label, path, promptAccessRules) {
+    const rule = promptAccessRules.get(path);
+    if (!rule) {
+        throw new Error(`${label} references undeclared context path "${path}".`);
+    }
+    if (rule.type !== "boolean") {
+        throw new Error(`${label} context path "${path}" must be declared with type "boolean".`);
+    }
+}
 function assertDeclaredBooleanWherePaths(agentId, paths, promptAccessRules) {
     for (const path of paths) {
-        const rule = promptAccessRules.get(path);
-        if (!rule) {
-            throw new Error(`Agent "${agentId}" instruction where references undeclared context path "${path}".`);
-        }
-        if (rule.type !== "boolean") {
-            throw new Error(`Agent "${agentId}" instruction where context path "${path}" must be declared with type "boolean".`);
-        }
+        assertDeclaredBooleanContextPath(`Agent "${agentId}" instruction where`, path, promptAccessRules);
     }
 }
-function shouldIncludeInstructionPart(part, context) {
-    if (!part.where) {
+function shouldIncludeWhere(where, context, label) {
+    if (!where) {
         return true;
     }
-    const path = normalizePromptPath(part.where.context, "Instruction where.context path");
+    const path = normalizePromptPath(where.context, "Instruction where.context path");
     const resolved = readWhereContextValue(context, path);
     if (!resolved.exists || typeof resolved.value === "undefined") {
-        throw new Error(`Harness descriptor instruction where could not resolve context path "${path}".`);
+        throw new Error(`${label} could not resolve context path "${path}".`);
     }
     if (typeof resolved.value !== "boolean") {
-        throw new Error(`Harness descriptor instruction where context path "${path}" must resolve to a boolean.`);
+        throw new Error(`${label} context path "${path}" must resolve to a boolean.`);
     }
     return resolved.value === true;
 }
+function shouldIncludeInstructionPart(part, context) {
+    return shouldIncludeWhere(part.where, context, "Harness descriptor instruction where");
+}
 function renderInstructionParts(instructions, context, promptAccessRules) {
     const parts = [];
     for (const part of instructions) {
@@ -323,6 +352,22 @@ function resolveTool(toolId, descriptor, registry) {
     }
     return toolDefinition;
 }
+function readHandoffEntry(entry) {
+    if (typeof entry === "string") {
+        return {
+            agent: entry,
+        };
+    }
+    return entry;
+}
+function compileHandoffCondition(fromAgentId, toAgentId, where, promptAccessRules) {
+    if (!where) {
+        return undefined;
+    }
+    const path = normalizePromptPath(where.context, `Agent "${fromAgentId}" handoff "${toAgentId}" where.context path`);
+    assertDeclaredBooleanContextPath(`Agent "${fromAgentId}" handoff "${toAgentId}" where`, path, promptAccessRules);
+    return (runContext) => shouldIncludeWhere(where, runContext.context, `Harness descriptor handoff "${fromAgentId}" to "${toAgentId}" where`);
+}
 function hashAgentHarnessDescriptor(descriptor) {
     return `sha256:${(0, canonical_json_1.hashCanonicalJsonValue)(descriptor)}`;
 }
@@ -350,13 +395,20 @@ function buildAgentHarness(descriptor, registry = {}) {
         if (!agent) {
             continue;
         }
-        agent.handoffs = (agentDescriptor.handoffs ?? []).map((handoffId) => {
+        const handoffRules = (agentDescriptor.handoffs ?? []).map((entry) => {
+            const handoff = readHandoffEntry(entry);
+            const handoffId = handoff.agent;
             const handoffAgent = agents.get(handoffId);
             if (!handoffAgent) {
                 throw new Error(`Harness descriptor references unknown handoff agent "${handoffId}" from "${agentId}".`);
             }
-            return handoffAgent;
+            return {
+                agent: handoffAgent,
+                enabled: compileHandoffCondition(agentId, handoffId, handoff.where, promptAccessRules),
+            };
         });
+        agent.handoffRules = handoffRules;
+        agent.handoffs = handoffRules.map((handoff) => handoff.agent);
     }
     const entryAgent = agents.get(descriptor.runtime.entry_agent);
     if (!entryAgent) {

package/dist/index.d.ts

@@ -18,6 +18,8 @@ export * from "./providers/mistral";
 export * from "./providers/openai";
 export * from "./run";
 export * from "./run-output-events";
+export * from "./run-regression";
+export * from "./run-regression-summary";
 export * from "./run-record";
 export * from "./run-record-utils";
 export * from "./run-context";

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,SAAS,CAAC;AACxB,cAAc,oBAAoB,CAAC;AACnC,cAAc,UAAU,CAAC;AACzB,cAAc,UAAU,CAAC;AACzB,cAAc,cAAc,CAAC;AAC7B,cAAc,sBAAsB,CAAC;AACrC,cAAc,6BAA6B,CAAC;AAC5C,cAAc,QAAQ,CAAC;AACvB,cAAc,UAAU,CAAC;AACzB,cAAc,YAAY,CAAC;AAC3B,cAAc,UAAU,CAAC;AACzB,cAAc,sBAAsB,CAAC;AACrC,cAAc,oBAAoB,CAAC;AACnC,cAAc,kBAAkB,CAAC;AACjC,cAAc,kBAAkB,CAAC;AACjC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,qBAAqB,CAAC;AACpC,cAAc,oBAAoB,CAAC;AACnC,cAAc,OAAO,CAAC;AACtB,cAAc,qBAAqB,CAAC;AACpC,cAAc,cAAc,CAAC;AAC7B,cAAc,oBAAoB,CAAC;AACnC,cAAc,eAAe,CAAC;AAC9B,cAAc,kBAAkB,CAAC;AACjC,cAAc,QAAQ,CAAC;AACvB,cAAc,SAAS,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,cAAc,SAAS,CAAC;AACxB,cAAc,oBAAoB,CAAC;AACnC,cAAc,UAAU,CAAC;AACzB,cAAc,UAAU,CAAC;AACzB,cAAc,cAAc,CAAC;AAC7B,cAAc,sBAAsB,CAAC;AACrC,cAAc,6BAA6B,CAAC;AAC5C,cAAc,QAAQ,CAAC;AACvB,cAAc,UAAU,CAAC;AACzB,cAAc,YAAY,CAAC;AAC3B,cAAc,UAAU,CAAC;AACzB,cAAc,sBAAsB,CAAC;AACrC,cAAc,oBAAoB,CAAC;AACnC,cAAc,kBAAkB,CAAC;AACjC,cAAc,kBAAkB,CAAC;AACjC,cAAc,8BAA8B,CAAC;AAC7C,cAAc,qBAAqB,CAAC;AACpC,cAAc,oBAAoB,CAAC;AACnC,cAAc,OAAO,CAAC;AACtB,cAAc,qBAAqB,CAAC;AACpC,cAAc,kBAAkB,CAAC;AACjC,cAAc,0BAA0B,CAAC;AACzC,cAAc,cAAc,CAAC;AAC7B,cAAc,oBAAoB,CAAC;AACnC,cAAc,eAAe,CAAC;AAC9B,cAAc,kBAAkB,CAAC;AACjC,cAAc,QAAQ,CAAC;AACvB,cAAc,SAAS,CAAC"}

package/dist/index.js

@@ -34,6 +34,8 @@ __exportStar(require("./providers/mistral"), exports);
 __exportStar(require("./providers/openai"), exports);
 __exportStar(require("./run"), exports);
 __exportStar(require("./run-output-events"), exports);
+__exportStar(require("./run-regression"), exports);
+__exportStar(require("./run-regression-summary"), exports);
 __exportStar(require("./run-record"), exports);
 __exportStar(require("./run-record-utils"), exports);
 __exportStar(require("./run-context"), exports);

package/dist/run-regression-summary.d.ts

@@ -0,0 +1,34 @@
+import type { RunRegressionResult, RunRegressionStatus } from "./run-regression";
+export interface RunRegressionCaseSummary {
+    name: string;
+    status: RunRegressionStatus;
+    baselineRunId: string;
+    candidateRunId: string;
+    signals: {
+        statusChanged: boolean;
+        toolsChanged: boolean;
+        policyChanged: boolean;
+        finalOutputChanged: boolean;
+    };
+    judge?: {
+        verdict: RunRegressionStatus;
+        summary: string;
+    };
+}
+export interface RunRegressionSummary {
+    suite?: string;
+    status: RunRegressionStatus;
+    totals: {
+        cases: number;
+        passed: number;
+        warned: number;
+        failed: number;
+    };
+    cases: RunRegressionCaseSummary[];
+}
+export interface SummarizeRunRegressionResultsOptions<TContext = unknown> {
+    suite?: string;
+    classifyCase?: (result: RunRegressionResult<TContext>) => RunRegressionStatus;
+}
+export declare function summarizeRunRegressionResults<TContext = unknown>(results: Array<RunRegressionResult<TContext>>, options?: SummarizeRunRegressionResultsOptions<TContext>): RunRegressionSummary;
+//# sourceMappingURL=run-regression-summary.d.ts.map

package/dist/run-regression-summary.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"run-regression-summary.d.ts","sourceRoot":"","sources":["../src/run-regression-summary.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,mBAAmB,EACnB,mBAAmB,EACpB,MAAM,kBAAkB,CAAC;AAE1B,MAAM,WAAW,wBAAwB;IACvC,IAAI,EAAE,MAAM,CAAC;IACb,MAAM,EAAE,mBAAmB,CAAC;IAC5B,aAAa,EAAE,MAAM,CAAC;IACtB,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,EAAE;QACP,aAAa,EAAE,OAAO,CAAC;QACvB,YAAY,EAAE,OAAO,CAAC;QACtB,aAAa,EAAE,OAAO,CAAC;QACvB,kBAAkB,EAAE,OAAO,CAAC;KAC7B,CAAC;IACF,KAAK,CAAC,EAAE;QACN,OAAO,EAAE,mBAAmB,CAAC;QAC7B,OAAO,EAAE,MAAM,CAAC;KACjB,CAAC;CACH;AAED,MAAM,WAAW,oBAAoB;IACnC,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,mBAAmB,CAAC;IAC5B,MAAM,EAAE;QACN,KAAK,EAAE,MAAM,CAAC;QACd,MAAM,EAAE,MAAM,CAAC;QACf,MAAM,EAAE,MAAM,CAAC;QACf,MAAM,EAAE,MAAM,CAAC;KAChB,CAAC;IACF,KAAK,EAAE,wBAAwB,EAAE,CAAC;CACnC;AAED,MAAM,WAAW,oCAAoC,CAAC,QAAQ,GAAG,OAAO;IACtE,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,YAAY,CAAC,EAAE,CAAC,MAAM,EAAE,mBAAmB,CAAC,QAAQ,CAAC,KAAK,mBAAmB,CAAC;CAC/E;AA2DD,wBAAgB,6BAA6B,CAAC,QAAQ,GAAG,OAAO,EAC9D,OAAO,EAAE,KAAK,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC,EAC7C,OAAO,GAAE,oCAAoC,CAAC,QAAQ,CAAM,GAC3D,oBAAoB,CA2BtB"}

package/dist/run-regression-summary.js

@@ -0,0 +1,71 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.summarizeRunRegressionResults = summarizeRunRegressionResults;
+function defaultClassifyRunRegressionCase(result) {
+    if (result.candidate.status === "failed" ||
+        result.judge?.verdict === "fail") {
+        return "fail";
+    }
+    if (!result.comparison.equal || result.judge?.verdict === "warn") {
+        return "warn";
+    }
+    return "pass";
+}
+function summarizeCase(result, status) {
+    const judge = result.judge
+        ? {
+            verdict: result.judge.verdict,
+            summary: result.judge.summary,
+        }
+        : undefined;
+    return {
+        name: result.name,
+        status,
+        baselineRunId: result.baseline.runId,
+        candidateRunId: result.candidate.runId,
+        signals: {
+            statusChanged: result.baseline.status !== result.candidate.status,
+            toolsChanged: !result.comparison.summary.sameToolCallShape,
+            policyChanged: !result.comparison.summary.samePolicyDecisions,
+            finalOutputChanged: !result.comparison.summary.sameFinalResponse,
+        },
+        ...(judge ? { judge } : {}),
+    };
+}
+function toSuiteStatus(totals) {
+    if (totals.failed > 0) {
+        return "fail";
+    }
+    if (totals.warned > 0) {
+        return "warn";
+    }
+    return "pass";
+}
+function summarizeRunRegressionResults(results, options = {}) {
+    const classifyCase = options.classifyCase ?? (defaultClassifyRunRegressionCase);
+    const totals = {
+        cases: results.length,
+        passed: 0,
+        warned: 0,
+        failed: 0,
+    };
+    const cases = results.map((result) => {
+        const status = classifyCase(result);
+        if (status === "pass") {
+            totals.passed += 1;
+        }
+        else if (status === "warn") {
+            totals.warned += 1;
+        }
+        else {
+            totals.failed += 1;
+        }
+        return summarizeCase(result, status);
+    });
+    return {
+        ...(typeof options.suite === "undefined" ? {} : { suite: options.suite }),
+        status: toSuiteStatus(totals),
+        totals,
+        cases,
+    };
+}

package/dist/run-regression.d.ts

@@ -0,0 +1,71 @@
+import type { RunRecord } from "./run-record";
+import { type RunRegressionSummary, type SummarizeRunRegressionResultsOptions } from "./run-regression-summary";
+import { type CompareRunRecordsOptions, type ReplayFromRunRecordInput, type RunRecordComparison } from "./run-record-utils";
+export type RunRegressionStatus = "pass" | "warn" | "fail";
+export interface RunRegressionExpectation {
+    intent?: string;
+    shouldUseTools?: string[];
+    shouldAvoidTools?: string[];
+    shouldPreserve?: string[];
+    shouldImprove?: string[];
+    notes?: string;
+}
+export interface RunJudgeFinding {
+    severity: "info" | "warn" | "error";
+    reason: string;
+    evidence?: string;
+}
+export interface RunJudgeResult {
+    verdict: RunRegressionStatus;
+    summary: string;
+    findings: RunJudgeFinding[];
+    score?: number;
+    judgeModel?: string;
+    judgePromptVersion?: string;
+}
+export interface RunJudgeInput<TContext = unknown, TDescriptor = unknown> {
+    baseline: RunRecord<TContext>;
+    candidate: RunRecord<TContext>;
+    comparison: RunRecordComparison;
+    expectation?: RunRegressionExpectation;
+    baselineDescriptor?: TDescriptor;
+    candidateDescriptor?: TDescriptor;
+}
+export type RunJudge<TContext = unknown, TDescriptor = unknown> = (input: RunJudgeInput<TContext, TDescriptor>) => Promise<RunJudgeResult> | RunJudgeResult;
+export interface RunRegressionResult<TContext = unknown> {
+    name: string;
+    baseline: RunRecord<TContext>;
+    candidate: RunRecord<TContext>;
+    comparison: RunRecordComparison;
+    judge?: RunJudgeResult;
+}
+export interface RunRegressionSuiteCase<TContext = unknown> {
+    name?: string;
+    baseline: RunRecord<TContext>;
+}
+export interface RunRegressionSuite<TContext = unknown> {
+    name?: string;
+    expectation?: RunRegressionExpectation;
+    cases: Array<RunRegressionSuiteCase<TContext>>;
+}
+export interface RunRegressionCaseInput<TContext = unknown> extends Omit<ReplayFromRunRecordInput<TContext>, "sourceRunRecord"> {
+    name?: string;
+    baseline: RunRecord<TContext>;
+    comparisonOptions?: CompareRunRecordsOptions;
+}
+export interface RunRegressionSuiteInput<TContext = unknown, TDescriptor = unknown> extends Omit<RunRegressionCaseInput<TContext>, "baseline" | "name"> {
+    suite: RunRegressionSuite<TContext>;
+    judge?: RunJudge<TContext, TDescriptor>;
+    baselineDescriptor?: TDescriptor;
+    candidateDescriptor?: TDescriptor;
+    summaryOptions?: Omit<SummarizeRunRegressionResultsOptions<TContext>, "suite">;
+}
+export interface RunRegressionSuiteResult<TContext = unknown> {
+    name?: string;
+    expectation?: RunRegressionExpectation;
+    results: Array<RunRegressionResult<TContext>>;
+    summary: RunRegressionSummary;
+}
+export declare function runRegressionCase<TContext = unknown>(input: RunRegressionCaseInput<TContext>): Promise<RunRegressionResult<TContext>>;
+export declare function runRegressionSuite<TContext = unknown, TDescriptor = unknown>(input: RunRegressionSuiteInput<TContext, TDescriptor>): Promise<RunRegressionSuiteResult<TContext>>;
+//# sourceMappingURL=run-regression.d.ts.map

package/dist/run-regression.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"run-regression.d.ts","sourceRoot":"","sources":["../src/run-regression.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,cAAc,CAAC;AAC9C,OAAO,EAEL,KAAK,oBAAoB,EACzB,KAAK,oCAAoC,EAC1C,MAAM,0BAA0B,CAAC;AAClC,OAAO,EAGL,KAAK,wBAAwB,EAC7B,KAAK,wBAAwB,EAC7B,KAAK,mBAAmB,EACzB,MAAM,oBAAoB,CAAC;AAE5B,MAAM,MAAM,mBAAmB,GAAG,MAAM,GAAG,MAAM,GAAG,MAAM,CAAC;AAE3D,MAAM,WAAW,wBAAwB;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,cAAc,CAAC,EAAE,MAAM,EAAE,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,EAAE,CAAC;IAC5B,cAAc,CAAC,EAAE,MAAM,EAAE,CAAC;IAC1B,aAAa,CAAC,EAAE,MAAM,EAAE,CAAC;IACzB,KAAK,CAAC,EAAE,MAAM,CAAC;CAChB;AAED,MAAM,WAAW,eAAe;IAC9B,QAAQ,EAAE,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC;IACpC,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC;CACnB;AAED,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,mBAAmB,CAAC;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,QAAQ,EAAE,eAAe,EAAE,CAAC;IAC5B,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,kBAAkB,CAAC,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,aAAa,CAAC,QAAQ,GAAG,OAAO,EAAE,WAAW,GAAG,OAAO;IACtE,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC9B,SAAS,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC/B,UAAU,EAAE,mBAAmB,CAAC;IAChC,WAAW,CAAC,EAAE,wBAAwB,CAAC;IACvC,kBAAkB,CAAC,EAAE,WAAW,CAAC;IACjC,mBAAmB,CAAC,EAAE,WAAW,CAAC;CACnC;AAED,MAAM,MAAM,QAAQ,CAAC,QAAQ,GAAG,OAAO,EAAE,WAAW,GAAG,OAAO,IAAI,CAChE,KAAK,EAAE,aAAa,CAAC,QAAQ,EAAE,WAAW,CAAC,KACxC,OAAO,CAAC,cAAc,CAAC,GAAG,cAAc,CAAC;AAE9C,MAAM,WAAW,mBAAmB,CAAC,QAAQ,GAAG,OAAO;IACrD,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC9B,SAAS,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC/B,UAAU,EAAE,mBAAmB,CAAC;IAChC,KAAK,CAAC,EAAE,cAAc,CAAC;CACxB;AAED,MAAM,WAAW,sBAAsB,CAAC,QAAQ,GAAG,OAAO;IACxD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;CAC/B;AAED,MAAM,WAAW,kBAAkB,CAAC,QAAQ,GAAG,OAAO;IACpD,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,wBAAwB,CAAC;IACvC,KAAK,EAAE,KAAK,CAAC,sBAAsB,CAAC,QAAQ,CAAC,CAAC,CAAC;CAChD;AAED,MAAM,WAAW,sBAAsB,CAAC,QAAQ,GAAG,OAAO,CAAE,SAAQ,IAAI,CACtE,wBAAwB,CAAC,QAAQ,CAAC,EAClC,iBAAiB,CAClB;IACC,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;IAC9B,iBAAiB,CAAC,EAAE,wBAAwB,CAAC;CAC9C;AAED,MAAM,WAAW,uBAAuB,CACtC,QAAQ,GAAG,OAAO,EAClB,WAAW,GAAG,OAAO,CACrB,SAAQ,IAAI,CAAC,sBAAsB,CAAC,QAAQ,CAAC,EAAE,UAAU,GAAG,MAAM,CAAC;IACnE,KAAK,EAAE,kBAAkB,CAAC,QAAQ,CAAC,CAAC;IACpC,KAAK,CAAC,EAAE,QAAQ,CAAC,QAAQ,EAAE,WAAW,CAAC,CAAC;IACxC,kBAAkB,CAAC,EAAE,WAAW,CAAC;IACjC,mBAAmB,CAAC,EAAE,WAAW,CAAC;IAClC,cAAc,CAAC,EAAE,IAAI,CACnB,oCAAoC,CAAC,QAAQ,CAAC,EAC9C,OAAO,CACR,CAAC;CACH;AAED,MAAM,WAAW,wBAAwB,CAAC,QAAQ,GAAG,OAAO;IAC1D,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,WAAW,CAAC,EAAE,wBAAwB,CAAC;IACvC,OAAO,EAAE,KAAK,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC,CAAC;IAC9C,OAAO,EAAE,oBAAoB,CAAC;CAC/B;AA0BD,wBAAsB,iBAAiB,CAAC,QAAQ,GAAG,OAAO,EACxD,KAAK,EAAE,sBAAsB,CAAC,QAAQ,CAAC,GACtC,OAAO,CAAC,mBAAmB,CAAC,QAAQ,CAAC,CAAC,CA6BxC;AAiBD,wBAAsB,kBAAkB,CACtC,QAAQ,GAAG,OAAO,EAClB,WAAW,GAAG,OAAO,EAErB,KAAK,EAAE,uBAAuB,CAAC,QAAQ,EAAE,WAAW,CAAC,GACpD,OAAO,CAAC,wBAAwB,CAAC,QAAQ,CAAC,CAAC,CAmD7C"}

package/dist/run-regression.js

@@ -0,0 +1,91 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.runRegressionCase = runRegressionCase;
+exports.runRegressionSuite = runRegressionSuite;
+const run_regression_summary_1 = require("./run-regression-summary");
+const run_record_utils_1 = require("./run-record-utils");
+function toRegressionCaseName(input) {
+    return input.name ?? input.baseline.runId;
+}
+function toReplayRunOptions(input) {
+    return {
+        ...(input.runOptions ?? {}),
+        record: input.runOptions?.record ?? {},
+    };
+}
+function toReplayAgentFields(input) {
+    return {
+        ...(input.agent ? { agent: input.agent } : {}),
+        ...(input.agentFactory ? { agentFactory: input.agentFactory } : {}),
+    };
+}
+async function runRegressionCase(input) {
+    const replay = await (0, run_record_utils_1.replayFromRunRecord)({
+        sourceRunRecord: input.baseline,
+        ...toReplayAgentFields(input),
+        mode: input.mode,
+        runOptions: toReplayRunOptions(input),
+        inputMode: input.inputMode,
+        metadataOverrides: input.metadataOverrides,
+        onMissingToolCall: input.onMissingToolCall,
+    });
+    if (!replay.replayRunRecord) {
+        throw new Error("runRegressionCase expected replayFromRunRecord to produce a candidate RunRecord.");
+    }
+    const comparison = (0, run_record_utils_1.compareRunRecords)(input.baseline, replay.replayRunRecord, input.comparisonOptions);
+    return {
+        name: toRegressionCaseName(input),
+        baseline: input.baseline,
+        candidate: replay.replayRunRecord,
+        comparison,
+    };
+}
+function resolveSuiteCase(caseDefinition) {
+    const baseline = caseDefinition.baseline;
+    const name = caseDefinition.name ?? baseline.runId;
+    return {
+        name,
+        baseline,
+    };
+}
+async function runRegressionSuite(input) {
+    const results = [];
+    const { suite, judge, baselineDescriptor, candidateDescriptor, summaryOptions, ...caseOptions } = input;
+    for (const caseDefinition of suite.cases) {
+        const regressionCase = resolveSuiteCase(caseDefinition);
+        const result = await runRegressionCase({
+            ...caseOptions,
+            name: regressionCase.name,
+            baseline: regressionCase.baseline,
+        });
+        const judgeResult = judge
+            ? await judge({
+                baseline: result.baseline,
+                candidate: result.candidate,
+                comparison: result.comparison,
+                ...(typeof suite.expectation === "undefined"
+                    ? {}
+                    : { expectation: suite.expectation }),
+                ...(typeof baselineDescriptor === "undefined"
+                    ? {}
+                    : { baselineDescriptor }),
+                ...(typeof candidateDescriptor === "undefined"
+                    ? {}
+                    : { candidateDescriptor }),
+            })
+            : undefined;
+        results.push(judgeResult ? { ...result, judge: judgeResult } : result);
+    }
+    const summary = (0, run_regression_summary_1.summarizeRunRegressionResults)(results, {
+        suite: suite.name,
+        ...(summaryOptions ?? {}),
+    });
+    return {
+        ...(typeof suite.name === "undefined" ? {} : { name: suite.name }),
+        ...(typeof suite.expectation === "undefined"
+            ? {}
+            : { expectation: suite.expectation }),
+        results,
+        summary,
+    };
+}

package/dist/run.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run.d.ts","sourceRoot":"","sources":["../src/run.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAE,MAAM,SAAS,CAAC;AAsChC,OAAO,EACL,cAAc,EAEd,mBAAmB,EAGnB,SAAS,EACT,cAAc,EACd,gBAAgB,EACjB,MAAM,SAAS,CAAC;AAUjB,KAAK,eAAe,CAAC,QAAQ,IAAI;IAC/B,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,SAAS,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;CAC5B,CAAC;AAGF,YAAY,EACV,kBAAkB,EAClB,wBAAwB,GACzB,MAAM,mBAAmB,CAAC;AAuwB3B,qBAAa,iBAAiB,CAAC,QAAQ,GAAG,OAAO;IAC/C,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,MAAM,CAA0C;IACxD,OAAO,CAAC,KAAK,CAA4B;gBAGvC,MAAM,EAAE,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC,EAC/C,KAAK,EAAE,eAAe,CAAC,QAAQ,CAAC;IAMlC,QAAQ,IAAI,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC;IAQnD,IAAI,OAAO,IAAI,cAAc,EAAE,CAE9B;IAED,IAAI,SAAS,IAAI,KAAK,CAAC,QAAQ,CAAC,CAE/B;CACF;AAED,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,EAAE,gBAAgB,CAAC,QAAQ,CAAC,GAClC,OAAO,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;AAExC,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,CAAC,EAAE,mBAAmB,CAAC,QAAQ,CAAC,GACtC,OAAO,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC"}
+{"version":3,"file":"run.d.ts","sourceRoot":"","sources":["../src/run.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,EAAqB,MAAM,SAAS,CAAC;AAsCnD,OAAO,EACL,cAAc,EAEd,mBAAmB,EAGnB,SAAS,EACT,cAAc,EACd,gBAAgB,EACjB,MAAM,SAAS,CAAC;AAUjB,KAAK,eAAe,CAAC,QAAQ,IAAI;IAC/B,OAAO,EAAE,cAAc,EAAE,CAAC;IAC1B,SAAS,EAAE,KAAK,CAAC,QAAQ,CAAC,CAAC;CAC5B,CAAC;AAGF,YAAY,EACV,kBAAkB,EAClB,wBAAwB,GACzB,MAAM,mBAAmB,CAAC;AAmxB3B,qBAAa,iBAAiB,CAAC,QAAQ,GAAG,OAAO;IAC/C,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,MAAM,CAA0C;IACxD,OAAO,CAAC,KAAK,CAA4B;gBAGvC,MAAM,EAAE,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC,EAC/C,KAAK,EAAE,eAAe,CAAC,QAAQ,CAAC;IAMlC,QAAQ,IAAI,aAAa,CAAC,cAAc,CAAC,QAAQ,CAAC,CAAC;IAQnD,IAAI,OAAO,IAAI,cAAc,EAAE,CAE9B;IAED,IAAI,SAAS,IAAI,KAAK,CAAC,QAAQ,CAAC,CAE/B;CACF;AAED,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,EAAE,gBAAgB,CAAC,QAAQ,CAAC,GAClC,OAAO,CAAC,iBAAiB,CAAC,QAAQ,CAAC,CAAC,CAAC;AAExC,wBAAsB,GAAG,CAAC,QAAQ,GAAG,OAAO,EAC1C,aAAa,EAAE,KAAK,CAAC,QAAQ,CAAC,EAC9B,KAAK,EAAE,MAAM,GAAG,cAAc,EAAE,EAChC,OAAO,CAAC,EAAE,mBAAmB,CAAC,QAAQ,CAAC,GACtC,OAAO,CAAC,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC"}

package/dist/run.js

@@ -62,11 +62,16 @@ function sanitizeToolSegment(input) {
         .replace(/^_+|_+$/g, "");
     return sanitized || "agent";
 }
-function buildTurnTools(agent) {
+function buildTurnTools(agent, runContext) {
     const handoffRegistry = new Map();
     const handoffTools = [];
     const reservedNames = new Set(agent.tools.map((tool) => tool.name));
-    for (const handoffAgent of agent.handoffs) {
+    const handoffRules = agent.handoffRules;
+    for (const handoffRule of handoffRules) {
+        if (handoffRule.enabled && !handoffRule.enabled(runContext)) {
+            continue;
+        }
+        const handoffAgent = handoffRule.agent;
         const baseName = `handoff_to_${sanitizeToolSegment(handoffAgent.name)}`;
         let toolName = baseName;
         let suffix = 2;
@@ -331,7 +336,7 @@ async function* runLoop(state, runId, provider, runContext, maxTurns, logger, po
             activeTurn = turn + 1;
             const currentAgent = state.lastAgent;
             await logEmitter.turnStarted(currentAgent.name, activeTurn);
-            const { providerTools, handoffRegistry } = buildTurnTools(currentAgent);
+            const { providerTools, handoffRegistry } = buildTurnTools(currentAgent, runContext);
             const model = resolveAgentModel(currentAgent);
             const systemPrompt = await currentAgent.resolveInstructions(runContext);
             onPromptSnapshot?.({

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@axiastudio/aioc",
-  "version": "0.2.5",
+  "version": "0.2.7",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
   "files": [
@@ -14,39 +14,45 @@
     "clean": "node -e \"require('fs').rmSync('dist',{recursive:true,force:true})\"",
     "build": "tsc -p tsconfig.json",
     "build:package": "npm run clean && tsc -p tsconfig.build.json",
+    "examples:check": "tsc -p tsconfig.examples.json --noEmit",
     "lint": "eslint ./src --ext .ts",
     "test:unit": "npm run build && node dist/tests/unit/index.js",
     "test:integration": "npm run build && node dist/tests/integration/index.js",
     "test:regression": "npm run build && node dist/tests/regression/index.js",
     "test:ci": "npm run lint && npm run build && node dist/tests/unit/index.js && node dist/tests/integration/index.js && node dist/tests/regression/index.js",
     "test:ci:stability": "npm run test:ci && npm run test:ci",
-    "example:hello": "tsx src/examples/basic/hello-world.ts",
-    "example:policy": "tsx src/examples/basic/policy.ts",
-    "example:approval-required": "tsx src/examples/basic/approval-required.ts",
-    "example:approval-evidence": "tsx src/examples/basic/approval-evidence.ts",
-    "example:tool-policy": "tsx src/examples/basic/tools.ts",
-    "example:policy-composition": "tsx src/examples/basic/policy-composition.ts",
-    "example:harness": "tsx src/examples/harness-descriptor/customer-support.ts",
-    "example:run-record": "tsx src/examples/basic/run-record-sink.ts",
-    "example:rru:01-extract": "tsx src/examples/run-record-utils-minimal/01-extract-tool-calls.ts",
-    "example:rru:02-compare": "tsx src/examples/run-record-utils-minimal/02-compare-run-records.ts",
-    "example:rru:03-replay-strict": "tsx src/examples/run-record-utils-minimal/03-replay-strict.ts",
-    "example:rru:04-replay-hybrid": "tsx src/examples/run-record-utils-minimal/04-replay-hybrid.ts",
-    "example:non-regression": "tsx src/examples/non-regression/v1-v2-runrecord-diff.ts",
+    "example:hello": "tsx examples/core/basic/hello-world.ts",
+    "example:policy": "tsx examples/core/basic/policy.ts",
+    "example:approval-required": "tsx examples/core/basic/approval-required.ts",
+    "example:approval-evidence": "tsx examples/core/basic/approval-evidence.ts",
+    "example:tool-policy": "tsx examples/core/basic/tools.ts",
+    "example:policy-composition": "tsx examples/core/basic/policy-composition.ts",
+    "example:harness": "tsx examples/core/harness-descriptor/customer-support.ts",
+    "example:harness-rerun": "tsx examples/core/harness-descriptor/rerun-modified-harness.ts",
+    "example:run-record": "tsx examples/core/basic/run-record-sink.ts",
+    "example:rru:01-extract": "tsx examples/core/run-record-utils-minimal/01-extract-tool-calls.ts",
+    "example:rru:02-compare": "tsx examples/core/run-record-utils-minimal/02-compare-run-records.ts",
+    "example:rru:03-replay-strict": "tsx examples/core/run-record-utils-minimal/03-replay-strict.ts",
+    "example:rru:04-replay-hybrid": "tsx examples/core/run-record-utils-minimal/04-replay-hybrid.ts",
+    "example:non-regression": "tsx examples/core/non-regression/v1-v2-runrecord-diff.ts",
+    "example:run-regression": "tsx examples/core/run-regression/age-adapted-suite.ts",
+    "example:run-regression-judge": "npm run regression-judge:build && tsx packages/aioc-regression-judge/examples/age-adapted-suite-with-judge.ts",
     "inspect:samples": "tsx scripts/generate-aioc-inspect-samples.ts",
     "docs:sync": "node scripts/sync-aioc-docs.mjs",
     "docs:dev": "npm run docs:sync && cd apps/aioc-docs && zsh -lc 'npm run dev'",
     "docs:build": "npm run docs:sync && cd apps/aioc-docs && zsh -lc 'npm run build'",
     "docs:preview": "cd apps/aioc-docs && zsh -lc 'npm run preview'",
-    "packages:build": "npm run governance-events:build && npm run build -w @axiastudio/aioc-export-otel && npm run build -w @axiastudio/aioc-inspect-ui",
-    "packages:test": "npm run governance-events:test && npm run export-otel:test",
+    "packages:build": "npm run governance-events:build && npm run regression-judge:build && npm run build -w @axiastudio/aioc-export-otel && npm run build -w @axiastudio/aioc-inspect-ui",
+    "packages:test": "npm run governance-events:test && npm run export-otel:test && npm run regression-judge:test",
     "governance-events:build": "npm run build && npm run build -w @axiastudio/aioc-governance-events",
     "governance-events:test": "npm run build && npm test -w @axiastudio/aioc-governance-events",
+    "regression-judge:build": "npm run build && npm run build -w @axiastudio/aioc-regression-judge",
+    "regression-judge:test": "npm run build && npm test -w @axiastudio/aioc-regression-judge",
     "export-otel:build": "npm run governance-events:build && npm run build -w @axiastudio/aioc-export-otel",
     "export-otel:test": "npm run governance-events:build && npm test -w @axiastudio/aioc-export-otel",
     "export-otel:console": "npm run governance-events:build && npm run console -w @axiastudio/aioc-export-otel",
     "export-otel:signoz": "npm run governance-events:build && npm run signoz -w @axiastudio/aioc-export-otel",
-    "test:mistral": "tsx src/examples/mistral-smoke.ts",
+    "test:mistral": "tsx examples/core/mistral-smoke.ts",
     "prepack": "npm run build:package"
   },
   "keywords": [