@tangle-network/agent-eval 0.28.0 → 0.29.0

package/CHANGELOG.md

@@ -1,5 +1,92 @@
 # Changelog
 
+## 0.29.0 — 2026-05-19
+
+### Analyst kinds + cross-run findings context
+
+Builds on 0.28.0's analyst registry. Ships four trace-analyst **kinds**
+that emit graded findings through native Ax structured output (no more
+flat-defaulted bullet lists) and a cross-run findings context the
+registry can inject into prompts so each kind sees what the prior run
+already surfaced.
+
+### Added
+
+- **`createTraceAnalystKind(spec, opts)`** (`src/analyst/kind-factory.ts`) —
+  turns a `TraceAnalystKindSpec` into a registry-ready
+  `Analyst<TraceAnalysisStore>`. Ax signature is
+  `'question:string -> findings:json[]'`; the Zod boundary in
+  `finding-signature.ts` rejects malformed rows instead of lifting them
+  with default severity. Supports `versionSuffix` for optimizer-fitted
+  prompts (MIPRO / GEPA / Bootstrap) and a per-row `postProcess` hook.
+- **`RawAnalystFinding`** Zod schema + **`RAW_FINDING_SCHEMA_PROMPT`**
+  string embedded into kind actor prompts so the model and the parser
+  share one source of truth.
+- **`TraceToolGroupName`** + **`buildTraceToolsForGroup`**
+  (`src/analyst/tool-groups.ts`) — five named tool subsets
+  (`all | discovery | discoveryAndRead | discoveryAndSearch | targeted`);
+  unknown group names throw.
+- **Four shipping kinds** (`src/analyst/kinds/`):
+  - `FAILURE_MODE_KIND_SPEC` — clusters dataset failures into distinct
+    modes (maxDepth 3, parallel 4, all tools).
+  - `KNOWLEDGE_GAP_KIND_SPEC` — attributes missing/stale knowledge to
+    `agent-knowledge:wiki:*`, `websearch:outdated:*`, `tool-doc:*`,
+    `system-prompt:*`, `memory:*` (maxDepth 2, discoveryAndSearch).
+  - `KNOWLEDGE_POISONING_KIND_SPEC` — dual-verify analyst for
+    confident-but-wrong actions (maxDepth 2, all tools).
+  - `IMPROVEMENT_KIND_SPEC` — converts upstream failure / gap /
+    poisoning findings into concrete locus-named edits with leverage
+    grades (maxDepth 3, all tools).
+- **`DEFAULT_TRACE_ANALYST_KINDS`** — the four specs in canonical run
+  order (failure-mode → gap → poisoning → improvement).
+- **`priorFindings` on `AnalystContext`** — registry injects findings
+  from a prior `AnalystRunResult` into every analyst's context, so an
+  improvement-kind run can see the failure-mode findings the previous
+  pass surfaced. Kinds reference prior findings via
+  `evidence_uri: "finding://<id>"`.
+
+### Deprecated
+
+- `createTraceAnalystAdapter` (`src/analyst/adapters.ts`) — the legacy
+  bullet-list lifter. Kept for one minor while consumers migrate to
+  `createTraceAnalystKind`.
+
+## 0.28.0 — 2026-05-19
+
+### Analyst registry + findings envelope
+
+A generic, model-agnostic orchestration layer over the existing
+analyzers (`analyzeTraces`, `MultiLayerVerifier`, `RunCritic`,
+`SemanticConceptJudge`, `JudgeFn`). One contract, one runner, one
+persistence path. Reusable by VB operator bench, leaderboard submission
+pipeline, and orchestrator on-completion reports with the same code.
+
+### Added
+
+- **`Analyst<TInput>`** contract + **`AnalystFinding`** envelope with
+  sha-stable `finding_id` (`src/analyst/types.ts`).
+- **`AnalystRegistry`** (`src/analyst/registry.ts`) — register/list/run
+  with input routing by `inputKind`, per-analyst isolation, equal-split
+  budget by default, per-analyst telemetry.
+- **`AnalystHooks`** — `onBeforeAnalyze | onAfterAnalyze | onError |
+  onComplete`. Generic seam for telemetry, cost ingestion, rotation,
+  error → finding conversion.
+- **`BudgetPolicy`** — `{ totalUsd, weights, allocate }`. Default
+  equal-split; weighted split or custom `allocate(args)` for precision.
+- **`ChatClient`** abstraction (`src/analyst/chat-client.ts`) over
+  `router | sandbox-sdk | cli-bridge | direct-provider | mock` so
+  analyst code is transport-agnostic; `wrapLlmClient` races the call
+  against `ChatCallOpts.signal`.
+- **`FindingsStore`** + **`diffFindings(prev, cur, { isMaterial })`**
+  (`src/analyst/findings-store.ts`) — locked JSONL persistence + cross-run
+  diff (appeared / disappeared / persisted / changed) with a pluggable
+  materiality predicate (`defaultIsMaterial` exported for layering).
+- Five **adapter** factories (`src/analyst/adapters.ts`) that lift
+  existing primitives into the contract without re-implementing them:
+  `createTraceAnalystAdapter`, `createVerifierAdapter`,
+  `createRunCriticAdapter`, `createJudgeAdapter`,
+  `createSemanticConceptJudgeAdapter`.
+
 ## 0.27.2 — 2026-05-17
 
 ### Corpus-wide inter-rater agreement primitive

package/dist/index.d.ts

@@ -690,6 +690,16 @@ interface AnalystContext {
      * analyst code.
      */
     chat?: ChatClient;
+    /**
+     * Findings from a prior run the operator wants the analyst to see as
+     * retrieval context. Kinds that take advantage of cross-run memory
+     * (failure-mode "I saw this cluster last run", knowledge-gap "the wiki
+     * page I asked for is still missing") render these into the actor's
+     * working set. Filtering is the operator's job: pass the slice that
+     * matches the analyst's id, or pass everything and let the kind
+     * filter. Empty / absent means no cross-run context.
+     */
+    priorFindings?: ReadonlyArray<AnalystFinding>;
     /** Free-form runtime tags (env, host, op). Findings can echo these into metadata. */
     tags?: Record<string, string>;
     /** Logger callback — analysts SHOULD prefer this over console.* for testability. */
@@ -842,6 +852,54 @@ interface SemanticConceptJudgeAdapterOpts {
 }
 declare function createSemanticConceptJudgeAdapter(opts?: SemanticConceptJudgeAdapterOpts): Analyst<SemanticConceptJudgeInput>;
 
+/**
+ * Typed Ax output for analyst findings.
+ *
+ * Replaces the legacy `findings:string[]` pattern (where every bullet
+ * became a flat-severity `AnalystFinding`) with a structured object
+ * array. Ax binds the field as `findings:json[]` so the provider emits
+ * native structured output; at the kind-factory boundary we Zod-validate
+ * each emitted finding so malformed rows fail loud instead of being
+ * silently lifted with default severity.
+ *
+ * Why not `f.object().array()` directly in the signature? The Ax
+ * signature string `question:string -> findings:json[]` already lets
+ * the provider emit JSON arrays. A Zod boundary is required either
+ * way (the provider can return any JSON), and Zod gives us a single
+ * validation surface independent of which Ax version is installed.
+ */
+
+declare const ANALYST_SEVERITIES: readonly ["critical", "high", "medium", "low", "info"];
+declare const RawAnalystFindingSchema: z.ZodObject<{
+    severity: z.ZodEnum<{
+        info: "info";
+        critical: "critical";
+        medium: "medium";
+        low: "low";
+        high: "high";
+    }>;
+    claim: z.ZodString;
+    subject: z.ZodOptional<z.ZodString>;
+    evidence_uri: z.ZodString;
+    evidence_excerpt: z.ZodOptional<z.ZodString>;
+    confidence: z.ZodNumber;
+    rationale: z.ZodOptional<z.ZodString>;
+    recommended_action: z.ZodOptional<z.ZodString>;
+}, z.core.$strict>;
+type RawAnalystFinding = z.infer<typeof RawAnalystFindingSchema>;
+/**
+ * Description embedded into the actor prompt so the LLM knows what
+ * shape to emit. Kept here so kinds share one source of truth rather
+ * than restating the schema in every prompt.
+ */
+declare const RAW_FINDING_SCHEMA_PROMPT = "Each finding MUST be a JSON object with these fields:\n  - severity: one of \"critical\" | \"high\" | \"medium\" | \"low\" | \"info\"\n  - claim: one-sentence statement (max 2000 chars)\n  - subject?: the leaf id, agent id, span id, tool name, or noun phrase the finding is about\n  - evidence_uri: \"span://<trace_id>/<span_id>\" for trace evidence, \"artifact://<relative-path>\" for files, \"metric://<name>\" for named scalars \u2014 ALWAYS cite a real id surfaced by the tools\n  - evidence_excerpt?: short quote (<=2000 chars) from the cited span/artifact\n  - confidence: number 0..1 \u2014 0.9+ when backed by exact quotes, 0.6-0.8 for inferred patterns, <0.5 for speculative\n  - rationale?: one or two sentences explaining the reasoning\n  - recommended_action?: concrete change phrased as an imperative (\"Add ...\", \"Replace ...\", \"Stop ...\") \u2014 omit when the finding is purely descriptive\n\nEmit an empty array when the question has no findings to report. Do not fabricate evidence.";
+/**
+ * Validate one row emitted by the LLM. Returns the typed finding on
+ * success; returns `null` and logs the reason on failure so the kind
+ * factory can skip-and-count rather than abort the whole analyst run.
+ */
+declare function parseRawFinding(row: unknown, log?: (msg: string, fields?: Record<string, unknown>) => void): RawAnalystFinding | null;
+
 /**
  * FindingsStore — durable persistence for AnalystFinding rows + a diff
  * helper so we can answer "what changed since the last run?" without
@@ -917,54 +975,6 @@ declare function defaultIsMaterial(a: AnalystFinding, b: AnalystFinding): boolea
  */
 declare function diffFindings(previous: PersistedFinding[], current: PersistedFinding[], policy?: DiffPolicy): FindingsDiff;
 
-/**
- * Typed Ax output for analyst findings.
- *
- * Replaces the legacy `findings:string[]` pattern (where every bullet
- * became a flat-severity `AnalystFinding`) with a structured object
- * array. Ax binds the field as `findings:json[]` so the provider emits
- * native structured output; at the kind-factory boundary we Zod-validate
- * each emitted finding so malformed rows fail loud instead of being
- * silently lifted with default severity.
- *
- * Why not `f.object().array()` directly in the signature? The Ax
- * signature string `question:string -> findings:json[]` already lets
- * the provider emit JSON arrays. A Zod boundary is required either
- * way (the provider can return any JSON), and Zod gives us a single
- * validation surface independent of which Ax version is installed.
- */
-
-declare const ANALYST_SEVERITIES: readonly ["critical", "high", "medium", "low", "info"];
-declare const RawAnalystFindingSchema: z.ZodObject<{
-    severity: z.ZodEnum<{
-        info: "info";
-        critical: "critical";
-        medium: "medium";
-        low: "low";
-        high: "high";
-    }>;
-    claim: z.ZodString;
-    subject: z.ZodOptional<z.ZodString>;
-    evidence_uri: z.ZodString;
-    evidence_excerpt: z.ZodOptional<z.ZodString>;
-    confidence: z.ZodNumber;
-    rationale: z.ZodOptional<z.ZodString>;
-    recommended_action: z.ZodOptional<z.ZodString>;
-}, z.core.$strict>;
-type RawAnalystFinding = z.infer<typeof RawAnalystFindingSchema>;
-/**
- * Description embedded into the actor prompt so the LLM knows what
- * shape to emit. Kept here so kinds share one source of truth rather
- * than restating the schema in every prompt.
- */
-declare const RAW_FINDING_SCHEMA_PROMPT = "Each finding MUST be a JSON object with these fields:\n  - severity: one of \"critical\" | \"high\" | \"medium\" | \"low\" | \"info\"\n  - claim: one-sentence statement (max 2000 chars)\n  - subject?: the leaf id, agent id, span id, tool name, or noun phrase the finding is about\n  - evidence_uri: \"span://<trace_id>/<span_id>\" for trace evidence, \"artifact://<relative-path>\" for files, \"metric://<name>\" for named scalars \u2014 ALWAYS cite a real id surfaced by the tools\n  - evidence_excerpt?: short quote (<=2000 chars) from the cited span/artifact\n  - confidence: number 0..1 \u2014 0.9+ when backed by exact quotes, 0.6-0.8 for inferred patterns, <0.5 for speculative\n  - rationale?: one or two sentences explaining the reasoning\n  - recommended_action?: concrete change phrased as an imperative (\"Add ...\", \"Replace ...\", \"Stop ...\") \u2014 omit when the finding is purely descriptive\n\nEmit an empty array when the question has no findings to report. Do not fabricate evidence.";
-/**
- * Validate one row emitted by the LLM. Returns the typed finding on
- * success; returns `null` and logs the reason on failure so the kind
- * factory can skip-and-count rather than abort the whole analyst run.
- */
-declare function parseRawFinding(row: unknown, log?: (msg: string, fields?: Record<string, unknown>) => void): RawAnalystFinding | null;
-
 /**
  * Analyst-kind factory — the typed, focused replacement for the
  * legacy `createTraceAnalystAdapter`.
@@ -1054,6 +1064,21 @@ interface CreateTraceAnalystKindOpts {
  * want shared across analyst runs).
  */
 declare function createTraceAnalystKind(spec: TraceAnalystKindSpec, opts: CreateTraceAnalystKindOpts): Analyst<TraceAnalysisStore>;
+/**
+ * Render a compact prior-findings block the actor reads alongside its
+ * brief. Each row is one line so the actor can scan dozens cheaply.
+ * The kind's prompt instructs the actor to (a) check whether a new
+ * cluster matches a prior `finding_id` (carry the id forward via
+ * `id_basis` to keep diffs stable) and (b) raise severity / confidence
+ * when a prior finding has reappeared without remediation.
+ *
+ * Returns the empty string when there are no prior findings — most
+ * runs are "first-of-its-kind" and the prompt stays unchanged.
+ *
+ * Exported for tests + for consumers that build their own actor
+ * prompts (e.g. specialized analysts living outside the default kinds).
+ */
+declare function renderPriorFindings(prior: AnalystContext['priorFindings']): string;
 
 /**
  * Failure-mode analyst — classifies what went wrong and why.
@@ -1074,6 +1099,29 @@ declare function createTraceAnalystKind(spec: TraceAnalystKindSpec, opts: Create
 
 declare const FAILURE_MODE_KIND_SPEC: TraceAnalystKindSpec;
 
+/**
+ * Improvement analyst — actionable, recursive self-improvement findings.
+ *
+ * Brief: read findings from upstream analysts (failure-mode,
+ * knowledge-gap, knowledge-poisoning) AND the trace dataset itself,
+ * then propose **concrete edits** to the agent's runtime: prompt
+ * additions, RAG documents to ingest, tool descriptions to rewrite,
+ * scaffolding changes to make, memory entries to invalidate. Each
+ * finding is one proposed edit with the locus, the diff, and the
+ * expected effect.
+ *
+ * This is the recursive-self-improvement loop's last mile: the prior
+ * kinds describe *what's wrong*; this kind describes *what to change*.
+ *
+ * Recursion is deep (`maxDepth: 3`) because real improvement proposals
+ * are competitive: for each failure-mode there are usually 2-3 viable
+ * fix directions (tighten prompt vs add tool vs adjust scaffolding),
+ * and the actor should explore each with a focused subagent before
+ * picking the highest-leverage one to recommend.
+ */
+
+declare const IMPROVEMENT_KIND_SPEC: TraceAnalystKindSpec;
+
 /**
  * Knowledge-gap analyst — what did the agent NOT know that it needed?
  *
@@ -1124,29 +1172,6 @@ declare const KNOWLEDGE_GAP_KIND_SPEC: TraceAnalystKindSpec;
 
 declare const KNOWLEDGE_POISONING_KIND_SPEC: TraceAnalystKindSpec;
 
-/**
- * Improvement analyst — actionable, recursive self-improvement findings.
- *
- * Brief: read findings from upstream analysts (failure-mode,
- * knowledge-gap, knowledge-poisoning) AND the trace dataset itself,
- * then propose **concrete edits** to the agent's runtime: prompt
- * additions, RAG documents to ingest, tool descriptions to rewrite,
- * scaffolding changes to make, memory entries to invalidate. Each
- * finding is one proposed edit with the locus, the diff, and the
- * expected effect.
- *
- * This is the recursive-self-improvement loop's last mile: the prior
- * kinds describe *what's wrong*; this kind describes *what to change*.
- *
- * Recursion is deep (`maxDepth: 3`) because real improvement proposals
- * are competitive: for each failure-mode there are usually 2-3 viable
- * fix directions (tighten prompt vs add tool vs adjust scaffolding),
- * and the actor should explore each with a focused subagent before
- * picking the highest-leverage one to recommend.
- */
-
-declare const IMPROVEMENT_KIND_SPEC: TraceAnalystKindSpec;
-
 /**
  * Default analyst kinds focused on agent failure + recursive
  * self-improvement.
@@ -1165,39 +1190,6 @@ declare const IMPROVEMENT_KIND_SPEC: TraceAnalystKindSpec;
  */
 declare const DEFAULT_TRACE_ANALYST_KINDS: readonly TraceAnalystKindSpec[];
 
-/**
- * Pre-curated tool subsets for analyst kinds.
- *
- * The full trace-analyst tool set is seven functions. Most kinds only
- * need three or four. Picking from named groups instead of importing
- * the whole bundle keeps every kind's actor-context budget tight and
- * makes "what can this analyst see?" obvious at registration time.
- *
- * Each function in the group keeps its full `name`/`description` from
- * `buildTraceAnalystTools` — we filter, we don't re-implement.
- */
-
-/** Named tool sets. Kinds pass `tools: TRACE_TOOL_GROUPS.failureForensics` etc. */
-type TraceToolGroupName = 
-/** All seven tools. Use for open-ended discovery kinds. */
-'all'
-/** Overview + paginated query + count. No deep reads. Cheap. */
- | 'discovery'
-/** Discovery + viewTrace + viewSpans. Deep-read but no regex search. */
- | 'discoveryAndRead'
-/** Discovery + search tools. For pattern-matching across many traces. */
- | 'discoveryAndSearch'
-/** Discovery + viewSpans + searchSpan. Targeted-span work after another kind narrows down. */
- | 'targeted';
-/**
- * Build the tool set for a named group bound to a specific trace store.
- *
- * `all` returns every tool. Other groups filter `buildTraceAnalystTools`
- * by name to the documented subset. An unrecognised group name throws —
- * silently returning all tools would defeat the cost-control point.
- */
-declare function buildTraceToolsForGroup(group: TraceToolGroupName, store: TraceAnalysisStore): AxFunction[];
-
 /**
  * AnalystRegistry — orchestrate N analysts against one run.
  *
@@ -1285,6 +1277,15 @@ interface RegistryRunOpts {
     signal?: AbortSignal;
     /** Tags echoed into AnalystContext.tags — useful for tracking environment/version in findings. */
     tags?: Record<string, string>;
+    /**
+     * Prior-run findings made available as retrieval context to every
+     * analyst via `ctx.priorFindings`. The registry forwards the slice
+     * whose `analyst_id` matches each registered analyst so a kind sees
+     * only its own history. Pass `{ '*': findings }` to broadcast to
+     * every analyst (useful for cross-kind chaining where the improvement
+     * analyst consumes upstream failure findings).
+     */
+    priorFindings?: ReadonlyArray<AnalystFinding> | Record<string, ReadonlyArray<AnalystFinding>>;
 }
 declare class AnalystRegistry {
     private readonly analysts;
@@ -1302,6 +1303,39 @@ declare class AnalystRegistry {
     private routeInput;
 }
 
+/**
+ * Pre-curated tool subsets for analyst kinds.
+ *
+ * The full trace-analyst tool set is seven functions. Most kinds only
+ * need three or four. Picking from named groups instead of importing
+ * the whole bundle keeps every kind's actor-context budget tight and
+ * makes "what can this analyst see?" obvious at registration time.
+ *
+ * Each function in the group keeps its full `name`/`description` from
+ * `buildTraceAnalystTools` — we filter, we don't re-implement.
+ */
+
+/** Named tool sets. Kinds pass `tools: TRACE_TOOL_GROUPS.failureForensics` etc. */
+type TraceToolGroupName = 
+/** All seven tools. Use for open-ended discovery kinds. */
+'all'
+/** Overview + paginated query + count. No deep reads. Cheap. */
+ | 'discovery'
+/** Discovery + viewTrace + viewSpans. Deep-read but no regex search. */
+ | 'discoveryAndRead'
+/** Discovery + search tools. For pattern-matching across many traces. */
+ | 'discoveryAndSearch'
+/** Discovery + viewSpans + searchSpan. Targeted-span work after another kind narrows down. */
+ | 'targeted';
+/**
+ * Build the tool set for a named group bound to a specific trace store.
+ *
+ * `all` returns every tool. Other groups filter `buildTraceAnalystTools`
+ * by name to the documented subset. An unrecognised group name throws —
+ * silently returning all tools would defeat the cost-control point.
+ */
+declare function buildTraceToolsForGroup(group: TraceToolGroupName, store: TraceAnalysisStore): AxFunction[];
+
 /**
  * Automated pull request opener for the production loop.
  *
@@ -5945,4 +5979,4 @@ declare function aggregateTrialsByMode(trials: TrialResult[], opts: {
     mode: AggregatorMode;
 }): TrialAggregate;
 
-export { ANALYST_SEVERITIES, ActionableSideInfo, type ActiveLearningOptions, type AdapterRun, AgentDriver, type AgentDriverConfig, AgentEvalError, type AggregatorMode, type AlignmentOp, type Analyst, type AnalystContext, type AnalystCost, type AnalystFinding, type AnalystHooks, type AnalystInputKind, AnalystRegistry, type AnalystRegistryOptions, type AnalystRequirements, type AnalystRunInputs, type AnalystRunResult, type AnalystRunSummary, type AnalystSeverity, type AntiSlopConfig, type AntiSlopIssue, type AntiSlopReport, Artifact$1 as Artifact, type ArtifactCheck, type Artifact as ArtifactCheckArtifact, type ArtifactResult, type ArtifactValidator, type AutoPrClient, AxGepaSteeringOptimizer, type AxSteeringOptimizerConfig, BaselineReport, BehaviorAssertion, type BenchmarkReport, BenchmarkRunner, type BenchmarkRunnerConfig, type BisectOptions, type BisectResult, type BisectStep, BudgetBreachError, BudgetGuard, BudgetLedgerEntry, type BudgetPolicy, BudgetSpec, CallExpectation, type CanaryAlert, type CanaryKind, type CanaryLeak, type CanaryOptions, type CanaryReport, type CanarySeverity, type CandidateScenario, type CausalAttributionReport, type ChatCallOpts, type ChatClient, type ChatRequest, type ChatResponse, type ChatTransport, type CheckResult, type CliBridgeTransportOpts, type CodeMutationOutcome, type CodeMutationRunner, type CollectedArtifacts, type CommandRunner, type CompletionCriterion, type CompositePolicy, type ConceptComplexity, type ConceptFinding, type ConceptSpec, type ConceptWeightStrategy, type ContinuityCheck, type ContinuityCheckResult, type ContinuityReport, type ContinuitySnapshotPair, ContinuousAgreement, ContinuousAgreementOptions, type ContractMetric, type ContractReport, ControlEvalResult, ConvergenceTracker, type CorpusAgreementOptions, type CorpusAgreementPerDimension, type CorpusAgreementReport, type CorpusScoreRecord, type CostEntry, CostLedger, type CostLedgerGeneration, type CostLedgerSnapshot, type CostSummary, CostTracker, type CounterfactualContext, type CounterfactualMutation, type CounterfactualResult, type CounterfactualRunner, type CreateChatClientOpts, type CreateCompositeMutatorOpts, type CreateDefaultReviewerOptions, type CreateSandboxCodeMutatorOpts, type CreateSandboxPoolOpts, type CreateTraceAnalystKindOpts, type CrossTraceDiff, type CrossTraceDiffOptions, D1ExperimentStore, type D1ExperimentStoreOptions, type D1Like, type D1PreparedStatementLike, DEFAULT_AGENT_SLOS, DEFAULT_COMPLEXITY_WEIGHTS, DEFAULT_FINDERS, DEFAULT_HARNESS_OBJECTIVES, DEFAULT_MUTATORS, DEFAULT_RUN_SCORE_WEIGHTS, DEFAULT_SEVERITY_WEIGHTS, DEFAULT_TRACE_ANALYST_KINDS, Dataset, DatasetScenario, type DeployFamily, type DeployGateLayerInput, type DeployRunResult, type DeployRunner, type DiffPolicy, type DirEntry, type DirectProviderTransportOpts, type DiscoverPersonasOptions, type DiscoveredPersona, type DriverResult, type DriverState, DualAgentBench, type DualAgentBenchConfig, type DualAgentReport, type DualAgentRound, type DualAgentScenario, type DualAgentScenarioResult, ERROR_COUNT_PATTERNS, type ErrorCountPattern, type EvalResult, type EvidenceRef, type EvolutionRound, EvolvableVariant, type ExecutorConfig, type Expectation, type Experiment, type Run as ExperimentRun, type ExperimentStore, ExperimentTracker, type ExportedRewardModel, type ExtractOptions, type ExtractResult, FAILURE_MODE_KIND_SPEC, type FactorContribution, type FactorialCell, type FailureClusterConfig, FeedbackLabel, type FeedbackPattern, FeedbackTrajectory, FeedbackTrajectoryStore, type FileChange, FileSystemExperimentStore, type FileSystemExperimentStoreOptions, type FindingsDiff, FindingsStore, type FlowAction, type FlowLayerEnv, type FlowLayerFactoryInput, type FlowRunner, type FlowRunnerStepResult, type FlowSpec, type FlowStep, GateDecision, type GhCliClientOptions, type GoldenSeverity, type GoldenSpec, type HarnessAdapter, HarnessConfig, type HarnessExperimentConfig, type HarnessExperimentResult, type HarnessIntervention, type HarnessRunRequest, type HarnessRunResult, type HarnessScenario, type HarnessSelection, type HarnessVariant, type HarnessVariantReport, HeldOutGateConfig, HoldoutAuditor, type HostedJudgeConfig, type HostedJudgeDimension, type HostedJudgeRequest, type HostedJudgeResponse, type HostedRunCriticConfig, type HostedRunScoreRequest, type HostedRunScoreResponse, type HttpGithubClientOptions, type HypothesisManifest, type HypothesisResult, IMPROVEMENT_KIND_SPEC, INTENT_MATCH_JUDGE_VERSION, type ImageData, InMemoryExperimentStore, InMemoryWorkspaceInspector, type InferenceScorer, type InspectorContext, type IntegrationGateSurface, type IntegrationInvokeFailureInput, type IntegrationManifestGateInput, type IntentMatchInput, type IntentMatchOptions, type IntentMatchResult, type InteractionContribution, JsonlTrialCache, type JudgeAdapterOpts, type JudgeConfig, type JudgeFleetOptions, type JudgeFn, type JudgeInput, type JudgeReplayResult, type JudgeRetryOutcome, type JudgeRetryPolicy, type JudgeRubric, JudgeRunner, type JudgeScore, KNOWLEDGE_GAP_KIND_SPEC, KNOWLEDGE_POISONING_KIND_SPEC, type KeywordConceptSpec, type KeywordCoverageFinding, type KeywordCoverageOptions, type KeywordCoverageResult, type LangfuseEnvelope, type LangfuseGeneration, type LangfuseScore, Layer, LayerResult, type LineageKind, type LineageKindResolver, type LineageNode, LineageRecorder, type LiveProofArtifact, type LiveProofConfig, type LiveProofContext, type LiveProofResult, LlmCallRequest, LlmCallResult, LlmClientOptions, LlmSpan, LockedJsonlAppender, MODEL_PRICING, type MatchResult, type MatcherResult, type MeasurementPolicy, type MergeOptions, MetricsCollector, type MockTransportOpts, type MuffledFinder, type MuffledFinding, MultiLayerVerifier, MultiShotMutateAdapter, MultiShotOptimizationResult, MultiShotRunner, MultiShotScorer, MultiShotTrialResult, type MultiToolchainLayerConfig, MutateAdapter, type MutationAttempt, type MutationChannel, MutationTelemetry, type Mutator, Mutex, Objective, type Oracle, type OracleObservation, type OracleReport, type OracleResult, type OrthogonalityInput, type OrthogonalityResult, PairwiseSteeringOptimizer, type ParaphraseRobustnessScenarioInput, type ParaphraseRobustnessScenarioResult, ParetoResult, type PersistedFinding, type PersonaConfig, type Playbook, type PlaybookEntry, type PoolSlot, ProductClient, type ProductClientConfig, type ProductionEvolveConfig, type ProductionLoopCronConfig, type ProductionLoopDecision, type ProductionLoopRenderContext, type ProductionLoopResult, type ProductionShipConfig, type PromptHandle, PromptRegistry, TrialResult as PromptTrialResult, type ProposeAutomatedPullRequestInput, type ProposeAutomatedPullRequestResult, RAW_FINDING_SCHEMA_PROMPT, type RawAnalystFinding, RawAnalystFindingSchema, type ReferenceMatchResult, type ReferenceReplayAdapter, type ReferenceReplayAdapterFn, type ReferenceReplayAdapterLike, type ReferenceReplayAggregate, type ReferenceReplayCandidate, type ReferenceReplayCase, type ReferenceReplayCaseRun, type ReferenceReplayExecutionScenario, type ReferenceReplayItem, type ReferenceReplayMatch, type ReferenceReplayMatchStrategy, type ReferenceReplayMatcher, type ReferenceReplayPromotionDecision, type ReferenceReplayPromotionPolicy, type ReferenceReplayRun, type ReferenceReplayRunContext, type ReferenceReplayRunOptions, type ReferenceReplayRunStore, type ReferenceReplayScenario, type ReferenceReplayScenarioScore, type ReferenceReplayScore, type ReferenceReplayScoreOptions, type ReferenceReplaySplit, type ReferenceReplaySplitComparison, type ReferenceReplaySteeringRowsOptions, type RegistryRunOpts, ReleaseConfidenceScorecard, ReleaseConfidenceThresholds, type RepoRef, type ReviewerMemoryEntry, type ReviewerOutput, type ReviewerPromptInput, type ReviewerSoftFailDefaults, type ReviewerVerificationSummary, type RobustnessResult, type RouteMap, type RouterTransportOpts, type RubricDimension, Run$1 as Run, type RunCommandInput, type RunCommandResult, type RunConfig, RunCritic, type RunCriticAdapterOpts, type RunCriticOptions, type RunDiff, RunFilter, type RunProductionLoopOptions, RunRecord, type RunScore, type RunScoreWeights, RunSplitTag, type RunTrace, SEMANTIC_CONCEPT_JUDGE_VERSION, SandboxDriver, SandboxHarnessResult, type SandboxJudgeKind, type SandboxJudgeResult, type SandboxJudgeSpec, type SandboxPool, type SandboxSdkTransportOpts, type ScanOptions, type Scenario, type ScenarioCost, type ScenarioFile, ScenarioRegistry, type ScenarioResult, type ScoredTarget, type SelfPlayOptions, type SelfPlayProposer, type SelfPlayScorer, type SemanticConceptJudgeAdapterOpts, type SemanticConceptJudgeInput, type SemanticConceptJudgeOptions, type SemanticConceptJudgeResult, type SeriesConvergenceOptions, type SeriesConvergenceResult, Severity, type SignedManifest, type SignedManifestAlgo, type Slo, type SloCheckResult, type SloComparator, type SloReport, type SloSeverity, type SlopCategory, type SlotFactory, Span, type SteeringBundle, type SteeringDelta, type SteeringOptimizationResult, type SteeringOptimizationRow, type SteeringOptimizationSelector, type SteeringOptimizerBackend, type SteeringOptimizerConfig, type SteeringRolePrompt, type StepAttribution, type SynthesisReason, type SynthesisTarget, type TestResult, type ThresholdContract, TokenCounter, type TokenSpec, type TraceAnalystAdapterOpts, type TraceAnalystGolden, type TraceAnalystKindSpec, TraceEmitter, TraceEvent, TraceStore, type TraceToolGroupName, Trajectory, TrajectoryStep, type TrialAggregate, type TrialAttempt, TrialCache, TrialTelemetry, type Turn, type TurnMetrics, type TurnResult, UNIVERSAL_FINDERS, type ValidationContext, type ValidationIssue, type ValidationResult, VariantAggregate, type VerifierAdapterOpts, VerifyContext, VerifyOptions, type VisualDiffOptions, type VisualDiffResult, type ViteDeployRunnerInput, type WorkflowTopology, type WorkspaceAssertion, type WorkspaceAssertionResult, type WorkspaceInspector, type WorkspaceSnapshot, type WranglerDeployRunnerInput, adversarialJudge, aggregateRunScore, aggregateTrialsByMode, analyzeAntiSlop, analyzeSeries, attributeCounterfactuals, benjaminiHochberg, bisect, bonferroni, buildReviewerPrompt, buildTraceToolsForGroup, byteLengthRange, canaryLeakView, canonicalize, causalAttribution, checkBehavioralCanary, checkCanaries, checkSlos, clamp01, codeExecutionJudge, cohensD, coherenceJudge, collectionPreserved, commitBisect, compareReferenceReplay, compilerJudge, composeValidators, computeFindingId, confidenceInterval, containsAll, corpusInterRaterAgreement, corpusInterRaterAgreementFromJudgeScores, createAntiSlopJudge, createChatClient, createCompositeMutator, createCustomJudge, createDefaultReviewer, createDomainExpertJudge, createIntentMatchJudge, createJudgeAdapter, createRunCriticAdapter, createSandboxCodeMutator, createSandboxPool, createSemanticConceptJudge, createSemanticConceptJudgeAdapter, createTraceAnalystAdapter, createTraceAnalystKind, createVerifierAdapter, crossTraceDiff, decideReferenceReplayPromotion, decideReferenceReplayRunPromotion, defaultIsMaterial, defaultJudges, defaultReferenceReplayMatcher, deployGateLayer, diffFindings, discoverPersonas, distillPlaybook, estimateCost, estimateTokens, evaluateContract, evaluateHypothesis, evaluateOracles, executeScenario, expectAgent, exportRewardModel, extractAssetUrls, extractErrorCount, fileContains, fileExists, findAutoMatchNoExpectation, findConstructorCwdDropped, findFallbackToPass, findLiteralTruePass, findSkipCountsAsPass, flowLayer, formatBenchmarkReport, formatDriverReport, formatFindings, ghCliClient, precision as goldenPrecision, hashContent, hashJson, htmlContainsElement, httpGithubClient, inMemoryReferenceReplayStore, integrationAsi, integrationGateEvals, integrationInvokeFailedPayload, integrationManifestResolvedPayload, integrationManifestValidatedPayload, interRaterReliability, jsonHasKeys, jsonShape, jsonlReferenceReplayStore, keyPreserved, liftSeverity, linterJudge, loadScorerFromGrader, localCommandRunner, lowercaseMutator, makeFinding, mannWhitneyU, matchGoldens, mergeLayerResults, mergeSteeringBundle, multiToolchainLayer, normalizeScores, notBlocked, pairedTTest, paraphraseRobustness, paraphraseRobustnessScenarios, parseRawFinding, partialCredit, passOrthogonality, pixelDeltaRatio, politenessPrefixMutator, printDriverSummary, promptBisect, proposeAutomatedPullRequest, proposeSynthesisTargets, referenceReplayRunsToSteeringRows, referenceReplayScenarioToRunScore, regexMatch, regexMatches, renderMarkdownReport, renderPlaybookMarkdown, renderSteeringText, replayScorerOverCorpus, replayTraceThroughJudge, requiredSampleSize, resetLockedAppendersForTesting, rowCount, rowWhere, runAssertions, runBehavioralCanaries, runCanaries, runCounterfactual, runE2EWorkflow, runExpectations, runHarnessExperiment, runIntentMatchJudge, runJudgeFleet, runKeywordCoverageJudge, runKeywordCoverageJudgeUrl, runLiveProof, runProductionLoop, runReferenceReplay, runSelfPlay, runSemanticConceptJudge, scanForMuffledGates, scoreContinuity, scoreReferenceReplay, securityJudge, selectHarnessVariant, sentenceReorderMutator, signManifest, statusAdvanced, summarizeHarnessResults, testJudge, textInSnapshot, toLangfuseEnvelope, toPrometheusText, typoMutator, urlContains, verifyManifest, visualDiff, viteDeployRunner, weightedMean, weightedRecall, whitespaceCollapseMutator, wilcoxonSignedRank, withJudgeRetry, wranglerDeployRunner };
+export { ANALYST_SEVERITIES, ActionableSideInfo, type ActiveLearningOptions, type AdapterRun, AgentDriver, type AgentDriverConfig, AgentEvalError, type AggregatorMode, type AlignmentOp, type Analyst, type AnalystContext, type AnalystCost, type AnalystFinding, type AnalystHooks, type AnalystInputKind, AnalystRegistry, type AnalystRegistryOptions, type AnalystRequirements, type AnalystRunInputs, type AnalystRunResult, type AnalystRunSummary, type AnalystSeverity, type AntiSlopConfig, type AntiSlopIssue, type AntiSlopReport, Artifact$1 as Artifact, type ArtifactCheck, type Artifact as ArtifactCheckArtifact, type ArtifactResult, type ArtifactValidator, type AutoPrClient, AxGepaSteeringOptimizer, type AxSteeringOptimizerConfig, BaselineReport, BehaviorAssertion, type BenchmarkReport, BenchmarkRunner, type BenchmarkRunnerConfig, type BisectOptions, type BisectResult, type BisectStep, BudgetBreachError, BudgetGuard, BudgetLedgerEntry, type BudgetPolicy, BudgetSpec, CallExpectation, type CanaryAlert, type CanaryKind, type CanaryLeak, type CanaryOptions, type CanaryReport, type CanarySeverity, type CandidateScenario, type CausalAttributionReport, type ChatCallOpts, type ChatClient, type ChatRequest, type ChatResponse, type ChatTransport, type CheckResult, type CliBridgeTransportOpts, type CodeMutationOutcome, type CodeMutationRunner, type CollectedArtifacts, type CommandRunner, type CompletionCriterion, type CompositePolicy, type ConceptComplexity, type ConceptFinding, type ConceptSpec, type ConceptWeightStrategy, type ContinuityCheck, type ContinuityCheckResult, type ContinuityReport, type ContinuitySnapshotPair, ContinuousAgreement, ContinuousAgreementOptions, type ContractMetric, type ContractReport, ControlEvalResult, ConvergenceTracker, type CorpusAgreementOptions, type CorpusAgreementPerDimension, type CorpusAgreementReport, type CorpusScoreRecord, type CostEntry, CostLedger, type CostLedgerGeneration, type CostLedgerSnapshot, type CostSummary, CostTracker, type CounterfactualContext, type CounterfactualMutation, type CounterfactualResult, type CounterfactualRunner, type CreateChatClientOpts, type CreateCompositeMutatorOpts, type CreateDefaultReviewerOptions, type CreateSandboxCodeMutatorOpts, type CreateSandboxPoolOpts, type CreateTraceAnalystKindOpts, type CrossTraceDiff, type CrossTraceDiffOptions, D1ExperimentStore, type D1ExperimentStoreOptions, type D1Like, type D1PreparedStatementLike, DEFAULT_AGENT_SLOS, DEFAULT_COMPLEXITY_WEIGHTS, DEFAULT_FINDERS, DEFAULT_HARNESS_OBJECTIVES, DEFAULT_MUTATORS, DEFAULT_RUN_SCORE_WEIGHTS, DEFAULT_SEVERITY_WEIGHTS, DEFAULT_TRACE_ANALYST_KINDS, Dataset, DatasetScenario, type DeployFamily, type DeployGateLayerInput, type DeployRunResult, type DeployRunner, type DiffPolicy, type DirEntry, type DirectProviderTransportOpts, type DiscoverPersonasOptions, type DiscoveredPersona, type DriverResult, type DriverState, DualAgentBench, type DualAgentBenchConfig, type DualAgentReport, type DualAgentRound, type DualAgentScenario, type DualAgentScenarioResult, ERROR_COUNT_PATTERNS, type ErrorCountPattern, type EvalResult, type EvidenceRef, type EvolutionRound, EvolvableVariant, type ExecutorConfig, type Expectation, type Experiment, type Run as ExperimentRun, type ExperimentStore, ExperimentTracker, type ExportedRewardModel, type ExtractOptions, type ExtractResult, FAILURE_MODE_KIND_SPEC, type FactorContribution, type FactorialCell, type FailureClusterConfig, FeedbackLabel, type FeedbackPattern, FeedbackTrajectory, FeedbackTrajectoryStore, type FileChange, FileSystemExperimentStore, type FileSystemExperimentStoreOptions, type FindingsDiff, FindingsStore, type FlowAction, type FlowLayerEnv, type FlowLayerFactoryInput, type FlowRunner, type FlowRunnerStepResult, type FlowSpec, type FlowStep, GateDecision, type GhCliClientOptions, type GoldenSeverity, type GoldenSpec, type HarnessAdapter, HarnessConfig, type HarnessExperimentConfig, type HarnessExperimentResult, type HarnessIntervention, type HarnessRunRequest, type HarnessRunResult, type HarnessScenario, type HarnessSelection, type HarnessVariant, type HarnessVariantReport, HeldOutGateConfig, HoldoutAuditor, type HostedJudgeConfig, type HostedJudgeDimension, type HostedJudgeRequest, type HostedJudgeResponse, type HostedRunCriticConfig, type HostedRunScoreRequest, type HostedRunScoreResponse, type HttpGithubClientOptions, type HypothesisManifest, type HypothesisResult, IMPROVEMENT_KIND_SPEC, INTENT_MATCH_JUDGE_VERSION, type ImageData, InMemoryExperimentStore, InMemoryWorkspaceInspector, type InferenceScorer, type InspectorContext, type IntegrationGateSurface, type IntegrationInvokeFailureInput, type IntegrationManifestGateInput, type IntentMatchInput, type IntentMatchOptions, type IntentMatchResult, type InteractionContribution, JsonlTrialCache, type JudgeAdapterOpts, type JudgeConfig, type JudgeFleetOptions, type JudgeFn, type JudgeInput, type JudgeReplayResult, type JudgeRetryOutcome, type JudgeRetryPolicy, type JudgeRubric, JudgeRunner, type JudgeScore, KNOWLEDGE_GAP_KIND_SPEC, KNOWLEDGE_POISONING_KIND_SPEC, type KeywordConceptSpec, type KeywordCoverageFinding, type KeywordCoverageOptions, type KeywordCoverageResult, type LangfuseEnvelope, type LangfuseGeneration, type LangfuseScore, Layer, LayerResult, type LineageKind, type LineageKindResolver, type LineageNode, LineageRecorder, type LiveProofArtifact, type LiveProofConfig, type LiveProofContext, type LiveProofResult, LlmCallRequest, LlmCallResult, LlmClientOptions, LlmSpan, LockedJsonlAppender, MODEL_PRICING, type MatchResult, type MatcherResult, type MeasurementPolicy, type MergeOptions, MetricsCollector, type MockTransportOpts, type MuffledFinder, type MuffledFinding, MultiLayerVerifier, MultiShotMutateAdapter, MultiShotOptimizationResult, MultiShotRunner, MultiShotScorer, MultiShotTrialResult, type MultiToolchainLayerConfig, MutateAdapter, type MutationAttempt, type MutationChannel, MutationTelemetry, type Mutator, Mutex, Objective, type Oracle, type OracleObservation, type OracleReport, type OracleResult, type OrthogonalityInput, type OrthogonalityResult, PairwiseSteeringOptimizer, type ParaphraseRobustnessScenarioInput, type ParaphraseRobustnessScenarioResult, ParetoResult, type PersistedFinding, type PersonaConfig, type Playbook, type PlaybookEntry, type PoolSlot, ProductClient, type ProductClientConfig, type ProductionEvolveConfig, type ProductionLoopCronConfig, type ProductionLoopDecision, type ProductionLoopRenderContext, type ProductionLoopResult, type ProductionShipConfig, type PromptHandle, PromptRegistry, TrialResult as PromptTrialResult, type ProposeAutomatedPullRequestInput, type ProposeAutomatedPullRequestResult, RAW_FINDING_SCHEMA_PROMPT, type RawAnalystFinding, RawAnalystFindingSchema, type ReferenceMatchResult, type ReferenceReplayAdapter, type ReferenceReplayAdapterFn, type ReferenceReplayAdapterLike, type ReferenceReplayAggregate, type ReferenceReplayCandidate, type ReferenceReplayCase, type ReferenceReplayCaseRun, type ReferenceReplayExecutionScenario, type ReferenceReplayItem, type ReferenceReplayMatch, type ReferenceReplayMatchStrategy, type ReferenceReplayMatcher, type ReferenceReplayPromotionDecision, type ReferenceReplayPromotionPolicy, type ReferenceReplayRun, type ReferenceReplayRunContext, type ReferenceReplayRunOptions, type ReferenceReplayRunStore, type ReferenceReplayScenario, type ReferenceReplayScenarioScore, type ReferenceReplayScore, type ReferenceReplayScoreOptions, type ReferenceReplaySplit, type ReferenceReplaySplitComparison, type ReferenceReplaySteeringRowsOptions, type RegistryRunOpts, ReleaseConfidenceScorecard, ReleaseConfidenceThresholds, type RepoRef, type ReviewerMemoryEntry, type ReviewerOutput, type ReviewerPromptInput, type ReviewerSoftFailDefaults, type ReviewerVerificationSummary, type RobustnessResult, type RouteMap, type RouterTransportOpts, type RubricDimension, Run$1 as Run, type RunCommandInput, type RunCommandResult, type RunConfig, RunCritic, type RunCriticAdapterOpts, type RunCriticOptions, type RunDiff, RunFilter, type RunProductionLoopOptions, RunRecord, type RunScore, type RunScoreWeights, RunSplitTag, type RunTrace, SEMANTIC_CONCEPT_JUDGE_VERSION, SandboxDriver, SandboxHarnessResult, type SandboxJudgeKind, type SandboxJudgeResult, type SandboxJudgeSpec, type SandboxPool, type SandboxSdkTransportOpts, type ScanOptions, type Scenario, type ScenarioCost, type ScenarioFile, ScenarioRegistry, type ScenarioResult, type ScoredTarget, type SelfPlayOptions, type SelfPlayProposer, type SelfPlayScorer, type SemanticConceptJudgeAdapterOpts, type SemanticConceptJudgeInput, type SemanticConceptJudgeOptions, type SemanticConceptJudgeResult, type SeriesConvergenceOptions, type SeriesConvergenceResult, Severity, type SignedManifest, type SignedManifestAlgo, type Slo, type SloCheckResult, type SloComparator, type SloReport, type SloSeverity, type SlopCategory, type SlotFactory, Span, type SteeringBundle, type SteeringDelta, type SteeringOptimizationResult, type SteeringOptimizationRow, type SteeringOptimizationSelector, type SteeringOptimizerBackend, type SteeringOptimizerConfig, type SteeringRolePrompt, type StepAttribution, type SynthesisReason, type SynthesisTarget, type TestResult, type ThresholdContract, TokenCounter, type TokenSpec, type TraceAnalystAdapterOpts, type TraceAnalystGolden, type TraceAnalystKindSpec, TraceEmitter, TraceEvent, TraceStore, type TraceToolGroupName, Trajectory, TrajectoryStep, type TrialAggregate, type TrialAttempt, TrialCache, TrialTelemetry, type Turn, type TurnMetrics, type TurnResult, UNIVERSAL_FINDERS, type ValidationContext, type ValidationIssue, type ValidationResult, VariantAggregate, type VerifierAdapterOpts, VerifyContext, VerifyOptions, type VisualDiffOptions, type VisualDiffResult, type ViteDeployRunnerInput, type WorkflowTopology, type WorkspaceAssertion, type WorkspaceAssertionResult, type WorkspaceInspector, type WorkspaceSnapshot, type WranglerDeployRunnerInput, adversarialJudge, aggregateRunScore, aggregateTrialsByMode, analyzeAntiSlop, analyzeSeries, attributeCounterfactuals, benjaminiHochberg, bisect, bonferroni, buildReviewerPrompt, buildTraceToolsForGroup, byteLengthRange, canaryLeakView, canonicalize, causalAttribution, checkBehavioralCanary, checkCanaries, checkSlos, clamp01, codeExecutionJudge, cohensD, coherenceJudge, collectionPreserved, commitBisect, compareReferenceReplay, compilerJudge, composeValidators, computeFindingId, confidenceInterval, containsAll, corpusInterRaterAgreement, corpusInterRaterAgreementFromJudgeScores, createAntiSlopJudge, createChatClient, createCompositeMutator, createCustomJudge, createDefaultReviewer, createDomainExpertJudge, createIntentMatchJudge, createJudgeAdapter, createRunCriticAdapter, createSandboxCodeMutator, createSandboxPool, createSemanticConceptJudge, createSemanticConceptJudgeAdapter, createTraceAnalystAdapter, createTraceAnalystKind, createVerifierAdapter, crossTraceDiff, decideReferenceReplayPromotion, decideReferenceReplayRunPromotion, defaultIsMaterial, defaultJudges, defaultReferenceReplayMatcher, deployGateLayer, diffFindings, discoverPersonas, distillPlaybook, estimateCost, estimateTokens, evaluateContract, evaluateHypothesis, evaluateOracles, executeScenario, expectAgent, exportRewardModel, extractAssetUrls, extractErrorCount, fileContains, fileExists, findAutoMatchNoExpectation, findConstructorCwdDropped, findFallbackToPass, findLiteralTruePass, findSkipCountsAsPass, flowLayer, formatBenchmarkReport, formatDriverReport, formatFindings, ghCliClient, precision as goldenPrecision, hashContent, hashJson, htmlContainsElement, httpGithubClient, inMemoryReferenceReplayStore, integrationAsi, integrationGateEvals, integrationInvokeFailedPayload, integrationManifestResolvedPayload, integrationManifestValidatedPayload, interRaterReliability, jsonHasKeys, jsonShape, jsonlReferenceReplayStore, keyPreserved, liftSeverity, linterJudge, loadScorerFromGrader, localCommandRunner, lowercaseMutator, makeFinding, mannWhitneyU, matchGoldens, mergeLayerResults, mergeSteeringBundle, multiToolchainLayer, normalizeScores, notBlocked, pairedTTest, paraphraseRobustness, paraphraseRobustnessScenarios, parseRawFinding, partialCredit, passOrthogonality, pixelDeltaRatio, politenessPrefixMutator, printDriverSummary, promptBisect, proposeAutomatedPullRequest, proposeSynthesisTargets, referenceReplayRunsToSteeringRows, referenceReplayScenarioToRunScore, regexMatch, regexMatches, renderMarkdownReport, renderPlaybookMarkdown, renderPriorFindings, renderSteeringText, replayScorerOverCorpus, replayTraceThroughJudge, requiredSampleSize, resetLockedAppendersForTesting, rowCount, rowWhere, runAssertions, runBehavioralCanaries, runCanaries, runCounterfactual, runE2EWorkflow, runExpectations, runHarnessExperiment, runIntentMatchJudge, runJudgeFleet, runKeywordCoverageJudge, runKeywordCoverageJudgeUrl, runLiveProof, runProductionLoop, runReferenceReplay, runSelfPlay, runSemanticConceptJudge, scanForMuffledGates, scoreContinuity, scoreReferenceReplay, securityJudge, selectHarnessVariant, sentenceReorderMutator, signManifest, statusAdvanced, summarizeHarnessResults, testJudge, textInSnapshot, toLangfuseEnvelope, toPrometheusText, typoMutator, urlContains, verifyManifest, visualDiff, viteDeployRunner, weightedMean, weightedRecall, whitespaceCollapseMutator, wilcoxonSignedRank, withJudgeRetry, wranglerDeployRunner };