@tangle-network/agent-eval 0.79.0 → 0.81.0

package/CHANGELOG.md

@@ -4,6 +4,20 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
 
 ---
 
+## [0.81.0] — 2026-06-05 — eval-campaign scaffold prep primitives
+
+### Added
+
+- **`aggregateJudgeVerdicts<D>` (root).** Generic judge-ensemble reducer: fan out N uncorrelated judges, mean each rubric dimension over the SURVIVORS, report the inter-rater disagreement spread, sum cost. Replaces the same reduction hand-rolled in legal (`aggregateEnsemble`), creative (`production-loop/judges.ts`), and tax (`judge-ensemble.ts`). Fail-loud: a failed judge (`perDimension: null`) is recorded in `failedJudges`, never folded into a zero; all-failed throws; a failed judge's cost is still summed. Composite reuses `weightedComposite`.
+- **`createTokenRecallChecker` (root).** The deterministic, no-LLM `CorrectnessChecker` — sibling of `createLlmCorrectnessChecker`. A produced item fulfils a requirement when its content is substantive and recalls ≥ `minRecall` of the requirement title's significant tokens. The default completion gate for apps/tests without an LLM judge.
+- **`ErrorCluster` (root + `/analyst`).** The failure-cluster element type is now a named export, so consumers import it instead of deriving `DatasetOverview['error_clusters'][number]`.
+
+### Fixed
+
+- **Lint drift + non-executable pre-commit hook.** `.husky/pre-commit` was tracked `100644`, so the hook silently no-op'd and unformatted code reached `main`; marked executable and reformatted the drift.
+
+---
+
 ## [0.72.3] — 2026-06-01 — workflow trace hardening and driver backtests
 
 ### Added

package/README.md

@@ -1,197 +1,106 @@
 # `@tangle-network/agent-eval`
 
-**Ship better agent prompts with statistical confidence.** One function call returns a decision packet: lift CI, judge calibration, contamination check, failure clusters, cost-quality Pareto, and a ranked action list. Same shape whether you've got a closed improvement loop or just production logs.
+Evaluate and improve AI agents from the runs they already produce.
+
+`agent-eval` turns agent outputs, traces, judge scores, and production feedback into a decision packet: did this change help, what failed, what should ship, and what needs more data?
 
 [![npm](https://img.shields.io/npm/v/@tangle-network/agent-eval.svg)](https://www.npmjs.com/package/@tangle-network/agent-eval)
 [![pypi](https://img.shields.io/pypi/v/agent-eval-rpc.svg)](https://pypi.org/project/agent-eval-rpc/)
 [![tests](https://github.com/tangle-network/agent-eval/actions/workflows/ci.yml/badge.svg)](https://github.com/tangle-network/agent-eval/actions/workflows/ci.yml)
 [![license: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
 
-> TypeScript first-class, Python (`agent-eval-rpc`) speaks the same wire protocol, hosted-tier-friendly, MIT, self-hostable, no SaaS dependency.
+Use it when you need to:
 
----
+- compare a candidate agent/prompt/model against a baseline,
+- turn production traces or human feedback into eval results,
+- run a gated self-improvement loop,
+- explain failures by cluster, cost, judge disagreement, and release risk.
 
-## Table of contents
-
-- [What you get back](#what-you-get-back-the-decision-packet)
-- [Quick start](#quick-start)
-  - [Closed loop — `selfImprove()`](#closed-loop--selfimprove)
-  - [Observed runs — `analyzeRuns()`](#observed-runs--analyzeruns)
-  - [Existing data — intake adapters](#existing-data--intake-adapters)
-- [How it compares](#how-it-compares)
-- [Customer journeys](#customer-journeys)
-- [Subpath entry points](#subpath-entry-points)
-- [Concepts + design](#concepts--design)
-- [Hosted tier](#hosted-tier)
-- [Install + run](#install--run)
-- [Stability + versioning](#stability--versioning)
-- [License](#license)
+It is a library, not a SaaS requirement. TypeScript is first-class; Python can call the same wire protocol through `agent-eval-rpc`.
 
 ---
 
-## What you get back: the decision packet
+## Install
 
-Whether you call `selfImprove()` (closed loop) or `analyzeRuns()` (observed runs), the report has the same shape. Here's a real one, abridged:
-
-```jsonc
-{
-  "n": 80,                                            // runs analyzed
-  "composite": {                                       // distributional summary
-    "mean": 0.62, "p50": 0.65, "p95": 0.88, "stddev": 0.17,
-    "histogram": [/* 12 bins */]
-  },
-  "lift": {                                            // paired bootstrap
-    "baselineMean": 0.58, "candidateMean": 0.65,
-    "delta": 0.07,
-    "ci95": [0.04, 0.10],                              // 95% CI on the delta
-    "pValue": 0.0008,                                  // paired-t
-    "cohensD": 0.41,
-    "n": 40,
-    "mde": 0.06,                                       // min detectable effect at 80% power
-    "requiredN": 38                                    // n needed to detect observed delta
-  },
-  "judges": {                                          // per-judge calibration
-    "domain-expert": { "n": 80, "meanScore": 0.64 },
-    "helpfulness-llm": { "n": 80, "meanScore": 0.61 }
-  },
-  "interRater": {                                      // multi-rater agreement
-    "raters": 3, "jointlyRated": 80, "kappa": 0.71,
-    "disagreementCases": [/* top 20 ranked by spread */]
-  },
-  "costQuality": {                                     // cost-vs-quality
-    "cost": { "mean": 0.024, "p95": 0.041, /* ... */ },
-    "pareto": { /* ParetoFigureSpec the dashboard renders */ }
-  },
-  "failureClusters": {                                 // when an AnalystRegistry is wired
-    "totalFailures": 11,
-    "clusters": [
-      { "name": "off-topic-drift",  "share": 0.45, "exemplars": ["run-12", "run-19"] },
-      { "name": "over-confidence",  "share": 0.27, "exemplars": ["run-3"] },
-      { "name": "format-mismatch",  "share": 0.18, "exemplars": ["run-41"] }
-    ]
-  },
-  "contamination": { "leaks": 0, "holdoutAuditPassed": true },
-  "outcomeCorrelation": {                              // when downstream metric supplied
-    "metric": "engagement_rate", "n": 80,
-    "pearson": 0.72, "spearman": 0.69,
-    "rewardModel": { "intercept": 0.04, "slope": 1.93, "r2": 0.52 }
-  },
-  "release": {
-    "status": "pass",
-    "axes": [
-      { "name": "quality-lift",          "status": "pass" },
-      { "name": "contamination",         "status": "pass" },
-      { "name": "composite-distribution","status": "pass" }
-    ]
-  },
-  "recommendations": [
-    { "priority": "critical", "kind": "ship",
-      "title": "Ship — lift 0.070 (95% CI 0.040..0.100)",
-      "detail": "Holdout lift exceeds threshold 0.02 with 95% bootstrap confidence (n=40, p=0.0008, d=0.41)." },
-    { "priority": "high", "kind": "investigate",
-      "title": "Top failure cluster: off-topic-drift (45% of failures)",
-      "detail": "11 runs failed. Drill into exemplars run-12 / run-19 to identify the pattern." }
-  ]
-}
+```sh
+pnpm add @tangle-network/agent-eval
 ```
 
-The `recommendations` array is the human-readable layer; everything above it is the evidence. Read the recs, act on them, the numbers are the proof.
+Python clients can use the RPC package:
+
+```sh
+pip install agent-eval-rpc
+```
 
 ---
 
 ## Quick start
 
-### Closed loop — `selfImprove()`
+### 1. Analyze runs you already have
 
-You have scenarios, a dispatch, judges, and want the loop to propose better prompts + tell you which to ship.
+Start here if you already have production logs, benchmark rows, human ratings, or agent run records.
 
 ```ts
-import { selfImprove } from '@tangle-network/agent-eval/contract'
+import { analyzeRuns } from '@tangle-network/agent-eval/contract'
 
-const result = await selfImprove({
-  scenarios,                                // your scenario corpus
-  dispatch: async ({ scenario }) =>          // your agent — anything that returns an artifact
-    await myAgent.run(scenario),
-  judges: [myJudge],                         // any JudgeConfig — LLM, rule, ensemble
-  baselineSurface: { systemPrompt: currentPrompt },
+const report = await analyzeRuns({
+  runs, // RunRecord[]
+  baselineRuns,
 })
 
-result.gateDecision         // 'ship' | 'hold' | 'need_more_work' | ...
-result.lift                 // raw delta on holdout
-result.insight              // the full decision packet above
+console.log(report.recommendations)
+console.log(report.lift)
+console.log(report.failureClusters)
 ```
 
-### Observed runs — `analyzeRuns()`
+The output includes score distributions, lift confidence intervals, failure modes, cost-quality tradeoffs, judge agreement, contamination checks, and release recommendations when the input supports them.
+
+### 2. Run a gated improvement loop
 
-You don't have a closed loop yet — you have observed runs (production traces, an approve/reject corpus, a CSV gold set). Same report shape, no agent invocation.
+Use this when you have scenarios, a runnable agent, and judges.
 
 ```ts
-import { analyzeRuns } from '@tangle-network/agent-eval/contract'
+import { selfImprove } from '@tangle-network/agent-eval/contract'
 
-const report = await analyzeRuns({
-  runs,                                     // RunRecord[]
-  outcomeSignal: {                          // optional — closes the loop on real outcomes
-    metric: 'engagement_rate',
-    valueByRunId: enrichedFromProd,
-  },
-  canaryScenarios,                          // optional — contamination probe
-  analyst: myAnalystRegistry,               // optional — AI-powered failure clustering
+const result = await selfImprove({
+  scenarios,
+  dispatch: async ({ scenario }) => myAgent.run(scenario),
+  judges: [myJudge],
+  baselineSurface: { systemPrompt: currentPrompt },
 })
 
-report.recommendations    // ranked actions
-report.failureClusters    // grouped failure modes
-report.outcomeCorrelation // judge↔outcome correlation + linear reward model
+console.log(result.gateDecision)
+console.log(result.winnerSurface)
+console.log(result.insight.recommendations)
 ```
 
-### Existing data — intake adapters
+`selfImprove()` evaluates candidates on held-out scenarios before recommending a winner.
 
-You have data already. Don't reshape it — pipe it through an adapter.
+### 3. Adapt existing data
 
 ```ts
-import {
-  fromFeedbackTable,
-  fromOtelSpans,
-  analyzeRuns,
-} from '@tangle-network/agent-eval/contract'
+import { analyzeRuns, fromFeedbackTable, fromOtelSpans } from '@tangle-network/agent-eval/contract'
 
-// Multi-rater approve/reject (Obsidian tags, Sheets, CSV, Postgres).
 const { runs, raterScores } = fromFeedbackTable({
-  ratings: parseYourFeedbackTable(),         // Array<{ runId, rater, rating }>
+  ratings: parseYourFeedbackTable(),
 })
-await analyzeRuns({ runs, raterScores })
 
-// Production OTel traces — group by tangle.runId or traceId.
-const runs2 = fromOtelSpans({ spans: yourOtelStream })
-await analyzeRuns({ runs: runs2 })
-```
+const traceRuns = fromOtelSpans({ spans: yourOtelSpans })
 
-Both intake adapters preserve every signal in the source — multi-rater scores stay rater-keyed so the report can compute inter-rater agreement and surface the disagreement triage list.
+await analyzeRuns({ runs: [...runs, ...traceRuns], raterScores })
+```
 
 ---
 
-## How it compares
-
-| | LangSmith | Braintrust | Phoenix | **agent-eval** |
-|---|:---:|:---:|:---:|:---:|
-| Closed-loop self-improvement | ✱ human-in-loop | ✱ experiment-driven | — | ✓ autonomous + gated |
-| Statistical lift CI (paired bootstrap) | — | partial | — | ✓ |
-| Judge calibration + bias detection | — | — | — | ✓ |
-| Inter-rater agreement + disagreement triage | — | — | — | ✓ |
-| Contamination / canary check | — | — | — | ✓ |
-| AI-driven failure clustering | partial | — | partial | ✓ |
-| Cost-quality Pareto | — | — | — | ✓ |
-| Multi-language clients (TS + Python) | TS only | TS only | TS + Py | ✓ TS + Py |
-| Self-hostable / no-SaaS option | — | — | OSS | ✓ MIT, OSS |
-| Substrate vs SaaS shape | SaaS | SaaS | OSS server | **library** |
-| Hosted tier (optional) | required | required | optional | optional |
+## Core concepts
 
-Position: agent-eval is the **substrate** (one library, decision-grade output) the others are SaaS *around* the substrate. If you want a closed loop that ships your prompt under statistical confidence, you call agent-eval. If you want a dashboard rendered from your data, you pipe agent-eval into the hosted tier or your own renderer.
+- **RunRecord**: the durable row for one agent run: model, prompt/config hashes, split, cost, tokens, outcome.
+- **Scenario**: one task or case the agent attempts.
+- **Judge**: a scoring function, rule-based or model-based.
+- **InsightReport**: the decision packet returned by `analyzeRuns()` and embedded in `selfImprove()`.
+- **Gate**: the policy that decides `ship`, `hold`, or `need_more_data`.
 
----
-
-## Customer journeys
-
-Three runnable examples — each is self-contained, each shows the actual output.
+## Examples
 
 | Journey | Example | Who it's for |
 |---|---|---|
@@ -207,28 +116,55 @@ Each example: `README.md` + a single `index.ts` runnable via `pnpm tsx`. Prints
 
 | Subpath | What it gives you |
 |---|---|
-| `@tangle-network/agent-eval/contract` | **The headline surface.** `selfImprove`, `analyzeRuns`, `runImprovementLoop`, `runCampaign`, `runEval`, `diffRuns`, intake adapters (`fromFeedbackTable`, `fromOtelSpans`), drivers (`gepaDriver`, `evolutionaryDriver`), gates (`defaultProductionGate`, `heldOutGate`, `composeGate`), storage. **New code starts here.** |
-| `@tangle-network/agent-eval/hosted` | Hosted-tier wire-format types + `createHostedClient` to ship eval-run events + trace spans to any orchestrator speaking the spec |
-| `@tangle-network/agent-eval/adapters/otel` | `createOtelBridge` — forwards OpenTelemetry-shape spans into the hosted-tier ingest |
-| `@tangle-network/agent-eval/adapters/langchain` | LangChain runnable → `Dispatch` adapter |
-| `@tangle-network/agent-eval/adapters/http` | `httpDispatch` + `runDispatchServer` for distributed campaigns across machines |
-| `@tangle-network/agent-eval/campaign` | Lower-level campaign primitives (storage, drivers, types) |
-| `@tangle-network/agent-eval/multishot` | N-shot persona × shot matrix runner |
-| `@tangle-network/agent-eval/control` | Agent control loop primitives (`runAgentControlLoop`, action policy, propose/review) |
-| `@tangle-network/agent-eval/traces` | Trace stores, emitters, OTLP-JSONL replay |
-| `@tangle-network/agent-eval/reporting` | Release confidence, paired stats, sequential e-values, launch reports |
-| `@tangle-network/agent-eval/rl` | RL bridge — verifiable rewards, preferences, OPE, PRM, tournaments, contamination, compute curves, auto-research |
-| `@tangle-network/agent-eval/matrix` | N-axis cartesian over substrate types |
-| `@tangle-network/agent-eval/wire` | HTTP/RPC server + Zod schemas (same protocol the Python client speaks) |
-| `@tangle-network/agent-eval/benchmarks` | Benchmark adapter contracts and reference wrappers |
-
-The root export remains available for backward compatibility; new code should prefer focused subpaths. Anything under `/rl`, `/pipelines`, `/meta-eval`, `/prm`, or `/builder-eval` is **only** reachable via its subpath.
+| `…/contract` | **The headline, frozen surface — new code starts here.** `selfImprove`, `analyzeRuns`, `runEval`, `runCampaign`, `runImprovementLoop`, `diffRuns`; intake adapters (`fromFeedbackTable`, `fromOtelSpans`); drivers (`gepaDriver`, `evolutionaryDriver`); gates (`defaultProductionGate`, `heldOutGate`, `paretoSignificanceGate`, `composeGate`); the deployment-outcome store; storage; and the five core types `Scenario` / `Dispatch` / `JudgeConfig` / `Mutator` / `Gate`. |
+| `…/hosted` | `createHostedClient` / `hostedClientFromEnv` + the wire types to ship eval-run events + trace spans to a hosted orchestrator (ours or your own implementation of the spec) |
+| `…/adapters/otel` | `createOtelBridge` — forwards OpenTelemetry-shape spans into the hosted-tier ingest, no `@opentelemetry/*` dependency |
+| `…/adapters/langchain` | Wrap any LangChain `Runnable` as a `Dispatch` (or `JudgeConfig`), no `@langchain/core` peer dep |
+| `…/adapters/http` | `httpDispatch` + `runDispatchServer` — run a campaign's worker on another machine (multi-region, driver-as-a-service) |
+| `…/campaign` | **The measurement + improvement engine** (`@experimental`): `runProfileMatrix`, `compareDrivers`, every driver (`gepaDriver`, `haloDriver`, `skillOptDriver`, `aceDriver`, `memoryCurationDriver`, …), the gates, storage backends, and loop provenance. `/contract` re-exports the stable subset. |
+| `…/rl` | RL bridge from eval artifacts to training signal: verifiable rewards, preferences, OPE, PRM, tournaments, contamination, compute curves, plus the durable corpus + `buildRlDataset` / datasheet bundle |
+| `…/reporting` | Release-decision statistics: `pairedBootstrap`, `benjaminiHochberg`, anytime-valid sequential e-values, `evaluateReleaseConfidence`, and the report renderers |
+| `…/analyst` | The trace-analyst surface: `AnalystRegistry` + `buildDefaultAnalystRegistry` (run the failure-clustering panel), `FindingsStore`, and the LLM chat transports |
+| `…/traces` | Trace stores + emitters, OTLP-JSONL deterministic replay, `analyzeTraces`, and the `traceAnalystOnRunComplete` hook |
+| `…/control` | Agent control loop: `runAgentControlLoop` (observe → validate → decide → act), action policy, propose/review |
+| `…/matrix` | `runAgentMatrix` — an N-axis cartesian over caller-supplied substrate values, per-axis pass/score/cost/duration |
+| `…/multishot` | N-shot persona × shot matrix runner (`runMultishot` / `runMultishotMatrix`) |
+| `…/wire` | The cross-language HTTP/RPC server + Zod schemas (the source-of-truth protocol the Python client speaks) + the built-in rubric registry |
+| `…/benchmarks` | `BenchmarkAdapter` contract + `deterministicSplit` + the bundled `routing` reference benchmark |
+
+**Specialized surfaces** (subpath-only): `…/prm` (process-reward grading + best-of-N), `…/meta-eval` (judge calibration + the deployment-outcome store), `…/pipelines` (trace-diagnostic views: budget breach, failure cluster, stuck loop, …), `…/governance` (EU AI Act / NIST AI RMF / SOC2 reports), `…/knowledge` (knowledge-readiness gating before a run), `…/builder-eval` (code-generator three-layer eval), `…/storyboard` (trace → watchable replay), `…/authenticity` (anti-Goodhart "real or convincing BS" scorer over produced files), `…/workflow` (workflow-trace eval + partner export), `…/telemetry` (Workers-safe telemetry client).
+
+The root export remains available for backward compatibility; new code should prefer the focused subpaths above — `/contract` first.
+
+---
+
+## Composition with the stack
+
+agent-eval is the bottom of the layering: consumers depend on it, it depends on none of them.
+
+```
+agent-runtime    Runs agents (chat turns, one-shot tasks, multi-attempt loops), captures every
+                 run as a trace, and calls optimizePrompt / runImprovementLoop. Produces the
+                 RunRecords + traces agent-eval scores. Depends on agent-eval.
+
+agent-eval       selfImprove, analyzeRuns, runCampaign + drivers (gepaDriver, …), the gates
+   (this repo)   (heldOutGate, defaultProductionGate, paretoSignificanceGate), the InsightReport
+                 decision packet, the RL bridge, the wire protocol. Depends on neither consumer.
+
+agent-knowledge  proposeKnowledgeWrites / applyKnowledgeWriteBlocks. agent-eval's analyst findings
+                 feed it; the knowledge gate consumes them. Depends on agent-eval.
+
+sandbox          AgentProfile, Sandbox.create, streamPrompt. The execution surface the runtime's
+                 loops run on; agent-eval scores what comes back.
+```
+
+The rule: **agent-eval has zero upward dependencies on a consumer.** A concept that makes sense *without* a running agent loop — a verdict, a run record, a scenario, a judge score — is substrate and lives here; a runtime-shaped one (a sandbox profile, a validation context with an abort signal) lives in agent-runtime. When in doubt, lean substrate.
 
 ---
 
 ## Concepts + design
 
-- [`docs/concepts.md`](./docs/concepts.md) — five types, three top-level functions, the layering rule, the wire protocol contract
+- [`docs/concepts.md`](./docs/concepts.md) — the three top-level functions, the layering rule, and the wire-protocol contract (the five core contract types are documented in the `/contract` barrel itself)
 - [`docs/insight-report.md`](./docs/insight-report.md) — annotated walkthrough of every section of the decision packet
 - [`docs/customer-journeys.md`](./docs/customer-journeys.md) — three end-to-end journeys with code + expected output
 - [`docs/adapters-observability.md`](./docs/adapters-observability.md) — composing agent-eval with LangSmith, Langfuse, Phoenix, OpenLLMetry, TraceAI
@@ -259,13 +195,7 @@ The substrate runs the loop in your process. Only the eval-run events + (optiona
 
 ---
 
-## Install + run
-
-```sh
-pnpm add @tangle-network/agent-eval
-# or, from Python:
-pip install agent-eval-rpc
-```
+## Development
 
 Run an example:
 
@@ -287,7 +217,9 @@ pnpm test
 
 ## Stability + versioning
 
-Public exports carry JSDoc stability markers visible in IDE hover + `.d.ts`:
+The `/contract` surface is the **stability contract**: its barrel freezes the API — a `0.x` minor only *adds*; nothing there changes shape or disappears. Depend on `/contract` (and the documented subpaths) rather than the root barrel.
+
+In the deeper subpaths, `@stable` / `@experimental` JSDoc markers (visible in IDE hover + `.d.ts`) call out what may still move — most granularly in `/rl` (tagged per export) and `/campaign` (whole barrel `@experimental`, since `/contract` re-exports only its settled subset).
 
 | Tag | Meaning |
 |---|---|

package/dist/adapters/http.d.ts

@@ -1,5 +1,5 @@
-import { S as Scenario, D as DispatchFn, b as DispatchContext } from '../types-QHG0KnkF.js';
-import '../run-record-sItO5ftF.js';
+import { S as Scenario, D as DispatchFn, b as DispatchContext } from '../types-D7lLRYe9.js';
+import '../run-record-De9VarXR.js';
 import '../errors-Dwqw-T_m.js';
 import '../schema-m0gsnbt3.js';
 

package/dist/adapters/langchain.d.ts

@@ -1,5 +1,5 @@
-import { S as Scenario, J as JudgeScore, D as DispatchFn, a as JudgeConfig } from '../types-QHG0KnkF.js';
-import '../run-record-sItO5ftF.js';
+import { S as Scenario, J as JudgeScore, D as DispatchFn, a as JudgeConfig } from '../types-D7lLRYe9.js';
+import '../run-record-De9VarXR.js';
 import '../errors-Dwqw-T_m.js';
 import '../schema-m0gsnbt3.js';
 

package/dist/adapters/otel.d.ts

@@ -1,10 +1,10 @@
 import { TraceSpanEvent, HostedClient } from '../hosted/index.js';
-import '../types-QHG0KnkF.js';
-import '../run-record-sItO5ftF.js';
+import '../types-D7lLRYe9.js';
+import '../run-record-De9VarXR.js';
 import '../errors-Dwqw-T_m.js';
 import '../schema-m0gsnbt3.js';
-import '../insight-report-dlpEzQDi.js';
-import '../summary-report-BTaXq1TS.js';
+import '../insight-report-3ADTfClO.js';
+import '../summary-report-Db0dDSWP.js';
 import '../failure-cluster-CL7IVgkJ.js';
 import '../store-CKUAgsJz.js';
 import '../judge-calibration-DilmB3Ml.js';

package/dist/{agent-profile-aSEaJ9Pl.d.ts → agent-profile-D0PBIWlV.d.ts}

@@ -1,5 +1,5 @@
 import { A as AgentEvalError } from './errors-Dwqw-T_m.js';
-import { R as RunRecord } from './run-record-sItO5ftF.js';
+import { R as RunRecord } from './run-record-De9VarXR.js';
 import { TCloud } from '@tangle-network/tcloud';
 
 /**
@@ -258,6 +258,18 @@ declare function parseCorrectnessResponse(raw: string): {
  * fulfil a requirement — the artifact must BE the deliverable.
  */
 declare function createLlmCorrectnessChecker(tc: TCloud, opts?: LlmCorrectnessCheckerOpts): CorrectnessChecker;
+/**
+ * Deterministic `CorrectnessChecker` — the no-LLM counterpart to
+ * `createLlmCorrectnessChecker`. A produced item fulfils a requirement when its
+ * content is substantive (≥ `minContentLength` chars) AND recalls ≥ `minRecall`
+ * of the requirement title's significant tokens. No network — the default gate
+ * for apps and tests without an LLM judge. Pass to `verifyCompletion` as the
+ * checker.
+ */
+declare function createTokenRecallChecker(opts?: {
+    minRecall?: number;
+    minContentLength?: number;
+}): CorrectnessChecker;
 
 /**
  * Produced-state extraction — normalize a run's runtime event stream into the
@@ -361,4 +373,4 @@ interface AgentProfile {
  */
 declare function agentProfileHash(profile: AgentProfile): string;
 
-export { type AgentProfile as A, type BackendIntegrityReport as B, type CompletionRequirement as C, type LlmCorrectnessCheckerOpts as L, type ProducedState as P, type RuntimeEventLike as R, type SatisfiedBy as S, type TaskGold as T, type ValidationContext as V, type CompletionVerdict as a, type CorrectnessChecker as b, type Artifact as c, type ArtifactEventLike as d, type ArtifactValidator as e, BackendIntegrityError as f, type ProducedProposal as g, type ProposalEventLike as h, type RequirementCheck as i, type ToolCallEventLike as j, type ValidationIssue as k, type ValidationResult as l, agentProfileHash as m, assertRealBackend as n, byteLengthRange as o, composeValidators as p, containsAll as q, createLlmCorrectnessChecker as r, extractProducedState as s, jsonHasKeys as t, parseCorrectnessResponse as u, regexMatch as v, summarizeBackendIntegrity as w, verifyCompletion as x };
+export { type AgentProfile as A, type BackendIntegrityReport as B, type CompletionRequirement as C, type LlmCorrectnessCheckerOpts as L, type ProducedState as P, type RuntimeEventLike as R, type SatisfiedBy as S, type TaskGold as T, type ValidationContext as V, type CompletionVerdict as a, type CorrectnessChecker as b, type Artifact as c, type ArtifactEventLike as d, type ArtifactValidator as e, BackendIntegrityError as f, type ProducedProposal as g, type ProposalEventLike as h, type RequirementCheck as i, type ToolCallEventLike as j, type ValidationIssue as k, type ValidationResult as l, agentProfileHash as m, assertRealBackend as n, byteLengthRange as o, composeValidators as p, containsAll as q, createLlmCorrectnessChecker as r, createTokenRecallChecker as s, extractProducedState as t, jsonHasKeys as u, parseCorrectnessResponse as v, regexMatch as w, summarizeBackendIntegrity as x, verifyCompletion as y };

package/dist/analyst/index.d.ts

@@ -1,21 +1,21 @@
 import { AxAIService, AxFunction } from '@ax-llm/ax';
 import { M as MultiLayerVerifier, V as VerifyOptions, S as Severity } from '../multi-layer-verifier-DlWCXuxL.js';
 import { c as RunCritic, a as RunTrace } from '../run-critic-BAIjX99r.js';
-import { S as SemanticConceptJudgeOptions, a as SemanticConceptJudgeInput, B as BehavioralMetrics } from '../semantic-concept-judge-Du4ZVyef.js';
-export { C as CreateAnalystAiConfig, D as DEFAULT_TRACE_ANALYST_KINDS, b as DefaultAnalystRegistryOptions, c as DiffPolicy, F as FAILURE_MODE_KIND_SPEC, d as FINDING_SUBJECT_GRAMMAR_PROMPT, e as FINDING_SUBJECT_KINDS, f as FindingSubject, g as FindingSubjectKind, h as FindingSubjectStringSchema, i as FindingsDiff, j as FindingsStore, I as IMPROVEMENT_KIND_SPEC, K as KIND_EXPECTED_SUBJECTS, k as KNOWLEDGE_GAP_KIND_SPEC, l as KNOWLEDGE_POISONING_KIND_SPEC, P as PersistedFinding, m as SKILL_USAGE_ANALYST, n as SkillUsageAnalyst, o as SkillUsageRecord, p as SkillUsageReport, q as SkillUsageScanConfig, r as buildDefaultAnalystRegistry, s as buildSkillUsageReport, t as createAnalystAi, u as defaultIsMaterial, v as diffFindings, w as emitSkillUsageFindings, x as parseFindingSubject, y as renderFindingSubject } from '../semantic-concept-judge-Du4ZVyef.js';
-import { A as AnalyzeTracesOptions } from '../analyst-t7zZS3TV.js';
-import { T as TraceAnalysisStore } from '../store-GmBE2pZZ.js';
+import { S as SemanticConceptJudgeOptions, a as SemanticConceptJudgeInput, B as BehavioralMetrics } from '../semantic-concept-judge-DIEgr_6v.js';
+export { C as CreateAnalystAiConfig, D as DEFAULT_TRACE_ANALYST_KINDS, b as DefaultAnalystRegistryOptions, c as DiffPolicy, F as FAILURE_MODE_KIND_SPEC, d as FINDING_SUBJECT_GRAMMAR_PROMPT, e as FINDING_SUBJECT_KINDS, f as FindingSubject, g as FindingSubjectKind, h as FindingSubjectStringSchema, i as FindingsDiff, j as FindingsStore, I as IMPROVEMENT_KIND_SPEC, K as KIND_EXPECTED_SUBJECTS, k as KNOWLEDGE_GAP_KIND_SPEC, l as KNOWLEDGE_POISONING_KIND_SPEC, P as PersistedFinding, m as SKILL_USAGE_ANALYST, n as SkillUsageAnalyst, o as SkillUsageRecord, p as SkillUsageReport, q as SkillUsageScanConfig, r as buildDefaultAnalystRegistry, s as buildSkillUsageReport, t as createAnalystAi, u as defaultIsMaterial, v as diffFindings, w as emitSkillUsageFindings, x as parseFindingSubject, y as renderFindingSubject } from '../semantic-concept-judge-DIEgr_6v.js';
+import { A as AnalyzeTracesOptions } from '../analyst-C8HHvfJp.js';
+import { T as TraceAnalysisStore } from '../store-C1YxJDEK.js';
 import { b as JudgeFn, a as JudgeInput } from '../types-Croy5h7V.js';
-import { A as Analyst, h as AnalystSeverity, c as AnalystFinding } from '../types-DRvV0zRo.js';
-export { a as AnalystContext, g as AnalystCost, i as AnalystInputKind, j as AnalystRequirements, f as AnalystRunEvent, e as AnalystRunInputs, d as AnalystRunResult, b as AnalystRunSummary, k as ChatCallOpts, C as ChatClient, l as ChatRequest, m as ChatResponse, n as ChatTransport, o as CliBridgeTransportOpts, p as CreateChatClientOpts, D as DirectProviderTransportOpts, E as EvidenceRef, M as MockTransportOpts, R as RouterTransportOpts, S as SandboxSdkTransportOpts, q as computeFindingId, r as createChatClient, s as makeFinding } from '../types-DRvV0zRo.js';
+import { A as Analyst, h as AnalystSeverity, c as AnalystFinding } from '../types-Cu3u_x59.js';
+export { a as AnalystContext, g as AnalystCost, i as AnalystInputKind, j as AnalystRequirements, f as AnalystRunEvent, e as AnalystRunInputs, d as AnalystRunResult, b as AnalystRunSummary, k as ChatCallOpts, C as ChatClient, l as ChatRequest, m as ChatResponse, n as ChatTransport, o as CliBridgeTransportOpts, p as CreateChatClientOpts, D as DirectProviderTransportOpts, E as EvidenceRef, M as MockTransportOpts, R as RouterTransportOpts, S as SandboxSdkTransportOpts, q as computeFindingId, r as createChatClient, s as makeFinding } from '../types-Cu3u_x59.js';
 import { TCloud } from '@tangle-network/tcloud';
-export { A as ANALYST_SEVERITIES, C as CreateTraceAnalystKindOpts, R as RAW_FINDING_SCHEMA_PROMPT, a as RawAnalystFinding, b as RawAnalystFindingSchema, T as TraceAnalystGolden, c as TraceAnalystKindSpec, d as createTraceAnalystKind, p as parseRawFinding, r as renderPriorFindings } from '../kind-factory-DqV2t1Xk.js';
-export { a as AnalystHooks, A as AnalystRegistry, b as AnalystRegistryOptions, B as BudgetPolicy, R as RegistryRunOpts } from '../registry-BmEuU94S.js';
-import { L as LlmClientOptions } from '../llm-client-DbjLfz-K.js';
+export { A as ANALYST_SEVERITIES, C as CreateTraceAnalystKindOpts, R as RAW_FINDING_SCHEMA_PROMPT, a as RawAnalystFinding, b as RawAnalystFindingSchema, T as TraceAnalystGolden, c as TraceAnalystKindSpec, d as createTraceAnalystKind, p as parseRawFinding, r as renderPriorFindings } from '../kind-factory-CVecZZG_.js';
+export { A as AnalystHooks, a as AnalystRegistry, b as AnalystRegistryOptions, B as BudgetPolicy, R as RegistryRunOpts } from '../registry-DrEQ3Luj.js';
+import { L as LlmClientOptions } from '../llm-client-CuUg2Mn3.js';
 import '../schema-m0gsnbt3.js';
 import '../store-CKUAgsJz.js';
 import 'zod';
-import '../run-record-sItO5ftF.js';
+import '../run-record-De9VarXR.js';
 import '../errors-Dwqw-T_m.js';
 import '../raw-provider-sink-C46HDghv.js';
 

package/dist/analyst/index.js

@@ -14,7 +14,7 @@ import {
   diffFindings,
   emitSkillUsageFindings,
   runSemanticConceptJudge
-} from "../chunk-5LVWPNS5.js";
+} from "../chunk-L5G7OUKD.js";
 import {
   ANALYST_SEVERITIES,
   AnalystRegistry,
@@ -41,8 +41,8 @@ import {
   renderPriorFindings,
   stripCodeFences,
   structureFindings
-} from "../chunk-CF67I6QY.js";
-import "../chunk-IHDHUN2X.js";
+} from "../chunk-VIDQF3F5.js";
+import "../chunk-CVVHBFGN.js";
 import {
   analyzeTraces
 } from "../chunk-VUINJM5M.js";

package/dist/{analyst-t7zZS3TV.d.ts → analyst-C8HHvfJp.d.ts}

@@ -1,5 +1,5 @@
 import { AxAIService } from '@ax-llm/ax';
-import { T as TraceAnalysisStore } from './store-GmBE2pZZ.js';
+import { T as TraceAnalysisStore } from './store-C1YxJDEK.js';
 
 interface AnalyzeTracesInput {
     /** The user-facing question. Domain framing belongs here, not in the