@tangle-network/agent-eval 0.50.0 → 0.50.1

package/docs/concepts.md

@@ -9,6 +9,26 @@ connected, or the answer lacks required sources. The package gives products a
 shared way to record runs, check outcomes, classify failures, compare variants,
 and make release decisions.
 
+## The three top-level functions
+
+Everything funnels through `/contract`. Three entries, one shape coming back:
+
+| Function | When to call it | What you give it | What you get back |
+|---|---|---|---|
+| **`selfImprove()`** | You have a closed loop — scenarios, judge, agent in hand, and you want the substrate to propose better candidates + gate them. | scenarios, agent, judge, baseline surface | `SelfImproveResult.insight: InsightReport` + ship/hold verdict + winner surface |
+| **`analyzeRuns()`** | You have observed runs (production traces, an approve/reject corpus, a CSV gold set) and want the same rigor packet without invoking an agent. | `RunRecord[]` + optional flags | `InsightReport` |
+| **Intake adapters** (`fromFeedbackTable`, `fromOtelSpans`) | Your data isn't already in `RunRecord` shape — it's in Obsidian, Sheets, an OTel collector, etc. | source-specific input | `RunRecord[]` ready to pipe into `analyzeRuns()` |
+
+The three customer maturity stages — logs only → ratings → closed loop — map exactly to the three functions. See [`customer-journeys.md`](./customer-journeys.md) for the runnable walkthroughs.
+
+The shape of the answer — `InsightReport` — is identical across all three paths. Distributional summary, paired-bootstrap lift CI, judge stats, inter-rater agreement, cost-quality Pareto, failure clusters, contamination check, outcome correlation, release axes, and a ranked recommendations array. Walked through section-by-section in [`insight-report.md`](./insight-report.md).
+
+## The layering rule
+
+`agent-eval` is the **substrate** at the bottom of the Tangle agent stack. `agent-runtime` and `agent-knowledge` depend on it; `agent-eval` MUST NOT import from either. Primitives that "feel like" they belong in a consumer but are actually substrate-shaped (validator verdicts, run records, scenarios, judge scores) live here. Primitives that genuinely require a running agent loop (`ValidationCtx` with iteration + signal + traceEmitter, sandbox `AgentRunSpec`) stay in `agent-runtime`.
+
+The test: *does this concept make sense WITHOUT a running agent loop?* If yes, it's substrate. If no, it's runtime. The full rule is in [`/CLAUDE.md`](../CLAUDE.md#repo-layering--this-package-is-the-substrate).
+
 ## Main Objects
 
 | Thing | What it is | One-line example |

package/docs/customer-journeys.md

@@ -0,0 +1,208 @@
+# Customer journeys
+
+Three end-to-end journeys covering the surface of `@tangle-network/agent-eval`. Each one is a runnable example under `examples/` — clone the repo and `pnpm tsx examples/<journey>/index.ts` to see the actual output.
+
+The three journeys map to three customer-maturity stages:
+
+1. **Logs but no eval discipline** → [Production traces journey](#1-production-traces-journey-customer-otel-traces)
+2. **Ratings but no closed loop** → [Feedback corpus journey](#2-feedback-corpus-journey-customer-feedback-loop)
+3. **Scenarios, judge, agent — full closed loop** → [Closed-loop journey](#3-closed-loop-journey-selfimprove-quickstart)
+
+Each section: what the customer has, what they want, the code, what the report looks like.
+
+---
+
+## 1. Production traces journey — `customer-otel-traces`
+
+**The customer:** an agentic GTM-as-a-service company. Multiple agent steps in prod (social media posting, image generation, translation). OTel observability piped to their collector. Doesn't run formal evals. CTO hand-rolled their tracing.
+
+**The frustration:** "Which step is unreliable? What's our cost-quality profile? Where do we fix next?" They have the data; they don't have the answer.
+
+**What they need from agent-eval:** day-1 analysis of their existing logs. No scenarios, no judges, no closed loop. Just turn the trace stream into a decision packet.
+
+### The code
+
+```ts
+import { analyzeRuns, fromOtelSpans } from '@tangle-network/agent-eval/contract'
+
+const runs = fromOtelSpans({ spans: yourOtelStream })
+const report = await analyzeRuns({ runs })
+
+// report.failureClusters → root causes
+// report.costQuality.pareto → cost-vs-quality scatter
+// report.composite → distribution
+// report.recommendations → top-3 actions
+```
+
+### What the report shows
+
+```
+Runs analyzed:     40
+Composite mean:    0.721 (p50: 0.717, p95: 0.925, stddev: 0.210)
+Cost mean:         $0.103 (p95: $0.131)
+
+── Failures ──
+6 runs with status=ERROR or failureMode set:
+  tool.search  (3x)
+  agent.turn   (3x)
+
+── Cost-quality Pareto ──
+1 candidate(s) plotted; 1 on the frontier
+  otel-default: cost=$0.103 quality=0.721  (frontier)
+
+── Recommendations ──
+[medium] expand-corpus — Mean composite 0.721 has room
+```
+
+### Next steps for this customer
+
+1. Wire an `AnalystRegistry` to cluster the 6 failures by root cause via LLM analysis.
+2. Add `outcomeSignal` once they have downstream conversion / engagement / post-engagement data, and the report fits a reward model showing whether their score predicts the customer outcome.
+3. Once they identify a step worth optimizing (translation, say), graduate to journey #3 — wrap that step in a `Dispatch` and call `selfImprove()`.
+
+**Runnable:** [`examples/customer-otel-traces/`](../examples/customer-otel-traces/)
+
+---
+
+## 2. Feedback corpus journey — `customer-feedback-loop`
+
+**The customer:** a research-validation team. A GitHub Action fires `claude -p` against the next claim, writes the research output to Obsidian. Three reviewers (Alice, Bob, Carol) tag results `#approved` or `#rejected`. Outputs feed a knowledge base. Knowledge feeds content. Content feeds engagement. The founder wants more engagement faster.
+
+**The frustration:** "We disagree on what's good. We don't know if our 'good' actually drives engagement. Reviewing every claim is slow."
+
+**What they need from agent-eval:** turn the approve/reject corpus into actionable signal:
+- Where do reviewers disagree? (triage list)
+- Can we synthesize each reviewer's taste into an LLM judge? (auto-grade)
+- Does the taste actually predict downstream engagement? (close the loop)
+
+### The code
+
+```ts
+import { analyzeRuns, fromFeedbackTable } from '@tangle-network/agent-eval/contract'
+
+// 1. Parse Obsidian #approved / #rejected tags into a flat table:
+const ratings = parseObsidianVault('./research-vault')
+// [{ runId: 'claim-1', rater: 'alice', rating: true }, ...]
+
+// 2. Pipe through the adapter:
+const { runs, raterScores } = fromFeedbackTable({ ratings })
+
+// 3. Analyze:
+const report = await analyzeRuns({
+  runs,
+  raterScores,
+  // Optional: close the loop with engagement data once you have it.
+  outcomeSignal: { metric: 'engagement_rate', valueByRunId: enrichedFromProd },
+})
+
+// report.interRater.disagreementCases → top 20 claims worth a meeting
+// report.outcomeCorrelation → does team taste predict engagement?
+// report.recommendations → action list
+```
+
+### What the report shows
+
+```
+Runs analyzed:     30
+Composite mean:    0.756 (approve rate ~76%)
+
+── Inter-rater agreement ──
+Raters:               3 (alice, bob, carol)
+Jointly rated runs:   30
+Pairwise pearson κ:
+  alice::bob     0.53
+  alice::carol   0.55
+  bob::carol     0.21
+Mean κ:               0.43
+
+── Top 5 disagreement cases ──
+  claim-1   range=1.00  ratings: alice=0, bob=0, carol=1
+  claim-7   range=1.00  ratings: alice=0, bob=1, carol=0
+  ...
+
+── Recommendations ──
+[high] recalibrate — Inter-rater agreement κ=0.43 is below 0.5
+  Raters disagree on what 'good' looks like. Refine the rubric or triage the disagreement cases.
+```
+
+### Next steps for this customer
+
+1. **Triage meeting on the disagreement cases.** Mean κ=0.43 means the rubric is ambiguous; clarify it on the cases that split.
+2. **Calibrate one LLM judge per reviewer.** Each reviewer's history is the gold signal — substrate primitive `calibrateJudge` against `raterScores` filtered to that reviewer.
+3. **Add engagement as `outcomeSignal`** once the content downstream is instrumented. The `outcomeCorrelation` section tells the team whether their taste predicts the founder's token-max goal — and if not, the linear reward model says how to retarget.
+4. **Graduate to journey #3** — wrap the research-generation Claude-P call as a `Dispatch`, use the calibrated judges, run `selfImprove()` nightly. Open a PR against the GitHub Action when the holdout approval rate beats baseline.
+
+**Runnable:** [`examples/customer-feedback-loop/`](../examples/customer-feedback-loop/)
+
+---
+
+## 3. Closed-loop journey — `selfimprove-quickstart`
+
+**The customer:** a team with a scenario corpus, a judge, and an agent. Wants to improve the prompt under statistical confidence — propose better candidates, gate on holdout lift, ship the winner.
+
+**The frustration:** "We can run an A/B by hand but we don't know if the improvement is real. We don't have time to run paired bootstrap by hand. We want a function that decides."
+
+**What they need from agent-eval:** the closed loop in one function — propose, score, gate, ship — with the full rigor packet on the way out.
+
+### The code
+
+```ts
+import { selfImprove } from '@tangle-network/agent-eval/contract'
+
+const result = await selfImprove({
+  scenarios,
+  agent: async (surface, scenario) =>
+    await myAgent.run({ systemPrompt: (surface as { systemPrompt: string }).systemPrompt, scenario }),
+  judge: {
+    name: 'rubric',
+    dimensions: [{ key: 'clarity', weight: 1 }, { key: 'concision', weight: 1 }],
+    score: async ({ artifact }) => myJudgeFn(artifact),
+  },
+  baselineSurface: { kind: 'prompt', systemPrompt: 'You write marketing copy...' },
+  budget: { generations: 3, populationSize: 2 },
+})
+
+result.gateDecision   // 'ship' | 'hold' | ...
+result.insight        // full decision packet
+```
+
+### What the report shows
+
+```
+═══ selfImprove() decision packet ═══
+
+Gate decision:        ship
+Raw lift:             +0.194
+
+── Statistical lift (paired bootstrap) ──
+delta:    +0.254
+CI95:     [0.254, 0.254]
+pValue:   1.0000
+Cohen's d: 0.00
+MDE @ 80% power: 2.802
+required n at observed effect: 244
+
+── Recommendations ──
+[critical] ship — Ship — lift 0.254 (95% CI 0.254..0.254)
+```
+
+### Next steps for this customer
+
+1. **Ship the winner.** Either accept `result.winner.surface` programmatically and roll it out, or pass `autoOnPromote: 'pr'` + a GitHub repo to have selfImprove open a PR for you.
+2. **Wire `hostedTenant`** to ship the decision packet to a dashboard (the hosted Intelligence orchestrator, or your own implementation of the wire spec).
+3. **Add `canaryScenarios`** to guard against the holdout leaking into the candidate prompt.
+4. **Add `outcomeSignal`** in `analyzeRuns()` for any post-deploy reruns to verify the predicted lift actually shows up in real outcomes.
+
+**Runnable:** [`examples/selfimprove-quickstart/`](../examples/selfimprove-quickstart/)
+
+---
+
+## How the three journeys compose
+
+Journey #1 + #2 + #3 are **maturity stages**, not exclusive products. A team typically:
+
+1. Starts with **#1** (analyze production logs) to find what's broken.
+2. Adds **#2** (feedback corpus) once they have a sense of where to improve, to calibrate what "good" means.
+3. Graduates to **#3** (closed loop) once they have scenarios + judges, to automate the improvement.
+
+Same substrate, same `InsightReport` shape, no rip-and-replace between stages. The data you collect in #1 informs the scenarios you derive in #2 which feed the loop in #3.

package/docs/insight-report.md

@@ -0,0 +1,337 @@
+# `InsightReport` — the decision packet
+
+The single shape every analysis call returns. `selfImprove()` embeds it in `SelfImproveResult.insight`; `analyzeRuns()` returns it directly. The hosted-tier wire format carries it on `EvalRunEvent.insightReport?`.
+
+Every section is **opt-in based on what your data supports** — the function never invents signal. If your runs don't carry judge scores, `judges` is empty. If there's no baseline/candidate split, `lift` is undefined. The shape is consistent; population is honest.
+
+This page walks every section with a real (synthetic) example and explains how to act on it.
+
+---
+
+## At a glance
+
+```ts
+interface InsightReport {
+  n: number                              // runs analyzed
+  composite: ScalarDistribution          // always
+  perDimension: Record<string, ScalarDistribution>   // when judgeScores carry dimensions
+  costQuality: { cost: ScalarDistribution; pareto: ParetoFigureSpec }   // always
+  judges: Record<string, JudgeInsight>   // when runs carry judge scores
+  interRater?: InterRaterInsight         // when raterScores supplied
+  lift?: LiftInsight                     // when baseline + candidate present
+  failureClusters?: FailureClusterInsight    // when AnalystRegistry wired
+  contamination?: ContaminationInsight   // when canaryScenarios supplied
+  outcomeCorrelation?: OutcomeCorrelationInsight   // when outcomeSignal supplied
+  release: ReleaseSummary                // always
+  recommendations: Recommendation[]      // always — read this FIRST
+}
+```
+
+---
+
+## `n` + `composite` + `perDimension` — distributional summary
+
+Always present. The basic "where are my numbers" view.
+
+```jsonc
+{
+  "n": 30,
+  "composite": {
+    "n": 30,
+    "mean": 0.683, "p50": 0.667, "p95": 1.000, "stddev": 0.231,
+    "min": 0.0, "max": 1.0,
+    "histogram": [
+      { "lo": 0.0,  "hi": 0.083, "count": 5 },
+      { "lo": 0.083, "hi": 0.167, "count": 0 },
+      // ...12 bins by default
+    ]
+  },
+  "perDimension": {
+    "clarity":   { "mean": 0.72, "p50": 0.75, "p95": 0.95, "stddev": 0.18, /* ... */ },
+    "concision": { "mean": 0.65, "p50": 0.68, "p95": 0.88, "stddev": 0.21, /* ... */ }
+  }
+}
+```
+
+**Read first:** the `composite.mean`. If it's < 0.5, your agent has a ceiling problem, not a tuning problem.
+
+**Read next:** `perDimension`. If `clarity` is high but `concision` is low, your prompts get the right ideas in too many words — different fix than "wrong ideas."
+
+**Use the histogram for:** finding bimodal failure modes. A bin with `count > 0` near zero and another > 0 near 1 means your agent has two distinct behaviors, not one noisy one.
+
+---
+
+## `costQuality` — cost-vs-quality Pareto
+
+Always present. `cost.histogram` is the per-run cost distribution; `pareto` is the substrate's `ParetoFigureSpec`.
+
+```jsonc
+{
+  "costQuality": {
+    "cost": {
+      "mean": 0.024, "p95": 0.041,
+      "histogram": [/* */]
+    },
+    "pareto": {
+      "kind": "pareto-cost-quality",
+      "split": "holdout",
+      "axes": { "x": "costUsd", "y": "score" },
+      "points": [
+        { "candidateId": "baseline", "cost": 0.018, "quality": 0.58, "n": 20, "onFrontier": true },
+        { "candidateId": "winner",   "cost": 0.027, "quality": 0.65, "n": 20, "onFrontier": true }
+      ]
+    }
+  }
+}
+```
+
+**Use this when:** comparing prompts, models, or candidate surfaces. The Pareto frontier is your menu of "best you can do at each cost level."
+
+**Render with:** any chart library — `points` is plain JSON. Hosted-tier dashboards render this as a scatter with the frontier highlighted.
+
+---
+
+## `judges` — per-judge mean
+
+Populated when run records carry `outcome.judgeScores`.
+
+```jsonc
+{
+  "judges": {
+    "domain-expert":   { "n": 30, "meanScore": 0.71 },
+    "helpfulness-llm": { "n": 30, "meanScore": 0.62 }
+  }
+}
+```
+
+The substrate's full judge-calibration suite (positional bias, self-preference, verbosity bias) lives in `/reporting` and operates on **paired-by-condition** inputs that `analyzeRuns` doesn't synthesize from raw `RunRecord[]`. Wire them yourself when you have the paired data; the report's `judges` map is the corpus-level slice.
+
+**Use this when:** comparing multiple judges over the same corpus. A big gap between two judges' means is the first signal that one of them is mis-calibrated.
+
+---
+
+## `interRater` — multi-rater agreement + disagreement triage
+
+Populated when `analyzeRuns({ raterScores })` is supplied — typically via `fromFeedbackTable()`.
+
+```jsonc
+{
+  "interRater": {
+    "raters": 3,
+    "jointlyRated": 30,
+    "kappa": 0.71,
+    "perPair": {
+      "alice::bob":   0.78,
+      "alice::carol": 0.65,
+      "bob::carol":   0.69
+    },
+    "disagreementCases": [
+      { "runId": "claim-7", "range": 1.00,
+        "ratings": [{"rater":"alice","score":1},{"rater":"bob","score":1},{"rater":"carol","score":0}] },
+      { "runId": "claim-13", "range": 1.00,
+        "ratings": [{"rater":"alice","score":0},{"rater":"bob","score":0},{"rater":"carol","score":1}] }
+      // ...top 20 by range
+    ]
+  }
+}
+```
+
+**Read first:** the mean `kappa`. < 0.5 means raters disagree on what "good" looks like — surface the disagreement cases at the next review meeting.
+
+**Use this when:** building per-rater LLM judges. Each rater's individual scores are the gold signal you calibrate against. Once a calibrated LLM matches the human ≥85%, you can auto-grade and escalate only the disagreement cases.
+
+---
+
+## `lift` — paired-bootstrap statistical lift
+
+Populated when baseline + candidate candidates are present (auto-detected from two distinct `candidateId`s, or explicit via `baselineCandidateId` + `candidateCandidateId`).
+
+```jsonc
+{
+  "lift": {
+    "baselineMean": 0.58,
+    "candidateMean": 0.65,
+    "delta": 0.07,
+    "ci95": [0.04, 0.10],          // bootstrap CI on the delta
+    "pValue": 0.0008,              // paired t-test
+    "n": 40,                       // paired observations
+    "cohensD": 0.41,
+    "mde": 0.06,                   // min detectable effect at current n, 80% power
+    "requiredN": 38                // n needed for observed delta at 80% power
+  }
+}
+```
+
+**Decision rule:**
+- `ci95[0] > threshold` → **SHIP.** Lower bound above your delta threshold means the lift is real at 95% confidence.
+- `ci95[0] ≤ threshold < ci95[1]` → **INCONCLUSIVE.** Expand the corpus or wait for more data.
+- `ci95[1] ≤ threshold` → **HOLD.** No evidence the candidate is better.
+
+The `recommendations` array surfaces exactly this decision (`kind: 'ship' | 'hold' | 'expand-corpus'`) — that's what consumers should read.
+
+**Why bootstrap, not t-test alone:** paired bootstrap is distribution-free. Your judge scores are bounded in [0,1] and almost never normal; the bootstrap CI is the honest one.
+
+---
+
+## `failureClusters` — grouped failure modes
+
+Populated when an `AnalystRegistry` is passed via `analyzeRuns({ analyst })`. The substrate runs each failed run through the registered analysts and groups findings by `analyst_id` / `area`.
+
+```jsonc
+{
+  "failureClusters": {
+    "totalFailures": 11,
+    "clusters": [
+      { "id": "off-topic-drift", "name": "off-topic-drift",
+        "share": 0.45, "exemplars": ["run-12", "run-19", "run-33"] },
+      { "id": "over-confidence", "name": "over-confidence",
+        "share": 0.27, "exemplars": ["run-3", "run-21"] },
+      { "id": "format-mismatch", "name": "format-mismatch",
+        "share": 0.18, "exemplars": ["run-41", "run-44"] }
+    ]
+  }
+}
+```
+
+**Read first:** the top cluster's `share`. If one cluster is > 40% of failures, fix that pattern before doing anything else.
+
+**Use this when:** triaging a regression. Failure clusters tell you "fix this kind of thing first."
+
+**To wire it:** register analysts in `AnalystRegistry`. See `src/analyst/registry.ts` and `src/analyst/kinds.ts` for the four built-in kinds (`failure-mode`, `improvement`, `knowledge-gap`, `knowledge-poisoning`).
+
+---
+
+## `contamination` — canary check
+
+Populated when canary scenarios are passed via `analyzeRuns({ canaryScenarios })`. Each canary carries a sentinel string the agent should never emit; the report counts leaks.
+
+```jsonc
+{
+  "contamination": {
+    "leaks": 0,
+    "holdoutAuditPassed": true,
+    "details": []
+  }
+}
+```
+
+When `leaks > 0`:
+
+```jsonc
+{
+  "contamination": {
+    "leaks": 2,
+    "holdoutAuditPassed": false,
+    "details": [
+      { "runId": "run-12", "canary": "xyz-secret-canary-123", "matched": "...the secret xyz-secret-canary-123 says..." }
+    ]
+  }
+}
+```
+
+**When this fails:** your holdout corpus has leaked into training context. The `lift` number is **unreliable**. Investigate before shipping anything.
+
+---
+
+## `outcomeCorrelation` — closing the loop on real outcomes
+
+Populated when `outcomeSignal: { metric, valueByRunId }` is supplied.
+
+```jsonc
+{
+  "outcomeCorrelation": {
+    "metric": "engagement_rate",
+    "n": 80,
+    "pearson": 0.72,           // linear correlation
+    "spearman": 0.69,          // rank correlation (robust to monotonic nonlinearity)
+    "rewardModel": {
+      "intercept": 0.04,
+      "slope": 1.93,
+      "r2": 0.52               // share of outcome variance the judge explains
+    }
+  }
+}
+```
+
+This is the layer that says **"does my judge's taste actually predict the metric the business cares about?"**
+
+**Read first:** `spearman`. If it's < 0.3 in absolute value, your judges are scoring something different from what wins downstream. Refit the judges (use the customer's downstream signal as gold) or change the rubric.
+
+**The reward model** is the simple linear `y = intercept + slope * composite`. Use it to:
+- Predict the engagement of a new run from its composite score alone.
+- Set a `composite` threshold for "must beat X to ship" based on the engagement equivalent.
+
+---
+
+## `release` — pass/warn/fail axes
+
+Always present. Roll-up across three axes — quality lift, contamination, composite distribution.
+
+```jsonc
+{
+  "release": {
+    "status": "pass",
+    "axes": [
+      { "name": "quality-lift", "status": "pass",
+        "detail": "delta=0.070, CI95=[0.040, 0.100], n=40" },
+      { "name": "contamination", "status": "pass",
+        "detail": "0 canary leak(s)" },
+      { "name": "composite-distribution", "status": "pass",
+        "detail": "mean=0.683, p50=0.667, p95=1.000 over n=30" }
+    ],
+    "issues": []
+  }
+}
+```
+
+Overall `status` is `fail` if any axis fails; `warn` if any warn; `pass` otherwise.
+
+**Use this when:** wiring agent-eval into CI. A `status === 'pass'` from `analyzeRuns` on the candidate vs baseline is your green-light gate.
+
+---
+
+## `recommendations` — the actionable layer
+
+Always present. Read this first.
+
+```jsonc
+{
+  "recommendations": [
+    { "priority": "critical", "kind": "ship",
+      "title": "Ship — lift 0.070 (95% CI 0.040..0.100)",
+      "detail": "Holdout lift exceeds threshold 0.02 with 95% bootstrap confidence (n=40, p=0.0008, d=0.41).",
+      "evidencePath": "lift" },
+    { "priority": "high", "kind": "investigate",
+      "title": "Top failure cluster: off-topic-drift (45% of failures)",
+      "detail": "11 runs failed. The largest cluster groups 3 exemplars under 'off-topic-drift'.",
+      "evidencePath": "failureClusters.clusters[0]" }
+  ]
+}
+```
+
+| `kind` | When emitted |
+|---|---|
+| `ship` | lift CI lower bound > threshold |
+| `hold` | lift CI upper bound ≤ threshold |
+| `expand-corpus` | lift CI straddles threshold — more data needed |
+| `fix` | canary contamination detected |
+| `recalibrate` | inter-rater κ < 0.5, OR outcome correlation < 0.3 |
+| `investigate` | top failure cluster > some-share |
+
+`evidencePath` points back into the report (`"lift"`, `"contamination"`, `"failureClusters.clusters[0]"`) so a UI can deep-link from each recommendation to its evidence.
+
+---
+
+## How `analyzeRuns` populates each section
+
+| Section | Required input |
+|---|---|
+| `composite`, `perDimension`, `costQuality`, `release`, `recommendations` | `runs` |
+| `judges` | `runs` with `outcome.judgeScores` |
+| `interRater` | `raterScores` (≥ 2 raters jointly rated some runs) |
+| `lift` | two distinct `candidateId`s in `runs` (or explicit baseline/candidate ids) |
+| `failureClusters` | `analyst` registry passed in |
+| `contamination` | `canaryScenarios` passed in |
+| `outcomeCorrelation` | `outcomeSignal` passed in |
+
+All sections beyond the always-present ones are `T | undefined`, never empty objects. If a section is missing, your inputs didn't support it — the report is honest about that.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tangle-network/agent-eval",
-  "version": "0.50.0",
+  "version": "0.50.1",
   "description": "Substrate for self-improving agents: traces, verifiable rewards, preferences, GEPA / reflective mutation, auto-research, replay, sequential anytime-valid stats, and release gates.",
   "homepage": "https://github.com/tangle-network/agent-eval#readme",
   "repository": {