@tangle-network/agent-eval 0.216.1 → 0.218.1

package/docs/changes/jev-product-adoption.md

@@ -9,7 +9,7 @@ Native extension metadata is preserved, not projected away; lossy non-JSON metad
 before consumers persist an apparently valid but changed observation. Already-paid receipts remain
 recorded when answer validation fails.
 
-`examples/jev-decision-benchmark.ts` compares caller-defined classifier configurations with the
+The decision-benchmark recipe (its example file is not in this tree) compares caller-defined classifier configurations with the
 existing matrix runner. It separates independent expected labels from model input, retains
 abstention, records complete observations and paid failures, and groups results by configuration.
 It is a recipe, not another optimizer or universal scoring policy.

package/docs/changes/jev-recorded-evidence.md

@@ -21,4 +21,4 @@ No live validation is relaxed. The shared implementation has one question/result
 
 Consumer order: publish the actual package version containing this export, update affected dependency pins/lockfiles, then replace archive-only calls to the live request validator. Do not invent a version or install an unpublished source import. Preserve output-vs-text consistency checks, tenant authorization, served-model policy, receipt ownership and the existing action archive. A historical decoder is not permission to repeat paid work.
 
-`tests/jev-recorded-request.test.ts` writes an observation to disk, reads it twice through the real parsers, rejects contradictory results, and confirms that the live path still rejects historical-only requests. No provider, database, scheduler, migration or new package dependency is added.
+The recorded-request test, which is no longer in this tree, writes an observation to disk, reads it twice through the real parsers, rejects contradictory results, and confirms that the live path still rejects historical-only requests. No provider, database, scheduler, migration or new package dependency is added.

package/docs/design/statistics-decisions.md

@@ -1,6 +1,6 @@
 # Statistics: adoption decisions
 
-This is the standing decision record for `src/statistics.ts` and the estimators layered on it.
+This is the standing decision record for `src/statistics/` and the estimators layered on it.
 It answers three questions: which statistic is trustworthy, where each number comes from, and what a caller is allowed to ask for at 3–10 repetitions per arm.
 The API itself is documented in [`concepts.md`](../concepts.md); this file records *why* each statistic is kept, fixed, or refused.
 
@@ -54,7 +54,7 @@ All are repaired, each with a regression test that fails on the pristine blob an
 | `studentTCdf` / `incompleteBeta` at `df ≤ 100` | defective, **fixed** | keep-and-fix | The continued fraction omits the mandatory symmetry branch (`x < (a+1)/(a+b+2)`, else `1 − I(1−x, b, a)`), so it is evaluated outside its convergence domain for small `|t|` and collapses as `x → 1`. `studentTCdf(0.005, 100)` returns `0.89152130` against a true `0.50198972`, an error of `0.3895`. Through `pairedTTest` at `df = 7`: `t = 0.001` reports `p = 0.15130173` against a true `0.99923002`, and `t = 1e-6` reports `p = 0.00015251`. There is a discontinuity at zero — `t = 1e-8` returns exactly `p = 1.0` because `x ≥ 1` short-circuits. | A perfectly null paired result reports `p < 0.05`. The materially wrong band is `|t| ≲ 0.02` at `df = 2–10`, widening to `|t| ≲ 0.058` near `df = 100`; outside it `pairedTTest` matched `scipy.stats.ttest_rel` to `≤ 1e-6` relative across 166 cases. |
 | `interRaterReliability` | defective, **fixed** | keep-and-fix | The grouping loop iterates judge-major and opens a new item bucket whenever the last holds `judgeScores.length` entries, so each bucket collects consecutive scores from the *same* judge. It measures within-judge spread, not between-judge agreement, and is anti-correlated with the truth. Two identical judges scoring `[0,100]` return **`−0.500000`** where the true α is `+1.0`; two maximally disagreeing judges (`[0,0]` versus `[100,100]`) return **`+1.000000`** where the true α is `−0.5`. Perfect agreement on three items returns `−0.250000`, on four items `+0.650000`. | The result is also unstable in the item count, because bucket boundaries depend on how the item count divides by the judge count. Zero test coverage repo-wide; consumed by `src/pipelines/judge-agreement.ts:73`. |
 | `mannWhitneyU` (NaN input) | defective, **fixed** | keep-and-fix | The tie-grouping loop advances via `while (j < len && combined[j].v === combined[i].v) j++` then `i = j`. `NaN === NaN` is false, so `j` never leaves `i` and the process spins forever. `mannWhitneyU([1,2,3,4,5,6],[NaN,2,3,4,5,6])` did not return within 8 s and blocked the event loop hard enough that a 4 s `setTimeout` never fired. | A single NaN score hangs a campaign rather than failing it. `ranks` is safe only because it uses `i = j + 1`. |
-| `wilcoxonSignedRank` (NaN input) | defective, **fixed** | keep-and-fix | The same construct at the absolute-rank grouping loop, same hang. | Reachable from `held-out-gate.ts:278`, so a NaN in a holdout set hangs the gate. |
+| `wilcoxonSignedRank` (NaN input) | defective, **fixed** | keep-and-fix | The same construct at the absolute-rank grouping loop, same hang. | Reachable from `HeldOutGate.decidePairedDelta` in `held-out-gate.ts`, so a NaN in a holdout set hangs the gate. |
 | `mannWhitneyU` (tie and continuity correction) | defective, **fixed** | keep-and-fix | The variance uses the no-ties formula `n₁n₂(n₁+n₂+1)/12` and there is no continuity correction. `[1,2,3]` versus `[4,5,6]` (zero ties) and `[0,0,0]` versus `[1,1,1]` (maximal ties) both return the byte-identical `p = 0.04953448`. scipy's tie-and-continuity-corrected asymptotic separates them at `0.080856` and `0.046854`. | Both omissions push p downward, so they compound the CDF defect rather than offsetting it. `u` is returned as `min(u1,u2)`, which discards the direction of the effect. |
 | `wilcoxonSignedRank` (tie and continuity correction) | defective, **fixed** | keep-and-fix | The variance uses `n(n+1)(2n+1)/24` with no tie term `−(1/48)Σ(t³−t)` and no continuity correction, despite the function computing average ranks for ties. | Same compounding direction. The statistic convention also differs from scipy without being documented: this returns `W⁺`, scipy returns `min(W⁺, W⁻)`. |
 | `wilcoxonSignedRank` (`n < 6` branch) | defective, **fixed** | **branch deleted** | Line 219 hard-returns `{w: 0, p: 1}` below six non-zero differences, with no exception, no flag, and nothing in the result to distinguish it from a measured null. A clean `+0.5` on all five of five pairs returns `p = 1` where the exact answer is `0.0625`. Because zero deltas are dropped first, ties silently push `n` under the threshold: ten pairs with five zero deltas also returned `p = 1`. | This is the single most consequential open defect at 3–10 reps, and it violates the package's own *no fallbacks, fail loud* rule. It is a false-negative generator precisely in the target regime. |
@@ -63,7 +63,7 @@ All are repaired, each with a regression test that fails on the pristine blob an
 | `pairedTTest` (zero-variance branch) | defective, **fixed** | keep-and-fix | Line 199 returns `p = 0` when the standard error is zero. `pairedTTest([0, 0, 0.5], [0.5, 0.5, 1])` — a constant `+0.5` shift on three pairs — returns `{t: null, df: 2, p: 0}`. That is absolute certainty from three observations where the exact signed-rank floor at `n = 3` is `0.25`, and the object is internally inconsistent (`t` serialized as `null` alongside `p = 0`). | `pairedCohensDz` handles the identical condition correctly by returning `null` and documents why. The module should answer the degenerate case once, the same way, everywhere. |
 | `cohensD` (degenerate branches) | defective, **fixed** | keep-and-fix | Returns a silent `0` twice: when either sample has fewer than two observations, and when the pooled standard deviation is zero. `cohensD([1,1,1],[2,2,2])` returns `0` — "no effect" for a maximal, zero-variance separation. | Third distinct answer to the same degenerate condition, after `pairedTTest`'s `p = 0` and `pairedCohensDz`'s `null`. Only `pairedCohensDz` is right. |
 | `mulberry32` | defective, **fixed** | keep-and-fix | `let s = seed \| 0 \|\| 0x9e3779b9` collapses seed `0` to the golden-ratio constant. `mulberry32(0)` and `mulberry32(0x9e3779b9 \| 0)` both emit `[0.3588899802, 0.1059032613, 0.6752904793]`. | Seed `0` is a common default, so two runs a caller believes are independent replicates are the same run. |
-| `makeRng` / bootstrap seeding | defective, **fixed** | keep-and-fix | `makeRng` returns raw `Math.random` when `opts.seed` is undefined. Two back-to-back `confidenceInterval` calls on identical input returned `lower = 0.30000000000000004` and `lower = 0.3`, `upper = 0.6749999999999999` and `upper = 0.6499999999999999`. Seeding works when supplied (`seed: 7` reproduced exactly). | This contradicts `mulberry32`'s own docstring, which states that a seed is required because unseeded randomness in gate verdicts is non-reproducible by construction. `src/contract/analyze-runs.ts:908` is the live call site that passes no seed; the gates in `held-out-gate.ts`, `promotion-policy.ts`, `statistical-heldout.ts`, `measured-comparison.ts` and `summary-report.ts` all thread one through. |
+| `makeRng` / bootstrap seeding | defective, **fixed** | keep-and-fix | `makeRng` returns raw `Math.random` when `opts.seed` is undefined. Two back-to-back `confidenceInterval` calls on identical input returned `lower = 0.30000000000000004` and `lower = 0.3`, `upper = 0.6749999999999999` and `upper = 0.6499999999999999`. Seeding works when supplied (`seed: 7` reproduced exactly). | This contradicts `mulberry32`'s own docstring, which states that a seed is required because unseeded randomness in gate verdicts is non-reproducible by construction. `src/contract/analyze-runs.ts` is the live call site that passes no seed; the gates in `held-out-gate.ts`, `promotion-policy.ts`, `statistical-heldout.ts`, `measured-comparison.ts` and `summary-report.ts` all thread one through. |
 
 ## Correct mathematics, misleading regime
 
@@ -100,12 +100,12 @@ These 17 matched their references with zero mismatches and stay as they are, pin
 | `cliffsDelta` | keep-with-CI-oracle | Matched a hand-computed `(#gt − #lt)/(n₁n₂)` reference exactly (`0.312500`). Note the orientation is *after over before*, the opposite sign to the textbook `δ` written over `(a, b)`; the docstring states this and the parameter names `(before, after)` carry it. |
 | `pairedCohensDz` | keep-with-CI-oracle | Matched `mean(d)/sd(d)` exactly (`2.184070`). It is the only function in the module that handles the zero-variance degenerate case correctly, returning `null` with a docstring explaining that the standardized effect is undefined rather than an arbitrarily large finite number. Treat it as the reference behaviour the other degenerate branches should adopt. |
 | `weightedMean`, `partialCredit`, `weightedComposite`, `interpretCliffs`, `normalizeScores` | keep-with-CI-oracle | Arithmetic and thresholding, verified by direct evaluation (`weightedMean` of `[{1,w2},{4,w1}]` is `2.000000`; `partialCredit(3,4)` is `0.750000`). |
-| `welchsTTest`, `compareToBaseline` (`src/baseline.ts`) | keep-with-CI-oracle, **covered** | Correct, and previously unguarded: no test file imported either, which is the structural reason a duplicated broken CDF survived here. It is exported publicly and it gates improved / regressed / stable verdicts. Now pinned against `scipy.stats.ttest_ind(equal_var=False)` in the oracle fixture and exercised through `compareToBaseline` in `tests/statistics.test.ts`. |
+| `welchsTTest`, `compareToBaseline` (`src/baseline.ts`) | keep-with-CI-oracle, **covered** | Correct, and previously unguarded: no test file imported either, which is the structural reason a duplicated broken CDF survived here. It is exported publicly and it gates improved / regressed / stable verdicts. Now pinned against `scipy.stats.ttest_ind(equal_var=False)` in the oracle fixture and exercised through `compareToBaseline` in a statistics test file that is no longer in the tree. |
 
 ## Public surface change
 
 `normalCdf` and `studentTCdf` were private on `origin/main` and are now exported with explicit accuracy contracts.
-`baseline.ts` owns the single Welch implementation, and `contract/analyze-runs.ts` consumes that result instead of carrying another normal-approximation copy.
+`baseline.ts` owns the single Welch implementation, and `src/contract/analyze-runs.ts` consumes that result instead of carrying another normal-approximation copy.
 That is the right trade: one implementation with one accuracy contract beats three copies of which two were wrong.
 It also means both functions are now public API and owe callers a stated accuracy bound.
 `normalCdf` is documented at `7.5e-8` absolute.
@@ -225,7 +225,7 @@ Promotion paths now use `pairedDeltaTest`: an exact one-sided sign test carries
 ## Consumer notice
 
 Every published version from **0.1.0 (2026-04-20)** through **0.133.0 (2026-07-27)** reported p-values that are too small from the functions listed below.
-The defect entered at the initial commit (`7d5032b`, `src/statistics.ts:747`) and was present in every release since.
+The defect entered at the initial commit (`7d5032b`, in the then single-file statistics module) and was present in every release since.
 
 The normal CDF itself was corrected in 0.133.1; every other row below was still open at that release.
 
@@ -290,18 +290,18 @@ That is why the option stays optional — a required seed would push the caller
 
 | Site | Formula | Consumer | Seed source |
 | --- | --- | --- | --- |
-| `statistics/internal.ts` `makeRng` | mulberry32 | every resampling path in `src/statistics` | caller seed, else FNV-1a over the IEEE-754 bytes of the observations |
-| `statistics/descriptive.ts` `confidenceInterval` | percentile bootstrap | reported interval | `makeRng(opts.seed, scores)` |
-| `statistics/paired-tests.ts` `pairedBootstrap` | paired percentile bootstrap | promotion intervals | `makeRng(opts.seed, deltas)` |
-| `statistics/rank-tests.ts` `mannWhitneyU`, `wilcoxonSignedRank` | permutation null | exact rank-test p | `makeRng(opts.seed, …)`, order-independent for the symmetric two-sample case |
+| `src/statistics/internal.ts` `makeRng` | mulberry32 | every resampling path in `src/statistics` | caller seed, else FNV-1a over the IEEE-754 bytes of the observations |
+| `src/statistics/descriptive.ts` `confidenceInterval` | percentile bootstrap | reported interval | `makeRng(opts.seed, scores)` |
+| `src/statistics/paired-tests.ts` `pairedBootstrap` | paired percentile bootstrap | promotion intervals | `makeRng(opts.seed, deltas)` |
+| `src/statistics/rank-tests.ts` `mannWhitneyU`, `wilcoxonSignedRank` | permutation null | exact rank-test p | `makeRng(opts.seed, …)`, order-independent for the symmetric two-sample case |
 | `promotion-gate.ts` | bootstrap over the two arms | ship / hold verdict | `options.seed`, else `hashSeed(baseline, candidate)` |
 | `held-out-gate.ts` | delegates to `pairedBootstrap` | held-out gate verdict | caller seed, else the paired observations |
 | `summary-report.ts` `bayesianBootstrapMeanSamples` | Dirichlet-weight bootstrap | reported posterior and gain histogram | `makeRng(seed, deltas)` |
-| `meta-eval/rubric-predictive-validity.ts` `bootstrapCi` | percentile bootstrap of Pearson r | rubric verdict (`load_bearing` / `informative`) | `makeRng(input.seed, xs, ys)` |
-| `meta-eval/correlation-study.ts` `bootstrapPearsonCi` | percentile bootstrap of Pearson r | reported correlation CI | `makeRng(options.seed, xs, ys)` |
-| `campaign/search-policy.ts` `aide` | Thompson draw over node posteriors, the debug coin, the debug and uniform draws | the parent an expansion extends (spends budget, claims nothing) | `mulberry32(seed ^ imul(completed expansions + 1, 0x9e3779b1) ^ salt)`, a pure function of the ledger |
-| `rl/adaptation-eval.ts` `compareAdaptationCurves` | bootstrap of per-k mean deltas | per-k CI and AUC delta | `makeRng(opts.seed, a-means, b-means)` |
-| `campaign/gates/sequential.ts` | seeded Fisher-Yates over paired deltas | exchangeability guard before the e-process | `shuffleSeed`, default 1337, data-independent by construction |
+| `src/meta-eval/rubric-predictive-validity.ts` `bootstrapCi` | percentile bootstrap of Pearson r | rubric verdict (`load_bearing` / `informative`) | `makeRng(input.seed, xs, ys)` |
+| `src/meta-eval/correlation-study.ts` `bootstrapPearsonCi` | percentile bootstrap of Pearson r | reported correlation CI | `makeRng(options.seed, xs, ys)` |
+| `src/campaign/search-policy.ts` `aide` | Thompson draw over node posteriors, the debug coin, the debug and uniform draws | the parent an expansion extends (spends budget, claims nothing) | `mulberry32(seed ^ imul(completed expansions + 1, 0x9e3779b1) ^ salt)`, a pure function of the ledger |
+| `src/rl/adaptation-eval.ts` `compareAdaptationCurves` | bootstrap of per-k mean deltas | per-k CI and AUC delta | `makeRng(opts.seed, a-means, b-means)` |
+| `src/campaign/gates/sequential.ts` | seeded Fisher-Yates over paired deltas | exchangeability guard before the e-process | `shuffleSeed`, default 1337, data-independent by construction |
 
 ### Draws that reach no result
 
@@ -319,9 +319,9 @@ These stay on `Math.random`. Each produces an identifier or a retry delay; no re
 
 ### What changed in this pass
 
-Five byte-identical private copies of mulberry32 existed alongside the canonical one, in `meta-eval/rubric-predictive-validity.ts`, `rl/active-curriculum.ts`, `rl/adaptation-eval.ts`, `summary-report.ts`, and `promotion-gate.ts`.
+Five byte-identical private copies of mulberry32 existed alongside the canonical one, in `src/meta-eval/rubric-predictive-validity.ts`, `src/rl/active-curriculum.ts`, `src/rl/adaptation-eval.ts`, `summary-report.ts`, and `promotion-gate.ts`.
 Four of them fell back to `Math.random` when the caller passed no seed, so four reported statistics were silently non-reproducible.
-`meta-eval/correlation-study.ts` called `Math.random` directly inside its bootstrap and had no seed option at all.
+`src/meta-eval/correlation-study.ts` called `Math.random` directly inside its bootstrap and had no seed option at all.
 All six now route through `makeRng`, and `correlationStudy` gained the `seed` option its sibling already had.
 `Math.random` references in `src/` fell from 19 to 10, and none of the remaining ten is result-bearing.
 
@@ -334,7 +334,7 @@ The audit in #411 counted assertions that prove nothing, module mocks, and condi
 - *A guard's accept case.* `assertLlmRoute`, `assertNoHiddenLeak`, `assertCapabilityHeadroom`, `assertMultishotShotResult`, `assertMatchedMethodLimits`, `assertGateReport`, `assertAnalystBenchmarkObservation`, `assertDeterministicOracle`, `assertDenominatorIntact`, and `assertJsonValue` return nothing. For these the no-throw IS the assertion: it pins the false-positive boundary of a fail-closed gate, and a gate that refuses a legal input is as broken as one that admits an illegal one. Each sits next to a `toThrow` sibling that proves the refusal. Kept.
 - *A value-returning call.* `validateRunRecord` and friends return the validated value, so the no-throw discarded the only thing worth checking. Converted to assert the returned value.
 
-**Module mocks.** 5 sites, each now carrying a one-line justification in its file header: four wrap a `node:` builtin to reproduce a race or an absent native binding that a real filesystem cannot be timed into (`trace/store.test.ts`, `analyst/benchmark-verification-artifacts.test.ts`, `analyst/benchmark-command-public-data.test.ts`, `rollout/readers/opencode-sqlite-lazy.test.ts`); one replaces the provider call in the wire handler test, which is about request validation and error mapping (`tests/wire/handlers.test.ts`).
+**Module mocks.** 5 sites, each now carrying a one-line justification in its file header: four wrap a `node:` builtin to reproduce a race or an absent native binding that a real filesystem cannot be timed into (`src/trace/store.test.ts`, `src/analyst/benchmark-verification-artifacts.test.ts`, `src/analyst/benchmark-command-public-data.test.ts`, the opencode-sqlite lazy-reader test (no longer in the tree)); one replaces the provider call in the wire handler test, which is about request validation and error mapping (`tests/wire/handlers.test.ts`).
 
 **Conditional skips.** 6 sites, all environment-gated, none silent:
 

package/docs/design.md

@@ -31,7 +31,7 @@ The stack context matters only if you adopt more of it later.
 ## The dependency rule
 
 This section is the public rationale.
-The enforceable maintainer rule lives in [`CLAUDE.md`](../CLAUDE.md#dependency-and-evidence-boundaries).
+The enforceable maintainer rule lives in [`AGENTS.md`](../AGENTS.md#dependency-and-evidence-boundaries).
 
 **`agent-eval` has zero upward dependencies on a consumer.**
 This is what keeps the package reusable outside our own stack: nothing in here imports from `agent-runtime`, `agent-knowledge`, or `sandbox`, whether at runtime, in development dependencies, or as type-only imports.

package/docs/eval-surface-map.md

@@ -85,6 +85,7 @@ Use the decision diagnostics and intervals to distinguish insufficient evidence
 | `/rl` | Build reward, preference, and supervised datasets from eligible evidence. |
 | `/profile-cell` | Create and validate portable agent-profile identities. |
 | `/authenticity`, `/ledger-core` | Check evidence authenticity and maintain canonical hash-chained journals. |
+| `/lock-holder` | Decide whether a lock's recorded holder is `dead`, `alive` or `unknown` with `lockHolderVerdict`; record a holder with `readLockHolderIdentity`. It imports only Node built-ins, so any lock owner can use it without loading the rest of the package. |
 | [`/rollout`](./rollout.md), `/storyboard` | Serialize training rows and render recorded work. |
 | `/hosted`, `/wire`, `/adapters/http` | Connect hosted storage or expose evaluation through HTTP and RPC. |
 

package/docs/evaluation-integrity.md

@@ -189,7 +189,7 @@ For outcome associations and direct score calibration, use [the outcome-validity
 
 These integrity checks establish execution and measurement behavior.
 They do not establish that a particular optimizer improves agents across domains.
-The [historical evidence audit](./design/self-improvement-evidence-audit.md) records prior gains, nulls, regressions, and their limits.
+The [historical evidence audit](../archive/2026-09-13-self-improvement-evidence-audit.md) records prior gains, nulls, regressions, and their limits.
 
 For a benefit experiment, define the user behavior and useful effect before search.
 Compare the starting agent, a direct edit or simple search baseline, and the proposed improvement method at equal actual resources.

package/docs/profile-json-identity.md

@@ -26,8 +26,7 @@ contributes to the cell ID, so a source identity join should compare both kind
 and hash. A loose product-specific fingerprint must not claim to be a validated
 canonical AgentProfile merely because it is JSON-serializable.
 
-Regression coverage: `tests/profile-json-canonical.test.ts`, together with the
-existing `tests/agent-profile-cell.test.ts`. Run the repository typecheck, full
+Regression coverage: `tests/agent-profile-cell.test.ts`. Run the repository typecheck, full
 suite, build and package checks before release. This document does not allocate
 or announce a package version; release metadata and downstream peer ranges must
 be updated together by the normal release process.

package/docs/proofs/campaign-attempt-retention-20261001/README.md

@@ -44,8 +44,8 @@ Run [qualify.mjs](artifacts/current/qualify.mjs) with these environment variable
 
 - `PROOF_ROOT`: a new output directory.
 - `BASELINE_ROOT`: the preserved predecessor evidence directory containing `before-resume`.
-- `EVAL_MODULE`: the candidate's built `dist/campaign/index.js`.
-- `EVAL_COST_MODULE`: the candidate's built `dist/index.js`.
+- `EVAL_MODULE`: the candidate's built campaign entry point.
+- `EVAL_COST_MODULE`: the candidate's built package entry point (`main` in `package.json`).
 
 Build, typecheck, and package verification completed successfully.
 No unit tests were added or run.