@tangle-network/agent-eval 0.145.19 → 0.145.20

package/CHANGELOG.md

@@ -8,6 +8,33 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
 
 ---
 
+## [0.145.20] — 2026-08-16
+
+### Fixed
+
+- A matrix cell that spent money and then threw recorded `costUsd: 0` and `durationMs: 0`. That spend left the cumulative sum `costCeiling` reads, so a run kept scheduling cells after real spend passed the ceiling. A failed cell is now billed for the spend it declares.
+- `runMultishot` lost the cost of every driver attempt that billed and returned empty content. Those calls are now charged as they happen, so `MultishotDriverEmptyError` no longer discards the whole conversation's spend.
+- A cell that returns a non-finite or negative `costUsd` no longer disables the ceiling for the rest of the run. `NaN` in the cumulative sum makes `>= costCeiling` false forever. Such a result now fails its own cell and the ceiling stays enforceable. An unusable `durationMs` reaches only `meanDurationMs`, so it is recorded as 0 with a warning and the cell keeps its verdict and its cost. A wall clock that steps back mid-cell makes an elapsed time negative, so this is a reachable state.
+- A shot whose call reported neither a cost nor usage priced that call at 0 and still reported the total as a complete estimate. `runMultishot` now reports `costProvenance` on success, so the cell records `uncaptured` and the run counts it. The error path already reported this; the success path did not.
+- `withCellSpend` lost the spend and replaced the original error when the thrown value was frozen, sealed, or otherwise non-extensible. Such a value is now wrapped in an `Error` that carries the spend and keeps the original as `cause`.
+
+### Added
+
+- `withCellSpend(error, spend)` and `readCellSpend(error)` on `@tangle-network/agent-eval/matrix`. A `runCell` implementation that spends before it throws declares that spend with `throw withCellSpend(err, { costUsd, durationMs, kind })`. The carrier is read structurally, so a throw that crosses a package boundary still bills.
+- `CellResult.costProvenance` names the origin of `costUsd`: `observed`, `estimated`, or `uncaptured`. A failure that declares no spend records `uncaptured`, which is distinct from a measured zero.
+- `MatrixResult.summary.costUncapturedCells` and `AxisSummary.costUncapturedCells` count the cells whose cost is a subtotal. Above 0, `totalCostUsd` is a floor on real spend, not the total.
+- `RunAgentMatrixOptions.maxCellCostUsd` bounds what one cell can spend. A cell whose cost is a subtotal is charged that bound against `costCeiling` instead of its known amount, so spend a cell hid cannot walk the run past its budget. It changes only what the ceiling reads: `CellResult.costUsd` and `summary.totalCostUsd` keep reporting known spend. `runMultishotMatrix` forwards the option.
+- `MatrixResult.summary.ceilingChargedUsd` reports the figure the ceiling read. It exceeds `totalCostUsd` only when `maxCellCostUsd` charged a bound.
+- `MultishotResult.costProvenance` lets a shot declare whether its `costUsd` is a total or a subtotal. It is optional, so an engine written before this field behaves as before. `assertMultishotShotResult` validates it when present and rejects an `uncaptured` provenance that carries a number.
+
+### Changed
+
+- The multishot cell declares the shot's own subtotal plus settled judge cost when it fails after the shot returns, and reports `costProvenance: uncaptured` when a judge cost was never reported. A shot that declares no spend is rethrown untouched, so the cell records as uncaptured rather than claiming a fabricated total.
+- `runMultishotMatrix` writes `costUncapturedCells` and `ceilingChargedUsd` into `summary.json`. `summary.md` labels the cost line "Cost (at least)" and carries a caveat line when any cell reported a subtotal, so a reader of the on-disk report cannot mistake the figure for the run's whole spend.
+- Refreshed the analyst benchmark dependency-lock hash for this version.
+
+---
+
 ## [0.145.19] — 2026-08-16
 
 ### Added

package/dist/analyst/index.d.ts

@@ -1299,7 +1299,7 @@ declare const ANALYST_BENCHMARK_HELP = "agent-eval analyst-benchmark\n\nRun the
 declare const ANALYST_BENCHMARK_IMPLEMENTATION_DIGEST_ALGORITHM = "sha256-canonical-source-manifest";
 declare const ANALYST_BENCHMARK_DEPENDENCY_LOCK_DIGEST_ALGORITHM = "sha256-canonical-file-manifest";
 declare const ANALYST_BENCHMARK_DEPENDENCY_LOCK_FILES: readonly string[];
-declare const ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256 = "d12f8c74a24183901623206f98a97bcf3d5c0ed4eb0d74ab09903d578605696e";
+declare const ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256 = "aa81bb8a9b2eca793a117fafbaf6d74d77402ad7f9fc197f7928a35e131984e6";
 /** The published benchmark evidence was produced at this package version, by
  * the retired one-shot direct runner, before trace analysts moved to the
  * recursive DSPy RLM engine. Both evidence digests below are historical facts

package/dist/analyst/index.js

@@ -7,7 +7,7 @@ import { a as assertExactRegistryRunOpts, c as KNOWLEDGE_GAP_KIND_SPEC, d as CON
 import { t as createDspyRlmTraceEngine } from "../dspy-rlm-engine-CqwhQQBw.js";
 import { a as diffFindings, i as defaultIsMaterial, n as runSemanticConceptJudge, r as FindingsStore, t as SEMANTIC_CONCEPT_JUDGE_VERSION } from "../semantic-concept-judge-C6M-qOeb.js";
 import { a as scoreAnalystFindings, i as summarizeAnalystBenchmarkRunner, n as runAnalystBenchmark, r as traceStoreEvidenceResolver, t as registryBenchmarkRunner } from "../benchmark-BhT16ep9.js";
-import { $ as analystBenchmarkDependencyLockDigest, A as compareAnalystRunners, B as DEFAULT_MAX_VERIFICATION_ARTIFACT_BYTES, C as analystDefinitionProtocolSha256, D as readAnalystBenchmarkArtifact, E as expandCodeTraceFailureBlocks, F as MAX_INCORRECT_BLOCKS, G as ANALYST_BENCHMARK_DEPENDENCY_LOCK_FILES, H as loadCodeTraceVerificationArtifacts, I as MAX_INCORRECT_BLOCK_STEPS, J as ANALYST_BENCHMARK_EVIDENCE_IMPLEMENTATION_SHA256, K as ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256, L as publicBenchmarkProtocolSha256, M as effectiveAnalystProtocolSha256, N as readAnalystInstructionsOverride, O as renderCodeTraceCalibrationMarkdown, P as CODE_TRACE_BENCH_ANALYST_PROMPT, Q as ANALYST_BENCHMARK_IMPLEMENTATION_SHA256, R as publicBenchmarkRlmInstructions, S as analystDefinitionAsymmetries, T as emptyPublicBenchmarkRunner, U as parseVerificationOutcome, V as appendVerificationArtifactsToOtlp, W as ANALYST_BENCHMARK_DEPENDENCY_LOCK_DIGEST_ALGORITHM, X as ANALYST_BENCHMARK_IMPLEMENTATION_DIGEST_ALGORITHM, Y as ANALYST_BENCHMARK_EVIDENCE_PACKAGE_VERSION, Z as ANALYST_BENCHMARK_IMPLEMENTATION_FILES, _ as runReplVariableAnalystDefinition, a as primeAnalystProtocolSha256, at as AGENT_RX_UPSTREAM_REVISION, b as runChunkedAnalystDefinition, c as nodeHttpPrimeBridgeTransport, ct as codeTraceBenchCase, d as publicBenchmarkDistributions, dt as agentRxPredictionsToFindings, et as analystBenchmarkImplementationDigest, f as publicBenchmarkSelectionReport, ft as normalizeAgentRxCategory, g as rlmEngineLimits, h as publicRlmAnalystDefinition, i as createPrimeBenchmarkRunner, it as ANALYST_BENCHMARK_OBSERVATIONS_FILE, j as analystInstructionsOverrideFromText, k as summarizeCodeTraceCalibration, l as loadPublicBenchmarkRows, lt as codeTracerPredictionsToFindings, m as createPublicBenchmarkRlmRunner, mt as normalizeBenchmarkLabel, n as runAnalystBenchmarkCommand, nt as ANALYST_BENCHMARK_LOCAL_RECEIPT_FILE, o as primeCodeTraceAnalystDefinition, ot as renderAgentRxCalibrationMarkdown, p as selectPublicBenchmarkRows, pt as roundAgentRxStep, q as ANALYST_BENCHMARK_EVIDENCE_DEPENDENCY_LOCK_SHA256, r as renderAnalystBenchmarkMarkdown, rt as ANALYST_BENCHMARK_MANIFEST_FILE, s as runInlineAnalystDefinition, st as summarizeAgentRxCalibration, t as ANALYST_BENCHMARK_HELP, tt as ANALYST_BENCHMARK_COST_LEDGER_FILE, u as preparePublicAnalystBenchmark, ut as agentRxBenchmarkCase, v as createPublicBenchmarkDirectRunner, w as adaptPublicBenchmarkFindings, x as AnalystExpressivenessError, y as publicDirectAnalystDefinition, z as publicBenchmarkSystemPrompt } from "../benchmark-command-Cq7sAba4.js";
+import { $ as analystBenchmarkDependencyLockDigest, A as compareAnalystRunners, B as DEFAULT_MAX_VERIFICATION_ARTIFACT_BYTES, C as analystDefinitionProtocolSha256, D as readAnalystBenchmarkArtifact, E as expandCodeTraceFailureBlocks, F as MAX_INCORRECT_BLOCKS, G as ANALYST_BENCHMARK_DEPENDENCY_LOCK_FILES, H as loadCodeTraceVerificationArtifacts, I as MAX_INCORRECT_BLOCK_STEPS, J as ANALYST_BENCHMARK_EVIDENCE_IMPLEMENTATION_SHA256, K as ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256, L as publicBenchmarkProtocolSha256, M as effectiveAnalystProtocolSha256, N as readAnalystInstructionsOverride, O as renderCodeTraceCalibrationMarkdown, P as CODE_TRACE_BENCH_ANALYST_PROMPT, Q as ANALYST_BENCHMARK_IMPLEMENTATION_SHA256, R as publicBenchmarkRlmInstructions, S as analystDefinitionAsymmetries, T as emptyPublicBenchmarkRunner, U as parseVerificationOutcome, V as appendVerificationArtifactsToOtlp, W as ANALYST_BENCHMARK_DEPENDENCY_LOCK_DIGEST_ALGORITHM, X as ANALYST_BENCHMARK_IMPLEMENTATION_DIGEST_ALGORITHM, Y as ANALYST_BENCHMARK_EVIDENCE_PACKAGE_VERSION, Z as ANALYST_BENCHMARK_IMPLEMENTATION_FILES, _ as runReplVariableAnalystDefinition, a as primeAnalystProtocolSha256, at as AGENT_RX_UPSTREAM_REVISION, b as runChunkedAnalystDefinition, c as nodeHttpPrimeBridgeTransport, ct as codeTraceBenchCase, d as publicBenchmarkDistributions, dt as agentRxPredictionsToFindings, et as analystBenchmarkImplementationDigest, f as publicBenchmarkSelectionReport, ft as normalizeAgentRxCategory, g as rlmEngineLimits, h as publicRlmAnalystDefinition, i as createPrimeBenchmarkRunner, it as ANALYST_BENCHMARK_OBSERVATIONS_FILE, j as analystInstructionsOverrideFromText, k as summarizeCodeTraceCalibration, l as loadPublicBenchmarkRows, lt as codeTracerPredictionsToFindings, m as createPublicBenchmarkRlmRunner, mt as normalizeBenchmarkLabel, n as runAnalystBenchmarkCommand, nt as ANALYST_BENCHMARK_LOCAL_RECEIPT_FILE, o as primeCodeTraceAnalystDefinition, ot as renderAgentRxCalibrationMarkdown, p as selectPublicBenchmarkRows, pt as roundAgentRxStep, q as ANALYST_BENCHMARK_EVIDENCE_DEPENDENCY_LOCK_SHA256, r as renderAnalystBenchmarkMarkdown, rt as ANALYST_BENCHMARK_MANIFEST_FILE, s as runInlineAnalystDefinition, st as summarizeAgentRxCalibration, t as ANALYST_BENCHMARK_HELP, tt as ANALYST_BENCHMARK_COST_LEDGER_FILE, u as preparePublicAnalystBenchmark, ut as agentRxBenchmarkCase, v as createPublicBenchmarkDirectRunner, w as adaptPublicBenchmarkFindings, x as AnalystExpressivenessError, y as publicDirectAnalystDefinition, z as publicBenchmarkSystemPrompt } from "../benchmark-command-CkgLXHoN.js";
 import { a as extractPrimeJsonObject, c as primeProtocolSha256, d as runPrimeExchange, f as decodeReplyRows, i as emptyPrimeRawUsage, l as primeReplyDefect, n as buildPrimePrompt, o as mergePrimeRawUsage, r as buildPrimeRepairPrompt, s as normalizePrimeUsage, t as analystUsageReceiptFromPrimeUsage, u as projectPrimeTrajectory } from "../prime-protocol-6tZTVsWm.js";
 import { existsSync, readFileSync, readdirSync, statSync } from "node:fs";
 import { join } from "node:path";

package/dist/{benchmark-command-Cq7sAba4.js → benchmark-command-CkgLXHoN.js}

@@ -1209,7 +1209,7 @@ const ANALYST_BENCHMARK_DEPENDENCY_LOCK_FILES = Object.freeze([
 	"package.json",
 	"pnpm-lock.yaml"
 ]);
-const ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256 = "d12f8c74a24183901623206f98a97bcf3d5c0ed4eb0d74ab09903d578605696e";
+const ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256 = "aa81bb8a9b2eca793a117fafbaf6d74d77402ad7f9fc197f7928a35e131984e6";
 /** The published benchmark evidence was produced at this package version, by
 * the retired one-shot direct runner, before trace analysts moved to the
 * recursive DSPy RLM engine. Both evidence digests below are historical facts
@@ -6369,4 +6369,4 @@ function shellQuote(value) {
 //#endregion
 export { analystBenchmarkDependencyLockDigest as $, compareAnalystRunners as A, DEFAULT_MAX_VERIFICATION_ARTIFACT_BYTES as B, analystDefinitionProtocolSha256 as C, readAnalystBenchmarkArtifact as D, expandCodeTraceFailureBlocks as E, MAX_INCORRECT_BLOCKS as F, ANALYST_BENCHMARK_DEPENDENCY_LOCK_FILES as G, loadCodeTraceVerificationArtifacts as H, MAX_INCORRECT_BLOCK_STEPS as I, ANALYST_BENCHMARK_EVIDENCE_IMPLEMENTATION_SHA256 as J, ANALYST_BENCHMARK_DEPENDENCY_LOCK_SHA256 as K, publicBenchmarkProtocolSha256 as L, effectiveAnalystProtocolSha256 as M, readAnalystInstructionsOverride as N, renderCodeTraceCalibrationMarkdown as O, CODE_TRACE_BENCH_ANALYST_PROMPT as P, ANALYST_BENCHMARK_IMPLEMENTATION_SHA256 as Q, publicBenchmarkRlmInstructions as R, analystDefinitionAsymmetries as S, emptyPublicBenchmarkRunner as T, parseVerificationOutcome as U, appendVerificationArtifactsToOtlp as V, ANALYST_BENCHMARK_DEPENDENCY_LOCK_DIGEST_ALGORITHM as W, ANALYST_BENCHMARK_IMPLEMENTATION_DIGEST_ALGORITHM as X, ANALYST_BENCHMARK_EVIDENCE_PACKAGE_VERSION as Y, ANALYST_BENCHMARK_IMPLEMENTATION_FILES as Z, runReplVariableAnalystDefinition as _, primeAnalystProtocolSha256 as a, AGENT_RX_UPSTREAM_REVISION as at, runChunkedAnalystDefinition as b, nodeHttpPrimeBridgeTransport as c, codeTraceBenchCase as ct, publicBenchmarkDistributions as d, agentRxPredictionsToFindings as dt, analystBenchmarkImplementationDigest as et, publicBenchmarkSelectionReport as f, normalizeAgentRxCategory as ft, rlmEngineLimits as g, publicRlmAnalystDefinition as h, createPrimeBenchmarkRunner as i, ANALYST_BENCHMARK_OBSERVATIONS_FILE as it, analystInstructionsOverrideFromText as j, summarizeCodeTraceCalibration as k, loadPublicBenchmarkRows as l, codeTracerPredictionsToFindings as lt, createPublicBenchmarkRlmRunner as m, normalizeBenchmarkLabel as mt, runAnalystBenchmarkCommand as n, ANALYST_BENCHMARK_LOCAL_RECEIPT_FILE as nt, primeCodeTraceAnalystDefinition as o, renderAgentRxCalibrationMarkdown as ot, selectPublicBenchmarkRows as p, roundAgentRxStep as pt, ANALYST_BENCHMARK_EVIDENCE_DEPENDENCY_LOCK_SHA256 as q, renderAnalystBenchmarkMarkdown as r, ANALYST_BENCHMARK_MANIFEST_FILE as rt, runInlineAnalystDefinition as s, summarizeAgentRxCalibration as st, ANALYST_BENCHMARK_HELP as t, ANALYST_BENCHMARK_COST_LEDGER_FILE as tt, preparePublicAnalystBenchmark as u, agentRxBenchmarkCase as ut, createPublicBenchmarkDirectRunner as v, adaptPublicBenchmarkFindings as w, AnalystExpressivenessError as x, publicDirectAnalystDefinition as y, publicBenchmarkSystemPrompt as z };
 
-//# sourceMappingURL=benchmark-command-Cq7sAba4.js.map
+//# sourceMappingURL=benchmark-command-CkgLXHoN.js.map