@tangle-network/agent-eval 0.123.7 → 0.124.0

package/CHANGELOG.md

@@ -6,7 +6,7 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
 
 ## [Unreleased]
 
-### Added
+### Changed
 
 - `selfImprove({ budget: { candidateConcurrency } })` exposes the existing `runOptimization()` control for scoring candidate campaigns in parallel; it remains opt-in and defaults to one candidate campaign at a time.
 - `llmPolicyEditProposer()` and `projectPolicyEditHistory()` accept `scenarioOrder: 'input'` when controlled comparisons must preserve first-occurrence caller order; ranked evidence selection remains the default.
@@ -14,9 +14,6 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
 - `llmPolicyEditProposer({ redactCurrentSurfaceForModel })` can remove credentials and unrelated fields from the current surface sent to the model while applying validated edits to the complete original surface.
 - `CostLedger.listPending()` exposes immutable pending paid calls and distinguishes calls that are active, late after cancellation, or interrupted by a prior process so durable workflows can reconcile exact reservations before resuming.
 - `traceAnalystProposer()` accepts an opt-in `resolvePriorFindings` callback that forwards canonical prior findings into the existing analyst registry.
-
-### Changed
-
 - Trace-analysis actors are instructed to emit one executable JavaScript program per turn, report named turn-limit exhaustion, and preserve per-analyst failure details when a proposer produces no findings.
 - Keep one live tip per lineage track when another track branches or merges from it, and compare those track tips when building the frontier.
 - Pass track identity, operation, vision, ancestry, and proposer choice to candidate generation so independent tracks can pursue distinct strategies.
@@ -53,6 +50,28 @@ All notable changes to `@tangle-network/agent-eval` and its sibling `agent-eval-
 - `runSkillOpt({ holdoutScenarios })` fails closed because those rows are adaptively reused.
   Pass `selectionScenarios`; selection result fields now use `Selection` instead of `Holdout`, and `lift` is now `selectionLift`.
 
+## [0.124.0] — 2026-07-24 — rollout and supervisor-run subpaths
+
+### Added
+
+- `callLlm()` and `callLlmJson()` accept request-level or client-default `thinking: 'enabled' | 'disabled'`; GEPA, SkillOpt, and policy-edit authors expose the same per-proposer override, and the exact mode is preserved in cost bounds, raw request capture, and provider traffic.
+- `@tangle-network/agent-eval/rollout` subpath: the single owner of the `tangle.rollout.v1` serialization — canonical schema + fail-loud validation, ledger file API (`writeRolloutLedger`/`appendRolloutLines`/`readRolloutLedger`), harness-store intake readers (opencode sqlite, Claude Code project jsonl), exporters (SFT, reward rows, Prime Intellect verifiers `RolloutOutput`, OpenAI RFT), deterministic 9-rule scrubber, HuggingFace dataset-card generation, and the `agent-eval rollout-release` CLI (build + optional `--push`). Ported from the agent-runtime bench rollout-ledger and reconciled with the PR #410 row shape; see `docs/rollout.md` for the decision table.
+- `@tangle-network/agent-eval/supervisor-run` subpath: supervision-tree analysis alongside single-rollout trace analysis. `analyzeSupervisorRun(runDir | reader | sources)` returns a `SupervisorRunReport` — steer count with per-worker breakdown, spawn waves + sizes, max concurrency, respawns/repeated labels/delegation depth, supervisor wall + idle wall + worker utilization, accepted vs rejected vs empty-pass, evidence→respawn vs blind respawn, tokens/USD by role, judge verdict + patch stats — and `rollupSupervisorRuns` aggregates across runs. Every metric is `Measured<T> = T | {unavailable: reason}`, so a missing artifact never reads as a measured zero. The input contract is a `SupervisorRunReader` over already-read bytes; `loopsSupervisorRunReader` is one implementation (the loops `.loops/supervisor/*` on-disk layout). `supervisorRunRolloutLines` emits the tree as `tangle.rollout.v1` rows keyed by `parent_rollout_id`, so a supervision tree lands in the same ledger as solo rollouts. Ported from the agent-runtime bench run-report; byte-identical on both committed backfill fixtures.
+
+### Changed
+
+- `mintRolloutRows` now emits canonical `tangle.rollout.v1` lines (snake_case wire shape) instead of the interim `RolloutRow`; records without trace spans become labeled gap lines AND are listed in `missingTraces` instead of being skipped. `toSftRows`/`toRewardRows` operate on the new lines; `toSftRows` additionally enforces the trainable-split filter (holdout/dev/canary never export). The realness gate still forces reward 0 and SFT exclusion.
+
+### Fixed
+
+- `openOpencodeDb` composes the `node:sqlite` specifier at runtime so neither esbuild nor Vite rewrites it, keeping the opencode rollout reader working in the bundled package.
+
+## [0.123.8] — 2026-07-23 — reasoning-token accounting
+
+### Fixed
+
+- Preserve OpenAI-compatible `completion_tokens_details.reasoning_tokens` through `LlmCallResult`, cost receipts, and cost-ledger summaries.
+
 ## [0.123.7] — 2026-07-23 — publishable GEPA bridge metadata
 
 ### Fixed

package/dist/analyst/index.d.ts

@@ -1020,6 +1020,7 @@ interface LlmMessage {
         };
     }>;
 }
+type LlmThinkingMode = 'enabled' | 'disabled';
 interface LlmCallRequest {
     model: string;
     messages: LlmMessage[];
@@ -1032,6 +1033,8 @@ interface LlmCallRequest {
     };
     temperature?: number;
     maxTokens?: number;
+    /** OpenAI-compatible reasoning mode. Omitted when the provider default should apply. */
+    thinking?: LlmThinkingMode;
     /** Per-call timeout, default 300s. */
     timeoutMs?: number;
 }
@@ -1041,6 +1044,8 @@ interface LlmUsage {
     totalTokens: number;
     /** False when the provider omitted or malformed prompt/completion usage. */
     captured?: boolean;
+    /** Reasoning-token subset of completionTokens, when reported. */
+    reasoningTokens?: number;
     /** Proxies populate this when prompt caching is on. */
     cachedPromptTokens?: number;
 }
@@ -1123,6 +1128,8 @@ interface LlmClientOptions {
      * Default: `extract`.
      */
     jsonPayloadMode?: 'extract' | 'exact';
+    /** Default provider reasoning mode. A per-call request value takes precedence. */
+    thinking?: LlmThinkingMode;
     /** Fetch implementation — defaults to global `fetch`. Override for custom transport (e.g. tests). */
     fetch?: typeof fetch;
     /**

package/dist/analyst/index.js

@@ -9,13 +9,13 @@ import {
   diffFindings,
   emitSkillUsageFindings,
   runSemanticConceptJudge
-} from "../chunk-QNVBALEX.js";
+} from "../chunk-W5B3ZGP3.js";
 import {
   behavioralAnalyst,
   buildDefaultAnalystRegistry,
   createChatClient,
   deriveEfficiencyFindings
-} from "../chunk-WP3GEXKI.js";
+} from "../chunk-A6GT67HT.js";
 import "../chunk-HHWE3POT.js";
 import {
   ANALYST_SEVERITIES,
@@ -71,8 +71,8 @@ import {
   validatePolicyEdit,
   validatePolicyEditCandidateRecord,
   validateUsageSettlementTimeout
-} from "../chunk-MHPEGJHC.js";
-import "../chunk-MK3Q6GCL.js";
+} from "../chunk-DT7OXY3C.js";
+import "../chunk-EQUK3RFS.js";
 import {
   CostLedger
 } from "../chunk-GRCDRKII.js";

package/dist/benchmarks/index.d.ts

@@ -204,6 +204,8 @@ interface LlmUsage {
     totalTokens: number;
     /** False when the provider omitted or malformed prompt/completion usage. */
     captured?: boolean;
+    /** Reasoning-token subset of completionTokens, when reported. */
+    reasoningTokens?: number;
     /** Proxies populate this when prompt caching is on. */
     cachedPromptTokens?: number;
 }

package/dist/benchmarks/index.js

@@ -17,12 +17,12 @@ import {
   runBenchmarkAdapter,
   summarizeBenchmarkCampaign
 } from "../chunk-JKDNAOF5.js";
-import "../chunk-VPDOSN3L.js";
-import "../chunk-KKPPFIDS.js";
+import "../chunk-5PVZVCZB.js";
+import "../chunk-PMITBABE.js";
 import "../chunk-D5JZ7UDZ.js";
-import "../chunk-MHPEGJHC.js";
+import "../chunk-DT7OXY3C.js";
 import "../chunk-ARU2PZFM.js";
-import "../chunk-MK3Q6GCL.js";
+import "../chunk-EQUK3RFS.js";
 import "../chunk-PJQFMIOX.js";
 import "../chunk-GRCDRKII.js";
 import "../chunk-VI2UW6B6.js";

package/dist/campaign/index.d.ts

@@ -1113,6 +1113,7 @@ interface LlmMessage {
         };
     }>;
 }
+type LlmThinkingMode = 'enabled' | 'disabled';
 interface LlmCallRequest {
     model: string;
     messages: LlmMessage[];
@@ -1125,6 +1126,8 @@ interface LlmCallRequest {
     };
     temperature?: number;
     maxTokens?: number;
+    /** OpenAI-compatible reasoning mode. Omitted when the provider default should apply. */
+    thinking?: LlmThinkingMode;
     /** Per-call timeout, default 300s. */
     timeoutMs?: number;
 }
@@ -1134,6 +1137,8 @@ interface LlmUsage {
     totalTokens: number;
     /** False when the provider omitted or malformed prompt/completion usage. */
     captured?: boolean;
+    /** Reasoning-token subset of completionTokens, when reported. */
+    reasoningTokens?: number;
     /** Proxies populate this when prompt caching is on. */
     cachedPromptTokens?: number;
 }
@@ -1217,6 +1222,8 @@ interface LlmClientOptions {
      * Default: `extract`.
      */
     jsonPayloadMode?: 'extract' | 'exact';
+    /** Default provider reasoning mode. A per-call request value takes precedence. */
+    thinking?: LlmThinkingMode;
     /** Fetch implementation — defaults to global `fetch`. Override for custom transport (e.g. tests). */
     fetch?: typeof fetch;
     /**
@@ -5453,6 +5460,8 @@ interface SkillOptProposerOptions {
     editBudget?: number;
     temperature?: number;
     maxTokens?: number;
+    /** Provider reasoning mode for each patch-authoring call. Omitted to preserve the provider default. */
+    thinking?: LlmCallRequest['thinking'];
     /** Top-K weak scenarios/dimensions surfaced as evidence. Default 3. */
     evidenceK?: number;
 }
@@ -5704,6 +5713,8 @@ interface GepaProposerOptions {
     temperature?: number;
     /** Reflection max tokens. Default 6000. */
     maxTokens?: number;
+    /** Provider reasoning mode for each reflection call. Omitted to preserve the provider default. */
+    thinking?: LlmCallRequest['thinking'];
     /** Structured-doc constraints. Candidates violating any are rejected
      *  post-parse and dropped from the returned population. */
     constraints?: GepaProposerConstraints;
@@ -5935,6 +5946,8 @@ interface LlmPolicyEditProposerOptions {
     maxCandidates?: number;
     temperature?: number;
     maxTokens?: number;
+    /** Provider reasoning mode for each policy-authoring call. Omitted to preserve the provider default. */
+    thinking?: LlmCallRequest['thinking'];
     timeoutMs?: number;
     /** Number of most recent scored generations sent to the author. Default: 4. */
     maxHistoryGenerations?: number;

package/dist/campaign/index.js

@@ -71,7 +71,7 @@ import {
   userStoryScoreboard,
   validateSearchLedgerEvent,
   verifyCodeSurface
-} from "../chunk-VPDOSN3L.js";
+} from "../chunk-5PVZVCZB.js";
 import {
   assertCodeSurfaceIdentity,
   buildEvidenceVector,
@@ -112,7 +112,7 @@ import {
   surfaceContentHash,
   surfaceHash,
   verifyLoopProvenanceRecord
-} from "../chunk-KKPPFIDS.js";
+} from "../chunk-PMITBABE.js";
 import {
   SearchLedgerConflictError,
   SearchLedgerError,
@@ -133,9 +133,9 @@ import {
 import {
   POLICY_EDIT_CANDIDATE_RECORD_SCHEMA,
   validatePolicyEditCandidateRecord
-} from "../chunk-MHPEGJHC.js";
+} from "../chunk-DT7OXY3C.js";
 import "../chunk-ARU2PZFM.js";
-import "../chunk-MK3Q6GCL.js";
+import "../chunk-EQUK3RFS.js";
 import "../chunk-PJQFMIOX.js";
 import "../chunk-GRCDRKII.js";
 import "../chunk-VI2UW6B6.js";