@tangle-network/agent-eval 0.125.0 → 0.126.1

package/docs/rollout.md

@@ -1,4 +1,4 @@
-# Rollout — `tangle.rollout.v1`
+# Rollout: `tangle.rollout.v1`
 
 `@tangle-network/agent-eval/rollout` is the single owner of the `tangle.rollout.v1` training-row serialization.
 One JSONL line per agent invocation, with the full transcript inline, a single scalar reward, and fail-closed split labels.
@@ -10,7 +10,7 @@ No other package may write rollout rows; domain repos (e.g. the bench swe-arena
 | --- | --- | --- |
 | schema + validation | `src/rollout/schema.ts` | `RolloutLine`, `validateRolloutLine`, `assertRolloutLine`, `isTrainableSplit` |
 | ledger file API | `src/rollout/ledger.ts` | `writeRolloutLedger`, `appendRolloutLines`, `readRolloutLedger` |
-| minting from records | `src/rollout/mint.ts` | `mintRolloutRows(records, traceStore)` — RunRecord ⋈ trace via shared `runId` |
+| minting from records | `src/rollout/mint.ts` | `mintRolloutRows(records, traceStore)`: RunRecord joined to trace via shared `runId` |
 | harness-store intake | `src/rollout/readers/` | `openOpencodeDb` + `readOpencodeSessionMessages` (opencode sqlite), `findClaudeTranscripts` + `readClaudeTranscript` (Claude Code project jsonl) |
 | exporters | `src/rollout/exporters.ts` | `toSftRows`, `toRewardRows`, `toVerifiersRolloutOutputs` (Prime Intellect), `toRftItems` (OpenAI RFT), `toJsonl` |
 | release pipeline | `src/rollout/release/` | `scrubLines` (9 deterministic rules), `buildDatasetCard`, `buildHfDataset`, CLI `agent-eval rollout-release … [--push org/name]` |
@@ -33,16 +33,16 @@ Rule applied: where fields conflicted, RunRecord-derived semantics won; the ledg
 | `policy.*` | ledger | + `prompt_hash`, `config_hash`, `agent_profile_cell_id` from PR #410's RunRecord provenance |
 | `messages` | ledger | canonical OpenAI chat-with-tools incl. `reasoning_content`; minted lines inline the final llm span's conversation |
 | `steps` | PR #410 | optional trace-span projections (llm/tool), absent on harness-store-derived lines |
-| `outcome.reward` | conflict | **merged**: `number \| null` — `null` means no verdict exists (a labeled gap, never 0); minted lines always carry a number (`holdoutScore ?? searchScore ?? 0`), forced to 0 when realness-gated |
+| `outcome.reward` | conflict | **merged**: `number \| null`; `null` means no verdict exists (a labeled gap, never 0); minted lines always carry a number (`holdoutScore ?? searchScore ?? 0`), forced to 0 when realness-gated |
 | `outcome.realness_gated` | PR #410 | added (optional-on-read, absent = false); the anti-Goodhart gate travels into the data and SFT export refuses gated lines |
 | `outcome.reward_source` / `verdict` / `metrics` / `is_completed` / `is_truncated` / `error` | ledger | unchanged; mint fills `metrics` from `RunRecord.outcome.raw` |
 | `cost.*` | ledger | superset of PR #410's costUsd/totalTokens; `cost.usd` is `null` when `costProvenance.kind === 'uncaptured'` (never a fake 0) |
 | `artifacts.*`, `provenance.*` | ledger | `provenance.capture` gains `mint` alongside `settle-time` / `backfill` |
-| gap discipline | ledger | records without trace spans become labeled gap lines (`messages: []`, `provenance.gap`) AND are listed in `missingTraces` — PR #410's silent skip retired |
+| gap discipline | ledger | records without trace spans become labeled gap lines (`messages: []`, `provenance.gap`) AND are listed in `missingTraces`; PR #410's silent skip retired |
 
 ## Export filters (fail-closed)
 
 - SFT: `reward ≥ 1` (configurable) ∧ not realness-gated ∧ trainable split ∧ non-empty transcript. Rows are `{messages, metadata}`.
 - Reward rows: every line with a non-null reward, failures included; unlabeled lines excluded (a gap is not a zero).
 - Verifiers / RFT: transcript-bearing lines only; RFT additionally requires prompt turns before the first assistant turn.
-- Release (`rollout-release`): trainable split only; proposer lines dropped unless `--include-proposers`; every string value scrubbed by the 9 deterministic rules (idempotent — a second pass counts zero); `--push` requires `huggingface-cli` + `HF_TOKEN` and never prints the token.
+- Release (`rollout-release`): trainable split only; proposer lines dropped unless `--include-proposers`; every string value scrubbed by the 9 deterministic rules (idempotent, so a second pass counts zero); `--push` requires `huggingface-cli` + `HF_TOKEN` and never prints the token.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tangle-network/agent-eval",
-  "version": "0.125.0",
+  "version": "0.126.1",
   "description": "Evaluate and improve AI agents from runs, traces, judges, and feedback. Compare candidates, cluster failures, measure lift, and gate releases.",
   "homepage": "https://github.com/tangle-network/agent-eval#readme",
   "repository": {
@@ -161,10 +161,10 @@
   },
   "dependencies": {
     "@asteasolutions/zod-to-openapi": "^8.5.0",
-    "@ax-llm/ax": "^23.0.0",
+    "@ax-llm/ax": "^23.0.1",
     "@hono/node-server": "^2.0.0",
-    "@tangle-network/agent-core": "^0.4.11",
-    "@tangle-network/agent-interface": "^0.31.0",
+    "@tangle-network/agent-core": "^0.4.20",
+    "@tangle-network/agent-interface": "^0.32.0",
     "@tangle-network/tcloud": "^0.4.14",
     "hono": "^4.12.30",
     "zod": "^4.3.6"