@bjoernrochel/sevro 0.1.0-rc.18 → 0.1.0-rc.19

package/docs/development-cli.md

@@ -8,10 +8,15 @@ grading, evidence, and extension lifecycle.
 [versioned report contract](report-v1.md). It emits Markdown by default or
 `sevro.report.v1` with `--json`.
 
+`sevro prune` [removes older retained results](reading-results.md#prune-retained-results)
+and maintains the latest report index. Bare pruning keeps the newest ten results;
+`--keep`, `--older-than`, and `--latest` select explicit retention policies.
+
 Examples assume [Sevro and Bun are on `PATH`](installing.md#make-the-command-available).
 For a contributor checkout, use `bun src/cli.ts` in place of `sevro`.
 
-Use `sevro --help`, `sevro run --help`, or `sevro report --help` for examples,
+Use `sevro --help`, `sevro run --help`, `sevro report --help`, or
+`sevro prune --help` for examples,
 required options, and defaults without preparing a run. `sevro --version` prints
 the installed package version. Invocation and configuration diagnostics go to
 stderr in both output modes; `--json` keeps one `sevro.cli-result.v1` document

package/docs/evaluation-workflow.md

@@ -81,7 +81,10 @@ an earlier failed assessment into success.
 `report --latest --results-root ./other-results` for another output root or working
 directory. `--result-file ./path/to/result.json` selects older runs and accepts
 relative paths. Reports validate referenced evidence; missing/moved evidence
-produces a diagnostic. [Report v1](report-v1.md) defines fields and limits.
+produces a diagnostic. To reclaim native bundle space while keeping reportable
+results and hashes, use [`sevro prune --native-transcripts`](reading-results.md#remove-native-transcript-bundles).
+Reports label bundles intentionally removed by that command as pruned.
+[Report v1](report-v1.md) defines fields and limits.
 
 ## Choose another starter
 

package/docs/installing.md

@@ -23,10 +23,10 @@ Local installation records the selected version in the project lockfile.
 
 ## Current release
 
-The current release is `0.1.0-rc.18`. Install this exact version:
+The current release is `0.1.0-rc.19`. Install this exact version:
 
 ```sh
-bun add --exact @bjoernrochel/sevro@0.1.0-rc.18
+bun add --exact @bjoernrochel/sevro@0.1.0-rc.19
 ```
 
 <!-- sevro-current-release:end -->

package/docs/native-hosts.md

@@ -80,7 +80,10 @@ Page matching uses complete nonblank lines as anchors and exact fragments of
 at least 32 string code units. Fragments with multiple possible positions in a
 body cannot establish coverage. Recognizable unfinished fragments appear in
 `partialSkills` and keep the observation partial; a later complete match resolves
-them. Comparison and anchor budgets also fail closed. Private receipts retain
+them. Within each tool result, text covered by unambiguous complete-body matches
+does not also imply partial reads of skills sharing that text. Fragments returned
+outside those complete-body spans or in other tool results remain independent
+partial-read evidence. Comparison and anchor budgets also fail closed. Private receipts retain
 body offsets rather than tool text; the public observation omits those offsets.
 Shorter, transformed or unanchored fragments remain outside this matching scope.
 Newly created or changed skill bodies are outside the prepared snapshot. A

package/docs/reading-results.md

@@ -43,6 +43,75 @@ For a contributor checkout, use `bun src/cli.ts` in place of `sevro`.
 duration is unknown, not zero. Advisory recommendations remain independent
 of the task verdict.
 
+## Prune retained results
+
+Run `sevro prune` to permanently delete older retained evidence. With no options,
+it keeps the newest **10 results** under `<cwd>/.sevro/results`. Each completed
+attempt counts separately, including repeated evaluations, dry runs, failures,
+and interrupted runs that retained a CLI result.
+
+```sh
+sevro prune
+sevro prune --results-root ./other-results --keep 5
+sevro prune --older-than 30d
+sevro prune --latest
+```
+
+Choose at most one explicit policy. `--keep` accepts a positive safe integer;
+`--latest` is equivalent to `--keep 1`. `--older-than` accepts a positive integer
+followed by `ms`, `s`, `m`, `h`, `d`, or `w` (milliseconds through weeks). It
+deletes results strictly older than that duration at invocation. Age and newest
+ordering use each retained `result.json` file's modification time; copying or
+editing that file can change its retention age. Equal timestamps sort by
+absolute result-directory path, ascending. Relative roots resolve from the
+invoking directory.
+
+Pruning finds retained result directories inside the selected root, including
+nested `attempts` trees, validates results and their colocated run evidence, and
+removes each selected result's entire directory. Directories without a retained
+`result.json` are not themselves deleted. Active and unrecognized ownership states
+are excluded from the retention count. Descendant directory links are not
+followed; linked result, evidence, ownership and lock files are refused. The
+selected root itself may be a directory alias. Invalid policies or unreadable,
+malformed, inconsistent or unsafe retained inputs return exit `64` with a
+diagnostic. Validation completes before any result deletion or latest update.
+
+Pruning and CLI result publication serialize access to the root. Before deleting
+evidence, pruning atomically points `latest.json` at the newest remaining
+completed result, or removes the index if none remain. `sevro report --latest`
+and explicit reports for surviving results keep working; reports naming deleted
+results fail. An empty or missing results root is a successful no-op. Ownership
+coordination metadata, partial attempts without CLI results, temporary fixtures,
+and a separate `--run-state-root` remain outside evidence pruning.
+
+### Remove native transcript bundles
+
+Use `--native-transcripts` to permanently remove captured native bundles while
+keeping every result, recorded hash, observation, and other artifact:
+
+```sh
+sevro prune --native-transcripts
+sevro prune --native-transcripts --older-than 30d
+sevro prune --native-transcripts --keep 5
+sevro report --latest
+```
+
+Without a policy, this mode removes bundles from all completed results. With
+`--keep 5`, it preserves the newest five results' bundles and removes bundles
+from older results. `--latest` preserves the newest result's bundles;
+`--older-than` removes bundles only from results older than the given duration.
+Age, ordering, root selection, active-run protection, and input validation follow
+the rules above. The mode includes candidate, semantic, and advisory bundles.
+
+`result.json`, `run.json`, trial evidence, and `latest.json` stay unchanged.
+Before removing bytes, pruning atomically saves a private
+`native-transcripts-pruned.json` receipt beside `run.json`. Keep that receipt
+with the evidence: reports require it to distinguish intentional pruning from
+missing evidence. Reports show retained and pruned bundle counts while preserving
+the original verdicts and measurements. Pruned native bytes cannot be decoded
+or recovered from hashes. Repeating the command safely resumes interrupted
+deletions; ordinary whole-result pruning still works afterward.
+
 ## Compare carefully
 
 [Identity v1](identity-v1.md) binds inputs, evaluator content, host/model/effort,

package/docs/report-v1.md

@@ -29,6 +29,22 @@ including dry and failed runs. Pre-run failures do not replace latest. Missing
 latest or referenced evidence is an error. See
 [the evaluation workflow](evaluation-workflow.md).
 
+[`sevro prune`](reading-results.md#prune-retained-results) maintains this index
+when removing retained evidence. After pruning, latest selects the newest
+remaining completed result; if none remain, the index is removed and `--latest`
+reports a missing-input diagnostic. Explicit reports still require their selected
+results and evidence to exist.
+
+After [`sevro prune --native-transcripts`](reading-results.md#remove-native-transcript-bundles),
+reports accept absent native bundles only with the matching
+[pruning receipt](results-v1.md#native-transcript-bundles). Present bundles must
+still match their recorded hashes; unexplained missing bytes remain an error.
+New report rows include optional `nativeTranscripts` with nonnegative integer
+`retained` and `pruned` bundle counts across the case's trials and host roles.
+Markdown shows the same counts beside retained evidence. Older report documents
+without this field remain valid. Pruning does not change verdicts, routes,
+identity, usage, or duration.
+
 Each case row keeps task verdict, execution, and grading separate. Task pass
 rate is measured only when every trial completed execution and grading with an
 assessed task verdict. Candidate duration sums the measured host call time

package/docs/results-v1.md

@@ -142,6 +142,13 @@ results and evidence remain intact. Failed cleanup is retried on later evaluatio
 `sevro.observation.workspaces` records candidate and retained auxiliary workspace
 paths for inspection. These paths are temporary and do not affect evaluation identity.
 
+Retained results have no automatic expiration. The explicit
+[`sevro prune` command](reading-results.md#prune-retained-results) applies an
+operator-selected age or count policy, defaulting to the newest ten results.
+It deletes complete retained result directories and maintains `latest.json`
+for the remaining results. Active or unrecognized attempts remain protected;
+fixture cleanup and run-state coordination metadata have separate lifecycles.
+
 Opted-in [candidate artifact runtimes](runtime-v1.md#candidate-artifact-access)
 remain available through grading and are removed afterward, including failed
 and cancelled trials. Failed-fixture retention does not preserve them.
@@ -217,14 +224,26 @@ failure; cleanup warnings contain no raw error/output bodies.
 With [runtime transcript opt-in](runtime-v1.md#native-transcript-access), each
 bundled host captures full native files before removing its private state.
 Candidate artifact `sevro.native-transcripts.bundle` has format
-`sevro.native-transcripts.v1`. It contains `host` (`codex` or `claude`), `role`,
+`sevro.native-transcripts.v2`. It contains `host` (`codex` or `claude`), `role`,
 native `rootSessionId` (or `null`), `completeness`, `issues`, and `files`.
-Each file records its native relative `path`, SHA-256 `sha256`, and exact bytes
-as `bytesBase64`. Decode the bytes without rewriting native records. Native
+Each file records its native relative `path`, SHA-256 `sha256` of the original
+bytes, and `gzipBase64`: the exact file bytes compressed with gzip, then encoded
+as base64. Decode base64, decompress gzip, and verify the original-byte hash
+before using a file. Bound decompression to 5 MiB across all files in the bundle.
+Decode bytes without rewriting native records. Native
 session paths, root/child records and sidecar metadata preserve associations;
 the bundle does not normalize model, effort, timing, token, or cost statistics.
 Existing unknown metrics remain unknown.
 
+Sevro reads both v2 bundles and existing `sevro.native-transcripts.v1` bundles.
+V1 files use uncompressed `bytesBase64`; decode that field directly without
+gzip decompression and verify the same original-byte hash. Readers must select
+the payload field and decoding by `format` and reject unsupported formats,
+corrupt compressed payloads, hash mismatches, or a decoded total above 5 MiB.
+New captures emit v2; reading retained v1 evidence does not rewrite it.
+The retained view exposes the same exact native files for either format, with
+the original bundle in `index.json`.
+
 `sevro.host.native-transcripts` records the same capture state and file hashes
 without raw bytes. Semantic and advisory bundles retain their role in the
 bundle and use the existing `sevro.semantic[.<source digest>].` and
@@ -235,7 +254,8 @@ benchmarks can use the same references and decode the native files.
 
 Capture includes native transcript-tree sidecars and refuses descendant links,
 special files and escaping paths. It is bounded to 4,096 directory entries,
-5 MiB of native file bytes and 7 MiB of encoded file metadata/content. The
+5 MiB of native file bytes before compression and 7 MiB of encoded file
+metadata/gzip content. The
 bundle remains inside the existing 8 MiB item and 32 MiB host artifact budgets;
 it groups sessions instead of spending an artifact per child. Limits produce
 `byte_limit`, `entry_limit`, or `bundle_limit` issues. Unreadable, unsafe or
@@ -254,3 +274,25 @@ structure, not native semantic correctness.
 An empty/missing native JSONL tree is `unavailable`. Failure, cancellation and
 timeout retain available native bytes with incomplete capture state. Raw
 bundles and views are private evidence and must stay out of Git.
+
+The explicit [`sevro prune --native-transcripts` mode](reading-results.md#remove-native-transcript-bundles)
+can remove these bundles without rewriting CLI results, run/trial evidence,
+capture observations, native file hashes, or whole-bundle hashes. It retains
+other artifacts and does not change evaluation identity or assessment states.
+Reports validate present native bundles against their recorded SHA-256 and
+require an exact pruning receipt for absent bundles.
+
+Before deletion, Sevro atomically writes private
+`native-transcripts-pruned.json` beside `run.json`, with format
+`sevro.pruned-native-transcripts.v1`, the original `runId`, and `artifacts`.
+Each entry contains the original artifact reference (`id`, `path`, `sha256`,
+and any optional reference flags), plus its `caseId` and `trial`. Entries follow
+run trial/reference order and cover every candidate, semantic (including
+source-specific), and advisory native bundle in that run. A receipt must match
+these original references exactly; malformed, inconsistent, or linked receipts
+are errors. Bundle references must name regular files directly inside their
+result directory. Missing bytes without a receipt, changed hashes, and escaping
+references remain errors. An interrupted deletion may leave some marked bundles
+present; reports count actual availability, and a repeated prune completes removal.
+Transcript references that alias result/trial metadata, raw responses, or other
+retained artifacts are refused before pruning.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bjoernrochel/sevro",
-  "version": "0.1.0-rc.18",
+  "version": "0.1.0-rc.19",
   "license": "SEE LICENSE IN LICENSE",
   "description": "A standalone evaluation runner for agentic coding capabilities",
   "type": "module",

package/schemas/report-v1.schema.json

@@ -110,6 +110,15 @@
         "domainOutcomes"
       ],
       "properties": {
+        "nativeTranscripts": {
+          "type": "object",
+          "required": ["retained", "pruned"],
+          "properties": {
+            "retained": { "type": "integer", "minimum": 0 },
+            "pruned": { "type": "integer", "minimum": 0 }
+          },
+          "additionalProperties": false
+        },
         "resultFile": { "$ref": "#/$defs/path" },
         "evidencePath": { "type": ["string", "null"] },
         "runId": { "type": ["string", "null"] },

package/src/cli/completion.ts

@@ -2,11 +2,14 @@ import { dirname, join } from "node:path";
 import { atomicWriteJson } from "../storage";
 import type { CliResult } from "../engine";
 import { createReport } from "../report";
+import { withResultsLock } from "../results-lock";
 
 /** Latest means the last successfully retained CLI result, including dry/failing runs. */
 export async function retainCliResult(result: CliResult, resultsRoot: string) {
-  const resultFile = join(dirname(result.evidencePath), "result.json");
-  await atomicWriteJson(resultFile, result);
-  await atomicWriteJson(join(resultsRoot, "latest.json"), result);
-  return { resultFile, report: await createReport([resultFile]) };
+  return withResultsLock(resultsRoot, async () => {
+    const resultFile = join(dirname(result.evidencePath), "result.json");
+    await atomicWriteJson(resultFile, result);
+    await atomicWriteJson(join(resultsRoot, "latest.json"), result);
+    return { resultFile, report: await createReport([resultFile]) };
+  });
 }

package/src/cli/help.ts

@@ -10,12 +10,14 @@ Examples:
     --condition passive --trials 1 --threshold 1
   sevro run --help
   sevro report --result-file /absolute/results/run/result.json
+  sevro prune
 
 Commands:
   demo      Run the packaged credential-free assessed example; use demo --help.
   init      Create a new evaluation project; use init --help.
   run       Evaluate one resolved case or extension case; use run --help.
   report    Summarize retained results; use report --help.
+  prune     Remove retained results using a retention policy; use prune --help.
 
 Options:
   -h, --help     Show help without preparing a run.
@@ -131,6 +133,36 @@ Options and defaults:
   -h, --help   Show this help without reading result files.
 `;
 
+const PRUNE_HELP = `Usage: sevro prune [options]
+
+Examples:
+  sevro prune
+  sevro prune --results-root ./other-results --keep 5
+  sevro prune --older-than 30d
+  sevro prune --latest
+  sevro prune --native-transcripts --older-than 30d
+
+Defaults: keep the newest 10 completed results under <cwd>/.sevro/results.
+Select at most one explicit retention policy:
+  --keep <positive-integer>  Keep this many newest results.
+  --latest                   Keep only the newest result (same as --keep 1).
+  --older-than <duration>    Remove results older than a positive integer plus ms, s, m, h, d or w.
+  --results-root <path>      Relative or absolute results root, resolved from cwd.
+  --native-transcripts       Remove only native transcript bundles; keep results and hashes.
+  -h, --help                 Show help without reading or deleting results.
+
+Age and ordering use result.json modification times; equal times sort by path.
+Each retained attempt counts separately, including dry, failed and interrupted runs.
+Active/unrecognized attempts and directories without result.json are preserved.
+Evidence is permanently deleted. Remaining results stay usable by sevro report;
+latest.json is updated, or removed when no completed results remain.
+With --native-transcripts, no policy removes bundles from all completed results;
+--keep/--latest preserve bundles for the newest results, and --older-than selects older results.
+Result/evidence files and latest.json stay unchanged. Reports label removed bundles as pruned.
+Missing or empty results roots are a successful no-op. Invalid or unreadable inputs exit 64.
+Reference: docs/reading-results.md in the installed package.
+`;
+
 const INIT_HELP = `Usage: sevro init [options]
 
 Examples:
@@ -169,6 +201,9 @@ const HELP_REQUESTS: Record<string, string> = {
   "report --help": REPORT_HELP,
   "report -h": REPORT_HELP,
   "help report": REPORT_HELP,
+  "prune --help": PRUNE_HELP,
+  "prune -h": PRUNE_HELP,
+  "help prune": PRUNE_HELP,
 };
 
 /** Handle informational commands before loading inputs or creating run state. */

package/src/cli.ts

@@ -4,6 +4,7 @@ import { reportCommand } from "./report";
 import { informationCommand } from "./cli/help";
 import { demoCommand } from "./cli/demo";
 import { initCommand } from "./cli/init";
+import { pruneCommand } from "./prune";
 
 if (informationCommand(process.argv.slice(2))) process.exitCode = 0;
 else if (process.argv[2] === "demo") await demoCommand(process.argv.slice(3));
@@ -11,4 +12,6 @@ else if (process.argv[2] === "init")
   process.exitCode = await initCommand(process.argv.slice(3));
 else if (process.argv[2] === "report")
   process.exitCode = await reportCommand(process.argv.slice(2));
+else if (process.argv[2] === "prune")
+  process.exitCode = await pruneCommand(process.argv.slice(3));
 else await main(process.argv.slice(2));