@evolvingmachines/evolve 0.0.60 → 0.0.61

package/skills/evolve-evals/references/cli-reference/check.mdx

@@ -17,6 +17,7 @@ evolve check ./tasks \
   -i 'abs-*' \
   -l 5 \
   --watch
+evolve check --show-defaults
 ```
 
 ## Options
@@ -25,6 +26,10 @@ evolve check ./tasks \
   Check a published dataset's tasks instead of a local path. The checker reads the version's retained task package, and the job page's CHECK tab lists these checks by task.
 </ParamField>
 
+<ParamField path="--name" type="name">
+  A name for the check, Harbor's `--job-name`. Omitted, the accept timestamp in Harbor's shape, `YYYY-MM-DD__HH-MM-SS`.
+</ParamField>
+
 <ParamField path="-m, --model" type="name" default="openrouter/deepseek/deepseek-v4.1-flash">
   Model the checker runs. `fireworks/deepseek-v4.1-flash` is the same model on its Fireworks route. Must be on the `claude` roster.
 </ParamField>
@@ -45,6 +50,10 @@ evolve check ./tasks \
   Sandbox provider the checker runs on. Defaults to the platform's analysis default.
 </ParamField>
 
+<ParamField path="--show-defaults">
+  Print the defaults a check runs under when you name none: model, effort, provider, the rubric's criteria and the built-in prompt template, then exit. With `--json`, one object. Takes no path and no other check flag.
+</ParamField>
+
 <ParamField path="-n, --n-concurrent" type="n">
   Max concurrent task checks, beneath the organization's ceiling. Defaults to the ceiling.
 </ParamField>
@@ -93,8 +102,13 @@ List your task quality checks, newest first.
   Filter by check status: `queued`, `running`, `completed`.
 </ParamField>
 
+<ParamField path="-d, --dataset" type="name[@version]">
+  Only checks on this dataset. A bare name is every version, `name@version` one version. A value with an empty half (`name@`, `@version`) is refused as `invalid_input` on `dataset`.
+</ParamField>
+
 ```bash
 evolve check list --status running
+evolve check list --dataset terminal-bench-4@4.0
 ```
 
 ### check show
@@ -144,3 +158,75 @@ Save a check as a check folder (`check_report.json` plus one folder per task) by
 ```bash
 evolve check download 3f9a1c2e-… -o checks/
 ```
+
+## check files
+
+A task check's run, by its task check id (`results[].id` on `evolve check show`).
+
+
+```bash
+evolve check files <status|ls|cat|search|changes|archive> <task-check-id> [path] [options]
+```
+
+The run's file system: while the box runs, `files` reads it as it changes; after the run, the same commands read the kept tree. `files status` says which you get.
+
+- `files status <task-check-id>` — `live`, `captured`, `capturing` (the kept tree is still being written) or `none`; the box, how change events arrive (`watcher`), and the capture record (`ready`, `incomplete` with what was left out, or `failed`).
+- `files ls <task-check-id> [path]` — one folder, sorted by name (default `/`). Each row shows the type, mode, size, mtime and name; `created (agent)` or `modified (verifier)` marks what the run changed and when, `image only` marks a file the run never touched (it lists but does not open on the kept tree).
+- `files cat <task-check-id> <path>` — the raw bytes of one file, to stdout. `--range bytes=a-b` (or `bytes=a-`, `bytes=-n`) reads a slice.
+- `files search <task-check-id> <text>` — every line matching the text, as `path:line: snippet`. `--path` narrows the search to one folder (the whole box takes seconds); `--regex` treats the text as a pattern; a `(truncated)` note means there was more.
+- `files changes <task-check-id>` — the files the run created, modified or removed, with the phase (`--phase agent|verifier`).
+- `files archive <task-check-id>` — a `.tar.gz` of one subtree (`--path`, default the whole tree) to stdout, or saved under `-o <dir>`. The kept tree's archive lists the files it does not carry in `MANIFEST.tsv`; while the box runs, a subtree too large to read out in time is refused (`feature_unsupported`).
+
+<ParamField path="--source" type="live|capture">
+  Force the source. Without it, whichever the run has. A source that is not there is refused with `filesystem_state`.
+</ParamField>
+
+<ParamField path="-l, --limit" type="n">
+  Page size on `ls`, `search` and `changes` (`ls`, `changes` default 500; `search` default 200; max 1000).
+</ParamField>
+
+<ParamField path="--cursor" type="c">
+  Resume a paged `ls` or `changes` after the previous page's `next_cursor`.
+</ParamField>
+
+```bash
+evolve check files ls 3f9a1c2e-… /app/work
+evolve check files search 3f9a1c2e-… 'permission denied' --path /app/work
+evolve check files archive 3f9a1c2e-… --path /app/work -o archives/
+```
+
+## check logs
+
+```bash
+evolve check logs <task-check-id> --stream <name> [--follow] [options]
+```
+
+The box's own streams: `agent` (the harness's stdout and stderr), `verifier` and `system` (the box's system log, only when the job asked for it with `--system-log`); `setup` and `metrics` are named but not recorded today, so their pages are empty and say why. One page by default; `--follow` keeps printing lines as they arrive while the box lives, and returns once it is gone.
+
+<ParamField path="--stream" type="name" required>
+  `agent`, `verifier`, `system`, `setup` or `metrics`.
+</ParamField>
+
+<ParamField path="-f, --follow">
+  Keep printing new lines while the box lives.
+</ParamField>
+
+<ParamField path="-l, --limit" type="n" default="1000">
+  Lines per page, max 1000.
+</ParamField>
+
+<ParamField path="--cursor" type="seq">
+  Resume after this line.
+</ParamField>
+
+```bash
+evolve check logs 3f9a1c2e-… --stream agent --follow
+```
+
+## check procs
+
+```bash
+evolve check procs <task-check-id>
+```
+
+The processes running in the box right now. Live only: once the box is gone the command is refused with `filesystem_state`.

package/skills/evolve-evals/references/cli-reference/dataset.mdx

@@ -122,6 +122,19 @@ Download the original corpus package. Owner only.
   Directory to save into. Defaults to the current directory.
 </ParamField>
 
+## dataset files
+
+```bash
+evolve dataset files <status|ls|cat> <name@version> <task> [path] [options]
+```
+
+A published task's files, exactly as the platform stored them and as every trial ran against them: `files ls` lists one folder of the task directory (default `/`: `instruction.md`, `task.toml`, `environment/`, `tests/`), `files cat` prints one file's bytes (`--range bytes=a-b` reads a slice), `files status` says whether the version keeps its package (`package_retained`). A version imported before packages were kept is refused with `task_package_not_retained`. The version must be named: `name@version`.
+
+```bash
+evolve dataset files ls my-swe@1.0 abs-1 /tests
+evolve dataset files cat my-swe@1.0 abs-1 /tests/test.sh
+```
+
 ## dataset activate
 
 ```bash

package/skills/evolve-evals/references/cli-reference/run.mdx

@@ -143,6 +143,12 @@ Each of these applies to every arm of the job.
   Reasoning effort the analyzer runs at. Defaults to the per-model default: `high` on `openrouter/deepseek/deepseek-v4.1-flash`, `max` on `glm-5.3-flash`. Implies `--analyze`.
 </ParamField>
 
+## Sandbox logs
+
+<ParamField path="--system-log">
+  Record the box's own system log beside the streams every run keeps (`agent`, `verifier`). Read it back with `evolve trial logs --stream system`. Off by default.
+</ParamField>
+
 ## Timeouts
 
 <ParamField path="--timeout-multiplier" type="x" default="1.0">

package/skills/evolve-evals/references/cli-reference/trial.mdx

@@ -19,7 +19,7 @@ evolve trial show <trial-id>
 
 Show one trial in full detail: its status, reward, `tokens`, `spent` and `spent (judge)` rows, `gpu compute (est.)` on a GPU trial, the exception when it failed, and its latest analysis with the verdicts, the summary and the typed failure. An uploaded trial prints its provider as `ported` and its reported figures apart from the metered ones.
 
-There is no `trial files` or `trial read` verb; listing and reading a trial's stored files is SDK-only, `trials().files()` and `file()`.
+The run's own files are under [`trial files`](#trial-files); the platform's stored artifacts (session files, the verifier log) are SDK-only, `trials().files()` and `file()`.
 
 ## trial trace
 
@@ -70,7 +70,7 @@ Save a trial as its trial tree plus `evolve.json`, or stream one artifact to std
 </ParamField>
 
 <ParamField path="--stream" type="artifact">
-  Print one artifact to stdout instead of saving: `trace-parsed`, `verifier`, `trace-stdout`, `trace-stderr`, `trace-atif` (the ATIF trajectory), `trajectory` (reserved: the harness-native session file) or `agent-home`.
+  Print one artifact to stdout instead of saving: `trace-parsed`, `verifier`, `trace-stdout`, `trace-stderr`, `trace-atif` (the ATIF trajectory), `trajectory` (reserved: the harness-native session file), `agent-home`, or `filesystem` (the run's file system as one `.tar.gz`, raw bytes — pipe it to a file; the same archive as `trial files archive`).
 </ParamField>
 
 <ParamField path="--cursor" type="seq">
@@ -85,6 +85,75 @@ Save a trial as its trial tree plus `evolve.json`, or stream one artifact to std
 evolve trial download d1a10c4e-… --stream trace-stdout
 ```
 
+## trial files
+
+```bash
+evolve trial files <status|ls|cat|search|changes|archive> <trial-id> [path] [options]
+```
+
+The run's file system: while the box runs, `files` reads it as it changes; after the run, the same commands read the kept tree. `files status` says which you get.
+
+- `files status <trial-id>` — `live`, `captured`, `capturing` (the kept tree is still being written) or `none`; the box, how change events arrive (`watcher`), and the capture record (`ready`, `incomplete` with what was left out, or `failed`).
+- `files ls <trial-id> [path]` — one folder, sorted by name (default `/`). Each row shows the type, mode, size, mtime and name; `created (agent)` or `modified (verifier)` marks what the run changed and when, `image only` marks a file the run never touched (it lists but does not open on the kept tree).
+- `files cat <trial-id> <path>` — the raw bytes of one file, to stdout. `--range bytes=a-b` (or `bytes=a-`, `bytes=-n`) reads a slice.
+- `files search <trial-id> <text>` — every line matching the text, as `path:line: snippet`. `--path` narrows the search to one folder (the whole box takes seconds); `--regex` treats the text as a pattern; a `(truncated)` note means there was more.
+- `files changes <trial-id>` — the files the run created, modified or removed, with the phase (`--phase agent|verifier`).
+- `files archive <trial-id>` — a `.tar.gz` of one subtree (`--path`, default the whole tree) to stdout, or saved under `-o <dir>`. The kept tree's archive lists the files it does not carry in `MANIFEST.tsv`; while the box runs, a subtree too large to read out in time is refused (`feature_unsupported`).
+
+<ParamField path="--source" type="live|capture">
+  Force the source. Without it, whichever the run has. A source that is not there is refused with `filesystem_state`.
+</ParamField>
+
+<ParamField path="-l, --limit" type="n">
+  Page size on `ls`, `search` and `changes` (`ls`, `changes` default 500; `search` default 200; max 1000).
+</ParamField>
+
+<ParamField path="--cursor" type="c">
+  Resume a paged `ls` or `changes` after the previous page's `next_cursor`.
+</ParamField>
+
+```bash
+evolve trial files ls d1a10c4e-… /app/work
+evolve trial files search d1a10c4e-… 'permission denied' --path /app/work
+evolve trial files archive d1a10c4e-… --path /app/work -o archives/
+```
+
+## trial logs
+
+```bash
+evolve trial logs <trial-id> --stream <name> [--follow] [options]
+```
+
+The box's own streams: `agent` (the harness's stdout and stderr), `verifier` and `system` (the box's system log, only when the job asked for it with `--system-log`); `setup` and `metrics` are named but not recorded today, so their pages are empty and say why. One page by default; `--follow` keeps printing lines as they arrive while the box lives, and returns once it is gone.
+
+<ParamField path="--stream" type="name" required>
+  `agent`, `verifier`, `system`, `setup` or `metrics`.
+</ParamField>
+
+<ParamField path="-f, --follow">
+  Keep printing new lines while the box lives.
+</ParamField>
+
+<ParamField path="-l, --limit" type="n" default="1000">
+  Lines per page, max 1000.
+</ParamField>
+
+<ParamField path="--cursor" type="seq">
+  Resume after this line.
+</ParamField>
+
+```bash
+evolve trial logs d1a10c4e-… --stream agent --follow
+```
+
+## trial procs
+
+```bash
+evolve trial procs <trial-id>
+```
+
+The processes running in the box right now. Live only: once the box is gone the command is refused with `filesystem_state`.
+
 ## trial retry
 
 ```bash

package/skills/evolve-evals/references/core-concepts/trial-outputs.mdx

@@ -101,3 +101,17 @@ evolve trial download <trial-id> --stream verifier
 ```
 
 From the SDK, `trials().artifact(id, "verifier")` returns the same text, and `null` when the trial never stored it.
+
+## The run's file system
+
+Beyond the artifacts above, every trial's whole file system can be browsed: while the box runs you read it live, and after the run you read the kept tree — the files the run created or modified, with the files it never touched listed but held only by the image. The same reads work on analysis runs and task checks, and a published task's own files can be read from its package.
+
+```bash
+evolve trial files status <trial-id>
+evolve trial files ls <trial-id> /app/work
+evolve trial files cat <trial-id> /app/work/main.py
+evolve trial files changes <trial-id>
+evolve trial files archive <trial-id> --path /app/work -o archives/
+```
+
+The box's own streams are read the same way: `evolve trial logs <trial-id> --stream agent --follow` prints the harness's output as it arrives; `verifier` once the verifier ran, `system` when the job was started with `--system-log`; `setup` and `metrics` are named but not recorded today, so their pages are empty and say why. `evolve trial procs <trial-id>` lists the box's processes while it runs. From the SDK it is `trials().filesystem(id)`; see the [SDK reference](/sdk-reference/trials#filesystem).

package/skills/evolve-evals/references/sdk/python.mdx

@@ -124,6 +124,28 @@ async for dataset in catalog.list():
 version = await catalog.get("terminal-bench-4@4.0")
 ```
 
+## Browse the run's files
+
+Every trial has a file system you can read while the box runs and after it is gone, plus its sandbox logs and process list. `filesystem(id)` returns one object for all of it.
+
+```python
+fs = t.filesystem(trial_id)
+print((await fs.status()).state)   # "live" | "captured" | "capturing" | "none"
+
+folder = await fs.list(path="/app/work")
+main = (await fs.read("/app/work/main.py")).decode()
+hits = await fs.search("permission denied", path="/app/work")
+changed = await fs.changes()
+await fs.archive(path="/app/work", to="./archives")
+
+async for frame in fs.events():
+    if frame.event == "fs":
+        print(frame.data["type"], frame.data["path"])
+agent_log = await fs.logs("agent")
+```
+
+Analysis runs (`analyses().filesystem(id)`) and task checks (`checks().task_filesystem(check_id, task_check_id)`) have the same surface; a published task's own files are `datasets().task_files("name@version", task)`.
+
 ## Everything else
 
 `analyses()`, `checks()`, `skills()`, `agents()`, `auth()` and `orgs()` are built the same way. `meta()` fetches the platform's capability document, the live list of harnesses, models and limits, and needs no key. `hosted()` builds every client from one configuration.

package/skills/evolve-evals/references/sdk/typescript.mdx

@@ -128,6 +128,28 @@ for await (const dataset of catalog.list()) {
 const version = await catalog.get("terminal-bench-4@4.0");
 ```
 
+## Browse the run's files
+
+Every trial has a file system you can read while the box runs and after it is gone, plus its sandbox logs and process list. `filesystem(id)` returns one object for all of it.
+
+```ts
+const fs = t.filesystem(trialId);
+console.log((await fs.status()).state);   // "live" | "captured" | "capturing" | "none"
+
+const folder = await fs.list({ path: "/app/work" });
+const main = (await fs.read("/app/work/main.py")).toString("utf8");
+const hits = await fs.search({ q: "permission denied", path: "/app/work" });
+const changed = await fs.changes();
+await fs.archive({ path: "/app/work", to: "./archives" });
+
+for await (const frame of fs.events()) {
+  if (frame.event === "fs") console.log(frame.data.type, frame.data.path);
+}
+const agentLog = await fs.logs({ stream: "agent" });
+```
+
+Analysis runs (`analyses().filesystem(id)`) and task checks (`checks().taskFilesystem(checkId, taskCheckId)`) have the same surface; a published task's own files are `datasets().taskFiles("name@version", task)`.
+
 ## Everything else
 
 `analyses()`, `checks()`, `skills()`, `agents()`, `auth()` and `orgs()` are built the same way. `meta()` fetches the platform's capability document, the live list of harnesses, models and limits, and needs no key. `hosted()` builds every client from one configuration.

package/skills/evolve-evals/references/sdk-reference/analyses.mdx

@@ -123,3 +123,73 @@ description: "Read trace-analysis runs."
     ```
   </Tab>
 </Tabs>
+
+## filesystem(analysisId: string)
+
+<Tabs>
+  <Tab title="TypeScript">
+    ```ts
+    filesystem(analysisId: string): RunFilesystem
+
+    interface RunFilesystem {
+      status(): Promise<FilesystemStatus>
+      list(options?: { path?: string; source?: "live" | "capture"; cursor?: string; limit?: number }): Promise<FilesystemListing>
+      read(path: string, options?: { source?: "live" | "capture"; range?: TrialFileRange }): Promise<Buffer>
+      search(options: { q: string; path?: string; regex?: boolean; limit?: number; source?: "live" | "capture" }): Promise<FilesystemSearchResult>
+      changes(options?: { source?: "live" | "capture"; phase?: "setup" | "agent" | "verifier" | "all"; cursor?: string; limit?: number }): Promise<FilesystemChanges>
+      archive(options?: { path?: string; source?: "live" | "capture" }): Promise<Buffer>
+      archive(options: { path?: string; source?: "live" | "capture"; to: string }): Promise<string>
+      watch(paths: string[]): Promise<FilesystemWatchResult>
+      events(options?: { lastEventId?: string; signal?: AbortSignal }): AsyncIterableIterator<FilesystemStreamEvent>
+      logs(options: { stream: SandboxLogStream; cursor?: string; limit?: number }): Promise<SandboxLogLines>
+      logEvents(options?: { lastEventId?: string; signal?: AbortSignal }): AsyncIterableIterator<SandboxLogEvent>
+      procs(): Promise<SandboxProcs>
+    }
+    ```
+
+    The analysis run's file system, sandbox logs and process list. While the box runs, every read sees it as it changes; after the run, the same reads answer from the kept tree. `status()` says which (`state`: `live`, `captured`, `capturing`, `none`), and `source` on a read forces one — a source that is not there is refused with `filesystem_state`.
+
+    `list` pages one folder by name; each entry says whether the run created or modified it and in which phase, and on the kept tree `captured: false` marks a file the run never touched (it lists but does not open — `not_captured`). `read` answers raw bytes, a slice with `range`. `search` finds text under a folder, or over the whole box when `path` is `/` (seconds); `truncated` says there was more. `changes` is the flat list of what the run created, modified or removed. `archive` is a `.tar.gz` of one subtree, in memory or saved under `to`; while the box runs, a subtree too large to read out in time is refused (`feature_unsupported`).
+
+    `events` yields change frames while the box lives (`state`, then one `fs` per change with its `seq`, `path` and `type`); on a provider without a native watcher, `watch(paths)` declares the folders you have open so their changes are reported (at most 8). `logs` pages one stream — `agent`, `verifier`, or `system` when the job asked for it; `setup` and `metrics` are named but not recorded today, so their page is empty with a `reason` — and `logEvents` follows every stream while the box lives. `procs` is the live process list.
+
+    ```ts
+    const fs = analyses().filesystem(analysis.id);
+    const { state } = await fs.status();
+    for (const entry of (await fs.list({ path: "/app/work" })).entries) console.log(entry.name, entry.changed);
+    const main = (await fs.read("/app/work/main.py")).toString("utf8");
+    for await (const frame of fs.events()) if (frame.event === "fs") console.log(frame.data.path, frame.data.type);
+    ```
+  </Tab>
+  <Tab title="Python">
+    ```python
+    def filesystem(self, analysis_id: str) -> RunFilesystem
+
+    class RunFilesystem:
+        async def status(self) -> FilesystemStatus
+        async def list(self, *, path="/", source=None, cursor=None, limit=None) -> FilesystemListing
+        async def read(self, path, *, source=None, start=None, end=None, suffix=None) -> bytes
+        async def search(self, q, *, path="/", regex=False, limit=None, source=None) -> FilesystemSearchResult
+        async def changes(self, *, source=None, phase=None, cursor=None, limit=None) -> FilesystemChanges
+        async def archive(self, *, path="/", source=None, to=None)  # bytes, or the saved path with to=
+        async def watch(self, paths) -> FilesystemWatchResult
+        def events(self, *, last_event_id=None) -> AsyncIterator[FilesystemStreamEvent]
+        async def logs(self, stream, *, cursor=None, limit=None) -> SandboxLogLines
+        def log_events(self, *, last_event_id=None) -> AsyncIterator[SandboxLogEvent]
+        async def procs(self) -> SandboxProcs
+    ```
+
+    The same surface, the same words: `status().state` says whether the reads answer from the running box (`live`) or the kept tree (`captured`); `source=` forces one. `read` answers bytes, a slice with `start`/`end` or `suffix`. `events` and `log_events` are async iterators over the live streams.
+
+    ```python
+    fs = analyses().filesystem(analysis.id)
+    print((await fs.status()).state)
+    for entry in (await fs.list(path="/app/work")).entries:
+        print(entry.name, entry.changed)
+    main = (await fs.read("/app/work/main.py")).decode()
+    async for frame in fs.events():
+        if frame.event == "fs":
+            print(frame.data["path"], frame.data["type"])
+    ```
+  </Tab>
+</Tabs>

package/skills/evolve-evals/references/sdk-reference/checks.mdx

@@ -10,7 +10,7 @@ description: "Start and read task quality checks."
 <Tabs>
   <Tab title="TypeScript">
     ```ts
-    create(input: { source: { directory: string } | { dataset: string }; model_name?: string; rubric?: Rubric; prompt?: string; reasoning_effort?: string; sandbox_provider?: EvalSandboxProvider; n_concurrent?: number; include_task_names?: string[]; exclude_task_names?: string[]; n_tasks?: number; onUploadProgress?: (sentBytes: number, totalBytes: number) => void }): Promise<Check>
+    create(input: { source: { directory: string } | { dataset: string }; name?: string; model_name?: string; rubric?: Rubric; prompt?: string; reasoning_effort?: string; sandbox_provider?: EvalSandboxProvider; n_concurrent?: number; include_task_names?: string[]; exclude_task_names?: string[]; n_tasks?: number; onUploadProgress?: (sentBytes: number, totalBytes: number) => void }): Promise<Check>
     ```
 
     Check a local task directory or a directory of them, uploaded as one archive (`upload_too_large` past the published ceiling), or a published dataset, nothing uploaded. Returns the accepted check with one `queued` result per task; an empty selection is refused with `no_checkable_tasks`, and a dataset version resolves as for a job (`dataset_not_found`, `no_active_version`, `version_not_ready`). A check on a dataset version also appears on every job spanning it, as `check` on `jobs().tasks()` rows.
@@ -25,6 +25,7 @@ description: "Start and read task quality checks."
         directory: Optional[str] = None,
         *,
         dataset: Optional[str] = None,
+        name=None,
         model_name=None,
         rubric: Optional[Rubric] = None,
         prompt=None,
@@ -52,14 +53,15 @@ description: "Start and read task quality checks."
   <Tab title="TypeScript">
     ```ts
     get(checkId: string): Promise<Check>
-    list(options?: { scope?: "my" | "shared"; status?: CheckStatus[]; limit?: number; cursor?: string }): CheckList
+    list(options?: { scope?: "my" | "shared"; status?: CheckStatus[]; dataset?: string; limit?: number; cursor?: string }): CheckList
     watch(checkId: string, options?: { onProgress?: (check: Check) => void; signal?: AbortSignal; pollIntervalMs?: number }): Promise<Check>
     ```
 
-    One check with its per-task results, your checks, or a poll until every task settled.
+    One check with its per-task results, your checks, or a poll until every task settled. `dataset` keeps only the checks on one dataset: a bare name is every version, `name@version` one version.
 
     ```ts
     const done = await checks().watch(check.id);
+    const onTb4 = await checks().list({ dataset: "terminal-bench-4@4.0" });
     ```
   </Tab>
   <Tab title="Python">
@@ -69,6 +71,7 @@ description: "Start and read task quality checks."
         *,
         scope: Optional[JobListScope] = None,
         status: Optional[List[CheckStatus]] = None,
+        dataset: Optional[str] = None,
         limit: Optional[int] = None,
         cursor: Optional[str] = None,
     )
@@ -81,10 +84,38 @@ description: "Start and read task quality checks."
     ) -> Check
     ```
 
-    One check with its per-task results, your checks, or a poll until every task settled.
+    One check with its per-task results, your checks, or a poll until every task settled. `dataset` keeps only the checks on one dataset: a bare name is every version, `name@version` one version.
 
     ```python
     done = await checks().watch(check.id)
+    on_tb4 = await checks().list(dataset="terminal-bench-4@4.0")
+    ```
+  </Tab>
+</Tabs>
+
+## defaults
+
+<Tabs>
+  <Tab title="TypeScript">
+    ```ts
+    defaults(): Promise<CheckDefaults>
+    ```
+
+    The defaults a check runs under when you name none: `model_name`, `reasoning_effort`, `sandbox_provider`, the platform's `rubric`, and `prompt`, the built-in check prompt as a template with `{task_path}`, `{file_tree}` and `{criteria_guidance}` unrendered. Edit the rubric or the prompt from here and pass it to `create`; passed back unchanged, they run exactly as the defaults do, and the check records them as named, so a later change of the platform default does not move it.
+
+    ```ts
+    const { rubric } = await checks().defaults();
+    ```
+  </Tab>
+  <Tab title="Python">
+    ```python
+    async def defaults() -> CheckDefaults
+    ```
+
+    The defaults a check runs under when you name none: `model_name`, `reasoning_effort`, `sandbox_provider`, the platform's `rubric`, and `prompt`, the built-in check prompt as a template with `{task_path}`, `{file_tree}` and `{criteria_guidance}` unrendered. Edit the rubric or the prompt from here and pass it to `create`; passed back unchanged, they run exactly as the defaults do, and the check records them as named, so a later change of the platform default does not move it.
+
+    ```python
+    rubric = (await checks().defaults())["rubric"]
     ```
   </Tab>
 </Tabs>
@@ -164,3 +195,73 @@ description: "Start and read task quality checks."
     ```
   </Tab>
 </Tabs>
+
+## taskFilesystem(checkId: string, taskCheckId: string)
+
+<Tabs>
+  <Tab title="TypeScript">
+    ```ts
+    taskFilesystem(checkId: string, taskCheckId: string): RunFilesystem
+
+    interface RunFilesystem {
+      status(): Promise<FilesystemStatus>
+      list(options?: { path?: string; source?: "live" | "capture"; cursor?: string; limit?: number }): Promise<FilesystemListing>
+      read(path: string, options?: { source?: "live" | "capture"; range?: TrialFileRange }): Promise<Buffer>
+      search(options: { q: string; path?: string; regex?: boolean; limit?: number; source?: "live" | "capture" }): Promise<FilesystemSearchResult>
+      changes(options?: { source?: "live" | "capture"; phase?: "setup" | "agent" | "verifier" | "all"; cursor?: string; limit?: number }): Promise<FilesystemChanges>
+      archive(options?: { path?: string; source?: "live" | "capture" }): Promise<Buffer>
+      archive(options: { path?: string; source?: "live" | "capture"; to: string }): Promise<string>
+      watch(paths: string[]): Promise<FilesystemWatchResult>
+      events(options?: { lastEventId?: string; signal?: AbortSignal }): AsyncIterableIterator<FilesystemStreamEvent>
+      logs(options: { stream: SandboxLogStream; cursor?: string; limit?: number }): Promise<SandboxLogLines>
+      logEvents(options?: { lastEventId?: string; signal?: AbortSignal }): AsyncIterableIterator<SandboxLogEvent>
+      procs(): Promise<SandboxProcs>
+    }
+    ```
+
+    The task check's file system, sandbox logs and process list. While the box runs, every read sees it as it changes; after the run, the same reads answer from the kept tree. `status()` says which (`state`: `live`, `captured`, `capturing`, `none`), and `source` on a read forces one — a source that is not there is refused with `filesystem_state`.
+
+    `list` pages one folder by name; each entry says whether the run created or modified it and in which phase, and on the kept tree `captured: false` marks a file the run never touched (it lists but does not open — `not_captured`). `read` answers raw bytes, a slice with `range`. `search` finds text under a folder, or over the whole box when `path` is `/` (seconds); `truncated` says there was more. `changes` is the flat list of what the run created, modified or removed. `archive` is a `.tar.gz` of one subtree, in memory or saved under `to`; while the box runs, a subtree too large to read out in time is refused (`feature_unsupported`).
+
+    `events` yields change frames while the box lives (`state`, then one `fs` per change with its `seq`, `path` and `type`); on a provider without a native watcher, `watch(paths)` declares the folders you have open so their changes are reported (at most 8). `logs` pages one stream — `agent`, `verifier`, or `system` when the job asked for it; `setup` and `metrics` are named but not recorded today, so their page is empty with a `reason` — and `logEvents` follows every stream while the box lives. `procs` is the live process list.
+
+    ```ts
+    const fs = checks().taskFilesystem(check.id, check.results[0].id);
+    const { state } = await fs.status();
+    for (const entry of (await fs.list({ path: "/app/work" })).entries) console.log(entry.name, entry.changed);
+    const main = (await fs.read("/app/work/main.py")).toString("utf8");
+    for await (const frame of fs.events()) if (frame.event === "fs") console.log(frame.data.path, frame.data.type);
+    ```
+  </Tab>
+  <Tab title="Python">
+    ```python
+    def task_filesystem(self, check_id: str, task_check_id: str) -> RunFilesystem
+
+    class RunFilesystem:
+        async def status(self) -> FilesystemStatus
+        async def list(self, *, path="/", source=None, cursor=None, limit=None) -> FilesystemListing
+        async def read(self, path, *, source=None, start=None, end=None, suffix=None) -> bytes
+        async def search(self, q, *, path="/", regex=False, limit=None, source=None) -> FilesystemSearchResult
+        async def changes(self, *, source=None, phase=None, cursor=None, limit=None) -> FilesystemChanges
+        async def archive(self, *, path="/", source=None, to=None)  # bytes, or the saved path with to=
+        async def watch(self, paths) -> FilesystemWatchResult
+        def events(self, *, last_event_id=None) -> AsyncIterator[FilesystemStreamEvent]
+        async def logs(self, stream, *, cursor=None, limit=None) -> SandboxLogLines
+        def log_events(self, *, last_event_id=None) -> AsyncIterator[SandboxLogEvent]
+        async def procs(self) -> SandboxProcs
+    ```
+
+    The same surface, the same words: `status().state` says whether the reads answer from the running box (`live`) or the kept tree (`captured`); `source=` forces one. `read` answers bytes, a slice with `start`/`end` or `suffix`. `events` and `log_events` are async iterators over the live streams.
+
+    ```python
+    fs = checks().task_filesystem(check["id"], check["results"][0]["id"])
+    print((await fs.status()).state)
+    for entry in (await fs.list(path="/app/work")).entries:
+        print(entry.name, entry.changed)
+    main = (await fs.read("/app/work/main.py")).decode()
+    async for frame in fs.events():
+        if frame.event == "fs":
+            print(frame.data["path"], frame.data["type"])
+    ```
+  </Tab>
+</Tabs>

package/skills/evolve-evals/references/sdk-reference/datasets.mdx

@@ -90,6 +90,46 @@ description: "The catalog: list, read, publish, download, activate, delete."
   </Tab>
 </Tabs>
 
+## taskFiles
+
+<Tabs>
+  <Tab title="TypeScript">
+    ```ts
+    taskFiles(ref: string, taskName: string): TaskPackageFiles
+
+    interface TaskPackageFiles {
+      status(): Promise<TaskPackageFilesystemStatus>
+      list(options?: { path?: string; cursor?: string; limit?: number }): Promise<FilesystemListing>
+      read(path: string, options?: { range?: TrialFileRange }): Promise<Buffer>
+    }
+    ```
+
+    A published task's files, exactly as the platform stored them and as every trial ran against them: `list` pages one folder of the task directory (`/` holds `instruction.md`, `task.toml`, `environment/`, `tests/`), `read` answers one file's bytes. `ref` must name the version (`name@version`); a version imported before packages were kept is refused with `task_package_not_retained` (`status().package_retained` says so first).
+
+    ```ts
+    const files = datasets().taskFiles("my-swe@1.0", "abs-1");
+    const tests = await files.list({ path: "/tests" });
+    const script = (await files.read("/tests/test.sh")).toString("utf8");
+    ```
+  </Tab>
+  <Tab title="Python">
+    ```python
+    def task_files(self, ref: str, task_name: str) -> TaskPackageFiles
+
+    class TaskPackageFiles:
+        async def status(self) -> TaskPackageFilesystemStatus
+        async def list(self, *, path="/", cursor=None, limit=None) -> FilesystemListing
+        async def read(self, path, *, start=None, end=None, suffix=None) -> bytes
+    ```
+
+    ```python
+    files = datasets().task_files("my-swe@1.0", "abs-1")
+    tests = await files.list(path="/tests")
+    script = (await files.read("/tests/test.sh")).decode()
+    ```
+  </Tab>
+</Tabs>
+
 ## preflight
 
 <Tabs>

package/skills/evolve-evals/references/sdk-reference/errors.mdx

@@ -88,6 +88,17 @@ Grouped by the door that returns them. The HTTP status is stated where it is fix
 | `insufficient_credits` | 402. The account's credit balance is zero at job create, resume or retry. |
 | `quota_exceeded` | 429, no retry delay. The organization's `max_queued_trials` would be crossed. `details` carry `quota`, `limit`, `used`, `requested` and `org`. The CLI prints `Launch quota exceeded:` and exits 2. |
 
+### Files and logs
+
+| Code | When |
+| --- | --- |
+| `not_found` | 404. A path the run's file system does not hold. |
+| `not_captured` | 404. A read of a file on the kept tree that the run never touched; only the image holds it. |
+| `filesystem_state` | 409. `source=live` while the box is gone, or `source=capture` before the kept tree exists; `details.state` says what the run has. |
+| `feature_unsupported` | 422. The provider or the run's shape has no live view of what was asked; `details.cause` names it. |
+| `provider_unreachable` | 503. The box's provider did not answer; wait `retryAfterSec`. |
+| `task_package_not_retained` | 409. The dataset version keeps no task package, so its task files cannot be served. |
+
 ### Input
 
 | Code | When |

package/skills/evolve-evals/references/sdk-reference/jobs.mdx

@@ -16,7 +16,7 @@ description: "Start, follow, and derive jobs."
     ): Promise<Job>
     ```
 
-    `JobCreate`: `datasets` (selectors: `name`, optional `version`, `task_names`, `exclude_task_names`, `n_tasks`), `agents` (arms: `name`, `model_name`, optional `version`, `reasoning_effort`, `kwargs`, `preset`, `skills`), and optional `job_name`, `n_attempts`, `n_concurrent_trials` (1 to 150), `max_trial_spend_usd`, `sandbox_provider`, `retry`, `analyze`, the five `*_timeout_multiplier` fields, `verifier_env` (`REWARDKIT_JUDGE` and `REWARDKIT_MODEL` only) and `secrets`. `agent_env` is in the shape but refused by the server.
+    `JobCreate`: `datasets` (selectors: `name`, optional `version`, `task_names`, `exclude_task_names`, `n_tasks`), `agents` (arms: `name`, `model_name`, optional `version`, `reasoning_effort`, `kwargs`, `preset`, `skills`), and optional `job_name`, `n_attempts`, `n_concurrent_trials` (1 to 150), `max_trial_spend_usd`, `sandbox_provider`, `retry`, `analyze`, `system_log` (record the box's own system log stream beside `agent`, `verifier`, `setup` and `metrics`; off by default), the five `*_timeout_multiplier` fields, `verifier_env` (`REWARDKIT_JUDGE` and `REWARDKIT_MODEL` only) and `secrets`. `agent_env` is in the shape but refused by the server.
 
     The response is the [Job](/sdk-reference/types#job), with the resolved `max_trial_spend_usd`, `worst_case_spend_usd`, `retry`, `analyze` and multipliers. `options.idempotencyKey` makes a repeated call return the same job, with `idempotent_replay` true; a different request under a used key is refused with `idempotency_key_reused`.