@evolvingmachines/evolve 0.0.65 → 0.0.67-train-harness-wave-2026-09-25.20260926.e00a8a6d

package/skills/evolve-evals/references/dashboard/jobs.mdx

@@ -1,65 +0,0 @@
----
-title: "Jobs"
-description: "Start a job from the browser, follow it, and read its results."
----
-
-The Jobs page lists your jobs, newest first. A task quality check is a job too, so the list holds both: the KIND column tells them apart, and a check row opens the [check's page](/dashboard/checks#the-check-page).
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="The jobs list">
-  <img src="/images/jobs.png" alt="The Jobs page: a NEW JOB button, a search box, a status filter, and a table of jobs and checks with kind, status, datasets, agents, trials, cost and creation time." />
-</Frame>
-
-Search by name, and filter by status: ALL, QUEUED, RUNNING, CANCELLING, COMPLETED, CANCELLED or FAILED. A job row shows its datasets, its agents with their models, how many trials have settled out of the total, what it has spent, and when it started; a check row shows what was checked, the checker's model and effort, how many task checks have settled out of the total, the checker's cost, and when it started. Tick rows to cancel live jobs or delete finished ones; a check row cannot be ticked, its verbs live on its page.
-
-## New job
-
-NEW JOB opens the form. It takes the same choices as `evolve run`.
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="The New job form">
-  <img src="/images/new-job.png" alt="The New job form: a Sources list of datasets to tick, an Agents section with agent, model and reasoning effort, an ADD AGENT button, and a Summary rail with LAUNCH and VALIDATE buttons." />
-</Frame>
-
-- **Sources**: tick one or more datasets, cap the task count, and narrow the tasks with include and exclude patterns over their names.
-- **Agents**: one arm per entry, with its agent, its model (required) and its reasoning effort. Arm options hold the version pin, a preset, a config file and skills. ADD AGENT adds another arm.
-- **Secrets**: attach a secret from your vault, or save and attach a one-off value.
-- **Job settings**: job name, attempts per task, parallel trials, auto-retries, and the spend cap per trial.
-- **Advanced**, collapsed by default: the sandbox provider, the timeout multipliers, the judge overrides for the verifier, and the exceptions to retry on.
-
-The Summary rail counts datasets × agents × attempts. VALIDATE checks the whole form without starting anything. LAUNCH starts the job and opens its page.
-
-## One job
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="A job's page, on its OVERVIEW tab">
-  <img src="/images/job.png" alt="A job's page: the job id and status in the header, CANCEL, RESUME, DOWNLOAD and RE-RUN JOB buttons, tiles for trials, errors, average reward, cost and max cost, and the OVERVIEW tab with a progress bar and a per-agent table." />
-</Frame>
-
-The header carries the job id and its status. CANCEL stops a running job. RESUME runs a finished job's failed or stopped trials again as a new job. Both appear for the job's creator and the organization's members; a job shared with you by address opens without them.
-
-DOWNLOAD saves a finished job's results as a `.tar.gz` in the standard job layout; while the job runs it is disabled, with the reason on hover. RE-RUN JOB opens the New job form filled in with this job's choices.
-
-Five tiles sum the job up: trials settled out of the total, errors, average reward, cost so far, and the most the job can cost. The average reward shows a value for a job with one agent and one dataset; with more, it shows a dash and the OVERVIEW rows carry the means. Below them, five tabs.
-
-- **OVERVIEW**: a progress bar by trial status, and one row per agent and dataset with its scored trials, errors, mean reward and pass@k.
-- **TRIALS**: every trial, with ALL or ERRORS, its task, agent, status, reward, cost, duration and token counts. Click a row to open the [trial viewer](/dashboard/trial-viewer).
-- **ANALYZE**: the trace analyses of the job's trials, below.
-- **CHECK**: the task quality checks of the job's datasets. See [checks](/dashboard/checks).
-- **CONFIG**: the stored job record as JSON: its id, status, counts, stats and resolved settings.
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="The TRIALS tab">
-  <img src="/images/job-trials.png" alt="The TRIALS tab: ALL and ERRORS filters and a table of trials with task, agent, status, reward, cost, duration, input tokens and output tokens." />
-</Frame>
-
-## The ANALYZE tab
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="The ANALYZE tab after every trial was analyzed">
-  <img src="/images/job-analyze.png" alt="The ANALYZE tab: tiles for completed, failed and pending analyses and the analyzer's cost, the policy with model, effort and criteria, and a table of pass, fail, not applicable and unknown counts per criterion." />
-</Frame>
-
-The tab counts completed, failed and pending analyses and the analyzer's own spend. The policy block names the analyzer's model, its reasoning effort and the rubric's criteria. The first table counts, per criterion, how many analyses ruled pass, fail, not applicable or unknown.
-
-Below it, one row per trial carries its label, its verdicts, the analyzer's spend and its summary; click a row to open the analysis in the trial viewer. Start analyses with `evolve analyze <job-id>` or with `--analyze` on `evolve run`, see [analyze](/core-concepts/analyze).

package/skills/evolve-evals/references/dashboard/traces.mdx

@@ -1,17 +0,0 @@
----
-title: "Traces"
-description: "Every recorded run in one list."
----
-
-The Traces page lists every run of the workspace you are in, newest first: the trials of its jobs, their regrades, analyses and checks, and the managed-agent sessions the platform recorded, which these pages do not cover. In a team it is one list of every member's runs, with a CREATED BY column after the trace tag. In your personal workspace it has two tabs: MINE, your own runs in any team, and SHARED WITH ME, what teammates made plus runs shared with you by email.
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="The traces list">
-  <img src="/images/traces.png" alt="The Traces page: a search box and a table of runs with a trace tag, the sandbox provider, the sandbox id, the agent and model, the start time, the cost and the status." />
-</Frame>
-
-Each row carries a tag naming the kind of run (EVAL for a trial, AGENT for a managed-agent session, REGRADE, ANALYZE, or CHECK), the task or session name, the dataset, the reward when there is one, the sandbox provider and sandbox id, the agent and model, when it started, what it spent, and its status. Search by name. Tick rows to stop running ones or delete finished ones; in a team's list and under SHARED WITH ME only the rows you created can be ticked, since stop and delete are the creator's.
-
-Click a row to open it in the [trial viewer](/dashboard/trial-viewer).
-
-A managed-agent session shares the two ways a [job](/core-concepts/jobs#share) does: `evolve session share <id> --link` turns on an unlisted link that opens the session, its transcript and its trace file without signing in, and `--email alice@example.org` shares it with a person by address. `evolve session shares <id>` prints the link and the addresses again, `evolve session unshare <id> --link` kills the link at once, and a session shared with you appears under `evolve session list --scope shared`. Sharing is read-only: neither the link nor an address can stop the session, and a link never reaches the session's file system or its browser replay.

package/skills/evolve-evals/references/dashboard/trial-viewer.mdx

@@ -1,30 +0,0 @@
----
-title: "Trial viewer"
-description: "Read one trial's trace event by event, its tool calls, its spend, and download its artifacts."
----
-
-Click a trial on a job's TRIALS tab, or a row on the Traces page, to open it.
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="A scored trial in the viewer">
-  <img src="/images/trial.png" alt="The trial viewer: a header with the task, agent, model, sandbox id, cost and status, a progress bar with previous and next event controls, an EVENTS list on the left, and the selected event's content on the right." />
-</Frame>
-
-The header names the run: its kind (EVAL for a trial, AGENT for a managed-agent session, REGRADE, ANALYZE, or CHECK), its task, its agent and model, its sandbox id, what it spent, and its status.
-
-BACK returns to where you came from. RETRY runs a settled trial again as a new one-trial job. STOP ends a run that is still going; its creator can stop it.
-
-DOWNLOAD on a trial offers the parsed trace, the ATIF trajectory, the raw stdout, the raw stderr, the agent home folder, the verifier log, or everything at once; on an analysis or a check it offers the verdict document, the raw stdout, the raw stderr, the agent home folder, or everything.
-
-Under the header, the reward and the event navigation: previous, next, and a progress bar you can drag. Type `/` to search the events.
-
-The EVENTS list runs down the left: the prompt, each assistant message and thought, each tool call and its result, and the gateway's own records: one per model call with its tokens and cost, and a run total at the end. Click an event to read it in full on the right. A running trial keeps appending events while you watch.
-
-{/* retake after the UI pass, from a non-admin account */}
-<Frame caption="An analysis run, with the TOOLS panel open">
-  <img src="/images/trial-analysis.png" alt="The trial viewer on an analysis run: a TOOLS summary with the number of calls, the success rate and a bar per tool, then the events list and the selected event's content." />
-</Frame>
-
-The panel above the events has two tabs. TOOLS counts the run's tool calls, its success rate and how many failed, with a bar per tool. SPEND shows what the run spent. The gateway meter at the end of the run states the prompt, cached, cache-write and completion tokens and the cost in dollars.
-
-An analysis, a check or a regrade is read the same way. An analysis's events are the analyzer's own trace, and its verdict document is on the DOWNLOAD menu.

package/skills/evolve-evals/references/getting-started/installation.mdx

@@ -1,56 +0,0 @@
----
-title: "Installation"
-description: "Install the evolve command and the SDKs, and set your API key."
----
-
-## The CLI
-
-The `evolve` command ships inside the npm package `@evolvingmachines/evolve`.
-
-```bash
-npm install -g @evolvingmachines/evolve
-evolve --version
-```
-
-Inside a project that has the package, `npx evolve` runs it. For a one-off without installing, name the package: `npx --package=@evolvingmachines/evolve evolve --help`. A bare `npx evolve` fetches an unrelated package of that name from the public registry.
-
-## The SDKs
-
-The same surface is available as a library in TypeScript and in Python.
-
-<CodeGroup>
-
-```bash TypeScript
-npm install @evolvingmachines/evolve
-```
-
-```bash Python
-pip install evolvingmachines-evolve
-```
-
-</CodeGroup>
-
-The Python package needs Python 3.10 or newer. The hosted clients on these pages talk to the platform directly. Node.js and npm are needed only to run agents on your own machine through `run()`, which these pages do not cover.
-
-## Your API key
-
-Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys), then export it. Every command and every SDK client reads `EVOLVE_API_KEY`.
-
-```bash
-export EVOLVE_API_KEY="<your key>"
-evolve auth status
-```
-
-`auth status` prints who the platform thinks you are and which key is in use. The key itself is never printed back. To use a different key for one call, pass `--api-key <key>` on the command line, or `{ apiKey }` to an SDK client.
-
-## Your coding agent
-
-```bash
-evolve skills install
-```
-
-One small skill, `evolve`, lands in the skill folder of every coding agent on your machine: Claude Code, Codex, Cursor, Copilot, Gemini CLI and OpenCode. From then on, when you mention Evolve, evals, datasets, jobs or tasks, the agent reads the manual from the command itself, `evolve skills get evals`, one page at a time. The manual ships inside the package, so it always matches the version you installed. See [evolve skills](/cli-reference/skills).
-
-<Card title="Quick start" icon="play" href="/getting-started/quick-start">
-  Start your first job.
-</Card>

package/skills/evolve-evals/references/getting-started/quick-start.mdx

@@ -1,94 +0,0 @@
----
-title: "Quick start"
-description: "Start your first job, watch it run, and download a trial."
----
-
-This page takes you from an empty terminal to a downloaded trial: one API key, any harness, any model, any sandbox provider, and every trial's trace streamed live. A trial is one agent's attempt at one task. A job is a set of trials: the tasks of one or more datasets, run by one or more agents.
-
-<Steps>
-  <Step title="Install the CLI and set your key">
-    ```bash
-    npm install -g @evolvingmachines/evolve
-    export EVOLVE_API_KEY="<your key>"
-    evolve auth status
-    ```
-
-    Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys), then export it. See [installation](/getting-started/installation) for the SDKs.
-  </Step>
-
-  <Step title="Give your coding agent the manual">
-    ```bash
-    evolve skills install
-    ```
-
-    Your agent then reads these docs with `evolve skills get evals`, and the `create-task`, `rewardkit`, `create-adapter` and `publish` skills by name. [The five skills](/cli-reference/skills).
-  </Step>
-
-  <Step title="Pick a dataset">
-    The catalog lists every dataset you can run, with its versions.
-
-    ```bash
-    evolve dataset list
-    evolve dataset show terminal-bench-4@4.0
-    ```
-
-    `dataset show` prints the version's tasks and, for each task, which sandbox providers can run it.
-  </Step>
-
-  <Step title="Start a job and watch it">
-    Name a dataset, an agent, and a model. `--watch` streams the job's events until it finishes; `-l 5` caps the run at five tasks and `--max-trial-spend 1` caps what each trial may spend on the model, so the first job is small.
-
-    ```bash
-    evolve run \
-      -d terminal-bench-4@4.0 \
-      -a codex \
-      -m gpt-6-astra \
-      -l 5 \
-      --max-trial-spend 1 \
-      --watch
-    ```
-
-    The final block prints the job id, how many trials landed in each status, and what the trials spent. Without `--watch` the command returns the id at once and the job keeps running.
-  </Step>
-
-  <Step title="Read the result">
-    ```bash
-    evolve job show <job-id>
-    evolve job trials <job-id>
-    evolve trial show <trial-id>
-    ```
-
-    `job show` is the whole job: how many trials landed in each status, the rewards, the spend. `job trials` lists the trials one per line with their ids. `trial show` is one trial in full, with its reward and, when it failed, why.
-  </Step>
-
-  <Step title="Download the trial">
-    A trial downloads as a trial tree: the trajectory, the agent's stdout and stderr, the verifier log, and the reward.
-
-    ```bash
-    evolve trial download <trial-id> -o trials/
-    ```
-
-    To read one artifact without saving anything, stream it instead.
-
-    ```bash
-    evolve trial download <trial-id> --stream verifier
-    ```
-  </Step>
-</Steps>
-
-## Next, by what you want to do
-
-<Columns cols={2}>
-  <Card title="Port a benchmark" icon="arrow-right-left" href="/core-concepts/tasks">
-    `evolve skills get create-adapter` turns it into Harbor-format tasks.
-  </Card>
-  <Card title="Write a task" icon="file-plus" href="/core-concepts/tasks">
-    `evolve skills get create-task`, and `rewardkit` for its verifier.
-  </Card>
-  <Card title="Publish a dataset" icon="upload" href="/core-concepts/datasets">
-    `evolve skills get publish`, then `evolve dataset publish`.
-  </Card>
-  <Card title="Upload a job you ran elsewhere" icon="folder-up" href="/core-concepts/upload">
-    `evolve upload <job dir>` brings a Harbor job into the dashboard.
-  </Card>
-</Columns>

package/skills/evolve-evals/references/index.mdx

@@ -1,48 +0,0 @@
----
-title: "Evolve documentation"
-description: "Environment-driven agent evaluations: any model, any harness, any sandbox, one API key. Every task versioned and checked, every trajectory analyzed."
----
-
-Evolve is hosted evaluation for agents: bring your own tasks or pick a public dataset, and run any model on any agent harness against them, in parallel, in cloud sandboxes. Each task is pinned to one versioned environment, so every run starts from the same place today or next month. Every trial returns its reward, its live trace, its files and its logs, with a quality verdict on any task before you spend on it and a judged analysis of any trajectory after.
-
-- **One API key.** Any model with `-m`, any agent harness with `-a`, any sandbox provider with `-e`; no provider keys to manage.
-- **Task quality control.** `evolve check` runs each task and judges it against a rubric before you spend a job on it.
-- **Full observability.** Every trial streams its trace live; its files, artifacts, agent logs and verifier logs are yours to read and download when it ends.
-- **Trace analysis after every trial.** `evolve analyze` judges every trial's trace against a rubric, cheap and scalable.
-
-```bash
-export EVOLVE_API_KEY="<your key>"   # from https://dashboard.evolvingmachines.ai/api-keys
-evolve run \
-  -d terminal-bench-4@4.0 \
-  -a codex \
-  -m gpt-6-astra \
-  -l 5 \
-  --watch
-```
-
-New here? The [quick start](/getting-started/quick-start) takes five minutes.
-
-## The whole process
-
-1. [Get tasks](/core-concepts/tasks): a public dataset, your own, or a converted benchmark.
-2. [Check](/core-concepts/check) each task against a rubric before you run it.
-3. [Publish](/core-concepts/datasets) the tasks as a versioned dataset.
-4. [Run](/core-concepts/jobs) any agent and model on it; every trial streams live.
-5. [Read](/core-concepts/trial-outputs) every trial's tree and [analyze](/core-concepts/analyze) its trace.
-
-A coding agent does all five from the same CLI with the [skills](/cli-reference/skills).
-
-<Columns cols={2}>
-  <Card title="Quick start" icon="play" href="/getting-started/quick-start">
-    Install the CLI, start a job, read its result.
-  </Card>
-  <Card title="Core concepts" icon="layers" href="/core-concepts">
-    How tasks, jobs, trials, agents and sandboxes fit together.
-  </Card>
-  <Card title="CLI reference" icon="terminal" href="/cli-reference">
-    Every verb and flag of the `evolve` command.
-  </Card>
-  <Card title="Dashboard" icon="layout-dashboard" href="/dashboard/datasets">
-    Datasets, jobs and the trial viewer in the browser.
-  </Card>
-</Columns>

package/skills/evolve-evals/references/sdk/python.mdx

@@ -1,161 +0,0 @@
----
-title: "Python SDK"
-description: "Start a job, wait for it, read the result, and download it from Python."
----
-
-```bash
-pip install evolvingmachines-evolve
-```
-
-The hosted surface is a set of clients, one per noun. Each one reads `EVOLVE_API_KEY` from the environment, or takes a `HostedClientConfig(api_key=..., base_url=...)`. Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys).
-
-Every call is a coroutine, and every client is an async context manager with `close()`. A request never follows a redirect: the client refuses every 3xx and raises it as an error, so the key only reaches the host you configured.
-
-```python
-import asyncio
-from evolve import jobs, trials, datasets
-
-evals = jobs()
-```
-
-## Start a job
-
-```python
-job = await evals.start(
-    datasets=[{"name": "terminal-bench-4", "version": "4.0", "n_tasks": 5}],
-    agents=[{"name": "codex", "model_name": "gpt-6-astra"}],
-    n_attempts=1,
-    n_concurrent_trials=4,
-    max_trial_spend_usd=25,
-)
-
-print(job.id, job.status)   # "QUEUED"
-```
-
-`datasets` is a list of selectors, as dicts or `DatasetSelector` dataclasses. A bare `name` resolves to the active version; `task_names`, `exclude_task_names` and `n_tasks` narrow the task set. Every arm in `agents`, a dict or an `AgentArm`, names a `model_name`, and `version`, `reasoning_effort`, `kwargs`, `preset` and `skills` are optional.
-
-`sandbox_provider` picks `"e2b"`, `"daytona"` or `"modal"`. The retry policy is a `JobRetryConfigInput` dict and comes back resolved as `JobRetryConfig`. The fields are the same ones a `-c` config file takes on the CLI.
-
-Results are dataclasses with the wire's `snake_case` names and `Literal` vocabularies for every closed set. Four keys the wire spells in camelCase are mapped: `next_cursor`, `has_more`, `by_status` and `task_matrix`. A refused request raises `EvolveAPIError`; see [errors](/sdk-reference/errors).
-
-## Wait for it
-
-`watch()` returns a handle you either await, for the final job, or iterate, for each event. Pick one form per call.
-
-```python
-final = await evals.watch(job.id)
-print(final.status, final.trials.by_status, final.stats.get("cost_usd"))
-```
-
-```python
-async for event in evals.watch(job.id):
-    if event.type == "trial.settled":
-        print(event.data["task_name"], event.data["status"], event.data.get("reward"))
-```
-
-The stream replays from the beginning, so attaching late loses nothing, and it resumes on its own after a disconnect. `event.data` is a plain dict; branch on `event.type` and read it by key.
-
-## Read the result
-
-```python
-detail = await evals.get(job.id)
-
-async for trial in evals.trials(job.id):
-    print(trial.task_name, trial.agent_info.name, trial.status, trial.reward)
-
-failures = await evals.trials(
-    job.id,
-    status=["INFRASTRUCTURE_ERROR", "SCORING_ERROR"],
-)
-```
-
-Every list on this surface works the same way: await it for one page, iterate it to walk every page. `pass_at_k(job)` reads a finished job's pass@k out of its stats as sorted numbers.
-
-A trial id is global, so `trials()` reaches one without the job.
-
-```python
-t = trials()
-
-trial = await t.get(trial_id)
-print(trial.reward, trial.exception_info and trial.exception_info.exception_type)
-
-verifier_log = await t.artifact(trial_id, "verifier")   # str | None
-home = await t.artifact(trial_id, "agent-home")         # dict[str, str] | None
-
-async for event in t.trace_events(trial_id):
-    print(event.seq, event.type)
-```
-
-`artifact()` takes the same names as `evolve trial download --stream`. None means the trial never stored that artifact; it is a normal answer, not an error.
-
-## Download
-
-```python
-path = await evals.download(
-    job.id,
-    to="./results",
-)   # the saved file's path
-data = await evals.download(job.id)                    # bytes
-```
-
-The archive is the standard job directory, the same tree `evolve job download` unpacks.
-
-## Analyze
-
-```python
-await evals.analyze(
-    job.id,
-    failing=True,
-    n_trials=20,
-)
-analyzed = await evals.watch_analysis(job.id)
-```
-
-`analyze()` enqueues one analysis per trial and returns at once. `watch_analysis()` follows the batch of analyses to its end.
-
-## The catalog
-
-```python
-catalog = datasets()
-
-async for dataset in catalog.list():
-    print(dataset.name, dataset.active_version and dataset.active_version.version)
-
-version = await catalog.get("terminal-bench-4@4.0")
-```
-
-## Browse the run's files
-
-Every trial has a file system you can read while the box runs and after it is gone, plus its sandbox logs and process list. `filesystem(id)` returns one object for all of it.
-
-```python
-fs = t.filesystem(trial_id)
-print((await fs.status()).state)   # "live" | "captured" | "capturing" | "none"
-
-folder = await fs.list(path="/app/work")
-main = (await fs.read("/app/work/main.py")).decode()
-hits = await fs.search("permission denied", path="/app/work")
-changed = await fs.changes()
-await fs.archive(path="/app/work", to="./archives")
-
-async for frame in fs.events():
-    if frame.event == "fs":
-        print(frame.data["type"], frame.data["path"])
-agent_log = await fs.logs("agent")
-```
-
-Analysis runs (`analyses().filesystem(id)`) and task checks (`checks().task_filesystem(check_id, task_check_id)`) have the same surface; a published task's own files are `datasets().task_files("name@version", task)`.
-
-## Everything else
-
-`analyses()`, `checks()`, `skills()`, `agents()`, `auth()` and `orgs()` are built the same way. `meta()` fetches the platform's capability document, the live list of harnesses, models and limits, and needs no key. `hosted()` builds every client from one configuration.
-
-```python
-from evolve import hosted
-
-evolve = hosted()
-doc = await evolve.meta()
-job = await evolve.jobs.start(...)
-```
-
-Run any of the snippets above inside an `async def main()` and start it with `asyncio.run(main())`.

package/skills/evolve-evals/references/sdk/typescript.mdx

@@ -1,163 +0,0 @@
----
-title: "TypeScript SDK"
-description: "Start a job, wait for it, read the result, and download it from TypeScript."
----
-
-```bash
-npm install @evolvingmachines/evolve
-```
-
-The hosted surface is a set of clients, one per noun. Each one reads `EVOLVE_API_KEY` from the environment, or takes `{ apiKey, baseUrl }`. Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys).
-
-```ts
-import { jobs, trials, datasets } from "@evolvingmachines/evolve";
-
-const evals = jobs();
-```
-
-## Start a job
-
-```ts
-const job = await evals.start({
-  datasets: [{ name: "terminal-bench-4", version: "4.0", n_tasks: 5 }],
-  agents: [{ name: "codex", model_name: "gpt-6-astra" }],
-  n_attempts: 1,
-  n_concurrent_trials: 4,
-  max_trial_spend_usd: 25,
-});
-
-console.log(job.id, job.status);   // "QUEUED"
-```
-
-`datasets` is a list of selectors. A bare `name` resolves to the active version; `task_names`, `exclude_task_names` and `n_tasks` narrow the task set.
-
-Every arm in `agents` names a `model_name`, and `version`, `reasoning_effort`, `kwargs`, `preset` and `skills` are optional. `sandbox_provider` picks `"e2b"`, `"daytona"` or `"modal"`. The fields are the same ones a `-c` config file takes on the CLI.
-
-## Wait for it
-
-`watch()` returns a handle you either await, for the final job, or iterate, for each event. Pick one form per call.
-
-```ts
-const final = await evals.watch(job.id);
-console.log(final.status, final.trials.byStatus, final.stats.cost_usd);
-```
-
-```ts
-for await (const event of evals.watch(job.id)) {
-  if (event.type === "trial.settled") {
-    console.log(event.data.task_name, event.data.status, event.data.reward);
-  }
-}
-```
-
-The stream replays from the beginning, so attaching late loses nothing, and it resumes on its own after a disconnect. `event.type` narrows `event.data`, so the fields above are typed.
-
-## Read the result
-
-```ts
-const detail = await evals.get(job.id);
-
-for await (const trial of evals.trials(job.id)) {
-  console.log(trial.task_name, trial.agent_info.name, trial.status, trial.reward);
-}
-
-const failures = await evals.trials(job.id, {
-  status: ["INFRASTRUCTURE_ERROR", "SCORING_ERROR"],
-});
-```
-
-Every list on this surface works the same way: await it for one page, iterate it to walk every page. `passAtK(job)` reads a finished job's pass@k out of its stats as sorted numbers. A refused request throws `EvolveApiError`; see [errors](/sdk-reference/errors).
-
-A trial id is global, so `trials()` reaches one without the job.
-
-```ts
-const t = trials();
-
-const trial = await t.get(trialId);
-console.log(trial.reward, trial.exception_info?.exception_type);
-
-const verifierLog = await t.artifact(trialId, "verifier");   // string | null
-const home = await t.artifact(
-  trialId,
-  "agent-home",
-);   // Record<string, string> | null
-
-for await (const event of t.traceEvents(trialId)) {
-  console.log(event.seq, event.type);
-}
-```
-
-`artifact()` takes the same names as `evolve trial download --stream`. Null means the trial never stored that artifact; it is a normal answer, not an error.
-
-## Download
-
-```ts
-const path = await evals.download(
-  job.id,
-  { to: "./results" },
-);   // the saved file's path
-const bytes = await evals.download(job.id);                        // a Buffer
-```
-
-The archive is the standard job directory, the same tree `evolve job download` unpacks.
-
-## Analyze
-
-```ts
-await evals.analyze(
-  job.id,
-  {
-    failing: true,
-    n_trials: 20,
-  },
-);
-const analyzed = await evals.watchAnalysis(job.id);
-```
-
-`analyze()` enqueues one analysis per trial and returns at once. `watchAnalysis()` follows the batch of analyses to its end.
-
-## The catalog
-
-```ts
-const catalog = datasets();
-
-for await (const dataset of catalog.list()) {
-  console.log(dataset.name, dataset.active_version?.version);
-}
-
-const version = await catalog.get("terminal-bench-4@4.0");
-```
-
-## Browse the run's files
-
-Every trial has a file system you can read while the box runs and after it is gone, plus its sandbox logs and process list. `filesystem(id)` returns one object for all of it.
-
-```ts
-const fs = t.filesystem(trialId);
-console.log((await fs.status()).state);   // "live" | "captured" | "capturing" | "none"
-
-const folder = await fs.list({ path: "/app/work" });
-const main = (await fs.read("/app/work/main.py")).toString("utf8");
-const hits = await fs.search({ q: "permission denied", path: "/app/work" });
-const changed = await fs.changes();
-await fs.archive({ path: "/app/work", to: "./archives" });
-
-for await (const frame of fs.events()) {
-  if (frame.event === "fs") console.log(frame.data.type, frame.data.path);
-}
-const agentLog = await fs.logs({ stream: "agent" });
-```
-
-Analysis runs (`analyses().filesystem(id)`) and task checks (`checks().taskFilesystem(checkId, taskCheckId)`) have the same surface; a published task's own files are `datasets().taskFiles("name@version", task)`.
-
-## Everything else
-
-`analyses()`, `checks()`, `skills()`, `agents()`, `auth()` and `orgs()` are built the same way. `meta()` fetches the platform's capability document, the live list of harnesses, models and limits, and needs no key. `hosted()` builds every client from one configuration.
-
-```ts
-import { hosted } from "@evolvingmachines/evolve";
-
-const evolve = hosted({ apiKey: process.env.EVOLVE_API_KEY });
-const doc = await evolve.meta();
-const job = await evolve.jobs.start({ /* … */ });
-```