@evolvingmachines/evolve 0.0.55-project-sable.20260916.db0bc0bb

package/docs-agents/typescript/index.md

@@ -0,0 +1,121 @@
+# Evolve TypeScript SDK
+
+Run CLI agents in secure sandboxes with built-in observability.
+
+```bash
+npm install @evolvingmachines/evolve
+```
+
+```ts
+import { Evolve } from "@evolvingmachines/evolve";
+
+const evolve = new Evolve();
+await evolve.run({ prompt: "Hello world" });
+```
+
+---
+
+## Reference
+
+### Getting Started
+
+| Topic | Link |
+|-------|------|
+| Installation (Node.js 18+, npm) | [Getting Started → Installation](./01-getting-started.md#installation) |
+| Quick Start (3 steps) | [Getting Started → Quick Start](./01-getting-started.md#quick-start) |
+| Core Lifecycle (run → getOutputFiles → kill) | [Getting Started → Core Lifecycle](./01-getting-started.md#core-lifecycle) |
+| Streaming basics | [Getting Started → Streaming](./01-getting-started.md#streaming) |
+| Gateway, managed BYO provider keys, and direct keys | [Getting Started → Authentication](./01-getting-started.md#authentication) |
+| Gateway mode (EVOLVE_API_KEY) | [Getting Started → Gateway Mode](./01-getting-started.md#gateway-mode-evolve_api_key) |
+| Managed BYO provider keys | [Getting Started → Managed BYO Provider Keys](./01-getting-started.md#managed-byo-provider-keys) |
+| Managed secrets | [Configuration → Managed Secrets](./02-configuration.md#managed-secrets) |
+| Direct provider key mode (local BYOK) | [Getting Started → Direct Provider Key Mode](./01-getting-started.md#direct-provider-key-mode-local-byok) |
+| BYO Claude Max / Codex / Gemini subscription | [Getting Started → BYO Claude Max Subscription](./01-getting-started.md#byo-claude-max-subscription) |
+| Supported agents & models | [Getting Started → Agent Reference](./01-getting-started.md#agent-reference) |
+| Agent-specific options (reasoningEffort) | [Getting Started → Agent Reference](./01-getting-started.md#agent-reference) |
+
+### Configuration
+
+| Topic | Link |
+|-------|------|
+| Sandbox providers (E2B, Modal, Daytona) | [Configuration → Sandbox Providers](./02-configuration.md#sandbox-providers) |
+| Provider auto-resolution from env | [Configuration → Sandbox Providers](./02-configuration.md#auto-resolution) |
+| `.withSandboxCreateOptions()` (image, network, user, homeDir) | [Configuration → Sandbox Create Options](./02-configuration.md#sandbox-create-options) |
+| `.withWorkspaceMode()` (knowledge / swe) | [Configuration → Workspace Modes](./02-configuration.md#workspace-modes) |
+| Evolve instance (full `.with*()` API) | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withAgent()` | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withSandbox()` | [Configuration → Sandbox Providers](./02-configuration.md#sandbox-providers) |
+| `.withContext()` / `.withFiles()` | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withSystemPrompt()` | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withSchema()` (Zod / JSON Schema) | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withBrowser()` browser guide | [Configuration → Browser Automation](./02-configuration.md#browser-automation) |
+| `.withBrowserCredentials()` browser logins | [Configuration → Browser Credentials](./02-configuration.md#browser-credentials) |
+| `.withPlugins()` | [Configuration → Agent Plugins](./02-configuration.md#agent-plugins) |
+| `.withSkills()` | [Configuration → Agent Skills](./02-configuration.md#agent-skills) |
+| `.withIntegrations()` (managed app integrations) | [Configuration → Managed Integrations](./02-configuration.md#managed-integrations) |
+| `.withManagedSecrets()` | [Configuration → Managed Secrets](./02-configuration.md#managed-secrets) |
+| `.withMcpServers()` (STDIO / HTTP / SSE) | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withSecrets()` | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+| `.withStorage()` (checkpointing) | [Runtime → Storage & Checkpointing](./03-runtime.md#storage--checkpointing) |
+| `.withSessionTagPrefix()` (observability) | [Runtime → Observability](./03-runtime.md#observability) |
+| Skills catalog (PDF, browser, research, etc.) | [Configuration → Agent Skills](./02-configuration.md#agent-skills) |
+| Integration auth/account helpers | [Configuration → Managed Integrations](./02-configuration.md#managed-integrations) |
+| Integration tool filtering | [Configuration → Managed Integrations](./02-configuration.md#managed-integrations) |
+| MCP server config (McpServerConfig) | [Configuration → Evolve Instance](./02-configuration.md#evolve-instance) |
+
+### Runtime
+
+| Topic | Link |
+|-------|------|
+| `run()` | [Runtime → run](./03-runtime.md#run) |
+| `executeCommand()` | [Runtime → executeCommand](./03-runtime.md#executecommand) |
+| Streaming events (content, lifecycle, stdout, stderr) | [Streaming Events](./04-streaming.md) |
+| OutputEvent / SessionUpdate types | [Streaming → SessionUpdate Types](./04-streaming.md#sessionupdate-types) |
+| LifecycleEvent / LifecycleReason | [Streaming → LifecycleEvent](./04-streaming.md#lifecycleevent) |
+| Tool events (ToolCall, ToolCallUpdate, ToolKind) | [Streaming → Tool Events](./04-streaming.md#tool-events) |
+| Browser lifecycle event fields | [Streaming → Browser Automation Streaming](./04-streaming.md#browser-automation-streaming) |
+| UI integration example | [Streaming → UI Integration Example](./04-streaming.md#ui-integration-example) |
+| Upload files (`uploadContext()`, `uploadFiles()`) | [Runtime → Upload](./03-runtime.md#upload-local--sandbox) |
+| Download files (`getOutputFiles()`, `saveLocalDir()`) | [Runtime → Download](./03-runtime.md#download-sandbox--local) |
+| Session controls (interrupt, pause, resume, kill) | [Runtime → Session Controls](./03-runtime.md#session-controls) |
+| `getHost()` (port forwarding) | [Runtime → getHost](./03-runtime.md#gethost) |
+| Workspace filesystem layout | [Runtime → Workspace & Structured Output](./03-runtime.md#workspace--structured-output) |
+| Structured output (Zod / JSON Schema) | [Runtime → Workspace & Structured Output](./03-runtime.md#structured-output) |
+| Multi-turn conversations | [Runtime → Session Management](./03-runtime.md#session-management) |
+| Pause / resume | [Runtime → Session Management](./03-runtime.md#session-management) |
+| Save and reconnect (`withSession`, `setSession`) | [Runtime → Session Management](./03-runtime.md#session-management) |
+| Storage & checkpointing (gateway) | [Runtime → Storage & Checkpointing](./03-runtime.md#storage--checkpointing) |
+| Auto-checkpoint, explicit checkpoint, restore | [Runtime → Storage & Checkpointing](./03-runtime.md#auto-checkpoint-via-run) |
+| Checkpoint lineage | [Runtime → Checkpoint Lineage](./03-runtime.md#checkpoint-lineage) |
+| `storage()` client, `evolve.storage()` | [Runtime → Listing & Browsing Checkpoints](./03-runtime.md#listing--browsing-checkpoints) |
+| `sessions()` client for past sessions & traces | [Runtime → Historical Sessions & Trace Download](./03-runtime.md#historical-sessions--trace-download) |
+| Observability (dashboard + local logs) | [Runtime → Observability](./03-runtime.md#observability) |
+| Error handling | [Runtime → Error Handling](./03-runtime.md#error-handling) |
+
+### Hosted evals
+
+Datasets, jobs, trials, checks, analyses and the `evolve` CLI are documented at https://docs.evolvingmachines.ai. The `evolve-evals` skill carries the same pages.
+
+### Swarm & Pipeline
+
+| Topic | Link |
+|-------|------|
+| Swarm setup (SwarmConfig) | [Swarm & Pipeline](./05-swarm-pipeline.md) |
+| Input types (FileMap, folders, chaining) | [Swarm → Input Types](./05-swarm-pipeline.md#input-types) |
+| `bestOf()` (N candidates + judge) | [Swarm → bestOf](./05-swarm-pipeline.md#bestof) |
+| `map()` (parallel processing) | [Swarm → map](./05-swarm-pipeline.md#map) |
+| `map()` + bestOf | [Swarm → map + bestOf](./05-swarm-pipeline.md#map--bestof) |
+| `filter()` (evaluate + threshold) | [Swarm → filter](./05-swarm-pipeline.md#filter) |
+| `reduce()` (synthesize many → one) | [Swarm → reduce](./05-swarm-pipeline.md#reduce) |
+| `verify` (quality gate with feedback loop) | [Swarm → verify](./05-swarm-pipeline.md#verify-quality-gate) |
+| Result types (SwarmResult, ReduceResult, BestOfResult) | [Swarm → Result Types](./05-swarm-pipeline.md#result-types) |
+| Chaining operations (result.json → data.json) | [Swarm → Chaining Operations](./05-swarm-pipeline.md#chaining-operations) |
+| AgentOverride (per-operation agent config) | [Swarm → AgentOverride](./05-swarm-pipeline.md#agentoverride) |
+| Concurrency (semaphore, ordering guarantees) | [Swarm → Concurrency](./05-swarm-pipeline.md#concurrency) |
+| RetryConfig (exponential backoff) | [Swarm & Pipeline](./05-swarm-pipeline.md) |
+| Pipeline (fluent chaining) | [Swarm → Pipeline](./05-swarm-pipeline.md#pipeline) |
+| Pipeline step configs (map, filter, reduce) | [Swarm → Step Configurations](./05-swarm-pipeline.md#step-configurations) |
+| Pipeline events (stepStart, stepComplete, etc.) | [Swarm → Events](./05-swarm-pipeline.md#events) |
+| Pipeline result (PipelineResult) | [Swarm → Result](./05-swarm-pipeline.md#result) |
+| TerminalPipeline (reduce is terminal) | [Swarm → Terminal Pipeline](./05-swarm-pipeline.md#terminal-pipeline) |
+| Filter `emit` option (success / filtered / all) | [Swarm → Pipeline](./05-swarm-pipeline.md#step-configurations) |

package/docs-evals/SKILL.md

@@ -0,0 +1,112 @@
+---
+# Generated by scripts/generate-skills.ts from docs-evals/docs.json. Do not edit; edit the source and run npm run generate:skills.
+name: docs-evals
+description: "Evolve hosted evals: run any model on any agent harness against datasets of Harbor-format tasks, in cloud sandboxes, from the evolve CLI and the TypeScript and Python SDKs. Use when the user wants to: (1) start, watch, or read a hosted eval job (evolve run, evolve job, jobs()), (2) browse the dataset catalog or publish their own dataset (evolve dataset), (3) check task quality (evolve check) or analyze trial traces (evolve analyze), (4) inspect or download trials, artifacts and traces (evolve trial, trials()), (5) register their own agent, mount skills, attach secrets, or choose models and sandbox providers, (6) upload a job run elsewhere (evolve upload), (7) use the dashboard's jobs, datasets, checks, traces and trial viewer pages. For running agents in sandboxes from code (the SDK's run(), Swarm, Pipeline) read `evolve skills get agents`."
+metadata:
+  internal: true
+---
+
+# Evolve hosted evals
+
+Hosted evaluation for agents: datasets of Harbor-format tasks, jobs that run any model on any agent harness against them in cloud sandboxes, and the trials, checks and analyses they produce — from the `evolve` CLI and the TypeScript and Python SDKs.
+
+This folder is the documentation site itself, page for page: every row below names a page by its site path, the file `<path>.mdx` beside this one, and `evolve skills get evals <path>` prints it. An `import` of `/snippets/<file>` is `snippets/<file>`.
+
+## How to use this skill
+
+1. Find the topic in the index below and read that page before writing any command or code: `evolve skills get evals <page>` prints it, the page named by its site path; `evolve skills get evals --full` prints every page.
+2. Every CLI verb is documented from its own `--help`; run `evolve <verb> --help` to confirm the flags of the installed version.
+3. Every command and every SDK client reads `EVOLVE_API_KEY`; the Installation page says where the key comes from.
+
+## Topic index
+
+## Docs
+
+### Getting started
+
+| Page | What it covers |
+| --- | --- |
+| [Evolve documentation](/index) | Environment-driven agent evaluations: any model, any harness, any sandbox, one API key. Every task versioned and checked, every trajectory analyzed. |
+| [Installation](/getting-started/installation) | Install the evolve command and the SDKs, and set your API key. |
+| [Quick start](/getting-started/quick-start) | Start your first job, watch it run, and download a trial. |
+
+### Core concepts
+
+| Page | What it covers |
+| --- | --- |
+| [Core concepts](/core-concepts/index) | How a dataset becomes trials, rewards, analyses and results. |
+| [Tasks](/core-concepts/tasks) | The task format: one directory with an instruction, an environment, a verifier, and an optional reference solution. |
+| [Datasets](/core-concepts/datasets) | The catalog, and publishing your own dataset. |
+| [Jobs](/core-concepts/jobs) | Start a job, watch it, read its results, and derive new jobs from it. |
+| [Trials](/core-concepts/trials) | One agent's attempt at one task: its record, its trace, and its artifacts. |
+| [Trial outputs](/core-concepts/trial-outputs) | What a finished trial gives you, file by file, and in which formats. |
+| [Agents](/core-concepts/agents) | The built-in harnesses, how to configure an arm, and how to register your own agent. |
+| [Models](/core-concepts/models) | How a model is named on -m, and why one API key covers every provider. |
+| [Sandboxes](/core-concepts/sandboxes) | Where trials run: e2b, daytona, or modal. |
+| [Analyze](/core-concepts/analyze) | Judge every trial's trace against a rubric, on the platform. |
+| [Check](/core-concepts/check) | Check the quality of tasks against a rubric before you spend a job on them. |
+| [Skills](/core-concepts/skills) | Mount skills into every agent run, from a git source or from your uploads. |
+| [Secrets](/core-concepts/secrets) | Store env secrets once and attach them to jobs by name. |
+| [Upload a job](/core-concepts/upload) | Bring a job you ran elsewhere onto the platform as a finished job. |
+
+### Dashboard
+
+| Page | What it covers |
+| --- | --- |
+| [Datasets](/dashboard/datasets) | Browse the catalog, and each dataset's tasks and versions, in the dashboard. |
+| [Jobs](/dashboard/jobs) | Start a job from the browser, follow it, and read its results. |
+| [Trial viewer](/dashboard/trial-viewer) | Read one trial's trace event by event, its tool calls, its spend, and download its artifacts. |
+| [Checks](/dashboard/checks) | Read a job's task quality checks in the dashboard. |
+| [Traces](/dashboard/traces) | Every recorded run in one list. |
+
+### SDK
+
+| Page | What it covers |
+| --- | --- |
+| [TypeScript SDK](/sdk/typescript) | Start a job, wait for it, read the result, and download it from TypeScript. |
+| [Python SDK](/sdk/python) | Start a job, wait for it, read the result, and download it from Python. |
+
+## CLI reference
+
+### The evolve command
+
+| Page | What it covers |
+| --- | --- |
+| [CLI reference](/cli-reference/index) | Every verb of the evolve command, from its own help output. |
+| [evolve run](/cli-reference/run) | Start a job, and optionally follow it to the end. |
+| [evolve analyze](/cli-reference/analyze) | Analyze a finished job's trial traces against a rubric, on the platform. |
+| [evolve check](/cli-reference/check) | Check task quality against a rubric, and read checks back. |
+| [evolve upload](/cli-reference/upload) | Upload a job directory as a finished job. |
+
+### Command groups
+
+| Page | What it covers |
+| --- | --- |
+| [evolve job](/cli-reference/job) | Start, follow, and derive jobs. |
+| [evolve trial](/cli-reference/trial) | Inspect, download, and act on single trials. |
+| [evolve analysis](/cli-reference/analysis) | List, inspect, and download trace-analysis runs. |
+| [evolve dataset](/cli-reference/dataset) | Browse and publish the dataset catalog. |
+| [evolve skill](/cli-reference/skill) | Upload and manage platform-stored skills. |
+| [evolve skills](/cli-reference/skills) | The skills the CLI serves to coding agents. |
+| [evolve agent](/cli-reference/agent) | Register and manage your own agents. |
+| [evolve auth](/cli-reference/auth) | Identity, API keys, and your organizations. |
+| [evolve secrets](/cli-reference/secrets) | Store and manage env secrets. Values are write-only; reads return metadata. |
+
+## SDK reference
+
+### Clients
+
+| Page | What it covers |
+| --- | --- |
+| [Clients](/sdk-reference/index) | How the hosted clients are built and configured. |
+| [jobs](/sdk-reference/jobs) | Start, follow, and derive jobs. |
+| [trials](/sdk-reference/trials) | Read, download, and act on single trials. |
+| [datasets](/sdk-reference/datasets) | The catalog: list, read, publish, download, activate, delete. |
+| [analyses](/sdk-reference/analyses) | Read trace-analysis runs. |
+| [checks](/sdk-reference/checks) | Start and read task quality checks. |
+| [skills](/sdk-reference/skills) | Upload and manage platform-stored skills. |
+| [agents](/sdk-reference/agents) | Register and manage your own agents. |
+| [auth and orgs](/sdk-reference/auth) | Who you are, and your organizations. |
+| [meta](/sdk-reference/meta) | The platform's capability document, no key needed. |
+| [Types](/sdk-reference/types) | Every field of the objects the clients return: name, type, meaning, and when it is null. |
+| [Errors](/sdk-reference/errors) | The error class every refused request raises, and every code it can carry. |

package/docs-evals/cli-reference/agent.mdx

@@ -0,0 +1,67 @@
+---
+title: "evolve agent"
+description: "Register and manage your own agents."
+---
+
+import GlobalOptions from '/snippets/global-options.mdx';
+import ListFlags from '/snippets/list-flags.mdx';
+
+```bash
+evolve agent <command> [options]
+```
+
+## agent list
+
+```bash
+evolve agent list [options]
+```
+
+List your registered agents.
+
+<ListFlags />
+
+## agent show
+
+```bash
+evolve agent show <name>
+```
+
+Show one registered agent.
+
+## agent add
+
+```bash
+evolve agent add <name> [options]
+```
+
+Register an agent from an install script or a local directory.
+
+<ParamField path="--install-script" type="path">
+  Install script file. Its contents are uploaded.
+</ParamField>
+
+<ParamField path="--dir" type="path">
+  Local agent directory, tarred and uploaded.
+</ParamField>
+
+<ParamField path="--run" type="command" required>
+  Run command, executed with `sh -c`.
+</ParamField>
+
+<ParamField path="--ae, --agent-env" type="KEY=VALUE">
+  Environment injected at run time. Repeatable. It may not override the six run-contract keys, and a credential-shaped key is refused, both with `agent_invalid_env`; see [the run contract](/core-concepts/agents#the-run-contract).
+</ParamField>
+
+```bash
+evolve agent add acme-cli --install-script ./install.sh --run "acme-cli --headless"
+```
+
+## agent remove
+
+```bash
+evolve agent remove <name>
+```
+
+Delete a registered agent. Past jobs keep their record.
+
+<GlobalOptions />

package/docs-evals/cli-reference/analysis.mdx

@@ -0,0 +1,91 @@
+---
+title: "evolve analysis"
+description: "List, inspect, and download trace-analysis runs."
+---
+
+import GlobalOptions from '/snippets/global-options.mdx';
+import ListFlags from '/snippets/list-flags.mdx';
+
+```bash
+evolve analysis <command> [options]
+```
+
+`show`, `trace` and `download` take an analysis id, or a trial id. An analysis id (or its unambiguous prefix) names that run. A trial id (or prefix) names the trial's latest analysis, the one `trial show` prints on its analysis row. A prefix matching both an analysis and a trial is refused as ambiguous, naming both.
+
+A trial nobody analyzed is refused with the remedy, `evolve analyze <job>`, and so is a regrade result's id. An id you cannot read answers `analysis_not_found`; `download` on a run still queued or running answers `analysis_not_terminal`.
+
+## analysis list
+
+```bash
+evolve analysis list [options]
+```
+
+List analysis runs, newest first, with the trial, job and task each one judged.
+
+<ListFlags />
+
+<ParamField path="--scope" type="my | shared">
+  Visibility scope: `my`, what you created (the default), or `shared`, your organizations' rows that teammates created.
+</ParamField>
+
+<ParamField path="--job" type="job-id">
+  Only analyses of this job's trials. An id, or an unambiguous prefix.
+</ParamField>
+
+<ParamField path="--status" type="s1,s2,...">
+  Filter by analysis status: `queued`, `running`, `completed`, `failed`.
+</ParamField>
+
+```bash
+evolve analysis list --job 3e1f9a2c-… --status failed
+```
+
+## analysis show
+
+```bash
+evolve analysis show <analysis-id | trial-id>
+```
+
+Show one analysis run in full: the verdict document.
+
+## analysis trace
+
+```bash
+evolve analysis trace <analysis-id | trial-id> [options]
+```
+
+Print the analyzer's own parsed transcript.
+
+<ParamField path="--since" type="n">
+  Skip the first N events. To resume, pass the count you already hold; sequence numbers are dense from 0, so N is also the next one.
+</ParamField>
+
+## analysis download
+
+```bash
+evolve analysis download <analysis-id | trial-id> [options]
+```
+
+Save an analysis run as a trial folder plus `evolve.json`, or stream one artifact.
+
+<ParamField path="-o, --output-dir" type="dir" default="analyses/">
+  Directory to save under. The run lands in `<dir>/analyze-<analyzed trial>__<7 chars>/`, the run's own folder name, read from the archive.
+</ParamField>
+
+<ParamField path="--overwrite">
+  Replace an existing folder of the same name.
+</ParamField>
+
+<ParamField path="--stream" type="artifact">
+  Print one artifact to stdout instead of saving: `analysis` (the verdict document), `trace-parsed`, `trace-stdout`, `trace-stderr` or `agent-home`.
+</ParamField>
+
+<ParamField path="--since" type="n">
+  With `--stream trace-parsed`: skip the first N events.
+</ParamField>
+
+```bash
+evolve analysis download a0a1b2c3-… --stream trace-stdout
+```
+
+<GlobalOptions />

package/docs-evals/cli-reference/analyze.mdx

@@ -0,0 +1,74 @@
+---
+title: "evolve analyze"
+description: "Analyze a finished job's trial traces against a rubric, on the platform."
+---
+
+import GlobalOptions from '/snippets/global-options.mdx';
+
+```bash
+evolve analyze <job-id> [options]
+```
+
+Analyze a terminal job's trial traces against a rubric, server-side. Without `--watch` the command returns at once with the job block and the hint `Follow it with: evolve job show <id>`; `--json` prints the accepted job body. Exit 0 on an accepted wave; with `--watch`, exit 1 when any analysis failed.
+
+```bash
+evolve analyze 3e1f9a2c-… \
+  -r rubric.toml \
+  -p prompt.txt \
+  --failing \
+  -l 20 \
+  -n 2 \
+  --watch
+```
+
+## Options
+
+<ParamField path="-m, --model" type="name" default="openrouter/deepseek/deepseek-v4.1-flash">
+  Model the analyzer runs. `glm-5.3-flash` and `haiku` are alternatives, `glm-5.3` escalates, and `fireworks/deepseek-v4.1-flash` is the same model on its Fireworks route. Must be on the `claude` roster.
+</ParamField>
+
+<ParamField path="--effort" type="value">
+  Reasoning effort the analyzer runs at. Defaults to the per-model default: `high` on `openrouter/deepseek/deepseek-v4.1-flash`, `max` on `glm-5.3-flash`.
+</ParamField>
+
+<ParamField path="-r, --rubric" type="path">
+  Rubric file: TOML (one `[[criteria]]` table per criterion), YAML or JSON in the `{criteria: [{name, description, guidance}]}` shape, parsed at the keyboard with unknown fields refused by name; the server owns the bounds. Defaults to the platform's analyze rubric, seven criteria with `score_is_earned` first.
+</ParamField>
+
+<ParamField path="-p, --prompt" type="path">
+  Prompt file for the evaluator agent; its text replaces the built-in prompt, with `{trial_path}`, `{task_section}` and `{criteria_guidance}` rendered, `{{` and `}}` writing a literal brace, and an unknown `{token}` rendering empty. The instructions for writing the result are appended after it. An empty file is refused; there is no length bound. Defaults to the built-in prompt.
+</ParamField>
+
+<ParamField path="-e, --env" type="provider">
+  Sandbox provider the analyzer runs on. Defaults to `daytona`.
+</ParamField>
+
+<ParamField path="-n, --n-concurrent" type="n">
+  Max concurrent trial analyses, 1 to 150, beneath the organization's `max_concurrent_analyses` (fleet default 60). Defaults to the ceiling.
+</ParamField>
+
+<ParamField path="--passing">
+  Only analyze passing trials (reward 1.0).
+</ParamField>
+
+<ParamField path="--failing">
+  Only analyze failing trials (reward below 1.0, or an exception).
+</ParamField>
+
+<ParamField path="-l, --n-trials" type="n">
+  Max trials to analyze, after `--passing` or `--failing`, in the job's trial order.
+</ParamField>
+
+<ParamField path="--watch">
+  Poll until every analysis settles, then print one row per analyzed trial: the label (`custom rubric` when there is none), the criterion outcomes, the analyzer's cost and a summary excerpt, with failed analyses listed below. With `--json`, NDJSON envelopes: `analysis.accepted`, `analysis.stats` per change, `analysis.final` with the job and the analyzed trials.
+</ParamField>
+
+<ParamField path="-q, --quiet">
+  With `--watch`: suppress the progress lines and print the final block only.
+</ParamField>
+
+<GlobalOptions />
+
+## Reading analyses back
+
+The [`analysis`](/cli-reference/analysis) group lists, shows, traces and downloads analysis runs.

package/docs-evals/cli-reference/auth.mdx

@@ -0,0 +1,60 @@
+---
+title: "evolve auth"
+description: "Identity, API keys, and your organizations."
+---
+
+import GlobalOptions from '/snippets/global-options.mdx';
+
+```bash
+evolve auth <command> [options]
+```
+
+## auth status
+
+```bash
+evolve auth status
+```
+
+Who am I: the caller and the API key in use.
+
+## auth org list
+
+```bash
+evolve auth org list [options]
+```
+
+List the organizations you belong to.
+
+<ParamField path="--search" type="text">
+  Free-text filter over slug, display name and role.
+</ParamField>
+
+<ParamField path="--columns" type="keys | all | help">
+  Choose and order columns, comma-separated. `help` lists the keys.
+</ParamField>
+
+<ParamField path="-q, --quiet">
+  Print only slugs, one per line, for piping.
+</ParamField>
+
+<ParamField path="--no-trunc">
+  Full cell content instead of one-line truncation.
+</ParamField>
+
+<ParamField path="--no-headers">
+  Omit the header row in piped (TSV) output.
+</ParamField>
+
+## auth org show
+
+```bash
+evolve auth org show <slug>
+```
+
+Show one organization: your role, its members, its quota and live usage.
+
+```bash
+evolve auth org show acme
+```
+
+<GlobalOptions />

package/docs-evals/cli-reference/check.mdx

@@ -0,0 +1,146 @@
+---
+title: "evolve check"
+description: "Check task quality against a rubric, and read checks back."
+---
+
+import GlobalOptions from '/snippets/global-options.mdx';
+import ListFlags from '/snippets/list-flags.mdx';
+
+```bash
+evolve check [<path>] [options]
+```
+
+Check task quality against a rubric, server-side. `<path>` is one task directory or a directory of task directories, uploaded as one archive (`upload_too_large` past the published ceiling); `-d` names a published dataset instead, one or the other. Without `--watch` the command prints the accepted check with the hint `Follow it with: evolve check show <id>`; `--json` prints the accepted check body. Exit 0 on an accepted check; with `--watch`, and on `check show`, exit 1 when any task errored.
+
+```bash
+evolve check ./tasks \
+  -i 'abs-*' \
+  -l 5 \
+  --watch
+```
+
+## Options
+
+<ParamField path="-d, --dataset" type="name[@version]">
+  Check a published dataset's tasks instead of a local path. The checker reads the version's retained task package, and the job page's CHECK tab lists these checks by task.
+</ParamField>
+
+<ParamField path="-m, --model" type="name" default="openrouter/deepseek/deepseek-v4.1-flash">
+  Model the checker runs. `fireworks/deepseek-v4.1-flash` is the same model on its Fireworks route. Must be on the `claude` roster.
+</ParamField>
+
+<ParamField path="--effort" type="value">
+  Reasoning effort the checker runs at. Defaults to the per-model default.
+</ParamField>
+
+<ParamField path="-r, --rubric" type="path">
+  Rubric file: TOML, YAML or JSON in the `{criteria: [{name, description, guidance}]}` shape. Defaults to the platform's check rubric, eleven criteria.
+</ParamField>
+
+<ParamField path="-p, --prompt" type="path">
+  Prompt file for the evaluator agent. Its text replaces the built-in check prompt, with `{task_path}`, `{file_tree}` and `{criteria_guidance}` rendered. Defaults to the built-in prompt.
+</ParamField>
+
+<ParamField path="-e, --env" type="provider">
+  Sandbox provider the checker runs on. Defaults to the platform's analysis default.
+</ParamField>
+
+<ParamField path="-n, --n-concurrent" type="n">
+  Max concurrent task checks, beneath the organization's ceiling. Defaults to the ceiling.
+</ParamField>
+
+<ParamField path="-i, --include-task-name" type="glob">
+  Only check tasks matching the glob. Repeatable.
+</ParamField>
+
+<ParamField path="-x, --exclude-task-name" type="glob">
+  Skip tasks matching the glob. Repeatable.
+</ParamField>
+
+<ParamField path="-l, --n-tasks" type="n">
+  Max tasks to check, after the globs, in sorted directory order.
+</ParamField>
+
+<ParamField path="--watch">
+  Poll until every task check settles, then print the report. One task prints its label line, a `CHECK | OUTCOME | EXPLANATION` table with the evidence beneath each verdict, and the cost; several tasks print a `TASK | LABEL | PASS | FAIL | N/A | UNKNOWN | COST ($) | TASK CHECK ID` table with each failed task's reason below it. With `--json`, NDJSON envelopes: `check.accepted`, `check.progress` per change, `check.final`.
+</ParamField>
+
+<ParamField path="-q, --quiet">
+  With `--watch`: suppress the progress lines and print the final report only.
+</ParamField>
+
+<GlobalOptions />
+
+## Reading checks back
+
+The same word, followed by `list`, `show`, `trace` or `download`, reads checks back. A task directory literally named like one of these verbs is written `./list`.
+
+### check list
+
+```bash
+evolve check list [options]
+```
+
+List your task quality checks, newest first.
+
+<ListFlags />
+
+<ParamField path="--scope" type="my | shared">
+  Visibility scope: `my`, what you created (the default), or `shared`, your organizations' rows that teammates created.
+</ParamField>
+
+<ParamField path="--status" type="s1,s2,...">
+  Filter by check status: `queued`, `running`, `completed`.
+</ParamField>
+
+```bash
+evolve check list --status running
+```
+
+### check show
+
+```bash
+evolve check show <check-id>
+```
+
+Show one task quality check in full, with one result per task.
+
+### check trace
+
+```bash
+evolve check trace <task-check-id> [options]
+```
+
+Print the checker's own parsed transcript for one task check.
+
+<ParamField path="--since" type="n">
+  Skip the first N events. To resume, pass the count you already hold.
+</ParamField>
+
+### check download
+
+```bash
+evolve check download <check-id | task-check-id> [options]
+```
+
+Save a check as a check folder (`check_report.json` plus one folder per task) by its check id, one task check's folder by its task check id, or stream one task check's artifact.
+
+<ParamField path="-o, --output-dir" type="dir" default="checks/">
+  Directory to save under. A check id lands in `<dir>/check-<id>/`, a task check id in `<dir>/check-<task>__<7 chars>/`, the run's own folder name.
+</ParamField>
+
+<ParamField path="--overwrite">
+  Replace an existing folder of the same name.
+</ParamField>
+
+<ParamField path="--stream" type="artifact">
+  Print one artifact of a task check to stdout instead of saving: `task-check` (the result document), `trace-parsed`, `trace-stdout`, `trace-stderr` or `agent-home`.
+</ParamField>
+
+<ParamField path="--since" type="n">
+  With `--stream trace-parsed`: skip the first N events.
+</ParamField>
+
+```bash
+evolve check download 3f9a1c2e-… -o checks/
+```