@evolvingmachines/evolve 0.0.65 → 0.0.67-train-harness-wave-2026-09-25.20260926.e00a8a6d

package/skills/evolve-evals/SKILL.md

@@ -1,113 +1,186 @@
 ---
 # Generated by scripts/generate-skills.ts from docs-evals/. Do not edit; edit the source and run npm run generate:skills.
 name: evolve-evals
-description: "Evolve hosted evals: run any model on any agent harness against datasets of Harbor-format tasks, in cloud sandboxes, from the evolve CLI and the TypeScript and Python SDKs. Use when the user wants to: (1) start, watch, or read a hosted eval job (evolve run, evolve job, jobs()), (2) browse the dataset catalog or publish their own dataset (evolve dataset), (3) check task quality (evolve check) or analyze trial traces (evolve analyze), (4) inspect or download trials, artifacts and traces (evolve trial, trials()), (5) register their own agent, mount skills, attach secrets, or choose models and sandbox providers, (6) upload a job run elsewhere (evolve upload), (7) use the dashboard's jobs, datasets, checks, traces and trial viewer pages. For running agents in sandboxes from code (the SDK's run(), Swarm, Pipeline) install the SDK skill: `npx skills add evolving-machines-lab/evolve --skill evolve-agents`."
+description: "Run managed evaluations with the Evolve CLI and Python or TypeScript SDK. Start and inspect jobs, publish Harbor-format datasets, configure agents and models, read trials and files, analyze traces, check task quality, and manage teams, sharing, skills, and secrets. For running agents directly with the SDK builder, Swarm, or Pipeline, use the separate evolve-agents skill."
 metadata:
   internal: true
 ---
 
-# Evolve hosted evals
+# Evolve managed evals
 
-Hosted evaluation for agents: datasets of Harbor-format tasks, jobs that run any model on any agent harness against them in cloud sandboxes, and the trials, checks and analyses they produce — from the `evolve` CLI and the TypeScript and Python SDKs.
+Run evaluations in cloud sandboxes. Read results, inspect files, check tasks, and analyze traces through the CLI or Python and TypeScript SDKs.
 
-The pages under `references/` are the documentation site's pages, byte for byte, at the site's paths: a site link to `/core-concepts/tasks` is `references/core-concepts/tasks.mdx`, and `evolve skills get evals core-concepts/tasks` prints it. An `import` of `/snippets/<file>` is `references/snippets/<file>`.
+## Find the right instructions
 
-## How to use this skill
+1. Choose a page from the index below. Read it with `evolve skills get evals <page>`.
+2. Follow links by site path: `/core-concepts/tasks` means `evolve skills get evals core-concepts/tasks` or `references/core-concepts/tasks.md` in this folder.
+3. Confirm installed command options with `evolve <command> --help`. Use `--json` when parsing command output.
 
-1. Find the topic in the index below and read that page before writing any command or code: `evolve skills get evals <page>` prints it, the page named by its site path; `evolve skills get evals --full` prints every page.
-2. Every CLI verb is documented from its own `--help`; run `evolve <verb> --help` to confirm the flags of the installed version.
-3. Every command and every SDK client reads `EVOLVE_API_KEY`; the Installation page says where the key comes from.
+These pages are generated from the same source as the website. Tabs become labeled sections, field details remain visible, and snippets are included in place. Read individual pages first; `--full` prints the whole manual.
 
+Hosted requests use `EVOLVE_API_KEY`. Start with Installation if authentication is missing. The managed-agent builder, Swarm, and Pipeline have a separate `evolve-agents` skill.
 
-## Docs
 
-### Getting started
+## Guides
+
+### Start here
 
 | Page | What it covers |
 | --- | --- |
-| [Evolve documentation](references/index.mdx) | Environment-driven agent evaluations: any model, any harness, any sandbox, one API key. Every task versioned and checked, every trajectory analyzed. |
-| [Installation](references/getting-started/installation.mdx) | Install the evolve command and the SDKs, and set your API key. |
-| [Quick start](references/getting-started/quick-start.mdx) | Start your first job, watch it run, and download a trial. |
+| [Evolve Documentation](references/index.md) | Environment-driven agent evaluation and optimization. |
+| [Installation](references/getting-started/installation.md) | Install your preferred client and connect it to your Evolve account. |
+| [Quick start](references/getting-started/quick-start.md) | Run one small task, inspect its result, and download the evidence. |
+| [How evaluations fit together](references/core-concepts/index.md) | The few concepts you need to read a job and understand its results. |
 
-### Core concepts
+### Run evaluations
 
 | Page | What it covers |
 | --- | --- |
-| [Core concepts](references/core-concepts/index.mdx) | How a dataset becomes trials, rewards, analyses and results. |
-| [Tasks](references/core-concepts/tasks.mdx) | The task format: one directory with an instruction, an environment, a verifier, and an optional reference solution. |
-| [Datasets](references/core-concepts/datasets.mdx) | The catalog, and publishing your own dataset. |
-| [Jobs](references/core-concepts/jobs.mdx) | Start a job, watch it, read its results, and derive new jobs from it. |
-| [Trials](references/core-concepts/trials.mdx) | One agent's attempt at one task: its record, its trace, and its artifacts. |
-| [Trial outputs](references/core-concepts/trial-outputs.mdx) | What a finished trial gives you, file by file, and in which formats. |
-| [Agents](references/core-concepts/agents.mdx) | The built-in harnesses, how to configure an arm, and how to register your own agent. |
-| [Models](references/core-concepts/models.mdx) | How a model is named on -m, and why one API key covers every provider. |
-| [Sandboxes](references/core-concepts/sandboxes.mdx) | Where trials run: e2b, daytona, or modal. |
-| [Analyze](references/core-concepts/analyze.mdx) | Judge every trial's trace against a rubric, on the platform. |
-| [Check](references/core-concepts/check.mdx) | Check the quality of tasks against a rubric before you spend a job on them. |
-| [Skills](references/core-concepts/skills.mdx) | Mount skills into every agent run, from a git source or from your uploads. |
-| [Secrets](references/core-concepts/secrets.mdx) | Store env secrets once and attach them to jobs by name. |
-| [Upload a job](references/core-concepts/upload.mdx) | Bring a job you ran elsewhere onto the platform: a Harbor job folder, or Evolve SDK runs packed as one. |
-| [Teams](references/core-concepts/teams.mdx) | Create a team, invite people, run under it, and see each other's work. |
-| [Sharing](references/core-concepts/sharing.mdx) | Share a run with people by email, or with anyone by link. |
-
-### Dashboard
+| [Datasets](references/core-concepts/datasets.md) | Browse ready-to-run tasks or publish your own versioned collection. |
+| [Jobs](references/core-concepts/jobs.md) | Run a task set across agents and models. Follow each attempt to its result. |
+| [Agents](references/core-concepts/agents.md) | Choose a built-in coding-agent harness or register your own. |
+| [Models](references/core-concepts/models.md) | Select models, compare them, and control metered spend. |
+| [Sandboxes](references/core-concepts/sandboxes.md) | Choose a cloud provider and understand how task requirements affect placement. |
+| [Agent skills](references/core-concepts/skills.md) | Give evaluated agents reusable instructions and supporting files. |
+| [Secrets](references/core-concepts/secrets.md) | Store a credential once and attach it to evaluation jobs by name. |
+
+### Read results
 
 | Page | What it covers |
 | --- | --- |
-| [Datasets](references/dashboard/datasets.mdx) | Browse the catalog, and each dataset's tasks and versions, in the dashboard. |
-| [Jobs](references/dashboard/jobs.mdx) | Start a job from the browser, follow it, and read its results. |
-| [Trial viewer](references/dashboard/trial-viewer.mdx) | Read one trial's trace event by event, its tool calls, its spend, and download its artifacts. |
-| [Checks](references/dashboard/checks.mdx) | Read task quality checks in the dashboard, on a job's CHECK tab or on the check's own page. |
-| [Traces](references/dashboard/traces.mdx) | Every recorded run in one list. |
+| [Trials](references/core-concepts/trials.md) | Inspect one agent's attempt: what it did, what it earned, and why it stopped. |
+| [Trial outputs](references/core-concepts/trial-outputs.md) | Find the record, trace, logs, and files from an evaluation. |
+| [Analyze trials](references/core-concepts/analyze.md) | Review a trial's behavior and score against a rubric, with evidence for every finding. |
+| [Upload a job](references/core-concepts/upload.md) | Bring existing Harbor-format results into Evolve for inspection, analysis, and sharing. |
+| [Upload an SDK run](references/core-concepts/upload-sdk-session.md) | Turn a recorded agent run into a job you can inspect and analyze. |
 
-### SDK
+### Work together
 
 | Page | What it covers |
 | --- | --- |
-| [TypeScript SDK](references/sdk/typescript.mdx) | Start a job, wait for it, read the result, and download it from TypeScript. |
-| [Python SDK](references/sdk/python.mdx) | Start a job, wait for it, read the result, and download it from Python. |
+| [Teams](references/core-concepts/teams.md) | Put evaluations in a shared workspace and give teammates access through membership. |
+| [Sharing](references/core-concepts/sharing.md) | Give someone read access to a job, check, or managed-agent session. |
+
+### Use code
+
+| Page | What it covers |
+| --- | --- |
+| [TypeScript](references/sdk/typescript.md) | Run one task, follow its progress, and inspect the result. |
+| [Python](references/sdk/python.md) | Run one task from an async Python program. No Node bridge is needed. |
+| [HTTP API](references/sdk/http.md) | Submit and inspect evaluations from any language. |
+
+## Build tasks
+
+### Task format
+
+| Page | What it covers |
+| --- | --- |
+| [Create your first task](references/getting-started/first-task.md) | Write a task, publish it, check its quality, and run one managed evaluation. |
+| [Tasks](references/core-concepts/tasks.md) | Define the work, the environment, and how to score it. |
+| [Task environment](references/core-concepts/task-environment.md) | Give the agent the tools, files, and services the task needs. |
+| [Multiple containers](references/core-concepts/compose.md) | Give a task its own API, database, or other service beside the agent. |
+| [Task configuration](references/core-concepts/task-config.md) | Set resources, timeouts, network access, and environment variables in task.toml. |
+| [Task field reference](references/core-concepts/task-reference.md) | Accepted task.toml fields, defaults, and the restrictions Evolve enforces. |
+| [Task verifier](references/core-concepts/task-verifiers.md) | Turn the agent's work into a reward you can compare. |
+| [Reward Kit](references/core-concepts/rewardkit.md) | Combine reusable checks, custom Python, and model judges into reward scores. |
+| [Multi-step tasks](references/core-concepts/multi-step.md) | Give the agent a sequence of instructions while preserving its files. |
+| [Compatibility](references/core-concepts/compatibility.md) | Supported Harbor-format task features and the boundaries of managed evaluation jobs. |
+
+### Validate your work
+
+| Page | What it covers |
+| --- | --- |
+| [Check tasks](references/core-concepts/check.md) | Review instructions, environments, solutions, and verifiers before an evaluation. |
 
 ## CLI reference
 
-### The evolve command
+### Start work
 
 | Page | What it covers |
 | --- | --- |
-| [CLI reference](references/cli-reference/index.mdx) | Every verb of the evolve command, from its own help output. |
-| [evolve run](references/cli-reference/run.mdx) | Start a job, and optionally follow it to the end. |
-| [evolve analyze](references/cli-reference/analyze.mdx) | Analyze a finished job's trial traces against a rubric, on the platform. |
-| [evolve check](references/cli-reference/check.mdx) | Check task quality against a rubric, and read checks back. |
-| [evolve upload](references/cli-reference/upload.mdx) | Upload a job folder as a finished job. |
+| [CLI reference](references/cli-reference/index.md) | Run evaluations, inspect results, and manage the resources around them. |
+| [evolve run](references/cli-reference/run.md) | Choose tasks and agents, set limits, and launch an evaluation. |
+| [evolve analyze](references/cli-reference/analyze.md) | Evaluate completed trial traces against a rubric. |
+| [evolve check](references/cli-reference/check.md) | Check task quality, inspect the report, and read the checker's trace. |
+| [evolve upload](references/cli-reference/upload.md) | Import completed evaluations into Evolve from a Harbor-format job folder. |
 
-### Command groups
+### Inspect results
 
 | Page | What it covers |
 | --- | --- |
-| [evolve job](references/cli-reference/job.mdx) | Start, follow, and derive jobs. |
-| [evolve trial](references/cli-reference/trial.mdx) | Inspect, download, and act on single trials. |
-| [evolve analysis](references/cli-reference/analysis.mdx) | List, inspect, and download trace-analysis runs. |
-| [evolve dataset](references/cli-reference/dataset.mdx) | Browse and publish the dataset catalog. |
-| [evolve skill](references/cli-reference/skill.mdx) | Upload and manage platform-stored skills. |
-| [evolve skills](references/cli-reference/skills.mdx) | The skills the CLI serves to coding agents. |
-| [evolve agent](references/cli-reference/agent.mdx) | Register and manage your own agents. |
-| [evolve auth](references/cli-reference/auth.mdx) | Identity, API keys, and your organizations. |
-| [evolve secrets](references/cli-reference/secrets.mdx) | Store and manage env secrets. Values are write-only; reads return metadata. |
+| [evolve job](references/cli-reference/job.md) | Inspect results, compare runs, and control or repeat evaluations. |
+| [evolve trial](references/cli-reference/trial.md) | Inspect one attempt: its score, trace, files, and execution. |
+| [evolve analysis](references/cli-reference/analysis.md) | Read an analysis result, transcript, or stored files. |
+| [Files, logs, and processes](references/cli-reference/filesystem.md) | Inspect a running sandbox, captured files, or a published task package. |
+| [evolve session](references/cli-reference/session.md) | Inspect and share the sessions recorded by managed-agent SDK runs. |
+
+### Manage resources
+
+| Page | What it covers |
+| --- | --- |
+| [evolve dataset](references/cli-reference/dataset.md) | Find tasks, validate a corpus, and publish a version for evaluation. |
+| [evolve agent](references/cli-reference/agent.md) | Register a custom agent so Evolve can install and run it for evaluations. |
+| [evolve skill](references/cli-reference/skill.md) | Upload skills for evaluated agents and reuse them by reference. |
+| [evolve skills](references/cli-reference/skills.md) | Read the bundled manual and install its small entry point for coding agents. |
+| [evolve auth](references/cli-reference/auth.md) | Check your identity, manage team membership, and choose a default organization. |
+| [evolve secrets](references/cli-reference/secrets.md) | Store environment secrets and attach them to evaluations by name. |
 
 ## SDK reference
 
-### Clients
+### Get started
+
+| Page | What it covers |
+| --- | --- |
+| [SDK clients](references/sdk-reference/index.md) | One configuration. Focused clients for jobs, datasets, trials, and results. |
+| [Platform capabilities](references/sdk-reference/meta.md) | Discover available harnesses, models, providers, and current limits. |
+| [Returned objects](references/sdk-reference/types.md) | Read jobs, trials, pages, costs, and results without guessing their shape. |
+| [Errors](references/sdk-reference/errors.md) | Handle API refusals separately from local, transfer, and wait failures. |
+| [Error codes](references/sdk-reference/error-codes.md) | Every known code, what it means, and what to do next. |
+
+### Run and inspect
+
+| Page | What it covers |
+| --- | --- |
+| [Jobs](references/sdk-reference/jobs.md) | Submit evaluations, follow progress, and work with results. |
+| [Trials](references/sdk-reference/trials.md) | Inspect one attempt, from its reward to its trace and files. |
+| [Filesystems and downloads](references/sdk-reference/filesystem.md) | Choose the right files: saved results, a sandbox, or a task package. |
+| [Analyses](references/sdk-reference/analyses.md) | Analyze trial traces, then inspect the result and the analyzer's own run. |
+| [Checks](references/sdk-reference/checks.md) | Review task quality before spending a benchmark run. |
+
+### Manage resources
+
+| Page | What it covers |
+| --- | --- |
+| [Datasets](references/sdk-reference/datasets.md) | Browse task versions, check a local corpus, and publish it. |
+| [Custom agents](references/sdk-reference/agents.md) | Register your own harness and use it in a job arm. |
+| [Uploaded skills](references/sdk-reference/skills.md) | Store skill folders and attach them to evaluation agents. |
+| [Secrets](references/sdk-reference/secrets.md) | Store a credential once, then attach it to evaluation jobs by name. |
+| [Identity and teams](references/sdk-reference/auth.md) | Read your identity, choose a team, and manage invitations. |
+
+### Method reference
+
+| Page | What it covers |
+| --- | --- |
+| [Jobs methods](references/sdk-reference/methods/jobs.md) | Every job method: inputs, results, and a small example in each SDK. |
+| [Trials methods](references/sdk-reference/methods/trials.md) | Read attempts, traces, files, and actions by trial id. |
+| [Filesystem methods](references/sdk-reference/methods/filesystem.md) | Read run files, captured changes, sandbox logs, and live processes. |
+| [Analyses methods](references/sdk-reference/methods/analyses.md) | Read the analyzer’s result, transcript, files, and current defaults. |
+| [Checks methods](references/sdk-reference/methods/checks.md) | Create task quality checks and read each checker’s result. |
+| [Datasets: methods](references/sdk-reference/methods/datasets.md) | Every dataset call, option, and returned field. |
+| [Agents: methods](references/sdk-reference/methods/agents.md) | Register, read, replace, and delete custom agents. |
+| [Uploaded skills: methods](references/sdk-reference/methods/skills.md) | Upload, read, and delete skill content used by evaluation agents. |
+| [Secrets: methods](references/sdk-reference/methods/secrets.md) | Store credentials and read or delete their metadata. |
+| [Identity and teams: methods](references/sdk-reference/methods/auth.md) | Read identity and manage organization membership. |
+| [Capabilities: method and fields](references/sdk-reference/methods/meta.md) | Read harness, model, provider, status, and limit information. |
+
+## Dashboard
+
+### Explore the dashboard
 
 | Page | What it covers |
 | --- | --- |
-| [Clients](references/sdk-reference/index.mdx) | How the hosted clients are built and configured. |
-| [jobs](references/sdk-reference/jobs.mdx) | Start, follow, and derive jobs. |
-| [trials](references/sdk-reference/trials.mdx) | Read, download, and act on single trials. |
-| [datasets](references/sdk-reference/datasets.mdx) | The catalog: list, read, publish, download, activate, delete. |
-| [analyses](references/sdk-reference/analyses.mdx) | Read trace-analysis runs. |
-| [checks](references/sdk-reference/checks.mdx) | Start and read task quality checks. |
-| [skills](references/sdk-reference/skills.mdx) | Upload and manage platform-stored skills. |
-| [agents](references/sdk-reference/agents.mdx) | Register and manage your own agents. |
-| [auth and orgs](references/sdk-reference/auth.mdx) | Who you are, and your organizations. |
-| [meta](references/sdk-reference/meta.mdx) | The platform's capability document, no key needed. |
-| [Types](references/sdk-reference/types.mdx) | Every field of the objects the clients return: name, type, meaning, and when it is null. |
-| [Errors](references/sdk-reference/errors.mdx) | The error class every refused request raises, and every code it can carry. |
+| [Datasets](references/dashboard/datasets.md) | Find a task set, inspect its versions, and check where its tasks can run. |
+| [Jobs](references/dashboard/jobs.md) | Launch an evaluation, follow its progress, and move from a summary to the evidence. |
+| [Trial viewer](references/dashboard/trial-viewer.md) | Follow the agent's actions and inspect the evidence behind a result. |
+| [Checks](references/dashboard/checks.md) | Read task-quality findings and open the evidence behind them. |
+| [Traces](references/dashboard/traces.md) | Find evaluation trials, analyses, checks, regrades, and agent sessions in one place. |

package/skills/evolve-evals/references/cli-reference/agent.md

@@ -0,0 +1,76 @@
+---
+title: "evolve agent"
+description: "Register a custom agent so Evolve can install and run it for evaluations."
+---
+
+Register an install script or agent directory, plus a command to run. Use the registration's name with `evolve run -a`.
+
+### Install script
+
+```bash
+evolve agent add my-agent \
+  --install-script ./install.sh \
+  --run "my-agent --headless"
+```
+
+Evolve uploads the script's contents.
+
+### Agent directory
+
+```bash
+evolve agent add my-agent \
+  --dir ./my-agent \
+  --run "./bin/my-agent --headless" \
+  --agent-env MODE=eval
+```
+
+Evolve uploads the directory as an archive.
+
+Use exactly one source. Both forms require `--run`.
+
+## Registration options
+
+| Option | Meaning |
+| --- | --- |
+| `--install-script <path>` | Local install script. |
+| `--dir <path>` | Local agent directory. |
+| `--run <command>` | Required command, executed with `sh -c`. |
+| `--agent-env <KEY=VALUE>`, `--ae <KEY=VALUE>` | Runtime environment variables. Repeatable. |
+| `--org <name>` | Owning organization; otherwise the saved CLI default, then personal. |
+
+The registration belongs to you and an organization. Its members can use it in jobs; only you can change or remove it.
+
+For the execution contract and model access, see [Agents](/core-concepts/agents).
+
+## List and inspect
+
+```bash
+evolve agent list --scope org
+evolve agent show my-agent
+```
+
+`agent list` accepts `--scope <my|shared|org>` and all shared [list options](/cli-reference/index#list-options).
+
+`agent show` takes the registered agent's name. This group manages custom registrations; it is not the list of built-in harnesses.
+
+## Run it
+
+```bash
+evolve run \
+  -d harbor-examples@1.0 -i hello-world \
+  -a my-agent -m gpt-5.6-luna \
+  --max-trial-spend 0.30 --max-retries 0 \
+  --watch
+```
+
+The supplied model must be suitable for your agent's implementation.
+
+## Remove a registration
+
+```bash
+evolve agent remove my-agent
+```
+
+Past jobs keep their recorded agent configuration. Removal has no CLI confirmation prompt.
+
+[Global options](/cli-reference/index#global-options) apply. The plural `agents` is reserved; use `agent`.

package/skills/evolve-evals/references/cli-reference/analysis.md

@@ -0,0 +1,78 @@
+---
+title: "evolve analysis"
+description: "Read an analysis result, transcript, or stored files."
+---
+
+Use [`evolve analyze`](/cli-reference/analyze) to start analyses. Use `evolve analysis` to read them.
+
+```bash
+evolve analysis list --job "$JOB_ID"
+evolve analysis show "$ANALYSIS_ID"
+```
+
+`$JOB_ID` is an evaluation job ID. `$ANALYSIS_ID` comes from the analysis list.
+
+## List analyses
+
+```bash
+evolve analysis list --status failed --scope org
+```
+
+| Option | Meaning |
+| --- | --- |
+| `--job <job-id>` | Analyses for this job. |
+| `--status <statuses>` | Comma-separated `queued`, `running`, `completed`, or `failed`. |
+| `--scope <my\|shared\|org>` | Records to list. Default `my`. |
+
+All shared [list options](/cli-reference/index#list-options) apply: `--limit`, `--cursor`, `--columns`, `--quiet`, `--no-trunc`, `--no-headers`.
+
+## Show the result
+
+```bash
+evolve analysis show "$ANALYSIS_ID" --json
+```
+
+`show`, `trace`, and `download` also accept a **trial ID**, which selects that trial's latest analysis. An analysis ID selects that specific run.
+
+Prefixes must identify only one analysis or analyzed trial. Use a full ID when a prefix is ambiguous.
+
+## Read the transcript
+
+```bash
+evolve analysis trace "$ANALYSIS_ID"
+evolve analysis trace "$ANALYSIS_ID" --since 200
+```
+
+`--since <n>` skips N transcript events. This command reads the available transcript and exits; it does not follow live output. With `--json`, events are printed one per line.
+
+## Download
+
+```bash
+evolve analysis download "$ANALYSIS_ID" -o analyses/
+```
+
+The archive contains the analyzer's run in a Harbor-compatible trial folder, plus an `evolve.json` record.
+
+| Option | Meaning |
+| --- | --- |
+| `-o`, `--output-dir <dir>` | Parent folder. Default `analyses/`. |
+| `--overwrite` | Allow replacing an existing folder. |
+| `--stream <artifact>` | Print one artifact: `analysis`, `trace-parsed`, `trace-stdout`, `trace-stderr`, or `agent-home`. |
+| `--since <n>` | With `--stream trace-parsed`, skip the first N events. |
+
+`--stream` cannot be combined with output directory or overwrite.
+
+```bash
+evolve analysis download "$ANALYSIS_ID" --stream analysis
+evolve analysis download "$ANALYSIS_ID" --stream trace-stderr
+```
+
+## Files and live output
+
+Use the shared [filesystem reference](/cli-reference/filesystem) for `analysis files`, `analysis logs`, and `analysis procs`.
+
+**Note:**
+
+These filesystem and log commands require an **analysis ID**. They do not select an analysis from a trial ID.
+
+[Global options](/cli-reference/index#global-options) apply.

package/skills/evolve-evals/references/cli-reference/analyze.md

@@ -0,0 +1,97 @@
+---
+title: "evolve analyze"
+description: "Evaluate completed trial traces against a rubric."
+---
+
+Start an analysis for a finished job. Use the job ID from `evolve job list` as `$JOB_ID`.
+
+```bash
+evolve analyze "$JOB_ID" --watch
+```
+
+Each selected trial gets its own analysis run. Read those runs with [`evolve analysis`](/cli-reference/analysis).
+
+## Inspect the defaults
+
+```bash
+evolve analyze --show-defaults
+evolve analyze --show-defaults --json
+```
+
+This returns the current agent, model, reasoning effort, sandbox provider, prompt, and rubric. Defaults mode takes no job ID or other analysis options.
+
+## Select trials
+
+### Failing results
+
+```bash
+evolve analyze "$JOB_ID" --failing --n-trials 20 --watch
+```
+
+### Specific trials
+
+```bash
+evolve job trials "$JOB_ID" --json
+evolve analyze "$JOB_ID" --trial "$TRIAL_ID" --watch
+```
+
+Copy the **full trial ID** into `$TRIAL_ID`. This flag does not expand prefixes.
+
+| Option | Meaning |
+| --- | --- |
+| `-t`, `--trial <trial-id>` | Select a trial in the job. Repeatable; full IDs required. |
+| `--passing` | Only trials with reward `1.0`. |
+| `--failing` | Eligible settled trials other than `SCORED` with reward exactly `1.0`. Includes missing rewards and execution errors. |
+| `-l`, `--n-trials <n>` | Maximum selected trials, after filters. |
+| `-n`, `--n-concurrent <n>` | Concurrent analyses, `1`–`150`. If omitted, use your organization's ceiling. |
+
+Cancelled trials are excluded. `--passing` and `--failing` cannot be combined. The server validates trial membership and eligibility. A job can have only one active analysis wave at a time.
+
+## Customize the analysis
+
+```bash
+evolve analyze "$JOB_ID" \
+  --agent codex --model gpt-5.5 \
+  --rubric rubric.toml \
+  --prompt prompt.txt \
+  --watch
+```
+
+| Option | Meaning |
+| --- | --- |
+| `-a`, `--agent <name>` | Agent the analyzer runs on. Default `claude`. |
+| `-m`, `--model <name>` | Analyzer model, from that agent's models. |
+| `--effort <value>` | Reasoning effort. |
+| `-r`, `--rubric <path>` | TOML, YAML, or JSON rubric. |
+| `-p`, `--prompt <path>` | Text file replacing the default prompt. |
+| `-e`, `--env <provider>` | Sandbox provider for analysis. |
+| `--show-defaults` | Print the default policy and exit. |
+
+A rubric contains a `criteria` list. Every criterion needs `name`, `description`, and `guidance`.
+
+```toml rubric.toml
+[[criteria]]
+name = "uses_evidence"
+description = "The final answer is supported by observed results."
+guidance = "Check whether the trace supports the claims in the final answer."
+```
+
+For prompt placeholders, rubric rules, and results, see [Analyze](/core-concepts/analyze).
+
+## Wait and read results
+
+| Option | Meaning |
+| --- | --- |
+| `--watch` | Wait until the selected analyses settle. |
+| `-q`, `--quiet` | With watch, suppress intermediate progress. |
+
+Without watch, the command returns the accepted job. With `--watch --json`, it prints `analysis.accepted`, progress records, and `analysis.final`.
+
+```bash
+evolve analysis list --job "$JOB_ID"
+evolve trial show "$TRIAL_ID"
+```
+
+A failed analysis execution produces exit code `1`. A rubric criterion marked `fail` is a finding, not an execution error.
+
+To analyze automatically during a new evaluation, use [`run --analyze`](/cli-reference/run). [Global options](/cli-reference/index#global-options) apply.

package/skills/evolve-evals/references/cli-reference/auth.md

@@ -0,0 +1,96 @@
+---
+title: "evolve auth"
+description: "Check your identity, manage team membership, and choose a default organization."
+---
+
+Set `EVOLVE_API_KEY`, then confirm which account and key the CLI is using.
+
+```bash
+export EVOLVE_API_KEY="<your-key>"
+evolve auth status
+```
+
+Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys). `--api-key <key>` overrides the environment for one command.
+
+## Find your organizations
+
+```bash
+evolve auth org list
+evolve auth org show acme
+```
+
+`org show <slug>` returns your role, members, quota, and current usage for that organization.
+
+| Option for `org list` | Meaning |
+| --- | --- |
+| `--search <text>` | Match slug, display name, or role. |
+| `--columns <keys\|all\|help>` | Select columns or list their names. |
+| `-q`, `--quiet` | Print organization slugs only. |
+| `--no-trunc` | Keep full terminal table cells. |
+| `--no-headers` | Omit piped TSV headers. |
+
+This list has no `--limit` or `--cursor` option.
+
+## Create a team
+
+```bash
+evolve auth org create acme --display-name "Acme"
+```
+
+You become the organization owner. `--display-name <text>` is optional and defaults to the supplied name.
+
+## Invite and join
+
+### 1. Owner creates an invitation
+
+```bash
+evolve auth org invite acme
+```
+
+The response includes an invite token and its expiry/use limits.
+
+### 2. Owner gives the token to a teammate
+
+The command prints the token; it does not send the invitation for you.
+
+### 3. Teammate joins with their own API key
+
+```bash
+evolve auth org join "$INVITE_TOKEN"
+```
+
+The teammate joins as a member. Set `$INVITE_TOKEN` to the token from the owner.
+
+## Set the default organization
+
+### Save a team
+
+```bash
+evolve auth org use acme
+```
+
+### Show the default
+
+```bash
+evolve auth org use
+```
+
+### Return to personal
+
+```bash
+evolve auth org use personal
+```
+
+This is a local preference. It writes to `$XDG_CONFIG_HOME/evolve/config.json`, or `~/.config/evolve/config.json`. The command does not verify membership; the server checks access when you create a resource.
+
+| Selection | Priority |
+| --- | --- |
+| `--org <name>` on a creation command | First. |
+| Saved CLI default | Used when no organization flag is supplied. |
+| Personal organization | Used when neither is set. |
+
+`--org personal` uses your personal organization for one command without changing the saved default.
+
+The default applies to job creation, dataset publishing, agent registration, and skill uploads. See [Teams](/core-concepts/teams) for ownership and permissions.
+
+[Global options](/cli-reference/index#global-options) apply to every auth command.