@evolvingmachines/evolve 0.0.65 → 0.0.67

package/skills/evolve-evals/references/cli-reference/trial.md

@@ -0,0 +1,115 @@
+---
+title: "evolve trial"
+description: "Inspect one attempt: its score, trace, files, and execution."
+---
+
+Get a trial ID from `evolve job trials "$JOB_ID"`. The examples use that ID as `$TRIAL_ID`.
+
+```bash
+evolve trial show "$TRIAL_ID"
+```
+
+The detail view includes status, reward, spend, execution details, and the latest analysis when one exists.
+
+## Read the trace
+
+```bash
+evolve trial trace "$TRIAL_ID" --grep 'permission denied' --tail 20
+```
+
+| Option | Meaning |
+| --- | --- |
+| `--type <event-type>` | Select an exact event type. |
+| `--grep <pattern>` | Case-insensitive regular expression over event type and content. |
+| `--tail <n>` | Keep the last N matching events. |
+| `--cursor <seq>` | Resume after this event sequence number. |
+| `-l`, `--limit <n>` | Events per API page. Default `200`, maximum `1000`. |
+
+The command reads all available pages and then exits. `--limit` sets page size, not a total event cap. Use [sandbox logs](/cli-reference/filesystem#logs) to follow live output.
+
+With `--json`, each trace event is a separate JSON line.
+
+## Download the trial
+
+```bash
+evolve trial download "$TRIAL_ID" -o trials/
+```
+
+```text
+trials/
+└── <trial-id>/
+    ├── config.json
+    ├── result.json
+    ├── evolve.json
+    ├── agent/
+    └── verifier/
+```
+
+`agent/` stores the trace and agent files. `verifier/` stores verifier output.
+
+Files are included when their artifacts exist.
+
+| Option | Meaning |
+| --- | --- |
+| `-o`, `--output-dir <dir>` | Parent folder. Default `trials/`. |
+| `--overwrite` | Allow writing into an existing trial folder. |
+| `--stream <artifact>` | Print one artifact instead of saving the trial tree. |
+| `--cursor <seq>` | With `--stream trace-parsed`, resume after this sequence. |
+| `--limit <n>` | With `--stream trace-parsed`, set API page size. |
+
+`--stream` cannot be combined with `--output-dir` or `--overwrite`.
+
+### Artifact names
+
+| Artifact | Content |
+| --- | --- |
+| `trace-parsed` | Parsed trace events. |
+| `trace-stdout` | Raw agent stdout. |
+| `trace-stderr` | Raw agent stderr. |
+| `trace-atif` | Normalized ATIF trajectory. |
+| `verifier` | Verifier output. |
+| `agent-home` | Stored agent-home files. |
+| `filesystem` | Sandbox filesystem archive, as `.tar.gz` bytes. |
+| `trajectory` | Reserved name; native trajectory download is not available through this selector. |
+
+```bash
+evolve trial download "$TRIAL_ID" --stream trace-stdout
+evolve trial download "$TRIAL_ID" --stream filesystem > trial-files.tar.gz
+```
+
+Missing stored logs can return `null` with `--json`. Binary filesystem output becomes a base64 JSON envelope when `--json` is used.
+
+## Files, logs, and processes
+
+**[Browse files](/cli-reference/filesystem)**
+
+List folders, read a file, search text, or download a subtree.
+
+**[Follow logs](/cli-reference/filesystem#logs)**
+
+Read agent, verifier, or system output and inspect live processes.
+
+## Retry or regrade
+
+```bash
+evolve trial retry "$TRIAL_ID"
+evolve trial regrade "$TRIAL_ID"
+```
+
+| Command | Result |
+| --- | --- |
+| `retry` | A new job running the settled trial again. |
+| `regrade` | A new job running only the verifier over eligible stored inputs. |
+
+Neither command changes the original trial. Regrade requires a supported single-step, separate verifier with stored inputs; judge-backed verifiers are excluded. See [Trials](/core-concepts/trials).
+
+## Stop a trial
+
+```bash
+evolve trial stop "$TRIAL_ID"
+evolve trial stop "$TRIAL_ID" "$OTHER_TRIAL_ID"
+```
+
+Stop selected trials while keeping their job. The response distinguishes stopped, already-terminal, and missing IDs. To stop an entire dataset's trials, use [`job stop`](/cli-reference/job#stop-work).
+
+[Global options and ID prefixes](/cli-reference/index) apply.

package/skills/evolve-evals/references/cli-reference/upload.md

@@ -0,0 +1,72 @@
+---
+title: "evolve upload"
+description: "Import completed evaluations into Evolve from a Harbor-format job folder."
+---
+
+Upload results from another machine or runner. Evolve imports their trials, rewards, and traces as a completed job.
+
+```bash
+evolve upload ./completed-job -d harbor-examples@1.0
+```
+
+The source must use the [supported job layout](/core-concepts/upload). A local directory needs `config.json` and `result.json` at its root.
+
+For an SDK session log, use the [packaging walkthrough](/core-concepts/upload-sdk-session) first.
+
+## Choose a source
+
+### Directory
+
+```bash
+evolve upload ./completed-job
+```
+
+### Archive
+
+```bash
+evolve upload ./completed-job.tar.gz
+```
+
+### Public URL
+
+```bash
+evolve upload --from https://example.com/completed-job.tar.gz
+```
+
+Use one source per invocation. `--from` requires a public HTTPS URL.
+
+## Options
+
+| Option | Meaning |
+| --- | --- |
+| `-d`, `--dataset <name[@version]>` | Link imported trial tasks to a published dataset version by task name. |
+| `--from <url>` | Have Evolve fetch a public job archive. Replaces the local path. |
+| `--no-wait` | Return the import record without waiting for the completed job. |
+
+Dataset linking makes the stored task available to later analysis. Review the report for unlinked tasks and skipped trial folders.
+
+## Follow an import
+
+By default, `upload` waits for the imported job. Use `--no-wait` when you want to reconnect later.
+
+```bash
+evolve upload ./completed-job --no-wait
+evolve job imports --status RUNNING
+evolve job import "$IMPORT_ID" --watch
+```
+
+Copy the complete import ID printed by `upload` into `$IMPORT_ID`.
+
+With `--json`, the default command prints one final job document. With `--no-wait --json`, it prints the import record instead.
+
+## Inspect the result
+
+```bash
+evolve job show "$JOB_ID"
+evolve job trials "$JOB_ID"
+evolve analyze "$JOB_ID" --watch
+```
+
+`$JOB_ID` is the ID of the imported job. Uploaded costs are reported by the source archive, not metered execution on Evolve. Uploaded jobs can be analyzed, but cannot be resumed, retried, or regraded.
+
+[Global options](/cli-reference/index#global-options) apply.

package/skills/evolve-evals/references/core-concepts/agents.md

@@ -0,0 +1,96 @@
+---
+title: "Agents"
+description: "Choose a built-in coding-agent harness or register your own."
+---
+
+An evaluation arm pairs an **agent harness** with a **model**. The harness reads the instruction, calls tools, and manages the conversation. The model supplies its decisions.
+
+```text
+One arm
+├── Harness: -a codex
+└── Model: -m gpt-5.6-luna
+```
+
+## Built-in harnesses
+
+| CLI name | Harness |
+| --- | --- |
+| `claude` | Claude Code |
+| `codex` | Codex |
+| `gemini` | Gemini CLI |
+| `qwen` | Qwen Code |
+| `kimi` | Kimi Code |
+| `opencode` | OpenCode |
+| `droid` | Droid |
+
+```bash
+evolve run -d harbor-examples@1.0 -i hello-world \
+  -a codex -m gpt-5.6-luna \
+  --max-trial-spend 1 --max-retries 0 --watch
+```
+
+Use `-a name@version` to pin a harness release. Without a version, Evolve resolves one at job creation and uses that version across the job. The trial records it in `agent_info.version`.
+
+## Configure an arm
+
+| Option | Purpose |
+| --- | --- |
+| `--effort <value>` | Set supported reasoning effort |
+| `--preset no-internet` | Disable vendor server-side web tools |
+| `--preset pinned-context` | Set the supported context window to 200,000 tokens |
+| `--ak config=<path-or-JSON>` | Supply a native harness configuration |
+
+Presets and native config are supported on Claude Code and Codex. The platform applies routing settings over your config, and preset settings take precedence where they overlap. Routing, billing, credential, and environment overrides are rejected.
+
+The live [capability document](/sdk-reference/meta) lists each harness's supported options. Unsupported effort or configuration is rejected when the job is created.
+
+**Note:**
+
+`no-internet` disables the harness's server-side web tools. The task's [network policy](/core-concepts/task-config#network-access) controls network access from the sandbox.
+
+## Register a custom agent
+
+Register an installation source and a command. The name then works with `-a`.
+
+```bash
+evolve agent add my-agent \
+  --install-script ./install.sh \
+  --run "my-agent --headless"
+```
+
+Use `--dir` instead of `--install-script` to upload a directory. The install runs with internet access and no secrets; place executables in `$PREFIX/bin`.
+
+### Execution contract
+
+1. Install the agent from its registered source.
+
+2. Prepare the instruction file and gateway environment.
+
+3. Start `run_command` with `sh -c` in the task working directory, with the instruction on stdin.
+
+4. Route model calls through the supplied gateway credentials.
+
+5. Collect outputs and run the task verifier.
+
+| Input | Meaning |
+| --- | --- |
+| Standard input | Task instruction |
+| `EVOLVE_INSTRUCTION_FILE` | Path to the same instruction |
+| `EVOLVE_MODEL` | Model selected for this arm |
+| `EVOLVE_GATEWAY_BASE_URL` | OpenAI-compatible URL, including `/v1` |
+| `EVOLVE_GATEWAY_API_KEY` | Gateway credential |
+| `OPENAI_BASE_URL`, `OPENAI_API_KEY` | Aliases for the same gateway values |
+
+Your agent must use these gateway settings for metered model calls. If it requires a config file, write that file from these values inside `run_command`.
+
+`--agent-env KEY=VALUE` adds non-secret runtime settings to the registration. It cannot override the contract variables or declare credential-like names. Attach [job secrets](/core-concepts/secrets) for credentials the task needs.
+
+### Custom-agent differences
+
+Custom agents do not have a built-in live trace parser or the shared `--effort` option. Put implementation-specific reasoning flags in the run command. Harness version pins are not accepted for registrations.
+
+Registrations belong to a user and an organization. Members can use them; the owner manages them. Deleting one preserves historical job records.
+
+**[Agent commands](/cli-reference/agent)**
+
+Register, inspect, and remove a custom agent.

package/skills/evolve-evals/references/core-concepts/analyze.md

@@ -0,0 +1,143 @@
+---
+title: "Analyze trials"
+description: "Review a trial's behavior and score against a rubric, with evidence for every finding."
+---
+
+An analysis reviews a trial's trace, result, and available task files. It produces explanations and evidence; it does not replace the trial's reward.
+
+```text
+1. Select settled trials
+           ↓
+2. Run one analyzer per trial
+           ↓
+3. Validate every criterion result
+           ↓
+4. Read findings + evidence + summary
+```
+
+## Start an analysis
+
+Use the ID returned by your evaluation as `$JOB_ID`:
+
+```bash
+evolve analyze "$JOB_ID" --watch
+```
+
+The manual command requires a finished job. Cancelled trials are excluded.
+
+### Choose trials
+
+```bash
+evolve analyze "$JOB_ID" --failing -l 20 --watch
+evolve analyze "$JOB_ID" -t "$TRIAL_ID" --watch
+```
+
+`--passing` selects scored trials with primary reward 1. `--failing` selects every other eligible outcome, including errors and missing rewards. They cannot be combined.
+
+Repeat `-t` with full trial IDs to select specific trials; selection happens before the `-l` cap.
+
+### Analyze automatically
+
+```bash
+evolve run \
+  -d harbor-examples@1.0 -i hello-world \
+  -a codex -m gpt-5.6-luna \
+  --max-trial-spend 1 -r 0 --analyze --watch
+```
+
+`--analyze` schedules analysis after each eligible trial settles. Analysis has its own lifecycle and cost.
+
+**Note:**
+
+When using embedded analysis through an SDK, wait for the job first, then its analyses. A temporary zero-pending analysis count does not mean a running job has finished producing trials.
+
+```typescript TypeScript
+import { jobs } from "@evolvingmachines/evolve";
+
+// job is your jobs().start(...) response.
+const finished = await jobs().watch(job.id);
+const reviewed = await jobs().watchAnalysis(finished.id);
+```
+
+```python Python
+from evolve import jobs
+
+# job is your jobs().start(...) response.
+finished = await jobs().watch(job.id)
+reviewed = await jobs().watch_analysis(finished.id)
+```
+
+## Rubrics and prompts
+
+Read the current defaults before customizing:
+
+```bash
+evolve analyze --show-defaults
+```
+
+The default rubric covers these questions:
+
+| Criterion | Question |
+| --- | --- |
+| `score_is_earned` | Did the agent do the task instead of manipulating grading? |
+| `score_is_correct` | Does the reward match the delivered result? |
+| `task_was_fair` | Could the agent know what was required? |
+| `environment_worked` | Did the environment and verifier work? |
+| `ended_by_its_own_decision` | Did the agent stop by choice? |
+| `report_is_truthful` | Do the agent's claims match the record? |
+| `worked_as_for_a_real_user` | Did it behave as it would on a real request? |
+
+### Use your own rubric or prompt
+
+A rubric is a list of uniquely named criteria. Supply TOML, YAML, or JSON:
+
+```toml rubric.toml
+[[criteria]]
+name = "changes_are_explained"
+description = "The final response explains the changes made."
+guidance = "Compare the final response with the recorded file edits."
+```
+
+```bash
+evolve analyze "$JOB_ID" -r rubric.toml -p prompt.txt --watch
+```
+
+An analyzer prompt can use `{trial_path}`, `{task_section}`, and `{criteria_guidance}`. Evolve appends the required output format.
+
+Use `-m`, `--effort`, `-e`, and `-n` for model, effort, provider, and concurrency. On job creation, use the corresponding `--analyze-*` flags. See the [full reference](/cli-reference/analyze).
+
+## The result
+
+Each criterion has an outcome, explanation, and evidence.
+
+| Outcome | Meaning |
+| --- | --- |
+| `pass` | The record supports the criterion. |
+| `fail` | The record contradicts it. |
+| `not_applicable` | The criterion has no subject in this trial. |
+| `unknown` | The available record cannot decide it. |
+
+The result is this per-criterion JSON and nothing else: Evolve derives no verdict from it. The dashboard shows a summary chip it computes from the outcomes. Under a custom rubric the chip is `flagged` when any criterion fails, `unclear` when none fails but at least one is unknown, and `clean` otherwise. Under a rubric with the default criterion names the chip reads only `score_is_earned`, `score_is_correct`, `task_was_fair`, `report_is_truthful` and `environment_worked`: a fail on the first four is `flagged`, else a failed `environment_worked` is `env_fault`, else an unknown on any of the five or a not applicable score criterion is `unclear`, else `clean`; the other two criteria are facts about the trial and never move the chip. Read the criteria and compute what you need from them. A missing or extra criterion makes the analysis result invalid. A run without a valid result can retry once.
+
+The analysis's `estimated_cost_usd` and the job's `stats.analysis.cost_usd` are separate from evaluation spend.
+
+## Read it back
+
+```bash
+evolve analysis list --job "$JOB_ID"
+evolve analysis show "$ANALYSIS_ID"
+evolve analysis trace "$ANALYSIS_ID"
+evolve analysis download "$ANALYSIS_ID" -o analyses/
+```
+
+`show`, `trace`, and `download` also accept a trial ID to read its latest analysis. Filesystem commands require the analysis ID itself.
+
+Run `evolve analyze` again to apply a new rubric or model. Only one analysis wave runs per job at a time. Earlier analyses remain accessible by their own IDs.
+
+**[Analyze reference](/cli-reference/analyze)**
+
+Selection, customization, and downloads.
+
+**[Check tasks](/core-concepts/check)**
+
+Review the benchmark before running agents.

package/skills/evolve-evals/references/core-concepts/check.md

@@ -0,0 +1,116 @@
+---
+title: "Check tasks"
+description: "Review instructions, environments, solutions, and verifiers before an evaluation."
+---
+
+A **check** reviews task quality. An [analysis](/core-concepts/analyze) reviews an agent's completed trial. Both use rubrics; they answer different questions.
+
+```bash Local task or task collection
+evolve check ./tasks --watch
+```
+
+```bash Published dataset
+evolve check -d harbor-examples@1.0 -i hello-world --watch
+```
+
+Local checks upload the selected task material. Dataset checks use the retained package; they do not publish a new version.
+
+## Choose the tasks
+
+### 1. Read the source
+
+| Source | Uses |
+| --- | --- |
+| Task directory | One task. |
+| Task collection | Top-level task directories. |
+| Published dataset | Retained task package. |
+
+### 2. Filter task names
+
+Apply include (`-i`) and exclude (`-x`) globs.
+
+### 3. Limit the selection
+
+Apply the task cap (`-l`).
+
+### 4. Run the checks
+
+Run one checker per selected task.
+
+```bash
+evolve check ./tasks -i 'auth-*' -x 'auth-legacy' -l 5 --watch
+```
+
+A check is a hosted model run and incurs cost. `evolve dataset check` is a separate metadata preflight; it does not perform this rubric review.
+
+## What it reviews
+
+| Review area | Default criteria |
+| --- | --- |
+| Instructions and tests | Sufficient instructions; tests match instructions; verifier correctness; data and labels support the work and its grading. |
+| Evaluation integrity | No answer leakage; grading is out of the agent's reach; each attempt starts clean and stays isolated from the others. |
+| Execution | Valid reference solution; rejection of non-solutions; environment works; stable verification; sufficient limits. |
+| Feasibility | The task is solvable. |
+
+Inspect the default rubric's criteria and their complete guidance, and the current model, effort, provider, and prompt:
+
+```bash
+evolve check --show-defaults
+```
+
+Use `-r rubric.toml` or `-p prompt.txt` to customize. Check prompt tokens are `{task_path}`, `{file_tree}`, and `{criteria_guidance}`. The required result format is appended automatically.
+
+## The result
+
+One parent check contains one **task check** per selected task:
+
+```text
+check
+├── task check: hello-world
+│   ├── criteria
+│   │   ├── outcomes
+│   │   ├── explanations
+│   │   └── evidence
+│   └── checker trace + files
+└── task check: another-task
+    └── ...
+```
+
+The result is this per-criterion JSON and nothing else: Evolve derives no verdict from it. The dashboard shows a summary chip it computes from the outcomes. Under a custom rubric the chip is `has_a_problem` when any criterion fails, `unclear` when none fails but at least one is unknown, and `no_problem_found` otherwise. Under a rubric with the default criterion names the same rule applies, with two exceptions: an unknown on one of the default rubric's execution criteria does not make the chip unclear but marks it not executed; and `attempt_isolation` counts only when it fails. Read the criteria and compute what you need from them.
+
+**Note:**
+
+**Read the default rubric's execution criteria** (`reference_solution_is_valid`, `verifier_rejects_non_solutions`, `environment_builds_and_runs`, `verification_is_stable`, `limits_allow_the_task`) before treating a result with no failure as proof that the task runs. They are `unknown` when decisive execution evidence is unavailable, for example when the checker could not run the environment, and that leaves execution questions unresolved.
+
+The parent lifecycle is `queued` → `running` → `completed`. A completed check can contain failed task checks; inspect each task's `status` and `failure`.
+
+## Read it back
+
+```bash
+evolve check list --status running
+evolve check show "$CHECK_ID"
+evolve check trace "$TASK_CHECK_ID"
+evolve check download "$CHECK_ID" -o checks/
+```
+
+Use the parent check ID for the whole report. Use a task-check ID for one checker's transcript, filesystem, or individual result:
+
+```bash
+evolve check download "$TASK_CHECK_ID" --stream task-check
+```
+
+## Share
+
+```bash
+evolve check share "$CHECK_ID" --link
+```
+
+The creator can share by link or email. Recipients can read the report and available files. Sharing does not give them permission to stop task checks.
+
+**[Check reference](/cli-reference/check)**
+
+Flags, custom rubrics, and task-check commands.
+
+**[Checks in the dashboard](/dashboard/checks)**
+
+Open reports and inspect findings.

package/skills/evolve-evals/references/core-concepts/compatibility.md

@@ -0,0 +1,39 @@
+---
+title: "Compatibility"
+description: "Supported Harbor-format task features and the boundaries of managed evaluation jobs."
+---
+
+Evolve imports Harbor-format task packages and exports Harbor-format results. Managed jobs use Evolve's own CLI and API contract. A local runner's configuration is not passed through automatically.
+
+Use the [task field reference](/core-concepts/task-reference) for exact accepted fields and defaults. Before publishing an existing task package, validate it:
+
+```bash
+evolve dataset check ./my-dataset
+```
+
+Validation checks declarations. Image builds, provider compatibility, and task execution still need their own checks.
+
+## Tasks and execution
+
+| Area | Managed Evolve support |
+| --- | --- |
+| Task packages | Instructions, `task.toml`, Dockerfiles or public pinned images, tests, optional solutions, skills, MCP, healthchecks, and supported [Compose](/core-concepts/compose) declarations. Unknown task fields are rejected. Windows, Apptainer, TPU configuration, and managed GUI desktop sessions are not provided. |
+| Dataset sources | [Publish](/core-concepts/datasets) local files, pinned Git sources, public HTTPS archives, or public Harbor Hub packages, then select the published dataset in a job. Arbitrary JSON task registries and direct local/repository task paths are not managed run sources. |
+| Job configuration | Use the documented [run flags](/cli-reference/run) or [SDK job fields](/sdk-reference/jobs). Host plugin classes, install-only runs, runtime mounts, extra Compose overlays, run-level artifact declarations, and per-arm concurrency groups are not accepted job options. |
+| Agent harnesses | Use a [built-in harness or registered agent](/core-concepts/agents). Registration takes an install script or uploaded files plus a headless run command. Python `BaseAgent` classes, `acp:<registry-id>` lookup, and `harbor-agent.json` manifests are not registration formats. Managed ACP simulated-user/persona configuration is not provided. |
+| Sandbox providers | [E2B, Daytona, and Modal](/core-concepts/sandboxes), subject to each task's resource and network requirements. A local sandbox runner or custom provider plugin cannot be selected for a managed job. |
+| Saved conversations | A task-level ATIF file can seed Claude Code or Codex. See [saved conversations](/core-concepts/task-config#start-from-a-saved-conversation). Run-level trajectory loading and a managed trial-to-local-CLI handoff operation are not provided. |
+| Multi-step tasks | Ordered steps share a filesystem, with per-step instructions, setup, checks, timeouts, and reward thresholds. [Multi-step tasks](/core-concepts/multi-step) use shared verification; Compose, separate verification, between-step conversation resume, and per-step artifact snapshots are not supported. |
+
+## Verification and records
+
+| Area | Managed Evolve support |
+| --- | --- |
+| Verifiers | Task `tests/test.sh` runs in a shared or separate sandbox and writes reward files. [Verifier scripts and supported judges](/core-concepts/task-verifiers) can implement custom grading; a host-side `BaseVerifier` class is not a managed input. |
+| Regrade | Eligible separate-mode trials can rerun their **recorded verifier** against retained inputs. Regrade does not select a replacement verifier and excludes shared-mode, multi-step, and judge-enabled trials. See [jobs](/core-concepts/jobs). |
+| Rewards and metrics | Tasks can produce named rewards, and jobs expose platform aggregates and pass@k. Dataset-level `metric.py` scripts are rejected. Compute additional aggregates from exported results. |
+| Artifacts | Root task declarations transfer files for separate verification and export retained payloads in the job archive. Shared verification uses the existing filesystem; inspect live or captured files through the filesystem API. See [trial outputs](/core-concepts/trial-outputs#declared-artifacts-in-the-job-archive). |
+| Credentials and images | [Job-attached secrets](/core-concepts/secrets) supply supported agent variables; managed judge credentials use the verifier contract. Task images must be publicly readable. User-managed private image-registry credential storage or selection is not exposed. |
+| Sharing and leaderboards | [Share jobs, task checks, and sessions](/core-concepts/sharing), or use team ownership. Public dataset visibility is platform-managed. Job comparison and exported results are available; a user-managed leaderboard object or ranking API is not provided. |
+
+These boundaries describe the managed evaluation service. Custom agent and verifier code still runs within the documented sandbox and credential contracts.

package/skills/evolve-evals/references/core-concepts/compose.md

@@ -0,0 +1,103 @@
+---
+title: "Multiple containers"
+description: "Give a task its own API, database, or other service beside the agent."
+---
+
+The agent works in a service called `main`. Other containers provide the services it needs. All belong to one trial.
+
+```mermaid
+%%{init: {"flowchart": {"nodeSpacing": 16, "rankSpacing": 20, "padding": 10}, "themeVariables": {"fontSize": "14px", "fontFamily": "Google Sans Flex, sans-serif"}}}%%
+flowchart LR
+  M["main<br/>Agent and workspace"] --> A["api:8000<br/>Task service"]
+```
+
+## Add a small HTTP service
+
+This environment gives the agent a JSON file through an HTTP service. Add it to a task with an [instruction and verifier](/core-concepts/tasks).
+
+- my-task/
+  - instruction.md
+  - task.toml
+  - environment/
+    - Dockerfile
+    - docker-compose.yaml
+    - api/
+      - Dockerfile
+      - status.json
+  - tests/
+    - test.sh
+
+```dockerfile environment/Dockerfile
+FROM python:3.12-slim
+WORKDIR /app
+```
+
+```dockerfile environment/api/Dockerfile
+FROM python:3.12-slim
+WORKDIR /srv
+COPY status.json .
+CMD ["python3", "-m", "http.server", "8000", "--bind", "0.0.0.0"]
+```
+
+```json environment/api/status.json
+{"status": "ready", "items": 3}
+```
+
+Name the Compose file exactly `environment/docker-compose.yaml`:
+
+```yaml environment/docker-compose.yaml
+services:
+  main:
+    depends_on:
+      api:
+        condition: service_healthy
+  api:
+    build:
+      context: ./api
+    healthcheck:
+      test:
+        - CMD
+        - python3
+        - -c
+        - >-
+          import urllib.request;
+          urllib.request.urlopen(
+            'http://localhost:8000/status.json'
+          )
+      interval: 2s
+      timeout: 5s
+      retries: 15
+```
+
+Evolve builds `main` from `environment/Dockerfile` and `api` from its own Dockerfile. `depends_on` waits for the API healthcheck. From `main`, the agent can read `http://api:8000/status.json` using the service name as the hostname.
+
+The `main` stanza above only adds a dependency. You can omit it when you have no overrides. To use a prebuilt main image, set `[environment].docker_image` in `task.toml`.
+
+## Images, files, and configuration
+
+| Choice | Rule |
+| --- | --- |
+| Service `image` | Use a public image with an explicit tag or digest |
+| Service `build` | Keep the context inside the task directory and the Dockerfile inside that context |
+| Build options | `context`, `dockerfile`, `args`, `target`, `no_cache`, `pull`, and `labels` |
+| Service dependencies | Use Compose `depends_on` and healthchecks |
+| Container files | Include bind-mounted source files inside the task's `environment/` directory |
+| Runtime configuration | Compose ports, volumes, networks, `environment`, and `env_file` are handled by Compose |
+
+Images are resolved during dataset import. Runtime tasks use the recorded images. Task files and image-build inputs are not a secret store; use [job secrets](/core-concepts/secrets) for agent credentials.
+
+### Configuration that import rejects
+
+- `include` and service `extends`, which can introduce unseen service images.
+- Interpolated image names, untagged images, and an unpinned `:latest` tag.
+- Build options such as `ssh`, `secrets`, additional contexts, and inline Dockerfiles.
+- A build context outside the task, or a Dockerfile outside its build context.
+- `main.build` together with `[environment].docker_image`, or conflicting main image declarations.
+
+## Provider and task limits
+
+Compose tasks run on **E2B or Daytona**. They do not support Modal, `no-network`, GPUs, or multi-step execution. [Sandbox capabilities](/core-concepts/sandboxes) describes provider selection and other network restrictions.
+
+## Score service state
+
+A verifier may need more than files from `main`: a request log, a database export, or an in-memory counter. Use [collection hooks and service artifacts](/core-concepts/task-verifiers#collect-from-a-service) to save that evidence before verification.