@evolvingmachines/evolve 0.0.65 → 0.0.66

package/skills/publish/SKILL.md

@@ -1,146 +1,170 @@
 ---
 name: publish
-description: Publish a dataset of Harbor-format tasks to Evolve, or upload a finished job folder — a Harbor job, or Evolve SDK runs packed as one. Use when the user wants to publish, upload, or share tasks, datasets/benchmarks, or job results on Evolve.
+description: Publish Harbor-format task datasets or upload existing results to Evolve, including recorded Evolve SDK runs packaged as jobs. Use when the user wants to publish a benchmark, upload tasks, or turn an SDK session into an evaluation job.
 metadata:
   internal: true
 ---
 
-Help the user publish a dataset of tasks to Evolve, or upload a job they ran elsewhere.
-Walk them through each step, checking prerequisites and confirming before running
-commands that upload.
+# Publish to Evolve
 
-## Prerequisites
+Publish tasks for future evaluations, or import results from a run that already happened. Both workflows use Evolve.
 
-1. **The CLI**: `npm install -g @evolvingmachines/evolve`; `evolve --version` succeeds.
+| What the user has | Start here |
+| --- | --- |
+| A folder or repository of tasks | Publish a dataset below. For the task layout, read `evolve skills get create-task`. |
+| A completed job folder or archive | Import results below. |
+| A recorded Evolve SDK session | Read `evolve skills get evals core-concepts/upload-sdk-session` before packaging it. |
 
-2. **API key**: create a key on the dashboard's API keys page
-   (https://dashboard.evolvingmachines.ai/api-keys) and export it. Every command reads
-   `EVOLVE_API_KEY`.
-   ```bash
-   export EVOLVE_API_KEY="<your key>"
-   evolve auth status
-   ```
-   `auth status` prints who the platform thinks you are and which key is in use.
+## Setup, when needed
 
-3. **Task layout**: a dataset is a folder of task directories, each with `task.toml`,
-   `instruction.md`, `tests/test.sh`, and an `environment/` folder unless `task.toml`
-   names a prebuilt `docker_image` (`evolve skills get create-task` has the format). The
-   directory name is the task's name: letters, digits, `.`, `_` and `-`, at most 128
-   characters, starting with a letter or digit; use lowercase (Harbor's convention). A
-   `dataset.toml` manifest at the root is optional.
+Use the installed CLI; install only if missing: `npm install -g @evolvingmachines/evolve`. Confirm authentication with `evolve auth status`. If needed, set `EVOLVE_API_KEY` using a key from https://dashboard.evolvingmachines.ai/api-keys.
 
-## Publishing a dataset
+Publishing or importing is a remote mutation. Proceed when the user's request authorizes it; otherwise prepare the exact command and explain what it uploads. An upload request does not authorize emailing or sharing results. For explicit sharing requests, read `evolve skills get evals core-concepts/sharing`.
 
-What you publish is private to your organization. There are no tags and no visibility
-flag.
+## Publish a dataset
 
-### 1. Check the folder first
+### 1. Check the tasks
+
+Inspect the task directories and any `dataset.toml`, then validate their metadata:
 
 ```bash
-evolve dataset check "<path/to/tasks>"
+evolve dataset check ./tasks
 ```
 
-A dry run: the pre-flight sends each task's `task.toml`, and the `dataset.toml` if there
-is one, to the server, which answers with a verdict per task and writes nothing. A refused
-task names the field to fix. `evolve check "<path/to/tasks>" --watch` goes further: it
-reads each task and, when it can, runs its environment, reference solution and verifier,
-then rules on a rubric.
+This sends each `task.toml` and the optional manifest to Evolve. It does not publish the corpus or run the tasks. Fix refused fields before continuing.
+
+For an agent-based quality review, use `evolve check ./tasks --watch`. This uploads tasks and incurs agent usage. Read the findings and trace: Evolve derives no verdict from the result, and `unknown` on the default rubric's execution criteria is not proof that the reference solution and verifier were executed successfully. See `evolve skills get evals core-concepts/check`.
 
-### 2. Publish
+### 2. Choose one source and publish
 
-From a local directory:
+**Local directory**
 
 ```bash
 evolve dataset publish \
-  --dir "<path/to/tasks>" \
-  --name "<dataset>" \
+  --dir ./tasks \
+  --name my-benchmark \
   --version 1.0 \
   --watch
 ```
 
-When the folder carries a `dataset.toml` manifest, `--name` and `--version` come from it
-and may be omitted. The pre-flight runs automatically before the upload;
-`--skip-preflight` uploads without it, and a task the check would have refused then fails
-at import instead.
+A `dataset.toml` can supply the name and version. Local publishing runs metadata preflight automatically; `--skip-preflight` skips that early check, not import validation.
 
-From a git repository:
+**Git repository**
 
 ```bash
 evolve dataset publish \
-  --git https://github.com/acme/my-swe.git \
+  --git https://github.com/acme/tasks.git \
   --ref v1.0.0 \
-  --name "<dataset>" \
+  --path benchmark \
+  --name my-benchmark \
   --version 1.0 \
   --watch
 ```
 
-`--ref` must be pinned: a tag, or a full 40-character commit sha. A branch name is
-refused. `--path <subfolder>` imports one folder of a larger repository.
+Pin a tag or a full 40-character commit SHA. Branch names are refused. Omit `--path` to use the repository root.
 
-From a source the server fetches itself:
+**Public HTTPS archive**
 
 ```bash
-evolve dataset publish --from hub:cookbook/hello-world --watch
+evolve dataset publish \
+  --from https://example.com/tasks.tar.gz \
+  --name my-benchmark \
+  --version 1.0 \
+  --watch
 ```
 
-`--from` takes a public https tarball URL, or `hub:org/name[@ref]` for a public package on
-the Harbor hub. For a hub package the name and version default to the package's own.
-
-### 3. Follow the publish
+Evolve fetches the archive. Both Git and HTTPS sources require an explicit name and version.
 
-`--watch` follows the publish until the version is `READY` or `FAILED`. Each task builds
-on its own, so one broken task does not block the others; `--watch` ends with how many
-built. If the terminal is gone, re-attach from any machine:
+**Optional: import a public Harbor Hub package into Evolve**
 
 ```bash
-evolve dataset watch "<dataset>"
+evolve dataset publish \
+  --from hub:cookbook/hello-world \
+  --watch
 ```
 
-The version lands `READY` when at least one task built, and `FAILED` only when none did.
-On your own dataset, `READY` also makes the version active, so the bare name runs it.
+`hub:org/name[@ref]` is an accepted source; the destination is Evolve. Name and version may come from the package. Keep this syntax when importing an actual Harbor package.
 
-## After publishing
+For a new dataset, add `--org <organization>` to select an organization. Otherwise, Evolve uses the saved `evolve auth org use` default, then your personal organization. An existing dataset keeps its organization; an explicit or saved default that conflicts is refused. See `evolve skills get evals cli-reference/dataset` for all options.
+
+### 3. Follow the import and inspect the result
+
+`--watch` follows the import and build. To reconnect, use the complete import ID printed by publishing:
 
 ```bash
-evolve dataset show "<dataset>@1.0"                       # versions, tasks, timeouts, providers per task
-evolve run -d "<dataset>@1.0" -a codex -m gpt-5.5 --watch   # run a job on it
+evolve dataset watch "$IMPORT_ID"
+evolve dataset show my-benchmark@1.0
 ```
 
-Each publish creates a version, named `<dataset>@<version>`; a bare name means the active
-version. To point the bare name at a different `READY` version:
+`evolve dataset watch my-benchmark` also finds a live import. Use the import ID after it has settled.
+
+A `READY` version needs at least one built task, but can contain failed tasks. Report the final state, name, version, built and failed counts, and failure reasons. A new ready version on your own dataset becomes active; a bare dataset name selects that active version.
+
+### 4. Run or manage the version
+
+When the user wants an evaluation, select the harness and model, then run the pinned version:
 
 ```bash
-evolve dataset activate "<dataset>" 1.0
+evolve run \
+  --dataset my-benchmark@1.0 \
+  --agent codex \
+  --model "<model>" \
+  --watch
 ```
 
-The owner of a dataset can download the original package back:
+Replace `<model>` with the selected model. Read `evolve skills get evals cli-reference/run` for job settings.
+
+| Need | Command |
+| --- | --- |
+| Point the bare name at another `READY` version | `evolve dataset activate my-benchmark 1.0` |
+| Download the original corpus package as its owner | `evolve dataset download my-benchmark@1.0 -o corpora/` |
+| Fix tasks after a partial build | Publish the corrected corpus under a new version. |
+
+## Import results
+
+### 1. Prepare the job folder
+
+An existing Harbor-format job folder can be uploaded directly. Before packaging another runner's files, read `evolve skills get evals core-concepts/upload` for the required layout and fields. The root needs `config.json` and `result.json`; each trial has its own `result.json`.
+
+For an SDK session, read `evolve skills get evals core-concepts/upload-sdk-session` first. It shows the local log location, field mapping, and packaging script. A session can contain several runs or commands; do not assume one file is one trial. The example script supports one successful Codex or Claude run; the guide covers other cases.
+
+Preserve recorded rewards in each trial's `verifier_result.rewards`. Without recorded rewards, the import has no score. Do not infer a score from agent completion.
+
+### 2. Upload and follow
 
 ```bash
-evolve dataset download "<dataset>@1.0" -o corpora/
+evolve upload ./completed-job -d my-benchmark@1.0
 ```
 
-## Uploading a job you ran elsewhere
+Use the job directory or its `.tar.gz`. Alternatively, use `evolve upload --from <public-https-archive-url>`. Omit `-d` if no published dataset should be linked; when supplied, it links trial tasks by name.
 
-A job run elsewhere, in the Harbor job layout, uploads as a finished job. Its trials,
-traces and rewards become a job you read like any other. Before packing a folder by
-hand — runs made with the Evolve SDK, or any other runner — read what a trial folder
-and its `result.json` must hold: `evolve skills get evals core-concepts/upload`. The
-score lives in `result.json` as `verifier_result.rewards`; without it a trial arrives
-with no score.
+The CLI waits for the import by default. `--no-wait` returns the import record instead. Reconnect with:
 
 ```bash
-evolve upload "<path/to/job-dir>" -d "<dataset>@1.0"
+evolve job imports --status RUNNING
+evolve job import "$IMPORT_ID" --watch
 ```
 
-`upload` takes the job directory, or its `.tar.gz`. With `--from <url>` it takes instead
-a public https URL of the archive, which the server fetches itself. `-d name[@version]`
-links the uploaded trials to a published dataset version by task name. The command
-follows the import until the job exists; `--no-wait` returns at once with the import id.
+### 3. Inspect the imported job
+
+Use the job ID from the completed import:
 
 ```bash
-evolve job imports --status RUNNING
-evolve job import <import-id> --watch
+evolve job show "$JOB_ID"
+evolve job trials "$JOB_ID"
 ```
 
-`job imports` lists your uploads, newest first. `job import --watch` re-attaches to one and
-follows it to the job, or to its typed failure.
+Choose a trial ID from the listing, then read its transcript with `evolve trial trace "$TRIAL_ID"`. Check the task, harness, model, prompt, agent activity, and recorded scores. Report skipped trial folders and missing task links.
+
+Imported jobs support analysis, but cannot be resumed, retried, or regraded. Read `evolve skills get evals cli-reference/upload` for all upload options.
+
+## Use Python or TypeScript instead
+
+Read `evolve skills get evals sdk-reference/index` for client setup, then the matching method reference:
+
+| Operation | Read |
+| --- | --- |
+| Validate, publish, follow, activate, or download datasets | `evolve skills get evals sdk-reference/methods/datasets` |
+| Upload a job and follow its import | `evolve skills get evals sdk-reference/methods/jobs` |
+
+Both SDKs expose `datasets().preflight()` and `datasets().publish()`, plus `jobs().upload()`. Call preflight separately when publishing through an SDK. Publishing and uploading return import records; follow with `watchImport` in TypeScript or `watch_import` in Python. These calls do not convert SDK session logs into job folders; package the session first.

package/skills/rewardkit/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: rewardkit
-description: Write Harbor-format task verifiers using Reward Kit. Use when creating or editing a 
+description: Write verifiers for Evolve's Harbor-format tasks using Reward Kit. Use when creating or editing a
   task's tests/ directory, adding grading criteria, setting up LLM/agent judges, or designing 
   verifiers that produce a reward score.
 metadata:
@@ -11,6 +11,10 @@ Help the user write task verifiers with Reward Kit. Reward Kit is a lightweight
 package that turns a directory of criteria files into a reward score. Each criterion is a 
 Python function call or a TOML judge file; folders become separate rewards.
 
+The package keeps its upstream name, `harbor-rewardkit`. For Evolve's supported
+setup and a complete example, read
+`evolve skills get evals core-concepts/rewardkit`.
+
 ## Setup in a task
 
 Put criteria alongside `test.sh` in the task's `tests/` directory:
@@ -22,17 +26,33 @@ tests/
 └── judge.toml        # optional LLM/agent judge
 ```
 
-`tests/test.sh`:
+Install the package when the image builds, using Python 3.12 or newer. In shared
+mode, add this to `environment/Dockerfile`; in separate mode, install it in the
+verifier image:
+
+```dockerfile
+RUN pip install --no-cache-dir \
+    'harbor-rewardkit==0.2.1'
+```
+
+Then write `tests/test.sh`:
 ```bash
 #!/bin/bash
-uvx --from 'harbor-rewardkit==0.2.*' rewardkit /tests
+set -euo pipefail
+python3 -m rewardkit /tests \
+  --workspace /app \
+  --output /logs/verifier/reward.json
 ```
 
 This runs all criteria in `/tests/` against the workspace at `/app` and writes 
-`/logs/verifier/reward.json`. Defaults match Harbor's conventions — no extra config needed.
+`/logs/verifier/reward.json`. Adjust `--workspace` if the task uses another directory.
 
-Run `evolve check "<task-path>" --watch` to run the verifier against the task's reference 
-solution before publishing the task.
+Run `evolve check "<task-path>" --watch` for task quality review. The checker
+may run the reference solution and verifier when its environment supports
+them; inspect its findings and evidence. The default rubric's execution
+criteria say whether it ran the task; `unknown` on them is not proof that the
+solution or verifier ran. Read `evolve skills get evals core-concepts/check`
+for the result semantics.
 
 If judge criteria need API keys, request them through `task.toml`:
 ```toml
@@ -40,11 +60,16 @@ If judge criteria need API keys, request them through `task.toml`:
 ANTHROPIC_API_KEY = "${ANTHROPIC_API_KEY}"
 ```
 
-On Evolve you never put a real key in the task: write the template exactly as above, as
-the whole value, and the judge's credential is supplied at run time. The rubric names the
-judge model, or Reward Kit's own default applies. A job can override the judge for every
-verifier with `evolve run --ve REWARDKIT_JUDGE=<judge> --ve REWARDKIT_MODEL=<model>`; no
-other verifier env key is accepted on a job.
+Write the template literally as the whole value. Evolve supplies a scoped gateway
+credential and base URL during verification. Set the judge model explicitly in
+the TOML. Use `OPENAI_API_KEY` instead for an OpenAI-family judge.
+
+On a job, `--ve REWARDKIT_JUDGE=<judge>` overrides the judge and
+`--ve REWARDKIT_MODEL=<model>` selects an agent judge's model. These are the only
+two job-level verifier environment overrides. Read
+`evolve skills get evals cli-reference/run` for CLI options or
+`evolve skills get evals sdk-reference/methods/jobs` for the Python and TypeScript
+`start` inputs.
 
 Ask whether Reward Kit should run in the agent's shared environment or in a
 separate verifier environment. Prefer a separate verifier environment when judge
@@ -53,14 +78,14 @@ available to the agent:
 
 ```toml
 [environment]
-network_mode = "no-network"   # Agent env baseline — offline during agent.run()
+network_mode = "no-network"   # Agent can reach the Evolve model gateway
 
 [verifier]
 environment_mode = "separate"
 
 [verifier.environment]
-network_mode = "public"     # Verifier env baseline — LLM judge API calls
-docker_image = "python:3.12-slim"
+network_mode = "no-network"   # Native judge calls use Evolve's gateway
+# With no docker_image, build the verifier from tests/Dockerfile.
 ```
 
 In shared mode, the verifier runs in the agent container and inherits
@@ -69,9 +94,9 @@ baseline is refused at import on Evolve, because a shared verify cannot switch
 egress. If agent and verifier need different network access, use
 `environment_mode = "separate"` and set `[verifier.environment].network_mode`.
 
-Judge criteria that call external APIs need a `public` baseline or allowlist on
-the verifier environment. Programmatic checks that only read local files can use
-`no-network`.
+Evolve's native judge gateway is reachable with `no-network`. Other external
+APIs need a `public` baseline or the required hosts in the verifier allowlist.
+Programmatic checks that only read local files can also use `no-network`.
 
 In separate mode with no `[verifier.environment].docker_image`, `tests/` is the
 verifier image's build context and its `tests/Dockerfile` must provide
@@ -79,6 +104,12 @@ verifier image's build context and its `tests/Dockerfile` must provide
 uploads `tests/` to `/tests` instead. When it pins a distinct image, that image
 boots as it is with nothing uploaded, so it must carry `/tests/test.sh` itself.
 
+For separate verification, declare the files to grade in the task's `artifacts`
+list. The verifier does not receive the agent's whole workspace. Read
+`evolve skills get evals core-concepts/task-verifiers` and
+`evolve skills get evals core-concepts/task-config` for Evolve's file-transfer and
+network rules.
+
 ## Programmatic criteria
 
 Call built-ins from any `.py` file in `tests/`:
@@ -95,6 +126,10 @@ rk.json_key_equals("result.json", "status", "ok")
 All criteria accept `weight` (default `1.0`) and `isolated` (default `False`, runs in 
 overlayfs so side effects don't leak).
 
+Isolation needs working overlayfs support. If the image needs `fuse-overlayfs`,
+install it at build time; do not rely on its runtime installer with restricted
+network access. Confirm the sandbox permits the required mounts.
+
 ### Available built-ins
 
 - **Files**: `file_exists`, `file_not_exists`, `file_contains`, `file_contains_regex`, 
@@ -102,12 +137,17 @@ overlayfs so side effects don't leak).
 - **Commands**: `command_succeeds`, `command_output_contains`, `command_output_matches`, 
   `command_output_matches_regex` (30s default timeout, optional `cwd`)
 - **Data**: `json_key_equals`, `json_path_equals`, `csv_cell_equals`, `xlsx_cell_equals` 
-  (needs `[office]` extra), `sqlite_query_equals`
+  (needs `[documents]` extra), `sqlite_query_equals`
 - **HTTP**: `http_status_equals`, `http_response_contains`
 - **Images**: `image_similarity`, `image_size_equals` (needs `[image]` extra)
 - **Trajectory**: `trajectory_tool_used`, `trajectory_tool_not_used`, `trajectory_turn_count`
 
-For extras, install with `uv tool install harbor-rewardkit[all]`.
+Install needed extras in that same image: `harbor-rewardkit[documents]==0.2.1`
+or `harbor-rewardkit[image]==0.2.1`. Use `[all]` only if both are needed.
+
+Do not depend on downloading packages during verification. Evolve's offline
+Reward Kit bundle is for judge-enabled runs and does not include optional
+extras. Its `uvx` path cannot fetch versions or extras absent from the bundle.
 
 ## Custom criteria
 
@@ -143,7 +183,7 @@ For subjective checks (quality, readability, edge cases), create a TOML file:
 
 ```toml
 [judge]
-judge = "anthropic/claude-sonnet-5"   # LiteLLM model string
+judge = "anthropic/claude-sonnet-4-6"
 files = ["/app/main.py"]
 
 [[criterion]]
@@ -169,7 +209,7 @@ Agent judges shell out to a CLI and can explore the filesystem:
 ```toml
 [judge]
 judge = "claude-code"
-model = "anthropic/claude-sonnet-5"
+model = "anthropic/claude-sonnet-4-6"
 isolated = true
 
 [[criterion]]
@@ -179,6 +219,10 @@ type = "binary"
 
 Slower and more expensive than LLM judges, but they can run commands and inspect files.
 
+Evolve supports `claude-code` and `codex` agent judges. Install the selected CLI
+in the verifier image and request its matching provider credential. The managed
+judge path does not supply verifier MCP tools or arbitrary provider credentials.
+
 ### Useful `[judge]` options
 
 `timeout` (default 300), `reasoning_effort` (`low`|`medium`|`high`), `reference` (path to 
@@ -289,8 +333,15 @@ weight unless that aggregation's inline map overrides them;
 - `/logs/verifier/reward.json` — per-reward scores
 - `/logs/verifier/reward-details.json` — per-criterion results, judge reasoning, errors
 
+Evolve uses `reward` as the primary score, or the only value when there is one
+key. Multiple dimensions without `reward` remain metrics but have no primary
+score. Keep the primary score in `[0, 1]`.
+
 ## Multi-step tasks
 
+Evolve currently requires shared verification for multi-step tasks; separate
+verifier environments are rejected.
+
 In a multi-step task, each step has its own `tests/` under
 `steps/{name}/tests/`, and the verifier runs once per step. Reward Kit behaves
 the same as in a single-step task: for each step it reads `/tests`, runs the
@@ -311,6 +362,9 @@ separate reward key for that step, and `multi_step_reward_strategy = "mean"`
 averages each key across steps. Use `"final"` when the last step is an
 end-to-end check whose rewards already represent the full task.
 
+Read `evolve skills get evals core-concepts/multi-step` for Evolve's supported
+step configuration and result layout.
+
 ## When to reach for what
 
 - **Use built-ins** for file existence, string matches, command output, JSON/CSV checks, 
@@ -326,5 +380,6 @@ end-to-end check whose rewards already represent the full task.
 
 ## Working example
 
-See https://github.com/laude-institute/harbor/tree/main/examples/tasks/reward-kit-example.
-Reward Kit's own documentation: https://docs.harborframework.com/core-concepts/rewardkit/quick-start.
+Read `evolve skills get evals core-concepts/rewardkit` for a complete verifier,
+or `evolve skills get evals getting-started/first-task` for the task's build,
+publish, check, and evaluation workflow.

package/spec/openapi.yaml

@@ -4994,8 +4994,8 @@ paths:
       description: |
         The resolved policy an empty `AnalyzeConfigInput` falls back to — the
         five knobs every analysis that names none is stamped with:
-        `model_name`, `rubric` (the platform's analyze rubric, the seven
-        criteria `AnalyzeConfigInput` lists), `prompt` (the built-in analyze
+        `model_name`, `rubric` (the platform's default analyze rubric),
+        `prompt` (the built-in analyze
         body, as the TEMPLATE a caller's own `prompt` replaces — with its
         `{trial_path}`, `{task_section}` and `{criteria_guidance}` slots
         unrendered; the output section Harbor appends is not part of it),
@@ -5835,8 +5835,8 @@ paths:
         refusal answers. The `config`
         part is the JSON `CheckConfigInput`; absent or
         `{}` means the defaults (openrouter/deepseek/deepseek-v4.1-flash at its
-        per-model effort, the platform's default check rubric — eleven
-        criteria, `CheckConfigInput.rubric` names them).
+        per-model effort, the platform's default check rubric, served by
+        `GET /api/checks/defaults`).
 
         The task bytes are never modified and never land anywhere but the
         checker's sandbox and the platform's rows: Harbor's own rule ("Do
@@ -5973,8 +5973,8 @@ paths:
       description: |
         The resolved policy an empty `CheckConfigInput` falls back to — the
         five knobs the create door stamps on every check that names none:
-        `model_name`, `rubric` (the platform's check rubric, the eleven
-        criteria `CheckConfigInput` lists), `prompt` (the built-in check
+        `model_name`, `rubric` (the platform's default check rubric),
+        `prompt` (the built-in check
         body, as the TEMPLATE a caller's own `prompt` replaces — with its
         `{task_path}`, `{file_tree}` and `{criteria_guidance}` slots
         unrendered; the output section Harbor appends is not part of it),
@@ -12318,13 +12318,10 @@ components:
           pattern: '^[a-z][a-z0-9_]*$'
           description: >
             Criterion identifier, snake_case (it keys the result's `checks`
-            object). The platform's default analyze rubric names seven
-            (score_is_earned, score_is_correct, task_was_fair,
-            environment_worked, ended_by_its_own_decision,
-            report_is_truthful, worked_as_for_a_real_user) and its default
-            check rubric eleven (CheckConfigInput lists them); the derived
-            labels read those names, and a rubric with other names carries
-            no label.
+            object). The platform's default rubrics are served by
+            `GET /api/analyses/defaults` and `GET /api/checks/defaults`; the
+            platform reads no criterion name — a run's result is its
+            per-criterion table under whatever names its rubric carries.
         description:
           type: string
           minLength: 1
@@ -12370,9 +12367,9 @@ components:
         it configures that manual wave. `{}` is legal and means "all
         defaults": openrouter/deepseek/deepseek-v4.1-flash at its per-model effort
         (high) over
-        the platform's default analyze rubric (the seven criteria
-        `RubricCriterion.name` lists; Harbor's own analyze-rubric.toml
-        still runs when passed as `rubric`, and carries no label).
+        the platform's default analyze rubric (served by
+        `GET /api/analyses/defaults`; Harbor's own analyze-rubric.toml
+        still runs when passed as `rubric`).
 
         The analyzer always runs the claude-code harness (Harbor's default
         analyze agent) in its own sandbox, under the network Harbor's
@@ -14786,8 +14783,7 @@ components:
         AnalyzeResult (their analyze/models.py: `summary`, `checks` keyed
         by criterion, `estimated_cost_usd`; `trial_name` is the enclosing
         trial) with each check carrying the platform's evidence list and
-        fourth outcome (AnalysisCheck), plus the derived `label`; the rest
-        is provenance — which model, rubric and prompt THIS analysis ran
+        fourth outcome (AnalysisCheck); the rest is provenance — which model, rubric and prompt THIS analysis ran
         under, its lifecycle status, and its typed failure when it failed.
 
         `estimated_cost_usd` is the analyzer agent's OWN metered spend (its
@@ -14807,7 +14803,6 @@ components:
         - prompt
         - summary
         - checks
-        - label
         - estimated_cost_usd
         - failure
         - created_at
@@ -14878,24 +14873,6 @@ components:
             criterion names (the frozen-criteria law). Null until completed.
           additionalProperties:
             $ref: '#/components/schemas/AnalysisCheck'
-        label:
-          type: [string, 'null']
-          enum: [flagged, env_fault, unclear, clean, null]
-          description: >
-            The derived label — computed by the platform from the outcomes
-            when the result is stored, never asked from the model, so it is
-            the same rule for every analysis and can never contradict the
-            checks. The rule reads the default analyze rubric's criteria,
-            precedence in this order: `flagged` on a fail of
-            score_is_earned, score_is_correct, task_was_fair or
-            report_is_truthful; else `env_fault` on a fail of
-            environment_worked; else `unclear` on an unknown of any of those
-            five, or a not_applicable of score_is_earned (the agent never
-            acted) or score_is_correct (no reward exists); else `clean`.
-            ended_by_its_own_decision and worked_as_for_a_real_user never
-            change it. Null until completed, and null on a completed
-            analysis whose rubric is not the default one — a custom rubric
-            carries its per-criterion outcomes and no label.
         estimated_cost_usd:
           type: [number, 'null']
         usage:
@@ -15036,14 +15013,8 @@ components:
         `claude-sonnet-4-6`, :101-103 — this platform's is the analyzer's
         `openrouter/deepseek/deepseek-v4.1-flash`, one roster and one default for both rubric
         agents, a recorded deviation), `rubric` (`-r/--rubric`; the default
-        is the platform's check rubric, eleven criteria:
-        instruction_is_sufficient, tests_match_the_instruction,
-        verifier_is_correct, no_answer_leakage, grader_is_out_of_reach,
-        reference_solution_is_valid, verifier_rejects_non_solutions,
-        environment_builds_and_runs, verification_is_stable,
-        limits_allow_the_task, task_is_solvable — the derived `label` and
-        `executed` read them, and a rubric with other names carries
-        neither; Harbor's own default-rubric.toml still runs when passed),
+        is the platform's check rubric, served by `GET /api/checks/defaults`;
+        Harbor's own default-rubric.toml still runs when passed),
         `prompt` (`-p/--prompt`, the file's TEXT: it replaces the built-in
         check prompt, in Harbor's check.txt slot, and is rendered with
         `{task_path}`, `{file_tree}`, `{criteria_guidance}` — checker.py:
@@ -15152,8 +15123,7 @@ components:
         One task's quality check — Harbor's QualityCheckResult shape
         (their cli/quality_checker/models.py:31-35: `task_name`, `checks`
         keyed by criterion, `cost_usd`), its checks extended by the result
-        schema and the derived `label` and `executed` beside them, plus the
-        hosted record: its own
+        schema, plus the hosted record: its own
         id, the check it belongs to, its lifecycle (the analysis ladder's
         four lowercase words — Harbor's dialect), the bounded attempt count,
         and a typed `failure` in place of Harbor's `error` string (the
@@ -15163,11 +15133,10 @@ components:
         (Harbor's check-task-template/tests/validate.py extended by the
         result schema): one key per rubric criterion, each an AnalysisCheck
         `{outcome, explanation, evidence}` — no summary (a check has none;
-        analyze does). `label` and `executed` are derived from it when the
-        result is stored. `cost_usd` is the checker agent's OWN metered
+        analyze does). `cost_usd` is the checker agent's OWN metered
         spend on its capped key; null when nothing was measured, never a
         fabricated 0.
-      required: [id, check_id, task_name, status, checks, label, executed, cost_usd, attempts, failure, created_at, finished_at]
+      required: [id, check_id, task_name, status, checks, cost_usd, attempts, failure, created_at, finished_at]
       properties:
         id:
           type: string
@@ -15186,30 +15155,6 @@ components:
           description: One entry per rubric criterion, keys exactly the frozen criteria. Null until completed.
           additionalProperties:
             $ref: '#/components/schemas/AnalysisCheck'
-        label:
-          type: [string, 'null']
-          enum: [has_a_problem, unclear, no_problem_found, null]
-          description: >
-            The derived label — computed by the platform from the outcomes
-            when the result is stored, never asked from the model. The rule
-            reads the default check rubric's criteria, precedence in this
-            order: `has_a_problem` on a fail of any criterion; else
-            `unclear` on an unknown of any of the six file-based criteria
-            (instruction_is_sufficient, tests_match_the_instruction,
-            verifier_is_correct, no_answer_leakage, grader_is_out_of_reach,
-            task_is_solvable); else `no_problem_found`. Null until
-            completed, and null on a completed check whose rubric is not
-            the default one — a custom rubric carries its per-criterion
-            outcomes and no label.
-        executed:
-          type: [boolean, 'null']
-          description: >
-            Whether the box ran the task's environment: true when none of
-            the five run-based criteria (reference_solution_is_valid,
-            verifier_rejects_non_solutions, environment_builds_and_runs,
-            verification_is_stable, limits_allow_the_task) is unknown, so a
-            reading-only `no_problem_found` is never mistaken for a run.
-            Null exactly when `label` is null.
         cost_usd:
           type: [number, 'null']
         attempts: