outerloop-science 0.4.0rc1__tar.gz → 0.6.0__tar.gz

outerloop_science-0.6.0/.claude-plugin/marketplace.json

@@ -0,0 +1,14 @@
+{
+  "name": "outerloop",
+  "owner": {
+    "name": "Outerloop"
+  },
+  "plugins": [
+    {
+      "name": "outerloop",
+      "source": "./plugins/outerloop",
+      "category": "Productivity"
+    }
+  ],
+  "description": "Set up and operate Outerloop from your coding agent."
+}

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/.pre-commit-config.yaml

@@ -14,7 +14,7 @@ repos:
       - id: check-toml
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.16.6
+    rev: v0.17.1
     hooks:
       - id: ruff-check
         args: [--fix]

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/CHANGELOG.md

@@ -6,6 +6,81 @@ Versions follow [SemVer](https://semver.org).
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-10-26
+
+### Upgrading
+
+No operator action is needed before upgrading; details in each entry.
+
+What changes for runs in flight: runs started before the upgrade have no
+trajectory until their next recorded event, and their earlier session events
+count as author sessions. The first run to end creates the attestation key
+`trajectory_attest.key`, owner-only, beside the operator settings file. On
+macOS without an image, sessions, evaluations and panel lenses now run under
+the Seatbelt profile, which allows outbound connections only to TCP port 443,
+DNS and local ports.
+
+Before rolling back: if you switched the author to a Claude subscription
+token, switch back to an API key, since older kernels send the token as
+`ANTHROPIC_API_KEY`. Older kernels ignore `trajectory.jsonl`, the attestation
+key and `OUTERLOOP_AUTHOR_EFFORT`, and pending ledger records are unchanged.
+
+### Changes
+
+- `OUTERLOOP_AUTHOR_EFFORT` sets the author's effort level (`low`, `medium`, `high`, `xhigh` or `max`) on every backend: Claude Code's `--effort` (the pinned 2.1.285 has the flag), Codex's `model_reasoning_effort` as a kernel-owned `-c` setting, and Hermes's `agent.reasoning_effort`. For Hermes the setting is written to the seeded `config.yaml`, and the capture wrapper also hands it to the agent, because `run_agent.main` builds its agent without a reasoning config and would otherwise ignore it. Unset keeps each harness's default. Panel judges and the steward ignore it, so a cheaper author does not change how its work is reviewed. Start, the tick and each wake refuse an unknown level. The setting is read when each session starts and is not stored in run records.
+- Run endings append one `attest` trajectory event: an HMAC-SHA256 over the run id and chain head with a deployment key created owner-only beside the operator settings file, and `outerloop trajectory --verify` reports `attested` or `attestation mismatch`. Older kernels ignore the event; no ledger or PR-body change.
+- The PR body gains a report-only Number check that lists values in the author's report matching no `measure` event or Experiments result when rounded to the report's decimals; nothing is blocked and no trajectory event is added.
+- Session end records each tool call from the retained Claude or Codex log as a `tool.call` trajectory event, once per call even when a session resumes. `outerloop trajectory` and a new Process section in the PR body show counts by tool. Older kernels ignore these events.
+- `outerloop status` shows a run's priced session cost as a lower bound when some legs are unpriced (a wake killed mid-turn leaves no usage event), for example `session-cost>=$4.2000 (1 unpriced)`, instead of hiding the priced part behind `unknown`; the status row and JSON gain the priced-leg count (#516).
+- The pre-PR panel accepts a composite change — a PR that carries several changes — when the report gives each component's isolated measurement and the combination's under the same protocol and the combination is what the kernel measured. A bundle without those isolated measurements stays blocked (#489).
+- Claude sessions can use a subscription token from `claude setup-token` (`sk-ant-oat...`). The harness sends it as `CLAUDE_CODE_OAUTH_TOKEN`, not `ANTHROPIC_API_KEY`, and `init` accepts it. A panel judge on a token runs `claude --safe-mode`, because `--bare` cannot read OAuth tokens. Both modes ignore the checkout's CLAUDE.md, hooks, plugins and MCP servers. The pinned Claude Code release (2.1.285) has `--safe-mode`.
+- A session now hears once when a live sibling states a new or changed hypothesis (#360). The note names the sibling, quotes its hypothesis, and points to `siblings` for its launches and score. It is sent only when `channels.siblings` is on. Upgrading: the note uses the existing `note` kind, so an older kernel shows it as ordinary inbox text.
+- A skills-only plugin for Claude Code and Codex sets up Outerloop, writes and validates the contract, and operates runs from the agent (#562). Install it with `/plugin marketplace add outerloop-science/outerloop` or `codex plugin marketplace add outerloop-science/outerloop`. Its setup skill offers a Claude subscription token from `claude setup-token` as the author credential.
+- `outerloop trajectory <run-id>` now ends with the run's refusal count and its author models, for release canary comparisons; the agent-text inventory includes operator notes (#487, part 3). Session events record `role` (`author` or `judge`). Upgrading: session events written before this kernel have no `role` and count as author sessions.
+- On macOS without an image, sessions, evaluations and panel lenses run under a packaged Seatbelt profile (`sandbox-exec`, #292): they write only the workspace and a scratch directory, read only system and toolchain paths, and connect out only to TCP port 443, DNS and local ports. Linux and Apptainer runs are unchanged.
+- Kernel events now append to a per-run `trajectory.jsonl` with sequence numbers and a SHA-256 chain. `outerloop trajectory <run-id> [--verify] [--json]` reads it without changing run state. Operational files and their formats are unchanged; older kernels ignore the new file. Upgrading: runs started before this kernel have no trajectory and read as "no trajectory" until their next recorded event. No backfill is attempted.
+- `measure` trajectory events carry provenance: the ledger's ruler digest (the same value a pending submission computes), the container image digest when an image is configured, and refs to the evaluation result files the kernel read. Self-reported measures record the ruler alone. Pending ledger records now ignore fields this kernel does not know, so a later release can add one without breaking this release's ledger read. Upgrading: no action needed; older kernels ignore the trajectory, and pending records are unchanged.
+- Add a printable agent-text inventory and protocol wording lint, with clearer refusal recovery guidance (#487, parts 1–2).
+- `BENCHMARKS.md` shows `—` instead of "provenance unknown" in the main-commit column of a baseline row that nothing has improved on yet, as on a freshly seeded repository.
+- The README and the getting-started guide link to the quickstart repository. It runs two CPU benchmarks with the local forge and local compute, and needs no GitHub account or cluster (#563). The docs and the startup warning no longer say an uncontained session runs in a sandbox: on Linux without an image, it gets a scrubbed environment and its own home, but can read any file the account can.
+- The advisory review covers exactly the head its triggering event named (`REVIEW_HEAD_SHA`, #462). If the PR has moved off that head, nothing is reviewed or posted. Otherwise the diff is read between fixed commits, every envelope records the reviewed `head_sha`, and the review is attached to that commit. When GitHub refuses its inline comments, the fallback is a body-only review on the same commit, never an unpinned comment (#569). The summarizer skips clean when its lenses read different heads. For a fork PR (`REVIEW_CHECKOUT_BASE=1`) the brief says the tree is the base branch and the change is only in the diff. Upgrading: envelopes without `head_sha` (from an older reviewer ref) still post; callers that do not pass REVIEW_HEAD_SHA behave as before.
+- When check runs cannot be read on a token deployment, the warning now says the token cannot read them (fine-grained tokens have no checks permission) instead of asking for an App permission.
+
+## [0.5.0] - 2026-10-23
+
+### Upgrading
+
+No operator action is needed before upgrading; details in each entry.
+
+What changes for runs in flight: kernel comments, issues and pull requests
+start carrying `<!-- outerloop:kernel -->`; the first tick republishes
+`climb/status.json` in the new shape; a resumed Claude session captured
+before the upgrade still overstates its cost.
+
+Before rolling back: remove `slots` keys from `limits.toml`, and let local
+jobs on GPU UUID ids (`GPU-...`) finish.
+
+### Changes
+
+- Docs for personal-token mode: the install guide's personal-mode section (setup, how kernel posts are told apart, what GitHub no longer enforces, token reach, switching modes), the safety defaults in `operate.md`, `OUTERLOOP_IDENTITY` in the environment reference, `init --personal` in the CLI reference, and `personal: true` in the reviewer setup.
+- Personal-token mode is selectable: `outerloop init --personal` takes the token `gh auth token` prints and writes it 0600 to the default PAT file (or uses a token file given with `--pat-file`), records its login as `OUTERLOOP_BOT_LOGIN`, and writes `OUTERLOOP_IDENTITY=personal`; `start` and the deploy step carry the setting into every tick. The kernel then posts as the person, so its own posts are told apart by the `outerloop:` marker rather than by login. The setup prints the trade-offs: no notifications for the kernel's activity, the `gh` token's repository-wide reach, and the reviewer workflow's `personal: true`. Upgrading: switching a personal deployment back to the App means releasing or closing claimed issues first — their claim comments were posted as the person, and the App's login check no longer counts them.
+- One rule, `forge.is_kernel_post`, now decides whether a comment, issue or pull request is the kernel's, wherever the kernel asks. In personal mode (`OUTERLOOP_IDENTITY=personal`) the login is the person's, so a post is the kernel's only when its body also carries an `outerloop:` marker. Alias logins keep the login check alone. Kernel comments, issues and pull requests now carry `<!-- outerloop:kernel -->` unless they already carry a marker. When the kernel arms auto-merge it records the PR in the run's stage (`auto_merge_armed`) and clears the record once it disarms or sees auto-merge off; under the auto dial it withdraws only an arm that record claims, so a human's later arm is left alone. Upgrading: kernel posts now carry a marker, which an older kernel ignores. A stage without an `auto_merge_armed` record (every run armed before the upgrade) keeps the old `enabled_by.login` rule outside personal mode, so a PR the kernel armed before the upgrade is still withdrawn under the auto dial; rolling back ignores the field.
+- The `siblings` view now shows each sibling's research instead of its last chat line: the run's stated hypothesis, its three most recent launches with `--why`, state and result line, the candidate score from the kernel's own gate measurement (never the agent's claim), and its PR, plus the target's five most recent endings with outcome and report title. `climb/status.json` gains `best_result` and `launches` per live run and a top-level `ended` list. Upgrading: no action needed; a snapshot written before this change reads as before (no launches, score or endings) and the first tick republishes it once in the new shape; rolling back is safe because an older reader ignores the added fields.
+- Session records and the session index now carry the Claude Code version that ran, read from the binary's resolved path (the native installer's `versions/<x.y.z>` and the kernel's managed `harnesses/claude/<x.y.z>/` installs; the pin is not used, since an operator can point the kernel at any binary), so a resumed Claude session prices its cumulative resume totals once at the newest leg when it ran on Claude Code 2.1.277 or later. Codex is unchanged, and a Claude entry without a usable version keeps summing its legs (#515). Upgrading: no action needed; index entries written before this change carry no version, so a resumed Claude session captured before the upgrade still overstates its cost until those entries age out.
+- `outerloop init` warns when the Claude or Codex CLI it finds is older than the pinned version and names `outerloop harness upgrade`, and says when the pre-PR panel stays off for lack of an image. The install guide covers seeding a private repository into the local forge.
+- `outerloop pr merge` on the local forge authors the merge commit with the operator's git identity from system and global config (a repo-local identity in the directory it runs from is ignored); automatic merges keep the kernel's. Candidate snapshot commits are named `candidate snapshot: <run id>` instead of `dispatch snapshot`.
+- Add an internal `Forge` seam behind the kernel's code-host operations; `OUTERLOOP_FORGE` accepts only `github` for now (no behavior change).
+- Add the local forge backend's transport, refs and files over bare repositories (clone, branch and file reads and compare-and-swap writes); `OUTERLOOP_FORGE=local` is not selectable until pull requests, threads and issues exist.
+- Add the local forge backend's records on `refs/outerloop/records`: pull requests, comment threads and the kernel's issues, with merge-commit transactions and recovery of merges made outside the kernel; user-visible inbox and self-merge text takes the forge's name.
+- The local forge is selectable: `OUTERLOOP_FORGE=local` builds it from the state root, entry points need no GitHub credential or bot login, `outerloop init --forge local` bootstraps the bare repository, and `outerloop pr` lists, shows, comments on, merges and closes its pull requests.
+- Local compute's GPU pool follows the loop's own `CUDA_VISIBLE_DEVICES`: when it is set at startup, those ids are the pool (jobs still get a subset), so two deployments on one host can be given disjoint GPUs. `OUTERLOOP_LOCAL_GPUS` caps the list, keeping the first ids; empty or `-1` means no GPUs. Jobs never see GPUs outside the loop's view: a job without an allocation inherits the loop's `CUDA_VISIBLE_DEVICES`. Fix: on local compute, the evaluated command now sees its job's GPU mask; the eval script's scrubbed environment (`env -i`, or apptainer `--cleanenv`) used to drop it, so evaluated code saw every GPU on the host. Slurm jobs are unchanged.
+- Upgrading: an existing pool file (`gpus.json`) reads correctly, since holder keys stay the GPU index strings. Before rolling back to an older kernel, let jobs on non-integer ids (`GPU-...` UUIDs) finish: the older kernel cannot sort those holder keys.
+- `limits.toml` accepts a per-target `slots` list (`[targets."owner/repo"]`, `slots = ["agent-04"]`): self-initiated runs take the first free listed agent identity, in order, instead of the lowest free `agent-01..agent-<width>`, so an operator can keep a slot's research line and author binding. Slots never add capacity (the attempt width still caps concurrency), running runs in unlisted slots are untouched, and `outerloop limits` shows the list. Without `slots`, behaviour is unchanged.
+- Upgrading: existing `limits.toml` files need no change. Remove any `slots` key before rolling back to an older kernel, which treats it as unknown and fails the file closed.
+- Refresh the toolchain within the existing bounds: pydantic 2.14.1, pydantic-core 2.50.1, ruff 0.17.1 and pre-commit 4.7.0; the ruff-pre-commit hook matches the locked Ruff. No source formatting or lint changes were needed.
+
+## [0.4.0] - 2026-10-20
+
 ### Upgrading
 
 Operator actions before upgrading (everything else needs no action; details
@@ -107,6 +182,38 @@ runs, and do not roll back while self-reported claims exist.
   without a compatibility change. Upgrading: no action needed; it appends an
   ordinary note from the kernel; a parked run that is not waiting on jobs wakes
   at the next tick to read it, and one waiting on jobs reads it when they end.
+- `OUTERLOOP_UNCONTAINED=1` runs a cluster deployment without a container
+  runtime: sessions and evaluations run as Slurm jobs the way local mode runs
+  them without an image, the image is ignored, and the panel stays opt-in
+  (`OUTERLOOP_PANEL_UNCONTAINED=1`, now also carried by the chain's deploy
+  step). Without it, a cluster deployment with no image still services nothing,
+  and its log now names the setting. `outerloop start` now checks the fleet
+  default author with the tick's own author check, as it already did for
+  overrides, reading settings with the precedence the launched tick applies,
+  so a default the tick would refuse (a Codex or Hermes author without an
+  image, a Codex author without a Codex model) fails at start instead of
+  idling every tick.
+- Multi-node benchmarks: a benchmark's `nodes` (default 1, up to 64) spreads
+  every dispatched job over that many nodes, with `gpus` per node. The eval
+  job `srun`s one task per node; each runs the command in its own checkout
+  with `NNODES`, `NODE_RANK`, `MASTER_ADDR`, `MASTER_PORT` and torchrun's
+  `PET_*` defaults set, node 0's output is the measurement, and a failing node
+  stops the rest. A requeued job clears the earlier attempt's per-node results
+  before it starts, and those rendezvous names (and any `PET_*`) are refused as
+  a `seed_env`. GPU-hour charges, budget checks, sweep pacing and the
+  operator's GPU ceiling count `nodes x gpus`. Uncontained Slurm deployments
+  only: a contained or local deployment refuses to launch attempts on such a
+  contract. Upgrading: one-node benchmarks keep their measurement signatures,
+  eval slots and cached baselines unchanged; cached baselines now record
+  their node count, and entries without one read as one node. A contract
+  that sets `nodes` fails to load on older kernels (unknown keys are
+  refused), so upgrade every fleet that services a target before adding it.
+- `OUTERLOOP_OPERATOR_HARNESSES` lists harness CLIs the operator supplies, such
+  as a site-managed `claude` that authenticates by itself: `outerloop harness
+  upgrade` (and the deployment's automatic `--used` run) never replaces them or
+  rewrites their recorded paths, and `outerloop harness status` reports
+  `OPERATOR` instead of drift. Unknown names are refused. The chain's deploy
+  step re-reads it from `.env` every tick.
 
 ### Documentation
 
@@ -221,6 +328,9 @@ runs, and do not roll back while self-reported claims exist.
   Upgrading: run `outerloop harness upgrade codex`; deploys that run harness
   upgrades do this automatically. Existing single-binary installs are detected
   as drift and replaced with a verified pair.
+- Uncontained sweep tasks now receive `SWEEP_INDEX`: the bare evaluation
+  branch runs the command under `env -i` and dropped it, so every task of a
+  sweep without an image ran the default configuration.
 
 ### Changed
 

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/CITATION.cff

@@ -13,5 +13,5 @@ authors:
 repository-code: "https://github.com/outerloop-science/outerloop"
 url: "https://outerloop.science"
 license: Apache-2.0
-version: 0.4.0rc1
-date-released: "2026-10-19"
+version: 0.6.0
+date-released: "2026-10-26"

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/CONTRIBUTING.md

@@ -61,6 +61,19 @@ uv run pytest
   `uv lock`; CI checks the lock.
 - Docs are plain prose. Short sentences, no metaphors, no internal jargon.
 
+New agent-facing text must be reachable from `outerloop.agent_text.inventory()`.
+Keep text with its renderer and add a representative rendering to the inventory;
+review it with `python -m outerloop.agent_text`. The fast lint in
+`tests/test_agent_text.py` checks protocol terms, syscall names, and refusal recovery
+guidance. Keep refusals inline: the inventory reads explicit AST constructs listed in
+`REFUSAL_CONSTRUCTS`, rendering f-string expressions as `{}`. Input constraints
+use category `validation` and need no redundant next step. The syscall CLI mixes
+constraints and state refusals in `ToolError`; `CLI_REFUSALS` identifies the latter
+by function and call ordinal, so update it when adding or reordering those calls. Delivery reasons composed into a
+refusal use category `reason`; the enclosing refusal carries recovery guidance.
+Kernel refusals require a next step; any exception needs a specific reason beside
+the lint rule.
+
 ## Tests
 
 | Tier | Marker | Runs |

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: outerloop-science
-Version: 0.4.0rc1
+Version: 0.6.0
 Summary: Autonomous research agents that improve the benchmark you point them at, one verified pull request at a time
 Project-URL: Homepage, https://outerloop.science
 Project-URL: Repository, https://github.com/outerloop-science/outerloop
@@ -41,6 +41,11 @@ change, runs the experiment on your cluster, and opens a pull request only
 when your benchmark actually improves. Every attempt ends in a short
 written report, including the attempts that failed.
 
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/outerloop-science/outerloop/main/docs/assets/loop-dark.svg">
+  <img alt="One attempt: an agent proposes a change, runs the experiment on your GPUs, the kernel re-measures it against the base, judges review the claim, and a pull request opens with a written report. Without a real gain the attempt ends in a report and the next hypothesis starts." src="https://raw.githubusercontent.com/outerloop-science/outerloop/main/docs/assets/loop.svg" width="880">
+</picture>
+
 You run it yourself, with your own keys, your own compute and your own
 repositories, and nothing reports back to us. It is built and used every
 day by the [Agentic Learning AI Lab](https://agenticlearning.ai) at NYU,
@@ -48,7 +53,24 @@ where it co-develops our research codebases.
 
 ## Get started
 
-Three commands and one file. You need a repo with a benchmark command, an
+To see a run first, the [quickstart](https://github.com/outerloop-science/quickstart)
+runs two CPU benchmarks on your own machine, with no GitHub account and no
+cluster. You need Python 3.12 or later, [uv](https://docs.astral.sh/uv/), Git
+2.38 or later and an Anthropic API key:
+
+```bash
+pip install outerloop-science
+outerloop init --forge local --compute local --no-image --target me/quickstart \
+  --forge-source https://github.com/outerloop-science/quickstart
+outerloop start
+```
+
+On macOS, sessions and evaluations run under a Seatbelt profile: they write their
+workspace and a scratch directory, read system and toolchain paths, and cannot read
+the rest of your home. On Linux without a container they can read any file your
+account can. Either way, use a spend-capped API key.
+
+For your own repository: three commands and one file. You need a repo with a benchmark command, an
 API key for the model that will write the code, and a Slurm cluster or one
 machine with a GPU.
 
@@ -60,8 +82,10 @@ outerloop init     # where the loop runs, which repo, which model and its key, y
 The wizard asks for a GitHub identity for the agents to open pull requests
 as. Pick `app` and it walks you through creating a GitHub App under your
 account and installing it on the repo. Pick `pat` if you already have a
-token, and it writes the config and the key files either way. Then add one
-file, `.outerloop.yaml`, to the repo you want improved:
+token, and it writes the config and the key files either way. To try it
+without GitHub, `outerloop init --forge local` keeps the repository and its
+pull requests in a git repository on the same machine. Then add one file,
+`.outerloop.yaml`, to the repo you want improved:
 
 ```yaml
 benchmarks:
@@ -128,8 +152,8 @@ build. See [the reviewer guide](https://docs.outerloop.science/reviewer/).
   output are rendered to the agent as fenced data, not instructions, which
   lowers the risk of injected text being followed without guaranteeing it.
   A contained session gets no GitHub credential in its environment and
-  cannot read the operator's key files; an uncontained local session runs
-  as your user and can read any file you can, key files included.
+  cannot read the operator's key files. Without a container, a macOS session
+  runs under Seatbelt; a Linux session can read any file you can, key files included.
 - **Budgets in code.** Launches, GPU-hours, and runs per week are enforced
   by the kernel, not left to the agent.
 - **No model lock-in.** Claude Code, Codex, and hermes-agent are wired

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/README.md

@@ -17,6 +17,11 @@ change, runs the experiment on your cluster, and opens a pull request only
 when your benchmark actually improves. Every attempt ends in a short
 written report, including the attempts that failed.
 
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/outerloop-science/outerloop/main/docs/assets/loop-dark.svg">
+  <img alt="One attempt: an agent proposes a change, runs the experiment on your GPUs, the kernel re-measures it against the base, judges review the claim, and a pull request opens with a written report. Without a real gain the attempt ends in a report and the next hypothesis starts." src="https://raw.githubusercontent.com/outerloop-science/outerloop/main/docs/assets/loop.svg" width="880">
+</picture>
+
 You run it yourself, with your own keys, your own compute and your own
 repositories, and nothing reports back to us. It is built and used every
 day by the [Agentic Learning AI Lab](https://agenticlearning.ai) at NYU,
@@ -24,7 +29,24 @@ where it co-develops our research codebases.
 
 ## Get started
 
-Three commands and one file. You need a repo with a benchmark command, an
+To see a run first, the [quickstart](https://github.com/outerloop-science/quickstart)
+runs two CPU benchmarks on your own machine, with no GitHub account and no
+cluster. You need Python 3.12 or later, [uv](https://docs.astral.sh/uv/), Git
+2.38 or later and an Anthropic API key:
+
+```bash
+pip install outerloop-science
+outerloop init --forge local --compute local --no-image --target me/quickstart \
+  --forge-source https://github.com/outerloop-science/quickstart
+outerloop start
+```
+
+On macOS, sessions and evaluations run under a Seatbelt profile: they write their
+workspace and a scratch directory, read system and toolchain paths, and cannot read
+the rest of your home. On Linux without a container they can read any file your
+account can. Either way, use a spend-capped API key.
+
+For your own repository: three commands and one file. You need a repo with a benchmark command, an
 API key for the model that will write the code, and a Slurm cluster or one
 machine with a GPU.
 
@@ -36,8 +58,10 @@ outerloop init     # where the loop runs, which repo, which model and its key, y
 The wizard asks for a GitHub identity for the agents to open pull requests
 as. Pick `app` and it walks you through creating a GitHub App under your
 account and installing it on the repo. Pick `pat` if you already have a
-token, and it writes the config and the key files either way. Then add one
-file, `.outerloop.yaml`, to the repo you want improved:
+token, and it writes the config and the key files either way. To try it
+without GitHub, `outerloop init --forge local` keeps the repository and its
+pull requests in a git repository on the same machine. Then add one file,
+`.outerloop.yaml`, to the repo you want improved:
 
 ```yaml
 benchmarks:
@@ -104,8 +128,8 @@ build. See [the reviewer guide](https://docs.outerloop.science/reviewer/).
   output are rendered to the agent as fenced data, not instructions, which
   lowers the risk of injected text being followed without guaranteeing it.
   A contained session gets no GitHub credential in its environment and
-  cannot read the operator's key files; an uncontained local session runs
-  as your user and can read any file you can, key files included.
+  cannot read the operator's key files. Without a container, a macOS session
+  runs under Seatbelt; a Linux session can read any file you can, key files included.
 - **Budgets in code.** Launches, GPU-hours, and runs per week are enforced
   by the kernel, not left to the agent.
 - **No model lock-in.** Claude Code, Codex, and hermes-agent are wired

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/RELEASING.md

@@ -43,21 +43,24 @@ wording does not reach a parked run unless its deduplication key changes.
    run with an open PR, a run whose PR merged, and an ended run. A dev or rc pre-release still bumps the version (PyPI never
    accepts a version twice, so the next one is `.dev1`, `rc2`, ...) but leaves
    `[Unreleased]` in place until the final release.
-3. `git tag vX.Y.Z && git push origin vX.Y.Z`. The `release` workflow builds
+3. Pre-release canary runs report `refusals` per run by author model from
+   `outerloop trajectory <run-id>`. If refusals per run rise against the previous
+   release, explain why in the release PR before shipping.
+4. `git tag vX.Y.Z && git push origin vX.Y.Z`. The `release` workflow builds
    and publishes `outerloop-science` to PyPI through Trusted Publishing; the
    one-time PyPI setup is described at the top of
    `.github/workflows/release.yml`.
-4. Once the `release` workflow run for the tag is green (`gh run watch` on
+5. Once the `release` workflow run for the tag is green (`gh run watch` on
    it), `pip install outerloop-science==X.Y.Z` in a fresh venv, then
    `outerloop --help`. This comes before the GitHub release: publishing it is
    what Discord announces, so nothing is announced that did not install.
-5. `gh release create vX.Y.Z --generate-notes`, with `--prerelease` for a dev
+6. `gh release create vX.Y.Z --generate-notes`, with `--prerelease` for a dev
    or rc tag.
    For a final release, also move the `v0` tag that external callers of the
    reusable workflows track: `git tag -f v0 vX.Y.Z && git push -f origin v0`.
    Dev and rc tags never move it. This tag alias is the one force-push the
    repository allows; branches are never force-pushed.
-6. Announce. Discord's `#announcements` gets the release from the GitHub
+7. Announce. Discord's `#announcements` gets the release from the GitHub
    webhook on its own (docs/community.md). For a final release, also write
    the post for X (`@outerloop_sci`) and Bluesky (`@outerloop.science`): one
    or two sentences on what changed for the reader, the install line, the

outerloop_science-0.6.0/docs/assets/loop-dark.svg

@@ -0,0 +1,86 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 880 210" width="880" height="210" role="img" aria-label="One Outerloop attempt: an agent proposes a change, runs the experiment on your GPUs, the kernel re-measures it against the base, judges review the claim, and a pull request opens with a written report. Without a real gain the attempt ends in a report and the next hypothesis.">
+<style>
+  svg { --fg: #e4e7eb; --muted: #9aa5b1; --line: #3e4c59; --card: #161b22; --accent: #6ea2ff; --good: #4cc38a; }
+  text { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif; fill: var(--fg); }
+  .title { font-size: 15px; font-weight: 600; }
+  .sub { font-size: 12px; fill: var(--muted); }
+  .node { fill: var(--card); stroke: var(--line); stroke-width: 1.5; }
+  .edge { stroke: var(--line); stroke-width: 1.5; fill: none; }
+  .back { stroke: var(--line); stroke-width: 1.5; fill: none; stroke-dasharray: 5 5; }
+  .head { fill: var(--line); }
+  .dot { fill: var(--accent); }
+  .score { font-size: 12px; font-variant-numeric: tabular-nums; fill: var(--fg); }
+  .ok { fill: var(--good); font-weight: 600; }
+  .pr { fill: var(--accent); font-weight: 600; font-size: 12px; }
+
+  .n1 { animation: lit1 12s infinite; } .n2 { animation: lit2 12s infinite; }
+  .n3 { animation: lit3 12s infinite; } .n4 { animation: lit4 12s infinite; }
+  .n5 { animation: lit5 12s infinite; }
+
+  @keyframes lit1 { 0%, 11% { stroke: var(--accent); stroke-width: 2.5; } 13%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit2 { 0%, 15% { stroke: var(--line); stroke-width: 1.5; } 16%, 31% { stroke: var(--accent); stroke-width: 2.5; } 33%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit3 { 0%, 35% { stroke: var(--line); stroke-width: 1.5; } 36%, 53% { stroke: var(--accent); stroke-width: 2.5; } 55%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit4 { 0%, 57% { stroke: var(--line); stroke-width: 1.5; } 58%, 71% { stroke: var(--accent); stroke-width: 2.5; } 73%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit5 { 0%, 75% { stroke: var(--line); stroke-width: 1.5; } 76%, 96% { stroke: var(--accent); stroke-width: 2.5; } 98%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+
+  @media (prefers-reduced-motion: reduce) {
+    .n1, .n2, .n3, .n4, .n5 { animation: none; }
+  }
+</style>
+
+<!-- edges between the five steps -->
+<g>
+  <line class="edge" x1="172" y1="68" x2="186" y2="68"/><polygon class="head" points="192,68 184,64 184,72"/>
+  <line class="edge" x1="344" y1="68" x2="358" y2="68"/><polygon class="head" points="364,68 356,64 356,72"/>
+  <line class="edge" x1="516" y1="68" x2="530" y2="68"/><polygon class="head" points="536,68 528,64 528,72"/>
+  <line class="edge" x1="688" y1="68" x2="702" y2="68"/><polygon class="head" points="708,68 700,64 700,72"/>
+</g>
+
+<!-- the steps -->
+<g>
+  <rect class="node n1" x="20" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="96" y="63" text-anchor="middle">Propose</text>
+  <text class="sub" x="96" y="83" text-anchor="middle">agent edits the solver</text>
+
+  <rect class="node n2" x="192" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="268" y="63" text-anchor="middle">Experiment</text>
+  <text class="sub" x="268" y="83" text-anchor="middle">jobs on your GPUs</text>
+
+  <rect class="node n3" x="364" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="440" y="63" text-anchor="middle">Measure</text>
+  <text class="sub" x="440" y="83" text-anchor="middle">Outerloop re-measures</text>
+
+  <rect class="node n4" x="536" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="612" y="63" text-anchor="middle">Review</text>
+  <text class="sub" x="612" y="83" text-anchor="middle">judges read the claim</text>
+
+  <rect class="node n5" x="708" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="784" y="63" text-anchor="middle">Pull request</text>
+  <text class="sub" x="784" y="83" text-anchor="middle">with a written report</text>
+</g>
+
+<!-- the attempt moving through -->
+<circle class="dot" cx="96" cy="22" r="6">
+  <animateTransform attributeName="transform" type="translate" dur="12s" repeatCount="indefinite"
+    values="0 0; 0 0; 172 0; 172 0; 344 0; 344 0; 516 0; 516 0; 688 0; 688 0"
+    keyTimes="0; 0.10; 0.16; 0.30; 0.36; 0.52; 0.58; 0.70; 0.76; 1"/>
+</circle>
+
+<!-- what the measure step reports -->
+<g class="readout" opacity="0">
+  <animate attributeName="opacity" dur="12s" repeatCount="indefinite"
+    values="0; 0; 1; 1; 0" keyTimes="0; 0.39; 0.42; 0.94; 1"/>
+  <text class="score" x="440" y="124" text-anchor="middle">base 9,472 steps → candidate 7,808</text>
+  <text class="score ok" x="440" y="142" text-anchor="middle">more than run-to-run noise</text>
+</g>
+<g class="badge" opacity="0">
+  <animate attributeName="opacity" dur="12s" repeatCount="indefinite"
+    values="0; 0; 1; 1; 0" keyTimes="0; 0.78; 0.81; 0.94; 1"/>
+  <text class="pr" x="784" y="124" text-anchor="middle">opened for a human to merge</text>
+</g>
+
+<!-- no gain: report and try again -->
+<path class="back" d="M 440 154 L 440 176 L 96 176 L 96 108"/>
+<polygon class="head" points="96,102 92,110 100,110"/>
+<text class="sub" x="268" y="196" text-anchor="middle">no real gain: the report says why, and the next hypothesis starts</text>
+</svg>

outerloop_science-0.6.0/docs/assets/loop.svg

@@ -0,0 +1,86 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 880 210" width="880" height="210" role="img" aria-label="One Outerloop attempt: an agent proposes a change, runs the experiment on your GPUs, the kernel re-measures it against the base, judges review the claim, and a pull request opens with a written report. Without a real gain the attempt ends in a report and the next hypothesis.">
+<style>
+  svg { --fg: #1f2933; --muted: #616e7c; --line: #c4cdd5; --card: #ffffff; --accent: #3c81f7; --good: #1f8a4c; }
+  text { font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif; fill: var(--fg); }
+  .title { font-size: 15px; font-weight: 600; }
+  .sub { font-size: 12px; fill: var(--muted); }
+  .node { fill: var(--card); stroke: var(--line); stroke-width: 1.5; }
+  .edge { stroke: var(--line); stroke-width: 1.5; fill: none; }
+  .back { stroke: var(--line); stroke-width: 1.5; fill: none; stroke-dasharray: 5 5; }
+  .head { fill: var(--line); }
+  .dot { fill: var(--accent); }
+  .score { font-size: 12px; font-variant-numeric: tabular-nums; fill: var(--fg); }
+  .ok { fill: var(--good); font-weight: 600; }
+  .pr { fill: var(--accent); font-weight: 600; font-size: 12px; }
+
+  .n1 { animation: lit1 12s infinite; } .n2 { animation: lit2 12s infinite; }
+  .n3 { animation: lit3 12s infinite; } .n4 { animation: lit4 12s infinite; }
+  .n5 { animation: lit5 12s infinite; }
+
+  @keyframes lit1 { 0%, 11% { stroke: var(--accent); stroke-width: 2.5; } 13%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit2 { 0%, 15% { stroke: var(--line); stroke-width: 1.5; } 16%, 31% { stroke: var(--accent); stroke-width: 2.5; } 33%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit3 { 0%, 35% { stroke: var(--line); stroke-width: 1.5; } 36%, 53% { stroke: var(--accent); stroke-width: 2.5; } 55%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit4 { 0%, 57% { stroke: var(--line); stroke-width: 1.5; } 58%, 71% { stroke: var(--accent); stroke-width: 2.5; } 73%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+  @keyframes lit5 { 0%, 75% { stroke: var(--line); stroke-width: 1.5; } 76%, 96% { stroke: var(--accent); stroke-width: 2.5; } 98%, 100% { stroke: var(--line); stroke-width: 1.5; } }
+
+  @media (prefers-reduced-motion: reduce) {
+    .n1, .n2, .n3, .n4, .n5 { animation: none; }
+  }
+</style>
+
+<!-- edges between the five steps -->
+<g>
+  <line class="edge" x1="172" y1="68" x2="186" y2="68"/><polygon class="head" points="192,68 184,64 184,72"/>
+  <line class="edge" x1="344" y1="68" x2="358" y2="68"/><polygon class="head" points="364,68 356,64 356,72"/>
+  <line class="edge" x1="516" y1="68" x2="530" y2="68"/><polygon class="head" points="536,68 528,64 528,72"/>
+  <line class="edge" x1="688" y1="68" x2="702" y2="68"/><polygon class="head" points="708,68 700,64 700,72"/>
+</g>
+
+<!-- the steps -->
+<g>
+  <rect class="node n1" x="20" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="96" y="63" text-anchor="middle">Propose</text>
+  <text class="sub" x="96" y="83" text-anchor="middle">agent edits the solver</text>
+
+  <rect class="node n2" x="192" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="268" y="63" text-anchor="middle">Experiment</text>
+  <text class="sub" x="268" y="83" text-anchor="middle">jobs on your GPUs</text>
+
+  <rect class="node n3" x="364" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="440" y="63" text-anchor="middle">Measure</text>
+  <text class="sub" x="440" y="83" text-anchor="middle">Outerloop re-measures</text>
+
+  <rect class="node n4" x="536" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="612" y="63" text-anchor="middle">Review</text>
+  <text class="sub" x="612" y="83" text-anchor="middle">judges read the claim</text>
+
+  <rect class="node n5" x="708" y="36" width="152" height="64" rx="10"/>
+  <text class="title" x="784" y="63" text-anchor="middle">Pull request</text>
+  <text class="sub" x="784" y="83" text-anchor="middle">with a written report</text>
+</g>
+
+<!-- the attempt moving through -->
+<circle class="dot" cx="96" cy="22" r="6">
+  <animateTransform attributeName="transform" type="translate" dur="12s" repeatCount="indefinite"
+    values="0 0; 0 0; 172 0; 172 0; 344 0; 344 0; 516 0; 516 0; 688 0; 688 0"
+    keyTimes="0; 0.10; 0.16; 0.30; 0.36; 0.52; 0.58; 0.70; 0.76; 1"/>
+</circle>
+
+<!-- what the measure step reports -->
+<g class="readout" opacity="0">
+  <animate attributeName="opacity" dur="12s" repeatCount="indefinite"
+    values="0; 0; 1; 1; 0" keyTimes="0; 0.39; 0.42; 0.94; 1"/>
+  <text class="score" x="440" y="124" text-anchor="middle">base 9,472 steps → candidate 7,808</text>
+  <text class="score ok" x="440" y="142" text-anchor="middle">more than run-to-run noise</text>
+</g>
+<g class="badge" opacity="0">
+  <animate attributeName="opacity" dur="12s" repeatCount="indefinite"
+    values="0; 0; 1; 1; 0" keyTimes="0; 0.78; 0.81; 0.94; 1"/>
+  <text class="pr" x="784" y="124" text-anchor="middle">opened for a human to merge</text>
+</g>
+
+<!-- no gain: report and try again -->
+<path class="back" d="M 440 154 L 440 176 L 96 176 L 96 108"/>
+<polygon class="head" points="96,102 92,110 100,110"/>
+<text class="sub" x="268" y="196" text-anchor="middle">no real gain: the report says why, and the next hypothesis starts</text>
+</svg>

{outerloop_science-0.4.0rc1 → outerloop_science-0.6.0}/docs/design/accelerators.md

@@ -17,23 +17,27 @@ placed and capped before any TPU contract is accepted.
 
 ## What assumes a GPU today
 
-- **Declaration.** `Benchmark.gpus: int (0..8)`; `Measure.gpus`,
-  `JobSpec.gpus`; the GPU type lives on the operator's per-target lane
+- **Declaration.** `Benchmark.gpus: int (0..8)` per node and
+  `Benchmark.nodes: int (1..64)`; `Measure.gpus`/`nodes`, `JobSpec.gpus`/`nodes`;
+  `Benchmark.total_gpus` is what one job holds. The GPU type lives on the operator's per-target lane
   (`OUTERLOOP_GPU_LANES`, `gpu_lanes.py`). Whether evals dispatch depends on
   `eval_minutes`, not on `gpus`.
-- **Metering.** Launches charge minutes × array × gpus / 60; gates charge the
-  main eval (one when a cached baseline applies, else two) and every suite
-  sibling pair × walltime × gpus (`syscall.py`). The run meter is
+- **Metering.** Launches charge minutes × array × total GPUs / 60; gates charge
+  the main eval (one when a cached baseline applies, else two) and every suite
+  sibling pair × walltime × total GPUs (`syscall.py`). The run meter is
   `stage.gpu_hours_used`; budgets are `gpu_hours_per_run` plus
   `review_topup.gpu_hours`; `max_concurrent_gpus` paces sweeps.
 - **Placement.** `JobSpec.to_argv` emits `--gres=gpu:TYPE:N` or
-  `--gpus-per-node=N`, single node; dispatched evals add `--nv` and size CPUs
-  and memory per GPU.
+  `--gpus-per-node=N`, plus `--nodes=M --ntasks-per-node=1` for a multi-node
+  benchmark; dispatched evals add `--nv` and size CPUs and memory per GPU, per
+  node. A multi-node eval `srun`s its per-node script once per node
+  (uncontained deployments only).
 - **Operator limits.** `limits.toml` `max_gpus`, counted from one
   `squeue --me --json` snapshot by parsing `gres/gpu` TRES, attributing jobs to
   this fleet by run id.
-- **Local backend.** `nvidia-smi` detection (or `OUTERLOOP_LOCAL_GPUS`),
-  `CUDA_VISIBLE_DEVICES` pinning.
+- **Local backend.** `nvidia-smi` detection (or `OUTERLOOP_LOCAL_GPUS`), the
+  loop's own `CUDA_VISIBLE_DEVICES` when it is set, `CUDA_VISIBLE_DEVICES`
+  pinning.
 - **Surfaces.** Briefs, wake lines, refusals, `budget.json`
   (`gpu_hours_remaining`), `outerloop status`, `outerloop limits` and the board
   carry GPU-named fields.