ctrlrelay 0.23.0__tar.gz → 0.24.0__tar.gz

{ctrlrelay-0.23.0 → ctrlrelay-0.24.0}/CHANGELOG.md

@@ -7,6 +7,72 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.24.0] - 2026-11-05
+
+### Added
+
+- Each poll's new issues run in priority order, oldest first (#356). The
+  batch is sorted by the position of the issue's priority label in
+  `intake.priority_labels` (default `["P0", "P1", "P2", "P3"]`, compared
+  without case; an issue with several runs at its earliest; unlabeled
+  last), then by when the issue was opened, then by number. An issue whose
+  creation time cannot be read runs after the others with its label. `[]`
+  keeps the oldest-first order only. Each poll that found issues logs the
+  order once as `poll.batch.order`. The re-read before each dispatch and
+  the blocker hold are unchanged; the order is per machine and fixed for
+  the batch.
+
+- Cost per run in the numbers (#230). Each 7- and 30-day window's
+  `cost_per_run` (in `/api/status` and the status page's **Numbers** box)
+  now gives the median and total tokens per run that ended (`done` or
+  `failed`), counted as the run budget counts them (#223), and the median
+  and total `total_cost_usd` where the agent reported one. Runs with no
+  usage recorded (from before #223, or on a backend that is not measured)
+  are left out and counted in `runs_without_usage`, never read as 0; a
+  window with no measured run is `null` with the reason in `unavailable`.
+  The weekly chat summary gets a `tokens per run` line, `unknown` when not
+  measured.
+
+- A dev run's PR goes ready only with proof the change works (#221). The
+  agent names a proving test in its DONE checkpoint. The relay runs it on
+  the pushed commit (must pass) and on the base with only the branch's test
+  changes (must fail), in fresh copies, with a time limit, without the
+  GitHub token, and under bwrap with no network when the agent was
+  sandboxed. It also flags deleted, skipped or loosened tests and empty or
+  whitespace-only diffs. A result counts only with evidence the named test
+  ran (pytest's junit report, or the exit code and the test in the output
+  for other runners), and the PR's head must still be the proven commit when
+  it is marked ready. The outcome is `proven`, `failed` or `not proven`;
+  only `proven` (or a docs-only diff) lets the PR go ready. Otherwise the PR
+  stays a draft and the run fails with `PR #N held as a draft`. The PR gets
+  one comment, edited in place, with the test, the command and both results.
+  Off by default: set `repos[].automation.require_proof: true` for a repo
+  whose tests can run offline in a fresh copy. With it off, the agent is not
+  asked for a proof and nothing is posted.
+
+- Each machine's agent sandbox mode is shown (#305). `/api/status` carries
+  `node.sandbox` (`"none"` or `"bwrap"`; `null` for a peer on an older
+  version), each machine's card on the status page says `sandbox off` in the
+  warning colour when the agent runs unconfined, and the poller logs
+  `poller.sandbox_off` once at start with what to do about it.
+- A token budget for each run (#223). `agent.budget.soft_tokens` warns once
+  in chat and the log; `agent.budget.hard_tokens` stops the agent, records the
+  run as failed with the reason `budget: used N tokens, hard limit M`, and
+  says `💸 Over budget` in chat. `agent.budget.rules` sets other limits by
+  issue label, which covers `size:*`; the first match wins. Nothing is limited
+  until a limit is set. The count is input + cache writes + output tokens,
+  read live from the agent's transcript, and covers the whole issue session
+  across resumes and fix rounds. Cache reads are recorded but not counted.
+  A run stopped over budget or as stuck after it opened a PR puts the PR back
+  to draft (or posts a "Do not merge" comment when that fails), starts no
+  merge watcher for it, and names it in chat and the issue comment.
+- Each session records the tokens it used (`sessions.usage_tokens` and
+  `sessions.usage_detail`, with the cost the agent CLI reported). Running
+  cards on the status page and `/api/status` show `tokens_used`,
+  `cache_read_tokens`, `cost_usd`, `token_soft_limit` and `token_hard_limit`.
+  A reading that could not be parsed shows as unknown (`null`), never as 0,
+  and never stops a run on its own (#223).
+
 ## [0.23.0] - 2026-11-05
 
 ### Added
@@ -2525,7 +2591,8 @@ pipeline).
   per-phase implementation plans (Phase 0 through Phase 4).
 - `docs/Claude_Code_Project_Guide.md` — project development guide.
 
-[Unreleased]: https://github.com/AInvirion/ctrlrelay/compare/v0.23.0...HEAD
+[Unreleased]: https://github.com/AInvirion/ctrlrelay/compare/v0.24.0...HEAD
+[0.24.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.23.0...v0.24.0
 [0.23.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.22.0...v0.23.0
 [0.22.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.21.0...v0.22.0
 [0.21.0]: https://github.com/AInvirion/ctrlrelay/compare/v0.20.2...v0.21.0

{ctrlrelay-0.23.0 → ctrlrelay-0.24.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ctrlrelay
-Version: 0.23.0
+Version: 0.24.0
 Summary: Local-first orchestrator for headless coding agents across multiple GitHub repos
 Project-URL: Homepage, https://github.com/AInvirion/ctrlrelay
 Project-URL: Documentation, https://ainvirion.github.io/ctrlrelay/

{ctrlrelay-0.23.0 → ctrlrelay-0.24.0}/docs/architecture.md

@@ -140,6 +140,14 @@ to confirm the PR is mergeable and CI is green. If not, it builds a fix prompt
 fix"), resumes the same agent session with that prompt, and re-verifies. Up
 to `DEFAULT_MAX_FIX_ATTEMPTS` (3) attempts.
 
+Then the proof step
+([`src/ctrlrelay/core/proof.py`](https://github.com/AInvirion/ctrlrelay/blob/main/src/ctrlrelay/core/proof.py),
+#221) runs the test the agent named, in fresh copies of the pushed commit
+(must pass) and of the base with only the branch's test changes (must fail),
+and scans the diff for weakened tests, when the repo turns it on. Only a proven run's PR is marked
+ready; any other outcome leaves it a draft and fails the run. See
+[`automation.require_proof`](configuration.md#reposautomationrequire_proof).
+
 `PRWatcher` (in
 [`src/ctrlrelay/core/pr_watcher.py`](https://github.com/AInvirion/ctrlrelay/blob/main/src/ctrlrelay/core/pr_watcher.py))
 polls a PR for merge status; the post-merge handler in

{ctrlrelay-0.23.0 → ctrlrelay-0.24.0}/docs/configuration.md

@@ -40,6 +40,7 @@ repos:        [ ... ]
 | `transport` | object | **yes** | — | See [transport](#transport). |
 | `dashboard` | object | no | (defaults) | See [dashboard](#dashboard). |
 | `web` | object | no | (defaults) | The read-only status page. See [web](#web). |
+| `intake` | object | no | (defaults) | The order each poll's new issues run in. See [intake](#intake). |
 | `claims` | object | no | (defaults) | Claim timings for several machines on one account. See [claims](#claims). |
 | `repos` | list | no | `[]` | See [repos](#repos). |
 | `schedules` | object | no | (defaults) | See [schedules](#schedules). |
@@ -217,6 +218,41 @@ modes either need every shell command allow-listed or filter by command text,
 which a wrapper script walks past, so they are not used as a guard. The guard
 is the next section: the agent holds no GitHub credentials.
 
+### agent.budget
+
+A token budget for each run (#223). Both limits unset (the default): no limit,
+and nothing changes. Usage is measured and shown on the status page either
+way. See [run budgets](operations.md#run-budgets) for what is counted.
+
+```yaml
+agent:
+  budget:
+    soft_tokens: 2000000     # warn once in chat and the log
+    hard_tokens: 6000000     # stop the agent
+    rules:                   # first rule naming any of the issue's labels wins
+      - labels: ["size:S", "size:XS"]
+        soft_tokens: 500000
+        hard_tokens: 1500000
+      - labels: ["size:L", "size:XL"]
+        hard_tokens: 15000000   # soft_tokens left out: 2000000 from above
+      - labels: ["no-budget"]
+        soft_tokens: null       # null: no limit for these runs
+        hard_tokens: null
+```
+
+| Key | Type | Default | Description |
+|---|---|---|---|
+| `soft_tokens` | int or null | `null` | When a run's total first reaches it, chat gets one line and the log gets `budget.soft_limit`. Once per session. |
+| `hard_tokens` | int or null | `null` | When a run's total reaches it, the agent is stopped and the run is recorded as failed with the reason `budget: used N tokens, hard limit M`. Must be at least `soft_tokens`. |
+| `rules` | list | `[]` | Other limits by issue label. Each rule has `labels` (a non-empty list) and optionally `soft_tokens` and `hard_tokens`. Labels match without regard to case. A size is a label (`size:S`), so this covers sizes too. A limit a rule leaves out comes from the top level; one set to `null` is no limit. |
+
+The count is input tokens + cache writes + output tokens; cache reads are
+recorded but not counted. The budget is for the whole issue session, across
+resumes and CI fix rounds. Unknown keys and a hard limit below the soft one
+(after a rule inherits from the top level) are refused when the config loads.
+Only the `claude` backend is measured. Runs with no issue (secops) get the
+top-level limits for each spawn.
+
 ### What the agent can reach (#219)
 
 The dev and task agents run **without GitHub access**. ctrlrelay starts them
@@ -290,6 +326,14 @@ your user. With `sandbox: "none"` a deliberately hostile agent can read
 and use them by hand, and it can write anywhere your user can, including your
 dotfiles and ctrlrelay's install. On macOS the login keychain is reachable
 whatever the environment says. `sandbox: "bwrap"` closes this on Linux.
+macOS has no sandbox yet, so a macOS node should only take issues from
+repositories you trust.
+
+So that an unconfined node is never a surprise (#305), each machine's mode is
+in `/api/status` as `node.sandbox` (`"none"` or `"bwrap"`; `null` for a peer on
+an older version), its card on the status page says `sandbox off` in the
+warning colour when it is `"none"`, and the poller logs `poller.sandbox_off`
+once at start.
 
 **`sandbox: "bwrap"` (#236).** The dev and task agents run under bubblewrap
 in a read-only copy of the machine (the secops agent, which needs your GitHub
@@ -545,6 +589,29 @@ web:
 The page has no login and shows repository names, issue titles and the
 questions agents asked. **Never expose it to the internet.**
 
+## intake
+
+The order each poll's new issues run in (#356). A poll gathers one batch of
+new issues and works through it before the next poll; this sets the order of
+that batch.
+
+```yaml
+intake:
+  priority_labels: ["P0", "P1", "P2", "P3"]
+```
+
+| Key | Type | Default | Description |
+|---|---|---|---|
+| `priority_labels` | list of strings | `["P0", "P1", "P2", "P3"]` | Issues carrying an earlier label in this list run first; labels are compared without case, and an issue carrying several runs at its earliest. Issues carrying none of them run last. Within the same label, the oldest issue runs first (by when it was opened, then by number). An issue whose creation time cannot be read runs after the others with its label. `[]` keeps the oldest-first order only. Each label must be a non-empty string and listed once. |
+
+Each poll that found new issues logs the order it will run them in, as one
+`poll.batch.order` event (`order`: `owner/repo#N` in run order). The order is
+fixed when the poll ends: an urgent issue opened while a batch runs waits for
+the next poll. Each issue is still read again just before it starts, and an
+issue blocked by an open issue is still held until the blocker closes; those
+can skip an issue, never reorder the batch. The order applies to this machine
+only, not across machines.
+
 ## claims
 
 How machines that share one GitHub account keep off each other's issues.
@@ -719,9 +786,11 @@ and ask the operator), or `never` (skip).
 | `include_labels` | `[]` | Issue labels that opt an issue **into** the dev pipeline regardless of who is (or isn't) assigned. See [include_labels](#reposautomationinclude_labels) below. |
 | `require_labels` | `[]` | Issue labels that gate the plain-assignment trigger: when set, assignment alone is no longer enough — the issue must also carry one of these labels. See [require_labels](#reposautomationrequire_labels) below. |
 | `ci_wait_timeout_seconds` | `600` | Unused since #219 (the agent no longer waits on CI; ctrlrelay does). Still accepted so existing configs load. See [ci_wait_timeout_seconds](#reposautomationci_wait_timeout_seconds) below. |
+| `require_proof` | `false` | A dev run's PR goes ready only when the relay itself proved the change: a test that fails on the base and passes on the branch, and no weakened tests. See [require_proof](#reposautomationrequire_proof) below. |
 
-The current secops and dev pipelines read these settings to bias their prompts
-to the agent — they're not enforced by hard-coded checks.
+The current secops and dev pipelines read the policy settings to bias their
+prompts to the agent — they're not enforced by hard-coded checks.
+`require_proof` is the exception: the relay enforces it.
 
 ### repos[].automation.operator_prs
 
@@ -922,13 +991,120 @@ repos:
 - This does not change the overall session timeout (`agent.default_timeout_seconds`,
   default 1800s) — it only bounds the one `ci wait` call.
 
+### repos[].automation.require_proof
+
+Off by default (#221). Green CI says a change did not break what was already
+tested; it does not say the issue was fixed. With `require_proof` the relay
+checks real evidence before it hands a dev run's PR over as ready.
+
+The agent names its proof in its DONE checkpoint:
+
+```json
+"proof": {"command": ["uv", "run", "--offline", "pytest", "tests/test_x.py::test_y"],
+          "tests": ["tests/test_x.py::test_y"]}
+```
+
+After CI is green, the relay runs `command` twice, as an argument list with
+no shell, each time in a fresh copy of committed content (never the agent's
+worktree):
+
+- on the commit it pushed, where it must pass;
+- on the base commit (the merge base with the default branch) with only the
+  branch's changes to test files applied, where it must fail.
+
+Each run has a 10-minute limit and never gets the operator's GitHub token.
+When the agent ran under `agent.sandbox: bwrap`, the proof runs under the
+same kind of bwrap confinement, with no network and only the copy and the
+agent's cache dir writable. When the agent ran with `sandbox: none`, it
+already ran arbitrary code as this user, so the proof runs unconfined too.
+The copies have no untracked files, so the command must set up what it needs
+offline.
+
+A run counts only with evidence that the named tests ran. For a pytest
+command (`pytest`, `uv run pytest`, `python -m pytest`) the relay adds
+`--junitxml` and reads the report: on the branch each named test must have
+passed, on the base each must have failed or errored. pytest exiting 2 to 5
+(a collection error, a usage error, no tests collected, or `uv run` failing
+to start it), no report, or a named test missing from it is no evidence.
+One exception, on the base only: a collection error in the file of a named
+test (a new test file that imports code only the branch has) counts as
+failing before, recorded as "could not import on base". A collection error
+only in other files does not. For any other runner the command must exit 0 on the
+branch and 1 on the base, and each named test (the part after the last
+`::`) must appear in its output.
+
+There are three outcomes:
+
+| Outcome | When |
+|---|---|
+| `proven` | The named tests failed on the base and passed on the branch. |
+| `failed` | The named tests ran on the branch and did not pass; they ran on the base and passed, so they prove nothing; the diff has no real change (empty or whitespace only); or the diff weakens tests. |
+| `not proven` | No proof was named, the proof is malformed or names a test outside the test files, there is no evidence the named tests ran (a setup, dependency or network failure, a usage error, a collection error), the command could not start, it timed out, or the branch moved after the proof ran. |
+
+Only `proven` lets the PR go ready. Two exemptions, recorded as `not
+required`: a diff that only touches docs (`*.md`, `*.rst`, `*.adoc`,
+`docs/**`), and a repo with `require_proof: false`.
+
+**The test weakening scan** reads the diff's test files (`tests/`, `test_*.py`,
+`*_test.py`, `conftest.py`, `*.test.ts`, `*.spec.js`, `__tests__/` and the
+like) and the test runner's config. These are blockers:
+
+- a deleted test file, in any language;
+- a test file renamed to a name the runner does not collect, or moved out
+  of the tests;
+- a removed test function;
+- an added skip, xfail or `.only` marker: decorators, a module-level
+  `pytestmark`, `pytest.importorskip`, unittest skips, and JS `.skip`,
+  `xit`, `it.only`;
+- an added line in runner config that can deselect or skip: `addopts`,
+  `--deselect`, `-k`, `testpaths`, `norecursedirs`, `collect_ignore` (in
+  `pytest.ini`, `pyproject.toml`, `setup.cfg`, `tox.ini`, `conftest.py`),
+  and jest's `testPathIgnorePatterns`, `testMatch`, `testRegex`;
+- a bare `return` added as the first statement of an existing Python test;
+- an assert line removed from a diff hunk that adds no assert.
+
+An assert line removed from a hunk that also adds one is an edit. The scan
+cannot tell a corrected expectation from a loosened one, so it is shown as a
+warning on the PR and does not block. Lines are matched by text, so a test or
+an assert moved to another file is not a hit. Any blocker makes the run
+`failed`, unless the plan's acceptance criteria say the change is intended
+with an item `Test removal intended: <why>`. Python and JS/TS are read; a
+test file in another language makes the scan report `not checked`, never
+`clean`.
+
+If no test can show the change, the plan's acceptance criteria can say `No
+test possible: <why>`. The run is still `not proven` and the PR stays a
+draft for a person to decide; the PR comment says why.
+
+**What the PR shows.** One comment, edited in place on a later run: the
+outcome and why, the proving test, the command, the fail-before and
+pass-after results with a short redacted tail of each, and any scan hits.
+A run that is not proven fails like any required step after the PR exists
+(see [below](#reposteps)): the PR stays a draft, and chat gets `❌ Failed on
+#N: PR #M held as a draft: not proven …`. A PR GitHub would not open as a
+draft gets a comment telling readers not to merge it.
+
+With `require_proof` on, the PR is opened as a draft, as with
+`steps.review: required`. When both are on, the review step marks it ready
+after the proof passed and the review posted. Right before it is marked
+ready, the relay reads the PR's head from GitHub: if it is not the commit the
+proof ran on, or cannot be read, the run is `not proven` and the PR stays a
+draft.
+
+Turn it on only for a repo whose tests can run offline in a fresh copy of
+the commit. With it off (the default) the agent is not asked for a proof,
+the PR is opened as before, and no proof comment is posted.
+
 ### repos[].steps
 
 The dev pipeline does its work in a fixed order: **plan** (an optional
 planning session, off unless asked for), **code** (the agent run and
 any questions it asks), **publish** (the relay pushes the branch and opens the
 pull request), **verify** (wait for CI and mergeability, asking the agent to
-fix what fails, up to `max_fix_attempts` rounds) and **review** (the
+fix what fails, up to `max_fix_attempts` rounds), **proof** (the relay runs
+the agent's proving test before and after; switched by
+[`automation.require_proof`](#reposautomationrequire_proof), not here) and
+**review** (the
 configured reviewer reads the pushed commit and posts its findings). The order
 is not configurable. `steps` says how strict each step is for this repo.
 Only the dev pipeline reads it. The task and secops pipelines have no
@@ -976,7 +1152,10 @@ that can each be checked by a command or by reading the diff. The relay then:
   outside the worktree, so it is never committed;
 - posts the criteria list as one comment on the issue;
 - adds the plan to the code session's prompt under `## Plan`, telling the
-  agent the criteria are what "done" means.
+  agent the criteria are what "done" means;
+- reads two items from the criteria for the proof step: `Test removal
+  intended: <why>` and `No test possible: <why>` (see
+  [require_proof](#reposautomationrequire_proof)).
 
 If the issue is too vague to plan, the planning session asks one question,
 through the same chat loop the code session uses. No code session runs and no
@@ -1047,7 +1226,7 @@ exists.** The ordinary failure path, unchanged:
 - chat gets `❌ Failed on #N: …`, never `✅ PR ready`.
 
 A resumed session (an answer that arrived after the run stopped to ask) goes
-through the same publish and verify steps. It did not run the review before
+through the same publish, verify and proof steps. It did not run the review before
 these switches, and with `review` unset or `on` it still does not; with
 `review: required` it does.
 
@@ -1101,8 +1280,10 @@ schedules:
 Once a week, each node sends one chat message with its own numbers for
 the last 7 days, the same ones its status page shows: runs started, done,
 failed, blocked and stuck, the success rate, the median time to PR and to
-merge, and how many issues the relay finished were reopened. A number the
-node cannot read is written `unknown`, with the reason, never 0.
+merge, how many issues the relay finished were reopened, and the median
+tokens per run (with the cost, when the agent reported one) over the runs
+that ended with their usage recorded. A number the node cannot read is
+written `unknown`, with the reason, never 0.
 
 The daemon checks every five minutes. It sends at the first check after
 `day` at `hour`:00 that finds nothing delivered since then, so a node that

{ctrlrelay-0.23.0 → ctrlrelay-0.24.0}/docs/operations.md

@@ -304,6 +304,18 @@ Added in #247:
 |---|---|
 | `node` | This machine's own block: `name` (`node_id`); `usage`, the agent subscription's usage as the last run or probe read it: `status`, `read_at`, and for `five_hour` and `seven_day` each `{used (0 to 1), resets_at}`, or `"unknown"` for a window it could not read; the whole `usage` is `"unknown"` before any reading. `pause_at` is `usage.pause_at`, and `paused` is `{reason, until}` while the machine takes no new issues, else `null`. The page shows it in the **Usage** box. `node` is `null`, with a line in `errors`, only if reading it failed outright. |
 
+Added in #223, on every `running` card (see [Run budgets](#run-budgets)):
+
+| Field | What |
+|---|---|
+| `tokens_used` | Tokens the session has used so far, over all its spawns: input + cache writes + output, what the budget counts. `null` before anything was measured, on a backend that is not measured, or when a transcript row could not be read (then `reason` gives the number of rows and the floor). Never `0` for unknown. |
+| `cache_read_tokens` | Cache reads, recorded but not counted. `null` as for `tokens_used`. |
+| `cost_usd` | The sum of `total_cost_usd` the agent CLI reported at the end of each spawn. Information only. `null` until a spawn reported one. |
+| `token_soft_limit`, `token_hard_limit` | The limits applied to this run. `null` when no limit applies or nothing was measured yet. |
+
+The page shows them in a `tokens:` row on each running card; the count is in
+the warning colour once it passes the soft limit.
+
 Added in #237: `node.github_api` is the account's GitHub API budget as this
 machine last read it: `read_at`, and for `core`, `search` and `graphql` each
 `{remaining, limit, resets_at, state}` with `state` one of `ok`, `low`, `out`,
@@ -340,6 +352,12 @@ Added in #307:
 | `node.config_fingerprint` | `{v, repos_hash, profiles, repos, schedules}`, `v` is `2`. `profiles` maps a profile hash to `{automation, steps}`, each a map of field name to a hash of its value; `repos` maps each repo name to its profile hash, so repos with the same configuration share one profile; `repos_hash` is the hash of the sorted, lowercased repo names; `schedules` maps each `schedules` field to a hash. Hashes and repo names only, never a value. `null` if it could not be built. A peer's fingerprint is unknown unless every hash is 16 lowercase hex characters, `repos_hash` is the hash of its own repo names, each profile key is the hash of that profile, and each repo names a profile it carries. A field's hash cannot be checked against its value, which is never sent. |
 | `fleet.peers[].drift` | `{status, reason, only_here, only_there, fields, lines}`. `status` is `same`, `differs` or `unknown` (with `reason`: not read yet, not reachable, the peer sends no fingerprint in this format, or the comparison failed); `fields` maps a field path such as `automation.exclude_labels` to the repos it differs on; `lines` is what the page shows. Leaves out `fleet.expected_drift`. The peer's own fingerprint is not repeated in `fleet.peers[].node`. |
 
+Added in #305:
+
+| Field | What |
+|---|---|
+| `node.sandbox` | This machine's `agent.sandbox`: `"none"` (the agent runs with the user's full access) or `"bwrap"`. In `fleet.peers[].node` it is `null` when the peer is on an older version or sent anything else, which means unknown, never off. The page shows `sandbox off` in the warning colour for `"none"`. |
+
 Added in #230: `numbers` is how this machine's runs went, for the last 7 and
 the last 30 days (`numbers.windows`, each with `days`, `since`/`since_ts`,
 `until`/`until_ts`). The page shows it in the **Numbers** box, with a table
@@ -354,7 +372,7 @@ control plane. Every field is defined in `src/ctrlrelay/web/numbers.py`.
 | `time_to_pr_s`, `time_to_merge_s` | `{median, n}`: seconds from the run's start to its first `pr_opened` (or `merged`) event, over the `n` runs that have one. Events stamped after now are not read. `null` when events cannot be read, or when a run in the window started before the oldest event still kept: its PR event may have been deleted. The daemon deletes events older than `web.history_days` (7 by default), so with the default the 30-day times are `null`. **To get them, set `web.history_days` to 30**; they appear once the events kept reach back past every run in the window (up to 30 days after the change, since deleted events do not come back). |
 | `came_back` | Distinct issues dispatched again in the window after an earlier run on the same issue had ended `done`. The window's `came_back_issues` lists them (`repo`, `issue_number`, `url`; at most 50). |
 | `reopened` | Issues the relay finished (it saw their PR merge and closed them) that were reopened on GitHub in the window, dated by GitHub's `reopened` event. Counted whether or not the relay picks the issue up again; one per finish. The window's `reopened_issues` lists them. The poller notices a reopen only for an issue its search returns (open and assigned to the operator, or wearing a trigger label), with no extra GitHub call unless a finished issue shows up open. Finishes are recorded in the state database from this version on, plus those still in the kept `merged` events when the table is created. `null` on a database the daemon has not migrated yet. A `reopened` event goes to the feed when one is recorded. |
-| `cost_per_run` | Always `null` for now: cost per run is not recorded yet (#223). The window's `unavailable` says this, and why any other field is `null`. |
+| `cost_per_run` | What the runs that ended (`done` or `failed`) in the window used, from the token usage the run budget records on each run (#223, see [Run budgets](#run-budgets)): `{tokens, cost_usd, runs_without_usage}`. `tokens` is `{median, total, n}` over the `n` ended runs whose usage was recorded and read in full, counted as the budget counts (input + cache writes + output; cache reads left out). `cost_usd` is `{median, total, n}` in US dollars over those of them whose agent reported a cost (`total_cost_usd`, information only), or `null` when none did. `runs_without_usage` is how many ended runs are left out: no usage recorded (runs from before #223, or a backend that is not measured, such as `opencode`) or usage that could not be read in full. They are never counted as 0. Blocked and running runs are left out, since their count still grows. `null` when no ended run in the window has its usage measured, or on a database the daemon has not migrated yet; the window's `unavailable` says which, and why any other field is `null`. |
 | `stuck` | Runs the relay stopped as stuck (#224), also counted in `failed`. `null` only on a database the daemon has not migrated yet. |
 
 `numbers` is `null`, with a line in `errors`, when the sessions cannot be read.
@@ -712,6 +730,57 @@ A stopped run is recorded as `failed` with a `stuck_reason` in the
 the reason, and chat gets `🔁 Stuck on #<n>, stopped: <reason>`. The status
 page's numbers count these runs in `stuck`.
 
+## Run budgets
+
+Each run can have a token budget (#223), set in
+[`agent.budget`]({{ '/configuration/' | relative_url }}#agentbudget). There is no
+limit until one is set.
+
+- **What is counted.** Every 30 seconds ctrlrelay adds up the usage in the
+  agent's live transcript, sub-agents included: input tokens, cache writes
+  and output tokens. Cache reads are recorded but not counted. One API
+  message written as several rows is counted once.
+- **Per session, not per spawn.** A dev or task run spawns the agent more
+  than once (the first run, resumes after a question, CI fix rounds). They
+  all count against the same budget, because each spawn starts from what the
+  session row already holds.
+- **Soft limit.** The first time the total passes it, chat gets one line
+  (`💰 #<n> (<repo>) has used <N> tokens, past its soft limit ...`) and the
+  log gets `budget.soft_limit`. Once per session.
+- **Hard limit.** The agent is stopped through the same path as a stuck run.
+  The run is recorded as `failed` with `stuck_reason` set to
+  `budget: used <N> tokens, hard limit <M>`, its summary starts with
+  `Stopped:`, the issue comment gives the reason, and chat gets
+  `💸 Over budget on #<n>, stopped: <reason>`. A checkpoint the agent wrote
+  just before the stop is not used: the work is not handed over as ready.
+  When no pull request exists yet, the unfinished work is not saved to a
+  branch (only a timeout does that). The status page's numbers count the
+  run in `failed`, not in `stuck`.
+- **A stop after the PR was opened** (over budget in a CI fix round, or
+  stuck there): the PR is put back to draft and kept with the work so far,
+  no merge watcher is started for it, and chat and the issue comment say
+  `PR #<n> was kept as a draft with the work so far.` If GitHub refuses the
+  draft, a "Do not merge" comment goes on the PR instead, and the messages
+  say so.
+- **A floor, not an exact bill.** Only API calls that write an assistant
+  row to the transcript are counted; calls the CLI makes on its own without
+  one (for example, short helper calls) are not. The count can be a little
+  under what the subscription was charged.
+- **Unknown is not zero.** A transcript row whose usage cannot be read adds
+  nothing and is counted as unreadable; the log gets `budget.usage_unreadable`
+  and the page shows the total as unknown. What could be read is still a
+  true floor, so a run is stopped only when that floor reaches the hard limit.
+  An unreadable row on its own never stops a run.
+- **What is stored.** `sessions.usage_tokens` is the counted total (a floor
+  when rows were unreadable) and `sessions.usage_detail` holds the breakdown,
+  the unreadable row count, the cost and the limits applied, as JSON.
+  `cost_usd` is what the agent CLI's final `result` event reported
+  (`total_cost_usd`), summed over the session's spawns; it is information
+  only.
+- **Backends.** Only the `claude` backend writes a transcript ctrlrelay can
+  read. A run on `opencode` is not measured; with a limit set, the log gets
+  `dispatcher.budget.unmeasured` and nothing is enforced.
+
 ## When to restart
 
 | You changed... | Restart... |

{ctrlrelay-0.23.0 → ctrlrelay-0.24.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ctrlrelay"
-version = "0.23.0"
+version = "0.24.0"
 description = "Local-first orchestrator for headless coding agents across multiple GitHub repos"
 readme = "README.md"
 requires-python = ">=3.12"