@wardby/cli 0.6.0 → 0.7.0

package/help/dependency-cache.md

@@ -0,0 +1,143 @@
+---
+id: dependency-cache
+title: Dependency cache for coding runs
+summary: Restore npm node_modules from a verified snapshot before a coding agent starts, so runs skip the install.
+audience: operator
+tags: [coding-agents, packages, npm, node_modules, dependency-cache, cache, npm-ci, lockfile, snapshot, restore, blob-store, workspaces, tool_unavailable, budget_too_small, snapshot_skipped, timeout, CODING_CPUS, CODING_MEMORY_MB, disk, sha_mismatch, unsafe_archive, dependencyCache, worker_input_failed, workerImageRef]
+appliesTo: >=0.7.0
+---
+
+# Dependency cache for coding runs
+
+With the dependency cache on, a coding run's `node_modules` is prepared
+**before the agent starts** from a snapshot in a blob store, so the agent
+doesn't spend minutes on `npm ci`. It covers npm only. Off by default.
+
+## How it behaves
+
+For each eligible project the run reports one result in `get_run`
+(`dependencyCache.projects[]`):
+
+- `hit`: a snapshot for this exact lockfile was restored; nothing was
+  installed. A snapshot that fails to download or verify isn't used: the
+  project installs `cold` instead.
+- `nearest`: a snapshot for an older lockfile of the same repository and folder
+  was restored, then updated to the committed lockfile.
+- `cold`: no snapshot, so `npm ci` ran and the result was stored (when it fit the disk).
+- `skipped` or `refused`: nothing was prepared and the agent installs for
+  itself. A skip never fails the run; its `reason` says why.
+
+The agent's task says dependencies are pre-installed and not to run `npm ci`
+there (it would delete the prepared tree); `npm install <package>` is fine.
+A pull request gets a "Dependency cache: restored `node_modules` for ..." line
+for folders restored from a snapshot.
+
+## Why it is safe
+
+- Snapshots are built only by Wardby, before the agent starts. Then the upload
+  window is sealed: the worker seals it, and the coding proxy seals it itself
+  on the run's first model request, so the agent can't read or write the
+  cache.
+- A restore happens only after the coding proxy re-approves the committed
+  lockfile under the agent's current allowlist, release age and advisory
+  policy. A policy that now refuses a package means no restore.
+- Agents on the same repository share snapshots (the key is the repository,
+  project folder, lockfile and image, not the package policy), and each
+  restore is still gated by that agent's own current policy.
+- Keys come from the server, not the worker. Downloads are checked against the
+  SHA-256 and size the proxy recorded (that is the integrity check), the
+  archive is scanned, and the tree's recorded package versions are compared
+  with the lockfile before use. A damaged snapshot is deleted and rebuilt.
+  A restore skips re-scanning an archive whose scan result was recorded at
+  upload and whose digest matches; extraction still enforces every archive
+  rule.
+- The packages are the ones the agent's policy approves; the cache doesn't vet
+  their contents beyond what a normal install through the registry does.
+- Install scripts are not cached. Snapshots are built with `--ignore-scripts`;
+  `npm rebuild` runs after the seal, only when needed, with a minimal
+  environment.
+
+## When it applies
+
+- npm with `package-lock.json` `lockfileVersion` 2 or 3, at the repository root
+  and/or up to 4 projects up to 3 folders deep.
+- Ignored: `node_modules`, dot-folders, symlinks, folders that already have a
+  `node_modules`, and npm workspaces monorepos (root and members are skipped).
+- The agent needs a package allowlist.
+- Every worker image must come from a release that includes the cache: the
+  Codex worker images, the Claude Code worker image and its tool runner
+  images. An older Codex or Claude Code worker image fails the run at input
+  (`worker_input_failed`). Rebuild a `workerImageRef` image `FROM` that
+  release's driver image (Codex) or tool runner image (Claude Code) first;
+  `create_agent` and `update_agent` warn when an agent has both.
+
+## Turn it on
+
+1. On the coding proxy: `WARDBY_DEPENDENCY_CACHE=true` plus a blob store
+   (`BLOB_STORE=gcs|s3|local` and its bucket or directory variables). GKE:
+   `dependency_cache_enabled` and `dependency_cache_bucket_name` in Terraform,
+   then `WARDBY_DEPENDENCY_CACHE=true deploy/gke/up.sh`. AWS: the same Terraform
+   variables, plus the `BLOB_STORE_S3_*` proxy variables. kind:
+   `WARDBY_DEPENDENCY_CACHE=true bash deploy/kind-coding/up.sh` (not durable).
+   Docker quickstart: add `WARDBY_DEPENDENCY_CACHE=true` to `.wardby/.env`
+   (requires wardby 0.7.0 or later).
+2. GKE: nothing extra. `up.sh` grants the proxy the `DependencySnapshot`
+   table before the rollout.
+3. Per agent, once the worker images are current: `update_agent` with
+   `codingProfile.dependencyCache: true`. This needs `packages:approve` or
+   `agents:admin`.
+
+## Disk use
+
+The cache works within a budget: the run's workspace disk
+(`workspaceDiskMb`) less what the checkout uses and a margin. Restores and the
+snapshot archive are held to it while they are written. An install is checked
+only before it starts: at least 1 GiB must be left (or twice the project's
+tree when a snapshot of it was seen), otherwise the project is
+`budget_too_small` and the agent installs for itself. npm's writes during the
+install itself aren't bounded. After its own install the worker deletes npm's
+download cache (`_cacache`), then archives the tree for upload only if the
+archive fits what is left. A snapshot skipped for space keeps the installed
+tree and the `cold` or `nearest` result; the worker log says
+`snapshot_skipped` (reason `budget`), not `get_run`.
+
+Tuning, on the proxy: `WARDBY_DEPENDENCY_CACHE_MAX_SNAPSHOT_MB` (1024),
+`_REPO_CAP_MB` (4096), `_TOTAL_CAP_MB` (20480), `_MAX_AGE_DAYS` (14, counted
+from creation, so snapshots are rebuilt periodically) and `_UPLOAD_SECONDS`
+(900). On GKE and AWS keep the Terraform `dependency_cache_max_age_days` equal
+to `_MAX_AGE_DAYS`; the bucket also deletes objects older than that plus 7 days.
+
+## Troubleshooting
+
+Read `dependencyCache` in `get_run`; then match the `reason`. A rejected `hit`
+or `nearest` snapshot is not a reason: the project shows `cold`.
+
+| Reason                                                                                       | What to do                                                                                                                                                                                                                                         |
+| -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `no_lockfile`, `lockfile_unreadable`, `no_package_json`, `no_dependencies`                   | Nothing to cache in that folder.                                                                                                                                                                                                                   |
+| `lockfile_version`                                                                           | Regenerate the lockfile with npm 7 or later.                                                                                                                                                                                                       |
+| `workspaces`, `workspace_member`, `shrinkwrap_present`                                       | Not supported; the agent installs itself.                                                                                                                                                                                                          |
+| `node_modules_present`, `cache_dir_unsafe`                                                   | Remove the committed `node_modules` or `.cache` (symlink) from the repository.                                                                                                                                                                     |
+| `too_many_projects`                                                                          | Only 4 projects per run are prepared.                                                                                                                                                                                                              |
+| `no_registry`, `plan_failed`, `plan_refused`                                                 | Give the agent a package allowlist and fix refused packages; see [Approve packages for coding agents](help://coding-packages). A plan the proxy left incomplete is retried for up to 10 minutes (at most 5 attempts per project, 8 plans per run). |
+| `cache_unavailable`, `plan_required`, `sealed`, `lookup_failed`, `download_failed`           | Check `WARDBY_DEPENDENCY_CACHE` and `BLOB_STORE` on the proxy, the store's permissions and the proxy log.                                                                                                                                          |
+| `size_mismatch`, `sha_mismatch`, `unsafe_archive`, `inconsistent`                            | The snapshot was not trusted; the project installed `cold` instead. It is rebuilt when it ages out or is deleted. Repeats point at the blob store.                                                                                                 |
+| `too_large`, `over_budget`, `budget_unknown`, `budget_too_small`                             | Raise `WARDBY_DEPENDENCY_CACHE_MAX_SNAPSHOT_MB` or the agent's `workspaceDiskMb` (see [Disk use](#disk-use)).                                                                                                                                      |
+| `install_failed`, `timeout`, `cancelled`, `internal_error`, `worker_error`, `report_invalid` | The agent installs for itself; check the worker log. The preparation gets at most 20 minutes or half the time left before the run's deadline. On Docker a huge restore can exhaust the tmpfs inode limit before its bytes.                         |
+| `tool_unavailable`                                                                           | Claude Code tool-runner image predates the cache or is custom; rebuild it from the current tool-runner image. The proxy still seals on the run's first model request.                                                                              |
+| `already_started`, `invalid_deadline`                                                        | Not expected: a command ran before preparation, or the worker and tool runner images are from different releases.                                                                                                                                  |
+| `worker_input_failed` (the run fails)                                                        | A Codex or Claude Code worker image, or a `workerImageRef` image, older than the cache. Update the images, or rebuild the custom image `FROM` the current release, before turning the cache on.                                                    |
+
+Every run of a large project installing `cold` again usually means its
+snapshot is never stored. Check the worker log for `snapshot_skipped` with
+reason `timeout`: its `limit` says which bound ran out. For `pack` or
+`deadline`, give coding pods more CPU (`CODING_CPUS=2`); for `upload_window`,
+raise `WARDBY_DEPENDENCY_CACHE_UPLOAD_SECONDS`. A large tree (about 1 GiB) can
+also need more than the default 2048 MiB of memory to install under gVisor;
+raise `CODING_MEMORY_MB`.
+
+`rebuild` of `failed`, `timeout` or `unsealed` means native add-ons may be
+unbuilt; the agent can run `npm rebuild`.
+
+The full reference, with every limit, is
+[`docs/coding-packages.md`](../docs/coding-packages.md#dependency-cache).

package/help/deploy-gke.md

@@ -3,7 +3,7 @@ id: deploy-gke
 title: Deploy Wardby on GKE Autopilot
 summary: Use the supported Google Cloud path for a private database, isolated coding workers, and HTTPS ingress.
 audience: operator
-tags: [deployment, gke, gcp, kubernetes, production, secrets, jira]
+tags: [deployment, gke, gcp, kubernetes, production, secrets, jira, upgrade, database-grants]
 appliesTo: >=0.2.1
 ---
 
@@ -25,22 +25,29 @@ The deployment process is:
 1. Install `gcloud`, Terraform, Docker with `linux/amd64` support, `kubectl`,
    Helm, Node.js 24, and authenticate to the target project.
 2. Configure `deploy/gke/terraform.tfvars`, apply Terraform, and prepare the
-   Gateway's address, certificate map, Cloud Armor policy, and DNS record.
+   Gateway's address, certificate map, Cloud Armor policy, and DNS record. On
+   a brand-new project, run `deploy/gke/bootstrap-database-iam.sh` once after
+   `terraform apply`, so the migrator has its database grants before the
+   first migration.
 3. Put first-time values in an untracked `.env.local`; `deploy/gke/up.sh`
    seeds Secret Manager without overwriting existing production values.
    Jira settings are optional there, all or none; see [Jira](jira.md).
 4. Run `WARDBY_HOSTNAME=wardby.example.com deploy/gke/up.sh`, then verify DNS,
-   certificate issuance, database IAM bootstrap, and service health.
+   certificate issuance, and service health.
    Use `WARDBY_HOSTNAME`, not `HOSTNAME` (shells set that to the machine name);
    the script refuses local names such as `*.lan` or `*.local`, and asks you
    to confirm (or set `WARDBY_HOSTNAME_CHANGE=<new hostname>`) before moving
    a deployment to a different hostname.
 
-When a release changes `deploy/gke/database-grants.sql`, re-run the database
-grants bootstrap **before** deploying that release, so the proxy role can
-already write the tables and columns it adds (such as per-model usage for
-[cost attribution](cost-attribution.md), or a run's live turn count). Until the
-grants are applied, the coding proxy's writes are refused and coding runs fail.
+To upgrade, re-run `deploy/gke/up.sh` from the new release. It applies the
+migrations, then the database grants (`deploy/gke/database-grants.sql`), and
+only then rolls out the new pods, so a release that adds tables or columns the
+coding proxy uses (such as per-model usage for
+[cost attribution](cost-attribution.md)) needs no manual grants step. If the
+grants fail, `up.sh` stops before the rollout with a non-zero exit;
+`deploy/gke/bootstrap-database-iam.sh --check` shows the refused statement
+without changing anything. See
+[Upgrading an existing deployment](../docs/getting-started-gke.md#upgrading-an-existing-deployment).
 
 The Kubernetes launcher runs Codex and Claude Code coding workers; a Claude
 Code pod adds a tool-runner sidecar with its own image. Configure an identity

package/help/errors/budget-exhausted.md

@@ -0,0 +1,187 @@
+---
+id: errors/budget-exhausted
+title: Run stopped out of budget (budget_exhausted)
+summary: A coding run stopped because its next model request did not fit what was left of its budget; get_run's budgetEvents says which limit and why.
+audience: operator
+tags: [budget, budget_exhausted, coding_budget_exhausted, overrun, allowedOverrunPercent, overrun_unavailable, draft]
+appliesTo: ">=0.7.0"
+---
+
+# Run stopped out of budget
+
+## What happened
+
+Before each model request the coding proxy holds an amount against the run's
+budget. The first request of a conversation (and any fallback) holds the worst
+case: the whole request body as new input plus the largest reply the request
+allows, priced at the model's output rate. After a mostly cached reply in the
+same conversation, the cached part of the prompt is held at the cache-read
+rate instead, so the hold is much smaller. A request whose hold does not fit in what is
+left is refused, and the run ends `budget_exhausted` (its error reads
+`coding_budget_exhausted: <sentence>`). The run stopped before a request whose
+hold did not fit. A cache miss can make a settled reply cost more than its
+hold, up to its worst case (see [Holds vs actual spend](#holds-vs-actual-spend)).
+
+## Find out why
+
+Call `get_run` and read `budgetEvents`. Each event has:
+
+- `kind`: `refused`, or `overrun_unavailable` for an overrun top-up its group
+  or run tree had no room for (see [Overrun not available](#overrun-not-available)).
+- `limit`: which cap bound the run.
+  - `run`: the agent's own `budgetUsd`.
+  - `run-tree`: what the lead agent had left for its whole run tree.
+  - `day`, `week`, `month`: the budget group's cap for that period.
+- `at`: when it was recorded.
+- `details`: for a refused model request, the budget, the amount spent, the
+  holds of requests in flight, the refused hold, the model and the turn. For a
+  refusal at dispatch, the cap, the amount spent, the amount held and
+  `heldRuns`, the number of runs holding budget. A refused model request also
+  carries `refusedHoldBasis` (`cached` or `full`) and `refusedHoldCeilingUsd`
+  (the worst-case hold for that request). Both also carry
+  `groupName` (the agent's budget group, when it has one) and, for a
+  `run-tree` limit, `leadAgentName`. A refused model request also carries
+  `agentBudgetUsd`, the agent's own `budgetUsd`. An `overrun_unavailable`
+  event carries the full cap or tree's `capUsd`, `spentUsd`, `heldUsd` and
+  `leftUsd`, the run's `baseBudgetUsd`, `grantedUsd` and `limitUsd`, and
+  `groupName`, the group whose cap was full (which can be the lead's).
+
+The run's error, and a lead's sub-agent result (in its `budget` field), carry
+the same facts as one sentence. Pull request and issue comments and check
+summaries show the amounts and the budget group's name only.
+
+## Holds vs actual spend
+
+A hold is not a charge. The run is charged the reply's real cost, and the hold
+is released when the reply is settled. A hold is larger than a typical reply
+because it must cover the whole output allowance (`maxOutputTokens`), and
+because the first request of a conversation, and any fallback, assumes the whole
+body is new input. Fallbacks apply when there is no previous reply in the
+conversation, the model changed, the previous reply was interrupted or mostly
+uncached, the previous request in the conversation was sent more than 5 minutes
+earlier, or the body got smaller.
+
+If the cache misses, a reply can cost more than its hold. It is still settled
+when it costs no more than its worst case, so a run can exceed its budget by at
+most the difference between a request's worst case and its hold, for each
+request in flight.
+
+Read `refusedHoldBasis`: `full` means the hold was a fallback, so the next
+request in a warm conversation would likely hold less; `cached` means it
+already used the cache-read rate, so the budget is simply low for the request.
+`refusedHoldCeilingUsd` is the most the request could ever hold.
+
+## What to do
+
+- `run`: raise `budgetUsd` on the agent, set `allowedOverrunPercent` (see
+  below), or lower `codingProfile.maxOutputTokens`
+  so each request holds less. If the sentence says the allowed overrun wasn't
+  available, raising the overrun will not help: raise the full group's cap, or
+  the lead's `budgetUsd` for a run tree, as the sentence says.
+- `run-tree`: raise `budgetUsd` on the lead agent; its sub-agents draw on it.
+- `day`, `week`, `month`: raise the budget group's cap for that period, or wait
+  for the period to reset. See [Budget group exhausted](budget-group-exhausted.md).
+
+A refused run is not resumed automatically; trigger a new one.
+
+## Allowed overrun
+
+A coding agent's `allowedOverrunPercent` (integer 0 to 100, default 0; set with
+`create_agent` or `update_agent`, same permission as `budgetUsd`) lets a run
+spend up to that percent beyond `budgetUsd`. On a lead agent that delegates,
+it raises its run tree's ceiling for coding sub-agents instead; native runs' own
+model calls are not affected. A delegated run tree's ceiling uses the lead
+agent's percent.
+
+The overrun is permission, not a reservation. A run reserves only its base
+budget (`budgetUsd`, trimmed to the room its budget group and run tree have).
+While it works, it is topped up in steps toward its limit (`budgetUsd` raised
+by the percent), each only from room the group and run tree still have, so
+group caps and run-tree ceilings stay hard. `get_run` shows `budgetBaseUsd`
+(before the overrun), `budgetLimitUsd` (the limit) and `budgetReservedUsd`
+(what has been granted so far). The sentence names what the run had, for
+example "this run's $1.00 budget and its 10% allowed overrun ($1.10 in all)",
+or "$1.00 budget and $0.05 of its 10% allowed overrun ($1.05 in all)".
+
+### Overrun not available
+
+When the group or run tree has no room left for the next top-up, `get_run`
+lists an `overrun_unavailable` event (its `limit` names the full cap or
+`run-tree`), and the run asks for no more, even if room frees up later. The
+proxy then stops it at what it was granted, and the sentence says why, for
+example "Only $0.40 of its 10% allowed overrun was available because the
+"team" group had no more left of its daily cap.", or "Its 10% allowed overrun
+wasn't available because the lead's run tree had nothing left of its $15.00
+budget." when it got none. A top-up takes whatever room is left, so the run is
+refused only once nothing is. A run whose base is smaller than its first model
+request cannot use its overrun: that request is refused before any top-up.
+To continue, raise that cap (or the lead's `budgetUsd`) and continue the draft
+pull request. Runs dispatched before this behaviour have no `budgetLimitUsd`:
+their reservation already included the overrun, and they finish as before.
+
+A worker that finishes is delivered even when it ended over the limit: the run
+is `succeeded` and its status comment says, for example, "Spent $1.04, $0.04
+(4%) over its $1.00 budget, within its allowed overrun."
+
+## Keeping the work of a stopped run
+
+When a run is stopped for budget and its worker ended cleanly, its changes so
+far are pushed as the usual draft pull request (or added to the one it was
+continuing). A new pull request's description starts "Stopped at its budget
+before finishing; the changes so far are in this draft." A continuation leaves
+the existing description alone. The run stays `budget_exhausted` and
+keeps its `budgetEvents`. Mentioning the agent on that pull request (a
+comment mentioning the App), or on the issue it came from (the App on a
+GitHub issue, the service account on a Jira issue), continues the work with
+a fresh budget. The pull
+request is not marked ready automatically until a later continuation finishes.
+
+A new pull request's description has a `### Task` section quoting the request
+that started the run (capped at 4 KiB, mentions defused), so a continuation
+knows what the draft was for. The request text is visible to anyone who can
+read the pull request. While the draft's latest commit is the stopped
+run's push, the automatic review is skipped with a neutral check, "Skipped:
+stopped at its budget", and no fix round starts. Marking the pull request ready
+or pushing a new commit gets a normal review. See
+[Run GitHub code-review agents](../code-review-agents.md#drafts-stopped-at-their-budget).
+
+If a push check refuses (protected paths, diff limits, access) or there are no
+changes, the run ends as above with nothing pushed. So does a continuation of a
+pull request that is no longer a draft (marked ready for review): unfinished
+work is never pushed onto it. A worker that crashed, timed out or was stopped
+is not kept.
+
+## Interrupted replies
+
+When a reply breaks off before its final usage, the proxy settles the request
+at an estimate instead of keeping its whole hold: the usage it already read,
+else the input of the same conversation's previous reply on the same model plus
+the body's growth since then, else the whole body as new input, plus the output
+relayed so far (text, tool-call input, and thinking or reasoning text) counted
+high, and never more than the request's worst case. With nothing relayed, the
+run's median completed reply on that model is used, or no output if there is
+none yet. The previous reply is used only if it started within the 5-minute
+prompt cache window before the request and the body has not shrunk since. New
+input is counted at one token per 3 bytes and priced at the higher of the
+model's input and cache-write rates. A streaming request that failed before the
+provider's response started books no output. The native sandbox gateway follows
+the same rule: a call cut after the response started but before any output (for
+example while the model was thinking) is booked at its input plus the run's
+median completed reply, or its input alone if there is none yet. The request is
+flagged as estimated and counted in the run's cost. If the estimate cannot be
+computed or recorded, the request keeps its full hold. A run can exceed its budget by at
+most the gap between the estimate and the real cost.
+
+## Group holds for delegated runs
+
+A lead and the sub-agents it starts in the same budget group hold the lead's
+budget once, at the lead (its reservation minus what the whole tree has spent),
+decided per period. If the tree's live sub-agent runs hold more than that
+(a coding run can be topped up past the lead's budget by the lead's
+`allowedOverrunPercent`), the lead holds their unspent holds instead. `get_budget_group` reports `heldRuns` for each period.
+
+See also [Troubleshoot budgets and reservations](../troubleshooting/budgets.md).
+
+A coding run refused by the model provider itself because its account is out of
+credit or over a spend limit is a different failure; see
+[errors/coding-stream-quota-exceeded](coding-stream-quota-exceeded.md).

package/help/errors/budget-group-exhausted.md

@@ -14,7 +14,9 @@ has no capacity left for that period after accounting for actual spending and
 active reservations. Wardby refuses the run before model work or a coding
 worker begins.
 
-1. Inspect the budget group's `spentUsd` and `reservedUsd` values.
+1. Inspect the budget group's `spentUsd`, `reservedUsd` and `heldRuns` values.
+   The refused run's `get_run` output lists the cap, spend and holds in
+   `budgetEvents`.
 2. Inspect active and recent runs that use the group.
 3. Wait for the period to reset, cancel unneeded active work, or deliberately
    adjust the agent or group budget.
@@ -23,3 +25,5 @@ worker begins.
 
 Do not raise a limit merely to clear a refusal without confirming the intended
 owner, schedule, and overlapping work. See [Budget troubleshooting](../troubleshooting/budgets.md).
+
+For a coding run stopped mid-run, see the help article `errors/budget-exhausted`.

package/help/errors/coding-stream-quota-exceeded.md

@@ -0,0 +1,33 @@
+---
+id: errors/coding-stream-quota-exceeded
+title: Coding provider out of credit or over its spend limit
+summary: A coding run's model provider refused it because the account is out of credit or over its spend limit; retrying does not help until the limit is raised.
+audience: operator
+tags: [error, coding-agents, billing, quota, spend-limit, coding_stream_quota_exceeded]
+appliesTo: ">=0.7.0"
+---
+
+# Coding provider out of credit or over its spend limit
+
+`job_coding_stream_quota_exceeded` in a coding run's diagnostic (the
+control-plane log line under the run's `coding_diag_…` id) means the model
+provider refused the run's requests because the provider account behind the
+coding proxy has run out of credit or reached a spend limit. OpenAI reports
+this as `insufficient_quota` or a project spend limit (with HTTP 429, like a
+rate limit); Anthropic as a credit balance that is too low. It is not wardby's
+own run budget: that ends the run as `budget_exhausted` (see
+[errors/budget-exhausted](budget-exhausted.md)).
+
+Retrying does not help: every coding run on that provider account fails the
+same way until the limit changes.
+
+1. Check the provider account the coding proxy uses: the OpenAI project's
+   billing and limits, or the Anthropic organization's credit balance and spend
+   limits.
+2. Add credit or raise the limit there.
+3. Trigger the run again.
+
+The coding proxy's log for the same request (`audit.*` events) shows the
+provider's status. See
+[Coding worker isolation](../../docs/coding-worker-isolation.md) for where the
+diagnostic and proxy logs live.

package/help/errors/coding-workspace-disk-exceeds-limit.md

@@ -0,0 +1,39 @@
+---
+id: errors/coding-workspace-disk-exceeds-limit
+title: Coding workspace larger than the deployment allows
+summary: create_agent and update_agent refuse workspaceDiskMb above CODING_MAX_DISK_MB; already-saved agents above the cap fail at run start.
+audience: operator
+tags: [error, coding-agents, workspace, disk, workspaceDiskMb, CODING_MAX_DISK_MB, create_agent, update_agent]
+appliesTo: ">=0.2.0"
+---
+
+# Coding workspace larger than the deployment allows
+
+`coding_workspace_disk_exceeds_limit` (on the run record as
+`coding_failure_workspace:<id>`) means the agent's
+`codingProfile.workspaceDiskMb` is larger than the deployment's
+`CODING_MAX_DISK_MB`. The run fails before a worker starts and costs nothing.
+When `CODING_MAX_DISK_MB` is unset, the limit is `CODING_DISK_MB` (2048 MiB by
+default).
+
+`create_agent` also checks this limit at save time. `update_agent` checks it
+when setting a numeric `workspaceDiskMb` or changing an agent's kind to coding.
+You can still make unrelated edits to an already-saved over-cap coding agent,
+lower its size to within the cap, or set it to `null`.
+The error names the requested size in MiB,
+the effective cap in MiB, and `CODING_MAX_DISK_MB`. A size exactly at the cap
+is allowed; `null` uses the deployment default. Agents already saved above
+the cap still fail at run time, including if the operator lowers the cap.
+
+Fix it one of two ways:
+
+1. Lower the agent's size to the limit or below with `update_agent`:
+   `{ "id": "<agent id>", "codingProfile": { "workspaceDiskMb": 2048 } }`.
+2. Or raise the deployment's `CODING_MAX_DISK_MB` on the control plane and
+   restart it. On GKE, deploy with `CODING_MAX_DISK_MB=8192 deploy/gke/up.sh`
+   (see the GKE deploy guide). On GKE Autopilot the value must leave room for the
+   worker's 1 GiB inside the 10 GiB pod ephemeral-storage ceiling.
+
+A run that installs more than its workspace holds is evicted mid-run instead;
+that needs a larger `workspaceDiskMb`, within this limit. See
+[Build a worker image](../build-worker-image.md).

package/help/errors/continuation-closed.md

@@ -19,15 +19,13 @@ an open pull request carrying that run's marker for the branch. Nothing from
 the run was pushed either way, so the pull request (and anyone who merged or
 closed it) is unaffected.
 
-Two things besides an actual merge or close can also make the check come back
-"no longer found," since it looks for an **open** pull request with that
-exact base branch and the run's hidden marker still in its description:
-
-- the pull request's base branch was retargeted to something other than the
-  one the run recorded;
-- the hidden `<!-- wardby:<run-id> -->` marker was removed or edited out of
-  the pull request's description (wardby relies on it to find its own PR;
-  nothing else identifies it).
+One thing besides an actual merge or close can also make the check come back
+"no longer found," since it looks for an **open** pull request from the run's
+branch with the run's hidden marker still in its description: the hidden
+`<!-- wardby:<run-id> -->` marker was removed or edited out of the pull
+request's description (wardby relies on it to find its own PR; nothing else
+identifies it). Retargeting the pull request to another base branch does not
+close it for wardby: the follow-up continues it and merges the new base.
 
 When the pull request was merged or closed **before the run started**, the
 check before cloning catches it: the run's status is `refused`, and nothing
@@ -66,9 +64,8 @@ sub-run instead, without the error code:
 If the change is still needed, ask again without `continuePriorRun` (or
 without naming the closed pull request). The new run opens a fresh pull
 request from the repository's default branch. If the check is wrongly
-finding nothing when the pull request is in fact open, check that its base
-branch still matches what wardby recorded and that the `<!-- wardby:... -->`
-marker is still in its description, unedited.
+finding nothing when the pull request is in fact open, check that the
+`<!-- wardby:... -->` marker is still in its description, unedited.
 
 Related: [Run GitHub code-review agents](../code-review-agents.md),
 [Automatic review fix rounds](../review-fix-rounds.md),

package/help/errors/delegation-conflict.md

@@ -0,0 +1,24 @@
+---
+id: errors/delegation-conflict
+title: Delegation conflict
+summary: A delegate_to_* call returned delegation_conflict because its call id is already used by a run of another sub-agent.
+audience: operator
+tags: [error, delegation, sub-agents, runs, delegation_conflict]
+appliesTo: ">=0.7.0"
+---
+
+# Delegation conflict
+
+`delegation_conflict` is the error a `delegate_to_*` call returns to the model when wardby could not create the sub-agent run: each sub-agent run records the parent's tool-call id, and that id is already used by a run of a different sub-agent of the same parent.
+
+It can happen when two executions of the same delegate step overlap (for example, a stalled process and the process that took the run over). When the other execution already started this sub-agent, wardby waits on that run instead and no error is returned. Only when the existing run belongs to a different sub-agent does the call return `delegation_conflict`.
+
+No new sub-agent run is started, and the parent run is not failed: the model gets the error as the tool result ("The sub-agent run could not be created: its call id is already used by another sub-agent run.") and can report it or try again.
+
+## What to do
+
+1. List the runs of the parent's sub-agents with `list_runs` (by each sub-agent's `agentId`) and look for those whose `parentRunId` is the parent run.
+2. Check the server logs for `delegation child create hit a unique key held by no child of this agent`, which names the run and the sub-agent.
+3. If the model did not delegate again on its own, trigger the request again.
+
+See [Run lost](run-lost.md) for how runs continue after a restart, and [Native capabilities](../native-capabilities.md) for sub-agents.

package/help/errors/file-list-truncated.md

@@ -0,0 +1,45 @@
+---
+id: errors/file-list-truncated
+title: APPROVE refused on a cut file list (file_list_truncated)
+summary: The pull request has more changed files than repo_pr_read lists, so repo_publish_review refuses the agent's APPROVE; it can still request changes or comment.
+audience: operator
+tags: [error, code-review, repo_publish_review, repo_pr_read, file_list_truncated, fileListComplete, large-pull-request]
+appliesTo: ">=0.7.0"
+---
+
+# APPROVE refused on a cut file list
+
+`repo_publish_review` returns the tool error `file_list_truncated` when a
+review agent publishes `APPROVE` on a pull request head after a
+`repo_pr_read` of that head in the run reported `fileListComplete: false`,
+and no complete read of the whole pull request (without `sinceSha`) has
+followed. The host lists at most
+1,000 changed files of a pull request (a GitHub compare, used when the agent
+passes `sinceSha`, lists at most 300). Files past that cut never reach the
+agent, so an approval would cover code nobody reviewed.
+
+Nothing is published and nothing else changes. `CHANGES_REQUESTED` and
+`COMMENT` are still allowed on the same head.
+
+## What the agent should do
+
+- If it passed `sinceSha`, call `repo_pr_read` again without it. When the
+  whole pull request fits in the list, `fileListComplete` becomes `true` and
+  `APPROVE` is allowed again (once every listed source, test or config file
+  is read; see `unread_files`). A complete `sinceSha` read does not lift the
+  refusal.
+- Otherwise publish `CHANGES_REQUESTED` on what it read, and say the pull
+  request is too large to review in full. That review does not cover the
+  head, so a later run cannot approve it through a `sinceSha` delta either
+  (see `full_read_required`). `COMMENT` completes the check as
+  neutral, which branch protection treats as passing.
+
+## What the owner does
+
+Split the change into smaller pull requests, each under the limit, or have
+a person review it. The limit is the same for GitHub and local repositories.
+
+## Related
+
+- [Run GitHub code-review agents](../code-review-agents.md)
+- [Large pull requests](../../docs/code-review-agents.md#large-pull-requests)

package/help/errors/full-read-required.md

@@ -0,0 +1,49 @@
+---
+id: errors/full-read-required
+title: APPROVE refused after a partial compare (full_read_required)
+summary: The review agent compared the pull request from a commit wardby did not vouch for, so it never saw the full change; it must read the whole pull request before approving.
+audience: operator
+tags: [error, code-review, repo_publish_review, repo_pr_read, full_read_required, sinceSha, lastCoveredSha]
+appliesTo: ">=0.7.0"
+---
+
+# APPROVE refused after a partial compare
+
+`repo_publish_review` returns the tool error `full_read_required` when a
+review agent publishes `APPROVE` on a pull request head and, in that run:
+
+- no `repo_pr_read` of the head was made without `sinceSha`, and
+- a `repo_pr_read` of the head passed a `sinceSha` other than the
+  `lastCoveredSha` that same read returned.
+
+A read with `sinceSha` lists only the files changed since that commit. Any
+earlier commit of the branch is accepted as `sinceSha`, so a compare from a
+commit wardby did not record as covered hides every change made before it.
+`lastCoveredSha` is the latest head this agent reviewed after reading every
+file. Any other commit, such as one named in the pull request's text, or a
+`lastReviewedSha` whose review did not read everything, is not a safe base. An approval would
+then cover code nobody reviewed.
+
+Nothing is published and nothing else changes. `CHANGES_REQUESTED` and
+`COMMENT` are still allowed on the same head.
+
+## What the agent should do
+
+Call `repo_pr_read` without `sinceSha`, read the source, test and config
+files it lists, and publish again. Or, for a re-review, pass the
+`lastCoveredSha` the read returns as `sinceSha`. If neither is possible,
+publish `CHANGES_REQUESTED` on what it read.
+
+## What the owner does
+
+Nothing, normally: the refusal tells the agent what to do. If it recurs, check
+the reviewer's system prompt: it should pass only the `lastCoveredSha` that
+`repo_pr_read` returned as `sinceSha`, never `lastReviewedSha` or a commit
+named in the pull request's text. Reviewer prompts written before 0.7.0 pass
+`lastReviewedSha`; update the re-review step from the shipped reviewer prompt.
+
+## Related
+
+- [Run GitHub code-review agents](../code-review-agents.md)
+- [APPROVE refused on a cut file list: `file_list_truncated`](file-list-truncated.md)
+- [Large pull requests](../../docs/code-review-agents.md#large-pull-requests)

package/help/errors/github-account-not-allowed.md

@@ -19,6 +19,10 @@ existing link to it are refused.
 
 - **The repository should be served:** an operator adds its owner's login to
   `WARDBY_GITHUB_ALLOWED_ACCOUNTS` and restarts the server, then links it again.
+- **Only sandboxed runs are refused** (the same link works for a
+  `control-plane` agent): the native sandbox gateway runs a sandboxed agent's
+  repository tools and needs the same `WARDBY_GITHUB_ALLOWED_ACCOUNTS` as the
+  control plane. Set it on both, then restart the gateway.
 - **It should not:** use a repository owned by a served account, or another
   wardby server with its own GitHub App.