@microck/raft 0.3.0 → 0.5.0

package/docs/reference/capacity.mdx

@@ -0,0 +1,198 @@
+---
+title: Box sizing and capacity
+description: Size boxes for the task, give them their own disks, and read the advisory host capacity estimate.
+---
+
+# Box sizing and capacity
+
+<Callout type="info" title="Advisory estimate only">
+Raft enforces no box count or size budget. Create the boxes your task needs, sized for the work. `raft capacity` is planning advice, never a reason to skip a box.
+</Callout>
+
+## Sizing a box
+
+Choose CPU, RAM and disk when you run `raft new`:
+
+- `--cpu N`: any whole number of CPUs, from `1` up to the host's CPU count (default: `1`). Boxes share host CPUs, so a limit is a ceiling, not a reservation.
+- `--memory SIZE`: any whole number of GiB, such as `3GiB` or `32GiB`, up to the host's RAM (default: `2GiB`). This is the box's cgroup memory limit.
+- `--disk SIZE`: give the box its own disk instead of the shared pool, on hosts that enable own box disks. See [Own box disks](#own-box-disks).
+
+A box from the image starts from these defaults, or from the controller's
+[`config.json` defaults](../guides/config-policy.mdx#defaults-for-new) when set.
+A box from a snapshot (`new --from`) or a fork keeps the source's CPU, RAM and
+disk instead of any default, and each flag you pass overrides only its own value.
+
+```sh
+box=$(raft new --cpu 4 --memory 16GiB --disk 40GiB --ttl 3600 --purpose "Large build")
+```
+
+A CPU or RAM request larger than the whole host fails before anything is
+created, because Incus would accept it and quietly give the box less:
+
+```text
+Requested 64 CPUs, but this host has 8
+```
+
+Change CPU and RAM when you resume a stopped box or fork one:
+
+```sh
+raft resume "$box" --ttl 1800 --cpu 2 --memory 4GiB
+copy=$(raft fork "$box" --ttl 1800 --memory 2GiB)
+```
+
+Resume requires a stopped box; fork requires a stopped source. Omitted values
+stay unchanged on resume and inherit the source limits on fork. Live resizing is
+not exposed by the CLI. See [Snapshots and forks](../guides/snapshots-forks.mdx).
+
+## Own box disks
+
+By default every box stores its files in the shared 60 GiB Btrfs pool
+`raft-data`. Boxes there grow until the pool is full, and Btrfs quotas cannot
+hold them back: Docker and other tools create nested subvolumes that escape
+Btrfs quotas.
+
+`raft new --disk SIZE` gives the box its own disk instead: an LVM thin volume
+of SIZE with an ext4 filesystem in the `raft-disks` pool. Everything the box
+writes, including Docker images, layers and container files, lands in that
+filesystem, so the box gets "No space left on device" at SIZE and cannot touch
+other boxes' space. Raft reserves the whole SIZE in the pool when the box is
+created, so other boxes cannot take it either.
+
+Own box disks are opt-in per host. A host deployed without `--disk-pool` has no
+`raft-disks` pool, which is a normal, healthy state: `raft doctor` and
+`raft capacity` say own box disks are not enabled, and `new --disk` fails before
+creating anything with the command that enables them.
+
+- **Size**: whole GiB, such as `20GiB`. The development image holds about 5 GiB of files, so give boxes at least `8GiB`; a disk too small for the image fails at creation and leaves nothing behind. Disks of 10 GiB or more start fastest: smaller ones are shrunk after the copy. ext4 keeps a little of the volume for itself, so `df` shows slightly less than SIZE.
+- **Copy-on-write**: a new own-disk box starts as a thin snapshot of the image, and forks, templates (`new --from`) and snapshots of own-disk boxes are thin snapshots of their source. They are created in seconds and share the source's blocks until they write their own.
+- **Reservations**: each own disk, and each snapshot of one, reserves its full size in the pool, even though it shares blocks. A request that would reserve more than the pool holds fails before anything is created. See [Reservation accounting](#reservation-accounting).
+- **Forks, templates and backups**: `fork` and `new --from` keep the source disk, shared or own, with the same size. `new --from BOX/SNAPSHOT --disk SIZE` copies a snapshot onto an own disk of a new size. `recover` puts an own-disk backup back onto an own disk of the same size, and needs the pool on the destination. Resume never changes the disk.
+- **Inspection**: `raft list` reports `disk` as `"shared"` or the size; `raft info` shows it in `expanded_devices.root`; `raft usage` reports the filesystem's space with a `filesystemScope` of `own SIZE box disk`.
+
+### Enabling the `raft-disks` pool
+
+Pass `--disk-pool SIZE` to `raft host add`, or to
+`python3 deploy/deploy.py --location LOCATION` on a host that is already
+deployed. The deployer creates `raft-disks` beside `raft-data`: a loop-backed
+LVM thin pool (`lvm.use_thinpool=true`) of SIZE. It allocates the pool's file on
+the host disk in full, so every reservation is backed by real host space, and
+it sets the thin pool to `--discards nopassdown`, so freed blocks return to the
+pool without punching holes back into that file. A fresh host then needs 80 GiB
+plus SIZE free; adding the pool later needs SIZE plus 20 GiB free.
+
+Redeploying never resizes the pool: an explicit `--disk-pool` that differs from
+the existing size is refused, and omitting it keeps the pool as it is. To change
+the size, back up the own-disk boxes, delete them, remove the pool explicitly
+with Incus and redeploy.
+
+### Reservation accounting
+
+A thin pool can promise its volumes more space than it holds, and when it fills
+up every volume in it fails at once. Raft never lets that happen. A thin volume
+can never hold more data than its own size, so as long as the sizes of all thin
+volumes in the pool add up to no more than the pool's data space, the pool
+cannot fill, however much the boxes write.
+
+Under the host lock, before Incus adds a thin volume to `raft-disks`, Raft adds
+up the size of every thin volume already there, as `lvs` reports it, and refuses
+the request if the new volumes would not fit. The sum covers everything in the
+pool: own-disk boxes, their snapshots, the cached image and anything an
+operator created. Each operation adds:
+
+| Operation | Adds |
+| --- | --- |
+| `new --disk SIZE` | The larger of SIZE and the cached image volume, plus the image volume itself the first time a box on this host needs it |
+| `new --from BOX/SNAPSHOT` of an own-disk snapshot | The larger of the snapshot's size and `--disk` |
+| `new --from BOX/SNAPSHOT --disk SIZE` of a shared-pool snapshot | SIZE |
+| `fork` of an own-disk box | The box's size |
+| `snapshot` of an own-disk box | The box's size |
+| `recover` of an own-disk backup | The box's size once for the box and once per snapshot in the backup |
+
+`restore` adds nothing: it swaps in a thin snapshot of the snapshot and removes
+the old volume. Destroying a box or deleting a snapshot frees its reservation.
+Incus caches the image in a 10 GiB volume (the pool's `volume.size` when set),
+which stays reserved while the image is in use; plan the pool size as 10 GiB
+plus the own disks and snapshots you want at once. The pool's data space is a
+little below SIZE, because LVM keeps part of the file for thin pool metadata.
+
+LVM sizes that metadata at 64 bytes per data block, above the 48 bytes per
+block the kernel documentation recommends for a fully mapped pool, and
+reservations keep the number of mapped blocks below the pool's block count. If
+metadata ever filled, every own-disk box would stop and the pool would need
+offline repair, so `raft capacity` warns when it passes 80%.
+
+## Inspecting capacity with `raft capacity`
+
+```sh title="CONTROLLER: View the advisory capacity estimate"
+raft capacity
+
+# Output JSON for programmatic inspection
+raft capacity --location lab --json
+```
+
+```text
+Advisory estimate only. Raft enforces no box count or size budget; create the boxes your task needs.
+lab: 8 CPUs, 31.3 GiB RAM (27.9 GiB available)
+Boxes: 3 saved, 2 active
+Allocated to active boxes: 3 CPUs, 6.0 GiB; host reserve: 1 CPU, 3.1 GiB; headroom: 4 CPUs, 22.2 GiB
+Disk free: 41.5 of 60.0 GiB shared pool, 48.2 GiB host
+Own box disks: 19.9 of 39.9 GiB unreserved
+CPU  RAM GiB  Fit on empty host  Fit now
+  1        2                  7        4
+  2        4                  3        2
+  4        8                  1        1
+Counts assume every box uses its full CPU and RAM at once. Boxes share host CPUs, so more can run when they are not all busy. Any whole CPU count and GiB size works.
+```
+
+The table lists example shapes that double from the 1 CPU / 2 GiB default at
+2 GiB per CPU, up to the largest that fits the host. For any other size, divide
+`headroom` by the box's CPU and RAM.
+
+### JSON fields
+
+| Field | Meaning |
+| --- | --- |
+| `notice` | The advisory sentence above |
+| `host` | CPUs, total and available RAM, free host disk, `sharedPool` space, and `boxDiskPool` |
+| `host.boxDiskPool` | `null` when own box disks are not enabled. Otherwise `totalBytes` (thin pool data space), `reservedBytes` (sizes of all its thin volumes), `usedBytes` (data written) and `metadataUsedPercent`; the last two are `null` while no own-disk volume is active. New own disks fit in `totalBytes - reservedBytes` |
+| `boxes`, `activeBoxes` | Saved boxes, and those not `Stopped` |
+| `allocated` | CPU and RAM limits of active boxes |
+| `reserve` | Host share left out of the estimates |
+| `headroom` | CPU and RAM left for more active boxes after the reserve |
+| `estimates` | Example shapes with `fitEmptyHost` and `fitNow` counts |
+| `warnings` | Low RAM or disk, overcommitted boxes, or thin pool metadata above 80% |
+
+### Calculations
+
+| Value | Calculation |
+| --- | --- |
+| CPU reserve | 1 CPU for the host OS |
+| Memory reserve | The larger of 2 GiB or 10% of host RAM |
+| Fit on empty host | `min(floor((CPUs - 1) / box CPUs), floor((RAM - reserve) / box RAM))` |
+| Fit now | The same with `headroom`, which also subtracts active allocations and keeps `MemAvailable - reserve` |
+
+All non-`Stopped` states, such as `Running` or `Frozen`, count as active.
+Stopped boxes use no CPU or RAM and keep their disk. Reserves are planning
+margins, not cgroup locks.
+
+### Illustrative hosts
+
+Fit on empty host for 1 CPU / 2 GiB boxes, using exact usable RAM values:
+
+| Host CPUs | Host RAM | Fit on empty host |
+| ---: | ---: | ---: |
+| 2 | 8 GiB | 1 |
+| 4 | 8 GiB | 3 |
+| 4 | 16 GiB | 3 |
+| 8 | 32 GiB | 7 |
+
+Operating systems report less usable RAM than the hardware size, so near
+boundaries the estimate can be one lower. Available RAM and host processes can
+lower it further. These are point-in-time estimates, not reservations.
+
+## Disk warnings
+
+`raft capacity` warns when free host disk or the shared pool falls below 5 GiB,
+and when `raft-disks` thin pool metadata passes 80%. Warnings do not block
+anything. `raft doctor` reports the shared pool and, when own box disks are
+enabled, `raft-disks`; a host without `raft-disks` is healthy.

package/docs/reference/cli.md

@@ -14,6 +14,12 @@ Syntax:
 and `raft --version` prints the installed version without loading configuration.
 Output is command-specific; see [Scripting and CI](../guides/scripting-ci.mdx).
 
+At most once a day, after a command finishes, Raft checks GitHub for a newer
+release and prints one line on stderr, such as
+`raft 0.6.0 is available (you have 0.5.0). Run: raft self-update`. It never
+installs anything or changes stdout, and network failures stay silent. See
+[`self-update`](#self-update) to turn it off.
+
 Workspaces are referenced by location-qualified handles formatted as `<location>:<instance-name>` (for example `lab:rf-a1b2c3d4e5f60718`).
 
 ## Command index
@@ -28,7 +34,9 @@ Workspaces are referenced by location-qualified handles formatted as `<location>
 | Port forwarding and desktop | [`forward`](#forward), [`desktop`](#desktop) |
 | Backup and recovery | [`backup`](#backup), [`recover`](#recover) |
 | Locations | [`host add`](#host-add), [`locations`](#locations), [`note`](#note) |
-| Host inspection and maintenance | [`limits`](#limits), [`doctor`](#doctor), [`list`](#list), [`gc`](#gc), [`prune`](#prune) |
+| Defaults and policies | [`config`](#config), [`policy`](#policy) |
+| Host inspection and maintenance | [`capacity`](#capacity), [`doctor`](#doctor), [`list`](#list), [`gc`](#gc), [`prune`](#prune) |
+| CLI maintenance | [`self-update`](#self-update) |
 
 ## Workspace lifecycle commands
 
@@ -36,18 +44,28 @@ Workspaces are referenced by location-qualified handles formatted as `<location>
 
 Create and launch a new workspace container.
 
-`raft new [--location LOCATION] [--ttl TTL] [--cpu {1,2}] [--memory {1GiB,2GiB,4GiB,8GiB}] [--disposable] [--from LOCATION:BOX/SNAPSHOT]`
+`raft new [--location LOCATION] [--ttl TTL] [--cpu N] [--memory SIZE] [--disk SIZE] [--disposable] [--from LOCATION:BOX/SNAPSHOT] [--purpose TEXT [--owner TEXT] [--next TEXT]]`
+
+- `--location LOCATION`: Target host location in `~/.config/raft/incus.json`. Built-in default: first configured location.
+- `--ttl TTL`: Lifetime in seconds before automatic stop (`60` to `2592000`). Built-in default: `600`.
+- `--cpu N`: CPU limit, any whole number from `1` up to the host's CPU count. Built-in default: `1`, or the snapshot's with `--from`.
+- `--memory SIZE`: Memory limit in whole GiB, for example `16GiB`, up to the host's RAM. Built-in default: `2GiB`, or the snapshot's with `--from`.
+- `--disk SIZE`: Give the box its own disk of SIZE in whole GiB, for example `20GiB`, instead of the shared pool. Needs a host deployed with `--disk-pool`. The box, and Docker inside it, cannot write past SIZE, and the pool reserves SIZE when the box is created. `--disk shared` asks for the shared pool, overriding a configured `disk`. Built-in default: the shared pool, or with `--from` the snapshot's disk. See [Box sizing and own disks](./capacity.mdx#own-box-disks).
 
-- `--location LOCATION`: Target host location in `~/.config/raft/incus.json`. Defaults to first configured location.
-- `--ttl TTL`: Lifetime in seconds before automatic stop (`60` to `2592000`). Default: `600`.
-- `--cpu {1,2}`: CPU core count managed by Incus. Default: `1`.
-- `--memory {1GiB,2GiB,4GiB,8GiB}`: Memory limit. Default: `2GiB`.
+An omitted option takes the controller default set with [`config`](#config),
+then the built-in default. With `--from`, configured `cpu`, `memory` and `disk`
+defaults do not apply: the box keeps the snapshot's size unless a flag overrides
+it, and `--disk shared` is refused.
 
 - `--disposable`: Delete the box and snapshots on stop or expiry. Off by default.
-- `--from LOCATION:BOX/SNAPSHOT`: Copy a named snapshot on its source host instead of the configured image. Normal creation sizing applies; the source can be running. Cannot target a different location.
+- `--from LOCATION:BOX/SNAPSHOT`: Copy a named snapshot on its source host instead of the configured image. The box keeps the snapshot's CPU, RAM and disk unless `--cpu`, `--memory` or `--disk` overrides them; the source can be running. Cannot target a different location.
+- `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Record a [box note](#note). `--owner` and `--next` need `--purpose`. Without them the box starts with no note, including boxes created `--from` a noted snapshot. A host [policy](#policy) can require fields.
 
 Returns the qualified workspace handle. If the location has a note, `new` prints
-it to stderr; stdout still contains only the handle.
+it to stderr; stdout still contains only the handle. Raft limits neither the
+number of boxes nor their size; requests larger than the whole host, own disks
+on a host without `raft-disks`, or own disks larger than the unreserved
+`raft-disks` space, fail with the reason before anything is created.
 
 ### `stop`
 
@@ -61,12 +79,13 @@ Terminates guest processes and stops the container. Persistent boxes retain file
 
 Start a stopped workspace with a renewed lifetime deadline.
 
-`raft resume <box> --ttl TTL [--cpu {1,2}] [--memory {1GiB,2GiB,4GiB,8GiB}]`
+`raft resume <box> --ttl TTL [--cpu N] [--memory SIZE]`
 
 - `--ttl TTL`: Lifetime in seconds from resumption (`60` to `2592000`). Required.
 
 CPU/RAM overrides on resume apply before startup. Omitted values stay unchanged.
-A running box is rejected before changing its limits.
+A running box is rejected before changing its limits. Resume never changes the
+box's disk.
 
 ### `extend`
 
@@ -82,7 +101,7 @@ Permanently delete a workspace container and its snapshots.
 
 `raft destroy <box>`
 
-Stops the container if running, removes all associated Btrfs snapshots, and releases host storage.
+Stops the container if running, removes all associated snapshots, and releases its storage, including an own disk.
 
 ## Execution and inspection commands
 
@@ -108,13 +127,13 @@ Open an interactive root terminal session inside the workspace over host SSH and
 
 ### `info`
 
-Display container metadata from Incus in JSON format, including status, IP addresses, resource limits, and timestamps.
+Display container metadata from Incus in JSON format, including status, IP addresses, resource limits, and timestamps. `expanded_devices.root` shows the disk: `pool: raft-data` for the shared pool, or `pool: raft-disks` with the own disk's `size`.
 
 `raft info <box>`
 
 ### `usage`
 
-Display active resource consumption for a running workspace: cgroup memory, CPU microseconds, CPU affinity, and shared-pool disk space.
+Display active resource consumption for a running workspace: cgroup memory, CPU microseconds, CPU affinity, and filesystem space. `filesystemScope` says whether the space is the shared pool or the box's own disk.
 
 `raft usage <box>`
 
@@ -174,6 +193,8 @@ Create a named point-in-time snapshot of a stopped workspace. Names use alphanum
 
 `raft snapshot <box> <name>`
 
+A snapshot of a box with its own disk is a thin snapshot that reserves the box's disk size in `raft-disks`; it fails before anything is created when the pool lacks unreserved space.
+
 ### `snapshots`
 
 List existing snapshots and creation timestamps for a workspace.
@@ -196,11 +217,13 @@ Roll back a stopped workspace filesystem to a named snapshot using `--diskonly`,
 
 Clone a stopped workspace into an independent container on the same host and start it.
 
-`raft fork <box> --ttl TTL [--cpu {1,2}] [--memory {1GiB,2GiB,4GiB,8GiB}] [--disposable]`
+`raft fork <box> --ttl TTL [--cpu N] [--memory SIZE] [--disposable] [--purpose TEXT [--owner TEXT] [--next TEXT]]`
 
 - `--ttl TTL`: Lifetime in seconds for the clone. Required.
+- `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Replace the clone's whole note. Without them the clone keeps the source note.
 
-Omitted CPU/RAM values inherit the source limits. The clone is persistent unless `--disposable` is supplied, regardless of the source policy.
+Omitted CPU/RAM values inherit the source limits; controller defaults never apply to forks. The clone keeps the source's disk: the shared pool, or an own disk of the same size as a thin snapshot, which reserves that size in `raft-disks`. To copy a box onto a disk of another size, snapshot it and use `new --from BOX/SNAPSHOT --disk SIZE`. The clone is persistent unless `--disposable` is supplied, regardless of the source policy.
+The note the clone ends up with must satisfy the host [policy](#policy).
 
 Returns the qualified handle of the clone.
 
@@ -239,12 +262,17 @@ Export a stopped workspace and its snapshots to a compressed archive on the cont
 
 Import a portable backup archive onto a host as a fresh stopped workspace.
 
-`raft recover <source> --location LOCATION`
+`raft recover <source> --location LOCATION [--purpose TEXT [--owner TEXT] [--next TEXT]]`
 
 - `<source>`: Path to archive file on controller.
 - `--location LOCATION`: Target host location. Required.
+- `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Replace the recovered box's whole note. Without them it keeps the archived note.
+
+Under a host [policy](#policy), a given note is checked before the upload. An
+archived note is checked after import; if it fails, the imported box is deleted
+in the same locked step.
 
-Returns the qualified handle of the imported stopped workspace.
+Returns the qualified handle of the imported stopped workspace. A backup of a box with its own disk is recovered onto an own disk of the same size in `raft-disks`, which must exist on the destination and have room to reserve that size for the box and each of its snapshots; any other backup lands in the shared pool.
 
 ## Location commands
 
@@ -252,19 +280,20 @@ Returns the qualified handle of the imported stopped workspace.
 
 Check, provision and register a new workspace host, then prove it works.
 
-`raft host add LOCATION --ssh TARGET [--image-release TAG | --image-archive PATH | --build] [--dry-run] [--yes]`
+`raft host add LOCATION --ssh TARGET [--image-release TAG | --image-archive PATH | --build] [--disk-pool SIZE] [--dry-run] [--yes]`
 
 - `LOCATION`: New location name. Existing locations are rejected.
 - `--ssh TARGET`: Key-based SSH target, such as `ubuntu@192.0.2.10` or an `~/.ssh/config` alias. Accept the host key interactively first.
 - `--image-release TAG`: Download `raft-dev-<arch>.tar.gz` from this release on the host and verify it against the release `SHA256SUMS`. Default: the CLI's own version.
 - `--image-archive PATH`: Upload and import a local image archive instead.
 - `--build`: Build the image on the host from Debian 13 instead.
+- `--disk-pool SIZE`: Enable [own box disks](./capacity.mdx#own-box-disks) with a `raft-disks` thin pool of SIZE, in whole GiB. Default: not enabled. The pool's file takes its full size on the host disk when created. To enable them on a host that is already deployed, run `python3 deploy/deploy.py --location LOCATION --disk-pool SIZE`. An existing pool keeps its size; a different explicit value is refused.
 - `--dry-run`: Probe the host and print the plan without changing anything.
 - `--yes`: Apply the plan without the interactive prompt. Without a terminal, `--yes` is required.
 
 Before any change, a read-only probe checks SSH, passwordless sudo, Ubuntu 22.04 or
 24.04, ARM64 or AMD64, python3, at least 2 CPUs, 8 GiB RAM and, for a fresh host,
-80 GiB free disk. Unsupported targets, including macOS, fail with the reasons and
+80 GiB of free disk, plus the `--disk-pool` size when given. Unsupported targets, including macOS, fail with the reasons and
 no changes. The command then runs the deployer, imports the image only if its
 checksum and architecture match, adds the location with the pinned fingerprint,
 and runs `doctor` plus a disposable create/exec/destroy smoke test. If the final
@@ -291,31 +320,101 @@ Prints one JSON object per location (JSONL) in configuration order, with
 
 ### `note`
 
-Set or clear operator guidance for a location in `~/.config/raft/incus.json`.
+Set or clear the note of a location or a box.
 
 `raft note LOCATION TEXT`
 `raft note LOCATION --clear`
+`raft note BOX --purpose TEXT [--owner TEXT] [--next TEXT]`
+`raft note BOX --clear`
+
+Location notes are operator guidance stored in `~/.config/raft/incus.json`. Use
+them to record which host is preferred or what a host should not run. Raft shows
+them in `locations`, `capacity` and `new`.
+
+Box notes tell other agents and people why a box exists. They follow one
+template, stored on the box as JSON in the Incus key `user.raft.note`:
+
+| Field | Meaning | Example |
+| --- | --- | --- |
+| `purpose` (required) | What the box is for | `Integration tests for PR 42` |
+| `owner` | Who uses it | `codex session in ~/work/app` |
+| `next` | What happens to it next | `Destroy after the tests pass` |
+
+Each call replaces the whole note, on running or stopped boxes. `raft list`
+shows it as `note`. Forks and recovered backups keep the source box's note
+unless `fork` or `recover` gets note flags; update it if the copy has a
+different job.
+
+Empty values are rejected; use `--clear` to remove a note. A host
+[policy](#policy) can require fields, and then rejects notes and `--clear` that
+would leave them out. Otherwise Raft does not interpret notes.
+
+### `config`
+
+Show or change the controller defaults for `new`.
+
+`raft config`
+`raft config set KEY VALUE`
+`raft config unset KEY`
+
+- `KEY`: One of `location`, `ttl`, `cpu`, `memory`, `disk`.
+- `VALUE`: Accepted exactly when the same text is accepted by the matching `new` flag. A `location` must be configured.
+
+`cpu`, `memory` and `disk` size only boxes created from the image. `new --from`
+and `fork` keep the source's CPU, RAM and disk; only explicit flags change them.
+A `disk` default on a location without `raft-disks`, or larger than its
+unreserved space, fails `new` with the same error as `--disk`, plus a sentence
+naming `defaults.disk` as the source. `--disk shared` overrides it for one box.
+
+Defaults live in `~/.config/raft/config.json` as `{"defaults": {"KEY": VALUE}}`.
+`set` and `unset` print the old and new value, write atomically under a
+controller lock and keep mode `0600`. An unknown key, bad value or malformed
+file makes `new` fail with the file path and key; `set` or `unset` that key to
+repair it.
+
+`raft config` prints one JSON object: `file`, `defaults` (each key's `value` and
+`source`, `config` or `built-in`) and `policy` (each location's `require` list
+and `changed` record). It contacts every configured host to read policies.
+See [Defaults and note policies](../guides/config-policy.mdx).
 
-Notes are advisory. Use them to record which host is preferred or what a host
-should not run. Raft shows notes in `locations`, `limits` and `new` but does not
-enforce them. Empty notes are rejected; use `--clear` to remove one.
+### `policy`
+
+Set or clear the box note fields a location's host requires.
+
+`raft policy LOCATION --require FIELDS`
+`raft policy LOCATION --clear`
+
+- `--require FIELDS`: Comma-separated fields from `purpose`, `owner`, `next`. Must include `purpose`.
+- `--clear`: Stop requiring fields.
+
+The policy is stored on the host in the Incus project key `user.raft.require`,
+so it applies to every controller. `new`, `fork`, `recover` and `note` check it
+on the controller for an actionable error, then again in the host transaction
+that writes the note. The command prints the change and the host records the
+time, SSH account and controller in `user.raft.require.changed` and the host
+journal (`journalctl -t raft`).
+
+This is a guardrail for agents, not a security boundary: SSH and sudo on the
+host can change the policy or bypass it, and older controllers do not check it.
 
 ## Host inspection and maintenance commands
 
-### `limits`
+### `capacity`
 
-Inspect host resource capacity, active allocations, and recommended workspace limits.
+Estimate how many more boxes a host can run well. The report is advice: Raft
+enforces no box count or size budget, so create the boxes your task needs.
 
-`raft limits [--location LOCATION] [--json]`
+`raft capacity [--location LOCATION] [--json]`
 
 - `--location LOCATION`: Filter report to a specific host.
 - `--json`: Output report as JSON.
 
-Each report includes the location's `note`, if any.
+Each report leads with that advisory notice and includes the location's `note`,
+if any. See [Box sizing and capacity](./capacity.mdx) for the fields.
 
 ### `doctor`
 
-Inspect host connectivity, Incus version, expiry-worker compatibility, active systemd units (`incus`, `raft-expire.timer`, `raft-network.service`), root filesystem usage, storage pool status, and saved box count across configured hosts. An incompatible worker returns a nonzero exit status and names the failing location without running expiry or modifying boxes.
+Inspect host connectivity, Incus version, expiry-worker compatibility, active systemd units (`incus`, `raft-expire.timer`, `raft-network.service`), root filesystem usage, status of the shared `raft-data` pool and, when own box disks are enabled, the `raft-disks` pool, and saved box count across configured hosts. Without `raft-disks` it prints `own box disks: not enabled (redeploy with --disk-pool SIZE)` and does not fail. An incompatible worker returns a nonzero exit status and names the failing location without running expiry or modifying boxes.
 
 `raft doctor`
 
@@ -325,7 +424,9 @@ List existing workspace containers across configured hosts.
 
 `raft list [--location LOCATION]`
 
-Prints one JSON object per box (JSONL), with `box`, `state`, and `stopAt` fields.
+Prints one JSON object per box (JSONL), with `box`, `state`, `stopAt`, `disk` and `note` fields.
+`disk` is `"shared"` for the shared pool or the own disk's size, such as `"20GiB"`.
+`note` is the box note object, or `null` when the box has none.
 An empty inventory prints no records. See [Scripting and CI](../guides/scripting-ci.mdx)
 for output formats, exit codes, readiness, and failure handling.
 
@@ -344,3 +445,39 @@ Preview deletion of persistent stopped boxes older than the selected number of d
 Days must be at least one. Without `--yes`, this only prints JSONL candidates. With `--yes`, it deletes matching boxes and their snapshots under the host lock. Running boxes and boxes without a recorded stop time are excluded. Stop timestamps come from Raft stop/expiry, not box creation. No automatic retention policy is enabled.
 
 Disposable creation/fork and prune require the current expiry worker installed through the deployer. A mismatch fails with deployment guidance; the CLI does not install or adapt an older worker.
+
+## CLI maintenance commands
+
+### `self-update`
+
+Upgrade the installed CLI to the latest GitHub release, or to a chosen one.
+
+`raft self-update [--check | --version X.Y.Z]`
+
+- `--check`: Report whether a newer release exists without installing it. Exits `0` when the CLI is current and `100` when an update is available.
+- `--version X.Y.Z`: Install this release instead of the latest, for example to roll back. Without it, `self-update` never downgrades.
+
+The latest release on GitHub decides what is current. `self-update` installs
+through the channel that installed the running copy and never switches channel:
+
+| Installed with | Detected by | Upgrade | Integrity |
+| --- | --- | --- | --- |
+| `npm install -g @microck/raft` | `PREFIX/lib/node_modules/@microck/raft`, linked from `PREFIX/bin/raft` | `npm install --global --prefix PREFIX @microck/raft@X.Y.Z` | npm checks the registry integrity hash; Raft publishes with npm provenance |
+| `uv tool install` of a release wheel | A uv tool environment named `raft` | Downloads the wheel and `SHA256SUMS` from the release, then `uv tool install --force` | Refuses a wheel whose SHA-256 differs from the release `SHA256SUMS` |
+
+Source checkouts, editable installs, pip and other layouts are refused with
+instructions. For a uv tool, `uv tool dir` must still name the original tool
+directory; set `UV_TOOL_DIR` if it moved.
+
+`self-update` prints `raft OLD -> NEW` and the release notes link. A current CLI
+is a no-op. Hosts are not changed: run [`doctor`](#doctor) next. If a host's
+worker differs from the new CLI, redeploy that host from the same release; see
+[Upgrades and image changes](../guides/upgrades.mdx). After an update,
+[`host add`](#host-add) downloads the new release's image by default. Existing
+image pins stay unchanged.
+
+Set `RAFT_NO_UPDATE_CHECK=1` to turn off the daily release notice, for example in
+pinned or scripted runs. The notice caches its last result in
+`$XDG_CACHE_HOME/raft/update.json` (default `~/.cache/raft/update.json`). It
+waits at most 2 seconds for GitHub and is skipped for `self-update` and
+`--version`.

package/docs/reference/configuration.mdx

@@ -40,7 +40,7 @@ The configuration file maps location names to host definition objects:
 ### Location keys
 
 - **Naming format**: Matches `^[a-zA-Z0-9][a-zA-Z0-9_-]*$`. Starts with an alphanumeric character and contains ASCII letters, digits, underscores, or hyphens.
-- **Default location**: The first location listed is the default for `raft new`. `raft list`, `raft limits`, `raft gc`, and `raft prune` accept `--location` filters. Without a filter they check all configured hosts; `raft doctor` always checks all configured hosts.
+- **Default location**: The first location listed is the default for `raft new`. `raft list`, `raft capacity`, `raft gc`, and `raft prune` accept `--location` filters. Without a filter they check all configured hosts; `raft doctor` always checks all configured hosts.
 
 ### Host entry fields
 
@@ -60,7 +60,7 @@ raft note spare --clear
 ```
 
 `raft note` rewrites the file with two-space indentation, keeps location order and
-mode `0600`. `raft locations`, `raft limits` and `raft new` show the note.
+mode `0600`. `raft locations`, `raft capacity` and `raft new` show the note.
 
 <Callout type="warn" title="Replace placeholder image fingerprint">
 The `image` field requires the full 64-character hash for `raft new`; aliases such as `raft-dev` are rejected. Replace the example hash with the actual imported image fingerprint for that host.
@@ -68,6 +68,40 @@ The `image` field requires the full 64-character hash for `raft new`; aliases su
 ARM64 and AMD64 images produce different fingerprints; do not reuse an ARM64 hash on an AMD64 host.
 </Callout>
 
+## Defaults for new
+
+`~/.config/raft/config.json` is optional. It sets defaults for `raft new` on this
+controller and is separate from the location inventory:
+
+```json title="Example ~/.config/raft/config.json"
+{
+  "defaults": {
+    "location": "lab",
+    "ttl": 3600,
+    "cpu": 2,
+    "memory": "4GiB",
+    "disk": "20GiB"
+  }
+}
+```
+
+| Key | Accepted values | Built-in default |
+| :--- | :--- | :--- |
+| `location` | A configured location name | First configured location |
+| `ttl` | `60` to `2592000` seconds | `600` |
+| `cpu` | A positive whole number | `1` |
+| `memory` | Whole GiB, e.g. `16GiB` | `2GiB` |
+| `disk` | Whole GiB for an own box disk, or `shared` | `shared` |
+
+Explicit flags win over these values. `cpu`, `memory` and `disk` size only boxes
+created from the image: `new --from` and `fork` keep the source's size unless a
+flag overrides it. A `disk` default needs the `raft-disks` pool on the target
+host, like `--disk`. Unknown keys and invalid values make
+`raft new` fail with the file path and key. Edit the file with
+`raft config set KEY VALUE` and `raft config unset KEY`, and inspect it with
+`raft config`. Required box note fields are a host policy, not part of this
+file; see [Defaults and note policies](../guides/config-policy.mdx).
+
 ## SSH configuration and connection behavior
 
 Primary remote commands (`raft ssh`, `upload`, `download`, and control commands) run OpenSSH with `BatchMode=yes` and `-o ConnectTimeout=10` to fail fast on unreachable hosts. The timeout applies to initial connection establishment, not session or transfer duration.

package/docs/reference/meta.json

@@ -5,6 +5,6 @@
   "pages": [
     "cli",
     "configuration",
-    "capacity-limits"
+    "capacity"
   ]
 }

package/docs/releases/0.4.0.md

@@ -0,0 +1,15 @@
+---
+title: Raft 0.4.0
+---
+
+# Raft 0.4.0
+
+- Boxes accept a note that says why they exist, so agents and people can tell which boxes are in use. The note follows one template: `purpose` (required), `owner` and `next`.
+- `raft new --purpose TEXT [--owner TEXT] [--next TEXT]` notes a box when it is created. `raft note BOX --purpose ...` replaces a box's note and `raft note BOX --clear` removes it. `raft note LOCATION ...` still edits location notes.
+- `raft list` adds a `note` field to each box: the note object, or `null`.
+- Forks and recovered backups keep the source box's note. `new --from` starts without one.
+- The `raft-cli` agent skill asks agents to note every box they create and to read notes before resuming, stopping, forking or destroying a box.
+
+Install on Linux with Python 3.11.8+ using `npm install -g @microck/raft@0.4.0`, or install the attached Python wheel. Box notes live in each box's Incus configuration; host workers are unchanged and need no redeployment.
+
+ARM64 and AMD64 development images accompany this release. Verify `SHA256SUMS` before importing an image for your host architecture.

package/docs/releases/0.5.0.md

@@ -0,0 +1,19 @@
+---
+title: Raft 0.5.0
+---
+
+# Raft 0.5.0
+
+- **No box cap.** A location no longer stops at four saved boxes. Agents create the boxes their task needs; creation only fails for real reasons such as a full pool.
+- **Any size.** `--cpu N` and `--memory NGiB` accept any positive size up to the host's CPUs and RAM. A box from the image gets 1 CPU, 2 GiB and the shared disk; `new --from` and `fork` keep the source's CPU, RAM and disk, and each flag overrides only its own value.
+- **Own box disks.** `raft new --disk SIZE` gives a box its own ext4 disk with a hard cap that also holds for Docker inside the box. Own disks are opt-in per host: redeploy with `--disk-pool SIZE` to create the thin LVM pool `raft-disks`. New boxes, forks, template copies and snapshots are copy-on-write, and a reservation check refuses any request that would overcommit the pool. `raft list` shows each box's `disk`.
+- **`raft capacity` replaces `raft limits`.** It is advisory only: it reports host CPUs, RAM, allocations, both pools and how many more boxes of example sizes fit, and never limits creation. JSON fields changed (`boxes`, `estimates`, `sharedPool`, `boxDiskPool`, `headroom`, `notice`).
+- **`raft self-update`.** Updates the CLI through the channel that installed it: npm, or a uv tool from the release wheel verified against `SHA256SUMS`. `--check` reports without installing (exit 100 when an update exists) and `--version X.Y.Z` installs a specific release. A notice on stderr at most once a day says when a newer release exists; `RAFT_NO_UPDATE_CHECK=1` turns it off. It never installs on its own and never changes hosts.
+- **Controller defaults.** `raft config`, `raft config set KEY VALUE` and `raft config unset KEY` manage `~/.config/raft/config.json` with defaults for `location`, `ttl`, `cpu`, `memory` and `disk`. A flag beats the config, which beats the built-in default. `--disk shared` overrides a configured disk default for one box.
+- **Note policies.** `raft policy LOCATION --require purpose,owner` makes a host require those box note fields. `new`, `fork`, `recover` and `note` refuse a box without them, with an error naming the missing flag, and the host checks again under its lock. Policy changes are recorded on the host and in its journal. This is a guardrail for agents, not a security boundary.
+
+Install on Linux with Python 3.11.8+ using `npm install -g @microck/raft@0.5.0`, or install the attached Python wheel; existing installs can run `raft self-update`.
+
+Redeploy each host with this release's `deploy/deploy.py`. It installs `lvm2` and `python3-yaml`, which `raft recover` now needs to read backups, and changes nothing else unless you add `--disk-pool SIZE` to enable own box disks. That needs SIZE plus 20 GiB free on the host; existing storage is never resized.
+
+ARM64 and AMD64 development images accompany this release. Verify `SHA256SUMS` before importing an image for your host architecture.

package/docs/requirements.mdx

@@ -17,11 +17,11 @@ Each workspace host must be a dedicated physical machine or virtual instance:
 | **Operating system** | Ubuntu 22.04 LTS or 24.04 LTS | The deployer installs Incus 6.0 LTS and nftables bridge isolation rules. |
 | **CPU** | At least 2 CPU cores | Capacity calculations reserve 1 CPU core for the host OS and services (an advisory estimate, not an enforced host reservation). |
 | **RAM** | At least 8 GiB | Sizing calculations reserve the larger of 2 GiB or 10% of host RAM (an advisory estimate, not an enforced host reservation). |
-| **Free disk** | At least 80 GiB on host disk | The deployer provisions a 60 GiB dedicated Btrfs pool (`raft-data`). |
+| **Free disk** | At least 80 GiB on host disk | For the 60 GiB shared Btrfs pool (`raft-data`), images and the host. [Own box disks](./reference/capacity.mdx#own-box-disks) are optional: `--disk-pool SIZE` adds a `raft-disks` pool of SIZE, so a fresh host then needs 80 GiB plus SIZE, and an existing host SIZE plus 20 GiB. |
 
 `raft host add` checks these requirements read-only before it changes a host.
 
-Host CPU and RAM reserves are advisory estimates for `raft limits`, not enforced cgroup locks.
+Host CPU and RAM reserves are advisory estimates for `raft capacity`, not enforced cgroup locks. Raft limits neither how many boxes a host runs nor their size.
 
 <Callout type="warn" title="Dedicated host requirement">
 Do not deploy Raft on hosts with existing Incus storage pools or shared virtualization workloads. Deployment configures dedicated nftables rules, storage pools, and system services.