@microck/raft 0.5.0 → 0.7.0

package/docs/guides/background-jobs.mdx

@@ -5,11 +5,11 @@ description: Run detached commands with systemd logging and process management.
 
 # Background jobs
 
-Raft runs detached background commands under the guest systemd init system using transient service units.
+Raft runs detached commands under guest systemd and captures their results with a transient host supervisor.
 
 ## Target workspace handle
 
-Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+Launching a job requires a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
 
 ```sh title="CONTROLLER: Select running workspace"
 raft list
@@ -33,49 +33,56 @@ The command returns a 32-hex systemd unit name prefixed with `rfcmd-` (such as `
 
 ## Checking job status
 
-Use `raft status` to query guest systemd unit properties:
+`raft status "$box" "$job"` returns a retained JSON record:
 
-```sh title="CONTROLLER: Inspect background job state"
-raft status "$box" "$job"
+```json
+{"job":"rfcmd-...","box":"lab:rf-...","state":"completed","exitCode":23,"startedAt":1793280000,"finishedAt":1793280010,"droppedBytes":0}
 ```
 
-Active jobs report running substate:
-```text
-ActiveState=active
-SubState=running
-ExecMainStatus=0
-```
-
-Because Raft runs detached commands with `RemainAfterExit=yes`, completed jobs retain their exit record:
-```text
-ActiveState=active
-SubState=exited
-ExecMainStatus=0
-```
-
-<Callout type="info" title="Exit status vs CLI return code">
-`ExecMainStatus` reports the command payload exit code inside the container. The `raft status` command returns exit code 0 when the query succeeds. It queries unit state and does not report process PIDs or start timestamps.
-</Callout>
-
-## Viewing job logs
+States are `starting`, `running`, `completed`, `cancelled`, `interrupted` and
+`failed`. `completed` means the command finished, not that it succeeded; inspect
+`exitCode`. An interrupted job has no confirmed payload exit code. A transport interruption can leave the guest command running;
+inspect or cancel it before rerunning the payload. Timestamps
+are host Unix seconds. A successful status query itself returns zero.
 
-Retrieve output recorded in the guest systemd journal:
+## Viewing retained output
 
-```sh title="CONTROLLER: Retrieve job log output"
+```sh
 raft logs "$box" "$job"
 ```
 
-The command runs `journalctl --no-pager -o cat -u <job>` inside the container. It prints recorded logs without a pager or stream follow mode.
+A host supervisor retains the first 1 MiB of combined stdout and stderr. Further
+output is drained and counted in `droppedBytes` when the job ends. Logs and final
+status remain readable after the workspace stops or is deleted; keep its exact
+original handle. They are not copied by workspace snapshots or forks.
 
-## Cancelling jobs
+Results live in the root-private host directory `/var/lib/raft-jobs`. They expire
+seven days after completion. Job commands prune expired records on access, so an
+idle host may retain expired files until the next access. The store holds at most
+512 records and refuses new jobs when full instead of evicting unexpired results.
+Unreadable records remain on the host for inspection and count toward the limit.
+They do not block other jobs. A query for a damaged record reports its host path;
+inspect that directory and its retained output before deciding how to recover it.
+Output can contain secrets; this is host-local retention, not an off-host backup.
 
-Terminate a running detached command:
+## Cancelling jobs
 
-```sh title="CONTROLLER: Terminate background job"
+```sh
 raft cancel "$box" "$job"
 ```
 
-`raft cancel` executes `systemctl stop <job>`, terminating processes in the job's systemd control group.
+Cancellation stops the guest systemd control group. Retained output remains
+available. If a job is still starting and its guest unit is not yet registered,
+cancellation reports an error; inspect its state before retrying. Unknown job
+IDs and IDs belonging to another workspace are rejected. Cancellation waits up to
+120 seconds. If it times out, the cancellation request remains recorded; inspect
+the job and retry cancellation.
+
+Host reboot or guest shutdown interrupts active jobs. The next inspection marks
+an absent supervisor as interrupted rather than assuming success. Completed
+results survive host reboot. This development implementation has no fallback to
+old guest-only journals; inspect old jobs with their controller version before
+upgrading.
 
 ## Extending workspace lifetime
 

package/docs/guides/backup-recovery.mdx

@@ -9,7 +9,8 @@ Raft exports portable workspace archives and restores them onto compatible hosts
 
 ## Target workspace handle
 
-Commands in this guide require a stopped workspace:
+Use a **persistent** workspace. Stop it before backup; stopping a disposable box
+deletes it and its snapshots, so copy files out instead.
 
 ```sh title="CONTROLLER: Select and stop target workspace"
 raft list
@@ -35,7 +36,7 @@ Backup archives contain the entire guest filesystem, including credentials, SSH
 
 Raft enforces safeguards during export:
 
-- **Private permissions**: Destination mode must be `0600`. Group and world permissions must be zero before export begins.
+- **Private permissions**: Raft creates the staged archive with mode `0600` and publishes it privately. Choose a new destination path; do not pre-create the file.
 - **Atomic link publication**: After export and `fsync`, Raft publishes the archive using `os.link(archive.name, destination)`. Hard-link support is checked at final publication rather than preflight; export can succeed while publication fails if the filesystem lacks hard-link support.
 - **No overwriting**: The destination file must not already exist.
 - **Lifecycle lock**: Export holds `/run/lock/raft-incus.lock`. Expiry checks for other containers on the host are delayed while the lock is held.
@@ -53,7 +54,7 @@ echo "Recovered workspace: $recovered"
 
 - **Architecture compatibility**: Target host must match the native CPU architecture (ARM64 to ARM64, AMD64 to AMD64). Cross-architecture recovery is not supported.
 - **Fresh network identity**: The container receives a new name in project `raft`, a new MAC address on `rfbr0`, and no inherited TTL deadline.
-- **Stopped initial state**: Recovered workspaces remain stopped until started with `raft resume "$recovered" --ttl <SECONDS>`.
+- **Stopped initial state**: Recovered workspaces remain stopped until started with `raft resume "$recovered" --ttl <SECONDS>`. Under a host [retention policy](./config-policy.mdx#delete-boxes-left-stopped), the recovered box's clock starts when the host worker first sees it, within about 15 seconds; resume it before the policy's hours pass.
 - **Disk placement**: A backup of a box with its own disk returns to an own disk of the same size in `raft-disks`. The destination needs own box disks enabled and enough unreserved pool space for the box and each of its snapshots; otherwise recovery fails before importing. Other backups land in the shared pool.
 - **Interrupted recovery**: Follow [Host crash during recovery](./troubleshooting.mdx#host-crash-during-recovery). Inspect the reported handle and container state before deleting any staging file.
 

package/docs/guides/ci-jobs.mdx

@@ -0,0 +1,209 @@
+---
+title: Disposable CI jobs
+description: Run trusted builds in temporary Raft boxes, collect results, and clean up with a bounded lifetime.
+---
+
+# Disposable CI jobs
+
+`raft run` runs one foreground command in a fresh disposable box. It creates the
+box, waits for command readiness, optionally uploads source, streams command
+output, collects requested result directories, and attempts ownership-checked
+cleanup even after failure or cancellation. It does not register a GitHub
+self-hosted runner or replace your CI scheduler.
+
+<Callout type="warn" title="Unreleased: requires the 0.7.0 development code">
+The published release is 0.6.0, which does not include `raft run`. As checked on
+2026-10-24, no 0.7.0 npm package or GitHub release assets are available. Use the
+[stable scripting workflow](./scripting-ci.mdx) unless you intend to test a reviewed
+source revision. Do not install a nonexistent 0.7.0 package.
+</Callout>
+
+For development, run `python3 /path/to/raft.py run` from the reviewed checkout.
+The CI example below pins the full source commit instead of an unpublished tag.
+
+## Trust boundary
+
+Use dedicated hosts and trusted jobs only. The controller's SSH identity has
+passwordless sudo and controls the host; it is not a scoped, per-job CI token.
+Boxes are unprivileged Incus containers sharing the host kernel, not separate
+kernel VMs. Do not expose this identity to unreviewed pull requests, fork jobs,
+or a `pull_request_target` workflow that checks out attacker-controlled code.
+
+Raft does not install or register a runner, configure repository secrets, forward
+your SSH agent, copy controller environment variables, or buy cloud capacity.
+The operator must approve and configure existing host access separately. Never
+copy a host private key or a GitHub token into the guest.
+
+## One command
+
+```sh title="CONTROLLER: Git working tree"
+RAFT_NO_UPDATE_CHECK=1 raft run \
+  --location lab --timeout 600 --cpu 2 --memory 4GiB \
+  --purpose "Project tests" --owner "Local CI" --next "Delete after collection" \
+  --push . --pull /workspace/project/reports ./ci-reports \
+  -- bash -lc 'set -euo pipefail; mkdir -p reports; npm ci; npm test 2>&1 | tee reports/test.log'
+```
+
+The controller needs Linux, Python 3.11.8+, OpenSSH, Git for `--push`, a configured
+Raft location, verified SSH host keys, batch authentication, and the current host
+expiry worker. Ordinary image/host sizing and note policies still apply. `run`
+always forces disposable behavior, regardless of controller lifetime defaults.
+It never retries the payload command.
+
+- stdout/stderr stream the payload; lifecycle messages and elapsed time go to
+  stderr. There is no cost estimate or JSON result protocol.
+- `--timeout` is one total deadline for Git filtering, creation, readiness, upload
+  and execution, not a fresh timeout for each step. Default: 600 seconds; range:
+  1 to 2,591,880 seconds.
+- `--push` includes tracked and unignored untracked files from the chosen Git
+  working tree, including working-tree edits. It excludes `.git` at every level
+  and refuses submodules. Tracked files remain included even if ignored later:
+  this is not a secret scanner. Deleted tracked files are absent.
+- `--push-all` explicitly copies the complete directory, including hidden and
+  ignored files and Git metadata. Inspect it for credentials first. This is the
+  opt-in alternative for non-Git or explicitly staged trees; it is not a silent
+  fallback when Git fails.
+- Uploads default to `/workspace/project`; without an upload, commands default
+  to `/workspace`. `--cwd` overrides this. An upload destination must not exist,
+  and its parent must already exist. Raft never merges into a stale checkout.
+- Repeat `--pull GUEST_DIRECTORY NEW_LOCAL_DIRECTORY` to collect directories,
+  including after a failing payload. Local parents must exist; destinations
+  must be new, including no existing symlinks. Files can be placed in a result
+  directory. Collection happens before deletion, with a shared 30-second grace.
+  A failed pull does not stop later pulls unless that grace has expired.
+- Exit status preserves a failing payload, including signal-style statuses.
+  Job timeout returns 124, Ctrl-C 130, SIGTERM 143. Setup/collection/cleanup
+  failure returns 1 when there is no earlier failing status. A successful
+  payload is not reported successful when cleanup cannot be confirmed.
+
+The host receives a unique ownership marker and a fixed expiry of timeout + 120
+seconds as part of creation, before startup. Cleanup checks ownership under the
+host lifecycle lock and has a separate 30-second grace. Repeated cancellation is
+ignored during these bounded collection/cleanup attempts. No lease renewal or
+`--keep` mode is implemented.
+
+If creation was not acknowledged, cleanup first records that run's cancellation
+under the same host lifecycle lock. Creation checks that record before allocating,
+so a queued SSH request cannot start the box after cleanup has confirmed absence.
+These empty per-run records live in the root-only `/run/raft-ci-cancelled`
+directory, use no-follow operations, and remain until host reboot, when queued
+processes also disappear. Ordinary acknowledged runs leave no cancellation file.
+
+If the controller is killed before cleanup, SSH is lost, or cleanup fails, inspect
+the exact handle printed on stderr. A disconnected create may still finish when
+cancellation could not be recorded. The disposable expiry worker is a backstop,
+not a guarantee of instantaneous deletion: an unreachable host, stopped worker or
+held lifecycle lock can delay it. Do not blindly rerun an uncertain job or assume
+its side effects were rolled back.
+
+## Prepared templates instead of shared mutable caches
+
+Prepare dependencies and tool versions in a persistent base box, stop it, then
+create a named snapshot. See [project setup](./project-setup.mdx) and
+[snapshots](./snapshots-forks.mdx). Keep the source box and snapshot available:
+
+```sh title="CONTROLLER: Use an existing trusted snapshot"
+raft run --from lab:rf-REPLACE_WITH_BASE_ID/ci-base --timeout 900 \
+  --purpose "Cached build" --owner "CI" --next "Delete after tests" \
+  --push . -- bash -lc 'npm ci --prefer-offline && npm test'
+```
+
+The template stays on its source host and retains its size unless explicitly
+overridden. Put reusable caches outside the upload destination, which must be
+new. Each job gets an independent copy; its mutations do not update the template
+or another job's cache. Refresh templates deliberately when dependencies change.
+
+Raft snapshots retain filesystem contents, including credentials and Docker
+storage. They do not save process memory. Inspect and sanitize the base before
+sharing it with jobs. Do not assume Boat's selective snapshot/cache rules apply
+to Raft's whole-filesystem copies.
+
+## GitHub Actions orchestration example
+
+The following is an example for a separate trusted project, not a change to
+Raft's own runner fleet. An ordinary `ubuntu-latest` job invokes Raft over SSH.
+Before using it, an operator must approve storing a dedicated CI host key as
+`RAFT_SSH_KEY`, verified host-key lines as `RAFT_KNOWN_HOSTS`, and the location
+inventory JSON as `RAFT_INVENTORY`. The inventory's SSH target must use that key
+and its image fingerprint must be pinned. Never generate trust with an unchecked
+`ssh-keyscan` or disable host-key checking. Restrict access to these secrets.
+
+The example pins development commit `7d678bb079a9b2dcde4e2eb9e4cd0468d00f50f3` in a
+separate Python environment. Review that code and its unreleased status
+before use; do not silently replace it with a moving branch.
+
+```yaml title="EXAMPLE: .github/workflows/trusted-build.yml"
+name: Trusted Raft build
+on:
+  push:
+    branches: [main]
+  workflow_dispatch:
+permissions:
+  contents: read
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    env:
+      RAFT_NO_UPDATE_CHECK: '1'
+    steps:
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262
+        with:
+          persist-credentials: false
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065
+        with:
+          python-version: '3.14'
+      - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020
+        with:
+          node-version: '24'
+      - name: Install reviewed development controller
+        run: |
+          python -m venv "$RUNNER_TEMP/raft-cli"
+          "$RUNNER_TEMP/raft-cli/bin/pip" install "git+https://github.com/Microck/raft.git@7d678bb079a9b2dcde4e2eb9e4cd0468d00f50f3"
+          echo "$RUNNER_TEMP/raft-cli/bin" >> "$GITHUB_PATH"
+      - name: Configure approved existing access
+        env:
+          SSH_KEY: ${{ secrets.RAFT_SSH_KEY }}
+          KNOWN_HOSTS: ${{ secrets.RAFT_KNOWN_HOSTS }}
+          INVENTORY: ${{ secrets.RAFT_INVENTORY }}
+        run: |
+          umask 077
+          mkdir -p ~/.ssh ~/.config/raft
+          printf '%s\n' "$SSH_KEY" > ~/.ssh/id_ed25519
+          printf '%s\n' "$KNOWN_HOSTS" > ~/.ssh/known_hosts
+          printf '%s\n' "$INVENTORY" > ~/.config/raft/incus.json
+          printf 'Host *\n  BatchMode yes\n  StrictHostKeyChecking yes\n  IdentitiesOnly yes\n' > ~/.ssh/config
+      - name: Run remote tests and collect logs
+        run: |
+          raft run --location ci --timeout 900 --cpu 2 --memory 4GiB \
+            --purpose "Trusted project tests" --owner "GitHub Actions" --next "Delete after tests" \
+            --push . --pull /workspace/project/reports ./ci-reports \
+            -- bash -lc 'set -euo pipefail; mkdir -p reports; npm ci; npm test 2>&1 | tee reports/test.log'
+      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
+        if: always()
+        with:
+          name: test-reports
+          path: ci-reports/
+          if-no-files-found: warn
+```
+
+No checkout credential is needed in the guest. The secret files are outside the
+uploaded tree, and `.git` is excluded. Set a CI job limit longer than Raft's
+command deadline plus collection and cleanup grace. A forcibly terminated CI
+runner cannot perform final cleanup; check the host expiry service and inventory.
+
+## Comparison and cost
+
+[Boat's CI guide](https://docs.boat.dev/ci) uses the same broad pattern: an existing
+CI job orchestrates a disposable remote workspace. Boat also provides a managed
+`run` command, scoped API keys, renewable leases and a different VM/snapshot
+model. Raft implements a fixed lifetime on operator-owned Incus hosts and uses
+host-level SSH access; these are meaningful differences, not interchangeable
+security or billing guarantees.
+
+There is no universal “10x cheaper” claim for Raft. Compare the complete host,
+storage, idle capacity, operations and orchestration-runner cost. Standard
+GitHub-hosted runners are already free for public repositories; remote capacity
+can still help with specific hardware, prepared environments or isolation from
+other trusted jobs. See [Boat pricing](https://docs.boat.dev/pricing) and
+[GitHub Actions billing](https://docs.github.com/en/billing/concepts/product-billing/github-actions).

package/docs/guides/config-policy.mdx

@@ -1,19 +1,20 @@
 ---
-title: Defaults and note policies
-description: Set controller defaults for new boxes and require box note fields on a host.
+title: Defaults and host policies
+description: Set controller defaults for new boxes, require box note fields on a host and delete boxes left stopped.
 ---
 
-# Defaults and note policies
+# Defaults and host policies
 
 Raft has two kinds of settings:
 
 | Setting | Stored in | Applies to | Change with |
 | --- | --- | --- | --- |
 | Defaults for `new` | `~/.config/raft/config.json` on the controller | That controller only | `raft config set` and `raft config unset` |
-| Required box note fields | Incus project `raft` on each host | Every controller that uses the host | `raft policy` |
+| Required box note fields | Incus project `raft` on each host | Every controller that uses the host | `raft policy --require` |
+| Stopped-box retention | Incus project `raft` on each host | Every box on the host | `raft policy --delete-stopped-after` |
 
-`raft config` prints both: the effective defaults with their source, and each
-location's policy.
+`raft config` prints both kinds: the effective defaults with their source, and
+each location's policies.
 
 ```sh title="CONTROLLER: Inspect defaults and policies"
 raft config
@@ -42,6 +43,10 @@ raft config
     "disk": {
       "value": "shared",
       "source": "built-in"
+    },
+    "disposable": {
+      "value": true,
+      "source": "config"
     }
   },
   "policy": {
@@ -50,7 +55,10 @@ raft config
         "purpose",
         "owner"
       ],
-      "changed": "2026-10-21T09:12:44Z by ubuntu from me@laptop"
+      "requireChanged": "2026-10-21T09:12:44Z by ubuntu from me@laptop",
+      "deleteStoppedAfterHours": 48,
+      "deleteStoppedAfterSince": 1792573964,
+      "deleteStoppedAfterChanged": "2026-10-21T09:12:44Z by ubuntu from me@laptop"
     }
   }
 }
@@ -61,33 +69,42 @@ names the location when a host is unreachable.
 
 ## Defaults for new
 
-Set a default for any of `location`, `ttl`, `cpu`, `memory` and `disk`:
+Set a default for any of `location`, `ttl`, `cpu`, `memory`, `disk` and
+`disposable`:
 
-```sh title="CONTROLLER: Prefer bigger, longer boxes on paris"
+```sh title="CONTROLLER: Prefer bigger, longer, temporary boxes on paris"
 raft config set location paris
 raft config set ttl 3600
 raft config set cpu 4
 raft config set memory 16GiB
 raft config set disk 20GiB
+raft config set disposable true
 raft config unset cpu
 ```
 
 Each command prints the old and new value. `raft new` takes an explicit flag
 first, then the configured default, then the built-in default (first configured
-location, 600 seconds, 1 CPU, 2 GiB, the shared pool). `resume`, `fork`,
-`extend` and `recover` take no defaults: their TTL and target are deliberate
-choices, and omitted sizes on resume and fork keep the source limits.
+location, 600 seconds, 1 CPU, 2 GiB, the shared pool, persistent). `resume`,
+`fork`, `extend` and `recover` take no defaults: their TTL and target are
+deliberate choices, and omitted sizes on resume and fork keep the source limits.
+
+With `disposable true`, every `raft new` box is deleted, with its snapshots, when
+it stops or expires. `raft new --persistent` keeps one box anyway, and
+`--disposable` overrides `disposable false` the same way. The default applies to
+`new --from` too: a snapshot has no lifecycle of its own. A fork stays
+persistent unless it gets `--disposable`, whatever the default says, like every
+other fork setting.
 
 Size defaults (`cpu`, `memory`, `disk`) apply only where the built-in sizes do:
 to boxes created from the image. A box from `new --from` keeps the snapshot's
 location, CPU, RAM and disk, and a fork keeps its source's, whatever
 `config.json` says. Only a flag changes them:
 
-| Command | CPU and RAM | Disk |
-| --- | --- | --- |
-| `new` | Flag, then config, then 1 CPU / 2 GiB | Flag, then config, then the shared pool |
-| `new --from` | Flag, then the snapshot's | `--disk SIZE`, then the snapshot's |
-| `fork` | Flag, then the source's | The source's |
+| Command | CPU and RAM | Disk | Disposable |
+| --- | --- | --- | --- |
+| `new` | Flag, then config, then 1 CPU / 2 GiB | Flag, then config, then the shared pool | Flag, then config, then persistent |
+| `new --from` | Flag, then the snapshot's | `--disk SIZE`, then the snapshot's | Flag, then config, then persistent |
+| `fork` | Flag, then the source's | The source's | `--disposable`, else persistent |
 
 A `disk` default needs own box disks on the target host. On a location without
 the `raft-disks` pool, or with too little unreserved space, `raft new` fails with
@@ -107,9 +124,9 @@ path and the key; Raft never ignores part of the file. Repair a bad key with
 `raft config set KEY VALUE` or `raft config unset KEY`. Writes are atomic, keep
 mode `0600` and remove the file when no default remains.
 
-Scripts and CI jobs that need stable sizing should pass `--location`, `--ttl`,
-`--cpu`, `--memory` and `--disk` explicitly, because defaults differ between
-controllers.
+Scripts and CI jobs that need stable boxes should pass `--location`, `--ttl`,
+`--cpu`, `--memory`, `--disk` and `--disposable` or `--persistent` explicitly,
+because defaults differ between controllers.
 
 ## Required box note fields
 
@@ -118,7 +135,7 @@ fields. `purpose` is always part of a note, so a policy must include it:
 
 ```sh title="CONTROLLER: Require an owner on lab"
 raft policy lab --require purpose,owner
-raft policy lab --clear
+raft policy lab --require none
 ```
 
 With that policy, a command that would leave a box without an owner fails before
@@ -151,29 +168,86 @@ so a policy change cannot race a creation. That transaction checks the
 requested size against the host and reserves own-disk space first; any refusal
 comes before Incus creates the box.
 
+## Delete boxes left stopped
+
+A retention policy makes a host delete boxes that stay stopped too long, with
+their snapshots. Use it when agents create boxes and forget them:
+
+```sh title="CONTROLLER: Delete boxes stopped for 48 hours on lab"
+raft policy lab --delete-stopped-after 48
+raft policy lab --delete-stopped-after never
+```
+
+The host expiry worker applies it every 15 seconds, and on `raft gc`, even when
+no controller is online. It deletes a stopped box once the given hours have
+passed since the later of:
+
+- the box's stop, recorded in `user.raft.stopped-at` by `raft stop` and expiry;
+- the policy change, recorded in `user.raft.delete-stopped-after.since`.
+
+So a new or shorter policy never deletes a box sooner than its hours after the
+change, including boxes that were stopped long before. Lengthening it or setting
+it again restarts the grace period the same way.
+
+| Box | What retention does |
+| --- | --- |
+| Running, including one whose TTL has not expired | Nothing |
+| Stopped with `raft stop` or expiry | Deleted HOURS after the later of the stop and the policy change |
+| Stopped another way: a poweroff inside the box, a native `incus stop`, a failed start or `raft recover` | The worker records the time it first sees the box stopped, and the clock starts then |
+| Resumed before its deadline | Nothing; `resume` clears the stop time, and the next stop starts a new clock |
+
+`raft list` shows when each stopped box will go, as `deleteAt`:
+
+```sh title="CONTROLLER: Stopped boxes and their deletion times"
+raft list --location lab | jq -r 'select(.deleteAt != "") | "\(.box) \(.deleteAt | tonumber | todate)"'
+```
+
+Retention deletes the whole box. Keep anything you need with
+[`raft backup`](./backup-recovery.mdx) before the deadline, or resume the box to
+reset its clock. Snapshots live inside the box and are deleted with it.
+
+Retention needs the current host worker. On a host deployed by an older release,
+`--delete-stopped-after HOURS` fails with
+`Host expiry worker differs from this CLI; deploy the current Raft checkout first`
+and changes nothing; redeploy the host, see
+[Upgrades and image changes](./upgrades.mdx). Clearing it with `never` works on
+any worker.
+
+`raft prune --older-than DAYS` remains a manual, previewed cleanup that works
+with or without a retention policy.
+
 ## Audit trail
 
-`raft policy` prints the change, for example
-`lab: required note fields none -> purpose,owner`. The host records the last
-change in the project key `user.raft.require.changed` (UTC time, the SSH account
-and the controller's self-reported user and hostname) and writes a `raft` entry
-to the host journal:
+`raft policy` prints each change, for example
+`lab: required note fields none -> purpose,owner` or
+`lab: delete stopped boxes after never -> 48 hours`. The host records the last
+change of each policy in a `.changed` project key (UTC time, the SSH account and
+the controller's self-reported user and hostname) and writes a `raft` entry to
+the host journal. The worker adds a journal entry for every box retention
+deletes:
 
-```sh title="HOST: Review policy changes"
+```sh title="HOST: Review policy changes and retention deletions"
 sudo incus project get raft user.raft.require
 sudo incus project get raft user.raft.require.changed
+sudo incus project get raft user.raft.delete-stopped-after
+sudo incus project get raft user.raft.delete-stopped-after.changed
 sudo journalctl -t raft
 ```
 
 ## Limits
 
-Note policies are a guardrail for agents, not a security boundary:
-
-- Anyone with SSH and sudo on the host can change or remove the policy, or create
-  boxes with native `incus` commands that skip it.
-- Controllers older than the release that added policies do not check them.
-  Upgrade every controller that uses a host before relying on its policy.
-- The controller identity in `user.raft.require.changed` is reported by the
-  controller and is not verified. The SSH account comes from `sudo`.
+Host policies are a guardrail for agents, not a security boundary:
+
+- Anyone with SSH and sudo on the host can change or remove a policy, or create
+  boxes with native `incus` commands that skip the note policy.
+- Retention trusts `user.raft.stopped-at`. The worker clears it when it sees a
+  box running, but a native `incus start` and stop between two worker runs (15
+  seconds apart) keeps the old stop time, so the box can be deleted on the next
+  run. Use `raft resume`, which clears the stop time.
+- Controllers older than the release that added note policies do not check
+  them. Upgrade every controller that uses a host before relying on one. The
+  host worker enforces retention whatever the controller version.
+- The controller identity in the `.changed` keys is reported by the controller
+  and is not verified. The SSH account comes from `sudo`.
 - Raft checks that required fields are present and nonempty, not that their text
   is accurate.

package/docs/guides/files.mdx

@@ -5,11 +5,11 @@ description: Transfer files between your controller machine and running Raft wor
 
 # File transfer
 
-Upload and download individual files between your local controller and running workspaces over verified host SSH and Incus instance APIs.
+Upload and download individual files between your controller and a box over verified host SSH and Incus instance APIs.
 
 ## Target workspace handle
 
-Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+Use a running box for all examples below. Individual-file transfers also work while stopped; recursive directory transfers require a running box. Select one from `raft list` (see [First working box](../first-box.mdx)):
 
 ```sh title="CONTROLLER: Select running workspace"
 raft list

package/docs/guides/index.mdx

@@ -0,0 +1,31 @@
+---
+title: Task guides
+description: Pick a workflow after setting up your first Raft host.
+---
+
+# Task guides
+
+Start with [your first box](../first-box.mdx) if you have not run a command yet.
+Commands run on the controller unless a block is labeled **HOST**.
+
+## Develop
+
+- [Project setup and templates](./project-setup.mdx): upload a project and reuse dependencies.
+- [Docker](./docker.mdx): build images and manage guest disk use.
+- [File transfer](./files.mdx): copy individual files or directory trees.
+- [Desktop and private services](./desktop-services.mdx): open a browser, desktop or local tunnel.
+
+## Automate
+
+- [Background jobs](./background-jobs.mdx): detach, inspect logs and cancel.
+- [Lifetime and persistent services](./long-running-services.mdx): set deadlines and restart services after resume.
+- [Scripting and CI](./scripting-ci.mdx): parse output and handle failures.
+- [Disposable CI jobs — unreleased 0.7.0](./ci-jobs.mdx): run and clean up one temporary build.
+
+## Operate
+
+- [Snapshots and forks](./snapshots-forks.mdx): save disk state or make a same-host copy.
+- [Backup and recovery](./backup-recovery.mdx): move a whole box to a compatible host.
+- [Defaults and policies](./config-policy.mdx): set creation defaults and retention rules.
+- [Upgrades](./upgrades.mdx): update the CLI, host worker and image separately.
+- [Troubleshooting](./troubleshooting.mdx): start from a symptom.

package/docs/guides/long-running-services.mdx

@@ -11,7 +11,7 @@ terminal, and detached jobs do not extend that deadline automatically.
 ## Choose a lifetime and retention policy
 
 ```sh title="CONTROLLER: Create a persistent box"
-box=$(raft new --location lab --ttl 3600)
+box=$(raft new --location lab --ttl 3600 --persistent)
 ```
 
 TTL is in seconds: 60 to 2,592,000 (30 days), with a creation default of 600.
@@ -24,13 +24,19 @@ Raft has no option to disable automatic expiry.
 | `resume --ttl SECONDS` | Starts the saved box with a new deadline | Only possible while the box still exists |
 | `destroy` | Deletes the box and snapshots | Deletes the box and snapshots |
 
-Use `raft new --disposable` for temporary work. A fork is persistent unless
+Use `raft new --disposable` for temporary work, or
+`raft config set disposable true` to make it the default on this controller;
+`raft new --persistent` then keeps one box. A fork is persistent unless
 `--disposable` is explicitly supplied; recovered boxes are persistent.
 Back up work before stopping a disposable box.
 
 Persistent stopped boxes still occupy storage. Raft does not cap how many boxes
 a host keeps. `raft prune --older-than 7` previews old stopped boxes;
-`--yes` applies deletion. There is no automatic persistent-box retention cleanup.
+`--yes` applies deletion. To delete them automatically, set a host retention
+policy: with `raft policy lab --delete-stopped-after 48`, the host deletes any
+box left stopped for 48 hours, and `raft list` shows each one's `deleteAt`.
+Resuming a box resets its clock. See
+[Defaults and host policies](./config-policy.mdx#delete-boxes-left-stopped).
 
 ## Read and reset the deadline