@microck/raft 0.4.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/README.md +30 -14
  2. package/deploy/deploy.py +112 -19
  3. package/deploy/install-incus.sh +10 -0
  4. package/deploy/raft-expire.py +109 -31
  5. package/deploy/verify-controls.py +34 -48
  6. package/deploy/verify-features.py +592 -3
  7. package/deploy/verify-fresh-host.py +64 -1
  8. package/deploy/verify-limits.py +327 -72
  9. package/deploy/verify-live.py +56 -31
  10. package/deploy/verify-self-update.py +372 -0
  11. package/docs/boat-capabilities.md +8 -7
  12. package/docs/boot-performance.mdx +1 -1
  13. package/docs/contract.md +33 -23
  14. package/docs/first-box.mdx +1 -1
  15. package/docs/guides/backup-recovery.mdx +2 -2
  16. package/docs/guides/config-policy.mdx +253 -0
  17. package/docs/guides/docker.mdx +9 -1
  18. package/docs/guides/long-running-services.mdx +10 -4
  19. package/docs/guides/meta.json +1 -0
  20. package/docs/guides/project-setup.mdx +4 -3
  21. package/docs/guides/scripting-ci.mdx +27 -10
  22. package/docs/guides/snapshots-forks.mdx +9 -3
  23. package/docs/guides/troubleshooting.mdx +37 -8
  24. package/docs/guides/upgrades.mdx +17 -5
  25. package/docs/index.mdx +1 -0
  26. package/docs/operations.mdx +16 -14
  27. package/docs/reference/capacity.mdx +198 -0
  28. package/docs/reference/cli.md +178 -35
  29. package/docs/reference/configuration.mdx +40 -2
  30. package/docs/reference/meta.json +1 -1
  31. package/docs/releases/0.5.0.md +19 -0
  32. package/docs/releases/0.6.0.md +16 -0
  33. package/docs/requirements.mdx +2 -2
  34. package/docs/verification.mdx +14 -8
  35. package/package.json +1 -1
  36. package/raft.py +1181 -128
  37. package/skills/raft-cli/SKILL.md +79 -13
  38. package/docs/reference/capacity-limits.mdx +0 -92
@@ -14,6 +14,12 @@ Syntax:
14
14
  and `raft --version` prints the installed version without loading configuration.
15
15
  Output is command-specific; see [Scripting and CI](../guides/scripting-ci.mdx).
16
16
 
17
+ At most once a day, after a command finishes, Raft checks GitHub for a newer
18
+ release and prints one line on stderr, such as
19
+ `raft 0.7.0 is available (you have 0.6.0). Run: raft self-update`. It never
20
+ installs anything or changes stdout, and network failures stay silent. See
21
+ [`self-update`](#self-update) to turn it off.
22
+
17
23
  Workspaces are referenced by location-qualified handles formatted as `<location>:<instance-name>` (for example `lab:rf-a1b2c3d4e5f60718`).
18
24
 
19
25
  ## Command index
@@ -28,7 +34,9 @@ Workspaces are referenced by location-qualified handles formatted as `<location>
28
34
  | Port forwarding and desktop | [`forward`](#forward), [`desktop`](#desktop) |
29
35
  | Backup and recovery | [`backup`](#backup), [`recover`](#recover) |
30
36
  | Locations | [`host add`](#host-add), [`locations`](#locations), [`note`](#note) |
31
- | Host inspection and maintenance | [`limits`](#limits), [`doctor`](#doctor), [`list`](#list), [`gc`](#gc), [`prune`](#prune) |
37
+ | Defaults and policies | [`config`](#config), [`policy`](#policy) |
38
+ | Host inspection and maintenance | [`capacity`](#capacity), [`doctor`](#doctor), [`list`](#list), [`gc`](#gc), [`prune`](#prune) |
39
+ | CLI maintenance | [`self-update`](#self-update) |
32
40
 
33
41
  ## Workspace lifecycle commands
34
42
 
@@ -36,19 +44,29 @@ Workspaces are referenced by location-qualified handles formatted as `<location>
36
44
 
37
45
  Create and launch a new workspace container.
38
46
 
39
- `raft new [--location LOCATION] [--ttl TTL] [--cpu {1,2}] [--memory {1GiB,2GiB,4GiB,8GiB}] [--disposable] [--from LOCATION:BOX/SNAPSHOT] [--purpose TEXT [--owner TEXT] [--next TEXT]]`
47
+ `raft new [--location LOCATION] [--ttl TTL] [--cpu N] [--memory SIZE] [--disk SIZE] [--disposable | --persistent] [--from LOCATION:BOX/SNAPSHOT] [--purpose TEXT [--owner TEXT] [--next TEXT]]`
48
+
49
+ - `--location LOCATION`: Target host location in `~/.config/raft/incus.json`. Built-in default: first configured location.
50
+ - `--ttl TTL`: Lifetime in seconds before automatic stop (`60` to `2592000`). Built-in default: `600`.
51
+ - `--cpu N`: CPU limit, any whole number from `1` up to the host's CPU count. Built-in default: `1`, or the snapshot's with `--from`.
52
+ - `--memory SIZE`: Memory limit in whole GiB, for example `16GiB`, up to the host's RAM. Built-in default: `2GiB`, or the snapshot's with `--from`.
53
+ - `--disk SIZE`: Give the box its own disk of SIZE in whole GiB, for example `20GiB`, instead of the shared pool. Needs a host deployed with `--disk-pool`. The box, and Docker inside it, cannot write past SIZE, and the pool reserves SIZE when the box is created. `--disk shared` asks for the shared pool, overriding a configured `disk`. Built-in default: the shared pool, or with `--from` the snapshot's disk. See [Box sizing and own disks](./capacity.mdx#own-box-disks).
40
54
 
41
- - `--location LOCATION`: Target host location in `~/.config/raft/incus.json`. Defaults to first configured location.
42
- - `--ttl TTL`: Lifetime in seconds before automatic stop (`60` to `2592000`). Default: `600`.
43
- - `--cpu {1,2}`: CPU core count managed by Incus. Default: `1`.
44
- - `--memory {1GiB,2GiB,4GiB,8GiB}`: Memory limit. Default: `2GiB`.
55
+ An omitted option takes the controller default set with [`config`](#config),
56
+ then the built-in default. With `--from`, configured `cpu`, `memory` and `disk`
57
+ defaults do not apply: the box keeps the snapshot's size unless a flag overrides
58
+ it, and `--disk shared` is refused.
45
59
 
46
- - `--disposable`: Delete the box and snapshots on stop or expiry. Off by default.
47
- - `--from LOCATION:BOX/SNAPSHOT`: Copy a named snapshot on its source host instead of the configured image. Normal creation sizing applies; the source can be running. Cannot target a different location.
48
- - `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Record a [box note](#note). `--owner` and `--next` need `--purpose`. Without them the box starts with no note, including boxes created `--from` a noted snapshot.
60
+ - `--disposable`: Delete the box and snapshots on stop or expiry. Built-in default: persistent.
61
+ - `--persistent`: Keep the box and its disk on stop or expiry, overriding a configured `disposable` default. Cannot be combined with `--disposable`.
62
+ - `--from LOCATION:BOX/SNAPSHOT`: Copy a named snapshot on its source host instead of the configured image. The box keeps the snapshot's CPU, RAM and disk unless `--cpu`, `--memory` or `--disk` overrides them; the source can be running. Cannot target a different location.
63
+ - `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Record a [box note](#note). `--owner` and `--next` need `--purpose`. Without them the box starts with no note, including boxes created `--from` a noted snapshot. A host [policy](#policy) can require fields.
49
64
 
50
65
  Returns the qualified workspace handle. If the location has a note, `new` prints
51
- it to stderr; stdout still contains only the handle.
66
+ it to stderr; stdout still contains only the handle. Raft limits neither the
67
+ number of boxes nor their size; requests larger than the whole host, own disks
68
+ on a host without `raft-disks`, or own disks larger than the unreserved
69
+ `raft-disks` space, fail with the reason before anything is created.
52
70
 
53
71
  ### `stop`
54
72
 
@@ -62,12 +80,13 @@ Terminates guest processes and stops the container. Persistent boxes retain file
62
80
 
63
81
  Start a stopped workspace with a renewed lifetime deadline.
64
82
 
65
- `raft resume <box> --ttl TTL [--cpu {1,2}] [--memory {1GiB,2GiB,4GiB,8GiB}]`
83
+ `raft resume <box> --ttl TTL [--cpu N] [--memory SIZE]`
66
84
 
67
85
  - `--ttl TTL`: Lifetime in seconds from resumption (`60` to `2592000`). Required.
68
86
 
69
87
  CPU/RAM overrides on resume apply before startup. Omitted values stay unchanged.
70
- A running box is rejected before changing its limits.
88
+ A running box is rejected before changing its limits. Resume never changes the
89
+ box's disk.
71
90
 
72
91
  ### `extend`
73
92
 
@@ -83,7 +102,7 @@ Permanently delete a workspace container and its snapshots.
83
102
 
84
103
  `raft destroy <box>`
85
104
 
86
- Stops the container if running, removes all associated Btrfs snapshots, and releases host storage.
105
+ Stops the container if running, removes all associated snapshots, and releases its storage, including an own disk.
87
106
 
88
107
  ## Execution and inspection commands
89
108
 
@@ -109,13 +128,13 @@ Open an interactive root terminal session inside the workspace over host SSH and
109
128
 
110
129
  ### `info`
111
130
 
112
- Display container metadata from Incus in JSON format, including status, IP addresses, resource limits, and timestamps.
131
+ Display container metadata from Incus in JSON format, including status, IP addresses, resource limits, and timestamps. `expanded_devices.root` shows the disk: `pool: raft-data` for the shared pool, or `pool: raft-disks` with the own disk's `size`.
113
132
 
114
133
  `raft info <box>`
115
134
 
116
135
  ### `usage`
117
136
 
118
- Display active resource consumption for a running workspace: cgroup memory, CPU microseconds, CPU affinity, and shared-pool disk space.
137
+ Display active resource consumption for a running workspace: cgroup memory, CPU microseconds, CPU affinity, and filesystem space. `filesystemScope` says whether the space is the shared pool or the box's own disk.
119
138
 
120
139
  `raft usage <box>`
121
140
 
@@ -175,6 +194,8 @@ Create a named point-in-time snapshot of a stopped workspace. Names use alphanum
175
194
 
176
195
  `raft snapshot <box> <name>`
177
196
 
197
+ A snapshot of a box with its own disk is a thin snapshot that reserves the box's disk size in `raft-disks`; it fails before anything is created when the pool lacks unreserved space.
198
+
178
199
  ### `snapshots`
179
200
 
180
201
  List existing snapshots and creation timestamps for a workspace.
@@ -197,11 +218,13 @@ Roll back a stopped workspace filesystem to a named snapshot using `--diskonly`,
197
218
 
198
219
  Clone a stopped workspace into an independent container on the same host and start it.
199
220
 
200
- `raft fork <box> --ttl TTL [--cpu {1,2}] [--memory {1GiB,2GiB,4GiB,8GiB}] [--disposable]`
221
+ `raft fork <box> --ttl TTL [--cpu N] [--memory SIZE] [--disposable] [--purpose TEXT [--owner TEXT] [--next TEXT]]`
201
222
 
202
223
  - `--ttl TTL`: Lifetime in seconds for the clone. Required.
224
+ - `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Replace the clone's whole note. Without them the clone keeps the source note.
203
225
 
204
- Omitted CPU/RAM values inherit the source limits. The clone is persistent unless `--disposable` is supplied, regardless of the source policy.
226
+ Omitted CPU/RAM values inherit the source limits; controller defaults never apply to forks. The clone keeps the source's disk: the shared pool, or an own disk of the same size as a thin snapshot, which reserves that size in `raft-disks`. To copy a box onto a disk of another size, snapshot it and use `new --from BOX/SNAPSHOT --disk SIZE`. The clone is persistent unless `--disposable` is supplied, regardless of the source and of a configured `disposable` default.
227
+ The note the clone ends up with must satisfy the host [policy](#policy).
205
228
 
206
229
  Returns the qualified handle of the clone.
207
230
 
@@ -240,12 +263,17 @@ Export a stopped workspace and its snapshots to a compressed archive on the cont
240
263
 
241
264
  Import a portable backup archive onto a host as a fresh stopped workspace.
242
265
 
243
- `raft recover <source> --location LOCATION`
266
+ `raft recover <source> --location LOCATION [--purpose TEXT [--owner TEXT] [--next TEXT]]`
244
267
 
245
268
  - `<source>`: Path to archive file on controller.
246
269
  - `--location LOCATION`: Target host location. Required.
270
+ - `--purpose TEXT`, `--owner TEXT`, `--next TEXT`: Replace the recovered box's whole note. Without them it keeps the archived note.
271
+
272
+ Under a host [policy](#policy), a given note is checked before the upload. An
273
+ archived note is checked after import; if it fails, the imported box is deleted
274
+ in the same locked step.
247
275
 
248
- Returns the qualified handle of the imported stopped workspace.
276
+ Returns the qualified handle of the imported stopped workspace. A backup of a box with its own disk is recovered onto an own disk of the same size in `raft-disks`, which must exist on the destination and have room to reserve that size for the box and each of its snapshots; any other backup lands in the shared pool.
249
277
 
250
278
  ## Location commands
251
279
 
@@ -253,19 +281,20 @@ Returns the qualified handle of the imported stopped workspace.
253
281
 
254
282
  Check, provision and register a new workspace host, then prove it works.
255
283
 
256
- `raft host add LOCATION --ssh TARGET [--image-release TAG | --image-archive PATH | --build] [--dry-run] [--yes]`
284
+ `raft host add LOCATION --ssh TARGET [--image-release TAG | --image-archive PATH | --build] [--disk-pool SIZE] [--dry-run] [--yes]`
257
285
 
258
286
  - `LOCATION`: New location name. Existing locations are rejected.
259
287
  - `--ssh TARGET`: Key-based SSH target, such as `ubuntu@192.0.2.10` or an `~/.ssh/config` alias. Accept the host key interactively first.
260
288
  - `--image-release TAG`: Download `raft-dev-<arch>.tar.gz` from this release on the host and verify it against the release `SHA256SUMS`. Default: the CLI's own version.
261
289
  - `--image-archive PATH`: Upload and import a local image archive instead.
262
290
  - `--build`: Build the image on the host from Debian 13 instead.
291
+ - `--disk-pool SIZE`: Enable [own box disks](./capacity.mdx#own-box-disks) with a `raft-disks` thin pool of SIZE, in whole GiB. Default: not enabled. The pool's file takes its full size on the host disk when created. To enable them on a host that is already deployed, run `python3 deploy/deploy.py --location LOCATION --disk-pool SIZE`. An existing pool keeps its size; a different explicit value is refused.
263
292
  - `--dry-run`: Probe the host and print the plan without changing anything.
264
293
  - `--yes`: Apply the plan without the interactive prompt. Without a terminal, `--yes` is required.
265
294
 
266
295
  Before any change, a read-only probe checks SSH, passwordless sudo, Ubuntu 22.04 or
267
296
  24.04, ARM64 or AMD64, python3, at least 2 CPUs, 8 GiB RAM and, for a fresh host,
268
- 80 GiB free disk. Unsupported targets, including macOS, fail with the reasons and
297
+ 80 GiB of free disk, plus the `--disk-pool` size when given. Unsupported targets, including macOS, fail with the reasons and
269
298
  no changes. The command then runs the deployer, imports the image only if its
270
299
  checksum and architecture match, adds the location with the pinned fingerprint,
271
300
  and runs `doctor` plus a disposable create/exec/destroy smoke test. If the final
@@ -301,7 +330,7 @@ Set or clear the note of a location or a box.
301
330
 
302
331
  Location notes are operator guidance stored in `~/.config/raft/incus.json`. Use
303
332
  them to record which host is preferred or what a host should not run. Raft shows
304
- them in `locations`, `limits` and `new`.
333
+ them in `locations`, `capacity` and `new`.
305
334
 
306
335
  Box notes tell other agents and people why a box exists. They follow one
307
336
  template, stored on the box as JSON in the Incus key `user.raft.note`:
@@ -313,28 +342,103 @@ template, stored on the box as JSON in the Incus key `user.raft.note`:
313
342
  | `next` | What happens to it next | `Destroy after the tests pass` |
314
343
 
315
344
  Each call replaces the whole note, on running or stopped boxes. `raft list`
316
- shows it as `note`. Forks and recovered backups keep the source box's note;
317
- update it if the copy has a different job.
345
+ shows it as `note`. Forks and recovered backups keep the source box's note
346
+ unless `fork` or `recover` gets note flags; update it if the copy has a
347
+ different job.
348
+
349
+ Empty values are rejected; use `--clear` to remove a note. A host
350
+ [policy](#policy) can require fields, and then rejects notes and `--clear` that
351
+ would leave them out. Otherwise Raft does not interpret notes.
352
+
353
+ ### `config`
354
+
355
+ Show or change the controller defaults for `new`.
356
+
357
+ `raft config`
358
+ `raft config set KEY VALUE`
359
+ `raft config unset KEY`
360
+
361
+ - `KEY`: One of `location`, `ttl`, `cpu`, `memory`, `disk`, `disposable`.
362
+ - `VALUE`: Accepted exactly when the same text is accepted by the matching `new` flag. A `location` must be configured. `disposable` takes `true` or `false`.
363
+
364
+ `disposable true` makes `new` create disposable boxes, including `new --from`,
365
+ because a snapshot carries no lifecycle; `--persistent` keeps one box. `fork`
366
+ ignores it: a fork is persistent unless it gets `--disposable`.
367
+
368
+ `cpu`, `memory` and `disk` size only boxes created from the image. `new --from`
369
+ and `fork` keep the source's CPU, RAM and disk; only explicit flags change them.
370
+ A `disk` default on a location without `raft-disks`, or larger than its
371
+ unreserved space, fails `new` with the same error as `--disk`, plus a sentence
372
+ naming `defaults.disk` as the source. `--disk shared` overrides it for one box.
373
+
374
+ Defaults live in `~/.config/raft/config.json` as `{"defaults": {"KEY": VALUE}}`.
375
+ `set` and `unset` print the old and new value, write atomically under a
376
+ controller lock and keep mode `0600`. An unknown key, bad value or malformed
377
+ file makes `new` fail with the file path and key; `set` or `unset` that key to
378
+ repair it.
379
+
380
+ `raft config` prints one JSON object: `file`, `defaults` (each key's `value` and
381
+ `source`, `config` or `built-in`) and `policy`. For each location, `policy` has
382
+ `require` (the required note fields), `deleteStoppedAfterHours` (`null` without
383
+ retention), `deleteStoppedAfterSince` (when retention was set, Unix seconds) and
384
+ the last change records `requireChanged` and `deleteStoppedAfterChanged`. It
385
+ contacts every configured host to read policies.
386
+ See [Defaults and host policies](../guides/config-policy.mdx).
387
+
388
+ ### `policy`
389
+
390
+ Set or clear a location's host policies: the box note fields it requires and
391
+ how long it keeps stopped boxes.
392
+
393
+ `raft policy LOCATION [--require FIELDS|none] [--delete-stopped-after HOURS|never]`
318
394
 
319
- Notes are advisory; Raft never enforces them. Empty values are rejected; use
320
- `--clear` to remove a note.
395
+ - `--require FIELDS`: Comma-separated fields from `purpose`, `owner`, `next`. Must include `purpose`. `--require none` stops requiring fields.
396
+ - `--delete-stopped-after HOURS`: Delete boxes that stay stopped for HOURS, a positive whole number, with their snapshots. `--delete-stopped-after never` keeps stopped boxes.
397
+
398
+ Give either option or both; each changes only its own policy. Both values are
399
+ checked before the host changes.
400
+
401
+ Policies are stored on the host in the Incus project `raft`, so they apply to
402
+ every controller. The command prints each change and the host records the time,
403
+ SSH account and controller in a `.changed` key beside the policy and in the host
404
+ journal (`journalctl -t raft`).
405
+
406
+ The note policy lives in `user.raft.require`. `new`, `fork`, `recover` and
407
+ `note` check it on the controller for an actionable error, then again in the
408
+ host transaction that writes the note.
409
+
410
+ Retention lives in `user.raft.delete-stopped-after`, with the time it was set in
411
+ `user.raft.delete-stopped-after.since`. The host expiry worker deletes a stopped
412
+ box once HOURS have passed since the later of its stop and that time, so a new
413
+ or shorter policy never deletes a box sooner than HOURS after the change. The
414
+ worker runs every 15 seconds and on [`gc`](#gc), logs each deletion to the host
415
+ journal and never touches running boxes. A stopped box with no recorded stop
416
+ time, such as one powered off from inside, stopped with native `incus` or
417
+ recovered from a backup, starts its clock when the worker first sees it.
418
+ [`list`](#list) shows each box's `deleteAt`. Setting retention needs the current
419
+ host worker; on an older one it fails with redeploy guidance.
420
+
421
+ Policies are guardrails for agents, not a security boundary: SSH and sudo on the
422
+ host can change or bypass them, and older controllers do not check note policies.
321
423
 
322
424
  ## Host inspection and maintenance commands
323
425
 
324
- ### `limits`
426
+ ### `capacity`
325
427
 
326
- Inspect host resource capacity, active allocations, and recommended workspace limits.
428
+ Estimate how many more boxes a host can run well. The report is advice: Raft
429
+ enforces no box count or size budget, so create the boxes your task needs.
327
430
 
328
- `raft limits [--location LOCATION] [--json]`
431
+ `raft capacity [--location LOCATION] [--json]`
329
432
 
330
433
  - `--location LOCATION`: Filter report to a specific host.
331
434
  - `--json`: Output report as JSON.
332
435
 
333
- Each report includes the location's `note`, if any.
436
+ Each report leads with that advisory notice and includes the location's `note`,
437
+ if any. See [Box sizing and capacity](./capacity.mdx) for the fields.
334
438
 
335
439
  ### `doctor`
336
440
 
337
- Inspect host connectivity, Incus version, expiry-worker compatibility, active systemd units (`incus`, `raft-expire.timer`, `raft-network.service`), root filesystem usage, storage pool status, and saved box count across configured hosts. An incompatible worker returns a nonzero exit status and names the failing location without running expiry or modifying boxes.
441
+ Inspect host connectivity, Incus version, expiry-worker compatibility, active systemd units (`incus`, `raft-expire.timer`, `raft-network.service`), root filesystem usage, status of the shared `raft-data` pool and, when own box disks are enabled, the `raft-disks` pool, and saved box count across configured hosts. Without `raft-disks` it prints `own box disks: not enabled (redeploy with --disk-pool SIZE)` and does not fail. An incompatible worker returns a nonzero exit status and names the failing location without running expiry or modifying boxes.
338
442
 
339
443
  `raft doctor`
340
444
 
@@ -344,14 +448,17 @@ List existing workspace containers across configured hosts.
344
448
 
345
449
  `raft list [--location LOCATION]`
346
450
 
347
- Prints one JSON object per box (JSONL), with `box`, `state`, `stopAt` and `note` fields.
451
+ Prints one JSON object per box (JSONL), with `box`, `state`, `stopAt`, `deleteAt`, `disk` and `note` fields.
452
+ `deleteAt` is the Unix time when the location's [retention](#policy) deletes a
453
+ stopped box, as a string like `stopAt`, or `""` when no deletion is due.
454
+ `disk` is `"shared"` for the shared pool or the own disk's size, such as `"20GiB"`.
348
455
  `note` is the box note object, or `null` when the box has none.
349
456
  An empty inventory prints no records. See [Scripting and CI](../guides/scripting-ci.mdx)
350
457
  for output formats, exit codes, readiness, and failure handling.
351
458
 
352
459
  ### `gc`
353
460
 
354
- Enforce deadlines immediately. Persistent boxes stop; disposable boxes are deleted.
461
+ Enforce deadlines and retention immediately. Persistent boxes stop; disposable boxes are deleted; stopped boxes past the location's [retention](#policy) are deleted.
355
462
 
356
463
  `raft gc [--location LOCATION]`
357
464
 
@@ -361,6 +468,42 @@ Preview deletion of persistent stopped boxes older than the selected number of d
361
468
 
362
469
  `raft prune --older-than DAYS [--location LOCATION] [--yes]`
363
470
 
364
- Days must be at least one. Without `--yes`, this only prints JSONL candidates. With `--yes`, it deletes matching boxes and their snapshots under the host lock. Running boxes and boxes without a recorded stop time are excluded. Stop timestamps come from Raft stop/expiry, not box creation. No automatic retention policy is enabled.
471
+ Days must be at least one. Without `--yes`, this only prints JSONL candidates. With `--yes`, it deletes matching boxes and their snapshots under the host lock. Running boxes and boxes without a recorded stop time are excluded. Stop timestamps come from Raft stop/expiry, or from the host worker when it first sees a box stopped another way, never from box creation. For automatic deletion, set a [retention policy](#policy).
472
+
473
+ Disposable creation/fork, prune and setting retention require the current expiry worker installed through the deployer. A mismatch fails with deployment guidance; the CLI does not install or adapt an older worker.
474
+
475
+ ## CLI maintenance commands
476
+
477
+ ### `self-update`
478
+
479
+ Upgrade the installed CLI to the latest GitHub release, or to a chosen one.
480
+
481
+ `raft self-update [--check | --version X.Y.Z]`
482
+
483
+ - `--check`: Report whether a newer release exists without installing it. Exits `0` when the CLI is current and `100` when an update is available.
484
+ - `--version X.Y.Z`: Install this release instead of the latest, for example to roll back. Without it, `self-update` never downgrades.
485
+
486
+ The latest release on GitHub decides what is current. `self-update` installs
487
+ through the channel that installed the running copy and never switches channel:
488
+
489
+ | Installed with | Detected by | Upgrade | Integrity |
490
+ | --- | --- | --- | --- |
491
+ | `npm install -g @microck/raft` | `PREFIX/lib/node_modules/@microck/raft`, linked from `PREFIX/bin/raft` | `npm install --global --prefix PREFIX @microck/raft@X.Y.Z` | npm checks the registry integrity hash; Raft publishes with npm provenance |
492
+ | `uv tool install` of a release wheel | A uv tool environment named `raft` | Downloads the wheel and `SHA256SUMS` from the release, then `uv tool install --force` | Refuses a wheel whose SHA-256 differs from the release `SHA256SUMS` |
493
+
494
+ Source checkouts, editable installs, pip and other layouts are refused with
495
+ instructions. For a uv tool, `uv tool dir` must still name the original tool
496
+ directory; set `UV_TOOL_DIR` if it moved.
497
+
498
+ `self-update` prints `raft OLD -> NEW` and the release notes link. A current CLI
499
+ is a no-op. Hosts are not changed: run [`doctor`](#doctor) next. If a host's
500
+ worker differs from the new CLI, redeploy that host from the same release; see
501
+ [Upgrades and image changes](../guides/upgrades.mdx). After an update,
502
+ [`host add`](#host-add) downloads the new release's image by default. Existing
503
+ image pins stay unchanged.
365
504
 
366
- Disposable creation/fork and prune require the current expiry worker installed through the deployer. A mismatch fails with deployment guidance; the CLI does not install or adapt an older worker.
505
+ Set `RAFT_NO_UPDATE_CHECK=1` to turn off the daily release notice, for example in
506
+ pinned or scripted runs. The notice caches its last result in
507
+ `$XDG_CACHE_HOME/raft/update.json` (default `~/.cache/raft/update.json`). It
508
+ waits at most 2 seconds for GitHub and is skipped for `self-update` and
509
+ `--version`.
@@ -40,7 +40,7 @@ The configuration file maps location names to host definition objects:
40
40
  ### Location keys
41
41
 
42
42
  - **Naming format**: Matches `^[a-zA-Z0-9][a-zA-Z0-9_-]*$`. Starts with an alphanumeric character and contains ASCII letters, digits, underscores, or hyphens.
43
- - **Default location**: The first location listed is the default for `raft new`. `raft list`, `raft limits`, `raft gc`, and `raft prune` accept `--location` filters. Without a filter they check all configured hosts; `raft doctor` always checks all configured hosts.
43
+ - **Default location**: The first location listed is the default for `raft new`. `raft list`, `raft capacity`, `raft gc`, and `raft prune` accept `--location` filters. Without a filter they check all configured hosts; `raft doctor` always checks all configured hosts.
44
44
 
45
45
  ### Host entry fields
46
46
 
@@ -60,7 +60,7 @@ raft note spare --clear
60
60
  ```
61
61
 
62
62
  `raft note` rewrites the file with two-space indentation, keeps location order and
63
- mode `0600`. `raft locations`, `raft limits` and `raft new` show the note.
63
+ mode `0600`. `raft locations`, `raft capacity` and `raft new` show the note.
64
64
 
65
65
  <Callout type="warn" title="Replace placeholder image fingerprint">
66
66
  The `image` field requires the full 64-character hash for `raft new`; aliases such as `raft-dev` are rejected. Replace the example hash with the actual imported image fingerprint for that host.
@@ -68,6 +68,44 @@ The `image` field requires the full 64-character hash for `raft new`; aliases su
68
68
  ARM64 and AMD64 images produce different fingerprints; do not reuse an ARM64 hash on an AMD64 host.
69
69
  </Callout>
70
70
 
71
+ ## Defaults for new
72
+
73
+ `~/.config/raft/config.json` is optional. It sets defaults for `raft new` on this
74
+ controller and is separate from the location inventory:
75
+
76
+ ```json title="Example ~/.config/raft/config.json"
77
+ {
78
+ "defaults": {
79
+ "location": "lab",
80
+ "ttl": 3600,
81
+ "cpu": 2,
82
+ "memory": "4GiB",
83
+ "disk": "20GiB",
84
+ "disposable": true
85
+ }
86
+ }
87
+ ```
88
+
89
+ | Key | Accepted values | Built-in default |
90
+ | :--- | :--- | :--- |
91
+ | `location` | A configured location name | First configured location |
92
+ | `ttl` | `60` to `2592000` seconds | `600` |
93
+ | `cpu` | A positive whole number | `1` |
94
+ | `memory` | Whole GiB, e.g. `16GiB` | `2GiB` |
95
+ | `disk` | Whole GiB for an own box disk, or `shared` | `shared` |
96
+ | `disposable` | `true` or `false` | `false` (persistent) |
97
+
98
+ Explicit flags win over these values; `--persistent` overrides `disposable`.
99
+ `cpu`, `memory` and `disk` size only boxes created from the image: `new --from`
100
+ and `fork` keep the source's size unless a flag overrides it. `disposable`
101
+ applies to `new` and `new --from`, never to `fork`. A `disk` default needs the `raft-disks` pool on the target
102
+ host, like `--disk`. Unknown keys and invalid values make
103
+ `raft new` fail with the file path and key. Edit the file with
104
+ `raft config set KEY VALUE` and `raft config unset KEY`, and inspect it with
105
+ `raft config`. Required box note fields and stopped-box retention are host
106
+ policies, not part of this file; see
107
+ [Defaults and host policies](../guides/config-policy.mdx).
108
+
71
109
  ## SSH configuration and connection behavior
72
110
 
73
111
  Primary remote commands (`raft ssh`, `upload`, `download`, and control commands) run OpenSSH with `BatchMode=yes` and `-o ConnectTimeout=10` to fail fast on unreachable hosts. The timeout applies to initial connection establishment, not session or transfer duration.
@@ -5,6 +5,6 @@
5
5
  "pages": [
6
6
  "cli",
7
7
  "configuration",
8
- "capacity-limits"
8
+ "capacity"
9
9
  ]
10
10
  }
@@ -0,0 +1,19 @@
1
+ ---
2
+ title: Raft 0.5.0
3
+ ---
4
+
5
+ # Raft 0.5.0
6
+
7
+ - **No box cap.** A location no longer stops at four saved boxes. Agents create the boxes their task needs; creation only fails for real reasons such as a full pool.
8
+ - **Any size.** `--cpu N` and `--memory NGiB` accept any positive size up to the host's CPUs and RAM. A box from the image gets 1 CPU, 2 GiB and the shared disk; `new --from` and `fork` keep the source's CPU, RAM and disk, and each flag overrides only its own value.
9
+ - **Own box disks.** `raft new --disk SIZE` gives a box its own ext4 disk with a hard cap that also holds for Docker inside the box. Own disks are opt-in per host: redeploy with `--disk-pool SIZE` to create the thin LVM pool `raft-disks`. New boxes, forks, template copies and snapshots are copy-on-write, and a reservation check refuses any request that would overcommit the pool. `raft list` shows each box's `disk`.
10
+ - **`raft capacity` replaces `raft limits`.** It is advisory only: it reports host CPUs, RAM, allocations, both pools and how many more boxes of example sizes fit, and never limits creation. JSON fields changed (`boxes`, `estimates`, `sharedPool`, `boxDiskPool`, `headroom`, `notice`).
11
+ - **`raft self-update`.** Updates the CLI through the channel that installed it: npm, or a uv tool from the release wheel verified against `SHA256SUMS`. `--check` reports without installing (exit 100 when an update exists) and `--version X.Y.Z` installs a specific release. A notice on stderr at most once a day says when a newer release exists; `RAFT_NO_UPDATE_CHECK=1` turns it off. It never installs on its own and never changes hosts.
12
+ - **Controller defaults.** `raft config`, `raft config set KEY VALUE` and `raft config unset KEY` manage `~/.config/raft/config.json` with defaults for `location`, `ttl`, `cpu`, `memory` and `disk`. A flag beats the config, which beats the built-in default. `--disk shared` overrides a configured disk default for one box.
13
+ - **Note policies.** `raft policy LOCATION --require purpose,owner` makes a host require those box note fields. `new`, `fork`, `recover` and `note` refuse a box without them, with an error naming the missing flag, and the host checks again under its lock. Policy changes are recorded on the host and in its journal. This is a guardrail for agents, not a security boundary.
14
+
15
+ Install on Linux with Python 3.11.8+ using `npm install -g @microck/raft@0.5.0`, or install the attached Python wheel; existing installs can run `raft self-update`.
16
+
17
+ Redeploy each host with this release's `deploy/deploy.py`. It installs `lvm2` and `python3-yaml`, which `raft recover` now needs to read backups, and changes nothing else unless you add `--disk-pool SIZE` to enable own box disks. That needs SIZE plus 20 GiB free on the host; existing storage is never resized.
18
+
19
+ ARM64 and AMD64 development images accompany this release. Verify `SHA256SUMS` before importing an image for your host architecture.
@@ -0,0 +1,16 @@
1
+ ---
2
+ title: Raft 0.6.0
3
+ ---
4
+
5
+ # Raft 0.6.0
6
+
7
+ - **Disposable by default.** `raft config set disposable true` makes `raft new` create disposable boxes, which are deleted with their snapshots when they stop or expire. `raft new --persistent` keeps one box anyway, and `--disposable` still works the other way. The default also applies to `new --from`; a fork stays persistent unless you pass `--disposable`. `raft config` shows the setting.
8
+ - **Delete boxes left stopped.** `raft policy LOCATION --delete-stopped-after 48` makes a host delete boxes, with their snapshots, once they have been stopped for 48 hours. The host applies it every 15 seconds and on `raft gc`, even with no controller online, logs each deletion to its journal (`journalctl -t raft`) and never touches running boxes. A new or shorter policy never deletes a box sooner than its hours after the change, and resuming a box resets its clock. `raft list` shows each stopped box's `deleteAt`. `--delete-stopped-after never` turns it off.
9
+ - **Stop times for every stopped box.** The host now records a stop time for boxes stopped outside `raft stop`, such as a poweroff inside the box, a native `incus stop` or a recovered backup, when it first sees them. `raft prune` can therefore select them later; before, it skipped them forever.
10
+ - **`raft policy` changes.** `--require` and `--delete-stopped-after` can be set together or alone, and each changes only its own policy. `--require none` replaces `--clear`, which is gone. In `raft config`, each location's `changed` field is now `requireChanged`, beside the new `deleteStoppedAfterHours`, `deleteStoppedAfterSince` and `deleteStoppedAfterChanged`.
11
+
12
+ Install on Linux with Python 3.11.8+ using `npm install -g @microck/raft@0.6.0`, or install the attached Python wheel; existing installs can run `raft self-update`.
13
+
14
+ Redeploy each host with this release's `deploy/deploy.py`. The host expiry worker changed, and until a host is redeployed `raft doctor` reports it, while `new --disposable` (including a disposable default), `fork --disposable`, `raft prune` and `raft policy --delete-stopped-after HOURS` fail on it with redeploy guidance. Redeploying changes nothing else on the host and sets no retention policy.
15
+
16
+ ARM64 and AMD64 development images accompany this release. Verify `SHA256SUMS` before importing an image for your host architecture.
@@ -17,11 +17,11 @@ Each workspace host must be a dedicated physical machine or virtual instance:
17
17
  | **Operating system** | Ubuntu 22.04 LTS or 24.04 LTS | The deployer installs Incus 6.0 LTS and nftables bridge isolation rules. |
18
18
  | **CPU** | At least 2 CPU cores | Capacity calculations reserve 1 CPU core for the host OS and services (an advisory estimate, not an enforced host reservation). |
19
19
  | **RAM** | At least 8 GiB | Sizing calculations reserve the larger of 2 GiB or 10% of host RAM (an advisory estimate, not an enforced host reservation). |
20
- | **Free disk** | At least 80 GiB on host disk | The deployer provisions a 60 GiB dedicated Btrfs pool (`raft-data`). |
20
+ | **Free disk** | At least 80 GiB on host disk | For the 60 GiB shared Btrfs pool (`raft-data`), images and the host. [Own box disks](./reference/capacity.mdx#own-box-disks) are optional: `--disk-pool SIZE` adds a `raft-disks` pool of SIZE, so a fresh host then needs 80 GiB plus SIZE, and an existing host SIZE plus 20 GiB. |
21
21
 
22
22
  `raft host add` checks these requirements read-only before it changes a host.
23
23
 
24
- Host CPU and RAM reserves are advisory estimates for `raft limits`, not enforced cgroup locks.
24
+ Host CPU and RAM reserves are advisory estimates for `raft capacity`, not enforced cgroup locks. Raft limits neither how many boxes a host runs nor their size.
25
25
 
26
26
  <Callout type="warn" title="Dedicated host requirement">
27
27
  Do not deploy Raft on hosts with existing Incus storage pools or shared virtualization workloads. Deployment configures dedicated nftables rules, storage pools, and system services.
@@ -18,11 +18,12 @@ Storage accounting covers shared pools rather than per-box quotas. Large backup
18
18
  | --- | --- |
19
19
  | Create/list/info/delete | Qualified handle, native state, ownership and cleanup |
20
20
  | Root terminal/exec | Real PTY, workspace directory, literal arguments and exit code |
21
- | Resource sizing | Effective CPU affinity and cgroup memory limit |
22
- | Host capacity | Mixed-size budgets, stopped/frozen boxes, low RAM/disk and four saved slots; live create/stop/resume/destroy changes in CPU/memory allocations |
21
+ | Resource sizing | Effective CPU affinity and cgroup memory limit for free-form sizes, and refusal of CPU or RAM beyond the host |
22
+ | Host capacity | Advisory notice, arbitrary sizes, no box count limit, example shapes, stopped/frozen boxes, low RAM/disk, hosts without own box disks and thin pool metadata warnings; live create/stop/resume/destroy changes in CPU/memory allocations |
23
+ | Own box disks | Guest and Docker writes stop at the disk size without changing pool reservations or taking shared-pool space; too-small disks and requests beyond the unreserved pool space fail before leaving anything behind; new boxes, forks and templates are thin snapshots of their source, and a template copy writes under 1 GiB; each reserves exactly its size; fork, template, resized template, snapshot and backup/recover keep the disk; on a fresh host, own box disks are off until a redeploy with `--disk-pool` enables them |
23
24
  | Background jobs | Journal output, nonzero exit status, cancellation of child processes, inspection after cancellation or log rotation, and rejection of unknown job IDs |
24
25
  | TTL/extend/resume | Host-enforced stop, manual `gc`, retained files, uninterrupted lifetime extension and 30-day resume |
25
- | Usage | Real cgroup counters and explicitly shared-pool disk scope |
26
+ | Usage | Real cgroup counters and shared-pool or own-disk scope |
26
27
  | File transfer | Binary content, paths containing spaces, stopped-workspace transfers, preservation of the destination after a failed download, and real controller write-error rejection |
27
28
  | Docker | Real image build and container execution |
28
29
  | Snapshot/restore/fork | File rollback, independent copy, retained network identity and deterministic concurrent-resume rejection under the real host lock |
@@ -33,7 +34,7 @@ Storage accounting covers shared pools rather than per-box quotas. Large backup
33
34
  | Development image | Real developer-user tool checks, immutable selection and credential-path archive rejection |
34
35
  | Deployment/firewall | Repeat deployment removes stale rules without restarting Incus; failed filtering blocks normal and socket startup; boot timestamps prove filtering precedes Incus |
35
36
  | Storage contract | Disposable oversized pool is rejected without automatic resizing |
36
- | CLI options and errors | Help for all 31 commands and `host add`; default location, TTL and desktop port; all eight sizing profiles; global and per-location reports; invalid TTLs, sizes, required arguments, handles, paths, snapshot names, jobs and ports |
37
+ | CLI options and errors | Help for all 34 commands and `host add`; default location, TTL and desktop port; all eight sizing profiles; global and per-location reports; invalid TTLs, sizes, required arguments, handles, paths, snapshot names, jobs and ports |
37
38
 
38
39
  ## Command and option audit (2026-10-06)
39
40
 
@@ -57,7 +58,7 @@ The [final runtime validation](https://github.com/Microck/raft/actions/runs/3754
57
58
 
58
59
  ## New workspace controls
59
60
 
60
- `python3 deploy/verify-features.py --location lab` verifies `doctor` readiness against the installed worker, recursive trees and unsafe-archive rejection, templates from a running source, snapshot deletion, CPU/RAM overrides, disposable timer deletion, and retention preview/apply. It uses only its own fixtures. Run the archive checks without `--location` to avoid allocating boxes.
61
+ `python3 deploy/verify-features.py --location lab` verifies `doctor` readiness against the installed worker, recursive trees and unsafe-archive rejection, templates from a running source, snapshot deletion, CPU/RAM overrides, disposable timer deletion, prune preview/apply, host note policies for `new`, `fork`, `note` and `recover`, and stopped-box retention: it sets a 1-hour policy, backdates one stopped box and the policy start, and checks that the next timer run deletes only that box, records the stop time of a box stopped natively and logs to the journal. It sets policies on the host and clears them at the end, and uses only its own fixtures. Every live suite runs the CLI in a private `HOME` holding only `incus.json`, so controller defaults such as `disposable true` never change their boxes. Run the archive, controller-default, policy-message and host worker retention checks without `--location` to avoid allocating boxes; the worker check runs the real worker against a fake `incus` on `PATH`.
61
62
 
62
63
  The live suite also holds a controller stdin pipe open while running real remote
63
64
  readers through both captured and streamed lifecycle transactions. Both must
@@ -87,8 +88,9 @@ The source-race fix also passed a real concurrent writer test that swaps files a
87
88
  Test commands require configured hosts and immutable image fingerprints. Suites destroy only the fixtures they create.
88
89
 
89
90
  Allocation and archive inspection:
90
- - `python3 deploy/verify-limits.py`: Checks allocation policies without a host. Add `--location lab` to verify live create, stop, resume, and destroy reporting.
91
+ - `python3 deploy/verify-limits.py`: Checks advisory capacity policies without a host. Add `--location lab` to verify live create, stop, resume, and destroy reporting, oversized-request refusal, and own box disks (needs a `raft-disks` pool with 42 GiB unreserved).
91
92
  - `python3 deploy/verify-image-tests.py`: Inspects archive structures, including symlinks, hardlinks, FIFOs, and credential detection.
93
+ - `python3 deploy/verify-self-update.py`: Upgrades real npm and uv installs in temporary directories from a local release server and npm registry. Checks SHA256SUMS refusals, install-channel detection, version comparison and the passive notice. Needs npm and uv, but no network or host.
92
94
 
93
95
  Live host suites (expand the check you need):
94
96
 
@@ -137,6 +139,8 @@ python3 deploy/verify-controls.py --source lab --target second-host --directory
137
139
 
138
140
  </details>
139
141
 
142
+ Recovery verification also sets a note policy on the target, proves that a backup without the required note is rejected and its imported box removed, recovers it with a note, then clears the policy.
143
+
140
144
  `verify-tools.py <qualified-box>` executes 31 tool checks as the unprivileged `developer` user inside a running workspace. It compiles and runs code across installed runtimes and package managers without external agent authentication.
141
145
 
142
146
  ## Extended lifecycle results (2026-10-05)
@@ -152,6 +156,8 @@ Desktop testing validated keyboard input, exact clipboard synchronization, point
152
156
 
153
157
  **Admission and backup runs**
154
158
 
159
+ These runs predate the removal of the saved-box admission cap.
160
+
155
161
  The four-saved-box admission test passed, including rejection of a fifth create, fork, or recovery attempt. Full development-image recovery succeeded with a 5,254,224,965-byte archive, verifying inventories, tool checks, permissions, fresh MACs, and snapshot rollback.
156
162
 
157
163
  The execution wrapper reported exit 143 after the success message; independent native inventory and archive checks confirmed that all fixtures and temporary archives were removed.
@@ -210,7 +216,7 @@ The [release run](https://github.com/Microck/raft/actions/runs/37469540182) vali
210
216
 
211
217
  Following a VM reboot, changed kernel boot IDs and active services were verified alongside retained files, snapshots, container states, Docker, and bridge isolation. The host test uses a sparse 100 GiB virtual disk and accessible KVM.
212
218
 
213
- It preserves the production deployer's 80 GiB free-disk check but does not prove physical storage capacity. ARM64 host reboot, host loss, and cross-architecture recovery remain unverified. The VM, SSH keys, and controller configuration were disposable; existing operator hosts were not rebooted or reconfigured.
219
+ It preserves the production deployer's 80 GiB free-disk check, and after enabling own box disks by redeploy checks that the pool is thin, does not pass discards to its file and has that file fully allocated, but does not prove physical storage capacity. ARM64 host reboot, host loss, and cross-architecture recovery remain unverified. The VM, SSH keys, and controller configuration were disposable; existing operator hosts were not rebooted or reconfigured.
214
220
 
215
221
  </details>
216
222
 
@@ -230,7 +236,7 @@ Packaging workflows passed on Python 3.11 and 3.14. Guidelines for visibility ch
230
236
  <details>
231
237
  <summary>Test evidence and run details</summary>
232
238
 
233
- On 2026-10-06, `raft limits` passed read-only text and JSON inspection and live lifecycle tests on two ARM64 Ubuntu hosts. Tests verified allocation adjustments during create, stop, resume, and destroy cycles.
239
+ On 2026-10-06, `raft limits` (since renamed `raft capacity`, with the admission cap removed) passed read-only text and JSON inspection and live lifecycle tests on two ARM64 Ubuntu hosts. Tests verified allocation adjustments during create, stop, resume, and destroy cycles.
234
240
 
235
241
  Allocation fixtures validated mixed sizing budgets, frozen containers, low RAM and disk conditions, host reserves, and admission limits. The four-stopped-box admission test verified that creation, forking, and recovery reject a fifth container when slots are exhausted. These are conservative allocation checks, not workload saturation benchmarks.
236
242
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@microck/raft",
3
- "version": "0.4.0",
3
+ "version": "0.6.0",
4
4
  "description": "Self-hosted Linux workspaces powered by Incus, an open-source alternative to boat.dev",
5
5
  "license": "MIT",
6
6
  "repository": {