@microck/raft 0.4.0 → 0.5.0

package/README.md

@@ -30,13 +30,16 @@ npm install -g @microck/raft
 raft --version
 ```
 
+`raft self-update` upgrades the CLI later. Raft prints a one-line stderr notice
+when a new release exists; set `RAFT_NO_UPDATE_CHECK=1` to turn it off.
+
 Python wheels and source archives are also available in
 [releases](https://github.com/Microck/raft/releases). npm installation does not set
 up hosts or download workspace images.
 
 | requirement | minimum |
 | --- | --- |
-| host | dedicated ARM64 or AMD64 Ubuntu host, 2 CPUs, 8 GiB RAM, 80 GiB free disk |
+| host | dedicated ARM64 or AMD64 Ubuntu host, 2 CPUs, 8 GiB RAM, 80 GiB free disk, plus the pool size if you enable own box disks |
 | host access | SSH and passwordless sudo |
 | controller | Linux, Python 3.11.8+, SSH and SCP; npm or uv for installation |
 
@@ -64,9 +67,19 @@ the box with a new lifetime. `raft destroy` deletes the box and its snapshots.
 disposable creation and retention cleanup require the current host worker;
 redeploy from the current checkout before using them.
 
-each host allows four saved boxes, including stopped ones. `raft limits` reports
-current allocations, free disk space and recommended running counts for each
-size. see [host capacity](docs/reference/capacity-limits.mdx).
+size boxes for the job: `--cpu`, `--memory` and `--disk` take any whole number
+up to the host. on hosts deployed with `--disk-pool SIZE`, `--disk 20GiB` gives a
+box its own disk that it, and Docker inside it, cannot grow past; forks and
+templates of it are copy-on-write. raft does not cap how many boxes you keep.
+`raft capacity` prints an advisory estimate of what fits; it never blocks a box.
+see [box sizing and capacity](docs/reference/capacity.mdx).
+
+`raft config set` stores your defaults for `new`, such as location, TTL and
+size. size defaults apply only to boxes from the image: `new --from` and `fork`
+keep the source's size. `raft policy lab --require purpose,owner` makes the host
+reject boxes whose note lacks those fields, with an error that names the missing
+flags. it is a guardrail for agents: anyone with sudo on the host can change it.
+see [defaults and note policies](docs/guides/config-policy.mdx).
 
 ## image and workspace control
 
@@ -131,8 +144,8 @@ records differences, sources and test coverage.
 | Create/list/inspect/delete | ✅ | ✅ | Single operator |
 | Persistent stop/resume | ✅ | ✅ | Shared host kernel |
 | TTL and lifetime extension | ✅ | ✅ | Persistent boxes retain disk; disposable boxes delete |
-| CPU/memory sizing and usage | ✅ | ✅ | 1 or 2 CPUs, 1, 2, 4 or 8 GiB RAM. Disk usage covers the shared pool |
-| Host capacity recommendations | ❌ | ✅ | Raft recommends running capacity and enforces four saved boxes. Boat reports plan limits |
+| CPU/memory sizing and usage | ✅ | ✅ | Any whole CPU count and GiB size up to the host. Usage covers the shared pool or the box's own disk |
+| Host capacity estimates | ❌ | ✅ | Advisory only; Raft has no box count limit. Boat reports plan limits |
 | Root terminal and command execution | ✅ | ✅ | Host SSH transport |
 | Background jobs, logs and cancellation | ✅ | ✅ | No process checkpoints |
 | File upload/download | ✅ | ✅ | Files and recursive trees; new directory destinations only |
@@ -144,17 +157,17 @@ records differences, sources and test coverage.
 | Desktop access | ✅ | ✅ | noVNC; see verification limits |
 | Reverse forwarding | ✅ | ❌ | Not implemented |
 | Public workspace hosting and browser-only streaming | ✅ | ❌ | Not implemented |
-| Resize on resume/fork | ✅ | ✅ | Optional CPU/RAM overrides within supported sizes |
-| CLI-wide JSON mode | ✅ | ❌ | Boat JSON/JSONL; Raft list/info/usage/limits plus prune JSONL |
+| Resize on resume/fork | ✅ | ✅ | Optional CPU/RAM overrides; disks keep their size |
+| CLI-wide JSON mode | ✅ | ❌ | Boat JSON/JSONL; Raft list/info/usage/capacity plus prune JSONL |
 | Snapshot file browsing/download | ✅ | ❌ | Whole-box backup only |
-| Strict per-box disk quotas | ✅ | ❌ | Shared 60 GiB Btrfs pool |
+| Strict per-box disk quotas | ✅ | ✅ | On hosts deployed with `--disk-pool`, `new --disk SIZE` gives a box its own fixed-size, copy-on-write disk; the default shared pool has none |
 | Separate-kernel VM isolation | ✅ | ❌ | Container runtime |
 | Named environments and managed secrets | ✅ | ❌ | Manual configuration |
 | Organizations and webhooks | ✅ | ❌ | Raft is single-operator; no event API |
 | Dashboard, remote API and SDK | ✅ | ❌ | CLI only |
 | Managed agent conversations | ✅ | ❌ | Not implemented |
 | Automatic deletion and retention | ✅ | ✅ | Disposable deletion; opt-in age-based prune with preview |
-| CLI self-update | ✅ | ❌ | Standard npm or Python package installation |
+| CLI self-update | ✅ | ✅ | Explicit `raft self-update` for npm and uv installs plus a daily stderr notice. Boat also updates itself after commands; Raft only notifies |
 
 ## verification and development
 

package/deploy/deploy.py

@@ -15,7 +15,7 @@ import sys
 
 ROOT = Path(__file__).resolve().parent.parent
 sys.path.insert(0, str(ROOT))
-from raft import configuration  # noqa: E402
+from raft import BOX_DISK_POOL, configuration, gib, gib_size  # noqa: E402
 
 RELEASES = "https://github.com/Microck/raft/releases/download/"
 # Matches the user agent install-incus.sh sends for its own downloads.
@@ -134,7 +134,86 @@ def import_archive(host, archive, architecture):
     return checked_image(host, fingerprint, architecture)
 
 
-def deploy(name, host):
+def free_disk(host):
+    return int(
+        remote(
+            host,
+            shlex.join(
+                [
+                    "python3",
+                    "-c",
+                    "import shutil; print(shutil.disk_usage('/var/lib/incus').free)",
+                ]
+            ),
+        )
+    )
+
+
+def box_disk_pool(host, pools, size):
+    """Create or verify the thin LVM pool that holds own box disks.
+
+    Own box disks are opt-in: without --disk-pool a host has no raft-disks pool,
+    which is a normal state, and --disk-pool on a later redeploy adds it. Thin
+    volumes make forks, templates and snapshots of own-disk boxes snapshots that
+    share blocks. Raft never overcommits the pool: every write path that adds a
+    thin volume checks that all virtual sizes still fit its data space (see
+    BOX_DISK_ACCOUNTING in raft.py), so the pool cannot fill.
+    """
+    pool = next((item for item in pools if item["name"] == BOX_DISK_POOL), None)
+    if pool is None:
+        if size is None:
+            return
+        # Same 20 GiB margin as the fresh-host check for the 60 GiB shared pool.
+        if free_disk(host) < (gib(size) + 20) * 1024**3:
+            raise RuntimeError(
+                f"Creating the {size} {BOX_DISK_POOL} pool needs {gib(size) + 20} GiB free disk; "
+                "pass a smaller --disk-pool or free host disk"
+            )
+        remote(
+            host,
+            f"sudo -n incus storage create {BOX_DISK_POOL} lvm size={size} lvm.use_thinpool=true",
+        )
+        pool = json.loads(remote(host, f"sudo -n incus query /1.0/storage-pools/{BOX_DISK_POOL}"))
+    config = pool["config"]
+    if pool["driver"] != "lvm" or config.get("lvm.use_thinpool") != "true":
+        raise RuntimeError(
+            f"{BOX_DISK_POOL} must be a thin LVM pool (lvm.use_thinpool=true); inspect it and "
+            "back up its boxes before explicit storage recovery"
+        )
+    if size is not None and config.get("size") != size:
+        raise RuntimeError(
+            f"{BOX_DISK_POOL} has size={config.get('size')}, not {size}. "
+            "Deployment does not resize existing storage; omit --disk-pool to keep it"
+        )
+    # The thin pool frees blocks that boxes discard, but must not pass discards on
+    # to its loop file: that would punch holes in the file, and a later write into
+    # a hole could find the host disk full and fail every own-disk box at once.
+    # Changing passdown to nopassdown is allowed while the pool is active.
+    thinpool = f"{config['lvm.vg_name']}/{config['lvm.thinpool_name']}"
+    discards = remote(host, f"sudo -n lvs --noheadings -o discards {thinpool}").strip()
+    if discards != "nopassdown":
+        remote(host, f"sudo -n lvchange --discards nopassdown {thinpool}")
+    # Incus creates the pool's loop file sparse. Allocating its full length makes
+    # the host disk hold the whole pool; fallocate only fills holes and never
+    # changes the file's length or data, so repeating it is a no-op. A pool an
+    # operator placed on a dedicated block device has no file to allocate.
+    remote(
+        host,
+        shlex.join(
+            [
+                "sudo",
+                "-n",
+                "sh",
+                "-c",
+                'test ! -f "$1" || fallocate -l "$(stat -c %s "$1")" "$1"',
+                "raft",
+                config["source"],
+            ]
+        ),
+    )
+
+
+def deploy(name, host, disk_pool=None):
     host_architecture(host)
     copy(host, ROOT / "deploy/install-incus.sh", "/tmp/raft-install-incus.sh")
     print(remote(host, "bash /tmp/raft-install-incus.sh </dev/null"), flush=True)
@@ -149,24 +228,29 @@ def deploy(name, host):
         if project.get("description") != "Raft managed workspaces":
             raise RuntimeError("Existing raft project needs ownership verification")
     else:
-        if pool is not None or network is not None:
-            raise RuntimeError(
-                "Raft resource names already exist without an owned project; review them explicitly"
-            )
-        available = int(
-            remote(
-                host,
-                shlex.join(
-                    [
-                        "python3",
-                        "-c",
-                        "import shutil; print(shutil.disk_usage('/var/lib/incus').free)",
-                    ]
+        leftovers = [
+            name
+            for name, found in [
+                ("storage pool raft-data", pool is not None),
+                ("network rfbr0", network is not None),
+                (
+                    f"storage pool {BOX_DISK_POOL}",
+                    any(item["name"] == BOX_DISK_POOL for item in pools),
                 ),
+            ]
+            if found
+        ]
+        if leftovers:
+            raise RuntimeError(
+                "Raft resource names already exist without an owned project: "
+                + ", ".join(leftovers)
+                + "; review them explicitly"
             )
-        )
-        if available < 80 * 1024**3:
-            raise RuntimeError("Fresh deployment needs at least 80 GiB free disk")
+        # 60 GiB shared pool, 20 GiB for images and the host, and the own-disk
+        # pool when --disk-pool enables it.
+        needed = 80 + (gib(disk_pool) if disk_pool else 0)
+        if free_disk(host) < needed * 1024**3:
+            raise RuntimeError(f"Fresh deployment needs at least {needed} GiB free disk")
         remote(
             host,
             "sudo -n incus project create raft -c features.images=true -c features.profiles=true </dev/null",
@@ -207,6 +291,7 @@ def deploy(name, host):
             "Raft pool must have configured size=60GiB; inspect raft-data and back up boxes "
             "before explicit storage recovery. Deployment does not resize existing storage."
         )
+    box_disk_pool(host, pools, disk_pool)
 
     devices = profile["devices"]
     if (
@@ -261,9 +346,17 @@ def main():
     parser = argparse.ArgumentParser(description=__doc__)
     configured = hosts()
     parser.add_argument("--location", choices=configured)
+    parser.add_argument(
+        "--disk-pool",
+        type=gib_size,
+        metavar="SIZE",
+        help=f"Enable own box disks: create the {BOX_DISK_POOL} pool with SIZE of host disk "
+        "if it is missing (default: leave own box disks as they are); an existing pool must "
+        "already have this size",
+    )
     args = parser.parse_args()
     for name in [args.location] if args.location else configured:
-        deploy(name, configured[name])
+        deploy(name, configured[name], args.disk_pool)
 
 
 if __name__ == "__main__":

package/deploy/install-incus.sh

@@ -15,6 +15,16 @@ if ! command -v incus >/dev/null; then
   sudo apt-get update -qq
   sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y --no-install-recommends incus btrfs-progs
 fi
+# raft-disks, the opt-in pool for own box disks, uses LVM thin provisioning.
+if ! command -v lvcreate >/dev/null; then
+  sudo apt-get update -qq
+  sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y --no-install-recommends lvm2
+fi
+# raft recover reads the backup index, which Incus writes as YAML.
+if ! python3 -c 'import yaml' 2>/dev/null; then
+  sudo apt-get update -qq
+  sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y --no-install-recommends python3-yaml
+fi
 if ! command -v nft >/dev/null; then
   sudo apt-get update -qq
   sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=l apt-get install -y --no-install-recommends nftables

package/deploy/verify-controls.py

@@ -9,7 +9,7 @@ import time
 import uuid
 
 sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
-from raft import ADMISSION, address, incus, inventory, parse_handle, transaction  # noqa: E402
+from raft import address, incus, inventory, parse_handle, transaction  # noqa: E402
 import importlib.util  # noqa: E402
 
 spec = importlib.util.spec_from_file_location(
@@ -32,61 +32,21 @@ def wait_address(box):
     raise RuntimeError("Resumed workspace has no managed IPv4 address after 30 seconds")
 
 
-def verify_admission(location, directory):
-    fixtures = []
-    saved = len(inventory(location))
-    if saved >= 4:
-        raise RuntimeError("Admission test needs at least one free slot on its target")
-    try:
-        for _ in range(4 - saved):
-            box = live.raft(
-                "new", "--location", location, "--memory", "1GiB", "--ttl", "600"
-            ).stdout.strip()
-            fixtures.append(box)
-            live.wait_running(box)
-            live.raft("stop", box)
-        rejected = live.raft("new", "--location", location, check=False)
-        if rejected.returncode == 0:
-            fixtures.append(rejected.stdout.strip())
-        assert rejected.returncode != 0 and "already holds four boxes" in rejected.stderr
-        rejected = live.raft("fork", fixtures[0], "--ttl", "600", check=False)
-        if rejected.returncode == 0:
-            fixtures.append(rejected.stdout.strip())
-        assert rejected.returncode != 0 and "already holds four boxes" in rejected.stderr
-        with tempfile.TemporaryDirectory(dir=directory, prefix="raft-admission-") as work:
-            # Admission must reject before native archive parsing, without creating a box.
-            archive = Path(work) / "rejected.tar.gz"
-            archive.write_bytes(b"not-an-incus-archive")
-            rejected = live.raft("recover", str(archive), "--location", location, check=False)
-            if rejected.returncode == 0:
-                fixtures.append(rejected.stdout.strip())
-            assert rejected.returncode != 0 and "already holds four boxes" in rejected.stderr
-        assert len(inventory(location)) == 4
-        report = json.loads(live.raft("limits", "--location", location, "--json").stdout)
-        assert report["savedBoxes"] == 4 and report["savedSlots"] == 0
-        assert all(size["newBoxes"] == 0 for size in report["recommendations"])
-        print(
-            f"{location}: stopped boxes count toward admission; new/fork/recover reject a fifth box",
-            flush=True,
-        )
-    finally:
-        for box in reversed(fixtures):
-            live.raft("destroy", box)
-
-
 def verify(source, target, directory, development_image=False):
     fixtures = []
     name = "rf-" + uuid.uuid4().hex[:16]
     box = source + ":" + name
     try:
         if development_image:
-            box = live.raft("new", "--location", source, "--ttl", "1800").stdout.strip()
+            box = live.raft(
+                "new", "--location", source, "--ttl", "1800", "--cpu", "1", "--disk", "shared"
+            ).stdout.strip()
             fixtures.append(box)
             live.wait_running(box)
         else:
             transaction(
                 source,
-                ADMISSION + 'incus --project raft launch images:debian/13 "$1" '
+                'incus --project raft launch images:debian/13 "$1" '
                 "-c user.raft=true -c limits.cpu=1 -c limits.memory=1GiB; "
                 'incus --project raft config set "$1" user.raft.expires="$(( $(date +%s) + 600 ))"',
                 name,
@@ -172,9 +132,36 @@ def verify(source, target, directory, development_image=False):
             print(f"{source}: archive completed ({size} bytes)", flush=True)
             assert live.raft("backup", box, str(archive), check=False).returncode != 0
             assert archive.stat().st_size == size
-            restored = live.raft("recover", str(archive), "--location", target).stdout.strip()
-            fixtures.append(restored)
+            # The archived box has no note, so a target policy rejects it after import
+            # and deletes the imported copy in the same locked transaction.
+            live.raft("policy", target, "--require", "purpose,owner")
+            try:
+                saved = len(inventory(target))
+                rejected = live.raft("recover", str(archive), "--location", target, check=False)
+                if rejected.returncode == 0:
+                    fixtures.append(rejected.stdout.strip())
+                assert rejected.returncode == 1, rejected
+                assert "requires box note fields purpose, owner" in rejected.stderr, rejected
+                assert len(inventory(target)) == saved, "Rejected recovery kept a box"
+                restored = live.raft(
+                    "recover",
+                    str(archive),
+                    "--location",
+                    target,
+                    "--purpose",
+                    "Recovery verification",
+                    "--owner",
+                    "verify-controls",
+                ).stdout.strip()
+                fixtures.append(restored)
+            finally:
+                live.raft("policy", target, "--clear")
             print(f"{target}: recovered stopped workspace", flush=True)
+            note = json.loads(live.raft("info", restored).stdout)["config"]["user.raft.note"]
+            assert json.loads(note) == {
+                "purpose": "Recovery verification",
+                "owner": "verify-controls",
+            }
             metadata = json.loads(live.raft("info", restored).stdout)
             assert metadata["status"] == "Stopped"
             assert not metadata["config"].get("user.raft.expires")
@@ -237,5 +224,4 @@ if __name__ == "__main__":
         help="Export the complete configured development image rather than a small fixture",
     )
     args = parser.parse_args()
-    verify_admission(args.target, args.directory)
     verify(args.source, args.target, args.directory, args.development_image)