@microck/raft 0.0.0-stage → 0.2.0

package/docs/site-development.md

@@ -0,0 +1,80 @@
+---
+title: Documentation site
+---
+
+# Documentation site
+
+The Fumadocs site in `website/` compiles Markdown and MDX files in
+`docs/` directly. Edit those files to update both GitHub documentation and the site.
+`docs/meta.json` and directory `meta.json` files control sidebar order and groups. Review reports stay outside the site navigation.
+
+## Run locally
+
+Use Node.js 24 and pnpm 11.25.0. Python 3 serves the local static preview.
+
+```sh
+cd website
+pnpm install --frozen-lockfile
+pnpm dev --hostname 127.0.0.1 --port 4300
+```
+
+Open `http://127.0.0.1:4300/docs/`. The site includes navigation, full-text search,
+code highlighting and a table of contents. Light mode is the default; the theme switch also offers dark mode.
+
+## Check and build
+
+```sh
+pnpm lint
+pnpm types:check
+pnpm build
+pnpm test
+pnpm start
+```
+
+### Build output
+
+The build copies benchmark reports and example configuration from `docs/` into the static export. Generated assets and Next.js output stay outside version control. The site is a separate package from the Python CLI.
+
+### Site tests
+
+`pnpm test` checks the exported pages, links, search, Markdown downloads, and 404s over controller loopback. It does not contact workspace hosts.
+
+Set `RAFT_DOCS_TEST_PORT` if port 4391 is occupied.
+
+## Machine-readable documentation
+
+- `/llms.txt` lists the Markdown files for every documentation page.
+- `/llms-full.txt` exports the complete documentation.
+- `/llms.mdx/docs/contract/content.md` exports the contract as Markdown.
+
+GitHub Pages serves fixed files. Use the explicit Markdown URLs; documentation
+pages return HTML regardless of the `Accept` header.
+
+## Hosting
+
+The public site is [raft.micr.dev](https://raft.micr.dev/). GitHub Pages serves
+`website/out/` at the domain root. Search runs in the browser using a build-time index.
+
+- Pull requests build and test the site. Pushes to `main` deploy the verified export.
+- Set the repository Pages publishing source to **GitHub Actions**.
+- For a fork, replace `website/public/CNAME` and the `metadataBase` URL in
+  `website/app/layout.tsx` with your domain, then configure Pages and DNS.
+
+For a local production preview, `pnpm start` serves the same export at
+`http://127.0.0.1:4300/`. Any static HTTP host can serve the exported files.
+Publishing the documentation does not provide public
+ingress to Raft workspaces.
+
+## Typography
+
+| Element | Font |
+| --- | --- |
+| Page titles | Funnel Display |
+| Body text | Inter |
+| Code blocks | Monospace |
+
+The pairing follows boat.dev. Upright Latin WOFF2 files from Fontsource live in `website/public/fonts/`; their SIL Open Font Licenses are in `boat-fonts-license.txt`.
+
+`next/font/local` serves the bundled fonts without third-party requests.
+
+Light mode uses Boat's white background, dark text, and thin borders. The theme switch also offers dark mode.

package/docs/verification.mdx

@@ -0,0 +1,231 @@
+---
+title: Verification
+---
+
+# Verification
+
+Raft is verified using real disposable Incus workspaces, guest processes, files, Docker builds, and network connections. Tests contain no mocked services and verify Raft behavior rather than conformance with the hosted Boat service.
+
+## Verification boundaries
+
+Tests do not certify arbitrary long-running workloads, host loss, multi-tenant security boundaries, or exact Boat image/version parity. ARM64 host reboot and physical storage capacity are not certified by the AMD64 VM test. Containers share the host kernel and do not offer VM isolation.
+
+Storage accounting covers shared pools rather than per-box quotas. Large backup transfers hold the host lifecycle lock, which delays expiration checks.
+
+## Coverage matrix
+
+| Implemented workflow | E2E assertion |
+| --- | --- |
+| Create/list/info/delete | Qualified handle, native state, ownership and cleanup |
+| Root terminal/exec | Real PTY, workspace directory, literal arguments and exit code |
+| Resource sizing | Effective CPU affinity and cgroup memory limit |
+| Host capacity | Mixed-size budgets, stopped/frozen boxes, low RAM/disk and four saved slots; live create/stop/resume/destroy changes in CPU/memory allocations |
+| Background jobs | Journal output, nonzero exit status, cancellation of child processes, inspection after cancellation or log rotation, and rejection of unknown job IDs |
+| TTL/extend/resume | Host-enforced stop, manual `gc`, retained files, uninterrupted lifetime extension and 30-day resume |
+| Usage | Real cgroup counters and explicitly shared-pool disk scope |
+| File transfer | Binary content, paths containing spaces, stopped-workspace transfers, preservation of the destination after a failed download, and real controller write-error rejection |
+| Docker | Real image build and container execution |
+| Snapshot/restore/fork | File rollback, independent copy, retained network identity and deterministic concurrent-resume rejection under the real host lock |
+| Backup/recovery | Stopped source, no overwrite, snapshots, content, permissions and fresh MAC |
+| Private forwarding | Guest HTTP response through controller loopback tunnel |
+| Desktop | WebSocket/RFB negotiation, framebuffer, pointer, keyboard, clipboard and reconnect |
+| Network separation | Positive service control, negative IPv4/IPv6 peer, host and metadata probes |
+| Development image | Real developer-user tool checks, immutable selection and credential-path archive rejection |
+| Deployment/firewall | Repeat deployment removes stale rules without restarting Incus; failed filtering blocks normal and socket startup; boot timestamps prove filtering precedes Incus |
+| Storage contract | Disposable oversized pool is rejected without automatic resizing |
+| CLI options and errors | Help for all 28 commands; default location, TTL and desktop port; all six sizing profiles; global and per-location reports; invalid TTLs, sizes, required arguments, handles, paths, snapshot names, jobs and ports |
+
+## Command and option audit (2026-10-06)
+
+<details>
+<summary>Coverage, regressions and remaining limits</summary>
+
+The audit checked help for all 26 commands and expanded the live suites to cover defaults, invalid arguments, manual garbage collection, maximum lifetimes and unknown job IDs. Two ARM64 hosts passed extended lifecycle tests, all six sizing profiles and all 31 development-tool checks.
+
+After the fixes, live lifecycle and same-host recovery passed again. Cross-host recovery passed with a 384,827,472-byte archive. Test boxes and archives were removed without rebooting or reconfiguring the existing hosts.
+
+The audit found and fixed two CLI bugs:
+
+- Unknown background jobs appeared to succeed. Job inspection now distinguishes unknown IDs, loaded jobs and canceled jobs with retained journal entries. Tests cover inspection after cancellation and journal rotation.
+- SSH could report success after a controller write failed. Downloads and backups now copy the stream through Python so local write errors reach the caller. Real `/dev/full` tests verify both receiving paths. The same Incus/SSH producer falsely succeeded with the previous implementation and raised `ENOSPC` with the fix.
+
+This covers implemented workflows and selected option combinations, not every possible workload or environment. A full-image export exceeded the local controller's free disk space and was interrupted. Full-sized archive validation runs on disposable CI hosts with sufficient scratch space.
+
+The [final runtime validation](https://github.com/Microck/raft/actions/runs/37542355027) tests commit `ac618e64`. ARM64 and AMD64 both passed extended lifecycle tests with daemon restarts, capacity transitions and full development-image backup/recovery. [Packaging checks](https://github.com/Microck/raft/actions/runs/37542355021) passed on Python 3.11 and 3.14, and [documentation checks](https://github.com/Microck/raft/actions/runs/37542355020) passed.
+
+</details>
+
+## New workspace controls
+
+`python3 deploy/verify-features.py --location lab` verifies recursive trees and unsafe-archive rejection, templates from a running source, snapshot deletion, CPU/RAM overrides, disposable timer deletion, and retention preview/apply. It uses only its own fixtures. Run the archive checks without `--location` to avoid allocating boxes.
+
+For an older deployment whose services should stay unchanged, `--isolated-worker` stages the current worker under a private temporary path. This checks worker logic without installing it; automatic timer integration is verified on disposable CI hosts. Current deployments use the normal verification mode; the isolated mode rejects them before allocation.
+
+## Workspace controls validation (2026-10-07)
+
+<details>
+<summary>Native feature tests and transfer regressions</summary>
+
+At runtime commit `a7fef1f5`, ARM64 and AMD64 passed the new workspace-control suite in [native CI](https://github.com/Microck/raft/actions/runs/37603576786). It verifies installed-timer deletion, retention preview and exclusions, live-source snapshot templates, snapshot deletion, and effective CPU/RAM limits after resume and fork.
+
+Directory tests cover binary files, permissions, relative symlinks, hardlinks, empty directories and existing destinations. Unix sockets and unsafe archive entries are rejected. A real Incus transfer truncated between complete archive entries fails without publishing a partial directory.
+
+Both architectures passed extended lifecycle tests, full development-image recovery, startup measurements and clean-template inspection. Fresh provisioning, image import, firewall ordering and reboot passed on a disposable AMD64 KVM host.
+
+The source-race fix also passed a real concurrent writer test that swaps files and directories for external symlinks. Accepted archives retain only the selected files. Two configured ARM64 hosts passed the final feature suite with a staged worker, without changing installed services. Local fixtures and staged files were removed.
+
+</details>
+
+## Running verification suites
+
+Test commands require configured hosts and immutable image fingerprints. Suites destroy only the fixtures they create.
+
+Allocation and archive inspection:
+- `python3 deploy/verify-limits.py`: Checks allocation policies without a host. Add `--location lab` to verify live create, stop, resume, and destroy reporting.
+- `python3 deploy/verify-image-tests.py`: Inspects archive structures, including symlinks, hardlinks, FIFOs, and credential detection.
+
+Live host suites (expand the check you need):
+
+<details>
+<summary>Lifecycle</summary>
+
+```sh
+python3 deploy/verify-live.py --location lab --extended
+```
+
+</details>
+
+<details>
+<summary>Daemon restart on a dedicated test host</summary>
+
+```sh
+python3 deploy/verify-live.py --location lab --restart-incus
+```
+
+</details>
+
+<details>
+<summary>Cross-host recovery</summary>
+
+```sh
+python3 deploy/verify-controls.py --source lab --target second-host --directory /path/to/archive-storage
+```
+
+</details>
+
+<details>
+<summary>Same-host recovery</summary>
+
+```sh
+python3 deploy/verify-controls.py --source lab --target lab --directory /path/to/archive-storage
+```
+
+</details>
+
+<details>
+<summary>Full development-image recovery</summary>
+
+```sh
+python3 deploy/verify-controls.py --source lab --target second-host --directory /path/to/archive-storage --development-image
+```
+
+</details>
+
+`verify-tools.py <qualified-box>` executes 31 tool checks as the unprivileged `developer` user inside a running workspace. It compiles and runs code across installed runtimes and package managers without external agent authentication.
+
+## Extended lifecycle results (2026-10-05)
+
+<details>
+<summary>Test evidence and run details</summary>
+
+The extended lifecycle suite and strengthened tool checks passed on two ARM64 Ubuntu hosts with Incus 6.0.6 on 2026-10-05.
+
+Both hosts executed all six sizing combinations, all 31 developer tool checks, Docker builds, file transfers, stopped snapshots, forks, restore, jobs, and private HTTP and desktop tunnels.
+
+Desktop testing validated keyboard input, exact clipboard synchronization, pointer coordinates, framebuffer updates, and secondary client connections. Headed Chromium testing passed with sandboxing enabled, confirming canvas rendering at 1280x720 over the noVNC interface.
+
+**Admission and backup runs**
+
+The four-saved-box admission test passed, including rejection of a fifth create, fork, or recovery attempt. Full development-image recovery succeeded with a 5,254,224,965-byte archive, verifying inventories, tool checks, permissions, fresh MACs, and snapshot rollback.
+
+The execution wrapper reported exit 143 after the success message; independent native inventory and archive checks confirmed that all fixtures and temporary archives were removed.
+
+Debian fixture backup and recovery passed across the following tests:
+- Same-host recovery: 385,278,177-byte archive with snapshot rollback and distinct MAC/IP identities.
+- Reverse cross-host recovery: 385,282,465-byte archive.
+- Forward cross-host admission run: 385,272,379-byte archive (exit code 0).
+
+All recovery fixtures were destroyed; existing operator hosts were not rebooted or reconfigured.
+
+</details>
+
+## Review-fix validation (2026-10-06)
+
+<details>
+<summary>Test evidence and run details</summary>
+
+On 2026-10-06, the extended suite passed on two configured ARM64 hosts after redeploying review fixes. Both ran all 31 tool checks, six sizing profiles, and deterministic race rejection when a source container started before obtaining the host lock. Cross-host recovery passed with a 385,265,470-byte archive.
+
+Both hosts had no remaining test boxes; existing hosts were not rebooted or reconfigured.
+
+The image inspector passed 15 test cases, including non-root SSH keys, symlinks, hardlinks, and clean controls. Network testing verified initial and repeated firewall application while preserving unrelated chains.
+
+The [review-fix native run](https://github.com/Microck/raft/actions/runs/37496577945) passed all native steps on ARM64 and AMD64: development-image builds, extended lifecycle with daemon restarts, capacity transitions, and clean-image inspection. Recovery archives were 3,844,136,311 bytes on ARM64 and 3,985,872,081 bytes on AMD64.
+
+**Disposable KVM host tests**
+
+Fresh provisioning on a disposable AMD64 KVM host confirmed:
+- Repeat deployment removed injected rules while leaving Incus daemon start timestamps unchanged.
+- Failed firewall installation blocked both normal and socket-activated Incus startup.
+- Host reboot verified firewall rules became active before Incus started.
+- A disposable pool resized to 61 GiB caused deployment to fail and preserved the existing size.
+
+ARM64 host reboot and physical disk capacity remain outside this verification.
+
+</details>
+
+## Native image validation (2026-10-06)
+
+<details>
+<summary>Test evidence and run details</summary>
+
+On 2026-10-06, the [native CI run](https://github.com/Microck/raft/actions/runs/37447415532) built ARM64 and AMD64 images on Ubuntu 24 and passed extended lifecycle tests on both. Workspaces executed all six sizing combinations, 31 tool checks, sandboxed Chromium, noVNC desktop sessions, and peer isolation checks. Timings are recorded in [Boot performance](./boot-performance.mdx).
+
+In the [release-validation run](https://github.com/Microck/raft/actions/runs/37469540182), native image jobs passed full development-image recovery on both architectures. Package inventories matched and all 31 tool checks passed. Archives measured 3,844,135,523 bytes on ARM64 and 3,985,894,517 bytes on AMD64.
+
+</details>
+
+## Fresh-host and reboot validation
+
+<details>
+<summary>Test evidence and run details</summary>
+
+The [release run](https://github.com/Microck/raft/actions/runs/37469540182) validated fresh-host setup on a disposable AMD64 KVM instance running Ubuntu 24. The test verified canonical deployment, repeat deployment, image import, and lifecycle execution.
+
+Following a VM reboot, changed kernel boot IDs and active services were verified alongside retained files, snapshots, container states, Docker, and bridge isolation. The host test uses a sparse 100 GiB virtual disk and accessible KVM.
+
+It preserves the production deployer's 80 GiB free-disk check but does not prove physical storage capacity. ARM64 host reboot, host loss, and cross-architecture recovery remain unverified. The VM, SSH keys, and controller configuration were disposable; existing operator hosts were not rebooted or reconfigured.
+
+</details>
+
+## Packaging and release checks
+
+<details>
+<summary>Test evidence and run details</summary>
+
+Package builds run via `uv build`. Source and wheel archives are inspected to exclude version-control files and local caches. Clean wheel installation is verified outside the worktree alongside Ruff, syntax checks, and secret scans.
+
+Packaging workflows passed on Python 3.11 and 3.14. Guidelines for visibility changes and repository hygiene are detailed in [Publishing](./publishing.md).
+
+</details>
+
+## Capacity command validation (2026-10-06)
+
+<details>
+<summary>Test evidence and run details</summary>
+
+On 2026-10-06, `raft limits` passed read-only text and JSON inspection and live lifecycle tests on two ARM64 Ubuntu hosts. Tests verified allocation adjustments during create, stop, resume, and destroy cycles.
+
+Allocation fixtures validated mixed sizing budgets, frozen containers, low RAM and disk conditions, host reserves, and admission limits. The four-stopped-box admission test verified that creation, forking, and recovery reject a fifth container when slots are exhausted. These are conservative allocation checks, not workload saturation benchmarks.
+
+</details>

package/images/incus/build.sh

@@ -0,0 +1,57 @@
+#!/bin/bash
+# Run inside a fresh native ARM64 or AMD64 Debian 13 Incus image, without host secrets.
+set -eu
+case "$(dpkg --print-architecture)" in
+  arm64) node_arch=arm64 ;;
+  amd64) node_arch=x64 ;;
+  *) echo 'Raft images support ARM64 and AMD64' >&2; exit 1 ;;
+esac
+export DEBIAN_FRONTEND=noninteractive
+apt-get update -qq
+apt-get install -y --no-install-recommends apparmor ca-certificates curl git openssh-server rsync jq ripgrep file less sudo locales python3 python3-pip python3-venv pipx build-essential clang cmake ninja-build golang rustc cargo default-jdk maven gradle kotlin scala ruby ruby-bundler php-cli composer erlang-base elixir r-base docker.io docker-cli docker-buildx chromium chromium-sandbox ffmpeg unzip xz-utils tar x11vnc xvfb x11-utils novnc websockify openbox dbus-x11
+# Ubuntu hosts restrict unprofiled user namespaces. Load a guest-local rule
+# for Chromium's exact executable so its sandbox can create its namespaces.
+# Incus's outer confinement and the host-wide restriction remain in force.
+cat > /etc/apparmor.d/raft-chromium <<'PROFILE'
+abi <abi/4.0>,
+include <tunables/global>
+profile raft-chromium /usr/lib/chromium/chromium flags=(unconfined) {
+  userns,
+}
+PROFILE
+systemctl enable apparmor
+systemctl restart apparmor
+mkdir -p /workspace /opt/raft
+node_archive="node-v24.21.0-linux-${node_arch}.tar.xz"
+curl -fsSL -A 'OpenAI File Downloader, XaiImageApiFetch/1.0' "https://nodejs.org/dist/v24.21.0/${node_archive}" -o /tmp/raft-node.tar.xz
+curl -fsSL -A 'OpenAI File Downloader, XaiImageApiFetch/1.0' https://nodejs.org/dist/v24.21.0/SHASUMS256.txt -o /tmp/raft-node.sha256
+cd /tmp
+awk -v archive="$node_archive" '$2 == archive {print $1 "  raft-node.tar.xz"}' raft-node.sha256 | sha256sum -c -
+tar -xJf /tmp/raft-node.tar.xz -C /usr/local --strip-components=1
+PIPX_HOME=/opt/raft/pipx PIPX_BIN_DIR=/usr/local/bin pipx install uv==0.12.23
+npm install -g pnpm@12.9.1 bun@1.4.2 deno@2.9.6
+curl -fsSL -A 'OpenAI File Downloader, XaiImageApiFetch/1.0' https://dot.net/v1/dotnet-install.sh -o /tmp/raft-dotnet-install.sh
+bash /tmp/raft-dotnet-install.sh --version 10.0.401 --install-dir /usr/local/share/dotnet
+ln -s /usr/local/share/dotnet/dotnet /usr/local/bin/dotnet
+useradd -m -s /bin/bash developer
+printf 'developer ALL=(ALL) NOPASSWD:ALL\n' > /etc/sudoers.d/raft-developer
+chmod 440 /etc/sudoers.d/raft-developer
+mkdir -p /etc/ssh/sshd_config.d /etc/systemd/system/ssh.service.d
+printf 'PasswordAuthentication no\nKbdInteractiveAuthentication no\nPermitRootLogin prohibit-password\nHostKey /etc/ssh/ssh_host_ed25519_key\n' > /etc/ssh/sshd_config.d/00-raft.conf
+# Generate keys before sshd validates them; published templates contain no host keys.
+# Only the advertised Ed25519 key is needed; generating unused RSA keys slows first boot.
+cat > /etc/systemd/system/ssh.service.d/raft-keys.conf <<'UNIT'
+[Service]
+ExecStartPre=
+ExecStartPre=/bin/sh -ec 'test -f /etc/ssh/ssh_host_ed25519_key || ssh-keygen -q -t ed25519 -N "" -f /etc/ssh/ssh_host_ed25519_key'
+ExecStartPre=/usr/sbin/sshd -t
+UNIT
+systemctl enable docker ssh
+systemctl start docker
+docker info --format '{{.ServerVersion}}'
+dpkg-query -W > /opt/raft/packages.tsv
+npm list -g --depth=0 --json > /opt/raft/npm-packages.json
+apt-get clean
+rm -rf /var/lib/apt/lists/* /tmp/raft-node* /tmp/raft-dotnet-install.sh /root/.npm/_cacache /root/.npm/_logs
+rm -f /etc/ssh/ssh_host_*
+truncate -s 0 /etc/machine-id

package/images/incus/desktop.sh

@@ -0,0 +1,21 @@
+#!/bin/sh
+set -eu
+export DISPLAY=:99
+Xvfb :99 -screen 0 1280x720x24 -nolisten tcp &
+xvfb_pid=$!
+trap 'kill "$xvfb_pid" 2>/dev/null || true' EXIT
+# Start clients as soon as X accepts requests, rather than sleeping on every launch.
+attempt=0
+until xdpyinfo >/dev/null 2>&1; do
+  attempt=$((attempt + 1))
+  if [ "$attempt" -ge 100 ]; then
+    echo 'Xvfb did not become ready within five seconds' >&2
+    exit 1
+  fi
+  sleep 0.05
+done
+runuser -u developer -- env DISPLAY=:99 openbox &
+# This dedicated Xvfb display has no login manager to wait for.
+X11VNC_AVOID_WINDOWS=never x11vnc -display :99 -localhost -nopw -forever -shared -rfbport 5900 &
+websockify --web=/usr/share/novnc 0.0.0.0:6080 127.0.0.1:5900 &
+wait

package/images/incus/raft-desktop.service

@@ -0,0 +1,7 @@
+[Unit]
+Description=Private Raft desktop on demand
+After=network.target
+[Service]
+ExecStart=/usr/local/bin/raft-desktop
+Restart=on-failure
+KillMode=control-group

package/package.json

@@ -1,6 +1,48 @@
 {
   "name": "@microck/raft",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.2.0",
+  "description": "Self-hosted Linux workspaces powered by Incus, an open-source alternative to boat.dev",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/Microck/raft.git"
+  },
+  "homepage": "https://raft.micr.dev",
+  "bugs": {
+    "url": "https://github.com/Microck/raft/issues"
+  },
+  "bin": {
+    "raft": "raft.py"
+  },
+  "os": [
+    "linux"
+  ],
+  "files": [
+    "raft.py",
+    "raft_files.py",
+    "deploy/*.py",
+    "deploy/*.sh",
+    "deploy/*.conf",
+    "deploy/*.service",
+    "deploy/*.timer",
+    "docs/**/*.md",
+    "docs/**/*.mdx",
+    "docs/**/*.json",
+    "images/incus/*.sh",
+    "images/incus/*.service",
+    "skills/raft-cli/SKILL.md",
+    ".github/assets/raft-logo.png"
+  ],
+  "keywords": [
+    "raft",
+    "sandbox",
+    "incus",
+    "self-hosted",
+    "linux",
+    "cli"
+  ],
+  "publishConfig": {
+    "access": "public",
+    "provenance": true
+  }
+}