@microck/raft 0.0.0-stage → 0.2.1

package/docs/guides/background-jobs.mdx

@@ -0,0 +1,95 @@
+---
+title: Background jobs
+description: Run detached commands with systemd logging and process management.
+---
+
+# Background jobs
+
+Raft runs detached background commands under the guest systemd init system using transient service units.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Launching detached commands
+
+Start a background command by passing `--detach` to `raft exec`, delimiting the payload with `--`:
+
+```sh title="CONTROLLER: Start a detached background job"
+job=$(raft exec "$box" --detach -- bash -lc '
+echo "Task started at $(date)"
+sleep 60
+echo "Task finished at $(date)"
+')
+echo "Started background job: $job"
+```
+
+The command returns a 32-hex systemd unit name prefixed with `rfcmd-` (such as `rfcmd-a1b2c3d4e5f60718293a4b5c6d7e8f90`).
+
+## Checking job status
+
+Use `raft status` to query guest systemd unit properties:
+
+```sh title="CONTROLLER: Inspect background job state"
+raft status "$box" "$job"
+```
+
+Active jobs report running substate:
+```text
+ActiveState=active
+SubState=running
+ExecMainStatus=0
+```
+
+Because Raft runs detached commands with `RemainAfterExit=yes`, completed jobs retain their exit record:
+```text
+ActiveState=active
+SubState=exited
+ExecMainStatus=0
+```
+
+<Callout type="info" title="Exit status vs CLI return code">
+`ExecMainStatus` reports the command payload exit code inside the container. The `raft status` command returns exit code 0 when the query succeeds. It queries unit state and does not report process PIDs or start timestamps.
+</Callout>
+
+## Viewing job logs
+
+Retrieve output recorded in the guest systemd journal:
+
+```sh title="CONTROLLER: Retrieve job log output"
+raft logs "$box" "$job"
+```
+
+The command runs `journalctl --no-pager -o cat -u <job>` inside the container. It prints recorded logs without a pager or stream follow mode.
+
+## Cancelling jobs
+
+Terminate a running detached command:
+
+```sh title="CONTROLLER: Terminate background job"
+raft cancel "$box" "$job"
+```
+
+`raft cancel` executes `systemctl stop <job>`, terminating processes in the job's systemd control group.
+
+## Extending workspace lifetime
+
+<Callout type="warn" title="No process checkpointing">
+Stopping a workspace with `raft stop` or allowing its TTL to expire terminates all running background jobs. Raft does not checkpoint running process memory.
+</Callout>
+
+Extend the workspace lifetime before it expires:
+
+```sh title="CONTROLLER: Reset workspace expiration"
+raft extend "$box" --ttl 1800
+```
+
+`raft extend` resets the container expiration deadline to `now + TTL` seconds using the host clock. It does not add time to the existing deadline.
+
+For deadline inspection and an enabled systemd service that starts again after
+resume or fork, see [Lifetime and persistent services](./long-running-services.mdx).

package/docs/guides/backup-recovery.mdx

@@ -0,0 +1,111 @@
+---
+title: Backup and recovery
+description: Export portable workspace archives and restore them across hosts.
+---
+
+# Backup and recovery
+
+Raft exports portable workspace archives and restores them onto compatible hosts for disaster recovery, migration, or offline archiving.
+
+## Target workspace handle
+
+Commands in this guide require a stopped workspace:
+
+```sh title="CONTROLLER: Select and stop target workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+raft stop "$box"
+```
+
+## Exporting a backup
+
+Specify a destination archive path on the controller:
+
+```sh title="CONTROLLER: Export stopped workspace to compressed archive"
+raft backup "$box" ./my-workspace-backup.tar.gz
+```
+
+`raft backup` exports container metadata, root filesystem, and snapshots into a gzip-compressed archive.
+
+### Security and storage requirements
+
+<Callout type="warn" title="Archives contain sensitive data">
+Backup archives contain the entire guest filesystem, including credentials, SSH keys, shell history, and private files. Native Incus archive configuration is not sanitized. Import only trusted archives created by Raft, and store backups on encrypted private storage off the source host for disaster recovery.
+</Callout>
+
+Raft enforces safeguards during export:
+
+- **Private permissions**: Destination mode must be `0600`. Group and world permissions must be zero before export begins.
+- **Atomic link publication**: After export and `fsync`, Raft publishes the archive using `os.link(archive.name, destination)`. Hard-link support is checked at final publication rather than preflight; export can succeed while publication fails if the filesystem lacks hard-link support.
+- **No overwriting**: The destination file must not already exist.
+- **Lifecycle lock**: Export holds `/run/lock/raft-incus.lock`. Expiry checks for other containers on the host are delayed while the lock is held.
+
+## Recovering an archive
+
+Import the archive onto the target location:
+
+```sh title="CONTROLLER: Import workspace backup onto target host"
+recovered=$(raft recover ./my-workspace-backup.tar.gz --location lab)
+echo "Recovered workspace: $recovered"
+```
+
+### Recovery process details
+
+- **Architecture compatibility**: Target host must match the native CPU architecture (ARM64 to ARM64, AMD64 to AMD64). Cross-architecture recovery is not supported.
+- **Fresh network identity**: The container receives a new name in project `raft`, a new MAC address on `rfbr0`, and no inherited TTL deadline.
+- **Stopped initial state**: Recovered workspaces remain stopped until started with `raft resume "$recovered" --ttl <SECONDS>`.
+- **Enforced admission limit**: The destination location must hold fewer than four boxes.
+- **Interrupted recovery**: Follow [Host crash during recovery](./troubleshooting.mdx#host-crash-during-recovery). Inspect the reported handle and container state before deleting any staging file.
+
+Start and verify the recovered workspace:
+
+```sh title="CONTROLLER: Start and verify recovered workspace"
+raft resume "$recovered" --ttl 600
+raft exec "$recovered" -- ls -la /workspace
+```
+
+## Cross-host migration workflow
+
+Follow these steps to migrate a workspace to a second host:
+
+### 1. Select and stop source workspace
+
+```sh title="CONTROLLER"
+raft list
+source_box="lab:rf-a1b2c3d4e5f60718"
+raft stop "$source_box"
+```
+
+### 2. Export archive to local controller storage
+
+```sh title="CONTROLLER"
+raft backup "$source_box" ./migration.tar.gz
+```
+
+### 3. Import archive onto destination host
+
+```sh title="CONTROLLER"
+target_box=$(raft recover ./migration.tar.gz --location second-host)
+```
+
+### 4. Resume workspace on destination host
+
+```sh title="CONTROLLER"
+raft resume "$target_box" --ttl 1800
+```
+
+### 5. Verify application files and operations
+
+```sh title="CONTROLLER"
+raft exec "$target_box" -- ls -la /workspace
+```
+
+After verifying the recovered workspace, optionally destroy the source container:
+
+```sh title="CONTROLLER: Optional: Destroy original container"
+raft destroy "$source_box"
+```
+
+<Callout type="warn" title="Preserve backup archives">
+Do not delete your local backup archive until you confirm that the destination workspace operates normally.
+</Callout>

package/docs/guides/desktop-services.mdx

@@ -0,0 +1,90 @@
+---
+title: Desktop and private services
+description: Forward network ports and connect to an on-demand graphical desktop.
+---
+
+# Desktop and private services
+
+Forward network ports and connect to an on-demand graphical desktop over authenticated host SSH tunnels bound to controller loopback.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Forwarding network ports
+
+Create a secure tunnel between your local controller and a workspace service:
+
+```text
+raft forward "$box" --remote <REMOTE_PORT> --local <LOCAL_PORT>
+```
+
+### End-to-end example: Running a web service
+
+Start a detached HTTP server inside the workspace:
+
+```sh title="CONTROLLER: Start detached HTTP server inside workspace"
+job=$(raft exec "$box" --detach -- python3 -m http.server 8080 --bind 0.0.0.0)
+echo "Started web service job: $job"
+```
+
+Open the forwarding tunnel:
+
+```sh title="CONTROLLER: Open port forwarding tunnel"
+raft forward "$box" --remote 8080 --local 8080
+```
+
+`raft forward` runs in the foreground. Open `http://127.0.0.1:8080` in your browser or run `curl http://127.0.0.1:8080` in another terminal.
+
+When finished:
+1. Press `Ctrl+C` to close the tunnel.
+2. Cancel the background job:
+   ```sh title="CONTROLLER: Stop the background server job"
+   raft cancel "$box" "$job"
+   ```
+
+### Binding requirement
+
+<Callout type="warn" title="Bind to all interfaces or guest IP">
+Services inside the container must bind to all interfaces (`0.0.0.0`) or to the container's private `eth0` IP address. A service listening strictly on container loopback (`127.0.0.1`) cannot receive traffic routed across the container bridge.
+</Callout>
+
+The controller listener binds strictly to `127.0.0.1` on your local workstation, without exposing ports to your local network.
+
+## On-demand desktop environment
+
+The development image includes a graphical desktop stack using Xvfb (display `:99`), Openbox, x11vnc, and noVNC.
+
+Start the desktop and establish a private tunnel:
+
+```sh title="CONTROLLER: Launch desktop and establish private tunnel"
+raft desktop "$box" --local 6080
+```
+
+`raft desktop` requests startup of `raft-desktop.service` inside the container if
+needed and opens a tunnel to local port `6080`. The service waits for Xvfb before
+launching its desktop clients. The CLI does not poll noVNC readiness; if the first
+browser request fails during startup, retry after the service initializes.
+
+Open the client in your browser:
+
+```text
+http://127.0.0.1:6080/vnc.html
+```
+
+<Callout type="info" title="Browser URL requirement">
+Include `/vnc.html` in the URL. The root path `/` displays an index directory; `/vnc.html` loads the desktop canvas with mouse, keyboard, and clipboard support.
+</Callout>
+
+### Desktop characteristics
+
+- **Display environment**: Virtual X display `:99` at 1280x720 resolution with 24-bit color.
+- **User session**: Openbox runs under the `developer` user account.
+- **Clipboard support**: The noVNC sidebar provides a clipboard drawer for text exchange.
+- **Tunnel lifecycle**: Pressing `Ctrl+C` closes the local tunnel. The desktop service keeps running until the workspace stops or expires.
+- **Reconnecting**: Running `raft desktop "$box" --local 6080` reconnects without interrupting open windows.

package/docs/guides/docker.mdx

@@ -0,0 +1,84 @@
+---
+title: Docker in workspaces
+description: Run Docker daemons and build container images inside unprivileged Raft workspaces.
+---
+
+# Docker in workspaces
+
+Raft development images include Docker daemon and CLI tooling running directly in the unprivileged container without mounting the host Docker socket.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Verifying Docker status
+
+Verify that the guest Docker daemon is initialized and responsive:
+
+```sh title="CONTROLLER: Check Docker version and daemon status"
+raft exec "$box" -- docker version
+raft exec "$box" -- docker info
+```
+
+<Callout type="info" title="Daemon readiness verification">
+`docker version` and `docker info` query guest `dockerd`. Running `docker --version` checks only the client binary without testing daemon health.
+</Callout>
+
+## Running containers
+
+Run containers inside your workspace:
+
+```sh title="CONTROLLER: Run an Alpine container"
+raft exec "$box" -- docker run --rm alpine uname -a
+```
+
+Start background services inside the workspace:
+
+```sh title="CONTROLLER: Start a background Redis container"
+raft exec "$box" -- docker run -d --name cache -p 6379:6379 redis:alpine
+raft exec "$box" -- docker ps
+```
+
+## Building container images
+
+Build container images within the workspace:
+
+```sh title="CONTROLLER: Create Dockerfile and build image"
+raft exec "$box" -- bash -lc '
+mkdir -p /workspace/app
+cat << "EOF" > /workspace/app/Dockerfile
+FROM alpine:latest
+RUN echo "hello from workspace container" > /message.txt
+CMD ["cat", "/message.txt"]
+EOF
+cd /workspace/app
+docker build -t my-app .
+docker run --rm my-app
+'
+```
+
+## Managing disk space
+
+Nested container layers and volumes reside on the shared 60 GiB `raft-data` Btrfs storage pool.
+
+Prune dangling images without deleting active container data:
+
+```sh title="CONTROLLER: Prune dangling images"
+raft exec "$box" -- docker image prune -f
+```
+
+<Callout type="warn" title="Data loss risk with system prune">
+Running `docker system prune -af --volumes` removes all stopped containers and unused volumes inside the workspace. Run full pruning only when you intend to discard local container state and caches.
+</Callout>
+
+## Isolation and runtime limits
+
+- **Daemon confinement**: The Docker daemon runs as a guest systemd unit using `/var/run/docker.sock`. The host Docker daemon is never mounted.
+- **Kernel sharing**: Workspaces share the host Linux kernel without host kernel module or KVM access.
+- **Syscall interception**: Incus sets `security.nesting=true`, `security.syscalls.intercept.mknod=true`, and `security.syscalls.intercept.setxattr=true` to manage image layers safely in unprivileged user namespaces.
+- **Shutdown behavior**: Stopping a workspace with `raft stop` stops guest processes and nested containers.

package/docs/guides/files.mdx

@@ -0,0 +1,91 @@
+---
+title: File transfer
+description: Transfer files between your controller machine and running Raft workspaces.
+---
+
+# File transfer
+
+Upload and download individual files between your local controller and running workspaces over verified host SSH and Incus instance APIs.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Uploading files
+
+Transfer a file from your controller to an absolute path inside the workspace:
+
+```sh title="CONTROLLER: Create fixture and upload to workspace"
+# Create local sample file on controller
+echo "Hello from controller" > ./local-file.txt
+
+# Upload to workspace
+raft upload "$box" ./local-file.txt /workspace/target-file.txt
+```
+
+Existing destination files are overwritten. Create parent directories before uploading if needed:
+
+```sh title="CONTROLLER: Create parent directory and upload"
+echo '{"mode": "debug"}' > ./settings.json
+raft exec "$box" -- mkdir -p /workspace/configs
+raft upload "$box" ./settings.json /workspace/configs/settings.json
+```
+
+<Callout type="info" title="Absolute guest paths required">
+Guest file paths must be absolute paths beginning with `/`. Relative paths inside the container are rejected by the CLI parser.
+</Callout>
+
+## Downloading files
+
+Download a file from a workspace to your controller:
+
+```sh title="CONTROLLER: Download a file from workspace"
+raft download "$box" /workspace/target-file.txt ./downloaded-file.txt
+```
+
+The destination path on the controller must specify the target filename.
+
+## Paths with spaces
+
+Wrap paths in quotes when transferring files with spaces:
+
+```sh title="CONTROLLER: Transfer files with spaces in paths"
+mkdir -p "./my documents"
+echo "col1,col2" > "./my documents/data file.csv"
+
+raft exec "$box" -- mkdir -p "/workspace/my documents"
+raft upload "$box" "./my documents/data file.csv" "/workspace/my documents/data file.csv"
+raft download "$box" "/workspace/my documents/data file.csv" "./downloaded-data.csv"
+```
+
+## Transferring directories
+
+Use `--recursive` to copy a tree into a new directory:
+
+```sh
+raft upload "$box" ./my-project /workspace/my-project --recursive
+raft download "$box" /workspace/my-project ./project-copy --recursive
+```
+
+The destination must not exist, and its parent must exist. Raft publishes the directory after the complete transfer; it does not merge or overwrite trees.
+
+| Preserved | Rejected |
+| --- | --- |
+| File contents and executable bits | Existing destination directories or symlinks |
+| Empty directories and internal hardlinks | Paths or links that escape the tree |
+| Relative symlinks within the tree | Devices, FIFOs and other special files |
+
+Recursive transfers require a running box with Python 3.11.8+ and temporary disk space on the controller. Individual-file transfers also work while a box is stopped.
+
+## File ownership
+
+Uploaded files are owned by `root` in the guest. Update ownership to `developer` when needed:
+
+```sh title="CONTROLLER: Change file ownership inside container"
+raft exec "$box" -- chown -R developer:developer /workspace/my-project
+```

package/docs/guides/long-running-services.mdx

@@ -0,0 +1,109 @@
+---
+title: Lifetime and persistent services
+description: Inspect deadlines, manage disposable and persistent boxes, and restart applications after resume.
+---
+
+# Lifetime and persistent services
+
+A box's lifetime controls when the host stops or deletes it. Activity, an open
+terminal, and detached jobs do not extend that deadline automatically.
+
+## Choose a lifetime and retention policy
+
+```sh title="CONTROLLER: Create a persistent box"
+box=$(raft new --location lab --ttl 3600)
+```
+
+TTL is in seconds: 60 to 2,592,000 (30 days), with a creation default of 600.
+The host sets the deadline after launch completes, using its own clock.
+Raft has no option to disable automatic expiry.
+
+| Operation | Persistent box (default) | Disposable box |
+| --- | --- | --- |
+| `stop` or TTL expiry | Stops processes; keeps disk and snapshots | Deletes the box and snapshots |
+| `resume --ttl SECONDS` | Starts the saved box with a new deadline | Only possible while the box still exists |
+| `destroy` | Deletes the box and snapshots | Deletes the box and snapshots |
+
+Use `raft new --disposable` for temporary work. A fork is persistent unless
+`--disposable` is explicitly supplied; recovered boxes are persistent.
+Back up work before stopping a disposable box.
+
+Persistent stopped boxes still occupy a saved slot and storage. Each host allows
+four saved boxes. `raft prune --older-than 7` previews old stopped boxes;
+`--yes` applies deletion. There is no automatic persistent-box retention cleanup.
+
+## Read and reset the deadline
+
+Select a running handle from `raft list`. These examples use controller `jq`:
+
+```sh title="CONTROLLER: Inspect and extend a running box"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual handle
+raft info "$box" | jq '{status, expires: .config["user.raft.expires"]}'
+raft extend "$box" --ttl 7200
+raft info "$box" | jq '{status, expires: .config["user.raft.expires"]}'
+```
+
+`user.raft.expires` is a Unix timestamp encoded as a string, not a remaining
+duration. `raft list` exposes it as `stopAt`; it may be empty on stopped boxes.
+`extend` sets `now + TTL`, rather than adding time to the old deadline.
+It requires a running box. Start a stopped box with `raft resume "$box" --ttl 7200`.
+
+The host timer checks every 15 seconds. Graceful shutdown, a host outage, or a
+long transaction holding the lifecycle lock can delay expiry. `raft gc --location lab`
+runs an immediate deadline check. A deadline is not a guarantee that a job will
+finish before shutdown; extend before starting work that needs more time.
+
+## Restart a service after resume
+
+One-off detached commands do not survive a stop. To start an application again
+after resume or fork, save a systemd unit on disk and enable it. This starts a new
+process; it does not recover process memory or an interrupted request.
+
+The following example serves `/workspace` as the guest `developer` user. Use a
+persistent running box and ensure files the application needs are readable by
+that user. For an actual application, replace `ExecStart` and the working directory.
+
+```sh title="CONTROLLER: Install and enable an example service"
+raft exec "$box" -- bash -lc 'cat > /etc/systemd/system/workspace-web.service <<"UNIT"
+[Unit]
+Description=Workspace HTTP example
+After=network.target
+
+[Service]
+User=developer
+WorkingDirectory=/workspace
+ExecStart=/usr/bin/python3 -m http.server 8080 --bind 0.0.0.0
+Restart=on-failure
+RestartSec=2
+
+[Install]
+WantedBy=multi-user.target
+UNIT
+systemctl daemon-reload
+systemctl enable --now workspace-web.service'
+raft exec "$box" -- systemctl status workspace-web.service --no-pager
+raft exec "$box" -- journalctl -u workspace-web.service -n 30 --no-pager
+```
+
+Open `raft forward "$box" --remote 8080 --local 8080` and visit
+`http://127.0.0.1:8080`. The tunnel runs in the foreground; close it with `Ctrl+C`.
+
+```sh title="CONTROLLER: Check service startup after resume"
+raft stop "$box"
+raft resume "$box" --ttl 3600
+raft exec "$box" -- systemctl is-active workspace-web.service
+```
+
+Reopen the tunnel after resume. A fork or snapshot template copies the enabled
+unit too, so its new box starts its own service. Application secrets on disk are
+also copied; see [Project setup and templates](./project-setup.mdx).
+
+To disable the example, run:
+
+```sh
+raft exec "$box" -- systemctl disable --now workspace-web.service
+```
+
+See [Background jobs](./background-jobs.mdx) for one-off tasks and
+[Desktop and private services](./desktop-services.mdx) for tunnel behavior.

package/docs/guides/meta.json

@@ -0,0 +1,18 @@
+{
+  "title": "Guides",
+  "collapsible": true,
+  "defaultOpen": false,
+  "pages": [
+    "project-setup",
+    "docker",
+    "files",
+    "background-jobs",
+    "long-running-services",
+    "scripting-ci",
+    "desktop-services",
+    "snapshots-forks",
+    "backup-recovery",
+    "upgrades",
+    "troubleshooting"
+  ]
+}