@microck/raft 0.0.0-stage → 0.2.0

package/docs/guides/background-jobs.mdx

@@ -0,0 +1,92 @@
+---
+title: Background jobs
+description: Run detached commands with systemd logging and process management.
+---
+
+# Background jobs
+
+Raft runs detached background commands under the guest systemd init system using transient service units.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Launching detached commands
+
+Start a background command by passing `--detach` to `raft exec`, delimiting the payload with `--`:
+
+```sh title="CONTROLLER: Start a detached background job"
+job=$(raft exec "$box" --detach -- bash -lc '
+echo "Task started at $(date)"
+sleep 60
+echo "Task finished at $(date)"
+')
+echo "Started background job: $job"
+```
+
+The command returns a 32-hex systemd unit name prefixed with `rfcmd-` (such as `rfcmd-a1b2c3d4e5f60718293a4b5c6d7e8f90`).
+
+## Checking job status
+
+Use `raft status` to query guest systemd unit properties:
+
+```sh title="CONTROLLER: Inspect background job state"
+raft status "$box" "$job"
+```
+
+Active jobs report running substate:
+```text
+ActiveState=active
+SubState=running
+ExecMainStatus=0
+```
+
+Because Raft runs detached commands with `RemainAfterExit=yes`, completed jobs retain their exit record:
+```text
+ActiveState=active
+SubState=exited
+ExecMainStatus=0
+```
+
+<Callout type="info" title="Exit status vs CLI return code">
+`ExecMainStatus` reports the command payload exit code inside the container. The `raft status` command returns exit code 0 when the query succeeds. It queries unit state and does not report process PIDs or start timestamps.
+</Callout>
+
+## Viewing job logs
+
+Retrieve output recorded in the guest systemd journal:
+
+```sh title="CONTROLLER: Retrieve job log output"
+raft logs "$box" "$job"
+```
+
+The command runs `journalctl --no-pager -o cat -u <job>` inside the container. It prints recorded logs without a pager or stream follow mode.
+
+## Cancelling jobs
+
+Terminate a running detached command:
+
+```sh title="CONTROLLER: Terminate background job"
+raft cancel "$box" "$job"
+```
+
+`raft cancel` executes `systemctl stop <job>`, terminating processes in the job's systemd control group.
+
+## Extending workspace lifetime
+
+<Callout type="warn" title="No process checkpointing">
+Stopping a workspace with `raft stop` or allowing its TTL to expire terminates all running background jobs. Raft does not checkpoint running process memory.
+</Callout>
+
+Extend the workspace lifetime before it expires:
+
+```sh title="CONTROLLER: Reset workspace expiration"
+raft extend "$box" --ttl 1800
+```
+
+`raft extend` resets the container expiration deadline to `now + TTL` seconds using the host clock. It does not add time to the existing deadline.

package/docs/guides/backup-recovery.mdx

@@ -0,0 +1,111 @@
+---
+title: Backup and recovery
+description: Export portable workspace archives and restore them across hosts.
+---
+
+# Backup and recovery
+
+Raft exports portable workspace archives and restores them onto compatible hosts for disaster recovery, migration, or offline archiving.
+
+## Target workspace handle
+
+Commands in this guide require a stopped workspace:
+
+```sh title="CONTROLLER: Select and stop target workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+raft stop "$box"
+```
+
+## Exporting a backup
+
+Specify a destination archive path on the controller:
+
+```sh title="CONTROLLER: Export stopped workspace to compressed archive"
+raft backup "$box" ./my-workspace-backup.tar.gz
+```
+
+`raft backup` exports container metadata, root filesystem, and snapshots into a gzip-compressed archive.
+
+### Security and storage requirements
+
+<Callout type="warn" title="Archives contain sensitive data">
+Backup archives contain the entire guest filesystem, including credentials, SSH keys, shell history, and private files. Native Incus archive configuration is not sanitized. Import only trusted archives created by Raft, and store backups on encrypted private storage off the source host for disaster recovery.
+</Callout>
+
+Raft enforces safeguards during export:
+
+- **Private permissions**: Destination mode must be `0600`. Group and world permissions must be zero before export begins.
+- **Atomic link publication**: After export and `fsync`, Raft publishes the archive using `os.link(archive.name, destination)`. Hard-link support is checked at final publication rather than preflight; export can succeed while publication fails if the filesystem lacks hard-link support.
+- **No overwriting**: The destination file must not already exist.
+- **Lifecycle lock**: Export holds `/run/lock/raft-incus.lock`. Expiry checks for other containers on the host are delayed while the lock is held.
+
+## Recovering an archive
+
+Import the archive onto the target location:
+
+```sh title="CONTROLLER: Import workspace backup onto target host"
+recovered=$(raft recover ./my-workspace-backup.tar.gz --location lab)
+echo "Recovered workspace: $recovered"
+```
+
+### Recovery process details
+
+- **Architecture compatibility**: Target host must match the native CPU architecture (ARM64 to ARM64, AMD64 to AMD64). Cross-architecture recovery is not supported.
+- **Fresh network identity**: The container receives a new name in project `raft`, a new MAC address on `rfbr0`, and no inherited TTL deadline.
+- **Stopped initial state**: Recovered workspaces remain stopped until started with `raft resume "$recovered" --ttl <SECONDS>`.
+- **Enforced admission limit**: The destination location must hold fewer than four boxes.
+- **Interrupted recovery**: Follow [Host crash during recovery](./troubleshooting.mdx#host-crash-during-recovery). Inspect the reported handle and container state before deleting any staging file.
+
+Start and verify the recovered workspace:
+
+```sh title="CONTROLLER: Start and verify recovered workspace"
+raft resume "$recovered" --ttl 600
+raft exec "$recovered" -- ls -la /workspace
+```
+
+## Cross-host migration workflow
+
+Follow these steps to migrate a workspace to a second host:
+
+### 1. Select and stop source workspace
+
+```sh title="CONTROLLER"
+raft list
+source_box="lab:rf-a1b2c3d4e5f60718"
+raft stop "$source_box"
+```
+
+### 2. Export archive to local controller storage
+
+```sh title="CONTROLLER"
+raft backup "$source_box" ./migration.tar.gz
+```
+
+### 3. Import archive onto destination host
+
+```sh title="CONTROLLER"
+target_box=$(raft recover ./migration.tar.gz --location second-host)
+```
+
+### 4. Resume workspace on destination host
+
+```sh title="CONTROLLER"
+raft resume "$target_box" --ttl 1800
+```
+
+### 5. Verify application files and operations
+
+```sh title="CONTROLLER"
+raft exec "$target_box" -- ls -la /workspace
+```
+
+After verifying the recovered workspace, optionally destroy the source container:
+
+```sh title="CONTROLLER: Optional: Destroy original container"
+raft destroy "$source_box"
+```
+
+<Callout type="warn" title="Preserve backup archives">
+Do not delete your local backup archive until you confirm that the destination workspace operates normally.
+</Callout>

package/docs/guides/desktop-services.mdx

@@ -0,0 +1,87 @@
+---
+title: Desktop and private services
+description: Forward network ports and connect to an on-demand graphical desktop.
+---
+
+# Desktop and private services
+
+Forward network ports and connect to an on-demand graphical desktop over authenticated host SSH tunnels bound to controller loopback.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Forwarding network ports
+
+Create a secure tunnel between your local controller and a workspace service:
+
+```text
+raft forward "$box" --remote <REMOTE_PORT> --local <LOCAL_PORT>
+```
+
+### End-to-end example: Running a web service
+
+Start a detached HTTP server inside the workspace:
+
+```sh title="CONTROLLER: Start detached HTTP server inside workspace"
+job=$(raft exec "$box" --detach -- python3 -m http.server 8080 --bind 0.0.0.0)
+echo "Started web service job: $job"
+```
+
+Open the forwarding tunnel:
+
+```sh title="CONTROLLER: Open port forwarding tunnel"
+raft forward "$box" --remote 8080 --local 8080
+```
+
+`raft forward` runs in the foreground. Open `http://127.0.0.1:8080` in your browser or run `curl http://127.0.0.1:8080` in another terminal.
+
+When finished:
+1. Press `Ctrl+C` to close the tunnel.
+2. Cancel the background job:
+   ```sh title="CONTROLLER: Stop the background server job"
+   raft cancel "$box" "$job"
+   ```
+
+### Binding requirement
+
+<Callout type="warn" title="Bind to all interfaces or guest IP">
+Services inside the container must bind to all interfaces (`0.0.0.0`) or to the container's private `eth0` IP address. A service listening strictly on container loopback (`127.0.0.1`) cannot receive traffic routed across the container bridge.
+</Callout>
+
+The controller listener binds strictly to `127.0.0.1` on your local workstation, without exposing ports to your local network.
+
+## On-demand desktop environment
+
+The development image includes a graphical desktop stack using Xvfb (display `:99`), Openbox, x11vnc, and noVNC.
+
+Start the desktop and establish a private tunnel:
+
+```sh title="CONTROLLER: Launch desktop and establish private tunnel"
+raft desktop "$box" --local 6080
+```
+
+`raft desktop` starts `raft-desktop.service` inside the container if needed, waits for Xvfb and VNC readiness, and opens a tunnel to local port `6080`.
+
+Open the client in your browser:
+
+```text
+http://127.0.0.1:6080/vnc.html
+```
+
+<Callout type="info" title="Browser URL requirement">
+Include `/vnc.html` in the URL. The root path `/` displays an index directory; `/vnc.html` loads the desktop canvas with mouse, keyboard, and clipboard support.
+</Callout>
+
+### Desktop characteristics
+
+- **Display environment**: Virtual X display `:99` at 1280x720 resolution with 24-bit color.
+- **User session**: Openbox runs under the `developer` user account.
+- **Clipboard support**: The noVNC sidebar provides a clipboard drawer for text exchange.
+- **Tunnel lifecycle**: Pressing `Ctrl+C` closes the local tunnel. The desktop service keeps running until the workspace stops or expires.
+- **Reconnecting**: Running `raft desktop "$box" --local 6080` reconnects without interrupting open windows.

package/docs/guides/docker.mdx

@@ -0,0 +1,84 @@
+---
+title: Docker in workspaces
+description: Run Docker daemons and build container images inside unprivileged Raft workspaces.
+---
+
+# Docker in workspaces
+
+Raft development images include Docker daemon and CLI tooling running directly in the unprivileged container without mounting the host Docker socket.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Verifying Docker status
+
+Verify that the guest Docker daemon is initialized and responsive:
+
+```sh title="CONTROLLER: Check Docker version and daemon status"
+raft exec "$box" -- docker version
+raft exec "$box" -- docker info
+```
+
+<Callout type="info" title="Daemon readiness verification">
+`docker version` and `docker info` query guest `dockerd`. Running `docker --version` checks only the client binary without testing daemon health.
+</Callout>
+
+## Running containers
+
+Run containers inside your workspace:
+
+```sh title="CONTROLLER: Run an Alpine container"
+raft exec "$box" -- docker run --rm alpine uname -a
+```
+
+Start background services inside the workspace:
+
+```sh title="CONTROLLER: Start a background Redis container"
+raft exec "$box" -- docker run -d --name cache -p 6379:6379 redis:alpine
+raft exec "$box" -- docker ps
+```
+
+## Building container images
+
+Build container images within the workspace:
+
+```sh title="CONTROLLER: Create Dockerfile and build image"
+raft exec "$box" -- bash -lc '
+mkdir -p /workspace/app
+cat << "EOF" > /workspace/app/Dockerfile
+FROM alpine:latest
+RUN echo "hello from workspace container" > /message.txt
+CMD ["cat", "/message.txt"]
+EOF
+cd /workspace/app
+docker build -t my-app .
+docker run --rm my-app
+'
+```
+
+## Managing disk space
+
+Nested container layers and volumes reside on the shared 60 GiB `raft-data` Btrfs storage pool.
+
+Prune dangling images without deleting active container data:
+
+```sh title="CONTROLLER: Prune dangling images"
+raft exec "$box" -- docker image prune -f
+```
+
+<Callout type="warn" title="Data loss risk with system prune">
+Running `docker system prune -af --volumes` removes all stopped containers and unused volumes inside the workspace. Run full pruning only when you intend to discard local container state and caches.
+</Callout>
+
+## Isolation and runtime limits
+
+- **Daemon confinement**: The Docker daemon runs as a guest systemd unit using `/var/run/docker.sock`. The host Docker daemon is never mounted.
+- **Kernel sharing**: Workspaces share the host Linux kernel without host kernel module or KVM access.
+- **Syscall interception**: Incus sets `security.nesting=true`, `security.syscalls.intercept.mknod=true`, and `security.syscalls.intercept.setxattr=true` to manage image layers safely in unprivileged user namespaces.
+- **Shutdown behavior**: Stopping a workspace with `raft stop` stops guest processes and nested containers.

package/docs/guides/files.mdx

@@ -0,0 +1,91 @@
+---
+title: File transfer
+description: Transfer files between your controller machine and running Raft workspaces.
+---
+
+# File transfer
+
+Upload and download individual files between your local controller and running workspaces over verified host SSH and Incus instance APIs.
+
+## Target workspace handle
+
+Commands in this guide target a running workspace. Select one from `raft list` (see [First working box](../first-box.mdx)):
+
+```sh title="CONTROLLER: Select running workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+```
+
+## Uploading files
+
+Transfer a file from your controller to an absolute path inside the workspace:
+
+```sh title="CONTROLLER: Create fixture and upload to workspace"
+# Create local sample file on controller
+echo "Hello from controller" > ./local-file.txt
+
+# Upload to workspace
+raft upload "$box" ./local-file.txt /workspace/target-file.txt
+```
+
+Existing destination files are overwritten. Create parent directories before uploading if needed:
+
+```sh title="CONTROLLER: Create parent directory and upload"
+echo '{"mode": "debug"}' > ./settings.json
+raft exec "$box" -- mkdir -p /workspace/configs
+raft upload "$box" ./settings.json /workspace/configs/settings.json
+```
+
+<Callout type="info" title="Absolute guest paths required">
+Guest file paths must be absolute paths beginning with `/`. Relative paths inside the container are rejected by the CLI parser.
+</Callout>
+
+## Downloading files
+
+Download a file from a workspace to your controller:
+
+```sh title="CONTROLLER: Download a file from workspace"
+raft download "$box" /workspace/target-file.txt ./downloaded-file.txt
+```
+
+The destination path on the controller must specify the target filename.
+
+## Paths with spaces
+
+Wrap paths in quotes when transferring files with spaces:
+
+```sh title="CONTROLLER: Transfer files with spaces in paths"
+mkdir -p "./my documents"
+echo "col1,col2" > "./my documents/data file.csv"
+
+raft exec "$box" -- mkdir -p "/workspace/my documents"
+raft upload "$box" "./my documents/data file.csv" "/workspace/my documents/data file.csv"
+raft download "$box" "/workspace/my documents/data file.csv" "./downloaded-data.csv"
+```
+
+## Transferring directories
+
+Use `--recursive` to copy a tree into a new directory:
+
+```sh
+raft upload "$box" ./my-project /workspace/my-project --recursive
+raft download "$box" /workspace/my-project ./project-copy --recursive
+```
+
+The destination must not exist, and its parent must exist. Raft publishes the directory after the complete transfer; it does not merge or overwrite trees.
+
+| Preserved | Rejected |
+| --- | --- |
+| File contents and executable bits | Existing destination directories or symlinks |
+| Empty directories and internal hardlinks | Paths or links that escape the tree |
+| Relative symlinks within the tree | Devices, FIFOs and other special files |
+
+Recursive transfers require a running box with Python 3.11.8+ and temporary disk space on the controller. Individual-file transfers also work while a box is stopped.
+
+## File ownership
+
+Uploaded files are owned by `root` in the guest. Update ownership to `developer` when needed:
+
+```sh title="CONTROLLER: Change file ownership inside container"
+raft exec "$box" -- chown -R developer:developer /workspace/my-project
+```

package/docs/guides/meta.json

@@ -0,0 +1,14 @@
+{
+  "title": "Guides",
+  "collapsible": true,
+  "defaultOpen": false,
+  "pages": [
+    "docker",
+    "files",
+    "background-jobs",
+    "desktop-services",
+    "snapshots-forks",
+    "backup-recovery",
+    "troubleshooting"
+  ]
+}

package/docs/guides/snapshots-forks.mdx

@@ -0,0 +1,107 @@
+---
+title: Snapshots and forks
+description: Capture point-in-time filesystem snapshots and fork independent workspace clones.
+---
+
+# Snapshots and forks
+
+Raft captures filesystem snapshots, rolls back changes, and clones workspaces on the same host.
+
+## Target workspace handle
+
+Commands in this guide require a stopped workspace:
+
+```sh title="CONTROLLER: Select and stop target workspace"
+raft list
+box="lab:rf-a1b2c3d4e5f60718"  # Replace with your actual workspace handle
+raft stop "$box"
+```
+
+## Stopped-source requirement
+
+<Callout type="warn" title="Workspace must be stopped">
+Creating a snapshot, restoring from a snapshot, or forking requires a stopped container. Operations run under host lock `/run/lock/raft-incus.lock` to prevent concurrent disk modification.
+</Callout>
+
+## Creating snapshots
+
+Take a point-in-time snapshot of the stopped workspace filesystem:
+
+```sh title="CONTROLLER: Create a named snapshot"
+raft snapshot "$box" pre-upgrade
+```
+
+Snapshot names must use letters, digits, underscores, or hyphens up to 64 characters.
+
+## Listing snapshots
+
+View existing snapshots for a workspace:
+
+```sh title="CONTROLLER: List existing snapshots"
+raft snapshots "$box"
+```
+
+The command prints snapshot names and creation timestamps from Incus metadata.
+
+## Restoring from a snapshot
+
+Roll back a stopped workspace to a previously recorded snapshot:
+
+```sh title="CONTROLLER: Restore snapshot"
+raft restore "$box" pre-upgrade
+```
+
+<Callout type="info" title="Configuration preservation during restore">
+`raft restore` executes Incus snapshot restore with `--diskonly`. It rolls back filesystem blocks while preserving container configuration, CPU and memory limits, and the `eth0` network MAC address.
+</Callout>
+
+Resume the workspace with a renewed lifetime:
+
+```sh title="CONTROLLER: Resume restored workspace"
+raft resume "$box" --ttl 600
+```
+
+## Forking a workspace
+
+Forking creates an independent clone on the same host:
+
+```sh title="CONTROLLER: Fork workspace into a new clone"
+copy=$(raft fork "$box" --ttl 600)
+echo "Created fork: $copy"
+```
+
+### Characteristics of forked workspaces
+
+- **Instance-only copy**: Uses Incus `--instance-only`. Filesystem data, installed packages, and `/workspace` files are copied; source snapshots are not inherited.
+- **Immediate start**: `raft fork` launches the new container immediately with the specified TTL while the source remains stopped.
+- **Fresh network identity**: The fork receives a new container name, a fresh MAC address on `rfbr0`, and an independent IP address.
+- **Capacity accounting**: The fork counts toward the location's four-saved-box limit and uses storage from the shared 60 GiB Btrfs pool.
+
+Destroy the clone when finished to release capacity:
+
+```sh title="CONTROLLER: Destroy forked workspace"
+raft destroy "$copy"
+```
+
+## Starting from a prepared snapshot
+
+```sh
+copy=$(raft new --from "$box/pre-upgrade" --ttl 600)
+```
+
+The source box can be running because Raft copies the saved snapshot. The new box stays on the same host, uses normal creation sizing defaults and has a fresh network identity. Filesystem secrets are copied too.
+
+Remove a snapshot you no longer need:
+
+```sh
+raft snapshot-delete "$box" pre-upgrade
+```
+
+## Choosing clone resources
+
+```sh
+copy=$(raft fork "$box" --ttl 600 --cpu 2 --memory 4GiB)
+raft resume "$box" --ttl 600 --memory 2GiB
+```
+
+Omitted sizing values keep the source limits. Forks are persistent by default; add `--disposable` to delete a clone when it stops or expires.