@microck/raft 0.4.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +30 -14
- package/deploy/deploy.py +112 -19
- package/deploy/install-incus.sh +10 -0
- package/deploy/raft-expire.py +109 -31
- package/deploy/verify-controls.py +34 -48
- package/deploy/verify-features.py +592 -3
- package/deploy/verify-fresh-host.py +64 -1
- package/deploy/verify-limits.py +327 -72
- package/deploy/verify-live.py +56 -31
- package/deploy/verify-self-update.py +372 -0
- package/docs/boat-capabilities.md +8 -7
- package/docs/boot-performance.mdx +1 -1
- package/docs/contract.md +33 -23
- package/docs/first-box.mdx +1 -1
- package/docs/guides/backup-recovery.mdx +2 -2
- package/docs/guides/config-policy.mdx +253 -0
- package/docs/guides/docker.mdx +9 -1
- package/docs/guides/long-running-services.mdx +10 -4
- package/docs/guides/meta.json +1 -0
- package/docs/guides/project-setup.mdx +4 -3
- package/docs/guides/scripting-ci.mdx +27 -10
- package/docs/guides/snapshots-forks.mdx +9 -3
- package/docs/guides/troubleshooting.mdx +37 -8
- package/docs/guides/upgrades.mdx +17 -5
- package/docs/index.mdx +1 -0
- package/docs/operations.mdx +16 -14
- package/docs/reference/capacity.mdx +198 -0
- package/docs/reference/cli.md +178 -35
- package/docs/reference/configuration.mdx +40 -2
- package/docs/reference/meta.json +1 -1
- package/docs/releases/0.5.0.md +19 -0
- package/docs/releases/0.6.0.md +16 -0
- package/docs/requirements.mdx +2 -2
- package/docs/verification.mdx +14 -8
- package/package.json +1 -1
- package/raft.py +1181 -128
- package/skills/raft-cli/SKILL.md +79 -13
- package/docs/reference/capacity-limits.mdx +0 -92
|
@@ -22,19 +22,19 @@ raft doctor
|
|
|
22
22
|
- Compatibility of the installed expiry worker with the CLI
|
|
23
23
|
- Active status of host systemd services (`incus`, `raft-expire.timer`, `raft-network.service`)
|
|
24
24
|
- Host root filesystem disk usage (`df -h /`)
|
|
25
|
-
- Status and usage of the `raft-data` Btrfs pool
|
|
25
|
+
- Status and usage of the shared `raft-data` Btrfs pool, and of the `raft-disks` LVM thin pool when own box disks are enabled (otherwise `own box disks: not enabled (redeploy with --disk-pool SIZE)`)
|
|
26
26
|
- Count of existing saved containers
|
|
27
27
|
|
|
28
28
|
<Callout type="info" title="Doctor scope">
|
|
29
|
-
`raft doctor` reports runtime daemon, expiry-worker compatibility, and storage status. It does not validate image fingerprints, inspect nftables bridge rules,
|
|
29
|
+
`raft doctor` reports runtime daemon, expiry-worker compatibility, and storage status. It does not validate image fingerprints, inspect nftables bridge rules, or emit 5 GiB low-disk warnings. A host without `raft-disks` is healthy: own box disks are opt-in.
|
|
30
30
|
</Callout>
|
|
31
31
|
|
|
32
32
|
### Expiry worker differs from the CLI
|
|
33
33
|
|
|
34
34
|
An active expiration timer can still run an older worker. Installing or updating
|
|
35
35
|
the controller CLI does not deploy host code. `doctor` checks the same worker
|
|
36
|
-
protocol as disposable creation and retention
|
|
37
|
-
changing any boxes. It exits nonzero and names the location on a mismatch.
|
|
36
|
+
protocol as disposable creation, prune and setting a retention policy, without
|
|
37
|
+
running expiry or changing any boxes. It exits nonzero and names the location on a mismatch.
|
|
38
38
|
|
|
39
39
|
Review the host deployment changes, then deploy from the checkout matching your
|
|
40
40
|
CLI. Replace `lab` with the location reported by the diagnostic:
|
|
@@ -106,6 +106,23 @@ files and snapshots. Expired disposable containers are deleted, including their
|
|
|
106
106
|
snapshots. Use `raft gc --location lab` to check only one host. The timer normally
|
|
107
107
|
checks every 15 seconds; host outages, graceful shutdown and long locked
|
|
108
108
|
operations can delay expiry. See [Lifetime and persistent services](./long-running-services.mdx).
|
|
109
|
+
`raft gc` also applies the location's stopped-box retention.
|
|
110
|
+
|
|
111
|
+
### A stopped box disappeared
|
|
112
|
+
|
|
113
|
+
A host retention policy deletes boxes left stopped for its hours. Check the
|
|
114
|
+
policy and the host journal, which records every deletion:
|
|
115
|
+
|
|
116
|
+
```sh title="CONTROLLER: Check retention"
|
|
117
|
+
raft config | jq '.policy'
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
```sh title="HOST: Find the deletion"
|
|
121
|
+
sudo journalctl -t raft | grep 'Deleted box'
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Deleted boxes cannot be recovered except from a `raft backup` archive. See
|
|
125
|
+
[Defaults and host policies](./config-policy.mdx#delete-boxes-left-stopped).
|
|
109
126
|
|
|
110
127
|
## Common error scenarios
|
|
111
128
|
|
|
@@ -120,13 +137,25 @@ The configuration file contains valid JSON that is `{}` or is not an object (an
|
|
|
120
137
|
#### "Build an image and configure its full immutable fingerprint before creating boxes"
|
|
121
138
|
The `image` field is missing or is not a 64-character hexadecimal SHA-256 fingerprint. Supply the full fingerprint from `sudo incus --project raft image list raft-dev --format json`.
|
|
122
139
|
|
|
123
|
-
###
|
|
140
|
+
### Sizing and disk errors
|
|
141
|
+
|
|
142
|
+
Raft has no box count limit and no size budget. These errors come from the
|
|
143
|
+
request itself or from the host:
|
|
144
|
+
|
|
145
|
+
#### "Requested N CPUs, but this host has M" or "Requested SIZE RAM, but this host has N GiB"
|
|
146
|
+
The request is larger than the whole host. Ask for at most the host's CPUs and RAM; `raft capacity` shows both.
|
|
147
|
+
|
|
148
|
+
#### "LOCATION has no raft-disks pool, so own box disks are not enabled there"
|
|
149
|
+
Own box disks are opt-in and this host was deployed without them. Nothing was created. Enable them with `python3 deploy/deploy.py --location <location> --disk-pool SIZE`, which needs SIZE plus 20 GiB free on the host, or create the box without `--disk`. Boxes without `--disk` keep working meanwhile.
|
|
150
|
+
|
|
151
|
+
#### "own box disks would need N GiB more of raft-disks"
|
|
152
|
+
Each own disk, each snapshot of one, and the cached image reserve their full size in `raft-disks`, so Raft refused the request before creating anything. Destroy own-disk boxes or snapshots you no longer need, choose a smaller `--disk`, or use the shared pool. `raft capacity` shows the unreserved space. See [Reservation accounting](../reference/capacity.mdx#reservation-accounting).
|
|
124
153
|
|
|
125
|
-
#### "
|
|
126
|
-
|
|
154
|
+
#### "No space left on device" inside an own-disk box
|
|
155
|
+
The box filled its own disk. Delete files or Docker data inside the box. For a persistent box you can also move to a larger disk: stop it with `raft stop "$box"` (snapshots need a stopped box), take `raft snapshot "$box" <snapshot>`, then create the copy with `raft new --from "$box/<snapshot>" --disk <larger size>`. Do not stop a `--disposable` box for this: stopping deletes it.
|
|
127
156
|
|
|
128
157
|
#### Low disk warnings
|
|
129
|
-
`raft
|
|
158
|
+
`raft capacity` warns when free space on host disk or the `raft-data` pool falls below 5 GiB. This warning is diagnostic and does not halt container operations. Clean up dangling images, old snapshots, or unused containers to free space.
|
|
130
159
|
|
|
131
160
|
### Network and forwarding issues
|
|
132
161
|
|
package/docs/guides/upgrades.mdx
CHANGED
|
@@ -10,7 +10,7 @@ parts of Raft. Updating one does not automatically update the others.
|
|
|
10
10
|
|
|
11
11
|
| Component | Update method | Effect on existing boxes |
|
|
12
12
|
| --- | --- | --- |
|
|
13
|
-
| Controller CLI |
|
|
13
|
+
| Controller CLI | `raft self-update`, or install the chosen npm or Python release | Changes controller commands; does not rewrite guest files |
|
|
14
14
|
| Host worker and firewall | Run the chosen release's deployer per host | Replaces host policy code and reapplies filtering |
|
|
15
15
|
| Pinned development image | Import an image and change the configuration fingerprint | Used for future image-based `new`; existing boxes keep their files |
|
|
16
16
|
|
|
@@ -33,7 +33,14 @@ stopping one. Schedule redeployment when interruptions are acceptable.
|
|
|
33
33
|
|
|
34
34
|
## Update the controller
|
|
35
35
|
|
|
36
|
-
|
|
36
|
+
`raft self-update` installs the latest release through the channel that
|
|
37
|
+
installed the CLI, and `raft self-update --version 0.2.0` installs a chosen
|
|
38
|
+
one. `raft self-update --check` only reports. For npm installs, npm checks the
|
|
39
|
+
registry integrity hash of a package published with provenance. For uv tools,
|
|
40
|
+
Raft downloads the release wheel and refuses it unless its SHA-256 matches the
|
|
41
|
+
release `SHA256SUMS`. See [`self-update`](../reference/cli.md#self-update).
|
|
42
|
+
|
|
43
|
+
To install a release by hand instead, for an npm installation:
|
|
37
44
|
|
|
38
45
|
```sh title="CONTROLLER: Install a pinned CLI release"
|
|
39
46
|
npm install -g @microck/raft@0.2.0
|
|
@@ -54,7 +61,10 @@ and check which executable your shell uses with `command -v raft`.
|
|
|
54
61
|
|
|
55
62
|
For an editable source installation, updating the checkout changes the installed
|
|
56
63
|
tool's source. Keep uncommitted work separate and run `uv sync --locked` for the
|
|
57
|
-
chosen checkout.
|
|
64
|
+
chosen checkout. `self-update` refuses source checkouts.
|
|
65
|
+
|
|
66
|
+
Updating the CLI changes no host. Continue below, and run `raft doctor` first: it
|
|
67
|
+
reports each host whose worker differs from the new CLI.
|
|
58
68
|
|
|
59
69
|
## Redeploy each host
|
|
60
70
|
|
|
@@ -74,8 +84,10 @@ timer. It does not import an image or migrate guest files.
|
|
|
74
84
|
|
|
75
85
|
Do not bypass a conflicting-resource or storage-size error. Inspect the host and
|
|
76
86
|
preserve backups before explicit recovery; redeployment does not repair arbitrary
|
|
77
|
-
infrastructure drift. Disposable creation/fork
|
|
78
|
-
and fail if it differs from the CLI. That check
|
|
87
|
+
infrastructure drift. Disposable creation/fork, prune and setting a retention
|
|
88
|
+
policy check the worker protocol and fail if it differs from the CLI. That check
|
|
89
|
+
is not an automatic upgrade. Raft 0.6.0 raised the protocol for stopped-box
|
|
90
|
+
retention, so hosts deployed by an earlier release need this redeploy.
|
|
79
91
|
|
|
80
92
|
## Replace the image pin
|
|
81
93
|
|
package/docs/index.mdx
CHANGED
|
@@ -33,6 +33,7 @@ Stopping a workspace preserves files and packages on disk. Resuming resets its r
|
|
|
33
33
|
| Run detached commands with systemd logs | [Background jobs](./guides/background-jobs.mdx) |
|
|
34
34
|
| Manage deadlines and restart services after resume | [Lifetime and persistent services](./guides/long-running-services.mdx) |
|
|
35
35
|
| Automate the CLI and parse its output | [Scripting and CI](./guides/scripting-ci.mdx) |
|
|
36
|
+
| Set defaults for new boxes, require box note fields or delete boxes left stopped | [Defaults and host policies](./guides/config-policy.mdx) |
|
|
36
37
|
| Forward guest ports or open noVNC desktop | [Desktop and private services](./guides/desktop-services.mdx) |
|
|
37
38
|
| Capture disk state or clone workspaces | [Snapshots and forks](./guides/snapshots-forks.mdx) |
|
|
38
39
|
| Export portable backups across hosts | [Backup and recovery](./guides/backup-recovery.mdx) |
|
package/docs/operations.mdx
CHANGED
|
@@ -15,7 +15,7 @@ Install the CLI from npm on your Linux controller, confirm key-based SSH to the
|
|
|
15
15
|
host, then let `raft host add` do the remaining steps:
|
|
16
16
|
|
|
17
17
|
```sh title="CONTROLLER: Onboard a host"
|
|
18
|
-
npm install -g @microck/raft@0.
|
|
18
|
+
npm install -g @microck/raft@0.6.0
|
|
19
19
|
ssh ubuntu@192.0.2.10 true
|
|
20
20
|
raft host add lab --ssh ubuntu@192.0.2.10 --dry-run
|
|
21
21
|
raft host add lab --ssh ubuntu@192.0.2.10
|
|
@@ -44,7 +44,7 @@ Connect to your host over SSH, verify its host key fingerprint, and install base
|
|
|
44
44
|
```sh title="HOST: Install baseline system packages"
|
|
45
45
|
sudo apt-get update
|
|
46
46
|
sudo apt-get install -y --no-install-recommends \
|
|
47
|
-
ca-certificates curl gnupg python3 openssh-server iptables btrfs-progs
|
|
47
|
+
ca-certificates curl gnupg python3 openssh-server iptables btrfs-progs lvm2 python3-yaml
|
|
48
48
|
```
|
|
49
49
|
|
|
50
50
|
Verify passwordless `sudo` on the host:
|
|
@@ -62,7 +62,7 @@ sudo -n true && echo "Passwordless sudo is functional"
|
|
|
62
62
|
On your Linux controller with Python 3.11.8+, install the published CLI:
|
|
63
63
|
|
|
64
64
|
```sh title="CONTROLLER: Install published CLI"
|
|
65
|
-
npm install -g @microck/raft@0.
|
|
65
|
+
npm install -g @microck/raft@0.6.0
|
|
66
66
|
raft --version
|
|
67
67
|
```
|
|
68
68
|
|
|
@@ -70,7 +70,7 @@ Installation does not configure hosts or download images. Clone the matching
|
|
|
70
70
|
release to obtain the deployment scripts and example configuration:
|
|
71
71
|
|
|
72
72
|
```sh title="CONTROLLER: Obtain deployment scripts"
|
|
73
|
-
git clone --branch v0.
|
|
73
|
+
git clone --branch v0.6.0 https://github.com/Microck/raft.git raft
|
|
74
74
|
cd raft
|
|
75
75
|
```
|
|
76
76
|
|
|
@@ -132,15 +132,17 @@ Run the automated host deployer from your controller:
|
|
|
132
132
|
|
|
133
133
|
```sh title="CONTROLLER: Provision host infrastructure"
|
|
134
134
|
python3 deploy/deploy.py --location lab
|
|
135
|
+
# Optional: enable own box disks with a pool of this size, now or on any later redeploy
|
|
136
|
+
python3 deploy/deploy.py --location lab --disk-pool 100GiB
|
|
135
137
|
```
|
|
136
138
|
|
|
137
139
|
<details>
|
|
138
140
|
<summary>Infrastructure created by the deployer</summary>
|
|
139
141
|
|
|
140
142
|
The deployer runs over SSH and configures the host:
|
|
141
|
-
1. Installs Incus 6.0 LTS
|
|
143
|
+
1. Installs Incus 6.0 LTS, LVM, nftables and python3-yaml packages.
|
|
142
144
|
2. Creates the isolated `raft` Incus project.
|
|
143
|
-
3. Creates
|
|
145
|
+
3. Creates the shared 60 GiB Btrfs storage pool (`raft-data`). With `--disk-pool SIZE`, also creates the LVM thin pool for [own box disks](./reference/capacity.mdx#own-box-disks) (`raft-disks`) of SIZE, allocates its file on the host disk in full and stops it passing discards to that file. Without `--disk-pool` own box disks stay disabled, which is a healthy state. A later redeploy with `--disk-pool` adds the pool; no redeploy resizes either pool.
|
|
144
146
|
4. Creates private bridge interface `rfbr0` (`10.232.0.1/24`) with NAT.
|
|
145
147
|
5. Configures nftables bridge filters to block inter-workspace traffic and restrict host access.
|
|
146
148
|
6. Installs and enables `raft-network.service` and `raft-expire.timer` units.
|
|
@@ -159,16 +161,16 @@ Workspaces require an unprivileged Debian 13 development image matching the host
|
|
|
159
161
|
|
|
160
162
|
<Tab value="Download release image">
|
|
161
163
|
|
|
162
|
-
Download the release archive, checksum, and manifest from [Raft releases](https://github.com/Microck/raft/releases). These examples pin `v0.
|
|
164
|
+
Download the release archive, checksum, and manifest from [Raft releases](https://github.com/Microck/raft/releases). These examples pin `v0.6.0`, matching the CLI and deployment checkout above. The `sha256sum` check gates import so corrupted archives are rejected:
|
|
163
165
|
|
|
164
166
|
<Tabs items={["ARM64", "AMD64"]}>
|
|
165
167
|
|
|
166
168
|
<Tab value="ARM64">
|
|
167
169
|
|
|
168
170
|
```sh title="HOST (ARM64): Download, verify, and import release image"
|
|
169
|
-
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.
|
|
170
|
-
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.
|
|
171
|
-
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.
|
|
171
|
+
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.6.0/raft-dev-arm64.tar.gz
|
|
172
|
+
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.6.0/SHA256SUMS
|
|
173
|
+
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.6.0/manifest-arm64.json
|
|
172
174
|
|
|
173
175
|
sha256sum --ignore-missing -c SHA256SUMS && \
|
|
174
176
|
sudo incus --project raft image import ./raft-dev-arm64.tar.gz --alias raft-dev
|
|
@@ -179,9 +181,9 @@ sha256sum --ignore-missing -c SHA256SUMS && \
|
|
|
179
181
|
<Tab value="AMD64">
|
|
180
182
|
|
|
181
183
|
```sh title="HOST (AMD64): Download, verify, and import release image"
|
|
182
|
-
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.
|
|
183
|
-
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.
|
|
184
|
-
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.
|
|
184
|
+
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.6.0/raft-dev-amd64.tar.gz
|
|
185
|
+
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.6.0/SHA256SUMS
|
|
186
|
+
curl -fsSL -O https://github.com/Microck/raft/releases/download/v0.6.0/manifest-amd64.json
|
|
185
187
|
|
|
186
188
|
sha256sum --ignore-missing -c SHA256SUMS && \
|
|
187
189
|
sudo incus --project raft image import ./raft-dev-amd64.tar.gz --alias raft-dev
|
|
@@ -260,4 +262,4 @@ Check that Incus, the network service, and the expiration timer are active. For
|
|
|
260
262
|
|
|
261
263
|
Proceed to [First working box](./first-box.mdx) to launch, verify, and manage your first workspace.
|
|
262
264
|
|
|
263
|
-
For later releases, follow [Upgrades and image changes](./guides/upgrades.mdx).
|
|
265
|
+
For later releases, run `raft self-update` and follow [Upgrades and image changes](./guides/upgrades.mdx).
|
|
@@ -0,0 +1,198 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Box sizing and capacity
|
|
3
|
+
description: Size boxes for the task, give them their own disks, and read the advisory host capacity estimate.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Box sizing and capacity
|
|
7
|
+
|
|
8
|
+
<Callout type="info" title="Advisory estimate only">
|
|
9
|
+
Raft enforces no box count or size budget. Create the boxes your task needs, sized for the work. `raft capacity` is planning advice, never a reason to skip a box.
|
|
10
|
+
</Callout>
|
|
11
|
+
|
|
12
|
+
## Sizing a box
|
|
13
|
+
|
|
14
|
+
Choose CPU, RAM and disk when you run `raft new`:
|
|
15
|
+
|
|
16
|
+
- `--cpu N`: any whole number of CPUs, from `1` up to the host's CPU count (default: `1`). Boxes share host CPUs, so a limit is a ceiling, not a reservation.
|
|
17
|
+
- `--memory SIZE`: any whole number of GiB, such as `3GiB` or `32GiB`, up to the host's RAM (default: `2GiB`). This is the box's cgroup memory limit.
|
|
18
|
+
- `--disk SIZE`: give the box its own disk instead of the shared pool, on hosts that enable own box disks. See [Own box disks](#own-box-disks).
|
|
19
|
+
|
|
20
|
+
A box from the image starts from these defaults, or from the controller's
|
|
21
|
+
[`config.json` defaults](../guides/config-policy.mdx#defaults-for-new) when set.
|
|
22
|
+
A box from a snapshot (`new --from`) or a fork keeps the source's CPU, RAM and
|
|
23
|
+
disk instead of any default, and each flag you pass overrides only its own value.
|
|
24
|
+
|
|
25
|
+
```sh
|
|
26
|
+
box=$(raft new --cpu 4 --memory 16GiB --disk 40GiB --ttl 3600 --purpose "Large build")
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
A CPU or RAM request larger than the whole host fails before anything is
|
|
30
|
+
created, because Incus would accept it and quietly give the box less:
|
|
31
|
+
|
|
32
|
+
```text
|
|
33
|
+
Requested 64 CPUs, but this host has 8
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Change CPU and RAM when you resume a stopped box or fork one:
|
|
37
|
+
|
|
38
|
+
```sh
|
|
39
|
+
raft resume "$box" --ttl 1800 --cpu 2 --memory 4GiB
|
|
40
|
+
copy=$(raft fork "$box" --ttl 1800 --memory 2GiB)
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
Resume requires a stopped box; fork requires a stopped source. Omitted values
|
|
44
|
+
stay unchanged on resume and inherit the source limits on fork. Live resizing is
|
|
45
|
+
not exposed by the CLI. See [Snapshots and forks](../guides/snapshots-forks.mdx).
|
|
46
|
+
|
|
47
|
+
## Own box disks
|
|
48
|
+
|
|
49
|
+
By default every box stores its files in the shared 60 GiB Btrfs pool
|
|
50
|
+
`raft-data`. Boxes there grow until the pool is full, and Btrfs quotas cannot
|
|
51
|
+
hold them back: Docker and other tools create nested subvolumes that escape
|
|
52
|
+
Btrfs quotas.
|
|
53
|
+
|
|
54
|
+
`raft new --disk SIZE` gives the box its own disk instead: an LVM thin volume
|
|
55
|
+
of SIZE with an ext4 filesystem in the `raft-disks` pool. Everything the box
|
|
56
|
+
writes, including Docker images, layers and container files, lands in that
|
|
57
|
+
filesystem, so the box gets "No space left on device" at SIZE and cannot touch
|
|
58
|
+
other boxes' space. Raft reserves the whole SIZE in the pool when the box is
|
|
59
|
+
created, so other boxes cannot take it either.
|
|
60
|
+
|
|
61
|
+
Own box disks are opt-in per host. A host deployed without `--disk-pool` has no
|
|
62
|
+
`raft-disks` pool, which is a normal, healthy state: `raft doctor` and
|
|
63
|
+
`raft capacity` say own box disks are not enabled, and `new --disk` fails before
|
|
64
|
+
creating anything with the command that enables them.
|
|
65
|
+
|
|
66
|
+
- **Size**: whole GiB, such as `20GiB`. The development image holds about 5 GiB of files, so give boxes at least `8GiB`; a disk too small for the image fails at creation and leaves nothing behind. Disks of 10 GiB or more start fastest: smaller ones are shrunk after the copy. ext4 keeps a little of the volume for itself, so `df` shows slightly less than SIZE.
|
|
67
|
+
- **Copy-on-write**: a new own-disk box starts as a thin snapshot of the image, and forks, templates (`new --from`) and snapshots of own-disk boxes are thin snapshots of their source. They are created in seconds and share the source's blocks until they write their own.
|
|
68
|
+
- **Reservations**: each own disk, and each snapshot of one, reserves its full size in the pool, even though it shares blocks. A request that would reserve more than the pool holds fails before anything is created. See [Reservation accounting](#reservation-accounting).
|
|
69
|
+
- **Forks, templates and backups**: `fork` and `new --from` keep the source disk, shared or own, with the same size. `new --from BOX/SNAPSHOT --disk SIZE` copies a snapshot onto an own disk of a new size. `recover` puts an own-disk backup back onto an own disk of the same size, and needs the pool on the destination. Resume never changes the disk.
|
|
70
|
+
- **Inspection**: `raft list` reports `disk` as `"shared"` or the size; `raft info` shows it in `expanded_devices.root`; `raft usage` reports the filesystem's space with a `filesystemScope` of `own SIZE box disk`.
|
|
71
|
+
|
|
72
|
+
### Enabling the `raft-disks` pool
|
|
73
|
+
|
|
74
|
+
Pass `--disk-pool SIZE` to `raft host add`, or to
|
|
75
|
+
`python3 deploy/deploy.py --location LOCATION` on a host that is already
|
|
76
|
+
deployed. The deployer creates `raft-disks` beside `raft-data`: a loop-backed
|
|
77
|
+
LVM thin pool (`lvm.use_thinpool=true`) of SIZE. It allocates the pool's file on
|
|
78
|
+
the host disk in full, so every reservation is backed by real host space, and
|
|
79
|
+
it sets the thin pool to `--discards nopassdown`, so freed blocks return to the
|
|
80
|
+
pool without punching holes back into that file. A fresh host then needs 80 GiB
|
|
81
|
+
plus SIZE free; adding the pool later needs SIZE plus 20 GiB free.
|
|
82
|
+
|
|
83
|
+
Redeploying never resizes the pool: an explicit `--disk-pool` that differs from
|
|
84
|
+
the existing size is refused, and omitting it keeps the pool as it is. To change
|
|
85
|
+
the size, back up the own-disk boxes, delete them, remove the pool explicitly
|
|
86
|
+
with Incus and redeploy.
|
|
87
|
+
|
|
88
|
+
### Reservation accounting
|
|
89
|
+
|
|
90
|
+
A thin pool can promise its volumes more space than it holds, and when it fills
|
|
91
|
+
up every volume in it fails at once. Raft never lets that happen. A thin volume
|
|
92
|
+
can never hold more data than its own size, so as long as the sizes of all thin
|
|
93
|
+
volumes in the pool add up to no more than the pool's data space, the pool
|
|
94
|
+
cannot fill, however much the boxes write.
|
|
95
|
+
|
|
96
|
+
Under the host lock, before Incus adds a thin volume to `raft-disks`, Raft adds
|
|
97
|
+
up the size of every thin volume already there, as `lvs` reports it, and refuses
|
|
98
|
+
the request if the new volumes would not fit. The sum covers everything in the
|
|
99
|
+
pool: own-disk boxes, their snapshots, the cached image and anything an
|
|
100
|
+
operator created. Each operation adds:
|
|
101
|
+
|
|
102
|
+
| Operation | Adds |
|
|
103
|
+
| --- | --- |
|
|
104
|
+
| `new --disk SIZE` | The larger of SIZE and the cached image volume, plus the image volume itself the first time a box on this host needs it |
|
|
105
|
+
| `new --from BOX/SNAPSHOT` of an own-disk snapshot | The larger of the snapshot's size and `--disk` |
|
|
106
|
+
| `new --from BOX/SNAPSHOT --disk SIZE` of a shared-pool snapshot | SIZE |
|
|
107
|
+
| `fork` of an own-disk box | The box's size |
|
|
108
|
+
| `snapshot` of an own-disk box | The box's size |
|
|
109
|
+
| `recover` of an own-disk backup | The box's size once for the box and once per snapshot in the backup |
|
|
110
|
+
|
|
111
|
+
`restore` adds nothing: it swaps in a thin snapshot of the snapshot and removes
|
|
112
|
+
the old volume. Destroying a box or deleting a snapshot frees its reservation.
|
|
113
|
+
Incus caches the image in a 10 GiB volume (the pool's `volume.size` when set),
|
|
114
|
+
which stays reserved while the image is in use; plan the pool size as 10 GiB
|
|
115
|
+
plus the own disks and snapshots you want at once. The pool's data space is a
|
|
116
|
+
little below SIZE, because LVM keeps part of the file for thin pool metadata.
|
|
117
|
+
|
|
118
|
+
LVM sizes that metadata at 64 bytes per data block, above the 48 bytes per
|
|
119
|
+
block the kernel documentation recommends for a fully mapped pool, and
|
|
120
|
+
reservations keep the number of mapped blocks below the pool's block count. If
|
|
121
|
+
metadata ever filled, every own-disk box would stop and the pool would need
|
|
122
|
+
offline repair, so `raft capacity` warns when it passes 80%.
|
|
123
|
+
|
|
124
|
+
## Inspecting capacity with `raft capacity`
|
|
125
|
+
|
|
126
|
+
```sh title="CONTROLLER: View the advisory capacity estimate"
|
|
127
|
+
raft capacity
|
|
128
|
+
|
|
129
|
+
# Output JSON for programmatic inspection
|
|
130
|
+
raft capacity --location lab --json
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
```text
|
|
134
|
+
Advisory estimate only. Raft enforces no box count or size budget; create the boxes your task needs.
|
|
135
|
+
lab: 8 CPUs, 31.3 GiB RAM (27.9 GiB available)
|
|
136
|
+
Boxes: 3 saved, 2 active
|
|
137
|
+
Allocated to active boxes: 3 CPUs, 6.0 GiB; host reserve: 1 CPU, 3.1 GiB; headroom: 4 CPUs, 22.2 GiB
|
|
138
|
+
Disk free: 41.5 of 60.0 GiB shared pool, 48.2 GiB host
|
|
139
|
+
Own box disks: 19.9 of 39.9 GiB unreserved
|
|
140
|
+
CPU RAM GiB Fit on empty host Fit now
|
|
141
|
+
1 2 7 4
|
|
142
|
+
2 4 3 2
|
|
143
|
+
4 8 1 1
|
|
144
|
+
Counts assume every box uses its full CPU and RAM at once. Boxes share host CPUs, so more can run when they are not all busy. Any whole CPU count and GiB size works.
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
The table lists example shapes that double from the 1 CPU / 2 GiB default at
|
|
148
|
+
2 GiB per CPU, up to the largest that fits the host. For any other size, divide
|
|
149
|
+
`headroom` by the box's CPU and RAM.
|
|
150
|
+
|
|
151
|
+
### JSON fields
|
|
152
|
+
|
|
153
|
+
| Field | Meaning |
|
|
154
|
+
| --- | --- |
|
|
155
|
+
| `notice` | The advisory sentence above |
|
|
156
|
+
| `host` | CPUs, total and available RAM, free host disk, `sharedPool` space, and `boxDiskPool` |
|
|
157
|
+
| `host.boxDiskPool` | `null` when own box disks are not enabled. Otherwise `totalBytes` (thin pool data space), `reservedBytes` (sizes of all its thin volumes), `usedBytes` (data written) and `metadataUsedPercent`; the last two are `null` while no own-disk volume is active. New own disks fit in `totalBytes - reservedBytes` |
|
|
158
|
+
| `boxes`, `activeBoxes` | Saved boxes, and those not `Stopped` |
|
|
159
|
+
| `allocated` | CPU and RAM limits of active boxes |
|
|
160
|
+
| `reserve` | Host share left out of the estimates |
|
|
161
|
+
| `headroom` | CPU and RAM left for more active boxes after the reserve |
|
|
162
|
+
| `estimates` | Example shapes with `fitEmptyHost` and `fitNow` counts |
|
|
163
|
+
| `warnings` | Low RAM or disk, overcommitted boxes, or thin pool metadata above 80% |
|
|
164
|
+
|
|
165
|
+
### Calculations
|
|
166
|
+
|
|
167
|
+
| Value | Calculation |
|
|
168
|
+
| --- | --- |
|
|
169
|
+
| CPU reserve | 1 CPU for the host OS |
|
|
170
|
+
| Memory reserve | The larger of 2 GiB or 10% of host RAM |
|
|
171
|
+
| Fit on empty host | `min(floor((CPUs - 1) / box CPUs), floor((RAM - reserve) / box RAM))` |
|
|
172
|
+
| Fit now | The same with `headroom`, which also subtracts active allocations and keeps `MemAvailable - reserve` |
|
|
173
|
+
|
|
174
|
+
All non-`Stopped` states, such as `Running` or `Frozen`, count as active.
|
|
175
|
+
Stopped boxes use no CPU or RAM and keep their disk. Reserves are planning
|
|
176
|
+
margins, not cgroup locks.
|
|
177
|
+
|
|
178
|
+
### Illustrative hosts
|
|
179
|
+
|
|
180
|
+
Fit on empty host for 1 CPU / 2 GiB boxes, using exact usable RAM values:
|
|
181
|
+
|
|
182
|
+
| Host CPUs | Host RAM | Fit on empty host |
|
|
183
|
+
| ---: | ---: | ---: |
|
|
184
|
+
| 2 | 8 GiB | 1 |
|
|
185
|
+
| 4 | 8 GiB | 3 |
|
|
186
|
+
| 4 | 16 GiB | 3 |
|
|
187
|
+
| 8 | 32 GiB | 7 |
|
|
188
|
+
|
|
189
|
+
Operating systems report less usable RAM than the hardware size, so near
|
|
190
|
+
boundaries the estimate can be one lower. Available RAM and host processes can
|
|
191
|
+
lower it further. These are point-in-time estimates, not reservations.
|
|
192
|
+
|
|
193
|
+
## Disk warnings
|
|
194
|
+
|
|
195
|
+
`raft capacity` warns when free host disk or the shared pool falls below 5 GiB,
|
|
196
|
+
and when `raft-disks` thin pool metadata passes 80%. Warnings do not block
|
|
197
|
+
anything. `raft doctor` reports the shared pool and, when own box disks are
|
|
198
|
+
enabled, `raft-disks`; a host without `raft-disks` is healthy.
|