bootproof 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- bootproof-0.2.0/LICENSE +21 -0
- bootproof-0.2.0/PKG-INFO +342 -0
- bootproof-0.2.0/README.md +324 -0
- bootproof-0.2.0/bootproof/__init__.py +15 -0
- bootproof-0.2.0/bootproof/__main__.py +8 -0
- bootproof-0.2.0/bootproof/capture.py +120 -0
- bootproof-0.2.0/bootproof/checklist.py +263 -0
- bootproof-0.2.0/bootproof/cli.py +322 -0
- bootproof-0.2.0/bootproof/errors.py +38 -0
- bootproof-0.2.0/bootproof/preflight.py +185 -0
- bootproof-0.2.0/bootproof/probes.py +399 -0
- bootproof-0.2.0/bootproof/report.py +79 -0
- bootproof-0.2.0/bootproof/runners.py +275 -0
- bootproof-0.2.0/bootproof/snapshots.py +129 -0
- bootproof-0.2.0/bootproof/verdict.py +108 -0
- bootproof-0.2.0/bootproof.egg-info/PKG-INFO +342 -0
- bootproof-0.2.0/bootproof.egg-info/SOURCES.txt +31 -0
- bootproof-0.2.0/bootproof.egg-info/dependency_links.txt +1 -0
- bootproof-0.2.0/bootproof.egg-info/entry_points.txt +2 -0
- bootproof-0.2.0/bootproof.egg-info/top_level.txt +1 -0
- bootproof-0.2.0/pyproject.toml +28 -0
- bootproof-0.2.0/setup.cfg +4 -0
- bootproof-0.2.0/tests/test_capture.py +109 -0
- bootproof-0.2.0/tests/test_checklist.py +100 -0
- bootproof-0.2.0/tests/test_cli.py +150 -0
- bootproof-0.2.0/tests/test_e2e.py +66 -0
- bootproof-0.2.0/tests/test_enabled.py +44 -0
- bootproof-0.2.0/tests/test_guest.py +276 -0
- bootproof-0.2.0/tests/test_preflight.py +125 -0
- bootproof-0.2.0/tests/test_probes.py +117 -0
- bootproof-0.2.0/tests/test_runners.py +106 -0
- bootproof-0.2.0/tests/test_snapshots.py +75 -0
- bootproof-0.2.0/tests/test_verdict.py +111 -0
bootproof-0.2.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Calas Automations Inc.
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
bootproof-0.2.0/PKG-INFO
ADDED
|
@@ -0,0 +1,342 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: bootproof
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Pre-reboot boot verification: boot a snapshot in an isolated VM, run the service-up checklist, and print GO or NO-GO before you reboot the real box
|
|
5
|
+
Author: Calas Automations Inc.
|
|
6
|
+
License: MIT
|
|
7
|
+
Keywords: reboot,boot,verification,qemu,kvm,systemd,ci
|
|
8
|
+
Classifier: Development Status :: 4 - Beta
|
|
9
|
+
Classifier: Environment :: Console
|
|
10
|
+
Classifier: Intended Audience :: System Administrators
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Topic :: System :: Systems Administration
|
|
14
|
+
Requires-Python: >=3.9
|
|
15
|
+
Description-Content-Type: text/markdown
|
|
16
|
+
License-File: LICENSE
|
|
17
|
+
Dynamic: license-file
|
|
18
|
+
|
|
19
|
+
# bootproof — pre-reboot boot verification
|
|
20
|
+
|
|
21
|
+
Every sysadmin fears rebooting a long-running server. Services that
|
|
22
|
+
"always worked" fail after a kernel update, a config edit, or a cert
|
|
23
|
+
rotation — and you only find out when prod is down.
|
|
24
|
+
|
|
25
|
+
bootproof snapshots the machine, boots the snapshot in an isolated
|
|
26
|
+
VM, runs a service-up checklist (ports listening, systemd units
|
|
27
|
+
active, health endpoints green), and gives a go/no-go **before**
|
|
28
|
+
you reboot the real box. One command: `bootproof check`.
|
|
29
|
+
|
|
30
|
+
The verdict is the whole UX: a single GO or NO-GO, with every
|
|
31
|
+
failing check named exactly — which port, which unit, which
|
|
32
|
+
endpoint, and what was observed.
|
|
33
|
+
|
|
34
|
+
**Status:** bootproof has not yet been run against a real KVM guest.
|
|
35
|
+
The `--boot qemu` path is unit-tested for command shape and failure
|
|
36
|
+
modes only; see "What was live-tested here vs. simulated" below.
|
|
37
|
+
|
|
38
|
+
## Install (local)
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
pip install .
|
|
42
|
+
# or, for development:
|
|
43
|
+
pip install -e .
|
|
44
|
+
# or without installing at all:
|
|
45
|
+
PYTHONPATH=. python -m bootproof check ...
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Requires Python 3.9+. No third-party dependencies, no keys, no
|
|
49
|
+
accounts, no telemetry, no phone-home. Linux-only in v1.
|
|
50
|
+
|
|
51
|
+
## Usage
|
|
52
|
+
|
|
53
|
+
```sh
|
|
54
|
+
# Write example files (checklist, snapshots, two fixtures)
|
|
55
|
+
bootproof init --output bootproof-examples
|
|
56
|
+
|
|
57
|
+
# List available snapshots
|
|
58
|
+
bootproof snapshots --snapshots snapshots.json
|
|
59
|
+
|
|
60
|
+
# Draft a checklist from this machine (review it before use)
|
|
61
|
+
bootproof capture # writes bootproof-checklist.yaml
|
|
62
|
+
|
|
63
|
+
# Static host boot checks, no VM (read-only)
|
|
64
|
+
bootproof preflight
|
|
65
|
+
|
|
66
|
+
# Validate a checklist without booting anything
|
|
67
|
+
bootproof validate --checklist bootproof-checklist.yaml
|
|
68
|
+
|
|
69
|
+
# The verdict: boot a snapshot, run the checklist
|
|
70
|
+
bootproof check --checklist bootproof-checklist.yaml \
|
|
71
|
+
--snapshots snapshots.json --snapshot snap-2026-10-27 \
|
|
72
|
+
--fixture fixture-go.json
|
|
73
|
+
|
|
74
|
+
# Real QEMU boot, with guest unit checks over SSH
|
|
75
|
+
bootproof check --checklist bootproof-checklist.yaml \
|
|
76
|
+
--snapshot-dir /var/lib/libvirt/images --boot qemu \
|
|
77
|
+
--ssh-user root --ssh-key ~/.ssh/id_ed25519
|
|
78
|
+
|
|
79
|
+
# Machine-readable output on every command
|
|
80
|
+
bootproof check ... --json
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Drafting a checklist: `bootproof capture`
|
|
84
|
+
|
|
85
|
+
`bootproof capture` writes a DRAFT `bootproof-checklist.yaml` from
|
|
86
|
+
this machine. It never runs a check.
|
|
87
|
+
|
|
88
|
+
- Ports: listening TCP sockets from `/proc/net/tcp` and
|
|
89
|
+
`/proc/net/tcp6`. Ports bound to `0.0.0.0`, `::` or loopback are
|
|
90
|
+
written with host `127.0.0.1`; a port bound only to one specific
|
|
91
|
+
address is written with that address.
|
|
92
|
+
- Units: `systemctl list-units --type=service --state=running`,
|
|
93
|
+
kept only if `systemctl is-enabled` says the unit starts at boot.
|
|
94
|
+
Running-but-disabled units are listed as comments, not checks.
|
|
95
|
+
Without `systemctl`, only ports are captured and the file says so.
|
|
96
|
+
- The first line reads `DRAFT captured from <host> at <UTC>; review
|
|
97
|
+
before use.` An existing file is not overwritten without
|
|
98
|
+
`--force` (exit 2).
|
|
99
|
+
|
|
100
|
+
### Static host checks: `bootproof preflight`
|
|
101
|
+
|
|
102
|
+
`bootproof preflight` reads the host's files and boots nothing. Its
|
|
103
|
+
report says `boot: preflight (static host checks — not a boot)`.
|
|
104
|
+
Same verdict rules and exit codes as `check`.
|
|
105
|
+
|
|
106
|
+
- **fstab:** every `UUID=`, `LABEL=` and `PARTUUID=` entry must exist
|
|
107
|
+
under `/dev/disk/by-*`, unless its options include `nofail` or
|
|
108
|
+
`noauto`. An unresolved entry is FAIL: a real boot would stop in
|
|
109
|
+
emergency mode.
|
|
110
|
+
- **initramfs:** every `/boot/vmlinuz-<v>` needs an `initrd.img-<v>`,
|
|
111
|
+
`initramfs-<v>.img` or `initrd-<v>`. A missing one is FAIL.
|
|
112
|
+
- **/boot space:** under 50 MB free is WARN. A warning does not
|
|
113
|
+
block GO.
|
|
114
|
+
- **dkms:** if `dkms` is installed, `dkms status` must show every
|
|
115
|
+
module `installed` for the newest `/boot` kernel (by version
|
|
116
|
+
sort). Anything else (for example `built`) is FAIL, naming the
|
|
117
|
+
module. Without `dkms` the check is reported as skipped, not passed.
|
|
118
|
+
|
|
119
|
+
`--root DIR` inspects another filesystem root instead of `/`.
|
|
120
|
+
|
|
121
|
+
### Checklist file
|
|
122
|
+
|
|
123
|
+
JSON, or a small YAML subset (no PyYAML needed — stdlib only).
|
|
124
|
+
Three check types:
|
|
125
|
+
|
|
126
|
+
```yaml
|
|
127
|
+
checks:
|
|
128
|
+
- type: port
|
|
129
|
+
port: 443
|
|
130
|
+
- type: port
|
|
131
|
+
port: 5432
|
|
132
|
+
- type: unit
|
|
133
|
+
name: nginx.service
|
|
134
|
+
- type: http
|
|
135
|
+
url: http://127.0.0.1:8080/health
|
|
136
|
+
expect_status: 200
|
|
137
|
+
body_contains: ok
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Defaults: port host `127.0.0.1`, http `expect_status` 200,
|
|
141
|
+
per-check timeout 5s. An empty checklist is an error, not a GO —
|
|
142
|
+
a verdict over zero checks would verify nothing (enforced in the
|
|
143
|
+
verdict engine itself, not just the CLI loader, so library use
|
|
144
|
+
cannot bypass it).
|
|
145
|
+
|
|
146
|
+
### QEMU mode: guest probing, not host probing (v0.2.0)
|
|
147
|
+
|
|
148
|
+
In `--boot qemu` mode, checklist checks are probed against the
|
|
149
|
+
**guest**, never the host:
|
|
150
|
+
|
|
151
|
+
- **Port forwarding map:** `QemuRunner` allocates a host port for
|
|
152
|
+
every loopback port/HTTP check in the checklist and forwards it
|
|
153
|
+
into the guest (`hostfwd`). Observed text names the *guest*
|
|
154
|
+
port, never the host port.
|
|
155
|
+
- **Unit checks over SSH:** `--ssh-user` + `--ssh-key` run
|
|
156
|
+
`systemctl is-active` inside the guest over the forwarded SSH
|
|
157
|
+
port (BatchMode, no password prompts, missing `ssh` binary is a
|
|
158
|
+
probe error). Without credentials, a unit check in qemu mode is
|
|
159
|
+
an ERROR — host units are never substituted for guest units.
|
|
160
|
+
- **Non-loopback and HTTPS checks are errors in qemu mode:** a
|
|
161
|
+
guest's LAN IP is not reachable from the host without bridging,
|
|
162
|
+
and TLS over a forwarded port cannot present the guest's
|
|
163
|
+
certificate. Use loopback `http://` URLs.
|
|
164
|
+
- **Readiness:** with SSH credentials, readiness is an actual SSH
|
|
165
|
+
command success (not just a TCP connect); without them, it is a
|
|
166
|
+
TCP connect to the forwarded SSH port.
|
|
167
|
+
- **Disk format** is detected from the snapshot suffix (`.img` /
|
|
168
|
+
`.raw` → raw, `.vmdk` → vmdk, else qcow2); **`-machine
|
|
169
|
+
accel=kvm`** is added only when `/dev/kvm` exists (otherwise
|
|
170
|
+
QEMU falls back to TCG — slow, but functional).
|
|
171
|
+
- **YAML `#` comments** are stripped only when the `#` starts the
|
|
172
|
+
line, follows whitespace, and is not inside quotes — URL
|
|
173
|
+
fragments (`page#frag`) and `body_contains` values containing
|
|
174
|
+
`#` are data, not comments.
|
|
175
|
+
- **Per-check latency** is measured and reported (slowest 3 checks
|
|
176
|
+
in the text report, per-check `seconds` in JSON), and the report
|
|
177
|
+
names the **disk path** that was booted.
|
|
178
|
+
|
|
179
|
+
### Snapshot sources
|
|
180
|
+
|
|
181
|
+
- `--snapshots FILE`: JSON file, `{"snapshots": [{"id", "name",
|
|
182
|
+
"source", "path"?}]}`. The fixture source used in this sandbox.
|
|
183
|
+
- `--snapshot-dir DIR`: scans for `*.qcow2` / `*.img` / `*.raw`
|
|
184
|
+
disk images; each entry carries its path, which is what a QEMU
|
|
185
|
+
boot needs.
|
|
186
|
+
|
|
187
|
+
`--snapshot` may be omitted when the source holds exactly one
|
|
188
|
+
snapshot. A missing id, or several snapshots with no id given, is
|
|
189
|
+
an error — never a silent default.
|
|
190
|
+
|
|
191
|
+
### Exit codes (CI-usable)
|
|
192
|
+
|
|
193
|
+
| Code | Meaning |
|
|
194
|
+
|------|---------|
|
|
195
|
+
| 0 | GO — every check passed |
|
|
196
|
+
| 1 | NO-GO — one or more checks observed down |
|
|
197
|
+
| 2 | Error — missing snapshot, bad checklist, boot timeout, or any check that could not be verified |
|
|
198
|
+
|
|
199
|
+
A GO requires every check to pass. A check that cannot be
|
|
200
|
+
determined (probe error, target not booted) is an error result,
|
|
201
|
+
never a pass — bootproof never prints GO on unverified checks.
|
|
202
|
+
|
|
203
|
+
## Example GO output
|
|
204
|
+
|
|
205
|
+
```
|
|
206
|
+
$ bootproof check --checklist examples/bootproof-checklist.yaml \
|
|
207
|
+
--snapshots examples/snapshots.json \
|
|
208
|
+
--fixture examples/fixture-go.json
|
|
209
|
+
bootproof verdict: GO
|
|
210
|
+
snapshot: snap-2026-10-27
|
|
211
|
+
boot: simulated
|
|
212
|
+
checks: 4 passed, 0 failed, 0 errored, 4 total
|
|
213
|
+
GO — every service-up check passed. Safe to reboot.
|
|
214
|
+
PASS: port 443 listening on 127.0.0.1 — port 443 on 127.0.0.1 listening
|
|
215
|
+
PASS: port 5432 listening on 127.0.0.1 — port 5432 on 127.0.0.1 listening
|
|
216
|
+
PASS: unit nginx.service active — unit nginx.service active
|
|
217
|
+
PASS: endpoint http://127.0.0.1:8080/health returns 200 containing 'ok' — http://127.0.0.1:8080/health HTTP 200
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
## Example NO-GO output (failing check named)
|
|
221
|
+
|
|
222
|
+
```
|
|
223
|
+
$ bootproof check --checklist examples/bootproof-checklist.yaml \
|
|
224
|
+
--snapshots examples/snapshots.json \
|
|
225
|
+
--fixture examples/fixture-nogo.json
|
|
226
|
+
bootproof verdict: NO-GO
|
|
227
|
+
snapshot: snap-2026-10-27
|
|
228
|
+
boot: simulated
|
|
229
|
+
checks: 3 passed, 1 failed, 0 errored, 4 total
|
|
230
|
+
NO-GO — the following checks failed:
|
|
231
|
+
FAIL: port 5432 listening on 127.0.0.1 — port 5432 on 127.0.0.1 closed
|
|
232
|
+
PASS: port 443 listening on 127.0.0.1 — port 443 on 127.0.0.1 listening
|
|
233
|
+
PASS: unit nginx.service active — unit nginx.service active
|
|
234
|
+
PASS: endpoint http://127.0.0.1:8080/health returns 200 containing 'ok' — http://127.0.0.1:8080/health HTTP 200
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
Exit code 1. `--json` gives the same verdict as JSON with a
|
|
238
|
+
`boot_mode` field, a per-check `status`, and the `observed` value
|
|
239
|
+
for each check.
|
|
240
|
+
|
|
241
|
+
## Error honesty
|
|
242
|
+
|
|
243
|
+
```
|
|
244
|
+
$ bootproof check ... --snapshot nope ...
|
|
245
|
+
error: snapshot not found: nope (available: snap-2026-10-27)
|
|
246
|
+
# exit 2
|
|
247
|
+
|
|
248
|
+
$ bootproof check ... --boot qemu # on a host without QEMU
|
|
249
|
+
error: QEMU binary not found: qemu-system-x86_64 — install qemu-system and enable KVM (/dev/kvm) to run a real boot
|
|
250
|
+
# exit 2 — never a silent fallback to simulation
|
|
251
|
+
```
|
|
252
|
+
|
|
253
|
+
Other specific errors: ambiguous snapshot inventory, missing
|
|
254
|
+
checklist file, unknown check type, bad port/status values, boot
|
|
255
|
+
timeout (guest not reachable within `--timeout`, default 120s).
|
|
256
|
+
|
|
257
|
+
## What was live-tested here vs. simulated — read this first
|
|
258
|
+
|
|
259
|
+
This build was developed in a sandbox with **no `/dev/kvm`, no
|
|
260
|
+
QEMU, no `virsh`, no restic/borg**. That constraint shaped what
|
|
261
|
+
could be proven live, and bootproof labels it everywhere instead
|
|
262
|
+
of papering over it:
|
|
263
|
+
|
|
264
|
+
**Live-tested in this sandbox (real code, real execution):**
|
|
265
|
+
- The full verdict engine: GO / NO-GO / ERROR, mixed results,
|
|
266
|
+
error-beats-fail, exit codes 0/1/2.
|
|
267
|
+
- Checklist parsing and validation: JSON + YAML subset, all three
|
|
268
|
+
check types, pass and fail cases per type, empty/unknown/malformed
|
|
269
|
+
checklists rejected.
|
|
270
|
+
- Real local probes: TCP port checks against a live socket server
|
|
271
|
+
and HTTP checks against a live local HTTP server (pass, wrong
|
|
272
|
+
status, unreachable — all observed, not mocked).
|
|
273
|
+
- Snapshot inventory: fixture JSON source and directory scan of
|
|
274
|
+
real `*.qcow2`/`*.img` files.
|
|
275
|
+
- Error honesty: missing snapshot, ambiguous snapshot, missing
|
|
276
|
+
source, bad checklist, boot timeout (via fixture boot delay),
|
|
277
|
+
missing QEMU binary.
|
|
278
|
+
- End-to-end CLI runs through `python -m bootproof` and through an
|
|
279
|
+
installed `bootproof` entry point in a clean venv.
|
|
280
|
+
- `capture` against fixture `/proc/net/tcp*` text and a fake
|
|
281
|
+
`systemctl`; never run on a real host here.
|
|
282
|
+
- `preflight` against fixture directory trees (fstab, `/dev/disk`,
|
|
283
|
+
`/boot`) and a fake `dkms`; never run on a real host here.
|
|
284
|
+
|
|
285
|
+
**Fixture / simulated (labelled `boot: simulated` in every report):**
|
|
286
|
+
- The booted guest itself. `--boot simulated` (the default) answers
|
|
287
|
+
probes from a fixture JSON file describing guest state. An
|
|
288
|
+
unlisted port/unit/endpoint fails loudly — fixture silence is
|
|
289
|
+
absence, never success.
|
|
290
|
+
- The QEMU boot path could not execute here: `QemuRunner` and
|
|
291
|
+
`build_qemu_command` are real code (command construction with
|
|
292
|
+
`snapshot=on` so the guest never writes the source image, user
|
|
293
|
+
networking with SSH forwarding, readiness loop, timeout kill,
|
|
294
|
+
teardown), unit-tested for command shape and failure modes, but
|
|
295
|
+
no guest was actually booted in this sandbox. **v0.2.0 fixed the
|
|
296
|
+
two real-path correctness holes the panel found in this code**
|
|
297
|
+
(host probing instead of guest probing for ports and units —
|
|
298
|
+
see "QEMU mode" above); the fix is unit-tested against real
|
|
299
|
+
sockets and fake `ssh` binaries, but a real guest boot still
|
|
300
|
+
awaits a host with KVM + a guest image.
|
|
301
|
+
|
|
302
|
+
**What a real host needs for the full path:**
|
|
303
|
+
- Linux host with KVM (`/dev/kvm`) and `qemu-system-x86_64`
|
|
304
|
+
installed.
|
|
305
|
+
- A snapshot with a disk path (directory snapshot source, or a
|
|
306
|
+
snapshots file entry carrying `path`).
|
|
307
|
+
- `bootproof check --checklist ... --snapshot-dir <dir> --boot qemu`
|
|
308
|
+
(loopback checklist ports/URLs are forwarded into the guest
|
|
309
|
+
automatically; add `--ssh-user` + `--ssh-key` for guest unit
|
|
310
|
+
checks and SSH-based readiness).
|
|
311
|
+
- On such a host the report reads `boot: qemu`, not `boot:
|
|
312
|
+
simulated`. Until then, treat every verdict here as a
|
|
313
|
+
checklist-engine verdict over simulated guest state — which is
|
|
314
|
+
exactly what its `boot:` line says.
|
|
315
|
+
|
|
316
|
+
`--local` is a third mode: it probes the current machine directly
|
|
317
|
+
(no snapshot, no boot) as a checklist dry run, labelled
|
|
318
|
+
`boot: local`. It is not a boot verification.
|
|
319
|
+
|
|
320
|
+
In `--local` mode a unit check also runs `systemctl is-enabled`.
|
|
321
|
+
`enabled`, `static`, `indirect`, `generated`, `alias` and
|
|
322
|
+
`enabled-runtime` pass. `disabled`, `masked` and `linked` fail with
|
|
323
|
+
"active now but <state> — will not start at boot unless pulled in by
|
|
324
|
+
another unit". Any other answer is an error, never a pass. Set
|
|
325
|
+
`require_enabled: false` on a unit check to skip this. QEMU mode
|
|
326
|
+
does not check it: the guest boot itself is the proof.
|
|
327
|
+
|
|
328
|
+
## What v1 does NOT do (deliberately)
|
|
329
|
+
|
|
330
|
+
- Fleet dashboards, scheduled checks, team-tier features — the
|
|
331
|
+
C$19/mo tier is a later product, not v1.
|
|
332
|
+
- Windows support — Windows boot semantics are a second product.
|
|
333
|
+
- VMware estates, restic/borg/Proxmox snapshot sources — the
|
|
334
|
+
`SnapshotSource` interface is defined for them, but they are not
|
|
335
|
+
built in v1.
|
|
336
|
+
- Any paid gating.
|
|
337
|
+
|
|
338
|
+
## Tests
|
|
339
|
+
|
|
340
|
+
```sh
|
|
341
|
+
python -m pytest -q
|
|
342
|
+
```
|
|
@@ -0,0 +1,324 @@
|
|
|
1
|
+
# bootproof — pre-reboot boot verification
|
|
2
|
+
|
|
3
|
+
Every sysadmin fears rebooting a long-running server. Services that
|
|
4
|
+
"always worked" fail after a kernel update, a config edit, or a cert
|
|
5
|
+
rotation — and you only find out when prod is down.
|
|
6
|
+
|
|
7
|
+
bootproof snapshots the machine, boots the snapshot in an isolated
|
|
8
|
+
VM, runs a service-up checklist (ports listening, systemd units
|
|
9
|
+
active, health endpoints green), and gives a go/no-go **before**
|
|
10
|
+
you reboot the real box. One command: `bootproof check`.
|
|
11
|
+
|
|
12
|
+
The verdict is the whole UX: a single GO or NO-GO, with every
|
|
13
|
+
failing check named exactly — which port, which unit, which
|
|
14
|
+
endpoint, and what was observed.
|
|
15
|
+
|
|
16
|
+
**Status:** bootproof has not yet been run against a real KVM guest.
|
|
17
|
+
The `--boot qemu` path is unit-tested for command shape and failure
|
|
18
|
+
modes only; see "What was live-tested here vs. simulated" below.
|
|
19
|
+
|
|
20
|
+
## Install (local)
|
|
21
|
+
|
|
22
|
+
```sh
|
|
23
|
+
pip install .
|
|
24
|
+
# or, for development:
|
|
25
|
+
pip install -e .
|
|
26
|
+
# or without installing at all:
|
|
27
|
+
PYTHONPATH=. python -m bootproof check ...
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Requires Python 3.9+. No third-party dependencies, no keys, no
|
|
31
|
+
accounts, no telemetry, no phone-home. Linux-only in v1.
|
|
32
|
+
|
|
33
|
+
## Usage
|
|
34
|
+
|
|
35
|
+
```sh
|
|
36
|
+
# Write example files (checklist, snapshots, two fixtures)
|
|
37
|
+
bootproof init --output bootproof-examples
|
|
38
|
+
|
|
39
|
+
# List available snapshots
|
|
40
|
+
bootproof snapshots --snapshots snapshots.json
|
|
41
|
+
|
|
42
|
+
# Draft a checklist from this machine (review it before use)
|
|
43
|
+
bootproof capture # writes bootproof-checklist.yaml
|
|
44
|
+
|
|
45
|
+
# Static host boot checks, no VM (read-only)
|
|
46
|
+
bootproof preflight
|
|
47
|
+
|
|
48
|
+
# Validate a checklist without booting anything
|
|
49
|
+
bootproof validate --checklist bootproof-checklist.yaml
|
|
50
|
+
|
|
51
|
+
# The verdict: boot a snapshot, run the checklist
|
|
52
|
+
bootproof check --checklist bootproof-checklist.yaml \
|
|
53
|
+
--snapshots snapshots.json --snapshot snap-2026-10-27 \
|
|
54
|
+
--fixture fixture-go.json
|
|
55
|
+
|
|
56
|
+
# Real QEMU boot, with guest unit checks over SSH
|
|
57
|
+
bootproof check --checklist bootproof-checklist.yaml \
|
|
58
|
+
--snapshot-dir /var/lib/libvirt/images --boot qemu \
|
|
59
|
+
--ssh-user root --ssh-key ~/.ssh/id_ed25519
|
|
60
|
+
|
|
61
|
+
# Machine-readable output on every command
|
|
62
|
+
bootproof check ... --json
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
### Drafting a checklist: `bootproof capture`
|
|
66
|
+
|
|
67
|
+
`bootproof capture` writes a DRAFT `bootproof-checklist.yaml` from
|
|
68
|
+
this machine. It never runs a check.
|
|
69
|
+
|
|
70
|
+
- Ports: listening TCP sockets from `/proc/net/tcp` and
|
|
71
|
+
`/proc/net/tcp6`. Ports bound to `0.0.0.0`, `::` or loopback are
|
|
72
|
+
written with host `127.0.0.1`; a port bound only to one specific
|
|
73
|
+
address is written with that address.
|
|
74
|
+
- Units: `systemctl list-units --type=service --state=running`,
|
|
75
|
+
kept only if `systemctl is-enabled` says the unit starts at boot.
|
|
76
|
+
Running-but-disabled units are listed as comments, not checks.
|
|
77
|
+
Without `systemctl`, only ports are captured and the file says so.
|
|
78
|
+
- The first line reads `DRAFT captured from <host> at <UTC>; review
|
|
79
|
+
before use.` An existing file is not overwritten without
|
|
80
|
+
`--force` (exit 2).
|
|
81
|
+
|
|
82
|
+
### Static host checks: `bootproof preflight`
|
|
83
|
+
|
|
84
|
+
`bootproof preflight` reads the host's files and boots nothing. Its
|
|
85
|
+
report says `boot: preflight (static host checks — not a boot)`.
|
|
86
|
+
Same verdict rules and exit codes as `check`.
|
|
87
|
+
|
|
88
|
+
- **fstab:** every `UUID=`, `LABEL=` and `PARTUUID=` entry must exist
|
|
89
|
+
under `/dev/disk/by-*`, unless its options include `nofail` or
|
|
90
|
+
`noauto`. An unresolved entry is FAIL: a real boot would stop in
|
|
91
|
+
emergency mode.
|
|
92
|
+
- **initramfs:** every `/boot/vmlinuz-<v>` needs an `initrd.img-<v>`,
|
|
93
|
+
`initramfs-<v>.img` or `initrd-<v>`. A missing one is FAIL.
|
|
94
|
+
- **/boot space:** under 50 MB free is WARN. A warning does not
|
|
95
|
+
block GO.
|
|
96
|
+
- **dkms:** if `dkms` is installed, `dkms status` must show every
|
|
97
|
+
module `installed` for the newest `/boot` kernel (by version
|
|
98
|
+
sort). Anything else (for example `built`) is FAIL, naming the
|
|
99
|
+
module. Without `dkms` the check is reported as skipped, not passed.
|
|
100
|
+
|
|
101
|
+
`--root DIR` inspects another filesystem root instead of `/`.
|
|
102
|
+
|
|
103
|
+
### Checklist file
|
|
104
|
+
|
|
105
|
+
JSON, or a small YAML subset (no PyYAML needed — stdlib only).
|
|
106
|
+
Three check types:
|
|
107
|
+
|
|
108
|
+
```yaml
|
|
109
|
+
checks:
|
|
110
|
+
- type: port
|
|
111
|
+
port: 443
|
|
112
|
+
- type: port
|
|
113
|
+
port: 5432
|
|
114
|
+
- type: unit
|
|
115
|
+
name: nginx.service
|
|
116
|
+
- type: http
|
|
117
|
+
url: http://127.0.0.1:8080/health
|
|
118
|
+
expect_status: 200
|
|
119
|
+
body_contains: ok
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Defaults: port host `127.0.0.1`, http `expect_status` 200,
|
|
123
|
+
per-check timeout 5s. An empty checklist is an error, not a GO —
|
|
124
|
+
a verdict over zero checks would verify nothing (enforced in the
|
|
125
|
+
verdict engine itself, not just the CLI loader, so library use
|
|
126
|
+
cannot bypass it).
|
|
127
|
+
|
|
128
|
+
### QEMU mode: guest probing, not host probing (v0.2.0)
|
|
129
|
+
|
|
130
|
+
In `--boot qemu` mode, checklist checks are probed against the
|
|
131
|
+
**guest**, never the host:
|
|
132
|
+
|
|
133
|
+
- **Port forwarding map:** `QemuRunner` allocates a host port for
|
|
134
|
+
every loopback port/HTTP check in the checklist and forwards it
|
|
135
|
+
into the guest (`hostfwd`). Observed text names the *guest*
|
|
136
|
+
port, never the host port.
|
|
137
|
+
- **Unit checks over SSH:** `--ssh-user` + `--ssh-key` run
|
|
138
|
+
`systemctl is-active` inside the guest over the forwarded SSH
|
|
139
|
+
port (BatchMode, no password prompts, missing `ssh` binary is a
|
|
140
|
+
probe error). Without credentials, a unit check in qemu mode is
|
|
141
|
+
an ERROR — host units are never substituted for guest units.
|
|
142
|
+
- **Non-loopback and HTTPS checks are errors in qemu mode:** a
|
|
143
|
+
guest's LAN IP is not reachable from the host without bridging,
|
|
144
|
+
and TLS over a forwarded port cannot present the guest's
|
|
145
|
+
certificate. Use loopback `http://` URLs.
|
|
146
|
+
- **Readiness:** with SSH credentials, readiness is an actual SSH
|
|
147
|
+
command success (not just a TCP connect); without them, it is a
|
|
148
|
+
TCP connect to the forwarded SSH port.
|
|
149
|
+
- **Disk format** is detected from the snapshot suffix (`.img` /
|
|
150
|
+
`.raw` → raw, `.vmdk` → vmdk, else qcow2); **`-machine
|
|
151
|
+
accel=kvm`** is added only when `/dev/kvm` exists (otherwise
|
|
152
|
+
QEMU falls back to TCG — slow, but functional).
|
|
153
|
+
- **YAML `#` comments** are stripped only when the `#` starts the
|
|
154
|
+
line, follows whitespace, and is not inside quotes — URL
|
|
155
|
+
fragments (`page#frag`) and `body_contains` values containing
|
|
156
|
+
`#` are data, not comments.
|
|
157
|
+
- **Per-check latency** is measured and reported (slowest 3 checks
|
|
158
|
+
in the text report, per-check `seconds` in JSON), and the report
|
|
159
|
+
names the **disk path** that was booted.
|
|
160
|
+
|
|
161
|
+
### Snapshot sources
|
|
162
|
+
|
|
163
|
+
- `--snapshots FILE`: JSON file, `{"snapshots": [{"id", "name",
|
|
164
|
+
"source", "path"?}]}`. The fixture source used in this sandbox.
|
|
165
|
+
- `--snapshot-dir DIR`: scans for `*.qcow2` / `*.img` / `*.raw`
|
|
166
|
+
disk images; each entry carries its path, which is what a QEMU
|
|
167
|
+
boot needs.
|
|
168
|
+
|
|
169
|
+
`--snapshot` may be omitted when the source holds exactly one
|
|
170
|
+
snapshot. A missing id, or several snapshots with no id given, is
|
|
171
|
+
an error — never a silent default.
|
|
172
|
+
|
|
173
|
+
### Exit codes (CI-usable)
|
|
174
|
+
|
|
175
|
+
| Code | Meaning |
|
|
176
|
+
|------|---------|
|
|
177
|
+
| 0 | GO — every check passed |
|
|
178
|
+
| 1 | NO-GO — one or more checks observed down |
|
|
179
|
+
| 2 | Error — missing snapshot, bad checklist, boot timeout, or any check that could not be verified |
|
|
180
|
+
|
|
181
|
+
A GO requires every check to pass. A check that cannot be
|
|
182
|
+
determined (probe error, target not booted) is an error result,
|
|
183
|
+
never a pass — bootproof never prints GO on unverified checks.
|
|
184
|
+
|
|
185
|
+
## Example GO output
|
|
186
|
+
|
|
187
|
+
```
|
|
188
|
+
$ bootproof check --checklist examples/bootproof-checklist.yaml \
|
|
189
|
+
--snapshots examples/snapshots.json \
|
|
190
|
+
--fixture examples/fixture-go.json
|
|
191
|
+
bootproof verdict: GO
|
|
192
|
+
snapshot: snap-2026-10-27
|
|
193
|
+
boot: simulated
|
|
194
|
+
checks: 4 passed, 0 failed, 0 errored, 4 total
|
|
195
|
+
GO — every service-up check passed. Safe to reboot.
|
|
196
|
+
PASS: port 443 listening on 127.0.0.1 — port 443 on 127.0.0.1 listening
|
|
197
|
+
PASS: port 5432 listening on 127.0.0.1 — port 5432 on 127.0.0.1 listening
|
|
198
|
+
PASS: unit nginx.service active — unit nginx.service active
|
|
199
|
+
PASS: endpoint http://127.0.0.1:8080/health returns 200 containing 'ok' — http://127.0.0.1:8080/health HTTP 200
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
## Example NO-GO output (failing check named)
|
|
203
|
+
|
|
204
|
+
```
|
|
205
|
+
$ bootproof check --checklist examples/bootproof-checklist.yaml \
|
|
206
|
+
--snapshots examples/snapshots.json \
|
|
207
|
+
--fixture examples/fixture-nogo.json
|
|
208
|
+
bootproof verdict: NO-GO
|
|
209
|
+
snapshot: snap-2026-10-27
|
|
210
|
+
boot: simulated
|
|
211
|
+
checks: 3 passed, 1 failed, 0 errored, 4 total
|
|
212
|
+
NO-GO — the following checks failed:
|
|
213
|
+
FAIL: port 5432 listening on 127.0.0.1 — port 5432 on 127.0.0.1 closed
|
|
214
|
+
PASS: port 443 listening on 127.0.0.1 — port 443 on 127.0.0.1 listening
|
|
215
|
+
PASS: unit nginx.service active — unit nginx.service active
|
|
216
|
+
PASS: endpoint http://127.0.0.1:8080/health returns 200 containing 'ok' — http://127.0.0.1:8080/health HTTP 200
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
Exit code 1. `--json` gives the same verdict as JSON with a
|
|
220
|
+
`boot_mode` field, a per-check `status`, and the `observed` value
|
|
221
|
+
for each check.
|
|
222
|
+
|
|
223
|
+
## Error honesty
|
|
224
|
+
|
|
225
|
+
```
|
|
226
|
+
$ bootproof check ... --snapshot nope ...
|
|
227
|
+
error: snapshot not found: nope (available: snap-2026-10-27)
|
|
228
|
+
# exit 2
|
|
229
|
+
|
|
230
|
+
$ bootproof check ... --boot qemu # on a host without QEMU
|
|
231
|
+
error: QEMU binary not found: qemu-system-x86_64 — install qemu-system and enable KVM (/dev/kvm) to run a real boot
|
|
232
|
+
# exit 2 — never a silent fallback to simulation
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
Other specific errors: ambiguous snapshot inventory, missing
|
|
236
|
+
checklist file, unknown check type, bad port/status values, boot
|
|
237
|
+
timeout (guest not reachable within `--timeout`, default 120s).
|
|
238
|
+
|
|
239
|
+
## What was live-tested here vs. simulated — read this first
|
|
240
|
+
|
|
241
|
+
This build was developed in a sandbox with **no `/dev/kvm`, no
|
|
242
|
+
QEMU, no `virsh`, no restic/borg**. That constraint shaped what
|
|
243
|
+
could be proven live, and bootproof labels it everywhere instead
|
|
244
|
+
of papering over it:
|
|
245
|
+
|
|
246
|
+
**Live-tested in this sandbox (real code, real execution):**
|
|
247
|
+
- The full verdict engine: GO / NO-GO / ERROR, mixed results,
|
|
248
|
+
error-beats-fail, exit codes 0/1/2.
|
|
249
|
+
- Checklist parsing and validation: JSON + YAML subset, all three
|
|
250
|
+
check types, pass and fail cases per type, empty/unknown/malformed
|
|
251
|
+
checklists rejected.
|
|
252
|
+
- Real local probes: TCP port checks against a live socket server
|
|
253
|
+
and HTTP checks against a live local HTTP server (pass, wrong
|
|
254
|
+
status, unreachable — all observed, not mocked).
|
|
255
|
+
- Snapshot inventory: fixture JSON source and directory scan of
|
|
256
|
+
real `*.qcow2`/`*.img` files.
|
|
257
|
+
- Error honesty: missing snapshot, ambiguous snapshot, missing
|
|
258
|
+
source, bad checklist, boot timeout (via fixture boot delay),
|
|
259
|
+
missing QEMU binary.
|
|
260
|
+
- End-to-end CLI runs through `python -m bootproof` and through an
|
|
261
|
+
installed `bootproof` entry point in a clean venv.
|
|
262
|
+
- `capture` against fixture `/proc/net/tcp*` text and a fake
|
|
263
|
+
`systemctl`; never run on a real host here.
|
|
264
|
+
- `preflight` against fixture directory trees (fstab, `/dev/disk`,
|
|
265
|
+
`/boot`) and a fake `dkms`; never run on a real host here.
|
|
266
|
+
|
|
267
|
+
**Fixture / simulated (labelled `boot: simulated` in every report):**
|
|
268
|
+
- The booted guest itself. `--boot simulated` (the default) answers
|
|
269
|
+
probes from a fixture JSON file describing guest state. An
|
|
270
|
+
unlisted port/unit/endpoint fails loudly — fixture silence is
|
|
271
|
+
absence, never success.
|
|
272
|
+
- The QEMU boot path could not execute here: `QemuRunner` and
|
|
273
|
+
`build_qemu_command` are real code (command construction with
|
|
274
|
+
`snapshot=on` so the guest never writes the source image, user
|
|
275
|
+
networking with SSH forwarding, readiness loop, timeout kill,
|
|
276
|
+
teardown), unit-tested for command shape and failure modes, but
|
|
277
|
+
no guest was actually booted in this sandbox. **v0.2.0 fixed the
|
|
278
|
+
two real-path correctness holes the panel found in this code**
|
|
279
|
+
(host probing instead of guest probing for ports and units —
|
|
280
|
+
see "QEMU mode" above); the fix is unit-tested against real
|
|
281
|
+
sockets and fake `ssh` binaries, but a real guest boot still
|
|
282
|
+
awaits a host with KVM + a guest image.
|
|
283
|
+
|
|
284
|
+
**What a real host needs for the full path:**
|
|
285
|
+
- Linux host with KVM (`/dev/kvm`) and `qemu-system-x86_64`
|
|
286
|
+
installed.
|
|
287
|
+
- A snapshot with a disk path (directory snapshot source, or a
|
|
288
|
+
snapshots file entry carrying `path`).
|
|
289
|
+
- `bootproof check --checklist ... --snapshot-dir <dir> --boot qemu`
|
|
290
|
+
(loopback checklist ports/URLs are forwarded into the guest
|
|
291
|
+
automatically; add `--ssh-user` + `--ssh-key` for guest unit
|
|
292
|
+
checks and SSH-based readiness).
|
|
293
|
+
- On such a host the report reads `boot: qemu`, not `boot:
|
|
294
|
+
simulated`. Until then, treat every verdict here as a
|
|
295
|
+
checklist-engine verdict over simulated guest state — which is
|
|
296
|
+
exactly what its `boot:` line says.
|
|
297
|
+
|
|
298
|
+
`--local` is a third mode: it probes the current machine directly
|
|
299
|
+
(no snapshot, no boot) as a checklist dry run, labelled
|
|
300
|
+
`boot: local`. It is not a boot verification.
|
|
301
|
+
|
|
302
|
+
In `--local` mode a unit check also runs `systemctl is-enabled`.
|
|
303
|
+
`enabled`, `static`, `indirect`, `generated`, `alias` and
|
|
304
|
+
`enabled-runtime` pass. `disabled`, `masked` and `linked` fail with
|
|
305
|
+
"active now but <state> — will not start at boot unless pulled in by
|
|
306
|
+
another unit". Any other answer is an error, never a pass. Set
|
|
307
|
+
`require_enabled: false` on a unit check to skip this. QEMU mode
|
|
308
|
+
does not check it: the guest boot itself is the proof.
|
|
309
|
+
|
|
310
|
+
## What v1 does NOT do (deliberately)
|
|
311
|
+
|
|
312
|
+
- Fleet dashboards, scheduled checks, team-tier features — the
|
|
313
|
+
C$19/mo tier is a later product, not v1.
|
|
314
|
+
- Windows support — Windows boot semantics are a second product.
|
|
315
|
+
- VMware estates, restic/borg/Proxmox snapshot sources — the
|
|
316
|
+
`SnapshotSource` interface is defined for them, but they are not
|
|
317
|
+
built in v1.
|
|
318
|
+
- Any paid gating.
|
|
319
|
+
|
|
320
|
+
## Tests
|
|
321
|
+
|
|
322
|
+
```sh
|
|
323
|
+
python -m pytest -q
|
|
324
|
+
```
|