@crvouga/dev-fleet 0.1.0 → 0.3.0

package/ansible/monitoring/README.md

@@ -1,19 +1,20 @@
 # Beszel owner and recovery
 
-Fleet's [manager guide](../../fleet/docs/ubuntu-sync.md) is the normal entry
-point: `devfleet fleet-sync`, run on the manager. Workstation owns the
-pinned Compose hub, native agent units and local Ansible tasks here. Fleet owns
-membership, registration and freshness evidence. `lock.json` pins Beszel 0.21.1
-and the native agent checksums. Never run an unpinned upstream installer
-alongside these services.
+The [README](../../README.md) and the [manager guide](../../docs/ubuntu-sync.md)
+are the normal entry points: `devfleet fleet-sync`, run on the manager. This
+directory holds the pinned Compose hub, the native agent unit and their
+playbooks. `src/monitoring/` enrolls machines and measures registration,
+connection, freshness and backups. `lock.json` pins the Beszel release, the
+agent archives and the agent executables by digest. Never run an unpinned
+upstream installer alongside these services.
 
 ## Manager and workers
 
-The fleet has exactly two roles (Fleet `definition.json`, version 3):
+The fleet has exactly two roles (`fleet.json`):
 
 - **Manager** — the human's macOS machine. It runs fleet-sync locally and hosts the
-  singleton hub (`cli/lib/monitoring/manager-hub.ts`, `convergeManagerHub`). Fleet
-  keeps it awake on AC power. Nothing is installed on it over SSH.
+  singleton hub (`src/monitoring/manager-hub.ts`, `convergeManagerHub`).
+  dev-fleet keeps it awake on AC power. Nothing is installed on it over SSH.
 - **Workers** — the stateless Linux boxes. They only run the native agent
   (`agent.yml`, a hardened systemd unit) and connect to the manager's private hub
   URL. A worker refuses hub mode (`BESZEL_NOT_HUB`: the hub runs only on the
@@ -77,13 +78,42 @@ unavailable the step warns and the hub shows its normal login page; create the
 user by hand (Settings > Users, role **Read-only**) as a fallback. Never point
 `AUTO_LOGIN` at an admin account. The fleet admin login is still needed to change anything.
 
+## The agent on a worker
+
 Native agents use outbound authenticated WebSockets, per-node tokens and
 persistent identity. They do not need inbound SSH monitoring ports, Docker socket
 access or journals.
-`SKIP_SYSTEMD_LOGS=true` deliberately excludes potentially sensitive service logs.
-System services may be visible; user T3 service information is not certified by
-that fact. Missing sensors, SMART privileges, Docker stats and user-service metrics
-remain unavailable until their specific supported permissions are reviewed.
+
+- **Unprivileged.** The unit runs as the `beszel` system account with no other
+  group, no capability, no journal and no Docker socket
+  (`features/monitoring/agent-privileges.feature`).
+  `SKIP_SYSTEMD_LOGS=true` deliberately excludes potentially sensitive service
+  logs. System services may be visible; user T3 service information is not
+  certified by that fact. Sensors, SMART, Docker stats and user-service metrics
+  stay unavailable until their specific permissions are reviewed: nothing grants
+  them by default.
+- **Repeatable.** `agent.yml` compares the installed executable with the digest
+  of the pinned build (`agentBinaries` in `lock.json`). An unchanged agent is left
+  running; one an interrupted upgrade or an edit left behind is removed,
+  extracted again from the checksummed archive and restarted once. Edited unit
+  or connection settings are restored the same way.
+- **Boots without a login.** It is a system unit (`WantedBy=multi-user.target`),
+  so it starts at boot with nobody signed in.
+- **Measured from the worker's side** (`src/monitoring/agent-evidence.ts`):
+
+  | Check                   | Summary                                                       | Meaning                                                              |
+  | ----------------------- | ------------------------------------------------------------- | -------------------------------------------------------------------- |
+  | `beszel:disk`           | `BESZEL_DISK_FULL`, `BESZEL_DISK_UNVERIFIED`                  | Less than 256 MiB free where the agent installs; nothing was changed |
+  | `beszel:service`        | `BESZEL_SERVICE_STOPPED`                                      | The unit is not enabled and active                                   |
+  | `beszel:hub-connection` | `BESZEL_AGENT_TOKEN_REJECTED`                                 | The hub answered the agent's token with 401                          |
+  |                         | `BESZEL_AGENT_KEY_REJECTED`                                   | The hub did not prove the key the agent was given                    |
+  |                         | `BESZEL_HUB_UNAVAILABLE_TO_AGENT` (warn)                      | The private route answered 502–504: the hub behind it is down        |
+  |                         | `BESZEL_AGENT_CONNECTION_PENDING`, `…_LOG_UNAVAILABLE` (skip) | No outcome yet, or the log could not be read                         |
+
+  The outcome is the newest of a few fixed texts the pinned agent logs, taken
+  from its source; nothing of the log is reported. The hub's own view
+  (registration, `BESZEL_AGENT_DISCONNECTED`, `BESZEL_METRICS_STALE`) is
+  measured separately by `fleet-status --workers`.
 
 Accounts need no manual step. On a hub's first start (empty data directory),
 `hub-darwin.yml` passes Beszel's `USER_EMAIL`/`USER_PASSWORD` once, so the hub
@@ -97,20 +127,32 @@ A legacy JSON `identity`/`password` value is still accepted. The manager's exist
 Vault credential must renew unattended. Superuser credentials are not distributed
 to nodes or needed for enrollment. Revocation produces action-required results.
 
-Configure network monitors and alert thresholds once in the stock UI; these
-settings are runtime application data, preserved in backups. Select explicit probe
-origins rather than an all-to-all mesh. Use sustained resource-pressure alerts,
-disk capacity, offline nodes and supported temperatures. Do not send notifications
-until an authorized destination exists. Service-failure alert delivery has not
-been accepted against this pinned release.
+## Settings kept only in the hub's UI
+
+Beszel keeps network monitors, alert rules and backup settings in its own data,
+set in its UI. dev-fleet does not write them: there is no reconciler and no
+endpoint beyond the stock API (`features/monitoring/stock-api.feature`). They
+are preserved by the hub's backups. Set them once, as the fleet account:
+
+- **Network monitors.** Add them per system from that system's page. Probe from
+  a few chosen origins (for example the manager to each worker) rather than an
+  all-to-all mesh, so a failed check names a path you can reason about.
+- **Alerts.** Prefer sustained thresholds over instant ones: CPU, memory and
+  disk above a level for several minutes, disk capacity, a system going offline,
+  and temperatures where the sensor is actually reported. Do not add a
+  notification destination until one you are allowed to use exists.
+- **Service alerts are not promised.** Alerting on a failed systemd service has
+  not been verified against the pinned release; the agent watches only the
+  services in `SERVICE_PATTERNS`, and none of them is relied on for alerting.
 
 ## Stock backup and restore
 
 1. Open the hub's private PocketBase administration route `/_/` and authenticate
    as its superuser. Keep that credential separate from fleet enrollment access.
 2. Open **Settings → Backups**. Create and download a backup before maintenance.
-   Set scheduled backups and retention there using existing authorized storage.
-   Do not provision a bucket or reuse unrelated credentials to bypass access gates.
+   Set a schedule, how many to keep, and S3-compatible storage there, using
+   storage you already approved. Do not provision a bucket or reuse unrelated
+   credentials to bypass access gates.
 3. Protect downloaded archives as secrets: they contain registrations, tokens,
    account data, alert configuration and history. Use the existing approved
    encrypted backup destination; a backup on the same disk is not disaster recovery.
@@ -122,6 +164,17 @@ been accepted against this pinned release.
 5. Run the manager command and require new metric timestamps after recovery.
    Old historical samples do not prove agent connectivity.
 
+Every hub converge in `fleet-sync` reads these settings back as the hub's
+superuser (`GET /api/settings`, `GET /api/backups`; `src/monitoring/hub-backup.ts`)
+and reports `beszel:backup`. It only warns, and never names the storage:
+
+| Summary                          | Meaning                                                  |
+| -------------------------------- | -------------------------------------------------------- |
+| `BESZEL_BACKUP_UNSCHEDULED`      | No backup schedule is set                                |
+| `BESZEL_BACKUP_OFFHOST_REQUIRED` | Backups stay on the manager's disk                       |
+| `BESZEL_BACKUP_STALE`            | A schedule is set but no backup is newer than 48 hours   |
+| `BESZEL_BACKUP_UNVERIFIED`       | The settings could not be read (no fleet login, refused) |
+
 ## Moving the hub to the manager
 
 The hub previously ran on worker `worker-one` (`/var/lib/personal-beszel`,
@@ -143,10 +196,18 @@ never run two copies of its identity or two tunnel connectors:
    Vault grant (OpenTofu).
 
 `tests/upstream.py` exercises real pinned binaries with disposable loopback data:
-authenticated registration, fresh metrics, restart identity preservation, no duplicate
-record, and the supported PocketBase backup/restore API. It does not contact live
-nodes, prove Tailscale routing, verify systemd reboot, or test an off-host encrypted
-backup destination. Those remain explicit release/operational gates.
+authenticated registration, fresh metrics, restart identity preservation, no
+duplicate record, the supported PocketBase backup/restore API, and the recovery
+drill above: a downloaded backup restored into a second, isolated hub after the
+first is fenced, keeping node IDs, fingerprints, history and settings. It does
+not contact live nodes, prove Tailscale routing, verify systemd reboot, or use an
+off-host storage service. Those remain operational checks:
+
+- From a second tailnet machine, `devfleet fleet-status --workers` reports
+  `beszel:registration`, `beszel:connection` and `beszel:freshness` passing for
+  every worker.
+- `beszel:backup` passes on the manager once off-host storage is configured.
+- After a worker reboots, the same checks pass again without anyone signing in.
 
 References: [agent](https://beszel.dev/guide/agent-installation),
 [systemd](https://beszel.dev/guide/systemd),

package/ansible/monitoring/agent.yml

@@ -13,6 +13,7 @@
         that:
           - ansible_facts.distribution == 'Ubuntu'
           - beszel_arch in beszel_lock.agents
+          - beszel_arch in beszel_lock.agentBinaries
           - beszel_node is match('^[a-z][a-z0-9-]{0,62}$')
           - beszel_hub_url is match('^https://[a-z0-9.-]+\.ts\.net:8443$')
     - name: Inspect existing native service ownership
@@ -46,7 +47,7 @@
       loop:
         - { path: /etc/personal-beszel, group: beszel, mode: '0750' }
         - {
-            path: '/usr/local/lib/personal-beszel/0.21.1',
+            path: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}',
             group: root,
             mode: '0755',
           }
@@ -88,12 +89,49 @@
         checksum: 'sha256:{{ beszel_lock.agents[beszel_arch] }}'
         mode: '0644'
         timeout: 60
-    - name: Extract the pinned executable and repair file drift
+    # The executable is compared with the digest of the pinned build, so an
+    # unchanged agent is left alone (no restart) and one that an interrupted
+    # upgrade or an edit left behind is replaced.
+    - name: Inspect the installed executable
+      ansible.builtin.stat:
+        path: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}/beszel-agent'
+        checksum_algorithm: sha256
+        follow: false
+      register: agent_binary
+    - name: Remove an executable that is not the pinned build
+      ansible.builtin.file:
+        path: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}/beszel-agent'
+        state: absent
+      when: >-
+        agent_binary.stat.exists and (not agent_binary.stat.isreg
+        or agent_binary.stat.checksum != beszel_lock.agentBinaries[beszel_arch])
+    - name: Extract the pinned executable when it is missing
       ansible.builtin.unarchive:
         src: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}/agent.tar.gz'
         dest: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}'
         remote_src: true
+        include: [beszel-agent]
+        creates: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}/beszel-agent'
       notify: Restart agent
+    - name: Inspect the extracted executable
+      ansible.builtin.stat:
+        path: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}/beszel-agent'
+        checksum_algorithm: sha256
+        follow: false
+      register: agent_extracted
+    - name: Require the pinned build before the service may start it
+      ansible.builtin.assert:
+        that:
+          - agent_extracted.stat.exists
+          - agent_extracted.stat.isreg
+          - agent_extracted.stat.checksum == beszel_lock.agentBinaries[beszel_arch]
+        fail_msg: The extracted Beszel agent does not match the pinned build.
+    - name: Keep the executable owned by root and not writable by the agent
+      ansible.builtin.file:
+        path: '/usr/local/lib/personal-beszel/{{ beszel_lock.version }}/beszel-agent'
+        owner: root
+        group: root
+        mode: '0755'
     - name: Install nonsecret node connection settings
       ansible.builtin.copy:
         dest: /etc/personal-beszel/agent.env

package/ansible/monitoring/hub-route-darwin.yml

@@ -0,0 +1,51 @@
+---
+# The manager's hub is reached over the tailnet only: Tailscale Serve
+# terminates HTTPS on the declared port and proxies to the loopback hub. Never
+# Funnel. A port that already has any owner is left alone: fleet-sync refuses
+# a conflicting owner before this runs and reads the route back afterwards.
+- name: Serve the manager hub privately over the tailnet
+  hosts: localhost
+  connection: local
+  gather_facts: false
+  vars:
+    beszel_serve:
+      [
+        '{{ beszel_tailscale_cli }}',
+        serve,
+        --bg,
+        '--https={{ beszel_route_port }}',
+        '{{ beszel_route_target }}',
+      ]
+  tasks:
+    - name: Require the declared loopback target and port
+      ansible.builtin.assert:
+        that:
+          - "beszel_route_target is match('^http://127\\.0\\.0\\.1:[0-9]+$')"
+          - "beszel_route_port is match('^[0-9]+$')"
+          - "beszel_tailscale_cli is match('^/')"
+
+    - name: Read this node's Serve configuration
+      ansible.builtin.command:
+        argv: ['{{ beszel_tailscale_cli }}', serve, status, --json]
+      register: beszel_serve_status
+      changed_when: false
+    - name: Publish the hub on the tailnet HTTPS port as the signed-in user
+      ansible.builtin.command:
+        argv: '{{ beszel_serve }}'
+      vars:
+        serve: "{{ beszel_serve_status.stdout | default('{}', true) | from_json }}"
+      when:
+        - beszel_route_port not in (serve.TCP | default({}))
+        - "(serve.Web | default({})) | select('search', ':' ~ beszel_route_port ~ '$') | list | length == 0"
+      register: beszel_served
+      changed_when: beszel_served.rc == 0
+      failed_when: false
+    # The standalone daemon accepts Serve changes from an administrator only.
+    - name: Publish the hub with administrator rights where the daemon requires them
+      ansible.builtin.command:
+        argv: '{{ beszel_serve }}'
+      become: true
+      when:
+        - beszel_served is not skipped
+        - beszel_served.rc != 0
+      changed_when: true

package/ansible/monitoring/lock.json

@@ -1,11 +1,16 @@
 {
   "version": "0.21.1",
   "image": "henrygd/beszel:0.21.1@sha256:8514a69d9bbbcb74f371a3205561ee231fc0289739cea64e8131f341d695b770",
+  "tunnelImage": "cloudflare/cloudflared:2026.9.3@sha256:072c067d25ccbe61d46e18f0d0723255f2bb5304f7317caa95b27031520ff92c",
   "agents": {
     "amd64": "3b6ee592632ec03b9a43312e99a40203a7b3d83fe40dd0c25acda1eb915ea7c0",
     "arm64": "a6ea544976b9a74398ac88d5da5bb96813d9a2e353a37bbfdb6a903fa8ba1838",
     "darwin-arm64": "9246065f680cdb0af2885b7bf5876dc45b754019f2c90eb28550cf45db978eb8",
     "darwin-amd64": "21f4a03fd90ae46869cfcbb69e21e8312f0f05deff49669d4602a3dee4866981"
   },
+  "agentBinaries": {
+    "amd64": "5c84416b63028f7ad86ebafa4c8a7a27977d485a838de357216ded3ab844b2b5",
+    "arm64": "eb4e7ac9340d380b19ab18261301b18813c1e46ebab5f0e4bf8cb6c86b4d5c20"
+  },
   "source": "https://github.com/henrygd/beszel/releases/tag/v0.21.1"
 }

package/ansible/pyproject.toml

@@ -0,0 +1,12 @@
+# The Ansible the manager runs these playbooks with: one release, on one
+# Python, with every dependency hash-locked in uv.lock. A worker runs the
+# exact ansible-core package ubuntu/policy.json names instead. Change the
+# version here, then run `uv lock` in this directory.
+[project]
+name = "dev-fleet-ansible"
+version = "0"
+requires-python = "==3.13.16"
+dependencies = ["ansible-core==2.19.9"]
+
+[tool.uv]
+package = false

package/ansible/ubuntu/baseline.yml

@@ -44,7 +44,7 @@
         that:
           - item.freeMiB >= minimumFreeDiskMiB
           - item.freeInodes >= minimumFreeInodes
-        fail_msg: RESOURCE_CAPACITY_REQUIRED; inspect disk space and inodes on the reported filesystem. Preserve developer work; no automatic cleanup or process killing.
+        fail_msg: RESOURCE_CAPACITY_REQUIRED; inspect disk space and inodes on the reported filesystem. Preserve developer work; the disk GC removes only build output and caches, so what remains needs a person.
       loop: '{{ capacity.stdout | from_json }}'
     - name: Inspect installed packages
       ansible.builtin.package_facts:

package/ansible/ubuntu/disk-gc.py

@@ -0,0 +1,336 @@
+#!/usr/bin/python3
+"""Recover disk space by deleting only what a build or install can recreate.
+
+Managed by personal fleet-sync (tasks/resources.yml) as
+personal-fleet-disk-gc.service, started hourly by personal-fleet-disk-gc.timer.
+
+What it deletes:
+  * dependency and build directories (node_modules, .turbo, dist, Rust target,
+    ...) inside git checkouts under the home directory, only when git says the
+    directory is ignored, the checkout has been idle for --idle-days and no
+    process is running inside it;
+  * Docker images and build cache that no container uses and that are older
+    than --docker-age-hours (never containers or volumes);
+  * the apt package cache.
+When a watched filesystem has less than --pressure-mib free, idle checkouts
+qualify after --pressure-idle-hours, Docker leftovers after
+--pressure-docker-age-hours, and aged entries of each --pressure-cache
+directory are removed too.
+
+It never deletes a checkout, a tracked file, uncommitted work, a container or
+a volume. Every decision is one JSON line on stdout.
+
+Usage: disk-gc.py --home H [--path P ...] [--idle-days N] [--pressure-mib N]
+                  [--pressure-idle-hours N] [--docker-age-hours N]
+                  [--pressure-docker-age-hours N] [--pressure-cache DIR ...]
+                  [--max-depth N] [--proc DIR] [--home-only] [--dry-run]
+
+--home-only skips the machine-wide Docker and apt steps.
+"""
+import argparse
+import json
+import os
+import shutil
+import stat
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+MIB = 1024 * 1024
+HOUR = 3600
+# Directory names that hold only installed dependencies or build output.
+REGENERABLE = frozenset({
+    'node_modules', '.cache', '.turbo', '.next', '.nuxt', '.svelte-kit', '.astro',
+    '.vite', '.parcel-cache', '.expo', '.gradle', 'dist', 'build', 'coverage',
+    '.pytest_cache', '.mypy_cache', '.ruff_cache', '__pycache__', '.venv',
+})
+# Directory names that are regenerable only beside the manifest that builds them.
+REGENERABLE_BESIDE = {'target': 'Cargo.toml'}
+# Hidden directories under the home directory that still hold checkouts.
+HIDDEN_ROOTS = frozenset({'.t3'})
+GIT = ['git', '-c', 'safe.directory=*']
+
+
+def log(**event):
+    print(json.dumps(event, sort_keys=True), flush=True)
+
+
+def free_bytes(path):
+    st = os.statvfs(path)
+    return st.f_bavail * st.f_frsize
+
+
+def mtime(path):
+    try:
+        return os.lstat(path).st_mtime
+    except OSError:
+        return 0.0
+
+
+def is_real_dir(path):
+    try:
+        return stat.S_ISDIR(os.lstat(path).st_mode)
+    except OSError:
+        return False
+
+
+def find_checkouts(home, max_depth):
+    """Git checkouts (repositories and worktrees) under home, outermost only."""
+    found = []
+
+    def walk(directory, depth):
+        if os.path.lexists(os.path.join(directory, '.git')):
+            found.append(directory)
+            return
+        if depth >= max_depth:
+            return
+        try:
+            entries = list(os.scandir(directory))
+        except OSError:
+            return
+        for entry in entries:
+            if not entry.is_dir(follow_symlinks=False):
+                continue
+            name = entry.name
+            if name in REGENERABLE or name in REGENERABLE_BESIDE:
+                continue
+            if name.startswith('.') and not (depth == 0 and name in HIDDEN_ROOTS):
+                continue
+            walk(entry.path, depth + 1)
+
+    walk(home, 0)
+    return sorted(found)
+
+
+def git_dir(checkout):
+    """The checkout's git directory, following a worktree's .git file."""
+    dot_git = os.path.join(checkout, '.git')
+    if is_real_dir(dot_git):
+        return dot_git
+    try:
+        text = Path(dot_git).read_text()
+    except OSError:
+        return None
+    if not text.startswith('gitdir:'):
+        return None
+    target = text[len('gitdir:'):].strip()
+    target = os.path.normpath(os.path.join(checkout, target))
+    return target if is_real_dir(target) else None
+
+
+def run_git(checkout, args, data=None):
+    try:
+        out = subprocess.run(GIT + ['-C', checkout] + args, input=data,
+                             capture_output=True, timeout=600)
+    except (OSError, subprocess.TimeoutExpired):
+        return None
+    return out
+
+
+def candidates(checkout):
+    """Regenerable directories in a checkout, outermost only, never through a link."""
+    found = []
+    for current, dirs, files in os.walk(checkout):
+        keep = []
+        for name in dirs:
+            path = os.path.join(current, name)
+            if name == '.git' or os.path.islink(path):
+                continue
+            if os.path.lexists(os.path.join(path, '.git')):
+                continue  # a nested repository answers for itself
+            beside = REGENERABLE_BESIDE.get(name)
+            if name in REGENERABLE or (beside and beside in files):
+                found.append(path)
+                continue
+            keep.append(name)
+        dirs[:] = keep
+    return found
+
+
+def ignored(checkout, paths):
+    """The subset of paths git ignores; tracked or unignored paths are kept."""
+    if not paths:
+        return []
+    rel = [os.path.relpath(p, checkout) for p in paths]
+    out = run_git(checkout, ['check-ignore', '--stdin', '-z'],
+                  data=('\0'.join(rel) + '\0').encode())
+    if out is None or out.returncode not in (0, 1):
+        return []
+    hits = {item for item in out.stdout.decode(errors='replace').split('\0') if item}
+    return [p for p, r in zip(paths, rel) if r in hits or r.rstrip('/') in hits]
+
+
+def git_activity(checkout, gdir):
+    """The latest commit, checkout or index refresh: cheap, read first."""
+    marks = [checkout, os.path.join(gdir, 'index'), os.path.join(gdir, 'HEAD'),
+             os.path.join(gdir, 'logs', 'HEAD')]
+    return max(mtime(p) for p in marks)
+
+
+def last_activity(checkout, gdir, dirs):
+    """The latest sign of work: git activity, changed files, build output writes."""
+    latest = max([git_activity(checkout, gdir)] + [mtime(p) for p in dirs])
+    out = run_git(checkout, ['status', '--porcelain=v1', '-z', '--untracked-files=all'])
+    if out is None or out.returncode != 0:
+        return None
+    entries = out.stdout.decode(errors='replace').split('\0')
+    for entry in entries:
+        if len(entry) > 3:
+            latest = max(latest, mtime(os.path.join(checkout, entry[3:])))
+    return latest
+
+
+def busy_roots(proc):
+    """Working directories of every running process."""
+    roots = set()
+    try:
+        pids = [p for p in os.listdir(proc) if p.isdigit()]
+    except OSError:
+        return roots
+    for pid in pids:
+        try:
+            roots.add(os.readlink(os.path.join(proc, pid, 'cwd')))
+        except OSError:
+            continue
+    return roots
+
+
+def in_use(checkout, roots):
+    prefix = checkout.rstrip('/') + '/'
+    return any(r == checkout or r.startswith(prefix) for r in roots)
+
+
+def size_of(path):
+    total = 0
+    for current, dirs, files in os.walk(path):
+        for name in dirs + files:
+            try:
+                total += os.lstat(os.path.join(current, name)).st_blocks * 512
+            except OSError:
+                continue
+    return total
+
+
+def remove(path, dry_run):
+    """Deletes one directory without following links; returns bytes freed."""
+    if not is_real_dir(path):
+        return 0
+    size = size_of(path)
+    if not dry_run:
+        shutil.rmtree(path, ignore_errors=True)
+        if os.path.lexists(path):
+            return 0
+    return size
+
+
+def collect_checkouts(home, idle_seconds, max_depth, proc, now, dry_run):
+    roots = busy_roots(proc)
+    freed = 0
+    for checkout in find_checkouts(home, max_depth):
+        gdir = git_dir(checkout)
+        if gdir is None:
+            log(event='checkout_skipped', checkout=checkout, reason='no_git_dir')
+            continue
+        if in_use(checkout, roots) or now - git_activity(checkout, gdir) < idle_seconds:
+            continue
+        dirs = ignored(checkout, candidates(checkout))
+        if not dirs:
+            continue
+        latest = last_activity(checkout, gdir, dirs)
+        if latest is None:
+            log(event='checkout_skipped', checkout=checkout, reason='git_status_failed')
+            continue
+        if now - latest < idle_seconds:
+            continue
+        for path in dirs:
+            got = remove(path, dry_run)
+            freed += got
+            log(event='removed', path=path, mib=got // MIB, dry_run=dry_run)
+    return freed
+
+
+def collect_cache(directory, age_seconds, now, dry_run):
+    """Removes top-level cache entries not written for age_seconds."""
+    freed = 0
+    try:
+        entries = list(os.scandir(directory))
+    except OSError:
+        return 0
+    for entry in entries:
+        if now - mtime(entry.path) < age_seconds:
+            continue
+        if entry.is_dir(follow_symlinks=False):
+            got = remove(entry.path, dry_run)
+        elif entry.is_file(follow_symlinks=False):
+            got = entry.stat(follow_symlinks=False).st_blocks * 512
+            if not dry_run:
+                try:
+                    os.unlink(entry.path)
+                except OSError:
+                    got = 0
+        else:
+            continue
+        freed += got
+    if freed:
+        log(event='cache_pruned', cache=directory, mib=freed // MIB, dry_run=dry_run)
+    return freed
+
+
+def command(argv, dry_run, name):
+    if shutil.which(argv[0]) is None:
+        return
+    if dry_run:
+        log(event='would_run', step=name)
+        return
+    try:
+        out = subprocess.run(argv, capture_output=True, timeout=1800)
+        log(event='ran', step=name, ok=out.returncode == 0)
+    except (OSError, subprocess.TimeoutExpired) as exc:
+        log(event='ran', step=name, ok=False, error=type(exc).__name__)
+
+
+def main(argv):
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--home', required=True)
+    parser.add_argument('--path', action='append', default=[])
+    parser.add_argument('--idle-days', type=float, default=7)
+    parser.add_argument('--pressure-mib', type=int, default=15360)
+    parser.add_argument('--pressure-idle-hours', type=float, default=12)
+    parser.add_argument('--docker-age-hours', type=int, default=168)
+    parser.add_argument('--pressure-docker-age-hours', type=int, default=24)
+    parser.add_argument('--pressure-cache', action='append', default=[])
+    parser.add_argument('--max-depth', type=int, default=4)
+    parser.add_argument('--proc', default='/proc')
+    parser.add_argument('--home-only', action='store_true')
+    parser.add_argument('--dry-run', action='store_true')
+    args = parser.parse_args(argv)
+    home = os.path.realpath(args.home)
+    paths = args.path or ['/', home]
+    now = time.time()
+    before = {p: free_bytes(p) for p in paths}
+    pressure = min(before.values()) < args.pressure_mib * MIB
+    idle = args.pressure_idle_hours * HOUR if pressure else args.idle_days * 24 * HOUR
+    docker_age = args.pressure_docker_age_hours if pressure else args.docker_age_hours
+    log(event='started', pressure=pressure, idle_hours=idle / HOUR,
+        docker_age_hours=docker_age, dry_run=args.dry_run,
+        free_mib={p: v // MIB for p, v in before.items()})
+
+    freed = collect_checkouts(home, idle, args.max_depth, args.proc, now, args.dry_run)
+    if pressure:
+        for cache in args.pressure_cache:
+            freed += collect_cache(os.path.join(home, cache), idle, now, args.dry_run)
+    if not args.home_only:
+        command(['docker', 'image', 'prune', '--all', '--force', '--filter',
+                 f'until={docker_age}h'], args.dry_run, 'docker_images')
+        command(['docker', 'builder', 'prune', '--all', '--force', '--filter',
+                 f'until={docker_age}h'], args.dry_run, 'docker_build_cache')
+        command(['apt-get', 'clean'], args.dry_run, 'apt_cache')
+
+    after = {p: free_bytes(p) for p in paths}
+    log(event='finished', checkout_mib=freed // MIB, dry_run=args.dry_run,
+        free_mib={p: v // MIB for p, v in after.items()})
+
+
+if __name__ == '__main__':
+    main(sys.argv[1:])