code-coordinator 0.5.46__py3-none-any.whl

coord/dead_end.py

@@ -0,0 +1,332 @@
+"""The driver's dead-end predicate (#2019).
+
+``coord drive`` could not tell **"still working"** apart from **"finished in
+a state I cannot act on."**  Both rendered as ``no state change in N m`` and
+the second one looped forever — a counter incrementing against an event that
+can never occur, holding a tmux session, a queue slot and (since #1972) an
+entire repo's capacity lane.  The live case this module is named for burned
+**140 minutes** on claude-coordinator#1956 with ``review=done/-`` on screen
+the whole time.
+
+WHAT THIS MODULE IS
+-------------------
+One pure function, :func:`detect_dead_end`, over the driver's already-computed
+:class:`coord.drive_state.IssueState`.  It answers a single question:
+
+    Is this row **terminal AND unactionable** — i.e. is there no board
+    transition left that any amount of polling could produce?
+
+It is **not** a stall detector.  Per #2019 ask 4, elapsed time is deliberately
+NOT an input: a healthy long-running Work stage is legitimately quiet for
+hours, and a dead end is knowable at minute zero.  The only clock-shaped
+guard here is the hard precondition ``active_count == 0`` — if anything at all
+is running on the fleet, this function returns ``None``, however long it has
+been running.  That guard is what makes a false positive on a healthy stage
+structurally impossible rather than merely unlikely.
+
+WHY THE 140 MINUTES HAPPENED (the field-mismatch bug)
+-----------------------------------------------------
+``coord.drive._decide_review`` has had a ``_die`` for "review finished with no
+verdict" since the original bash port — but it keys on
+``state.work_review_state``, the **work** row's projected ``review_state``.
+The incident's board showed ``review=done/-``, which is
+``state.review_status`` — the **review assignment's own** status.  On #1956
+the review row reached ``status="done"`` while the work row's ``review_state``
+was never advanced (advancing it is exactly what recording the verdict does),
+so the die never fired and ``_decide_review`` fell through to a bare
+``_wait()``.  Two readings of "the review is done", one checked, one not.
+:data:`_TERMINAL_REVIEW_STATUSES` closes that by keying on the review row.
+
+WHAT IS DELIBERATELY *NOT* COVERED
+----------------------------------
+* **A Test stage that was never dispatched, with no policy saying so**
+  (``test_state=""``, no smoke row, no ``test-mode:*`` label) — from board
+  state alone that is indistinguishable from "the daemon will dispatch it on
+  the next tick", and the rest of the dispatcher's refusal set
+  (``coord.smoke.dispatch_smoke`` returns ``None`` for a superseded row, a
+  capability-rule miss on a non-``work`` type, a missing smoke command, …) is
+  not re-derivable here without duplicating it.  Guessing would escalate
+  healthy rows.  Two variants ARE covered, both because something on the board
+  positively states that no dispatch is coming: ``test_state="blocked"``, the
+  marker the dispatcher leaves when it gives up (#1672, shape 2), and
+  ``test-mode:smoke``, the per-issue POLICY that switches the headless Test
+  stage off entirely (#685/#2024, shape 3).
+* **A ``done`` smoke row whose verdict has not landed yet** — #1605 already
+  established that this has an expected, bounded propagation lag and is not a
+  defect.  Only the contradictions #1605 itself detects are terminal.
+* **A zero-commit advisory / a refused pre-dispatch guard** — already
+  escalated correctly by ``coord.drive._decide_advisory``, the "finished with
+  no branch" die, and #1844's ``EXIT_DISPATCH_REFUSED``.  vimcode#634 proved
+  that path works; this module deliberately does not touch it.
+
+The registry below is meant to grow one *proven* shape at a time.  A shape
+belongs here only when the board state alone makes the row unactionable — not
+when it merely looks quiet.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+# A review assignment that reached one of these is FINISHED — no worker is
+# coming back to add a verdict.  `"failed"` is deliberately absent: #1584
+# owns it with a bounded `coord review` re-dispatch, and a dead-end verdict
+# there would steal a retry that genuinely can succeed.
+_TERMINAL_REVIEW_STATUSES = frozenset({"done", "cancelled"})
+
+
+@dataclass(frozen=True)
+class DeadEnd:
+    """One terminal-and-unactionable board shape, fully described.
+
+    ``reason`` names the SPECIFIC dead end (#2019 ask 3) — "review reached
+    done with no verdict" rather than "no state change in 140.558m" — and
+    ``recovery`` is the command an operator can paste.  Both end up on the
+    ``coord escalate record`` row, in the drive-queue entry's ``last_reason``,
+    and in the GitHub escalation comment, so all three surfaces say the same
+    thing.
+    """
+
+    kind: str
+    """Stable slug for the shape, e.g. ``review_terminal_no_verdict``."""
+
+    stage: str
+    """Pipeline stage the dead end sits in — ``coord escalate record --stage``."""
+
+    reason: str
+    """Why no poll can change this.  One paragraph, operator-facing."""
+
+    recovery: str
+    """The command that unblocks it, ready to paste."""
+
+    assignment_id: str = ""
+    """The row an operator should look at first (``""`` when unknown)."""
+
+    gates: tuple[tuple[str, str], ...] = ()
+    """Observed gate readings, ``(key, value)`` — mirrors ``_escalate_merge``."""
+
+
+def _gates(state) -> tuple[tuple[str, str], ...]:
+    """The board readings worth recording on any dead end.
+
+    Same shape as ``coord.drive._escalate_merge``'s ``gate_pairs`` so the two
+    escalation kinds render identically in ``coord escalate list``.
+    """
+    return (
+        ("work_status", state.work_status or "(empty)"),
+        ("test_state", state.work_test_state or "(none)"),
+        # #2024: the per-issue Test-stage POLICY is a reading in its own right
+        # — for shape 3 it is the entire proof, and on every other shape it is
+        # the difference between "no verdict yet" and "no verdict, ever".
+        ("issue_test_mode", state.issue_test_mode or "(none)"),
+        ("review_status", state.review_status or "(none)"),
+        ("review_verdict", state.review_verdict or "(none)"),
+        ("active", str(state.active_count)),
+    )
+
+
+def detect_dead_end(state, *, can_waive_test_gate: bool = False) -> DeadEnd | None:
+    """The predicate: ``None`` when the row can still move on its own.
+
+    *state* is a :class:`coord.drive_state.IssueState` (untyped here purely to
+    keep this module import-light — it reads a handful of attributes and
+    nothing else).
+
+    *can_waive_test_gate* (#2024) is the caller saying "I still have a Test-
+    stage move of my own" — today that is ``coord drive --skip-test``, whose
+    ``_decide_test`` records ``skipped`` for a verdict-less row. Shape 3 below
+    is exactly the shape that flag exists to unblock, so with it set that shape
+    is not a dead end: the driver must be allowed to use the move the operator
+    explicitly asked for rather than escalate past it.
+
+    Pure, cheap, and safe to call on every poll: no I/O, no clock, no
+    counters.  The caller (``coord.drive.decide``) turns a non-``None`` result
+    into an ``EXIT`` action carrying ``coord.drive.EXIT_DEAD_END``.
+    """
+    # #1672's "the fleet cannot route this Test stage" marker.  IMPORTED (not
+    # mirrored as a literal) so the two can never drift: `dispatch_smoke`
+    # bails out on this exact value, and its refusal to re-probe is precisely
+    # why a driver polling against it can never make progress.  Deferred to
+    # call time purely to keep THIS module importable on its own without
+    # dragging in `coord.smoke`'s `httpx` dependency — `coord.drive` already
+    # pulls httpx in via `coord.usage_limits`, so nothing is saved there, but
+    # a predicate this small should stay cheap to import and test in
+    # isolation.
+    from coord.smoke import TEST_STATE_BLOCKED  # noqa: PLC0415
+    # THE conservative guard, and it stays first.  #2019 acceptance: "a
+    # genuinely long-running work stage (active=1) does NOT escalate, however
+    # long it runs."  Anything in flight — work, test, review, a fix round,
+    # the acceptance author — makes every question below premature.
+    if state.active_count > 0:
+        return None
+
+    # ── shape 1: the review row is terminal and carries no verdict ──────────
+    # claude-coordinator#1956, 2026-08-08.  The driver can neither dispatch a
+    # fix (there is no `request-changes` to fix) nor proceed to merge (there
+    # is no `approve` to merge on), and the row that would have supplied
+    # either is finished.  Nothing about waiting changes that.
+    if (
+        state.review_aid
+        and state.review_status in _TERMINAL_REVIEW_STATUSES
+        and not state.review_verdict
+        # Belt-and-braces: an actionable failed test outranks a stale review
+        # on the same work row — let `_decide_test`'s bounded fix loop have
+        # it rather than dead-ending a row that still has a live move.
+        and state.work_test_state != "failed"
+    ):
+        if state.review_status == "cancelled":
+            return DeadEnd(
+                kind="review_cancelled_no_verdict",
+                stage="review",
+                reason=(
+                    f"review {state.review_aid} was CANCELLED before recording "
+                    "a verdict — no verdict is coming, and this driver has "
+                    "nothing to approve, fix, or merge on. Terminal on the "
+                    "board; polling cannot change it (#2019)."
+                ),
+                recovery=f"coord review {state.work_aid}",
+                assignment_id=state.review_aid,
+                gates=_gates(state),
+            )
+        # status == "done": the session finalised CLEANLY.  Saying so matters
+        # — #2019 acceptance requires this text to distinguish an
+        # END_REVIEW-without-verdict from a crashed session, and to stop
+        # citing the CLOSED #812 (which was about INTERACTIVE reviews that
+        # never started) at a headless review that ran to completion.  A
+        # review worker that actually died lands `status="failed"`, which is
+        # `_decide_review`'s #1584 bounded-retry arm, not this one.
+        #
+        # #812 is deliberately absent from the text below — even as a
+        # disclaimer.  An operator who greps the reason for an issue number
+        # to open must never land on a closed one, and "not #812" reads as
+        # "#812" to every mechanical reader (and to a tired one at 2am).  The
+        # correction lives here, in the source, where it belongs.
+        return DeadEnd(
+            kind="review_terminal_no_verdict",
+            stage="review",
+            reason=(
+                f"review {state.review_aid} reached status=done carrying NO "
+                "verdict. The review session finalised cleanly — a crashed, "
+                "killed or never-started session lands status='failed' (which "
+                "this driver retries), so this is the END_REVIEW-without-"
+                "verdict class: the reviewer finished and its REVIEW_VERDICT "
+                "header was never emitted or never parsed (#1956, "
+                "coord.review.detect_end_review_without_verdict). The verdict "
+                "is very likely already sitting in the transcript. With no "
+                "verdict there is no fix to dispatch and no approval to merge "
+                "on, and the review row is terminal — no number of polls "
+                "changes that (#2019)."
+            ),
+            # Straight out of docs/OPERATING_GOTCHAS.md's "Recovery — do NOT
+            # re-dispatch" block: re-running the review costs a full cycle to
+            # re-derive a conclusion already in the log, and the drop
+            # reproduces at a documented ~14% rate (#873).
+            recovery=(
+                f"coord report-result --assignment {state.review_aid} "
+                "--status done --verdict <approve|request-changes> "
+                "--verdict-source recovered --verdict-reason 'REVIEW_VERDICT "
+                "header missing, recovered from transcript (#1956)' "
+                "--body-file <extracted-review.md>"
+            ),
+            assignment_id=state.review_aid,
+            gates=_gates(state),
+        )
+
+    # ── shape 2: the Test stage is BLOCKED (#1672) ─────────────────────────
+    # `dispatch_smoke` stamps `test_state="blocked"` when no capability-
+    # matched machine could run the suite, and then REFUSES to re-probe on
+    # every tick — deliberately, that spin is what #1672 closed.  So the
+    # driver polling for a verdict is polling for something the dispatcher
+    # has already decided not to produce.  vimcode#635's shape, in the one
+    # variant the board makes provable.
+    if state.work_test_state == TEST_STATE_BLOCKED:
+        return DeadEnd(
+            kind="test_stage_blocked",
+            stage="test",
+            reason=(
+                f"the Test stage for {state.work_aid} is BLOCKED: "
+                f"{state.work_test_reason or 'no reason recorded'} — "
+                "coord.smoke.dispatch_smoke found no capability-matched "
+                "machine and recorded 'blocked' rather than re-probing a "
+                "broken fleet on every tick (#1672). It will not try again "
+                "on its own, so waiting for a verdict here waits forever "
+                "(#2019). Fix the fleet (or the capability rules), then "
+                "clear the marker."
+            ),
+            recovery=(
+                f"coord diagnose {state.repo} {state.issue} --stage test --reset"
+            ),
+            assignment_id=state.work_aid,
+            gates=_gates(state),
+        )
+
+    # ── shape 3: the Test stage is HUMAN-ATTENDED by policy (#685/#2024) ────
+    # `test-mode:smoke` means "the headless Test stage does not run for this
+    # issue; the TUI offers an interactive smoke agent instead" —
+    # `dispatch_pending_smoke` skips the row unconditionally, on every tick,
+    # forever. Review dispatch is meanwhile gated on a passed/skipped verdict
+    # for THIS work row (`pipeline.test_precedes_review()`, honoured by both
+    # `dispatch_pending_reviews` and `auto_loop.run_for_fix_transition`), so a
+    # completed row with no verdict of its own cannot advance: one component
+    # requires a verdict and, by policy, no component will produce one.
+    #
+    # Why this bites `--fix-of` rounds specifically (#2024): a fix round is a
+    # NEW work row on the SAME branch, and it carries its own empty
+    # `test_state`. The parent's verdict satisfies the branch-scoped MERGE
+    # gate (`coord gates` reads `test: passed` off it — see the note
+    # `coord.gates.build_gate_report` now emits), which is why the stall reads
+    # as "slow" rather than "blocked". Round 0 gets attended because a human is
+    # watching the first pass; rounds 1..N complete unattended at 3am and sit
+    # there. Observed twice on JDonaghy/vimcode#635 (2026-08-08): 25 minutes,
+    # then 160 minutes on the same issue, each cleared within minutes of an
+    # operator running `coord test <fix_aid> --passed` by hand.
+    #
+    # Deliberately narrow, so it can never fire on a healthy row:
+    #   * `active_count == 0` (above) — an interactive smoke session IS a live
+    #     board row, so an attended Test stage in progress never reaches here.
+    #   * `smoke_aid` empty — a Test-stage child dispatched for THIS row (by a
+    #     human, `--smoke-of`, or an earlier auto pass) means the stage did
+    #     happen; #1605 owns a `done` smoke whose verdict hasn't landed yet.
+    #   * `work_test_state` empty — not "running" (#1395's transient marker,
+    #     `_decide_test` waits on it), not a terminal verdict, not "blocked"
+    #     (shape 2 above owns that, and reads better).
+    if (
+        not can_waive_test_gate
+        and state.issue_test_mode == "smoke"
+        and state.work_status == "done"
+        and state.work_aid
+        and not state.work_test_state
+        and not state.smoke_aid
+    ):
+        return DeadEnd(
+            kind="test_stage_human_attended",
+            stage="test",
+            reason=(
+                f"work {state.work_aid} finished with NO Test verdict, and "
+                f"{state.repo}#{state.issue} is labelled `test-mode:smoke` — "
+                "the per-issue policy (#685) that switches the HEADLESS Test "
+                "stage off: coord.smoke.dispatch_pending_smoke skips this "
+                "issue on every tick by design, so no smoke assignment is "
+                "coming. Review dispatch is meanwhile held until THIS row "
+                "carries a passed/skipped verdict "
+                "(pipeline.test_precedes_review), so nothing can advance "
+                "without a human. This is the #2024 shape: a --fix-of round "
+                "is a new work row with its own empty test_state, and the "
+                "parent's verdict satisfies only the branch-scoped merge gate "
+                "(`coord gates` reads `test: passed` off the parent row) — "
+                "which is why this reads as slow rather than blocked (#2019)."
+            ),
+            # Two doors, cheapest first: record the verdict directly, or run
+            # the attended Test stage the label asked for. Both are real; the
+            # operator picks by whether the suite actually needs running.
+            recovery=(
+                f"coord test {state.work_aid} --passed   # or --skipped "
+                "--reason '<why>'; to actually run it, `coord assign "
+                f"<machine> {state.repo} {state.issue} --smoke-of "
+                f"{state.work_aid} --interactive`"
+            ),
+            assignment_id=state.work_aid,
+            gates=_gates(state),
+        )
+
+    return None

coord/deploy/README.md

@@ -0,0 +1,33 @@
+# `coord/deploy/` — the packaged systemd units (#1927)
+
+**This directory is a byte-identical copy of the repo-root `deploy/` unit
+files, and it is the copy that ships in the wheel.**
+
+Edit `deploy/<name>` at the repo root — that is the reviewed source of
+truth — then copy it here:
+
+```
+cp deploy/*.service deploy/*.timer coord/deploy/
+```
+
+`tests/test_packaged_deploy_units.py` fails if the two ever disagree, so the
+copy cannot drift silently.
+
+## Why the copy exists
+
+`coord release verify`'s unit-drift check
+(`coord/health/checks/unit_drift.py`, #1831) diffs each host's *installed*
+unit under `~/.config/systemd/user/` against a reference. Until #1927 that
+reference was `<checkout>/deploy/<name>` — a file in the host's own git
+working copy, which nothing verifies is current.
+
+Installed units and checkouts go stale for the same reason (nobody pulled),
+so they go stale *together*, and the diff then reports clean — the check was
+least reliable in exactly the #1831 case it exists to catch, and its printed
+remedy (`cp <checkout>/deploy/... ~/.config/systemd/user/...`) cemented the
+stale unit.
+
+Files under `coord/` ship in the wheel, so on a pip-installed host this
+directory is the unit set *as of the installed version*. It cannot drift
+with the host. The `*.sh` helpers in the repo-root `deploy/` are not copied
+here: the drift check only reads `*.service`/`*.timer`.

coord/deploy/coord-agent.service

@@ -0,0 +1,89 @@
+# coord agent — per-machine worker/dispatch server (#1570/#1671).
+#
+# A systemd *user* unit — the FIRST of the four coord services (every
+# worker box runs this one; coord-serve/coord-web/coord-notify run only on
+# the box that owns ~/.coord/coord.db). Runs `coord agent`, which accepts
+# dispatched assignments, spawns `claude -p` workers, and answers /health
+# with this machine's resolved tool versions (coord/prereqs.py) so
+# `coord doctor` can see capability drift without SSHing in.
+#
+# `install-agent.sh` generates a unit like this one inline (heredoc) as
+# part of the curl|bash installer flow and stays the SOURCE OF TRUTH for a
+# fresh install — keep the two in sync by hand. This checked-in copy
+# exists so:
+#   (a) a manual install / install from a checkout has something to `cp`
+#       (mirrors coord-serve/coord-web/coord-notify below, all `cp`-installed
+#       from this same directory), and
+#   (b) the PATH fix below is documented in the repo instead of living only
+#       inside a shell heredoc nobody reads until #1671 bites again.
+#
+# #1671: PATH MUST include ~/.cargo/bin (and ~/.local/bin). A systemd user
+# unit's default PATH is minimal — it omits anything rustup/pipx/etc.
+# installed under $HOME. Without this, the `rust` capability probe
+# (coord/prereqs.py, literal `cargo --version`) reads "not found" even
+# though cargo IS installed, and #1570 D's dispatch guard then refuses to
+# route ANY tui/** work to this machine forever — a silent 30s retry loop,
+# no smoke row at all (see #1671).
+#
+# Do NOT "fix" this by symlinking just the `cargo` binary onto the
+# existing PATH instead of adding the whole directory. `cargo build` also
+# shells out to `rustc`, which lives in the same `~/.cargo/bin` — a probe
+# that resolves `cargo` alone reports the capability MET while
+# `cargo build` still fails inside the worker. That is a false green,
+# strictly worse than the honest refusal this issue started from. Also
+# note #402: a worker's PATH is derived from *this agent's own* PATH
+# (venv stripped) — so probe-side and worker-side resolution move
+# together only if the fix is "widen the agent's PATH here," not "make
+# the probe search harder while the worker still can't find the tool."
+#
+# #1678: the `browser` capability (Playwright acceptance suites, e.g.
+# `coord/dashboard/webapp` -> `npm run test:e2e`) needs `node`/`npm` here
+# too, but Node must NOT be added the way ~/.cargo/bin was. nvm installs
+# into a version-stamped directory (~/.nvm/versions/node/vX.Y.Z/bin), so a
+# literal entry on the PATH line below would break silently at the next
+# `nvm install` and put the capability right back where #1678 found it.
+# Instead `install-agent.sh` drops run-time-resolving shims (node/npm/npx ->
+# coord-node-shim, see deploy/node-shim.sh) into ~/.local/bin, which is
+# ALREADY on the PATH line below — so nothing here changes and a Node
+# version bump is a no-op. Keep ~/.local/bin on the PATH for that reason.
+#
+# Placeholders to fill in before installing (install-agent.sh fills the
+# equivalents in automatically from its --machine/--port flags):
+#   <MACHINE_NAME>  — this machine's name in coordinator.yml (often the
+#                     short hostname; `hostname -s`)
+#   <PORT>          — 7433 unless this box intentionally runs more than
+#                     one agent
+#
+# Install (manual / from a checkout — normally use install-agent.sh
+# instead, which does not need this file):
+#   mkdir -p ~/.config/systemd/user
+#   sed -e "s/<MACHINE_NAME>/$(hostname -s)/" -e 's/<PORT>/7433/' \
+#       deploy/coord-agent.service > ~/.config/systemd/user/coord-agent.service
+#   loginctl enable-linger "$USER"          # survive logout / reboot
+#   systemctl --user daemon-reload
+#   systemctl --user enable --now coord-agent
+#
+# Already have a coord-agent unit without this PATH entry (e.g. installed
+# before #1671)? A systemd drop-in is the least-disruptive fix — no need
+# to replace the whole unit:
+#   mkdir -p ~/.config/systemd/user/coord-agent.service.d
+#   printf '[Service]\nEnvironment=PATH=%%h/.coord-venv/bin:%%h/.cargo/bin:%%h/.local/bin:/usr/local/bin:/usr/bin:/bin\n' \
+#       > ~/.config/systemd/user/coord-agent.service.d/path.conf
+#   systemctl --user daemon-reload
+#   systemctl --user restart coord-agent
+#
+# See docs/AGENT_OPERATIONS.md → "Install a new agent (first time)".
+
+[Unit]
+Description=Coordinator agent server (port <PORT>)
+After=network-online.target
+
+[Service]
+Type=simple
+ExecStart=%h/.coord-venv/bin/coord agent --machine <MACHINE_NAME> --port <PORT>
+Restart=on-failure
+RestartSec=5
+Environment=PATH=%h/.coord-venv/bin:%h/.cargo/bin:%h/.local/bin:/usr/local/bin:/usr/bin:/bin
+
+[Install]
+WantedBy=default.target

coord/deploy/coord-db-backup.service

@@ -0,0 +1,60 @@
+# coord-db-backup — hourly coord.db snapshot to the external SSD, interim
+# stopgap pending #1822 (continuous backup + verified restore, tier:large,
+# unstarted), paired with coord-db-backup.timer.
+#
+# A systemd *user* unit (mirrors coord-serve/coord-web/coord-notify/
+# coord-drive-queue), Type=oneshot, timer-fired. Runs
+# ~/.local/bin/coord-db-backup.sh (deploy/coord-db-backup.sh), which:
+#   - refuses to run unless the destination directory's parent is an actual
+#     mountpoint (default /media/crucial) — the external SSD must be
+#     physically present, or it would otherwise write "backups" onto the
+#     very disk (the daemon host's root filesystem) it exists to protect
+#     against,
+#   - takes a live, WAL-safe snapshot via `sqlite3 ... VACUUM INTO`, never
+#     `cp` (a plain copy of a WAL-mode db under concurrent writes can capture
+#     a torn file — the failure you only discover at restore),
+#   - `PRAGMA integrity_check`s the snapshot and sanity-checks it has a
+#     non-empty `assignments` table before it counts; either failure keeps
+#     the bad file on disk renamed `.REJECTED` rather than deleting it or
+#     leaving it indistinguishable from a good backup,
+#   - prunes snapshots beyond `COORD_BACKUP_RETAIN` (default 168 ≈ 7 days
+#     hourly), never touching `.REJECTED` files.
+#
+# Install ONLY on the box that owns ~/.coord/coord.db (dellserver in
+# production — the same host that runs coord-serve/coord-web/coord-notify/
+# coord-drive-queue).
+#
+# NOT a substitute for off-box backup: this protects against db corruption,
+# a bad migration, accidental deletion and OS-disk failure. It does NOT
+# protect against the machine itself being lost, stolen or burned — see
+# #1822, which owns the real (continuous + verified-restore + off-box)
+# answer. This interim unit is deliberately small in scope.
+#
+# Install:
+#   mkdir -p ~/.local/bin ~/.config/systemd/user
+#   cp deploy/coord-db-backup.sh ~/.local/bin/ && chmod +x ~/.local/bin/coord-db-backup.sh
+#   cp deploy/coord-db-backup.service deploy/coord-db-backup.timer ~/.config/systemd/user/
+#   loginctl enable-linger "$USER"          # survive logout / reboot
+#   systemctl --user daemon-reload
+#   systemctl --user enable --now coord-db-backup.timer
+#
+# This is a Type=oneshot service activated BY THE TIMER — do not
+# `systemctl --user enable coord-db-backup.service` directly (it has no
+# [Install] section; the timer unit is what you enable).
+#
+# See docs/AGENT_OPERATIONS.md → "coord.db backups to the external SSD".
+
+[Unit]
+Description=Snapshot coord.db to the external SSD (interim, pending #1822)
+Documentation=https://github.com/JDonaghy/claude-coordinator/issues/1822
+
+[Service]
+Type=oneshot
+ExecStart=%h/.local/bin/coord-db-backup.sh
+# `VACUUM INTO` on a live, busy coord.db can stall under lock contention.
+# Generous relative to a normal run (a snapshot this size takes seconds),
+# well below the timer's hourly cadence (coord-db-backup.timer) so a
+# genuinely wedged run can't run into the next fire indefinitely — same
+# pattern as every other oneshot unit in this directory (see
+# coord-drive-queue.service, coord-web-dist-build.service).
+TimeoutStartSec=600

coord/deploy/coord-db-backup.sh

@@ -0,0 +1,74 @@
+#!/usr/bin/env bash
+# Interim coord.db snapshot to the external SSD (pending #1822).
+#
+# VACUUM INTO, not cp: it takes a consistent snapshot of a live SQLite database
+# while coord-serve keeps writing. A plain cp of a WAL-mode db under load can
+# capture a torn file, which is the failure mode you only discover at restore.
+#
+# Every snapshot is integrity-checked and sanity-checked (assignments table
+# present and non-empty) before it is allowed to count, and any failed check
+# leaves the snapshot on disk named .REJECTED so it can be looked at rather
+# than silently deleted OR silently left lying around under its normal name,
+# indistinguishable from a good backup.
+#
+# NOT a substitute for off-box backup: this protects against db corruption, a
+# bad migration, accidental deletion and OS-disk failure. It does NOT protect
+# against the machine being lost, stolen or burned. See #1822.
+set -uo pipefail
+
+SRC="${COORD_DB:-$HOME/.coord/coord.db}"
+DEST_DIR="${COORD_BACKUP_DIR:-/media/crucial/coord-backups}"
+RETAIN="${COORD_BACKUP_RETAIN:-168}"        # hourly x 7 days
+STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
+OUT="$DEST_DIR/coord.db.$STAMP"
+
+fail() { echo "coord-db-backup: FAILED: $*" >&2; exit 1; }
+
+[ -f "$SRC" ] || fail "source db not found: $SRC"
+
+# The mount must actually be a mount. If the SSD is unplugged, /media/crucial
+# is still a directory on the root filesystem, and we would cheerfully write
+# "backups" onto the very disk we are protecting against.
+mountpoint -q "$(dirname "$DEST_DIR")" || fail "$(dirname "$DEST_DIR") is not a mountpoint — external SSD not mounted"
+
+mkdir -p "$DEST_DIR" || fail "cannot create $DEST_DIR"
+
+sqlite3 "$SRC" "VACUUM INTO '$OUT';" || fail "VACUUM INTO failed"
+
+CHECK="$(sqlite3 "$OUT" 'PRAGMA integrity_check;' 2>&1)"
+if [ "$CHECK" != "ok" ]; then
+  mv "$OUT" "$OUT.REJECTED"
+  fail "integrity_check on snapshot: $CHECK (kept as $OUT.REJECTED)"
+fi
+
+# Prove it is a coord db and not an empty file that passed integrity_check.
+# Same "keep it as .REJECTED, never silently delete or silently leave it
+# lying around under the normal name" contract as the integrity_check
+# rejection above -- an unlinked, unmarked snapshot sitting next to good
+# ones is exactly the kind of thing retention pruning would eventually
+# rotate out indistinguishably from a real backup.
+ROWS="$(sqlite3 "$OUT" 'SELECT COUNT(*) FROM assignments;' 2>&1)"
+RC=$?
+if [ "$RC" -ne 0 ]; then
+  mv "$OUT" "$OUT.REJECTED"
+  fail "snapshot has no assignments table: $ROWS (kept as $OUT.REJECTED)"
+fi
+case "$ROWS" in
+  ''|*[!0-9]*)
+    mv "$OUT" "$OUT.REJECTED"
+    fail "unexpected assignments count: $ROWS (kept as $OUT.REJECTED)"
+    ;;
+esac
+if [ "$ROWS" -eq 0 ]; then
+  mv "$OUT" "$OUT.REJECTED"
+  fail "snapshot has 0 assignments — refusing to count this as a backup (kept as $OUT.REJECTED)"
+fi
+
+ln -sfn "$OUT" "$DEST_DIR/coord.db.latest"
+
+# Prune oldest beyond RETAIN. Never touches .REJECTED files.
+mapfile -t OLD < <(ls -1 "$DEST_DIR"/coord.db.2* 2>/dev/null | grep -v '\.REJECTED$' | sort | head -n -"$RETAIN")
+for f in "${OLD[@]:-}"; do [ -n "$f" ] && rm -f "$f"; done
+
+SIZE="$(du -h "$OUT" | cut -f1)"
+echo "coord-db-backup: ok $OUT ($SIZE, ${ROWS} assignments, $(ls -1 "$DEST_DIR"/coord.db.2* 2>/dev/null | grep -vc '\.REJECTED$') snapshots retained)"

coord/deploy/coord-db-backup.timer

@@ -0,0 +1,18 @@
+# Fires coord-db-backup.service hourly. See that unit's header for the full
+# rationale, install steps and safety guarantees.
+#
+# Persistent=true: if the daemon host was down/rebooted when a fire was
+# missed, the next boot catches it up immediately rather than waiting up to
+# an hour for the next scheduled tick.
+
+[Unit]
+Description=Hourly coord.db snapshot to the external SSD (interim, pending #1822)
+
+[Timer]
+OnBootSec=5min
+OnUnitActiveSec=1h
+AccuracySec=5min
+Persistent=true
+
+[Install]
+WantedBy=timers.target