@tikoci/quickchr 0.4.7 → 0.5.0

package/DESIGN.md

@@ -97,6 +97,19 @@ Modules (src/lib/)      ← qemu, images, versions, network, state, ...
 
     ⚠️ **Testing note.** `Bun.serve` **drops an explicit `content-length` when the body is a `ReadableStream`** and frames the response `transfer-encoding: chunked`, so a stub built on it cannot exercise the size-derived budget at all — every response looks unknown-length. The anchor tests use a raw `Bun.listen` HTTP stub for that reason.
 
+12. **Two locks, one primitive** (`src/lib/lock.ts`; #140, #181, #180).
+    - **Machine lock** — `machines/<name>/.lock/`, so it goes with the machine. One mutex for every lifecycle operation: `add`, `start`, relaunch, `setDeviceMode`, `stop`, `clean`, `remove`, orphan removal. It was `.start-lock` and taken only by starts, so `stop`/`clean`/`remove` raced every relaunch. For one release a live `.start-lock` is still honoured; a dead one is ignored, not deleted.
+    - **Allocation lock** — `<dataDir>/.allocate.lock/`, held across read → probe → claim when a machine is created. Without it two creations probe the same free ports and pick the same block. The claim itself has to outlive the lock, because `start()` persists `machine.json` only after the download and the spawn, so the creator's machine lock carries `{ portBase, macs }` as a reservation and `listReservations()` reads it. A reservation ends with its lock, and dies with a crashed creator's pid.
+    - **Who waits.** `stop`/`clean`/`remove` wait, bounded by the machine's boot budget, so a `stop` issued during a `start` runs after it. A second `start`, `setDeviceMode` and orphan removal fail fast with `MACHINE_LOCKED`: waiting would only boot the machine twice, or wait out a create that proves the directory is not an orphan. Nobody waits for a `foreground` holder, which keeps the lock until its VM exits. `remove` has no override for a live holder: deleting a directory under a live spawn is the bug the lock exists to prevent. A broken machine has no live holder, so the lock never blocks clearing it.
+    - **Not reentrant.** Two async operations in one process conflict as much as two processes do, so a lock held by this pid counts as held. Code inside a locked operation calls `stopUnderLock()`/`removeUnderLock()`/`cleanUnderLock()`, never the public `ChrInstance` methods.
+    - **State is re-read under the lock.** A handle's snapshot predates whatever the previous holder did, and a stale `state.pid` is a process that has exited or one the OS has reused.
+    - **An append-only log, so nothing is deleted from under a holder.** A lock is a directory of numbered records, and the highest decides the state: a live holder means held; a released marker, a dead holder's record or nothing means free. Acquiring creates record top+1 with `link()`, so exactly one contender wins and the file never exists half-written. Releasing appends a released marker, and records below the top are collected. A claim stands only if nothing higher exists and its predecessor is still there, which turns away a contender that slept through other claims, or through a remove and re-create of the machine.
+    - **Why not delete a stale lock?** A path has no compare-and-delete. Every deleting recovery tried here had a window between judging a holder dead and deleting its file, in which a successor could be installed and then deleted: rename-aside-and-compare, then a reap guard recovered by age. The Codex review of #190 reproduced both. Taking over a dead holder is now just claiming the next number. A cross-process stress run (6 processes, 360 critical sections, about 15% simulated crashes taken over mid-run) showed no overlap.
+    - **Records are named for the lock's incarnation, so a log can be deleted.** A machine's lock lives in `machines/<name>/` and goes with it. A machine can be removed and created again under the same name while some process is paused holding paths into the old lock, so every record is named `<id>.<gen>`, with an id published once per lock directory. A stale process cannot reach the new incarnation's records: not with a claim, a collect, a rejected-claim cleanup or a released marker. Codex reproduced two such cases while records were named by number alone: a claim matching the new record 1, and a collect run after the released marker that deleted the new holder's record. Before that, the logs were briefly pushed out to a never-deleted `locks/<name>/`, which kept a file for every name ever used.
+    - **A claim stands only on what it judged, and a holder cleans up while it still holds.** After creating its record, a claimer checks three things: nothing higher exists, its predecessor is byte-for-byte the record it judged free, and its id is still the directory's incarnation. Release collects old records *before* publishing its marker, because from that moment it is no longer the holder and nothing may be deleted by path. A released lock therefore keeps two records: the last holder's record and its marker.
+    - **`remove()` renames, then deletes, with no in-place fallback.** `removeMachine()` moves `machines/<name>/` to `<dataDir>/.removing/` in one step before deleting it. A recursive delete in place has a window where `.lock/` is gone but the directory is not, and a waiter or same-named create could take a fresh lock in a directory being deleted. A rename that fails is retried for about 2 s, which covers a scanner holding a handle on Windows. After that, `remove` fails with `STATE_ERROR` and leaves the directory in place, rather than falling back to the unsafe delete (Codex review of #190). Once the rename succeeds the name is free, so a failed delete only warns, giving the staged path. Each removal sweeps `.removing/` for entries left by a delete that failed or a process that stopped after its rename, skipping any whose pid is still alive in another process. A waiter never re-creates a removed machine's directory, because the lock directory is created with a non-recursive `mkdir`: it gets `MACHINE_NOT_FOUND`. `assertSameMachine()` catches one that waited out a remove and re-create.
+    - **Unreadable means held.** A record or legacy `.start-lock` that cannot be read, or that does not parse and is younger than 500 ms, blocks. It is never treated as absent.
+
 ## Port Layout
 
 | Offset | Service    | Guest Port |
@@ -177,6 +190,14 @@ See MANUAL.md's CLI reference and environment-variables sections for the full su
 Full artifact map, dispatch recipes, and failure-diagnosis guide live in
 `.github/instructions/ci.instructions.md` — this section is the high-level rationale only.
 
+**The toolchain is part of the repository (#148).** `.bun-version` pins the exact Bun that
+CI, integration and release all install, enforced by `scripts/check-bun-pin.ts` in
+`bun run check`. Before this, `setup-bun` was called with no version anywhere, so the tested
+runtime could change between two identical commits — and did: a six-week gap with no `ci.yml`
+runs spanned the Bun 1.4.0 release, and the next PR inherited a 1.3.14 → 1.4.2 jump as if it
+were its own regression. Anything that can turn the build red is a dependency and belongs in
+a reviewable diff; "whatever is newest at run time" is not a version.
+
 **The 2026-07 refactor (#29)** replaced the organically-grown scheme (integration on every
 PR push, three parallel copies of the runner logic, `continue-on-error` green-washing)
 with a layered design:
@@ -208,7 +229,9 @@ with a layered design:
 - **`release.yml`** — one-click publish from committed release state: freshness gate
   (no suite re-run — main is kept continuously release-able) + `package.json` version
   with a matching non-empty CHANGELOG section → tag + GitHub Release + `npm publish
-  --provenance` (odd minor → `next`, even → `latest`). CI never bumps versions or pushes
+  --provenance` (odd minor → `next`, even → `latest`; a stable release is then also
+  tagged `next`, since there is no separately maintained pre-release line and `next`
+  must not point at something older than `latest`). CI never bumps versions or pushes
   to protected `main`.
 - **`ros-versions.yml`** — daily: new RouterOS versions (per channel) with no linux-x86
   record in `ci-data/tested-versions.json` ride the `routeros-targets` matrix of a
@@ -347,9 +370,17 @@ boot went silent:
    paths (#106).
 
 Corollary for CI cache keys: `actions/cache` only saves when the primary key
-*missed*, so a static key means a newly-resolved RouterOS version re-downloads
-every run forever (#91). Primary keys must carry a rotating component, with
-`restore-keys` doing the prefix fallback.
+*missed*, so the concrete resolved RouterOS version belongs in the key (#91).
+The writable owner must prefetch and verify a declared image-and-package
+manifest before saving; saving opportunistic test byproducts makes a red or
+lost suite permanently cold, while saving only images makes immutable entries
+permanently package-incomplete. Acquisition is therefore a named pre-test step,
+restore-key extras are reconciled away before save so auto-prune cannot evict
+fixed fixtures later, and a manifest change bumps the cache-key generation.
+Raw images must have in-bounds DOS partition extents; package directories are
+published atomically with an exact filename/size manifest. Tests boot the same
+concrete pin named by the immutable key rather than resolving the channel again
+(#144).
 
 7. **The boot probe can break the service it is waiting for.** `restGet()`
    implements its deadline as `req.destroy()` — a mid-flight TCP teardown. On
@@ -455,6 +486,48 @@ The `default-route-distance=2` ensures the shared route is backup — SLiRP ethe
 
 **TCG hazard:** SLiRP half-open connections (TCP connect succeeds, data never flows) burn the full per-probe HTTP timeout in `waitForBoot`. Under cross-arch TCG where TCP round-trips are slow, this compounds badly. Lab: `test/lab/slirp-hostfwd/`.
 
+#### NIC MAC Addresses — Stable Per Machine, Assigned Once
+
+QEMU's default NIC address is `52:54:00:12:34:56`, incremented per NIC **within** a
+guest. NIC index *N* therefore holds the same address in *every* guest. That is
+harmless while machines only have `user` NICs (each SLiRP netdev is its own
+namespace and nothing is shared), and fatal the moment two machines share an L2
+segment — a named socket, a `socket:listen`/`socket:connect` pair, a TAP, or a
+RouterOS bridge on a hub VM. Forwarding then half-works, and every instinct points
+at RouterOS or the overlay rather than at the launcher (#154).
+
+quickchr assigns each NIC a locally-administered address: `02:` (bit 1 set =
+locally administered, bit 0 clear = unicast, so it can never collide with a
+vendor-assigned address) followed by five octets of a SHA-256 over the machine
+name, NIC index and a collision salt. Deriving rather than randomizing means a lab
+rebuilt under the same machine names presents the same addresses. The address is
+persisted in `machine.json` and `deriveMac()` is consulted only when a NIC has
+none, so a later change to the derivation cannot move an existing machine's
+addresses.
+
+**Assigned at creation only — never backfilled onto an existing machine.**
+RouterOS ties its persisted interface identity to the MAC. Change it and the guest
+treats the NIC as new: the `ether1` holding the DHCP client is orphaned, no address
+is obtained, and hostfwd accepts on the host side while every guest SYN is dropped
+— the same half-open signature as the SLiRP/ether1 hazard above, from a different
+cause.
+
+Grounded by A/B against a live CHR (7.24.2 x86, HVF), same machine and same disk:
+
+| MAC | Result |
+|---|---|
+| original | boots, REST ready |
+| changed | boot fails — 36 probe-timeouts over 178 s, all forwarded ports `dropped` |
+| restored | boots, REST ready |
+
+So a machine created before MACs existed keeps QEMU's defaults for its lifetime and
+must be recreated to get a stable address. Silently rewriting a working lab's L2
+identity on upgrade would be the worse trade.
+
+`deviceArgs()` in `network.ts` is the single place a `-device virtio-net-pci` is
+emitted; `qemu.ts`'s fallback path builds its own `-netdev` strings but calls into
+it, so a NIC cannot reach QEMU without its address.
+
 #### Host-Side L2 Capture (MNDP, MAC-Telnet)
 
 A caller can receive the guest's raw Layer-2 frames — RouterOS MNDP (UDP/5678
@@ -477,7 +550,7 @@ capture on any platform. Evidence: `test/lab/mndp/REPORT.md`.
 The dual of `hostfwd`: SLIRP's gateway `10.0.2.2` *is* the host from inside the
 guest, so guest-originated UDP to `10.0.2.2:<port>` reaches a host socket bound on
 loopback `<port>` with **no forward and no extra NIC**. This generalizes the TZSP
-path (`tzspGatewayIp`/`captureInterface`) — it is not TZSP-specific and reaches an
+path (`hostGatewayIp`/`captureInterface`) — it is not TZSP-specific and reaches an
 ordinary bound socket, not just a `tshark`/pcap capture. The host socket **must be
 unconnected**: SLIRP re-emits from a rewritten loopback source
 (`127.0.0.1:<ephemeral>`), which a `connect()`-ed socket would filter. This closed
@@ -519,13 +592,57 @@ Always read back what we write. One extra REST API call after a provisioning act
 
 ### Provisioning Scope
 
-quickchr provisions at first boot and (optionally) on restart:
+quickchr provisions at a machine's **first boot** — see "The provisioning window"
+below for why, and for what happens to an option that arrives later:
 - **User creation** — create user, set password, optionally disable admin
 - **Package install** — SCP `.npk` files, reboot to activate
 - **License** — `/system/license/renew` for trial
 - **Device-mode** — `/system/device-mode/update mode=rose container=yes ...` for restricted features (containers, traffic-gen, routerboard). Opt-in: not configured unless explicitly requested via CLI `--device-mode` or API `deviceMode` option. CHR ships with `mode=advanced` which is sufficient for most use cases. Device-mode requires a hard QEMU power-cycle to confirm changes — this is the MikroTik-mandated confirmation mechanism (physical button press on real hardware, cold reboot on VM). The wizard defaults to `rose` when the user opts in, since it enables containers. See: https://help.mikrotik.com/docs/spaces/ROS/pages/93749258/Device-mode
 - **Config import** — planned: load `.rsc` or `.backup` at creation time
 
+#### The provisioning window — and why `lastStartedAt` could not be the gate
+
+Every provisioning step assumes the guest holds RouterOS' default configuration, so
+provisioning runs in a window that closes at **first boot** — not at `add`. A machine
+created with no provisioning flags can still take them on its first `start`, which is
+the behaviour agents actually rely on.
+
+The gate was `!state.lastStartedAt`, and that field does not mean what the gate needed.
+It is stamped immediately after `spawnQemu`, before the guest is known to have booted
+and before provisioning runs, so it records "QEMU has been launched" rather than "the
+guest has been configured". The two agree most of the time, which is why the
+conflation survived — and come apart in three ways, each a defect (#176):
+
+- **`clean()`** replaces the disk with the factory image but left `lastStartedAt`
+  behind, so a genuinely fresh guest could never be provisioned again. `clean()` now
+  clears the gate along with the credentials it already cleared, keeping `cleanedAt`
+  for the forensic clue. A cleaned machine's next start re-applies the retained
+  provisioning *intent* and costs what a first boot costs.
+- **Provisioning that throws after a successful boot** leaves a running,
+  half-provisioned machine with the gate closed. The record is stamped only on
+  success, so this case is at least *visible* now; making it retryable — re-running
+  only the steps that did not land — is the remaining half of #176 and is what
+  `provisioning.steps` exists to support.
+- **A first start with no provisioning options** closed the gate although nothing had
+  been configured.
+
+`MachineState.provisioning` is the positive record: when provisioning last completed
+and which steps ran. `isProvisioningWindowOpen()` reads it and falls back to
+`lastStartedAt` for machines that predate the field, so no existing machine
+re-provisions itself on its next start. This is the desired-config/runtime-state split
+of #58, applied to the one field where the conflation was already producing bugs
+rather than to `machine.json` wholesale.
+
+**Nothing may drop a provisioning option in silence.** `start()` used to pass
+`undefined` in place of the caller's options on every path but a first boot, so all
+seven were discarded with no warning, no error and a normal-length boot — the only way
+to notice was to read the value back. An option outside the window now throws
+`PROVISIONING_WINDOW_CLOSED`, naming each option and the route that still works
+(`quickchr set <name> --license`, the library's `setDeviceMode()` / `installPackage()`,
+or `clean()` for anything else). An option state already records is treated as a
+no-op, so passing the same flags on every start — the shape a script naturally takes —
+is not an error.
+
 Provisioning via REST API is preferred (simple HTTP calls). Serial console provisioning (prompt detection + buffer tracking, as in chr-armed) is a fallback for locked environments. Key lessons from chr-armed serial work: use `\r` not `\r\n` on PTY; accumulate buffer with offset tracking to prevent re-matching; detect prompts dynamically, don't use fixed delays.
 
 ### Exec Transport Design
@@ -592,6 +709,383 @@ the CLI/library surface is tracked in
 
 A recurring lesson from downstream agents (donny, centrs, restraml): the friction is usually **finding** an existing capability, not a missing one. Issue #18 (centrs) nearly rebuilt UDP forwarding, `socket-connect` L2, and the guest→host gateway — all of which already existed — because the only way to choose among them was reading `src/lib/network.ts`. Rule: **every CLI-documented capability also needs a library-facing, by-goal surface** — JSDoc at the call site, a by-goal recipe (`docs/networking-recipes.md`), and coverage in the `routeros-quickchr` skill — or agents won't find it. Connection handoff for harnesses goes through `ChrInstance.descriptor()` / `quickchr inspect` / `quickchr env` (credential-bearing by design), never by reading `machine.json`. The structured, versioned **descriptor v1** contract that centrs consumes for `--quickchr` targets — its shape, per-service semantics, scope boundaries, and staged implementation plan — is specified in [`docs/centrs-interface.md`](./docs/centrs-interface.md) (issue [#71](https://github.com/tikoci/quickchr/issues/71)).
 
+### Tips — pointing at the better path without becoming chrome
+
+`quickchr exec` is a raw `/rest/execute` pipe: whatever the caller types is what RouterOS is asked to run, and quickchr has no opinion about whether the command is well-formed. centrs does — it checks a command's shape against the device's own command tree before running it and has per-path help — so `centrs --quickchr <name> <command>` is the better path for anything more than a one-liner. An agent that does not know this reads quickchr's source to find out, which is the discoverability failure described above, one layer up.
+
+The answer is a **tip**: one dim line, at the moment the better path applies. Three rules keep tips from turning into noise (`src/cli/tips.ts`):
+
+1. **stderr, always.** stdout is the command's result and must stay parseable — `quickchr inspect --json | jq` cannot be made to filter advice out of its own input.
+2. **A tip names a next command or a better-suited tool.** It never restates what just happened, and never appears where the command already said the thing.
+3. **Suppressible via `QUICKCHR_NO_TIPS=1`, and not gated on TTY.** The audience for "you probably wanted X" is agents and first-time users, and both usually run without one.
+
+### A mistyped or failed command leaves no state
+
+Three CLI defects reported from one external agent's 3-CHR lab were the same defect: quickchr turned a mistake into durable state and then could not describe it.
+
+- `--help` reached subcommands as an ordinary argument, so `add --help` created a machine and `networks sockets create --help` persisted a socket named `--help`. Help is now answered in the dispatcher, before dispatch, so a new subcommand cannot miss the guard.
+- An unrecognised flag on `add`/`start` was inert, so a typo downloaded 43 MB and created a machine. Those two commands — the ones that create machines — now check their flags against a registry (`src/cli/flags.ts`) first. The read-only commands stay tolerant.
+- A failure between `ensureDir()` and `saveMachine()` stranded a directory that `list` could not show, `remove` would not delete, and `add` still refused to reuse. `add` now cleans up after itself, and — because a SIGKILL still can strand one — `remove` clears a directory with no readable `machine.json`, `MACHINE_EXISTS` distinguishes the two cases, and `doctor` names `quickchr remove`, not `rm -rf`.
+
+The rule the three share: **clearing quickchr's own mess must never require knowing that the data dir exists.** A recovery step whose only form is `rm -rf ~/.local/share/quickchr/machines/<name>` is not a recovery step; it is a leak of an internal layout into the user's hands. The same rule is why machine and socket names are validated before anything is written — both become a path segment, so "what names are legal" is a storage-layout question the user should never have to answer.
+
+### A flag's arity is declared, not guessed
+
+`parseFlags` had one rule for two jobs: *if the next argument does not start with `--`, consume it as this flag's value.* That is right for `--version 7.24.3` and wrong for every boolean flag, which then eats the following positional — `quickchr start --vmnet-shared lab` set `vmnet-shared="lab"` and lost the machine name (#164). Arity now comes from `VALUE_FLAGS` in `src/cli/flags.ts`; a flag absent from that table is boolean and leaves the next argument alone. A typo is boolean too, which is why a misspelled `--version` followed by `lab` can now be reported as an unknown flag *with* the machine name intact, instead of silently consuming it.
+
+The audit matters as much as the table. `--vmnet-shared` was found by accident while fixing something else, and the only way to find its siblings was to re-read every `flag()` call site by eye. `test/unit/cli-flag-arity.test.ts` scans the CLI source for every flag read and cross-checks each name against the registry in both directions — so the next flag cannot skip the table by being forgotten, and nobody has to do that reading again. Writing it paid for itself immediately: the first version scanned only the `flag()`/`flagBool()` helpers, pronounced the registry complete, and missed three `cache prune` flags read straight off the object (`flags["older-than"]`), which the arity change had already broken. An audit is only as wide as the ways the code actually reads a flag.
+
+`--vmnet-shared` / `--vmnet-bridge <iface>` were removed rather than repaired: they were special cases for one network type that `--add-network` already expresses in general, both were already documented as deprecated aliases, and nothing generated them. A removed name is still recognized and errors with its replacement — deleting it outright would leave the flag "unknown", and for the tolerant read-only commands, ignored again. `vmnet-shared` is simultaneously the removed flag, QEMU's own netdev name, and a legacy `machine.json` value; only the first went, and a test pins the other two because the collision makes over-deletion easy.
+
+### An enumeration and a lookup fail differently
+
+`loadMachine()` parsed `machine.json` with no guard, so one corrupt file — a `saveMachine()` that died mid-write — threw a raw `SyntaxError` out of `quickchr list` and destroyed the state of every healthy machine alongside it (#165). One policy was being applied to two contracts:
+
+- **`get(name)` is a targeted lookup.** You named *that* machine, it is corrupt, and answering `null` would report a directory that exists as one that never did. It still throws — now `STATE_ERROR` naming the file and `quickchr remove <name>`, rather than a parse trace.
+- **`list()` is an enumeration** — best-effort across many. One bad entry aborting it is the enumeration failing at its own job, and it costs more signal than it keeps.
+
+Skipping alone would have traded a loud failure for a quiet one: a machine that was there yesterday just disappears, which is the shape of trouble that does *not* look like trouble. So the unreadable entry keeps its place in `list` as a row — marked, named, with the remedy attached — because it still holds a disk image and `list` is where anyone looks first. Deferring it to `doctor` assumes you already suspect something. Exit stays 0: the listing succeeded, and a non-zero exit would punish the working machines and break every script piping `list`.
+
+"Unreadable" is not the same as "unparseable", and review caught the first version confusing them: it guarded `JSON.parse` and rejected primitives and arrays, so `{"name":"lab3"}` was still cast to `MachineState`, still enumerated, and still reached `formatPorts(m.ports)` — `Object.entries(undefined)`, the same wholesale abort with a different stack. `loadMachine()` now checks the fields callers dereference unguarded (`REQUIRED_STATE_FIELDS`). That list is deliberately not a schema: it is the set that turns a bad file into a crash somewhere else, and every field in it has been required on `MachineState` since the commit that introduced the type, so it cannot condemn state any version of quickchr wrote. `cpu`/`mem` are left out precisely because a missing one prints oddly rather than throwing.
+
+Internally this is a three-way `tryLoadMachine()` (`ok` / `missing` / `unreadable`) that `loadMachine()` still collapses into "undefined or throw". Keeping the discriminated result unexported is the small diff; promoting it to public API is a rename if #58 lands.
+
+### A named socket says what it is
+
+An external agent's 3-CHR lab brought up two CHRs on `--add-network socket::skylab-mir`,
+got interfaces up, addresses assigned, and 100% packet loss with no error anywhere. It
+found the cause by opening `machine.json`: the link was UDP multicast, and the sandbox
+denied the send.
+
+The lab re-ran the question against its own VM on 2026-09-20, which turned the original
+hand-wave ("the sandbox blocks UDP") into something exact — and narrower:
+
+- The host is **Linux** (Ubuntu 24.04, seccomp-bpf active, `NoNewPrivs=1`, four filters).
+  `iptables` and `nft` are empty, so no firewall is involved. **This lab was never on
+  macOS**, which is why its finding is independent of the macOS defect below.
+- The denied call is **an unconnected `sendto()`/`sendmsg()` on an `AF_INET` `SOCK_DGRAM`
+  socket**, returning `EPERM` — strace-confirmed. Not "UDP". A `connect()`-ed UDP socket's
+  `send()` passes the filter, as does TCP, as does `AF_UNIX` `SOCK_DGRAM`.
+- `socket()`, `bind()` and `IP_ADD_MEMBERSHIP` all succeed; only the send is refused. So
+  the group joins cleanly and then silently carries nothing.
+- Loopback multicast fails **independently** of internet egress: `sendto()` to
+  `230.0.0.1:4000` gets the same `EPERM` as `sendto()` to `8.8.8.8:53`. It is not an
+  egress-only policy, which was the intuition that had to be tested rather than assumed.
+
+That maps onto one line of QEMU, which is why `mcast` died there and `dgram` did not
+(`net/socket.c`, `net_socket_receive_dgram()`):
+
+```c
+ret = RETRY_ON_EINTR(
+    s->dgram_dst.sin_family != AF_UNIX ?
+        sendto(s->fd, buf, size, 0, (struct sockaddr *)&s->dgram_dst, sizeof(s->dgram_dst)) :
+        send(s->fd, buf, size, 0)
+);
+```
+
+`mcast` takes the `AF_INET` branch — the exact call the filter denies. `dgram` over unix
+takes `send()`. `EPERM` is neither `EINTR` nor `EAGAIN`, so the frame is dropped and
+nothing is logged: the 100% loss, mechanically.
+
+One thing the lab explicitly withdrew: the failed 7.24.4 image downloads in the same
+session were **TCP egress-proxy stalls**, not UDP. Do not cite them as evidence here.
+
+Three separate defects converged on that one silence.
+
+1. **The only transport the CLI could reach was the one that fails silently.**
+   `createNamedSocket()` defaulted to `mcast`, and `networks sockets create` had no flag
+   to pick anything else.
+2. **`listen-connect`, the alternative the registry already modelled, had never worked.**
+   The role came from `entry.members.length === 0` at resolve time, but
+   `registerSocketMembers()` runs *before* `resolveAllNetworks()`, so the starting
+   machine had always added itself by then. `isFirst` was never true; every member
+   resolved to `connect=`; nobody ever listened.
+3. **Nothing printed the transport.** `start` did not name it, and `format.ts` tested
+   `s.type === "socket-named"` while the parser emits `{ type: "socket" }`, so `list`
+   and `info` fell through to `JSON.stringify` for every named socket.
+
+#### What the transports actually do
+
+Measured on QEMU 11.1.1, macOS Intel, CHR 7.24.4, rather than reasoned about — except
+the `listen-connect` column, measured later on `windows-x86`/TCG, CHR 7.24.4 (see below):
+
+| | `mcast` | `listen-connect` | `dgram` over unix |
+|---|---|---|---|
+| start order | any | listener first | **either end first** |
+| frames verified | broken on macOS | ICMP both ways | ICMP both ways, plus MNDP and IPv6 ND |
+| host ports | a multicast group | a TCP port | none |
+| send call | unconnected `sendto()`, `AF_INET` | TCP `send()` | `send()`, `AF_UNIX` |
+| machines | any number | 2 | 2 |
+| where a failure lives | host kernel / sandbox policy | quickchr's own registry | quickchr's own registry |
+
+`listen-connect` read `(was unreachable)` in that row until #172. It is the Windows
+default, so it is the one transport whose first end-to-end proof had to come from CI:
+run [35545697945](https://github.com/tikoci/quickchr/actions/runs/35545697945) created
+the link with no `--mode` at all, started slot 0 and slot 1 in that order, and pinged
+both ways over it. The QEMU side of the pair is not exotic — `network-mac.test.ts` had
+long covered the raw `socket-listen`/`socket-connect` specifiers on Windows — but the
+named-socket path around it (persisted mode, slot roles, auto-allocated port) had never
+been exercised on a real boot anywhere.
+
+The decisive point is not "TCP works on macOS" — it is **which silence quickchr can
+see.** A TCP pair does not fail loudly either: `connect=` to a dead port runs on with
+no error, no log and no exit, never retries when the listener appears later, and
+`listen=` accepts exactly one peer while leaving the listening socket open, so a third
+member's connect succeeds at the kernel level and receives nothing. But those failures
+are member count and endpoint occupancy, which are already persisted. A multicast
+failure lives in the host kernel, and no amount of bookkeeping can see it.
+
+#### The decision
+
+**A named socket defaults to `dgram` on macOS and Linux, and `listen-connect` on
+Windows** (AF_UNIX there has no `SOCK_DGRAM`). `mcast` stays first-class as the only
+N-way segment, and says at create time that it fails silently.
+
+The resolved default is **persisted at creation, never re-derived**. A
+`networks/<name>.json` states the transport it was made with, so one specifier never
+silently means two things — the objection that ruled out a platform-sniffing default
+in the first place.
+
+Two dgram failure modes the design has to own, both found by probing rather than
+reading:
+
+- **A duplicate `local.path` silently steals the link.** A second QEMU unlinks the
+  first's socket and binds its own: the inode changes, both processes stay alive, and
+  neither logs anything. This is why a third machine is a refusal rather than a
+  warning, and why endpoints are claimed before QEMU is spawned.
+- **`sun_path` is 104 bytes.** Checked at create time so the error names the data dir,
+  instead of arriving later as `UNIX socket path '...' is too long` against a path the
+  caller never chose.
+
+Claiming an endpoint is a cross-process read-modify-write, so it takes a **registry
+lock** and re-reads from disk inside it. Without one, `quickchr start a & quickchr start
+b &` — the shape the field report actually used — has both machines read `[null, null]`,
+both take slot 0, and the second write win: both QEMUs get the same `local.path`, and
+the duplicate-path theft above does the rest. A two-process repro lost an endpoint on
+12 of 12 rounds. Entries are also written with write-then-rename, because a truncating
+write let a concurrent reader see invalid JSON and report the socket as missing.
+
+The lock itself had the same class of bug on its first cut: a waiter that found the
+lock file created but not yet written to judged it stale and deleted it, so two
+processes believed they held it. A lock file whose pid is unreadable is now given a
+grace period before it counts as abandoned, which still reclaims a crashed holder.
+
+Endpoints are held in **numbered slots** rather than derived from live membership.
+Slot 0 is the listener (`listen-connect`) or endpoint A (`dgram`); the first machine
+to start has to name its peer's path before that peer exists, and a slot is the only
+thing both ends can agree on in advance. It also fixes (2) properly: a listener that
+stops and restarts comes back as the listener instead of demoting itself to a second
+connector.
+
+A consequence worth stating, because it is easy to document wrongly: the roles are
+fixed when the two machines *first* join, so `listen-connect` still has a start order
+afterwards — the listening machine must be up before the connecting one, since
+`connect=` fails silently and never retries. `dgram` has no ordering constraint at any
+point. A claim is also released if the launch that made it never reaches persisted
+state, so a start that fails after claiming an endpoint does not leave a link occupied
+by a machine that is not running. The invariant is the plain one — **a machine that is
+not running holds no endpoint** — and every path that reaches "stopped" has to honour
+it, including the two that do not go through `stop()`: a foreground run whose QEMU has
+exited, and `clean()`.
+
+The registry needs a second, wider lock as well. Automatic port allocation is
+`max(existing ports) + 1`, which reads *every* entry, so the per-entry lock does not
+help when the contenders are different names — three `mcast` links created at once all
+took port 4000 (15 of 15 rounds), silently collapsing three segments into one group.
+The registry-wide lock is always taken *inside* the per-entry one, so the order is
+fixed and two holders cannot deadlock.
+
+The lock alone is not enough, though: allocation also has to *see* the current
+registry. `listNamedSockets()` seeds itself from the in-memory cache and only reads
+disk for names it does not already hold, so a cached entry whose on-disk port has
+changed is never re-read — and the next allocation hands out a port that is taken.
+Reading only from disk is not the fix either, because that cache exists to work around
+Bun returning stale data from a read that follows a write on Windows. The two sources
+go stale in opposite directions, so allocation takes the **union** of both: it can only
+over-estimate the maximum, which wastes a port number, where trusting either one alone
+under-estimates it and collides.
+
+The acceptance bar, and the reason the visibility work is not cosmetic: **nothing about
+a named socket should require opening a file under the data dir.** Every place the
+field report had to read `machine.json` is a place quickchr knew the answer and did not
+print it. The same reasoning applies to the CLI's own options — a `--group` that cannot
+apply is an error rather than a silently dropped argument.
+
+#### Still open
+
+`dgram` and `listen-connect` both cap at two machines, and `mcast` — the N-way answer —
+is broken on macOS. **quickchr still has no 3-node rootless L2 segment on macOS.** That
+was true before this change; it is now explicit rather than presenting as a RouterOS
+fault. Tracked in #167.
+
+The in-repo alternative is a frame-repeating hub, and since `start` returns and `--bg` is
+not a detach (#159), there is no process to host one — a hub is a daemon, and "no daemon"
+is a standing design decision. So the work goes into making `mcast` usable instead, via
+two independent defects.
+
+**Ours:** quickchr never passes `localaddr=`, so `net_socket_mcast_create()` falls back to
+`imr_interface = INADDR_ANY` and sets no `IP_MULTICAST_IF`. The group therefore rides the
+host's default multicast interface rather than loopback. `IP_MULTICAST_LOOP` governs local
+delivery, not scoping. A `dgram` link cannot leave the filesystem; an `mcast` one does.
+The consequence — two people on one LAN using the default group joining each other's
+segments — is **a reading of the code plus the lab's `netstat -gn`, not a demonstrated
+two-host repro**; #167 step 1 is the probe that settles it.
+
+**Upstream:** `net_socket_mcast_create()` sets only `SO_REUSEADDR`, on a stated assumption
+that this suffices to share a multicast port — true on Linux, false on BSD/macOS, which
+require `SO_REUSEPORT` on every such socket. Still that way on QEMU master
+`c1c18d1e640b64292859ce9f30f3c344edfb0294` (2026-09-19). `test/lab/mndp/REPORT.md`
+isolates it to QEMU's side: the host joined on every interface *with* `SO_REUSEPORT` and
+still received nothing, and two CHRs on one group never discovered each other.
+
+**The two are independent, and it matters.** The macOS defect is a *delivery* failure —
+the send succeeds, only one socket receives. The Linux sandbox defect is a *permission*
+failure — the send never happens. They were found in different environments by different
+labs, neither reproduces the other, and fixing either leaves the other intact. Reading
+"mcast is broken" as one story is how the sandbox finding ended up attached to a macOS
+issue in the first place.
+
+**`mcast` stays opt-in either way.** Fixing both removes specific causes of silent failure,
+but neither overturns the asymmetry that chose the `dgram` default — and the sandbox lab is
+itself the cleanest evidence for that default, since `dgram` carried frames in the exact
+environment where `mcast` could not send at all.
+
+The precise form of the asymmetry, corrected: it is **not** that mcast failures are
+undetectable. An active probe at create or start time — one unconnected `sendto()` to a
+loopback group — catches the whole permission class cheaply, including the one that
+actually bit (#169). What a probe cannot catch is the *delivery* class: `SO_REUSEPORT`,
+a downstream firewall, a VPN capturing the group, or another host already on it. The
+durable difference is that a pair transport's misconfiguration is endpoint occupancy —
+persisted, visible, and refused before spawn — whereas every mcast failure mode needs
+an active measurement, and some are not reachable by one at all.
+
+Nor is `mcast` the simpler mechanism it first appears: it removes slots, roles and start
+order, but trades them for group and port allocation, which is where the
+three-links-one-port collapse above came from — and a group collision merges segments
+silently where a slot collision refuses.
+
+One correction on the record, since the earlier draft of this section got it backwards.
+The external lab's sandbox report was marked *unverified*, with the guess that it probably
+only restricted internet-bound UDP. Both halves were wrong: the block is real, it is at the
+syscall, and loopback multicast fails independently of egress. The lesson is the one this
+section already argues elsewhere — the answer came from running the commands on the actual
+host, not from reasoning about what a sandbox is likely to permit.
+
+### Background is not detachment, and `--bg` never claimed to be
+
+`spawnQemu()` backgrounds QEMU and calls `unref()`. That is enough for the parent to
+*exit* — QEMU is adopted by init/launchd and runs on — and it was read as detachment for
+as long as nobody sent a signal. It leaves QEMU in the caller's process group, so anything
+addressed to the group takes the VM with it: Ctrl-C in the terminal, a shell `timeout`, a
+CI step teardown, an agent harness killing a command it thinks is stuck. The external lab
+that hit this worked around it by wrapping every start in `nohup` (#159).
+
+Windows had the right spawn for the wrong-looking reason. A Job Object terminates
+everything inside it when the parent goes, and `Bun.spawn().unref()` does not escape one,
+so that branch already used `node:child_process` with `detached: true` — and carried a
+comment describing precisely the hazard the POSIX branch was exposed to. The two platforms
+fail differently (a Job Object on parent *exit*; a process group on a *signal*), which is
+why the asymmetry survived review: the Windows comment reads as Windows trivia rather than
+as the general rule. It is the general rule, and `detached: true` is now set on both.
+
+The *spawner* still differs, and deliberately. POSIX uses `Bun.spawn`, per the repo's
+Bun-first rule, with its `detached` verified here on Bun 1.4.2 rather than taken from the
+documentation — `test/unit/posix-detach.test.ts` is that verification. Windows keeps
+`node:child_process`, because its Job Object escape is load-bearing and can only be
+demonstrated on Windows. Bun documents `detached` as `UV_PROCESS_DETACHED` there, which
+would probably collapse the two branches into one; that is a change to make from a Windows
+reproduction, not from a doc claim. The first draft of this fix did unify them on
+`node:child_process`, which review correctly called out as reaching for Node where Bun has
+the API — the unification was the easy half, and the wrong half to keep.
+
+Detaching does not weaken cleanup, because nothing that sweeps QEMU sweeps by group —
+`scripts/ci-file-watchdog.ts` and the integration workflow both kill by process name. That
+is worth stating rather than assuming: "the child outlives its parent's signal" and "a
+wedged run leaks VMs" would otherwise look like the same change.
+
+The naming half was not fixed quietly. `--bg` is already the default and only ever meant
+"serial console goes to a log file instead of stdio" — it does not make `start` return
+early, because `start` waits for readiness by contract. Both an external agent and the
+maintainer believed it bought them something. The repair (#159, 0.5.0) is a separate flag
+rather than a redefinition of `--bg` that would change what existing scripts get:
+
+- **`start --no-wait`** (`StartOptions.wait: false`) returns once QEMU is confirmed
+  running — past `spawnQemu()`'s 1.5 s early-exit check, with state and ports persisted —
+  and leaves readiness to the caller. Measured on CHR 7.24.5 under HVF: the CLI returns in
+  ~2 s, REST answers ~17 s later on its own.
+- **Spelled `--no-wait`, not `--detach`.** QEMU *is* detached (above), so `--detach` would
+  name something every start already does. What changes is whether the CLI waits, and
+  `--no-wait` is the other half of centrs' `--wait <deadline>`:
+  `centrs retrieve --quickchr <name> /system/resource --wait 5m` is the documented way to
+  own readiness afterwards, and the CLI prints it as a tip.
+- **Provisioning is refused with `--no-wait`, never skipped.** Provisioning runs against
+  the booted guest, so the pair cannot both be honoured. That includes first-boot intent
+  `add` recorded, which the caller did not retype — the case where skipping would be
+  silent. `--fg` is refused too: a foreground start holds the console until QEMU exits.
+- **`--bg` stays accepted as a deprecated no-op** that warns on stderr, so existing scripts
+  keep working and learn the flag was never doing what they thought. The library default
+  is unchanged: `start()` resolves ready.
+
+### RouterOS 6 is a boot-only target, not supported RouterOS (#182)
+
+A 6.x CHR downloads and boots under quickchr, but the REST readiness probe can never
+succeed against it (6.x has no `/rest`), so `start` ran to its 180 s timeout and then
+deleted a guest that had answered www, ssh and api from 15 s in. The decision is a narrow
+**"track and boot"** contract, for one consumer: centrs needs a disposable 6.x router to
+test the operations an agent needs for a v6→v7 migration (tikoci/centrs#386).
+
+- **Readiness is a native-API `/login`** on guest 8728, built as a general alternative
+  probe (`waitForApiReady()`, chosen by `readinessProbeFor(version)`) rather than a v6
+  special case — a 7.x guest with REST disabled has the same problem. Two consecutive
+  answers; a credential `!trap` counts as an answer, as a REST 401 does. Grounded on CHR
+  6.49.23 under HVF (2026-10-21): empty `admin` gets `!done`; a bad password gets
+  `!trap =message=invalid user name or password (6)` then `!done`; before RouterOS is up,
+  slirp accepts and the probe times out (~20 s), then ready at ~22–25 s.
+- **Not a bare TCP accept.** Under user-mode networking slirp accepts on the host side
+  whatever the guest is doing (see `probeTcpPort()`), so a connect proves nothing.
+- **Everything else is refused loudly, before download where possible**: every
+  provisioning option and the post-boot `set` routes (`PROVISIONING_VERSION_UNSUPPORTED`,
+  checked before the provisioning-window refusal, whose `clean` route would only lead to a
+  second refusal), `arm64` (MikroTik publishes no arm64 CHR for 6.x), and every REST
+  operation (`REST_UNAVAILABLE`, instead of a request that times out). The `secure-login`
+  setting is a default, not a request, so it is not applied to 6.x.
+- **CLI only.** The wizard refuses 6.x — every step it offers is provisioning.
+
+### A failed first boot resets an `add()`-created machine, whatever failed (#182)
+
+#187 made a first boot whose *provisioning* threw reset the machine instead of removing
+it: disk back to the factory image, `machine.json` back to the record `add()` wrote. A
+*boot timeout* on the same first boot still called `remove()`, so the two failure modes of
+one boot disagreed, and the timeout threw away the options `add` recorded. Now
+`_launchExisting()` snapshots the record whenever the provisioning window is open — not
+only when there are provisioning options — and a timeout resets from it. A machine
+`start()` created in the same call is still removed, and one that has booted before is
+unchanged. The first-boot cleanup messages name `QUICKCHR_PRESERVE_ON_FAILURE=1`, because an
+auto-removed machine is what hid #182's report.
+
+### A restart timeout preserves the configured guest (#189)
+
+Boot-timeout cleanup carries explicit intent: remove a machine created by this
+`start()`, factory-reset and restore an existing never-booted machine, or stop and
+preserve an existing machine whose provisioning window was closed. `_launchExisting()`
+classifies from the refreshed authoritative record under the lifecycle lock,
+before launch stamps `lastStartedAt`, PID or status. Supplying a restore record
+for every restart would be destructive too: that branch resets the disk.
+
+Preservation retains disks, credentials, provisioning history and persistent
+configuration, while the shared stop path clears PID/status and releases socket
+memberships. Older records with only `lastStartedAt` remain outside the provisioning
+window. Diagnostics survive; recovery is retry or explicit removal, never an
+implicit factory reset. The environment override still suppresses destructive
+first-boot cleanup. Provisioning rollback is unchanged.
+
+Synthetic launch tests cover all three cases, the override, stale snapshots in
+both directions, disk sentinels and credential retention. The focused live
+`restart-timeout.test.ts` injects a zero readiness-probe budget after a real
+configured boot, exercises real launch/diagnostics/cleanup, then restarts and
+reads back the guest identity with retained credentials. Whole-disk equality is
+only asserted using synthetic sentinels; a live guest can write during boot.
+
 ### Out of Scope (decided)
 
 Explicitly rejected, with rationale, so they aren't re-proposed:
@@ -608,3 +1102,25 @@ Explicitly rejected, with rationale, so they aren't re-proposed:
 ### Document Maintenance
 
 DESIGN.md is the living home for design decisions and rationale. At the end of any significant work session, agents should review whether new implementation details, design decisions, or discovered constraints belong here. Open/close work in **GitHub Issues** — not BACKLOG.md (see CONTRIBUTING.md "Tracking work"); record grounded RouterOS/QEMU behaviour facts in the narrowest scoped doc (`.github/instructions/*.md`, `docs/`, or `test/lab/<topic>/REPORT.md`); add a CHANGELOG.md entry for user-facing changes. Treat this as a lightweight checklist, not a gate.
+
+## Explicit guest shutdown (#143)
+
+`stop()` remains a power cut (QEMU SIGTERM, then SIGKILL), the primary teardown
+primitive and the mechanism used for device-mode confirmation. `shutdown()` is a
+separate bounded operation: one `/system/shutdown` via REST and independent process
+exit observation share a 10-second default grace budget, followed by the existing
+hard-stop escalation if needed. CLI seconds and library milliseconds are explicit;
+the library does not resolve the CLI's environment override.
+
+The HTTP result is diagnostic evidence, not the completion signal. On CHR 7.24.5,
+x86/HVF and arm64/TCG returned HTTP 400 `interrupted` while successfully halting
+(see `test/lab/shutdown/REPORT.md`). No QGA/ACPI/console selection framework is
+needed. Request failure still waits out the grace budget and reports forced
+fallback if QEMU survives. `guest-exited` means no host stop signal was required;
+it does not promise ROSE, container or filesystem durability.
+
+The entire operation holds the existing per-machine lock, refreshes the machine
+before acting, and uses shared stopped-state cleanup without recursively taking
+the non-reentrant stop lock. It confirms QEMU exit before clearing the PID or
+releasing named-network membership. Lock wait, hard-stop escalation and final
+exit verification are separate bounded phases outside the guest grace budget.