cargo-hauler 0.4.4 → 0.4.6

package/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 0.4.6
+
+### Patch Changes
+
+- 4257982: Survive a CLI or plugin upgrade under a daemon left running from the older install, and add `hauler daemon restart`. The client schemas now default the fields older daemons never send (`outputPath`, `after`), so `hauler status`, `hauler result`, `hauler await`, and the `hauler_*` tools read a 0.4.1 daemon's replies instead of printing a Zod issue array; a reply that still cannot be read fails as `daemon is 0.4.1 (pid N, since 3h ago), this CLI is 0.4.4 — restart it with \`hauler daemon restart\`` with the first mismatch (`active[0].outputPath expected string, received undefined`) on a second line, and `hauler daemon status` says the same. The status report carries the daemon `version` (older daemons are asked with one `ping`), `hauler status --json` exposes it as `daemonVersion`, and every document header — plus the status summary — warns `daemon 0.4.2 ≠ cli 0.4.4 — restart it with \`hauler daemon restart\`` while the versions differ. `hauler daemon restart` sends the graceful stop, waits up to 5 s for the old pid to exit, starts a daemon from this install, and prints both pids and versions (`previousPid` in the JSON result; exit `1` when the old daemon did not exit, in which case nothing is started). Tickets in flight at a restart are not handed over: the new daemon marks them `killed` with `orphaned by daemon restart`, and `hauler result` / `hauler_result` explain that instead of showing a plain kill. The shell header ends with `state dir <path>` so a moved `CARGO_HAULER_STATE_DIR` is visible on every command. (#75)
+
+## 0.4.5
+
+### Patch Changes
+
+- 62d4e16: Add `--after <ticket>[,<ticket>…]` to `hauler exec` and `hauler request` (and `after: string[]` on `hauler_request`) to declare explicit ticket dependencies: the request stays `queued` — skipped by lane admission and batch folding — until every named ticket has settled, fails with `prerequisite cc-N failed`/`killed` (exit code `null`) if one of them does, resolves at once when they already finished, and is rejected as a `bad-intent` error naming an unknown ticket. Prerequisites may live in another lane; the ledger stores them (`after` on every request record) and `hauler status`/`hauler result`/await heartbeats show `waits for cc-N (running 2m/~5m)` while a dependent is held. The `exec --bg` and `request` acknowledgements now name the tickets ahead in the lane (`queued behind cc-3281 (~13m)`, `ahead` on the ack and `queue` on the request result) so a cost-ordered reorder is visible, and `hauler request` reports the daemon's rejection reason instead of a generic submit failure. (#45)
+- 1135585: The daemon rejects a path-shaped program that is not cargo (`bad-intent`) instead of running it and recording the path as the subcommand; the dashboard's by-command table shows legacy rows by basename, and the all-time latency tile reads "latency added by attaching" when riders waited longer than their solo estimate instead of showing a negative "saved" value. Dashboard screenshots refreshed.
+- d86d8b6: The shared fifo jobserver is armed only when the host `make` can speak it (GNU make 4.4+, or no make at all); on older makes the daemon falls back to per-run `CARGO_BUILD_JOBS` grants so `-sys` crates whose build scripts run `make` (jemalloc, openssl, …) build again. `CARGO_HAULER_JOBSERVER=fifo|off|auto` overrides the detection (#76).
+- b13db35: Detect stalled tickets and end orphaned ones. The daemon samples every running ticket's process-tree CPU time (Linux `/proc`, macOS `ps`) and flags a run `stalled` once it is past `CARGO_HAULER_STALL_ESTIMATE_FACTOR` (3) times its estimate with no CPU and no output for `CARGO_HAULER_STALL_IDLE_MS` (10 min). `hauler status` / `hauler_status`, the dashboard, and `hauler await` heartbeats show `stalled` with the idle duration; `hauler result` / `hauler_result` answer `ticket looks stalled (no CPU for Nm) — hauler kill cc-N`. A stalled ticket whose submitting connection has disconnected is killed automatically through the `hauler kill` path with the error `stalled: no CPU for Nm after owner disconnected; killed automatically`; `CARGO_HAULER_STALL_AUTO_KILL=0` only flags. Status records gain optional `stall` and `orphaned` fields. (#46)
+- cfb6967: Keep every ticket's whole output on disk and make it readable after the fact. The daemon writes a leader run's combined stdout+stderr, as emitted (rendered diagnostics for demultiplexed runs, ANSI as captured), to `<state dir>/tickets/<ticket>.log`, bounded by the new `CARGO_HAULER_TICKET_LOG_MAX_BYTES` (default 64 MiB, then one truncation line; `0` disables). The ledger gains `output_path`, exposed as `outputPath` on every request record (attached followers carry their leader's path); the startup retention pass removes the logs of pruned rows and any log without a row. `hauler result <ticket>` now shows `Full output: <path> (size)` and `--json` carries `request.outputPath`; `hauler result <ticket> --full` and `hauler_result { ticket, full: true }` render the whole log as the document body (the last ~768 KiB when it does not fit, with the path for the rest). When a synchronous command is auto-backgrounded and the caller's stdout is not a terminal, the exit-75 notice adds that the redirect receives no output and to read it with `hauler result cc-N --full`. (#68)
+
 ## 0.4.4
 
 ### Patch Changes

package/README.md

@@ -41,11 +41,14 @@ src/
 Every rendered route — MCP tool, CLI command, rendered script — composes
 through one layout, the way a page framework's `layout.tsx` wraps every page:
 
-- **Header:** `<DaemonBadge>` prints what the request-start probe proved:
-  `cargo-hauler · daemon running (pid 4021) · 2/5 permits +1 riding, 1 queued
-  · 2 lanes busy · up since 3h ago`, or `daemon stopped · no socket; it starts
-  on demand…`, or `daemon unresponsive · did not accept a connection within
-  750ms (machine saturated)…`.
+- **Header:** `<DaemonBadge>` prints what the request-start probe proved and
+  which state directory it is: `cargo-hauler · daemon running (pid 4021) ·
+  2/5 permits +1 riding, 1 queued · 2 lanes busy · up since 3h ago · state dir
+  /fast/cache/cargo-hauler`, or `daemon stopped · no socket; it starts on
+  demand…`, or `daemon unresponsive · did not accept a connection within
+  750ms (machine saturated)…`. When the daemon is another build than the CLI
+  or MCP server rendering the document, a second line says so: `cargo-hauler
+  · daemon 0.4.2 ≠ cli 0.4.4 — restart it with \`hauler daemon restart\``.
 - **Body:** the route's own document, unchanged. The route keeps its
   `<Agent.Result value>`; the runtime merges it into the shell so
   `structuredContent` and `--json` are exactly what the route declared.
@@ -66,7 +69,7 @@ and a `health` value from one bounded `status` probe:
 
 | `health.state` | meaning |
 | --- | --- |
-| `running` | `pid`, `startedAtMs`, `latencyMs`, `running` (permit holders), `riding` (attached), `queued`, `busyLanes`, `maxConcurrent` |
+| `running` | `pid`, `startedAtMs`, `latencyMs`, `running` (permit holders), `riding` (attached), `queued`, `busyLanes`, `maxConcurrent`, and `version` when the daemon states one (on the status report from 0.4.5, by one extra `ping` for older daemons) |
 | `stopped` | `socket-missing` (starts on demand) or `connection-refused` (stale socket) |
 | `unresponsive` | `accept-timeout` (never accepted), `answer-timeout` (accepted, no `status-result`), or `connection-closed` within the probe budget (750 ms for the accept and for the answer); ledger reads still work |
 | `unreachable` | `open-failed` with the errno (`EACCES`, `EMFILE`, …): the socket is present but could not be opened, which is not evidence the daemon is down |
@@ -90,6 +93,7 @@ assertion share one derivation.
 | `<AdmissionState>` | permits in use, load, memory clamp, sharing savings; calls out a paused admission gate |
 | `<KacheStats>` | kache coverage and freshness, slowest crates by profile, or an honest "not detected" |
 | `<LogTail>` | the captured output tail, labelled live while the run is in progress |
+| `<FullOutput>` | where the ticket's whole output log lives and how large it is; under `full`, the log itself in code-block chunks |
 | `<BuildDiagnostics>` | an index of cargo `error[E…]`/`warning:` blocks (level / code / message / location) followed by every captured block verbatim |
 | `<DashboardLink>` | where the MCP App lives and how to open it elsewhere |
 | `<TicketGuidance>` | what to do next, one component per ticket status |
@@ -135,7 +139,7 @@ the same filter as its `session` field). Results carry
 | `tool:hauler/hauler_log` · `cli:log` | recent requests | `LogStream` → `LogDocument` |
 | `tool:hauler/hauler_last` · `cli:last` | most recent request | `LastDocument` |
 | `tool:hauler/hauler_await` · `cli:await` | long-poll a ticket (≤ 2 h) | `AwaitStream` → `AwaitDocument` |
-| `tool:hauler/hauler_result` · `cli:result` | one ticket, live tail while running | `ResultDocument` |
+| `tool:hauler/hauler_result` · `cli:result` | one ticket, live tail while running; `full` renders the whole on-disk output log | `ResultDocument` (`<FullOutput>`) |
 | `tool:hauler/hauler_kill` · `cli:kill` | stop a queued or running ticket | `KillDocument` |
 | `tool:hauler/hauler_request` · `cli:request` | submit a background request | `RequestDocument` |
 | `cli:daemon` | `run` / `start` / `stop` / `status` | plain JSON, exit code from the result |
@@ -216,15 +220,15 @@ executable beside it (`dist/bin/cargo-hauler.js` in the package,
 
 | Command | Behavior |
 | --- | --- |
-| `hauler exec [--session ID] [--host HOST] [--cwd DIR] [--bg] -- <cargo …>` | Submit Cargo through the daemon and stream output; hooks rewrite commands to this form. A relative `--cwd` is resolved against the caller's directory. Exits with cargo's code; `130`/`143` after a SIGINT/SIGTERM (the ticket is killed first); `75` when auto-backgrounded. |
+| `hauler exec [--session ID] [--host HOST] [--cwd DIR] [--bg] [--after TICKET[,TICKET…]] -- <cargo …>` | Submit Cargo through the daemon and stream output; hooks rewrite commands to this form. A relative `--cwd` is resolved against the caller's directory. `--after` (repeatable or comma-separated) keeps the request queued until every named ticket has finished; it fails with `prerequisite cc-N <status>` if one of them fails or is killed, and an unknown ticket is rejected as a bad intent. Exits with cargo's code; `130`/`143` after a SIGINT/SIGTERM (the ticket is killed first); `75` when auto-backgrounded. |
 | `hauler status [--limit N] [--cwd DIR] [--session ID] [--lane KEY] [--ticket ID …] [--status S …] [--command-contains TEXT]` | Queue, active runs, lanes, admission, kache, optionally filtered. |
 | `hauler log [--limit N]` | Recent requests from the ledger. |
 | `hauler last` | The most recent request. |
 | `hauler await <ticket> [--max-wait-ms N]` | Long-poll until the ticket finishes or the wait expires (default 30 s, ceiling 55 s per call — the rendered-route budget; call again to keep waiting). |
-| `hauler result <ticket>` | A stored ticket; running tickets include a live output tail. |
+| `hauler result <ticket> [--full]` | A stored ticket; running tickets include a live output tail. The document names the full on-disk output log (`Full output: <path> (size)`) and `--json` carries it as `request.outputPath`; `--full` prints that whole log instead of the tail (the last ~768 KiB when it does not fit, with the path for the rest). |
 | `hauler kill <ticket>` | Stop a ticket: drop it from the queue or SIGTERM (then SIGKILL) its cargo process group, freeing the lane. Riders return to their lane or fail with it. |
-| `hauler request [--session ID] [--host HOST] [--cwd DIR] -- <cargo …>` | Submit a background request and return its ticket. |
-| `hauler daemon <run\|start\|stop\|status>` | Manage the daemon lifecycle. |
+| `hauler request [--session ID] [--host HOST] [--cwd DIR] [--after TICKET …] -- <cargo …>` | Submit a background request and return its ticket, with where it landed in its lane (`queued behind cc-3281 (~13m)`, `waiting for cc-3281`, or `attached to cc-3281`). `--after` works as for `exec`. |
+| `hauler daemon <run\|start\|stop\|status\|restart>` | Manage the daemon lifecycle. `restart` sends the graceful stop, waits up to 5 s for the old pid to exit, then starts a daemon from this install and prints both (`restarted: pid 741314 (0.4.1) → pid 742001 (0.4.4)`); a daemon that has not exited by then is reported, not killed, and nothing is started (exit `1`). Tickets in flight at the restart are not handed over: the new daemon marks them `killed` with the error `orphaned by daemon restart`. |
 | `hauler install-shim [--dir DIR] [--real-cargo PATH] [--force]` | Install the optional PATH shim. |
 
 The `hauler` MCP server projects the same operations as `hauler_status`,
@@ -330,6 +334,26 @@ callers, dependency-unblocking work, and recently edited packages receive a
 lower scheduling score. Waiting time lowers the score further so broad work
 eventually runs.
 
+Admission within a lane is therefore cost-ordered, not first-in-first-out: a
+queued `cargo test -p foo` (a cheap estimate) normally starts before a queued
+`cargo build --workspace` submitted a minute earlier, even when the test
+spawns a binary that build produces. The acknowledgement makes the order
+visible — `ticket cc-3289 queued behind cc-3281 (1 ahead, wait ~780s)` on
+`exec`, `cc-3289 submitted, queued behind cc-3281 (~13m)` on `request` — and
+`--after` makes the dependency explicit: `hauler exec --after cc-3281 -- cargo
+test …` (or `hauler request --after`, or `after: ["cc-3281"]` on
+`hauler_request`) stays `queued` and is skipped by admission and batch folding
+until every named ticket has settled. Prerequisites may live in any lane; a
+prerequisite in the same lane also scores higher while a ticket waits on it,
+like a dependency-closure leaf. A prerequisite
+that ends `failed` or `killed` settles the dependent `failed` with
+`prerequisite cc-N failed` and no exit code, without spawning cargo; its riders
+follow the normal attachment rules. A prerequisite that already finished
+resolves immediately; an unknown ticket is rejected as a bad intent. A blocked
+ticket never attaches to a run already in flight (that run started before the
+prerequisite finished), and `hauler status`, `hauler result`, and await
+heartbeats show `waits for cc-N (running 2m/~5m)` while it is held.
+
 Admission is separate from lane scheduling. It observes one-minute load per
 core and, on Linux, CPU PSI `some avg10`, then applies the configured thresholds
 and the global permit cap. Load and CPU pressure never defer below
@@ -363,18 +387,31 @@ own `-j` flag or `CARGO_BUILD_JOBS` always wins over both.
 | Lane isolation | A workspace-root and target-directory pair is serialized independently from other lanes. |
 | Admission | Per-core load, Linux CPU PSI, Linux memory PSI and `MemAvailable`, macOS VM pressure, configured thresholds, and the global permit cap control new starts. |
 | Parallelism | One daemon-owned jobserver FIFO shared by every spawned Cargo; a per-run `CARGO_BUILD_JOBS` grant only when the FIFO could not be armed. |
-| Scheduling | EWMA estimates, optional kache priors, fan-out, dependency topology, recent edits, and request age determine lane order. |
-| Persistence | Tickets, output tails, timings, outcomes, and savings are stored in SQLite. |
+| Scheduling | EWMA estimates, optional kache priors, fan-out, dependency topology, recent edits, and request age determine lane order; `--after cc-N` holds a request until the named tickets settle. |
+| Persistence | Tickets, output tails, timings, outcomes, and savings are stored in SQLite; every leader run's whole combined output is kept on disk as `<state dir>/tickets/<ticket>.log`. |
 | Caller output and status | Output streams to attached callers; late callers receive buffered replay. After 30 seconds without output, the client emits a progress heartbeat every 15 seconds with lane queue position, the lane-head ticket, and an aggregate wait ETA. |
-| Wait escalation | A queued request waiting longer than the larger of twice its own estimate and ten minutes is flagged as delayed; running jobs silent for more than five minutes show a quiet-duration hint. Nothing is killed automatically. |
+| Wait escalation | A queued request waiting longer than the larger of twice its own estimate and ten minutes is flagged as delayed; running jobs silent for more than five minutes show a quiet-duration hint. A running job past three times its estimate whose process tree has burned no CPU and printed nothing for ten minutes is flagged `stalled`; only a stalled job whose submitting connection is gone is killed automatically. |
 | Daemon status | `running`, `stopped`, or `unresponsive`: a socket that exists but does not answer within its budget is reported as unresponsive, never as stopped. |
 
 ### Tickets and long-running requests
 
+![cargo-hauler dashboard detail drawer streaming a running ticket's live output](docs/media/dashboard-live-output.png)
+
 Every request has a durable ticket (`cc-<n>`). Its status, exit code, output
 tail, estimate, and timestamps are stored in SQLite and can be read from later
-sessions. `hauler exec --bg -- cargo …` and `hauler_request` return the ticket
-immediately. A synchronous request also switches to background mode when a
+sessions. The ledger keeps only a bounded tail (16 KiB); the run's whole
+combined stdout+stderr goes to `<state dir>/tickets/<ticket>.log` as the daemon
+emits it — for a demultiplexed `check`/`build`/`clippy` that is the rendered
+diagnostics stream, not cargo's JSON — up to `CARGO_HAULER_TICKET_LOG_MAX_BYTES`
+(64 MiB by default; the file then ends with one truncation line). A request
+that attached to an in-flight run shares its leader's log, and the row records
+that path. `hauler result cc-N` names the file and its size, `--json` carries
+it as `request.outputPath`, and `hauler result cc-N --full` (or `hauler_result`
+with `full: true`) renders the log itself — so a red `cargo test` is triaged
+from the ticket's own `failures:` list and panic sections instead of a second
+run. The startup retention pass that prunes old ledger rows removes their logs
+too, along with any log whose row is gone. `hauler exec --bg -- cargo …` and
+`hauler_request` return the ticket immediately. A synchronous request also switches to background mode when a
 *measured* estimate — EWMA history or kache priors, never the cold-start
 default — exceeds the host's shell-tool cap (nine minutes for Claude, ten for
 Codex, fourteen for Cursor; the PATH shim uses `CARGO_HAULER_HOST` when it is
@@ -383,7 +420,9 @@ whole wait: the work queued ahead in the lane plus the job's own runtime,
 which the queued line reports as `wait ~Ns, run ~Ns`. That conversion exits
 `75` (`EX_TEMPFAIL`) with the ticket on stderr, so `cargo build && …` chains
 and scripts cannot mistake "submitted" for "built"; explicit `--bg` keeps exit
-`0`. Failed runs feed the estimate history too, so a broken build is not
+`0`. When the caller's stdout is not a terminal (`cargo test > out.log`), the
+notice adds that the redirect receives no output and to read it with
+`hauler result cc-N --full`. Failed runs feed the estimate history too, so a broken build is not
 re-estimated cold on every retry.
 
 A foreground `hauler exec` that receives SIGINT or SIGTERM (Ctrl-C, or a
@@ -396,6 +435,45 @@ start cargo at all. If the connection drops after the ticket was accepted, the
 client prints `connection to daemon lost; ticket cc-N continues — hauler
 result cc-N` and exits `1`; the daemon finishes the ticket on its own.
 
+A deadlocked test binary holds its lane for ever at 0% CPU with nothing on
+stdout, and neither the estimate overrun nor the output silence alone can
+tell it from a slow build. The daemon therefore samples the CPU time of every
+running ticket's process tree every 30 seconds (`/proc` on Linux, `ps` on
+macOS; other platforms do not detect stalls). A ticket is flagged `stalled`
+when its elapsed time exceeds `CARGO_HAULER_STALL_ESTIMATE_FACTOR` (3) times
+its estimate, the tree's CPU time has not changed for
+`CARGO_HAULER_STALL_IDLE_MS` (ten minutes), and it printed nothing in that
+window. `hauler status`, `hauler_status`, and the dashboard show `stalled`
+with the idle duration; `hauler result` and `hauler_result` answer `ticket
+looks stalled (no CPU for Nm) — hauler kill cc-N`; `hauler await` heartbeats
+say the same. Riders of a stalled leader report the leader's stall and its
+ticket, since killing a rider only detaches it. A stalled ticket whose
+submitting connection is still open is only flagged. When that connection has
+disconnected (a dead agent shell, a killed hook), the ticket is marked
+orphaned, and once it is also stalled the daemon kills it through the normal
+`hauler kill` path — riders settle or requeue as for any kill — with the
+error `stalled: no CPU for Nm after owner disconnected; killed automatically`.
+`CARGO_HAULER_STALL_AUTO_KILL=0` keeps the flag and never kills. Background
+tickets (`--bg`, `hauler_request`) have no streaming connection and are only
+ever flagged.
+
+Tickets do not survive a daemon restart. `hauler daemon restart` (or `stop`
+then `start`) ends every queued or running ticket: the cargo processes die
+with the daemon, and the new daemon's first ledger pass marks each of them
+`killed` with the error `orphaned by daemon restart`, so `hauler result cc-N`
+answers `cc-N killed — orphaned by daemon restart: the daemon stopped while it
+was in flight and does not hand runs over; resubmit if the work is still
+needed` rather than looking like a failure of the command itself. Restart when
+the CLI or plugin was upgraded under a running daemon: every document then
+carries `daemon 0.4.2 ≠ cli 0.4.4 — restart it with \`hauler daemon restart\``,
+and a reply the newer client still cannot read is reported as `daemon is 0.4.2
+(pid N, since 3h ago), this CLI is 0.4.4 — restart it with \`hauler daemon
+restart\`` with the first schema mismatch on a second line — never as a raw
+validation dump. The client schemas default the fields older daemons never
+send (`outputPath`, `after`), so a plain version difference alone does not
+break `status`, `result`, or `await`; finish or `hauler kill` what is in
+flight before restarting if the work matters.
+
 The `tool/after` route checks the session's background tickets — `--bg`,
 `hauler_request`, and synchronous requests the client converted to a ticket
 — and, on the first tool call after one finishes, adds its result to the
@@ -469,10 +547,11 @@ is reported as unavailable and never rejects a request.
 
 | Variable | Default | Meaning |
 | --- | --- | --- |
-| `CARGO_HAULER_STATE_DIR` | Per-user cache directory | Unix socket or Windows named pipe source, SQLite ledger, daemon log, pid lock, `hook-state.json`, and `hook-events.jsonl`. No legacy alias. |
+| `CARGO_HAULER_STATE_DIR` | Per-user cache directory | Unix socket or Windows named pipe source, SQLite ledger, daemon log, pid lock, `hook-state.json`, `hook-events.jsonl`, and the per-ticket output logs under `tickets/`. No legacy alias. |
 | `CARGO_HAULER_CARGO_BIN` | `$CARGO_HOME/bin/cargo` | Cargo binary for daemon-started work; bare `cargo` is the last fallback. Never resolved through `PATH`. Read from the daemon's own environment (export it where the daemon starts, or before `hauler daemon start`); clients do not forward it. |
 | `CARGO_HAULER_MAX_CONCURRENT` | `5` | Global admission permits for Cargo processes across all lanes; an integer >= 1. |
 | `CARGO_HAULER_JOBS_GRANT` | `max(4, cores / max concurrent)` | `CARGO_BUILD_JOBS` added to each Cargo process only while the shared jobserver FIFO is not armed; an armed daemon injects `MAKEFLAGS` instead and leaves `CARGO_BUILD_JOBS` unset. `0` disables injection. |
+| `CARGO_HAULER_JOBSERVER` | `auto` | Machine-wide fifo jobserver for daemon-spawned cargo: `auto` arms it only when the host `make` is 4.4+ (or absent) because older makes reject `--jobserver-auth=fifo:` in build scripts; `fifo` forces it on, `off` disables it (per-run `CARGO_BUILD_JOBS` grants apply instead). |
 | `CARGO_HAULER_LOAD_THRESHOLD` | Disabled | Per-core one-minute load threshold for deferring new admissions. |
 | `CARGO_HAULER_LOAD_MIN` | `2` | Active Cargo processes below which load, CPU PSI, and soft memory pressure do not defer admission. |
 | `CARGO_HAULER_CPU_PRESSURE_THRESHOLD` | `75` | Linux CPU PSI `some avg10` percentage for deferring new admissions; `0` or `off` disables. |
@@ -487,9 +566,13 @@ is reported as unavailable and never rejects a request.
 | `CARGO_HAULER_BATCH` | Enabled | `0`, `false`, `off`, or `no` disables the batch composer. |
 | `CARGO_HAULER_BATCH_WINDOW_MS` | `150` | Delay applied to a batchable lane head so nearby requests can fold; `0` disables. |
 | `CARGO_HAULER_KILL_GRACE_MS` | `8000` | Time between SIGTERM and SIGKILL when the daemon stops a Cargo process. |
+| `CARGO_HAULER_STALL_ESTIMATE_FACTOR` | `3` | A running ticket becomes a stall candidate once its elapsed time exceeds this multiple of its estimate. |
+| `CARGO_HAULER_STALL_IDLE_MS` | `600000` | Window with no process-tree CPU time and no output after which a stall candidate is flagged `stalled`; `0` or `off` disables stall detection. |
+| `CARGO_HAULER_STALL_AUTO_KILL` | Enabled | Kill a stalled ticket automatically once the connection that submitted it has disconnected. `0`, `false`, `off`, or `no` only flags it. |
 | `CARGO_HAULER_STOP_WAIT_MS` | `30000` | Maximum wait for one stop-hook invocation; values above the 7200000 ms await ceiling are clamped. |
 | `CARGO_HAULER_LEDGER_RETENTION_DAYS` | `30` | Finished ledger rows older than this many days are deleted when the daemon starts; `0` disables the age limit. |
-| `CARGO_HAULER_LEDGER_MAX_ROWS` | `50000` | Total ledger rows beyond which the oldest finished rows are deleted when the daemon starts; `0` disables the row cap. |
+| `CARGO_HAULER_LEDGER_MAX_ROWS` | `50000` | Total ledger rows beyond which the oldest finished rows are deleted when the daemon starts; `0` disables the row cap. Pruned rows take their `tickets/<ticket>.log` files with them. |
+| `CARGO_HAULER_TICKET_LOG_MAX_BYTES` | `67108864` (64 MiB) | Bytes of a leader run's combined output written to `<state dir>/tickets/<ticket>.log` before the log stops with one truncation line; `0` writes no ticket logs (`hauler result` then has only the tail). |
 | `CARGO_HAULER_LOG_LEVEL` | `Info` | Daemon log level. |
 | `CARGO_HAULER_HOST`, `CARGO_HAULER_SESSION` | Unset | Default `--host` and `--session` attribution for `hauler exec`; the PATH shim also borrows `CARGO_HAULER_HOST`'s shell cap for auto-background. |
 
@@ -519,7 +602,12 @@ unset, the daemon reads kache's configured local store from
   `hauler_await` fail loudly when the daemon is unreachable instead of
   reporting a ticket as not found; `hauler_status`, `hauler_log`, and
   `hauler_last` read the ledger with the daemon marked `stopped` or
-  `unresponsive`.
+  `unresponsive`. A daemon whose reply the client cannot read fails as a
+  version difference naming both versions and `hauler daemon restart`.
+- The state directory is not migrated between installs. Every rendered
+  document names the one in use (`state dir …` in the header; `stateRoot` in
+  `--json`), so a `CARGO_HAULER_STATE_DIR` change is visible on the next
+  command rather than discovered from an empty ledger.
 - Test sharing uses identity attachment or batch folding, never coverage.
   Folded `test` and `nextest` requests receive the composite output and exit
   code, so a failure may come from another package in the batch.