agent-dispatch 0.12.0__tar.gz → 0.13.0__tar.gz

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/.github/workflows/publish.yml

@@ -26,5 +26,12 @@ jobs:
           pip install build
           python -m build
 
+      # Keep this current with the build backend. The pinned action bundles its
+      # own twine, and twine rejects a Metadata-Version it does not know: the
+      # v0.13.0 publish failed because hatchling 1.32 emits `Metadata-Version:
+      # 2.5` while the previous pin (Feb 2026) shipped twine 6.1.0 / packaging
+      # 25.0. `python -m build` resolves the newest hatchling at build time, so
+      # this drifts on its own with no repo change — see the release checklist's
+      # `twine check` step, which catches it before a tag is cut.
       - name: Publish to PyPI
-        uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # release/v1
+        uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/AGENTS.md

@@ -28,7 +28,7 @@ pip install -e ".[dev]"
 
 ```bash
 ruff check src/ tests/
-python3 -m pytest tests/ -v   # 561 tests, ~4s
+python3 -m pytest tests/ -v   # 578 tests, ~5s
 ```
 
 Tests must **never** invoke the real `claude` CLI. Runner tests mock `shutil.which` + `subprocess.run`/`Popen`; server tests mock `_get_config` + `runner.dispatch`. The one exception is `TestStreamPipeHandling`, which spawns a short-lived *python* subprocess: a pipe deadlock lives in the OS pipe buffer, so a mocked `Popen` structurally cannot reproduce it.
@@ -52,16 +52,25 @@ Tests must **never** invoke the real `claude` CLI. Runner tests mock `shutil.whi
 - **Never `close()` a pipe another thread may still be reading.** `close()` waits on the reader's buffer lock with *no timeout*, so it would hang the dispatch forever — the bounded `join()` before it buys nothing. `dispatch_stream` skips the stderr close while the drain thread is alive and lets the daemon reader + Popen finalizer release the fd. This is not hypothetical: a stdio MCP server inherits the child's `stderr`, so the pipe often has no EOF even after `claude` exits cleanly.
 - **No `await` inside `config_lock()`.** `ProcessLock`'s in-process guard is a `threading.RLock` — re-entrant per *thread* — and every MCP tool coroutine runs on the one event-loop thread. Suspending in the critical section lets a second coroutine re-enter the "held" lock and interleave its own load/mutate/save. Collect warnings as data, emit them after the `with` block (`test_no_await_inside_the_config_lock` enforces this by AST).
 - Cross-process locks are acquired with a **bounded** wait, never a blocking `flock`: the server takes them on its event-loop thread, so a wedged holder would freeze every tool. After the deadline it proceeds unlocked and logs — a possible lost update beats a permanent freeze.
-- `recover_stale` sweeps `pending` on a **much longer** threshold than `running`: the jobs directory is shared by every `agent-dispatch serve`, so an hours-old pending job may still be queued behind another live server's semaphore.
+- `recover_stale` sweeps `pending` on a **much longer** threshold than `running`: the jobs directory is shared by every `agent-dispatch serve`, so an hours-old pending job may still be queued behind another live server's semaphore. For a *running* job, `started_at` alone does **not** prove abandonment — a dispatch may legitimately run to the 7200s timeout ceiling, so the file's own mtime is checked too (a live worker rewrites it on every progress flush). That check can only ever *skip* a recovery, never add one.
+- Deleting job records is **opt-in** (`settings.job_retention_days`, default `0` = off) and only ever happens at server start, never inside a tool. They are the user's dispatch history and the deletion is irreversible, so an unreadable config is treated as "do nothing" rather than falling back to a default retention.
+- `max_concurrency` must bound *subprocesses*, not coroutines. Cancelling a coroutine does not stop the thread behind `asyncio.to_thread`, so `async with sem:` gave the slot away while `claude` kept running and billing. Dispatches go through `_dispatch_guarded`, which releases from the future's done-callback and `shield`s the await. Never "simplify" it back to `async with`.
+- Tool responses are serialized through `server._dumps`, never bare `json.dumps`: the stdlib default (`ensure_ascii=True`) turns every non-ASCII character into a `\uXXXX` escape, tripling the bytes and tokenizing badly, for no gain — the stdio transport emits raw UTF-8 via pydantic anyway. A real `list_groups()` carried 8520 escapes and weighed 59 KB instead of 25 KB.
+- `agents.yaml` is parsed with libyaml's `CSafeLoader` when available (`config._YamlLoader`), falling back to the pure-Python **safe** loader — never `yaml.Loader`. The config is re-read on every tool call, so this parse is on the hot path of all 21 tools *and* blocks the event-loop thread: 9.80 ms → 0.76 ms on a real 38 KB config.
 - Pydantic does **not** validate on assignment. `Field(ge=...)` guards only the *load* path; every mutation surface (CLI `add`/`update`, MCP `add_agent`/`update_agent`) needs its own boundary check, or the bound escapes as a raw `ValidationError`.
 - Every state file (`agents.yaml`, job files) is written **temp file + `os.replace`**, never in place, and every load/mutate/save is wrapped in `config.ProcessLock` — the CLI and the MCP server are separate processes writing the same files, so a thread lock alone loses updates.
 - Anything that changes an agent's config must call `_invalidate_agent_cache` — the cache key holds the agent *name*, not its directory or permissions.
 - Only *clean* successes are cached: `cache.put` refuses failures, `denied_tools` results, and `budget_exceeded` results, so the documented "grant access, then re-dispatch" recovery is never short-circuited.
 - Remediation text is a contract: a hint that names a flag must name one that exists (`test_printed_budget_hint_is_a_runnable_command` feeds the printed flags back into the CLI). Run the command you print.
-- MCP tools that load config carry `@_config_guard` under `@mcp.tool()` so a broken `agents.yaml` returns the `{"error": ...}` envelope instead of a raw traceback.
+- The config error sets are declared **once** and in two halves: `config.CONFIG_LOAD_ERRORS` (read) and `config.CONFIG_SAVE_ERRORS` (write — `yaml.dump`'s `RepresenterError` is a `yaml.YAMLError`, therefore neither `OSError` nor `ConfigLoadError`, and used to escape both the MCP guard and the CLI's `_save_or_exit`). Two halves, not one set, because the remediations differ: a failed write is atomic so the old config survives, while a failed read needs the YAML fixed.
+- MCP tools that load config carry `@_config_guard` under `@mcp.tool()` so a broken `agents.yaml` — or a failed *write* — returns the `{"error": ...}` envelope instead of a raw traceback. The set of load errors lives in one place (`config.CONFIG_LOAD_ERRORS`) because three surfaces handle it: **`UnicodeDecodeError` is a `ValueError`, not an `OSError`**, and listing types per-site is exactly how a cp1251 config slipped past all three.
 
 - Tests must not touch anything outside `tmp_path`. `test_server.py`'s autouse `_reset_globals` and `test_cli.py`'s `_isolated_config` redirect **both** `AGENT_DISPATCH_CONFIG` and `AGENT_DISPATCH_JOBS_DIR`: a mutation tool that bails out early (unknown agent) still takes `config_lock()` first, which would otherwise create a lock file beside the developer's real config.
 
+- `mcp` is pinned **`>=1.2.0,<2`** deliberately: 2.0 removed `mcp.server.fastmcp`, which `server.py` imports, so an unbounded range gives every fresh install a dead `agent-dispatch serve`. Lifting the cap means porting to `mcp.server.mcpserver.MCPServer` — it is not a dependency bump.
+- Verify packaging in a **clean venv**, never the dev machine: build the wheel, install it fresh, import the server. A stale pin in local site-packages hides exactly the failure a new user hits first.
+- Run **`twine check dist/*`** before cutting the tag, with a *current* twine. The clean-venv check does not cover this: it proves the wheel installs, not that PyPI's uploader will accept its metadata. `python -m build` pulls the newest hatchling at build time, so the emitted `Metadata-Version` climbs on its own, and `pypa/gh-action-pypi-publish` is pinned to a SHA that bundles a fixed twine — v0.13.0's publish failed on exactly that mismatch (hatchling 1.32 → metadata 2.5; pinned twine 6.1.0 → "not a valid metadata version"). When it happens, bump the action pin rather than pinning the backend down.
+
 ## Deliberately not built
 
 These were considered — some fully implemented — and cut on purpose: an agent router / auto-dispatch (`recommend_agent` / `dispatch_auto`, removed before 0.8.0 — a keyword scorer adds little over the calling LLM at a handful of agents, and auto-dispatch can spend money or mutate a repo on a guess); groups as an execution engine (they are a descriptive layer — no routing, no per-group settings); an agent-dispatch-side budget ledger across dispatches (the CLI's own `--max-budget-usd` covers a single run; anything cumulative would need state we deliberately don't keep). Please open an issue with the use case before adding any of them.
@@ -76,4 +85,4 @@ Python ≥ 3.10 · `from __future__ import annotations` everywhere · Pydantic v
 
 ## More detail
 
-[README.md](README.md) documents every MCP tool with parameter tables, response shapes, and the error-recovery map — it doubles as the behavioral spec. The test suite (`tests/`, 561 tests) encodes the exact expected behavior of every layer: when in doubt, read the tests for the module you're touching (`test_runner.py`, `test_server.py`, `test_cli.py`, ...).
+[README.md](README.md) documents every MCP tool with parameter tables, response shapes, and the error-recovery map — it doubles as the behavioral spec. The test suite (`tests/`, 578 tests) encodes the exact expected behavior of every layer: when in doubt, read the tests for the module you're touching (`test_runner.py`, `test_server.py`, `test_cli.py`, ...).

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/CHANGELOG.md

@@ -7,6 +7,134 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.13.0] - 2026-08-13
+
+An efficiency round, measured against a real 38 KB config (4 agents, 6 groups),
+a 294-record jobs directory and 14 concurrently running servers — not against
+synthetic fixtures. Findings came from a five-axis audit (caller context,
+latency/IO, spend, CPU, concurrency) whose every claim was re-measured
+adversarially before implementation; most were confirmed as real but too small
+to matter next to a 10–120 s subprocess and were deliberately left alone.
+
+### Changed
+- **Tool responses no longer escape non-ASCII, cutting discovery payloads
+  roughly in half.** `json.dumps` defaults to `ensure_ascii=True`, so every
+  Cyrillic character left as a `\uXXXX` escape — 6 bytes for what UTF-8 stores
+  in 2, and a far worse token sequence. One `list_groups()` carried **8520**
+  such escapes: 59 KB where 25 KB was enough. Measured over a full discovery
+  pass (`list_agents` + `list_groups` + `inspect_group` + `inspect_agent`):
+  104 KB → 51 KB, about **13 000 tokens saved per pass**, on every call, in
+  every session. Nothing is lost: the stdio transport serializes the JSON-RPC
+  envelope with pydantic's `model_dump_json`, which already emits raw UTF-8 —
+  which is also why the `dispatch` family never had this problem while every
+  `json.dumps` tool did. All 57 call sites now go through one `_dumps` helper
+  so the two agree. `_dumps` probes encodability and falls back to the escaped
+  form for the one payload that needs it — a **lone surrogate**, which an agent
+  produces by printing a literal `\udXXX` escape in its JSON output (`json.loads`
+  manufactures it from perfectly ASCII input, so the subprocess-level
+  `errors="replace"` cannot help). Unescaped it would raise `UnicodeEncodeError`
+  inside the stdio transport *after* the tool returned — past every guard.
+  The saving is proportional to how much non-ASCII your config and results
+  carry: on an all-ASCII setup it is exactly zero.
+- **`agents.yaml` parses ~13x faster.** The config is re-read on every single
+  tool call (deliberately — that is how a new agent is picked up without a
+  restart), so the parse sits on the hot path of all 21 tools *and* runs on the
+  event-loop thread, where it blocks every other tool. Switching to libyaml's
+  `CSafeLoader` takes the real config from **9.80 ms to 0.76 ms** per call.
+  Falls back to the pure-Python safe loader when PyYAML was built without
+  libyaml; both are the *safe* loader, so a config still cannot construct
+  arbitrary objects.
+
+### Added
+- **`settings.job_retention_days`** — when > 0, terminal job records older than
+  N days are deleted at server start. Every async dispatch and every
+  `return_ref` dispatch leaves a job file that nothing removed on its own, so
+  the directory grew forever while `dispatch_jobs` and stale-job recovery
+  parsed every file in it. **Defaults to `0` (off)**: those records are the
+  user's own dispatch history and deleting them is irreversible, so it is an
+  explicit opt-in rather than something a version bump starts doing to an
+  existing install. An unreadable config is treated as `0`, never guessed.
+
+### Fixed
+- **`max_concurrency` bounds live subprocesses again, not live coroutines.**
+  `async with sem:` released the slot when the *coroutine* unwound, but
+  cancelling a coroutine does not stop the thread behind `asyncio.to_thread` —
+  so an interrupted turn or a closed session handed the slot to the next
+  dispatch while the `claude` subprocess kept running and kept being billed. N
+  cancellations meant up to N extra concurrent subprocesses, and each abandoned
+  worker also held a thread out of the default executor (`min(32, cpu_count+4)`,
+  which `to_thread` needs for *every* dispatch) for the rest of its timeout —
+  up to 7200s. The slot is now tied to the worker's real lifetime via
+  `_dispatch_guarded`; cancellation still reaches the caller immediately.
+- **A legitimately long dispatch is no longer flipped to `failed` by an
+  unrelated server starting up.** `recover_stale` judged abandonment from
+  `started_at` alone, but a dispatch may legitimately run up to the 7200s
+  timeout ceiling — so any of the (routinely 14+) other servers booting would
+  mark a live 2-hour job `failed`, after which `finish()` refuses the real,
+  already-paid-for result. A *running* job whose file was modified within the
+  threshold is now left alone: a live worker rewrites it on every progress
+  flush, so only the file's own age proves abandonment. This can only ever skip
+  a recovery — a genuinely abandoned job is picked up on a later start.
+- **A config that cannot be *rendered* now returns an error envelope instead of
+  a traceback.** `_get_config` wraps read-side YAML failures into
+  `ConfigLoadError`, but `save_config` → `yaml.dump` raises `RepresenterError` —
+  a `yaml.YAMLError`, so neither an `OSError` nor a `ConfigLoadError`. It
+  therefore slipped past *both* arms of the MCP guard on
+  `add_agent`/`update_agent`/`remove_agent` **and** past the CLI's
+  `_save_or_exit`. Unreachable today (only JSON-native types are ever dumped)
+  and closed now because it is the same "an exception type escapes the handler
+  meant to catch it" class that produced the 0.12.0 and 0.12.1 rounds. The set
+  is declared once as `config.CONFIG_SAVE_ERRORS`, next to its read-side twin,
+  so the two surfaces cannot drift.
+- **Stale-job recovery scans the jobs directory once instead of twice.** It
+  called `list()` per status, and `list()` reads and parses every file in the
+  directory (~85 ms at 294 records) — at the start of every server process.
+
+## [0.12.1] - 2026-07-30
+
+Two holes in 0.12.0's own "tools always return a clean error" fix.
+
+### Fixed
+- **A non-UTF-8 `agents.yaml` no longer crashes every surface.** `UnicodeDecodeError`
+  is a `ValueError`, not an `OSError`, so a config saved as cp1251/UTF-16 slipped
+  past all three handlers added in 0.12.0 and raised a bare traceback out of every
+  MCP tool, `agent-dispatch list` — and `doctor`, the command you run *because*
+  the config is broken. The set of load errors is now declared once
+  (`config.CONFIG_LOAD_ERRORS`) and shared by all three, and `doctor` reports the
+  encoding (and an unreadable file) as a normal FAIL with a fix. The shared set
+  lives in `config.CONFIG_LOAD_ERRORS`; the CLI still branches per type because
+  each one deserves a different remediation line.
+- **An undecodable byte from the `claude` CLI no longer kills a paid-for
+  dispatch.** `text=True` decodes strictly, so one invalid byte on stdout raised
+  `UnicodeDecodeError` — a `ValueError`, caught by nothing in the runner — out of
+  `dispatch`, `dispatch_stream`, the MCP tools and `agent-dispatch test`. Both
+  spawn sites now decode with `errors="replace"`, so a mangled byte becomes
+  U+FFFD instead of discarding the run. (Present since the initial commit.)
+- **`dispatch` classifies spawn failures like `dispatch_stream` already did.**
+  A directory that passes `is_dir()` but cannot be entered — or that vanishes
+  between the check and the spawn — made `subprocess.run` raise straight through
+  the `except subprocess.TimeoutExpired`. It now returns `not_found` /
+  `permission` / `cli_error` like the streaming path.
+- **The six job tools got the I/O envelope too.** `dispatch_status`, `_wait`,
+  `_cancel`, `_jobs`, `fetch_result` and `dispatch_gc` never load config, so they
+  carried no guard — an unwritable or misconfigured jobs directory raised out of
+  them. The guard is now split (`_io_guard` / `_config_guard`) and both halves
+  are applied where each is needed.
+- **`add_agent` reports an unresolvable path** (`~unknown-user`) as an error
+  envelope instead of raising `RuntimeError`.
+- **A failed config *write* is reported, not raised.** The same contract had the
+  other half missing: a full disk or a read-only volume made `save_config`
+  raise OSError straight out of `add_agent`/`update_agent`/`remove_agent` and out
+  of the matching CLI commands. Both surfaces now report it — the CLI adds that
+  the previous config is intact, which the atomic write guarantees.
+
+### Changed
+- Docs corrected against the code: stale-job recovery now documents the
+  `pending` sweep and its 24h threshold; the config-lock guarantee is stated as
+  best-effort (it proceeds unlocked after 10s rather than freezing the server);
+  `add_agent(timeout=0)` documents that it stores the literal 300, not
+  `settings.default_timeout`.
+
 ## [0.12.0] - 2026-07-29
 
 Reliability pass over the streaming path, config durability, and the tool
@@ -546,7 +674,8 @@ cache bounding, and stale-job recovery.
 - Dependabot for `pip` + `github-actions`, GitHub Actions pinned to
   commit SHAs for supply-chain integrity.
 
-[Unreleased]: https://github.com/ginkida/agent-dispatch/compare/v0.12.0...HEAD
+[Unreleased]: https://github.com/ginkida/agent-dispatch/compare/v0.12.1...HEAD
+[0.12.1]: https://github.com/ginkida/agent-dispatch/compare/v0.12.0...v0.12.1
 [0.12.0]: https://github.com/ginkida/agent-dispatch/compare/v0.11.0...v0.12.0
 [0.11.0]: https://github.com/ginkida/agent-dispatch/compare/v0.10.0...v0.11.0
 [0.10.0]: https://github.com/ginkida/agent-dispatch/compare/v0.9.0...v0.10.0

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: agent-dispatch
-Version: 0.12.0
+Version: 0.13.0
 Summary: MCP server that lets Claude Code agents delegate tasks to agents in other project directories
 Project-URL: Homepage, https://github.com/ginkida/agent-dispatch
 Project-URL: Repository, https://github.com/ginkida/agent-dispatch
@@ -388,8 +388,8 @@ Register a new project directory as an agent. Description is auto-generated from
 | `name` | string | yes | Agent name (letters, digits, hyphens, underscores) |
 | `directory` | string | yes | Path to an existing project directory (`~` is expanded, relative paths resolved) |
 | `description` | string | no | What this agent can do — auto-generated if empty |
-| `timeout` | int | no | Timeout in seconds (0 = use global default) |
-| `max_budget_usd` | float | no | Max cost in USD per dispatch (0 = no limit) |
+| `timeout` | int | no | Timeout in seconds (0 = 300; this is a literal default, not `settings.default_timeout`) |
+| `max_budget_usd` | float | no | Max cost in USD per dispatch (0 = inherit `settings.default_max_budget_usd`; no cap only when that is unset too) |
 | `permission_mode` | string | no | Permission mode (e.g. `default`, `plan`, `bypassPermissions`) |
 | `allowed_tools` | string | no | Comma-separated allowed tools (e.g. `"Bash,Read,Edit"`) |
 | `disallowed_tools` | string | no | Comma-separated disallowed tools |
@@ -405,7 +405,7 @@ Update an existing agent's configuration. Only non-empty fields are changed. Pas
 | `name` | string | yes | Agent name to update |
 | `description` | string | no | New description |
 | `timeout` | int | no | New timeout (0 = don't change) |
-| `max_budget_usd` | float | no | New budget limit (0 = don't change, negative = clear the limit) |
+| `max_budget_usd` | float | no | New budget limit (0 = don't change; negative clears the *per-agent* cap, after which `settings.default_max_budget_usd` applies) |
 | `model` | string | no | Model override. `"none"` to clear |
 | `permission_mode` | string | no | Permission mode. `"none"` to clear |
 | `allowed_tools` | string | no | Comma-separated. `"none"` to clear |
@@ -474,7 +474,7 @@ Async workers run with streaming under the hood: the job file keeps a rolling ta
 
 `dispatch_jobs(status?)` lists recent jobs as summaries (filter by `pending` / `running` / `done` / `failed` / `cancelled`). `dispatch_gc(max_age_days=7)` purges terminal jobs older than the threshold — pending and running jobs are never deleted.
 
-Job state persists to disk at `~/.config/agent-dispatch/jobs/` (override with `AGENT_DISPATCH_JOBS_DIR`). One JSON file per job, written owner-only (`0o600`) with atomic writes — safe to read or `ls` while jobs are in flight. Caller-supplied `job_id`s are validated as 32-char hex before any file access (no path traversal). On startup the server marks jobs left in `running` by a crashed instance as `failed` once they are stale (stuck for over an hour).
+Job state persists to disk at `~/.config/agent-dispatch/jobs/` (override with `AGENT_DISPATCH_JOBS_DIR`). One JSON file per job, written owner-only (`0o600`) with atomic writes — safe to read or `ls` while jobs are in flight. Caller-supplied `job_id`s are validated as 32-char hex before any file access (no path traversal). On startup the server recovers jobs a crashed instance abandoned: `running` ones stuck over an hour, and `pending` ones over 24 hours, are marked `failed` so they stop being polled forever and become collectable by `dispatch_gc`. (The `pending` threshold is deliberately long — the jobs directory is shared by every running server, so a job queued behind another server's concurrency limit must not be swept.)
 
 | When to use async | When to use `dispatch` |
 |-------------------|------------------------|
@@ -574,6 +574,7 @@ settings:
   #   - Edit
   max_dispatch_depth: 3     # recursion protection
   max_concurrency: 5        # max parallel claude -p processes (per dispatch path)
+  # job_retention_days: 30  # 0 (default) = never prune. See "Job retention" below.
   cache:
     enabled: true
     ttl: 300                # seconds
@@ -582,6 +583,30 @@ settings:
 
 Config is reloaded on every tool call — add agents without restarting.
 
+### Job retention
+
+Every `dispatch_async` **and** every `dispatch(..., return_ref=True)` writes a
+record to `~/.config/agent-dispatch/jobs/`, and nothing deletes it on its own —
+`dispatch_gc` has to be run by hand. The directory therefore grows without
+bound, and `dispatch_jobs` plus the stale-job recovery that runs at every server
+start read and parse *every* file in it.
+
+Set `job_retention_days` to prune terminal (done/failed/cancelled) records older
+than N days when a server starts:
+
+```yaml
+settings:
+  job_retention_days: 30
+```
+
+It defaults to `0` — **off** — because those records are your own history of
+past dispatches and deleting them cannot be undone. Pending and running jobs are
+never touched.
+
+`agent-dispatch gc --days N` and the `dispatch_gc` tool apply the same rule as a
+one-off. Both *delete* immediately and report the count; neither previews, so
+check what is there first with `agent-dispatch jobs`.
+
 ### Auto-Description
 
 `agent-dispatch add` without `--description` generates one from:
@@ -635,7 +660,7 @@ agent-dispatch MCP server
 - **Concurrency** — `max_concurrency` (default: 5) caps parallel `claude -p` processes. Note: the sync and async dispatch paths use separate semaphores, so the worst-case total is `2 × max_concurrency`.
 - **Timeout** — per-agent or global (default: 300s). A streaming dispatch runs the agent in its own process group, so the deadline kills the whole tree: a process the agent left running in the background can't hold the dispatch (and its concurrency slot) open past the timeout.
 - **Caching** — identical `(agent, task, context, caller, goal, response_format)` requests return cached results, bounded by `cache.max_size` (oldest entry evicted first). Only clean successes are cached: failures, results with `denied_tools`, and results flagged `budget_exceeded` are not, so the documented "grant access / raise the cap, then re-dispatch" recovery is never served a stale crippled answer. Changing an agent's config invalidates its entries. Sessions and dialogues are never cached. A `group=` dispatch folds the group's `shared_context` into `context`, so different groups cache separately and a plain dispatch is unaffected.
-- **Durable config** — `agents.yaml` is written atomically (temp file + rename), and every mutation path (CLI and MCP server alike) holds a cross-process advisory lock, so concurrent edits cannot truncate the file or silently drop one another's agents.
+- **Durable config** — `agents.yaml` is written atomically (temp file + rename), so an interrupted write can never truncate it. Every mutation path (CLI and MCP server alike) also takes a cross-process advisory lock, so concurrent edits don't drop one another's agents. The lock is best-effort by design: after waiting 10 seconds it logs a warning and proceeds anyway, because a wedged lock holder must not freeze the MCP server — so on a heavily contended config a lost update is possible, while a truncated one is not.
 
 See [SECURITY.md](SECURITY.md) for the full threat model (including the `bypassPermissions` escalation risk and on-disk job files).
 

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/README.md

@@ -358,8 +358,8 @@ Register a new project directory as an agent. Description is auto-generated from
 | `name` | string | yes | Agent name (letters, digits, hyphens, underscores) |
 | `directory` | string | yes | Path to an existing project directory (`~` is expanded, relative paths resolved) |
 | `description` | string | no | What this agent can do — auto-generated if empty |
-| `timeout` | int | no | Timeout in seconds (0 = use global default) |
-| `max_budget_usd` | float | no | Max cost in USD per dispatch (0 = no limit) |
+| `timeout` | int | no | Timeout in seconds (0 = 300; this is a literal default, not `settings.default_timeout`) |
+| `max_budget_usd` | float | no | Max cost in USD per dispatch (0 = inherit `settings.default_max_budget_usd`; no cap only when that is unset too) |
 | `permission_mode` | string | no | Permission mode (e.g. `default`, `plan`, `bypassPermissions`) |
 | `allowed_tools` | string | no | Comma-separated allowed tools (e.g. `"Bash,Read,Edit"`) |
 | `disallowed_tools` | string | no | Comma-separated disallowed tools |
@@ -375,7 +375,7 @@ Update an existing agent's configuration. Only non-empty fields are changed. Pas
 | `name` | string | yes | Agent name to update |
 | `description` | string | no | New description |
 | `timeout` | int | no | New timeout (0 = don't change) |
-| `max_budget_usd` | float | no | New budget limit (0 = don't change, negative = clear the limit) |
+| `max_budget_usd` | float | no | New budget limit (0 = don't change; negative clears the *per-agent* cap, after which `settings.default_max_budget_usd` applies) |
 | `model` | string | no | Model override. `"none"` to clear |
 | `permission_mode` | string | no | Permission mode. `"none"` to clear |
 | `allowed_tools` | string | no | Comma-separated. `"none"` to clear |
@@ -444,7 +444,7 @@ Async workers run with streaming under the hood: the job file keeps a rolling ta
 
 `dispatch_jobs(status?)` lists recent jobs as summaries (filter by `pending` / `running` / `done` / `failed` / `cancelled`). `dispatch_gc(max_age_days=7)` purges terminal jobs older than the threshold — pending and running jobs are never deleted.
 
-Job state persists to disk at `~/.config/agent-dispatch/jobs/` (override with `AGENT_DISPATCH_JOBS_DIR`). One JSON file per job, written owner-only (`0o600`) with atomic writes — safe to read or `ls` while jobs are in flight. Caller-supplied `job_id`s are validated as 32-char hex before any file access (no path traversal). On startup the server marks jobs left in `running` by a crashed instance as `failed` once they are stale (stuck for over an hour).
+Job state persists to disk at `~/.config/agent-dispatch/jobs/` (override with `AGENT_DISPATCH_JOBS_DIR`). One JSON file per job, written owner-only (`0o600`) with atomic writes — safe to read or `ls` while jobs are in flight. Caller-supplied `job_id`s are validated as 32-char hex before any file access (no path traversal). On startup the server recovers jobs a crashed instance abandoned: `running` ones stuck over an hour, and `pending` ones over 24 hours, are marked `failed` so they stop being polled forever and become collectable by `dispatch_gc`. (The `pending` threshold is deliberately long — the jobs directory is shared by every running server, so a job queued behind another server's concurrency limit must not be swept.)
 
 | When to use async | When to use `dispatch` |
 |-------------------|------------------------|
@@ -544,6 +544,7 @@ settings:
   #   - Edit
   max_dispatch_depth: 3     # recursion protection
   max_concurrency: 5        # max parallel claude -p processes (per dispatch path)
+  # job_retention_days: 30  # 0 (default) = never prune. See "Job retention" below.
   cache:
     enabled: true
     ttl: 300                # seconds
@@ -552,6 +553,30 @@ settings:
 
 Config is reloaded on every tool call — add agents without restarting.
 
+### Job retention
+
+Every `dispatch_async` **and** every `dispatch(..., return_ref=True)` writes a
+record to `~/.config/agent-dispatch/jobs/`, and nothing deletes it on its own —
+`dispatch_gc` has to be run by hand. The directory therefore grows without
+bound, and `dispatch_jobs` plus the stale-job recovery that runs at every server
+start read and parse *every* file in it.
+
+Set `job_retention_days` to prune terminal (done/failed/cancelled) records older
+than N days when a server starts:
+
+```yaml
+settings:
+  job_retention_days: 30
+```
+
+It defaults to `0` — **off** — because those records are your own history of
+past dispatches and deleting them cannot be undone. Pending and running jobs are
+never touched.
+
+`agent-dispatch gc --days N` and the `dispatch_gc` tool apply the same rule as a
+one-off. Both *delete* immediately and report the count; neither previews, so
+check what is there first with `agent-dispatch jobs`.
+
 ### Auto-Description
 
 `agent-dispatch add` without `--description` generates one from:
@@ -605,7 +630,7 @@ agent-dispatch MCP server
 - **Concurrency** — `max_concurrency` (default: 5) caps parallel `claude -p` processes. Note: the sync and async dispatch paths use separate semaphores, so the worst-case total is `2 × max_concurrency`.
 - **Timeout** — per-agent or global (default: 300s). A streaming dispatch runs the agent in its own process group, so the deadline kills the whole tree: a process the agent left running in the background can't hold the dispatch (and its concurrency slot) open past the timeout.
 - **Caching** — identical `(agent, task, context, caller, goal, response_format)` requests return cached results, bounded by `cache.max_size` (oldest entry evicted first). Only clean successes are cached: failures, results with `denied_tools`, and results flagged `budget_exceeded` are not, so the documented "grant access / raise the cap, then re-dispatch" recovery is never served a stale crippled answer. Changing an agent's config invalidates its entries. Sessions and dialogues are never cached. A `group=` dispatch folds the group's `shared_context` into `context`, so different groups cache separately and a plain dispatch is unaffected.
-- **Durable config** — `agents.yaml` is written atomically (temp file + rename), and every mutation path (CLI and MCP server alike) holds a cross-process advisory lock, so concurrent edits cannot truncate the file or silently drop one another's agents.
+- **Durable config** — `agents.yaml` is written atomically (temp file + rename), so an interrupted write can never truncate it. Every mutation path (CLI and MCP server alike) also takes a cross-process advisory lock, so concurrent edits don't drop one another's agents. The lock is best-effort by design: after waiting 10 seconds it logs a warning and proceeds anyway, because a wedged lock holder must not freeze the MCP server — so on a heavily contended config a lost update is possible, while a truncated one is not.
 
 See [SECURITY.md](SECURITY.md) for the full threat model (including the `bypassPermissions` escalation risk and on-disk job files).
 

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/agents.example.yaml

@@ -100,6 +100,12 @@ settings:
   #   - Bash
   #   - Read
   #   - Edit
+  # job_retention_days: 30         # OFF by default (0). When > 0, terminal job records
+                                   # older than this are deleted at server start. Every
+                                   # async and every return_ref dispatch leaves a job
+                                   # file behind and nothing removes it otherwise, so
+                                   # the directory grows forever — but the records are
+                                   # your dispatch history, so pruning is opt-in.
   cache:
     enabled: true
     ttl: 300                       # seconds; identical (agent, task, context) requests are cached

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "agent-dispatch"
-version = "0.12.0"
+version = "0.13.0"
 description = "MCP server that lets Claude Code agents delegate tasks to agents in other project directories"
 readme = "README.md"
 license = "MIT"

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/src/agent_dispatch/__init__.py

@@ -1,3 +1,3 @@
 """agent-dispatch: Delegate tasks between Claude Code agents across projects."""
 
-__version__ = "0.12.0"
+__version__ = "0.13.0"

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/src/agent_dispatch/cli.py

@@ -13,7 +13,14 @@ import click
 import yaml
 from pydantic import ValidationError
 
-from .config import auto_describe, config_lock, config_path, load_config, save_config
+from .config import (
+    CONFIG_SAVE_ERRORS,
+    auto_describe,
+    config_lock,
+    config_path,
+    load_config,
+    save_config,
+)
 from .jobs import JobStore, default_jobs_dir, is_valid_job_id
 from .models import (
     AgentConfig,
@@ -29,6 +36,39 @@ def _parse_csv(value: str | None) -> list[str] | None:
     return [t.strip() for t in value.split(",") if t.strip()] if value else None
 
 
+def _save_or_exit(config: DispatchConfig) -> None:
+    """Persist the config, turning an I/O failure into a message, not a traceback.
+
+    A full disk or a read-only volume makes `save_config` raise OSError. The
+    write is atomic, so the previous config survives — say so, since that is the
+    one thing the user needs to know.
+    """
+    try:
+        save_config(config)
+    except OSError as e:
+        click.echo(
+            click.style(
+                f"Error: could not write {config_path()}: {e}\n"
+                "Nothing was changed — the previous config is intact "
+                "(the write is atomic). Check free space and permissions.",
+                fg="red",
+            )
+        )
+        raise SystemExit(1) from None
+    except CONFIG_SAVE_ERRORS as e:
+        # A value YAML cannot represent — not an OSError, so the arm above never
+        # saw it and it escaped as a traceback. See config.CONFIG_SAVE_ERRORS.
+        click.echo(
+            click.style(
+                f"Error: could not serialize the config to YAML: {e}\n"
+                "Nothing was changed — the previous config is intact. "
+                "This is a bug: please report the field that failed.",
+                fg="red",
+            )
+        )
+        raise SystemExit(1) from None
+
+
 def _check_budget_or_exit(max_budget: float | None) -> None:
     """Reject a negative spend cap before it reaches `AgentConfig` (ge=0).
 
@@ -75,6 +115,15 @@ def _load_or_exit() -> DispatchConfig:
         click.echo(click.style(f"Error: config at {config_path()} is not valid YAML:", fg="red"))
         click.echo(str(e))
         raise SystemExit(1) from None
+    except UnicodeDecodeError as e:
+        click.echo(
+            click.style(
+                f"Error: config at {config_path()} is not valid UTF-8 (re-save it as UTF-8):",
+                fg="red",
+            )
+        )
+        click.echo(str(e))
+        raise SystemExit(1) from None
     except OSError as e:
         click.echo(click.style(f"Error: config at {config_path()} could not be read:", fg="red"))
         click.echo(str(e))
@@ -236,7 +285,7 @@ def add(
         if warning := check_permission_mode(permission_mode):
             click.echo(click.style(f"Warning: {warning}", fg="yellow"))
 
-        save_config(config)
+        _save_or_exit(config)
     click.echo(f"Added agent '{name}' -> {dir_path}")
 
 
@@ -251,7 +300,7 @@ def remove(name: str) -> None:
             raise SystemExit(1)
 
         del config.agents[name]
-        save_config(config)
+        _save_or_exit(config)
     click.echo(f"Removed agent '{name}'.")
 
 
@@ -381,7 +430,7 @@ def update(
             click.echo("Nothing to update. Pass at least one option (see --help).")
             raise SystemExit(1)
 
-        save_config(config)
+        _save_or_exit(config)
     click.echo(f"Updated agent '{name}': {', '.join(updated)}")
 
 
@@ -639,7 +688,7 @@ def group_add(name: str, description: str, shared_context: str, members: tuple[s
             shared_context=shared_context,
             members=member_objs,
         )
-        save_config(config)
+        _save_or_exit(config)
     click.echo(f"Added group '{name}' ({len(member_objs)} member(s)).")
     if not member_objs:
         click.echo(
@@ -737,7 +786,7 @@ def group_update(name: str, description: str | None, shared_context: str | None)
             click.echo("Nothing to update. Pass --description and/or --shared-context.")
             raise SystemExit(1)
 
-        save_config(config)
+        _save_or_exit(config)
     click.echo(f"Updated group '{name}': {', '.join(updated)}")
 
 
@@ -752,7 +801,7 @@ def group_remove(name: str) -> None:
             raise SystemExit(1)
 
         del config.groups[name]
-        save_config(config)
+        _save_or_exit(config)
     click.echo(f"Removed group '{name}'.")
 
 
@@ -807,6 +856,15 @@ def doctor() -> None:
         except yaml.YAMLError as e:
             fail(f"Config not valid YAML: {cp}")
             click.echo(f"    {e}")
+        except UnicodeDecodeError as e:
+            # A ValueError, not an OSError — it slipped past both handlers above
+            # and crashed the one command meant to diagnose a broken config.
+            fail(f"Config is not valid UTF-8: {cp}")
+            click.echo(f"    {e}")
+            click.echo("    Re-save the file as UTF-8.")
+        except OSError as e:
+            fail(f"Config could not be read: {cp}")
+            click.echo(f"    {e}")
 
     section("MCP registration")
     if claude_path is None:

{agent_dispatch-0.12.0 → agent_dispatch-0.13.0}/src/agent_dispatch/config.py

@@ -13,6 +13,7 @@ from collections.abc import Iterator
 from pathlib import Path
 
 import yaml
+from pydantic import ValidationError
 
 from .models import DispatchConfig
 
@@ -21,6 +22,19 @@ try:  # pragma: no cover - platform dependent
 except ImportError:  # pragma: no cover - Windows has no fcntl
     fcntl = None  # type: ignore[assignment]
 
+# Parse with libyaml when PyYAML was built against it. Every MCP tool call
+# reloads agents.yaml from scratch (deliberately — that is how a new agent is
+# picked up without a restart), so this parse is on the hot path of all 21
+# tools. On a real 38 KB config the pure-Python SafeLoader takes ~9.8 ms and
+# CSafeLoader ~0.6 ms: a 16x saving repeated on every single call, and it runs
+# on the event-loop thread where it blocks every other tool. The fallback is
+# mandatory — a PyYAML installed from source without libyaml headers has no
+# C extension. Same semantics either way: both are the *safe* loader.
+try:  # pragma: no cover - depends on how PyYAML was built
+    from yaml import CSafeLoader as _YamlLoader
+except ImportError:  # pragma: no cover - pure-Python PyYAML
+    from yaml import SafeLoader as _YamlLoader  # type: ignore[assignment]
+
 logger = logging.getLogger(__name__)
 
 DEFAULT_CONFIG_DIR = Path.home() / ".config" / "agent-dispatch"
@@ -150,12 +164,36 @@ def file_lock(path: Path) -> Iterator[None]:
         _release_lock(fd)
 
 
+# Everything load_config() can raise for a config a human can plausibly produce.
+# Listed once because three surfaces handle it independently (server._get_config,
+# cli._load_or_exit, cli.doctor) and they must not drift: UnicodeDecodeError, in
+# particular, is a ValueError — NOT an OSError — so a file saved in cp1251 or
+# UTF-16 used to escape every one of them as a raw traceback.
+CONFIG_LOAD_ERRORS = (ValidationError, yaml.YAMLError, UnicodeDecodeError, OSError)
+
+# The *write* half, and deliberately not a copy of the read half. `save_config`
+# can fail two ways with two different remediations: OSError (full disk,
+# read-only volume — handled separately, and the atomic rename means the old
+# file survives), or a rendering failure. `yaml.dump` raises RepresenterError —
+# a `yaml.YAMLError`, so neither an OSError nor a ValueError — for any value it
+# cannot represent. That is unreachable today because save_config only ever
+# feeds it `model_dump(mode="json")` output, i.e. JSON-native types. It becomes
+# reachable the moment a field lands whose JSON dump is not one of those, and it
+# would then escape *both* server guards and the CLI's `_save_or_exit` as a raw
+# traceback — the exact "an exception type escapes the handler meant to catch
+# it" class this codebase has been bitten by twice. Declared once, next to its
+# read-side twin, so the two surfaces that handle it cannot drift.
+CONFIG_SAVE_ERRORS = (yaml.YAMLError,)
+
+
 def load_config(path: Path | None = None) -> DispatchConfig:
     """Load config from YAML file. Returns empty config if file missing."""
     p = path or config_path()
     if not p.exists():
         return DispatchConfig()
-    raw = yaml.safe_load(p.read_text(encoding="utf-8"))
+    # yaml.load with the safe loader == yaml.safe_load; _YamlLoader is the C
+    # one when available (see its definition for why this is worth doing).
+    raw = yaml.load(p.read_text(encoding="utf-8"), Loader=_YamlLoader)  # noqa: S506
     if raw is None:
         return DispatchConfig()
     return DispatchConfig.model_validate(raw)