@matt82198/aesop 0.3.2 → 0.4.1

package/docs/THE-AESOP-HYPOTHESIS.md

@@ -0,0 +1,156 @@
+# The Aesop Hypothesis: Why Crash-Recoverable Systems Outrun Distributed Ones
+
+**Expanded from the original essay:** https://medium.com/@matt82198/the-aesop-hypothesis-ai-agents-that-survive-because-theyre-designed-to-fail-de5f033369d4
+
+---
+
+## The Hypothesis
+
+**Agent behavior is source code.** Everything a fleet does — every decision, every checkpoint, every recovery path — lives in durable, human-diffable files: git history, plain-text STATE.md, append-only BUILDLOG.md, Python scripts, shell hooks. No vector embeddings, no distributed consensus, no magic. When a machine fails, you re-read from disk. When a human operator needs to audit a decision, they grep the git log or read the state file. When you need to reason about cost, you look at the dispatch rules in code.
+
+## Ancestors: Naming the Ideas
+
+Aesop does not invent crash-recoverable orchestration. It **composes and measures** four proven ideas from distributed systems research:
+
+1. **Temporal** (Czezatke & Stengel, 2019) — durable execution via plain-text event logs, essential for crash recovery without external state.
+2. **Crash-only software** (Candea & Fox, 2003) — design every component as stateless; recovery is the normal startup path.
+3. **Erlang/OTP supervision trees** (Armstrong et al., 1996) — organize fault tolerance as a hierarchy of restarts, each with a bounded retry policy.
+4. **Kubernetes controller reconciliation** (Burns et al., 2015) — durable desired state + controllers that converge reality to it via append-only logs.
+
+Aesop ports these patterns to agent orchestration: the wave loop is a reconciliation controller; STATE.md + BUILDLOG.md are the durable event log; workers are supervised processes with retry caps; crashes trigger re-reads from disk. The contribution is not any single idea, but the composition, measurement (benchmarks with committed artifacts), and the discipline of naming ancestors instead of claiming novelty.
+
+---
+
+This hypothesis rests on five pillars:
+
+1. **Durable plain-text state** — git + POSIX text as the state layer, not Postgres or vector DBs.
+2. **Stateless runtimes** — agents execute one request at a time; permanent state lives on disk.
+3. **Cost-aware parallelism** — cheap Haiku subagents in parallel, not serial Opus.
+4. **Guardrails in code, not prose** — pre-push secret gates, kill-switches, cost ceilings: all executable.
+5. **Observable signals** — heartbeats, append-only logs, drift detectors, crash-recovery as the normal startup path.
+
+The bet is this: **a small, crash-recoverable system running on git and plain text outperforms a distributed one** in latency, debuggability, cost, and trust — because the simpler system fails loudly and often, learns from every failure, and never hides state in a database you can't grep.
+
+---
+
+## (1) Git + POSIX Text as the State Layer
+
+**Why not Postgres? Why not vector DBs?**
+
+Aesop's core state lives in git-committed files: `STATE.md` (intent, phase, NEXT STEPS), `BUILDLOG.md` (append-only progress snapshots), Python scripts (cost rules, dispatch logic), shell hooks (pre-push gates). This is not a limitation. It is the whole idea.
+
+**Durability.** Postgres fails when the connection pool is exhausted or the database is unreachable. Git fails when the filesystem is corrupted — a far rarer event on any modern machine. State committed to git survives machine wipes, container restarts, session loss. You clone a repository from 2026-07-18, and you know *exactly* what the system was doing on that date. No migration scripts, no schema versioning, no eventual consistency.
+
+**Human-diffable forensics.** When something goes wrong, you run `git log -p` and read the actual changes that led to the broken state. You see not just what happened, but *why* the system made each decision (because the humans who designed it wrote it in code and commit messages). A vector DB stores embeddings; a BUILDLOG.md stores human-readable decisions.
+
+**Single-box by explicit design choice.** Aesop is not "not distributed yet." Multi-instance coordination is *deliberately unscheduled*. The system runs on one machine. When team scale requires multi-instance support, the real work is not "add Postgres"; it is "redesign state to support leases and event-sourcing on SQLite." That redesign is on the roadmap, not an architectural gap. Postgres is a refactoring target *after* the single-box proves the core loop works. Premature distribution is premature optimization.
+
+**Cite:** [`docs/CHECKPOINTING.md`](./CHECKPOINTING.md) — the durable state strategy; [`docs/CARDINAL-RULES.md`](./CARDINAL-RULES.md) § 5 — handoff discipline.
+
+---
+
+## (2) Stateless Runtimes + Persistent Filesystem Brain
+
+**The architecture is simple:** agents are processes. Each agent receives a scoped task, reads the filesystem for context, makes decisions, writes results, and exits. The filesystem is the only source of truth.
+
+When an agent crashes (or hits a timeout, or the user kills it), the next agent picks up from the checkpoint files on disk. There is no agent state in memory, no distributed transaction, no graceful shutdown protocol. Dead is dead; reading from disk is the recovery protocol *and the normal startup path*.
+
+**Why this matters:** the system never invents state. An agent that hangs for 3 minutes and is forcibly killed is indistinguishable from one that exits normally — both leave state on disk, and the next reader validates what's there. No "check if this agent is still alive" logic, no heartbeat-based membership. The watchdog's job is simple: if a task hangs, kill it; the orchestrator will re-read the checkpoint and decide what to do next.
+
+**Crash recovery is not a special path.** On resume after a crash (or user interruption, or session loss), the orchestrator reads STATE.md and BUILDLOG.md from disk, verifies them against git log, and proceeds. If STATE.md is stale, it updates it. If BUILDLOG.md shows a half-completed task, it completes it or rolls back and retries. This is the same code path that runs on normal startup. No two code paths, no special-case recovery hooks, no "was I shut down cleanly?" flag.
+
+**Cite:** [`docs/CHECKPOINTING.md`](./CHECKPOINTING.md) — recovery workflow; [`docs/RELIABILITY.md`](./RELIABILITY.md) — inputs-always-produce-outputs principle.
+
+---
+
+## (3) Cost Architecture: Haiku-First, Flat Fan-Out, the Cancelled Hierarchical Design
+
+**The cost model is the heart of the system.** Subagents are *always* Haiku (1/3 the per-token cost of Opus), spawned in parallel (5–8 agents per wave). This single rule, more than any other, determines whether agent-driven work scales or burns money.
+
+**The A/B that killed hierarchical dispatch:** Earlier designs proposed a three-tier model — Fable orchestrator + Sonnet supervisors (splitting work into domains) + Haiku workers. Lab testing showed **4.3× cost increase for identical quality**. The hierarchical design was cancelled. (Cancelled architectures with published data is engineering honesty, not weakness.)
+
+Today's dispatch is flat: one Opus/Fable orchestrator on the main thread, 5–8 parallel Haiku workers per wave, no intermediate supervisors. Cost per wave: roughly $0.01–0.02 USD. Scaling to 10 waves per day still costs less than a single Opus API call.
+
+**The benchmark proves Haiku is good enough.** The held-out judgment benchmark (v3 = 28 additional tasks, building on v2 = 11 prior) tested Haiku, Sonnet, and Opus across 39 combined judgment tasks: bug-in-diff (with concurrency races and resource leaks), finding-inflation, acceptance-criteria coverage, severity calibration, root-cause-from-trace, refactor-equivalence, security issue spotting. All three models converged on identical answers for all 28 v3 tasks. Combined score: **Haiku 39/39** vs **Opus 38/39** (Opus erred on one severity call; Haiku did not). At ~1/3 the per-token cost.
+
+**Ceiling rule: the benchmark demonstrates sufficiency, not equivalence.** A pre-declared ceiling rule (when ≥2 tiers score ≥92%, the instrument failed to discriminate) trips on this result: both Haiku and Sonnet achieved 39/39 (100%), confirming the benchmark found a *convergence zone* where both models ace the task set, not a *separating frontier* where one outperforms the other. This is the intended result — the honest interpretation is that **Haiku is sufficient for the judgment shapes measured here**, not that it is tier-equivalent to Sonnet or Opus everywhere. See [`bench/results/2026-07-26-judgment-v3-ceiling-addendum.md`](../bench/results/2026-07-26-judgment-v3-ceiling-addendum.md) for the full forensic analysis.
+
+**Honest limits on the benchmark:** Curated (N=39), not sampled from real fleet transcripts. No frontier-reaching task found where Opus beats Haiku in this set. The benchmark maps a floor ("Haiku is sufficient for scoped judgment and extraction tasks with context at the seam"), not the absolute frontier. **What it does NOT test**: open-ended synthesis (designing novel systems from scratch), frontier reasoning (problems requiring 100+ steps of chaining or novel proof techniques), long-horizon planning (multi-phase dependency graphs). Cost is token-price ratio, not wall-clock latency. For detailed equivalence analysis, see [`bench/EQUIVALENCE-MARGIN.md`](../bench/EQUIVALENCE-MARGIN.md). These caveats are not hidden; they are load-bearing.
+
+**Cite:** [`docs/DISPATCH-MODEL.md`](./DISPATCH-MODEL.md) — cost model and patterns; [`bench/results/2026-07-17-judgment-v3-haiku-sonnet-opus.md`](../bench/results/2026-07-17-judgment-v3-haiku-sonnet-opus.md) — benchmark run and interpretation.
+
+---
+
+## (4) Guardrails Enforced in Code, Not Prose
+
+**Safety rules live in executable code**, not documentation:
+
+- **Pre-push secret gate** (`tools/secret_scan.py`): scans staged files for 50+ secret patterns (AWS keys, Anthropic keys, tokens). Exits with failure on file-read errors; never silently passes.
+- **Kill-switch** (`tools/halt.py`): wired into the live dispatch path. When triggered, aborts all pending work with zero new workers spawned. Operator-triggered (manual brake), not autonomous.
+- **Cost ceiling** (`tools/cost_ceiling.py`): halts dispatch when the configured per-wave budget is exceeded. Enforces a *configured* ceiling, not live-metered spend.
+- **Pre-push branch checks**: run before every push (enforced via git hooks). No committing to main, no force-push without explicit approval.
+
+The key insight: **fail-closed by default.** A secret-scan that silently passes when the file is unreadable is worse than a crash. A kill-switch that doesn't trip is useless. A cost ceiling that is "maybe" enforced wastes tokens. Aesop inverts the default: safety rules are executable and logged; unsafe paths are explicitly rejected; a gate that fails triggers an immediate backout.
+
+**Cite:** [`docs/CARDINAL-RULES.md`](./CARDINAL-RULES.md) § 7 — security and version control; [`tools/halt.py`](../tools/halt.py), [`tools/cost_ceiling.py`](../tools/cost_ceiling.py) — implementations.
+
+---
+
+## (5) Observability: Heartbeats, Append-Only Logs, Drift Signals, and CI Sharding
+
+**Every action produces a signal.** Daemons emit heartbeats every cycle (even on error). Logs are append-only; every task appended with a timestamp. Stalled agents trigger automatic watchdog restarts (3 retries, then escalate to human).
+
+**Drift detection.** The orchestrator compares expected state (BUILDLOG.md) against reality (git log, filesystem timestamps). Drift = stale checkpoint, incomplete work, or a half-written file. On detection, the system does not guess: it re-reads from disk and either rolls forward (if work completed) or rolls back (if interrupted).
+
+**CI sharding story.** Early on, Aesop's test suite ran serially on Windows, wall-clock time ~11 minutes. A single spawn-semantics bug hit Windows harder than Linux (process group cleanup behaved differently). Rather than paper over it with retries, the team:
+1. Diagnosed the root cause (Windows process tree cleanup).
+2. Fixed it (explicit cleanup in the test harness).
+3. Added sharding (4-way split, ~3 min wall-clock with 80–180s per shard).
+4. Made Windows a required check (previously optional).
+
+The point: **observability means you see the real bottleneck**, and you fix it, not the symptom. Aesop's CI reports job timings for every shard; the orchestrator reads those and can rebalance if a shard drifts >20% off baseline.
+
+**Cite:** [`docs/CARDINAL-RULES.md`](./CARDINAL-RULES.md) § 3 — reliability core and heartbeats; [`docs/RELIABILITY.md`](./RELIABILITY.md) — inputs-always-produce-outputs, never-wait discipline.
+
+---
+
+## Proof: What Ships with the System
+
+These are not claims about what Aesop *could* do. They are receipts:
+
+- **1,181 commits, 387 merged PRs, 30 waves** (verified by anyone who clones; `tools/self_stats.py`).
+- **173,035 lines of code** across 642 files tracked, delivered end-to-end: from feature intake to merge.
+- **Benchmark results** committed in `bench/results/` — 39 judgment tasks, all models scored by deterministic Python scoring (no LLM in the grading loop).
+- **Kill-switch proof** — `tools/halt.py` is wired into the live dispatch path and was exercised on a real wave.
+- **Cost ceiling** — implemented in `tools/cost_ceiling.py`, enforced per-wave.
+- **Windows CI sharding** — reduced wall-clock time from ~11 min to ~3 min (4-way shard); now a required check.
+- **Durable state** — STATE.md, BUILDLOG.md, and all orchestration rules are git-committed and human-readable.
+
+---
+
+## Honest Limits
+
+This is not a universal solution. The system has explicit boundaries:
+
+1. **Single-box by design.** Aesop runs on one machine. Multi-instance coordination is on the roadmap, not shipped. If you need 100-machine scale today, this is not the tool.
+2. **Small-N benchmarks.** 39 judgment tasks is directional evidence, not statistical proof. Frontier reasoning (where Opus depth might matter 3×) is not tested here.
+3. **Scored judgment vs. open-ended generation — a measured boundary.** An all-Haiku audit (wave-24) reported four P0 issues, verification found zero real (2 hallucinated, 2 severity-inflated). This is not a Haiku failure — it is a *cheap-model failure mode* when the model chooses what to report in open-ended generation. The benchmark proves Haiku holds on *scored, single-shot judgment* (when the task structure and rubric are explicit). The architecture mitigates this boundary by pairing cheap generation (workers) with independent verification (multi-tier review gate) — the separation of concerns is precisely designed for this measured failure mode. Selection (deciding what to report) is where cheap models diverge from expensive ones; scoring (evaluating a bounded task with a rubric) is where they converge.
+4. **Lab-measured multi-writer throughput.** 800 events/sec is measured in a stress test, not production. Team scale beyond one machine requires additional work (leases, event-sourcing, distributed consensus).
+5. **No third-party verification yet.** The artifacts are committed so a skeptic can reproduce — that is transparency, not independent replication.
+6. **Release candidate.** APIs, config, and dashboard contracts may still shift. Pin the exact version if you need stability.
+
+---
+
+## The Bet
+
+**Simple systems that fail loudly and often outrun complex ones that hide state in databases.**
+
+Aesop bets on:
+- **Transparency over abstraction.** Every decision is code. Every state is a file you can read and diff.
+- **Crash recovery as design principle.** If you build for recovery from scratch, you build for reliability. Distributed systems hide failures; crash-recoverable ones surface them.
+- **Small is faster than smart.** Flat fan-out (5–8 Haiku agents) beats hierarchical dispatch (4.3× cost), even at scale, because the simpler system has fewer failure modes.
+- **Cost as a first-class constraint.** The whole system is designed around $0.01–0.02 per wave. Expensive paths are rejected before they ship.
+
+The evidence is in the receipts: 1,181 commits, 387 PRs, 30 waves, zero hallucinated audits (via adversarial verification), and a benchmark that proves Haiku is good enough for scoped judgment work.
+
+**Read more:** [`docs/autonomous-swe.md`](./autonomous-swe.md) — honest account of what shipped, what didn't, and where the gaps are.

package/driver/CLAUDE.md

@@ -1,150 +1,150 @@
 # driver/ — AgentDriver backend-portability seam
 
-**What**: the domain contract letting aesop's wave loop run on non-Claude backends
-(Codex, open models); the loop dispatches only through the `AgentDriver` interface.
-**Phases 1-3 shipped** (interface + reference adapter; Codex Chat Completions; wave bridge).
+**What**: the domain contract letting aesop's wave loop run on non-Claude backends; the loop dispatches only through the `AgentDriver` interface. **Phases 1-3 shipped**.
 
 ## Files
 
-- **agent_driver.py** — the `AgentDriver` ABC + capability/request/result
-  dataclasses. The contract. stdlib-only, no provider SDKs.
-- **claude_code_driver.py** — reference adapter (Claude Code parity). Thin +
-  documented: two ops are concrete Python, three are serviced by the harness.
-- **codex_driver.py** — Phase 2 IMPLEMENTATION: OpenAI Chat Completions HTTP
-  backend. Fully wired: dispatch_worker (file injection, JSON validation with
-  retry, full-file replacement), run_command (subprocess), worker_status
-  (in-memory registry), get_tokens_spent (aggregate usage). Transport injectable
-  for offline testing.
-- **openai_transport.py** — stdlib urllib transport for OpenAI Chat Completions
-  endpoint. Injectable seam so tests feed canned responses (FakeTransport) with
-  no API key or network.
-- **openai_compatible_driver.py** — OpenAI-compatible backend (Ollama, OpenRouter, etc.).
-- **verification_policy.py** — Maps verification tier -> orchestrator tuning (validate_all_json,
-  spot_check_frac, repair_cap, require_adversarial_review).
+- **agent_driver.py** — the `AgentDriver` ABC + capability/request/result dataclasses.
+  The contract. stdlib-only, no provider SDKs.
+- **claude_code_driver.py** — reference adapter (Claude Code parity): two ops concrete
+  Python, three serviced by the harness.
+- **codex_driver.py** — Phase 2: OpenAI Chat Completions HTTP backend: dispatch_worker (file
+  injection, JSON retry), run_command (own `command_timeout_s` knob, defaults to timeout_s), worker_status, get_tokens_spent. Transport injectable.
+- **proc_util.py** — `run_shell_bounded()`: shared run_command backing (RS-A F1/F7). Timeout
+  truly bounds wall-clock: child in own group/session; on expiry the WHOLE tree is killed
+  (taskkill /T /F Windows, killpg POSIX), exit 124 promptly, partial output preserved. Never
+  subprocess.run(shell=True, timeout=) — Windows kills only cmd.exe then re-blocks on the orphaned grandchild.
+- **openai_transport.py** — stdlib urllib transport for the OpenAI endpoint; injectable
+  seam (FakeTransport in tests: no API key or network).
+- **openai_compatible_driver.py** — OpenAI-compatible backend (Ollama, OpenRouter, etc.);
+  construct-time validate_base_url; key waiver only via loopback-validated is_local.
+- **verification_policy.py** — Maps verification tier -> orchestrator tuning (validate_all_json, spot_check_frac, repair_cap, require_adversarial_review).
 - **wave_loop.py** — the wave ENGINE: preflight ownership guard, parallel build,
-  bounded repair, adversarial review, per-repo batched git ship, recovery journal.
-- **wave_bridge.py** — Phase 3: bridges AgentDriver backends to wave manifest items.
-  build_manifest_item() enriches with verificationTier + model; dispatch_item() routes
-  by capability and decides green ONLY from test exit code (not model's say-so).
-- **backend_config.py** — Per-deployment model resolution (role → model id, API key/base URL).
-- **README.md** — the abstraction, the phased roadmap, the verification thesis.
-- **../tests/test_agent_driver.py** — the contract's test suite.
-- **../tests/test_codex_driver_e2e.py** — Phase 2 end-to-end offline tests
-  (FakeTransport, red-to-green verification, retry logic, ownership enforcement)
-  + gated live test (AESOP_CODEX_LIVE env var).
-- **../tests/test_wave_bridge.py** — Phase 3 offline e2e (honest green: exit 0 only).
-  no network). Tests: manifest building, routing, fail-safe, ownership enforcement,
-  headline test (red stub + FakeTransport fix + test pass -> ok=True).
+  bounded repair, per-repo batched git ship, recovery journal. HS-2 (live seat swap):
+  `run_wave(orchestrator_backend=...)` Phase 6 routes ONE final_catch decision per
+  test-VERIFIED item through a configured orchestrator seat (schema: decisions/
+  final_catch.schema.json): merge=approve; block=verified flipped False, not shipped,
+  journal rewritten, AND written files QUARANTINED (FILE paths only — empty/dot/directory
+  pathspecs rejected with per-file error records, no cross-item destruction; tracked ->
+  git checkout, untracked deleted ONLY on clean ls-files exit-1, ambiguous = error never
+  delete; globs `:(literal)`-neutralized; backslashes normalized; outside git = honest
+  skip, surfaced in Report.blocked); escalate/undetermined/DECISION_FAILED=ship with
+  honest record (crash-only degradation — a seat outage never fabricates or blocks).
+  None/null harness backend -> "deferred", byte-identical to pre-HS-2 (no key, no
+  backend constructed). Repair stays mechanical on both seats. orchestrator_review
+  carries verdict_counts, blocked_detail {slug,reason}, seat_tokens_spent and
+  gate_status ("degraded" when all decisions failed); after decisions the ceiling is
+  re-checked with driver+seat spend (abort_reason=cost_ceiling_exceeded_after_decisions;
+  an unmetered None driver spend never crashes — seat-only spend or ledger fallback) —
+  live-seat path only. RS3-W round-2 robustness: claim gate REAL + fail-CLOSED
+  (coordination reads EventStore streams, TTL expiry, stale-lease release by slug key;
+  claim errors SKIP — never claim-less dispatch/repair); journal entries
+  fingerprint-bound + atomic (reused slug never inherits stale verified state; torn
+  writes can't corrupt resume); resumed-verified items restore filesWritten and always
+  reach a terminal shipped record (honest no_changes no-op ship — no recovery livelock);
+  dup slugs rejected at preflight; executor failures recorded, green never vacuous;
+  Windows quoting doubles only quote-preceding backslashes. RS5 claim lifecycle:
+  wave-level instance id, ttl sized to the driver command timeout (10x, 1h floor — never
+  the 300s default a real build outlives); claim HELD across build -> repair -> ship,
+  released exactly once in run_wave's finally (every exit path); repair re-dispatch and
+  ship FENCED via current_holder (lost claim = honest claim_lost abort, never
+  double-ship; try_claim retracts on error, no phantom holder). Tests: test_wave_loop_rs3.
+- **wave_bridge.py** — Phase 3: bridges AgentDriver backends to wave manifest items
+  (build_manifest_item / dispatch_item; green ONLY from test exit code, see below).
+- **backend_config.py** — HS-1 unified two-seat config: `seats.worker` + `seats.orchestrator`
+  in aesop.config.json select BOTH seats from ONE block (legacy flat block parses but is
+  INERT in the scheduler default; seats.worker wins when both present). `build_driver()` =
+  worker seat (raw-dict paths run the loader's validation); `build_orchestrator_backend()`
+  = decision seat (absent/harness/claude → null `HarnessOrchestratorBackend`;
+  openai-compatible → configured backend). Guards: base_url SSRF check incl. time-bounded
+  DNS resolution; `is_local` pinned to loopback (key waiver keys off VALIDATED is_local,
+  never URL text); `api_key_env` allowlist-primary (SECRET/TOKEN/... fragments
+  hard-reject, other key-shaped names loud NOTICE). NO seats block = byte-identical to
+  today (no key needed).
+- **context_pack.py** — build_context_pack() reads ONLY allowlisted control files
+  (STATE.md, BUILDLOG.md, tracker.json, MEMORY.md, explicit brief: under repo/conductor
+  roots) — cardinal rule 4 in code. Size-bounded, deterministic truncation, manifest.
+- **orchestrator_backend.py** — OrchestratorBackend protocol: decide_call(prompt, schema)
+  → raw text. Real impl: OpenAICompatibleOrchestratorBackend (gpt-5 temperature fallback;
+  api_key_env + is_local dummy-key; validate_base_url in __init__). HarnessOrchestratorBackend
+  = null default seat (decide_call raises: the live harness IS the orchestrator). Fake for tests.
+- **orchestrator_driver.py** — OrchestratorDriver: structured verdicts via the
+  OrchestratorBackend protocol (no AgentDriver coupling).
+- **adjudication_gate.py** — increment 3 (conservative): two-tier escalation gate — cheaper
+  challenger decides; undetermined/low-conf/disallowed-type/content-seeded-spot-check calls
+  escalate to the incumbent (frontier). Never emits an unconfident verdict as final.
+- **decisions/** — Decision type schema registry (sibling lane owns schemas; absent = optional).
+- **../tests/** — test_agent_driver (contract), test_codex_driver_e2e (offline + gated
+  live), test_wave_bridge, test_orchestrator_driver, test_adjudication_gate,
+  test_hs2_swap_proof, test_hs2_block_gate, test_wave_loop_rs3 (round-2 robustness:
+  N1/N3/N4/N5/N6/N7/N10 — all offline). Details: tests/CLAUDE.md.
 
 ## The five operations (what the wave loop needs from ANY backend)
 
-1. `probe_capabilities() -> DriverCapabilities` — honest self-report (parallel?
-   filesystem? shell? structured output? worktree? cost tracking? accuracy? →
-   recommended verification tier). Read once; everything keys off it.
-2. `dispatch_worker(request) -> WorkerResult` — spawn ONE isolated worker over a
-   prompt + owned_files + workdir; the worker may read/write files, run a shell
-   command, and return a **structured** result (extent reported by the probe).
-3. `worker_status(worker_id) -> WorkerStatus` — liveness / stall detection for
-   the watchdog.
-4. `run_command(command, cwd, shell) -> CommandResult` — ORCHESTRATOR-side
-   command execution (tests, git, verification). Distinct from a worker shell.
-5. `resolve_model(role) -> str` — map an abstract role (`worker`/`setup`/
-   `verify`) to a concrete backend model id.
-
-Optional (non-abstract): `get_tokens_spent()`.
+1. `probe_capabilities() -> DriverCapabilities` — honest self-report (parallel? fs? shell? structured? worktree? cost? accuracy? → verification tier). Read once; everything keys off it.
+2. `dispatch_worker(request) -> WorkerResult` — spawn ONE isolated worker (prompt + owned_files + workdir); may read/write/run + return a **structured** result (extent per probe).
+3. `worker_status(worker_id) -> WorkerStatus` — liveness / stall detection for the watchdog.
+4. `run_command(command, cwd, shell) -> CommandResult` — ORCHESTRATOR-side exec (tests, git, verify). Distinct from a worker shell.
+5. `resolve_model(role) -> str` — map `worker`/`setup`/`verify` to a concrete backend model id. Optional (non-abstract): `get_tokens_spent()`.
 
 ## Invariants
 
-- The wave loop calls **only** `AgentDriver` methods — never `agent()`,
-  `parallel()`, Read/Write/Bash tools, or `budget.spent()` directly. That is
-  the seam.
-- `probe_capabilities()` must be **honest**. Defaults are conservative (no
-  native abilities, accuracy 0.0, tier 4) — optimism is opt-in, never default.
-- **Weaker workers → higher verification tier.** Lower `tool_use_accuracy`
-  raises `recommended_verification_tier`. Cheaper/weaker backends RAISE the
-  orchestrator's burden; they do not lower it.
-- Unknown roles in `resolve_model()` fall back to the worker model — a mis-typed
-  role can never silently escalate cost.
-- stdlib-only (`abc`, `dataclasses`, `typing`, `subprocess`), ASCII-only,
-  Windows + Linux safe. Concrete adapters own any provider SDK, not this layer.
-
-## Per-backend capability matrix (as encoded)
-
-| Capability            | claude-code  | codex (Phase 2)       |
-|-----------------------|:------------:|:---------------------:|
-| parallel_dispatch     | yes          | no (ext loop)         |
-| worker filesystem     | yes          | no (orch injects)      |
-| worker shell          | yes          | no (orch runs)         |
-| structured_output     | yes (~perfect)| yes (JSON schema)      |
-| worktree_isolation    | yes          | no (temp-dir)         |
-| native_cost_tracking  | yes          | yes (usage metadata)   |
-| tool_use_accuracy     | ~0.99        | ~0.92                 |
-| verification_tier     | 1            | 2                     |
-
-## Phase 2 Codex Implementation Details
+- The wave loop calls **only** `AgentDriver` methods — never `agent()`, `parallel()`,
+  Read/Write/Bash tools, or `budget.spent()` directly. That is the seam.
+- The orchestrator calls **only** `OrchestratorDriver.decide()`. Context packs are allowlist-only
+  (STATE.md, BUILDLOG.md, tracker.json, MEMORY.md, explicit brief); arbitrary reads raise
+  `ContextPackViolation` — **cardinal rule 4 in code**.
+- `probe_capabilities()` must be **honest**. Defaults conservative (no native abilities, accuracy 0.0, tier 4) — optimism is opt-in, never default.
+- **Weaker workers → higher verification tier.** Lower `tool_use_accuracy` raises `recommended_verification_tier`: cheaper backends RAISE the orchestrator's burden.
+- Unknown roles in `resolve_model()` fall back to the worker model — a mis-typed role can never silently escalate cost.
+- **Fail-safe verdicts**: `OrchestratorDriver.decide()` returns DECISION_FAILED after retries exhausted; never fabricates a passing verdict (never-green principle).
+- **AdjudicationGate safety invariant** (increment 3): the final verdict is EITHER a confident
+  challenger verdict OR the incumbent's; undetermined/DECISION_FAILED/low-confidence is NEVER final.
+- stdlib-only, ASCII-only, Windows + Linux safe. Concrete adapters own any provider SDK, not this layer.
+
+## Phase 2 (Codex) + Phase 3 (Bridge) Implementation Details
+
+Capability matrix (as encoded): claude-code = parallel, worker fs/shell, worktree isolation,
+native cost, accuracy ~0.99, tier 1; codex = none natively (orchestrator injects/runs; temp-dir;
+usage-metadata cost), accuracy ~0.92, tier 2. NOTE: ClaudeCodeDriver.get_tokens_spent() is None BY CONTRACT (cost_ceiling ledger fallback).
 
 Codex driver (Tier 2): injects file contents into prompt, calls OpenAI Chat Completions via injectable transport, validates JSON with bounded retry, enforces ownership. CRITICAL: Green = exit 0 only. Verification policy: tier 2 -> {validate_all_json:True, spot_check_frac:0.50, repair_cap:2, require_adversarial_review:True}. **P1 Security**: Default model map uses gpt-4o-mini (worker, supports json_schema); init-time guard rejects models lacking json_schema support unless `allow_unverified_models=True` (P1 gate: prevent gpt-3.5-turbo silent failures).
 
-## Phase 3 Bridge Implementation Details
-
-Connects AgentDriver backends to wave-flat-dispatch manifest items; verification
-tier is driven by backend capability, not config.
-
-**build_manifest_item(driver, item) -> dict**: enriches a backlog item with model
-(driver.resolve_model), verificationTier (probe), and the four policy knobs from
-verification_policy(caps) — RESOLVED ONCE, carried as literal manifest fields so the
-template cannot recompute/drift; tier-1/Claude path stays byte-identical (repairCap=1,
-requireAdversarialReview=false, spotCheckFrac=0.10, validateAllJson=false).
-
-**dispatch_item(driver, item) -> dict**:
-- Routes by worker_filesystem_access: True→{route:'harness'}; False→orchestrator-managed
-  (dispatch_worker + run_command test). Returns {route, ok, testExit, filesWritten, verified, ...}.
-- HONESTY: Green (ok=True) ONLY on test exit 0; never from model's done:true. No testCmd
-  → unverified (ok=False, verified=False, reason='no_test_command'): "no test" ≠ "verified."
-- Fail-safe: exception → ok=False, verified=False. Ownership at driver level.
-
-**Tests**: prove a non-Claude backend (Codex + FakeTransport) takes a RED unittest
-stub, applies a fix, runs the test, and returns ok=True ONLY because the test passed
-(exit 0). All offline, no API key, no network.
+**build_manifest_item(driver, item)**: enriches a backlog item with model, verificationTier,
+and the four verification_policy knobs — resolved ONCE as literal manifest fields (no
+recompute drift); Claude tier-1 path stays byte-identical. **dispatch_item(driver, item)**:
+routes by worker_filesystem_access. HONESTY: ok=True ONLY on test exit 0, never from the
+model's done:true; no testCmd -> ok/verified=False (reason='no_test_command'); exception ->
+fail-safe False. Ownership enforced at driver level. Offline tests prove Codex+FakeTransport
+takes a RED stub to green via real test exit 0.
 
 ## Wave Scheduler (WS3a Pilot) + GATE-1 Handoff Kit
 
 **wave_scheduler.py** — single-cycle backlog orchestration: intake up to N file-disjoint todo items from tracker.json (empty/missing ownsFiles REJECTED; paths normalized posix+casefold-on-Windows before overlap checks; required fields pre-validated) -> manifest via build_manifest_item (model + verificationTier from driver.probe) -> HALT + cost-ceiling gates (fail-CLOSED: module import failure or check exception = abort with honest Report, phase=gate_unavailable) -> run_wave (recovery journal + git ship) -> STOP before merge; Report JSON with per-item observability (GATE-1). After ship, selected items atomically marked in_progress in tracker (temp+os.replace; dry-run never mutates) so a second run cannot double-dispatch.
 
-**CLI** (GATE-1): `python driver/wave_scheduler.py --tracker <path> --max-items N --dry-run|--execute --driver claude|codex` (default: claude). For codex+execute, requires OPENAI_API_KEY env var; dry-run works without it.
-
-**Tests** (35+): disjoint/normalization/rejection, gate fail-closed, dry-run, GATE-1 per-item/driver/ceiling/codex tests; module-tmpdir hygiene; all TestCase.
+**CLI** (HS-1): `python driver/wave_scheduler.py --tracker <path> --max-items N --dry-run|--execute [--driver claude|codex] [--config <path>]`. Default: worker seat from aesop.config.json seats.worker ONLY (no seats block → claude; a bare legacy flat block stays inert — migrate to seats.worker) — the seats path also reaches openai-compatible; `--driver` OVERRIDES the config. Hosted seat + --execute requires the seat's api_key_env (is_local: none); dry-run never needs a key. HS-2: seats.orchestrator resolved by `resolve_orchestrator_backend()` (absent/harness/claude → None = live harness stays orchestrator; openai-compatible → live backend, same --execute key gate) and passed into run_wave; Report JSON shape is IDENTICAL either way (swap transparency — seat activity lives in run_wave's result: orchestrator_review + per-item final_catch, plus a stderr notice).
 
-**Invariants**: stdlib-only, ASCII, Windows+Linux safe (list-form subprocess); manifest items carry resolved policy knobs from verification_policy (no recompute drift); merge stays manual in the pilot.
+**Tests** (35+): disjoint/normalization/rejection, gate fail-closed, dry-run, GATE-1 per-item/driver/ceiling/codex tests; module-tmpdir hygiene; all TestCase. **Invariants**: stdlib-only, ASCII, Windows+Linux safe (list-form subprocess); manifest items carry resolved policy knobs from verification_policy (no recompute drift); merge stays manual in the pilot.
 
 ### REPORT-CONTRACT (GATE-1 Orchestrator Handoff)
 
-The scheduler emits a Report JSON consumed by the orchestrator to decide merge eligibility:
-```
-{
-  "phase": "dispatch"|"intake"|"halt"|"ceiling"|"gate_unavailable"|"manifest",
-  "wave_id": "<uuid>",
-  "items_selected": ["id1", "id2"],  // IDs selected for this wave
-  "items_shipped": [
-    { "slug": "feat/x", "backend": "claude-code|codex", "tier": 1|2|3|4, "verified": true|false (false = NOT PROVEN, not necessarily failed; tier null = no build record), "testExit": 0|1|null },
-    ...
-  ],
-  "merged": false, "tracker_update_attempted": true|absent, "tracker_unmapped_slugs": [..]|absent (LOUD -> success false),  // pilot always false (manual merge)
-  "success": true|false,
-  "timestamp": "<ISO8601>",
-  "branch": "<branch_name>",  // optional; set when ship succeeds
-  "sha": "<git_sha>",  // optional; set when ship succeeds
-  "halt_reason": "...",  // optional; set if halted
-  "ceiling_reason": "...",  // optional; ceiling exceeded before dispatch (abort-before-dispatch only)
-  "error": "..."  // optional; unexpected error
-}
-```
-Ceiling semantics: checked BEFORE run_wave dispatch (phase=ceiling, ceiling_reason populated). Mid-wave ceiling trips are run_wave's responsibility. items_shipped[] carries per-item observability: slug, backend (driver name), tier (verificationTier), verified (from test exit 0), testExit (test command exit code or null if not run).
+Scheduler emits a Report JSON the orchestrator uses for merge eligibility. Fields: phase
+(dispatch|intake|halt|ceiling|gate_unavailable|manifest), wave_id, items_selected[],
+items_shipped[] ({slug, backend, tier 1-4|null, verified — test-exit-0-only, false = NOT
+PROVEN, testExit}), merged (pilot: always false, manual merge), success, timestamp,
+branch/sha (set on ship), halt_reason/ceiling_reason/error (optional). Live seat ONLY
+(HS-2 block gate; default shape unchanged): blocked[] ({slug, reason, quarantine:
+clean|errors|skipped|unknown, + quarantine_errors / quarantine_skipped_reason —
+a failed quarantine means refused code may still be in the tree}; item also gets
+TERMINAL tracker status "blocked" — never re-selected) + orchestrator_gate {seat, model,
+decisions, verdict_counts, blocked, decision_failed, seat_tokens_spent, status
+active|degraded|no_decisions}; any block -> success false. Ceiling is checked
+BEFORE run_wave dispatch (phase=ceiling); mid-wave trips are run_wave's responsibility.
+Tracker sync: LOUD on unmapped slugs (tracker_unmapped_slugs -> success false). Full JSON
+shape lives in wave_scheduler.py's module docstring.
 
 ## Status
 
-- **Phase 1**: shipped. Interface + Claude reference adapter + contract tests.
-- **Phase 2**: shipped. Codex OpenAI Chat Completions implementation. Offline tests GREEN.
-- **Phase 3**: shipped. Wave bridge: driver → manifest, orchestrator-side dispatch.
-  Proves non-Claude backends drive items end-to-end with honest green (test exit 0 only).
-- **Wave Scheduler (WS3a) + GATE-1**: shipped. Single-cycle orchestration: intake → manifest → dispatch → report (manual merge). Per-item observability, driver injection (--driver claude|codex), ceiling semantics documented.
+Phases 1-3 + Wave Scheduler (WS3a/GATE-1) + HS-1 + HS-2 seat swap shipped (proof:
+bench/results/hs2-swap-proof-2026-07-25.md; merge manual in pilot) + RS3-W round-2
+robustness + RS5 claim lifecycle (see wave_loop.py bullet).

package/driver/README.md

@@ -225,7 +225,45 @@ switch backends without changing code. The configuration is **offline-safe**:
 building a driver requires no API key; keys are read from environment variables
 at call time during live dispatch.
 
-**Configuration schema** (backend block):
+**Unified two-seat config (0.4.0, HS-1)** — one namespaced block selects BOTH
+seats: `seats.worker` (the coding agents; same fields as the legacy block
+below, wins over it when both are present) and `seats.orchestrator` (the
+decision seat; `"harness"` = the live Claude Code session, the default;
+`"openai-compatible"` routes `OrchestratorDriver.decide()` to an API model
+with `model`/`base_url`/`api_key_env`/`is_local`):
+
+```json
+{
+  "seats": {
+    "worker": { "backend": "claude" },
+    "orchestrator": { "backend": "openai-compatible", "model": "gpt-4o-mini" }
+  }
+}
+```
+
+`build_driver(load_backend_config())` builds the worker seat;
+`build_orchestrator_backend(load_backend_config())` builds the orchestrator
+seat (returning the null `HarnessOrchestratorBackend` -- whose `decide_call`
+raises -- when the seat is absent or `harness`/`claude`). With **no** `seats`
+block, behavior is byte-identical to today: Claude Code worker + harness
+orchestrator, no OpenAI backend constructed, no key required -- and a bare
+legacy flat block (below) stays INERT in `wave_scheduler`'s default path
+(it was dead config before 0.4.0; migrate it to `seats.worker` to opt in).
+Consumers: `wave_scheduler.py` (worker seat from `seats.worker` only;
+`--driver` overrides) and the shadow adjudication tools (orchestrator seat;
+`--model` overrides).
+
+Guardrails: `base_url` is SSRF-checked (scheme, IP literals, AND
+DNS-resolved addresses; TTL-0 rebinding residual documented in
+`validate_base_url`), `is_local: true` requires a loopback `base_url`
+(localhost/127.0.0.1/::1 -- it disables the key requirement), and
+`api_key_env` must be an LLM-key-shaped name (`*_KEY`/`*_API_KEY`, no
+SECRET/TOKEN/... fragments) so a config cannot exfiltrate arbitrary env
+secrets as Bearer tokens.
+
+**Configuration schema** (legacy/flat worker block; parses + validates, honored
+by direct `build_driver()` callers, but inert in the scheduler default -- see
+above):
 ```json
 {
   "backend": "claude" | "codex" | "openai-compatible",
@@ -274,7 +312,7 @@ print(describe_backend(config))
 ```json
 {
   "backend": "codex",
-  "model": "gpt-3.5-turbo"
+  "model": "gpt-4o-mini"
 }
 ```
 - Requires `OPENAI_API_KEY` environment variable set