cowork-harness 0.17.0 → 0.19.0

package/CHANGELOG.md

@@ -6,6 +6,114 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [0.19.0] — 2026-06-30
+
+### Changed
+
+- **The default `--decider-llm` answering model is now Sonnet (`claude-sonnet-4-5`), not Haiku.** A
+  measurement on a real-doc skill with judgment-heavy gates found the prior Haiku default
+  **prose-declined ~50% of them** (replying "I don't have information…" instead of picking an option →
+  a fail-loud whiff); a Sonnet decider binds those gates and the run proceeds. ⚠️ This **raises per-gate
+  token cost** for every `--decider-llm` run, and a gate a weaker model would have prose-declined (and
+  failed fast) is now more likely to be answered and the run to continue (longer/costlier). Pin a
+  cheaper model to restore the old cost/behavior: `--decider-model <haiku-id>` or
+  `COWORK_HARNESS_DECIDER_MODEL`. (n is small — read this as "Haiku is too weak for judgment-heavy
+  gates," not a precise rate.) The whiff error now also names the `--decider-model` lever.
+  ⚠️ **False-green caveat (new guidance, no behavior change):** binding-and-proceeding is the upside,
+  but a *decided* answer is the decider's best guess from the transcript tail — it never sees the
+  mounted documents — so it can fabricate (oracle-less gate) or get a doc-answerable fact wrong, and a
+  green run resting on it is a false pass. **Script any gate whose answer feeds a _semantic_ assertion
+  (`--answer` / `--answer-policy`); reserve `--decider-llm` for structural-assertion runs.** See
+  `references/fidelity-and-answers.md` in the skill.
+- **Stall detector now also flags a stall AFTER an answered gate (H3).** The `stalled` verdict signal
+  previously fired only when a run ended on a question having made *no tool calls at all*. It now fires
+  when a run ends on a question and made **no productive tool call after its last `AskUserQuestion` gate** —
+  catching the case where the agent answers a gate, then re-asks in plain text and stops with the
+  deliverable never produced (previously a `result: "success"` false-green). The no-gate case is unchanged.
+  Re-derives identically on replay. ⚠️ This can flip a previously-green scenario to red — set
+  `allow_stall: true` to restore the prior verdict if ending on a question is the intended terminal state
+  (e.g. a skill that writes its output and then ends on a confirmation/"anything else?" question). The signal
+  is a *tool-position heuristic*, not deliverable detection, so it is imprecise in both directions: a post-gate
+  tool *call* (successful or errored) clears the flag even if nothing useful was produced (false negative);
+  and a deliverable produced *before* the final gate does NOT clear it, so a write-then-confirm-then-question
+  run is flagged (false positive). Assertions are the real guard — assert the deliverable
+  (`file_exists`/`artifact_json`), never `result` alone — and use `allow_stall` for a deliberate
+  question-terminal skill.
+
+### Fixed
+
+- **`--decider-llm` now binds an echoed grant label on a `web_fetch` approval gate** instead of failing
+  loud. The web_fetch permission-approval path matched the model's reply with an exact-label check only,
+  so a reply that echoed the option plus a self-glossed tail past a `:` boundary (e.g.
+  `Allow once: fetch this URL one time` against option `Allow once`) aborted the run — the same echo shape
+  the `AskUserQuestion` path already tolerates. The permission path now applies the same `echoPrefixMatch`
+  backstop, so the echoed label binds. Out-of-set replies still fail loud.
+- **The three non-retried `--decider-llm` transport failures now name their mitigation in the error.** A
+  timeout, a `maxBytes` overflow, and a spawn failure (e.g. `ENOENT`) forfeit the run by design — they are
+  not transient — but the surfaced message previously named only the failure. It now points at the lever:
+  `COWORK_HARNESS_LLM_TIMEOUT_MS`, `COWORK_HARNESS_LLM_MAX_BYTES`, and `PATH` / `COWORK_HARNESS_CLAUDE_BIN`
+  respectively. No retry behavior changed.
+- `doctor`'s "no token, but a Claude Code Keychain entry exists" remedy now also names the
+  `--dotenv` workaround. The macOS Keychain branch previously printed only "copy the token into
+  `./.env`", which led an operator whose token lived in a *different* file to conclude `doctor`
+  ignores `--dotenv` — it does not (the global `--dotenv <path>` is honored by `doctor` exactly as
+  by `skill`/`run`, since it loads into `process.env` before dispatch). The remedy now reads
+  "… or, if the token is already in another file, point at it: `cowork-harness --dotenv <path> <cmd>`".
+
+## [0.18.0] — 2026-06-27
+
+### Fixed
+
+- `sync` no longer flags an unknown delta when `coworkEgressAllowedHosts` is absent from
+  `config.json` (the normal state for a fresh install with no user-configured custom egress
+  hosts). Previously the absent key was treated the same as a wrong-typed value and blocked
+  the baseline write entirely.
+- `sync` now regenerates the baseline `$comment` with the current capture date instead of
+  carrying the stale string forward from the prior baseline via the `...base` spread.
+- A misplaced GLOBAL flag (the space form `--dotenv <path>` / `--run-dir <path>` placed *after* the
+  subcommand, where the pre-0.17.0 docs put `--dotenv`) now fails with a position hint —
+  `--dotenv is a GLOBAL flag and must come BEFORE the subcommand (e.g. \`cowork-harness --dotenv <path> doctor …\`)`
+  — instead of a bare `unknown flag: --dotenv` (or, for some commands, an unrelated positional /
+  "unexpected argument" error) that sent users hunting for a per-command flag that doesn't exist. The
+  hint honors `--output-format json`, never pre-empts `--version`/`--help`, and only fires for a known
+  subcommand (a junk command still gets the accurate "unknown command"). The `--dotenv=<path>` equals
+  form is not matched — to avoid hijacking a legitimate value like `--answer "--dotenv=x=y"` — so a
+  misplaced equals form still gets the plain unknown-flag rejection. (A bare `--dotenv`/`--run-dir`
+  token used as another flag's omitted value, e.g. `decide --question --dotenv`, is pre-empted by the
+  hint rather than the more specific "requires a value" error; both exit 2.)
+- `doctor`'s no-token remedies now show `--dotenv` in its correct **leading** position
+  (`cowork-harness --dotenv <path> <cmd>`). The git-worktree remedy previously printed the pre-0.17.0
+  `<cmd> --dotenv` form — which the new position hint above now rejects — so `doctor` was suggesting a
+  command the harness refuses. The generic no-token remedy also now advertises the
+  `--dotenv <path> <cmd>` form, so pointing at a non-cwd `.env` is discoverable.
+- `skill --help` / `run --help` now label `--run-dir` as a **GLOBAL** flag that must precede the
+  subcommand. It was listed in each command's local "Output:" flag block, implying
+  `skill … --run-dir <path>`, which the command rejects (`--run-dir` is honored only before the
+  subcommand, like `--dotenv`).
+
+### Changed
+
+- Baseline bumped to `desktop-1.15962.0` (agent `2.1.187`). Content is unchanged from
+  `1.15200.0` — host-loop generator, system prompt, identity, gates, and egress domains are
+  all byte-identical per asar analysis. Version and fingerprint fields only. The live-contract
+  suite was re-run green against the staged `2.1.187` agent on this baseline, so the
+  "verified end-to-end" claims are earned. Cassettes recorded against the `1.15200.0` baseline will
+  report a non-failing baseline-drift warning on `replay` (a hard fail under `--strict` /
+  `verify-cassettes`); re-record to clear, or ignore it since the asar content is byte-identical.
+
+### Documentation
+
+- Audit-validated doc/DX fixes: corrected the `docs/cassette.md` cassette-version example
+  (`6`→`7`); rewrote the `protocol-smoke` row to stop referencing a rejected
+  `transcript_no_host_path: false` line; scoped the SKILL.md "agent binary" prerequisite to the
+  sandboxed live tiers (protocol/replay need none); added `python3` to the README requirements
+  (the `lint` linter shells out to it); moved the `/plugin` slash-command block off the `bash`
+  fence; pinned `cowork-harness@>=0.17.0` in the CI recipe; noted that `chat` excludes
+  `microvm`/`cowork`; and other small corrections (see `docs/internal/2026-06-26-doc-audit-*`).
+- Refreshed all verification/version stamps that still pinned `1.15200.0` / `2.1.181`
+  (README, DESIGN, SPEC, the spawn-contract reference, `docs/cassette.md` fingerprint example,
+  and the `hostloop-prompt.ts` re-verified comment) to `1.15962.0` / `2.1.187`.
+
 ## [0.17.0] — 2026-06-26
 
 ### Upgrade notes

package/README.md

@@ -17,6 +17,7 @@ Scriptable, CI-friendly test harness that reproduces **Claude Cowork's observabl
 
 > **Requirements at a glance**
 > - **Free demo (`replay`):** Node ≥ 20 — nothing else (no Docker, token, or Claude Desktop).
+> - **`lint` (optional, token-free):** also needs **`python3`** on PATH — the scenario linter shells out to it (PyYAML is bundled); a missing `python3` is a hard `exit 127`.
 > - **Live tiers**: **Claude Desktop opened once** (stages the agent — nothing is bundled) · a **Claude token** (real per-run cost; runs take minutes — mint one with `claude setup-token`, which needs the **`claude` CLI**: `npm i -g @anthropic-ai/claude-code`) · plus a runtime — **Docker (arm64)** for `container` (default) / `hostloop`, or **Lima (Apple-VZ)** for `microvm`. The `protocol` tier skips the runtime + the staged agent but still calls a real model, so it needs the token. (`doctor --tier <t>` reports exactly what a given tier needs.)
 > - **Platform:** best on **macOS Apple Silicon**; **Windows is not supported** for the live tiers (use the token-free `replay`); `sync` and `microvm` are **macOS-arm64 only**. Full detail in [Quick start → prerequisites](#quick-start).
 
@@ -92,7 +93,7 @@ Skill testing is the headline use, but the tool is a general harness over the Co
 |---|---|---|
 | `skill <folder> "<prompt>"` | Run a local skill/plugin folder once against the staged agent | ad-hoc "is the skill alive / does it do X?" — the fast inner loop |
 | `run <scenario.yaml \| dir/>` | Run authored scenarios with `assert:` + a CI-ready exit code | you want a repeatable, **asserted regression test** |
-| `chat <folder> [prompt]` | Interactive multi-turn REPL against a skill (TTY); optional seed prompt is sent as the first turn. `--upload <file>` / `--folder <dir>` (repeatable) attach files/project folders; `--verbose` / `-V` shows thinking blocks + tool inputs; `--fidelity protocol\|container\|hostloop` | debugging a multi-turn flow by hand |
+| `chat <folder> [prompt]` | Interactive multi-turn REPL against a skill (TTY); optional seed prompt is sent as the first turn. `--upload <file>` / `--folder <dir>` (repeatable) attach files/project folders; `--verbose` / `-V` shows thinking blocks + tool inputs; `--fidelity protocol\|container\|hostloop` (no `microvm`/`cowork` in the REPL) | debugging a multi-turn flow by hand |
 | `record` / `replay` | **Record a live run once → replay it token-free, Docker-free thereafter.** `record` saves a control-protocol cassette (one scenario, or batch a `dir/`; `--rerecord-stale` refreshes only drifted ones; `--concurrency <N>` records a batch N at a time — runs are fully isolated, so it just bounds Docker/API pressure; `--max-artifact-bytes` raises the 64 KiB inline-body cap; `--dry-run` previews the scenarios + token/binary checks without recording; answer gates **live during the recording** with `--decider-dir`/`--decider-llm`/`--on-unanswered first` instead of pre-scripting them — a live decider flags the cassette non-deterministic but it still replays deterministically); `replay <file.cassette.json \| dir/>` runs a cassette **file or a `dir/` of `*.cassette.json`** deterministically (`--strict` fails on any stale one, `--fail-on-skill-drift` only on skill-source drift; staleness is also reported per-result in `staleness[]` for a JSON gate; exits on the worst verdict) | **token-free, Docker-free CI** from a once-recorded run |
 | `verify-cassettes <file\|dir>` | Token-free CI gate over committed cassettes: a privacy scan (email/currency/domain → exit 1; whole-token allows via `--allow` / class-scoped `--allow-domain` / `--allow-email` / `--allow-file`) + a staleness check (`--skip-privacy` or `--skip-staleness` to run only one half). A dir argument scans `*.cassette.json` in that dir only (**non-recursive**) | gating **committed cassettes** against PII leaks + "edited the skill, forgot to re-record" |
 | `verify-run <run-dir> <scenario.yaml>` | Re-evaluate a scenario's `assert:` (and, when the scenario declares `answers:`, whether they still match the run's actual gates) against an already-kept run dir — **no live agent, no tokens, no Docker** (~1s) | iterating on a wrong assertion or a drifted `answer` without a full live re-record |
@@ -143,12 +144,12 @@ Only the committed-cassette `replay` above is fully self-contained. Live `run`/`
 
 This repo ships a **companion skill** (`.claude/skills/cowork-harness/`) that teaches an agent how to drive the harness — author scenarios, pick a fidelity tier, script answers, place assertions in the right CI lane, and avoid the "✓ passed ≠ correct" traps. Install it via the bundled marketplace — run these **in Claude Code** (slash commands, not your shell):
 
-```bash
+```text
 /plugin marketplace add yaniv-golan/cowork-harness
 /plugin install cowork-harness@cowork-harness
 ```
 
-The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx cowork-harness@>=0.17.0` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
+The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx cowork-harness@>=0.19.0` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
 
 It also follows the open [Agent Skills](https://github.com/vercel-labs/skills) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) by pointing the `npx skills` CLI at `.claude/skills/cowork-harness` in this repo. (Working *inside* this repo, the skill auto-loads as a project skill — no install needed.)
 
@@ -336,7 +337,7 @@ assert:
 # Relative paths below resolve from THIS file's dir (absolute and ~ are used as-is).
 model: claude-opus-4-8
 effort: high
-max_thinking_tokens: 8000
+max_thinking_tokens: 8000               # kept low for example cost; real default 31999 (see docs/session.md)
 permission_mode: default
 permission_parity: cowork                   # cowork (allow unscripted tool calls, the default) | strict (deny unscripted)
 folders:
@@ -380,7 +381,7 @@ The harness builds a **clean managed `CLAUDE_CONFIG_DIR` per run** (with a gener
 
 ## What you get out (inspectable output)
 
-Every run writes to `~/.cowork-harness/runs/<scenario>/<sessionId>/` (out of any working tree; relocate with `--run-dir <path>` or `COWORK_HARNESS_RUNS_DIR`):
+Every run writes to `~/.cowork-harness/runs/<scenario>/<sessionId>/` (out of any working tree; relocate with the global `--run-dir <path>` flag — it goes *before* the subcommand — or `COWORK_HARNESS_RUNS_DIR`):
 
 ```
 events.jsonl        full stream-json event log (child→driver; the cassette source)
@@ -481,8 +482,8 @@ Most runs need **none** of these — the defaults are correct. They're grouped b
 - `COWORK_CONTAINER_RUNTIME=podman` — use Podman instead of Docker.
 - `COWORK_AGENT_IMAGE=<tag>` — override the agent image name (default `cowork-agent-base:2`); `COWORK_AGENT_BINARY=<path>` — override the auto-detected staged agent ELF.
 - `COWORK_SKIP_CAPABILITY_PROBE=1` — skip the per-run capability probe (the harness otherwise probes the agent image/VM for the document/OCR/Office capabilities the real Cowork rootfs ships and **fails a run that uses one the image omits** — a likely false negative; suppress per-scenario with `allow_missing_capability: true`, or rebuild full parity).
-- `COWORK_HARNESS_DECIDER_MODEL` — the default `--decider-llm` answering model (overridden by the `--decider-model` flag; falls back to the Haiku default). `COWORK_HARNESS_DECIDER_DIR_POLL_MS` / `_TIMEOUT_MS` — tune the `--decider-dir` rendezvous poll/backstop; `COWORK_HARNESS_DECIDER_CMD_TIMEOUT_MS` / `COWORK_HARNESS_LLM_TIMEOUT_MS` — backstop a hung `--decider-cmd` helper / `--decider-llm` model call (default 600 s, fail loud); `COWORK_HARNESS_LLM_RETRIES` — bounded retries for a transient non-zero `claude -p` exit in the `--decider-llm` transport (default 2, clamped to 0–10; set `0` to disable, e.g. deterministic CI); `COWORK_HARNESS_DIALOG_TIMEOUT_MS` — override the 6 s dialog auto-cancel.
-- `COWORK_HARNESS_RUNS_DIR` (or the `--run-dir <path>` flag) — override the default run-output root `~/.cowork-harness/runs` (kept out of any working tree so sensitive skill inputs/outputs don't land in a repo). Precedence: `--run-dir` > env > default. The root is flat and machine-global (shared across projects); pinned `--session-id` runs are guarded against cross-project overwrite, and `prune` never prunes them. In CI, set it to a workspace path (e.g. `runs`) so artifact upload can collect the runs. `COWORK_HARNESS_ALLOW_FOREIGN_RESUME=1` overrides the guard that blocks `--resume` onto another project's pinned session.
+- `COWORK_HARNESS_DECIDER_MODEL` — the default `--decider-llm` answering model (overridden by the `--decider-model` flag; falls back to the Sonnet default — pin a cheaper model for simple gates to cut cost). `COWORK_HARNESS_DECIDER_DIR_POLL_MS` / `_TIMEOUT_MS` — tune the `--decider-dir` rendezvous poll/backstop; `COWORK_HARNESS_DECIDER_CMD_TIMEOUT_MS` / `COWORK_HARNESS_LLM_TIMEOUT_MS` — backstop a hung `--decider-cmd` helper / `--decider-llm` model call (default 600 s, fail loud); `COWORK_HARNESS_LLM_RETRIES` — bounded retries for a transient non-zero `claude -p` exit in the `--decider-llm` transport (default 2, clamped to 0–10; set `0` to disable, e.g. deterministic CI); `COWORK_HARNESS_LLM_MAX_BYTES` — stdout cap on a `--decider-llm` model call (default 8 MiB, fail loud past it); `COWORK_HARNESS_DIALOG_TIMEOUT_MS` — override the 6 s dialog auto-cancel.
+- `COWORK_HARNESS_RUNS_DIR` (or the `--run-dir <path>` flag — a **global** flag that must precede the subcommand, like `--dotenv`) — override the default run-output root `~/.cowork-harness/runs` (kept out of any working tree so sensitive skill inputs/outputs don't land in a repo). Precedence: `--run-dir` > env > default. The root is flat and machine-global (shared across projects); pinned `--session-id` runs are guarded against cross-project overwrite, and `prune` never prunes them. In CI, set it to a workspace path (e.g. `runs`) so artifact upload can collect the runs. `COWORK_HARNESS_ALLOW_FOREIGN_RESUME=1` overrides the guard that blocks `--resume` onto another project's pinned session.
 - **Networking / loop:** `COWORK_EGRESS_PROXY` overrides the egress-proxy URL injected into the sandbox; `COWORK_PROXY_IMAGE` overrides the egress proxy Docker image name (default `cowork-egress-proxy:1`); `COWORK_DOCKER_NETWORK` pins the Docker network the agent container joins; `CLAUDE_FORCE_HOST_LOOP=1` forces the host-loop path regardless of the baseline's loop decision (the `cowork` tier's auto-pick). `COWORK_LIMACTL` overrides the `limactl` binary path (default `/opt/homebrew/bin/limactl`).
 - **Strictness escape hatches** (the harness fails loud by default): `COWORK_HARNESS_SOFT_MISSING=1` downgrades a missing mount source from a hard error to warn-and-exclude; `COWORK_HARNESS_ALLOW_CONFIG_DIR_WRITE=1` permits writing into an existing pinned `plugins.config_dir` (otherwise refused, to avoid clobbering a real Claude config).
 - **Staleness boundary:** the cassette-staleness skill hash uses the **git-tracked** file set of each skill/plugin source dir by default (a non-repo dir falls back to a raw walk; OS-junk like `.DS_Store` is always excluded). `COWORK_HARNESS_GITSET=0` opts out to the legacy raw walk for every dir. `COWORK_HARNESS_DEBUG_SKILLHASH=1` dumps the exact file set feeding the hash on a staleness mismatch (and flags OS-junk) so a drift source is one line. Declare per-plugin non-runtime paths in a `.cowork-hashignore` file (or the session `staleness.hash_ignore`). `COWORK_HARNESS_AGENT_SCOPE=skill` (opt-in) refines a scenario's `skills:` scope so a **skill-named** `agents/<name>.md` re-stales only that skill's cassettes instead of the whole fleet (generic agents stay shared; stamped into the fingerprint, so flipping it is a one-time re-record like `GITSET`).
@@ -565,11 +566,11 @@ This repo is built to be driven by agents, not just read by humans:
 
 ## Status
 
-**Verified end-to-end against the live staged agent (2.1.181 / asar 1.15200.0).** Host-side parity (egress/gates/mount/web_fetch facts + the reconstructed prompt appends) re-derived for 1.15200.0 via `sync` + asar inspection; the live scenario suite passes against this baseline (the staged agent ELF is unchanged at 2.1.181).
+**Verified end-to-end against the live staged agent (2.1.187 / asar 1.15962.0).** Host-side parity (egress/gates/mount/web_fetch facts + the reconstructed prompt appends) re-derived for 1.15962.0 via `sync` + asar inspection; the live scenario suite passes against this baseline (the 1.15962.0 asar content is byte-identical to 1.15200.0 — all runtime-shape fields unchanged).
 
 - ✅ **Three isolation tiers (L0/L1/L2) + two loop-mode overlays** — `protocol` (L0 control loop), `container` (L1 sandboxed arm64 + per-run default-deny egress sidecar), `microvm` (L2 real Apple-VZ Linux microVM + guest firewall); plus the loop-mode overlays `hostloop` (production split-execution: agent loop on host, shell/web via the workspace SDK-MCP server) and `cowork` (auto-picks host-loop vs container the way Cowork does — gate `1143815894`). Egress enforced at container/microvm/hostloop; `boundary-check` reports **ALL CONSTRAINTS ENFORCED**.
 - ✅ **Three-seam driver** — `AgentSession` (control protocol) → `Decider` (scripted + `on_unanswered` policy, no silent false-greens) → `Run` (turn loop, sub-agent dispatch tree, `RunRecord`). Multi-turn `chat`, deterministic cassette `record`/`replay` (no token), `run.jsonl`/`trace.json` logging with secret-scrub.
-- ✅ **Answering live questions, every way you'd need** — `--decider-llm --intent "<one line>"` (a small model picks per question, steered by your test intent — the ergonomic default; it replies with the option **number**, mapped to the exact label, so it can't whiff by parroting the label text, and an out-of-set reply fails loud; raise the answering model for ambiguous judgment gates with **`--decider-model <id>`**; non-determinism is flagged so a green isn't mistaken for a scripted pass), `--answer-policy <yaml>`/`--answer "rx=c"` (declarative regex→label, deterministic CI — accepts a **stable partial anchor** for skills whose labels drift run-to-run), `--decider-cmd '<helper>'` (custom logic — the Python `serve_decider(fn)` adapter pre-builds the wire loop so the helper writes only the decision), and **`--decider-dir <dir>`** (the *driving agent* answers each gate **in-band** with full context — it arms a Monitor on `cowork-harness gates <dir> --follow` and replies with `cowork-harness answer <dir> --gate <N> --choose <label>`; the session-under-test stays live, no resume/re-worded question — binary-verified). **multiSelect gates** are answerable on every channel — repeat `--choose` (or, for `--decider-llm`, a comma-list of option numbers; or a JSON-array reply from a `--decider-cmd` helper / hand-written `resp-N.json`); all deliver the binary-verified `", "`-joined wire shape. Every channel keeps stdout free, so all compose with `--output-format json`. A question is **never silently answered with option 1** — unhandled fails loud. Validate any decider in ~2s with `cowork-harness decide`.
+- ✅ **Answering live questions, every way you'd need** — `--decider-llm --intent "<one line>"` (a model picks per question, steered by your test intent — the ergonomic default; defaults to **Sonnet** (a weaker model prose-declines ambiguous judgment gates → fail-loud), override with **`--decider-model <id>`** — pin a cheaper model for simple gates or a stronger one for the hardest; it replies with the option **number**, mapped to the exact label, so it can't whiff by parroting the label text, and an out-of-set reply fails loud; non-determinism is flagged so a green isn't mistaken for a scripted pass — and ⚠️ a *decided* gate whose answer feeds a **semantic** assertion can still false-green, so script those), `--answer-policy <yaml>`/`--answer "rx=c"` (declarative regex→label, deterministic CI — accepts a **stable partial anchor** for skills whose labels drift run-to-run), `--decider-cmd '<helper>'` (custom logic — the Python `serve_decider(fn)` adapter pre-builds the wire loop so the helper writes only the decision), and **`--decider-dir <dir>`** (the *driving agent* answers each gate **in-band** with full context — it arms a Monitor on `cowork-harness gates <dir> --follow` and replies with `cowork-harness answer <dir> --gate <N> --choose <label>`; the session-under-test stays live, no resume/re-worded question — binary-verified). **multiSelect gates** are answerable on every channel — repeat `--choose` (or, for `--decider-llm`, a comma-list of option numbers; or a JSON-array reply from a `--decider-cmd` helper / hand-written `resp-N.json`); all deliver the binary-verified `", "`-joined wire shape. Every channel keeps stdout free, so all compose with `--output-format json`. A question is **never silently answered with option 1** — unhandled fails loud. Validate any decider in ~2s with `cowork-harness decide`.
 - ✅ **Sub-agent aggregation + `trace`** — recognizes the real `Agent` dispatch tool (binary-verified; `Task` is its alias) so `subagent_dispatched`/`dispatch_count_max` fire under `--fidelity cowork`, excluding the `TaskCreate` todo list; `cowork-harness trace <id> --view tools` digests `events.jsonl` (each tool row now shows its **result status** `ok`/`error`), and `trace <id> --view questions` shows the gate lifecycle (**question → injected answer → delivered result**); `result.json`/`--keep` surface the deep `mnt/outputs` deliverable path.
 - ✅ **Answer delivery is verified, not assumed** — an AskUserQuestion answer is injected as the binary's full tool input (`{questions, answers}`, ELF-verified), so the answer actually reaches the model; `RunResult.gateDeliveries[]` + the `gate_answers_delivered` assertion catch any delivery failure, and `RunResult.toolCounts` gives the **truthful** per-tool call count (host-routed `WebSearch` shows here, not the always-0 `usage.server_tool_use`).
 - ✅ **Binary-grounded fidelity** — cwd `/sessions/<id>`, the three-channel MCP model (`--mcp-config` honored in plain cowork mode; host/API-routed `web_fetch`; SDK-server delivery), host-loop tool partition, auth-env token-only drop, and production GrowthBook gates pinned per release.

package/baselines/desktop-1.15962.0.json

@@ -0,0 +1,220 @@
+{
+  "baselineVersion": 1,
+  "appVersion": "1.15962.0",
+  "agentVersion": "2.1.187",
+  "agentBinary": {
+    "stagedPath": "~/Library/Application Support/Claude/claude-code-vm/2.1.187/claude",
+    "format": "elf-aarch64"
+  },
+  "guest": {
+    "os": "linux",
+    "arch": "arm64",
+    "baseImage": "ubuntu:22.04"
+  },
+  "spawn": {
+    "configDirInGuest": "mnt/.claude",
+    "settingSources": [
+      "user"
+    ],
+    "permissionMode": "default",
+    "maxThinkingTokens": 31999,
+    "effortDefault": "medium",
+    "tools": [
+      "Task",
+      "Bash",
+      "Glob",
+      "Grep",
+      "Read",
+      "Edit",
+      "Write",
+      "NotebookEdit",
+      "WebFetch",
+      "TaskCreate",
+      "TaskUpdate",
+      "TaskGet",
+      "TaskList",
+      "TaskStop",
+      "WebSearch",
+      "Skill",
+      "REPL",
+      "JavaScript",
+      "AskUserQuestion",
+      "ToolSearch"
+    ],
+    "allowedTools": [
+      "Task",
+      "Bash",
+      "Glob",
+      "Grep",
+      "Read",
+      "Edit",
+      "Write",
+      "NotebookEdit",
+      "WebFetch",
+      "TaskCreate",
+      "TaskUpdate",
+      "TaskGet",
+      "TaskList",
+      "TaskStop",
+      "WebSearch",
+      "Skill",
+      "REPL",
+      "JavaScript",
+      "ToolSearch"
+    ],
+    "env": {
+      "CLAUDE_CODE_IS_COWORK": "1",
+      "CLAUDE_CODE_ENTRYPOINT": "local-agent",
+      "CLAUDE_CODE_TAGS": "lam_session_type:chat",
+      "CLAUDE_CODE_PROVIDER_MANAGED_BY_HOST": "1",
+      "CLAUDE_CODE_ENABLE_ASK_USER_QUESTION_TOOL": "true",
+      "CLAUDE_CODE_DISABLE_CRON": "1",
+      "CLAUDE_CODE_DISABLE_BACKGROUND_TASKS": "1",
+      "CLAUDE_CODE_DISABLE_AGENTS_FLEET": "1",
+      "CLAUDE_CODE_ENABLE_APPEND_SUBAGENT_PROMPT": "1",
+      "CLAUDE_CODE_ENABLE_TASKS": "true",
+      "CLAUDE_CODE_DISABLE_TERMINAL_TITLE": "1",
+      "ENABLE_PROMPT_CACHING_1H": "1",
+      "DISABLE_MICROCOMPACT": "1",
+      "MCP_CONNECTION_NONBLOCKING": "true"
+    },
+    "promptTemplate": "prompts/desktop-1.15200.0/system-prompt-append.md",
+    "subagentAppend": "prompts/desktop-1.15200.0/subagent-append-vm.md",
+    "$comment": "Binary-verified Desktop->agent spawn contract (asar 1.12603.1). See docs/cowork-spawn-contract-1.12603.1.md.",
+    "$comment_notSet": "Deliberately NOT set: CLAUDE_CODE_USE_COWORK_PLUGINS (Desktop never sets it; would flip the agent to cowork_settings.json/cowork_plugins). Host-derived (TZ, account UUIDs, WORKSPACE_HOST_PATHS, OTEL) injected at runtime, not pinned.",
+    "$comment_prompts": "Reconstructed cowork-specific sections, re-paraphrased from asar 1.15200.0 const MZi (system prompt) + generator CVr (subagent) — not the full base prompt (not cleanly extractable), and excluding the generic refusal/safety tail. The main append is delivered via the --append-system-prompt CLI flag (layered on the agent's built-in base prompt), NOT the initialize handshake; only the subagent append goes over initialize (appendSubagentSystemPrompt), gated on CLAUDE_CODE_ENABLE_APPEND_SUBAGENT_PROMPT."
+  },
+  "mountLayout": {
+    "sessionRoot": "/sessions/{sessionId}",
+    "cwd": "/sessions/{sessionId}",
+    "mntRoot": "/sessions/{sessionId}/mnt",
+    "mounts": [
+      {
+        "name": "uploads",
+        "mountPath": "uploads",
+        "mode": "r",
+        "purpose": "user-uploaded files (read-only — asar 'ro')"
+      },
+      {
+        "name": "projects",
+        "mountPath": ".projects/{projectId}",
+        "mode": "rw",
+        "purpose": "RESERVED namespace (and the separate UUID project-sync feature) — NOT the work-folder path. From Desktop 1.14271.0 selected work folders mount at mnt/<collision-resolved-basename> (dynamic, derived per session by buildLaunchPlan; see MOUNT_BARE_NAME_MIN_VERSION). This decorative row is not consumed for binding (staged paths come from plan.mounts)."
+      },
+      {
+        "name": "local-plugins",
+        "mountPath": ".local-plugins/marketplaces",
+        "mode": "r",
+        "purpose": "marketplace skills/plugins, runtime-discovered"
+      },
+      {
+        "name": "remote-plugins",
+        "mountPath": ".remote-plugins",
+        "mode": "r",
+        "purpose": "org-remote plugins, runtime-discovered"
+      },
+      {
+        "name": "outputs",
+        "mountPath": "outputs",
+        "mode": "rw",
+        "purpose": "session outputs/artifacts — delete denied by default (asar IX); rwd only when approved"
+      }
+    ]
+  },
+  "network": {
+    "mode": "gvisor",
+    "allowKind": "allowlist",
+    "allowDomains": [
+      "sentry.io",
+      "preview.claude.ai",
+      "downloads.claude.ai",
+      "api.anthropic.com",
+      "a-cdn.anthropic.com",
+      "a-api.anthropic.com",
+      "assets.claude.ai",
+      "console.anthropic.com",
+      "api-staging.anthropic.com",
+      "www.anthropic.com",
+      "docs.anthropic.com",
+      "mcp-proxy.anthropic.com",
+      "pivot.claude.ai",
+      "support.anthropic.com"
+    ]
+  },
+  "bgEnvStrip": {
+    "knownVars": [
+      "CLAUDE_CODE_OAUTH_TOKEN",
+      "CLAUDE_CODE_SESSION_KIND",
+      "CLAUDE_CODE_SESSION_ID",
+      "CLAUDE_CODE_SESSION_NAME",
+      "CLAUDE_CODE_SESSION_LOG"
+    ]
+  },
+  "$comment": "Platform baseline auto-derived by `cowork-harness sync` from a live Claude Desktop install + app.asar. VOLATILE per-release facts only. Regenerate per release; review the diff. Captured 2026-06-26 on macOS arm64.",
+  "capturedAt": "2026-06-26",
+  "platform": "darwin-arm64",
+  "settings": {
+    "autoMountFolders": {
+      "key": "autoMountFolders",
+      "default": false
+    },
+    "localAgentModeTrustedFolders": {
+      "key": "localAgentModeTrustedFolders",
+      "default": []
+    }
+  },
+  "provenance": {
+    "asarPath": "/Applications/Claude.app/Contents/Resources/app.asar",
+    "asarFingerprint": "0fe570e50d97404f",
+    "gates": {
+      "$comment": "Production GrowthBook gate states decoded from ~/Library/Application Support/Claude/fcache (standard interactive Anthropic account, 2026-06-13; binary-verified app.asar 1.12603.1). Pin per release. Behavior-affecting gates the harness models: 1143815894 (loop), 1648655587 (dispatch cap), 1978029737 (web_fetch routing). Telemetry/auth-internal gates omitted.",
+      "bridgeSdkTransport:583857784": {
+        "on": true,
+        "source": "force",
+        "value": true,
+        "note": "— Cowork uses the SDK-based transport (control protocol), confirming the harness's sdkMcpServers/mcp_message path is the production transport."
+      },
+      "hostLoop:1143815894": {
+        "on": true,
+        "source": "force",
+        "value": true
+      },
+      "taskDispatchLimiter:1648655587": {
+        "on": true,
+        "source": "force",
+        "value": {
+          "perTask": 1,
+          "global": 3
+        },
+        "note": "perTask=1 global=3 (host-side SKIP: recordSkipAndEmit/GCA.PerTaskLimit — NOT queue/deny). A dispatch session launches <=1 sub-task; <=3 concurrent globally."
+      },
+      "coworkRuntimeConfig:1978029737": {
+        "on": true,
+        "source": "force",
+        "value": {
+          "sessionsBridgePollBlockMs": 30,
+          "coworkNativeFilePreview": true,
+          "coworkWebFetchViaApi": true,
+          "coworkWebFetchPrompt": true,
+          "workspaceBashWaitLonger": true
+        },
+        "note": "coworkWebFetchViaApi=true coworkWebFetchPrompt=true workspaceBashWaitLonger=true sessionsBridgePollBlockMs=30 — web_fetch is host/API-routed (POST /api/organizations/<org>/cowork/web_fetch), NOT container egress; gated by a separate web-fetch hostname allowlist + URL provenance."
+      },
+      "cliPlugin:2307090146": {
+        "on": false,
+        "source": "defaultValue",
+        "value": false,
+        "note": "— the CLI-plugin credential broker is dark-launched off for standard interactive accounts (Ch23/L106)."
+      },
+      "pluginSyncSparkplug:2340532315": {
+        "on": true,
+        "source": "force",
+        "value": true,
+        "note": "— startup syncPlugins(); plugins load via --plugin-dir (registry inert in-VM)."
+      }
+    },
+    "eipcChannelUuid": "4f426349-8d6f-45f3-ae22-280fef323564",
+    "$comment": "eipcChannelUuid is per-build; recorded for provenance only — the harness does not use Desktop IPC."
+  },
+  "requireFullVmSandbox": null
+}

package/baselines/desktop-1.15962.1.json

@@ -0,0 +1,220 @@
+{
+  "baselineVersion": 1,
+  "appVersion": "1.15962.1",
+  "agentVersion": "2.1.187",
+  "agentBinary": {
+    "stagedPath": "~/Library/Application Support/Claude/claude-code-vm/2.1.187/claude",
+    "format": "elf-aarch64"
+  },
+  "guest": {
+    "os": "linux",
+    "arch": "arm64",
+    "baseImage": "ubuntu:22.04"
+  },
+  "spawn": {
+    "configDirInGuest": "mnt/.claude",
+    "settingSources": [
+      "user"
+    ],
+    "permissionMode": "default",
+    "maxThinkingTokens": 31999,
+    "effortDefault": "medium",
+    "tools": [
+      "Task",
+      "Bash",
+      "Glob",
+      "Grep",
+      "Read",
+      "Edit",
+      "Write",
+      "NotebookEdit",
+      "WebFetch",
+      "TaskCreate",
+      "TaskUpdate",
+      "TaskGet",
+      "TaskList",
+      "TaskStop",
+      "WebSearch",
+      "Skill",
+      "REPL",
+      "JavaScript",
+      "AskUserQuestion",
+      "ToolSearch"
+    ],
+    "allowedTools": [
+      "Task",
+      "Bash",
+      "Glob",
+      "Grep",
+      "Read",
+      "Edit",
+      "Write",
+      "NotebookEdit",
+      "WebFetch",
+      "TaskCreate",
+      "TaskUpdate",
+      "TaskGet",
+      "TaskList",
+      "TaskStop",
+      "WebSearch",
+      "Skill",
+      "REPL",
+      "JavaScript",
+      "ToolSearch"
+    ],
+    "env": {
+      "CLAUDE_CODE_IS_COWORK": "1",
+      "CLAUDE_CODE_ENTRYPOINT": "local-agent",
+      "CLAUDE_CODE_TAGS": "lam_session_type:chat",
+      "CLAUDE_CODE_PROVIDER_MANAGED_BY_HOST": "1",
+      "CLAUDE_CODE_ENABLE_ASK_USER_QUESTION_TOOL": "true",
+      "CLAUDE_CODE_DISABLE_CRON": "1",
+      "CLAUDE_CODE_DISABLE_BACKGROUND_TASKS": "1",
+      "CLAUDE_CODE_DISABLE_AGENTS_FLEET": "1",
+      "CLAUDE_CODE_ENABLE_APPEND_SUBAGENT_PROMPT": "1",
+      "CLAUDE_CODE_ENABLE_TASKS": "true",
+      "CLAUDE_CODE_DISABLE_TERMINAL_TITLE": "1",
+      "ENABLE_PROMPT_CACHING_1H": "1",
+      "DISABLE_MICROCOMPACT": "1",
+      "MCP_CONNECTION_NONBLOCKING": "true"
+    },
+    "promptTemplate": "prompts/desktop-1.15200.0/system-prompt-append.md",
+    "subagentAppend": "prompts/desktop-1.15200.0/subagent-append-vm.md",
+    "$comment": "Binary-verified Desktop->agent spawn contract (asar 1.12603.1). See docs/cowork-spawn-contract-1.12603.1.md.",
+    "$comment_notSet": "Deliberately NOT set: CLAUDE_CODE_USE_COWORK_PLUGINS (Desktop never sets it; would flip the agent to cowork_settings.json/cowork_plugins). Host-derived (TZ, account UUIDs, WORKSPACE_HOST_PATHS, OTEL) injected at runtime, not pinned.",
+    "$comment_prompts": "Reconstructed cowork-specific sections, re-paraphrased from asar 1.15200.0 const MZi (system prompt) + generator CVr (subagent) — not the full base prompt (not cleanly extractable), and excluding the generic refusal/safety tail. The main append is delivered via the --append-system-prompt CLI flag (layered on the agent's built-in base prompt), NOT the initialize handshake; only the subagent append goes over initialize (appendSubagentSystemPrompt), gated on CLAUDE_CODE_ENABLE_APPEND_SUBAGENT_PROMPT."
+  },
+  "mountLayout": {
+    "sessionRoot": "/sessions/{sessionId}",
+    "cwd": "/sessions/{sessionId}",
+    "mntRoot": "/sessions/{sessionId}/mnt",
+    "mounts": [
+      {
+        "name": "uploads",
+        "mountPath": "uploads",
+        "mode": "r",
+        "purpose": "user-uploaded files (read-only — asar 'ro')"
+      },
+      {
+        "name": "projects",
+        "mountPath": ".projects/{projectId}",
+        "mode": "rw",
+        "purpose": "RESERVED namespace (and the separate UUID project-sync feature) — NOT the work-folder path. From Desktop 1.14271.0 selected work folders mount at mnt/<collision-resolved-basename> (dynamic, derived per session by buildLaunchPlan; see MOUNT_BARE_NAME_MIN_VERSION). This decorative row is not consumed for binding (staged paths come from plan.mounts)."
+      },
+      {
+        "name": "local-plugins",
+        "mountPath": ".local-plugins/marketplaces",
+        "mode": "r",
+        "purpose": "marketplace skills/plugins, runtime-discovered"
+      },
+      {
+        "name": "remote-plugins",
+        "mountPath": ".remote-plugins",
+        "mode": "r",
+        "purpose": "org-remote plugins, runtime-discovered"
+      },
+      {
+        "name": "outputs",
+        "mountPath": "outputs",
+        "mode": "rw",
+        "purpose": "session outputs/artifacts — delete denied by default (asar IX); rwd only when approved"
+      }
+    ]
+  },
+  "network": {
+    "mode": "gvisor",
+    "allowKind": "allowlist",
+    "allowDomains": [
+      "sentry.io",
+      "preview.claude.ai",
+      "downloads.claude.ai",
+      "api.anthropic.com",
+      "a-cdn.anthropic.com",
+      "a-api.anthropic.com",
+      "assets.claude.ai",
+      "console.anthropic.com",
+      "api-staging.anthropic.com",
+      "www.anthropic.com",
+      "docs.anthropic.com",
+      "mcp-proxy.anthropic.com",
+      "pivot.claude.ai",
+      "support.anthropic.com"
+    ]
+  },
+  "bgEnvStrip": {
+    "knownVars": [
+      "CLAUDE_CODE_OAUTH_TOKEN",
+      "CLAUDE_CODE_SESSION_KIND",
+      "CLAUDE_CODE_SESSION_ID",
+      "CLAUDE_CODE_SESSION_NAME",
+      "CLAUDE_CODE_SESSION_LOG"
+    ]
+  },
+  "$comment": "Platform baseline auto-derived by `cowork-harness sync` from a live Claude Desktop install + app.asar. VOLATILE per-release facts only. Regenerate per release; review the diff. Captured 2026-06-27 on macOS arm64.",
+  "capturedAt": "2026-06-27",
+  "platform": "darwin-arm64",
+  "settings": {
+    "autoMountFolders": {
+      "key": "autoMountFolders",
+      "default": false
+    },
+    "localAgentModeTrustedFolders": {
+      "key": "localAgentModeTrustedFolders",
+      "default": []
+    }
+  },
+  "provenance": {
+    "asarPath": "/Applications/Claude.app/Contents/Resources/app.asar",
+    "asarFingerprint": "687dfcfca5bdfc0f",
+    "gates": {
+      "$comment": "Production GrowthBook gate states decoded from ~/Library/Application Support/Claude/fcache (standard interactive Anthropic account, 2026-06-13; binary-verified app.asar 1.12603.1). Pin per release. Behavior-affecting gates the harness models: 1143815894 (loop), 1648655587 (dispatch cap), 1978029737 (web_fetch routing). Telemetry/auth-internal gates omitted.",
+      "bridgeSdkTransport:583857784": {
+        "on": true,
+        "source": "force",
+        "value": true,
+        "note": "— Cowork uses the SDK-based transport (control protocol), confirming the harness's sdkMcpServers/mcp_message path is the production transport."
+      },
+      "hostLoop:1143815894": {
+        "on": true,
+        "source": "force",
+        "value": true
+      },
+      "taskDispatchLimiter:1648655587": {
+        "on": true,
+        "source": "force",
+        "value": {
+          "perTask": 1,
+          "global": 3
+        },
+        "note": "perTask=1 global=3 (host-side SKIP: recordSkipAndEmit/GCA.PerTaskLimit — NOT queue/deny). A dispatch session launches <=1 sub-task; <=3 concurrent globally."
+      },
+      "coworkRuntimeConfig:1978029737": {
+        "on": true,
+        "source": "force",
+        "value": {
+          "sessionsBridgePollBlockMs": 30,
+          "coworkNativeFilePreview": true,
+          "coworkWebFetchViaApi": true,
+          "coworkWebFetchPrompt": true,
+          "workspaceBashWaitLonger": true
+        },
+        "note": "coworkWebFetchViaApi=true coworkWebFetchPrompt=true workspaceBashWaitLonger=true sessionsBridgePollBlockMs=30 — web_fetch is host/API-routed (POST /api/organizations/<org>/cowork/web_fetch), NOT container egress; gated by a separate web-fetch hostname allowlist + URL provenance."
+      },
+      "cliPlugin:2307090146": {
+        "on": false,
+        "source": "defaultValue",
+        "value": false,
+        "note": "— the CLI-plugin credential broker is dark-launched off for standard interactive accounts (Ch23/L106)."
+      },
+      "pluginSyncSparkplug:2340532315": {
+        "on": true,
+        "source": "force",
+        "value": true,
+        "note": "— startup syncPlugins(); plugins load via --plugin-dir (registry inert in-VM)."
+      }
+    },
+    "eipcChannelUuid": "4f426349-8d6f-45f3-ae22-280fef323564",
+    "$comment": "eipcChannelUuid is per-build; recorded for provenance only — the harness does not use Desktop IPC."
+  },
+  "requireFullVmSandbox": null
+}