cowork-harness 0.10.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,113 @@ All notable changes to this project are documented here. The format is based on
4
4
  [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The project uses
5
5
  [Semantic Versioning](https://semver.org/); pre-1.0 minor versions may include breaking changes.
6
6
 
7
+ ## [0.12.0] — 2026-06-24
8
+
9
+ ### Added
10
+
11
+ - **`record` now shows the offered options when a scripted answer matches none.** A `choose:` that names no
12
+ offered option previously failed with just "matched no offered option" — you had to dig through
13
+ `events.jsonl` for the real labels. The error now lists the **valid labels** and suggests the **closest
14
+ match**, so you can fix the anchor from the error alone. (The labels were already on the error object; the
15
+ record path just wasn't printing them — `run`/`skill` already did.)
16
+ - **`doctor` detects the git-worktree `.env` trap.** Running from a git worktree where `./.env` is gitignored
17
+ (so absent) yields "no token"; when the **main checkout** has a `.env`, `doctor` now points you at
18
+ `--dotenv <main>/.env` instead of the generic remedy. (Keychain hint still takes precedence on macOS.)
19
+ - **`scaffold` stamps a provenance header on generated scenarios.** Output now carries a
20
+ `# generated by cowork-harness v<ver> (scaffold)` comment (a comment, since the scenario schema is
21
+ `additionalProperties: false`), mirroring the cassette's `generator`/`$schema`/`cassetteVersion` provenance.
22
+ The shipped `example-pdf-skill` replay cassette — previously a hand-authored legacy fixture with no
23
+ signature and a stale embedded scenario — was re-recorded live so it now carries a genuine v6 signature +
24
+ fingerprint/artifacts/userVisibleRoots and matches the current scenario YAML (replays green;
25
+ `verify-cassettes` clean).
26
+
27
+ ### Changed
28
+
29
+ - **`verify-run` answer-coverage now refuses against a *stale* kept run.** Every run persists a skill
30
+ fingerprint in `result.json`; on the answer-coverage path (`answers:` declared), `verify-run` recomputes it
31
+ live and, if the skill source changed since the run was kept, **exits 2** ("the kept run predates the current
32
+ skill") instead of vouching for answers against a stale gate snapshot. ⚠️ This closes a false-green: a
33
+ reworded/moved gate after `--keep` would previously green against the old labels. The plain `assert:`-only
34
+ re-eval (no `answers:`) is unaffected; a kept run recorded by an older harness (no fingerprint) → a warning,
35
+ not a refusal. `RunResult.fingerprint` is the new persisted field.
36
+
37
+ ### Parity
38
+
39
+ - **Synced the platform baseline to Claude Desktop 1.15200.0** (`baselines/desktop-1.15200.0.json`). The
40
+ staged agent ELF is **unchanged (2.1.181)** — the bump is host-side: `sync` re-derived egress / gates /
41
+ mount / web_fetch facts (no `unknown delta`; only the `asarFingerprint` moved). Re-verified end-to-end —
42
+ the live scenario suite (`protocol` + `container` tiers) passes against the new baseline.
43
+ - **Re-paraphrased the reconstructed system-prompt + subagent appends for 1.15200.0**
44
+ (`baselines/prompts/desktop-1.15200.0/`). Cowork's identity constant was rewritten **first→third person**
45
+ (it no longer says "powering Cowork mode"; the load-bearing "is NOT Claude Code" correction is kept);
46
+ `file_handling` became a scratchpad-vs-workspace split, and `working_with_user_files` / `product_information`
47
+ blocks were added; the subagent-append VM clause changed to "not on the user's real computer." Host-only
48
+ affordances (`request_cowork_directory`, `computer://` links) are described as behavior, **not** injected as
49
+ instructions, so they don't induce dead tool calls on the container / microvm / protocol tiers. The host-loop
50
+ `## Shell access` generator was re-verified **byte-identical** (one new conditional sentence stays omitted —
51
+ our single-container topology never triggers it; comment + verification stamp refreshed).
52
+
53
+ ### Fixed
54
+
55
+ - **Baseline-bump-stable staleness tests.** `staleness-roundtrip`, `manifest`, and `agent-scope` round-trip
56
+ tests hardcoded the baseline version (`1.14271.0`), so adding a new latest baseline re-staled them; they now
57
+ source it from `loadBaseline("latest")`, so a parity bump no longer breaks the green round-trips.
58
+ - **`example-pdf-skill` asserts the workspace deliverable correctly.** The scenario connects a folder, so
59
+ `{{workspaceFolder}} = mnt/project` and the model writes the deliverable into the folder
60
+ (`mnt/project/outputs/actions.md`), not `mnt/outputs` — but the assert was `file_exists: outputs/actions.md`
61
+ (anchored at `mnt/`), so it failed. Switched to `user_visible_artifact: project/outputs/actions.md` (spans
62
+ the user-visible roots). A pre-existing scenario bug (fails identically on the old prompt), not a parity
63
+ regression; the `{{workspaceFolder}}` resolution is faithful to the Desktop builder `y8r`. Docs + skill now
64
+ teach `user_visible_artifact` vs `file_exists` for folder-connected deliverables.
65
+
66
+ ### Docs
67
+
68
+ - **Baseline-pin sweep to `desktop-1.15200.0`** across README / DESIGN / SPEC / spawn-contract / skill docs
69
+ (agent ELF still 2.1.181). The `≥1.14271.0` mount/bare-name gate-boundary references are intentionally left
70
+ as-is — that boundary did not move.
71
+ - Skill + scenario docs: the gate-centric answer-coverage currency rule; the cheap `--keep` → `trace
72
+ --view questions`/`verify-run` authoring loop (no token-free gate probe exists — gates are model-decided);
73
+ the mismatch-vs-unanswered hard-fail gotcha; "anchor only assert-relevant gates, prefer `on_unanswered:
74
+ first` elsewhere"; the keep-going batch semantics (`record <dir>`/`--rerecord-stale` record all and report
75
+ at the end — no one-at-a-time wrapper needed); and the git-worktree `.env` gotcha.
76
+
77
+ ### Security
78
+
79
+ - **Hardened HTML-comment stripping in prompt assets (CodeQL `js/incomplete-multi-character-sanitization`).**
80
+ A one-shot `.replace(/<!--[\s\S]*?-->/g, "")` is incomplete multi-character sanitization: removing an inner
81
+ comment can recombine surrounding fragments into a fresh `<!--` the single pass leaves behind (e.g.
82
+ `<!<!-- -->-- x -->` → `<!-- x -->`). `stripComments` now loops until the string stabilizes and is shared
83
+ from `prompt.ts`; `hostloop.ts` reuses it instead of a duplicated inline regex. Resolves the finding at both
84
+ call sites; a regression test covers the recombination case.
85
+
86
+ ## [0.11.0] — 2026-06-24
87
+
88
+ ### Added
89
+
90
+ - **`record --concurrency <N>` — parallel fleet re-records.** A directory batch (or `--rerecord-stale`) can
91
+ now record N cassettes at a time (`record cassettes/ --rerecord-stale --concurrency 3`) instead of one ~7–8
92
+ min run after another. Every run is already fully isolated (its own per-run Docker networks + egress proxy,
93
+ its own session dir), so parallelism is safe; `--concurrency` is purely a **bound** against Docker's address
94
+ pool and model API rate limits. Default `1` (unchanged behavior + ordered output), max `8`. A dir batch
95
+ where two scenarios' `name:` slugify to the same cassette path is now rejected up front (they would clobber
96
+ each other — a pre-existing footgun parallelism would have surfaced).
97
+ - **Opt-in per-skill agent scoping for cassette staleness (`COWORK_HARNESS_AGENT_SCOPE=skill`).** By default a
98
+ plugin's `agents/` directory is a fleet-wide staleness root, so editing one skill's sub-agent contract
99
+ (`agents/cap-table.md`) re-stales *every* cassette. With this env set, a **skill-named** `agents/<name>.md`
100
+ is treated as skill `<name>`'s private input (refining a scenario's `skills:` scope), so it re-stales only
101
+ that skill's cassettes; generic (non-skill-named) agents stay shared. The setting is stamped into the
102
+ cassette fingerprint (`agentScope`), so flipping it is an honest one-time "re-record under the same setting"
103
+ (like `COWORK_HARNESS_GITSET`); cassettes recorded without it are unaffected. Caveat: assumes an agent named
104
+ after a skill belongs to that skill — keep it off if you share a skill-named agent across skills.
105
+
106
+ ### Fixed
107
+
108
+ - **Clarified `verify-run` answer-coverage docs (gate-centric, not rule-centric).** `verify-run` checks that
109
+ every gate the run *actually fired* is covered by a matching `answer`; it does **not** penalize answer rules
110
+ that no fired gate matched (e.g. rules for conditional gates that didn't fire). The behavior was always
111
+ correct; `docs/scenario.md` now states it precisely (a scenario with 5 rules whose run fired 2 gates passes
112
+ at "2/2 matched").
113
+
7
114
  ## [0.10.0] — 2026-06-23
8
115
 
9
116
  ### Added
package/README.md CHANGED
@@ -24,13 +24,18 @@ Scriptable, CI-friendly test harness that reproduces **Claude Cowork's observabl
24
24
 
25
25
  > **What this is and isn't.** This is an *emulator of the contract*, not the Desktop runtime. Cowork's real session control plane lives behind the Desktop renderer's IPC (per-build UUID + `senderFrame` origin checks) and the app ships with remote debugging disabled (verified: `--remote-debugging-port` opens no listener; Electron `EnableNodeCliInspectArguments` fuse is OFF). So you **cannot** drive the real Apple Virtualization.framework microVM from a script. What you *can* faithfully reproduce is everything that actually changes how a **skill** behaves: the same agent binary in **cowork mode** (`CLAUDE_CODE_IS_COWORK=1` — there is no `--cowork` flag), the same mount layout, the same egress allowlist, and the same permission/question protocol. That's what this project does.
26
26
 
27
- **Zero-friction preview — no token, no Docker.** A committed cassette replays on a fresh clone:
27
+ **Zero-friction preview — no token, no Docker.** A committed cassette replays from a fresh clone (the example
28
+ cassette ships in the repo; just Node ≥ 20):
28
29
 
29
30
  ```bash
30
- npm install -g cowork-harness
31
- cowork-harness replay examples/replays/example-pdf-skill.cassette.json
31
+ git clone https://github.com/yaniv-golan/cowork-harness && cd cowork-harness
32
+ npm ci && npm run build
33
+ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
32
34
  ```
33
35
 
36
+ (Installing globally — `npm install -g cowork-harness` — gives you the `cowork-harness` CLI for your own
37
+ scenarios and cassettes; the bundled example above is replayed from a checkout.)
38
+
34
39
  Full setup → [Quick start](#quick-start).
35
40
 
36
41
  ---
@@ -88,14 +93,14 @@ Skill testing is the headline use, but the tool is a general harness over the Co
88
93
  | `skill <folder> "<prompt>"` | Run a local skill/plugin folder once against the staged agent | ad-hoc "is the skill alive / does it do X?" — the fast inner loop |
89
94
  | `run <scenario.yaml \| dir/>` | Run authored scenarios with `assert:` + a CI-ready exit code | you want a repeatable, **asserted regression test** |
90
95
  | `chat <folder> [prompt]` | Interactive multi-turn REPL against a skill (TTY); optional seed prompt is sent as the first turn. `--upload <file>` / `--folder <dir>` (repeatable) attach files/project folders; `--verbose` / `-V` shows thinking blocks + tool inputs; `--fidelity protocol\|container\|hostloop` | debugging a multi-turn flow by hand |
91
- | `record` / `replay` | `record` saves a control-protocol cassette (one scenario, or batch a `dir/`; `--rerecord-stale` refreshes only drifted ones; `--max-artifact-bytes` raises the 64 KiB inline-body cap; `--dry-run` previews the scenarios + token/binary checks without recording; answer gates **live during the recording** with `--decider-dir`/`--decider-llm`/`--on-unanswered first` instead of pre-scripting them — a live decider flags the cassette non-deterministic but it still replays deterministically); `replay <file.cassette.json \| dir/>` runs a cassette **file or a `dir/` of `*.cassette.json`** deterministically (`--strict` fails on a stale one; exits on the worst verdict) | **token-free, Docker-free CI** from a once-recorded run |
96
+ | `record` / `replay` | `record` saves a control-protocol cassette (one scenario, or batch a `dir/`; `--rerecord-stale` refreshes only drifted ones; `--concurrency <N>` records a batch N at a time — runs are fully isolated, so it just bounds Docker/API pressure; `--max-artifact-bytes` raises the 64 KiB inline-body cap; `--dry-run` previews the scenarios + token/binary checks without recording; answer gates **live during the recording** with `--decider-dir`/`--decider-llm`/`--on-unanswered first` instead of pre-scripting them — a live decider flags the cassette non-deterministic but it still replays deterministically); `replay <file.cassette.json \| dir/>` runs a cassette **file or a `dir/` of `*.cassette.json`** deterministically (`--strict` fails on a stale one; exits on the worst verdict) | **token-free, Docker-free CI** from a once-recorded run |
92
97
  | `verify-cassettes <file\|dir>` | Token-free CI gate over committed cassettes: a privacy scan (email/currency/domain → exit 1; whole-token allows via `--allow` / class-scoped `--allow-domain` / `--allow-email` / `--allow-file`) + a staleness check (`--skip-privacy` or `--skip-staleness` to run only one half). A dir argument scans `*.cassette.json` in that dir only (**non-recursive**) | gating **committed cassettes** against PII leaks + "edited the skill, forgot to re-record" |
93
98
  | `verify-run <run-dir> <scenario.yaml>` | Re-evaluate a scenario's `assert:` (and, when the scenario declares `answers:`, whether they still match the run's actual gates) against an already-kept run dir — **no live agent, no tokens, no Docker** (~1s) | iterating on a wrong assertion or a drifted `answer` without a full live re-record |
94
99
  | `trace <run-id>` | Digest a run's `events.jsonl` (`--view tools\|questions\|dispatches`; legacy `--tools` / `--gates` / `--dispatches` aliases still accepted) | "how many sub-agents *actually* dispatched, and which?" |
95
100
  | `scaffold <run-id>` | Turn a kept run into a starter scenario YAML (gates→answers, artifacts→`file_exists`); `--from-run <id>` is a deprecated alias for the positional | authoring a scenario from a real run instead of guessing |
96
101
  | `python3 …/scenario.py scaffold --name <name> --skill <dir>` | Generate a starter scenario skeleton from scratch (the `…` is `.claude/skills/cowork-harness/scripts/`) | starting a new scenario when you have no prior run |
97
102
  | `lint <scenario.yaml \| dir/>…` | Check scenarios for silent false-greens — assertions placed on the wrong CI lane, mixed content/live keys, missing `controlOut`-required keys (files or a directory of `*.yaml`/`*.yml`; bundled `scenario.py`; needs python3 — PyYAML is bundled) | before committing a new scenario or after changing assertions |
98
- | `assertions --list` | List the available scenario assertions (generated from the schema); `assert` is a deprecated alias | "what can I assert?" without grepping the source |
103
+ | `assertions --list` | List the available scenario assertions (generated from the schema) | "what can I assert?" without grepping the source |
99
104
  | `decide` | Validate a decider against a sample question in ~2 s (no run) | sanity-check a `--decider-*` / `--answer` wiring before a long run |
100
105
  | `gates` / `answer` | Stream / answer in-band gates for `--decider-dir` | a **driving agent** answers live questions via a Monitor |
101
106
  | `boundary-check [baseline] [--session <file>]` | Prove the sandbox enforces Cowork's limitations; `--session` folds a session's `egress.extra_allow` into the probe allowlist | verifying the harness's own fidelity |
@@ -202,7 +207,7 @@ This repo ships a **companion skill** (`.claude/skills/cowork-harness/`) that te
202
207
  /plugin install cowork-harness@cowork-harness
203
208
  ```
204
209
 
205
- The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx cowork-harness@>=0.9.0` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
210
+ The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx cowork-harness@>=0.12.0` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
206
211
 
207
212
  It also follows the open [Agent Skills](https://github.com/vercel-labs/skills) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) by pointing the `npx skills` CLI at `.claude/skills/cowork-harness` in this repo. (Working *inside* this repo, the skill auto-loads as a project skill — no install needed.)
208
213
 
@@ -261,7 +266,7 @@ cowork-harness lint examples/scenarios/*.yaml
261
266
  > `serializeDecision` as a token-free O7 guard (the AskUserQuestion `{questions,answers}` answer-shape
262
267
  > invariant). Evaluated on replay: `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`,
263
268
  > `result`, and the verdict modifiers `allow_permissive_auto_allow` / `allow_missing_capability` /
264
- `allow_l0_plugin_divergence` / `allow_stall` (no-op passes, kept on replay). **`question_asked`,
269
+ > `allow_l0_plugin_divergence` / `allow_stall` (no-op passes, kept on replay). **`question_asked`,
265
270
  > `questions_count_max`, and `gate_answers_delivered` are also evaluated —
266
271
  > but only when the cassette carries `controlOut` (full-fidelity)**; old cassettes without it get a
267
272
  > loud warning and those three keys are excluded (not vacuously passed). **Filesystem assertions
@@ -293,7 +298,7 @@ Configuration splits the way Cowork itself splits — *what you set up before th
293
298
  # scenarios/pdf.yaml ← the filename is the test's identity (name: is an optional override)
294
299
  baseline: latest # platform baseline (auto-synced from Desktop)
295
300
  session: ../sessions/default.yaml # pre-prompt setup, resolved relative to THIS file
296
- fidelity: container # protocol | container | microvm
301
+ fidelity: container # protocol | container | microvm | hostloop | cowork
297
302
 
298
303
  prompt: |
299
304
  Summarize report.pdf and write the action items to outputs/actions.md
@@ -471,7 +476,7 @@ Unit tests cover the scripted-answer logic, the egress allowlist matcher, the se
471
476
  - `COWORK_HARNESS_RUNS_DIR` (or the `--run-dir <path>` flag) — override the default run-output root `~/.cowork-harness/runs` (kept out of any working tree so sensitive skill inputs/outputs don't land in a repo). Precedence: `--run-dir` > env > default. The root is flat and machine-global (shared across projects); pinned `--session-id` runs are guarded against cross-project overwrite, and `runs gc` never prunes them. In CI, set it to a workspace path (e.g. `runs`) so artifact upload can collect the runs. `COWORK_HARNESS_ALLOW_FOREIGN_RESUME=1` overrides the guard that blocks `--resume` onto another project's pinned session.
472
477
  - **Networking / loop:** `COWORK_EGRESS_PROXY` overrides the egress-proxy URL injected into the sandbox; `COWORK_PROXY_IMAGE` overrides the egress proxy Docker image name (default `cowork-egress-proxy:1`); `COWORK_DOCKER_NETWORK` pins the Docker network the agent container joins; `CLAUDE_FORCE_HOST_LOOP=1` forces the host-loop path regardless of the baseline's loop decision (the `cowork` tier's auto-pick). `COWORK_LIMACTL` overrides the `limactl` binary path (default `/opt/homebrew/bin/limactl`).
473
478
  - **Strictness escape hatches** (the harness fails loud by default): `COWORK_HARNESS_SOFT_MISSING=1` downgrades a missing mount source from a hard error to warn-and-exclude; `COWORK_HARNESS_ALLOW_CONFIG_DIR_WRITE=1` permits writing into an existing pinned `plugins.config_dir` (otherwise refused, to avoid clobbering a real Claude config).
474
- - **Staleness boundary:** the cassette-staleness skill hash uses the **git-tracked** file set of each skill/plugin source dir by default (a non-repo dir falls back to a raw walk; OS-junk like `.DS_Store` is always excluded). `COWORK_HARNESS_GITSET=0` opts out to the legacy raw walk for every dir. `COWORK_HARNESS_DEBUG_SKILLHASH=1` dumps the exact file set feeding the hash on a staleness mismatch (and flags OS-junk) so a drift source is one line. Declare per-plugin non-runtime paths in a `.cowork-hashignore` file (or the session `staleness.hash_ignore`).
479
+ - **Staleness boundary:** the cassette-staleness skill hash uses the **git-tracked** file set of each skill/plugin source dir by default (a non-repo dir falls back to a raw walk; OS-junk like `.DS_Store` is always excluded). `COWORK_HARNESS_GITSET=0` opts out to the legacy raw walk for every dir. `COWORK_HARNESS_DEBUG_SKILLHASH=1` dumps the exact file set feeding the hash on a staleness mismatch (and flags OS-junk) so a drift source is one line. Declare per-plugin non-runtime paths in a `.cowork-hashignore` file (or the session `staleness.hash_ignore`). `COWORK_HARNESS_AGENT_SCOPE=skill` (opt-in) refines a scenario's `skills:` scope so a **skill-named** `agents/<name>.md` re-stales only that skill's cassettes instead of the whole fleet (generic agents stay shared; stamped into the fingerprint, so flipping it is a one-time re-record like `GITSET`).
475
480
  - **`skill` / `chat` defaults:** `COWORK_HARNESS_FIDELITY` sets the default fidelity tier for ad-hoc `skill`/`chat` runs (a `--fidelity` flag or a scenario's `fidelity:` still wins); `COWORK_HARNESS_MODEL` sets the default model; `COWORK_HARNESS_OUTPUT_FORMAT` (`text`|`json`) sets the default output format. Each is overridden by the matching explicit flag.
476
481
  - **Secret scrubbing:** `COWORK_HARNESS_SCRUB_KEYS=<KEY1,KEY2>` adds extra env-var names whose values are redacted from logs (beyond the known auth tokens + `ANTHROPIC_CUSTOM_HEADERS`); `COWORK_HARNESS_SCRUB_VALUES=<v1,v2>` redacts literal values regardless of env. **Committed-cassette redaction:** `COWORK_HARNESS_REDACT_PATTERNS=<rx1,rx2>` / `COWORK_HARNESS_REDACT_KEYS=<k1,k2>` extend the privacy layer that scrubs recorded `controlOut` before a cassette is written for commit.
477
482
  - L2 microVM: `COWORK_VM_GATEWAY` overrides the Lima host-proxy gateway IP (default `192.168.5.2`); `COWORK_VM_PROXY_PORT` the proxy port. The Lima instance is named `cowork-vm-<config-hash>` (a config change → a fresh VM); `COWORK_LIMA_INSTANCE` pins a fixed name, and `vm prune` removes orphaned ones.
@@ -549,7 +554,7 @@ This repo is built to be driven by agents, not just read by humans:
549
554
 
550
555
  ## Status
551
556
 
552
- **Verified end-to-end against the live staged agent (2.1.181 / asar 1.14271.0).**
557
+ **Verified end-to-end against the live staged agent (2.1.181 / asar 1.15200.0).** Host-side parity (egress/gates/mount/web_fetch facts + the reconstructed prompt appends) re-derived for 1.15200.0 via `sync` + asar inspection; the live scenario suite passes against this baseline (the staged agent ELF is unchanged at 2.1.181).
553
558
 
554
559
  - ✅ **Three isolation tiers (L0/L1/L2) + two loop-mode overlays** — `protocol` (L0 control loop), `container` (L1 sandboxed arm64 + per-run default-deny egress sidecar), `microvm` (L2 real Apple-VZ Linux microVM + guest firewall); plus the loop-mode overlays `hostloop` (production split-execution: agent loop on host, shell/web via the workspace SDK-MCP server) and `cowork` (auto-picks host-loop vs container the way Cowork does — gate `1143815894`). Egress enforced at container/microvm/hostloop; `boundary-check` reports **ALL CONSTRAINTS ENFORCED**.
555
560
  - ✅ **Three-seam driver** — `AgentSession` (control protocol) → `Decider` (scripted + `on_unanswered` policy, no silent false-greens) → `Run` (turn loop, sub-agent dispatch tree, `RunRecord`). Multi-turn `chat`, deterministic cassette `record`/`replay` (no token), `run.jsonl`/`trace.json` logging with secret-scrub.
@@ -0,0 +1,220 @@
1
+ {
2
+ "baselineVersion": 1,
3
+ "appVersion": "1.15200.0",
4
+ "agentVersion": "2.1.181",
5
+ "agentBinary": {
6
+ "stagedPath": "~/Library/Application Support/Claude/claude-code-vm/2.1.181/claude",
7
+ "format": "elf-aarch64"
8
+ },
9
+ "guest": {
10
+ "os": "linux",
11
+ "arch": "arm64",
12
+ "baseImage": "ubuntu:22.04"
13
+ },
14
+ "spawn": {
15
+ "configDirInGuest": "mnt/.claude",
16
+ "settingSources": [
17
+ "user"
18
+ ],
19
+ "permissionMode": "default",
20
+ "maxThinkingTokens": 31999,
21
+ "effortDefault": "medium",
22
+ "tools": [
23
+ "Task",
24
+ "Bash",
25
+ "Glob",
26
+ "Grep",
27
+ "Read",
28
+ "Edit",
29
+ "Write",
30
+ "NotebookEdit",
31
+ "WebFetch",
32
+ "TaskCreate",
33
+ "TaskUpdate",
34
+ "TaskGet",
35
+ "TaskList",
36
+ "TaskStop",
37
+ "WebSearch",
38
+ "Skill",
39
+ "REPL",
40
+ "JavaScript",
41
+ "AskUserQuestion",
42
+ "ToolSearch"
43
+ ],
44
+ "allowedTools": [
45
+ "Task",
46
+ "Bash",
47
+ "Glob",
48
+ "Grep",
49
+ "Read",
50
+ "Edit",
51
+ "Write",
52
+ "NotebookEdit",
53
+ "WebFetch",
54
+ "TaskCreate",
55
+ "TaskUpdate",
56
+ "TaskGet",
57
+ "TaskList",
58
+ "TaskStop",
59
+ "WebSearch",
60
+ "Skill",
61
+ "REPL",
62
+ "JavaScript",
63
+ "ToolSearch"
64
+ ],
65
+ "env": {
66
+ "CLAUDE_CODE_IS_COWORK": "1",
67
+ "CLAUDE_CODE_ENTRYPOINT": "local-agent",
68
+ "CLAUDE_CODE_TAGS": "lam_session_type:chat",
69
+ "CLAUDE_CODE_PROVIDER_MANAGED_BY_HOST": "1",
70
+ "CLAUDE_CODE_ENABLE_ASK_USER_QUESTION_TOOL": "true",
71
+ "CLAUDE_CODE_DISABLE_CRON": "1",
72
+ "CLAUDE_CODE_DISABLE_BACKGROUND_TASKS": "1",
73
+ "CLAUDE_CODE_DISABLE_AGENTS_FLEET": "1",
74
+ "CLAUDE_CODE_ENABLE_APPEND_SUBAGENT_PROMPT": "1",
75
+ "CLAUDE_CODE_ENABLE_TASKS": "true",
76
+ "CLAUDE_CODE_DISABLE_TERMINAL_TITLE": "1",
77
+ "ENABLE_PROMPT_CACHING_1H": "1",
78
+ "DISABLE_MICROCOMPACT": "1",
79
+ "MCP_CONNECTION_NONBLOCKING": "true"
80
+ },
81
+ "promptTemplate": "prompts/desktop-1.15200.0/system-prompt-append.md",
82
+ "subagentAppend": "prompts/desktop-1.15200.0/subagent-append-vm.md",
83
+ "$comment": "Binary-verified Desktop->agent spawn contract (asar 1.12603.1). See docs/cowork-spawn-contract-1.12603.1.md.",
84
+ "$comment_notSet": "Deliberately NOT set: CLAUDE_CODE_USE_COWORK_PLUGINS (Desktop never sets it; would flip the agent to cowork_settings.json/cowork_plugins). Host-derived (TZ, account UUIDs, WORKSPACE_HOST_PATHS, OTEL) injected at runtime, not pinned.",
85
+ "$comment_prompts": "Reconstructed cowork-specific sections, re-paraphrased from asar 1.15200.0 const MZi (system prompt) + generator CVr (subagent) — not the full base prompt (not cleanly extractable), and excluding the generic refusal/safety tail. The main append is delivered via the --append-system-prompt CLI flag (layered on the agent's built-in base prompt), NOT the initialize handshake; only the subagent append goes over initialize (appendSubagentSystemPrompt), gated on CLAUDE_CODE_ENABLE_APPEND_SUBAGENT_PROMPT."
86
+ },
87
+ "mountLayout": {
88
+ "sessionRoot": "/sessions/{sessionId}",
89
+ "cwd": "/sessions/{sessionId}",
90
+ "mntRoot": "/sessions/{sessionId}/mnt",
91
+ "mounts": [
92
+ {
93
+ "name": "uploads",
94
+ "mountPath": "uploads",
95
+ "mode": "r",
96
+ "purpose": "user-uploaded files (read-only — asar 'ro')"
97
+ },
98
+ {
99
+ "name": "projects",
100
+ "mountPath": ".projects/{projectId}",
101
+ "mode": "rw",
102
+ "purpose": "RESERVED namespace (and the separate UUID project-sync feature) — NOT the work-folder path. From Desktop 1.14271.0 selected work folders mount at mnt/<collision-resolved-basename> (dynamic, derived per session by buildLaunchPlan; see MOUNT_BARE_NAME_MIN_VERSION). This decorative row is not consumed for binding (staged paths come from plan.mounts)."
103
+ },
104
+ {
105
+ "name": "local-plugins",
106
+ "mountPath": ".local-plugins/marketplaces",
107
+ "mode": "r",
108
+ "purpose": "marketplace skills/plugins, runtime-discovered"
109
+ },
110
+ {
111
+ "name": "remote-plugins",
112
+ "mountPath": ".remote-plugins",
113
+ "mode": "r",
114
+ "purpose": "org-remote plugins, runtime-discovered"
115
+ },
116
+ {
117
+ "name": "outputs",
118
+ "mountPath": "outputs",
119
+ "mode": "rw",
120
+ "purpose": "session outputs/artifacts — delete denied by default (asar IX); rwd only when approved"
121
+ }
122
+ ]
123
+ },
124
+ "network": {
125
+ "mode": "gvisor",
126
+ "allowKind": "allowlist",
127
+ "allowDomains": [
128
+ "sentry.io",
129
+ "preview.claude.ai",
130
+ "downloads.claude.ai",
131
+ "api.anthropic.com",
132
+ "a-cdn.anthropic.com",
133
+ "a-api.anthropic.com",
134
+ "assets.claude.ai",
135
+ "console.anthropic.com",
136
+ "api-staging.anthropic.com",
137
+ "www.anthropic.com",
138
+ "docs.anthropic.com",
139
+ "mcp-proxy.anthropic.com",
140
+ "pivot.claude.ai",
141
+ "support.anthropic.com"
142
+ ]
143
+ },
144
+ "bgEnvStrip": {
145
+ "knownVars": [
146
+ "CLAUDE_CODE_OAUTH_TOKEN",
147
+ "CLAUDE_CODE_SESSION_KIND",
148
+ "CLAUDE_CODE_SESSION_ID",
149
+ "CLAUDE_CODE_SESSION_NAME",
150
+ "CLAUDE_CODE_SESSION_LOG"
151
+ ]
152
+ },
153
+ "$comment": "Platform baseline auto-derived by `cowork-harness sync` from a live Claude Desktop install + app.asar. VOLATILE per-release facts only. Regenerate per release; review the diff. Captured 2026-06-12 on macOS arm64.",
154
+ "capturedAt": "2026-06-24",
155
+ "platform": "darwin-arm64",
156
+ "settings": {
157
+ "autoMountFolders": {
158
+ "key": "autoMountFolders",
159
+ "default": false
160
+ },
161
+ "localAgentModeTrustedFolders": {
162
+ "key": "localAgentModeTrustedFolders",
163
+ "default": []
164
+ }
165
+ },
166
+ "provenance": {
167
+ "asarPath": "/Applications/Claude.app/Contents/Resources/app.asar",
168
+ "asarFingerprint": "636f468d72f314f7",
169
+ "gates": {
170
+ "$comment": "Production GrowthBook gate states decoded from ~/Library/Application Support/Claude/fcache (standard interactive Anthropic account, 2026-06-13; binary-verified app.asar 1.12603.1). Pin per release. Behavior-affecting gates the harness models: 1143815894 (loop), 1648655587 (dispatch cap), 1978029737 (web_fetch routing). Telemetry/auth-internal gates omitted.",
171
+ "bridgeSdkTransport:583857784": {
172
+ "on": true,
173
+ "source": "force",
174
+ "value": true,
175
+ "note": "— Cowork uses the SDK-based transport (control protocol), confirming the harness's sdkMcpServers/mcp_message path is the production transport."
176
+ },
177
+ "hostLoop:1143815894": {
178
+ "on": true,
179
+ "source": "force",
180
+ "value": true
181
+ },
182
+ "taskDispatchLimiter:1648655587": {
183
+ "on": true,
184
+ "source": "force",
185
+ "value": {
186
+ "perTask": 1,
187
+ "global": 3
188
+ },
189
+ "note": "perTask=1 global=3 (host-side SKIP: recordSkipAndEmit/GCA.PerTaskLimit — NOT queue/deny). A dispatch session launches <=1 sub-task; <=3 concurrent globally."
190
+ },
191
+ "coworkRuntimeConfig:1978029737": {
192
+ "on": true,
193
+ "source": "force",
194
+ "value": {
195
+ "sessionsBridgePollBlockMs": 30,
196
+ "coworkNativeFilePreview": true,
197
+ "coworkWebFetchViaApi": true,
198
+ "coworkWebFetchPrompt": true,
199
+ "workspaceBashWaitLonger": true
200
+ },
201
+ "note": "coworkWebFetchViaApi=true coworkWebFetchPrompt=true workspaceBashWaitLonger=true sessionsBridgePollBlockMs=30 — web_fetch is host/API-routed (POST /api/organizations/<org>/cowork/web_fetch), NOT container egress; gated by a separate web-fetch hostname allowlist + URL provenance."
202
+ },
203
+ "cliPlugin:2307090146": {
204
+ "on": false,
205
+ "source": "defaultValue",
206
+ "value": false,
207
+ "note": "— the CLI-plugin credential broker is dark-launched off for standard interactive accounts (Ch23/L106)."
208
+ },
209
+ "pluginSyncSparkplug:2340532315": {
210
+ "on": true,
211
+ "source": "force",
212
+ "value": true,
213
+ "note": "— startup syncPlugins(); plugins load via --plugin-dir (registry inert in-VM)."
214
+ }
215
+ },
216
+ "eipcChannelUuid": "4f426349-8d6f-45f3-ae22-280fef323564",
217
+ "$comment": "eipcChannelUuid is per-build; recorded for provenance only — the harness does not use Desktop IPC."
218
+ },
219
+ "requireFullVmSandbox": null
220
+ }
@@ -0,0 +1,18 @@
1
+ <!-- Reconstructed (PARAPHRASED, not verbatim) from the subagent system-prompt append in
2
+ Claude.app app.asar 1.15200.0 — generator `CVr({vmProcessName, hostLoopMode, ...})` at
3
+ byte offset ~9,305,235, VM branch (the text below corresponds to hostLoopMode=false).
4
+ Delivered to spawned sub-agents over the `initialize` control_request field
5
+ `appendSubagentSystemPrompt` (NOT --append-system-prompt). In 1.15200.0 this text is the
6
+ HARDCODED FALLBACK behind GrowthBook flag 124685897 / key `subagent_env_vm` — used when the
7
+ flag is off or no server override is supplied (the shipped default). A separate host-loop
8
+ branch (key `subagent_env_hl`, different wording about reaching the user's real filesystem)
9
+ exists in the asar but is NOT modeled here — the harness wires a single subagentAppend.
10
+
11
+ 1.12603.1 → 1.15200.0 drift folded in: the middle clause changed from "are not visible to
12
+ the user unless written to the outputs directory" to "not on the user's real computer".
13
+ The real cwd renders as /sessions/<vmProcessName>; {{cwd}} is the harness reconstruction
14
+ token (substituted by src/prompt.ts). Paraphrased per the repo's no-bundling rule. -->
15
+
16
+ ## Cowork environment
17
+
18
+ You are running as a subagent inside a Cowork session. Shell commands execute in an isolated Linux sandbox rooted at {{cwd}} — files created there (or under /tmp) exist only in the sandbox, not on the user's real computer. User-attached folders are mounted under {{cwd}}/mnt/.
@@ -0,0 +1,58 @@
1
+ <!-- Reconstructed (PARAPHRASED, not verbatim) from the real Cowork system prompt in
2
+ Claude.app app.asar 1.15200.0. The cowork body is module constant `MZi` at byte offset
3
+ ~6,583,523, emitted locally by the desktop app's 3P bootstrap builder (consumed at
4
+ ~6,632,043) as `cowork_system_prompt:{value:{prompt:MZi}, on:true, source:"defaultValue",
5
+ ruleId:null}` — the shipped DEFAULT, layered through a server-overridable envelope (not
6
+ itself a GrowthBook/spVariant lookup, but the server COULD replace it). Cross-checked
7
+ against the 1.13576.1 reconstruction this supersedes.
8
+
9
+ Scope: only the cowork-specific, behavior-driving sections (identity, product info,
10
+ skills, computer use, file handling, user files, uploads). MZi continues into ~28KB of
11
+ GENERIC refusal/safety policy — deliberately EXCLUDED here: it is not Cowork-specific and
12
+ the agent's built-in base prompt already carries safety. Delivered via
13
+ `--append-system-prompt` (layered on the agent's base prompt), NOT the initialize
14
+ handshake. Tokens substituted by src/prompt.ts mirroring the Desktop builder `y8r`. -->
15
+
16
+ <!-- 1.14271.0 → 1.15200.0 drift folded in: the identity constant was rewritten FIRST→THIRD
17
+ person and no longer says "powering Cowork mode" / "made by Anthropic" / "refer to
18
+ yourself as Claude"; it KEEPS the load-bearing "is NOT Claude Code / should not refer to
19
+ itself as such" correction (without it the base claude_code preset makes the agent say
20
+ "I'm Claude Code"). high_level_computer_use_explanation expanded; file_handling_rules
21
+ became a scratchpad-vs-workspace split; a working_with_user_files block was added.
22
+ The real asar's `{{workspaceContext}}` token (Desktop host workspace state the harness
23
+ does not have) is resolved INLINE below to the single-container model rather than wired as
24
+ a renderer token. The asar's `request_cowork_directory` tool and `computer://` link scheme
25
+ are Desktop/host-loop-only affordances — they are described as behavior, NOT emitted as
26
+ instructions, so the model is never told to call a tool / emit a URI that does not resolve
27
+ on the container / microvm / protocol tiers. Paraphrased per the repo's no-bundling rule;
28
+ model name comes from the <env> block, not hardcoded here. -->
29
+ <application_details>
30
+ Claude is operating as an agent inside the Claude desktop app; this agent capability is currently a research preview. Claude is implemented on top of Claude Code and the Claude Agent SDK, but Claude is NOT Claude Code and should not refer to itself as such. Claude runs in a lightweight Linux VM on the user's computer — a secure sandbox for executing code with controlled access to a workspace folder. Claude should not mention implementation details like this (including Claude Code or the Claude Agent SDK) unless it is relevant to the user's request.
31
+ </application_details>
32
+
33
+ <product_information>
34
+ Claude runs inside the Claude desktop app to help automate file and task management. The desktop app supports plugins — installable bundles of MCP servers, skills, and tools. The model powering this session is given in the <env> section at the end of the prompt; public documentation for configuration, policy keys, and troubleshooting is published by Anthropic for the Claude desktop app. When it helps, Claude can offer guidance on effective prompting — being clear and detailed, giving positive and negative examples, encouraging step-by-step reasoning, requesting specific XML tags, and stating the desired length or format — with concrete examples where possible.
35
+ </product_information>
36
+
37
+ <skills>
38
+ Skills are folders of condensed best practices for producing high-quality outputs (for example a docx skill for Word documents, a PDF skill for creating and filling PDFs). They live under {{skillsDir}}/skills/. Before writing any code, creating files, or using computer tools, Claude should first check which available skills are relevant and Read the relevant SKILL.md — the documentation is worth reading up front, and more than one skill may apply to a task.
39
+ </skills>
40
+
41
+ <high_level_computer_use_explanation>
42
+ Claude runs in a lightweight Linux VM (Ubuntu 22) on the user's computer — a secure sandbox for executing code with controlled access to user files. Available tools: Bash (run commands), Edit (edit existing files), Write (create files), and Read (read files — not directories; use `ls` via Bash for those). The working directory {{cwd}} is for all temporary work. The VM's internal file system resets between tasks, but the workspace folder ({{workspaceFolder}}) persists on the user's actual computer, so files saved there remain accessible after the session ends. Claude can create files such as docx, pptx, and xlsx for the user to open from their selected folder.
43
+ </high_level_computer_use_explanation>
44
+
45
+ <file_handling_rules>
46
+ FILE LOCATIONS AND ACCESS:
47
+ 1. Claude's work — {{cwd}}: create new files here first; this is a temporary scratchpad and the user cannot see files in it.
48
+ 2. Workspace folder — {{workspaceFolder}}: where final outputs and deliverables belong. Files written here are surfaced to the user; files written elsewhere in the sandbox are not — so saving final work here is essential, or the user won't see it. For simple tasks (a single short file) write directly here. If the user mounted a folder from their computer, this IS that folder and Claude can both read from and write to it.
49
+ - Never expose absolute /sessions/ sandbox paths to the user; refer to files by name or workspace-relative location.
50
+ </file_handling_rules>
51
+
52
+ <working_with_user_files>
53
+ The user connects folders to the session; a connected work folder is mounted under {{workspaceFolder}} and is where user-visible deliverables belong. Files written to the workspace folder are surfaced to the user; files written elsewhere in the sandbox (the scratchpad at {{cwd}}, or /tmp) are not. When no folder is connected, work in the scratchpad and tell the user a folder is needed to persist results. When referring to file locations, Claude should say "the folder you selected" if it has access to the user's files, or "my working folder" if it only has a temporary folder — and never expose internal /sessions/ paths, which look like backend infrastructure and cause confusion.
54
+ </working_with_user_files>
55
+
56
+ <notes_on_user_uploaded_files>
57
+ Files the user uploads are placed under {{cwd}}/mnt/uploads and can be accessed programmatically at that path.
58
+ </notes_on_user_uploaded_files>
@@ -0,0 +1,27 @@
1
+ /** Bounded-concurrency async map. Runs `fn` over `items` with at most `concurrency` in flight at once,
2
+ * preserving result order (results[i] corresponds to items[i]). A worker that throws rejects the whole
3
+ * pool — callers that want per-item error capture (the batch-record paths do) should make `fn` return a
4
+ * discriminated result and never throw. `concurrency` is clamped to [1, items.length].
5
+ *
6
+ * Note on in-process safety: the record setup (egress sidecar, image build) is `spawnSync` — it blocks the
7
+ * event loop, so the synchronous setup of one item completes before another item's worker can run. Only the
8
+ * long agent run is async and therefore actually overlaps. That's where the wall-clock win comes from, and
9
+ * it's also why a cold-start proxy-image build can't race in-process (the first worker builds it synchronously
10
+ * before yielding). Separate OS processes (`xargs -P`) do NOT get this serialization — see docs/cassette.md. */
11
+ export async function pMapBounded(items, concurrency, fn) {
12
+ const results = new Array(items.length);
13
+ if (items.length === 0)
14
+ return results;
15
+ const limit = Math.max(1, Math.min(concurrency, items.length));
16
+ let next = 0;
17
+ const worker = async () => {
18
+ for (;;) {
19
+ const i = next++;
20
+ if (i >= items.length)
21
+ return;
22
+ results[i] = await fn(items[i], i);
23
+ }
24
+ };
25
+ await Promise.all(Array.from({ length: limit }, () => worker()));
26
+ return results;
27
+ }
package/dist/cli.js CHANGED
@@ -14,7 +14,7 @@ import { vmInit, vmDelete, vmStatus, vmPrune, instanceName } from "./runtime/lim
14
14
  import { sync } from "./sync/cowork-sync.js";
15
15
  import { runBoundaryChecks, formatBoundary } from "./boundary.js";
16
16
  import { cmdChat } from "./run/chat.js";
17
- import { cmdRecord, cmdReplay, cmdVerifyCassettes, cmdRehash } from "./run/cassette.js";
17
+ import { cmdRecord, cmdReplay, cmdVerifyCassettes, cmdRehash, buildFingerprint, fingerprintSkillDrift } from "./run/cassette.js";
18
18
  import { cmdRunsGc } from "./run/runs-gc.js";
19
19
  import { resolveInputs } from "./run/inputs.js";
20
20
  import { cmdLint } from "./run/scenario-tool.js";
@@ -64,6 +64,8 @@ const HELP = `cowork-harness <command> (v${"$VERSION"})
64
64
  record <scenario.yaml> run + save a control-protocol cassette
65
65
  [--out <file>] cassette path (default: cassettes/<scenario-name>.cassette.json)
66
66
  [--max-artifact-bytes <n>] inline-body cap (default 65536 / $COWORK_HARNESS_MAX_ARTIFACT_BYTES)
67
+ [--concurrency <N>] record a dir/ batch (or --rerecord-stale) N at a time (default 1; runs are
68
+ fully isolated, so this only bounds Docker address-pool / API-rate pressure)
67
69
  [--decider-dir <dir>] answer gates LIVE in-band during the recording (single scenario; then use
68
70
  'gates'/'answer' to stream/respond) — one pass, no scripted-answer guesswork
69
71
  [--decider-llm [--intent "…"]] | [--on-unanswered fail|first] answer live via a model / auto-pick
@@ -78,7 +80,7 @@ const HELP = `cowork-harness <command> (v${"$VERSION"})
78
80
  runs gc [--keep-last <n>] prune accumulated run dirs, keeping N most recent per scenario (default: 5)
79
81
 
80
82
  ── CI lint + assertion reference ──────────────────────────────────────────────
81
- lint <scenario.yaml>… check scenarios for silent false-greens (bundled scenario.py; needs python3 — PyYAML is bundled)
83
+ lint <scenario.yaml | dir/>… check scenarios for silent false-greens (bundled scenario.py; needs python3 — PyYAML is bundled)
82
84
  [--strict] escalate cassette-staleness warning to failure
83
85
  NOTE: exit 127 means python3 itself is missing — treat any non-zero exit as a CI failure, do not swallow it.
84
86
  assertions --list list available scenario assertions (generated from Zod schema)
@@ -118,7 +120,7 @@ const HELP = `cowork-harness <command> (v${"$VERSION"})
118
120
  --version print version --help, -h print this help
119
121
 
120
122
  Env-var defaults (CLI flags take precedence):
121
- COWORK_HARNESS_FIDELITY default --fidelity tier (skill/run/chat)
123
+ COWORK_HARNESS_FIDELITY default --fidelity tier (skill/chat; run takes fidelity from the scenario)
122
124
  COWORK_HARNESS_OUTPUT_FORMAT default --output-format (text|json)
123
125
  COWORK_HARNESS_MODEL default --model
124
126
  NO_COLOR=1 disable ANSI output`;
@@ -252,7 +254,8 @@ const SUBCOMMAND_USAGE = {
252
254
  "boundary-check": "usage: boundary-check [<baseline>] [--session <file>] [--output-format text|json]",
253
255
  vm: "usage: vm <init|status|delete|prune> [--output-format text|json] (macOS arm64 only)\n init create the L2 Apple-VZ microVM\n status show running VM state\n delete remove a named VM\n prune drop all orphaned VMs",
254
256
  chat: "usage: chat <skill-folder> [prompt] [--fidelity protocol|container|hostloop] [--model <id>]\n [--upload <file>]... [--folder <dir>]... [--plugin <dir>]... [--verbose] [--raw]\n --raw: native cowork mode via docker run -it; egress sandbox NOT applied; rejects --upload/--folder/--plugin/--fidelity (only --model applies)\n --fidelity: protocol/container/hostloop only (no microvm/cowork); protocol = no Docker, no sandbox",
255
- record: "usage: record <scenario.yaml | dir/> [--out <file>] [--output-format text|json] [--rerecord-stale] [--no-redact] [--allow-failing] [--max-artifact-bytes <n>] [--dry-run]\n" +
257
+ record: "usage: record <scenario.yaml | dir/> [--out <file>] [--output-format text|json] [--rerecord-stale] [--no-redact] [--allow-failing] [--max-artifact-bytes <n>] [--dry-run] [--concurrency <N>]\n" +
258
+ " --concurrency <N>: record a dir/ batch (or --rerecord-stale) N at a time (default 1, max 8). Runs are fully isolated; the bound is for Docker address pool + API rate limits.\n" +
256
259
  ' answer gates LIVE: [--decider-dir <dir>] (single scenario only) | [--decider-llm [--intent "<one line>"]] | [--on-unanswered fail|first]\n' +
257
260
  " (a live decider flags the cassette non-deterministic — re-recording may drift; replay stays deterministic. --rerecord-stale rejects these flags.)\n" +
258
261
  " NOTE: --allow-failing only relaxes the post-run VERDICT gate; it does NOT salvage an unanswered gate (that throws before any cassette is written — use --on-unanswered first / a decider).",
@@ -1891,6 +1894,24 @@ async function cmdVerifyRun(args) {
1891
1894
  // behaves EXACTLY as before (assert-only; events.jsonl irrelevant; no refusal), preserving existing runs.
1892
1895
  let answerCoverage;
1893
1896
  if (scenario.answers.length > 0) {
1897
+ // CURRENCY: answer-coverage validates against the kept run's gate SNAPSHOT (its events.jsonl). If the skill
1898
+ // changed since the run was kept, those gates are stale and a green here is false confidence — the real
1899
+ // gates moved. Refuse rather than vouch (can't verify ⇒ not green). Every run as of this version persists a
1900
+ // fingerprint; an older run without one → warn (can't check). A run with no skill dirs → nothing to drift.
1901
+ const recFp = result.fingerprint;
1902
+ if (recFp === undefined) {
1903
+ log(`verify-run: ::warning:: this kept run carries no skill fingerprint (recorded by an older harness) — ` +
1904
+ `cannot confirm it is current vs the skill; re-keep a fresh run to be sure answer-coverage is against live gates.`);
1905
+ }
1906
+ else if (recFp.skillHash !== undefined) {
1907
+ const liveFp = buildFingerprint(scenario.session, recFp.baseline, undefined, scenario.skills);
1908
+ const drift = fingerprintSkillDrift(recFp, liveFp);
1909
+ if (drift) {
1910
+ log(`verify-run: the kept run predates the current skill — ${drift}. Its gate snapshot is stale, so ` +
1911
+ `answer-coverage can't be trusted; re-keep a fresh run (or re-record). (can't verify ⇒ not green)`);
1912
+ return process.exit(2);
1913
+ }
1914
+ }
1894
1915
  const gates = parseGatesFromEvents(join(runDir, "events.jsonl"));
1895
1916
  if (gates === null) {
1896
1917
  log(`verify-run: scenario declares answers but ${runDir} has no events.jsonl — cannot verify answer coverage ` +