cowork-harness 0.29.0 → 0.30.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/.claude/skills/cowork-harness/SKILL.md +25 -11
  2. package/.claude/skills/cowork-harness/references/ci-recipe.md +4 -4
  3. package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +1 -1
  4. package/.claude/skills/cowork-harness/references/scenario-schema.md +29 -9
  5. package/.claude/skills/cowork-harness/references/task-recipes.md +2 -2
  6. package/.claude/skills/cowork-harness/scripts/assertion-keys.json +6 -1
  7. package/.claude/skills/cowork-harness/scripts/scenario.py +5 -0
  8. package/CHANGELOG.md +169 -0
  9. package/CONTRIBUTING.md +1 -1
  10. package/README.md +11 -28
  11. package/RELEASING.md +27 -0
  12. package/SPEC.md +14 -9
  13. package/baselines/desktop-1.18286.2.json +1 -0
  14. package/baselines/desktop-1.19367.0.json +1 -0
  15. package/baselines/desktop-1.20186.0.json +1 -0
  16. package/baselines/desktop-1.20186.1.json +380 -0
  17. package/baselines/prompts/cowork-system-prompt-fingerprints.json +4 -0
  18. package/baselines/prompts/desktop-1.18286.2/subagent-append-hl.md +16 -0
  19. package/dist/agent/session.js +70 -9
  20. package/dist/agent/timeline.js +6 -1
  21. package/dist/assert.js +95 -5
  22. package/dist/cli.js +48 -7
  23. package/dist/decide/decider.js +1 -1
  24. package/dist/hostloop/canusetool-gate.js +63 -0
  25. package/dist/hostloop/pretooluse-path-hook.js +62 -22
  26. package/dist/prompt.js +33 -10
  27. package/dist/run/cassette.js +236 -103
  28. package/dist/run/chat-result.js +2 -0
  29. package/dist/run/chat.js +30 -11
  30. package/dist/run/execute.js +702 -656
  31. package/dist/run/renderer.js +1 -1
  32. package/dist/run/run.js +267 -44
  33. package/dist/run/trace-view.js +27 -7
  34. package/dist/run/verdict.js +16 -2
  35. package/dist/runtime/argv.js +6 -1
  36. package/dist/runtime/container.js +7 -2
  37. package/dist/runtime/hostloop-prompt.js +5 -5
  38. package/dist/runtime/hostloop.js +51 -13
  39. package/dist/runtime/microvm.js +3 -1
  40. package/dist/runtime/protocol.js +17 -1
  41. package/dist/session.js +42 -0
  42. package/dist/sync/cowork-sync.js +296 -5
  43. package/dist/types.js +48 -2
  44. package/docker/Dockerfile.agent +1 -1
  45. package/docs/README.md +2 -1
  46. package/docs/boundary.md +21 -2
  47. package/docs/cassette.md +35 -17
  48. package/docs/cowork-spawn-contract-1.12603.1.md +3 -2
  49. package/docs/fidelity-gaps.md +85 -0
  50. package/docs/gotchas.md +0 -3
  51. package/docs/invariants.md +4 -2
  52. package/docs/maintenance.md +19 -0
  53. package/docs/plugin-root.md +16 -7
  54. package/docs/scenario.md +31 -6
  55. package/docs/session.md +37 -0
  56. package/docs/subagents.md +154 -0
  57. package/examples/replays/README.md +1 -1
  58. package/examples/replays/example-multiselect-gate.cassette.json +1 -1
  59. package/examples/replays/example-pdf-skill.cassette.json +1 -1
  60. package/examples/replays/hostloop-computer-links.cassette.json +1 -1
  61. package/package.json +3 -1
  62. package/schema/cassette.v10.json +4 -0
  63. package/schema/protocol.v1.json +1 -0
  64. package/schema/run-result.json +65 -7
  65. package/schema/scenario.schema.json +72 -2
  66. package/schema/session.schema.json +20 -0
  67. package/scripts/check-versions.ts +8 -15
  68. package/dist/canary/grants.js +0 -37
  69. package/dist/critique/evaluator.js +0 -321
  70. package/dist/critique/evidence.js +0 -175
  71. package/dist/critique/package-evidence.js +0 -198
  72. package/docs/assets/banner.png +0 -0
  73. package/fixtures/subagent-grants.json +0 -5
  74. package/schema/cassette.v2.json +0 -19
  75. package/schema/cassette.v3.json +0 -30
  76. package/schema/cassette.v5.json +0 -107
  77. package/schema/cassette.v6.json +0 -131
  78. package/schema/cassette.v7.json +0 -156
  79. package/schema/cassette.v8.json +0 -166
  80. package/scripts/boot-rootfs-vz.ts +0 -99
@@ -3,8 +3,8 @@ name: cowork-harness
3
3
  description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
4
4
  metadata:
5
5
  author: cowork-harness
6
- version: 0.29.0
7
- tracks-harness: cowork-harness 0.29.0 (baseline desktop-1.20186.0)
6
+ version: 0.30.0
7
+ tracks-harness: cowork-harness 0.30.0 (baseline desktop-1.20186.1)
8
8
  ---
9
9
 
10
10
  # cowork-harness
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
22
22
  allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
23
23
  the highest-value part. Read it.
24
24
 
25
- > **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.29.0` (baseline
25
+ > **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.30.0` (baseline
26
26
  > `desktop-1.20186.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
27
27
  > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
28
28
 
@@ -38,7 +38,7 @@ cowork-harness skill ./my-skill "do X" # run the skill once against the sta
38
38
  Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
39
39
 
40
40
  - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
41
- - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.29.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.29.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.29.0"`. **Pin `@>=0.29.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.29.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-patterns-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0; `--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, `computer_links_resolve` (new in 0.22.0), and `semantic_matches` (an LLM judge grades a fixed `rubric` of claims against the run's answer — **live-only**, so it is evidence-unavailable / skipped-loud on replay, never a vacuous pass; new in 0.27.0), and **glob-matched** `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` (a pattern like `mcp__workspace__*` matches any tool in the family; exact names match exactly; new in 0.28.0).)
41
+ - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.30.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.30.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.30.0"`. **Pin `@>=0.30.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.30.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-patterns-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0; `--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, `computer_links_resolve` (new in 0.22.0), `semantic_matches` (an LLM judge grades a fixed `rubric` of claims against the run's answer — **live-only**, so it is evidence-unavailable / skipped-loud on replay, never a vacuous pass; new in 0.27.0), **glob-matched** `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` (a pattern like `mcp__workspace__*` matches any tool in the family; exact names match exactly; new in 0.28.0), the five path-gate assertion keys `no_vm_path_file_op`/`vm_path_denied`/`path_denied`/`no_path_denied`/`subagent_file_write`, the session-level `agent_env` knob, and resolved sub-agent identity on dispatch records (`resolvedAgentType`/`resolvedModel`; new in 0.30.0).)
42
42
  - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
43
43
  - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
44
44
  - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
@@ -63,7 +63,7 @@ reproducible regression (Part II), and **debug** a run that misbehaved or greene
63
63
  the path, `trace <run-id>` finds it). **Localize the failure post-hoc** from that evidence:
64
64
  `cowork-harness trace <run-dir>`'s views + the emitted `result.json` to see what the run actually did,
65
65
  then `verify-run` to re-check a suspect assertion — all token-free, no Docker, no re-record. This is
66
- the loop 0.29.0's observability is built for; the *Triage* and *Inspecting a run's observability
66
+ the loop 0.30.0's observability is built for; the *Triage* and *Inspecting a run's observability
67
67
  output* sections in **Part III — Debug** are the detail (the fuller human-facing map lives in
68
68
  `docs/debugging.md` — repo-only, not shipped with the installed skill).
69
69
  - **Multi-turn / interactive reproduction** → `cowork-harness chat` (interactive; gates answered at the
@@ -292,7 +292,9 @@ scenario's gates or their labels: `--keep` ONE run, then `cowork-harness trace <
292
292
  your `answers:` against that kept run, then record once. **But the kept run is a snapshot:** if you change the
293
293
  skill's gate phrasing afterward, re-`--keep` — verify-run's answer-coverage *refuses* (exit 2, "predates the
294
294
  current skill") rather than vouch against stale labels, but the trace/inspect path can't warn you, so re-keep
295
- deliberately. (A token-free probe of "which gates fire" isn't possible — gates are model-decided per run.)
295
+ deliberately. (Same fail-closed family: corrupt gate evidence — unparseable `events.jsonl` lines, or fewer
296
+ gates than `trace.json` recorded questions — and a structurally invalid `result.json` also refuse rather
297
+ than certify.) (A token-free probe of "which gates fire" isn't possible — gates are model-decided per run.)
296
298
 
297
299
  Run artifacts are written to `~/.cowork-harness/runs/…` by default — **outside any working tree**, so a run
298
300
  launched from a repo root never drops sensitive skill inputs/outputs into it. Pass `--run-dir <path>` (or set
@@ -420,8 +422,8 @@ decide which assertions from *Assertions: two orthogonal axes* are worth adding)
420
422
  - **`cowork-harness stats [--metric <m>]`** — aggregate across the run index: `cost`, `duration`,
421
423
  `tokens`, `cache-tokens`, `model-cost`, `turns`, `pass-rate`.
422
424
  - **`result.json` carries the raw fields** the assertions read: `toolDurations`, `models`, `toolErrors`,
423
- `redundantToolCalls`, `modelUsage`, `thinking`, `skillActivity`, `subagents[]` (prompt/model/output/
424
- `attributedSkillId`, `outputTruncated`), `context` (tools/mcpServers/availableSkills), `tasks`,
425
+ `redundantToolCalls`, `modelUsage`, `thinking`, `skillActivity`, `subagents[]` (prompt/`dispatchModel`/
426
+ `resolvedModel`/output/`attributedSkillId`, `outputTruncated`), `context` (tools/mcpServers/availableSkills), `tasks`,
425
427
  `workspaceFiles`, `presentedFiles`, `hookEvents`, `mcpErrors`, `contextEvents`, `resources`
426
428
  (`probeFailures` distinguishes a failed sample from a tier that was never sampleable). Provenance/
427
429
  evidence-health fields: `command` (`run`/`skill`/`record`/`chat`/`replay` — finer than `mode`),
@@ -439,7 +441,7 @@ decide which assertions from *Assertions: two orthogonal axes* are worth adding)
439
441
  than debugging; `"transport"`/`"agent"` means something actually broke, worth localizing before
440
442
  re-running.
441
443
  - **Attributing cost to sub-agent work.** `subagents[]` gives the dispatch tree — each sub-agent's
442
- `model`, `toolsUsed`, `prompt`/`output`, and `attributedSkillId` — but **not** its own token/cost;
444
+ `dispatchModel`/`resolvedModel`, `toolsUsed`, `prompt`/`output`, and `attributedSkillId` — but **not** its own token/cost;
443
445
  aggregate cost is per-**model** in `modelUsage` (and `trace --view usage`), not per-sub-agent. So a
444
446
  cost spike from fan-out reads as `trace --view dispatches` (how many, which agent) against that model's
445
447
  per-model usage — the harness doesn't line-item each sub-agent's tokens.
@@ -514,8 +516,20 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
514
516
  to prove an attempt; it proves the tool actually ran.
515
517
 
516
518
  5. **`subagent_declared_but_unused` fires on declared-but-didn't-use-THAT-tool**, even if the
517
- sub-agent used other tools. And `subagent_dispatched` matches by dispatch **description** too —
518
- skills often dispatch with no `subagent_type` (`agentType:"unknown"`), so match the description.
519
+ sub-agent used other tools. `subagent_dispatched` / `subagent_output_contains` match on dispatch
520
+ type (`dispatchAgentType`), the binary-*resolved* type (`resolvedAgentType`), *or* the dispatch
521
+ **description** — so a type-less dispatch that resolved to e.g. `general-purpose` is still
522
+ selectable, by either the resolved type or the description. A `Task` dispatch that carries NO
523
+ `subagent_type` at all falls back to the built-in `general-purpose` agent with a **wildcard tool
524
+ surface** (`tools:["*"]`, including workspace bash) — faithful production behavior, and it fires
525
+ routinely. The harness warns loudly on this fallback and records `subagents[].dispatchTypeOmitted`;
526
+ an *explicit* `subagent_type: "general-purpose"` is a deliberate author choice and does not warn.
527
+ Implication: `subagent_tool_absent` on a type-less dispatch is weaker evidence (wildcard surface) —
528
+ pin `subagent_type` explicitly when you need a tight tool-absence guarantee.
529
+
530
+ **Cross-tier "no shell" caveat.** On `hostloop`, native `Bash` calls route through the
531
+ `mcp__workspace__bash` alias, so a "sub-agent used no shell" check must glob **both** `Bash` and
532
+ `mcp__workspace__*` to hold across every tier.
519
533
 
520
534
  6. **`dispatch_count_max` is an author-chosen budget, not a production cap.** It's a post-hoc count
521
535
  assertion: passing means "happened to dispatch ≤N this run," nothing more. Cowork imposes **no**
@@ -1,6 +1,6 @@
1
1
  # CI recipe — replay vs live lanes
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 0.29.0` (baseline `desktop-1.20186.0`).
3
+ Self-contained reference. Tracks `cowork-harness 0.30.0` (baseline `desktop-1.20186.0`).
4
4
 
5
5
  **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
6
6
  job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -54,7 +54,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
54
54
  GitHub-hosted runners, no token/Docker/agent:
55
55
 
56
56
  ```yaml
57
- - run: npm i -g "cowork-harness@>=0.29.0"
57
+ - run: npm i -g "cowork-harness@>=0.30.0"
58
58
  - run: cowork-harness lint scenarios/*.yaml # no silent false-greens
59
59
  - run: cowork-harness verify-cassettes cassettes/ # privacy + staleness
60
60
  - run: cowork-harness replay cassettes/ # token-free content/structure
@@ -188,7 +188,7 @@ jobs:
188
188
  with: { node-version: '20' }
189
189
  - uses: actions/setup-python@v5
190
190
  with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
191
- - run: npm i -g "cowork-harness@>=0.29.0"
191
+ - run: npm i -g "cowork-harness@>=0.30.0"
192
192
  - run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
193
193
  - run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
194
194
  - run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
@@ -217,7 +217,7 @@ jobs:
217
217
  echo "live=true" >> "$GITHUB_OUTPUT"
218
218
  fi
219
219
  - if: steps.guard.outputs.live == 'true'
220
- run: npm i -g "cowork-harness@>=0.29.0"
220
+ run: npm i -g "cowork-harness@>=0.30.0"
221
221
  - if: steps.guard.outputs.live == 'true'
222
222
  run: cowork-harness run scenarios/ --output-format json
223
223
  env:
@@ -1,6 +1,6 @@
1
1
  # Fidelity tiers & answer paths
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 0.29.0` (baseline `desktop-1.20186.0`).
3
+ Self-contained reference. Tracks `cowork-harness 0.30.0` (baseline `desktop-1.20186.0`).
4
4
 
5
5
  ## Fidelity tiers (`fidelity:` in the scenario)
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Scenario & session schema, assertion catalog, web_fetch, full gotchas
2
2
 
3
- Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.29.0`
3
+ Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.30.0`
4
4
  (baseline `desktop-1.20186.0`). If your checkout is newer, prefer the live `docs/scenario.md`,
5
5
  `docs/session.md`, and `SPEC.md`.
6
6
 
@@ -105,6 +105,12 @@ agent_max_turns: 500 # optional turn ceiling -> agent --max-turns;
105
105
  permission_mode: default # default | acceptEdits | plan | bypassPermissions
106
106
  permission_parity: cowork # cowork (unscripted tool calls allowed) | strict (deny unscripted)
107
107
 
108
+ # sub-agent / tool-search env knob (tier-uniform; maps to agent env vars)
109
+ agent_env:
110
+ subagent_model: claude-opus-4-8 # -> CLAUDE_CODE_SUBAGENT_MODEL
111
+ tool_search: auto # auto | off -> ENABLE_TOOL_SEARCH; omit = binary default (ToolSearch ON)
112
+ disable_experimental_betas: false # true -> CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1 (also disables ToolSearch)
113
+
108
114
  # fenced debug escape hatch (NOT reachable via Cowork's UI)
109
115
  debug:
110
116
  max_thinking_tokens: 50000 # overrides --max-thinking-tokens directly; bypasses extended_thinking's
@@ -254,9 +260,11 @@ same set live from the schema.
254
260
  | `tool_result_not_contains: <str>` | no tool result includes the literal string (content / replay-checkable; fails loud when tool results are absent) |
255
261
  | `subagent_tool_used: <glob>` | a sub-agent used a tool matching this glob (same `*`/`?`, anchored, case-sensitive semantics as `tool_called`, including the empty/regex-ish rejection) |
256
262
  | `subagent_tool_absent: <glob>` | no sub-agent used a tool matching this glob (same rejection) |
257
- | `subagent_dispatched: <regex>` | a sub-agent whose `agentType` **or dispatch description** matches |
263
+ | `no_vm_path_file_op: true` | **`fidelity: hostloop` only** — NO gated file tool attempted a `/sessions`(-prefixed) path (`RunResult.fileToolAttempts`) — content-class, replay-checkable without `controlOut`; any other tier FAILS "cannot verify" (`/sessions/...` is valid there). **Only `true` is valid** |
264
+ | `subagent_file_write: {path?, path_suffix?, tool?}` | a sub-agent-origin write attempt whose raw path equals `path` (exact) or ends with `path_suffix` has a paired non-error tool_result — the causal half of a delivery probe; requires one of `path`/`path_suffix`; `tool` defaults to Write/Edit/MultiEdit; content-class; tier-agnostic |
265
+ | `subagent_dispatched: <regex>` | a sub-agent whose `dispatchAgentType`, binary-*resolved* `resolvedAgentType`, **or dispatch description** matches |
258
266
  | `subagent_declared_but_unused: <Tool>` | a sub-agent declared the tool but never used **that** tool (even if it used others) |
259
- | `subagent_output_contains: {match?, contains}` | a dispatched sub-agent's own output contains the substring `contains` — `match` (optional regex over `agentType`/`description`) narrows to specific dispatch(es); omitted, checks whether ANY dispatch's output contains it (existence check, not "all"); a miss against an output that was **truncated at the assert cap** reports evidence-unavailable instead of a proven absence — the substring could lie past the cut |
267
+ | `subagent_output_contains: {match?, contains}` | a dispatched sub-agent's own output contains the substring `contains` — `match` (optional regex over `dispatchAgentType`/`resolvedAgentType`/`description`) narrows to specific dispatch(es); omitted, checks whether ANY dispatch's output contains it (existence check, not "all"); a miss against an output that was **truncated at the assert cap** reports evidence-unavailable instead of a proven absence — the substring could lie past the cut |
260
268
  | `dispatch_count_max: <N>` | at most N sub-agents dispatched — an author-chosen budget (Cowork imposes no in-conversation Task-dispatch cap; records only, enforces nothing — see gotcha 12) |
261
269
  | `skill_triggered: <regex>` | a skill matching the regex (invoked id, e.g. `"plugin:skill"`) was invoked via the `Skill` tool — evidence-unavailable (not a normal fail) if the agent's init tools have no `Skill` tool |
262
270
  | `no_skill_triggered: <regex>` | no invoked skill id matched — the negative-control / description-collision catcher; evidence-unavailable (never a vacuous pass) if invocation data is absent or the `Skill` tool is unobservable |
@@ -291,6 +299,9 @@ same set live from the schema.
291
299
  | `gate_answer_count_min: <N>` | at least N AskUserQuestion gates fired AND were delivered non-error — presence companion to `gate_answers_delivered`'s vacuous-pass |
292
300
  | `hook_blocked: <regex>` | a PreToolUse hook blocked a tool whose name matches the regex (`RunResult.hookEvents`) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette (a custom hook's decision lives only there, not the recorded stream) |
293
301
  | `no_hook_blocked: true` | no tool was hook-blocked during the run (distinguishes a real tool crash from an intentional hook block) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette. **Only `true` is valid** |
302
+ | `vm_path_denied: true` | **`fidelity: hostloop` only** — at least one recorded path denial (`RunResult.pathDenials`, any source) targeted a `/sessions` VM path — evidence-unavailable if path-denial telemetry is absent. Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify". **Only `true` is valid** |
303
+ | `path_denied: {tool?, path_matches?, source?, agent_scope?}` | **`fidelity: hostloop` only** — a path denial matching ALL given matchers (`tool` glob, `path_matches` regex, `source` ∈ pretooluse/can_use_tool/permission_denied, `agent_scope` ∈ main/subagent/any) was recorded. Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify" |
304
+ | `no_path_denied: true` | **`fidelity: hostloop` only** — NO path denial was recorded at all (the channel is already path-scoped, unlike `no_hook_blocked`'s indiscriminate reject). Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify". **Only `true` is valid** |
294
305
  | `allow_permissive_auto_allow: true` | verdict modifier — suppresses the default-fail when the run recorded a cowork-parity permissive auto-allow; for tests that deliberately assert Cowork's permissive behavior |
295
306
  | `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
296
307
  | `allow_l0_plugin_divergence: true` | verdict modifier — opt into L0/protocol plugin divergence: suppresses the default-fail when a plugin behaves differently at `protocol` (L0) fidelity than under a sandboxed tier. Live tiers only |
@@ -319,7 +330,7 @@ dotted path.
319
330
 
320
331
  **VerdictSignals in `result.signals`:** `computeVerdict` pushes signals into `result.signals`; most
321
332
  are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
322
- `"success"`) and only two are **warn**-severity (informational, never flip pass/fail). Current signal
333
+ `"success"`) and only three are **warn**-severity (informational, never flip pass/fail). Current signal
323
334
  codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
324
335
 
325
336
  | Code | Severity | Meaning |
@@ -337,10 +348,11 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
337
348
  | `stalled` | fail | The run ended on an unanswered question (opt out: `allow_stall`) |
338
349
  | `non_deterministic` | warn | The run was LLM/external/human-decided — not reproducible |
339
350
  | `prompt_asset_missing` | warn | The run proceeded with a missing prompt asset (`COWORK_HARNESS_ALLOW_MISSING_PROMPT=1`); fidelity is degraded |
351
+ | `scan_unavailable` | warn | Post-run scan evidence unavailable (`RunResult.scan` undefined) — the host-path and outputs-delete guards did not run this run |
340
352
 
341
353
  A **fail**-severity signal does not change `result.result` (still `"success"`), but it DOES fail the
342
354
  overall run verdict and exit code — `assert result: success` alone won't catch it; check
343
- `result.signals[].severity` or the run's exit code. Only the two **warn** codes are truly benign.
355
+ `result.signals[].severity` or the run's exit code. Only the three **warn** codes are truly benign.
344
356
 
345
357
  ## Replay class
346
358
 
@@ -356,7 +368,8 @@ on recording-shaping drift (`prompt`/`answers`/`baseline`/`skills`) or skill-con
356
368
  sourcing ≠ evaluation (replay warns when you edit one). `verify-run` is the on-disk-`assert:` path for a kept
357
369
  *run dir*; `--assert-from` is the equivalent for a *cassette*.
358
370
 
359
- **Evaluated on replay (content):** `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`,
371
+ **Evaluated on replay (content):** `transcript_*`, `tool_*`, `subagent_*`, `subagent_file_write`,
372
+ `no_vm_path_file_op`, `dispatch_count_max`,
360
373
  `skill_triggered`, `no_skill_triggered`, `skill_available`, `connector_available`, `tool_available`,
361
374
  `skill_tool_used`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `tool_no_error`,
362
375
  `max_tool_errors`, `max_redundant_tool_calls`, `max_turns`, `compaction_occurred`, `all_tasks_completed`, `task_status`, `task_count_min`, `no_scratchpad_leak`, `present_files_called`, `result`
@@ -365,7 +378,9 @@ modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0
365
378
  `allow_stall` are also kept on replay, evaluated as no-op passes.
366
379
 
367
380
  **Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `questions_count_max`,
368
- `gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked`. With `controlOut` present they evaluate; on an old
381
+ `gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked`, `vm_path_denied`,
382
+ `path_denied`, `no_path_denied` (the latter three are also `fidelity: hostloop`-only — see the assertion
383
+ table). With `controlOut` present they evaluate; on an old
369
384
  cassette without it, a **loud warning** fires and they are **excluded** (not vacuously passed). Re-record to enable them.
370
385
  `questions_count_max` counts sub-questions, not gates/tool-calls — see the catalog row above and
371
386
  `trace --view questions`. `hook_blocked`/`no_hook_blocked` need `controlOut` for a different reason than
@@ -498,8 +513,13 @@ reference omits). Neither list is a strict superset of the other — reach for t
498
513
  `artifact_json` (dotted `path` + operator); use the pytest lane (`assert_artifact_json` with a real
499
514
  predicate) only for checks too complex for a dotted path. Find field paths via `--keep`.
500
515
 
501
- 10. **`subagent_dispatched` matches by `description` too** — skills often dispatch with no
502
- `subagent_type` (`agentType:"unknown"`), so match the dispatch description.
516
+ 10. **`subagent_dispatched` matches by `resolvedAgentType` or `description` too** — a `Task` dispatch
517
+ with no `subagent_type` at all falls back to the built-in `general-purpose` agent (a WILDCARD tool
518
+ surface, `tools:["*"]`, incl. workspace bash) rather than leaving the type unresolved; the harness
519
+ warns loudly on this fallback and records `subagents[].dispatchTypeOmitted`, so match on the
520
+ resolved type or the dispatch description. `subagent_tool_absent` on a type-less dispatch is
521
+ correspondingly weaker evidence (wildcard surface) — pin `subagent_type` explicitly when you need a
522
+ tight guarantee.
503
523
 
504
524
  11. **`subagent_declared_but_unused` fires on declared-but-didn't-use-THAT-tool**, even if the
505
525
  sub-agent used other tools.
@@ -64,8 +64,8 @@ Top-level fields of a `*.cassette.json` (schema `schema/cassette.v10.json`):
64
64
  | `preRunHashes` | Pre-run per-path sha256 baseline for `input_unmodified` (added 0.27.0, no cassetteVersion bump); a `null` value marks a path whose recorded artifact body was secret-scrubbed (evidence-unavailable, never a false "modified") |
65
65
  | `scenarioSource` | Relative path to the authored YAML this was recorded from |
66
66
  | `authoring` | Present iff a live decider answered ≥1 gate during recording (`nonDeterministic: true`) |
67
- | `sessionFingerprint` | v9+: hash of the session's content-relevant SHAPE (folders/plugins/skills/mcp/egress). Checked ONLY by `verify-cassettes`, never the default replay verdict; absent on a pre-v9 cassette → not checked |
68
- | `folderPrefixMap` | v9+: the record-time connected-folder host-path → mount-name map. Replay's `computer_links_resolve` uses THIS (never the current session file) for a v9+ cassette; a pre-v9 cassette keeps reconstructing from the current session |
67
+ | `sessionFingerprint` | Optional even on v9+ (the minimum readable version): hash of the session's content-relevant SHAPE (folders/plugins/skills/mcp/egress). Checked ONLY by `verify-cassettes`, never the default replay verdict; absent → not checked |
68
+ | `folderPrefixMap` | Optional even on v9+: the record-time connected-folder host-path → mount-name map. Replay's `computer_links_resolve` uses THIS (never the current session file); absent → the link is treated as evidence-unavailable, never reconstructed from the current session |
69
69
  | `timeline`, `timelineHeader` | The recorded per-event timeline (harness-observation timestamps for tool_use/tool_result/subagent_dispatch/thinking/decision/result, in total order) plus its header (`startedAtWall`/`startedAtMono` anchors); informational only — never affects the replay verdict. Absent on a cassette recorded before this field existed |
70
70
  | `environment` | Recording provenance: `location` (`"local"` on every cassette this harness produces), plus the resolved `tier` and `agentBinaryFormat` |
71
71
 
@@ -28,9 +28,12 @@
28
28
  "no_delete_in_outputs",
29
29
  "no_hook_blocked",
30
30
  "no_mcp_error",
31
+ "no_path_denied",
31
32
  "no_scratchpad_leak",
32
33
  "no_skill_triggered",
33
34
  "no_unexpected_files",
35
+ "no_vm_path_file_op",
36
+ "path_denied",
34
37
  "present_files_called",
35
38
  "question_asked",
36
39
  "questions_count_max",
@@ -43,6 +46,7 @@
43
46
  "skill_triggered",
44
47
  "subagent_declared_but_unused",
45
48
  "subagent_dispatched",
49
+ "subagent_file_write",
46
50
  "subagent_output_contains",
47
51
  "subagent_tool_absent",
48
52
  "subagent_tool_used",
@@ -61,7 +65,8 @@
61
65
  "transcript_no_host_path",
62
66
  "transcript_not_contains",
63
67
  "transcript_not_matches",
64
- "user_visible_artifact"
68
+ "user_visible_artifact",
69
+ "vm_path_denied"
65
70
  ],
66
71
  "topLevelKeys": [
67
72
  "allow_host_writes",
@@ -64,6 +64,8 @@ CONTENT_KEYS = {
64
64
  "subagent_dispatched",
65
65
  "subagent_declared_but_unused",
66
66
  "subagent_output_contains",
67
+ "no_vm_path_file_op",
68
+ "subagent_file_write",
67
69
  "dispatch_count_max",
68
70
  "skill_triggered",
69
71
  "no_skill_triggered",
@@ -94,6 +96,9 @@ GATE_KEYS = {
94
96
  "gate_answer_count_min",
95
97
  "hook_blocked",
96
98
  "no_hook_blocked",
99
+ "vm_path_denied",
100
+ "path_denied",
101
+ "no_path_denied",
97
102
  }
98
103
  # manifest-backed: replay-checkable when the cassette carries an `artifacts` manifest (record snapshots one);
99
104
  # a manifest-less cassette skips them. Since the 0.3.0 artifact-manifest these are NOT always live-only.
package/CHANGELOG.md CHANGED
@@ -6,6 +6,175 @@ All notable changes to this project are documented here. The format is based on
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.30.0] — 2026-07-12
10
+
11
+ > **Upgrade notes.**
12
+ > - **Breaking (pre-1.0): `subagents[].agentType` renamed to `dispatchAgentType`, and
13
+ > `subagents[].model` renamed to `dispatchModel`.** A consumer reading either old field name off
14
+ > `result.json` now gets `undefined`. Both keep their original (dispatch-input) meaning — the
15
+ > resolved counterparts (`resolvedAgentType`/`resolvedModel`, below) are new, separate fields.
16
+
17
+ ### Added
18
+
19
+ - **Parity baseline for Claude Desktop 1.20186.1.** `baseline: latest` now resolves to `desktop-1.20186.1`
20
+ — a patch-only Desktop release (egress allowlist, spawn config, and the Cowork system-prompt
21
+ fingerprint are unchanged from 1.20186.0). The pinned VM agent ELF was re-synced from 2.1.202 to
22
+ 2.1.205 after a live Desktop self-update within the same Desktop build, restoring agreement between
23
+ the pinned agent version and what's staged on disk so the host-loop live lane resolves its binary.
24
+ - **Cassette prompt-asset fingerprint.** Prompt identity was previously keyed on `appVersion` alone, so
25
+ editing a committed prompt-asset file (`spawn.promptTemplate` / `subagentAppend` /
26
+ `subagentAppendHostLoop`) under the *same* app version silently replayed old-prompt behavior. A cassette
27
+ now records a hash over the referenced asset files (`fingerprint.promptAssetsHash`); a mismatch is a new
28
+ warn-by-default staleness finding (`prompt-assets`, fails under `--strict`), an unhashable pointer is a
29
+ distinct can't-verify finding (`unverifiable-prompt-assets`), and a cassette recorded before this field
30
+ existed surfaces a non-failing informational note instead of either.
31
+ - **Host-loop sub-agent tool aliasing and gated `web_fetch`.** The host-loop initialize request now
32
+ carries production's `Bash → mcp__workspace__bash` / `WebFetch → mcp__workspace__web_fetch` aliases
33
+ (single-hop; an alias never grants a tool a sub-agent's frontmatter didn't already bind).
34
+ - **Five new assertion keys for sub-agent path fidelity.** `no_vm_path_file_op: true` (hostloop-only —
35
+ no gated file tool attempted a `/sessions`-exact-or-prefixed path); `vm_path_denied: true`
36
+ (hostloop-only — at least one recorded denial targeted a `/sessions` path; needs `controlOut` on
37
+ replay); `path_denied: {tool?, path_matches?, source?, agent_scope?}` (hostloop-only — a denial
38
+ matching all given matchers, `source` ∈ `pretooluse`/`can_use_tool`/`permission_denied`); `no_path_denied:
39
+ true` (hostloop-only — no path denial recorded at all); and `subagent_file_write: {path?, path_suffix?,
40
+ tool?}` (**tier-agnostic** — a sub-agent-origin write/edit attempt at the given path, exact or suffix,
41
+ with a paired non-error tool result — the causal half of a delivery probe, pairs with `artifact_json` for
42
+ content). The four hostloop-only keys fail "cannot verify" (never vacuous-pass) on any other tier, since
43
+ `/sessions/...` is a valid path there.
44
+ - **`RunResult.fileToolAttempts`** — attempt-level telemetry (raw `file_path`/`path` as sent, the
45
+ first-match gate path, and `origin: main|subagent|unknown`) for every gated file-tool call
46
+ (Read/Write/Edit/Glob/Grep/MultiEdit), re-derivable on replay from the frozen tool_use stream.
47
+ - **`RunResult.pathDenials`** — decision-level path-denial telemetry merged from all three producers that
48
+ can deny a gated file-tool call on path grounds: the PreToolUse path gate's own callback, a denied
49
+ `can_use_tool` ask on a gated file tool with a path, and a pre-ask `permission_denied` correlated to a
50
+ recorded attempt. The `can_use_tool` source is reconstructible on replay only when the cassette carries
51
+ `controlOut`; without it the field is undefined (evidence-unavailable), never a vacuous `[]`.
52
+ - **`RunResult.subagents[].resolvedAgentType` / `.resolvedModel`.** The agent stream states the
53
+ binary-*resolved* child type (via `task_started`) and the resolved child model (via the dispatch's own
54
+ `tool_use_result` envelope) — previously discarded. Both are joined strictly by `toolUseId` and
55
+ corroborate each other without overwriting stronger evidence. `subagent_dispatched` and
56
+ `subagent_output_contains` now match on dispatch type, resolved type, *or* description, so a type-less
57
+ dispatch that resolved to e.g. `general-purpose` is selectable by either assertion.
58
+ - **A loud warning on type-omitted sub-agent dispatches.** A `Task` dispatch that carries no
59
+ `subagent_type` at all falls back to the built-in `general-purpose` agent (`tools:["*"]` — the wildcard
60
+ surface, including workspace bash) — faithful production behavior, and one that fires routinely, not
61
+ just at the edges. The harness now warns loudly when this fallback is observed (an *explicit*
62
+ `subagent_type: "general-purpose"` is a deliberate author choice and does not warn) and records the
63
+ omission on the run (`subagents[].dispatchTypeOmitted`), so it's visible in the run's own record, not
64
+ only in a live terminal warning.
65
+ - **`agent_env` session config knob** (`subagent_model`, `tool_search: "auto"|"off"`,
66
+ `disable_experimental_betas`) applies `CLAUDE_CODE_SUBAGENT_MODEL` / `ENABLE_TOOL_SEARCH` /
67
+ `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS` **uniformly across all four fidelity tiers**. Previously an
68
+ operator-exported value of any of these three leaked into hostloop/protocol (which spawn over the full
69
+ operator shell env) but never into container/microvm (a constructed allowlist) — the same session
70
+ behaved differently depending only on which tier it ran at. The three keys are now also scrubbed from
71
+ the operator layer on the two inheriting tiers before any baseline/knob overlay, so a stray shell value
72
+ can never leak through on some tiers and not others; setting any `agent_env` field moves the session
73
+ fingerprint so `verify-cassettes` surfaces the drift on an older cassette.
74
+ - **`sync` now guards sub-agent prompt and path-gate fidelity.** Two new structural sentinels pin the
75
+ two-branch (`subagent_env_hl`/`subagent_env_vm`) sub-agent append — the branch-selection ternary,
76
+ the per-branch content fingerprint, and the substitution map's keys *and* values (a host/VM cwd swap
77
+ fails) — and the host-loop path gate's module-bounded shape (gated/excluded tool sets, every deny text,
78
+ the read-only-guard-before-containment order). A separate pinned gate hard-stops the sync (not just
79
+ warns) if production flips on a server-delivered override of the sub-agent append text, since that flip
80
+ is invisible to the content sentinel and the harness has no captured override text to fall back to.
81
+ - **`docs/subagents.md`** — the sub-agent capability/path reference: the tier-qualified outputs-addressing
82
+ contract (host-loop's cwd-relative `artifacts/...` vs. the VM loop's `mnt/outputs/artifacts/...` —
83
+ there is no single cross-tier literal form), a full capability/path matrix, the sub-agent tool-composition
84
+ rules (frontmatter allowlists, the `"*"` wildcard, `disallowedTools`, tool aliases, never-subagent
85
+ tools), the type-less dispatch trap, model-resolution precedence, and lifecycle notes (no resume, a
86
+ depth cap of 5 with no fan-out cap, fork-dispatch environment-append exclusion).
87
+
88
+ ### Changed
89
+
90
+ - **Cassette v9 read floor.** `readCassette` now refuses cassettes recorded below v9
91
+ (`MIN_SUPPORTED_CASSETTE_VERSION`) with a clear re-record error instead of silently tolerating
92
+ legacy formats; the pre-v9 reconstruction branches (`contentSigAlgoOf`, `buildFolderPrefixMap`'s
93
+ session-fallback, `cmdRehash`'s pre-v3/pre-v6 checks) are deleted as unreachable, and SPEC.md's
94
+ stability contract states the floor. The superseded `schema/cassette.v2`–`v8.json` files are
95
+ removed (v9/v10 remain).
96
+ - **`verify-run` fails closed on degraded evidence.** A `result.json` that parses but is
97
+ structurally invalid (no `"success" | "error"` result field) refuses instead of being certified
98
+ as success; an `events.jsonl` with unparseable lines, or one yielding fewer gates than
99
+ `trace.json` recorded questions, refuses instead of passing answer coverage at a hollow 0/0.
100
+ - **Verdict honesty for missing scan evidence.** When post-run scan evidence is unavailable, the
101
+ `host-path` / `outputs-delete` guards report `?` (unverified) instead of a false ✓, and a new
102
+ warn-severity `scan_unavailable` signal surfaces in the verdict JSON.
103
+ - **Release gates hardened.** The CI scenario suite hard-fails on the canonical repo when
104
+ `ANTHROPIC_API_KEY` is missing on non-PR events (fork and Dependabot PRs keep the warn+skip);
105
+ image publishing now waits on ci.yml success for the tagged commit and verifies tag↔package
106
+ lockstep via a composite action shared with the npm release gate (`workflow_dispatch` remains
107
+ the documented manual-republish escape hatch). An admin can deliberately skip the live scenario
108
+ suite for a single release by setting a `SKIP_LIVE_SCENARIOS` repo variable to that exact commit
109
+ SHA — an explicit, auditable, per-commit override (a later commit can't inherit it), not a silent
110
+ skip; see [RELEASING.md](./RELEASING.md).
111
+ - `ResourceSummary.probeFailures` (shipped at runtime in 0.29.0) is now declared on the RunResult
112
+ type and JSON Schema; assertion-side consumption remains deferred.
113
+ - **Breaking (pre-1.0): `subagents[].agentType` renamed to `dispatchAgentType`, and `subagents[].model`
114
+ renamed to `dispatchModel`.** Both keep their original (dispatch-input) meaning — the rename exists only
115
+ because a *resolved* type/model now lands beside each (`resolvedAgentType`/`resolvedModel`, above) and
116
+ the old names would have been ambiguous between "what the dispatch asked for" and "what the binary
117
+ actually ran." No deprecation window; a consumer reading the old field names gets `undefined`.
118
+ - **The host-loop `PreToolUse` path gate is re-ported to Desktop 1.20186.1 semantics.** Plugin/skill
119
+ content loses its blanket path exemption and becomes a read-only category instead (a mutating tool
120
+ targeting it is now denied, with its own message, rather than silently passing through); the read-only
121
+ guard gains the two other production categories — an uploads hardlink write-block and spooled tool
122
+ results (`projects`) — each with per-session-type deny text; the spool dir joins the readable roots. The
123
+ harness's own `mode: r` connected-folder extension and the `/sessions` MultiEdit message-selection
124
+ nuance are preserved across the re-port.
125
+
126
+ ### Fixed
127
+
128
+ - **Host-loop sub-agent dispatches now receive the correct environment description.** The harness always
129
+ sent sub-agents the VM-branch environment text, telling them their files exist only in a sandbox while
130
+ their file tools actually reach the real host filesystem — one of two prompt bugs that could seed
131
+ `/sessions/...` paths into a host-loop sub-agent's dispatch. A new host-loop-specific append
132
+ (`subagent_env_hl`) is now selected whenever `effectiveFidelity === "hostloop"` (container/microvm keep
133
+ the existing VM-branch text; `protocol` sends neither, a documented divergence), delivered on **both**
134
+ the `run`/`skill` and `chat` lanes (previously `chat` sent no sub-agent append at all).
135
+ - **The host-loop "Shell access" prompt's outputs bullet no longer teaches VM paths to native file tools.**
136
+ Its file-tool side substituted the VM session root where production substitutes the host outputs
137
+ directory — the other of the two prompt bugs behind stray `/sessions/...` sub-agent paths.
138
+ - **`web_fetch` on host-loop is now gated through a single `can_use_tool` decision, matching production**,
139
+ instead of being pre-approved directly by the workspace handler. Bash stays pre-approved for the whole
140
+ session; a fetch now emits a real permission ask, answered once by the shared provenance/domain
141
+ decision (which marks provenance on allow) — closing the prior two-decision, self-approving shape.
142
+ - **Raw run logs are scrubbed on every exit path.** Previously only the success and
143
+ unanswered-gate paths scrubbed `events.jsonl`/`control-out.jsonl`, and `agent.stderr.log` was
144
+ never scrubbed — a mid-run fault kept raw (potentially secret-bearing) logs on disk. An
145
+ outermost finally now scrubs all three on success, salvage, and every rethrown fault, in the
146
+ run and chat lanes alike; leak/capability scanners still read the raw stream first.
147
+ - A truncated `present_files` result (fewer returned paths than inputs, or extras) now counts as
148
+ malformed evidence, so `no_scratchpad_leak` fails "cannot verify" instead of green-lighting
149
+ only the pairs that came back.
150
+ - Baseline pins caught up to `desktop-1.20186.1` (the parity sync landed the baseline file
151
+ without bumping them, leaving `check:versions` red).
152
+ - `docker/Dockerfile.agent` preinstalls `pyyaml==6.0.3`, matching real rootfs provisioning (the
153
+ container tier no longer exercises the vendored-YAML fallback the real environment never needs).
154
+
155
+ ### Removed
156
+
157
+ - The subagent-grant canary (`src/canary/`, its fixture, test, and empty snapshot) — never wired
158
+ to `sync`, and its drift snapshot could never fail.
159
+ - `scripts/boot-rootfs-vz.ts` — the tested-infeasible generic-VZ boot attempt; its finding is
160
+ preserved in `docs/fidelity-gaps.md`.
161
+ - From the npm tarball: the 1.5 MB README banner and the compiled critique evaluator (relocated
162
+ to `scripts/lib/critique/`, dev-only) — the package drops from 2.57 MB to ~1.05 MB compressed.
163
+
164
+ ### Docs
165
+
166
+ - README's `## Status` section collapsed to the baseline-pin sentence (the perishable
167
+ verification narrative lives in this changelog; the feature catalogue lives in the sections
168
+ above it).
169
+ - `docs/cowork-spawn-contract-1.12603.1.md` is frozen as historical research and no longer a
170
+ release version-pin; `check:versions` gates on the SKILL.md and README pins only.
171
+ - Dropped the stale first-run egress-race gotcha (proxy startup has been synchronous for a
172
+ while); documented `verify-run`'s fail-closed refusal family in `docs/scenario.md` and the
173
+ companion skill.
174
+ - `docs/subagents.md` (new — see Added) is cross-linked from `docs/boundary.md` and `docs/scenario.md`;
175
+ `docs/plugin-root.md`'s tier table is corrected — `CLAUDE_PLUGIN_ROOT` is absent from a Bash-tool
176
+ subprocess's env on **every** tier, not host-loop only, as the table previously implied.
177
+
9
178
  ## [0.29.0] — 2026-07-11
10
179
 
11
180
  ### Added
package/CONTRIBUTING.md CHANGED
@@ -51,7 +51,7 @@ test/ vitest unit tests
51
51
  baselines/ committed platform baselines (one per Desktop release)
52
52
  examples/ user-facing worked examples (CI-verified): scenarios/ sessions/ skills/ data/
53
53
  e2e/ the harness's own fidelity self-tests: scenarios/ sessions/
54
- fixtures/ harness runtime fixtures (e.g. subagent-grants.json)
54
+ fixtures/ harness runtime fixtures (protocol/ golden control-response vectors)
55
55
  docs/ guides + references
56
56
  ```
57
57
 
package/README.md CHANGED
@@ -91,7 +91,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
91
91
 
92
92
  > **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
93
93
  > <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
94
- > From a global install (`npm i -g "cowork-harness@>=0.29.0"`), point at the package root instead:
94
+ > From a global install (`npm i -g "cowork-harness@>=0.30.0"`), point at the package root instead:
95
95
  > `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
96
96
  > (or copy the cassette into your own project and pass that path).
97
97
 
@@ -101,7 +101,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
101
101
  > - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
102
102
  > - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
103
103
  > - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
104
- > - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=0.29.0"`.
104
+ > - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=0.30.0"`.
105
105
 
106
106
  Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
107
107
  test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
@@ -126,7 +126,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
126
126
  claude plugin install cowork-harness@cowork-harness
127
127
  ```
128
128
 
129
- The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=0.29.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
129
+ The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=0.30.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
130
130
 
131
131
  It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
132
132
 
@@ -147,7 +147,7 @@ A global install is enough for CI `lint`, reading the teaching skill, and replay
147
147
  To `run` the worked examples live or copy them as a starting point, use a source checkout. (The marketplace
148
148
  skill install itself only pulls `.claude/skills/cowork-harness/` — SKILL.md + `references/` + `scenario.py`/
149
149
  assertion keys, per `.claude-plugin/marketplace.json`'s `source` — not the rest of this table; the full set
150
- above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=0.29.0"` — see
150
+ above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=0.30.0"` — see
151
151
  [above](#drive-it-from-claude-code-companion-skill) — which pulls the same npm package as the global-install row.)
152
152
 
153
153
  ### Prerequisites for anything above `protocol` fidelity
@@ -225,8 +225,8 @@ cowork-harness lint examples/scenarios/*.yaml
225
225
  >
226
226
  > | When | Assertions |
227
227
  > |---|---|
228
- > | Always | `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `skill_triggered`/`no_skill_triggered`, `max_cost_usd`/`max_tokens`/`tool_calls_max`/`max_turns` (against the *frozen recording's* spend, not fresh spend — a live `run` catches a real budget regression), `max_tool_errors`, `max_redundant_tool_calls`, `skill_available`, `connector_available`, `skill_tool_used`, `compaction_occurred`, `all_tasks_completed`, `task_count_min`, `task_status`, `no_scratchpad_leak`, `present_files_called`, `result`, the verdict modifiers |
229
- > | Only if the cassette carries `controlOut` | `question_asked`, `questions_count_max`, `gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked` |
228
+ > | Always | `transcript_*`, `tool_*`, `subagent_*`, `no_vm_path_file_op`, `dispatch_count_max`, `skill_triggered`/`no_skill_triggered`, `max_cost_usd`/`max_tokens`/`tool_calls_max`/`max_turns` (against the *frozen recording's* spend, not fresh spend — a live `run` catches a real budget regression), `max_tool_errors`, `max_redundant_tool_calls`, `skill_available`, `connector_available`, `skill_tool_used`, `compaction_occurred`, `all_tasks_completed`, `task_count_min`, `task_status`, `no_scratchpad_leak`, `present_files_called`, `result`, the verdict modifiers |
229
+ > | Only if the cassette carries `controlOut` | `question_asked`, `questions_count_max`, `gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked`, `vm_path_denied`, `path_denied`, `no_path_denied` |
230
230
  > | Only if the cassette carries an `artifacts` manifest | `file_exists`, `user_visible_artifact`, `artifact_json`, `computer_links_resolve`, `computer_links_resolve_if_present`, `no_unexpected_files`, `input_unmodified` |
231
231
  > | Always skipped (live-only) | `egress_*`, `expect_denied`, `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`, `no_mcp_error`, `max_peak_rss_bytes`, `semantic_matches` — keep these in a periodic live `run` |
232
232
  >
@@ -766,6 +766,7 @@ a global install has them locally too, not just on GitHub.
766
766
  | [docs/boundary.md](./docs/boundary.md) | The limitations model — sealed FS, default-deny egress, MCP-only crossing; how each tier enforces it; how to verify. |
767
767
  | [docs/session.md](./docs/session.md) | Every `sessions/*.yaml` field and its Cowork mapping. |
768
768
  | [docs/scenario.md](./docs/scenario.md) | `scenarios/*.yaml` — prompt, scripted answers, assertions. |
769
+ | [docs/subagents.md](./docs/subagents.md) | The sub-agent capability/path model — tier-qualified outputs contract, tool-composition rules, the type-less dispatch trap. |
769
770
  | [docs/chat.md](./docs/chat.md) | The interactive `chat` REPL — multi-turn debugging, flags, attaching files/folders. |
770
771
  | [docs/debugging.md](./docs/debugging.md) | Debugging a run — `inspect`/`trace`/`chat`/`verify-run` for a misbehaving skill; the false-green hunt for a green you don't trust. |
771
772
  | [docs/cassette.md](./docs/cassette.md) | `record`/`replay` cassettes — what replay checks, which assertions are skipped. |
@@ -798,25 +799,7 @@ inputs/outputs. Human-readable terminal text is explicitly **not** part of the c
798
799
 
799
800
  ## Status
800
801
 
801
- **Verified end-to-end against the live staged agent (VM ELF 2.1.202 / native host app 2.1.205 / asar 1.20186.0).** Host-side parity (egress/gates/mount/web_fetch facts) re-derived via `sync` — no unknown deltas; `asarFingerprint` moved (a 1.19367.0→1.20186.0 minifier re-anchor: hoisted helpers became namespace-method calls, spreads became export aliases — value-resolved contract unchanged, extractor re-anchored in `src/sync/cowork-sync.ts`), and `nativeStagedPath` moved to the natively-staged `2.1.205` (the native macOS app and the VM ELF version on independent cadences). The reconstructed prompt appends were re-checked against the new asar: the system-prompt source added a trailing `{{modelIdentity}}` placeholder, but it is deployment-gated and stripped on the first-party Cowork emulated here, so the rendered prompt is byte-identical — the load-bearing "is NOT Claude Code" correction, the host-loop `## Shell access` generator, and the web_fetch/mount facts all still hold. The live scenario suite (`protocol` + `container` + `hostloop` tiers) passes against this baseline (fresh live re-run 2026-07-11; 5/6 scenarios, `example-pdf-skill` needs the full-parity image). The latest shipped baseline — what `baseline: latest` resolves to (`cowork-harness list`) — is **`desktop-1.20186.0`** (the committed baseline file and version-lockstep checks track it; it additionally carries the synced per-model effort config, and there are no known spawn-contract deltas — the `1.20186.0` bundle's anchor drift was benign and is resolved in `src/sync/cowork-sync.ts`).
802
-
803
- - ✅ **Three isolation tiers (L0/L1/L2) + two loop-mode overlays** — see [Fidelity tiers](#fidelity-tiers-pick-per-scenario--per-ci-job) for the full decision guide; in short:
804
- - `protocol` (L0, control loop only), `container` (L1, sandboxed arm64 + per-run default-deny egress sidecar), `microvm` (L2, real Apple-VZ Linux microVM + guest firewall).
805
- - `hostloop` (production split-execution: native host agent process, file tools routed host-side via the workspace SDK-MCP server into a Docker VM sidecar) and `cowork` (auto-picks `hostloop` vs `container` the way Cowork does, via gate `1143815894`).
806
- - Egress enforced at `container`/`microvm`/`hostloop`; `boundary-check` reports **ALL CONSTRAINTS ENFORCED**.
807
- - ✅ **Three-seam driver** — `AgentSession` (control protocol) → `Decider` (scripted + `on_unanswered` policy, no silent false-greens) → `Run` (turn loop, sub-agent dispatch tree, `RunRecord`). Multi-turn `chat`, deterministic cassette `record`/`replay` (no token), `run.jsonl`/`trace.json` logging with secret-scrub.
808
- - ✅ **Answering live questions, every way you'd need** — four independent channels, all incapable of silently picking option 1 (an unhandled question always fails loud):
809
- - `--decider-llm --intent "<one line>"` — a model picks per question, steered by your test intent (the ergonomic default); defaults to **Sonnet** (a weaker model prose-declines ambiguous judgment gates → fail-loud); override with `--decider-model <id>` to pin a cheaper or stronger model. It replies with the option **number** (mapped to the exact label, so it can't whiff by parroting label text); an out-of-set reply fails loud, and non-determinism is flagged so a green isn't mistaken for a scripted pass. ⚠️ A *decided* gate whose answer feeds a **semantic** assertion can still false-green — script those instead.
810
- - `--answer-policy <yaml>` / `--answer "rx=c"` — declarative regex→label, deterministic for CI (accepts a **stable partial anchor** for skills whose labels drift run-to-run).
811
- - `--decider-cmd '<helper>'` — custom logic via the Python `serve_decider(fn)` adapter, which pre-builds the wire loop so the helper writes only the decision.
812
- - `--decider-dir <dir>` — the *driving agent* answers each gate **in-band** with full context: it arms a Monitor on `cowork-harness gates <dir> --follow` and replies with `cowork-harness answer <dir> --gate <N> --choose <label>`; the session-under-test stays live, no resume/re-worded question (binary-verified).
813
- - **multiSelect gates** are answerable on every channel — repeat `--choose` (or, for `--decider-llm`, a comma-list of option numbers; or a JSON-array reply from a `--decider-cmd` helper / hand-written `resp-N.json`); all deliver the binary-verified `", "`-joined wire shape. Every channel keeps stdout free, so all compose with `--output-format json`. Validate any decider in ~2s with `cowork-harness decide`.
814
- - ✅ **Sub-agent aggregation + `trace`** — recognizes the real `Agent` dispatch tool (binary-verified; `Task` is its alias) so `subagent_dispatched`/`dispatch_count_max` fire under `--fidelity cowork`, excluding the `TaskCreate` todo list; `cowork-harness trace <id> --view tools` digests `events.jsonl` (each tool row now shows its **result status** `ok`/`error`), and `trace <id> --view questions` shows the gate lifecycle (**question → injected answer → delivered result**); `result.json`/`--keep` surface the deep `mnt/outputs` deliverable path.
815
- - ✅ **Answer delivery is verified, not assumed** — an AskUserQuestion answer is injected as the binary's full tool input (`{questions, answers}`, ELF-verified), so the answer actually reaches the model; `RunResult.gateDeliveries[]` + the `gate_answers_delivered` assertion catch any delivery failure, and `RunResult.toolCounts` gives the **truthful** per-tool call count (host-routed `WebSearch` shows here, not the always-0 `usage.server_tool_use`).
816
- - ✅ **Binary-grounded fidelity** — cwd `/sessions/<id>`, the three-channel MCP model (`--mcp-config` honored in plain cowork mode; host/API-routed `web_fetch`; SDK-server delivery), host-loop tool partition, auth-env token-only drop, and production GrowthBook gates pinned per release.
817
- - ✅ The agent binary is **bind-mounted from your own install** at run time — nothing Anthropic-owned is in any image or distributed.
818
- - ✅ **File provision & session resume** — `--upload <file>` / `--folder <dir>` attach files & connect folders (`mnt/uploads`, `mnt/<folder-name>`); `--session-id <id>` + `--resume` persist and continue a session via the agent's *native* resume (binary-verified), so checkpoint-and-resume gated skills are testable. Demonstrated end-to-end in the live-contract suite (`test/live-contract.test.ts`: codeword established → resumed → recalled).
819
- - ✅ **The full unit suite + the live-contract suite green**; a `cowork` pytest lane (`python/`) for skill authors.
820
- - ℹ️ **Auth:** a `claude setup-token` OAuth token (or `ANTHROPIC_API_KEY`), provided via the env, `--dotenv <path>`, `./.env`, or the install's own `.env` (gitignored; keep it out of mounted folders). It's passed into the sandbox **off the process argv** (Docker: `-e KEY` inherit-by-name; microVM: a stdin prologue) so the token isn't visible via `ps`/`/proc`, scrubbed from logs, never persisted in a runtime path; the token-only path mirrors the desktop. A fresh `CLAUDE_CONFIG_DIR` alone breaks local OAuth.
821
-
822
- See [SPEC.md](./SPEC.md) (the testable contract), [DESIGN.md](./DESIGN.md) (architecture + parity), and [CHANGELOG.md](./CHANGELOG.md).
802
+ The latest shipped baseline — what `baseline: latest` resolves to (`cowork-harness list`) — is
803
+ **`desktop-1.20186.1`**. Release-by-release verification notes (what was re-verified against
804
+ which live agent/asar) are recorded in [CHANGELOG.md](./CHANGELOG.md); the feature catalogue
805
+ this section used to duplicate lives in the sections above.