cowork-harness 4.13.1 → 4.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. package/.claude/skills/cowork-harness/SKILL.md +5 -5
  2. package/.claude/skills/cowork-harness/references/assertion-catalog-agents-skills-budgets.md +1 -1
  3. package/.claude/skills/cowork-harness/references/assertion-catalog-gates-hooks-modifiers.md +1 -1
  4. package/.claude/skills/cowork-harness/references/assertion-catalog-outcome-files-tools.md +1 -1
  5. package/.claude/skills/cowork-harness/references/assertion-catalog.md +1 -1
  6. package/.claude/skills/cowork-harness/references/assertions-guide.md +1 -1
  7. package/.claude/skills/cowork-harness/references/authoring.md +1 -1
  8. package/.claude/skills/cowork-harness/references/ci-recipe.md +6 -6
  9. package/.claude/skills/cowork-harness/references/critique.md +1 -1
  10. package/.claude/skills/cowork-harness/references/debugging.md +1 -1
  11. package/.claude/skills/cowork-harness/references/eval.md +1 -1
  12. package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +12 -1
  13. package/.claude/skills/cowork-harness/references/gotchas.md +8 -1
  14. package/.claude/skills/cowork-harness/references/hillclimb-recipe.md +1 -1
  15. package/.claude/skills/cowork-harness/references/hillclimb.md +1 -1
  16. package/.claude/skills/cowork-harness/references/measurement.md +1 -1
  17. package/.claude/skills/cowork-harness/references/run-record-replay.md +2 -2
  18. package/.claude/skills/cowork-harness/references/scenario-schema.md +2 -2
  19. package/.claude/skills/cowork-harness/references/semantic-judging.md +1 -1
  20. package/.claude/skills/cowork-harness/references/task-recipes.md +1 -1
  21. package/CHANGELOG.md +54 -0
  22. package/CONTRIBUTING.md +9 -1
  23. package/DESIGN.md +2 -2
  24. package/README.md +9 -5
  25. package/baselines/desktop-4.16336.0.json +2056 -0
  26. package/baselines/prompts/cowork-system-prompt-fingerprints.json +9 -1
  27. package/dist/assert.js +4 -2
  28. package/dist/baseline.js +4 -0
  29. package/dist/cli.js +15 -2
  30. package/dist/hostloop/process-cwd.js +85 -2
  31. package/dist/prompt/subagent-manifest.js +28 -2
  32. package/dist/prompt.js +7 -4
  33. package/dist/run/cassette.js +9 -1
  34. package/dist/run/execute.js +2 -0
  35. package/dist/runtime/hostloop.js +34 -9
  36. package/dist/sync/baseline-diff.js +8 -0
  37. package/dist/sync/cowork-sync.js +292 -9
  38. package/dist/sync/hostloop-fence.js +262 -0
  39. package/dist/sync/spawn-env-registry.js +24 -2
  40. package/dist/types.js +3 -0
  41. package/docs/boundary.md +2 -0
  42. package/docs/cassette.md +3 -1
  43. package/docs/ci.md +2 -2
  44. package/docs/cli.md +5 -5
  45. package/docs/companion-skill.md +2 -2
  46. package/docs/fidelity-gaps.md +72 -4
  47. package/docs/maintenance.md +7 -2
  48. package/docs/scenario.md +1 -1
  49. package/examples/replays/README.md +1 -1
  50. package/examples/replays/example-multiselect-gate.cassette.json +1 -1
  51. package/examples/replays/example-pdf-skill.cassette.json +1 -1
  52. package/examples/replays/hostloop-computer-links.cassette.json +68 -62
  53. package/package.json +1 -1
  54. package/scripts/ci/hang-dump.sh +73 -0
@@ -3,8 +3,8 @@ name: cowork-harness
3
3
  description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, asserting artifacts, egress, or sub-agent dispatch, measuring how long each tool call took (toolDurations / trace), or debugging a failed run or verdict from its result.json or transcript. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. Also for comparing two versions of a skill before merging an edit — did it make answers worse? (`eval`: paired A/B, pinned models) — or improving one round by round (`hillclimb`, the `/claude-api hillclimb` runner). NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold / critique / stats / eval / hillclimb commands and the session-vs-scenario split.
4
4
  metadata:
5
5
  author: cowork-harness
6
- version: 4.13.1
7
- tracks-harness: cowork-harness 4.13.1 (baseline desktop-4.4212.0)
6
+ version: 4.14.0
7
+ tracks-harness: cowork-harness 4.14.0 (baseline desktop-4.16336.0)
8
8
  ---
9
9
 
10
10
  # cowork-harness
@@ -26,8 +26,8 @@ allowlist). This skill exists mostly to keep you out of those traps — the *Inv
26
26
  full landmine catalog in [`references/gotchas.md`](references/gotchas.md) are the highest-value part.
27
27
  Read them.
28
28
 
29
- > **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.13.1` (baseline
30
- > `desktop-4.4212.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
29
+ > **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.14.0` (baseline
30
+ > `desktop-4.16336.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
31
31
  > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
32
32
 
33
33
  ## Preflight — make sure the harness can actually run
@@ -43,7 +43,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
43
43
 
44
44
  - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, the baseline and the `device_bridge` image (built, or built on the first bridge run) in one pass. The bullets below explain each thing it checks (and how to fix it).
45
45
  - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
46
- - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.13.1**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.13.1" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.13.1"`. **Pin `@^4.13.1`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
46
+ - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.14.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.14.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.14.0"`. **Pin `@^4.14.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
47
47
 
48
48
  This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
49
49
  OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -1,6 +1,6 @@
1
1
  # Assertion catalog: sub-agents, skills and connectors, budgets, tasks, delivery
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
4
4
  keys is an AND; glob vs regex name matching) and the verdict-signal table are in
5
5
  [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Assertion catalog: gates, hooks, path denial, verdict modifiers, egress, judged keys
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
4
4
  keys is an AND; glob vs regex name matching) and the verdict-signal table are in
5
5
  [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md).
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Assertion catalog: outcome, transcript, files and artifacts, tools, tool results
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
4
4
  keys is an AND; glob vs regex name matching) and the verdict-signal table are in
5
5
  [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Assertion catalog
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Every `assert:` key with its semantics, and the
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Every `assert:` key with its semantics, and the
4
4
  verdict-signal table. Which keys survive `replay` is in [`scenario-schema.md`](./scenario-schema.md#replay-class);
5
5
  the scenario and session YAML fields are there too.
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Assertions guide
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
4
4
 
5
5
  ### Assertions: two orthogonal axes
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Authoring a scenario
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
4
4
 
5
5
  ## Part I — AUTHOR a scenario
6
6
 
@@ -1,6 +1,6 @@
1
1
  # CI recipe — replay vs live lanes
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`).
3
+ Self-contained reference. Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`).
4
4
 
5
5
  **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
6
6
  job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
17
17
  CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
18
18
  copy-pasted recipe that omits the input takes a major bump with no say in it. `^4` holds the major, needs no
19
19
  patch number to remember, and only wants a human decision at the next major. Pin an exact version
20
- (e.g. `version: "4.13.1"`) instead when you want byte-reproducible CI.
20
+ (e.g. `version: "4.14.0"`) instead when you want byte-reproducible CI.
21
21
 
22
22
  Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
23
23
  cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -36,7 +36,7 @@ jobs:
36
36
  - uses: actions/checkout@v4
37
37
  - name: Stage the agent binary (official channel, sha256-verified against the pinned baseline)
38
38
  run: |
39
- V=2.1.309 # match your scenario's pinned baseline's agentVersion
39
+ V=2.1.317 # match your scenario's pinned baseline's agentVersion
40
40
  # The release channel is NOT always the stable one. Desktop also stages release CANDIDATES,
41
41
  # served from .../claude-code-releases/rc/<commit>/. For some versions the stable path 404s
42
42
  # (2.1.255); for others it returns 200 and serves a DIFFERENT BUILD UNDER THE SAME VERSION
@@ -82,7 +82,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
82
82
  GitHub-hosted runners, no token/Docker/agent:
83
83
 
84
84
  ```yaml
85
- - run: npm i -g "cowork-harness@^4.13.1"
85
+ - run: npm i -g "cowork-harness@^4.14.0"
86
86
  - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
87
87
  # no silent false-greens. WITHOUT --strict this
88
88
  # step cannot fail on a WARN-class rule (e.g.
@@ -411,7 +411,7 @@ jobs:
411
411
  with: { node-version: '24' }
412
412
  - uses: actions/setup-python@v5
413
413
  with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
414
- - run: npm i -g "cowork-harness@^4.13.1"
414
+ - run: npm i -g "cowork-harness@^4.14.0"
415
415
  - run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
416
416
  - run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
417
417
  - run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
@@ -440,7 +440,7 @@ jobs:
440
440
  echo "live=true" >> "$GITHUB_OUTPUT"
441
441
  fi
442
442
  - if: steps.guard.outputs.live == 'true'
443
- run: npm i -g "cowork-harness@^4.13.1"
443
+ run: npm i -g "cowork-harness@^4.14.0"
444
444
  - if: steps.guard.outputs.live == 'true'
445
445
  run: cowork-harness run scenarios/ --output-format json
446
446
  env:
@@ -1,6 +1,6 @@
1
1
  # Critique — the facts a plugin install can't otherwise reach
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). This is **not** a trim of the full
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). This is **not** a trim of the full
4
4
  [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
5
5
  flags, cost, reading several critiques, known limitations all live there). This file covers exactly what a
6
6
  plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's
@@ -1,6 +1,6 @@
1
1
  # Debugging a run
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
4
4
 
5
5
  ## Part III — Debug
6
6
 
@@ -1,6 +1,6 @@
1
1
  # `eval` — paired before/after comparison of a skill edit (EXPERIMENTAL)
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). The full guide is
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). The full guide is
4
4
  [docs/eval.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/eval.md); this is the part you
5
5
  need while running it.
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Fidelity tiers & answer paths
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`).
3
+ Self-contained reference. Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`).
4
4
 
5
5
  > **This page vs. the repo docs.** This is the **offline snapshot** that ships inside the installed
6
6
  > plugin — it is self-contained on purpose. The repo carries four other fidelity views, each answering a
@@ -55,6 +55,17 @@ and sees `/sessions/<id>/...` paths. Consequences an agent (or a debugging autho
55
55
  staged host uploads dir; it does not need to be copied into `outputs/` first.
56
56
  - In real Cowork an upload is a **hardlink** in the session's uploads dir; the harness **copies** it
57
57
  instead. Read-reachability is the same; only the advertised parent dir can differ.
58
+ - **A second layer on `desktop-4.16336.0` and later**, when the native agent is 2.1.315 or newer: the agent
59
+ itself refuses file reads and writes outside its working directories (outputs and every connected folder),
60
+ in addition to the path hook above. Uploads, the spooled tool results, skills and staged plugins are exempt from
61
+ its read block. Such a refusal comes from the agent, so its wording is not the hook's message; a path the hook
62
+ allows can still be refused here if it sits outside all of those roots. Older baselines and agents run without it.
63
+ An agent binary outside Desktop's staging layout (typically a `COWORK_HOST_AGENT_BINARY` override) is asked for its
64
+ version once; if it gives none, this layer is left off and the run warns that it is.
65
+ - **The agent's own claude.ai Artifacts server (`claude-artifacts`) is not attached on any tier.** From
66
+ `desktop-4.16336.0`, an ordinary Cowork session lists it among its MCP servers; the harness leaves it off so a test
67
+ run cannot touch the operator's real Artifacts. A skill that calls it fails here and works in Cowork. See
68
+ [`docs/fidelity-gaps.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/fidelity-gaps.md) § "The agent's own Artifacts server is not attached."
58
69
 
59
70
  ### `microvm` prerequisites & lifecycle
60
71
 
@@ -1,6 +1,6 @@
1
1
  # Gotchas
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). The full "✓ passed ≠ correct" landmine catalog.
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). The full "✓ passed ≠ correct" landmine catalog.
4
4
 
5
5
  ## Gotchas — the "✓ passed ≠ correct" landmines
6
6
 
@@ -324,6 +324,13 @@ authorable). Reach for this list when debugging a run's behavior, that one while
324
324
  `probe-dispatch`, which have no `assert:` block. The failure message names the spelling for the lane
325
325
  you ran.
326
326
 
327
+ 28. **A `hostloop` file tool is refused with a message that is not the path hook's.** *Why:* on
328
+ `desktop-4.16336.0` and later, with a native agent 2.1.315 or newer, the agent's own outside-folder fence
329
+ refuses reads and writes outside outputs and the connected folders, in addition to the harness's path hook; only
330
+ uploads, the spooled tool results, skills and staged plugins are exempt from its read block. *Fix:* point the
331
+ tool at one of those roots by its host path (see `references/fidelity-and-answers.md`, "hostloop paths"); a
332
+ file the run needs from elsewhere belongs in a connected folder or an upload.
333
+
327
334
  For the assertion catalog, the YAML schema, the fidelity/answer tables, and the CI recipe, read the
328
335
  files in `references/` (the gotchas above are the full list; the references repeat only the
329
336
  assertion/replay-relevant ones).
@@ -1,6 +1,6 @@
1
1
  # Recipe 7 — Climb a skill with `/claude-api hillclimb` and the harness as its runner
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). It needs a `cowork-harness` whose
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). It needs a `cowork-harness` whose
4
4
  `hillclimb --help` lists `--skill` (help goes to stderr). This page is the loop's procedure, step by step, in the order of the
5
5
  `/claude-api hillclimb` guide. Every mechanic (flags, refusals, the gate, `regrade`, `freeze-ref`, exit codes,
6
6
  row keys) is in [`hillclimb.md`](hillclimb.md); the setup and the full list of differences from the guide's own
@@ -1,6 +1,6 @@
1
1
  # `hillclimb` — the runner for a `/claude-api hillclimb` loop
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). It needs a `cowork-harness` whose
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). It needs a `cowork-harness` whose
4
4
  `hillclimb --help` lists `--skill` (help goes to stderr). The command reference is
5
5
  [docs/cli.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/cli.md); this is the part a loop needs
6
6
  while it runs. It covers `run`, `check`, `state-template`, `freeze-ref` and `regrade`.
@@ -1,6 +1,6 @@
1
1
  # Measurement
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
4
4
 
5
5
  ### Measure — before/after, with/without (`--repeat`, `--ablate-skill`)
6
6
 
@@ -1,6 +1,6 @@
1
1
  # Run, record and lock
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
4
4
 
5
5
  ## Part II — RUN, RECORD & LOCK
6
6
 
@@ -229,7 +229,7 @@ Recognize these before "fixing" a non-bug:
229
229
  `markitdown`/`magika` (`ml_extract`), `cv2` (`cv`), `camelot`/`tabula` (`pdf_tables`), or `wand`
230
230
  (`magick`) can trip this even though real Cowork **ships** those (per the rootfs manifest captured at
231
231
  Desktop `2.9939.2` — `baselines/provisioning/rootfs-provisioning.json`, which is the dated evidence
232
- behind that sentence; 13 baselines have shipped since without a re-capture). The message says so ("likely a FALSE
232
+ behind that sentence; 14 baselines have shipped since without a re-capture). The message says so ("likely a FALSE
233
233
  NEGATIVE (real Cowork ships them)"). Fix: rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`, point
234
234
  `COWORK_AGENT_IMAGE` at it), or — if the skill's fallback is genuinely equivalent — assert
235
235
  `allow_missing_capability: true`. (Two sources: a skill *observed using* an omitted family, live lane;
@@ -1,7 +1,7 @@
1
1
  # Scenario & session schema, replay class, web_fetch, authoring gotchas
2
2
 
3
- Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.13.1`
4
- (baseline `desktop-4.4212.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
3
+ Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.14.0`
4
+ (baseline `desktop-4.16336.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
5
5
  [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
6
6
 
7
7
  **Minimal scenario** — `prompt` and `fidelity` are required:
@@ -1,6 +1,6 @@
1
1
  # `semantic_matches` — the full semantics
2
2
 
3
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). The one-line summary is in
3
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). The one-line summary is in
4
4
  [assertion-catalog-gates-hooks-modifiers.md](assertion-catalog-gates-hooks-modifiers.md); this is the whole contract, split out so the catalog stays within the
5
5
  agent's single-read size.
6
6
 
@@ -2,7 +2,7 @@
2
2
 
3
3
  Each recipe composes facts that live scattered across SKILL.md and the other references into one
4
4
  decision path. Every one answers a question a real fleet owner had to work out the hard way.
5
- Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
5
+ Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
6
6
  Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
7
7
  needed if your CLI meets SKILL.md's version floor.
8
8
 
package/CHANGELOG.md CHANGED
@@ -6,6 +6,60 @@ All notable changes to this project are documented here. The format is based on
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [4.14.0] - 2026-11-05
10
+
11
+ ### Upgrade notes
12
+
13
+ - **Cassettes: re-stamp or re-record.** `latest` moves to `desktop-4.16336.0`, so a cassette recorded through
14
+ `baseline: latest` reports `[stale] baseline moved 4.4212.0 → 4.16336.0 since record — re-record`; re-stamping
15
+ `fingerprint.baseline` clears that. The agent moves from 2.1.309 to 2.1.317, so a re-stamped cassette recorded on
16
+ 2.1.309 gets an `agent-version:` note; re-record it to clear the note. The host-loop retry sentence (see Added)
17
+ counts toward `promptAssetsHash` only where it is sent: a `hostloop` cassette on a 4.16336.0 or later baseline. A
18
+ re-stamped `hostloop` cassette therefore also reports that the baseline's prompt assets changed since record, and
19
+ `verify-cassettes` fails on it until it is re-recorded. A `container`, `microvm` or `protocol` cassette's hash is
20
+ the same on 4.16336.0 as on 4.4212.0, so re-stamping its baseline is enough, and a cassette pinned to an older
21
+ baseline is unaffected on every tier. The bundled cassettes are re-stamped and re-hashed, and the `hostloop` one is
22
+ re-recorded on 4.16336.0.
23
+ - **On `hostloop`, an agent binary outside Desktop's staging layout is now run once with `--version`** when the baseline
24
+ records the outside-folder fence, typically a `COWORK_HOST_AGENT_BINARY` override. Its answer decides whether the
25
+ fence is sent; if it gives no version, the fence is not sent and a warning says so. A baseline without a fence runs
26
+ nothing.
27
+
28
+ ### Added
29
+
30
+ - **Baseline for Claude Desktop 4.16336.0 (agent 2.1.317).** `desktop-4.16336.0` is the new newest baseline. It has
31
+ not had a live pass.
32
+ - **A host-loop agent now receives Cowork's outside-folder blocks** (configured and acceptance-verified; behaviour not
33
+ measured). From Desktop 4.16336.0, Desktop turns on the agent's own refusal of file reads and writes outside the
34
+ task's folders when the host CLI is new enough. The baseline records that floor as `hostLoopFence.minCli`, and on
35
+ `hostloop` (and `chat`, which spawns the same way) the harness sends the same settings when the agent it runs is at
36
+ or above it: outputs and every connected folder as working directories, and uploads, the spooled tool results,
37
+ skills and staged plugins as readable roots. A probe against the staged agent shows it accepts these settings. What
38
+ a tool call then does has not been measured in a live run. See docs/fidelity-gaps.md.
39
+ - **The host-loop sub-agent append gains Desktop's retry sentence for a workspace that is not ready yet.** On a
40
+ 4.16336.0 or later baseline, `hostloop` sub-agents receive it between the folder list and the skill sentence, with
41
+ the download-status instruction when the baseline records that gate on. VM tiers and older baselines are unchanged.
42
+
43
+ ### Changed
44
+
45
+ - **`sync` reads the shapes Desktop 4.16336.0 moved.** It reads the empty-key cleanup's new fourth member, the
46
+ hoisted read-only tail in the path hook, and the claude.ai memory-reset check in front of the auto-memory switch.
47
+ It classifies four new spawn env keys, and pins where Desktop sets the key that lets the agent attach its own
48
+ claude.ai Artifacts server. The harness leaves that key unset, so a test run cannot reach the operator's real
49
+ Artifacts; a skill that uses that server finds it missing here (docs/fidelity-gaps.md).
50
+ - **`hook_decision` and the hook-block keys follow agent 2.1.317**, which lets an SDK callback hook with
51
+ `onFailure: "block"` block on failure, as http hooks already could. No hook a plugin or settings file declares is
52
+ affected.
53
+ - **A test that hangs now fails with evidence instead of running to the CI time limit.** A synchronous child call
54
+ blocks its vitest process, so `testTimeout` could never fire on it, and on Node 24 and later a CLI process can
55
+ deadlock inside its own exit (an upstream Node bug, nodejs/node#64274). A run then sat until the job limit and lost
56
+ its log. The unit lane now watches for a blocked process: it saves stack dumps of the stuck children and the test
57
+ process, kills the children, and fails the test, or the run for a hang in a hook. The CI test jobs and the release's
58
+ Build + test also install gdb, take a job-level dump of every node process shortly before the test step's limit,
59
+ and upload the dumps and the test log when the step fails or reaches its own time limit. A run whose runner stops
60
+ responding keeps neither. This contains the hang; it does not fix the upstream bug. Contributor knobs are in
61
+ CONTRIBUTING.md.
62
+
9
63
  ## [4.13.1] - 2026-11-02
10
64
 
11
65
  A dependency-only patch: no code, baseline, result field or cassette changes.
package/CONTRIBUTING.md CHANGED
@@ -86,6 +86,14 @@ Paths inside a scenario/session resolve relative to that file (see [docs/session
86
86
  - **Consumer-visible workflow changes update the skill.** A change a scenario author would act on — a new assertion key, cassette field, CLI command, or a changed record/replay/verify workflow — must land with a matching update to `.claude/skills/cowork-harness/` (SKILL.md or `references/`). The machine-checkable slices are enforced (`test/skill-docs-sync.test.ts` pins the skill against the assertion-key catalog and the cassette schema's field list; `test/cli-help.test.ts` pins the `docs/cli.md` command table); prose workflows are on you — this checklist line exists because `effectiveFidelity` shipped consumer-visible and stayed undocumented in the skill until an external consumer flagged it.
87
87
  - **Typecheck.** `npm test` is `vitest run` — it strips types and does **not** typecheck. `tsconfig.json` covers only `src`; test files are typechecked solely by `tsconfig.test.json`, i.e. only via `npm run typecheck` (which `npm run ci` runs). After editing `src/`, run `npm run typecheck`, not just `npm test`.
88
88
  - **Format.** `npm run format:check` must pass (`npm run format:write` to fix).
89
+ - **A test that hangs fails instead (macOS and Linux).** A synchronous child call (`spawnSync`, `execFileSync`,
90
+ `execSync`) blocks its vitest process, so `testTimeout` cannot fire on it. The unit lane's first setup file,
91
+ `test/setup/sync-child-watchdog.ts`, watches for that: past the limit it saves stack dumps, kills the stuck children,
92
+ and fails the test (or the run, for a hang in a hook) with the dump directory. It needs `ps` (and `sample` on macOS,
93
+ `gdb` on Linux for backtraces), so on other platforms a hang is not caught. The limit is the largest of 90 s, the test's timeout + 30 s, and the
94
+ call's own `timeout` + 30 s, so give a sync call that can legitimately run longer than 90 s an explicit `timeout`.
95
+ Dumps go under `COWORK_TEST_HANG_DIR` (default `<tmpdir>/cowork-test-hangs`); `COWORK_TEST_HANG_T_MS` changes the
96
+ 90 s; `COWORK_TEST_WATCHDOG=0` turns it off.
89
97
 
90
98
  ## Validating a companion-skill edit (answer quality)
91
99
 
@@ -131,7 +139,7 @@ The provided [GitHub Actions workflow](https://github.com/yaniv-golan/cowork-har
131
139
  | Stage | Runs | Needs | Gates |
132
140
  |---|---|---|---|
133
141
  | **build** | format check · version-lockstep guard · typecheck · source guards · build · CLI smoke · token-free `replay` · `verify-cassettes` · `lint` | nothing | every push/PR |
134
- | **test** | the unit suite (vitest), sharded 4-way | nothing | every push/PR |
142
+ | **test** | the unit suite (vitest), sharded 4-way. If a test step fails or reaches its own time limit, its log and any hang dumps are uploaded as an artifact (`hangs-shard-N`); the floor and release jobs do the same. A runner that stops responding uploads nothing | nothing | every push/PR |
135
143
  | **floor** | the unit suite once, unsharded, on Node 22 — the version `engines.node` declares — so the floor is exercised rather than asserted (other jobs run Node 24, the Active LTS line) | nothing | every push/PR; gates the merge context |
136
144
  | **metric-runtimes** | the command-metric runtimes at their floors: every shipped Python metric file compiles on 3.9, the Python runner's audit hook and the closure tests run on 3.9, and the Node permission tests run on 22.12 (the `--experimental-permission` branch) | nothing | every push/PR; gates the merge context |
137
145
  | **action-self-test** | packs this commit and runs the packaged `uses: ./` Action across its full case set — pass (committed example cassette), usage-error fail (nonexistent path), assertion fail (checks the reporter renders a ❌ row), `lint`, and `analyze-skill`; `ci.yml` currently runs the packaged action **5** times (3 × `replay`, then `lint` and `analyze-skill`) — count the `with: command:` keys, not a bare `grep -c 'command:'`, which also matches step names and log strings | nothing | every push/PR |
package/DESIGN.md CHANGED
@@ -55,7 +55,7 @@ in how a file reaches the user, which is what changes skill behaviour: see
55
55
  [docs/scenario.md](./docs/scenario.md)'s `lane:` key for holding a run to either contract. The local lane:
56
56
 
57
57
  - VM bundle: `~/Library/Application Support/Claude/vm_bundles/claudevm.bundle/` (`rootfs.img`, `sessiondata.img`, `efivars.fd`, `machineIdentifier`, `gvisorMacAddress`, `vmIP`); a warm pool at `vm_bundles/warm/<sha>/`.
58
- - In-VM agent: `~/Library/Application Support/Claude/claude-code-vm/<ver>/claude` (currently **2.1.309**, per `baselines/desktop-4.4212.0.json`, an **ELF aarch64** binary), spawned by the host **in cowork mode via the `CLAUDE_CODE_IS_COWORK=1` env var** — *not* a `--cowork` flag (that flag is plugin-scope and the staged agent rejects it; see the control-protocol note below). Each baseline records this ELF's `sha256` (`agentBinary.sha256`/`shaProvenance`), and the resolver integrity-checks the binary it's about to run against that hash by default (opt out `COWORK_HARNESS_VERIFY_AGENT_SHA=0`), so "the same pinned agent" is enforced, not just asserted. Old versions are re-downloadable + verifiable from the official release channel — from the channel that baseline's `agentBinary.releaseBaseUrl` names, which is **not always the stable path** (Desktop stages release candidates too); the exact command is in `docs/maintenance.md`.
58
+ - In-VM agent: `~/Library/Application Support/Claude/claude-code-vm/<ver>/claude` (currently **2.1.317**, per `baselines/desktop-4.16336.0.json`, an **ELF aarch64** binary), spawned by the host **in cowork mode via the `CLAUDE_CODE_IS_COWORK=1` env var** — *not* a `--cowork` flag (that flag is plugin-scope and the staged agent rejects it; see the control-protocol note below). Each baseline records this ELF's `sha256` (`agentBinary.sha256`/`shaProvenance`), and the resolver integrity-checks the binary it's about to run against that hash by default (opt out `COWORK_HARNESS_VERIFY_AGENT_SHA=0`), so "the same pinned agent" is enforced, not just asserted. Old versions are re-downloadable + verifiable from the official release channel — from the channel that baseline's `agentBinary.releaseBaseUrl` names, which is **not always the stable path** (Desktop stages release candidates too); the exact command is in `docs/maintenance.md`.
59
59
  - Network: `vm_network_mode: "gvisor"`, egress through a userspace netstack with a **compiled domain allowlist**; off-list partners rejected (`partner rejected: entry not on compiled allowlist`).
60
60
  - Control plane: Electron renderer→main typed IPC on channels named `$eipc_message$_<per-build-UUID>_$_claude.web_$_<Class>_$_<method>`, every handler validating `event.senderFrame.url` against a trusted-origin allowlist. The session manager is `LocalAgentModeSessions` (80 methods: `start`, `sendMessage`, `setDraftSessionFolders`, `onToolPermissionRequest`, `respondToToolPermission`, `getTranscript`, `onEvent`, …), bridged to the renderer as `window.cowork`.
61
61
 
@@ -209,7 +209,7 @@ The policy that produces those `allow`/`deny` responses is the **Decider** seam
209
209
  > Cowork system-prompt fingerprint all unchanged from `1.20186.0`, with the staged VM ELF re-synced
210
210
  > 2.1.202 → 2.1.205 — and the live pass of that era was deliberately **not** restamped onto it.)
211
211
 
212
- > **Scope of that claim.** `2026-11-01 / desktop-4.4212.0` is the baseline carrying the latest live pass, run against agent **2.1.309** (the staged Linux ELF for `container`, and the staged native macOS build `5af865b9901c` for `hostloop`, with no override of either); the `protocol` tier runs the HOST `claude`, **2.1.316** for this pass. No baselines have shipped since. The `3.6556.2` sync reported two unknown deltas, both classified before the baseline was written: the host-loop permission wrapper's new Artifact file capture (pinned, not modelled) and a project-attribution spawn key set only when one claude.ai Project is selected (allowlisted). The `4.4212.0` sync (read from a scheduled task, the only local session the account runs) classified a second scheduled-run spawn key, a canUseTool link skipped under the host CLI launcher (inert), the frame-tools predicate admitting chat sessions, and the prompt template's move to a lazy chunk, and ported the `web_fetch` vendor-host refusal (Path B). The Cowork system prompt bundled with Desktop and the egress allowlist are unchanged in both; a signed-in session can receive its prompt from Anthropic's servers instead, which no sync sees. **The 4.13.0 pass, 2026-11-01, on `desktop-4.4212.0` (agent 2.1.309; host `claude` 2.1.316 for `protocol`), on the release-prep branch at `c3e2a1a0`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with eight warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error or warning; the debugging prompt read `debugging.md` and answered correctly; the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist; (iv) `cowork-harness run examples/scenarios/`, the first live regression on this baseline: seven of eight passed (`container` ×5, `hostloop-computer-links`, `protocol-smoke`). `subagent-manifest-probe` (`hostloop`) failed twice, both times after the parent's dispatch prompt spelled out a `/sessions/` path for the note: once the sub-agent did all of its work in the shell and never called `Read` or `Write`, and once it called `Write` on that path and the path gate refused it. The total was right both times. The probe passed on a re-run in the 4.11.0 pass and failed in the same way once there; its header says a green shows the sub-agent used the host path, not where it learned it. The live suite itself was not re-run, and no cassette was re-recorded (`verify-cassettes` reports none stale). Live spend about **$3.33**. **The 4.11.0 pass, 2026-10-23, on `desktop-3.1623.4` (agent 2.1.308; host `claude` 2.1.309 for `protocol`), on the release-prep branch at `a1e3c98d`, with the debugging-reference fix `c1f31ae4` for the steps marked below**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with eight warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error; the debugging prompt read `debugging.md` but again named a `trace --view gates` that does not exist, so `debugging.md` and `docs/debugging.md` now name the gate view (`questions`), and the re-run at `c1f31ae4` read `debugging.md` and named `--view questions` and `--view dispatches`; the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist; (iv) `cowork-harness run examples/scenarios/` at `c1f31ae4`, the first live regression on this baseline: six of eight passed on the first run (`container` ×5, `hostloop-computer-links`). `subagent-manifest-probe` (`hostloop`) failed `no_vm_path_file_op` once: the parent's dispatch prompt named the `/sessions/` path for the shell, the sub-agent tried a `Write` there, the path gate refused it, and the sub-agent then wrote the host path; the re-run passed every assertion. `protocol-smoke` errored with "Invalid API key" because the operator `.env` it loaded also held a stale `ANTHROPIC_API_KEY`, which `protocol` passes to the host `claude` ahead of the OAuth token (the agent reported `apiKeySource: ANTHROPIC_API_KEY`); with only the token loaded it passed. The live suite itself was not re-run, and no cassette was re-recorded (`verify-cassettes` reports none stale). Live spend about **$3.53**. **The 4.10.0 pass, 2026-10-16, on `desktop-2.42304.0` (agent 2.1.299; host `claude` 2.1.301 for `protocol`), on the release-prep branch at `737d4f35`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error; the debugging prompt read `debugging.md` and gave the right steps, but also named a `trace --view gates` that does not exist (`trace` refuses it, exit 2, and lists the views; neither the skill nor the docs name it); the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist; (iv) the frozen search corpus on `hostloop` (`claude-sonnet-5`): a seed run with two real searches built a two-record pool. A run of the same prompt from that pool was served both searches as exact matches (`served 2, missed 0, refused 0`), every live verdict equal to the ranking replayed offline from the built `dist/`, with the label on `verify-run`, `status`, `trace` and the grouped `stats` summary, whose `searchCorpus.sha256` equals the pool's. A run with a shifted prompt was served one nearby round and one miss; `verify-run` then graded both keys on each side of that run's real values: `search_corpus_max_miss_rate` failed at 0.49 and passed at 0.5 on a miss rate of 1/2, and `search_corpus_nearby_citations` failed at 8 and passed at 9 on nine cited URLs served only on nearby rounds. Both keys were refused (exit 2) by `run` without `--search-corpus` and by `verify-run` on a run served without one, and a judged assert on a frozen run carried `search_corpus_semantic_judge`. `record --search-corpus` stays refused. The three cassette re-records of 2026-10-16 on agent 2.1.299 (`example-pdf-skill` and `test/fixtures/tool-call-dispatch/dispatch-shell.cassette.json` on `container`, `hostloop-computer-links` on `hostloop`) carry no `memory_paths`. The full live suite was not re-run. Live spend about **$1.51**, plus **$0.24** on a first attempt stopped when Claude Desktop updated mid-gate. **The 4.9.1 pass, 2026-10-14, on `desktop-2.37337.1` (agent 2.1.295; host `claude` 2.1.299 for `protocol`), on the release-prep branch at `cc753ad5`** (a patch; the commits after it change docs, the CHANGELOG and the `critique` help line only): (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every run `delivered_clean` and every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error, the debugging prompt read `debugging.md` and answered correctly, and the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist. The full live suite was not re-run: the code changes in this release are to help text, `search-pool`, the companion skill's linter, the sync extractors, a hillclimb refactor that leaves every flag unchanged, and maintainer scripts, none of which a live scenario run reaches. Live spend about **$1.06**. **The 4.7.1 pass, 2026-10-10, on `desktop-2.31226.1` (agent 2.1.295; host `claude` 2.1.296 for `protocol`), on the release-prep branch at `3c13a4f8`** (a patch: a smoke on two tiers, not the live suite; the one code change after it widens which task types the usage-limit check reads, on a path neither run reached): (i) `container`, the sub-agent dispatch scenario (`test/fixtures/tool-call-dispatch/dispatch-shell.yaml`, `claude-sonnet-5`): a real sub-agent ran and completed, the stream carried `task_started` (`task_type: local_agent`), a `task_updated` whose patch was `completed` with no error, and a completed `task_notification`; the run passed, `delivered_clean`, with no `subagent_usage_limit` row; (ii) `protocol`, `examples/scenarios/protocol-smoke.yaml`: the first run had no credential in its environment, so it read the operator's real config dir and its verdict failed on `l0_host_config_contamination` alone (`result: success`, every assertion passed, no `subagent_usage_limit` row); the re-run with `COWORK_MANAGED_CONFIG=1` and the credential loaded read a sealed config dir and passed, `delivered_clean`, with no fail signal. A sub-agent that hits the usage limit was not triggered live (it needs a spent quota); that path is covered by tests built from the agent's frame shapes. Live spend about **$0.88**. **The 4.7.0 pass, 2026-10-10, on `desktop-2.31226.1` (agent 2.1.295; host `claude` 2.1.296 for `protocol`), on the release-prep branch at `2dba294f`** (the commits after it change docs only): (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 24 passed, one pre-registered skip; (iii) `artifact_json` `schema:` on `container`: a whole-document schema passed, and a schema of `type: string` on the integer at `count` failed alone, naming the path and "must be string"; a schema using `format` was refused at load (exit 2) before any spend, and `lint` reported it as an error; (iv) `fixture export --session-paths` on `container`: without the flag, export refused the file holding the run's outputs path (`run_path`) and wrote nothing; with it, the path became `__COWORK_HARNESS_VM_SESSION_ROOT__/mnt/outputs`, and a next step staged from that fixture read its own new session's path, with no token left; (v) `record <dir/> --case`: the `--dry-run` budget check priced the named scenario alone and passed where the whole directory was refused, and a real `--case` record wrote one cassette for the named scenario only; it carries `hookFailureBlocks: {"events": []}` and stamps v12, as before; (vi) `onFailure: "block"` on `container`: two runs identical but for that field on a PreToolUse hook that exits 1 on the prompted Bash call. Both recorded the same failed hook frame (exit 1, `outcome: error`). With the field, the agent blocked the call, the run's inventory listed `PreToolUse`, and both forms of `no_hook_event_blocked` failed as evidence-unavailable; without it, the call ran, the inventory was empty, and both passed. Recorded with `--allow-failing`, the first run stamped v16, and its replay reproduced the two evidence-unavailable results; (vii) the companion skill's router check, the authoring prompt on `container`: the first run's first `AskUserQuestion` held a single option and was rejected by the agent's own input validation, and the run ended on a question asked in prose (graded `stalled`), with `authoring.md` read but no scenario written, as in 4.6.1. The release checklist's prompt now names the skill folder and the report path; the re-run read `authoring.md` (and `scenario-schema.md` and an assertion-catalog part) and wrote a session file and a scenario that pass `lint` with no error or warning. Live spend about **$2.07**. **The 4.6.1 pass, 2026-10-09, on `desktop-2.31226.0` (agent 2.1.293; host `claude` 2.1.295 for `protocol`), on the release-prep branch at `e5ca4bd9`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip, and one red in the plugin hook-decision test: the model made a second, unprompted Bash call, the probe's hook denied both, and the harness counted both, so the probe's exact count of one Bash denial failed. The probe now denies only the calls its prompt asks for, and the re-run passed all twelve assertions with exactly one denial each from the Bash, Write and Stop hooks; (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iv) Ctrl-C on `hostloop` on macOS, SIGINT to the harness's whole process group while the agent ran `sleep 120` in the sandbox: exit 130 in 1.28 s, and in 1.33 s with twelve concurrent `ps -A -o tty=` loops running, where the same stop took about 8 s before 4.6.1; both times nothing in the process tree captured before the signal survived, the sandbox container was gone and `status.json` ended `error`; (v) a `lane: remote` scenario asserting `artifact_text`, and one asserting `semantic_matches` with `evidence_files`, are refused by `run` at load (exit 2) before any spend, naming the key and a remedy, and `lint` reports them as errors (exit 1); (vi) the companion skill's router check, the authoring prompt on `container`: one run's first `AskUserQuestion` was rejected by the agent's own input validation and the run ended by asking for the skill's path, with no reference read and no scenario written; the re-run read `authoring.md` (and `scenario-schema.md` and an assertion-catalog part) and wrote a scenario that passes `lint` with no error or warning. Live spend about **$2.06**. **The 4.6.0 pass, 2026-10-08, on `desktop-2.26454.2` (agent 2.1.293; host `claude` 2.1.294 for `protocol`), on the release-prep branch at `daf3479d`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 24 passed, one pre-registered skip, including the plugin hook-decision test on `container` (a PreToolUse hook denying by JSON, one exiting 2, and a Stop hook blocking once, graded by `hook_decision` and both forms of `hook_event_blocked`); (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iv) the companion skill's router check, three prompts on `container`, against the split assertion catalog: the authoring prompt read `authoring.md` and `scenario-schema.md` and wrote a session file and a scenario that pass `lint` with no error or warning, the debugging prompt read `debugging.md` and asked for the run's `result.json`, and the tool-timing prompt read `measurement.md` and answered with `trace … --view tool-durations`; (v) `gates_all_scripted` on `container`: a scripted multi-select gate passed, and the same scenario with its answer removed and `on_unanswered: first` failed on exactly that assertion, naming the gate as answered by `first`; (vi) `answer_channel: none` on `container`, with a probe skill that records a status file before asking: the run passed with a `parked_at_question` warning and no `stalled` fail, carried the headless label and `answerChannel: "none"`, graded the status file through an `artifact_json` glob, and its init frame listed no `AskUserQuestion`. Live spend about **$2.21**. The 4.4.1, 4.4.0 and 4.3.0 passes below ran on `desktop-2.19675.0` (agent 2.1.286). **The 4.5.0 pass, 2026-10-07, on `desktop-2.26454.0` (agent 2.1.289; host `claude` 2.1.292 for `protocol`), on the release-prep branch at `af26c3c7`** (the commits after it change docs and the companion skill only): (i) `vitest list --staticParse=false` listed all 23 live tests with no `SKIPPED` warning when the token was exported, and 6 with four warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip; (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iv) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `assertion-catalog.md`, searching `scenario-schema.md`) but wrote a scenario with an inline `session:` block, which fails `lint`, so it failed; `authoring.md` now says `session:` is a path, and one re-run read `authoring.md` and `scenario-schema.md` and asked for the skill's path instead of writing a scenario, so the corrected sentence is checked by `lint`, not by a live answer. The debugging prompt read no reference and answered with correct clarifying questions; the tool-timing prompt loaded the skill, read no reference and asked which run to time, so `measurement.md` routing was not exercised in this pass; (v) the forced-compaction recipe on `container`: a task turn, a resumed `/compact` turn whose `result.json` records a manual `compact_boundary` (40,266 → 2,896 tokens; the check exits 1 on the task turn), and a resumed turn that still named the command it had described before the compaction. The recipe first said to check this with `verify-run`, which refuses a run dir with more than one turn; it now reads the `/compact` turn's own `result.json`. Live spend about **$2.53**. **The 4.4.1 pass, 2026-10-05, on the release-prep commit `3e6fe443`** (a companion-skill-only release; same baseline and staged agent; host `claude` 2.1.289 for `protocol`): (i) `vitest list --staticParse=false` listed all 23 live tests with no `SKIPPED` warning when the token was exported, and 6 with four warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`, searching `assertion-catalog.md`) and wrote a scenario that passes `lint` with no error or warning, the debugging prompt read `debugging.md` and answered with correct clarifying questions, and the tool-timing prompt read `measurement.md` (and `debugging.md`, which documents `trace`'s views) and answered with `trace … --view tool-durations`. The live suite itself was not re-run: this release changes no code that runs. Live spend about **$1.08**. **The 4.4.0 pass, 2026-10-05, on the release-prep commit `e9c516df`** (same baseline and staged agent; host `claude` 2.1.289 for `protocol`): (i) `vitest list --staticParse=false` listed all 23 live tests with no `SKIPPED` warning when the token was exported, and 6 with four warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip; (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each; (iv) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` and wrote a scenario that passes `lint`, the tool-timing prompt read `measurement.md` and answered with `trace … --view tool-durations`, and the debugging prompt read no reference and answered with correct clarifying questions, so `debugging.md` routing was not exercised in this pass; (v) the re-grade scrub-set check: a live run on the default runs root created `scrubset.key` beside the runs directory (mode 0600) and recorded `scrubSet` (`v: 1`) with the configured scrub literal absent from `result.json`; a re-grade of that run with an edited rubric line and a smaller scrub set was refused as `rubric_unverifiable` (exit 2) naming only the edited line, with no judge call and the run directory byte-identical; and a re-grade of a run recorded before 4.4 with an edited rubric line was refused with the `legacy` reason and the one-line remedy, again with no judge call and the run directory byte-identical. Live spend about **$2.83** recorded. **The 4.3.0 pass, on the release commit `8a5e479f`:** (1) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip (the auto-memory prerequisites check, which runs only under `COWORK_LIVE_REQUIRE=1`); before it ran, `vitest list --staticParse=false` listed all 23 with no `SKIPPED` warning, and 6 with four warnings when no token was resolvable, so a silent skip would have shown; (2) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (3) the graders' pinned effort: every judged assert that called the judge in the live hillclimb acceptance runs (45, Opus 4.8: 25 `semantic_matches`, 20 `semantic_pairwise`; host `claude` 2.1.288 and 2.1.289) records `judgeTransport.effort: "high"`; the five pairwise asserts without a `judgeTransport` are the reference variant's own rows, which call no judge. **Earlier in the same release, on the release candidate's tree:** (4) a `microvm` smoke on a freshly booted VM (agent 2.1.286); (5) `boundary-check`, 6/6 constraints enforced; (6) an `eval` A/A at `--reps 4` on `container`, every row `no detectable change`, its report rebuilt byte-identical; (7) the companion skill's router check, three prompts on `container`: routing was correct on all three; one answer was wrong because a reference showed `artifact_text`'s `contains` without its list shape (fixed in the references, with a test that keeps every list-typed field shown as a list), and the same prompt's verdict went red on a false `host_path_leak`, because a reference spelled out the host roots that check looks for and the agent echoed it (fixed in the skill's files and in the check, which no longer counts a literal that comes from the plugin's or a local skill's own staged files); re-run after the fixes, the prompt routed correctly, wrote a scenario that passes `lint` and loads, and carried no `host_path_leak`; (8) the `hostloop` detached-agent kill check: the first pass found the workspace sidecar container, its `docker run` client and its network surviving SIGINT, which was fixed, and the re-run passed on a normal run, SIGINT to the harness process and SIGINT to the whole process group (exit 130, nothing left behind); (9) `vm delete`'s usage refusal; (10) `chat` under a real terminal, on `protocol`: Ctrl-C mid-turn exits 130 in about 2 s with nothing left running and no `result.json`; `/exit` followed by Ctrl-C exits 130 after writing the turn's `result.json`; Ctrl-C at the idle prompt ends the session normally (exit 0, `result.json` written); SIGHUP mid-turn exits 129 with nothing left running; and closing the terminal window mid-turn (run by hand) leaves no harness or agent process and no `result.json`. Total live spend about **$11.20**, including the re-runs. **Scope-out, so this is not read as more than it is.** (a) A live pass verifies observed behaviour, not the whole spawn contract by construction, and these checks are model-dependent, so a single red is evidence of model variance until a re-run says otherwise. (b) **CI does not live-validate anything.** Its "scenario suite (… live inference)" job is skipped as a whole without an `ANTHROPIC_API_KEY` repository secret, and none is set, so it shows as *skipped*. Before 2026-09 it instead ran with every real step skipped and reported *success*, so an older green check there means nothing ran. This note, not CI, is the live evidence. Separately and not a live matter: all four committed cassettes were **re-recorded** on 2026-10-03 against `2.19675.0` with auto-memory off, each on its original model: `example-pdf-skill` and `test/fixtures/tool-call-dispatch/dispatch-shell.cassette.json` (`container`, agent 2.1.286), `hostloop-computer-links` (`hostloop`, agent 2.1.286) and `example-multiselect-gate` (`protocol`, which runs the host CLI, here 2.1.287), about $0.75 in all. Their init frames carry no `memory_paths`. The recorder's own scrub (now applied to every recording) removed, across the four, the account's model menu, a subscription account's `rate_limit_info` (from `example-pdf-skill`, `dispatch-shell` and `hostloop-computer-links`) and the agent's sub-agent hand-back frame (from `dispatch-shell`). `verify-cassettes` exits 0 on all four, with one accepted `unscanned` entry (`example-pdf-skill`'s uploaded artifact body, too large to commit). Re-stamp this paragraph, naming the baseline, whenever a live pass is actually re-run.
212
+ > **Scope of that claim.** `2026-11-01 / desktop-4.4212.0` is the baseline carrying the latest live pass, run against agent 2.1.309 (the staged Linux ELF for `container`, and the staged native macOS build `5af865b9901c` for `hostloop`, with no override of either); the `protocol` tier runs the HOST `claude`, **2.1.316** for this pass. **1** baseline has shipped since (`4.16336.0`), **1** of which moved the agent ELF, most recently to **2.1.317**; nothing in it has had a live pass. Every unknown delta its `sync` reported was classified or modelled before the baseline was written. The `3.6556.2` sync reported two unknown deltas, both classified before the baseline was written: the host-loop permission wrapper's new Artifact file capture (pinned, not modelled) and a project-attribution spawn key set only when one claude.ai Project is selected (allowlisted). The `4.4212.0` sync (read from a scheduled task, the only local session the account runs) classified a second scheduled-run spawn key, a canUseTool link skipped under the host CLI launcher (inert), the frame-tools predicate admitting chat sessions, and the prompt template's move to a lazy chunk, and ported the `web_fetch` vendor-host refusal (Path B). The Cowork system prompt bundled with Desktop and the egress allowlist are unchanged in both; a signed-in session can receive its prompt from Anthropic's servers instead, which no sync sees. **The 4.13.0 pass, 2026-11-01, on `desktop-4.4212.0` (agent 2.1.309; host `claude` 2.1.316 for `protocol`), on the release-prep branch at `c3e2a1a0`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with eight warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error or warning; the debugging prompt read `debugging.md` and answered correctly; the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist; (iv) `cowork-harness run examples/scenarios/`, the first live regression on this baseline: seven of eight passed (`container` ×5, `hostloop-computer-links`, `protocol-smoke`). `subagent-manifest-probe` (`hostloop`) failed twice, both times after the parent's dispatch prompt spelled out a `/sessions/` path for the note: once the sub-agent did all of its work in the shell and never called `Read` or `Write`, and once it called `Write` on that path and the path gate refused it. The total was right both times. The probe passed on a re-run in the 4.11.0 pass and failed in the same way once there; its header says a green shows the sub-agent used the host path, not where it learned it. The live suite itself was not re-run, and no cassette was re-recorded (`verify-cassettes` reports none stale). Live spend about **$3.33**. **The 4.11.0 pass, 2026-10-23, on `desktop-3.1623.4` (agent 2.1.308; host `claude` 2.1.309 for `protocol`), on the release-prep branch at `a1e3c98d`, with the debugging-reference fix `c1f31ae4` for the steps marked below**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with eight warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error; the debugging prompt read `debugging.md` but again named a `trace --view gates` that does not exist, so `debugging.md` and `docs/debugging.md` now name the gate view (`questions`), and the re-run at `c1f31ae4` read `debugging.md` and named `--view questions` and `--view dispatches`; the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist; (iv) `cowork-harness run examples/scenarios/` at `c1f31ae4`, the first live regression on this baseline: six of eight passed on the first run (`container` ×5, `hostloop-computer-links`). `subagent-manifest-probe` (`hostloop`) failed `no_vm_path_file_op` once: the parent's dispatch prompt named the `/sessions/` path for the shell, the sub-agent tried a `Write` there, the path gate refused it, and the sub-agent then wrote the host path; the re-run passed every assertion. `protocol-smoke` errored with "Invalid API key" because the operator `.env` it loaded also held a stale `ANTHROPIC_API_KEY`, which `protocol` passes to the host `claude` ahead of the OAuth token (the agent reported `apiKeySource: ANTHROPIC_API_KEY`); with only the token loaded it passed. The live suite itself was not re-run, and no cassette was re-recorded (`verify-cassettes` reports none stale). Live spend about **$3.53**. **The 4.10.0 pass, 2026-10-16, on `desktop-2.42304.0` (agent 2.1.299; host `claude` 2.1.301 for `protocol`), on the release-prep branch at `737d4f35`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error; the debugging prompt read `debugging.md` and gave the right steps, but also named a `trace --view gates` that does not exist (`trace` refuses it, exit 2, and lists the views; neither the skill nor the docs name it); the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist; (iv) the frozen search corpus on `hostloop` (`claude-sonnet-5`): a seed run with two real searches built a two-record pool. A run of the same prompt from that pool was served both searches as exact matches (`served 2, missed 0, refused 0`), every live verdict equal to the ranking replayed offline from the built `dist/`, with the label on `verify-run`, `status`, `trace` and the grouped `stats` summary, whose `searchCorpus.sha256` equals the pool's. A run with a shifted prompt was served one nearby round and one miss; `verify-run` then graded both keys on each side of that run's real values: `search_corpus_max_miss_rate` failed at 0.49 and passed at 0.5 on a miss rate of 1/2, and `search_corpus_nearby_citations` failed at 8 and passed at 9 on nine cited URLs served only on nearby rounds. Both keys were refused (exit 2) by `run` without `--search-corpus` and by `verify-run` on a run served without one, and a judged assert on a frozen run carried `search_corpus_semantic_judge`. `record --search-corpus` stays refused. The three cassette re-records of 2026-10-16 on agent 2.1.299 (`example-pdf-skill` and `test/fixtures/tool-call-dispatch/dispatch-shell.cassette.json` on `container`, `hostloop-computer-links` on `hostloop`) carry no `memory_paths`. The full live suite was not re-run. Live spend about **$1.51**, plus **$0.24** on a first attempt stopped when Claude Desktop updated mid-gate. **The 4.9.1 pass, 2026-10-14, on `desktop-2.37337.1` (agent 2.1.295; host `claude` 2.1.299 for `protocol`), on the release-prep branch at `cc753ad5`** (a patch; the commits after it change docs, the CHANGELOG and the `critique` help line only): (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every run `delivered_clean` and every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`) and wrote a scenario that passes `lint` with no error, the debugging prompt read `debugging.md` and answered correctly, and the tool-timing prompt read `measurement.md` (and `debugging.md`) and answered correctly, with `trace` flags that exist. The full live suite was not re-run: the code changes in this release are to help text, `search-pool`, the companion skill's linter, the sync extractors, a hillclimb refactor that leaves every flag unchanged, and maintainer scripts, none of which a live scenario run reaches. Live spend about **$1.06**. **The 4.7.1 pass, 2026-10-10, on `desktop-2.31226.1` (agent 2.1.295; host `claude` 2.1.296 for `protocol`), on the release-prep branch at `3c13a4f8`** (a patch: a smoke on two tiers, not the live suite; the one code change after it widens which task types the usage-limit check reads, on a path neither run reached): (i) `container`, the sub-agent dispatch scenario (`test/fixtures/tool-call-dispatch/dispatch-shell.yaml`, `claude-sonnet-5`): a real sub-agent ran and completed, the stream carried `task_started` (`task_type: local_agent`), a `task_updated` whose patch was `completed` with no error, and a completed `task_notification`; the run passed, `delivered_clean`, with no `subagent_usage_limit` row; (ii) `protocol`, `examples/scenarios/protocol-smoke.yaml`: the first run had no credential in its environment, so it read the operator's real config dir and its verdict failed on `l0_host_config_contamination` alone (`result: success`, every assertion passed, no `subagent_usage_limit` row); the re-run with `COWORK_MANAGED_CONFIG=1` and the credential loaded read a sealed config dir and passed, `delivered_clean`, with no fail signal. A sub-agent that hits the usage limit was not triggered live (it needs a spent quota); that path is covered by tests built from the agent's frame shapes. Live spend about **$0.88**. **The 4.7.0 pass, 2026-10-10, on `desktop-2.31226.1` (agent 2.1.295; host `claude` 2.1.296 for `protocol`), on the release-prep branch at `2dba294f`** (the commits after it change docs only): (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 24 passed, one pre-registered skip; (iii) `artifact_json` `schema:` on `container`: a whole-document schema passed, and a schema of `type: string` on the integer at `count` failed alone, naming the path and "must be string"; a schema using `format` was refused at load (exit 2) before any spend, and `lint` reported it as an error; (iv) `fixture export --session-paths` on `container`: without the flag, export refused the file holding the run's outputs path (`run_path`) and wrote nothing; with it, the path became `__COWORK_HARNESS_VM_SESSION_ROOT__/mnt/outputs`, and a next step staged from that fixture read its own new session's path, with no token left; (v) `record <dir/> --case`: the `--dry-run` budget check priced the named scenario alone and passed where the whole directory was refused, and a real `--case` record wrote one cassette for the named scenario only; it carries `hookFailureBlocks: {"events": []}` and stamps v12, as before; (vi) `onFailure: "block"` on `container`: two runs identical but for that field on a PreToolUse hook that exits 1 on the prompted Bash call. Both recorded the same failed hook frame (exit 1, `outcome: error`). With the field, the agent blocked the call, the run's inventory listed `PreToolUse`, and both forms of `no_hook_event_blocked` failed as evidence-unavailable; without it, the call ran, the inventory was empty, and both passed. Recorded with `--allow-failing`, the first run stamped v16, and its replay reproduced the two evidence-unavailable results; (vii) the companion skill's router check, the authoring prompt on `container`: the first run's first `AskUserQuestion` held a single option and was rejected by the agent's own input validation, and the run ended on a question asked in prose (graded `stalled`), with `authoring.md` read but no scenario written, as in 4.6.1. The release checklist's prompt now names the skill folder and the report path; the re-run read `authoring.md` (and `scenario-schema.md` and an assertion-catalog part) and wrote a session file and a scenario that pass `lint` with no error or warning. Live spend about **$2.07**. **The 4.6.1 pass, 2026-10-09, on `desktop-2.31226.0` (agent 2.1.293; host `claude` 2.1.295 for `protocol`), on the release-prep branch at `e5ca4bd9`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip, and one red in the plugin hook-decision test: the model made a second, unprompted Bash call, the probe's hook denied both, and the harness counted both, so the probe's exact count of one Bash denial failed. The probe now denies only the calls its prompt asks for, and the re-run passed all twelve assertions with exactly one denial each from the Bash, Write and Stop hooks; (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iv) Ctrl-C on `hostloop` on macOS, SIGINT to the harness's whole process group while the agent ran `sleep 120` in the sandbox: exit 130 in 1.28 s, and in 1.33 s with twelve concurrent `ps -A -o tty=` loops running, where the same stop took about 8 s before 4.6.1; both times nothing in the process tree captured before the signal survived, the sandbox container was gone and `status.json` ended `error`; (v) a `lane: remote` scenario asserting `artifact_text`, and one asserting `semantic_matches` with `evidence_files`, are refused by `run` at load (exit 2) before any spend, naming the key and a remedy, and `lint` reports them as errors (exit 1); (vi) the companion skill's router check, the authoring prompt on `container`: one run's first `AskUserQuestion` was rejected by the agent's own input validation and the run ended by asking for the skill's path, with no reference read and no scenario written; the re-run read `authoring.md` (and `scenario-schema.md` and an assertion-catalog part) and wrote a scenario that passes `lint` with no error or warning. Live spend about **$2.06**. **The 4.6.0 pass, 2026-10-08, on `desktop-2.26454.2` (agent 2.1.293; host `claude` 2.1.294 for `protocol`), on the release-prep branch at `daf3479d`**: (i) `vitest list --staticParse=false` listed all 24 live tests with no `SKIPPED` warning when the token was exported, and 6 with five warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 24 passed, one pre-registered skip, including the plugin hook-decision test on `container` (a PreToolUse hook denying by JSON, one exiting 2, and a Stop hook blocking once, graded by `hook_decision` and both forms of `hook_event_blocked`); (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iv) the companion skill's router check, three prompts on `container`, against the split assertion catalog: the authoring prompt read `authoring.md` and `scenario-schema.md` and wrote a session file and a scenario that pass `lint` with no error or warning, the debugging prompt read `debugging.md` and asked for the run's `result.json`, and the tool-timing prompt read `measurement.md` and answered with `trace … --view tool-durations`; (v) `gates_all_scripted` on `container`: a scripted multi-select gate passed, and the same scenario with its answer removed and `on_unanswered: first` failed on exactly that assertion, naming the gate as answered by `first`; (vi) `answer_channel: none` on `container`, with a probe skill that records a status file before asking: the run passed with a `parked_at_question` warning and no `stalled` fail, carried the headless label and `answerChannel: "none"`, graded the status file through an `artifact_json` glob, and its init frame listed no `AskUserQuestion`. Live spend about **$2.21**. The 4.4.1, 4.4.0 and 4.3.0 passes below ran on `desktop-2.19675.0` (agent 2.1.286). **The 4.5.0 pass, 2026-10-07, on `desktop-2.26454.0` (agent 2.1.289; host `claude` 2.1.292 for `protocol`), on the release-prep branch at `af26c3c7`** (the commits after it change docs and the companion skill only): (i) `vitest list --staticParse=false` listed all 23 live tests with no `SKIPPED` warning when the token was exported, and 6 with four warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip; (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iv) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `assertion-catalog.md`, searching `scenario-schema.md`) but wrote a scenario with an inline `session:` block, which fails `lint`, so it failed; `authoring.md` now says `session:` is a path, and one re-run read `authoring.md` and `scenario-schema.md` and asked for the skill's path instead of writing a scenario, so the corrected sentence is checked by `lint`, not by a live answer. The debugging prompt read no reference and answered with correct clarifying questions; the tool-timing prompt loaded the skill, read no reference and asked which run to time, so `measurement.md` routing was not exercised in this pass; (v) the forced-compaction recipe on `container`: a task turn, a resumed `/compact` turn whose `result.json` records a manual `compact_boundary` (40,266 → 2,896 tokens; the check exits 1 on the task turn), and a resumed turn that still named the command it had described before the compaction. The recipe first said to check this with `verify-run`, which refuses a run dir with more than one turn; it now reads the `/compact` turn's own `result.json`. Live spend about **$2.53**. **The 4.4.1 pass, 2026-10-05, on the release-prep commit `3e6fe443`** (a companion-skill-only release; same baseline and staged agent; host `claude` 2.1.289 for `protocol`): (i) `vitest list --staticParse=false` listed all 23 live tests with no `SKIPPED` warning when the token was exported, and 6 with four warnings when none was resolvable; (ii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (iii) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` (and `scenario-schema.md`, searching `assertion-catalog.md`) and wrote a scenario that passes `lint` with no error or warning, the debugging prompt read `debugging.md` and answered with correct clarifying questions, and the tool-timing prompt read `measurement.md` (and `debugging.md`, which documents `trace`'s views) and answered with `trace … --view tool-durations`. The live suite itself was not re-run: this release changes no code that runs. Live spend about **$1.08**. **The 4.4.0 pass, 2026-10-05, on the release-prep commit `e9c516df`** (same baseline and staged agent; host `claude` 2.1.289 for `protocol`): (i) `vitest list --staticParse=false` listed all 23 live tests with no `SKIPPED` warning when the token was exported, and 6 with four warnings when none was resolvable; (ii) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip; (iii) the auto-memory check on all three tiers, `Tests 2 passed (2)` each; (iv) the companion skill's router check, three prompts on `container`: the authoring prompt read `authoring.md` and wrote a scenario that passes `lint`, the tool-timing prompt read `measurement.md` and answered with `trace … --view tool-durations`, and the debugging prompt read no reference and answered with correct clarifying questions, so `debugging.md` routing was not exercised in this pass; (v) the re-grade scrub-set check: a live run on the default runs root created `scrubset.key` beside the runs directory (mode 0600) and recorded `scrubSet` (`v: 1`) with the configured scrub literal absent from `result.json`; a re-grade of that run with an edited rubric line and a smaller scrub set was refused as `rubric_unverifiable` (exit 2) naming only the edited line, with no judge call and the run directory byte-identical; and a re-grade of a run recorded before 4.4 with an edited rubric line was refused with the `legacy` reason and the one-line remedy, again with no judge call and the run directory byte-identical. Live spend about **$2.83** recorded. **The 4.3.0 pass, on the release commit `8a5e479f`:** (1) the live suite on `protocol`, `container` and `hostloop`: 23 passed, one pre-registered skip (the auto-memory prerequisites check, which runs only under `COWORK_LIVE_REQUIRE=1`); before it ran, `vitest list --staticParse=false` listed all 23 with no `SKIPPED` warning, and 6 with four warnings when no token was resolvable, so a silent skip would have shown; (2) the auto-memory check on all three tiers, `Tests 2 passed (2)` each, every init frame without `memory_paths`; (3) the graders' pinned effort: every judged assert that called the judge in the live hillclimb acceptance runs (45, Opus 4.8: 25 `semantic_matches`, 20 `semantic_pairwise`; host `claude` 2.1.288 and 2.1.289) records `judgeTransport.effort: "high"`; the five pairwise asserts without a `judgeTransport` are the reference variant's own rows, which call no judge. **Earlier in the same release, on the release candidate's tree:** (4) a `microvm` smoke on a freshly booted VM (agent 2.1.286); (5) `boundary-check`, 6/6 constraints enforced; (6) an `eval` A/A at `--reps 4` on `container`, every row `no detectable change`, its report rebuilt byte-identical; (7) the companion skill's router check, three prompts on `container`: routing was correct on all three; one answer was wrong because a reference showed `artifact_text`'s `contains` without its list shape (fixed in the references, with a test that keeps every list-typed field shown as a list), and the same prompt's verdict went red on a false `host_path_leak`, because a reference spelled out the host roots that check looks for and the agent echoed it (fixed in the skill's files and in the check, which no longer counts a literal that comes from the plugin's or a local skill's own staged files); re-run after the fixes, the prompt routed correctly, wrote a scenario that passes `lint` and loads, and carried no `host_path_leak`; (8) the `hostloop` detached-agent kill check: the first pass found the workspace sidecar container, its `docker run` client and its network surviving SIGINT, which was fixed, and the re-run passed on a normal run, SIGINT to the harness process and SIGINT to the whole process group (exit 130, nothing left behind); (9) `vm delete`'s usage refusal; (10) `chat` under a real terminal, on `protocol`: Ctrl-C mid-turn exits 130 in about 2 s with nothing left running and no `result.json`; `/exit` followed by Ctrl-C exits 130 after writing the turn's `result.json`; Ctrl-C at the idle prompt ends the session normally (exit 0, `result.json` written); SIGHUP mid-turn exits 129 with nothing left running; and closing the terminal window mid-turn (run by hand) leaves no harness or agent process and no `result.json`. Total live spend about **$11.20**, including the re-runs. **Scope-out, so this is not read as more than it is.** (a) A live pass verifies observed behaviour, not the whole spawn contract by construction, and these checks are model-dependent, so a single red is evidence of model variance until a re-run says otherwise. (b) **CI does not live-validate anything.** Its "scenario suite (… live inference)" job is skipped as a whole without an `ANTHROPIC_API_KEY` repository secret, and none is set, so it shows as *skipped*. Before 2026-09 it instead ran with every real step skipped and reported *success*, so an older green check there means nothing ran. This note, not CI, is the live evidence. Separately and not a live matter: all four committed cassettes were **re-recorded** on 2026-10-03 against `2.19675.0` with auto-memory off, each on its original model: `example-pdf-skill` and `test/fixtures/tool-call-dispatch/dispatch-shell.cassette.json` (`container`, agent 2.1.286), `hostloop-computer-links` (`hostloop`, agent 2.1.286) and `example-multiselect-gate` (`protocol`, which runs the host CLI, here 2.1.287), about $0.75 in all. Their init frames carry no `memory_paths`. The recorder's own scrub (now applied to every recording) removed, across the four, the account's model menu, a subscription account's `rate_limit_info` (from `example-pdf-skill`, `dispatch-shell` and `hostloop-computer-links`) and the agent's sub-agent hand-back frame (from `dispatch-shell`). `verify-cassettes` exits 0 on all four, with one accepted `unscanned` entry (`example-pdf-skill`'s uploaded artifact body, too large to commit). Re-stamp this paragraph, naming the baseline, whenever a live pass is actually re-run.
213
213
 
214
214
  > The staged agent ELF is unchanged (2.1.181) across the 1.14271.0→1.15200.0 asar bump, and 2.1.187 across the 1.15200.0→1.15962.0 bump. The live scenario suite (`protocol` + `container` tiers) was re-run against the 1.15200.0 baseline; the 1.15962.0 bump was verified via asar analysis (content byte-identical: host-loop generator, system prompt, identity string, gates, and egress domains all unchanged) plus a full local test suite pass. The 1.15962.1→1.17377.1 bump moved the staged agent to **2.1.197** and added `api.claude.ai` to the egress allowlist; re-verified via `sync` (no unknown deltas) plus a manual asar spot-check of the reconstructed prompt content (substantively unchanged — see the Parity entry in CHANGELOG.md) and a full live scenario-suite pass (`protocol` + `container` tiers).
215
215
 
package/README.md CHANGED
@@ -36,7 +36,7 @@ npm ci && npm run build
36
36
  node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
37
37
  ```
38
38
 
39
- (Installing globally — `npm install -g "cowork-harness@^4.13.1"` — gives you the `cowork-harness` CLI for your own
39
+ (Installing globally — `npm install -g "cowork-harness@^4.14.0"` — gives you the `cowork-harness` CLI for your own
40
40
  scenarios and cassettes; the bundled example above also replays from a global install — see the `$(npm root -g)` path below.)
41
41
 
42
42
  Full setup → [Quick start](./docs/cli.md#quick-start).
@@ -49,8 +49,8 @@ Three ways to use this project. Each row is the whole hook — follow the link f
49
49
 
50
50
  | I want to… | Start here | Needs |
51
51
  |---|---|---|
52
- | **Run scenarios myself** from a terminal | **[docs/cli.md](./docs/cli.md)**<br><br>`npm i -g "cowork-harness@^4.13.1"`<br>`cowork-harness replay examples/replays/example-pdf-skill.cassette.json` | Node ≥ 22. The replay demo above is token-free and needs nothing else; live tiers above `protocol` need Docker + a staged agent binary |
53
- | **Have Claude Code drive it** for me | **[docs/companion-skill.md](./docs/companion-skill.md)**<br><br>`/plugin marketplace add yaniv-golan/cowork-harness`<br>`/plugin install cowork-harness@cowork-harness` | Claude Code. The skill self-bootstraps the CLI via `npx "cowork-harness@^4.13.1"` |
52
+ | **Run scenarios myself** from a terminal | **[docs/cli.md](./docs/cli.md)**<br><br>`npm i -g "cowork-harness@^4.14.0"`<br>`cowork-harness replay examples/replays/example-pdf-skill.cassette.json` | Node ≥ 22. The replay demo above is token-free and needs nothing else; live tiers above `protocol` need Docker + a staged agent binary |
53
+ | **Have Claude Code drive it** for me | **[docs/companion-skill.md](./docs/companion-skill.md)**<br><br>`/plugin marketplace add yaniv-golan/cowork-harness`<br>`/plugin install cowork-harness@cowork-harness` | Claude Code. The skill self-bootstraps the CLI via `npx "cowork-harness@^4.14.0"` |
54
54
  | **Gate my skill in CI** | **[docs/ci.md](./docs/ci.md)**<br><br>`- uses: yaniv-golan/cowork-harness@v4`<br>` with: { command: replay, path: cassettes/ }` | Nothing for the token-free gate; the live lane needs a self-hosted runner with Docker + an agent binary |
55
55
 
56
56
  Improving a skill round by round with Claude Code's `/claude-api hillclimb` loop? The harness is its runner, with
@@ -249,7 +249,11 @@ L2 microvm parity Optional. Agent inside a real Linux microVM (Lima/Apple-VZ
249
249
  server — an in-process MCP server the Agent SDK talks to directly (tool name
250
250
  mcp__workspace__bash), whose bash calls route into a Docker VM sidecar; web_fetch
251
251
  routes via host curl. A PreToolUse path-containment hook is the security boundary
252
- for real filesystem access at this tier — see docs/boundary.md.
252
+ for real filesystem access at this tier — see docs/boundary.md. From Desktop
253
+ 4.16336.0 (with an agent at or above the baseline's fence floor), the agent also
254
+ refuses file reads and writes outside the task's folders itself, as a second layer
255
+ on top of the hook. That is configured and acceptance-verified, not measured in a
256
+ live run — see docs/fidelity-gaps.md.
253
257
 
254
258
  Both container and hostloop also declare the skills/plugins skill/plugin-discovery SDK-MCP servers
255
259
  (mcp__skills__list_skills/suggest_skills, mcp__plugins__list_plugins/search_plugins/
@@ -457,6 +461,6 @@ inputs/outputs. Human-readable terminal text is explicitly **not** part of the c
457
461
  ## Status
458
462
 
459
463
  The latest shipped baseline — what `baseline: latest` resolves to, the highest version under `baselines/` — is
460
- **`desktop-4.4212.0`**. Release-by-release verification notes (what was re-verified against
464
+ **`desktop-4.16336.0`**. Release-by-release verification notes (what was re-verified against
461
465
  which live agent/asar) are recorded in [CHANGELOG.md](./CHANGELOG.md); the feature catalogue
462
466
  this section would otherwise duplicate lives in the sections above.