cowork-harness 1.7.0 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (49) hide show
  1. package/.claude/skills/cowork-harness/SKILL.md +39 -11
  2. package/.claude/skills/cowork-harness/references/ci-recipe.md +5 -5
  3. package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +17 -1
  4. package/.claude/skills/cowork-harness/references/scenario-schema.md +2 -2
  5. package/.claude/skills/cowork-harness/references/task-recipes.md +2 -2
  6. package/CHANGELOG.md +170 -0
  7. package/CONTRIBUTING.md +7 -6
  8. package/DESIGN.md +1 -1
  9. package/README.md +17 -11
  10. package/RELEASING.md +7 -5
  11. package/dist/cli.js +24 -5
  12. package/dist/critique/command.js +394 -40
  13. package/dist/critique/evaluator.js +111 -43
  14. package/dist/critique/evidence.js +15 -2
  15. package/dist/critique/limitations.js +43 -27
  16. package/dist/critique/package-evidence.js +120 -12
  17. package/dist/decide/llm-transport.js +3 -1
  18. package/dist/hostloop/canusetool-gate.js +4 -4
  19. package/dist/hostloop/safety.js +6 -5
  20. package/dist/run/analyze-artifact-runtime.js +3 -4
  21. package/dist/run/analyze-artifact.js +1 -1
  22. package/dist/run/artifacts.js +12 -0
  23. package/dist/run/chat-result.js +8 -5
  24. package/dist/run/doctor.js +65 -0
  25. package/dist/run/execute.js +48 -16
  26. package/dist/run/skill-flag-surface.js +13 -2
  27. package/dist/run/subagent-reasoning.js +58 -3
  28. package/dist/run/trace-view.js +38 -0
  29. package/dist/runtime/hostloop-prompt.js +23 -4
  30. package/dist/runtime/hostloop.js +9 -3
  31. package/dist/sync/cowork-sync.js +5 -5
  32. package/dist/types.js +1 -1
  33. package/docker/Dockerfile.agent +9 -0
  34. package/docs/README.md +8 -3
  35. package/docs/boundary.md +1 -1
  36. package/docs/chat.md +1 -1
  37. package/docs/critique.md +134 -23
  38. package/docs/debugging.md +36 -27
  39. package/docs/gotchas.md +2 -0
  40. package/docs/subagents.md +1 -1
  41. package/examples/README.md +1 -1
  42. package/examples/replays/README.md +1 -1
  43. package/llms.txt +1 -1
  44. package/package.json +1 -1
  45. package/python/README.md +7 -2
  46. package/schema/run-result.json +27 -0
  47. package/scripts/bump-version.ts +5 -3
  48. package/scripts/check-versions.ts +45 -11
  49. package/scripts/release-preflight.ts +0 -2
@@ -3,8 +3,8 @@ name: cowork-harness
3
3
  description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
4
4
  metadata:
5
5
  author: cowork-harness
6
- version: 1.7.0
7
- tracks-harness: cowork-harness 1.7.0 (baseline desktop-1.24012.1)
6
+ version: 1.8.0
7
+ tracks-harness: cowork-harness 1.8.0 (baseline desktop-1.24012.1)
8
8
  ---
9
9
 
10
10
  # cowork-harness
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
22
22
  allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
23
23
  the highest-value part. Read it.
24
24
 
25
- > **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.7.0` (baseline
25
+ > **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.8.0` (baseline
26
26
  > `desktop-1.24012.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
27
27
  > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
28
28
 
@@ -39,9 +39,30 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
39
39
 
40
40
  - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
41
41
  - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
42
- - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.7.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.7.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=1.7.0"`. **Pin `@>=1.7.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
43
-
44
- What the ≥ 1.7.0 floor gates, by release:
42
+ - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.8.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.8.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=1.8.0"`. **Pin `@>=1.8.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
43
+
44
+ What the ≥ 1.8.0 floor gates, by release:
45
+
46
+ - **1.8.0 (critique hardening + hostloop unpin):** `critique --skill <name>` (multi-skill-plugin
47
+ grading — a plugin root with no `--skill` is refused pre-spend; the evidence package gains the
48
+ invoked skill's `agents/<name>.md` + bounded `references/*.md` CONTENT), run-dir artifacts
49
+ (`critique-report.json` always, `critique-evidence-package.txt` when the evaluator ran,
50
+ `critique-salvage.json` on exit 2) + `--out <path>`, per-critique `costUsd` across all four
51
+ workloads, `findingFingerprint` per item (cross-INPUT clustering; skillHash stays the cross-FIX
52
+ key), gate-answer `--answer` echo in the report, per-item-tolerant evaluator parse
53
+ (`droppedEvaluatorItems`), `skillMdTruncated` (a readable-but-oversized SKILL.md is flagged
54
+ "graded a cut copy" — distinct from missing/unreadable), `subagents[].webSearches` +
55
+ `trace --view subagent-research` (live/record lane), and the hostloop uploads-bullet fix (Read of
56
+ an uploaded file at the advertised path works — no copy-into-outputs workaround needed). Also:
57
+ **`critique --fidelity hostloop`** (the container-tier pin is lifted — resume-continuity proven
58
+ live on the native binary; `microvm`/`protocol`/`cowork` stay refused, each with a tagged reason),
59
+ **`skill --allow-host-writes`** (hostloop writable-folder consent; `critique` forwards it to both
60
+ turns), `verdictProvenance` stamped on every critique report (advisory self-run, not an
61
+ attestation), evidence caps 64KB SKILL.md / 32KB transcript / 144KB package (a flagship-sized
62
+ SKILL.md grades untruncated; ~2–2.5× evaluator cost on large skills), pinned sessions stamp their
63
+ fidelity and a cross-tier `--resume` fails loud pre-spawn, and `doctor`'s advisory
64
+ `image-freshness` check (pulled agent image behind the published GHCR one → warn + re-pull
65
+ remedy).
45
66
 
46
67
  - **1.7.0 (per-turn run-directory layout, single shape):** every run dir — `run`/`skill`/`chat`,
47
68
  single-turn or multi-turn (any `--session-id` + `--resume`, and **every `critique`**: task turn +
@@ -61,8 +82,9 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
61
82
  - **1.7.0 (limitation provenance):** every `critique` limitation in `critique --help` and
62
83
  docs/critique.md is tagged with WHY it exists — `[structural]` (permanent), `[unverified]` (unproven,
63
84
  **not** known-impossible), `[deliberate]`, `[not-built]`. **Read the tag before telling a user to
64
- design around a limitation.** In particular `critique`'s container-tier pin is `[unverified]`, not
65
- permanent: do not advise building a second hostloop test lane on the assumption it can never lift.
85
+ design around a limitation.** Worked example of why the tag matters: `critique`'s container-tier pin
86
+ was tagged `[unverified]`, not permanent — and 1.8.0 lifted it (hostloop allowed, see above) exactly
87
+ as the tag predicted a proof could.
66
88
 
67
89
  - **1.6.0 (`critique`'s skill-flag parity):** `critique` accepts most `skill` flags under the same names,
68
90
  which is what makes a document-analysis skill (cap table, deck, financial model, transcript)
@@ -679,9 +701,15 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
679
701
  7. **`protocol` is rejected (not silently passed) if the scenario asserts egress** — boundary
680
702
  assertions need a sandboxed tier (`container`+). Good: this one fails loud by design.
681
703
 
682
- 8. **Read-only mounts are enforced; delete-deny is not.** `mode:r` mounts get a real `:ro` bind
683
- (a write fails in-guest). But `rw` vs `rwd` (write-but-no-delete on `outputs/` / connected folders) is
684
- *not* mount-enforced — `rm` succeeds and is only caught post-hoc by `no_delete_in_outputs`.
704
+ 8. **Read-only mounts are enforced; delete-deny is a HARNESS gap — production DOES enforce it.**
705
+ `mode:r` mounts get a real `:ro` bind (a write fails in-guest). But `rw` vs `rwd`
706
+ (write-but-no-delete on `outputs/` / connected folders) is *not* mount-enforced **in the harness** —
707
+ `rm` succeeds and is only caught post-hoc by `no_delete_in_outputs`. **Real Cowork enforces it live:**
708
+ `rm` under `outputs/` fails `Operation not permitted`, and a skill must request approval via
709
+ `allow_cowork_file_delete` to delete. Two consequences: a skill should not stage disposable scratch
710
+ under `outputs/` (in production, cleanup there costs an approval prompt), and a skill's
711
+ "catch-EPERM-then-request-approval" branch cannot be exercised at any harness tier (the `rm` just
712
+ succeeds here). Do not read this gotcha as "delete-deny may not be real in production" — it is real.
685
713
 
686
714
  9. **Keep `.env` out of any mounted folder** — it is copied into the sandbox and the token could
687
715
  leak. Put it at a working-dir or install root (token resolution: env > `--dotenv` > `./.env` >
@@ -1,6 +1,6 @@
1
1
  # CI recipe — replay vs live lanes
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 1.7.0` (baseline `desktop-1.20186.1`).
3
+ Self-contained reference. Tracks `cowork-harness 1.8.0` (baseline `desktop-1.24012.1`).
4
4
 
5
5
  **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
6
6
  job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -13,7 +13,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
13
13
  ```
14
14
 
15
15
  The Action's `version` input defaults to `latest` — intentional so a copy-pasted recipe tracks the current
16
- release; pin an exact version (e.g. `version: "1.7.0"`) for reproducible CI.
16
+ release; pin an exact version (e.g. `version: "1.8.0"`) for reproducible CI.
17
17
 
18
18
  Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
19
19
  cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -57,7 +57,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
57
57
  GitHub-hosted runners, no token/Docker/agent:
58
58
 
59
59
  ```yaml
60
- - run: npm i -g "cowork-harness@>=1.7.0"
60
+ - run: npm i -g "cowork-harness@>=1.8.0"
61
61
  - run: cowork-harness lint scenarios/*.yaml # no silent false-greens
62
62
  - run: cowork-harness verify-cassettes cassettes/ # privacy + staleness
63
63
  - run: cowork-harness replay cassettes/ # token-free content/structure
@@ -197,7 +197,7 @@ jobs:
197
197
  with: { node-version: '20' }
198
198
  - uses: actions/setup-python@v5
199
199
  with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
200
- - run: npm i -g "cowork-harness@>=1.7.0"
200
+ - run: npm i -g "cowork-harness@>=1.8.0"
201
201
  - run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
202
202
  - run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
203
203
  - run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
@@ -226,7 +226,7 @@ jobs:
226
226
  echo "live=true" >> "$GITHUB_OUTPUT"
227
227
  fi
228
228
  - if: steps.guard.outputs.live == 'true'
229
- run: npm i -g "cowork-harness@>=1.7.0"
229
+ run: npm i -g "cowork-harness@>=1.8.0"
230
230
  - if: steps.guard.outputs.live == 'true'
231
231
  run: cowork-harness run scenarios/ --output-format json
232
232
  env:
@@ -1,6 +1,6 @@
1
1
  # Fidelity tiers & answer paths
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 1.7.0` (baseline `desktop-1.20186.1`).
3
+ Self-contained reference. Tracks `cowork-harness 1.8.0` (baseline `desktop-1.24012.1`).
4
4
 
5
5
  ## Fidelity tiers (`fidelity:` in the scenario)
6
6
 
@@ -24,6 +24,22 @@ Self-contained reference. Tracks `cowork-harness 1.7.0` (baseline `desktop-1.201
24
24
  `skill` (any tier) and `chat` (`protocol`/`container`/`hostloop`; only `microvm`/`cowork` unsupported); `run` rejects an extra `--fidelity`
25
25
  positional ("Fidelity is set by the scenario's `fidelity:` field, not a flag").
26
26
 
27
+ ### hostloop paths: what Read sees vs what bash sees (uploads included)
28
+
29
+ At `hostloop` the native file tools (Read/Write/Edit/Glob/Grep) run **on the host** and are gated by a
30
+ path-containment hook (production's own model), while `mcp__workspace__bash` runs **in the VM sidecar**
31
+ and sees `/sessions/<id>/...` paths. Consequences an agent (or a debugging author) must know:
32
+
33
+ - A `/sessions/...` path handed to `Read` fails ("is a VM path") — that split is **production-faithful**,
34
+ not a harness bug. Translate via the prompt's path-mapping table.
35
+ - **Uploads ARE Read-able at hostloop**: the staged uploads dir on the host is in the containment
36
+ allowlist (read-only — writes/edits there are blocked, matching production). If a Read of an uploads
37
+ path fails "outside this session's connected folders", the path *form* is wrong (e.g. a relative
38
+ `mnt/uploads/...`, which resolves against the outputs cwd) — the file itself is reachable at the
39
+ staged host uploads dir; it does not need to be copied into `outputs/` first.
40
+ - In real Cowork an upload is a **hardlink** in the session's uploads dir; the harness **copies** it
41
+ instead. Read-reachability is the same; only the advertised parent dir can differ.
42
+
27
43
  ### `microvm` prerequisites & lifecycle
28
44
 
29
45
  Requires macOS on arm64 and Lima (`brew install lima`; binary expected at
@@ -1,7 +1,7 @@
1
1
  # Scenario & session schema, assertion catalog, web_fetch, full gotchas
2
2
 
3
- Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.7.0`
4
- (baseline `desktop-1.20186.1`). If your checkout is newer, prefer the live `docs/scenario.md`,
3
+ Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.8.0`
4
+ (baseline `desktop-1.24012.1`). If your checkout is newer, prefer the live `docs/scenario.md`,
5
5
  `docs/session.md`, and `SPEC.md`.
6
6
 
7
7
  **Minimal scenario** — `prompt` is the only required field:
@@ -1,8 +1,8 @@
1
1
  # Task recipes — end-to-end paths for the jobs consumers actually do
2
2
 
3
3
  Each recipe composes facts that live scattered across SKILL.md and the other references into one
4
- decision path. Every one answers a question a real fleet owner had to work out the hard way. Facts track the harness version in SKILL.md's
5
- front-matter (currently 1.5.0). Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
4
+ decision path. Every one answers a question a real fleet owner had to work out the hard way.
5
+ Tracks `cowork-harness 1.8.0` (baseline `desktop-1.24012.1`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
6
6
  Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
7
7
  needed if your CLI meets SKILL.md's version floor.
8
8
 
package/CHANGELOG.md CHANGED
@@ -6,6 +6,176 @@ All notable changes to this project are documented here. The format is based on
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [1.8.0] — 2026-07-23
10
+
11
+ ### Added
12
+
13
+ - **`check:versions` closes the stale-pin gap a docs audit found.** Three new enforcement surfaces:
14
+ every companion-skill `references/*.md` `(baseline desktop-X.Y.Z)` pin must match SKILL.md's
15
+ `tracks-harness` baseline; `task-recipes.md` now carries the same guarded
16
+ `` Tracks `cowork-harness X.Y.Z` `` stamp as the other references (and is a `bump-version` target);
17
+ DESIGN.md's one present-tense "currently **X**, per `baselines/desktop-Y.json`" sentence must name the
18
+ max committed baseline + its `agentVersion` (its dated verification-pass notes stay exempt). A new
19
+ docs-index sync test keeps README's Documentation table and docs/README.md's Guides table from
20
+ drifting apart (that drift is how `critique.md` went missing from the README table).
21
+
22
+ - **`doctor` warns when a *pulled* agent image is behind the published one.** For the
23
+ `container`/`hostloop`/`cowork` tiers, a new advisory `image-freshness` check compares the local agent
24
+ image's registry digest against the current published `ghcr.io/yaniv-golan/cowork-agent-base:2` (or
25
+ `-full`) and warns — with a re-pull + retag remedy — when they diverge. Best-effort and never blocking:
26
+ a locally-built image (no registry digest), an offline host, `docker buildx` being unavailable, or a
27
+ custom `COWORK_AGENT_IMAGE` is a quiet skip, never a false "stale"; only a pulled image incurs a network
28
+ probe. The `doctor --output-format json` envelope gains an `image-freshness` entry (the open
29
+ `checks[].id` set — SPEC §12 — already permits this).
30
+ - `critique` now stamps `verdictProvenance` on every report (JSON key + text "verdict scope" line): the
31
+ verdict is an advisory self-run, not an independent attestation.
32
+ - **`critique --skill <name>` — plugin-aware grading.** A multi-skill plugin root (`skills/<name>/SKILL.md`,
33
+ no root SKILL.md) previously graded a missing file, downgrading every coverage finding to "not
34
+ adjudicable". `--skill` selects the invoked skill's folder for the packager (selection only — the
35
+ positional folder is still what both turns mount, and `fingerprint.skillHash` is unchanged: per-plugin,
36
+ not per-skill); a multi-skill root with no `--skill` is refused **before any model spend**, naming the
37
+ available skills; a single-skill plugin auto-selects with a notice. The evidence package now also
38
+ carries the invoked skill's `agents/<name>.md` body and bounded `references/*.md` **content** (not just
39
+ filenames), and the report carries an advisory `skillInvocationObserved` (false = the graded run's own
40
+ `skillActivity` never mentions the selected skill).
41
+ - **`critique` persists run-dir artifacts.** Every critique writes `critique-report.json` (the
42
+ machine-readable report); when the evaluator ran, `critique-evidence-package.txt` (the exact armored
43
+ corpus it graded against — re-grade a disputed finding offline); on an instrument failure (exit 2),
44
+ `critique-salvage.json` (the self-report + each evaluator pass's RAW reply captured **pre-parse**).
45
+ New `--out <path>` also writes the selected-format report to a file. The two `not-built` limitations
46
+ these close (`evidence-not-persisted`, `report-stdout-only`) are removed from the registry.
47
+ - **Per-critique cost across all four workloads.** The report's `costUsd` sums the task turn, reflection
48
+ turn, and BOTH evaluator passes (whose usage the transport previously discarded), marked `INCOMPLETE`
49
+ whenever any workload is unpriced. The header also carries the pinned `fidelity` plus the graded turn's
50
+ recorded `gradedEffectiveFidelity`/`gradedBaseline`, and the graded run's resolved gate answers are
51
+ echoed as copy-pasteable `--answer` lines (JSON: `gateAnswers`).
52
+ - **`critique` evaluator parse is per-item tolerant.** One malformed item in an `{"items":[...]}` reply
53
+ previously discarded the whole document ("no valid JSON found", a broken discovery run). Valid items now
54
+ survive; malformed ones are dropped AND counted (`droppedEvaluatorItems`, surfaced in both report
55
+ formats). The integrity canary is recognized by its `idea` alone (a mutated echo still proves
56
+ instruction-following) and stripped before dedup, so a full-document + canary-only-restatement reply no
57
+ longer trips the ambiguity throw. Fail-loud preserved: garbage, or all-malformed-with-no-canary, still
58
+ throws — now naming which field check failed.
59
+ - **Sub-agent research is observable end-to-end.** A sub-agent's WebSearch never enters the main
60
+ `toolCounts`/`webSearches[]`; its query + result are now captured from the child session transcript as
61
+ `subagents[].webSearches` (bounded, live/record lane only — absent on replay, and absence is never
62
+ evidence of no research), surfaced by the new **`trace --view subagent-research`**, and packaged into
63
+ critique's evidence as a "Sub-agent research" section so "researched" claims are groundable.
64
+ - Each critique item now carries a **`findingFingerprint`** (sha16 over the normalized
65
+ idea + classification + recommendedAction, deliberately excluding the input-specific evidence excerpt) —
66
+ clusters the SAME finding across DIFFERENT inputs, complementing `skillHash` (same skill across fixes).
67
+ docs/critique.md gains a "Reproduction" section documenting the ≥2-run discipline.
68
+ - SKILL.md truncation is now reported distinctly: a readable-but-oversized SKILL.md marks the report
69
+ `skillMdTruncated` ("the evaluator graded a cut copy") instead of being indistinguishable from a fully
70
+ packaged one; only missing/unreadable still forces the mechanical "already-covered" downgrade — the
71
+ `--help`/docs limitation wording now says so.
72
+
73
+ ### Changed
74
+
75
+ - **Published agent images (`cowork-agent-base` / `cowork-agent-full`) now carry OCI metadata labels**
76
+ (`org.opencontainers.image.{title,description,source,documentation,licenses}`), so their GHCR package
77
+ pages render a description, repo link, license, and the "contains no Anthropic binary" provenance note
78
+ instead of appearing bare. The `publish-image` workflow sets the `full` variant's title/description via
79
+ `--label` so its page isn't mislabeled by the base image's baked-in defaults.
80
+ - `critique` evidence caps raised: SKILL.md 16KB→64KB, transcript 16KB→32KB, overall package 48KB→144KB
81
+ (the overall cap sits above the worst-case per-section sum, agents/references/research sections included), so
82
+ a flagship-sized (~51KB) SKILL.md no longer grades permanently truncated. Increases per-critique evaluator
83
+ token cost on large skills (~2–2.5×).
84
+ - `sync`'s two code-tripwire warnings (the `getMcpSkillSources` caller-count and MCP-skills-capability
85
+ checks) now carry a self-contained instruction — re-verify whether MCP servers can contribute skills and
86
+ whether the harness must model MCP-contributed skill sources — instead of pointing at a reference the
87
+ published repo does not carry.
88
+ - **Pinned sessions stamp their fidelity tier on `session.json`, and a cross-tier `--resume` fails loud**
89
+ (pre-spawn, with a "re-run at `--fidelity <stamped>`" remedy). The agent's native conversation store is
90
+ tier-LOCAL — container persists it under the work tree, hostloop under the host config dir — so resuming
91
+ a session at a different tier would hand the binary a `--resume` for a conversation its store has never
92
+ seen. Legacy stampless manifests (pre-dating the stamp) are let through with a warning; every manifest
93
+ written from now on carries the stamp. `readSessionManifest` gains a required `expectedFidelity`
94
+ argument.
95
+
96
+ ### Docs
97
+
98
+ - README documents that the agent images are **published to GHCR** and can be pulled + retagged instead of
99
+ built from scratch (`:2` floats to the latest release; `:2-<version>` pins an immutable per-release
100
+ build), and that the harness resolves the *local* tag — so a stale local image shadows the published one;
101
+ re-pull after upgrading. The `doctor` command row notes the new freshness warning.
102
+
103
+ ### Fixed
104
+
105
+ - **hostloop uploads bullet advertised a non-readable path.** The dynamic "## Shell access" section
106
+ rendered the uploads mapping's file-tool side as `dirname(upload.hostPath)` — the user's original
107
+ source dir, which the path-containment gate does not allow — while the base prompt pointed at the
108
+ correct staged dir. An agent following the bullet got "outside this session's connected folders" and
109
+ worked around it via copy-into-outputs + `rm`, tripping a spurious `outputs-delete` fail. The bullet
110
+ now advertises the staged uploads dir — the SAME hoisted value the path gate allows, so the prompt and
111
+ the gate cannot disagree. (Live-verified: hostloop `--upload` + Read-tool read succeeds at the
112
+ advertised path.)
113
+ - The task-turn timeout kill message now names `--timeout` and the 10-minute default; a "missing"
114
+ SKILL.md report note now points at `--skill` / the invoked skill's folder (the multi-skill-plugin-root
115
+ cause) instead of only stating the symptom.
116
+ - Shipped-doc corrections that induced field misdiagnoses: the cowork-harness skill's gotcha #8 now states
117
+ that **production enforces** outputs delete-deny (EPERM + approval) and the harness gap is
118
+ detection-only; docs/critique.md documents that a WebSearch produces **no search-host entries in the
119
+ container egress.log** (an `api.anthropic.com`-only log is consistent with research working —
120
+ live-verified with a first-party capture) and that sub-agent searches don't increment the main
121
+ `toolCounts.WebSearch`; the shipped fidelity reference explains the hostloop Read-vs-bash path split
122
+ including uploads readability.
123
+ - **`critique` now runs at `--fidelity hostloop` as well as `container` — the container-tier pin is
124
+ lifted.** The pin existed because the reflection turn *resumes* the task turn and resume-continuity was
125
+ only proven for the container Linux ELF; a live two-turn proof at hostloop's **native** agent binary
126
+ (`test/live-contract.test.ts`, "resume-continuity proof at hostloop") cleared it, and a live
127
+ `critique --fidelity hostloop` e2e validates the full protocol there. `microvm`/`protocol`/`cowork` stay
128
+ refused, each with its own stated reason: the single `container-tier-only` limitation is replaced by
129
+ three tagged ones — `microvm-tier-refused` `[unverified]` (a resume-continuity proof at the microVM guest
130
+ would lift it), `protocol-tier-refused` `[not-built]` (protocol never plumbs a session id/`--resume`),
131
+ and `cowork-tier-refused` `[deliberate]` (the synced loop gate would make the graded tier
132
+ baseline-dependent, adding noise to skillHash-paired generation comparisons).
133
+ - **`skill` accepts `--allow-host-writes`** (and `critique` forwards it to BOTH turns) — the
134
+ hostloop writable-connected-folder consent that previously only `chat` (its own flag) and `run` (the
135
+ `allow_host_writes: true` scenario field) could grant; a plain `skill --fidelity hostloop --folder X`
136
+ was refused with no way to consent at all. Folder-less runs (skill dir + uploads) still need no consent —
137
+ uploads and skill/plugin mounts are read-only.
138
+ - **`critique`'s spaced flag parser no longer silently grabs the next flag as a value.** `critique
139
+ <folder> --prompt --output-format json` (a forgotten `--prompt` value) swallowed `--output-format` as the
140
+ prompt AND dropped the real flag, then ran a four-workload critique on the wrong input. The spaced form
141
+ now fails loud and points at the equals escape hatch (`--prompt=<value>`) for a value that intentionally
142
+ starts with `-`.
143
+ - **`critique`'s subprocess byte cap is now one combined stdout+stderr budget.** `boundedSpawn` kept two
144
+ independent per-stream counters, so a looping or hostile child splitting output across both streams could
145
+ buffer ~2× the documented cap before either tripped. It now charges both streams against a single budget
146
+ and slices the terminal chunk to the remaining room, so captured output never exceeds the cap.
147
+ - **`critique`'s verbatim citation-grounding is now case-sensitive.** The mechanical check that drops any
148
+ finding whose cited evidence is not a verbatim excerpt normalized whitespace AND folded case — so a
149
+ case-altered (paraphrased) citation resolved as "grounded," weakening the principal defense against
150
+ evaluator hallucination. It now matches case-exact; whitespace reflow stays tolerated (models reflow
151
+ spacing when quoting — they don't change case).
152
+ - **critique's "Attached inputs" evidence no longer reports `(none)` when the uploads dir is unreadable.**
153
+ `listAttachedInputs` caught every `readdirSync` failure identically, so an unreadable uploads dir
154
+ (`EACCES`/`ENOTDIR`/…) collapsed to the same `(none)` as a legitimately-absent one — telling the
155
+ evaluator "the agent correctly saw no file" when attachment presence was actually UNKNOWN. It now
156
+ distinguishes `ENOENT` (absent) from a genuine read fault and surfaces the uncertainty loudly.
157
+ - **critique flags a partly-corrupt archived turn-1 transcript instead of grading it as clean.**
158
+ `readTurn1Transcript` skipped malformed JSONL rows (resilient) but returned `degraded: false` on the
159
+ first valid transcript record regardless — so archive corruption vanished from evidence health and the
160
+ evaluator graded a silently-incomplete transcript as solid ground truth. It now scans the whole
161
+ (turn-1-only) archive and sets `turn1SliceDegraded` when a malformed row was skipped or the archive
162
+ doesn't hold exactly one transcript record; the transcript is still delivered, just flagged.
163
+ - **critique flags a crashed task turn instead of grading evidence from it.** `taskTurnInfraFailure` only
164
+ treated a *killed* (timed-out / byte-capped) task turn as an instrument failure — a task that exited
165
+ nonzero without ever printing a parseable result envelope (a crash after the early `[status]` line)
166
+ slipped through, and its `[status]`-recovered run dir was reflected on and graded as a legitimate task.
167
+ It now also flags a nonzero exit with no parseable envelope. Deliberately narrow: a completed run that
168
+ reported a failing verdict (`ok:false` / `result:"error"` with a valid envelope) stays gradeable.
169
+ - **A nested unreadable output subtree no longer persists a partial file list as complete.** The
170
+ `workspaceFiles` / `artifacts` list was recorded as UNAVAILABLE (`undefined`, the evidence-unavailable
171
+ convention) only when the workspace ROOT was unobservable; a nested `EACCES`/`EIO` subtree left the walk
172
+ partial (`walkComplete: false`) but still persisted the incompletely-enumerated list as if complete. An
173
+ authored file inside the unreadable subtree then vanished with no signal, so an absence-sensitive
174
+ consumer (`delivered_clean` / `ended_with_question`, the replay `diff`, `scaffold`, `file_exists`) could
175
+ read it as absent — a silent false-clean. All three RunResult producers (run, partial-salvage, chat) now
176
+ route through one shared `trustedWorkspaceFiles` gate that collapses a missing root OR any incomplete
177
+ walk to `undefined`, and the run lanes emit a `::warning::` naming the unreadable subtree.
178
+
9
179
  ## [1.7.0] — 2026-07-22
10
180
 
11
181
  ### Added
package/CONTRIBUTING.md CHANGED
@@ -12,7 +12,7 @@ npm run format:check # prettier over src/ + test/ .ts — the most common first
12
12
  npm run typecheck # tsc --noEmit
13
13
  npm test # vitest
14
14
  npm run build # -> dist/
15
- npm run ci # typecheck + build + test — the core TS gate (CI's `unit` job runs more; see below)
15
+ npm run ci # typecheck + build + test — the core TS gate (CI's `build` + `test` jobs run more; see below)
16
16
  ```
17
17
 
18
18
  Before pushing: `npm run format:check` and `npm run ci` (see RELEASING.md's release checklist for the full pre-tag list).
@@ -25,10 +25,11 @@ Extra prerequisites for specific stages:
25
25
  - **Docker (arm64)** — required for `boundary-check` and the **L1 `container`** + `hostloop` fidelity tiers (the container sandbox + agent image).
26
26
  - **Lima (`limactl`, macOS arm64)** — required only for the **L2 `microvm`** tier and the `vm` commands; the guest runs on Apple Virtualization.framework (`vmType: vz`). `microvm` does **not** use Docker. (`cowork-harness doctor --tier microvm` checks for Lima, not Docker.)
27
27
 
28
- CI Stage 1 (the `unit` job in `.github/workflows/ci.yml`) does **not** invoke `npm run ci`. It runs the
29
- gate steps individually — e.g. `check:versions`, `format:check`, `typecheck`, `npm test`, `build`, a CLI
30
- smoke (`node dist/cli.js list`), three token-free `replay` gate fixtures, `verify-cassettes`, `lint`, and
31
- source-guard checks (see `ci.yml` for the authoritative list). Only `release.yml` calls `npm run ci`.
28
+ CI Stage 1 (the `build` job in `.github/workflows/ci.yml`) does **not** invoke `npm run ci`. It runs the
29
+ gate steps individually — e.g. `check:versions`, `format:check`, `typecheck`, `build`, a CLI smoke
30
+ (`node dist/cli.js list`), three token-free `replay` gate fixtures, `verify-cassettes`, `lint`, and
31
+ source-guard checks — while the unit suite runs separately as the 4-shard `test` job (see `ci.yml` for
32
+ the authoritative list). Only `release.yml` calls `npm run ci`.
32
33
 
33
34
  > **Cutting a release?** See [RELEASING.md](./RELEASING.md) for the branch → PR → tag → publish flow.
34
35
 
@@ -85,7 +86,7 @@ When a Desktop release moves something `sync` doesn't read, it reports an `unkno
85
86
  ## Commit & PR
86
87
 
87
88
  - Conventional, imperative commit subjects (`add …`, `fix …`, `parity: sync to <ver>`).
88
- - Open PRs against `main`. CI runs a six-stage pipeline (`unit`, `action-self-test`, `python`, `boundary`, `scenarios`, `parity-drift` — see `ci.yml`) on every PR including forks (no secrets needed) except `scenarios`; live scenarios only run on same-repo PRs/pushes with `ANTHROPIC_API_KEY` set.
89
+ - Open PRs against `main`. CI runs a seven-stage pipeline (`build`, `test`, `action-self-test`, `python`, `boundary`, `scenarios`, `parity-drift` — see `ci.yml`) on every PR including forks (no secrets needed) except `scenarios`; live scenarios only run on same-repo PRs/pushes with `ANTHROPIC_API_KEY` set.
89
90
  - Describe *what changed and why*; link issues.
90
91
 
91
92
  ## Reporting issues
package/DESIGN.md CHANGED
@@ -38,7 +38,7 @@ flowchart TB
38
38
  A Cowork session is the Desktop app driving an agent that runs **inside an Apple Virtualization.framework microVM**:
39
39
 
40
40
  - VM bundle: `~/Library/Application Support/Claude/vm_bundles/claudevm.bundle/` (`rootfs.img`, `sessiondata.img`, `efivars.fd`, `machineIdentifier`, `gvisorMacAddress`, `vmIP`); a warm pool at `vm_bundles/warm/<sha>/`.
41
- - In-VM agent: `~/Library/Application Support/Claude/claude-code-vm/<ver>/claude` (currently **2.1.205**, per `baselines/desktop-1.20186.1.json`, an **ELF aarch64** binary), spawned by the host **in cowork mode via the `CLAUDE_CODE_IS_COWORK=1` env var** — *not* a `--cowork` flag (that flag is plugin-scope and the staged agent rejects it; see the control-protocol note below). Each baseline records this ELF's `sha256` (`agentBinary.sha256`/`shaProvenance`), and the resolver integrity-checks the binary it's about to run against that hash by default (opt out `COWORK_HARNESS_VERIFY_AGENT_SHA=0`), so "the same pinned agent" is enforced, not just asserted. Old versions are re-downloadable + verifiable from the official release channel (see `docs/maintenance.md`).
41
+ - In-VM agent: `~/Library/Application Support/Claude/claude-code-vm/<ver>/claude` (currently **2.1.217**, per `baselines/desktop-1.24012.1.json`, an **ELF aarch64** binary), spawned by the host **in cowork mode via the `CLAUDE_CODE_IS_COWORK=1` env var** — *not* a `--cowork` flag (that flag is plugin-scope and the staged agent rejects it; see the control-protocol note below). Each baseline records this ELF's `sha256` (`agentBinary.sha256`/`shaProvenance`), and the resolver integrity-checks the binary it's about to run against that hash by default (opt out `COWORK_HARNESS_VERIFY_AGENT_SHA=0`), so "the same pinned agent" is enforced, not just asserted. Old versions are re-downloadable + verifiable from the official release channel (see `docs/maintenance.md`).
42
42
  - Network: `vm_network_mode: "gvisor"`, egress through a userspace netstack with a **compiled domain allowlist**; off-list partners rejected (`partner rejected: entry not on compiled allowlist`).
43
43
  - Control plane: Electron renderer→main typed IPC on channels named `$eipc_message$_<per-build-UUID>_$_claude.web_$_<Class>_$_<method>`, every handler validating `event.senderFrame.url` against a trusted-origin allowlist. The session manager is `LocalAgentModeSessions` (80 methods: `start`, `sendMessage`, `setDraftSessionFolders`, `onToolPermissionRequest`, `respondToToolPermission`, `getTranscript`, `onEvent`, …), bridged to the renderer as `window.cowork`.
44
44
 
package/README.md CHANGED
@@ -91,7 +91,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
91
91
 
92
92
  > **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
93
93
  > <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
94
- > From a global install (`npm i -g "cowork-harness@>=1.7.0"`), point at the package root instead:
94
+ > From a global install (`npm i -g "cowork-harness@>=1.8.0"`), point at the package root instead:
95
95
  > `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
96
96
  > (or copy the cassette into your own project and pass that path).
97
97
 
@@ -101,7 +101,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
101
101
  > - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
102
102
  > - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
103
103
  > - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
104
- > - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=1.7.0"`.
104
+ > - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=1.8.0"`.
105
105
 
106
106
  Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
107
107
  test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
@@ -126,7 +126,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
126
126
  claude plugin install cowork-harness@cowork-harness
127
127
  ```
128
128
 
129
- The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=1.7.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
129
+ The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=1.8.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
130
130
 
131
131
  It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
132
132
 
@@ -147,7 +147,7 @@ A global install is enough for CI `lint`, reading the teaching skill, and replay
147
147
  To `run` the worked examples live or copy them as a starting point, use a source checkout. (The marketplace
148
148
  skill install itself only pulls `.claude/skills/cowork-harness/` — SKILL.md + `references/` + `scenario.py`/
149
149
  assertion keys, per `.claude-plugin/marketplace.json`'s `source` — not the rest of this table; the full set
150
- above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=1.7.0"` — see
150
+ above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=1.8.0"` — see
151
151
  [above](#drive-it-from-claude-code-companion-skill) — which pulls the same npm package as the global-install row.)
152
152
 
153
153
  ### Prerequisites for anything above `protocol` fidelity
@@ -190,7 +190,11 @@ So **Linux live == `container` only**: `microvm` is Apple-VZ (macOS), and `hostl
190
190
 
191
191
  > The `pytest` lane (`python/README.md`) has its own, slightly different prerequisite list (adds `pytest`
192
192
  > + package importability) — see there if you're driving tests via `pytest` instead of the CLI directly.
193
- 3. **An auth token** — either `export CLAUDE_CODE_OAUTH_TOKEN=$(claude setup-token)` (the `claude setup-token` step needs the **`claude` CLI**: `npm i -g @anthropic-ai/claude-code`) or a **`.env`** file (copy `.env.example` → `.env`; gitignored). (`.env.example` lists all three accepted vars — `CLAUDE_CODE_OAUTH_TOKEN`, then `ANTHROPIC_API_KEY`, then `ANTHROPIC_AUTH_TOKEN` — in that precedence order.) The token resolves in priority order: exported env > `--dotenv <path>` > `./.env` (cwd) > `<install>/.env` (the package root), so a `npm link`ed install works from any directory. That chain is about *precedence*, not error recovery: a `--dotenv` path you passed explicitly must be readable — if it is missing, unreadable, or a directory, the run stops with a usage error rather than quietly falling through to `./.env` and running against different credentials. The automatic `./.env` / `<install>/.env` locations remain best-effort. Keep `.env` at a working-dir or install root, never inside a mounted skill/project folder. (Use `--dotenv`, not `--env-file` — Node reserves the latter.) For a global `npm install -g` install, find the package root with `` `$(npm root -g)/cowork-harness` `` (e.g. `$(npm root -g)/cowork-harness/.env`) — or simpler, just use `--dotenv <path>` / `./.env` in your working directory, which take priority over the package root anyway.
193
+ 3. **An auth token** — either `export CLAUDE_CODE_OAUTH_TOKEN=$(claude setup-token)` (the `claude setup-token` step needs the **`claude` CLI**: `npm i -g @anthropic-ai/claude-code`) or a **`.env`** file (copy `.env.example` → `.env`; gitignored). `.env.example` lists all three accepted vars — `CLAUDE_CODE_OAUTH_TOKEN`, then `ANTHROPIC_API_KEY`, then `ANTHROPIC_AUTH_TOKEN` — in that precedence order.
194
+ - **Resolution order:** exported env > `--dotenv <path>` > `./.env` (cwd) > `<install>/.env` (the package root) — so a `npm link`ed install works from any directory. (Use `--dotenv`, not `--env-file` — Node reserves the latter.)
195
+ - **Explicit beats best-effort:** that chain is about *precedence*, not error recovery. A `--dotenv` path you passed explicitly must be readable — if it is missing, unreadable, or a directory, the run stops with a usage error rather than quietly falling through to `./.env` and running against different credentials. The automatic `./.env` / `<install>/.env` locations remain best-effort.
196
+ - **Placement:** keep `.env` at a working-dir or install root, never inside a mounted skill/project folder.
197
+ - **Global install:** find the package root with `` `$(npm root -g)/cowork-harness` `` (e.g. `$(npm root -g)/cowork-harness/.env`) — or simpler, just use `--dotenv <path>` / `./.env` in your working directory, which take priority over the package root anyway.
194
198
 
195
199
  > `sync` (below) is **optional for a first run** — the repo ships `baselines/desktop-*.json`, so `baseline: latest` already resolves. Run `sync` only to refresh the platform baseline after Claude Desktop updates. (`sync` is **macOS-only** today; on Linux/Windows use the committed baselines — they work cross-platform.)
196
200
 
@@ -398,7 +402,7 @@ Skill testing is the headline use, but the tool is a general harness over the Co
398
402
  | `record` / `replay` | **Record a live run once → replay it token-free, Docker-free thereafter** (key flags below; `replay --explain` prints the evidence behind every passing assert) | **token-free, Docker-free CI** from a once-recorded run |
399
403
  | `verify-cassettes <file\|dir>` | Token-free CI gate over committed cassettes: a privacy scan (email/currency/domain/path/machine-inventory) + a staleness check (allowlist and skip flags below); a dir argument scans `*.cassette.json` non-recursively | gating **committed cassettes** against PII leaks + "edited the skill, forgot to re-record" |
400
404
  | `verify-run <run-dir> <scenario.yaml>` | Re-evaluate a scenario's `assert:` (and, when the scenario declares `answers:`, whether they still match the run's actual gates) against an already-kept run dir — **no live agent, no tokens, no Docker** (~1s) | iterating on a wrong assertion or a drifted `answer` without a full live re-record |
401
- | `trace <run-id>` | Digest a run's `events.jsonl` (`--view tools\|questions\|dispatches\|tool-durations\|tool-errors\|files\|usage`; `--translate-paths` rewrites VM paths to host paths in the text `tools`/default views; `--full-results` captures the full input + result of every tool call — successful ones too, not just errors — so an external grader can ground a self-critique finding against the call it cites) | "how many sub-agents *actually* dispatched, and which?" — per-tool call/timing stats with `--view tool-durations`, full stderr per failed call with `--view tool-errors`, a workspace-file diff with `--view files`, or per-model token/cost with `--view usage` |
405
+ | `trace <run-id>` | Digest a run's `events.jsonl` (`--view tools\|questions\|dispatches\|tool-durations\|tool-errors\|files\|usage\|subagent-research`; `--translate-paths` rewrites VM paths to host paths in the text `tools`/default views; `--full-results` captures the full input + result of every tool call — successful ones too, not just errors — so an external grader can ground a self-critique finding against the call it cites) | "how many sub-agents *actually* dispatched, and which?" — per-tool call/timing stats with `--view tool-durations`, full stderr per failed call with `--view tool-errors`, a workspace-file diff with `--view files`, per-model token/cost with `--view usage`, or each dispatch's own WebSearch query+result with `--view subagent-research` (live/record lane) |
402
406
  | `inspect <run-id>` | Show what a run **produced**: the artifacts + a shallow field preview of each JSON artifact (`--output-format json` for a digest). Works on a salvaged partial run too | "did it do the job?" — without hand-parsing `…/mnt/outputs/…` |
403
407
  | `scaffold <run-id>` | Turn a kept run into a starter scenario YAML (gates→answers, artifacts→`file_exists`) | authoring a scenario from a real run instead of guessing |
404
408
  | `python3 …/scenario.py scaffold --name <name> --skill <dir>` | Generate a starter scenario skeleton from scratch (the `…` is `.claude/skills/cowork-harness/scripts/`) | starting a new scenario when you have no prior run |
@@ -415,8 +419,8 @@ Skill testing is the headline use, but the tool is a general harness over the Co
415
419
  | `boundary-check [baseline] [--session <file>]` | Prove the **L1 Docker** sandbox enforces Cowork's limitations (sealed FS + default-deny egress; `container`/`hostloop` share this sandbox — `microvm`'s guest firewall is not probed here); `--session` folds a session's `egress.extra_allow` into the probe allowlist | verifying the harness's own fidelity |
416
420
  | `sync` / `list` | Derive/refresh (`sync [--diff] [--allow-empty\|--force]`) & list platform baselines from the Desktop install | after Claude Desktop updates (baselines ship, so it's optional otherwise) |
417
421
  | `diff <a> <b>` | Compare two baselines, two runs, two cassettes, or a run+cassette — kind auto-detected by content (view/normalization flags below). Token-free, no live Desktop/Docker needed | "what changed between two runs/cassettes/baselines?" |
418
- | `critique <skill-folder>` | **EXPERIMENTAL.** Run a skill, ask the agent what confused it, then grade that self-report against a frozen record of the run — blinded evaluator, mechanical citation checking. Accepts the `skill` flags a graded run needs — `--upload`/`--folder`/`--plugin` reach both spawned turns, `--answer`/`--decider-*` the graded turn only; anything that can't work is refused with a reason (full table in `critique --help`). Discovery instrument, never a gate (findings always exit 0). Costs four model workloads; see [docs/critique.md](./docs/critique.md) | "what confused the agent about my skill?" — including "does it even see the attached document?" |
419
- | `doctor [--tier <t>]` | Read-only prerequisite check, per tier (Docker + agent image for `container`/`hostloop`/`cowork`; **Lima** for `microvm`; plus staged agent, token, baseline); prints the exact `docker build` line if the agent image is missing | "can I run the live tiers — what's missing?" before a first live run |
422
+ | `critique <skill-folder>` | **EXPERIMENTAL.** Run a skill, ask the agent what confused it, then grade that self-report against a frozen record of the run — blinded evaluator, mechanical citation checking. Accepts the `skill` flags a graded run needs — `--upload`/`--folder`/`--plugin` reach both spawned turns, `--answer`/`--decider-*` the graded turn only; anything that can't work is refused with a reason (full table in `critique --help`). Runs at `--fidelity container` (default) or `hostloop` (a writable `--folder` there additionally needs `--allow-host-writes`); `microvm`/`protocol`/`cowork` are refused, each with a stated reason. A multi-skill plugin needs `--skill <name>` (refused pre-spend otherwise). Persists `critique-report.json` + the armored evidence package to the run dir (`--out` copies the report; a salvage file lands on exit 2), reports per-critique cost across all four workloads, and stamps each finding with a cross-input `findingFingerprint`. Discovery instrument, never a gate (findings always exit 0). Advisory — a discovery lead, not an independent attestation (the skill under review can steer the grade). See [docs/critique.md](./docs/critique.md) | "what confused the agent about my skill?" — including "does it even see the attached document?" |
423
+ | `doctor [--tier <t>]` | Read-only prerequisite check, per tier (Docker + agent image for `container`/`hostloop`/`cowork`; **Lima** for `microvm`; plus staged agent, token, baseline); prints the exact `docker build` line if the agent image is missing, and — for a **pulled** agent image — warns (advisory, best-effort) when its digest doesn't match the current published GHCR `:2`, so you can re-pull | "can I run the live tiers — what's missing?" before a first live run |
420
424
  | `migrate-run-dir [<runs-dir>] [--scenario <name>] [--write]` | Convert pre-layout run dirs (artifacts at the run-dir root) to the per-turn `turns/<N>/` layout, in place — **dry-run by default**, preserving the mtimes `stats`/`--latest-for` rank by, and refusing any dir it cannot resolve rather than guessing | upgrading a runs root written by an older version, so `verify-run`/`diff`/`inspect` can read it again; `--scenario` scopes it so you can migrate one, verify, then do the rest |
421
425
  | `prune [<runs-dir>] [--keep-last <n>] [--pinned-older-than <N>d\|h\|m]` | Prune accumulated run dirs (keeps the N most recent per scenario; pinned `--session-id` runs are never pruned unless `--pinned-older-than` opts in to reclaiming stale ones by last-activity age); the optional positional overrides the runs root; `--dry-run` | the machine-global runs root has grown and you want space back |
422
426
  | `rehash <dir/>` | Migrate cassette fingerprints to the current format version when the content is provably unchanged (`--dry-run`); no re-record needed | a cassette-format bump flagged committed fixtures as stale |
@@ -686,13 +690,14 @@ jobs:
686
690
  anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
687
691
  ```
688
692
 
689
- Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — intentional so recipes track the current release; pin an exact version for reproducible CI. The companion skill's `cowork-harness@>=1.7.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). See [`action.yml`](./action.yml) for the full input reference.
693
+ Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — intentional so recipes track the current release; pin an exact version for reproducible CI. The companion skill's `cowork-harness@>=1.8.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). See [`action.yml`](./action.yml) for the full input reference.
690
694
 
691
- The provided [GitHub Actions workflow](.github/workflows/ci.yml) runs a **six-stage pipeline**. The **unit** stage is the token-free gate you can copy into your skill repo; the `action-self-test`, `python`, `boundary`, `scenarios`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):
695
+ The provided [GitHub Actions workflow](.github/workflows/ci.yml) runs a **seven-stage pipeline**. The **build** + **test** stages are the token-free gate you can copy into your skill repo; the `action-self-test`, `python`, `boundary`, `scenarios`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):
692
696
 
693
697
  | Stage | Runs | Needs | Gates |
694
698
  |---|---|---|---|
695
- | **unit** | format check · typecheck · unit tests · build · CLI smoke · token-free `replay` · `verify-cassettes` · `lint` | nothing | every push/PR |
699
+ | **build** | format check · version-lockstep guard · typecheck · source guards · build · CLI smoke · token-free `replay` · `verify-cassettes` · `lint` | nothing | every push/PR |
700
+ | **test** | the unit suite (vitest), sharded 4-way | nothing | every push/PR |
696
701
  | **action-self-test** | packs this commit and runs the packaged `uses: ./` Action against the committed example-pdf-skill cassette (pass case) + a nonexistent path (fail case) | nothing | every push/PR |
697
702
  | **python** | `pytest` helper self-checks (`python/`, run with `-m 'not cowork'` — the token-free subset; the Docker/token `@pytest.mark.cowork` tests are excluded) | nothing (token-free assertions only) | every push/PR |
698
703
  | **boundary** | builds the pinned agent image, brings up the default-deny network, runs `boundary-check`, then `npm run test:live` (live contract tests that guard the binary-resolution assumptions, no token needed) | Docker, arm64 runner | proves the sandbox enforces Cowork's limits — **no API key** |
@@ -815,6 +820,7 @@ a global install has them locally too, not just on GitHub.
815
820
  | [docs/chat.md](./docs/chat.md) | The interactive `chat` REPL — multi-turn debugging, flags, attaching files/folders. |
816
821
  | [docs/debugging.md](./docs/debugging.md) | Debugging a run — `inspect`/`trace`/`verify-run`/`diff`/`chat` for a misbehaving skill; the false-green hunt for a green you don't trust; and the **iterate-across-fixes verification loop** (ground findings in run evidence; pair by `fingerprint.skillHash`). |
817
822
  | [docs/cassette.md](./docs/cassette.md) | `record`/`replay` cassettes — what replay checks, which assertions are skipped. |
823
+ | [docs/critique.md](./docs/critique.md) | **EXPERIMENTAL** — `critique`: run a skill, ask the agent what confused it, then grade that self-report against a frozen record. Its verdict is an advisory lead, not an attestation. |
818
824
  | [docs/run-status.md](./docs/run-status.md) | Checking whether a background run is alive — the `status.json` file + `cowork-harness status [--follow]`. |
819
825
  | [docs/stats.md](./docs/stats.md) | The `stats` command + `index.jsonl` — querying pass rate, cost/duration/token/turn percentiles, and last-green across every past run. |
820
826
  | [docs/gotchas.md](./docs/gotchas.md) | Troubleshooting FAQ — exit 127, empty skill mount, arm64 Docker issues, git-worktree token traps, scenarioDrift after an edit, plus skill-authoring/host-loop footguns. |
package/RELEASING.md CHANGED
@@ -104,10 +104,12 @@ tagging `1.0.0`, deliberately review and freeze the surfaces with no machine-rea
104
104
  `npx "cowork-harness@>=X.Y.Z"`).
105
105
  5. `.claude/skills/cowork-harness/references/scenario-schema.md` → the
106
106
  "Tracks `cowork-harness X.Y.Z`" line.
107
- 6. `.claude/skills/cowork-harness/references/fidelity-and-answers.md` → the
108
- "Tracks `cowork-harness X.Y.Z`" line.
107
+ 6. `.claude/skills/cowork-harness/references/fidelity-and-answers.md` and
108
+ `.claude/skills/cowork-harness/references/task-recipes.md` → the
109
+ "Tracks `cowork-harness X.Y.Z`" line in each.
109
110
  7. The baseline these track (`tracks-harness … (baseline desktop-<ver>)`) — keep in sync with the
110
- newest `baselines/desktop-*.json`.
111
+ newest `baselines/desktop-*.json`. The `check:versions` guard enforces this for SKILL.md, every
112
+ `references/*.md` baseline pin, and DESIGN.md's current-state sentence — a lagging pin reds CI.
111
113
  8. `.claude/skills/cowork-harness/references/ci-recipe.md` → all `npm i -g "cowork-harness@>=X.Y.Z"` floors
112
114
  (currently 3 occurrences).
113
115
  9. `examples/replays/README.md` → the `npm i -g "cowork-harness@>=X.Y.Z"` floor.
@@ -185,5 +187,5 @@ tagging `1.0.0`, deliberately review and freeze the surfaces with no machine-rea
185
187
  (`git tag -d vX.Y.Z`), re-create it on the correct commit, and push it.
186
188
  - The live `scenario suite` CI stage runs on **same-repo** PRs (where the `ANTHROPIC_API_KEY` secret is
187
189
  available) and on pushes to `main`; it is skipped only on **fork** PRs (or when the secret is unset). So
188
- a same-repo release-branch PR DOES exercise the live suite and spend API budget — the `unit` + `boundary`
189
- stages alone are sufficient to gate a release if you'd rather not.
190
+ a same-repo release-branch PR DOES exercise the live suite and spend API budget — the `build` + `test` +
191
+ `boundary` stages alone are sufficient to gate a release if you'd rather not.