cowork-harness 0.21.0 → 0.22.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 0.21.0
-  tracks-harness: cowork-harness 0.21.0 (baseline desktop-1.17377.2)
+  version: 0.22.0
+  tracks-harness: cowork-harness 0.22.0 (baseline desktop-1.18286.0)
 ---
 
 # cowork-harness
@@ -22,8 +22,8 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.21.0` (baseline
-> `desktop-1.17377.2`). If your checkout is newer, prefer the live `--help`, `SPEC.md`, and
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.22.0` (baseline
+> `desktop-1.18286.0`). If your checkout is newer, prefer the live `--help`, `SPEC.md`, and
 > `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## 0. Preflight — make sure the harness can actually run
@@ -38,7 +38,7 @@ cowork-harness skill ./my-skill "do X"      # run the skill once against the sta
 Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.21.0**. If it's missing or older, prefix every command with the version floor `npx cowork-harness@>=0.21.0 <cmd>` (Node ≥ 20), or install once with `npm i -g cowork-harness@>=0.21.0`. **Pin `@>=0.21.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.21.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator, `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, and `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field.)
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.22.0**. If it's missing or older, prefix every command with the version floor `npx cowork-harness@>=0.22.0 <cmd>` (Node ≥ 20), or install once with `npm i -g cowork-harness@>=0.22.0`. **Pin `@>=0.22.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.22.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator, `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, and `computer_links_resolve` (new in 0.22.0).)
 - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
 - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
 - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
@@ -54,8 +54,8 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
   debugging map).
 
 Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · lint ·
-verify-run · trace · inspect · decide · gates · answer · scaffold · assertions --list · sync · list ·
-boundary-check · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
+verify-run · trace · inspect · diff · stats · decide · gates · answer · scaffold · assertions --list · sync ·
+list · boundary-check · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
 
 ## 2. Two files: session vs scenario
 
@@ -288,7 +288,9 @@ keys). Exit code is non-zero on errors (CI-friendly). `scaffold` auto-upgrades t
 for egress on `protocol`, so it never emits a scenario `lint` would reject.
 
 CI placement: a **token-free `replay` PR gate** (content/structure only) + a **nightly live `run`**
-(filesystem/egress). See `references/ci-recipe.md` for the four-stage pipeline.
+(filesystem/egress). Fastest setup: `uses: yaniv-golan/cowork-harness@v1` (a packaged GitHub Action with a
+PR job-summary reporter). See `references/ci-recipe.md` for the Action, the manual step-by-step form, and
+the four-stage pipeline.
 
 ## 10. Debugging with `chat`
 
@@ -326,7 +328,8 @@ are the ones that bite hardest.
    `replay` skips filesystem/egress keys (`file_exists`, `user_visible_artifact`, `artifact_json`,
    `egress_*`, `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`); a *mixed* item like
    `{result, egress_denied}` greens on `result` while its `egress_denied` half is dropped. (`record`
-   snapshots an `artifacts` manifest, which makes `file_exists`/`user_visible_artifact`/`artifact_json`
+   snapshots an `artifacts` manifest, which makes
+   `file_exists`/`user_visible_artifact`/`artifact_json`/`computer_links_resolve`
    replay-checkable — but the live-only egress keys stay skipped.) *Fix:* put egress/live-only checks on
    a live gate; keep one concern per `assert:` item; run the linter. The harness warns loudly on skip.
 
@@ -444,7 +447,8 @@ and the CI recipe, read the files in `references/`.
 - `references/scenario-schema.md` — scenario/session YAML schema, full assertion catalog (with each
   key's replay class), the web_fetch model, and the complete gotcha list.
 - `references/fidelity-and-answers.md` — fidelity tiers, answer paths, the determinism contract.
-- `references/ci-recipe.md` — replay-vs-live lane split and the four-stage GitHub Actions pipeline.
+- `references/ci-recipe.md` — the packaged GitHub Action, replay-vs-live lane split, and the four-stage
+  GitHub Actions pipeline.
 - `scripts/scenario.py` — `scaffold` a valid scenario skeleton and `lint` scenarios for the
   no-silent-false-green invariants (both usable as CI steps).
 - `../../../docs/run-status.md` — checking a background run's status without `ps aux`.

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,12 +1,59 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 0.21.0` (baseline `desktop-1.17377.2`).
+Self-contained reference. Tracks `cowork-harness 0.22.0` (baseline `desktop-1.18286.0`).
 
-**Minimal token-free PR gate** — the smallest thing worth committing; runs on stock GitHub-hosted runners,
-no token/Docker/agent:
+**Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
+job-summary reporter (verdict table, staleness findings, cost/turns when available):
 
 ```yaml
-- run: npm i -g cowork-harness@>=0.21.0
+- uses: yaniv-golan/cowork-harness@v1
+  with:
+    command: replay
+    path: cassettes/
+```
+
+Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
+cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
+independently instead of as one action run per command).
+
+**Live lane (`command: run`) with the packaged Action — self-hosted runner, you stage the binary.** The
+Action never downloads or provisions the agent ELF itself — by design, not an oversight (see "the
+realistic CI shape" below for why). Stage it yourself as an explicit step in *your own* workflow, then
+point the Action at it:
+
+```yaml
+jobs:
+  live:
+    runs-on: [self-hosted, linux, arm64]   # needs Docker + this staged ELF; not a stock GitHub-hosted runner
+    steps:
+      - uses: actions/checkout@v4
+      - name: Stage the agent binary (official channel, sha256-verified — see docs/maintenance.md)
+        run: |
+          V=2.1.197   # match your scenario's pinned baseline's agentVersion
+          curl -fSL "https://downloads.claude.ai/claude-code-releases/$V/linux-arm64/claude" -o "$RUNNER_TEMP/claude-$V"
+          chmod +x "$RUNNER_TEMP/claude-$V"
+          # verify against the committed baseline's sha256 (baselines/desktop-*.json → agentBinary.sha256)
+          # before trusting it — see docs/maintenance.md's "Agent-binary provenance" section.
+          echo "COWORK_AGENT_BINARY=$RUNNER_TEMP/claude-$V" >> "$GITHUB_ENV"
+      - uses: yaniv-golan/cowork-harness@v1
+        with:
+          command: run
+          path: scenarios/
+          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
+```
+
+Why this is a step *you* write, not an Action input the harness provides for you: pulling Anthropic's
+binary is a call about your own relationship with their distribution terms — keeping it in your own
+version-controlled workflow keeps that decision and its execution yours, auditable, and outside any
+third-party action's code. See the [agent-binary provenance runbook](../../../../docs/maintenance.md) for
+the full recovery/verification story (including why `COWORK_AGENT_BINARY` substitutions are
+sha256-*checked* but not hard-blocking on mismatch — it's advisory for an intentional substitution).
+
+**Minimal token-free PR gate (manual form)** — the smallest thing worth committing; runs on stock
+GitHub-hosted runners, no token/Docker/agent:
+
+```yaml
+- run: npm i -g cowork-harness@>=0.22.0
 - run: cowork-harness lint scenarios/*.yaml          # no silent false-greens
 - run: cowork-harness verify-cassettes cassettes/    # privacy + staleness
 - run: cowork-harness replay cassettes/              # token-free content/structure
@@ -22,7 +69,8 @@ The split is not just about tokens — it decides **where each lane can run**:
 - **`replay` / `verify-cassettes` (token-free, agent-free).** Replays a recorded cassette
   (`events.jsonl` + `control-out.jsonl`) and lints the committed cassettes. **No model tokens, no
   Docker, no agent binary** — runs on a stock GitHub Actions runner. Evaluates **content** assertions
-  only (`transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `result`, and the verdict
+  only (`transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `skill_triggered`,
+  `no_skill_triggered`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `result`, and the verdict
   modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
   `allow_stall` (no-op passes); plus the gate keys `question_asked` / `questions_count_max` /
   `gate_answers_delivered` **if** the cassette has `controlOut`). Filesystem/egress assertions are
@@ -110,6 +158,12 @@ A typical skill repo runs four stages, fastest/cheapest first:
 
 ## GitHub Actions sketch
 
+The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v1` does
+in one step (see the top of this doc) — reach for this form when you need independent per-command
+gating/annotations rather than one action run per command. The nightly live job has no packaged-Action
+equivalent yet (the Action's `command: run` mode needs a self-hosted runner with Docker + the agent binary
+already provisioned, same as the manual form below).
+
 PR gate (token-free — runs on every push):
 
 ```yaml
@@ -124,7 +178,7 @@ jobs:
         with: { node-version: '20' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g cowork-harness@>=0.21.0
+      - run: npm i -g cowork-harness@>=0.22.0
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -153,7 +207,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g cowork-harness@>=0.21.0
+        run: npm i -g cowork-harness@>=0.22.0
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 0.21.0` (baseline `desktop-1.17377.2`).
+Self-contained reference. Tracks `cowork-harness 0.22.0` (baseline `desktop-1.18286.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,7 +1,7 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.21.0`
-(baseline `desktop-1.17377.2`). If your checkout is newer, prefer the live `docs/scenario.md`,
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.22.0`
+(baseline `desktop-1.18286.0`). If your checkout is newer, prefer the live `docs/scenario.md`,
 `docs/session.md`, and `SPEC.md`.
 
 **Minimal scenario** — `prompt` is the only required field:
@@ -220,6 +220,12 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `subagent_dispatched: <regex>` | a sub-agent whose `agentType` **or dispatch description** matches |
 | `subagent_declared_but_unused: <Tool>` | a sub-agent declared the tool but never used **that** tool (even if it used others) |
 | `dispatch_count_max: <N>` | at most N sub-agents dispatched (records only — does NOT enforce a cap) |
+| `skill_triggered: <regex>` | a skill matching the regex (invoked id, e.g. `"plugin:skill"`) was invoked via the `Skill` tool — evidence-unavailable (not a normal fail) if the agent's init tools have no `Skill` tool |
+| `no_skill_triggered: <regex>` | no invoked skill id matched — the negative-control / description-collision catcher; evidence-unavailable (never a vacuous pass) if invocation data is absent or the `Skill` tool is unobservable |
+| `max_cost_usd: <N>` | the run's SDK-reported cost is ≤ N USD — evidence-unavailable if cost telemetry is absent. **Replay asserts the frozen recording's cost, not fresh spend** — a real regression needs a live `run` |
+| `max_tokens: <N>` | `usage.input_tokens + usage.output_tokens` ≤ N (cache tokens excluded) — same replay caveat as `max_cost_usd` |
+| `tool_calls_max: <N>` | total top-level tool calls (sum of `toolCounts`) ≤ N — meaningfully replay-checkable (re-drive recomputes `toolCounts` deterministically) |
+| `max_turns: <N>` | the SDK-reported (or fallback-counted) turn count ≤ N — meaningfully replay-checkable (re-drive recounts turns deterministically, same as `tool_calls_max`) |
 | `question_asked: <regex>` | the agent asked an AskUserQuestion whose text matches |
 | `questions_count_max: <N>` | the agent asked at most N questions |
 | `gate_answers_delivered: true` | every answered gate's answer reached the model (observed `tool_result`; unobserved = fail) |
@@ -232,6 +238,7 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `egress_denied: <host>` | the host was blocked by the egress proxy |
 | `egress_allowed: <host>` | the host was allowed through |
 | `artifact_json: {artifact, path, …}` | assert a JSON artifact's contents — `equals`/`gt`/`in`/`exists`/`absent`/`is_null` over a dotted `path` (`in` = membership in a list, for a stochastic/LLM value; `absent` ≠ `is_null`; an unresolved intermediate fails loud) |
+| `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). Zero links in the transcript **passes** (presence-gated separately — pair with `transcript_contains` if you also need a link to show up). **Only `true` is valid** (`false` is rejected by the schema) |
 
 `expect_denied: [host, …]` adds one `egress_denied` per host. Run `cowork-harness assertions --list` for this
 table from the live schema. Example: `artifact_json: { artifact: outputs/cap.json, path: me.run_id, equals: "r1" }`.
@@ -270,21 +277,26 @@ sourcing ≠ evaluation (replay warns when you edit one). `verify-run` is the on
 *run dir*; `--assert-from` is the equivalent for a *cassette*.
 
 **Evaluated on replay (content):** `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`,
-`result`. The verdict modifiers `allow_permissive_auto_allow` / `allow_missing_capability` /
-`allow_l0_plugin_divergence` / `allow_stall` are also kept on replay, evaluated as no-op passes.
+`skill_triggered`, `no_skill_triggered`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `max_turns`, `result`
+(`max_cost_usd`/`max_tokens` assert the frozen recording's spend on replay, not fresh spend). The verdict
+modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
+`allow_stall` are also kept on replay, evaluated as no-op passes.
 
 **Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `questions_count_max`,
 `gate_answers_delivered`. With `controlOut` present they evaluate; on an old cassette without it, a
 **loud warning** fires and they are **excluded** (not vacuously passed). Re-record to enable them.
 
 **Filesystem — replay-checkable WITH an artifact manifest:** `file_exists`, `user_visible_artifact`,
-`artifact_json` run on replay when the cassette carries an `artifacts` snapshot (`record` captures
-`outputs/` + connected folders; `replay` materializes it). `artifact_json` needs the small-file JSON `body`
-inlined; a hash-only entry still satisfies `file_exists`. Without a manifest (older cassettes) they're
-skipped. A green replay re-confirms *record-time* artifacts, not that the current skill still produces them
-— `replay --strict` fails when the staleness `fingerprint` shows ANY skill/baseline drift, or
-`replay --fail-on-skill-drift` only on skill-source drift; every replay result also reports it class-tagged
-in `staleness[]` for a JSON gate.
+`artifact_json`, `computer_links_resolve` run on replay when the cassette carries an `artifacts` snapshot
+(`record` captures `outputs/` + connected folders; `replay` materializes it). `artifact_json` needs the
+small-file JSON `body` inlined; a hash-only entry still satisfies `file_exists`. `computer_links_resolve`
+resolves a `/sessions/…/mnt/…`-shaped link directly against the manifest, and a host-shaped (hostloop) link
+by first normalizing it to a mount-relative path (recorded connected-folder prefixes + the outputs/uploads
+mounts) — replay has no live filesystem to check a host path against directly (that only happens on a live
+`run`/`verify-run`). Without a manifest (older cassettes) all four are skipped. A green replay re-confirms
+*record-time* artifacts, not that the current skill still produces them — `replay --strict` fails when the
+staleness `fingerprint` shows ANY skill/baseline drift, or `replay --fail-on-skill-drift` only on
+skill-source drift; every replay result also reports it class-tagged in `staleness[]` for a JSON gate.
 
 **Egress + other filesystem — still skipped on replay (live-only):** `no_delete_in_outputs`,
 `self_heal_ran`, `transcript_no_host_path`, `egress_*` / `expect_denied`. These run only on a live `run`/`record`.

package/.claude/skills/cowork-harness/scripts/assertion-keys.json

@@ -6,22 +6,29 @@
     "allow_permissive_auto_allow",
     "allow_stall",
     "artifact_json",
+    "computer_links_resolve",
     "dispatch_count_max",
     "egress_allowed",
     "egress_denied",
     "file_exists",
     "gate_answers_delivered",
+    "max_cost_usd",
+    "max_tokens",
+    "max_turns",
     "no_delete_in_outputs",
+    "no_skill_triggered",
     "question_asked",
     "questions_count_max",
     "replay_protocol_fidelity",
     "result",
     "self_heal_ran",
+    "skill_triggered",
     "subagent_declared_but_unused",
     "subagent_dispatched",
     "subagent_tool_absent",
     "subagent_tool_used",
     "tool_called",
+    "tool_calls_max",
     "tool_not_called",
     "tool_result_contains",
     "tool_result_not_contains",

package/.claude/skills/cowork-harness/scripts/scenario.py

@@ -59,12 +59,20 @@ CONTENT_KEYS = {
     "subagent_dispatched",
     "subagent_declared_but_unused",
     "dispatch_count_max",
+    "skill_triggered",
+    "no_skill_triggered",
+    "max_cost_usd",
+    "max_tokens",
+    "tool_calls_max",
+    "max_turns",
 }
 # content keys, but only evaluated on replay when the cassette carries controlOut
 GATE_KEYS = {"question_asked", "questions_count_max", "gate_answers_delivered"}
 # manifest-backed: replay-checkable when the cassette carries an `artifacts` manifest (record snapshots one);
 # a manifest-less cassette skips them. Since the 0.3.0 artifact-manifest these are NOT always live-only.
-MANIFEST_KEYS = {"file_exists", "user_visible_artifact", "artifact_json"}
+# computer_links_resolve joins this bucket (not CONTENT_KEYS): resolving a non-empty link set needs
+# either a live filesystem or the cassette's artifacts manifest — see cassette.ts's manifestKeys comment.
+MANIFEST_KEYS = {"file_exists", "user_visible_artifact", "artifact_json", "computer_links_resolve"}
 # live-only: ALWAYS skipped on replay, with a loud warning (no filesystem, no network on the token-free lane)
 LIVE_ONLY_KEYS = {
     "egress_denied",

package/CHANGELOG.md

@@ -6,6 +6,186 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [0.22.0] — 2026-07-03
+
+### Added
+
+- **`computer://` link modeling — the prompt now instructs file links exactly as production does.**
+  Four pieces landed together (design + binary research in
+  `docs/internal/2026-07-03-computer-link-scheme-research-and-plan.md`):
+  - `src/vm-paths.ts` — a faithful port of Desktop's display-side VM→host path transform
+    (`deepTranslateVMPaths` / `mapVMPathToHostPath` / `encodeComputerUrlsForHostLoop`): markdown-link,
+    backtick, bare-token, and prose rewrite positions; per-segment percent-encoding; traversal
+    rejection; dormant `.host-home` / `.auto-memory` mount branches.
+  - **Hostloop display translation** (`src/run/display-translate.ts`): at hostloop fidelity the
+    `run`/`chat` renderer shows production-identical host paths in assistant text and tool lines.
+    Hostloop-only by design (container staging paths would be less faithful than VM paths), identity on
+    replay (no live ctx), suppressed in `--compact`/`--demo` (the shareable no-host-paths contract).
+    Hostloop prompt tokens now render HOST paths (`{{cwd}}` / `{{workspaceFolder}}` / `{{skillsDir}}` +
+    the dedicated `{{cwd}}/mnt/uploads` pre-replacement), matching the Desktop builder's own host-loop
+    substitution recipe.
+  - **New assertion `computer_links_resolve: true`** — every `computer://` link in the transcript must
+    resolve to an artifact that exists (live/verify-run: filesystem; replay: the cassette's artifact
+    manifest, with host-shaped links normalized via the recorded session folders). Zero links pass;
+    dangling links report which target was checked. Assert links with this key, not literal link text.
+  - The `sharing_files` prompt section now instructs `[View your X](computer://{{workspaceFolder}}/x.ext)`
+    links faithfully — the prompt-reconstruction divergence for links is retired (docs/fidelity-gaps.md
+    updated).
+- **Dark-feature gate sentinels.** Four newly discovered GrowthBook gates pinned in the baseline
+  (host-fs skeleton `2614807392`, standard-session auto-memory `123929380`, memory-guidelines env
+  `1696890383`, memory extra-guidelines `2860753854`) — all dark or inert-default for standard cowork
+  sessions today; pinned so a production flip surfaces as a `sync --diff` delta. Absent-from-fcache
+  dark gates record an explicit `source:"absent"` marker (the fcache re-key guard's semantics are
+  preserved).
+
+- **`stats` command + cross-run result index.** Every `run`/`skill` invocation (and `record`'s live
+  execution) now appends one JSON line to `<runsRoot>/index.jsonl` at the same moment it writes
+  `result.json` — a durable, queryable history independent of whether the run dir itself survives a later
+  `prune`. `cowork-harness stats [<scenario>]` reads it back: run count, pass rate, cost/duration/token/turn
+  p50/p95, and the last-green timestamp, filterable by `--since`/`--baseline`/`--branch` and windowable by
+  `--last <n>` (per-scenario, not globally). `--reindex` rebuilds the index from the physical run-dir tree
+  — the migration path for runs that predate the index. `trace`/`inspect`/`scaffold`/`status`'s existing
+  run-id/fragment resolution now checks the index FIRST (faster, and the source of truth going forward),
+  falling through to the pre-index filesystem walk automatically and unchanged for any run that predates
+  the index or was never indexed — same commands, same output, same ambiguity-handling behavior either
+  way. See [docs/stats.md](./docs/stats.md).
+- **`run --matrix` matrix runner.** Runs one scenario across the cross-product of baseline/model/skill_dir
+  axes declared in a `matrix.yaml` file (any axis optional; an absent axis contributes one unmodified
+  cell) and reports one row per cell instead of a single pass/fail. `--max-cells` caps the cross-product
+  (default 16, warns and truncates rather than silently dropping cells); `--concurrency` (default 1, max 8)
+  runs cells N at a time via the same bounded pool `record --concurrency` uses. Exit is non-zero if ANY
+  cell fails — a real assertion failure or a cell-level infrastructure error (e.g. the pinned baseline's
+  agent binary isn't staged), rendered as a distinct `cell error: …` line rather than a fake assertion
+  failure. The `skill_dirs` axis substitutes the session's single `plugins.local_plugins` entry; candidates
+  must share that entry's directory basename (the mount name derives from it, with no author-chosen
+  override anywhere in the harness) — a mismatch is a loud, explicit usage error. `--concurrency > 1`
+  cannot combine with `--decider-dir`/`--decider-cmd` (the external decider channel is one shared object
+  across every cell, and every channel implementation is strictly serial over shared mutable state — not
+  safe for concurrent gate answers; `--concurrency 1`, the default, is genuinely serial and fine). The
+  JSON envelope gains an additive `matrix: {cells[]}` field; `ok`/the exit code are `!matrix.anyFail` for
+  this mode.
+- **`--matrix` composes with `--repeat`.** Each cell now runs as its own repeat batch (N iterations of that
+  cell's axes-overridden scenario) through the same `runRepeatBatch` helper standalone `--repeat` uses —
+  same unanswered-gate, error, and budget-cap handling — with `MatrixCellRepeatResult`/`MatrixRepeatRollup`
+  carrying each cell's full `RepeatRollup` (pass rate, per-assertion attribution, signal histogram,
+  stoppedEarly) rather than a single pass/fail. The matrix verdict judges each cell's rollup against
+  `--min-pass-rate`; the JSON envelope gains an additive `matrixRepeat: {cells[]}` field, checked before
+  `matrix`/`rollups` when present. Also closes the previously-ungated `--repeat` + `--decider-cmd`
+  combination (rejected for the same live-decider reasoning as `--decider-dir`).
+- **Packaged GitHub Action** (`uses: yaniv-golan/cowork-harness@v1`, [`action.yml`](./action.yml)) wrapping
+  `replay`/`lint`/`verify-cassettes`/`run` with a PR job-summary reporter (verdict table, staleness
+  findings, the skipped-live-only-assertions honesty line, cost/turns when available). Token-free lane runs
+  on any `ubuntu-latest` runner; `run` (live lane) needs a self-hosted runner with Docker + the agent binary
+  already provisioned — the action does not stage either, by design (staging Anthropic's binary is a call
+  about your own distribution-terms relationship, so it stays a step in your own workflow, not something a
+  third-party action automates for you). README and the companion skill's `ci-recipe.md` both carry a
+  worked self-hosted-runner example for the live lane. Self-tested in CI (`uses: ./` against a packed
+  tarball of the current commit, a passing case, a usage-error case, and a genuine assertion-failure case).
+  A `publish-image.yml` workflow pushes `ghcr.io/yaniv-golan/cowork-agent-base:2`/`cowork-agent-full:2` on
+  release tags for consumers (and this repo's own CI) to `docker pull` instead of building from scratch.
+- **`skill_triggered` / `no_skill_triggered` assertions.** Skill invocation (the top-level `Skill` tool_use)
+  is now a first-class assertable event, recorded as `RunResult.skillsInvoked[]` and evaluated as a regex
+  match, matching the `subagent_dispatched` convention. Fails as evidence-unavailable (never a vacuous pass)
+  when the agent's init tool list has no `Skill` tool (agent-version drift) or, for the negative form, when
+  invocation data itself is absent (an old run predating this key). Replay-checkable (content key).
+- **`max_cost_usd` / `max_tokens` / `tool_calls_max` / `max_turns` budget assertions**, built on Wave 0's
+  cost/turns seam. Each fails as evidence-unavailable (never a vacuous pass) when the underlying telemetry
+  is absent. `max_cost_usd`/`max_tokens` are honest about the replay lane: they assert the *frozen
+  recording's* spend, not fresh spend — a live `run` is where a real budget regression is caught.
+  `tool_calls_max`/`max_turns` are meaningfully replay-checkable (the re-drive recomputes `toolCounts`/turn
+  count deterministically).
+- **`diff <a> <b>` command.** Compares two committed platform baselines (`--changelog` renders known-field
+  prose — agent/Desktop version bumps, egress allowlist changes, gate flips — from a proper recursive
+  structural differ, replacing the old one-level diff that dumped a whole subtree on any nested change;
+  `sync --diff` now uses the same differ), two runs, two cassettes, or a run and a cassette (kind
+  auto-detected by CONTENT, not filename — a cassette-shaped file not literally named `*.cassette.json`
+  still detects correctly). Run/cassette mode has four views (`tools`/`transcript`/`artifacts`/`meta`, or
+  `all`) with normalization masking per-run noise (tool-use ids, UUIDs, session-dir markers, timestamps,
+  host paths) so two runs of the *same* scenario diff as identical despite that noise; `--no-normalize`
+  compares raw values for forensics. Comparing runs of two *different* scenarios is allowed (useful for
+  skill-variant comparison) but warns on stderr — added/removed rows may then reflect scenario
+  differences, not drift. Token-free — no live Desktop install or Docker needed either way.
+- **`run --repeat N` variance rollup.** Runs each resolved scenario N times (2-100) and aggregates a
+  rollup (pass rate, per-assertion pass/fail attribution, a verdict-signal histogram, cost/token totals,
+  non-deterministic-run count) instead of a single pass/fail. `--min-pass-rate` sets the batch threshold
+  (default 1.0 — no flakiness tolerance); `--stop-on-diverge` stops the loop as soon as both a pass and a
+  fail are observed (that batch always fails — divergence IS the failure being measured for);
+  `--max-budget-usd` stops the loop once cumulative cost would exceed it (an incomplete-but-clean stop is a
+  warning, not a failure by itself). `--repeat` rejects `--decider-dir`/`--decider-cmd` (an interactive
+  driving agent × N runs is not a measurement). The JSON envelope gains an optional `rollups[]` array;
+  `ok`/the exit code are
+  redefined for this mode from the rollups, not from `results.every(pass)` — `results[]` still holds every
+  raw run.
+- **E9: a hand-authored draft-07 JSON Schema for the harness's own control-channel wire protocol**
+  (`schema/protocol.v1.json`) — the `initialize` handshake, `can_use_tool` permission/question gates
+  (incl. AskUserQuestion's `questions[]`), `hook_callback`/`mcp_message` round-trips, and the nested
+  `control_response` envelope + the `answers` wire-shape — formalizing the prose in DESIGN.md §6/SPEC.md
+  §4-5. Deliberately does NOT schema the Claude Agent SDK's own event stream (Anthropic's surface).
+  Ships with a golden vector pack (`fixtures/protocol/v1/*.json` — real cassette-extracted where
+  possible, synthetic-via-the-real-`session.ts`-envelope-builders otherwise) and conformance tests
+  (`test/protocol-schema.test.ts`) that validate every committed cassette's control-channel lines plus
+  the real envelope-builder functions' actual output, and guard the schema/vector-pack lockstep. See
+  [docs/protocol.md](./docs/protocol.md) for the versioning policy and the explicit
+  descriptive-not-normative scope statement.
+
+### Parity
+
+- **Synced the platform baseline to Claude Desktop 1.18286.0** (`baselines/desktop-1.18286.0.json`).
+  The staged agent ELF is unchanged (`2.1.197`; measured sha256 matches the official release manifest).
+  `sync` re-derived egress/gates/mount/web_fetch facts — no unknown deltas; the egress allowlist (15
+  domains) and all 6 pinned GrowthBook gates held. `asarFingerprint` moved (`0b2f2fb6 → edff6926`);
+  the host-loop `## Shell access` generator and the subagent append were re-verified against the new
+  asar (unconditional fragments still byte-faithful).
+- **Re-authored the system-prompt append reconstruction for 1.18286.0**
+  (`baselines/prompts/desktop-1.18286.0/system-prompt-append.md`). The real append was RESTRUCTURED
+  at this release (constant `aui`, 37.9KB): a new `<claude_behavior>` wrapper plus new
+  behavior-driving sections — AskUserQuestion-before-work, task-list/verification, citation,
+  file-creation/computer-use guidance, web-fetch no-fallback restrictions, sharing/package/examples,
+  an `<env>` block — and the skills/file-handling sections moved inside `<computer_use>`. The
+  reconstruction (paraphrased per the no-bundling rule) now carries these; generic refusal/safety
+  policy stays elided, and the deliberate divergences (artifacts renderer catalog trimmed,
+  `computer://` links described-not-instructed) are logged in the asset header. New `<env>` tokens
+  `{{currentDateTime}}` / `{{currentTimezone}}` / `{{accountName}}` (session `account_name`,
+  default `"User"`) render in `src/prompt.ts`. New tests guard baseline→asset references, token
+  hygiene, and the `/sessions/` link-leak trade.
+- **Annotated `desktop-1.17377.2` as append-unverified** (`$comment_prompts_unverified`): its own
+  asar was never prompt-spot-checked and is no longer obtainable locally, so whether the 1.18286.0
+  restructure landed there or later is unverifiable; the pin carries the last-verified 1.15200.0
+  reconstruction.
+
+### Added
+
+- **Agent-binary provenance in baselines.** Each baseline's `agentBinary` now records the Linux/arm64 ELF's
+  `sha256` plus `shaProvenance` — `measured-local` (hashed from the staged binary at `sync` and cross-checked
+  against the official release manifest) or `official-manifest` (copied from the manifest for a version not
+  staged on the syncing machine; staging-identity unverified) — and, on measured rows, `manifestChecksumMatch`.
+  `sync` computes these and stays offline-capable (an unreachable manifest records `"unknown"` and never fails
+  the sync). All committed baselines back-filled. (No `nativeSha256`: the signed native Mach-O never equals a
+  manifest hash.)
+- **Default-on agent-ELF integrity check.** The resolved ELF is verified against the recorded `sha256` at run
+  time **by default** (opt out with `COWORK_HARNESS_VERIFY_AGENT_SHA=0`; ELF only). Hard-fails only on a
+  `measured-local` mismatch at the baseline's own staged path; intentional substitutions (`COWORK_AGENT_BINARY`
+  / newest-sibling fallback) and `official-manifest` hashes advisory-warn. `doctor` now shows a
+  `[sha256 ✓ vs baseline, …]` provenance line. Old agent versions are re-downloadable and verifiable — recovery
+  runbook in `docs/maintenance.md`.
+
+### Changed
+
+- **`RunResult.cost`/`.usage` retyped** from opaque `Record<string, unknown>` to structured shapes.
+  `cost` is now `{ usd?, raw? }`: `usd` is the SDK result message's `total_cost_usd`, newly extracted (was
+  previously dropped on the floor); `raw` is the pre-existing `api_metrics` payload, now nested under `raw`
+  instead of being the whole `cost` object. `usage` gains a `turns?: number` field, from the SDK's
+  `num_turns` (also newly extracted). Breaking shape change for anything reading `result.json`'s `cost`
+  field directly — see SPEC.md's `RunResult` reference for the new shape. No cassette-format bump (derived
+  reporting, not a stored format change).
+
+### Fixed
+
+- **Replay now surfaces `usage`/`cost` in `result.json`.** `replayCassette` previously omitted both fields
+  entirely from every replayed `RunResult`, regardless of what the cassette recorded — a replay-lane blind
+  spot, not a live-only limitation. Both are now re-derived from the cassette's re-driven record, same as
+  the live/partial-run lanes.
+
 ## [0.21.0] — 2026-07-03
 
 ### Added

package/DESIGN.md

@@ -38,7 +38,7 @@ flowchart TB
 A Cowork session is the Desktop app driving an agent that runs **inside an Apple Virtualization.framework microVM**:
 
 - VM bundle: `~/Library/Application Support/Claude/vm_bundles/claudevm.bundle/` (`rootfs.img`, `sessiondata.img`, `efivars.fd`, `machineIdentifier`, `gvisorMacAddress`, `vmIP`); a warm pool at `vm_bundles/warm/<sha>/`.
-- In-VM agent: `~/Library/Application Support/Claude/claude-code-vm/<ver>/claude` (currently **2.1.197**, an **ELF aarch64** binary), spawned by the host **in cowork mode via the `CLAUDE_CODE_IS_COWORK=1` env var** — *not* a `--cowork` flag (that flag is plugin-scope and the staged agent rejects it; see the control-protocol note below).
+- In-VM agent: `~/Library/Application Support/Claude/claude-code-vm/<ver>/claude` (currently **2.1.197**, an **ELF aarch64** binary), spawned by the host **in cowork mode via the `CLAUDE_CODE_IS_COWORK=1` env var** — *not* a `--cowork` flag (that flag is plugin-scope and the staged agent rejects it; see the control-protocol note below). Each baseline records this ELF's `sha256` (`agentBinary.sha256`/`shaProvenance`), and the resolver integrity-checks the binary it's about to run against that hash by default (opt out `COWORK_HARNESS_VERIFY_AGENT_SHA=0`), so "the same pinned agent" is enforced, not just asserted. Old versions are re-downloadable + verifiable from the official release channel (see `docs/maintenance.md`).
 - Network: `vm_network_mode: "gvisor"`, egress through a userspace netstack with a **compiled domain allowlist**; off-list partners rejected (`partner rejected: entry not on compiled allowlist`).
 - Control plane: Electron renderer→main typed IPC on channels named `$eipc_message$_<per-build-UUID>_$_claude.web_$_<Class>_$_<method>`, every handler validating `event.senderFrame.url` against a trusted-origin allowlist. The session manager is `LocalAgentModeSessions` (80 methods: `start`, `sendMessage`, `setDraftSessionFolders`, `onToolPermissionRequest`, `respondToToolPermission`, `getTranscript`, `onEvent`, …), bridged to the renderer as `window.cowork`.
 
@@ -163,6 +163,8 @@ L1 reproduces this as a **default-deny forward proxy**: the agent's `HTTP(S)_PRO
 
 The policy that produces those `allow`/`deny` responses is the **Decider** seam (see the architecture diagram); to smoke-test a decider against a sample question without a full run, use `cowork-harness decide`.
 
+> **Machine-readable form:** the five shapes below are schema'd as `schema/protocol.v1.json`, with a golden vector pack at `fixtures/protocol/v1/` — see [docs/protocol.md](./docs/protocol.md) for scope, versioning, and how to conformance-test against them.
+
 ### Control protocol — VERIFIED end-to-end against the live host CLI (macOS build 2.1.177+; the staged in-VM agent that L1/L2 run is 2.1.197, baseline `desktop-1.17377.1`)
 
 > The staged agent ELF is unchanged (2.1.181) across the 1.14271.0→1.15200.0 asar bump, and 2.1.187 across the 1.15200.0→1.15962.0 bump. The live scenario suite (`protocol` + `container` tiers) was re-run against the 1.15200.0 baseline; the 1.15962.0 bump was verified via asar analysis (content byte-identical: host-loop generator, system prompt, identity string, gates, and egress domains all unchanged) plus a full local test suite pass. The 1.15962.1→1.17377.1 bump moved the staged agent to **2.1.197** and added `api.claude.ai` to the egress allowlist; re-verified via `sync` (no unknown deltas) plus a manual asar spot-check of the reconstructed prompt content (substantively unchanged — see the Parity entry in CHANGELOG.md) and a full live scenario-suite pass (`protocol` + `container` tiers).