cowork-harness 2.4.0 → 3.0.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 2.4.0
-  tracks-harness: cowork-harness 2.4.0 (baseline desktop-1.37937.1)
+  version: 3.0.0
+  tracks-harness: cowork-harness 3.0.0 (baseline desktop-1.40609.0)
 ---
 
 # cowork-harness
@@ -25,8 +25,8 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 2.4.0` (baseline
-> `desktop-1.37937.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 3.0.0` (baseline
+> `desktop-1.40609.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## Preflight — make sure the harness can actually run
@@ -42,13 +42,13 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 2.4.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^2.4.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^2.4.0"`. **Pin `@^2.4.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 3.0.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^3.0.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^3.0.0"`. **Pin `@^3.0.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
   — upgrade rather than work around it, since this skill's `file:line` pointers and flag names track the floor.
 - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
-- **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
+- **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass. At `protocol` the plugin under test **is** delivered (`--plugin-dir`), but its hooks then run as native host processes — so a plugin declaring hooks is refused there until you pass `--allow-host-hooks` / `allow_host_hooks: true`.
 - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred), or `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
 - **`--dotenv` is a GLOBAL flag — put it BEFORE the subcommand.** `cowork-harness --dotenv .env record …`, never `cowork-harness record … --dotenv .env`. Every *other* flag is subcommand-level, so muscle memory fights this one; the harness rejects the misplaced form with an exact-fix error, but placing it first avoids the round-trip. **One exception: `critique` also accepts `--dotenv` per-command** (`critique <folder> --prompt "…" --dotenv <path>`) — available as of **1.6.0** (documented but unreachable before then); `--run-dir` stays global-only everywhere.
 
@@ -147,7 +147,7 @@ the folder basename (collision-resolved); there is no `to:` override. See `refer
 | Tier | What it gives you | Use when |
 |---|---|---|
 | `protocol` | Fastest; no sandbox, no egress | Pure protocol/answer-shape tests. **Rejected** if the scenario asserts egress. |
-| `container` | Real sandbox + real default-deny egress (**default**). Models the **VM loop**: keeps the built-in `Bash`, but `WebFetch` is replaced by `mcp__workspace__web_fetch` — assert on that name, not `WebFetch`. (`run`/`record` only; `chat --fidelity container` still offers the built-in.) **The name is tier-specific**: `microvm`/`protocol` never offer it, so the same `tool_not_called` is vacuous there — moving a scenario between tiers can silently void a web-fetch assertion | Most functional + boundary tests. |
+| `container` | Real sandbox + real default-deny egress (**default**). Models the **VM loop**: keeps the built-in `Bash`, but `WebFetch` is replaced by `mcp__workspace__web_fetch` — assert on that name, not `WebFetch`. (`run`/`record` only; `chat --fidelity container` still offers the built-in.) **The name is tier-specific**: `microvm` never offers it and `protocol` serves the operator's own host registry. A `tool_not_called`/`subagent_tool_absent` naming a tool its tier does not serve is **REFUSED at scenario load** (the message names what to write instead), so moving a scenario between tiers now errors rather than silently voiding the assertion — except at `protocol`, which is never judged | Most functional + boundary tests. |
 | `microvm` | VM-grade escape **isolation** (macOS arm64). Egress transport is the *same allowlist proxy as `container`* — not better network fidelity. Unlike `container`, still offers the built-in `WebFetch` | Testing untrusted code escape, not network behavior. |
 | `hostloop` / `cowork` | Production split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production), with **both** native `Bash` and `WebFetch` disabled and routed host-side via the workspace SDK-MCP server into a Docker VM sidecar (the VM loop replaces web_fetch only) | Highest-fidelity / parity runs. A writable connected folder needs `allow_host_writes: true` (see scenario-schema.md). |
 
@@ -679,7 +679,7 @@ than a stuck `"running"`.)
 ### Place assertions in the right CI lane
 
 CI placement: a **token-free `replay` PR gate** (content/structure only) + a **nightly live `run`**
-(filesystem/egress). Fastest setup: `uses: yaniv-golan/cowork-harness@v2` (a packaged GitHub Action with a
+(filesystem/egress). Fastest setup: `uses: yaniv-golan/cowork-harness@v3` (a packaged GitHub Action with a
 PR job-summary reporter). See `references/ci-recipe.md` for the Action, the manual step-by-step form, and
 the four-stage pipeline.
 

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,23 +1,23 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`).
+Self-contained reference. Tracks `cowork-harness 3.0.0` (baseline `desktop-1.40609.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
 
 ```yaml
-- uses: yaniv-golan/cowork-harness@v2
+- uses: yaniv-golan/cowork-harness@v3
   with:
     command: replay
     path: cassettes/
-    version: "^2"              # hold the major; see below
+    version: "^3"              # hold the major; see below
 ```
 
-**These recipes pin `version: "^2"`.** The Action's `version` input *defaults* to `latest`, which means a
+**These recipes pin `version: "^3"`.** The Action's `version` input *defaults* to `latest`, which means a
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
 copy-pasted recipe that omits the input takes a major bump with no say in it. `^2` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "2.4.0"`) instead when you want byte-reproducible CI.
+(e.g. `version: "3.0.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -36,7 +36,7 @@ jobs:
       - uses: actions/checkout@v4
       - name: Stage the agent binary (official channel, sha256-verified against the pinned baseline)
         run: |
-          V=2.1.246   # match your scenario's pinned baseline's agentVersion
+          V=2.1.247   # match your scenario's pinned baseline's agentVersion
           # The expected digest is baselines/desktop-<ver>.json -> agentBinary.sha256. Paste it here, or
           # read it with jq if you vendor the baseline. An unverified download is an unverified agent:
           # this step FAILS rather than staging one, which is the whole point of naming it "verified".
@@ -47,11 +47,11 @@ jobs:
           echo "COWORK_AGENT_BINARY=$RUNNER_TEMP/claude-$V" >> "$GITHUB_ENV"
           # Background on the provenance chain: the "Agent-binary provenance" section of
           # https://github.com/yaniv-golan/cowork-harness/blob/main/docs/maintenance.md
-      - uses: yaniv-golan/cowork-harness@v2
+      - uses: yaniv-golan/cowork-harness@v3
         with:
           command: run
           path: scenarios/
-          version: "^2"
+          version: "^3"
           anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
 ```
 
@@ -67,7 +67,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^2.4.0"
+- run: npm i -g "cowork-harness@^3.0.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -119,18 +119,18 @@ Action has no input for, and it creates a coupling nothing checks:
 you have a reason:
 
 ```yaml
-- uses: yaniv-golan/cowork-harness@v2
+- uses: yaniv-golan/cowork-harness@v3
   with:
     command: lint
     path: scenarios/
-    version: "^2"                       # holds the major
-    extra-args: --min-severity WARN     # needs a CLI >= 1.11.0; any 2.x satisfies that
+    version: "^3"                       # holds the major
+    extra-args: --min-severity WARN     # needs a CLI >= 1.11.0; any 3.x satisfies that
 ```
 
 **If a flag you pass in `extra-args` landed in a specific release, bound the range — don't write a bare
 floor.** `>=1.11.0` reads as "at least 1.11.0" and silently means "and every future major too", so a
 recipe written that way hands a copy-paster the next major with no say in it. Anchor it at the current
-major instead — `version: "^2"`, which is what the steps above use — keeping the floor's intent while
+major instead — `version: "^3"`, which is what the steps above use — keeping the floor's intent while
 stopping at the major boundary. An exact
 pin (`version: "2.0.1"`) is the right choice when you want byte-reproducible CI, at the cost of rotting the
 moment a recipe adopts a newer flag.
@@ -147,7 +147,7 @@ The split is not just about tokens — it decides **where each lane can run**:
   Docker, no agent binary** — runs on a stock GitHub Actions runner. Evaluates **content** assertions —
   `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `skill_triggered`, `no_skill_triggered`,
   `max_cost_usd`, `max_tokens`, `tool_calls_max`, `result`, and the verdict modifiers
-  `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
+  `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_host_config_contamination` /
   `allow_stall` (no-op passes); plus the gate keys `question_asked` / `question_options` /
   `question_context` / `questions_count_max` / `gate_answers_delivered` **if** the cassette has `controlOut`, and the manifest keys
   (`file_exists` / `user_visible_artifact` / `artifact_json` / `artifact_text`) **if** it carries an artifact
@@ -322,7 +322,7 @@ A typical skill repo runs four stages, fastest/cheapest first:
 
 ## GitHub Actions sketch
 
-The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v2` does
+The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v3` does
 in one step (see the top of this doc) — reach for this form when you need independent per-command
 gating/annotations rather than one action run per command. The nightly live job has no packaged-Action
 equivalent yet (the Action's `command: run` mode needs a self-hosted runner with Docker + the agent binary
@@ -342,7 +342,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^2.4.0"
+      - run: npm i -g "cowork-harness@^3.0.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -371,7 +371,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^2.4.0"
+        run: npm i -g "cowork-harness@^3.0.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`). This is **not** a trim of the full
+Tracks `cowork-harness 3.0.0` (baseline `desktop-1.40609.0`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reproduction discipline, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's
@@ -34,6 +34,63 @@ computed over raw rows** — that exclusion also governs `stats --group-by skill
 **total spend** still needs the raw rows (only the roll-up carries the evaluator passes); and a roll-up with `result:"error"` had an unpriced workload, so its totals
 UNDERCOUNT. The index is the only cost record that survives run-dir pruning.
 
+## An exit-2 report — which turn failed, and whether it was really infrastructure
+
+Exit 2 means no findings were produced, so the report's only job is to say what went wrong. Three fields
+carry that; read all three before touching anything.
+
+| Field | Meaning |
+|---|---|
+| `infraFailure` | the reason |
+| `infraFailurePhase` | `task turn` (the graded run) or `reflection turn` (critique's own protocol turn) |
+| `infraFailureKind` | why it failed — a harness `ErrCategory` (error envelope, exit 2/3) **or** a `resultErrorKind` (`usage_limit`/`transport`/`agent`) from a turn that RAN and errored (exit 1, top-level `error: null`). **Absent** = killed, or no envelope |
+| `gradedErrorReason` | on a `taskResult: "error"` run (still gradeable, exit 0): why the GRADED turn errored, so a quota exhaustion is not read as a skill defect |
+
+**Do NOT read "has a kind" as "the instrument is fine".** The CLI's top-level catch turns every
+unexpected throw into category **`internal`** — Docker down, container start failure, missing staged
+agent, harness bug — and `runtime` carries a refused run dir. Only **`unanswered`, `usage`, `boundary`**
+and, from the result-row taxonomy, **`usage_limit`** (quota exhausted — retry after reset) and
+**`transport`** (a tail-end drop) are the caller's problem. `agent` is not: for critique's own protocol
+turn that IS the instrument breaking. The header encodes exactly that split and fails closed (an
+unrecognized kind renders as infrastructure):
+
+- `RUN FAILED (<turn>, <kind>): …` → ordinary, actionable, instrument healthy.
+- `INFRASTRUCTURE/PROTOCOL FAILURE (<turn>): …` → `internal`/`runtime`/`agent`/unknown/killed/no-envelope.
+
+**A turn that exits 1 is not a crash.** It RAN and reported an errored result, with `error: null` and a
+full `results[0]` — an exit-code-only reading of that path is what leaves an exhausted quota looking like
+a broken instrument.
+
+**Read the reason, not the category.** The reason carries the failed turn's own message *and* hint
+verbatim. That matters most for `unanswered`, which is 36 distinct throw sites and only ONE of them is
+"the skill asked an unscripted question" — the others are a mis-typed `--answer` label, malformed
+`--answer-policy` YAML, a crashed or bad-JSON `--decider-cmd` helper, an out-of-set `--decider-llm`
+reply, an unanswered dialog/elicit, even a self-declared harness bug. A remedy picked from the category
+is wrong for nearly all of them; each site's own hint is written for its case. (Also note `--on-unanswered`
+*conflicts* with `--decider-dir`/`--decider-cmd`, so it is not a blanket fallback.)
+
+For the genuine unscripted-gate case: script it (`--answer`, `--answer-policy`), or, when the skill's
+gates are LLM-authored and reworded every run so a literal regex will not match twice, use `--decider-llm`
+(or the scenario's `on_unanswered: llm`).
+
+## Which model was graded — `gradedModels`
+
+**The two turns are a SUBPROCESS.** They inherit no model from whatever invoked `critique` — not your
+session, not a project setting. With no `--model`, the graded run uses the spawned agent's own default,
+which may not be the model you are otherwise working under, and nothing about the run announces it.
+
+`gradedModels` (text header: `graded model(s):`) is read back from the graded turn's own `result.json` and
+is the only record of which model produced the behaviour being graded — distinct from the evaluator's
+resolved model, which is a **different workload with its own default** (`claude-opus-4-8`), reported
+separately. An evaluator line naming a model you did not pass is therefore expected, not evidence your
+`--model` was ignored. Pin with `--model <id>` whenever a critique will be compared against another, and
+read `gradedModels` back to confirm it took.
+
+It is **observed, not requested** — the ids come from the model stamped on the graded turn's assistant
+messages, never from the flag. So `graded model(s): unknown` means no assistant message reached the run
+(crash, kill, or a gate before the first reply); passing `--model` does not change that line. Past runs
+can be checked without re-running: the same ids are in each kept run dir's `turns/1/result.json`.
+
 ## The report's item shape — no `title`, no `summary`
 
 Each `items[]` entry's prose fields are **`idea`** and **`recommendedAction`** — there is no `title` field
@@ -80,6 +137,20 @@ false `already-covered` verdict. It is named in `corpusExcluded` instead, and an
 specifically reports `skillMdStatus: "untracked"`, forcing the mechanical `already-covered` →
 `not-adjudicable` downgrade. `git add` it (or commit before critiquing) if it should count as evidence.
 
+## Read `referencesAccessed`, not `referencesRead`
+
+`referencesRead` counts the **`Read` tool only**. An agent that reaches a reference with a `Bash cat`, a
+`Grep` or a `Glob` leaves nothing in it, so its emptiness is **not** evidence the content went unread.
+`referencesAccessed` is the wide signal — every file reached, with the channel each was reached through
+(`read` / `grep` / `bash`) — and it is what the critique headline is computed from.
+
+Two properties to carry: only the `read` channel is strong evidence the agent opened the file (a `bash`
+entry means a command named the path); and detection **under-approximates** — a `cd` into the skill dir
+then a bare relative `cat`, a heredoc body and a `$VAR`-built path are all invisible. So an absent path is
+weak evidence, never proof. **Presence is the cannot-verify channel:** `[]` means the drive ran and saw
+nothing (a real negative); an ABSENT field means there was no observable drive, and must never be read as
+"none".
+
 ## `referencesRead` is main-agent-only — `noSkillFilesRead` is not
 
 `result.json`'s top-level `referencesRead` lists **main-agent Reads only**. A dispatcher-style skill does

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`).
+Self-contained reference. Tracks `cowork-harness 3.0.0` (baseline `desktop-1.40609.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 
@@ -20,6 +20,11 @@ Self-contained reference. Tracks `cowork-harness 2.4.0` (baseline `desktop-1.379
 - A `hostloop` scenario with a **writable** connected folder (`mode: rw`/`rwd`) needs `allow_host_writes:
   true` — with no container around the native file tools, that combination gives the agent genuine,
   software-checked-only host filesystem access. Read-only folders and folder-less runs need no opt-in.
+- A `protocol` scenario staging a plugin that declares runnable hooks needs `allow_host_hooks: true`
+  (`--allow-host-hooks` on `chat`/`skill`). L0 passes `--plugin-dir`, so the CLI runs those
+  hooks as native host processes under your account with no sandbox. Plugins without hooks need no opt-in.
+  Needs cowork-harness >= 3.0.0 — an older CLI rejects the key outright (`Unrecognized key`, exit 2), it
+  does not fall back to the default.
 - **Set the tier in the scenario's `fidelity:` field — not a flag.** `--fidelity` is accepted only by
   `skill` (any tier) and `chat` (`protocol`/`container`/`hostloop`; only `microvm`/`cowork` unsupported); `run` rejects an extra `--fidelity`
   positional ("Fidelity is set by the scenario's `fidelity:` field, not a flag").

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,7 +1,7 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 2.4.0`
-(baseline `desktop-1.37937.1`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 3.0.0`
+(baseline `desktop-1.40609.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 
 **Minimal scenario** — `prompt` is the only required field:
@@ -98,6 +98,17 @@ allow_host_writes: true             # OPTIONAL — required to run `hostloop` fi
                                     # around hostloop's native file tools, that combination gives the
                                     # agent genuine, software-checked-only host filesystem access.
                                     # Read-only folders and folder-less runs need no opt-in.
+
+allow_host_hooks: true              # OPTIONAL — required to run `protocol` fidelity when a staged plugin
+                                    # declares runnable hooks (`<plugin>/hooks/hooks.json`): L0 passes
+                                    # --plugin-dir, so the CLI executes those hooks as NATIVE HOST
+                                    # processes under your account, with no container sandbox. A plugin
+                                    # that declares no hooks needs no opt-in, and a misplaced root-level
+                                    # `hooks.json` cannot execute so it does not trigger the gate.
+                                    # Use `--fidelity container` to run them sandboxed instead.
+                                    # NEEDS cowork-harness >= 3.0.0. The loader is a strict object, so an
+                                    # OLDER CLI does not default it — it hard-errors
+                                    # `Unrecognized key: "allow_host_hooks"` and exits 2.
 ```
 
 Relative paths resolve from the file's own directory, so a scenario + session + referenced files
@@ -305,6 +316,10 @@ same set live from the schema.
 | `no_lost_write_back: true` | fails if the run authored an interactive HTML artifact (or a `.py`/`.js` generator of one) whose **relative** Submit/POST write-back is lost under Cowork (served from Cowork's own origin → resolves non-ok, a "Saved!" is silently false). Runs the shipped **static Tier A** analyzer over the files the run authored (diffed vs the pre-run manifest). A lost write-back on an **added** agent-authored source (`outputs/`, scratchpad) **fails**; a **pre-existing** file the skill only modified on a read-write mount is **advisory**; `-suspect` findings surface but pass. **Only `true` is valid**. **Live/verify-run only** — skipped-loud on replay; runs on every live sandbox tier including **microvm** (its outputs are snapshotted from the VM into the run dir); could-not-verify (fail-closed) on a `--resume` scratchpad or an unanalyzable candidate |
 | `tool_called: <glob>` | a tool the agent ran matched this **glob** — `*` = any run, `?` = one char, exact when literal, anchored + case-sensitive. Exact name (`Write`) matches only that tool; `mcp__workspace__*` matches any workspace tool. GLOB, not regex (`.` is literal) — an empty glob, or one containing a regex/brace-expansion metacharacter (`.*`, `.+`, `\|`, `()`, `[]`, `+`, `^`, `$`, `{}`, `\d`/`\w`/`\s`/`\b`), is **rejected at load** (a hard schema error, not a runtime warning) — it would match no real tool name and pass a `_not_`/`_absent` assert vacuously. Applies whether the glob comes from an authored scenario or a recorded cassette's frozen assert. The bundled `scenario.py lint` does NOT perform this check — only the harness enforces it, at actual load (`run`/`skill`/`record`) |
 | `tool_not_called: <glob>` | NO tool the agent ran matched this glob (`mcp__*` = "no MCP tool ran"). Same glob semantics as `tool_called`, including the empty/regex-ish rejection |
+| **legacy tool names** | The agent binary canonicalizes legacy spellings (`Task`→`Agent`, `KillShell`/`KillBash`→`TaskStop`, …) while the spawn tool list still declares the LEGACY one — so the init inventory shows `Task` and every call is emitted as `Agent`. `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` match EITHER spelling, globs included. Before this, `tool_called: "Task"` could never pass and `tool_not_called: "Task"` always did. |
+| **tier vacuity — now REFUSED** | `tool_not_called` **and `subagent_tool_absent`** naming a tool the tier does not serve is rejected at scenario load (`Bash`/`WebFetch`/`NotebookEdit` at `hostloop`; `mcp__workspace__bash` at `container`/`microvm`), because it could never be violated. `scenario.py lint` WARNs on the same set before any run. The table is CLOSED to tools the harness itself removes or registers — `--tools` gates only the built-in set while every tier separately passes `--mcp-config`, so a session-MCP tool name is offered without appearing in any tool list and is never refused. Globs are never refused. `subagent_tool_absent` is covered because it reads the tools sub-agents actually USED, not a per-dispatch declared list. Under-approximates on purpose: `REPL` at hostloop is vacuous too and is not caught, which is the right side to err on for a hard refusal. |
+| `reference_read: <regex>` | a skill `references/`/`scripts/` file whose path matches this **regex** was ACCESSED — main agent or sub-agents, via `Read`, `Grep`/`Glob` (`path` input), or a `Bash`/`mcp__workspace__bash` command naming the path. Regex is **unanchored + case-insensitive** (the shared helper every regex key uses). **Under-approximates by design:** the path must be rooted in the mounted plugin, so a `cd` into the skill dir then a bare `cat references/x.md`, a heredoc body, and a `$VAR`-built path are invisible. Fails **evidence unavailable** when the run recorded no observable tool stream. Replay-capable (cassettes freeze whole tool inputs) |
+| `no_observed_reference_access: <regex>` | no OBSERVED access matched the regex — the progressive-disclosure check: a reference the skill's routing never reaches. Named `observed` because detection under-approximates (see above), so it is **not proof the file went unread** — an agent that `cd`s and `cat`s it passes. Fails **evidence unavailable** rather than passing vacuously when no observable tool stream was recorded |
 | `tool_result_contains: <str>` | a tool result includes the literal string (content / replay-checkable — substring match) |
 | `tool_result_not_contains: <str>` | no tool result includes the literal string (content / replay-checkable; fails loud when tool results are absent) |
 | `tool_result_matches: <regex>` | the regex sibling of `tool_result_contains` — case-insensitive regex matches at least one tool result; use for an error-signature family, not just one literal string |
@@ -358,7 +373,7 @@ same set live from the schema.
 | `no_path_denied: true` | **`fidelity: hostloop` only** — NO path denial was recorded at all (the channel is already path-scoped, unlike `no_hook_blocked`'s indiscriminate reject). Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify". **Only `true` is valid** |
 | `allow_permissive_auto_allow: true` | verdict modifier — suppresses the default-fail when the run recorded a cowork-parity permissive auto-allow; for tests that deliberately assert Cowork's permissive behavior |
 | `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
-| `allow_l0_plugin_divergence: true` | verdict modifier — opt into L0/protocol plugin divergence: suppresses the default-fail when a plugin behaves differently at `protocol` (L0) fidelity than under a sandboxed tier. Live tiers only |
+| `allow_l0_host_config_contamination: true` | verdict modifier — accept a contaminated L0 environment: suppresses the default-fail when `protocol` runs against your REAL config dir, where your installed plugins/skills/auto-memory/MCP servers are visible and may answer instead of the thing under test. Live tiers only |
 | `allow_stall: true` | verdict modifier — suppresses the `stalled` default-fail when a run ends on a question having done no productive tool work after its last gate (the agent asked for input and stopped — incl. re-asking in plain text after answering an `AskUserQuestion`); assert only when ending on a question is intended, else script the answer (`answer:` / `--answer` / a decider). **Scenario-only:** an open-ended `skill` run has no `assert:` block, so it cannot opt out — a helpful closing offer ("want me to run this through a structured pass?") fails `stalled` there with no suppressor. Read the final message before believing it, or move the check to a `run` scenario |
 | `allow_undelivered_deliverables: true` | verdict modifier — suppresses the `undelivered_deliverables` WARN. Working in the scratchpad is Cowork's designed pattern, so a skill that legitimately leaves intermediates, caches or downloaded inputs behind can say so instead of carrying permanent noise. The signal is warn-only and never fails a run on its own; reach for this when the scratch activity is intentional, not to silence a real delivery gap |
 | `allow_outputs_delete: true` | verdict modifier — accepts a detected outputs delete instead of failing the run, for a skill whose deletion is intended. Omitting `no_delete_in_outputs` does **not** permit deletes (a detected delete fails via the `outputs_delete` signal precisely because the key was not authored), so this is the way to accept one. **Mutually exclusive** with `no_delete_in_outputs`. Waives the harness's post-hoc detection; it does not model Cowork's `allow_cowork_file_delete` approval handshake |
@@ -400,7 +415,7 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 | `outputs_delete` | fail | An unauthorized delete touched `mnt/outputs` (opt out: author `no_delete_in_outputs`) |
 | `mount_delete` | warn | A delete touched a delete-denied mount other than `outputs` (a `rw` connected folder). Production denies `unlink`/`rmdir` there until per-mount approval, so the run diverged. Warn, not fail: the harness detects post-hoc what production enforces. Author `no_delete_in_mounts` to hard-fail, or `allow_delete_in` to waive |
 | `host_path_leak` | fail | A host path leaked into model-visible text (opt out: author `transcript_no_host_path`) |
-| `l0_plugin_divergence` | fail | L0/protocol plugin loading diverged from Cowork (opt out: `allow_l0_plugin_divergence`) |
+| `l0_host_config_contamination` | fail | `protocol` ran against the operator's REAL config dir, so host-installed plugins/skills/memory/MCP may have answered instead of the thing under test — the run did not necessarily measure it (opt out: `allow_l0_host_config_contamination`). Name predates the meaning: delivery at L0 works, contamination is what this reports |
 | `missing_capability` | fail | A `requires_capabilities` need was unmet, or the skill used a capability the image omits (opt out: `allow_missing_capability`, or `skill --allow-missing-capability` on an open-ended run) |
 | `infra_error` | fail | A supervising process died mid-run (VM/egress sidecar) — the run's evidence is contaminated, not author-suppressible |
 | `stalled` | fail | The run ended on an unanswered question, or on a trailing-`?` final turn with no tool work after the last gate (opt out: `allow_stall`) |
@@ -439,7 +454,7 @@ sourcing ≠ evaluation (replay warns when you edit one). `verify-run` is the on
 `skill_tool_used`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `tool_no_error`,
 `max_tool_errors`, `max_redundant_tool_calls`, `max_turns`, `compaction_occurred`, `all_tasks_completed`, `task_status`, `task_count_min`, `no_scratchpad_leak`, `present_files_called`, `result`
 (`max_cost_usd`/`max_tokens` assert the frozen recording's spend on replay, not fresh spend). The verdict
-modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
+modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_host_config_contamination` /
 `allow_stall` are also kept on replay, evaluated as no-op passes.
 
 **Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `question_options`, `question_context`, `questions_count_max`,

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 3.0.0` (baseline `desktop-1.40609.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.
 

package/.claude/skills/cowork-harness/scripts/assertion-keys.json

@@ -3,7 +3,7 @@
   "keys": [
     "all_tasks_completed",
     "allow_delete_in",
-    "allow_l0_plugin_divergence",
+    "allow_l0_host_config_contamination",
     "allow_missing_capability",
     "allow_outputs_delete",
     "allow_permissive_auto_allow",
@@ -35,6 +35,7 @@
     "no_hook_blocked",
     "no_lost_write_back",
     "no_mcp_error",
+    "no_observed_reference_access",
     "no_path_denied",
     "no_scratchpad_leak",
     "no_skill_triggered",
@@ -46,6 +47,7 @@
     "question_context",
     "question_options",
     "questions_count_max",
+    "reference_read",
     "replay_protocol_fidelity",
     "result",
     "self_heal_ran",
@@ -81,6 +83,7 @@
     "vm_path_denied"
   ],
   "topLevelKeys": [
+    "allow_host_hooks",
     "allow_host_writes",
     "answers",
     "assert",
@@ -99,7 +102,7 @@
   ],
   "verdictModifierKeys": [
     "allow_delete_in",
-    "allow_l0_plugin_divergence",
+    "allow_l0_host_config_contamination",
     "allow_missing_capability",
     "allow_outputs_delete",
     "allow_permissive_auto_allow",

package/.claude/skills/cowork-harness/scripts/scenario.py

@@ -80,6 +80,10 @@ CONTENT_KEYS = {
     "tool_result_not_matches",
     "tool_called",
     "tool_not_called",
+    # Replay re-derives these from the SAME frozen tool inputs the live run used (a cassette stores whole
+    # tool inputs), so they are content keys, not live-only.
+    "reference_read",
+    "no_observed_reference_access",
     "subagent_tool_used",
     "subagent_tool_absent",
     "subagent_dispatched",
@@ -169,7 +173,7 @@ LANE_REMOTE_INCOMPATIBLE_KEYS = {"present_files_called", "no_scratchpad_leak", "
 # verdict modifiers — don't verify anything themselves (e.g. suppress a default-fail)
 VERDICT_MODIFIER_KEYS = {
     "allow_permissive_auto_allow",
-    "allow_l0_plugin_divergence",
+    "allow_l0_host_config_contamination",
     "allow_missing_capability",
     "allow_stall",
     "allow_undelivered_deliverables",
@@ -258,7 +262,8 @@ _EMBEDDED_TOP_LEVEL_KEYS = {
     "assert",
     "skills",  # opt-in skill-staleness hash scope
     "requires_capabilities",  # Fix 4b: scenario-level required-capability declaration (pre-flight gate)
-    "allow_host_writes",  # hostloop native-split: consent for a writable connected folder (pre-run gate)
+    "allow_host_writes",
+    "allow_host_hooks",  # protocol consent: a staged plugin's hooks run as NATIVE HOST processes  # hostloop native-split: consent for a writable connected folder (pre-run gate)
 }
 
 
@@ -294,6 +299,8 @@ REGEX_KEYS = {
     "hook_blocked",
     "tool_result_matches",
     "tool_result_not_matches",
+    "reference_read",
+    "no_observed_reference_access",
 }
 VALID_ON_UNANSWERED = {"fail", "prompt", "first", "llm"}
 VALID_TIERS = ("protocol", "container", "microvm", "hostloop", "cowork")
@@ -583,6 +590,44 @@ def lint_doc(doc, path, raw_lines):
     # warns at run start, after authoring). Lint is deliberately STRICTER than the runtime: the docs
     # declare the combination incompatible, so authoring it is a bug even if a tool-free run could
     # accidentally pass. `cowork` gets a WARN naming the baseline-gate resolution dependency (the
+    # `tool_not_called` naming a tool the TIER does not serve can never be violated — it passes
+    # vacuously and verifies nothing. Expressible offline because the mapping is a harness constant
+    # (WORKSPACE_TOOL_ALIASES / VM_LOOP_TOOL_ALIASES), NOT a baseline read. Deliberately literals only,
+    # and deliberately a closed table: `--tools` gates the BUILT-IN set alone while every tier separately
+    # passes --mcp-config, so a session-MCP tool name is offered without appearing in any tool list and
+    # must never be flagged here. The harness refuses these at load; this catches them before any run.
+    _TIER_VACUOUS = {
+        "hostloop": {"Bash": "mcp__workspace__bash", "WebFetch": "mcp__workspace__web_fetch", "NotebookEdit": None},
+        "container": {"mcp__workspace__bash": "Bash"},
+        "microvm": {"mcp__workspace__bash": "Bash", "mcp__workspace__web_fetch": "WebFetch"},
+        # `protocol` absent on purpose: it passes no tool flags, so its surface is the operator's own
+        # host CLI registry — machine-dependent and about a different product.
+    }
+    # BOTH negative tool keys: `subagent_tool_absent` is judged against the tools sub-agents actually
+    # USED, not a per-dispatch declared list, so a tool the tier never serves makes it equally vacuous.
+    for _key in ("tool_not_called", "subagent_tool_absent"):
+        for _v in _assert_values(items, _key):
+            if not isinstance(_v, str) or "*" in _v or "?" in _v:
+                continue  # a glob is not a literal claim about one tool
+            _repl = _TIER_VACUOUS.get(fidelity, {})
+            if _v not in _repl:
+                continue
+            _instead = _repl[_v]
+            findings.append(
+                Finding(
+                    "WARN",
+                    "tool-not-called-tier-vacuous",
+                    f"`{_key}: {_v}` on `fidelity: {fidelity}` — that tier does not serve "
+                    f"`{_v}` at all, so this can never be violated and verifies nothing.",
+                    (
+                        f"Assert `{_key}: {_instead}` instead — that is what the tier serves in its place."
+                        if _instead
+                        else f"The {fidelity} tier removes `{_v}` outright; drop this assertion."
+                    ),
+                    path,
+                )
+            )
+
     # linter stays offline — the message carries the gate fact instead of reading a baseline).
     if "transcript_no_host_path" in assert_keys:
         if fidelity in ("hostloop", "protocol"):
@@ -1016,6 +1061,26 @@ def lint_doc(doc, path, raw_lines):
             )
         )
 
+    # Same shape for the reference-access pair: `reference_read: R` and `no_observed_reference_access: R`
+    # with the IDENTICAL regex cannot both hold. Compared as raw pattern strings — two different regexes
+    # that happen to match the same path are NOT a contradiction (the linter cannot know the paths), so
+    # this only fires on the case that is unambiguously self-defeating.
+    read_pats = {v for v in _assert_values(items, "reference_read") if isinstance(v, str)}
+    unread_pats = {v for v in _assert_values(items, "no_observed_reference_access") if isinstance(v, str)}
+    both_refs = sorted(read_pats & unread_pats)
+    if both_refs:
+        findings.append(
+            Finding(
+                "ERROR",
+                "reference-access-contradiction",
+                f"assert requires {both_refs} to be both accessed (`reference_read`) and never observed "
+                "(`no_observed_reference_access`) — no run can satisfy that, so this would spend a run to fail.",
+                "Drop whichever half the scenario does not mean. To check that ONE reference is reached while "
+                "another is not, give the two keys different patterns.",
+                path,
+            )
+        )
+
     # W: double-quoted regex with a backslash (raw-text scan — the parser already ate it)
     findings.extend(_lint_regex_quoting(path, raw_lines))