cowork-harness 2.3.0 → 2.4.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 2.3.0
-  tracks-harness: cowork-harness 2.3.0 (baseline desktop-1.37937.1)
+  version: 2.4.0
+  tracks-harness: cowork-harness 2.4.0 (baseline desktop-1.37937.1)
 ---
 
 # cowork-harness
@@ -25,7 +25,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 2.3.0` (baseline
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 2.4.0` (baseline
 > `desktop-1.37937.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
@@ -42,7 +42,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 2.3.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^2.3.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^2.3.0"`. **Pin `@^2.3.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 2.4.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^2.4.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^2.4.0"`. **Pin `@^2.4.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -147,9 +147,9 @@ the folder basename (collision-resolved); there is no `to:` override. See `refer
 | Tier | What it gives you | Use when |
 |---|---|---|
 | `protocol` | Fastest; no sandbox, no egress | Pure protocol/answer-shape tests. **Rejected** if the scenario asserts egress. |
-| `container` | Real sandbox + real default-deny egress (**default**) | Most functional + boundary tests. |
-| `microvm` | VM-grade escape **isolation** (macOS arm64). Egress transport is the *same allowlist proxy as `container`* — not better network fidelity | Testing untrusted code escape, not network behavior. |
-| `hostloop` / `cowork` | Production split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production), with native Bash/WebFetch disabled and routed host-side via the workspace SDK-MCP server into a Docker VM sidecar | Highest-fidelity / parity runs. A writable connected folder needs `allow_host_writes: true` (see scenario-schema.md). |
+| `container` | Real sandbox + real default-deny egress (**default**). Models the **VM loop**: keeps the built-in `Bash`, but `WebFetch` is replaced by `mcp__workspace__web_fetch` — assert on that name, not `WebFetch`. (`run`/`record` only; `chat --fidelity container` still offers the built-in.) **The name is tier-specific**: `microvm`/`protocol` never offer it, so the same `tool_not_called` is vacuous there — moving a scenario between tiers can silently void a web-fetch assertion | Most functional + boundary tests. |
+| `microvm` | VM-grade escape **isolation** (macOS arm64). Egress transport is the *same allowlist proxy as `container`* — not better network fidelity. Unlike `container`, still offers the built-in `WebFetch` | Testing untrusted code escape, not network behavior. |
+| `hostloop` / `cowork` | Production split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production), with **both** native `Bash` and `WebFetch` disabled and routed host-side via the workspace SDK-MCP server into a Docker VM sidecar (the VM loop replaces web_fetch only) | Highest-fidelity / parity runs. A writable connected folder needs `allow_host_writes: true` (see scenario-schema.md). |
 
 Set the tier in the **scenario's `fidelity:` field**, not a flag — `run` rejects `--fidelity`
 (it's a `skill`/`chat` flag; `run` takes fidelity only from the scenario). See
@@ -550,8 +550,15 @@ Recognize these before "fixing" a non-bug:
   only when you thought to ask for it, and the runs that most need this are the ones where nobody did.
   **Silent when the evidence cannot answer the question** (no workspace walk, or a tier that runs no
   scratchpad walk, absent delivery telemetry, or a resumed turn) — "cannot tell" never reads as "clean".
-  **The fix is lane-dependent.** On `lane: local`, write deliverables under `outputs/` or a connected
-  folder, or deliver them explicitly. **On `lane: remote`, moving a file under `outputs/` does NOT help** —
+  **The fix is lane-dependent — and so is the PATH.** On `lane: local`, write deliverables where the user
+  can see them, but do **not** hardcode the literal prefix `outputs/`: on the desktop-local host-loop lane
+  (what production runs) the file tools are ALREADY rooted at `outputs/`, so `outputs/x.md` doubles to
+  `outputs/outputs/x.md` and the user never sees it — a **bare filename** is correct there. At
+  `fidelity: container`/`microvm` (VM-loop, the harness default) the base is the session root instead, so a
+  bare name lands in the scratchpad and you want `{{workspaceFolder}}` or an explicit delivery. Addressing
+  a connected folder by name (`<folder>/x.md`) never reaches it on either lane — it builds a same-named
+  decoy inside `outputs`, reports success, and gives no signal. Measured 2026-08-27; see
+  [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md), "Where a relative path actually lands". **On `lane: remote`, moving a file under `outputs/` does NOT help** —
   nothing is delivered by location there, so only an explicit delivery counts. Assert
   **`allow_undelivered_deliverables: true`** when the leftovers are intentional (intermediates, caches,
   downloaded inputs) rather than a delivery gap.

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 2.3.0` (baseline `desktop-1.37937.1`).
+Self-contained reference. Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
 copy-pasted recipe that omits the input takes a major bump with no say in it. `^2` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "2.3.0"`) instead when you want byte-reproducible CI.
+(e.g. `version: "2.4.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -67,7 +67,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^2.3.0"
+- run: npm i -g "cowork-harness@^2.4.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -342,7 +342,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^2.3.0"
+      - run: npm i -g "cowork-harness@^2.4.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -371,7 +371,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^2.3.0"
+        run: npm i -g "cowork-harness@^2.4.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 2.3.0` (baseline `desktop-1.37937.1`). This is **not** a trim of the full
+Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reproduction discipline, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 2.3.0` (baseline `desktop-1.37937.1`).
+Self-contained reference. Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,6 +1,6 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 2.3.0`
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 2.4.0`
 (baseline `desktop-1.37937.1`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 2.3.0` (baseline `desktop-1.37937.1`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 2.4.0` (baseline `desktop-1.37937.1`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.
 

package/.claude/skills/cowork-harness/scripts/scenario.py

@@ -488,6 +488,30 @@ def lint_doc(doc, path, raw_lines):
 
     fidelity = (doc.get("fidelity") or "container")
     lane = (doc.get("lane") or "local")
+
+    # W: no `fidelity:` — the default models the WRONG LANE.
+    # `container` (the schema default) models VM-loop; production runs HOST-LOOP, gate 1143815894 is
+    # force-ON in every shipped baseline. So an omitted key measures the scenario against a lane real
+    # users are not on: the file tools resolve a bare relative path differently, the shell starts
+    # somewhere else, and the offered tool set differs (measured 2026-08-27).
+    # Read the KEY, not the resolved value: `fidelity: container` is a deliberate choice and must not warn.
+    # DEPRECATION — `fidelity:` becomes REQUIRED in the next major; this is the warning window.
+    if "fidelity" not in doc:
+        findings.append(
+            Finding(
+                "WARN",
+                "fidelity-defaulted",
+                "no `fidelity:` — defaulting to `container`, which models the VM-LOOP lane. Production "
+                "runs HOST-LOOP by default (gate 1143815894), so this scenario is likely measured "
+                "against a lane your users are not on.",
+                "Name a tier: `fidelity: hostloop` to match production, `fidelity: cowork` to auto-pick "
+                "the way Cowork does, or `fidelity: container` to keep today's behaviour deliberately. "
+                "Switching tiers can COST you assertions: `no_scratchpad_leak` is container-only (an "
+                "error elsewhere) and `transcript_no_host_path` fails by design at hostloop/protocol. "
+                "The default is being removed — `fidelity:` becomes REQUIRED in the next major.",
+                path,
+            )
+        )
     items = _assert_items(doc)
     assert_keys = _all_assert_keys(items)
     has_expect_denied = bool(doc.get("expect_denied"))

package/CHANGELOG.md

@@ -6,6 +6,136 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [2.4.0] — 2026-08-27
+
+### Fidelity
+
+- **Path resolution: the shell and the file tools use DIFFERENT roots, and the harness now models that.**
+  Measured on desktop-local Cowork 2026-08-27: `mcp__workspace__bash` starts every call at the bare
+  session root (`/sessions/<id>`), while the agent process sits at the outputs dir, so a bare `Write`
+  lands in `mnt/outputs` and is user-visible immediately. The two are different path spaces — a file the
+  shell creates with a relative path is *not* where a relative `Write` puts one.
+
+  `hostloop` ran its workspace bash at `${sessionRoot}/mnt/<firstFolder ?? outputs>`, collapsing the two.
+  The replaced derivation came from the asar's `cwd: c.vmCwd` spawn argument, which is **not
+  load-bearing on the cowork path** — only the `chat` branch prepends an explicit `cd ${vmCwd}`, which
+  would be redundant if the argument worked. It reproduced a prompt claim rather than an observed
+  behaviour. Both cwds now come from one function (`hostLoopCwds`) and are pinned together in a single
+  test: a single-value assertion cannot express "shell and file tools disagree, on purpose", which is the
+  contract every previous version of this bug flattened.
+
+  **What it changes for you:** a skill that writes deliverables from a shell script using relative paths
+  looked correct at `hostloop` and delivered nothing in production. It now fails here too.
+
+- **`container` models production's VM-loop `web_fetch` swap.** When `coworkWebFetchViaApi` resolves on
+  (true in every shipped baseline), the tier registers a workspace SDK-MCP server exposing **web_fetch
+  only**, disallows the built-in `WebFetch`, and aliases the name to `mcp__workspace__web_fetch`
+  (`VM_LOOP_TOOL_ALIASES`). Bash is deliberately untouched — "Bash is the only tool that truly diverges
+  between loops" — which is why `container` keeps the built-in shell while `hostloop` replaces both.
+
+  The three parts ship together by necessity: disallowing the built-in without the alias turns a fidelity
+  fix into a regression, because the bare name stops resolving instead of landing on the workspace tool.
+  The tool is advertised but deliberately **not** pre-approved — production's VM-loop registration passes
+  the same approval hook the host loop does, so the call is gated at `can_use_tool`. Pre-approving it
+  would make a scripted `webfetch:<domain>` answer, and `decide: deny` on it, silently inert.
+
+  Its fetch is bound to the session egress allowlist and to URL provenance, exactly as the host loop's is.
+  Both matter: this fetch runs in the harness's own process, outside the container network namespace,
+  so the sidecar proxy never sees it and only these two gates constrain it. The handler's allowlist now
+  defaults to **deny-all** rather than `["*"]`, so a caller that forgets to pass one gets nothing through
+  instead of everything.
+
+  The workspace handler now gates **dispatch** on the same set it advertises, not just `tools/list`. An
+  unadvertised tool that still executed when named was a real hole: the VM-loop registration exposes
+  web_fetch only, and a `bash` call arriving there must be refused rather than quietly exec'd into the
+  container.
+
+  **`microvm` is unchanged** and still offers the built-in `WebFetch` — `spawnMicroVm` does not receive
+  the gate. **`chat` is unchanged too**: the swap applies to `run`/`record`, so `chat --fidelity container`
+  still offers the built-in and the two surfaces differ at the same declared tier.
+
+### Upgrade impact
+
+- **At `fidelity: container` under `run`/`record`, `WebFetch` is no longer in the offered tool set.** An assertion naming it
+  no longer describes a callable tool. `tool_not_called: WebFetch` is the dangerous direction: it now
+  passes **vacuously** rather than failing loudly, so a scenario that was genuinely testing "this skill
+  does not fetch from the web" silently stops testing anything. Rename to
+  `tool_not_called: mcp__workspace__web_fetch`. The shipped `example-pdf-skill` scenario carried exactly
+  this defect and is fixed.
+
+  `verify-cassettes` grew a `replaced-builtin` note for this class: it reads a cassette's recorded init
+  inventory and reports built-in names the current build no longer offers at that tier. It is a **note,
+  not a finding** — the swap is gate-conditional, so a recording made with the gate off is legitimately
+  different, and an init event carrying no `tools` key is *no evidence* rather than a missing surface.
+  Neither should be told to re-record.
+
+  This does not break a covered surface ([SPEC.md §12](./SPEC.md#12-versioning--the-10-compatibility-contract)):
+  no command, flag, schema or exit-code meaning changes. The tier's modelled tool inventory is a fidelity
+  property, and moving it toward production is the project's purpose — so it ships in a minor.
+
+- **DEPRECATION: `fidelity:` becomes REQUIRED in the next major.** A scenario that omits it currently
+  defaults to `container`, which models the **VM loop** — but production runs the **host loop** (gate
+  `1143815894` is force-ON in every shipped baseline). So an omitted key silently measures the scenario
+  against a lane real users are not on, and the two lanes differ in exactly the ways that bite: where a
+  bare relative path lands, where the shell starts, and which tools are offered.
+
+  Both `run` and the skill's `scenario.py` lint now warn (`fidelity-defaulted`). The check reads the RAW
+  document, because Zod's `.default()` makes an omitted key indistinguishable from a deliberate
+  `fidelity: container` — and those two deserve different treatment. Naming the tier explicitly silences
+  it; `fidelity: container` remains a valid, non-warning choice.
+
+### Fixed
+
+- **The semantic judge was told which authored files were never delivered.** The authored-file capture
+  deliberately includes the scratchpad, but production discards anything outside `mnt/` — "never reaches
+  the user or your file tools". Unlabelled, a rubric like "the report was written" graded TRUE on a file
+  the user never receives: a false green inside the one evaluator that reads free-form prose and cannot
+  infer the convention. Scratch files are now tagged `— SCRATCH, NOT delivered to the user`, with a note
+  explaining they are evidence of what the run DID and not that anything was delivered.
+
+- **`sync` no longer refuses to write on the `subagentPromptServerOverride` gate.** Gate-ON only enables
+  the lookup; the payload that would actually override is delivered **per session by the server** and
+  appears in neither the asar, the fcache nor `config.json`. So gate state alone cannot separate
+  "override active" from "gate on, no payload, fallback still correct" — the guard blocked forever
+  rather than tripping, because it could never clear itself from its own inputs. It is now a
+  non-blocking note that says what it can and cannot know.
+
+  Settled by a live sub-agent probe instead: a real sub-agent's environment section matched the committed
+  paraphrase on all four load-bearing claims, so no override was reaching that account. That is evidence,
+  not proof — one account, one session, and a server rule can be segment-targeted. The note says so, and
+  says to re-probe if the sub-agent append matters to what you are shipping.
+
+### Documentation
+
+- **[docs/scenario.md](./docs/scenario.md) now states where a relative path actually lands**, as a
+  measured table per tier, replacing a sentence that was simply wrong for the file tools. The guidance
+  that follows it is lane-dependent, because the correct answer inverts between lanes: on the desktop
+  host loop the file tools are already rooted at `outputs/`, so writing `outputs/x.md` doubles to
+  `outputs/outputs/x.md` and the user never sees it — a **bare filename** is right there. At
+  `container`/`microvm` a bare name lands in the scratchpad instead. Addressing a connected folder by
+  name never reaches it on either lane: it builds a same-named decoy inside `outputs`, reports success,
+  and gives no signal.
+
+- **[docs/fidelity-gaps.md](./docs/fidelity-gaps.md)** gained the path-resolution split, and its
+  "VM tiers have no workspace tool aliases" entry is updated — closed at `container`, still open at
+  `microvm`.
+
+- The skill's undelivered-deliverables guidance no longer hardcodes the literal prefix `outputs/`, which
+  was wrong on the lane production actually runs.
+
+- **The product's vocabulary is mapped to this project's**, because one directory had four names and none
+  of them was the one Cowork's UI shows. "Working folder" is the user-visible roots (`outputs/` plus each
+  connected folder); "Scratchpad" is everything outside `mnt/`; `{{workspaceFolder}}` is a prompt token
+  that renders to the *first* user-visible root. Also recorded: Cowork's **Scratchpad panel is an activity
+  log, not a location listing** — it lists files as "wrote to" wherever they landed, so a file appearing
+  there is not evidence it was undelivered.
+
+- **`Write`'s tool result echoes the raw path it was given and never absolutizes** (read from the agent
+  binary). Nothing in the harness parses a path out of a `Write` result; this is recorded so nothing
+  starts, since such an assertion would be reading something production does not emit. Cowork's own
+  chat-surface prompt claims the opposite, so the product's description of its own tool is wrong here.
+
+
 ## [2.3.0] — 2026-08-26
 
 ### Parity

package/README.md

@@ -120,7 +120,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
 
 > **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
 > <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
-> From a global install (`npm i -g "cowork-harness@^2.3.0"`), point at the package root instead:
+> From a global install (`npm i -g "cowork-harness@^2.4.0"`), point at the package root instead:
 > `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
 > (or copy the cassette into your own project and pass that path).
 
@@ -130,7 +130,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
 > - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
 > - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
 > - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
-> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@^2.3.0"`.
+> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@^2.4.0"`.
 
 Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
 test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
@@ -155,7 +155,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
 claude plugin install cowork-harness@cowork-harness
 ```
 
-The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@^2.3.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 22). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
+The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@^2.4.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 22). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
 
 It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
 
@@ -180,7 +180,7 @@ global install puts nothing in your working directory. The matrix, answer-policy
 ones that still need a source checkout. (The marketplace
 skill install itself only pulls `.claude/skills/cowork-harness/` — SKILL.md + `references/` + `scenario.py`/
 assertion keys, per `.claude-plugin/marketplace.json`'s `source` — not the rest of this table; the full set
-above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@^2.3.0"` — see
+above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@^2.4.0"` — see
 [above](#drive-it-from-claude-code-companion-skill) — which pulls the same npm package as the global-install row.)
 
 ### Prerequisites for anything above `protocol` fidelity
@@ -802,7 +802,7 @@ jobs:
           anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
 ```
 
-Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — the recipes above pin `^2` instead, because leaving it at `latest` takes a CLI major the moment it is promoted even though your `uses:` ref never changed; pin an exact version for byte-reproducible CI. The companion skill's `cowork-harness@^2.3.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any **`error`**-severity finding — advisory findings are precisely the class that does NOT gate); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). Outputs: `ok` (`"true"`/`"false"`, mirrors the exit code), `envelope-path` (path to the raw JSON envelope, for post-processing), `summary-md` (the rendered verdict table, exposed as an output — not just written to `$GITHUB_STEP_SUMMARY` — because that file is scoped to this action's own invocation and a caller's later step gets a fresh, empty one). See [`action.yml`](https://github.com/yaniv-golan/cowork-harness/blob/main/action.yml) for the full input/output reference.
+Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — the recipes above pin `^2` instead, because leaving it at `latest` takes a CLI major the moment it is promoted even though your `uses:` ref never changed; pin an exact version for byte-reproducible CI. The companion skill's `cowork-harness@^2.4.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any **`error`**-severity finding — advisory findings are precisely the class that does NOT gate); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). Outputs: `ok` (`"true"`/`"false"`, mirrors the exit code), `envelope-path` (path to the raw JSON envelope, for post-processing), `summary-md` (the rendered verdict table, exposed as an output — not just written to `$GITHUB_STEP_SUMMARY` — because that file is scoped to this action's own invocation and a caller's later step gets a fresh, empty one). See [`action.yml`](https://github.com/yaniv-golan/cowork-harness/blob/main/action.yml) for the full input/output reference.
 
 The provided [GitHub Actions workflow](https://github.com/yaniv-golan/cowork-harness/blob/main/.github/workflows/ci.yml) runs a **nine-stage pipeline**. The **build** + **test** stages are the token-free gate you can copy into your skill repo; the `floor`, `action-self-test`, `python`, `image-recipe`, `boundary`, `scenarios`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):
 

package/dist/assert.js

@@ -373,7 +373,7 @@ function capForJudge(text, cap) {
         return text;
     return `${text.slice(0, cap)}\n…[${text.length - cap} chars truncated for the judge input budget — evidence beyond this point was NOT shown; do not infer absence from this cut]`;
 }
-function buildJudgedDocument(ctx, includeSubagentText = false) {
+export function buildJudgedDocument(ctx, includeSubagentText = false) {
     // SCRUB BEFORE CAP: scrub is exact-string replacement, so a secret straddling a cap boundary would
     // be truncated mid-token and slip past scrub into the doc sent to the (external) judge. Scrub each raw
     // section FIRST, then cap the already-redacted text — capping redacted content can never re-expose a secret.
@@ -405,8 +405,33 @@ function buildJudgedDocument(ctx, includeSubagentText = false) {
             parts.push(`## Sub-agent output: ${s(label)}\n${capForJudge(s(text), JUDGE_SUBAGENT_CAP)}`);
         }
     }
-    for (const f of ctx.authoredFiles ?? [])
-        parts.push(`## Authored file: ${s(f.path)}${f.truncated ? " (truncated)" : ""}\n${s(f.content)}`);
+    // AUTHORED is not DELIVERED. The capture deliberately includes the scratchpad (the run DID write those
+    // files, and production's sandbox would too), but production DISCARDS anything outside `mnt/` — its own
+    // sub-agent prompt: "never reaches the user or your file tools". Unlabelled, a rubric like "the report
+    // was written" grades TRUE on a file the user never receives — the harness's own false-green, inside the
+    // one evaluator that reads free-form prose and cannot infer the convention.
+    //
+    // The distinction is already carried in the path: the scratchpad walk emits a synthetic `scratchpad/`
+    // prefix (run/artifacts.ts). This only makes it legible to the judge, rather than restructuring the set.
+    // `scratchpad` is NOT in RESERVED_MOUNT_NAMES, so a user may connect a folder with that exact name.
+    // Its files then arrive workRoot-relative as `scratchpad/…` — indistinguishable by prefix from the
+    // synthetic walk. Labelling those would hand the judge a FALSE statement ("NOT delivered" about a file
+    // the user does receive) and false-RED a "was the report delivered?" rubric. A wrong claim is worse
+    // than a missing one, so the collision disables the label rather than guessing.
+    // `?? []` because a partial AssertContext (tests, and any caller building one by hand) may omit this;
+    // an absent prefix list means "nothing is user-visible", which keeps the label ON — the safe direction.
+    const scratchIsUserVisible = (ctx.userVisiblePrefixes ?? []).some((p) => p === "scratchpad" || p === SCRATCHPAD_PREFIX);
+    let sawScratch = false;
+    for (const f of ctx.authoredFiles ?? []) {
+        const scratch = !scratchIsUserVisible && f.path.startsWith(SCRATCHPAD_PREFIX);
+        sawScratch ||= scratch;
+        const tag = scratch ? " — SCRATCH, NOT delivered to the user" : "";
+        parts.push(`## Authored file: ${s(f.path)}${tag}${f.truncated ? " (truncated)" : ""}\n${s(f.content)}`);
+    }
+    if (sawScratch)
+        parts.push("## Note on scratch files\nFiles marked SCRATCH were written to the session's scratch area, which is " +
+            "discarded and never reaches the user. Treat them as working intermediates: they are evidence of what " +
+            "the run DID, and are NOT evidence that anything was delivered, saved, shared, or produced for the user.");
     // Surface authored-file incompleteness to the judge so it never reads an omitted/unreadable file's
     // ABSENCE as evidence the skill didn't produce it (#14/#16). The verdict is separately forced to
     // evidence-unavailable in the semantic_matches check; this note keeps a still-produced grade honest.

package/dist/hostloop/workspace-handler.js

@@ -228,10 +228,12 @@ const defaultRawFetch = async (url, pinnedAddresses) => {
 const BASH_DESC = "Run a shell command in the session's isolated Linux workspace. Your connected folders are mounted under {{mnt}}/ — the Shell access section of your system prompt lists the exact path for each folder. Each bash call is independent (no cwd/env carryover). Use absolute paths.";
 const FETCH_DESC = "Fetch a URL from the session network (subject to the egress allowlist). web_fetch can only retrieve URLs that appeared in a user message or a prior result.";
 export function makeWorkspaceHandler(opts) {
-    const { containerName, vmMnt, runner = "docker", webFetchAllow = ["*"], onEgress, onInfraError, provenanceRef, dedup, rawFetch = defaultRawFetch, resolve = defaultResolver, execCwd = vmMnt, } = opts;
+    const { containerName, vmMnt, runner = "docker", webFetchAllow = [], // DENY-ALL when unset — see the option doc; an open default was a real hole
+    onEgress, onInfraError, provenanceRef, dedup, rawFetch = defaultRawFetch, resolve = defaultResolver, execCwd = vmMnt, } = opts;
     // Per-handler (per-spawn) latch for the provenance-unenforced warning — was module-level, which
     // silenced the gap after the first run in a long-lived process. Each fresh handler warns once.
     const provWarned = { value: false };
+    const exposed = opts.tools ?? ["bash", "web_fetch"];
     const tools = [
         {
             name: "bash",
@@ -243,7 +245,7 @@ export function makeWorkspaceHandler(opts) {
             description: FETCH_DESC,
             inputSchema: { type: "object", properties: { url: { type: "string" } }, required: ["url"] },
         },
-    ];
+    ].filter((td) => exposed.includes(td.name));
     return async (_server, jr) => {
         const method = jr.method;
         if (method === "initialize")
@@ -259,6 +261,12 @@ export function makeWorkspaceHandler(opts) {
         if (method === "tools/call") {
             const name = jr.params?.name;
             const a = jr.params?.arguments ?? {};
+            // Gate the DISPATCH on the same set as the advertisement, not just the tools/list response. An
+            // unadvertised tool that still executes when named is a real hole: the VM-loop registration exposes
+            // web_fetch ONLY, and a `bash` call arriving there must be refused, not quietly exec'd into the
+            // container.
+            if (!exposed.includes(name))
+                return { result: textResult(`error: unknown tool "${String(name)}"`, true) };
             if (name === "bash")
                 return {
                     result: await execInContainer(runner, containerName, execCwd, String(a.command ?? ""), clampTimeout(a.timeout_ms), onInfraError),
@@ -267,6 +275,8 @@ export function makeWorkspaceHandler(opts) {
                 return {
                     result: await fetchViaHost(String(a.url ?? ""), webFetchAllow, onEgress, provenanceRef?.current, provWarned, rawFetch, resolve, dedup),
                 };
+            // Unreachable: the dispatch gate above rejects every name outside `exposed`, and `exposed` is a
+            // subset of the two handled here. Kept as a total-function backstop if that gate is ever loosened.
             return { error: { code: -32602, message: `unknown tool: ${name}` } };
         }
         return { result: {} }; // ping / notifications

package/dist/run/cassette.js

@@ -1484,10 +1484,58 @@ function computeDiscoverySurfaceNote(cassette) {
             `Re-record if this scenario asserts on those tools; harmless otherwise.`,
     ];
 }
+/** Built-ins this build REPLACES with a workspace MCP tool, per tier. Production swaps different sets per
+ *  loop, so the expectation is tier-specific:
+ *    host-loop — the patch disallows Bash AND WebFetch and aliases both;
+ *    VM-loop   — the site registers web_fetch ONLY (gated on coworkWebFetchViaApi) and never touches Bash,
+ *                which is why container legitimately keeps the built-in shell. */
+const REPLACED_BUILTINS_BY_TIER = {
+    // NotebookEdit is in spawnHostLoop's `disallowed` alongside Bash/WebFetch, so a cassette asserting
+    // `tool_not_called: NotebookEdit` at this tier is just as vacuous. It has no workspace replacement —
+    // the tier simply removes it — which the note's wording accommodates.
+    hostloop: ["Bash", "WebFetch", "NotebookEdit"],
+    container: ["WebFetch"],
+    // microvm is deliberately ABSENT, not an oversight: spawnMicroVm never receives `webFetchViaApi`
+    // (execute.ts), so that tier still offers the built-in WebFetch. Listing it here would tell users to
+    // re-record a cassette that faithfully describes what this build produces.
+};
+/** A cassette whose recorded init inventory names a built-in this build no longer offers at that tier.
+ *
+ *  The gap this closes, hit for real: `example-pdf-skill` recorded `WebFetch` at `container` and asserted
+ *  `tool_not_called: WebFetch`. When the harness started modelling production's VM-loop swap, that tool
+ *  stopped existing there — so the assertion could never be violated and passed VACUOUSLY, while
+ *  `verify-cassettes` exited 0 throughout. Staleness keys on baseline/skillHash, so a fixture can describe
+ *  a tool surface the harness no longer produces and nothing notices.
+ *
+ *  A NOTE, not a finding, deliberately: the swap is gate-conditional (`coworkWebFetchViaApi`), so a
+ *  cassette recorded with the gate off is legitimately different rather than stale. A false positive costs
+ *  a line of prose; a false FINDING would red a correct fixture. Mirrors computeDiscoverySurfaceNote. */
+function computeReplacedBuiltinNote(cassette) {
+    const tools = recordedInitTools(cassette);
+    if (!tools || tools.length === 0)
+        return []; // no evidence — NOT "the tools are missing"
+    const authored = cassette.scenario.fidelity;
+    const tier = cassette.environment?.tier ?? cassette.effectiveFidelity ?? (authored !== "cowork" ? authored : undefined);
+    if (!tier)
+        return [];
+    const replaced = REPLACED_BUILTINS_BY_TIER[tier];
+    if (!replaced)
+        return [];
+    const stale = replaced.filter((name) => tools.includes(name));
+    if (stale.length === 0)
+        return [];
+    return [
+        `replaced-builtin: this cassette's init inventory names ${stale.join(", ")} at ${tier}, which this build ` +
+            `no longer offers there — Bash/WebFetch are replaced by workspace MCP tools (mcp__workspace__*), and ` +
+            `NotebookEdit is removed outright. An assertion naming one of them can no longer be violated: it would ` +
+            `pass VACUOUSLY. Re-record if this scenario asserts on those names; harmless if it does not, and ` +
+            `expected if the recording predates the swap or ran with coworkWebFetchViaApi off.`,
+    ];
+}
 export function computeStaleness(cassette, cassetteDir, sessionOverride) {
     const tier = computeTierStaleness(cassette);
     const findings = [...tier.findings];
-    const notes = [...tier.notes, ...computeDiscoverySurfaceNote(cassette)];
+    const notes = [...tier.notes, ...computeDiscoverySurfaceNote(cassette), ...computeReplacedBuiltinNote(cassette)];
     const fp = cassette.fingerprint;
     // BEFORE the fingerprint guard on purpose — same rationale as the tier check above: fingerprint-less
     // cassettes are the OLDEST, i.e. exactly the population the discovery-surface note targets.