cowork-harness 4.8.1 → 4.9.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, asserting artifacts, egress, or sub-agent dispatch, measuring how long each tool call took (toolDurations / trace), or debugging a failed run or verdict from its result.json or transcript. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. Also for comparing two versions of a skill before merging an edit — did it make answers worse? (`eval`: paired A/B, pinned models) — or improving one round by round (`hillclimb`, the `/claude-api hillclimb` runner). NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold / critique / stats / eval / hillclimb commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 4.8.1
-  tracks-harness: cowork-harness 4.8.1 (baseline desktop-2.31226.1)
+  version: 4.9.0
+  tracks-harness: cowork-harness 4.9.0 (baseline desktop-2.31226.1)
 ---
 
 # cowork-harness
@@ -26,7 +26,7 @@ allowlist). This skill exists mostly to keep you out of those traps — the *Inv
 full landmine catalog in [`references/gotchas.md`](references/gotchas.md) are the highest-value part.
 Read them.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.8.1` (baseline
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.9.0` (baseline
 > `desktop-2.31226.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
@@ -35,15 +35,15 @@ Read them.
 The 10-second inner loop, once the CLI is on PATH:
 
 ```bash
-cowork-harness doctor                       # prerequisites OK? (Docker, agent, token, baseline)
+cowork-harness doctor                       # prerequisites OK? (Docker, agent, token, baseline, device image)
 cowork-harness skill ./my-skill "do X" --model claude-sonnet-5   # run once (or set COWORK_HARNESS_MODEL)
 ```
 
 Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
 
-- **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
+- **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, the baseline and the `device_bridge` image (built, or built on the first bridge run) in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.8.1**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.8.1" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.8.1"`. **Pin `@^4.8.1`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.9.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.9.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.9.0"`. **Pin `@^4.9.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -106,7 +106,7 @@ CI-grade scenario, and the post-hoc debug loop; the rest are narrower tools that
 
 Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · migrate-run-dir · lint ·
 lint-skill · analyze-skill · probe-dispatch ·
-verify-run · regrade · fixture · ref <freeze|verify> · trace · inspect · diff · critique · eval · eval report · hillclimb <run|check|state-template|freeze-ref|regrade> · stats · decide · gates · answer · scaffold · assertions --list · sync ·
+verify-run · regrade · fixture · search-pool <build|export|import> · ref <freeze|verify> · trace · inspect · diff · critique · eval · eval report · hillclimb <run|check|state-template|freeze-ref|regrade> · stats · decide · gates · answer · scaffold · assertions --list · sync ·
 list · boundary-check · status · vm <init|status|delete|prune> · doctor · init-redact`. Always check `cowork-harness <cmd> --help`.
 
 ## Invariants — how a green run lies
@@ -155,23 +155,23 @@ behind each, is [`references/gotchas.md`](references/gotchas.md).
 
 | File | Read it for |
 |---|---|
-| [`references/authoring.md`](references/authoring.md) | session vs scenario, discovery, fidelity tier, the answer-channel decision tree, `web_fetch`, scaffold + lint |
+| [`references/authoring.md`](references/authoring.md) | session vs scenario, discovery, fidelity tier, the answer-channel decision tree, `web_fetch`, a skill that resumes in a later session (`legs:`), scaffold + lint |
 | [`references/assertions-guide.md`](references/assertions-guide.md) | the two assertion axes, the goal → key map |
-| [`references/run-record-replay.md`](references/run-record-replay.md) | run / `verify-run`, recording and cassette placement, real-document validation, verdict signals (incl. where a relative or `outputs/` path lands per tier), background-run liveness, CI lanes |
+| [`references/run-record-replay.md`](references/run-record-replay.md) | run / `verify-run`, multi-leg runs and `--leg`, recording and cassette placement, real-document validation, verdict signals (incl. where a relative or `outputs/` path lands per tier), background-run liveness, CI lanes |
 | [`references/measurement.md`](references/measurement.md) | `--repeat`, `--ablate-skill`, measurement hygiene |
 | [`references/debugging.md`](references/debugging.md) | triage, `result.json` fields and `trace` views, `chat` |
 | [`references/gotchas.md`](references/gotchas.md) | the full "✓ passed ≠ correct" landmine catalog |
-| [`references/task-recipes.md`](references/task-recipes.md) | start here for "how do I X": evolve `assert:`, audit tier drift, redaction, budgets, answer quality, and goals with no flag (force a compaction, ablate a section, a form reply, resume in a new conversation, hook JSON decisions, schema checks, unattended runs) |
+| [`references/task-recipes.md`](references/task-recipes.md) | start here for "how do I X": evolve `assert:`, audit tier drift, redaction, budgets, answer quality, and goals with no flag (force a compaction, ablate a section, a form reply, resume in a new conversation, hook JSON decisions, schema checks, unattended runs), keeping a skill's web searches (`search-pool`), and a skill that parks and resumes in a later session (`legs:`) |
 | [`references/assertion-catalog.md`](references/assertion-catalog.md) | the assertion catalog's index: conventions shared by every key, the verdict-signal table, and links to the per-family files below |
 | [`references/assertion-catalog-outcome-files-tools.md`](references/assertion-catalog-outcome-files-tools.md) | per-key rows: outcome, transcript, files and artifacts, tools |
 | [`references/assertion-catalog-agents-skills-budgets.md`](references/assertion-catalog-agents-skills-budgets.md) | per-key rows: sub-agents, skills and connectors, budgets, tasks, delivery |
 | [`references/assertion-catalog-gates-hooks-modifiers.md`](references/assertion-catalog-gates-hooks-modifiers.md) | per-key rows: gates, hooks, path denial, verdict modifiers, egress, judged keys |
 | [`references/semantic-judging.md`](references/semantic-judging.md) | `semantic_matches` in full: what the judge reads, fork results, refusal reasons, provenance |
-| [`references/scenario-schema.md`](references/scenario-schema.md) | every YAML field, which keys survive `replay`, the `web_fetch` model |
+| [`references/scenario-schema.md`](references/scenario-schema.md) | every YAML field, the multi-leg (`legs:`, `host_action`, `workdir`) form, which keys survive `replay`, the `web_fetch` model |
 | [`references/fidelity-and-answers.md`](references/fidelity-and-answers.md) | tier semantics, answer paths, the determinism contract |
 | [`references/ci-recipe.md`](references/ci-recipe.md) | the GitHub Action, replay-vs-live lanes, the four-stage pipeline |
 | [`references/critique.md`](references/critique.md) | `critique` report and evidence-package shapes |
-| [`references/eval.md`](references/eval.md) | `eval`: paired before/after of two plugin versions — labels, refusals, exit codes, files |
+| [`references/eval.md`](references/eval.md) | `eval`: paired before/after of two plugin versions — labels, metric rows, refusals, exit codes, files |
 | [`references/hillclimb.md`](references/hillclimb.md) | `hillclimb run` / `check` / `state-template` / `freeze-ref` / `regrade` for a `/claude-api hillclimb` loop — pass `--flow` to each, the harness gate, resume, pairwise references, re-grading, refusals, exit codes |
 | [`references/hillclimb-recipe.md`](references/hillclimb-recipe.md) | Recipe 7: the `/claude-api hillclimb` loop with the harness as its runner, step by step — the runner command, Step 0.5 checks, metrics, spend, rounds, stalls |
 | `scripts/scenario.py` | `scaffold`, `lint`, `lint-skill`, `resolve-agent-types <plugin-dir>` (validates a pinned `subagent_type` against `plugin.json` + `agents/*.md`) |

package/.claude/skills/cowork-harness/references/assertion-catalog-agents-skills-budgets.md

@@ -1,6 +1,6 @@
 # Assertion catalog: sub-agents, skills and connectors, budgets, tasks, delivery
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 

package/.claude/skills/cowork-harness/references/assertion-catalog-gates-hooks-modifiers.md

@@ -1,6 +1,6 @@
 # Assertion catalog: gates, hooks, path denial, verdict modifiers, egress, judged keys
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md).
 
@@ -17,7 +17,7 @@ refused at load: no question reaches the harness to grade.
 | `gate_answers_delivered: true` | every answered gate's answer reached the model (observed `tool_result`; unobserved = fail); **zero gates fired passes vacuously** — pair with `gate_answer_count_min: >= 1` to also require a gate, or drop this key and declare `questions_count_max: 0` in a scenario that expects none |
 | `gate_answers_delivered: false` | asserts at least one answered gate's answer was **confirmed not delivered** (an observed delivery failure); an unobserved/null delivery does **not** satisfy this — for negative-path delivery tests. Requires a gate, so **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
 | `gate_answer_count_min: <N>` | at least N AskUserQuestion gates fired AND were delivered non-error — presence companion to `gate_answers_delivered`'s vacuous-pass. **`: 0` asserts nothing** and does not satisfy that pairing; `>= 1` is **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
-| `gates_all_scripted: true \| {include_permissions: true}` | every AskUserQuestion gate that fired was answered by a scripted rule (`answers:` / `--answer`) — not the LLM decider, `first`, or an external or human decider; a failure names each such gate and who answered it. A gate asked but never answered fails. **Evidence:** the run's decisions — a result with no decisions record, or an answer with no recognisable source, fails evidence-unavailable, never passes. On replay each gate is re-classified against the cassette's frozen `answers:`; that is evidence-unavailable when those answers were redacted, or when the cassette records that a live decider answered a gate yet its frozen rules cover every gate (they are not the rules it recorded with). **`{include_permissions: true}`** also checks tool permissions: a scripted `when_tool` rule or a fixed rule counts (a Read/Glob/Grep registry allow, a strict-parity deny, the harness's own path-gate deny, the fail-closed deny when nothing answered live — on replay that row means the recording held no answer, evidence-unavailable); a web_fetch deny that no answer source decided fails (`answered by abstain-fallback`), because real Cowork asks the user; cowork parity's permissive off-registry auto-allow never does, and a replayed web_fetch permission cannot be attributed (evidence-unavailable). Zero gates passes (nothing needed a person), so pair it with `gate_answer_count_min: >= 1` or `questions_count_max` to state whether gates were expected (`lint` warns `unpaired-gates-all-scripted`). |
+| `gates_all_scripted: true \| {include_permissions: true}` | every AskUserQuestion gate that fired was answered by a scripted rule (`answers:` / `--answer`) — not the LLM decider, `first`, or an external or human decider; a failure names each such gate and who answered it. A gate asked but never answered fails. **Evidence:** the run's decisions — a result with no decisions record, or an answer with no recognisable source, fails evidence-unavailable, never passes. On replay each gate is re-classified against the cassette's frozen `answers:`; that is evidence-unavailable when those answers were redacted, or when the cassette records that a live decider answered a gate yet its frozen rules cover every gate (they are not the rules it recorded with). **`{include_permissions: true}`** also checks tool permissions: a scripted `when_tool` rule or a fixed rule counts (a Read/Glob/Grep registry allow, a strict-parity deny, the harness's own path-gate deny, the fail-closed deny when nothing answered live — on replay that row means the recording held no answer, evidence-unavailable); a web_fetch deny that no answer source decided fails (`answered by abstain-fallback`), because real Cowork asks the user (under the default `on_unanswered: fail` such a fetch ends the run as an unanswered gate before any deny, so this arises only under `first` or `llm`); cowork parity's permissive off-registry auto-allow never does, and a replayed web_fetch permission cannot be attributed (evidence-unavailable). Zero gates passes (nothing needed a person), so pair it with `gate_answer_count_min: >= 1` or `questions_count_max` to state whether gates were expected (`lint` warns `unpaired-gates-all-scripted`). |
 | `hook_blocked: <regex>` | one of the harness's own PreToolUse hook callbacks blocked a tool whose name matches the regex (`RunResult.hookEvents`; a plugin's hook never reaches it: use `hook_event_blocked`) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette (a custom hook's decision lives only there, not the recorded stream). **Does not see the agent's own refusal**: on `hostloop` against Desktop 2.7032.0+ a relative `Read`/`Write`/`Edit` is denied by the agent's permission rules before any hook runs, so neither this key nor `path_denied` records it — assert it with `tool_result_contains: "denied by your permission settings"` |
 | `no_hook_blocked: true` | no tool was blocked by the harness's own hook callbacks (a plugin's hook never reaches it: use `no_hook_event_blocked`) (distinguishes a real tool crash from an intentional hook block) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette. **Only `true` is valid** |
 | `hook_event_fired: <HookEvent>` | a **command hook** for this event (a plugin's `hooks/hooks.json` or manifest hook — `Stop`, `SessionStart`, `PostToolUse`, …) ran: a `hook_response` system frame with that `hook_event` was recorded (`RunResult.contextEvents`). Any outcome counts. The harness passes `--include-hook-events` whenever a staged plugin declares hooks — that is what puts events other than SessionStart/Setup on the stream — so a recording made without it reports "never fired". Content-class, grades on replay. Recorded end-to-end for `Stop` ([stop-hook-probe.scenario.yaml](https://github.com/yaniv-golan/cowork-harness/blob/main/examples/probes/stop-hook-probe.scenario.yaml)) and for `PreToolUse` at `container` (deciding by JSON and by exit 2); the other names match the same frame but have not each been recorded |

package/.claude/skills/cowork-harness/references/assertion-catalog-outcome-files-tools.md

@@ -1,6 +1,6 @@
 # Assertion catalog: outcome, transcript, files and artifacts, tools, tool results
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 
@@ -14,7 +14,7 @@ keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 | `file_exists: <path>` | the path exists under the run's `work/` (anchored at `mnt/`, e.g. `outputs/x.md`). For a user-facing deliverable prefer `user_visible_artifact` — with a connected folder the file lands in `mnt/<folder>` (= `{{workspaceFolder}}`), not `mnt/outputs`, so `file_exists: outputs/x.md` misses it. Object form `{path, authored}`: `authored: true` also requires that THIS run created or rewrote the file (an untouched pre-run file — e.g. from a `workspace_fixture` — fails; no pre-run manifest ⇒ evidence-unavailable); `authored: false` states that inheriting it is fine. On a `workspace_fixture` path one of the two is REQUIRED (refused at load otherwise). On `lane: remote` plain `file_exists` is the proxy for a written path; with `authored: true` it is rejected at load.  Under `device_bridge`, a path in a connected folder is refused: the folders live in the device container; assert the `device_bash` call instead. |
 | `user_visible_artifact: <path>` | exists **and** under a user-visible root (`outputs/` + each connected folder's mount name) — the right primitive for a workspace deliverable when a folder is connected. Takes the same `{path, authored}` object form as `file_exists` (and `artifact_text`/`artifact_json` take an `authored` field) |
 | `no_delete_in_outputs: true` | no delete op touched `mnt/outputs` — **only `true` is valid**; `false` is rejected. Checks on **every** baseline. Omitting it: on a baseline recording outputs `rw` (Desktop before 2.16120.0) a detected delete still fails via the `outputs_delete` verdict signal, which fires *because* the key was not authored (`allow_outputs_delete: true` accepts an intended one); on `rwd` (2.16120.0+, including `latest`, where Cowork allows deletes in outputs) nothing checks outputs deletes, so author this key to keep the check. Detection is a post-run bash-command scan plus a per-turn filesystem diff of `outputs/`, not mount enforcement, so a green means none was *detected*. Fails when the diff proves the delete, when a delete in command/call position has an `outputs/` path as its own operand, or when the diff could not verify; a hit resting only on the detector's inference (e.g. a Python variable named `rm`) with a clean diff passes, and the `outputs_delete_unconfirmed` warn is still raised in the run output — see that code for the classes of real delete that land there |
-| `no_delete_in_mounts: true` | no delete op touched `outputs` or any `rw` connected folder, except mounts waived by `allow_delete_in`. Covers `outputs` on **every** baseline, including those where Cowork allows an outputs delete (unless outputs is waived, authoring it arms the outputs check, so a filesystem-proven delete fails as the `outputs_delete` signal, which `allow_outputs_delete` waives). Production denies `unlink`/`rmdir` on a `rw` connected folder until approval, so `no_delete_in_outputs` covers only part of the rule. Under `device_bridge` it also reads `device_bash` commands and compares the device copy of each `rw` folder before and after the run. **only `true` is valid** |
+| `no_delete_in_mounts: true` | no delete op touched `outputs` or any `rw` connected folder, except mounts waived by `allow_delete_in`. Covers `outputs` on **every** baseline, including those where Cowork allows an outputs delete (unless outputs is waived, authoring it arms the outputs check, so a filesystem-proven delete fails as the `outputs_delete` signal, which `allow_outputs_delete` waives). Production denies `unlink`/`rmdir` on a `rw` connected folder until approval, so `no_delete_in_outputs` covers only part of the rule. Under `device_bridge` it compares the device copy of each `rw` folder before and after the run (a delete the user allowed through the delete card counts), and reads `device_bash` commands only on a run whose device did not refuse deletes (`deleteRefusal: "reported"`). **only `true` is valid** |
 | `no_unexpected_files: [<glob>, …]` | every **newly created** file under a user-visible root matches ≥1 glob (workRoot-relative paths; `**` = whole path segment for any depth — use `outputs/handoff/**` for per-run subdirs); `[]` = no new files; **new-files-only** — overwriting a pre-existing file in place is invisible (use content-level producer stamping); live/verify-run without a pre-run manifest ⇒ evidence-unavailable hard-fail (live runs capture the baseline only when this key is asserted; recordings always capture); an **incomplete post-run filesystem walk** (an unreadable subtree — a permission/I-O error) also fails evidence-unavailable rather than reporting "no strays" over a partial tree — distinct from the missing-manifest case (a `--resume` run), which fails for a different reason; captured on every live sandbox tier including microvm (its outputs are snapshotted from the VM into the run dir); replay needs `cassette.preRunPaths` (≥0.24 recordings) — cassettes without it **exclude** the key with a loud warning. ⚠️ **Not a stand-in for "file X must not exist":** there is no negative-existence key, and inverting this one is a different claim — it is new-files-only (a pre-existing file is invisible to it) and needs a pre-run manifest. Assert the content-level consequence instead (`tool_not_called`, or the absent `artifact_json`) rather than widening an allowlist for every incidental lock/temp file. Rejected at load on `lane: remote`. |
 | `input_unmodified: <glob> \| [<glob>, …]` | a single glob or a list; every **pre-existing** file (incl. uploaded files under `uploads/**`) whose workRoot-relative path matches ≥1 glob keeps an unchanged content hash after the run — the in-place-mutation companion to `no_unexpected_files`'s new-files check (`[]` is rejected by the schema — list at least one glob); a glob that matches **no** pre-run path fails loud (a typo or renamed mount would otherwise verify zero files and pass vacuously); a matched file that was deleted counts as a content change (fails); live/verify-run without a pre-run hash manifest ⇒ evidence-unavailable hard-fail (a `--resume` run); captured on every live sandbox tier including microvm; replay needs `cassette.preRunHashes` — cassettes without it **exclude** the key with a loud warning; on replay it compares against the manifest's recorded `sha256`, never a re-hash of the materialized tree. On `lane: remote` it still reads the local stand-in for the inputs, which is right there: an edit to a staged copy inside the cloud container does not reach the device. Under `device_bridge`, a glob on a connected folder's path is refused (the folders live in the device container). |
 | `self_heal_ran: <bool>` | a bash command the model wrote did (not) name a plugin under `/sessions/<id>/mnt/.local-plugins/` or `/sessions/<id>/mnt/.remote-plugins/` — the plugin-root self-heal path. It reads the command as the model wrote it, so a host plugin path that `hostloop`'s bash tool rewrote to the VM mount does not count |
@@ -26,7 +26,7 @@ keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 | `tool_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?, count?}` | **object form** — asserts what a call CARRIED, where it RAN and what its paired result SAID, which no name glob and no `transcript_*` key can see (`transcript_*` reads top-level prose only, so it passes when the agent merely *says* it ran a command). `tool`: a glob or a list of globs (any-of; list both shells, `[Bash, mcp__workspace__bash]`, for a claim that must hold at hostloop too). `input: {<field>: <regex>}`: every named top-level input field must match (case-insensitive, unanchored; a missing field is no match; a non-string value is matched as JSON). `input_any: <regex>`: some top-level field matches. `result: {matches?, not_matches?, is_error?}`: predicates on the call's paired `tool_result` (by `toolUseId`, 10,240 characters of text — 32,768 for a top-level `Skill` result). `scope`: `main` (default — the main agent, including a `Skill`'s or `Agent(fork)`'s children, the set the string form reads), `subagent` (the parent is a sub-agent dispatch this run recorded, at any depth), or `any` (also calls whose parent is not a recorded dispatch, e.g. a `Skill` invoked inside a sub-agent). `subagent_type` (with `scope: subagent`): regex over the IMMEDIATE parent dispatch's type or description. `count: {min?, max?}` (default `min: 1`). **Fails closed:** an unpaired call never satisfies `result`; a truncated result or input that cannot settle a predicate, or a result.json without `toolCalls`, reports *evidence unavailable*, never a pass or a plain "not called". A red lists the calls it considered and names matches in another scope ("1 matching call in scope subagent — set `scope: any`"). `{tool: X}` alone is exactly `tool_called: X`. A cassette using this form stamps **v13**. |
 | `tool_not_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?}` | **object form** — no call in scope satisfies every predicate (no `count`). Same fields and fail-closed rules as above: a candidate whose result is unpaired or truncated, or whose input was truncated, is *evidence unavailable*, never a pass. Refused at load when EVERY listed tool is one the tier does not serve. ⚠️ **Redaction hazard — this is the dangerous direction.** A committed cassette is redacted, which rewrites the recorded inputs: an `input` regex naming a literal the policy rewrites (a home path, an email) cannot see its target on replay. On replay, any candidate call whose field (or paired result, for `result.matches`) carries a redaction token is *evidence unavailable*, whatever the regex — the evaluator cannot know what the token replaced — so a negative check never passes over rewritten bytes. `record` warns — naming the redacted call and the ways out (narrow `scope`/`tool`, the string form, or live-only) — and refuses the write (the exact guard), and `lint` flags a regex naming a redactable literal (`tool-input-regex-redactable`) — a heuristic that reads `.cowork-redact.json` from the current and scenario directories only, not the cassette's. Match a part of the input redaction leaves alone (the verb and flags, a workspace-relative path), or keep the check on a live gate. Note also that the default `scope: main` does not see a sub-agent's call — use `scope: any` for "nothing anywhere ran X". |
 | **legacy tool names** | The agent binary canonicalizes legacy spellings (`Task`→`Agent`, `KillShell`/`KillBash`→`TaskStop`, …) while the spawn tool list still declares the LEGACY one — so the init inventory shows `Task` and every call is emitted as `Agent`. `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` match EITHER spelling, globs included. Before this, `tool_called: "Task"` could never pass and `tool_not_called: "Task"` always did. |
-| **tier vacuity — now REFUSED** | `tool_not_called` **and `subagent_tool_absent`** naming a tool the tier does not serve is rejected at scenario load (`Bash`/`WebFetch`/`NotebookEdit` at `hostloop`; `mcp__workspace__bash` at `container`/`microvm`; on those three tiers, `mcp__remote-devices__device_bash`/`get_device_info` without `device_bridge: true`, and every other `mcp__remote-devices__*` tool the harness never serves, bridge or not), because it could never be violated. `scenario.py lint` WARNs on the same set before any run. The table is CLOSED to tools the harness itself removes or registers — `--tools` gates only the built-in set while every tier separately passes `--mcp-config`, so a session-MCP tool name is offered without appearing in any tool list and is never refused. Globs are never refused. `subagent_tool_absent` is covered because it reads the tools sub-agents actually USED, not a per-dispatch declared list. Under-approximates on purpose: `REPL` at hostloop is vacuous too and is not caught, which is the right side to err on for a hard refusal. |
+| **tier vacuity — now REFUSED** | `tool_not_called` **and `subagent_tool_absent`** naming a tool the tier does not serve is rejected at scenario load (`Bash`/`WebFetch`/`NotebookEdit` at `hostloop`; `mcp__workspace__bash` at `container`/`microvm`; on those three tiers, `mcp__remote-devices__device_bash`/`get_device_info`/`device_request_delete_permission` without `device_bridge: true`, and every other `mcp__remote-devices__*` tool the harness never serves, bridge or not; and `mcp__visualize__read_me`/`show_widget`, production's skill-argument form tools, which the harness never serves unless the session's own MCP config declares a `visualize` server), because it could never be violated. `scenario.py lint` WARNs on the same set before any run. The table is CLOSED to tools the harness itself removes or registers — `--tools` gates only the built-in set while every tier separately passes `--mcp-config`, so a session-MCP tool name is offered without appearing in any tool list and is never refused. Globs are never refused. `subagent_tool_absent` is covered because it reads the tools sub-agents actually USED, not a per-dispatch declared list. Under-approximates on purpose: `REPL` at hostloop is vacuous too and is not caught, which is the right side to err on for a hard refusal. |
 | `reference_read: <regex>` | a skill `references/`/`scripts/` file whose path matches this **regex** was ACCESSED — main agent or sub-agents, via `Read`, `Grep`/`Glob` (`path` input), or a `Bash`/`mcp__workspace__bash` command naming the path. Regex is **unanchored + case-insensitive** (the shared helper every regex key uses). **Under-approximates by design:** the path must be rooted in the mounted plugin, so a `cd` into the skill dir then a bare `cat references/x.md`, a heredoc body, and a `$VAR`-built path are invisible. Fails **evidence unavailable** when the run recorded no observable tool stream. Replay-capable (cassettes freeze whole tool inputs) |
 | `no_observed_reference_access: <regex>` | no OBSERVED access matched the regex — the progressive-disclosure check: a reference the skill's routing never reaches. Named `observed` because detection under-approximates (see above), so it is **not proof the file went unread** — an agent that `cd`s and `cat`s it passes. Fails **evidence unavailable** rather than passing vacuously when no observable tool stream was recorded |
 | `tool_result_contains: <str>` | a tool result includes the literal string (content / replay-checkable — substring match) |

package/.claude/skills/cowork-harness/references/assertion-catalog.md

@@ -1,6 +1,6 @@
 # Assertion catalog
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Every `assert:` key with its semantics, and the
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Every `assert:` key with its semantics, and the
 verdict-signal table. Which keys survive `replay` is in [`scenario-schema.md`](./scenario-schema.md#replay-class);
 the scenario and session YAML fields are there too.
 
@@ -23,6 +23,18 @@ The per-key table is split by assertion family so each file stays well within wh
 - [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md): subagent_*, dispatch, skills and connectors, cost / token / turn / tool-error budgets, tasks, scratchpad and present_files delivery.
 - [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md): AskUserQuestion gates, hooks, path denial, allow_* verdict modifiers, host-path leaks, egress, MCP, RSS, semantic judging, artifact_json, computer:// links.
 
+**A multi-leg scenario's `host_action` leg** takes its own keys plus the file keys only (it has no transcript):
+
+| key | passes when |
+|---|---|
+| `exit_code: <0-255>` | the action exited with that code (with no `exit_code` item, 0 is expected) |
+| `stdout_matches: <regex>` | the action's stdout (capped, secret-scrubbed) matches; a no-hit on a truncated stdout is evidence-unavailable, not a fail |
+| `stdout_not_matches: <regex>` | the action's stdout does not match |
+| `file_exists`, `file_absent`, `artifact_json`, `artifact_text` | as below, against the leg's tree after the action |
+
+A prompt leg takes every key in the per-family files. The generated lists are `multiLeg.*` in
+`scripts/assertion-keys.json`; the form is in [`scenario-schema.md`](./scenario-schema.md#multi-leg-scenario-yaml-legs).
+
 **Name-matching styles differ by key (don't mix them up):** `tool_called`, `tool_not_called`,
 `subagent_tool_used`, `subagent_tool_absent` are **glob** (anchored, case-sensitive, `*`/`?`). `tool_available`,
 `skill_triggered`/`no_skill_triggered`, `skill_available`, `connector_available`, `skill_tool_used`,
@@ -38,9 +50,9 @@ paraphrases (that re-records red). Structured JSON → assert it in YAML with **
 `path` + operator, or `schema:` for a whole value's shape); use the pytest lane (`assert_artifact_json`) only for
 predicates neither can express.
 
-**VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; eleven
+**VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; twelve
 are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
-`"success"`) and thirteen are **warn**-severity (informational, never flip pass/fail). All twenty-four signal
+`"success"`) and fifteen are **warn**-severity (informational, never flip pass/fail). All twenty-seven signal
 codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 | Code | Severity | Meaning |
@@ -53,7 +65,10 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 | `outputs_delete` | fail | An unauthorized delete touched `mnt/outputs` on a baseline that records outputs `rw` (not raised on `rwd`, Desktop 2.16120.0+, unless `no_delete_in_outputs` or `no_delete_in_mounts` (outputs not waived) is authored — with `no_delete_in_outputs` that assertion fails instead; with `no_delete_in_mounts` this signal itself fires, and its message names that key), confirmed: the per-turn filesystem diff proves it, a delete in command/call position has an `outputs/` path as its own operand, or the diff could not verify the turn. Authoring `no_delete_in_outputs` moves it into that assertion; `allow_outputs_delete` waives it |
 | `outputs_delete_unconfirmed` | warn | A delete-shaped command near `mnt/outputs` nothing confirms: no output present at turn start was deleted and no flagged delete has an outputs path as its own operand (e.g. a Python variable named `rm` next to an outputs path, quoted prose, a `sed`/`grep` pattern). A *statement* is a fragment split on newline/`;`/`&&`/`\|\|`, quote-blind, so some real deletes also land here — a loop body whose operand is the loop variable (`for f in …; do rm "$f"; done`), a `cd` then a relative path, chained variables (`A=…; B=$A/x; rm "$B"`), a Python path held in a variable set on another line (`p = …` then `os.remove(p)`, or `for p in …:` then `p.unlink()`), wrappers with flag combinations the classifier does not model (`sudo -Hu user rm`, `git -C dir rm`), and calls outside the modelled set such as Node's `fs.promises.rm(…)` — read the command. Raised even when `no_delete_in_outputs` is authored (that assertion passes on it); not raised on an `rwd` baseline (Desktop 2.16120.0+) unless that key, or `no_delete_in_mounts` with outputs not waived, is authored. Kept false positives that still fail `outputs_delete`: quoted text in which a delete command with an outputs operand follows a shell separator, subshell or keyword — the classifier does not track quotes (`echo 'note; rm mnt/outputs/x'`, `echo "a & rm …/outputs/x"`), and a heredoc that *writes* a script rather than running it (`cat <<EOF > clean.sh` with an `rm …/outputs/x` line). Waive: `allow_outputs_delete` |
 | `outputs_diff_unavailable` | warn | The per-turn filesystem diff of `outputs/` could not verify this turn and the text scan flagged nothing — a delete made without a bash command would have gone undetected. With a text hit the turn fails `outputs_delete` instead. Like `outputs_delete_unconfirmed`, not raised on an `rwd` baseline unless `no_delete_in_outputs` or `no_delete_in_mounts` (outputs not waived) is authored |
-| `mount_delete` | warn | A delete touched a delete-denied mount other than `outputs` (a `rw` connected folder). Production denies `unlink`/`rmdir` there until per-mount approval, so the run diverged. Warn, not fail: the harness detects post-hoc what production enforces. Author `no_delete_in_mounts` to hard-fail, or `allow_delete_in` to waive |
+| `mount_delete` | warn | A delete touched a delete-denied mount other than `outputs` (a `rw` connected folder). Production denies `unlink`/`rmdir` there until per-mount approval, so the run diverged. Warn, not fail: the harness detects post-hoc what production enforces. Author `no_delete_in_mounts` to hard-fail, or `allow_delete_in` to waive. Not raised under `device_bridge` once the device refuses deletes (`deleteRefusal: "enforced"`): a delete left there is one the user allowed |
+| `device_delete_bypassed` | fail | `device_bridge`: a file is gone from a connected folder deletion was never allowed in, past the device's refusal (a static binary or raw system call; `deviceBridge.bypassedDeletes` names them). Production would have refused it. Not waivable by `allow_delete_in`. Where the filesystem can read a rename followed by an edit as a delete (`rename-reads-as-delete`) it is `device_delete_unconfirmed` instead |
+| `device_delete_unconfirmed` | warn | `device_bridge`, on a filesystem that can give a new file a deleted file's identity (`rename-reads-as-delete`): files are gone from a connected folder deletion was never allowed in, either deleted past the device's refusal or renamed and then edited, which that filesystem cannot tell apart |
+| `device_audit_unreadable` | warn | `device_bridge`: the shim's delete log was replaced by the device shell (a link, a FIFO), so the number of refused deletes, and deletes of files the session made after a grant, are unknown (`deviceBridge.auditUnreadable`; `refusedDeletes` is left out). The before-and-after comparison of the folders still ran |
 | `host_path_leak` | fail | A host path leaked into model-visible text (opt out: author `transcript_no_host_path`). A path copied verbatim from the scenario's own uploads, connected folders, prompt, or declared plugins' or local skills' files is not a leak |
 | `l0_host_config_contamination` | fail | `protocol` ran against the operator's REAL config dir, so host-installed plugins/skills/MCP servers may have answered instead of the thing under test — the run did not necessarily measure it (opt out: `allow_l0_host_config_contamination`). Name predates the meaning: delivery at L0 works, contamination is what this reports |
 | `missing_capability` | fail | A `requires_capabilities` need was unmet, or the skill used a capability the image omits (opt out: `allow_missing_capability`, or `skill --allow-missing-capability` on an open-ended run) |
@@ -72,4 +87,4 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 A **fail**-severity signal does not change `result.result` (still `"success"`), but it DOES fail the
 overall run verdict and exit code — `assert result: success` alone won't catch it; check
-`result.verdict.signals[].severity` or the run's exit code. Only the thirteen **warn** codes are truly benign.
+`result.verdict.signals[].severity` or the run's exit code. Only the fifteen **warn** codes are truly benign.

package/.claude/skills/cowork-harness/references/assertions-guide.md

@@ -1,6 +1,6 @@
 # Assertions guide
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
 
 ### Assertions: two orthogonal axes
 

package/.claude/skills/cowork-harness/references/authoring.md

@@ -1,6 +1,6 @@
 # Authoring a scenario
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
 
 ## Part I — AUTHOR a scenario
 
@@ -79,12 +79,24 @@ computer, `mcp__remote-devices__device_bash`, from a second container holding CO
 `$HOME/mnt/<folder-name>`; the agent's own container gets no folder. Assert on that call, never on the folder's
 files: `tool_called: {tool: mcp__remote-devices__device_bash, result: {matches: <text>}}`, plus `transcript_matches`.
 `file_exists` / `input_unmodified` on a connected folder's path are refused under the bridge (they would read the
-agent's container). `no_delete_in_mounts` covers the folders (the device copy allows a delete production refuses).
-`tool_not_called` on `device_bash`/`get_device_info` without the bridge, or on a device tool the harness never serves
+agent's container). A delete in an `rw` folder is refused until the user allows it through the delete card,
+`mcp__remote-devices__device_request_delete_permission`, as in production: a skill that deletes there needs a
+scripted answer for the card (`when_tool:` it, `decide: allow` or `deny`) or the run fails as unanswered; an `rwd`
+folder allows deletes from the start. A move between connected folders copies and cannot remove its source, and git
+leaves its lock files so the next commit fails on `HEAD.lock`, as in production; a rename within one folder
+succeeds. `no_delete_in_mounts` covers the folders, an allowed delete included. A delete
+that gets past the refusal fails the run (`device_delete_bypassed`). The first bridge run builds the device image
+locally (the agent image plus the delete library; needs Docker Hub and apt, takes a few minutes) and rebuilds it when
+the agent image changes.
+`tool_not_called` on `device_bash`/`get_device_info`/`device_request_delete_permission` without the bridge, or on a device tool the harness never serves
 (with it or without), is refused as vacuous. Egress follows the session allowlist;
 `egress.unrestricted: true` matches production's pass-through. With no `permission_mode` in the session the agent
-runs in `acceptEdits` (a production cloud task's mode under Manual); a mode you set wins. The run's `deviceBridge` lists every known
-difference from production.
+runs in `acceptEdits` (a production cloud task's mode under Manual); a mode you set wins. The delete card
+(`mcp__remote-devices__device_request_delete_permission`) is served and never answered by the harness: script it with
+`when_tool: … decide: allow|deny`, or an unscripted card fails the run as an unanswered gate (`first` declines it).
+Under `permission_mode: bypassPermissions` a delete is granted at once with no card (`answeredBy: bypassPermissions`).
+A `--decider-cmd` reply to the card is `{behavior}` only; a `grant` scope in it fails the run. The
+run's `deviceBridge` lists every known difference from production, and `grants` who allowed what.
 
 ### Choose an answer path (gates: AskUserQuestion + tool-permission)
 
@@ -204,7 +216,7 @@ The `"first"` shorthand remains active only for the built-in `--on-unanswered fi
 write an external helper, return a label name or option index — never the bare word `"first"` unless
 your gate actually has a label called `"first"`.
 
-### web_fetch (fail-closed, two-path)
+### web_fetch (never auto-allowed, two-path)
 
 `web_fetch` behaves unlike `curl`. A URL is gated by **provenance**, not the egress allowlist:
 
@@ -212,14 +224,29 @@ your gate actually has a label called `"first"`.
   make a fetch succeed, put the URL in the prompt.
 - **Provenanced** → fetches (still SSRF-guarded per redirect hop); the egress hostname allowlist is
   **not consulted**.
-- **Not provenanced** → raises a per-domain approval gate (`webfetch:<domain>`) that is
-  **fail-closed** (it is *not* auto-allowed; `--on-unanswered first` won't allow it). Answer it with
-  a scripted rule (`when_tool: "webfetch:<domain>"` + `grant: domain|once`), a session
+- **Not provenanced** → raises a per-domain approval gate (`webfetch:<domain>`) that is never
+  auto-allowed. Unanswered, it **ends the run as an unanswered gate** under `on_unanswered: fail`
+  (the `run` and `record` default), naming the domain, the URL and the rule to add; `--on-unanswered first` denies
+  it, and `llm` picks one of its three buttons (`Allow once`, `Allow all for website`, `Deny`). Answer it with a
+  scripted rule (`when_tool: "webfetch:<domain>"` + `decide: allow` and `grant: once|domain`, or `decide: deny`), a
+  session
   `web_fetch.approved_domains`, or a live decider.
 
 Surprise to remember: adding a host to `egress.extra_allow` is a **no-op** for a provenanced fetch.
 Full model in `references/scenario-schema.md`.
 
+### A skill that parks and resumes in a later session: `legs:`
+
+When the skill stops (a question, a gate, a status file) and a NEW session resumes it, author ONE multi-leg scenario
+instead of two scenarios glued together: `legs:` in place of `prompt:`, each leg a `prompt` (a fresh agent session)
+or a `host_action` (an argv command in the previous leg's runtime, asserted with `exit_code`, `stdout_matches`,
+`stdout_not_matches` and the file keys). Pick `workdir: persistent` (one tree across legs — a persistent workspace,
+not Cowork) or `workdir: cowork-task` (a new tree per prompt leg, folders carried — a model of a new Cowork task).
+Judge completion from the skill's own status file and ledger (`artifact_json`), never from the agent's prose. The
+YAML is in [`scenario-schema.md`](./scenario-schema.md#multi-leg-scenario-yaml-legs), the walkthrough is Recipe 10 in
+[`task-recipes.md`](./task-recipes.md), and the rules are in [docs/scenario.md → "Multi-leg scenarios"](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md#multi-leg-scenarios-legs).
+`scaffold` does not generate the multi-leg form; start from the resumable-probe example Recipe 10 links.
+
 ### Scaffold a valid scenario, then lint before you push
 
 Don't hand-write the YAML from memory — that's how invented keys (`assertions:` vs `assert:`,

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`).
+Self-contained reference. Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
 copy-pasted recipe that omits the input takes a major bump with no say in it. `^4` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "4.8.1"`) instead when you want byte-reproducible CI.
+(e.g. `version: "4.9.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -82,7 +82,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^4.8.1"
+- run: npm i -g "cowork-harness@^4.9.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -411,7 +411,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^4.8.1"
+      - run: npm i -g "cowork-harness@^4.9.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -440,7 +440,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^4.8.1"
+        run: npm i -g "cowork-harness@^4.9.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). This is **not** a trim of the full
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reading several critiques, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's
@@ -224,6 +224,9 @@ harness majors, mixed `corpusHash` within a group, the same critique twice, a su
 field, a pre-4.8 report. Marked, and refused
 by `--strict`: mixed evaluator models / hash bases / skill trees / probes, a pass-1-only member. Marked only:
 `mixedPackagedCorpus`, `probeUnverified` (no prompt hash — the summary default).
+`--cluster lexical` (default `none`) adds `candidateGroups`: citation-resolved findings with text, from full reports, whose wording is close, from at least two reports
+(complete linkage, every pair at or above a fixed cut of 0.20). A judgement from word overlap, not a match: no k/N, input order, a
+finding in no group may still have recurred in other words. Local, nothing sent; `--cluster judge` is refused.
 Its output carries finding text — not public-safe. Details: [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only).
 
 ## An untracked skill file is not graded

package/.claude/skills/cowork-harness/references/debugging.md

@@ -1,6 +1,6 @@
 # Debugging a run
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
 
 ## Part III — Debug
 
@@ -109,7 +109,9 @@ decide which assertions from *Assertions: two orthogonal axes* in `assertions-gu
   to check a skill's delivery survives the cloud lane, where new Pro and Max tasks run from 2026-10-06. Orthogonal to `fidelity` — a `lane: remote`
   scenario still runs locally. Add `device_bridge: true` (with `fidelity: container`) to serve `device_bash`, the
   cloud task's shell on the user's computer; a red there reads from the `device_bash` tool calls and the run's
-  `deviceBridge` record, and a `device-container` / `device-exec` infra error is that second container.
+  `deviceBridge` record, and a `device-container` / `device-exec` infra error is that second container (`device-shim`: the device image could not be prepared, or the bridge's probe delete was not refused). The bridge
+  also serves the delete card (`device_request_delete_permission`): an unscripted one ends the run as an unanswered
+  gate, so script it (`when_tool: … decide: allow|deny`).
 - **`cowork-harness stats [--metric <m>]`** — aggregate across the run index: `cost`, `duration`,
   `tokens`, `cache-tokens`, `model-cost`, `turns`, `pass-rate`. Filters: `--since`/`--baseline`/`--branch`,
   plus `--skill-hash <prefix>`/`--label <tag>` to narrow to ONE skill generation and

package/.claude/skills/cowork-harness/references/eval.md

@@ -1,6 +1,6 @@
 # `eval` — paired before/after comparison of a skill edit (EXPERIMENTAL)
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). The full guide is
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). The full guide is
 [docs/eval.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/eval.md); this is the part you
 need while running it.
 
@@ -20,8 +20,9 @@ cowork-harness eval report <eval-dir>     # rebuild the report from the eval dir
   not the `remote_plugins` path a UI-installed plugin has (`mnt/.remote-plugins/plugin_<id>`). A skill that
   locates its own files at runtime sees a different path under each, so the comparison holds for the
   `local_plugins` layout only.
-- Every row is a pass or a fail per rep, and `eval` compares pass rates. A scenario's `metrics:` are not graded or
-  compared here; `hillclimb` records and compares them.
+- Pass rows are a pass or a fail per rep, and `eval` compares pass rates. Each number a scenario declares under
+  `metrics:` is a metric row, compared with an exact rank test in its own correction family (see
+  [Metric rows](#metric-rows)).
 - scenarios × 2 × `--reps` live runs (10 per scenario at the default `--reps 5`), interleaved, plus one judge
   call per `semantic_matches` assert per run. The start-up line prints the job count.
 - Before an A/B, plan it: `eval … --dry-run --target-effect 30pp` runs nothing and prints, from the runs
@@ -35,7 +36,8 @@ cowork-harness eval report <eval-dir>     # rebuild the report from the eval dir
 - Run an A/A first (`--allow-identical-arms`, the same source twice) to see your scenarios' noise.
 - A directory arm's snapshot leaves out untracked files unless you pass `--include-untracked` (not with a
   `git:` arm). `--correction bh|holm` picks the multiple-comparison correction behind `confirmed` (default
-  `bh` at q = 0.10; `holm` is stricter).
+  `bh` at q = 0.10; `holm` is stricter). Metric rows default to holm at `--alpha` split across the section's
+  families (0.025 beside pass rows); an explicit `--correction bh` grades them with BH at q = 0.10, unsplit.
 
 ## Reading the labels
 
@@ -47,6 +49,9 @@ cowork-harness eval report <eval-dir>     # rebuild the report from the eval dir
 | `underpowered` | no outcome at these sizes could reach `--alpha` — NOT "no change" |
 | `insufficient` | too few valid reps in an arm (4 of 5 needed by default) |
 
+Metric rows say `confirmed/possible worse|better` (read through the metric's `better`), plus `unresolved`,
+`insufficient_unmeasured` and `not compared` ([below](#metric-rows)).
+
 A drop is a signal to investigate, not proof: open the run dirs the report links for that row. In order,
 first match wins: an infrastructure failure is excluded and reported — including a usage limit the run lane
 named (the main loop's or a sub-agent's, rule `kind_usage_limit`, even on a rep that otherwise timed out or hit
@@ -70,10 +75,11 @@ names the arm, scenario, dominant error and a matching hint (`Every rep of arm <
 - a model or judge that is an alias (`opus`, `best`) rather than a concrete id;
 - an eval dir inside any git work tree (the snapshots would mount empty);
 - identical arms (unless `--allow-identical-arms`);
-- an arm that contains the eval's own scenario or session files (by location, copy or symlink), an
-  `evals.json`, or a symlink resolving outside it;
+- an arm that contains the eval's own scenario or session files, or a command metric's script or deps (by location,
+  copy or symlink), an `evals.json`, or a symlink resolving outside it;
 - a scenario input a run would refuse (a missing path, a `tool_not_called` the tier can never violate);
-- `--fail-on confirmed` when no row could reach `confirmed` at this `--reps`;
+- `--fail-on confirmed` when no family (pass rows or metric rows, in any section) could reach `confirmed` at this
+  `--reps`; when only some can, the start-up line names the families the gate can fire on;
 - no usable agent credential for a scenario's tier — the same check as `doctor --tier <tier>`'s `token` row,
   with its fix. A Keychain login or a `.credentials.json` in the config dir, without an env/.env token,
   passes only at `protocol`;
@@ -81,10 +87,35 @@ names the arm, scenario, dominant error and a matching hint (`Every rep of arm <
   one `local_plugins` entry). Workaround: eval a copy of the session that declares the same directory under
   `local_plugins`, and check the `remote_plugins` path handling with an ordinary `run`.
 
+## Metric rows
+
+- One row per (scenario, declared metric). The test is exact two-sided Wilcoxon–Mann–Whitney, so ties and skewed
+  numbers are fine. Each row also shows the Hodges–Lehmann shift with its ACHIEVED coverage (96.8% at 5 v 5), each
+  arm's SD and the minimum detectable shift (`mds`).
+- **At 5 reps a metric row reliably flags only a shift of about 2.4 SD** (2.0 at 6 reps): near-complete separation.
+  Every metric row is `underpowered` at `--reps 3`; use `--reps 5`, since at `--reps 4` one lost rep silences it.
+  A metric whose every value is equal is `underpowered`, never "no change".
+- A rep where the agent failed, or the skill did not produce the number (`missing_artifact`, `missing_path`,
+  `not_json`, `not_a_number`, `pre_run`), ranks WORST, below every measured value. A rep whose number is missing
+  because of the scorer, the capture or a cap (`size`, `readonly`, `remote`, and a command metric's
+  `command_failed` / `command_timeout` / `command_output` / `nondeterministic` / `no_events`) is EXCLUDED and
+  guarded: no verdict may rest on it, so a row that would flag only while those reps stay unknown is `unresolved`
+  (never gates). The candidate's ≥ 2 extra excluded reps that took a row below the threshold: `insufficient_unmeasured`
+  (`--fail-on possible` gates). `pruned` / `no_manifest`, a completed rep with no entry for the id
+  (`metric_unrecorded`), or a command metric's `command_changed` / `command_missing` / `command_not_allowed` (a rep
+  scored by another script than the manifest froze): `not compared`, exit 1.
+- At 5 reps a metric can be `confirmed` beside pass rows only when the section declares 3 metrics or fewer; 4–11
+  need `--reps 6`. Every report states each family's correction, row count and A/A false-confirm rate.
+- Worth it for count metrics on a targeted defect fix (a ×3.3 rate change is `possible` at 5 reps, ×4.1
+  confirms). Otherwise run an A/A first and read each row's `sd` and `mds`: if `MDS possible` (in the metric's units)
+  is larger than the change you hope for, raise `--reps` or change the metric.
+- An eval dir written before metrics were graded reports them as not recorded (exit code unchanged).
+
 ## Exit codes
 
-`0` completed — no drop fails the eval unless you pass `--fail-on`. `1` a drop at the `--fail-on` level,
-every row `insufficient`, a scenario that compared nothing, or the judge model differed across reps (an A/A run under `--fail-on possible`
+`0` completed — no drop fails the eval unless you pass `--fail-on`. `1` a drop or a worse metric at the `--fail-on`
+level, every row `insufficient`, a scenario that compared nothing, a metric row `not compared`, or the judge model
+differed across reps (an A/A run under `--fail-on possible`
 can exit 1 on noise). `2` usage or a refusal. `3` an arm snapshot could not be copied or staged.
 
 ## Files

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`).
+Self-contained reference. Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`).
 
 > **This page vs. the repo docs.** This is the **offline snapshot** that ships inside the installed
 > plugin — it is self-contained on purpose. The repo carries four other fidelity views, each answering a
@@ -243,7 +243,8 @@ prevent, so use it to unblock once, never in CI.
   at the cost of non-determinism (one model call per gate). Correct, but flaky for skills whose gates
   appear stochastically.
 - `first` — picks option 1 and warns loudly. **Flagged `nonDeterministic`** — not a deterministic
-  substitute for scripted answers. (For a web_fetch approval gate it abstains → fail-closed.)
+  substitute for scripted answers. (For a web_fetch approval gate it abstains, and the fetch is denied; for the device bridge's delete card it picks
+  `Decline`.)
 - `prompt` — asks at the TTY (`skill` only).
 - `llm` — a model answers; flagged `nonDeterministic`.
 

package/.claude/skills/cowork-harness/references/gotchas.md

@@ -1,6 +1,6 @@
 # Gotchas
 
-Tracks `cowork-harness 4.8.1` (baseline `desktop-2.31226.1`). The full "✓ passed ≠ correct" landmine catalog.
+Tracks `cowork-harness 4.9.0` (baseline `desktop-2.31226.1`). The full "✓ passed ≠ correct" landmine catalog.
 
 ## Gotchas — the "✓ passed ≠ correct" landmines
 
@@ -69,9 +69,10 @@ authorable). Reach for this list when debugging a run's behavior, that one while
 7. **`protocol` is rejected (not silently passed) if the scenario asserts egress** — boundary
    assertions need a sandboxed tier (`container`+). Good: this one fails loud by design.
 
-8. **Read-only mounts are enforced; delete-deny is a HARNESS gap — production DOES enforce it.**
-   Production enforces it on `rw` mounts; which mounts are `rw` is below. `mode:r` mounts get a real `:ro` bind (a write fails in-guest). But `rw` vs `rwd`
-   (write-but-no-delete) is *not* mount-enforced **in the harness** — `rm` succeeds and is only caught
+8. **Read-only mounts are enforced; delete-deny is enforced only under `device_bridge` — production enforces it everywhere.**
+   Production enforces it on `rw` mounts; which mounts are `rw` is below. `mode:r` mounts get a real `:ro` bind (a write fails in-guest). On the local lane
+   (every tier without `device_bridge`) `rw` vs `rwd` (write-but-no-delete) is *not* mount-enforced — `rm` succeeds
+   and is only caught
    post-hoc. **Real Cowork enforces it live on a `rw` mount:** a FUSE mount where `unlink`/`rmdir` fail
    `Operation not permitted`; a skill must request approval via `allow_cowork_file_delete` (which
    re-mounts the folder `rwd` mid-session) to delete. **Which mounts are `rw` depends on the release:** a
@@ -87,8 +88,8 @@ authorable). Reach for this list when debugging a run's behavior, that one while
    (`EXDEV`, then `EPERM` on the copy-then-unlink fallback), so that stays a delete; on an `rwd` outputs
    mount the fallback's unlink should succeed, but that is not probed. Consequences on a `rw` mount: a
    skill should not stage disposable scratch there (cleanup costs an approval prompt), and a skill's
-   "catch-EPERM-then-request-approval" branch cannot be exercised at any harness tier (the `rm` just
-   succeeds here). On an older `rw` baseline, if a scenario's deletion IS intended, assert
+   "catch-EPERM-then-request-approval" branch can be exercised only under `device_bridge`, where the device refuses
+   the delete and serves the delete card; on the local-lane tiers the `rm` just succeeds. On an older `rw` baseline, if a scenario's deletion IS intended, assert
    `allow_outputs_delete: true` rather than dropping `no_delete_in_outputs` (on `rwd` it is a no-op unless
    `no_delete_in_mounts` arms the outputs check, where it waives that check as on `rw`).