cowork-harness 4.14.0 → 5.0.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, asserting artifacts, egress, or sub-agent dispatch, measuring how long each tool call took (toolDurations / trace), or debugging a failed run or verdict from its result.json or transcript. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. Also for comparing two versions of a skill before merging an edit — did it make answers worse? (`eval`: paired A/B, pinned models) — or improving one round by round (`hillclimb`, the `/claude-api hillclimb` runner). NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold / critique / stats / eval / hillclimb commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 4.14.0
-  tracks-harness: cowork-harness 4.14.0 (baseline desktop-4.16336.0)
+  version: 5.0.0
+  tracks-harness: cowork-harness 5.0.0 (baseline desktop-4.23118.0)
 ---
 
 # cowork-harness
@@ -26,8 +26,8 @@ allowlist). This skill exists mostly to keep you out of those traps — the *Inv
 full landmine catalog in [`references/gotchas.md`](references/gotchas.md) are the highest-value part.
 Read them.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.14.0` (baseline
-> `desktop-4.16336.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 5.0.0` (baseline
+> `desktop-4.23118.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## Preflight — make sure the harness can actually run
@@ -43,14 +43,14 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, the baseline and the `device_bridge` image (built, or built on the first bridge run) in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.14.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.14.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.14.0"`. **Pin `@^4.14.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 5.0.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^5.0.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^5.0.0"`. **Pin `@^5.0.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
   — upgrade rather than work around it, since this skill's `file:line` pointers and flag names track the floor.
 - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
 - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass. At `protocol` the plugin under test **is** delivered (`--plugin-dir`), but its hooks then run as native host processes — so a plugin declaring hooks is refused there until you pass `--allow-host-hooks` / `allow_host_hooks: true`.
-- **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred), or `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
+- **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY` (`ANTHROPIC_AUTH_TOKEN`: `protocol` only), via env or `.env` — at every tier, `protocol` included (a Keychain or `/login` session is never used). With the token set, the agent gets it alone at `container`/`microvm`/`hostloop` (at `hostloop` it gets only Desktop's named shell vars plus `agent_env.passthrough`; at `protocol`, with `ANTHROPIC_API_KEY` set, the agent gets the key and not the token, since the harness passes the token there only when no key is set), and the graders (judge, LLM decider, evaluator) get it alone — `COWORK_HARNESS_GRADER_AUTH=inherit` passes every key ([semantic-judging](references/semantic-judging.md)). Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
 - **Point at another `.env`** with `--dotenv <path>`, and relocate run output with `--run-dir <path>`; both work before or after the subcommand.
 
 ## Orient — the three loops
@@ -67,6 +67,7 @@ CI-grade scenario, and the post-hoc debug loop; the rest are narrower tools that
   no scenario file.
 - **Repeatable, asserted regression** → author a `scenarios/*.yaml` and run `cowork-harness run`.
   This is the CI-grade path and most of this skill.
+- **Against the plugins and skills you have installed** → `plugins.from_host` ([schema](references/scenario-schema.md)).
 - **A run failed — or greened and you don't trust it** (the debugging loop) → **Read
   [`references/debugging.md`](references/debugging.md) before touching the run.** Its *Triage* table first
   splits "the skill misbehaved" from "a green you don't trust" — they need different tools — then names the
@@ -104,9 +105,9 @@ CI-grade scenario, and the post-hoc debug loop; the rest are narrower tools that
 > `node_modules/cowork-harness/docs/<name>.md` before assuming a pointer dangles. A **plugin**
 > install loads a trimmed source-only cache where those pointers genuinely do dangle.
 
-Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · migrate-run-dir · lint ·
+Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · lint ·
 lint-skill · analyze-skill · probe-dispatch ·
-verify-run · regrade · fixture · search-pool <build|export|import> · ref <freeze|verify> · trace · inspect · diff · critique · eval · eval report · hillclimb <run|check|state-template|freeze-ref|regrade> · stats · decide · gates · answer · scaffold · assertions --list · sync ·
+verify-run · regrade · fixture · search-pool <build|export|import> · ref <freeze|verify> · trace · inspect · diff · critique · eval · eval report · hillclimb <run|check|state-template|freeze-ref|regrade|seed> · stats · decide · gates · answer · scaffold · assertions --list · sync ·
 list · boundary-check · status · vm <init|status|delete|prune> · doctor · init-redact`. Always check `cowork-harness <cmd> --help`.
 
 ## Invariants — how a green run lies

package/.claude/skills/cowork-harness/references/assertion-catalog-agents-skills-budgets.md

@@ -1,6 +1,6 @@
 # Assertion catalog: sub-agents, skills and connectors, budgets, tasks, delivery
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 
@@ -8,7 +8,7 @@ keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 |---|---|
 | `subagent_tool_used: <glob>` | a sub-agent used a tool matching this glob (same `*`/`?`, anchored, case-sensitive semantics as `tool_called`, including the empty/regex-ish rejection) |
 | `subagent_tool_absent: <glob>` | no sub-agent used a tool matching this glob (same rejection) |
-| `no_vm_path_file_op: true` | **`fidelity: hostloop` only** — NO gated file tool attempted a `/sessions`(-prefixed) path (`RunResult.fileToolAttempts`) — content-class, replay-checkable without `controlOut`; any other tier FAILS "cannot verify" (`/sessions/...` is valid there). **Only `true` is valid** |
+| `no_vm_path_file_op: true` | **`fidelity: hostloop` only** — NO gated file tool attempted a `/sessions`(-prefixed) path (`RunResult.fileToolAttempts`) — content-class, replay-checkable; any other tier FAILS "cannot verify" (`/sessions/...` is valid there). **Only `true` is valid** |
 | `subagent_file_write: {path?, path_suffix?, tool?}` | a sub-agent-origin write attempt whose raw path equals `path` (exact) or ends with `path_suffix` has a paired non-error tool_result — the causal half of a delivery probe; requires one of `path`/`path_suffix`; `tool` defaults to Write/Edit/MultiEdit; content-class; tier-agnostic |
 | `subagent_dispatch_healthy: {type?, delivered?, path?, path_suffix?, no_vm_paths?}` | **`fidelity: hostloop` only** — composite: selects dispatch(es) via `type` (same matching as `subagent_dispatched`; omit to require every dispatch) and, for EACH selected dispatch, checks it (not just any sub-agent) delivered a paired non-error write (`delivered`, default true — narrowed by `path`/`path_suffix`, same exact-vs-suffix precedence as `subagent_file_write`) and made no `/sessions` VM-path attempt (`no_vm_paths`, default true) — both scoped to that dispatch's OWN `parentToolUseId`, the per-dispatch correlation `subagent_file_write` (which matches ANY sub-agent write) cannot express; a `type` that matches no dispatch FAILS; content-class (`RunResult.fileToolAttempts` + `RunResult.toolResults`); any non-hostloop tier FAILS "cannot verify" |
 | `subagent_dispatched: <regex>` | a sub-agent whose `dispatchAgentType`, binary-*resolved* `resolvedAgentType`, **or dispatch description** matches |

package/.claude/skills/cowork-harness/references/assertion-catalog-gates-hooks-modifiers.md

@@ -1,6 +1,6 @@
 # Assertion catalog: gates, hooks, path denial, verdict modifiers, egress, judged keys
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md).
 
@@ -18,20 +18,20 @@ refused at load: no question reaches the harness to grade.
 | `gate_answers_delivered: false` | asserts at least one answered gate's answer was **confirmed not delivered** (an observed delivery failure); an unobserved/null delivery does **not** satisfy this — for negative-path delivery tests. Requires a gate, so **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
 | `gate_answer_count_min: <N>` | at least N AskUserQuestion gates fired AND were delivered non-error — presence companion to `gate_answers_delivered`'s vacuous-pass. **`: 0` asserts nothing** and does not satisfy that pairing; `>= 1` is **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
 | `gates_all_scripted: true \| {include_permissions: true}` | every fired AskUserQuestion gate was answered by a scripted rule (`answers:`/`--answer`), not the LLM decider, `first` or a human; `include_permissions` adds tool permissions (`when_tool`/fixed rules count; a web_fetch deny that no answer source decided fails (`answered by abstain-fallback`), reachable only under `first`, not `on_unanswered: fail`). Replay re-classifies on the frozen `answers:`; zero gates passes, so pair with `gate_answer_count_min: >= 1` or `questions_count_max` (`lint`: `unpaired-gates-all-scripted`); details in the scenario reference (linked above) |
-| `hook_blocked: <regex>` | one of the harness's own PreToolUse hook callbacks blocked a tool whose name matches the regex (`RunResult.hookEvents`; a plugin's hook never reaches it: use `hook_event_blocked`) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette (a custom hook's decision lives only there, not the recorded stream). **Does not see the agent's own refusal**: on `hostloop` against Desktop 2.7032.0+ a relative `Read`/`Write`/`Edit` is denied by the agent's permission rules before any hook runs, so neither this key nor `path_denied` records it — assert it with `tool_result_contains: "denied by your permission settings"` |
-| `no_hook_blocked: true` | no tool was blocked by the harness's own hook callbacks (a plugin's hook never reaches it: use `no_hook_event_blocked`) (distinguishes a real tool crash from an intentional hook block) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette. **Only `true` is valid** |
+| `hook_blocked: <regex>` | one of the harness's own PreToolUse hook callbacks blocked a tool whose name matches the regex (`RunResult.hookEvents`; a plugin's hook never reaches it: use `hook_event_blocked`) — evidence-unavailable if hook telemetry is absent. Replay reads a custom hook's decision from `controlOut` (not the recorded stream). **Does not see the agent's own refusal**: on `hostloop` against Desktop 2.7032.0+ a relative `Read`/`Write`/`Edit` is denied by the agent's permission rules before any hook runs, so neither this key nor `path_denied` records it — assert it with `tool_result_contains: "denied by your permission settings"` |
+| `no_hook_blocked: true` | no tool was blocked by the harness's own hook callbacks (a plugin's hook never reaches it: use `no_hook_event_blocked`) (distinguishes a real tool crash from an intentional hook block) — evidence-unavailable if hook telemetry is absent. **Only `true` is valid** |
 | `hook_event_fired: <HookEvent>` | a **command hook** for this event (a plugin's `hooks/hooks.json` or manifest hook — `Stop`, `SessionStart`, `PostToolUse`, …) ran: a `hook_response` system frame with that `hook_event` was recorded (`RunResult.contextEvents`). Any outcome counts. The harness passes `--include-hook-events` whenever a staged plugin declares hooks — that is what puts events other than SessionStart/Setup on the stream — so a recording made without it reports "never fired". Content-class, grades on replay. Recorded end-to-end for `Stop` ([stop-hook-probe.scenario.yaml](https://github.com/yaniv-golan/cowork-harness/blob/main/examples/probes/stop-hook-probe.scenario.yaml)) and for `PreToolUse` at `container` (deciding by JSON and by exit 2); the other names match the same frame but have not each been recorded |
 | `hook_event_blocked: <HookEvent> \| {event, tool?, via?, min?, max?}` | a command hook for `event` blocked: the bare form counts exit-2 frames only; the object form also counts a JSON deny on an `outcome: success` frame (`via: exit2`/`json`/`any`, default any), `tool` = the tool that fired (`Bash` at container, `mcp__workspace__bash` at hostloop), `min`/`max` a range (`{event: Stop, max: 0}` is the negative). No frame in scope fails "never fired"; an unreadable frame (incl. a failed one on an `onFailure: "block"` hook) is evidence-unavailable when it could change the verdict. Content-class; details in the scenario reference (linked above) |
-| `no_hook_event_blocked: true \| {event, tool?}` | no command hook blocked: every `hook_response` frame (or every frame for `event`) neither exited 2 nor denied by JSON, the same rule as `hook_event_blocked`'s object form. **Never vacuous**: no frame in scope is evidence-unavailable, and `true` also needs a frame of an event other than SessionStart/Setup, because those stream even when hook events were not requested (they stream only when a staged plugin declares hooks). A frame whose outcome cannot be read (no exit code, a hook that started and never answered, stdout a redaction policy rewrote or the agent truncated, output the agent did not apply as a verdict, or a failed or timed-out frame on an event with an `onFailure: "block"` hook — see `hook_decision`) is evidence-unavailable. Hooks on PreCompact, PostCompact, ConfigChange, DirectoryAdded, Elicitation, ElicitationResult, InstructionsLoaded, Notification, SessionEnd, StopFailure, WorktreeCreate, WorktreeRemove, CwdChanged and FileChanged stream no frame at all, so a block there never shows: `true` is evidence-unavailable when one of those events has an `onFailure: "block"` hook, or when the run's hook sources could not all be read. Frames carry no plugin id, so a second staged plugin's hook, or a host plugin's at `protocol`, counts as a frame here (the run warns). Distinct from `no_hook_blocked`, which reads the harness's **own** PreToolUse callbacks from `controlOut`. Content-class |
+| `no_hook_event_blocked: true \| {event, tool?}` | no command hook blocked: every `hook_response` frame (or every frame for `event`) neither exited 2 nor denied by JSON, the same rule as `hook_event_blocked`'s object form. **Never vacuous**: no frame in scope is evidence-unavailable, and `true` also needs a frame of an event other than SessionStart/Setup, because those stream even when hook events were not requested (they stream only when a staged plugin declares hooks). A frame whose outcome cannot be read (no exit code, a hook that started and never answered, stdout a redaction policy rewrote or the agent truncated, output the agent did not apply as a verdict, or a failed or timed-out frame on an event with an `onFailure: "block"` hook — see `hook_decision`) is evidence-unavailable. Hooks on PreCompact, PostCompact, ConfigChange, DirectoryAdded, Elicitation, ElicitationResult, InstructionsLoaded, Notification, SessionEnd, StopFailure, WorktreeCreate, WorktreeRemove, CwdChanged and FileChanged stream no frame at all, so a block there never shows: `true` is evidence-unavailable when one of those events has an `onFailure: "block"` hook, or when the run's hook sources could not all be read. Frames carry no plugin id, so a second staged plugin's hook counts as a frame here (the run warns). Distinct from `no_hook_blocked`, which reads the harness's **own** PreToolUse callbacks from `controlOut`. Content-class |
 | `hook_decision: {event, decision, tool?, min?, max?}` | counts `hook_response` frames for `event` whose decision (stdout JSON on an `outcome: success` frame, or exit 2 = deny) is `decision`: `allow`, `deny`, `ask`, `defer`, or the aliases `block` (deny), `approve` (allow); `tool`, `min`/`max` as for `hook_event_blocked` (default: at least one). Empty/non-JSON stdout is no decision; no frame fails "never fired"; an unreadable frame (incl. a failed one on an `onFailure: "block"` hook) is evidence-unavailable when it could change the verdict. Content-class, grades on replay; details in the scenario reference (linked above) |
-| `hook_output_contains: {event, stream?, text \| matches}` / `hook_output_not_contains: {…}` | a command hook **printed** (or never printed) a text on `stdout` / `stderr` / either (`stream`, default `any`) — some (or no) `hook_response` frame for `event` carries `text` (literal, case-sensitive) or `matches` (regex, case-insensitive, no multiline flag — `^`/`$` anchor the whole stream). For a hook that fails open on stderr while exiting 0. No frame for the event FAILS both. On a redacted stream, a literal miss and any `matches` result are evidence-unavailable for either key, while a literal hit outside a token counts; a miss is also evidence-unavailable on a missing field, a hook that started without a response, or output the agent truncated. Frames carry no plugin id (a second plugin, or at `protocol` without a sealed config dir a host-installed one, can answer). Content-class. Recorded end-to-end for `Stop` |
-| `vm_path_denied: true` | **`fidelity: hostloop` only** — at least one recorded path denial (`RunResult.pathDenials`, any source) targeted a `/sessions` VM path — evidence-unavailable if path-denial telemetry is absent. Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify". **Only `true` is valid** |
-| `path_denied: {tool?, path_matches?, source?, agent_scope?}` | **`fidelity: hostloop` only** — a path denial matching ALL given matchers (`tool` glob, `path_matches` regex, `source` ∈ pretooluse/can_use_tool/permission_denied, `agent_scope` ∈ main/subagent/any) was recorded. Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify" |
-| `no_path_denied: true` | **`fidelity: hostloop` only** — NO path denial was recorded at all (the channel is already path-scoped, unlike `no_hook_blocked`'s indiscriminate reject). Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify". **Only `true` is valid** |
+| `hook_output_contains: {event, stream?, text \| matches}` / `hook_output_not_contains: {…}` | a command hook **printed** (or never printed) a text on `stdout` / `stderr` / either (`stream`, default `any`) — some (or no) `hook_response` frame for `event` carries `text` (literal, case-sensitive) or `matches` (regex, case-insensitive, no multiline flag — `^`/`$` anchor the whole stream). For a hook that fails open on stderr while exiting 0. No frame for the event FAILS both. On a redacted stream, a literal miss and any `matches` result are evidence-unavailable for either key, while a literal hit outside a token counts; a miss is also evidence-unavailable on a missing field, a hook that started without a response, or output the agent truncated. Frames carry no plugin id (a second staged plugin can answer). Content-class. Recorded end-to-end for `Stop` |
+| `vm_path_denied: true` | **`fidelity: hostloop` only** — at least one recorded path denial (`RunResult.pathDenials`, any source) targeted a `/sessions` VM path — evidence-unavailable if path-denial telemetry is absent. Any other tier FAILS "cannot verify". **Only `true` is valid** |
+| `path_denied: {tool?, path_matches?, source?, agent_scope?}` | **`fidelity: hostloop` only** — a path denial matching ALL given matchers (`tool` glob, `path_matches` regex, `source` ∈ pretooluse/can_use_tool/permission_denied, `agent_scope` ∈ main/subagent/any) was recorded. Any other tier FAILS "cannot verify" |
+| `no_path_denied: true` | **`fidelity: hostloop` only** — NO path denial was recorded at all (the channel is already path-scoped, unlike `no_hook_blocked`'s indiscriminate reject). Any other tier FAILS "cannot verify". **Only `true` is valid** |
 | `allow_permissive_auto_allow: true` | verdict modifier — suppresses the default-fail when the run recorded a cowork-parity permissive auto-allow; for tests that deliberately assert Cowork's permissive behavior |
 | `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
-| `allow_l0_host_config_contamination: true` | verdict modifier — accept a contaminated L0 environment: suppresses the default-fail when `protocol` runs against your REAL config dir, where your installed plugins/skills/MCP servers are visible and may answer instead of the thing under test. Live tiers only |
 | `allow_stall: true` | verdict modifier — suppresses the `stalled` default-fail when a run ends on a question or (after an `AskUserQuestion` gate) a closing request for input having done no productive tool work after its last gate (the agent asked for input and stopped — incl. re-asking in plain text after answering an `AskUserQuestion`); assert only when ending on a question is intended, else script the answer (`answer:` / `--answer` / a decider). On an open-ended `skill` / `probe-dispatch` run (no `assert:` block) pass `--allow-stall` instead. A helpful closing offer ("want me to run this through a structured pass?") also fails `stalled` — read the final message before believing it |
+| `allow_max_turns: true` | verdict modifier — a run stopped by the session's `agent_max_turns` cap completes instead of failing (`eval`: `turn_cap_allowed`). `result` stays `error`: never beside `result: success` |
 | `allow_undelivered_deliverables: true` | verdict modifier — suppresses the `undelivered_deliverables` WARN. Working in the scratchpad is Cowork's designed pattern, so a skill that legitimately leaves intermediates, caches or downloaded inputs behind can say so instead of carrying permanent noise. The signal is warn-only and never fails a run on its own; reach for this when the scratch activity is intentional, not to silence a real delivery gap |
 | `allow_outputs_delete: true` | verdict modifier — accepts a detected outputs delete instead of failing the run, for a skill whose deletion is intended. Has an effect only on a baseline recording outputs `rw` (Desktop before 2.16120.0), where omitting `no_delete_in_outputs` does **not** permit deletes; on `rwd` (2.16120.0+) an outputs delete does not fail by default, so it is an accepted no-op (no warning), unless `no_delete_in_mounts` arms the outputs check, where it waives that check's signal as on `rw`. **Mutually exclusive** with `no_delete_in_outputs`. Silences `outputs_delete`, `outputs_delete_unconfirmed` and `outputs_diff_unavailable`. Waives the harness's post-hoc detection; it does not model Cowork's `allow_cowork_file_delete` approval handshake |
 | `allow_delete_in: [<mount>…]` | verdict modifier — accepts detected deletes in the named mounts (the per-mount analogue of `allow_outputs_delete`, mirroring production's `fileDeleteApprovedMounts`). Waives the verdict only; detection still runs and the hits stay in `result.json`. Listing `outputs` conflicts with `no_delete_in_outputs` |

package/.claude/skills/cowork-harness/references/assertion-catalog-outcome-files-tools.md

@@ -1,6 +1,6 @@
 # Assertion catalog: outcome, transcript, files and artifacts, tools, tool results
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 

package/.claude/skills/cowork-harness/references/assertion-catalog.md

@@ -1,6 +1,6 @@
 # Assertion catalog
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Every `assert:` key with its semantics, and the
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Every `assert:` key with its semantics, and the
 verdict-signal table. Which keys survive `replay` is in [`scenario-schema.md`](./scenario-schema.md#replay-class);
 the scenario and session YAML fields are there too.
 
@@ -50,9 +50,9 @@ paraphrases (that re-records red). Structured JSON → assert it in YAML with **
 `path` + operator, or `schema:` for a whole value's shape); use the pytest lane (`assert_artifact_json`) only for
 predicates neither can express.
 
-**VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; twelve
+**VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; eleven
 are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
-`"success"`) and eighteen are **warn**-severity (informational, never flip pass/fail). All thirty signal
+`"success"`) and eighteen are **warn**-severity (informational, never flip pass/fail). All twenty-nine signal
 codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 | Code | Severity | Meaning |
@@ -70,7 +70,6 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 | `device_delete_unconfirmed` | warn | `device_bridge`, on a filesystem that can give a new file a deleted file's identity (`rename-reads-as-delete`): files are gone from a connected folder deletion was never allowed in, either deleted past the device's refusal or renamed and then edited, which that filesystem cannot tell apart |
 | `device_audit_unreadable` | warn | `device_bridge`: the shim's delete log was replaced by the device shell (a link, a FIFO), so the number of refused deletes, and deletes of files the session made after a grant, are unknown (`deviceBridge.auditUnreadable`; `refusedDeletes` is left out). The before-and-after comparison of the folders still ran |
 | `host_path_leak` | fail | A host path leaked into model-visible text (opt out: author `transcript_no_host_path`). A path copied verbatim from the scenario's own uploads, connected folders, prompt, or declared plugins' or local skills' files is not a leak |
-| `l0_host_config_contamination` | fail | `protocol` ran against the operator's REAL config dir, so host-installed plugins/skills/MCP servers may have answered instead of the thing under test — the run did not necessarily measure it (opt out: `allow_l0_host_config_contamination`). Name predates the meaning: delivery at L0 works, contamination is what this reports |
 | `missing_capability` | fail | A `requires_capabilities` need was unmet, or the skill used a capability the image omits (opt out: `allow_missing_capability`, or `skill --allow-missing-capability` on an open-ended run) |
 | `infra_error` | fail | A supervising process died mid-run (the VM/egress sidecar, or the `device_bridge` container) — the run's evidence is contaminated, not author-suppressible |
 | `stalled` | fail | The run ended asking for input with no tool work after the last gate. "Asking" = the final turn's closing sentence ends in `?`; or, once an `AskUserQuestion` gate has fired, it ends in `?` after trailing bold/quotes/`)`/emoji, is a `?` followed only by a `For example: …`/parenthetical aside, or is a request that says the input comes back to the agent (`Please share/provide/upload… so I can…`/`…here`, `Let me know which… you'd like me to use`, `Once you share… I'll…`, `I need X to proceed`, `Once I have the file, I'll…` after a sentence asking for it). Polite closers and hand-offs (`Let me know if…`, `Feel free…`, `with your`/`to your`, `before sending`, `If you…`) and a closing code block or blockquote never count. Hand-offs to a named third party (`with the team`, `to the founders`, `with the CFO`) never count either; `here` is a cue only after share/paste/upload/drop/reply/type/send or as the last word. English-only. Opt out with `allow_stall` (on `run`, `replay` and `eval`); without it, under `eval` a stalled rep is `errored_agent` and fails every row |

package/.claude/skills/cowork-harness/references/assertions-guide.md

@@ -1,6 +1,6 @@
 # Assertions guide
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
 
 ### Assertions: two orthogonal axes
 
@@ -50,15 +50,15 @@ them by what you're trying to prove:
 | the user was **shown** the right choices, in order | `question_options: {when_question, equals: [..]}` — the option SET/ORDER a gate offered (`question_asked` matches the text only); order is compared by default |
 | the user was **told something specific** at a gate | `question_context: {when_question, matches}` — a regex over the question label + option labels + option **descriptions**. Reach for this when the wording may land in an option's `description`, which `question_asked` and `question_options` cannot see |
 | every gate offered exactly one (or at most N) options of a kind | `question_option_count: {matches, exactly}` — counts the option LABELS matching a regex on EVERY sub-question asked (`when_question` narrows); zero sub-questions asked fails |
-| every gate was answered by a scripted rule (an unattended run that still asks) | `gates_all_scripted: true` with `gate_answer_count_min: 1` — fails naming any gate the LLM decider, `first`, or an external or human decider answered; `{include_permissions: true}` also fails cowork parity's permissive auto-allow (replay needs a `controlOut` cassette) |
-| a **plugin's** command hook (any event: PreToolUse, Stop, …) blocked, didn't block, or decided | `hook_event_blocked: <event>` (exit 2) or `{event, tool?, via?, max: 0}`, `no_hook_event_blocked: true`, `hook_decision: {event, decision}` — stream content, no `controlOut` needed. A failed or timed-out frame on an event with an `onFailure: "block"` hook is evidence-unavailable, and so is unscoped `no_hook_event_blocked: true` when such a hook sits on an event that streams no frame |
-| the **harness's own** hook callbacks (the built-in Task hook, a custom hook bundle) blocked / didn't block a tool | `hook_blocked: <regex>`, `no_hook_blocked: true` (replay needs a `controlOut` cassette) — a plugin's hook never reaches this list, so `no_hook_blocked` passes over a plugin block |
+| every gate was answered by a scripted rule (an unattended run that still asks) | `gates_all_scripted: true` with `gate_answer_count_min: 1` — fails naming any gate the LLM decider, `first`, or an external or human decider answered; `{include_permissions: true}` also fails cowork parity's permissive auto-allow |
+| a **plugin's** command hook (any event: PreToolUse, Stop, …) blocked, didn't block, or decided | `hook_event_blocked: <event>` (exit 2) or `{event, tool?, via?, max: 0}`, `no_hook_event_blocked: true`, `hook_decision: {event, decision}` — stream content. A failed or timed-out frame on an event with an `onFailure: "block"` hook is evidence-unavailable, and so is unscoped `no_hook_event_blocked: true` when such a hook sits on an event that streams no frame |
+| the **harness's own** hook callbacks (the built-in Task hook, a custom hook bundle) blocked / didn't block a tool | `hook_blocked: <regex>`, `no_hook_blocked: true` — a plugin's hook never reaches this list, so `no_hook_blocked` passes over a plugin block |
 | every MCP round-trip succeeded | `no_mcp_error: true` (**live-only**) |
 | a context compaction happened | `compaction_occurred: true` |
 | THIS run wrote the file (not just that it is there) | `file_exists: {path, authored: true}` (also `user_visible_artifact`, and `authored: true` on `artifact_text`/`artifact_json`) — an untouched pre-run file fails; needs the pre-run manifest, which `authored` arms |
 
-Every one of these still obeys the two axes above — several are live-only or need a `controlOut`
-cassette on replay, so check the catalog's replay class before putting one on a PR gate.
+Every one of these still obeys the two axes above — several are live-only or need an artifact
+manifest on replay, so check the catalog's replay class before putting one on a PR gate.
 `cowork-harness assertions --list` prints the full, always-current key set with one-line semantics
 straight from the schema — treat it (and the catalog) as the source of truth; this map is a
 goal-oriented index into it, not a second catalog.

package/.claude/skills/cowork-harness/references/authoring.md

@@ -1,6 +1,6 @@
 # Authoring a scenario
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
 
 ## Part I — AUTHOR a scenario
 
@@ -45,7 +45,7 @@ the folder basename (collision-resolved); there is no `to:` override. See `refer
 
 | Tier | What it gives you | Use when |
 |---|---|---|
-| `protocol` | Fastest; no sandbox, no egress | Pure protocol/answer-shape tests. **Rejected** if the scenario asserts egress. |
+| `protocol` | Fastest; no sandbox, no egress; needs a credential in env/.env (no Keychain/login) | Pure protocol/answer-shape tests. **Rejected** if the scenario asserts egress. |
 | `container` | Real sandbox + real default-deny egress (the pre-4.0 default; every scenario now names its tier). Models the **VM loop**: keeps the built-in `Bash`, but `WebFetch` is replaced by `mcp__workspace__web_fetch` — assert on that name, not `WebFetch`. (`run`/`record` only; `chat --fidelity container` still offers the built-in.) **The name is tier-specific**: `microvm` never offers it and `protocol` serves the operator's own host registry. A `tool_not_called`/`subagent_tool_absent` naming a tool its tier does not serve is **REFUSED at scenario load** (the message names what to write instead), so moving a scenario between tiers now errors rather than silently voiding the assertion — except at `protocol`, which is never judged | Most functional + boundary tests. |
 | `microvm` | VM-grade escape **isolation** (macOS arm64). Egress transport is the *same allowlist proxy as `container`* — not better network fidelity. Unlike `container`, still offers the built-in `WebFetch` | Testing untrusted code escape, not network behavior. |
 | `hostloop` / `cowork` | Production split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production), with **both** native `Bash` and `WebFetch` disabled and routed host-side via the workspace SDK-MCP server into a Docker VM sidecar (the VM loop replaces web_fetch only) | Highest-fidelity / parity runs. A writable connected folder needs `allow_host_writes: true` (see scenario-schema.md). |
@@ -288,12 +288,12 @@ cowork-harness lint scenarios/*.yaml
 | `capabilities-on-protocol` | ERROR | non-empty `requires_capabilities` on `protocol` without `allow_missing_capability` — the probe cannot run there, so the run fails as unverifiable |
 | `cassette-evidence-skipped` | INFO | with `--cassette-dir`: a cassette (or the directory) could not be read, so it cannot quiet replay-evidence advice |
 | `container-only-key-off-container` | ERROR | `no_scratchpad_leak` off `container` — hostloop's `present_files` never promotes, so there is nothing to leak (WARN on `cowork`, whose tier resolves per the baseline gate) |
+| `contradictory-turn-cap` | WARN | `allow_max_turns: true` beside `result: success` (a capped run is `error`) |
 | `egress-on-protocol` | ERROR | an egress assertion (`egress_*` / `expect_denied`) on `protocol`, which enforces no egress |
 | `enum-value-invalid` | ERROR | a field value outside its allowed set |
 | `artifact-json-match` | ERROR | an `artifact_json` glob `artifact` (with `*` or `?`) with no `match` or ending in `/`, or a literal one that sets `match` |
 | `fidelity-missing` | ERROR | no `fidelity:` (required since 4.0.0) |
 | `file-absent-contradiction` | ERROR | one path under both `file_exists` and `file_absent` |
-| `gate-needs-controlout` | INFO | gate assertions, which evaluate on replay only when the cassette has `controlOut` |
 | `hook-output-control-char` | ERROR | a `hook_output_contains` / `hook_output_not_contains` `text` or `matches` holding a control character (a double-quoted YAML `\b` is a backspace) — the harness refuses it at load |
 | `host-path-assert-cowork` | WARN | `transcript_no_host_path` on `cowork` — it fails by design if the tier resolves to hostloop |
 | `host-path-assert-tier` | ERROR | `transcript_no_host_path` on `hostloop` / `protocol`, where it fails by design |
@@ -304,6 +304,7 @@ cowork-harness lint scenarios/*.yaml
 | `manifest-needs-snapshot` | INFO | manifest-backed keys, which evaluate on replay only when the cassette carries an `artifacts` manifest (not reported on `lane: remote` for `user_visible_artifact` / `artifact_json` / `artifact_text`, which cannot pass there) |
 | `mixed-assert-item` | WARN | one assert item mixing replay-checkable and live-only keys (replay drops the live-only half) |
 | `no-scenarios` | ERROR | a linted directory with no `*.yaml` / `*.yml` |
+| `not-a-scenario` | INFO | a session file (only session keys); not checked |
 | `not-found` | ERROR | a named file that does not exist |
 | `parse` | ERROR | a file that is not YAML, or not a mapping |
 | `positional-choose-order` | INFO | an answer rule with a positional `choose` (first / index), which option re-ordering can move |
@@ -369,13 +370,13 @@ a reserved value — is ✗ ERROR `scenario-invalid` (exit 1, with or without `-
 naming no baseline this installed CLI ships is ✗ ERROR `baseline-unknown` (`latest` always resolves). It
 does not check what depends on the machine the run happens on (the session file and its mounts, an
 absolute `baseline:` path that does not exist here — one that exists is checked (4.1.1 and later) —
-environment variables). A session or matrix YAML in a linted directory is not
-a scenario and is reported as one that does not load — keep those out of the linted set. `python3
+environment variables). A session file in a linted directory is one INFO `not-a-scenario`; a matrix YAML is
+reported as one that does not load — keep those out of the linted set. `python3
 scenario.py lint` run directly stays offline and lenient: there an unknown key is only a ⚠ WARN (exit 0).
 `cowork-harness record <file.yaml> --dry-run` also runs the loader and adds the pre-spend refusals (exit 2
 on a schema error; a directory reports each `✗ broken:` file and exits 1). **Read the exit code, not just
 its sign:** `record <file>` — with or without
-`--dry-run` — answers `2` for "did not load" and `1` for "loaded fine, but this record is refused" (a
+`--dry-run` — answers `2` for "did not load" or an unknown baseline name and `1` for "loaded fine, but this record is refused" (a
 pre-spend policy refusal, including `--max-budget-usd`, which exited 2 before 4.0.0; `skill`/`run` still
 exit 2 on it). Treating any non-zero
 as "scenario broken" mis-reports every refused-but-valid scenario. Corollary: **the loader** fails LOUD on an unknown key (never silently).
@@ -384,6 +385,6 @@ decided when the cassette is recorded. A key that changes what a verdict means (
 the cassette's version stamp, so the older `replay` / `verify-cassettes` refuses it as too new instead of
 evaluating it (`replay --best-effort-future-cassette` overrides that and names the key). For `lane: remote`
 that holds for a cassette recorded on ≥ 1.16.0 (stamped v11); one recorded by 1.14.0 or 1.15.0 is stamped
-v10, a pre-`lane` CLI ignores the key there, and `rehash` re-stamps it. A meaning-neutral key leaves the
+v10, and a pre-`lane` CLI ignores the key there (5.0 reads v12+ only). A meaning-neutral key leaves the
 stamp alone and is ignored by design. Full split:
 [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md#unknown-keys-the-loader-is-strict-lint-is-lenient).

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,23 +1,23 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`).
+Self-contained reference. Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
 
 ```yaml
-- uses: yaniv-golan/cowork-harness@v4
+- uses: yaniv-golan/cowork-harness@v5
   with:
     command: replay
     path: cassettes/
-    version: "^4"              # hold the major; see below
+    version: "^5"              # hold the major; see below
 ```
 
-**These recipes pin `version: "^4"`.** The Action's `version` input *defaults* to `latest`, which means a
+**These recipes pin `version: "^5"`.** The Action's `version` input *defaults* to `latest`, which means a
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
-copy-pasted recipe that omits the input takes a major bump with no say in it. `^4` holds the major, needs no
+copy-pasted recipe that omits the input takes a major bump with no say in it. `^5` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "4.14.0"`) instead when you want byte-reproducible CI.
+(e.g. `version: "5.0.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -57,11 +57,11 @@ jobs:
           echo "COWORK_AGENT_BINARY=$RUNNER_TEMP/claude-$V" >> "$GITHUB_ENV"
           # Background on the provenance chain: the "Agent-binary provenance" section of
           # https://github.com/yaniv-golan/cowork-harness/blob/main/docs/maintenance.md
-      - uses: yaniv-golan/cowork-harness@v4
+      - uses: yaniv-golan/cowork-harness@v5
         with:
           command: run
           path: scenarios/
-          version: "^4"
+          version: "^5"
           anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
           model: claude-sonnet-5 # used only where a scenario's session sets no `model:`
 ```
@@ -82,7 +82,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^4.14.0"
+- run: npm i -g "cowork-harness@^5.0.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -157,21 +157,21 @@ Action has no input for, and it creates a coupling nothing checks:
 you have a reason:
 
 ```yaml
-- uses: yaniv-golan/cowork-harness@v4
+- uses: yaniv-golan/cowork-harness@v5
   with:
     command: lint
     path: scenarios/
-    version: "^4"                       # holds the major
+    version: "^5"                       # holds the major
     strict: true                        # without it, only ERROR fails the step
-    extra-args: --min-severity WARN     # needs a CLI >= 1.11.0; any 4.x satisfies that
+    extra-args: --min-severity WARN     # needs a CLI >= 1.11.0; any 5.x satisfies that
 ```
 
 **If a flag you pass in `extra-args` landed in a specific release, bound the range — don't write a bare
 floor.** `>=1.11.0` reads as "at least 1.11.0" and silently means "and every future major too", so a
 recipe written that way hands a copy-paster the next major with no say in it. Anchor it at the current
-major instead — `version: "^4"`, which is what the steps above use — keeping the floor's intent while
+major instead — `version: "^5"`, which is what the steps above use — keeping the floor's intent while
 stopping at the major boundary. An exact
-pin (`version: "4.0.0"`) is the right choice when you want byte-reproducible CI, at the cost of rotting the
+pin (`version: "5.0.0"`) is the right choice when you want byte-reproducible CI, at the cost of rotting the
 moment a recipe adopts a newer flag.
 
 Without a satisfied floor, an older CLI fails the step with `unrecognized arguments: --min-severity WARN`
@@ -186,9 +186,8 @@ The split is not just about tokens — it decides **where each lane can run**:
   Docker, no agent binary** — runs on a stock GitHub Actions runner. Evaluates **content** assertions —
   `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `skill_triggered`, `no_skill_triggered`,
   `max_cost_usd`, `max_tokens`, `tool_calls_max`, `result`, and the verdict modifiers
-  `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_host_config_contamination` /
-  `allow_stall` (no-op passes); plus the gate keys `question_asked` / `question_options` /
-  `question_context` / `question_option_count` / `questions_count_max` / `gate_answers_delivered` **if** the cassette has `controlOut`, and the manifest keys
+  `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_stall` (no-op passes); plus the gate keys `question_asked` / `question_options` /
+  `question_context` / `question_option_count` / `questions_count_max` / `gate_answers_delivered`, and the manifest keys
   (`file_exists` / `user_visible_artifact` / `artifact_json` / `artifact_text`) **if** it carries an artifact
   manifest. `file_absent` is in neither class — it is live/verify-run only.
   **That list is illustrative, not the authoritative set** — more keys are replay-checkable than fit a
@@ -199,7 +198,7 @@ The split is not just about tokens — it decides **where each lane can run**:
   ```
 
   It emits `{key, description}` — there is no structured replay-class field to filter on; the
-  live-only / manifest / `controlOut` preconditions are stated in each key's `description` prose and,
+  live-only / manifest preconditions are stated in each key's `description` prose and,
   in full, in the catalog's replay-class tables in
   [`references/scenario-schema.md`](./scenario-schema.md).
 
@@ -252,8 +251,7 @@ cowork-harness replay cassettes/my-test.cassette.json  # token-free re-evaluatio
 cowork-harness replay cassettes/                   # replay every *.cassette.json in the committed dir
 ```
 
-Re-record whenever the protocol or your scenario's expected content changes. An old cassette without
-`controlOut` excludes the gate keys (with a loud warning) — re-record to enable them. `record` **refuses
+Re-record whenever the protocol or your scenario's expected content changes. `record` **refuses
 to freeze a failing live run** into a cassette (pass `--allow-failing` to override) — a committed red
 cassette is a latent false-signal. An `--allow-failing` recording of a red run exits 0 with `ok: true`:
 `record`'s `ok` means a cassette was written, and the run's own verdict is `results[0].verdict.pass`
@@ -290,6 +288,7 @@ dollar figures). In a skill repo these cassettes get **committed**. So:
   still refuses the write and quarantines it. **You do not need to audit the session by hand to pass this
   flag**; that precondition was never decidable by the operator, and the scan is the real gate. Writing a
   recording the scan DID flag is the separate `--allow-host-inventory-findings`.
+  A `plugins.from_host` session is host-inheriting at **every** tier: its cassette names your installed set.
 
   Two details that matter for a re-record loop. The pre-spend check **warns rather than refuses when the
   cassette already exists**, deliberately: refusing there would fire on every `--rerecord-stale` pass and
@@ -359,18 +358,11 @@ A typical skill repo runs four stages, fastest/cheapest first:
 
    This is the shape a CI step wants: **silent on success (no output, exit 0), loud and specific on
    failure** — `--quiet` suppresses the readiness preview but never the `✗ broken:` lines, which name the
-   offending file *and* the rejected key, one line per file, and the step still exits 1. It exits 0,
-   though, on an input the real record would refuse (a `session:` file that cannot be read — 4.1.1 and
-   later — a missing path, an unknown baseline name, a tier-vacuous negative tool check): that prints a `⚠ input error:` line and lands in `inputErrors[]`. To
-   gate on those too, use the JSON form (4.1.0 and later):
-
-   ```bash
-   cowork-harness record scenarios/ --dry-run --output-format json | jq -e '.ok and (.inputErrors == [])'
-   ```
-
-   Point `lint`
-   at scenarios only: a session or matrix YAML in the linted set is reported as a file that does not
-   load.
+   offending file *and* the rejected key, one line per file, and the step still exits 1. An input the real record would refuse
+   (an unreadable `session:` file, a missing path, a tier-vacuous negative tool check) exits 1 too, with a `✗ refused:`
+   line and an entry in `refusals[]`; an unknown baseline name exits 2. The exit code is the gate. Point `lint`
+   at scenarios only: a session file in the linted set is one INFO, and a matrix YAML is reported as a file
+   that does not load.
 
    **If the repo pays for `critique`, gate the evidence corpus here first, for free:**
 
@@ -391,7 +383,7 @@ A typical skill repo runs four stages, fastest/cheapest first:
 
 ## GitHub Actions sketch
 
-The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v4` does
+The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v5` does
 in one step (see the top of this doc) — reach for this form when you need independent per-command
 gating/annotations rather than one action run per command. The nightly live job has no packaged-Action
 equivalent yet (the Action's `command: run` mode needs a self-hosted runner with Docker + the agent binary
@@ -411,7 +403,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^4.14.0"
+      - run: npm i -g "cowork-harness@^5.0.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -440,7 +432,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^4.14.0"
+        run: npm i -g "cowork-harness@^5.0.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:
@@ -560,7 +552,7 @@ moved — and `unverifiable-fixture`, a `workspace_fixture` whose signature cann
 | `skill` / `shared-root` | the skill source the assertions validate drifted | **high** (assertions may validate dead code) |
 | `format` | the git/raw file-set mode or agent-scope differs from the recording | re-record under the same setting (waivable) |
 | `unverifiable-baseline` | the latest baseline couldn't be loaded | couldn't verify (env, not skill) |
-| `unverifiable-skill` | skill dirs unresolvable, **or** the cassette predates the hash-format epoch (v12) so its digest is not comparable | couldn't verify the skill. **Fails a bare `replay`.** For the epoch case try `rehash` first — it migrates without a re-record where it can prove the content unchanged (`rehash <file> --session <s.yaml>` if the cassette moved) |
+| `unverifiable-skill` | skill dirs unresolvable | couldn't verify the skill. **Fails a bare `replay`.** Point at a moved cassette's session with `--session <s.yaml>` |
 | `resolved-tier` | a `fidelity: cowork` cassette's recorded `effectiveFidelity` no longer matches what the baseline resolves to today (the host-loop gate flipped) | **high** (the recording exercises the wrong tier) |
 | `unverifiable-tier` | tier check couldn't run for a `fidelity: cowork` cassette (no recorded `effectiveFidelity`, or its pinned baseline failed to load) | couldn't verify the tier — re-record |
 | `prompt-assets` | the baseline's committed prompt assets changed since record under the same appVersion | re-record (warns; `--strict` fails) |

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). This is **not** a trim of the full
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reading several critiques, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/debugging.md

@@ -1,6 +1,6 @@
 # Debugging a run
 
-Tracks `cowork-harness 4.14.0` (baseline `desktop-4.16336.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
+Tracks `cowork-harness 5.0.0` (baseline `desktop-4.23118.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
 
 ## Part III — Debug
 
@@ -28,6 +28,7 @@ Docker, no re-record.
 |---|---|---|
 | **The skill misbehaved** | wrong output, an unexpected gate, a denied tool, an opaque crash | `inspect` — what did it produce? · `trace <run-dir> --view <view>` — what did it actually do (tools; gates are the questions view, as there is no gates view; the sub-agent tree is the dispatches view)? · `verify-run` — re-assert cheaply when only an assertion is wrong · `diff <old-run> <new-run>` — what changed since it worked · `chat` — reproduce it by hand |
 | **A green you don't trust** | an assert that may have tested nothing, a stale cassette, an auto-answered or decided gate | `replay --explain` — the evidence trail behind each *passing* assert · `replay --mutate` — perturbs a CAPPED SAMPLE of recorded JSON values (10/file, 50 total) and reports which perturbations NOTHING caught; the report names the sample size, so read it as a sample not a total (reporting only; never moves the verdict/exit code) · `lint` — assertions on the wrong CI lane / mixed-class keys · `verify-cassettes` — privacy + staleness over committed cassettes · the Gotchas landmine catalog — how a check passes vacuously · `run --repeat N` / `skill --repeat N` — did it pass, or pass once? · `stats` — flaky or expensive over time |
+| **A judge or grader call failed** | an assert reads `judge call FAILED (API 401, …)`, or `eval` stops at its credential check; often a stale key, such as an old one in `.env` | `cowork-harness doctor` — which credential the graders and the agent each use · the graders get `CLAUDE_CODE_OAUTH_TOKEN` alone when it is set, so a stale token fails even with a good API key · `COWORK_HARNESS_GRADER_AUTH=inherit` passes every key |
 
 A failed run also records `errorSource` (where the failure originated) and `stderrLogPath` (the captured
 agent stderr) — read those before re-running (`answer_channel_violation` means a run under `answer_channel: none`
@@ -141,7 +142,7 @@ decide which assertions from *Assertions: two orthogonal axes* in `assertions-gu
   `modelUsage[].costUSD`, which is what `trace --view usage` reports, so the two can differ),
   `usage` (`input_tokens`/`output_tokens`/`turns`), `toolDurations` (with `toolDurationsBasis`), `models`,
   `toolCalls` (every tool call in stream order: `name`, top-level `input` fields each capped at 10 KB, and
-  `origin` `main`/`subagent`/`unknown` — what the object form of `tool_called`/`tool_not_called` reads), `toolErrors`,
+  `origin` `main`/`subagent`/`unknown` — what the object form of `tool_called`/`tool_not_called` reads; `denied: true` means the agent received a deny answer to that call's permission request, which a hook block never sets; the footer's `denied: N` counts those answers), `toolErrors`,
   `redundantToolCalls`, `modelUsage`, `thinking`, `skillActivity`, `subagents[]` (prompt/`dispatchModel`/
   `resolvedModel`/output/`attributedSkillId`, `outputTruncated`, `referencesRead`, `reasoning`/`reasoningElided`),
   `context` (tools/mcpServers/availableSkills), `tasks`,
@@ -184,6 +185,8 @@ decide which assertions from *Assertions: two orthogonal axes* in `assertions-gu
   UNAVAILABLE marker (`workspaceFilesRecorded: false` in JSON, and no phantom "removed" diff rows) and
   `inspect` prints `artifacts: UNAVAILABLE` (`artifactsRecorded: false`) instead of `artifacts (0):`.
 
+To grade `assertions[]` by hand, see [docs/debugging.md → Reading a run's assertions by hand](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/debugging.md#reading-a-runs-assertions-by-hand).
+
 ### Debugging with `chat`
 
 `cowork-harness chat` opens an interactive multi-turn REPL against a live Cowork session. It is