cowork-harness 4.9.1 → 4.10.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, asserting artifacts, egress, or sub-agent dispatch, measuring how long each tool call took (toolDurations / trace), or debugging a failed run or verdict from its result.json or transcript. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. Also for comparing two versions of a skill before merging an edit — did it make answers worse? (`eval`: paired A/B, pinned models) — or improving one round by round (`hillclimb`, the `/claude-api hillclimb` runner). NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold / critique / stats / eval / hillclimb commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 4.9.1
-  tracks-harness: cowork-harness 4.9.1 (baseline desktop-2.37337.1)
+  version: 4.10.0
+  tracks-harness: cowork-harness 4.10.0 (baseline desktop-2.42304.0)
 ---
 
 # cowork-harness
@@ -26,8 +26,8 @@ allowlist). This skill exists mostly to keep you out of those traps — the *Inv
 full landmine catalog in [`references/gotchas.md`](references/gotchas.md) are the highest-value part.
 Read them.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.9.1` (baseline
-> `desktop-2.37337.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.10.0` (baseline
+> `desktop-2.42304.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## Preflight — make sure the harness can actually run
@@ -43,7 +43,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, the baseline and the `device_bridge` image (built, or built on the first bridge run) in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.9.1**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.9.1" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.9.1"`. **Pin `@^4.9.1`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.10.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.10.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.10.0"`. **Pin `@^4.10.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -161,7 +161,7 @@ behind each, is [`references/gotchas.md`](references/gotchas.md).
 | [`references/measurement.md`](references/measurement.md) | `--repeat`, `--ablate-skill`, measurement hygiene |
 | [`references/debugging.md`](references/debugging.md) | triage, `result.json` fields and `trace` views, `chat` |
 | [`references/gotchas.md`](references/gotchas.md) | the full "✓ passed ≠ correct" landmine catalog |
-| [`references/task-recipes.md`](references/task-recipes.md) | start here for "how do I X": evolve `assert:`, audit tier drift, redaction, budgets, answer quality, and goals with no flag (force a compaction, ablate a section, a form reply, resume in a new conversation, hook JSON decisions, schema checks, unattended runs), keeping a skill's web searches (`search-pool`), and a skill that parks and resumes in a later session (`legs:`) |
+| [`references/task-recipes.md`](references/task-recipes.md) | start here for "how do I X": evolve `assert:`, audit tier drift, redaction, budgets, answer quality, and goals with no flag (force a compaction, ablate a section, a form reply, resume in a new conversation, hook JSON decisions, schema checks, unattended runs), keeping a skill's web searches (`search-pool`) and serving a run's searches from them (`--search-corpus`), and a skill that parks and resumes in a later session (`legs:`) |
 | [`references/assertion-catalog.md`](references/assertion-catalog.md) | the assertion catalog's index: conventions shared by every key, the verdict-signal table, and links to the per-family files below |
 | [`references/assertion-catalog-outcome-files-tools.md`](references/assertion-catalog-outcome-files-tools.md) | per-key rows: outcome, transcript, files and artifacts, tools |
 | [`references/assertion-catalog-agents-skills-budgets.md`](references/assertion-catalog-agents-skills-budgets.md) | per-key rows: sub-agents, skills and connectors, budgets, tasks, delivery |

package/.claude/skills/cowork-harness/references/assertion-catalog-agents-skills-budgets.md

@@ -1,6 +1,6 @@
 # Assertion catalog: sub-agents, skills and connectors, budgets, tasks, delivery
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 
@@ -33,5 +33,5 @@ keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 | `all_tasks_completed: true` | every task in `RunResult.tasks[]` reached status `"completed"` — **requires ≥1 task** (a zero-task run fails; assert `task_count_min` for presence); evidence-unavailable if tasks telemetry is absent |
 | `task_count_min: <N>` | at least N tasks were created (`RunResult.tasks.length >= N`) — the presence companion for task assertions |
 | `task_status: {match, status}` | a task whose `subject` OR `id` matches the regex `match` reached `status` — evidence-unavailable if tasks telemetry is absent; also fails **malformed** when a TaskCreate result was unparseable (corrupt task telemetry), mirroring the guard `all_tasks_completed`/`task_count_min` already had |
-| `no_scratchpad_leak: true` | every file presented via `present_files` that was in the scratchpad was successfully promoted to `mnt/outputs` (none left behind) — vacuously passes if nothing was presented (pair with a presence check to require a delivery); content-class: both the `present_files` tool_use and its own tool_result live in the ordinary events stream, so `RunResult.presentedFiles` re-derives on replay at container, where the agent's cwd IS the session root the live lane measures from; at hostloop the live lane measures from the session root while a re-drive has only the recorded cwd (`mnt/outputs` before Desktop 2.7032.0, an empty macOS system dir from it), so the promoted/leaked booleans there are not equivalent — immaterial to this key, which evaluates at container only (meaningfully replay-checkable, same as `skill_triggered`); evidence-unavailable if `presentedFiles` telemetry is absent (an older run predating the feature). **Container-only on the merits**: hostloop serves `present_files` but never promotes (its handler passes a validated path through), so there is no scratch→outputs copy that could leak. On microvm/protocol a scratchpad-delivered file is neither promoted to `mnt/outputs` nor detected there (a skill that delivers via write-to-cwd→`present_files` will false-red `user_visible_artifact` on those tiers; use `container`, or write directly to `outputs/`). Hostloop is not one of them: at hostloop a delivered file under the outputs dir is visible there immediately, so `user_visible_artifact` passes (before Desktop 2.7032.0 a write-to-cwd landed there; from it the agent runs at `/var/empty` and a relative write is refused). **The tool name is lane-specific:** `present_files` is the local lane's tool (the one this harness emulates); remote Cowork delivers via the agent-native `SendUserFile` instead, so a skill should describe the delivery outcome rather than naming either tool — this key asserts the harness-side delivery record either way. **Only `true` is valid** |
+| `no_scratchpad_leak: true` | every file presented via `present_files` from the scratchpad was promoted to `mnt/outputs`; vacuous when nothing was presented (pair with a presence check); content-class, re-derived on replay at `container`, the only tier it evaluates on (hostloop never promotes; microvm/protocol do not serve `present_files`); evidence-unavailable without `presentedFiles` telemetry. Remote Cowork delivers via `SendUserFile`: describe the outcome, not the tool. Only `true` is valid; details in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
 | `present_files_called: true` | at least one file was actually delivered via the `present_files` tool (`RunResult.presentFilesCalls > 0` — the count of invocations that carried a well-formed `file_path`). Presence is read at the **invocation**, deliberately not off the classified `presentedFiles` list: that list drops any path it cannot resolve, and a host-path redaction policy makes that certain at `hostloop`, where a real host path redacts to `[REDACTED:…]/mnt/outputs/f`. So this key is safe to assert in a scenario you intend to `record` with redaction active — `record` refuses a cassette whose verdict redaction changed, and an invocation count cannot be changed by it. A run that called the tool but whose every call carried an unusable path reports **cannot verify**, never a claim of non-delivery. The presence companion to `no_scratchpad_leak` (which passes vacuously when nothing was presented). Pair them to require a delivery **and** require it not to leak. Content-class (re-derives identically on replay). **`fidelity: container` or `hostloop`** — the harness serves `present_files` at both (hostloop via a handler mirroring production's own host-loop branch: validate the path, pass it through, no promotion). `protocol` and `microvm` report cannot-verify. See the `no_scratchpad_leak` row, which stays container-only for a different reason; and see the lane note there — the tool name differs on remote Cowork. **Only `true` is valid** |

package/.claude/skills/cowork-harness/references/assertion-catalog-gates-hooks-modifiers.md

@@ -1,6 +1,6 @@
 # Assertion catalog: gates, hooks, path denial, verdict modifiers, egress, judged keys
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md).
 
@@ -17,13 +17,13 @@ refused at load: no question reaches the harness to grade.
 | `gate_answers_delivered: true` | every answered gate's answer reached the model (observed `tool_result`; unobserved = fail); **zero gates fired passes vacuously** — pair with `gate_answer_count_min: >= 1` to also require a gate, or drop this key and declare `questions_count_max: 0` in a scenario that expects none |
 | `gate_answers_delivered: false` | asserts at least one answered gate's answer was **confirmed not delivered** (an observed delivery failure); an unobserved/null delivery does **not** satisfy this — for negative-path delivery tests. Requires a gate, so **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
 | `gate_answer_count_min: <N>` | at least N AskUserQuestion gates fired AND were delivered non-error — presence companion to `gate_answers_delivered`'s vacuous-pass. **`: 0` asserts nothing** and does not satisfy that pairing; `>= 1` is **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
-| `gates_all_scripted: true \| {include_permissions: true}` | every AskUserQuestion gate that fired was answered by a scripted rule (`answers:` / `--answer`) — not the LLM decider, `first`, or an external or human decider; a failure names each such gate and who answered it. A gate asked but never answered fails. **Evidence:** the run's decisions — a result with no decisions record, or an answer with no recognisable source, fails evidence-unavailable, never passes. On replay each gate is re-classified against the cassette's frozen `answers:`; that is evidence-unavailable when those answers were redacted, or when the cassette records that a live decider answered a gate yet its frozen rules cover every gate (they are not the rules it recorded with). **`{include_permissions: true}`** also checks tool permissions: a scripted `when_tool` rule or a fixed rule counts (a Read/Glob/Grep registry allow, a strict-parity deny, the harness's own path-gate deny, the fail-closed deny when nothing answered live — on replay that row means the recording held no answer, evidence-unavailable); a web_fetch deny that no answer source decided fails (`answered by abstain-fallback`), because real Cowork asks the user (under the default `on_unanswered: fail` such a fetch ends the run as an unanswered gate before any deny, so this arises only under `first`: the LLM decider always picks a button or fails the run); cowork parity's permissive off-registry auto-allow never does, and a replayed web_fetch permission cannot be attributed (evidence-unavailable). Zero gates passes (nothing needed a person), so pair it with `gate_answer_count_min: >= 1` or `questions_count_max` to state whether gates were expected (`lint` warns `unpaired-gates-all-scripted`). |
+| `gates_all_scripted: true \| {include_permissions: true}` | every fired AskUserQuestion gate was answered by a scripted rule (`answers:`/`--answer`), not the LLM decider, `first` or a human; `include_permissions` adds tool permissions (`when_tool`/fixed rules count; a web_fetch deny that no answer source decided fails (`answered by abstain-fallback`), reachable only under `first`, not `on_unanswered: fail`). Replay re-classifies on the frozen `answers:`; zero gates passes, so pair with `gate_answer_count_min: >= 1` or `questions_count_max` (`lint`: `unpaired-gates-all-scripted`); details in the scenario reference (linked above) |
 | `hook_blocked: <regex>` | one of the harness's own PreToolUse hook callbacks blocked a tool whose name matches the regex (`RunResult.hookEvents`; a plugin's hook never reaches it: use `hook_event_blocked`) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette (a custom hook's decision lives only there, not the recorded stream). **Does not see the agent's own refusal**: on `hostloop` against Desktop 2.7032.0+ a relative `Read`/`Write`/`Edit` is denied by the agent's permission rules before any hook runs, so neither this key nor `path_denied` records it — assert it with `tool_result_contains: "denied by your permission settings"` |
 | `no_hook_blocked: true` | no tool was blocked by the harness's own hook callbacks (a plugin's hook never reaches it: use `no_hook_event_blocked`) (distinguishes a real tool crash from an intentional hook block) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette. **Only `true` is valid** |
 | `hook_event_fired: <HookEvent>` | a **command hook** for this event (a plugin's `hooks/hooks.json` or manifest hook — `Stop`, `SessionStart`, `PostToolUse`, …) ran: a `hook_response` system frame with that `hook_event` was recorded (`RunResult.contextEvents`). Any outcome counts. The harness passes `--include-hook-events` whenever a staged plugin declares hooks — that is what puts events other than SessionStart/Setup on the stream — so a recording made without it reports "never fired". Content-class, grades on replay. Recorded end-to-end for `Stop` ([stop-hook-probe.scenario.yaml](https://github.com/yaniv-golan/cowork-harness/blob/main/examples/probes/stop-hook-probe.scenario.yaml)) and for `PreToolUse` at `container` (deciding by JSON and by exit 2); the other names match the same frame but have not each been recorded |
-| `hook_event_blocked: <HookEvent> \| {event, tool?, via?, min?, max?}` | that command hook **blocked**: a `hook_response` frame for the event denied — by exit code 2, or (object form only) by a JSON decision on stdout on a frame the agent marks `outcome: success` (exit 0, or an HTTP hook's 2xx status): a PreToolUse `permissionDecision: "deny"` or a top-level `decision: "block"`, which the agent treats alike. A JSON decision on any other frame, such as exit 1, is unreadable, since the frame does not show whether the agent applied it; a hook the agent cancelled (timed out or aborted) decided nothing by its output. A hook that sets `onFailure: "block"` (agent 2.1.295 and later; a command hook that is not `async` or `asyncRewake`, or an http hook) is the exception: the agent turns its failure or its timeout into a block after it emits the frame. So on an event the run has such a hook on, a timed-out frame and a failed one (any exit but 0 and 2, or an HTTP status outside 2xx) are unreadable. The same holds when the run's hook sources could not all be read (`RunResult.hookFailureBlocks`). Stop, SubagentStop, TaskCompleted and TeammateIdle are exempt, as the agent does not block there. Only stdout that parses whole as a JSON object decides, since stdout is the hook's ordinary output too: a word like `deny`, prose, or JSON after other text is no decision. The bare event means at least one frame with exit code 2, the channel this key has always counted; a JSON deny is not counted there, and when the bare form fails it names any JSON deny it saw. The **object form** counts blocking frames by either channel, one per hook run (PreToolUse runs once per matching tool call, subagent calls included); `via: exit2` counts exit code 2 alone, `via: json` the JSON deny alone, and `via: any` (the default) either. `{event: Stop, max: 0}` is the per-event negative, and fails on a hook that denied by JSON alone. `tool` keeps only frames whose `hook_name` is `<event>:<tool>`: the tool that **fired** (or the SessionStart source), not the matcher in hooks.json. The shell is `Bash` at `container` and `mcp__workspace__bash` at `hostloop`, so the wrong name reports "never fired". An event whose frames carry no tool name (`Stop` is named `Stop`) reports evidence-unavailable for any `tool`. Fails naming the exit codes seen when the hook fired without blocking; fails "never fired" when no frame is in scope, `max: 0` included, so a disabled hook never passes. A frame whose outcome cannot be read (no exit code, a hook that started and never answered, stdout a redaction policy rewrote or the agent truncated, output the agent did not apply as a verdict) is evidence-unavailable when it could change the verdict. A Python hook whose script is missing (`python3 missing.py`) also exits 2, so it reads as a block. Cannot-verify when the run has no context events. Frames carry no plugin id: when a second staged plugin declares the event, or a `protocol` run can see hooks installed on the host, the run warns that the frame cannot be attributed to the plugin under test; the verdict does not change. Content-class |
+| `hook_event_blocked: <HookEvent> \| {event, tool?, via?, min?, max?}` | a command hook for `event` blocked: the bare form counts exit-2 frames only; the object form also counts a JSON deny on an `outcome: success` frame (`via: exit2`/`json`/`any`, default any), `tool` = the tool that fired (`Bash` at container, `mcp__workspace__bash` at hostloop), `min`/`max` a range (`{event: Stop, max: 0}` is the negative). No frame in scope fails "never fired"; an unreadable frame (incl. a failed one on an `onFailure: "block"` hook) is evidence-unavailable when it could change the verdict. Content-class; details in the scenario reference (linked above) |
 | `no_hook_event_blocked: true \| {event, tool?}` | no command hook blocked: every `hook_response` frame (or every frame for `event`) neither exited 2 nor denied by JSON, the same rule as `hook_event_blocked`'s object form. **Never vacuous**: no frame in scope is evidence-unavailable, and `true` also needs a frame of an event other than SessionStart/Setup, because those stream even when hook events were not requested (they stream only when a staged plugin declares hooks). A frame whose outcome cannot be read (no exit code, a hook that started and never answered, stdout a redaction policy rewrote or the agent truncated, output the agent did not apply as a verdict, or a failed or timed-out frame on an event with an `onFailure: "block"` hook — see `hook_decision`) is evidence-unavailable. Hooks on PreCompact, PostCompact, ConfigChange, DirectoryAdded, Elicitation, ElicitationResult, InstructionsLoaded, Notification, SessionEnd, StopFailure, WorktreeCreate, WorktreeRemove, CwdChanged and FileChanged stream no frame at all, so a block there never shows: `true` is evidence-unavailable when one of those events has an `onFailure: "block"` hook, or when the run's hook sources could not all be read. Frames carry no plugin id, so a second staged plugin's hook, or a host plugin's at `protocol`, counts as a frame here (the run warns). Distinct from `no_hook_blocked`, which reads the harness's **own** PreToolUse callbacks from `controlOut`. Content-class |
-| `hook_decision: {event, decision, tool?, min?, max?}` | counts the `hook_response` frames for `event` whose decision is `decision`, read from the hook's stdout JSON on a frame the agent marks `outcome: success` (exit 0, or an HTTP hook's 2xx status): `hookSpecificOutput.permissionDecision` on `PreToolUse` and `PreModelSwitch`, where it overrides a top-level `decision`, and which the agent ignores on other events; for `PermissionRequest` `hookSpecificOutput.decision.behavior`; for `Elicitation` and `ElicitationResult` an `action: decline`, a deny; else the top-level `decision`; or from exit code 2, which is a deny. A JSON decision on any other frame, such as exit 1, is unreadable, since the frame does not show whether the agent applied it; a hook the agent cancelled (timed out or aborted) decided nothing by its output. A hook that sets `onFailure: "block"` (agent 2.1.295 and later; a command hook that is not `async` or `asyncRewake`, or an http hook) is the exception: the agent turns its failure or its timeout into a block after it emits the frame. So on an event the run has such a hook on, a timed-out frame and a failed one (any exit but 0 and 2, or an HTTP status outside 2xx) are unreadable. The same holds when the run's hook sources could not all be read (`RunResult.hookFailureBlocks`). Stop, SubagentStop, TaskCompleted and TeammateIdle are exempt, as the agent does not block there. A run whose agents all report 2.1.293 or earlier in their init frames has no `onFailure`, so its inventory is empty whatever its sources say. The inventory reads as unknown when a hook source exists but cannot be read, or when there is too much to walk. The likeliest case is one installed plugin with a broken manifest or `hooks.json`. Rarer: a settings file that does not parse; a skill, command or agent whose frontmatter declares `hooks:` and does not parse; more than 50,000 entries walked in all, across the staged plugins and the config dir's skills, commands, agents and plugins. That needs a very large tree, or a plugins dir whose `installed_plugins.json` is missing or does not parse, so every plugin dir is walked. `decision` is one of `allow`, `deny`, `ask`, `defer`, as the agent applies them, plus two aliases: `block` means `deny` and `approve` means `allow`. So `deny` matches exit 2, `permissionDecision: "deny"` and `decision: "block"` alike, and the message shows which. Range and `tool` as for `hook_event_blocked` (default: at least one). Only stdout that parses whole as a JSON object decides; empty or non-JSON stdout is no decision, not an error, unless the agent rejected the frame's output, so `allow` never matches a hook that printed nothing. Fails "never fired" when no frame is in scope. A frame whose decision cannot be read (stdout a redaction policy rewrote so it does not parse, truncated output, a frame with no `stdout` field, a decision shape it does not model, a top-level `decision` other than `approve` or `block`, output the agent did not apply (`outcome: "error"` with exit 0, or stderr that opens with the agent's rejection of the JSON, its refusal to read an incomplete capture, or its failure to run the hook), no exit code, a hook that started and never answered, a `hookSpecificOutput` whose `hookEventName` is missing or names another event) is evidence-unavailable when it could change the verdict. Frames carry no plugin id: when a second staged plugin declares the event, or a `protocol` run can see hooks installed on the host, the run warns that the frame cannot be attributed to the plugin under test; the verdict does not change. `record` warns when its redaction policy rewrites hook stdout that this key, `hook_event_blocked` or `no_hook_event_blocked` reads, since the committed cassette's verdict could then only be evidence-unavailable. Content-class, grades on replay |
+| `hook_decision: {event, decision, tool?, min?, max?}` | counts `hook_response` frames for `event` whose decision (stdout JSON on an `outcome: success` frame, or exit 2 = deny) is `decision`: `allow`, `deny`, `ask`, `defer`, or the aliases `block` (deny), `approve` (allow); `tool`, `min`/`max` as for `hook_event_blocked` (default: at least one). Empty/non-JSON stdout is no decision; no frame fails "never fired"; an unreadable frame (incl. a failed one on an `onFailure: "block"` hook) is evidence-unavailable when it could change the verdict. Content-class, grades on replay; details in the scenario reference (linked above) |
 | `hook_output_contains: {event, stream?, text \| matches}` / `hook_output_not_contains: {…}` | a command hook **printed** (or never printed) a text on `stdout` / `stderr` / either (`stream`, default `any`) — some (or no) `hook_response` frame for `event` carries `text` (literal, case-sensitive) or `matches` (regex, case-insensitive, no multiline flag — `^`/`$` anchor the whole stream). For a hook that fails open on stderr while exiting 0. No frame for the event FAILS both. On a redacted stream, a literal miss and any `matches` result are evidence-unavailable for either key, while a literal hit outside a token counts; a miss is also evidence-unavailable on a missing field, a hook that started without a response, or output the agent truncated. Frames carry no plugin id (a second plugin, or at `protocol` without a sealed config dir a host-installed one, can answer). Content-class. Recorded end-to-end for `Stop` |
 | `vm_path_denied: true` | **`fidelity: hostloop` only** — at least one recorded path denial (`RunResult.pathDenials`, any source) targeted a `/sessions` VM path — evidence-unavailable if path-denial telemetry is absent. Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify". **Only `true` is valid** |
 | `path_denied: {tool?, path_matches?, source?, agent_scope?}` | **`fidelity: hostloop` only** — a path denial matching ALL given matchers (`tool` glob, `path_matches` regex, `source` ∈ pretooluse/can_use_tool/permission_denied, `agent_scope` ∈ main/subagent/any) was recorded. Replay: needs a `controlOut` cassette. Any other tier FAILS "cannot verify" |
@@ -39,10 +39,12 @@ refused at load: no question reaches the harness to grade.
 | `egress_denied: <host>` | the host was blocked by the egress proxy |
 | `egress_allowed: <host>` | the host was allowed through |
 | `no_mcp_error: true` | no MCP round-trip failed (`RunResult.mcpErrors` is empty — no unhandled server, no handler throw) — live-only: MCP round-trips are harness-computed, not in the SDK stdout stream, so evidence-unavailable on replay (never a vacuous pass). **Only `true` is valid** |
+| `search_corpus_max_miss_rate: <0..1>` | a `--search-corpus` run's misses / (served + misses) ≤ N; never clean with a refused search. Live-only; full rules in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
+| `search_corpus_nearby_citations: {artifact, max?}` | a `--search-corpus` run's text deliverable cites ≤ `max` URLs served only on nearby rounds. Live-only; rejected at load on `lane: remote`; full rules in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
 | `max_peak_rss_bytes: <N>` | peak sampled RSS of the agent sandbox ≤ N bytes (`RunResult.resources.peakRssBytes`) — live-only: replay never spawns a sandbox to sample, so evidence-unavailable on replay/protocol (never a vacuous pass); also evidence-unavailable when sampling captured no RSS value |
 | `semantic_matches: {rubric: [...], min_pass?, judge_model?, include_subagent_text?, include_fork_results?, evidence_files?: [..]}` | a pinned LLM judge grades each fixed `rubric` claim against the run's answer: the final result text, the transcript and the files the agent authored. The transcript is top-level `assistant_text` only — it excludes every `tool_use`/`tool_result` and sub-agent text unless opted in. Full semantics (evidence scope, fork results, refusals, judge provenance, cost): [semantic-judging.md](semantic-judging.md). On `lane: remote`: judged on the transcript only; `evidence_files` is rejected at load. |
-| `semantic_pairwise: {refs?: [..], rubric?: [..], pass_if?, order?, judge_model?, include_subagent_text?, include_fork_results?, evidence_files?: [..]}` | a pinned LLM judge compares this run's judged document with a **frozen reference** — the same document an earlier run (usually the baseline) produced, written once by `ref freeze` and never regenerated — and answers win, tie, loss or `both_bad`. **The judged document is the one `semantic_matches` builds** (final message + transcript + authored files, same `evidence_files` / `include_subagent_text` / `include_fork_results` options), so the transcript **excludes every `tool_use`/`tool_result`** and a criterion about whether a tool was called cannot be judged from it. `refs` are reference stores relative to the scenario file; the run is judged against every one (under `hillclimb run`, against the flow's own references instead — only the baseline's gates `pass_if`, a later variant's is a metric recorded with `gate: false`), and `pass_if` (default `not_worse`) must hold against each gating one: `win`, `not_worse` (win or tie), or `any` (graded at all — a metric only). `both_bad` fails `win` and `not_worse`. Which output the judge sees first is a seeded coin per run, assert and reference (`order: both` judges both orders: a win in one and a loss in the other is position bias and scores as a tie; any other disagreement keeps the worse outcome; each order's own outcome is recorded as `pairwise[].orders.candidate_first` / `.ref_first`, absent for a single-order grade, and a judge that favours whichever output it sees first shows across runs as `candidate_first` winning more often than `ref_first`). The judge never sees the words "reference" or "baseline", and the stored reference is scrubbed with the grading process's secret set before it reads it (`pairwise[].refRedactions`). A reference must have been frozen with the same evidence options and for the same prompt; a missing, damaged, differently-scoped or other-prompt one **refuses the run before it spends anything**, as does a reference store inside any mounted source (the agent could read its own answer key). Unavailable evidence refuses with `semantic_matches`' typed reasons. Per-reference outcomes land in `assertions[].pairwise`. **LIVE-ONLY** — skipped on replay. |
-| `artifact_json: {artifact, path, …}` | assert a JSON artifact's contents — `equals`/`gt`/`in`/`exists`/`absent`/`is_null` over a dotted `path` (`in` = membership in a list, for a stochastic/LLM value; `absent` ≠ `is_null`; an unresolved intermediate fails loud). A glob `artifact` (`*`, `?`, `**`; `[` is literal) needs `match: each` (every match) or `match: any` (≥1), e.g. `{artifact: outputs/artifacts/runs/*/run_status.json, match: each, path: status, equals: complete}`. It matches files under the user-visible roots only; uploaded inputs are not matched. Zero matches fail; >200 matches, a link (a matched file, or a symlinked directory on the way to one), a body-less match or an incomplete walk (over 32 levels below the glob's fixed part, over 20,000 entries, or an unreadable directory) is evidence-unavailable. Names match exactly, with no case or Unicode folding. `authored: true` applies to every match. A glob that can reach no user-visible root (an `uploads/` glob) fails and says why. At `record`, a match stored hash-only over the body cap, or an artifact walk that could not see the whole tree, is refused, as for a literal path. Rejected at load on `lane: remote`, in every form (that lane's container filesystem is not locally observable). `schema: <JSON Schema>` (draft 2020-12, inline or `{file: <path>}` read at load and inlined) checks the value at `path` (or the whole document): the first 5 errors with JSON paths; refused at load: `format` (use `pattern`), `$id`, a non-local `$ref`, `$dynamicRef`/`$dynamicAnchor`/`$recursiveRef`/`$recursiveAnchor`, `nullable`, the `content*` keywords, the draft-07 list-form `items` (use `prefixItems`), a `__proto__` key, NaN/Infinity in `enum`/`const`/`default`/`examples`, a `$ref` to the root with nothing to stop the recursion, an unknown keyword, a type-specific keyword (`required`, `properties`, `items`, `minimum`, …) with no `type`, a root with no validating keyword, a `$schema` other than draft 2020-12, a schema deeper than 64 levels or one that contains itself (a YAML alias loop; write recursion as a local `$ref` to `$defs`) |
+| `semantic_pairwise: {refs?: [..], rubric?: [..], pass_if?, order?, judge_model?, include_subagent_text?, include_fork_results?, evidence_files?: [..]}` | a pinned LLM judge compares the document `semantic_matches` builds (it excludes every `tool_use`/`tool_result`) with each frozen reference in `refs` (written once by `ref freeze`; under `hillclimb run` the flow's own, later variants `gate: false`): win, tie, loss or `both_bad`; `pass_if`: `win`, `not_worse` (default) or `any`; `order: both` judges both orders (position bias = tie). A missing or mismatched reference refuses the run before it spends. LIVE-ONLY, skipped on replay; details in the scenario reference (linked above) |
+| `artifact_json: {artifact, path, …}` | a JSON artifact's value at a dotted `path` (or the whole document, with no `path`) satisfies `equals`/`gt`/`in`/`exists`/`absent`/`is_null` or `schema` (inline or `{file}`); a glob `artifact` needs `match: each`/`any` and fails on zero matches; `authored: true` applies to every match. Rejected at load on `lane: remote`; details in the scenario reference (linked above) |
 | `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). **Requires ≥1 link** (zero links fails — use `computer_links_resolve_if_present` for the presence-free variant). **Only `true` is valid** (`false` is rejected by the schema) **Sees top-level `assistant_text` only — it excludes every `tool_use`/`tool_result`**, so a `computer://` link that appeared only inside a tool call or its result is invisible to it. Rejected at load on `lane: remote`. |
 | `computer_links_resolve_if_present: true` | like `computer_links_resolve` but passes vacuously when the transcript has zero `computer://` links — the presence-free variant. **Only `true` is valid** **Sees top-level `assistant_text` only — it excludes every `tool_use`/`tool_result`**, so a `computer://` link that appeared only inside a tool call or its result is invisible to it. Rejected at load on `lane: remote`. |
 

package/.claude/skills/cowork-harness/references/assertion-catalog-outcome-files-tools.md

@@ -1,6 +1,6 @@
 # Assertion catalog: outcome, transcript, files and artifacts, tools, tool results
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 
@@ -23,7 +23,7 @@ keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 | `no_lost_write_back: true` | fails if the run authored an interactive HTML artifact (or a `.py`/`.js` generator of one) whose **relative** Submit/POST write-back is lost under Cowork (served from Cowork's own origin → resolves non-ok, a "Saved!" is silently false). Runs the shipped **static Tier A** analyzer over the files the run authored (diffed vs the pre-run manifest). A lost write-back on an **added** agent-authored source (`outputs/`, scratchpad) **fails**; a **pre-existing** file the skill only modified on a read-write mount is **advisory**; `-suspect` findings surface but pass. **Only `true` is valid**. **Live/verify-run only** — skipped-loud on replay; runs on every live sandbox tier including **microvm** (its outputs are snapshotted from the VM into the run dir); could-not-verify (fail-closed) on a `--resume` scratchpad or an unanalyzable candidate. Rejected at load on `lane: remote`. |
 | `tool_called: <glob>` | a tool the agent ran matched this **glob** — `*` = any run, `?` = one char, exact when literal, anchored + case-sensitive. Exact name (`Write`) matches only that tool; `mcp__workspace__*` matches any workspace tool. GLOB, not regex (`.` is literal) — an empty glob, or one containing a regex/brace-expansion metacharacter (`.*`, `.+`, `\|`, `()`, `[]`, `+`, `^`, `$`, `{}`, `\d`/`\w`/`\s`/`\b`), is **rejected at load** (a hard schema error, not a runtime warning) — it would match no real tool name and pass a `_not_`/`_absent` assert vacuously. Applies whether the glob comes from an authored scenario or a recorded cassette's frozen assert. `cowork-harness lint` reports it too (ERROR `scenario-invalid` — the wrapper runs the same loader); the bundled `scenario.py lint` run directly does NOT |
 | `tool_not_called: <glob>` | NO tool the agent ran matched this glob (`mcp__*` = "no MCP tool ran"). Same glob semantics as `tool_called`, including the empty/regex-ish rejection |
-| `tool_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?, count?}` | **object form** — asserts what a call CARRIED, where it RAN and what its paired result SAID, which no name glob and no `transcript_*` key can see (`transcript_*` reads top-level prose only, so it passes when the agent merely *says* it ran a command). `tool`: a glob or a list of globs (any-of; list both shells, `[Bash, mcp__workspace__bash]`, for a claim that must hold at hostloop too). `input: {<field>: <regex>}`: every named top-level input field must match (case-insensitive, unanchored; a missing field is no match; a non-string value is matched as JSON). `input_any: <regex>`: some top-level field matches. `result: {matches?, not_matches?, is_error?}`: predicates on the call's paired `tool_result` (by `toolUseId`, 10,240 characters of text — 32,768 for a top-level `Skill` result). `scope`: `main` (default — the main agent, including a `Skill`'s or `Agent(fork)`'s children, the set the string form reads), `subagent` (the parent is a sub-agent dispatch this run recorded, at any depth), or `any` (also calls whose parent is not a recorded dispatch, e.g. a `Skill` invoked inside a sub-agent). `subagent_type` (with `scope: subagent`): regex over the IMMEDIATE parent dispatch's type or description. `count: {min?, max?}` (default `min: 1`). **Fails closed:** an unpaired call never satisfies `result`; a truncated result or input that cannot settle a predicate, or a result.json without `toolCalls`, reports *evidence unavailable*, never a pass or a plain "not called". A red lists the calls it considered and names matches in another scope ("1 matching call in scope subagent — set `scope: any`"). `{tool: X}` alone is exactly `tool_called: X`. A cassette using this form stamps **v13**. |
+| `tool_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?, count?}` | **object form**: what a call CARRIED, where it RAN and what its paired result SAID. `tool`: glob or any-of list; `input: {<field>: <regex>}`; `input_any: <regex>`; `result: {matches?, not_matches?, is_error?}` over the paired `tool_result`; `scope`: `main` (default)/`subagent`/`any`; `subagent_type` (with `scope: subagent`): regex over the IMMEDIATE parent dispatch's type or description; `count: {min?, max?}` (default `min: 1`). Fails closed: an unpaired call never satisfies `result`; truncated evidence is evidence unavailable, never a pass. Stamps cassette v13; details in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
 | `tool_not_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?}` | **object form** — no call in scope satisfies every predicate (no `count`). Same fields and fail-closed rules as above: a candidate whose result is unpaired or truncated, or whose input was truncated, is *evidence unavailable*, never a pass. Refused at load when EVERY listed tool is one the tier does not serve. ⚠️ **Redaction hazard — this is the dangerous direction.** A committed cassette is redacted, which rewrites the recorded inputs: an `input` regex naming a literal the policy rewrites (a home path, an email) cannot see its target on replay. On replay, any candidate call whose field (or paired result, for `result.matches`) carries a redaction token is *evidence unavailable*, whatever the regex — the evaluator cannot know what the token replaced — so a negative check never passes over rewritten bytes. `record` warns — naming the redacted call and the ways out (narrow `scope`/`tool`, the string form, or live-only) — and refuses the write (the exact guard), and `lint` flags a regex naming a redactable literal (`tool-input-regex-redactable`) — a heuristic that reads `.cowork-redact.json` from the current and scenario directories only, not the cassette's. Match a part of the input redaction leaves alone (the verb and flags, a workspace-relative path), or keep the check on a live gate. Note also that the default `scope: main` does not see a sub-agent's call — use `scope: any` for "nothing anywhere ran X". |
 | **legacy tool names** | The agent binary canonicalizes legacy spellings (`Task`→`Agent`, `KillShell`/`KillBash`→`TaskStop`, …) while the spawn tool list still declares the LEGACY one — so the init inventory shows `Task` and every call is emitted as `Agent`. `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` match EITHER spelling, globs included. Before this, `tool_called: "Task"` could never pass and `tool_not_called: "Task"` always did. |
 | **tier vacuity — now REFUSED** | `tool_not_called` **and `subagent_tool_absent`** naming a tool the tier does not serve is rejected at scenario load (`Bash`/`WebFetch`/`NotebookEdit` at `hostloop`; `mcp__workspace__bash` at `container`/`microvm`; on those three tiers, `mcp__remote-devices__device_bash`/`get_device_info`/`device_request_delete_permission` without `device_bridge: true`, and every other `mcp__remote-devices__*` tool the harness never serves, bridge or not; and `mcp__visualize__read_me`/`show_widget`, production's skill-argument form tools, which the harness never serves unless the session's own MCP config declares a `visualize` server), because it could never be violated. `scenario.py lint` WARNs on the same set before any run. The table is CLOSED to tools the harness itself removes or registers — `--tools` gates only the built-in set while every tier separately passes `--mcp-config`, so a session-MCP tool name is offered without appearing in any tool list and is never refused. Globs are never refused. `subagent_tool_absent` is covered because it reads the tools sub-agents actually USED, not a per-dispatch declared list. Under-approximates on purpose: `REPL` at hostloop is vacuous too and is not caught, which is the right side to err on for a hard refusal. |

package/.claude/skills/cowork-harness/references/assertion-catalog.md

@@ -1,6 +1,6 @@
 # Assertion catalog
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Every `assert:` key with its semantics, and the
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Every `assert:` key with its semantics, and the
 verdict-signal table. Which keys survive `replay` is in [`scenario-schema.md`](./scenario-schema.md#replay-class);
 the scenario and session YAML fields are there too.
 
@@ -52,7 +52,7 @@ predicates neither can express.
 
 **VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; twelve
 are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
-`"success"`) and fifteen are **warn**-severity (informational, never flip pass/fail). All twenty-seven signal
+`"success"`) and eighteen are **warn**-severity (informational, never flip pass/fail). All thirty signal
 codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 | Code | Severity | Meaning |
@@ -82,9 +82,12 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 | `undelivered_deliverables` | warn | The skill produced file(s) OUTSIDE every user-visible root and never delivered them, so they stay invisible to the user. Fires on every run without opting in, because the scenarios that most need it are the ones whose author never considered delivery. Silent when the evidence cannot answer the question (no workspace walk, a tier that runs no scratchpad walk, absent delivery telemetry, a resumed turn, or a lane where delivery is unobservable — see `delivery_unobservable`) — never a vacuous clean. **`lane: local` only**: on remote, delivery cannot be measured at all, so that lane reports `delivery_unobservable` instead of guessing. Opt out: `allow_undelivered_deliverables` |
 | `delivery_unobservable` | warn | `lane: remote` only — the run produced file(s), but whether any reached the user CANNOT be verified: nothing is delivered by location on that lane and the harness models no remote delivery tool (production uses the agent-native `SendUserFile`). The honest counterpart to `undelivered_deliverables`, which would otherwise fire on every remote run that writes anything — a signal that always fires carries no information. Mutually exclusive with it; quiet when the run produced nothing to deliver. A harness coverage gap, not a skill defect. Opt out: `allow_undelivered_deliverables` |
 | `partly_scripted_gate` | warn | A question batch (one `AskUserQuestion` with several sub-questions) that the scenario's `answers:` matched only PART of. Answers are delivered atomically, so the whole batch went to the `on_unanswered` fallback and the matched answers were NOT delivered — the fallback may contradict them. The message names the matched and unmatched sub-questions and who answered instead; `result.partlyScriptedGates` carries the full lists. Fires only on a partial match within one batch (a single-question gate no rule matched is the ordinary unanswered case). Re-derived on `replay` (from the cassette's frozen answers) and `verify-run` (from the current scenario). Fix: script every sub-question of the batch |
+| `search_corpus_refused` | warn | A `--search-corpus` run refused some searches ("Web search error: unavailable"); all refused usually means an agent newer than the baseline's anchors: re-sync. Never changes the verdict; details in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
+| `search_corpus_attribution_uncertain` | warn | A `--search-corpus` run in which some sub-agent search results could not be settled to their request; those `searchCorpus.calls[]` entries stay `provisional`. Never changes the verdict; details in the scenario reference (linked above) |
+| `search_corpus_semantic_judge` | warn | A judge (`semantic_matches`/`semantic_pairwise`) graded a `--search-corpus` run: its searches were recorded answers, so the grade is not research quality. Also from `regrade`. Never changes the verdict; details in the scenario reference (linked above) |
 | `parked_at_question` | warn | The session declares `answer_channel: none` and the run ended on a question nobody can answer: its last message ends in `?`, even after tools ran (a finished answer ending on an offer warns too; a request without `?` does not fire). Takes the place of the `stalled` fail and never changes the verdict: with no answer channel, stopping at the question is the contract. Completion is judged from the file assertions such a scenario must carry (refused at load without one). No opt-out needed |
 | `exec_infra_error` | warn | Host-loop, or a `device_bash` call under `device_bridge`: one or more container `exec` calls failed for infrastructure reasons, so those tool calls returned an error to the agent. Warns rather than fails because the run's other evidence is intact — unlike `infra_error`, where a dead supervisor contaminates everything. Caveat: if *every* exec failed, the agent ran nothing and this still only warns — check `result.infraErrors` |
 
 A **fail**-severity signal does not change `result.result` (still `"success"`), but it DOES fail the
 overall run verdict and exit code — `assert result: success` alone won't catch it; check
-`result.verdict.signals[].severity` or the run's exit code. Only the fifteen **warn** codes are truly benign.
+`result.verdict.signals[].severity` or the run's exit code. Only the eighteen **warn** codes are truly benign.

package/.claude/skills/cowork-harness/references/assertions-guide.md

@@ -1,6 +1,6 @@
 # Assertions guide
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
 
 ### Assertions: two orthogonal axes
 

package/.claude/skills/cowork-harness/references/authoring.md

@@ -1,6 +1,6 @@
 # Authoring a scenario
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
 
 ## Part I — AUTHOR a scenario
 
@@ -327,6 +327,7 @@ cowork-harness lint scenarios/*.yaml
 | `workspace-fixture-invalid` | ERROR | the run refuses the scenario's `workspace_fixture` directory — missing, a symlink, a hard link, agent-config paths, untracked files in git mode, empty, over a size cap, a binary file holding a session-path token, or session-path tokens at `fidelity: protocol` (via `cowork-harness lint` only) |
 | `workspace-fixture-not-relative` | WARN | `workspace_fixture` is an absolute or `~/` path rather than one relative to the scenario file — it names a directory on one machine; when it does not exist where lint runs it is not checked (via `cowork-harness lint` only) |
 | `workspace-fixture-vacuous-assert` | ERROR | a `file_exists` / `user_visible_artifact` / `artifact_text` / `artifact_json` names a file (or directory) the `workspace_fixture` provides without stating `authored:` — it would pass on the fixture alone (via `cowork-harness lint` only) |
+| `search-corpus-key-needs-flag` | INFO | a `search_corpus_*` assertion: `run`/`skill` refuse it without `--search-corpus` (exit 2); `record` refuses the key (exit 1), and the flag itself (exit 2) |
 | `lint-loader-internal` | ERROR | the wrapper could not run its loader check on a file — a harness bug; it never falls back to a lint that skipped the loader |
 
 `scaffold` auto-upgrades the tier if you ask for egress on `protocol`, so it never emits a scenario `lint`

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`).
+Self-contained reference. Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
 copy-pasted recipe that omits the input takes a major bump with no say in it. `^4` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "4.9.1"`) instead when you want byte-reproducible CI.
+(e.g. `version: "4.10.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -36,7 +36,7 @@ jobs:
       - uses: actions/checkout@v4
       - name: Stage the agent binary (official channel, sha256-verified against the pinned baseline)
         run: |
-          V=2.1.295   # match your scenario's pinned baseline's agentVersion
+          V=2.1.299   # match your scenario's pinned baseline's agentVersion
           # The release channel is NOT always the stable one. Desktop also stages release CANDIDATES,
           # served from .../claude-code-releases/rc/<commit>/. For some versions the stable path 404s
           # (2.1.255); for others it returns 200 and serves a DIFFERENT BUILD UNDER THE SAME VERSION
@@ -82,7 +82,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^4.9.1"
+- run: npm i -g "cowork-harness@^4.10.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -411,7 +411,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^4.9.1"
+      - run: npm i -g "cowork-harness@^4.10.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -440,7 +440,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^4.9.1"
+        run: npm i -g "cowork-harness@^4.10.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). This is **not** a trim of the full
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reading several critiques, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/debugging.md

@@ -1,6 +1,6 @@
 # Debugging a run
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
 
 ## Part III — Debug
 
@@ -131,7 +131,10 @@ decide which assertions from *Assertions: two orthogonal axes* in `assertions-gu
 - **`result.json` carries the raw fields** the assertions read: `verdict`, `lane` (which Cowork delivery
   contract the run was held to — see gotcha 24 in `gotchas.md`), `answerChannel` (`"none"` on a run with nobody to
   answer; such a run is labelled `[headless, no answer channel — not Cowork]`, and `stats` lists it as its own row
-  under every `--group-by`), `scratchpadEvidenceComplete` (did a COMPLETE scratchpad
+  under every `--group-by`), `searchCorpus` (present when `--search-corpus` served the run's web searches from a pool:
+  the pool, its hash, the `frozen search corpus …` label and the counts served, missed and refused; `stats` groups such
+  runs per pool, never with live ones, and leaves a run with a refused search out of its pass rate, as `--repeat` does),
+  `scratchpadEvidenceComplete` (did a COMPLETE scratchpad
   walk observe this run — what distinguishes "nothing was left undelivered" from "cannot tell"), `cost` (`cost.usd` = the SDK's
   `total_cost_usd` for the run — the authoritative single-run spend of the agent session, which leaves out the `semantic_matches` judge and the LLM decider calls; NOT the same source as summing
   `modelUsage[].costUSD`, which is what `trace --view usage` reports, so the two can differ),

package/.claude/skills/cowork-harness/references/eval.md

@@ -1,6 +1,6 @@
 # `eval` — paired before/after comparison of a skill edit (EXPERIMENTAL)
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). The full guide is
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). The full guide is
 [docs/eval.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/eval.md); this is the part you
 need while running it.
 
@@ -87,6 +87,23 @@ names the arm, scenario, dominant error and a matching hint (`Every rep of arm <
   one `local_plugins` entry). Workaround: eval a copy of the session that declares the same directory under
   `local_plugins`, and check the `remote_plugins` path handling with an ordinary `run`.
 
+## Frozen search corpus (`--search-corpus`)
+
+- **Not for a change to what or when the skill searches**: a frozen corpus holds search fixed. Evaluate that live,
+  or split it: a live eval of the search change, then a frozen eval with a pool built evenly from both versions' runs.
+- `--search-corpus <pool>` (hostloop; `--search-corpus-miss`, `--search-corpus-min-score` as on `run`): one pool for
+  both arms, copied into the eval dir and pinned by hash; every rep checks the copy before it spends. `--dry-run`
+  copies and verifies it too, so it refuses where the real eval would. An older harness refuses to read such an eval
+  dir.
+- Refused before spend: a pool drawn from one arm, or with more than 60% of a scenario's compared-arm records from one
+  arm (duplicates counted the worst way) — `--allow-confounded-pool` runs it, labelled CONFOUNDED; a mixed pool
+  without `--allow-mixed-pool`; a pool built or exported before 4.10 that would read balanced (rebuild or re-export
+  it).
+- Buckets: `search_pool_mismatch` (not served from the pinned pool; any one exits 1) and `search_refused` (a search
+  was refused; never scored). Either filling an arm makes that scenario compare nothing.
+- The report prints both arms' served/miss/refused and mean top score, and marks asymmetric service CONFOUNDED
+  (placeholder thresholds; under 11 searches a side, not judged). CONFOUNDED never changes the exit code.
+
 ## Metric rows
 
 - One row per (scenario, declared metric). The test is exact two-sided Wilcoxon–Mann–Whitney, so ties and skewed
@@ -115,8 +132,8 @@ names the arm, scenario, dominant error and a matching hint (`Every rep of arm <
 
 `0` completed — no drop fails the eval unless you pass `--fail-on`. `1` a drop or a worse metric at the `--fail-on`
 level, every row `insufficient`, a scenario that compared nothing, a metric row `not compared`, or the judge model
-differed across reps (an A/A run under `--fail-on possible`
-can exit 1 on noise). `2` usage or a refusal. `3` an arm snapshot could not be copied or staged.
+differed across reps, or a rep was not served from the pinned pool (an A/A run under `--fail-on possible`
+can exit 1 on noise). `2` usage or a refusal. `3` an arm snapshot or the pool copy could not be made or staged.
 
 ## Files
 

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`).
+Self-contained reference. Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`).
 
 > **This page vs. the repo docs.** This is the **offline snapshot** that ships inside the installed
 > plugin — it is self-contained on purpose. The repo carries four other fidelity views, each answering a
@@ -297,7 +297,7 @@ up often enough to spell out:
   a filesystem or network — it re-evaluates assertions from the frozen cassette. A fixed set of
   keys is live-only and **skipped outright** on replay (absent from `assertions[]`, not vacuously
   passed): `file_absent`, `no_delete_in_outputs`, `no_delete_in_mounts`, `self_heal_ran`, `transcript_no_host_path`, `egress_denied`,
-  `egress_allowed`, `no_mcp_error`, `max_peak_rss_bytes`, `semantic_matches`, `semantic_pairwise`, `no_lost_write_back`, and `expect_denied`.
+  `egress_allowed`, `no_mcp_error`, `max_peak_rss_bytes`, `semantic_matches`, `semantic_pairwise`, `no_lost_write_back`, `search_corpus_max_miss_rate`, `search_corpus_nearby_citations`, and `expect_denied`.
   Everything else that *is* evaluated is checked against the **recording**, not fresh behavior — a
   green replay says the skill produced these events when it was recorded, not that it still does
   (`staleness[]` flags skill/baseline drift as a hint; only a live `run` re-confirms current

package/.claude/skills/cowork-harness/references/gotchas.md

@@ -1,6 +1,6 @@
 # Gotchas
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). The full "✓ passed ≠ correct" landmine catalog.
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). The full "✓ passed ≠ correct" landmine catalog.
 
 ## Gotchas — the "✓ passed ≠ correct" landmines
 

package/.claude/skills/cowork-harness/references/hillclimb-recipe.md

@@ -1,6 +1,6 @@
 # Recipe 7 — Climb a skill with `/claude-api hillclimb` and the harness as its runner
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). It needs a `cowork-harness` whose
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). It needs a `cowork-harness` whose
 `hillclimb --help` lists `--skill` (help goes to stderr). This page is the loop's procedure, step by step, in the order of the
 `/claude-api hillclimb` guide. Every mechanic (flags, refusals, the gate, `regrade`, `freeze-ref`, exit codes,
 row keys) is in [`hillclimb.md`](hillclimb.md); the setup and the full list of differences from the guide's own
@@ -132,6 +132,8 @@ Each line matches one entry of the list in [the hillclimb guide](https://github.
 
 ## Step 2 — stopping condition and budget
 
+- **Frozen search** (`run --search-corpus`): read each row's `meta.search_corpus` beside its score, and never
+  climb a search change on it (hillclimb reference).
 - **If the goal is cost** (the guide's cost search): run each model × effort cell as its own variant with
   `--model <id> --effort <level>`, one setting per variant. Prompt caching is the agent's, not the skill's: read
   `usage.cache_read_input_tokens` on the rows after each round, and treat a drop as the skill's content breaking the

package/.claude/skills/cowork-harness/references/hillclimb.md

@@ -1,6 +1,6 @@
 # `hillclimb` — the runner for a `/claude-api hillclimb` loop
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). It needs a `cowork-harness` whose
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). It needs a `cowork-harness` whose
 `hillclimb --help` lists `--skill` (help goes to stderr). The command reference is
 [docs/cli.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/cli.md); this is the part a loop needs
 while it runs. It covers `run`, `check`, `state-template`, `freeze-ref` and `regrade`.
@@ -79,6 +79,17 @@ firing first is a scored `errored_agent` row), `--model ID` and `--judge-model I
 `--case ID`, `--skill NAME`, `--approve-harness`, `--ablate`, `--dry-run`, `--no-copy-inputs`, `--decider-cmd CMD` or
 `--decider-dir DIR`, `--output-format text|json`, `--dotenv FILE`, `--run-dir DIR`.
 
+- **`--search-corpus POOL`** (with `--search-corpus-miss`, `--search-corpus-min-score`; hostloop) serves every rep's
+  searches from a pool copied under the snapshot root and pinned by hash. Not for climbing a change to what or when
+  the skill searches (eval reference, frozen corpus). Refused (exit 2): a pool from one variant the flow compares, or
+  with more than 60% of a scenario's compared-variant records from one (duplicates counted the worst way; a pool of
+  the baseline's rounds judging v1 included), without `--allow-confounded-pool`; a mixed pool on the baseline, or
+  without `--allow-mixed-pool`; a pass served differently from the flow's rows (another pool, or frozen and live
+  either way round) without `--allow-mixed-search`. Rows carry `meta.search_corpus`; error rows `search_pool_mismatch`
+  / `search_refused` are never scored; the cost line adds `search frozen: … refused`; `check` errors on mismatches,
+  and on mixed pools or frozen beside live (only a warning when every frozen row was written under
+  `--allow-mixed-search`); warns on CONFOUNDED.
+
 - **Resume is by slot.** A re-run of the same variant runs only the (case, rep) slots with no row in
   `results.jsonl`. A slot with only an `errors.jsonl` row re-runs on every pass, so a permanent fault re-runs
   forever; the scope line names those slots. Raising `--reps` adds reps.
@@ -425,7 +436,8 @@ the session's `outputs/` before turn 1, so the prompt asks for the late step alo
 - `run`: `0` every attempted (case, rep) was scored; `1` an attempt failed (an `errors.jsonl` row, or a scored
   row whose trace or copies could not be written), the pass stopped mid-run (rows already written are kept),
   a baseline pass could not freeze a pairwise case's reference, or `summary.json` could not be written; `2`
-  refused before spending. `--dry-run` exits `0` unless a refusal fires. The `done - N ok, M failed` line and
+  refused before spending (a frozen-corpus refusal or a pool copy that cannot be written included). `--dry-run`
+  exits `0` unless a refusal fires; it verifies a `--search-corpus` pool but never copies it. The `done - N ok, M failed` line and
   the envelope's `failed` count (case, rep) slots; a reference freeze reports on its own line
   (`reference freeze: skipped — …` when the case has no good row to freeze from, `failed — …` otherwise).
 - `check`: `0` clean, `1` an error finding, `2` usage.

package/.claude/skills/cowork-harness/references/measurement.md

@@ -1,6 +1,6 @@
 # Measurement
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
 
 ### Measure — before/after, with/without (`--repeat`, `--ablate-skill`)
 

package/.claude/skills/cowork-harness/references/run-record-replay.md

@@ -1,6 +1,6 @@
 # Run, record and lock
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
 
 ## Part II — RUN, RECORD & LOCK
 
@@ -229,7 +229,7 @@ Recognize these before "fixing" a non-bug:
   `markitdown`/`magika` (`ml_extract`), `cv2` (`cv`), `camelot`/`tabula` (`pdf_tables`), or `wand`
   (`magick`) can trip this even though real Cowork **ships** those (per the rootfs manifest captured at
   Desktop `2.9939.2` — `baselines/provisioning/rootfs-provisioning.json`, which is the dated evidence
-  behind that sentence; 9 baselines have shipped since without a re-capture). The message says so ("likely a FALSE
+  behind that sentence; 10 baselines have shipped since without a re-capture). The message says so ("likely a FALSE
   NEGATIVE (real Cowork ships them)"). Fix: rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`, point
   `COWORK_AGENT_IMAGE` at it), or — if the skill's fallback is genuinely equivalent — assert
   `allow_missing_capability: true`. (Two sources: a skill *observed using* an omitted family, live lane;
@@ -369,7 +369,7 @@ Recognize these before "fixing" a non-bug:
   A recording made under the key freezes `answerChannel: "none"` at the cassette's top level and stamps v15, so
   replay runs with the channel still absent and an older build refuses the cassette as too new.
 
-The full 27-code signal table (severity + per-signal opt-out) is in
+The full 30-code signal table (severity + per-signal opt-out) is in
 [`references/assertion-catalog.md`](./assertion-catalog.md); [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) (repo-only) carries
 the fuller narrative.
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,7 +1,7 @@
 # Scenario & session schema, replay class, web_fetch, authoring gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.9.1`
-(baseline `desktop-2.37337.1`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.10.0`
+(baseline `desktop-2.42304.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 
 **Minimal scenario** — `prompt` and `fidelity` are required:
@@ -163,6 +163,9 @@ metrics:                            # OPTIONAL — numbers read from JSON files
                                     # hillclimb regrade carry it forward unless --allow-metric-commands. Its own reasons:
                                     # command_failed/_timeout/_output/_changed/_missing/_not_allowed, nondeterministic,
                                     # no_events. Editing the script mid-flow changes its signature: new id or new flow.
+                                    # The slice keeps the harness's own `_emu` rows (under --search-corpus, a
+                                    # search_corpus_serve line per answered search; a sub-agent's is provisional):
+                                    # skip lines with an `_emu` key to count the agent's frames.
 ```
 
 **Writing a scorer (a command metric's script).** The script measures; it never judges.

package/.claude/skills/cowork-harness/references/semantic-judging.md

@@ -1,6 +1,6 @@
 # `semantic_matches` — the full semantics
 
-Tracks `cowork-harness 4.9.1` (baseline `desktop-2.37337.1`). The one-line summary is in
+Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). The one-line summary is in
 [assertion-catalog-gates-hooks-modifiers.md](assertion-catalog-gates-hooks-modifiers.md); this is the whole contract, split out so the catalog stays within the
 agent's single-read size.