cowork-harness 4.10.0 → 4.11.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, asserting artifacts, egress, or sub-agent dispatch, measuring how long each tool call took (toolDurations / trace), or debugging a failed run or verdict from its result.json or transcript. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. Also for comparing two versions of a skill before merging an edit — did it make answers worse? (`eval`: paired A/B, pinned models) — or improving one round by round (`hillclimb`, the `/claude-api hillclimb` runner). NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold / critique / stats / eval / hillclimb commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 4.10.0
-  tracks-harness: cowork-harness 4.10.0 (baseline desktop-2.42304.0)
+  version: 4.11.0
+  tracks-harness: cowork-harness 4.11.0 (baseline desktop-3.1623.4)
 ---
 
 # cowork-harness
@@ -26,8 +26,8 @@ allowlist). This skill exists mostly to keep you out of those traps — the *Inv
 full landmine catalog in [`references/gotchas.md`](references/gotchas.md) are the highest-value part.
 Read them.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.10.0` (baseline
-> `desktop-2.42304.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.11.0` (baseline
+> `desktop-3.1623.4`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## Preflight — make sure the harness can actually run
@@ -43,7 +43,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, the baseline and the `device_bridge` image (built, or built on the first bridge run) in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.10.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.10.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.10.0"`. **Pin `@^4.10.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.11.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.11.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.11.0"`. **Pin `@^4.11.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)

package/.claude/skills/cowork-harness/references/assertion-catalog-agents-skills-budgets.md

@@ -1,6 +1,6 @@
 # Assertion catalog: sub-agents, skills and connectors, budgets, tasks, delivery
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 

package/.claude/skills/cowork-harness/references/assertion-catalog-gates-hooks-modifiers.md

@@ -1,6 +1,6 @@
 # Assertion catalog: gates, hooks, path denial, verdict modifiers, egress, judged keys
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md).
 
@@ -42,7 +42,7 @@ refused at load: no question reaches the harness to grade.
 | `search_corpus_max_miss_rate: <0..1>` | a `--search-corpus` run's misses / (served + misses) ≤ N; never clean with a refused search. Live-only; full rules in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
 | `search_corpus_nearby_citations: {artifact, max?}` | a `--search-corpus` run's text deliverable cites ≤ `max` URLs served only on nearby rounds. Live-only; rejected at load on `lane: remote`; full rules in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
 | `max_peak_rss_bytes: <N>` | peak sampled RSS of the agent sandbox ≤ N bytes (`RunResult.resources.peakRssBytes`) — live-only: replay never spawns a sandbox to sample, so evidence-unavailable on replay/protocol (never a vacuous pass); also evidence-unavailable when sampling captured no RSS value |
-| `semantic_matches: {rubric: [...], min_pass?, judge_model?, include_subagent_text?, include_fork_results?, evidence_files?: [..]}` | a pinned LLM judge grades each fixed `rubric` claim against the run's answer: the final result text, the transcript and the files the agent authored. The transcript is top-level `assistant_text` only — it excludes every `tool_use`/`tool_result` and sub-agent text unless opted in. Full semantics (evidence scope, fork results, refusals, judge provenance, cost): [semantic-judging.md](semantic-judging.md). On `lane: remote`: judged on the transcript only; `evidence_files` is rejected at load. |
+| `semantic_matches: {rubric: [...], min_pass?, judge_model?, include_subagent_text?, include_fork_results?, evidence_files?: [..]}` | a pinned LLM judge grades each fixed `rubric` claim against the run's answer: the final result text, the transcript and the files the agent authored. The transcript is top-level `assistant_text` only — it excludes every `tool_use`/`tool_result` and sub-agent text unless opted in. Full semantics (evidence scope, fork results, refusals, judge provenance, cost): [semantic-judging.md](semantic-judging.md). On `lane: remote`: judged on the transcript only; `evidence_files` is rejected at load. A `min_pass` above the number of rubric claims is refused at load. |
 | `semantic_pairwise: {refs?: [..], rubric?: [..], pass_if?, order?, judge_model?, include_subagent_text?, include_fork_results?, evidence_files?: [..]}` | a pinned LLM judge compares the document `semantic_matches` builds (it excludes every `tool_use`/`tool_result`) with each frozen reference in `refs` (written once by `ref freeze`; under `hillclimb run` the flow's own, later variants `gate: false`): win, tie, loss or `both_bad`; `pass_if`: `win`, `not_worse` (default) or `any`; `order: both` judges both orders (position bias = tie). A missing or mismatched reference refuses the run before it spends. LIVE-ONLY, skipped on replay; details in the scenario reference (linked above) |
 | `artifact_json: {artifact, path, …}` | a JSON artifact's value at a dotted `path` (or the whole document, with no `path`) satisfies `equals`/`gt`/`in`/`exists`/`absent`/`is_null` or `schema` (inline or `{file}`); a glob `artifact` needs `match: each`/`any` and fails on zero matches; `authored: true` applies to every match. Rejected at load on `lane: remote`; details in the scenario reference (linked above) |
 | `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). **Requires ≥1 link** (zero links fails — use `computer_links_resolve_if_present` for the presence-free variant). **Only `true` is valid** (`false` is rejected by the schema) **Sees top-level `assistant_text` only — it excludes every `tool_use`/`tool_result`**, so a `computer://` link that appeared only inside a tool call or its result is invisible to it. Rejected at load on `lane: remote`. |

package/.claude/skills/cowork-harness/references/assertion-catalog-outcome-files-tools.md

@@ -1,6 +1,6 @@
 # Assertion catalog: outcome, transcript, files and artifacts, tools, tool results
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
 keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 [`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
 
@@ -23,10 +23,10 @@ keys is an AND; glob vs regex name matching) and the verdict-signal table are in
 | `no_lost_write_back: true` | fails if the run authored an interactive HTML artifact (or a `.py`/`.js` generator of one) whose **relative** Submit/POST write-back is lost under Cowork (served from Cowork's own origin → resolves non-ok, a "Saved!" is silently false). Runs the shipped **static Tier A** analyzer over the files the run authored (diffed vs the pre-run manifest). A lost write-back on an **added** agent-authored source (`outputs/`, scratchpad) **fails**; a **pre-existing** file the skill only modified on a read-write mount is **advisory**; `-suspect` findings surface but pass. **Only `true` is valid**. **Live/verify-run only** — skipped-loud on replay; runs on every live sandbox tier including **microvm** (its outputs are snapshotted from the VM into the run dir); could-not-verify (fail-closed) on a `--resume` scratchpad or an unanalyzable candidate. Rejected at load on `lane: remote`. |
 | `tool_called: <glob>` | a tool the agent ran matched this **glob** — `*` = any run, `?` = one char, exact when literal, anchored + case-sensitive. Exact name (`Write`) matches only that tool; `mcp__workspace__*` matches any workspace tool. GLOB, not regex (`.` is literal) — an empty glob, or one containing a regex/brace-expansion metacharacter (`.*`, `.+`, `\|`, `()`, `[]`, `+`, `^`, `$`, `{}`, `\d`/`\w`/`\s`/`\b`), is **rejected at load** (a hard schema error, not a runtime warning) — it would match no real tool name and pass a `_not_`/`_absent` assert vacuously. Applies whether the glob comes from an authored scenario or a recorded cassette's frozen assert. `cowork-harness lint` reports it too (ERROR `scenario-invalid` — the wrapper runs the same loader); the bundled `scenario.py lint` run directly does NOT |
 | `tool_not_called: <glob>` | NO tool the agent ran matched this glob (`mcp__*` = "no MCP tool ran"). Same glob semantics as `tool_called`, including the empty/regex-ish rejection |
-| `tool_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?, count?}` | **object form**: what a call CARRIED, where it RAN and what its paired result SAID. `tool`: glob or any-of list; `input: {<field>: <regex>}`; `input_any: <regex>`; `result: {matches?, not_matches?, is_error?}` over the paired `tool_result`; `scope`: `main` (default)/`subagent`/`any`; `subagent_type` (with `scope: subagent`): regex over the IMMEDIATE parent dispatch's type or description; `count: {min?, max?}` (default `min: 1`). Fails closed: an unpaired call never satisfies `result`; truncated evidence is evidence unavailable, never a pass. Stamps cassette v13; details in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
-| `tool_not_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?}` | **object form** — no call in scope satisfies every predicate (no `count`). Same fields and fail-closed rules as above: a candidate whose result is unpaired or truncated, or whose input was truncated, is *evidence unavailable*, never a pass. Refused at load when EVERY listed tool is one the tier does not serve. ⚠️ **Redaction hazard — this is the dangerous direction.** A committed cassette is redacted, which rewrites the recorded inputs: an `input` regex naming a literal the policy rewrites (a home path, an email) cannot see its target on replay. On replay, any candidate call whose field (or paired result, for `result.matches`) carries a redaction token is *evidence unavailable*, whatever the regex — the evaluator cannot know what the token replaced — so a negative check never passes over rewritten bytes. `record` warns — naming the redacted call and the ways out (narrow `scope`/`tool`, the string form, or live-only) — and refuses the write (the exact guard), and `lint` flags a regex naming a redactable literal (`tool-input-regex-redactable`) — a heuristic that reads `.cowork-redact.json` from the current and scenario directories only, not the cassette's. Match a part of the input redaction leaves alone (the verb and flags, a workspace-relative path), or keep the check on a live gate. Note also that the default `scope: main` does not see a sub-agent's call — use `scope: any` for "nothing anywhere ran X". |
+| `tool_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?, count?}` | **object form**: what a call CARRIED, where it RAN and what its paired result SAID. `tool`: glob or any-of list; `input: {<field>: <regex>}`; `input_any: <regex>`; `result: {matches?, not_matches?, is_error?}` over the paired `tool_result`; `scope`: `main` (default)/`subagent`/`any`; `subagent_type` (with `scope: subagent`): regex over the IMMEDIATE parent dispatch's type or description; `count: {min?, max?}` (default `min: 1`; `{max: 0}` alone, or `{min: 0, max: 0}`, means never called and is graded as `tool_not_called`, tier and redaction rules included; `min` above `max` is refused at load). Fails closed: an unpaired call never satisfies `result`; truncated evidence is evidence unavailable, never a pass. Stamps cassette v13; details in [docs/scenario.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) |
+| `tool_not_called: {tool: <glob> \| [..], input?, input_any?, result?, scope?, subagent_type?}` | **object form** — no call in scope satisfies every predicate (no `count`; `tool_called` with `count: {max: 0}` is the same check). Same fields and fail-closed rules as above: a candidate whose result is unpaired or truncated, or whose input was truncated, is *evidence unavailable*, never a pass. Refused at load when EVERY listed tool is one the tier does not serve. ⚠️ **Redaction hazard — this is the dangerous direction.** A committed cassette is redacted, which rewrites the recorded inputs: an `input` regex naming a literal the policy rewrites (a home path, an email) cannot see its target on replay. On replay, any candidate call whose field (or paired result, for `result.matches`) carries a redaction token is *evidence unavailable*, whatever the regex — the evaluator cannot know what the token replaced — so a negative check never passes over rewritten bytes. `record` warns — naming the redacted call and the ways out (narrow `scope`/`tool`, the string form, or live-only) — and refuses the write (the exact guard), and `lint` flags a regex naming a redactable literal (`tool-input-regex-redactable`) — a heuristic that reads `.cowork-redact.json` from the current and scenario directories only, not the cassette's. Match a part of the input redaction leaves alone (the verb and flags, a workspace-relative path), or keep the check on a live gate. Note also that the default `scope: main` does not see a sub-agent's call — use `scope: any` for "nothing anywhere ran X". |
 | **legacy tool names** | The agent binary canonicalizes legacy spellings (`Task`→`Agent`, `KillShell`/`KillBash`→`TaskStop`, …) while the spawn tool list still declares the LEGACY one — so the init inventory shows `Task` and every call is emitted as `Agent`. `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` match EITHER spelling, globs included. Before this, `tool_called: "Task"` could never pass and `tool_not_called: "Task"` always did. |
-| **tier vacuity — now REFUSED** | `tool_not_called` **and `subagent_tool_absent`** naming a tool the tier does not serve is rejected at scenario load (`Bash`/`WebFetch`/`NotebookEdit` at `hostloop`; `mcp__workspace__bash` at `container`/`microvm`; on those three tiers, `mcp__remote-devices__device_bash`/`get_device_info`/`device_request_delete_permission` without `device_bridge: true`, and every other `mcp__remote-devices__*` tool the harness never serves, bridge or not; and `mcp__visualize__read_me`/`show_widget`, production's skill-argument form tools, which the harness never serves unless the session's own MCP config declares a `visualize` server), because it could never be violated. `scenario.py lint` WARNs on the same set before any run. The table is CLOSED to tools the harness itself removes or registers — `--tools` gates only the built-in set while every tier separately passes `--mcp-config`, so a session-MCP tool name is offered without appearing in any tool list and is never refused. Globs are never refused. `subagent_tool_absent` is covered because it reads the tools sub-agents actually USED, not a per-dispatch declared list. Under-approximates on purpose: `REPL` at hostloop is vacuous too and is not caught, which is the right side to err on for a hard refusal. |
+| **tier vacuity — now REFUSED** | `tool_not_called` **and `subagent_tool_absent`** (and `tool_called` with `count: {max: 0}`) naming a tool the tier does not serve is rejected at scenario load (`Bash`/`WebFetch`/`NotebookEdit` at `hostloop`; `mcp__workspace__bash` at `container`/`microvm`; on those three tiers, `mcp__remote-devices__device_bash`/`get_device_info`/`device_request_delete_permission` without `device_bridge: true`, and every other `mcp__remote-devices__*` tool the harness never serves, bridge or not; and `mcp__visualize__read_me`/`show_widget`, production's skill-argument form tools, which the harness never serves unless the session's own MCP config declares a `visualize` server), because it could never be violated. `scenario.py lint` WARNs on the same set before any run. The table is CLOSED to tools the harness itself removes or registers — `--tools` gates only the built-in set while every tier separately passes `--mcp-config`, so a session-MCP tool name is offered without appearing in any tool list and is never refused. Globs are never refused. `subagent_tool_absent` is covered because it reads the tools sub-agents actually USED, not a per-dispatch declared list. Under-approximates on purpose: `REPL` at hostloop is vacuous too and is not caught, which is the right side to err on for a hard refusal. |
 | `reference_read: <regex>` | a skill `references/`/`scripts/` file whose path matches this **regex** was ACCESSED — main agent or sub-agents, via `Read`, `Grep`/`Glob` (`path` input), or a `Bash`/`mcp__workspace__bash` command naming the path. Regex is **unanchored + case-insensitive** (the shared helper every regex key uses). **Under-approximates by design:** the path must be rooted in the mounted plugin, so a `cd` into the skill dir then a bare `cat references/x.md`, a heredoc body, and a `$VAR`-built path are invisible. Fails **evidence unavailable** when the run recorded no observable tool stream. Replay-capable (cassettes freeze whole tool inputs) |
 | `no_observed_reference_access: <regex>` | no OBSERVED access matched the regex — the progressive-disclosure check: a reference the skill's routing never reaches. Named `observed` because detection under-approximates (see above), so it is **not proof the file went unread** — an agent that `cd`s and `cat`s it passes. Fails **evidence unavailable** rather than passing vacuously when no observable tool stream was recorded |
 | `tool_result_contains: <str>` | a tool result includes the literal string (content / replay-checkable — substring match) |

package/.claude/skills/cowork-harness/references/assertion-catalog.md

@@ -1,6 +1,6 @@
 # Assertion catalog
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Every `assert:` key with its semantics, and the
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Every `assert:` key with its semantics, and the
 verdict-signal table. Which keys survive `replay` is in [`scenario-schema.md`](./scenario-schema.md#replay-class);
 the scenario and session YAML fields are there too.
 

package/.claude/skills/cowork-harness/references/assertions-guide.md

@@ -1,6 +1,6 @@
 # Assertions guide
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
 
 ### Assertions: two orthogonal axes
 

package/.claude/skills/cowork-harness/references/authoring.md

@@ -1,6 +1,6 @@
 # Authoring a scenario
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
 
 ## Part I — AUTHOR a scenario
 
@@ -308,15 +308,17 @@ cowork-harness lint scenarios/*.yaml
 | `positional-choose-order` | INFO | an answer rule with a positional `choose` (first / index), which option re-ordering can move |
 | `present-files-key-off-tier` | ERROR | `present_files_called` on `protocol` / `microvm` (served only at `container` / `hostloop`) |
 | `prompt-slash-not-leading` | WARN | a `prompt:` that names `/<skill>` without starting with it, so it is never expanded |
+| `count-bound-unsatisfiable` | ERROR | an assertion whose own bounds no count meets: `tool_called` `count.min` above `count.max`, or `semantic_matches` `min_pass` above the number of rubric claims (the loader refuses both) |
 | `reference-access-contradiction` | ERROR | one reference under both `reference_read` and `no_observed_reference_access` |
 | `regex-double-quoted` | WARN | a double-quoted regex with an unescaped backslash (YAML strips it) |
 | `replay-noop` | WARN | every assertion is live-only or a verdict modifier, so a replay gate verifies nothing |
 | `slash-prompt-forked-result-anchor` | WARN | a `prompt:` starting with `/<skill>` plus a `tool_result_*` anchored on `forked execution` — a slash-invoked skill makes no `Skill` call, so that tool result never exists; assert `skill_triggered` instead |
 | `slash-skill-name-differs-from-plugin` | WARN | a `prompt:` starting with a bare `/<skill>` that names a skill of a plugin the scenario's session stages, where the plugin's name differs — the agent expands it, but real Cowork's app has refused that typed form; pick it from the slash menu or name the skill like its plugin (`/<plugin>:<skill>` was not measured with a single copy installed). Names follow the agent: `.claude-plugin/plugin.json` `name`/`skills`, else the dir name; a skill's sanitized directory name. Reads the `session:` file and its `local_plugins`/`remote_plugins`; silent for an inline `session:`, for marketplace-delivered plugins, and when the files are not on this machine |
 | `tool-called-always-passes` | INFO | `tool_called` with `count: {min: 0}` and no `max` — it asserts nothing |
-| `tool-input-regex-redactable` | WARN | a `tool_not_called` input literal the redaction policy rewrites in the committed cassette (or a policy pattern it cannot check offline) |
+| `tool-called-max-zero` | INFO | `tool_called` with `count: {max: 0}` — a negative check, graded as `tool_not_called` with the same fields (its tier and redaction rules apply) |
+| `tool-input-regex-redactable` | WARN | a negative tool check's input literal (`tool_not_called`, or `tool_called` with `count: {max: 0}`) the redaction policy rewrites in the committed cassette (or a policy pattern it cannot check offline) |
 | `tool-input-shell-tier` | INFO | the object form with `tool: Bash` and a `command` on `hostloop` / `cowork`, where shell runs as `mcp__workspace__bash` — list both |
-| `tool-not-called-tier-vacuous` | WARN | `tool_not_called` / `subagent_tool_absent` naming a tool the tier never serves |
+| `tool-not-called-tier-vacuous` | WARN | `tool_not_called`, `subagent_tool_absent` or `tool_called` with `count: {max: 0}` naming a tool the tier never serves |
 | `transcript-command-shaped` | WARN | a `transcript_*` value shaped like a shell command — those keys read prose only, never a tool call |
 | `unknown-assert-key` | WARN | an assertion key not in the catalog (the loader rejects it) |
 | `unknown-top-key` | WARN | a scenario key not in the schema |

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`).
+Self-contained reference. Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
 copy-pasted recipe that omits the input takes a major bump with no say in it. `^4` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "4.10.0"`) instead when you want byte-reproducible CI.
+(e.g. `version: "4.11.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -36,7 +36,7 @@ jobs:
       - uses: actions/checkout@v4
       - name: Stage the agent binary (official channel, sha256-verified against the pinned baseline)
         run: |
-          V=2.1.299   # match your scenario's pinned baseline's agentVersion
+          V=2.1.308   # match your scenario's pinned baseline's agentVersion
           # The release channel is NOT always the stable one. Desktop also stages release CANDIDATES,
           # served from .../claude-code-releases/rc/<commit>/. For some versions the stable path 404s
           # (2.1.255); for others it returns 200 and serves a DIFFERENT BUILD UNDER THE SAME VERSION
@@ -82,7 +82,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^4.10.0"
+- run: npm i -g "cowork-harness@^4.11.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -361,7 +361,7 @@ A typical skill repo runs four stages, fastest/cheapest first:
    failure** — `--quiet` suppresses the readiness preview but never the `✗ broken:` lines, which name the
    offending file *and* the rejected key, one line per file, and the step still exits 1. It exits 0,
    though, on an input the real record would refuse (a `session:` file that cannot be read — 4.1.1 and
-   later — a missing path, an unknown baseline name, a tier-vacuous `tool_not_called`): that prints a `⚠ input error:` line and lands in `inputErrors[]`. To
+   later — a missing path, an unknown baseline name, a tier-vacuous negative tool check): that prints a `⚠ input error:` line and lands in `inputErrors[]`. To
    gate on those too, use the JSON form (4.1.0 and later):
 
    ```bash
@@ -411,7 +411,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^4.10.0"
+      - run: npm i -g "cowork-harness@^4.11.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -440,7 +440,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^4.10.0"
+        run: npm i -g "cowork-harness@^4.11.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). This is **not** a trim of the full
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reading several critiques, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/debugging.md

@@ -1,6 +1,6 @@
 # Debugging a run
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
 
 ## Part III — Debug
 
@@ -26,7 +26,7 @@ Docker, no re-record.
 
 | Situation | Symptom | Reach for (in order) |
 |---|---|---|
-| **The skill misbehaved** | wrong output, an unexpected gate, a denied tool, an opaque crash | `inspect` — what did it produce? · `trace <run-dir> --view <view>` — what did it actually do (tools, gates, sub-agent tree)? · `verify-run` — re-assert cheaply when only an assertion is wrong · `diff <old-run> <new-run>` — what changed since it worked · `chat` — reproduce it by hand |
+| **The skill misbehaved** | wrong output, an unexpected gate, a denied tool, an opaque crash | `inspect` — what did it produce? · `trace <run-dir> --view <view>` — what did it actually do (tools; gates are the questions view, as there is no gates view; the sub-agent tree is the dispatches view)? · `verify-run` — re-assert cheaply when only an assertion is wrong · `diff <old-run> <new-run>` — what changed since it worked · `chat` — reproduce it by hand |
 | **A green you don't trust** | an assert that may have tested nothing, a stale cassette, an auto-answered or decided gate | `replay --explain` — the evidence trail behind each *passing* assert · `replay --mutate` — perturbs a CAPPED SAMPLE of recorded JSON values (10/file, 50 total) and reports which perturbations NOTHING caught; the report names the sample size, so read it as a sample not a total (reporting only; never moves the verdict/exit code) · `lint` — assertions on the wrong CI lane / mixed-class keys · `verify-cassettes` — privacy + staleness over committed cassettes · the Gotchas landmine catalog — how a check passes vacuously · `run --repeat N` / `skill --repeat N` — did it pass, or pass once? · `stats` — flaky or expensive over time |
 
 A failed run also records `errorSource` (where the failure originated) and `stderrLogPath` (the captured
@@ -98,7 +98,8 @@ already-written evidence*, not re-run-and-hope. Use them to diagnose a failure (
 decide which assertions from *Assertions: two orthogonal axes* in `assertions-guide.md` are worth adding):
 
 - **`cowork-harness trace <run-dir> --view <view>`** — focuses one of the run's rollups (the per-tool
-  call-count/timing table, the sub-agent dispatch tree, the gate lifecycle, the tool/error rollups, …);
+  call-count/timing table, the sub-agent dispatch tree, the gate lifecycle as `--view questions` (not `gates`), the
+  tool/error rollups, …);
   bare `trace` digests the whole run. The view set is actively being extended — run `trace --help` for
   the current list rather than relying on a fixed enumeration here.
 - **`lane: local|remote`** (scenario key, default `local`) — which Cowork lane's DELIVERY CONTRACT the run

package/.claude/skills/cowork-harness/references/eval.md

@@ -1,6 +1,6 @@
 # `eval` — paired before/after comparison of a skill edit (EXPERIMENTAL)
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). The full guide is
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). The full guide is
 [docs/eval.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/eval.md); this is the part you
 need while running it.
 
@@ -77,7 +77,7 @@ names the arm, scenario, dominant error and a matching hint (`Every rep of arm <
 - identical arms (unless `--allow-identical-arms`);
 - an arm that contains the eval's own scenario or session files, or a command metric's script or deps (by location,
   copy or symlink), an `evals.json`, or a symlink resolving outside it;
-- a scenario input a run would refuse (a missing path, a `tool_not_called` the tier can never violate);
+- a scenario input a run would refuse (a missing path, a negative tool check the tier can never violate);
 - `--fail-on confirmed` when no family (pass rows or metric rows, in any section) could reach `confirmed` at this
   `--reps`; when only some can, the start-up line names the families the gate can fire on;
 - no usable agent credential for a scenario's tier — the same check as `doctor --tier <tier>`'s `token` row,

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`).
+Self-contained reference. Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`).
 
 > **This page vs. the repo docs.** This is the **offline snapshot** that ships inside the installed
 > plugin — it is self-contained on purpose. The repo carries four other fidelity views, each answering a
@@ -13,7 +13,7 @@ Self-contained reference. Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42
 
 | Tier | What runs | Use it for |
 |---|---|---|
-| `protocol` | L0 — agent on the host, no sandbox, no egress enforcement | Fastest control-loop / answer-shape checks. **Rejected** if the scenario asserts egress / `expect_denied` (would false-pass). |
+| `protocol` | L0 — agent on the host, no sandbox, no egress enforcement | Fastest control-loop / answer-shape checks. **Rejected** if the scenario asserts egress / `expect_denied` (would false-pass). No background shells or tasks, as on every tier and in Cowork (since 4.11.0), except on a baseline that pins no switch (`desktop-1.11847.5`), where no tier sets it. |
 | `container` | L1 — agent in a Docker container with a per-run default-deny egress proxy | The everyday tier: real sandbox, real egress allowlist. |
 | `microvm` | L2 — agent in an Apple-VZ Lima microVM with a guest firewall | VM-grade escape **isolation** of untrusted code. macOS arm64 only; needs `cowork-harness vm init`. Network transport **equals `container`** (same allowlist proxy) — *not* better network fidelity. |
 | `hostloop` | Host-loop split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production); shell/web routed host-side via the workspace SDK-MCP server (`mcp__workspace__bash`) into a Docker VM sidecar | Reproduce Cowork's **production** split-execution model. |

package/.claude/skills/cowork-harness/references/gotchas.md

@@ -1,6 +1,6 @@
 # Gotchas
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). The full "✓ passed ≠ correct" landmine catalog.
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). The full "✓ passed ≠ correct" landmine catalog.
 
 ## Gotchas — the "✓ passed ≠ correct" landmines
 

package/.claude/skills/cowork-harness/references/hillclimb-recipe.md

@@ -1,6 +1,6 @@
 # Recipe 7 — Climb a skill with `/claude-api hillclimb` and the harness as its runner
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). It needs a `cowork-harness` whose
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). It needs a `cowork-harness` whose
 `hillclimb --help` lists `--skill` (help goes to stderr). This page is the loop's procedure, step by step, in the order of the
 `/claude-api hillclimb` guide. Every mechanic (flags, refusals, the gate, `regrade`, `freeze-ref`, exit codes,
 row keys) is in [`hillclimb.md`](hillclimb.md); the setup and the full list of differences from the guide's own

package/.claude/skills/cowork-harness/references/hillclimb.md

@@ -1,6 +1,6 @@
 # `hillclimb` — the runner for a `/claude-api hillclimb` loop
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). It needs a `cowork-harness` whose
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). It needs a `cowork-harness` whose
 `hillclimb --help` lists `--skill` (help goes to stderr). The command reference is
 [docs/cli.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/cli.md); this is the part a loop needs
 while it runs. It covers `run`, `check`, `state-template`, `freeze-ref` and `regrade`.

package/.claude/skills/cowork-harness/references/measurement.md

@@ -1,6 +1,6 @@
 # Measurement
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
 
 ### Measure — before/after, with/without (`--repeat`, `--ablate-skill`)
 

package/.claude/skills/cowork-harness/references/run-record-replay.md

@@ -1,6 +1,6 @@
 # Run, record and lock
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
 
 ## Part II — RUN, RECORD & LOCK
 
@@ -53,12 +53,12 @@ cassette-portability pre-flight below**, so it cannot green something a paid run
 guarantee is the SINGLE-FILE form only** — it takes the real `--out` and the real flags, so its verdict is the
 one a paid run would give. Two inputs it reports rather than refuses, at exit 0 under `inputErrors[]` with a
 `⚠ input error:` line: a `session:` file that cannot be read (missing, a directory, not valid YAML — 4.1.1
-and later) and a `tool_not_called` the tier can never violate — `run` and the real `record` refuse both. On a **directory** the path-dependent verdicts (host-inventory, cassette
+and later) and a negative tool check the tier can never violate — `run` and the real `record` refuse both. On a **directory** the path-dependent verdicts (host-inventory, cassette
 portability) are reported as `⚠ would-refuse (advisory)` / `⚠ would-warn (advisory)` notes — the label follows
 the verdict kind, and portability can only ever warn — that do NOT affect the exit code — a dir target
 takes no `--out`, so the destination is a guess — and only the path-independent ones (prompt policy, assert
 contradiction, duplicate cassette target) gate the batch. An input the real record would refuse — a
-`session:` file that cannot be read (4.1.1 and later), a missing path, an unknown baseline name, a `tool_not_called` the tier can never violate — is listed under
+`session:` file that cannot be read (4.1.1 and later), a missing path, an unknown baseline name, a negative tool check the tier can never violate — is listed under
 `inputErrors[]` with a `⚠ input error:` line, also at exit 0. So a directory dry-run CAN exit 0 on a scenario the
 real `record` would refuse; re-run that one file with its real flags for a binding answer, or gate on
 `.ok and (.inputErrors == [])` in the JSON payload (4.1.0 and later). A directory also
@@ -229,7 +229,7 @@ Recognize these before "fixing" a non-bug:
   `markitdown`/`magika` (`ml_extract`), `cv2` (`cv`), `camelot`/`tabula` (`pdf_tables`), or `wand`
   (`magick`) can trip this even though real Cowork **ships** those (per the rootfs manifest captured at
   Desktop `2.9939.2` — `baselines/provisioning/rootfs-provisioning.json`, which is the dated evidence
-  behind that sentence; 10 baselines have shipped since without a re-capture). The message says so ("likely a FALSE
+  behind that sentence; 11 baselines have shipped since without a re-capture). The message says so ("likely a FALSE
   NEGATIVE (real Cowork ships them)"). Fix: rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`, point
   `COWORK_AGENT_IMAGE` at it), or — if the skill's fallback is genuinely equivalent — assert
   `allow_missing_capability: true`. (Two sources: a skill *observed using* an omitted family, live lane;

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,7 +1,7 @@
 # Scenario & session schema, replay class, web_fetch, authoring gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.10.0`
-(baseline `desktop-2.42304.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.11.0`
+(baseline `desktop-3.1623.4`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 
 **Minimal scenario** — `prompt` and `fidelity` are required:

package/.claude/skills/cowork-harness/references/semantic-judging.md

@@ -1,6 +1,6 @@
 # `semantic_matches` — the full semantics
 
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`). The one-line summary is in
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`). The one-line summary is in
 [assertion-catalog-gates-hooks-modifiers.md](assertion-catalog-gates-hooks-modifiers.md); this is the whole contract, split out so the catalog stays within the
 agent's single-read size.
 
@@ -12,4 +12,4 @@ so. A graded assert reports `semanticEvidence: {reason: "graded", paths: []}` an
 only (lane: remote)". `evidence_files` is rejected at load on that lane (`lane_remote_evidence_files` if it reaches
 the evaluator another way), and a `semantic_pairwise` reference is stored per lane.
 
-a pinned LLM judge grades each fixed `rubric` claim against the run's answer — the **union of the agent's final result text (`RunResult.finalMessage`), the transcript, and the final on-disk content of any files the agent authored during the run** — so a claim about content the skill led the agent to *write to a file* grades as reliably as one about inlined prose. **"The transcript" is narrower than it reads: top-level `assistant_text` ONLY.** It excludes every `tool_use`/`tool_result` and **all sub-agent-originated text** (including fork-scoped `Skill`/`Agent(fork)` dispatches — the harness attributes their *tool* calls to the main agent, but not their text). **A `context: fork` skill's own answer is not graded by default**, even with `include_subagent_text: true`: it is not a dispatch (so it has no `subagents[]` entry to fold in), and it reaches the main agent as the `Skill` tool result, which the judged document excludes. **Opt in with `include_fork_results: true`** to grade a FOREGROUND fork's answer: every top-level `Skill` call is joined to its result by `toolUseId` and appended as a `Fork skill result: <skill>` section (`Skill result: <skill>` when the result lacks the agent's `completed (forked execution)` marker — an inline skill's result is only its launch line). A top-level `Skill` result is captured up to 32,768 characters (every other tool result keeps 10,240); one cut at that cap fails evidence-unavailable (`semanticEvidence.reason: "fork_result_truncated"`), a call with no paired result `fork_result_unpaired`, a fork launched in the background (its result is only the `launched (forked execution, running in the background)` line) `fork_result_background`, a result.json with no tool-call record `fork_calls_unrecorded` — `paths` then lists the `Skill` calls' skill names; never a grade over part of an answer. A fork invoked by a leading `/<skill>` prompt makes no `Skill` call, so there is nothing to join: no section is added, the fork's answer is already top-level transcript text, and a pass's evidence reads `graded Skill results: (none — no Skill call)`. Without the key the judge sees the fork's answer only if the main agent restates it; `tool_result_matches` checks it structurally (see `subagent_output_contains`). ⚠️ **Consequence: a rubric claim about whether a tool was called cannot grade true** — the evidence is not in the judged document (one exception: with `include_fork_results: true`, a `## Skill result: X` / `## Fork skill result: X` section lets the judge confirm that skill X ran, from its result). Such a claim looks reasonable and silently caps your pass rate; assert tool use with `tool_called` / `present_files_called` / `subagent_dispatched` instead, and `hook_event_blocked` / `hook_decision` for a plugin's hook (`hook_blocked` reads only the harness's own). Sub-agent text is captured in `RunResult.subagents[].reasoning` and reaches the judge only via opt-in `include_subagent_text: true` (`kind:"text"` turns only — sub-agent *thinking* arrives empty with `redacted:true`, so it would pad the document with blanks) (authored-file evidence is captured on every live sandbox tier including **microvm** — its session tree is snapshotted from the VM into the run dir). When the authored-file evidence backing the judged document is **incomplete** — a file dropped at the capture-size cap, unreadable at read-back, or (on `--resume`) the scratchpad walk skipped — the assert fails evidence-unavailable rather than trusting a judge grade over a partial document, and the judge is not called for it (no `semanticClaims`, no judge spend or `judgedDoc` — `semanticEvidence.reason` says why); this is separate from the malformed-grade `judgeInvalid` path below. The assert passes iff ≥ `min_pass` claims pass (default: all — avoid for a gating scenario). Results align by claim index and are recorded per-claim in `RunResult.assertions[].semanticClaims` (`[{index, claim, pass, rationale?}]`, so a consumer can diff the per-claim profile across runs; `rationale` is the judge's one-sentence reason, printed under each failed claim in the failure footer: untrusted model text that can quote the judged document, whose content never affects `pass`, absent when the judge gave none (a reply whose shape is broken, such as unparseable JSON, a malformed `{"results": …}` group beside a valid grade, or a partial restatement that contradicts it, is retried once and then marked `judgeInvalid`), and comparable only between runs that share `judgePromptHash`); a rep whose grade can't be parsed (after one retry) is marked `RunResult.assertions[].judgeInvalid` and **never silently dropped** — it is excluded from the pass denominator, and the guard against a misleading score from that exclusion is the gate's minimum-valid-rep floor (`MIN_VALID` ≥ 4) plus this visibility, not a claim that denominator-shrinking inflation is impossible. Within a rep, a grade that's still unparseable after the retry **fails that assert outright** (evidence-unavailable, not a vacuous pass) — a persistently-flaky judge reds the run rather than silently passing. `judge_model` pins the grader (default when neither it nor `COWORK_HARNESS_JUDGE_MODEL` is set: `claude-opus-4-8`; a dated id keeps a before/after comparison reproducible — or let `eval` compare two skill versions per claim, with the judge pinned). Each graded assert records the judge's provenance: `RunResult.assertions[].judgeModel` (the resolved model), `judgeCostUsd` (judge spend over both attempts, reported beside `cost.usd` and never inside it; absent when unpriced) and `judgePromptHash` (the grading-prompt template — compare only runs that share it). Live-only: the judge is a live model call, so evidence-unavailable / skipped-loud on replay (never a vacuous pass) **`evidence_files: [globs]` scopes which authored files are graded** — reach for it the moment a run authors more than a couple of files. The capture budget (64 KiB total by default) is spent prefix-major then alphabetically, so a pipeline that stages intermediates (`outputs/_work/*.json`) exhausts it before reaching its own deliverable and the verdict is refused evidence-unavailable over files no rubric mentions. Scoping also makes the capture spend the budget on the named files FIRST and exempts them from the per-file cap. Paths are `<root>/<rel>` (`outputs/report.md`, never a bare `report.md`; session-root writes are `scratchpad/<rel>`); globs are `*`/`?`/`**`, not regex. A glob matching nothing FAILS and the message lists every authored path — read it rather than guessing. Still too big? Raise `$COWORK_HARNESS_AUTHORED_TOTAL_BYTES`. The typed reason is on `RunResult.assertions[].semanticEvidence` — check `.reason` instead of parsing the message
+a pinned LLM judge grades each fixed `rubric` claim against the run's answer — the **union of the agent's final result text (`RunResult.finalMessage`), the transcript, and the final on-disk content of any files the agent authored during the run** — so a claim about content the skill led the agent to *write to a file* grades as reliably as one about inlined prose. **"The transcript" is narrower than it reads: top-level `assistant_text` ONLY.** It excludes every `tool_use`/`tool_result` and **all sub-agent-originated text** (including fork-scoped `Skill`/`Agent(fork)` dispatches — the harness attributes their *tool* calls to the main agent, but not their text). **A `context: fork` skill's own answer is not graded by default**, even with `include_subagent_text: true`: it is not a dispatch (so it has no `subagents[]` entry to fold in), and it reaches the main agent as the `Skill` tool result, which the judged document excludes. **Opt in with `include_fork_results: true`** to grade a FOREGROUND fork's answer: every top-level `Skill` call is joined to its result by `toolUseId` and appended as a `Fork skill result: <skill>` section (`Skill result: <skill>` when the result lacks the agent's `completed (forked execution)` marker — an inline skill's result is only its launch line). A top-level `Skill` result is captured up to 32,768 characters (every other tool result keeps 10,240); one cut at that cap fails evidence-unavailable (`semanticEvidence.reason: "fork_result_truncated"`), a call with no paired result `fork_result_unpaired`, a fork launched in the background (its result is only the `launched (forked execution, running in the background)` line) `fork_result_background`, a result.json with no tool-call record `fork_calls_unrecorded` — `paths` then lists the `Skill` calls' skill names; never a grade over part of an answer. A fork invoked by a leading `/<skill>` prompt makes no `Skill` call, so there is nothing to join: no section is added, the fork's answer is already top-level transcript text, and a pass's evidence reads `graded Skill results: (none — no Skill call)`. Without the key the judge sees the fork's answer only if the main agent restates it; `tool_result_matches` checks it structurally (see `subagent_output_contains`). ⚠️ **Consequence: a rubric claim about whether a tool was called cannot grade true** — the evidence is not in the judged document (one exception: with `include_fork_results: true`, a `## Skill result: X` / `## Fork skill result: X` section lets the judge confirm that skill X ran, from its result). Such a claim looks reasonable and silently caps your pass rate; assert tool use with `tool_called` / `present_files_called` / `subagent_dispatched` instead, and `hook_event_blocked` / `hook_decision` for a plugin's hook (`hook_blocked` reads only the harness's own). Sub-agent text is captured in `RunResult.subagents[].reasoning` and reaches the judge only via opt-in `include_subagent_text: true` (`kind:"text"` turns only — sub-agent *thinking* arrives empty with `redacted:true`, so it would pad the document with blanks) (authored-file evidence is captured on every live sandbox tier including **microvm** — its session tree is snapshotted from the VM into the run dir). When the authored-file evidence backing the judged document is **incomplete** — a file dropped at the capture-size cap, unreadable at read-back, or (on `--resume`) the scratchpad walk skipped — the assert fails evidence-unavailable rather than trusting a judge grade over a partial document, and the judge is not called for it (no `semanticClaims`, no judge spend or `judgedDoc` — `semanticEvidence.reason` says why); this is separate from the malformed-grade `judgeInvalid` path below. The assert passes iff ≥ `min_pass` claims pass (default: all — avoid for a gating scenario; above the number of rubric claims it can never pass and is refused at load). Results align by claim index and are recorded per-claim in `RunResult.assertions[].semanticClaims` (`[{index, claim, pass, rationale?}]`, so a consumer can diff the per-claim profile across runs; `rationale` is the judge's one-sentence reason, printed under each failed claim in the failure footer: untrusted model text that can quote the judged document, whose content never affects `pass`, absent when the judge gave none (a reply whose shape is broken, such as unparseable JSON, a malformed `{"results": …}` group beside a valid grade, or a partial restatement that contradicts it, is retried once and then marked `judgeInvalid`), and comparable only between runs that share `judgePromptHash`); a rep whose grade can't be parsed (after one retry) is marked `RunResult.assertions[].judgeInvalid` and **never silently dropped** — it is excluded from the pass denominator, and the guard against a misleading score from that exclusion is the gate's minimum-valid-rep floor (`MIN_VALID` ≥ 4) plus this visibility, not a claim that denominator-shrinking inflation is impossible. Within a rep, a grade that's still unparseable after the retry **fails that assert outright** (evidence-unavailable, not a vacuous pass) — a persistently-flaky judge reds the run rather than silently passing. `judge_model` pins the grader (default when neither it nor `COWORK_HARNESS_JUDGE_MODEL` is set: `claude-opus-4-8`; a dated id keeps a before/after comparison reproducible — or let `eval` compare two skill versions per claim, with the judge pinned). Each graded assert records the judge's provenance: `RunResult.assertions[].judgeModel` (the resolved model), `judgeCostUsd` (judge spend over both attempts, reported beside `cost.usd` and never inside it; absent when unpriced) and `judgePromptHash` (the grading-prompt template — compare only runs that share it). Live-only: the judge is a live model call, so evidence-unavailable / skipped-loud on replay (never a vacuous pass) **`evidence_files: [globs]` scopes which authored files are graded** — reach for it the moment a run authors more than a couple of files. The capture budget (64 KiB total by default) is spent prefix-major then alphabetically, so a pipeline that stages intermediates (`outputs/_work/*.json`) exhausts it before reaching its own deliverable and the verdict is refused evidence-unavailable over files no rubric mentions. Scoping also makes the capture spend the budget on the named files FIRST and exempts them from the per-file cap. Paths are `<root>/<rel>` (`outputs/report.md`, never a bare `report.md`; session-root writes are `scratchpad/<rel>`); globs are `*`/`?`/`**`, not regex. A glob matching nothing FAILS and the message lists every authored path — read it rather than guessing. Still too big? Raise `$COWORK_HARNESS_AUTHORED_TOTAL_BYTES`. The typed reason is on `RunResult.assertions[].semanticEvidence` — check `.reason` instead of parsing the message

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 4.10.0` (baseline `desktop-2.42304.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 4.11.0` (baseline `desktop-3.1623.4`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.