cowork-harness 0.25.0 → 0.27.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 0.25.0
-  tracks-harness: cowork-harness 0.25.0 (baseline desktop-1.18286.0)
+  version: 0.27.0
+  tracks-harness: cowork-harness 0.27.0 (baseline desktop-1.18286.0)
 ---
 
 # cowork-harness
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.25.0` (baseline
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.27.0` (baseline
 > `desktop-1.18286.0`). If your checkout is newer, prefer the live `--help`, `SPEC.md`, and
 > `docs/*.md` over this snapshot, and re-run the bundled linter.
 
@@ -38,10 +38,11 @@ cowork-harness skill ./my-skill "do X"      # run the skill once against the sta
 Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.25.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.25.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.25.0"`. **Pin `@>=0.25.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.25.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator, `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-patterns-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0; `--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, and `computer_links_resolve` (new in 0.22.0).)
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.27.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.27.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.27.0"`. **Pin `@>=0.27.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.27.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-patterns-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0; `--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, and `computer_links_resolve` (new in 0.22.0).)
 - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
 - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
 - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
+- **`--dotenv` is a GLOBAL flag — put it BEFORE the subcommand.** `cowork-harness --dotenv .env record …`, never `cowork-harness record … --dotenv .env`. Every *other* flag is subcommand-level, so muscle memory fights this one; the harness rejects the misplaced form with an exact-fix error, but placing it first avoids the round-trip.
 
 ## 1. Pick the loop
 
@@ -55,7 +56,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · lint ·
 verify-run · trace · inspect · diff · stats · decide · gates · answer · scaffold · assertions --list · sync ·
-list · boundary-check · status · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
+list · boundary-check · status · vm <init|status|delete|prune> · doctor · init-redact`. Always check `cowork-harness <cmd> --help`.
 
 **Two different `scaffold` tools — don't confuse them.** The native `cowork-harness scaffold <run-id>`
 above turns an already-*recorded* run into a scenario (needs a run to exist first). The bundled
@@ -157,6 +158,12 @@ checkable regardless of PID namespace. The harness prints `[status] <outDir>` to
 run starts, so capture stderr to get the exact directory. `--follow` fails loud on a timeout/staleness
 rather than hanging forever. See `docs/run-status.md` for the full recipe.
 
+**A multi-minute `record`/`run` outlives a short-lived wrapper.** Don't launch a long record from a
+subagent that returns before it finishes — the returning agent tears down its process tree and kills the
+in-flight run mid-artifact-write. Run it foreground, or detached from any process that will exit first.
+(The `status.json` liveness above is exactly what surfaces such a teardown as `"error"`/`stale` rather
+than a stuck `"running"`.)
+
 ## 6. Assertions: two orthogonal axes
 
 Conflating these is the **biggest landmine**. An assertion key has two independent properties:
@@ -177,6 +184,35 @@ catches it before you push — run it (§9).
 
 See `references/scenario-schema.md` for the full assertion catalog with each key's replay class.
 
+### Which assertion for which question (goal → key)
+
+Beyond the outcome/content keys most scenarios reach for first (`result`, `transcript_*`,
+`file_exists`/`user_visible_artifact`, `artifact_json`), the harness surfaces the agent's *behavior*
+— tool health, sub-agent work, panels, skill attribution, resources — as assertable keys. Reach for
+them by what you're trying to prove:
+
+| You want to check that… | Reach for |
+|---|---|
+| the skill didn't error out of a tool | `tool_no_error: <regex>`, `max_tool_errors: <N>` |
+| it didn't waste repeated identical calls | `max_redundant_tool_calls: <N>` |
+| a deliverable reached the user | `user_visible_artifact: <path>` (+ `no_scratchpad_leak: true` if it delivers via `present_files` — **`container` only**) |
+| a to-do workflow finished | `all_tasks_completed: true`, `task_status: {match, status}` |
+| a skill / connector / tool was **offered** | `skill_available`, `connector_available`, `tool_available` (all `<regex>`) |
+| a skill actually **ran** (or must NOT) | `skill_triggered: <regex>`, `no_skill_triggered: <regex>` |
+| a tool ran **inside** a skill's scope | `skill_tool_used: {skill, tool}` |
+| a sub-agent did the work | `subagent_output_contains: {contains}`, `subagent_dispatched: <regex>`, `dispatch_count_max: <N>` |
+| a pre-existing input wasn't mutated | `input_unmodified: [<glob>]` (live/verify-run; not microvm) |
+| a resource ceiling held | `max_peak_rss_bytes: <N>` (**live-only**) |
+| a hook blocked / didn't block a tool | `hook_blocked: <regex>`, `no_hook_blocked: true` (replay needs a `controlOut` cassette) |
+| every MCP round-trip succeeded | `no_mcp_error: true` (**live-only**) |
+| a context compaction happened | `compaction_occurred: true` |
+
+Every one of these still obeys the two axes above — several are live-only or need a `controlOut`
+cassette on replay, so check the catalog's replay class before putting one on a PR gate.
+`cowork-harness assertions --list` prints the full, always-current key set with one-line semantics
+straight from the schema — treat it (and the catalog) as the source of truth; this map is a
+goal-oriented index into it, not a second catalog.
+
 ## 7. web_fetch (fail-closed, two-path)
 
 `web_fetch` behaves unlike `curl`. A URL is gated by **provenance**, not the egress allowlist:
@@ -263,6 +299,26 @@ The run verdict may include `WARN`-severity signals in addition to pass/fail. On
   the run can still green. If you see it, fix the asset path — a green with a missing asset is
   not a valid pass.
 
+### Inspecting a run's observability output
+
+A verdict is only the top of what a run records. Beyond pass/fail, every `run`/`skill`/`chat` writes a
+`result.json` and a trace you can read without a re-record — use these to *observe* behavior (and to decide
+which assertions from §6 are worth adding), and to diagnose a failure:
+
+- **`cowork-harness trace <run-dir> --view <view>`** — `tool-durations` (per-tool call count + timing),
+  `dispatches` (the sub-agent tree, each dispatch's prompt / model / output), `questions` (gates +
+  sub-question totals), `tools`. Default digests the run.
+- **`cowork-harness stats [--metric <m>]`** — aggregate across the run index: `cost`, `duration`,
+  `tokens`, `cache-tokens`, `model-cost`, `turns`, `pass-rate`.
+- **`result.json` carries the raw fields** the assertions read: `toolDurations`, `models`, `toolErrors`,
+  `redundantToolCalls`, `modelUsage`, `thinking`, `skillActivity`, `subagents[]` (prompt/model/output/
+  `attributedSkillId`), `context` (tools/mcpServers/availableSkills), `tasks`, `workspaceFiles`,
+  `presentedFiles`, `hookEvents`, `mcpErrors`, `contextEvents`, `resources` — see the README's
+  "Observability fields" for the full field semantics.
+- **Opaque failure?** A failed run also records **`errorSource`** (where the failure originated) and
+  **`stderrLogPath`** (the captured agent stderr) — read those and `trace <run-dir>` *before* re-running;
+  a re-record rarely tells you more than the captured stderr already does.
+
 ## 9. Scaffold a valid scenario, lint before you push, then place in CI
 
 Don't hand-write the YAML from memory — that's how invented keys (`assertions:` vs `assert:`,
@@ -308,6 +364,11 @@ the four-stage pipeline.
 **not** an asserted test — no `assert:` block, no cassette. Use it to explore behavior, reproduce a
 bug interactively, or test a prompt before committing it to a scenario.
 
+Each session still writes an informational `result.json` (`mode: "chat"`, no `assertions`) plus a
+trace and index row under its run dir — the same telemetry (tool durations, model usage, resources,
+etc.) that `run`/`skill` produce — so `cowork-harness trace <chat-run-dir>` / `stats` work on a chat
+session too, even though it never yields a verdict.
+
 **`--plugin <dir>` flag (repeatable).** Load additional skill folders alongside the primary session
 plugin. Each `--plugin <dir>` appends the folder to `local_plugins`. Useful when the skill-under-test
 depends on a sibling plugin:
@@ -479,6 +540,14 @@ are the ones that bite hardest.
     `RunResult.artifacts`). A `mode: rw`/`rwd` folder's contents are captured with a full body, same as
     `outputs/`.
 
+21. **A `fidelity: cowork` cassette can go stale in a way `skill`/`format` drift won't catch.** Its recorded
+    `effectiveFidelity` field pins which concrete tier (`hostloop` or `container`) the baseline resolved to
+    AT RECORD TIME. If a later Desktop baseline flips that resolution, `verify-cassettes` reports it as a
+    `resolved-tier` finding (re-record — the recording now exercises the wrong tier); a cassette with no
+    `effectiveFidelity` at all, or an unloadable pinned `baseline:`, reports `unverifiable-tier` instead
+    (couldn't check — also re-record). Both are `fidelity: cowork`-only; an explicit-tier scenario never
+    produces them. See `docs/cassette.md` § tier staleness.
+
 For the complete gotcha list, the assertion catalog, the YAML schema, the fidelity/answer tables,
 and the CI recipe, read the files in `references/`.
 

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 0.25.0` (baseline `desktop-1.18286.0`).
+Self-contained reference. Tracks `cowork-harness 0.27.0` (baseline `desktop-1.18286.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -53,7 +53,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@>=0.25.0"
+- run: npm i -g "cowork-harness@>=0.27.0"
 - run: cowork-harness lint scenarios/*.yaml          # no silent false-greens
 - run: cowork-harness verify-cassettes cassettes/    # privacy + staleness
 - run: cowork-harness replay cassettes/              # token-free content/structure
@@ -187,7 +187,7 @@ jobs:
         with: { node-version: '20' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@>=0.25.0"
+      - run: npm i -g "cowork-harness@>=0.27.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -216,7 +216,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@>=0.25.0"
+        run: npm i -g "cowork-harness@>=0.27.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 0.25.0` (baseline `desktop-1.18286.0`).
+Self-contained reference. Tracks `cowork-harness 0.27.0` (baseline `desktop-1.18286.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,6 +1,6 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.25.0`
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.27.0`
 (baseline `desktop-1.18286.0`). If your checkout is newer, prefer the live `docs/scenario.md`,
 `docs/session.md`, and `SPEC.md`.
 
@@ -209,6 +209,11 @@ that accidentally returns `"first"` will fail the gate rather than silently gree
 Each list item under `assert:` is one assertion. **An item with multiple keys is an AND** — it
 passes only if every key passes. Keep one concern per item unless you mean conjunction.
 
+Looking for a key *by what you want to prove* (tool health, sub-agent work, panels, skill
+attribution, resources, diagnostics)? SKILL.md §6's "goal → key" map is the by-purpose index into this
+table; the table below is the full per-key reference, and `cowork-harness assertions --list` prints the
+same set live from the schema.
+
 | Assertion | Passes when |
 |---|---|
 | `result: success \| error` | the run ended with that status |
@@ -220,6 +225,7 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `user_visible_artifact: <path>` | exists **and** under a user-visible root (`outputs/` + each connected folder's mount name) — the right primitive for a workspace deliverable when a folder is connected |
 | `no_delete_in_outputs: true` | no delete op touched `mnt/outputs` — **only `true` is valid**; `false` is rejected (omit to allow deletes) |
 | `no_unexpected_files: [<glob>, …]` | every **newly created** file under a user-visible root matches ≥1 glob (workRoot-relative paths; `**` = whole path segment for any depth — use `outputs/handoff/**` for per-run subdirs); `[]` = no new files; **new-files-only** — overwriting a pre-existing file in place is invisible (use content-level producer stamping); live/verify-run without a pre-run manifest ⇒ evidence-unavailable hard-fail (live runs capture the baseline only when this key is asserted; recordings always capture); **microvm cannot capture** (use container/hostloop); replay needs `cassette.preRunPaths` (≥0.24 container/hostloop recordings) — cassettes without it **exclude** the key with a loud warning |
+| `input_unmodified: [<glob>, …]` | every **pre-existing** file whose workRoot-relative path matches ≥1 glob keeps an unchanged content hash after the run — the in-place-mutation companion to `no_unexpected_files`'s new-files check (`[]` is rejected by the schema — list at least one glob); a matched file that was deleted counts as a content change (fails); live/verify-run without a pre-run hash manifest ⇒ evidence-unavailable hard-fail; **microvm cannot capture**; replay needs `cassette.preRunHashes` — cassettes without it **exclude** the key with a loud warning; on replay it compares against the manifest's recorded `sha256`, never a re-hash of the materialized tree |
 | `self_heal_ran: <bool>` | a plugin-root self-heal script was (not) invoked |
 | `tool_called: <Tool>` | the agent invoked the tool (actually ran it) |
 | `tool_not_called: <Tool>` | the agent never invoked it |
@@ -229,18 +235,34 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `subagent_tool_absent: <Tool>` | no sub-agent used the tool |
 | `subagent_dispatched: <regex>` | a sub-agent whose `agentType` **or dispatch description** matches |
 | `subagent_declared_but_unused: <Tool>` | a sub-agent declared the tool but never used **that** tool (even if it used others) |
+| `subagent_output_contains: {match?, contains}` | a dispatched sub-agent's own output contains the substring `contains` — `match` (optional regex over `agentType`/`description`) narrows to specific dispatch(es); omitted, checks whether ANY dispatch's output contains it (existence check, not "all") |
 | `dispatch_count_max: <N>` | at most N sub-agents dispatched — an author-chosen budget (Cowork imposes no in-conversation Task-dispatch cap; records only, enforces nothing — see gotcha 12) |
 | `skill_triggered: <regex>` | a skill matching the regex (invoked id, e.g. `"plugin:skill"`) was invoked via the `Skill` tool — evidence-unavailable (not a normal fail) if the agent's init tools have no `Skill` tool |
 | `no_skill_triggered: <regex>` | no invoked skill id matched — the negative-control / description-collision catcher; evidence-unavailable (never a vacuous pass) if invocation data is absent or the `Skill` tool is unobservable |
+| `skill_available: <regex>` | a staged skill's id matched the regex (offered, not necessarily invoked — see `skill_triggered` for invocation) — content-class: the id list comes from the agent's init `skills` listing, so it replays from the frozen init event (id-only; the `whenToUse` enrichment is live-disk and thus absent on replay, but the id is what's matched); evidence-unavailable only if `RunResult.context.availableSkills` is absent entirely (an older cassette recorded before the available-skills listing was captured) |
+| `connector_available: <regex>` | an MCP server/connector's name matched the regex (available, not necessarily used) — evidence-unavailable if `RunResult.context.mcpServers` is absent |
+| `tool_available: <regex>` | a tool in the init manifest matched the regex (available, not necessarily called — see `tool_called` for invocation) — evidence-unavailable if `RunResult.context.tools` is absent |
+| `skill_tool_used: {skill, tool}` | a tool whose name matches `tool` ran inside a skill-activation window whose `skillId` matches `skill` (`RunResult.skillActivity`) — evidence-unavailable if skill-activity telemetry is absent; heuristic for inline skills (a sticky, sequential window matching the agent's `activeSkill` scope, not an exact per-tool boundary) |
 | `max_cost_usd: <N>` | the run's SDK-reported cost is ≤ N USD — evidence-unavailable if cost telemetry is absent. **Replay asserts the frozen recording's cost, not fresh spend** — a real regression needs a live `run` |
 | `max_tokens: <N>` | `usage.input_tokens + usage.output_tokens` ≤ N (cache tokens excluded) — same replay caveat as `max_cost_usd` |
 | `tool_calls_max: <N>` | total top-level tool calls (sum of `toolCounts`) ≤ N — meaningfully replay-checkable (re-drive recomputes `toolCounts` deterministically) |
+| `tool_no_error: <regex>` | no tool whose name matches the regex recorded any error (`RunResult.toolErrors[name].errors === 0` for every match) — **requires ≥1 matching tool call** (a regex matching nothing fails, so a typo can't silently pass); evidence-unavailable if tool-error telemetry is absent |
+| `tool_no_error_if_called: <regex>` | like `tool_no_error` but passes vacuously when no tool matches the regex — the presence-free variant |
+| `max_tool_errors: <N>` | total tool errors across all tools (sum of `RunResult.toolErrors[*].errors`) ≤ N — evidence-unavailable if tool-error telemetry is absent |
+| `max_redundant_tool_calls: <N>` | total WASTED repeated tool calls (sum of `(count-1)` across every redundant `{name,args}` group in `RunResult.redundantToolCalls`) ≤ N — not the raw count of redundant groups; evidence-unavailable if redundant-call telemetry is absent |
 | `max_turns: <N>` | the SDK-reported (or fallback-counted) turn count ≤ N — meaningfully replay-checkable (re-drive recounts turns deterministically, same as `tool_calls_max`) |
+| `compaction_occurred: true` | a context-compaction boundary occurred (a `compact_boundary` system event was recorded) — lives in the stdout stream, so meaningfully replay-checkable; evidence-unavailable if context-event telemetry is absent. **Only `true` is valid** — omit to not require it |
+| `all_tasks_completed: true` | every task in `RunResult.tasks[]` reached status `"completed"` — **requires ≥1 task** (a zero-task run fails; assert `task_count_min` for presence); evidence-unavailable if tasks telemetry is absent |
+| `task_count_min: <N>` | at least N tasks were created (`RunResult.tasks.length >= N`) — the presence companion for task assertions |
+| `task_status: {match, status}` | a task whose `subject` OR `id` matches the regex `match` reached `status` — evidence-unavailable if tasks telemetry is absent |
+| `no_scratchpad_leak: true` | every file presented via `present_files` that was in the scratchpad was successfully promoted to `mnt/outputs` (none left behind) — vacuously passes if nothing was presented (pair with a presence check to require a delivery); content-class: both the `present_files` tool_use and its own tool_result live in the ordinary events stream, so `RunResult.presentedFiles` re-derives identically on replay (meaningfully replay-checkable, same as `skill_triggered`); evidence-unavailable if `presentedFiles` telemetry is absent (an older run predating the feature). **`fidelity: container` only** — `present_files` is not served on hostloop/microvm, so a scratchpad-delivered file is neither promoted to `mnt/outputs` nor detected there (a skill that delivers via write-to-cwd→`present_files` will false-red `user_visible_artifact` on those tiers; use `container`, or write directly to `outputs/`). **Only `true` is valid** |
 | `question_asked: <regex>` | the agent asked an AskUserQuestion whose text matches |
 | `questions_count_max: <N>` | at most N **sub-questions** asked — a bundled `AskUserQuestion` with K sub-questions counts as K, not 1; `trace --view questions`'s footer total uses the same definition |
 | `gate_answers_delivered: true` | every answered gate's answer reached the model (observed `tool_result`; unobserved = fail); **zero gates fired passes vacuously** — pair with `gate_answer_count_min` to also require a gate |
 | `gate_answers_delivered: false` | asserts at least one answered gate's answer was **confirmed not delivered** (an observed delivery failure); an unobserved/null delivery does **not** satisfy this — for negative-path delivery tests |
 | `gate_answer_count_min: <N>` | at least N AskUserQuestion gates fired AND were delivered non-error — presence companion to `gate_answers_delivered`'s vacuous-pass |
+| `hook_blocked: <regex>` | a PreToolUse hook blocked a tool whose name matches the regex (`RunResult.hookEvents`) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette (a custom hook's decision lives only there, not the recorded stream) |
+| `no_hook_blocked: true` | no tool was hook-blocked during the run (distinguishes a real tool crash from an intentional hook block) — evidence-unavailable if hook telemetry is absent. Replay: needs a `controlOut` cassette. **Only `true` is valid** |
 | `allow_permissive_auto_allow: true` | verdict modifier — suppresses the default-fail when the run recorded a cowork-parity permissive auto-allow; for tests that deliberately assert Cowork's permissive behavior |
 | `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
 | `allow_l0_plugin_divergence: true` | verdict modifier — opt into L0/protocol plugin divergence: suppresses the default-fail when a plugin behaves differently at `protocol` (L0) fidelity than under a sandboxed tier. Live tiers only |
@@ -248,8 +270,11 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `transcript_no_host_path: true` | no host path (`/Users/`, `/opt/cowork/`, `/home/`, `/root/`) leaked into model-visible text — **incompatible with `hostloop` AND `protocol`**: hostloop's native file tools legitimately expose real host paths, and protocol (L0) runs the agent's file tools on the real host cwd with no sealed filesystem, so this fails BY DESIGN on both (the harness warns at run start if asserted anyway); use `container`/`microvm` for this check |
 | `egress_denied: <host>` | the host was blocked by the egress proxy |
 | `egress_allowed: <host>` | the host was allowed through |
+| `no_mcp_error: true` | no MCP round-trip failed (`RunResult.mcpErrors` is empty — no unhandled server, no handler throw) — live-only: MCP round-trips are harness-computed, not in the SDK stdout stream, so evidence-unavailable on replay (never a vacuous pass). **Only `true` is valid** |
+| `max_peak_rss_bytes: <N>` | peak sampled RSS of the agent sandbox ≤ N bytes (`RunResult.resources.peakRssBytes`) — live-only: replay never spawns a sandbox to sample, so evidence-unavailable on replay/protocol (never a vacuous pass); also evidence-unavailable when sampling captured no RSS value |
 | `artifact_json: {artifact, path, …}` | assert a JSON artifact's contents — `equals`/`gt`/`in`/`exists`/`absent`/`is_null` over a dotted `path` (`in` = membership in a list, for a stochastic/LLM value; `absent` ≠ `is_null`; an unresolved intermediate fails loud) |
-| `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). Zero links in the transcript **passes** (presence-gated separately — pair with `transcript_contains` if you also need a link to show up). **Only `true` is valid** (`false` is rejected by the schema) |
+| `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). **Requires ≥1 link** (zero links fails — use `computer_links_resolve_if_present` for the presence-free variant). **Only `true` is valid** (`false` is rejected by the schema) |
+| `computer_links_resolve_if_present: true` | like `computer_links_resolve` but passes vacuously when the transcript has zero `computer://` links — the presence-free variant. **Only `true` is valid** |
 
 `expect_denied: [host, …]` adds one `egress_denied` per host. Run `cowork-harness assertions --list` for this
 table from the live schema. Example: `artifact_json: { artifact: outputs/cap.json, path: me.run_id, equals: "r1" }`.
@@ -288,16 +313,21 @@ sourcing ≠ evaluation (replay warns when you edit one). `verify-run` is the on
 *run dir*; `--assert-from` is the equivalent for a *cassette*.
 
 **Evaluated on replay (content):** `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`,
-`skill_triggered`, `no_skill_triggered`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `max_turns`, `result`
+`skill_triggered`, `no_skill_triggered`, `skill_available`, `connector_available`, `tool_available`,
+`skill_tool_used`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `tool_no_error`,
+`max_tool_errors`, `max_redundant_tool_calls`, `max_turns`, `compaction_occurred`, `all_tasks_completed`, `task_status`, `result`
 (`max_cost_usd`/`max_tokens` assert the frozen recording's spend on replay, not fresh spend). The verdict
 modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
 `allow_stall` are also kept on replay, evaluated as no-op passes.
 
 **Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `questions_count_max`,
-`gate_answers_delivered`, `gate_answer_count_min`. With `controlOut` present they evaluate; on an old
+`gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked`. With `controlOut` present they evaluate; on an old
 cassette without it, a **loud warning** fires and they are **excluded** (not vacuously passed). Re-record to enable them.
 `questions_count_max` counts sub-questions, not gates/tool-calls — see the catalog row above and
-`trace --view questions`.
+`trace --view questions`. `hook_blocked`/`no_hook_blocked` need `controlOut` for a different reason than
+the question keys: a custom hook's block/allow decision is an opaque async reply recorded only in
+`control-out.jsonl`, not the `events` stream — reconstructing from the stream alone would show only the
+built-in Task hook's view and could vacuously pass `no_hook_blocked` even if a custom hook genuinely blocked.
 
 **Filesystem — replay-checkable WITH an artifact manifest:** `file_exists`, `user_visible_artifact`,
 `artifact_json`, `computer_links_resolve` run on replay when the cassette carries an `artifacts` snapshot
@@ -308,13 +338,16 @@ by first normalizing it to a mount-relative path (recorded connected-folder pref
 mounts) — replay has no live filesystem to check a host path against directly (that only happens on a live
 `run`/`verify-run`). Without a manifest (older cassettes) all five are skipped; `no_unexpected_files` also
 needs `preRunPaths` (≥0.24 recordings) — without it the key is excluded with a loud warning (live/verify-run
-hard-fails evidence-unavailable instead). A green replay re-confirms
+hard-fails evidence-unavailable instead). `input_unmodified` is the same shape but needs `preRunHashes`
+(the pre-run per-path sha256 baseline) instead of `preRunPaths`; without it, likewise excluded with a loud
+warning. A green replay re-confirms
 *record-time* artifacts, not that the current skill still produces them — `replay --strict` fails when the
 staleness `fingerprint` shows ANY skill/baseline drift, or `replay --fail-on-skill-drift` only on
 skill-source drift; every replay result also reports it class-tagged in `staleness[]` for a JSON gate.
 
 **Egress + other filesystem — still skipped on replay (live-only):** `no_delete_in_outputs`,
-`self_heal_ran`, `transcript_no_host_path`, `egress_*` / `expect_denied`. These run only on a live `run`/`record`.
+`self_heal_ran`, `transcript_no_host_path`, `egress_*` / `expect_denied`, `no_mcp_error`, `max_peak_rss_bytes`.
+These run only on a live `run`/`record`.
 
 **Mixed assertions on replay:** before evaluating, `replay` strips each assertion to its replay-checkable
 keys and drops any left empty. So `{result, egress_denied}` evaluates on replay as `{result}` alone — its
@@ -377,8 +410,9 @@ at the top of this file.
    concern per item; run the linter. (`contentKeys` in `src/run/cassette.ts`.)
 
 2. **Gate keys need a `controlOut` cassette.** `question_asked`, `questions_count_max`,
-   `gate_answers_delivered`, `gate_answer_count_min` only evaluate on replay with `controlOut`; on an
-   old cassette they warn and are excluded (not passed). `gate_answers_delivered` **fails on
+   `gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked` only evaluate on
+   replay with `controlOut`; on an old cassette they warn and are excluded (not passed).
+   `gate_answers_delivered` **fails on
    unobserved delivery** (`delivered: null`) — absence of evidence is failure — but **passes
    vacuously when zero gates fired**; use `gate_answer_count_min: 1` to also require a gate to have
    fired. A **header-only gate** (empty `question`, only `header`) can never be keyed and is rejected

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -47,11 +47,11 @@ what production would do. Two layers of defense:
 
 ### Cassette anatomy (what you're looking at when you open one)
 
-Top-level fields of a `*.cassette.json` (schema `schema/cassette.v7.json`):
+Top-level fields of a `*.cassette.json` (schema `schema/cassette.v9.json`):
 
 | Field | What it is |
 |---|---|
-| `$schema`, `generator`, `cassetteVersion` | Provenance: schema URL, producing tool, format version (current: 7) |
+| `$schema`, `generator`, `cassetteVersion` | Provenance: schema URL, producing tool, format version (current: 9) |
 | `scenario` | The embedded scenario snapshot at record time |
 | `events` | The recorded agent event stream (the replay source) |
 | `controlOut` | Driver→agent control responses — presence unlocks gate asserts on replay |
@@ -63,6 +63,8 @@ Top-level fields of a `*.cassette.json` (schema `schema/cassette.v7.json`):
 | `preRunPaths` | Pre-run file-path baseline for `no_unexpected_files` (workRoot-relative; co-present with `userVisibleRoots`) |
 | `scenarioSource` | Relative path to the authored YAML this was recorded from |
 | `authoring` | Present iff a live decider answered ≥1 gate during recording (`nonDeterministic: true`) |
+| `sessionFingerprint` | v9+: hash of the session's content-relevant SHAPE (folders/plugins/skills/mcp/egress). Checked ONLY by `verify-cassettes`, never the default replay verdict; absent on a pre-v9 cassette → not checked |
+| `folderPrefixMap` | v9+: the record-time connected-folder host-path → mount-name map. Replay's `computer_links_resolve` uses THIS (never the current session file) for a v9+ cassette; a pre-v9 cassette keeps reconstructing from the current session |
 
 ## Recipe 3 — Set up redaction BEFORE your first hostloop/protocol record
 

package/.claude/skills/cowork-harness/scripts/assertion-keys.json

@@ -1,22 +1,34 @@
 {
   "$comment": "GENERATED from the Zod schemas (src/types.ts) by scripts/gen-schema.ts — do not edit; run `npm run schema`.",
   "keys": [
+    "all_tasks_completed",
     "allow_l0_plugin_divergence",
     "allow_missing_capability",
     "allow_permissive_auto_allow",
     "allow_stall",
     "artifact_json",
+    "compaction_occurred",
     "computer_links_resolve",
+    "computer_links_resolve_if_present",
+    "connector_available",
     "dispatch_count_max",
     "egress_allowed",
     "egress_denied",
     "file_exists",
     "gate_answer_count_min",
     "gate_answers_delivered",
+    "hook_blocked",
+    "input_unmodified",
     "max_cost_usd",
+    "max_peak_rss_bytes",
+    "max_redundant_tool_calls",
     "max_tokens",
+    "max_tool_errors",
     "max_turns",
     "no_delete_in_outputs",
+    "no_hook_blocked",
+    "no_mcp_error",
+    "no_scratchpad_leak",
     "no_skill_triggered",
     "no_unexpected_files",
     "question_asked",
@@ -24,13 +36,21 @@
     "replay_protocol_fidelity",
     "result",
     "self_heal_ran",
+    "skill_available",
+    "skill_tool_used",
     "skill_triggered",
     "subagent_declared_but_unused",
     "subagent_dispatched",
+    "subagent_output_contains",
     "subagent_tool_absent",
     "subagent_tool_used",
+    "task_count_min",
+    "task_status",
+    "tool_available",
     "tool_called",
     "tool_calls_max",
+    "tool_no_error",
+    "tool_no_error_if_called",
     "tool_not_called",
     "tool_result_contains",
     "tool_result_not_contains",
@@ -53,7 +73,8 @@
     "prompt",
     "requires_capabilities",
     "session",
-    "skills"
+    "skills",
+    "timeout_ms"
   ],
   "verdictModifierKeys": [
     "allow_l0_plugin_divergence",

package/.claude/skills/cowork-harness/scripts/scenario.py

@@ -63,21 +63,50 @@ CONTENT_KEYS = {
     "subagent_tool_absent",
     "subagent_dispatched",
     "subagent_declared_but_unused",
+    "subagent_output_contains",
     "dispatch_count_max",
     "skill_triggered",
     "no_skill_triggered",
+    "skill_available",
+    "connector_available",
+    "tool_available",
+    "skill_tool_used",
     "max_cost_usd",
     "max_tokens",
     "tool_calls_max",
+    "tool_no_error",
+    "tool_no_error_if_called",
+    "max_tool_errors",
+    "max_redundant_tool_calls",
     "max_turns",
+    "compaction_occurred",
+    "all_tasks_completed",
+    "task_count_min",
+    "task_status",
+    "no_scratchpad_leak",
 }
 # content keys, but only evaluated on replay when the cassette carries controlOut
-GATE_KEYS = {"question_asked", "questions_count_max", "gate_answers_delivered", "gate_answer_count_min"}
+GATE_KEYS = {
+    "question_asked",
+    "questions_count_max",
+    "gate_answers_delivered",
+    "gate_answer_count_min",
+    "hook_blocked",
+    "no_hook_blocked",
+}
 # manifest-backed: replay-checkable when the cassette carries an `artifacts` manifest (record snapshots one);
 # a manifest-less cassette skips them. Since the 0.3.0 artifact-manifest these are NOT always live-only.
 # computer_links_resolve joins this bucket (not CONTENT_KEYS): resolving a non-empty link set needs
 # either a live filesystem or the cassette's artifacts manifest — see cassette.ts's manifestKeys comment.
-MANIFEST_KEYS = {"file_exists", "user_visible_artifact", "artifact_json", "computer_links_resolve", "no_unexpected_files"}
+MANIFEST_KEYS = {
+    "file_exists",
+    "user_visible_artifact",
+    "artifact_json",
+    "computer_links_resolve",
+    "computer_links_resolve_if_present",
+    "no_unexpected_files",
+    "input_unmodified",
+}
 # live-only: ALWAYS skipped on replay, with a loud warning (no filesystem, no network on the token-free lane)
 LIVE_ONLY_KEYS = {
     "egress_denied",
@@ -85,6 +114,8 @@ LIVE_ONLY_KEYS = {
     "no_delete_in_outputs",
     "self_heal_ran",
     "transcript_no_host_path",
+    "no_mcp_error",
+    "max_peak_rss_bytes",
 }
 EGRESS_KEYS = {"egress_denied", "egress_allowed"}
 # verdict modifiers — don't verify anything themselves (e.g. suppress a default-fail)
@@ -141,6 +172,7 @@ _EMBEDDED_TOP_LEVEL_KEYS = {
     "fidelity",
     "on_unanswered",
     "prompt",
+    "timeout_ms",  # wall-clock budget → kill + errorSource:timeout on expiry
     "answers",
     "expect_denied",
     "assert",
@@ -178,6 +210,7 @@ REGEX_KEYS = {
     "when_question",
     "subagent_dispatched",
     "question_asked",
+    "hook_blocked",
 }
 VALID_ON_UNANSWERED = {"fail", "prompt", "first", "llm"}
 VALID_TIERS = ("protocol", "container", "microvm", "hostloop", "cowork")

package/.env.example

@@ -1,6 +1,8 @@
 # cowork-harness host-side credentials.
 #
-# Copy to `.env` (auto-loaded from the working dir at startup; already exported vars win).
+# Copy to `.env`. Resolution order: exported env > `--dotenv <path>` > `./.env` (cwd) >
+# `<install-root>/.env` — so a global/npm-linked install still finds this file at the package
+# root even when you run from an unrelated directory. Already-exported vars always win.
 # `.env` is gitignored. SECURITY: keep this at your repo/working-dir ROOT — never inside a
 # mounted skill/project folder, or its contents would be copied into the agent's sandbox.
 # The token value is also scrubbed from all persisted run logs regardless of source.
@@ -12,5 +14,9 @@ CLAUDE_CODE_OAUTH_TOKEN=
 # token is present, mirroring the desktop).
 # ANTHROPIC_API_KEY=
 
+# Alternative: a raw Anthropic auth token. Same precedence as ANTHROPIC_API_KEY (used only when
+# no OAuth token is set) — set at most one of the two.
+# ANTHROPIC_AUTH_TOKEN=
+
 # Optional: timezone passed through to the agent.
 # TZ=America/New_York