cowork-harness 1.18.0 → 1.19.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 1.18.0
-  tracks-harness: cowork-harness 1.18.0 (baseline desktop-1.25927.0)
+  version: 1.19.0
+  tracks-harness: cowork-harness 1.19.0 (baseline desktop-1.25927.0)
 ---
 
 # cowork-harness
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.18.0` (baseline
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.19.0` (baseline
 > `desktop-1.25927.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
@@ -39,7 +39,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.18.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.18.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@>=1.18.0"`. **Pin `@>=1.18.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.19.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.19.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@>=1.19.0"`. **Pin `@>=1.19.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -547,7 +547,7 @@ Docker, no re-record.
 | Situation | Symptom | Reach for (in order) |
 |---|---|---|
 | **The skill misbehaved** | wrong output, an unexpected gate, a denied tool, an opaque crash | `inspect` — what did it produce? · `trace <run-dir> --view <view>` — what did it actually do (tools, gates, sub-agent tree)? · `verify-run` — re-assert cheaply when only an assertion is wrong · `diff <old-run> <new-run>` — what changed since it worked · `chat` — reproduce it by hand |
-| **A green you don't trust** | an assert that may have tested nothing, a stale cassette, an auto-answered or decided gate | `replay --explain` — the evidence trail behind each *passing* assert · `replay --mutate` — perturbs each recorded JSON artifact value and reports which perturbations NOTHING caught (reporting only; never moves the verdict/exit code) · `lint` — assertions on the wrong CI lane / mixed-class keys · `verify-cassettes` — privacy + staleness over committed cassettes · the Gotchas landmine catalog — how a check passes vacuously · `run --repeat N` / `skill --repeat N` — did it pass, or pass once? · `stats` — flaky or expensive over time |
+| **A green you don't trust** | an assert that may have tested nothing, a stale cassette, an auto-answered or decided gate | `replay --explain` — the evidence trail behind each *passing* assert · `replay --mutate` — perturbs a CAPPED SAMPLE of recorded JSON values (10/file, 50 total) and reports which perturbations NOTHING caught; the report names the sample size, so read it as a sample not a total (reporting only; never moves the verdict/exit code) · `lint` — assertions on the wrong CI lane / mixed-class keys · `verify-cassettes` — privacy + staleness over committed cassettes · the Gotchas landmine catalog — how a check passes vacuously · `run --repeat N` / `skill --repeat N` — did it pass, or pass once? · `stats` — flaky or expensive over time |
 
 A failed run also records `errorSource` (where the failure originated) and `stderrLogPath` (the captured
 agent stderr) — read those before re-running; a re-record rarely tells you more than the captured stderr

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 1.18.0` (baseline `desktop-1.25927.0`).
+Self-contained reference. Tracks `cowork-harness 1.19.0` (baseline `desktop-1.25927.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -13,7 +13,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 ```
 
 The Action's `version` input defaults to `latest` — intentional so a copy-pasted recipe tracks the current
-release; pin an exact version (e.g. `version: "1.18.0"`) for reproducible CI.
+release; pin an exact version (e.g. `version: "1.19.0"`) for reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -58,7 +58,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@>=1.18.0"
+- run: npm i -g "cowork-harness@>=1.19.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -279,7 +279,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@>=1.18.0"
+      - run: npm i -g "cowork-harness@>=1.19.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -308,7 +308,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@>=1.18.0"
+        run: npm i -g "cowork-harness@>=1.19.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 1.18.0` (baseline `desktop-1.25927.0`). This is **not** a trim of the full
+Tracks `cowork-harness 1.19.0` (baseline `desktop-1.25927.0`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reproduction discipline, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 1.18.0` (baseline `desktop-1.25927.0`).
+Self-contained reference. Tracks `cowork-harness 1.19.0` (baseline `desktop-1.25927.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,6 +1,6 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.18.0`
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.19.0`
 (baseline `desktop-1.25927.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 1.18.0` (baseline `desktop-1.25927.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 1.19.0` (baseline `desktop-1.25927.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.
 

package/CHANGELOG.md

@@ -6,6 +6,61 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [1.19.0] — 2026-08-06
+
+Follow-up to the consumer report against published 1.18.0: two false-positive/misreporting fixes in the
+tooling 1.18.0 introduced, plus the scoping and summary controls those reports asked for.
+
+### Upgrade notes
+
+- **The new `skills[]` axis can fail a cassette that passed under 1.18.0.** It applies only at
+  host-inheriting tiers (`protocol`, `hostloop`, or `cowork` resolving to hostloop). A flagged entry is a
+  skill name from the recording machine; re-record against a clean environment, or scope an allow with
+  `--allow-host-inventory <regex>` after reviewing the finding.
+- **Conversely, `host-inventory` failures on a plugin's own agents go away.** If 1.18.0 forced you to add
+  an allow regex for `<plugin>:<agent>` entries, that exemption is now automatic and the regex can be
+  removed. No re-record is needed.
+
+### Added
+
+- **`verify-cassettes` reads `skills[]` as a `host-inventory` axis.** A skill name discloses what is
+  installed on the recording machine, the same class of disclosure as an MCP server name, and the field
+  was previously read by no axis. Two exemptions: the agent's built-in skills, and a `<plugin>:<skill>`
+  whose plugin the same recording declares in `plugins[]`. Host-inheriting tiers only — at `protocol`
+  with local OAuth the harness keeps the operator's real `CLAUDE_CONFIG_DIR`, so personally installed
+  skills are discoverable there. `plugins[].name` is deliberately not scanned: that array holds only
+  entries the harness supplied via `--plugin-dir`, so it carries no host inventory to find.
+- **`replay --mutate` accepts scoping and cap overrides.** `--mutate-include <glob>` and
+  `--mutate-exclude <glob>` (both repeatable; exclude applied last) restrict which artifact paths are
+  perturbed — `*` matches within a path segment, `**` across them. Filtering is applied before planning,
+  so an out-of-scope artifact is excluded from the eligible count as well. `--mutate-max-per-file <n>`
+  and `--mutate-max-total <n>` raise the sampling caps (defaults 10 and 50); prefer the per-file one,
+  which binds first. Any of these without `--mutate` is a usage error. Cost is linear — one full
+  assertion re-run per perturbation.
+
+### Changed
+
+- **`verify-cassettes` prints a per-class findings count before the per-file listing**, e.g.
+  `findings by class: host-inventory 240`. Informational classes such as `unscanned` are counted too, so
+  the header agrees with the rows beneath it. The per-file rows are unchanged — the rollup is additive.
+  JSON output is unchanged; `findings[].cls` already carried this.
+
+### Fixed
+
+- **`replay --mutate` reported its sample as though it were the full set.** The plan is capped at 10
+  values per file and 50 in total, with the per-file cap applied first, but the report showed only
+  `<uncaught>/<planned>`. It now appends the eligible total and names the binding cap — `(sampled 30 of
+  120 eligible value(s); per-file cap 10 reached on 3 file(s))` — and omits the note entirely when
+  nothing was truncated. A bare ratio from 1.18.0 or earlier describes the sample, not the corpus.
+- **`replay --mutate` produced no machine-readable output.** Coverage now rides the JSON envelope as
+  `mutation`, carrying `sampled`, `eligible`, `truncatedBy`, `caps`, and `uncaught`.
+- **The `host-inventory` check flagged the scenario's own plugin agents.** At `hostloop` the agent roster
+  necessarily includes the agents of the mounted plugin, so any plugin declaring agents failed the gate
+  on upgrade to 1.18.0. An agent named `<plugin>:<agent>` whose plugin the same recording declares in
+  `plugins[]` is now exempt, matching the existing carve-out for `mcp.config`-attached servers. The
+  exemption is derived from the cassette, so it applies to recordings made before this release without
+  re-recording. A foreign agent, or one namespaced to a plugin the run never declared, still flags.
+
 ## [1.18.0] — 2026-08-06
 
 ### Upgrade notes
@@ -22,6 +77,11 @@ All notable changes to this project are documented here. The format is based on
 
 ### Fixed
 
+- **`critique --help` and `docs/critique.md` misstated the tier prerequisites.** Both said the `container`
+  and `hostloop` tiers need an authenticated `claude` CLI on PATH. They need a token in the environment
+  or `.env` (`CLAUDE_CODE_OAUTH_TOKEN`, or `ANTHROPIC_API_KEY` as a CI fallback) — the graded turns run
+  the staged agent binary, not the host CLI. The evaluator passes are what require `claude` on PATH,
+  overridable with `COWORK_HARNESS_CLAUDE_BIN`.
 - **A scenario's `on_unanswered:` overrode an explicit `--on-unanswered` in silence.** The precedence is
   intentional and documented (`run --help`: "per-scenario answers/on_unanswered in the YAML take
   precedence where set") — a committed scenario is the reproducible definition of its own test — but a

package/README.md

@@ -92,7 +92,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
 
 > **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
 > <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
-> From a global install (`npm i -g "cowork-harness@>=1.18.0"`), point at the package root instead:
+> From a global install (`npm i -g "cowork-harness@>=1.19.0"`), point at the package root instead:
 > `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
 > (or copy the cassette into your own project and pass that path).
 
@@ -102,7 +102,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
 > - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
 > - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
 > - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
-> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=1.18.0"`.
+> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=1.19.0"`.
 
 Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
 test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
@@ -127,7 +127,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
 claude plugin install cowork-harness@cowork-harness
 ```
 
-The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=1.18.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 22). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
+The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=1.19.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 22). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
 
 It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
 
@@ -149,7 +149,7 @@ A global install is enough for CI `lint`, reading the teaching skill, and replay
 To `run` the worked examples live or copy them as a starting point, use a source checkout. (The marketplace
 skill install itself only pulls `.claude/skills/cowork-harness/` — SKILL.md + `references/` + `scenario.py`/
 assertion keys, per `.claude-plugin/marketplace.json`'s `source` — not the rest of this table; the full set
-above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=1.18.0"` — see
+above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=1.19.0"` — see
 [above](#drive-it-from-claude-code-companion-skill) — which pulls the same npm package as the global-install row.)
 
 ### Prerequisites for anything above `protocol` fidelity
@@ -708,7 +708,7 @@ jobs:
           anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
 ```
 
-Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — intentional so recipes track the current release; pin an exact version for reproducible CI. The companion skill's `cowork-harness@>=1.18.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). Outputs: `ok` (`"true"`/`"false"`, mirrors the exit code), `envelope-path` (path to the raw JSON envelope, for post-processing), `summary-md` (the rendered verdict table, exposed as an output — not just written to `$GITHUB_STEP_SUMMARY` — because that file is scoped to this action's own invocation and a caller's later step gets a fresh, empty one). See [`action.yml`](./action.yml) for the full input/output reference.
+Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — intentional so recipes track the current release; pin an exact version for reproducible CI. The companion skill's `cowork-harness@>=1.19.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). Outputs: `ok` (`"true"`/`"false"`, mirrors the exit code), `envelope-path` (path to the raw JSON envelope, for post-processing), `summary-md` (the rendered verdict table, exposed as an output — not just written to `$GITHUB_STEP_SUMMARY` — because that file is scoped to this action's own invocation and a caller's later step gets a fresh, empty one). See [`action.yml`](./action.yml) for the full input/output reference.
 
 The provided [GitHub Actions workflow](.github/workflows/ci.yml) runs an **eight-stage pipeline**. The **build** + **test** stages are the token-free gate you can copy into your skill repo; the `floor`, `action-self-test`, `python`, `boundary`, `scenarios`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):
 

package/dist/run/cassette.js

@@ -32,7 +32,7 @@ import { isVmSessionsPath } from "../vm-paths.js";
  *  pool (each run creates two networks) and press model API rate limits — both surface as actionable errors. */
 const MAX_RECORD_CONCURRENCY = 8;
 import { evaluate, budgetFields } from "../assert.js";
-import { planMutations, applyMutation, explainNoMutations } from "./mutate.js";
+import { planMutationsWithStats, summarizeMutationPlan, matchesAnyGlob, applyMutation, explainNoMutations, } from "./mutate.js";
 import { anyGlobMatches } from "../glob.js";
 import { extractComputerLinks } from "./computer-links.js";
 import { makeRenderer, renderFooter } from "./renderer.js";
@@ -787,6 +787,43 @@ function isCapabilityManifest(line) {
  *  machine's own inventory. `cowork` is included because it resolves to container OR hostloop via a baseline
  *  gate — the privacy scan fails closed rather than loading a baseline to find out. */
 const HOST_INHERITING_TIERS = new Set(["protocol", "hostloop", "cowork"]);
+/** Plugin names the scenario mounted, harvested from anywhere in the event stream.
+ *
+ *  `system/init` carries `plugins[]` beside `agents[]`, so that surface is self-sufficient — but the
+ *  registry `control_response` lists agents with NO plugin array of its own. Collecting once over the
+ *  whole stream and passing the result into every scan covers both, and keeps `scanHostInventory` a pure
+ *  function of one payload plus explicit context. Tolerant by construction: a non-JSON line, a missing
+ *  `plugins`, or a bare-string entry are all normal, not errors. */
+export function collectDeclaredPlugins(events) {
+    const names = new Set();
+    const visit = (node) => {
+        if (Array.isArray(node)) {
+            for (const v of node)
+                visit(v);
+            return;
+        }
+        if (node === null || typeof node !== "object")
+            return;
+        const o = node;
+        if (Array.isArray(o.plugins))
+            for (const p of o.plugins) {
+                const n = typeof p === "string" ? p : p?.name;
+                if (typeof n === "string" && n)
+                    names.add(n);
+            }
+        for (const v of Object.values(o))
+            visit(v);
+    };
+    for (const line of events ?? []) {
+        try {
+            visit(JSON.parse(line));
+        }
+        catch {
+            continue; // a non-JSON transcript line declares no plugins
+        }
+    }
+    return [...names];
+}
 export function scanCassette(cassette, allow) {
     const findings = [];
     const FULL = DEFAULT_SCAN_PATTERNS; // email + currency + domain + path + machine-inventory
@@ -823,6 +860,11 @@ export function scanCassette(cassette, allow) {
     // identify as sealed (`container`, `microvm`) is exempt. Same fail-closed reasoning as `cowork`.
     const tier = (cassette.effectiveFidelity ?? cassette.scenario.fidelity);
     if (tier === undefined || HOST_INHERITING_TIERS.has(tier)) {
+        // Harvest the plugin names the SCENARIO mounted, once, across the whole event stream — then hand them
+        // to every scan. `system/init` carries `plugins[]` beside `agents[]` so it is self-sufficient, but the
+        // registry `control_response` lists agents with no plugin array of its own; without this pre-pass that
+        // surface keeps flagging the scenario's own plugin agents. See scanHostInventory's A4.
+        const declaredPlugins = collectDeclaredPlugins(cassette.events);
         const structural = (lines, key) => lines?.forEach((l, i) => {
             let decoded;
             try {
@@ -831,7 +873,7 @@ export function scanCassette(cassette, allow) {
             catch {
                 return; // a non-JSON transcript line has no name fields to read
             }
-            findings.push(...scanHostInventory(decoded, `${key}[${i}]`, allow));
+            findings.push(...scanHostInventory(decoded, `${key}[${i}]`, allow, declaredPlugins));
         });
         structural(cassette.events, "events");
         structural(cassette.controlOut, "controlOut");
@@ -3134,6 +3176,7 @@ function replayErrorResult(file) {
         effectiveFidelity: undefined,
         fidelityWarnings: undefined,
         staleness: undefined,
+        mutation: undefined, // the protocol-error stub carries no mutation report
         skippedAssertions: undefined,
         toolResults: undefined,
         l0PluginDivergence: undefined,
@@ -3479,7 +3522,13 @@ export const REPLAY_BOOLEAN_FLAGS = [
     "--quiet",
     "--verbose",
 ];
-export const REPLAY_VALUE_FLAGS = ["--output-format", "--assert-from"];
+export const REPLAY_VALUE_FLAGS = ["--output-format", "--assert-from", "--mutate-max-per-file", "--mutate-max-total"];
+// A SEPARATE axis from REPLAY_VALUE_FLAGS, for the same reason VERIFY_CASSETTES_REPEATED_FLAGS is one:
+// parseArgs collects every occurrence of a repeated flag into p.repeated[], while a value flag keeps only
+// the LAST. Scoping is inherently multi-pattern ("everything but handoff/ and tmp/"), so folding these in
+// would silently honour one pattern and drop the rest — a semantic regression, not a refactor. `replay`
+// had no repeated-flag axis at all before these.
+export const REPLAY_REPEATED_FLAGS = ["--mutate-include", "--mutate-exclude"];
 // Both inert on `replay` — parsed into p.flags, never read anywhere in cmdReplay below (verified by
 // reading every `p.flags["--quiet"]`/`p.flags["--verbose"]` site in this file: none exists outside
 // cmdRecord). Contrast RECORD_ALLOWLIST's --verbose entry: record's --quiet IS wired up there, so it's
@@ -3494,7 +3543,9 @@ export const REPLAY_ALLOWLIST = [
 // one; this one was missing --best-effort-future-cassette entirely, undiscoverable at the exact point
 // (`cassette format too new: …`) that tells a user to pass it — the P9 bug this generalization fixes).
 export const REPLAY_USAGE = "usage: replay <file.cassette.json | dir/> [--strict] [--fail-on-skill-drift] [--mutate] [--assert-from <scenario.yaml> | --reassert] [--write [--allow-failing]] [--explain] [--best-effort-future-cassette] [--output-format text|json]\n" +
-    "       --mutate: perturb each recorded JSON artifact value, re-run the assertions, and report which perturbations NOTHING caught — those fields are unguarded. Reporting only; never changes the verdict or exit code.\n" +
+    "       --mutate: perturb recorded JSON artifact values, re-run the assertions, and report which perturbations NOTHING caught — those fields are unguarded. SAMPLES: max 10 values per file, 50 total (per-file applies first), so `N/N caught by nothing` is N of the sample, not of your corpus; when a cap binds the report names it and the eligible total. Reporting only; never changes the verdict or exit code.\n" +
+    "       --mutate-include <glob> / --mutate-exclude <glob>: scope which artifact paths are perturbed (repeatable; exclude wins). `*` stays inside a path segment, `**` crosses them — so `--mutate-exclude 'handoff/**'` drops per-run internals nobody should assert on, and the sample is spent on deliverables instead.\n" +
+    "       --mutate-max-per-file <n> / --mutate-max-total <n>: raise the sampling caps (default 10 / 50). Per-file is applied FIRST, so with a handful of artifacts raising only --mutate-max-total changes nothing. Cost is linear — one full assertion re-run per perturbation.\n" +
     "       --explain: after the footer, print the evidence trail for each PASSING assert (which link resolved, which file matched, which value satisfied a bound) — text mode; json already carries assertions[].evidence.\n" +
     "       by default the assertions FROZEN in the cassette drive the verdict (deterministic); a sibling scenario whose assert: differs only prints a notice.\n" +
     `       --assert-from <file> / --reassert: token-free re-check against the on-disk assert:/expect_denied: — recording-shaping drift (${RECORDING_SHAPING_FIELDS.join("/")}) and skill staleness HARD-FAIL.\n` +
@@ -3510,6 +3561,7 @@ export async function cmdReplay(args) {
             // fork this list back into a local literal (that's the drift P9 generalized P3's fix to prevent).
             booleans: [...REPLAY_BOOLEAN_FLAGS],
             values: [...REPLAY_VALUE_FLAGS],
+            repeated: [...REPLAY_REPEATED_FLAGS],
             enums: { "--output-format": ["text", "json"] },
             aliases: { "-q": "--quiet" },
         });
@@ -3552,6 +3604,32 @@ export async function cmdReplay(args) {
         warn("::notice:: [replay] --allow-failing only affects --write's verdict gate; it is a no-op without --write\n");
     const failOnSkillDrift = (p.flags["--fail-on-skill-drift"] ?? false) || reassertMode; // narrower gate: only skill-source drift fails
     const mutate = p.flags["--mutate"] ?? false;
+    // Scoping + cap overrides for --mutate. Repeated flags come back as arrays; exclude is applied after
+    // include so "everything under outputs/, except the scratch subtree" reads the obvious way.
+    const mutateInclude = p.repeated["--mutate-include"] ?? [];
+    const mutateExclude = p.repeated["--mutate-exclude"] ?? [];
+    const mutateCap = (flag) => {
+        const raw = p.options[flag];
+        if (raw === undefined)
+            return undefined;
+        const n = Number(raw);
+        if (!Number.isInteger(n) || n < 1)
+            return fail("replay", "usage", `${flag} must be a positive integer (got "${raw}")`, undefined, asJson);
+        return n;
+    };
+    const mutateMaxPerFile = mutateCap("--mutate-max-per-file");
+    const mutateMaxTotal = mutateCap("--mutate-max-total");
+    // These only shape a --mutate pass; accepting them silently without it would look like scoping worked.
+    if (!mutate) {
+        const orphan = [
+            mutateInclude.length ? "--mutate-include" : "",
+            mutateExclude.length ? "--mutate-exclude" : "",
+            mutateMaxPerFile !== undefined ? "--mutate-max-per-file" : "",
+            mutateMaxTotal !== undefined ? "--mutate-max-total" : "",
+        ].filter(Boolean);
+        if (orphan.length)
+            return fail("replay", "usage", `${orphan.join(", ")} require(s) --mutate (they only shape a mutation pass).`, undefined, asJson);
+    }
     if (strict && p.flags["--fail-on-skill-drift"])
         warn("::notice:: [replay] --strict and --fail-on-skill-drift both passed — --strict is the superset (fails on every class), so --fail-on-skill-drift is redundant here\n");
     const resolved = resolveInputs(target, ".cassette.json");
@@ -3707,6 +3785,10 @@ export async function cmdReplay(args) {
                 strict,
                 failOnSkillDrift,
                 mutate,
+                mutateInclude,
+                mutateExclude,
+                mutateMaxPerFile,
+                mutateMaxTotal,
                 cassetteDir: dirname(f),
                 bestEffortFutureCassette,
                 notesSink: collectNotes,
@@ -3875,7 +3957,7 @@ export const USAGE_GUARD_REGISTRY = [
         command: "replay",
         booleanFlags: REPLAY_BOOLEAN_FLAGS,
         valueFlags: REPLAY_VALUE_FLAGS,
-        repeatedFlags: [],
+        repeatedFlags: REPLAY_REPEATED_FLAGS,
         aliases: { "-q": "--quiet" },
         usage: REPLAY_USAGE,
         allowlist: REPLAY_ALLOWLIST,
@@ -4126,6 +4208,20 @@ export async function cmdVerifyCassettes(args) {
             log("⚠ cowork-harness: --skip-privacy: privacy scan was skipped");
         if (!doScenarioDrift)
             log("⚠ cowork-harness: --skip-scenario-drift: scenario prompt-drift check was skipped");
+        // A per-class count, printed ONCE ahead of the per-file rows. A sweep that surfaces hundreds of
+        // findings of a single class reads as hundreds of separate problems; a consumer piped the output
+        // through `uniq -c` to discover 240 findings were one class with one cause. This is additive — every
+        // per-file row below still prints, so the attribution the `notes` rationale protects is untouched.
+        // It answers "what kind, and how many" before the reader starts scrolling, not instead of it.
+        const byClass = new Map();
+        for (const r of results)
+            for (const f of r.findings)
+                byClass.set(f.cls, (byClass.get(f.cls) ?? 0) + 1);
+        if (byClass.size)
+            log(`findings by class: ${[...byClass.entries()]
+                .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
+                .map(([cls, n]) => `${cls} ${n}`)
+                .join(" · ")}`);
         for (const r of results) {
             if (r.error)
                 log(`✗ ${r.file}: [error] ${r.error}`);
@@ -4787,6 +4883,9 @@ export async function replayCassette(cassette, hooks = [], opts = {}) {
     // outside the try, since it doesn't depend on anything materializeManifest produced.
     const folderPrefixResolution = buildFolderPrefixMap(cassette);
     // Populated only under --mutate; surfaced after the verdict so it reads as coverage info, not a failure.
+    // Surfaced on the RunResult so `--mutate --output-format json` is machine-readable. It was previously
+    // assigned here and never read by anything, so every consumer had to scrape stderr — which is how the
+    // capped denominator went unnoticed in the first place.
     let mutationReport;
     // materializeManifest created a temp dir (`replayWorkRoot`) above; everything below uses it and
     // then returns. Wrap the rest in try/finally so the temp dir is removed on every exit path (normal
@@ -4955,7 +5054,21 @@ export async function replayCassette(cassette, hooks = [], opts = {}) {
             const inlined = (cassette.artifacts ?? [])
                 .filter((a) => typeof a.body === "string" && !a.truncated)
                 .map((a) => ({ path: a.path, body: a.body }));
-            const plan = planMutations(inlined);
+            // planMutationsWITHSTATS, not planMutations: the plan is CAPPED (10 per file, 50 total), so
+            // reporting `uncaught/plan.length` alone reads as "N of your N fields are unguarded" when N is a
+            // sample. A consumer aggregated twenty such lines into "1,020 perturbations, 0 caught" and came one
+            // step from concluding their assertions verified nothing — the truth was their asserted paths were
+            // never in the sample. The stats form exists precisely to make that discoverable; it was simply
+            // never wired to its only caller.
+            const include = opts.mutateInclude ?? [];
+            const exclude = opts.mutateExclude ?? [];
+            // Filter BEFORE planning so the caps bind on what survived, not on the whole corpus. An artifact
+            // dropped here is absent from `eligible` too — the report describes the scope you asked for.
+            const scoped = inlined.filter((a) => (include.length === 0 || matchesAnyGlob(a.path, include)) && !matchesAnyGlob(a.path, exclude));
+            const planOpts = { maxPerFile: opts.mutateMaxPerFile, maxTotal: opts.mutateMaxTotal };
+            const stats = planMutationsWithStats(scoped, planOpts);
+            const plan = stats.mutations;
+            const coverage = summarizeMutationPlan(stats, planOpts);
             const uncaught = [];
             const baselineFailed = new Set(assertions.map((a, i) => (a.pass ? -1 : i)).filter((i) => i >= 0));
             for (const m of plan) {
@@ -4980,16 +5093,36 @@ export async function replayCassette(cassette, hooks = [], opts = {}) {
                     writeFileSync(abs, original); // restore unconditionally — a thrown assert must not leave a mutated tree
                 }
             }
-            mutationReport = { planned: plan.length, uncaught };
+            mutationReport = {
+                sampled: plan.length,
+                eligible: coverage.eligible,
+                truncatedBy: coverage.truncatedBy,
+                caps: coverage.caps,
+                uncaught,
+            };
+            // Only when truncation actually happened: on a corpus under both caps the counts ARE the whole
+            // truth and the parenthetical is noise. Naming the binding cap matters — the per-file cap is
+            // checked first, so "raise --mutate-max-total" is inert advice whenever per-file bound.
+            const scope = coverage.truncatedBy
+                ? ` (sampled ${coverage.sampled} of ${coverage.eligible} eligible value(s)` +
+                    (coverage.truncatedBy === "per-file"
+                        ? `; per-file cap ${coverage.caps.perFile} reached on ${coverage.filesAtPerFileCap} file(s)`
+                        : `; total cap ${coverage.caps.total} reached`) +
+                    `)`
+                : "";
             if (!plan.length)
-                log(`::notice:: [mutate] ${explainNoMutations(cassette.artifacts ?? [], inlined)}`);
+                // Attribute an empty plan to the FILTERS when they are what emptied it — otherwise the generic
+                // "no perturbable values" explanation sends the reader looking for a JSON deliverable they have.
+                log(scoped.length === 0 && inlined.length > 0
+                    ? `::notice:: [mutate] --mutate-include/--mutate-exclude filtered out all ${inlined.length} inlined JSON artifact(s) — nothing left to perturb`
+                    : `::notice:: [mutate] ${explainNoMutations(cassette.artifacts ?? [], scoped)}`);
             else if (uncaught.length) {
-                log(`::warning:: [mutate] ${uncaught.length}/${plan.length} perturbation(s) CAUGHT BY NOTHING — these fields are unguarded:`);
+                log(`::warning:: [mutate] ${uncaught.length}/${plan.length} sampled perturbation(s) CAUGHT BY NOTHING — these fields are unguarded${scope}:`);
                 for (const u of uncaught)
                     log(`    ${u}`);
             }
             else
-                log(`::notice:: [mutate] all ${plan.length} perturbation(s) were caught — assertions cover every perturbed field`);
+                log(`::notice:: [mutate] all ${plan.length} sampled perturbation(s) were caught — assertions cover every perturbed field${scope}`);
         }
         // under --strict, EVERY staleness finding becomes a failing assertion (non-zero exit), not just a
         // warning. --fail-on-skill-drift is the narrower gate: only the skill-source classes fail (incl.
@@ -5087,6 +5220,7 @@ export async function replayCassette(cassette, hooks = [], opts = {}) {
         return assembleRunResult({
             turn: undefined, // replay reconstructs one recorded run; no multi-turn attribution
             command: "replay", // #48
+            mutation: mutationReport, // --mutate only; undefined otherwise
             // A replay is held to the lane the RECORDED scenario declared — the frozen contract, not the
             // replaying machine's. Absent on a cassette recorded before the axis existed ⇒ local.
             lane: cassette.scenario.lane,

package/dist/run/chat-result.js

@@ -143,6 +143,7 @@ export function buildChatResult(record, opts) {
         partial: undefined,
         unansweredGate: undefined,
         staleness: undefined,
+        mutation: undefined, // replay --mutate only
         skippedAssertions: undefined,
         outcome: undefined, // rollup of `verdict`; chat has none, so this is absent too
         verdict: undefined, // chat carries NO verdict (no assertions were evaluated) — left absent, never a vacuous {pass:true,...}

package/dist/run/execute.js

@@ -1351,6 +1351,7 @@ export async function executeScenario(scenario, opts = {}) {
             partial: undefined,
             unansweredGate: undefined,
             staleness: undefined,
+            mutation: undefined, // replay --mutate only
             skippedAssertions: undefined,
             outcome: undefined, // stamped alongside the verdict just below (derived from it)
             verdict: undefined, // computed just below (after assertions are evaluated / the object is fully assembled) and stored — see the comment there
@@ -1717,6 +1718,7 @@ export function buildPartialResult(args) {
         l0PluginDivergence: undefined,
         missingCapabilityUse: undefined,
         staleness: undefined,
+        mutation: undefined, // replay --mutate only
         skippedAssertions: undefined,
         outcome: undefined, // stamped alongside the verdict just below (derived from it)
         verdict: undefined, // computed just below (after every other field is assembled) and stored — see the comment there

package/dist/run/mutate.js

@@ -126,6 +126,42 @@ export function planMutationsWithStats(artifacts, opts) {
     }
     return { mutations, eligibleLeafCounts, truncatedTotal };
 }
+/** Glob matcher for `--mutate-include` / `--mutate-exclude`, over cassette-recorded artifact paths.
+ *
+ *  Deliberately its own, rather than reusing `analyze-skill`'s: that one is module-private, EXPANDS the
+ *  filesystem (`readdirSync`) rather than matching a string, accepts only `dir/*.ext`-shaped patterns,
+ *  and is case-insensitive. Artifact paths are recorded strings — matching them is a pure predicate, and
+ *  case is meaningful.
+ *
+ *  `*` matches within one path segment; `**` crosses segments (so `handoff/**` scopes a whole subtree,
+ *  which is the shape someone reaches for to exclude per-run internals). Every other character is
+ *  literal, including regex metacharacters. */
+export function matchesAnyGlob(path, patterns) {
+    return patterns.some((pattern) => globToRegExp(pattern).test(path));
+}
+function globToRegExp(pattern) {
+    // Split on the ** token first so it can be expanded to a cross-segment match; everything between is
+    // escaped literally, with single `*` limited to non-separator characters.
+    const body = pattern
+        .split("**")
+        .map((part) => part.replace(/[.+?^${}()|[\]\\]/g, "\\$&").replace(/\*/g, "[^/]*"))
+        .join(".*");
+    return new RegExp(`^${body}$`); // case-SENSITIVE by construction
+}
+export function summarizeMutationPlan(stats, opts) {
+    const perFile = opts?.maxPerFile ?? DEFAULT_MAX_PER_FILE;
+    const total = opts?.maxTotal ?? DEFAULT_MAX_TOTAL;
+    const eligible = Object.values(stats.eligibleLeafCounts).reduce((a, b) => a + b, 0);
+    const takenPerFile = new Map();
+    for (const m of stats.mutations)
+        takenPerFile.set(m.file, (takenPerFile.get(m.file) ?? 0) + 1);
+    let filesAtPerFileCap = 0;
+    for (const [file, count] of takenPerFile)
+        if (count >= perFile && (stats.eligibleLeafCounts[file] ?? 0) > count)
+            filesAtPerFileCap++;
+    const truncatedBy = stats.mutations.length >= total && eligible > stats.mutations.length ? "total" : filesAtPerFileCap > 0 ? "per-file" : null;
+    return { sampled: stats.mutations.length, eligible, truncatedBy, filesAtPerFileCap, caps: { perFile, total } };
+}
 /** Pure planner: decides which leaves across a set of artifacts are worth perturbing, bounded by
  *  maxPerFile/maxTotal (defaults 10/50 — a full assertion-suite re-run per mutation, so cost is linear
  *  in the length of this list). Convenience wrapper over planMutationsWithStats for callers that don't

package/dist/scan.js

@@ -91,6 +91,14 @@ export const KNOWN_BUILTIN_AGENTS = new Set([
     "Plan",
     "statusline-setup",
 ]);
+/** Skills the AGENT itself ships, present in a clean recording regardless of what a scenario mounts.
+ *  Observed in every committed fixture across the sealed `container` tier (where no host config dir is
+ *  reachable) and `hostloop`, and confirmed in the staged agent ELF — so a bare name outside this set
+ *  came from the operator's own config dir, not the product.
+ *
+ *  Closed set, same contract as KNOWN_BUILTIN_AGENTS: extend it deliberately when the agent's built-in
+ *  roster changes. A new built-in would surface on the first fresh recording after a `sync`. */
+export const KNOWN_BUILTIN_SKILLS = new Set(["deep-research"]);
 /** `account` keys that identify the OPERATOR. A clean recording's account block is `{tokenSource,
  *  apiProvider}` only. `email` is usually redacted upstream by the time it reaches here — the load-bearing
  *  members are `organization` and `subscriptionType`, which no redaction rule touches. */
@@ -109,13 +117,17 @@ const ACCOUNT_IDENTITY_KEYS = ["email", "organization", "subscriptionType"];
  * necessarily one the scenario attached on purpose via `mcp.config` — a documented, supported feature —
  * and flagging it would red CI on a legitimate fixture.
  */
-export function scanHostInventory(decoded, where, allow) {
+export function scanHostInventory(decoded, where, allow, declaredPlugins = []) {
     const out = [];
     const norm = allow.map(normAllow);
     const push = (sample, detail) => {
         if (!allowed(sample, HOST_INVENTORY_CLS, norm))
             out.push({ where: `${where} ${detail}`, cls: HOST_INVENTORY_CLS, sample });
     };
+    // Plugin names the SCENARIO mounted. `declaredPlugins` covers payloads with no sibling `plugins[]` —
+    // notably the registry `control_response`, whose agents arrive as `{name,description,model}` with no
+    // plugin list beside them; the caller harvests those names from the events array and passes them here.
+    const declared = new Set(declaredPlugins);
     const visit = (node) => {
         if (Array.isArray(node)) {
             for (const v of node)
@@ -151,13 +163,58 @@ export function scanHostInventory(decoded, where, allow) {
                 if (acct[k] !== undefined)
                     push(k, `account.${k}`);
         }
-        // A4 — an agent outside the built-in roster.
-        if (Array.isArray(o.agents)) {
-            for (const a of o.agents) {
-                const name = typeof a === "string" ? a : a?.name;
-                if (typeof name === "string" && !KNOWN_BUILTIN_AGENTS.has(name))
+        // A4 — an agent outside the built-in roster, EXCEPT one the scenario itself mounted.
+        //
+        // A plugin contributes its agents to the roster, and at `hostloop` — the tier a fleet actually
+        // records at — that roster is the fixture, not the recording machine's inventory. This is the same
+        // carve-out A1 makes for an `mcp.config`-attached server, and without it every plugin-with-agents
+        // consumer hits a wall of false positives whose only documented remedy is inventing a regex.
+        //
+        // The provenance is in the payload already: `plugins[]` sits beside `agents[]` in the same init
+        // object, and a plugin's agents are namespaced `<plugin>:<agent>`. So the subtraction is derivable
+        // at scan time from an ALREADY-RECORDED cassette — no new cassette field, and no re-record.
+        //
+        // Deliberate residual: a host-leaked plugin's agents are namespaced too, so this trades a narrow
+        // false-negative for the false-positive. That is the right trade — this check exists to catch a
+        // FOREIGN machine's inventory, and an agent the scenario mounted is by construction not that.
+        //
+        // A5 — a SKILL outside the agent's built-ins, same two exemptions. This axis matters because at
+        // `protocol` with local OAuth the harness keeps the operator's REAL CLAUDE_CONFIG_DIR (a fresh one
+        // breaks OAuth — see src/runtime/protocol.ts), so the personal skills installed there are
+        // discoverable and would be frozen into a committed fixture. A skill name is inventory in exactly
+        // the way an MCP server name is: it says what the operator has installed.
+        if (Array.isArray(o.agents) || Array.isArray(o.skills)) {
+            // Plugins declared by THIS payload, plus any harvested from elsewhere in the stream.
+            const local = new Set(declared);
+            if (Array.isArray(o.plugins))
+                for (const p of o.plugins) {
+                    const n = typeof p === "string" ? p : p?.name;
+                    if (typeof n === "string")
+                        local.add(n);
+                }
+            /** A `<plugin>:<name>` whose plugin the recording declares is the fixture, not the host's. */
+            const fromDeclaredPlugin = (name) => {
+                const sep = name.indexOf(":");
+                return sep > 0 && local.has(name.slice(0, sep));
+            };
+            if (Array.isArray(o.agents))
+                for (const a of o.agents) {
+                    const name = typeof a === "string" ? a : a?.name;
+                    if (typeof name !== "string" || KNOWN_BUILTIN_AGENTS.has(name))
+                        continue;
+                    if (fromDeclaredPlugin(name))
+                        continue;
                     push(name, "agents[]");
-            }
+                }
+            if (Array.isArray(o.skills))
+                for (const s of o.skills) {
+                    const name = typeof s === "string" ? s : s?.name;
+                    if (typeof name !== "string" || KNOWN_BUILTIN_SKILLS.has(name))
+                        continue;
+                    if (fromDeclaredPlugin(name))
+                        continue;
+                    push(name, "skills[]");
+                }
         }
         for (const v of Object.values(o))
             visit(v);

package/docs/cassette.md

@@ -541,11 +541,32 @@ scenario's `assert:` block; on the live path it would fail as an empty assertion
 
 ### Mutation coverage — `replay --mutate`
 
-`replay --mutate` perturbs each recorded, inlined JSON artifact value in turn, re-runs the scenario's
+`replay --mutate` perturbs recorded, inlined JSON artifact values one at a time, re-runs the scenario's
 assertions against the perturbed tree, and reports which perturbations nothing catches — those are the
 fields your `assert:` block leaves unguarded. Each perturbation is applied, evaluated, and restored
 before the next one runs, so the materialized tree ends the pass unchanged.
 
+> **It samples. Read the ratio accordingly.** The plan is capped at **10 values per file and 50 in
+> total** — the per-file cap is applied first, so a corpus of N JSON artifacts yields at most `10 × N`
+> perturbations no matter how much total budget is left. A line reading `50/50 … CAUGHT BY NOTHING` means
+> 50 of the *sampled* values, not 50 of your fields, and aggregating such lines across many cassettes
+> produces a number that describes the sample rather than the corpus. Whenever a cap binds, the report
+> appends the eligible total and names the cap that bound — `(sampled 30 of 120 eligible value(s);
+> per-file cap 10 reached on 3 file(s))` — and omits that note entirely when nothing was truncated, so
+> its absence means the counts are the whole truth. `--output-format json` carries the same facts under
+> `mutation` (`sampled` / `eligible` / `truncatedBy` / `caps` / `uncaught`), which is the right surface to
+> aggregate over.
+
+**Scope it, and raise the cap that binds.** `--mutate-include <glob>` / `--mutate-exclude <glob>` (both
+repeatable, exclude applied last) restrict which artifact paths are perturbed — `*` stays inside a path
+segment, `**` crosses them, so `--mutate-exclude 'handoff/**'` drops per-run internals nobody should
+assert on and spends the sample on deliverables instead. Filtering happens before planning, so an
+out-of-scope artifact is absent from `eligible` too: the report describes the scope you asked for rather
+than counting what you deliberately excluded as missed. `--mutate-max-per-file <n>` / `--mutate-max-total
+<n>` raise the caps; reach for the per-file one first, since it is applied first and with a handful of
+artifacts the total is never the binding constraint. Cost is linear — one full assertion re-run per
+perturbation.
+
 This is coverage reporting, not verdict reporting: an uncaught perturbation is a gap in the scenario's
 assertions, not a failure of the run, so `--mutate` never changes replay's verdict or exit code — a green
 replay stays green regardless of what it finds.
@@ -800,9 +821,25 @@ counts) — committed PII surface. Two layers, distinct from secret-scrub (which
   token is ever written and `grep mcp__` over the cassette reads clean. The inventory lives in **name
   fields**. So this check reads specific name fields of the decoded events and flags: an `mcp_servers[].name`
   outside the harness's own servers, a `mcp__<server>__…` tool naming a foreign server, `account.email` /
-  `.organization` / `.subscriptionType`, and an `agents[]` entry outside the built-in roster. Suppress with
+  `.organization` / `.subscriptionType`, an `agents[]` entry outside the built-in roster, and a `skills[]`
+  entry outside the agent's own built-ins. The skills axis is not theoretical: at `protocol` with local
+  OAuth the harness keeps the operator's **real `CLAUDE_CONFIG_DIR`** (a fresh one breaks OAuth), so the
+  personal skills installed there are discoverable, and a skill name says what you have installed exactly
+  as an MCP server name does. Suppress with
   `--allow-host-inventory <regex>`; if the flagged name is a genuine Cowork server, add it to
   `KNOWN_COWORK_SERVERS` instead of allowing it.
+  **A plugin the scenario mounted is not host inventory.** Its agents join the roster — at `hostloop`,
+  that roster *is* the fixture — so an agent namespaced `<plugin>:<agent>` whose plugin the same recording
+  declares in `plugins[]` is exempt, exactly as an `mcp.config`-attached server is. The provenance comes
+  from the cassette itself, so this applies to recordings you already have; no re-record, and no allow
+  regex to invent. A foreign agent, or one namespaced to a plugin the run never mounted, still flags. The
+  same exemption covers `skills[]`.
+  **`plugins[].name` is deliberately NOT an axis.** That array is the harness's own declaration channel —
+  every entry arrives from a `--plugin-dir` the harness passed, and the settings it writes carry an
+  explicit `enabledPlugins` allowlist. Across every committed fixture, including `protocol` and `hostloop`
+  recordings made on a machine with many host plugins installed, it holds only the scenario's own. A field
+  that carries nothing but declarations would produce false positives and catch nothing — the failure this
+  class has already had once.
   **`record` carries its own preflight for the same risk.** Recording at a host-inheriting tier
   (`protocol`, `hostloop`, or `cowork` resolving to hostloop) into a repo-visible cassette path is
   refused by default — that recording would freeze this machine's MCP servers, agents, and account
@@ -874,6 +911,13 @@ leak/drift":
 |---|---|---|
 | `0` | clean | none |
 | `1` | **verification RAN and found a real problem** | `findings[]` (a PII/privacy match), `staleness[]` (a genuine, non-`unverifiable-*` drift finding), and/or `scenarioDrift[]` |
+
+> **Triage first, then read the rows.** Text output opens with a per-class count —
+> `findings by class: host-inventory 240` — before the per-file listing. A sweep that surfaces hundreds
+> of findings of one class reads as hundreds of separate problems until you know that; the header says
+> what kind and how many in one line. It is additive: every per-file row still prints, because which
+> file carries which finding is the answer a per-file audit exists to give. JSON consumers already have
+> `findings[].cls` and need no rollup.
 | `2` | usage error (e.g. `--skip-privacy`+`--skip-staleness` together, zero cassettes under a dir) | n/a |
 | `3` | **verification could NOT complete** | `unverifiable[]` (an `unverifiable-*`-class staleness finding), `version[]` (the cassette is from a newer harness than this one understands), and/or `error` (a malformed/unreadable cassette, or a per-file crash) |
 

package/docs/debugging.md

@@ -19,7 +19,7 @@ Docker, no re-record.
 | Situation | Symptom | Reach for (in order) |
 |---|---|---|
 | **The skill misbehaved** | wrong output, an unexpected gate, a denied tool, an opaque crash | `inspect` — what did it produce? · `trace <run-dir> --view <view>` — what did it actually do (tools, gates, sub-agent tree)? · `verify-run` — re-assert cheaply when only an assertion is wrong · `diff <old-run> <new-run>` — what changed since it worked · `chat` — reproduce it by hand |
-| **A green you don't trust** | an assert that may have tested nothing, a stale cassette, an auto-answered or decided gate | `replay --explain` — the evidence trail behind each *passing* assert · `replay --mutate` — perturbs each recorded JSON artifact value and reports which perturbations NOTHING caught (reporting only; never moves the verdict/exit code) · `lint` — assertions on the wrong CI lane / mixed-class keys · `verify-cassettes` — privacy + staleness over committed cassettes · the Gotchas landmine catalog — how a check passes vacuously · `run --repeat N` / `skill --repeat N` — did it pass, or pass once? · `stats` — flaky or expensive over time |
+| **A green you don't trust** | an assert that may have tested nothing, a stale cassette, an auto-answered or decided gate | `replay --explain` — the evidence trail behind each *passing* assert · `replay --mutate` — perturbs a CAPPED SAMPLE of recorded JSON values (10/file, 50 total) and reports which perturbations NOTHING caught; the report names the sample size, so read it as a sample not a total (reporting only; never moves the verdict/exit code) · `lint` — assertions on the wrong CI lane / mixed-class keys · `verify-cassettes` — privacy + staleness over committed cassettes · the Gotchas landmine catalog — how a check passes vacuously · `run --repeat N` / `skill --repeat N` — did it pass, or pass once? · `stats` — flaky or expensive over time |
 
 A failed run also records `errorSource` (where the failure originated) and `stderrLogPath` (the captured
 agent stderr) — read those before re-running; a re-record rarely tells you more than the captured stderr

package/examples/replays/README.md

@@ -16,7 +16,7 @@ DOES exercise a real gate exchange, see `example-multiselect-gate.cassette.json`
 
 Run it with:
 
-> Assumes the `cowork-harness` CLI is available — from a source checkout run `npm ci && npm run build && npm link` first, or `npm i -g "cowork-harness@>=1.18.0"`. (`replay` itself needs nothing else — no token, no Docker.)
+> Assumes the `cowork-harness` CLI is available — from a source checkout run `npm ci && npm run build && npm link` first, or `npm i -g "cowork-harness@>=1.19.0"`. (`replay` itself needs nothing else — no token, no Docker.)
 
 ```sh
 cowork-harness replay examples/replays/example-pdf-skill.cassette.json

package/llms.txt

@@ -30,7 +30,7 @@ It is a fidelity fixture, not the Desktop runtime. The CLI binary is `cowork-har
 - [docs/discovery.md](docs/discovery.md): how marketplaces / plugins / skills / MCP are discovered
 - [docs/plugin-root.md](docs/plugin-root.md): `${CLAUDE_PLUGIN_ROOT}` resolves differently host-side vs in-VM bash — the single most common Cowork authoring footgun
 - [docs/cassette.md](docs/cassette.md): record/replay cassettes and the token-free PR gate
-- `replay --mutate` perturbs each recorded JSON artifact value, re-runs the assertions, and reports which perturbations NOTHING caught — i.e. which fields your assertions leave unguarded. Reporting only: it never moves the verdict or the exit code, so it is safe to add to a gate.
+- `replay --mutate` perturbs recorded JSON artifact values, re-runs the assertions, and reports which perturbations NOTHING caught — i.e. which fields your assertions leave unguarded. **It samples: at most 10 values per file and 50 in total**, so `N/N caught by nothing` is N of the SAMPLE, never of your whole corpus. When a cap binds, the report says which one and how many values were eligible; `--output-format json` carries the same under `mutation`. Reporting only: it never moves the verdict or the exit code, so it is safe to add to a gate.
 - Host-inventory consent — recording a host-inheriting tier (`protocol`, `hostloop`, or `cowork` resolving to hostloop) into a repo-visible path freezes the recording machine's own MCP servers / agents / account into the fixture, so `record` refuses without `--allow-host-inventory-fixture`; `verify-cassettes --allow-host-inventory <regex>` suppresses a single such finding on an already-committed cassette. The two are distinct flags on distinct commands and do not interchange.
 - [docs/decider-dir.md](docs/decider-dir.md): in-band gate answering by a driving agent
 - [docs/maintenance.md](docs/maintenance.md): keeping parity across Claude Desktop releases (`sync`)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cowork-harness",
-  "version": "1.18.0",
+  "version": "1.19.0",
   "description": "Scriptable, CI-friendly harness for Claude Cowork's runtime contract for testing skills across scenarios — same agent, mounts, egress allowlist, permission protocol, and sandbox limitations.",
   "license": "MIT",
   "type": "module",

package/schema/run-result.json

@@ -1406,6 +1406,46 @@
           }
         }
       }
+    },
+    "mutation": {
+      "type": "object",
+      "description": "Replay lane only, and only under --mutate: which recorded values were perturbed and which perturbations no assertion caught. `sampled` is post-cap; `eligible` is the pre-cap total, so a consumer can distinguish 'N unguarded fields' from 'N unguarded OUT OF eligible'. `truncatedBy` names the cap that bound \u2014 raising the other one would not change the sample.",
+      "properties": {
+        "sampled": {
+          "type": "number"
+        },
+        "eligible": {
+          "type": "number"
+        },
+        "truncatedBy": {
+          "type": [
+            "string",
+            "null"
+          ],
+          "enum": [
+            "per-file",
+            "total",
+            null
+          ]
+        },
+        "caps": {
+          "type": "object",
+          "properties": {
+            "perFile": {
+              "type": "number"
+            },
+            "total": {
+              "type": "number"
+            }
+          }
+        },
+        "uncaught": {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        }
+      }
     }
   }
 }