cowork-harness 1.8.0 → 1.9.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 1.8.0
-  tracks-harness: cowork-harness 1.8.0 (baseline desktop-1.24012.1)
+  version: 1.9.0
+  tracks-harness: cowork-harness 1.9.0 (baseline desktop-1.24012.1)
 ---
 
 # cowork-harness
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.8.0` (baseline
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.9.0` (baseline
 > `desktop-1.24012.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
@@ -39,16 +39,28 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.8.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.8.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=1.8.0"`. **Pin `@>=1.8.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.9.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.9.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=1.9.0"`. **Pin `@>=1.9.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
-  What the ≥ 1.8.0 floor gates, by release:
+  What the ≥ 1.9.0 floor gates, by release:
+
+  - **1.9.0 (critique report schema + shareable, screenshot-safe output):**
+    `schema/critique-report.json` (descriptive, test-pinned — parse the report against it, not prose;
+    NOT §12-frozen while critique is EXPERIMENTAL), `gradedSkill` in the report (text header + JSON;
+    multi-skill pairing key — pair by `(gradedSkillHash, gradedSkill)`, since the hash alone keys the whole
+    plugin and cross-pairs its skills). Critique's text report and stderr diagnostics now **collapse host
+    paths to `~`** (a shared report or screenshot no longer leaks the username + filesystem layout; the JSON
+    report and persisted-artifact paths stay raw for machine consumers), a **missing or non-directory skill
+    folder is refused pre-spawn** (fast exit 2, no stray run dir left behind), and the report header + stderr
+    prefixes now read **`critique:`** (were `skill-critique:`).
 
   - **1.8.0 (critique hardening + hostloop unpin):** `critique --skill <name>` (multi-skill-plugin
     grading — a plugin root with no `--skill` is refused pre-spend; the evidence package gains the
     invoked skill's `agents/<name>.md` + bounded `references/*.md` CONTENT), run-dir artifacts
     (`critique-report.json` always, `critique-evidence-package.txt` when the evaluator ran,
     `critique-salvage.json` on exit 2) + `--out <path>`, per-critique `costUsd` across all four
-    workloads, `findingFingerprint` per item (cross-INPUT clustering; skillHash stays the cross-FIX
+    workloads, `findingFingerprint` per item (cross-INPUT clustering; HIGH-precision
+    LOW-recall — a match proves reproduction, a mismatch does NOT prove non-reproduction, the same
+    finding reworded fingerprints differently; skillHash stays the cross-FIX
     key), gate-answer `--answer` echo in the report, per-item-tolerant evaluator parse
     (`droppedEvaluatorItems`), `skillMdTruncated` (a readable-but-oversized SKILL.md is flagged
     "graded a cut copy" — distinct from missing/unreadable), `subagents[].webSearches` +
@@ -436,7 +448,9 @@ cassette — has its own recipe:
    input + result are captured, not just errored ones. When iterating, tag generations with `--label` and
    pair a critique only with a `result.json` whose `fingerprint.skillHash` **matches** the skill that
    produced it (`inspect`/the run-index row surface a short `skillHash` prefix; `verify-run` warns when a
-   kept run predates the current skill). **On a pre-1.5.0 CLI the `skill` lane emits no `fingerprint.skillHash` at all**, so a
+   kept run predates the current skill). **Multi-skill plugin caveat (post-1.7.0 CLIs with `--skill`):**
+   skillHash keys the whole MOUNTED plugin, so on a multi-skill plugin the hash alone cross-pairs
+   critiques of DIFFERENT skills — pair by the report's `(gradedSkillHash, gradedSkill)` pair. **On a pre-1.5.0 CLI the `skill` lane emits no `fingerprint.skillHash` at all**, so a
    pairing step there silently groups on an absent key instead of erroring — check the field is present, or
    require ≥ 1.5.0. See `docs/debugging.md`
    (repo-only) for the full loop.

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 1.8.0` (baseline `desktop-1.24012.1`).
+Self-contained reference. Tracks `cowork-harness 1.9.0` (baseline `desktop-1.24012.1`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -13,7 +13,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 ```
 
 The Action's `version` input defaults to `latest` — intentional so a copy-pasted recipe tracks the current
-release; pin an exact version (e.g. `version: "1.8.0"`) for reproducible CI.
+release; pin an exact version (e.g. `version: "1.9.0"`) for reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -57,7 +57,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@>=1.8.0"
+- run: npm i -g "cowork-harness@>=1.9.0"
 - run: cowork-harness lint scenarios/*.yaml          # no silent false-greens
 - run: cowork-harness verify-cassettes cassettes/    # privacy + staleness
 - run: cowork-harness replay cassettes/              # token-free content/structure
@@ -197,7 +197,7 @@ jobs:
         with: { node-version: '20' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@>=1.8.0"
+      - run: npm i -g "cowork-harness@>=1.9.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -226,7 +226,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@>=1.8.0"
+        run: npm i -g "cowork-harness@>=1.9.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 1.8.0` (baseline `desktop-1.24012.1`).
+Self-contained reference. Tracks `cowork-harness 1.9.0` (baseline `desktop-1.24012.1`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,6 +1,6 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.8.0`
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.9.0`
 (baseline `desktop-1.24012.1`). If your checkout is newer, prefer the live `docs/scenario.md`,
 `docs/session.md`, and `SPEC.md`.
 

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 1.8.0` (baseline `desktop-1.24012.1`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 1.9.0` (baseline `desktop-1.24012.1`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.
 

package/CHANGELOG.md

@@ -6,6 +6,85 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [1.9.0] — 2026-07-24
+
+### Added
+
+- **`schema/critique-report.json`** — a descriptive, test-pinned schema for `critique`'s JSON report /
+  `critique-report.json` artifact, so automation consumers (budget pacers gating on `costUsd.complete`,
+  harvesters) parse field names/shapes from a schema instead of prose. Deliberately **not** a SPEC
+  §12-frozen surface (unlike `doctor.json`) — critique is EXPERIMENTAL and additive field changes are
+  expected; the schema says so in its own description, and a two-way sync test pins it against the
+  actual report builder on every branch (findings / infraFailure / evaluatorError).
+- **`gradedSkill` in the critique report** (text header + JSON): the resolved `skills/<name>` the
+  packager graded under `--skill`/auto-selection. Load-bearing for multi-skill plugins:
+  `gradedSkillHash` keys the whole mounted plugin, so pairing by hash alone cross-pairs critiques of
+  *different* skills — pair by `(gradedSkillHash, gradedSkill)`. Docs updated accordingly.
+
+### Docs
+
+- Truncation→DROPPED mechanics: a finding whose `evidence` quotes SKILL.md text past the packaging cap
+  fails citation-resolution and lands in DROPPED (the check runs against the *cut* copy) — documented
+  next to `skillMdTruncated` so a back-half DROPPED skew on an oversized skill has its cause named.
+- `findingFingerprint` direction-of-inference: high-precision, LOW-RECALL — a match proves
+  reproduction; a mismatch does NOT prove non-reproduction (the same finding reworded fingerprints
+  differently). The Reproduction section says so before anyone concludes "didn't reproduce".
+- Evaluator cost share: the two evaluator passes were ~3/4 of a measured e2e total — the
+  calibrate-then-`--evaluator-model` strategy is now in the cost section and `critique --help`, with
+  the armor-verification-is-default-evaluator-only caveat.
+- SPEC §12 now names the critique report explicitly under **NOT covered**: `schema/critique-report.json`
+  is descriptive (parse against it, not prose) but not the compatibility contract while critique is
+  EXPERIMENTAL — its surface-baseline presence is for change visibility, and it is the promotion
+  candidate on the `doctor.json` template once critique stabilizes. README, llms.txt, and the shipped
+  skill point at the schema and the `(gradedSkillHash, gradedSkill)` pairing rule.
+
+### Fixed
+
+- **critique's "Attached inputs" evidence no longer reports `(none)` when `mounts.json` is corrupt.**
+  `listAttachedInputs` derived connected-folder names from `loadVmPathContext`, which returns `null` for
+  BOTH an absent `mounts.json` (legitimately no mounts) and a present-but-unparseable one (the folder map
+  is UNKNOWN). Uploads already distinguished these (the ENOENT-vs-read-fault split), but folders — which
+  have no fixed-layout fallback — silently collapsed to `[]`, so a corrupt `mounts.json` rendered `(none)`,
+  telling the evaluator "the agent correctly saw no connected folder" when the truth was unknown. It now
+  surfaces a corrupt `mounts.json` as an explicit UNKNOWN note, completing the same
+  confabulation-vs-correct guard the uploads path already applies.
+- **`diff` no longer treats two tool inputs that differ only past the ~2000-char cap as the same call.**
+  `canonicalizeInput` truncated a tool input's canonical JSON to a 2000-char cap and used that truncated
+  string as the tool-sequence equality key, so two `Write`/`Bash` calls sharing a long identical prefix
+  but differing only in the dropped tail compared as `op: "same"` in `diffToolSequence` — a false "no
+  change" that could flip the advisory `diff` exit code to 0. A truncated key now folds in a
+  `#<len>·<sha16>` hash of the full canonical string, so the key depends on the entire content while the
+  visible prefix stays readable in the hunk; both diff sides canonicalize identically, so the comparison
+  stays consistent.
+- **`COWORK_VM_GATEWAY` is now validated as a canonical IPv4 literal.** The L2-microVM gateway override was
+  interpolated verbatim into a root-run guest `iptables -A OUTPUT -d <gateway>` command (via `sh -c`), so a
+  malformed or hostile value could inject shell syntax into privileged provisioning — or, more mundanely,
+  leave the firewall in an unknown state. `vmGatewayIp()` now rejects anything that isn't a canonical IPv4
+  literal (digits-and-dots only, octets 0–255, no leading zeros), failing loud instead of reaching the
+  shell. Operator-set env var, so this is defense-in-depth; no valid gateway value is affected.
+- **The `agent.stderr.log` sink is now flushed before the teardown secret-scrub reads it.** The stderr sink
+  was piped fire-and-forget and never awaited, so bytes still buffered when `scrubRawRunLogs` read the file
+  could land raw *afterwards* — a persisted-secret leak in a narrow teardown window. `LiveAgentSession` now
+  pipes it with `{ end: false }` and ends+awaits it in the same session-teardown drain that already flushes
+  `events.jsonl` / `control-out.jsonl`, so the session generator resolves only after the sink is fully
+  flushed — the scrub always sees the complete log.
+- **`critique` no longer prints raw host paths in its report or diagnostics.** The text report's `run dir:`
+  line, the `inspect <dir>` hints, the write-failure diagnostics, and the echoed skill-folder path all
+  printed absolute `$HOME`-rooted paths, so a shared report or screenshot leaked the username + filesystem
+  layout (it landed in a video frame). critique was the one rendering path in the CLI that never called
+  `tildeify`, while `skill`/`run` scrub unconditionally. Every human-facing path is now collapsed to `~`;
+  the JSON report and persisted-artifact paths stay raw (machine data a consumer feeds back to a tool). The
+  `--demo` rejection is unchanged — it was never the fix — but its reason now notes the report already
+  collapses paths.
+- **`critique` fails fast on a missing or non-directory skill folder.** A typo'd/absent positional folder
+  previously minted a session and spawned the task turn before infra-failing (exit 2), leaving a stray run
+  dir behind. `resolveCritiquedSkillDir` now `existsSync`/`isDirectory`-checks the folder up front — before
+  any session is minted or spawned — so a bad path exits 2 immediately with nothing left on disk. A
+  present-but-`SKILL.md`-less folder still defers to the packager's degraded flow, unchanged.
+- **`critique`'s report header and stderr diagnostics now say `critique:` (were `skill-critique:`).** A
+  leftover label from the `scripts/skill-critique.ts` instrument; the invoked command is `critique`.
+  Cosmetic, no schema change.
+
 ## [1.8.0] — 2026-07-23
 
 ### Added

package/README.md

@@ -91,7 +91,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
 
 > **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
 > <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
-> From a global install (`npm i -g "cowork-harness@>=1.8.0"`), point at the package root instead:
+> From a global install (`npm i -g "cowork-harness@>=1.9.0"`), point at the package root instead:
 > `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
 > (or copy the cassette into your own project and pass that path).
 
@@ -101,7 +101,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
 > - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
 > - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
 > - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
-> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=1.8.0"`.
+> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=1.9.0"`.
 
 Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
 test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
@@ -126,7 +126,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
 claude plugin install cowork-harness@cowork-harness
 ```
 
-The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=1.8.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
+The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=1.9.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
 
 It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
 
@@ -147,7 +147,7 @@ A global install is enough for CI `lint`, reading the teaching skill, and replay
 To `run` the worked examples live or copy them as a starting point, use a source checkout. (The marketplace
 skill install itself only pulls `.claude/skills/cowork-harness/` — SKILL.md + `references/` + `scenario.py`/
 assertion keys, per `.claude-plugin/marketplace.json`'s `source` — not the rest of this table; the full set
-above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=1.8.0"` — see
+above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=1.9.0"` — see
 [above](#drive-it-from-claude-code-companion-skill) — which pulls the same npm package as the global-install row.)
 
 ### Prerequisites for anything above `protocol` fidelity
@@ -690,7 +690,7 @@ jobs:
           anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
 ```
 
-Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — intentional so recipes track the current release; pin an exact version for reproducible CI. The companion skill's `cowork-harness@>=1.8.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). See [`action.yml`](./action.yml) for the full input reference.
+Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — intentional so recipes track the current release; pin an exact version for reproducible CI. The companion skill's `cowork-harness@>=1.9.0` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). See [`action.yml`](./action.yml) for the full input reference.
 
 The provided [GitHub Actions workflow](.github/workflows/ci.yml) runs a **seven-stage pipeline**. The **build** + **test** stages are the token-free gate you can copy into your skill repo; the `action-self-test`, `python`, `boundary`, `scenarios`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):
 
@@ -752,7 +752,7 @@ Most runs need **none** of these — the defaults are correct. They're grouped b
 - **Staleness boundary** (the git-tracked-files-only default and its `git add`/hard-fail rationale are explained in [Test a local skill](#test-a-local-skill-in-one-command)): a **non-repo** source dir falls back to a raw walk instead; OS-junk like `.DS_Store` is always excluded; a partial exclusion emits a `::notice:: [stage]`. `COWORK_HARNESS_GITSET=0` opts out to a raw walk for every dir (and copies untracked files too); `COWORK_HARNESS_DEBUG_SKILLHASH=1` dumps the exact file set feeding the staleness hash on a mismatch (and flags OS-junk) so a drift source is one line. Declare per-plugin non-runtime paths in a `.cowork-hashignore` file (or the session `staleness.hash_ignore`). `COWORK_HARNESS_AGENT_SCOPE=skill` (opt-in) refines a scenario's `skills:` scope so a **skill-named** `agents/<name>.md` re-stales only that skill's cassettes instead of the whole fleet (generic agents stay shared; stamped into the fingerprint, so flipping it is a one-time re-record like `GITSET`).
 - **`skill` / `chat` defaults:** `COWORK_HARNESS_FIDELITY` sets the default fidelity tier for ad-hoc `skill`/`chat` runs (a `--fidelity` flag or a scenario's `fidelity:` still wins); `COWORK_HARNESS_MODEL` sets the default model; `COWORK_HARNESS_OUTPUT_FORMAT` (`text`|`json`) sets the default output format. Each is overridden by the matching explicit flag. **Caveat:** `chat` only accepts `protocol`/`container`/`hostloop` — `microvm` and `cowork` are rejected (no Lima/auto-pick plumbing in the interactive REPL), so a `COWORK_HARNESS_FIDELITY` set to either is rejected loudly for `chat` even though `skill` accepts the full tier set.
 - **Secret scrubbing:** `COWORK_HARNESS_SCRUB_KEYS=<KEY1,KEY2>` adds extra env-var names whose values are redacted from logs (beyond the known auth tokens + `ANTHROPIC_CUSTOM_HEADERS`); `COWORK_HARNESS_SCRUB_VALUES=<v1,v2>` redacts literal values regardless of env. **Committed-cassette redaction:** `COWORK_HARNESS_REDACT_PATTERNS=<rx1,rx2>` / `COWORK_HARNESS_REDACT_KEYS=<k1,k2>` extend the privacy layer that scrubs recorded `controlOut` before a cassette is written for commit.
-- L2 microVM: `COWORK_VM_GATEWAY` overrides the Lima host-proxy gateway IP (default `192.168.5.2`); `COWORK_VM_PROXY_PORT` pins the egress-proxy port (unset, the host binds an OS-assigned free port and threads that same value into the guest firewall + `HTTP(S)_PROXY`). The Lima instance is named `cowork-vm-<config-hash>` (a config change → a fresh VM); `COWORK_LIMA_INSTANCE` pins a fixed name, and `vm prune` removes orphaned ones.
+- L2 microVM: `COWORK_VM_GATEWAY` overrides the Lima host-proxy gateway IP (default `192.168.5.2`; must be a canonical IPv4 literal — an invalid value is rejected, since it is interpolated into the guest firewall rule); `COWORK_VM_PROXY_PORT` pins the egress-proxy port (unset, the host binds an OS-assigned free port and threads that same value into the guest firewall + `HTTP(S)_PROXY`). The Lima instance is named `cowork-vm-<config-hash>` (a config change → a fresh VM); `COWORK_LIMA_INSTANCE` pins a fixed name, and `vm prune` removes orphaned ones.
 - **Advanced / internal escape hatches** (rarely needed): `PYTHON` overrides the interpreter for `lint` / scenario tooling (default `python3`); `COWORK_HARNESS_DEBUG=1` surfaces which `.env` files were loaded; `COWORK_HARNESS_CLAUDE_BIN=<path>` points the `--decider-llm` transport at a specific `claude` binary; `COWORK_HARNESS_ALLOW_AGENT_FALLBACK=1` lets the harness use the newest sibling agent binary when the baseline-pinned version is missing (a fidelity compromise — off by default) — a same-major.minor **patch** bump of the staged NATIVE binary is auto-accepted without this flag (the native binary carries no sha256 pin, so a patch drift is safe by default; it prints a loud stderr note naming the pinned and substituted versions). At `hostloop`, and at `cowork` **only when it resolves to host-loop** on the synced baseline, the staged **VM ELF** is auto-accepted on a patch bump too, because on that path it's a non-executed parity mount into the bash sidecar; a `cowork` baseline that resolves to VM-loop instead executes the ELF directly, so it keeps the strict sha-pinned exact-version match, same as `container`/`microvm`, which always keep it (the ELF is the executed agent there), so the flag remains required for any ELF drift on those tiers/paths, and for a major/minor gap everywhere; `COWORK_MANAGED_CONFIG=1` forces the managed-config path on `protocol`; `COWORK_HARNESS_ALLOW_MISSING_PROMPT=1` downgrades a missing prompt asset to a warning; `COWORK_HARNESS_SAFE_STAGING_PREFIX=<a,b>` whitelists prefixes under which a delete in `outputs/` is allowed (otherwise delete-in-outputs fails loud); `COWORK_HARNESS_NO_HEARTBEAT=1` disables the idle-run heartbeat, `COWORK_HARNESS_HEARTBEAT_MS` tunes its interval (default 30000ms / 30s); `COWORK_HARNESS_CLI=/path/to/cli.js` overrides which built CLI the Python `cowork` pytest lane drives (see python/README.md).
 - Pin `baseline: desktop-<ver>` and `model:` in a session for byte-stable runs; use `latest` to track.
 
@@ -801,7 +801,7 @@ This repo is built to be driven by agents, not just read by humans:
 - **[AGENTS.md](./AGENTS.md)** — the canonical agent-instructions file (architecture seams, the build gate, invariants, ethos). Read it before changing code. Also indexed in **[llms.txt](./llms.txt)**.
 - **Companion skill** — [`.claude/skills/cowork-harness/`](./.claude/skills/cowork-harness/SKILL.md) teaches an agent to drive the harness; install it via the marketplace (see [above](#drive-it-from-claude-code-companion-skill)).
 - **Machine-readable interfaces** — stable `--output-format json` envelope on stdout, deterministic exit codes (`0`/`1`/`2`/`3`, with a couple of documented per-command exceptions — see [SPEC.md](./SPEC.md) for the full table), and `--help` on every command.
-- **JSON Schemas** — [`schema/scenario.schema.json`](./schema/scenario.schema.json) and [`schema/session.schema.json`](./schema/session.schema.json) describe every field of the YAML you author (generated from the source schemas; `npm run schema`). [`schema/protocol.v1.json`](./schema/protocol.v1.json) (hand-authored) schemas the harness's own control-channel wire protocol, with a golden vector pack at [`fixtures/protocol/v1/`](./fixtures/protocol/v1/) — see [docs/protocol.md](./docs/protocol.md).
+- **JSON Schemas** — [`schema/scenario.schema.json`](./schema/scenario.schema.json) and [`schema/session.schema.json`](./schema/session.schema.json) describe every field of the YAML you author (generated from the source schemas; `npm run schema`). [`schema/protocol.v1.json`](./schema/protocol.v1.json) (hand-authored) schemas the harness's own control-channel wire protocol, with a golden vector pack at [`fixtures/protocol/v1/`](./fixtures/protocol/v1/) — see [docs/protocol.md](./docs/protocol.md). [`schema/critique-report.json`](./schema/critique-report.json) describes `critique`'s JSON report / `critique-report.json` artifact for automation consumers (budget pacers, harvesters) — **descriptive, not §12-frozen** while critique is EXPERIMENTAL (see [SPEC.md §12](./SPEC.md#12-versioning--the-10-compatibility-contract)).
 
 `AGENTS.md`, `SPEC.md`, and `DESIGN.md` **are** shipped in the npm package (see `package.json` `files`) —
 a global install has them locally too, not just on GitHub.

package/SPEC.md

@@ -793,6 +793,13 @@ Covered-surface changes follow semver as of `1.0.0` — see [RELEASING.md](./REL
   the `doctor`/`verify-cassettes`/RunResult envelopes above, these have no `schema/*.json` and may change
   (fields, rule ids, the artifact-write-back finding shape) while the analyzers stabilize. Parse at your
   own risk until they are promoted to a covered surface.
+- **The `critique` report** (`--output-format json` / the `critique-report.json` run-dir artifact) —
+  NOT yet frozen, and — uniquely — it DOES have a schema file: `schema/critique-report.json` is
+  **descriptive** (authoritative field names/shapes, test-pinned against the builder) so automation can
+  parse against a schema rather than prose, but it is NOT this section's compatibility contract —
+  critique is EXPERIMENTAL and additive field changes may land in any minor release (the schema's own
+  `description` says so). Its presence in the surface-drift baseline is for change *visibility*, not
+  coverage. It is the promotion CANDIDATE once critique stabilizes, on the `doctor.json` template.
 - **`docs/internal/**`** — untracked working notes.
 - **The reconstructed system-prompt append text** — a paraphrase by design (see
   [docs/fidelity-gaps.md](./docs/fidelity-gaps.md)); behaviorally equivalent, not byte-stable.

package/dist/agent/session.js

@@ -242,6 +242,7 @@ export class LiveAgentSession {
     proc;
     events;
     controlOut;
+    errLog;
     timeline;
     lineIndex = 0;
     reqById = new Map();
@@ -275,8 +276,11 @@ export class LiveAgentSession {
         this.events = createWriteStream(join(outDir, "events.jsonl"), { flags: "a" });
         this.controlOut = createWriteStream(join(outDir, "control-out.jsonl"), { flags: "a" });
         this.timeline = new TimelineWriter(outDir);
-        const errLog = createWriteStream(join(outDir, "agent.stderr.log"), { flags: "a" });
-        this.proc.stderr.pipe(errLog);
+        // #60: keep a reference and pipe with `{ end: false }` so WE own the close — the drain in `start()`'s
+        // finally ends this stream and AWAITS its flush before the generator resolves, so a subsequent
+        // `scrubRawRunLogs` can't read the file while bytes are still buffered (a raw-secret-after-scrub leak).
+        this.errLog = createWriteStream(join(outDir, "agent.stderr.log"), { flags: "a" });
+        this.proc.stderr.pipe(this.errLog, { end: false });
         // keep a bounded stderr tail and capture the exit code/signal so a child that dies nonzero
         // (with no structured {type:"result"} error) is surfaced as a typed error event, not a silent stop.
         this.proc.stderr.on("data", (d) => {
@@ -404,11 +408,14 @@ export class LiveAgentSession {
             this.closing = true; // discard any late write() from translate()'s async hook/mcp paths
             await this.drainAll(); // queue empty + all callbacks confirmed before ending either stream
             // AWAIT the stream flush before the generator resolves. executeScenario reads/scans/scrubs
-            // events.jsonl + control-out.jsonl immediately after `drive()` returns; a fire-and-forget end()
-            // races the final buffered writes. end(cb) fires the callback on 'finish' (fully flushed).
+            // events.jsonl + control-out.jsonl + agent.stderr.log immediately after `drive()` returns; a
+            // fire-and-forget end() races the final buffered writes. end(cb) fires the callback on 'finish'
+            // (fully flushed). #60: agent.stderr.log is ended here too — piped with `{ end: false }` (see the
+            // ctor) so its last buffered bytes flush BEFORE the scrub reads it, never landing raw afterwards.
             await Promise.all([
                 new Promise((res) => this.events.end(() => res())),
                 new Promise((res) => this.controlOut.end(() => res())),
+                new Promise((res) => this.errLog.end(() => res())),
                 new Promise((res) => this.timeline.end(() => res())),
             ]);
         }

package/dist/critique/command.js

@@ -17,7 +17,8 @@ import { fileURLToPath } from "node:url";
 import { lookupSkillFlag } from "../run/skill-flag-surface.js";
 import { gradedAliasPath, turnArtifactPath } from "../run/turn-layout.js";
 import { renderKnownLimitations } from "./limitations.js";
-import { existsSync, readFileSync, copyFileSync, writeFileSync, readdirSync } from "node:fs";
+import { tildeify } from "../io.js";
+import { existsSync, readFileSync, copyFileSync, writeFileSync, readdirSync, statSync } from "node:fs";
 import { randomUUID } from "node:crypto";
 import { writeSync } from "node:fs";
 import { basename, join } from "node:path";
@@ -98,6 +99,7 @@ Not accepted (each errors with its reason rather than being silently ignored):
   --repeat + companions     fixed two-turn protocol — loop critique itself; pair by fingerprint.skillHash
   --ablate-skill            grading a skill you removed is incoherent
   --quiet/--verbose/--compact/--demo/--dry-run   inner-turn rendering or preview — no effect on the report
+                                                 (which already collapses host paths to ~)
 
 Repeating a flag: --upload/--folder/--plugin/--marketplace/--enable/--answer accumulate (that is how you
   pass several). Every other value-taking flag is single-valued and repeating it is a USAGE ERROR rather
@@ -107,8 +109,10 @@ Repeating a flag: --upload/--folder/--plugin/--marketplace/--enable/--answer acc
 COST AND PREREQUISITES — read before running:
   * Each critique is FOUR model workloads: two graded runs (task + reflection) at the chosen tier and two
     evaluator passes over an evidence package of up to ${MAX_PACKAGE_BYTES / 1024}KB.
-  * The evaluator defaults to ${DEFAULT_EVALUATOR_MODEL} — the most expensive tier. Override it if
-    that is not what you want.
+  * The evaluator defaults to ${DEFAULT_EVALUATOR_MODEL} — the most expensive tier — and the two
+    evaluator passes DOMINATE spend (~3/4 of a measured e2e total). For a batch: calibrate with run 1's
+    costUsd (gate on costUsd.complete), then consider a cheaper --evaluator-model for the sweep — noting
+    the armor's injection-resistance is verified for the DEFAULT evaluator only.
   * container needs Docker/Lima; hostloop needs Docker (the bash/web_fetch sidecar) PLUS the staged native
     agent binary, and writes to the real host FS (a writable --folder requires --allow-host-writes). Both
     tiers need an authenticated \`claude\` CLI on PATH.
@@ -361,6 +365,22 @@ function parseArgs(argv) {
  *  folder → same hash — so generation pairing keeps working; it is a per-plugin key, not per-skill.
  *  Exported for unit tests. */
 export function resolveCritiquedSkillDir(skillFolder, skillSelector) {
+    // Fail-fast on a typo'd / absent path BEFORE the caller mints a session and spawns the task turn — a
+    // missing folder otherwise only surfaces as a mid-run mount failure that leaves a stray run dir behind.
+    // This lives here (not in parseArgs) on purpose: parseArgs is unit-tested with fictitious paths, whereas
+    // this resolver is only ever called with a real folder. One statSync in a try/catch also covers a broken
+    // symlink; the existsSync guard just gives the common typo the clearer "not found" message.
+    if (!existsSync(skillFolder))
+        throw new Error(`skill folder not found: ${tildeify(skillFolder)}`);
+    let stat;
+    try {
+        stat = statSync(skillFolder);
+    }
+    catch {
+        throw new Error(`skill folder not found: ${tildeify(skillFolder)}`);
+    }
+    if (!stat.isDirectory())
+        throw new Error(`not a directory: ${tildeify(skillFolder)}`);
     const agentsMdFor = (name) => {
         const p = join(skillFolder, "agents", `${name}.md`);
         return existsSync(p) ? p : undefined;
@@ -380,7 +400,7 @@ export function resolveCritiquedSkillDir(skillFolder, skillSelector) {
         const candidate = join(skillFolder, "skills", skillSelector);
         if (!existsSync(join(candidate, "SKILL.md"))) {
             const available = listPluginSkills();
-            throw new Error(`--skill ${skillSelector}: no skills/${skillSelector}/SKILL.md under ${skillFolder}` +
+            throw new Error(`--skill ${skillSelector}: no skills/${skillSelector}/SKILL.md under ${tildeify(skillFolder)}` +
                 (available.length ? ` — available skills: ${available.join(", ")}` : ` — no skills/<name>/SKILL.md found at all`));
         }
         return { skillDir: candidate, agentsMdPath: agentsMdFor(skillSelector) };
@@ -391,7 +411,7 @@ export function resolveCritiquedSkillDir(skillFolder, skillSelector) {
     if (skills.length === 1)
         return { skillDir: join(skillFolder, "skills", skills[0]), agentsMdPath: agentsMdFor(skills[0]), autoSelectedSkill: skills[0] };
     if (skills.length > 1)
-        throw new Error(`${skillFolder} is a multi-skill plugin root (no root SKILL.md; skills: ${skills.join(", ")}) — ` +
+        throw new Error(`${tildeify(skillFolder)} is a multi-skill plugin root (no root SKILL.md; skills: ${skills.join(", ")}) — ` +
             `pass --skill <name> so critique grades the INVOKED skill's SKILL.md instead of a missing root one`);
     return { skillDir: skillFolder }; // no SKILL.md anywhere — the packager's existing missing/degraded flow reports it
 }
@@ -702,10 +722,10 @@ export const VERDICT_PROVENANCE = {
 export function buildTextReport(state) {
     const { skillFolder, prompt, sessionId, outDir, taskResult, gradedOutcome, gradedSkillHash, selfReportStatus, items, evaluatorModel, requestedModel, evaluatorError, infraFailure, turn1ResultDegraded, turn1SliceDegraded, skillMdStatus, } = state;
     const out = [];
-    out.push(`skill-critique: ${skillFolder}`);
+    out.push(`critique: ${tildeify(skillFolder)}`);
     out.push(`  probe: ${prompt}`);
     out.push(`  session: ${sessionId}`);
-    out.push(`  run dir: ${outDir}`);
+    out.push(`  run dir: ${tildeify(outDir)}`);
     out.push(`  fidelity: ${state.fidelity}` +
         (state.gradedEffectiveFidelity
             ? ` (graded turn recorded ${state.gradedEffectiveFidelity}${state.gradedBaseline ? `, baseline ${state.gradedBaseline}` : ""})`
@@ -722,6 +742,8 @@ export function buildTextReport(state) {
     // reflection turn's result.json for them).
     if (gradedOutcome)
         out.push(`  graded outcome: ${gradedOutcome}`);
+    if (state.gradedSkill)
+        out.push(`  graded skill: ${state.gradedSkill} (pair by skillHash + this name — skillHash keys the whole mounted plugin)`);
     if (gradedSkillHash)
         out.push(`  graded skillHash: ${gradedSkillHash.slice(0, 12)}`);
     if (evaluatorModel)
@@ -750,7 +772,7 @@ export function buildTextReport(state) {
     // The dominant real-world cause of a "missing" SKILL.md is pointing critique at a MULTI-SKILL PLUGIN
     // root (skills/<name>/SKILL.md, no root SKILL.md) — name the cause and the fix, not just the symptom.
     if (skillMdStatus === "missing")
-        out.push(`  NOTE: if ${skillFolder} is a multi-skill plugin root, pass --skill <name> (or point critique at <plugin>/skills/<name> directly) so the invoked skill's SKILL.md is graded.`);
+        out.push(`  NOTE: if ${tildeify(skillFolder)} is a multi-skill plugin root, pass --skill <name> (or point critique at <plugin>/skills/<name> directly) so the invoked skill's SKILL.md is graded.`);
     out.push(`  verdict scope: advisory self-run — NOT an independent attestation (never gate a skill on it)`);
     out.push("");
     const dropped = state.droppedEvaluatorItems;
@@ -768,12 +790,12 @@ export function buildTextReport(state) {
     }
     if (infraFailure) {
         out.push(`INFRASTRUCTURE/PROTOCOL FAILURE (reflection turn): ${infraFailure}`);
-        out.push(`The evaluator was NOT invoked — this is a broken discovery run, not a critique. Re-run, or inspect ${outDir} directly.`);
+        out.push(`The evaluator was NOT invoked — this is a broken discovery run, not a critique. Re-run, or inspect ${tildeify(outDir)} directly.`);
         return out.join("\n");
     }
     if (evaluatorError) {
         out.push(`EVALUATOR FAILED: ${evaluatorError}`);
-        out.push(`No critique items were produced. Re-run, or inspect ${outDir} directly.`);
+        out.push(`No critique items were produced. Re-run, or inspect ${tildeify(outDir)} directly.`);
         return out.join("\n");
     }
     const byBucket = new Map();
@@ -828,6 +850,7 @@ export function buildJsonReport(state) {
         gradedEffectiveFidelity: state.gradedEffectiveFidelity,
         gradedBaseline: state.gradedBaseline,
         costUsd: state.costUsd,
+        gradedSkill: state.gradedSkill,
         skillInvocationObserved: state.skillInvocationObserved,
         gateAnswers: state.gateAnswers,
         taskResult,
@@ -894,7 +917,7 @@ function writeRunArtifact(outDir, name, content) {
         writeFileSync(join(outDir, name), content);
     }
     catch (e) {
-        process.stderr.write(`skill-critique: could not write ${name} under ${outDir}: ${String(e)}\n`);
+        process.stderr.write(`critique: could not write ${name} under ${tildeify(outDir)}: ${String(e)}\n`);
     }
 }
 /** Persist the run-dir artifacts every critique leaves behind (all best-effort):
@@ -925,7 +948,7 @@ function writeOutFile(outPath, state, outputFormat) {
         writeFileSync(outPath, content);
     }
     catch (e) {
-        process.stderr.write(`skill-critique: --out ${outPath} could not be written: ${String(e)}\n`);
+        process.stderr.write(`critique: --out ${tildeify(outPath)} could not be written: ${String(e)}\n`);
     }
 }
 export function buildTaskTurnArgs(opts, sessionId) {
@@ -1000,7 +1023,7 @@ async function main(argv = process.argv.slice(2)) {
         return;
     }
     if (resolvedSkill.autoSelectedSkill)
-        process.stderr.write(`::notice:: [critique] ${opts.skillFolder} is a single-skill plugin — grading skills/${resolvedSkill.autoSelectedSkill}/SKILL.md (pass --skill to be explicit)\n`);
+        process.stderr.write(`::notice:: [critique] ${tildeify(opts.skillFolder)} is a single-skill plugin — grading skills/${resolvedSkill.autoSelectedSkill}/SKILL.md (pass --skill to be explicit)\n`);
     const sessionId = `crit-${randomUUID()}`;
     try {
         // 1. Task turn.
@@ -1017,7 +1040,7 @@ async function main(argv = process.argv.slice(2)) {
             ]
                 .filter(Boolean)
                 .join("; ");
-            process.stderr.write(`skill-critique: could not determine the task run's directory (no envelope outDir and no [status] line)${diag ? ` [${diag}]` : ""}.\n` +
+            process.stderr.write(`critique: could not determine the task run's directory (no envelope outDir and no [status] line)${diag ? ` [${diag}]` : ""}.\n` +
                 `--- task stdout ---\n${task.stdout}\n--- task stderr (tail) ---\n${task.stderr.slice(-4000)}\n`);
             process.exit(EXIT_INSTRUMENT_FAILURE);
             return;
@@ -1216,6 +1239,7 @@ async function main(argv = process.argv.slice(2)) {
             gradedEffectiveFidelity,
             gradedBaseline,
             costUsd,
+            gradedSkill: gradedSkillName,
             skillInvocationObserved,
             gateAnswers: gateAnswers?.length ? gateAnswers : undefined,
             taskResult,
@@ -1254,7 +1278,7 @@ async function main(argv = process.argv.slice(2)) {
             process.exit(EXIT_INSTRUMENT_FAILURE);
     }
     catch (e) {
-        process.stderr.write(`skill-critique: unexpected failure: ${e.stack ?? String(e)}\n`);
+        process.stderr.write(`critique: unexpected failure: ${e.stack ?? String(e)}\n`);
         process.exit(EXIT_INSTRUMENT_FAILURE); // an unexpected throw means no critique was produced
     }
     // FINDINGS never gate: any classification — including a task run that ERRORED, which is itself a

package/dist/critique/package-evidence.js

@@ -95,6 +95,13 @@ function listAttachedInputs(runDir) {
     const loaded = loadVmPathContext(runDir);
     const uploadsDir = loaded?.ctx.uploadsHostDir ?? join(runDir, "work", "session", "mnt", "uploads");
     const folderNames = loaded ? Array.from(loaded.ctx.folders.keys()).sort() : [];
+    // `loadVmPathContext` returns null for BOTH an absent mounts.json (legitimately no recorded mount
+    // context — folders default to []) AND a present-but-corrupt one (the folder map is UNKNOWN, not empty).
+    // Uploads has a fixed-layout fallback dir it can still probe; connected FOLDERS have none, so a corrupt
+    // mounts.json would silently render "(none)" — telling the evaluator "the agent correctly saw no
+    // connected folder" when the truth is UNKNOWN. That is the same confabulation-vs-correct false-clean the
+    // uploads path guards against below (the ENOENT-vs-read-fault split); mirror it for folders. #14
+    const mountsCorrupt = loaded === null && existsSync(join(runDir, "mounts.json"));
     const lines = [];
     try {
         const uploadNames = readdirSync(uploadsDir, { withFileTypes: true })
@@ -124,6 +131,9 @@ function listAttachedInputs(runDir) {
     }
     for (const name of folderNames)
         lines.push(`${name} (connected folder)`);
+    if (mountsCorrupt)
+        lines.push(`(connected-folder context could not be read: mounts.json present but unparseable — ` +
+            `folder attachment presence UNKNOWN, not confirmed absent)`);
     return lines.join("\n");
 }
 /** Byte-bound a text section, appending a loud (never silent) truncation marker so the evaluator knows the

package/dist/run/diff.js

@@ -1,6 +1,7 @@
 // Run/cassette diff engine with normalization. Compares two runs, two cassettes, or a run and a
 // cassette (both reduce to an event stream + result metadata through parseMessage/buildTrace, the same
 // typed model `trace` uses — so this stays correct as the SDK schema evolves).
+import { createHash } from "node:crypto";
 import { DEFAULT_SCAN_PATTERNS } from "../scan.js";
 import { diffFileSigsPaths } from "./cassette.js";
 const HOST_PATH_RE = DEFAULT_SCAN_PATTERNS.find((p) => p.cls === "path").re;
@@ -56,7 +57,13 @@ const CANON_CAP = 2000;
 /** Bounded, key-aware canonicalization of a tool's `input` (or any structured value) for tool-sequence
  *  comparison — masks volatile string spans AND volatile key names, then caps the length. The 100-char
  *  `summarize()` in trace-view.ts is display-lossy by design; this needs its own, larger, comparison-safe
- *  cap (still bounded — never an unbounded dump into a diff hunk). */
+ *  cap (still bounded — never an unbounded dump into a diff hunk).
+ *
+ *  #41: when the value exceeds the cap, the truncated prefix ALONE is not a safe equality key — two inputs
+ *  sharing the first `CANON_CAP` chars but differing only in the dropped tail would compare equal (a
+ *  false "same" tool call in `diffToolSequence`). So a truncated key folds in a hash of the FULL canonical
+ *  string: the prefix stays human-readable in a diff hunk, the `#<len>·<sha16>` suffix makes the key
+ *  depend on the entire content. Both diff sides run this identically, so the comparison stays consistent. */
 export function canonicalizeInput(input, normalize = true) {
     let s;
     try {
@@ -65,7 +72,10 @@ export function canonicalizeInput(input, normalize = true) {
     catch {
         s = String(input);
     }
-    return s.length > CANON_CAP ? s.slice(0, CANON_CAP) + "…" : s;
+    if (s.length <= CANON_CAP)
+        return s;
+    const fullHash = createHash("sha256").update(s).digest("hex").slice(0, 16);
+    return `${s.slice(0, CANON_CAP)}…#${s.length}·${fullHash}`;
 }
 /** LCS over full-row equality (name AND canon must match for "same"), then a post-pass over each gap
  *  between LCS matches: a 1-vs-1 gap with the SAME tool name is a "changed" input, not a remove+add pair

package/dist/run/execute.js

@@ -1664,10 +1664,11 @@ function scrubFileInPlace(path, secrets) {
  *  executeScenario's outermost `finally` (and the chat lane's teardown) so every exit path AFTER the
  *  agent session exists scrubs — success, the unanswered-gate salvage rethrow, and any rethrown fault
  *  (agent crash, infra error, hostloop snapshot failure). Deliberately NOT total coverage: a throw
- *  before that try has no raw logs yet; a SIGKILL of the harness process skips any finally; and the
- *  agent.stderr.log write stream is never awaited, so bytes still buffered at scrub time can land raw
- *  afterwards (closing that needs an awaitable stderr-sink close on the session — a follow-up, not
- *  this seam). Exported for tests. */
+ *  before that try has no raw logs yet, and a SIGKILL of the harness process skips any finally. #60: the
+ *  agent.stderr.log sink IS now flushed before this runs — `LiveAgentSession` pipes it with `{ end: false }`
+ *  and ends+awaits it in `start()`'s drain, so no buffered stderr byte lands raw after the scrub reads the
+ *  file (the session generator resolves only after that flush, and this runs after it resolves). Exported
+ *  for tests. */
 export function scrubRawRunLogs(outDir, secrets) {
     scrubFileInPlace(join(outDir, "events.jsonl"), secrets);
     scrubFileInPlace(join(outDir, "control-out.jsonl"), secrets);

package/dist/run/skill-flag-surface.js

@@ -91,7 +91,7 @@ export const SKILL_FLAG_SURFACE = [
         arity: 0,
         critique: {
             kind: "reject",
-            reason: "critique produces its own report; inner-turn rendering flags have no effect on it",
+            reason: "critique produces its own report (host paths in it are already collapsed to ~); inner-turn rendering flags have no effect on it",
         },
     })),
     {

package/dist/runtime/lima.js

@@ -51,7 +51,19 @@ export function instanceName(baseline) {
  * result of this one helper into both.
  */
 export function vmGatewayIp() {
-    return process.env.COWORK_VM_GATEWAY ?? "192.168.5.2";
+    const raw = process.env.COWORK_VM_GATEWAY ?? "192.168.5.2";
+    // #95: this value is interpolated into a root-run iptables command inside the guest
+    // (guestFirewallScript → `iptables -A OUTPUT -d ${gatewayIp}` executed via `sh -c`). Validate it as a
+    // canonical IPv4 literal and reject everything else, so a malformed or hostile override can never inject
+    // shell syntax into privileged provisioning. Defense-in-depth: the var is operator-set, but an
+    // unvalidated string reaching a root `sh -c` should be impossible by construction, not by trust. IPv4
+    // only — the Apple VZ user-network gateway is IPv4, and the digits-and-dots grammar excludes every shell
+    // metacharacter (`;`, `$`, backtick, whitespace, …).
+    const octets = raw.split(".");
+    const canonicalIPv4 = octets.length === 4 && octets.every((o) => /^\d{1,3}$/.test(o) && Number(o) <= 255 && String(Number(o)) === o);
+    if (!canonicalIPv4)
+        throw new Error(`COWORK_VM_GATEWAY must be a canonical IPv4 literal (e.g. 192.168.5.2); got ${JSON.stringify(raw)}`);
+    return raw;
 }
 export function vmStatus(instance) {
     const r = spawnSync(limaPath(), ["list", instance, "--format", "{{.Status}}"], { encoding: "utf8" });

package/docs/critique.md

@@ -124,7 +124,7 @@ ignored.
 | `--session-id` / `--resume` | critique mints and manages its own session — the reflection turn *is* a resume of it |
 | `--repeat` + companions | fixed two-turn protocol; loop `critique` itself and pair by `fingerprint.skillHash` |
 | `--ablate-skill` | grading a skill you removed is incoherent |
-| `--quiet`/`-q` / `--verbose` / `--compact` / `--demo` / `--dry-run` | inner-turn rendering or preview — no effect on the report |
+| `--quiet`/`-q` / `--verbose` / `--compact` / `--demo` / `--dry-run` | inner-turn rendering or preview — no effect on the report (which already collapses host paths to `~`) |
 
 **Repeating a flag.** `--upload`, `--folder`, `--plugin`, `--marketplace`, `--enable` and `--answer` accumulate,
 so repeating them is how you pass several. Every other value-taking flag is single-valued and repeating it is
@@ -144,8 +144,11 @@ adjudicable". So:
   auto-selects with a notice.
 - **Selection only:** the positional folder is still what both turns mount (session identity is
   unchanged), and **`fingerprint.skillHash` is unchanged by `--skill`** — it keys the *mounted folder*,
-  so it pairs generations per-plugin, not per-skill. Pair per-skill runs by `--label` if you need finer
-  grouping.
+  so it pairs generations per-plugin, not per-skill. **Workflow implication: pairing critiques of a
+  multi-skill plugin by skillHash alone CROSS-PAIRS different skills** — pair by
+  **(`gradedSkillHash`, `gradedSkill`)**; the report's `gradedSkill` field carries the resolved
+  `skills/<name>` (`--skill` or the auto-selection). `--label` remains available for coarser
+  generation tags.
 - The report carries an advisory **`skillInvocationObserved`**: `false` means the graded run's own
   `skillActivity` never mentions the selected skill — the critique may be grading a run that did not
   actually invoke it.
@@ -170,6 +173,12 @@ It does **not** record their contents — see Known limitations.
   evaluator passes.
 - The evaluator defaults to the most expensive tier. Override with `--evaluator-model <id>` or
   **`COWORK_HARNESS_EVALUATOR_MODEL`**.
+- **The evaluator passes dominate spend** — on a measured end-to-end (trivial probe, default evaluator)
+  the two evaluator passes were ~3/4 of the total. For a wide batch: calibrate with run 1's `costUsd`
+  (gate on `costUsd.complete` — `false` means the total undercounts), then consider a cheaper
+  `--evaluator-model` for the sweep. Caveat: the armor's injection-resistance is verified for the
+  shipped **default** evaluator model only — changing it voids that specific verification (matters when
+  critiquing skills you did not write).
 - **container** needs Docker/Lima; **hostloop** needs Docker (the bash/web_fetch sidecar) **plus** the
   staged native agent binary, and writes to the real host filesystem — a writable `--folder` there requires
   `--allow-host-writes`. Both tiers need an authenticated `claude` CLI on PATH.
@@ -227,6 +236,14 @@ malformed evaluator items (the surviving findings are then not necessarily the c
 readable-but-oversized SKILL.md is flagged **`skillMdTruncated`** ("the evaluator graded a cut copy") —
 distinct from missing/unreadable, which alone force the mechanical `"already-covered"` downgrade.
 
+**Truncation has a second, sharper consequence than the not-adjudicable steer: DROPPED findings.**
+Citation validation checks each finding's `evidence` excerpt verbatim against the *packaged* (cut)
+copy — so a finding that quotes text past the cut cannot resolve and lands in **DROPPED**, even when
+the quote is a perfectly accurate excerpt of the real file. A skill well over the cap should expect a
+not-adjudicable/DROPPED skew concentrated on its back half; if you see back-half findings in DROPPED
+on a `skillMdTruncated` run, this is why — front-load the operative guidance, split the skill, or
+treat those items as leads to re-check by hand.
+
 ### Run-dir artifacts
 
 Beyond stdout, every critique leaves durable artifacts at the run-dir root (best-effort writes —
@@ -240,6 +257,10 @@ Beyond stdout, every critique leaves durable artifacts at the run-dir root (best
 
 These artifacts (and the report's JSON shape) are part of critique's **EXPERIMENTAL** surface — useful
 and stable in practice, but not yet a frozen SPEC §12 covered surface; field additions are expected.
+The report's field names and shapes are authoritatively described by
+[`schema/critique-report.json`](../schema/critique-report.json) (descriptive + test-pinned against the
+actual builder, unlike the §12-frozen `doctor.json`), so automation consumers — budget pacers gating on
+`costUsd.complete`, harvesters pairing on `gradedSkill` — parse against a schema, not prose.
 
 ## Reproduction — the ≥2-run discipline
 
@@ -258,8 +279,15 @@ Then pair/cluster across the reports:
 - **Same finding across runs/inputs?** cluster by each item's **`findingFingerprint`** (sha over the
   normalized idea + classification + recommendedAction, deliberately excluding the input-specific
   `evidence` excerpt — so the same finding matches across different decks/transcripts).
+- **The fingerprint is high-precision, LOW-RECALL — read the direction correctly.** `idea` is
+  model-authored free text, so the same underlying finding *reworded* across runs fingerprints
+  differently. A **match proves** reproduction; a **mismatch does NOT prove** non-reproduction — before
+  concluding "didn't reproduce", skim the unmatched items for rewordings of the same substance.
 - A finding that recurs across ≥2 runs with the same `findingFingerprint` meets the reproduction bar;
-  a one-off is a lead, not a conclusion.
+  a fingerprint one-off is a lead — possibly a real one-off, possibly a reworded repeat.
+- **Multi-skill plugins: never pair by `gradedSkillHash` alone.** The hash keys the whole mounted
+  plugin, so it cross-pairs critiques of *different* skills in the same plugin — pair by
+  **(`gradedSkillHash`, `gradedSkill`)**; `gradedSkill` is the report's resolved `skills/<name>`.
 - To make the graded runs deterministic across repeats, copy the report's echoed `--answer` lines
   (the graded run's resolved gate answers) into the next invocation.
 

package/docs/scenario.md

@@ -952,7 +952,8 @@ with `COWORK_LIMA_INSTANCE`.
 - **A run errors with "not mounted — VM not provisioned for this harness config"** — the VM predates a
   config change (its mounts don't match). Recreate it: `cowork-harness vm delete && cowork-harness vm init`.
 - **Egress allowed/denied looks wrong** — the guest firewall and the proxy URL must point at the same
-  gateway. The default Apple-VZ user-network gateway is `192.168.5.2`; override with `COWORK_VM_GATEWAY`,
+  gateway. The default Apple-VZ user-network gateway is `192.168.5.2`; override with `COWORK_VM_GATEWAY`
+  (a canonical IPv4 literal — an invalid value is rejected, as it feeds the guest iptables rule),
   and the proxy port with `COWORK_VM_PROXY_PORT` (unset, the host binds an OS-assigned free port;
   `8899` is only the guest-config fallback when a VM is spawned without an explicit port — not the
   effective default of a normal run). The harness threads one resolved

package/examples/replays/README.md

@@ -16,7 +16,7 @@ DOES exercise a real gate exchange, see `example-multiselect-gate.cassette.json`
 
 Run it with:
 
-> Assumes the `cowork-harness` CLI is available — from a source checkout run `npm ci && npm run build && npm link` first, or `npm i -g "cowork-harness@>=1.8.0"`. (`replay` itself needs nothing else — no token, no Docker.)
+> Assumes the `cowork-harness` CLI is available — from a source checkout run `npm ci && npm run build && npm link` first, or `npm i -g "cowork-harness@>=1.9.0"`. (`replay` itself needs nothing else — no token, no Docker.)
 
 ```sh
 cowork-harness replay examples/replays/example-pdf-skill.cassette.json

package/llms.txt

@@ -17,6 +17,7 @@ It is a fidelity fixture, not the Desktop runtime. The CLI binary is `cowork-har
 - [examples/README.md](examples/README.md): worked, copyable example scenarios + sessions + skills (from a source checkout — the npm package ships only examples/replays/; protocol/container tiers, token-free replay, prerequisites)
 - [schema/scenario.schema.json](schema/scenario.schema.json): JSON Schema for scenario files (machine-readable)
 - [schema/session.schema.json](schema/session.schema.json): JSON Schema for session files (machine-readable)
+- [schema/critique-report.json](schema/critique-report.json): descriptive schema for `critique`'s JSON report / `critique-report.json` artifact (EXPERIMENTAL — not §12-frozen; field additions expected)
 - [.claude/skills/cowork-harness/references/ci-recipe.md](.claude/skills/cowork-harness/references/ci-recipe.md): copy-paste GitHub Actions — token-free replay PR gate + nightly live lane
 
 ## Concepts & internals

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "cowork-harness",
-  "version": "1.8.0",
+  "version": "1.9.0",
   "description": "Scriptable, CI-friendly harness for Claude Cowork's runtime contract for testing skills across scenarios \u2014 same agent, mounts, egress allowlist, permission protocol, and sandbox limitations.",
   "license": "MIT",
   "type": "module",

package/schema/critique-report.json

@@ -0,0 +1,221 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "$id": "critique-report.json",
+  "title": "cowork-harness critique --output-format json report (and critique-report.json run-dir artifact)",
+  "description": "EXPERIMENTAL, descriptive schema — NOT a SPEC §12-frozen surface: additive field changes are expected between minor versions (unlike doctor.json). It exists so automation consumers (budget pacers, harvesters) have authoritative field names/shapes instead of prose. Exactly one of infraFailure / evaluatorError / evaluatorModel-with-findings describes the outcome; items is [] on both failure branches.",
+  "type": "object",
+  "required": ["skillFolder", "prompt", "sessionId", "outDir", "fidelity", "selfReportStatus", "verdictProvenance", "items"],
+  "additionalProperties": false,
+  "properties": {
+    "skillFolder": {
+      "type": "string",
+      "description": "the positional folder BOTH turns mounted (a plain skill, or a plugin root)"
+    },
+    "prompt": {
+      "type": "string"
+    },
+    "sessionId": {
+      "type": "string"
+    },
+    "outDir": {
+      "type": "string",
+      "description": "the kept run dir (turns/1 = graded, turns/2 = reflection; critique-* artifacts at its root)"
+    },
+    "fidelity": {
+      "type": "string",
+      "description": "the tier critique pinned for both turns (cowork is refused, so requested == resolved)",
+      "enum": ["container", "hostloop"]
+    },
+    "gradedEffectiveFidelity": {
+      "type": "string",
+      "description": "best-effort: the tier the graded turn's own result.json RECORDS (should equal fidelity; both surfaced so a mismatch is visible)"
+    },
+    "gradedBaseline": {
+      "type": "string",
+      "description": "best-effort: the graded turn's fingerprint.baseline (Desktop appVersion)"
+    },
+    "costUsd": {
+      "type": "object",
+      "description": "per-critique cost across the FOUR model workloads. GATE ON `complete` BEFORE TRUSTING totalUsd: false means one or more workloads were unpriced and the total UNDERCOUNTS true spend.",
+      "required": ["totalUsd", "complete"],
+      "additionalProperties": false,
+      "properties": {
+        "taskTurnUsd": {
+          "type": "number"
+        },
+        "reflectionTurnUsd": {
+          "type": "number"
+        },
+        "evaluatorPass1Usd": {
+          "type": "number"
+        },
+        "evaluatorPass2Usd": {
+          "type": "number"
+        },
+        "totalUsd": {
+          "type": "number"
+        },
+        "complete": {
+          "type": "boolean"
+        }
+      }
+    },
+    "gradedSkill": {
+      "type": "string",
+      "description": "the resolved skills/<name> the packager graded (--skill or single-skill auto-selection); ABSENT for a plain skill folder. Multi-skill-plugin pairing key: skillHash keys the whole mounted plugin, so pair by (gradedSkillHash, gradedSkill), never skillHash alone"
+    },
+    "skillInvocationObserved": {
+      "type": "boolean",
+      "description": "advisory: whether the graded run's own skillActivity mentions gradedSkill; false = the critique may be grading a run that never invoked the selected skill"
+    },
+    "gateAnswers": {
+      "type": "array",
+      "description": "the graded run's resolved gate answers, for a deterministic follow-up run",
+      "items": {
+        "type": "object",
+        "required": ["question", "answer", "answeredBy"],
+        "additionalProperties": false,
+        "properties": {
+          "question": {
+            "type": "string"
+          },
+          "answer": {
+            "type": "string"
+          },
+          "answeredBy": {
+            "type": "string"
+          }
+        }
+      }
+    },
+    "taskResult": {
+      "type": "string",
+      "description": "the graded turn's own result — error is a GRADEABLE outcome, not an instrument failure",
+      "enum": ["success", "error"]
+    },
+    "gradedOutcome": {
+      "type": "string",
+      "description": "the graded turn's result.json outcome (e.g. delivered_clean)"
+    },
+    "gradedSkillHash": {
+      "type": "string",
+      "description": "content-exact hash of the MOUNTED folder — the cross-FIX pairing key (per-plugin under --skill)"
+    },
+    "selfReportStatus": {
+      "type": "string",
+      "description": "unavailable = pass 2 was skipped; findings are pass-1-only",
+      "enum": ["captured", "unavailable"]
+    },
+    "evaluatorIntegrity": {
+      "type": "object",
+      "description": "mechanical canary check per pass; false = that pass ignored a trusted instruction — an empty critique may be adversarial silencing, not a clean skill",
+      "required": ["pass1Canary"],
+      "additionalProperties": false,
+      "properties": {
+        "pass1Canary": {
+          "type": "boolean"
+        },
+        "pass2Canary": {
+          "type": "boolean"
+        }
+      }
+    },
+    "droppedEvaluatorItems": {
+      "type": "object",
+      "description": "malformed items the per-item-tolerant parse dropped; non-zero means the surviving findings under-represent the full reply",
+      "required": ["pass1"],
+      "additionalProperties": false,
+      "properties": {
+        "pass1": {
+          "type": "number"
+        },
+        "pass2": {
+          "type": "number"
+        }
+      }
+    },
+    "turn1ResultDegraded": {
+      "type": "boolean",
+      "description": "the graded turn's canonical result was corrupted/never archived — result-derived sections are empty DEFAULTS, treat as unknown"
+    },
+    "turn1SliceDegraded": {
+      "type": "boolean",
+      "description": "the turn-1 transcript slice's boundary could not be trusted — treat gaps as unknown"
+    },
+    "skillMdStatus": {
+      "type": "string",
+      "description": "readability of the packaged SKILL.md; missing/unreadable force the mechanical already-covered -> not-adjudicable downgrade",
+      "enum": ["readable", "missing", "unreadable"]
+    },
+    "skillMdTruncated": {
+      "type": "boolean",
+      "description": "SKILL.md was READABLE but over its cap — the evaluator graded a CUT copy. Consequences: claims about past-cut content are steered to not-adjudicable, and a finding QUOTING past-cut text fails citation-resolution and lands in DROPPED"
+    },
+    "verdictProvenance": {
+      "type": "object",
+      "description": "advisory scoping: the verdict is a self-run discovery lead, never an independent attestation",
+      "required": ["kind", "advisory", "caveat"],
+      "additionalProperties": false,
+      "properties": {
+        "kind": {
+          "type": "string",
+          "enum": ["self-run"]
+        },
+        "advisory": {
+          "type": "boolean"
+        },
+        "caveat": {
+          "type": "string"
+        }
+      }
+    },
+    "infraFailure": {
+      "type": "string",
+      "description": "instrument failure (task turn killed / reflection protocol broke) — no critique was produced; items is []; process exits 2"
+    },
+    "evaluatorError": {
+      "type": "string",
+      "description": "the evaluator threw (embeds the raw reply) — no critique was produced; items is []; process exits 2; see critique-salvage.json"
+    },
+    "evaluatorModel": {
+      "type": "string",
+      "description": "the transport-RESOLVED evaluator model (never the requested alias); present only when the evaluator completed"
+    },
+    "items": {
+      "type": "array",
+      "description": "the citation-validated findings from both passes. citationResolved:false = the cited excerpt did not resolve verbatim against the evidence package (DROPPED section — transparency only, do not act on as-is)",
+      "items": {
+        "type": "object",
+        "required": ["source", "idea", "classification", "evidence", "recommendedAction"],
+        "additionalProperties": false,
+        "properties": {
+          "source": {
+            "type": "string",
+            "enum": ["evaluator", "self-report"]
+          },
+          "idea": {
+            "type": "string"
+          },
+          "classification": {
+            "type": "string",
+            "enum": ["grounded-and-actionable", "grounded-but-not-worth-it", "confabulated", "already-covered", "not-adjudicable"]
+          },
+          "evidence": {
+            "type": "string",
+            "description": "the model's cited excerpt — verbatim-checked against the evidence package"
+          },
+          "recommendedAction": {
+            "type": "string"
+          },
+          "citationResolved": {
+            "type": "boolean"
+          },
+          "findingFingerprint": {
+            "type": "string",
+            "description": "sha256/16 over normalized idea+classification+recommendedAction (evidence excluded). Cross-INPUT clustering key — HIGH-PRECISION, LOW-RECALL: idea is model-authored free text, so a MATCH proves the same finding recurred; a MISMATCH does NOT prove non-reproduction (the same finding reworded fingerprints differently)"
+          }
+        }
+      }
+    }
+  }
+}