cowork-harness 3.1.0 → 3.2.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 3.1.0
-  tracks-harness: cowork-harness 3.1.0 (baseline desktop-1.40609.0)
+  version: 3.2.0
+  tracks-harness: cowork-harness 3.2.0 (baseline desktop-1.40609.0)
 ---
 
 # cowork-harness
@@ -25,7 +25,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 3.1.0` (baseline
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 3.2.0` (baseline
 > `desktop-1.40609.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
@@ -42,7 +42,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 3.1.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^3.1.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^3.1.0"`. **Pin `@^3.1.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 3.2.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^3.2.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^3.2.0"`. **Pin `@^3.2.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -57,21 +57,15 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 Everything you do with the harness is one of **three loops**, and the rest of this skill is organized
 into three Parts to match: **author** a scenario (Part I), **run / record / lock** it into a
 reproducible regression (Part II), and **debug** a run that misbehaved or greened when it shouldn't
-(Part III — reachable straight from here in one hop). Pick the loop you're in:
+(Part III — reachable straight from here in one hop).
+
+Pick the entry point you need. The first three are the everyday path — a quick liveness check, the
+CI-grade scenario, and the post-hoc debug loop; the rest are narrower tools that hang off them:
 
 - **"Is it even alive?"** (inner loop) → `cowork-harness skill <folder> "<prompt>"`. Fastest; no
   scenario file.
 - **Repeatable, asserted regression** → author a `scenarios/*.yaml` and run `cowork-harness run`.
   This is the CI-grade path and most of this skill.
-- **Regression-test your skill's ANSWER quality** (not just its behavior — does its guidance still lead to
-  correct answers after you edit it?) → author `semantic_matches` scenarios and gate on the per-claim
-  profile. See **Recipe 5** in `references/task-recipes.md` (validity, N≥3, discrimination — the traps).
-- **"What is WRONG with this skill?"** (a graded critique, not a pass/fail) → `cowork-harness critique
-  <folder> --prompt "<probe>"`. Four model workloads and 10–20 minutes; budget from
-  `report.costUsd.totalUsd`. Reach for it when you want **findings**. **For "what does this skill
-  **DO**" — routing, artifact location, narration — use `skill` instead**: no evaluator, a fraction of
-  the cost, and it answers that question directly. Report and evidence-package shapes:
-  `references/critique.md`.
 - **A run failed — or greened and you don't trust it** (the debugging loop) → don't re-run and hope.
   The run already wrote its evidence to a **kept run dir** (`~/.cowork-harness/runs/…`; `--keep` prints
   the path, `trace <run-id>` finds it). **Localize the failure post-hoc** from that evidence:
@@ -83,6 +77,15 @@ reproducible regression (Part II), and **debug** a run that misbehaved or greene
   **"Evidence" here means the RUN's own record** — events, trace, transcript. `critique`'s evaluator
   grades against a different artifact, `critique-evidence-package.txt`, which none of these tools
   surface; see `references/critique.md`.
+- **Regression-test your skill's ANSWER quality** (not just its behavior — does its guidance still lead to
+  correct answers after you edit it?) → author `semantic_matches` scenarios and gate on the per-claim
+  profile. See **Recipe 5** in `references/task-recipes.md` (validity, N≥3, discrimination — the traps).
+- **"What is WRONG with this skill?"** (a graded critique, not a pass/fail) → `cowork-harness critique
+  <folder> --prompt "<probe>"`. Four model workloads and 10–20 minutes; budget from
+  `report.costUsd.totalUsd`. Reach for it when you want **findings**. **For "what does this skill
+  **DO**" — routing, artifact location, narration — use `skill` instead**: no evaluator, a fraction of
+  the cost, and it answers that question directly. Report and evidence-package shapes:
+  `references/critique.md`.
 - **Multi-turn / interactive reproduction** → `cowork-harness chat` (interactive; gates answered at the
   TTY, **not** an asserted test — see *Debugging with `chat`* in **Part III — Debug**).
   **"Interactive" splits two ways — don't take the wrong branch.** Want to answer gates yourself *and*
@@ -380,7 +383,10 @@ emits a scenario `lint` would reject.
 `lint` (exit 0) but a **hard error** in the runtime (`Unrecognized key: "<k>"`, exit 2) — so a scenario
 that lints with warnings may still not run. To check whether a scenario actually loads, without spending:
 `cowork-harness record <file.yaml> --dry-run` (exit 2 on a schema error; a directory reports each
-`✗ broken:` file and exits 1). Corollary: **the loader** fails LOUD on an unknown key (never silently) —
+`✗ broken:` file and exits 1). **Read the exit code, not just its sign:** `record <file>` — with or without
+`--dry-run` — answers `2` for "did not load" and `1` for "loaded fine, but this record is refused" (a
+pre-spend policy refusal; `--max-budget-usd` is the one refusal that keeps exit 2). Treating any non-zero
+as "scenario broken" mis-reports every refused-but-valid scenario. Corollary: **the loader** fails LOUD on an unknown key (never silently) —
 but **`replay` does not**: a frozen top-level key it doesn't recognize (e.g. `lane:` recorded pre-1.16.0) is
 silently ignored and can flip a lane-sensitive verdict green; only frozen **assertion** keys stay
 hard-rejected there. Full split + the v11 version-regime:
@@ -416,6 +422,28 @@ contradiction, duplicate cassette target) gate the batch. So a directory dry-run
 real `record` would refuse; re-run that one file with its real flags for a binding answer. A directory also
 reports every offender and the batch cost estimate. `lint` checks the assertion invariants (both above).
 
+**Which arm to reach for.** They answer different questions, and picking the wrong one is why a consumer
+concluded the free pre-flight was unavailable:
+- **"Does my whole corpus still load?"** → the **directory** arm (`record scenarios/ --dry-run --quiet`,
+  the CI shape in `references/ci-recipe.md`). It reports every offender in one pass, and the
+  destination-policy verdict cannot red it: that arm knows no `--out`, so host-inventory and portability
+  are advisory `notes[]` at exit 0 while a file that cannot load is `✗ broken:` at exit 1. Limits worth
+  knowing: it is **non-recursive** (`readdirSync` — scenarios in subdirectories are never opened), a file
+  with no `prompt:` key reports as `· skipped:` rather than broken (so a renamed or mis-indented
+  `prompt:` reads as "not a scenario" and the batch still exits 0), exit 1 covers **refused**
+  (path-independent only — prompt policy, assert contradiction, duplicate target) as well as broken, and
+  `--quiet` suppresses the advisory notes entirely — it keeps `✗ broken:` / `✗ refused:` / `· skipped:`,
+  which is what you want in CI but means the notes are not a thing you will see there.
+- **"Would THIS record be refused?"** → the **single-file** arm **with the flags and `--out` the real
+  record will get**. The destination it evaluates is `--out` if given, else `cassettes/<slug>.cassette.json`
+  *relative to your cwd* — so previewing from the repo root a record that really runs from a subdirectory
+  asks about a path that may not even exist, and a scenario whose cassette IS committed can come back
+  refused. Point it at the real destination and the answer is binding.
+
+Neither question is answered by passing `--allow-host-inventory-fixture` to get past the refusal: that
+flag is consent for a recording you intend to make, and reaching for it as a load-check habit is how it
+stops meaning anything.
+
 **Decide WHERE the cassette lives before you record it — a cassette cannot be moved afterwards.**
 Without `--out`, `record` writes `cassettes/<scenario-name-slug>.cassette.json` (gitignored by
 default); pass `--out <path>` to put it somewhere tracked, e.g. `examples/replays/<name>.cassette.json`.
@@ -429,7 +457,7 @@ warns when the cassette would be written outside the scenario's tree, or when `s
 outside it (an absolute or `~` path: the mirror case, invisible to a check that only looks at where the
 cassette lands). A warning, not a refusal — an out-of-tree throwaway cassette is legitimate; what was
 missing was anything saying so while you could still act. Related: recording at a **host-inheriting** tier
-(`protocol`/`hostloop`/`cowork`→hostloop) into a repo-visible path is refused outright (gotcha 25).
+(`protocol`/`hostloop`/`cowork`→hostloop) into a repo-visible path is refused outright (gotcha 25 below).
 The clean answer there is `fidelity: container` (sealed, `HOME=/tmp`, nothing to leak) — **not**
 redirecting `--out` outside the repo and moving the file in afterwards, which trades a loud refusal
 for a cassette that cannot verify staleness from its own location — recoverable only by passing
@@ -600,7 +628,7 @@ Recognize these before "fixing" a non-bug:
   `RunResult.scan` is undefined and the host-path + outputs-delete guards **did not run this run**. Not a
   pass or a defect — assert `no_delete_in_outputs` / `transcript_no_host_path` to hard-fail on it instead.
 
-The full 17-code signal table (severity + per-signal opt-out) is in
+The full 20-code signal table (severity + per-signal opt-out) is in
 [`references/scenario-schema.md`](./references/scenario-schema.md); [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) (repo-only) carries
 the fuller narrative.
 
@@ -750,7 +778,8 @@ decide which assertions from *Assertions: two orthogonal axes* are worth adding)
   `tokens`, `cache-tokens`, `model-cost`, `turns`, `pass-rate`. Filters: `--since`/`--baseline`/`--branch`,
   plus `--skill-hash <prefix>`/`--label <tag>` to narrow to ONE skill generation and
   `--group-by scenario|skill-hash|label|fidelity` to split per generation — or per effective fidelity
-  tier — instead of aggregating across them (a window spanning >1 generation warns — see Gotcha 6;
+  tier — instead of aggregating across them (a window spanning >1 generation warns, so an un-split A/B
+  average announces itself rather than passing as one number;
   >1 tier warns too, independently, with `--group-by fidelity` as its own remedy). `--runs` lists the
   individual runs behind each summary with their `skillHash`/`runLabel`, so
   you can tell which arm a run belonged to without opening its `result.json`. `--last <n>` windows per group.
@@ -773,7 +802,7 @@ decide which assertions from *Assertions: two orthogonal axes* are worth adding)
   `outputTruncated` on a matched tool result). Three separately-shaped rollups, easy to conflate in a
   `jq` recipe: `toolCounts` is a flat `{tool: number}` call-count map, `toolErrors` is
   `{tool: {calls, errors}}`, and `toolDurations` is `{tool: {calls, totalMs, maxMs}}`. (Full per-field
-  semantics: the README's "Observability fields" section — repo-only; [`schema/run-result.json`](https://github.com/yaniv-golan/cowork-harness/blob/main/schema/run-result.json) is the
+  semantics: [`docs/cli.md` → What you get out](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/cli.md#what-you-get-out-inspectable-output) (repo-only); [`schema/run-result.json`](https://github.com/yaniv-golan/cowork-harness/blob/main/schema/run-result.json) is the
   machine source.)
 - **Opaque failure?** A failed run also records **`errorSource`** (where the failure originated) and
   **`stderrLogPath`** (the captured agent stderr) — read those and `trace <run-dir>` *before* re-running;
@@ -830,8 +859,12 @@ The startup banner now reads `type your message (/help for commands)` as a remin
 
 ## Gotchas — the "✓ passed ≠ correct" landmines
 
-Stated as *symptom → why → fix*. **This is the full landmine catalog;** `references/scenario-schema.md`
-repeats the assertion/replay-relevant ones alongside the schema (a scoped subset, not a fuller list).
+Stated as *symptom → why → fix*. This is the **workflow/record/answer-path** view — the broader of the
+two lists, but **not a strict superset**: `references/scenario-schema.md`'s *Authoring gotcha list*
+carries a few assertion-level landmines this one omits (`transcript_no_host_path`'s scan width,
+`egress.extra_allow`'s no-op on the provenanced `web_fetch` path, `replay_protocol_fidelity` not being
+authorable). Reach for this list when debugging a run's behavior, that one while authoring `assert:`.
+**The two lists are numbered independently** — a bare "gotcha N" means the list you are reading.
 
 1. **An assertion passed but tested nothing on the PR gate.** *Why:* on a manifest-less cassette
    `replay` skips filesystem/egress keys (`file_exists`, `user_visible_artifact`, `artifact_json`,
@@ -857,7 +890,7 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
 
 3. **A multi-key `assert:` item is an AND.** A single list item with more than one key passes iff
    **every** key passes. *Fix:* one concern per item unless you genuinely mean conjunction (and a
-   mixed-class conjunction still loses its filesystem half on replay — see gotcha 1).
+   mixed-class conjunction still loses its filesystem half on replay — see gotcha 1 below).
 
 4. **`tool_called` doesn't mean "attempted".** Tool counts are authoritative and de-duped: a tool
    that was *requested then denied* does **not** register as called. *Fix:* don't assert `tool_called`

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 3.1.0` (baseline `desktop-1.40609.0`).
+Self-contained reference. Tracks `cowork-harness 3.2.0` (baseline `desktop-1.40609.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
 copy-pasted recipe that omits the input takes a major bump with no say in it. `^2` holds the major, needs no
 patch number to remember, and only wants a human decision at the next major. Pin an exact version
-(e.g. `version: "3.1.0"`) instead when you want byte-reproducible CI.
+(e.g. `version: "3.2.0"`) instead when you want byte-reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -67,7 +67,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@^3.1.0"
+- run: npm i -g "cowork-harness@^3.2.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
@@ -312,7 +312,9 @@ A typical skill repo runs four stages, fastest/cheapest first:
 
    This is the shape a CI step wants: **silent on success (no output, exit 0), loud and specific on
    failure** — `--quiet` suppresses the readiness preview but never the `✗ broken:` lines, which name the
-   offending file *and* the rejected key, and the step still exits 1. A scenario that lints with only
+   offending file *and* the rejected key, one line per file, and the step still exits 1. (Silent is
+   literal only when your scenarios name a `fidelity:`; one still in the deprecation window prints one
+   defaulted-fidelity notice per scenario.) A scenario that lints with only
    warnings can still be unloadable, so a green `lint` is not evidence the suite runs.
 3. **Scenarios (replay)** — `cowork-harness replay cassettes/` on every PR (the committed `*.cassette.json`).
    Token-free; content + structure + gate delivery.
@@ -342,7 +344,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@^3.1.0"
+      - run: npm i -g "cowork-harness@^3.2.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -371,7 +373,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@^3.1.0"
+        run: npm i -g "cowork-harness@^3.2.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 3.1.0` (baseline `desktop-1.40609.0`). This is **not** a trim of the full
+Tracks `cowork-harness 3.2.0` (baseline `desktop-1.40609.0`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reproduction discipline, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 3.1.0` (baseline `desktop-1.40609.0`).
+Self-contained reference. Tracks `cowork-harness 3.2.0` (baseline `desktop-1.40609.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,6 +1,6 @@
-# Scenario & session schema, assertion catalog, web_fetch, full gotchas
+# Scenario & session schema, assertion catalog, web_fetch, authoring gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 3.1.0`
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 3.2.0`
 (baseline `desktop-1.40609.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 
@@ -22,7 +22,7 @@ Everything below is the full field + assertion catalog.
 - [Replay class — which assertions survive `replay`](#replay-class)
 - [The web_fetch model](#the-web_fetch-model)
 - [Scenario YAML vs the pytest lane](#scenario-yaml-vs-the-pytest-lane)
-- [Full gotcha list](#full-gotcha-list)
+- [Authoring gotcha list](#authoring-gotcha-list)
 
 ## Scenario YAML
 
@@ -307,7 +307,7 @@ same set live from the schema.
 | `transcript_not_matches: <regex>` | it does not match (e.g. no leaked stack trace) **Sees top-level `assistant_text` only — it excludes every `tool_use`/`tool_result`**, so text emitted inside a tool call (a gate question, an option label or description) can never match; use `question_context`/`question_asked` or `tool_result_contains` for those. |
 | `file_exists: <path>` | the path exists under the run's `work/` (anchored at `mnt/`, e.g. `outputs/x.md`). For a user-facing deliverable prefer `user_visible_artifact` — with a connected folder the file lands in `mnt/<folder>` (= `{{workspaceFolder}}`), not `mnt/outputs`, so `file_exists: outputs/x.md` misses it |
 | `user_visible_artifact: <path>` | exists **and** under a user-visible root (`outputs/` + each connected folder's mount name) — the right primitive for a workspace deliverable when a folder is connected |
-| `no_delete_in_outputs: true` | no delete op touched `mnt/outputs` — **only `true` is valid**; `false` is rejected (omit to allow deletes) |
+| `no_delete_in_outputs: true` | no delete op touched `mnt/outputs` — **only `true` is valid**; `false` is rejected. **Omitting it does NOT allow deletes** — a detected delete still fails via the `outputs_delete` verdict signal, which fires *because* the key was not authored; use `allow_outputs_delete: true` to accept an intended one. Detection is a post-run bash-command scan, not mount enforcement, so a green means none was *detected* |
 | `no_delete_in_mounts: true` | no delete op touched ANY delete-denied mount — `outputs` plus every `rw` connected folder — except those waived by `allow_delete_in`. Production denies `unlink`/`rmdir` on every such mount, so `no_delete_in_outputs` covers only part of the real rule. **only `true` is valid** |
 | `no_unexpected_files: [<glob>, …]` | every **newly created** file under a user-visible root matches ≥1 glob (workRoot-relative paths; `**` = whole path segment for any depth — use `outputs/handoff/**` for per-run subdirs); `[]` = no new files; **new-files-only** — overwriting a pre-existing file in place is invisible (use content-level producer stamping); live/verify-run without a pre-run manifest ⇒ evidence-unavailable hard-fail (live runs capture the baseline only when this key is asserted; recordings always capture); an **incomplete post-run filesystem walk** (an unreadable subtree — a permission/I-O error) also fails evidence-unavailable rather than reporting "no strays" over a partial tree — distinct from the missing-manifest case (a `--resume` run), which fails for a different reason; captured on every live sandbox tier including microvm (its outputs are snapshotted from the VM into the run dir); replay needs `cassette.preRunPaths` (≥0.24 recordings) — cassettes without it **exclude** the key with a loud warning. ⚠️ **Not a stand-in for "file X must not exist":** there is no negative-existence key, and inverting this one is a different claim — it is new-files-only (a pre-existing file is invisible to it) and needs a pre-run manifest. Assert the content-level consequence instead (`tool_not_called`, or the absent `artifact_json`) rather than widening an allowlist for every incidental lock/temp file |
 | `input_unmodified: <glob>` or `[<glob>, …]` | a single glob or a list; every **pre-existing** file (incl. uploaded files under `uploads/**`) whose workRoot-relative path matches ≥1 glob keeps an unchanged content hash after the run — the in-place-mutation companion to `no_unexpected_files`'s new-files check (`[]` is rejected by the schema — list at least one glob); a glob that matches **no** pre-run path fails loud (a typo or renamed mount would otherwise verify zero files and pass vacuously); a matched file that was deleted counts as a content change (fails); live/verify-run without a pre-run hash manifest ⇒ evidence-unavailable hard-fail (a `--resume` run); captured on every live sandbox tier including microvm; replay needs `cassette.preRunHashes` — cassettes without it **exclude** the key with a loud warning; on replay it compares against the manifest's recorded `sha256`, never a re-hash of the materialized tree |
@@ -401,9 +401,9 @@ paraphrases (that re-records red). Structured JSON → assert it in YAML with **
 `path` + operator); use the pytest lane (`assert_artifact_json`) only for predicates too complex for a
 dotted path.
 
-**VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; most
+**VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; eleven
 are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
-`"success"`) and only four are **warn**-severity (informational, never flip pass/fail). Current signal
+`"success"`) and nine are **warn**-severity (informational, never flip pass/fail). All twenty signal
 codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 | Code | Severity | Meaning |
@@ -431,7 +431,7 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 A **fail**-severity signal does not change `result.result` (still `"success"`), but it DOES fail the
 overall run verdict and exit code — `assert result: success` alone won't catch it; check
-`result.verdict.signals[].severity` or the run's exit code. Only the five **warn** codes are truly benign.
+`result.verdict.signals[].severity` or the run's exit code. Only the nine **warn** codes are truly benign.
 
 ## Replay class
 
@@ -546,14 +546,14 @@ real predicate over a skill's **structured JSON output**:
 Find an artifact's real field paths by running once with `--keep`, then `cowork-harness inspect <run-dir>`
 (a shallow field preview of each JSON artifact) or by reading the JSON directly.
 
-## Full gotcha list
+## Authoring gotcha list
 
 The "✓ passed ≠ correct" landmines relevant to **scenario/assertion authoring**, as
 *symptom → why → fix*. `file:line` pointers track the version at the top of this file.
-**Scope note:** this is the assertion/replay-focused view; the **companion `SKILL.md`'s Gotchas
-section is the full landmine catalog** (it adds workflow/record/answer-path landmines this schema
-reference omits). Neither list is a strict superset of the other — reach for this one while authoring
-`assert:`, and SKILL's when debugging a run's behavior.
+**Scope note:** this is the assertion/replay-focused view; the companion `SKILL.md`'s Gotchas section
+is the broader one (it adds workflow/record/answer-path landmines this reference omits). **Neither list
+is a strict superset of the other** — reach for this one while authoring `assert:`, and SKILL's when
+debugging a run's behavior. The two are **numbered independently**: a bare "gotcha N" means this list.
 
 1. **Replay skips filesystem/egress assertions (two shapes) — with a loud warning.** *Full skip:* a pure
    live-only `egress_*`/`no_delete_in_outputs`/`self_heal_ran`/`transcript_no_host_path` item on a
@@ -591,7 +591,8 @@ reference omits). Neither list is a strict superset of the other — reach for t
    one concatenated string → use `[\s\S]`, not `.`. `transcript_matches` is case-insensitive.
 
 7. **Multi-key assertion item = AND.** Passes iff every key passes. One concern per item unless
-   conjunction is intended (and a mixed-class conjunction loses its filesystem half on replay — gotcha 1).
+   conjunction is intended (and a mixed-class conjunction loses its filesystem half on replay — gotcha 1
+   of THIS list; the two gotcha lists are numbered independently).
 
 8. **`tool_called` proves a tool ran, not that it was attempted.** Tool counts are authoritative and
    de-duped: a requested-then-denied tool does NOT register as called; the synthetic

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 3.1.0` (baseline `desktop-1.40609.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 3.2.0` (baseline `desktop-1.40609.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.
 

package/.claude/skills/cowork-harness/scripts/assertion-keys.json

@@ -122,5 +122,58 @@
     "Stop",
     "SubagentStop",
     "UserPromptSubmit"
-  ]
+  ],
+  "enums": {
+    "answers.decide": [
+      "allow",
+      "deny"
+    ],
+    "answers.else": [
+      "allow",
+      "deny"
+    ],
+    "answers.grant": [
+      "once",
+      "domain"
+    ],
+    "assert.path_denied.agent_scope": [
+      "main",
+      "subagent",
+      "any"
+    ],
+    "assert.path_denied.source": [
+      "pretooluse",
+      "can_use_tool",
+      "permission_denied"
+    ],
+    "assert.question_options.order": [
+      "exact",
+      "any"
+    ],
+    "assert.result": [
+      "success",
+      "error"
+    ],
+    "execution": [
+      "local",
+      "cloud-describe"
+    ],
+    "fidelity": [
+      "protocol",
+      "container",
+      "microvm",
+      "hostloop",
+      "cowork"
+    ],
+    "lane": [
+      "local",
+      "remote"
+    ],
+    "on_unanswered": [
+      "fail",
+      "prompt",
+      "llm",
+      "first"
+    ]
+  }
 }

package/.claude/skills/cowork-harness/scripts/scenario.py

@@ -23,7 +23,10 @@ lint flags (see references/scenario-schema.md for the why of each):
                                                   (runtime rejects at LOAD time; tier rules suppressed)
   E  `requires_capabilities` on `fidelity: protocol` (probe can't run → hard-fails
                                                       unless allow_missing_capability)
-  E  on_unanswered: agent / invalid value            (schema rejects `agent`)
+  E  `enum-value-invalid`      any enum-valued field carries a value the schema rejects (fidelity,
+                               execution, lane, on_unanswered, answers[].decide/else/grant,
+                               assert[].result/path_denied.*/question_options.order) — `agent` gets a
+                               `on_unanswered: agent` → `llm` rename hint
   E  authored `replay_protocol_fidelity` assertion   (replay-synthesized only)
   E  `assertions:` instead of `assert:`              (block ignored → every check no-ops)
   E  a presence assert + its absence sibling            (unsatisfiable; run/skill/record refuse it:
@@ -302,8 +305,82 @@ REGEX_KEYS = {
     "reference_read",
     "no_observed_reference_access",
 }
-VALID_ON_UNANSWERED = {"fail", "prompt", "first", "llm"}
-VALID_TIERS = ("protocol", "container", "microvm", "hostloop", "cowork")
+# The authoritative enum-value map: field id -> allowed values, generated from the Zod schemas into
+# `assertion-keys.json`'s `enums` block by walking BOTH the top-level ScenarioObject fields and the
+# nested answers[]/assert[] item shapes (gen-schema.ts's buildEnumMap). Backs the generic
+# `enum-value-invalid` lint rule below, and replaces two independent hand-copies that used to drift from
+# the schema on their own: VALID_TIERS (now `ENUM_VALUES["fidelity"]`) and the old bespoke
+# on_unanswered check's VALID_ON_UNANSWERED set (now `ENUM_VALUES["on_unanswered"]`).
+_EMBEDDED_ENUMS = {
+    "fidelity": ["protocol", "container", "microvm", "hostloop", "cowork"],
+    # "cloud-describe" IS a valid schema value here -- the runtime rejects it as RESERVED at load
+    # (src/run/execute.ts: no cloud runner exists yet), so it must never be handed back to an author as
+    # the FIX for some other invalid `execution:` value. See _enum_fix_values below, which is where that
+    # carve-out is applied -- this map itself stays a faithful mirror of what the schema accepts.
+    "execution": ["local", "cloud-describe"],
+    "lane": ["local", "remote"],
+    "on_unanswered": ["fail", "prompt", "llm", "first"],
+    "answers.decide": ["allow", "deny"],
+    "answers.else": ["allow", "deny"],
+    "answers.grant": ["once", "domain"],
+    "assert.result": ["success", "error"],
+    "assert.path_denied.source": ["pretooluse", "can_use_tool", "permission_denied"],
+    "assert.path_denied.agent_scope": ["main", "subagent", "any"],
+    "assert.question_options.order": ["exact", "any"],
+}
+
+
+def _load_enums():
+    """The authoritative enum-value map, from the generated assertion-keys.json sidecar. Falls back to
+    the embedded `_EMBEDDED_ENUMS` (kept equal to the generated map) with a loud warning if the file is
+    missing or predates the `enums` field."""
+    p = Path(__file__).resolve().parent / "assertion-keys.json"
+    try:
+        enums = json.loads(p.read_text(encoding="utf-8")).get("enums")
+        if not enums:
+            raise KeyError("enums")
+        return enums
+    except Exception:
+        print(
+            f"::warning:: assertion-keys.json missing or has no enums next to scenario.py ({p}) — "
+            "using a built-in enum-value map that may be stale (run `npm run schema`).",
+            file=sys.stderr,
+        )
+        return dict(_EMBEDDED_ENUMS)
+
+
+# field id -> allowed values (generated from the zod schemas; see _load_enums)
+ENUM_VALUES = _load_enums()
+VALID_TIERS = tuple(ENUM_VALUES.get("fidelity", _EMBEDDED_ENUMS["fidelity"]))
+
+
+def _enum_fix_values(field, allowed):
+    """Allowed values to SHOW an author as the fix for an invalid `field`. Identical to the validation
+    set except for `execution`, where `cloud-describe` is schema-valid but runtime-RESERVED (a load-time
+    error at record/run/skill — src/run/execute.ts) — offering it as a remedy would just relocate the
+    failure from `lint` to `record --dry-run` rather than resolve it."""
+    if field == "execution":
+        return [v for v in allowed if v != "cloud-describe"]
+    return allowed
+
+
+def _enum_finding(field, value, path):
+    """An `enum-value-invalid` Finding if `value` is not in `field`'s allowed set, else None. Unknown
+    `field`s (not in ENUM_VALUES) are silently skipped -- that would be a linter bug, not an authoring
+    one, and every field this is called with is a schema enum by construction."""
+    allowed = ENUM_VALUES.get(field)
+    if not allowed or value in allowed:
+        return None
+    hint = " (`agent` was renamed to `llm`)" if field == "on_unanswered" and value == "agent" else ""
+    # A present-but-null key reads as `fidelity: None` in Python; show the YAML the author wrote.
+    shown = "null" if value is None else value
+    return Finding(
+        "ERROR",
+        "enum-value-invalid",
+        f"`{field}: {shown}` is not a valid value{hint}.",
+        f"Use one of: {' | '.join(_enum_fix_values(field, allowed))}.",
+        path,
+    )
 
 # Gate-id tripwire: the `host-path-assert-cowork` WARN below embeds Cowork's
 # host-loop gate id in offline Python (the linter never reads a baseline). The
@@ -747,19 +824,54 @@ def lint_doc(doc, path, raw_lines):
                 )
             )
 
-    # E: retired/invalid on_unanswered
-    ou = doc.get("on_unanswered")
-    if ou is not None and ou not in VALID_ON_UNANSWERED:
-        extra = " (`agent` was renamed to `llm`)" if ou == "agent" else ""
-        findings.append(
-            Finding(
-                "ERROR",
-                "on-unanswered-invalid",
-                f"on_unanswered: {ou} is not a valid value{extra}.",
-                "Use one of: fail | prompt | first | llm (YAML). For a live model use on_unanswered: llm.",
-                path,
-            )
-        )
+    # E: enum-value-invalid — every enum-valued scenario field, top-level and nested. Schema-invalid
+    # values here are refused at load (`run`/`skill`/`record` all raise zod's `invalid_value`, exit 2)
+    # regardless of what `lint` says, so a scenario `lint --strict` calls clean while the loader hard-
+    # rejects it is exactly the false-green class this linter exists to catch. Generalizes what used to
+    # be a single bespoke check for `on_unanswered` alone — a fifth of the schema's actual enum surface
+    # (see ENUM_VALUES above); the `agent` -> `llm` rename hint survives as a field-specific case in
+    # _enum_finding.
+    # Membership, NOT `.get(...) is not None`: a key present with an empty value (`fidelity:` and
+    # nothing after it -- a one-keystroke slip) parses as null, which the loader rejects and which the
+    # retired on_unanswered check let through. Absent keys stay absent; they default legitimately.
+    for _field in ("fidelity", "execution", "lane", "on_unanswered"):
+        if _field in doc:
+            _value = doc[_field]
+            _f = _enum_finding(_field, _value, path)
+            if _f is not None:
+                findings.append(_f)
+    _answers = doc.get("answers")
+    if isinstance(_answers, list):
+        for _rule in _answers:
+            if not isinstance(_rule, dict):
+                continue
+            for _key in ("decide", "else", "grant"):
+                if _key in _rule:
+                    _value = _rule[_key]
+                    _f = _enum_finding(f"answers.{_key}", _value, path)
+                    if _f is not None:
+                        findings.append(_f)
+    for _item in items:
+        if "result" in _item:
+            _value = _item["result"]
+            _f = _enum_finding("assert.result", _value, path)
+            if _f is not None:
+                findings.append(_f)
+        _path_denied = _item.get("path_denied")
+        if isinstance(_path_denied, dict):
+            for _key in ("source", "agent_scope"):
+                if _key in _path_denied:
+                    _value = _path_denied[_key]
+                    _f = _enum_finding(f"assert.path_denied.{_key}", _value, path)
+                    if _f is not None:
+                        findings.append(_f)
+        _question_options = _item.get("question_options")
+        if isinstance(_question_options, dict):
+            if "order" in _question_options:
+                _value = _question_options["order"]
+                _f = _enum_finding("assert.question_options.order", _value, path)
+                if _f is not None:
+                    findings.append(_f)
 
     # E: authored replay_protocol_fidelity
     if "replay_protocol_fidelity" in assert_keys: