cowork-harness 1.22.0 → 1.24.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 1.22.0
-  tracks-harness: cowork-harness 1.22.0 (baseline desktop-1.28929.0)
+  version: 1.24.0
+  tracks-harness: cowork-harness 1.24.0 (baseline desktop-1.32352.0)
 ---
 
 # cowork-harness
@@ -22,8 +22,8 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.22.0` (baseline
-> `desktop-1.28929.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.24.0` (baseline
+> `desktop-1.32352.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## Preflight — make sure the harness can actually run
@@ -39,7 +39,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.22.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.22.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@>=1.22.0"`. **Pin `@>=1.22.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.24.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.24.0" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@>=1.24.0"`. **Pin `@>=1.24.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
   This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
   OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
@@ -293,6 +293,8 @@ them by what you're trying to prove:
 | the skill didn't error out of a tool | `tool_no_error: <regex>`, `max_tool_errors: <N>` |
 | it didn't waste repeated identical calls | `max_redundant_tool_calls: <N>` |
 | a deliverable reached the user | `user_visible_artifact: <path>` (+ `no_scratchpad_leak: true` if it delivers via `present_files` — **`container` only**) |
+| an internal name/path did **not** leak into a delivered file | `artifact_text: {artifact, not_contains}` — `artifact_json`'s companion for non-JSON bodies; literal path, no glob, so one entry per delivered surface |
+| a named path must **not** exist after the run | `file_absent: <path>` (**live/verify-run only**) — do NOT invert `no_unexpected_files`: that is an allowlist over *newly created* files and needs a pre-run manifest |
 | a to-do workflow finished | `all_tasks_completed: true`, `task_status: {match, status}` |
 | a skill / connector / tool was **offered** | `skill_available`, `connector_available`, `tool_available` (all `<regex>`) |
 | a skill actually **ran** (or must NOT) | `skill_triggered: <regex>`, `no_skill_triggered: <regex>` |
@@ -301,6 +303,7 @@ them by what you're trying to prove:
 | a pre-existing input wasn't mutated (incl. `uploads/**`) | `input_unmodified: <glob>` or `[<glob>, …]` (live/verify-run) |
 | no authored interactive artifact silently loses its Submit under Cowork | `no_lost_write_back: true` (**live-only**; static Tier A over the run's authored `.html`/`.py`/`.js`; per-scenario gate for the same class `analyze-skill` scans) |
 | a resource ceiling held | `max_peak_rss_bytes: <N>` (**live-only**) |
+| the user was **shown** the right choices, in order | `question_options: {when_question, equals}` — the option SET/ORDER a gate offered (`question_asked` matches the text only); order is compared by default |
 | a hook blocked / didn't block a tool | `hook_blocked: <regex>`, `no_hook_blocked: true` (replay needs a `controlOut` cassette) |
 | every MCP round-trip succeeded | `no_mcp_error: true` (**live-only**) |
 | a context compaction happened | `compaction_occurred: true` |
@@ -393,10 +396,28 @@ the discovery/encode/record dance entirely and answer gates **live during the re
 `record --decider-dir`/`--decider-llm` (the cassette is flagged non-deterministic but replays deterministically).
 `run` takes no `--dry-run`: to check that a scenario **loads** without spending, use
 `cowork-harness record <file.yaml> --dry-run` — it runs the real loader AND the same scenario-level
-refusals the real `record` applies (`on_unanswered: prompt`, and an unsatisfiable assert pairing), so it
-cannot green something a paid run would reject. On a directory it reports every offender and the batch
+refusals the real `record` applies (`on_unanswered: prompt`, and an unsatisfiable assert pairing) **plus the
+cassette-portability pre-flight below**, so it cannot green something a paid run would reject. On a directory it reports every offender and the batch
 cost estimate. `lint` checks the assertion invariants (both above).
 
+**Decide WHERE the cassette lives before you record it — a cassette cannot be moved afterwards.**
+Without `--out`, `record` writes `cassettes/<scenario-name-slug>.cassette.json` (gitignored by
+default); pass `--out <path>` to put it somewhere tracked, e.g. `examples/replays/<name>.cassette.json`.
+That choice is permanent: the cassette rewrites `scenario.session` and `scenarioSource` **relative to
+its own directory** at record time, so moving the file later — a different `--out`, a `git mv`, a copy
+into another repo — leaves those unresolvable and
+`verify-cassettes` reports `unverifiable-skill` ("can't verify ⇒ not green", exit 3) until you
+re-record at the new location. **`record` now says so BEFORE it spends:** a pre-flight — at the same
+pre-spend point as the host-inventory refusal, and in `record --dry-run`, so the rehearsal is free —
+warns when the cassette would be written outside the scenario's tree, or when `session:` itself lives
+outside it (an absolute or `~` path: the mirror case, invisible to a check that only looks at where the
+cassette lands). A warning, not a refusal — an out-of-tree throwaway cassette is legitimate; what was
+missing was anything saying so while you could still act. Related: recording at a **host-inheriting** tier
+(`protocol`/`hostloop`/`cowork`→hostloop) into a repo-visible path is refused outright (gotcha 25).
+The clean answer there is `fidelity: container` (sealed, `HOME=/tmp`, nothing to leak) — **not**
+redirecting `--out` outside the repo and moving the file in afterwards, which trades a loud refusal
+for a permanently unverifiable cassette.
+
 **Author answers WITHOUT re-paying — the cheap loop.** You don't need a fresh paid record to discover a
 scenario's gates or their labels: `--keep` ONE run, then `cowork-harness trace <run-dir> --view questions`
 (and `verify-run`) read the gates + offered option labels out of that run's `events.jsonl` for free. Iterate
@@ -548,6 +569,42 @@ The full 17-code signal table (severity + per-signal opt-out) is in
 [`references/scenario-schema.md`](./references/scenario-schema.md); [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md) (repo-only) carries
 the fuller narrative.
 
+### Measure — before/after, with/without (`--repeat`, `--ablate-skill`)
+
+A single green proves the run passed **once**. Two questions need more than that, and both have a
+discipline that is cheap to follow and expensive to skip.
+
+**"Did it pass, or pass once?"** → `--repeat N` (2-100, on `skill` AND `run`) samples the same
+skill+prompt N times and prints a variance rollup instead of a single verdict. `--min-pass-rate` sets
+the batch threshold, `--stop-on-diverge` stops the moment flakiness is proven, `--max-budget-usd` caps
+spend.
+
+**"Does the skill actually help?"** → `--ablate-skill` runs the prompt with every skill/plugin
+discovery source removed, so the agent answers from its own priors. **It is ONE arm, not a paired
+experiment**: this invocation is the control. Run the same prompt a second time *without* the flag for
+the treatment arm and compare them yourself. Composed with `--repeat 5` it produces **5 ablated runs
+and 0 treatment runs** — N samples of the control, which is the intended reading and is not an A/B.
+Every ablated run is stamped `ablated: true` in `result.json`; a run that isn't stamped is a real run.
+What the harness gives you here is the run execution and the control arm — designing the comparison
+(scrubbing giveaways, shuffling, judging blind, unblinding only after grading) is still yours.
+
+**Measurement hygiene — four things that silently invalidate a batch:**
+
+1. **Pin the model.** With no `model:` in the session (or `--model` on the `skill` lane) the run uses
+   whatever the staged agent binary defaults to — not a harness constant, and it can move under a
+   baseline bump. Read `result.json`'s `models` back before believing any cross-run comparison — and when
+   you do, **ignore any entry wrapped in angle brackets**: `<synthetic>` is the agent marking a turn it
+   fabricated locally (no API call), not a model, so two runs of the same pinned model can differ on this
+   array purely by whether such a turn occurred.
+2. **Commit the skill first.** `fingerprint.skillHash` is content-exact, so an edit mid-batch silently
+   splits your dataset into two generations — and a hash whose source was never committed identifies a
+   generation that is unrecoverable. `stats --group-by skill-hash` separates them after the fact;
+   nothing recovers the source.
+3. **Check which arm you actually ran** before analysing anything: `ablated` and
+   `context.availableSkills` in each `result.json`.
+4. **Read `skillsInvoked`.** A rep where the skill never triggered is a measurement of the model, not
+   of your skill — discard or re-run it.
+
 ### Checking whether a background run is alive
 
 Never use `ps aux` to check on a `cowork-harness` run you launched in the background — it only sees
@@ -741,11 +798,13 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
 
 1. **An assertion passed but tested nothing on the PR gate.** *Why:* on a manifest-less cassette
    `replay` skips filesystem/egress keys (`file_exists`, `user_visible_artifact`, `artifact_json`,
-   `egress_*`, `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`); a *mixed* item like
+   `artifact_text`, `egress_*`, `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`); a
+   *mixed* item like
    `{result, egress_denied}` greens on `result` while its `egress_denied` half is dropped. (`record`
    snapshots an `artifacts` manifest, which makes
-   `file_exists`/`user_visible_artifact`/`artifact_json`/`computer_links_resolve`
-   replay-checkable — but the live-only egress keys stay skipped.) *Fix:* put egress/live-only checks on
+   `file_exists`/`user_visible_artifact`/`artifact_json`/`artifact_text`/`computer_links_resolve`
+   replay-checkable — but the live-only egress keys stay skipped, and `file_absent` is never
+   replay-checkable at all: proving absence needs an exhaustive, healthy walk a manifest does not record.) *Fix:* put egress/live-only checks on
    a live gate; keep one concern per `assert:` item; run the linter. The harness warns loudly on skip.
 
 2. **A steered gate answer never reached the model.** *Why:* `serializeDecision` must emit
@@ -754,8 +813,8 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
    channel: scripted `choose:` list, in-band `--decider-dir` via a repeated `--choose` / a JSON-array
    reply, and `--decider-cmd` via a JSON-array reply — all deliver the same `", "`-joined wire shape.
    Free-text "Other" via `answer:`. Do NOT hand-write a multiSelect reply as a bare comma-joined
-   string — send an array; a scalar is treated as one selection.) `question_asked` / `questions_count_max` /
-   `gate_answers_delivered` only evaluate on replay **with a `controlOut` cassette** — re-record an
+   string — send an array; a scalar is treated as one selection.) `question_asked` / `question_options` /
+   `questions_count_max` / `gate_answers_delivered` only evaluate on replay **with a `controlOut` cassette** — re-record an
    old cassette or they're excluded (loudly), not vacuously passed. `gate_answers_delivered` *fails*
    on unobserved delivery (absence of evidence is failure, not neutral).
 
@@ -865,7 +924,9 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
 14. **A positional `choose` (`first` / index) is order-dependent.** `choose: "2"` survives label drift
     but NOT option *re-ordering* — if the gate presents its options in a different order run-to-run, the
     index lands on a different option (a silent re-record flake). Prefer an exact label when order is
-    stable; `lint` flags positional `choose` with an advisory.
+    stable; `lint` flags positional `choose` with an advisory. Unstable option order is also what the
+    **user** sees — a reordered gate puts a different choice in the default slot — so pin what was shown
+    with `question_options`, rather than only hardening the answer rule against it.
 15. **A scripted `choose:` matching no offered option HARD-fails the run — `on_unanswered: first` does NOT
     backstop it.** This is distinct from an *unanswered* gate (no rule matched → falls to `on_unanswered`): a
     rule that DID match the gate but whose `choose:` names a label the gate never offered (the model reworded
@@ -970,6 +1031,26 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
     scanner's `host-inventory` class on an already-committed cassette. Passing one where the other command
     wants it fails as an unrecognized flag — they don't interchange. Depth: `references/ci-recipe.md`.
 
+26. **A `skill`-lane `PASS` does not mean the skill ran, or that the run was the one you wanted.** *Why:*
+    an open-ended `skill` run has no `assert:` block, so its verdict reports only that **no guard fired**
+    (no error, stall, host-path leak, `outputs/` delete, permissive auto-allow or capability gap). On
+    `run` the same word additionally means *your assertions held*; on `skill --repeat N`, `PASS — N/N`
+    means N runs cleared the guards — it says nothing about which model served them, whether the skill
+    was invoked, or whether they were the ablated arm. *Fix:* read the three fields the record already
+    carries before drawing any conclusion — `skillsInvoked` / `skillActivity` (was it invoked at all),
+    `models` (which model), `ablated` + `context.availableSkills` (which arm). An answer that reads
+    exactly like skill output is not evidence: the skill's own source is mounted where the model can
+    read it — in production too — so on a self-referential prompt it may read `SKILL.md` and answer
+    directly, with `skillActivity` empty.
+
+27. **`allow_stall: true` is a scenario assertion, so the `skill` lane cannot use it.** *Why:* the
+    `stalled` guard fires when a run's final message ends in `?` with no productive tool call after the
+    last gate — which includes a complete answer that closes by *offering* a follow-up ("want me to run
+    this through a structured pass?"). The documented opt-out lives in an `assert:` block, and an
+    open-ended `skill` run has none, so the failure message names a remedy that lane can't perform.
+    *Fix:* on `skill`, read the final message before believing `stalled`, or move the check to a
+    `run` scenario where `allow_stall: true` is authorable.
+
 For the assertion catalog, the YAML schema, the fidelity/answer tables, and the CI recipe, read the
 files in `references/` (the gotchas above are the full list; the references repeat only the
 assertion/replay-relevant ones).

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 1.22.0` (baseline `desktop-1.28929.0`).
+Self-contained reference. Tracks `cowork-harness 1.24.0` (baseline `desktop-1.32352.0`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -13,7 +13,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 ```
 
 The Action's `version` input defaults to `latest` — intentional so a copy-pasted recipe tracks the current
-release; pin an exact version (e.g. `version: "1.22.0"`) for reproducible CI.
+release; pin an exact version (e.g. `version: "1.24.0"`) for reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -32,7 +32,7 @@ jobs:
       - uses: actions/checkout@v4
       - name: Stage the agent binary (official channel, sha256-verified — see https://github.com/yaniv-golan/cowork-harness/blob/main/docs/maintenance.md)
         run: |
-          V=2.1.227   # match your scenario's pinned baseline's agentVersion
+          V=2.1.229   # match your scenario's pinned baseline's agentVersion
           curl -fSL "https://downloads.claude.ai/claude-code-releases/$V/linux-arm64/claude" -o "$RUNNER_TEMP/claude-$V"
           chmod +x "$RUNNER_TEMP/claude-$V"
           # verify against the committed baseline's sha256 (baselines/desktop-*.json → agentBinary.sha256)
@@ -58,13 +58,17 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@>=1.22.0"
+- run: npm i -g "cowork-harness@>=1.24.0"
 - run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
                                                     # no silent false-greens. WITHOUT --strict this
                                                     # step cannot fail on a WARN-class rule (e.g.
                                                     # vacuous-gate-assert) — it would print the
                                                     # finding and still exit 0. --min-severity WARN
-                                                    # keeps the advisory INFO class advisory.
+                                                    # keeps the advisory INFO class advisory — pair them.
+                                                    # Bare `lint --strict` fails on INFO too, which reds
+                                                    # on scenarios that are perfectly fine. (`lint-skill
+                                                    # --strict` never fails on INFO: same flag name, a
+                                                    # different rule. Do not carry one over to the other.)
 - run: cowork-harness verify-cassettes cassettes/    # privacy + staleness — FAILS on a stale recording
                                                     # ALSO fails on a leaked host inventory: recording at
                                                     # protocol/hostloop freezes YOUR machine's MCP servers,
@@ -114,16 +118,22 @@ The split is not just about tokens — it decides **where each lane can run**:
   `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `skill_triggered`, `no_skill_triggered`,
   `max_cost_usd`, `max_tokens`, `tool_calls_max`, `result`, and the verdict modifiers
   `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
-  `allow_stall` (no-op passes); plus the gate keys `question_asked` / `questions_count_max` /
-  `gate_answers_delivered` **if** the cassette has `controlOut`, and the manifest keys
-  (`file_exists` / `user_visible_artifact` / `artifact_json`) **if** it carries an artifact manifest.
+  `allow_stall` (no-op passes); plus the gate keys `question_asked` / `question_options` /
+  `questions_count_max` / `gate_answers_delivered` **if** the cassette has `controlOut`, and the manifest keys
+  (`file_exists` / `user_visible_artifact` / `artifact_json` / `artifact_text`) **if** it carries an artifact
+  manifest. `file_absent` is in neither class — it is live/verify-run only.
   **That list is illustrative, not the authoritative set** — more keys are replay-checkable than fit a
   paragraph, and a hand-typed enumeration is exactly what goes stale. For the current set, ask the CLI:
 
   ```bash
-  cowork-harness assertions --list --output-format json   # every key, with its replay class
+  cowork-harness assertions --list --output-format json   # every key + its one-line semantics
   ```
 
+  It emits `{key, description}` — there is no structured replay-class field to filter on; the
+  live-only / manifest / `controlOut` preconditions are stated in each key's `description` prose and,
+  in full, in the catalog's replay-class tables in
+  [`references/scenario-schema.md`](./scenario-schema.md).
+
   **`replay --mutate`** is a distinct, reporting-only diagnostic on this same lane: it perturbs each
   recorded JSON artifact value one at a time, re-runs the assertions against the perturbed cassette,
   and reports which perturbations NOTHING caught — those are the fields your `assert:` block leaves
@@ -279,7 +289,7 @@ jobs:
         with: { node-version: '24' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@>=1.22.0"
+      - run: npm i -g "cowork-harness@>=1.24.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -308,7 +318,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@>=1.22.0"
+        run: npm i -g "cowork-harness@>=1.24.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:
@@ -335,6 +345,31 @@ scenario = `result === "success" && assertions.every(pass)`. Exit code is non-ze
 fails or a run errors, so a plain `cowork-harness run scenarios/` is already CI-ready without parsing
 JSON.
 
+**Telling *why* a run failed, without scraping stderr.** Each result carries a `verdict` whose
+`failures[]` is the one place every failure reason is enumerated, in one shape (the same object lands
+in `result.json` and in the stdout envelope, by construction). Each entry is
+`{kind, assertion?: "<key>", message}`, and **`kind` is the discriminator**:
+
+- **`assertion`** — one of *your* `assert:` items failed; `assertion` names its key (`file_exists`,
+  `semantic_matches`, …).
+- **`guard`** — a signal you didn't author: `stalled`, `permissive_auto_allow`, `missing_capability`, a
+  host-path leak, an `outputs/` delete, an infra or transport error, an unanswered gate.
+- **`staleness`** — on `replay --strict` / `--assert-from` / `--reassert`, skill or baseline drift,
+  which those modes escalate to a hard failure on purpose (frozen events must not green an edited
+  assert against a skill whose current source produces something else).
+- **`cassette-format`** — the cassette is too new for this build to interpret.
+- **`coverage`** — a `verify-run` answer-coverage miss: a gate the run fired that your `answers:` block
+  does not cover.
+
+So `jq '[.verdict.failures[] | select(.kind=="assertion")]'` answers "did MY assertions pass?" and
+`select(.kind=="staleness")` answers "is the cassette stale?", from the envelope alone. Do not infer
+either from the exit code: every kind lands on exit 1.
+
+> **Do not filter on whether `assertion` is present.** That was the only discriminator before `kind`
+> existed and it never worked in both directions: `coverage` entries carry a key too (an internal
+> `answer_coverage` marker), so they read as authored asserts, while `guard`, `staleness` and
+> `cassette-format` all arrive key-less and indistinguishable from one another.
+
 **For the commands in this recipe, stdout carries the machine envelope and nothing else.** Without
 `--output-format json`, `run` / `record` / `replay` / `verify-cassettes` / `status` write their whole
 human rendering — warnings, verdict, `status`'s summary line — to **stderr**, and stdout stays empty.

package/.claude/skills/cowork-harness/references/critique.md

@@ -1,6 +1,6 @@
 # Critique — the facts a plugin install can't otherwise reach
 
-Tracks `cowork-harness 1.22.0` (baseline `desktop-1.28929.0`). This is **not** a trim of the full
+Tracks `cowork-harness 1.24.0` (baseline `desktop-1.32352.0`). This is **not** a trim of the full
 [`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
 flags, cost, reproduction discipline, known limitations all live there). This file covers exactly what a
 plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 1.22.0` (baseline `desktop-1.28929.0`).
+Self-contained reference. Tracks `cowork-harness 1.24.0` (baseline `desktop-1.32352.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 
@@ -210,7 +210,7 @@ up often enough to spell out:
 - **A green `replay` proves "same as when recorded," not "correct today."** `replay` never touches
   a filesystem or network — it re-evaluates assertions from the frozen cassette. A fixed set of
   keys is live-only and **skipped outright** on replay (absent from `assertions[]`, not vacuously
-  passed): `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`, `egress_denied`,
+  passed): `file_absent`, `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`, `egress_denied`,
   `egress_allowed`, `no_mcp_error`, `max_peak_rss_bytes`, `semantic_matches`, `no_lost_write_back`, and `expect_denied`.
   Everything else that *is* evaluated is checked against the **recording**, not fresh behavior — a
   green replay says the skill produced these events when it was recorded, not that it still does

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,7 +1,7 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.22.0`
-(baseline `desktop-1.28929.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.24.0`
+(baseline `desktop-1.32352.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
 [`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
 
 **Minimal scenario** — `prompt` is the only required field:
@@ -103,6 +103,13 @@ allow_host_writes: true             # OPTIONAL — required to run `hostloop` fi
 Relative paths resolve from the file's own directory, so a scenario + session + referenced files
 form a relocatable bundle. `~` expands to home.
 
+> **A recorded cassette is NOT part of that bundle and is NOT relocatable.** It rewrites its own
+> references relative to **its own** directory at record time (`scenario.session` and
+> `scenarioSource`), so moving it afterwards — a different `--out`, a
+> `git mv`, a copy into another repo — leaves them unresolvable and `verify-cassettes` reports
+> `unverifiable-skill` ("can't verify ⇒ not green", exit 3) until you re-record at the new location.
+> Decide where a cassette will live *before* you record it.
+
 ## Session YAML
 
 A session (`sessions/*.yaml`) captures everything you'd configure in Cowork **before the first
@@ -111,7 +118,15 @@ session = your setup.**
 
 ```yaml
 # model & reasoning
-model: claude-opus-4-8           # omit for the agent default
+model: claude-opus-4-8           # omit ONLY to track the agent default: with no `model:` the harness emits
+                                  # no --model flag at all, so the run uses whatever the STAGED AGENT BINARY
+                                  # defaults to — not a harness constant, and it can move under a baseline
+                                  # bump. PIN IT for anything you will compare (a --repeat batch, a
+                                  # before/after, a with/without); read result.json's `models` back to confirm,
+                                  # IGNORING any `<…>`-wrapped entry (`<synthetic>` = a turn the agent
+                                  # fabricated locally, not a model id).
+                                  # On the ad-hoc `skill` lane there is no session file, so `--model <id>`
+                                  # (or COWORK_HARNESS_MODEL) is the ONLY way to pin it.
 account_name: my-account         # OPTIONAL — display name rendered into {{accountName}} / the prompt's
                                     # "User name:" line; NOT a credential/identity selector (see src/prompt.ts,
                                     # https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md)
@@ -282,9 +297,11 @@ same set live from the schema.
 | `user_visible_artifact: <path>` | exists **and** under a user-visible root (`outputs/` + each connected folder's mount name) — the right primitive for a workspace deliverable when a folder is connected |
 | `no_delete_in_outputs: true` | no delete op touched `mnt/outputs` — **only `true` is valid**; `false` is rejected (omit to allow deletes) |
 | `no_delete_in_mounts: true` | no delete op touched ANY delete-denied mount — `outputs` plus every `rw` connected folder — except those waived by `allow_delete_in`. Production denies `unlink`/`rmdir` on every such mount, so `no_delete_in_outputs` covers only part of the real rule. **only `true` is valid** |
-| `no_unexpected_files: [<glob>, …]` | every **newly created** file under a user-visible root matches ≥1 glob (workRoot-relative paths; `**` = whole path segment for any depth — use `outputs/handoff/**` for per-run subdirs); `[]` = no new files; **new-files-only** — overwriting a pre-existing file in place is invisible (use content-level producer stamping); live/verify-run without a pre-run manifest ⇒ evidence-unavailable hard-fail (live runs capture the baseline only when this key is asserted; recordings always capture); an **incomplete post-run filesystem walk** (an unreadable subtree — a permission/I-O error) also fails evidence-unavailable rather than reporting "no strays" over a partial tree — distinct from the missing-manifest case (a `--resume` run), which fails for a different reason; captured on every live sandbox tier including microvm (its outputs are snapshotted from the VM into the run dir); replay needs `cassette.preRunPaths` (≥0.24 recordings) — cassettes without it **exclude** the key with a loud warning |
+| `no_unexpected_files: [<glob>, …]` | every **newly created** file under a user-visible root matches ≥1 glob (workRoot-relative paths; `**` = whole path segment for any depth — use `outputs/handoff/**` for per-run subdirs); `[]` = no new files; **new-files-only** — overwriting a pre-existing file in place is invisible (use content-level producer stamping); live/verify-run without a pre-run manifest ⇒ evidence-unavailable hard-fail (live runs capture the baseline only when this key is asserted; recordings always capture); an **incomplete post-run filesystem walk** (an unreadable subtree — a permission/I-O error) also fails evidence-unavailable rather than reporting "no strays" over a partial tree — distinct from the missing-manifest case (a `--resume` run), which fails for a different reason; captured on every live sandbox tier including microvm (its outputs are snapshotted from the VM into the run dir); replay needs `cassette.preRunPaths` (≥0.24 recordings) — cassettes without it **exclude** the key with a loud warning. ⚠️ **Not a stand-in for "file X must not exist":** there is no negative-existence key, and inverting this one is a different claim — it is new-files-only (a pre-existing file is invisible to it) and needs a pre-run manifest. Assert the content-level consequence instead (`tool_not_called`, or the absent `artifact_json`) rather than widening an allowlist for every incidental lock/temp file |
 | `input_unmodified: <glob>` or `[<glob>, …]` | a single glob or a list; every **pre-existing** file (incl. uploaded files under `uploads/**`) whose workRoot-relative path matches ≥1 glob keeps an unchanged content hash after the run — the in-place-mutation companion to `no_unexpected_files`'s new-files check (`[]` is rejected by the schema — list at least one glob); a glob that matches **no** pre-run path fails loud (a typo or renamed mount would otherwise verify zero files and pass vacuously); a matched file that was deleted counts as a content change (fails); live/verify-run without a pre-run hash manifest ⇒ evidence-unavailable hard-fail (a `--resume` run); captured on every live sandbox tier including microvm; replay needs `cassette.preRunHashes` — cassettes without it **exclude** the key with a loud warning; on replay it compares against the manifest's recorded `sha256`, never a re-hash of the materialized tree |
 | `self_heal_ran: <bool>` | a plugin-root self-heal script was (not) invoked |
+| `file_absent: <path>` | the named path does **not** exist under the work root after the run — the direct negative-existence key. Do NOT invert `no_unexpected_files` for this: that is an allowlist over NEW files only, and it needs a pre-run manifest. **LIVE/verify-run only** (a cassette records no walk health, so absence is unprovable on replay); evidence-unavailable on `lane: remote` / `preRunOrigin: remote-unavailable` |
+| `artifact_text: {artifact, contains?, not_contains?, matches?, not_matches?}` | assert over a delivered artifact's TEXT body — `artifact_json`'s companion for non-JSON files, and how you prove an internal name did not leak into a file the user receives. Literal path (no glob), so one entry per delivered surface. Manifest-class; body-less / symlinked / over-cap targets fail evidence-unavailable, and a non-UTF-8 body fails the NEGATIVE matchers rather than passing against bytes it never read |
 | `no_lost_write_back: true` | fails if the run authored an interactive HTML artifact (or a `.py`/`.js` generator of one) whose **relative** Submit/POST write-back is lost under Cowork (served from Cowork's own origin → resolves non-ok, a "Saved!" is silently false). Runs the shipped **static Tier A** analyzer over the files the run authored (diffed vs the pre-run manifest). A lost write-back on an **added** agent-authored source (`outputs/`, scratchpad) **fails**; a **pre-existing** file the skill only modified on a read-write mount is **advisory**; `-suspect` findings surface but pass. **Only `true` is valid**. **Live/verify-run only** — skipped-loud on replay; runs on every live sandbox tier including **microvm** (its outputs are snapshotted from the VM into the run dir); could-not-verify (fail-closed) on a `--resume` scratchpad or an unanalyzable candidate |
 | `tool_called: <glob>` | a tool the agent ran matched this **glob** — `*` = any run, `?` = one char, exact when literal, anchored + case-sensitive. Exact name (`Write`) matches only that tool; `mcp__workspace__*` matches any workspace tool. GLOB, not regex (`.` is literal) — an empty glob, or one containing a regex/brace-expansion metacharacter (`.*`, `.+`, `\|`, `()`, `[]`, `+`, `^`, `$`, `{}`, `\d`/`\w`/`\s`/`\b`), is **rejected at load** (a hard schema error, not a runtime warning) — it would match no real tool name and pass a `_not_`/`_absent` assert vacuously. Applies whether the glob comes from an authored scenario or a recorded cassette's frozen assert. The bundled `scenario.py lint` does NOT perform this check — only the harness enforces it, at actual load (`run`/`skill`/`record`) |
 | `tool_not_called: <glob>` | NO tool the agent ran matched this glob (`mcp__*` = "no MCP tool ran"). Same glob semantics as `tool_called`, including the empty/regex-ish rejection |
@@ -312,7 +329,7 @@ same set live from the schema.
 `skill_triggered`/`no_skill_triggered`, `skill_available`, `connector_available`, `skill_tool_used`,
 `subagent_type` are **regex** (unanchored, case-insensitive). So `tool_called: mcp__workspace__*` (glob) but
 `tool_available: mcp__workspace__.*` (regex) — a `.*` in a `tool_called` glob is a load-time schema error, not silently-matches-nothing.
-| `skill_tool_used: {skill, tool}` | a tool whose name matches `tool` ran inside a skill-activation window whose `skillId` matches `skill` (`RunResult.skillActivity`) — evidence-unavailable if skill-activity telemetry is absent; heuristic for inline skills (a sticky, sequential window matching the agent's `activeSkill` scope, not an exact per-tool boundary) |
+| `skill_tool_used: {skill, tool}` | a tool whose name matches `tool` ran inside a skill-activation window whose `skillId` matches `skill` (`RunResult.skillActivity`) — evidence-unavailable if skill-activity telemetry is absent; heuristic for inline skills (a sticky, sequential window matching the agent's `activeSkill` scope, not an exact per-tool boundary). **Scope:** the window's tool counts **include sub-agent calls** made during it, so this key can't say which agent called (`subagent_tool_used` is the sub-agent-only claim), and it matches tool **names** only — never the path/args, so "did it read *this* file" isn't expressible (per-sub-agent reads are recorded at `subagents[].referencesRead`, readable but not assertable) |
 | `max_cost_usd: <N>` | the run's SDK-reported cost is ≤ N USD — evidence-unavailable if cost telemetry is absent. **Replay asserts the frozen recording's cost, not fresh spend** — a real regression needs a live `run` |
 | `max_tokens: <N>` | `usage.input_tokens + usage.output_tokens` ≤ N (cache tokens excluded) — same replay caveat as `max_cost_usd` |
 | `tool_calls_max: <N>` | total top-level tool calls (sum of `toolCounts`) ≤ N — meaningfully replay-checkable (re-drive recomputes `toolCounts` deterministically) |
@@ -327,7 +344,8 @@ same set live from the schema.
 | `task_status: {match, status}` | a task whose `subject` OR `id` matches the regex `match` reached `status` — evidence-unavailable if tasks telemetry is absent; also fails **malformed** when a TaskCreate result was unparseable (corrupt task telemetry), mirroring the guard `all_tasks_completed`/`task_count_min` already had |
 | `no_scratchpad_leak: true` | every file presented via `present_files` that was in the scratchpad was successfully promoted to `mnt/outputs` (none left behind) — vacuously passes if nothing was presented (pair with a presence check to require a delivery); content-class: both the `present_files` tool_use and its own tool_result live in the ordinary events stream, so `RunResult.presentedFiles` re-derives identically on replay (meaningfully replay-checkable, same as `skill_triggered`); evidence-unavailable if `presentedFiles` telemetry is absent (an older run predating the feature). **Container-only on the merits**: hostloop serves `present_files` but never promotes (its handler passes a validated path through), so there is no scratch→outputs copy that could leak. On microvm/protocol a scratchpad-delivered file is neither promoted to `mnt/outputs` nor detected there (a skill that delivers via write-to-cwd→`present_files` will false-red `user_visible_artifact` on those tiers; use `container`, or write directly to `outputs/`). Hostloop is not one of them: the agent's cwd there already *is* the outputs dir, so a write-to-cwd delivery is under a user-visible root from the start and `user_visible_artifact` passes. **The tool name is lane-specific:** `present_files` is the desktop-local lane's tool (the one this harness emulates); remote Cowork delivers via the agent-native `SendUserFile` instead, so a skill should describe the delivery outcome rather than naming either tool — this key asserts the harness-side delivery record either way. **Only `true` is valid** |
 | `present_files_called: true` | at least one file was actually delivered via the `present_files` tool (`RunResult.presentedFiles` is non-empty) — the presence companion to `no_scratchpad_leak` (which passes vacuously when nothing was presented). Pair them to require a delivery **and** require it not to leak. Content-class (re-derives identically on replay). **`fidelity: container` or `hostloop`** — the harness serves `present_files` at both (hostloop via a handler mirroring production's own host-loop branch: validate the path, pass it through, no promotion). `protocol` and `microvm` report cannot-verify. See the `no_scratchpad_leak` row, which stays container-only for a different reason; and see the lane note there — the tool name differs on remote Cowork. **Only `true` is valid** |
-| `question_asked: <regex>` | the agent asked an AskUserQuestion whose text matches |
+| `question_asked: <regex>` | the agent asked an AskUserQuestion whose **question text** matches (`question`, falling back to `header`). Text only — for the options it offered, use `question_options` |
+| `question_options: {when_question?, equals?, contains?, order?}` | the option SET and ORDER a gate offered — what the user was actually SHOWN. `when_question` selects the sub-question by the same label `question_asked` matches (omit only if exactly one fired; ambiguity FAILS). Exactly one of `equals` (complete set) / `contains` (subset); `order: exact` is the default because a re-ordered list is the defect this exists for. Captured at ask time, so a gate that was shown then denied/stalled still counts; unreadable evidence fails evidence-unavailable |
 | `questions_count_max: <N>` | at most N **sub-questions** asked — a bundled `AskUserQuestion` with K sub-questions counts as K, not 1; `trace --view questions`'s footer total uses the same definition |
 | `gate_answers_delivered: true` | every answered gate's answer reached the model (observed `tool_result`; unobserved = fail); **zero gates fired passes vacuously** — pair with `gate_answer_count_min: >= 1` to also require a gate, or drop this key and declare `questions_count_max: 0` in a scenario that expects none |
 | `gate_answers_delivered: false` | asserts at least one answered gate's answer was **confirmed not delivered** (an observed delivery failure); an unobserved/null delivery does **not** satisfy this — for negative-path delivery tests. Requires a gate, so **mutually exclusive** with `questions_count_max: 0` (refused by `run`/`skill`/`record`) |
@@ -340,7 +358,7 @@ same set live from the schema.
 | `allow_permissive_auto_allow: true` | verdict modifier — suppresses the default-fail when the run recorded a cowork-parity permissive auto-allow; for tests that deliberately assert Cowork's permissive behavior |
 | `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
 | `allow_l0_plugin_divergence: true` | verdict modifier — opt into L0/protocol plugin divergence: suppresses the default-fail when a plugin behaves differently at `protocol` (L0) fidelity than under a sandboxed tier. Live tiers only |
-| `allow_stall: true` | verdict modifier — suppresses the `stalled` default-fail when a run ends on a question having done no productive tool work after its last gate (the agent asked for input and stopped — incl. re-asking in plain text after answering an `AskUserQuestion`); assert only when ending on a question is intended, else script the answer (`answer:` / `--answer` / a decider) |
+| `allow_stall: true` | verdict modifier — suppresses the `stalled` default-fail when a run ends on a question having done no productive tool work after its last gate (the agent asked for input and stopped — incl. re-asking in plain text after answering an `AskUserQuestion`); assert only when ending on a question is intended, else script the answer (`answer:` / `--answer` / a decider). **Scenario-only:** an open-ended `skill` run has no `assert:` block, so it cannot opt out — a helpful closing offer ("want me to run this through a structured pass?") fails `stalled` there with no suppressor. Read the final message before believing it, or move the check to a `run` scenario |
 | `allow_undelivered_deliverables: true` | verdict modifier — suppresses the `undelivered_deliverables` WARN. Working in the scratchpad is Cowork's designed pattern, so a skill that legitimately leaves intermediates, caches or downloaded inputs behind can say so instead of carrying permanent noise. The signal is warn-only and never fails a run on its own; reach for this when the scratch activity is intentional, not to silence a real delivery gap |
 | `allow_outputs_delete: true` | verdict modifier — accepts a detected outputs delete instead of failing the run, for a skill whose deletion is intended. Omitting `no_delete_in_outputs` does **not** permit deletes (a detected delete fails via the `outputs_delete` signal precisely because the key was not authored), so this is the way to accept one. **Mutually exclusive** with `no_delete_in_outputs`. Waives the harness's post-hoc detection; it does not model Cowork's `allow_cowork_file_delete` approval handshake |
 | `allow_delete_in: [<mount>…]` | verdict modifier — accepts detected deletes in the named mounts (the per-mount analogue of `allow_outputs_delete`, mirroring production's `fileDeleteApprovedMounts`). Waives the verdict only; detection still runs and the hits stay in `result.json`. Listing `outputs` conflicts with `no_delete_in_outputs` |
@@ -423,7 +441,7 @@ sourcing ≠ evaluation (replay warns when you edit one). `verify-run` is the on
 modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
 `allow_stall` are also kept on replay, evaluated as no-op passes.
 
-**Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `questions_count_max`,
+**Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `question_options`, `questions_count_max`,
 `gate_answers_delivered`, `gate_answer_count_min`, `hook_blocked`, `no_hook_blocked`, `vm_path_denied`,
 `path_denied`, `no_path_denied` (the latter three are also `fidelity: hostloop`-only — see the assertion
 table). With `controlOut` present they evaluate; on an old
@@ -435,13 +453,16 @@ the question keys: a custom hook's block/allow decision is an opaque async reply
 built-in Task hook's view and could vacuously pass `no_hook_blocked` even if a custom hook genuinely blocked.
 
 **Filesystem — replay-checkable WITH an artifact manifest:** `file_exists`, `user_visible_artifact`,
-`artifact_json`, `computer_links_resolve` run on replay when the cassette carries an `artifacts` snapshot
+`artifact_json`, `artifact_text`, `computer_links_resolve` (+ `computer_links_resolve_if_present`) run on
+replay when the cassette carries an `artifacts` snapshot
 (`record` captures `outputs/` + connected folders; `replay` materializes it). `artifact_json` needs the
 small-file JSON `body` inlined; a hash-only entry still satisfies `file_exists`. `computer_links_resolve`
 resolves a `/sessions/…/mnt/…`-shaped link directly against the manifest, and a host-shaped (hostloop) link
 by first normalizing it to a mount-relative path (recorded connected-folder prefixes + the outputs/uploads
 mounts) — replay has no live filesystem to check a host path against directly (that only happens on a live
-`run`/`verify-run`). Without a manifest (older cassettes) all five are skipped (five need the manifest; two
+`run`/`verify-run`). `artifact_text` is manifest-class for the same reason `artifact_json` is — a
+body-less, symlinked or over-cap entry fails evidence-unavailable rather than passing.
+Without a manifest (older cassettes) all six are skipped (six need the manifest; two
 more — `no_unexpected_files` and `input_unmodified` — need the pre-run path/hash capture, below); `no_unexpected_files` also
 needs `preRunPaths` (≥0.24 recordings) — without it the key is excluded with a loud warning (live/verify-run
 hard-fails evidence-unavailable instead). `input_unmodified` is the same shape but needs `preRunHashes`
@@ -451,7 +472,9 @@ warning. A green replay re-confirms
 staleness `fingerprint` shows ANY skill/baseline drift, or `replay --fail-on-skill-drift` only on
 skill-source drift; every replay result also reports it class-tagged in `staleness[]` for a JSON gate.
 
-**Egress + other filesystem — still skipped on replay (live-only):** `no_delete_in_outputs`,
+**Egress + other filesystem — still skipped on replay (live-only):** `file_absent` (proving a path is
+ABSENT needs an exhaustive, healthy walk; a manifest records no walk health, so "not captured" and "not
+there" are indistinguishable and the key would pass while proving nothing), `no_delete_in_outputs`,
 `self_heal_ran`, `transcript_no_host_path`, `egress_*` / `expect_denied`, `no_mcp_error`, `max_peak_rss_bytes`,
 `no_lost_write_back`. These run only on a live `run`/`record`.
 

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -2,7 +2,7 @@
 
 Each recipe composes facts that live scattered across SKILL.md and the other references into one
 decision path. Every one answers a question a real fleet owner had to work out the hard way.
-Tracks `cowork-harness 1.22.0` (baseline `desktop-1.28929.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
+Tracks `cowork-harness 1.24.0` (baseline `desktop-1.32352.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
 Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
 needed if your CLI meets SKILL.md's version floor.
 
@@ -17,11 +17,12 @@ a paid live re-record? Walk this tree — the answer is usually no:
    `cowork-harness replay <cassette> --assert-from <scenario.yaml>`. Token-free, no re-record.
    If the recording genuinely lacks the telemetry a key needs (very old cassettes), the key fails
    **loud** as `evidence-unavailable` — that is correct behavior, not a bug; only then re-record.
-2. **Gate keys** (`question_asked`, `questions_count_max`, `gate_answers_delivered`) on a cassette
+2. **Gate keys** (`question_asked`, `question_options`, `questions_count_max`, `gate_answers_delivered`) on a cassette
    **with `controlOut`** (any modern recording) → same token-free `--assert-from` path.
 3. **Gate keys** on a **pre-`controlOut`** cassette → one re-record unlocks gate asserts for that
    cassette permanently.
-4. **Filesystem / egress keys** (`file_exists` without an artifact manifest, `egress_allowed`,
+4. **Filesystem / egress keys** (`file_exists` without an artifact manifest, `file_absent` — live-only
+   whatever the cassette carries — `egress_allowed`,
    `egress_denied`, `no_delete_in_outputs`) → live lane **by design**; replay skips them with a
    loud `::warning::`. Keep them in the scenario, run them on the nightly live gate.
 
@@ -69,6 +70,7 @@ Top-level fields of a `*.cassette.json` (schema [`schema/cassette.v11.json`](htt
 | `folderPrefixMap` | Optional even on v9+: the record-time connected-folder host-path → mount-name map. Replay's `computer_links_resolve` uses THIS (never the current session file); absent → the link is treated as evidence-unavailable, never reconstructed from the current session |
 | `timeline`, `timelineHeader` | The recorded per-event timeline (harness-observation timestamps for tool_use/tool_result/subagent_dispatch/thinking/decision/result, in total order) plus its header (`startedAtWall`/`startedAtMono` anchors); informational only — never affects the replay verdict. Absent on a cassette recorded before this field existed |
 | `environment` | Recording provenance: `location` (`"local"` on every cassette this harness produces), the resolved `tier`, `agentBinaryFormat`, and `harnessVersion` — the CLI that RECORDED the cassette (≥1.11.0). A harness-code change can shift recorded behaviour at an UNCHANGED baseline, which no staleness class keys off, so this is the only provenance for that class. **Absent** on pre-1.11.0 cassettes, and never backfilled — the absence is itself the signal |
+| `environment.agentImage` | The `agentImage` sub-block records the rootfs image the recording ran against, stamped only for the tiers whose capabilities come from it (`container`, `hostloop`; `microvm` probes the Lima guest instead). `ref` is the resolved image ref, `configId` the local config id (built **and** pulled images, NOT comparable across machines), `registryDigest` the registry manifest digest (pulled only, and the only cross-machine-comparable identity). The image decides `missingCapabilityUse`, which `computeVerdict` fails on, so replaying against a different rootfs can move a verdict with nothing in the cassette having changed — `replay` compares `registryDigest` first and prints an advisory note, never a failure. Re-inspected at cassette-WRITE time, so a rebuild between run and record records the later image. **Absent** on cassettes recorded before this field existed, and never backfilled |
 
 ## Recipe 3 — Set up redaction BEFORE your first hostloop/protocol record
 
@@ -175,9 +177,16 @@ degrade the advice. It is real work to calibrate; these steps are the traps that
    from `RunResult.assertions[].semanticClaims`. Do **not** chase single-run all-pass — set `min_pass` to
    the reliably-hit core for a green verdict, and treat the per-claim rates as the real signal. (N=1
    routinely mislabels a stable 0/3 as "intermittent" and vice-versa.)
-5. **Check discrimination — does the skill actually help?** Run one rep with the skill NOT installed (or
-   inspect a not-invoked rep). If the answer still scores high, that claim is answerable from priors and
-   tests the model, not your skill — strengthen it (a skill-specific fact) or drop it.
+5. **Check discrimination — does the skill actually help?** Run one rep with the skill NOT installed and
+   compare. `--ablate-skill` is the flag for it: it empties every skill/plugin discovery source for
+   **that one invocation**, so the agent answers from its own priors, and stamps the result
+   `ablated: true`. It is the **control arm only** — run the same prompt again *without* the flag for
+   the treatment arm; `--ablate-skill --repeat 5` gives you 5 control runs and 0 treatment runs, not an
+   A/B. (Inspecting an organically not-invoked rep works too, but is not a substitute: that rep may
+   differ for other reasons.) If the answer still scores high without the skill, that claim is
+   answerable from priors and tests the model, not your skill — strengthen it (a skill-specific fact) or
+   drop it. Everything past "run both arms" — scrubbing giveaways, shuffling, judging blind, unblinding
+   after grading — is yours to build; the harness supplies the runs and the control.
 6. **Gate a change on the profile diff.** Capture the per-claim profile before your edit (the baseline),
    make the edit, re-capture, and compare per claim: a claim that DROPPED (e.g. 3/3 → 0/3) is a regression
    your edit caused; a claim already at 0/3 (a known gap) cannot regress. That turns "did my SKILL.md
@@ -228,7 +237,15 @@ Hardening a skill is a loop: run → read what it did → fix → run again. Two
    - `cowork-harness inspect <run-dir>` → what the run produced, plus the run's `label` and `skillHash`.
    - In-run alternative: dispatch a checker **sub-agent** (maker/checker) whose result folds into the
      verdict.
-2. **Don't cross-pair generations.** When you run the same skill across fixes, never pair a *pre-fix*
+2. **Commit the skill before a measurement batch, and pin the model.** Two cheap disciplines that are
+   unrecoverable if skipped. `skillHash` is content-exact, so an edit mid-batch silently splits the
+   dataset into two generations — `stats --group-by skill-hash` separates them afterwards, but a hash
+   whose source was never committed names a generation that is unrecoverable, which makes the
+   comparison uninterpretable rather than merely noisy. And with no `model:` in the session (or
+   `--model` on the `skill` lane) each run uses whatever the staged agent binary defaults to, so a
+   before/after can silently straddle two models; read `result.json`'s `models` back to confirm —
+   ignoring any `<…>`-wrapped entry (`<synthetic>` marks a turn the agent fabricated locally, not a model).
+3. **Don't cross-pair generations.** When you run the same skill across fixes, never pair a *pre-fix*
    `result.json` with a *post-fix* critique. The authoritative version key is `fingerprint.skillHash` —
    content-exact, on every live `run`/`skill` run that mounts a skill or plugin — **but only on ≥ 1.5.0; earlier CLIs emit
    no `skillHash` on the `skill` lane at all, so verify the field is present before pairing on it** (a run that mounts nothing

package/.claude/skills/cowork-harness/scripts/assertion-keys.json

@@ -10,6 +10,7 @@
     "allow_stall",
     "allow_undelivered_deliverables",
     "artifact_json",
+    "artifact_text",
     "compaction_occurred",
     "computer_links_resolve",
     "computer_links_resolve_if_present",
@@ -17,6 +18,7 @@
     "dispatch_count_max",
     "egress_allowed",
     "egress_denied",
+    "file_absent",
     "file_exists",
     "gate_answer_count_min",
     "gate_answers_delivered",
@@ -41,6 +43,7 @@
     "path_denied",
     "present_files_called",
     "question_asked",
+    "question_options",
     "questions_count_max",
     "replay_protocol_fidelity",
     "result",