cowork-harness 1.2.0 → 1.4.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -1,10 +1,10 @@
 ---
 name: cowork-harness
-description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
+description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 1.2.0
-  tracks-harness: cowork-harness 1.2.0 (baseline desktop-1.21459.0)
+  version: 1.4.0
+  tracks-harness: cowork-harness 1.4.0 (baseline desktop-1.22209.3)
 ---
 
 # cowork-harness
@@ -22,8 +22,8 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.2.0` (baseline
-> `desktop-1.21459.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 1.4.0` (baseline
+> `desktop-1.22209.3`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
 > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## Preflight — make sure the harness can actually run
@@ -39,9 +39,9 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
 - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.2.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.2.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=1.2.0"`. **Pin `@>=1.2.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 1.4.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=1.4.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=1.4.0"`. **Pin `@>=1.4.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
 
-  What the ≥ 1.2.0 floor gates, by release:
+  What the ≥ 1.4.0 floor gates, by release:
 
   - **core set (pre-0.21.0 vintage, or mixed):** `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes` incl. the `--allow-domain`/`--allow-email`/`--allow-patterns-file` allows (`--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, and `/help` in the REPL.
   - **0.21.0:** `verify-cassettes --allow-path` (`path` — local absolute filesystem paths — is the scanner's 4th class), and `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field.
@@ -54,6 +54,8 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
   - **1.0.0:** first stable release — the SPEC §12 compatibility contract takes effect (covered CLI/schema/env/Action surfaces are now stable; breaking changes need a major bump). No new author-facing command; the floor simply tracks the 1.0 release.
   - **1.1.0:** `analyze-skill` now also flags **interactive-artifact write-backs lost under Cowork** — a relative `fetch`/XHR/`sendBeacon`/`<form method=post>` in an emitted `.html` (or its `.py`/`.js` generator) that silently fails when the artifact is served from Cowork's own origin. `artifact-write-back-lost` gates under `--strict`; `artifact-write-back-suspect` is advisory; an unanalyzable candidate is a could-not-verify exit 3. An optional **`analyze-skill --runtime`** drives the artifact in a headless DOM (needs `jsdom`) to *observe* the lost write-back — enrichment only, never changes the exit code. Plus a `lint` check for a container-only assertion key (`no_scratchpad_leak`/`present_files_called`) used off the `container` tier, and the `doctor --output-format json` envelope frozen as a covered SPEC §12 surface (`schema/doctor.json`).
   - **1.2.0:** three new assertion keys — `no_lost_write_back: true` (the write-back detector above wired as a per-scenario gate over the run's authored files, live-only) and the regex siblings `tool_result_matches`/`tool_result_not_matches` (case-insensitive per-result, for an error-signature *family* a literal substring can't express). **`microvm` outputs are now observable** — its session tree is snapshotted from the VM into the run dir, so `file_exists`/`artifact_json`/`user_visible_artifact`/`no_unexpected_files`/`input_unmodified`/`no_lost_write_back`/`semantic_matches` all work there, no longer `container`/`hostloop`-only. `status <dir>` also resolves the newest session under a `--run-dir` root; the completion footer prints a `→ result: …/result.json` pointer; the `on_unanswered=fail` error also points to `on_unanswered: llm`. `analyze-skill` hardening: a phantom `<script>` prose block no longer sinks a real verdict, a delete/remove flow claiming success classifies as lost (error) not just suspect, and write-back detection widened (optional-call `?.` spellings, member-spelled/aliased `fetch`/`sendBeacon`, axios instance/config forms).
+  - **1.3.0:** run-identity for the iterate-across-fixes loop — `skill`/`run` take `--label <tag>` (a generation tag surfaced in `result.json` `runLabel`, the run-index row, `inspect`, and `status.json`) alongside an auto-recorded `skillCommit`, layered on the **authoritative** content-exact `fingerprint.skillHash` a harvest step should pair critiques by (`inspect`/index surface a short `skillHash` prefix). `trace --full-results` captures the full input+result of **every** tool call — successful ones too, not just errors — so an external grader can ground a self-critique finding against the call it cites. `verify-run` now **warns** on skillHash drift for answer-less scenarios (was silent unless the scenario declared scripted `answers`). Plus `skill --allow-missing-capability` (the open-ended-run opt-out for a capability FALSE-NEGATIVE on the lean `core` image), a new **warn-severity `ended_with_question`** verdict signal (the agent's final answer contains a question and the run wrote no `outputs/` deliverable — the lenient sibling of the strict `stalled`), and LLM-decider `OTHER:` free-text answers now marked `[via Other free-text]` in gate provenance.
+  - **1.4.0:** platform baseline synced to Desktop **1.22209.3** (agent 2.1.215; no prompt/spawn/egress drift), and **`coworkWebFetchDedup` is now enacted** on the hostloop `web_fetch` path — a repeat fetch of the same normalized URL within the TTL (15 min, cap 100) makes **no network request** and returns a marker to re-use the earlier result, with no egress event. Baseline-gated (Desktop ≥ 1.22209.3), keyed under the request + terminal `destination_url`, never caching errors/empty; a hit is assertable via `tool_result_contains: "Already fetched"`.
 - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
 - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
 - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred), or `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
@@ -311,8 +313,10 @@ your `answers:` against that kept run, then record once. **But the kept run is a
 skill's gate phrasing afterward, re-`--keep` — verify-run's answer-coverage *refuses* (exit 2, "predates the
 current skill") rather than vouch against stale labels, but the trace/inspect path can't warn you, so re-keep
 deliberately. (Same fail-closed family: corrupt gate evidence — unparseable `events.jsonl` lines, or fewer
-gates than `trace.json` recorded questions — and a structurally invalid `result.json` also refuse rather
-than certify.) (A token-free probe of "which gates fire" isn't possible — gates are model-decided per run.)
+gates than `trace.json` recorded questions — a structurally invalid `result.json`, a `command:"replay"`
+result (a replay is a re-check of a recorded cassette, not run evidence — verify the original live run dir),
+and a `mode:"chat"` result (chat carries no assertions or verdict by contract) also refuse rather than
+certify.) (A token-free probe of "which gates fire" isn't possible — gates are model-decided per run.)
 
 Run artifacts are written to `~/.cowork-harness/runs/…` by default — **outside any working tree**, so a run
 launched from a repo root never drops sensitive skill inputs/outputs into it. Pass `--run-dir <path>` (or set
@@ -354,6 +358,15 @@ cassette — has its own recipe:
    unless the scenario asserts `allow_missing_capability: true`, which downgrades it to a notice and
    proceeds. Rebuild with `--build-arg COWORK_FULL_PARITY=1` and point `COWORK_AGENT_IMAGE` at it for those
    skills.
+6. **Iterate across fixes — verify before you trust, and don't cross-pair generations.** A green run is
+   not a correct run, and a skill's self-reported finding is not real until its cited evidence is found in
+   the run's own output. Ground each finding against `result.json` (`finalMessage` = the skill's own
+   answer/critique; `toolResults` = tool outputs) and the tool-call stream via
+   `cowork-harness trace <run-dir> --output-format json` — add `--full-results` so a successful call's full
+   input + result are captured, not just errored ones. When iterating, tag generations with `--label` and
+   pair a critique only with a `result.json` whose `fingerprint.skillHash` **matches** the skill that
+   produced it (`inspect`/the run-index row surface a short `skillHash` prefix; `verify-run` warns when a
+   kept run predates the current skill). See `docs/debugging.md` (repo-only) for the full loop.
 
 #### Interpreting verdict signals
 
@@ -364,6 +377,42 @@ The run verdict may include `WARN`-severity signals in addition to pass/fail. On
   the run can still green. If you see it, fix the asset path — a green with a missing asset is
   not a valid pass.
 
+**False negatives — signals that are tier/image artifacts, not skill defects.** Some fail-severity
+signals read like a skill gap but are really a property of the reduced test image or the fidelity tier.
+Recognize these before "fixing" a non-bug:
+
+- **`missing_capability`** — the lean `core` agent image is a deliberate partial mirror of real Cowork's
+  rootfs, so a skill that used `soffice`/LibreOffice (`office_convert`), `tesseract` (`ocr`),
+  `markitdown`/`magika` (`ml_extract`), `cv2` (`cv`), `camelot`/`tabula` (`pdf_tables`), or `wand`
+  (`magick`) can trip this even though real Cowork **ships** those. The message says so ("likely a FALSE
+  NEGATIVE (real Cowork ships them)"). Fix: rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`, point
+  `COWORK_AGENT_IMAGE` at it), or — if the skill's fallback is genuinely equivalent — assert
+  `allow_missing_capability: true`. (Two sources: a skill *observed using* an omitted family, live lane;
+  or a declared `requires_capabilities` the tier can't provide, both lanes — an unknown family name
+  hard-fails rather than silently passing.) **On an open-ended `skill` run** (no `assert:` block to carry
+  the modifier), pass **`--allow-missing-capability`** — the CLI equivalent of the assertion.
+- **`ended_with_question`** (`WARN`, live lane) — a heuristic: the agent's final answer contains a
+  question and the run wrote **no deliverable to `outputs/`** — it may have ended on a request for input
+  instead of finishing. Warn-only; the fix is scripting/steering the answer (`answer:` / `--answer` / a
+  decider, or `--decider-llm --intent`), not editing the skill's prose. The strict, fail-severity sibling
+  `stalled` already catches a *trailing*-`?` final turn that did no tool work after the last gate; this
+  covers the residual (a mid-message `?`, or tool work after the last gate that still ended asking). Read
+  the final message before acting — a legitimate question-posing answer that wrote a file never fires.
+  Assert `allow_stall: true` if ending on a question is the intended terminal state.
+- **`host_path_leak`** — skipped at **`hostloop` and `protocol`** fidelity (the agent runs on real host
+  paths there, so a host path in model-visible text is expected, not a leak); it is *armed* at
+  `container`/`microvm`, but only *fires* on an actual scanned leak with no authored
+  `transcript_no_host_path`. At `fidelity: cowork` the skip follows the **resolved** tier, so a `cowork`
+  run that lands on `container` is armed. Author `transcript_no_host_path` to enforce cleanliness where
+  it's valid.
+- **`scan_unavailable`** (`WARN`) — emitted only on the live lane: `events.jsonl` was missing/corrupt, so
+  `RunResult.scan` is undefined and the host-path + outputs-delete guards **did not run this run**. Not a
+  pass or a defect — assert `no_delete_in_outputs` / `transcript_no_host_path` to hard-fail on it instead.
+
+The full 14-code signal table (severity + per-signal opt-out) is in
+[`references/scenario-schema.md`](./references/scenario-schema.md); `docs/scenario.md` (repo-only) carries
+the fuller narrative.
+
 ### Checking whether a background run is alive
 
 Never use `ps aux` to check on a `cowork-harness` run you launched in the background — it only sees
@@ -472,7 +521,11 @@ decide which assertions from *Assertions: two orthogonal axes* are worth adding)
 - **Debugging a wrong Cowork UI panel.** Each panel is reconstructed in `result.json`: **Progress** =
   `tasks[]`, **Working folder** = `workspaceFiles[]` (classified output/mount/input, with a
   `trace --view files` diff), **Context / Connectors** = `context` (tools / mcpServers / availableSkills),
-  **Scratch-pad → outputs** = `presentedFiles[]`. If a panel looks wrong in a run, read its field.
+  **Scratch-pad → outputs** = `presentedFiles[]`. If a panel looks wrong in a run, read its field. An
+  **absent** `workspaceFiles`/`artifacts` (a replay result, or a run whose workspace root was missing at
+  collection) is evidence **UNAVAILABLE**, not an empty run — `trace --view files` reports a loud
+  UNAVAILABLE marker (`workspaceFilesRecorded: false` in JSON, and no phantom "removed" diff rows) and
+  `inspect` prints `artifacts: UNAVAILABLE` (`artifactsRecorded: false`) instead of `artifacts (0):`.
 
 ### Debugging with `chat`
 
@@ -615,9 +668,11 @@ repeats the assertion/replay-relevant ones alongside the schema (a scoped subset
     - `on_unanswered` governs **unanswered** `AskUserQuestion` gates; the `stalled` signal covers
       stalling *after* one is answered — two different failure modes.
     - **Free-text aside:** a "type-it-in-notes" option has **no scripted deterministic answer** today
-      (the `OTHER:` directive works only on the LLM-decider path, not scripted `choose:`; on an
-      options-bearing gate a bare out-of-set LLM answer fails loud (exit 2) — see the LLM-decider
-      free-text note in `references/fidelity-and-answers.md`).
+      (the `OTHER:` directive works only on the LLM-decider path, not scripted `choose:`, and only on
+      **single-select** gates — a **multi-select** gate is index-only, so `OTHER:` fails loud there; on an
+      options-bearing single-select gate a bare out-of-set LLM answer also fails loud (exit 2) — see the
+      LLM-decider free-text note in `references/fidelity-and-answers.md`). An LLM decision answered via
+      `OTHER:` is marked `[via Other free-text]` in its `gateProvenance` rationale.
 14. **A positional `choose` (`first` / index) is order-dependent.** `choose: "2"` survives label drift
     but NOT option *re-ordering* — if the gate presents its options in a different order run-to-run, the
     index lands on a different option (a silent re-record flake). Prefer an exact label when order is

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,6 +1,6 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 1.2.0` (baseline `desktop-1.20186.1`).
+Self-contained reference. Tracks `cowork-harness 1.4.0` (baseline `desktop-1.20186.1`).
 
 **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
 job-summary reporter (verdict table, staleness findings, cost/turns when available):
@@ -13,7 +13,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
 ```
 
 The Action's `version` input defaults to `latest` — intentional so a copy-pasted recipe tracks the current
-release; pin an exact version (e.g. `version: "1.2.0"`) for reproducible CI.
+release; pin an exact version (e.g. `version: "1.4.0"`) for reproducible CI.
 
 Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
 cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
@@ -32,7 +32,7 @@ jobs:
       - uses: actions/checkout@v4
       - name: Stage the agent binary (official channel, sha256-verified — see docs/maintenance.md)
         run: |
-          V=2.1.209   # match your scenario's pinned baseline's agentVersion
+          V=2.1.215   # match your scenario's pinned baseline's agentVersion
           curl -fSL "https://downloads.claude.ai/claude-code-releases/$V/linux-arm64/claude" -o "$RUNNER_TEMP/claude-$V"
           chmod +x "$RUNNER_TEMP/claude-$V"
           # verify against the committed baseline's sha256 (baselines/desktop-*.json → agentBinary.sha256)
@@ -57,7 +57,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
 GitHub-hosted runners, no token/Docker/agent:
 
 ```yaml
-- run: npm i -g "cowork-harness@>=1.2.0"
+- run: npm i -g "cowork-harness@>=1.4.0"
 - run: cowork-harness lint scenarios/*.yaml          # no silent false-greens
 - run: cowork-harness verify-cassettes cassettes/    # privacy + staleness
 - run: cowork-harness replay cassettes/              # token-free content/structure
@@ -197,7 +197,7 @@ jobs:
         with: { node-version: '20' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g "cowork-harness@>=1.2.0"
+      - run: npm i -g "cowork-harness@>=1.4.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -226,7 +226,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g "cowork-harness@>=1.2.0"
+        run: npm i -g "cowork-harness@>=1.4.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 1.2.0` (baseline `desktop-1.20186.1`).
+Self-contained reference. Tracks `cowork-harness 1.4.0` (baseline `desktop-1.20186.1`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 
@@ -164,10 +164,13 @@ Caveat: `decide` only builds a **single-select** sample (set choices with `--opt
 multiSelect flag), so its printed request shows `options[].label` but never `multiSelect:true` — to
 exercise the array reply path, run a real multiSelect gate or unit-test the helper directly.
 
-LLM-decider free-text goes via `OTHER: <value>` on an **options-bearing** gate; a bare out-of-set
-answer (no matching label, no `OTHER:`) fails loud (`UnansweredError` → exit 2) — it never stalls or
-guesses an option. Open-ended (no-option) gates need no `OTHER:` prefix: free text is delivered
-verbatim. (Scripted scenarios use the separate `answer:` escape hatch.)
+LLM-decider free-text goes via `OTHER: <value>` on an **options-bearing single-select** gate; a bare
+out-of-set answer (no matching label, no `OTHER:`) fails loud (`UnansweredError` → exit 2) — it never
+stalls or guesses an option. A **multi-select** gate is **index-only**: it accepts comma-separated option
+numbers, and `OTHER:` fails loud there (no free-text escape on that path). Open-ended (no-option) gates
+need no `OTHER:` prefix: free text is delivered verbatim. A decision answered via any free-text path is
+marked `[via Other free-text]` in its `gateProvenance` rationale, so a `result.json` consumer can tell it
+from an offered-option pick. (Scripted scenarios use the separate `answer:` escape hatch.)
 
 A gate that fails loud (`on_unanswered: fail`, the default) still **salvages a PARTIAL run**: the harness
 writes a `result.json` (marked `partial: true`) with the artifacts the agent produced before the whiff, so

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,6 +1,6 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.2.0`
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 1.4.0`
 (baseline `desktop-1.20186.1`). If your checkout is newer, prefer the live `docs/scenario.md`,
 `docs/session.md`, and `SPEC.md`.
 
@@ -252,7 +252,7 @@ same set live from the schema.
 | `user_visible_artifact: <path>` | exists **and** under a user-visible root (`outputs/` + each connected folder's mount name) — the right primitive for a workspace deliverable when a folder is connected |
 | `no_delete_in_outputs: true` | no delete op touched `mnt/outputs` — **only `true` is valid**; `false` is rejected (omit to allow deletes) |
 | `no_unexpected_files: [<glob>, …]` | every **newly created** file under a user-visible root matches ≥1 glob (workRoot-relative paths; `**` = whole path segment for any depth — use `outputs/handoff/**` for per-run subdirs); `[]` = no new files; **new-files-only** — overwriting a pre-existing file in place is invisible (use content-level producer stamping); live/verify-run without a pre-run manifest ⇒ evidence-unavailable hard-fail (live runs capture the baseline only when this key is asserted; recordings always capture); an **incomplete post-run filesystem walk** (an unreadable subtree — a permission/I-O error) also fails evidence-unavailable rather than reporting "no strays" over a partial tree — distinct from the missing-manifest case (a `--resume` run), which fails for a different reason; captured on every live sandbox tier including microvm (its outputs are snapshotted from the VM into the run dir); replay needs `cassette.preRunPaths` (≥0.24 recordings) — cassettes without it **exclude** the key with a loud warning |
-| `input_unmodified: <glob>` or `[<glob>, …]` | a single glob or a list; every **pre-existing** file (incl. uploaded files under `uploads/**`) whose workRoot-relative path matches ≥1 glob keeps an unchanged content hash after the run — the in-place-mutation companion to `no_unexpected_files`'s new-files check (`[]` is rejected by the schema — list at least one glob); a matched file that was deleted counts as a content change (fails); live/verify-run without a pre-run hash manifest ⇒ evidence-unavailable hard-fail (a `--resume` run); captured on every live sandbox tier including microvm; replay needs `cassette.preRunHashes` — cassettes without it **exclude** the key with a loud warning; on replay it compares against the manifest's recorded `sha256`, never a re-hash of the materialized tree |
+| `input_unmodified: <glob>` or `[<glob>, …]` | a single glob or a list; every **pre-existing** file (incl. uploaded files under `uploads/**`) whose workRoot-relative path matches ≥1 glob keeps an unchanged content hash after the run — the in-place-mutation companion to `no_unexpected_files`'s new-files check (`[]` is rejected by the schema — list at least one glob); a glob that matches **no** pre-run path fails loud (a typo or renamed mount would otherwise verify zero files and pass vacuously); a matched file that was deleted counts as a content change (fails); live/verify-run without a pre-run hash manifest ⇒ evidence-unavailable hard-fail (a `--resume` run); captured on every live sandbox tier including microvm; replay needs `cassette.preRunHashes` — cassettes without it **exclude** the key with a loud warning; on replay it compares against the manifest's recorded `sha256`, never a re-hash of the materialized tree |
 | `self_heal_ran: <bool>` | a plugin-root self-heal script was (not) invoked |
 | `no_lost_write_back: true` | fails if the run authored an interactive HTML artifact (or a `.py`/`.js` generator of one) whose **relative** Submit/POST write-back is lost under Cowork (served from Cowork's own origin → resolves non-ok, a "Saved!" is silently false). Runs the shipped **static Tier A** analyzer over the files the run authored (diffed vs the pre-run manifest). A lost write-back on an **added** agent-authored source (`outputs/`, scratchpad) **fails**; a **pre-existing** file the skill only modified on a read-write mount is **advisory**; `-suspect` findings surface but pass. **Only `true` is valid**. **Live/verify-run only** — skipped-loud on replay; runs on every live sandbox tier including **microvm** (its outputs are snapshotted from the VM into the run dir); could-not-verify (fail-closed) on a `--resume` scratchpad or an unanalyzable candidate |
 | `tool_called: <glob>` | a tool the agent ran matched this **glob** — `*` = any run, `?` = one char, exact when literal, anchored + case-sensitive. Exact name (`Write`) matches only that tool; `mcp__workspace__*` matches any workspace tool. GLOB, not regex (`.` is literal) — an empty glob, or one containing a regex/brace-expansion metacharacter (`.*`, `.+`, `\|`, `()`, `[]`, `+`, `^`, `$`, `{}`, `\d`/`\w`/`\s`/`\b`), is **rejected at load** (a hard schema error, not a runtime warning) — it would match no real tool name and pass a `_not_`/`_absent` assert vacuously. Applies whether the glob comes from an authored scenario or a recorded cassette's frozen assert. The bundled `scenario.py lint` does NOT perform this check — only the harness enforces it, at actual load (`run`/`skill`/`record`) |
@@ -334,7 +334,7 @@ dotted path.
 
 **VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; most
 are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
-`"success"`) and only three are **warn**-severity (informational, never flip pass/fail). Current signal
+`"success"`) and only four are **warn**-severity (informational, never flip pass/fail). Current signal
 codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 
 | Code | Severity | Meaning |
@@ -347,16 +347,17 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
 | `outputs_delete` | fail | An unauthorized delete touched `mnt/outputs` (opt out: author `no_delete_in_outputs`) |
 | `host_path_leak` | fail | A host path leaked into model-visible text (opt out: author `transcript_no_host_path`) |
 | `l0_plugin_divergence` | fail | L0/protocol plugin loading diverged from Cowork (opt out: `allow_l0_plugin_divergence`) |
-| `missing_capability` | fail | A `requires_capabilities` need was unmet, or the skill used a capability the image omits (opt out: `allow_missing_capability`) |
+| `missing_capability` | fail | A `requires_capabilities` need was unmet, or the skill used a capability the image omits (opt out: `allow_missing_capability`, or `skill --allow-missing-capability` on an open-ended run) |
 | `infra_error` | fail | A VM/egress sidecar crashed mid-run — not author-suppressible |
-| `stalled` | fail | The run ended on an unanswered question (opt out: `allow_stall`) |
+| `stalled` | fail | The run ended on an unanswered question, or on a trailing-`?` final turn with no tool work after the last gate (opt out: `allow_stall`) |
 | `non_deterministic` | warn | The run was LLM/external/human-decided — not reproducible |
 | `prompt_asset_missing` | warn | The run proceeded with a missing prompt asset (`COWORK_HARNESS_ALLOW_MISSING_PROMPT=1`); fidelity is degraded |
 | `scan_unavailable` | warn | Post-run scan evidence unavailable (`RunResult.scan` undefined) — the host-path and outputs-delete guards did not run this run |
+| `ended_with_question` | warn | Live-lane heuristic: the final answer contains a question and the run wrote no deliverable to `outputs/` — the lenient sibling of `stalled` (covers a mid-message `?`, or tool work after the last gate that still ended asking). Opt out: `allow_stall` |
 
 A **fail**-severity signal does not change `result.result` (still `"success"`), but it DOES fail the
 overall run verdict and exit code — `assert result: success` alone won't catch it; check
-`result.verdict.signals[].severity` or the run's exit code. Only the three **warn** codes are truly benign.
+`result.verdict.signals[].severity` or the run's exit code. Only the four **warn** codes are truly benign.
 
 ## Replay class
 

package/.claude/skills/cowork-harness/references/task-recipes.md

@@ -163,3 +163,30 @@ degrade the advice. It is real work to calibrate; these steps are the traps that
 **Lane note:** `semantic_matches` is **live-only** (the judge is a live model call), so these scenarios
 run on the `run` lane, never token-free `replay` — the linter's "all assertions live-only" warning is
 expected and correct here.
+
+## Recipe 6 — Iterate a skill across fixes (ground findings, don't cross-pair generations)
+
+Hardening a skill is a loop: run → read what it did → fix → run again. Two disciplines keep it honest.
+
+1. **Verify before you trust.** A green run is not a correct run, and a skill's self-reported finding (a
+   self-critique appendix, "I extracted X") is not real until its cited evidence is found in the run's own
+   output. The harness emits the substrate; the grader is yours (it lives outside the harness):
+   - `result.json` → `finalMessage` (the skill's own answer/critique) + `toolResults[]` (tool outputs).
+   - `cowork-harness trace <run-dir> --output-format json` → the tool-call stream. Add `--full-results` so
+     a **successful** call's full input + result are captured (the default view slices them to ~100/120
+     chars) — this is what lets your grader confirm "the skill claims it read X and derived Y" against the
+     actual call.
+   - `cowork-harness inspect <run-dir>` → what the run produced, plus the run's `label` and `skillHash`.
+   - In-run alternative: dispatch a checker **sub-agent** (maker/checker) whose result folds into the
+     verdict.
+2. **Don't cross-pair generations.** When you run the same skill across fixes, never pair a *pre-fix*
+   `result.json` with a *post-fix* critique. The authoritative version key is `fingerprint.skillHash` —
+   content-exact, on every live run, changes on any tracked edit. **Group/pair on it** (`inspect` and the
+   run-index row surface a short prefix). Add `--label <tag>` for a human-readable generation name
+   (skillHash is the correctness key; the label is ergonomics). `cowork-harness verify-run <run-dir>
+   <scenario.yaml>` is the native staleness guard: it **warns** when a kept run predates the current
+   skill, and with scripted `answers` **hard-fails** rather than vouch for a stale gate snapshot.
+
+**Lane note:** the exploratory driver is `skill <dir> --decider-llm --intent "<what this run tests>"`,
+which is flagged non-deterministic (a green here is exploration, not a scripted pass) — pin the
+load-bearing gates with `--answer` once you know which fire.

package/CHANGELOG.md

@@ -6,6 +6,137 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [1.4.0] — 2026-07-19
+
+### Added
+
+- **`coworkWebFetchDedup` enacted (hostloop `web_fetch`).** Real Cowork keeps a per-session negative-work
+  cache: a repeat `web_fetch` of the same normalized URL within a TTL (default 15 min; cap 100; FIFO
+  eviction; a hit does not refresh recency) makes **no network request** and returns a marker telling the
+  model to re-use the earlier result. The harness now reproduces this on the host-API (`coworkWebFetchViaApi`)
+  path — **baseline-gated** (only when the resolved baseline's `coworkWebFetchDedup` gate is on, i.e. Desktop
+  ≥ 1.22209.3), keyed under both the request URL and the terminal `destination_url`, never caching errors /
+  empty / non-2xx responses, and emitting **no egress event** on a hit (matching production's zero-network
+  dedup). A hit is observable via the marker text (`tool_result_contains: "Already fetched"`).
+
+### Changed
+
+- **Platform baseline synced to Desktop 1.22209.3** (agent `2.1.215`). No prompt / spawn-env / egress-allowlist
+  drift vs 1.21459.0; the sync captured the new `coworkWebFetchDedup` runtime config (enacted above) plus a
+  few new (off) GrowthBook gates. The skill/README/reference version floors and agent-binary pins track the
+  new baseline.
+
+## [1.3.0] — 2026-07-19
+
+### Added
+
+- **`skill --allow-missing-capability`** — the open-ended-run equivalent of a scenario asserting
+  `allow_missing_capability: true`. An open-ended `skill` run has no `assert:` block to carry the opt-out,
+  so a self-flagged capability FALSE-NEGATIVE on the lean `core` image would hard-fail with no escape
+  hatch; the flag merges the modifier onto the synthesized success assertion, suppressing both the
+  post-run `missing_capability` fail and the pre-run capability abort.
+- **`ended_with_question` verdict signal (WARN)** — a heuristic run-level classifier: the agent's final
+  answer contains a question and the run wrote no deliverable to `outputs/` — a likely conversational
+  dead-end that still exited `result:"success"`. The lenient warn-severity sibling of the strict, fail
+  `stalled` (which already catches a trailing-`?` final turn with no post-gate tool work); this covers the
+  residual — a mid-message `?`, or tool work after the last gate that still ended asking. Never flips a
+  verdict; `verify-run` over historical results may surface it on matching old runs (benign).
+- **`--decider-llm` "Other" answers are marked in gate provenance** — a decision answered via the
+  `OTHER:` free-text path (or a no-option free-text gate) now carries `[via Other free-text]` in its
+  `rationale`, so a `result.json` consumer can distinguish it from an offered-option pick. (Multi-select
+  gates remain index-only — `OTHER:` is rejected there, now pinned by a test.)
+
+- **Run-identity metadata for the iterate-across-fixes loop.** `skill`/`run` accept `--label <tag>`, a
+  human-readable generation tag surfaced in `result.json` (`runLabel`), the run-index row, `inspect`, and
+  `status.json`. Each live run also records `skillCommit` — best-effort git `HEAD` of the session's skill
+  source dirs (commit provenance; `null` when the dirs span >1 repo or aren't a git work tree). These are
+  ergonomics on top of the **authoritative** content-exact version key `fingerprint.skillHash` (already
+  recorded on every run): a harvest step should group/pair a critique against a matching `skillHash`.
+  `inspect` and the run-index row now surface a short `skillHash` prefix so a pairing check needs no
+  `result.json` open. (Chat runs carry no `skillHash` and take no `--label`.)
+- **`trace --full-results`** captures the FULL input + result of every tool call — successful ones too,
+  not just errors (`resultTextFull`/`detailFull`, 4 KB cap) — so an external grader can ground a
+  self-critique finding against the call it cites. The default view keeps its 100/120-char slices, so
+  existing JSON consumers are unaffected.
+- **`verify-run` now warns on skill drift for answer-less scenarios.** Previously the skillHash-drift
+  check ran only when a scenario declared scripted `answers` (a hard fail). An answer-less `verify-run`
+  now emits a `::warning::` ("the kept run predates the current skill … findings describe an older skill
+  version") instead of staying silent — a WARN, not a fail, since re-asserting a new `assert:` block
+  against a frozen run dir is legitimate.
+
+### Fixed
+
+- **Filesystem-evidence assertions no longer pass on incomplete evidence.** `input_unmodified` now fails
+  loud when its glob matches **no** pre-run path (a typo or renamed mount was a silent vacuous pass), and
+  fails evidence-unavailable when a manifest path escapes the workspace root or a matched file exceeds the
+  hash cap. The pre-run baseline records its provenance, so `no_unexpected_files` / `input_unmodified` fail
+  evidence-unavailable on an unreadable connected-folder baseline instead of diffing a partial tree
+  (`RunResult` gains `preRunOrigin`). `no_lost_write_back` no longer silently misses a modified-but-unreadable
+  or over-cap file, an authored file under an unreadable subtree, or a scratchpad deliverable behind a
+  symlink/hardlink — each now surfaces as could-not-verify.
+- **Run-dir consumers no longer read absent evidence as empty, or a replay re-check as run evidence.**
+  `verify-run` now refuses (exit `2`, "can't verify ⇒ not green") a run dir whose `result.json` was produced
+  by `replay` (`command:"replay"` — a re-check of a recorded cassette, not run evidence) or by `chat`
+  (`mode:"chat"` — no assertions or verdict by contract); the refusal keys on `command`/`mode`, never on
+  `workspaceFiles`, so a live run merely lacking optional evidence fields still verifies. `stats --reindex`
+  skips a stray `command:"replay"` `result.json` instead of stripping the label and relabeling it `"run"`,
+  and its report line separates `skipped — replay re-check, not evidence` from `skipped — missing/corrupt
+  result.json`. `trace --view files` reports workspace-file evidence **UNAVAILABLE**
+  (`workspaceFilesRecorded: false` in JSON) when `workspaceFiles` is absent — distinct from a run that
+  genuinely wrote nothing — and no longer emits phantom "removed" diff rows against a persisted
+  `preRunHashes` in that case. `inspect` likewise prints `artifacts: UNAVAILABLE`
+  (`artifactsRecorded: false`) instead of `artifacts (0):` when `result.artifacts` is absent.
+- **Static artifact analysis (`analyze-skill` / `no_lost_write_back` Tier A)** closes several false-green and
+  false-positive holes: recognizes `axios`/jQuery and library write-backs, computed member calls
+  (`xhr["open"]`), unquoted and submitter-overridden `<form>` actions, and ES-module sources; classifies URLs
+  with the WHATWG parser so `localhost.evil.com`, protocol-relative `//host`, and `mailto:`/`data:` are no
+  longer misread as local; scope-aware constant folding and member-mutation tracking stop live code being
+  proven dead; an unresolved URL or method (including spread request options) yields could-not-verify instead
+  of a silent clean; and every write-back in a file is reported, not just the first. The advertised
+  `.ts/.tsx/.jsx` source extensions are dropped (no compatible parser) — treated as out-of-scope rather than
+  parse-noise.
+- **Runtime artifact confirmation (Tier B)** now confirms edit-fired autosaves and load-time write-backs (not
+  only explicit commits), handles `fetch(Request)`, models browser-faithful XHR response/listener semantics,
+  skips disabled/hidden controls and honors a submitter's `formaction`, analyzes observed writes before
+  downgrading on an external script, matches loopback hosts exactly, and no longer swallows unrelated harness
+  exceptions.
+- **`analyze-skill` orchestration** fails could-not-verify (exit `3`) when a `references`/`agents`/`commands`/
+  `skills` subtree is unreadable (was a silent clean); an ignore marker inside a fenced or block-quoted
+  example no longer suppresses real findings; a mixed invocation fails on a positional that resolves to no
+  scannable source; JSON coverage lists clean artifact sources; and runtime mode honors the 3 MB read cap.
+- **Protocol / decision handling fails closed on drift.** Unknown control-request subtypes and duplicate
+  outstanding request IDs are rejected as protocol errors; malformed user/tool-result blocks increment a new
+  `evidenceErrors.protocolMalformed` counter; the ABSTAIN permission/dialog fallback reconciles answer
+  delivery like a normal decision; `present_files` leak classification normalizes `..` paths; gate answers
+  require a request id; and an `allow_if` expression no longer fails to compile when a permission input key is
+  a reserved word.
+- **CI / release gates.** The baseline-staleness check rejects non-finite, future-dated, and corrupt
+  timestamps (extracted to a unit-tested `scripts/check-baseline-staleness.ts`); manual container-image
+  publishes require a green CI run unless an explicit break-glass input is set; and the composite Action
+  passes inputs via the environment (closing a shell-injection surface) and accepts a JSON-array `extra-args`
+  that preserves quoting and spaces.
+- **`analyze-skill` top-level `--help`** now documents the `--runtime` flag and exit code `3`
+  (could-not-verify); previously these appeared only in the per-command `analyze-skill --help`.
+- Corrected a phantom assertion key in the `record --margins` documentation
+  (`max_tool_calls` → the real `tool_calls_max`).
+
+### Documentation
+
+- Documented previously-undocumented CLI flags: `record --force` (narrowly overrides the
+  different-scenario slug-collision overwrite refusal) and `record --decider-model`; `probe-dispatch`'s
+  inherited `--decider-cmd` / `--decider-dir` / `--on-unanswered` / `--ablate-skill`; and
+  `skill --timeout` / `--answer-policy`.
+- Clarified the scenario-schema descriptions for `user_visible_artifact` (the assertion value is
+  workRoot-relative — e.g. `outputs/x.md`, not `mnt/`-prefixed) and `gate_answers_delivered`
+  (documented the `: false` confirmed-non-delivery inverse). Regenerated `schema/scenario.schema.json`.
+- Qualified the stable JSON-envelope contract: SPEC §11 now maps commands to envelope families by
+  mechanism (`jsonEnvelope` / `jsonPayloadEnvelope` / dedicated), and README defers to it; clarified
+  that `replay` exit `2` is a whole-cassette operational failure, distinct from an in-cassette
+  malformation (which fails as an exit-`1` assertion).
+- Added a `docs/decisions/` ADR index, `python/README.md` cross-links into the main doc spine, and
+  `RELEASING.md` to `llms.txt`; signposted the specialized `docs/*.md` guides and added `lint-skill` /
+  `analyze-skill` rows to the docs index.
+
 ## [1.2.0] — 2026-07-18
 
 ### Added