cowork-harness 0.21.0 → 0.23.0

package/.claude/skills/cowork-harness/SKILL.md

@@ -3,8 +3,8 @@ name: cowork-harness
 description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
 metadata:
   author: cowork-harness
-  version: 0.21.0
-  tracks-harness: cowork-harness 0.21.0 (baseline desktop-1.17377.2)
+  version: 0.23.0
+  tracks-harness: cowork-harness 0.23.0 (baseline desktop-1.18286.0)
 ---
 
 # cowork-harness
@@ -22,8 +22,8 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
 allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
 the highest-value part. Read it.
 
-> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.21.0` (baseline
-> `desktop-1.17377.2`). If your checkout is newer, prefer the live `--help`, `SPEC.md`, and
+> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.23.0` (baseline
+> `desktop-1.18286.0`). If your checkout is newer, prefer the live `--help`, `SPEC.md`, and
 > `docs/*.md` over this snapshot, and re-run the bundled linter.
 
 ## 0. Preflight — make sure the harness can actually run
@@ -38,7 +38,7 @@ cowork-harness skill ./my-skill "do X"      # run the skill once against the sta
 Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
 
 - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
-- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.21.0**. If it's missing or older, prefix every command with the version floor `npx cowork-harness@>=0.21.0 <cmd>` (Node ≥ 20), or install once with `npm i -g cowork-harness@>=0.21.0`. **Pin `@>=0.21.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.21.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator, `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, and `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field.)
+- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.23.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.23.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.23.0"`. **Pin `@>=0.23.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.23.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator, `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, and `computer_links_resolve` (new in 0.22.0).)
 - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
 - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
 - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
@@ -54,8 +54,13 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
   debugging map).
 
 Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · lint ·
-verify-run · trace · inspect · decide · gates · answer · scaffold · assertions --list · sync · list ·
-boundary-check · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
+verify-run · trace · inspect · diff · stats · decide · gates · answer · scaffold · assertions --list · sync ·
+list · boundary-check · status · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
+
+**Two different `scaffold` tools — don't confuse them.** The native `cowork-harness scaffold <run-id>`
+above turns an already-*recorded* run into a scenario (needs a run to exist first). The bundled
+`scripts/scenario.py scaffold --name … --skill …` in §9 builds a scenario from flags alone, no run
+required. Passing §9's flag set to the native command fails with `unknown flag: --name` (exit 2).
 
 ## 2. Two files: session vs scenario
 
@@ -94,7 +99,8 @@ the folder basename (collision-resolved); there is no `to:` override. See `refer
 | `hostloop` / `cowork` | Production split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production), with native Bash/WebFetch disabled and routed host-side via the workspace SDK-MCP server into a Docker VM sidecar | Highest-fidelity / parity runs. A writable connected folder needs `allow_host_writes: true` (see scenario-schema.md). |
 
 Set the tier in the **scenario's `fidelity:` field**, not a flag — `run` rejects `--fidelity`
-(it's a `skill`-only flag). See `references/fidelity-and-answers.md`.
+(it's a `skill`/`chat` flag; `run` takes fidelity only from the scenario). See
+`references/fidelity-and-answers.md`.
 
 ## 5. Choose an answer path (gates: AskUserQuestion + tool-permission)
 
@@ -265,7 +271,7 @@ known-good skeleton (right tier, scripted `answers:` + `on_unanswered: fail`, co
 separated from live-only ones, one concern per item) and **self-lints its own output**:
 
 ```bash
-S="${CLAUDE_PLUGIN_ROOT}/skills/cowork-harness/scripts/scenario.py"
+S="${CLAUDE_PLUGIN_ROOT}/scripts/scenario.py"
 # Working from a repo checkout instead of an installed plugin?
 # S=".claude/skills/cowork-harness/scripts/scenario.py"
 python3 "$S" scaffold --name report-check --skill ./skills/report-gen \
@@ -288,7 +294,9 @@ keys). Exit code is non-zero on errors (CI-friendly). `scaffold` auto-upgrades t
 for egress on `protocol`, so it never emits a scenario `lint` would reject.
 
 CI placement: a **token-free `replay` PR gate** (content/structure only) + a **nightly live `run`**
-(filesystem/egress). See `references/ci-recipe.md` for the four-stage pipeline.
+(filesystem/egress). Fastest setup: `uses: yaniv-golan/cowork-harness@v1` (a packaged GitHub Action with a
+PR job-summary reporter). See `references/ci-recipe.md` for the Action, the manual step-by-step form, and
+the four-stage pipeline.
 
 ## 10. Debugging with `chat`
 
@@ -326,7 +334,8 @@ are the ones that bite hardest.
    `replay` skips filesystem/egress keys (`file_exists`, `user_visible_artifact`, `artifact_json`,
    `egress_*`, `no_delete_in_outputs`, `self_heal_ran`, `transcript_no_host_path`); a *mixed* item like
    `{result, egress_denied}` greens on `result` while its `egress_denied` half is dropped. (`record`
-   snapshots an `artifacts` manifest, which makes `file_exists`/`user_visible_artifact`/`artifact_json`
+   snapshots an `artifacts` manifest, which makes
+   `file_exists`/`user_visible_artifact`/`artifact_json`/`computer_links_resolve`
    replay-checkable — but the live-only egress keys stay skipped.) *Fix:* put egress/live-only checks on
    a live gate; keep one concern per `assert:` item; run the linter. The harness warns loudly on skip.
 
@@ -389,23 +398,29 @@ are the ones that bite hardest.
     prompt. *Fix:* treat `prompt_asset_missing` as a blocking error in CI by checking the signals
     array.
 13. **`result: success` means the agent didn't error, NOT that the task completed — always assert on
-    artifacts/content.** A turn that ends on a plain-text re-ask ("which file did you mean?") still
-    reports `result: success`. The harness catches a run that ends on a question and did **no productive
-    work after its last gate** — both the no-gate case ("which file?" with no tool calls) AND the
-    *answered-gate-then-re-ask* case (the agent answers an `AskUserQuestion`, then asks again in plain text
-    and stops) — with a **`stalled`** verdict signal (suppress with `allow_stall: true` if ending on a
-    question is intended). The signal is a **tool-position heuristic**, not deliverable detection, so it is
-    imprecise both ways: (a) a post-gate tool *call* clears the flag whether it **succeeded or errored** (false
-    negative — an agent that ran a tool after the gate and still stalled is not caught); and (b) a deliverable
-    written *before* a final confirmation gate does **not** clear it, so a write-then-confirm-then-question run
-    is flagged (false positive — use `allow_stall: true` for a deliberate confirm-terminal skill). The broad
-    guard is therefore YOUR assertions — assert the deliverable
-    (`file_exists` / `artifact_json` / `transcript_matches`), never just `result: success`. Note:
-    `on_unanswered` governs **unanswered** `AskUserQuestion` gates; the `stalled` signal covers stalling
-    *after* one is answered. A "type-it-in-notes" option has **no scripted deterministic answer** today (the
-    `OTHER:` directive works only on the LLM-decider path, not scripted `choose:`; on an options-bearing gate
-    a bare out-of-set LLM answer fails loud (exit 2) — see the LLM-decider free-text note in
-    `references/fidelity-and-answers.md`).
+    artifacts/content.**
+    - A turn that ends on a plain-text re-ask ("which file did you mean?") still reports
+      `result: success`.
+    - The harness catches this with a **`stalled`** verdict signal: a run that ends on a question and
+      did **no productive work after its last gate** — both the no-gate case ("which file?" with no
+      tool calls) AND the *answered-gate-then-re-ask* case (the agent answers an `AskUserQuestion`,
+      then asks again in plain text and stops). Suppress with `allow_stall: true` if ending on a
+      question is intended.
+    - The signal is a **tool-position heuristic**, not deliverable detection, so it is imprecise both
+      ways:
+      - **False negative:** a post-gate tool *call* clears the flag whether it **succeeded or
+        errored** — an agent that ran a tool after the gate and still stalled is not caught.
+      - **False positive:** a deliverable written *before* a final confirmation gate does **not**
+        clear it, so a write-then-confirm-then-question run is flagged — use `allow_stall: true` for a
+        deliberate confirm-terminal skill.
+    - The broad guard is therefore YOUR assertions — assert the deliverable (`file_exists` /
+      `artifact_json` / `transcript_matches`), never just `result: success`.
+    - `on_unanswered` governs **unanswered** `AskUserQuestion` gates; the `stalled` signal covers
+      stalling *after* one is answered — two different failure modes.
+    - **Free-text aside:** a "type-it-in-notes" option has **no scripted deterministic answer** today
+      (the `OTHER:` directive works only on the LLM-decider path, not scripted `choose:`; on an
+      options-bearing gate a bare out-of-set LLM answer fails loud (exit 2) — see the LLM-decider
+      free-text note in `references/fidelity-and-answers.md`).
 14. **A positional `choose` (`first` / index) is order-dependent.** `choose: "2"` survives label drift
     but NOT option *re-ordering* — if the gate presents its options in a different order run-to-run, the
     index lands on a different option (a silent re-record flake). Prefer an exact label when order is
@@ -444,7 +459,8 @@ and the CI recipe, read the files in `references/`.
 - `references/scenario-schema.md` — scenario/session YAML schema, full assertion catalog (with each
   key's replay class), the web_fetch model, and the complete gotcha list.
 - `references/fidelity-and-answers.md` — fidelity tiers, answer paths, the determinism contract.
-- `references/ci-recipe.md` — replay-vs-live lane split and the four-stage GitHub Actions pipeline.
+- `references/ci-recipe.md` — the packaged GitHub Action, replay-vs-live lane split, and the four-stage
+  GitHub Actions pipeline.
 - `scripts/scenario.py` — `scaffold` a valid scenario skeleton and `lint` scenarios for the
   no-silent-false-green invariants (both usable as CI steps).
 - `../../../docs/run-status.md` — checking a background run's status without `ps aux`.

package/.claude/skills/cowork-harness/references/ci-recipe.md

@@ -1,12 +1,59 @@
 # CI recipe — replay vs live lanes
 
-Self-contained reference. Tracks `cowork-harness 0.21.0` (baseline `desktop-1.17377.2`).
+Self-contained reference. Tracks `cowork-harness 0.23.0` (baseline `desktop-1.18286.0`).
 
-**Minimal token-free PR gate** — the smallest thing worth committing; runs on stock GitHub-hosted runners,
-no token/Docker/agent:
+**Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
+job-summary reporter (verdict table, staleness findings, cost/turns when available):
 
 ```yaml
-- run: npm i -g cowork-harness@>=0.21.0
+- uses: yaniv-golan/cowork-harness@v1
+  with:
+    command: replay
+    path: cassettes/
+```
+
+Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
+cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
+independently instead of as one action run per command).
+
+**Live lane (`command: run`) with the packaged Action — self-hosted runner, you stage the binary.** The
+Action never downloads or provisions the agent ELF itself — by design, not an oversight (see "the
+realistic CI shape" below for why). Stage it yourself as an explicit step in *your own* workflow, then
+point the Action at it:
+
+```yaml
+jobs:
+  live:
+    runs-on: [self-hosted, linux, arm64]   # needs Docker + this staged ELF; not a stock GitHub-hosted runner
+    steps:
+      - uses: actions/checkout@v4
+      - name: Stage the agent binary (official channel, sha256-verified — see docs/maintenance.md)
+        run: |
+          V=2.1.197   # match your scenario's pinned baseline's agentVersion
+          curl -fSL "https://downloads.claude.ai/claude-code-releases/$V/linux-arm64/claude" -o "$RUNNER_TEMP/claude-$V"
+          chmod +x "$RUNNER_TEMP/claude-$V"
+          # verify against the committed baseline's sha256 (baselines/desktop-*.json → agentBinary.sha256)
+          # before trusting it — see docs/maintenance.md's "Agent-binary provenance" section.
+          echo "COWORK_AGENT_BINARY=$RUNNER_TEMP/claude-$V" >> "$GITHUB_ENV"
+      - uses: yaniv-golan/cowork-harness@v1
+        with:
+          command: run
+          path: scenarios/
+          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
+```
+
+Why this is a step *you* write, not an Action input the harness provides for you: pulling Anthropic's
+binary is a call about your own relationship with their distribution terms — keeping it in your own
+version-controlled workflow keeps that decision and its execution yours, auditable, and outside any
+third-party action's code. See the [agent-binary provenance runbook](../../../../docs/maintenance.md) for
+the full recovery/verification story (including why `COWORK_AGENT_BINARY` substitutions are
+sha256-*checked* but not hard-blocking on mismatch — it's advisory for an intentional substitution).
+
+**Minimal token-free PR gate (manual form)** — the smallest thing worth committing; runs on stock
+GitHub-hosted runners, no token/Docker/agent:
+
+```yaml
+- run: npm i -g "cowork-harness@>=0.23.0"
 - run: cowork-harness lint scenarios/*.yaml          # no silent false-greens
 - run: cowork-harness verify-cassettes cassettes/    # privacy + staleness
 - run: cowork-harness replay cassettes/              # token-free content/structure
@@ -22,7 +69,8 @@ The split is not just about tokens — it decides **where each lane can run**:
 - **`replay` / `verify-cassettes` (token-free, agent-free).** Replays a recorded cassette
   (`events.jsonl` + `control-out.jsonl`) and lints the committed cassettes. **No model tokens, no
   Docker, no agent binary** — runs on a stock GitHub Actions runner. Evaluates **content** assertions
-  only (`transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `result`, and the verdict
+  only (`transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`, `skill_triggered`,
+  `no_skill_triggered`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `result`, and the verdict
   modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
   `allow_stall` (no-op passes); plus the gate keys `question_asked` / `questions_count_max` /
   `gate_answers_delivered` **if** the cassette has `controlOut`). Filesystem/egress assertions are
@@ -80,8 +128,9 @@ dollar figures). In a skill repo these cassettes get **committed**. So:
   `_KEYS`); empty by default. Redaction is **verdict-preserving** — `record` refuses to write if
   redaction would flip an assertion (a manufactured green). `--no-redact` skips it for known-synthetic
   inputs.
-- **Always-on scan gate** — `verify-cassettes` flags email / currency / bare-domain / local-path matches
-  it finds in the committed cassettes and **exits non-zero**, so "no leak" is a gate, not discipline.
+- **Always-on scan gate** — `verify-cassettes` flags email / currency / bare-domain / local-path /
+  machine-inventory matches it finds in the committed cassettes and **exits non-zero**, so "no leak" is
+  a gate, not discipline.
   Suppress synthetic / public reference names (NVCA, Cooley GO, …) with `--allow <regex>`. (Multi-word
   proper names are NOT a default class — too noisy to gate on; add a pattern via config if your corpus
   needs it.)
@@ -110,6 +159,12 @@ A typical skill repo runs four stages, fastest/cheapest first:
 
 ## GitHub Actions sketch
 
+The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v1` does
+in one step (see the top of this doc) — reach for this form when you need independent per-command
+gating/annotations rather than one action run per command. The nightly live job has no packaged-Action
+equivalent yet (the Action's `command: run` mode needs a self-hosted runner with Docker + the agent binary
+already provisioned, same as the manual form below).
+
 PR gate (token-free — runs on every push):
 
 ```yaml
@@ -124,7 +179,7 @@ jobs:
         with: { node-version: '20' }
       - uses: actions/setup-python@v5
         with: { python-version: '3.x' }                                       # python3 only — PyYAML is bundled with the linter
-      - run: npm i -g cowork-harness@>=0.21.0
+      - run: npm i -g "cowork-harness@>=0.23.0"
       - run: cowork-harness lint scenarios/*.yaml                              # no-silent-false-green (needs python3; PyYAML bundled)
       - run: cowork-harness verify-cassettes cassettes/ --output-format json   # privacy + staleness gate
       - run: cowork-harness replay cassettes/ --output-format json             # token-free content/structure
@@ -153,7 +208,7 @@ jobs:
             echo "live=true" >> "$GITHUB_OUTPUT"
           fi
       - if: steps.guard.outputs.live == 'true'
-        run: npm i -g cowork-harness@>=0.21.0
+        run: npm i -g "cowork-harness@>=0.23.0"
       - if: steps.guard.outputs.live == 'true'
         run: cowork-harness run scenarios/ --output-format json
         env:

package/.claude/skills/cowork-harness/references/fidelity-and-answers.md

@@ -1,6 +1,6 @@
 # Fidelity tiers & answer paths
 
-Self-contained reference. Tracks `cowork-harness 0.21.0` (baseline `desktop-1.17377.2`).
+Self-contained reference. Tracks `cowork-harness 0.23.0` (baseline `desktop-1.18286.0`).
 
 ## Fidelity tiers (`fidelity:` in the scenario)
 
@@ -187,5 +187,6 @@ and `scaffold` refuse to treat a partial run's half-finished output as a passing
   redact from logs (beyond the auth tokens + `ANTHROPIC_CUSTOM_HEADERS`).
 - `COWORK_HARNESS_SOFT_MISSING` — downgrade a missing mount source from hard-error to warn-and-skip.
 - `COWORK_VM_GATEWAY` / `COWORK_VM_PROXY_PORT` / `COWORK_LIMA_INSTANCE` — L2 (microVM) knobs.
-- `COWORK_LOCKDOWN` — default `on`: **aborts loudly** if the L2 guest firewall fails to apply (no
-  silent unprotected run). Set `=off` to opt out and run without isolation deliberately.
+- `COWORK_LOCKDOWN` — default `on`: gates sandbox hardening on every isolated tier — **aborts loudly**
+  if the L2 guest firewall fails to apply (no silent unprotected run), and also gates the
+  `container`/`hostloop` Docker hardening. Set `=off` to opt out and run without isolation deliberately.

package/.claude/skills/cowork-harness/references/scenario-schema.md

@@ -1,7 +1,7 @@
 # Scenario & session schema, assertion catalog, web_fetch, full gotchas
 
-Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.21.0`
-(baseline `desktop-1.17377.2`). If your checkout is newer, prefer the live `docs/scenario.md`,
+Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.23.0`
+(baseline `desktop-1.18286.0`). If your checkout is newer, prefer the live `docs/scenario.md`,
 `docs/session.md`, and `SPEC.md`.
 
 **Minimal scenario** — `prompt` is the only required field:
@@ -61,6 +61,12 @@ skills: [report-gen]                # OPTIONAL — scope the cassette-staleness
                                     # `skills/<name>` dir under a mounted plugin-root) + the plugin's shared
                                     # roots. Fail-closed to whole-tree on an unknown name. Omit = whole tree.
 
+requires_capabilities: [ocr]        # OPTIONAL — declare a capability the skill needs (e.g. office_convert,
+                                    # ocr, pdf_tables, opencv). If the running agent image provably omits
+                                    # one, the harness ABORTS before the paid run (exit 3) — unless the
+                                    # scenario also asserts `allow_missing_capability: true`, which downgrades
+                                    # the abort to a notice and proceeds. Live tiers only.
+
 allow_host_writes: true             # OPTIONAL — required to run `hostloop` fidelity with a WRITABLE
                                     # connected folder (session `mode: rw`/`rwd`): with no container
                                     # around hostloop's native file tools, that combination gives the
@@ -80,8 +86,10 @@ session = your setup.**
 ```yaml
 # model & reasoning
 model: claude-opus-4-8           # omit for the agent default
+account_name: my-account         # OPTIONAL — which configured account/credential set to run as
 effort: high                     # low | medium | high | xhigh
 max_thinking_tokens: 31999       # positive int, or per-model map {default, <model>: <n>}; default 31999
+extended_thinking: true          # INERT — not a real Cowork toggle; use max_thinking_tokens instead
 permission_mode: default         # default | acceptEdits | plan | bypassPermissions
 permission_parity: cowork        # cowork (unscripted tool calls allowed) | strict (deny unscripted)
 
@@ -97,8 +105,9 @@ plugins:
   marketplaces: []               # plugin_marketplaces (git URLs or local paths)
   local_marketplaces: []         # local marketplace dirs (each has a marketplace.json)
   enabled: [my-skill@local]      # enabledPlugins (name@marketplace)
-  local_plugins: [./skills/my-skill]   # host plugin dirs → mnt/.local-plugins/marketplaces/<marketplace>/<plugin>
-                                       #   (≥1.14271.0; older baselines use mnt/.local-plugins/cache)
+  local_plugins: [./skills/my-skill]   # host plugin dirs → mnt/.local-plugins/marketplaces/local-desktop-app-uploads/<plugin>
+                                       #   (the marketplace segment is that fixed synthetic name; ≥1.14271.0 —
+                                       #   older baselines use mnt/.local-plugins/cache)
   remote_plugins: []
 skills:
   local: []                      # extra host skill dirs
@@ -220,6 +229,12 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `subagent_dispatched: <regex>` | a sub-agent whose `agentType` **or dispatch description** matches |
 | `subagent_declared_but_unused: <Tool>` | a sub-agent declared the tool but never used **that** tool (even if it used others) |
 | `dispatch_count_max: <N>` | at most N sub-agents dispatched (records only — does NOT enforce a cap) |
+| `skill_triggered: <regex>` | a skill matching the regex (invoked id, e.g. `"plugin:skill"`) was invoked via the `Skill` tool — evidence-unavailable (not a normal fail) if the agent's init tools have no `Skill` tool |
+| `no_skill_triggered: <regex>` | no invoked skill id matched — the negative-control / description-collision catcher; evidence-unavailable (never a vacuous pass) if invocation data is absent or the `Skill` tool is unobservable |
+| `max_cost_usd: <N>` | the run's SDK-reported cost is ≤ N USD — evidence-unavailable if cost telemetry is absent. **Replay asserts the frozen recording's cost, not fresh spend** — a real regression needs a live `run` |
+| `max_tokens: <N>` | `usage.input_tokens + usage.output_tokens` ≤ N (cache tokens excluded) — same replay caveat as `max_cost_usd` |
+| `tool_calls_max: <N>` | total top-level tool calls (sum of `toolCounts`) ≤ N — meaningfully replay-checkable (re-drive recomputes `toolCounts` deterministically) |
+| `max_turns: <N>` | the SDK-reported (or fallback-counted) turn count ≤ N — meaningfully replay-checkable (re-drive recounts turns deterministically, same as `tool_calls_max`) |
 | `question_asked: <regex>` | the agent asked an AskUserQuestion whose text matches |
 | `questions_count_max: <N>` | the agent asked at most N questions |
 | `gate_answers_delivered: true` | every answered gate's answer reached the model (observed `tool_result`; unobserved = fail) |
@@ -228,10 +243,11 @@ passes only if every key passes. Keep one concern per item unless you mean conju
 | `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
 | `allow_l0_plugin_divergence: true` | verdict modifier — opt into L0/protocol plugin divergence: suppresses the default-fail when a plugin behaves differently at `protocol` (L0) fidelity than under a sandboxed tier. Live tiers only |
 | `allow_stall: true` | verdict modifier — suppresses the `stalled` default-fail when a run ends on a question having done no productive tool work after its last gate (the agent asked for input and stopped — incl. re-asking in plain text after answering an `AskUserQuestion`); assert only when ending on a question is intended, else script the answer (`answer:` / `--answer` / a decider) |
-| `transcript_no_host_path: true` | no host path (`/Users/`, `/opt/cowork/`, `/home/`, `/root/`) leaked into model-visible text — **incompatible with `hostloop`**: its native file tools legitimately expose real host paths, so this fails BY DESIGN there (the harness warns at run start if asserted anyway); use `container`/`microvm` for this check |
+| `transcript_no_host_path: true` | no host path (`/Users/`, `/opt/cowork/`, `/home/`, `/root/`) leaked into model-visible text — **incompatible with `hostloop` AND `protocol`**: hostloop's native file tools legitimately expose real host paths, and protocol (L0) runs the agent's file tools on the real host cwd with no sealed filesystem, so this fails BY DESIGN on both (the harness warns at run start if asserted anyway); use `container`/`microvm` for this check |
 | `egress_denied: <host>` | the host was blocked by the egress proxy |
 | `egress_allowed: <host>` | the host was allowed through |
 | `artifact_json: {artifact, path, …}` | assert a JSON artifact's contents — `equals`/`gt`/`in`/`exists`/`absent`/`is_null` over a dotted `path` (`in` = membership in a list, for a stochastic/LLM value; `absent` ≠ `is_null`; an unresolved intermediate fails loud) |
+| `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). Zero links in the transcript **passes** (presence-gated separately — pair with `transcript_contains` if you also need a link to show up). **Only `true` is valid** (`false` is rejected by the schema) |
 
 `expect_denied: [host, …]` adds one `egress_denied` per host. Run `cowork-harness assertions --list` for this
 table from the live schema. Example: `artifact_json: { artifact: outputs/cap.json, path: me.run_id, equals: "r1" }`.
@@ -270,21 +286,26 @@ sourcing ≠ evaluation (replay warns when you edit one). `verify-run` is the on
 *run dir*; `--assert-from` is the equivalent for a *cassette*.
 
 **Evaluated on replay (content):** `transcript_*`, `tool_*`, `subagent_*`, `dispatch_count_max`,
-`result`. The verdict modifiers `allow_permissive_auto_allow` / `allow_missing_capability` /
-`allow_l0_plugin_divergence` / `allow_stall` are also kept on replay, evaluated as no-op passes.
+`skill_triggered`, `no_skill_triggered`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `max_turns`, `result`
+(`max_cost_usd`/`max_tokens` assert the frozen recording's spend on replay, not fresh spend). The verdict
+modifiers `allow_permissive_auto_allow` / `allow_missing_capability` / `allow_l0_plugin_divergence` /
+`allow_stall` are also kept on replay, evaluated as no-op passes.
 
 **Gate keys — replay only with a `controlOut` cassette:** `question_asked`, `questions_count_max`,
 `gate_answers_delivered`. With `controlOut` present they evaluate; on an old cassette without it, a
 **loud warning** fires and they are **excluded** (not vacuously passed). Re-record to enable them.
 
 **Filesystem — replay-checkable WITH an artifact manifest:** `file_exists`, `user_visible_artifact`,
-`artifact_json` run on replay when the cassette carries an `artifacts` snapshot (`record` captures
-`outputs/` + connected folders; `replay` materializes it). `artifact_json` needs the small-file JSON `body`
-inlined; a hash-only entry still satisfies `file_exists`. Without a manifest (older cassettes) they're
-skipped. A green replay re-confirms *record-time* artifacts, not that the current skill still produces them
-— `replay --strict` fails when the staleness `fingerprint` shows ANY skill/baseline drift, or
-`replay --fail-on-skill-drift` only on skill-source drift; every replay result also reports it class-tagged
-in `staleness[]` for a JSON gate.
+`artifact_json`, `computer_links_resolve` run on replay when the cassette carries an `artifacts` snapshot
+(`record` captures `outputs/` + connected folders; `replay` materializes it). `artifact_json` needs the
+small-file JSON `body` inlined; a hash-only entry still satisfies `file_exists`. `computer_links_resolve`
+resolves a `/sessions/…/mnt/…`-shaped link directly against the manifest, and a host-shaped (hostloop) link
+by first normalizing it to a mount-relative path (recorded connected-folder prefixes + the outputs/uploads
+mounts) — replay has no live filesystem to check a host path against directly (that only happens on a live
+`run`/`verify-run`). Without a manifest (older cassettes) all four are skipped. A green replay re-confirms
+*record-time* artifacts, not that the current skill still produces them — `replay --strict` fails when the
+staleness `fingerprint` shows ANY skill/baseline drift, or `replay --fail-on-skill-drift` only on
+skill-source drift; every replay result also reports it class-tagged in `staleness[]` for a JSON gate.
 
 **Egress + other filesystem — still skipped on replay (live-only):** `no_delete_in_outputs`,
 `self_heal_ran`, `transcript_no_host_path`, `egress_*` / `expect_denied`. These run only on a live `run`/`record`.
@@ -360,7 +381,8 @@ at the top of this file.
    the stochastic path flags the run `nonDeterministic`. The LLM decider is one mechanism, two
    spellings: `on_unanswered: llm` (YAML) and `--decider-llm` (CLI). The bare `--on-unanswered llm`
    is rejected (use `--decider-llm`). `agent` is **retired** — `on_unanswered: agent` is rejected by
-   the schema. (`src/types.ts:276`; `src/cli.ts:652`.)
+   the schema. (`src/types.ts:365` — the `on_unanswered` enum; `src/cli.ts:899` — the CLI-side
+   `--on-unanswered` value check.)
 
 4. **`--on-unanswered first` is non-deterministic too** — it picks option 1 and is flagged
    `nonDeterministic`; not a deterministic substitute for scripted answers.

package/.claude/skills/cowork-harness/scripts/assertion-keys.json

@@ -6,22 +6,29 @@
     "allow_permissive_auto_allow",
     "allow_stall",
     "artifact_json",
+    "computer_links_resolve",
     "dispatch_count_max",
     "egress_allowed",
     "egress_denied",
     "file_exists",
     "gate_answers_delivered",
+    "max_cost_usd",
+    "max_tokens",
+    "max_turns",
     "no_delete_in_outputs",
+    "no_skill_triggered",
     "question_asked",
     "questions_count_max",
     "replay_protocol_fidelity",
     "result",
     "self_heal_ran",
+    "skill_triggered",
     "subagent_declared_but_unused",
     "subagent_dispatched",
     "subagent_tool_absent",
     "subagent_tool_used",
     "tool_called",
+    "tool_calls_max",
     "tool_not_called",
     "tool_result_contains",
     "tool_result_not_contains",

package/.claude/skills/cowork-harness/scripts/scenario.py

@@ -59,12 +59,20 @@ CONTENT_KEYS = {
     "subagent_dispatched",
     "subagent_declared_but_unused",
     "dispatch_count_max",
+    "skill_triggered",
+    "no_skill_triggered",
+    "max_cost_usd",
+    "max_tokens",
+    "tool_calls_max",
+    "max_turns",
 }
 # content keys, but only evaluated on replay when the cassette carries controlOut
 GATE_KEYS = {"question_asked", "questions_count_max", "gate_answers_delivered"}
 # manifest-backed: replay-checkable when the cassette carries an `artifacts` manifest (record snapshots one);
 # a manifest-less cassette skips them. Since the 0.3.0 artifact-manifest these are NOT always live-only.
-MANIFEST_KEYS = {"file_exists", "user_visible_artifact", "artifact_json"}
+# computer_links_resolve joins this bucket (not CONTENT_KEYS): resolving a non-empty link set needs
+# either a live filesystem or the cassette's artifacts manifest — see cassette.ts's manifestKeys comment.
+MANIFEST_KEYS = {"file_exists", "user_visible_artifact", "artifact_json", "computer_links_resolve"}
 # live-only: ALWAYS skipped on replay, with a loud warning (no filesystem, no network on the token-free lane)
 LIVE_ONLY_KEYS = {
     "egress_denied",
@@ -118,8 +126,9 @@ if UNCLASSIFIED_KEYS:
     )
 # Embedded fallback for the top-level scenario keys — kept EQUAL to the generated `topLevelKeys`
 # (test-enforced, like _CLASSIFIED_KEYS for assert keys) so a missing assertion-keys.json can't silently
-# reintroduce key drift. `profile`/`assertions` are NOT here — they are deprecated aliases handled by the
-# special-cases in the unknown-key check below (`k != "assertions" and k != "profile"`).
+# reintroduce key drift. `assertions` is NOT here — it's a hard error handled by its own special-case
+# in the unknown-key check below (`k != "assertions"`). `profile` is retired vocabulary and now falls
+# through to that same unknown-key check like any other typo — no special-case for it.
 _EMBEDDED_TOP_LEVEL_KEYS = {
     "name",
     "baseline",
@@ -252,7 +261,7 @@ def lint_doc(doc, path, raw_lines):
 
     # W: unknown top-level keys (typo or hallucinated schema)
     for k in doc:
-        if k not in TOP_LEVEL_KEYS and k != "assertions" and k != "profile":
+        if k not in TOP_LEVEL_KEYS and k != "assertions":
             findings.append(
                 Finding(
                     "WARN",
@@ -418,19 +427,28 @@ def lint_doc(doc, path, raw_lines):
 _DQ_REGEX_LINE = re.compile(
     r'^\s*-?\s*(' + "|".join(sorted(REGEX_KEYS)) + r')\s*:\s*"([^"]*\\[^"]*)"'
 )
+# A run of an EVEN number of consecutive backslashes in a double-quoted YAML scalar is a properly
+# paired escape (`\\` -> a literal `\`), so e.g. "\\d+ items" decodes to the valid regex `\d+ items` —
+# not a mistake. An ODD run leaves one backslash unpaired, which is the actual footgun (YAML either
+# eats it or errors, depending on what follows). Only flag the odd case.
+_ODD_BACKSLASH_RUN = re.compile(r"\\+")
+
+
+def _has_unpaired_backslash(s):
+    return any(len(m.group(0)) % 2 == 1 for m in _ODD_BACKSLASH_RUN.finditer(s))
 
 
 def _lint_regex_quoting(path, raw_lines):
     out = []
     for i, line in enumerate(raw_lines, start=1):
         m = _DQ_REGEX_LINE.match(line)
-        if m:
+        if m and _has_unpaired_backslash(m.group(2)):
             out.append(
                 Finding(
                     "WARN",
                     "regex-double-quoted",
-                    f"`{m.group(1)}` uses a DOUBLE-quoted regex containing a backslash "
-                    f'("{m.group(2)}") — YAML strips the backslash, so the regex is wrong.',
+                    f"`{m.group(1)}` uses a DOUBLE-quoted regex containing an unescaped backslash "
+                    f'("{m.group(2)}") — YAML strips it, so the regex is wrong.',
                     "Single-quote the regex (e.g. '\\d+') or use a block scalar. Use [\\s\\S] not . to span turns.",
                     path,
                     i,

package/AGENTS.md

@@ -39,8 +39,8 @@ protocol layer or run-loop bookkeeping in the CLI.
   gate-steering silently no-ops (the O7 bug). ELF-verified; a regression test pins it.
 - **"profile" is retired vocabulary.** Synced release ground truth = **`PlatformBaseline`** (`baseline:` /
   `baselines/`); authored setup = **`SessionConfig`** (`session:` / `sessions/`). The `profile:`
-  scenario key is retired vocabulary — still accepted as an alias for `baseline:` with a warning — do not
-  reintroduce the term.
+  scenario key is retired vocabulary — the alias is gone and it is now rejected as an unknown key — do
+  not reintroduce the term.
 - **A new assertion must pick its replay class.** *Content* assertions (read only `ctx.transcript` / the
   record) go in `src/run/cassette.ts` `contentKeys` so they run on the token-free `replay` PR gate;
   *filesystem / egress* assertions do NOT — they are excluded **by omission** from `contentKeys`, so they