cowork-harness 0.31.0 → 0.33.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/.claude/skills/cowork-harness/SKILL.md +23 -11
  2. package/.claude/skills/cowork-harness/references/ci-recipe.md +11 -8
  3. package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +6 -6
  4. package/.claude/skills/cowork-harness/references/scenario-schema.md +6 -6
  5. package/.claude/skills/cowork-harness/references/task-recipes.md +1 -1
  6. package/.claude/skills/cowork-harness/scripts/scenario.py +32 -19
  7. package/CHANGELOG.md +225 -0
  8. package/CONTRIBUTING.md +2 -0
  9. package/DESIGN.md +2 -2
  10. package/README.md +62 -41
  11. package/SPEC.md +6 -4
  12. package/dist/agent/session.js +11 -2
  13. package/dist/cli.js +21 -10
  14. package/dist/run/analyze-skill.js +520 -46
  15. package/dist/run/cassette.js +4 -2
  16. package/dist/run/doctor.js +2 -2
  17. package/dist/run/run.js +3 -3
  18. package/dist/run/scenario-tool.js +90 -25
  19. package/dist/run/subagent-reasoning.js +38 -1
  20. package/dist/runtime/argv.js +3 -0
  21. package/dist/session.js +9 -0
  22. package/dist/types.js +2 -2
  23. package/docs/README.md +5 -5
  24. package/docs/cassette.md +1 -1
  25. package/docs/chat.md +4 -0
  26. package/docs/debugging.md +11 -9
  27. package/docs/gotchas.md +5 -1
  28. package/docs/maintenance.md +1 -1
  29. package/docs/plugin-root.md +4 -0
  30. package/docs/protocol.md +5 -0
  31. package/docs/run-status.md +24 -0
  32. package/docs/scenario.md +10 -4
  33. package/docs/session.md +11 -4
  34. package/docs/subagents.md +167 -16
  35. package/examples/README.md +21 -1
  36. package/examples/replays/README.md +1 -1
  37. package/llms.txt +7 -2
  38. package/package.json +1 -1
  39. package/python/README.md +4 -1
  40. package/schema/run-result.json +10 -1
  41. package/schema/scenario.schema.json +2 -2
  42. package/schema/session.schema.json +7 -0
  43. package/scripts/check-versions.ts +35 -16
@@ -3,8 +3,8 @@ name: cowork-harness
3
3
  description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
4
4
  metadata:
5
5
  author: cowork-harness
6
- version: 0.31.0
7
- tracks-harness: cowork-harness 0.31.0 (baseline desktop-1.20186.1)
6
+ version: 0.33.0
7
+ tracks-harness: cowork-harness 0.33.0 (baseline desktop-1.20186.1)
8
8
  ---
9
9
 
10
10
  # cowork-harness
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
22
22
  allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
23
23
  the highest-value part. Read it.
24
24
 
25
- > **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.31.0` (baseline
25
+ > **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.33.0` (baseline
26
26
  > `desktop-1.20186.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
27
27
  > `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
28
28
 
@@ -38,10 +38,22 @@ cowork-harness skill ./my-skill "do X" # run the skill once against the sta
38
38
  Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
39
39
 
40
40
  - **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
41
- - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.31.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.31.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.31.0"`. **Pin `@>=0.31.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.31.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-patterns-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0; `--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, `computer_links_resolve` (new in 0.22.0), `semantic_matches` (an LLM judge grades a fixed `rubric` of claims against the run's answer — **live-only**, so it is evidence-unavailable / skipped-loud on replay, never a vacuous pass; new in 0.27.0), **glob-matched** `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` (a pattern like `mcp__workspace__*` matches any tool in the family; exact names match exactly; new in 0.28.0), the five path-gate assertion keys `no_vm_path_file_op`/`vm_path_denied`/`path_denied`/`no_path_denied`/`subagent_file_write`, the session-level `agent_env` knob, and resolved sub-agent identity on dispatch records (`resolvedAgentType`/`resolvedModel`; new in 0.30.0), the `lint-skill` (static host-loop footgun + `subagent_type` resolution linter) / `analyze-skill` (advisory `/sessions`-path static scan, `--strict`, `analyze-skill: ignore` marker) / `probe-dispatch` (single-dispatch mechanics probe) commands, `status --latest-for` (resolve a scenario's newest run dir by run time, not directory mtime), the `subagent_dispatch_healthy` composite assertion, and the persisted `result.json` fields `verdict`, `subagents[].referencesRead`/`subagents[].reasoning`, plus the `toolCounts`/`toolErrors`/`toolDurations` shape distinction (all new in 0.31.0).)
41
+ - **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
42
+ - **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.33.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.33.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.33.0"`. **Pin `@>=0.33.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
43
+
44
+ What the ≥ 0.33.0 floor gates, by release:
45
+
46
+ - **core set (pre-0.21.0 vintage, or mixed):** `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes` incl. the `--allow-domain`/`--allow-email`/`--allow-patterns-file` allows (`--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, and `/help` in the REPL.
47
+ - **0.21.0:** `verify-cassettes --allow-path` (`path` — local absolute filesystem paths — is the scanner's 4th class), and `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field.
48
+ - **0.22.0:** `computer_links_resolve`.
49
+ - **0.28.0:** `semantic_matches` (an LLM judge grades a fixed `rubric` of claims against the run's answer — **live-only**, so it is evidence-unavailable / skipped-loud on replay, never a vacuous pass), and **glob-matched** `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` (a pattern like `mcp__workspace__*` matches any tool in the family; exact names match exactly).
50
+ - **0.30.0:** resolved sub-agent identity on dispatch records (`resolvedAgentType`/`resolvedModel`), the five path-gate assertion keys `no_vm_path_file_op`/`vm_path_denied`/`path_denied`/`no_path_denied`/`subagent_file_write`, and the session-level `agent_env` knob.
51
+ - **0.31.0:** the `lint-skill` (static host-loop footgun + `subagent_type` resolution linter) / `analyze-skill` (advisory `/sessions`-path static scan, `--strict`, `analyze-skill: ignore` marker) / `probe-dispatch` (single-dispatch mechanics probe) commands, `status --latest-for` (resolve a scenario's newest run dir by run time, not directory mtime), the `subagent_dispatch_healthy` composite assertion, and the persisted `result.json` fields `verdict`, `subagents[].referencesRead`/`subagents[].reasoning`, plus the `toolCounts`/`toolErrors`/`toolDurations` shape distinction.
52
+ - **0.32.0:** `analyze-skill`'s directory scan now covering a skill/plugin's full contract surface (recursive `agents/`/`references/`/`commands/`, plugin-root-aware, symlink-following) with line/block-scoped `analyze-skill: ignore-next-line`/`ignore-start`/`ignore-end` markers and multi-path/glob input, and `lint-skill`'s provable in-plugin `subagent_type` typo now a WARN that gates under `--strict`.
53
+ - **0.33.0:** the `redacted` marker on display-omitted reasoning — `subagents[].reasoning` and the top-level `thinking[]` now carry `{text:"", redacted:true}` when the model returns a signed-but-empty thought (so "reasoned, text omitted" is distinct from "no thought"), plus the fenced `debug.thinking_display` escape hatch.
42
54
  - **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
43
55
  - **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
44
- - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
56
+ - **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred), or `ANTHROPIC_API_KEY` / `ANTHROPIC_AUTH_TOKEN`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
45
57
  - **`--dotenv` is a GLOBAL flag — put it BEFORE the subcommand.** `cowork-harness --dotenv .env record …`, never `cowork-harness record … --dotenv .env`. Every *other* flag is subcommand-level, so muscle memory fights this one; the harness rejects the misplaced form with an exact-fix error, but placing it first avoids the round-trip.
46
58
 
47
59
  ## Orient — the three loops
@@ -63,7 +75,7 @@ reproducible regression (Part II), and **debug** a run that misbehaved or greene
63
75
  the path, `trace <run-id>` finds it). **Localize the failure post-hoc** from that evidence:
64
76
  `cowork-harness trace <run-dir>`'s views + the emitted `result.json` to see what the run actually did,
65
77
  then `verify-run` to re-check a suspect assertion — all token-free, no Docker, no re-record. This is
66
- the loop 0.31.0's observability is built for; the *Triage* and *Inspecting a run's observability
78
+ the loop 0.32.0's observability is built for; the *Triage* and *Inspecting a run's observability
67
79
  output* sections in **Part III — Debug** are the detail (the fuller human-facing map lives in
68
80
  `docs/debugging.md` — repo-only, not shipped with the installed skill).
69
81
  - **Multi-turn / interactive reproduction** → `cowork-harness chat` (interactive; gates answered at the
@@ -240,12 +252,12 @@ Full model in `references/scenario-schema.md`.
240
252
  Don't hand-write the YAML from memory — that's how invented keys (`assertions:` vs `assert:`,
241
253
  `json_file`, `answer_policy`) creep in. Start from the bundled generator, which emits the
242
254
  known-good skeleton (right tier, scripted `answers:` + `on_unanswered: fail`, content assertions
243
- separated from live-only ones, one concern per item) and **self-lints its own output**:
255
+ separated from live-only ones, one concern per item) and **self-lints its own output**. The
256
+ generator is the bundled `scripts/scenario.py` — installed as a plugin, point `S` at
257
+ `${CLAUDE_PLUGIN_ROOT}/scripts/scenario.py`; from a repo checkout, use the literal path below:
244
258
 
245
259
  ```bash
246
- S="${CLAUDE_PLUGIN_ROOT}/scripts/scenario.py"
247
- # Working from a repo checkout instead of an installed plugin?
248
- # S=".claude/skills/cowork-harness/scripts/scenario.py"
260
+ S=".claude/skills/cowork-harness/scripts/scenario.py"
249
261
  python3 "$S" scaffold --name report-check --skill ./skills/report-gen \
250
262
  --prompt "Generate the weekly report to outputs/report.md." \
251
263
  --content 'weekly report' --artifact outputs/report.md \
@@ -372,7 +384,7 @@ than a stuck `"running"`.)
372
384
  ### Place assertions in the right CI lane
373
385
 
374
386
  CI placement: a **token-free `replay` PR gate** (content/structure only) + a **nightly live `run`**
375
- (filesystem/egress). Fastest setup: `uses: yaniv-golan/cowork-harness@v1` (a packaged GitHub Action with a
387
+ (filesystem/egress). Fastest setup: `uses: yaniv-golan/cowork-harness@main` (a packaged GitHub Action with a
376
388
  PR job-summary reporter). See `references/ci-recipe.md` for the Action, the manual step-by-step form, and
377
389
  the four-stage pipeline.
378
390
 
@@ -1,17 +1,20 @@
1
1
  # CI recipe — replay vs live lanes
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 0.31.0` (baseline `desktop-1.20186.1`).
3
+ Self-contained reference. Tracks `cowork-harness 0.33.0` (baseline `desktop-1.20186.1`).
4
4
 
5
5
  **Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
6
6
  job-summary reporter (verdict table, staleness findings, cost/turns when available):
7
7
 
8
8
  ```yaml
9
- - uses: yaniv-golan/cowork-harness@v1
9
+ - uses: yaniv-golan/cowork-harness@main
10
10
  with:
11
11
  command: replay
12
12
  path: cassettes/
13
13
  ```
14
14
 
15
+ The Action's `version` input defaults to `latest` — intentional so a copy-pasted recipe tracks the current
16
+ release; pin an exact version (e.g. `version: "0.33.0"`) for reproducible CI.
17
+
15
18
  Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
16
19
  cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
17
20
  independently instead of as one action run per command).
@@ -29,13 +32,13 @@ jobs:
29
32
  - uses: actions/checkout@v4
30
33
  - name: Stage the agent binary (official channel, sha256-verified — see docs/maintenance.md)
31
34
  run: |
32
- V=2.1.197 # match your scenario's pinned baseline's agentVersion
35
+ V=2.1.205 # match your scenario's pinned baseline's agentVersion
33
36
  curl -fSL "https://downloads.claude.ai/claude-code-releases/$V/linux-arm64/claude" -o "$RUNNER_TEMP/claude-$V"
34
37
  chmod +x "$RUNNER_TEMP/claude-$V"
35
38
  # verify against the committed baseline's sha256 (baselines/desktop-*.json → agentBinary.sha256)
36
39
  # before trusting it — see docs/maintenance.md's "Agent-binary provenance" section.
37
40
  echo "COWORK_AGENT_BINARY=$RUNNER_TEMP/claude-$V" >> "$GITHUB_ENV"
38
- - uses: yaniv-golan/cowork-harness@v1
41
+ - uses: yaniv-golan/cowork-harness@main
39
42
  with:
40
43
  command: run
41
44
  path: scenarios/
@@ -54,7 +57,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
54
57
  GitHub-hosted runners, no token/Docker/agent:
55
58
 
56
59
  ```yaml
57
- - run: npm i -g "cowork-harness@>=0.31.0"
60
+ - run: npm i -g "cowork-harness@>=0.33.0"
58
61
  - run: cowork-harness lint scenarios/*.yaml # no silent false-greens
59
62
  - run: cowork-harness verify-cassettes cassettes/ # privacy + staleness
60
63
  - run: cowork-harness replay cassettes/ # token-free content/structure
@@ -174,7 +177,7 @@ A typical skill repo runs four stages, fastest/cheapest first:
174
177
 
175
178
  ## GitHub Actions sketch
176
179
 
177
- The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@v1` does
180
+ The PR gate below is the manual, step-by-step version of what `uses: yaniv-golan/cowork-harness@main` does
178
181
  in one step (see the top of this doc) — reach for this form when you need independent per-command
179
182
  gating/annotations rather than one action run per command. The nightly live job has no packaged-Action
180
183
  equivalent yet (the Action's `command: run` mode needs a self-hosted runner with Docker + the agent binary
@@ -194,7 +197,7 @@ jobs:
194
197
  with: { node-version: '20' }
195
198
  - uses: actions/setup-python@v5
196
199
  with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
197
- - run: npm i -g "cowork-harness@>=0.31.0"
200
+ - run: npm i -g "cowork-harness@>=0.33.0"
198
201
  - run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
199
202
  - run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
200
203
  - run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
@@ -223,7 +226,7 @@ jobs:
223
226
  echo "live=true" >> "$GITHUB_OUTPUT"
224
227
  fi
225
228
  - if: steps.guard.outputs.live == 'true'
226
- run: npm i -g "cowork-harness@>=0.31.0"
229
+ run: npm i -g "cowork-harness@>=0.33.0"
227
230
  - if: steps.guard.outputs.live == 'true'
228
231
  run: cowork-harness run scenarios/ --output-format json
229
232
  env:
@@ -1,6 +1,6 @@
1
1
  # Fidelity tiers & answer paths
2
2
 
3
- Self-contained reference. Tracks `cowork-harness 0.31.0` (baseline `desktop-1.20186.1`).
3
+ Self-contained reference. Tracks `cowork-harness 0.33.0` (baseline `desktop-1.20186.1`).
4
4
 
5
5
  ## Fidelity tiers (`fidelity:` in the scenario)
6
6
 
@@ -13,10 +13,10 @@ Self-contained reference. Tracks `cowork-harness 0.31.0` (baseline `desktop-1.20
13
13
  | `cowork` | Auto-picks `hostloop` vs `container` the way Cowork itself does for the synced release | "Do what real Cowork does for this release." |
14
14
 
15
15
  - `hostloop` / `cowork` are the production-faithful path; `container` is the practical default.
16
- - Boundary assertions (`egress_*`, `expect_denied`) are enforced at `container`, `microvm`, and
17
- `hostloop` (`container`'s and `hostloop`'s `bash` share the same Docker sandbox + egress proxy —
18
- `hostloop`'s native file tools run with no container at all, gated instead by a path-containment hook).
19
- Only `protocol` is rejected.
16
+ - Boundary assertions (`egress_*`, `expect_denied`) are enforced at `container`, `microvm`, `hostloop`,
17
+ and `cowork` (`cowork` auto-resolves to a sandboxed tier; `container`'s and `hostloop`'s `bash` share
18
+ the same Docker sandbox + egress proxy — `hostloop`'s native file tools run with no container at all,
19
+ gated instead by a path-containment hook). Only `protocol` is rejected.
20
20
  - A `hostloop` scenario with a **writable** connected folder (`mode: rw`/`rwd`) needs `allow_host_writes:
21
21
  true` — with no container around the native file tools, that combination gives the agent genuine,
22
22
  software-checked-only host filesystem access. Read-only folders and folder-less runs need no opt-in.
@@ -176,7 +176,7 @@ and `scaffold` refuse to treat a partial run's half-finished output as a passing
176
176
  - `COWORK_HARNESS_RUNS_DIR` (or `--run-dir <path>`) — override the default run-output root `~/.cowork-harness/runs` (out of any working tree). flag > env > default.
177
177
  - `COWORK_HARNESS_DECIDER_CMD_TIMEOUT_MS` / `COWORK_HARNESS_LLM_TIMEOUT_MS` — decider backstops
178
178
  (default 600 s; **fail loud** on timeout).
179
- - `COWORK_HARNESS_DECIDER_DIR_POLL_MS` / `_TIMEOUT_MS` — the `--decider-dir` rendezvous.
179
+ - `COWORK_HARNESS_DECIDER_DIR_POLL_MS` / `_TIMEOUT_MS` — the `--decider-dir` rendezvous (poll defaults: 300 ms for the run-side rendezvous, 500 ms for `gates --follow`).
180
180
  - `COWORK_HARNESS_DIALOG_TIMEOUT_MS` — dialog auto-cancel (default 6 s).
181
181
  - `COWORK_HARNESS_LLM_MAX_BYTES` — stdout bound on `--decider-llm` (default 8 MiB).
182
182
  - `COWORK_HARNESS_LLM_RETRIES` — bounded retries for a transient non-zero `claude -p` exit on the
@@ -1,6 +1,6 @@
1
1
  # Scenario & session schema, assertion catalog, web_fetch, full gotchas
2
2
 
3
- Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.31.0`
3
+ Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.33.0`
4
4
  (baseline `desktop-1.20186.1`). If your checkout is newer, prefer the live `docs/scenario.md`,
5
5
  `docs/session.md`, and `SPEC.md`.
6
6
 
@@ -37,7 +37,7 @@ fidelity: container # protocol | container | microvm | hostloop
37
37
  execution: local # OPTIONAL — orthogonal to fidelity (a privilege/sandbox tier, all
38
38
  # local): local (default) | cloud-describe (RESERVED — no runner
39
39
  # exists yet; authoring it is a load-time error, not a silent no-op)
40
- on_unanswered: fail # policy for unscripted gates: fail | prompt | first | llm
40
+ on_unanswered: fail # policy for unscripted gates: fail | prompt | first | llm — run rejects prompt
41
41
  # ("agent" is retired — no longer a valid value)
42
42
 
43
43
  prompt: | # the user turn
@@ -312,7 +312,7 @@ same set live from the schema.
312
312
  | `egress_allowed: <host>` | the host was allowed through |
313
313
  | `no_mcp_error: true` | no MCP round-trip failed (`RunResult.mcpErrors` is empty — no unhandled server, no handler throw) — live-only: MCP round-trips are harness-computed, not in the SDK stdout stream, so evidence-unavailable on replay (never a vacuous pass). **Only `true` is valid** |
314
314
  | `max_peak_rss_bytes: <N>` | peak sampled RSS of the agent sandbox ≤ N bytes (`RunResult.resources.peakRssBytes`) — live-only: replay never spawns a sandbox to sample, so evidence-unavailable on replay/protocol (never a vacuous pass); also evidence-unavailable when sampling captured no RSS value |
315
- | `semantic_matches: {rubric: [...], min_pass?, judge_model?}` | a pinned LLM judge grades each fixed `rubric` claim against the run's answer — the **union of the agent's final result text (`RunResult.finalMessage`), the transcript, and the final on-disk content of any files the agent authored during the run** — so a claim about content the skill led the agent to *write to a file* grades as reliably as one about inlined prose (authored-file evidence is unavailable on the **microvm** tier — no pre-run manifest to diff against, so no files are captured; `container`/`hostloop` do capture them). Beyond the microvm case, when the authored-file evidence backing the judged document is **incomplete** — a file dropped at the capture-size cap, unreadable at read-back, or (on `--resume`) the scratchpad walk skipped — the assert fails evidence-unavailable rather than trusting a judge grade over a partial document; this is separate from the malformed-grade `judgeInvalid` path below. The assert passes iff ≥ `min_pass` claims pass (default: all — avoid for a gating scenario). Results align by claim index and are recorded per-claim in `RunResult.assertions[].semanticClaims` (`[{index, claim, pass}]`, so a consumer can diff the per-claim profile across runs); a rep whose grade can't be parsed (after one retry) is marked `RunResult.assertions[].judgeInvalid` and **never silently dropped** — it is excluded from the pass denominator, and the guard against a misleading score from that exclusion is the gate's minimum-valid-rep floor (`MIN_VALID` ≥ 4) plus this visibility, not a claim that denominator-shrinking inflation is impossible. Within a rep, a grade that's still unparseable after the retry **fails that assert outright** (evidence-unavailable, not a vacuous pass) — a persistently-flaky judge reds the run rather than silently passing. `judge_model` pins the grader (a dated id keeps a before/after comparison reproducible). Live-only: the judge is a live model call, so evidence-unavailable / skipped-loud on replay (never a vacuous pass) |
315
+ | `semantic_matches: {rubric: [...], min_pass?, judge_model?}` | a pinned LLM judge grades each fixed `rubric` claim against the run's answer — the **union of the agent's final result text (`RunResult.finalMessage`), the transcript, and the final on-disk content of any files the agent authored during the run** — so a claim about content the skill led the agent to *write to a file* grades as reliably as one about inlined prose (authored-file evidence is unavailable on the **microvm** tier — no pre-run manifest to diff against, so no files are captured; `container`/`hostloop` do capture them). Beyond the microvm case, when the authored-file evidence backing the judged document is **incomplete** — a file dropped at the capture-size cap, unreadable at read-back, or (on `--resume`) the scratchpad walk skipped — the assert fails evidence-unavailable rather than trusting a judge grade over a partial document; this is separate from the malformed-grade `judgeInvalid` path below. The assert passes iff ≥ `min_pass` claims pass (default: all — avoid for a gating scenario). Results align by claim index and are recorded per-claim in `RunResult.assertions[].semanticClaims` (`[{index, claim, pass}]`, so a consumer can diff the per-claim profile across runs); a rep whose grade can't be parsed (after one retry) is marked `RunResult.assertions[].judgeInvalid` and **never silently dropped** — it is excluded from the pass denominator, and the guard against a misleading score from that exclusion is the gate's minimum-valid-rep floor (`MIN_VALID` ≥ 4) plus this visibility, not a claim that denominator-shrinking inflation is impossible. Within a rep, a grade that's still unparseable after the retry **fails that assert outright** (evidence-unavailable, not a vacuous pass) — a persistently-flaky judge reds the run rather than silently passing. `judge_model` pins the grader (default when neither it nor `COWORK_HARNESS_JUDGE_MODEL` is set: `claude-opus-4-8`; a dated id keeps a before/after comparison reproducible). Live-only: the judge is a live model call, so evidence-unavailable / skipped-loud on replay (never a vacuous pass) |
316
316
  | `artifact_json: {artifact, path, …}` | assert a JSON artifact's contents — `equals`/`gt`/`in`/`exists`/`absent`/`is_null` over a dotted `path` (`in` = membership in a list, for a stochastic/LLM value; `absent` ≠ `is_null`; an unresolved intermediate fails loud) |
317
317
  | `computer_links_resolve: true` | every `computer://` link in the model-visible transcript resolves to an artifact that exists in the run's collected outputs/mounts — a dangling link fails, naming which target was checked (a live host path, the collected work tree, or the replay manifest). **Requires ≥1 link** (zero links fails — use `computer_links_resolve_if_present` for the presence-free variant). **Only `true` is valid** (`false` is rejected by the schema) |
318
318
  | `computer_links_resolve_if_present: true` | like `computer_links_resolve` but passes vacuously when the transcript has zero `computer://` links — the presence-free variant. **Only `true` is valid** |
@@ -329,7 +329,7 @@ paraphrases (that re-records red). Structured JSON → assert it in YAML with **
329
329
  `path` + operator); use the pytest lane (`assert_artifact_json`) only for predicates too complex for a
330
330
  dotted path.
331
331
 
332
- **VerdictSignals in `result.signals`:** `computeVerdict` pushes signals into `result.signals`; most
332
+ **VerdictSignals in `result.verdict.signals`:** `computeVerdict` pushes signals into `result.verdict.signals`; most
333
333
  are **fail**-severity (they flip the run's pass/exit code even though `result.result` itself stays
334
334
  `"success"`) and only three are **warn**-severity (informational, never flip pass/fail). Current signal
335
335
  codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
@@ -353,7 +353,7 @@ codes (`VerdictSignal["code"]` in `src/run/verdict.ts`):
353
353
 
354
354
  A **fail**-severity signal does not change `result.result` (still `"success"`), but it DOES fail the
355
355
  overall run verdict and exit code — `assert result: success` alone won't catch it; check
356
- `result.signals[].severity` or the run's exit code. Only the three **warn** codes are truly benign.
356
+ `result.verdict.signals[].severity` or the run's exit code. Only the three **warn** codes are truly benign.
357
357
 
358
358
  ## Replay class
359
359
 
@@ -491,7 +491,7 @@ reference omits). Neither list is a strict superset of the other — reach for t
491
491
  the stochastic path flags the run `nonDeterministic`. The LLM decider is one mechanism, two
492
492
  spellings: `on_unanswered: llm` (YAML) and `--decider-llm` (CLI). The bare `--on-unanswered llm`
493
493
  is rejected (use `--decider-llm`). `agent` is **retired** — `on_unanswered: agent` is rejected by
494
- the schema. (`src/types.ts:365` — the `on_unanswered` enum; `src/cli.ts:899` — the CLI-side
494
+ the schema. (`src/types.ts` — the `on_unanswered` enum; `src/cli.ts:899` — the CLI-side
495
495
  `--on-unanswered` value check.)
496
496
 
497
497
  4. **`--on-unanswered first` is non-deterministic too** — it picks option 1 and is flagged
@@ -2,7 +2,7 @@
2
2
 
3
3
  Each recipe composes facts that live scattered across SKILL.md and the other references into one
4
4
  decision path. Every one answers a question a real fleet owner had to work out the hard way. Facts track the harness version in SKILL.md's
5
- front-matter (currently 0.31.0). Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
5
+ front-matter (currently 0.32.0). Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
6
6
  Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
7
7
  needed if your CLI meets SKILL.md's version floor.
8
8
 
@@ -1047,9 +1047,10 @@ def _finding_subagent_not_found_in_plugin(path, line, value, plugin_name, agent,
1047
1047
  own enclosing plugin) but whose agent isn't in its enumerated agents/*.md set. Unlike
1048
1048
  `subagent-type-unknown`, this can never be another binary's built-in — the namespace prefix
1049
1049
  already commits it to this plugin, and the plugin's agent set was fully enumerable — so it's a
1050
- provable typo, not an unconfirmable unknown."""
1050
+ provable typo, not an unconfirmable unknown. Provable means it can be a hard gate: WARN (not
1051
+ INFO), so `lint-skill --strict` fails on it."""
1051
1052
  return Finding(
1052
- "INFO",
1053
+ "WARN",
1053
1054
  "subagent-type-not-found-in-plugin",
1054
1055
  f"pinned type `{value}` names this plugin (`{plugin_name}`) but `{agent}` is not among its "
1055
1056
  f"agents [{', '.join(sorted_agents)}] — likely a typo.",
@@ -1063,20 +1064,23 @@ def _finding_subagent_not_found_in_plugin(path, line, value, plugin_name, agent,
1063
1064
 
1064
1065
  def _classify_subagent_type(value, plugin_name, plugin_agent_types):
1065
1066
  """subagent_type severity ladder. Returns a Finding-builder (path, line, value) -> Finding, or None if
1066
- clean. Severity is ALWAYS INFO — see the HONEST LIMIT note above this section; never WARN/ERROR.
1067
+ clean. Only ONE outcome is a hard gate: `subagent-type-not-found-in-plugin` is WARN (a provable
1068
+ typo, so `lint-skill --strict` fails on it); the two unconfirmable outcomes
1069
+ (`subagent-type-unresolvable`, `subagent-type-unknown`) stay INFO — see the HONEST LIMIT note
1070
+ above this section for why those two can never be proven.
1067
1071
 
1068
1072
  Precedence:
1069
1073
  1. Resolves in-plugin, or is `general-purpose` -> clean (None).
1070
1074
  2. Has a `<prefix>:<agent>` shape whose prefix EQUALS the resolved plugin's own name AND the
1071
- plugin's agent set was non-empty (i.e. enumerable) -> `subagent-type-not-found-in-plugin`.
1072
- A value namespaced under this plugin's own name can never be another binary's built-in, so
1073
- once the plugin is fully enumerated a miss here is a provable typo, not an unconfirmable
1074
- unknown.
1075
+ plugin's agent set was non-empty (i.e. enumerable) -> `subagent-type-not-found-in-plugin`
1076
+ (WARN — provable typo, gates under `--strict`). A value namespaced under this plugin's own
1077
+ name can never be another binary's built-in, so once the plugin is fully enumerated a miss
1078
+ here is a provable typo, not an unconfirmable unknown.
1075
1079
  3. Has a `<prefix>:<agent>` shape whose prefix does NOT equal the resolved plugin's name (or
1076
- there's no resolved plugin at all) -> `subagent-type-unresolvable` (belongs to another
1080
+ there's no resolved plugin at all) -> `subagent-type-unresolvable` (INFO — belongs to another
1077
1081
  plugin, or an empty plugin_name means we truly can't tell whose namespace it is).
1078
1082
  4. Otherwise (no colon, or same-plugin prefix but the plugin set was empty/unenumerable) ->
1079
- `subagent-type-unknown` — genuinely can't confirm statically.
1083
+ `subagent-type-unknown` (INFO) — genuinely can't confirm statically.
1080
1084
  """
1081
1085
  if value == "general-purpose" or value in plugin_agent_types:
1082
1086
  return None
@@ -1170,10 +1174,10 @@ def cmd_lint_skill(args):
1170
1174
  _print_findings(all_findings, n_files, kind="skill file", clean_suffix=" — no Cowork host-loop footguns.")
1171
1175
  has_error = any(x.severity == "ERROR" for x in all_findings)
1172
1176
  # --strict fails on WARN too, per its own --help text ("exit non-zero on WARN too, not just ERROR")
1173
- # — but NEVER on INFO. The subagent_type ladder (subagent-type-unresolvable /
1174
- # -not-found-in-plugin / -unknown) and plugin-root-guarded are always INFO by design (there is no
1175
- # harness registry to disprove an unknown value against — see the subparser help above), so they
1176
- # must always be surfaced, never failed, even under --strict.
1177
+ # — but NEVER on INFO. Of the subagent_type ladder, only `subagent-type-not-found-in-plugin` is
1178
+ # WARN (a provable in-plugin typo — see the subparser help above); `subagent-type-unresolvable` and
1179
+ # `subagent-type-unknown` stay INFO by design (there is no harness registry to disprove an unknown
1180
+ # value against), so those two must always be surfaced, never failed, even under --strict.
1177
1181
  has_warn = any(x.severity == "WARN" for x in all_findings)
1178
1182
  if has_error or (args.strict and has_warn):
1179
1183
  return 1
@@ -1364,17 +1368,26 @@ def main(argv=None):
1364
1368
  "Also statically resolves any pinned `subagent_type` value in the SKILL.md against the "
1365
1369
  "enclosing plugin's `agents/*.md` (see `resolve-agent-types`): a value that resolves in-plugin "
1366
1370
  "or is `general-purpose` is clean; a `<this-plugin>:<agent>` whose agent is missing from an "
1367
- "enumerable plugin is a provable typo, reported as `subagent-type-not-found-in-plugin`; a "
1368
- "`<other-plugin>:<agent>` is reported as `subagent-type-unresolvable`; any other unresolved "
1369
- "value (including a same-plugin prefix the linter couldn't enumerate) is "
1370
- "`subagent-type-unknown` — all three are always INFO, never WARN, since there is no harness "
1371
- "registry of built-in agent types to disprove an unknown value against."
1371
+ "enumerable plugin is a provable typo, reported as `subagent-type-not-found-in-plugin` "
1372
+ "(WARN — gates under `--strict`, since the namespace prefix already commits it to this "
1373
+ "plugin and its agent set was fully enumerated); a `<other-plugin>:<agent>` is reported as "
1374
+ "`subagent-type-unresolvable` (INFO); any other unresolved value (including a same-plugin "
1375
+ "prefix the linter couldn't enumerate) is `subagent-type-unknown` (INFO) — those two stay "
1376
+ "INFO, never WARN, since there is no harness registry of built-in agent types to disprove "
1377
+ "an unknown value against.\n\n"
1378
+ "Plain `lint-skill` (no `--strict`) is ADVISORY — it prints findings but exits 0 on "
1379
+ "WARN/INFO. CI should invoke `lint-skill --strict` to actually gate on the WARN-class "
1380
+ "findings above (the two host-loop footguns and the provable subagent_type typo)."
1372
1381
  ),
1373
1382
  formatter_class=argparse.RawDescriptionHelpFormatter,
1374
1383
  )
1375
1384
  lsp.add_argument("paths", nargs="+", help="SKILL.md file(s) or skill director(ies) to inspect")
1376
1385
  lsp.add_argument("--json", action="store_true", help="emit findings as JSON")
1377
- lsp.add_argument("--strict", action="store_true", help="exit non-zero on WARN too, not just ERROR")
1386
+ lsp.add_argument(
1387
+ "--strict",
1388
+ action="store_true",
1389
+ help="exit non-zero on WARN too, not just ERROR (CI-recommended invocation; plain lint-skill is advisory-only)",
1390
+ )
1378
1391
  lsp.set_defaults(func=cmd_lint_skill)
1379
1392
 
1380
1393
  rap = sub.add_parser(
package/CHANGELOG.md CHANGED
@@ -6,6 +6,226 @@ All notable changes to this project are documented here. The format is based on
6
6
 
7
7
  ## [Unreleased]
8
8
 
9
+ ## [0.33.0] — 2026-07-13
10
+
11
+ ### Fixed
12
+
13
+ - **Empty thinking was surfaced as if the model hadn't reasoned — both `subagents[].reasoning` and the
14
+ top-level `thinking[]`.** The 0.31.0 sub-agent field captured TEXT turns fine but every `thinking` turn
15
+ came through as `{kind:"thinking", text:""}`, and the older top-level `thinking[]` field does the same
16
+ on current-gen models — a consumer couldn't tell "reasoned but text unavailable" from "no thought."
17
+ Root cause (binary-verified against the staged 2.1.205 agent, corpus-corroborated): it is a
18
+ **request-side display mode**, not a persist-time strip — the API's `thinking.display` resolves to
19
+ `"omitted"` (empty thinking text + a signature) for sub-agent turns always, and for the MAIN loop too
20
+ on models whose API default is `"omitted"` (Opus 4.8, Sonnet 5; Sonnet 4.6 defaulted to `"summarized"`).
21
+ Corpus: main-loop thinking text is 1810/1810 on Sonnet-4.6 but **0/2 on Opus-4.8 and 0/15 on Sonnet-5**,
22
+ every empty carrying a signature; sub-agents 0/230. The harness passes no `--thinking-display` —
23
+ faithfully to real Cowork, whose spawn passes none either. Both fields now mark such blocks
24
+ `{text:"", redacted:true}` (`kind:"thinking"` for the sub-agent turn shape) so they read as "reasoned,
25
+ text omitted by request"; TEXT turns are never redacted, and `redacted` is omitted (not `false`) on
26
+ blocks that carry text.
27
+ - **New `debug.thinking_display` escape hatch** (fenced, non-Cowork — like `debug.max_thinking_tokens`).
28
+ Set it to `"summarized"` to emit `--thinking-display summarized`, which flips **both** loops to
29
+ summarized thinking text (the API returns no raw chain-of-thought — `summarized` is the ceiling). It
30
+ diverges from real Cowork (which passes no such flag) and adds token cost, so it is a debug-only opt-in;
31
+ the default stays `"omitted"`, byte-identical argv. `docs/subagents.md`, `docs/session.md`,
32
+ `schema/run-result.json`, and the `RunResult`/`SessionConfig` types state the real capture semantics;
33
+ the released 0.31.0 note is corrected in place.
34
+
35
+ - **Docs pointed the GitHub Action at a ref that doesn't exist.** README, `SKILL.md`, and
36
+ `references/ci-recipe.md` all said `uses: yaniv-golan/cowork-harness@v1`, but no `v1` tag has ever
37
+ been published — a copy-pasted workflow failed with "unable to resolve action" before running
38
+ anything. All six references now bind `@main`; a moving major tag is a 1.0.0 question. A new
39
+ token-free guard (`test/action-docs-sync.test.ts`) locks the ref policy and also validates that
40
+ every documented `with:` key is a real `action.yml` input and every documented `command:` value is
41
+ one the Action describes.
42
+ - **`llms.txt` command list was three commands stale** — `lint-skill`, `analyze-skill`, and
43
+ `probe-dispatch` were missing. Now lists all 29, locked by a new COMMANDS ↔ llms.txt sync test
44
+ (set-equality, so stale names fail too), and the README "Commands at a glance" guard gained the
45
+ reverse direction (a README-only row now fails).
46
+ - **README observability prose still called a `subagents[]` field `model`** — renamed to
47
+ `dispatchModel` (beside `resolvedModel`) in 0.30.0; the prose now names the real fields. Also
48
+ fixed: the effort enum shorthand now lists `xhigh` (`extra` is the accepted alias) in both places,
49
+ the CI stage table matches `ci.yml`'s declaration order, the exit-code summary notes
50
+ `verify-cassettes`' distinct exit-`3` meaning and the reserved exit `4`, a duplicated `--keep`
51
+ example was collapsed, and `on_unanswered: llm` is explicitly marked as a scenario-YAML key (not a
52
+ CLI flag).
53
+ - **`status --latest-for` was undocumented in its own guide** — now covered in `docs/run-status.md`
54
+ (with the real output shape), the `docs/README.md` index row, and the top-level `--help` status
55
+ entry (with its own `0` found / `2` not found exit semantics).
56
+ - **`probe-dispatch --help` didn't start with a `usage:` line** like every other command; it does
57
+ now.
58
+ - **`debugging.md` showed only the scenario run-dir layout** — it now also names `chat`'s
59
+ `runs/chat/<sessionId>/` path. `docs/session.md`'s "all four tiers" now spells out the four
60
+ execution tiers and that `fidelity: cowork` resolves to one of them. `examples/README.md`'s
61
+ answer-policies pointer is now a real link. `effectiveFidelity` (the recorded resolved tier behind
62
+ the `resolved-tier`/`unverifiable-tier` staleness classes) is now mentioned in the README fidelity
63
+ section and `python/README.md`, not only in `docs/cassette.md`.
64
+ - **The docs told a false "default `8899`" story for `COWORK_VM_PROXY_PORT`** — when the var is
65
+ unset, the host binds the egress proxy on an OS-assigned free port and threads that value into the
66
+ guest firewall + `HTTP(S)_PROXY`; `8899` is only the guest-config fallback when a VM is spawned
67
+ without an explicit port, which never happens on a normal run. README and `docs/scenario.md` now
68
+ say so. Also closed: `COWORK_HARNESS_STATUS_CORRUPT_TIMEOUT_MS` (30s corrupt-`status.json`
69
+ backstop on `status --follow`) was the one `COWORK_*` env var documented nowhere — now in
70
+ `docs/run-status.md` + the README knob list; the `COWORK_HARNESS_DECIDER_DIR_POLL_MS` docs name
71
+ the per-subsystem defaults (300 ms rendezvous / 500 ms `gates --follow`); the `semantic_matches`
72
+ judge docs name the pinned default (`claude-opus-4-8`) in README, `docs/scenario.md`, and the
73
+ skill's schema reference; the heartbeat bullet states its 30s default instead of pointing at a
74
+ source file.
75
+ - **`replay --explain` was invisible from the command catalog** — the flagship false-green
76
+ diagnosis flag was documented only in debugging prose; it's now in the `record`/`replay` command
77
+ table row and leads the record/replay "Flags worth knowing" bullet.
78
+ - **`examples/README.md` ships in the npm tarball while the trees it describes don't** — it now
79
+ opens with a "Reading this on npm?" callout (clone for `scenarios/`/`sessions/`/`skills/`/`data/`),
80
+ documents `probes/` (previously invisible: used by `test/live-contract.test.ts` but absent from
81
+ the layout and from schema validation — `examples/probes` is now in `test/examples.test.ts`'s
82
+ scenario sweep), and links the matrices worked example; the README documentation-table row carries
83
+ the same source-checkout caveat.
84
+ - **The companion skill's preflight sent replay-only users through `doctor`**, whose token check
85
+ hard-fails on every tier — a new "Replay-only? Skip `doctor`" bullet carries the same carve-out
86
+ `docs/README.md` already had. The 3,200-character single-bullet 0.32.0 feature parenthetical is
87
+ now a scannable by-release sub-list, with every release bucket tag-verified against git history —
88
+ which caught and fixed a long-standing wrong tag: `semantic_matches` was labeled "new in 0.27.0"
89
+ but shipped in 0.28.0; the five path-gate keys, `agent_env`, and the `hostloop` split/
90
+ `allow_host_writes:` now sit under their real releases (0.30.0 / 0.21.0) too.
91
+ - **`DESIGN.md` still said the staged in-VM agent is 2.1.202** — the `desktop-1.20186.1` baseline
92
+ re-synced it to 2.1.205; DESIGN.md and `docs/protocol.md` now carry dated patch-only notes for
93
+ 1.20186.1 while the 2026-07-11 live pass stays scoped to `desktop-1.20186.0` (not restamped).
94
+
95
+ ### Added
96
+
97
+ - **`test/docs-index-sync.test.ts`** — three token-free guards: every `COWORK_*` env var read in
98
+ `src/` (dot-access **or** helper-read string literal) must be documented in README/`docs/*.md`;
99
+ the judge-model default id in the docs must match `semantic-judge.ts` (fails loud on a const
100
+ rename); `llms.txt` must link every top-level `docs/*.md` guide (both directions — `gotchas.md`,
101
+ `subagents.md`, and `plugin-root.md` were missing and are now linked).
102
+
103
+ - **The Action's `version` default (`latest`) is now documented as intentional** — in `action.yml`'s
104
+ input description, the README inputs sentence, and `references/ci-recipe.md` — with
105
+ pin-an-exact-version guidance for reproducible CI, and why it deliberately differs from the
106
+ companion skill's `@>=0.32.0` floor for ad-hoc CLI installs.
107
+ - **`--help` and unknown-flag structural-guard test coverage now spans all 29 commands** (previously
108
+ 16 and 17 respectively); `lint`/`lint-skill` are asserted against their `scenario.py` passthrough
109
+ usage lines and skip cleanly when `python3` is absent.
110
+
111
+ ### Documentation
112
+
113
+ - **`on_unanswered: prompt` was described as "only valid for `chat`" — wrong on two counts.** `chat`
114
+ never reads a scenario YAML (it runs an inline interactive scenario), and `prompt` is really a
115
+ `skill`-command policy (the adaptive-TTY default, or explicit `skill --on-unanswered prompt`);
116
+ `run` rejects it. The schema `.describe()` (and regenerated `scenario.schema.json`) now say so.
117
+ - **`ANTHROPIC_AUTH_TOKEN` is an accepted auth source but was under-documented.** It resolves
118
+ identically to `ANTHROPIC_API_KEY` (used only when no OAuth token is set) and was already in
119
+ `.env.example`, but the README auth text, the `record`/`doctor` `--help` blurbs, the `doctor`
120
+ no-token detail/remedy, the two `record` credential messages, `docs/cassette.md`, and the companion
121
+ skill's Auth note named only the other two — all now list it as the third alternative.
122
+ - **`llms.txt` exit-code line** now flags that `3`/`1` carry per-command meanings (e.g.
123
+ `verify-cassettes` `3` = could-not-verify, `sync` hard-fail = `1`) and links the authoritative
124
+ SPEC §11 text, instead of implying one global meaning for `3`.
125
+ - **README** gains a platform × tier support matrix (making explicit that Linux live runs are
126
+ `container`-only), doc-index rows for the spawn contract and `docs/decisions/`, the
127
+ `probe-dispatch` fidelity set, and a clearer global-install-ships-`examples/replays/`-only warning;
128
+ `docs/chat.md` notes that scaffolding a `chat` run yields an empty `assert:` block.
129
+
130
+ ## [0.32.0] — 2026-07-13
131
+
132
+ ### Added
133
+
134
+ - **`assertions --list` now documents that `subagent_dispatch_healthy`'s `type` matches the dispatch
135
+ description** (not just resolvedAgentType/dispatchAgentType), so a regex can narrow to one dispatch in a
136
+ same-agent-type fleet — previously only in `docs/scenario.md`.
137
+ - **`analyze-skill` line- and block-scoped ignore markers** — `analyze-skill: ignore-next-line` and
138
+ `analyze-skill: ignore-start` / `analyze-skill: ignore-end`, alongside the existing file-wide
139
+ `analyze-skill: ignore`. The file-wide marker silences EVERY finding in the file, which was too coarse
140
+ for a SKILL.md that carries just one `/sessions`-addressed teaching example (e.g. a "don't do this"
141
+ callout in `references/`/`agents/`): the whole file went blind to any OTHER, genuine finding. The two
142
+ new markers suppress only the exact line (`ignore-next-line`) or fenced range (`ignore-start`/
143
+ `ignore-end`, inclusive) they scope, reusing the same line-anchored marker matching as the file-wide
144
+ marker (bare, `#`-prefixed, list-bullet, HTML comment, or markdown reference-link comment — never
145
+ triggered by a marker merely documented mid-prose). An `ignore-start` with no matching `ignore-end`
146
+ before EOF emits its own gating `unclosed-ignore-fence` finding — it still suppresses to EOF (fail-open),
147
+ but the missing-`ignore-end` mistake itself prints and fails under `--strict` like any other finding,
148
+ never a silent notice on a green exit.
149
+
150
+ - **`analyze-skill <dir>` now scans the UNION of every contract-bearing markdown file in the directory,
151
+ not just `SKILL.md`.** A plugin's `/sessions` dispatch/output contracts often live in `agents/**` or
152
+ `references/**` — scanning only `SKILL.md` there was a false green (a consumer's `agents/sub/x.md`
153
+ could hand a `/sessions` path straight to a file tool and `analyze-skill` would never see it). A
154
+ directory target is now expanded to every shape it matches, deduped by resolved absolute path: a
155
+ top-level `SKILL.md` (+ its `references/**`); a plugin root (`.claude-plugin/plugin.json` or
156
+ `plugin.json`, + `agents/**`, `references/**`, `commands/**`, and each `skills/*/SKILL.md` + that
157
+ skill's own `references/**`); and a skill dir living inside a plugin also pulls in the enclosing
158
+ plugin's `agents/**`, `references/**`, and `commands/**`. Every walk is RECURSIVE (Claude Code discovers
159
+ namespaced commands/agents in subdirectories — `commands/tasks/build.md`, `agents/sub/x.md` — so a
160
+ single-level listing silently narrowed the scan) and FOLLOWS directory symlinks, loop-guarded against a
161
+ self-referencing symlink via a realpath visited-set, so a symlinked `references/`/`skills/<name>` is
162
+ scanned rather than silently dropped. Zero scannable files under a directory target is now a usage error
163
+ (exit 2) that enumerates the shapes it looked for — never a silent clean pass.
164
+
165
+ - **`analyze-skill` accepts MULTIPLE positionals + a simple `*` glob, matching `lint-skill`'s
166
+ `nargs="+"`.** A consumer with several explicit paths (or a directory's flat `*.md` set) no longer has
167
+ to loop `analyze-skill` once per path: `analyze-skill a/SKILL.md b/SKILL.md`, `analyze-skill skill-a/
168
+ skill-b/`, and `analyze-skill "plug/agents/*.md"` are all one call now. Each positional resolves
169
+ independently — a file or directory via the existing rules above, or a `*`-bearing target via a small
170
+ HAND-ROLLED glob matcher (no new dependency, no `engines.node` bump): `dir/*.md` matches shallowly,
171
+ `dir/**/*.md` matches recursively (reusing the same symlink-following, loop-guarded walker the
172
+ directory-target union scan already uses). Every positional's files are UNIONed and deduped by
173
+ resolved absolute path across the WHOLE invocation, extending the existing single-target dedup to
174
+ span all of them — a file reached
175
+ both directly and through a dir/glob positional is analyzed once. Zero scannable files across ALL
176
+ positionals remains the usage error (exit 2); a bad single positional (missing path, unrecognized glob
177
+ shape) fails the whole invocation.
178
+
179
+ ### Changed (breaking, pre-1.0)
180
+
181
+ - **`lint-skill`'s `subagent-type-not-found-in-plugin` finding is now WARN, not INFO** — so
182
+ `lint-skill --strict` newly gates on it. This is the one `subagent_type` outcome that's a *provable*
183
+ typo: a `<this-plugin>:<agent>` value whose prefix names the SKILL.md's own enclosing plugin, where
184
+ that plugin's `agents/*.md` was fully enumerated and doesn't contain the agent — the namespace prefix
185
+ already commits the value to this plugin, so a miss can never be another binary's built-in hiding
186
+ from static analysis. The other two `subagent_type` outcomes stay INFO, unchanged, because they
187
+ genuinely can't be disproven: `subagent-type-unresolvable` (a `<other-plugin>:<agent>` — belongs to
188
+ a plugin this linter didn't scan) and `subagent-type-unknown` (a bare value with no in-plugin match —
189
+ may be an agent-binary built-in). A `lint-skill --strict` run that previously passed clean may now
190
+ fail if it carries this exact typo shape; fix the agent name (or add the missing `agents/<agent>.md`)
191
+ to restore a clean `--strict` run.
192
+ - **`analyze-skill <dir> --strict` may newly fail where it passed clean before**, if `references/`/
193
+ `agents/`/`commands/` carries a `/sessions`-addressed teaching example that the old SKILL.md-only scan
194
+ never looked at — including one nested under a subdirectory or reached only via a symlink, both now
195
+ scanned. Annotate the example with `analyze-skill: ignore-next-line` (or an `ignore-start`/`ignore-end`
196
+ fence) to restore a clean `--strict` run — there is no `--skill-md-only` compatibility flag; the
197
+ narrower scan was the bug this closes.
198
+ - **`analyze-skill --output-format json`'s single-file shape changed.** The flat
199
+ `{file, findings, suppressed, strict}` payload is now always `{files: [{file, findings, suppressed}],
200
+ scanned, unscanned, strict}`, for uniformity with a directory target's multi-file result. An external
201
+ `jq '.findings'` / `jq '.file'` recipe needs `jq '.files[0].findings'` / `jq '.files[0].file'` instead.
202
+ - **`analyze-skill`'s JSON envelope `ok` now mirrors the exit code, not "zero findings."** Previously
203
+ `ok = findings.length === 0`, so an ADVISORY run (the default — findings print but exit 0) reported
204
+ `ok:false` even though the process exited 0 and `action.yml`'s own documented contract (`ok` "mirrors
205
+ the command's exit code") said otherwise; the packaged Action's job-summary reporter then rendered
206
+ "Overall: ❌ fail" for a run that hadn't actually failed. `ok` is now `true` on an advisory exit-0 run
207
+ even with findings present, and `false` only under `--strict` with findings or the exit-2 usage error
208
+ — the same rule `lint`/`lint-skill --output-format json` follow (see the Fixed entry below). A `jq
209
+ '.ok'` consumer that was treating `ok:false` as "any findings at all" should switch to checking
210
+ `.files[].findings` directly; the exit code (and `ok`) now only reflect `--strict` gating.
211
+
212
+ ### Fixed
213
+
214
+ - **The packaged Action's `lint`/`lint-skill` lanes were broken.** `action.yml` always appends
215
+ `--output-format json` to every command it drives, but the bundled `scenario.py`'s `lint`/`lint-skill`
216
+ only understand `--json` — `--output-format` was an unrecognized argument to python's argparse (exit 2,
217
+ empty stdout), so `command: lint` / `command: lint-skill` in the Action always failed with an
218
+ unhelpful `ok:false` and no findings. `cowork-harness lint`/`lint-skill` now detect json mode
219
+ themselves, strip `--output-format json|text|=…` before forwarding to python, pass python's own
220
+ `--json` in json mode, and re-wrap the child's bare findings array in the harness's standard
221
+ `jsonPayloadEnvelope("lint"/"lint-skill", ok, { findings })` (`ok` mirrors the child's exit code). Text
222
+ mode (the default) is unaffected in output — `stdio: "inherit"` as before — but now also strips the
223
+ flag first, since python didn't know the text form either. A python-level usage error in json mode
224
+ (e.g. a missing required path — argparse exit 2, empty stdout) now surfaces as a `jsonError("usage",
225
+ …)` envelope instead of a raw `JSON.parse` crash. `action.yml` also now documents `lint-skill` and
226
+ `analyze-skill` in its `command` input, and no longer forwards the replay-only `--fail-on-skill-drift`
227
+ flag to any command other than `replay`.
228
+
9
229
  ## [0.31.0] — 2026-07-12
10
230
 
11
231
  ### Added
@@ -26,6 +246,11 @@ All notable changes to this project are documented here. The format is based on
26
246
  `thinking[]` field is (~50 entries, ~10KB/entry each), with `reasoningElided` counting the overflow. A
27
247
  missing or malformed child transcript never fails the run — the affected dispatch's `reasoning` is just
28
248
  left absent.
249
+ - **Correction (see Unreleased):** in practice only the TEXT turns carry content — the harness's
250
+ non-interactive spawn forces the API's `thinking.display` to `"omitted"` for sub-agent turns, so
251
+ `thinking` turns come through empty (signature-only). The Unreleased `redacted` marker distinguishes
252
+ "reasoned, text omitted by request" from "no thought"; the "THINKING … turns" wording above
253
+ overstated what this captures by default.
29
254
 
30
255
  - **`status --latest-for <scenario-name-or-slug>`** — resolves and prints the NEWEST run dir for a
31
256
  scenario by actual run time, replacing the fragile `ls -td runs/<scenario>/* | head -1` idiom: bare