cowork-harness 0.31.0 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/cowork-harness/SKILL.md +5 -5
- package/.claude/skills/cowork-harness/references/ci-recipe.md +4 -4
- package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +1 -1
- package/.claude/skills/cowork-harness/references/scenario-schema.md +1 -1
- package/.claude/skills/cowork-harness/references/task-recipes.md +1 -1
- package/.claude/skills/cowork-harness/scripts/scenario.py +32 -19
- package/CHANGELOG.md +99 -0
- package/README.md +9 -9
- package/dist/cli.js +14 -5
- package/dist/run/analyze-skill.js +520 -46
- package/dist/run/scenario-tool.js +90 -25
- package/dist/types.js +1 -1
- package/docs/plugin-root.md +4 -0
- package/docs/subagents.md +125 -16
- package/examples/replays/README.md +1 -1
- package/package.json +1 -1
- package/schema/scenario.schema.json +1 -1
|
@@ -3,8 +3,8 @@ name: cowork-harness
|
|
|
3
3
|
description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
|
|
4
4
|
metadata:
|
|
5
5
|
author: cowork-harness
|
|
6
|
-
version: 0.
|
|
7
|
-
tracks-harness: cowork-harness 0.
|
|
6
|
+
version: 0.32.0
|
|
7
|
+
tracks-harness: cowork-harness 0.32.0 (baseline desktop-1.20186.1)
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# cowork-harness
|
|
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
|
|
|
22
22
|
allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
|
|
23
23
|
the highest-value part. Read it.
|
|
24
24
|
|
|
25
|
-
> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.
|
|
25
|
+
> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.32.0` (baseline
|
|
26
26
|
> `desktop-1.20186.1`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
|
|
27
27
|
> `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
|
|
28
28
|
|
|
@@ -38,7 +38,7 @@ cowork-harness skill ./my-skill "do X" # run the skill once against the sta
|
|
|
38
38
|
Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
|
|
39
39
|
|
|
40
40
|
- **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
|
|
41
|
-
- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.
|
|
41
|
+
- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.32.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.32.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.32.0"`. **Pin `@>=0.32.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.32.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator (passes when the resolved value deep-equals one of the listed members — value ∈ your list, not the reverse), `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-patterns-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0; `--allow-patterns-file <path>` is a FILE of patterns, one regex per line — not a path to allow, unlike `--allow <regex>`), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, `computer_links_resolve` (new in 0.22.0), `semantic_matches` (an LLM judge grades a fixed `rubric` of claims against the run's answer — **live-only**, so it is evidence-unavailable / skipped-loud on replay, never a vacuous pass; new in 0.27.0), **glob-matched** `tool_called`/`tool_not_called`/`subagent_tool_used`/`subagent_tool_absent` (a pattern like `mcp__workspace__*` matches any tool in the family; exact names match exactly; new in 0.28.0), the five path-gate assertion keys `no_vm_path_file_op`/`vm_path_denied`/`path_denied`/`no_path_denied`/`subagent_file_write`, the session-level `agent_env` knob, and resolved sub-agent identity on dispatch records (`resolvedAgentType`/`resolvedModel`; new in 0.30.0), the `lint-skill` (static host-loop footgun + `subagent_type` resolution linter) / `analyze-skill` (advisory `/sessions`-path static scan, `--strict`, `analyze-skill: ignore` marker) / `probe-dispatch` (single-dispatch mechanics probe) commands, `status --latest-for` (resolve a scenario's newest run dir by run time, not directory mtime), the `subagent_dispatch_healthy` composite assertion, and the persisted `result.json` fields `verdict`, `subagents[].referencesRead`/`subagents[].reasoning`, plus the `toolCounts`/`toolErrors`/`toolDurations` shape distinction (all new in 0.31.0), `analyze-skill`'s directory scan now covering a skill/plugin's full contract surface (recursive `agents/`/`references/`/`commands/`, plugin-root-aware, symlink-following) with line/block-scoped `analyze-skill: ignore-next-line`/`ignore-start`/`ignore-end` markers and multi-path/glob input, and `lint-skill`'s provable in-plugin `subagent_type` typo now a WARN that gates under `--strict` (new in 0.32.0).)
|
|
42
42
|
- **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
|
|
43
43
|
- **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
|
|
44
44
|
- **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
|
|
@@ -63,7 +63,7 @@ reproducible regression (Part II), and **debug** a run that misbehaved or greene
|
|
|
63
63
|
the path, `trace <run-id>` finds it). **Localize the failure post-hoc** from that evidence:
|
|
64
64
|
`cowork-harness trace <run-dir>`'s views + the emitted `result.json` to see what the run actually did,
|
|
65
65
|
then `verify-run` to re-check a suspect assertion — all token-free, no Docker, no re-record. This is
|
|
66
|
-
the loop 0.
|
|
66
|
+
the loop 0.32.0's observability is built for; the *Triage* and *Inspecting a run's observability
|
|
67
67
|
output* sections in **Part III — Debug** are the detail (the fuller human-facing map lives in
|
|
68
68
|
`docs/debugging.md` — repo-only, not shipped with the installed skill).
|
|
69
69
|
- **Multi-turn / interactive reproduction** → `cowork-harness chat` (interactive; gates answered at the
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# CI recipe — replay vs live lanes
|
|
2
2
|
|
|
3
|
-
Self-contained reference. Tracks `cowork-harness 0.
|
|
3
|
+
Self-contained reference. Tracks `cowork-harness 0.32.0` (baseline `desktop-1.20186.1`).
|
|
4
4
|
|
|
5
5
|
**Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
|
|
6
6
|
job-summary reporter (verdict table, staleness findings, cost/turns when available):
|
|
@@ -54,7 +54,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
|
|
|
54
54
|
GitHub-hosted runners, no token/Docker/agent:
|
|
55
55
|
|
|
56
56
|
```yaml
|
|
57
|
-
- run: npm i -g "cowork-harness@>=0.
|
|
57
|
+
- run: npm i -g "cowork-harness@>=0.32.0"
|
|
58
58
|
- run: cowork-harness lint scenarios/*.yaml # no silent false-greens
|
|
59
59
|
- run: cowork-harness verify-cassettes cassettes/ # privacy + staleness
|
|
60
60
|
- run: cowork-harness replay cassettes/ # token-free content/structure
|
|
@@ -194,7 +194,7 @@ jobs:
|
|
|
194
194
|
with: { node-version: '20' }
|
|
195
195
|
- uses: actions/setup-python@v5
|
|
196
196
|
with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
|
|
197
|
-
- run: npm i -g "cowork-harness@>=0.
|
|
197
|
+
- run: npm i -g "cowork-harness@>=0.32.0"
|
|
198
198
|
- run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
|
|
199
199
|
- run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
|
|
200
200
|
- run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
|
|
@@ -223,7 +223,7 @@ jobs:
|
|
|
223
223
|
echo "live=true" >> "$GITHUB_OUTPUT"
|
|
224
224
|
fi
|
|
225
225
|
- if: steps.guard.outputs.live == 'true'
|
|
226
|
-
run: npm i -g "cowork-harness@>=0.
|
|
226
|
+
run: npm i -g "cowork-harness@>=0.32.0"
|
|
227
227
|
- if: steps.guard.outputs.live == 'true'
|
|
228
228
|
run: cowork-harness run scenarios/ --output-format json
|
|
229
229
|
env:
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Fidelity tiers & answer paths
|
|
2
2
|
|
|
3
|
-
Self-contained reference. Tracks `cowork-harness 0.
|
|
3
|
+
Self-contained reference. Tracks `cowork-harness 0.32.0` (baseline `desktop-1.20186.1`).
|
|
4
4
|
|
|
5
5
|
## Fidelity tiers (`fidelity:` in the scenario)
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Scenario & session schema, assertion catalog, web_fetch, full gotchas
|
|
2
2
|
|
|
3
|
-
Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.
|
|
3
|
+
Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.32.0`
|
|
4
4
|
(baseline `desktop-1.20186.1`). If your checkout is newer, prefer the live `docs/scenario.md`,
|
|
5
5
|
`docs/session.md`, and `SPEC.md`.
|
|
6
6
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Each recipe composes facts that live scattered across SKILL.md and the other references into one
|
|
4
4
|
decision path. Every one answers a question a real fleet owner had to work out the hard way. Facts track the harness version in SKILL.md's
|
|
5
|
-
front-matter (currently 0.
|
|
5
|
+
front-matter (currently 0.32.0). Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
|
|
6
6
|
Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
|
|
7
7
|
needed if your CLI meets SKILL.md's version floor.
|
|
8
8
|
|
|
@@ -1047,9 +1047,10 @@ def _finding_subagent_not_found_in_plugin(path, line, value, plugin_name, agent,
|
|
|
1047
1047
|
own enclosing plugin) but whose agent isn't in its enumerated agents/*.md set. Unlike
|
|
1048
1048
|
`subagent-type-unknown`, this can never be another binary's built-in — the namespace prefix
|
|
1049
1049
|
already commits it to this plugin, and the plugin's agent set was fully enumerable — so it's a
|
|
1050
|
-
provable typo, not an unconfirmable unknown.
|
|
1050
|
+
provable typo, not an unconfirmable unknown. Provable means it can be a hard gate: WARN (not
|
|
1051
|
+
INFO), so `lint-skill --strict` fails on it."""
|
|
1051
1052
|
return Finding(
|
|
1052
|
-
"
|
|
1053
|
+
"WARN",
|
|
1053
1054
|
"subagent-type-not-found-in-plugin",
|
|
1054
1055
|
f"pinned type `{value}` names this plugin (`{plugin_name}`) but `{agent}` is not among its "
|
|
1055
1056
|
f"agents [{', '.join(sorted_agents)}] — likely a typo.",
|
|
@@ -1063,20 +1064,23 @@ def _finding_subagent_not_found_in_plugin(path, line, value, plugin_name, agent,
|
|
|
1063
1064
|
|
|
1064
1065
|
def _classify_subagent_type(value, plugin_name, plugin_agent_types):
|
|
1065
1066
|
"""subagent_type severity ladder. Returns a Finding-builder (path, line, value) -> Finding, or None if
|
|
1066
|
-
clean.
|
|
1067
|
+
clean. Only ONE outcome is a hard gate: `subagent-type-not-found-in-plugin` is WARN (a provable
|
|
1068
|
+
typo, so `lint-skill --strict` fails on it); the two unconfirmable outcomes
|
|
1069
|
+
(`subagent-type-unresolvable`, `subagent-type-unknown`) stay INFO — see the HONEST LIMIT note
|
|
1070
|
+
above this section for why those two can never be proven.
|
|
1067
1071
|
|
|
1068
1072
|
Precedence:
|
|
1069
1073
|
1. Resolves in-plugin, or is `general-purpose` -> clean (None).
|
|
1070
1074
|
2. Has a `<prefix>:<agent>` shape whose prefix EQUALS the resolved plugin's own name AND the
|
|
1071
|
-
plugin's agent set was non-empty (i.e. enumerable) -> `subagent-type-not-found-in-plugin
|
|
1072
|
-
A value namespaced under this plugin's own
|
|
1073
|
-
once the plugin is fully enumerated a miss
|
|
1074
|
-
unknown.
|
|
1075
|
+
plugin's agent set was non-empty (i.e. enumerable) -> `subagent-type-not-found-in-plugin`
|
|
1076
|
+
(WARN — provable typo, gates under `--strict`). A value namespaced under this plugin's own
|
|
1077
|
+
name can never be another binary's built-in, so once the plugin is fully enumerated a miss
|
|
1078
|
+
here is a provable typo, not an unconfirmable unknown.
|
|
1075
1079
|
3. Has a `<prefix>:<agent>` shape whose prefix does NOT equal the resolved plugin's name (or
|
|
1076
|
-
there's no resolved plugin at all) -> `subagent-type-unresolvable` (belongs to another
|
|
1080
|
+
there's no resolved plugin at all) -> `subagent-type-unresolvable` (INFO — belongs to another
|
|
1077
1081
|
plugin, or an empty plugin_name means we truly can't tell whose namespace it is).
|
|
1078
1082
|
4. Otherwise (no colon, or same-plugin prefix but the plugin set was empty/unenumerable) ->
|
|
1079
|
-
`subagent-type-unknown` — genuinely can't confirm statically.
|
|
1083
|
+
`subagent-type-unknown` (INFO) — genuinely can't confirm statically.
|
|
1080
1084
|
"""
|
|
1081
1085
|
if value == "general-purpose" or value in plugin_agent_types:
|
|
1082
1086
|
return None
|
|
@@ -1170,10 +1174,10 @@ def cmd_lint_skill(args):
|
|
|
1170
1174
|
_print_findings(all_findings, n_files, kind="skill file", clean_suffix=" — no Cowork host-loop footguns.")
|
|
1171
1175
|
has_error = any(x.severity == "ERROR" for x in all_findings)
|
|
1172
1176
|
# --strict fails on WARN too, per its own --help text ("exit non-zero on WARN too, not just ERROR")
|
|
1173
|
-
# — but NEVER on INFO.
|
|
1174
|
-
#
|
|
1175
|
-
#
|
|
1176
|
-
# must always be surfaced, never failed, even under --strict.
|
|
1177
|
+
# — but NEVER on INFO. Of the subagent_type ladder, only `subagent-type-not-found-in-plugin` is
|
|
1178
|
+
# WARN (a provable in-plugin typo — see the subparser help above); `subagent-type-unresolvable` and
|
|
1179
|
+
# `subagent-type-unknown` stay INFO by design (there is no harness registry to disprove an unknown
|
|
1180
|
+
# value against), so those two must always be surfaced, never failed, even under --strict.
|
|
1177
1181
|
has_warn = any(x.severity == "WARN" for x in all_findings)
|
|
1178
1182
|
if has_error or (args.strict and has_warn):
|
|
1179
1183
|
return 1
|
|
@@ -1364,17 +1368,26 @@ def main(argv=None):
|
|
|
1364
1368
|
"Also statically resolves any pinned `subagent_type` value in the SKILL.md against the "
|
|
1365
1369
|
"enclosing plugin's `agents/*.md` (see `resolve-agent-types`): a value that resolves in-plugin "
|
|
1366
1370
|
"or is `general-purpose` is clean; a `<this-plugin>:<agent>` whose agent is missing from an "
|
|
1367
|
-
"enumerable plugin is a provable typo, reported as `subagent-type-not-found-in-plugin
|
|
1368
|
-
"
|
|
1369
|
-
"
|
|
1370
|
-
"`subagent-type-
|
|
1371
|
-
"
|
|
1371
|
+
"enumerable plugin is a provable typo, reported as `subagent-type-not-found-in-plugin` "
|
|
1372
|
+
"(WARN — gates under `--strict`, since the namespace prefix already commits it to this "
|
|
1373
|
+
"plugin and its agent set was fully enumerated); a `<other-plugin>:<agent>` is reported as "
|
|
1374
|
+
"`subagent-type-unresolvable` (INFO); any other unresolved value (including a same-plugin "
|
|
1375
|
+
"prefix the linter couldn't enumerate) is `subagent-type-unknown` (INFO) — those two stay "
|
|
1376
|
+
"INFO, never WARN, since there is no harness registry of built-in agent types to disprove "
|
|
1377
|
+
"an unknown value against.\n\n"
|
|
1378
|
+
"Plain `lint-skill` (no `--strict`) is ADVISORY — it prints findings but exits 0 on "
|
|
1379
|
+
"WARN/INFO. CI should invoke `lint-skill --strict` to actually gate on the WARN-class "
|
|
1380
|
+
"findings above (the two host-loop footguns and the provable subagent_type typo)."
|
|
1372
1381
|
),
|
|
1373
1382
|
formatter_class=argparse.RawDescriptionHelpFormatter,
|
|
1374
1383
|
)
|
|
1375
1384
|
lsp.add_argument("paths", nargs="+", help="SKILL.md file(s) or skill director(ies) to inspect")
|
|
1376
1385
|
lsp.add_argument("--json", action="store_true", help="emit findings as JSON")
|
|
1377
|
-
lsp.add_argument(
|
|
1386
|
+
lsp.add_argument(
|
|
1387
|
+
"--strict",
|
|
1388
|
+
action="store_true",
|
|
1389
|
+
help="exit non-zero on WARN too, not just ERROR (CI-recommended invocation; plain lint-skill is advisory-only)",
|
|
1390
|
+
)
|
|
1378
1391
|
lsp.set_defaults(func=cmd_lint_skill)
|
|
1379
1392
|
|
|
1380
1393
|
rap = sub.add_parser(
|
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,105 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.32.0] — 2026-07-13
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **`assertions --list` now documents that `subagent_dispatch_healthy`'s `type` matches the dispatch
|
|
14
|
+
description** (not just resolvedAgentType/dispatchAgentType), so a regex can narrow to one dispatch in a
|
|
15
|
+
same-agent-type fleet — previously only in `docs/scenario.md`.
|
|
16
|
+
- **`analyze-skill` line- and block-scoped ignore markers** — `analyze-skill: ignore-next-line` and
|
|
17
|
+
`analyze-skill: ignore-start` / `analyze-skill: ignore-end`, alongside the existing file-wide
|
|
18
|
+
`analyze-skill: ignore`. The file-wide marker silences EVERY finding in the file, which was too coarse
|
|
19
|
+
for a SKILL.md that carries just one `/sessions`-addressed teaching example (e.g. a "don't do this"
|
|
20
|
+
callout in `references/`/`agents/`): the whole file went blind to any OTHER, genuine finding. The two
|
|
21
|
+
new markers suppress only the exact line (`ignore-next-line`) or fenced range (`ignore-start`/
|
|
22
|
+
`ignore-end`, inclusive) they scope, reusing the same line-anchored marker matching as the file-wide
|
|
23
|
+
marker (bare, `#`-prefixed, list-bullet, HTML comment, or markdown reference-link comment — never
|
|
24
|
+
triggered by a marker merely documented mid-prose). An `ignore-start` with no matching `ignore-end`
|
|
25
|
+
before EOF emits its own gating `unclosed-ignore-fence` finding — it still suppresses to EOF (fail-open),
|
|
26
|
+
but the missing-`ignore-end` mistake itself prints and fails under `--strict` like any other finding,
|
|
27
|
+
never a silent notice on a green exit.
|
|
28
|
+
|
|
29
|
+
- **`analyze-skill <dir>` now scans the UNION of every contract-bearing markdown file in the directory,
|
|
30
|
+
not just `SKILL.md`.** A plugin's `/sessions` dispatch/output contracts often live in `agents/**` or
|
|
31
|
+
`references/**` — scanning only `SKILL.md` there was a false green (a consumer's `agents/sub/x.md`
|
|
32
|
+
could hand a `/sessions` path straight to a file tool and `analyze-skill` would never see it). A
|
|
33
|
+
directory target is now expanded to every shape it matches, deduped by resolved absolute path: a
|
|
34
|
+
top-level `SKILL.md` (+ its `references/**`); a plugin root (`.claude-plugin/plugin.json` or
|
|
35
|
+
`plugin.json`, + `agents/**`, `references/**`, `commands/**`, and each `skills/*/SKILL.md` + that
|
|
36
|
+
skill's own `references/**`); and a skill dir living inside a plugin also pulls in the enclosing
|
|
37
|
+
plugin's `agents/**`, `references/**`, and `commands/**`. Every walk is RECURSIVE (Claude Code discovers
|
|
38
|
+
namespaced commands/agents in subdirectories — `commands/tasks/build.md`, `agents/sub/x.md` — so a
|
|
39
|
+
single-level listing silently narrowed the scan) and FOLLOWS directory symlinks, loop-guarded against a
|
|
40
|
+
self-referencing symlink via a realpath visited-set, so a symlinked `references/`/`skills/<name>` is
|
|
41
|
+
scanned rather than silently dropped. Zero scannable files under a directory target is now a usage error
|
|
42
|
+
(exit 2) that enumerates the shapes it looked for — never a silent clean pass.
|
|
43
|
+
|
|
44
|
+
- **`analyze-skill` accepts MULTIPLE positionals + a simple `*` glob, matching `lint-skill`'s
|
|
45
|
+
`nargs="+"`.** A consumer with several explicit paths (or a directory's flat `*.md` set) no longer has
|
|
46
|
+
to loop `analyze-skill` once per path: `analyze-skill a/SKILL.md b/SKILL.md`, `analyze-skill skill-a/
|
|
47
|
+
skill-b/`, and `analyze-skill "plug/agents/*.md"` are all one call now. Each positional resolves
|
|
48
|
+
independently — a file or directory via the existing rules above, or a `*`-bearing target via a small
|
|
49
|
+
HAND-ROLLED glob matcher (no new dependency, no `engines.node` bump): `dir/*.md` matches shallowly,
|
|
50
|
+
`dir/**/*.md` matches recursively (reusing the same symlink-following, loop-guarded walker the
|
|
51
|
+
directory-target union scan already uses). Every positional's files are UNIONed and deduped by
|
|
52
|
+
resolved absolute path across the WHOLE invocation, extending the existing single-target dedup to
|
|
53
|
+
span all of them — a file reached
|
|
54
|
+
both directly and through a dir/glob positional is analyzed once. Zero scannable files across ALL
|
|
55
|
+
positionals remains the usage error (exit 2); a bad single positional (missing path, unrecognized glob
|
|
56
|
+
shape) fails the whole invocation.
|
|
57
|
+
|
|
58
|
+
### Changed (breaking, pre-1.0)
|
|
59
|
+
|
|
60
|
+
- **`lint-skill`'s `subagent-type-not-found-in-plugin` finding is now WARN, not INFO** — so
|
|
61
|
+
`lint-skill --strict` newly gates on it. This is the one `subagent_type` outcome that's a *provable*
|
|
62
|
+
typo: a `<this-plugin>:<agent>` value whose prefix names the SKILL.md's own enclosing plugin, where
|
|
63
|
+
that plugin's `agents/*.md` was fully enumerated and doesn't contain the agent — the namespace prefix
|
|
64
|
+
already commits the value to this plugin, so a miss can never be another binary's built-in hiding
|
|
65
|
+
from static analysis. The other two `subagent_type` outcomes stay INFO, unchanged, because they
|
|
66
|
+
genuinely can't be disproven: `subagent-type-unresolvable` (a `<other-plugin>:<agent>` — belongs to
|
|
67
|
+
a plugin this linter didn't scan) and `subagent-type-unknown` (a bare value with no in-plugin match —
|
|
68
|
+
may be an agent-binary built-in). A `lint-skill --strict` run that previously passed clean may now
|
|
69
|
+
fail if it carries this exact typo shape; fix the agent name (or add the missing `agents/<agent>.md`)
|
|
70
|
+
to restore a clean `--strict` run.
|
|
71
|
+
- **`analyze-skill <dir> --strict` may newly fail where it passed clean before**, if `references/`/
|
|
72
|
+
`agents/`/`commands/` carries a `/sessions`-addressed teaching example that the old SKILL.md-only scan
|
|
73
|
+
never looked at — including one nested under a subdirectory or reached only via a symlink, both now
|
|
74
|
+
scanned. Annotate the example with `analyze-skill: ignore-next-line` (or an `ignore-start`/`ignore-end`
|
|
75
|
+
fence) to restore a clean `--strict` run — there is no `--skill-md-only` compatibility flag; the
|
|
76
|
+
narrower scan was the bug this closes.
|
|
77
|
+
- **`analyze-skill --output-format json`'s single-file shape changed.** The flat
|
|
78
|
+
`{file, findings, suppressed, strict}` payload is now always `{files: [{file, findings, suppressed}],
|
|
79
|
+
scanned, unscanned, strict}`, for uniformity with a directory target's multi-file result. An external
|
|
80
|
+
`jq '.findings'` / `jq '.file'` recipe needs `jq '.files[0].findings'` / `jq '.files[0].file'` instead.
|
|
81
|
+
- **`analyze-skill`'s JSON envelope `ok` now mirrors the exit code, not "zero findings."** Previously
|
|
82
|
+
`ok = findings.length === 0`, so an ADVISORY run (the default — findings print but exit 0) reported
|
|
83
|
+
`ok:false` even though the process exited 0 and `action.yml`'s own documented contract (`ok` "mirrors
|
|
84
|
+
the command's exit code") said otherwise; the packaged Action's job-summary reporter then rendered
|
|
85
|
+
"Overall: ❌ fail" for a run that hadn't actually failed. `ok` is now `true` on an advisory exit-0 run
|
|
86
|
+
even with findings present, and `false` only under `--strict` with findings or the exit-2 usage error
|
|
87
|
+
— the same rule `lint`/`lint-skill --output-format json` follow (see the Fixed entry below). A `jq
|
|
88
|
+
'.ok'` consumer that was treating `ok:false` as "any findings at all" should switch to checking
|
|
89
|
+
`.files[].findings` directly; the exit code (and `ok`) now only reflect `--strict` gating.
|
|
90
|
+
|
|
91
|
+
### Fixed
|
|
92
|
+
|
|
93
|
+
- **The packaged Action's `lint`/`lint-skill` lanes were broken.** `action.yml` always appends
|
|
94
|
+
`--output-format json` to every command it drives, but the bundled `scenario.py`'s `lint`/`lint-skill`
|
|
95
|
+
only understand `--json` — `--output-format` was an unrecognized argument to python's argparse (exit 2,
|
|
96
|
+
empty stdout), so `command: lint` / `command: lint-skill` in the Action always failed with an
|
|
97
|
+
unhelpful `ok:false` and no findings. `cowork-harness lint`/`lint-skill` now detect json mode
|
|
98
|
+
themselves, strip `--output-format json|text|=…` before forwarding to python, pass python's own
|
|
99
|
+
`--json` in json mode, and re-wrap the child's bare findings array in the harness's standard
|
|
100
|
+
`jsonPayloadEnvelope("lint"/"lint-skill", ok, { findings })` (`ok` mirrors the child's exit code). Text
|
|
101
|
+
mode (the default) is unaffected in output — `stdio: "inherit"` as before — but now also strips the
|
|
102
|
+
flag first, since python didn't know the text form either. A python-level usage error in json mode
|
|
103
|
+
(e.g. a missing required path — argparse exit 2, empty stdout) now surfaces as a `jsonError("usage",
|
|
104
|
+
…)` envelope instead of a raw `JSON.parse` crash. `action.yml` also now documents `lint-skill` and
|
|
105
|
+
`analyze-skill` in its `command` input, and no longer forwards the replay-only `--fail-on-skill-drift`
|
|
106
|
+
flag to any command other than `replay`.
|
|
107
|
+
|
|
9
108
|
## [0.31.0] — 2026-07-12
|
|
10
109
|
|
|
11
110
|
### Added
|
package/README.md
CHANGED
|
@@ -91,7 +91,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
|
|
|
91
91
|
|
|
92
92
|
> **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
|
|
93
93
|
> <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
|
|
94
|
-
> From a global install (`npm i -g "cowork-harness@>=0.
|
|
94
|
+
> From a global install (`npm i -g "cowork-harness@>=0.32.0"`), point at the package root instead:
|
|
95
95
|
> `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
|
|
96
96
|
> (or copy the cassette into your own project and pass that path).
|
|
97
97
|
|
|
@@ -101,7 +101,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
|
|
|
101
101
|
> - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
|
|
102
102
|
> - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
|
|
103
103
|
> - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
|
|
104
|
-
> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=0.
|
|
104
|
+
> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@>=0.32.0"`.
|
|
105
105
|
|
|
106
106
|
Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
|
|
107
107
|
test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
|
|
@@ -126,7 +126,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
|
|
|
126
126
|
claude plugin install cowork-harness@cowork-harness
|
|
127
127
|
```
|
|
128
128
|
|
|
129
|
-
The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=0.
|
|
129
|
+
The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@>=0.32.0"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
|
|
130
130
|
|
|
131
131
|
It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
|
|
132
132
|
|
|
@@ -147,7 +147,7 @@ A global install is enough for CI `lint`, reading the teaching skill, and replay
|
|
|
147
147
|
To `run` the worked examples live or copy them as a starting point, use a source checkout. (The marketplace
|
|
148
148
|
skill install itself only pulls `.claude/skills/cowork-harness/` — SKILL.md + `references/` + `scenario.py`/
|
|
149
149
|
assertion keys, per `.claude-plugin/marketplace.json`'s `source` — not the rest of this table; the full set
|
|
150
|
-
above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=0.
|
|
150
|
+
above becomes available once the skill's first command self-bootstraps `npx "cowork-harness@>=0.32.0"` — see
|
|
151
151
|
[above](#drive-it-from-claude-code-companion-skill) — which pulls the same npm package as the global-install row.)
|
|
152
152
|
|
|
153
153
|
### Prerequisites for anything above `protocol` fidelity
|
|
@@ -380,9 +380,9 @@ Skill testing is the headline use, but the tool is a general harness over the Co
|
|
|
380
380
|
| `scaffold <run-id>` | Turn a kept run into a starter scenario YAML (gates→answers, artifacts→`file_exists`) | authoring a scenario from a real run instead of guessing |
|
|
381
381
|
| `python3 …/scenario.py scaffold --name <name> --skill <dir>` | Generate a starter scenario skeleton from scratch (the `…` is `.claude/skills/cowork-harness/scripts/`) | starting a new scenario when you have no prior run |
|
|
382
382
|
| `lint <scenario.yaml \| dir/>…` | Check scenarios for silent false-greens — assertions placed on the wrong CI lane, mixed content/live keys, missing `controlOut`-required keys (files or a directory of `*.yaml`/`*.yml`; bundled `scenario.py`; needs python3 — PyYAML is bundled); `--json` emits findings as machine-readable JSON instead of the text report | before committing a new scenario or after changing assertions |
|
|
383
|
-
| `lint-skill <SKILL.md \| skill-dir/>…` | Lint a skill body (and any sibling `hooks.json`) for two Cowork host-loop footguns — a `${CLAUDE_PLUGIN_ROOT}` path used in an in-VM bash context, and a hook command that exports an env var or writes into `/tmp` for the in-VM agent — plus static resolution of any pinned `subagent_type` against the enclosing plugin's `agents/*.md` (bundled `scenario.py`; needs python3); the two footguns are WARN-only,
|
|
383
|
+
| `lint-skill <SKILL.md \| skill-dir/>…` | Lint a skill body (and any sibling `hooks.json`) for two Cowork host-loop footguns — a `${CLAUDE_PLUGIN_ROOT}` path used in an in-VM bash context, and a hook command that exports an env var or writes into `/tmp` for the in-VM agent — plus static resolution of any pinned `subagent_type` against the enclosing plugin's `agents/*.md` (bundled `scenario.py`; needs python3); the two footguns are WARN-only, and of the three `subagent_type` outcomes, an in-plugin-prefixed agent missing from the enclosing plugin's fully-enumerated `agents/*.md` (`subagent-type-not-found-in-plugin`) is a **provable typo and is WARN too**, while a cross-plugin (`subagent-type-unresolvable`) or unknown-bare (`subagent-type-unknown`) value stays INFO (no built-in agent-type registry to disprove it against) — pass `--strict` (the CI-recommended invocation; plain `lint-skill` is advisory-only) to fail on any WARN, or `--json` for machine-readable findings | authoring or reviewing a skill before a paid Cowork host-loop run exposes the footgun, or before a pinned `subagent_type` typo breaks a `Task` dispatch |
|
|
384
384
|
| `python3 …/scenario.py resolve-agent-types <plugin-dir> [--json]` | Token-free: print a plugin's valid `<plugin>:<agent>` subagent types, resolved from `.claude-plugin/plugin.json` + `agents/*.md` frontmatter (filename-stem fallback when an agent file has no `name:`); the `…` is `.claude/skills/cowork-harness/scripts/` | "does `founder-skills:deck-review` resolve within this plugin?" without a live dispatch |
|
|
385
|
-
| `analyze-skill <SKILL.md \| skill-dir
|
|
385
|
+
| `analyze-skill <SKILL.md \| skill-dir/ \| glob>…` | Token-free ADVISORY scan: flags a `/sessions/...` path handed to a file tool or used as a dispatch/sub-agent output path — that path class is DENIED on host-loop (reuses the harness's own ported `/sessions` path-gate predicate as the deny decision; only the text extraction is heuristic, and it's conservative — bash fences, negation idioms ("never write to..."), and plain prose are exempt). A DIRECTORY target scans the UNION of every contract-bearing markdown file present, walked RECURSIVELY (agents/commands/references are `**` walks, not a single-level listing) and FOLLOWING directory symlinks (loop-guarded against a self-referencing symlink) — top-level `SKILL.md` + `references/**`; a plugin root's `agents/**` + `references/**` + `commands/**` + `skills/*/SKILL.md`(+`references/**`); a skill-dir-inside-a-plugin also pulls the enclosing plugin's `agents/**`, `references/**`, and `commands/**`. MULTIPLE positionals are accepted (matches `lint-skill`'s `nargs="+"`), including a simple hand-rolled `*` glob — `dir/*.md` (shallow) or `dir/**/*.md` (recursive, no dependency, no `engines.node` bump) — with every positional's files UNIONed and deduped by resolved absolute path across the whole invocation; zero scannable files across ALL positionals is a usage error (exit 2), never a silent clean pass. Findings print but exit 0 by default (advisory-only); `--strict` — the CI-recommended invocation — fails (exit 1) if ANY scanned file (across all positionals) has an unsuppressed finding (mirrors `lint-skill --strict`). Three ignore markers, three scopes, applied PER FILE: `analyze-skill: ignore-next-line` silences just the following line, `analyze-skill: ignore-start`/`ignore-end` silences a fenced block (an unclosed `ignore-start` is itself a gating `unclosed-ignore-fence` finding — never a silent notice), and the file-wide `analyze-skill: ignore` silences every finding for that one file, even under `--strict` (all three: bare, or in an HTML comment). `--output-format json` nests results under `files: [{file, findings, suppressed}]` (plus `scanned`/`unscanned`) — see [docs/subagents.md](./docs/subagents.md#static-path-fidelity-check-analyze-skill) | catching the exact "skill hands `/sessions/...` to a file tool" defect statically, across a whole plugin's contract surface (or several paths/globs in one call), before paying for a live host-loop run to discover it |
|
|
386
386
|
| `probe-dispatch <skill-dir> "<prompt>"` | Cheap single-dispatch mechanics probe: a THIN wrapper over `skill` (default fidelity `hostloop`) that scopes a prompt to trigger ONE `Task` dispatch, then prints just that dispatch's `{resolvedAgentType, pathDenials, delivered}` — no new data model, a pure projection of the same `RunResult` `skill` already produces; `--expect-write <suffix>` narrows `delivered`; "one dispatch" is prompt-scoped, not enforced (`--output-format json` for machine consumption) | "did THIS dispatch resolve to the type I expect, avoid a path denial, and actually deliver its write?" without hand-writing a scenario or reading `trace --view dispatches` |
|
|
387
387
|
| `assertions --list` | List the available scenario assertions (generated from the schema) | "what can I assert?" without grepping the source |
|
|
388
388
|
| `decide` | Validate a decider against a sample question in ~2 s (no run) | sanity-check a `--decider-*` / `--answer` wiring before a long run |
|
|
@@ -613,13 +613,13 @@ The fastest path to CI: a composite action wrapping the token-free lane, with a
|
|
|
613
613
|
```yaml
|
|
614
614
|
- uses: yaniv-golan/cowork-harness@v1
|
|
615
615
|
with:
|
|
616
|
-
command: replay # replay | lint | verify-cassettes | run
|
|
616
|
+
command: replay # replay | lint | lint-skill | analyze-skill | verify-cassettes | run
|
|
617
617
|
path: cassettes/my-skill.cassette.json
|
|
618
618
|
```
|
|
619
619
|
|
|
620
620
|
| Lane | Commands | Runner requirements | What you get |
|
|
621
621
|
|---|---|---|---|
|
|
622
|
-
| **Token-free** (the headline lane) | `replay`, `lint`, `verify-cassettes` | any `ubuntu-latest` | deterministic, no Docker, no API key, no agent binary — this is what most skill repos want |
|
|
622
|
+
| **Token-free** (the headline lane) | `replay`, `lint`, `lint-skill`, `analyze-skill`, `verify-cassettes` | any `ubuntu-latest` | deterministic, no Docker, no API key, no agent binary — this is what most skill repos want. `lint`/`lint-skill` are thin passthroughs to the bundled `scenario.py` (python3, preinstalled on `ubuntu-latest`); `analyze-skill` is pure TS (no python3 needed) |
|
|
623
623
|
| **Live** (`command: run`) | `run` | Docker + a provisioned agent binary + `anthropic-api-key` input | real inference against a live scenario — the action does **not** provision the agent binary or build the image for you (see [Fidelity tiers](#fidelity-tiers-pick-per-scenario--per-ci-job) below and the agent-binary provenance runbook in [`docs/maintenance.md`](./docs/maintenance.md)); this is for a self-hosted runner that already has both staged, not a stock GitHub-hosted runner |
|
|
624
624
|
|
|
625
625
|
**Live lane, by design not oversight:** the action never downloads or stages the agent ELF itself.
|
|
@@ -646,7 +646,7 @@ jobs:
|
|
|
646
646
|
anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
|
|
647
647
|
```
|
|
648
648
|
|
|
649
|
-
Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest`), `strict`, `fail-on-skill-drift
|
|
649
|
+
Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest`), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN/INFO), and `analyze-skill` (any advisory finding); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only). See [`action.yml`](./action.yml) for the full input reference.
|
|
650
650
|
|
|
651
651
|
The provided [GitHub Actions workflow](.github/workflows/ci.yml) runs a **six-stage pipeline**. The **unit** stage is the token-free gate you can copy into your skill repo; the `python`, `boundary`, `scenarios`, `action-self-test`, and `parity-drift` stages are this repo's own fidelity self-tests and are not directly portable (they build the harness's Docker image and run harness-specific e2e scenarios — see [`ci-recipe.md`](./.claude/skills/cowork-harness/references/ci-recipe.md) for the skill-repo template):
|
|
652
652
|
|
package/dist/cli.js
CHANGED
|
@@ -107,10 +107,11 @@ const HELP = `cowork-harness <command> (v${"$VERSION"})
|
|
|
107
107
|
NOTE: exit 127 means python3 itself is missing — treat any non-zero exit as a CI failure, do not swallow it.
|
|
108
108
|
lint-skill <SKILL.md | skill-dir/>… lint a skill body (and any sibling hooks.json) for Cowork host-loop footguns (bundled scenario.py; needs python3)
|
|
109
109
|
[--strict] fail on any finding (the two footguns are WARN-only by default), not just ERROR
|
|
110
|
-
analyze-skill <SKILL.md | skill-dir
|
|
110
|
+
analyze-skill <SKILL.md | skill-dir/ | glob>… ADVISORY token-free scan: warns on a /sessions/... path handed to a file tool or dispatch/sub-agent output (denied on host-loop) — reuses the ported /sessions path-gate predicate; only the extraction is heuristic
|
|
111
|
+
A directory target scans the UNION of every contract-bearing markdown file present, not just SKILL.md: top-level SKILL.md + references/**, a plugin root's agents/*.md + references/**/commands/*.md/skills/*/SKILL.md(+references/**), and (for a skill dir inside a plugin) the enclosing plugin's agents/*.md. Multiple positionals are accepted (matches lint-skill's nargs="+"), incl. a simple hand-rolled '*' glob — "dir/*.md" (shallow) or "dir/**/*.md" (recursive) — with the results of ALL positionals UNIONed + deduped by resolved path. Zero scannable files across every positional is a usage error (exit 2), never a silent clean pass.
|
|
111
112
|
[--strict] fail (exit 1) on any finding instead of just warning (mirrors lint-skill's --strict)
|
|
112
113
|
analyze-skill: ignore a line with this marker (bare or in an HTML comment) in a SKILL.md silences EVERY finding for that file, even under --strict
|
|
113
|
-
[--output-format json] exit 0 = default (even with findings) · exit 1 = --strict + finding(s) · exit 2 = usage; a clean/suppressed result is a PRE-FLIGHT signal, not proof of on-tier resolution
|
|
114
|
+
[--output-format json] {tool,version,command,ok,files:[{file,findings,suppressed}],scanned,unscanned,strict,error} — exit 0 = default (even with findings) · exit 1 = --strict + finding(s) · exit 2 = usage; a clean/suppressed result is a PRE-FLIGHT signal, not proof of on-tier resolution
|
|
114
115
|
assertions --list list available scenario assertions (generated from Zod schema)
|
|
115
116
|
[--output-format json]
|
|
116
117
|
|
|
@@ -403,10 +404,18 @@ const SUBCOMMAND_USAGE = {
|
|
|
403
404
|
rehash: "usage: rehash <dir/> [--dry-run] [--output-format text|json] (migrate cassettes across format bumps using contentSig verification; no re-record needed)",
|
|
404
405
|
prune: "usage: prune [--keep-last <n>] [--dry-run] [<runs-dir>] (prune accumulated run dirs; default --keep-last 5)",
|
|
405
406
|
"init-redact": "usage: init-redact [--force] [--output-format json] (copy the packaged reference .cowork-redact.json into the cwd; refuses to overwrite an existing one without --force)",
|
|
406
|
-
"analyze-skill": "usage: analyze-skill <SKILL.md | skill-dir
|
|
407
|
+
"analyze-skill": "usage: analyze-skill <SKILL.md | skill-dir/ | glob>… [--strict] [--output-format text|json] (ADVISORY token-free scan: flags a /sessions/... path handed to a file tool or dispatch/sub-agent output — denied on host-loop)\n" +
|
|
407
408
|
" reuses the harness's own ported /sessions path-gate predicate (isVmSessionsPath) as the deny decision; only the EXTRACTION from SKILL.md text is heuristic.\n" +
|
|
408
|
-
"
|
|
409
|
-
"
|
|
409
|
+
" a DIRECTORY target scans the UNION of every contract-bearing markdown file present, not just SKILL.md — a plugin's dispatch/output contracts often live in agents/*.md or references/*.md:\n" +
|
|
410
|
+
" · top-level SKILL.md present → + top-level references/**/*.md\n" +
|
|
411
|
+
" · .claude-plugin/plugin.json or plugin.json present (a plugin root) → + agents/*.md, references/**/*.md, commands/*.md, skills/*/SKILL.md (+ that skill's own references/**/*.md)\n" +
|
|
412
|
+
" · a skill dir INSIDE a plugin (SKILL.md present, walking up finds an enclosing plugin manifest) → + that plugin's root agents/*.md\n" +
|
|
413
|
+
' MULTIPLE positionals are accepted (matches lint-skill\'s nargs="+"), each resolved independently (file/dir via the rules above, or a `*`-bearing target via a small hand-rolled glob matcher — `dir/*.md` shallow, `dir/**/*.md` recursive; no dependency, no engines bump).\n' +
|
|
414
|
+
" every positional's file set is UNIONed and deduped by resolved absolute path across the WHOLE invocation (a file reached both directly and via a dir/glob is analyzed once). ZERO scannable files across ALL positionals is a USAGE ERROR (exit 2), never a silent clean pass; an unresolvable single positional (bad path, unrecognized glob shape) fails the whole invocation the same way.\n" +
|
|
415
|
+
" findings are WARNINGS: exit 0 by default even when findings are printed. --strict fails (exit 1) if ANY scanned file (across ALL positionals) has an unsuppressed finding (mirrors lint-skill's --strict). exit 2 = usage error.\n" +
|
|
416
|
+
" plain `analyze-skill` (no --strict) is ADVISORY ONLY — CI should invoke `analyze-skill ... --strict` to actually gate on findings, same as `lint-skill --strict`.\n" +
|
|
417
|
+
" a line containing `analyze-skill: ignore` (bare, or inside an HTML comment) anywhere in one of the scanned files silences EVERY path-fidelity finding for THAT file — even under --strict, even a genuine true positive (an explicit author override); it does not affect other scanned files.\n" +
|
|
418
|
+
" --output-format json emits {tool,version,command,ok,files:[{file,findings,suppressed}],scanned,unscanned,strict,error} — NOT a bare array and NOT results[]; `scanned`/`unscanned` name the files analyzed and any contract dirs left out of scope (printed as a banner in text mode too).\n" +
|
|
410
419
|
" a clean/suppressed result is a PRE-FLIGHT signal only — the runtime no_vm_path_file_op / vm_path_denied asserts remain authoritative (see docs/subagents.md).",
|
|
411
420
|
};
|
|
412
421
|
// Known subcommands — used by the global value-flag parsers (`--dotenv`, `--run-dir`) to reject a command
|