cowork-harness 4.13.0 → 4.13.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/cowork-harness/SKILL.md +4 -4
- package/.claude/skills/cowork-harness/references/assertion-catalog-agents-skills-budgets.md +1 -1
- package/.claude/skills/cowork-harness/references/assertion-catalog-gates-hooks-modifiers.md +1 -1
- package/.claude/skills/cowork-harness/references/assertion-catalog-outcome-files-tools.md +1 -1
- package/.claude/skills/cowork-harness/references/assertion-catalog.md +1 -1
- package/.claude/skills/cowork-harness/references/assertions-guide.md +1 -1
- package/.claude/skills/cowork-harness/references/authoring.md +1 -1
- package/.claude/skills/cowork-harness/references/ci-recipe.md +5 -5
- package/.claude/skills/cowork-harness/references/critique.md +1 -1
- package/.claude/skills/cowork-harness/references/debugging.md +1 -1
- package/.claude/skills/cowork-harness/references/eval.md +1 -1
- package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +1 -1
- package/.claude/skills/cowork-harness/references/gotchas.md +1 -1
- package/.claude/skills/cowork-harness/references/hillclimb-recipe.md +1 -1
- package/.claude/skills/cowork-harness/references/hillclimb.md +1 -1
- package/.claude/skills/cowork-harness/references/measurement.md +1 -1
- package/.claude/skills/cowork-harness/references/run-record-replay.md +1 -1
- package/.claude/skills/cowork-harness/references/scenario-schema.md +1 -1
- package/.claude/skills/cowork-harness/references/semantic-judging.md +1 -1
- package/.claude/skills/cowork-harness/references/task-recipes.md +1 -1
- package/CHANGELOG.md +17 -0
- package/README.md +3 -3
- package/docs/ci.md +1 -1
- package/docs/cli.md +5 -5
- package/docs/companion-skill.md +2 -2
- package/examples/replays/README.md +1 -1
- package/package.json +1 -1
|
@@ -3,8 +3,8 @@ name: cowork-harness
|
|
|
3
3
|
description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, asserting artifacts, egress, or sub-agent dispatch, measuring how long each tool call took (toolDurations / trace), or debugging a failed run or verdict from its result.json or transcript. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. Also when iterating or hardening a skill across fixes, or grounding a skill's self-critique against its own run evidence — including a document-analysis skill (cap table, deck, financial model, transcript) that needs an uploaded file attached to be critiqued at all. Also for comparing two versions of a skill before merging an edit — did it make answers worse? (`eval`: paired A/B, pinned models) — or improving one round by round (`hillclimb`, the `/claude-api hillclimb` runner). NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold / critique / stats / eval / hillclimb commands and the session-vs-scenario split.
|
|
4
4
|
metadata:
|
|
5
5
|
author: cowork-harness
|
|
6
|
-
version: 4.13.
|
|
7
|
-
tracks-harness: cowork-harness 4.13.
|
|
6
|
+
version: 4.13.1
|
|
7
|
+
tracks-harness: cowork-harness 4.13.1 (baseline desktop-4.4212.0)
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# cowork-harness
|
|
@@ -26,7 +26,7 @@ allowlist). This skill exists mostly to keep you out of those traps — the *Inv
|
|
|
26
26
|
full landmine catalog in [`references/gotchas.md`](references/gotchas.md) are the highest-value part.
|
|
27
27
|
Read them.
|
|
28
28
|
|
|
29
|
-
> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.13.
|
|
29
|
+
> **Version note:** the facts and `file:line` pointers here track `cowork-harness 4.13.1` (baseline
|
|
30
30
|
> `desktop-4.4212.0`). If your checkout is newer, prefer the live `--help` and — in a repo checkout —
|
|
31
31
|
> `SPEC.md` / `docs/*.md` over this snapshot, and re-run the bundled linter.
|
|
32
32
|
|
|
@@ -43,7 +43,7 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
|
|
|
43
43
|
|
|
44
44
|
- **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, the baseline and the `device_bridge` image (built, or built on the first bridge run) in one pass. The bullets below explain each thing it checks (and how to fix it).
|
|
45
45
|
- **Replay-only? Skip `doctor`.** Replaying committed cassettes needs no Docker, no staged agent, and no token — and every tier's `doctor` validates the auth token (the live tiers also Docker + the staged agent), so a ✗ there is expected, not a blocker. Go straight to `cowork-harness replay <cassette>`.
|
|
46
|
-
- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.13.
|
|
46
|
+
- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 4.13.1**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@^4.13.1" <cmd>` (Node ≥ 22), or install once with `npm i -g "cowork-harness@^4.13.1"`. **Pin `@^4.13.1`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published.
|
|
47
47
|
|
|
48
48
|
This skill documents the CURRENT surface, not release history. If `cowork-harness --version` is
|
|
49
49
|
OLDER than the floor, the per-release record of what you are missing is [CHANGELOG.md](https://github.com/yaniv-golan/cowork-harness/blob/main/CHANGELOG.md)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Assertion catalog: sub-agents, skills and connectors, budgets, tasks, delivery
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Part 2 of 3 of the per-key table. The conventions every key shares (an item with several
|
|
4
4
|
keys is an AND; glob vs regex name matching) and the verdict-signal table are in
|
|
5
5
|
[`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Assertion catalog: gates, hooks, path denial, verdict modifiers, egress, judged keys
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Part 3 of 3 of the per-key table. The conventions every key shares (an item with several
|
|
4
4
|
keys is an AND; glob vs regex name matching) and the verdict-signal table are in
|
|
5
5
|
[`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-outcome-files-tools.md`](./assertion-catalog-outcome-files-tools.md), [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md).
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Assertion catalog: outcome, transcript, files and artifacts, tools, tool results
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Part 1 of 3 of the per-key table. The conventions every key shares (an item with several
|
|
4
4
|
keys is an AND; glob vs regex name matching) and the verdict-signal table are in
|
|
5
5
|
[`assertion-catalog.md`](./assertion-catalog.md); the other parts are [`assertion-catalog-agents-skills-budgets.md`](./assertion-catalog-agents-skills-budgets.md), [`assertion-catalog-gates-hooks-modifiers.md`](./assertion-catalog-gates-hooks-modifiers.md).
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Assertion catalog
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Every `assert:` key with its semantics, and the
|
|
4
4
|
verdict-signal table. Which keys survive `replay` is in [`scenario-schema.md`](./scenario-schema.md#replay-class);
|
|
5
5
|
the scenario and session YAML fields are there too.
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Assertions guide
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when choosing assertion keys: the two orthogonal axes and the goal → key map. The full catalog starts at `assertion-catalog.md`, which indexes the per-family files that hold every key's row.
|
|
4
4
|
|
|
5
5
|
### Assertions: two orthogonal axes
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Authoring a scenario
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when composing a `scenarios/*.yaml`: session vs scenario, discovery, the fidelity tier, the answer path, `web_fetch`, and scaffold + lint.
|
|
4
4
|
|
|
5
5
|
## Part I — AUTHOR a scenario
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# CI recipe — replay vs live lanes
|
|
2
2
|
|
|
3
|
-
Self-contained reference. Tracks `cowork-harness 4.13.
|
|
3
|
+
Self-contained reference. Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`).
|
|
4
4
|
|
|
5
5
|
**Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
|
|
6
6
|
job-summary reporter (verdict table, staleness findings, cost/turns when available):
|
|
@@ -17,7 +17,7 @@ job-summary reporter (verdict table, staleness findings, cost/turns when availab
|
|
|
17
17
|
CLI major reaches your workflow the moment it is promoted even though your `uses:` ref never changed — so a
|
|
18
18
|
copy-pasted recipe that omits the input takes a major bump with no say in it. `^4` holds the major, needs no
|
|
19
19
|
patch number to remember, and only wants a human decision at the next major. Pin an exact version
|
|
20
|
-
(e.g. `version: "4.13.
|
|
20
|
+
(e.g. `version: "4.13.1"`) instead when you want byte-reproducible CI.
|
|
21
21
|
|
|
22
22
|
Reach for the manual multi-step form below only when you need per-step control the Action's inputs don't
|
|
23
23
|
cover (a custom flag combination, a different runner matrix per step, or `lint`/`verify-cassettes` gated
|
|
@@ -82,7 +82,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
|
|
|
82
82
|
GitHub-hosted runners, no token/Docker/agent:
|
|
83
83
|
|
|
84
84
|
```yaml
|
|
85
|
-
- run: npm i -g "cowork-harness@^4.13.
|
|
85
|
+
- run: npm i -g "cowork-harness@^4.13.1"
|
|
86
86
|
- run: cowork-harness lint scenarios/*.yaml --strict --min-severity WARN
|
|
87
87
|
# no silent false-greens. WITHOUT --strict this
|
|
88
88
|
# step cannot fail on a WARN-class rule (e.g.
|
|
@@ -411,7 +411,7 @@ jobs:
|
|
|
411
411
|
with: { node-version: '24' }
|
|
412
412
|
- uses: actions/setup-python@v5
|
|
413
413
|
with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
|
|
414
|
-
- run: npm i -g "cowork-harness@^4.13.
|
|
414
|
+
- run: npm i -g "cowork-harness@^4.13.1"
|
|
415
415
|
- run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
|
|
416
416
|
- run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
|
|
417
417
|
- run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
|
|
@@ -440,7 +440,7 @@ jobs:
|
|
|
440
440
|
echo "live=true" >> "$GITHUB_OUTPUT"
|
|
441
441
|
fi
|
|
442
442
|
- if: steps.guard.outputs.live == 'true'
|
|
443
|
-
run: npm i -g "cowork-harness@^4.13.
|
|
443
|
+
run: npm i -g "cowork-harness@^4.13.1"
|
|
444
444
|
- if: steps.guard.outputs.live == 'true'
|
|
445
445
|
run: cowork-harness run scenarios/ --output-format json
|
|
446
446
|
env:
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Critique — the facts a plugin install can't otherwise reach
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). This is **not** a trim of the full
|
|
4
4
|
[`docs/critique.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/critique.md) (repo-only —
|
|
5
5
|
flags, cost, reading several critiques, known limitations all live there). This file covers exactly what a
|
|
6
6
|
plugin install cannot otherwise discover: the run-dir artifact a harvester actually reads, the report's
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Debugging a run
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when a run misbehaved or a green looks wrong: triage, the observability output, and `chat`.
|
|
4
4
|
|
|
5
5
|
## Part III — Debug
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# `eval` — paired before/after comparison of a skill edit (EXPERIMENTAL)
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). The full guide is
|
|
4
4
|
[docs/eval.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/eval.md); this is the part you
|
|
5
5
|
need while running it.
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Fidelity tiers & answer paths
|
|
2
2
|
|
|
3
|
-
Self-contained reference. Tracks `cowork-harness 4.13.
|
|
3
|
+
Self-contained reference. Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`).
|
|
4
4
|
|
|
5
5
|
> **This page vs. the repo docs.** This is the **offline snapshot** that ships inside the installed
|
|
6
6
|
> plugin — it is self-contained on purpose. The repo carries four other fidelity views, each answering a
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Gotchas
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). The full "✓ passed ≠ correct" landmine catalog.
|
|
4
4
|
|
|
5
5
|
## Gotchas — the "✓ passed ≠ correct" landmines
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Recipe 7 — Climb a skill with `/claude-api hillclimb` and the harness as its runner
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). It needs a `cowork-harness` whose
|
|
4
4
|
`hillclimb --help` lists `--skill` (help goes to stderr). This page is the loop's procedure, step by step, in the order of the
|
|
5
5
|
`/claude-api hillclimb` guide. Every mechanic (flags, refusals, the gate, `regrade`, `freeze-ref`, exit codes,
|
|
6
6
|
row keys) is in [`hillclimb.md`](hillclimb.md); the setup and the full list of differences from the guide's own
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# `hillclimb` — the runner for a `/claude-api hillclimb` loop
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). It needs a `cowork-harness` whose
|
|
4
4
|
`hillclimb --help` lists `--skill` (help goes to stderr). The command reference is
|
|
5
5
|
[docs/cli.md](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/cli.md); this is the part a loop needs
|
|
6
6
|
while it runs. It covers `run`, `check`, `state-template`, `freeze-ref` and `regrade`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Measurement
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it before comparing runs: `--repeat`, `--ablate-skill`, and the hygiene that keeps a batch valid.
|
|
4
4
|
|
|
5
5
|
### Measure — before/after, with/without (`--repeat`, `--ablate-skill`)
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Run, record and lock
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). Read it when running a scenario, recording or placing a cassette, reading verdict signals, checking a background run, or choosing CI lanes.
|
|
4
4
|
|
|
5
5
|
## Part II — RUN, RECORD & LOCK
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Scenario & session schema, replay class, web_fetch, authoring gotchas
|
|
2
2
|
|
|
3
|
-
Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.13.
|
|
3
|
+
Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 4.13.1`
|
|
4
4
|
(baseline `desktop-4.4212.0`). If your checkout is newer, prefer the live [`docs/scenario.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/scenario.md),
|
|
5
5
|
[`docs/session.md`](https://github.com/yaniv-golan/cowork-harness/blob/main/docs/session.md), and `SPEC.md`.
|
|
6
6
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# `semantic_matches` — the full semantics
|
|
2
2
|
|
|
3
|
-
Tracks `cowork-harness 4.13.
|
|
3
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`). The one-line summary is in
|
|
4
4
|
[assertion-catalog-gates-hooks-modifiers.md](assertion-catalog-gates-hooks-modifiers.md); this is the whole contract, split out so the catalog stays within the
|
|
5
5
|
agent's single-read size.
|
|
6
6
|
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Each recipe composes facts that live scattered across SKILL.md and the other references into one
|
|
4
4
|
decision path. Every one answers a question a real fleet owner had to work out the hard way.
|
|
5
|
-
Tracks `cowork-harness 4.13.
|
|
5
|
+
Tracks `cowork-harness 4.13.1` (baseline `desktop-4.4212.0`), same as SKILL.md's front-matter. Recipe 2's `resolved-tier`/`unverifiable-tier` staleness classes and
|
|
6
6
|
Recipe 3's `init-redact` shipped in 0.24.0 and are part of the current feature set — no version gate
|
|
7
7
|
needed if your CLI meets SKILL.md's version floor.
|
|
8
8
|
|
package/CHANGELOG.md
CHANGED
|
@@ -6,6 +6,23 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [4.13.1] - 2026-11-02
|
|
10
|
+
|
|
11
|
+
A dependency-only patch: no code, baseline, result field or cassette changes.
|
|
12
|
+
|
|
13
|
+
### Security
|
|
14
|
+
|
|
15
|
+
- **The lockfile moves two runtime dependencies past known advisories:** `zod` 4.6.5 → 4.6.7, and `fast-uri`
|
|
16
|
+
3.1.8 → 3.1.9 (reached through `ajv`). The package ships no shrinkwrap, so an npm install already resolved the
|
|
17
|
+
newest versions its ranges allow; this changes what a source checkout and this repository's CI install.
|
|
18
|
+
- **Development dependencies:** `undici` 8.11.2 → 8.12.3 (past a known advisory), `vitest` 5.0.3 → 5.0.5 and
|
|
19
|
+
`jsdom` 30.1.2 → 30.1.3, with their transitive updates. None is in the published package.
|
|
20
|
+
|
|
21
|
+
### Changed
|
|
22
|
+
|
|
23
|
+
- **CI action pins:** `actions/setup-python` v7.0.0 → v7.1.0 and `docker/login-action` v4.6.0 → v4.7.0, each pinned
|
|
24
|
+
by commit SHA.
|
|
25
|
+
|
|
9
26
|
## [4.13.0] - 2026-11-01
|
|
10
27
|
|
|
11
28
|
### Upgrade notes
|
package/README.md
CHANGED
|
@@ -36,7 +36,7 @@ npm ci && npm run build
|
|
|
36
36
|
node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
-
(Installing globally — `npm install -g "cowork-harness@^4.13.
|
|
39
|
+
(Installing globally — `npm install -g "cowork-harness@^4.13.1"` — gives you the `cowork-harness` CLI for your own
|
|
40
40
|
scenarios and cassettes; the bundled example above also replays from a global install — see the `$(npm root -g)` path below.)
|
|
41
41
|
|
|
42
42
|
Full setup → [Quick start](./docs/cli.md#quick-start).
|
|
@@ -49,8 +49,8 @@ Three ways to use this project. Each row is the whole hook — follow the link f
|
|
|
49
49
|
|
|
50
50
|
| I want to… | Start here | Needs |
|
|
51
51
|
|---|---|---|
|
|
52
|
-
| **Run scenarios myself** from a terminal | **[docs/cli.md](./docs/cli.md)**<br><br>`npm i -g "cowork-harness@^4.13.
|
|
53
|
-
| **Have Claude Code drive it** for me | **[docs/companion-skill.md](./docs/companion-skill.md)**<br><br>`/plugin marketplace add yaniv-golan/cowork-harness`<br>`/plugin install cowork-harness@cowork-harness` | Claude Code. The skill self-bootstraps the CLI via `npx "cowork-harness@^4.13.
|
|
52
|
+
| **Run scenarios myself** from a terminal | **[docs/cli.md](./docs/cli.md)**<br><br>`npm i -g "cowork-harness@^4.13.1"`<br>`cowork-harness replay examples/replays/example-pdf-skill.cassette.json` | Node ≥ 22. The replay demo above is token-free and needs nothing else; live tiers above `protocol` need Docker + a staged agent binary |
|
|
53
|
+
| **Have Claude Code drive it** for me | **[docs/companion-skill.md](./docs/companion-skill.md)**<br><br>`/plugin marketplace add yaniv-golan/cowork-harness`<br>`/plugin install cowork-harness@cowork-harness` | Claude Code. The skill self-bootstraps the CLI via `npx "cowork-harness@^4.13.1"` |
|
|
54
54
|
| **Gate my skill in CI** | **[docs/ci.md](./docs/ci.md)**<br><br>`- uses: yaniv-golan/cowork-harness@v4`<br>` with: { command: replay, path: cassettes/ }` | Nothing for the token-free gate; the live lane needs a self-hosted runner with Docker + an agent binary |
|
|
55
55
|
|
|
56
56
|
Improving a skill round by round with Claude Code's `/claude-api hillclimb` loop? The harness is its runner, with
|
package/docs/ci.md
CHANGED
|
@@ -147,7 +147,7 @@ jobs:
|
|
|
147
147
|
model: claude-sonnet-5 # used only where a scenario's session sets no `model:`
|
|
148
148
|
```
|
|
149
149
|
|
|
150
|
-
Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — the recipes above pin `^4` instead, because leaving it at `latest` takes a CLI major the moment it is promoted even though your `uses:` ref never changed; pin an exact version for byte-reproducible CI. The companion skill's `cowork-harness@^4.13.
|
|
150
|
+
Every run writes a Markdown verdict table (scenario, pass/fail, signals, cost/turns when available, staleness findings, and the replay-skipped-assertions honesty line) to the job summary. Inputs: `command`, `path` (required), `version` (npm dist-tag/version, default `latest` — the recipes above pin `^4` instead, because leaving it at `latest` takes a CLI major the moment it is promoted even though your `uses:` ref never changed; pin an exact version for byte-reproducible CI. The companion skill's `cowork-harness@^4.13.1` floor guidance applies to ad-hoc CLI installs, not this input), `strict` (applies to `replay` (staleness findings), `lint`/`lint-skill` (WARN; INFO never gates unless `lint` gets `--min-severity INFO` via `extra-args`; a reviewed `lint-skill` finding you accept is suppressed with `extra-args: --suppressions <file> --strict-ignores` (one entry per accepted site, in a JSON file kept outside the plugin; a stale entry fails too) or `--ignore-rule <rule>[=<glob>]`, either of which keeps every other WARN gating — see [cli.md](./cli.md#flags-worth-knowing)), and `analyze-skill` (any **`error`**-severity finding — advisory findings are precisely the class that does NOT gate); IGNORED — not forwarded — for `verify-cassettes`/`run`, which don't accept the flag), `fail-on-skill-drift` (**`replay`-only** — never forwarded to the analyzers), `extra-args`, `summary` (default `true`), `anthropic-api-key` (live lane only), `model` (live lane only: exported as `COWORK_HARNESS_MODEL`, which fills in where a session sets no `model:` and no `--model` is passed; a `run` that resolves no model is refused with exit 2). Outputs: `ok` (`"true"`/`"false"`, mirrors the exit code), `envelope-path` (path to the raw JSON envelope, for post-processing), `summary-md` (the rendered verdict table, exposed as an output — not just written to `$GITHUB_STEP_SUMMARY` — because that file is scoped to this action's own invocation and a caller's later step gets a fresh, empty one). See [`action.yml`](https://github.com/yaniv-golan/cowork-harness/blob/main/action.yml) for the full input/output reference.
|
|
151
151
|
|
|
152
152
|
CI uses `ANTHROPIC_API_KEY` specifically because there's no interactive browser available to run
|
|
153
153
|
`claude setup-token`'s OAuth flow in a GitHub Actions runner; locally, the OAuth token is preferred because
|
package/docs/cli.md
CHANGED
|
@@ -18,7 +18,7 @@ companion skill, CI). This page is the CLI one.
|
|
|
18
18
|
**Install from npm:**
|
|
19
19
|
|
|
20
20
|
```bash
|
|
21
|
-
npm install -g "cowork-harness@^4.13.
|
|
21
|
+
npm install -g "cowork-harness@^4.13.1" # puts the `cowork-harness` command on your PATH
|
|
22
22
|
```
|
|
23
23
|
|
|
24
24
|
**Or build from source:**
|
|
@@ -38,7 +38,7 @@ node dist/cli.js replay examples/replays/example-pdf-skill.cassette.json
|
|
|
38
38
|
|
|
39
39
|
> **Installed globally instead?** Once linked/installed, the same command is `cowork-harness replay
|
|
40
40
|
> <cassette>` — but the relative path above only resolves from a source checkout's `examples/replays/`.
|
|
41
|
-
> From a global install (`npm i -g "cowork-harness@^4.13.
|
|
41
|
+
> From a global install (`npm i -g "cowork-harness@^4.13.1"`), point at the package root instead:
|
|
42
42
|
> `cowork-harness replay "$(npm root -g)/cowork-harness/examples/replays/example-pdf-skill.cassette.json"`
|
|
43
43
|
> (or copy the cassette into your own project and pass that path).
|
|
44
44
|
|
|
@@ -48,7 +48,7 @@ Live `run`/`skill` need the prerequisites in the next section — note the `prot
|
|
|
48
48
|
> - **Replay only (zero setup):** `cowork-harness replay <cassette>` — no token, no Docker, no agent. The command above.
|
|
49
49
|
> - **`protocol` (real model, no Docker):** needs only the auth token (item 3 below).
|
|
50
50
|
> - **Live `container` / `microvm` / `hostloop` / `cowork`:** needs Docker (or Lima for `microvm`), a staged agent, and the token — run `cowork-harness doctor` first.
|
|
51
|
-
> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@^4.13.
|
|
51
|
+
> - **Invocation:** from a source checkout, `node dist/cli.js <cmd>` (or `npm link` to get the `cowork-harness` command); from a global install, `cowork-harness <cmd>`; the companion skill falls back to `npx "cowork-harness@^4.13.1"`.
|
|
52
52
|
|
|
53
53
|
Two more worked examples worth knowing about: `examples/scenarios/protocol-smoke.yaml` (zero-Docker smoke
|
|
54
54
|
test) and `examples/scenarios/skill-loads.yaml` (container-tier acceptance check) — see
|
|
@@ -126,7 +126,7 @@ The parts people ask about. **`package.json`'s `files[]` is the exhaustive, mach
|
|
|
126
126
|
this table is the readable summary of it, and deliberately omits the infrastructure that always ships
|
|
127
127
|
(`baselines/`, `schema/`, `fixtures/`, `scripts/`, `docker/`).
|
|
128
128
|
|
|
129
|
-
| What ships | npm global (`npm install -g "cowork-harness@^4.13.
|
|
129
|
+
| What ships | npm global (`npm install -g "cowork-harness@^4.13.1"`) | Source checkout (`git clone` + `npm ci`) |
|
|
130
130
|
|---|---|---|
|
|
131
131
|
| CLI, `scenario.py` + assertion keys (enough for `lint` in CI) | ✓ | ✓ |
|
|
132
132
|
| `SKILL.md`, all of `docs/`, `SPEC.md`/`DESIGN.md`/`AGENTS.md` | ✓ | ✓ |
|
|
@@ -141,7 +141,7 @@ since a global install puts nothing in your working directory. The matrix, answe
|
|
|
141
141
|
examples are the only ones that still need a source checkout. The **marketplace skill install** is
|
|
142
142
|
narrower again — it pulls only `.claude/skills/cowork-harness/` (SKILL.md + `references/` +
|
|
143
143
|
`scenario.py`/assertion keys, per `.claude-plugin/marketplace.json`'s `source`); everything in the npm column
|
|
144
|
-
arrives when the skill's first command self-bootstraps `npx "cowork-harness@^4.13.
|
|
144
|
+
arrives when the skill's first command self-bootstraps `npx "cowork-harness@^4.13.1"` — the last row stays
|
|
145
145
|
✗ either way, since `matrices/`, `answer-policies/` and `probes/` are not published at all. See
|
|
146
146
|
[docs/companion-skill.md](./companion-skill.md) for that install path.
|
|
147
147
|
|
package/docs/companion-skill.md
CHANGED
|
@@ -28,7 +28,7 @@ claude plugin marketplace add yaniv-golan/cowork-harness
|
|
|
28
28
|
claude plugin install cowork-harness@cowork-harness
|
|
29
29
|
```
|
|
30
30
|
|
|
31
|
-
The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@^4.13.
|
|
31
|
+
The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx "cowork-harness@^4.13.1"` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 22). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
|
|
32
32
|
|
|
33
33
|
It also follows the open [Agent Skills](https://agentskills.io) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) via [`npx skills`](https://github.com/vercel-labs/skills) (Vercel Labs' CLI implementation of that spec):
|
|
34
34
|
|
|
@@ -46,7 +46,7 @@ hook-event list verifies none). SKILL.md is a
|
|
|
46
46
|
short entrypoint (routing, the false-green invariants, short workflows); the detail it routes to lives in
|
|
47
47
|
`references/`, which the agent reads on demand. Everything
|
|
48
48
|
else (the CLI, `docs/`, the worked examples, the pytest lane) arrives when the skill's first command
|
|
49
|
-
self-bootstraps `npx "cowork-harness@^4.13.
|
|
49
|
+
self-bootstraps `npx "cowork-harness@^4.13.1"`, which pulls the same npm package as a global install.
|
|
50
50
|
|
|
51
51
|
For the full package-contents table — what a global install gives you versus a source checkout — see
|
|
52
52
|
[docs/cli.md → What ships](./cli.md#what-ships). It is maintained there, once.
|
|
@@ -16,7 +16,7 @@ DOES exercise a real gate exchange, see `example-multiselect-gate.cassette.json`
|
|
|
16
16
|
|
|
17
17
|
Run it with:
|
|
18
18
|
|
|
19
|
-
> Assumes the `cowork-harness` CLI is available — from a source checkout run `npm ci && npm run build && npm link` first, or `npm i -g "cowork-harness@^4.13.
|
|
19
|
+
> Assumes the `cowork-harness` CLI is available — from a source checkout run `npm ci && npm run build && npm link` first, or `npm i -g "cowork-harness@^4.13.1"`. (`replay` itself needs nothing else — no token, no Docker.)
|
|
20
20
|
|
|
21
21
|
```sh
|
|
22
22
|
cowork-harness replay examples/replays/example-pdf-skill.cassette.json
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "cowork-harness",
|
|
3
|
-
"version": "4.13.
|
|
3
|
+
"version": "4.13.1",
|
|
4
4
|
"description": "Scriptable, CI-friendly harness for Claude Cowork's runtime contract for testing skills across scenarios — same agent, mounts, egress allowlist, permission protocol, and sandbox limitations.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|