cowork-harness 0.1.0 → 0.2.0

package/CHANGELOG.md

@@ -4,187 +4,97 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). The project uses
 [Semantic Versioning](https://semver.org/); pre-1.0 minor versions may include breaking changes.
 
-## [0.1.0] — 2026-06-13
+## [Unreleased]
 
-Initial public release. A faithful, headless, scriptable harness for Claude Cowork's runtime for
-testing Claude Code **skills** outside the Desktop app — same staged agent, same spawn/control-protocol
-contract, same sandbox limitations, binary-grounded against `app.asar` 1.12603.1 / agent ELF 2.1.170.
+## [0.2.0] — 2026-06-17
+
+Binary-verified the AskUserQuestion answer wire shape (agent ELF 2.1.170), implemented the
+harness-improvements plan, and resolved a 39-finding code-review pass behind two centralizing seams.
 
 ### Added
 
-- **Skill & scenario testing.** `cowork-harness skill <folder> "<prompt>"` runs a local skill folder
-  directly (copied fresh each run — no install, marketplace, or version bump). `cowork-harness run
-  <scenario.yaml | dir/>` runs authored, asserted regression scenarios with a CI-ready exit code.
-  `--prompt-file <path>` passes a prompt **verbatim** (raw bytes, no shell `$`/backtick expansion) — the
-  faithful-relay path for prompts with shell metacharacters. Per-command `skill --help` / `run --help`.
-- **Unified output.** `skill` renders the agent's work (assistant text + tool calls) + a metered
-  footer; `run` is verdict-first but prints the **failing transcript inline** on a `FAIL`. `--quiet` /
-  `--verbose` / `NO_COLOR`. `--output-format json` emits a stable, pipe-safe **compact single-line** envelope on
-  stdout (`{tool, version, command, ok, results[], error}`; errors carry `{category, message, hint}`) —
-  human output stays on stderr (see [SPEC §11](./SPEC.md)). Exit codes `0`/`1`/`2`.
-- **Five fidelity tiers** (`fidelity:`): `protocol` (L0, no sandbox), `container` (L1 Docker + per-run
-  default-deny egress proxy), `microvm` (L2 Apple-VZ Lima microVM + guest firewall), `hostloop`
-  (Cowork's production split-execution: agent loop on the host, shell/web routed into the VM via the
-  workspace SDK-MCP server), and `cowork` (auto-picks host-loop vs container the way Cowork does, via
-  GrowthBook gate `1143815894` decoded from the synced baseline).
-- **Three-seam driver.** `AgentSession` (typed event stream over the stream-json control protocol) →
-  `Decider` (policy) → `Run` (turn loop + `RunRecord`). Multi-turn capable; the sub-agent dispatch tree,
-  decisions (with who-decided + rationale), and cost are recorded.
-- **Input policy — no silent false-greens.** Scripted `answers:` / `--answer "q=choice"` resolve the
-  agent's questions and tool-permission requests. An unscripted question follows an explicit
-  `on_unanswered` policy: `fail` (error + the exact `--answer` to add — always the default for `run`),
-  `prompt` (ask at the TTY), or `first` (pick option 1, loudly warn). Left unset, `skill` is adaptive
-  (`prompt` on a TTY, `fail` when piped/CI). Exit codes: `0` pass · `1` assertion/agent failure · `2` usage / unanswered-under-`fail`.
-  Tool permissions follow a `cowork` (allow-unscripted with an audit finding) or `strict` (deny) parity.
-- **In-band gate answering by the driving agent — `--decider-dir <dir>` + Monitor.** For the cases where
-  the right answer encodes the *driving agent's test intent* (branch-steer / reproduce / boundary /
-  differential), the harness writes each live gate to `<dir>/req-N.json` and blocks for `<dir>/resp-N.json`;
-  the driving Claude session arms a **Monitor** on the dir (each gate wakes it via a task-notification) and
-  writes the answer — the LIVE `AskUserQuestion` is answered **in-band**, no resume, no re-worded question
-  (mechanism verified live). Same wire protocol as the other channels (reuses `ExternalDecider` whole);
-  stdout stays free so it composes with `--output-format json` + `run_in_background`. The run is flagged
-  non-deterministic. (Replaces the deferred, resume-brittle exit-at-gate approach.) Emits `[gate] req-N
-  emitted` / `[gate] resp-N consumed` to stderr (visible even under `--output-format json`) and renames a
-  consumed `req-N.json` → `.done` so a watcher can't re-emit it. The harness **owns the transport** so
-  the driver only reads-and-decides: **`cowork-harness gates <dir> --follow`** streams one clean JSON
-  line per pending gate + a terminal `{"done":true}` (point a single Monitor at it — no hand-written
-  zsh/find/seen-set loop), and **`cowork-harness answer <dir> --gate <N> --choose <label>`** writes the
-  answer atomically with the right wire shape.
-- **LLM decider — state the test's intent in one line (`--decider-llm --intent "<text>"`).** For
-  agent-driven runs where writing a `--decider-cmd` helper is overkill: a small model (host `claude -p`,
-  `COWORK_HARNESS_DECIDER_MODEL`) picks an option **by label** per live question, optionally steered by a
-  one-line intent (e.g. "test the not_ai branch" → picks `not_ai`). Scripted `--answer`/`--answer-policy`
-  still resolve first; an out-of-set answer **fails loud** (never a silent default). Because it's
-  non-deterministic, the run is flagged `nonDeterministic` and the footer prints `⚠ non-deterministic
-  (LLM-decided)` so a green can't be mistaken for a reproducible/scripted pass. Validate it in ~2s with
-  `cowork-harness decide --decider-llm --intent …`. (`--decider-llm` is the only user-facing spelling;
-  the internal `agent` policy it rides is not a CLI flag — `--on-unanswered agent` is rejected with a redirect.)
-- **Safety fix — a question is never silently answered with option 1.** A question that reaches the
-  decider chain's terminal unanswered now **fails loud** (`UnansweredError`) instead of the prior
-  silent option-1 fallback in the run loop — closing the worst failure mode (a wrong-branch run printing
-  `✓ success`). Permissions/dialogs still fail *closed* (deny/cancel), which is correct.
-- **External decider — answer the LIVE question (the stochastic-question fix).** Because a skill's
-  AskUserQuestions are LLM-generated and vary run to run, a pre-written `--answer` regex is brittle.
-  **`--decider-cmd '<helper>'`** spawns a helper once and pipes each *actual* live question (with options
-  + a scrubbed transcript `context` + a literal `reply_with` template) to it, reading the answer back —
-  the agent-usable, one-shot path for custom logic (even an LLM call). The Python package's
-  **`serve_decider(fn)`** pre-builds the wire loop so a helper writes only the decision function (the
-  spawn-helper analogue of the `gates`/`answer` commands). Replies are lenient (label OR 1-based index,
-  `id` optional); scripted `--answer` + permission parity still apply first; the request is
-  secret-scrubbed before it leaves the process. The helper owns its own pipes, so **`--decider-cmd`
-  keeps the CLI's stdout free and composes with `--output-format json`** — as does `--decider-dir`; both
-  external channels are stdout-free and orthogonal (pick one terminal).
-- **Egress sandbox.** Default-deny outbound, enforced against the **synced** Cowork domain allowlist
-  (plus per-scenario `extra_allow`); `egress_*` / `expect_denied` assertions; `web_fetch` modeled
-  host/API-routed (gated by a web-fetch allowlist) as in real Cowork, distinct from container-sandboxed
-  `bash`.
-- **Assertions** (`assert:`): transcript, files, user-visible artifacts, tool / sub-agent usage,
-  `subagent_dispatched` / `subagent_declared_but_unused` / `dispatch_count_max`, egress, no-delete-in-
-  outputs, self-heal, host-path-leak, question count, `gate_answers_delivered`, result status, and
-  **`transcript_matches`/`transcript_not_matches`** (case-insensitive regex over the transcript — the
-  drift-tolerant content check for stochastic prose; replay-safe, so it runs on the token-free PR gate).
-- **AskUserQuestion answer delivery (correctness fix).** The answer to an AskUserQuestion gate is now
-  injected as the binary's COMPLETE tool input — `updatedInput:{questions, answers}`, not `{answers}` —
-  matching the ELF's built-in handler, which does `questions.map(…)` over the input (verified against
-  `claude-code-vm` 2.1.170). Dropping `questions` threw `undefined is not an object (evaluating 'q.map')`,
-  so the answer never reached the model and gate-steering silently no-oped. (The earlier golden snapshot
-  had blessed the `{answers}`-only shape as "faithful"; it was the bug — corrected, with a regression test
-  asserting `questions` is preserved.) **New verification surfaces:** `tool_result` blocks are now captured;
-  `RunResult.gateDeliveries[]` + the `gate_answers_delivered` assertion confirm each answer actually
-  reached the model (a `::warning:: [gate] DELIVERY FAILED` fires in real time on an errored result);
-  `cowork-harness trace <id> --tools` shows each tool's result status; `trace <id> --gates` shows the gate
-  lifecycle (question → injected answer → delivered result); the gate rendezvous wire shapes are
-  written into `<run>/gates/` on every run (so the forensic evidence survives the channel's cleanup, even
-  without `--keep`).
-- **Truthful tool counts (`RunResult.toolCounts`).** Per-tool call counts from the actual tool_use stream
-  (top-level only). On the cowork path `usage.server_tool_use.web_search_requests` is 0 (it counts the
-  Anthropic *server* tool; WebSearch is a host-routed *client* tool) — `toolCounts.WebSearch` is the real
-  count to assert on.
-- **Run-once-then-script.** Every question the agent asks that wasn't pre-scripted (auto-answered by
-  `first`, or answered interactively) is echoed on the footer as a copy-pasteable `--answer "<q>=<choice>"`
-  line — turning an exploratory run into a deterministic one. An **idle heartbeat** (`… still running
-  (Xs · N tools)` on stderr after ~30s of silence) keeps long 5–20 min runs legible; disable with
-  `COWORK_HARNESS_NO_HEARTBEAT`, tune with `COWORK_HARNESS_HEARTBEAT_MS`.
-- **File provision.** `--upload <file>` attaches a file at `mnt/uploads/<name>` (the "attach a file"
-  path skills like deck-review require) and `--project <dir>` connects a folder at `mnt/.projects/<id>`,
-  ad-hoc on the `skill` command (parity with the scenario `session.uploads`/`folders`).
-- **Resume-after-failure hardening.** Ephemeral Docker resources (egress networks/proxy + the host-loop
-  container) are named by a unique per-invocation token (not the session id), and the agent container is
-  reaped on teardown — so a `--resume` after a failed/interrupted run no longer collides with a leftover
-  orphaned container/network.
-- **Session persistence & resume.** `--session-id <id>` pins a stable run dir + the agent's native
-  session UUID (persisted in a `session.json` manifest); `--resume` reuses that work dir — preserving
-  `mnt/.claude/projects/<uuid>.jsonl`, `gate_state.json`, and `mnt/outputs` — and passes the agent's
-  own `--resume` so it reloads the conversation. This is how checkpoint-and-resume skills (a gate that
-  writes state, ends, and is re-invoked later with the prior RUN_ID) are tested. The harness leans on
-  the agent's native resume rather than reimplementing it (binary-verified).
-- **Interactive `chat`** — multi-turn REPL keeping the full harness (egress sandbox + control protocol);
-  `chat --raw` drops to the agent's native interactive cowork mode via `docker run -it`.
-- **Cassettes + full-fidelity replay.** `record` saves a control-protocol cassette; `replay --cassette`
-  plays it back deterministically (no token, no Docker) and re-evaluates content assertions.
-  - *The cassette captures both protocol directions:* `events` (child→driver, the assistant turn stream)
-    AND `controlOut` (driver→child decision responses). Both are recorded; `replay` now **consumes** both.
-  - *Full-fidelity replay (C1 false-green fix).* Consuming `controlOut` re-runs the decision pipeline on
-    replay, populating `rec.questions`/`rec.gateAnswers`/`rec.gateDeliveries`. Previously,
-    `question_asked` silently false-failed (questions invisible), `questions_count_max` passed vacuously
-    (0 ≤ max), and `gate_answers_delivered: true` passed vacuously (no deliveries recorded) — a silent
-    false-green violating the project's core principle. All three now genuinely evaluate when `controlOut`
-    is present.
-  - *The O7 guard on the token-free lane (`replay_protocol_fidelity`).* `replay` re-serializes each
-    decision response via `serializeDecision` and compares to the frozen `controlOut` envelope. A mismatch
-    (e.g. `serializeDecision` dropping `questions` from the AskUserQuestion `updatedInput`) appends a
-    `{ assertion: { replay_protocol_fidelity: true }, pass: false, message }` entry and exits 1 — catching
-    the O7 answer-shape regression without a live model.
-  - *Backward compatibility.* Old cassettes without `controlOut` get a loud `::warning::` on stderr;
-    `question_asked`, `questions_count_max`, and `gate_answers_delivered` are excluded from evaluation
-    (not vacuously passed). Re-record to enable full-fidelity mode.
-  - *Committed synthetic fixture + CI replay gate.* `examples/replays/example-pdf-skill.cassette.json`
-    is a hand-authored fixture (permission gate + AskUserQuestion gate + `tool_result`) committed to the
-    repo and replayed in the token-free CI job — dogfooding the documented PR-gate pattern and pinning
-    the fixture against `parseMessage`/assertion/`Run` regressions on every push.
-- **pytest `cowork` lane** (`python/`) — `@pytest.mark.cowork` + a `cowork` fixture over the
-  `--output-format json` surface, selectable with `-m cowork` beside your fast tests.
-- **Faithful sub-agent aggregation.** Recognizes the real cowork dispatch tool — **`Agent`**
-  (`{description, subagent_type, prompt}`; binary-verified primary name, with `Task` as its legacy
-  alias) and any tool carrying `subagent_type` — so `subagents[]` and the `subagent_dispatched` /
-  `dispatch_count_max` / `subagent_tool_*` assertions fire under `--fidelity cowork`. The cowork
-  `TaskCreate`/`TaskUpdate` **todo list** and `Monitor` are correctly excluded (no over-counting). Each
-  dispatch also captures its **`description`** — so a dispatch the skill made with no `subagent_type`
-  (`agentType:"unknown"`) is still self-explaining in `trace` and assertable: `subagent_dispatched`
-  matches the agentType **OR** the description.
-- **`trace <run-id | dir | events.jsonl> [--tools]`** — digests a run's `events.jsonl` into tool calls,
-  sub-agent dispatches (deduped), decisions, and questions (reuses the live `parseMessage` so it tracks
-  the schema); `--output-format json` for structured rows. Plus `result.json`/the json envelope now expose
-  `workDir`/`outputsDir`, and `--keep` prints the deep `mnt/outputs` deliverable path.
-- **Per-run artifacts** under `runs/<scenario>/<id>/`: `events.jsonl` + `control-out.jsonl` (the cassette
-  source), `run.jsonl` (harness-observability log), `trace.json` (structured trace), `egress.log`,
-  `result.json`, `agent.stderr.log`. Injected secrets (OAuth token / API key) are scrubbed from every
-  persisted log by value.
-- **Auth via env or `.env`.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, resolved in
-  priority order: exported env > `--dotenv <path>` > `./.env` (cwd) > `<install>/.env` (package root),
-  so a run from any directory still finds the install's credentials. Host-side, gitignored, exported
-  vars win, never mounted into the sandbox; passed via argv/env, never written to a runtime path. (The
-  flag is `--dotenv`, not `--env-file` — Node reserves the latter.)
-- **Platform baselines** (`cowork-harness sync`) derive the release-specific facts (agent version, domain
-  allowlist, mount layout, the production GrowthBook gate states) from your installed Claude Desktop, so
-  the code rides the stable protocol while data tracks each release. `--diff` previews changes; an
-  `asarFingerprint` tripwire flags unrecognized deltas.
-- **Sandbox self-verification** — `cowork-harness boundary-check` proves the sandbox enforces Cowork's
-  limitations; `cowork-harness vm <init|status|delete|prune>` manages the L2 microVM (`prune` drops orphaned VMs).
-- **CI** (`.github/workflows/ci.yml`): typecheck · format · unit + golden snapshots · build · boundary
-  parity (Docker) · live-contract guards · scenario suite (gated on a key) · the pytest lane.
+- **AskUserQuestion answer shapes.** `multiSelect` gates (answer with a list of labels → the verified
+  comma-joined wire shape); free-text **"Other"** via `answer:` (distinct from the label-validated
+  `choose:`); `choose` tolerates the `(Recommended)` suffix + `recommended`/`first` keywords. A partial
+  match on a batched gate now **names the unmatched sub-questions**.
+- **`artifact_json` assertion** — assert a JSON artifact's contents via a dotted path
+  (`equals`/`gt`/`exists`/`absent`/`is_null`); `absent`, `is_null`, and an unresolved intermediate are
+  distinct (the last fails loud, never a vacuous pass).
+- **Artifact manifest in cassettes** — `record` snapshots `outputs/`/`.projects/` (paths + hashes + small
+  JSON bodies) so `file_exists`/`user_visible_artifact`/`artifact_json` run on token-free `replay`. A
+  cassette→skill/baseline **staleness fingerprint** warns on drift; `replay --strict` fails on it. Cassettes
+  now carry a `cassetteVersion` (forward-compat guard).
+- **`RunResult.artifacts`** (ENV-MANIFEST) — observed user-visible files (path + bytes); also surfaced as
+  `Result.artifacts` in the Python helper.
+- **`allow_permissive_auto_allow` assertion + `RunResult.scan`** — a security-scan surface for the
+  Cowork-parity verdict (below); the assertion opts a scenario into a permissive auto-allow on purpose.
+- **CLI:** `trace --dispatches` (sub-agent dispatch tree + real total), `assert --list` (schema-generated),
+  `scaffold --from-run <id>` (kept run → starter scenario YAML).
+- **Python:** `run_scenario()` — run an authored scenario YAML and get the typed `Result`.
+
+### Changed
+
+- **Single verdict source (`computeVerdict()`)** wired into all five pass/fail sites (run/skill exit, footer,
+  replay exit, JSON-envelope `ok`) plus the Python `assert_success`. A Cowork-parity violation — a permissive
+  auto-allow, a recorded `outputs/` delete, or a host-path leak — now **default-fails** the run unless the
+  scenario explicitly asserts about it.
+- **Single fail-loud staging policy (`src/staging/resolve.ts`)** for every declared input (marketplace
+  manifest, enabled-plugin resolution, local skills, `mcp.config`, uploads, folders), with a Docker-safe
+  marketplace charset.
+- The run root honors `COWORK_HARNESS_RUNS_DIR`.
+
+### Fixed
+
+- **Egress / runtime hardening:** per-hop redirect egress logging, allowlist validation, a per-run proxy
+  port, proxy/sidecar readiness handshakes, fail-loud Lima provisioning, and boundary teardown in
+  `try/finally`.
+- **Protocol / decider hardening:** oversized control-frame hard-fail, a nonzero child-exit error event,
+  provenance untruncation, TTY-elicit cancel, and a JSON-safe `reply_with` key.
+- **Detection / packaging:** `%2F`/backslash decode in the outputs-delete detector; the npm package now
+  ships `schema/`, `docs/`, `python/`, and `scripts/`; assertion path containment; resume empty-tree warning.
 
 ### Notes
 
-- **Two concepts, two names (no "profile" overload).** The synced per-release snapshot is the **platform
-  baseline** (`baselines/desktop-*.json`, scenario field `baseline:`, type `PlatformBaseline`); the
-  hand-authored pre-prompt config is the **session** (`sessions/*.yaml`, scenario field `session:`, type
-  `SessionConfig`). For back-compat, a scenario's deprecated `profile:` key is still accepted for one minor
-  (mapped to `baseline:` with a stderr deprecation warning) and `Profile` is re-exported as an alias of
-  `PlatformBaseline`; both are removed next minor. The `--output-format json` `RunResult` field is `baseline`
-  (was `profile`).
-- The agent binary is **bind-mounted from your own Claude Desktop install** (`claude-code-vm/<ver>/claude`)
-  or `COWORK_AGENT_BINARY` — nothing Anthropic-owned is bundled or distributed. There is no npm path.
-- Cowork mode is enabled by `CLAUDE_CODE_IS_COWORK=1` (env), **not** a `--cowork` flag.
-- This is a fixture for testing, **not a security boundary** — see [SECURITY.md](./SECURITY.md) and the
-  fidelity caveats in [DESIGN.md](./DESIGN.md).
+- Held/deferred per the plan's gating: composed partial-gate answering, `decider_intent:` in scenario YAML,
+  a whole-gate `response:` freeform, and `artifacts_share_field`. All additive/opt-in when built.
+
+## [0.1.1] — 2026-06-16
+
+Docs, distribution, and packaging. No CLI behavior change.
+
+### Added
+
+- **Companion Claude Code skill, installable.** A `.claude-plugin/marketplace.json` + skills-directory
+  plugin make the bundled skill installable via `/plugin marketplace add yaniv-golan/cowork-harness`;
+  the skill self-bootstraps the CLI (`npx cowork-harness@latest`) and fails loud on missing tier deps.
+- **`AGENTS.md`** — canonical, cross-tool agent instructions — and **`llms.txt`** doc index.
+- **JSON Schema for scenario & session YAML** (`schema/*.schema.json`, generated via `npm run schema`,
+  pinned by a token-free drift-guard); `# yaml-language-server: $schema=` hints in the example scenarios.
+- README banner, badges, an "For AI agents" section, and `npm install` instructions.
+
+### Changed
+
+- Release pipeline publishes via npm **Trusted Publishing (OIDC)** with provenance (no stored token).
+- GitHub Actions bumped off the deprecated Node 20 runtime; CI live-scenario job skips cleanly without a key.
+
+## [0.1.0] — 2026-06-16
+
+Initial public release. A faithful, headless, scriptable harness for Claude Cowork's runtime — for
+testing Claude Code **skills** outside the Desktop app with the same staged agent, spawn/control-protocol
+contract, egress allowlist, permission protocol, and sandbox limitations. Binary-grounded against
+`app.asar` 1.12603.1 / agent ELF 2.1.170.
+
+### Added
+
+- Commands: `skill`, `run`, `chat`, `record`, `replay`, `trace`, and `decide`, plus `sync`,
+  `boundary-check`, and `vm` management. Stable `--output-format json` envelope and CI-ready exit codes.
+- Five fidelity tiers (`fidelity:`): `protocol`, `container`, `microvm`, `hostloop`, and `cowork`
+  (auto-picks host-loop vs container the way Cowork does).
+- Scenario YAML — prompt + scripted answers + `assert:` (transcript, files, artifacts, tool / sub-agent
+  usage, egress, and more) for authored, asserted regression runs.
+- Input policy with no silent false-greens: scripted, LLM, and in-band (`--decider-dir`) answering for
+  AskUserQuestion / tool-permission gates; an unanswered gate fails loud.
+- Default-deny egress sandbox enforced against the synced Cowork domain allowlist.
+- Token-free, Docker-free cassette `record` / `replay` for the PR gate.
+- Platform baselines synced from a local Claude Desktop install — nothing Anthropic-owned is bundled
+  or distributed.

package/README.md

@@ -1,8 +1,15 @@
+<p align="center">
+  <img src="docs/assets/banner.png" alt="cowork-harness — headless, scriptable, CI-ready test harness for Claude Cowork skills" width="100%">
+</p>
+
 # cowork-harness
 
 [![ci](https://github.com/yaniv-golan/cowork-harness/actions/workflows/ci.yml/badge.svg)](https://github.com/yaniv-golan/cowork-harness/actions/workflows/ci.yml)
 [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
 [![node: >=20](https://img.shields.io/badge/node-%3E%3D20-339933.svg)](#quick-start)
+[![Claude Code plugin](https://img.shields.io/badge/Claude_Code-plugin-F97316)](#drive-it-from-claude-code-companion-skill)
+[![Built with Skill Creator Plus](https://img.shields.io/badge/Built_with-Skill_Creator_Plus-4ecdc4)](https://github.com/yaniv-golan/skill-creator-plus)
+[![Agent Skills compatible](https://img.shields.io/badge/Agent_Skills-compatible-4A90D9)](https://agentskills.io)
 
 Scriptable, CI-friendly test harness that reproduces **Claude Cowork's observable runtime contract** closely enough to test the skills you write — across many scenarios, headless, in CI — without the (locked) Desktop app. It reproduces not just Cowork's *behavior* but its *limitations*: sealed filesystem, default-deny egress, MCP-only cross-boundary — so a green test means green in real Cowork.
 
@@ -67,8 +74,10 @@ Skill testing is the headline use, but the tool is a general harness over the Co
 | `skill <folder> "<prompt>"` | Run a local skill/plugin folder once against the staged agent | ad-hoc "is the skill alive / does it do X?" — the fast inner loop |
 | `run <scenario.yaml \| dir/>` | Run authored scenarios with `assert:` + a CI-ready exit code | you want a repeatable, **asserted regression test** |
 | `chat <folder>` | Interactive multi-turn REPL against a skill (TTY) | debugging a multi-turn flow by hand |
-| `record` / `replay` | Save a control-protocol cassette, then replay it deterministically | **token-free, Docker-free CI** from a once-recorded run |
-| `trace <run-id>` | Digest a run's `events.jsonl` (tools, sub-agent dispatches, decisions) | "how many sub-agents *actually* dispatched, and which?" |
+| `record` / `replay` | Save a control-protocol cassette, then replay it deterministically (`replay --strict` fails on a stale cassette) | **token-free, Docker-free CI** from a once-recorded run |
+| `trace <run-id>` | Digest a run's `events.jsonl` (`--tools`, `--gates`, `--dispatches` for the sub-agent dispatch tree + total) | "how many sub-agents *actually* dispatched, and which?" |
+| `scaffold --from-run <id>` | Turn a kept run into a starter scenario YAML (gates→answers, artifacts→`file_exists`) | authoring a scenario from a real run instead of guessing |
+| `assert --list` | List the available scenario assertions (generated from the schema) | "what can I assert?" without grepping the source |
 | `decide` | Validate a decider against a sample question in ~2 s (no run) | sanity-check a `--decider-*` / `--answer` wiring before a long run |
 | `gates` / `answer` | Stream / answer in-band gates for `--decider-dir` | a **driving agent** answers live questions via a Monitor |
 | `boundary-check [baseline]` | Prove the sandbox enforces Cowork's limitations | verifying the harness's own fidelity |
@@ -141,7 +150,13 @@ It mounts the folder(s) at the Cowork plugin path, runs the staged agent in cowo
 
 ## Quick start
 
-**Install from source** (not yet published to npm):
+**Install from npm:**
+
+```bash
+npm install -g cowork-harness    # puts the `cowork-harness` command on your PATH
+```
+
+**Or build from source:**
 
 ```bash
 git clone https://github.com/yaniv-golan/cowork-harness && cd cowork-harness
@@ -149,8 +164,18 @@ npm install && npm run build && npm link    # puts the `cowork-harness` command
 # …or skip the link and call it directly: node dist/cli.js <cmd>
 ```
 
-> A global `npm install -g cowork-harness` will work once the package is published; for now, build from source.
-> (Heads-up: the repo folder is `claude-cowork-headless-emulator`, the package/CLI is `cowork-harness`, and the GitHub repo is `yaniv-golan/cowork-harness`.)
+### Drive it from Claude Code (companion skill)
+
+This repo ships a **companion skill** (`.claude/skills/cowork-harness/`) that teaches an agent how to drive the harness — author scenarios, pick a fidelity tier, script answers, place assertions in the right CI lane, and avoid the "✓ passed ≠ correct" traps. Install it into Claude Code via the bundled marketplace:
+
+```bash
+/plugin marketplace add yaniv-golan/cowork-harness
+/plugin install cowork-harness@cowork-harness
+```
+
+The skill **self-bootstraps the CLI**: if `cowork-harness` isn't on your PATH it falls back to `npx cowork-harness@>=0.2.0` (a version floor that fails loud rather than silently fetching a too-old CLI; Node ≥ 20). Tiers above `protocol` still need Docker/Lima and a Claude Desktop agent binary — see the prerequisites below.
+
+It also follows the open [Agent Skills](https://github.com/vercel-labs/skills) spec, so it installs cross-editor (Cursor, Codex, OpenCode, …) by pointing the `npx skills` CLI at `.claude/skills/cowork-harness` in this repo. (Working *inside* this repo, the skill auto-loads as a project skill — no install needed.)
 
 **Prerequisites for anything above `protocol` fidelity** (the `protocol` tier needs none of these — it's pure logic iteration):
 1. **Claude Desktop, opened once.** The Cowork agent binary is **bind-mounted from your own install** at run time — nothing Anthropic-owned is bundled. Open Cowork once so the agent ELF is staged (`…/claude-code-vm/<ver>/claude`); the harness auto-detects it, or set `COWORK_AGENT_BINARY=<path>` to point at it. Without a staged agent, container/cowork runs fail with "Open Cowork once to stage it…".
@@ -309,32 +334,26 @@ Secrets (the injected OAuth token / API key) are scrubbed from every persisted l
 ## Architecture
 
 ```
-                       ┌──────────────────────────────────────────────┐
-  scenario.yaml ─────► │  cowork-harness (TS CLI)                     │
-                       │                                              │
-                       │  baseline loader ── baselines/desktop-*.json   │  ◄── cowork-sync
-                       │                       (agent ver, mounts,    │      (reads live
-                       │                        egress allowlist)     │       Desktop install
-                       │                                              │       + app.asar)
-                       │  scenario → runtime selector (L0/L1/L2)      │
-                       └───────────────┬──────────────────────────────┘
-                                       │ spawns + speaks stream-json
-                       ┌───────────────▼──────────────────────────────┐
-                       │  Agent: claude -p  (CLAUDE_CODE_IS_COWORK=1)  │
-                       │    --input-format stream-json                 │
-                       │    --output-format stream-json                │
-                       │                                              │
-                       │  cwd = /sessions/<id>/mnt                     │
-                       │  mnt/uploads, mnt/.projects/*, plugin mounts  │
-                       └───────────────┬──────────────────────────────┘
-         decision control req (tool/  │   egress
-         question/dialog/elicit)      │
-                ┌─────────────────────▼──────────────┐   ┌───────────────────────┐
-                │  AgentSession → Decider → Run       │   │  Egress proxy         │
-                │  (protocol seam · policy seam ·     │   │  default-deny,        │
-                │   turn loop + RunRecord)            │   │  allowlist = synced   │
-                └─────────────────────────────────────┘   │  vmAllowedDomains()   │
-                                                           └───────────────────────┘
+                      ┌────────────────────────────────────────────────┐
+  scenario.yaml ────► │  cowork-harness  (TypeScript CLI)              │
+                      │    baseline loader ◄── baselines/desktop-*.json│
+                      │    runtime selector  ──►  L0 / L1 / L2         │
+                      └───────────────────────┬────────────────────────┘
+                                              │  spawns + speaks stream-json
+                      ┌───────────────────────▼────────────────────────┐
+                      │  Agent:  claude -p   (CLAUDE_CODE_IS_COWORK=1) │
+                      │    --input-format / --output-format stream-json│
+                      │    cwd = /sessions/<id>/mnt                    │
+                      │    mnt/uploads · mnt/.projects/* · plugins     │
+                      └───────────────────────┬────────────────────────┘
+            decision control request          │  outbound network (egress)
+            (tool · question · dialog)        │  default-deny → allowlist
+                      ┌───────────────────────▼────────────┐    ┌────────────────────────┐
+                      │  AgentSession ──► Decider ──► Run  │    │  Egress proxy          │
+                      │  protocol · policy · turn loop     │    │  default-deny;         │
+                      │  + RunRecord                       │    │  allowlist = synced    │
+                      │                                    │    │  vmAllowedDomains()    │
+                      └────────────────────────────────────┘    └────────────────────────┘
 ```
 
 - **AgentSession** speaks the Agent SDK control protocol over stream-json, emitting a typed event
@@ -424,7 +443,7 @@ The diff shows exactly what moved (agent bump, allowlist change, new mount). You
 
 ---
 
-## Honest limitations
+## Limitations
 
 - **Not the full Desktop network transport.** L1 is a container, not a VM; L2 *is* a real Apple-VZ microVM but still does not reproduce Cowork's gVisor netstack — its egress is the same allowlist proxy as L1 (with a guest iptables firewall in front). If your skill depends on VM-kernel specifics, validate at L2; if it depends on packet-level gVisor behavior, no tier reproduces it.
 - **Cowork in-guest context is partial.** Desktop supplies host-loop staging, runtime `mountPath` RPC, and the bridge. We reproduce the *filesystem and cowork mode*, not those host-side services. Skills that call Desktop-only host RPCs won't run here (they wouldn't be portable anyway).
@@ -435,6 +454,17 @@ These are documented per-tier in [DESIGN.md](./DESIGN.md) so a green test means
 
 ---
 
+## For AI agents
+
+This repo is built to be driven by agents, not just read by humans:
+
+- **[AGENTS.md](./AGENTS.md)** — the canonical agent-instructions file (architecture seams, the build gate, invariants, ethos). Read it before changing code. Also indexed in **[llms.txt](./llms.txt)**.
+- **Companion skill** — [`.claude/skills/cowork-harness/`](./.claude/skills/cowork-harness/SKILL.md) teaches an agent to drive the harness; install it via the marketplace (see [above](#drive-it-from-claude-code-companion-skill)).
+- **Machine-readable interfaces** — stable `--output-format json` envelope on stdout, deterministic exit codes (`0`/`1`/`2`), and `--help` on every command.
+- **JSON Schemas** — [`schema/scenario.schema.json`](./schema/scenario.schema.json) and [`schema/session.schema.json`](./schema/session.schema.json) describe every field of the YAML you author (generated from the source schemas; `npm run schema`).
+
+---
+
 ## Documentation
 
 | Doc | Read it for |

package/dist/agent/session.js

@@ -1,3 +1,4 @@
+import { warn } from "../io.js";
 import { createWriteStream } from "node:fs";
 import { join } from "node:path";
 import readline from "node:readline";
@@ -159,6 +160,8 @@ export class LiveAgentSession {
     /** Reject function set when proc emits an error — bridges the callback into the async generator.
      *  Set before the generator loop starts; called at most once (the Promise settles once). */
     rejectError;
+    /** Bounded tail of the child's stderr, for the nonzero-exit error message. */
+    stderrTail = "";
     constructor(proc, outDir) {
         this.proc = proc;
         this.outDir = outDir;
@@ -166,6 +169,11 @@ export class LiveAgentSession {
         this.controlOut = createWriteStream(join(outDir, "control-out.jsonl"), { flags: "a" });
         const errLog = createWriteStream(join(outDir, "agent.stderr.log"), { flags: "a" });
         this.proc.stderr.pipe(errLog);
+        // keep a bounded stderr tail and capture the exit code/signal so a child that dies nonzero
+        // (with no structured {type:"result"} error) is surfaced as a typed error event, not a silent stop.
+        this.proc.stderr.on("data", (d) => {
+            this.stderrTail = (this.stderrTail + d.toString()).slice(-2000);
+        });
         // #15: attach stdin error listener once at construction so dead-child writes don't produce
         // unhandled process errors. Routes to the same error path as spawn errors when possible.
         this.proc.stdin.on("error", (e) => {
@@ -241,6 +249,20 @@ export class LiveAgentSession {
                 }
                 yield* this.translate(msg);
             }
+            // stdout closed. Give a pending 'exit' one tick to land (NOT a blocking wait on 'close' — a
+            // mock/fake child may never emit it), then surface a nonzero/signal exit as a typed error — a
+            // crashed child that emitted no {type:"result"} error line would otherwise be a silent stop.
+            await new Promise((res) => setImmediate(res));
+            const code = this.proc.exitCode;
+            const signal = this.proc.signalCode;
+            if (signal || (code !== null && code !== 0)) {
+                const tail = this.stderrTail.trim();
+                yield {
+                    type: "error",
+                    source: "exit",
+                    message: `agent process exited ${signal ? `on signal ${signal}` : `with code ${code}`}${tail ? ` — stderr tail: ${tail}` : ""}`,
+                };
+            }
         }
         finally {
             this.rejectError = undefined; // generator is done; stop routing errors here
@@ -274,7 +296,7 @@ export class LiveAgentSession {
                     // reply — an unanswered mcp_message blocks the in-VM agent on the round-trip forever (deadlock).
                     // Reply with a JSON-RPC error instead, mirroring the no-handler defense below.
                     const message = e?.message ?? String(e);
-                    process.stderr.write(`::warning:: sdkMcp.handle threw for "${server}" — replying with a JSON-RPC error: ${message}\n`);
+                    warn(`::warning:: sdkMcp.handle threw for "${server}" — replying with a JSON-RPC error: ${message}\n`);
                     out = { error: { code: -32603, message: `handler error: ${message}` } };
                 }
                 this.write(mcpResponseEnvelope(msg.request_id, out, jr.id));
@@ -294,7 +316,7 @@ export class LiveAgentSession {
             // #10: an mcp_message arrived but no sdkMcp handler is configured. Reply with a JSON-RPC error
             // (well-formed via mcpResponseEnvelope) instead of silently dropping it — a dropped request
             // leaves the in-VM agent waiting on the round-trip forever (protocol deadlock in host-loop mode).
-            process.stderr.write(`::warning:: mcp_message for server "${server}" arrived but no sdkMcp handler is configured — replying with a JSON-RPC error (would otherwise deadlock)\n`);
+            warn(`::warning:: mcp_message for server "${server}" arrived but no sdkMcp handler is configured — replying with a JSON-RPC error (would otherwise deadlock)\n`);
             this.write(mcpResponseEnvelope(msg.request_id, { error: { code: -32601, message: "no sdkMcp handler configured" } }, jr.id));
             return;
         }
@@ -312,7 +334,7 @@ export class LiveAgentSession {
         if (!req) {
             // #13: an id with no matching request_id is a protocol drift. Writing a guessed envelope would
             // be worse, but a silent return leaves the agent blocked until timeout (looks like a hang).
-            process.stderr.write(`::warning:: respond() for unknown decision id "${decisionId}" — no matching request_id was seen; the agent may block until timeout (protocol drift)\n`);
+            warn(`::warning:: respond() for unknown decision id "${decisionId}" — no matching request_id was seen; the agent may block until timeout (protocol drift)\n`);
             return;
         }
         // #14: serializeDecision returns a safe deny envelope on a kind mismatch (defense in depth). That
@@ -320,7 +342,7 @@ export class LiveAgentSession {
         // while the agent actually received a deny. (serializeDecision stays a pure declared inverse of
         // deserializeDecision; the warning lives here in the caller, not in the pure function.)
         if (req.kind !== r.kind)
-            process.stderr.write(`::warning:: decider returned kind "${r.kind}" for a "${req.kind}" request (id ${decisionId}) → sending a safe deny/cancel; the agent did NOT receive an answer\n`);
+            warn(`::warning:: decider returned kind "${r.kind}" for a "${req.kind}" request (id ${decisionId}) → sending a safe deny/cancel; the agent did NOT receive an answer\n`);
         this.write(serializeDecision(req, r));
     }
     close() {
@@ -333,13 +355,12 @@ export class LiveAgentSession {
     }
     write(obj) {
         const line = JSON.stringify(obj);
-        // #54: the control protocol writes small single-line JSON frames, so stdin backpressure
-        // effectively never engages; we intentionally ignore the write() return / drain here. If frame
-        // sizes ever grow, revisit with a drain-aware queue (which would make write()/respond() async).
-        // #47: guard that assumption — a frame past the threshold warns loudly so the "revisit" trigger
-        // fires instead of silently risking partial buffering on a frame far larger than expected.
+        // The control protocol writes small single-line JSON frames, so stdin backpressure effectively never
+        // engages; we ignore the write() return / drain here. A frame past the safe threshold is anomalous —
+        // hard-FAIL rather than risk a partially-buffered write that silently corrupts the protocol stream.
+        // (If large control frames ever become legitimate, switch to a drain-aware queue, making writes async.)
         if (line.length > 256 * 1024)
-            process.stderr.write(`::warning:: control frame is ${line.length} bytes (> 256 KiB) — stdin backpressure may engage; revisit write() with a drain-aware queue\n`);
+            throw new Error(`control frame is ${line.length} bytes (> 256 KiB safe limit) — refusing to write to avoid partial stdin buffering; this indicates an unexpectedly large control payload`);
         this.controlOut.write(line + "\n");
         this.proc.stdin.write(line + "\n");
     }
@@ -393,6 +414,7 @@ export function parseMessage(msg) {
                         ev.push({
                             type: "subagent_dispatch",
                             toolUseId: String(block.id ?? ""),
+                            parentToolUseId,
                             // Skills often dispatch with only {description, prompt} (no subagent_type) → agentType is
                             // "unknown" but the description still identifies the dispatch (e.g. "TOP_DOWN market sizing").
                             agentType: String(inp.subagent_type ?? inp.subagentType ?? "unknown"),
@@ -415,6 +437,7 @@ export function parseMessage(msg) {
                         toolUseId: block.tool_use_id ? String(block.tool_use_id) : undefined,
                         isError: !!block.is_error,
                         text: toolResultText(block.content),
+                        provenanceText: toolResultRaw(block.content),
                     });
             }
             break;
@@ -424,17 +447,27 @@ export function parseMessage(msg) {
     }
     return ev;
 }
-/** Flatten a tool_result `content` (a string, or an array of `{type:"text",text}` blocks) to a short string. */
-function toolResultText(content) {
+/** Flatten a tool_result `content` (a string, or an array of `{type:"text",text}` blocks), capped at
+ *  `max` chars. The 500-char DISPLAY value (toolResultText) keeps the recorder/trace compact; the larger
+ *  PROVENANCE value (toolResultRaw) is what seeds web_fetch provenance, so a URL past char 500 isn't lost. */
+function flattenToolResult(content, max) {
     if (typeof content === "string")
-        return content.slice(0, 500);
+        return content.slice(0, max);
     if (Array.isArray(content))
         return content
             .map((b) => (b && typeof b === "object" && "text" in b ? String(b.text) : ""))
             .join(" ")
-            .slice(0, 500);
+            .slice(0, max);
     return "";
 }
+function toolResultText(content) {
+    return flattenToolResult(content, 500);
+}
+/** Larger cap for provenance (URL extraction) — matches the web_fetch body cap so any URL the agent
+ *  could realistically act on is seeded; still bounded so a pathological result can't blow up memory. */
+function toolResultRaw(content) {
+    return flattenToolResult(content, 200_000);
+}
 export function toDecisionRequest(msg) {
     const sub = msg.request?.subtype;
     const id = msg.request_id;