cowork-harness 0.22.0 → 0.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/cowork-harness/SKILL.md +36 -24
- package/.claude/skills/cowork-harness/references/ci-recipe.md +7 -6
- package/.claude/skills/cowork-harness/references/fidelity-and-answers.md +4 -3
- package/.claude/skills/cowork-harness/references/scenario-schema.md +15 -5
- package/.claude/skills/cowork-harness/scripts/scenario.py +16 -6
- package/AGENTS.md +2 -2
- package/CHANGELOG.md +214 -4
- package/DESIGN.md +2 -2
- package/README.md +32 -17
- package/SPEC.md +62 -16
- package/baselines/desktop-1.18286.0.json +41 -5
- package/dist/cli-args.js +1 -1
- package/dist/cli.js +66 -37
- package/dist/decide/llm-transport.js +19 -4
- package/dist/errors.js +12 -0
- package/dist/loop-decision.js +0 -3
- package/dist/run/cassette.js +114 -58
- package/dist/run/chat.js +13 -3
- package/dist/run/display-translate.js +180 -1
- package/dist/run/doctor.js +25 -1
- package/dist/run/execute.js +29 -5
- package/dist/run/renderer.js +8 -3
- package/dist/run/trace-view.js +17 -7
- package/dist/run/verdict.js +8 -6
- package/dist/run/vm-path-ctx-file.js +123 -0
- package/dist/runtime/argv.js +13 -0
- package/dist/runtime/hostloop.js +3 -0
- package/dist/scan.js +31 -9
- package/dist/sync/cowork-sync.js +544 -10
- package/dist/types.js +2 -20
- package/dist/vm-paths.js +8 -4
- package/docs/README.md +1 -1
- package/docs/cassette.md +31 -23
- package/docs/chat.md +4 -3
- package/docs/cowork-spawn-contract-1.12603.1.md +1 -1
- package/docs/debugging.md +10 -0
- package/docs/discovery.md +6 -0
- package/docs/fidelity-gaps.md +46 -0
- package/docs/invariants.md +1 -1
- package/docs/maintenance.md +1 -1
- package/docs/run-status.md +33 -0
- package/docs/scenario.md +13 -9
- package/docs/session.md +2 -0
- package/examples/replays/README.md +1 -1
- package/examples/replays/hostloop-computer-links.cassette.json +77 -0
- package/llms.txt +1 -1
- package/package.json +2 -2
- package/python/test_scenario_lint.py +21 -0
- package/schema/scenario.schema.json +1 -1
- package/scripts/check-versions.ts +80 -3
- package/scripts/gen-schema.ts +4 -3
|
@@ -3,8 +3,8 @@ name: cowork-harness
|
|
|
3
3
|
description: Test or debug a Claude Code skill/plugin under Claude Cowork's runtime — sandboxed agent, default-deny egress, the can_use_tool permission/question protocol — using the cowork-harness CLI. Use when validating or regression-testing a skill, authoring or debugging a scenario YAML (prompt + scripted answers + assert:), choosing a fidelity tier, scripting AskUserQuestion / tool-permission answers, or asserting artifacts, egress, or sub-agent dispatch. Especially when a harness run no-ops an assertion, fails on an unanswered gate, false-greens, a steered answer never reaches the model, or a web_fetch is unexpectedly denied or gated. NOT for generic unit testing (pytest/vitest of your own scripts) or non-Cowork CI. Covers the skill / run / chat / record / replay / trace / decide / assertions / scaffold commands and the session-vs-scenario split.
|
|
4
4
|
metadata:
|
|
5
5
|
author: cowork-harness
|
|
6
|
-
version: 0.
|
|
7
|
-
tracks-harness: cowork-harness 0.
|
|
6
|
+
version: 0.23.0
|
|
7
|
+
tracks-harness: cowork-harness 0.23.0 (baseline desktop-1.18286.0)
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# cowork-harness
|
|
@@ -22,7 +22,7 @@ flagged with a loud `::warning::`, not silent — auto-answer a gate, observe an
|
|
|
22
22
|
allowlist). This skill exists mostly to keep you out of those traps — the Gotchas section below is
|
|
23
23
|
the highest-value part. Read it.
|
|
24
24
|
|
|
25
|
-
> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.
|
|
25
|
+
> **Version note:** the facts and `file:line` pointers here track `cowork-harness 0.23.0` (baseline
|
|
26
26
|
> `desktop-1.18286.0`). If your checkout is newer, prefer the live `--help`, `SPEC.md`, and
|
|
27
27
|
> `docs/*.md` over this snapshot, and re-run the bundled linter.
|
|
28
28
|
|
|
@@ -38,7 +38,7 @@ cowork-harness skill ./my-skill "do X" # run the skill once against the sta
|
|
|
38
38
|
Before the first command, confirm the CLI is reachable and **fail loud** (never fake a pass) when a tier's dependencies are missing:
|
|
39
39
|
|
|
40
40
|
- **One-shot check.** Run `cowork-harness doctor [--tier <tier>]` first — a read-only prerequisite check that inspects Docker, the staged agent, the token, and the baseline in one pass. The bullets below explain each thing it checks (and how to fix it).
|
|
41
|
-
- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.
|
|
41
|
+
- **CLI on PATH, recent enough?** Run `cowork-harness --version` — this skill needs **≥ 0.23.0**. If it's missing or older, prefix every command with the version floor `npx "cowork-harness@>=0.23.0" <cmd>` (Node ≥ 20), or install once with `npm i -g "cowork-harness@>=0.23.0"`. **Pin `@>=0.23.0`, never `@latest`** — `@latest` can silently fetch an older CLI and the new commands fail as "unknown command", whereas the floor **fails loud** if no compatible version is published. (≥ 0.23.0 is what gates the commands/assertions this skill teaches: `assertions --list`, `scaffold <run-id>`, `trace --view dispatches`, `artifact_json` incl. the `in:` operator, `verify-cassettes`, batch `record <dir>`/`--rerecord-stale`, `record --concurrency <N>`, record-time redaction, multiSelect/`answer:`, `verify-run` answer-coverage, `record --max-artifact-bytes`, live record-time deciders, `verify-cassettes --allow-domain`/`--allow-email`/`--allow-path`/`--allow-file` (`path` — local absolute filesystem paths — is the scanner's 4th class, new in 0.21.0), scenario `skills:` staleness scoping with `COWORK_HARNESS_AGENT_SCOPE=skill`, `chat --plugin`, `/help` in the REPL, `hostloop`'s native host/VM process split with its `allow_host_writes:` consent field, and `computer_links_resolve` (new in 0.22.0).)
|
|
42
42
|
- **Agent binary (sandboxed live tiers — `container`/`microvm`/`hostloop`/`cowork`).** The staged Claude Code agent is **bind-mounted** from a local Claude Desktop install, or point `COWORK_AGENT_BINARY` at a `claude-code-vm/<ver>/claude` ELF. Nothing is bundled. `protocol` (L0) and `replay` need no staged agent; for the sandboxed tiers, no agent → no run; report that, don't skip silently.
|
|
43
43
|
- **Docker / Lima.** Only `--fidelity protocol` (L0) runs without them. `container` / `microvm` / `hostloop` / `cowork` need Docker (Lima for L2). If they're absent, drop to `--fidelity protocol` and **say so** — a green that never exercised the sandbox is not a sandbox pass.
|
|
44
44
|
- **Auth.** `CLAUDE_CODE_OAUTH_TOKEN` (preferred) or `ANTHROPIC_API_KEY`, via env or `.env`. Minting an OAuth token needs the **`claude` CLI** (`npm i -g @anthropic-ai/claude-code`, then `claude setup-token`).
|
|
@@ -55,7 +55,12 @@ Before the first command, confirm the CLI is reachable and **fail loud** (never
|
|
|
55
55
|
|
|
56
56
|
Full command set: `skill · run · chat · record · replay · verify-cassettes · rehash · prune · lint ·
|
|
57
57
|
verify-run · trace · inspect · diff · stats · decide · gates · answer · scaffold · assertions --list · sync ·
|
|
58
|
-
list · boundary-check · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
|
|
58
|
+
list · boundary-check · status · vm <init|status|delete|prune> · doctor`. Always check `cowork-harness <cmd> --help`.
|
|
59
|
+
|
|
60
|
+
**Two different `scaffold` tools — don't confuse them.** The native `cowork-harness scaffold <run-id>`
|
|
61
|
+
above turns an already-*recorded* run into a scenario (needs a run to exist first). The bundled
|
|
62
|
+
`scripts/scenario.py scaffold --name … --skill …` in §9 builds a scenario from flags alone, no run
|
|
63
|
+
required. Passing §9's flag set to the native command fails with `unknown flag: --name` (exit 2).
|
|
59
64
|
|
|
60
65
|
## 2. Two files: session vs scenario
|
|
61
66
|
|
|
@@ -94,7 +99,8 @@ the folder basename (collision-resolved); there is no `to:` override. See `refer
|
|
|
94
99
|
| `hostloop` / `cowork` | Production split-exec: the agent loop is a **native process on the host** (no container around the file tools — matching production), with native Bash/WebFetch disabled and routed host-side via the workspace SDK-MCP server into a Docker VM sidecar | Highest-fidelity / parity runs. A writable connected folder needs `allow_host_writes: true` (see scenario-schema.md). |
|
|
95
100
|
|
|
96
101
|
Set the tier in the **scenario's `fidelity:` field**, not a flag — `run` rejects `--fidelity`
|
|
97
|
-
(it's a `skill
|
|
102
|
+
(it's a `skill`/`chat` flag; `run` takes fidelity only from the scenario). See
|
|
103
|
+
`references/fidelity-and-answers.md`.
|
|
98
104
|
|
|
99
105
|
## 5. Choose an answer path (gates: AskUserQuestion + tool-permission)
|
|
100
106
|
|
|
@@ -265,7 +271,7 @@ known-good skeleton (right tier, scripted `answers:` + `on_unanswered: fail`, co
|
|
|
265
271
|
separated from live-only ones, one concern per item) and **self-lints its own output**:
|
|
266
272
|
|
|
267
273
|
```bash
|
|
268
|
-
S="${CLAUDE_PLUGIN_ROOT}/
|
|
274
|
+
S="${CLAUDE_PLUGIN_ROOT}/scripts/scenario.py"
|
|
269
275
|
# Working from a repo checkout instead of an installed plugin?
|
|
270
276
|
# S=".claude/skills/cowork-harness/scripts/scenario.py"
|
|
271
277
|
python3 "$S" scaffold --name report-check --skill ./skills/report-gen \
|
|
@@ -392,23 +398,29 @@ are the ones that bite hardest.
|
|
|
392
398
|
prompt. *Fix:* treat `prompt_asset_missing` as a blocking error in CI by checking the signals
|
|
393
399
|
array.
|
|
394
400
|
13. **`result: success` means the agent didn't error, NOT that the task completed — always assert on
|
|
395
|
-
artifacts/content.**
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
|
|
409
|
-
|
|
410
|
-
|
|
411
|
-
|
|
401
|
+
artifacts/content.**
|
|
402
|
+
- A turn that ends on a plain-text re-ask ("which file did you mean?") still reports
|
|
403
|
+
`result: success`.
|
|
404
|
+
- The harness catches this with a **`stalled`** verdict signal: a run that ends on a question and
|
|
405
|
+
did **no productive work after its last gate** — both the no-gate case ("which file?" with no
|
|
406
|
+
tool calls) AND the *answered-gate-then-re-ask* case (the agent answers an `AskUserQuestion`,
|
|
407
|
+
then asks again in plain text and stops). Suppress with `allow_stall: true` if ending on a
|
|
408
|
+
question is intended.
|
|
409
|
+
- The signal is a **tool-position heuristic**, not deliverable detection, so it is imprecise both
|
|
410
|
+
ways:
|
|
411
|
+
- **False negative:** a post-gate tool *call* clears the flag whether it **succeeded or
|
|
412
|
+
errored** — an agent that ran a tool after the gate and still stalled is not caught.
|
|
413
|
+
- **False positive:** a deliverable written *before* a final confirmation gate does **not**
|
|
414
|
+
clear it, so a write-then-confirm-then-question run is flagged — use `allow_stall: true` for a
|
|
415
|
+
deliberate confirm-terminal skill.
|
|
416
|
+
- The broad guard is therefore YOUR assertions — assert the deliverable (`file_exists` /
|
|
417
|
+
`artifact_json` / `transcript_matches`), never just `result: success`.
|
|
418
|
+
- `on_unanswered` governs **unanswered** `AskUserQuestion` gates; the `stalled` signal covers
|
|
419
|
+
stalling *after* one is answered — two different failure modes.
|
|
420
|
+
- **Free-text aside:** a "type-it-in-notes" option has **no scripted deterministic answer** today
|
|
421
|
+
(the `OTHER:` directive works only on the LLM-decider path, not scripted `choose:`; on an
|
|
422
|
+
options-bearing gate a bare out-of-set LLM answer fails loud (exit 2) — see the LLM-decider
|
|
423
|
+
free-text note in `references/fidelity-and-answers.md`).
|
|
412
424
|
14. **A positional `choose` (`first` / index) is order-dependent.** `choose: "2"` survives label drift
|
|
413
425
|
but NOT option *re-ordering* — if the gate presents its options in a different order run-to-run, the
|
|
414
426
|
index lands on a different option (a silent re-record flake). Prefer an exact label when order is
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# CI recipe — replay vs live lanes
|
|
2
2
|
|
|
3
|
-
Self-contained reference. Tracks `cowork-harness 0.
|
|
3
|
+
Self-contained reference. Tracks `cowork-harness 0.23.0` (baseline `desktop-1.18286.0`).
|
|
4
4
|
|
|
5
5
|
**Fastest path: the packaged Action.** One step gets you `replay`/`lint`/`verify-cassettes` plus a PR
|
|
6
6
|
job-summary reporter (verdict table, staleness findings, cost/turns when available):
|
|
@@ -53,7 +53,7 @@ sha256-*checked* but not hard-blocking on mismatch — it's advisory for an inte
|
|
|
53
53
|
GitHub-hosted runners, no token/Docker/agent:
|
|
54
54
|
|
|
55
55
|
```yaml
|
|
56
|
-
- run: npm i -g cowork-harness@>=0.
|
|
56
|
+
- run: npm i -g "cowork-harness@>=0.23.0"
|
|
57
57
|
- run: cowork-harness lint scenarios/*.yaml # no silent false-greens
|
|
58
58
|
- run: cowork-harness verify-cassettes cassettes/ # privacy + staleness
|
|
59
59
|
- run: cowork-harness replay cassettes/ # token-free content/structure
|
|
@@ -128,8 +128,9 @@ dollar figures). In a skill repo these cassettes get **committed**. So:
|
|
|
128
128
|
`_KEYS`); empty by default. Redaction is **verdict-preserving** — `record` refuses to write if
|
|
129
129
|
redaction would flip an assertion (a manufactured green). `--no-redact` skips it for known-synthetic
|
|
130
130
|
inputs.
|
|
131
|
-
- **Always-on scan gate** — `verify-cassettes` flags email / currency / bare-domain / local-path
|
|
132
|
-
it finds in the committed cassettes and **exits non-zero**, so "no leak" is
|
|
131
|
+
- **Always-on scan gate** — `verify-cassettes` flags email / currency / bare-domain / local-path /
|
|
132
|
+
machine-inventory matches it finds in the committed cassettes and **exits non-zero**, so "no leak" is
|
|
133
|
+
a gate, not discipline.
|
|
133
134
|
Suppress synthetic / public reference names (NVCA, Cooley GO, …) with `--allow <regex>`. (Multi-word
|
|
134
135
|
proper names are NOT a default class — too noisy to gate on; add a pattern via config if your corpus
|
|
135
136
|
needs it.)
|
|
@@ -178,7 +179,7 @@ jobs:
|
|
|
178
179
|
with: { node-version: '20' }
|
|
179
180
|
- uses: actions/setup-python@v5
|
|
180
181
|
with: { python-version: '3.x' } # python3 only — PyYAML is bundled with the linter
|
|
181
|
-
- run: npm i -g cowork-harness@>=0.
|
|
182
|
+
- run: npm i -g "cowork-harness@>=0.23.0"
|
|
182
183
|
- run: cowork-harness lint scenarios/*.yaml # no-silent-false-green (needs python3; PyYAML bundled)
|
|
183
184
|
- run: cowork-harness verify-cassettes cassettes/ --output-format json # privacy + staleness gate
|
|
184
185
|
- run: cowork-harness replay cassettes/ --output-format json # token-free content/structure
|
|
@@ -207,7 +208,7 @@ jobs:
|
|
|
207
208
|
echo "live=true" >> "$GITHUB_OUTPUT"
|
|
208
209
|
fi
|
|
209
210
|
- if: steps.guard.outputs.live == 'true'
|
|
210
|
-
run: npm i -g cowork-harness@>=0.
|
|
211
|
+
run: npm i -g "cowork-harness@>=0.23.0"
|
|
211
212
|
- if: steps.guard.outputs.live == 'true'
|
|
212
213
|
run: cowork-harness run scenarios/ --output-format json
|
|
213
214
|
env:
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Fidelity tiers & answer paths
|
|
2
2
|
|
|
3
|
-
Self-contained reference. Tracks `cowork-harness 0.
|
|
3
|
+
Self-contained reference. Tracks `cowork-harness 0.23.0` (baseline `desktop-1.18286.0`).
|
|
4
4
|
|
|
5
5
|
## Fidelity tiers (`fidelity:` in the scenario)
|
|
6
6
|
|
|
@@ -187,5 +187,6 @@ and `scaffold` refuse to treat a partial run's half-finished output as a passing
|
|
|
187
187
|
redact from logs (beyond the auth tokens + `ANTHROPIC_CUSTOM_HEADERS`).
|
|
188
188
|
- `COWORK_HARNESS_SOFT_MISSING` — downgrade a missing mount source from hard-error to warn-and-skip.
|
|
189
189
|
- `COWORK_VM_GATEWAY` / `COWORK_VM_PROXY_PORT` / `COWORK_LIMA_INSTANCE` — L2 (microVM) knobs.
|
|
190
|
-
- `COWORK_LOCKDOWN` — default `on`:
|
|
191
|
-
|
|
190
|
+
- `COWORK_LOCKDOWN` — default `on`: gates sandbox hardening on every isolated tier — **aborts loudly**
|
|
191
|
+
if the L2 guest firewall fails to apply (no silent unprotected run), and also gates the
|
|
192
|
+
`container`/`hostloop` Docker hardening. Set `=off` to opt out and run without isolation deliberately.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Scenario & session schema, assertion catalog, web_fetch, full gotchas
|
|
2
2
|
|
|
3
|
-
Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.
|
|
3
|
+
Self-contained reference for authoring `cowork-harness` scenarios. Tracks `cowork-harness 0.23.0`
|
|
4
4
|
(baseline `desktop-1.18286.0`). If your checkout is newer, prefer the live `docs/scenario.md`,
|
|
5
5
|
`docs/session.md`, and `SPEC.md`.
|
|
6
6
|
|
|
@@ -61,6 +61,12 @@ skills: [report-gen] # OPTIONAL — scope the cassette-staleness
|
|
|
61
61
|
# `skills/<name>` dir under a mounted plugin-root) + the plugin's shared
|
|
62
62
|
# roots. Fail-closed to whole-tree on an unknown name. Omit = whole tree.
|
|
63
63
|
|
|
64
|
+
requires_capabilities: [ocr] # OPTIONAL — declare a capability the skill needs (e.g. office_convert,
|
|
65
|
+
# ocr, pdf_tables, opencv). If the running agent image provably omits
|
|
66
|
+
# one, the harness ABORTS before the paid run (exit 3) — unless the
|
|
67
|
+
# scenario also asserts `allow_missing_capability: true`, which downgrades
|
|
68
|
+
# the abort to a notice and proceeds. Live tiers only.
|
|
69
|
+
|
|
64
70
|
allow_host_writes: true # OPTIONAL — required to run `hostloop` fidelity with a WRITABLE
|
|
65
71
|
# connected folder (session `mode: rw`/`rwd`): with no container
|
|
66
72
|
# around hostloop's native file tools, that combination gives the
|
|
@@ -80,8 +86,10 @@ session = your setup.**
|
|
|
80
86
|
```yaml
|
|
81
87
|
# model & reasoning
|
|
82
88
|
model: claude-opus-4-8 # omit for the agent default
|
|
89
|
+
account_name: my-account # OPTIONAL — which configured account/credential set to run as
|
|
83
90
|
effort: high # low | medium | high | xhigh
|
|
84
91
|
max_thinking_tokens: 31999 # positive int, or per-model map {default, <model>: <n>}; default 31999
|
|
92
|
+
extended_thinking: true # INERT — not a real Cowork toggle; use max_thinking_tokens instead
|
|
85
93
|
permission_mode: default # default | acceptEdits | plan | bypassPermissions
|
|
86
94
|
permission_parity: cowork # cowork (unscripted tool calls allowed) | strict (deny unscripted)
|
|
87
95
|
|
|
@@ -97,8 +105,9 @@ plugins:
|
|
|
97
105
|
marketplaces: [] # plugin_marketplaces (git URLs or local paths)
|
|
98
106
|
local_marketplaces: [] # local marketplace dirs (each has a marketplace.json)
|
|
99
107
|
enabled: [my-skill@local] # enabledPlugins (name@marketplace)
|
|
100
|
-
local_plugins: [./skills/my-skill] # host plugin dirs → mnt/.local-plugins/marketplaces/<
|
|
101
|
-
# (≥1.14271.0
|
|
108
|
+
local_plugins: [./skills/my-skill] # host plugin dirs → mnt/.local-plugins/marketplaces/local-desktop-app-uploads/<plugin>
|
|
109
|
+
# (the marketplace segment is that fixed synthetic name; ≥1.14271.0 —
|
|
110
|
+
# older baselines use mnt/.local-plugins/cache)
|
|
102
111
|
remote_plugins: []
|
|
103
112
|
skills:
|
|
104
113
|
local: [] # extra host skill dirs
|
|
@@ -234,7 +243,7 @@ passes only if every key passes. Keep one concern per item unless you mean conju
|
|
|
234
243
|
| `allow_missing_capability: true` | verdict modifier — suppresses the default-fail when the (partial "core") agent image omits a capability the skill used but real Cowork ships (OCR/LibreOffice/markitdown/opencv/PDF-tables). Assert only when the skill's fallback is genuinely equivalent; otherwise rebuild full parity (`--build-arg COWORK_FULL_PARITY=1`). Also opts out of the `requires_capabilities` declared-need check. Live tiers only |
|
|
235
244
|
| `allow_l0_plugin_divergence: true` | verdict modifier — opt into L0/protocol plugin divergence: suppresses the default-fail when a plugin behaves differently at `protocol` (L0) fidelity than under a sandboxed tier. Live tiers only |
|
|
236
245
|
| `allow_stall: true` | verdict modifier — suppresses the `stalled` default-fail when a run ends on a question having done no productive tool work after its last gate (the agent asked for input and stopped — incl. re-asking in plain text after answering an `AskUserQuestion`); assert only when ending on a question is intended, else script the answer (`answer:` / `--answer` / a decider) |
|
|
237
|
-
| `transcript_no_host_path: true` | no host path (`/Users/`, `/opt/cowork/`, `/home/`, `/root/`) leaked into model-visible text — **incompatible with `hostloop`**:
|
|
246
|
+
| `transcript_no_host_path: true` | no host path (`/Users/`, `/opt/cowork/`, `/home/`, `/root/`) leaked into model-visible text — **incompatible with `hostloop` AND `protocol`**: hostloop's native file tools legitimately expose real host paths, and protocol (L0) runs the agent's file tools on the real host cwd with no sealed filesystem, so this fails BY DESIGN on both (the harness warns at run start if asserted anyway); use `container`/`microvm` for this check |
|
|
238
247
|
| `egress_denied: <host>` | the host was blocked by the egress proxy |
|
|
239
248
|
| `egress_allowed: <host>` | the host was allowed through |
|
|
240
249
|
| `artifact_json: {artifact, path, …}` | assert a JSON artifact's contents — `equals`/`gt`/`in`/`exists`/`absent`/`is_null` over a dotted `path` (`in` = membership in a list, for a stochastic/LLM value; `absent` ≠ `is_null`; an unresolved intermediate fails loud) |
|
|
@@ -372,7 +381,8 @@ at the top of this file.
|
|
|
372
381
|
the stochastic path flags the run `nonDeterministic`. The LLM decider is one mechanism, two
|
|
373
382
|
spellings: `on_unanswered: llm` (YAML) and `--decider-llm` (CLI). The bare `--on-unanswered llm`
|
|
374
383
|
is rejected (use `--decider-llm`). `agent` is **retired** — `on_unanswered: agent` is rejected by
|
|
375
|
-
the schema. (`src/types.ts:
|
|
384
|
+
the schema. (`src/types.ts:365` — the `on_unanswered` enum; `src/cli.ts:899` — the CLI-side
|
|
385
|
+
`--on-unanswered` value check.)
|
|
376
386
|
|
|
377
387
|
4. **`--on-unanswered first` is non-deterministic too** — it picks option 1 and is flagged
|
|
378
388
|
`nonDeterministic`; not a deterministic substitute for scripted answers.
|
|
@@ -126,8 +126,9 @@ if UNCLASSIFIED_KEYS:
|
|
|
126
126
|
)
|
|
127
127
|
# Embedded fallback for the top-level scenario keys — kept EQUAL to the generated `topLevelKeys`
|
|
128
128
|
# (test-enforced, like _CLASSIFIED_KEYS for assert keys) so a missing assertion-keys.json can't silently
|
|
129
|
-
# reintroduce key drift. `
|
|
130
|
-
#
|
|
129
|
+
# reintroduce key drift. `assertions` is NOT here — it's a hard error handled by its own special-case
|
|
130
|
+
# in the unknown-key check below (`k != "assertions"`). `profile` is retired vocabulary and now falls
|
|
131
|
+
# through to that same unknown-key check like any other typo — no special-case for it.
|
|
131
132
|
_EMBEDDED_TOP_LEVEL_KEYS = {
|
|
132
133
|
"name",
|
|
133
134
|
"baseline",
|
|
@@ -260,7 +261,7 @@ def lint_doc(doc, path, raw_lines):
|
|
|
260
261
|
|
|
261
262
|
# W: unknown top-level keys (typo or hallucinated schema)
|
|
262
263
|
for k in doc:
|
|
263
|
-
if k not in TOP_LEVEL_KEYS and k != "assertions"
|
|
264
|
+
if k not in TOP_LEVEL_KEYS and k != "assertions":
|
|
264
265
|
findings.append(
|
|
265
266
|
Finding(
|
|
266
267
|
"WARN",
|
|
@@ -426,19 +427,28 @@ def lint_doc(doc, path, raw_lines):
|
|
|
426
427
|
_DQ_REGEX_LINE = re.compile(
|
|
427
428
|
r'^\s*-?\s*(' + "|".join(sorted(REGEX_KEYS)) + r')\s*:\s*"([^"]*\\[^"]*)"'
|
|
428
429
|
)
|
|
430
|
+
# A run of an EVEN number of consecutive backslashes in a double-quoted YAML scalar is a properly
|
|
431
|
+
# paired escape (`\\` -> a literal `\`), so e.g. "\\d+ items" decodes to the valid regex `\d+ items` —
|
|
432
|
+
# not a mistake. An ODD run leaves one backslash unpaired, which is the actual footgun (YAML either
|
|
433
|
+
# eats it or errors, depending on what follows). Only flag the odd case.
|
|
434
|
+
_ODD_BACKSLASH_RUN = re.compile(r"\\+")
|
|
435
|
+
|
|
436
|
+
|
|
437
|
+
def _has_unpaired_backslash(s):
|
|
438
|
+
return any(len(m.group(0)) % 2 == 1 for m in _ODD_BACKSLASH_RUN.finditer(s))
|
|
429
439
|
|
|
430
440
|
|
|
431
441
|
def _lint_regex_quoting(path, raw_lines):
|
|
432
442
|
out = []
|
|
433
443
|
for i, line in enumerate(raw_lines, start=1):
|
|
434
444
|
m = _DQ_REGEX_LINE.match(line)
|
|
435
|
-
if m:
|
|
445
|
+
if m and _has_unpaired_backslash(m.group(2)):
|
|
436
446
|
out.append(
|
|
437
447
|
Finding(
|
|
438
448
|
"WARN",
|
|
439
449
|
"regex-double-quoted",
|
|
440
|
-
f"`{m.group(1)}` uses a DOUBLE-quoted regex containing
|
|
441
|
-
f'("{m.group(2)}") — YAML strips
|
|
450
|
+
f"`{m.group(1)}` uses a DOUBLE-quoted regex containing an unescaped backslash "
|
|
451
|
+
f'("{m.group(2)}") — YAML strips it, so the regex is wrong.',
|
|
442
452
|
"Single-quote the regex (e.g. '\\d+') or use a block scalar. Use [\\s\\S] not . to span turns.",
|
|
443
453
|
path,
|
|
444
454
|
i,
|
package/AGENTS.md
CHANGED
|
@@ -39,8 +39,8 @@ protocol layer or run-loop bookkeeping in the CLI.
|
|
|
39
39
|
gate-steering silently no-ops (the O7 bug). ELF-verified; a regression test pins it.
|
|
40
40
|
- **"profile" is retired vocabulary.** Synced release ground truth = **`PlatformBaseline`** (`baseline:` /
|
|
41
41
|
`baselines/`); authored setup = **`SessionConfig`** (`session:` / `sessions/`). The `profile:`
|
|
42
|
-
scenario key is retired vocabulary —
|
|
43
|
-
reintroduce the term.
|
|
42
|
+
scenario key is retired vocabulary — the alias is gone and it is now rejected as an unknown key — do
|
|
43
|
+
not reintroduce the term.
|
|
44
44
|
- **A new assertion must pick its replay class.** *Content* assertions (read only `ctx.transcript` / the
|
|
45
45
|
record) go in `src/run/cassette.ts` `contentKeys` so they run on the token-free `replay` PR gate;
|
|
46
46
|
*filesystem / egress* assertions do NOT — they are excluded **by omission** from `contentKeys`, so they
|
package/CHANGELOG.md
CHANGED
|
@@ -6,13 +6,223 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.23.0] — 2026-07-04
|
|
10
|
+
|
|
11
|
+
### Added
|
|
12
|
+
|
|
13
|
+
- **The baseline `spawn.env` is now binary-derived and drift-alarmed, not hand-transcribed.** `sync`
|
|
14
|
+
enumerates the Desktop→agent spawn env directly from the `app.asar` construction (three windows +
|
|
15
|
+
gate/const value resolution) and writes the resolved map into the baseline, guarded by a
|
|
16
|
+
`checkSpawnContractFacts` sentinel over the scalar options/tools/prompt structure. Additions,
|
|
17
|
+
removals, and value changes in the spawn contract now surface as loud `sync`-time signals instead of
|
|
18
|
+
drifting silently. This closed real, already-present drift: seven env keys the production agent
|
|
19
|
+
receives (`MCP_TOOL_TIMEOUT`, `API_TIMEOUT_MS`, `CLAUDE_CODE_ENABLE_FINE_GRAINED_TOOL_STREAMING`,
|
|
20
|
+
`DISABLE_AUTOUPDATER`, `CLAUDE_CODE_EMIT_TOOL_USE_SUMMARIES`, `USE_STAGING_OAUTH`, `USE_LOCAL_OAUTH`)
|
|
21
|
+
were missing from the committed baseline and are now pinned. A hand-transcribed golden fixture
|
|
22
|
+
cross-checks the generator so a resolver bug can't silently rewrite the contract.
|
|
23
|
+
- **`sync` surfaces stale-allowlist prune hints instead of discarding them.** When a
|
|
24
|
+
`SPAWN_ENV_ALLOWLIST` entry is no longer constructed anywhere in the asar, `partitionSpawnFlags`
|
|
25
|
+
routes it into `SyncResult.notes` (distinct from the hard-fail deltas above) and the CLI prints it
|
|
26
|
+
under a non-blocking `notes (non-blocking):` line — a prune candidate to review, not a drift alarm.
|
|
27
|
+
- **TUI forward-compatibility set** (three items hardening the display seam for future frontends):
|
|
28
|
+
- **The display-policy seam is now an explicit contract.** `src/run/display-translate.ts`'s header
|
|
29
|
+
states it is the single policy seam for translating model-visible VM paths for human display
|
|
30
|
+
(hostloop-only / identity-without-ctx / identity-when-shareable), locked by a 20-row table-driven
|
|
31
|
+
contract test; `docs/debugging.md` explains why hostloop shows `/Users/…` while other tiers show
|
|
32
|
+
`/sessions/…`.
|
|
33
|
+
- **Per-run `mounts.json`.** Every run dir now records the VM-path context (`{v:1, sessionId,
|
|
34
|
+
effectiveFidelity, outputsHostDir, uploadsHostDir, folders}`) — best-effort, never load-bearing,
|
|
35
|
+
derived from the same call that feeds the live display translator. `loadVmPathContext` lets any
|
|
36
|
+
historical-run consumer rebuild the context (null → degrade to identity). First consumer:
|
|
37
|
+
`trace --translate-paths` (text output only) renders hostloop runs with host paths, threaded into
|
|
38
|
+
row construction pre-slice. Cassettes structurally cannot carry the file.
|
|
39
|
+
- **OSC 8 terminal hyperlinks.** On a real TTY (not CI, not `--compact`/`--demo`, opt-out
|
|
40
|
+
`COWORK_HARNESS_NO_HYPERLINKS=1`), host-shaped `computer://` links in assistant text render as
|
|
41
|
+
clickable `file://` hyperlinks (`normalizeEncodePath`, decode-then-re-encode; backtick code spans
|
|
42
|
+
and VM-shaped links pass through; tool lines are excluded — they truncate at ~80 chars and a
|
|
43
|
+
sliced URL would link to a wrong target). Piped/non-TTY output stays byte-identical.
|
|
44
|
+
|
|
45
|
+
- **`machine-inventory` cassette privacy scan class.** `verify-cassettes` now flags the sentinel
|
|
46
|
+
phrases a capability-manifest recording leaks ("applications on this machine", "installed
|
|
47
|
+
integrations/apps/extensions", …) — prose mentions of an app never trip it — with a scoped
|
|
48
|
+
`--allow-machine-inventory <regex>` to whitelist provably-synthetic values. The capability-manifest
|
|
49
|
+
filter line is recognized so the manifest itself doesn't false-trip the other classes. Tightening the
|
|
50
|
+
privacy gate before the 1.0 contract freezes keeps it from becoming a breaking change to consumers'
|
|
51
|
+
committed cassettes later.
|
|
52
|
+
- **The 1.0 compatibility contract (SPEC.md §12).** Enumerates the surfaces semver covers from `1.0.0` —
|
|
53
|
+
CLI commands/flags + exit codes, the scenario/session/baseline/`RunResult`/cassette(v7)/protocol
|
|
54
|
+
schemas, the documented `COWORK_HARNESS_*` (+ `COWORK_AGENT_BINARY`/`COWORK_AGENT_IMAGE`) env vars, and
|
|
55
|
+
the packaged Action's inputs/outputs — and states what is explicitly NOT covered (human-readable
|
|
56
|
+
terminal text, `trace` row shapes, the paraphrased prompt append). Cross-linked from README and
|
|
57
|
+
RELEASING.md.
|
|
58
|
+
- **First committed `hostloop`-tier replay cassette + live two-tier `computer_links_resolve`
|
|
59
|
+
coverage.** `examples/replays/hostloop-computer-links.cassette.json` (from a new purpose-built
|
|
60
|
+
`fidelity: hostloop` scenario) is the first committed cassette at the newest/headline tier — the one
|
|
61
|
+
the token-free replay lane never exercised — and asserts that a `computer://` link the model shares
|
|
62
|
+
resolves to its real collected artifact. Verified live at both `container` (VM-shaped link) and
|
|
63
|
+
`hostloop` (host-shaped link); wired into CI's replay + privacy-scan gates.
|
|
64
|
+
- **Host-platform / workspace-host-paths identity env vars.** The spawn env now emits
|
|
65
|
+
`CLAUDE_CODE_HOST_PLATFORM` (`process.platform`, on every tier that assembles the Cowork spawn env —
|
|
66
|
+
container/microvm/hostloop; protocol (L0) spawns with the plain base env) and `CLAUDE_CODE_WORKSPACE_HOST_PATHS`
|
|
67
|
+
(connected-folder host paths, hostloop only when folders are present) — matching what the real Cowork
|
|
68
|
+
spawn sets, binary-verified against the in-VM ELF and Desktop asar. The account-identity and `OTEL_*`
|
|
69
|
+
vars stay unset (they need live Desktop account state the headless harness can't know; documented in
|
|
70
|
+
`docs/fidelity-gaps.md`).
|
|
71
|
+
- **Reserved exit code `4` on the `run`/`skill` family** for a future "needs input / surfaced question"
|
|
72
|
+
outcome — documented in SPEC.md so a later addition is additive rather than a renumbering of the
|
|
73
|
+
burned `0`/`1`/`2`/`3` space.
|
|
74
|
+
|
|
75
|
+
### Removed
|
|
76
|
+
|
|
77
|
+
- **The `profile:` scenario-field alias.** The top-level `profile:` key (an earlier name for
|
|
78
|
+
`baseline:`) is no longer accepted — it was silently remapped with a deprecation warning; it now
|
|
79
|
+
errors as an unknown key. Use `baseline:`.
|
|
80
|
+
- **The deprecated `Profile` re-export (library API).** The `Profile` const/type in `src/types.ts` was
|
|
81
|
+
renamed to `PlatformBaseline` with a "remove next minor" promise that never fired; removed now so the
|
|
82
|
+
1.0 API contract doesn't freeze retired vocabulary in. Import `PlatformBaseline`.
|
|
83
|
+
- **The `scaffold --from-run <id>` flag.** `scaffold` had two spellings for one thing; the canonical
|
|
84
|
+
positional `scaffold <run-id | run-dir>` stays, and `--from-run` now errors as an unknown flag
|
|
85
|
+
(exit 2) with the usage string pointing at the positional form.
|
|
86
|
+
- **The `-V` short form for `--verbose`.** `-v` (version, per `node -v`/`npm -v`) and `-V` (verbose)
|
|
87
|
+
were a shift-key-typo collision that silently flipped meaning. `--verbose` is now long-only on every
|
|
88
|
+
command that accepts it (`run`/`skill`/`chat`/`decide`/`record`/`replay`/`verify-cassettes`); `-v`
|
|
89
|
+
still prints the version and `-q` still means `--quiet`.
|
|
90
|
+
- **The dead `forceDisableHostLoop` loop-decision key.** It was never populated by `sync` and its
|
|
91
|
+
branch could never fire — a config key that silently does nothing is a trap in a 1.0 schema. The
|
|
92
|
+
field and its branch are removed (re-add with real semantics if `sync` ever derives it).
|
|
93
|
+
|
|
94
|
+
### Changed
|
|
95
|
+
|
|
96
|
+
- **CI's boundary job now pulls the published GHCR agent image the packaged Action pins**, retagging it
|
|
97
|
+
for the sandbox probes so a bad publish surfaces in our own CI instead of only in a consumer's runner
|
|
98
|
+
(it previously only ever `docker build`t the image locally). A pull failure now hard-fails
|
|
99
|
+
(`::error::` + `exit 1`) on the canonical repo instead of silently rebuilding; forks and pre-publish
|
|
100
|
+
runs (no GHCR read access yet) keep the local-build fallback with a warning. Live-verified on the
|
|
101
|
+
0.23.0 release PR's CI run: `docker pull ghcr.io/yaniv-golan/cowork-agent-base:2` resolved and
|
|
102
|
+
retagged successfully (image name, `linux/arm64` platform, and the default `GITHUB_TOKEN`'s GHCR
|
|
103
|
+
read access are all confirmed working, not just implemented).
|
|
104
|
+
|
|
105
|
+
### Fixed
|
|
106
|
+
|
|
107
|
+
- **Redaction could destroy a `computer://` link and manufacture a VACUOUS `computer_links_resolve`
|
|
108
|
+
pass on replay.** The repo's local-path redaction pattern didn't exclude `)` from its character
|
|
109
|
+
class, so it ate a markdown link's closing paren — replay's extractor then saw an unterminated link,
|
|
110
|
+
found zero links, and the presence-gated assertion passed while checking nothing (the first committed
|
|
111
|
+
hostloop cassette shipped exactly this). Three-part fix: the redaction patterns now redact only the
|
|
112
|
+
machine-specific path prefix (stopping before `/mnt/`, so replay's structural-marker resolution still
|
|
113
|
+
works) and exclude link delimiters; `record`'s verdict-preservation guard gained a fourth check that
|
|
114
|
+
compares `computer://` link counts pre/post redaction and refuses to write a cassette whose links
|
|
115
|
+
redaction destroyed; and the hostloop cassette was re-recorded — its replay now extracts and resolves
|
|
116
|
+
the link for real.
|
|
117
|
+
- **Scenario-schema violations now surface as category `usage`, not `internal`.** A typo'd or retired
|
|
118
|
+
key (e.g. `profile:`) threw an uncaught Zod error that the top-level catch labeled `internal` — a
|
|
119
|
+
user mistake masquerading as a harness bug. `parseScenarioFile` now wraps schema errors in a
|
|
120
|
+
`UsageError` that names the offending file; exit stays 2.
|
|
121
|
+
- **The `protocol-smoke` example no longer fails by design on a live run.** `protocol` (L0) runs the
|
|
122
|
+
agent's file tools on the real host cwd with no sealed filesystem — exactly like `hostloop` — so a
|
|
123
|
+
host path in a tool result is expected there, not a leak. The `host_path_leak` default-fail is now
|
|
124
|
+
exempted at `protocol` as well as `hostloop` (emitting a notice), so the flagship example passes its
|
|
125
|
+
own assertions on every advertised lane. The signal stays a hard fail at the sandboxed
|
|
126
|
+
`container`/`microvm` tiers (where a host path IS a regression), and an explicit
|
|
127
|
+
`transcript_no_host_path` assertion still enforces cleanliness at any tier.
|
|
128
|
+
- **The LLM decider prompt no longer travels via `argv`.** `claude -p <prompt>` put the gate/skill
|
|
129
|
+
text in the process's argument vector, world-readable via `ps` on shared hosts. The prompt is now
|
|
130
|
+
delivered on stdin (the same channel the microvm auth-token uses); argv carries only
|
|
131
|
+
`-p --model … --output-format json`. Off-brand for a tool that privacy-scans its own cassettes.
|
|
132
|
+
- **Malformed decider env knobs now fail loud instead of silently reverting.**
|
|
133
|
+
`COWORK_HARNESS_LLM_TIMEOUT_MS` / `COWORK_HARNESS_LLM_MAX_BYTES` went through `Number(…) || default`,
|
|
134
|
+
so a typo (`5m`) or an explicit `0` silently became the default. Both now route through
|
|
135
|
+
`envPositiveNumber`, which warns loud on a set-but-unparseable/non-positive value (an unset var still
|
|
136
|
+
uses the same default).
|
|
137
|
+
- **The bundled `cowork-harness` skill's CI recipe no longer breaks under bash/zsh.** The recommended
|
|
138
|
+
`npm i -g cowork-harness@>=0.22.0` was unquoted — `>=` is a shell redirection, so the snippet failed
|
|
139
|
+
as written in bash (GitHub Actions' default `run:` shell) and zsh alike. Now quoted at all sites. Same pass corrected the skill's command inventory (`status` was
|
|
140
|
+
missing), a wrong `CLAUDE_PLUGIN_ROOT` scaffold path, missing `requires_capabilities` /
|
|
141
|
+
`extended_thinking` / `account_name` schema docs, stale gotcha citations, a `scenario.py` regex-lint
|
|
142
|
+
false-positive, and thin eval coverage.
|
|
143
|
+
- **`doctor --tier microvm` now detects an unprovisioned Lima instance.** It previously checked only
|
|
144
|
+
for `limactl` itself, not whether `vm init` had actually provisioned the instance for the current
|
|
145
|
+
config — a missing VM image could slip past `doctor` and only surface as first-run VM-boot latency
|
|
146
|
+
on the next live `microvm` run (which self-provisions). New `vm-instance` check is advisory (`warn`,
|
|
147
|
+
non-blocking, matching `microvm.ts`'s self-provisioning behavior), skipped when `limactl` itself is
|
|
148
|
+
already the reported problem. Live-verified against a real, unprovisioned Lima install.
|
|
149
|
+
- **Top-level `--help` printed an invalid combined flag shorthand.** `--allow-domain/-email/-path
|
|
150
|
+
<regex>` is not something the parser accepts — the three flags are independent and parsed
|
|
151
|
+
separately (`verify-cassettes`'s own usage string already had this right). Fixed to list the three
|
|
152
|
+
flags separately, matching the dedicated usage string.
|
|
153
|
+
|
|
154
|
+
### Documentation
|
|
155
|
+
|
|
156
|
+
- **Documented the Task-dispatch cap divergence in `docs/fidelity-gaps.md`.** Real Cowork skips agent
|
|
157
|
+
Task dispatches beyond `{perTask:1, global:3}` (a binary-verified gate); the harness does not cap at
|
|
158
|
+
runtime, mitigated by the `dispatch_count_max` assertion. The faithful runtime limiter is deferred
|
|
159
|
+
post-1.0.
|
|
160
|
+
- **Doc-vs-code audit (post-0.22.0) — corrected several doc claims that had drifted from the
|
|
161
|
+
implementation, found by a systematic docs-and-skill sweep.**
|
|
162
|
+
- **Baseline pin staleness.** README, DESIGN.md, SPEC.md, the companion skill's `SKILL.md`, and
|
|
163
|
+
`docs/cowork-spawn-contract-1.12603.1.md` all still pinned `desktop-1.17377.1`/`.2` after the
|
|
164
|
+
platform baseline had moved on to `desktop-1.18286.0`. Reconciled the plain "current baseline"
|
|
165
|
+
pins; deliberately left DESIGN.md's point-in-time verification stamps (§ Control protocol /
|
|
166
|
+
Spawn contract) untouched, since bumping those would assert re-verification work that hasn't
|
|
167
|
+
actually happened. README's "Status" paragraph had the same unresolved tension one section
|
|
168
|
+
down (claiming `1.18286.0` is latest two sentences after a `1.17377.1` verification stamp) —
|
|
169
|
+
added a clarifying parenthetical instead of silently picking one number.
|
|
170
|
+
- **`docs/chat.md` self-contradiction.** Its `--folder` behavior rows named `protocol`/`container`/
|
|
171
|
+
`microvm` in two places despite `chat` never accepting `microvm` (already correctly excluded
|
|
172
|
+
elsewhere in the same file).
|
|
173
|
+
- **`docs/cassette.md`'s assertion table** was missing six replay-evaluated keys (`skill_triggered`,
|
|
174
|
+
`no_skill_triggered`, `max_cost_usd`, `max_tokens`, `tool_calls_max`, `max_turns`) despite already
|
|
175
|
+
documenting all six correctly in `docs/scenario.md`.
|
|
176
|
+
- **`docs/session.md`** never documented the real, live `account_name` session field.
|
|
177
|
+
- **`docs/scenario.md`'s `run --matrix` example** pointed `skill_dirs` at a fabricated
|
|
178
|
+
`../variants/v1/…` path that doesn't exist anywhere in the repo. Repointed the section at the new
|
|
179
|
+
`examples/matrices/csv-metrics-matrix.yaml` fixture (baselines-only axis — this repo has no second
|
|
180
|
+
`csv-metrics` variant to matrix against, so the shipped example omits `skill_dirs` rather than
|
|
181
|
+
inventing fake paths).
|
|
182
|
+
- **`docs/discovery.md`** left the `<proj-slug>` placeholder in its "find the VM session log" path
|
|
183
|
+
unexplained. Documented what it is (Claude Code's own project-slug derivation, opaque to this
|
|
184
|
+
repo) and gave a practical `ls`-based workaround instead of guessing at undocumented CLI internals.
|
|
185
|
+
- **Doc-index and cross-link gaps.** `docs/README.md`'s guide table and `llms.txt`'s command list
|
|
186
|
+
were both missing `stats`/`status`/`diff`; the `decide` reference row cited only `decide --help`
|
|
187
|
+
despite a full worked "dry-running a decider" subsection already existing in `docs/scenario.md`;
|
|
188
|
+
the README architecture diagram had no `hostloop` representation; `docs/maintenance.md`'s
|
|
189
|
+
`sync --diff` example (real but the two oldest baseline files in the repo) wasn't flagged as
|
|
190
|
+
illustrative; and README's `doctor` section read as if bare `doctor` were more general than
|
|
191
|
+
`--tier container`, when bare `doctor` **is** `--tier container` by default.
|
|
192
|
+
- **README command-table / flag-reference gaps.** `--compact`/`--demo` (output trimmed for
|
|
193
|
+
shareable screenshots/GIFs) were undocumented in the `skill`/`run` command-table rows;
|
|
194
|
+
`record`'s row omitted `--no-redact`/`--allow-failing`/`--dry-run`; the reproducibility-knobs
|
|
195
|
+
section omitted `COWORK_HARNESS_VERIFY_AGENT_SHA`; the exit-code summary claimed a uniform
|
|
196
|
+
`0`/`1`/`2`/`3` "on every command" when `diff`/`lint` have documented per-command exceptions
|
|
197
|
+
(SPEC.md already had the accurate table — README's summary now points at it instead of
|
|
198
|
+
overstating); and a note was added near the Prerequisites block that the worked
|
|
199
|
+
`examples/scenarios/...` commands need a source checkout, not a global `npm install -g`.
|
|
200
|
+
- Added **`examples/matrices/csv-metrics-matrix.yaml`**, a worked example for `run --matrix` (no
|
|
201
|
+
prior fixture existed) — live-verified: both baseline cells pass.
|
|
202
|
+
|
|
203
|
+
### Internal
|
|
204
|
+
|
|
205
|
+
- **`scripts/check-versions.ts`'s version-lockstep guard now also cross-checks the `(baseline
|
|
206
|
+
desktop-X)` pins** across README/`SKILL.md`/the spawn-contract doc against the newest committed
|
|
207
|
+
`baselines/*.json`, closing the exact class of drift the doc audit above found so it can't silently
|
|
208
|
+
recur. Deliberately excludes DESIGN.md's verification-stamp lines (see above).
|
|
209
|
+
- `alwaysContentKeys`/`questionGateKeys`/`manifestKeys` in `src/run/cassette.ts` are now exported
|
|
210
|
+
(previously function-local); `test/cassette-docs-sync.test.ts` asserts `docs/cassette.md`'s
|
|
211
|
+
assertion table stays in sync with their union.
|
|
212
|
+
- `test/cli-help.test.ts` gained a check that every CLI command appears in README's "Commands at a
|
|
213
|
+
glance" table, on the same "doc can't silently drift from code" principle.
|
|
214
|
+
- **`test/vm-path-ctx-file.test.ts` gained a structural cassette-privacy regression test.** Asserts
|
|
215
|
+
the committed `examples/replays/example-pdf-skill.cassette.json`'s top-level key set is closed and
|
|
216
|
+
contains no `mount`-named field — guards the cassette assembler itself (not just `buildManifest`'s
|
|
217
|
+
walk scope), so a future edit that adds a mounts-bearing field to the cassette literal is caught
|
|
218
|
+
without needing a live re-record.
|
|
219
|
+
|
|
9
220
|
## [0.22.0] — 2026-07-03
|
|
10
221
|
|
|
11
222
|
### Added
|
|
12
223
|
|
|
13
224
|
- **`computer://` link modeling — the prompt now instructs file links exactly as production does.**
|
|
14
|
-
Four pieces landed together
|
|
15
|
-
`docs/internal/2026-07-03-computer-link-scheme-research-and-plan.md`):
|
|
225
|
+
Four pieces landed together, grounded in binary research against the Desktop app:
|
|
16
226
|
- `src/vm-paths.ts` — a faithful port of Desktop's display-side VM→host path transform
|
|
17
227
|
(`deepTranslateVMPaths` / `mapVMPathToHostPath` / `encodeComputerUrlsForHostLoop`): markdown-link,
|
|
18
228
|
backtick, bare-token, and prose rewrite positions; per-segment percent-encoding; traversal
|
|
@@ -558,7 +768,7 @@ All notable changes to this project are documented here. The format is based on
|
|
|
558
768
|
sandboxed live tiers (protocol/replay need none); added `python3` to the README requirements
|
|
559
769
|
(the `lint` linter shells out to it); moved the `/plugin` slash-command block off the `bash`
|
|
560
770
|
fence; pinned `cowork-harness@>=0.17.0` in the CI recipe; noted that `chat` excludes
|
|
561
|
-
`microvm`/`cowork`; and other small corrections
|
|
771
|
+
`microvm`/`cowork`; and other small corrections from a full documentation audit.
|
|
562
772
|
- Refreshed all verification/version stamps that still pinned `1.15200.0` / `2.1.181`
|
|
563
773
|
(README, DESIGN, SPEC, the spawn-contract reference, `docs/cassette.md` fingerprint example,
|
|
564
774
|
and the `hostloop-prompt.ts` re-verified comment) to `1.15962.0` / `2.1.187`.
|
|
@@ -1418,7 +1628,7 @@ All notable changes to this project are documented here. The format is based on
|
|
|
1418
1628
|
|
|
1419
1629
|
### Internal
|
|
1420
1630
|
|
|
1421
|
-
- The npm tarball no longer ships
|
|
1631
|
+
- The npm tarball no longer ships internal planning notes that were accidentally being published.
|
|
1422
1632
|
|
|
1423
1633
|
### Added
|
|
1424
1634
|
|
package/DESIGN.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# DESIGN — parity model, deltas, and the maintenance contract
|
|
2
2
|
|
|
3
|
-
This document is the reference for *how faithful* each tier is, *what we deliberately don't reproduce*, and *why the chosen seams keep parity cheap to maintain*. Everything here is grounded in analysis of the live Claude Desktop `app.asar` (spawn contract and gates first verified at build 1.12603.1; updated through build 1.
|
|
3
|
+
This document is the reference for *how faithful* each tier is, *what we deliberately don't reproduce*, and *why the chosen seams keep parity cheap to maintain*. Everything here is grounded in analysis of the live Claude Desktop `app.asar` (spawn contract and gates first verified at build 1.12603.1; updated through build 1.18286.0 — volatile fields tracked in `baselines/desktop-1.18286.0.json`) and the on-disk runtime state on macOS.
|
|
4
4
|
|
|
5
5
|
> **Just want to pick a tier or write a scenario?** This doc is the *why*. For the *how*, start at the
|
|
6
6
|
> [README](./README.md) (tiers, quick start) and [docs/](./docs/README.md) (scenario/session reference).
|
|
@@ -187,6 +187,6 @@ The full Desktop→agent spawn contract (cwd `/sessions/<id>`, `CLAUDE_CONFIG_DI
|
|
|
187
187
|
|
|
188
188
|
- `fidelity: container | microvm` → **VM-loop** (the whole agent in the sandbox).
|
|
189
189
|
- `fidelity: hostloop` → **host-loop**: the agent LOOP is a **native process spawned directly on the host** (Desktop stages this same native macOS binary alongside the Linux/arm64 ELF the other tiers use), with native Bash/WebFetch disabled (`--disallowedTools`) and the agent's shell routed through the **workspace SDK-MCP server** — declared via `sdkMcpServers:["workspace"]` in the `initialize` handshake, with the driver (`src/agent/session.ts` + `src/hostloop/workspace-handler.ts`) handling `mcp_message` JSON-RPC and executing `bash` via `docker exec` into a VM sidecar container (no agent runs inside it) at `/sessions/<id>/mnt`. `${CLAUDE_PLUGIN_ROOT}` for the native process points at the staged plugin copy directly (a real host path); bash's `docker exec` still gets an intentionally-unresolvable sentinel so it self-heals via `find /sessions/<id>/mnt …`, exactly like production. Connected folders are bind-mounted (never copied) into both the native process's view and the VM sidecar, so the native file tools and bash see the same bytes. With no container around the native file tools, a PreToolUse hook (`src/hostloop/pretooluse-path-hook.ts`, a byte-faithful port of production's own containment check) is the security boundary for real filesystem access — see [docs/boundary.md](./docs/boundary.md) for the full safety posture (writable-folder consent, the runtime tripwire).
|
|
190
|
-
- `fidelity: cowork` → **auto-picks** host-loop vs VM-loop using Cowork's own decision logic (`src/loop-decision.ts`, an exact replica of Desktop's minified `f_()` loop-decision function): `requireFullVmSandbox
|
|
190
|
+
- `fidelity: cowork` → **auto-picks** host-loop vs VM-loop using Cowork's own decision logic (`src/loop-decision.ts`, an exact replica of Desktop's minified `f_()` loop-decision function): `requireFullVmSandbox ? vm : (dev override ? host : gate 1143815894)`. With the synced gate forced on, `cowork → hostloop`. (The replicated `f_()` **decision shape** is pinned to asar 1.12603.1 per `src/loop-decision.ts`; the 1.15200.0→1.15962.0 sync re-derives only the **gate value** it reads, not the logic.)
|
|
191
191
|
|
|
192
192
|
The **bash-visible world is identical** in both (`/sessions/<id>/mnt/...`); the agent-loop world differs (`${CLAUDE_PLUGIN_ROOT}` resolution, the shell tool). Use `hostloop` (or `cowork`) for production-faithful skill testing; `container` for fast VM-loop.
|