cool-workflow 0.1.82 → 0.1.84
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +2 -2
- package/.codex-plugin/plugin.json +4 -4
- package/README.md +128 -120
- package/apps/architecture-review/app.json +1 -1
- package/apps/architecture-review-fast/app.json +1 -1
- package/apps/end-to-end-golden-path/app.json +1 -1
- package/apps/pr-review-fix-ci/app.json +1 -1
- package/apps/release-cut/app.json +1 -1
- package/apps/research-synthesis/app.json +1 -1
- package/dist/capability-core.js +16 -8
- package/dist/capability-registry.js +270 -0
- package/dist/cli/command-surface.js +1320 -0
- package/dist/cli.js +2 -1307
- package/dist/commit.js +5 -1
- package/dist/doctor.js +153 -0
- package/dist/mcp-server.js +15 -1451
- package/dist/mcp-surface.js +1441 -0
- package/dist/orchestrator.js +13 -0
- package/dist/reclamation/hash.js +72 -0
- package/dist/reclamation.js +25 -78
- package/dist/run-registry/queue.js +6 -7
- package/dist/run-registry.js +35 -24
- package/dist/scheduler.js +78 -53
- package/dist/version.js +1 -1
- package/dist/worker-accept/acceptance.js +114 -0
- package/dist/worker-accept/blackboard-fanout.js +80 -0
- package/dist/worker-accept/blackboard-linkage.js +19 -0
- package/dist/worker-accept/context.js +2 -0
- package/dist/worker-accept/telemetry-ledger.js +116 -0
- package/dist/worker-accept/validation.js +77 -0
- package/dist/worker-accept/verifier-completion.js +73 -0
- package/dist/worker-isolation.js +41 -446
- package/docs/agent-delegation-drive.7.md +94 -86
- package/docs/agent-framework.md +33 -32
- package/docs/candidate-scoring.7.md +26 -24
- package/docs/canonical-workflow-apps.7.md +40 -40
- package/docs/capability-topology-registry.7.md +24 -24
- package/docs/cli-mcp-parity.7.md +230 -154
- package/docs/contract-migration-tooling.7.md +52 -41
- package/docs/control-plane-scheduling.7.md +49 -41
- package/docs/coordinator-blackboard.7.md +30 -30
- package/docs/dogfood-one-real-repo.7.md +44 -44
- package/docs/durable-state-and-locking.7.md +38 -30
- package/docs/end-to-end-golden-path.7.md +29 -29
- package/docs/error-feedback.7.md +27 -27
- package/docs/evidence-adoption-reasoning-chain.7.md +66 -58
- package/docs/execution-backends.7.md +88 -80
- package/docs/getting-started.md +35 -18
- package/docs/index.md +3 -3
- package/docs/mcp-app-surface.7.md +64 -64
- package/docs/multi-agent-cli-mcp-surface.7.md +86 -77
- package/docs/multi-agent-eval-replay-harness.7.md +63 -55
- package/docs/multi-agent-operator-ux.7.md +73 -65
- package/docs/multi-agent-runtime-core.7.md +39 -39
- package/docs/multi-agent-topologies.7.md +24 -24
- package/docs/multi-agent-trust-policy-audit.7.md +38 -38
- package/docs/node-snapshot-diff-replay.7.md +30 -22
- package/docs/observability-cost-accounting.7.md +53 -45
- package/docs/operator-ux.7.md +30 -30
- package/docs/pipeline-runner.7.md +31 -31
- package/docs/project-index.md +16 -5
- package/docs/real-execution-backends.7.md +51 -43
- package/docs/release-and-migration.7.md +46 -38
- package/docs/release-tooling.7.md +67 -50
- package/docs/routines.md +16 -16
- package/docs/run-registry-control-plane.7.md +124 -116
- package/docs/run-retention-reclamation.7.md +49 -41
- package/docs/sandbox-profiles.7.md +32 -32
- package/docs/scheduled-tasks.md +14 -14
- package/docs/security-trust-hardening.7.md +29 -29
- package/docs/source-context-profiles.7.md +28 -28
- package/docs/state-explosion-management.7.md +67 -59
- package/docs/state-node.7.md +8 -8
- package/docs/team-collaboration.7.md +66 -58
- package/docs/trust-model.md +126 -126
- package/docs/unix-principles.md +80 -80
- package/docs/vendor-manifest-loadability.7.md +20 -20
- package/docs/verifier-gated-commit.7.md +16 -16
- package/docs/web-desktop-workbench.7.md +73 -65
- package/docs/worker-isolation.7.md +34 -37
- package/docs/workflow-app-framework.7.md +38 -38
- package/manifest/plugin.manifest.json +4 -4
- package/package.json +3 -2
- package/scripts/bump-version.js +9 -1
- package/scripts/canonical-apps.js +4 -4
- package/scripts/dogfood-release.js +1 -1
- package/scripts/gen-parity-doc.js +106 -0
- package/scripts/golden-path.js +4 -4
- package/scripts/parity-check.js +27 -57
- package/scripts/release-flow.js +7 -6
- package/scripts/sync-project-index.js +1 -1
- package/dist/verifier-registry.js +0 -46
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
# Multi-Agent Trust / Policy / Audit
|
|
2
2
|
|
|
3
|
-
CW v0.1.22
|
|
4
|
-
multi-agent policy, provenance, blackboard write audit, and judge
|
|
5
|
-
It does not
|
|
6
|
-
CW v0.1.24
|
|
7
|
-
missing provenance, changed policy violations, or missing judge rationale
|
|
8
|
-
the regression gate.
|
|
3
|
+
CW v0.1.22 adds to the trust-audit layer that is already there. It gives
|
|
4
|
+
first-class multi-agent policy, provenance, blackboard write audit, and judge
|
|
5
|
+
rationale. It does not add a second audit subsystem.
|
|
6
|
+
CW v0.1.24 puts these trust projections into eval/replay comparison, so
|
|
7
|
+
missing provenance, changed policy violations, or missing judge rationale make
|
|
8
|
+
the regression gate fail.
|
|
9
9
|
|
|
10
10
|
## Model
|
|
11
11
|
|
|
12
|
-
Multi-agent trust state is plain JSON
|
|
12
|
+
Multi-agent trust state is plain JSON joined to records that are already there:
|
|
13
13
|
|
|
14
14
|
- `AgentRole.policy`
|
|
15
15
|
- `AgentGroup.policy`
|
|
@@ -18,7 +18,7 @@ Multi-agent trust state is plain JSON attached to existing records:
|
|
|
18
18
|
- candidate score and selection audit links
|
|
19
19
|
- append-friendly trust events in `.cw/runs/<run-id>/audit/events.jsonl`
|
|
20
20
|
|
|
21
|
-
Policies
|
|
21
|
+
Policies say clearly what is let through:
|
|
22
22
|
|
|
23
23
|
- allowed blackboard topic ids
|
|
24
24
|
- allowed write operations: `message`, `context`, `artifact`, `snapshot`,
|
|
@@ -30,11 +30,11 @@ Policies describe explicit authority:
|
|
|
30
30
|
- denied operations and reasons
|
|
31
31
|
|
|
32
32
|
Missing policy, missing role authority, out-of-scope topics, missing evidence,
|
|
33
|
-
or missing judge rationale fail closed and
|
|
33
|
+
or missing judge rationale fail closed and make audit records.
|
|
34
34
|
|
|
35
35
|
## Audit Events
|
|
36
36
|
|
|
37
|
-
The
|
|
37
|
+
The audit log that is already there keeps multi-agent dimensions with stable ids:
|
|
38
38
|
|
|
39
39
|
- `multi-agent.role-policy`
|
|
40
40
|
- `multi-agent.permission`
|
|
@@ -44,19 +44,19 @@ The existing audit log records multi-agent dimensions with stable ids:
|
|
|
44
44
|
- `judge.panel-decision`
|
|
45
45
|
- `policy.violation`
|
|
46
46
|
|
|
47
|
-
Events
|
|
47
|
+
Events hold ids such as `multiAgentRunId`, `agentRoleId`, `agentGroupId`,
|
|
48
48
|
`agentMembershipId`, `agentFanoutId`, `agentFaninId`, blackboard record ids,
|
|
49
49
|
candidate/score/selection/commit ids, topology ids, `sandboxProfileId`, and
|
|
50
|
-
`policyRef` when
|
|
50
|
+
`policyRef` when they have a part to play.
|
|
51
51
|
|
|
52
|
-
Audit events do not copy large blackboard bodies. Message provenance
|
|
53
|
-
author kind/id, role/group/membership/worker ids when known, source,
|
|
54
|
-
evidence refs, parent message ids, topic scope, a body hash, and a short
|
|
52
|
+
Audit events do not copy large blackboard bodies. Message provenance keeps
|
|
53
|
+
author kind/id, role/group/membership/worker ids when these are known, source,
|
|
54
|
+
linked evidence refs, parent message ids, topic scope, a body hash, and a short
|
|
55
55
|
summary.
|
|
56
56
|
|
|
57
57
|
## Blackboard Writes
|
|
58
58
|
|
|
59
|
-
Every blackboard write
|
|
59
|
+
Every blackboard write goes through audit:
|
|
60
60
|
|
|
61
61
|
- topic create/update
|
|
62
62
|
- message post
|
|
@@ -66,24 +66,24 @@ Every blackboard write is audited:
|
|
|
66
66
|
- coordinator decision
|
|
67
67
|
|
|
68
68
|
The audit record says who wrote, under which role or membership, which policy
|
|
69
|
-
|
|
70
|
-
the write was accepted, denied, superseded, conflicting, or blocked.
|
|
69
|
+
let it through or said no to it, what evidence was given, what record changed,
|
|
70
|
+
and if the write was accepted, denied, superseded, conflicting, or blocked.
|
|
71
71
|
|
|
72
|
-
Denied writes are
|
|
73
|
-
`policy.violation` and `blackboard.write` audit projections.
|
|
72
|
+
Denied writes are turned away before state mutation, and you can see them
|
|
73
|
+
through `policy.violation` and `blackboard.write` audit projections.
|
|
74
74
|
|
|
75
75
|
## Judge Rationale
|
|
76
76
|
|
|
77
|
-
Judge-panel scoring
|
|
78
|
-
score evidence and chair rationale. Accepted judge and panel records
|
|
79
|
-
score, candidate, evidence refs, role policy, and parent audit events.
|
|
77
|
+
Judge-panel scoring needs evidence and rationale. Panel selection needs
|
|
78
|
+
score evidence and chair rationale. Accepted judge and panel records point to
|
|
79
|
+
the score, candidate, evidence refs, role policy, and parent audit events.
|
|
80
80
|
|
|
81
81
|
Missing rationale or evidence blocks score, selection, fanin readiness, and
|
|
82
|
-
verifier-gated commit readiness where those gates
|
|
82
|
+
verifier-gated commit readiness where those gates lean on judge evidence.
|
|
83
83
|
|
|
84
84
|
## CLI
|
|
85
85
|
|
|
86
|
-
|
|
86
|
+
The commands that are already there still work the same way:
|
|
87
87
|
|
|
88
88
|
```bash
|
|
89
89
|
node scripts/cw.js audit summary <run-id>
|
|
@@ -104,7 +104,7 @@ node scripts/cw.js audit judge <run-id>
|
|
|
104
104
|
|
|
105
105
|
Use `--json` or `--format json` for deterministic machine output.
|
|
106
106
|
|
|
107
|
-
Human output
|
|
107
|
+
Human output has stable panels:
|
|
108
108
|
|
|
109
109
|
- Multi-Agent Trust
|
|
110
110
|
- Role Policies
|
|
@@ -117,7 +117,7 @@ Human output includes stable panels:
|
|
|
117
117
|
|
|
118
118
|
## Verify (fail-closed)
|
|
119
119
|
|
|
120
|
-
`audit summary`
|
|
120
|
+
`audit summary` puts in an `integrity` field but is a *reader* — it always exits 0,
|
|
121
121
|
so it cannot gate a script. `audit verify` is the gate:
|
|
122
122
|
|
|
123
123
|
```bash
|
|
@@ -125,9 +125,9 @@ node scripts/cw.js audit verify <run-id> # exit 1 if the chain is forged
|
|
|
125
125
|
node scripts/cw.js audit verify <run-id> --json
|
|
126
126
|
```
|
|
127
127
|
|
|
128
|
-
It
|
|
129
|
-
hash from genesis, checks `prevEventHash` linkage, and
|
|
130
|
-
forgery (an `eventHash`-less line
|
|
128
|
+
It proves the run's trust-audit hash chain again offline: it works out every event
|
|
129
|
+
hash from genesis, checks `prevEventHash` linkage, and finds the unchained-event
|
|
130
|
+
forgery (an `eventHash`-less line put into a chained log to be waved through as
|
|
131
131
|
"legacy"). The JSON reports `present`, `verified`, `eventCount`, `chained`,
|
|
132
132
|
`unchained`, `corruptLines`, and `failedChecks[]`.
|
|
133
133
|
|
|
@@ -135,9 +135,9 @@ Exit-code contract (the peer of `telemetry verify`):
|
|
|
135
135
|
|
|
136
136
|
- ANY **unverified** chain exits **1** — forged / edited / truncated / unchained-injected,
|
|
137
137
|
*and* a fully-corrupt log (every line unparseable, which reports `present:false` but
|
|
138
|
-
`verified:false`). The gate keys on `verified`, not `present`, so the
|
|
139
|
-
tamper —
|
|
140
|
-
`cw audit verify <run> && deploy` stops
|
|
138
|
+
`verified:false`). The gate keys on `verified`, not `present`, so the worst kind of
|
|
139
|
+
tamper — making a mess of the whole log — cannot get away by looking "absent". So
|
|
140
|
+
`cw audit verify <run> && deploy` stops when there is tampering.
|
|
141
141
|
- Only a truly **absent / empty** chain is `verified:true` / exit **0** — a run with
|
|
142
142
|
no audit log (or a blank one) has nothing to prove (no false-red).
|
|
143
143
|
|
|
@@ -151,10 +151,10 @@ MCP parity tools:
|
|
|
151
151
|
- `cw_audit_blackboard`
|
|
152
152
|
- `cw_audit_judge`
|
|
153
153
|
|
|
154
|
-
The older audit tools
|
|
154
|
+
The older audit tools are still there to use:
|
|
155
155
|
|
|
156
156
|
- `cw_audit_summary`
|
|
157
|
-
- `cw_audit_verify` — fail-closed
|
|
157
|
+
- `cw_audit_verify` — fail-closed proof again of the trust-audit hash chain (peer of `cw_telemetry_verify`)
|
|
158
158
|
- `cw_audit_worker`
|
|
159
159
|
- `cw_audit_provenance`
|
|
160
160
|
- `cw_audit_attest`
|
|
@@ -163,9 +163,9 @@ The older audit tools remain available:
|
|
|
163
163
|
## Operator Questions
|
|
164
164
|
|
|
165
165
|
The combined `multi-agent status`, `multi-agent evidence`, `report --show`,
|
|
166
|
-
`audit summary`, and `audit provenance` views
|
|
166
|
+
`audit summary`, and `audit provenance` views give the answers to:
|
|
167
167
|
|
|
168
|
-
- Which role was
|
|
168
|
+
- Which role was let to do this?
|
|
169
169
|
- Which blackboard message came from which role, member, or worker?
|
|
170
170
|
- Which write was denied and why?
|
|
171
171
|
- Which judge rationale was accepted?
|
|
@@ -173,7 +173,7 @@ The combined `multi-agent status`, `multi-agent evidence`, `report --show`,
|
|
|
173
173
|
|
|
174
174
|
## Regression
|
|
175
175
|
|
|
176
|
-
`test/multi-agent-trust-policy-audit-smoke.js`
|
|
176
|
+
`test/multi-agent-trust-policy-audit-smoke.js` makes a judge-panel run with
|
|
177
177
|
allowed and denied blackboard writes, message provenance, role/membership/worker
|
|
178
178
|
links, accepted judge rationale, missing-rationale and missing-evidence failure
|
|
179
179
|
paths, CLI output assertions, MCP parity assertions, report assertions, and
|
|
@@ -2,42 +2,42 @@
|
|
|
2
2
|
|
|
3
3
|
CW v0.1.35 adds Node Snapshot / Diff / Replay: per-NODE granularity over the
|
|
4
4
|
v0.1.23 eval/replay harness. Before v0.1.35 the harness worked only at RUN/SUITE
|
|
5
|
-
granularity — `createMultiAgentReplaySnapshot(run)`
|
|
6
|
-
was no way to snapshot, fingerprint, diff, or replay a single `StateNode`. This
|
|
7
|
-
release adds that
|
|
5
|
+
granularity — `createMultiAgentReplaySnapshot(run)` took a picture of a whole run;
|
|
6
|
+
there was no way to snapshot, fingerprint, diff, or replay a single `StateNode`. This
|
|
7
|
+
release adds that. It reuses the harness's normalize/stable-stringify way of working and
|
|
8
8
|
the v0.1.25 fingerprint/freshness pattern — without forking `StateNode`, the eval
|
|
9
9
|
harness, or the run-state schema (all additive).
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
This keeps the same base-system split used in other places: the mechanism
|
|
12
12
|
captures/diffs/replays one node by id; nothing decides which node "matters".
|
|
13
13
|
|
|
14
14
|
## Snapshot — derived + fingerprinted
|
|
15
15
|
|
|
16
16
|
A `NodeSnapshot` is a DERIVED projection of one `StateNode`: its body is normalized
|
|
17
|
-
(timestamps/paths
|
|
17
|
+
(timestamps/paths taken out via the eval harness's `normalizeValue`), so it is
|
|
18
18
|
byte-stable across captures of the same logical state. It carries a
|
|
19
19
|
`sourceFingerprint` — sha256 over the RAW node (`id:status:updatedAt` + artifact
|
|
20
|
-
and evidence ids/paths) — so any
|
|
20
|
+
and evidence ids/paths) — so any change to the state turns it over.
|
|
21
21
|
|
|
22
22
|
```text
|
|
23
23
|
node snapshot <run-id> <node-id> [--json]
|
|
24
24
|
```
|
|
25
25
|
|
|
26
|
-
|
|
26
|
+
Kept under `<run>/nodes/snapshots/<node-id>/<snapshot-id>.json`; the source
|
|
27
27
|
`<run>/nodes/<id>.json` stays the truth. The `snapshot-id` is content-addressed
|
|
28
|
-
(`snap-<node>-<fingerprint>`), so
|
|
28
|
+
(`snap-<node>-<fingerprint>`), so taking a new snapshot of an unchanged node is idempotent.
|
|
29
29
|
|
|
30
30
|
## Freshness — fail closed on drift
|
|
31
31
|
|
|
32
|
-
Every load
|
|
32
|
+
Every load works out the fingerprint again from the current source and gives a
|
|
33
33
|
freshness verdict:
|
|
34
34
|
|
|
35
35
|
- `valid` — source matches the snapshot.
|
|
36
|
-
- `stale` — the source node changed
|
|
36
|
+
- `stale` — the source node changed after capture.
|
|
37
37
|
- `absent` — the node, or a referenced artifact path, is gone/unreadable.
|
|
38
38
|
|
|
39
39
|
`stale` and `absent` both REFUSE diff/replay with a structured `NodeSnapshotError`
|
|
40
|
-
naming the divergence — never a
|
|
40
|
+
naming the divergence — never a quiet stale replay, never a best-effort partial.
|
|
41
41
|
|
|
42
42
|
## Diff — stable + structural
|
|
43
43
|
|
|
@@ -46,7 +46,7 @@ node diff <run-id> <baseline-snapshot-id> <candidate-snapshot-id> [--json]
|
|
|
46
46
|
```
|
|
47
47
|
|
|
48
48
|
Per-section (`status`/`inputs`/`outputs`/`artifacts`/`evidence`/`errors`/`links`/
|
|
49
|
-
`metadata`) `added|removed|changed|same`,
|
|
49
|
+
`metadata`) `added|removed|changed|same`, put in order deterministically by the same
|
|
50
50
|
`stableStringify` the eval comparison uses. Byte-identical across repeated runs.
|
|
51
51
|
|
|
52
52
|
## Replay — isolated + deterministic
|
|
@@ -55,10 +55,10 @@ Per-section (`status`/`inputs`/`outputs`/`artifacts`/`evidence`/`errors`/`links`
|
|
|
55
55
|
node replay <run-id> <snapshot-id> [--json]
|
|
56
56
|
```
|
|
57
57
|
|
|
58
|
-
|
|
58
|
+
Builds the normalized node again from the snapshot with `now` INJECTED — no
|
|
59
59
|
ambient `new Date()` in the deterministic payload. The result carries an
|
|
60
60
|
`outputFingerprint` over the normalized body, so two replays of one snapshot are
|
|
61
|
-
byte-identical (only `replayedAt`/`replayId`, which
|
|
61
|
+
byte-identical (only `replayedAt`/`replayId`, which come from now, are different).
|
|
62
62
|
Replaying a `stale`/`absent` snapshot fails closed.
|
|
63
63
|
|
|
64
64
|
## Verify — replay vs source
|
|
@@ -67,7 +67,7 @@ Replaying a `stale`/`absent` snapshot fails closed.
|
|
|
67
67
|
node verify <run-id> <replay-id> [--json]
|
|
68
68
|
```
|
|
69
69
|
|
|
70
|
-
Compares a replay to a FRESH snapshot of the source node and
|
|
70
|
+
Compares a replay to a FRESH snapshot of the source node and gives a pass/fail
|
|
71
71
|
verdict plus findings in the eval harness's `severity/category/reason/baselineRef/
|
|
72
72
|
replayRef` shape.
|
|
73
73
|
|
|
@@ -77,9 +77,9 @@ replayRef` shape.
|
|
|
77
77
|
capability registry as `surface: "both"`, so `cw node <verb> --json` and the
|
|
78
78
|
`cw_node_*` MCP tools render one core (`src/node-snapshot.ts`). Additive: no change
|
|
79
79
|
to `StateNode`, `STATE_NODE_SCHEMA_VERSION`, the run-state schema, the pipeline
|
|
80
|
-
contract, or existing eval-suite artifacts; pre-0.1.35 runs and snapshots
|
|
81
|
-
|
|
82
|
-
(`normalizeValue`/`stableStringify`/`lines`) and `fingerprintStrings` is purely
|
|
80
|
+
contract, or existing eval-suite artifacts; pre-0.1.35 runs and snapshots can still
|
|
81
|
+
be loaded. Making the once-private eval-harness helpers
|
|
82
|
+
(`normalizeValue`/`stableStringify`/`lines`) and `fingerprintStrings` public is purely
|
|
83
83
|
additive and changes no behavior.
|
|
84
84
|
|
|
85
85
|
## See Also
|
|
@@ -113,7 +113,7 @@ evidence grounding + durable audit append + symlink-hardened containment + deter
|
|
|
113
113
|
|
|
114
114
|
## Robust Result Ingest (v0.1.42)
|
|
115
115
|
|
|
116
|
-
capture findings/evidence from any reasonable agent shape (alt keys + prose), CW
|
|
116
|
+
capture findings/evidence from any reasonable agent shape (alt keys + prose), CW works out grounded evidence by itself, gives a warning on empty capture — closes the v0.1.41 live-drive 'accepted with 0 captured' failure
|
|
117
117
|
|
|
118
118
|
## No-False-Green Gate & Launch Prep (v0.1.43)
|
|
119
119
|
|
|
@@ -121,7 +121,7 @@ Hard gate blocking empty-capture verifier-gated commits, plus quickstart and lau
|
|
|
121
121
|
|
|
122
122
|
## Release-Gate Determinism & Agents Vendor (v0.1.44)
|
|
123
123
|
|
|
124
|
-
Release-readiness checks now
|
|
124
|
+
Release-readiness checks now check the committed blob (`git show HEAD:<path>`) instead of the mutable working tree — getting rid of false-red/false-green from working-tree writes that happen at the same time (iCloud/Spotlight/editor). Adds the `agents` vendor manifest target: a generated `.agents/plugins/cool-workflow/` adapter that gives any non-Claude AI agent one common interface to CW.
|
|
125
125
|
|
|
126
126
|
## P1-P2 Fixes & CI Content Surfaces (v0.1.49)
|
|
127
127
|
|
|
@@ -138,7 +138,15 @@ Migration DAG with reversible edges (v0.1.45), capability auto-discovery (v0.1.4
|
|
|
138
138
|
|
|
139
139
|
## Fast Architecture Review (v0.1.80)
|
|
140
140
|
|
|
141
|
-
Adds the opt-in fast architecture-review lane: scoped JSONL source contexts, diff-aware exports, reusable Map and Assess results,
|
|
141
|
+
Adds the opt-in fast architecture-review lane: scoped JSONL source contexts, diff-aware exports, reusable Map and Assess results, wrapper metrics you can measure, a background full-review handoff you can act on, and userland model policy flags for routing fast/strong workers without changing the full review contract.
|
|
142
142
|
|
|
143
143
|
_No changes to node-snapshot diff/replay in v0.1.81._
|
|
144
|
-
_No
|
|
144
|
+
_No change in behavior in v0.1.82 (the node projection field set was brought together into one source of truth; snapshot/replay digests are byte-identical)._
|
|
145
|
+
|
|
146
|
+
## Hardening and Onboarding (v0.1.83)
|
|
147
|
+
|
|
148
|
+
Loaders fail closed on corrupt state; store writes are made safe under more than one writer; a new cw doctor checks your setup; help lists every command; and the docs are put into Basic English.
|
|
149
|
+
|
|
150
|
+
## Privacy Release (v0.1.84)
|
|
151
|
+
|
|
152
|
+
No other change to this page in v0.1.84.
|
|
@@ -3,28 +3,28 @@
|
|
|
3
3
|
CW v0.1.31 adds Observability + Cost Accounting: time/duration, failure rate,
|
|
4
4
|
verifier pass rate, candidate acceptance rate, and token/cost — all DERIVED from
|
|
5
5
|
the run state CW already keeps. Before v0.1.31 there was no metrics module and no
|
|
6
|
-
token or cost field anywhere; run state already
|
|
6
|
+
token or cost field anywhere; run state already had `createdAt`/`updatedAt`/
|
|
7
7
|
`completedAt`/`dispatchedAt` and outcome statuses on tasks, workers, verifier
|
|
8
|
-
nodes, candidates, memberships, and feedback. This release
|
|
9
|
-
report — and adds an
|
|
10
|
-
|
|
11
|
-
taking
|
|
8
|
+
nodes, candidates, memberships, and feedback. This release turns those into a
|
|
9
|
+
report — and adds an added-on, host-attested usage record so cost can be
|
|
10
|
+
worked out in an honest way — without changing the `ResultEnvelope` schema and without
|
|
11
|
+
taking over the source truth.
|
|
12
12
|
|
|
13
|
-
The design
|
|
13
|
+
The design keeps to the same base-system observability idea as
|
|
14
14
|
[State Explosion Management](state-explosion-management.7.md) and the
|
|
15
15
|
[Run Registry / Control Plane](run-registry-control-plane.7.md):
|
|
16
16
|
|
|
17
17
|
- the per-run `.cw/runs/<id>/state.json` is the SINGLE source of truth
|
|
18
|
-
- metrics are a DERIVED
|
|
18
|
+
- metrics are a DERIVED view of source records, never a separate database
|
|
19
19
|
- no telemetry pipeline, no background collector daemon, no hidden counters
|
|
20
20
|
- plain files, stable JSON, deterministic output
|
|
21
|
-
- fail closed: a rate over zero samples is `n/a`, never a
|
|
22
|
-
- cost is ATTESTED, never measured or
|
|
23
|
-
- backward compatible; usage/cost fields are
|
|
21
|
+
- fail closed: a rate over zero samples is `n/a`, never a made-up 0%/100%
|
|
22
|
+
- cost is ATTESTED, never measured or made up; missing usage is `unreported`
|
|
23
|
+
- backward compatible; usage/cost fields are added-on and optional
|
|
24
24
|
|
|
25
25
|
## Derived, not a telemetry pipeline
|
|
26
26
|
|
|
27
|
-
Every number
|
|
27
|
+
Every number comes from durable state that is already there:
|
|
28
28
|
|
|
29
29
|
- durations come from recorded timestamps — `dispatchedAt`→`completedAt` for
|
|
30
30
|
tasks, `createdAt`→worker output `recordedAt` for workers, `createdAt`→
|
|
@@ -34,31 +34,31 @@ Every number is a projection of existing durable state:
|
|
|
34
34
|
total of those samples;
|
|
35
35
|
- the verifier pass rate counts `verifier` state nodes whose status is a pass
|
|
36
36
|
(`verified`/`committed`) against decided gates (pass + `failed`/`rejected`/
|
|
37
|
-
`blocked`); pending/running gates are
|
|
37
|
+
`blocked`); pending/running gates are not yet decided and left out;
|
|
38
38
|
- the candidate acceptance rate counts `selected`/`verified` candidates over all
|
|
39
39
|
candidate records.
|
|
40
40
|
|
|
41
41
|
There is no metrics store. `deriveMetricsReport(run, { now, policy })` is a PURE
|
|
42
42
|
function of one run's state, an injected `now`, and an optional pricing policy.
|
|
43
|
-
The only now-derived field is `generatedAt`; durations are
|
|
43
|
+
The only now-derived field is `generatedAt`; durations are worked out from recorded
|
|
44
44
|
timestamps, so a report over a fixed snapshot is byte-reproducible (eval/replay
|
|
45
|
-
agnostic). The per-run report is
|
|
45
|
+
agnostic). The per-run report is kept as a rebuildable, fingerprinted
|
|
46
46
|
snapshot under `.cw/runs/<id>/metrics/metrics-report.json`; the cross-repo
|
|
47
47
|
summary reports each run's snapshot freshness as `valid|stale|absent` against
|
|
48
|
-
current source — fail closed,
|
|
48
|
+
current source — fail closed, just like the registry.
|
|
49
49
|
|
|
50
50
|
## A counter you cannot trust is worse than none
|
|
51
51
|
|
|
52
52
|
Each rate is a `RateMetric` carrying `state` (`ok`/`n/a`), `count`, `total`,
|
|
53
53
|
`rate`, and per-bucket sample counts. Over zero samples the state is `n/a` and
|
|
54
|
-
`count`/`rate` are `null` — never `0`. No divide-by-zero, no
|
|
55
|
-
|
|
56
|
-
can
|
|
54
|
+
`count`/`rate` are `null` — never `0`. No divide-by-zero, and no part-data rate
|
|
55
|
+
shown as if it were complete. Sample counts and buckets come with every rate so a reader
|
|
56
|
+
can check the numerator and denominator.
|
|
57
57
|
|
|
58
58
|
## Cost is attested, never measured or fabricated
|
|
59
59
|
|
|
60
60
|
CW does not call the model; the host/worker does. Token usage is recorded as
|
|
61
|
-
HOST-ATTESTED provenance — a `UsageRecord`
|
|
61
|
+
HOST-ATTESTED provenance — a `UsageRecord` taken in on the existing intake path
|
|
62
62
|
and stored on the task or worker record (never on `ResultEnvelope`):
|
|
63
63
|
|
|
64
64
|
```
|
|
@@ -68,15 +68,15 @@ cw result <run-id> <task-id> <file> \
|
|
|
68
68
|
cw worker output <run-id> <worker-id> <file> --usage-input-tokens N --usage-output-tokens M --usage-model ID
|
|
69
69
|
```
|
|
70
70
|
|
|
71
|
-
CW records what the host attests,
|
|
72
|
-
host reports no usage the value is
|
|
73
|
-
|
|
74
|
-
carrying attested usage) and `usage.unreportedUnits` so the gap
|
|
71
|
+
CW records what the host attests, word for word, and makes up nothing. When the
|
|
72
|
+
host reports no usage the value is a clear `unreported` — never `0`, never a
|
|
73
|
+
quiet guess. The report shows `usage.coverage` (the part of work units
|
|
74
|
+
carrying attested usage) and `usage.unreportedUnits` so the gap can be seen.
|
|
75
75
|
|
|
76
|
-
A
|
|
76
|
+
A money figure is `attested` ONLY when it comes from attested usage × a
|
|
77
77
|
recorded pricing policy with an EXACT model match. When a model is priced by the
|
|
78
|
-
policy's `defaultPrice` fallback, that
|
|
79
|
-
and the cost `state` becomes `estimated`; the two USD figures are never
|
|
78
|
+
policy's `defaultPrice` fallback, that part is a SEPARATE `estimated` figure
|
|
79
|
+
and the cost `state` becomes `estimated`; the two USD figures are never mixed
|
|
80
80
|
into one. Cost states:
|
|
81
81
|
|
|
82
82
|
- `attested` — every attested model exact-matched a policy entry;
|
|
@@ -87,23 +87,23 @@ into one. Cost states:
|
|
|
87
87
|
## Mechanism vs policy: pricing is data
|
|
88
88
|
|
|
89
89
|
The runtime is MECHANISM: it records attested usage and derives rates/durations.
|
|
90
|
-
The pricing table is POLICY —
|
|
91
|
-
kernel. The same attested usage
|
|
90
|
+
The pricing table is POLICY — given as DATA (`CostPolicy`), not built into the
|
|
91
|
+
kernel. The same attested usage gives different cost reports under different
|
|
92
92
|
pricing without touching the runtime. A bundled EXAMPLE policy lives at
|
|
93
|
-
`manifest/pricing.policy.json` (USD per 1e6 tokens,
|
|
93
|
+
`manifest/pricing.policy.json` (USD per 1e6 tokens, a starting point you can edit —
|
|
94
94
|
not a live price feed); pass `--pricing <path>` to use your own, or
|
|
95
|
-
`--pricing default` for the bundled example. With no policy
|
|
95
|
+
`--pricing default` for the bundled example. With no policy given, cost is
|
|
96
96
|
`unpriced`/`unreported`, never guessed.
|
|
97
97
|
|
|
98
98
|
## One source, every surface
|
|
99
99
|
|
|
100
100
|
The metrics verbs are declared once in `src/capability-registry.ts`, so the CLI
|
|
101
|
-
and MCP surfaces are two
|
|
101
|
+
and MCP surfaces are two views of one core (`src/observability.ts`) and pass
|
|
102
102
|
the v0.1.27 parity gate — `cw <cmd> --json` is byte-identical to `cw_<tool>`
|
|
103
103
|
(durations are integers from recorded timestamps; only the ISO `generatedAt` is
|
|
104
|
-
now-derived and
|
|
104
|
+
now-derived and made neutral by the parity probe). The v0.1.30 Workbench renders a
|
|
105
105
|
read-only metrics panel from the same payload, showing coverage and
|
|
106
|
-
`unreported`/`n/a`
|
|
106
|
+
`unreported`/`n/a` in an honest way — it shows nothing the CLI/MCP cannot.
|
|
107
107
|
|
|
108
108
|
## Commands
|
|
109
109
|
|
|
@@ -112,11 +112,11 @@ read-only metrics panel from the same payload, showing coverage and
|
|
|
112
112
|
the canonical payload; `--pricing <path>|default` to price attested usage.
|
|
113
113
|
- `cw metrics summary` — the cross-repo rollup over the v0.1.28 run registry:
|
|
114
114
|
pooled rates, summed attested usage/cost with coverage, and per-app and
|
|
115
|
-
per-backend breakdowns. `--scope repo|home`;
|
|
116
|
-
(`unreadableRuns`), never
|
|
115
|
+
per-backend breakdowns. `--scope repo|home`; runs that cannot be read are counted
|
|
116
|
+
(`unreadableRuns`), never quietly dropped.
|
|
117
117
|
|
|
118
|
-
MCP hosts call `cw_metrics_show` and `cw_metrics_summary` with the
|
|
119
|
-
payloads. Old runs load and report `unreported` cost while still
|
|
118
|
+
MCP hosts call `cw_metrics_show` and `cw_metrics_summary` with the same
|
|
119
|
+
payloads. Old runs load and report `unreported` cost while still giving correct
|
|
120
120
|
time and rate metrics from their existing timestamps and outcomes.
|
|
121
121
|
|
|
122
122
|
This document targets CW 0.1.31.
|
|
@@ -127,8 +127,8 @@ This document targets CW 0.1.31.
|
|
|
127
127
|
v0.1.32 adds Team Collaboration: a host-attested actor and append-only
|
|
128
128
|
approvals/rejections/comments/handoffs provenance-linked to a durable target,
|
|
129
129
|
plus a review gate that STACKS ON the verifier gate — required approvals from
|
|
130
|
-
authorized roles,
|
|
131
|
-
and never
|
|
130
|
+
authorized roles, made to hold inside `resolveCommitGate` AFTER the verifier checks
|
|
131
|
+
and never in place of them, failing closed on quorum/authority/self-approval and
|
|
132
132
|
recording who approved the very artifact that shipped. Policy (required approvals,
|
|
133
133
|
authorized roles, self-approval) is data, default off (pre-v0.1.32 behavior
|
|
134
134
|
unchanged). The verbs are parity-gated and render read-only in the v0.1.30
|
|
@@ -136,11 +136,11 @@ Workbench. See [Team Collaboration](team-collaboration.7.md).
|
|
|
136
136
|
|
|
137
137
|
## Release Tooling (v0.1.33)
|
|
138
138
|
|
|
139
|
-
the per-tag mechanical surfaces (version bump across 17 surfaces, feature scaffold, and the forward-reference docs) become deterministic scripts, with a
|
|
139
|
+
the per-tag mechanical surfaces (version bump across 17 surfaces, feature scaffold, and the forward-reference docs) become deterministic scripts, with a release gate that drops repeats. See release-tooling(7).
|
|
140
140
|
|
|
141
141
|
## Real Execution Backend Integrations (v0.1.34)
|
|
142
142
|
|
|
143
|
-
container/remote/ci backends really execute (docker/podman run, remote/CI POST-and-poll) under the sandbox contract, with byte-stable evidence vs node and fail-closed refusal when a runtime/endpoint
|
|
143
|
+
container/remote/ci backends really execute (docker/podman run, remote/CI POST-and-poll) under the sandbox contract, with byte-stable evidence vs node and fail-closed refusal when a runtime/endpoint cannot be reached. See real-execution-backends(7).
|
|
144
144
|
|
|
145
145
|
## Node Snapshot / Diff / Replay (v0.1.35)
|
|
146
146
|
|
|
@@ -172,7 +172,7 @@ evidence grounding + durable audit append + symlink-hardened containment + deter
|
|
|
172
172
|
|
|
173
173
|
## Robust Result Ingest (v0.1.42)
|
|
174
174
|
|
|
175
|
-
capture findings/evidence from any reasonable agent shape (alt keys + prose), CW derives grounded evidence itself,
|
|
175
|
+
capture findings/evidence from any reasonable agent shape (alt keys + prose), CW derives grounded evidence itself, give a warning on empty capture — closes the v0.1.41 live-drive 'accepted with 0 captured' failure
|
|
176
176
|
|
|
177
177
|
## No-False-Green Gate & Launch Prep (v0.1.43)
|
|
178
178
|
|
|
@@ -180,7 +180,7 @@ Hard gate blocking empty-capture verifier-gated commits, plus quickstart and lau
|
|
|
180
180
|
|
|
181
181
|
## Release-Gate Determinism & Agents Vendor (v0.1.44)
|
|
182
182
|
|
|
183
|
-
Release-readiness checks now validate the committed blob (`git show HEAD:<path>`) instead of the mutable working tree —
|
|
183
|
+
Release-readiness checks now validate the committed blob (`git show HEAD:<path>`) instead of the mutable working tree — taking away false-red/false-green from concurrent working-tree writes (iCloud/Spotlight/editor). Adds the `agents` vendor manifest target: a generated `.agents/plugins/cool-workflow/` adapter giving any non-Claude AI agent one common interface to CW.
|
|
184
184
|
|
|
185
185
|
## P1-P2 Fixes & CI Content Surfaces (v0.1.49)
|
|
186
186
|
|
|
@@ -197,7 +197,15 @@ Migration DAG with reversible edges (v0.1.45), capability auto-discovery (v0.1.4
|
|
|
197
197
|
|
|
198
198
|
## Fast Architecture Review (v0.1.80)
|
|
199
199
|
|
|
200
|
-
Adds the opt-in fast architecture-review lane: scoped JSONL source contexts, diff-aware exports, reusable Map and Assess results, measurable wrapper metrics,
|
|
200
|
+
Adds the opt-in fast architecture-review lane: scoped JSONL source contexts, diff-aware exports, reusable Map and Assess results, measurable wrapper metrics, background full-review handoff you can act on, and userland model policy flags for routing fast/strong workers without changing the full review contract.
|
|
201
201
|
|
|
202
|
-
_No changes to the observability + cost-accounting surface in v0.1.81 (the observability module was
|
|
202
|
+
_No changes to the observability + cost-accounting surface in v0.1.81 (the observability module was split into behavior-preserving siblings; output is byte-identical)._
|
|
203
203
|
_No changes in v0.1.82._
|
|
204
|
+
|
|
205
|
+
## Hardening and Onboarding (v0.1.83)
|
|
206
|
+
|
|
207
|
+
Loaders fail closed on corrupt state; store writes are made safe under more than one writer; a new cw doctor checks your setup; help lists every command; and the docs are put into Basic English.
|
|
208
|
+
|
|
209
|
+
## Privacy Release (v0.1.84)
|
|
210
|
+
|
|
211
|
+
No other change to this page in v0.1.84.
|