@journeykit/canon 0.9.1 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
|
@@ -51,6 +51,36 @@ diagnostic, at `canon inventory sync`.
|
|
|
51
51
|
attest it. If you can't, it is a `process` rule, or it is a gap — record
|
|
52
52
|
the gap, do not write the scenario.
|
|
53
53
|
|
|
54
|
+
## Cases: which path of the behavior is this?
|
|
55
|
+
|
|
56
|
+
A second, independent axis, tagged `@case:<value>`:
|
|
57
|
+
|
|
58
|
+
| Case | The scenario specifies |
|
|
59
|
+
|---|---|
|
|
60
|
+
| `happy` | The expected path: valid input, the outcome the user wants. |
|
|
61
|
+
| `failure` | A refusal, an error, a denied or blocked action — what happens when the input is wrong, missing, unauthorized, or stale. |
|
|
62
|
+
| `edge` | A boundary that is neither: empty sets, limits, duplicates, re-runs, ordering. |
|
|
63
|
+
|
|
64
|
+
A spec can be fully attested and still describe only its golden path.
|
|
65
|
+
Evidence presence cannot see that gap; `require_cases` can:
|
|
66
|
+
|
|
67
|
+
```yaml
|
|
68
|
+
# .canon/policy.yaml
|
|
69
|
+
spec_coverage:
|
|
70
|
+
require_evidence: true
|
|
71
|
+
scope: [building, verifying]
|
|
72
|
+
require_cases: [failure]
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Every feature surface (`<area>.<surface>`) with in-scope scenarios must
|
|
76
|
+
then carry at least one scenario of each listed case, or `canon gate
|
|
77
|
+
check` reports it as `uncovered-cell`, and `canon subject status <id>
|
|
78
|
+
shipped` refuses for a subject that owns such a surface. An untagged
|
|
79
|
+
scenario counts toward its surface but satisfies no case, so an
|
|
80
|
+
untagged corpus reports every surface instead of passing silently. The
|
|
81
|
+
base set comes from `canon.core`; a repo replaces it with its own `case`
|
|
82
|
+
enum, like `lane`.
|
|
83
|
+
|
|
54
84
|
## Subject pinning
|
|
55
85
|
|
|
56
86
|
Every scenario belongs to a Subject (the durable product unit, see
|
|
@@ -64,7 +94,9 @@ a scoped gate.
|
|
|
64
94
|
```bash
|
|
65
95
|
canon feature new checkout.cart --title "Cart" # once per surface
|
|
66
96
|
canon scenario new checkout.cart.01 --title "Adding an item raises the count" \
|
|
67
|
-
--subject checkout-core --lane behavior
|
|
97
|
+
--subject checkout-core --lane behavior --case happy # once per behavior
|
|
98
|
+
canon scenario new checkout.cart.04 --title "Adding an out-of-stock item is refused" \
|
|
99
|
+
--subject checkout-core --lane behavior --case failure # and its failure path
|
|
68
100
|
canon format specs # clean?
|
|
69
101
|
canon inventory sync # index it
|
|
70
102
|
```
|
|
@@ -78,6 +110,7 @@ canon inventory sync # index it
|
|
|
78
110
|
# canon: {"schema":1,"at":"2026-09-16T10:02:11Z","actor":{"agent_id":"Main"}}
|
|
79
111
|
@subject:checkout-core
|
|
80
112
|
@lane:behavior
|
|
113
|
+
@case:happy
|
|
81
114
|
@checkout.cart.01
|
|
82
115
|
Scenario: Adding an item raises the count
|
|
83
116
|
Given an empty cart
|
|
@@ -107,9 +140,13 @@ canon inventory sync # index it
|
|
|
107
140
|
- **Coverage by splitting.** One behavior as two scenarios so two rows go
|
|
108
141
|
green. One behavior, one id.
|
|
109
142
|
- **Runner tags as classification.** `@design`, `@p2`, `@wip` are
|
|
110
|
-
invisible to canon — they index nothing. Only `@lane
|
|
111
|
-
are read; any other `@name:value` namespace is counted as a
|
|
112
|
-
at sync so you can see it was dropped.
|
|
143
|
+
invisible to canon — they index nothing. Only `@lane:`, `@case:`, and
|
|
144
|
+
`@subject:` are read; any other `@name:value` namespace is counted as a
|
|
145
|
+
diagnostic at sync so you can see it was dropped.
|
|
146
|
+
- **The golden-path-only surface.** Five scenarios, all of the form
|
|
147
|
+
"doing X works". Nothing says what happens on a missing field, a
|
|
148
|
+
denied user, a duplicate submit. Write the refusal as its own
|
|
149
|
+
`@case:failure` scenario; `require_cases` makes the gap a gate failure.
|
|
113
150
|
|
|
114
151
|
## Worked example
|
|
115
152
|
|
|
@@ -21,7 +21,7 @@ Every violation carries one of these stable, grep-able strings:
|
|
|
21
21
|
|
|
22
22
|
| Class | Meaning |
|
|
23
23
|
|---|---|
|
|
24
|
-
| `uncovered-cell` | Either a policy-required evidence cell (role × artifact) with no matching record, or — with `spec_coverage` enabled — a spec scenario that is unimplemented or mismatched. The detail string distinguishes them. Coverage means "a test exists", not "a test passed": even a `Divergent` verdict satisfies the role-cell form. |
|
|
24
|
+
| `uncovered-cell` | Either a policy-required evidence cell (role × artifact) with no matching record, or — with `spec_coverage` enabled — a spec scenario that is unimplemented or mismatched, or a feature surface missing a `require_cases` case. The detail string distinguishes them. Coverage means "a test exists", not "a test passed": even a `Divergent` verdict satisfies the role-cell form. |
|
|
25
25
|
| `unreviewed-promotion` | An artifact tagged `reviewed` has no matching ledger review record. |
|
|
26
26
|
| `trust-below-required` | Achieved trust level is below `policy.yaml`'s `trust_required` for its class — RELEASE-scoped only (`canon gate check --release`); never fires on an ordinary run. |
|
|
27
27
|
| `stale-evidence` | A passing record degraded to stale: its declared surface changed since its `evidence_sha`, or HEAD moved past `staleness.max_commits_behind`. Only degrades an already-green record. |
|
|
@@ -63,6 +63,8 @@ an unimplemented or mismatched spec becomes a reported violation.
|
|
|
63
63
|
spec_coverage:
|
|
64
64
|
require_evidence: true
|
|
65
65
|
scope: [building, verifying] # optional; omit for the whole corpus
|
|
66
|
+
exclude_lanes: [process] # optional
|
|
67
|
+
require_cases: [failure] # optional; see Golden-path only below
|
|
66
68
|
```
|
|
67
69
|
|
|
68
70
|
- **Unimplemented** — no `EvidenceRecord` carries the scenario's
|
|
@@ -78,6 +80,15 @@ spec_coverage:
|
|
|
78
80
|
set, an untagged scenario is out of scope; with `scope` omitted, every
|
|
79
81
|
scenario is in scope. A tag naming no Subject record is reported as a
|
|
80
82
|
dangling link, never silently skipped.
|
|
83
|
+
- **Golden-path only** (`require_cases`) — every feature surface
|
|
84
|
+
(`<area>.<surface>`) among the in-scope scenarios must carry at least
|
|
85
|
+
one scenario tagged `@case:<v>` for each listed value, or the SURFACE
|
|
86
|
+
is reported (subject `<area>.<surface>`, detail naming the missing
|
|
87
|
+
case). This is the gap evidence presence cannot see: every scenario
|
|
88
|
+
attested, none of them specifying a refusal. Untagged scenarios satisfy
|
|
89
|
+
no case, so an unclassified corpus is reported, not passed. Works with
|
|
90
|
+
`require_evidence: false` too. `canon subject status <id> shipped`
|
|
91
|
+
applies the same rule to the scenarios the subject owns.
|
|
81
92
|
|
|
82
93
|
Two refusals rather than a silent pass, both surfacing as
|
|
83
94
|
`uncovered-cell` on the subject `spec_coverage`:
|
|
@@ -93,6 +104,54 @@ Malformed rows in those three kinds do NOT surface as
|
|
|
93
104
|
`malformed-evidence`; they are kept off the evidence violation set on
|
|
94
105
|
purpose, so enabling nothing changes no existing verdict.
|
|
95
106
|
|
|
107
|
+
## Experimental: evidence binding (`experimental.evidence_binding`, off by default)
|
|
108
|
+
|
|
109
|
+
An evidence record is an attestation: canon never runs `--ref`, and it
|
|
110
|
+
never will — it cannot run every language and every agent-driven QA
|
|
111
|
+
tool a team uses. Binding lets a record point at files the team's own
|
|
112
|
+
runner or agent ALREADY produced, so a reviewer (and the gate) can check
|
|
113
|
+
what backed the claim:
|
|
114
|
+
|
|
115
|
+
```bash
|
|
116
|
+
# any file: a Playwright trace, screenshots, an agent QA log — by sha256
|
|
117
|
+
canon evidence add --scenario-id cart.add.04 --project-id root \
|
|
118
|
+
--kind agent-qa --ref "stagehand run 7" --role implementer \
|
|
119
|
+
--artifact test-results/cart-trace.zip
|
|
120
|
+
# a JUnit XML or Cucumber JSON report: also records the matched case + outcome
|
|
121
|
+
canon evidence add ... --report junit:target/nextest/default/junit.xml
|
|
122
|
+
canon evidence add ... --report cucumber:reports/cucumber.json
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
The case is found by the scenario id in its name, classname, or Cucumber
|
|
126
|
+
tag — dotted (`cart.add.04`) or underscored (`cart_add_04`, the form a
|
|
127
|
+
test function carries) — or named exactly with `--report-case <name>`
|
|
128
|
+
(a Rust test by its function name). No match refuses (exit 2); a
|
|
129
|
+
`faithful` verdict over a failed or skipped case refuses (exit 1). Files
|
|
130
|
+
must be inside the repository and are recorded by relative path.
|
|
131
|
+
|
|
132
|
+
Strength, weakest first: `attested` (no attachment) < `artifact` (any
|
|
133
|
+
bound file) < `report` (a parsed report whose case passed). The policy
|
|
134
|
+
decides whether the gate cares:
|
|
135
|
+
|
|
136
|
+
```yaml
|
|
137
|
+
experimental:
|
|
138
|
+
evidence_binding:
|
|
139
|
+
mode: warn # off (default) | warn | require
|
|
140
|
+
strength: artifact # artifact (default) | report
|
|
141
|
+
case: [failure] # optional filters — only these scenarios are held
|
|
142
|
+
lane: [behavior]
|
|
143
|
+
scope: [verifying] # subject statuses
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
`off` checks nothing. `warn` prints the distribution and lists each
|
|
147
|
+
scenario below `strength` as `warn <id>` after the gate result, never
|
|
148
|
+
failing it. `require` reports each as `uncovered-cell`. The latest
|
|
149
|
+
evidence record per scenario decides; scenarios with no evidence are
|
|
150
|
+
`spec_coverage`'s finding, not this one's. A malformed section is a
|
|
151
|
+
violation in every mode. Canon still cannot know whether the bound test
|
|
152
|
+
actually exercises the scenario — binding narrows the trust gap, it does
|
|
153
|
+
not close it.
|
|
154
|
+
|
|
96
155
|
## Effect-aware risk approvals (`risk_tiers`)
|
|
97
156
|
|
|
98
157
|
`risk_tiers` is absent by default and remains a no-op when absent. When it
|
|
@@ -110,11 +169,14 @@ binding. This is still attestation input, not proof of the changed paths;
|
|
|
110
169
|
run/diff metadata is authoritative when available, and an unavailable
|
|
111
170
|
binding is a gate violation rather than a clean result.
|
|
112
171
|
|
|
113
|
-
Approval identity is separate from attestation text.
|
|
114
|
-
|
|
115
|
-
`
|
|
116
|
-
|
|
117
|
-
|
|
172
|
+
Approval identity is separate from attestation text. An approval counts
|
|
173
|
+
only when it carries a detached SSH signature (`ssh-keygen -Y sign -n
|
|
174
|
+
canon-approval-v1` over `canon evidence approval-payload`'s bytes) that
|
|
175
|
+
verifies against the signers pinned by policy `approval.allowed_signers`,
|
|
176
|
+
with `--approval-role human`. `CANON_ACTOR`, caller-supplied strings, and
|
|
177
|
+
`agent` roles never satisfy a risk tier, and canon never signs on a
|
|
178
|
+
user's behalf.
|
|
179
|
+
|
|
118
180
|
Malformed `risk_tiers` sections produce a stable `uncovered-cell` policy
|
|
119
181
|
violation (`risk_tiers policy-invalid: ...`) instead of degrading to an
|
|
120
182
|
empty clean map.
|
|
@@ -42,9 +42,9 @@ Per configured root, in order:
|
|
|
42
42
|
2. **Scan** — walks `<root>/features/**/*.feature`, pairing each
|
|
43
43
|
`@<area>.<surface>.<nn>` tag with its header label as `title` and a
|
|
44
44
|
`source_digest` (sha256 over the raw `.feature` bytes). `@subject:<id>`
|
|
45
|
-
→ `subject_id`, `@lane:<v>` → `lane` (first tag
|
|
46
|
-
malformed value, or a value outside a declared
|
|
47
|
-
with a counted, non-fatal diagnostic). Any other `@name:value`
|
|
45
|
+
→ `subject_id`, `@lane:<v>` → `lane`, `@case:<v>` → `case` (first tag
|
|
46
|
+
wins; a second, a malformed value, or a value outside a declared
|
|
47
|
+
`lane`/`case` enum is dropped with a counted, non-fatal diagnostic). Any other `@name:value`
|
|
48
48
|
namespace is counted as a diagnostic so a dropped classification is
|
|
49
49
|
visible; plain tags (`@p2`) are ignored. The index derives from the
|
|
50
50
|
`.feature` corpus alone. See `canon-authoring` for the organizing rule.
|
|
@@ -113,7 +113,16 @@ not proof that the caller's path is authoritative; configured rules cannot
|
|
|
113
113
|
be bypassed by declaring a safe ref.
|
|
114
114
|
|
|
115
115
|
Approval text is also only an attestation unless identity is verified.
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
116
|
+
An approval counts only with a detached SSH signature that verifies
|
|
117
|
+
against the signers pinned by `approval.allowed_signers`, and
|
|
118
|
+
`--approval-role human`. `CANON_ACTOR`, arbitrary caller strings, and
|
|
119
|
+
agent roles do not count.
|
|
120
|
+
|
|
121
|
+
## Experimental: `experimental.evidence_binding`
|
|
122
|
+
|
|
123
|
+
Off by default. Holds scenario evidence to a minimum strength —
|
|
124
|
+
`artifact` (a file bound by `canon evidence add --artifact`) or `report`
|
|
125
|
+
(a JUnit/Cucumber report whose matched case passed, `--report`) — in
|
|
126
|
+
`mode: warn` (advisories only) or `mode: require` (gate violations),
|
|
127
|
+
optionally narrowed by `case`, `lane`, and subject-status `scope`. See
|
|
128
|
+
`canon-gate` for the full contract.
|
|
@@ -132,11 +132,14 @@ what brings a scenario into that scope. See `canon-gate`.
|
|
|
132
132
|
requires that the Subject OWNS at least one scenario (one whose latest
|
|
133
133
|
synced generation carries `@subject:<id>`), and that EVERY owned
|
|
134
134
|
scenario carries a latest, non-`Divergent` verdict in the ledger (the
|
|
135
|
-
same last-wins rule `canon gate check` uses).
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
135
|
+
same last-wins rule `canon gate check` uses). With
|
|
136
|
+
`spec_coverage.require_cases` set, every feature surface the Subject
|
|
137
|
+
owns must also carry a scenario of each required `@case:` (e.g. one
|
|
138
|
+
`@case:failure`) — attested golden-path scenarios alone do not ship. A
|
|
139
|
+
Subject with no tagged scenario is refused rather than shipped on an
|
|
140
|
+
empty set; so is one whose `scenario` records route away from the
|
|
141
|
+
gate's rung. Each refusal prints by failure class (`uncovered-cell`),
|
|
142
|
+
exits 1, and the status stays `verifying`. `retired` is not gated.
|
|
140
143
|
|
|
141
144
|
## Adopt flow
|
|
142
145
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@journeykit/canon",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.11.0",
|
|
4
4
|
"description": "canon — harness knowledge substrate. JS launcher: platform detection + native binary resolution.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"type": "module",
|
|
@@ -30,9 +30,9 @@
|
|
|
30
30
|
},
|
|
31
31
|
"dependencies": {},
|
|
32
32
|
"optionalDependencies": {
|
|
33
|
-
"@journeykit/canon-core-darwin-arm64": "0.
|
|
34
|
-
"@journeykit/canon-core-linux-x64": "0.
|
|
35
|
-
"@journeykit/canon-core-win32-x64": "0.
|
|
33
|
+
"@journeykit/canon-core-darwin-arm64": "0.11.0",
|
|
34
|
+
"@journeykit/canon-core-linux-x64": "0.11.0",
|
|
35
|
+
"@journeykit/canon-core-win32-x64": "0.11.0"
|
|
36
36
|
},
|
|
37
37
|
"devDependencies": {
|
|
38
38
|
"@types/node": "^20.0.0",
|