@journeykit/canon 0.14.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/skills/SKILL.src.md +92 -30
- package/dist/skills/reference/canon-authoring.md +10 -3
- package/dist/skills/reference/canon-gate.md +238 -19
- package/dist/skills/reference/canon-inventory.md +35 -8
- package/dist/skills/reference/canon-policy.md +114 -7
- package/dist/skills/reference/canon-review.md +198 -21
- package/dist/skills/reference/canon-storage.md +124 -1
- package/dist/skills/reference/canon-subject.md +173 -45
- package/package.json +4 -4
package/dist/skills/SKILL.src.md
CHANGED
|
@@ -13,60 +13,122 @@ where each subject stands and lists the next commands.
|
|
|
13
13
|
## The working loop
|
|
14
14
|
|
|
15
15
|
Follow these steps in order for every request that builds, changes or fixes
|
|
16
|
-
a feature, a one-line request included.
|
|
17
|
-
|
|
18
|
-
- **The
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
16
|
+
a feature, a one-line request included. A human is needed once per scope:
|
|
17
|
+
|
|
18
|
+
- **The approval** (step 1). A new subject, or a change to an approved
|
|
19
|
+
subject's scope, needs the human to approve its brief: the request, your
|
|
20
|
+
assumptions, your open questions. Write the brief, then stop and report
|
|
21
|
+
it: build nothing for that scope (no scenarios, code or evidence) before
|
|
22
|
+
the human answers. The approval is the one thing you never assume, even
|
|
23
|
+
when the human said they cannot answer until you report: the brief is
|
|
24
|
+
that report. In the same answer the human approves the policy,
|
|
25
|
+
`.canon/policy.yaml` (written by `canon init`), which is what the gate
|
|
26
|
+
grades the work against. When the human answers, record both at once:
|
|
27
|
+
an `Approved: <who> <when> — "<the human's words>"` line in the change's
|
|
28
|
+
`proposal.md`, and
|
|
29
|
+
`canon policy approve --by <the human> --note "<their words>"`. Never
|
|
30
|
+
approve without a human answer, and never edit the policy to make the
|
|
31
|
+
gate pass.
|
|
32
|
+
- **Inside an approved scope, never stop for a human.** Fixes, follow-ups,
|
|
33
|
+
review rounds and re-reviews go straight on: add or extend the scenario,
|
|
34
|
+
implement, gather evidence, and get an independent review in another
|
|
35
|
+
session. Independent review happens on every change. A fix or follow-up
|
|
36
|
+
stays in the approved change: add it as tasks in that change's
|
|
37
|
+
`tasks.md`, never with `canon change new`, because a new change is a new
|
|
38
|
+
scope and needs the human's approval.
|
|
39
|
+
|
|
40
|
+
When no human can answer one of the brief's questions, write the question
|
|
41
|
+
into the brief, state the assumption you will build on, and say so in your
|
|
42
|
+
report. Proceeding on an assumption applies to the brief's questions and to
|
|
43
|
+
work inside an approved scope, never to the approval itself.
|
|
44
|
+
|
|
45
|
+
1. **Brief.** For a new subject or a scope change, write down the request
|
|
46
|
+
verbatim, your assumptions, and the questions you could not ask. It
|
|
47
|
+
becomes the change's `openspec/changes/<slug>/proposal.md`, which
|
|
29
48
|
`canon change new <slug> --subject <id> --title "…"` scaffolds once the
|
|
30
|
-
subject exists (step 2).
|
|
49
|
+
subject exists (step 2). Once it is there, stop and report the brief,
|
|
50
|
+
asking the human once for the brief and the policy; go on to step 3
|
|
51
|
+
only after the human answers, and record the answer as above. Inside an
|
|
52
|
+
approved scope there is no new brief: go to step 3 under the existing
|
|
53
|
+
subject and change.
|
|
31
54
|
2. **Subject.** `canon subject new <id> --domain <domain> --title "…"`, then
|
|
32
55
|
`canon change new <slug> --subject <id> --title "…"`, and put the brief in
|
|
33
|
-
the change's `proposal.md`.
|
|
56
|
+
the change's `proposal.md`. For a new subject or a scope change, this is
|
|
57
|
+
where you stop until the human answers (step 1).
|
|
34
58
|
3. **Scenarios.** `canon feature new <area>.<surface> --title "…"`, then one
|
|
35
59
|
`canon scenario new <area>.<surface>.<nn> --subject <id> --case <case> --title "…"`
|
|
36
60
|
per behavior, with failure paths (`--case failure`) as well as `happy`
|
|
37
61
|
and `edge`. When the gate asks for a missing case (`require_cases`), add
|
|
38
|
-
a new scenario; never retag an existing one to satisfy it.
|
|
39
|
-
|
|
62
|
+
a new scenario; never retag an existing one to satisfy it. There is no
|
|
63
|
+
sync or format step: `canon status`, `canon gate check`,
|
|
64
|
+
`canon evidence add`, `canon review add` and `canon subject status`
|
|
65
|
+
index added, edited or deleted `.feature` files themselves. Only when
|
|
66
|
+
one of them prints `NOT refreshed` (a format violation, such as a
|
|
67
|
+
hand-written header with no `# canon:` provenance comment, or a
|
|
68
|
+
duplicate id) run `canon inventory sync`: it names each problem; fix
|
|
69
|
+
them, and the next command indexes the files.
|
|
40
70
|
4. **Units.** Split the work into units (for example specs, domain code,
|
|
41
71
|
assets, review). Give each unit its own actor id and session id, and pass
|
|
42
72
|
them on everything the unit records (`--actor-id <unit> --session-id <session>`),
|
|
43
73
|
so the ledger shows which unit did what. A delegated unit attests its own
|
|
44
74
|
work; never attest another unit's work under your id.
|
|
45
75
|
5. **Implement.** Write the code with tests whose names carry the scenario
|
|
46
|
-
id (`game.run.01: …`), so a test report binds to its scenario.
|
|
76
|
+
id (`game.run.01: …`), so a test report binds to its scenario. Record a
|
|
77
|
+
defect you find in your own work as a finding the moment you find it,
|
|
78
|
+
before review:
|
|
79
|
+
`canon finding add --change-id <slug> --round <n> --seq <n> --severity <s> --reviewer <me> --actor-id <me> --session-id <session> --summary "…"`.
|
|
47
80
|
6. **Evidence.** Per scenario:
|
|
48
81
|
`canon evidence add --project-id <root-id> --scenario-id <id> --kind test-run --role implementer --ref "<command>" --actor-id <unit> --session-id <session>`,
|
|
49
82
|
with `--report junit:<path>` or `--artifact <path>` when files prove it.
|
|
50
83
|
canon stores those bytes in `.canon/artifacts/sha256/` (commit it with the
|
|
51
|
-
ledger) and `canon gate check` re-checks them.
|
|
84
|
+
ledger) and `canon gate check` re-checks them. For a scenario about what
|
|
85
|
+
users see, bind the screenshot or recording that shows it
|
|
86
|
+
(`--artifact <path>`). Under `write: { promote: auto }` (what
|
|
87
|
+
`canon init` writes) the record is final when the command returns; only
|
|
88
|
+
in a repo with `write.promote: manual` (or no `write:` key) follow it
|
|
89
|
+
with `canon gate promote`.
|
|
90
|
+
An external evaluator's audit is recorded with `canon evidence add --kind
|
|
91
|
+
audit`; canon never generates one.
|
|
92
|
+
Evidence for a plan task (`--task <id>`, before `canon gate task`) must
|
|
93
|
+
bind the artifact or report that proves the task; a canon command
|
|
94
|
+
(`canon status`, `canon gate check`) is never that proof. Both read the
|
|
95
|
+
change's `tasks.md` as it is now, so a task row you add needs no
|
|
96
|
+
`canon ingest plans` first.
|
|
52
97
|
7. **Independent review.** A different session from the one that wrote the
|
|
53
|
-
code and evidence reviews the change.
|
|
54
|
-
|
|
55
|
-
`canon finding add --change-id <slug> --round <n> --seq <n> --severity <blocker|should-fix|note> --reviewer <me> --actor-id <me> --session-id <session> --summary "…"
|
|
56
|
-
|
|
98
|
+
code and evidence reviews the change. The reviewer records every defect
|
|
99
|
+
the review finds as a finding:
|
|
100
|
+
`canon finding add --change-id <slug> --round <n> --seq <n> --severity <blocker|should-fix|note> --reviewer <me> --actor-id <me> --session-id <session> --summary "…"`;
|
|
101
|
+
once fixed,
|
|
57
102
|
`canon finding close --change-id <slug> --round <n> --seq <n> --disposition fixed --resolution-sha <sha>`
|
|
58
|
-
|
|
103
|
+
(a short sha is fine; canon resolves it). With `write.promote: manual`,
|
|
104
|
+
follow each with `canon gate promote`. Every bug fix adds or extends a scenario. The
|
|
59
105
|
reviewer signs off each scenario:
|
|
60
106
|
`canon review add --project-id <root-id> --scenario-id <id> --reviewer <who> --actor-id <who> --role reviewer --session-id <review-session> --pin <sha> --upstream-ref <ref>`
|
|
61
|
-
(written directly; nothing to promote).
|
|
107
|
+
(written directly; nothing to promote). A reviewer of user-visible
|
|
108
|
+
behavior attaches what it looked at with `--artifact <path>`
|
|
109
|
+
(repeatable); canon stores and re-checks those bytes and never compares
|
|
110
|
+
images.
|
|
62
111
|
8. **Transition.** `canon subject status <id> <state>` along
|
|
63
112
|
`proposed → specced → building → verifying → shipped`. It refuses a move
|
|
64
113
|
the policy does not yet allow (missing evidence, missing reviews, open
|
|
65
114
|
blocker findings); fix the gap rather than overriding, and finish with
|
|
66
|
-
`canon gate check` clean.
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
`canon
|
|
115
|
+
`canon gate check` clean. A root whose `.feature` edits could not be
|
|
116
|
+
indexed fails the gate and refuses `shipped` (`uncovered-cell <root>`;
|
|
117
|
+
see step 3). `shipped` also refuses while the change's `proposal.md`
|
|
118
|
+
has no `Approved:` line, and `canon evidence add` refuses the subject's
|
|
119
|
+
scenarios and tasks the same way. Both are backstops: the stop itself
|
|
120
|
+
is at step 1, so you never reach them for a brief the human has not
|
|
121
|
+
answered. You record the line only from the human's answer (with no
|
|
122
|
+
`proposal.md` there is no brief to answer: neither refuses, and the
|
|
123
|
+
ship guard only prints an advisory). Move the subject to
|
|
124
|
+
`shipped` yourself once `canon subject status <id> shipped` passes its
|
|
125
|
+
guard, unless the brief says the human ships. `canon status` shows
|
|
126
|
+
whether each subject's brief is approved, by the same check, and names
|
|
127
|
+
the next step.
|
|
128
|
+
|
|
129
|
+
Every write command says whether the record is final (`written directly`,
|
|
130
|
+
or `staged and promoted in this run`) or staged; only a staged record
|
|
131
|
+
needs `canon gate promote`, and only `write.promote: manual` leaves one.
|
|
70
132
|
|
|
71
133
|
## Command reference
|
|
72
134
|
|
|
@@ -75,7 +137,7 @@ Load only the reference the current step needs.
|
|
|
75
137
|
### Setup and provider projection
|
|
76
138
|
|
|
77
139
|
```bash
|
|
78
|
-
canon init # canon.yaml, starter policy, AGENTS.md block, plans home
|
|
140
|
+
canon init # canon.yaml, starter policy (--policy light|standard|strict, default strict; see canon-policy), AGENTS.md block, plans home
|
|
79
141
|
canon skills install # detects .claude/.agents|.codex/.omp/.pi; if none exists, installs Claude+Codex
|
|
80
142
|
canon skills install --providers=claude,codex,omp,pi
|
|
81
143
|
canon skills check # read-only drift check
|
|
@@ -85,7 +85,7 @@ enum, like `lane`.
|
|
|
85
85
|
|
|
86
86
|
Every scenario belongs to a Subject (the durable product unit, see
|
|
87
87
|
`canon-subject`). Write the Subject first, then pin with `--subject`;
|
|
88
|
-
|
|
88
|
+
the scenario index carries the tag onto the index record and
|
|
89
89
|
`spec_coverage.scope` filters on it. An unpinned scenario is invisible to
|
|
90
90
|
a scoped gate.
|
|
91
91
|
|
|
@@ -97,10 +97,17 @@ canon scenario new checkout.cart.01 --title "Adding an item raises the count" \
|
|
|
97
97
|
--subject checkout-core --lane behavior --case happy # once per behavior
|
|
98
98
|
canon scenario new checkout.cart.04 --title "Adding an out-of-stock item is refused" \
|
|
99
99
|
--subject checkout-core --lane behavior --case failure # and its failure path
|
|
100
|
-
canon format specs # clean?
|
|
101
|
-
canon inventory sync # index it
|
|
102
100
|
```
|
|
103
101
|
|
|
102
|
+
There is no format or sync step after these: the scaffold writes a
|
|
103
|
+
format-clean scenario, and `canon status`, `canon gate check`,
|
|
104
|
+
`canon evidence add`, `canon review add` and `canon subject status`
|
|
105
|
+
index added, edited and deleted `.feature` files before they read. When
|
|
106
|
+
one of them prints `NOT refreshed`, the corpus has a format violation
|
|
107
|
+
(for example a hand-written header with no `# canon:` line) or a
|
|
108
|
+
duplicate id; `canon inventory sync` (or `canon format specs`) names it.
|
|
109
|
+
Fix it and the next command indexes the files (see `canon-inventory`).
|
|
110
|
+
|
|
104
111
|
- `--actor <id>` (or `CANON_ACTOR` in the environment) puts YOUR id on the
|
|
105
112
|
provenance line. Default is `canon-scaffold`.
|
|
106
113
|
- **Never hand-type, move, or edit a `# canon:` line.** The scaffold
|
|
@@ -15,21 +15,22 @@ release-scoped check (below) enforces it. A human-only `flagged` overlay
|
|
|
15
15
|
overrides the ladder — a flagged artifact is never green regardless of
|
|
16
16
|
evidence. (Author `policy.yaml`'s trust/staleness fields via `canon-policy`.)
|
|
17
17
|
|
|
18
|
-
## The
|
|
18
|
+
## The ten failure classes
|
|
19
19
|
|
|
20
20
|
Every violation carries one of these stable, grep-able strings:
|
|
21
21
|
|
|
22
22
|
| Class | Meaning |
|
|
23
23
|
|---|---|
|
|
24
24
|
| `uncovered-cell` | Either a policy-required evidence cell (role × artifact) with no matching record, or — with `spec_coverage` enabled — a spec scenario that is unimplemented or mismatched, or a feature surface missing a `require_cases` case. The detail string distinguishes them. Coverage means "a test exists", not "a test passed": even a `Divergent` verdict satisfies the role-cell form. |
|
|
25
|
-
| `unreviewed-promotion` | An artifact tagged `reviewed` has no matching ledger review record — or, with `spec_coverage.require_review` set, an in-scope scenario has no qualifying `Review
|
|
25
|
+
| `unreviewed-promotion` | An artifact tagged `reviewed` has no matching ledger review record — or, with `spec_coverage.require_review` set, an in-scope scenario has no qualifying `Review`: none at all, only self-reviews, only stale reviews (the spec or evidence it pinned changed since), or only `indeterminate`/`unavailable` ones. See `canon-review`. |
|
|
26
26
|
| `trust-below-required` | Achieved trust level is below `policy.yaml`'s `trust_required` for its class — RELEASE-scoped only (`canon gate check --release`); never fires on an ordinary run. |
|
|
27
|
-
| `stale-evidence` | A passing record degraded to stale: its declared surface changed since its `evidence_sha`, or HEAD moved past `staleness.max_commits_behind`. Only degrades an already-green record. Also: a file bound to a latest evidence record whose bytes are gone — no intact blob in `.canon/artifacts/sha256/` and no working-tree copy with the recorded digest (see Bound evidence stays provable below). |
|
|
27
|
+
| `stale-evidence` | A passing record degraded to stale: its declared surface changed since its `evidence_sha`, or HEAD moved past `staleness.max_commits_behind`. Only degrades an already-green record. Also: a file bound to a latest evidence record whose bytes are gone — no intact blob in `.canon/artifacts/sha256/` and no working-tree copy with the recorded digest — or a file bound to an in-effect review with no intact blob (see Bound evidence stays provable below). |
|
|
28
28
|
| `malformed-evidence` | A candidate record doesn't parse, is misfiled, or carries an unparseable interim tag. Malformed evidence is no evidence. |
|
|
29
29
|
| `flagged` | The human-only `flagged` overlay is set — never green regardless of passing evidence. |
|
|
30
30
|
| `unevidenced-flip` | `canon gate task <task_id>` was asked to flip a checkbox with no matching, non-`Divergent` evidence record. |
|
|
31
31
|
| `fabricated-evidence` | An evidence note contains a blocklisted marker (`"would pass"`, `"TBD"`, `"n/a"`) or a bare `verified` claim with no attached command result. |
|
|
32
32
|
| `open-blocker` | With `spec_coverage.require_review` set (and `block_on_findings`, the default), a Finding whose latest version is severity `blocker`, disposition `open`, on a change an in-scope subject adopted. Subject string `<change_id>#<round>.<seq>`. See `canon-review`. |
|
|
33
|
+
| `false-success` | The latest audit record (`canon evidence add --kind audit`) of a scenario says `refuted`: an external evaluator found a counterexample to the claimed success. Always on, regardless of policy or sampling, and takes precedence over any declared faithful verdict; only a newer audit (`confirmed` or `indeterminate`) clears it. The detail names the evaluator, its version, its sha256 and the counterexample paths. |
|
|
33
34
|
|
|
34
35
|
## `canon gate check [--repo <dir>] [--release]`
|
|
35
36
|
|
|
@@ -50,6 +51,32 @@ Exit `0` clean, `1` gate-red (any violation), `2` usage/load failure
|
|
|
50
51
|
resolves through the nearest-ancestor `canon.yaml` walk, so it reads the
|
|
51
52
|
repo ROOT's `.canon/policy.yaml` and `.canon/ledger` from any subdirectory.
|
|
52
53
|
|
|
54
|
+
When `.canon/policy.yaml` exists but has no approval, or its bytes no
|
|
55
|
+
longer match the approved digest, the gate prints `advisory: policy not
|
|
56
|
+
approved: …` or `advisory: policy changed since approval: …` after its
|
|
57
|
+
result, naming `canon policy approve --by <who>`; the exit code is
|
|
58
|
+
unchanged (see `canon-policy`).
|
|
59
|
+
|
|
60
|
+
Before it reads the scenario index, the gate rebuilds any stale part of
|
|
61
|
+
it from the changed `.feature` files (the same code `canon inventory
|
|
62
|
+
sync` runs) and says so in one stderr line, silent when the index is
|
|
63
|
+
fresh:
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
canon gate check: refreshed the scenario index from 1 changed feature file(s) (specs/features/kind=feature/area=cart/add.feature): 1 record(s) written
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
A root that fails to sync (format violations, a duplicate id) is named
|
|
70
|
+
instead, ending ``NOT refreshed: root `x` (N format violation(s), 0 sync
|
|
71
|
+
error(s)) — `canon inventory sync` lists them``, and the gate fails
|
|
72
|
+
closed: it adds `uncovered-cell <root> — the scenario index for spec root
|
|
73
|
+
`<root>` is stale and could not be refreshed (…); `canon inventory sync`
|
|
74
|
+
lists the problems to fix` and exits 1, since the index it would grade
|
|
75
|
+
does not hold the edits. The `verifying → shipped` guard refuses with
|
|
76
|
+
the same violation, and `canon status` lists the sync as its only step.
|
|
77
|
+
Fix what the sync lists and run the gate again. See `canon-storage`
|
|
78
|
+
(The scenario index).
|
|
79
|
+
|
|
53
80
|
### The spec-derived worklist (`spec_coverage`, opt-in)
|
|
54
81
|
|
|
55
82
|
Coverage groups the EVIDENCE corpus, so it can only ever enumerate
|
|
@@ -111,14 +138,18 @@ A present section, even `require_evidence: false` or a malformed one
|
|
|
111
138
|
- **Independent review** (`require_review`) — scenarios whose subject is
|
|
112
139
|
in `require_review.scope` (default `[verifying, shipped]`; `[]` = every
|
|
113
140
|
scenario; independent of `scope` above) need a `Review` record
|
|
114
|
-
(`canon review add`)
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
(default `true`)
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
141
|
+
(`canon review add`) whose latest version per reviewer is `signed`,
|
|
142
|
+
still matches the scenario digest and latest evidence it pinned
|
|
143
|
+
(a review written before 0.15 has no pins and skips this check), and,
|
|
144
|
+
when `distinct_actor` (default `true`), is by a reviewer and actor
|
|
145
|
+
other than the scenario's evidence actor and from a session other than
|
|
146
|
+
any evidence session when both carry one (`--session-id`);
|
|
147
|
+
gaps are `unreviewed-promotion`, and the detail says whether the
|
|
148
|
+
review is missing, a self-review, stale (naming whether the spec, the
|
|
149
|
+
evidence or both changed), or not signed. With `block_on_findings`
|
|
150
|
+
(default `true`), an open `blocker` Finding on a change an in-scope
|
|
151
|
+
subject adopted is `open-blocker`. `exclude_lanes` also exempts
|
|
152
|
+
scenarios from review. Full contract: `canon-review`.
|
|
122
153
|
|
|
123
154
|
```yaml
|
|
124
155
|
spec_coverage:
|
|
@@ -129,12 +160,28 @@ A present section, even `require_evidence: false` or a malformed one
|
|
|
129
160
|
block_on_findings: true # default
|
|
130
161
|
```
|
|
131
162
|
|
|
163
|
+
- **Sampled audits** (`require_audit`) — a deterministic sample of
|
|
164
|
+
in-scope scenarios (`require_audit.scope`, default `[verifying,
|
|
165
|
+
shipped]`; `exclude_lanes` applies) must carry an external audit
|
|
166
|
+
(`canon evidence add --kind audit`, see Audit records below). A sampled
|
|
167
|
+
scenario whose latest audit is missing or `indeterminate` is
|
|
168
|
+
`uncovered-cell`:
|
|
169
|
+
|
|
170
|
+
```
|
|
171
|
+
uncovered-cell cart.add.01 — sampled for audit by spec_coverage.require_audit (rate 0.1, seed `s1`) and has no audit record — …
|
|
172
|
+
uncovered-cell cart.add.01 — sampled for audit by spec_coverage.require_audit (rate 0.1, seed `s1`), but its latest audit is indeterminate (evaluator `judge` 1.2) — …
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
Keys, defaults and the exact sampling formula: `canon-policy`.
|
|
176
|
+
|
|
132
177
|
Two refusals rather than a silent pass, both surfacing as
|
|
133
178
|
`uncovered-cell` on the subject `spec_coverage`:
|
|
134
179
|
|
|
135
180
|
- A **malformed section** (unknown key, unknown `scope` status, or a
|
|
136
|
-
malformed `require_cases`/`require_review` — unknown
|
|
137
|
-
`null`, non-mapping
|
|
181
|
+
malformed `require_cases`/`require_review`/`require_audit` — unknown
|
|
182
|
+
key, bad status, `null`, non-mapping, a missing or out-of-range
|
|
183
|
+
`rate`) refuses at check time (`uncovered-cell spec_coverage — ...
|
|
184
|
+
unusable`). It is never treated as
|
|
138
185
|
absent — a typo must not be silently equivalent to not opting in.
|
|
139
186
|
- A **corpus kind routed off the gate's rung** (`scenario`,
|
|
140
187
|
`divergence`, or `subject` sent anywhere but `local`) refuses, because
|
|
@@ -144,6 +191,96 @@ Malformed rows in those three kinds do NOT surface as
|
|
|
144
191
|
`malformed-evidence`; they are kept off the evidence violation set on
|
|
145
192
|
purpose, so enabling nothing changes no existing verdict.
|
|
146
193
|
|
|
194
|
+
### Audit records (`--kind audit`)
|
|
195
|
+
|
|
196
|
+
canon never generates or runs an audit. An external evaluator does, and
|
|
197
|
+
canon records its result:
|
|
198
|
+
|
|
199
|
+
```bash
|
|
200
|
+
canon evidence add --kind audit --project-id root --scenario-id cart.add.01 \
|
|
201
|
+
--ref "judge run 42" --role reviewer \
|
|
202
|
+
--evaluator-id judge --evaluator-version 1.2 \
|
|
203
|
+
--evaluator-digest <64 lowercase hex sha256> \
|
|
204
|
+
--audit-result refuted --report junit:audit/counterexample.xml
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
- `--kind audit` requires `--evaluator-id`, `--evaluator-version`,
|
|
208
|
+
`--evaluator-digest` (sha256, 64 lowercase hex), `--audit-result
|
|
209
|
+
confirmed|refuted|indeterminate`, and `--scenario-id` with
|
|
210
|
+
`--project-id`. It refuses `--task`, `--verdict`, `--surface-ref` and
|
|
211
|
+
`--approval-*`; the four evaluator flags are refused on any other kind.
|
|
212
|
+
All refusals exit `2`.
|
|
213
|
+
- The record carries `audit: {evaluator_id, evaluator_version,
|
|
214
|
+
evaluator_digest, result}`; its verdict is derived (`confirmed` →
|
|
215
|
+
faithful, `refuted` → divergent, `indeterminate` → not-applicable).
|
|
216
|
+
- Optional `--report`/`--artifact` counterexamples are bound and stored
|
|
217
|
+
like any report; the latest audit's bindings are re-hashed by the
|
|
218
|
+
artifact check.
|
|
219
|
+
- Audits fold apart: per `(project_id, scenario_id)` there is a latest
|
|
220
|
+
declared record and a latest audit. An audit never becomes the
|
|
221
|
+
scenario's latest declared verdict.
|
|
222
|
+
- `canon subject status <id> shipped` refuses `false-success`, and an
|
|
223
|
+
owed sampled audit when `require_audit.scope` covers `shipped`.
|
|
224
|
+
- `canon status` shows an `audits` section (sampled / confirmed / owed /
|
|
225
|
+
refuted) and suggests `canon evidence add --kind audit …` in `next:`.
|
|
226
|
+
|
|
227
|
+
### Dropping a binding (`--drop-attachment`)
|
|
228
|
+
|
|
229
|
+
`canon evidence add` for a scenario refuses (exit `2`) when the
|
|
230
|
+
scenario's current latest record of the same fold (declared records, or
|
|
231
|
+
audit records for an audit) — committed ledger plus `_staging` — binds
|
|
232
|
+
files the new record does not. The refusal names the paths and two ways
|
|
233
|
+
out: re-bind them (`--artifact`/`--report`; new bytes at the same path
|
|
234
|
+
are fine), or drop them on purpose with `--drop-attachment <path>`
|
|
235
|
+
(repeatable) and one single-line `--drop-reason <text>`. The drop is
|
|
236
|
+
recorded as `attachment_drop: {paths, reason}`. Refused (`2`): a drop
|
|
237
|
+
without a reason, a reason without a drop, a drop path that is not
|
|
238
|
+
actually dropped, a multi-line reason, or a drop without
|
|
239
|
+
`--scenario-id` (task-only evidence has no drop guard). `canon status`
|
|
240
|
+
lists recorded drops (`dropped bindings (N):`; `drops[]` in `--json`).
|
|
241
|
+
|
|
242
|
+
### Nothing is attested before the brief is approved (`--override-reason`)
|
|
243
|
+
|
|
244
|
+
The approval is never assumed: the agent writes the brief, stops and
|
|
245
|
+
reports it, and builds nothing for that scope until the human answers.
|
|
246
|
+
`canon evidence add` is the backstop for that stop (0.16 M7). It
|
|
247
|
+
refuses (exit `1`, nothing staged or stored) while a subject the record
|
|
248
|
+
attests for has an enforced, unapproved brief: the newest adopted
|
|
249
|
+
change has a `proposal.md` under the openspec plan sources with no
|
|
250
|
+
`Approved:` line (the check `canon status` and the ship guard use; see
|
|
251
|
+
`canon-subject`). The subjects are the scenario's `@subject` owner for
|
|
252
|
+
`--scenario-id`, and every subject adopting the task's change for
|
|
253
|
+
`--task`:
|
|
254
|
+
|
|
255
|
+
```
|
|
256
|
+
canon evidence add: unreviewed-promotion demo-subject — evidence for world.demo.01: brief not approved — openspec/changes/c-demo/proposal.md has no `Approved:` line, so the human has not answered it
|
|
257
|
+
canon evidence add: refused — stop, report the brief to the human and wait for their answer: nothing is built or attested for a subject before its brief is approved. Record their words as `Approved: <who> <when> — "<their words>"` in that proposal.md only from that answer; to attest anyway, pass --override-reason <text>, which is recorded on the evidence record
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
A subject with no brief to answer (no openspec source, no adopted change,
|
|
261
|
+
no `proposal.md`) and a scenario no subject owns are unaffected.
|
|
262
|
+
`canon review add` and `canon finding add` are never refused.
|
|
263
|
+
`--override-reason "<one line>"` attests anyway, prints the `waived …`
|
|
264
|
+
lines and `override recorded by `<actor>` for the N brief refusal(s)
|
|
265
|
+
above: <reason>`, and records on the evidence record:
|
|
266
|
+
|
|
267
|
+
```yaml
|
|
268
|
+
brief_override:
|
|
269
|
+
reason: the human approved in chat; the line follows
|
|
270
|
+
waived:
|
|
271
|
+
- { class: unreviewed-promotion, subject: demo-subject }
|
|
272
|
+
actor: { agent_id: lead } # from --actor-id
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
A blank or multi-line reason exits 2; an override when nothing refused is
|
|
276
|
+
not recorded. `canon status` lists each such record as a `warning:`, and
|
|
277
|
+
`canon gate check` prints them after the result without failing:
|
|
278
|
+
|
|
279
|
+
```
|
|
280
|
+
brief overrides: 1 evidence record(s) attested before their brief was approved — not failing the gate:
|
|
281
|
+
evidence for world.demo.01 was attested by `lead` at 2026-10-12T07:00:00Z while the brief of demo-subject was not approved (override: the human approved in chat; the line follows)
|
|
282
|
+
```
|
|
283
|
+
|
|
147
284
|
### Review-waiver advisories
|
|
148
285
|
|
|
149
286
|
A subject moved into a review-scoped status with `canon subject status
|
|
@@ -160,6 +297,58 @@ review waivers: 1 advisory(ies) — not failing the gate:
|
|
|
160
297
|
|
|
161
298
|
The waiver lasts only until the subject's next status write.
|
|
162
299
|
|
|
300
|
+
`canon subject status <id> shipped` also refuses where a brief exists and
|
|
301
|
+
has no `Approved:` line (the subject's newest adopted change has a
|
|
302
|
+
`proposal.md`; see `canon-subject`), as an `unreviewed-promotion` naming
|
|
303
|
+
the subject. With no brief to answer it prints an advisory and does not
|
|
304
|
+
refuse. Like the evidence refusal above, it is a backstop: the stop is at
|
|
305
|
+
the brief. `canon gate check` does not grade briefs, so a waived ship
|
|
306
|
+
brief prints no advisory here; `canon status` keeps showing the brief as
|
|
307
|
+
not approved.
|
|
308
|
+
|
|
309
|
+
### Reviews that predate pinning (checked against their pin)
|
|
310
|
+
|
|
311
|
+
With `require_review` on, the reviews in effect (latest per reviewer and
|
|
312
|
+
scenario) that carry no pins were written before 0.15. Each still names
|
|
313
|
+
`pin`, the commit it reviewed, and the gate checks it there (0.16):
|
|
314
|
+
|
|
315
|
+
- **current** — the scenario's digest at that commit (read with `git`,
|
|
316
|
+
normalized exactly as `canon inventory sync` computes
|
|
317
|
+
`scenario_digest`) equals its current digest, and no declared evidence
|
|
318
|
+
record of the scenario is newer than the review. It counts. When the
|
|
319
|
+
scenario lived in another `.feature` file at that commit, the spec
|
|
320
|
+
root's files at that commit are searched for its id.
|
|
321
|
+
- **stale** — the scenario changed since the commit, or it has newer
|
|
322
|
+
evidence. It no longer counts, and an in-scope scenario left with no
|
|
323
|
+
other qualifying review is `unreviewed-promotion`:
|
|
324
|
+
|
|
325
|
+
```
|
|
326
|
+
unreviewed-promotion gate.check.15 — stale review by `canon-reviewer-g1` (written before review pinning; checked against commit b556ab7529a332aa0b28d0a0f98bdd62fcbb9a63): it has declared evidence newer than the review; a review counts only while what it saw is current — review it again (`canon review add`)
|
|
327
|
+
```
|
|
328
|
+
|
|
329
|
+
A spec change reads `its spec changed since <pin> (the scenario's
|
|
330
|
+
digest at that commit is not its current one)`.
|
|
331
|
+
- **undecidable** — the pin is not a commit in this clone (a shallow
|
|
332
|
+
clone, rewritten history), or no feature file of the scenario's root
|
|
333
|
+
holds its id at that commit. It keeps counting, as in 0.15: a missing
|
|
334
|
+
commit never becomes a violation.
|
|
335
|
+
|
|
336
|
+
One advisory line after the violations counts the three. A stale one
|
|
337
|
+
whose scenario already has another qualifying review (current, signed,
|
|
338
|
+
distinct) is counted as superseded, since nothing is owed for it; only
|
|
339
|
+
the scenarios a stale one leaves without a qualifying review are named
|
|
340
|
+
for re-review. The exit code is unchanged by the line itself:
|
|
341
|
+
|
|
342
|
+
```
|
|
343
|
+
advisory: 338 review(s) predate review pinning (0.15) and were checked against the commit each one pinned: 318 current, 12 stale, 8 undecidable; 7 stale one(s) superseded by a current review of the same scenario, so nothing is owed for them; a stale one with no current review in its place no longer counts — review its scenario again (`canon review add`): platformer.movement.01, platformer.movement.02, …; an undecidable one still counts unchecked, because its pinned commit or the scenario at it is not in this clone — check out full history (`fetch-depth: 0` in CI) to check it
|
|
344
|
+
```
|
|
345
|
+
|
|
346
|
+
A review of a scenario no feature file declares any more is not counted.
|
|
347
|
+
In CI, check out full history (`actions/checkout` with `fetch-depth: 0`)
|
|
348
|
+
for the job that runs `canon gate check`, or every legacy review is
|
|
349
|
+
undecidable. Each pinned commit is read once per run through one `git
|
|
350
|
+
cat-file --batch` process, and never fetched over the network.
|
|
351
|
+
|
|
163
352
|
## Experimental: evidence binding (`experimental.evidence_binding`, off by default)
|
|
164
353
|
|
|
165
354
|
An evidence record is an attestation: canon never runs `--ref`, and it
|
|
@@ -252,6 +441,15 @@ subject's status, and a repo with no `spec_coverage` would otherwise never
|
|
|
252
441
|
have its bindings checked. A superseded record is not checked —
|
|
253
442
|
re-attesting is the fix.
|
|
254
443
|
|
|
444
|
+
Files bound to a review (`canon review add --artifact`, see
|
|
445
|
+
`canon-review`) are checked the same way for every in-effect review (the
|
|
446
|
+
latest per `(project_id, scenario_id, reviewer)`), with one difference:
|
|
447
|
+
they were stored when the review was written, so only an intact blob is
|
|
448
|
+
clean. A missing or altered blob is `stale-evidence` on the review's
|
|
449
|
+
scenario, and the detail names the review: ``review by `<reviewer>`
|
|
450
|
+
(actor `<id>`, <time>)``. The working-tree file is reported but never
|
|
451
|
+
vouches for it.
|
|
452
|
+
|
|
255
453
|
`canon evidence vault [--max-artifact-mib <N>]` stores the bytes for
|
|
256
454
|
existing records: every attachment whose working-tree file still matches
|
|
257
455
|
its digest is copied in, and everything else is listed as not stored,
|
|
@@ -313,6 +511,14 @@ The success line is `canon gate task: <task_id> flipped — written
|
|
|
313
511
|
directly; nothing to promote`. An already-`[x]` row is an idempotent no-op (exit `0`). An unknown
|
|
314
512
|
`task_id` is reported (exit `1`).
|
|
315
513
|
|
|
514
|
+
A flip on evidence that cannot prove the task still succeeds, but warns
|
|
515
|
+
on stderr: when none of the task's evidence records binds a file and one
|
|
516
|
+
of them cites a canon command as its `--ref` (`canon gate check`,
|
|
517
|
+
`canon status`), the warning says to bind the artifact or report that
|
|
518
|
+
proves the task (`canon evidence add --task <id> --artifact <path>` or
|
|
519
|
+
`--report <format>:<path>`). A canon command reports on canon's own
|
|
520
|
+
records, so it is never the proof of a task.
|
|
521
|
+
|
|
316
522
|
```bash
|
|
317
523
|
canon gate task my-change#5.2 --repo .
|
|
318
524
|
```
|
|
@@ -337,13 +543,26 @@ canon gate promote --repo . --dry-run # preview only
|
|
|
337
543
|
```
|
|
338
544
|
|
|
339
545
|
Which writes need this step: `evidence add`, `finding add` and `finding
|
|
340
|
-
close` stage, and
|
|
341
|
-
|
|
342
|
-
|
|
546
|
+
close` stage, and under `write: { promote: manual }` (or an absent
|
|
547
|
+
`write:` key) their success line ends ``— run `canon gate promote` to
|
|
548
|
+
commit it``. Under `write: { promote: auto }` (what `canon init` writes)
|
|
549
|
+
or with `--promote`, they promote the one record they just staged —
|
|
550
|
+
same validation and `run_seq` assignment as this command — and print
|
|
551
|
+
one success line naming the committed record (`wrote kind=…/….json`), not
|
|
552
|
+
the staged file, ending `— staged and promoted in this run; nothing to
|
|
553
|
+
promote`, so no separate
|
|
554
|
+
`canon gate promote` is needed for it; `--no-promote` keeps one write
|
|
555
|
+
staged. A record another run staged is not touched: only `gate promote`
|
|
556
|
+
commits everything staged. It stays safe to run anyway: with nothing
|
|
557
|
+
staged it prints
|
|
558
|
+
`nothing staged` and exits `0`. `divergence stage` stages for `canon
|
|
559
|
+
divergence promote` and names it, under the same `write:` rule. Every
|
|
560
|
+
other write — `review add`, `divergence
|
|
343
561
|
resolve|defer`, `subject new|adopt|status`, `change new`, `scenario new`, `feature new`,
|
|
344
562
|
`gate task` (it edits the plan document and re-ingests it)
|
|
345
563
|
— is final when it returns, and its success line ends `— written
|
|
346
|
-
directly; nothing to promote`.
|
|
564
|
+
directly; nothing to promote`. Full promote-on-write contract:
|
|
565
|
+
`canon-storage` (Staged writes and promotion).
|
|
347
566
|
|
|
348
567
|
## `canon gate install-hooks [--repo] [--event] [--matcher] [--command] [--timeout]`
|
|
349
568
|
|
|
@@ -367,8 +586,8 @@ Prefer this over hand-editing `settings.json`/`hooks.json`.
|
|
|
367
586
|
|
|
368
587
|
## `canon gate selftest`
|
|
369
588
|
|
|
370
|
-
Runs the shipped fixture corpus — one fixture per failure class (
|
|
371
|
-
including `
|
|
589
|
+
Runs the shipped fixture corpus — one fixture per failure class (ten,
|
|
590
|
+
including `false-success`), each a
|
|
372
591
|
deliberately broken corpus proving that class fires and ONLY that class
|
|
373
592
|
(both under-detection and over-triggering fail the run). Takes no
|
|
374
593
|
`--repo`; self-contained. Run it before trusting any other `canon gate
|
|
@@ -1,16 +1,43 @@
|
|
|
1
1
|
# canon-inventory
|
|
2
2
|
|
|
3
|
-
`canon inventory sync`
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
`canon inventory sync` builds canon's scenario ledger-INDEX: it turns a
|
|
4
|
+
hand-authored, `canon format`-validated `.feature` corpus into the scenario
|
|
5
|
+
records every downstream stage joins against.
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## You rarely run it yourself
|
|
8
|
+
|
|
9
|
+
Since 0.16 the commands that read the index refresh it first:
|
|
10
|
+
`canon status`, `canon gate check`, `canon evidence add`,
|
|
11
|
+
`canon review add` and `canon subject status` index added, edited and
|
|
12
|
+
deleted `.feature` files themselves, with the same code this command runs,
|
|
13
|
+
scoped to the changed files, and say so in one stderr line:
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
canon status: refreshed the scenario index from 1 changed feature file(s) (specs/features/…): 1 record(s) written
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
So the working loop has no sync step and no `canon format` step: write the
|
|
20
|
+
scenario (`canon scenario new` writes it format-clean) and run the next
|
|
21
|
+
command. Run `canon inventory sync` when:
|
|
22
|
+
|
|
23
|
+
- a reader prints `NOT refreshed` — a root has a format violation (a
|
|
24
|
+
hand-written `Feature:`/`Scenario:` header with no `# canon:` provenance
|
|
25
|
+
comment, an emptied file, …) or a duplicate id, so its files were not
|
|
26
|
+
indexed. `canon gate check` then fails (`uncovered-cell <root>`),
|
|
27
|
+
`shipped` is refused and `canon status` lists only the sync. `canon
|
|
28
|
+
inventory sync` names each problem (`canon format specs` names the format ones);
|
|
29
|
+
fix them and the next command indexes the files;
|
|
30
|
+
- `canon status` says the ledger holds no scenario records while the
|
|
31
|
+
`.feature` files hold some (the same refusal, on an index never built);
|
|
32
|
+
- you want the whole index rebuilt from every root at once.
|
|
33
|
+
|
|
34
|
+
## Where the index sits in the loop
|
|
8
35
|
|
|
9
36
|
```
|
|
10
|
-
author .feature corpus
|
|
37
|
+
author .feature corpus (canon scenario new)
|
|
11
38
|
│
|
|
12
39
|
▼
|
|
13
|
-
|
|
40
|
+
scenario index # refreshed by every reader; `canon inventory sync` rebuilds it
|
|
14
41
|
│
|
|
15
42
|
▼
|
|
16
43
|
canon review add # attests a review — see canon-inventory downstream / canon-gate
|
|
@@ -23,8 +50,8 @@ canon ingest artifacts # with artifacts.native_records: true in canon.y
|
|
|
23
50
|
# (see canon-artifact-ingest)
|
|
24
51
|
```
|
|
25
52
|
|
|
26
|
-
Every stage after
|
|
27
|
-
pair
|
|
53
|
+
Every stage after the index joins on the SAME `(project_id, scenario_id)`
|
|
54
|
+
pair it materializes — one identity scheme, no second one.
|
|
28
55
|
|
|
29
56
|
## `canon inventory sync [--repo <dir>] [--spec-root <dir>]`
|
|
30
57
|
|