canon-ai 3.4.0 → 3.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/README.md +1 -1
- package/dist/cli/index.js +7 -29
- package/dist/orchestrator/run-task.js +217 -92
- package/package.json +1 -1
- package/templates/.canon/templates/spec.md +2 -0
- package/templates/.claude/agents/code-review-anchored.md +12 -0
- package/templates/.claude/agents/code-review-cold.md +1 -0
- package/templates/.claude/agents/spec-review-lens.md +33 -0
- package/templates/.claude/skills/canon-pipeline/recovery.md +21 -0
- package/templates/.claude/skills/canon-spec/SKILL.md +12 -1
- package/templates/.claude/skills/canon-spec-review/SKILL.md +38 -46
- package/templates/docs/pipeline-orchestrator.md +28 -11
package/package.json
CHANGED
|
@@ -22,6 +22,7 @@ What are we explicitly NOT doing in this task? This prevents scope creep.
|
|
|
22
22
|
Checklist of verifiable outcomes. Each item must be testable.
|
|
23
23
|
|
|
24
24
|
> **For a bug or flake fix:** Include a regression-test AC that fails on the pre-fix code for the stated reason and passes after the fix (red-first). If the mechanism is environment-bound and a faithful repro is impractical, the AC must say so and name the deterministic alternative rather than omitting verification.
|
|
25
|
+
> **For a refactor:** Before any "preserve behavior" AC, run a correctness audit. For each behavior a refactor declares preserved, record in Problem whether it is correct today, and state one outcome: correct as-is, fixed deliberately (with its own AC), split into a separate task, or kept as a named quirk. A behavior confirmed correct needs only a one-line record. N/A for features and bug fixes.
|
|
25
26
|
|
|
26
27
|
- [ ] AC-1: ...
|
|
27
28
|
- [ ] AC-2: ...
|
|
@@ -98,3 +99,4 @@ Steps the human should perform to verify the feature works as intended. Written
|
|
|
98
99
|
- [ ] Human Test Plan uses product language only (no code, no file names)
|
|
99
100
|
- [ ] Validation Required has at least one entry marked `- [x]` (not `- [ ]`). `- [ ]` is a placeholder; the spec author flips required checks to `- [x]` before marking spec done. The orchestrator's code_review pre-flight blocks if no `[x]` items are present.
|
|
100
101
|
- [ ] (Bug/flake fixes; N/A for features/refactors) *Problem* states the confirmed mechanism and how it was confirmed, with evidence matching the mechanism class (runtime-dependent mechanisms need executed confirmation), not merely a plausible cause; *Acceptance Criteria* includes a red-first regression-test AC or an explicit environment-bound and faithful-repro-impractical escape with a deterministic alternative
|
|
102
|
+
- [ ] (Refactors; N/A for features/bug fixes) *Problem* records the correctness audit of each behavior declared preserved, with one outcome each (correct as-is, fixed deliberately (with its own AC), split into a separate task, or kept as a named quirk) stated before any "preserve" AC
|
|
@@ -8,6 +8,7 @@ You are the anchored code-review lens in canon's three-lens review pipeline. You
|
|
|
8
8
|
Do not write `review.md`.
|
|
9
9
|
Do not run `canon task phase`.
|
|
10
10
|
Do not ask the user for permission to edit files.
|
|
11
|
+
Do this review yourself: spawn no sub-agents and run no review skills or other reviewers.
|
|
11
12
|
|
|
12
13
|
This agent intentionally declares no model override; it inherits the foreman's selected `code_review` model and effort.
|
|
13
14
|
|
|
@@ -54,3 +55,14 @@ OVERALL_SIGNAL: approve | changes_requested
|
|
|
54
55
|
```
|
|
55
56
|
|
|
56
57
|
If Stage 1 failed, omit Stage 2 findings and set `OVERALL_SIGNAL: changes_requested`.
|
|
58
|
+
|
|
59
|
+
If Stage 1 passes and you find no Stage 2 issues, return:
|
|
60
|
+
|
|
61
|
+
```text
|
|
62
|
+
STAGE_1: pass
|
|
63
|
+
AC_TABLE:
|
|
64
|
+
| AC | Met/Partial/Not Met | note |
|
|
65
|
+
STAGE_1_GAPS: (none)
|
|
66
|
+
STAGE_2_FINDINGS: (none)
|
|
67
|
+
OVERALL_SIGNAL: approve
|
|
68
|
+
```
|
|
@@ -8,6 +8,7 @@ You are the cold-Claude code-review lens in canon's three-lens review pipeline.
|
|
|
8
8
|
Do not write `review.md`.
|
|
9
9
|
Do not run `canon task phase`.
|
|
10
10
|
Do not ask the user for permission to edit files.
|
|
11
|
+
Do this review yourself: spawn no sub-agents and run no review skills or other reviewers.
|
|
11
12
|
|
|
12
13
|
This agent intentionally declares no model override; it inherits the foreman's selected `code_review` model and effort.
|
|
13
14
|
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: spec-review-lens
|
|
3
|
+
description: Read-only spec reviewer for the /canon-spec-review skill. Dispatched once per angle (shape, factual, spec-quality) with the spec inline; returns cited findings to the synthesizing session.
|
|
4
|
+
tools: Read, Grep, Glob, Bash
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
You are one lens in `/canon-spec-review`. The prompt gives you the spec text, your angle, and your rubric.
|
|
8
|
+
|
|
9
|
+
Review the spec yourself and spawn no sub-agents.
|
|
10
|
+
Invoke no skills.
|
|
11
|
+
Write no files. Run no `canon` commands and ask the user nothing.
|
|
12
|
+
Bash is read-only: search, list, print file ranges, and inspect read-only git history. Use no redirects, in-place edits, or state-changing git.
|
|
13
|
+
Inherit the session's model and effort.
|
|
14
|
+
|
|
15
|
+
Every finding cites `file:line` or an AC number. Re-open the current file before making any claim about code; do not rely on memory.
|
|
16
|
+
|
|
17
|
+
Coverage first: report every finding you can back with a citation, each tagged with a severity. Filtering is the synthesizer's job. `[NO FINDINGS]` is a valid return; do not invent findings.
|
|
18
|
+
|
|
19
|
+
**Severity definitions**
|
|
20
|
+
- **BLOCKING:** would cause wrong behavior or a silent bug, or makes an AC unimplementable as written.
|
|
21
|
+
- **NIT:** an implementation detail the implementer resolves by reading the codebase, a minor ambiguity with an obvious default, or a question the plan phase should answer.
|
|
22
|
+
- **STRONG:** nit-grade under those rules, but likely to cost a review round, so cheaper to fix now.
|
|
23
|
+
|
|
24
|
+
Behavior the spec's Non-Goals explicitly exclude and verify as unaffected is a NIT at most. An omitted required change (such as a caller, parser, migration or test surface), a transitive effect of the change, or an internal contradiction between spec sections stays BLOCKING.
|
|
25
|
+
|
|
26
|
+
Return format:
|
|
27
|
+
|
|
28
|
+
```text
|
|
29
|
+
- [BLOCKING|STRONG|NIT] <one-line finding> — <file:line or AC#>
|
|
30
|
+
<2-3 sentence rationale, citing evidence>
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Return exactly `- [NO FINDINGS]` if you find nothing. Add no preamble or closing remarks.
|
|
@@ -5,6 +5,7 @@ Patterns from real production use for when the pipeline gets stuck. Use them as
|
|
|
5
5
|
## Contents
|
|
6
6
|
|
|
7
7
|
- Auto-block on `spec_review` or `code_review` (loop cap hit)
|
|
8
|
+
- Code review stopped without a verdict (reviewer-set `blocked`)
|
|
8
9
|
- Findings keep clustering in the same mechanism (try a size bump before another loop)
|
|
9
10
|
- Phase mismatch — pipeline routes to `spec` when you expected `spec_review`
|
|
10
11
|
- `--ship` refuses: wrong phase
|
|
@@ -35,6 +36,26 @@ Stop the reviewer loop once findings turn wording-only. Self-grep for flagged ph
|
|
|
35
36
|
|
|
36
37
|
**Never reset the iteration counter** to bypass the cap. Counter is durable signal of how many review rounds the task has burned — losing it hides cost from future operators.
|
|
37
38
|
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## Code review stopped without a verdict (reviewer-set `blocked`)
|
|
42
|
+
|
|
43
|
+
**Symptom:** the run exits with `✋ CODE REVIEW STOPPED — the reviewer could not reach a verdict`, and `code_review` is `blocked` with an escalation for every task in the bundle. No verdict box is checked in `review.md`.
|
|
44
|
+
|
|
45
|
+
**Cause:** one of the Claude review lenses returned nothing usable twice (absent, truncated, or not in its return format), so the foreman stopped rather than approve on partial evidence or invent findings. The pipeline deliberately does not retry this stop.
|
|
46
|
+
|
|
47
|
+
**Fix:**
|
|
48
|
+
1. Read `tasks/<id>/review.md` — it names the missing lens and what came back. Look for the cause: a budget cap hit, a timeout, or a harness error in the run log. Raise `CLAUDE_BUDGET` if the lens ran out of budget.
|
|
49
|
+
2. Reset every task in the bundle, then run the whole bundle again. The stop's banner prints these exact commands:
|
|
50
|
+
```bash
|
|
51
|
+
canon task reset-code-review <task-id-1>
|
|
52
|
+
canon task reset-code-review <task-id-2>
|
|
53
|
+
canon run <task-id-1> <task-id-2>
|
|
54
|
+
```
|
|
55
|
+
Run all bundle members together. A `canon run` with only one ID reviews the shared branch against that one task's spec and Affected Files, so the siblings' changes look out of scope.
|
|
56
|
+
|
|
57
|
+
`reset-code-review` archives the stopped `review.md` and starts a fresh review pass. Re-running without the reset also starts a fresh pass, but without archiving the stopped review. Don't hand-check a verdict box or set a verdict with `canon task phase` to get past the stop: the review that would justify it never happened.
|
|
58
|
+
|
|
38
59
|
## Findings keep clustering in the same mechanism (try a size bump before another loop)
|
|
39
60
|
|
|
40
61
|
Most multi-round tasks are healthy — each round closes findings scattered across different, previously-untouched parts of the diff. That's normal iterative discovery, not a sizing problem; raising the loop cap (above) is the right tool for it.
|
|
@@ -146,6 +146,7 @@ Self-check before presenting:
|
|
|
146
146
|
- [ ] Non-Goals rules out the most tempting scope expansions — back load-bearing exclusions with a positive scope-bound or grep AC, not prose "NOT" alone
|
|
147
147
|
- [ ] Symbols named in ACs actually exist in the codebase — grep-verify before presenting
|
|
148
148
|
- [ ] (Bug/flake fixes; N/A for features/refactors) *Problem* states the confirmed mechanism and how it was confirmed, with evidence matching the mechanism class (runtime-dependent mechanisms need executed confirmation), not merely a plausible cause; *Acceptance Criteria* includes a red-first regression-test AC or an explicit environment-bound and faithful-repro-impractical escape with a deterministic alternative
|
|
149
|
+
- [ ] (Refactors; N/A for features/bug fixes) *Problem* records the correctness audit of each behavior declared preserved, with one outcome each (correct as-is, fixed deliberately (with its own AC), split into a separate task, or kept as a named quirk) stated before any "preserve" AC
|
|
149
150
|
|
|
150
151
|
**Spec-writing rules of thumb** (apply when writing ACs and structure):
|
|
151
152
|
- **Name effects to DELETE**: frame supersession as replacement ("replace `oldFn` with `newFn`; `oldFn` must not exist after"), not separate add/remove bullets.
|
|
@@ -156,6 +157,7 @@ Self-check before presenting:
|
|
|
156
157
|
- **Bug and flake-fix specs need a confirmed mechanism and red-first test**: For a bug or flake fix, the spec author must state, in *Problem*, both the confirmed mechanism and how it was confirmed — not merely a plausible cause. Evidence must match the mechanism class: a deterministic mechanism (fixed inputs hit the same wrong branch every run) may cite a trace with the verified trigger values; a runtime-dependent mechanism (race, timing, environment/config interaction) needs executed confirmation — a throwaway prototype-fix spike that makes the symptom vanish, or a deterministic forced repro. The author must satisfy that checkpoint before the spec is marked done; on fast-tier (XS, non-delicate) tasks the `spec_review` checkpoint is skipped, so no reviewer will catch an unverified mechanism. The *Acceptance Criteria* must include a red-first regression-test AC: a test that fails on the pre-fix code for the stated reason and passes after the fix. If the mechanism is environment-bound and a faithful repro is impractical, *Problem* must say so and name a deterministic alternative (integration fixture or documented manual repro) instead of skipping verification silently.
|
|
157
158
|
- **At ≥3 spec_review iterations, read the rounds' content, not the count.** Label each round *edge-fine-tune* (it narrows an already-identified case — converging, safe to finish once the findings are addressed) or *scope-expansion* (it names a **new** structural case the spec hadn't covered — a real design gap). Round count alone is not the signal; a 6-round run of successive fine-tunes is convergence. On genuine scope-expansion, don't refine again: check whether the last two rejections share one structural root, and whether an existing layer already owns the concern. Dropping the whole mechanism class — or deleting the mechanism in favor of the layer that already owns it — is usually cheaper than another round of hardening.
|
|
158
159
|
- **Refactor specs need hard structural caps**: size cap, explicit deletion expectations per symbol, grep AC for disappeared symbols.
|
|
160
|
+
- **Refactor specs need a correctness audit before "preserve behavior"**: For each behavior a refactor declares preserved, record in Problem whether it is correct today, and state one outcome: correct as-is, fixed deliberately (with its own AC), split into a separate task, or kept as a named quirk. A behavior confirmed correct needs only a one-line record. N/A for features and bug fixes.
|
|
159
161
|
- **Codebase-wide term renames: gate per string family, not per enumerated hit.** Hand-enumerated grep lists cause round-over-round scope expansion — each review round surfaces a new label *family*, not a missed instance. Decompose the stale term into its distinct string families up front via a structural sweep, then express each family's post-change invariant as a zero-result grep AC with a word-bounded pattern that catches all separator variants and cannot match the new term. Where the old string is a substring of the new one, zero-result gating is structurally impossible — verify those surfaces with positive targeted ACs instead, and document the asymmetry in the spec so reviewers don't read it as a gap. Two checks before finalizing any zero-result grep AC: (1) on prose surfaces (`docs/`, templates, backlog), read every affected file for *paraphrases* of the retired claim — an exact-string grep proves only that the literal phrase is gone, not that the retired behavior is undocumented everywhere; (2) scan the spec's own other ACs, Docs Impact, and Non-Goals for hits they already tolerate (historical docs, a release-time step, incidental test fixtures) and fold those into an explicit permitted-to-remain bucket inside the same AC, so a later section doesn't quietly contradict the gate.
|
|
160
162
|
|
|
161
163
|
**⛔ STOP — present the spec and wait for approval.**
|
|
@@ -165,7 +167,16 @@ Self-check before presenting:
|
|
|
165
167
|
### Phase 6 — After spec approval
|
|
166
168
|
|
|
167
169
|
**XS tasks:**
|
|
168
|
-
1. Write `tasks/TASK-ID/plan.md` using `.canon/templates/plan.md` as structure.
|
|
170
|
+
1. Write `tasks/TASK-ID/plan.md` using `.canon/templates/plan.md` as structure. Before writing steps, run the same feasibility check the pipeline's planner runs, and record the results in a few lines in the plan's Approach section. Mark an item that doesn't apply `N/A` in a word:
|
|
171
|
+
- **Exists:** the functions, files, and patterns the plan relies on exist — found by searching the code, not recalled.
|
|
172
|
+
- **Real path:** the actual runtime call path the change sits on, including any existing code on it that already does part of the work.
|
|
173
|
+
- **Callers:** every caller of each function whose behavior or error contract changes, found and accounted for.
|
|
174
|
+
- **Tests that can fail:** for each regression test the plan prescribes, the input or state that sends it through the changed path, so it fails without the change.
|
|
175
|
+
- **Predicates:** for each condition or state check the plan writes out, its boundary values (for example zero, empty, or non-numeric) and every state it must handle.
|
|
176
|
+
- **Scope:** every file the steps change is inside the spec's Affected Files.
|
|
177
|
+
- **Async/stateful:** for async or stateful changes — re-entry, cancellation or unmount, stale state, and ownership.
|
|
178
|
+
|
|
179
|
+
A finding that changes a step goes in that step. If the check contradicts the spec, don't work around it in the plan — on the fast tier no Codex `spec_review` will catch the mismatch. Amend the spec, re-present the change to the human, and wait for renewed approval. The earlier Phase 5 approval does not cover a spec changed after it, so don't record approval in step 2 until the human has approved the amended spec.
|
|
169
180
|
2. Record the human's approval in `tasks/TASK-ID/spec-review.md`: check the **Approved** box and add a one-line note ("Fast tier — human conversational spec approval; Codex spec review skipped"). The phase gate reads this artifact before letting `spec_review` advance.
|
|
170
181
|
3. Advance the phases with the helpers (they rederive the top-level `status` pointer):
|
|
171
182
|
```bash
|
|
@@ -1,29 +1,33 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: canon-spec-review
|
|
3
3
|
description: Use when a canon task spec is written and the human wants to surface BLOCKING issues before invoking `canon run <id>`. Triggers on "/canon-spec-review", "review the spec", "pre-flight the spec", "what would Codex catch", or before kicking off the pipeline. Useful for any spec with logic — for full-tier (S/M/L/XL/delicate) it pre-empts Codex spec_review iterations, for fast-tier (XS non-delicate) it's the only automated review layer since Codex spec_review auto-approves.
|
|
4
|
-
argument-hint: "
|
|
5
|
-
allowed-tools: Read Glob Grep Bash(canon task list*) Bash(git status*) Agent
|
|
4
|
+
argument-hint: "[task-id]"
|
|
5
|
+
allowed-tools: Read Glob Grep Bash(canon task list*) Bash(git status*) Agent AskUserQuestion
|
|
6
6
|
effort: high
|
|
7
7
|
---
|
|
8
8
|
|
|
9
9
|
# canon-spec-review
|
|
10
10
|
|
|
11
|
-
Previews what Codex's `spec_review` phase would surface — BEFORE `canon run <id>` — by dispatching three parallel sub-agents at the spec from different angles. Returns one inline report; the human revises spec
|
|
11
|
+
Previews what Codex's `spec_review` phase would surface — BEFORE `canon run <id>` — by dispatching three parallel sub-agents at the spec from different angles. Returns one inline report; the human revises the spec or proceeds.
|
|
12
12
|
|
|
13
13
|
## When to use
|
|
14
14
|
|
|
15
15
|
- Spec is written, `human_spec_gate` not yet cleared
|
|
16
|
-
- Task is **full-tier (S/M/L/XL/delicate)** — pre-empts Codex spec_review iterations; saves
|
|
16
|
+
- Task is **full-tier (S/M/L/XL/delicate)** — pre-empts Codex spec_review iterations; saves time and cost
|
|
17
17
|
- Task is **fast-tier (XS non-delicate)** — this is the *only* automated review layer (fast-tier auto-approves `spec_review`, so without this skill the human's read is the only gate before Codex hits the spec at implement-time)
|
|
18
18
|
- After a reroute amendment, before re-invoking the pipeline
|
|
19
19
|
|
|
20
|
-
Don't use for
|
|
20
|
+
Don't use for code-diff review (use `/canon-inline-review`), or for XS changes with no logic to vet (typo fixes, version bumps).
|
|
21
21
|
|
|
22
22
|
## Workflow
|
|
23
23
|
|
|
24
24
|
### 1. Verify inputs
|
|
25
25
|
|
|
26
|
-
|
|
26
|
+
Resolve the task ID:
|
|
27
|
+
|
|
28
|
+
- **`$ARGUMENTS` names a task** → use it. This is how to review a task other than the one under discussion, or to run from a fresh session.
|
|
29
|
+
- **`$ARGUMENTS` is empty** → take the task from the conversation: the one just authored with `/canon-spec`, or the one the human has been discussing. Don't re-confirm an unambiguous pick.
|
|
30
|
+
- **Still unclear** (fresh session, or several tasks in play) → run `canon task list`, narrow to tasks at `spec` or `spec_review`, plus tasks at `plan` whose `status.json` shows `phases.plan.status: "pending"` (full tier waits at the human spec gate in that state). Use `AskUserQuestion` to pick one. If none match, say so rather than offering tasks already past the gate. Don't guess.
|
|
27
31
|
|
|
28
32
|
Verify `tasks/<id>/spec.md` exists and is filled out (no `<placeholder>` text or "TBD" stubs). If it's still a template, stop and say so.
|
|
29
33
|
|
|
@@ -35,65 +39,54 @@ State briefly: task ID, size, delicate flag, "dispatching 3 parallel sub-agents.
|
|
|
35
39
|
|
|
36
40
|
### 2. Dispatch 3 sub-agents in one message
|
|
37
41
|
|
|
38
|
-
Send a SINGLE message with three Agent tool calls so they run concurrently.
|
|
39
|
-
|
|
40
|
-
Each sub-agent returns findings in this format (no preamble, no closing remarks):
|
|
41
|
-
|
|
42
|
-
```
|
|
43
|
-
- [BLOCKING|STRONG|NIT] <one-line finding> — <file:line or AC#>
|
|
44
|
-
<2-3 sentence rationale, citing evidence>
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
**Calibration applied to every angle**: silence is the default. Only flag issues that would cause real problems during implementation. A real shape concern becomes the lead reason for a `changes_requested` verdict. Don't manufacture findings to look thorough.
|
|
42
|
+
Send a SINGLE message with three Agent tool calls so they run concurrently. Every call uses `subagent_type: spec-review-lens` and carries the full spec text inline, its angle, scope and rubric. Each lens returns cited findings in its own format. If the Agent tool rejects `spec-review-lens`, stop and tell the human to run `canon upgrade`, then start a new Claude Code session; do not fall back to another agent type.
|
|
48
43
|
|
|
49
44
|
#### Agent A — Structural / shape
|
|
50
45
|
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
Goal: answer "is this spec solving the right problem in the right shape?" Four questions: (1) is the problem real, (2) is the framing right, (3) is there a materially simpler solution, (4) is the AC decomposition right (compound ACs, missing ACs, ACs solving symptoms not causes)?
|
|
46
|
+
Scope: review whether the spec solves the right problem in the right shape. Answer four questions: is the problem real, is the framing right, is there a materially simpler solution, and is the AC decomposition right (compound ACs, missing ACs, ACs solving symptoms rather than causes)?
|
|
54
47
|
|
|
55
48
|
Constraints: don't audit factual claims (Agent B's job); don't audit completeness against canon's spec-writing rules (Agent C's job). Stay in shape territory.
|
|
56
49
|
|
|
57
|
-
Output: the format above. If no shape concerns, return exactly `- [NO FINDINGS]`.
|
|
58
|
-
|
|
59
50
|
#### Agent B — Factual / ground-truth
|
|
60
51
|
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
Goal: for each named symbol, file path, function, or behavior the spec mentions — does it exist? Is the description correct? Are Affected Files entries real paths? For symbols in ACs (e.g. "extend `parseFooBar` to handle X"): grep for the symbol, read the function signature, verify the spec's assumptions about return shape and call sites.
|
|
64
|
-
|
|
65
|
-
Constraints: cite `file:line` for every claim. Don't audit shape (Agent A) or stylistic completeness (Agent C).
|
|
52
|
+
Scope: verify every factual claim in the spec against the actual codebase. For each named symbol, file path, function, or behavior: does it exist, is the description correct, and are Affected Files entries real paths? For symbols in ACs, verify signatures, return shapes, and call sites.
|
|
66
53
|
|
|
67
|
-
|
|
54
|
+
Read whole functions and call sites, not excerpts, when verifying return shapes. Cite `file:line` for every claim. Don't audit shape (Agent A) or stylistic completeness (Agent C).
|
|
68
55
|
|
|
69
|
-
This angle catches the highest-value class:
|
|
56
|
+
This angle catches the highest-value class: a spec that assumes something exists in one place when it is elsewhere, or describes its contract incorrectly.
|
|
70
57
|
|
|
71
58
|
#### Agent C — Spec-quality completeness
|
|
72
59
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
Goal: check these spec-quality rules:
|
|
60
|
+
Scope: audit against canon's spec-writing rules of thumb. Check these ten items:
|
|
76
61
|
(1) **Name effects to DELETE** — when a change supersedes prior code, is it framed as a single replacement, not separate add/remove bullets?
|
|
77
62
|
(2) **Prefer positive or structural assertions** — are load-bearing "must not" constraints backed by a grep AC or positive reframe, not just prose negation?
|
|
78
|
-
(3) **Affected Files** —
|
|
63
|
+
(3) **Affected Files** — the table sits under a heading that is exactly `### Affected Files`, inside `## Design` or any amendment section (an H2 whose first word is `Amendment`, such as `## Amendment`, `## Amendment Round N` or `## Amendment (full-send scope)`). It lists every file the change writes, including generated mirrors and rebuilt outputs. Files only read for context stay out. A mismatched heading or a missing generated output is BLOCKING, because a pipeline gate rejects it later.
|
|
79
64
|
(4) **Validation Required** — section present AND has at least one `- [x]` checked entry (or an explicit checked "None — <reason>"). A section with zero `[x]` entries is a failing check.
|
|
80
65
|
(5) **Non-goals** — rule out the most tempting scope expansions.
|
|
81
66
|
(6) **Human Test Plan** — product language only; no code, no file paths.
|
|
82
67
|
(7) **Known Risks** — names actual failure modes for the trickiest ACs.
|
|
83
|
-
(8) **Symbols in ACs exist** — for any named function or symbol, has the author
|
|
84
|
-
(9) **Bug/flake-fix evidence** (N/A for features/refactors) — *Problem* states the confirmed mechanism and how it was confirmed, with evidence matching the mechanism class: a deterministic mechanism
|
|
68
|
+
(8) **Symbols in ACs exist** — for any named function or symbol, has the author verified the return shape matches the spec's assumed data contract?
|
|
69
|
+
(9) **Bug/flake-fix evidence** (N/A for features/refactors) — *Problem* states the confirmed mechanism and how it was confirmed, with evidence matching the mechanism class: a deterministic mechanism may cite a trace with verified trigger values; a runtime-dependent mechanism needs executed confirmation (a throwaway prototype-fix spike or a deterministic forced repro). *Acceptance Criteria* includes a red-first regression-test AC or the explicit environment-bound-and-impractical escape with a named deterministic alternative. Missing or under-rung evidence, or a missing red-first AC without the escape, is BLOCKING.
|
|
70
|
+
(10) **Refactor correctness audit evidence** (N/A for features/bug fixes) — does the spec record, for each behavior it declares preserved, whether it is correct today and one outcome: correct as-is, fixed deliberately (with its own AC), split into a separate task, or kept as a named quirk? A behavior confirmed correct needs only a one-line record. Missing audit evidence is STRONG, never BLOCKING.
|
|
71
|
+
|
|
72
|
+
Constraints: stay structural/completeness. Don't second-guess shape (Agent A) or independently re-verify symbols against the codebase (Agent B's job). Check (8) audits whether the author verified before writing the AC.
|
|
73
|
+
|
|
74
|
+
### Severity definitions and scope boundary
|
|
85
75
|
|
|
86
|
-
|
|
76
|
+
- **BLOCKING:** would cause wrong behavior or a silent bug, or makes an AC unimplementable as written.
|
|
77
|
+
- **NIT:** an implementation detail the implementer resolves by reading the codebase, a minor ambiguity with an obvious default, or a question the plan phase should answer.
|
|
78
|
+
- **STRONG:** nit-grade under those rules, but likely to cost a review round, so cheaper to fix now.
|
|
87
79
|
|
|
88
|
-
|
|
80
|
+
Behavior the spec's Non-Goals explicitly exclude and verify as unaffected is a NIT at most. An omitted required change (such as a caller, parser, migration or test surface), a transitive effect of the change, or an internal contradiction between spec sections stays BLOCKING.
|
|
89
81
|
|
|
90
82
|
### 3. Synthesize and report
|
|
91
83
|
|
|
92
|
-
|
|
84
|
+
Silence is the default for what reaches the report. Apply these filters:
|
|
93
85
|
|
|
94
|
-
1.
|
|
95
|
-
2.
|
|
96
|
-
3.
|
|
86
|
+
1. Drop any finding with no citation (uncited).
|
|
87
|
+
2. Downgrade any finding inside the scope boundary to NIT (scope-boundary downgrade).
|
|
88
|
+
3. de-dupe overlapping findings, keeping the more specific rationale.
|
|
89
|
+
4. Re-classify on the strongest evidence, then order BLOCKING, STRONG, NIT.
|
|
97
90
|
|
|
98
91
|
Print this inline (do NOT write to a file):
|
|
99
92
|
|
|
@@ -123,22 +116,21 @@ Print this inline (do NOT write to a file):
|
|
|
123
116
|
- Worth a quick pass — STRONG items will probably surface in spec_review; cheaper to fix now
|
|
124
117
|
```
|
|
125
118
|
|
|
126
|
-
If
|
|
119
|
+
If a lens returned `[NO FINDINGS]`, say so under its section. Don't pad. If all three returned no findings, the report is two lines: header + "Proceed — three sub-agents found no issues."
|
|
127
120
|
|
|
128
121
|
## Output principles
|
|
129
122
|
|
|
130
123
|
- **Inline report only.** No file writes. No status.json updates. No spec-review.md mutations.
|
|
131
124
|
- **Operator owns the decision.** This skill surfaces findings; it doesn't revise specs or invoke the pipeline.
|
|
132
|
-
- **One example of what this catches** (from canon-ai's own worktree-canonical-task-state spec authoring): a Pass 1 multi-agent run caught a `taskDirFor → resolveTaskCwd` rename that would have produced infinite recursion at every pipeline phase entry, plus 6 other BLOCKING items. Codex's spec_review would have surfaced these across 2-3 iterations (~30 min each); the skill closed them in one ~15-min pass.
|
|
133
125
|
|
|
134
126
|
## Anti-patterns
|
|
135
127
|
|
|
136
128
|
| Anti-pattern | Why it's wrong | Do instead |
|
|
137
129
|
|---|---|---|
|
|
138
|
-
| Run on
|
|
139
|
-
| Use to revise the spec | The skill doesn't edit files | Read the report, edit spec
|
|
140
|
-
| Manufacture findings to look thorough | Dilutes signal |
|
|
141
|
-
| Use for code-diff review | Spec mode only
|
|
130
|
+
| Run on XS typo fixes or version bumps | Overhead exceeds value when there is no logic to vet | Skip when there's no logic to vet; use freely on any spec that has decisions in it, XS included |
|
|
131
|
+
| Use to revise the spec | The skill doesn't edit files | Read the report, edit the spec manually |
|
|
132
|
+
| Manufacture findings to look thorough | Dilutes signal | `[NO FINDINGS]` is a real verdict; report only cited findings |
|
|
133
|
+
| Use for code-diff review | Spec mode only | Use `/canon-inline-review` for diff review |
|
|
142
134
|
|
|
143
135
|
---
|
|
144
136
|
|
|
@@ -168,8 +168,8 @@ Claude writes QA summary → Human tests
|
|
|
168
168
|
|
|
169
169
|
- Spec and plan are written in separate Claude sessions.
|
|
170
170
|
- Codex runs a real spec review before the gate. Spec review starts with a **Shape Check** (is the problem real? is the framing right? is there a materially simpler solution? is the AC decomposition right?) before the implementability probe.
|
|
171
|
-
- Codex model/effort
|
|
172
|
-
- **Optional pre-pipeline self-review**: before invoking `canon run`, the operator can run `/canon-spec-review
|
|
171
|
+
- Effective size selects both the Codex and Claude model/effort cells in the matrices below.
|
|
172
|
+
- **Optional pre-pipeline self-review**: before invoking `canon run`, the operator can run `/canon-spec-review` (task ID optional — it defaults to the task under discussion) to dispatch three parallel sub-agents (structural / factual / spec-quality) at the spec and surface BLOCKING / STRONG / NIT findings inline. Catches the class of issues Codex's spec_review would surface across 2-3 iterations in one ~15-min pass. Opt-in; most valuable when iteration cost is real.
|
|
173
173
|
|
|
174
174
|
**Where validation happens**: Project-specific checks (lint, type-check, unit tests, e2e, etc.) run inside agent phases — Codex runs them during `implement` and records outcomes in the handoff; Claude verifies the outcomes table in Stage 1 code review and re-runs selectively when anything looks off. There is no separate orchestrator-run validation phase.
|
|
175
175
|
|
|
@@ -183,8 +183,8 @@ Set in `status.json` at task creation:
|
|
|
183
183
|
|
|
184
184
|
| Field | Values | Purpose |
|
|
185
185
|
|---|---|---|
|
|
186
|
-
| `task_size` | `XS \| S \| M \| L \| XL` | Drives
|
|
187
|
-
| `delicate` | `true \| false` | Forces the XL bucket (full Codex model
|
|
186
|
+
| `task_size` | `XS \| S \| M \| L \| XL` | Drives the pipeline tier and effective-size model/effort cells for Codex and Claude. XS is fast-tier; S+ runs the full pipeline (including Codex spec review). |
|
|
187
|
+
| `delicate` | `true \| false` | Forces the XL bucket (full Codex model and high Claude spec/plan/code-review effort) regardless of nominal size. Set when an undetected bug has materially harder-to-recover blast radius than a normal bug — common examples: auth, payments, premium gating, persistent storage migrations, security-sensitive cryptography. Project-specific surfaces also qualify (medical PHI, scientific reproducibility, regulated data). The bar is *blast radius*, not difficulty. |
|
|
188
188
|
| `human_spec_gate` | `true \| false` | **Single-use latch**, not a persistent toggle. `true` arms a one-time halt after `spec_review`, before planning (default: `true`). The orchestrator flips it to `false` *at the moment it halts* — so `false` means "the gate already fired (or was pre-cleared)," not "review was skipped." See [Spec gate is a single-use latch](#spec-gate-is-a-single-use-latch). |
|
|
189
189
|
| `worktree` | `true \| false` | Worktree isolation. `canon task new` scaffolds this to `true`, so worktree mode is the effective default for every scaffolded task — set it to `false` to opt out and run in the main checkout. The orchestrator treats an *absent* field as `false`, but that fallback only applies to hand-rolled `status.json` files. See Worktree Isolation below. |
|
|
190
190
|
| `base_branch` | string (default `"main"`) | Branch the task branches off and PRs against. Auto-set by `canon task new` from the current git checkout at task creation. |
|
|
@@ -223,7 +223,20 @@ Codex is tuned for token efficiency — the mini model handles most phases; the
|
|
|
223
223
|
|
|
224
224
|
The cold-Codex `code_review` lens is the exception to size scaling: as a mandatory hard-fail gate, it runs at flat `high` effort and stays on the mini model at every size, including XL/delicate.
|
|
225
225
|
|
|
226
|
-
`spec_review` runs at `high` from M upward. Task-history analysis found M's excess code_review iterations weren't an implement-quality gap (non-rerouted M and L tasks ran at nearly identical iteration counts) but a reroute-severity gap, with M's lighter spec_review effort the leading hypothesis. M and L also differ on loop cap
|
|
226
|
+
`spec_review` runs at `high` from M upward. Task-history analysis found M's excess code_review iterations weren't an implement-quality gap (non-rerouted M and L tasks ran at nearly identical iteration counts) but a reroute-severity gap, with M's lighter spec_review effort the leading hypothesis. M and L also differ on loop cap and budget, so this isn't a proven sole cause; if you retune the matrix, re-measure your own M vs. L reroute rate.
|
|
227
|
+
|
|
228
|
+
## Claude Model/Effort Matrix
|
|
229
|
+
|
|
230
|
+
Claude model and effort scale with effective task size. The light tier defaults to `sonnet`, and the strong tier defaults to `opus`:
|
|
231
|
+
|
|
232
|
+
| Phase | XS | S | M | L | XL / delicate |
|
|
233
|
+
|---|---|---|---|---|---|
|
|
234
|
+
| `spec` | sonnet / medium | sonnet / medium | opus / medium | opus / medium | opus / high |
|
|
235
|
+
| `plan` | sonnet / medium | sonnet / medium | opus / medium | opus / medium | opus / high |
|
|
236
|
+
| `code_review` | sonnet / medium | sonnet / medium | opus / medium | opus / medium | opus / high |
|
|
237
|
+
| `qa` | sonnet / medium | sonnet / medium | sonnet / medium | sonnet / medium | sonnet / medium |
|
|
238
|
+
|
|
239
|
+
`spec`, `plan`, and `code_review` move to the strong tier at M, where sustained judgment matters; only XL/delicate raises effort to `high`. `qa` writes completion artifacts and cannot block or reroute a task, so it stays light/medium at every size. No Claude cell runs above `high`.
|
|
227
240
|
|
|
228
241
|
## Claude Budget Matrix
|
|
229
242
|
|
|
@@ -238,15 +251,18 @@ Claude phase budgets scale with task size. `spec`/`plan`/`qa` are single-pass Cl
|
|
|
238
251
|
|
|
239
252
|
## Environment Variables
|
|
240
253
|
|
|
241
|
-
Claude
|
|
254
|
+
Claude uses the light tier for XS/S spec, plan, and code review and for QA at every size. M and above use the strong tier for spec, plan, and code review. A phase pin changes only the model; effort comes from the matrix.
|
|
242
255
|
|
|
243
256
|
| Variable | Default | Purpose |
|
|
244
257
|
|---|---|---|
|
|
245
|
-
| `
|
|
246
|
-
| `
|
|
247
|
-
| `
|
|
248
|
-
| `
|
|
249
|
-
| `
|
|
258
|
+
| `CLAUDE_MODEL_LIGHT` | `sonnet` | Model for light-tier cells: XS/S `spec`/`plan`/`code_review`, and `qa` at every size, unless a phase pin applies. |
|
|
259
|
+
| `CLAUDE_MODEL_STRONG` | `opus` | Model for strong-tier cells: M and above `spec`/`plan`/`code_review`, unless a phase pin applies. |
|
|
260
|
+
| `CLAUDE_MODEL_SPEC` | _(unset)_ | Pins the `spec` model at every size, replacing the model only; effort still comes from the matrix. |
|
|
261
|
+
| `CLAUDE_MODEL_PLAN` | _(unset)_ | Pins the `plan` model at every size where a separate plan phase runs (XS fast tier combines spec and plan under the `spec` cell), replacing the model only; effort still comes from the matrix. |
|
|
262
|
+
| `CLAUDE_MODEL_REVIEW` | _(unset)_ | Pins the `code_review` model for XS/S/M/L only, replacing the model only; effort still comes from the matrix. Does not reach XL/delicate. |
|
|
263
|
+
| `CLAUDE_MODEL_REVIEW_LARGE` | _(unset)_ | Pins the `code_review` model for XL/delicate only, replacing the model only; effort still comes from the matrix. `_LARGE` means task size, not model tier. Does not reach XS–L. |
|
|
264
|
+
| `CLAUDE_MODEL_QA` | _(unset)_ | Pins the `qa` model at every size, replacing the model only; effort still comes from the matrix. |
|
|
265
|
+
| `CLAUDE_MODEL` | _(unset; deprecated)_ | Legacy catch-all model for every Claude phase, including `qa`, only where no phase pin or light/strong tier variable is set. Phase pin → tier variable → this fallback → tier default. |
|
|
250
266
|
| `CLAUDE_BUDGET` | _(phase- and size-aware)_ | Max spend per Claude phase (USD). Unset → resolved from the Claude Budget Matrix above (phase × size). Set → flat cap applied uniformly across every phase and size (e.g. `CLAUDE_BUDGET=15.00` overrides every phase to $15). |
|
|
251
267
|
| `CANON_PROJECT_NAME` | _(reads `package.json` "name" or "your project")_ | Name injected into agent prompts. |
|
|
252
268
|
| `CANON_WORKTREES_ROOT` | .canon/worktrees | Where task worktrees are created. When overridden, the orchestrator warns if the path isn't in `.claude/settings*.json` `additionalDirectories`. |
|
|
@@ -361,6 +377,7 @@ After `spec_review` or `code_review`, the orchestrator checks the verdict.
|
|
|
361
377
|
| `spec_review` | `changes_requested` | Loop back to `spec` automatically, except reroute-amendment reviews block for human revision. |
|
|
362
378
|
| `code_review` | `changes_requested` / `needs_re_review` | Loop back to `implement` automatically. |
|
|
363
379
|
| `code_review` | `spec_gap` | Block the whole `code_review` bundle with an escalation. Fix path: amend the `spec_gap` task specs and run `canon run <ids> --reroute`. Bless path: `canon task accept <ids> code_review --reason "<why>"`. |
|
|
380
|
+
| `code_review` | reviewer-set `blocked` (no verdict) | Block the whole `code_review` bundle with an escalation and exit `2`; no evidence recovery or retry is attempted. The stop names each task's `review.md`. Recovery: `canon task reset-code-review <id>` for each task, then `canon run <ids>`. |
|
|
364
381
|
| `spec_review` / `code_review` | `approved` / `approved_with_nits` / `sanctioned` | Continue to the next phase. `sanctioned` is status-only and written by `canon task accept`, not by review artifacts; `canon task accept` refuses a review phase with no recorded verdict unless `--force` is passed. |
|
|
365
382
|
|
|
366
383
|
**Auto-block on runaway loops**: When spec review or code review reaches or exceeds the size-aware cap (3 for XS/S/M, 5 for L/XL, or `MAX_REVIEW_LOOPS` if set), the orchestrator auto-blocks that review phase and appends an entry to `escalations` in `status.json`. The primary check runs at the next revision phase's entry, before a spec revision or re-implementation starts; the review phase retains the same check as a defense-in-depth backstop for unexpected state. For code review the cap check is the **sum** of `iterations_current_loop` and `preflight_rejections_current_loop` — handoffs bounced by the deterministic pre-flight gate count toward the same budget, and recovery requires resetting both counters (the block message says which). Separately, a pre-flight where every blocker is a `blocked` validation row auto-blocks immediately without consuming loop budget. On approval, `iterations_current_loop` resets to `0` while `iterations_total` (lifetime verdict count) and `auto_block_count` are preserved — history is never erased.
|