@seanyao/roll 3.628.1 → 4.629.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/README.md +71 -11
- package/dist/roll.mjs +19867 -18764
- package/package.json +1 -1
- package/skills/roll-.qa/SKILL.md +4 -0
- package/skills/roll-.review/SKILL.md +4 -0
- package/skills/roll-build/SKILL.md +6 -1
- package/skills/roll-build/references/full-contract.md +79 -8
- package/skills/roll-design/SKILL.md +6 -1
- package/skills/roll-design/references/full-contract.md +56 -0
- package/skills/roll-fix/SKILL.md +4 -0
- package/skills/roll-fix/references/full-contract.md +6 -5
- package/skills/roll-peer/SKILL.md +4 -0
- package/skills/roll-propose/SKILL.md +6 -0
package/package.json
CHANGED
package/skills/roll-.qa/SKILL.md
CHANGED
|
@@ -45,3 +45,7 @@ Load when a build, fix, review, or story needs Roll QA coverage standards: test
|
|
|
45
45
|
- Description changes require updates in `route-cases/skills.json`.
|
|
46
46
|
- New observed failures should add a gotcha and the matching positive or negative route case.
|
|
47
47
|
- Heavy examples, templates, recovery paths, and deterministic snippets belong in `references/`, `assets/`, or `scripts/`, not in this hub.
|
|
48
|
+
|
|
49
|
+
## Role in v4 execution profiles
|
|
50
|
+
|
|
51
|
+
These QA standards are an **Evaluator capability** in the `verified`/`planned` execution profiles: the independent Evaluator applies them to judge story satisfaction and produces an `eval-report.md` (blocking findings, advisory findings, score, attest/evidence status, recommendation) — distinct from CI feedback and from the Builder's self-report. (Roles: Supervisor Agent / Planner / Builder / Evaluator; never Prime/Watchman/Dispatcher/Governor.)
|
|
@@ -146,3 +146,7 @@ In each micro-step of `$roll-build`:
|
|
|
146
146
|
- **Check Phase**: Local quality control
|
|
147
147
|
- **Micro-steps**: Small fast steps, each review < 100 lines
|
|
148
148
|
- **TCR**: Can only commit after passing self-check, ensuring repo quality
|
|
149
|
+
|
|
150
|
+
## Role in v4 execution profiles
|
|
151
|
+
|
|
152
|
+
Self-review is a TCR-step quality gate **within the Builder role** — it is NOT story acceptance. In the `verified`/`planned` execution profiles the independent **Evaluator** (a fresh session, never the Builder's) judges whether the delivery satisfies the contract; blocking review, score, and attest stay three SEPARATE dimensions (never one pass/fail). This skill is one Evaluator capability among review/qa/scoring. (Roles: Supervisor Agent / Planner / Builder / Evaluator; never Prime/Watchman/Dispatcher/Governor.)
|
|
@@ -26,7 +26,7 @@ Load when a user gives a US-XXX story or asks to ship a feature through Roll TCR
|
|
|
26
26
|
|
|
27
27
|
1. Classify input as Story mode or Fly mode.
|
|
28
28
|
2. Read backlog/spec and decide if the story fits one cycle.
|
|
29
|
-
3. Split into small TCR actions and define verification.
|
|
29
|
+
3. Split into small TCR actions; per Action route objective work to the 4-stage TCR loop and judgment-dependent work to a 3-stage criteria+Evaluator loop, then define verification.
|
|
30
30
|
4. Run test-first implementation, self-review, evidence, and write-back.
|
|
31
31
|
5. Commit on green, open PR, and leave remote evidence.
|
|
32
32
|
|
|
@@ -34,6 +34,7 @@ Load when a user gives a US-XXX story or asks to ship a feature through Roll TCR
|
|
|
34
34
|
|
|
35
35
|
- TCR for every micro-step.
|
|
36
36
|
- No WIP commits or bypassed verification.
|
|
37
|
+
- In the test-gated loop the tests are the weakest link: an isolated Test Adequacy reviewer (Phase 6 Agent 4) audits the tests against the AC, seeing only AC + test files — never the implementation diff or builder reasoning. Self-review of one's own tests (Phase 2) is necessary but not isolation.
|
|
37
38
|
- Self-review, attest, and E2E evidence remain required. The Review Score is produced by the runner's fresh-session peer Reviewer — the agent does NOT self-score.
|
|
38
39
|
- Docs/code/product alignment is a DoD gate: user-visible behavior, command, output-copy, site, or delivery-view changes update the touched README/docs/guide/site/help in the same delivery.
|
|
39
40
|
|
|
@@ -47,3 +48,7 @@ Load when a user gives a US-XXX story or asks to ship a feature through Roll TCR
|
|
|
47
48
|
- Description changes require updates in `route-cases/skills.json`.
|
|
48
49
|
- New observed failures should add a gotcha and the matching positive or negative route case.
|
|
49
50
|
- Heavy examples, templates, recovery paths, and deterministic snippets belong in `references/`, `assets/`, or `scripts/`, not in this hub.
|
|
51
|
+
|
|
52
|
+
## Role in v4 execution profiles
|
|
53
|
+
|
|
54
|
+
**roll-build is the Builder capability in every execution profile** (`standard` / `verified` / `planned`). Under `planned` it receives the Planner's `planner-contract.md` via artifact refs (no shared raw session). Under `verified`/`planned` an independent **Evaluator** (a fresh session) judges the delivery and may open a BOUNDED repair round — you address the blocking findings and write a repair note mapping findings → changes. `standard` is Builder-only. (Roles: Supervisor Agent / Planner / Builder / Evaluator; never Prime/Watchman/Dispatcher/Governor.)
|
|
@@ -117,6 +117,7 @@ the command decides. The cap exists purely to stop infinite split chains.
|
|
|
117
117
|
1. Open `.roll/backlog.md`, find the US row, follow the link to `.roll/features/<epic>/<story>/spec.md`
|
|
118
118
|
2. Read the full AC / Files / Dependencies section
|
|
119
119
|
3. If a plan doc (`<feature>-plan.md`) exists, read it for context
|
|
120
|
+
4. **Read the Evaluation contract (US-SKILL-030)**: if the spec contains an `**Evaluation contract:**` block, read `expected_evidence` and `scorer_focus` before writing any code. This is the planner's artifact contract — use it to inform test design and evidence planning. Map each `expected_evidence` item to a planned Action or test; if an item is impossible given the actual scope, note the deviation in your report/ac-map. `builder_notes` carry planner hints — treat them as bounded guidance, not hidden requirements.
|
|
120
121
|
|
|
121
122
|
### Step A3: Split into Actions
|
|
122
123
|
|
|
@@ -180,6 +181,43 @@ When parallel conditions are not met, execute Actions sequentially.
|
|
|
180
181
|
- What "online verification" means for this repo (URL, endpoint, UI flow, log signal)
|
|
181
182
|
- Reference `$roll-.qa` for test pyramid (unit → E2E → visual → smoke)
|
|
182
183
|
|
|
184
|
+
#### A4.1 Testability routing (objective 4-stage vs subjective 3-stage)
|
|
185
|
+
|
|
186
|
+
Before defining the verification, classify **each Action** on one axis:
|
|
187
|
+
|
|
188
|
+
> **Can this Action's acceptance be expressed as deterministic pass/fail tests?**
|
|
189
|
+
|
|
190
|
+
```
|
|
191
|
+
Action acceptance
|
|
192
|
+
├── YES (deterministic) → objective 4-stage loop ← DEFAULT
|
|
193
|
+
│ Planner (AC/contract) → Tester (Phase 2 + write RED)
|
|
194
|
+
│ → Builder (Phase 3 TCR) → Evaluator (Phase 6, incl. Agent 4 test audit)
|
|
195
|
+
│
|
|
196
|
+
└── NO (judgment-dependent) → subjective 3-stage loop
|
|
197
|
+
Planner emits EXPLICIT evaluation criteria up front
|
|
198
|
+
→ Builder produces the output
|
|
199
|
+
→ an ISOLATED Evaluator scores the output against those criteria
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
Most code Actions are objective — the test IS the contract; stay on the default
|
|
203
|
+
path. Route to the 3-stage loop only for the genuinely judgment-dependent slices
|
|
204
|
+
that still ride inside a delivery: output-copy wording, layout / visual polish,
|
|
205
|
+
docs readability, recommendation quality. For those:
|
|
206
|
+
|
|
207
|
+
- **Write the criteria before building** — the same discipline as a test, but in
|
|
208
|
+
prose: a short checklist of what "good" means for this Action (e.g. the Phase 11
|
|
209
|
+
design-QA checklist is exactly such a criteria set for a visual surface).
|
|
210
|
+
- **Do not fake a test** to force the work onto the TCR path, and **do not let the
|
|
211
|
+
building agent grade its own output** — that is the self-score red line (FIX-343).
|
|
212
|
+
- **An isolated Evaluator scores against the criteria** — a fresh session or
|
|
213
|
+
`$roll-peer` (which already enforces the Independent Judgment Rule: the evaluator
|
|
214
|
+
is not seeded with the builder's reasoning). Its verdict feeds the same
|
|
215
|
+
fix-and-recheck loop as a failing test.
|
|
216
|
+
|
|
217
|
+
A single Story usually mixes both: the logic is objective (4-stage), while a copy
|
|
218
|
+
or readability slice inside it is subjective (3-stage). Classify per Action, not
|
|
219
|
+
per Story.
|
|
220
|
+
|
|
183
221
|
Proceed to the **Shared TCR Workflow** (Phase 1 onward).
|
|
184
222
|
|
|
185
223
|
---
|
|
@@ -297,6 +335,12 @@ Reference `$roll-.qa` for coverage requirements and test pyramid strategy.
|
|
|
297
335
|
|
|
298
336
|
**Why this phase**: TCR only guarantees code passes tests — verify tests are correct first.
|
|
299
337
|
|
|
338
|
+
> This self-review is necessary but **not** sufficient: the same agent that
|
|
339
|
+
> wrote the tests judging its own tests is not isolation. It catches obvious
|
|
340
|
+
> gaps now; the *independent* adequacy audit (no implementation, no builder
|
|
341
|
+
> reasoning) happens at **Phase 6 Agent 4 — Test Adequacy Review**. Treat this
|
|
342
|
+
> step as "don't ship obviously-wrong tests," not as the final word on coverage.
|
|
343
|
+
|
|
300
344
|
### Phase 3: TCR Implementation Loop
|
|
301
345
|
|
|
302
346
|
```
|
|
@@ -477,27 +521,53 @@ across files, copy-paste patterns, cross-file N+1, etc.).
|
|
|
477
521
|
git diff main...HEAD
|
|
478
522
|
```
|
|
479
523
|
|
|
480
|
-
**Launch
|
|
524
|
+
**Launch four review agents in parallel:**
|
|
481
525
|
|
|
482
526
|
```
|
|
483
|
-
Agent 1: Reuse Review
|
|
527
|
+
Agent 1: Reuse Review (receives: full diff)
|
|
484
528
|
→ Search for existing utilities / helpers the new code could use instead
|
|
485
529
|
→ Flag any new function that duplicates existing functionality
|
|
486
530
|
→ Flag inline logic replaceable by existing tools
|
|
487
531
|
|
|
488
|
-
Agent 2: Quality Review
|
|
532
|
+
Agent 2: Quality Review (receives: full diff)
|
|
489
533
|
→ Redundant state, Parameter sprawl, Copy-paste near-duplicate,
|
|
490
534
|
Leaky abstraction, Stringly-typed, JSX nesting,
|
|
491
535
|
Nested conditionals ≥3 deep, Unnecessary comments
|
|
492
536
|
|
|
493
|
-
Agent 3: Efficiency Review
|
|
537
|
+
Agent 3: Efficiency Review (receives: full diff)
|
|
494
538
|
→ Redundant computation / N+1, Missed concurrency,
|
|
495
539
|
Hot-path bloat, Loop no-op updates, TOCTOU existence pre-check,
|
|
496
540
|
Memory leaks, Overly broad operations
|
|
541
|
+
|
|
542
|
+
Agent 4: Test Adequacy Review (receives: AC/contract + test files ONLY)
|
|
543
|
+
→ Audit the TESTS, not the implementation. This agent does NOT receive
|
|
544
|
+
the implementation diff or any builder reasoning — only (a) the Story's
|
|
545
|
+
AC / interface contract (the Planner artifact) and (b) the test files.
|
|
546
|
+
→ AC ↔ assertion coverage: every AC has at least one test that would fail
|
|
547
|
+
if that AC regressed. Flag any AC with no corresponding assertion.
|
|
548
|
+
→ Tautological / vacuous assertions: tests that only assert "did not throw",
|
|
549
|
+
re-assert the mock, or compare a value to itself — green but proves nothing.
|
|
550
|
+
→ Tests loosened to pass: an assertion weakened (widened matcher, deleted
|
|
551
|
+
case, `.skip`, snapshot blindly updated) so a thin implementation goes green.
|
|
552
|
+
→ Missing edge / failure cases the AC names (boundaries, invalid input,
|
|
553
|
+
concurrency, error paths) that have no test.
|
|
497
554
|
```
|
|
498
555
|
|
|
499
|
-
|
|
500
|
-
|
|
556
|
+
**Why Agent 4 is isolated (the test-gated-loop insight)**: in the TCR loop the
|
|
557
|
+
implementation is *already* constrained by the tests — a thin or wrong
|
|
558
|
+
implementation cannot stay green. The weakest link is therefore the **tests
|
|
559
|
+
themselves**, and Phase 2's test-design review is a *self*-review by the same
|
|
560
|
+
agent that wrote them (no isolation). Agent 4 closes that gap: a fresh,
|
|
561
|
+
context-isolated reviewer that sees only the contract and the tests audits
|
|
562
|
+
whether the tests actually pin the behavior the AC promises. Seeding it with the
|
|
563
|
+
implementation diff or the builder's reasoning would collapse the independence
|
|
564
|
+
(same red line as the FIX-343 Review Score and roll-peer's Independent Judgment
|
|
565
|
+
Rule) — so it is deliberately withheld.
|
|
566
|
+
|
|
567
|
+
Wait for all four agents to complete. Aggregate findings → fix each issue
|
|
568
|
+
(false positives: note and skip, no debate) → summarize what was fixed. A
|
|
569
|
+
genuine test-adequacy gap (uncovered AC, vacuous assertion) is fixed by a new
|
|
570
|
+
TCR cycle that adds the missing test FIRST (RED), never by relaxing the audit.
|
|
501
571
|
|
|
502
572
|
**Fallback**: If parallel agent invocation fails, run `$roll-.review staged` on
|
|
503
573
|
the full diff as a single-pass fallback — do not skip review entirely.
|
|
@@ -622,8 +692,8 @@ story (loop or manual Phase 10.6) and earn the report at delivery time.
|
|
|
622
692
|
tmux observation window `roll-loop-<slug>` is a natural target — display the
|
|
623
693
|
proof there), then `screencapture -x -R <window-rect>` (macOS) into
|
|
624
694
|
`screenshots/`. Capture ONLY the relevant work area — a focused window, not
|
|
625
|
-
the whole desktop. Unattended cycles: drive the capture from the
|
|
626
|
-
(deterministic), never hand-craft an image; if the capture channel is
|
|
695
|
+
the whole desktop. Unattended cycles: drive the capture from the runner's
|
|
696
|
+
capture lane (deterministic), never hand-craft an image; if the capture channel is
|
|
627
697
|
unavailable (no GUI session / no permission), fall back to text evidence and
|
|
628
698
|
mark the AC `partial` with a note — never fake a screenshot.
|
|
629
699
|
|
|
@@ -650,6 +720,7 @@ story (loop or manual Phase 10.6) and earn the report at delivery time.
|
|
|
650
720
|
] }]
|
|
651
721
|
```
|
|
652
722
|
|
|
723
|
+
**Cross-reference with the Evaluation contract** (when present): each `expected_evidence` item in the contract maps to an AC via its `proves` field. When writing ac-map entries, ensure every `expected_evidence` item is addressed — the attest gate later surfaces a planned-vs-delivered delta from this mapping (US-SKILL-030).
|
|
653
724
|
No evidence for an AC → say `claimed` yourself; the renderer enforces that downgrade anyway (red line) and lists it under Discrepancies.
|
|
654
725
|
3. **Run** `roll attest {ID}` (add `--deploy-url <url>` when one exists). The report lands at `.roll/features/<epic>/{ID}/latest/{ID}-report.html` (archive-per-card layout, US-META-001). The report is now layered (US-ATTEST-013): card context + conclusion/business badges + key screenshots up front, technical ANSI/command output folded into collapsed `<details>`, and a closing block (quality gate + evidence index + Review Score).
|
|
655
726
|
4. **Design QA checklist (US-ATTEST-013) — READABILITY ONLY**. After the report
|
|
@@ -28,7 +28,7 @@ Load when the user wants to discuss approaches, design a solution, model domains
|
|
|
28
28
|
1. Clarify only when intent or boundaries are unclear.
|
|
29
29
|
2. Model domain depth proportional to risk and novelty.
|
|
30
30
|
3. Detailed design before decomposition — for any non-trivial work, produce a concrete, implementable design artifact and get owner sign-off: (a) data/contract schema, (b) AT LEAST ONE complete worked sample of the intended output/behavior, (c) key interface signatures, (d) mapping/normalization rules, (e) edge cases & failure modes. Depth scales with risk/novelty; trivial work may be light.
|
|
31
|
-
4. Split into INVEST stories — each a slice of the agreed detailed design.
|
|
31
|
+
4. Split into INVEST stories — each a slice of the agreed detailed design. For every newly split story, write an `**Evaluation contract:**` block (expected_evidence + scorer_focus) — see the full contract's Story Format for the template. Genuinely trivial/internal stories may use a one-item minimal block, but never omit it.
|
|
32
32
|
5. Write specs through roll story new and update backlog.
|
|
33
33
|
|
|
34
34
|
## Hard Gates
|
|
@@ -37,6 +37,7 @@ Load when the user wants to discuss approaches, design a solution, model domains
|
|
|
37
37
|
- Backlog rows and spec files must stay consistent.
|
|
38
38
|
- Peer review gates apply only when explicitly available/requested.
|
|
39
39
|
- No story decomposition until a detailed design exists and the owner has signed off (proportional to risk). Decomposition slices an agreed design — it is NOT a substitute for designing. If you cannot show at least one complete worked sample of the intended output/behavior, the design is NOT done.
|
|
40
|
+
- **Evaluation contract (US-SKILL-030 — planner-builder-evaluator artifact contract)**: every newly split story spec MUST include an `**Evaluation contract:**` block with `expected_evidence` (each item: `kind`, `target`, `proves`) and `scorer_focus`. This artifact is authored by the planner and consumed by builder (roll-build/roll-fix reads before coding) and evaluator (peer score prompt reads the contract; attest surfaces planned-vs-delivered mapping) — not a fixed three-agent collaboration model. Genuinely trivial/internal stories may carry a one-item minimal block, but never omit it. See `references/full-contract.md` for the full template and rules.
|
|
40
41
|
- **Visual-evidence contract (FIX-311 — design-phase gate)**: every story spec is born honest. By default each story MUST carry one AC that captures its user-visible surface (web/CLI/TUI), and a web/visual card MUST declare the real product surface in its spec frontmatter — `deliverable_url:` (alias `screenshot_url:`) pointing at the actual deliverable page (e.g. `.roll/features/index.html#casting`), NEVER the card's own dossier/report page. A card with genuinely no visual surface writes `screenshot_exempt: <reason>` (a naked `true`/`yes` is NOT a valid exemption — it must carry a reason). This is enforced, not advisory: `validateStoryVisualEvidence(specText)` in `packages/cli/src/lib/design-visual-evidence.ts` returns `ok:false` for a non-exempt spec with no visual-evidence AC, or one that declares a visual surface but no `deliverable_url`. Keyword matching may only RECOGNISE an exemption / an existing visual-evidence AC — it may NEVER be used to decide a card needs a screenshot (it always does, by default). This is the same contract the runtime enforce gate (FIX-309) and archive gate (FIX-334) hold; the three must agree.
|
|
41
42
|
|
|
42
43
|
## Gotchas
|
|
@@ -50,3 +51,7 @@ Load when the user wants to discuss approaches, design a solution, model domains
|
|
|
50
51
|
- Description changes require updates in `route-cases/skills.json`.
|
|
51
52
|
- New observed failures should add a gotcha and the matching positive or negative route case.
|
|
52
53
|
- Heavy examples, templates, recovery paths, and deterministic snippets belong in `references/`, `assets/`, or `scripts/`, not in this hub.
|
|
54
|
+
|
|
55
|
+
## Role in v4 execution profiles
|
|
56
|
+
|
|
57
|
+
In the `planned` execution profile, **roll-design is the Planner capability**: in a FRESH session before the Builder it writes a `planner-contract.md` (scope boundary, acceptance/evaluation contract, expected evidence, risks, out-of-scope items, resize/split guidance) that the Builder consumes via artifact refs. roll-design stays a skill — the TS engine owns orchestration and validates the contract fail-closed before the Builder starts. For pure design/backlog planning the user loads roll-design directly; it does not auto-trigger. (Roles: Supervisor Agent / Planner / Builder / Evaluator; never Prime/Watchman/Dispatcher/Governor.)
|
|
@@ -721,6 +721,20 @@ Note: `{DOMAIN}` maps to the Bounded Context name identified in DDD analysis.
|
|
|
721
721
|
- [ ] {measurable criteria 3}
|
|
722
722
|
- [ ] {visual-evidence AC — REQUIRED by default: a captured screenshot of this story's user-visible surface (web/CLI/TUI). For a web/visual card this is "screenshot of <the deliverable page> is captured", paired with the `deliverable_url:` frontmatter below. Omit this AC ONLY when the card is `screenshot_exempt:` (see frontmatter).}
|
|
723
723
|
|
|
724
|
+
**Evaluation contract:**
|
|
725
|
+
- expected_evidence:
|
|
726
|
+
- kind: test | command | screenshot | document | diff | ci | manual
|
|
727
|
+
target: {file/command/surface/report expected to prove an AC}
|
|
728
|
+
proves: {AC id or short AC phrase this evidence proves}
|
|
729
|
+
- scorer_focus:
|
|
730
|
+
- {what the peer Review Score should judge beyond generic code quality}
|
|
731
|
+
- builder_notes:
|
|
732
|
+
- {bounded implementation/evidence hints; no hidden requirements}
|
|
733
|
+
|
|
734
|
+
> **强制规则 — Evaluation contract 必须填**:每张新 story 必须带 `**Evaluation contract:**` 块(expected_evidence + scorer_focus)。这是 planner 写给 builder 和 evaluator 的验收契约——builder 编码前读 contract、交付后把 ac-map 项映射回 contract 的 expected_evidence;evaluator(peer score / attest gate)读 contract 对账。不是固定三 agent 协同模型,是 artifact-based 契约。真正 trivial/internal 故事可放一项 minimal block,但不许省略。
|
|
735
|
+
>
|
|
736
|
+
> **MUST fill** the `**Evaluation contract:**` block on every newly split story. This is an artifact contract authored by roll-design, consumed by builder (read before coding, map delivered evidence back) and evaluator (peer score prompt includes it, attest surfaces planned-vs-delivered). Genuinely trivial/internal stories may use a one-item minimal block, never omit it.
|
|
737
|
+
|
|
724
738
|
**Spec frontmatter (visual-evidence contract — FIX-311):**
|
|
725
739
|
- A web/visual card MUST declare the real product surface it delivers:
|
|
726
740
|
```yaml
|
|
@@ -757,6 +771,25 @@ Note: `{DOMAIN}` maps to the Bounded Context name identified in DDD analysis.
|
|
|
757
771
|
>
|
|
758
772
|
> **MUST fill** the `**Agent profile:**` block on every newly split US. `est_min` is the **sole loop-routing input** — `lib/loop_pick_agent.py` maps it onto a four-slot complexity tier (easy/default/hard/fallback) on the est_min axis alone (missing/illegal → default). Fill `risk_zone` too, but note it does NOT feed routing; it only informs the roll-build / roll-fix pre-flight self-eval (US-AGENT-007).
|
|
759
773
|
|
|
774
|
+
> **强制规则 — Evaluation contract 必须填(US-SKILL-030)**:每张新 story spec 都必须包含 `**Evaluation contract:**` 块,明确列出 `expected_evidence`(每项 kind/target/proves 三字段)和 `scorer_focus`(peer scorer 评分口径)。纯内部/无可见面的 trivial story 可使用 one-item 最小块,但不可省略。该块由 planner(`roll-design`)撰写,供 builder 和 evaluator 作为共享证据契约消费,不触发三 agent 协同会话。
|
|
775
|
+
|
|
776
|
+
> **Evaluation contract is required (US-SKILL-030)**: every newly split story spec MUST carry an `**Evaluation contract:**` block with `expected_evidence` (each item: `kind`, `target`, `proves`) and `scorer_focus` (scorer rubric). Genuinely trivial/internal stories may use a one-item minimal block but must not omit the section entirely. This block is authored by the planner (`roll-design`) and consumed as a shared artifact contract by builder (`roll-build`/`roll-fix`) and evaluator (peer scorer, attest gate) — no three-agent chat pipeline is introduced.
|
|
777
|
+
>
|
|
778
|
+
> **Block template:**
|
|
779
|
+
> ```markdown
|
|
780
|
+
> **Evaluation contract:**
|
|
781
|
+
> - expected_evidence:
|
|
782
|
+
> - kind: test | command | screenshot | document | diff | ci | manual
|
|
783
|
+
> target: <file/command/surface/report that proves an AC>
|
|
784
|
+
> proves: <AC id or short AC phrase>
|
|
785
|
+
> - scorer_focus:
|
|
786
|
+
> - <what the peer Review Score should judge beyond generic code quality>
|
|
787
|
+
> - builder_notes:
|
|
788
|
+
> - <bounded implementation/evidence hints; no hidden requirements>
|
|
789
|
+
> ```
|
|
790
|
+
>
|
|
791
|
+
> **Rules**: (1) `expected_evidence` items are **binding for evidence expectations** — if an item becomes impossible, the builder updates the spec or explains the deviation in the report/ac-map. (2) `scorer_focus` items extend, not replace, the generic scoring rubric. (3) This block complements the visual-evidence contract; `deliverable_url`/`screenshot_exempt` rules are unchanged. (4) Legacy specs without the block degrade gracefully — builder and scorer fall back to the generic rubric with no behavior change.
|
|
792
|
+
|
|
760
793
|
### Closing Doc-Refresh Story Template — Phase N.M 收尾文档
|
|
761
794
|
|
|
762
795
|
When any preceding US in the batch changes user-visible behavior, append this template story at the end of the batch. Wire it as `depends-on:` against every preceding user-facing US so it runs last.
|
|
@@ -837,6 +870,29 @@ The design-side peer Review Score path is tracked by **FIX-344** (the runner's
|
|
|
837
870
|
score stage does not yet cover `roll-design`). Until FIX-344 lands, this skill
|
|
838
871
|
emits no quality score; the agent just delivers the split + specs and stops.
|
|
839
872
|
|
|
873
|
+
### The 3-stage shape (subjective deliverable)
|
|
874
|
+
|
|
875
|
+
A design has **no executable pass/fail test** — correctness is judgment. So it is
|
|
876
|
+
the canonical **3-stage loop**, and naming the stages keeps the roles honest:
|
|
877
|
+
|
|
878
|
+
- **Planner / Builder = this design session.** Besides the split + specs, it must
|
|
879
|
+
emit **explicit evaluation criteria for the design itself** — the prose
|
|
880
|
+
equivalent of a test contract: what would make this design wrong or incomplete
|
|
881
|
+
(e.g. "every Bounded Context boundary is justified", "the worked sample covers
|
|
882
|
+
the hardest edge case", "no story depends on a parked parent", "each
|
|
883
|
+
user-facing story carries a visual-evidence AC"). Write these down so the
|
|
884
|
+
Evaluator has something concrete to check against rather than a vibe.
|
|
885
|
+
- **Evaluator = an ISOLATED reviewer.** It receives the design artifact + the
|
|
886
|
+
criteria, and **must not be seeded with the designer's reasoning** (roll-peer's
|
|
887
|
+
Independent Judgment Rule already enforces exactly this — see the two
|
|
888
|
+
`[peer]` checkpoints in the Workflow: Direction Review and Plan Review). A
|
|
889
|
+
fresh same-vendor session is the minimum; a different agent+model is encouraged.
|
|
890
|
+
|
|
891
|
+
Until the runner's FIX-344 stage lands, the isolated Evaluator is the
|
|
892
|
+
`$roll-peer` Plan Review checkpoint, not an automated score — but the criteria
|
|
893
|
+
above are what it should grade against either way. Emitting the criteria now is
|
|
894
|
+
cheap and makes the future automated Evaluator a drop-in.
|
|
895
|
+
|
|
840
896
|
---
|
|
841
897
|
|
|
842
898
|
## Integration
|
package/skills/roll-fix/SKILL.md
CHANGED
|
@@ -47,3 +47,7 @@ Load when the user provides a FIX-XXX or BUG-XXX item, or asks for a focused hot
|
|
|
47
47
|
- Description changes require updates in `route-cases/skills.json`.
|
|
48
48
|
- New observed failures should add a gotcha and the matching positive or negative route case.
|
|
49
49
|
- Heavy examples, templates, recovery paths, and deterministic snippets belong in `references/`, `assets/`, or `scripts/`, not in this hub.
|
|
50
|
+
|
|
51
|
+
## Role in v4 execution profiles
|
|
52
|
+
|
|
53
|
+
**roll-fix is the Builder capability for hot-fix scope** (a single issue), normally under the `standard` execution profile — the same TCR/test/attest flow as roll-build, with no Planner or Evaluator. A fix that turns out user-visible or evidence-risky is selected into `verified`; one that turns out cross-module or ambiguous into `planned`. (Roles: Supervisor Agent / Planner / Builder / Evaluator; never Prime/Watchman/Dispatcher/Governor.)
|
|
@@ -20,9 +20,10 @@ Use when:
|
|
|
20
20
|
|
|
21
21
|
**Workflow:**
|
|
22
22
|
1. Read .roll/backlog.md index → Find FIX/BUG row → Follow link to `.roll/features/<epic>/<story>/spec.md`
|
|
23
|
-
2.
|
|
24
|
-
3.
|
|
25
|
-
4.
|
|
23
|
+
2. **Read the Evaluation contract (US-SKILL-030)** if the spec contains an `**Evaluation contract:**` block. Read `expected_evidence` and `scorer_focus` before writing code; use them to guide test design and evidence collection. Map each evidence item to a planned change and map delivered evidence back to the contract in the ac-map; note any deviation if an item turns out to be N/A for a fix.
|
|
24
|
+
3. Single Action (no splitting)
|
|
25
|
+
4. Execute via TCR workflow
|
|
26
|
+
5. Write back: update .roll/backlog.md status column + update FIX section in Feature file
|
|
26
27
|
|
|
27
28
|
Do not use for:
|
|
28
29
|
|
|
@@ -389,8 +390,8 @@ Runs ONLY on a ✅ Gate PASS (a FAIL retry must not mint a misleading report). N
|
|
|
389
390
|
tmux observation window `roll-loop-<slug>` is a natural target — display the
|
|
390
391
|
proof there), then `screencapture -x -R <window-rect>` (macOS) into
|
|
391
392
|
`screenshots/`. Capture ONLY the relevant work area — a focused window, not
|
|
392
|
-
the whole desktop. Unattended cycles: drive the capture from the
|
|
393
|
-
(deterministic), never hand-craft an image; if the capture channel is
|
|
393
|
+
the whole desktop. Unattended cycles: drive the capture from the runner's
|
|
394
|
+
capture lane (deterministic), never hand-craft an image; if the capture channel is
|
|
394
395
|
unavailable (no GUI session / no permission), fall back to text evidence and
|
|
395
396
|
mark the AC `partial` with a note — never fake a screenshot.
|
|
396
397
|
2. **Write the intent map** `.roll/features/<epic>/{ID}/ac-map.json` — for EVERY AC (ids `{ID}:AC1..n`) pick `pass|readonly|partial|claimed|missing` and reference only evidence that exists (paths relative to the run dir; story-level dirs are reachable as `../evidence/...` / `../screenshots/...`):
|
|
@@ -45,3 +45,7 @@ Load when the user explicitly asks for peer review, cross-agent negotiation, /pe
|
|
|
45
45
|
- Description changes require updates in `route-cases/skills.json`.
|
|
46
46
|
- New observed failures should add a gotcha and the matching positive or negative route case.
|
|
47
47
|
- Heavy examples, templates, recovery paths, and deterministic snippets belong in `references/`, `assets/`, or `scripts/`, not in this hub.
|
|
48
|
+
|
|
49
|
+
## Role in v4 execution profiles
|
|
50
|
+
|
|
51
|
+
Cross-agent peer review is an **Evaluator capability**: an independent fresh-session reviewer (never the Builder's session) scores the delivery. The Evaluator's judgment is binding for story satisfaction; peer negotiation is the mechanism. In `verified`/`planned` profiles this feeds the `eval-report.md`. (Roles: Supervisor Agent / Planner / Builder / Evaluator; never Prime/Watchman/Dispatcher/Governor.)
|
|
@@ -125,6 +125,12 @@ To reject: annotate with "Rejected: {reason}" to suppress future re-proposals.
|
|
|
125
125
|
- Never write directly to .roll/backlog.md — .roll/proposals.md is the staging area.
|
|
126
126
|
- If a similar proposal already exists in .roll/proposals.md (pending or rejected), note similarity and skip or merge rather than creating a duplicate.
|
|
127
127
|
- Aim for 2 proposals by default; generate 1 if context is thin, 3 if focus hint suggests a rich area.
|
|
128
|
+
- **3-stage discipline**: a proposal is a subjective deliverable, so the drafting
|
|
129
|
+
session must NOT also approve it. The owner moving an entry into .roll/backlog.md
|
|
130
|
+
IS the isolated Evaluator (the human is the fresh, unbiased session the 3-stage
|
|
131
|
+
loop requires). The proposal's **Acceptance Criteria (draft)** are the explicit
|
|
132
|
+
criteria that approval is graded against — keep them concrete and checkable so
|
|
133
|
+
the decision is against stated criteria, not a vibe; never self-promote a draft.
|
|
128
134
|
|
|
129
135
|
## .roll/proposals.md Format
|
|
130
136
|
|