@qball-inc/the-bulwark 1.0.1 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +2 -3
- package/.gitattributes +48 -0
- package/CHANGELOG.md +121 -0
- package/LICENSE +21 -0
- package/README.md +426 -368
- package/agents/bulwark-fix-validator.md +643 -633
- package/agents/bulwark-implementer.md +407 -391
- package/agents/bulwark-issue-analyzer.md +310 -308
- package/agents/bulwark-standards-reviewer.md +305 -221
- package/agents/plan-creation-architect.md +325 -323
- package/agents/plan-creation-eng-lead.md +354 -352
- package/agents/plan-creation-po.md +302 -300
- package/agents/plan-creation-qa-critic.md +336 -334
- package/agents/product-ideation-competitive-analyzer.md +2 -0
- package/agents/product-ideation-idea-validator.md +2 -0
- package/agents/product-ideation-market-researcher.md +2 -0
- package/agents/product-ideation-pattern-documenter.md +2 -0
- package/agents/product-ideation-segment-analyzer.md +2 -0
- package/agents/product-ideation-strategist.md +2 -0
- package/agents/statusline-setup.md +99 -97
- package/hooks/hooks.json +30 -1
- package/package.json +6 -5
- package/scripts/apply-section.sh +243 -0
- package/scripts/hooks/check-template-drift.sh +191 -0
- package/scripts/hooks/cleanup-review-registry.sh +106 -0
- package/scripts/hooks/cleanup-stale.sh +19 -2
- package/scripts/hooks/enforce-quality.sh +72 -23
- package/scripts/hooks/lib/coverage_check.py +513 -0
- package/scripts/hooks/suggest-pipeline-stop.sh +234 -0
- package/scripts/hooks/suggest-pipeline.sh +12 -0
- package/scripts/init.sh +64 -0
- package/scripts/install-bun.sh +327 -0
- package/scripts/install-just.sh +404 -0
- package/scripts/toolchain-smoke-run.sh +219 -0
- package/scripts/update.sh +342 -0
- package/skills/anthropic-validator/SKILL.md +497 -607
- package/skills/anthropic-validator/references/agents-checklist.md +144 -131
- package/skills/anthropic-validator/references/agents-validation.md +90 -0
- package/skills/anthropic-validator/references/commands-checklist.md +102 -102
- package/skills/anthropic-validator/references/commands-validation.md +42 -0
- package/skills/anthropic-validator/references/hooks-checklist.md +160 -151
- package/skills/anthropic-validator/references/hooks-validation.md +82 -0
- package/skills/anthropic-validator/references/mcp-checklist.md +136 -136
- package/skills/anthropic-validator/references/mcp-validation.md +39 -0
- package/skills/anthropic-validator/references/plugins-checklist.md +154 -148
- package/skills/anthropic-validator/references/plugins-validation.md +68 -0
- package/skills/anthropic-validator/references/skills-checklist.md +105 -85
- package/skills/anthropic-validator/references/skills-validation.md +79 -0
- package/skills/assertion-patterns/SKILL.md +298 -296
- package/skills/bug-magnet-data/SKILL.md +286 -284
- package/skills/bug-magnet-data/context/cli-args.md +91 -91
- package/skills/bug-magnet-data/context/db-query.md +104 -104
- package/skills/bug-magnet-data/context/file-contents.md +103 -103
- package/skills/bug-magnet-data/context/http-body.md +91 -91
- package/skills/bug-magnet-data/context/process-spawn.md +123 -123
- package/skills/bug-magnet-data/data/booleans/boundaries.yaml +143 -143
- package/skills/bug-magnet-data/data/collections/arrays.yaml +114 -114
- package/skills/bug-magnet-data/data/collections/objects.yaml +123 -123
- package/skills/bug-magnet-data/data/concurrency/race-conditions.yaml +118 -118
- package/skills/bug-magnet-data/data/concurrency/state-machines.yaml +115 -115
- package/skills/bug-magnet-data/data/dates/boundaries.yaml +137 -137
- package/skills/bug-magnet-data/data/dates/invalid.yaml +132 -132
- package/skills/bug-magnet-data/data/dates/timezone.yaml +118 -118
- package/skills/bug-magnet-data/data/encoding/charset.yaml +79 -79
- package/skills/bug-magnet-data/data/encoding/normalization.yaml +105 -105
- package/skills/bug-magnet-data/data/formats/email.yaml +154 -154
- package/skills/bug-magnet-data/data/formats/json.yaml +187 -187
- package/skills/bug-magnet-data/data/formats/url.yaml +165 -165
- package/skills/bug-magnet-data/data/language-specific/javascript.yaml +182 -182
- package/skills/bug-magnet-data/data/language-specific/python.yaml +174 -174
- package/skills/bug-magnet-data/data/language-specific/rust.yaml +148 -148
- package/skills/bug-magnet-data/data/numbers/boundaries.yaml +161 -161
- package/skills/bug-magnet-data/data/numbers/precision.yaml +89 -89
- package/skills/bug-magnet-data/data/numbers/special.yaml +69 -69
- package/skills/bug-magnet-data/data/strings/boundaries.yaml +109 -109
- package/skills/bug-magnet-data/data/strings/injection.yaml +208 -208
- package/skills/bug-magnet-data/data/strings/special-chars.yaml +190 -190
- package/skills/bug-magnet-data/data/strings/unicode.yaml +139 -139
- package/skills/bug-magnet-data/references/external-lists.md +115 -115
- package/skills/bulwark-brainstorm/SKILL.md +566 -563
- package/skills/bulwark-brainstorm/references/at-teammate-prompts.md +95 -60
- package/skills/bulwark-brainstorm/references/role-critical-analyst.md +78 -78
- package/skills/bulwark-brainstorm/references/role-development-lead.md +66 -66
- package/skills/bulwark-brainstorm/references/role-product-delivery-lead.md +79 -79
- package/skills/bulwark-brainstorm/references/role-product-manager.md +62 -62
- package/skills/bulwark-brainstorm/references/role-project-sme.md +59 -59
- package/skills/bulwark-brainstorm/references/role-technical-architect.md +66 -66
- package/skills/bulwark-research/SKILL.md +300 -298
- package/skills/bulwark-research/references/viewpoint-contrarian.md +63 -63
- package/skills/bulwark-research/references/viewpoint-direct-investigation.md +62 -62
- package/skills/bulwark-research/references/viewpoint-first-principles.md +65 -65
- package/skills/bulwark-research/references/viewpoint-practitioner.md +62 -62
- package/skills/bulwark-research/references/viewpoint-prior-art.md +66 -66
- package/skills/bulwark-scaffold/SKILL.md +483 -330
- package/skills/bulwark-statusline/SKILL.md +166 -161
- package/skills/bulwark-statusline/scripts/statusline.sh +1 -1
- package/skills/bulwark-verify/SKILL.md +532 -519
- package/skills/code-review/SKILL.md +488 -428
- package/skills/code-review/examples/anti-patterns/linting.ts +181 -181
- package/skills/code-review/examples/anti-patterns/security.ts +91 -91
- package/skills/code-review/examples/anti-patterns/standards.ts +195 -195
- package/skills/code-review/examples/anti-patterns/type-safety.ts +108 -108
- package/skills/code-review/examples/recommended/linting.ts +195 -195
- package/skills/code-review/examples/recommended/security.ts +154 -154
- package/skills/code-review/examples/recommended/standards.ts +231 -231
- package/skills/code-review/examples/recommended/type-safety.ts +181 -181
- package/skills/code-review/frameworks/angular.md +218 -218
- package/skills/code-review/frameworks/django.md +235 -235
- package/skills/code-review/frameworks/express.md +207 -207
- package/skills/code-review/frameworks/fastapi.md +326 -0
- package/skills/code-review/frameworks/flask.md +298 -298
- package/skills/code-review/frameworks/generic.md +146 -146
- package/skills/code-review/frameworks/react.md +152 -152
- package/skills/code-review/frameworks/vue.md +244 -244
- package/skills/code-review/references/linting-patterns.md +221 -221
- package/skills/code-review/references/security-patterns.md +125 -125
- package/skills/code-review/references/standards-patterns.md +246 -246
- package/skills/code-review/references/type-safety-patterns.md +130 -130
- package/skills/component-patterns/SKILL.md +133 -131
- package/skills/component-patterns/references/pattern-cli-command.md +118 -118
- package/skills/component-patterns/references/pattern-database.md +166 -166
- package/skills/component-patterns/references/pattern-external-api.md +139 -139
- package/skills/component-patterns/references/pattern-file-parser.md +168 -168
- package/skills/component-patterns/references/pattern-http-server.md +162 -162
- package/skills/component-patterns/references/pattern-process-spawner.md +133 -133
- package/skills/continuous-feedback/SKILL.md +329 -327
- package/skills/continuous-feedback/references/collect-instructions.md +81 -81
- package/skills/continuous-feedback/references/specialize-code-review.md +82 -82
- package/skills/continuous-feedback/references/specialize-general.md +98 -98
- package/skills/continuous-feedback/references/specialize-test-audit.md +81 -81
- package/skills/create-skill/SKILL.md +550 -359
- package/skills/create-skill/agents/skill-eval-comparator.md +158 -0
- package/skills/create-skill/agents/skill-eval-grader.md +168 -0
- package/skills/create-skill/references/agent-conventions.md +194 -194
- package/skills/create-skill/references/agent-template.md +195 -195
- package/skills/create-skill/references/content-guidance.md +541 -291
- package/skills/create-skill/references/decision-framework.md +232 -124
- package/skills/create-skill/references/eval-scaffolding.md +468 -0
- package/skills/create-skill/references/eval-shape.md +383 -0
- package/skills/create-skill/references/scripts-conventions.md +142 -0
- package/skills/create-skill/references/template-generator.md +183 -0
- package/skills/create-skill/references/template-inversion.md +269 -0
- package/skills/create-skill/references/template-pipeline.md +248 -217
- package/skills/create-skill/references/template-research.md +234 -210
- package/skills/create-skill/references/template-reviewer.md +231 -0
- package/skills/create-skill/references/template-script-driven.md +185 -172
- package/skills/create-skill/references/template-tool-wrapper.md +199 -0
- package/skills/create-skill/scripts/check-description.ts +238 -0
- package/skills/create-skill/scripts/check-skill-size.ts +201 -0
- package/skills/create-skill/scripts/grade.ts +855 -0
- package/skills/create-skill/scripts/run-loop.ts +297 -0
- package/skills/create-subagent/SKILL.md +355 -353
- package/skills/create-subagent/references/agent-conventions.md +268 -268
- package/skills/create-subagent/references/content-guidance.md +232 -232
- package/skills/create-subagent/references/decision-framework.md +134 -134
- package/skills/create-subagent/references/template-single-agent.md +194 -192
- package/skills/fix-bug/SKILL.md +243 -241
- package/skills/governance-protocol/SKILL.md +118 -116
- package/skills/init/SKILL.md +519 -341
- package/skills/init/references/update-askuser-prompts.md +198 -0
- package/skills/init/references/update-mode.md +305 -0
- package/skills/init/references/update-section-anchor-diff.md +163 -0
- package/skills/issue-debugging/SKILL.md +387 -385
- package/skills/issue-debugging/references/anti-patterns.md +245 -245
- package/skills/issue-debugging/references/debug-report-schema.md +227 -227
- package/skills/mock-detection/SKILL.md +528 -511
- package/skills/mock-detection/references/false-positive-prevention.md +402 -402
- package/skills/mock-detection/references/stub-patterns.md +236 -236
- package/skills/pipeline-templates/SKILL.md +262 -215
- package/skills/pipeline-templates/references/code-change-workflow.md +277 -277
- package/skills/pipeline-templates/references/code-review.md +348 -336
- package/skills/pipeline-templates/references/fix-validation.md +421 -421
- package/skills/pipeline-templates/references/new-feature.md +335 -335
- package/skills/pipeline-templates/references/research-brainstorm.md +161 -161
- package/skills/pipeline-templates/references/research-planning.md +257 -257
- package/skills/pipeline-templates/references/test-audit.md +389 -389
- package/skills/pipeline-templates/references/test-execution-fix.md +238 -238
- package/skills/plan-creation/SKILL.md +531 -497
- package/skills/plan-to-tasks/SKILL.md +151 -0
- package/skills/plan-to-tasks/references/askuserquestion-prompts.md +75 -0
- package/skills/plan-to-tasks/references/transform.md +253 -0
- package/skills/product-ideation/SKILL.md +2 -0
- package/skills/session-handoff/SKILL.md +167 -139
- package/skills/session-handoff/references/examples.md +223 -223
- package/skills/setup-lsp/SKILL.md +314 -312
- package/skills/setup-lsp/references/server-registry.md +85 -85
- package/skills/setup-lsp/references/troubleshooting.md +135 -135
- package/skills/spec-drift-check/SKILL.md +287 -0
- package/skills/spec-drift-check/evals/evals.json +33 -0
- package/skills/spec-drift-check/evals/triggers.json +19 -0
- package/skills/spec-drift-check/examples/clean-spec.md +52 -0
- package/skills/spec-drift-check/examples/expected-output-clean.yaml +96 -0
- package/skills/spec-drift-check/examples/expected-output-high-drift.yaml +78 -0
- package/skills/spec-drift-check/examples/expected-output-low-drift.yaml +67 -0
- package/skills/spec-drift-check/examples/high-drift-spec.md +49 -0
- package/skills/spec-drift-check/examples/low-drift-spec.md +39 -0
- package/skills/spec-drift-check/references/anti-patterns.md +65 -0
- package/skills/spec-drift-check/references/output-template.md +142 -0
- package/skills/spec-drift-check/references/step-1-claim-extraction.md +147 -0
- package/skills/spec-drift-check/references/step-2-verification-methods.md +203 -0
- package/skills/spec-drift-check/references/step-3-categorization.md +105 -0
- package/skills/spec-drift-check/references/step-4-plan-adjustment.md +122 -0
- package/skills/spec-drift-check/references/step-5-log-template.md +220 -0
- package/skills/spec-drift-check/references/step-6-decision-matrix.md +136 -0
- package/skills/subagent-output-templating/SKILL.md +417 -415
- package/skills/subagent-output-templating/references/examples.md +440 -440
- package/skills/subagent-prompting/SKILL.md +366 -364
- package/skills/subagent-prompting/references/examples.md +342 -342
- package/skills/test-audit/SKILL.md +545 -531
- package/skills/test-audit/references/known-limitations.md +41 -41
- package/skills/test-audit/references/priority-classification.md +30 -30
- package/skills/test-audit/references/prompts/deep-mode-detection.md +83 -83
- package/skills/test-audit/references/prompts/synthesis.md +58 -57
- package/skills/test-audit/references/rewrite-instructions.md +46 -46
- package/skills/test-audit/references/schemas/audit-output.yaml +131 -100
- package/skills/test-audit/references/schemas/diagnostic-output.yaml +56 -49
- package/skills/test-audit/references/two-gate-logic.md +43 -0
- package/skills/test-audit/scripts/data-flow-analyzer.ts +508 -509
- package/skills/test-audit/scripts/integration-mock-detector.ts +462 -462
- package/skills/test-audit/scripts/skip-detector.ts +211 -211
- package/skills/test-audit/scripts/verification-counter.ts +295 -295
- package/skills/test-classification/SKILL.md +326 -310
- package/skills/test-fixture-creation/SKILL.md +297 -295
- package/Infographics/01_product-ideation.png +0 -0
- package/Infographics/02_feature-research.png +0 -0
- package/Infographics/03_brainstorm.png +0 -0
- package/Infographics/04_plan-creation.png +0 -0
- package/Infographics/05_code-review.png +0 -0
- package/Infographics/06_test-audit.png +0 -0
- package/Infographics/07_fix-bug.png +0 -0
- package/skills/create-skill/references/template-reference-heavy.md +0 -111
- package/skills/create-skill/references/template-simple.md +0 -80
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# Step 3 — Categorization
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
For every Stage 2 verification result, assign:
|
|
6
|
+
|
|
7
|
+
1. A **category** — what kind of drift (or non-drift) is this?
|
|
8
|
+
2. A **severity** — how much does it block?
|
|
9
|
+
3. A **decision implication** — what does the orchestrator do about it?
|
|
10
|
+
|
|
11
|
+
Categorization is the bridge from raw verification evidence to the verdict. The category dictates how Stage 4 rewrites the plan; the severity dictates whether Stage 6 emits PROCEED, PROCEED_ADJUSTED, or STOP_USER_APPROVAL.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Category Table
|
|
16
|
+
|
|
17
|
+
| Category | Definition | Severity | Example | Decision Implication |
|
|
18
|
+
|----------|------------|----------|---------|----------------------|
|
|
19
|
+
| **CONFIRMED** | Claim matches reality (Stage 2 returned exact verbatim match) | n/a | Spec says `coverage_check.py:88`; Read at line 88 shows the asserted content. | No action; carry the original plan forward |
|
|
20
|
+
| **DRIFT — line-ref** | File correct; content has moved to a different line OR a literal value off by a small amount | LOW | Spec says line 88; actual content is at line 91 (file is otherwise unchanged). | Doc fix in implementation comments; PROCEED_ADJUSTED |
|
|
21
|
+
| **DRIFT — wrong file** | File path stale; content lives at a different path | HIGH | Spec says `scripts/hooks/foo.sh`; actual file is `scripts/foo.sh`. | STOP_USER_APPROVAL — wrong-file drifts often signal larger refactors the spec doesn't reflect |
|
|
22
|
+
| **DRIFT — missing scope** | Spec lists deliverable X but X is a no-op (already done, or no longer needed) | MEDIUM | Spec says "add X function"; X already exists at the asserted path with the asserted shape. | Drop the deliverable; PROCEED_ADJUSTED |
|
|
23
|
+
| **DRIFT — undeclared scope** | Spec missing a deliverable that real implementation requires | HIGH | Spec says "modify A"; modifying A safely also requires modifying B (uncovered by spec). | STOP_USER_APPROVAL — scope expansion needs sign-off |
|
|
24
|
+
| **AC re-interpretation** | Acceptance criterion is ambiguous; can resolve against current code with documented choice | MEDIUM | Spec says "update the schema"; current code has two schemas; AC is unclear which one. | Resolve explicitly with rationale; PROCEED_ADJUSTED |
|
|
25
|
+
| **GAP** | Claim references a thing that does not exist anywhere | HIGH | Spec says `parseFooBar()`; no such function in the repo. | STOP_USER_APPROVAL — the spec is referencing fiction |
|
|
26
|
+
|
|
27
|
+
---
|
|
28
|
+
|
|
29
|
+
## Severity Decision Tree
|
|
30
|
+
|
|
31
|
+
Use this tree when a finding is borderline. The tree maps **finding shape** → **severity tier** deterministically.
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
Is the claim CONFIRMED (verbatim match)?
|
|
35
|
+
├── YES → category=CONFIRMED, severity=n/a, no decision impact
|
|
36
|
+
└── NO → drift exists; continue
|
|
37
|
+
│
|
|
38
|
+
Does the claim reference a file path?
|
|
39
|
+
├── YES → does the file exist at the asserted path?
|
|
40
|
+
│ ├── YES (file ok, content drift only)
|
|
41
|
+
│ │ └── Is the content at a different line in the SAME file?
|
|
42
|
+
│ │ ├── YES → DRIFT-line-ref, severity LOW
|
|
43
|
+
│ │ └── NO → content gone or wrong → GAP or DRIFT-wrong-file (case below)
|
|
44
|
+
│ └── NO (file path wrong)
|
|
45
|
+
│ └── Does similar content exist at a DIFFERENT path?
|
|
46
|
+
│ ├── YES → DRIFT-wrong-file, severity HIGH
|
|
47
|
+
│ └── NO → GAP, severity HIGH
|
|
48
|
+
│
|
|
49
|
+
Does the claim reference a deliverable / scope item?
|
|
50
|
+
├── Is the deliverable a no-op (already done)?
|
|
51
|
+
│ └── YES → DRIFT-missing-scope, severity MEDIUM
|
|
52
|
+
├── Does completing the spec REQUIRE additional work the spec didn't list?
|
|
53
|
+
│ └── YES → DRIFT-undeclared-scope, severity HIGH
|
|
54
|
+
│
|
|
55
|
+
Is the claim ambiguous (multiple plausible readings against current code)?
|
|
56
|
+
├── YES → AC re-interpretation, severity MEDIUM (must document chosen reading)
|
|
57
|
+
│
|
|
58
|
+
Does the claim reference a function/symbol/value that does not exist anywhere?
|
|
59
|
+
└── YES → GAP, severity HIGH
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
**Tie-breaker rules**:
|
|
63
|
+
- When in doubt between LOW and MEDIUM, choose **MEDIUM** (you can downgrade in Stage 4 if it's truly trivial; you can't upgrade after the verdict ships).
|
|
64
|
+
- When in doubt between MEDIUM and HIGH, choose **HIGH** (the user should see the finding; over-stopping is recoverable, under-stopping ships bad work).
|
|
65
|
+
- CRITICAL is reserved (the rubric in SKILL.md notes it escalates to HIGH for verdict purposes). Do not use CRITICAL unless an explicit safety/security concern is in play; otherwise, HIGH is the top tier.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## Severity Tier Reference
|
|
70
|
+
|
|
71
|
+
| Severity | Definition | Verdict Implication |
|
|
72
|
+
|----------|------------|---------------------|
|
|
73
|
+
| **CRITICAL** | Reserved — used only for explicit safety/security blockers; escalates to HIGH for verdict purposes | STOP_USER_APPROVAL |
|
|
74
|
+
| **HIGH** | Wrong-file DRIFT, undeclared-scope DRIFT, GAP — scope-changing or block-level | STOP_USER_APPROVAL |
|
|
75
|
+
| **MEDIUM** | Missing-scope DRIFT, AC re-interpretation — adjustable without scope expansion | PROCEED_ADJUSTED |
|
|
76
|
+
| **LOW** | Line-ref DRIFT — file correct, line off | PROCEED_ADJUSTED (doc fix in implementation comments) |
|
|
77
|
+
|
|
78
|
+
This tier table is mirrored in SKILL.md. If you change one, change both (cross-file consistency).
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## Output Format Per Finding
|
|
83
|
+
|
|
84
|
+
Each finding from Stage 3 carries:
|
|
85
|
+
|
|
86
|
+
```yaml
|
|
87
|
+
- id: D-NN # sequential within this run
|
|
88
|
+
claim_id: C-NN # back-reference to the Stage 1 claim
|
|
89
|
+
category: confirmed | drift-line-ref | drift-wrong-file | drift-missing-scope | drift-undeclared-scope | ac-reinterpretation | gap
|
|
90
|
+
severity: LOW | MEDIUM | HIGH # n/a for CONFIRMED
|
|
91
|
+
spec_claim: "<verbatim quote from spec>"
|
|
92
|
+
actual_state: "<verbatim evidence from Stage 2>"
|
|
93
|
+
resolution: "<doc fix | plan adjustment | scope expansion | ask user | none>"
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
Multiple findings can map to a single claim if the claim has compound assertions (e.g., "the function `foo` at `bar.ts:42`" yields one finding for the function name and one for the line ref).
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## Anti-Pattern Reminders for This Stage
|
|
101
|
+
|
|
102
|
+
- Do NOT mark a finding CONFIRMED based on recall. Stage 2 evidence must be verbatim.
|
|
103
|
+
- Do NOT skip the severity field. Severity drives the verdict; an unsevered finding is invisible to Stage 6.
|
|
104
|
+
- Do NOT invent new categories. The 7 categories above cover all observed drift shapes; extending the taxonomy mid-run breaks downstream consumers (Stage 6 decision matrix and the YAML schema).
|
|
105
|
+
- Do NOT auto-downgrade HIGH → MEDIUM to "save the user a question". HIGH is HIGH because the orchestrator cannot proceed without a sign-off.
|
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# Step 4 — Plan Adjustment
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Rewrite the spec's implementation-plan section based on the Stage 3 findings. The output of this stage becomes the **adjusted plan** — the source of truth for the rest of the WP. Stage 7 binds it: the implementer follows this plan, not the original spec.
|
|
6
|
+
|
|
7
|
+
If Stage 3 produced zero non-CONFIRMED findings, Stage 4 is a no-op (the original plan stands). Otherwise, every finding must be reflected in the rewrite.
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
## Procedure
|
|
12
|
+
|
|
13
|
+
For each finding category, apply the matching adjustment:
|
|
14
|
+
|
|
15
|
+
### DRIFT-missing-scope (deliverable is a no-op)
|
|
16
|
+
|
|
17
|
+
**Action**: Drop the deliverable from the plan. Note the drop in the `verification_checklist:` so the implementer doesn't accidentally re-add it.
|
|
18
|
+
|
|
19
|
+
**Token-budget impact**: subtract the original estimate for the dropped deliverable.
|
|
20
|
+
|
|
21
|
+
### DRIFT-undeclared-scope (deliverable missing from spec)
|
|
22
|
+
|
|
23
|
+
**Action**: Add the new deliverable to the plan. If severity is HIGH (which it should be by default), the verdict is STOP_USER_APPROVAL and the user must sign off before this addition is binding.
|
|
24
|
+
|
|
25
|
+
**Token-budget impact**: add an estimate for the new deliverable.
|
|
26
|
+
|
|
27
|
+
### DRIFT-wrong-file (path target is wrong)
|
|
28
|
+
|
|
29
|
+
**Action**: Re-target the deliverable's path in the plan to the correct path. Note the re-target in the `verification_checklist:`.
|
|
30
|
+
|
|
31
|
+
**Token-budget impact**: usually zero (same scope, different target).
|
|
32
|
+
|
|
33
|
+
### DRIFT-line-ref (line off, file correct)
|
|
34
|
+
|
|
35
|
+
**Action**: Update the line reference (or the contextual cue, if the spec used "around line 200" framing) to match current state. This is typically a doc fix; the deliverable itself is unchanged.
|
|
36
|
+
|
|
37
|
+
**Token-budget impact**: zero.
|
|
38
|
+
|
|
39
|
+
### AC re-interpretation (ambiguous AC)
|
|
40
|
+
|
|
41
|
+
**Action**: Resolve the ambiguity explicitly. Document the chosen reading + the rationale in the `ac_reinterpretations:` section of the log. Update any plan deliverables that depend on the resolved AC.
|
|
42
|
+
|
|
43
|
+
**Token-budget impact**: usually zero, occasionally negative (a tighter interpretation drops scope).
|
|
44
|
+
|
|
45
|
+
### GAP (claim references nothing)
|
|
46
|
+
|
|
47
|
+
**Action**: STOP. GAP findings escalate to STOP_USER_APPROVAL; the user must clarify the spec. Do NOT silently drop the deliverable that referenced the GAP — surface it.
|
|
48
|
+
|
|
49
|
+
**Token-budget impact**: deferred until the user clarifies.
|
|
50
|
+
|
|
51
|
+
### CONFIRMED
|
|
52
|
+
|
|
53
|
+
**Action**: None. Carry the original deliverable forward unchanged.
|
|
54
|
+
|
|
55
|
+
---
|
|
56
|
+
|
|
57
|
+
## Re-Estimate Tokens
|
|
58
|
+
|
|
59
|
+
After applying the adjustments above, re-estimate the token budget for the WP. The adjusted budget appears in the log's `adjusted_plan.estimated_token_delta:` field as a signed delta from the original estimate (`+5K`, `-10K`, `0`).
|
|
60
|
+
|
|
61
|
+
A non-trivial delta (≥10K either direction) is itself a signal worth surfacing in the verdict summary, even if the finding mix doesn't otherwise force STOP_USER_APPROVAL.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Before / After Example
|
|
66
|
+
|
|
67
|
+
### Original spec plan section
|
|
68
|
+
|
|
69
|
+
```markdown
|
|
70
|
+
## Implementation Plan
|
|
71
|
+
|
|
72
|
+
### Step 1 — Add `parseFollowupEdits()` helper to `coverage_check.py`
|
|
73
|
+
- 80 lines, ~5K tokens
|
|
74
|
+
- File: `scripts/hooks/coverage_check.py:88`
|
|
75
|
+
|
|
76
|
+
### Step 2 — Update `code-review` skill schema
|
|
77
|
+
- Add `followup_edits_expected` field to diagnostic YAML template
|
|
78
|
+
- File: `skills/code-review/SKILL.md:391`
|
|
79
|
+
|
|
80
|
+
### Step 3 — Test coverage
|
|
81
|
+
- Add 5 test cases to `tests/hooks/test-suggest-pipeline-stop.sh`
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### Stage 3 findings (hypothetical)
|
|
85
|
+
|
|
86
|
+
- D-01: `coverage_check.py:88` actually shows the helper at line 91 → DRIFT-line-ref, LOW
|
|
87
|
+
- D-02: `code-review` SKILL.md does not have a line 391; the schema section is at line 416 → DRIFT-line-ref, LOW
|
|
88
|
+
- D-03: `parseFollowupEdits()` already exists at `coverage_check.py:91` from a prior session → DRIFT-missing-scope, MEDIUM (Step 1 is a no-op)
|
|
89
|
+
- D-04: tests file `tests/hooks/test-suggest-pipeline-stop.sh` exists, but adding tests requires the file to be writable and a `set -e` audit per `process_test_harness_set_e_pattern.md` — undeclared in spec → DRIFT-undeclared-scope, HIGH
|
|
90
|
+
|
|
91
|
+
### Adjusted plan after Stage 4
|
|
92
|
+
|
|
93
|
+
```markdown
|
|
94
|
+
## Adjusted Implementation Plan
|
|
95
|
+
(supersedes original spec; binding per Stage 7)
|
|
96
|
+
|
|
97
|
+
### Step 1 — DROPPED — `parseFollowupEdits()` already exists
|
|
98
|
+
- Dropped per finding D-03 (DRIFT-missing-scope, MEDIUM)
|
|
99
|
+
- Token savings: -5K
|
|
100
|
+
|
|
101
|
+
### Step 2 — Update `code-review` skill schema
|
|
102
|
+
- Add `followup_edits_expected` field to diagnostic YAML template
|
|
103
|
+
- File: `skills/code-review/SKILL.md` (line ref updated; current location ~line 416 per finding D-02)
|
|
104
|
+
|
|
105
|
+
### Step 3 — Test coverage
|
|
106
|
+
- Add 5 test cases to `tests/hooks/test-suggest-pipeline-stop.sh`
|
|
107
|
+
- ALSO: confirm `set -e` is present in the test harness per `process_test_harness_set_e_pattern.md` (undeclared in original spec; finding D-04 surfaces this)
|
|
108
|
+
- Token addition: +1K
|
|
109
|
+
|
|
110
|
+
estimated_token_delta: -4K
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
If finding D-04 is severity HIGH (undeclared scope), the verdict is STOP_USER_APPROVAL and the user must approve adding the `set -e` audit task before Stage 7 binding takes effect.
|
|
114
|
+
|
|
115
|
+
---
|
|
116
|
+
|
|
117
|
+
## What NOT To Do at This Stage
|
|
118
|
+
|
|
119
|
+
- Do NOT auto-apply LOW findings to the original spec by editing the brief. The skill is read-only. Capture the corrections in the log, not in the source.
|
|
120
|
+
- Do NOT silently merge HIGH findings into the adjusted plan — they require Stage 6 to emit STOP_USER_APPROVAL and the user to confirm.
|
|
121
|
+
- Do NOT carry forward a deliverable that DRIFT-missing-scope flagged as a no-op. Doing so wastes the next session's tokens.
|
|
122
|
+
- Do NOT estimate token deltas with false precision. Round to the nearest 1K; the goal is signal, not accounting.
|
|
@@ -0,0 +1,220 @@
|
|
|
1
|
+
# Step 5 — Verification Log Template
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Capture the audit's findings, AC re-interpretations, adjusted plan, verification checklist, decision, and ROI estimate in a single canonical log file. The log is the deliverable; downstream sessions and implementers consume it directly.
|
|
6
|
+
|
|
7
|
+
**Log path**: `$PROJECT_DIR/logs/spec-verify-{session}-{topic}.md`
|
|
8
|
+
|
|
9
|
+
- `{session}` is the current Bulwark session id (e.g., `122`).
|
|
10
|
+
- `{topic}` is a short slug derived from the subject spec filename (e.g., `P10.16` for `P10.16-statusline-lock-cleanup-observability.md`).
|
|
11
|
+
|
|
12
|
+
Example: `logs/spec-verify-122-P10.16.md`.
|
|
13
|
+
|
|
14
|
+
---
|
|
15
|
+
|
|
16
|
+
## Full Template
|
|
17
|
+
|
|
18
|
+
```yaml
|
|
19
|
+
# logs/spec-verify-{session}-{topic}.md
|
|
20
|
+
# Verification log for spec-drift-check run.
|
|
21
|
+
# Subject: {path-to-spec}
|
|
22
|
+
# Run timestamp: {ISO-8601}
|
|
23
|
+
# Skill version: 1.0.0
|
|
24
|
+
|
|
25
|
+
metadata:
|
|
26
|
+
reviewer: spec-drift-check
|
|
27
|
+
subject: {path-to-spec}
|
|
28
|
+
session: {session-id}
|
|
29
|
+
timestamp: {ISO-8601}
|
|
30
|
+
spec_word_count: {N}
|
|
31
|
+
claims_extracted: {N}
|
|
32
|
+
claims_verified: {N}
|
|
33
|
+
|
|
34
|
+
findings:
|
|
35
|
+
- id: D-01
|
|
36
|
+
claim_id: C-01
|
|
37
|
+
category: drift-line-ref | drift-wrong-file | drift-missing-scope | drift-undeclared-scope | ac-reinterpretation | gap | confirmed
|
|
38
|
+
severity: LOW | MEDIUM | HIGH # n/a for confirmed
|
|
39
|
+
spec_claim: "<verbatim quote from spec>"
|
|
40
|
+
actual_state: "<what current code/state shows, verbatim>"
|
|
41
|
+
resolution: "<doc fix | plan adjustment | scope expansion | ask user | none>"
|
|
42
|
+
# ... D-02, D-03, ...
|
|
43
|
+
|
|
44
|
+
ac_reinterpretations:
|
|
45
|
+
- ac: AC-N
|
|
46
|
+
ambiguity: "<what's unclear in the original AC>"
|
|
47
|
+
chosen_interpretation: "<the reading we will execute against>"
|
|
48
|
+
rationale: "<why this reading; what it implies for the plan>"
|
|
49
|
+
# ... AC-M, ...
|
|
50
|
+
|
|
51
|
+
adjusted_plan:
|
|
52
|
+
binding_status: "supersedes original spec for rest of WP"
|
|
53
|
+
deliverables:
|
|
54
|
+
- id: AD-01
|
|
55
|
+
description: "<deliverable text>"
|
|
56
|
+
original_spec_step: <number or null if newly added>
|
|
57
|
+
change_from_spec: dropped | re-targeted | unchanged | added
|
|
58
|
+
target_path: <path>
|
|
59
|
+
token_estimate: "~5K"
|
|
60
|
+
# ... AD-02, ...
|
|
61
|
+
estimated_token_delta: "+5K | -10K | 0"
|
|
62
|
+
|
|
63
|
+
verification_checklist:
|
|
64
|
+
- "<item the implementer must confirm at end of WP — typically per finding>"
|
|
65
|
+
# ...
|
|
66
|
+
|
|
67
|
+
proceed_decision: PROCEED | PROCEED_ADJUSTED | STOP_USER_APPROVAL
|
|
68
|
+
|
|
69
|
+
decision_rationale: |
|
|
70
|
+
{1-3 sentences explaining the verdict in terms of the finding mix.}
|
|
71
|
+
|
|
72
|
+
roi:
|
|
73
|
+
spent_tokens_estimate: "~{N}K"
|
|
74
|
+
estimated_savings: "~{N}K"
|
|
75
|
+
net: positive | break-even | negative
|
|
76
|
+
rationale: |
|
|
77
|
+
{1-3 sentences explaining the savings estimate — what rework was avoided.}
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## How to Fill Each Section
|
|
83
|
+
|
|
84
|
+
### `metadata`
|
|
85
|
+
|
|
86
|
+
Captured as the run begins. `claims_extracted` is the count from Stage 1; `claims_verified` should equal `claims_extracted` unless a verification command failed.
|
|
87
|
+
|
|
88
|
+
**Filled-in example**:
|
|
89
|
+
```yaml
|
|
90
|
+
metadata:
|
|
91
|
+
reviewer: spec-drift-check
|
|
92
|
+
subject: plans/task-briefs/P10.16-statusline-lock-cleanup-observability.md
|
|
93
|
+
session: 122
|
|
94
|
+
timestamp: 2026-05-09T14:32:00Z
|
|
95
|
+
spec_word_count: 4521
|
|
96
|
+
claims_extracted: 23
|
|
97
|
+
claims_verified: 23
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### `findings`
|
|
101
|
+
|
|
102
|
+
One entry per claim from Stage 1. CONFIRMED claims SHOULD be included (with `severity: n/a`) so the log is a complete record of what was checked. Drift findings carry the actual_state verbatim.
|
|
103
|
+
|
|
104
|
+
**Filled-in example**:
|
|
105
|
+
```yaml
|
|
106
|
+
findings:
|
|
107
|
+
- id: D-01
|
|
108
|
+
claim_id: C-03
|
|
109
|
+
category: drift-line-ref
|
|
110
|
+
severity: LOW
|
|
111
|
+
spec_claim: "The recursion bug is at coverage_check.py:88."
|
|
112
|
+
actual_state: "Line 88 is blank (the helper function moved to line 91 in S121 cleanup; verbatim line 91: `def parse_followup_edits_expected(diagnostic_path):`)."
|
|
113
|
+
resolution: "doc fix — update spec line ref to 91 in implementation comments"
|
|
114
|
+
|
|
115
|
+
- id: D-02
|
|
116
|
+
claim_id: C-07
|
|
117
|
+
category: drift-undeclared-scope
|
|
118
|
+
severity: HIGH
|
|
119
|
+
spec_claim: "Add 5 test cases to test-suggest-pipeline-stop.sh."
|
|
120
|
+
actual_state: "Test harness `tests/hooks/test-suggest-pipeline-stop.sh:1` sets `set -euo pipefail` per process_test_harness_set_e_pattern.md. Adding test cases requires assertion-counter pattern (failures via counter, not exit), which the spec does not mention."
|
|
121
|
+
resolution: "scope expansion — add `set -e` audit task to plan; ask user"
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### `ac_reinterpretations`
|
|
125
|
+
|
|
126
|
+
One entry per acceptance criterion that Stage 3 flagged as ambiguous. The `chosen_interpretation` becomes binding for the WP via Stage 7.
|
|
127
|
+
|
|
128
|
+
**Filled-in example**:
|
|
129
|
+
```yaml
|
|
130
|
+
ac_reinterpretations:
|
|
131
|
+
- ac: AC-3
|
|
132
|
+
ambiguity: "Spec says 'update the schema' but two schemas exist (diagnostic schema in code-review/SKILL.md and findings schema in templates/findings-output.yaml)."
|
|
133
|
+
chosen_interpretation: "Update the diagnostic schema (code-review/SKILL.md). The findings template already has the field."
|
|
134
|
+
rationale: "Stage 2 verification of C-12 showed the findings template already includes `followup_edits_expected`; only the diagnostic surface is missing."
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
### `adjusted_plan`
|
|
138
|
+
|
|
139
|
+
The binding plan for the rest of the WP. Each deliverable has an explicit `change_from_spec` field so the implementer (and future readers) can audit the rewrite.
|
|
140
|
+
|
|
141
|
+
**Filled-in example**:
|
|
142
|
+
```yaml
|
|
143
|
+
adjusted_plan:
|
|
144
|
+
binding_status: "supersedes original spec for rest of WP"
|
|
145
|
+
deliverables:
|
|
146
|
+
- id: AD-01
|
|
147
|
+
description: "Update diagnostic schema in code-review/SKILL.md to include followup_edits_expected"
|
|
148
|
+
original_spec_step: 2
|
|
149
|
+
change_from_spec: re-targeted
|
|
150
|
+
target_path: skills/code-review/SKILL.md
|
|
151
|
+
token_estimate: "~3K"
|
|
152
|
+
- id: AD-02
|
|
153
|
+
description: "Add 5 test cases to test-suggest-pipeline-stop.sh + audit set -e"
|
|
154
|
+
original_spec_step: 3
|
|
155
|
+
change_from_spec: unchanged
|
|
156
|
+
target_path: tests/hooks/test-suggest-pipeline-stop.sh
|
|
157
|
+
token_estimate: "~6K"
|
|
158
|
+
estimated_token_delta: "-4K"
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
### `verification_checklist`
|
|
162
|
+
|
|
163
|
+
Items the implementer (or a downstream reviewer) must confirm at the end of the WP to verify the verified plan was executed. Typically one item per drift finding that survives into the adjusted plan.
|
|
164
|
+
|
|
165
|
+
**Filled-in example**:
|
|
166
|
+
```yaml
|
|
167
|
+
verification_checklist:
|
|
168
|
+
- "Diagnostic schema in skills/code-review/SKILL.md includes followup_edits_expected (field, not just prose)"
|
|
169
|
+
- "test-suggest-pipeline-stop.sh has 5 new test cases, all using assertion-counter pattern (no premature exit)"
|
|
170
|
+
- "AC-3 resolution committed: only diagnostic schema modified; findings template unchanged"
|
|
171
|
+
```
|
|
172
|
+
|
|
173
|
+
### `proceed_decision` + `decision_rationale`
|
|
174
|
+
|
|
175
|
+
The verdict itself + a 1-3 sentence summary tying the finding mix to the verdict. See `step-6-decision-matrix.md` for the matrix.
|
|
176
|
+
|
|
177
|
+
**Filled-in example**:
|
|
178
|
+
```yaml
|
|
179
|
+
proceed_decision: STOP_USER_APPROVAL
|
|
180
|
+
decision_rationale: |
|
|
181
|
+
Finding D-02 is HIGH (DRIFT-undeclared-scope: set -e audit task missing from spec).
|
|
182
|
+
Per the decision matrix, any HIGH finding triggers STOP_USER_APPROVAL.
|
|
183
|
+
Surface the scope expansion to the user via AskUserQuestion before binding the adjusted plan.
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
### `roi`
|
|
187
|
+
|
|
188
|
+
The cost-vs-savings estimate. Approximate; the goal is signal, not accounting. Round to the nearest 1K. `net: positive` means estimated savings exceed spent tokens.
|
|
189
|
+
|
|
190
|
+
**Filled-in example**:
|
|
191
|
+
```yaml
|
|
192
|
+
roi:
|
|
193
|
+
spent_tokens_estimate: "~6K"
|
|
194
|
+
estimated_savings: "~20K"
|
|
195
|
+
net: positive
|
|
196
|
+
rationale: |
|
|
197
|
+
Caught D-02 (undeclared scope: set -e audit) before implementation.
|
|
198
|
+
Without this finding, the implementer would have added tests, hit a fail-fast
|
|
199
|
+
on the missing audit, debugged for ~15K tokens, then re-implemented. ROI = +14K net.
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Field-Level Validation
|
|
205
|
+
|
|
206
|
+
Before writing the log, confirm:
|
|
207
|
+
|
|
208
|
+
- Every `findings` entry has BOTH `spec_claim` AND `actual_state` populated verbatim.
|
|
209
|
+
- Every `severity` value is one of LOW / MEDIUM / HIGH (or `n/a` for CONFIRMED).
|
|
210
|
+
- Every `category` value matches the 7-entry taxonomy from `step-3-categorization.md`.
|
|
211
|
+
- `proceed_decision` value matches the matrix in `step-6-decision-matrix.md` (any HIGH → STOP_USER_APPROVAL).
|
|
212
|
+
- `adjusted_plan.deliverables[*].change_from_spec` is set on every entry.
|
|
213
|
+
|
|
214
|
+
If any check fails, fix before writing — the log is binding (Stage 7) and downstream consumers parse it.
|
|
215
|
+
|
|
216
|
+
---
|
|
217
|
+
|
|
218
|
+
## After Writing the Log
|
|
219
|
+
|
|
220
|
+
The log path is the artifact. Surface it to the user. If the verdict is STOP_USER_APPROVAL, immediately follow with the AskUserQuestion flow per `step-6-decision-matrix.md`. The implementer (next session, next command) reads from this log, not from the original spec.
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# Step 6 — Decision Matrix
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Map the **mix of findings** from Stage 3 to a verdict. The verdict is one of three values:
|
|
6
|
+
|
|
7
|
+
- **PROCEED** — original plan stands; no adjustments needed.
|
|
8
|
+
- **PROCEED_ADJUSTED** — adjusted plan in the verification log supersedes the original; flag in summary; no user gate.
|
|
9
|
+
- **STOP_USER_APPROVAL** — surface findings to the user via AskUserQuestion; do NOT bind the adjusted plan until the user signs off.
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Decision Matrix
|
|
14
|
+
|
|
15
|
+
| Finding mix | Verdict | Rationale |
|
|
16
|
+
|-------------|---------|-----------|
|
|
17
|
+
| All CONFIRMED | **PROCEED** | The spec is in alignment with current code; no plan rewrite needed. |
|
|
18
|
+
| Only LOW + MEDIUM findings | **PROCEED_ADJUSTED** | Drift exists but is contained; the adjusted plan resolves it without scope changes. |
|
|
19
|
+
| Any HIGH finding | **STOP_USER_APPROVAL** | Scope changes, wrong-file drift, GAPs, or undeclared-scope additions need explicit user sign-off. |
|
|
20
|
+
|
|
21
|
+
Reading order: **HIGH wins**. If the run produces 30 LOW findings and 1 HIGH, the verdict is STOP_USER_APPROVAL. The matrix is not a vote count.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Tie-Breaking
|
|
26
|
+
|
|
27
|
+
The matrix above resolves cleanly — there are no genuine ties. Two situations that look like ties:
|
|
28
|
+
|
|
29
|
+
- **All CONFIRMED + a single AC-reinterpretation MEDIUM** → still PROCEED_ADJUSTED. The ambiguity must be resolved in the log; that resolution is binding via Stage 7.
|
|
30
|
+
- **Mostly CONFIRMED + a borderline finding (LOW vs MEDIUM)** → choose MEDIUM (per `step-3-categorization.md` tie-breaker). Verdict stays PROCEED_ADJUSTED in either case.
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## STOP_USER_APPROVAL Escalation
|
|
35
|
+
|
|
36
|
+
When the verdict is STOP_USER_APPROVAL, the orchestrator must:
|
|
37
|
+
|
|
38
|
+
1. **Write the verification log first** (Stage 5). The log is the source of truth; the user reviews from it.
|
|
39
|
+
2. **Surface each HIGH finding individually** to the user via AskUserQuestion. Do NOT batch HIGH findings into a single yes/no — the user must adjudicate each.
|
|
40
|
+
3. **Propose an adjusted scope** for each HIGH finding: drop the deliverable / add the deliverable / re-target the path / clarify the AC.
|
|
41
|
+
4. **Wait for user input** before binding the adjusted plan via Stage 7. Do NOT proceed to implementation.
|
|
42
|
+
|
|
43
|
+
### AskUserQuestion Template
|
|
44
|
+
|
|
45
|
+
Use this fragment, customized per finding. One AskUserQuestion call per HIGH finding (or per logically-coupled group of findings).
|
|
46
|
+
|
|
47
|
+
```text
|
|
48
|
+
question: "spec-drift-check found HIGH-severity drift in {spec-path}. Approve scope adjustment?"
|
|
49
|
+
|
|
50
|
+
multiSelect: false
|
|
51
|
+
|
|
52
|
+
header: "Spec Drift — HIGH ({finding-id})"
|
|
53
|
+
|
|
54
|
+
options:
|
|
55
|
+
- label: "Approve adjusted scope"
|
|
56
|
+
description: |
|
|
57
|
+
Finding {D-NN} ({category}): {short summary}.
|
|
58
|
+
Spec claim: "{verbatim spec quote, truncated to ~120 chars}"
|
|
59
|
+
Actual state: "{verbatim actual quote, truncated to ~120 chars}"
|
|
60
|
+
Proposed adjustment: {proposed adjustment in 1-2 sentences}
|
|
61
|
+
Token-budget impact: {+NK | -NK | 0}
|
|
62
|
+
|
|
63
|
+
- label: "Reject adjustment — keep original spec"
|
|
64
|
+
description: |
|
|
65
|
+
Implementation will follow the original spec verbatim despite the drift.
|
|
66
|
+
The orchestrator will document the override in the verification log.
|
|
67
|
+
|
|
68
|
+
- label: "Need more detail"
|
|
69
|
+
description: |
|
|
70
|
+
Open the verification log at logs/spec-verify-{session}-{topic}.md and
|
|
71
|
+
review finding {D-NN} in full before deciding.
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### Worked AskUserQuestion Example
|
|
75
|
+
|
|
76
|
+
For a hypothetical D-02 (DRIFT-undeclared-scope: `set -e` audit task missing from spec):
|
|
77
|
+
|
|
78
|
+
```text
|
|
79
|
+
question: "spec-drift-check found HIGH-severity drift in plans/task-briefs/P10.16-...md. Approve scope adjustment?"
|
|
80
|
+
|
|
81
|
+
multiSelect: false
|
|
82
|
+
|
|
83
|
+
header: "Spec Drift — HIGH (D-02)"
|
|
84
|
+
|
|
85
|
+
options:
|
|
86
|
+
- label: "Approve adjusted scope"
|
|
87
|
+
description: |
|
|
88
|
+
Finding D-02 (drift-undeclared-scope): the test plan implicitly requires
|
|
89
|
+
a set -e audit per process_test_harness_set_e_pattern.md, which the spec
|
|
90
|
+
does not mention.
|
|
91
|
+
Spec claim: "Add 5 test cases to test-suggest-pipeline-stop.sh."
|
|
92
|
+
Actual state: "Harness uses set -euo pipefail; new tests must use the
|
|
93
|
+
assertion-counter pattern, not naive expect-fail."
|
|
94
|
+
Proposed adjustment: add a Step 3a "audit set -e + counter pattern"
|
|
95
|
+
task before adding test cases.
|
|
96
|
+
Token-budget impact: +1K
|
|
97
|
+
|
|
98
|
+
- label: "Reject adjustment — keep original spec"
|
|
99
|
+
description: |
|
|
100
|
+
Implementation will follow the original spec verbatim. The orchestrator
|
|
101
|
+
will document the override in the verification log.
|
|
102
|
+
|
|
103
|
+
- label: "Need more detail"
|
|
104
|
+
description: |
|
|
105
|
+
Open the verification log at logs/spec-verify-122-P10.16.md and
|
|
106
|
+
review finding D-02 in full before deciding.
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
---
|
|
110
|
+
|
|
111
|
+
## Procedure: STOP and Surface
|
|
112
|
+
|
|
113
|
+
When the verdict fires STOP_USER_APPROVAL, follow this exact sequence:
|
|
114
|
+
|
|
115
|
+
1. **Stage 5 log written.** Confirm the log file exists and contains every HIGH finding with both `spec_claim` and `actual_state` populated verbatim.
|
|
116
|
+
2. **List each HIGH finding** to the user in the orchestrator's text response. Format: `D-NN ({category}, severity HIGH): {one-line summary}`. This gives the user a quick scan before the AskUserQuestion fires.
|
|
117
|
+
3. **Propose adjusted scope** in the same text response. One bullet per HIGH finding.
|
|
118
|
+
4. **Fire AskUserQuestion** per HIGH finding (in the next assistant message). Use the template above.
|
|
119
|
+
5. **Wait for the response** to each AskUserQuestion before proceeding. Do NOT chain multiple AskUserQuestion calls in parallel — sequential adjudication is intentional.
|
|
120
|
+
6. **Bind the adjusted plan (Stage 7)** only after every HIGH finding has an explicit user decision recorded in the log.
|
|
121
|
+
|
|
122
|
+
If the user rejects an adjustment, document the rejection in the log under `decision_rationale:` and either:
|
|
123
|
+
- Carry the original spec deliverable forward unchanged (and accept the implementation risk), OR
|
|
124
|
+
- Defer the entire WP back to the user for spec revision.
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## Decision Rationale Field
|
|
129
|
+
|
|
130
|
+
Every verdict — even PROCEED — should populate `decision_rationale:` in the log with a 1-3 sentence summary. Examples:
|
|
131
|
+
|
|
132
|
+
- **PROCEED**: "All 23 claims CONFIRMED. Spec aligns with current code; no adjustments needed."
|
|
133
|
+
- **PROCEED_ADJUSTED**: "5 LOW findings (line-ref drift), 2 MEDIUM (1 missing-scope, 1 AC-reinterpretation). Adjusted plan drops the no-op deliverable and resolves AC-3 explicitly. No scope changes."
|
|
134
|
+
- **STOP_USER_APPROVAL**: "1 HIGH finding (D-02, drift-undeclared-scope: set -e audit). Surface to user before binding adjusted plan."
|
|
135
|
+
|
|
136
|
+
The rationale becomes part of the audit trail; downstream sessions read it to understand why this WP's plan was rewritten.
|