@rune-kit/rune 2.2.6 → 2.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +395 -389
- package/compiler/__tests__/tier-override.test.js +158 -0
- package/compiler/adapters/antigravity.js +3 -8
- package/compiler/adapters/codex.js +3 -8
- package/compiler/adapters/cursor.js +3 -8
- package/compiler/adapters/generic.js +3 -8
- package/compiler/adapters/openclaw.js +4 -9
- package/compiler/adapters/opencode.js +3 -8
- package/compiler/adapters/windsurf.js +3 -8
- package/compiler/bin/rune.js +34 -1
- package/compiler/emitter.js +94 -5
- package/compiler/transforms/branding.js +10 -3
- package/docs/ARCHITECTURE.md +3 -3
- package/docs/SKILL-TEMPLATE.md +15 -0
- package/docs/VISION.md +3 -3
- package/docs/guides/index.html +14 -14
- package/docs/index.html +82 -13
- package/docs/script.js +33 -6
- package/docs/skills/index.html +832 -832
- package/docs/style.css +62 -0
- package/extensions/ai-ml/PACK.md +7 -0
- package/extensions/content/PACK.md +7 -0
- package/extensions/mobile/PACK.md +9 -9
- package/extensions/zalo/PACK.md +9 -0
- package/package.json +8 -6
- package/skills/adversary/SKILL.md +12 -0
- package/skills/audit/SKILL.md +526 -467
- package/skills/autopsy/SKILL.md +12 -0
- package/skills/ba/SKILL.md +349 -342
- package/skills/brainstorm/SKILL.md +11 -0
- package/skills/completion-gate/SKILL.md +260 -249
- package/skills/context-engine/SKILL.md +77 -1
- package/skills/context-pack/SKILL.md +160 -0
- package/skills/cook/SKILL.md +648 -958
- package/skills/cook/references/deviation-rules.md +19 -0
- package/skills/cook/references/error-recovery.md +37 -0
- package/skills/cook/references/exit-conditions.md +31 -0
- package/skills/cook/references/loop-detection.md +39 -0
- package/skills/cook/references/mid-run-signals.md +31 -0
- package/skills/cook/references/output-format.md +40 -0
- package/skills/cook/references/pack-detection.md +82 -0
- package/skills/cook/references/pause-resume-template.md +38 -0
- package/skills/cook/references/rfc-template.md +52 -0
- package/skills/cook/references/sharp-edges.md +24 -0
- package/skills/cook/references/subagent-status.md +38 -0
- package/skills/db/SKILL.md +12 -0
- package/skills/debug/SKILL.md +392 -362
- package/skills/deploy/SKILL.md +10 -0
- package/skills/deploy/references/post-deploy-integration.md +192 -0
- package/skills/design/SKILL.md +9 -0
- package/skills/docs/SKILL.md +12 -0
- package/skills/docs-seeker/SKILL.md +11 -0
- package/skills/fix/SKILL.md +281 -249
- package/skills/incident/SKILL.md +10 -0
- package/skills/launch/SKILL.md +12 -0
- package/skills/logic-guardian/SKILL.md +11 -0
- package/skills/marketing/SKILL.md +13 -0
- package/skills/mcp-builder/SKILL.md +13 -0
- package/skills/onboard/SKILL.md +50 -2
- package/skills/perf/SKILL.md +11 -0
- package/skills/plan/SKILL.md +342 -688
- package/skills/plan/references/completeness-scoring.md +36 -0
- package/skills/plan/references/outcome-block.md +40 -0
- package/skills/plan/references/plan-templates.md +193 -0
- package/skills/plan/references/wave-planning.md +44 -0
- package/skills/plan/references/workflow-registry.md +52 -0
- package/skills/preflight/SKILL.md +360 -280
- package/skills/rescue/SKILL.md +11 -0
- package/skills/research/SKILL.md +149 -150
- package/skills/retro/SKILL.md +11 -0
- package/skills/review/SKILL.md +489 -396
- package/skills/review-intake/SKILL.md +11 -0
- package/skills/safeguard/SKILL.md +12 -0
- package/skills/scaffold/SKILL.md +10 -0
- package/skills/scope-guard/SKILL.md +11 -0
- package/skills/scout/SKILL.md +9 -0
- package/skills/sentinel/SKILL.md +296 -425
- package/skills/sentinel/references/config-protection.md +52 -0
- package/skills/sentinel/references/destructive-commands.md +39 -0
- package/skills/sentinel/references/domain-hooks.md +73 -0
- package/skills/sentinel/references/framework-patterns.md +46 -0
- package/skills/sentinel/references/owasp-patterns.md +69 -0
- package/skills/sentinel/references/secret-patterns.md +40 -0
- package/skills/sentinel/references/skill-content-guard.md +54 -0
- package/skills/session-bridge/SKILL.md +56 -2
- package/skills/skill-forge/SKILL.md +47 -2
- package/skills/skill-router/{SKILL.md → skill.md} +446 -365
- package/skills/surgeon/SKILL.md +12 -0
- package/skills/team/SKILL.md +34 -1
- package/skills/test/SKILL.md +585 -427
- package/skills/watchdog/references/webhook-health-checks.md +243 -0
|
@@ -299,6 +299,15 @@ Choose Option B if [specific hedge condition].
|
|
|
299
299
|
Proceeding to rune:plan with Option A. Constraints to honor: [list].
|
|
300
300
|
```
|
|
301
301
|
|
|
302
|
+
## Returns
|
|
303
|
+
|
|
304
|
+
| Artifact | Format | Location |
|
|
305
|
+
|----------|--------|----------|
|
|
306
|
+
| Option matrix (2-3 Discovery / 3-5 Rescue) | Markdown sections | inline (chat output) |
|
|
307
|
+
| Trade-off analysis per option | Markdown (pros/cons/effort/risk) | inline |
|
|
308
|
+
| Single recommendation with hedge condition | Markdown | inline |
|
|
309
|
+
| Approved design document | Markdown | `docs/plans/<feature>.md` |
|
|
310
|
+
|
|
302
311
|
## Sharp Edges
|
|
303
312
|
|
|
304
313
|
Known failure modes for this skill. Check these before declaring done.
|
|
@@ -326,3 +335,5 @@ Known failure modes for this skill. Check these before declaring done.
|
|
|
326
335
|
## Cost Profile
|
|
327
336
|
|
|
328
337
|
~2000-5000 tokens input, ~1000-2500 tokens output. Opus for creative reasoning depth. Runs infrequently — only when creative exploration is needed.
|
|
338
|
+
|
|
339
|
+
**Scope guardrail:** Brainstorm produces options and a recommendation — never implementation code or an execution plan. All code and planning begins only after user approves an approach and `rune:plan` is invoked.
|
|
@@ -1,249 +1,260 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: completion-gate
|
|
3
|
-
description: "Validates agent claims against evidence trail. Catches 'done' without proof, 'tests pass' without output, 'fixed' without verification. Called by cook and team at workflow end."
|
|
4
|
-
user-invocable: false
|
|
5
|
-
metadata:
|
|
6
|
-
author: runedev
|
|
7
|
-
version: "1.
|
|
8
|
-
layer: L3
|
|
9
|
-
model: haiku
|
|
10
|
-
group: validation
|
|
11
|
-
tools: "Read, Bash, Glob, Grep"
|
|
12
|
-
---
|
|
13
|
-
|
|
14
|
-
# completion-gate
|
|
15
|
-
|
|
16
|
-
## Purpose
|
|
17
|
-
|
|
18
|
-
The lie detector for agent claims. Validates that what an agent says it did actually happened — with evidence. Catches the #1 failure mode in AI coding: claiming completion without proof.
|
|
19
|
-
|
|
20
|
-
<HARD-GATE>
|
|
21
|
-
Every claim requires evidence. No evidence = UNCONFIRMED = BLOCK.
|
|
22
|
-
"I ran the tests and they pass" without stdout = UNCONFIRMED.
|
|
23
|
-
"I fixed the bug" without before/after diff = UNCONFIRMED.
|
|
24
|
-
"Build succeeds" without build output = UNCONFIRMED.
|
|
25
|
-
</HARD-GATE>
|
|
26
|
-
|
|
27
|
-
## Triggers
|
|
28
|
-
|
|
29
|
-
- Called by `cook` in Phase 5d (quality gate)
|
|
30
|
-
- Called by `team` before merging stream results
|
|
31
|
-
- Called by any skill that reports "done" to an orchestrator
|
|
32
|
-
- Auto-trigger: when agent says "done", "complete", "fixed", "passing"
|
|
33
|
-
|
|
34
|
-
## Calls (outbound)
|
|
35
|
-
|
|
36
|
-
None — pure validator. Reads evidence, produces verdict.
|
|
37
|
-
|
|
38
|
-
## Called By (inbound)
|
|
39
|
-
|
|
40
|
-
- `cook` (L1): Phase 5d — validate completion claims before commit
|
|
41
|
-
- `team` (L1): validate cook reports from parallel streams
|
|
42
|
-
|
|
43
|
-
## Execution
|
|
44
|
-
|
|
45
|
-
### Step 1 — Collect Claims
|
|
46
|
-
|
|
47
|
-
Parse the agent's output for completion claims. Common claim patterns:
|
|
48
|
-
|
|
49
|
-
```
|
|
50
|
-
CLAIM PATTERNS:
|
|
51
|
-
"tests pass" / "all tests passing" / "test suite green"
|
|
52
|
-
"build succeeds" / "build complete" / "compiles clean"
|
|
53
|
-
"no lint errors" / "lint clean"
|
|
54
|
-
"fixed" / "resolved" / "bug is gone"
|
|
55
|
-
"implemented" / "feature complete" / "done"
|
|
56
|
-
"no security issues" / "sentinel passed"
|
|
57
|
-
```
|
|
58
|
-
|
|
59
|
-
Extract each claim as: `{ claim: string, source_skill: string }`
|
|
60
|
-
|
|
61
|
-
### Step 1b — Stub Detection (Existence Theater Check)
|
|
62
|
-
|
|
63
|
-
Before checking claims, scan all files created/modified in this workflow for stubs:
|
|
64
|
-
|
|
65
|
-
```
|
|
66
|
-
Grep for stub patterns in new/modified files:
|
|
67
|
-
- "Placeholder" | "TODO" | "Not implemented" | "NotImplementedError"
|
|
68
|
-
- Functions with body: only `return null` / `return {}` / `pass` / `throw`
|
|
69
|
-
- Components returning only a single div with no logic
|
|
70
|
-
```
|
|
71
|
-
|
|
72
|
-
If ANY stub detected:
|
|
73
|
-
- Add synthetic claim: "implemented [filename]" → CONTRADICTED (file is a stub)
|
|
74
|
-
- This catches agents that create files but don't implement them
|
|
75
|
-
|
|
76
|
-
### Step 1c — Self-Validation Check
|
|
77
|
-
|
|
78
|
-
If the skill that just ran has a `## Self-Validation` section, extract its checklist and treat each item as an implicit claim:
|
|
79
|
-
|
|
80
|
-
```
|
|
81
|
-
For each Self-Validation check in the skill's SKILL.md:
|
|
82
|
-
1. Read the check (e.g., "at least one assertion per test")
|
|
83
|
-
2. Look for evidence in tool output that this check was satisfied
|
|
84
|
-
3. If evidence found → add as CONFIRMED claim
|
|
85
|
-
4. If no evidence → add as UNCONFIRMED claim ("Self-Validation: [check] — no evidence")
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
Why: Self-Validation catches domain-specific quality issues that generic claim matching (Step 2) cannot detect. A test skill knows "no assertions = useless test" but completion-gate doesn't — unless the skill's Self-Validation tells it to check.
|
|
89
|
-
|
|
90
|
-
<HARD-GATE>
|
|
91
|
-
If a skill has Self-Validation and ANY check is UNCONFIRMED or CONTRADICTED → overall verdict cannot be CONFIRMED, even if all explicit claims pass.
|
|
92
|
-
</HARD-GATE>
|
|
93
|
-
|
|
94
|
-
### Step 2 — Match Evidence
|
|
95
|
-
|
|
96
|
-
For each claim, look for corresponding evidence in the conversation context:
|
|
97
|
-
|
|
98
|
-
| Claim Type | Required Evidence | Where to Find |
|
|
99
|
-
|---|---|---|
|
|
100
|
-
| "tests pass" | Test runner stdout with pass count | Bash output from test command |
|
|
101
|
-
| "build succeeds" | Build command stdout showing success | Bash output from build command |
|
|
102
|
-
| "lint clean" | Linter stdout (even if empty = 0 errors) | Bash output from lint command |
|
|
103
|
-
| "fixed" | Git diff showing the change + test proving fix | Edit/Write tool calls + test output |
|
|
104
|
-
| "implemented" | Files created/modified matching the plan | Write/Edit tool calls vs plan |
|
|
105
|
-
| "no security issues" | Sentinel report with PASS verdict | Sentinel skill output |
|
|
106
|
-
| "coverage ≥ X%" | Coverage tool output with actual percentage | Test runner with coverage flag |
|
|
107
|
-
|
|
108
|
-
### Step 3 — Validate Each Claim (Default-FAIL Mindset)
|
|
109
|
-
|
|
110
|
-
<HARD-GATE>
|
|
111
|
-
Default posture is FAIL, not PASS. Actively seek 3-5 issues per review.
|
|
112
|
-
Zero issues found = red flag — look harder, not a sign of quality.
|
|
113
|
-
This prevents rubber-stamping where the gate confirms everything without scrutiny.
|
|
114
|
-
</HARD-GATE>
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
-
|
|
178
|
-
-
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
-
|
|
242
|
-
|
|
243
|
-
-
|
|
244
|
-
|
|
245
|
-
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
1
|
+
---
|
|
2
|
+
name: completion-gate
|
|
3
|
+
description: "Validates agent claims against evidence trail. Catches 'done' without proof, 'tests pass' without output, 'fixed' without verification. Called by cook and team at workflow end."
|
|
4
|
+
user-invocable: false
|
|
5
|
+
metadata:
|
|
6
|
+
author: runedev
|
|
7
|
+
version: "1.6.0"
|
|
8
|
+
layer: L3
|
|
9
|
+
model: haiku
|
|
10
|
+
group: validation
|
|
11
|
+
tools: "Read, Bash, Glob, Grep"
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# completion-gate
|
|
15
|
+
|
|
16
|
+
## Purpose
|
|
17
|
+
|
|
18
|
+
The lie detector for agent claims. Validates that what an agent says it did actually happened — with evidence. Catches the #1 failure mode in AI coding: claiming completion without proof.
|
|
19
|
+
|
|
20
|
+
<HARD-GATE>
|
|
21
|
+
Every claim requires evidence. No evidence = UNCONFIRMED = BLOCK.
|
|
22
|
+
"I ran the tests and they pass" without stdout = UNCONFIRMED.
|
|
23
|
+
"I fixed the bug" without before/after diff = UNCONFIRMED.
|
|
24
|
+
"Build succeeds" without build output = UNCONFIRMED.
|
|
25
|
+
</HARD-GATE>
|
|
26
|
+
|
|
27
|
+
## Triggers
|
|
28
|
+
|
|
29
|
+
- Called by `cook` in Phase 5d (quality gate)
|
|
30
|
+
- Called by `team` before merging stream results
|
|
31
|
+
- Called by any skill that reports "done" to an orchestrator
|
|
32
|
+
- Auto-trigger: when agent says "done", "complete", "fixed", "passing"
|
|
33
|
+
|
|
34
|
+
## Calls (outbound)
|
|
35
|
+
|
|
36
|
+
None — pure validator. Reads evidence, produces verdict.
|
|
37
|
+
|
|
38
|
+
## Called By (inbound)
|
|
39
|
+
|
|
40
|
+
- `cook` (L1): Phase 5d — validate completion claims before commit
|
|
41
|
+
- `team` (L1): validate cook reports from parallel streams
|
|
42
|
+
|
|
43
|
+
## Execution
|
|
44
|
+
|
|
45
|
+
### Step 1 — Collect Claims
|
|
46
|
+
|
|
47
|
+
Parse the agent's output for completion claims. Common claim patterns:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
CLAIM PATTERNS:
|
|
51
|
+
"tests pass" / "all tests passing" / "test suite green"
|
|
52
|
+
"build succeeds" / "build complete" / "compiles clean"
|
|
53
|
+
"no lint errors" / "lint clean"
|
|
54
|
+
"fixed" / "resolved" / "bug is gone"
|
|
55
|
+
"implemented" / "feature complete" / "done"
|
|
56
|
+
"no security issues" / "sentinel passed"
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Extract each claim as: `{ claim: string, source_skill: string }`
|
|
60
|
+
|
|
61
|
+
### Step 1b — Stub Detection (Existence Theater Check)
|
|
62
|
+
|
|
63
|
+
Before checking claims, scan all files created/modified in this workflow for stubs:
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
Grep for stub patterns in new/modified files:
|
|
67
|
+
- "Placeholder" | "TODO" | "Not implemented" | "NotImplementedError"
|
|
68
|
+
- Functions with body: only `return null` / `return {}` / `pass` / `throw`
|
|
69
|
+
- Components returning only a single div with no logic
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
If ANY stub detected:
|
|
73
|
+
- Add synthetic claim: "implemented [filename]" → CONTRADICTED (file is a stub)
|
|
74
|
+
- This catches agents that create files but don't implement them
|
|
75
|
+
|
|
76
|
+
### Step 1c — Self-Validation Check
|
|
77
|
+
|
|
78
|
+
If the skill that just ran has a `## Self-Validation` section, extract its checklist and treat each item as an implicit claim:
|
|
79
|
+
|
|
80
|
+
```
|
|
81
|
+
For each Self-Validation check in the skill's SKILL.md:
|
|
82
|
+
1. Read the check (e.g., "at least one assertion per test")
|
|
83
|
+
2. Look for evidence in tool output that this check was satisfied
|
|
84
|
+
3. If evidence found → add as CONFIRMED claim
|
|
85
|
+
4. If no evidence → add as UNCONFIRMED claim ("Self-Validation: [check] — no evidence")
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Why: Self-Validation catches domain-specific quality issues that generic claim matching (Step 2) cannot detect. A test skill knows "no assertions = useless test" but completion-gate doesn't — unless the skill's Self-Validation tells it to check.
|
|
89
|
+
|
|
90
|
+
<HARD-GATE>
|
|
91
|
+
If a skill has Self-Validation and ANY check is UNCONFIRMED or CONTRADICTED → overall verdict cannot be CONFIRMED, even if all explicit claims pass.
|
|
92
|
+
</HARD-GATE>
|
|
93
|
+
|
|
94
|
+
### Step 2 — Match Evidence
|
|
95
|
+
|
|
96
|
+
For each claim, look for corresponding evidence in the conversation context:
|
|
97
|
+
|
|
98
|
+
| Claim Type | Required Evidence | Where to Find |
|
|
99
|
+
|---|---|---|
|
|
100
|
+
| "tests pass" | Test runner stdout with pass count | Bash output from test command |
|
|
101
|
+
| "build succeeds" | Build command stdout showing success | Bash output from build command |
|
|
102
|
+
| "lint clean" | Linter stdout (even if empty = 0 errors) | Bash output from lint command |
|
|
103
|
+
| "fixed" | Git diff showing the change + test proving fix | Edit/Write tool calls + test output |
|
|
104
|
+
| "implemented" | Files created/modified matching the plan | Write/Edit tool calls vs plan |
|
|
105
|
+
| "no security issues" | Sentinel report with PASS verdict | Sentinel skill output |
|
|
106
|
+
| "coverage ≥ X%" | Coverage tool output with actual percentage | Test runner with coverage flag |
|
|
107
|
+
|
|
108
|
+
### Step 3 — Validate Each Claim (Default-FAIL Mindset)
|
|
109
|
+
|
|
110
|
+
<HARD-GATE>
|
|
111
|
+
Default posture is FAIL, not PASS. Actively seek 3-5 issues per review.
|
|
112
|
+
Zero issues found = red flag — look harder, not a sign of quality.
|
|
113
|
+
This prevents rubber-stamping where the gate confirms everything without scrutiny.
|
|
114
|
+
</HARD-GATE>
|
|
115
|
+
|
|
116
|
+
|
|
117
|
+
For each claim + evidence pair:
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
IF evidence exists AND evidence supports claim:
|
|
121
|
+
→ CONFIRMED
|
|
122
|
+
IF evidence exists BUT contradicts claim:
|
|
123
|
+
→ CONTRADICTED (most serious — agent is wrong)
|
|
124
|
+
IF no evidence found:
|
|
125
|
+
→ UNCONFIRMED (agent may be right but didn't prove it)
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
**3-Axis verification** — categorize each claim into one of three axes, then ensure all axes are covered:
|
|
129
|
+
|
|
130
|
+
| Axis | Question | Example Claims |
|
|
131
|
+
|------|----------|----------------|
|
|
132
|
+
| **Completeness** | Were all planned tasks done? All specs implemented? | "implemented feature X", "all TODO items done", "migration created" |
|
|
133
|
+
| **Correctness** | Does output match spec intent? Do tests verify real behavior? | "tests pass", "build succeeds", "lint clean", "fixed the bug" |
|
|
134
|
+
| **Coherence** | Does it follow project patterns? Consistent with existing code? | "follows conventions", "uses existing patterns", "no new deps needed" |
|
|
135
|
+
|
|
136
|
+
If an axis has ZERO claims → flag as gap: "No [Completeness/Correctness/Coherence] evidence found — agent may have skipped this dimension."
|
|
137
|
+
|
|
138
|
+
**Adversarial validation checklist** (run AFTER initial verdicts):
|
|
139
|
+
1. Re-read each CONFIRMED claim — is the evidence actually proving THIS claim, or a different one?
|
|
140
|
+
2. Check for **partial completion** — did the agent do 80% but claim 100%? (e.g., "implemented feature" but only the happy path)
|
|
141
|
+
3. Check for **scope mismatch** — does the evidence prove the SPECIFIC claim or a broader/narrower version?
|
|
142
|
+
4. If all claims are CONFIRMED on first pass, apply **skeptic sweep**: re-examine the weakest 2 claims with heightened scrutiny
|
|
143
|
+
5. Check **axis coverage** — are all 3 axes (Completeness/Correctness/Coherence) represented? Missing axis = investigation gap
|
|
144
|
+
|
|
145
|
+
### Step 4 — Report
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
## Completion Gate Report
|
|
149
|
+
- **Status**: CONFIRMED | UNCONFIRMED | CONTRADICTED
|
|
150
|
+
- **Claims Checked**: [count]
|
|
151
|
+
- **Confirmed**: [count] | **Unconfirmed**: [count] | **Contradicted**: [count]
|
|
152
|
+
|
|
153
|
+
### Claim Validation
|
|
154
|
+
| # | Claim | Evidence | Verdict |
|
|
155
|
+
|---|---|---|---|
|
|
156
|
+
| 1 | "All tests pass" | Bash: `npm test` → "42 passed, 0 failed" | CONFIRMED |
|
|
157
|
+
| 2 | "Build succeeds" | No build command output found | UNCONFIRMED |
|
|
158
|
+
| 3 | "No lint errors" | Bash: `npm run lint` → "3 errors" | CONTRADICTED |
|
|
159
|
+
|
|
160
|
+
### Gaps (if any)
|
|
161
|
+
- Claim 2: Re-run `npm run build` and capture output
|
|
162
|
+
- Claim 3: Agent claimed clean but lint shows 3 errors — fix required
|
|
163
|
+
|
|
164
|
+
### Verdict
|
|
165
|
+
UNCONFIRMED — 1 claim lacks evidence, 1 contradicted. Cannot proceed to commit.
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
### Step 4.5 — Cross-Phase Integration Check
|
|
169
|
+
|
|
170
|
+
> From GSD (gsd-build/get-shit-done, 30.8k★): "Phase boundaries are where integration bugs hide."
|
|
171
|
+
|
|
172
|
+
When validating a completed phase in a multi-phase plan, check for integration gaps between phases:
|
|
173
|
+
|
|
174
|
+
1. **Orphaned exports** — files/functions created in this phase that claim to be used by future phases (see `## Cross-Phase Context → Exports`) but are not yet importable:
|
|
175
|
+
```
|
|
176
|
+
Grep for the export name in the current codebase:
|
|
177
|
+
- If export exists AND is importable → CONFIRMED
|
|
178
|
+
- If export exists but has wrong signature vs phase file contract → CONTRADICTED
|
|
179
|
+
- Expected export missing entirely → UNCONFIRMED ("Phase N claims to export X but X not found")
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
2. **Uncalled routes** — API endpoints added in this phase but not wired to any frontend/consumer yet:
|
|
183
|
+
- This is OK if a future phase handles wiring (check master plan)
|
|
184
|
+
- Flag as WARN if no future phase mentions consuming this route
|
|
185
|
+
|
|
186
|
+
3. **Auth gaps** — new endpoints or pages without authentication/authorization:
|
|
187
|
+
- `Grep` for route handlers without auth middleware
|
|
188
|
+
- Flag as WARN (may be intentional for public endpoints, but worth checking)
|
|
189
|
+
|
|
190
|
+
4. **E2E flow trace** — for the primary user flow this phase enables:
|
|
191
|
+
- Trace: entry point → business logic → data layer → response
|
|
192
|
+
- If any step in the chain is missing or stubbed → CONTRADICTED
|
|
193
|
+
|
|
194
|
+
**This step is OPTIONAL for single-phase tasks and MANDATORY for multi-phase master plans.**
|
|
195
|
+
|
|
196
|
+
### Step 5 — Evidence Quality Gate
|
|
197
|
+
|
|
198
|
+
Before emitting verdict, verify evidence quality:
|
|
199
|
+
|
|
200
|
+
1. **IDENTIFY** — list every claim the agent made (Step 1 output)
|
|
201
|
+
2. **RUN** — confirm verification commands were actually executed (not just planned)
|
|
202
|
+
3. **READ** — read every line of command output (not just exit code)
|
|
203
|
+
4. **VERIFY** — match each claim to a specific evidence quote (file:line or output snippet)
|
|
204
|
+
5. **CLAIM** — only mark CONFIRMED if evidence quote directly supports the claim
|
|
205
|
+
|
|
206
|
+
| Evidence Quality | Verdict |
|
|
207
|
+
|-----------------|---------|
|
|
208
|
+
| Exit code 0 only, no output read | INSUFFICIENT — re-run and read output |
|
|
209
|
+
| Output read but no quote matched to claim | UNCONFIRMED — cite specific evidence |
|
|
210
|
+
| Quote matches claim exactly | CONFIRMED |
|
|
211
|
+
| Quote contradicts claim | CONTRADICTED |
|
|
212
|
+
|
|
213
|
+
## Verdict Rules
|
|
214
|
+
|
|
215
|
+
```
|
|
216
|
+
ALL claims CONFIRMED → overall CONFIRMED (proceed)
|
|
217
|
+
ANY claim CONTRADICTED → overall CONTRADICTED (BLOCK — fix the contradiction)
|
|
218
|
+
ANY claim UNCONFIRMED → overall UNCONFIRMED (BLOCK — provide evidence)
|
|
219
|
+
(no CONTRADICTED)
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
## Output Format
|
|
223
|
+
|
|
224
|
+
Completion Gate Report with status (CONFIRMED/UNCONFIRMED/CONTRADICTED), claim validation table, gaps, and verdict. See Step 4 Report above for full template.
|
|
225
|
+
|
|
226
|
+
## Constraints
|
|
227
|
+
|
|
228
|
+
1. MUST check every completion claim against actual tool output — not agent narrative
|
|
229
|
+
2. MUST flag missing evidence as UNCONFIRMED — absence of proof is not proof of absence
|
|
230
|
+
3. MUST flag contradictions as CONTRADICTED — this is more serious than missing evidence
|
|
231
|
+
4. MUST NOT accept "I verified it" as evidence — show the command output
|
|
232
|
+
5. MUST be fast (haiku) — this runs on every cook completion
|
|
233
|
+
|
|
234
|
+
## Sharp Edges
|
|
235
|
+
|
|
236
|
+
| Failure Mode | Severity | Mitigation |
|
|
237
|
+
|---|---|---|
|
|
238
|
+
| Agent rephrases claim to avoid detection | MEDIUM | Pattern matching covers common phrasings — extend as new patterns emerge |
|
|
239
|
+
| Evidence from a DIFFERENT test run (stale) | HIGH | Check that evidence timestamp/context matches current changes |
|
|
240
|
+
| Agent pre-generates evidence by running commands proactively | LOW | This is actually GOOD behavior — we want agents to provide evidence |
|
|
241
|
+
| Completion-gate itself claims "all confirmed" without evidence | CRITICAL | Gate report MUST include the evidence table — no table = report is invalid |
|
|
242
|
+
| Existence Theater — agent creates files but they're stubs | HIGH | Step 1b stub detection: grep for Placeholder/TODO/NotImplementedError in new files |
|
|
243
|
+
| Cross-phase integration gaps — exports exist but wrong signature | HIGH | Step 4.5: verify exports match Code Contracts from phase file |
|
|
244
|
+
| Phase complete but E2E flow broken — missing link in the chain | MEDIUM | Step 4.5 E2E flow trace: entry → logic → data → response must all be connected |
|
|
245
|
+
| Rubber-stamping — all CONFIRMED without scrutiny | HIGH | Default-FAIL mindset: actively seek 3-5 issues. Zero issues = red flag, apply skeptic sweep on weakest 2 claims |
|
|
246
|
+
| Partial completion claimed as full — 80% done but "implemented" | HIGH | Adversarial checklist: check for partial completion, scope mismatch, evidence-claim alignment |
|
|
247
|
+
| Self-Validation skipped — skill has checks but gate ignores them | HIGH | Step 1c: extract Self-Validation from skill's SKILL.md, treat each as implicit claim. Missing = UNCONFIRMED |
|
|
248
|
+
|
|
249
|
+
## Done When
|
|
250
|
+
|
|
251
|
+
- All completion claims extracted from agent output
|
|
252
|
+
- Each claim matched against tool output evidence
|
|
253
|
+
- Verdict table emitted with claim/evidence/verdict for each item
|
|
254
|
+
- All 3 verification axes (Completeness/Correctness/Coherence) have at least one claim checked
|
|
255
|
+
- Overall verdict: CONFIRMED / UNCONFIRMED / CONTRADICTED
|
|
256
|
+
- If not CONFIRMED: specific gaps listed with remediation steps
|
|
257
|
+
|
|
258
|
+
## Cost Profile
|
|
259
|
+
|
|
260
|
+
~500-1000 tokens input, ~200-500 tokens output. Haiku for speed. Runs frequently as part of cook's quality phase.
|