@tianhai/pi-workflow-kit 0.16.0 → 0.17.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -9
- package/docs/plans/2026-06-03-karpathy-guidelines-ab-comparison.md +166 -0
- package/docs/plans/completed/2026-06-03-add-verify-skill-design.md +51 -0
- package/docs/plans/completed/2026-06-03-add-verify-skill-implementation.md +111 -0
- package/docs/plans/completed/2026-06-03-add-verify-skill-progress.md +11 -0
- package/docs/plans/completed/2026-06-03-verify-skill-design.md +176 -0
- package/package.json +1 -1
- package/skills/verify/SKILL.md +170 -0
- package/skills/writing-plans/SKILL.md +51 -0
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# pi-workflow-kit
|
|
2
2
|
|
|
3
|
-
> Stop AI agents from rushing to code. Enforce a structured brainstorm→plan→execute→finalize workflow with TDD discipline.
|
|
3
|
+
> Stop AI agents from rushing to code. Enforce a structured brainstorm→plan→execute→verify→finalize workflow with TDD discipline.
|
|
4
4
|
|
|
5
5
|
AI coding agents tend to skip design and jump straight into implementation, producing over-engineered or misaligned code. **pi-workflow-kit** solves this by hard-blocking write operations during brainstorm and planning phases — the agent *literally cannot modify your source files* until you approve the design.
|
|
6
6
|
|
|
@@ -33,15 +33,13 @@ Enforces phase-appropriate tool access — not just guidelines, but hard blocks:
|
|
|
33
33
|
|
|
34
34
|
The agent can read code and discuss design with you during brainstorm/plan, but it physically cannot modify source files or run mutating commands.
|
|
35
35
|
|
|
36
|
-
### 🧠
|
|
36
|
+
### 🧠 7 Workflow Skills
|
|
37
37
|
|
|
38
38
|
Guide the agent through a disciplined development process:
|
|
39
39
|
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
diagnose (anytime)
|
|
44
|
-
```
|
|
40
|
+
brainstorm → design-review → plan → execute → verify → finalize
|
|
41
|
+
↕
|
|
42
|
+
diagnose (anytime)
|
|
45
43
|
|
|
46
44
|
| Phase | Trigger | What Happens |
|
|
47
45
|
|-------|---------|--------------|
|
|
@@ -49,6 +47,7 @@ brainstorm → design-review → plan → execute → finalize
|
|
|
49
47
|
| **Design Review** | `/skill:design-review` | Audit design for production risks (security, scalability, fault tolerance) |
|
|
50
48
|
| **Plan** | `/skill:writing-plans` | Break design into bite-sized TDD tasks with acceptance criteria and concrete code |
|
|
51
49
|
| **Execute** | `/skill:executing-tasks` | Implement tasks one-by-one with TDD discipline and pre-commit checkpoint review gates |
|
|
50
|
+
| **Verify** | `/skill:verify` | Three expert review passes (security, optimization, traceability) on implemented code |
|
|
52
51
|
| **Finalize** | `/skill:finalizing` | Archive plan docs, update README/CHANGELOG, create PR |
|
|
53
52
|
| **Diagnose** | `/skill:diagnose` | 6-phase debugging loop: reproduce → hypothesize → instrument → fix → verify |
|
|
54
53
|
|
|
@@ -58,13 +57,12 @@ brainstorm → design-review → plan → execute → finalize
|
|
|
58
57
|
|
|
59
58
|
You control each phase — the agent never advances on its own. Invoke a skill to move forward:
|
|
60
59
|
|
|
61
|
-
```
|
|
62
60
|
/skill:brainstorming → discuss and design
|
|
63
61
|
/skill:design-review → audit for production risks (non-trivial designs)
|
|
64
62
|
/skill:writing-plans → break into tasks
|
|
65
63
|
/skill:executing-tasks → implement with TDD
|
|
64
|
+
/skill:verify → review code for security, optimization, and traceability issues
|
|
66
65
|
/skill:finalizing → ship it
|
|
67
|
-
```
|
|
68
66
|
|
|
69
67
|
### TDD Three-Scenario Model
|
|
70
68
|
|
|
@@ -129,6 +127,9 @@ pi install npm:@tianhai/pi-workflow-kit
|
|
|
129
127
|
> /skill:executing-tasks
|
|
130
128
|
|
|
131
129
|
# (agent implements with TDD, cognitive persona shifts, all tools unlocked)
|
|
130
|
+
> /skill:verify
|
|
131
|
+
|
|
132
|
+
# (agent runs security, optimization, and traceability reviews on implemented code)
|
|
132
133
|
> /skill:finalizing
|
|
133
134
|
|
|
134
135
|
# (agent archives docs, curates lessons, creates PR)
|
|
@@ -152,6 +153,7 @@ pi-workflow-kit/
|
|
|
152
153
|
│ ├── design-review/SKILL.md
|
|
153
154
|
│ ├── writing-plans/SKILL.md
|
|
154
155
|
│ ├── executing-tasks/SKILL.md
|
|
156
|
+
│ ├── verify/SKILL.md
|
|
155
157
|
│ ├── finalizing/SKILL.md
|
|
156
158
|
│ └── diagnose/SKILL.md
|
|
157
159
|
├── tests/
|
|
@@ -0,0 +1,166 @@
|
|
|
1
|
+
# A/B Comparison: Writing Plans — Karpathy Behavioral Guidelines
|
|
2
|
+
|
|
3
|
+
## Setup
|
|
4
|
+
- **Same design doc** (bookmarks: CRUD + search)
|
|
5
|
+
- **Same Go project scaffold**
|
|
6
|
+
- **Same prompt** (no questions, full plan with concrete code)
|
|
7
|
+
- **Variant A** (WITHOUT guidelines): 292-line SKILL.md — original writing-plans skill
|
|
8
|
+
- **Variant B** (WITH guidelines): 354-line SKILL.md — with Behavioral Guidelines section appended
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## Structural Comparison
|
|
13
|
+
|
|
14
|
+
| Dimension | A (Without) | B (With) |
|
|
15
|
+
|---|---|---|
|
|
16
|
+
| **Total tasks** | 4 | 6 |
|
|
17
|
+
| **Lines in plan** | ~1,054 | ~1,019 |
|
|
18
|
+
| **New files per plan** | 7 files in Task 1 alone | 1-2 files per task |
|
|
19
|
+
| **External dependency** | None (stdlib only) | `github.com/google/uuid` |
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Task Decomposition
|
|
24
|
+
|
|
25
|
+
### A (Without) — 4 tasks
|
|
26
|
+
| Task | Scope | Files touched |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| 1 | Bookmark + ALL infrastructure (model, store interface, mem store with full CRUD, service, handler, errors, route, tests) | 7 files |
|
|
29
|
+
| 2 | Delete bookmark | 3 files |
|
|
30
|
+
| 3 | List bookmarks (paginated, cursor) | 3 files |
|
|
31
|
+
| 4 | Search bookmarks (keyword + pagination) | 3 files |
|
|
32
|
+
|
|
33
|
+
### B (With) — 6 tasks
|
|
34
|
+
| Task | Scope | Files touched |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| 1 | Scaffold (go.mod + model only) | 2 files |
|
|
37
|
+
| 2 | Bookmark a message (store + handler + test + route) | 4 files |
|
|
38
|
+
| 3 | List bookmarks (offset/limit pagination) | 4 files |
|
|
39
|
+
| 4 | Remove a bookmark | 4 files |
|
|
40
|
+
| 5 | Search bookmarks (keyword) | 4 files |
|
|
41
|
+
| 6 | Final wiring + integration lifecycle test | 2 files |
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Detailed Analysis by Guideline
|
|
46
|
+
|
|
47
|
+
### Simplicity First
|
|
48
|
+
|
|
49
|
+
**A (Without):** ⚠️ **Overbuilt in Task 1.** Task 1 creates a `BookmarkStore` interface with 4 methods (Create, Delete, ListByUser, SearchByUser) — methods that won't be used until Tasks 2-4. It also creates the full `MemoryStore` implementation with all 4 methods, an `errors.go` file, a `Service` struct, AND the handler — all in a single task. The store interface is the full contract upfront before any task exercises most of it.
|
|
50
|
+
|
|
51
|
+
**B (With):** ✅ **Minimal per task.** Task 1 only creates `go.mod` + the `Bookmark` struct. Task 2 introduces `Store` with only `Create`, and `MemStore` with only `Create`. `List` is added to the interface in Task 3, `Delete` in Task 4, `Search` in Task 5 — each method appears when it's needed, not before.
|
|
52
|
+
|
|
53
|
+
**Verdict:** Guidelines had a clear positive effect. Plan B builds only what each task needs.
|
|
54
|
+
|
|
55
|
+
### Surgical Changes
|
|
56
|
+
|
|
57
|
+
**A (Without):** ⚠️ Task 1 touches 7 files in one go (model, store interface, store mem, errors, service, handler, main.go). The Task 1 description says "create the full vertical slice" which bundles infrastructure that isn't tested yet.
|
|
58
|
+
|
|
59
|
+
**B (With):** ✅ Each task touches 1-2 files for new code. Task 1 creates 2 files (go.mod, model.go). Task 2 adds 3 new files + modifies main.go. No task creates more than 4 files.
|
|
60
|
+
|
|
61
|
+
**Verdict:** Guidelines had a clear positive effect. Plan B has tighter blast radius per task.
|
|
62
|
+
|
|
63
|
+
### Think Before Coding (surface assumptions)
|
|
64
|
+
|
|
65
|
+
**A (Without):** ❌ Silent assumptions throughout:
|
|
66
|
+
- Used cursor-based pagination without noting the design just said "paginated" — didn't surface that offset-based vs cursor-based is a choice
|
|
67
|
+
- Added `sync.RWMutex` and concurrent safety without the design mentioning concurrency
|
|
68
|
+
- Created a `Service` layer between handler and store without justification
|
|
69
|
+
|
|
70
|
+
**B (With):** ⚠️ Still has assumptions but more defensible:
|
|
71
|
+
- Used offset/limit pagination (simpler, matches "paginated" literally)
|
|
72
|
+
- No concurrency concerns added (store uses `sync.Mutex` only, no RWMutex overhead)
|
|
73
|
+
- No `Service` layer — handler calls store directly
|
|
74
|
+
- Did add `github.com/google/uuid` dependency without asking — minor assumption
|
|
75
|
+
|
|
76
|
+
**Verdict:** Marginal positive effect. Plan B is less presumptuous but both plans made assumptions. Neither explicitly surfaced tradeoffs to the user.
|
|
77
|
+
|
|
78
|
+
### Goal-Driven Execution
|
|
79
|
+
|
|
80
|
+
**A (Without):** ✅ Good acceptance criteria with Given/When/Then. Has a `checkpoint: test` on 3/4 tasks and `checkpoint: done` on the last task.
|
|
81
|
+
|
|
82
|
+
**B (With):** ✅ Good acceptance criteria. Has `checkpoint: test` on 3 tasks, `checkpoint: done` on 1, and no checkpoint on 2 simpler tasks. Added a full lifecycle integration test in Task 6 that wasn't in A.
|
|
83
|
+
|
|
84
|
+
**Verdict:** Roughly equivalent. Both plans have strong acceptance criteria (required by the base skill). The lifecycle test in B is a nice bonus that catches integration issues.
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## Unrelated Observations (noise, not guidelines)
|
|
89
|
+
|
|
90
|
+
| Observation | A (Without) | B (With) |
|
|
91
|
+
|---|---|---|
|
|
92
|
+
| Pagination style | Cursor-based (more complex) | Offset-based (simpler) |
|
|
93
|
+
| External deps | None | `google/uuid` |
|
|
94
|
+
| Handler method naming | `Create`, `Delete`, `List`, `Search` | `CreateBookmark`, `DeleteBookmark`, `ListBookmarks`, `SearchBookmarks` |
|
|
95
|
+
| Test structure | Single `TestXxx` with `t.Run` subtests | Separate top-level test functions |
|
|
96
|
+
| `make([]T, 0, len)` usage | Yes (mem store candidates) | Yes (list handler, search handler) |
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## Overall Assessment
|
|
101
|
+
|
|
102
|
+
| Guideline | Effect | Evidence |
|
|
103
|
+
|---|---|---|
|
|
104
|
+
| **Simplicity First** | ✅ Strong positive | B builds incrementally; A front-loads the full store interface |
|
|
105
|
+
| **Surgical Changes** | ✅ Positive | B touches fewer files per task (1-4 vs 7 in Task 1) |
|
|
106
|
+
| **Think Before Coding** | ⚠️ Marginal | B made fewer silent assumptions but neither surfaced tradeoffs explicitly |
|
|
107
|
+
| **Goal-Driven Execution** | ≈ Neutral | Both strong; base skill already enforces acceptance criteria |
|
|
108
|
+
|
|
109
|
+
**Bottom line (iteration 1):** The guidelines measurably improved the plan. The biggest win is **Simplicity First** — Plan B's incremental interface growth (adding methods to `Store` as each task needs them) is clearly better than Plan A's upfront full-contract approach. This is exactly the kind of thing "no abstractions for single-use code" catches.
|
|
110
|
+
|
|
111
|
+
**Weakness:** Neither plan explicitly called out assumptions or asked clarifying questions — the "Think Before Coding" guideline had the weakest signal. The guidelines alone may not be enough to overcome the model's tendency to fill gaps silently.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## Iteration 2: Revised Guidelines
|
|
116
|
+
|
|
117
|
+
### What changed
|
|
118
|
+
|
|
119
|
+
The guidelines were reworked from 4 generic coding rules to 3 planning-specific principles:
|
|
120
|
+
|
|
121
|
+
| v1 (Generic) | v2 (Planning-Specific) | Why |
|
|
122
|
+
|---|---|---|
|
|
123
|
+
| Think Before Coding | **Surface Assumptions** | v1 said "ask" — the agent ignores this when told not to ask. v2 says "annotate in the plan" with a concrete `> **Assumption:** ...` format and examples of what to annotate. |
|
|
124
|
+
| Simplicity First | **Build Only What Each Task Needs** | Kept the same core principle but added the specific anti-pattern from the v1 A/B test: "don't define interface methods that no task exercises yet." |
|
|
125
|
+
| Surgical Changes | **One Task, One Change** | Reframed from "don't touch adjacent code" to "each task should trace to exactly one user-facing behavior" with a concrete guardrail (max 4 new files). |
|
|
126
|
+
| Goal-Driven Execution | *(removed)* | Redundant — the base skill already enforces Given/When/Then acceptance criteria. |
|
|
127
|
+
|
|
128
|
+
### Iteration 2 Plan (v2 guidelines) vs Iteration 1 Plans
|
|
129
|
+
|
|
130
|
+
| Dimension | A (No guidelines) | B1 (v1 guidelines) | B2 (v2 guidelines) |
|
|
131
|
+
|---|---|---|---|
|
|
132
|
+
| **Total tasks** | 4 | 6 | 4 |
|
|
133
|
+
| **Max files/task** | 7 (Task 1) | 4 | 4 |
|
|
134
|
+
| **Assumptions annotated** | 0 | 0 | **4** (header below) |
|
|
135
|
+
| **External deps** | None | `google/uuid` | None |
|
|
136
|
+
| **Store interface** | 4 methods upfront in Task 1 | 1 method per task | 1 method per task |
|
|
137
|
+
| **Service layer** | Yes (unjustified) | No | No |
|
|
138
|
+
|
|
139
|
+
### The big win: Surface Assumptions
|
|
140
|
+
|
|
141
|
+
Plan B2 opens with four explicit assumption annotations:
|
|
142
|
+
|
|
143
|
+
```
|
|
144
|
+
> **Assumption:** User identification via X-User-ID request header since
|
|
145
|
+
> no auth system exists in the project.
|
|
146
|
+
|
|
147
|
+
> **Assumption:** Bookmarks include a Note field so users can annotate
|
|
148
|
+
> bookmarks. The design says "search by keyword" but doesn't specify
|
|
149
|
+
> the field.
|
|
150
|
+
|
|
151
|
+
> **Assumption:** Offset/limit pagination (not cursor-based).
|
|
152
|
+
|
|
153
|
+
> **Assumption:** In-memory store behind a Store interface.
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
None of the previous plans (A or B1) did this. The v1 "Think Before Coding" guideline was completely invisible in output. The v2 "Surface Assumptions" guideline produced visible, reviewable annotations on the first run.
|
|
157
|
+
|
|
158
|
+
### Iteration 2 Assessment
|
|
159
|
+
|
|
160
|
+
| Guideline | v1 Effect | v2 Effect | Improvement |
|
|
161
|
+
|---|---|---|---|
|
|
162
|
+
| **Surface Assumptions** (was Think Before Coding) | ⚠️ Invisible | ✅ 4 explicit annotations | Complete turnaround — concrete format + examples fixed the weakest signal |
|
|
163
|
+
| **Build Only What's Needed** (was Simplicity First) | ✅ Strong | ✅ Strong | Maintained — interface still grows incrementally |
|
|
164
|
+
| **One Task, One Change** (was Surgical Changes) | ✅ Positive | ✅ Positive | Maintained — max 4 files/task |
|
|
165
|
+
|
|
166
|
+
**Bottom line (iteration 2):** The v2 guidelines fixed the weakest signal from v1. "Surface Assumptions" went from invisible to producing 4 explicit, reviewable annotations. The other two principles maintained their positive effect. The removal of "Goal-Driven Execution" (redundant) reduced noise without losing signal.
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# Add Verify Skill — Design Doc
|
|
2
|
+
|
|
3
|
+
## Context
|
|
4
|
+
|
|
5
|
+
Based on [Chris LeMa's "The Last Prompt"](https://chrislema.com/the-last-prompt-you-need-when-building-software-with-ai), we need a post-implementation code verification phase in pi-workflow-kit. The existing `design-review` skill validates architecture *intentions* at the design-doc level, but there's no review of the *actual implemented code*. This is where the most dangerous bugs hide: signature mismatches between layers, dead code, duplicated logic, and security holes that pass tests but break in production.
|
|
6
|
+
|
|
7
|
+
## Decision
|
|
8
|
+
|
|
9
|
+
### Add a `verify` skill (new)
|
|
10
|
+
|
|
11
|
+
A single skill triggered by `/skill:verify` that runs three sequential expert review passes over implemented code:
|
|
12
|
+
|
|
13
|
+
1. **Security** 🔴 — adversarial review as if a junior wrote it and the best security expert is auditing
|
|
14
|
+
2. **Optimization** 🟡 — dead code, duplication, over/under-engineering, performance
|
|
15
|
+
3. **Traceability** 🔵 — end-to-end call chain verification across every layer boundary
|
|
16
|
+
|
|
17
|
+
Output: structured markdown report at `docs/plans/*-verification-report.md` with findings and actionable task list.
|
|
18
|
+
|
|
19
|
+
### Keep `design-review` unchanged
|
|
20
|
+
|
|
21
|
+
`design-review` stays between brainstorm and plan — it validates architecture before task breakdown. Moving it would lose the cheap "catch it before you build it" value.
|
|
22
|
+
|
|
23
|
+
### Update README
|
|
24
|
+
|
|
25
|
+
Add `verify` to the workflow diagram, skill table, and quick start. The pipeline becomes:
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
brainstorm → design-review → plan → execute → verify → finalize
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
## Workflow Integration
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
brainstorm → design-review (optional) → plan → execute → verify → finalize
|
|
35
|
+
↑ ↑
|
|
36
|
+
existing new
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
- `verify` runs after `executing-tasks` and before `finalizing`
|
|
40
|
+
- It's optional — trivial changes can skip it
|
|
41
|
+
- The report's remediation task list feeds directly into a follow-up `/skill:writing-plans` if fixes are needed
|
|
42
|
+
- Read-only: can write to `docs/plans/` only, cannot modify source code
|
|
43
|
+
|
|
44
|
+
## Files to Change
|
|
45
|
+
|
|
46
|
+
1. **`skills/verify/SKILL.md`** — new skill (full content in `docs/plans/2026-06-03-verify-skill-design.md`)
|
|
47
|
+
2. **`README.md`** — update workflow diagram, skill table, quick start, and project structure
|
|
48
|
+
|
|
49
|
+
## Production Risks
|
|
50
|
+
|
|
51
|
+
Simple change — no design review needed. We're adding a new SKILL.md and updating documentation. No code execution, no external integrations, no security surface.
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Implementation Plan: Add Verify Skill
|
|
2
|
+
|
|
3
|
+
Design: `docs/plans/2026-06-03-add-verify-skill-design.md`
|
|
4
|
+
|
|
5
|
+
## Overview
|
|
6
|
+
|
|
7
|
+
Add a `verify` skill to pi-workflow-kit — a post-implementation code verification phase that runs three expert review passes (security, optimization, traceability) over implemented code. Also update the README to reflect the expanded workflow pipeline.
|
|
8
|
+
|
|
9
|
+
Full SKILL.md content is in `docs/plans/2026-06-03-verify-skill-design.md` (lines 7-176, inside the code fence).
|
|
10
|
+
|
|
11
|
+
## Task 1: Create the verify skill
|
|
12
|
+
|
|
13
|
+
<!-- tdd: trivial -->
|
|
14
|
+
|
|
15
|
+
Acceptance Criteria (QA Engineer Hat):
|
|
16
|
+
- **Happy Path**:
|
|
17
|
+
- Given: No `skills/verify/` directory exists
|
|
18
|
+
- When: `skills/verify/SKILL.md` is created
|
|
19
|
+
- Then: The file contains valid YAML frontmatter with `name: verify` and a description mentioning security, optimization, and traceability. The file body contains all three review pass sections, the report format template, and the principles section.
|
|
20
|
+
- **Edge Case (skill already exists)**:
|
|
21
|
+
- Given: `skills/verify/SKILL.md` already exists
|
|
22
|
+
- When: Task runs
|
|
23
|
+
- Then: The existing file is overwritten with the new content
|
|
24
|
+
|
|
25
|
+
Files:
|
|
26
|
+
- `skills/verify/SKILL.md`
|
|
27
|
+
|
|
28
|
+
Steps:
|
|
29
|
+
1. Create the directory `skills/verify/`
|
|
30
|
+
2. Create `skills/verify/SKILL.md` with the full content from the design draft. The content is the markdown inside the code fence in `docs/plans/2026-06-03-verify-skill-design.md` (lines 8-176). Copy it exactly — it includes:
|
|
31
|
+
- YAML frontmatter with name and description
|
|
32
|
+
- # Verify heading and intro paragraph
|
|
33
|
+
- ## Process section (5 steps)
|
|
34
|
+
- ## Pass 1 — Security Review 🔴 (framing, what to look for, severity table)
|
|
35
|
+
- ## Pass 2 — Optimization Review 🟡 (framing, what to look for, priority table)
|
|
36
|
+
- ## Pass 3 — Traceability Review 🔵 (framing, what to look for 4 sub-items, severity table)
|
|
37
|
+
- ## Report Format section (full template with summary table, findings sections, remediation task list)
|
|
38
|
+
- ## Principles section (5 bullets)
|
|
39
|
+
|
|
40
|
+
## Task 2: Update README with verify skill
|
|
41
|
+
|
|
42
|
+
<!-- tdd: trivial -->
|
|
43
|
+
|
|
44
|
+
Acceptance Criteria (QA Engineer Hat):
|
|
45
|
+
- **Happy Path**:
|
|
46
|
+
- Given: README.md has the current workflow (brainstorm → design-review → plan → execute → finalize)
|
|
47
|
+
- When: README is updated
|
|
48
|
+
- Then: All five sections are updated — tagline, workflow diagram, skill table, phase control, quick start, and project structure — to include `verify` between execute and finalize.
|
|
49
|
+
- **Edge Case (verify already in README)**:
|
|
50
|
+
- Given: README already contains verify references
|
|
51
|
+
- When: Task runs
|
|
52
|
+
- Then: No duplicate entries are introduced
|
|
53
|
+
|
|
54
|
+
Files:
|
|
55
|
+
- `README.md`
|
|
56
|
+
|
|
57
|
+
Steps:
|
|
58
|
+
|
|
59
|
+
1. Update the tagline (line 3) — change `brainstorm→plan→execute→finalize` to `brainstorm→plan→execute→verify→finalize`:
|
|
60
|
+
```
|
|
61
|
+
> Stop AI agents from rushing to code. Enforce a structured brainstorm→plan→execute→verify→finalize workflow with TDD discipline.
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
2. Update the "🧠 6 Workflow Skills" heading (line 36) to "🧠 7 Workflow Skills"
|
|
65
|
+
|
|
66
|
+
3. Update the workflow diagram (lines 40-44) to:
|
|
67
|
+
```
|
|
68
|
+
brainstorm → design-review → plan → execute → verify → finalize
|
|
69
|
+
↕
|
|
70
|
+
diagnose (anytime)
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
4. Add verify to the skill table (after the Execute row, before Finalize):
|
|
74
|
+
```
|
|
75
|
+
| **Verify** | `/skill:verify` | Three expert review passes (security, optimization, traceability) on implemented code |
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
5. Update the phase control section (lines 61-67) to add verify:
|
|
79
|
+
```
|
|
80
|
+
/skill:brainstorming → discuss and design
|
|
81
|
+
/skill:design-review → audit for production risks (non-trivial designs)
|
|
82
|
+
/skill:writing-plans → break into tasks
|
|
83
|
+
/skill:executing-tasks → implement with TDD
|
|
84
|
+
/skill:verify → review code for security, optimization, and traceability issues
|
|
85
|
+
/skill:finalizing → ship it
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
6. Update the quick start section (lines 110-135) to add verify between executing-tasks and finalizing:
|
|
89
|
+
```
|
|
90
|
+
> /skill:executing-tasks
|
|
91
|
+
|
|
92
|
+
# (agent implements with TDD, cognitive persona shifts, all tools unlocked)
|
|
93
|
+
> /skill:verify
|
|
94
|
+
|
|
95
|
+
# (agent runs security, optimization, and traceability reviews on implemented code)
|
|
96
|
+
> /skill:finalizing
|
|
97
|
+
|
|
98
|
+
# (agent archives docs, curates lessons, creates PR)
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
7. Update the project structure (lines 146-161) to add verify:
|
|
102
|
+
```
|
|
103
|
+
├── skills/
|
|
104
|
+
│ ├── brainstorming/SKILL.md
|
|
105
|
+
│ ├── design-review/SKILL.md
|
|
106
|
+
│ ├── writing-plans/SKILL.md
|
|
107
|
+
│ ├── executing-tasks/SKILL.md
|
|
108
|
+
│ ├── verify/SKILL.md
|
|
109
|
+
│ ├── finalizing/SKILL.md
|
|
110
|
+
│ └── diagnose/SKILL.md
|
|
111
|
+
```
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# Progress: Add Verify Skill
|
|
2
|
+
|
|
3
|
+
Plan: docs/plans/2026-06-03-add-verify-skill-implementation.md
|
|
4
|
+
Branch: add-verify-skill
|
|
5
|
+
Started: 2026-06-03T13:00:00Z
|
|
6
|
+
Last updated: 2026-06-03T13:00:00Z
|
|
7
|
+
|
|
8
|
+
| # | Status | Task | Commit |
|
|
9
|
+
|---|--------|------|--------|
|
|
10
|
+
| 1 | ✅ done | Create the verify skill | c48d47a |
|
|
11
|
+
| 2 | ✅ done | Update README with verify skill | ea37ea8 |
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
# Verify Skill — Draft SKILL.md
|
|
2
|
+
|
|
3
|
+
> **Target path:** `skills/verify/SKILL.md` (to be created during executing-tasks)
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
```markdown
|
|
8
|
+
---
|
|
9
|
+
name: verify
|
|
10
|
+
description: "Post-implementation code verification with three expert review passes — security, optimization, and traceability. Use after executing-tasks and before finalizing to catch issues that pass tests but break in production. Runs the 'last prompt' pattern: adversarial security review, dead code and duplication audit, and end-to-end contract verification across every layer. Use this skill whenever the user says 'verify', 'review the code', 'check for issues', 'security review', 'the last prompt', 'audit', or when code has been implemented and needs a quality gate before shipping."
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
# Verify
|
|
14
|
+
|
|
15
|
+
Three expert review passes over the implemented codebase. Read-only — you **may** write the verification report to `docs/plans/`, but you **may not** modify source code.
|
|
16
|
+
|
|
17
|
+
The core insight: code that passes tests is not code that's ready. Working code can have security holes, dead branches, duplicated logic, and broken contracts between layers — especially when AI generates across many files without maintaining a single mental model of the whole system. This skill catches what tests miss.
|
|
18
|
+
|
|
19
|
+
## Process
|
|
20
|
+
|
|
21
|
+
1. **Check what's been done** — run `git log --oneline` and `git diff --stat` to understand the scope of recent changes. If nothing has been implemented, say "No code changes found. Run `/skill:executing-tasks` first." and stop.
|
|
22
|
+
|
|
23
|
+
2. **Identify the project's layers** — before reviewing, map the codebase's architecture. Look for layer boundaries: UI/handlers/routes → services/business logic → repositories/data access → database/models. Note the patterns: does the project use controllers, handlers, or routes? Services or use cases? Repositories or DAOs? This map drives the traceability pass.
|
|
24
|
+
|
|
25
|
+
3. **Run three expert review passes** — each pass adopts a distinct adversarial framing. Do them sequentially. For each pass, read the relevant code deeply — don't skim. Then write findings.
|
|
26
|
+
|
|
27
|
+
4. **Compile the report** — write all findings to `docs/plans/*-verification-report.md`. Present the report to the user and wait for feedback.
|
|
28
|
+
|
|
29
|
+
5. **Offer to create a remediation plan** — after the report, ask: "Want me to create a fix plan from these findings? Run `/skill:writing-plans` to turn the task list into executable tasks."
|
|
30
|
+
|
|
31
|
+
## Pass 1 — Security Review 🔴
|
|
32
|
+
|
|
33
|
+
**Framing:** A junior developer wrote this code. Now the best security expert on the team is reviewing it — adversarial, suspicious of everything. Trust nothing.
|
|
34
|
+
|
|
35
|
+
**What to look for:**
|
|
36
|
+
|
|
37
|
+
- **Input validation** — every external input (HTTP params, form data, headers, query strings, environment variables) must be validated and sanitized. Unvalidated input is a critical finding.
|
|
38
|
+
- **Authentication & authorization** — every endpoint that handles user data must have auth checks. Are there endpoints that skip auth? Can one user access another user's data by changing an ID?
|
|
39
|
+
- **Injection** — SQL queries built by string concatenation, unsanitized shell commands, template injection, XSS in HTML output. Any raw variable interpolated into a query or command is critical.
|
|
40
|
+
- **Secrets** — API keys, passwords, tokens hardcoded in source files. Check environment variable loading — are defaults set to empty or to actual secrets?
|
|
41
|
+
- **Data exposure** — are sensitive fields (passwords, tokens, PII) logged, returned in API responses, or stored unencrypted?
|
|
42
|
+
- **Dependency risks** — known-vulnerable packages (if `package.json`/`go.mod`/`requirements.txt` is present).
|
|
43
|
+
|
|
44
|
+
**Severity classification:**
|
|
45
|
+
|
|
46
|
+
| Severity | Definition |
|
|
47
|
+
|----------|-----------|
|
|
48
|
+
| Critical | Exploitable right now — auth bypass, injection, data leak |
|
|
49
|
+
| High | Likely exploitable — missing validation on sensitive endpoint, weak auth |
|
|
50
|
+
| Medium | Harder to exploit but real risk — verbose error messages leaking internals, missing rate limits |
|
|
51
|
+
| Low | Best practice violations — missing CSP headers, no HSTS, long session timeouts |
|
|
52
|
+
|
|
53
|
+
## Pass 2 — Optimization Review 🟡
|
|
54
|
+
|
|
55
|
+
**Framing:** A code quality expert looking for waste — things that make the codebase harder to maintain, slower to run, or more confusing than necessary.
|
|
56
|
+
|
|
57
|
+
**What to look for:**
|
|
58
|
+
|
|
59
|
+
- **Dead code** — functions, methods, types, or exports that are never called anywhere in the codebase. Search for definitions and verify they have callers.
|
|
60
|
+
- **Duplication** — the same logic implemented in slightly different ways across multiple files. AI-generated code is especially prone to this — if context was lost between sessions, the AI solved the same sub-problem differently in two places. Flag each pair with file paths and line numbers.
|
|
61
|
+
- **Over-engineering** — abstractions, interfaces, or layers that add complexity without earning their keep (only one implementation, no real variation across the seam).
|
|
62
|
+
- **Under-engineering** — god functions, 200-line blocks, deeply nested conditionals that should be extracted.
|
|
63
|
+
- **Performance concerns** — N+1 queries, unbounded loops, unnecessary copies of large data structures, missing pagination on list endpoints.
|
|
64
|
+
|
|
65
|
+
**Priority classification:**
|
|
66
|
+
|
|
67
|
+
| Priority | Definition |
|
|
68
|
+
|----------|-----------|
|
|
69
|
+
| P0 | Dead code in a critical path or duplicated logic that will diverge |
|
|
70
|
+
| P1 | Significant duplication or over-engineering that increases maintenance cost |
|
|
71
|
+
| P2 | Minor cleanups — long functions, missing pagination, style inconsistencies |
|
|
72
|
+
|
|
73
|
+
## Pass 3 — Traceability Review 🔵
|
|
74
|
+
|
|
75
|
+
**Framing:** An integration expert tracing every user-facing action end-to-end — from UI to database and back. The AI generates code file-by-file, and the seams between files are where bugs hide.
|
|
76
|
+
|
|
77
|
+
**What to look for:**
|
|
78
|
+
|
|
79
|
+
1. **Map every entry point** — list all handlers, routes, controllers, or event listeners that receive external input.
|
|
80
|
+
2. **Trace each call chain** — for each entry point, follow the call: handler → service → repository → database. At each boundary, verify:
|
|
81
|
+
- **Function name** — does the caller use the exact function name the callee exposes?
|
|
82
|
+
- **Argument names** — does the caller pass `userId` when the function expects `user_id`? Does `id` mean the same thing in both layers?
|
|
83
|
+
- **Argument types** — is a string passed where an integer is expected? Is an object shape different from what the next layer destructures?
|
|
84
|
+
- **Return shape** — does the caller expect fields that the callee actually returns? Are response DTOs consistent across layers?
|
|
85
|
+
3. **Check error propagation** — when a database query returns no results, does the service layer handle it? Does the handler return 404 or 500? Do errors propagate cleanly or get swallowed silently?
|
|
86
|
+
4. **Verify the round-trip** — if the UI calls `getUser(id)` and displays `user.name`, trace that `name` actually exists in the DB schema, gets selected by the query, mapped by the repository, passed through the service, included in the response, and rendered by the UI.
|
|
87
|
+
|
|
88
|
+
**This is the pass that catches the most bugs.** AI-generated code will often have a frontend calling `getUserProfile(userId)` and a backend exposing `get_user_profile(user_id)` — both work in isolation, neither works together.
|
|
89
|
+
|
|
90
|
+
**Severity classification:**
|
|
91
|
+
|
|
92
|
+
| Severity | Definition |
|
|
93
|
+
|----------|-----------|
|
|
94
|
+
| Critical | Call chain is completely broken — function doesn't exist or signature is fundamentally wrong |
|
|
95
|
+
| High | Signature mismatch — wrong arg names, wrong types, missing required fields |
|
|
96
|
+
| Medium | Silent error handling — errors swallowed without logging or user feedback |
|
|
97
|
+
| Low | Inconsistent naming conventions that could confuse future developers |
|
|
98
|
+
|
|
99
|
+
## Report Format
|
|
100
|
+
|
|
101
|
+
Write findings to `docs/plans/*-verification-report.md` using this structure:
|
|
102
|
+
|
|
103
|
+
# Verification Report: <feature/topic>
|
|
104
|
+
|
|
105
|
+
**Date:** <ISO date>
|
|
106
|
+
**Scope:** <summary of what was reviewed>
|
|
107
|
+
**Reviewer:** AI verify skill (security + optimization + traceability)
|
|
108
|
+
|
|
109
|
+
## Summary
|
|
110
|
+
|
|
111
|
+
| Pass | Critical | High | Medium | Low |
|
|
112
|
+
|------|----------|------|--------|-----|
|
|
113
|
+
| Security | X | X | X | X |
|
|
114
|
+
| Optimization | — | X | X | X |
|
|
115
|
+
| Traceability | X | X | X | X |
|
|
116
|
+
| **Total** | **X** | **X** | **X** | **X** |
|
|
117
|
+
|
|
118
|
+
## 🔴 Security Findings
|
|
119
|
+
|
|
120
|
+
### [S-001] Critical — <short title>
|
|
121
|
+
|
|
122
|
+
**Location:** `path/to/file.ts:line`
|
|
123
|
+
|
|
124
|
+
**Issue:** <what's wrong and why it matters>
|
|
125
|
+
|
|
126
|
+
**Fix:** <concrete remediation step>
|
|
127
|
+
|
|
128
|
+
### [S-002] High — <short title>
|
|
129
|
+
...
|
|
130
|
+
|
|
131
|
+
## 🟡 Optimization Findings
|
|
132
|
+
|
|
133
|
+
### [O-001] P0 — <short title>
|
|
134
|
+
|
|
135
|
+
**Location:** `path/to/file.ts:line` and `path/to/other.ts:line`
|
|
136
|
+
|
|
137
|
+
**Issue:** <what's wrong>
|
|
138
|
+
|
|
139
|
+
**Fix:** <concrete remediation step>
|
|
140
|
+
|
|
141
|
+
### [O-002] P1 — <short title>
|
|
142
|
+
...
|
|
143
|
+
|
|
144
|
+
## 🔵 Traceability Findings
|
|
145
|
+
|
|
146
|
+
### [T-001] Critical — <short title>
|
|
147
|
+
|
|
148
|
+
**Entry point:** `path/to/handler.ts:line`
|
|
149
|
+
**Call chain:** handler → service → repository → DB
|
|
150
|
+
**Broken at:** <which boundary>
|
|
151
|
+
**Issue:** <what's wrong — e.g., handler passes `userId` but service expects `user_id`>
|
|
152
|
+
|
|
153
|
+
**Fix:** <concrete remediation step>
|
|
154
|
+
|
|
155
|
+
### [T-002] High — <short title>
|
|
156
|
+
...
|
|
157
|
+
|
|
158
|
+
## Remediation Task List
|
|
159
|
+
|
|
160
|
+
Convert findings into actionable tasks:
|
|
161
|
+
|
|
162
|
+
| ID | Priority | Finding | Estimated Effort |
|
|
163
|
+
|----|----------|---------|-----------------|
|
|
164
|
+
| S-001 | Critical | <one-liner> | <small/medium/large> |
|
|
165
|
+
| T-001 | Critical | <one-liner> | <small/medium/large> |
|
|
166
|
+
| O-001 | P0 | <one-liner> | <small/medium/large> |
|
|
167
|
+
| ...
|
|
168
|
+
|
|
169
|
+
## Principles
|
|
170
|
+
|
|
171
|
+
- **Be specific** — every finding must include a file path and line reference. "There might be security issues" is useless.
|
|
172
|
+
- **Be adversarial** — actively look for problems. If you don't find any, say so — but don't phone it in.
|
|
173
|
+
- **Be proportional** — a small config change doesn't need the same depth as a new API endpoint. Adjust your review depth to the scope of changes.
|
|
174
|
+
- **Don't fix anything** — this is read-only. Find and report. The user decides what to fix and when.
|
|
175
|
+
- **Focus on seams** — the traceability pass is where the most value lives. Code within a single file is usually coherent; the bugs hide between files.
|
|
176
|
+
```
|
package/package.json
CHANGED
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: verify
|
|
3
|
+
description: "Post-implementation code verification with three expert review passes — security, optimization, and traceability. Use after executing-tasks and before finalizing to catch issues that pass tests but break in production. Runs the 'last prompt' pattern: adversarial security review, dead code and duplication audit, and end-to-end contract verification across every layer. Use this skill whenever the user says 'verify', 'review the code', 'check for issues', 'security review', 'the last prompt', 'audit', or when code has been implemented and needs a quality gate before shipping."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Verify
|
|
7
|
+
|
|
8
|
+
Three expert review passes over the implemented codebase. Read-only — you **may** write the verification report to `docs/plans/`, but you **may not** modify source code.
|
|
9
|
+
|
|
10
|
+
The core insight: code that passes tests is not code that's ready. Working code can have security holes, dead branches, duplicated logic, and broken contracts between layers — especially when AI generates across many files without maintaining a single mental model of the whole system. This skill catches what tests miss.
|
|
11
|
+
|
|
12
|
+
## Process
|
|
13
|
+
|
|
14
|
+
1. **Check what's been done** — run `git log --oneline` and `git diff --stat` to understand the scope of recent changes. If nothing has been implemented, say "No code changes found. Run `/skill:executing-tasks` first." and stop.
|
|
15
|
+
|
|
16
|
+
2. **Identify the project's layers** — before reviewing, map the codebase's architecture. Look for layer boundaries: UI/handlers/routes → services/business logic → repositories/data access → database/models. Note the patterns: does the project use controllers, handlers, or routes? Services or use cases? Repositories or DAOs? This map drives the traceability pass.
|
|
17
|
+
|
|
18
|
+
3. **Run three expert review passes** — each pass adopts a distinct adversarial framing. Do them sequentially. For each pass, read the relevant code deeply — don't skim. Then write findings.
|
|
19
|
+
|
|
20
|
+
4. **Compile the report** — write all findings to `docs/plans/*-verification-report.md`. Present the report to the user and wait for feedback.
|
|
21
|
+
|
|
22
|
+
5. **Offer to create a remediation plan** — after the report, ask: "Want me to create a fix plan from these findings? Run `/skill:writing-plans` to turn the task list into executable tasks."
|
|
23
|
+
|
|
24
|
+
## Pass 1 — Security Review 🔴
|
|
25
|
+
|
|
26
|
+
**Framing:** A junior developer wrote this code. Now the best security expert on the team is reviewing it — adversarial, suspicious of everything. Trust nothing.
|
|
27
|
+
|
|
28
|
+
**What to look for:**
|
|
29
|
+
|
|
30
|
+
- **Input validation** — every external input (HTTP params, form data, headers, query strings, environment variables) must be validated and sanitized. Unvalidated input is a critical finding.
|
|
31
|
+
- **Authentication & authorization** — every endpoint that handles user data must have auth checks. Are there endpoints that skip auth? Can one user access another user's data by changing an ID?
|
|
32
|
+
- **Injection** — SQL queries built by string concatenation, unsanitized shell commands, template injection, XSS in HTML output. Any raw variable interpolated into a query or command is critical.
|
|
33
|
+
- **Secrets** — API keys, passwords, tokens hardcoded in source files. Check environment variable loading — are defaults set to empty or to actual secrets?
|
|
34
|
+
- **Data exposure** — are sensitive fields (passwords, tokens, PII) logged, returned in API responses, or stored unencrypted?
|
|
35
|
+
- **Dependency risks** — known-vulnerable packages (if `package.json`/`go.mod`/`requirements.txt` is present).
|
|
36
|
+
|
|
37
|
+
**Severity classification:**
|
|
38
|
+
|
|
39
|
+
| Severity | Definition |
|
|
40
|
+
|----------|-----------|
|
|
41
|
+
| Critical | Exploitable right now — auth bypass, injection, data leak |
|
|
42
|
+
| High | Likely exploitable — missing validation on sensitive endpoint, weak auth |
|
|
43
|
+
| Medium | Harder to exploit but real risk — verbose error messages leaking internals, missing rate limits |
|
|
44
|
+
| Low | Best practice violations — missing CSP headers, no HSTS, long session timeouts |
|
|
45
|
+
|
|
46
|
+
## Pass 2 — Optimization Review 🟡
|
|
47
|
+
|
|
48
|
+
**Framing:** A code quality expert looking for waste — things that make the codebase harder to maintain, slower to run, or more confusing than necessary.
|
|
49
|
+
|
|
50
|
+
**What to look for:**
|
|
51
|
+
|
|
52
|
+
- **Dead code** — functions, methods, types, or exports that are never called anywhere in the codebase. Search for definitions and verify they have callers.
|
|
53
|
+
- **Duplication** — the same logic implemented in slightly different ways across multiple files. AI-generated code is especially prone to this — if context was lost between sessions, the AI solved the same sub-problem differently in two places. Flag each pair with file paths and line numbers.
|
|
54
|
+
- **Over-engineering** — abstractions, interfaces, or layers that add complexity without earning their keep (only one implementation, no real variation across the seam).
|
|
55
|
+
- **Under-engineering** — god functions, 200-line blocks, deeply nested conditionals that should be extracted.
|
|
56
|
+
- **Performance concerns** — N+1 queries, unbounded loops, unnecessary copies of large data structures, missing pagination on list endpoints.
|
|
57
|
+
|
|
58
|
+
**Priority classification:**
|
|
59
|
+
|
|
60
|
+
| Priority | Definition |
|
|
61
|
+
|----------|-----------|
|
|
62
|
+
| P0 | Dead code in a critical path or duplicated logic that will diverge |
|
|
63
|
+
| P1 | Significant duplication or over-engineering that increases maintenance cost |
|
|
64
|
+
| P2 | Minor cleanups — long functions, missing pagination, style inconsistencies |
|
|
65
|
+
|
|
66
|
+
## Pass 3 — Traceability Review 🔵
|
|
67
|
+
|
|
68
|
+
**Framing:** An integration expert tracing every user-facing action end-to-end — from UI to database and back. The AI generates code file-by-file, and the seams between files are where bugs hide.
|
|
69
|
+
|
|
70
|
+
**What to look for:**
|
|
71
|
+
|
|
72
|
+
1. **Map every entry point** — list all handlers, routes, controllers, or event listeners that receive external input.
|
|
73
|
+
2. **Trace each call chain** — for each entry point, follow the call: handler → service → repository → database. At each boundary, verify:
|
|
74
|
+
- **Function name** — does the caller use the exact function name the callee exposes?
|
|
75
|
+
- **Argument names** — does the caller pass `userId` when the function expects `user_id`? Does `id` mean the same thing in both layers?
|
|
76
|
+
- **Argument types** — is a string passed where an integer is expected? Is an object shape different from what the next layer destructures?
|
|
77
|
+
- **Return shape** — does the caller expect fields that the callee actually returns? Are response DTOs consistent across layers?
|
|
78
|
+
3. **Check error propagation** — when a database query returns no results, does the service layer handle it? Does the handler return 404 or 500? Do errors propagate cleanly or get swallowed silently?
|
|
79
|
+
4. **Verify the round-trip** — if the UI calls `getUser(id)` and displays `user.name`, trace that `name` actually exists in the DB schema, gets selected by the query, mapped by the repository, passed through the service, included in the response, and rendered by the UI.
|
|
80
|
+
|
|
81
|
+
**This is the pass that catches the most bugs.** AI-generated code will often have a frontend calling `getUserProfile(userId)` and a backend exposing `get_user_profile(user_id)` — both work in isolation, neither works together.
|
|
82
|
+
|
|
83
|
+
**Severity classification:**
|
|
84
|
+
|
|
85
|
+
| Severity | Definition |
|
|
86
|
+
|----------|-----------|
|
|
87
|
+
| Critical | Call chain is completely broken — function doesn't exist or signature is fundamentally wrong |
|
|
88
|
+
| High | Signature mismatch — wrong arg names, wrong types, missing required fields |
|
|
89
|
+
| Medium | Silent error handling — errors swallowed without logging or user feedback |
|
|
90
|
+
| Low | Inconsistent naming conventions that could confuse future developers |
|
|
91
|
+
|
|
92
|
+
## Report Format
|
|
93
|
+
|
|
94
|
+
Write findings to `docs/plans/*-verification-report.md` using this structure:
|
|
95
|
+
|
|
96
|
+
```markdown
|
|
97
|
+
# Verification Report: <feature/topic>
|
|
98
|
+
|
|
99
|
+
**Date:** <ISO date>
|
|
100
|
+
**Scope:** <summary of what was reviewed>
|
|
101
|
+
**Reviewer:** AI verify skill (security + optimization + traceability)
|
|
102
|
+
|
|
103
|
+
## Summary
|
|
104
|
+
|
|
105
|
+
| Pass | Critical | High | Medium | Low |
|
|
106
|
+
|------|----------|------|--------|-----|
|
|
107
|
+
| Security | X | X | X | X |
|
|
108
|
+
| Optimization | — | X | X | X |
|
|
109
|
+
| Traceability | X | X | X | X |
|
|
110
|
+
| **Total** | **X** | **X** | **X** | **X** |
|
|
111
|
+
|
|
112
|
+
## 🔴 Security Findings
|
|
113
|
+
|
|
114
|
+
### [S-001] Critical — <short title>
|
|
115
|
+
|
|
116
|
+
**Location:** `path/to/file.ts:line`
|
|
117
|
+
|
|
118
|
+
**Issue:** <what's wrong and why it matters>
|
|
119
|
+
|
|
120
|
+
**Fix:** <concrete remediation step>
|
|
121
|
+
|
|
122
|
+
### [S-002] High — <short title>
|
|
123
|
+
...
|
|
124
|
+
|
|
125
|
+
## 🟡 Optimization Findings
|
|
126
|
+
|
|
127
|
+
### [O-001] P0 — <short title>
|
|
128
|
+
|
|
129
|
+
**Location:** `path/to/file.ts:line` and `path/to/other.ts:line`
|
|
130
|
+
|
|
131
|
+
**Issue:** <what's wrong>
|
|
132
|
+
|
|
133
|
+
**Fix:** <concrete remediation step>
|
|
134
|
+
|
|
135
|
+
### [O-002] P1 — <short title>
|
|
136
|
+
...
|
|
137
|
+
|
|
138
|
+
## 🔵 Traceability Findings
|
|
139
|
+
|
|
140
|
+
### [T-001] Critical — <short title>
|
|
141
|
+
|
|
142
|
+
**Entry point:** `path/to/handler.ts:line`
|
|
143
|
+
**Call chain:** handler → service → repository → DB
|
|
144
|
+
**Broken at:** <which boundary>
|
|
145
|
+
**Issue:** <what's wrong — e.g., handler passes `userId` but service expects `user_id`>
|
|
146
|
+
|
|
147
|
+
**Fix:** <concrete remediation step>
|
|
148
|
+
|
|
149
|
+
### [T-002] High — <short title>
|
|
150
|
+
...
|
|
151
|
+
|
|
152
|
+
## Remediation Task List
|
|
153
|
+
|
|
154
|
+
Convert findings into actionable tasks:
|
|
155
|
+
|
|
156
|
+
| ID | Priority | Finding | Estimated Effort |
|
|
157
|
+
|----|----------|---------|-----------------|
|
|
158
|
+
| S-001 | Critical | <one-liner> | <small/medium/large> |
|
|
159
|
+
| T-001 | Critical | <one-liner> | <small/medium/large> |
|
|
160
|
+
| O-001 | P0 | <one-liner> | <small/medium/large> |
|
|
161
|
+
| ...
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
## Principles
|
|
165
|
+
|
|
166
|
+
- **Be specific** — every finding must include a file path and line reference. "There might be security issues" is useless.
|
|
167
|
+
- **Be adversarial** — actively look for problems. If you don't find any, say so — but don't phone it in.
|
|
168
|
+
- **Be proportional** — a small config change doesn't need the same depth as a new API endpoint. Adjust your review depth to the scope of changes.
|
|
169
|
+
- **Don't fix anything** — this is read-only. Find and report. The user decides what to fix and when.
|
|
170
|
+
- **Focus on seams** — the traceability pass is where the most value lives. Code within a single file is usually coherent; the bugs hide between files.
|
|
@@ -290,3 +290,54 @@ Use judgment when assigning checkpoints. Prefer `checkpoint: test` for new featu
|
|
|
290
290
|
## After the plan
|
|
291
291
|
|
|
292
292
|
Ask: "Ready to execute? Run `/skill:executing-tasks`"
|
|
293
|
+
|
|
294
|
+
## Behavioral Guidelines
|
|
295
|
+
|
|
296
|
+
Guidelines to reduce overcomplication and hidden assumptions in plans. Derived from [Andrej Karpathy's observations](https://x.com/karpathy/status/2015883857489522876) on LLM coding pitfalls, adapted for the planning context.
|
|
297
|
+
|
|
298
|
+
**Tradeoff:** These guidelines bias toward caution over speed. For trivial plans (1-2 tasks), use judgment.
|
|
299
|
+
|
|
300
|
+
### Surface Assumptions
|
|
301
|
+
|
|
302
|
+
**When the design is ambiguous, annotate — don't silently pick.**
|
|
303
|
+
|
|
304
|
+
When writing a plan, you'll encounter gaps: the design says "paginated" but doesn't specify how, says "validate input" but doesn't say which fields, or leaves the data layer unspecified. Your instinct will be to fill the gap and keep writing. Resist that.
|
|
305
|
+
|
|
306
|
+
Instead, add a brief `> **Assumption:** ...` note in the plan at the point where you made the call:
|
|
307
|
+
|
|
308
|
+
```
|
|
309
|
+
> **Assumption:** Using offset/limit pagination because the design just says
|
|
310
|
+
> "paginated". Cursor-based would be better for large datasets.
|
|
311
|
+
```
|
|
312
|
+
|
|
313
|
+
```
|
|
314
|
+
> **Assumption:** No service layer — handler calls store directly. Add one
|
|
315
|
+
> if cross-cutting concerns (logging, auth checks) emerge later.
|
|
316
|
+
```
|
|
317
|
+
|
|
318
|
+
This lets the reviewer see what you chose and why, without blocking progress. Common gaps worth annotating:
|
|
319
|
+
- Pagination style, error handling strategy, concurrency model
|
|
320
|
+
- Whether to add a service/middleware layer
|
|
321
|
+
- Whether to add external dependencies
|
|
322
|
+
- Naming conventions when the design doesn't specify
|
|
323
|
+
|
|
324
|
+
### Build Only What Each Task Needs
|
|
325
|
+
|
|
326
|
+
**Minimum code to deliver the task's observable behavior. Nothing more.**
|
|
327
|
+
|
|
328
|
+
- No interface methods that no task exercises yet. If Task 2 creates a `Store` interface, it should have only the methods Task 2 calls. Add methods in the task that first needs them.
|
|
329
|
+
- No layers (service, middleware, repository) unless the design explicitly requires them.
|
|
330
|
+
- No error types, helper files, or shared packages until a task actually uses them.
|
|
331
|
+
- No external dependencies when stdlib suffices. Every `go get` or `npm install` is a choice — default to no.
|
|
332
|
+
- No "flexible" or "configurable" code that wasn't requested.
|
|
333
|
+
|
|
334
|
+
If you find yourself writing a store with 4 methods where only 1 is used in this task, stop. Write 1 method. Add the rest when the tasks that need them arrive.
|
|
335
|
+
|
|
336
|
+
### One Task, One Change
|
|
337
|
+
|
|
338
|
+
**Each task should trace to exactly one user-facing behavior.**
|
|
339
|
+
|
|
340
|
+
- If a task creates more than 4 new files, it's probably doing too much — split it.
|
|
341
|
+
- If a task modifies existing files unrelated to its acceptance criteria, trim the scope.
|
|
342
|
+
- Infrastructure (types, interfaces, module scaffolding) should live in the same task as the first code that uses it, not in a separate "setup" task — unless the infrastructure alone is complex enough to warrant its own task.
|
|
343
|
+
- Every file listed in a task's `Files:` section should be directly necessary for that task's acceptance criteria to pass.
|