@sienklogic/plan-build-run 2.21.0 → 2.21.2

package/plugins/pbr/references/verification-patterns.md

@@ -1,198 +1,614 @@
-# Goal-Backward Verification Patterns
+# Verification Patterns
 
-Reference patterns for deriving verification criteria from goals. Used by the planner to create `<verify>` and `<done>` elements, and by the verifier to check phase completion.
+How to verify different types of artifacts are real implementations, not stubs or placeholders.
 
----
+<core_principle>
+**Existence != Implementation**
 
-## The Three-Layer Check
+A file existing does not mean the feature works. Verification must check:
+1. **Exists** - File is present at expected path
+2. **Substantive** - Content is real implementation, not placeholder
+3. **Wired** - Connected to the rest of the system
+4. **Functional** - Actually works when invoked
 
-Every must-have is verified through three layers, checked in order:
+Levels 1-3 can be checked programmatically. Level 4 often requires human verification.
+</core_principle>
 
-### Layer 1: Existence
+<stub_detection>
 
-Does the artifact exist?
+<!-- stub detection patterns -->
+## Universal Stub Patterns
 
+These patterns indicate placeholder code regardless of file type:
+
+**Comment-based stubs:**
 ```bash
-# File existence
-ls -la src/auth/discord.ts
+# Grep patterns for stub comments
+grep -E "(TODO|FIXME|XXX|HACK|PLACEHOLDER)" "$file"
+grep -E "implement|add later|coming soon|will be" "$file" -i
+grep -E "// \.\.\.|/\* \.\.\. \*/|# \.\.\." "$file"
+```
 
-# Module export existence
-grep -q "export.*authenticateWithDiscord" src/auth/discord.ts
+**Placeholder text in output:**
+```bash
+# UI placeholder patterns
+grep -E "placeholder|lorem ipsum|coming soon|under construction" "$file" -i
+grep -E "sample|example|test data|dummy" "$file" -i
+grep -E "\[.*\]|<.*>|\{.*\}" "$file"  # Template brackets left in
+```
 
-# Database table existence
-npx prisma db execute --stdin <<< "SELECT 1 FROM users LIMIT 1"
+**Empty or trivial implementations:**
+```bash
+# Functions that do nothing
+grep -E "return null|return undefined|return \{\}|return \[\]" "$file"
+grep -E "pass$|\.\.\.|\bnothing\b" "$file"
+grep -E "console\.(log|warn|error).*only" "$file"  # Log-only functions
+```
 
-# Route existence
-curl -s -o /dev/null -w "%{http_code}" http://localhost:3000/api/auth/login
+**Hardcoded values where dynamic expected:**
+```bash
+# Hardcoded IDs, counts, or content
+grep -E "id.*=.*['\"].*['\"]" "$file"  # Hardcoded string IDs
+grep -E "count.*=.*\d+|length.*=.*\d+" "$file"  # Hardcoded counts
+grep -E "\\\$\d+\.\d{2}|\d+ items" "$file"  # Hardcoded display values
 ```
 
-### Layer 2: Substantiveness
+</stub_detection>
 
-Is the artifact more than a stub?
+<react_components>
 
+## React/Next.js Components
+
+**Existence check:**
+```bash
+# File exists and exports component
+[ -f "$component_path" ] && grep -E "export (default |)function|export const.*=.*\(" "$component_path"
+```
+
+**Substantive check:**
 ```bash
-# File has real content (not just exports)
-wc -l src/auth/discord.ts  # Should be > 10 lines for a real implementation
+# Returns actual JSX, not placeholder
+grep -E "return.*<" "$component_path" | grep -v "return.*null" | grep -v "placeholder" -i
+
+# Has meaningful content (not just wrapper div)
+grep -E "<[A-Z][a-zA-Z]+|className=|onClick=|onChange=" "$component_path"
+
+# Uses props or state (not static)
+grep -E "props\.|useState|useEffect|useContext|\{.*\}" "$component_path"
+```
 
-# Function has a body (not just a signature)
-grep -A5 "authenticateWithDiscord" src/auth/discord.ts | grep -q "return"
+**Stub patterns specific to React:**
+```javascript
+// RED FLAGS - These are stubs:
+return <div>Component</div>
+return <div>Placeholder</div>
+return <div>{/* TODO */}</div>
+return <p>Coming soon</p>
+return null
+return <></>
+
+// Also stubs - empty handlers:
+onClick={() => {})
+onChange={() => console.log('clicked')}
+onSubmit={(e) => e.preventDefault()}  // Only prevents default, does nothing
+```
+
+**Wiring check:**
+```bash
+# Component imports what it needs
+grep -E "^import.*from" "$component_path"
 
-# Test file has actual test cases
-grep -c "it(" tests/auth.test.ts  # Should be > 0
+# Props are actually used (not just received)
+# Look for destructuring or props.X usage
+grep -E "\{ .* \}.*props|\bprops\.[a-zA-Z]+" "$component_path"
 
-# API returns real data (not just 200 OK)
-curl -s http://localhost:3000/api/health | jq '.status' | grep -q "ok"
+# API calls exist (for data-fetching components)
+grep -E "fetch\(|axios\.|useSWR|useQuery|getServerSideProps|getStaticProps" "$component_path"
 ```
 
-### Layer 3: Wiring
+**Functional verification (human required):**
+- Does the component render visible content?
+- Do interactive elements respond to clicks?
+- Does data load and display?
+- Do error states show appropriately?
+
+</react_components>
+
+<api_routes>
+
+## API Routes (Next.js App Router / Express / etc.)
+
+**Existence check:**
+```bash
+# Route file exists
+[ -f "$route_path" ]
+
+# Exports HTTP method handlers (Next.js App Router)
+grep -E "export (async )?(function|const) (GET|POST|PUT|PATCH|DELETE)" "$route_path"
 
-Are components connected to each other?
+# Or Express-style handlers
+grep -E "\.(get|post|put|patch|delete)\(" "$route_path"
+```
 
+**Substantive check:**
 ```bash
-# Module is imported where needed
-grep -q "from.*auth/discord" src/routes/auth.ts
+# Has actual logic, not just return statement
+wc -l "$route_path"  # More than 10-15 lines suggests real implementation
 
-# Middleware is applied to routes
-grep -q "requireAuth" src/routes/protected.ts
+# Interacts with data source
+grep -E "prisma\.|db\.|mongoose\.|sql|query|find|create|update|delete" "$route_path" -i
 
-# Database is configured in app init
-grep -q "prisma" src/app.ts
+# Has error handling
+grep -E "try|catch|throw|error|Error" "$route_path"
 
-# Environment variables are referenced
-grep -q "DISCORD_CLIENT_ID" src/auth/discord.ts
+# Returns meaningful response
+grep -E "Response\.json|res\.json|res\.send|return.*\{" "$route_path" | grep -v "message.*not implemented" -i
 ```
 
----
+**Stub patterns specific to API routes:**
+```typescript
+// RED FLAGS - These are stubs:
+export async function POST() {
+  return Response.json({ message: "Not implemented" })
+}
+
+export async function GET() {
+  return Response.json([])  // Empty array with no DB query
+}
+
+export async function PUT() {
+  return new Response()  // Empty response
+}
+
+// Console log only:
+export async function POST(req) {
+  console.log(await req.json())
+  return Response.json({ ok: true })
+}
+```
 
-## Verification by Feature Type
+**Wiring check:**
+```bash
+# Imports database/service clients
+grep -E "^import.*prisma|^import.*db|^import.*client" "$route_path"
 
-### API Endpoint
+# Actually uses request body (for POST/PUT)
+grep -E "req\.json\(\)|req\.body|request\.json\(\)" "$route_path"
 
+# Validates input (not just trusting request)
+grep -E "schema\.parse|validate|zod|yup|joi" "$route_path"
 ```
-Existence:  curl returns non-404 status
-Substance:  curl returns expected response shape (correct fields)
-Wiring:     endpoint calls the right service, middleware is applied
+
+**Functional verification (human or automated):**
+- Does GET return real data from database?
+- Does POST actually create a record?
+- Does error response have correct status code?
+- Are auth checks actually enforced?
+
+</api_routes>
+
+<database_schema>
+
+## Database Schema (Prisma / Drizzle / SQL)
+
+**Existence check:**
+```bash
+# Schema file exists
+[ -f "prisma/schema.prisma" ] || [ -f "drizzle/schema.ts" ] || [ -f "src/db/schema.sql" ]
+
+# Model/table is defined
+grep -E "^model $model_name|CREATE TABLE $table_name|export const $table_name" "$schema_path"
 ```
 
-### Database Schema
+**Substantive check:**
+```bash
+# Has expected fields (not just id)
+grep -A 20 "model $model_name" "$schema_path" | grep -E "^\s+\w+\s+\w+"
+
+# Has relationships if expected
+grep -E "@relation|REFERENCES|FOREIGN KEY" "$schema_path"
 
+# Has appropriate field types (not all String)
+grep -A 20 "model $model_name" "$schema_path" | grep -E "Int|DateTime|Boolean|Float|Decimal|Json"
 ```
-Existence:  table/collection exists, can query without error
-Substance:  columns/fields match specification, constraints are applied
-Wiring:     application code references the schema, migrations run cleanly
+
+**Stub patterns specific to schemas:**
+```prisma
+// RED FLAGS - These are stubs:
+model User {
+  id String @id
+  // TODO: add fields
+}
+
+model Message {
+  id        String @id
+  content   String  // Only one real field
+}
+
+// Missing critical fields:
+model Order {
+  id     String @id
+  // No: userId, items, total, status, createdAt
+}
 ```
 
-### Authentication
+**Wiring check:**
+```bash
+# Migrations exist and are applied
+ls prisma/migrations/ 2>/dev/null | wc -l  # Should be > 0
+npx prisma migrate status 2>/dev/null | grep -v "pending"
 
+# Client is generated
+[ -d "node_modules/.prisma/client" ]
 ```
-Existence:  auth routes exist, auth module exports functions
-Substance:  login flow returns token, invalid creds return error
-Wiring:     protected routes use auth middleware, tokens are validated
+
+**Functional verification:**
+```bash
+# Can query the table (automated)
+npx prisma db execute --stdin <<< "SELECT COUNT(*) FROM $table_name"
 ```
 
-### UI Component
+</database_schema>
 
+<hooks_utilities>
+
+## Custom Hooks and Utilities
+
+**Existence check:**
+```bash
+# File exists and exports function
+[ -f "$hook_path" ] && grep -E "export (default )?(function|const)" "$hook_path"
 ```
-Existence:  component file exists, exports default component
-Substance:  component renders expected elements (test or visual check)
-Wiring:     component is imported in parent, receives correct props, routes to it
+
+**Substantive check:**
+```bash
+# Hook uses React hooks (for custom hooks)
+grep -E "useState|useEffect|useCallback|useMemo|useRef|useContext" "$hook_path"
+
+# Has meaningful return value
+grep -E "return \{|return \[" "$hook_path"
+
+# More than trivial length
+[ $(wc -l < "$hook_path") -gt 10 ]
+```
+
+**Stub patterns specific to hooks:**
+```typescript
+// RED FLAGS - These are stubs:
+export function useAuth() {
+  return { user: null, login: () => {}, logout: () => {} }
+}
+
+export function useCart() {
+  const [items, setItems] = useState([])
+  return { items, addItem: () => console.log('add'), removeItem: () => {} }
+}
+
+// Hardcoded return:
+export function useUser() {
+  return { name: "Test User", email: "test@example.com" }
+}
+```
+
+**Wiring check:**
+```bash
+# Hook is actually imported somewhere
+grep -r "import.*$hook_name" src/ --include="*.tsx" --include="*.ts" | grep -v "$hook_path"
+
+# Hook is actually called
+grep -r "$hook_name()" src/ --include="*.tsx" --include="*.ts" | grep -v "$hook_path"
+```
+
+</hooks_utilities>
+
+<environment_config>
+
+## Environment Variables and Configuration
+
+**Existence check:**
+```bash
+# .env file exists
+[ -f ".env" ] || [ -f ".env.local" ]
+
+# Required variable is defined
+grep -E "^$VAR_NAME=" .env .env.local 2>/dev/null
 ```
 
-### Configuration
+**Substantive check:**
+```bash
+# Variable has actual value (not placeholder)
+grep -E "^$VAR_NAME=.+" .env .env.local 2>/dev/null | grep -v "your-.*-here|xxx|placeholder|TODO" -i
+
+# Value looks valid for type:
+# - URLs should start with http
+# - Keys should be long enough
+# - Booleans should be true/false
+```
 
+**Stub patterns specific to env:**
+```bash
+# RED FLAGS - These are stubs:
+DATABASE_URL=your-database-url-here
+STRIPE_SECRET_KEY=sk_test_xxx
+API_KEY=placeholder
+NEXT_PUBLIC_API_URL=http://localhost:3000  # Still pointing to localhost in prod
 ```
-Existence:  config file exists, environment variables documented
-Substance:  config values are used (not dead code), defaults are sensible
-Wiring:     application reads config at startup, config changes take effect
+
+**Wiring check:**
+```bash
+# Variable is actually used in code
+grep -r "process\.env\.$VAR_NAME|env\.$VAR_NAME" src/ --include="*.ts" --include="*.tsx"
+
+# Variable is in validation schema (if using zod/etc for env)
+grep -E "$VAR_NAME" src/env.ts src/env.mjs 2>/dev/null
 ```
 
----
+</environment_config>
+
+<wiring_verification>
+
+<!-- wiring verification patterns -->
+## Wiring Verification Patterns
+
+Wiring verification checks that components actually communicate. This is where most stubs hide.
 
-## Verify Command Patterns
+### Pattern: Component -> API
 
-### TypeScript/JavaScript
+**Check:** Does the component actually call the API?
 
 ```bash
-# Type checking
-npx tsc --noEmit
+# Find the fetch/axios call
+grep -E "fetch\(['\"].*$api_path|axios\.(get|post).*$api_path" "$component_path"
 
-# Unit tests
-npm test
-npm test -- --grep "pattern"
-npx jest path/to/test
+# Verify it's not commented out
+grep -E "fetch\(|axios\." "$component_path" | grep -v "^.*//.*fetch"
 
-# Linting
-npx eslint src/auth/
+# Check the response is used
+grep -E "await.*fetch|\.then\(|setData|setState" "$component_path"
+```
+
+**Red flags:**
+```typescript
+// Fetch exists but response ignored:
+fetch('/api/messages')  // No await, no .then, no assignment
 
-# Build check
-npm run build
+// Fetch in comment:
+// fetch('/api/messages').then(r => r.json()).then(setMessages)
+
+// Fetch to wrong endpoint:
+fetch('/api/message')  // Typo - should be /api/messages
 ```
 
-### Python
+### Pattern: API -> Database
+
+**Check:** Does the API route actually query the database?
 
 ```bash
-# Type checking
-mypy src/
+# Find the database call
+grep -E "prisma\.$model|db\.query|Model\.find" "$route_path"
+
+# Verify it's awaited
+grep -E "await.*prisma|await.*db\." "$route_path"
+
+# Check result is returned
+grep -E "return.*json.*data|res\.json.*result" "$route_path"
+```
 
-# Unit tests
-pytest tests/
-pytest tests/test_auth.py -v
-pytest -k "test_login"
+**Red flags:**
+```typescript
+// Query exists but result not returned:
+await prisma.message.findMany()
+return Response.json({ ok: true })  // Returns static, not query result
 
-# Linting
-flake8 src/
-pylint src/auth/
+// Query not awaited:
+const messages = prisma.message.findMany()  // Missing await
+return Response.json(messages)  // Returns Promise, not data
 ```
 
-### General
+### Pattern: Form -> Handler
+
+**Check:** Does the form submission actually do something?
 
 ```bash
-# File existence
-ls -la path/to/file
+# Find onSubmit handler
+grep -E "onSubmit=\{|handleSubmit" "$component_path"
+
+# Check handler has content
+grep -A 10 "onSubmit.*=" "$component_path" | grep -E "fetch|axios|mutate|dispatch"
 
-# Content check
-grep -q "pattern" path/to/file
+# Verify not just preventDefault
+grep -A 5 "onSubmit" "$component_path" | grep -v "only.*preventDefault" -i
+```
 
-# HTTP endpoint
-curl -s -o /dev/null -w "%{http_code}" http://localhost:PORT/path
+**Red flags:**
+```typescript
+// Handler only prevents default:
+onSubmit={(e) => e.preventDefault()}
 
-# Process running
-pgrep -f "process-name"
+// Handler only logs:
+const handleSubmit = (data) => {
+  console.log(data)
+}
 
-# Port listening
-netstat -an | grep LISTEN | grep PORT
+// Handler is empty:
+onSubmit={() => {}}
 ```
 
----
+### Pattern: State -> Render
 
-## Done Condition Patterns
+**Check:** Does the component render state, not hardcoded content?
 
-### Good Done Conditions (Observable, Falsifiable)
+```bash
+# Find state usage in JSX
+grep -E "\{.*messages.*\}|\{.*data.*\}|\{.*items.*\}" "$component_path"
 
-- "User can log in with Discord and see their dashboard"
-- "API returns paginated list of items with correct total count"
-- "Database migration creates users table with all required columns"
-- "Protected routes return 401 without valid JWT"
-- "Build completes without TypeScript errors"
+# Check map/render of state
+grep -E "\.map\(|\.filter\(|\.reduce\(" "$component_path"
 
-### Bad Done Conditions (Vague, Not Falsifiable)
+# Verify dynamic content
+grep -E "\{[a-zA-Z_]+\." "$component_path"  # Variable interpolation
+```
+
+**Red flags:**
+```tsx
+// Hardcoded instead of state:
+return <div>
+  <p>Message 1</p>
+  <p>Message 2</p>
+</div>
+
+// State exists but not rendered:
+const [messages, setMessages] = useState([])
+return <div>No messages</div>  // Always shows "no messages"
+
+// Wrong state rendered:
+const [messages, setMessages] = useState([])
+return <div>{otherData.map(...)}</div>  // Uses different data
+```
 
-- "Authentication is implemented" (how do you test this?)
-- "Code is clean" (subjective)
-- "Database is set up" (what does "set up" mean?)
-- "Tests pass" (which tests? what do they test?)
-- "File was created" (created with what content?)
+</wiring_verification>
 
-### Transformation Rule
+<verification_checklist>
 
+## Quick Verification Checklist
+
+For each artifact type, run through this checklist:
+
+### Component Checklist
+- [ ] File exists at expected path
+- [ ] Exports a function/const component
+- [ ] Returns JSX (not null/empty)
+- [ ] No placeholder text in render
+- [ ] Uses props or state (not static)
+- [ ] Event handlers have real implementations
+- [ ] Imports resolve correctly
+- [ ] Used somewhere in the app
+
+### API Route Checklist
+- [ ] File exists at expected path
+- [ ] Exports HTTP method handlers
+- [ ] Handlers have more than 5 lines
+- [ ] Queries database or service
+- [ ] Returns meaningful response (not empty/placeholder)
+- [ ] Has error handling
+- [ ] Validates input
+- [ ] Called from frontend
+
+### Schema Checklist
+- [ ] Model/table defined
+- [ ] Has all expected fields
+- [ ] Fields have appropriate types
+- [ ] Relationships defined if needed
+- [ ] Migrations exist and applied
+- [ ] Client generated
+
+### Hook/Utility Checklist
+- [ ] File exists at expected path
+- [ ] Exports function
+- [ ] Has meaningful implementation (not empty returns)
+- [ ] Used somewhere in the app
+- [ ] Return values consumed
+
+### Wiring Checklist
+- [ ] Component -> API: fetch/axios call exists and uses response
+- [ ] API -> Database: query exists and result returned
+- [ ] Form -> Handler: onSubmit calls API/mutation
+- [ ] State -> Render: state variables appear in JSX
+
+</verification_checklist>
+
+<automated_verification_script>
+
+## Automated Verification Approach
+
+For the verification subagent, use this pattern:
+
+```bash
+# 1. Check existence
+check_exists() {
+  [ -f "$1" ] && echo "EXISTS: $1" || echo "MISSING: $1"
+}
+
+# 2. Check for stub patterns
+check_stubs() {
+  local file="$1"
+  local stubs=$(grep -c -E "TODO|FIXME|placeholder|not implemented" "$file" 2>/dev/null || echo 0)
+  [ "$stubs" -gt 0 ] && echo "STUB_PATTERNS: $stubs in $file"
+}
+
+# 3. Check wiring (component calls API)
+check_wiring() {
+  local component="$1"
+  local api_path="$2"
+  grep -q "$api_path" "$component" && echo "WIRED: $component -> $api_path" || echo "NOT_WIRED: $component -> $api_path"
+}
+
+# 4. Check substantive (more than N lines, has expected patterns)
+check_substantive() {
+  local file="$1"
+  local min_lines="$2"
+  local pattern="$3"
+  local lines=$(wc -l < "$file" 2>/dev/null || echo 0)
+  local has_pattern=$(grep -c -E "$pattern" "$file" 2>/dev/null || echo 0)
+  [ "$lines" -ge "$min_lines" ] && [ "$has_pattern" -gt 0 ] && echo "SUBSTANTIVE: $file" || echo "THIN: $file ($lines lines, $has_pattern matches)"
+}
 ```
-Bad:  "Authentication is implemented"
-Good: "User can complete Discord OAuth flow and receive a valid JWT"
 
-Bad:  "Database is set up"
-Good: "Users table exists with id, email, name columns and can accept INSERT"
+Run these checks against each must-have artifact. Aggregate results into VERIFICATION.md.
+
+</automated_verification_script>
+
+<human_verification_triggers>
 
-Bad:  "Tests pass"
-Good: "All 5 auth middleware tests pass: valid token, expired token,
-       missing token, malformed token, and correct user extraction"
+## When to Require Human Verification
+
+Some things can't be verified programmatically. Flag these for human testing:
+
+**Always human:**
+- Visual appearance (does it look right?)
+- User flow completion (can you actually do the thing?)
+- Real-time behavior (WebSocket, SSE)
+- External service integration (Stripe, email sending)
+- Error message clarity (is the message helpful?)
+- Performance feel (does it feel fast?)
+
+**Human if uncertain:**
+- Complex wiring that grep can't trace
+- Dynamic behavior depending on state
+- Edge cases and error states
+- Mobile responsiveness
+- Accessibility
+
+**Format for human verification request:**
+```markdown
+## Human Verification Required
+
+### 1. Chat message sending
+**Test:** Type a message and click Send
+**Expected:** Message appears in list, input clears
+**Check:** Does message persist after refresh?
+
+### 2. Error handling
+**Test:** Disconnect network, try to send
+**Expected:** Error message appears, message not lost
+**Check:** Can retry after reconnect?
 ```
+
+</human_verification_triggers>
+
+<checkpoint_automation_reference>
+
+## Pre-Checkpoint Automation
+
+For automation-first checkpoint patterns, server lifecycle management, CLI installation handling, and error recovery protocols, see:
+
+**references/checkpoints.md** -> `<automation_reference>` section
+
+Key principles:
+- Claude sets up verification environment BEFORE presenting checkpoints
+- Users never run CLI commands (visit URLs only)
+- Server lifecycle: start before checkpoint, handle port conflicts, keep running for duration
+- CLI installation: auto-install where safe, checkpoint for user choice otherwise
+- Error handling: fix broken environment before checkpoint, never present checkpoint with failed setup
+
+</checkpoint_automation_reference>