anvil-dev-framework 0.1.6

package/docs/specifications/SPEC-ANV-verification-ralph-wiggum.md

@@ -0,0 +1,789 @@
+---
+spec_id: SPEC-ANV-VRW
+title: Verification Feedback Loops & Ralph Wiggum Mode
+status: draft
+created: 2026-01-04
+updated: 2026-01-04
+linear_issue: ANV-140
+references:
+  - https://x.com/bcherny/status/2007179832300581177
+  - https://github.com/anthropics/claude-code/tree/main/plugins/ralph-wiggum
+  - docs/boris-cherny-ralph-wiggum-insights.md
+---
+
+# Verification Feedback Loops & Ralph Wiggum Mode
+
+## Executive Summary
+
+This specification implements proven methodologies from Boris Cherny (Claude Code creator) and Geoffrey Huntley's Ralph Wiggum technique to achieve **2-3x quality improvements** through verification feedback loops and enable **long-running unattended execution**.
+
+### Key Outcomes
+- **Verification-first workflow** that catches issues before they compound
+- **Stop hook gates** that prevent premature task completion
+- **Ralph Wiggum mode** for overnight/unattended execution
+- **Atomic action commands** for rapid inner-loop execution
+- **Self-improving CLAUDE.md** via automated pattern extraction
+
+---
+
+## Problem Statement
+
+### Current Gaps vs. Best Practices
+
+| Gap | Impact | Boris/Ralph Evidence |
+|-----|--------|---------------------|
+| No verification feedback loop | Errors compound, manual QA required | Boris: "2-3x quality improvement" |
+| No stop hook gating | Claude exits with failing tests | Boris: Non-zero exit blocks stopping |
+| No PostToolUse formatting | Manual formatting, inconsistent code | Boris: Auto-format on every edit |
+| No long-running mode | Requires constant supervision | Geoffrey: 6 repos overnight, $50k→$297 |
+| Scattered atomic actions | Slow inner loops | Boris: `/commit-push-pr` one-liners |
+| Manual CLAUDE.md updates | Stale project context | Boris: GitHub Actions extraction |
+
+### User Pain Points
+
+1. **Quality Drift**: Without verification loops, small errors accumulate into large fixes
+2. **Context Pollution**: Main session bloated with research that should be in subagents
+3. **Supervision Burden**: Can't run Claude overnight without babysitting
+4. **Slow Iterations**: Common multi-step actions (test→commit→push→PR) require multiple commands
+
+---
+
+## Goals
+
+| Goal | Success Metric |
+|------|---------------|
+| Verification-first workflow | All PRs pass CI before human review |
+| Stop hook enforcement | Zero exits with failing tests |
+| Unattended execution | 4+ hour autonomous sessions |
+| Rapid inner loops | Single command for test→commit→push→PR |
+| Self-improving context | CLAUDE.md updated weekly from commits |
+
+---
+
+## Component 1: Verification Command (`/verify`)
+
+### Overview
+
+A new slash command that runs verification checks with automatic iteration on failure. This is the **highest-impact** feature from Boris Cherny's methodology.
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-V-100 | Run configurable verification suite (tests, lint, types) | P0 |
+| REQ-V-101 | On failure, re-read failing files and attempt fix | P0 |
+| REQ-V-102 | Maximum 3 automatic fix attempts before escalating | P0 |
+| REQ-V-103 | Track verification state (pass/fail/iterations) | P0 |
+| REQ-V-104 | Integrate with stop hook for exit gating | P1 |
+| REQ-V-105 | Support custom verification commands per project | P1 |
+| REQ-V-106 | Report verification results in statusline | P2 |
+
+### Behavior
+
+```
+User: /verify
+
+Claude:
+1. Run verification suite:
+   - npm test (or configured test command)
+   - npm run lint
+   - npm run typecheck
+
+2. If ALL pass:
+   ✅ Verification passed (tests: 47, lint: clean, types: clean)
+   [Mark task ready for completion]
+
+3. If ANY fail:
+   ❌ Verification failed - 3 test failures
+
+   Analyzing failures...
+   - src/auth/login.test.ts:45 - Expected 200, got 401
+   - src/auth/login.test.ts:67 - Timeout
+
+   Attempting fix (iteration 1/3)...
+   [Re-read failing files]
+   [Apply targeted fix]
+   [Re-run verification]
+
+4. After 3 failed iterations:
+   ⚠️ Verification failed after 3 attempts
+
+   Remaining issues:
+   - [List of unfixed issues]
+
+   Manual intervention required.
+```
+
+### Configuration
+
+```yaml
+# .claude/settings.yaml
+verification:
+  commands:
+    test: "npm test"
+    lint: "npm run lint"
+    types: "npm run typecheck"
+  max_iterations: 3
+  required_for_completion: true
+```
+
+### Skill Implementation
+
+```markdown
+# /verify - Verification Feedback Loop
+
+> Run verification suite with automatic iteration on failure.
+
+## Execution Steps
+
+### Step 1: Run Verification Suite
+Execute all configured verification commands:
+- Test suite
+- Linter
+- Type checker
+
+### Step 2: Analyze Results
+If all pass → Report success, allow completion
+If any fail → Parse errors, identify failing files
+
+### Step 3: Iterate on Failures (max 3 times)
+1. Re-read each failing file
+2. Analyze the specific error
+3. Apply targeted fix
+4. Re-run failing command only
+5. Repeat until pass or max iterations
+
+### Step 4: Report Final State
+- Success: List all passing checks
+- Failure: List remaining issues, request manual help
+
+## Anti-Patterns
+- ❌ Making broad changes instead of targeted fixes
+- ❌ Continuing past max iterations
+- ❌ Ignoring failing checks
+```
+
+---
+
+## Component 2: Stop Hook Verification Gate
+
+### Overview
+
+A stop hook that prevents Claude from exiting when verification fails. This ensures the AI cannot prematurely complete tasks with broken code.
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-SH-100 | Run verification checks before allowing exit | P0 |
+| REQ-SH-101 | Return non-zero exit code to block stopping | P0 |
+| REQ-SH-102 | Provide clear feedback on what's blocking exit | P0 |
+| REQ-SH-103 | Allow override with explicit user command | P1 |
+| REQ-SH-104 | Configure which checks are mandatory | P1 |
+| REQ-SH-105 | Log all gate events for debugging | P2 |
+
+### Hook Implementation
+
+```bash
+#!/bin/bash
+# global/hooks/stop_gate.sh
+# Stop hook that gates on verification
+
+set -e
+
+# Check if verification is required
+REQUIRE_VERIFY="${ANVIL_REQUIRE_VERIFICATION:-true}"
+
+if [[ "$REQUIRE_VERIFY" != "true" ]]; then
+    exit 0  # Allow exit
+fi
+
+# Check for explicit override
+if [[ "$ANVIL_FORCE_EXIT" == "true" ]]; then
+    echo "⚠️ Force exit requested - bypassing verification"
+    exit 0
+fi
+
+echo "🔒 Running verification gate..."
+
+# Run test suite
+if ! npm test --silent 2>/dev/null; then
+    echo "❌ Tests failing - cannot exit"
+    echo "Fix failing tests or use /force-exit to bypass"
+    exit 1  # Block exit
+fi
+
+# Run lint
+if ! npm run lint --silent 2>/dev/null; then
+    echo "❌ Lint errors - cannot exit"
+    echo "Fix lint errors or use /force-exit to bypass"
+    exit 1
+fi
+
+# Run typecheck
+if ! npm run typecheck --silent 2>/dev/null; then
+    echo "❌ Type errors - cannot exit"
+    echo "Fix type errors or use /force-exit to bypass"
+    exit 1
+fi
+
+echo "✅ Verification gate passed"
+exit 0
+```
+
+### Settings Configuration
+
+```json
+{
+  "hooks": {
+    "Stop": [{
+      "type": "command",
+      "command": "bash global/hooks/stop_gate.sh"
+    }]
+  }
+}
+```
+
+---
+
+## Component 3: PostToolUse Formatting Hook
+
+### Overview
+
+Automatically format code files after every Edit or Write operation. This catches the "last 10%" of formatting issues that Claude often misses.
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-PTU-100 | Run formatter on files after Edit tool | P0 |
+| REQ-PTU-101 | Run formatter on files after Write tool | P0 |
+| REQ-PTU-102 | Support project-specific formatter (prettier, black, etc.) | P0 |
+| REQ-PTU-103 | Execute in <500ms to avoid slowdown | P1 |
+| REQ-PTU-104 | Skip binary files and node_modules | P1 |
+| REQ-PTU-105 | Log formatting changes for visibility | P2 |
+
+### Hook Implementation
+
+```bash
+#!/bin/bash
+# global/hooks/post_tool_format.sh
+# PostToolUse hook for auto-formatting
+
+FILE_PATH="$CLAUDE_FILE_PATH"
+
+# Skip if no file path
+if [[ -z "$FILE_PATH" ]]; then
+    exit 0
+fi
+
+# Skip non-existent files
+if [[ ! -f "$FILE_PATH" ]]; then
+    exit 0
+fi
+
+# Skip binary and excluded paths
+case "$FILE_PATH" in
+    */node_modules/*|*/.git/*|*.png|*.jpg|*.ico|*.woff*)
+        exit 0
+        ;;
+esac
+
+# Determine formatter based on file type
+case "$FILE_PATH" in
+    *.ts|*.tsx|*.js|*.jsx|*.json|*.md|*.css|*.html)
+        if command -v npx &> /dev/null; then
+            npx prettier --write "$FILE_PATH" 2>/dev/null || true
+        fi
+        ;;
+    *.py)
+        if command -v black &> /dev/null; then
+            black "$FILE_PATH" 2>/dev/null || true
+        elif command -v ruff &> /dev/null; then
+            ruff format "$FILE_PATH" 2>/dev/null || true
+        fi
+        ;;
+    *.go)
+        if command -v gofmt &> /dev/null; then
+            gofmt -w "$FILE_PATH" 2>/dev/null || true
+        fi
+        ;;
+    *.sh)
+        if command -v shfmt &> /dev/null; then
+            shfmt -w "$FILE_PATH" 2>/dev/null || true
+        fi
+        ;;
+esac
+
+exit 0
+```
+
+### Settings Configuration
+
+```json
+{
+  "hooks": {
+    "PostToolUse": [{
+      "matcher": "Edit|Write",
+      "hooks": [{
+        "type": "command",
+        "command": "bash global/hooks/post_tool_format.sh"
+      }]
+    }]
+  }
+}
+```
+
+---
+
+## Component 4: Ralph Wiggum Mode (`/ralph`)
+
+### Overview
+
+Long-running unattended execution mode based on Geoffrey Huntley's technique. Uses a bash loop with completion promises and stop hook interception for multi-hour autonomous sessions.
+
+### Core Mechanism
+
+```bash
+# The Ralph Wiggum loop
+while :; do
+    cat PROMPT.md | npx --yes @anthropic/claude-code --print
+done
+```
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-RW-100 | Create PROMPT.md with structured task format | P0 |
+| REQ-RW-101 | Implement completion promise pattern | P0 |
+| REQ-RW-102 | Stop hook intercepts exit, restarts on incomplete | P0 |
+| REQ-RW-103 | Maximum iterations safety (default: 50) | P0 |
+| REQ-RW-104 | Track iteration count and progress | P0 |
+| REQ-RW-105 | Single-item focus per iteration (critical) | P0 |
+| REQ-RW-106 | Context management via subagents | P1 |
+| REQ-RW-107 | Self-learning AGENT.md updates | P2 |
+
+### Three-Phase Architecture
+
+Based on Ralph Wiggum's proven pattern:
+
+```
+Phase 1: Requirements → specs/*.md
+Phase 2: Specs → fix_plan.md (TODO list)
+Phase 3: TODO items → Implementation (one at a time)
+```
+
+### PROMPT.md Format
+
+```markdown
+# Task: [Feature Name]
+
+## Objective
+[Clear, single-item objective]
+
+## Current State
+- Iteration: {iteration_count}
+- Last action: {last_action}
+- Remaining items: {remaining_count}
+
+## Instructions
+1. Read fix_plan.md for current TODO list
+2. Complete ONLY the first unchecked item
+3. Run tests after completion
+4. If tests pass, mark item complete
+5. If all items complete, output: <promise>COMPLETE</promise>
+
+## Constraints
+- ONE item per iteration (critical)
+- Do not skip verification
+- Commit on each successful item
+- Use subagents for research (max 500)
+- Use only 1 agent for build/test
+
+## Completion Signal
+When ALL items are done and tests pass:
+<promise>COMPLETE</promise>
+```
+
+### Stop Hook Integration
+
+```bash
+#!/bin/bash
+# global/hooks/ralph_stop.sh
+# Ralph Wiggum stop hook with completion promise
+
+TRANSCRIPT="$CLAUDE_TRANSCRIPT"
+MAX_ITERATIONS="${RALPH_MAX_ITERATIONS:-50}"
+ITERATION_FILE="/tmp/ralph_iteration_count"
+
+# Read or initialize iteration count
+if [[ -f "$ITERATION_FILE" ]]; then
+    ITERATION=$(cat "$ITERATION_FILE")
+else
+    ITERATION=0
+fi
+
+# Increment iteration
+ITERATION=$((ITERATION + 1))
+echo "$ITERATION" > "$ITERATION_FILE"
+
+# Safety: Max iterations check
+if [[ $ITERATION -ge $MAX_ITERATIONS ]]; then
+    echo "⚠️ Ralph Wiggum: Maximum iterations ($MAX_ITERATIONS) reached"
+    echo "Manual review required"
+    rm -f "$ITERATION_FILE"
+    exit 0  # Allow exit after max
+fi
+
+# Check for completion promise
+if grep -q "<promise>COMPLETE</promise>" "$TRANSCRIPT" 2>/dev/null; then
+    echo "✅ Ralph Wiggum: Task complete after $ITERATION iterations"
+    rm -f "$ITERATION_FILE"
+    exit 0  # Allow exit
+fi
+
+# Check for fatal error signals
+if grep -q "<fatal>" "$TRANSCRIPT" 2>/dev/null; then
+    echo "❌ Ralph Wiggum: Fatal error detected"
+    rm -f "$ITERATION_FILE"
+    exit 0  # Allow exit on fatal
+fi
+
+echo "🔄 Ralph Wiggum: Iteration $ITERATION - task incomplete, restarting..."
+exit 1  # Block exit, trigger restart
+```
+
+### `/ralph` Skill Implementation
+
+```markdown
+# /ralph - Ralph Wiggum Long-Running Mode
+
+> Enable long-running unattended execution with completion promises.
+
+## Arguments
+- `start [task-description]` - Initialize Ralph mode with task
+- `status` - Show current iteration and progress
+- `stop` - Force exit Ralph mode
+
+## Execution: /ralph start
+
+### Step 1: Create Task Structure
+Create PROMPT.md with:
+- Clear single objective
+- Current state tracking
+- Constraints and completion signal
+
+### Step 2: Enable Stop Hook
+Activate ralph_stop.sh hook that:
+- Checks for <promise>COMPLETE</promise>
+- Blocks exit if incomplete
+- Tracks iteration count
+- Has max iterations safety
+
+### Step 3: Create fix_plan.md
+Break task into atomic TODO items:
+- [ ] Item 1
+- [ ] Item 2
+- ...
+
+### Step 4: Start Loop
+Execute bash loop that:
+- Feeds PROMPT.md to Claude
+- Processes one item per iteration
+- Commits on success
+- Restarts until complete
+
+## Critical Rules
+1. **ONE item per loop** - Never try to do multiple items
+2. **Subagents for research** - Don't pollute main context
+3. **Commit on success** - Each item gets committed
+4. **Tests gate progress** - No marking complete without tests
+5. **500 research / 1 build** - Subagent ratio
+
+## Example
+```
+/ralph start "Implement user authentication with OAuth"
+
+Creates:
+- PROMPT.md with structured task
+- fix_plan.md with TODO items
+- Enables ralph_stop.sh hook
+- Starts execution loop
+
+Claude will iterate until:
+- All TODOs complete
+- Tests pass
+- <promise>COMPLETE</promise> output
+```
+```
+
+---
+
+## Component 5: Atomic Action Commands
+
+### Overview
+
+Single commands that execute complete workflows for rapid inner-loop execution.
+
+### Commands
+
+#### `/test-and-commit`
+
+```markdown
+# /test-and-commit - Test Then Commit
+
+> Run tests, and if passing, commit changes.
+
+## Execution
+1. Run test suite
+2. If pass → Create commit with conventional message
+3. If fail → Report failures, do not commit
+
+## Usage
+/test-and-commit "feat: add user authentication"
+```
+
+#### `/commit-push-pr`
+
+```markdown
+# /commit-push-pr - Full Commit Workflow
+
+> Commit, push, and create PR in one command.
+
+## Execution
+1. Stage all changes
+2. Create commit with message
+3. Push to remote
+4. Create PR with description
+
+## Usage
+/commit-push-pr "feat: add OAuth support"
+```
+
+#### `/lint-fix`
+
+```markdown
+# /lint-fix - Auto-Fix Lint Issues
+
+> Run linter with auto-fix enabled.
+
+## Execution
+1. Run linter with --fix flag
+2. Report remaining issues
+3. Stage auto-fixed changes
+
+## Usage
+/lint-fix
+```
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-AA-100 | `/test-and-commit` runs tests before committing | P0 |
+| REQ-AA-101 | `/commit-push-pr` handles full workflow | P0 |
+| REQ-AA-102 | `/lint-fix` auto-fixes lintable issues | P1 |
+| REQ-AA-103 | All commands have clear success/failure output | P0 |
+| REQ-AA-104 | Commands respect git conventions | P0 |
+
+---
+
+## Component 6: Verification Subagent
+
+### Overview
+
+A dedicated subagent for verification that keeps research out of the main context.
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-VS-100 | Subagent runs all verification checks | P0 |
+| REQ-VS-101 | Returns structured pass/fail report | P0 |
+| REQ-VS-102 | Isolates verification context from main session | P0 |
+| REQ-VS-103 | Supports custom verification commands | P1 |
+
+### Agent Configuration
+
+```json
+{
+  "name": "verify-app",
+  "description": "Verification subagent for running tests, lint, and type checks",
+  "tools": ["Bash", "Read", "Glob"],
+  "prompt": "You are a verification agent. Run all project checks and report results in structured format."
+}
+```
+
+### Invocation Pattern
+
+```bash
+# Use Task tool with verify-app subagent
+Task(
+  subagent_type="verify-app",
+  prompt="Run full verification suite and report results"
+)
+```
+
+---
+
+## Component 7: Automated CLAUDE.md Updates
+
+### Overview
+
+GitHub Action that extracts patterns from commits and updates CLAUDE.md automatically.
+
+### Requirements
+
+| ID | Requirement | Priority |
+|----|-------------|----------|
+| REQ-AC-100 | Weekly GitHub Action runs pattern extraction | P1 |
+| REQ-AC-101 | Extracts common patterns from commit messages | P1 |
+| REQ-AC-102 | Updates CLAUDE.md Project-Learned Patterns section | P1 |
+| REQ-AC-103 | Creates PR for human review (no auto-merge) | P0 |
+
+### GitHub Action
+
+```yaml
+# .github/workflows/update-claude-md.yml
+name: Update CLAUDE.md
+
+on:
+  schedule:
+    - cron: '0 0 * * 0'  # Weekly on Sunday
+  workflow_dispatch:
+
+jobs:
+  extract-patterns:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Extract patterns from recent commits
+        run: |
+          # Get commits from last week
+          COMMITS=$(git log --since="1 week ago" --pretty=format:"%s")
+
+          # Run pattern extraction script
+          python scripts/extract_patterns.py "$COMMITS" > patterns.json
+
+      - name: Update CLAUDE.md
+        run: |
+          python scripts/update_claude_md.py patterns.json
+
+      - name: Create PR
+        uses: peter-evans/create-pull-request@v5
+        with:
+          title: "chore: update CLAUDE.md patterns"
+          body: "Automated pattern extraction from weekly commits"
+          branch: auto/claude-md-update
+          labels: automated
+```
+
+---
+
+## Implementation Phases
+
+### Phase 1: Verification Foundation (P0)
+- `/verify` command implementation
+- Stop hook verification gate
+- Basic integration tests
+
+**Estimated effort**: 4-6 hours
+**Dependencies**: None
+
+### Phase 2: PostToolUse Hooks (P0)
+- Formatting hook implementation
+- Multi-language formatter support
+- Performance optimization
+
+**Estimated effort**: 2-3 hours
+**Dependencies**: None (parallel with Phase 1)
+
+### Phase 3: Ralph Wiggum Mode (P1)
+- PROMPT.md templating
+- Stop hook with completion promises
+- Iteration tracking
+- fix_plan.md integration
+
+**Estimated effort**: 6-8 hours
+**Dependencies**: Phase 1 (verification)
+
+### Phase 4: Atomic Actions (P1)
+- `/test-and-commit` command
+- `/commit-push-pr` command
+- `/lint-fix` command
+
+**Estimated effort**: 3-4 hours
+**Dependencies**: Phase 1 (verification)
+
+### Phase 5: Subagents & Automation (P2)
+- verify-app subagent configuration
+- GitHub Action for CLAUDE.md updates
+- Pattern extraction scripts
+
+**Estimated effort**: 4-5 hours
+**Dependencies**: Phase 1, Phase 4
+
+---
+
+## File Structure
+
+```
+global/
+├── commands/
+│   ├── verify.md           # /verify skill
+│   ├── ralph.md            # /ralph skill
+│   ├── test-and-commit.md  # Atomic action
+│   ├── commit-push-pr.md   # Atomic action
+│   └── lint-fix.md         # Atomic action
+├── hooks/
+│   ├── stop_gate.sh        # Verification stop hook
+│   ├── ralph_stop.sh       # Ralph Wiggum stop hook
+│   └── post_tool_format.sh # PostToolUse formatter
+└── agents/
+    └── verify-app.json     # Verification subagent
+
+.github/
+└── workflows/
+    └── update-claude-md.yml
+
+scripts/
+├── extract_patterns.py     # Pattern extraction
+└── update_claude_md.py     # CLAUDE.md updater
+```
+
+---
+
+## Success Metrics
+
+| Metric | Target | Measurement |
+|--------|--------|-------------|
+| PR CI Pass Rate | >95% | GitHub Actions |
+| Verification Iterations | <2 avg | Command logs |
+| Ralph Mode Completion | >80% tasks | Completion promises |
+| Format Violations | 0 post-save | Git diff checks |
+| CLAUDE.md Freshness | <7 days | Last update date |
+
+---
+
+## Risk Assessment
+
+| Risk | Mitigation |
+|------|------------|
+| Stop hook blocks legitimate exits | Provide `/force-exit` override |
+| Ralph mode infinite loops | Max iterations safety (50 default) |
+| Formatter breaks code | Run tests after formatting |
+| Pattern extraction noise | Human review via PR |
+| Performance degradation | Async hooks, time limits |
+
+---
+
+## References
+
+- Boris Cherny's Claude Code Setup: https://x.com/bcherny/status/2007179832300581177
+- Ralph Wiggum Plugin: https://github.com/anthropics/claude-code/tree/main/plugins/ralph-wiggum
+- Research Document: `.claude/research/boris-cherny-ralph-wiggum-insights.md`
+- Claude Code Best Practices: https://docs.anthropic.com/en/docs/claude-code/best-practices