anvil-dev-framework 0.1.6

package/docs/specifications/CCS-RESEARCH-REPORT.md

@@ -0,0 +1,169 @@
+# Exploration Report: Context Checkpoint System Research
+
+**Date**: 2026-01-07
+**Topic**: Auto-compact behavior and context preservation strategies
+**Trigger**: Tweet from @dani_avila7 about disabling auto-compact
+
+---
+
+## Research Question
+
+Can disabling Claude Code's auto-compact improve context preservation, and should this be incorporated into the Anvil framework?
+
+---
+
+## Sources Consulted
+
+### Primary Sources
+
+| Source | Type | Key Finding |
+|--------|------|-------------|
+| GitHub Issue #6689 | Feature request | Users requesting `--no-auto-compact` flag |
+| GitHub Issue #12053 | Feature request | Request to disable auto-compact buffer |
+| Claude Code `/config` | Official feature | Auto-compact CAN be disabled via toggle |
+| DEV.to article | User report | Disabling recovers ~22.5% context (45k tokens) |
+
+### Secondary Sources
+
+| Source | Type | Key Finding |
+|--------|------|-------------|
+| matsen.fhcrc.org | Best practices | Planning docs preserve understanding across sessions |
+| arsturn.com | Developer guide | Manual `/compact` at milestones recommended |
+| ClaudeLog documentation | Community resource | Strategic manual compacting over auto-compact |
+
+### Existing Anvil Components Analyzed
+
+| Component | Location | Relevance |
+|-----------|----------|-----------|
+| `statusline.sh` | `global/hooks/` | Has 70/85/95% thresholds, visual context bar |
+| `shard.md` | `global/commands/` | Task breakdown to prevent context overrun |
+| `handoff.md` | `global/commands/` | Session continuity documents |
+| `pre_compact.py` | `project/hooks/` | Backup transcript before compaction |
+
+---
+
+## Key Findings
+
+### Finding 1: Auto-Compact Can Be Disabled
+
+**Method**: `/config` command → toggle "Auto-compact enabled" OFF
+**Storage**: `~/.claude.json` as `{"autoCompactEnabled": false}`
+**Effect**: Recovers ~22.5% of context window (~45k tokens on 200k model)
+
+### Finding 2: Context Loss After Compaction is Well-Documented
+
+Multiple users report Claude "forgets" after auto-compaction:
+- CLAUDE.md instructions
+- Tool configurations
+- Learned patterns and decisions
+- Project-specific rules
+
+**Evidence quality**: Anecdotal but consistent across multiple sources
+
+### Finding 3: Trade-off Between Approaches
+
+| Approach | Pros | Cons |
+|----------|------|------|
+| Auto-compact ON | Sessions continue indefinitely | Lossy summarization, context degradation |
+| Auto-compact OFF | More usable context, no degradation | Sessions end hard at limit |
+| Manual compaction | Control over timing | Requires user awareness |
+| Structured handoff | Zero-loss preservation | Requires session restart |
+
+### Finding 4: Best Practices from External Research
+
+1. **Planning documents** preserve understanding across sessions
+2. **Small components** reduce context overhead per task
+3. **Front-load instructions** in CLAUDE.md for persistence
+4. **Subagents** for specialized tasks reduce main context usage
+5. **One task per chat** prevents context bloat
+6. **Checkpoint at milestones** not arbitrary points
+
+### Finding 5: Existing Anvil Coverage
+
+Anvil already has partial solutions:
+- `pre_compact.py` - Backs up before compaction (reactive)
+- `/handoff` - Creates continuity documents (manual)
+- Anti-pattern in CLAUDE.md - "Uncommitted work before compaction"
+- `statusline.sh` - Visual context monitoring
+
+**Gap**: No proactive, automatic checkpoint triggering
+
+---
+
+## Analysis
+
+### Why Simply Disabling Auto-Compact is Not Enough
+
+1. **Hard session endings** - Work stops abruptly at limit
+2. **No continuation path** - Next session starts cold
+3. **Lost work risk** - Uncommitted changes may be lost
+4. **No Linear integration** - Issue state not preserved
+
+### What's Needed: Intelligent Context Checkpoint System
+
+A system that:
+1. Monitors context proactively (not just visually)
+2. Triggers structured handoff at optimal breakpoints
+3. Commits WIP before checkpoint
+4. Updates Linear with progress
+5. Provides clear continuation path for next session
+6. Integrates with sharding to prevent overruns
+
+---
+
+## Threshold Model
+
+Based on existing `statusline.sh` thresholds:
+
+| Level | Threshold | Current Behavior | Proposed CCS Behavior |
+|-------|-----------|------------------|----------------------|
+| L0 | 0-69% | Green indicator | Normal operation |
+| L1 | 70-84% | Yellow warning | Alert + prepare handoff |
+| L2 | 85-94% | Red critical | Initiate handoff sequence |
+| L3 | 95%+ | COMPACT warning | Force immediate handoff |
+
+---
+
+## Recommendations
+
+### Recommendation 1: Do NOT Disable Auto-Compact by Default
+
+**Rationale**: Without a robust alternative, disabling creates worse UX (hard session endings).
+
+### Recommendation 2: Build Context Checkpoint System (CCS)
+
+**Rationale**: Provides intelligent alternative that works with or without auto-compact.
+
+### Recommendation 3: Integrate CCS with Ralph Wiggum
+
+**Rationale**: Ralph's continuous loops will frequently hit context limits; needs built-in checkpointing.
+
+### Recommendation 4: Enhance Existing Components
+
+Build on:
+- `statusline.sh` for monitoring
+- `/handoff` for document generation
+- `/shard` for task breakdown
+- `pre_compact.py` for backup patterns
+
+---
+
+## Open Questions for Design
+
+1. How does CCS integrate with Ralph Wiggum's loop architecture?
+2. Should CCS work alongside auto-compact or replace it entirely?
+3. How do we estimate task size before starting?
+4. What's the checkpoint document format for Ralph continuity?
+
+---
+
+## Next Steps
+
+1. Deep analysis of Ralph Wiggum system architecture
+2. Design CCS-Ralph integration points
+3. Update specification with Ralph requirements
+4. Create implementation plan
+
+---
+
+*Research conducted 2026-01-07. Specification draft created at `.claude/specs/current/SPEC-ANV-context-checkpoints.md`*

package/docs/specifications/PLAN-ANV-verification-ralph-wiggum.md

@@ -0,0 +1,403 @@
+---
+plan_id: PLAN-ANV-VRW
+spec_id: SPEC-ANV-VRW
+title: Verification Feedback Loops & Ralph Wiggum Implementation Plan
+status: draft
+created: 2026-01-04
+linear_parent: ANV-140
+---
+
+# Implementation Plan: Verification Feedback Loops & Ralph Wiggum Mode
+
+## Overview
+
+This plan breaks down the SPEC-ANV-VRW specification into actionable implementation phases. Each phase is designed to be completed in a single focused session (2-4 hours).
+
+---
+
+## Phase 1: Verification Command Foundation
+
+**Linear Issue**: ANV-141
+**Estimated Time**: 4-6 hours
+**Dependencies**: None
+
+### Objectives
+- Implement `/verify` skill with full feedback loop
+- Create verification configuration schema
+- Build iteration tracking system
+
+### Tasks
+
+1. **Create Verification Skill** (`global/commands/verify.md`)
+   - [ ] Define skill frontmatter and metadata
+   - [ ] Document execution steps
+   - [ ] Add argument parsing (custom commands)
+   - [ ] Include anti-patterns and best practices
+
+2. **Create Verification Runner** (`global/lib/verification_runner.py`)
+   - [ ] Load verification config from `.claude/settings.yaml`
+   - [ ] Execute test suite with output capture
+   - [ ] Execute lint with output parsing
+   - [ ] Execute typecheck with error extraction
+   - [ ] Return structured results (pass/fail/errors)
+
+3. **Implement Iteration Logic** (`global/lib/verify_iteration.py`)
+   - [ ] Parse verification failures
+   - [ ] Identify failing files and line numbers
+   - [ ] Track iteration count (max 3)
+   - [ ] Generate fix suggestions
+   - [ ] Re-run verification after fix attempts
+
+4. **Add Configuration Support**
+   - [ ] Create schema for `.claude/settings.yaml` verification section
+   - [ ] Support project-specific commands
+   - [ ] Add `max_iterations` setting
+   - [ ] Add `required_for_completion` flag
+
+### Acceptance Criteria
+- [ ] `/verify` runs configured test/lint/type commands
+- [ ] On failure, automatically reads failing files
+- [ ] Attempts up to 3 fix iterations
+- [ ] Reports clear pass/fail status with details
+- [ ] Respects project configuration
+
+### Test Cases
+- `/verify` with all passing checks
+- `/verify` with single failing test
+- `/verify` with multiple failures
+- `/verify` reaching max iterations
+- `/verify` with custom commands
+
+---
+
+## Phase 2: Stop Hook Verification Gate
+
+**Linear Issue**: ANV-142
+**Estimated Time**: 3-4 hours
+**Dependencies**: Phase 1
+
+### Objectives
+- Create stop hook that gates on verification
+- Implement force-exit override
+- Add gate event logging
+
+### Tasks
+
+1. **Create Stop Gate Hook** (`global/hooks/stop_gate.sh`)
+   - [ ] Check `ANVIL_REQUIRE_VERIFICATION` env var
+   - [ ] Check `ANVIL_FORCE_EXIT` override
+   - [ ] Run test suite verification
+   - [ ] Run lint verification
+   - [ ] Run typecheck verification
+   - [ ] Return non-zero on any failure
+
+2. **Create Force Exit Command** (`global/commands/force-exit.md`)
+   - [ ] Set `ANVIL_FORCE_EXIT=true`
+   - [ ] Log override reason
+   - [ ] Trigger actual exit
+
+3. **Update Settings Hook Registration**
+   - [ ] Add stop_gate.sh to Stop hooks
+   - [ ] Document hook ordering
+
+4. **Add Gate Logging**
+   - [ ] Create log file at `.claude/logs/stop_gate.log`
+   - [ ] Log each gate event with timestamp
+   - [ ] Log pass/fail status and reason
+
+### Acceptance Criteria
+- [ ] Cannot exit with failing tests (exit code 1)
+- [ ] Clear message shows what's blocking exit
+- [ ] `/force-exit` bypasses gate with warning
+- [ ] All gate events logged
+
+### Test Cases
+- Exit attempt with passing verification
+- Exit attempt with failing tests
+- Exit attempt with lint errors
+- Force exit override
+- Verification disabled scenario
+
+---
+
+## Phase 3: PostToolUse Formatting Hook
+
+**Linear Issue**: ANV-143
+**Estimated Time**: 2-3 hours
+**Dependencies**: None (can parallel Phase 1-2)
+
+### Objectives
+- Auto-format code after every Edit/Write
+- Support multiple formatters per language
+- Ensure <500ms execution
+
+### Tasks
+
+1. **Create PostToolUse Hook** (`global/hooks/post_tool_format.sh`)
+   - [ ] Extract `CLAUDE_FILE_PATH` from environment
+   - [ ] Skip non-existent and binary files
+   - [ ] Skip node_modules, .git, etc.
+   - [ ] Detect file type from extension
+   - [ ] Run appropriate formatter
+
+2. **Formatter Configuration**
+   - [ ] TypeScript/JavaScript: prettier
+   - [ ] Python: black or ruff
+   - [ ] Go: gofmt
+   - [ ] Bash: shfmt
+   - [ ] Add fallback when formatter unavailable
+
+3. **Performance Optimization**
+   - [ ] Measure execution time
+   - [ ] Skip if formatter not installed
+   - [ ] Use `--silent` flags to reduce output
+
+4. **Register Hook**
+   - [ ] Add to PostToolUse with matcher "Edit|Write"
+   - [ ] Test with both Edit and Write operations
+
+### Acceptance Criteria
+- [ ] All JS/TS files formatted after Edit/Write
+- [ ] Python files formatted after Edit/Write
+- [ ] Execution completes in <500ms
+- [ ] No errors on missing formatters
+
+### Test Cases
+- Edit TypeScript file → formatted
+- Write new Python file → formatted
+- Edit binary file → skipped
+- Write to node_modules → skipped
+- No prettier installed → graceful skip
+
+---
+
+## Phase 4: Ralph Wiggum Core Infrastructure
+
+**Linear Issue**: ANV-144
+**Estimated Time**: 4-5 hours
+**Dependencies**: Phase 1, Phase 2
+
+### Objectives
+- Implement Ralph Wiggum stop hook
+- Create PROMPT.md templating
+- Build iteration tracking system
+
+### Tasks
+
+1. **Create Ralph Stop Hook** (`global/hooks/ralph_stop.sh`)
+   - [ ] Read iteration count from temp file
+   - [ ] Increment iteration counter
+   - [ ] Check max iterations limit
+   - [ ] Check for `<promise>COMPLETE</promise>`
+   - [ ] Check for `<fatal>` error signal
+   - [ ] Return appropriate exit code
+
+2. **Create Prompt Templates** (`global/templates/ralph/`)
+   - [ ] `PROMPT.md.template` - Main task prompt
+   - [ ] `fix_plan.md.template` - TODO list template
+   - [ ] Variable substitution support
+
+3. **Create `/ralph` Skill** (`global/commands/ralph.md`)
+   - [ ] `start [description]` - Initialize Ralph mode
+   - [ ] `status` - Show current progress
+   - [ ] `stop` - Force exit Ralph mode
+   - [ ] Document constraints and rules
+
+4. **Iteration State Management**
+   - [ ] Store state in `.claude/ralph-state.json`
+   - [ ] Track: iteration count, last action, remaining items
+   - [ ] Clean up on completion
+
+### Acceptance Criteria
+- [ ] `/ralph start` creates PROMPT.md and fix_plan.md
+- [ ] Stop hook blocks exit until `<promise>COMPLETE</promise>`
+- [ ] Max iterations safety stops runaway execution
+- [ ] `/ralph status` shows current progress
+- [ ] `/ralph stop` cleanly exits
+
+### Test Cases
+- Start Ralph mode with simple task
+- Verify stop hook blocks incomplete exit
+- Verify completion promise allows exit
+- Verify max iterations trigger
+- Status shows accurate progress
+
+---
+
+## Phase 5: Ralph Wiggum Execution Loop
+
+**Linear Issue**: ANV-145
+**Estimated Time**: 3-4 hours
+**Dependencies**: Phase 4
+
+### Objectives
+- Implement the bash execution loop
+- Add subagent integration for research
+- Enable self-learning AGENT.md updates
+
+### Tasks
+
+1. **Create Execution Script** (`global/tools/ralph-loop.sh`)
+   - [ ] Read PROMPT.md
+   - [ ] Pipe to Claude CLI
+   - [ ] Check exit code
+   - [ ] Loop until completion or max iterations
+
+2. **Subagent Integration**
+   - [ ] Document research subagent usage (max 500)
+   - [ ] Document build subagent limit (1)
+   - [ ] Add context pollution warnings
+
+3. **Self-Learning Updates**
+   - [ ] Allow Claude to append to AGENT.md
+   - [ ] Extract learnings from completed iterations
+   - [ ] Persist patterns across sessions
+
+4. **Progress Reporting**
+   - [ ] Real-time iteration updates
+   - [ ] TODO completion tracking
+   - [ ] Time per iteration metrics
+
+### Acceptance Criteria
+- [ ] Loop executes until completion
+- [ ] Each iteration processes one TODO item
+- [ ] Subagent usage follows limits
+- [ ] AGENT.md updated with learnings
+- [ ] Progress visible during execution
+
+### Test Cases
+- Complete 3-item TODO list
+- Verify one-item-per-iteration constraint
+- Test subagent spawning during research
+- Test AGENT.md learning updates
+
+---
+
+## Phase 6: Atomic Action Commands
+
+**Linear Issue**: ANV-146
+**Estimated Time**: 3-4 hours
+**Dependencies**: Phase 1
+
+### Objectives
+- Create rapid inner-loop commands
+- Ensure atomic execution (all-or-nothing)
+
+### Tasks
+
+1. **Create `/test-and-commit`** (`global/commands/test-and-commit.md`)
+   - [ ] Run test suite
+   - [ ] If pass: stage and commit
+   - [ ] If fail: report errors, no commit
+   - [ ] Accept commit message argument
+
+2. **Create `/commit-push-pr`** (`global/commands/commit-push-pr.md`)
+   - [ ] Stage all changes
+   - [ ] Create commit with message
+   - [ ] Push to remote
+   - [ ] Create PR with `gh pr create`
+   - [ ] Return PR URL
+
+3. **Create `/lint-fix`** (`global/commands/lint-fix.md`)
+   - [ ] Run linter with `--fix`
+   - [ ] Report remaining unfixable issues
+   - [ ] Stage auto-fixed changes
+
+4. **Error Handling**
+   - [ ] Clear failure messages
+   - [ ] No partial state on failure
+   - [ ] Rollback on error
+
+### Acceptance Criteria
+- [ ] `/test-and-commit` only commits if tests pass
+- [ ] `/commit-push-pr` creates complete PR
+- [ ] `/lint-fix` auto-fixes lintable issues
+- [ ] All commands have clear success/failure output
+
+### Test Cases
+- `/test-and-commit` with passing tests
+- `/test-and-commit` with failing tests
+- `/commit-push-pr` full workflow
+- `/lint-fix` with fixable issues
+
+---
+
+## Phase 7: Verify Subagent & CLAUDE.md Automation
+
+**Linear Issue**: ANV-147
+**Estimated Time**: 4-5 hours
+**Dependencies**: Phase 1, Phase 6
+
+### Objectives
+- Create dedicated verification subagent
+- Implement GitHub Action for CLAUDE.md updates
+
+### Tasks
+
+1. **Create Verify Subagent** (`global/agents/verify-app.json`)
+   - [ ] Define agent configuration
+   - [ ] Limit to verification tools
+   - [ ] Create invocation pattern
+
+2. **Create Pattern Extraction Script** (`scripts/extract_patterns.py`)
+   - [ ] Parse commit messages
+   - [ ] Identify recurring patterns
+   - [ ] Output structured JSON
+
+3. **Create CLAUDE.md Updater** (`scripts/update_claude_md.py`)
+   - [ ] Read patterns.json
+   - [ ] Update Project-Learned Patterns table
+   - [ ] Preserve existing content
+
+4. **Create GitHub Action** (`.github/workflows/update-claude-md.yml`)
+   - [ ] Weekly schedule (Sundays)
+   - [ ] Manual trigger support
+   - [ ] Pattern extraction step
+   - [ ] CLAUDE.md update step
+   - [ ] PR creation (no auto-merge)
+
+### Acceptance Criteria
+- [ ] verify-app subagent runs all checks
+- [ ] Pattern extraction identifies common patterns
+- [ ] CLAUDE.md updated with new patterns
+- [ ] PR created for human review
+
+### Test Cases
+- Invoke verify-app subagent
+- Extract patterns from test commits
+- Update CLAUDE.md with new patterns
+- GitHub Action runs on schedule
+
+---
+
+## Implementation Schedule
+
+| Phase | Name | Blocked By | Priority |
+|-------|------|------------|----------|
+| 1 | Verification Command | None | P0 |
+| 2 | Stop Hook Gate | Phase 1 | P0 |
+| 3 | PostToolUse Hook | None | P0 |
+| 4 | Ralph Core | Phases 1, 2 | P1 |
+| 5 | Ralph Execution | Phase 4 | P1 |
+| 6 | Atomic Actions | Phase 1 | P1 |
+| 7 | Subagent & Automation | Phases 1, 6 | P2 |
+
+**Note**: Phases 1, 2, and 3 can start in parallel. Phase 6 can parallel with Phases 4-5.
+
+---
+
+## Definition of Done
+
+For each phase:
+- [ ] All tasks completed
+- [ ] All acceptance criteria met
+- [ ] All test cases verified
+- [ ] Documentation updated
+- [ ] Code reviewed
+- [ ] Changes committed and pushed
+
+For overall feature:
+- [ ] All 7 phases complete
+- [ ] Integration testing passed
+- [ ] Demo to stakeholders
+- [ ] Retrospective written