anvil-dev-framework 0.1.6

package/docs/INSTALLATION-RETRO-NOTES.md

@@ -0,0 +1,458 @@
+# Anvil Installation Retrospective Notes
+
+> Running notes during installation. Will be used to update the Anvil repo's INSTALLATION.md after completion.
+
+---
+
+## Phase 0: Backup & Prerequisites
+
+**Date**: 2025-12-17
+
+### What Worked Well
+- Backup script worked cleanly
+- All quality gate tools were already installed (may not be typical)
+- Linear MCP verification was straightforward
+
+### Issues Encountered
+- None
+
+### Changes for Updated Installation Plan
+1. **Team name discovery**: The installation plan assumes user knows their Linear team name. Added step to list teams first (`Linear:list_teams`) before verifying connection. User's team was "MyTeam" not "baby-gift-garden" (that's a project name, not team).
+
+2. **Prerequisite installation timing**: Current plan lists `brew install` commands but doesn't handle the case where tools are already installed. Consider adding version checks with "already installed" messaging.
+
+3. **Backup verification**: Should add a step to verify backup contents (file count, size) not just that directory was created.
+
+---
+
+## Phase 1: Global Installation
+
+**Date**: 2025-12-17
+
+### What Worked Well
+- Directory creation was idempotent (mkdir -p handles existing dirs)
+- Commands were already installed from earlier partial attempt — verified with diff
+- Template installation was clean (no conflicts with existing templates)
+- Existing skills preserved alongside new skill-template
+
+### Issues Encountered
+1. **cp -n confusion**: Used `cp -n` (no clobber) for commands, but all showed "not overwritten" which looked like failure. Actually the files were already identical from earlier run.
+2. **Grep verification failed misleadingly**: Initial grep-based verification said files were "different versions" but diff showed they were identical. Grep patterns were too specific.
+
+### Changes for Updated Installation Plan
+1. **Better idempotency messaging**: Instead of `cp -n`, use a proper check-then-copy approach
+2. **Add verification step to plan**: After each install section, include verification
+3. **Note about existing commands**: Plan should mention users may have existing commands and explain merge strategy
+4. **Simpler verification**: Use `diff -q` not grep patterns
+5. **Show what's NEW vs what EXISTS**: Installation output should clearly distinguish "newly installed" vs "already present"
+
+---
+
+## Phase 2: Project Installation
+
+**Date**: 2025-12-17
+
+### What Worked Well
+- Directory creation was straightforward
+- Existing hooks were more advanced than Anvil's — correctly kept them
+- Convention examples copied cleanly
+- product.md and constitution.md creation was guided by existing README
+
+### Issues Encountered
+1. **CLAUDE.md confusion**: Project's CLAUDE.md was actually a design extension config, not project context. Had to look at README.md instead.
+2. **Existing hooks decision**: Plan didn't address what to do when project already has more advanced hooks than Anvil provides.
+
+### Changes for Updated Installation Plan
+1. **Hooks merge strategy**: Add explicit guidance: "If your project already has hooks, compare functionality. Keep whichever is more complete. Anvil hooks are minimal starting points."
+
+2. **product.md guidance**: Add prompts/questions to help users fill in product.md:
+   - What does this project do in one sentence?
+   - Who are the primary users?
+   - What tech stack and why?
+   - What are you working on now vs later?
+
+3. **constitution.md should be project-specific**: Current template is generic. Consider making it more of a questionnaire:
+   - What security rules are absolute?
+   - What code quality rules matter most?
+   - What will you never do?
+
+4. **Check for existing docs**: Add step to look for README, existing CLAUDE.md, or other docs to inform product.md creation.
+
+5. **Convention examples note**: Add reminder that these are STARTING POINTS and should be customized to match actual project patterns.
+
+6. **Add coordination.md to Phase 2**: This file was created AFTER Phase 2 when we realized parallel coordination was still needed. Installation plan should include:
+   - Copy `project/coordination.md` to `.claude/coordination.md`
+   - Explain when it's needed (multiple terminals)
+   - Explain what it's NOT for (agent roles)
+
+---
+
+## Phase 3: Agent → Skill Conversion
+
+**Date**: 2025-12-17
+
+### What Worked Well
+- Skills already existed! Previous work had extracted domain knowledge from agents into 21 skills
+- Sub-agents were already decoupled from specialist agents (15 preserved)
+- Archive approach preserved all original files for reference
+- Rewrite of generalist was straightforward — mostly removing coordination, keeping skill triggers
+
+### Issues Encountered
+1. **"Conversion" was mostly archiving**: The real work (skill extraction) had already been done. This phase was really "remove multi-agent coordination."
+2. **Redundant command**: `explore-plan-code-test.md` duplicated generalist workflow — archived it.
+
+### Changes for Updated Installation Plan
+1. **Rename Phase 3**: Should be "Architecture Transition" not "Agent → Skill Conversion"
+   - For fresh installs: "Create Your First Skill"
+   - For migrations: "Archive Multi-Agent System"
+
+2. **Add pre-check**: Before this phase, check if skills already exist. If yes, phase is mostly archiving.
+
+3. **Make archive optional**: Some users may want to delete rather than archive old agents.
+
+4. **Clarify what's preserved**:
+   - Skills = domain knowledge (KEEP)
+   - Sub-agents = focused task execution (KEEP)
+   - Specialist agents = coordination overhead (ARCHIVE)
+   - Coordination files (comms.md, session-*) = (ARCHIVE)
+
+5. **For fresh installs**: This phase should be "Skill Creation Tutorial" instead:
+   - When to create a skill vs use sub-agent
+   - Skill template walkthrough
+   - Example: creating first project-specific skill
+
+6. **IMPORTANT: Don't conflate role-based agents with parallel coordination**
+   - Role-based agents (Backend, Frontend) = coordination overhead we removed
+   - Parallel instance coordination = still needed for multiple terminals
+   - Created `coordination.md` as lightweight replacement for `comms.md`
+   - Tracks: active sessions, files being worked on, interface contracts, activity log
+   - Does NOT track: agent roles, module ownership by role
+
+---
+
+## Phase 4: Quality Gates Setup
+
+**Date**: 2025-12-17
+
+### What Worked Well
+- CodeRabbit config installed cleanly
+- Pre-commit installation and activation was straightforward
+- Existing CI workflows were already comprehensive — no additions needed
+- Pre-commit auto-fixed trailing whitespace issues
+
+### Issues Encountered
+1. **Pre-commit deprecated stage names**: The template used `stages: [commit]` which is deprecated. Had to run `pre-commit migrate-config` to fix. Updated Anvil template to use `stages: [pre-commit]`.
+
+2. **Semgrep rule errors**: Original rules had multiple issues:
+   - Path patterns needed anchoring (`/src/**` not `src/**`)
+   - `as any` pattern was invalid TypeScript for semgrep
+   - `floating-promise` rule was too broad (14,000+ false positives)
+   
+3. **JSON check fails on tsconfig**: Pre-commit's `check-json` hook fails on TypeScript config files because they use JSONC (JSON with Comments). This is expected behavior.
+
+### Changes for Updated Installation Plan
+1. **Pre-commit template already fixed**: Use `stages: [pre-commit]` not `stages: [commit]`
+
+2. **Semgrep rules simplified**: Reduced from 187 lines to 99 lines. Focus on high-value rules:
+   - console.log detection
+   - hardcoded secrets
+   - any type annotation (not `as any`)
+   - SQL template injection
+   - Promise .then() without .catch()
+   - dangerouslySetInnerHTML
+   
+3. **Add note about JSONC**: Installation plan should mention that check-json will fail on tsconfig files — this is expected. Options:
+   - Exclude tsconfig files from check-json
+   - Accept the warnings
+   - Use a JSONC-aware checker
+
+4. **Semgrep rule testing**: Add step to test semgrep rules after installation:
+   ```bash
+   semgrep scan --config .semgrep/rules/ src/ --max-target-bytes 1000000
+   ```
+
+5. **CI workflow note**: If project already has CI, don't blindly overwrite. Check existing workflows first.
+
+---
+
+## Phase 5: Validation & Testing
+
+**Date**: 2025-12-17
+
+### What Worked Well
+- All 14 commands present and accessible
+- All 4 templates present
+- Project structure complete (specs, changes, handoffs, examples, skills, agents)
+- Skills (21) and sub-agents (15) preserved from original system
+- Linear integration working (connected, statuses accessible)
+- Hooks all present and configured
+- TypeScript passes, ESLint passes (warnings only)
+
+### Issues Encountered
+1. **Build fails**: Pre-existing project issue (missing SuperAdminSettingsPage import). Not related to Anvil installation.
+
+2. **ESLint warnings**: 1015 warnings in project — these are pre-existing, not from installation.
+
+### Changes for Updated Installation Plan
+1. **Add validation checklist**: Installation plan should include the verification steps run in Phase 5:
+   - Commands accessible (14)
+   - Templates present (4)
+   - Project directories created
+   - Skills inventory populated
+   - Linear connection verified
+   - Hooks configured
+   - Quality gates functional
+
+2. **Note about pre-existing issues**: Validation may surface pre-existing project issues (build failures, lint warnings). These are not Anvil problems.
+
+3. **Add "Quick Test" section**: Simple commands to verify installation:
+   ```bash
+   # Global commands
+   ls ~/.claude/commands/*.md | wc -l  # Should be 14+
+   
+   # Templates
+   ls ~/.claude/templates/*-template.md | wc -l  # Should be 4
+   
+   # Project structure
+   ls .claude/{specs,changes,handoffs,examples,skills} 2>/dev/null
+   
+   # Quality gates
+   pre-commit run --all-files
+   ```
+
+---
+
+## General Observations
+
+### CRITICAL: Role-Based Agents vs Parallel Coordination
+
+**This is the most important learning from the installation.**
+
+During Phase 3, we archived the multi-agent system including `comms.md`. This was almost a significant mistake. We conflated two separate concerns:
+
+#### Concern 1: Role-Based Agent Specialization (REMOVE)
+The original system had specialist agents:
+- Backend agent (agent-backend-001 through agent-backend-010)
+- Frontend agent (agent-frontend-001 through agent-frontend-010)
+- Core agent, Infra agent, Generalist agent
+
+**Problems with this approach:**
+- Overhead deciding "is this a backend or frontend task?"
+- Module ownership created artificial boundaries
+- Role assignment added complexity without proportional value
+- Single developer doesn't benefit from role specialization
+- Research showed single generalist + skills outperforms multi-agent
+
+**What we removed:**
+- Specialist agent definitions (backend.md, frontend.md, core.md, infra.md)
+- Role-based agent ID pools
+- Module ownership by role
+- Role-specific workflows
+
+#### Concern 2: Parallel Instance Coordination (KEEP)
+Claude Code users often run multiple terminals simultaneously:
+- Terminal A working on feature X
+- Terminal B working on feature Y
+- Terminal C doing a refactor
+
+**Problems without coordination:**
+- Two terminals modify the same file → merge conflicts
+- Terminal A creates an API, Terminal B doesn't know → duplicate work
+- No record of what each instance accomplished → lost context
+- No way to prevent overlapping work claims
+
+**This is NOT about roles. This is about multiple instances of the SAME generalist agent.**
+
+#### The Solution: coordination.md
+
+We created a NEW file `.claude/coordination.md` that handles parallel coordination WITHOUT role complexity:
+
+```
+# Session Coordination
+
+## Active Sessions
+| Session ID | Started | Working On | Status |
+|------------|---------|------------|--------|
+| session-001 | 14:30 | ENG-123: Button component | active |
+
+## Current Work (file-level conflict prevention)
+| Session ID | Files/Areas | Issue |
+|------------|-------------|-------|
+| session-001 | src/components/Button.tsx | ENG-123 |
+
+## Interface Contracts (new APIs being created)
+[contracts posted here before implementation]
+
+## Session Log (activity record)
+[append-only log of significant events]
+```
+
+**Key differences from comms.md:**
+
+| Old (comms.md) | New (coordination.md) |
+|----------------|----------------------|
+| Role-based agent registry | Session-based (any instance) |
+| Agent ID pools per role (001-010) | Simple session IDs |
+| Module ownership by role | File-level conflict tracking |
+| Role-specific workflows | Same workflow for all |
+| ~200+ lines | ~70 lines |
+
+#### Why This Matters for Anvil
+
+**The installation plan MUST clearly distinguish:**
+
+1. **Single-agent architecture** = Use ONE generalist agent definition, not specialists
+   - This is the "single agent + skills" model
+   - Skills provide domain expertise on-demand
+   - Sub-agents handle focused tasks
+   - No role-based coordination overhead
+
+2. **Parallel coordination** = Still needed when running multiple terminals
+   - coordination.md handles this
+   - Lightweight, role-agnostic
+   - Tracks sessions, files, contracts, activity
+   - Prevents conflicts without adding role complexity
+
+**Without this distinction, users will either:**
+- Remove coordination entirely → conflicts when running parallel instances
+- Keep role-based agents → unnecessary complexity for solo devs
+
+#### Implementation Requirements for Anvil Repo
+
+1. **Add coordination.md template** to `project/` directory
+2. **Add coordination section** to generalist agent template
+3. **Document in installation plan**:
+   - "If you ever run multiple Claude Code terminals simultaneously, you need coordination.md"
+   - "This is NOT about agent roles — it's about parallel instances"
+4. **Update README** to explain the distinction
+5. **Add to daily workflow**:
+   - Session start: Check coordination.md, register session
+   - Session end: Update coordination.md, log completion
+
+#### Example Scenario
+
+**Without coordination.md:**
+```
+You: "Work on the login page" (Terminal A)
+You: "Fix the auth bug" (Terminal B)
+Both terminals: Modify src/services/auth.ts simultaneously
+Result: Merge conflict, wasted work, confusion
+```
+
+**With coordination.md:**
+```
+Terminal A: Checks coordination.md, registers session-001, claims src/pages/Login.tsx
+Terminal B: Checks coordination.md, sees Terminal A is in auth area
+Terminal B: Either waits, picks different work, or coordinates via the file
+Result: No conflicts, clear record of who did what
+```
+
+#### Key Quotes to Remember
+
+> "Role-based agents add overhead. Parallel coordination prevents conflicts. These are not the same thing."
+
+> "A single generalist with 21 skills beats 5 specialists arguing about boundaries."
+
+> "You don't need a Backend Agent and a Frontend Agent. You need to know if another terminal is editing the same file."
+
+---
+
+## Final Recommendations for Repo Update
+
+> Completed after Phase 5
+
+### High Priority Changes
+
+1. **Add coordination.md template** — Essential for parallel instance coordination
+   - Location: `project/coordination.md`
+   - Status: ✅ DONE during installation
+
+2. **Fix pre-commit stage names** — Use `pre-commit` not deprecated `commit`
+   - Status: ✅ DONE during installation
+
+3. **Simplify Semgrep rules** — Remove false-positive-prone rules
+   - Reduced from 187 to 99 lines
+   - Status: ✅ DONE during installation
+
+4. **Document role-based vs parallel coordination distinction**
+   - This is the most important learning
+   - Status: ✅ Documented in retro notes (General Observations section)
+
+5. **Rename Phase 3** — "Architecture Transition" not "Agent → Skill Conversion"
+   - For fresh installs: "Create Your First Skill"
+   - For migrations: "Archive Multi-Agent System"
+
+### Medium Priority Changes
+
+1. **Add product.md prompts** — Questions to help users fill in their product.md
+2. **Add constitution.md questionnaire** — Help users define their non-negotiables
+3. **Better idempotency messaging** — Show "already installed" vs "newly installed"
+4. **Add Linear team discovery step** — Users may not know their team name
+5. **Exclude tsconfig from JSON check** — They use JSONC with comments
+
+### Nice to Have
+
+1. **Installation script** — Single command to run all phases
+2. **Verification script** — Automated Phase 5 checks
+3. **Rollback script** — Restore from backup if needed
+
+### Phase 3 Replacement Decision
+
+For the public Anvil repo:
+
+- [ ] Move current Phase 3 to "Appendix: Migration Guide"
+- [x] Replace with "Create Your First Skill" tutorial for fresh installs
+- [ ] Keep migration guide for teams coming from multi-agent systems
+
+**Recommended approach**: Split into two paths:
+1. **Fresh Install Path**: Phases 0-2, then "Create Your First Skill"
+2. **Migration Path**: Phases 0-2, then "Archive Multi-Agent System", then existing Phase 3
+
+---
+
+## Installation Time Summary
+
+| Phase | Planned | Actual | Notes |
+|-------|---------|--------|-------|
+| 0: Backup & Prerequisites | 30 min | 15 min | Tools already installed |
+| 1: Global Installation | 30 min | 20 min | Commands already present from earlier attempt |
+| 2: Project Installation | 1 hour | 30 min | Hooks already configured |
+| 3: Agent → Skill Conversion | 2-3 hours | 45 min | Skills already extracted |
+| 4: Quality Gates Setup | 1 hour | 40 min | CI already existed |
+| 5: Validation & Testing | 1-2 hours | 30 min | Straightforward verification |
+| **Total** | **6-8 hours** | **~3 hours** | Migration from existing setup |
+
+**Note**: Actual time was shorter because this was a migration from an existing multi-agent system, not a fresh install. Skills were already extracted, hooks were already configured, CI was already in place.
+
+
+---
+
+## Post-Installation Findings
+
+### Linear CLI vs Linear MCP
+
+**Issue discovered during `/orient` test:**
+
+The `/orient` command said "Query Linear MCP" but Claude Code in terminal tried to use a Linear CLI script. The CLI call failed because:
+1. Wrong subcommand: `list` instead of `list-issues`
+2. Wrong flag: `--state` instead of `--status`
+
+**Root cause**: Commands need explicit CLI syntax for Claude Code, not just "Query Linear MCP".
+
+**Fix applied to orient.md** (both global and Anvil repo):
+```bash
+# Correct syntax
+python3 ~/.claude/skills/linear-skill/scripts/linear.py list-issues --status "In Progress" --limit 10
+```
+
+**Lesson for Anvil**: Commands should provide explicit bash examples for any tool integrations, because:
+- Claude.ai uses MCP tools
+- Claude Code uses CLI/bash directly
+- Without explicit syntax, Claude Code will guess (often incorrectly)
+
+**Commands to review for similar issues**:
+- `/ready` - May also reference Linear
+- `/tasks` - Creates Linear issues
+- Any other commands that integrate with external tools
+
+**Design principle**: Anvil commands should be "context-aware" - providing both MCP instructions AND explicit CLI syntax where applicable.