forgedev 1.1.3 → 1.2.0

package/docs/04-errata-and-verification-checklist.md

@@ -0,0 +1,284 @@
+# Errata & Verification Checklist
+## What to Test Before You Trust Any of These Documents
+
+---
+
+## Status: These Documents Are a Starting Framework, Not Tested Production Code
+
+I generated these documents by synthesizing best practices from Anthropic's
+official docs, community guides, and your specific CLAUDE.md. However:
+
+- The hook scripts have NOT been tested against a live Claude Code session
+- The subagent definitions have NOT been tested with actual task delegation
+- The prompt templates are patterns that need tuning to your codebase
+- Claude Code's API surface changes frequently — verify against current docs
+
+**Your job before relying on any of this: run each piece in isolation, confirm
+it works, then integrate.**
+
+---
+
+## Known Errors to Fix
+
+### 1. PostToolUse Hook — Wrong Variable Reference
+
+**File:** `multi-agent-verification.md` and `claude-code-mastery-playbook.md`
+
+**Problem:** The auto-lint PostToolUse hook references `$TOOL_INPUT_FILE` which
+may not exist as an environment variable. Hook input comes via JSON on stdin.
+
+**Fix:** Replace the post-edit.sh hook with:
+
+```bash
+#!/bin/bash
+# .claude/hooks/post-edit.sh
+INPUT=$(cat)
+FILE_PATH=$(echo "$INPUT" | jq -r '.tool_input.file_path // empty')
+
+if [ -z "$FILE_PATH" ]; then exit 0; fi
+
+# Auto-lint TypeScript files
+if [[ "$FILE_PATH" == *.ts || "$FILE_PATH" == *.tsx ]]; then
+  npx eslint --fix "$FILE_PATH" 2>&1 || true
+fi
+
+# Auto-lint Python files
+if [[ "$FILE_PATH" == *.py ]]; then
+  cd "$CLAUDE_PROJECT_DIR/backend" && ruff check --fix "$FILE_PATH" 2>&1 || true
+fi
+
+exit 0
+```
+
+**Dependency:** Requires `jq` installed (`sudo apt install jq` or `brew install jq`).
+
+### 2. Stop Hook — Infinite Loop Risk
+
+**File:** `claude-code-mastery-playbook-v2.md`
+
+**Problem:** The stop hook must check `stop_hook_active` to prevent infinite loops.
+The scripts include this check, but if your stop hook is slow (full test suite)
+AND also blocks with exit code 2, Claude will run the hook again after fixing,
+creating a potential long loop.
+
+**Mitigation:** Keep stop hooks fast (under 30 seconds). If your full test suite
+takes >60 seconds, move it to TaskCompleted instead of Stop. Use Stop only for
+fast checks (typecheck, lint).
+
+### 3. Agent `disallowedTools` Format
+
+**File:** `multi-agent-verification.md`
+
+**Problem:** The subagent frontmatter uses `disallowedTools: Write, Edit, MultiEdit, Notebook`.
+Verify the exact YAML format against your Claude Code version. The official docs show
+the format as a YAML list:
+
+```yaml
+disallowedTools:
+  - Write
+  - Edit
+  - MultiEdit
+  - Notebook
+```
+
+Or as a comma-separated string — check which your version supports.
+
+### 4. `$CLAUDE_PROJECT_DIR` in settings.json
+
+**File:** Both playbooks
+
+**Problem:** Hook commands in settings.json use `$CLAUDE_PROJECT_DIR`. This IS a valid
+environment variable provided by Claude Code to hooks, but verify it resolves
+correctly in your environment. Some users report issues when running from nested directories.
+
+**Test:** Add a simple hook first that just echoes the variable:
+```json
+{
+  "hooks": {
+    "SessionStart": [{
+      "hooks": [{
+        "type": "command",
+        "command": "echo \"Project dir: $CLAUDE_PROJECT_DIR\" >&2"
+      }]
+    }]
+  }
+}
+```
+
+### 5. Mermaid Diagram Rendering
+
+**File:** `universal-prompt-library.md`
+
+**Problem:** The Mermaid diagram uses `style` directives with `fill` colors. Not all
+Mermaid renderers support this (GitHub does, some VS Code extensions don't).
+
+**Test:** Paste the Mermaid block into https://mermaid.live to verify rendering.
+If your tool doesn't support `style`, remove those lines — the diagram works without them.
+
+---
+
+## Verification Checklist (Test Each Piece Before Trusting It)
+
+### Phase 1: Verify Hooks (Do This First)
+
+```bash
+# Step 1: Verify jq is installed
+jq --version
+# If not: brew install jq (Mac) or sudo apt install jq (Linux)
+
+# Step 2: Create the hooks directory
+mkdir -p .claude/hooks
+chmod +x .claude/hooks/*.sh
+
+# Step 3: Test protected files hook manually
+echo '{"tool_input":{"file_path":".env"}}' | .claude/hooks/protect-files.sh
+echo $?  # Should print 2 (blocked)
+
+echo '{"tool_input":{"file_path":"src/App.tsx"}}' | .claude/hooks/protect-files.sh
+echo $?  # Should print 0 (allowed)
+
+# Step 4: Test post-edit hook manually
+echo '{"tool_input":{"file_path":"src/App.tsx"}}' | .claude/hooks/post-edit.sh
+echo $?  # Should print 0
+
+# Step 5: Test stop hook manually
+echo '{"stop_hook_active": false}' | .claude/hooks/stop-quality-gate.sh
+# Should run your 5 checks (or fail if you're not in the right directory)
+
+echo '{"stop_hook_active": true}' | .claude/hooks/stop-quality-gate.sh
+echo $?  # Should print 0 (skip when already in stop hook)
+
+# Step 6: Add hooks to settings.json
+# Start with JUST the protected files hook
+# Run a Claude Code session and try to edit .env
+# Verify it gets blocked
+
+# Step 7: Add remaining hooks one at a time
+# Test each in a real session before adding the next
+```
+
+### Phase 2: Verify Subagents
+
+```bash
+# Step 1: Create one agent file
+mkdir -p .claude/agents
+# Copy the code-quality-reviewer.md content to .claude/agents/code-quality-reviewer.md
+
+# Step 2: In a Claude Code session, test delegation
+# Type: "Task the code-quality-reviewer with: review src/App.tsx"
+# Verify:
+#   - It creates a separate context (you see "Delegating to code-quality-reviewer")
+#   - It can READ files
+#   - It CANNOT write/edit files (disallowedTools enforced)
+#   - It returns a structured report
+
+# Step 3: If disallowedTools aren't enforced, check the YAML format
+# Try both formats:
+#   disallowedTools: Write, Edit     (comma-separated)
+#   disallowedTools:                  (YAML list)
+#     - Write
+#     - Edit
+
+# Step 4: Test each agent one at a time before relying on the chain
+```
+
+### Phase 3: Verify Commands
+
+```bash
+# Step 1: Create one command
+mkdir -p .claude/commands
+# Copy audit-spec.md content to .claude/commands/audit-spec.md
+
+# Step 2: In a Claude Code session, test it
+# Type: /project:audit-spec phases/some-spec.md
+# Verify it produces a structured table
+
+# Step 3: Test each command one at a time
+```
+
+### Phase 4: Verify Skills
+
+```bash
+# Step 1: Create one skill
+mkdir -p .claude/skills/playwright
+# Copy playwright testing skill content to .claude/skills/playwright/SKILL.md
+
+# Step 2: In a Claude Code session, ask Claude to write a Playwright test
+# Verify it loaded the skill (it should follow your patterns, not generic ones)
+
+# Step 3: Test each skill one at a time
+```
+
+### Phase 5: Verify Directory-Scoped CLAUDE.md
+
+```bash
+# Step 1: Create backend/CLAUDE.md with backend rules
+# Step 2: In a Claude Code session, ask Claude to edit a backend file
+# Step 3: Ask Claude: "What rules are you following for this file?"
+# Verify it mentions the backend-specific rules
+
+# Step 4: Do the same for src/CLAUDE.md and e2e/CLAUDE.md
+```
+
+---
+
+## Things That Are Intentionally Generic (And Need Your Customization)
+
+| Document Section | What You Need to Customize |
+|-----------------|---------------------------|
+| Stop hook — 5 checks | Replace with YOUR project's actual lint/type/test commands |
+| Protected files list | Add YOUR project's protected paths (migrations, configs) |
+| Subagent security checklist | Add YOUR project's security patterns (auth helpers, RBAC checks) |
+| Playwright skill patterns | Add YOUR project's fixture patterns, auth setup, test IDs |
+| Frontend/Backend skill patterns | Add YOUR project's actual component patterns, API patterns |
+| Prompt templates | Replace `[bracketed placeholders]` with your real context |
+| Spec file paths | Replace `phases/[module].md` with your actual spec file paths |
+
+---
+
+## Things I Cannot Verify (You Must Test Yourself)
+
+1. **Your Claude Code version** — hooks API has changed across versions.
+   Run `claude --version` and check the changelog for your version.
+
+2. **Your project's test suite speed** — if tests take >60 seconds,
+   the Stop hook will slow down every task completion significantly.
+   Move slow tests to TaskCompleted or a manual command instead.
+
+3. **Your token budget** — running 5 subagents on every feature adds
+   ~50-80K tokens. On Max plan that's fine. On Pro plan, you'll hit
+   limits faster. Adjust by using fewer agents or running them less often.
+
+4. **Subagent model availability** — the agent files specify `model: opus`
+   or `model: sonnet`. Verify these model names match what your plan supports.
+
+5. **VS Code extension version** — the extension updates independently of
+   the CLI. Some features (hooks, subagents) may behave slightly differently
+   in the VS Code panel vs terminal.
+
+6. **Team/Enterprise features** — Claude Code Review ($15-25/PR) requires
+   Team or Enterprise plan. Subagents and hooks work on all plans.
+
+---
+
+## Recommended Rollout Order
+
+Don't try to implement everything at once. Do this over 1-2 weeks:
+
+**Week 1:**
+- Day 1: Slim down your CLAUDE.md to ~150 lines (move specs to skills)
+- Day 2: Set up and test the 3 essential hooks (protect, post-edit, stop)
+- Day 3: Create the backend/CLAUDE.md and src/CLAUDE.md directory scoping
+- Day 4-5: Use the new setup on a real task. Note what works and what doesn't.
+
+**Week 2:**
+- Day 1: Create and test 2 subagents (code-quality + security)
+- Day 2: Create the audit commands (audit-spec, audit-wiring)
+- Day 3: Create remaining skills and subagents
+- Day 4-5: Run the full verification chain on an existing module. Tune based on results.
+
+**Ongoing:**
+- Treat CLAUDE.md, hooks, skills, and agents like code: review when things
+  go wrong, prune regularly, and test changes by observing behavior.
+- After fixing a recurring issue, add it to Known Pitfalls in CLAUDE.md
+  or create a new hook to prevent it deterministically.