@zigrivers/scaffold 2.1.2 → 2.28.1

package/knowledge/core/ai-memory-management.md

@@ -0,0 +1,246 @@
+---
+name: ai-memory-management
+description: AI memory and context management patterns for Claude Code projects including modular rules, MCP memory servers, lifecycle hooks, decision logging, and external context integration
+topics: [ai-memory, claude-code, claude-rules, mcp-servers, lifecycle-hooks, context-management, session-handoff, decision-logging, mcp-knowledge-graph, context7]
+---
+
+# AI Memory Management
+
+AI coding agents forget everything between sessions. Memory management is the practice of making that forgetting graceful — ensuring the right context is available at the right time without drowning the agent in irrelevant information. This knowledge covers the full spectrum from lightweight rule files to persistent memory servers.
+
+## Summary
+
+### The Memory Hierarchy
+
+AI memory operates in layers, each with different persistence and token cost:
+
+| Layer | Persistence | Token Cost | Examples |
+|-------|------------|------------|----------|
+| **Context window** | Current session only | High (1:1) | Files read, conversation history |
+| **CLAUDE.md / rules** | Permanent (git-tracked) | Medium (loaded at start) | Conventions, commands, workflow |
+| **Auto-memory** | Cross-session (local) | Low (loaded on demand) | User preferences, project patterns |
+| **MCP memory server** | Cross-session (structured) | Low (queried on demand) | Decisions, lessons, error patterns |
+| **External docs** | Always current | Zero until queried | Library APIs, framework docs |
+
+### Core Principle: Signal Over Volume
+
+ETH Zurich research (2026) found that dumping context into AI sessions hurts more than it helps — 3% performance decrease with 20% cost increase for LLM-generated context files. The key insight: **only store what cannot be derived from the code itself.** Custom build commands, project-specific conventions, decision rationale, and team agreements are high-signal. Code patterns, file structure, and API shapes are low-signal (the agent can read the code).
+
+### The Three-Tier Memory Stack
+
+**Tier 1 — Modular Rules** (`.claude/rules/`): Path-scoped convention files loaded automatically based on what files the agent is working with. Zero manual effort after setup. Keeps CLAUDE.md lean.
+
+**Tier 2 — Persistent Memory** (MCP server + hooks): Structured cross-session memory that captures decisions, lessons, and error patterns automatically via lifecycle hooks. Survives session boundaries.
+
+**Tier 3 — External Context** (library doc servers): Version-specific documentation for project dependencies, queried on demand. Prevents API hallucination.
+
+## Deep Guidance
+
+### Tier 1: Modular Rules
+
+#### `.claude/rules/` Architecture
+
+Claude Code loads rule files from `.claude/rules/` based on path-scoping defined in YAML frontmatter. This replaces the pattern of stuffing everything into CLAUDE.md.
+
+**File structure:**
+```
+.claude/
+  rules/
+    code-style.md          # Always active — naming, formatting, imports
+    testing.md             # Active when working in test files
+    api-endpoints.md       # Active when working in API route files
+    database.md            # Active when working with schema/migration files
+    frontend.md            # Active when working in UI component files
+    memory-hygiene.md      # Always active — what to save/not save in memory
+```
+
+**Rule file format:**
+```markdown
+---
+description: TypeScript naming and import conventions
+globs: ["src/**/*.ts", "src/**/*.tsx"]
+---
+
+- Use camelCase for variables/functions, PascalCase for types/classes
+- Import order: external packages, then internal modules, then relative imports
+- Prefer named exports over default exports
+```
+
+**Key principles:**
+- Each rule file targets a specific concern and file pattern
+- Rules activate only when the agent works on matching files — no wasted tokens
+- Total rule content across all files should stay under 500 lines
+- Rules state what to do differently from defaults — don't restate obvious conventions
+- Extract rules from existing docs (coding-standards.md, tech-stack.md, git-workflow.md) rather than writing from scratch
+
+#### CLAUDE.md Optimization
+
+After creating rules, CLAUDE.md should use the pointer pattern:
+
+```markdown
+## Coding Conventions
+See `docs/coding-standards.md` for full reference. Key rules in `.claude/rules/code-style.md`.
+```
+
+This replaces inline convention blocks, keeping CLAUDE.md under 200 lines (the empirically-validated adherence threshold).
+
+### Tier 2: Persistent Memory
+
+#### MCP Memory Servers
+
+**Recommended: MCP Knowledge Graph** (`@modelcontextprotocol/server-memory`)
+- Official MCP server from the Model Context Protocol project
+- Stores entities, relations, and observations in a local JSON file
+- Zero setup: `npx -y @modelcontextprotocol/server-memory`
+- Entities persist across sessions — decisions, patterns, project facts
+- Best for: All projects (stable, official, zero dependencies beyond Node)
+
+**Alternative: Custom MCP server**
+- If the user has a preferred MCP memory server already installed, use it
+- The key requirement is that it exposes MCP tools for storing and retrieving structured memory
+- Examples from the ecosystem: Engram (if installed), hmem (if installed), ContextVault
+
+**Configuration pattern** (`.claude/settings.json`):
+```json
+{
+  "mcpServers": {
+    "memory": {
+      "command": "npx",
+      "args": ["-y", "@modelcontextprotocol/server-memory"],
+      "env": {
+        "MEMORY_FILE_PATH": ".claude/memory-graph.json"
+      }
+    }
+  }
+}
+```
+
+#### Lifecycle Hooks
+
+Hooks automate memory capture at key session events:
+
+**PreCompact** (highest value) — Triggers before context compression. Logs when compaction occurs for debugging context loss.
+
+```json
+{
+  "hooks": {
+    "PreCompact": [{
+      "type": "command",
+      "command": "echo \"$(date '+%Y-%m-%d %H:%M:%S') — Context compacting\" >> .claude/compaction-log.txt",
+      "timeout": 5000
+    }]
+  }
+}
+```
+
+File-logging for compaction events:
+```json
+{
+  "hooks": {
+    "PreCompact": [{
+      "type": "command",
+      "command": "date '+%Y-%m-%d %H:%M' >> .claude/compaction-log.txt && echo 'Context compacted' >> .claude/compaction-log.txt",
+      "timeout": 5000
+    }]
+  }
+}
+```
+
+**Stop** — Triggers when a session ends. Good for capturing session-level summaries.
+
+**PreToolUse** — Triggers before tool calls. Can log decisions about file modifications. Use sparingly — high frequency means high overhead.
+
+**Hook selection guidance:**
+- Start with PreCompact only — it captures the most value with least noise
+- Hook commands must produce a side effect (write to MCP server, append to file) — echoing to `/dev/null` provides zero value
+- Add Stop if sessions frequently end with unrecorded decisions
+- Avoid PreToolUse unless you have a specific logging need — it fires constantly
+
+#### Decision Logging
+
+Decisions are the highest-value memory type because they cannot be derived from code. A decision log captures what was chosen, what was rejected, and why.
+
+**Structure:**
+```
+docs/decisions/
+  DECISIONS.md          # Index of all decisions
+  001-auth-strategy.md  # Individual decision records
+  002-database-choice.md
+```
+
+**Decision entry format:**
+```markdown
+## DEC-001: JWT over session cookies for auth
+
+**Date:** 2026-03-27
+**Context:** Need stateless auth for API-first architecture
+**Decision:** Use JWT with short-lived access tokens + refresh tokens
+**Rejected:** Session cookies (requires sticky sessions), OAuth-only (too complex for MVP)
+**Consequences:** Need token refresh logic in frontend, need secure token storage
+```
+
+This complements ADRs (which cover architecture-level decisions) by capturing day-to-day implementation decisions that would otherwise be lost between sessions.
+
+#### Session Handoff Patterns
+
+When context hits limits, structured handoff preserves continuity:
+
+1. **Before compaction**: Save current task state, open questions, and recent decisions
+2. **After compaction**: Claude Code auto-reloads CLAUDE.md and auto-memory, but loses working context
+3. **Recovery**: Agent reads decision log and memory server to reconstruct working state
+
+The `/compact` command is the natural handoff point. A PreCompact hook that saves session state ensures nothing critical is lost.
+
+### Tier 3: External Context
+
+#### Library Documentation Servers
+
+AI agents hallucinate APIs — they generate plausible but incorrect function signatures, especially for rapidly-evolving libraries. External doc servers solve this by providing current, version-specific documentation on demand.
+
+**Context7** (by Upstash) — Most popular, fetches current library docs via MCP
+- Covers major frameworks (React, Next.js, Vue, Angular, etc.)
+- Free tier: 1,000 requests/month
+- Caution: had a security vulnerability (patched) — review before enabling
+
+**Nia** (by Nozomio) — Indexes codebases + 3,000+ pre-indexed packages
+- Cross-session context persistence
+- Deep research agent for complex questions
+- Y Combinator backed, more comprehensive than Context7
+
+**Docfork** — 9,000+ libraries, MIT license
+- "Cabinets" for project-specific documentation isolation
+- Self-hostable
+
+**Configuration pattern:**
+```json
+{
+  "mcpServers": {
+    "context7": {
+      "command": "npx",
+      "args": ["-y", "@upstash/context7-mcp@latest"]
+    }
+  }
+}
+```
+
+**When to enable:** Projects with 3+ external dependencies, especially rapidly-evolving frameworks (React, Next.js, Svelte). Skip for standard library-only projects or well-established stable APIs.
+
+### Anti-Patterns
+
+| Anti-Pattern | Why It Fails | Instead |
+|-------------|-------------|---------|
+| Dumping entire codebase into context | Drowns signal in noise, costs tokens | Let the agent read files on demand |
+| Storing code patterns in memory | Duplicates what's in the code; goes stale | Store decisions and rationale only |
+| Huge CLAUDE.md (500+ lines) | Adherence drops sharply above 200 lines | Use .claude/rules/ for specifics |
+| Memory without structure | Unstructured notes become unsearchable noise | Use categories (decision, lesson, error) |
+| Capturing everything | Token cost with diminishing returns | Capture what can't be derived from code |
+| Multiple overlapping memory tools | Conflicting context, duplicated entries | Pick one MCP server, use it consistently |
+
+### Integration with Beads
+
+When Beads is configured, memory complements task tracking:
+- **Beads** tracks what work to do (tasks, dependencies, status)
+- **Memory** tracks how to do work better (patterns, decisions, lessons)
+- Decision log entries can reference Beads task IDs for traceability
+- `tasks/lessons.md` remains the cross-session learning file; MCP memory adds structured queryability
+- Don't duplicate: if a pattern is in `tasks/lessons.md`, don't also store it in the MCP server

package/knowledge/core/api-design.md

@@ -499,3 +499,7 @@ Design APIs so that transient failures can be safely retried:
 **Missing rate limiting.** No protection against abusive or buggy clients sending excessive requests. Fix: implement rate limiting on all public endpoints. Return 429 with `Retry-After` header.
 
 **Ignoring CORS.** Frontend can't call the API because CORS headers are missing. Fix: configure CORS at API design time. Be specific about allowed origins — don't use `*` in production.
+
+## See Also
+
+- [testing-strategy](../core/testing-strategy.md) — Contract testing and API test patterns

package/knowledge/core/claude-md-patterns.md

@@ -0,0 +1,254 @@
+---
+name: claude-md-patterns
+description: Patterns for structuring CLAUDE.md files including section organization, rule authoring, pointer patterns, and merge strategies
+topics: [claude-md, ai-configuration, rule-files, memory-management, project-setup]
+---
+
+# CLAUDE.md Patterns
+
+CLAUDE.md is the primary instruction file for AI coding agents. It is loaded at the start of every session and defines how the agent should behave within a project. A well-structured CLAUDE.md dramatically improves agent adherence; a poorly structured one gets ignored or causes conflicts. This knowledge covers structure, authoring, the pointer pattern, and the merge strategy for multi-step pipeline updates.
+
+## Summary
+
+### Purpose
+
+CLAUDE.md is a project-level instruction file that AI agents (Claude Code, Codex, etc.) read at session start. It answers three questions:
+1. **What are the rules?** — Coding conventions, git workflow, testing requirements
+2. **How do I do common tasks?** — Key commands, PR workflow, deployment
+3. **What should I avoid?** — Anti-patterns, forbidden operations, common pitfalls
+
+### Section Organization
+
+A well-structured CLAUDE.md follows this order, from most-referenced to least:
+
+| Section | Purpose | Example Content |
+|---------|---------|-----------------|
+| **Core Principles** | 3-5 non-negotiable tenets | TDD, simplicity, no laziness |
+| **Project Overview** | What this project is (1-2 sentences) | "Prompt pipeline for scaffolding projects" |
+| **Key Commands** | Commands the agent runs constantly | `make check`, `make test`, `npm run dev` |
+| **Workflow** | How to do common operations | Branch, commit, PR, merge flow |
+| **Structure Quick Reference** | Where files go | Directory table with purpose |
+| **Environment** | Dev setup specifics | Build tool, test runner, linter |
+| **Rules** | Specific do/don't instructions | "Never push to main directly" |
+| **Self-Improvement** | Learning feedback loop | Lessons file, correction capture |
+| **Autonomous Behavior** | What the agent should do proactively | Fix bugs on sight, use subagents |
+| **Doc Lookup Table** | Where to find detailed docs | Question-to-document mapping |
+
+### Rule Authoring Best Practices
+
+Rules must be specific, actionable, and testable:
+
+**Good rules:**
+- "Run `make check` before every commit"
+- "Never push directly to main — always use branch + PR"
+- "Every commit message starts with `[BD-xxx]` task ID"
+
+**Bad rules:**
+- "Write clean code" — what does clean mean?
+- "Be careful with git" — what specific actions to take/avoid?
+- "Follow best practices" — which ones?
+
+### The Pointer Pattern
+
+Reference external docs instead of duplicating content inline:
+
+```markdown
+## Coding Conventions
+See `docs/coding-standards.md` for full reference. Key rules in `.claude/rules/code-style.md`.
+```
+
+This keeps CLAUDE.md under 200 lines (the empirically-validated adherence threshold) while preserving access to detailed docs. The agent reads referenced docs on demand rather than processing everything at session start.
+
+## Deep Guidance
+
+### Section Organization — Extended
+
+#### Front-Loading Critical Information
+
+Agents skim CLAUDE.md. The first 50 lines get the most attention. Place the most violated rules and most-used commands at the top. Core Principles and Key Commands should appear before any detailed documentation.
+
+#### The 200-Line Threshold
+
+Research and practical experience show that agent adherence drops sharply when CLAUDE.md exceeds ~200 lines. Beyond that length, agents start selectively ignoring instructions — particularly those in the middle or bottom of the file.
+
+Strategies to stay under 200 lines:
+- Use the pointer pattern for anything longer than 5 lines
+- Move path-scoped conventions to `.claude/rules/` files
+- Keep tables compact (no verbose descriptions)
+- Eliminate redundancy (same rule stated multiple ways)
+
+#### Section Templates
+
+**Core Principles** — 3-5 tenets, each a single sentence with a bold label:
+```markdown
+## Core Principles
+- **Simplicity First**: Make every change as simple as possible.
+- **TDD Always**: Write failing tests first, then make them pass.
+- **Prove It Works**: Never mark a task complete without demonstrating correctness.
+```
+
+**Key Commands** — Table format, sorted by frequency of use:
+```markdown
+## Key Commands
+| Command | Purpose |
+|---------|---------|
+| `make check` | Run all quality gates |
+| `make test` | Run test suite |
+| `make lint` | Run linters |
+```
+
+**Doc Lookup Table** — Question-to-document mapping:
+```markdown
+## When to Consult Other Docs
+| Question | Document |
+|----------|----------|
+| How do I branch and commit? | `docs/git-workflow.md` |
+| What are the coding conventions? | `docs/coding-standards.md` |
+```
+
+### Rule Authoring — Extended
+
+#### The Testability Criterion
+
+Every rule should be verifiable. If you cannot check whether the rule was followed, the rule is too vague.
+
+| Rule | Testable? | Fix |
+|------|-----------|-----|
+| "Write good tests" | No | "Every new function has at least one unit test" |
+| "Use proper naming" | No | "Use camelCase for variables, PascalCase for types" |
+| "Run `make check` before commits" | Yes | — |
+| "Never commit `.env` files" | Yes | — |
+
+#### Conflict Resolution
+
+Rules can conflict. When they do, the resolution order is:
+1. CLAUDE.md rules override general conventions
+2. More specific rules override more general rules
+3. Later rules override earlier rules (if truly contradictory)
+4. Project-specific rules override ecosystem defaults
+
+Document known conflicts explicitly: "This project uses tabs despite the TypeScript convention of spaces — see `.editorconfig`."
+
+#### Negative Rules vs. Positive Rules
+
+Prefer positive rules ("always do X") over negative rules ("never do Y") when possible. Positive rules tell the agent what to do; negative rules only eliminate one option from an infinite set.
+
+Exception: safety-critical negative rules are valuable. "Never push to main directly" and "Never commit secrets" are clearer as negatives.
+
+### Pointer Pattern — Extended
+
+#### When to Inline vs. Point
+
+| Content Type | Inline in CLAUDE.md | Point to External Doc |
+|-------------|--------------------|-----------------------|
+| Core principles | Yes | No |
+| Key commands table | Yes | No |
+| Workflow summary (5-10 lines) | Yes | Detailed version elsewhere |
+| Coding conventions (full) | No | `docs/coding-standards.md` |
+| Git workflow (full) | No | `docs/git-workflow.md` |
+| Project structure (full) | No | `docs/project-structure.md` |
+| Design system rules | No | `docs/design-system.md` |
+
+The rule: if the content is referenced multiple times per session, inline a summary. If it is referenced occasionally, point to it.
+
+#### Cross-Reference Format
+
+Use consistent pointer format throughout:
+```markdown
+See `docs/coding-standards.md` for full reference.
+```
+
+Not:
+```markdown
+Refer to the coding standards document for more details.
+```
+
+The first format gives the agent an exact file path to read. The second requires the agent to search for the file.
+
+### Merge Strategy for Multi-Step Pipeline Updates
+
+Seven pipeline steps modify CLAUDE.md during project scaffolding. Each step owns specific sections and must not overwrite sections owned by other steps. This section ownership model prevents destructive overwrites when steps execute sequentially.
+
+#### Section Ownership Map
+
+| Pipeline Step | CLAUDE.md Sections Owned | Operation |
+|--------------|-------------------------|-----------|
+| **beads** | Core Principles, Task Management (Beads commands), Self-Improvement, Autonomous Behavior | Creates initial skeleton |
+| **project-structure** | Project Structure Quick Reference | Adds/updates directory table |
+| **dev-env-setup** | Key Commands, Dev Environment | Adds/updates command table and env section |
+| **git-workflow** | Committing and Creating PRs, Parallel Sessions (Worktrees) | Adds/updates workflow sections |
+| **design-system** | Design System, Browser Testing | Adds/updates design system section |
+| **ai-memory-setup** | Pointer restructuring (cross-cutting) | Replaces inline content with pointers to `.claude/rules/` |
+| **automated-pr-review** | Code Review workflow | Adds/updates review workflow section |
+
+#### Merge Rules
+
+1. **Additive by default.** Each step adds its sections without modifying sections owned by other steps. If a section does not exist, create it. If it exists and belongs to this step, update it in-place.
+
+2. **Never delete unrecognized sections.** If CLAUDE.md contains sections not in the ownership map (user customizations, project-specific sections), preserve them. Move them to the end if they conflict with the expected layout, but never remove them.
+
+3. **Beads goes first.** The `beads` step creates the initial CLAUDE.md skeleton. All subsequent steps add to this skeleton. If `beads` was skipped (project does not use Beads), subsequent steps must still create their sections — they just skip the Beads-specific content.
+
+4. **ai-memory-setup is cross-cutting.** Unlike other steps that add sections, `ai-memory-setup` restructures existing sections by replacing inline content blocks with pointer references to `.claude/rules/` files. It operates across sections owned by other steps but only changes the representation (inline → pointer), not the substance.
+
+5. **claude-md-optimization consolidates.** The final consolidation step (`claude-md-optimization`) reviews the accumulated CLAUDE.md, removes redundancy introduced by incremental additions, fixes inconsistencies in terminology, and reorders for scannability. It operates on all sections but does not add new workflow steps or rules — only consolidates and clarifies what exists.
+
+6. **Preserve tracking comments.** Steps that add tracking comments (`<!-- scaffold:step-name v1 YYYY-MM-DD -->`) must preserve comments from other steps. These comments enable update detection.
+
+7. **Update mode is the norm.** After initial creation by `beads`, all subsequent steps operate in update mode. They check for existing content, preserve customizations, and update in-place rather than replacing.
+
+#### Conflict Scenarios
+
+**Two steps reference the same command.** Example: `dev-env-setup` adds `make check` to Key Commands and `git-workflow` references `make check` in the PR workflow. Resolution: the Key Commands table (owned by `dev-env-setup`) is the single source of truth for command definitions. Other sections reference commands but do not redefine them.
+
+**ai-memory-setup restructures a section another step just added.** This is expected and by design. The `ai-memory-setup` step runs after environment steps and converts verbose inline blocks to compact pointer references. The referenced docs must exist before the pointer is valid.
+
+**User adds custom sections between pipeline runs.** Subsequent pipeline steps must detect and preserve custom sections. Use the tracking comment (`<!-- scaffold:step-name -->`) to identify pipeline-managed sections vs. user-added sections.
+
+### Update Mode Handling
+
+#### Detecting Existing Content
+
+Every pipeline step that modifies CLAUDE.md implements mode detection:
+- If the file does not exist → create mode (write full skeleton)
+- If the file exists → update mode (modify owned sections in-place)
+
+Update mode is the common case. After the first `beads` run, every subsequent step encounters an existing CLAUDE.md.
+
+#### Preserving Custom Sections
+
+Users customize CLAUDE.md between pipeline runs. Common customizations:
+- Adding project-specific rules
+- Adding custom command aliases
+- Adding team-specific workflow notes
+- Adding integration-specific sections (deployment, monitoring)
+
+Pipeline steps must preserve all content they do not own. The safest pattern is:
+1. Read the existing CLAUDE.md
+2. Identify sections owned by this step (by heading or tracking comment)
+3. Replace only those sections with updated content
+4. Leave everything else untouched
+
+#### Additive Updates
+
+When updating a section, prefer additive changes over destructive ones:
+- Add new table rows rather than replacing the entire table
+- Add new subsections rather than rewriting the section
+- Append to lists rather than replacing them
+- Only remove content if it is demonstrably wrong or duplicated
+
+### Common Anti-Patterns
+
+**Inline everything.** CLAUDE.md becomes 500+ lines with full coding standards, complete git workflow, entire project structure. Agent adherence drops, load time increases, signal drowns in noise. Fix: use the pointer pattern. Keep CLAUDE.md under 200 lines.
+
+**Stale commands.** Key Commands table references `npm test` but the project switched to `bun test` two months ago. The agent runs the wrong command and wastes a cycle. Fix: keep Key Commands in sync with actual build tool configuration. The `claude-md-optimization` step verifies this.
+
+**Conflicting rules.** CLAUDE.md says "always use conventional commits" in one section and "use `[BD-xxx]` prefix" in another, with no guidance on which takes precedence. Fix: consolidate commit message rules in one place. If both apply, show the combined format: `[BD-42] feat(api): implement endpoint`.
+
+**Redundant instructions.** The same rule appears in Core Principles, Workflow, and Rules sections with slightly different wording. The agent may follow one version and violate another. Fix: state each rule once in its canonical section. Other sections reference it.
+
+**Missing doc lookup.** CLAUDE.md references "the git workflow" but does not specify the file path. The agent searches, guesses, or ignores the reference. Fix: always include exact file paths in references.
+
+**No update mode.** A pipeline step blindly writes a complete CLAUDE.md, overwriting sections added by earlier steps. Fix: every step that modifies CLAUDE.md must read it first, identify its owned sections, and update only those sections.
+
+**Over-specifying autonomous behavior.** CLAUDE.md micro-manages every agent decision: "If you see a typo, fix it. If you see a missing import, add it. If you see..." This wastes lines on things the agent would do anyway. Fix: autonomous behavior should cover non-obvious expectations — "fix bugs on sight," "use subagents for research," "re-plan when stuck." Skip obvious behaviors.