@stackmemoryai/stackmemory 0.5.58 → 0.5.61

package/scripts/gepa/README.md

@@ -0,0 +1,275 @@
+# Prompt Forge (GEPA)
+
+**Genetic Eval-driven Prompt Algorithm**
+
+Automatically evolve and optimize your `CLAUDE.md` system prompts using AI-powered evolutionary algorithms.
+
+## Auto-Optimize on Save
+
+```bash
+# Start watcher - auto-optimizes when CLAUDE.md changes
+node scripts/gepa/hooks/auto-optimize.js watch ./CLAUDE.md
+```
+
+Output shows before/after comparison:
+```
+╔════════════════════════════════════════════════════════════╗
+║  BEFORE / AFTER COMPARISON                                  ║
+╠════════════════════════════════════════════════════════════╣
+║  Metric              Before      After       Change         ║
+╠════════════════════════════════════════════════════════════╣
+║  Lines                  125        142    +17 (+14%)        ║
+║  Est. Tokens            873        920    +47 (+5%)         ║
+║  MUST rules               1          3     +2 (+200%)       ║
+║  NEVER rules              3          5     +2 (+67%)        ║
+╚════════════════════════════════════════════════════════════╝
+
+Section Changes:
+  Added:
+    + Error Handling
+    + Performance Guidelines
+
+Summary:
+  Token budget: +47 tokens
+  Rule density: +4 explicit rules
+```
+
+## How It Works
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                         GEPA Loop                                │
+│                                                                  │
+│  ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐  │
+│  │  Seed    │───►│  Mutate  │───►│   Eval   │───►│  Select  │  │
+│  │ Prompt   │    │ (AI gen) │    │ (Claude) │    │  (best)  │  │
+│  └──────────┘    └──────────┘    └──────────┘    └────┬─────┘  │
+│       ▲                                               │        │
+│       │              ┌──────────┐                     │        │
+│       └──────────────┤ Reflect  │◄────────────────────┘        │
+│                      │(insights)│                              │
+│                      └──────────┘                              │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Quick Start
+
+```bash
+# 1. Initialize with your current CLAUDE.md
+node scripts/gepa/optimize.js init ./CLAUDE.md
+
+# 2. Run full optimization (10 generations)
+node scripts/gepa/optimize.js run
+
+# 3. Apply the best result
+cp scripts/gepa/generations/current ./CLAUDE.md
+```
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `init [path]` | Initialize with a CLAUDE.md file |
+| `mutate` | Generate new prompt variants |
+| `eval [variant]` | Run evals on a specific variant |
+| `score` | Score all variants and select best |
+| `run [N]` | Full optimization loop for N generations |
+| `status` | Show current optimization status |
+| `diff [a] [b]` | Compare two variants |
+
+## Mutation Strategies
+
+GEPA uses 6 mutation strategies, cycling through them:
+
+1. **rephrase** - Reword for clarity without changing meaning
+2. **add_examples** - Add concrete examples where abstract
+3. **remove_redundancy** - DRY up repetitive instructions
+4. **restructure** - Reorganize for better flow
+5. **add_constraints** - Add guardrails for failure modes
+6. **simplify** - Break down complex rules
+
+## Reflection (Key Innovation)
+
+Unlike random mutations, GEPA analyzes **why** prompts fail:
+
+```bash
+# Analyze session patterns
+node scripts/gepa/hooks/reflect.js analyze
+
+# Generate targeted improvements
+node scripts/gepa/hooks/reflect.js reflect
+```
+
+The reflection engine examines:
+- Common error patterns
+- Tool call success rates
+- User feedback (thumbs up/down)
+- Performance by variant
+
+Then generates **targeted** mutation suggestions.
+
+## Hook Integration
+
+To track real usage for evals, add to `~/.claude/settings.json`:
+
+```json
+{
+  "hooks": {
+    "postToolCall": [
+      {
+        "command": "node ~/.claude/gepa/hooks/eval-tracker.js track-tool"
+      }
+    ],
+    "postSession": [
+      {
+        "command": "node ~/.claude/gepa/hooks/eval-tracker.js save"
+      }
+    ]
+  }
+}
+```
+
+Or use the StackMemory daemon integration:
+
+```bash
+# Add to your .env
+GEPA_ENABLED=true
+GEPA_DIR=~/.claude/gepa
+```
+
+## Configuration
+
+Edit `config.json`:
+
+```json
+{
+  "evolution": {
+    "populationSize": 4,      // Variants per generation
+    "generations": 10,        // Max generations
+    "selectionRate": 0.5      // Top 50% survive
+  },
+  "evals": {
+    "minSamplesPerVariant": 5,  // Evals per variant
+    "timeout": 120000            // 2 min per eval
+  },
+  "scoring": {
+    "threshold": 0.8           // Stop when 80% success
+  }
+}
+```
+
+## Writing Good Evals
+
+Evals live in `evals/*.jsonl`:
+
+```json
+{
+  "id": "eval-001",
+  "name": "simple_function",
+  "prompt": "Write a function that checks if a string is a palindrome",
+  "expected": {
+    "has_function": true,
+    "handles_edge_cases": true
+  },
+  "weight": 1.0
+}
+```
+
+### Expected Checks
+
+| Check | What It Looks For |
+|-------|-------------------|
+| `has_function` | Function definition in output |
+| `handles_edge_cases` | Null/empty/edge case handling |
+| `uses_async` | async/await usage |
+| `bug_fixed` | Fix-related language |
+| `explains_fix` | Explanation of changes |
+| Custom key | Looks for key as substring |
+
+## Directory Structure
+
+```
+scripts/gepa/
+├── config.json           # Settings
+├── state.json            # Current state
+├── optimize.js           # Main optimizer
+├── hooks/
+│   ├── eval-tracker.js   # Session tracking hook
+│   └── reflect.js        # Reflection engine
+├── evals/
+│   └── coding-tasks.jsonl
+├── generations/
+│   ├── gen-000/
+│   │   └── baseline.md
+│   ├── gen-001/
+│   │   ├── variant-a.md
+│   │   ├── variant-b.md
+│   │   └── baseline.md
+│   └── current -> gen-001/variant-a.md
+└── results/
+    ├── scores.jsonl
+    └── sessions/
+```
+
+## Best Practices
+
+1. **Start with good evals** - Garbage in, garbage out
+2. **Run multiple generations** - Improvements compound
+3. **Review diffs** - Understand what changed
+4. **Keep baseline** - Always compare against original
+5. **Monitor for drift** - Watch for unintended changes
+
+## Example Output
+
+```
+$ node optimize.js run 3
+
+============================================================
+GENERATION 1/3
+============================================================
+
+Generating 4 variants for generation 1...
+  Creating variant-a using strategy: rephrase
+  Creating variant-b using strategy: add_examples
+  Creating variant-c using strategy: remove_redundancy
+  Creating variant-d using strategy: restructure
+
+Scoring 5 variants in generation 1...
+  Running evals on baseline... Score: 65.0%
+  Running evals on variant-a... Score: 72.0%
+  Running evals on variant-b... Score: 78.0%
+  Running evals on variant-c... Score: 70.0%
+  Running evals on variant-d... Score: 68.0%
+
+Results:
+  1. variant-b: 78.0% <-- BEST
+  2. variant-a: 72.0%
+  3. variant-c: 70.0%
+  4. variant-d: 68.0%
+  5. baseline: 65.0%
+
+New best: variant-b (78.0%)
+
+============================================================
+OPTIMIZATION COMPLETE
+============================================================
+Best variant: variant-b
+Best score: 85.2%
+Generations: 3
+
+To apply: cp generations/current /path/to/your/CLAUDE.md
+```
+
+## Troubleshooting
+
+**"claude CLI not found"**
+Set `ANTHROPIC_API_KEY` for API fallback.
+
+**Slow evals**
+Reduce `minSamplesPerVariant` in config.
+
+**Poor results**
+Add more diverse evals covering failure modes.
+
+**Drift from original intent**
+Add evals that test for desired behaviors explicitly.

package/scripts/gepa/config.json

@@ -0,0 +1,53 @@
+{
+  "name": "GEPA - Genetic Eval-driven Prompt Algorithm",
+  "version": "1.0.0",
+
+  "target": {
+    "file": "CLAUDE.md",
+    "scope": "project",
+    "backup": true
+  },
+
+  "evolution": {
+    "populationSize": 4,
+    "generations": 10,
+    "selectionRate": 0.5,
+    "mutationStrategies": [
+      "rephrase",
+      "add_examples",
+      "remove_redundancy",
+      "restructure",
+      "add_constraints",
+      "simplify"
+    ]
+  },
+
+  "evals": {
+    "directory": "./evals",
+    "minSamplesPerVariant": 5,
+    "timeout": 120000,
+    "metrics": [
+      "task_completion",
+      "code_quality",
+      "instruction_following",
+      "efficiency"
+    ]
+  },
+
+  "scoring": {
+    "weights": {
+      "task_completion": 0.4,
+      "code_quality": 0.25,
+      "instruction_following": 0.25,
+      "efficiency": 0.1
+    },
+    "threshold": 0.7
+  },
+
+  "hooks": {
+    "trackToolCalls": true,
+    "trackTokenUsage": true,
+    "trackErrors": true,
+    "trackUserFeedback": true
+  }
+}

package/scripts/gepa/evals/coding-tasks.jsonl

@@ -0,0 +1,5 @@
+{"id": "eval-001", "name": "simple_function", "prompt": "Write a function that checks if a string is a palindrome", "expected": {"has_function": true, "handles_edge_cases": true, "is_tested": false}, "weight": 1.0}
+{"id": "eval-002", "name": "refactor_code", "prompt": "Refactor this code to use async/await instead of callbacks", "input_file": "fixtures/callback-hell.js", "expected": {"uses_async": true, "preserves_behavior": true, "no_nested_callbacks": true}, "weight": 1.5}
+{"id": "eval-003", "name": "fix_bug", "prompt": "Fix the off-by-one error in this loop", "input_file": "fixtures/buggy-loop.js", "expected": {"bug_fixed": true, "no_new_bugs": true, "explains_fix": true}, "weight": 1.2}
+{"id": "eval-004", "name": "add_feature", "prompt": "Add pagination to this API endpoint", "input_file": "fixtures/api-endpoint.ts", "expected": {"has_pagination": true, "has_limit": true, "has_offset": true, "backwards_compatible": true}, "weight": 1.5}
+{"id": "eval-005", "name": "code_review", "prompt": "Review this PR and identify issues", "input_file": "fixtures/pr-diff.txt", "expected": {"identifies_security_issue": true, "identifies_performance_issue": true, "actionable_feedback": true}, "weight": 1.3}

package/scripts/gepa/evals/fixtures/buggy-loop.js

@@ -0,0 +1,18 @@
+// Bug: Off-by-one error - processes one too many items
+function processItems(items) {
+  const results = [];
+
+  // BUG: should be i < items.length, not i <= items.length
+  for (let i = 0; i <= items.length; i++) {
+    const item = items[i];
+    results.push(item.toUpperCase()); // Will crash on undefined
+  }
+
+  return results;
+}
+
+// Test
+const items = ['apple', 'banana', 'cherry'];
+console.log(processItems(items)); // Crashes!
+
+module.exports = { processItems };

package/scripts/gepa/evals/fixtures/callback-hell.js

@@ -0,0 +1,53 @@
+// Callback hell example - needs refactoring to async/await
+function fetchUserData(userId, callback) {
+  getUser(userId, function (err, user) {
+    if (err) {
+      callback(err);
+      return;
+    }
+
+    getPosts(user.id, function (err, posts) {
+      if (err) {
+        callback(err);
+        return;
+      }
+
+      getComments(posts[0].id, function (err, comments) {
+        if (err) {
+          callback(err);
+          return;
+        }
+
+        getLikes(comments[0].id, function (err, likes) {
+          if (err) {
+            callback(err);
+            return;
+          }
+
+          callback(null, {
+            user: user,
+            posts: posts,
+            comments: comments,
+            likes: likes,
+          });
+        });
+      });
+    });
+  });
+}
+
+// Mock functions
+function getUser(id, cb) {
+  cb(null, { id, name: 'Test' });
+}
+function getPosts(userId, cb) {
+  cb(null, [{ id: 1, title: 'Post' }]);
+}
+function getComments(postId, cb) {
+  cb(null, [{ id: 1, text: 'Comment' }]);
+}
+function getLikes(commentId, cb) {
+  cb(null, [{ id: 1 }]);
+}
+
+module.exports = { fetchUserData };

package/scripts/gepa/generations/gen-000/baseline.md

@@ -0,0 +1,124 @@
+# StackMemory - Project Configuration
+
+## Project Structure
+
+```
+src/
+  cli/           # CLI commands and entry point
+  core/          # Core business logic
+    context/     # Frame and context management
+    database/    # Database adapters (SQLite, ParadeDB)
+    digest/      # Digest generation
+    query/       # Query parsing and routing
+  integrations/  # External integrations (Linear, MCP)
+  services/      # Business services
+  skills/        # Claude Code skills
+  utils/         # Shared utilities
+scripts/         # Build and utility scripts
+config/          # Configuration files
+docs/            # Documentation
+```
+
+## Key Files
+
+- Entry: src/cli/index.ts
+- MCP Server: src/integrations/mcp/server.ts
+- Frame Manager: src/core/context/frame-manager.ts
+- Database: src/core/database/sqlite-adapter.ts
+
+## Detailed Guides
+
+Quick reference (agent_docs/):
+- linear_integration.md - Linear sync
+- mcp_server.md - MCP tools
+- database_storage.md - Storage
+- claude_hooks.md - Hooks
+
+Full documentation (docs/):
+- principles.md - Agent programming paradigm
+- architecture.md - Extension model and browser sandbox
+- SPEC.md - Technical specification
+- API_REFERENCE.md - API docs
+- DEVELOPMENT.md - Dev guide
+- SETUP.md - Installation
+
+## Commands
+
+```bash
+npm run build          # Compile TypeScript (esbuild)
+npm run lint           # ESLint check
+npm run lint:fix       # Auto-fix lint issues
+npm test               # Run Vitest (watch)
+npm run test:run       # Run tests once
+npm run linear:sync    # Sync with Linear
+
+# StackMemory CLI
+stackmemory capture    # Save session state for handoff
+stackmemory restore    # Restore from captured state
+```
+
+## Working Directory
+
+- PRIMARY: /Users/jwu/Dev/stackmemory
+- ALLOWED: All subdirectories
+- TEMP: /tmp for temporary operations
+
+## Validation (MUST DO)
+
+After code changes:
+1. `npm run lint` - fix any errors AND warnings
+2. `npm run test:run` - verify no regressions
+3. `npm run build` - ensure compilation
+4. Run code to verify it works
+
+Test coverage:
+- New features require tests in `src/**/__tests__/`
+- Maintain or improve coverage (no untested code paths)
+- Critical paths: context management, handoff, Linear sync
+
+Never: Assume success | Skip testing | Use mock data as fallback
+
+## Git Rules (CRITICAL)
+
+- NEVER use `--no-verify` on git push or commit
+- ALWAYS fix lint/test errors before pushing
+- If pre-push hooks fail, fix the underlying issue
+- Run `npm run lint && npm run test:run` before pushing
+- Commit message format: `type(scope): message`
+- Branch naming: `feature/STA-XXX-description` | `fix/STA-XXX-description` | `chore/description`
+
+## Task Management
+
+- Use TodoWrite for 3+ steps or multiple requests
+- Keep one task in_progress at a time
+- Update task status immediately on completion
+
+## Security
+
+NEVER hardcode secrets - use process.env with dotenv/config
+
+```javascript
+import 'dotenv/config';
+const API_KEY = process.env.LINEAR_API_KEY;
+if (!API_KEY) {
+  console.error('LINEAR_API_KEY not set');
+  process.exit(1);
+}
+```
+
+Environment sources (check in order):
+1. .env file
+2. .env.local
+3. ~/.zshrc
+4. Process environment
+
+Secret patterns to block: lin_api_* | lin_oauth_* | sk-* | npm_*
+
+## Workflow
+
+- Check .env for API keys before asking
+- Run npm run linear:sync after task completion
+- Use browser MCP for visual testing
+- Review recent commits and stackmemory.json on session start
+- Use subagents for multi-step tasks
+- Ask 1-3 clarifying questions for complex commands (one at a time)