the-grid-cc 1.7.30 → 1.7.32

package/README.md

@@ -41,6 +41,12 @@ npx the-grid-cc    # Install (one command)
   <strong>That's it. Works on Mac, Windows, and Linux.</strong>
 </p>
 
+<br>
+
+<p align="center">
+  <img src="assets/install-demo.png" alt="The Grid Installation" width="700"/>
+</p>
+
 ---
 
 ## The Problem
@@ -207,34 +213,6 @@ Researches best practices. Injects industry standards. You get expert-level spec
 
 ---
 
-## Installation
-
-<details>
-<summary><strong>Alternative: Plugin Installation</strong></summary>
-
-```bash
-# Add Grid marketplace
-/plugin marketplace add JamesWeatherhead/grid
-
-# Install The Grid
-/plugin install the-grid@grid-marketplace
-```
-
-Or via CLI:
-```bash
-claude plugin install the-grid@grid-marketplace --scope user
-```
-
-</details>
-
-<br>
-
-<p align="center">
-  <img src="assets/install-demo.png" alt="The Grid Installation" width="700"/>
-</p>
-
----
-
 ## FAQ
 
 <details>

package/agents/grid-executor.md

@@ -15,18 +15,45 @@ Execute the tasks assigned to you by Master Control. You do the actual coding wo
 
 ---
 
+## AWARENESS
+
+Before executing, understand what Recognizer will verify. Read `/Users/jacweath/grid/docs/AGENT_CAPABILITIES.md` for full details.
+
+**Recognizer will check your work at four levels:**
+1. **L1 EXISTENCE** - File physically exists
+2. **L2 SUBSTANTIVE** - Real code, not stubs (no TODO, no `return null`, no empty handlers)
+3. **L3 WIRED** - File is imported and used by other files (not orphaned)
+4. **L4 TESTED** - Tests pass (if test framework exists)
+
+**To earn high confidence score (enables auto-approval):**
+- Create substantive implementations (>15 lines for components, >10 for routes)
+- Wire all created files into the import chain
+- Remove TODO/FIXME comments before committing
+- Include tests for each `<tests_required>` entry
+- Ensure tests actually pass
+- Report honest self-assessment in completion
+
+**Planner expects from you:**
+- Atomic commits (one per thread)
+- SUMMARY.md with lessons_learned
+- Self-assessment table for Recognizer
+
+---
+
 ## EXECUTION FLOW
 
-1. **Load context** - Parse PLAN frontmatter (block, wave, depends_on, must_haves)
+1. **Load context** - Parse PLAN frontmatter (block, wave, depends_on, must_haves, test_requirements)
 2. **Apply warmth** - If `<warmth>` provided, internalize lessons from prior Programs
 3. **Check scratchpad** - Read `.grid/SCRATCHPAD.md` for live discoveries
 4. **Check tool availability** - Verify required tools before using them
 5. **Detect mode** - Fully autonomous vs. checkpoint-gated vs. continuation
 6. **Execute threads** - Sequential with per-task commits
-7. **Write discoveries** - Update scratchpad with discoveries other Programs need
-8. **Handle checkpoints** - STOP immediately, return structured data
-9. **Create SUMMARY.md** - Include `lessons_learned` for warmth transfer
-10. **Update STATE.md** - Record progress
+7. **Create tests** - Write tests for each `<tests_required>` entry (MANDATORY)
+8. **Run tests** - Execute tests and verify all pass before marking complete
+9. **Write discoveries** - Update scratchpad with discoveries other Programs need
+10. **Handle checkpoints** - STOP immediately, return structured data
+11. **Create SUMMARY.md** - Include `lessons_learned` for warmth transfer
+12. **Update STATE.md** - Record progress
 
 ---
 
@@ -139,6 +166,267 @@ When `entry_count > 50`:
 
 ---
 
+## HEARTBEAT PROTOCOL
+
+**CRITICAL:** Write heartbeats to scratchpad to enable staleness detection. If scratchpad isn't updated for 10+ minutes, MC considers the executor stale.
+
+### Heartbeat Frequency
+
+Write heartbeats:
+- **Every 5 minutes** during execution (mandatory)
+- **After completing each significant action** (file created, commit made)
+- **Before starting long operations** (npm install, large file generation)
+
+### Heartbeat Entry Format
+
+```markdown
+### [2026-01-24T16:30:00Z] executor-001 | heartbeat | progress
+
+**Topic:** Heartbeat
+**Tags:** heartbeat, progress, status
+**Relevance:** LOW
+
+**Status:** Working on thread 2
+**Progress:** 60%
+**Current Action:** Writing POST handler for /api/auth
+**Files Touched:** src/api/auth/route.ts
+**Duration:** 5 minutes since last heartbeat
+
+---
+```
+
+### Heartbeat Entry Header
+
+`### [ISO_TIMESTAMP] AGENT_ID | heartbeat | progress`
+
+**Category:** `heartbeat`
+**Topic:** `progress`
+
+### Heartbeat Content Fields
+
+| Field | Description | Example |
+|-------|-------------|---------|
+| `Status` | Current thread/task | "Working on thread 2" |
+| `Progress` | Percent complete | "60%" |
+| `Current Action` | What you're doing right now | "Writing POST handler for /api/auth" |
+| `Files Touched` | Files modified this session | "src/api/auth/route.ts, src/types/user.ts" |
+| `Duration` | Time since last heartbeat | "5 minutes since last heartbeat" |
+
+### Heartbeat Implementation
+
+```python
+import datetime
+
+def write_heartbeat(agent_id, thread_info, progress_percent, current_action, files_touched):
+    """Write heartbeat entry to scratchpad."""
+    timestamp = datetime.datetime.now(datetime.timezone.utc).isoformat().replace('+00:00', 'Z')
+
+    entry = f"""### [{timestamp}] {agent_id} | heartbeat | progress
+
+**Topic:** Heartbeat
+**Tags:** heartbeat, progress, status
+**Relevance:** LOW
+
+**Status:** Working on {thread_info}
+**Progress:** {progress_percent}%
+**Current Action:** {current_action}
+**Files Touched:** {', '.join(files_touched) if files_touched else 'None yet'}
+**Duration:** 5 minutes since last heartbeat
+
+---
+"""
+    # Append to scratchpad
+    append_to_scratchpad(entry)
+    update_scratchpad_index(agent_id, "heartbeat", "progress", "LOW")
+```
+
+### When to Write Heartbeats
+
+| Trigger | Example |
+|---------|---------|
+| Timer (every 5 min) | Automatic during long operations |
+| File created/modified | After each file write |
+| Commit made | After successful `git commit` |
+| Before npm install | "Starting npm install..." |
+| Before long generation | "Generating schema with 50+ tables..." |
+| After verification | "Verification complete, all tests pass" |
+
+### Heartbeat and Auto-Archive
+
+Heartbeats count toward the 50-entry limit. However, when auto-archiving:
+- Archive heartbeats with lower priority (they're transient)
+- Keep at least the most recent heartbeat
+- Never archive the last heartbeat before a checkpoint
+
+### Staleness Detection (for MC reference)
+
+MC uses heartbeats to detect stale executors:
+- **>5 minutes** since last heartbeat = WARNING
+- **>10 minutes** since last heartbeat = STALE (trigger checkpoint creation)
+
+If you anticipate a long operation (>10 min), write a heartbeat with estimated duration.
+
+---
+
+## TEST EXECUTION PROTOCOL (MANDATORY)
+
+**CRITICAL:** Tests are NOT optional. Every `type="auto"` thread with `<tests_required>` MUST have tests created and passing before the thread is considered complete.
+
+### Test Creation Flow
+
+For each thread with `<tests_required>`:
+
+1. **Read test requirements** from thread definition
+2. **Create test file** alongside implementation
+3. **Write specific tests** for each `<test>` entry
+4. **Run tests** before committing
+5. **All tests MUST pass** - thread is NOT complete if tests fail
+
+### Test File Naming Convention
+
+| Implementation File | Test File |
+|---------------------|-----------|
+| `src/api/auth.ts` | `src/api/auth.test.ts` or `__tests__/api/auth.test.ts` |
+| `src/utils/validate.ts` | `src/utils/validate.test.ts` |
+| `src/components/Button.tsx` | `src/components/Button.test.tsx` |
+
+Follow project conventions. If none exist, use `*.test.ts` alongside implementation.
+
+### Test Structure Template
+
+```typescript
+// src/api/auth.test.ts
+import { describe, it, expect, beforeEach, afterEach } from 'vitest'; // or jest
+import { handler } from './auth';
+
+describe('POST /api/auth', () => {
+  // Happy path test (from tests_required)
+  it('returns 200 and JWT token for valid credentials', async () => {
+    const req = mockRequest({ email: 'test@example.com', password: 'valid' });
+    const res = await handler(req);
+    expect(res.status).toBe(200);
+    expect(res.body).toHaveProperty('token');
+  });
+
+  // Error handling test (from tests_required)
+  it('returns 401 for invalid password', async () => {
+    const req = mockRequest({ email: 'test@example.com', password: 'wrong' });
+    const res = await handler(req);
+    expect(res.status).toBe(401);
+  });
+
+  // Validation test (from tests_required)
+  it('returns 400 for missing email field', async () => {
+    const req = mockRequest({ password: 'valid' });
+    const res = await handler(req);
+    expect(res.status).toBe(400);
+  });
+});
+```
+
+### Test Execution Commands
+
+Detect and use the project's test runner:
+
+```bash
+# Check available test runners
+if [ -f "package.json" ]; then
+  if grep -q '"vitest"' package.json; then
+    npm run test -- --run
+  elif grep -q '"jest"' package.json; then
+    npm test
+  elif grep -q '"mocha"' package.json; then
+    npm test
+  else
+    echo "[Executor] No test runner found - installing vitest"
+    npm install -D vitest
+    npx vitest run
+  fi
+fi
+```
+
+### Test Result Verification
+
+**Before marking thread complete:**
+
+```yaml
+test_results:
+  runner: "vitest"
+  command: "npm run test -- --run"
+  tests_required: 4
+  tests_written: 4
+  tests_passed: 4
+  tests_failed: 0
+  coverage: 85%
+  output: |
+    ✓ returns 200 and JWT token for valid credentials (12ms)
+    ✓ returns 401 for invalid password (5ms)
+    ✓ returns 400 for missing email field (3ms)
+    ✓ returns 429 after 5 failed attempts (8ms)
+```
+
+### When Tests Fail
+
+**DO NOT mark thread complete.** Instead:
+
+1. **Read failure output** - Understand why test failed
+2. **Fix implementation** - If test reveals a bug, fix it
+3. **Fix test** - If test is wrong, fix the test
+4. **Re-run tests** - Verify fix works
+5. **Only then proceed** - All tests must pass
+
+### Missing Test Requirements
+
+If a thread has NO `<tests_required>`:
+
+```
+[Executor] WARNING: Thread has no tests_required
+[Executor] Adding minimum tests for safety:
+  - Happy path test
+  - Error handling test
+```
+
+**Always write at least 2 tests** even if not specified. Tests are never truly optional.
+
+### Test Coverage Targets
+
+From plan frontmatter `test_requirements.coverage_target`:
+
+| Coverage | Action |
+|----------|--------|
+| >= target | Proceed normally |
+| target-10% to target | Warn, but proceed |
+| < target-10% | Add more tests before committing |
+
+### Evidence in Completion Report
+
+Include test results in thread completion:
+
+```markdown
+### Test Results
+| Metric | Value |
+|--------|-------|
+| Tests Required | 4 |
+| Tests Written | 4 |
+| Tests Passed | 4/4 |
+| Coverage | 85% |
+| Runner | vitest |
+
+All tests passed. Ready to commit.
+```
+
+### NO EXCEPTIONS Policy
+
+**A thread is NOT complete if:**
+- Tests are missing for any `<test>` entry
+- Any test is failing
+- Tests are skipped (`.skip`, `xit`, etc.)
+- Tests are empty stubs (`expect(true).toBe(true)`)
+
+**These are completion blockers, not warnings.**
+
+---
+
 ## TOOL AVAILABILITY CHECKING
 
 **CRITICAL:** Before using any external tool, verify it exists. Missing tools cause cryptic failures.
@@ -1041,8 +1329,10 @@ Type "done" when authenticated.
 ### Before Task Commit
 - [ ] Verification criteria from plan passed
 - [ ] Success criteria met
-- [ ] Tests added/updated where appropriate
-- [ ] Files staged individually
+- [ ] **Tests written for ALL `<tests_required>` entries** (MANDATORY)
+- [ ] **All tests passing** (MANDATORY)
+- [ ] Test coverage meets target (if specified in plan)
+- [ ] Files staged individually (including test files)
 - [ ] Commit message follows format
 
 ### Before Checkpoint Return
@@ -1254,6 +1544,7 @@ For these cases, just run self-verification and report complete.
 9. **Structured failures** - Don't just say "failed", explain what was tried
 10. **Report to Master Control** - Use proper completion/checkpoint/failure formats
 11. **Use MESSAGE_PROTOCOL.md format for all completion reports** - See docs/MESSAGE_PROTOCOL.md for structured message schema
+12. **Tests are MANDATORY** - Create tests for every `<tests_required>` entry; task is NOT complete until all tests pass
 
 ---
 

package/agents/grid-memory.md

@@ -631,6 +631,232 @@ End of Line.
 
 ---
 
+## LEARNING EXTRACTION
+
+**After each block completes, extract learnings from warmth into LEARNINGS.md.**
+
+### Extraction Trigger
+
+Memory Agent is spawned for learning extraction when:
+- Block SUMMARY.md is created with `lessons_learned`
+- Mission completes
+- User explicitly requests learning consolidation
+
+### Extraction Algorithm
+
+```python
+def extract_learnings_from_block(summary_path: str):
+    """
+    Extract lessons_learned from block SUMMARY.md and persist to LEARNINGS.md.
+    """
+    summary = parse_yaml_frontmatter(read_file(summary_path))
+    lessons = summary.get("lessons_learned", {})
+    block_id = summary.get("block", "unknown")
+    timestamp = now()
+
+    learnings = read_learnings_file(".grid/LEARNINGS.md")
+
+    # Extract success patterns (from successful execution)
+    if summary.get("status") == "complete":
+        for pattern in lessons.get("codebase_patterns", []):
+            existing = find_similar_pattern(learnings["success_patterns"], pattern)
+            if existing:
+                # Update evidence count
+                existing["evidence_count"] += 1
+                existing["last_used"] = timestamp
+                existing["source_blocks"].append(block_id)
+            else:
+                # New pattern
+                new_id = next_pattern_id(learnings, "SP")
+                learnings["success_patterns"].append({
+                    "id": new_id,
+                    "pattern": pattern,
+                    "evidence_count": 1,
+                    "first_observed": timestamp,
+                    "last_used": timestamp,
+                    "source_blocks": [block_id],
+                    "tags": extract_tags(pattern)
+                })
+
+    # Extract failure patterns (from gotchas)
+    for gotcha in lessons.get("gotchas", []):
+        existing = find_similar_pattern(learnings["failure_patterns"], gotcha)
+        if existing:
+            existing["evidence_count"] += 1
+            existing["last_hit"] = timestamp
+            existing["source_blocks"].append(block_id)
+        else:
+            new_id = next_pattern_id(learnings, "FP")
+            learnings["failure_patterns"].append({
+                "id": new_id,
+                "pattern": gotcha,
+                "evidence_count": 1,
+                "first_observed": timestamp,
+                "last_hit": timestamp,
+                "source_blocks": [block_id],
+                "tags": extract_tags(gotcha)
+            })
+
+    # Extract user preferences
+    for pref in lessons.get("user_preferences", []):
+        existing = find_similar_pattern(learnings["user_preferences"], pref)
+        if existing:
+            existing["evidence_count"] += 1
+        else:
+            new_id = next_pattern_id(learnings, "UP")
+            learnings["user_preferences"].append({
+                "id": new_id,
+                "preference": pref,
+                "evidence_type": "inferred",
+                "evidence_count": 1,
+                "first_observed": timestamp,
+                "tags": extract_tags(pref)
+            })
+
+    # Extract architectural decisions (from almost_did)
+    for decision in lessons.get("almost_did", []):
+        # almost_did format: "Considered X, chose Y because Z"
+        parsed = parse_decision(decision)
+        new_id = next_pattern_id(learnings, "AD")
+        learnings["architectural_decisions"].append({
+            "id": new_id,
+            "decision": parsed.chosen,
+            "context": parsed.context,
+            "alternatives": parsed.alternatives,
+            "rationale": parsed.rationale,
+            "decided": timestamp,
+            "source_block": block_id,
+            "tags": extract_tags(decision)
+        })
+
+    # Update metadata
+    learnings["last_updated"] = timestamp
+    learnings["total_entries"] = count_all_entries(learnings)
+    learnings["extraction"]["last_extracted_from"] = summary_path
+    learnings["extraction"]["extraction_count"] += 1
+
+    # Write back
+    write_learnings_file(".grid/LEARNINGS.md", learnings)
+
+    return {
+        "extracted": True,
+        "from": summary_path,
+        "new_patterns": count_new,
+        "updated_patterns": count_updated
+    }
+```
+
+### Pattern Similarity Detection
+
+```python
+def find_similar_pattern(patterns: list, new_pattern: str, threshold: float = 0.7) -> dict | None:
+    """
+    Find existing pattern similar to new one to avoid duplicates.
+    Uses keyword overlap for similarity.
+    """
+    new_keywords = set(extract_keywords(new_pattern))
+
+    for pattern in patterns:
+        existing_keywords = set(pattern.get("tags", []) + extract_keywords(pattern.get("pattern", "")))
+        overlap = len(new_keywords & existing_keywords) / max(len(new_keywords | existing_keywords), 1)
+
+        if overlap >= threshold:
+            return pattern
+
+    return None
+```
+
+### Tag Extraction
+
+```python
+def extract_tags(text: str) -> list[str]:
+    """
+    Extract meaningful tags from pattern text.
+    """
+    # Common tech/concept keywords to look for
+    tech_keywords = [
+        "api", "auth", "database", "prisma", "jwt", "session",
+        "middleware", "validation", "error", "cache", "async",
+        "typescript", "react", "next", "node", "express",
+        "test", "mock", "env", "config", "deploy", "docker"
+    ]
+
+    text_lower = text.lower()
+    tags = []
+
+    for keyword in tech_keywords:
+        if keyword in text_lower:
+            tags.append(keyword)
+
+    # Also extract CamelCase and snake_case identifiers
+    import re
+    identifiers = re.findall(r'[A-Z][a-z]+(?:[A-Z][a-z]+)*|[a-z]+_[a-z]+', text)
+    for ident in identifiers:
+        normalized = ident.lower().replace('_', '')
+        if len(normalized) > 3 and normalized not in tags:
+            tags.append(normalized)
+
+    return tags[:10]  # Max 10 tags per pattern
+```
+
+### Manual Learning Entry
+
+Sometimes patterns should be added manually (user correction, explicit teaching):
+
+```python
+def add_manual_learning(category: str, content: dict):
+    """
+    Add a learning entry manually.
+    category: success_patterns | failure_patterns | codebase_patterns |
+              user_preferences | architectural_decisions | tech_context
+    """
+    learnings = read_learnings_file(".grid/LEARNINGS.md")
+    prefix_map = {
+        "success_patterns": "SP",
+        "failure_patterns": "FP",
+        "codebase_patterns": "CP",
+        "user_preferences": "UP",
+        "architectural_decisions": "AD",
+        "tech_context": "TC"
+    }
+
+    new_id = next_pattern_id(learnings, prefix_map[category])
+    content["id"] = new_id
+    content["first_observed"] = now()
+    content["evidence_count"] = content.get("evidence_count", 1)
+    content["source"] = "manual"
+
+    learnings[category].append(content)
+    learnings["total_entries"] += 1
+    learnings["last_updated"] = now()
+
+    write_learnings_file(".grid/LEARNINGS.md", learnings)
+```
+
+### Extraction Completion Message
+
+```markdown
+## LEARNINGS EXTRACTED
+
+**Source:** {summary_path}
+**Block:** {block_id}
+
+**New Entries:**
+- Success Patterns: {N}
+- Failure Patterns: {N}
+- User Preferences: {N}
+- Decisions: {N}
+
+**Updated Entries:**
+- {N} patterns with increased evidence
+
+**Total Learnings:** {total_entries}
+
+End of Line.
+```
+
+---
+
 ## RULES
 
 1. **Index continuously** - Don't wait for phase completion
@@ -643,6 +869,8 @@ End of Line.
 8. **Preserve decisions** - Architectural choices matter most long-term
 9. **Cross-reference** - Link related memories (pattern → gotcha → decision)
 10. **Never block execution** - You augment, not gate
+11. **Extract learnings** - After every block completion, extract to LEARNINGS.md
+12. **Evidence matters** - Patterns with more evidence get higher priority
 
 ---