the-grid-cc 1.7.13 → 1.7.14

package/docs/PERSISTENCE.md

@@ -0,0 +1,962 @@
+# Grid State Persistence - Technical Design Document
+
+**Version:** 1.0
+**Author:** Program 5 (Persistence Orchestration)
+**Date:** 2026-01-23
+
+---
+
+## Executive Summary
+
+This document specifies the state persistence system that enables Grid missions to survive session death. When a session ends unexpectedly (context exhaustion, timeout, user disconnect), the Grid preserves complete state in `.grid/` files. A fresh Master Control can resume any in-progress mission from the exact stopping point using the `/grid:resume` command.
+
+**Design Principles:**
+- **File-only persistence** - No external databases, Redis, or services
+- **Human-readable state** - All files are markdown/YAML for debugging
+- **Checkpoint-based recovery** - Saga pattern with compensating actions
+- **Warmth preservation** - Institutional knowledge survives across sessions
+
+---
+
+## Architecture Overview
+
+### Saga Pattern Implementation
+
+The Grid implements the **orchestration saga pattern** for distributed workflow management:
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                         SAGA ORCHESTRATOR                           │
+│                        (Master Control)                             │
+├─────────────────────────────────────────────────────────────────────┤
+│                                                                     │
+│   Step 1        Step 2        Step 3        Step N                  │
+│  ┌──────┐      ┌──────┐      ┌──────┐      ┌──────┐                │
+│  │Plan  │ ──▶  │Exec  │ ──▶  │Verify│ ──▶  │ ...  │                │
+│  └──────┘      └──────┘      └──────┘      └──────┘                │
+│      │              │              │              │                 │
+│      ▼              ▼              ▼              ▼                 │
+│  ┌──────┐      ┌──────┐      ┌──────┐      ┌──────┐                │
+│  │State │      │State │      │State │      │State │                │
+│  │ File │      │ File │      │ File │      │ File │                │
+│  └──────┘      └──────┘      └──────┘      └──────┘                │
+│                                                                     │
+│                    STATE.md (Central State)                         │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+**Recovery Modes:**
+1. **Forward Recovery (Continuation)** - Resume from last checkpoint
+2. **Backward Recovery (Compensation)** - Rollback partial work if needed
+
+### Checkpoint Hierarchy
+
+State is captured at multiple granularities:
+
+| Level | Frequency | File | Purpose |
+|-------|-----------|------|---------|
+| **Session** | On start/end | `STATE.md` | Top-level position tracking |
+| **Wave** | After each wave | `WAVE_STATUS.md` | Wave completion status |
+| **Block** | After each block | `*-SUMMARY.md` | Detailed work records |
+| **Thread** | On checkpoint | `CHECKPOINT.md` | Interrupted thread state |
+| **Discovery** | During work | `SCRATCHPAD.md` | Live findings |
+
+---
+
+## State That Must Survive
+
+### 1. Execution Position
+
+```yaml
+# .grid/STATE.md
+---
+cluster: "james-weatherhead-blog"
+session_id: "2026-01-23T14:30:00Z-blog"
+status: interrupted  # active | interrupted | completed | failed
+
+position:
+  phase: 2
+  phase_total: 4
+  phase_name: "Authentication"
+  block: 1
+  block_total: 3
+  wave: 2
+  wave_total: 3
+  thread: 3
+  thread_total: 5
+  thread_name: "Implement JWT refresh"
+
+progress_percent: 45
+energy_remaining: 7500
+---
+```
+
+### 2. Completed Work
+
+All completed work persists in SUMMARY files with commit hashes:
+
+```yaml
+# .grid/phases/01-foundation/01-SUMMARY.md
+---
+cluster: james-weatherhead-blog
+block: 01
+status: complete
+completed_at: "2026-01-23T14:00:00Z"
+
+commits:
+  - hash: "abc123"
+    message: "feat(01): Initialize Astro project"
+    files: ["package.json", "astro.config.mjs"]
+  - hash: "def456"
+    message: "feat(01): Configure Tailwind dark mode"
+    files: ["tailwind.config.mjs"]
+
+artifacts_created:
+  - path: "package.json"
+    lines: 45
+    verified: true
+  - path: "astro.config.mjs"
+    lines: 32
+    verified: true
+---
+```
+
+### 3. Pending Work
+
+Plans not yet executed are preserved in their original form:
+
+```
+.grid/plans/
+├── blog-block-01.md  # Contains full PLAN spec
+├── blog-block-02.md
+├── blog-block-03.md
+├── blog-block-04.md
+├── blog-block-05.md
+├── blog-block-06.md
+└── blog-PLAN-SUMMARY.md  # Master plan with wave structure
+```
+
+### 4. Interrupted Thread State
+
+When a checkpoint or interruption occurs mid-thread:
+
+```yaml
+# .grid/CHECKPOINT.md
+---
+type: session_death  # checkpoint | session_death | failure
+timestamp: "2026-01-23T15:45:00Z"
+block: "02"
+thread: 3
+thread_name: "Implement dark mode toggle"
+
+completed_threads:
+  - id: "02.1"
+    name: "Create BaseLayout"
+    commit: "ghi789"
+    status: complete
+  - id: "02.2"
+    name: "Build Header component"
+    commit: "jkl012"
+    status: complete
+
+current_thread:
+  id: "02.3"
+  name: "Implement dark mode toggle"
+  status: in_progress
+  partial_work:
+    files_created: ["src/components/DarkModeToggle.astro"]
+    files_modified: ["src/layouts/BaseLayout.astro"]
+    staged_changes: true
+    last_action: "Writing localStorage persistence logic"
+
+pending_threads:
+  - id: "02.4"
+    name: "CHECKPOINT: Human verification"
+    status: pending
+---
+```
+
+### 5. Warmth (Institutional Knowledge)
+
+Accumulated learnings from all Programs:
+
+```yaml
+# .grid/WARMTH.md
+---
+cluster: james-weatherhead-blog
+accumulated_from:
+  - executor-01 (block 01)
+  - executor-02 (block 02)
+  - executor-03 (block 02 parallel)
+---
+
+## Codebase Patterns
+- "This project uses Astro content collections for blog posts"
+- "Dark mode uses class strategy with localStorage"
+- "Tailwind typography plugin required for prose styling"
+
+## Gotchas
+- "Astro config must use .mjs extension for ESM"
+- "Shiki syntax highlighting is build-time only"
+- "Content collection schema in src/content/config.ts"
+
+## User Preferences
+- "User prefers minimal dependencies"
+- "User wants dark mode as default"
+- "No unnecessary abstractions"
+
+## Decisions Made
+- "Chose serverless adapter over static for future flexibility"
+- "Using Shiki over Prism for better VS Code parity"
+
+## Almost Did (Rejected)
+- "Considered MDX but stuck with plain MD for simplicity"
+- "Considered Zustand for state but localStorage sufficient"
+```
+
+### 6. User Decisions
+
+Critical decisions made by the user during I/O Tower interactions:
+
+```yaml
+# .grid/DECISIONS.md
+---
+cluster: james-weatherhead-blog
+---
+
+## Decision 1: 2026-01-23T14:15:00Z
+**Question:** Deploy to Vercel or Netlify?
+**Options Presented:**
+  - vercel: "Native Astro support, edge functions"
+  - netlify: "Simpler config, build plugins"
+**User Choice:** vercel
+**Rationale:** "Already have Vercel account"
+**Affects:** Block 01 (adapter choice), Block 06 (deploy config)
+
+## Decision 2: 2026-01-23T14:30:00Z
+**Question:** Dark mode default state?
+**Options Presented:**
+  - light: "Traditional default"
+  - dark: "Modern preference"
+  - system: "Respect OS setting"
+**User Choice:** dark
+**Rationale:** "Developer blog, dark is expected"
+**Affects:** Block 02 (toggle default)
+```
+
+### 7. Blockers Encountered
+
+Issues that blocked progress:
+
+```yaml
+# .grid/BLOCKERS.md
+---
+cluster: james-weatherhead-blog
+---
+
+## Blocker 1: RESOLVED
+**Block:** 02
+**Thread:** 02.3
+**Type:** dependency_missing
+**Description:** @tailwindcss/typography not installed
+**Resolution:** Added dependency in thread 02.1
+**Resolved At:** 2026-01-23T14:20:00Z
+
+## Blocker 2: ACTIVE
+**Block:** 06
+**Thread:** 06.3
+**Type:** human_action_required
+**Description:** Vercel CLI authentication needed
+**Waiting For:** User to run `vercel login`
+**Created At:** 2026-01-23T16:00:00Z
+```
+
+---
+
+## Directory Structure
+
+### Complete .grid/ Layout
+
+```
+.grid/
+├── STATE.md                    # Central state file (ALWAYS read first)
+├── CHECKPOINT.md               # Current checkpoint if interrupted
+├── WARMTH.md                   # Accumulated institutional knowledge
+├── DECISIONS.md                # User decisions log
+├── BLOCKERS.md                 # Active and resolved blockers
+├── SCRATCHPAD.md               # Live discoveries during execution
+├── SCRATCHPAD_ARCHIVE.md       # Archived scratchpad entries
+├── LEARNINGS.md                # Cross-project patterns
+├── config.json                 # Grid configuration (model tier, etc.)
+│
+├── plans/                      # Execution plans
+│   ├── {cluster}-PLAN-SUMMARY.md
+│   ├── {cluster}-block-01.md
+│   ├── {cluster}-block-02.md
+│   └── ...
+│
+├── phases/                     # Execution artifacts
+│   ├── 01-foundation/
+│   │   ├── 01-PLAN.md         # Plan (copy from plans/)
+│   │   └── 01-SUMMARY.md      # Completion record
+│   ├── 02-auth/
+│   │   ├── 02-PLAN.md
+│   │   └── 02-SUMMARY.md
+│   └── ...
+│
+├── debug/                      # Debug session state
+│   └── {timestamp}-{slug}/
+│       └── session.md
+│
+└── refinement/                 # Refinement swarm outputs
+    ├── screenshots/
+    ├── e2e/
+    └── personas/
+```
+
+### File Ownership and Locking
+
+| File | Written By | Read By | Update Frequency |
+|------|------------|---------|------------------|
+| `STATE.md` | MC | MC, Resume | Every wave |
+| `CHECKPOINT.md` | Executor | MC, Resume | On interrupt |
+| `WARMTH.md` | MC | Executors | After each block |
+| `*-SUMMARY.md` | Executor | MC, Recognizer | On block complete |
+| `SCRATCHPAD.md` | Executors | MC, Executors | During execution |
+| `DECISIONS.md` | MC | All | On user decision |
+
+---
+
+## Resume Protocol
+
+### Pre-Resume State Validation
+
+Before resuming, validate state consistency:
+
+```python
+def validate_state():
+    """Validate .grid/ state is consistent and resumable."""
+
+    checks = [
+        # 1. STATE.md exists and is parseable
+        ("STATE.md exists", file_exists(".grid/STATE.md")),
+
+        # 2. Claimed commits actually exist
+        ("Commits exist", verify_commits(get_claimed_commits())),
+
+        # 3. Claimed files exist
+        ("Files exist", verify_files(get_claimed_artifacts())),
+
+        # 4. No conflicting checkpoints
+        ("Single checkpoint", count_checkpoints() <= 1),
+
+        # 5. Plan files exist for pending blocks
+        ("Plans available", verify_pending_plans()),
+    ]
+
+    return all(check[1] for check in checks)
+```
+
+### Resume Flow
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                         /grid:resume                                │
+├─────────────────────────────────────────────────────────────────────┤
+│                                                                     │
+│  1. READ STATE                                                      │
+│     ├─ Load .grid/STATE.md                                          │
+│     ├─ Parse position (phase/block/wave/thread)                     │
+│     └─ Determine status (interrupted/checkpoint/failed)             │
+│                                                                     │
+│  2. VALIDATE STATE                                                  │
+│     ├─ Verify commits exist in git                                  │
+│     ├─ Verify claimed files exist                                   │
+│     ├─ Check for state corruption                                   │
+│     └─ If invalid: ENTER RECOVERY MODE                              │
+│                                                                     │
+│  3. RECONSTRUCT CONTEXT                                             │
+│     ├─ Load WARMTH.md (institutional knowledge)                     │
+│     ├─ Load DECISIONS.md (user decisions)                           │
+│     ├─ Load CHECKPOINT.md if exists (interrupted thread)            │
+│     ├─ Collect all SUMMARY.md files (completed work)                │
+│     └─ Build execution context object                               │
+│                                                                     │
+│  4. DETERMINE RESUME POINT                                          │
+│     ├─ If CHECKPOINT: Resume from interrupted thread                │
+│     ├─ If wave complete: Start next wave                            │
+│     ├─ If block complete: Start next block in wave                  │
+│     └─ If session death: Resume from last checkpoint                │
+│                                                                     │
+│  5. SPAWN CONTINUATION                                              │
+│     ├─ Pass warmth to fresh executor                                │
+│     ├─ Pass completed_threads table                                 │
+│     ├─ Pass remaining plan                                          │
+│     └─ Execute from resume point                                    │
+│                                                                     │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+### Context Reconstruction
+
+When resuming, MC must rebuild complete context from files:
+
+```python
+def reconstruct_context():
+    """Rebuild execution context from .grid/ files."""
+
+    context = {
+        "cluster": None,
+        "position": None,
+        "completed_blocks": [],
+        "completed_threads": [],
+        "pending_plans": [],
+        "warmth": None,
+        "decisions": [],
+        "blockers": [],
+        "checkpoint": None,
+    }
+
+    # 1. Load central state
+    state = parse_yaml(read(".grid/STATE.md"))
+    context["cluster"] = state["cluster"]
+    context["position"] = state["position"]
+
+    # 2. Collect completed work
+    for summary_path in glob(".grid/phases/*/SUMMARY.md"):
+        summary = parse_yaml(read(summary_path))
+        context["completed_blocks"].append(summary)
+
+        # Extract completed threads from summary
+        for commit in summary.get("commits", []):
+            context["completed_threads"].append({
+                "block": summary["block"],
+                "commit": commit["hash"],
+                "files": commit["files"],
+            })
+
+    # 3. Load warmth
+    if file_exists(".grid/WARMTH.md"):
+        context["warmth"] = read(".grid/WARMTH.md")
+
+    # 4. Load decisions
+    if file_exists(".grid/DECISIONS.md"):
+        context["decisions"] = parse_decisions(".grid/DECISIONS.md")
+
+    # 5. Load checkpoint if exists
+    if file_exists(".grid/CHECKPOINT.md"):
+        context["checkpoint"] = parse_yaml(read(".grid/CHECKPOINT.md"))
+
+    # 6. Identify pending plans
+    for plan_path in glob(".grid/plans/*-block-*.md"):
+        block_num = extract_block_number(plan_path)
+        if block_num not in [b["block"] for b in context["completed_blocks"]]:
+            context["pending_plans"].append({
+                "path": plan_path,
+                "block": block_num,
+                "content": read(plan_path),
+            })
+
+    return context
+```
+
+---
+
+## Checkpoint Persistence
+
+### Session Death Detection
+
+Grid cannot always write a checkpoint before death. Detection strategies:
+
+1. **Graceful shutdown** - MC writes checkpoint before context exhaustion
+2. **Heartbeat timeout** - Scratchpad not updated in 10 minutes
+3. **Partial state** - CHECKPOINT.md exists but no SUMMARY.md
+
+### Checkpoint Types
+
+| Type | Trigger | State Captured | Resume Strategy |
+|------|---------|----------------|-----------------|
+| `human_verify` | Thread requires verification | Full thread state | Wait for user approval, continue |
+| `human_action` | External action needed | Blocker details | Wait for user to act, continue |
+| `decision` | Architectural fork | Options, context | Present options, continue on choice |
+| `session_death` | Context exhaustion | Last known state | Validate and continue |
+| `failure` | Unrecoverable error | Error details | Present to user, may require rollback |
+
+### Checkpoint File Format
+
+```yaml
+# .grid/CHECKPOINT.md
+---
+# Checkpoint metadata
+type: human_verify
+timestamp: "2026-01-23T15:45:00Z"
+expires: null  # or ISO timestamp if time-sensitive
+
+# Position
+cluster: james-weatherhead-blog
+block: "02"
+wave: 2
+thread: 4
+thread_name: "CHECKPOINT: Human verification of layout"
+
+# Progress
+progress:
+  blocks_complete: 1
+  blocks_total: 6
+  threads_complete: 3
+  threads_total: 4
+
+# Completed work (essential for continuation)
+completed_threads:
+  - id: "02.1"
+    name: "Create BaseLayout"
+    commit: "abc123"
+    files: ["src/layouts/BaseLayout.astro"]
+    status: complete
+  - id: "02.2"
+    name: "Build Header component"
+    commit: "def456"
+    files: ["src/components/Header.astro"]
+    status: complete
+  - id: "02.3"
+    name: "Implement dark mode toggle"
+    commit: "ghi789"
+    files: ["src/components/DarkModeToggle.astro"]
+    status: complete
+
+# Current checkpoint details
+checkpoint_details:
+  verification_instructions: |
+    1. Run: npm run dev
+    2. Visit: http://localhost:4321
+    3. Click dark mode toggle
+    4. Verify theme persists on refresh
+  expected_behavior: |
+    - Toggle switches between light/dark
+    - Theme persists after page reload
+    - No flash of wrong theme (FOUC)
+
+# User action needed
+awaiting: "User to test dark mode and respond 'approved' or describe issues"
+
+# Warmth for continuation
+warmth:
+  codebase_patterns:
+    - "Astro components use .astro extension"
+    - "Dark mode uses class strategy"
+  gotchas:
+    - "Must inline theme script in <head> to prevent FOUC"
+  user_preferences:
+    - "User prefers dark as default"
+---
+```
+
+---
+
+## State Update Protocol
+
+### When to Update STATE.md
+
+| Event | Update Action |
+|-------|---------------|
+| Wave starts | Set status: active, update wave position |
+| Wave completes | Increment wave, update progress |
+| Block completes | Increment block, log completion |
+| Checkpoint hit | Set status: checkpoint, write CHECKPOINT.md |
+| Session ending | Set status: interrupted, write checkpoint |
+| Mission complete | Set status: completed, archive warmth |
+| Failure | Set status: failed, write failure details |
+
+### Atomic State Updates
+
+State updates must be atomic to prevent corruption:
+
+```python
+def update_state(updates: dict):
+    """Atomically update STATE.md."""
+
+    # 1. Read current state
+    current = parse_yaml(read(".grid/STATE.md"))
+
+    # 2. Apply updates
+    merged = deep_merge(current, updates)
+    merged["updated_at"] = datetime.now().isoformat()
+
+    # 3. Write to temp file first
+    temp_path = ".grid/STATE.md.tmp"
+    write(temp_path, to_yaml(merged))
+
+    # 4. Atomic rename (POSIX atomic)
+    rename(temp_path, ".grid/STATE.md")
+```
+
+---
+
+## Warmth Aggregation
+
+### Warmth Collection
+
+After each block completes, aggregate warmth:
+
+```python
+def aggregate_warmth(block_summary_path: str):
+    """Aggregate warmth from block summary into WARMTH.md."""
+
+    # 1. Parse block summary
+    summary = parse_yaml(read(block_summary_path))
+    block_warmth = summary.get("lessons_learned", {})
+
+    # 2. Load existing warmth
+    warmth_path = ".grid/WARMTH.md"
+    if file_exists(warmth_path):
+        existing = parse_yaml(read(warmth_path))
+    else:
+        existing = {
+            "codebase_patterns": [],
+            "gotchas": [],
+            "user_preferences": [],
+            "decisions_made": [],
+            "almost_did": [],
+        }
+
+    # 3. Merge (deduplicate)
+    for category in ["codebase_patterns", "gotchas", "user_preferences", "almost_did"]:
+        new_items = block_warmth.get(category, [])
+        for item in new_items:
+            if item not in existing[category]:
+                existing[category].append(item)
+
+    # 4. Write aggregated warmth
+    write(warmth_path, to_yaml(existing))
+```
+
+### Warmth Injection
+
+When spawning continuation, inject aggregated warmth:
+
+```python
+def build_continuation_prompt(context):
+    """Build prompt with warmth for continuation executor."""
+
+    return f"""
+First, read ~/.claude/agents/grid-executor.md for your role.
+
+<warmth>
+Accumulated knowledge from prior Programs:
+
+{context["warmth"]}
+
+Apply this warmth. Don't repeat mistakes. Build on discoveries.
+</warmth>
+
+<completed_threads>
+{format_completed_threads(context["completed_threads"])}
+</completed_threads>
+
+<resume_point>
+Continue from: Thread {context["checkpoint"]["thread"]}
+User feedback: {context.get("user_feedback", "None")}
+</resume_point>
+
+<plan>
+{context["pending_plan"]["content"]}
+</plan>
+
+Execute the remaining threads. Verify previous commits exist before starting.
+"""
+```
+
+---
+
+## Recovery Scenarios
+
+### Scenario 1: Clean Checkpoint Resume
+
+**Situation:** User approved checkpoint, MC context expired before continuation.
+
+**State Files Present:**
+- STATE.md (status: checkpoint)
+- CHECKPOINT.md (type: human_verify, user_response: approved)
+- WARMTH.md (accumulated)
+- 01-SUMMARY.md (complete)
+
+**Resume Action:**
+1. Load state, verify checkpoint approved
+2. Load warmth
+3. Spawn executor with resume_point = next thread after checkpoint
+4. Continue execution
+
+### Scenario 2: Session Death Recovery
+
+**Situation:** MC context exhausted mid-execution, no checkpoint written.
+
+**State Files Present:**
+- STATE.md (status: active, stale timestamp)
+- SCRATCHPAD.md (recent entries)
+- 01-SUMMARY.md (complete)
+- No CHECKPOINT.md
+
+**Resume Action:**
+1. Detect stale state (no updates in 10+ minutes)
+2. Read last scratchpad entry for position
+3. Check git log for recent commits
+4. Reconstruct position from commits + scratchpad
+5. Create synthetic checkpoint
+6. Resume from last known good state
+
+### Scenario 3: Failure Recovery
+
+**Situation:** Executor failed, partial work exists.
+
+**State Files Present:**
+- STATE.md (status: active)
+- CHECKPOINT.md (type: failure, partial_work present)
+- Uncommitted changes in working directory
+
+**Resume Action:**
+1. Present failure report to user
+2. Options:
+   - Rollback: `git reset --hard` to last commit
+   - Retry: Spawn executor with failure context
+   - Manual: User fixes, then continue
+3. Update state based on choice
+4. Continue or restart
+
+### Scenario 4: Corrupted State
+
+**Situation:** STATE.md is invalid or missing.
+
+**Recovery Action:**
+1. Scan for SUMMARY.md files to find completed work
+2. Scan git log for Grid commits (commit message patterns)
+3. Rebuild STATE.md from discovered state
+4. Present reconstruction to user for approval
+5. Resume or restart based on confidence
+
+```python
+def recover_corrupted_state():
+    """Attempt to rebuild state from artifacts."""
+
+    recovered = {
+        "cluster": "unknown",
+        "position": {},
+        "completed_blocks": [],
+        "recovered": True,
+        "confidence": "medium",
+    }
+
+    # 1. Find summaries
+    summaries = glob(".grid/phases/*/SUMMARY.md")
+    for s in summaries:
+        try:
+            data = parse_yaml(read(s))
+            recovered["completed_blocks"].append(data["block"])
+            recovered["cluster"] = data.get("cluster", recovered["cluster"])
+        except:
+            pass
+
+    # 2. Find commits
+    commits = bash("git log --oneline --grep='feat(' --grep='fix(' -20")
+    # Parse commits for block references
+
+    # 3. Calculate position
+    if recovered["completed_blocks"]:
+        last_block = max(recovered["completed_blocks"])
+        recovered["position"]["block"] = last_block + 1
+
+    return recovered
+```
+
+---
+
+## State File Schemas
+
+### STATE.md Schema
+
+```yaml
+# Required fields
+cluster: string          # Cluster name
+session_id: string       # Unique session identifier
+status: enum             # active | checkpoint | interrupted | completed | failed
+created_at: ISO8601
+updated_at: ISO8601
+
+# Position tracking
+position:
+  phase: integer         # Current phase number
+  phase_total: integer   # Total phases
+  phase_name: string     # Phase name
+  block: integer         # Current block number
+  block_total: integer   # Total blocks
+  wave: integer          # Current wave number
+  wave_total: integer    # Total waves
+  thread: integer        # Current thread number (optional)
+  thread_total: integer  # Total threads in block (optional)
+
+# Progress
+progress_percent: integer  # 0-100
+energy_remaining: integer  # Energy budget
+
+# Execution mode
+mode: enum               # autopilot | guided | hands_on
+
+# Optional
+error: string           # If status is failed
+last_checkpoint: string # Path to last checkpoint file
+```
+
+### CHECKPOINT.md Schema
+
+```yaml
+# Required
+type: enum              # human_verify | decision | human_action | session_death | failure
+timestamp: ISO8601
+cluster: string
+block: string
+thread: integer
+thread_name: string
+
+# Progress snapshot
+progress:
+  blocks_complete: integer
+  blocks_total: integer
+  threads_complete: integer
+  threads_total: integer
+
+# Completed work
+completed_threads:
+  - id: string
+    name: string
+    commit: string
+    files: [string]
+    status: enum  # complete | partial
+
+# Checkpoint-specific
+checkpoint_details: object  # Type-specific content
+awaiting: string           # What user needs to do
+
+# Optional
+user_response: string      # User's response when resuming
+warmth: object            # Warmth for continuation
+expires: ISO8601          # If time-sensitive
+```
+
+### WARMTH.md Schema
+
+```yaml
+# Metadata
+cluster: string
+accumulated_from: [string]  # List of program IDs
+last_updated: ISO8601
+
+# Knowledge categories
+codebase_patterns: [string]
+gotchas: [string]
+user_preferences: [string]
+decisions_made: [string]
+almost_did: [string]
+fragile_areas: [string]
+```
+
+---
+
+## Implementation Checklist
+
+### Phase 1: Core Persistence (Required)
+
+- [ ] STATE.md write on every wave complete
+- [ ] CHECKPOINT.md write on checkpoint/interrupt
+- [ ] WARMTH.md aggregation after block complete
+- [ ] /grid:resume command implementation
+- [ ] Basic context reconstruction
+
+### Phase 2: Enhanced Recovery (Recommended)
+
+- [ ] Session death detection via scratchpad staleness
+- [ ] Git-based state reconstruction
+- [ ] Corrupted state recovery
+- [ ] Rollback support
+
+### Phase 3: Advanced Features (Future)
+
+- [ ] Multi-cluster support (multiple .grid/ directories)
+- [ ] State diff visualization
+- [ ] Time-travel debugging (restore any checkpoint)
+- [ ] Cross-session analytics
+
+---
+
+## Testing the Persistence System
+
+### Test 1: Clean Resume
+
+```bash
+# Start a mission
+/grid
+# Build a blog
+
+# Wait for checkpoint
+# User approves checkpoint
+# Close terminal (simulate session death)
+
+# New session
+/grid:resume
+# Should continue from approved checkpoint
+```
+
+### Test 2: Session Death Recovery
+
+```bash
+# Start a mission
+/grid
+# Build a blog
+
+# Wait for mid-execution
+# Kill terminal (SIGKILL)
+
+# New session
+/grid:resume
+# Should detect stale state
+# Should reconstruct from scratchpad + git
+# Should continue from last known point
+```
+
+### Test 3: Failure Recovery
+
+```bash
+# Start a mission that will fail
+# (e.g., missing API key)
+
+# Executor returns failure
+# MC writes failure checkpoint
+
+# New session
+/grid:resume
+# Should present failure report
+# Should offer rollback/retry/manual options
+```
+
+---
+
+## References
+
+### Research Sources
+
+- [Mastering LangGraph Checkpointing: Best Practices for 2025](https://sparkco.ai/blog/mastering-langgraph-checkpointing-best-practices-for-2025) - LangGraph built-in checkpointing patterns
+- [Build durable AI agents with LangGraph and Amazon DynamoDB](https://aws.amazon.com/blogs/database/build-durable-ai-agents-with-langgraph-and-amazon-dynamodb/) - AWS production persistence patterns
+- [LangGraph & Redis: Build smarter AI agents](https://redis.io/blog/langgraph-redis-build-smarter-ai-agents-with-memory-persistence/) - Thread-level persistence
+- [Saga Pattern in Microservices](https://microservices.io/patterns/data/saga.html) - Orchestration saga pattern
+- [Saga Design Pattern - Azure Architecture Center](https://learn.microsoft.com/en-us/azure/architecture/patterns/saga) - Microsoft saga implementation
+- [Saga orchestration pattern - AWS](https://docs.aws.amazon.com/prescriptive-guidance/latest/cloud-design-patterns/saga-orchestration.html) - AWS saga orchestration
+
+### Key Principles Applied
+
+1. **Checkpoint-based recovery** (LangGraph) - State saved at each step
+2. **Orchestration saga** (microservices) - Central coordinator with compensation
+3. **Human-in-the-loop** (LangGraph) - Pause for approval, resume without context loss
+4. **Idempotent operations** (saga pattern) - Retryable steps
+5. **File-based persistence** (Grid constraint) - No external services
+
+---
+
+End of Line.