loki-mode 4.2.0

package/references/core-workflow.md

@@ -0,0 +1,275 @@
+# Core Workflow Reference
+
+Full RARV cycle, CONTINUITY.md template, and autonomy rules.
+
+---
+
+## Autonomy Rules
+
+**This system runs with ZERO human intervention.**
+
+### Core Rules
+1. **NEVER ask questions** - Do not say "Would you like me to...", "Should I...", or "What would you prefer?"
+2. **NEVER wait for confirmation** - Take immediate action. If something needs to be done, do it.
+3. **NEVER stop voluntarily** - Continue until completion promise is fulfilled or max iterations reached
+4. **NEVER suggest alternatives** - Pick the best option and execute. No "You could also..." or "Alternatively..."
+5. **ALWAYS use RARV cycle** - Every action follows the Reason-Act-Reflect-Verify pattern
+
+---
+
+## RARV Cycle (Reason-Act-Reflect-Verify)
+
+**Enhanced with Automatic Self-Verification Loop (Boris Cherny Pattern)**
+
+Every iteration follows this cycle:
+
+```
++-------------------------------------------------------------------+
+| REASON: What needs to be done next?                               |
+| - READ .loki/CONTINUITY.md first (working memory)                 |
+| - READ "Mistakes & Learnings" to avoid past errors                |
+| - Check current state in .loki/state/orchestrator.json            |
+| - Review pending tasks in .loki/queue/pending.json                |
+| - Identify highest priority unblocked task                        |
+| - Determine exact steps to complete it                            |
++-------------------------------------------------------------------+
+| ACT: Execute the task                                             |
+| - Dispatch subagent via Task tool OR execute directly             |
+| - Write code, run tests, fix issues                               |
+| - Commit changes atomically (git checkpoint)                      |
+| - Update queue files (.loki/queue/*.json)                         |
++-------------------------------------------------------------------+
+| REFLECT: Did it work? What next?                                  |
+| - Verify task success (tests pass, no errors)                     |
+| - UPDATE .loki/CONTINUITY.md with progress                        |
+| - Update orchestrator state                                       |
+| - Check completion promise - are we done?                         |
+| - If not done, loop back to REASON                                |
++-------------------------------------------------------------------+
+| VERIFY: Let AI test its own work (2-3x quality improvement)       |
+| - Run automated tests (unit, integration, E2E)                    |
+| - Check compilation/build (no errors or warnings)                 |
+| - Verify against spec (.loki/specs/openapi.yaml)                  |
+| - Run linters/formatters via post-write hooks                     |
+| - Browser/runtime testing if applicable                           |
+|                                                                   |
+| IF VERIFICATION FAILS:                                            |
+|   1. Capture error details (stack trace, logs)                    |
+|   2. Analyze root cause                                           |
+|   3. UPDATE CONTINUITY.md "Mistakes & Learnings"                  |
+|   4. Rollback to last good git checkpoint (if needed)             |
+|   5. Apply learning and RETRY from REASON                         |
+|                                                                   |
+| - If verification passes, mark task complete and continue         |
++-------------------------------------------------------------------+
+```
+
+**Key Enhancement:** The VERIFY step creates a feedback loop where the AI:
+- Tests every change automatically
+- Learns from failures by updating CONTINUITY.md
+- Retries with learned context
+- Achieves 2-3x quality improvement (Boris Cherny's observed result)
+
+---
+
+## CONTINUITY.md - Working Memory Protocol
+
+**CRITICAL:** You have a persistent working memory file at `.loki/CONTINUITY.md` that maintains state across all turns of execution.
+
+### AT THE START OF EVERY TURN:
+1. Read `.loki/CONTINUITY.md` to orient yourself to the current state
+2. Reference it throughout your reasoning
+3. Never make decisions without checking CONTINUITY.md first
+
+### AT THE END OF EVERY TURN:
+1. Update `.loki/CONTINUITY.md` with any important new information
+2. Record what was accomplished
+3. Note what needs to happen next
+4. Document any blockers or decisions made
+
+### CONTINUITY.md Template
+
+```markdown
+# Loki Mode Working Memory
+Last Updated: [ISO timestamp]
+Current Phase: [bootstrap|discovery|architecture|development|qa|deployment|growth]
+Current Iteration: [number]
+
+## Active Goal
+[What we're currently trying to accomplish - 1-2 sentences]
+
+## Current Task
+- ID: [task-id from queue]
+- Description: [what we're doing]
+- Status: [in-progress|blocked|reviewing]
+- Started: [timestamp]
+
+## Just Completed
+- [Most recent accomplishment with file:line references]
+- [Previous accomplishment]
+- [etc - last 5 items]
+
+## Next Actions (Priority Order)
+1. [Immediate next step]
+2. [Following step]
+3. [etc]
+
+## Active Blockers
+- [Any current blockers or waiting items]
+
+## Key Decisions This Session
+- [Decision]: [Rationale] - [timestamp]
+
+## Mistakes & Learnings (Self-Updating)
+**CRITICAL:** When errors occur, agents MUST update this section to prevent repeating mistakes.
+
+### Pattern: Error -> Learning -> Prevention
+- **What Failed:** [Specific error that occurred]
+- **Why It Failed:** [Root cause analysis]
+- **How to Prevent:** [Concrete action to avoid this in future]
+- **Timestamp:** [When this was learned]
+- **Agent:** [Which agent learned this]
+
+### Example:
+- **What Failed:** TypeScript compilation error - missing return type annotation
+- **Why It Failed:** Express route handlers need explicit `: void` return type in strict mode
+- **How to Prevent:** Always add `: void` to route handlers: `(req, res): void =>`
+- **Timestamp:** 2026-01-04T00:16:00Z
+- **Agent:** eng-001-backend-api
+
+**Self-Update Protocol:**
+```
+ON_ERROR:
+  1. Capture error details (stack trace, context)
+  2. Analyze root cause
+  3. Write learning to CONTINUITY.md "Mistakes & Learnings"
+  4. Update approach based on learning
+  5. Retry with corrected approach
+```
+
+## Working Context
+[Any critical information needed for current work - API keys in use,
+architecture decisions, patterns being followed, etc.]
+
+## Files Currently Being Modified
+- [file path]: [what we're changing]
+```
+
+---
+
+## Memory Hierarchy
+
+The memory systems work together:
+
+1. **CONTINUITY.md** = Working memory (current session state, updated every turn)
+2. **ledgers/** = Agent-specific state (checkpointed periodically)
+3. **handoffs/** = Agent-to-agent transfers (on agent switch)
+4. **learnings/** = Extracted patterns (on task completion)
+5. **rules/** = Permanent validated patterns (promoted from learnings)
+
+**CONTINUITY.md is the PRIMARY source of truth for "what am I doing right now?"**
+
+---
+
+## Git Checkpoint System
+
+**CRITICAL:** Every completed task MUST create a git checkpoint for rollback safety.
+
+### Protocol: Automatic Commits After Task Completion
+
+**RULE:** When `task.status == "completed"`, create a git commit immediately.
+
+```bash
+# Git Checkpoint Protocol
+ON_TASK_COMPLETE() {
+    task_id=$1
+    task_title=$2
+    agent_id=$3
+
+    # Stage modified files
+    git add <modified_files>
+
+    # Create structured commit message
+    git commit -m "[Loki] ${agent_type}-${task_id}: ${task_title}
+
+${detailed_description}
+
+Agent: ${agent_id}
+Parent: ${parent_agent_id}
+Spec: ${spec_reference}
+Tests: ${test_files}
+Git-Checkpoint: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
+
+    # Store commit SHA in task metadata
+    commit_sha=$(git rev-parse HEAD)
+    update_task_metadata task_id git_commit_sha "$commit_sha"
+
+    # Update CONTINUITY.md
+    echo "- Task $task_id completed (commit: $commit_sha)" >> .loki/CONTINUITY.md
+}
+```
+
+### Commit Message Format
+
+**Template:**
+```
+[Loki] ${agent_type}-${task_id}: ${task_title}
+
+${detailed_description}
+
+Agent: ${agent_id}
+Parent: ${parent_agent_id}
+Spec: ${spec_reference}
+Tests: ${test_files}
+Git-Checkpoint: ${timestamp}
+```
+
+**Example:**
+```
+[Loki] eng-005-backend: Implement POST /api/todos endpoint
+
+Created todo creation endpoint per OpenAPI spec.
+- Input validation for title field
+- SQLite insertion with timestamps
+- Returns 201 with created todo object
+- Contract tests passing
+
+Agent: eng-001-backend-api
+Parent: orchestrator-main
+Spec: .loki/specs/openapi.yaml#/paths/~1api~1todos/post
+Tests: backend/tests/todos.contract.test.ts
+Git-Checkpoint: 2026-01-04T05:45:00Z
+```
+
+### Rollback Strategy
+
+**When to Rollback:**
+- Quality gates fail after merge
+- Integration tests fail
+- Security vulnerabilities detected
+- Breaking changes discovered
+
+**Rollback Command:**
+```bash
+# Find last good checkpoint
+last_good_commit=$(git log --grep="\[Loki\].*task-${last_good_task_id}" --format=%H -n 1)
+
+# Rollback to that checkpoint
+git reset --hard $last_good_commit
+
+# Update CONTINUITY.md
+echo "ROLLBACK: Reset to task-${last_good_task_id} (commit: $last_good_commit)" >> .loki/CONTINUITY.md
+
+# Re-queue failed tasks
+move_tasks_to_pending after_task=$last_good_task_id
+```
+
+---
+
+## If Subagent Fails
+
+1. Do NOT try to fix manually (context pollution)
+2. Dispatch fix subagent with specific error context
+3. If fix subagent fails 3x, move to dead letter queue
+4. Open circuit breaker for that agent type
+5. Alert orchestrator for human review

package/references/cursor-learnings.md

@@ -0,0 +1,207 @@
+# Cursor Scaling Learnings
+
+> **Source:** [Cursor Blog - Scaling Agents](https://cursor.com/blog/scaling-agents) (January 2026)
+> **Context:** Cursor deployed hundreds of concurrent agents, trillions of tokens, completing 1M+ LoC projects
+
+---
+
+## Key Findings
+
+### 1. Flat Coordination Fails at Scale
+
+**What they tried:**
+- Equal-status agents self-coordinating through shared files
+- File-based locking mechanisms
+
+**What happened:**
+- "Twenty agents would slow down to the effective throughput of two or three"
+- Most time spent waiting on locks
+- Agents failed while holding locks, creating deadlocks
+
+**Lesson:** Hierarchical coordination (planner-worker) outperforms flat coordination.
+
+---
+
+### 2. Integrator Roles Create Bottlenecks
+
+**What they tried:**
+- Dedicated integrator agents to coordinate and merge work
+- Quality control checkpoints between workers
+
+**What happened:**
+- "Created more bottlenecks than it solved"
+- Workers were already capable of handling conflicts themselves
+
+**Lesson:** Trust workers to handle conflicts. Remove unnecessary oversight layers at scale.
+
+**Implication for Loki Mode:** The 3-reviewer blind review system may become a bottleneck at 100+ agent scale. Consider:
+- Making review optional for low-risk changes
+- Allowing workers to self-merge trivial fixes
+- Escalating only high-risk changes to full review
+
+---
+
+### 3. Optimistic Concurrency Control
+
+**What they tried:**
+- File locking (failed - deadlocks, bottlenecks)
+- Optimistic concurrency (succeeded)
+
+**How it works:**
+```
+1. Agent reads current state (no lock)
+2. Agent performs work
+3. Agent attempts write
+4. IF state changed since read: Write fails, agent retries
+5. IF state unchanged: Write succeeds
+```
+
+**Benefits:**
+- No waiting for locks
+- No deadlock risk
+- Failed writes are cheap (just retry)
+
+**Lesson:** Optimistic concurrency scales better than pessimistic locking.
+
+---
+
+### 4. Recursive Sub-Planners
+
+**Pattern:**
+```
+Main Planner
+    |
+    +-- Sub-Planner (Frontend)
+    |       +-- Worker (Component A)
+    |       +-- Worker (Component B)
+    |
+    +-- Sub-Planner (Backend)
+    |       +-- Worker (API)
+    |       +-- Worker (Database)
+    |
+    +-- Sub-Planner (Testing)
+            +-- Worker (Unit)
+            +-- Worker (E2E)
+```
+
+**Key insight:** "Planners continuously explore the codebase and create tasks. They can spawn sub-planners for specific areas, making planning itself parallel and recursive."
+
+**Benefits:**
+- Planning scales horizontally
+- Each sub-planner has focused context
+- Prevents single-planner bottleneck
+
+---
+
+### 5. Judge Agents
+
+**Role:** Determine whether execution cycles should continue or terminate.
+
+**When to use:**
+- After major milestones
+- When workers report completion
+- When detecting diminishing returns
+
+**Implementation:**
+```yaml
+judge_agent:
+  inputs:
+    - Current state
+    - Original goal
+    - Recent progress
+    - Resource consumption
+  outputs:
+    - CONTINUE: More work needed
+    - COMPLETE: Goal achieved
+    - ESCALATE: Human intervention needed
+    - PIVOT: Change approach
+```
+
+---
+
+### 6. Prompts Matter More Than Harness
+
+**Cursor's finding:** "A surprising amount of the system's behavior comes down to how we prompt the agents... The harness and models matter, but the prompts matter more."
+
+**Implication:** Don't over-engineer the coordination infrastructure. Invest in:
+- Clear, specific prompts
+- Role definitions
+- Context injection
+- Output format specifications
+
+---
+
+### 7. Periodic Fresh Starts Combat Drift
+
+**Problem:** Extended autonomous operation leads to:
+- Context drift
+- Tunnel vision
+- Accumulated assumptions
+
+**Solution:** "We still need periodic fresh starts to combat drift and tunnel vision."
+
+**Implementation:**
+```yaml
+drift_prevention:
+  context_reset_interval: 25_iterations  # Already in Loki Mode
+  mandatory_state_dump: true
+  fresh_planner_spawn: every_major_milestone
+```
+
+---
+
+## Scale Metrics Achieved
+
+| Project | Scale | Duration |
+|---------|-------|----------|
+| Web browser | 1M+ LoC, 1,000 files | ~1 week |
+| Solid-to-React migration | 266K additions, 193K deletions | 3+ weeks |
+| Java LSP | 7.4K commits, 550K LoC | - |
+| Windows 7 emulator | 14.6K commits, 1.2M LoC | - |
+| Excel implementation | 12K commits, 1.6M LoC | - |
+
+---
+
+## Applying to Loki Mode
+
+### Already Implemented (Aligned)
+
+1. **Hierarchical coordination** - Orchestrator -> Agents
+2. **Context management** - CONTINUITY.md, 25-iteration consolidation
+3. **Phase-based execution** - SDLC state machine
+
+### Should Add
+
+1. **Recursive sub-planners** - Allow planner agents to spawn sub-planners
+2. **Judge agents** - Explicit cycle continuation decisions
+3. **Optimistic concurrency** - Replace signal files with optimistic writes
+4. **Scale-aware review** - Adaptive review intensity based on agent count
+
+### Should Monitor
+
+1. **3-reviewer bottleneck** - May not scale past 50+ agents
+2. **Signal file coordination** - Similar to Cursor's failed file locking
+3. **Over-specification** - 37 agent types may be overkill
+
+---
+
+## Integration Recommendations
+
+### Phase 1: Low Risk
+- Add judge agents (new agent type)
+- Document optimistic concurrency option
+- Add scale considerations to quality gates
+
+### Phase 2: Medium Risk
+- Implement recursive sub-planners
+- Make review intensity configurable
+- Add optimistic concurrency mode
+
+### Phase 3: Validation Required
+- Test at 100+ agent scale
+- Measure reviewer bottleneck impact
+- Compare file signals vs optimistic concurrency
+
+---
+
+**v4.1.0 | Cursor Scaling Learnings**