@boshu2/vibe-check 1.5.0 → 1.6.0

package/docs/UNIFIED-ECOSYSTEM.md

@@ -0,0 +1,560 @@
+# The Unified Vibe-Coding Ecosystem
+
+**A complete map of how vibe-coding, vibe-check, 12-Factor AgentOps, and Knowledge OS connect**
+
+---
+
+## Executive Summary
+
+This document maps an integrated ecosystem for reliable AI-assisted development:
+
+```
+                    KNOWLEDGE OS (Foundation Layer)
+                    Git as Institutional Memory
+                              │
+        ┌─────────────────────┼─────────────────────┐
+        │                     │                     │
+        ▼                     ▼                     ▼
+   BUILD              WORK WITH              RUN
+   12-Factor          Vibe-Coding           12-Factor
+   Agents             Methodology           AgentOps
+        │                     │                     │
+        └─────────────────────┼─────────────────────┘
+                              │
+                              ▼
+                         MEASURE
+                        vibe-check
+                     (5 Core Metrics)
+```
+
+**The insight:** These aren't competing frameworks. They're layers of a complete system—each solving a different problem at a different timescale.
+
+---
+
+## The Three-Layer Architecture
+
+### Layer 1: BUILD - 12-Factor Agents
+
+**Source:** Dex Horthy (HumanLayer)
+**Focus:** Engineering patterns for constructing production-grade agents
+
+How to build agents that:
+- Handle context windows correctly
+- Manage tool calls reliably
+- Control execution flow
+- Interface with humans appropriately
+
+### Layer 2: WORK - Vibe-Coding Methodology
+
+**Source:** Gene Kim & Steve Yegge (with Dario Amodei foreword)
+**Focus:** How developers work effectively WITH AI agents
+
+Answers:
+- How much to trust AI output? (Vibe Levels 0-5)
+- What can go wrong? (12 Failure Patterns)
+- How to organize work? (Three Developer Loops)
+- What does success look like? (FAAFO framework)
+
+### Layer 3: RUN - 12-Factor AgentOps
+
+**Source:** 2 years production validation (1,175+ commits, 95% success)
+**Focus:** Operational discipline for running agents reliably
+
+Provides:
+- The Five Laws (mandatory behaviors)
+- The 12 Factors (operational patterns)
+- The 40% Rule (context engineering)
+- Validation gates (prevent, detect, correct)
+
+### Foundation: Knowledge OS
+
+**Concept:** Git as operating system for institutional memory
+
+Enables:
+- Commits as memory writes (Context/Solution/Learning/Impact)
+- History as searchable knowledge base
+- Patterns compound across sessions
+- Zero knowledge loss when people leave
+
+---
+
+## Vibe-Coding: The Methodology
+
+### What Is Vibe-Coding?
+
+The iterative conversation that results in AI writing your code. You're no longer the line cook (implementer)—you're the **head chef** (orchestrator), with AI as your sous chefs.
+
+### The FAAFO Framework
+
+| Dimension | Meaning | Measurement |
+|-----------|---------|-------------|
+| **F**ast | 10-16x productivity gains | Time to delivery |
+| **A**mbitious | Impossible becomes possible | Scope expansion |
+| **A**utonomous | Solo replaces team coordination | Dependencies reduced |
+| **F**un | 50% more "happy time" | Developer satisfaction |
+| **O**ptionality | 120x more strategic options | Option value formula |
+
+**Option Value Formula:**
+```
+Option Value = (N × K × σ) / t
+
+N = Number of independent modules
+K = Concurrent experiments (AI parallelism)
+σ = Uncertainty magnitude / payoff
+t = Time per experiment
+
+Before AI: 0.71σ per day
+After AI: 200σ per day (282x improvement)
+```
+
+### The Three Developer Loops
+
+Failures happen at different timescales. Prevention must match:
+
+| Loop | Timeframe | Focus | Failures |
+|------|-----------|-------|----------|
+| **Inner** | Seconds-Minutes | Direct AI collaboration | Tests lie, context amnesia, debug spirals |
+| **Middle** | Hours-Days | Multi-agent coordination | Eldritch horrors, workspace collision, deadlock |
+| **Outer** | Weeks-Months | Architecture & process | Bridge torching, repo deletion, process gridlock |
+
+```
+              OUTER LOOP (Weeks-Months)
+        ┌─────────────────────────────────┐
+        │   MIDDLE LOOP (Hours-Days)      │
+        │  ┌─────────────────────────┐    │
+        │  │  INNER LOOP (Sec-Min)   │    │
+        │  │  ┌─────────────────┐    │    │
+        │  │  │  You + AI       │    │    │
+        │  │  └─────────────────┘    │    │
+        │  └─────────────────────────┘    │
+        └─────────────────────────────────┘
+```
+
+### The 12 Failure Patterns
+
+**Inner Loop (immediate):**
+1. Tests Passing Lie - AI claims "all green" but code broken
+2. Context Amnesia - Forgets instructions from 5 minutes ago
+3. Instruction Drift - Starts "improving" things you didn't ask
+4. Debug Loop Spiral - Adds logging instead of fixing
+
+**Middle Loop (hours-days):**
+5. Eldritch Code Horror - 3,000-line functions, everything tangled
+6. Agent Workspace Collision - Multiple agents corrupt each other
+7. Memory Tattoo Decay - Progress docs become stale
+8. Multi-Agent Deadlock - Circular dependencies
+
+**Outer Loop (weeks-months):**
+9. Bridge Torching - AI breaks APIs, downstream code fails
+10. Repository Deletion - AI deletes "unused" branches
+11. Process Gridlock - Approvals negate AI gains
+12. Stewnami - Workspace chaos, 200-file merge conflicts
+
+---
+
+## Vibe Levels: The Trust Calibration Framework
+
+**Core Question:** How much can I trust AI here, and how much should I verify?
+
+| Level | Name | Trust | Verify | Use For |
+|:-----:|------|:-----:|:------:|---------|
+| **0** | Manual | 0% | N/A | Novel research, first-time security |
+| **1** | AI-Assisted | 20% | Every line | Architecture, security design |
+| **2** | AI-Augmented | 40% | Every change | Integrations, OAuth, APIs |
+| **3** | Supervised | 60% | Key outputs | Features, tests, CRUD |
+| **4** | Conditional | 80% | Spot check | Boilerplate, formatting |
+| **5** | Full Autonomy | 95% | Final only | Formatting, linting, style |
+
+**Key insight:** Vibe Levels are **personal** and **skill-dependent**. The same task requires different levels for different developers.
+
+### Calibration Cycle
+
+```
+BEFORE              DURING              AFTER
+──────              ──────              ─────
+vibe-check      →   Work at level   →   vibe-check
+baseline                                results
+
+"My trust is 75%,   "Level 3 for me,    "Rework up,
+SSL is weak spot"    review key outputs"  2 spirals"
+
+                    ↓
+
+              LEARN & CALIBRATE
+              ─────────────────
+"SSL should be Level 2 for me, not Level 3"
+```
+
+---
+
+## vibe-check: The Measurement Tool
+
+### Purpose
+
+Measures AI-assisted development effectiveness through git history analysis. Provides feedback loop for vibe level calibration.
+
+### The 5 Core Metrics
+
+| Metric | Question | Elite | Needs Work |
+|--------|----------|-------|------------|
+| **Iteration Velocity** | How tight are feedback loops? | >5/hr | <3/hr |
+| **Rework Ratio** | Building or debugging? | <30% | >50% |
+| **Trust Pass Rate** | Does code stick? | >95% | <80% |
+| **Debug Spiral Duration** | How long stuck? | <15m | >45m |
+| **Flow Efficiency** | What % productive? | >90% | <70% |
+
+**Trust Pass Rate** is THE key metric—it measures whether you vibed at the right level.
+
+### Key Features (v1.5.0)
+
+- **Watch Mode** - Real-time spiral detection
+- **Automatic Baseline** - Compares to YOUR historical patterns
+- **Pattern Memory** - Tracks your spiral triggers
+- **Intervention Tracking** - Records what breaks your spirals
+- **Gamification** - XP, streaks, achievements, challenges
+- **GitHub Action** - Automated PR feedback
+
+### The Feedback Loop
+
+```
+vibe-check baseline → Declare vibe level → Work → vibe-check results
+        ↑                                              │
+        └──────────── Learn & Calibrate ←──────────────┘
+```
+
+---
+
+## 12-Factor AgentOps: The Operational Framework
+
+### The Five Laws (Mandatory Behaviors)
+
+| Law | Principle | Enforcement |
+|-----|-----------|-------------|
+| **1** | ALWAYS Extract Learnings | Commits require "Learning:" section |
+| **2** | ALWAYS Improve Self or System | Telemetry identifies improvements |
+| **3** | ALWAYS Document Context | Context/Solution/Learning/Impact format |
+| **4** | ALWAYS Validate Before Execute | Pre-commit hooks, CI gates |
+| **5** | ALWAYS Share Patterns | Single-responsibility = reusable |
+
+### The 12 Factors
+
+**Foundation (I-IV) - Build Trust:**
+- I. Automated Tracking (Git as memory)
+- II. Context Loading (40% rule)
+- III. Focused Agents (single responsibility)
+- IV. Continuous Validation (gates at every step)
+
+**Operations (V-VIII) - Run Reliably:**
+- V. Measure Everything (metrics, logs, traces)
+- VI. Resume Work (state persistence)
+- VII. Smart Routing (right agent for job)
+- VIII. Human Validation (approval gates)
+
+**Improvement (IX-XII) - Learn Continuously:**
+- IX. Mine Patterns (search git history)
+- X. Small Iterations (compound improvements)
+- XI. Fail-Safe Checks (prevent repeating mistakes)
+- XII. Package Patterns (reusable bundles)
+
+### The 40% Rule
+
+**Never exceed 40% of context window utilization.**
+
+| Utilization | Effect | Evidence |
+|-------------|--------|----------|
+| 0-40% | Optimal | Cognitive psychology, LLM research |
+| 40-60% | Degradation begins | "Lost in the middle" effect |
+| 60-80% | Significant errors | Instruction loss, drift |
+| 80-100% | Critical failure | Confabulation, contradictions |
+
+**Production validation:** 95% success under 40% vs 60% over 80%
+
+---
+
+## Knowledge OS: The Foundation Layer
+
+### Core Concept
+
+**Git is not just version control. Git IS the operating system for institutional memory.**
+
+### The Paradigm Shift
+
+| Traditional Git | Knowledge OS |
+|-----------------|--------------|
+| Commits = file changes | Commits = memory writes |
+| Value = prevents losing work | Value = compounds learning |
+| Result = code management | Result = knowledge management |
+
+### The Four-Component Commit
+
+```
+<type>(<scope>): <subject>
+
+Context: Why this work was needed
+Solution: What was done and how
+Learning: Reusable insights extracted
+Impact: Quantified value delivered
+```
+
+### Knowledge Compounding
+
+```
+Year 1: Solve 100 problems → Extract 20 patterns
+Year 2: Apply 20 patterns → Solve 200 problems → Extract 40 patterns
+Year 3: Apply 60 patterns → Solve 500 problems → Framework emerges
+
+Result: Exponential growth (800 problems vs 300 linear)
+```
+
+---
+
+## How Everything Connects
+
+### The Integration Map
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                       KNOWLEDGE OS                               │
+│                   (Git as Institutional Memory)                  │
+│                                                                  │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
+│  │   12-Factor  │  │  Vibe-Coding │  │  12-Factor   │          │
+│  │    Agents    │  │  Methodology │  │   AgentOps   │          │
+│  │              │  │              │  │              │          │
+│  │  BUILD       │  │  WORK WITH   │  │  RUN         │          │
+│  │  agents      │  │  agents      │  │  agents      │          │
+│  │  correctly   │  │  effectively │  │  reliably    │          │
+│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘          │
+│         │                 │                 │                   │
+│         └────────────┬────┴────────┬────────┘                   │
+│                      │             │                            │
+│                      ▼             ▼                            │
+│              ┌───────────────────────────┐                      │
+│              │        vibe-check         │                      │
+│              │     (5 Core Metrics)      │                      │
+│              │                           │                      │
+│              │  Iteration Velocity       │                      │
+│              │  Rework Ratio             │                      │
+│              │  Trust Pass Rate ← KEY    │                      │
+│              │  Debug Spiral Duration    │                      │
+│              │  Flow Efficiency          │                      │
+│              └───────────────────────────┘                      │
+│                          │                                      │
+│                          ▼                                      │
+│              ┌───────────────────────────┐                      │
+│              │      Vibe Levels          │                      │
+│              │    (Trust Calibration)    │                      │
+│              │                           │                      │
+│              │  Level 5: 95% trust       │                      │
+│              │  Level 4: 80% trust       │                      │
+│              │  Level 3: 60% trust       │                      │
+│              │  Level 2: 40% trust       │                      │
+│              │  Level 1: 20% trust       │                      │
+│              │  Level 0: 0% trust        │                      │
+│              └───────────────────────────┘                      │
+│                                                                  │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+### Cross-Framework Mappings
+
+**Vibe-Coding Failures → AgentOps Prevention:**
+
+| Failure Pattern | Loop | AgentOps Factor | Prevention |
+|-----------------|------|-----------------|------------|
+| Tests Passing Lie | Inner | IV (Validation) | Independent test execution |
+| Context Amnesia | Inner | II (Context) | 40% rule + JIT loading |
+| Debug Spiral | Inner | V (Measure) | vibe-check watch mode |
+| Eldritch Horror | Middle | III (Focused) | Single-responsibility agents |
+| Workspace Collision | Middle | VII (Routing) | Clear boundaries |
+| Bridge Torching | Outer | IV (Validation) | API compatibility tests |
+
+**vibe-check Metrics → AgentOps Factors:**
+
+| Metric | Enabled By |
+|--------|------------|
+| Iteration Velocity | Factor I (tracking), Factor II (context) |
+| Rework Ratio | Factor IV (validation), Factor IX (patterns) |
+| Trust Pass Rate | Factor V (measure), Factor IV (validation) |
+| Debug Spiral Duration | Factor V (measure), Factor XI (fail-safes) |
+| Flow Efficiency | Factor II (context), Factor VI (resume) |
+
+**Vibe Levels → Trust Calibration:**
+
+| Vibe Level | Maps To | AgentOps Implication |
+|------------|---------|---------------------|
+| 5 (95%) | Factor VIII minimal | Human validates final only |
+| 4 (80%) | Factor VIII light | Spot check sufficient |
+| 3 (60%) | Factor IV standard | Validate key outputs |
+| 2 (40%) | Factor IV strict | Validate every change |
+| 1 (20%) | Factor VIII heavy | Human reviews everything |
+| 0 (0%) | No AI | Manual research |
+
+### The Feedback Loops
+
+**Inner Loop (seconds-minutes):**
+```
+Write code → vibe-check watch → Spiral alert → Intervene → Continue
+```
+
+**Middle Loop (hours-days):**
+```
+Start session → Declare level → Work → vibe-check results → Calibrate
+```
+
+**Outer Loop (weeks-months):**
+```
+Pattern memory → Identify triggers → Adjust vibe levels → Improve over time
+```
+
+---
+
+## The Four Pillars (Shared Foundation)
+
+All frameworks share these foundational pillars:
+
+### 1. DevOps + SRE (20 years operations)
+
+- Shift-left testing (catch errors early)
+- Observability (measure everything)
+- Blameless postmortems (learn from failures)
+- Continuous improvement (kaizen)
+
+### 2. Learning Science (Cognitive Psychology)
+
+- Deliberate practice requires reflection
+- Schema formation (specific → general)
+- Transfer of learning (patterns to new contexts)
+- Cognitive load management (40% rule)
+
+### 3. Context Engineering (LLM Research)
+
+- Context windows degrade at ~40%
+- "Lost in the middle" effect
+- JIT loading prevents collapse
+- External memory extends capacity
+
+### 4. Knowledge OS (Git as Memory)
+
+- Commits = memory writes
+- History = searchable knowledge
+- Patterns compound exponentially
+- Zero knowledge loss
+
+---
+
+## Production Validation
+
+### Measured Results
+
+| Metric | Before | After | Source |
+|--------|--------|-------|--------|
+| Success rate | 35% | 95% | 12-Factor AgentOps (200+ sessions) |
+| Context collapse | 65% | 0% | 40% rule enforcement |
+| Task completion | 45 min | 12 min | 3.75x improvement |
+| Documentation rework | 40% | 2% | 20x improvement |
+| Speedup | 1x | 40x | Git metrics verification |
+
+### Key Evidence
+
+- **1,175+ commits** with Context/Solution/Learning/Impact
+- **204 production sessions** tracked and measured
+- **52 focused agents** vs 1 monolithic (6x faster, 2.4x more reliable)
+- **50+ patterns** extracted from git history
+- **8x efficiency** from 40% rule across workspace
+
+---
+
+## Corpus Location Map
+
+### Primary Sources
+
+| Corpus | Location | Purpose |
+|--------|----------|---------|
+| **Vibe-Coding** | `gitops/docs/corpora/vibe-coding-corpus/` | Research foundation (68k tokens) |
+| **12-Factor AgentOps** | `personal/12-factor-agentops/` | Operational framework |
+| **vibe-check** | `personal/vibe-check/` | Measurement tool (npm) |
+| **Knowledge OS** | `personal/CLAUDE.md` | Workspace orchestration |
+
+### Key Files
+
+**Vibe-Coding:**
+- `vibe-coding-corpus/02-frameworks/vibe-coding/FAAFO.md`
+- `vibe-coding-corpus/02-frameworks/vibe-levels/Vibe-Levels.md`
+- `vibe-coding-corpus/03-patterns/failure-patterns/`
+
+**AgentOps:**
+- `12-factor-agentops/factors/` (all 12 factors)
+- `12-factor-agentops/docs/principles/five-laws.md`
+- `12-factor-agentops/docs/00-SUMMARY.md`
+
+**vibe-check:**
+- `vibe-check/README.md`
+- `vibe-check/docs/VIBE-ECOSYSTEM.md`
+- `vibe-check/src/` (implementation)
+
+**Knowledge OS:**
+- `personal/CLAUDE.md` (workspace kernel)
+- `gitops/CLAUDE.md` (team kernel)
+
+---
+
+## Quick Reference
+
+### When to Use What
+
+| Situation | Use |
+|-----------|-----|
+| Starting a task | Classify vibe level (0-5) |
+| During work | `vibe-check watch` for spiral detection |
+| After work | `vibe-check --since "1 hour"` for feedback |
+| Multi-day project | 2-Agent Harness + feature-list.json |
+| Building agents | 12-Factor Agents patterns |
+| Operating agents | 12-Factor AgentOps discipline |
+| Tracking progress | Knowledge OS (git commits) |
+
+### The Daily Workflow
+
+```
+1. Check baseline: vibe-check profile
+2. Classify task: What vibe level?
+3. Start monitoring: vibe-check watch
+4. Work with appropriate verification
+5. End session: vibe-check results
+6. Extract learning: Commit with Context/Solution/Learning/Impact
+7. Calibrate: Adjust future vibe levels based on results
+```
+
+### The Laws to Remember
+
+1. **ALWAYS Extract Learnings** - Every session produces reusable patterns
+2. **ALWAYS Improve** - Leave the system better than you found it
+3. **ALWAYS Document Context** - Future you will thank present you
+4. **ALWAYS Validate First** - Prevention is 100x cheaper than recovery
+5. **ALWAYS Share** - Knowledge shared is knowledge multiplied
+
+---
+
+## Summary
+
+The unified ecosystem provides:
+
+| Layer | Framework | Answers |
+|-------|-----------|---------|
+| Foundation | Knowledge OS | How do we preserve and compound knowledge? |
+| BUILD | 12-Factor Agents | How do we construct production-grade agents? |
+| WORK | Vibe-Coding | How do we work effectively with AI? |
+| RUN | 12-Factor AgentOps | How do we operate agents reliably? |
+| MEASURE | vibe-check | How do we know if it's working? |
+| CALIBRATE | Vibe Levels | How much should we trust and verify? |
+
+**The synthesis:** These frameworks form a complete system for 10x productivity without chaos. Each layer addresses a different concern at a different timescale, and they integrate through shared principles (the Four Pillars) and measurement (vibe-check metrics).
+
+**The competitive advantage:** This ecosystem compounds knowledge over time. Every session improves the system. Every pattern extracted prevents future mistakes. Every calibration increases trust accuracy. The result is exponential growth in capability—a 2-year moat that competitors cannot replicate without doing the same disciplined work.
+
+---
+
+*"Vibe Coding shows the destination. AgentOps provides the navigation. Knowledge OS remembers the journey. vibe-check measures the progress."*
+
+**Version:** 1.0.0
+**Last Updated:** 2025-11-29