@polymorphism-tech/morph-spec 4.2.0 → 4.3.1

package/docs/next-generation/CONTEXT-OPTIMIZATION.md

@@ -0,0 +1,267 @@
+# Context Optimization — MORPH-SPEC v3.0
+
+Context optimization is the practice of maximizing what useful work fits in the AI model's context window. v3.0 introduces structured techniques to reduce token waste and increase implementation quality per session.
+
+---
+
+## The Context Budget Problem
+
+Claude's context window has a hard limit. A typical morph-next-gen feature session consumes:
+
+| Component | Tokens (approx) |
+|-----------|----------------|
+| CLAUDE.md (full) | ~23,000 |
+| spec.md (large) | ~8,000 |
+| tasks.json | ~3,000 |
+| Standards (3 files) | ~6,000 |
+| Conversation history | Grows per turn |
+| **Total overhead** | **~40,000+** |
+
+Without optimization, implementation tokens are crowded out by reference material.
+
+---
+
+## 12 Optimization Techniques
+
+### 1. Context Priming (Biggest Win: ~80% reduction)
+
+Instead of loading full CLAUDE.md, use a priming file (500 tokens vs 23,000):
+
+```bash
+# Output a focused priming file for your current task type
+morph-spec prime feature
+morph-spec prime bug
+morph-spec prime refactor
+morph-spec prime design
+morph-spec prime infra
+```
+
+Each prime file contains only the essential rules for that task type. Paste the output at the start of your session.
+
+**Priming file locations:** `framework/templates/prime/{type}.md`
+
+### 2. Context Bundles
+
+Pre-compress feature context from ~180K tokens to ~15K:
+
+```bash
+# Create a bundle (compress all outputs for a feature)
+morph-spec context bundle create user-auth
+
+# Load a bundle at session start
+morph-spec context bundle load user-auth
+
+# List available bundles
+morph-spec context bundle list
+```
+
+Bundle structure:
+
+```json
+{
+  "id": "bundle-user-auth-2026-02",
+  "feature": "user-auth",
+  "created": "2026-02-19T10:00:00Z",
+  "tokenCount": 14800,
+  "includes": ["spec-summary", "decisions-summary", "task-list", "key-standards"],
+  "compressed": true
+}
+```
+
+### 3. Selective Standards Loading
+
+Don't load all standards — load only what's relevant to the current task:
+
+```bash
+# Bad: loads everything
+# cat framework/standards/*.md
+
+# Good: load only relevant standards
+morph-spec prime feature | head -20
+# Then manually load: coding.md, and ONE domain standard
+```
+
+Standards priority for implementation tasks:
+1. `coding.md` — always load
+2. Domain standard (e.g., `supabase-rls.md` if working on auth)
+3. Skip everything else
+
+### 4. Micro-Agents for Narrow Tasks
+
+Create ultra-specialized agents with minimal context (1-3 standards, narrow mission):
+
+```bash
+morph-spec micro-agent create rls-writer \
+  --base-agent supabase-expert \
+  --mission "Write RLS policies only" \
+  --standards supabase-rls.md \
+  --tools Read,Write
+
+morph-spec micro-agent show rls-writer --show-prompt
+```
+
+Micro-agents have 90%+ fewer tokens than full agents.
+
+### 5. MCP Server Hygiene
+
+Unnecessary MCP servers consume tokens via tool definitions:
+
+```bash
+# Analyze current MCP overhead
+morph-spec mcp optimize
+
+# List configured servers with usage stats
+morph-spec mcp list
+```
+
+Disable MCP servers not used in current task via `.claude/settings.local.json`.
+
+### 6. Checkpoint Summaries Instead of Full History
+
+At each checkpoint, generate a compact summary instead of keeping full conversation:
+
+```bash
+morph-spec session-summary user-auth
+```
+
+This produces a ~500-token summary replacing thousands of tokens of history.
+
+### 7. Core Four Dashboard
+
+Before starting a session, check the Core Four to identify context wastage:
+
+```bash
+morph-spec core-four user-auth
+```
+
+Output shows: Context efficiency %, active model, prompt quality score, tool count.
+
+**Context efficiency** = (useful tokens) / (total tokens). Target: >70%.
+
+### 8. Thread-Based Context Isolation
+
+Each parallel thread (P-Thread) gets a clean context window. Instead of one massive session, split into 3 parallel sessions:
+
+```bash
+morph-spec parallel plan user-auth
+# Output: 3 squads × ~20K tokens each = 60K total but 20K per session
+```
+
+vs. single session with 60K+ tokens of context.
+
+### 9. Artifact Trail
+
+Track what was produced to avoid re-reading files:
+
+```bash
+# The artifact-trail.js lib tracks: file path, producer agent, timestamp, token cost
+# Access via:
+morph-spec analytics feature user-auth
+```
+
+This shows which files were produced and their approximate token cost.
+
+### 10. Output-First Prompting
+
+In your prompts to agents, always list expected outputs first:
+
+```markdown
+## Your Deliverables (produce these files):
+1. Services/UserService.cs
+2. DTOs/LoginRequest.cs
+
+## Then implement following these specs...
+```
+
+This front-loads the goal, keeping the model focused and reducing verbose exploration.
+
+### 11. Incremental Spec Loading
+
+Don't load the full spec at once. Load sections on demand:
+
+```markdown
+# Load only the section relevant to current task
+## API Contracts (relevant to T003)
+POST /api/auth/login
+Request: { email, password }
+Response: { token, refreshToken }
+```
+
+### 12. Session Handoffs
+
+When approaching context limit, generate a handoff document:
+
+```bash
+morph-spec generate recap user-auth
+```
+
+The recap captures: progress %, tasks completed, decisions made, next steps. Start a new session with just the recap (~1,000 tokens).
+
+---
+
+## Context Analytics
+
+Monitor context efficiency over time:
+
+```bash
+# Feature-level analytics
+morph-spec analytics context user-auth
+
+# Project-level (30-day)
+morph-spec analytics project
+```
+
+Analytics track:
+- Context efficiency score (0-100%)
+- Token usage by phase
+- Optimization opportunities detected
+
+**Optimization triggers** (auto-detected):
+- Efficiency < 60% → Recommend priming
+- Session > 50K tokens → Recommend bundle
+- Standards loaded > 5 → Recommend selective loading
+- MCP servers > 3 active → Recommend MCP hygiene
+
+---
+
+## Context Bundle Format
+
+```json
+{
+  "id": "bundle-{feature}-{date}",
+  "feature": "user-auth",
+  "created": "2026-02-19T10:00:00Z",
+  "tokenCount": 14800,
+  "sections": {
+    "specSummary": "...",
+    "keyDecisions": ["ADR-001: JWT over sessions", "ADR-002: BCrypt for hashing"],
+    "taskProgress": "8/12 completed (67%)",
+    "standards": ["coding.md summary", "supabase-auth.md summary"],
+    "nextTasks": ["T009", "T010", "T011"]
+  }
+}
+```
+
+Store bundles at: `.morph/project/bundles/{feature}/`
+
+---
+
+## Quick Reference
+
+| Technique | Token Savings | When to Use |
+|-----------|--------------|-------------|
+| Context Priming | ~80% | Always — start of every session |
+| Context Bundles | ~70% | Features with large outputs |
+| Selective Standards | ~50% | Implementation phases |
+| Micro-Agents | ~90% | Narrow, repetitive tasks |
+| MCP Hygiene | ~10-20% | When >3 MCP servers active |
+| Thread Isolation | Resets per thread | Multi-domain features |
+| Session Handoffs | Resets context | Long-running features |
+
+---
+
+## Related
+
+- [Thread Management](THREAD-MANAGEMENT.md) — Thread isolation for context efficiency
+- [Meta-Prompts](META-PROMPTS.md) — HOP templates for structured agent prompting
+- [Migration Guide](MIGRATION-GUIDE.md) — Upgrading to v3.0 context features

package/docs/next-generation/EXECUTION-FLOW.md

@@ -0,0 +1,274 @@
+# Execution Flow — MORPH-SPEC v3.0
+
+This document describes the complete execution flow of a v3.0 feature from first request to sync, including all new v3.0 decision points and automation paths.
+
+---
+
+## High-Level Flow
+
+```
+User Request
+    │
+    ▼
+┌─────────────┐
+│  WORKFLOW   │  Auto-detect: fast-track / standard / full-morph
+│  DETECTION  │  Command: morph-spec detect-workflow "<request>"
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   CONTEXT   │  Load priming file (500 tokens vs 23K CLAUDE.md)
+│   PRIMING   │  Command: morph-spec prime feature
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   PROPOSAL  │  Agents detected, workflow confirmed
+│    PHASE    │  Gate: proposal approval (auto if trust >= high)
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   DESIGN    │  spec.md + contracts + decisions.md
+│    PHASE    │  Gate: design approval (auto if trust >= medium)
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   CLARIFY   │  Resolve ambiguities, update spec
+│    PHASE    │  (no gate, auto-continues)
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│    TASKS    │  tasks.json: IDs, deps, phases
+│    PHASE    │  Gate: tasks approval (auto if trust >= high)
+└──────┬──────┘
+       │
+       ▼
+┌─────────────────────────────────────────────────────────┐
+│                   IMPLEMENT PHASE                       │
+│                                                         │
+│   ┌─────────────┐    ┌─────────────┐                   │
+│   │  Single     │    │  Parallel   │                   │
+│   │  B-Thread   │    │  P-Threads  │                   │
+│   │  (simple)   │    │  (complex)  │                   │
+│   └──────┬──────┘    └──────┬──────┘                   │
+│          │                  │                           │
+│          │         ┌────────┴────────┐                  │
+│          │         │  Squad Planning │                  │
+│          │         │ morph-spec      │                  │
+│          │         │ parallel plan   │                  │
+│          │         └────────┬────────┘                  │
+│          │                  │                           │
+│          │    ┌─────────────┼─────────────┐             │
+│          │    ▼             ▼             ▼             │
+│          │  Backend     Frontend       Infra            │
+│          │  P-Thread    P-Thread     P-Thread           │
+│          │    │             │             │             │
+│          │    └─────────────┴─────────────┘             │
+│          │                  │                           │
+│          │              Coordinator                     │
+│          │              Merges results                  │
+│          │                  │                           │
+│          └──────────────────┘                           │
+│                             │                           │
+│                    Every 3 tasks:                       │
+│                   CHECKPOINT ──> Validators             │
+│                             │    - architecture         │
+│                             │    - security             │
+│                             │    - packages             │
+│                             │                           │
+│                   Trust updates automatically           │
+└─────────────────────────────┬───────────────────────────┘
+                              │
+                              ▼
+                      ┌─────────────┐
+                      │    SYNC     │  Recap + decisions archive
+                      │    PHASE    │
+                      └──────┬──────┘
+                             │
+                             ▼
+                      ┌─────────────┐
+                      │  ARCHIVED   │  Feature complete
+                      └─────────────┘
+```
+
+---
+
+## v3.0 Decision Points
+
+### 1. Workflow Detection
+
+```
+morph-spec detect-workflow "<request>"
+   │
+   ├─ fast-track  → 2 phases (skip most)
+   ├─ standard    → 4 phases (skip UIUX, CLARIFY, SYNC)
+   ├─ full-morph  → ALL phases
+   ├─ design-impl → prototype-focused
+   └─ ui-refresh  → visual-only
+```
+
+### 2. Trust-Based Auto-Approval
+
+```
+Phase gate reached?
+   │
+   ├─ Trust: low (<80%)     → Manual approval required
+   ├─ Trust: medium (>=80%) → Auto-approve: design gate
+   ├─ Trust: high (>=90%)   → Auto-approve: design + tasks gates
+   └─ Trust: maximum (>=95%)→ Auto-approve: all gates + zero-touch eligible
+```
+
+Check current trust: `morph-spec trust status`
+
+### 3. Parallel vs Sequential Implementation
+
+```
+Task count > 10 AND multi-domain?
+   YES → morph-spec parallel plan {feature}
+         → Spawn P-Threads per domain
+         → Wait: morph-spec threads wait-all {feature}
+         → Merge results
+   NO  → Single B-Thread sequential implementation
+```
+
+### 4. Fusion Decision
+
+```
+Need highest-quality output for critical component?
+   YES → morph-spec agents-fuse run --count 3 --strategy best-of-n
+         → 3 F-Threads run independently
+         → Aggregator picks best result
+   NO  → Standard single-agent approach
+```
+
+### 5. Checkpoint Gate
+
+```
+Task 3, 6, 9, ... completed?
+   │
+   ├─ Run validators (architecture, security, packages)
+   ├─ ALL PASS → Continue, update trust metrics
+   └─ FAIL     → Block progress
+                  → Attempt 2: fix violations
+                  → Attempt 3: fix violations
+                  → Attempt 4: AskUserQuestion
+```
+
+---
+
+## Zero-Touch Path (Maximum Trust)
+
+When trust level is `maximum` (>95% checkpoint pass rate across 10+ checkpoints):
+
+```
+Feature request
+    │
+    ▼ morph-spec feature create --auto
+┌──────────────────────────────────────┐
+│         ZERO-TOUCH EXECUTION         │
+│                                      │
+│  Proposal auto-approved              │
+│  Design auto-approved                │
+│  Tasks auto-approved                 │
+│  Commits auto-signed                 │
+│  Rollback auto-triggered on fail     │
+│                                      │
+│  Escalation triggers:                │
+│  - Checkpoint fails 3x → pause       │
+│  - Architecture error → pause        │
+│  - Security error → BLOCK            │
+│  - Out-of-scope change → pause       │
+└──────────────────────────────────────┘
+    │
+    ▼
+Feature complete (no human intervention)
+```
+
+Configure zero-touch: `framework/workflows/configs/zero-touch.json`
+
+---
+
+## Analytics & Observability
+
+At every step, analytics events are logged to `.morph/analytics/`:
+
+| Event | Log File | When |
+|-------|----------|------|
+| Phase advanced | `threads-log.jsonl` | Each phase transition |
+| Task completed | `threads-log.jsonl` | Each `task done` |
+| Checkpoint passed/failed | `threads-log.jsonl` | Every 3 tasks |
+| Trust level changed | `trust-log.jsonl` | After each checkpoint |
+| Auto-approval used | `trust-log.jsonl` | When gate auto-approved |
+| Thread created/killed | `threads-log.jsonl` | Thread lifecycle events |
+
+View analytics: `morph-spec analytics feature {feature-name}`
+
+---
+
+## Context Efficiency at Each Phase
+
+Recommended context loading per phase:
+
+| Phase | Context Budget | Load |
+|-------|---------------|------|
+| Proposal | 5K tokens | `morph-spec prime feature` output only |
+| Design | 15K tokens | Prime + relevant 2-3 standards |
+| Clarify | 10K tokens | Spec summary + decisions |
+| Tasks | 8K tokens | Spec summary + existing tasks |
+| Implement | 20K/thread | Bundle + task-specific standards |
+| Sync | 5K tokens | Recap + new decisions |
+
+---
+
+## CLI Quick Reference (v3.0 New Commands)
+
+```bash
+# Workflow detection
+morph-spec detect-workflow "implement X"
+
+# Context
+morph-spec prime feature
+morph-spec context bundle create {feature}
+
+# Threads
+morph-spec threads list
+morph-spec threads wait-all {feature}
+
+# Trust
+morph-spec trust status
+morph-spec trust auto-calculate {feature}
+
+# Parallel
+morph-spec parallel plan {feature}
+morph-spec squad-template list
+
+# Fusion
+morph-spec agents-fuse run --count 3 --prompt "..."
+
+# HOPs
+morph-spec hop list
+morph-spec hop render hop-backend-squad output.md --variables '{...}'
+
+# Migration
+morph-spec migrate status
+morph-spec migrate v2-to-v3
+
+# Analytics
+morph-spec analytics feature {feature}
+morph-spec core-four {feature}
+
+# Doctor
+morph-spec doctor --v3
+```
+
+---
+
+## Related
+
+- [Thread Management](THREAD-MANAGEMENT.md) — Thread types and lifecycle
+- [Context Optimization](CONTEXT-OPTIMIZATION.md) — Token efficiency techniques
+- [Meta-Prompts](META-PROMPTS.md) — HOP template system
+- [Migration Guide](MIGRATION-GUIDE.md) — Upgrading from v2.x