claude-flow-novice 2.4.2 → 2.5.0

package/CLAUDE.md

@@ -0,0 +1,346 @@
+# Claude Flow Novice — AI Agent Orchestration
+
+**🚀 Production Status:** Redis coordination system fully deployed (Phase 7 - 2025-10-17)
+
+---
+
+## 1) Critical Rules (Single Source of Truth)
+
+**Main Chat Role (Thin Orchestration Layer):**
+* Main chat does ONLY: minimal investigation → determine task type → spawn coordinator + agents in single message → wait for results
+* ALL coordination happens via Redis between coordinator and agents
+* Main chat does NOT orchestrate agents directly - coordinator handles all agent coordination
+* Agents communicate via Redis pub/sub with explicit dependencies (see `.claude/redis-agent-dependencies.md`)
+
+**Core Principles:**
+* Use agents for all non-trivial work (≥3 steps or multi-file/research/testing/architecture/security)
+* **PRIMARY COORDINATOR: Use coordinator-hybrid for all multi-agent coordination** (cost-optimized CLI spawning with typed agents)
+* Initialize swarm before multi-agent work
+* Batch operations: spawn ALL agents (coordinator + workers) in single message
+* Run post-edit hook after every file edit
+* **REQUIRED: All CLI agent spawning must use explicit --agents flag with typed agents**
+* **Context continuity:** Redis/SQLite persistence enables unlimited continuation — never stop due to context/token concerns
+
+**Prohibited Patterns:**
+* Main chat orchestrating agents directly — spawn coordinator to handle orchestration
+* Spawning agents across multiple messages — use single message for coordinator + all agents
+* Working solo on multi-step tasks — spawn parallel specialists via coordinator-hybrid
+* Using generic "coordinator" fallback when coordinator-hybrid available
+* Spawning agents without explicit --agents flag (must specify types from AVAILABLE-AGENTS.md)
+* Agent coordination without Redis pub/sub messaging — ALL agents must use Redis
+* Running tests inside agents — coordinator runs tests ONCE; workers read cached results from test-results.json
+* Concurrent test runs — terminate previous runs first
+* **Saving to project root — check `.artifacts/logs/post-edit-pipeline.log` after writes; move files if ROOT_WARNING**
+* Creating guides/summaries/reports unless explicitly asked
+* Asking permission to retry/advance when criteria/iterations allow — **relaunch agents immediately when consensus <threshold and iterations <max**
+* Stopping work due to context/token concerns — Redis/SQLite persistence handles continuation automatically
+
+**Communication:**
+* Use spartan language
+* Redis persistence enables swarm recovery — state survives interruptions
+* ALL agent communication MUST use Redis pub/sub — no direct file coordination
+
+**Consensus thresholds** (mode-dependent)
+
+* Standard mode: Gate ≥0.75 • Consensus ≥0.90 • 4 validators • single PO
+* MVP mode: Gate ≥0.65 • Consensus ≥0.85 • 2 validators • single PO
+* Enterprise mode: Gate ≥0.85 • Consensus ≥0.95 • 5 validators • 4-person board • Loop 0.5 planning
+
+---
+
+## 2) When Agents Are Mandatory (Triggers)
+
+If **any** apply, spawn coordinator-hybrid (which spawns typed specialist agents):
+
+* > 3 distinct steps • multiple files • research+implement+test • design decisions • code review/quality • security/performance/compliance • system integration • docs generation • refactor/optimize • any feature work
+
+**Coordinator Selection Priority:**
+1. **coordinator-hybrid** (PRIMARY) - All multi-agent work, CLI spawning with --agents flag, cost-optimized ($0 coordinator + $0.50 workers)
+2. **adaptive-coordinator** - Only for 8+ agents with dynamic topology switching
+3. **coordinator** - FALLBACK ONLY when specialized coordinators unavailable
+
+**Required Spawning Pattern:**
+```bash
+# ✅ CORRECT: Explicit typed agents
+node src/cli/hybrid-routing/spawn-workers.js \
+  "Task description" \
+  --agents=analyst,architect,coder \
+  --provider zai
+
+# ❌ WRONG: Missing --agents flag (will error)
+node src/cli/hybrid-routing/spawn-workers.js "Task description" --max-agents 3
+```
+
+---
+
+## 3) Execution Patterns
+
+### 3.1 Swarm Init → Spawn (Single Message)
+
+**Swarm Init Pattern: ONCE per phase, not per round**
+```bash
+# Phase-level initialization (persistent through all loops)
+executeSwarm({
+  swarmId: "phase-0-mcp-less-foundation",
+  objective: "Phase 0: MCP-Less Foundation",
+  strategy: "development",
+  mode: "mesh",
+  persistence: true
+})
+```
+
+**Redis-backed Swarm Execution**:
+```bash
+npx claude-flow-novice swarm "Create REST API with authentication" --strategy development --max-agents 3
+```
+
+**Topology**: mesh (2–7), hierarchical (8+)
+
+### 3.2 Post-Edit Hook (Mandatory)
+
+```bash
+node config/hooks/post-edit-pipeline.js "[FILE]" --memory-key "swarm/[agent]/[step]"
+```
+
+**Hook runs automatically after Edit/Write/MultiEdit** but output is NOT captured by default.
+
+**Root Directory Warnings:**
+If file created in root, hook returns `status: "ROOT_WARNING"` with `rootWarning.suggestions[]`. Agent MUST:
+1. Check log file: `.artifacts/logs/post-edit-pipeline.log` (last entry)
+2. If `status: "ROOT_WARNING"`, move file to suggested location
+3. Common suggestions: `src/`, `docs/`, `config/`, `tests/`, `scripts/`
+
+**Example Response:**
+```json
+{
+  "status": "ROOT_WARNING",
+  "rootWarning": {
+    "suggestions": [
+      {"location": "src/example.js", "reason": "Source code directory"},
+      {"location": "docs/example.md", "reason": "Documentation directory"}
+    ]
+  }
+}
+```
+
+### 3.3 Safe Test Execution
+
+**Pattern: Coordinator runs tests ONCE before spawning workers**
+
+```bash
+# 1. Coordinator terminates any existing test runs
+pkill -f vitest; pkill -f "npm test"
+
+# 2. Coordinator runs tests once and caches results
+npm test -- --run --reporter=json > test-results.json 2>&1
+
+# 3. Workers read cached results only (no test execution)
+cat test-results.json
+
+# 4. Cleanup after all work complete
+pkill -f vitest; pkill -f "npm test"
+```
+
+**Rules:**
+* Coordinator executes tests before worker spawn
+* Workers ONLY read test-results.json (never run tests)
+* Single test execution prevents concurrent run conflicts
+* Cache results in test-results.json for worker consumption
+
+### 3.4 Batching (One message = all related ops)
+
+* Spawn all agents with Task tool in one message
+* Batch file ops, bash, todos, memory ops
+
+---
+
+## 4) CFN Loop Overview
+
+**→ Full CFN Loop rules: `.claude/cfn-loop-rules.md` (auto-injected by CFN commands)**
+
+**Mode Comparison:**
+
+| Mode | Gate | Consensus | Iterations | Validators |
+|------|------|-----------|------------|------------|
+| MVP | ≥0.65 | ≥0.85 | 5 | 2 |
+| Standard | ≥0.75 | ≥0.90 | 10 | 4 |
+| Enterprise | ≥0.85 | ≥0.95 | 15 | 5 |
+
+**Coordinator Patterns:** See `.claude/coordinator-patterns.md`
+
+---
+
+## 5) Coordination Checklist
+
+**Before**: initialize SQLite memory system → assess complexity → set agent count/types → choose topology → prepare single spawn message
+
+**During**: coordinate via SwarmMemory → post-edit hook after every edit → self-validate and report confidence
+
+**After**: achieve ≥0.80-0.95 validator consensus → store results → auto next steps
+
+---
+
+## 6) Hook Feedback System (Phase 4.5)
+
+**Auto-enabled:** Agents receive real-time feedback from post-edit hook via Redis.
+
+### CLI Mode (Direct Subscription)
+CLI-spawned agents auto-subscribe to `agent:{agentId}:feedback`:
+- Feedback delivered within 100ms
+- Written to `.artifacts/agents/{agentId}/pending-feedback.json`
+
+### Task Mode (Coordinator-Mediated)
+Task-spawned agents receive feedback via coordinator wake:
+- Coordinator polls `coordinator:{id}:feedback` every 5s
+- On feedback: Coordinator wakes agent with system reminder
+
+### Feedback Types (Priority Order)
+
+| Type | Severity | Action Required |
+|------|----------|-----------------|
+| `ROOT_WARNING` | High | Move file from root to suggested location |
+| `TDD_VIOLATION` | High | Write tests before continuing |
+| `LOW_COVERAGE` | Medium | Increase test coverage to threshold |
+| `RUST_QUALITY` | Medium | Fix code quality issues |
+| `LINT_ISSUES` | Low | Fix linting errors |
+
+### Handling Feedback (Agents)
+
+**ROOT_WARNING example:**
+```bash
+# Feedback: File created in root
+mv test.txt src/test.txt  # Move to suggested location
+```
+
+**→ Pattern: `.claude/coordinator-feedback-pattern.md`**
+
+### 6.3 Dashboard Integration (Phase 5)
+
+**Real-time Dashboard:**
+- URL: http://localhost:3001 (RealtimeServer)
+- WebSocket: ws://localhost:3001/ws
+- Components: RedisCoordinationMonitor.tsx
+
+**REST API Endpoints:**
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| /api/redis/feedback | GET | Recent feedback (limit=100) |
+| /api/redis/metrics | GET | Current metrics snapshot |
+| /api/swarm/status | GET | Current swarm coordination status |
+
+**CLI Monitoring Commands:**
+```bash
+# Monitor feedback
+./scripts/monitor-swarm-redis.sh feedback
+
+# Monitor CFN Loop
+./scripts/monitor-swarm-redis.sh coordination
+
+# Realtime dashboard launch
+npx claude-flow-novice dashboard --port 3001
+```
+
+**Post-Spawn Validation:**
+```bash
+# Validate CLI agent
+node config/hooks/post-spawn-validation.js coder-1
+
+# Validate Task agent
+node config/hooks/post-spawn-validation.js task_abc123 --coordinator-id coordinator-cfn
+```
+
+**WebSocket Event Types:**
+- `swarm:coordination`
+- `agent:feedback`
+- `system:metrics`
+- `dashboard:status`
+
+---
+
+## 7) Commands & Setup
+
+**Swarm Execution:**
+```bash
+npx claude-flow-novice swarm "Create REST API" --strategy development --max-agents 3
+redis-cli publish "swarm:coordination" '{"agent":"id","status":"message"}'
+```
+
+**CFN Loop Commands:**
+```bash
+/cfn-loop "Task" --mode=mvp|standard|enterprise
+/cfn-loop-sprints "Phase" --sprints=3
+/cfn-loop-epic "Epic" --phases=4
+```
+
+---
+
+## 8) SQLite Memory System
+
+**5-Level ACL:**
+
+| Level | Scope | Encryption | Loop Use |
+|-------|-------|------------|----------|
+| 1 | Agent | AES-256 | Loop 3 |
+| 2 | Team | AES-256 | Team sync |
+| 3 | Swarm | None | Loop 2 |
+| 4 | Project | None | Loop 4 |
+| 5 | System | Master key | Audit |
+
+**Agent Usage:**
+```javascript
+const memory = new SQLiteMemorySystem({ swarmId, agentId, dbPath });
+await memory.initialize();
+await memory.memoryAdapter.set(key, value, { agentId, aclLevel, namespace });
+const data = await memory.memoryAdapter.get(key, { agentId });
+```
+
+**Architecture**: Redis (hot, 1h TTL) + SQLite (persistent, 30-365d)
+**Performance**: Write <60ms, Read <5ms (Redis) / <20ms (SQLite)
+
+**→ Detailed commands: `readme/additional-commands.md` Section "SQLite Memory & ACL Commands"**
+
+---
+
+## 8) Output & Telemetry (Concise)
+
+**Agent confidence JSON (per agent)**
+```json
+{ "agent": "coder-1", "confidence": 0.85, "reasoning": "tests pass; security clean", "blockers": [] }
+```
+
+**Next steps block**
+* ✅ Completed: brief list
+* 📊 Validation: confidence, coverage, consensus
+* 🔍 Issues: debt/warnings
+* 💡 Recommendations: prioritized
+
+---
+
+## 9) CLI Command Reference
+
+### Swarm Management
+```bash
+# Initialize and execute swarms
+npx claude-flow-novice swarm "Objective description" --strategy development --max-agents 5
+npx claude-flow-novice swarm "Research cloud patterns" --strategy research
+
+# Monitor swarm status
+claude-flow-novice swarm status
+claude-flow-novice metrics --format=json
+```
+
+### Development Workflows
+```bash
+# Execute CFN Loop
+/cfn-loop "Implement authentication system" --phase=auth --mode=standard
+/cfn-loop "Build MVP prototype" --mode=mvp
+/cfn-loop "Production API" --mode=enterprise
+
+# Epic and sprint orchestration
+/cfn-loop-sprints "E-commerce platform" --sprints=3 --mode=enterprise
+/cfn-loop-epic "User management system" --phases=4 --mode=standard
+```
+
+---
+
+For specialized commands (fullstack development, SPARC methodology, fleet management, event bus, compliance, performance, markdown validation, utilities, metrics reporting, WASM optimization, build/deployment, neural operations, GitHub integration, workflow automation, security/monitoring, debugging, and SDK integration), see `readme/additional-commands.md`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-flow-novice",
-  "version": "2.4.2",
+  "version": "2.5.0",
   "description": "AI Agent Orchestration CLI",
   "main": "src/index.ts",
   "bin": {
@@ -14,6 +14,7 @@
     "src",
     "README.md",
     "CHANGELOG.md",
+    "CLAUDE.md",
     "LICENSE",
     "SYNC_USAGE.md",
     "config",
@@ -22,6 +23,7 @@
     "readme"
   ],
   "scripts": {
+    "postinstall": "node scripts/postinstall.js",
     "build": "npm run clean && tsc && rollup -c",
     "clean": "rimraf dist",
     "lint": "eslint . --ext .ts",
@@ -74,6 +76,16 @@
     "typescript": "^5.9.3"
   },
   "dependencies": {
-    "ioredis": "^5.8.1"
+    "ioredis": "^5.8.1",
+    "redis": "^4.7.0",
+    "socket.io-client": "^4.8.1"
+  },
+  "peerDependencies": {
+    "better-sqlite3": "^11.0.0"
+  },
+  "peerDependenciesMeta": {
+    "better-sqlite3": {
+      "optional": true
+    }
   }
 }

package/scripts/postinstall.js

@@ -1,94 +1,122 @@
 #!/usr/bin/env node
+const fs = require('node:fs');
+const path = require('node:path');
+const { execSync } = require('node:child_process');
+
+// Logging utility
+function log(message, type = 'info') {
+  const colors = {
+    info: '\x1b[36m',  // Cyan
+    warn: '\x1b[33m', // Yellow
+    error: '\x1b[31m' // Red
+  };
+  console.log(`${colors[type]}[Claude Flow] ${message}\x1b[0m`);
+}
 
-/**
- * Post-installation script for claude-flow-novice
- * Copies .claude directory from package to project root
- * Overwrites existing files to ensure updates work correctly
- */
+// Generate timestamped backup filename
+function getBackupFilename(basePath) {
+  const timestamp = new Date().toISOString()
+    .replace(/:/g, '-')
+    .replace(/\./g, '_');
+  return `${basePath}_backup_${timestamp}`;
+}
 
-import { existsSync, readdirSync, statSync, copyFileSync, mkdirSync } from 'fs';
-import { dirname, join, relative } from 'path';
-import { fileURLToPath } from 'url';
+// Check if running in development mode (not as installed dependency)
+function isDevMode() {
+  try {
+    const packageJson = require(path.resolve(process.cwd(), 'package.json'));
+    // We're in dev mode if the package name matches (developing the package itself)
+    return packageJson.name === 'claude-flow-novice';
+  } catch {
+    return false;
+  }
+}
 
-const __filename = fileURLToPath(import.meta.url);
-const __dirname = dirname(__filename);
+// Perform backup before sync
+async function backupDirectory(sourcePath, targetPath) {
+  if (fs.existsSync(targetPath)) {
+    const backupPath = getBackupFilename(targetPath);
+    try {
+      fs.renameSync(targetPath, backupPath);
+      log(`Created backup: ${backupPath}`, 'info');
+    } catch (err) {
+      log(`Failed to create backup: ${err.message}`, 'warn');
+    }
+  }
+}
 
+// Sync directories
+async function syncDirectories() {
+  // Skip if we're in development mode (developing the package itself)
+  if (isDevMode()) {
+    log('Development mode detected - skipping auto-sync', 'info');
+    return;
+  }
 
-/**
- * Copy .claude directory from node_modules to project root
- * Shows progress for each file copied
- */
-function copyClaudeDirectory() {
-  try {
-    const projectRoot = process.cwd();
-
-    // Try multiple source locations (handles both dev and installed package scenarios)
-    const possibleSources = [
-      join(__dirname, '../dist/.claude'),  // Installed package: node_modules/claude-flow-novice/scripts/ → dist/.claude
-      join(__dirname, '../.claude'),       // Direct execution from repo root
-      join(__dirname, '../../.claude'),    // Execution from dist/scripts/
-    ];
-
-    const sourceDir = possibleSources.find(dir => existsSync(dir));
-    const targetDir = join(projectRoot, '.claude');
-
-    console.log('🚀 claude-flow-novice post-install: Setting up .claude directory...');
-    console.log('📂 Source:', sourceDir);
-    console.log('📁 Target:', targetDir);
-    console.log('');
-
-    // Check if source directory was found
-    if (!sourceDir) {
-      console.error('❌ Source .claude directory not found. Tried:');
-      possibleSources.forEach(dir => console.error(`   - ${dir}`));
-      console.log('   This indicates a broken npm package installation.');
-      process.exit(1);
+  log('Auto-syncing agents, commands, and hooks to your project...', 'info');
+
+  const projectRoot = process.cwd();
+
+  const syncConfigs = [
+    {
+      source: path.join(__dirname, '..', '.claude', 'agents'),
+      target: path.join(projectRoot, '.claude', 'agents')
+    },
+    {
+      source: path.join(__dirname, '..', '.claude', 'commands'),
+      target: path.join(projectRoot, '.claude', 'commands')
+    },
+    {
+      source: path.join(__dirname, '..', 'config', 'hooks'),
+      target: path.join(projectRoot, 'config', 'hooks')
     }
-
-    let fileCount = 0;
-
-    /**
-     * Recursively copy directory with progress output
-     */
-    function copyWithProgress(source, target) {
-      const stat = statSync(source);
-
-      if (stat.isDirectory()) {
-        // Create directory if it doesn't exist
-        if (!existsSync(target)) {
-          mkdirSync(target, { recursive: true });
-        }
-
-        // Copy all entries in directory
-        const entries = readdirSync(source);
-        for (const entry of entries) {
-          copyWithProgress(join(source, entry), join(target, entry));
-        }
-      } else {
-        // Copy file and show progress
-        const relativePath = relative(sourceDir, source);
-        const status = existsSync(target) ? '♻️  Overwriting' : '📄 Copying';
-        console.log(`${status}: ${relativePath}`);
-
-        copyFileSync(source, target);
-        fileCount++;
+  ];
+
+  for (const config of syncConfigs) {
+    try {
+      // Create target directory if it doesn't exist
+      fs.mkdirSync(path.dirname(config.target), { recursive: true });
+
+      // Backup existing directory
+      await backupDirectory(config.source, config.target);
+
+      // Perform directory copy using cp command
+      const sourcePath = config.source;
+      const targetPath = config.target;
+
+      try {
+        execSync(`cp -r "${sourcePath}" "${targetPath}"`, { stdio: 'ignore' });
+        log(`Synced ${path.basename(config.source)}/ to project`, 'info');
+      } catch (cpError) {
+        // Fallback to recursive copy for Windows
+        copyDirRecursive(sourcePath, targetPath);
+        log(`Synced ${path.basename(config.source)}/ to project`, 'info');
       }
+    } catch (err) {
+      log(`Sync failed for ${path.basename(config.source)}: ${err.message}`, 'error');
     }
+  }
+}
 
-    // Start copying
-    copyWithProgress(sourceDir, targetDir);
+// Fallback recursive copy for Windows
+function copyDirRecursive(src, dest) {
+  fs.mkdirSync(dest, { recursive: true });
+  const entries = fs.readdirSync(src, { withFileTypes: true });
 
-    console.log('');
-    console.log(`✅ Successfully copied ${fileCount} files`);
-    console.log('📁 Location:', targetDir);
-    console.log('🎉 Installation complete! claude-flow-novice is ready to use.');
+  for (const entry of entries) {
+    const srcPath = path.join(src, entry.name);
+    const destPath = path.join(dest, entry.name);
 
-  } catch (error) {
-    console.error('❌ Post-install script failed:', error.message);
-    console.error('   Please run manually: cp -r node_modules/claude-flow-novice/dist/.claude .claude');
-    process.exit(1);
+    if (entry.isDirectory()) {
+      copyDirRecursive(srcPath, destPath);
+    } else {
+      fs.copyFileSync(srcPath, destPath);
+    }
   }
 }
 
-// Run the installation
-copyClaudeDirectory();
+// Run sync
+syncDirectories().catch(err => {
+  log(`Sync encountered an error: ${err.message}`, 'error');
+  process.exit(1);
+});

package/scripts/toggle-cost-savings.cjs

@@ -1,4 +1,4 @@
-#!/usr/bin/env node
+#!/usr/bin/env node
 /**
  * Cost-Savings Mode Toggle Script
  * Switches between CLI-based (cost-savings) and Task-tool coordination

package/scripts/validate-agent-hooks.js

@@ -1,4 +1,4 @@
-#!/usr/bin/env node
+#!/usr/bin/env node
 
 /**
  * Agent Lifecycle Hooks Validator

package/src/cli/hybrid-routing/spawn-workers.js

@@ -1,4 +1,4 @@
-#!/usr/bin/env node
+#!/usr/bin/env node
 
 /**
  * Real Hybrid Routing CLI - Spawns actual Claude agents with bash execution