claude-flow 2.7.33 → 2.7.35

package/docs/troubleshooting/wsl-better-sqlite3-error.md

@@ -0,0 +1,239 @@
+# WSL better-sqlite3 Error - Troubleshooting Guide
+
+## ⚡ Automatic Error Recovery (v2.7.35+)
+
+**Good news!** Starting with v2.7.35, claude-flow includes **automatic error recovery** that handles this issue without manual intervention.
+
+### What Happens Automatically:
+1. ✅ Detects ENOTEMPTY and better-sqlite3 errors
+2. ✅ Cleans npm/npx cache automatically
+3. ✅ Applies WSL-specific fixes
+4. ✅ Retries initialization (up to 5 times with `--force`)
+5. ✅ Falls back to JSON storage if SQLite fails
+
+### Just Run:
+```bash
+npx claude-flow@alpha init --force
+```
+
+The `--force` flag enables **automatic error recovery** and will:
+- Detect and clean npm cache errors
+- Apply WSL environment optimizations
+- Retry up to 5 times with exponential backoff
+- Automatically switch to JSON storage if needed
+
+---
+
+## Error Description
+```
+[Error: ENOTEMPTY: directory not empty, rmdir '/home/username/.npm/_npx/xxxxx/node_modules/better-sqlite3']
+errno: -39
+```
+
+When running: `npx claude-flow@alpha init --force` on Windows Subsystem for Linux (WSL)
+
+**Note:** If you're using v2.7.35+, automatic recovery handles this. Manual fixes below are only needed for older versions or edge cases.
+
+## Root Causes
+
+1. **File locking conflicts** between Windows and WSL filesystems
+2. **NPX cache corruption** due to interrupted installations
+3. **Permission issues** with npm cache directories
+4. **Native module compilation** issues specific to WSL
+
+## Solutions (Try in order)
+
+### Solution 1: Clear NPM/NPX Cache
+```bash
+# Clear npm cache
+npm cache clean --force
+
+# Remove npx cache directory
+rm -rf ~/.npm/_npx
+
+# Retry installation
+npx claude-flow@alpha init --force
+```
+
+### Solution 2: Use npm instead of npx
+```bash
+# Install globally first
+npm install -g claude-flow@alpha
+
+# Then run init
+claude-flow init --force
+```
+
+### Solution 3: Manual Directory Cleanup
+```bash
+# Find the problematic directory
+ls -la ~/.npm/_npx/
+
+# Force remove with elevated permissions if needed
+sudo rm -rf ~/.npm/_npx/*/node_modules/better-sqlite3
+
+# Clear entire npx cache
+rm -rf ~/.npm/_npx
+
+# Retry
+npx claude-flow@alpha init --force
+```
+
+### Solution 4: Fix WSL File Permissions
+```bash
+# Ensure proper ownership
+sudo chown -R $(whoami) ~/.npm
+
+# Fix permissions
+chmod -R 755 ~/.npm
+
+# Clear and retry
+npm cache clean --force
+npx claude-flow@alpha init --force
+```
+
+### Solution 5: Rebuild better-sqlite3
+```bash
+# Install build tools if missing
+sudo apt-get update
+sudo apt-get install -y build-essential python3
+
+# Clear cache and retry with rebuild flag
+npm cache clean --force
+rm -rf ~/.npm/_npx
+npx claude-flow@alpha init --force
+```
+
+### Solution 6: Use WSL2 with Proper Node Version
+```bash
+# Check WSL version
+wsl --list --verbose
+
+# Ensure using WSL2 (not WSL1)
+wsl --set-version Ubuntu 2
+
+# Use Node 18+ (better-sqlite3 compatibility)
+node --version
+
+# Install/update node if needed via nvm
+curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
+source ~/.bashrc
+nvm install 20
+nvm use 20
+
+# Retry installation
+npx claude-flow@alpha init --force
+```
+
+### Solution 7: Run from Linux Filesystem (Not Windows Mount)
+```bash
+# Bad: Running from /mnt/c/Users/... (Windows filesystem)
+cd /mnt/c/Users/username/project  # ❌
+
+# Good: Running from WSL filesystem
+cd ~/projects/your-project  # ✅
+
+# Copy project to WSL if needed
+cp -r /mnt/c/Users/username/project ~/projects/
+
+# Run from WSL filesystem
+cd ~/projects/project
+npx claude-flow@alpha init --force
+```
+
+## Prevention
+
+### Best Practices for WSL Users
+
+1. **Always work in WSL filesystem** (`~/` not `/mnt/c/`)
+2. **Use Node 18+** for better native module support
+3. **Keep npm updated**: `npm install -g npm@latest`
+4. **Regular cache cleanup**: Add to `.bashrc`:
+   ```bash
+   alias npm-clean="npm cache clean --force && rm -rf ~/.npm/_npx"
+   ```
+
+## Quick Fix Script
+
+Create a script to automate the fix:
+
+```bash
+#!/bin/bash
+# wsl-fix-npx.sh
+
+echo "🔧 Fixing WSL NPX cache issues..."
+
+# Stop any running node processes
+pkill -f node || true
+
+# Clean npm cache
+echo "📦 Cleaning npm cache..."
+npm cache clean --force
+
+# Remove npx cache
+echo "🗑️  Removing npx cache..."
+rm -rf ~/.npm/_npx
+
+# Fix permissions
+echo "🔐 Fixing permissions..."
+sudo chown -R $(whoami) ~/.npm
+chmod -R 755 ~/.npm
+
+# Verify node/npm
+echo "✅ Verifying Node.js..."
+node --version
+npm --version
+
+echo "🎉 Cleanup complete! Try running your command again."
+echo "Command: npx claude-flow@alpha init --force"
+```
+
+Usage:
+```bash
+chmod +x wsl-fix-npx.sh
+./wsl-fix-npx.sh
+npx claude-flow@alpha init --force
+```
+
+## Still Having Issues?
+
+### Report the Issue
+If none of the solutions work, gather this information:
+
+```bash
+# System info
+cat /etc/os-release
+node --version
+npm --version
+wsl --list --verbose  # Run from Windows PowerShell
+
+# Error details
+npx claude-flow@alpha init --force --verbose 2>&1 | tee error-log.txt
+```
+
+Then report at: https://github.com/ruvnet/claude-flow/issues
+
+### Alternative: Use Docker
+If WSL issues persist, consider using Docker:
+
+```bash
+# Pull claude-flow Docker image (if available)
+docker pull ruvnet/claude-flow:latest
+
+# Run in container
+docker run -it -v $(pwd):/workspace ruvnet/claude-flow:latest init --force
+```
+
+## Technical Background
+
+The `ENOTEMPTY` error occurs because:
+
+1. **WSL filesystem translation layer** can cause delays in file operations
+2. **better-sqlite3** is a native Node.js module requiring compilation
+3. **NPX temporary directories** may not be fully cleaned before reuse
+4. **Windows Defender** or antivirus may lock files during scanning
+
+The error code `-39` (ENOTEMPTY) means the system tried to remove a directory that still contains files, typically due to:
+- Race conditions in cleanup
+- File handles still open
+- Filesystem caching inconsistencies

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-flow",
-  "version": "2.7.33",
+  "version": "2.7.35",
   "description": "Enterprise-grade AI agent orchestration with WASM-powered ReasoningBank memory and AgentDB vector database (always uses latest agentic-flow)",
   "mcpName": "io.github.ruvnet/claude-flow",
   "main": "cli.mjs",
@@ -121,6 +121,9 @@
     "@anthropic-ai/sdk": "^0.65.0",
     "@modelcontextprotocol/sdk": "^1.0.4",
     "agentic-flow": "^1.9.4",
+    "ajv": "^8.17.1",
+    "ajv-errors": "^3.0.0",
+    "ajv-formats": "^3.0.1",
     "blessed": "^0.1.81",
     "chalk": "^4.1.2",
     "cli-table3": "^0.6.3",
@@ -137,6 +140,7 @@
     "ora": "^7.0.1",
     "p-queue": "^8.1.0",
     "ruv-swarm": "^1.0.14",
+    "uuid": "^13.0.0",
     "ws": "^8.18.3",
     "yaml": "^2.8.0",
     "yoctocolors-cjs": "^2.1.3"
@@ -168,7 +172,6 @@
     "@typescript-eslint/eslint-plugin": "^6.21.0",
     "@typescript-eslint/parser": "^6.21.0",
     "@vercel/ncc": "^0.38.3",
-    "agentic-jujutsu": "^2.3.2",
     "babel-jest": "^29.7.0",
     "eslint": "^8.57.1",
     "eslint-plugin-react": "^7.37.5",

package/scripts/create-github-issue.sh

@@ -0,0 +1,64 @@
+#!/bin/bash
+# Create GitHub Issue for Automatic Error Recovery
+# Run this after confirming fix works in Docker/CLI
+
+set -e
+
+echo "📝 Creating GitHub Issue for Automatic Error Recovery"
+echo ""
+
+# Check if gh CLI is installed
+if ! command -v gh &> /dev/null; then
+    echo "❌ GitHub CLI (gh) is not installed"
+    echo "   Install: https://cli.github.com/"
+    exit 1
+fi
+
+# Check if authenticated
+if ! gh auth status &> /dev/null; then
+    echo "❌ Not authenticated with GitHub CLI"
+    echo "   Run: gh auth login"
+    exit 1
+fi
+
+echo "✅ GitHub CLI ready"
+echo ""
+
+# Issue title
+TITLE="✅ Fixed: Automatic recovery for WSL better-sqlite3 ENOTEMPTY error during init"
+
+# Issue body (from template)
+BODY=$(cat docs/github-issues/wsl-enotempty-automatic-recovery.md)
+
+# Labels
+LABELS="enhancement,bug-fix,wsl,user-experience,v2.7.35"
+
+# Milestone
+MILESTONE="v2.7.35"
+
+echo "Creating issue..."
+echo "Title: $TITLE"
+echo "Labels: $LABELS"
+echo "Milestone: $MILESTONE"
+echo ""
+
+# Create the issue
+gh issue create \
+    --title "$TITLE" \
+    --body "$BODY" \
+    --label "$LABELS" \
+    --milestone "$MILESTONE"
+
+if [ $? -eq 0 ]; then
+    echo ""
+    echo "✅ GitHub issue created successfully!"
+    echo ""
+    echo "Next steps:"
+    echo "1. Add test results to the issue"
+    echo "2. Attach screenshots if available"
+    echo "3. Request review from maintainers"
+else
+    echo ""
+    echo "❌ Failed to create issue"
+    exit 1
+fi

package/scripts/test-docker-wsl.sh

@@ -0,0 +1,198 @@
+#!/bin/bash
+# Test automatic error recovery in Docker (simulates WSL)
+# Run this before creating GitHub issue
+
+set -e
+
+echo "🐳 Testing Automatic Error Recovery in Docker"
+echo ""
+
+# Colors for output
+GREEN='\033[0;32m'
+RED='\033[0;31m'
+YELLOW='\033[1;33m'
+NC='\033[0m' # No Color
+
+# Test configurations
+DISTROS=("ubuntu:22.04" "ubuntu:20.04" "debian:11" "debian:12")
+TEST_RESULTS=()
+
+# Function to test on a distro
+test_distro() {
+    local distro=$1
+    echo ""
+    echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+    echo "Testing on: $distro"
+    echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+    echo ""
+
+    # Run test in Docker
+    docker run --rm -i $distro bash -c "
+        set -e
+
+        echo '📦 Installing dependencies...'
+        apt-get update -qq
+        apt-get install -y -qq curl build-essential python3 git > /dev/null 2>&1
+
+        echo '📥 Installing Node.js...'
+        curl -fsSL https://deb.nodesource.com/setup_20.x | bash - > /dev/null 2>&1
+        apt-get install -y -qq nodejs > /dev/null 2>&1
+
+        echo '🔍 Node version:'
+        node --version
+        npm --version
+
+        echo ''
+        echo '🚀 Running claude-flow init --force...'
+        echo ''
+
+        # Run the actual test
+        npx claude-flow@alpha init --force
+
+        echo ''
+        echo '✅ Test completed successfully on $distro'
+    " && {
+        echo -e "${GREEN}✅ PASS: $distro${NC}"
+        TEST_RESULTS+=("PASS: $distro")
+    } || {
+        echo -e "${RED}❌ FAIL: $distro${NC}"
+        TEST_RESULTS+=("FAIL: $distro")
+    }
+}
+
+# Function to test with corrupted cache simulation
+test_corrupted_cache() {
+    echo ""
+    echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+    echo "Testing with corrupted npm cache simulation"
+    echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+    echo ""
+
+    docker run --rm -i ubuntu:22.04 bash -c "
+        set -e
+
+        apt-get update -qq
+        apt-get install -y -qq curl build-essential python3 git > /dev/null 2>&1
+        curl -fsSL https://deb.nodesource.com/setup_20.x | bash - > /dev/null 2>&1
+        apt-get install -y -qq nodejs > /dev/null 2>&1
+
+        echo '💥 Simulating corrupted npm cache...'
+        mkdir -p ~/.npm/_npx/test-corrupt/node_modules/better-sqlite3
+        touch ~/.npm/_npx/test-corrupt/node_modules/better-sqlite3/.lock
+
+        echo ''
+        echo '🚀 Running claude-flow init --force with corrupted cache...'
+        echo ''
+
+        npx claude-flow@alpha init --force
+
+        echo ''
+        echo '✅ Recovery from corrupted cache successful'
+    " && {
+        echo -e "${GREEN}✅ PASS: Corrupted cache recovery${NC}"
+        TEST_RESULTS+=("PASS: Corrupted cache recovery")
+    } || {
+        echo -e "${RED}❌ FAIL: Corrupted cache recovery${NC}"
+        TEST_RESULTS+=("FAIL: Corrupted cache recovery")
+    }
+}
+
+# Function to test without --force flag
+test_without_force() {
+    echo ""
+    echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+    echo "Testing without --force flag (3 retries max)"
+    echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+    echo ""
+
+    docker run --rm -i ubuntu:22.04 bash -c "
+        set -e
+
+        apt-get update -qq
+        apt-get install -y -qq curl build-essential python3 git > /dev/null 2>&1
+        curl -fsSL https://deb.nodesource.com/setup_20.x | bash - > /dev/null 2>&1
+        apt-get install -y -qq nodejs > /dev/null 2>&1
+
+        echo '🚀 Running claude-flow init (no --force)...'
+        echo ''
+
+        npx claude-flow@alpha init
+
+        echo ''
+        echo '✅ Init without --force successful'
+    " && {
+        echo -e "${GREEN}✅ PASS: Init without --force${NC}"
+        TEST_RESULTS+=("PASS: Init without --force")
+    } || {
+        echo -e "${RED}❌ FAIL: Init without --force${NC}"
+        TEST_RESULTS+=("FAIL: Init without --force")
+    }
+}
+
+# Main test execution
+echo "Starting comprehensive Docker tests..."
+echo ""
+echo "This will test:"
+echo "  - Multiple Linux distributions"
+echo "  - Corrupted cache recovery"
+echo "  - With and without --force flag"
+echo ""
+
+read -p "Continue? (y/n) " -n 1 -r
+echo ""
+
+if [[ ! $REPLY =~ ^[Yy]$ ]]; then
+    echo "Aborted"
+    exit 1
+fi
+
+# Run tests
+for distro in "${DISTROS[@]}"; do
+    test_distro "$distro"
+done
+
+test_corrupted_cache
+test_without_force
+
+# Print summary
+echo ""
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+echo "TEST SUMMARY"
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+echo ""
+
+PASS_COUNT=0
+FAIL_COUNT=0
+
+for result in "${TEST_RESULTS[@]}"; do
+    if [[ $result == PASS* ]]; then
+        echo -e "${GREEN}✅ $result${NC}"
+        ((PASS_COUNT++))
+    else
+        echo -e "${RED}❌ $result${NC}"
+        ((FAIL_COUNT++))
+    fi
+done
+
+echo ""
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+echo "Total: $((PASS_COUNT + FAIL_COUNT)) tests"
+echo -e "${GREEN}Passed: $PASS_COUNT${NC}"
+echo -e "${RED}Failed: $FAIL_COUNT${NC}"
+echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
+echo ""
+
+if [ $FAIL_COUNT -eq 0 ]; then
+    echo -e "${GREEN}✅ All tests passed!${NC}"
+    echo ""
+    echo "Next steps:"
+    echo "1. Review test results above"
+    echo "2. Run: bash scripts/create-github-issue.sh"
+    echo "3. Update issue with test results"
+    exit 0
+else
+    echo -e "${RED}❌ Some tests failed${NC}"
+    echo ""
+    echo "Please fix failing tests before creating GitHub issue"
+    exit 1
+fi

package/src/cli/commands/mcp.ts

@@ -6,10 +6,17 @@ import { Command } from '@cliffy/command';
 import chalk from 'chalk';
 import { logger } from '../../core/logger.js';
 import { configManager } from '../../core/config.js';
-import { MCPServer } from '../../mcp/server.js';
+import { MCPServer, type IMCPServer } from '../../mcp/server.js';
 import { eventBus } from '../../core/event-bus.js';
+import {
+  createMCPServer,
+  isMCP2025Available,
+  getServerCapabilities,
+  type ExtendedMCPConfig,
+} from '../../mcp/server-factory.js';
+import type { MCP2025Server } from '../../mcp/server-mcp-2025.js';
 
-let mcpServer: MCPServer | null = null;
+let mcpServer: IMCPServer | MCP2025Server | null = null;
 
 export const mcpCommand = new Command()
   .description('Manage MCP server and tools')
@@ -32,29 +39,95 @@ export const mcpCommand = new Command()
       .option('--transport <transport:string>', 'Transport type (stdio, http)', {
         default: 'stdio',
       })
+      .option('--mcp2025', 'Enable MCP 2025-11 features (version negotiation, async jobs, etc.)', {
+        default: false,
+      })
+      .option('--no-legacy', 'Disable legacy client support', { default: false })
       .action(async (options: any) => {
         try {
           const config = await configManager.load();
 
-          // Override with CLI options
-          const mcpConfig = {
+          // Check if MCP 2025-11 dependencies are available
+          const mcp2025Available = isMCP2025Available();
+          const enableMCP2025 = options.mcp2025 && mcp2025Available;
+
+          if (options.mcp2025 && !mcp2025Available) {
+            console.log(
+              chalk.yellow(
+                '⚠️  MCP 2025-11 dependencies not found. Install with: npm install uuid ajv ajv-formats ajv-errors'
+              )
+            );
+            console.log(chalk.yellow('   Falling back to legacy MCP server...'));
+          }
+
+          // Build extended configuration
+          const mcpConfig: ExtendedMCPConfig = {
             ...config.mcp,
             port: options.port,
             host: options.host,
             transport: options.transport,
+            features: {
+              enableMCP2025,
+              supportLegacyClients: options.legacy !== false,
+              enableVersionNegotiation: enableMCP2025,
+              enableAsyncJobs: enableMCP2025,
+              enableRegistryIntegration: false, // Opt-in via env var
+              enableSchemaValidation: enableMCP2025,
+              enableProgressiveDisclosure: true, // Phase 1 feature (always enabled)
+            },
+            mcp2025: enableMCP2025
+              ? {
+                  async: {
+                    enabled: true,
+                    maxJobs: 100,
+                    jobTTL: 3600000,
+                  },
+                  registry: {
+                    enabled: process.env.MCP_REGISTRY_ENABLED === 'true',
+                    url: process.env.MCP_REGISTRY_URL,
+                    apiKey: process.env.MCP_REGISTRY_API_KEY,
+                  },
+                  validation: {
+                    enabled: true,
+                    strictMode: false,
+                  },
+                }
+              : undefined,
           };
 
-          mcpServer = new MCPServer(mcpConfig, eventBus, logger);
+          // Create server using factory
+          mcpServer = await createMCPServer(mcpConfig, eventBus, logger, {
+            autoDetectFeatures: false, // Use explicit config
+          });
+
           await mcpServer.start();
 
+          // Get capabilities
+          const capabilities = getServerCapabilities(mcpConfig);
+
           console.log(chalk.green(`✅ MCP server started on ${options.host}:${options.port}`));
-          console.log(chalk.cyan(`📡 Server URL: http://${options.host}:${options.port}`));
-          console.log(chalk.cyan(`🔧 Available tools: Research, Code, Terminal, Memory`));
           console.log(
-            chalk.cyan(`📚 API documentation: http://${options.host}:${options.port}/docs`),
+            chalk.cyan(`🎯 Mode: ${enableMCP2025 ? 'MCP 2025-11 Enhanced' : 'Legacy Compatible'}`)
           );
+          console.log(chalk.cyan(`📡 Transport: ${options.transport}`));
+
+          if (capabilities.length > 0) {
+            console.log(chalk.cyan(`✨ Capabilities: ${capabilities.join(', ')}`));
+          }
+
+          if (enableMCP2025) {
+            console.log(chalk.green('   • Version negotiation (YYYY-MM format)'));
+            console.log(chalk.green('   • Async job support (poll/resume)'));
+            console.log(chalk.green('   • JSON Schema 1.1 validation'));
+            console.log(chalk.green('   • Progressive disclosure (98.7% token reduction)'));
+          }
+
+          if (options.transport === 'http') {
+            console.log(chalk.cyan(`📚 Server URL: http://${options.host}:${options.port}`));
+          }
         } catch (error) {
           console.error(chalk.red(`❌ Failed to start MCP server: ${(error as Error).message}`));
+          logger.error('MCP server startup failed', { error });
           process.exit(1);
         }
       }),
@@ -151,7 +224,11 @@ export const mcpCommand = new Command()
 
         console.log(chalk.yellow('🔄 Starting MCP server...'));
         const config = await configManager.load();
-        mcpServer = new MCPServer(config.mcp, eventBus, logger);
+
+        // Use factory to create server with same capabilities as before
+        mcpServer = await createMCPServer(config.mcp, eventBus, logger, {
+          autoDetectFeatures: true, // Auto-detect on restart
+        });
         await mcpServer.start();
 
         console.log(