claude-flow-novice 2.14.18 → 2.14.20

package/claude-assets/root-claude-distribute/CFN-CLAUDE.md

@@ -8,6 +8,7 @@
 
 ### Core Operational Rules
 * **Use agents for all non-trivial work** (≥4 steps or any multi-file / research / testing / architecture / security / integration / refactor / feature)
+* **🚨 FOR CFN LOOP CLI WORKFLOWS: Use CLI commands** - `/cfn-loop-cli "task"` (NEVER manual Task() spawning)
 * **Initialize swarm before any multi-agent work**
 * **Batch operations**: one message per related batch (spawn, file edits, bash, todos, memory ops)
 * **Run post-edit hook after every file edit** inclusive of .md files and await the response
@@ -38,10 +39,12 @@
 * Validators consensus: **≥0.90**
 
 ### CTO Delegation Persona
-* **Act as a busy CTO** who delegates all non-trivial work to specialized agents or a cfn-coordinator
+* **Act as a busy CTO** who delegates all non-trivial work to specialized agents or CFN Loop CLI commands
+* **For multi-agent workflows**: Use `/cfn-loop-cli "task description"` (automatically handles coordinator spawning)
+* **For single agent tasks**: Use `Task("agent-type", "specific task")` directly
 * **Define clear success criteria** for implementation (working code, passing tests, documented features)
 * **Never define adoption criteria** (user engagement, rollout strategy, training plans)
-* **Ruthlessly delegate** - if task requires >3 steps, spawn agents immediately
+* **Ruthlessly delegate** - if task requires >3 steps, use CLI commands immediately
 * **Provide context, not solutions** - agents figure out implementation details
 * **Success = implementation complete** - not "users love it" or "team adopts it"
 
@@ -168,32 +171,42 @@ When spawned via CLI (`npx claude-flow-novice`), you automatically benefit from
 /switch-api status
 ```
 
-**CRITICAL: Single Coordinator Pattern (v2)**
+**🚨 CRITICAL: Main Chat MUST Use CLI Mode Commands**
 
-Main Chat spawns ONLY the coordinator agent. The coordinator handles all agent spawning internally via CLI and  .claude/skills/cfn-loop-orchestration/orchestrate.sh
+**DO NOT spawn Task() agents directly for CFN Loop workflows.**
+Instead, use the dedicated CLI mode slash commands that handle coordinator spawning automatically.
 
-**❌ FORBIDDEN - Main Chat Spawning Workers:**
+**❌ FORBIDDEN - Manual Task() Spawning:**
 ```javascript
-// WRONG in v2 - Don't spawn workers from Main Chat
-Task("coordinator", "Coordinate task...")
-Task("backend-dev", "Implement feature...")  // ❌ NO
-Task("tester", "Test feature...")            // ❌ NO
+// WRONG - Don't spawn CFN Loop agents manually from Main Chat
+Task("cfn-v3-coordinator", "Execute CFN Loop...")           // ❌ NO
+Task("backend-developer", "Implement feature...")          // ❌ NO
+Task("tester", "Test feature...")                         // ❌ NO
 ```
 
-**✅ REQUIRED - Single Coordinator:**
-```javascript
-// CORRECT - Main Chat spawns only coordinator
-Task("cfn-v3-coordinator", `
-  Execute CFN Loop for: Implement authentication
-
-  Coordinator will:
-  1. Invoke .claude/skills/cfn-loop-orchestration/orchestrate.sh
-  2. Orchestrator spawns agents via CLI
-  3. Coordinator manages all Redis coordination
-  4. Return structured result to Main Chat
-`)
+**✅ REQUIRED - Use CLI Mode Slash Commands:**
+```bash
+# PRODUCTION - Enhanced CLI mode v3.0 (default)
+/cfn-loop-cli "Implement JWT authentication" --mode=standard
+
+# DEBUGGING - Task mode (full visibility)
+/cfn-loop-task "Fix security bug in auth module" --mode=standard
+
+# QUICK TASKS - Single iteration
+/cfn-loop-single "Update documentation"
+
+# LARGE EPICS - Multi-phase
+/cfn-loop-epic "Build complete authentication system"
 ```
 
+**Why CLI Mode Commands?**
+- ✅ Automatic coordinator spawning with enhanced monitoring v3.0
+- ✅ Real-time agent progress tracking and automatic recovery
+- ✅ Protocol compliance (prevents "consensus on vapor" anti-patterns)
+- ✅ 95-98% cost savings with Z.ai routing
+- ✅ Background execution with Redis persistence
+- ✅ Built-in parameter validation and success criteria templates
+
 **Why This Pattern:**
 - Coordinator controls spawn timing via orchestrate.sh (no timeout issues)
 - 95-98% cost savings vs Task() spawning
@@ -210,6 +223,7 @@ BACKUP_PATH=$(./.claude/hooks/cfn-invoke-pre-edit.sh "$FILE_TO_EDIT" --agent-id
 **Why:** Enables safe file revert without git operations during parallel sessions.
 **Location:** `.backups/[agent-id]/[timestamp]_[hash]/`
 **Retention:** 24h TTL (configurable)
+**Injection:** Automatically included in all agent prompts via `src/cli/agent-prompt-builder.ts`
 
 **Revert Instead of Git:**
 ```bash
@@ -249,10 +263,21 @@ fi
 
 ## 2) When Agents Are Mandatory (Triggers)
 
-If **any** apply, spawn agents:
+If **any** apply, use CFN Loop CLI commands:
 
 * > 3 distinct steps • multiple files • research+implement+test • design decisions • code review/quality • security/performance/compliance • system integration • docs generation • refactor/optimize • any feature work
 
+**🚨 IMPORTANT: For complex multi-agent workflows, use CLI mode commands:**
+```bash
+# Production with enhanced monitoring v3.0
+/cfn-loop-cli "Complex task description" --mode=standard
+
+# Debugging with full visibility
+/cfn-loop-task "Complex task description" --mode=standard
+```
+
+**Do NOT manually spawn Task() agents for CFN Loop workflows - the CLI commands handle coordination automatically.**
+
 ### Skill Selection Criteria
 **Mandatory Skill Spawning Triggers:**
 - Complex tasks (>3 steps)
@@ -273,16 +298,26 @@ npx claude-flow-novice swarm "Task Description" \
 
 ### Single Agent vs Coordinator
 
-**Use Single Agent:**
+**Use Single Agent (Task() directly):**
 * 1 specialized task (coding, reviewing, testing)
 * No dependencies on other agents
 * Straightforward execution
+* Simple, isolated work
 
-**Use Coordinator:**
+**Use Coordinator (CLI Commands):**
 * Multiple agents needed (2+)
 * Sequential dependencies (Loop 3 → Loop 2 → Product Owner)
 * Iteration/consensus required
-* CFN Loop workflows
+* **ALL CFN Loop workflows**
+
+**🚨 FOR CFN LOOP WORKFLOWS: Always use CLI commands - never manual Task() spawning**
+```bash
+# Multi-agent workflows (coordinator handles everything)
+/cfn-loop-cli "Build authentication system" --mode=standard
+
+# Single agent tasks (direct Task() is fine)
+Task("reviewer", "Review this specific file")
+```
 
 ## 3) Coordination Patterns
 
@@ -312,40 +347,48 @@ Refer to `.claude/skills/cfn-redis-coordination/SKILL.md` for:
 
 ### CFN Loop Orchestration Pattern
 
-**CLI Mode (Production):**
-Main Chat spawns cfn-v3-coordinator → Coordinator spawns workers via CLI → Workers exit after reporting confidence
+**CLI Mode (Production) - Enhanced v3.0:**
+Main Chat spawns cfn-v3-coordinator → Enhanced orchestrator with monitoring → Workers via CLI with progress tracking → Automatic recovery from stuck agents
 
 **Task Mode (Debugging):**
 Main Chat spawns all agents directly via Task() → No coordinator → Full visibility
 
-**Orchestrator:**
+**Enhanced Orchestrator v3.0:**
 ```bash
 ./.claude/skills/cfn-loop-orchestration/orchestrate.sh
 ```
-- Spawns Loop 3 agents (implementers)
-- Collects confidence scores
-- Gate check: spawn Loop 2 if ≥threshold
-- Spawns Loop 2 agents (validators)
-- Collects consensus
+- ✅ **Enhanced Monitoring**: Real-time agent progress tracking with stuck detection
+- ✅ **Automatic Recovery**: Dead process cleanup and agent restart capabilities
+- ✅ **Protocol Compliance**: Prevents "consensus on vapor" anti-patterns
+- ✅ **Enhanced Spawning**: Context validation and broadcast message injection
+- ✅ **Progress Visibility**: Detailed progress reports with timestamps
+- Spawns Loop 3 agents with protocol enforcement
+- Enhanced waiting with progress tracking and recovery
+- Collects confidence scores with metadata validation
+- Gate check: spawn Loop 2 if ≥threshold (with health verification)
+- Spawns Loop 2 agents (validators) with monitoring
+- Collects consensus with stuck agent detection
 - Spawns Product Owner for decision
-- Manages iterations based on PROCEED/ITERATE/ABORT
+- Manages iterations based on PROCEED/ITERATE/ABORT with timeout handling
 
 **Agent Completion Protocol (Mode-Specific):**
 
-**CLI Mode** (spawned via `npx claude-flow-novice agent-spawn`):
+**CLI Mode v3.0** (spawned via `npx claude-flow-novice agent-spawn`):
 ```bash
-# 1. Complete work
-# 2. Signal done
+# 1. Complete work with enhanced context
+# 2. Automatic context validation (prevents "consensus on vapor")
+# 3. Signal completion
 redis-cli lpush "swarm:${TASK_ID}:${AGENT_ID}:done" "complete"
 
-# 3. Report confidence and exit
-./.claude/skills/cfn-redis-coordination/invoke-waiting-mode.sh report \
+# 4. Report confidence with metadata
+./.claude/skills/cfn-redis-coordination/report-completion.sh \
   --task-id "$TASK_ID" \
   --agent-id "$AGENT_ID" \
   --confidence 0.85 \
-  --iteration 1
+  --iteration 1 \
+  --result '{"deliverables_created": ["file.ts"], "status": "complete"}'
 
-# Agent exits cleanly (no waiting mode)
+# 5. Agent exits cleanly (orchestrator monitors via enhanced waiting)
 ```
 
 **Task Mode** (spawned via Task() tool in Main Chat):
@@ -356,6 +399,12 @@ redis-cli lpush "swarm:${TASK_ID}:${AGENT_ID}:done" "complete"
 # NO explicit completion protocol needed
 ```
 
+**Enhanced Agent Protocol Requirements:**
+- ✅ **Mandatory completion signaling**: `report-completion.sh` call required
+- ✅ **Context awareness**: Broadcast messages automatically injected
+- ✅ **Metadata tracking**: Agent status and process PID monitored
+- ✅ **Health checking**: Process health validated during execution
+
 **Orchestration Flow (CORRECTED - Self-Validation Pattern):**
 1. Loop 3 agents complete work and report confidence
 2. **Gate Check:** Loop 3 self-validation scores checked
@@ -686,4 +735,28 @@ Out of Scope:
 - **Confidence:** 0.91
 - **Priority:** 8
 - **Insight**: Decompose complex systems into independent skills (20 skills in CFN v3: task-classifier, playbook, validation-templates, etc.). Enables reuse, testing isolation, and incremental enhancement.
-- **Tags**: modularity, skills, architecture, reusability
+- **Tags**: modularity, skills, architecture, reusability
+## Sprint 10 Adaptive Context Lessons (Phase X - Defensive Programming)
+
+### Defensive Programming Patterns
+
+#### PATTERN-025: Comprehensive File Validation
+- **Context**: Defensive File Handling
+- **Insight**: Implement comprehensive file validation techniques that go beyond basic existence checks. Use multi-stage validation including file type, permissions, size constraints, and content integrity checks. Create a robust validation pipeline that prevents potential security vulnerabilities and unexpected system behavior.
+- **Tags**: file-handling, defensive-programming, validation, security, system-integrity
+- **Confidence**: 0.92
+- **Priority**: 9/10
+
+#### PATTERN-026: Shell Strict Mode
+- **Context**: Bash Script Reliability
+- **Insight**: Enable shell strict mode using `set -euo pipefail` to create more robust and predictable shell scripts. This approach forces immediate exit on errors, prevents unset variable usage, and ensures pipeline failures are properly captured. Dramatically improves script reliability and makes error conditions explicit.
+- **Tags**: bash, shell-scripting, error-handling, defensive-programming, reliability
+- **Confidence**: 0.90
+- **Priority**: 9/10
+
+#### PATTERN-028: Process Group Management
+- **Context**: Background Process Handling
+- **Insight**: Implement comprehensive process group management techniques to ensure clean termination and resource cleanup. Use techniques like `trap` for signal handling, process substitution, and explicit process group management to prevent zombie processes and resource leaks in complex multi-process environments.
+- **Tags**: process-management, bash, background-processes, resource-cleanup, defensive-programming
+- **Confidence**: 0.86
+- **Priority**: 8/10

package/claude-assets/skills/cfn-docker-agent-spawning/SKILL.md

@@ -0,0 +1,394 @@
+# CFN Docker Agent Spawning Skill
+
+**Purpose:** Spawn agents in isolated Docker containers with skill-based MCP selection, resource management, and authentication.
+
+## Overview
+
+This skill manages the lifecycle of container-based agents, providing isolated execution environments with controlled resource usage, secure MCP access, and comprehensive monitoring capabilities.
+
+## Architecture
+
+```bash
+Agent Spawning Request
+    ↓
+Container Configuration (memory, CPU, volumes)
+    ↓
+Docker Container Creation
+    ↓
+MCP Token Generation & Injection
+    ↓
+Agent Initialization (claude-flow-novice agent-spawn)
+    ↓
+Resource Monitoring & Management
+```
+
+## Core Functions
+
+### 1. Container Configuration
+Generate Docker container specifications based on agent type and requirements:
+
+```bash
+# Configure container for frontend engineer
+cfn-docker-agent-spawn configure \
+  --agent-type react-frontend-engineer \
+  --memory-limit 1g \
+  --cpu-limit 1.0 \
+  --network mcp-network
+```
+
+### 2. Container Creation
+Create and start Docker containers with proper isolation:
+
+```bash
+# Spawn agent container
+cfn-docker-agent-spawn create \
+  --agent-id agent-frontend-001 \
+  --agent-type react-frontend-engineer \
+  --task-id task-authentication \
+  --context "${TASK_CONTEXT}"
+```
+
+### 3. MCP Integration
+Configure secure MCP server access with authentication tokens:
+
+```bash
+# Setup MCP access for container
+cfn-docker-agent-spawn setup-mcp \
+  --container-id agent-frontend-001 \
+  --mcp-servers playwright \
+  --token-file /tmp/mcp-tokens.json
+```
+
+### 4. Resource Management
+Monitor and manage container resources:
+
+```bash
+# Monitor container resources
+cfn-docker-agent-spawn monitor \
+  --container-id agent-frontend-001 \
+  --alert-threshold 90%
+```
+
+## Container Specification
+
+### Standard Configuration
+```yaml
+# Docker container specification
+agent-container:
+  image: claude-flow-novice:agent
+  hostname: agent-{{AGENT_ID}}
+  networks:
+    - mcp-network
+  volumes:
+    - ./.claude:/app/.claude:ro
+    - ./src:/app/src:ro
+    - agent-workspace-{{AGENT_ID}}:/app/workspace
+  environment:
+    - AGENT_ID={{AGENT_ID}}
+    - AGENT_TYPE={{AGENT_TYPE}}
+    - TASK_ID={{TASK_ID}}
+    - REDIS_URL=redis://redis:6379
+    - MCP_TOKENS_FILE=/tmp/mcp-tokens.json
+  resources:
+    memory: {{MEMORY_LIMIT}}
+    cpu: {{CPU_LIMIT}}
+  restart_policy: unless-stopped
+```
+
+### Volume Mounts
+- **Codebase**: Read-only mount for source code and skills
+- **Agent Configuration**: Read-only mount for .claude directory
+- **Workspace**: Writable mount for agent work output
+- **Token Store**: Temporary file for MCP authentication tokens
+
+### Environment Variables
+- `AGENT_ID`: Unique container identifier
+- `AGENT_TYPE`: Agent type for skill-based selection
+- `TASK_ID`: CFN Loop task identifier
+- `REDIS_URL`: Redis connection string
+- `MCP_TOKENS_FILE`: Path to MCP authentication tokens
+
+## Usage Patterns
+
+### Basic Agent Spawning
+```bash
+# Spawn single agent
+cfn-docker-agent-spawn \
+  --agent-type react-frontend-engineer \
+  --task-id "implement-ui" \
+  --memory-limit 1g
+```
+
+### Batch Agent Spawning
+```bash
+# Spawn team of agents
+cfn-docker-agent-spawn batch \
+  --team frontend \
+  --agents 3 \
+  --task-id "ui-development" \
+  --memory-limit 1g \
+  --network mcp-network
+```
+
+### Custom Configuration
+```bash
+# Spawn with custom configuration
+cfn-docker-agent-spawn \
+  --agent-type security-specialist \
+  --custom-config config/security-agent.json \
+  --environment "DEBUG=true,LOG_LEVEL=verbose" \
+  --volume /data/secrets:/app/secrets:ro
+```
+
+## Resource Management
+
+### Memory Limits
+| Agent Type | Default Limit | Maximum Recommended |
+|------------|---------------|---------------------|
+| **Frontend Engineer** | 1GB | 2GB |
+| **Backend Developer** | 768MB | 1.5GB |
+| **Security Specialist** | 1.5GB | 3GB |
+| **DevOps Engineer** | 1GB | 2GB |
+
+### CPU Limits
+- **Standard Agents**: 0.5-1.0 CPU units
+- **Resource-Intensive Agents**: 1.0-2.0 CPU units
+- **Batch Operations**: 0.3-0.5 CPU units per agent
+
+### Network Configuration
+```bash
+# Create isolated network for MCP communication
+docker network create mcp-network --driver bridge
+
+# Connect containers to MCP network
+docker network connect mcp-network agent-frontend-001
+docker network connect mcp-network playwright-mcp
+```
+
+## Integration with CFN Docker Skills
+
+### Skill-Based MCP Selection
+```bash
+# Get MCP configuration for agent
+MCP_CONFIG=$(cfn-docker-skill-mcp-selector select --agent-type ${AGENT_TYPE})
+
+# Spawn with MCP configuration
+cfn-docker-agent-spawn \
+  --agent-type ${AGENT_TYPE} \
+  --mcp-config "${MCP_CONFIG}" \
+  --auto-tokens
+```
+
+### Redis Coordination
+```bash
+# Register agent in Redis
+cfn-docker-redis-coordination register \
+  --agent-id ${AGENT_ID} \
+  --container-id ${CONTAINER_ID} \
+  --status "spawning"
+
+# Update agent status
+cfn-docker-redis-coordination update-status \
+  --agent-id ${AGENT_ID} \
+  --status "running"
+```
+
+### Loop Orchestration
+```bash
+# Spawn agents for Loop 3 implementation
+cfn-docker-loop-orchestration spawn-loop3 \
+  --task-context "${TASK_CONTEXT}" \
+  --agent-count 3 \
+  --parallel
+```
+
+## Monitoring and Observability
+
+### Resource Monitoring
+```bash
+# Real-time resource usage
+cfn-docker-agent-spawn stats \
+  --agent-id agent-frontend-001 \
+  --format json
+
+# Historical resource data
+cfn-docker-agent-spawn history \
+  --agent-id agent-frontend-001 \
+  --duration 1h
+```
+
+### Health Checks
+```bash
+# Container health status
+cfn-docker-agent-spawn health \
+  --agent-id agent-frontend-001
+
+# MCP server connectivity
+cfn-docker-agent-spawn check-mcp \
+  --agent-id agent-frontend-001 \
+  --servers playwright,redis
+```
+
+### Logging
+```bash
+# Container logs
+cfn-docker-agent-spawn logs \
+  --agent-id agent-frontend-001 \
+  --tail 100
+
+# Structured logs for monitoring
+cfn-docker-agent-spawn logs \
+  --agent-id agent-frontend-001 \
+  --format json \
+  --output /var/log/agents/${AGENT_ID}.log
+```
+
+## Error Handling
+
+### Container Failure Recovery
+```bash
+# Automatic restart on failure
+cfn-docker-agent-spawn \
+  --agent-type ${AGENT_TYPE} \
+  --restart-policy on-failure \
+  --restart-count 3
+
+# Manual recovery
+cfn-docker-agent-spawn recover \
+  --agent-id agent-frontend-001 \
+  --backup-state
+```
+
+### Resource Exhaustion Handling
+```bash
+# Memory pressure handling
+cfn-docker-agent-spawn \
+  --memory-limit 1g \
+  --memory-swap 1.5g \
+  --oom-kill-disable
+
+# CPU throttling
+cfn-docker-agent-spawn \
+  --cpu-limit 1.0 \
+  --cpu-shares 1024
+```
+
+### Network Isolation Issues
+```bash
+# Network connectivity validation
+cfn-docker-agent-spawn validate-network \
+  --agent-id agent-frontend-001 \
+  --targets redis:6379,playwright-mcp:3000
+
+# Network repair
+cfn-docker-agent-spawn repair-network \
+  --agent-id agent-frontend-001 \
+  --recreate-network
+```
+
+## Performance Optimization
+
+### Startup Optimization
+- **Pre-warmed Images**: Use Docker image layer caching
+- **Parallel Startup**: Spawn multiple containers concurrently
+- **Lazy Loading**: Load MCP servers on-demand
+- **Resource Pre-allocation**: Reserve resources in advance
+
+### Memory Optimization
+- **Selective MCP Loading**: Only load required MCP servers
+- **Workspace Cleanup**: Clean temporary files automatically
+- **Garbage Collection**: Optimize Node.js memory management
+- **Shared Libraries**: Use shared mounts for common dependencies
+
+### Network Optimization
+- **Local Network**: Use Docker bridge networks for MCP communication
+- **Connection Pooling**: Reuse MCP server connections
+- **DNS Caching**: Cache MCP server DNS resolution
+- **Compression**: Compress large data transfers
+
+## Security Considerations
+
+### Container Isolation
+- **Read-only Codebase**: Prevent code modification
+- **Limited Privileges**: Run as non-root user
+- **Resource Limits**: Prevent DoS attacks
+- **Network Segmentation**: Isolate agent networks
+
+### Token Security
+- **Secure Token Storage**: Use tmpfs for token files
+- **Token Expiration**: Automatic token rotation
+- **Access Logging**: Log all token usage
+- **Revocation**: Immediate token invalidation
+
+### File System Security
+- **Restricted Access**: Limit file system access
+- **Workspace Isolation**: Isolate agent workspaces
+- **Temporary Files**: Secure cleanup of temporary files
+- **Audit Logging**: Log all file system operations
+
+## Testing and Validation
+
+### Unit Tests
+- Container creation and configuration
+- Resource limit enforcement
+- Network connectivity validation
+- Token injection and validation
+
+### Integration Tests
+- End-to-end agent spawning workflow
+- MCP server connectivity and authentication
+- Resource monitoring and alerting
+- Error handling and recovery
+
+### Performance Tests
+- Container startup time measurement
+- Resource usage benchmarking
+- Concurrent spawning scalability
+- Network performance validation
+
+## Troubleshooting
+
+### Common Issues
+1. **Container Won't Start**: Check image availability and resource limits
+2. **MCP Connection Failed**: Verify network configuration and token validity
+3. **Memory Issues**: Monitor usage and adjust limits
+4. **Permission Errors**: Check volume mounts and user permissions
+
+### Debug Commands
+```bash
+# Debug container creation
+cfn-docker-agent-spawn debug \
+  --agent-type ${AGENT_TYPE} \
+  --verbose \
+  --dry-run
+
+# Inspect container configuration
+cfn-docker-agent-spawn inspect \
+  --container-id ${CONTAINER_ID} \
+  --format json
+
+# Validate MCP connectivity
+cfn-docker-agent-spawn test-mcp \
+  --container-id ${CONTAINER_ID} \
+  --all-servers
+```
+
+## Best Practices
+
+### Resource Planning
+- **Conservative Limits**: Start with lower memory limits and increase as needed
+- **Monitoring**: Implement comprehensive resource monitoring
+- **Capacity Planning**: Plan for peak usage scenarios
+- **Resource Cleanup**: Clean up unused containers and volumes
+
+### Security Hardening
+- **Minimal Images**: Use minimal Docker images
+- **Regular Updates**: Keep base images updated
+- **Scanning**: Regularly scan images for vulnerabilities
+- **Access Control**: Implement proper access controls
+
+### Operational Excellence
+- **Automation**: Automate container lifecycle management
+- **Observability**: Implement comprehensive monitoring
+- **Documentation**: Maintain detailed configuration documentation
+- **Backup**: Backup critical container configurations