claude-flow-novice 2.14.28 → 2.14.30

package/claude-assets/skills/cfn-environment-sanitization/SKILL.md

@@ -0,0 +1,200 @@
+# CFN Environment Sanitization Skill
+
+## Purpose
+
+Automatically sanitizes environment variables and enforces resource limits to prevent memory leaks in CFN Loop orchestration workflows. This is a critical component of the ANTI-023 memory leak protection system.
+
+## Core Functions
+
+### Environment Sanitization
+- Removes or redacts sensitive environment variables
+- Enforces memory limits for Node.js processes
+- Sets CFN-specific configuration limits
+- Preserves critical coordination variables
+
+### Resource Limit Enforcement
+- Node.js heap size: 2GB max
+- Maximum agents: 10 concurrent
+- Operation timeout: 600 seconds
+- Memory limits: 2GB per process
+
+### Sensitive Data Protection
+- Detects and redacts passwords, secrets, tokens
+- Validates environment for exposed credentials
+- Prevents sensitive data leakage in logs
+
+## Usage Patterns
+
+### Integration in Orchestration Scripts
+```bash
+#!/usr/bin/env bash
+
+# Load sanitization at script start
+source "$(dirname "$0")/../cfn-environment-sanitization/sanitize-environment.sh"
+
+# Rest of orchestration logic...
+```
+
+### Standalone Validation
+```bash
+# Check current environment state
+./cfn-environment-sanitization/sanitize-environment.sh check
+
+# Apply strict sanitization
+./cfn-environment-sanitization/sanitize-environment.sh --strict
+```
+
+### CFN Loop Integration Points
+- **Orchestration Scripts**: Apply before agent spawning
+- **Agent Spawning**: Enforce limits during process creation
+- **Validation Scripts**: Check environment before operations
+- **Cleanup Scripts**: Sanitize before process exit
+
+## Configuration
+
+### Environment Variables
+- `CFN_MAX_AGENTS`: Maximum concurrent agents (default: 10)
+- `CFN_TIMEOUT`: Operation timeout in seconds (default: 600)
+- `CFN_MEMORY_LIMIT`: Memory limit per process (default: 2GB)
+- `CFN_MODE`: Execution mode preservation
+- `NODE_OPTIONS`: Node.js runtime options with memory limits
+
+### Sanitization Rules
+The script uses a rule-based system for variable handling:
+- `sanitize`: Remove the variable
+- `sanitize_if_sensitive`: Remove if contains sensitive data
+- `preserve`: Keep the variable unchanged
+- `enforce_*`: Set to specific limit value
+
+## Safety Features
+
+### Strict Mode
+When enabled (`--strict` flag):
+- Validates required CLI mode variables
+- Enforces additional security checks
+- Prevents execution without proper configuration
+
+### Environment Validation
+- Detects sensitive data exposure
+- Validates memory limit configuration
+- Checks CFN coordination variables
+
+### Automatic Protection
+- Applies sanitization automatically when sourced
+- Preserves critical coordination variables
+- Enforces limits without manual configuration
+
+## Integration Requirements
+
+### Prerequisites
+- Bash 4.0+ for associative arrays
+- Standard Unix tools (grep, cut, sed)
+- Node.js environment for memory limits
+
+### Dependencies
+- CFN Mode Detection skill for coordination
+- CFN Redis Coordination for state management
+- CFN Agent Spawning for process creation
+
+## Usage Examples
+
+### Orchestration Script Integration
+```bash
+#!/usr/bin/env bash
+
+# Load environment sanitization
+source "$(dirname "$0")/../cfn-environment-sanitization/sanitize-environment.sh"
+
+# Now safe to proceed with orchestration
+echo "Environment sanitized, starting orchestration..."
+```
+
+### Agent Spawning with Limits
+```bash
+#!/usr/bin/env bash
+
+# Ensure sanitized environment
+source "./cfn-environment-sanitization/sanitize-environment.sh" --strict
+
+# Spawn agent with enforced limits
+npx claude-flow-novice agent "$AGENT_TYPE" \
+    --max-memory "$CFN_MEMORY_LIMIT" \
+    --timeout "$CFN_TIMEOUT"
+```
+
+### Environment Validation
+```bash
+# Pre-execution validation
+if ! ./cfn-environment-sanitization/sanitize-environment.sh check; then
+    echo "Environment validation failed" >&2
+    exit 1
+fi
+```
+
+## Testing
+
+### Unit Tests
+```bash
+# Test sanitization rules
+./tests/test-environment-sanitization.sh
+
+# Test sensitive data detection
+./tests/test-sensitive-data-redaction.sh
+
+# Test memory limit enforcement
+./tests/test-memory-limits.sh
+```
+
+### Integration Tests
+```bash
+# Test with orchestration script
+./tests/test-orchestration-integration.sh
+
+# Test with agent spawning
+./tests/test-agent-spawning-integration.sh
+```
+
+## Troubleshooting
+
+### Common Issues
+1. **Missing required variables**: Use strict mode to validate
+2. **Memory limits not applied**: Check Node.js options
+3. **Sensitive data leakage**: Run environment check
+4. **Agent spawning failures**: Verify resource limits
+
+### Debug Mode
+```bash
+# Enable debug output
+export CFN_DEBUG=1
+./sanitize-environment.sh --strict
+```
+
+## Monitoring and Telemetry
+
+### Metrics Collected
+- Environment sanitization changes
+- Resource limit violations
+- Sensitive data detections
+- Memory usage patterns
+
+### Log Integration
+- Structured logging with severity levels
+- Integration with CFN logging system
+- Audit trail for security compliance
+
+## Security Considerations
+
+### Data Protection
+- Automatic redaction of sensitive data
+- Prevention of credential exposure
+- Secure environment variable handling
+
+### Resource Protection
+- Memory limit enforcement prevents OOM
+- Timeout protection prevents hanging
+- Agent count limits prevent resource exhaustion
+
+### Compliance
+- Audit logging for security reviews
+- Environment state validation
+- Secure by default configuration

package/claude-assets/skills/cfn-environment-sanitization/sanitize-environment.sh

@@ -0,0 +1,244 @@
+#!/usr/bin/env bash
+
+##############################################################################
+# CFN Environment Sanitization
+# Part of ANTI-023 Memory Leak Protection System
+#
+# Automatically sanitizes environment variables and prevents memory leaks
+# in CFN Loop orchestration workflows.
+#
+# Usage:
+#   source ./sanitize-environment.sh [--strict]
+#   ./sanitize-environment.sh --check
+##############################################################################
+
+set -euo pipefail
+
+# Configuration
+STRICT_MODE=${1:-"false"}
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+
+# Environment sanitization rules
+declare -A SANITIZATION_RULES=(
+    # Clear potentially problematic variables
+    ["NODE_OPTIONS"]="sanitize"
+    ["UV_THREADPOOL_SIZE"]="sanitize"
+    ["REDIS_URL"]="sanitize_if_sensitive"
+
+    # Preserve critical CFN variables
+    ["CFN_MODE"]="preserve"
+    ["TASK_ID"]="preserve"
+    ["AGENT_ID"]="preserve"
+    ["LOOP3_AGENTS"]="preserve"
+    ["LOOP2_AGENTS"]="preserve"
+    ["PRODUCT_OWNER"]="preserve"
+
+    # Memory and process limits
+    ["NODE_HEAP_LIMIT"]="enforce_2gb"
+    ["MAX_AGENTS"]="enforce_10"
+    ["CFN_TIMEOUT"]="enforce_600"
+)
+
+# Sensitive patterns to redact
+SENSITIVE_PATTERNS=(
+    "password="
+    "secret="
+    "token="
+    "key="
+    "auth="
+    "credential="
+)
+
+# Color coding for output
+readonly RED='\033[0;31m'
+readonly GREEN='\033[0;32m'
+readonly YELLOW='\033[1;33m'
+readonly BLUE='\033[0;34m'
+readonly NC='\033[0m' # No Color
+
+# Logging functions
+log_info() {
+    echo -e "${BLUE}[SANITIZE]${NC} $1" >&2
+}
+
+log_success() {
+    echo -e "${GREEN}[SANITIZE]${NC} $1" >&2
+}
+
+log_warning() {
+    echo -e "${YELLOW}[SANITIZE]${NC} $1" >&2
+}
+
+log_error() {
+    echo -e "${RED}[SANITIZE]${NC} $1" >&2
+}
+
+# Check if value contains sensitive information
+is_sensitive() {
+    local value="$1"
+    for pattern in "${SENSITIVE_PATTERNS[@]}"; do
+        if [[ "$value" =~ $pattern ]]; then
+            return 0
+        fi
+    done
+    return 1
+}
+
+# Sanitize environment variable
+sanitize_var() {
+    local var_name="$1"
+    local var_value="${!var_name:-}"
+    local rule="${SANITIZATION_RULES[$var_name]:-"preserve"}"
+
+    case "$rule" in
+        "sanitize")
+            if [[ -n "$var_value" ]]; then
+                log_info "Sanitizing $var_name"
+                unset "$var_name"
+            fi
+            ;;
+        "sanitize_if_sensitive")
+            if is_sensitive "$var_value"; then
+                log_warning "Redacting sensitive $var_name"
+                unset "$var_name"
+            fi
+            ;;
+        "preserve")
+            # Keep the variable as-is
+            ;;
+        "enforce_2gb")
+            export "$var_name"="${var_value:-2048}"
+            log_info "Enforcing 2GB heap limit: $var_name=${!var_name}"
+            ;;
+        "enforce_10")
+            export "$var_name"="${var_value:-10}"
+            log_info "Enforcing max 10 agents: $var_name=${!var_value}"
+            ;;
+        "enforce_600")
+            export "$var_name"="${var_value:-600}"
+            log_info "Enforcing 600s timeout: $var_name=${!var_value}"
+            ;;
+    esac
+}
+
+# Apply environment sanitization
+sanitize_environment() {
+    log_info "Starting environment sanitization..."
+
+    # Count changes for reporting
+    local changes=0
+
+    for var_name in "${!SANITIZATION_RULES[@]}"; do
+        local old_value="${!var_name:-}"
+        sanitize_var "$var_name"
+        local new_value="${!var_name:-}"
+
+        if [[ "$old_value" != "$new_value" ]]; then
+            ((changes++))
+        fi
+    done
+
+    # Enforce memory limits for Node.js processes
+    export NODE_OPTIONS="--max-old-space-size=2048 --max-new-space-size=512 ${NODE_OPTIONS:-}"
+
+    # Set CFN-specific limits
+    export CFN_MAX_AGENTS="${CFN_MAX_AGENTS:-10}"
+    export CFN_TIMEOUT="${CFN_TIMEOUT:-600}"
+    export CFN_MEMORY_LIMIT="${CFN_MEMORY_LIMIT:-2GB}"
+
+    log_success "Environment sanitization complete ($changes changes applied)"
+
+    if [[ "$STRICT_MODE" == "true" ]]; then
+        log_info "Strict mode enabled - additional validations applied"
+
+        # Validate critical variables are set in CLI mode
+        if [[ -n "${TASK_ID:-}" ]]; then
+            for required_var in AGENT_ID LOOP3_AGENTS; do
+                if [[ -z "${!required_var:-}" ]]; then
+                    log_error "Required variable $required_var not set in CLI mode"
+                    return 1
+                fi
+            done
+        fi
+    fi
+
+    return 0
+}
+
+# Check current environment state
+check_environment() {
+    log_info "Checking environment state..."
+
+    local issues=0
+
+    # Check for sensitive data exposure
+    for var_name in $(env | grep -E "(password|secret|token|key|auth|credential)" | cut -d= -f1); do
+        log_warning "Potential sensitive data in $var_name"
+        ((issues++))
+    done
+
+    # Check Node.js memory settings
+    if [[ -n "${NODE_OPTIONS:-}" && ! "$NODE_OPTIONS" =~ "max-old-space-size" ]]; then
+        log_warning "NODE_OPTIONS missing heap limit"
+        ((issues++))
+    fi
+
+    # Check CFN configuration
+    if [[ -z "${CFN_MAX_AGENTS:-}" ]]; then
+        log_warning "CFN_MAX_AGENTS not set"
+        ((issues++))
+    fi
+
+    if [[ $issues -eq 0 ]]; then
+        log_success "Environment check passed"
+        return 0
+    else
+        log_error "Environment check failed ($issues issues found)"
+        return 1
+    fi
+}
+
+# Main execution
+main() {
+    local action="${1:-"sanitize"}"
+
+    case "$action" in
+        "sanitize")
+            sanitize_environment
+            ;;
+        "check")
+            check_environment
+            ;;
+        "--strict")
+            STRICT_MODE="true"
+            sanitize_environment
+            ;;
+        "--help"|"-h")
+            cat << EOF
+CFN Environment Sanitization Script
+
+Usage:
+  $0                    # Apply standard sanitization
+  $0 --strict          # Apply strict sanitization
+  $0 check             # Check environment state
+  $0 --help            # Show this help
+
+This script sanitizes the environment to prevent memory leaks and
+ensure secure CFN Loop execution.
+EOF
+            ;;
+        *)
+            log_error "Unknown action: $action"
+            return 1
+            ;;
+    esac
+}
+
+# Execute main function if run directly
+if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
+    main "$@"
+else
+    # When sourced, automatically apply sanitization
+    sanitize_environment
+fi

package/claude-assets/skills/cfn-hybrid-routing/README.md

@@ -20,7 +20,7 @@ Refer to `config.json` for detailed routing configuration parameters.
 
 ### Spawning Workers
 ```bash
-./spawn-worker.sh
+./scripts/spawn-worker.sh
 ```
 
 ### Dependency Check

package/claude-assets/skills/cfn-loop-orchestration/orchestrate.sh

@@ -21,6 +21,10 @@
 
 set -euo pipefail
 
+# Determine PROJECT_ROOT first before any other operations
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
+
 # ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Block Task Mode agents
 # Task Mode agents spawn via Task() tool and should NOT use orchestration scripts
 if [[ -z "${TASK_ID:-}" || -z "${LOOP3_AGENTS:-}" ]]; then
@@ -31,14 +35,39 @@ if [[ -z "${TASK_ID:-}" || -z "${LOOP3_AGENTS:-}" ]]; then
     exit 1
 fi
 
+# ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Environment Sanitization
+# Load and apply environment sanitization to prevent memory leaks
+if [[ -f "$PROJECT_ROOT/.claude/skills/cfn-task-mode-sanitize/task-mode-env-sanitizer.sh" ]]; then
+    source "$PROJECT_ROOT/.claude/skills/cfn-task-mode-sanitize/task-mode-env-sanitizer.sh"
+    sanitize_task_mode_environment "cli"
+    echo "✅ Environment sanitization applied" >&2
+else
+    echo "⚠️ Environment sanitization not available - proceeding without protection" >&2
+fi
+
+# ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Process Instrumentation
+# Load process instrumentation and monitoring for the orchestrator
+if [[ -f "$PROJECT_ROOT/.claude/skills/cfn-validation-runner-instrumentation/wrapped-executor.sh" ]]; then
+    source "$PROJECT_ROOT/.claude/skills/cfn-validation-runner-instrumentation/wrapped-executor.sh"
+    echo "✅ Orchestrator process instrumentation enabled" >&2
+else
+    echo "⚠️ Process instrumentation not available - proceeding without monitoring" >&2
+fi
+
+# ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Environment Configuration
+# Set stabilization environment variables with sensible defaults
+export CFN_VALIDATION_TIMEOUT="${CFN_VALIDATION_TIMEOUT:-300}"   # 5 minutes
+export CFN_MEMORY_LIMIT="${CFN_MEMORY_LIMIT:-2048}"              # 2GB memory limit
+export CFN_CPU_LIMIT="${CFN_CPU_LIMIT:-80}"                      # 80% CPU limit
+export CFN_TELEMETRY_DIR="${CFN_TELEMETRY_DIR:-$PROJECT_ROOT/.artifacts/telemetry}"
+mkdir -p "$CFN_TELEMETRY_DIR"
+
 # Load security utilities
 # shellcheck source=./security_utils.sh
 source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/security_utils.sh"
 
-# Determine script directory
-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 HELPERS_DIR="$SCRIPT_DIR/helpers"
-PROJECT_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)" && REDIS_COORD_SKILL="$PROJECT_ROOT/.claude/skills/cfn-redis-coordination"
+REDIS_COORD_SKILL="$PROJECT_ROOT/.claude/skills/cfn-redis-coordination"
 
 # Configuration
 TASK_ID=""
@@ -250,6 +279,16 @@ if [ -z "$TASK_ID" ] || [ -z "$LOOP3_AGENTS" ] || [ -z "$LOOP2_AGENTS" ] || [ -z
   exit 1
 fi
 
+# ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Process Instrumentation
+# Load process instrumentation and monitoring for the orchestrator
+# shellcheck source=../cfn-process-instrumentation/instrument-process.sh
+if [[ -f "$PROJECT_ROOT/.claude/skills/cfn-process-instrumentation/instrument-process.sh" ]]; then
+    source "$PROJECT_ROOT/.claude/skills/cfn-process-instrumentation/instrument-process.sh"
+    echo "✅ Orchestrator process instrumentation enabled" >&2
+else
+    echo "⚠️ Process instrumentation not available - proceeding without monitoring" >&2
+fi
+
 # Get thresholds for mode
 # Add additional mode validation with safe fallback
 case "$MODE" in
@@ -435,12 +474,22 @@ function spawn_loop3_agents() {
     safe_task_id=$(sanitize_input "$task_id") || continue
     safe_agent_id=$(sanitize_input "$UNIQUE_AGENT_ID") || continue
 
-    # Spawn agent in background with explicit agent ID
-    npx claude-flow-novice agent "$safe_agent_type" \
-      --task-id "$safe_task_id" \
-      --agent-id "$safe_agent_id" \
-      --iteration "$iteration" \
-      --context "$(build_agent_context "$safe_task_id" "$iteration" "$safe_agent_type" "" "loop3")" &
+    # Spawn agent in background with process instrumentation and memory limits
+    if command -v execute_instrumented >/dev/null 2>&1; then
+        execute_instrumented "npx" "$CFN_VALIDATION_TIMEOUT" "$CFN_MEMORY_LIMIT" \
+          claude-flow-novice agent "$safe_agent_type" \
+          --task-id "$safe_task_id" \
+          --agent-id "$safe_agent_id" \
+          --iteration "$iteration" \
+          --context "$(build_agent_context "$safe_task_id" "$iteration" "$safe_agent_type" "" "loop3")" &
+    else
+        # Fallback to raw spawn if instrumentation unavailable
+        npx claude-flow-novice agent "$safe_agent_type" \
+          --task-id "$safe_task_id" \
+          --agent-id "$safe_agent_id" \
+          --iteration "$iteration" \
+          --context "$(build_agent_context "$safe_task_id" "$iteration" "$safe_agent_type" "" "loop3")" &
+    fi
 
     # Store PID for monitoring using unique agent ID
     AGENT_PID=$!
@@ -450,6 +499,17 @@ function spawn_loop3_agents() {
       --value "{\"pid\": $AGENT_PID}" \
       --namespace "swarm" >/dev/null
 
+    # ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Start telemetry monitoring
+    if [[ -f "$PROJECT_ROOT/.claude/skills/cfn-telemetry/collect-metrics.sh" ]]; then
+        MONITOR_PID=$("$PROJECT_ROOT/.claude/skills/cfn-telemetry/collect-metrics.sh" start-monitoring "$UNIQUE_AGENT_ID" "$AGENT_PID" "$iteration" "$safe_agent_type")
+        "$REDIS_COORD_SKILL/store-context.sh" \
+          --task-id "$task_id" \
+          --key "${UNIQUE_AGENT_ID}:monitor_pid" \
+          --value "{\"pid\": $MONITOR_PID}" \
+          --namespace "swarm" >/dev/null
+        echo "🔍 Started monitoring for $UNIQUE_AGENT_ID (Agent PID: $AGENT_PID, Monitor PID: $MONITOR_PID)" >&2
+    fi
+
     # Store agent ID mapping for later retrieval using Redis SADD for set storage
     redis-cli SADD "swarm:${task_id}:loop3:agent_ids:iteration${iteration}" "$UNIQUE_AGENT_ID" >/dev/null
   done
@@ -524,6 +584,16 @@ function wait_for_agents() {
     wait "$pid" 2>/dev/null || true
   done
 
+  # ⚠️ ANTI-023 MEMORY LEAK PROTECTION: Stop monitoring for all agents
+  echo "  Stopping telemetry monitoring for Loop 3 agents..." >&2
+  for unique_agent_id in "${AGENT_IDS[@]}"; do
+    local monitor_pid=$("$REDIS_COORD_SKILL/get-context.sh" --task-id "$task_id" --key "${unique_agent_id}:monitor_pid" --namespace "swarm" 2>/dev/null | jq -r '.pid // 0' || echo "0")
+    if [[ "$monitor_pid" -gt 0 ]] && kill -0 "$monitor_pid" 2>/dev/null; then
+      "$PROJECT_ROOT/.claude/skills/cfn-telemetry/collect-metrics.sh" stop-monitoring "$monitor_pid" >/dev/null 2>&1 || true
+      echo "    Stopped monitoring for $unique_agent_id (Monitor PID: $monitor_pid)" >&2
+    fi
+  done
+
   # Calculate actual elapsed time
   local end_time=$(date +%s)
   local elapsed=$((end_time - start_time))
@@ -668,12 +738,22 @@ function spawn_loop2_agents() {
 
     echo "  Spawning: $agent_type (ID: $UNIQUE_VALIDATOR_ID)"
 
-    # Spawn agent in background with explicit agent ID
-    npx claude-flow-novice agent "$agent_type" \
-      --task-id "$task_id" \
-      --agent-id "$UNIQUE_VALIDATOR_ID" \
-      --iteration "$iteration" \
-      --context "$(build_agent_context "$task_id" "$iteration" "$agent_type" "" "loop2")" &
+    # Spawn validator in background with process instrumentation and memory limits
+    if command -v execute_instrumented >/dev/null 2>&1; then
+        execute_instrumented "npx" "$CFN_VALIDATION_TIMEOUT" "$CFN_MEMORY_LIMIT" \
+          claude-flow-novice agent "$agent_type" \
+          --task-id "$task_id" \
+          --agent-id "$UNIQUE_VALIDATOR_ID" \
+          --iteration "$iteration" \
+          --context "$(build_agent_context "$task_id" "$iteration" "$agent_type" "" "loop2")" &
+    else
+        # Fallback to raw spawn if instrumentation unavailable
+        npx claude-flow-novice agent "$agent_type" \
+          --task-id "$task_id" \
+          --agent-id "$UNIQUE_VALIDATOR_ID" \
+          --iteration "$iteration" \
+          --context "$(build_agent_context "$task_id" "$iteration" "$agent_type" "" "loop2")" &
+    fi
 
     # Store PID for monitoring using unique agent ID
     AGENT_PID=$!