claude-flow-novice 2.15.2 → 2.15.4

package/.claude/skills/cfn-redis-coordination/CENTRALIZED_REDIS_WRAPPER.md

@@ -0,0 +1,319 @@
+# Centralized Redis Wrapper - Graceful Fallback for Task Mode
+
+**Version:** 1.0
+**Date:** 2025-11-16
+**Status:** Production Ready
+
+---
+
+## Overview
+
+This document describes the centralized Redis wrapper implementation that provides graceful fallback for Task mode execution, solving the ANTI-023 memory leak issue without requiring mode detection.
+
+## Problem Statement
+
+**Original Issue:**
+- CLI mode agents (spawned via `npx claude-flow-novice`) need Redis for coordination
+- Task mode agents (spawned via `Task()` tool) should NOT use Redis (causes memory leaks)
+- Mode detection was unreliable (both modes could have TASK_ID/AGENT_ID)
+
+**Solution:**
+Instead of trying to detect mode, **make Redis commands fail gracefully** when Redis is unavailable.
+
+## Architecture
+
+### Components
+
+**1. redis-cli-wrapper.sh** (Centralized wrapper)
+- Checks if Redis is available (1 second timeout)
+- If available → execute command normally (CLI/Docker mode)
+- If unavailable → soft fail with informative message (Task mode)
+
+**2. redis-functions.sh** (Function override)
+- Sources the wrapper
+- Exports `redis-cli()` function that overrides the system command
+- All scripts using redis-cli automatically get graceful fallback
+
+**3. Coordination scripts** (Updated to source redis-functions.sh)
+- `report-completion.sh` - Main agent completion protocol
+- Other scripts can be updated as needed
+
+### How It Works
+
+```bash
+# Script sources redis-functions.sh
+source ./.claude/skills/cfn-redis-coordination/redis-functions.sh
+
+# All redis-cli calls now use the wrapper
+redis-cli LPUSH "key" "value"  # Fails gracefully if Redis unavailable
+```
+
+**Execution Flow:**
+
+```text
+1. Script calls redis-cli LPUSH "key" "value"
+2. redis-functions.sh overrides redis-cli → calls wrapper
+3. Wrapper checks: timeout 1 redis-cli ping
+   ├─ Success? → Execute command (CLI/Docker mode)
+   └─ Timeout? → Print warning, exit 0 (Task mode)
+4. Script continues (no fatal error)
+```
+
+## Benefits
+
+✅ **No mode detection needed** - Just check if Redis responds
+✅ **Simpler** - One code path, graceful degradation
+✅ **Self-healing** - Works in both Task mode (no Redis) AND CLI mode (has Redis)
+✅ **No memory leak** - If Redis isn't available, commands fail fast (exit 0)
+✅ **Clear feedback** - Informative messages explain what's happening
+✅ **Centralized** - One place to change behavior (redis-cli-wrapper.sh)
+✅ **Backward compatible** - Scripts without redis-functions.sh still work (use system redis-cli)
+
+## Usage
+
+### For New Scripts
+
+Add this at the top of any script that uses Redis:
+
+```bash
+#!/bin/bash
+set -euo pipefail
+
+# Source centralized Redis functions (provides graceful fallback for Task mode)
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/redis-functions.sh"
+
+# Now use redis-cli normally
+redis-cli LPUSH "swarm:task-123:done" "complete"
+redis-cli SET "key" "value" EX 3600
+```
+
+### For Existing Scripts
+
+1. Add the source block after `set -euo pipefail`
+2. Remove manual ANTI-023 checks (wrapper handles it)
+3. Simplify redis-cli calls (no need for `-h` / `-p` flags)
+
+**Before:**
+```bash
+if [[ -z "${TASK_ID:-}" || -z "${AGENT_ID:-}" ]]; then
+    echo "❌ TASK MODE DETECTED - Redis coordination forbidden" >&2
+    exit 1
+fi
+
+redis-cli -h "${REDIS_HOST:-localhost}" -p "${REDIS_PORT:-6379}" LPUSH "key" "value"
+```
+
+**After:**
+```bash
+source "${SCRIPT_DIR}/redis-functions.sh"
+
+redis-cli LPUSH "key" "value"
+```
+
+### Testing Graceful Fallback
+
+Test that your script handles Redis unavailability:
+
+```bash
+# Simulate Task mode (Redis unavailable)
+REDIS_HOST="nonexistent" bash your-script.sh --args
+
+# Expected output:
+⚠️ Redis unavailable - command skipped (soft fail)
+💡 This is expected in Task mode (Main Chat coordination)
+🔧 Agents should output JSON directly instead of Redis coordination
+✅ Script completed successfully
+```
+
+## Implementation Details
+
+### redis-cli-wrapper.sh
+
+```bash
+#!/bin/bash
+set -euo pipefail
+
+REDIS_HOST="${REDIS_HOST:-localhost}"
+REDIS_PORT="${REDIS_PORT:-6379}"
+
+# Graceful Redis availability check (1 second timeout)
+if ! timeout 1 redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" ping &>/dev/null; then
+    # Redis unavailable - likely Task mode
+    echo "⚠️ Redis unavailable - command skipped (soft fail)" >&2
+    echo "💡 This is expected in Task mode (Main Chat coordination)" >&2
+    echo "🔧 Agents should output JSON directly instead of Redis coordination" >&2
+    exit 0  # Soft fail - don't break agent execution
+fi
+
+# Redis available - execute command normally (CLI/Docker mode)
+exec redis-cli -h "$REDIS_HOST" -p "$REDIS_PORT" "$@"
+```
+
+**Key Features:**
+- 1 second timeout (prevents hanging)
+- Informative error messages
+- Exit 0 (soft fail, agent continues)
+- Respects REDIS_HOST/REDIS_PORT environment variables
+
+### redis-functions.sh
+
+```bash
+#!/bin/bash
+
+# Get the directory where this script is located
+REDIS_FUNCTIONS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Override redis-cli to always use the wrapper
+redis-cli() {
+    "${REDIS_FUNCTIONS_DIR}/redis-cli-wrapper.sh" "$@"
+}
+
+# Export the function so it's available in subshells
+export -f redis-cli
+
+# Helper: Check if Redis is available (useful for conditional logic)
+is_redis_available() {
+    timeout 1 "${REDIS_FUNCTIONS_DIR}/redis-cli-wrapper.sh" ping &>/dev/null
+    return $?
+}
+
+export -f is_redis_available
+```
+
+**Key Features:**
+- Function override (all redis-cli calls use wrapper)
+- Exported for subshells
+- Helper function `is_redis_available()` for conditional logic
+
+## Migration Guide
+
+### Step 1: Update Core Scripts
+
+Update the most critical coordination scripts first:
+
+1. `report-completion.sh` ✅ (Already updated)
+2. `collect-confidence-scores.sh`
+3. `invoke-waiting-mode.sh`
+4. `store-context.sh`
+5. `get-context.sh`
+
+### Step 2: Test Each Script
+
+For each updated script:
+
+```bash
+# Test with Redis available (CLI mode simulation)
+bash script.sh --args
+
+# Test with Redis unavailable (Task mode simulation)
+REDIS_HOST="nonexistent" bash script.sh --args
+```
+
+### Step 3: Update Agent Profiles (Optional)
+
+Agent profiles can now simplify their Redis usage examples:
+
+**Before:**
+```markdown
+redis-cli -h "${REDIS_HOST:-localhost}" -p "${REDIS_PORT:-6379}" LPUSH "key" "value"
+```
+
+**After:**
+```markdown
+# Redis coordination (automatically fails gracefully in Task mode)
+redis-cli LPUSH "key" "value"
+```
+
+## Troubleshooting
+
+### Issue: "Permission denied" when running wrapper
+
+**Solution:**
+```bash
+chmod +x .claude/skills/cfn-redis-coordination/redis-cli-wrapper.sh
+chmod +x .claude/skills/cfn-redis-coordination/redis-functions.sh
+```
+
+### Issue: "$'\r': command not found" (CRLF line endings)
+
+**Solution:**
+```bash
+sed -i 's/\r$//' .claude/skills/cfn-redis-coordination/redis-cli-wrapper.sh
+sed -i 's/\r$//' .claude/skills/cfn-redis-coordination/redis-functions.sh
+```
+
+### Issue: Redis commands still hanging
+
+**Cause:** Script not sourcing redis-functions.sh
+
+**Solution:** Add source line at top of script:
+```bash
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/redis-functions.sh"
+```
+
+## Performance Impact
+
+**Redis Available (CLI/Docker mode):**
+- Ping check: ~1-5ms overhead per script execution
+- Command execution: No overhead (exec replaces process)
+
+**Redis Unavailable (Task mode):**
+- Ping timeout: 1 second (one-time per script, not per command)
+- Total overhead: ~1 second per script execution
+- Acceptable for Task mode (not performance-critical)
+
+## Security Considerations
+
+**Timeout Protection:**
+- 1 second timeout prevents indefinite hanging
+- Protects against network issues
+
+**Environment Variables:**
+- Respects REDIS_HOST/REDIS_PORT for configuration
+- No hardcoded credentials
+
+**Soft Failure:**
+- Exit 0 prevents cascade failures
+- Agent can continue execution
+
+## Success Metrics
+
+**Test Results:**
+- ✅ report-completion.sh tested with nonexistent Redis host
+- ✅ All 6 redis-cli calls failed gracefully (exit 0)
+- ✅ Script completed successfully with informative warnings
+- ✅ No memory leaks (commands timeout after 1 second)
+
+**Expected Behavior:**
+- CLI mode: Redis available → normal execution
+- Docker mode: Redis available → normal execution
+- Task mode: Redis unavailable → graceful fallback, agent continues
+
+## Conclusion
+
+The centralized Redis wrapper provides a robust, simple solution to the ANTI-023 memory leak issue. By focusing on **graceful failure** instead of **mode detection**, we achieve:
+
+1. **Reliability** - Works regardless of mode detection accuracy
+2. **Simplicity** - One code path, clear behavior
+3. **Maintainability** - Centralized in one wrapper file
+4. **Safety** - No memory leaks, no hanging processes
+
+This approach is **production-ready** and can be rolled out incrementally (script by script) without disrupting existing workflows.
+
+---
+
+**See Also:**
+- `docs/architecture/task-mode-redis-safety-patterns.md` - Original ANTI-023 analysis
+- `CLAUDE.md` - CFN Loop execution modes
+- `.claude/skills/cfn-redis-coordination/SKILL.md` - Coordination protocols
+
+**Next Steps:**
+1. Update remaining coordination scripts
+2. Test in production CFN Loop execution
+3. Document in agent profiles
+4. Monitor for any edge cases
+
+**Version History:**
+- v1.0 (2025-11-16): Initial implementation with graceful fallback

package/.claude/skills/cfn-redis-coordination/agent-log.sh

@@ -12,6 +12,10 @@
 
 set -euo pipefail
 
+# Source centralized Redis functions (provides graceful fallback for Task mode)
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/redis-functions.sh"
+
 # Color codes for terminal output
 COLOR_RESET="\033[0m"
 COLOR_DEBUG="\033[0;36m"   # Cyan

package/.claude/skills/cfn-redis-coordination/agent-log.sh.bak

@@ -0,0 +1,124 @@
+#!/usr/bin/env bash
+
+# Agent Logging Utility - Dual output (terminal + Redis)
+#
+# Usage:
+#   source agent-log.sh
+#   agent_log "info" "Message here"
+#   agent_log "error" "Error occurred"
+#
+# Or as standalone:
+#   ./agent-log.sh "info" "Agent started" --agent-id "researcher-1" --task-id "task-123"
+
+set -euo pipefail
+
+# Color codes for terminal output
+COLOR_RESET="\033[0m"
+COLOR_DEBUG="\033[0;36m"   # Cyan
+COLOR_INFO="\033[0;32m"    # Green
+COLOR_WARN="\033[0;33m"    # Yellow
+COLOR_ERROR="\033[0;31m"   # Red
+COLOR_AGENT="\033[0;35m"   # Magenta
+
+# Get color for log level
+get_log_color() {
+  local level="$1"
+  case "$level" in
+    debug) echo "$COLOR_DEBUG" ;;
+    info)  echo "$COLOR_INFO" ;;
+    warn)  echo "$COLOR_WARN" ;;
+    error) echo "$COLOR_ERROR" ;;
+    *) echo "$COLOR_RESET" ;;
+  esac
+}
+
+# Main logging function
+# Usage: agent_log <level> <message> [--agent-id <id>] [--task-id <id>] [--no-terminal] [--no-redis]
+agent_log() {
+  local LEVEL="${1:-info}"
+  local MESSAGE="${2:-}"
+  local AGENT_ID="${AGENT_ID:-unknown}"
+  local TASK_ID="${TASK_ID:-unknown}"
+  local REPOSITORY=$(basename "$(pwd)")
+  local NO_TERMINAL=false
+  local NO_REDIS=false
+
+  # Parse optional arguments
+  shift 2 || true
+  while [[ $# -gt 0 ]]; do
+    case "$1" in
+      --agent-id)
+        AGENT_ID="$2"
+        shift 2
+        ;;
+      --task-id)
+        TASK_ID="$2"
+        shift 2
+        ;;
+      --no-terminal)
+        NO_TERMINAL=true
+        shift
+        ;;
+      --no-redis)
+        NO_REDIS=true
+        shift
+        ;;
+      *)
+        shift
+        ;;
+    esac
+  done
+
+  local TIMESTAMP=$(date -u +"%Y-%m-%dT%H:%M:%SZ")
+  local COLOR=$(get_log_color "$LEVEL")
+
+  # 1. Terminal output (with colors)
+  if [ "$NO_TERMINAL" = false ]; then
+    echo -e "${COLOR}[${LEVEL^^}]${COLOR_RESET} ${COLOR_AGENT}[${AGENT_ID}]${COLOR_RESET} ${MESSAGE}"
+  fi
+
+  # 2. File output (no colors, in /tmp)
+  local LOG_FILE="/tmp/agent-${AGENT_ID}.log"
+  echo "[${TIMESTAMP}] [${LEVEL^^}] [${AGENT_ID}] ${MESSAGE}" >> "$LOG_FILE"
+
+  # 3. Redis pub/sub (for portal)
+  if [ "$NO_REDIS" = false ]; then
+    local LOG_PAYLOAD=$(cat <<EOF
+{
+  "level": "${LEVEL}",
+  "message": "${MESSAGE}",
+  "agentId": "${AGENT_ID}",
+  "taskId": "${TASK_ID}",
+  "repository": "${REPOSITORY}",
+  "timestamp": "${TIMESTAMP}"
+}
+EOF
+)
+
+    # Publish to Redis channel: swarm:<task-id>:logs
+    redis-cli publish "swarm:${TASK_ID}:logs" "$LOG_PAYLOAD" > /dev/null 2>&1 || true
+
+    # Also add to sorted set for persistence (with timestamp as score)
+    local SCORE=$(date +%s)
+    redis-cli zadd "swarm:${TASK_ID}:logs:history" "$SCORE" "$LOG_PAYLOAD" > /dev/null 2>&1 || true
+
+    # Set TTL on history (7 days)
+    redis-cli expire "swarm:${TASK_ID}:logs:history" 604800 > /dev/null 2>&1 || true
+  fi
+}
+
+# If script is executed directly (not sourced)
+if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
+  if [ $# -lt 2 ]; then
+    echo "Usage: $0 <level> <message> [--agent-id <id>] [--task-id <id>]"
+    echo ""
+    echo "Levels: debug, info, warn, error"
+    echo ""
+    echo "Examples:"
+    echo "  $0 info \"Agent started\" --agent-id researcher-1 --task-id task-123"
+    echo "  $0 error \"Connection failed\" --agent-id reviewer-2 --task-id task-456"
+    exit 1
+  fi
+
+  agent_log "$@"
+fi

package/.claude/skills/cfn-redis-coordination/agent-recovery.sh

@@ -1,75 +1,75 @@
-#!/bin/bash
-# Redis Coordination Skill - Agent Emergency Recovery
-# Version: 1.0.0
-# Last Updated: 2025-10-19
-
-# Strict error handling
-set -euo pipefail
-
-# Default values
-TASK_ID=""
-AGENT_ID=""
-RECOVERY_MODE="soft"  # Options: soft, hard
-
-# Parse command-line arguments
-while [[ $# -gt 0 ]]; do
-    case "$1" in
-        --task-id)
-            TASK_ID="$2"
-            shift 2
-            ;;
-        --agent-id)
-            AGENT_ID="$2"
-            shift 2
-            ;;
-        --mode)
-            RECOVERY_MODE="$2"
-            shift 2
-            ;;
-        *)
-            echo "Unknown parameter: $1"
-            exit 1
-            ;;
-    esac
-done
-
-# Validate required parameters
-if [[ -z "$TASK_ID" || -z "$AGENT_ID" ]]; then
-    echo "Error: task-id and agent-id are required"
-    exit 1
-fi
-
-# Log recovery attempt
-echo "[$(date -u)] Attempting ${RECOVERY_MODE} recovery for agent: ${AGENT_ID}" >> /var/log/claude-flow/agent-recovery.log
-
-# Retrieve agent context before recovery
-AGENT_CONTEXT=$(redis-cli get "swarm:${TASK_ID}:${AGENT_ID}:context")
-
-# Recovery actions based on mode
-case "$RECOVERY_MODE" in
-    "soft")
-        # Soft recovery: Spawn new agent with previous context
-        npx claude-flow@alpha spawn "$AGENT_ID" \
-            --task-id "$TASK_ID" \
-            --recovery-context "$AGENT_CONTEXT"
-        ;;
-    "hard")
-        # Hard recovery: Full agent replacement
-        npx claude-flow@alpha spawn "$AGENT_ID" \
-            --task-id "$TASK_ID" \
-            --mode replace \
-            --recovery-context "$AGENT_CONTEXT"
-        ;;
-    *)
-        echo "Invalid recovery mode: ${RECOVERY_MODE}"
-        exit 1
-        ;;
-esac
-
-# Update active agents set
-redis-cli sadd "swarm:${TASK_ID}:active-agents" "$AGENT_ID"
-
-# Log recovery completion
-echo "[$(date -u)] Recovery completed for agent: ${AGENT_ID}" >> /var/log/claude-flow/agent-recovery.log
-
+#!/bin/bash
+# Redis Coordination Skill - Agent Emergency Recovery
+# Version: 1.0.0
+# Last Updated: 2025-10-19
+
+# Strict error handling
+set -euo pipefail
+
+# Default values
+TASK_ID=""
+AGENT_ID=""
+RECOVERY_MODE="soft"  # Options: soft, hard
+
+# Parse command-line arguments
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        --task-id)
+            TASK_ID="$2"
+            shift 2
+            ;;
+        --agent-id)
+            AGENT_ID="$2"
+            shift 2
+            ;;
+        --mode)
+            RECOVERY_MODE="$2"
+            shift 2
+            ;;
+        *)
+            echo "Unknown parameter: $1"
+            exit 1
+            ;;
+    esac
+done
+
+# Validate required parameters
+if [[ -z "$TASK_ID" || -z "$AGENT_ID" ]]; then
+    echo "Error: task-id and agent-id are required"
+    exit 1
+fi
+
+# Log recovery attempt
+echo "[$(date -u)] Attempting ${RECOVERY_MODE} recovery for agent: ${AGENT_ID}" >> /var/log/claude-flow/agent-recovery.log
+
+# Retrieve agent context before recovery
+AGENT_CONTEXT=$(redis-cli get "swarm:${TASK_ID}:${AGENT_ID}:context")
+
+# Recovery actions based on mode
+case "$RECOVERY_MODE" in
+    "soft")
+        # Soft recovery: Spawn new agent with previous context
+        cfn-spawn agent "$AGENT_ID" \
+            --task-id "$TASK_ID" \
+            --recovery-context "$AGENT_CONTEXT"
+        ;;
+    "hard")
+        # Hard recovery: Full agent replacement
+        cfn-spawn agent "$AGENT_ID" \
+            --task-id "$TASK_ID" \
+            --mode replace \
+            --recovery-context "$AGENT_CONTEXT"
+        ;;
+    *)
+        echo "Invalid recovery mode: ${RECOVERY_MODE}"
+        exit 1
+        ;;
+esac
+
+# Update active agents set
+redis-cli sadd "swarm:${TASK_ID}:active-agents" "$AGENT_ID"
+
+# Log recovery completion
+echo "[$(date -u)] Recovery completed for agent: ${AGENT_ID}" >> /var/log/claude-flow/agent-recovery.log
+
 exit 0

package/.claude/skills/cfn-redis-coordination/collect-confidence-scores.sh

@@ -18,6 +18,36 @@
 
 set -euo pipefail
 
+# Validate required tools are available
+MISSING_TOOLS=()
+for TOOL in redis-cli jq bc; do
+    if ! command -v "$TOOL" >/dev/null 2>&1; then
+        MISSING_TOOLS+=("$TOOL")
+    fi
+done
+
+if [ ${#MISSING_TOOLS[@]} -gt 0 ]; then
+    echo "❌ ERROR: Required tools not found: ${MISSING_TOOLS[*]}" >&2
+    echo "   Install missing tools and retry" >&2
+    exit 1
+fi
+
+# Source centralized Redis functions (provides graceful fallback for Task mode)
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Verify redis-functions.sh exists and is readable before sourcing
+REDIS_FUNCTIONS="${SCRIPT_DIR}/redis-functions.sh"
+if [ ! -r "$REDIS_FUNCTIONS" ]; then
+    echo "❌ ERROR: Cannot read redis-functions.sh at: $REDIS_FUNCTIONS" >&2
+    exit 1
+fi
+
+# Source with explicit error handling
+if ! source "$REDIS_FUNCTIONS"; then
+    echo "❌ ERROR: Failed to source redis-functions.sh" >&2
+    exit 1
+fi
+
 # Debug mode (set DEBUG=true for verbose output)
 DEBUG="${DEBUG:-false}"