claude-flow-novice 2.8.1 → 2.9.0

package/.claude/skills/redis-coordination/SKILL.md

@@ -1,720 +0,0 @@
-# Redis Coordination Skill
-
-**Version:** 3.0.0 (Pure Coordination Primitives)
-**Last Updated:** 2025-10-23
-
-**⚠️ DEPRECATION NOTICE:**
-The monolithic `orchestrate-cfn-loop.sh` (70KB) has been replaced by the modular CFN v3 orchestrator at `.claude/skills/cfn-loop-orchestration/orchestrate.sh` (26KB). See CFN v3 documentation for details.
-
----
-
-## Overview
-
-The Redis Coordination Skill provides low-level coordination primitives for distributed agent orchestration. This skill is framework-agnostic and supports any coordination pattern (swarm, mesh, hierarchical).
-
-**Key Capabilities:**
-- Generic JSON context storage and retrieval
-- Pub/sub signaling (LPUSH/BLPOP)
-- Agent result collection and aggregation
-- Zero-token waiting mechanisms
-- Consensus calculation
-
-**Design Philosophy:**
-- Pure primitives, no workflow-specific logic
-- Minimal assumptions about coordination patterns
-- Clean, composable interfaces
-- Framework-agnostic implementations
-
----
-
-## Core Primitives
-
-### 1. Context Storage (`store-context.sh`)
-
-Store arbitrary JSON context in Redis with automatic TTL management.
-
-**Usage:**
-```bash
-./store-context.sh \
-  --task-id "unique-task-123" \
-  --key "epic-context" \
-  --value '{"goal": "Build feature X", "scope": ["A", "B"]}' \
-  --ttl 86400
-```
-
-**Parameters:**
-- `--task-id` (required): Unique task identifier
-- `--key` (required): Context key name
-- `--value` (required): Valid JSON string
-- `--ttl` (optional): Time-to-live in seconds (default: 86400 = 24h)
-- `--namespace` (optional): Redis key namespace (default: "context")
-
-**Returns:**
-- Redis key on success: `context:unique-task-123:epic-context`
-- Exit code 0 on success, 1 on error
-
-**Redis Key Format:**
-```
-{namespace}:{task_id}:{key}
-  └─ Hash fields:
-     - value: JSON payload
-     - metadata: {stored_at, ttl_seconds}
-```
-
-**Example:**
-```bash
-# Store epic context
-REDIS_KEY=$(./store-context.sh \
-  --task-id "task-001" \
-  --key "epic" \
-  --value '{"goal": "Authentication system"}')
-
-echo "Stored at: $REDIS_KEY"
-# Output: Stored at: context:task-001:epic
-```
-
----
-
-### 2. Context Retrieval (`retrieve-context.sh`)
-
-Retrieve JSON context from Redis with optional metadata.
-
-**Usage:**
-```bash
-./retrieve-context.sh \
-  --task-id "unique-task-123" \
-  --key "epic-context"
-```
-
-**Parameters:**
-- `--task-id` (required): Unique task identifier
-- `--key` (required): Context key name
-- `--namespace` (optional): Redis key namespace (default: "context")
-- `--with-metadata` (optional): Include storage metadata in output
-
-**Returns:**
-- JSON value on success
-- Exit code 0 on success, 1 if key not found or invalid
-
-**Example:**
-```bash
-# Retrieve context
-CONTEXT=$(./retrieve-context.sh \
-  --task-id "task-001" \
-  --key "epic")
-
-echo "$CONTEXT" | jq .
-# Output: {"goal": "Authentication system"}
-
-# With metadata
-./retrieve-context.sh \
-  --task-id "task-001" \
-  --key "epic" \
-  --with-metadata | jq .
-# Output:
-# {
-#   "value": {"goal": "Authentication system"},
-#   "metadata": {"stored_at": "2025-10-23T10:30:00Z", "ttl_seconds": 86400},
-#   "ttl_remaining": 85000
-# }
-```
-
----
-
-### 3. Signaling (`signal.sh`)
-
-Generic pub/sub signaling for agent coordination using LPUSH/BLPOP.
-
-**Commands:**
-- `send`: Send signal to a queue (LPUSH)
-- `wait`: Wait for signal with timeout (BLPOP)
-- `broadcast`: Send signal to multiple agents
-
-**Usage - Send:**
-```bash
-./signal.sh send \
-  --task-id "task-001" \
-  --signal "gate-passed" \
-  --payload '{"threshold": 0.75, "passed": true}'
-```
-
-**Usage - Wait:**
-```bash
-# Wait indefinitely (timeout=0)
-PAYLOAD=$(./signal.sh wait \
-  --task-id "task-001" \
-  --signal "gate-passed")
-
-# Wait with timeout (120 seconds)
-PAYLOAD=$(./signal.sh wait \
-  --task-id "task-001" \
-  --signal "gate-passed" \
-  --timeout 120)
-```
-
-**Usage - Broadcast:**
-```bash
-./signal.sh broadcast \
-  --task-id "task-001" \
-  --signal "iteration-start" \
-  --agents "agent-1,agent-2,agent-3" \
-  --payload '{"iteration": 2, "reason": "improve_quality"}'
-```
-
-**Parameters:**
-- `--task-id` (required): Unique task identifier
-- `--signal` (required): Signal name
-- `--payload` (optional): JSON payload (default: `{}`)
-- `--timeout` (optional): Wait timeout in seconds (default: 0 = infinite)
-- `--agents` (required for broadcast): Comma-separated agent IDs
-- `--namespace` (optional): Redis key namespace (default: "signal")
-
-**Redis Key Formats:**
-```
-# Send/Wait:
-signal:{task_id}:{signal_name}
-
-# Broadcast:
-signal:{task_id}:{agent_id}:{signal_name}
-```
-
-**Example - Simple Coordination:**
-```bash
-# Agent 1: Wait for signal
-(
-  ./signal.sh wait --task-id "task-001" --signal "start-work"
-  echo "Received start signal, beginning work..."
-) &
-
-# Agent 2: Send signal after preparation
-sleep 2
-./signal.sh send --task-id "task-001" --signal "start-work" --payload '{}'
-```
-
----
-
-### 4. Result Collection (`collect-results.sh`)
-
-Collect and aggregate agent results with confidence scoring and consensus calculation.
-
-**Usage:**
-```bash
-./collect-results.sh \
-  --task-id "task-001" \
-  --agent-ids "coder-1,reviewer-1,tester-1"
-```
-
-**Parameters:**
-- `--task-id` (required): Unique task identifier
-- `--agent-ids` (required): Comma-separated agent IDs
-- `--namespace` (optional): Redis key namespace (default: "result")
-- `--calculate-consensus` (optional): Calculate average confidence score
-- `--min-confidence` (optional): Minimum confidence threshold
-- `--include-metadata` (optional): Include agent metadata in output
-- `--timeout` (optional): Wait timeout per agent in seconds (default: 0)
-
-**Expected Redis Structure (Per Agent):**
-```
-result:{task_id}:{agent_id}
-  └─ Hash fields:
-     - confidence: 0.85
-     - iteration: 1
-     - timestamp: 2025-10-23T10:30:00Z
-     - output: "Implementation complete"
-```
-
-**Returns JSON:**
-```json
-{
-  "task_id": "task-001",
-  "total_agents": 3,
-  "successful_agents": 3,
-  "failed_agents": [],
-  "results": [
-    {
-      "agent_id": "coder-1",
-      "confidence": 0.92,
-      "iteration": 1,
-      "timestamp": "2025-10-23T10:30:00Z",
-      "output": "Implementation complete"
-    },
-    {
-      "agent_id": "reviewer-1",
-      "confidence": 0.88,
-      "iteration": 1,
-      "timestamp": "2025-10-23T10:31:00Z",
-      "output": "Code review passed"
-    }
-  ],
-  "consensus": 0.90
-}
-```
-
-**Example - Calculate Consensus:**
-```bash
-# Collect results with consensus calculation
-RESULTS=$(./collect-results.sh \
-  --task-id "task-001" \
-  --agent-ids "coder-1,reviewer-1,tester-1" \
-  --calculate-consensus \
-  --min-confidence 0.75)
-
-CONSENSUS=$(echo "$RESULTS" | jq -r '.consensus')
-echo "Team consensus: $CONSENSUS"
-
-# Check if consensus meets threshold
-if (( $(echo "$CONSENSUS >= 0.90" | bc -l) )); then
-  echo "✅ Consensus threshold met"
-else
-  echo "❌ Needs iteration"
-fi
-```
-
----
-
-## Coordination Patterns
-
-### Pattern 1: Simple Chain (Sequential)
-
-Agents execute sequentially, each waiting for the previous to complete.
-
-```bash
-TASK_ID="task-001"
-
-# Agent 1: Execute and signal completion
-(
-  echo "Agent 1: Working..."
-  sleep 2
-  ./signal.sh send --task-id "$TASK_ID" --signal "agent-1-done" --payload '{}'
-) &
-
-# Agent 2: Wait for Agent 1, then execute
-(
-  ./signal.sh wait --task-id "$TASK_ID" --signal "agent-1-done"
-  echo "Agent 2: Working..."
-  sleep 2
-  ./signal.sh send --task-id "$TASK_ID" --signal "agent-2-done" --payload '{}'
-) &
-
-# Wait for all agents
-wait
-echo "Chain complete"
-```
-
-### Pattern 2: Broadcast with Consensus
-
-Coordinator broadcasts work to multiple agents, then collects consensus.
-
-```bash
-TASK_ID="task-002"
-AGENTS="agent-1,agent-2,agent-3"
-
-# Store context for agents
-./store-context.sh \
-  --task-id "$TASK_ID" \
-  --key "work-context" \
-  --value '{"task": "Review code", "file": "main.py"}'
-
-# Broadcast start signal to all agents
-./signal.sh broadcast \
-  --task-id "$TASK_ID" \
-  --signal "start-work" \
-  --agents "$AGENTS" \
-  --payload '{"iteration": 1}'
-
-# Wait for agents to complete (agents must report results to Redis)
-sleep 10
-
-# Collect results and calculate consensus
-RESULTS=$(./collect-results.sh \
-  --task-id "$TASK_ID" \
-  --agent-ids "$AGENTS" \
-  --calculate-consensus \
-  --min-confidence 0.75)
-
-echo "$RESULTS" | jq .
-```
-
-### Pattern 3: Hierarchical (Loop Dependencies)
-
-Implement multi-stage workflows with gate checks between stages.
-
-```bash
-TASK_ID="task-003"
-
-# Stage 1: Implementation agents
-LOOP3_AGENTS="coder-1,researcher-1"
-./signal.sh broadcast \
-  --task-id "$TASK_ID" \
-  --signal "stage-1-start" \
-  --agents "$LOOP3_AGENTS" \
-  --payload '{}'
-
-# Wait for implementation results
-sleep 10
-LOOP3_RESULTS=$(./collect-results.sh \
-  --task-id "$TASK_ID" \
-  --agent-ids "$LOOP3_AGENTS" \
-  --calculate-consensus)
-
-LOOP3_CONSENSUS=$(echo "$LOOP3_RESULTS" | jq -r '.consensus')
-
-# Gate check
-GATE_THRESHOLD=0.75
-if (( $(echo "$LOOP3_CONSENSUS >= $GATE_THRESHOLD" | bc -l) )); then
-  echo "✅ Gate passed: $LOOP3_CONSENSUS"
-
-  # Stage 2: Validation agents (only if gate passed)
-  LOOP2_AGENTS="reviewer-1,tester-1"
-  ./signal.sh send \
-    --task-id "$TASK_ID" \
-    --signal "gate-passed" \
-    --payload "{\"loop3_consensus\": $LOOP3_CONSENSUS}"
-
-  ./signal.sh broadcast \
-    --task-id "$TASK_ID" \
-    --signal "stage-2-start" \
-    --agents "$LOOP2_AGENTS" \
-    --payload '{}'
-else
-  echo "❌ Gate failed: $LOOP3_CONSENSUS - relaunch Stage 1"
-fi
-```
-
----
-
-## Agent Integration Examples
-
-### Example 1: Agent Storing Context
-
-```bash
-#!/bin/bash
-# Agent: backend-dev
-
-TASK_ID="$1"
-AGENT_ID="backend-dev-1"
-
-# Retrieve work context
-CONTEXT=$(./retrieve-context.sh --task-id "$TASK_ID" --key "work-context")
-GOAL=$(echo "$CONTEXT" | jq -r '.goal')
-
-echo "Working on: $GOAL"
-
-# Do work...
-sleep 5
-
-# Report confidence to Redis
-redis-cli HSET "result:${TASK_ID}:${AGENT_ID}" \
-  confidence 0.92 \
-  iteration 1 \
-  timestamp "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
-  output "Implementation complete"
-
-echo "Reported confidence: 0.92"
-```
-
-### Example 2: Agent Waiting for Signal
-
-```bash
-#!/bin/bash
-# Agent: validator
-
-TASK_ID="$1"
-AGENT_ID="validator-1"
-
-echo "Waiting for gate to pass..."
-
-# Block until signal received
-SIGNAL_PAYLOAD=$(./signal.sh wait \
-  --task-id "$TASK_ID" \
-  --signal "gate-passed" \
-  --timeout 300)
-
-if [ $? -eq 0 ]; then
-  LOOP3_CONSENSUS=$(echo "$SIGNAL_PAYLOAD" | jq -r '.loop3_consensus')
-  echo "Gate passed with consensus: $LOOP3_CONSENSUS"
-
-  # Retrieve implementation results
-  CONTEXT=$(./retrieve-context.sh --task-id "$TASK_ID" --key "implementation")
-
-  # Validate...
-  echo "Validating implementation..."
-  sleep 3
-
-  # Report result
-  redis-cli HSET "result:${TASK_ID}:${AGENT_ID}" \
-    confidence 0.88 \
-    iteration 1 \
-    timestamp "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
-    output "Validation passed"
-else
-  echo "Timeout waiting for gate signal"
-  exit 1
-fi
-```
-
----
-
-## Deprecated Features (v3.0.0)
-
-### Waiting Mode (invoke-waiting-mode.sh)
-
-**Status:** Partially deprecated (report/collect still active)
-
-The `enter` and `wake` subcommands are deprecated as they caused agent lifecycle issues (indefinite blocking). Agents should now exit cleanly after reporting results.
-
-**Still Supported:**
-- `report`: Report confidence scores
-- `collect`: Collect consensus
-- `shutdown`: Clean shutdown
-
-**No Longer Supported:**
-- `enter`: Agents should exit instead
-- `wake`: Use broadcast signaling instead
-
-**Migration Path:**
-```bash
-# OLD (deprecated):
-./invoke-waiting-mode.sh enter --task-id "$TASK_ID" --agent-id "$AGENT_ID"
-
-# NEW (use signal.sh):
-./signal.sh wait --task-id "$TASK_ID" --signal "wake-signal"
-```
-
-### CFN Loop Orchestrator (orchestrate-cfn-loop.sh)
-
-**Status:** Deprecated (kept for backward compatibility)
-
-The orchestrator script contains CFN-specific workflow logic and will be moved to the `cfn-loop-validation` skill in a future release.
-
-**Recommendation:** Build custom orchestrators using the core primitives instead of relying on this monolithic script.
-
----
-
-## Best Practices
-
-### 1. Always Set TTL
-
-Context and signals should have reasonable TTLs to prevent Redis memory leaks:
-
-```bash
-# Good: 24-hour TTL
-./store-context.sh --task-id "$TASK_ID" --key "context" --value "$JSON" --ttl 86400
-
-# Signals auto-expire after 1 hour
-./signal.sh send --task-id "$TASK_ID" --signal "start" --payload '{}'
-```
-
-### 2. Validate JSON Before Storage
-
-```bash
-# Validate JSON structure
-if echo "$CONTEXT_JSON" | jq empty 2>/dev/null; then
-  ./store-context.sh --task-id "$TASK_ID" --key "context" --value "$CONTEXT_JSON"
-else
-  echo "Error: Invalid JSON"
-  exit 1
-fi
-```
-
-### 3. Use Namespaces for Isolation
-
-```bash
-# Separate production and test environments
-./store-context.sh \
-  --task-id "$TASK_ID" \
-  --key "config" \
-  --value "$JSON" \
-  --namespace "prod:context"
-
-./store-context.sh \
-  --task-id "$TASK_ID" \
-  --key "config" \
-  --value "$JSON" \
-  --namespace "test:context"
-```
-
-### 4. Handle Timeouts Gracefully
-
-```bash
-# Always check signal.sh exit code
-if SIGNAL=$(./signal.sh wait --task-id "$TASK_ID" --signal "ready" --timeout 60); then
-  echo "Signal received: $SIGNAL"
-else
-  echo "Timeout - proceeding with default behavior"
-fi
-```
-
-### 5. Clean Up After Task Completion
-
-```bash
-# Delete task-specific keys after completion
-redis-cli DEL "context:${TASK_ID}:*"
-redis-cli DEL "signal:${TASK_ID}:*"
-redis-cli DEL "result:${TASK_ID}:*"
-```
-
----
-
-## Testing
-
-### Unit Tests
-
-Each primitive has corresponding tests in `tests/primitives/`:
-
-```bash
-# Test context storage
-./tests/primitives/test-store-context.sh
-
-# Test context retrieval
-./tests/primitives/test-retrieve-context.sh
-
-# Test signaling
-./tests/primitives/test-signal.sh
-
-# Test result collection
-./tests/primitives/test-collect-results.sh
-```
-
-### Integration Tests
-
-Test coordination patterns end-to-end:
-
-```bash
-# Test chain pattern
-./tests/integration/test-chain-pattern.sh
-
-# Test broadcast pattern
-./tests/integration/test-broadcast-pattern.sh
-
-# Test hierarchical pattern
-./tests/integration/test-hierarchical-pattern.sh
-```
-
----
-
-## Performance
-
-### Benchmarks (100 operations)
-
-| Operation | Avg Latency | Throughput |
-|-----------|-------------|------------|
-| store-context.sh | 3ms | 333 ops/s |
-| retrieve-context.sh | 2ms | 500 ops/s |
-| signal.sh send | 2ms | 500 ops/s |
-| signal.sh wait (immediate) | 3ms | 333 ops/s |
-| collect-results.sh (3 agents) | 8ms | 125 ops/s |
-
-**Zero-Token Waiting:**
-- BLPOP blocks without API calls (0 token cost)
-- Immediate wake-up (<100ms latency)
-- Scales to 100+ concurrent agents
-
----
-
-## Security Considerations
-
-### 1. No Sensitive Data in Payloads
-
-Never store API keys, credentials, or secrets in Redis context:
-
-```bash
-# BAD:
-./store-context.sh --key "config" --value '{"api_key": "sk-xxx"}'
-
-# GOOD:
-./store-context.sh --key "config" --value '{"api_key_ref": "env:API_KEY"}'
-```
-
-### 2. Namespace Isolation
-
-Use namespaces to prevent cross-task contamination:
-
-```bash
---namespace "user-${USER_ID}:context"
-```
-
-### 3. TTL Enforcement
-
-Always set TTLs to prevent indefinite data retention:
-
-```bash
---ttl 3600  # 1 hour maximum
-```
-
----
-
-## Error Handling
-
-### Common Errors
-
-**Error:** `Context key does not exist`
-```bash
-# Check if key exists before retrieval
-if redis-cli EXISTS "context:${TASK_ID}:${KEY}" | grep -q "1"; then
-  ./retrieve-context.sh --task-id "$TASK_ID" --key "$KEY"
-else
-  echo "Key not found, using default"
-fi
-```
-
-**Error:** `Timeout waiting for signal`
-```bash
-# Always provide timeout for production
-./signal.sh wait --task-id "$TASK_ID" --signal "start" --timeout 300 || {
-  echo "Timeout - sending fallback signal"
-  ./signal.sh send --task-id "$TASK_ID" --signal "timeout-fallback" --payload '{}'
-}
-```
-
-**Error:** `Failed to retrieve context value`
-```bash
-# Validate Redis connection
-if ! redis-cli PING | grep -q "PONG"; then
-  echo "Redis unavailable"
-  exit 1
-fi
-```
-
----
-
-## Version History
-
-### v3.0.0 (2025-10-23) - Pure Coordination Primitives
-- **Breaking:** Removed CFN-specific logic from primitives
-- **Added:** `store-context.sh` - Generic JSON storage
-- **Added:** `retrieve-context.sh` - Generic JSON retrieval
-- **Added:** `signal.sh` - Pub/sub signaling
-- **Added:** `collect-results.sh` - Result aggregation
-- **Deprecated:** `invoke-waiting-mode.sh` enter/wake commands
-- **Deprecated:** `orchestrate-cfn-loop.sh` (moved to cfn-loop-validation skill)
-- **Migration:** CFN-specific logic moved to separate orchestration layer
-
-### v2.1.0 (2025-10-20) - Agent Completion Protocol
-- Three-layer timeout protection
-- Process-based completion detection
-- Heartbeat monitoring
-
-### v2.0.0 (2025-10-18) - Metrics & Observability
-- Comprehensive metrics collection
-- Prometheus export support
-- Deliverable verification (BUG #11 fix)
-
----
-
-## Support
-
-**Skill Owner:** Redis Coordination Team
-**Documentation:** `.claude/skills/redis-coordination/SKILL.md`
-**Issues:** Tag with `skill:redis-coordination`
-
-**Related Skills:**
-- `cfn-loop-validation` - CFN Loop orchestration (uses these primitives)
-- `agent-spawning` - Agent lifecycle management
-- `hook-pipeline` - Post-edit validation
-
----
-
-## License
-
-Part of Claude Flow Novice - AI Agent Orchestration Framework