claude-flow-novice 2.9.1 → 2.10.1

package/.claude/skills/cfn-agent-output-processing/SKILL.md

@@ -0,0 +1,359 @@
+# Agent Output Processing Skill (Universal)
+
+**Version:** 1.0.0
+**Status:** Production
+**Purpose:** Guaranteed structured output extraction from any agent type
+
+---
+
+## Overview
+
+**Problem Solved:** Agent templates cannot force tool execution or guarantee output format.
+
+**Solution:** Orchestrator-controlled output processing with robust parsing, validation, and Redis coordination.
+
+**Applicable To:** Any agent requiring structured output (Product Owner, validators, analyzers, decision-makers)
+
+---
+
+## Core Principle
+
+```
+Agent Templates → Focus on analysis/decision-making
+Skills → Handle execution/coordination/formatting
+Orchestrators → Control workflow and Redis state
+```
+
+**Why This Works:**
+- ✅ Agents do what they're good at (analysis)
+- ✅ Skills enforce structure (parsing, validation)
+- ✅ Orchestrators maintain coordination (Redis, timing)
+- ✅ No reliance on agent tool usage compliance
+
+---
+
+## Architecture
+
+### Generic Flow
+
+```
+1. Orchestrator → Spawn agent with structured context
+2. Skill → Capture agent stdout/stderr
+3. Skill → Parse output (pattern matching with fallbacks)
+4. Skill → Validate output (business logic checks)
+5. Skill → Transform to required format (JSON, etc.)
+6. Skill → Push to Redis (coordination state)
+7. Skill → Signal completion
+```
+
+### Skill Components
+
+```
+.claude/skills/agent-output-processing/
+├── SKILL.md                          # This file
+├── execute-with-parsing.sh           # Generic agent wrapper
+├── parse-structured-output.sh        # Multi-pattern parser
+├── validate-output.sh                # Validation framework
+└── patterns/                         # Agent-specific patterns
+    ├── product-owner.json            # PROCEED/ITERATE/ABORT
+    ├── validator.json                # confidence + feedback
+    ├── analyzer.json                 # metrics + recommendations
+    └── decision-maker.json           # choice + reasoning
+```
+
+---
+
+## Usage Patterns
+
+### Pattern 1: Decision Output (Product Owner)
+
+**Output Requirements:** Must extract PROCEED/ITERATE/ABORT
+
+```bash
+RESULT=$(./.claude/skills/agent-output-processing/execute-with-parsing.sh \
+  --agent-type "product-owner" \
+  --pattern-file "patterns/product-owner.json" \
+  --task-id "$TASK_ID" \
+  --agent-id "$AGENT_ID" \
+  --context "Loop 2 consensus: 0.92" \
+  --redis-key "swarm:${TASK_ID}:${AGENT_ID}:decision")
+
+DECISION=$(echo "$RESULT" | jq -r '.decision')
+```
+
+**Pattern File (patterns/product-owner.json):**
+```json
+{
+  "name": "product-owner-decision",
+  "required_fields": ["decision"],
+  "patterns": [
+    {
+      "field": "decision",
+      "regex": "Decision:\\s*(PROCEED|ITERATE|ABORT)",
+      "extract": "keyword",
+      "fallbacks": [
+        "(PROCEED|ITERATE|ABORT)",
+        "(?i)(proceed|iterate|abort)"
+      ]
+    },
+    {
+      "field": "reasoning",
+      "regex": "Reasoning:\\s*(.+)",
+      "extract": "capture_group"
+    }
+  ],
+  "validation": {
+    "decision": {
+      "type": "enum",
+      "values": ["PROCEED", "ITERATE", "ABORT"]
+    }
+  }
+}
+```
+
+### Pattern 2: Confidence + Feedback (Validators)
+
+**Output Requirements:** Must extract confidence score and feedback list
+
+```bash
+RESULT=$(./.claude/skills/agent-output-processing/execute-with-parsing.sh \
+  --agent-type "reviewer" \
+  --pattern-file "patterns/validator.json" \
+  --task-id "$TASK_ID" \
+  --agent-id "$AGENT_ID" \
+  --context "Review Loop 3 implementation" \
+  --redis-key "swarm:${TASK_ID}:${AGENT_ID}:result")
+
+CONFIDENCE=$(echo "$RESULT" | jq -r '.confidence')
+FEEDBACK=$(echo "$RESULT" | jq -r '.feedback[]')
+```
+
+**Pattern File (patterns/validator.json):**
+```json
+{
+  "name": "validator-result",
+  "required_fields": ["confidence"],
+  "patterns": [
+    {
+      "field": "confidence",
+      "regex": "Confidence:\\s*([0-9.]+)",
+      "extract": "float",
+      "fallbacks": [
+        "confidence[:\\s]+([0-9.]+)",
+        "score[:\\s]+([0-9.]+)"
+      ]
+    },
+    {
+      "field": "feedback",
+      "regex": "Feedback:\\s*\\n([\\s\\S]+?)(?=\\n\\n|$)",
+      "extract": "multiline_list"
+    }
+  ],
+  "validation": {
+    "confidence": {
+      "type": "float",
+      "min": 0.0,
+      "max": 1.0
+    }
+  }
+}
+```
+
+### Pattern 3: Metrics + Recommendations (Analyzers)
+
+**Output Requirements:** Must extract metrics object and recommendation list
+
+```bash
+RESULT=$(./.claude/skills/agent-output-processing/execute-with-parsing.sh \
+  --agent-type "code-analyzer" \
+  --pattern-file "patterns/analyzer.json" \
+  --task-id "$TASK_ID" \
+  --agent-id "$AGENT_ID" \
+  --context "Analyze codebase quality" \
+  --redis-key "swarm:${TASK_ID}:${AGENT_ID}:analysis")
+
+COMPLEXITY=$(echo "$RESULT" | jq -r '.metrics.complexity')
+RECOMMENDATIONS=$(echo "$RESULT" | jq -r '.recommendations[]')
+```
+
+---
+
+## Implementation
+
+### execute-with-parsing.sh
+
+```bash
+#!/bin/bash
+set -euo pipefail
+
+# Parse arguments
+AGENT_TYPE=""
+PATTERN_FILE=""
+TASK_ID=""
+AGENT_ID=""
+CONTEXT=""
+REDIS_KEY=""
+TIMEOUT=900
+
+while [[ $# -gt 0 ]]; do
+  case $1 in
+    --agent-type) AGENT_TYPE="$2"; shift 2 ;;
+    --pattern-file) PATTERN_FILE="$2"; shift 2 ;;
+    --task-id) TASK_ID="$2"; shift 2 ;;
+    --agent-id) AGENT_ID="$2"; shift 2 ;;
+    --context) CONTEXT="$2"; shift 2 ;;
+    --redis-key) REDIS_KEY="$2"; shift 2 ;;
+    --timeout) TIMEOUT="$2"; shift 2 ;;
+    *) echo "Unknown: $1"; exit 1 ;;
+  esac
+done
+
+# Spawn agent and capture output
+AGENT_OUTPUT=$(timeout "$TIMEOUT" npx claude-flow-novice agent "$AGENT_TYPE" \
+  --task-id "$TASK_ID" \
+  --agent-id "$AGENT_ID" \
+  --context "$CONTEXT" 2>&1 || true)
+
+# Parse structured output using pattern file
+PARSED_OUTPUT=$(./parse-structured-output.sh \
+  --pattern-file "$PATTERN_FILE" \
+  --output "$AGENT_OUTPUT")
+
+# Validate parsed output
+if ! ./validate-output.sh --pattern-file "$PATTERN_FILE" --data "$PARSED_OUTPUT"; then
+  echo "ERROR: Output validation failed" >&2
+  exit 1
+fi
+
+# Push to Redis if key provided
+if [ -n "$REDIS_KEY" ]; then
+  echo "$PARSED_OUTPUT" | redis-cli -x LPUSH "$REDIS_KEY" >/dev/null
+  redis-cli LPUSH "swarm:${TASK_ID}:${AGENT_ID}:done" "complete" >/dev/null
+fi
+
+# Output structured result
+echo "$PARSED_OUTPUT"
+```
+
+---
+
+## Benefits Over Template-Based
+
+| Aspect | Template-Based | Skill-Based (Universal) |
+|--------|----------------|-------------------------|
+| **Execution Guarantee** | ❌ Agent decides | ✅ Skill enforces |
+| **Output Format** | ❌ Inconsistent | ✅ Guaranteed structure |
+| **Parsing Robustness** | ❌ None | ✅ Multi-pattern fallbacks |
+| **Validation** | ❌ Manual | ✅ Automated schema checks |
+| **Redis Coordination** | ❌ Agent-dependent | ✅ Skill-controlled |
+| **Reusability** | ❌ Per-agent templates | ✅ Universal pattern system |
+| **Testability** | ❌ Hard to test | ✅ Unit tests per pattern |
+| **Extensibility** | ❌ Template changes | ✅ Add pattern files |
+
+---
+
+## Agent-Specific Implementations
+
+### Existing: Product Owner Decision
+**Location:** `.claude/skills/product-owner-decision/`
+**Pattern:** PROCEED/ITERATE/ABORT extraction
+**Status:** ✅ Operational (BUG #11 fix)
+
+### Future: Validator Agents
+**Pattern:** Confidence + feedback extraction
+**Use Case:** Loop 2 validators (reviewer, tester, security)
+**Benefit:** Guaranteed confidence scoring
+
+### Future: Analyzer Agents
+**Pattern:** Metrics + recommendations extraction
+**Use Case:** Code quality, performance, security analyzers
+**Benefit:** Structured analysis output
+
+---
+
+## Migration Strategy
+
+### Phase 1: Product Owner (✅ Complete)
+- Created dedicated skill
+- Integrated with orchestrator
+- Solves BUG #11
+
+### Phase 2: Validators (Planned)
+- Extract confidence scores reliably
+- Parse feedback into structured lists
+- Prevent 0.0 confidence issues
+
+### Phase 3: All Decision-Making Agents (Future)
+- Standardize output processing
+- Centralize pattern definitions
+- Enable cross-agent consistency
+
+---
+
+## Testing
+
+### Unit Tests
+```bash
+# Test pattern parsing
+./test-parse-structured-output.sh --pattern-file patterns/product-owner.json
+
+# Test validation logic
+./test-validate-output.sh --pattern-file patterns/validator.json
+```
+
+### Integration Tests
+```bash
+# Test full agent execution with parsing
+./test-execute-with-parsing.sh --agent-type product-owner
+```
+
+---
+
+## Related Skills
+
+- **Product Owner Decision** (`.claude/skills/product-owner-decision/SKILL.md`) - Specific implementation
+- **Redis Coordination** (`.claude/skills/redis-coordination/SKILL.md`) - State management
+- **CFN Loop Validation** (`.claude/skills/cfn-loop-validation/SKILL.md`) - Quality gates
+
+---
+
+## Lessons Learned
+
+### ANTI-PATTERN: Template-Forced Tool Usage
+**What Doesn't Work:** Telling agents "Use Bash tool to execute X"
+
+**Why:** Agents interpret instructions autonomously, treating code blocks as documentation
+
+**Better:** Capture agent output and parse it with robust patterns
+
+### PATTERN: Orchestrator Control
+**Principle:** Coordination belongs in orchestrators/skills, not agent templates
+
+**Rationale:**
+- Agents focus on analysis (what they do well)
+- Skills handle structure (parsing, validation)
+- Orchestrators manage coordination (Redis, timing)
+
+### PATTERN: Multi-Fallback Parsing
+**Principle:** Use multiple pattern matching strategies with increasing leniency
+
+**Example:**
+1. Strict: "Decision: PROCEED" (labeled, exact case)
+2. Moderate: "PROCEED" (standalone keyword)
+3. Lenient: "proceed" (case-insensitive)
+4. Desperate: JSON extraction
+
+---
+
+## Version History
+
+### 1.0.0 (2025-10-20)
+- Initial skill creation
+- Generalized from Product Owner fix
+- Universal pattern-based output processing
+- Validation framework
+- Redis coordination integration
+
+---
+
+**Summary:** This skill provides universal output processing for any agent type, eliminating reliance on template-based tool usage compliance and guaranteeing structured, validated results.

package/.claude/skills/cfn-agent-selector/SKILL.md

@@ -0,0 +1,90 @@
+# Agent Selector Skill
+
+**Version:** 1.0.0
+**Purpose:** Select optimal agents for CFN Loop v3 based on task type and requirements
+
+## Overview
+
+Recommends Loop 3 (producers) and Loop 2 (evaluators) agents based on:
+- Task type (from task-classifier)
+- Task description keywords
+- Complexity requirements
+
+## Usage
+
+```bash
+AGENTS=$(./.claude/skills/agent-selector/select-agents.sh \
+  --task-type "software-development" \
+  --description "Implement JWT authentication with refresh tokens")
+
+echo "$AGENTS" | jq '.loop3[]'  # ["backend-dev", "security-specialist"]
+echo "$AGENTS" | jq '.loop2[]'  # ["reviewer", "tester", "security-auditor"]
+```
+
+## Output Format
+
+```json
+{
+  "loop3": ["agent1", "agent2", "agent3"],
+  "loop2": ["validator1", "validator2", "validator3"],
+  "loop4": "product-owner",
+  "reasoning": "Explanation of agent selection"
+}
+```
+
+## Agent Selection Rules
+
+### Software Development
+**Base Loop 3:** backend-dev, coder
+**Add if keywords:**
+- "security", "authentication", "JWT" → security-specialist
+- "database", "SQL", "schema" → database-engineer (if exists)
+- "deploy", "CI/CD", "infrastructure" → devops-engineer
+- "frontend", "React", "UI" → react-frontend-engineer
+
+**Loop 2:** reviewer, tester, security-auditor
+
+### Content Creation
+**Base Loop 3:** copywriter, content-strategist
+**Add if keywords:**
+- "SEO", "search", "keywords" → seo-specialist
+- "technical", "documentation" → technical-writer (if exists)
+
+**Loop 2:** editor, brand-reviewer, compliance-checker
+
+### Research
+**Base Loop 3:** researcher, data-analyst
+**Add if keywords:**
+- "statistics", "data analysis" → statistician (if exists)
+- "domain-specific" → domain-expert
+
+**Loop 2:** fact-checker, methodology-reviewer, statistician
+
+### Design
+**Base Loop 3:** ui-designer, ux-researcher
+**Add if keywords:**
+- "visual", "branding" → visual-designer
+- "accessibility" → accessibility-advocate
+
+**Loop 2:** accessibility-advocate, design-critic, user-tester
+
+### Infrastructure
+**Base Loop 3:** devops-engineer, terraform-engineer
+**Add if keywords:**
+- "Kubernetes", "k8s", "container" → kubernetes-architect
+- "network", "security" → network-engineer (if exists)
+
+**Loop 2:** security-auditor, cost-optimizer, compliance-checker
+
+### Data Engineering
+**Base Loop 3:** data-engineer, pipeline-builder
+**Add if keywords:**
+- "ETL", "transformation" → etl-specialist
+- "streaming", "real-time" → streaming-specialist (if exists)
+
+**Loop 2:** data-quality-validator, schema-reviewer, performance-tester
+
+## Integration
+
+Used by:
+- `.claude/agents/cfn-v3-coordinator.md` - Agent selection

package/.claude/skills/cfn-agent-selector/select-agents.sh

@@ -0,0 +1,112 @@
+#!/usr/bin/env bash
+# Dynamic Agent Selection with JSON Registry
+
+set -euo pipefail
+
+# Paths
+REGISTRY_PATH=".claude/skills/cfn-agent-discovery/agents-registry.json"
+LOG_PATH="/tmp/agent_selection_$(date +%s).log"
+
+# Validate inputs
+if [[ $# -lt 2 ]]; then
+    echo "Usage: $0 --task-type TYPE --description 'task description'" >&2
+    exit 1
+fi
+
+# Parse arguments
+while [[ $# -gt 0 ]]; do
+    case "$1" in
+        --task-type) TASK_TYPE="$2"; shift 2 ;;
+        --description) DESCRIPTION="$2"; shift 2 ;;
+        *) echo "Unknown parameter: $1" >&2; exit 1 ;;
+    esac
+done
+
+# Validate registry
+if [[ ! -f "$REGISTRY_PATH" ]]; then
+    echo "Error: Agent registry not found at $REGISTRY_PATH. Run agent discovery first." >&2
+    exit 1
+fi
+
+# Score agents function with improved flat namespace matching
+score_agents() {
+    local registry_path="$1"
+    local description="$2"
+    local task_type="$3"
+
+    # Complex JQ query for flexible matching
+    jq -r --arg desc "$description" --arg task_type "$task_type" '
+        [
+            .agents[]
+            | select(
+                # Type match using flat namespace
+                (
+                    # Direct type match
+                    .type == $task_type or
+                    # Case-insensitive substring match
+                    (.type | ascii_downcase | contains($task_type | ascii_downcase))
+                )
+            )
+            | {
+                name: .name,
+                file: .file,
+                score: (
+                    # Base type match score
+                    10 +
+                    # Filename-based scoring (flat namespace detection)
+                    (
+                        if (.file | ascii_downcase | contains($task_type | ascii_downcase))
+                        then 5
+                        else 0
+                        end
+                    ) +
+                    # Keyword matching
+                    (
+                        [.keywords[]]
+                        | map(select(
+                            ($desc | ascii_downcase | contains(. | ascii_downcase))
+                        ))
+                        | length * 3
+                    )
+                )
+            }
+        ]
+        | sort_by(.score)
+        | reverse
+        | .[0:3]
+        | map(.name)
+    ' "$registry_path"
+}
+
+# Score agents with strategic fallback
+LOOP3_AGENTS=$(score_agents "$REGISTRY_PATH" "$DESCRIPTION" "coder,developer")
+LOOP2_AGENTS=$(score_agents "$REGISTRY_PATH" "$DESCRIPTION" "reviewer,validator")
+
+# Default fallback
+if [[ -z "$LOOP3_AGENTS" || "$LOOP3_AGENTS" == "[]" ]]; then
+    LOOP3_AGENTS='["product-owner"]'
+fi
+
+if [[ -z "$LOOP2_AGENTS" || "$LOOP2_AGENTS" == "[]" ]]; then
+    LOOP2_AGENTS='["product-owner"]'
+fi
+
+# Log the results
+{
+    echo "Task Description: $DESCRIPTION"
+    echo "Loop 3 Agents: $LOOP3_AGENTS"
+    echo "Loop 2 Agents: $LOOP2_AGENTS"
+} >> "$LOG_PATH"
+
+# Output JSON result with detailed reasoning
+jq -n \
+    --argjson loop3 "$LOOP3_AGENTS" \
+    --argjson loop2 "$LOOP2_AGENTS" \
+    --arg desc "$DESCRIPTION" \
+    '{
+        "loop3": $loop3,
+        "loop2": $loop2,
+        "loop4": "product-owner",
+        "reasoning": "Selected agents by scoring against task description: \($desc)",
+        "log_path": "'"$LOG_PATH"'"
+    }'

package/.claude/skills/cfn-agent-spawning/SKILL.md

@@ -0,0 +1,135 @@
+---
+name: Agent Spawning
+version: 1.4.0
+complexity: High
+keywords: [
+    "dynamic agent deployment",
+    "dependency management",
+    "resource optimization",
+    "multi-agent initialization",
+    "dependency validation"
+]
+triggers: [
+    "complex system design",
+    "dependency resolution",
+    "agent lifecycle management"
+]
+performance_targets: {
+    "dependency_check_time_ms": 50,
+    "spawn_time_ms": 200,
+    "stop_time_ms": 150,
+    "success_rate": 99.5
+}
+---
+
+# Agent Spawning Skill - Dependency Enhanced
+
+## Dependency Management Overview
+
+Version 1.4.0 introduces comprehensive dependency management for agent spawning, ensuring robust and reliable multi-agent deployment.
+
+### Dependency Checks
+
+The updated `spawn-agent.sh` now includes thorough dependency validation:
+
+1. **System Requirements**
+   - Bash 4.0+
+   - Node.js (Latest LTS)
+   - Required CLI tools
+
+2. **Node.js Module Validation**
+   - redis
+   - dotenv
+   - Other required modules
+
+3. **Claude Flow Prerequisites**
+   - Configured `.claude` directory
+   - Coordination scripts
+   - Task tool accessibility
+
+## Dependency Validation Process
+
+### System Check
+```bash
+# Validates:
+# - Bash version
+# - Required command-line tools
+# - Node.js module availability
+# - Claude Flow configuration
+./spawn-agent.sh
+```
+
+### Error Handling Example
+```
+[ERROR] Missing Dependencies:
+  - bash>=4.0
+  - redis
+  - dotenv
+
+[WARNING] Recommended Installation:
+  1. Install Node.js and npm (latest LTS version)
+  2. Run: npm install redis dotenv
+  3. Clone Claude Flow Novice repository
+```
+
+## Usage Examples (Unchanged from Previous Version)
+
+Spawn multiple agents:
+```bash
+./spawn-agent.sh \
+  --task "Implement user authentication" \
+  --agents coder,security-specialist,tester \
+  --agent-id coordinator-1
+```
+
+## Performance Targets
+
+- Dependency check time: <50ms
+- Spawn time: <200ms
+- Stop time: <150ms
+- Success rate: 99.5%
+
+## Changelog
+
+### Version 1.4.0 (2025-10-18)
+- Added comprehensive dependency validation
+- Enhanced system and module checking
+- Improved error reporting
+- Clear installation instructions
+- Reduced false-positive spawn attempts
+
+### Version 1.3.0
+- Agent termination capabilities
+- Stop specific or all agents
+- Enhanced agent lifecycle management
+
+### Version 1.2.0
+- CLI wrapper implementation
+- Agent type validation
+- Template discovery
+
+## Status
+- **Operational Status:** ✅ FULLY OPERATIONAL (v2.0.0)
+- **CLI Command:** `npx claude-flow-spawn` (production ready)
+- **Dependency Management:** FULLY IMPLEMENTED
+- **Error Handling:** HIGH PRECISION
+- **Last Verified:** 2025-10-19
+
+## Implementation Details
+
+**CLI Entry Point:** `src/cli/spawn.ts` → `dist/cli/spawn.js`
+**Worker Implementation:** `src/cli/hybrid-routing/spawn-workers.cjs`
+**Package Bin:** `"claude-flow-spawn": "dist/cli/spawn.js"`
+
+**Usage:**
+```bash
+# Spawn multiple agents
+npx claude-flow-spawn "Implement auth" --agents=coder,tester --provider zai
+
+# Using skill wrapper
+./.claude/skills/agent-spawning/spawn-agent.sh \
+  --task "Implement auth" \
+  --agents coder,tester \
+  --provider zai \
+  --redis-channel swarm:auth
+```