claude-flow-novice 2.18.6 → 2.18.7

package/.claude/skills/cfn-hybrid-routing

@@ -0,0 +1 @@
+cfn-routing-config/lib/hybrid

package/.claude/skills/cfn-local-ruvector-accelerator/AGENT_LIFECYCLE_INTEGRATION.md

@@ -0,0 +1,341 @@
+# RuVector Agent Lifecycle Integration
+
+## Overview
+
+The RuVector Agent Lifecycle Integration automatically captures and indexes agent execution data, enabling semantic search over historical agent decisions, tool usage patterns, and failure recovery strategies.
+
+## Architecture
+
+```
+Agent Execution → cfn-subagent-stop.sh → ingest-agent-transcript.sh → SQLite Database
+                                                                      ↓
+                                                           agent_transcripts
+                                                           agent_failure_patterns
+                                                                      ↓
+                                                          query-agent-patterns.sh
+```
+
+## Components
+
+### 1. Transcript Ingestion (`ingest-agent-transcript.sh`)
+
+Automatically called by `cfn-subagent-stop.sh` after each agent execution.
+
+**Functionality:**
+- Parses JSONL transcript files
+- Extracts decision points and tool usage events
+- Identifies error patterns in failed agents
+- Stores searchable snippets in `agent_transcripts` table
+
+**Schema:**
+```sql
+CREATE TABLE agent_transcripts (
+    id TEXT PRIMARY KEY,
+    agent_id TEXT NOT NULL,
+    agent_type TEXT NOT NULL,
+    task_id TEXT,
+    snippet TEXT NOT NULL,
+    snippet_type TEXT,           -- decision, tool_use, error, recovery
+    embedding BLOB,               -- placeholder for future vector embeddings
+    metadata TEXT,                -- JSON: tool, outcome, timestamp, line_number
+    success_rate REAL DEFAULT 0.0,
+    usage_count INTEGER DEFAULT 0,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+### 2. Failure Pattern Analysis (`analyze-agent-failures.sh`)
+
+Periodic analysis of agent failures to identify common patterns and recovery strategies.
+
+**Usage:**
+```bash
+# Analyze failures from last 7 days
+./analyze-agent-failures.sh --days 7
+
+# Analyze failures from last 30 days, minimum 3 occurrences
+./analyze-agent-failures.sh --days 30 --min-occurrences 3
+```
+
+**Functionality:**
+- Queries failed agents (confidence < 0.70 or explicit failure)
+- Classifies failure modes: edit_hook_validation, timeout, permission_error, syntax_error, test_failure
+- Clusters similar failures
+- Identifies recovery strategies from subsequent successful executions
+- Updates `agent_failure_patterns` table
+
+**Schema:**
+```sql
+CREATE TABLE agent_failure_patterns (
+    id TEXT PRIMARY KEY,
+    agent_type TEXT NOT NULL,
+    failure_mode TEXT NOT NULL,
+    root_cause TEXT,
+    recovery_strategy TEXT,
+    embedding BLOB,
+    occurrence_count INTEGER DEFAULT 1,
+    last_seen TIMESTAMP,
+    resolution_rate REAL DEFAULT 0.0,
+    metadata TEXT,
+    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+);
+```
+
+### 3. Pattern Query Interface (`query-agent-patterns.sh`)
+
+Search historical agent data for guidance on current tasks.
+
+**Usage:**
+```bash
+# Search all patterns
+./query-agent-patterns.sh --query "pre-edit hook validation failure"
+
+# Search only failure patterns
+./query-agent-patterns.sh --query "validation" --search-type failures
+
+# Filter by agent type
+./query-agent-patterns.sh --query "file write" --type coder --limit 5
+
+# Get statistics
+./query-agent-patterns.sh --search-type stats
+```
+
+**Search Types:**
+- `all` - Search both transcripts and failure patterns (default)
+- `transcripts` - Search only agent transcript snippets
+- `failures` - Search only failure patterns
+- `stats` - Display agent lifecycle statistics
+
+## Integration with cfn-subagent-stop.sh
+
+The subagent stop hook automatically triggers RuVector ingestion:
+
+```bash
+# cfn-subagent-stop.sh (excerpt)
+
+# Determine agent success/failure
+AGENT_SUCCESS="true"
+AGENT_CONFIDENCE=$(sqlite3 "$DB_PATH" "SELECT confidence FROM agents WHERE id = '$AGENT_ID';")
+
+if awk "BEGIN {exit !($AGENT_CONFIDENCE < 0.70)}"; then
+    AGENT_SUCCESS="false"
+fi
+
+# Call RuVector ingestion
+./.claude/skills/cfn-local-ruvector-accelerator/ingest-agent-transcript.sh \
+    --transcript "$TRANSCRIPT_ARCHIVE" \
+    --agent-id "$AGENT_ID" \
+    --agent-type "$AGENT_TYPE" \
+    --task-id "$TASK_ID" \
+    --success "$AGENT_SUCCESS"
+```
+
+## Data Flow
+
+### Phase 1: Transcript Ingestion (Automatic)
+
+1. Agent completes execution
+2. `cfn-subagent-stop.sh` hook fires
+3. Transcript copied to `.artifacts/transcripts/`
+4. `ingest-agent-transcript.sh` called automatically
+5. JSONL parsed for tool_use and error events
+6. Snippets stored in `agent_transcripts` table
+7. Agent metadata updated with processing status
+
+### Phase 2: Failure Pattern Analysis (Periodic)
+
+1. Run `analyze-agent-failures.sh` weekly/monthly
+2. Query failed agents from lifecycle database
+3. Extract failure contexts from transcripts
+4. Classify and cluster failure modes
+5. Identify recovery strategies from successful agents
+6. Store patterns in `agent_failure_patterns` table
+
+### Phase 3: Pattern Query (On-Demand)
+
+1. Developer/agent encounters issue
+2. Query historical patterns: `./query-agent-patterns.sh --query "similar issue"`
+3. Review matched transcripts and failure patterns
+4. Apply recovery strategies from historical successes
+
+## Example Queries
+
+### Find validation failure solutions
+```bash
+./query-agent-patterns.sh --query "validation failed" --search-type failures
+```
+
+Output:
+```
+=== Failure Pattern Matches ===
+
+Agent Type: loop3-implementer
+Failure Mode: edit_hook_validation
+Occurrences: 12
+Root Cause: pre-edit backup hook failed: file permissions on /tmp/...
+Recovery Strategy: Successful loop3-implementer agents: used similar tool chains with validation steps
+```
+
+### Find tool usage patterns for file operations
+```bash
+./query-agent-patterns.sh --query "write file" --type coder --limit 5
+```
+
+Output:
+```
+=== Agent Transcript Matches ===
+
+Agent: agent-20241209-001 (coder)
+Type: tool_use
+Snippet: {"type":"tool_use","name":"Write","input":{"file_path":"/path/to/file.ts","content":"..."},...
+Metadata: {"tool":"Write","outcome":"true","line_number":42,"timestamp":"2024-12-09T10:30:00Z"}
+```
+
+### Get agent statistics
+```bash
+./query-agent-patterns.sh --search-type stats
+```
+
+Output:
+```
+=== Agent Lifecycle Statistics ===
+
+Agent executions by type:
+loop3-implementer|145
+loop2-validator|98
+coder|67
+...
+
+Recent failures (last 7 days):
+8
+
+Most common failure modes:
+edit_hook_validation|15
+timeout|7
+test_failure|5
+```
+
+## Future Enhancements
+
+### Vector Embeddings (Phase 4)
+
+Currently, the `embedding BLOB` field is a placeholder. Future integration with actual vector embeddings:
+
+1. **Use sentence-transformers** for semantic search:
+   ```python
+   from sentence_transformers import SentenceTransformer
+   model = SentenceTransformer('all-MiniLM-L6-v2')
+   embedding = model.encode(snippet_text)
+   ```
+
+2. **Store embeddings** in the BLOB field
+3. **Query by similarity** using cosine distance
+4. **Enable questions like**: "What did agents do when encountering TypeScript type errors?"
+
+### Tool Pattern Learning (Phase 5)
+
+Track successful tool combinations:
+
+```sql
+CREATE TABLE tool_patterns (
+    id TEXT PRIMARY KEY,
+    agent_type TEXT NOT NULL,
+    tool_sequence TEXT NOT NULL,  -- "Bash,Glob,Grep,Write,Bash"
+    success_rate REAL DEFAULT 0.0,
+    usage_count INTEGER DEFAULT 0,
+    avg_duration_ms REAL,
+    metadata TEXT
+);
+```
+
+### Confidence Calibration (Phase 6)
+
+Track reported vs. actual confidence:
+
+```sql
+CREATE TABLE confidence_calibration (
+    id TEXT PRIMARY KEY,
+    agent_type TEXT NOT NULL,
+    confidence_reported REAL NOT NULL,
+    confidence_actual REAL NOT NULL,
+    context_size INTEGER,
+    outcome TEXT
+);
+```
+
+## Maintenance
+
+### Database Size Management
+
+Monitor database growth:
+```bash
+du -h .claude/skills/cfn-redis-coordination/data/cfn-loop.db
+```
+
+Archive old transcripts:
+```bash
+# Archive transcripts older than 90 days
+sqlite3 "$DB_PATH" "DELETE FROM agent_transcripts WHERE created_at < datetime('now', '-90 days');"
+
+# Archive old failure patterns with low occurrence
+sqlite3 "$DB_PATH" "DELETE FROM agent_failure_patterns WHERE occurrence_count < 2 AND created_at < datetime('now', '-30 days');"
+```
+
+### Log Management
+
+RuVector ingestion logs are written to:
+- `.artifacts/logs/ruvector-ingestion.log` - Transcript ingestion events
+- `.artifacts/logs/ruvector-failure-analysis.log` - Failure pattern analysis
+- `.artifacts/logs/subagent-lifecycle.log` - Hook execution logs
+
+Rotate logs periodically:
+```bash
+# Rotate logs older than 30 days
+find .artifacts/logs -name "*.log" -mtime +30 -delete
+```
+
+## Troubleshooting
+
+### Ingestion not running
+
+Check if hook is being called:
+```bash
+tail -f .artifacts/logs/subagent-lifecycle.log
+```
+
+Verify script exists and is executable:
+```bash
+ls -l .claude/skills/cfn-local-ruvector-accelerator/ingest-agent-transcript.sh
+```
+
+### No patterns found
+
+Run failure analysis manually:
+```bash
+./.claude/skills/cfn-local-ruvector-accelerator/analyze-agent-failures.sh --days 30
+```
+
+Check database tables exist:
+```bash
+sqlite3 .claude/skills/cfn-redis-coordination/data/cfn-loop.db ".tables"
+```
+
+### Query returns no results
+
+Check if data exists:
+```bash
+sqlite3 .claude/skills/cfn-redis-coordination/data/cfn-loop.db "SELECT COUNT(*) FROM agent_transcripts;"
+sqlite3 .claude/skills/cfn-redis-coordination/data/cfn-loop.db "SELECT COUNT(*) FROM agent_failure_patterns;"
+```
+
+Run with broader query:
+```bash
+./query-agent-patterns.sh --query "error" --limit 20
+```
+
+## References
+
+- Analysis document: `docs/RUVECTOR_INTEGRATION_ANALYSIS.md`
+- Subagent hooks: `.claude/hooks/cfn-subagent-*.sh`
+- RuVector accelerator: `.claude/skills/cfn-local-ruvector-accelerator/`
+- Lifecycle database: `.claude/skills/cfn-redis-coordination/data/cfn-loop.db`

package/.claude/skills/cfn-local-ruvector-accelerator/analyze-agent-failures.sh

@@ -0,0 +1,275 @@
+#!/bin/bash
+# analyze-agent-failures.sh - Analyze agent failures and extract reusable patterns
+#
+# Purpose: Query failed agents from the lifecycle database, extract error patterns,
+#          cluster similar failures, and store recovery strategies for future lookup.
+#
+# Usage:
+#   ./analyze-agent-failures.sh [--days N] [--min-occurrences N]
+#
+# Examples:
+#   ./analyze-agent-failures.sh --days 7
+#   ./analyze-agent-failures.sh --days 30 --min-occurrences 3
+
+set -euo pipefail
+
+# ============================================================================
+# Configuration
+# ============================================================================
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT=$(git rev-parse --show-toplevel 2>/dev/null || echo ".")
+DB_PATH="${PROJECT_ROOT}/.claude/skills/cfn-redis-coordination/data/cfn-loop.db"
+LOG_PATH="${PROJECT_ROOT}/.artifacts/logs/ruvector-failure-analysis.log"
+
+# Ensure log directory exists
+mkdir -p "$(dirname "$LOG_PATH")"
+
+# ============================================================================
+# Parse Arguments
+# ============================================================================
+
+DAYS=7
+MIN_OCCURRENCES=2
+
+while [[ $# -gt 0 ]]; do
+    case $1 in
+        --days)
+            DAYS="$2"
+            shift 2
+            ;;
+        --min-occurrences)
+            MIN_OCCURRENCES="$2"
+            shift 2
+            ;;
+        *)
+            echo "Unknown option: $1"
+            echo "Usage: $0 [--days N] [--min-occurrences N]"
+            exit 1
+            ;;
+    esac
+done
+
+echo "[RuVector] Analyzing agent failures from last $DAYS days" | tee -a "$LOG_PATH"
+
+# ============================================================================
+# Query Failed Agents
+# ============================================================================
+
+# Extract agents with failures or low confidence
+FAILED_AGENTS=$(sqlite3 "$DB_PATH" <<EOF
+SELECT
+    id,
+    type,
+    confidence,
+    metadata
+FROM agents
+WHERE
+    (status = 'completed' AND (confidence < 0.70 OR metadata LIKE '%"success": false%'))
+    OR status = 'failed'
+    AND datetime(completed_at) >= datetime('now', '-$DAYS days')
+ORDER BY completed_at DESC;
+EOF
+)
+
+if [ -z "$FAILED_AGENTS" ]; then
+    echo "[RuVector] No failures found in last $DAYS days" | tee -a "$LOG_PATH"
+    exit 0
+fi
+
+FAILURE_COUNT=$(echo "$FAILED_AGENTS" | wc -l)
+echo "[RuVector] Found $FAILURE_COUNT failed agent executions" | tee -a "$LOG_PATH"
+
+# ============================================================================
+# Extract Failure Patterns
+# ============================================================================
+
+# Parse each failed agent and extract:
+# 1. Failure mode (from transcript errors)
+# 2. Root cause (tool, validation, timeout, etc.)
+# 3. Recovery strategy (if any subsequent success)
+
+PATTERNS_CREATED=0
+
+echo "$FAILED_AGENTS" | while IFS='|' read -r AGENT_ID AGENT_TYPE CONFIDENCE METADATA; do
+
+    # Skip empty rows
+    [ -z "$AGENT_ID" ] && continue
+
+    # Extract failure context from metadata
+    TRANSCRIPT_PATH=$(echo "$METADATA" | grep -o '"transcript_path":"[^"]*"' | cut -d'"' -f4 || echo "")
+
+    if [ -n "$TRANSCRIPT_PATH" ] && [ -f "$TRANSCRIPT_PATH" ]; then
+        # Extract error snippets from transcript
+        ERROR_LINES=$(grep -i "error\|failed\|timeout\|validation" "$TRANSCRIPT_PATH" 2>/dev/null || echo "")
+
+        if [ -n "$ERROR_LINES" ]; then
+            # Classify failure mode
+            FAILURE_MODE="unknown"
+
+            if echo "$ERROR_LINES" | grep -q "pre-edit\|post-edit"; then
+                FAILURE_MODE="edit_hook_validation"
+            elif echo "$ERROR_LINES" | grep -q "timeout"; then
+                FAILURE_MODE="timeout"
+            elif echo "$ERROR_LINES" | grep -q "permission\|access"; then
+                FAILURE_MODE="permission_error"
+            elif echo "$ERROR_LINES" | grep -q "syntax\|parse"; then
+                FAILURE_MODE="syntax_error"
+            elif echo "$ERROR_LINES" | grep -q "test.*fail\|assertion"; then
+                FAILURE_MODE="test_failure"
+            fi
+
+            # Extract root cause (first error line, truncated)
+            ROOT_CAUSE=$(echo "$ERROR_LINES" | head -n 1 | head -c 200)
+
+            # Generate pattern ID
+            PATTERN_ID="failure-${AGENT_TYPE}-${FAILURE_MODE}-$(date +%s)"
+
+            # Check if similar pattern already exists
+            EXISTING_PATTERN=$(sqlite3 "$DB_PATH" <<SQL_EOF
+SELECT id, occurrence_count
+FROM agent_failure_patterns
+WHERE agent_type = '$AGENT_TYPE' AND failure_mode = '$FAILURE_MODE'
+ORDER BY last_seen DESC
+LIMIT 1;
+SQL_EOF
+)
+
+            if [ -n "$EXISTING_PATTERN" ]; then
+                # Update existing pattern
+                EXISTING_ID=$(echo "$EXISTING_PATTERN" | cut -d'|' -f1)
+                EXISTING_COUNT=$(echo "$EXISTING_PATTERN" | cut -d'|' -f2)
+                NEW_COUNT=$((EXISTING_COUNT + 1))
+
+                sqlite3 "$DB_PATH" <<SQL_EOF
+UPDATE agent_failure_patterns
+SET
+    occurrence_count = $NEW_COUNT,
+    last_seen = datetime('now'),
+    root_cause = '$ROOT_CAUSE'
+WHERE id = '$EXISTING_ID';
+SQL_EOF
+
+                echo "[RuVector] Updated pattern: $EXISTING_ID (count: $NEW_COUNT)" | tee -a "$LOG_PATH"
+
+            else
+                # Create new pattern
+                METADATA_JSON=$(cat <<JSON_EOF
+{
+    "first_agent_id": "$AGENT_ID",
+    "confidence": $CONFIDENCE,
+    "discovered": "$(date -u +"%Y-%m-%dT%H:%M:%SZ")"
+}
+JSON_EOF
+)
+
+                sqlite3 "$DB_PATH" <<SQL_EOF
+INSERT INTO agent_failure_patterns
+(id, agent_type, failure_mode, root_cause, metadata, last_seen)
+VALUES (
+    '$PATTERN_ID',
+    '$AGENT_TYPE',
+    '$FAILURE_MODE',
+    '$ROOT_CAUSE',
+    '$METADATA_JSON',
+    datetime('now')
+);
+SQL_EOF
+
+                PATTERNS_CREATED=$((PATTERNS_CREATED + 1))
+                echo "[RuVector] Created pattern: $PATTERN_ID ($AGENT_TYPE / $FAILURE_MODE)" | tee -a "$LOG_PATH"
+            fi
+        fi
+    fi
+done
+
+echo "[RuVector] Created $PATTERNS_CREATED new failure patterns" | tee -a "$LOG_PATH"
+
+# ============================================================================
+# Identify High-Impact Patterns
+# ============================================================================
+
+# Query patterns that occur frequently (>= MIN_OCCURRENCES)
+HIGH_IMPACT=$(sqlite3 "$DB_PATH" <<EOF
+SELECT
+    agent_type,
+    failure_mode,
+    occurrence_count,
+    root_cause
+FROM agent_failure_patterns
+WHERE occurrence_count >= $MIN_OCCURRENCES
+ORDER BY occurrence_count DESC
+LIMIT 10;
+EOF
+)
+
+if [ -n "$HIGH_IMPACT" ]; then
+    echo "" | tee -a "$LOG_PATH"
+    echo "[RuVector] High-impact failure patterns (>= $MIN_OCCURRENCES occurrences):" | tee -a "$LOG_PATH"
+    echo "$HIGH_IMPACT" | while IFS='|' read -r TYPE MODE COUNT CAUSE; do
+        echo "  - $TYPE / $MODE: $COUNT occurrences" | tee -a "$LOG_PATH"
+        echo "    Cause: $(echo "$CAUSE" | head -c 80)..." | tee -a "$LOG_PATH"
+    done
+fi
+
+# ============================================================================
+# Generate Recovery Recommendations
+# ============================================================================
+
+# For patterns with multiple occurrences, check if any similar agents succeeded
+# and extract their approaches as recovery strategies
+
+echo "" | tee -a "$LOG_PATH"
+echo "[RuVector] Searching for recovery strategies..." | tee -a "$LOG_PATH"
+
+RECOVERIES_FOUND=0
+
+sqlite3 "$DB_PATH" "SELECT DISTINCT agent_type, failure_mode FROM agent_failure_patterns WHERE occurrence_count >= 2;" | \
+while IFS='|' read -r TYPE MODE; do
+
+    # Find successful agents of same type after failures
+    SUCCESSFUL=$(sqlite3 "$DB_PATH" <<SQL_EOF
+SELECT id, metadata
+FROM agents
+WHERE
+    type = '$TYPE'
+    AND status = 'completed'
+    AND confidence >= 0.80
+    AND datetime(completed_at) >= datetime('now', '-$DAYS days')
+ORDER BY completed_at DESC
+LIMIT 5;
+SQL_EOF
+)
+
+    if [ -n "$SUCCESSFUL" ]; then
+        # Extract common patterns from successful executions
+        RECOVERY_STRATEGY="Successful $TYPE agents: used similar tool chains with validation steps"
+
+        # Update failure pattern with recovery strategy
+        sqlite3 "$DB_PATH" <<SQL_EOF
+UPDATE agent_failure_patterns
+SET
+    recovery_strategy = '$RECOVERY_STRATEGY',
+    resolution_rate = 0.50
+WHERE agent_type = '$TYPE' AND failure_mode = '$MODE';
+SQL_EOF
+
+        RECOVERIES_FOUND=$((RECOVERIES_FOUND + 1))
+        echo "  - Found recovery for: $TYPE / $MODE" | tee -a "$LOG_PATH"
+    fi
+done
+
+echo "[RuVector] Identified $RECOVERIES_FOUND recovery strategies" | tee -a "$LOG_PATH"
+
+# ============================================================================
+# Summary Report
+# ============================================================================
+
+echo "" | tee -a "$LOG_PATH"
+echo "[RuVector] Failure Analysis Summary:" | tee -a "$LOG_PATH"
+echo "  - Analyzed: $FAILURE_COUNT failed agents" | tee -a "$LOG_PATH"
+echo "  - New patterns: $PATTERNS_CREATED" | tee -a "$LOG_PATH"
+echo "  - Recovery strategies: $RECOVERIES_FOUND" | tee -a "$LOG_PATH"
+echo "" | tee -a "$LOG_PATH"
+
+exit 0