claude-flow-novice 2.10.5 → 2.10.6

package/claude-assets/skills/cfn-analytics/README.md

@@ -0,0 +1,58 @@
+# Skill Invocation Logging Infrastructure
+
+## Overview
+This infrastructure captures and analyzes skill invocation metrics for AI agent performance tracking.
+
+## Components
+1. `skill-invocations.sql`: SQLite schema for logging skill invocations
+2. `log-skill-invocation.js`: Script to log individual skill invocations
+3. `skill-analytics-dashboard.js`: Generate performance analytics reports
+4. `test-data-generator.js`: Create sample test data for analysis
+5. `skill-invocation-hook.sh`: Bash wrapper for logging skill invocations
+
+## Usage
+
+### Logging a Skill Invocation
+```bash
+node log-skill-invocation.js \
+    "skill_name" \
+    "user_prompt" \
+    "outcome" \
+    input_tokens \
+    output_tokens \
+    confidence_score \
+    context_reduction_percentage
+```
+
+### Generate Analytics Report
+```bash
+node skill-analytics-dashboard.js
+```
+
+### Generate Test Data
+```bash
+node test-data-generator.js
+```
+
+## Key Metrics Tracked
+- Skill invocation success rate
+- Average confidence score
+- Context reduction percentage
+- Token usage
+
+## Redis Notifications
+Publishes to `swarm:analytics:skill-invoked` with:
+- Skill name
+- Outcome
+- Timestamp
+
+## Recommended Workflow
+1. Invoke skill with logging
+2. Publish Redis notification
+3. Periodically run analytics dashboard
+4. Use report for skill refinement
+
+## Performance Considerations
+- WAL journal mode for SQLite
+- Asynchronous logging
+- Hashed user prompts for privacy

package/claude-assets/skills/cfn-analytics/description-refinement-guide.md

@@ -0,0 +1,164 @@
+# Skill Description Refinement Guide
+
+## Objective
+Establish best practices for creating high-accuracy skill descriptions that enable precise skill selection through automated classification.
+
+## Refinement Patterns
+
+### 1. Keyword Selection
+- Focus on unique, domain-specific terms
+- Include technical jargon and specific methodologies
+- Prioritize terms that clearly distinguish the skill
+- Aim for 6-10 keywords per skill
+- Use lowercase, hyphen-separated terms
+
+**Good Keywords Examples:**
+- "multi-agent initialization"
+- "real-time pub/sub"
+- "5-level ACL"
+- "coordinated test execution"
+
+**Anti-Pattern Keywords:**
+- Generic terms like "system", "management"
+- Overly broad descriptors
+- Marketing language
+
+### 2. Trigger Selection
+- Identify specific scenarios that require the skill
+- Use concise, action-oriented phrases
+- Represent real-world problem domains
+- Focus on pain points the skill solves
+
+**Good Triggers:**
+- "preventing test conflicts"
+- "secure data storage needs"
+- "complex system architecture validation"
+
+**Anti-Pattern Triggers:**
+- Vague descriptions
+- Overly generic scenarios
+- Non-actionable statements
+
+### 3. Performance Targets
+- Include 3-4 quantitative metrics
+- Use standard units (ms, pct, bits)
+- Represent key performance dimensions
+- Be specific and measurable
+
+**Good Performance Targets:**
+- "query_time_ms": 20
+- "consensus_accuracy": 90
+- "spawn_time_ms": 200
+- "max_concurrent_agents": 15
+
+### 4. Frontmatter Structure
+```yaml
+---
+name: Skill Name
+version: X.Y.Z
+complexity: [Low|Medium|High]
+keywords: [
+    "term1",
+    "term2",
+    ...
+]
+triggers: [
+    "scenario1",
+    "scenario2",
+    ...
+]
+performance_targets: {
+    "metric1": value,
+    "metric2": value,
+    ...
+}
+---
+```
+
+## Validation Strategy
+
+### Accuracy Calculation
+1. Prompt matching percentage
+2. Keyword coverage
+3. Trigger relevance score
+4. Performance target precision
+
+### Target Metrics
+- Overall Accuracy: ≥95%
+- Skill-Specific Accuracy:
+  - Lowest acceptable: ≥90%
+  - Ideal target: ≥97%
+
+### Continuous Improvement
+- Quarterly review of skill descriptions
+- Machine learning model refinement
+- Periodic re-validation with expanded test corpus
+
+## Maintenance Guidelines
+- Update with each major version
+- Reflect emerging technology trends
+- Incorporate feedback from actual usage
+- Maintain consistency across skills
+
+## Example Refinement Process
+
+### Before Refinement
+```
+keywords: ["system", "management", "coordination"]
+```
+
+### After Refinement
+```
+keywords: [
+    "multi-agent initialization",
+    "dynamic topology",
+    "dependency resolution",
+    "resource optimization"
+]
+```
+
+## Tool Integration
+Use `validate-skill-selection.js` for:
+- Automated description validation
+- Accuracy reporting
+- Suggestions for improvement
+
+## References
+- Current Test Corpus: `.claude/skills/analytics/test-corpus.json`
+- Validation Script: `.claude/skills/analytics/validate-skill-selection.js`
+- Accuracy Report: `.artifacts/analytics/skill-description-accuracy.json`
+
+## Lessons Learned in Skill Description Refinement
+
+### Keyword Strategy
+1. **Granularity**: Use 6-10 keywords that represent specific, actionable aspects
+2. **Context Alignment**: Ensure keywords match real-world problem statements
+3. **Token-Based Matching**: Break keywords into meaningful tokens
+4. **Semantic Proximity**: Include related terms and variations
+
+### Validation Insights
+- Exact matching provides limited accuracy
+- Partial token matching increases precision
+- Combine keywords and triggers for comprehensive matching
+- Target accuracy ranges:
+  - Overall: 70-90%
+  - Per Skill: 60-85%
+
+### Continuous Improvement
+- Quarterly review of skill descriptions
+- Expand test corpus with diverse scenarios
+- Machine learning-assisted keyword suggestion
+- Track evolution of skill descriptions over time
+
+### Anti-Patterns to Avoid
+- Generic, overused terminology
+- Marketing language
+- Overly broad descriptions
+- Lack of specificity
+- Keywords that don't reflect actual implementation
+
+### Future Research
+- Develop ML model for automatic keyword generation
+- Create dynamic skill description generator
+- Build comprehensive taxonomy of skill descriptions
+- Develop cross-referencing mechanism for skill selection

package/claude-assets/skills/cfn-analytics/log-skill-invocation.js

@@ -0,0 +1,122 @@
+#!/usr/bin/env node
+const sqlite3 = require('sqlite3').verbose();
+const redis = require('redis');
+const crypto = require('crypto');
+const path = require('path');
+
+class SkillInvocationLogger {
+    constructor(dbPath, redisConfig) {
+        this.dbPath = dbPath || path.join(__dirname, '../../../.artifacts/analytics/skill-invocations.sqlite');
+        this.redisClient = redis.createClient(redisConfig);
+    }
+
+    async initialize() {
+        this.db = new sqlite3.Database(this.dbPath);
+        return new Promise((resolve, reject) => {
+            this.db.run(`
+                PRAGMA journal_mode = WAL;
+                PRAGMA synchronous = NORMAL;
+                PRAGMA cache_size = -2000;
+            `, (err) => {
+                if (err) reject(err);
+                else resolve(this);
+            });
+        });
+    }
+
+    hashPrompt(prompt) {
+        return crypto.createHash('sha256').update(prompt).digest('hex');
+    }
+
+    async logSkillInvocation(params) {
+        const {
+            skill_name,
+            user_prompt,
+            outcome,
+            input_tokens,
+            output_tokens,
+            confidence_score,
+            context_reduction_percentage
+        } = params;
+
+        const user_prompt_hash = this.hashPrompt(user_prompt);
+
+        return new Promise((resolve, reject) => {
+            this.db.run(
+                `INSERT INTO skill_invocations (
+                    skill_name,
+                    user_prompt_hash,
+                    outcome,
+                    input_tokens,
+                    output_tokens,
+                    confidence_score,
+                    context_reduction_percentage
+                ) VALUES (?, ?, ?, ?, ?, ?, ?)`,
+                [
+                    skill_name,
+                    user_prompt_hash,
+                    outcome,
+                    input_tokens,
+                    output_tokens,
+                    confidence_score,
+                    context_reduction_percentage
+                ],
+                (err) => {
+                    if (err) reject(err);
+                    else {
+                        this.publishRedisNotification(skill_name, outcome);
+                        resolve();
+                    }
+                }
+            );
+        });
+    }
+
+    publishRedisNotification(skill_name, outcome) {
+        this.redisClient.publish(
+            'swarm:analytics:skill-invoked',
+            JSON.stringify({ skill_name, outcome, timestamp: new Date().toISOString() })
+        );
+    }
+
+    async close() {
+        return new Promise((resolve, reject) => {
+            this.db.close((err) => {
+                if (err) reject(err);
+                else {
+                    this.redisClient.quit();
+                    resolve();
+                }
+            });
+        });
+    }
+
+    static async run(params) {
+        const logger = await new SkillInvocationLogger().initialize();
+        try {
+            await logger.logSkillInvocation(params);
+            await logger.close();
+        } catch (error) {
+            console.error('Skill Invocation Logging Error:', error);
+            await logger.close();
+            process.exit(1);
+        }
+    }
+}
+
+// Allow direct script execution with params
+if (require.main === module) {
+    const params = {
+        skill_name: process.argv[2],
+        user_prompt: process.argv[3],
+        outcome: process.argv[4],
+        input_tokens: parseInt(process.argv[5], 10) || 0,
+        output_tokens: parseInt(process.argv[6], 10) || 0,
+        confidence_score: parseFloat(process.argv[7]) || 0.0,
+        context_reduction_percentage: parseFloat(process.argv[8]) || 0.0
+    };
+
+    SkillInvocationLogger.run(params);
+}
+
+module.exports = SkillInvocationLogger;

package/claude-assets/skills/cfn-analytics/run-production-criteria-tests.sh

@@ -0,0 +1,126 @@
+#!/bin/bash
+set -e
+
+echo "═══════════════════════════════════════════════════════════"
+echo "Production Criteria Test Suite"
+echo "Claude Code Skills Integration - Epic Validation"
+echo "═══════════════════════════════════════════════════════════"
+echo ""
+
+RESULTS_DIR=".artifacts/analytics"
+mkdir -p $RESULTS_DIR
+
+PASSED=0
+FAILED=0
+declare -a TEST_RESULTS
+
+run_test() {
+  local test_name=$1
+  local test_command=$2
+
+  echo ""
+  echo "──────────────────────────────────────"
+  echo "TEST: $test_name"
+  echo "──────────────────────────────────────"
+
+  if eval "$test_command"; then
+    echo "✅ PASSED: $test_name"
+    PASSED=$((PASSED + 1))
+    TEST_RESULTS+=("PASS:$test_name")
+  else
+    echo "❌ FAILED: $test_name"
+    FAILED=$((FAILED + 1))
+    TEST_RESULTS+=("FAIL:$test_name")
+  fi
+}
+
+# Run all 4 production criteria tests
+run_test "Manual Override Rate <5%" \
+  "node .claude/skills/cfn-analytics/test-manual-override-rate.js"
+
+run_test "Zero Concurrent Test Conflicts" \
+  ".claude/skills/cfn-test-execution/test-concurrent-conflicts.sh"
+
+run_test "100% ROOT_WARNING Auto-Resolution" \
+  ".claude/skills/cfn-hook-pipeline/test-root-warning-resolution.sh"
+
+run_test "Agents Persist State Autonomously" \
+  "node .claude/skills/cfn-sqlite-memory/test-state-persistence.js"
+
+# Calculate success rate
+TOTAL_TESTS=$((PASSED + FAILED))
+SUCCESS_RATE=$(echo "scale=2; ($PASSED / $TOTAL_TESTS) * 100" | bc)
+
+# Generate summary report
+echo ""
+echo "═══════════════════════════════════════════════════════════"
+echo "TEST SUITE SUMMARY"
+echo "═══════════════════════════════════════════════════════════"
+echo "Total Tests: $TOTAL_TESTS"
+echo "Passed: $PASSED"
+echo "Failed: $FAILED"
+echo "Success Rate: ${SUCCESS_RATE}%"
+echo ""
+
+# Individual test results
+echo "Individual Test Results:"
+for result in "${TEST_RESULTS[@]}"; do
+  IFS=':' read -r status test_name <<< "$result"
+  if [ "$status" = "PASS" ]; then
+    echo "  ✅ $test_name"
+  else
+    echo "  ❌ $test_name"
+  fi
+done
+echo ""
+
+# Generate JSON report
+cat > $RESULTS_DIR/production-criteria-summary.json << REPORT
+{
+  "test_suite": "production_criteria_validation",
+  "epic": "claude-code-skills-integration",
+  "total_tests": $TOTAL_TESTS,
+  "passed": $PASSED,
+  "failed": $FAILED,
+  "success_rate": $SUCCESS_RATE,
+  "tests": [
+    {
+      "name": "Manual Override Rate",
+      "result_file": "$RESULTS_DIR/manual-override-test.json",
+      "status": "$(echo "${TEST_RESULTS[0]}" | cut -d: -f1)"
+    },
+    {
+      "name": "Concurrent Test Conflicts",
+      "result_file": "$RESULTS_DIR/test-conflict-results.json",
+      "status": "$(echo "${TEST_RESULTS[1]}" | cut -d: -f1)"
+    },
+    {
+      "name": "ROOT_WARNING Auto-Resolution",
+      "result_file": "$RESULTS_DIR/root-warning-test.json",
+      "status": "$(echo "${TEST_RESULTS[2]}" | cut -d: -f1)"
+    },
+    {
+      "name": "State Persistence",
+      "result_file": "$RESULTS_DIR/state-persistence-test.json",
+      "status": "$(echo "${TEST_RESULTS[3]}" | cut -d: -f1)"
+    }
+  ],
+  "test_date": "$(date -Iseconds)",
+  "test_duration_seconds": $SECONDS
+}
+REPORT
+
+echo "📄 Summary report saved to: $RESULTS_DIR/production-criteria-summary.json"
+echo ""
+
+if [ $FAILED -eq 0 ]; then
+  echo "✅ ALL PRODUCTION CRITERIA TESTS PASSED ($PASSED/$TOTAL_TESTS)"
+  echo ""
+  echo "🚀 Epic is ready for production deployment!"
+  exit 0
+else
+  echo "❌ SOME PRODUCTION CRITERIA TESTS FAILED ($FAILED/$TOTAL_TESTS)"
+  echo ""
+  echo "📋 Review individual test results for details"
+  exit 1
+fi

package/claude-assets/skills/cfn-analytics/skill-analytics-dashboard.js

@@ -0,0 +1,113 @@
+#!/usr/bin/env node
+const sqlite3 = require('sqlite3').verbose();
+const path = require('path');
+const fs = require('fs').promises;
+
+class SkillAnalyticsDashboard {
+    constructor(dbPath, reportPath) {
+        this.dbPath = dbPath || path.join(__dirname, '../../../.artifacts/analytics/skill-invocations.sqlite');
+        this.reportPath = reportPath || path.join(__dirname, '../../../.artifacts/analytics/skill-invocation-report.json');
+    }
+
+    async connect() {
+        return new Promise((resolve, reject) => {
+            this.db = new sqlite3.Database(this.dbPath, (err) => {
+                if (err) reject(err);
+                else resolve(this);
+            });
+        });
+    }
+
+    async generateSkillPerformanceReport() {
+        return new Promise((resolve, reject) => {
+            this.db.all(`
+                SELECT
+                    skill_name,
+                    total_invocations,
+                    successful_invocations,
+                    ROUND(successful_invocations * 100.0 / total_invocations, 2) as success_rate,
+                    ROUND(average_confidence, 2) as average_confidence
+                FROM skill_accuracy_summary
+                ORDER BY success_rate DESC
+            `, async (err, rows) => {
+                if (err) reject(err);
+
+                const contextReductionReport = await this.getContextReductionAnalysis();
+                const report = {
+                    timestamp: new Date().toISOString(),
+                    skill_performance: rows,
+                    context_reduction: contextReductionReport,
+                    recommendations: this.generateRecommendations(rows)
+                };
+
+                await this.writeReportToFile(report);
+                resolve(report);
+            });
+        });
+    }
+
+    async getContextReductionAnalysis() {
+        return new Promise((resolve, reject) => {
+            this.db.all(`
+                SELECT
+                    skill_name,
+                    ROUND(AVG(reduction_percentage), 2) as avg_reduction,
+                    ROUND(MAX(reduction_percentage), 2) as max_reduction
+                FROM context_reduction_metrics
+                GROUP BY skill_name
+                ORDER BY avg_reduction DESC
+            `, (err, rows) => {
+                if (err) reject(err);
+                else resolve(rows);
+            });
+        });
+    }
+
+    generateRecommendations(performanceData) {
+        const recommendations = [];
+        performanceData.forEach(skill => {
+            if (skill.success_rate < 70) {
+                recommendations.push(`Low performance for ${skill.skill_name}: Consider skill refinement`);
+            }
+            if (skill.average_confidence < 0.7) {
+                recommendations.push(`Low confidence for ${skill.skill_name}: Review skill implementation`);
+            }
+        });
+        return recommendations;
+    }
+
+    async writeReportToFile(report) {
+        await fs.mkdir(path.dirname(this.reportPath), { recursive: true });
+        await fs.writeFile(this.reportPath, JSON.stringify(report, null, 2));
+    }
+
+    async close() {
+        return new Promise((resolve, reject) => {
+            this.db.close((err) => {
+                if (err) reject(err);
+                else resolve();
+            });
+        });
+    }
+
+    static async run() {
+        const dashboard = await new SkillAnalyticsDashboard().connect();
+        try {
+            const report = await dashboard.generateSkillPerformanceReport();
+            console.log('Skill Analytics Report Generated');
+            await dashboard.close();
+            return report;
+        } catch (error) {
+            console.error('Skill Analytics Dashboard Error:', error);
+            await dashboard.close();
+            process.exit(1);
+        }
+    }
+}
+
+// Allow direct script execution
+if (require.main === module) {
+    SkillAnalyticsDashboard.run();
+}
+
+module.exports = SkillAnalyticsDashboard;

package/claude-assets/skills/cfn-analytics/skill-invocation-hook.sh

@@ -0,0 +1,28 @@
+#!/bin/bash
+# Skill Invocation Logging Hook
+
+# Ensure required arguments are provided
+if [ $# -lt 5 ]; then
+    echo "Usage: $0 <skill_name> <user_prompt> <outcome> <input_tokens> <output_tokens> [confidence_score] [context_reduction_percentage]"
+    exit 1
+fi
+
+# Default values
+CONFIDENCE_SCORE=${6:-0.0}
+CONTEXT_REDUCTION=${7:-0.0}
+
+# Path to the logging script (adjust as needed)
+LOGGING_SCRIPT="/mnt/c/Users/masha/Documents/claude-flow-novice/.claude/skills/cfn-analytics/log-skill-invocation.js"
+
+# Execute logging script
+node "$LOGGING_SCRIPT" \
+    "$1" \
+    "$2" \
+    "$3" \
+    "$4" \
+    "$5" \
+    "$CONFIDENCE_SCORE" \
+    "$CONTEXT_REDUCTION_PERCENTAGE"
+
+# Exit with the script's return code
+exit $?

package/claude-assets/skills/cfn-analytics/skill-invocations.sql

@@ -0,0 +1,58 @@
+-- Skill Invocation Logging Schema
+-- ACL Level 4: Project-wide Visibility
+
+-- Skill Invocations Table
+CREATE TABLE skill_invocations (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    skill_name TEXT NOT NULL,
+    user_prompt_hash TEXT NOT NULL,
+    timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
+    outcome TEXT NOT NULL,
+    input_tokens INTEGER,
+    output_tokens INTEGER,
+    confidence_score REAL,
+    context_reduction_percentage REAL
+);
+
+-- Skill Accuracy Summary Table
+CREATE TABLE skill_accuracy_summary (
+    skill_name TEXT PRIMARY KEY,
+    total_invocations INTEGER DEFAULT 0,
+    successful_invocations INTEGER DEFAULT 0,
+    average_confidence REAL DEFAULT 0.0,
+    last_updated DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Context Reduction Metrics Table
+CREATE TABLE context_reduction_metrics (
+    skill_name TEXT,
+    initial_context_tokens INTEGER,
+    final_context_tokens INTEGER,
+    reduction_percentage REAL,
+    timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
+    PRIMARY KEY (skill_name, timestamp)
+);
+
+-- Indexes for Performance
+CREATE INDEX idx_skill_invocations_timestamp ON skill_invocations(timestamp);
+CREATE INDEX idx_skill_invocations_outcome ON skill_invocations(outcome);
+CREATE INDEX idx_skill_accuracy_summary_success_rate ON skill_accuracy_summary(successful_invocations * 1.0 / total_invocations);
+
+-- Triggers for Automatic Summary Updates
+CREATE TRIGGER update_skill_accuracy_summary
+AFTER INSERT ON skill_invocations
+BEGIN
+    INSERT OR REPLACE INTO skill_accuracy_summary (
+        skill_name,
+        total_invocations,
+        successful_invocations,
+        average_confidence,
+        last_updated
+    ) VALUES (
+        NEW.skill_name,
+        (SELECT COALESCE(total_invocations, 0) + 1 FROM skill_accuracy_summary WHERE skill_name = NEW.skill_name),
+        (SELECT COALESCE(successful_invocations, 0) + (CASE WHEN NEW.outcome = 'success' THEN 1 ELSE 0 END) FROM skill_accuracy_summary WHERE skill_name = NEW.skill_name),
+        (SELECT (COALESCE(average_confidence, 0) * COALESCE(total_invocations, 0) + NEW.confidence_score) / (COALESCE(total_invocations, 0) + 1) FROM skill_accuracy_summary WHERE skill_name = NEW.skill_name),
+        CURRENT_TIMESTAMP
+    );
+END;