claude-flow-novice 2.18.13 → 2.18.14

package/.claude/skills/cfn-local-ruvector-accelerator/AGENT_INTEGRATION_PATTERNS.md

@@ -0,0 +1,255 @@
+# Agent Integration Patterns for AST-Aware RuVector
+
+This document describes how agents can integrate with the AST-aware RuVector indexer to achieve sub-50ms query performance for code intelligence tasks.
+
+## Overview
+
+The AST-Aware RuVector Accelerator provides:
+- Entity-based indexing (functions, structs, traits, etc.)
+- Reference tracking (calls, imports, type usage)
+- Vector embeddings for semantic search
+- Structured query interface
+
+## Integration Patterns
+
+### 1. Agent Setup
+
+```bash
+# Initialize RuVector in the project
+./target/release/local-ruvector init
+
+# Index the codebase
+./target/release/local-ruvector index --path . --types rs,ts,tsx,js,jsx --force
+```
+
+### 2. Query Patterns
+
+#### A. Find Functions Using a Type
+
+```bash
+# Find all functions that use the `Album` type
+./target/release/local-ruvector find --kind function --uses-type Album
+
+# Example SQL query behind the scenes:
+SELECT e.* FROM entities e
+JOIN type_usage tu ON e.id = tu.entity_id
+WHERE e.kind = 'function' AND tu.type_name = 'Album';
+```
+
+#### B. Find Callers of a Function
+
+```bash
+# Find all functions that call `create_album`
+./target/release/local-ruvector refs --target create_album --kind calls
+
+# Exclude calls within the same module
+./target/release/local-ruvector refs --target create_album --kind calls --exclude-module src/album/
+
+# Example SQL query:
+SELECT r.* FROM refs r
+JOIN entities e ON r.target_entity_id = e.id
+WHERE e.name = 'create_album' AND r.ref_kind = 'call'
+  AND r.file_path NOT LIKE '%src/album/%';
+```
+
+#### C. Refactoring Workflow
+
+```bash
+# Step 1: Find all references to a function
+./target/release/local-ruvector refs --target function_name
+
+# Step 2: Find all implementations of a trait
+./target/release/local-ruvector find --kind impl --implements Trait
+
+# Step 3: Find types used in a module
+./target/release/local-ruvector find --file-path src/module/ --kind struct
+
+# Step 4: Verify no breaks after refactoring
+./target/release/local-ruvector query "broken reference error" --threshold 0.9
+```
+
+### 3. Programmatic Integration
+
+Agents can integrate using the Rust API directly:
+
+```rust
+use local_ruvector::store_v2::StoreV2;
+use local_ruvector::query_api::QueryEngine;
+
+let store = StoreV2::new(&db_path)?;
+let query_engine = QueryEngine::new(store);
+
+// Find functions using a type
+let functions = query_engine.find_functions_using_type("Album")?;
+
+// Find callers of a function
+let callers = query_engine.find_callers("create_album", Some("src/other/"))?;
+
+// Search by semantic similarity
+let results = query_engine.semantic_search("database transaction", 10)?;
+```
+
+### 4. Performance Optimization Tips
+
+#### A. Use Database Indexes
+The schema includes optimized indexes:
+- `idx_entities_kind_name` for fast entity lookups
+- `idx_type_usage_type_name` for type usage queries
+- `idx_refs_target_kind` for reference queries
+
+#### B. Batch Operations
+```rust
+// Batch insert entities
+let entities: Vec<Entity> = vec![...];
+store.insert_entities_batch(entities)?;
+
+// Batch query multiple entities
+let ids = vec![1, 2, 3, 4, 5];
+let entities = store.get_entity_batch(&ids)?;
+```
+
+#### C. Query Result Caching
+```rust
+use std::time::Duration;
+use cached::proc_macro::cached;
+
+#[cached(size = 1000, time = 300)]
+pub fn find_functions_using_type(type_name: &str) -> Result<Vec<Entity>> {
+    // Query implementation
+}
+```
+
+## Agent Workflow Examples
+
+### 1. Code Review Agent
+
+```rust
+pub async fn review_pull_request(pr_id: i32) -> Result<ReviewResult> {
+    // 1. Get changed files
+    let changed_files = get_pr_files(pr_id).await?;
+
+    // 2. Query for potentially problematic patterns
+    let issues = query_engine.search_patterns([
+        "TODO:",
+        "FIXME:",
+        "unwrap()",
+        "panic!",
+        "expect(",
+    ])?;
+
+    // 3. Check for breaking changes
+    let public_api_changes = query_engine.find_public_api_changes(&changed_files)?;
+
+    // 4. Verify imports are correct
+    let unused_imports = query_engine.find_unused_imports(&changed_files)?;
+
+    Ok(ReviewResult { issues, public_api_changes, unused_imports })
+}
+```
+
+### 2. Refactoring Agent
+
+```rust
+pub async fn extract_function(
+    file_path: &str,
+    start_line: usize,
+    end_line: usize,
+    function_name: &str,
+) -> Result<RefactorResult> {
+    // 1. Analyze the selected code
+    let entities = query_engine.find_entities_in_range(file_path, start_line, end_line)?;
+
+    // 2. Find all external dependencies
+    let dependencies = query_engine.find_dependencies(&entities)?;
+
+    // 3. Check if extraction is safe
+    let safe_to_extract = query_engine.verify_extraction_safety(&entities)?;
+
+    if safe_to_extract {
+        // 4. Perform refactoring
+        let new_content = extract_function_to_module(file_path, start_line, end_line, function_name)?;
+
+        // 5. Update imports in dependent files
+        for dep in dependencies {
+            update_imports(&dep.file_path, function_name)?;
+        }
+
+        Ok(RefactorResult::Success)
+    } else {
+        Ok(RefactorResult::Unsafe)
+    }
+}
+```
+
+### 3. Documentation Agent
+
+```rust
+pub async fn generate_documentation(entity_name: &str) -> Result<Documentation> {
+    // 1. Find the entity
+    let entity = query_engine.find_entity_by_name(entity_name)?;
+
+    // 2. Get all related documentation
+    let related_docs = query_engine.find_related_documentation(&entity)?;
+
+    // 3. Find usage examples
+    let examples = query_engine.find_usage_examples(&entity)?;
+
+    // 4. Check for undocumented public APIs
+    if entity.visibility == Visibility::Public && entity.doc_comment.is_none() {
+        return Err(anyhow!("Public entity lacks documentation"));
+    }
+
+    // 5. Generate comprehensive docs
+    Ok(Documentation {
+        entity,
+        related_docs,
+        examples,
+    })
+}
+```
+
+## Performance Benchmarks
+
+Based on the current implementation:
+
+- **Index Size**: 175MB for 31 Rust files with 5,897 embeddings
+- **Index Time**: ~90 seconds for full reindex
+- **Query Performance**: Target <50ms for indexed queries
+- **Entity Coverage**: Functions, structs, traits, impls, enums
+
+## Future Enhancements
+
+1. **Incremental Updates**: Only reindex changed files
+2. **Cross-Language Support**: TypeScript, JavaScript support
+3. **Enhanced Query API**: More complex query builders
+4. **Integration with LLMs**: Use embeddings for AI-assisted coding
+5. **Real-time Updates**: File watching for automatic reindexing
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Compilation Errors**: Ensure tree-sitter parsers are properly linked
+2. **Database Locks**: Use WAL mode for concurrent access
+3. **Memory Usage**: Limit batch sizes for large codebases
+4. **Slow Queries**: Check EXPLAIN QUERY PLAN and add indexes
+
+### Debug Commands
+
+```bash
+# Check database health
+./target/release/local-ruvector stats --detailed
+
+# Rebuild index
+./target/release/local-ruvector index --force
+
+# Query optimization
+sqlite3 .ruvector/index.db "EXPLAIN QUERY PLAN SELECT ..."
+
+# Check embeddings
+./target/release/local-ruvector query "test query" --format json
+```
+
+## Conclusion
+
+The AST-Aware RuVector Accelerator provides a powerful foundation for agent-driven code intelligence. By leveraging entity-based indexing and structured queries, agents can achieve sub-50ms response times for complex code analysis tasks.

package/.claude/skills/cfn-local-ruvector-accelerator/Cargo.toml

@@ -37,4 +37,11 @@ chrono = { version = "0.4", features = ["serde"] }
 # File operations
 fs_extra = "1.3"
 # Environment
-dotenvy = "0.15"
+dotenvy = "0.15"
+# Tree-sitter for AST parsing
+# Tree-sitter for AST parsing
+
+# Tree-sitter for AST parsing
+tree-sitter = "0.20"
+tree-sitter-rust = "0.20"
+tree-sitter-typescript = "0.20"

package/.claude/skills/cfn-local-ruvector-accelerator/Cargo.toml.backup

@@ -0,0 +1,47 @@
+[package]
+name = "local-ruvector"
+version = "0.1.0"
+edition = "2021"
+
+[dependencies]
+# Database
+rusqlite = { version = "0.31", features = ["bundled", "vtab"] }
+# Vector operations
+ndarray = { version = "0.15", features = ["rayon"], default-features = false }
+# Serialization
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+# CLI
+clap = { version = "4.4", features = ["derive"] }
+anyhow = "1.0"
+# File system
+walkdir = "2.4"
+# Hashing
+sha2 = "0.10"
+# Async (for embedding API)
+tokio = { version = "1.0", features = ["full"] }
+reqwest = { version = "0.11", features = ["json"] }
+# Logging
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["json", "env-filter"] }
+# Parallel processing
+rayon = "1.8"
+# Memory mapping
+memmap2 = "0.9"
+# Binary serialization
+byteorder = "1.5"
+# Regular expressions
+regex = "1.10"
+# Date/time
+chrono = { version = "0.4", features = ["serde"] }
+# File operations
+fs_extra = "1.3"
+# Environment
+dotenvy = "0.15"
+# Tree-sitter for AST parsing
+# Tree-sitter for AST parsing
+
+# Tree-sitter for AST parsing
+tree-sitter = "0.20"
+tree-sitter-rust = "0.20"
+tree-sitter-typescript = "0.20"

package/.claude/skills/cfn-local-ruvector-accelerator/EXTRACTION_EXAMPLES.md

@@ -0,0 +1,287 @@
+# RuVector Extraction Examples
+
+This document demonstrates what the TypeScript/JavaScript and text-based fallback extractors capture from different file types.
+
+## TypeScript Extraction Example
+
+### Input File: `hello.ts`
+
+```typescript
+/**
+ * Simple hello module
+ */
+
+export function sayHello(name: string): void {
+  console.log(`Hello, ${name}!`);
+}
+
+export interface Greeter {
+  greet(name: string): void;
+}
+
+export class SimpleGreeter implements Greeter {
+  greet(name: string): void {
+    sayHello(name);
+  }
+}
+
+export type GreetingFunction = (name: string) => void;
+
+import { Logger } from './logger';
+```
+
+### Extracted Entities
+
+| Name | Kind | Line | Visibility | Metadata |
+|------|------|------|------------|----------|
+| `sayHello` | Function | 6 | Public | exported: true |
+| `Greeter` | Interface | 10 | Public | exported: true |
+| `SimpleGreeter` | Class | 14 | Public | exported: true, implements: Greeter |
+| `GreetingFunction` | TypeAlias | 22 | Public | exported: true, type_definition: "(name: string) => void" |
+
+### Extracted References
+
+| Target | Type | File | Line |
+|--------|------|------|------|
+| `Logger` | Import | ./logger | 24 |
+
+## JSON Extraction Example
+
+### Input File: `package.json`
+
+```json
+{
+  "name": "hello-world",
+  "version": "1.0.0",
+  "description": "Simple greeting library",
+  "main": "dist/hello.js",
+  "scripts": {
+    "build": "tsc",
+    "test": "jest"
+  },
+  "dependencies": {
+    "typescript": "^5.0.0",
+    "jest": "^29.0.0"
+  }
+}
+```
+
+### Extracted Entities
+
+| Name | Kind | Data Type | Metadata |
+|------|------|-----------|----------|
+| `name` | Variable | string | chunk_type: json |
+| `version` | Variable | string | chunk_type: json |
+| `description` | Variable | string | chunk_type: json |
+| `main` | Variable | string | chunk_type: json |
+| `scripts` | Variable | object | chunk_type: json |
+| `dependencies` | Variable | object | chunk_type: json |
+
+**Benefit**: All top-level keys are indexed for semantic search, enabling queries like "find configuration with name hello-world" or "locate all JSON files with dependencies".
+
+## YAML Extraction Example
+
+### Input File: `config.yaml`
+
+```yaml
+version: 1.0.0
+name: hello-app
+environments:
+  development:
+    debug: true
+    port: 3000
+  production:
+    debug: false
+    port: 8080
+features:
+  logging: enabled
+  caching: enabled
+```
+
+### Extracted Entities
+
+| Name | Line | Indent Level | Metadata |
+|------|------|--------------|----------|
+| `version` | 1 | 0 | chunk_type: yaml |
+| `name` | 2 | 0 | chunk_type: yaml |
+| `environments` | 3 | 0 | chunk_type: yaml |
+| `development` | 4 | 1 | chunk_type: yaml |
+| `debug` | 5 | 2 | chunk_type: yaml |
+| `port` | 6 | 2 | chunk_type: yaml |
+| `production` | 7 | 1 | chunk_type: yaml |
+| `debug` | 8 | 2 | chunk_type: yaml |
+| `port` | 9 | 2 | chunk_type: yaml |
+| `features` | 10 | 0 | chunk_type: yaml |
+| `logging` | 11 | 1 | chunk_type: yaml |
+| `caching` | 12 | 1 | chunk_type: yaml |
+
+**Benefit**: Configuration structure is indexed with indentation levels, enabling hierarchical searches and environment-specific queries.
+
+## Markdown Extraction Example
+
+### Input File: `README.md`
+
+```markdown
+# Getting Started
+
+This guide helps you set up the project.
+
+## Installation
+
+```bash
+npm install
+npm run build
+```
+
+## Configuration
+
+Edit the config.yaml file to customize settings.
+
+### Advanced Setup
+
+For advanced users, use environment variables.
+```
+
+### Extracted Entities
+
+| Name | Type | Line | Metadata |
+|------|------|------|----------|
+| `Getting Started` | Module | 1 | chunk_type: markdown, heading_level: 1 |
+| `Installation` | Namespace | 5 | chunk_type: markdown, heading_level: 2 |
+| `code_bash` | Variable | 7 | chunk_type: markdown_code, language: bash |
+| `Configuration` | Namespace | 12 | chunk_type: markdown, heading_level: 2 |
+| `Advanced Setup` | Namespace | 15 | chunk_type: markdown, heading_level: 3 |
+
+**Benefit**: Documentation structure is preserved, enabling navigation and contextual search across documentation files.
+
+## Shell Script Extraction Example
+
+### Input File: `setup.sh`
+
+```bash
+#!/bin/bash
+
+DATABASE_URL="postgres://localhost:5432/mydb"
+API_KEY="secret-key-123"
+DEBUG_MODE=true
+
+function setup_environment() {
+  echo "Setting up environment..."
+  export DATABASE_URL
+  export API_KEY
+}
+
+function cleanup() {
+  rm -rf /tmp/cache
+  unset DEBUG_MODE
+}
+
+setup_environment
+```
+
+### Extracted Entities
+
+| Name | Kind | Type | Line |
+|------|------|------|------|
+| `DATABASE_URL` | Variable | shell_variable | 3 |
+| `API_KEY` | Variable | shell_variable | 4 |
+| `DEBUG_MODE` | Variable | shell_variable | 5 |
+| `setup_environment` | Function | shell_function | 7 |
+| `cleanup` | Function | shell_function | 12 |
+
+**Benefit**: Shell script structure is indexed, enabling queries like "find all setup functions" or "locate database configuration variables".
+
+## Extraction Statistics
+
+### Coverage by File Type
+
+| File Type | Extractor | Entity Types | References | Support Level |
+|-----------|-----------|--------------|------------|----------------|
+| `.ts, .tsx, .js, .jsx, .mjs, .cjs` | TypeScript | Functions, Classes, Interfaces, Types | Imports | Full |
+| `.json` | JSON | Top-level keys | None | Full |
+| `.yaml, .yml` | YAML | Keys (hierarchical) | None | Full |
+| `.md, .markdown` | Markdown | Headings, Code Blocks | None | Full |
+| `.sh, .bash` | Shell | Functions, Variables | None | Full |
+| `.rs` | Rust | Functions, Structs, Traits, Imports | References | Full |
+| Others | Text Fallback | Meaningful text chunks | None | Partial |
+
+## Indexing Results
+
+When you run `local-ruvector index`, the system:
+
+1. **Scans** all files in the project
+2. **Routes** files to appropriate extractors based on extension
+3. **Extracts** entities and references using language-specific logic
+4. **Generates** embeddings for all extracted entities
+5. **Stores** results in SQLite database with vector search capability
+
+### Example Output
+
+```
+Starting index process
+File types: [".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs", ".rs", ".json", ".yaml", ".yml", ".md", ".markdown", ".sh", ".bash", ".txt", ".config", ".conf", ".env"]
+
+Processing 1000 files...
+✓ hello.ts: 4 entities, 1 reference, 4 embeddings
+✓ package.json: 6 entities, 0 references, 6 embeddings
+✓ config.yaml: 11 entities, 0 references, 11 embeddings
+✓ setup.sh: 5 entities, 0 references, 5 embeddings
+...
+
+Index complete: 1000 files processed
+- Entities extracted: 12,345
+- References extracted: 2,108
+- Embeddings generated: 12,345
+```
+
+## Semantic Search Capabilities
+
+After indexing, you can search for:
+
+### TypeScript/JavaScript Specific
+- Functions: "Find all exported functions that handle authentication"
+- Classes: "Locate all classes that implement Observable"
+- Interfaces: "Find all interfaces defined in the API layer"
+- Types: "Search for union types used in state management"
+- Imports: "Show all files importing from react"
+
+### JSON Configuration
+- "Find all configuration files with database settings"
+- "Locate package.json files with specific dependencies"
+- "Search for API endpoint definitions"
+
+### YAML Configuration
+- "Find all environment-specific configurations"
+- "Locate service definitions in docker-compose files"
+- "Search for feature flag configurations"
+
+### Markdown Documentation
+- "Find installation guides"
+- "Locate API documentation sections"
+- "Search for code examples in specific languages"
+
+### Shell Scripts
+- "Find all setup functions"
+- "Locate scripts with specific environment variables"
+- "Search for database initialization scripts"
+
+## Performance Notes
+
+- **TypeScript files**: <1ms per file (regex-based)
+- **JSON files**: <0.5ms per file (native parsing)
+- **YAML files**: <1ms per file (line-based)
+- **Markdown files**: <1ms per file (pattern-based)
+- **Shell files**: <1ms per file (regex-based)
+
+Complete project indexing (10,000 files): ~30-50 seconds depending on file types and complexity.
+
+## Next Steps
+
+After indexing:
+
+1. **Query** using `local-ruvector query`
+2. **Export** results using `local-ruvector export`
+3. **Analyze** dependencies using `local-ruvector refs`
+4. **Monitor** with `local-ruvector stats`
+
+See the main README for detailed usage instructions.