claude-flow 2.7.0-alpha.13 → 2.7.0-alpha.14

package/.claude/skills/agentdb-vector-search/SKILL.md

@@ -7,38 +7,113 @@ description: "Implement semantic vector search with AgentDB for intelligent docu
 
 ## What This Skill Does
 
-Implements vector-based semantic search using AgentDB's high-performance vector database with 150x-12,500x faster operations than traditional solutions. Enables similarity search, hybrid search (vector + metadata), and real-time embedding generation.
+Implements vector-based semantic search using AgentDB's high-performance vector database with **150x-12,500x faster** operations than traditional solutions. Features HNSW indexing, quantization, and sub-millisecond search (<100µs).
 
 ## Prerequisites
 
-- agentic-flow v1.5.11+ or agentdb v1.0.4+
 - Node.js 18+
+- AgentDB v1.0.7+ (via agentic-flow or standalone)
 - OpenAI API key (for embeddings) or custom embedding model
 
-## Quick Start
+## Quick Start with CLI
+
+### Initialize Vector Database
+
+```bash
+# Initialize with default dimensions (1536 for OpenAI ada-002)
+npx agentdb@latest init ./vectors.db
+
+# Custom dimensions for different embedding models
+npx agentdb@latest init ./vectors.db --dimension 768  # sentence-transformers
+npx agentdb@latest init ./vectors.db --dimension 384  # all-MiniLM-L6-v2
+
+# Use preset configurations
+npx agentdb@latest init ./vectors.db --preset small   # <10K vectors
+npx agentdb@latest init ./vectors.db --preset medium  # 10K-100K vectors
+npx agentdb@latest init ./vectors.db --preset large   # >100K vectors
+
+# In-memory database for testing
+npx agentdb@latest init ./vectors.db --in-memory
+```
+
+### Query Vector Database
+
+```bash
+# Basic similarity search
+npx agentdb@latest query ./vectors.db "[0.1,0.2,0.3,...]"
+
+# Top-k results
+npx agentdb@latest query ./vectors.db "[0.1,0.2,0.3]" -k 10
+
+# With similarity threshold (cosine similarity)
+npx agentdb@latest query ./vectors.db "0.1 0.2 0.3" -t 0.75 -m cosine
+
+# Different distance metrics
+npx agentdb@latest query ./vectors.db "[...]" -m euclidean  # L2 distance
+npx agentdb@latest query ./vectors.db "[...]" -m dot        # Dot product
+
+# JSON output for automation
+npx agentdb@latest query ./vectors.db "[...]" -f json -k 5
+
+# Verbose output with distances
+npx agentdb@latest query ./vectors.db "[...]" -v
+```
+
+### Import/Export Vectors
+
+```bash
+# Export vectors to JSON
+npx agentdb@latest export ./vectors.db ./backup.json
+
+# Import vectors from JSON
+npx agentdb@latest import ./backup.json
+
+# Get database statistics
+npx agentdb@latest stats ./vectors.db
+```
+
+## Quick Start with API
 
 ```typescript
-import { AgentDB } from 'agentdb';
+import { createAgentDBAdapter, computeEmbedding } from 'agentic-flow/reasoningbank';
 
-// Initialize AgentDB with vector support
-const db = new AgentDB({
-  persist: true,
-  vectorDimensions: 1536, // OpenAI ada-002 dimensions
-  enableVectorIndex: true
+// Initialize with vector search optimizations
+const adapter = await createAgentDBAdapter({
+  dbPath: '.agentdb/vectors.db',
+  enableLearning: false,       // Vector search only
+  enableReasoning: true,       // Enable semantic matching
+  quantizationType: 'binary',  // 32x memory reduction
+  cacheSize: 1000,             // Fast retrieval
 });
 
-// Store documents with vectors
-await db.storeMemory({
-  text: "The quantum computer achieved 100 qubits",
-  metadata: { category: "technology", date: "2025-01-15" },
-  embedding: await generateEmbedding(text) // Your embedding function
+// Store document with embedding
+const text = "The quantum computer achieved 100 qubits";
+const embedding = await computeEmbedding(text);
+
+await adapter.insertPattern({
+  id: '',
+  type: 'document',
+  domain: 'technology',
+  pattern_data: JSON.stringify({
+    embedding,
+    text,
+    metadata: { category: "quantum", date: "2025-01-15" }
+  }),
+  confidence: 1.0,
+  usage_count: 0,
+  success_count: 0,
+  created_at: Date.now(),
+  last_used: Date.now(),
 });
 
-// Semantic search
-const results = await db.searchSimilar(
-  queryEmbedding,
-  { limit: 10, threshold: 0.7 }
-);
+// Semantic search with MMR (Maximal Marginal Relevance)
+const queryEmbedding = await computeEmbedding("quantum computing advances");
+const results = await adapter.retrieveWithReasoning(queryEmbedding, {
+  domain: 'technology',
+  k: 10,
+  useMMR: true,              // Diverse results
+  synthesizeContext: true,    // Rich context
+});
 ```
 
 ## Core Features
@@ -104,23 +179,161 @@ await db.batchStore(documents.map(doc => ({
 })));
 ```
 
+## MCP Server Integration
+
+```bash
+# Start AgentDB MCP server for Claude Code
+npx agentdb@latest mcp
+
+# Add to Claude Code (one-time setup)
+claude mcp add agentdb npx agentdb@latest mcp
+
+# Now use MCP tools in Claude Code:
+# - agentdb_query: Semantic vector search
+# - agentdb_store: Store documents with embeddings
+# - agentdb_stats: Database statistics
+```
+
+## Performance Benchmarks
+
+```bash
+# Run comprehensive benchmarks
+npx agentdb@latest benchmark
+
+# Results:
+# ✅ Pattern Search: 150x faster (100µs vs 15ms)
+# ✅ Batch Insert: 500x faster (2ms vs 1s for 100 vectors)
+# ✅ Large-scale Query: 12,500x faster (8ms vs 100s at 1M vectors)
+# ✅ Memory Efficiency: 4-32x reduction with quantization
+```
+
+## Quantization Options
+
+AgentDB provides multiple quantization strategies for memory efficiency:
+
+### Binary Quantization (32x reduction)
+```typescript
+const adapter = await createAgentDBAdapter({
+  quantizationType: 'binary',  // 768-dim → 96 bytes
+});
+```
+
+### Scalar Quantization (4x reduction)
+```typescript
+const adapter = await createAgentDBAdapter({
+  quantizationType: 'scalar',  // 768-dim → 768 bytes
+});
+```
+
+### Product Quantization (8-16x reduction)
+```typescript
+const adapter = await createAgentDBAdapter({
+  quantizationType: 'product',  // 768-dim → 48-96 bytes
+});
+```
+
+## Distance Metrics
+
+```bash
+# Cosine similarity (default, best for most use cases)
+npx agentdb@latest query ./db.sqlite "[...]" -m cosine
+
+# Euclidean distance (L2 norm)
+npx agentdb@latest query ./db.sqlite "[...]" -m euclidean
+
+# Dot product (for normalized vectors)
+npx agentdb@latest query ./db.sqlite "[...]" -m dot
+```
+
+## Advanced Features
+
+### HNSW Indexing
+- **O(log n) search complexity**
+- **Sub-millisecond retrieval** (<100µs)
+- **Automatic index building**
+
+### Caching
+- **1000 pattern in-memory cache**
+- **<1ms pattern retrieval**
+- **Automatic cache invalidation**
+
+### MMR (Maximal Marginal Relevance)
+- **Diverse result sets**
+- **Avoid redundancy**
+- **Balance relevance and diversity**
+
 ## Performance Tips
 
-- **Indexing**: Enable vector index for 10-100x faster searches
-- **Batch Size**: Use batch operations for 1000+ documents
-- **Dimensions**: Match embedding model (1536 for OpenAI ada-002)
-- **Threshold**: Start at 0.7 for quality results
+1. **Enable HNSW indexing**: Automatic with AgentDB, 10-100x faster
+2. **Use quantization**: Binary (32x), Scalar (4x), Product (8-16x) memory reduction
+3. **Batch operations**: 500x faster for bulk inserts
+4. **Match dimensions**: 1536 (OpenAI), 768 (sentence-transformers), 384 (MiniLM)
+5. **Similarity threshold**: Start at 0.7 for quality, adjust based on use case
+6. **Enable caching**: 1000 pattern cache for frequent queries
 
 ## Troubleshooting
 
 ### Issue: Slow search performance
-**Solution**: Enable vector index: `enableVectorIndex: true`
+```bash
+# Check if HNSW indexing is enabled (automatic)
+npx agentdb@latest stats ./vectors.db
+
+# Expected: <100µs search time
+```
+
+### Issue: High memory usage
+```bash
+# Enable binary quantization (32x reduction)
+# Use in adapter: quantizationType: 'binary'
+```
 
 ### Issue: Poor relevance
-**Solution**: Adjust similarity threshold or use hybrid search
+```bash
+# Adjust similarity threshold
+npx agentdb@latest query ./db.sqlite "[...]" -t 0.8  # Higher threshold
+
+# Or use MMR for diverse results
+# Use in adapter: useMMR: true
+```
+
+### Issue: Wrong dimensions
+```bash
+# Check embedding model dimensions:
+# - OpenAI ada-002: 1536
+# - sentence-transformers: 768
+# - all-MiniLM-L6-v2: 384
+
+npx agentdb@latest init ./db.sqlite --dimension 768
+```
+
+## Database Statistics
+
+```bash
+# Get comprehensive stats
+npx agentdb@latest stats ./vectors.db
+
+# Shows:
+# - Total patterns/vectors
+# - Database size
+# - Average confidence
+# - Domains distribution
+# - Index status
+```
+
+## Performance Characteristics
+
+- **Vector Search**: <100µs (HNSW indexing)
+- **Pattern Retrieval**: <1ms (with cache)
+- **Batch Insert**: 2ms for 100 vectors
+- **Memory Efficiency**: 4-32x reduction with quantization
+- **Scalability**: Handles 1M+ vectors efficiently
+- **Latency**: Sub-millisecond for most operations
 
 ## Learn More
 
-- AgentDB Docs: packages/agentdb/README.md
-- Vector DB API: packages/agentdb/docs/vector-api.md
-- Performance Guide: docs/agentdb/performance.md
+- GitHub: https://github.com/ruvnet/agentic-flow/tree/main/packages/agentdb
+- Documentation: node_modules/agentic-flow/docs/AGENTDB_INTEGRATION.md
+- MCP Integration: `npx agentdb@latest mcp` for Claude Code
+- Website: https://agentdb.ruv.io
+- CLI Help: `npx agentdb@latest --help`
+- Command Help: `npx agentdb@latest help <command>`