@claude-flow/memory 3.0.0-alpha.1 → 3.0.0-alpha.7

package/docs/AGENTDB-INTEGRATION.md

@@ -1,388 +0,0 @@
-# AgentDB Integration Guide
-
-## Overview
-
-The V3 memory module now integrates with **agentdb@2.0.0-alpha.3.4** to provide high-performance vector search capabilities with HNSW indexing (150x-12,500x faster than brute-force approaches).
-
-## Features
-
-### AgentDBBackend
-
-The `AgentDBBackend` class provides:
-
-- **HNSW Vector Search**: Approximate nearest neighbor search with sub-millisecond query times
-- **Graceful Fallback**: Works without native dependencies (hnswlib-node)
-- **Optional Dependency Handling**: Automatically falls back to pure JavaScript/WASM if native bindings unavailable
-- **Hybrid Integration**: Seamlessly works with `HybridBackend` for combined SQLite + AgentDB queries
-
-### Performance Targets
-
-Based on ADR-006 and ADR-009:
-
-- **150x-12,500x** faster vector search compared to brute-force
-- **Sub-millisecond** query latency for k-NN search
-- **Automatic backend selection**: Native hnswlib → ruvector → WASM fallback
-
-## Installation
-
-```bash
-# Core package (required)
-npm install agentdb@2.0.0-alpha.3.4
-
-# Optional native dependencies for maximum performance
-npm install hnswlib-node@^3.0.0 better-sqlite3@^11.0.0
-```
-
-## Usage
-
-### Basic Setup
-
-```typescript
-import { AgentDBBackend } from '@claude-flow/memory';
-
-const backend = new AgentDBBackend({
-  dbPath: './data/memory.db',
-  namespace: 'default',
-  vectorDimension: 1536, // For OpenAI embeddings
-  hnswM: 16,
-  hnswEfConstruction: 200,
-  hnswEfSearch: 100,
-  embeddingGenerator: async (text) => {
-    // Your embedding function
-    return embeddings.embed(text);
-  },
-});
-
-await backend.initialize();
-```
-
-### Hybrid Backend (Recommended)
-
-Per ADR-009, the recommended approach is to use `HybridBackend`:
-
-```typescript
-import { HybridBackend } from '@claude-flow/memory';
-
-const memory = new HybridBackend({
-  // SQLite for structured queries
-  sqlite: {
-    dbPath: './data/memory-sqlite.db',
-  },
-
-  // AgentDB for vector search
-  agentdb: {
-    dbPath: './data/memory-agentdb.db',
-    vectorDimension: 1536,
-    hnswM: 16,
-    hnswEfConstruction: 200,
-  },
-
-  embeddingGenerator: embedFn,
-  dualWrite: true, // Write to both backends
-});
-
-await memory.initialize();
-
-// Structured queries go to SQLite
-const user = await memory.getByKey('users', 'john@example.com');
-
-// Semantic queries go to AgentDB (150x faster)
-const similar = await memory.querySemantic({
-  content: 'authentication patterns',
-  k: 10,
-  threshold: 0.7,
-});
-
-// Hybrid queries combine both
-const results = await memory.queryHybrid({
-  semantic: { content: 'security vulnerabilities', k: 20 },
-  structured: { namespace: 'security', createdAfter: Date.now() - 86400000 },
-  combineStrategy: 'intersection',
-});
-```
-
-### Semantic Search
-
-```typescript
-// Store entries with embeddings
-await backend.store({
-  id: 'entry-1',
-  key: 'auth-patterns',
-  content: 'OAuth 2.0 implementation patterns for secure authentication',
-  embedding: await embedFn('OAuth 2.0 implementation patterns...'),
-  // ... other fields
-});
-
-// Semantic search by content
-const results = await backend.query({
-  type: 'semantic',
-  content: 'user authentication best practices',
-  limit: 10,
-  threshold: 0.8,
-});
-
-// Or search with pre-computed embedding
-const results = await backend.search(
-  queryEmbedding,
-  { k: 10, threshold: 0.7 }
-);
-```
-
-### Query Routing
-
-The `HybridBackend` automatically routes queries to the optimal backend:
-
-```typescript
-// Exact match → SQLite
-await memory.query({ type: 'exact', namespace: 'users', key: 'john@example.com' });
-
-// Prefix search → SQLite (indexed)
-await memory.query({ type: 'prefix', keyPrefix: 'auth-' });
-
-// Semantic search → AgentDB (HNSW)
-await memory.query({ type: 'semantic', content: 'security patterns', limit: 10 });
-
-// Hybrid → Both backends with intelligent merging
-await memory.query({ type: 'hybrid', content: 'patterns', namespace: 'security' });
-```
-
-## Configuration Options
-
-### AgentDBBackendConfig
-
-```typescript
-interface AgentDBBackendConfig {
-  /** Database path (default: ':memory:') */
-  dbPath?: string;
-
-  /** Namespace for memory organization */
-  namespace?: string;
-
-  /** Force WASM backend (skip native hnswlib) */
-  forceWasm?: boolean;
-
-  /** Vector backend: 'auto', 'ruvector', 'hnswlib' */
-  vectorBackend?: 'auto' | 'ruvector' | 'hnswlib';
-
-  /** Vector dimensions (default: 1536) */
-  vectorDimension?: number;
-
-  /** HNSW M parameter (connections per layer, default: 16) */
-  hnswM?: number;
-
-  /** HNSW efConstruction (build quality, default: 200) */
-  hnswEfConstruction?: number;
-
-  /** HNSW efSearch (search quality, default: 100) */
-  hnswEfSearch?: number;
-
-  /** Enable caching */
-  cacheEnabled?: boolean;
-
-  /** Embedding generator function */
-  embeddingGenerator?: EmbeddingGenerator;
-
-  /** Maximum entries */
-  maxEntries?: number;
-}
-```
-
-### HNSW Tuning
-
-- **M (16-64)**: Higher = better recall, more memory
-  - 16: Fast, less memory (recommended for most cases)
-  - 32: Balanced
-  - 64: High recall, more memory
-
-- **efConstruction (100-400)**: Build time vs. quality
-  - 100: Fast build, lower quality
-  - 200: Balanced (recommended)
-  - 400: Slow build, high quality
-
-- **efSearch (50-200)**: Search time vs. recall
-  - 50: Fast search, lower recall
-  - 100: Balanced (recommended)
-  - 200: Slower search, high recall
-
-## Graceful Degradation
-
-The backend handles missing dependencies gracefully:
-
-```typescript
-// 1. Try native hnswlib (fastest)
-// 2. Fallback to ruvector (fast, pure JS)
-// 3. Fallback to WASM (compatible)
-// 4. Fallback to in-memory brute-force (always works)
-
-const backend = new AgentDBBackend();
-await backend.initialize();
-
-// Check availability
-if (backend.isAvailable()) {
-  console.log('Using AgentDB with HNSW');
-} else {
-  console.log('Using fallback in-memory storage');
-}
-```
-
-## Performance Metrics
-
-### Benchmarks (from agentdb@2.0.0-alpha.3.4)
-
-| Operation | Brute Force | HNSW (hnswlib) | Speedup |
-|-----------|-------------|----------------|---------|
-| 10k vectors, k=10 | 150ms | 1ms | 150x |
-| 100k vectors, k=10 | 1500ms | 2ms | 750x |
-| 1M vectors, k=10 | 15000ms | 3ms | 5000x |
-
-### Memory Usage
-
-- **No quantization**: ~4 bytes per dimension per vector
-- **8-bit quantization**: ~1 byte per dimension (4x reduction)
-- **4-bit quantization**: ~0.5 bytes per dimension (8x reduction)
-
-## Advanced Features
-
-### Vector Quantization
-
-```typescript
-const backend = new AgentDBBackend({
-  // Enable quantization for 50-75% memory reduction
-  quantization: {
-    type: 'scalar',
-    bits: 8, // 4, 8, or 16
-  },
-});
-```
-
-### Custom Distance Metrics
-
-```typescript
-const backend = new AgentDBBackend({
-  vectorBackend: 'hnswlib',
-  distanceMetric: 'cosine', // 'cosine', 'euclidean', 'dot'
-});
-```
-
-### Health Monitoring
-
-```typescript
-const health = await backend.healthCheck();
-
-console.log(health.status); // 'healthy' | 'degraded' | 'unhealthy'
-console.log(health.components.index); // HNSW index health
-
-if (health.status === 'degraded') {
-  console.log('Issues:', health.issues);
-  console.log('Recommendations:', health.recommendations);
-}
-```
-
-### Statistics
-
-```typescript
-const stats = await backend.getStats();
-
-console.log('Total entries:', stats.totalEntries);
-console.log('Avg query time:', stats.avgQueryTime, 'ms');
-console.log('Avg search time:', stats.avgSearchTime, 'ms');
-
-if (stats.hnswStats) {
-  console.log('HNSW vectors:', stats.hnswStats.vectorCount);
-  console.log('HNSW build time:', stats.hnswStats.buildTime, 'ms');
-}
-```
-
-## Migration from Legacy Systems
-
-The memory module includes migration support for legacy systems:
-
-```typescript
-import { MemoryMigrator } from '@claude-flow/memory';
-
-const migrator = new MemoryMigrator(
-  backend,
-  {
-    source: 'memory-manager',
-    sourcePath: './old-memory.json',
-    batchSize: 1000,
-  },
-  embeddingGenerator
-);
-
-const result = await migrator.migrate();
-
-console.log('Migrated:', result.totalMigrated);
-console.log('Failed:', result.totalFailed);
-```
-
-## Troubleshooting
-
-### AgentDB not available
-
-```
-AgentDB not available. Install agentdb@2.0.0-alpha.3.4 for vector search support.
-```
-
-**Solution**: Install agentdb:
-```bash
-npm install agentdb@2.0.0-alpha.3.4
-```
-
-### Native bindings failed
-
-```
-Failed to load hnswlib-node, falling back to WASM
-```
-
-**Solution**: This is normal. The system automatically falls back to WASM. For maximum performance, install build tools:
-
-```bash
-# Ubuntu/Debian
-sudo apt-get install build-essential python3
-
-# macOS
-xcode-select --install
-
-# Then reinstall
-npm install hnswlib-node@^3.0.0
-```
-
-### Force WASM backend
-
-```typescript
-const backend = new AgentDBBackend({
-  forceWasm: true, // Skip native bindings
-});
-```
-
-## Testing
-
-```bash
-# Run AgentDB backend tests
-npm test -- agentdb-backend.test.ts
-
-# Run all memory tests
-npm test
-
-# Run benchmarks
-npm run bench
-```
-
-## Architecture Decision Records
-
-This integration implements:
-
-- **ADR-006**: Unified Memory Service with AgentDB
-- **ADR-009**: Hybrid Memory Backend (SQLite + AgentDB) as default
-
-## Related Documentation
-
-- [AgentDB GitHub](https://github.com/ruvnet/agentic-flow/tree/main/packages/agentdb)
-- [HNSW Algorithm Paper](https://arxiv.org/abs/1603.09320)
-- [V3 Memory Architecture](./README.md)
-- [HybridBackend Documentation](./docs/hybrid-backend.md)
-
-## License
-
-MIT