@ruvector/ruvllm 0.2.0

package/README.md

@@ -0,0 +1,406 @@
+# @ruvector/ruvllm
+
+**Build AI that learns and improves from every interaction.**
+
+RuvLLM is a self-learning language model toolkit that gets smarter over time. Unlike traditional LLMs that remain static after training, RuvLLM continuously adapts to your use case while remembering what it learned before.
+
+## What Makes RuvLLM Different?
+
+Traditional LLMs forget old knowledge when learning new things (called "catastrophic forgetting"). RuvLLM solves this with three key innovations:
+
+1. **It Learns Without Forgetting** - Uses tiny parameter updates (LoRA) and memory protection (EWC++) to learn new patterns while preserving existing knowledge
+
+2. **It Remembers Context** - Built-in vector memory stores and retrieves relevant information instantly using similarity search
+
+3. **It Routes Intelligently** - Automatically selects the right model size and parameters based on query complexity, saving resources on simple tasks
+
+## Key Features
+
+| Feature | What It Does | Why It Matters |
+|---------|-------------|----------------|
+| **Adaptive Learning** | Learns from user feedback in real-time | Improves accuracy over time without retraining |
+| **Memory System** | Stores context with instant similarity search | Finds relevant information in microseconds |
+| **Smart Routing** | Picks optimal model/settings per query | Reduces costs, improves response quality |
+| **SIMD Acceleration** | Uses CPU vector instructions (AVX2/NEON) | 10-50x faster vector operations |
+| **Federated Learning** | Train across devices without sharing data | Privacy-preserving distributed learning |
+| **LoRA Adapters** | Parameter-efficient fine-tuning with low-rank matrices | Fast adaptation with minimal memory |
+| **EWC++ Protection** | Elastic Weight Consolidation prevents forgetting | Learn new tasks without losing old knowledge |
+| **SafeTensors Export** | HuggingFace-compatible model serialization | Share models with the ML ecosystem |
+| **Training Pipeline** | Full training infrastructure with schedulers | Production-ready model training |
+| **Session Management** | Stateful conversations with streaming | Build chat applications easily |
+
+## Installation
+
+```bash
+npm install @ruvector/ruvllm
+```
+
+Or run directly:
+
+```bash
+npx @ruvector/ruvllm info
+```
+
+## Quick Start Tutorial
+
+### 1. Basic Query
+
+```typescript
+import { RuvLLM } from '@ruvector/ruvllm';
+
+const llm = new RuvLLM();
+
+// Ask a question - routing happens automatically
+const response = llm.query('Explain neural networks simply');
+console.log(response.text);
+// Output: "Neural networks are computing systems inspired by..."
+
+console.log(`Used model: ${response.model}`);
+console.log(`Confidence: ${(response.confidence * 100).toFixed(1)}%`);
+```
+
+### 2. Teaching the System
+
+```typescript
+// Query and get a response
+const response = llm.query('What is the capital of France?');
+
+// Provide feedback - the system learns from this
+llm.feedback({
+  requestId: response.requestId,
+  rating: 5,  // 1-5 scale
+  correction: 'Paris is the capital and largest city of France'
+});
+
+// Future similar queries will be more accurate
+```
+
+### 3. Using Memory
+
+```typescript
+// Store important context
+llm.addMemory('Company policy: All returns accepted within 30 days', {
+  category: 'policy',
+  department: 'customer-service'
+});
+
+llm.addMemory('Product X launched in March 2024 with features A, B, C', {
+  category: 'product',
+  name: 'Product X'
+});
+
+// Search memory for relevant context
+const results = llm.searchMemory('return policy', 5);
+console.log(results[0].content);
+// Output: "Company policy: All returns accepted within 30 days"
+console.log(`Relevance: ${(results[0].score * 100).toFixed(1)}%`);
+```
+
+### 4. Computing Similarity
+
+```typescript
+import { SimdOps } from '@ruvector/ruvllm';
+
+const simd = new SimdOps();
+
+// Compare two texts
+const score = llm.similarity(
+  'How do I reset my password?',
+  'I forgot my login credentials'
+);
+console.log(`Similarity: ${(score * 100).toFixed(1)}%`);
+// Output: "Similarity: 78.3%"
+
+// Fast vector operations
+const embedding1 = llm.embed('machine learning');
+const embedding2 = llm.embed('deep learning');
+const similarity = simd.cosineSimilarity(embedding1, embedding2);
+```
+
+### 5. Batch Processing
+
+```typescript
+// Process multiple queries efficiently
+const batch = llm.batchQuery({
+  queries: [
+    'What is AI?',
+    'Explain machine learning',
+    'How do neural networks work?'
+  ],
+  config: { temperature: 0.7 }
+});
+
+batch.responses.forEach((r, i) => {
+  console.log(`Query ${i + 1}: ${r.text.slice(0, 50)}...`);
+});
+console.log(`Total time: ${batch.totalLatencyMs}ms`);
+```
+
+## CLI Commands
+
+```bash
+# Get system information
+ruvllm info
+
+# Query the model
+ruvllm query "What is quantum computing?"
+
+# Generate text with custom settings
+ruvllm generate "Write a product description for:" --temperature 0.8 --max-tokens 200
+
+# Memory operations
+ruvllm memory add "Important fact to remember"
+ruvllm memory search "fact" --k 10
+
+# Compare texts
+ruvllm similarity "hello world" "hi there"
+
+# Get embeddings
+ruvllm embed "your text here"
+
+# Run performance benchmark
+ruvllm benchmark --dims 768 --iterations 5000
+
+# View statistics
+ruvllm stats --json
+```
+
+## Benchmarks
+
+*Benchmarked in Docker (node:20-alpine, x64) - December 2024*
+
+### Core Operations
+
+| Operation | Time | Throughput |
+|-----------|------|------------|
+| Query (short) | 1.49μs | **670K ops/s** |
+| Query (long) | 874ns | **1.14M ops/s** |
+| Generate | 88ns | **11.4M ops/s** |
+| Route | 92ns | **10.9M ops/s** |
+| Embed (256d) | 10.6μs | **94K ops/s** |
+| Embed (768d) | 7.1μs | **140K ops/s** |
+
+### SIMD Vector Operations
+
+| Operation | 128d | 256d | 512d | 768d |
+|-----------|------|------|------|------|
+| Dot Product | 214ns / **4.67M ops/s** | 318ns / **3.15M ops/s** | 609ns / **1.64M ops/s** | 908ns / **1.10M ops/s** |
+| Cosine Similarity | 233ns / **4.30M ops/s** | 335ns / **2.99M ops/s** | 652ns / **1.53M ops/s** | 972ns / **1.03M ops/s** |
+| L2 Distance | 195ns / **5.14M ops/s** | 315ns / **3.18M ops/s** | 612ns / **1.63M ops/s** | 929ns / **1.08M ops/s** |
+
+### LoRA Adapter Performance
+
+| Operation | 64d | 128d | 256d |
+|-----------|-----|------|------|
+| Forward (r=4) | 6.09μs / **164K ops/s** | 2.74μs / **365K ops/s** | 4.83μs / **207K ops/s** |
+| Forward (r=8) | 2.17μs / **462K ops/s** | 4.30μs / **233K ops/s** | 8.99μs / **111K ops/s** |
+| Forward (r=16) | 4.85μs / **206K ops/s** | 9.05μs / **111K ops/s** | 18.3μs / **55K ops/s** |
+| Backward (r=8) | - | 110μs / **9.1K ops/s** | - |
+| Batch (100) | - | 467μs / **2.1K ops/s** | - |
+
+### Memory Operations
+
+| Operation | Time | Throughput |
+|-----------|------|------------|
+| Add Memory | 5.3μs | **189K ops/s** |
+| Search (k=5) | 45.6μs | **21.9K ops/s** |
+| Search (k=10) | 28.3μs | **35.3K ops/s** |
+| Search (k=20) | 33.1μs | **30.2K ops/s** |
+
+### SONA Learning System
+
+| Operation | Time | Throughput |
+|-----------|------|------------|
+| Pattern Store | 14.4μs | **69.5K ops/s** |
+| Pattern Find Similar | 224μs | **4.5K ops/s** |
+| EWC Register Task | 6.5μs | **154K ops/s** |
+| EWC Compute Penalty | 501μs | **2.0K ops/s** |
+| Trajectory Build | 1.24μs | **807K ops/s** |
+
+### Federated Learning
+
+| Operation | Time | Throughput |
+|-----------|------|------------|
+| Agent Create | 7.8μs | **128K ops/s** |
+| Process Task | 7.9μs | **126K ops/s** |
+| Apply LoRA | 12.6μs | **79.6K ops/s** |
+| Export State | 48.9μs | **20.4K ops/s** |
+| Aggregate | 5.26ms | **190 ops/s** |
+
+### Session & Streaming
+
+| Operation | Time | Throughput |
+|-----------|------|------------|
+| Session Create | 1.45μs | **690K ops/s** |
+| Session Chat | 3.28μs | **305K ops/s** |
+| Session Export | 3.91ms | **255 ops/s** |
+| Session Import | 1.60ms | **625 ops/s** |
+
+### Training Pipeline
+
+| Operation | Time |
+|-----------|------|
+| Pipeline Create | 70.6μs |
+| Add Data (100 samples) | 70.6μs |
+| Train (32 samples, 3 epochs) | 1.33s |
+
+### Export/Import
+
+| Operation | Time | Throughput |
+|-----------|------|------------|
+| SafeTensors Write | 67.3μs | **14.9K ops/s** |
+| SafeTensors Read | 102μs | **9.8K ops/s** |
+| LoRA to JSON | 87.9μs | **11.4K ops/s** |
+| LoRA from JSON | 86.0μs | **11.6K ops/s** |
+
+### Performance Highlights
+
+- **Fastest**: Generate at **11.4M ops/s**, Route at **10.9M ops/s**
+- **Vector Ops**: Up to **5.14M ops/s** for L2 distance (128d)
+- **LoRA Forward**: Up to **462K ops/s** (64d, rank-8)
+- **Memory Search**: **35K ops/s** (k=10)
+- **Session Create**: **690K ops/s**
+
+## Configuration
+
+```typescript
+const llm = new RuvLLM({
+  // Embedding settings
+  embeddingDim: 768,        // Vector dimensions (384, 768, 1024)
+
+  // Memory settings
+  hnswM: 16,                // Graph connectivity (higher = better recall, more memory)
+  hnswEfConstruction: 100,  // Build quality (higher = better index, slower build)
+  hnswEfSearch: 64,         // Search quality (higher = better recall, slower search)
+
+  // Learning settings
+  learningEnabled: true,    // Enable adaptive learning
+  qualityThreshold: 0.7,    // Min confidence to skip learning
+  ewcLambda: 2000,          // Memory protection strength
+
+  // Router settings
+  routerHiddenDim: 128,     // Router network size
+});
+```
+
+## Platform Support
+
+Native acceleration available on:
+
+| Platform | Architecture | SIMD Support |
+|----------|-------------|--------------|
+| macOS | Apple Silicon (M1/M2/M3) | NEON |
+| macOS | Intel x64 | AVX2, SSE4.1 |
+| Linux | x64 | AVX2, AVX-512, SSE4.1 |
+| Linux | ARM64 | NEON |
+| Windows | x64 | AVX2, SSE4.1 |
+
+Falls back to optimized JavaScript on unsupported platforms.
+
+## Real-World Use Cases
+
+### Customer Support Bot
+```typescript
+// Store FAQ and policies
+faqs.forEach(faq => llm.addMemory(faq.answer, { question: faq.question }));
+
+// Answer questions with context
+function answerQuestion(question: string) {
+  const context = llm.searchMemory(question, 3);
+  const prompt = `Context:\n${context.map(c => c.content).join('\n')}\n\nQuestion: ${question}`;
+  return llm.query(prompt);
+}
+```
+
+### Document Search
+```typescript
+// Index documents
+documents.forEach(doc => {
+  llm.addMemory(doc.content, {
+    title: doc.title,
+    path: doc.path
+  });
+});
+
+// Semantic search
+const results = llm.searchMemory('quarterly revenue growth', 10);
+```
+
+### Personalized Recommendations
+```typescript
+// Learn from user interactions
+function recordInteraction(userId: string, itemId: string, rating: number) {
+  const response = llm.query(`User ${userId} rated ${itemId}`);
+  llm.feedback({ requestId: response.requestId, rating });
+}
+
+// Get recommendations
+function recommend(userId: string) {
+  return llm.searchMemory(`preferences for user ${userId}`, 10);
+}
+```
+
+## API Reference
+
+### RuvLLM Class
+
+| Method | Description |
+|--------|-------------|
+| `query(text, config?)` | Query with automatic model routing |
+| `generate(prompt, config?)` | Generate text with given prompt |
+| `route(text)` | Get routing decision without executing |
+| `addMemory(content, metadata?)` | Store content in vector memory |
+| `searchMemory(text, k?)` | Find similar content (default k=10) |
+| `feedback(fb)` | Submit feedback for learning |
+| `embed(text)` | Get embedding vector for text |
+| `similarity(t1, t2)` | Compute similarity between texts |
+| `stats()` | Get engine statistics |
+| `forceLearn()` | Trigger immediate learning cycle |
+
+### SimdOps Class
+
+| Method | Description |
+|--------|-------------|
+| `dotProduct(a, b)` | Vector dot product |
+| `cosineSimilarity(a, b)` | Cosine similarity (0-1) |
+| `l2Distance(a, b)` | Euclidean distance |
+| `normalize(v)` | Normalize to unit length |
+| `softmax(v)` | Softmax activation |
+| `relu(v)` | ReLU activation |
+| `gelu(v)` | GELU activation |
+| `layerNorm(v, eps?)` | Layer normalization |
+| `matvec(m, v)` | Matrix-vector multiply |
+
+## Troubleshooting
+
+**Q: Native module not loading?**
+```bash
+ruvllm info  # Check if native is loaded
+```
+If "Native: Fallback", install platform-specific package manually:
+```bash
+npm install @ruvector/ruvllm-darwin-arm64  # For Apple Silicon
+```
+
+**Q: Memory usage too high?**
+Reduce HNSW parameters:
+```typescript
+const llm = new RuvLLM({ hnswM: 8, hnswEfConstruction: 50 });
+```
+
+**Q: Learning not improving results?**
+Check that feedback is being processed:
+```typescript
+const stats = llm.stats();
+console.log(`Patterns learned: ${stats.patternsLearned}`);
+```
+
+## License
+
+MIT OR Apache-2.0
+
+## Links
+
+- [GitHub Repository](https://github.com/ruvnet/ruvector)
+- [Documentation](https://github.com/ruvnet/ruvector/tree/main/examples/ruvLLM)
+- [Issue Tracker](https://github.com/ruvnet/ruvector/issues)