ruvector 0.2.27 → 0.2.29
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +2270 -2270
- package/bin/cli.js +9570 -9479
- package/bin/mcp-server.js +3854 -3854
- package/dist/core/intelligence-engine.d.ts +13 -0
- package/dist/core/intelligence-engine.d.ts.map +1 -1
- package/dist/core/intelligence-engine.js +38 -0
- package/dist/core/onnx/bundled-parallel.mjs +164 -164
- package/dist/core/onnx/embed-worker.mjs +67 -67
- package/dist/core/onnx/loader.js +434 -434
- package/dist/core/onnx/package.json +3 -3
- package/dist/core/onnx/pkg/LICENSE +21 -21
- package/dist/core/onnx/pkg/loader.js +348 -348
- package/dist/core/onnx/pkg/package.json +3 -3
- package/dist/core/onnx/pkg/ruvector_onnx_embeddings_wasm.d.ts +112 -112
- package/dist/core/onnx/pkg/ruvector_onnx_embeddings_wasm.js +5 -5
- package/dist/core/onnx/pkg/ruvector_onnx_embeddings_wasm_bg.js +638 -638
- package/dist/core/onnx/pkg/ruvector_onnx_embeddings_wasm_bg.wasm.d.ts +29 -29
- package/dist/core/onnx-embedder.d.ts.map +1 -1
- package/dist/core/onnx-embedder.js +24 -30
- package/dist/core/parallel-workers.js +439 -439
- package/dist/workers/benchmark.js +15 -15
- package/package.json +122 -122
- package/src/decompiler/api-prober.js +302 -302
- package/src/decompiler/index.js +463 -463
- package/src/decompiler/metrics.js +86 -86
- package/src/decompiler/model-decompiler.js +423 -423
- package/src/decompiler/module-splitter.js +498 -498
- package/src/decompiler/module-tree.js +142 -142
- package/src/decompiler/name-predictor.js +400 -400
- package/src/decompiler/npm-fetch.js +176 -176
- package/src/decompiler/reconstructor.js +499 -499
- package/src/decompiler/reference-tracker.js +285 -285
- package/src/decompiler/statement-parser.js +285 -285
- package/src/decompiler/style-improver.js +438 -438
- package/src/decompiler/subcategories.js +339 -339
- package/src/decompiler/validator.js +379 -379
- package/src/decompiler/witness.js +140 -140
- package/wasm/package.json +26 -26
- package/wasm/ruvector_decompiler_wasm.d.ts +27 -27
- package/wasm/ruvector_decompiler_wasm.js +220 -220
- package/wasm/ruvector_decompiler_wasm_bg.wasm.d.ts +16 -16
- package/dist/core/onnx/pkg/ruvector.db +0 -0
package/README.md
CHANGED
|
@@ -1,2270 +1,2270 @@
|
|
|
1
|
-
# ruvector
|
|
2
|
-
|
|
3
|
-
[](https://www.npmjs.com/package/ruvector)
|
|
4
|
-
[](https://opensource.org/licenses/MIT)
|
|
5
|
-
[](https://nodejs.org)
|
|
6
|
-
[](https://www.npmjs.com/package/ruvector)
|
|
7
|
-
[](https://github.com/ruvnet/ruvector)
|
|
8
|
-
[](https://github.com/ruvnet/ruvector)
|
|
9
|
-
[](https://github.com/ruvnet/ruvector)
|
|
10
|
-
|
|
11
|
-
**The fastest vector database for Node.js—built in Rust, runs everywhere**
|
|
12
|
-
|
|
13
|
-
Ruvector is a self-learning vector database with **enterprise-grade semantic search**, hybrid retrieval (sparse + dense), Graph RAG, FlashAttention-3, and DiskANN — all in a single npm package. Unlike cloud-only solutions or Python-first databases, Ruvector is designed for JavaScript/TypeScript developers who need **blazing-fast vector search** without external services.
|
|
14
|
-
|
|
15
|
-
> 🚀 **Sub-millisecond queries** • 🎯 **52,000+ inserts/sec** • 💾 **~50 bytes per vector** • 🌍 **Runs anywhere** • 🧠 **859 tests passing**
|
|
16
|
-
|
|
17
|
-
Built by [rUv](https://ruv.io) with production-grade Rust performance and intelligent platform detection—**automatically uses native bindings when available, falls back to WebAssembly when needed**.
|
|
18
|
-
|
|
19
|
-
🌐 **[Visit ruv.io](https://ruv.io)** | 📦 **[GitHub](https://github.com/ruvnet/ruvector)** | 📚 **[Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)**
|
|
20
|
-
|
|
21
|
-
---
|
|
22
|
-
|
|
23
|
-
## 🧠 Claude Code Intelligence v2.0
|
|
24
|
-
|
|
25
|
-
**Self-learning intelligence for Claude Code** — RuVector provides optimized hooks with ONNX embeddings, AST analysis, and coverage-aware routing.
|
|
26
|
-
|
|
27
|
-
```bash
|
|
28
|
-
# One-command setup with pretrain and agent generation
|
|
29
|
-
npx ruvector hooks init --pretrain --build-agents quality
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
### Core Features
|
|
33
|
-
- 🎯 **Smart Agent Routing** — Q-learning optimized suggestions with 80%+ accuracy
|
|
34
|
-
- 📚 **9-Phase Pretrain** — AST, diff, coverage, neural, and graph analysis
|
|
35
|
-
- 🤖 **Agent Builder** — Generates optimized `.claude/agents/` configs
|
|
36
|
-
- 🔗 **Co-edit Patterns** — Learns file relationships from git history
|
|
37
|
-
- 💾 **Vector Memory** — HNSW-indexed semantic recall (150x faster)
|
|
38
|
-
|
|
39
|
-
### New in v2.1 — SOTA Vector Search
|
|
40
|
-
- **FlashAttention-3** — IO-aware tiled attention, O(N) memory instead of O(N^2)
|
|
41
|
-
- **Graph RAG** — Knowledge graph + community detection for multi-hop queries (30-60% improvement)
|
|
42
|
-
- **Hybrid Search** — Sparse + dense vectors with RRF fusion (20-49% better retrieval)
|
|
43
|
-
- **DiskANN / Vamana** — SSD-friendly ANN graph with PQ compression for large-scale search
|
|
44
|
-
- **ColBERT Multi-Vector** — Per-token late interaction retrieval (MaxSim)
|
|
45
|
-
- **Matryoshka Embeddings** — Adaptive-dimension search with funnel/cascade modes
|
|
46
|
-
- **MLA** — Multi-Head Latent Attention with ~93% KV-cache compression (DeepSeek-V2/V3)
|
|
47
|
-
- **Mamba SSM** — Selective State Space Models for linear-time sequence processing
|
|
48
|
-
- **TurboQuant** — 2-4 bit KV-cache quantization, 6-8x memory reduction
|
|
49
|
-
- **OPQ** — Optimized Product Quantization with learned rotation (10-30% error reduction)
|
|
50
|
-
- **GraphMAE** — Graph Masked Autoencoder for self-supervised node learning
|
|
51
|
-
|
|
52
|
-
### New in v2.0
|
|
53
|
-
- **ONNX WASM Embeddings** — all-MiniLM-L6-v2 (384d) runs locally, no API needed
|
|
54
|
-
- **AST Analysis** — Symbol extraction, complexity metrics, import graphs
|
|
55
|
-
- **Diff Embeddings** — Semantic change classification with risk scoring
|
|
56
|
-
- **Coverage Routing** — Test coverage-aware agent selection
|
|
57
|
-
- **Graph Algorithms** — MinCut boundaries, Louvain communities, Spectral clustering
|
|
58
|
-
- 🛡️ **Security Scanning** — Parallel vulnerability pattern detection
|
|
59
|
-
- 🎯 **RAG Context** — Semantic retrieval with HNSW indexing
|
|
60
|
-
|
|
61
|
-
### Performance
|
|
62
|
-
| Backend | Read Time | Speedup |
|
|
63
|
-
|---------|-----------|---------|
|
|
64
|
-
| ONNX inference | ~400ms | baseline |
|
|
65
|
-
| HNSW search | ~0.045ms | 8,800x |
|
|
66
|
-
| Memory cache | ~0.01ms | **40,000x** |
|
|
67
|
-
|
|
68
|
-
📖 **[Full Hooks Documentation →](https://github.com/ruvnet/ruvector/blob/main/npm/packages/ruvector/HOOKS.md)**
|
|
69
|
-
|
|
70
|
-
### MCP Server Integration
|
|
71
|
-
|
|
72
|
-
RuVector includes an MCP server for Claude Code with 103 tools:
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
# Add to Claude Code
|
|
76
|
-
claude mcp add ruvector -- npx ruvector mcp start
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
**Available MCP Tools:**
|
|
80
|
-
- `hooks_route`, `hooks_route_enhanced` — Agent routing with signals
|
|
81
|
-
- `hooks_ast_analyze`, `hooks_ast_complexity` — Code structure analysis
|
|
82
|
-
- `hooks_diff_analyze`, `hooks_diff_classify` — Change classification
|
|
83
|
-
- `hooks_coverage_route`, `hooks_coverage_suggest` — Test-aware routing
|
|
84
|
-
- `hooks_graph_mincut`, `hooks_graph_cluster` — Code boundaries
|
|
85
|
-
- `hooks_security_scan` — Vulnerability detection
|
|
86
|
-
- `hooks_rag_context` — Semantic context retrieval
|
|
87
|
-
- `hooks_attention_info`, `hooks_gnn_info` — Neural capabilities
|
|
88
|
-
- `brain_search`, `brain_share`, `brain_status` — Shared brain knowledge
|
|
89
|
-
- `brain_agi_status`, `brain_sona_stats`, `brain_temporal`, `brain_explore` — AGI diagnostics
|
|
90
|
-
- `brain_midstream`, `brain_flags` — Midstream platform + feature flags
|
|
91
|
-
- `midstream_status`, `midstream_attractor`, `midstream_scheduler` — Streaming analysis
|
|
92
|
-
- `midstream_benchmark`, `midstream_search`, `midstream_health` — Latency benchmarks + health
|
|
93
|
-
|
|
94
|
-
### Brain AGI Commands
|
|
95
|
-
|
|
96
|
-
Access all 8 AGI subsystems deployed at π.ruv.io:
|
|
97
|
-
|
|
98
|
-
```bash
|
|
99
|
-
npx ruvector brain agi status # Combined AGI + midstream diagnostics
|
|
100
|
-
npx ruvector brain agi sona # SONA patterns, trajectories, ticks
|
|
101
|
-
npx ruvector brain agi temporal # Knowledge evolution velocity
|
|
102
|
-
npx ruvector brain agi explore # Meta-learning curiosity & regret
|
|
103
|
-
npx ruvector brain agi midstream # Scheduler, attractor, solver, strange-loop
|
|
104
|
-
npx ruvector brain agi flags # Feature flag state
|
|
105
|
-
```
|
|
106
|
-
|
|
107
|
-
### Midstream Commands
|
|
108
|
-
|
|
109
|
-
Real-time streaming analysis platform:
|
|
110
|
-
|
|
111
|
-
```bash
|
|
112
|
-
npx ruvector midstream status # Platform overview
|
|
113
|
-
npx ruvector midstream attractor # Lyapunov attractor analysis
|
|
114
|
-
npx ruvector midstream scheduler # Nanosecond scheduler metrics
|
|
115
|
-
npx ruvector midstream benchmark # Latency benchmark (p50/p90/p99)
|
|
116
|
-
```
|
|
117
|
-
|
|
118
|
-
---
|
|
119
|
-
|
|
120
|
-
## 🌟 Why Ruvector?
|
|
121
|
-
|
|
122
|
-
### The Problem with Existing Vector Databases
|
|
123
|
-
|
|
124
|
-
Most vector databases force you to choose between three painful trade-offs:
|
|
125
|
-
|
|
126
|
-
1. **Cloud-Only Services** (Pinecone, Weaviate Cloud) - Expensive, vendor lock-in, latency issues, API rate limits
|
|
127
|
-
2. **Python-First Solutions** (ChromaDB, Faiss) - Poor Node.js support, require separate Python processes
|
|
128
|
-
3. **Self-Hosted Complexity** (Milvus, Qdrant) - Heavy infrastructure, Docker orchestration, operational overhead
|
|
129
|
-
|
|
130
|
-
**Ruvector eliminates these trade-offs.**
|
|
131
|
-
|
|
132
|
-
### The Ruvector Advantage
|
|
133
|
-
|
|
134
|
-
Ruvector is purpose-built for **modern JavaScript/TypeScript applications** that need vector search:
|
|
135
|
-
|
|
136
|
-
🎯 **Native Node.js Integration**
|
|
137
|
-
- Drop-in npm package—no Docker, no Python, no external services
|
|
138
|
-
- Full TypeScript support with complete type definitions
|
|
139
|
-
- Automatic platform detection with native Rust bindings
|
|
140
|
-
- Seamless WebAssembly fallback for universal compatibility
|
|
141
|
-
|
|
142
|
-
⚡ **Production-Grade Performance**
|
|
143
|
-
- **52,000+ inserts/second** with native Rust (10x faster than Python alternatives)
|
|
144
|
-
- **<0.5ms query latency** with HNSW indexing and SIMD optimizations
|
|
145
|
-
- **~50 bytes per vector** with advanced memory optimization
|
|
146
|
-
- Scales from edge devices to millions of vectors
|
|
147
|
-
|
|
148
|
-
🧠 **Built for AI Applications**
|
|
149
|
-
- Optimized for LLM embeddings (OpenAI, Cohere, Hugging Face)
|
|
150
|
-
- Perfect for RAG (Retrieval-Augmented Generation) systems
|
|
151
|
-
- Agent memory and semantic caching
|
|
152
|
-
- Real-time recommendation engines
|
|
153
|
-
|
|
154
|
-
🌍 **Universal Deployment**
|
|
155
|
-
- **Linux, macOS, Windows** with native performance
|
|
156
|
-
- **Browser support** via WebAssembly (experimental)
|
|
157
|
-
- **Edge computing** and serverless environments
|
|
158
|
-
- **Alpine Linux** and non-glibc systems supported
|
|
159
|
-
|
|
160
|
-
💰 **Zero Operational Costs**
|
|
161
|
-
- No cloud API fees or usage limits
|
|
162
|
-
- No infrastructure to manage
|
|
163
|
-
- No separate database servers
|
|
164
|
-
- Open source MIT license
|
|
165
|
-
|
|
166
|
-
### Key Advantages
|
|
167
|
-
|
|
168
|
-
- ⚡ **Blazing Fast**: <0.5ms p50 latency with native Rust, 10-50ms with WASM fallback
|
|
169
|
-
- 🎯 **Automatic Platform Detection**: Uses native when available, falls back to WASM seamlessly
|
|
170
|
-
- 🧠 **AI-Native**: Built specifically for embeddings, RAG, semantic search, and agent memory
|
|
171
|
-
- 🔧 **CLI Tools Included**: Full command-line interface for database management
|
|
172
|
-
- 🌍 **Universal Deployment**: Works on all platforms—Linux, macOS, Windows, even browsers
|
|
173
|
-
- 💾 **Memory Efficient**: ~50 bytes per vector with advanced quantization
|
|
174
|
-
- 🚀 **Production Ready**: Battle-tested algorithms with comprehensive benchmarks
|
|
175
|
-
- 🔓 **Open Source**: MIT licensed, community-driven
|
|
176
|
-
|
|
177
|
-
## 🚀 Quick Start Tutorial
|
|
178
|
-
|
|
179
|
-
### Step 1: Installation
|
|
180
|
-
|
|
181
|
-
Install Ruvector with a single npm command:
|
|
182
|
-
|
|
183
|
-
```bash
|
|
184
|
-
npm install ruvector
|
|
185
|
-
```
|
|
186
|
-
|
|
187
|
-
**What happens during installation:**
|
|
188
|
-
- npm automatically detects your platform (Linux, macOS, Windows)
|
|
189
|
-
- Downloads the correct native binary for maximum performance
|
|
190
|
-
- Falls back to WebAssembly if native binaries aren't available
|
|
191
|
-
- No additional setup, Docker, or external services required
|
|
192
|
-
|
|
193
|
-
**Windows Installation (without build tools):**
|
|
194
|
-
```bash
|
|
195
|
-
# Skip native compilation, use WASM fallback
|
|
196
|
-
npm install ruvector --ignore-scripts
|
|
197
|
-
|
|
198
|
-
# The ONNX WASM runtime (7.4MB) works without build tools
|
|
199
|
-
# Memory cache provides 40,000x speedup over inference
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
**Verify installation:**
|
|
203
|
-
```bash
|
|
204
|
-
npx ruvector info
|
|
205
|
-
```
|
|
206
|
-
|
|
207
|
-
You should see your platform and implementation type (native Rust or WASM fallback).
|
|
208
|
-
|
|
209
|
-
### Step 2: Your First Vector Database
|
|
210
|
-
|
|
211
|
-
Let's create a simple vector database and perform basic operations. This example demonstrates the complete CRUD (Create, Read, Update, Delete) workflow:
|
|
212
|
-
|
|
213
|
-
```javascript
|
|
214
|
-
const { VectorDb } = require('ruvector');
|
|
215
|
-
|
|
216
|
-
async function tutorial() {
|
|
217
|
-
// Step 2.1: Create a new vector database
|
|
218
|
-
// The 'dimensions' parameter must match your embedding model
|
|
219
|
-
// Common sizes: 128, 384 (sentence-transformers), 768 (BERT), 1536 (OpenAI)
|
|
220
|
-
const db = new VectorDb({
|
|
221
|
-
dimensions: 128, // Vector size - MUST match your embeddings
|
|
222
|
-
maxElements: 10000, // Maximum vectors (can grow automatically)
|
|
223
|
-
storagePath: './my-vectors.db' // Persist to disk (omit for in-memory)
|
|
224
|
-
});
|
|
225
|
-
|
|
226
|
-
console.log('✅ Database created successfully');
|
|
227
|
-
|
|
228
|
-
// Step 2.2: Insert vectors
|
|
229
|
-
// In real applications, these would come from an embedding model
|
|
230
|
-
const documents = [
|
|
231
|
-
{ id: 'doc1', text: 'Artificial intelligence and machine learning' },
|
|
232
|
-
{ id: 'doc2', text: 'Deep learning neural networks' },
|
|
233
|
-
{ id: 'doc3', text: 'Natural language processing' },
|
|
234
|
-
];
|
|
235
|
-
|
|
236
|
-
for (const doc of documents) {
|
|
237
|
-
// Generate random vector for demonstration
|
|
238
|
-
// In production: use OpenAI, Cohere, or sentence-transformers
|
|
239
|
-
const vector = new Float32Array(128).map(() => Math.random());
|
|
240
|
-
|
|
241
|
-
await db.insert({
|
|
242
|
-
id: doc.id,
|
|
243
|
-
vector: vector,
|
|
244
|
-
metadata: {
|
|
245
|
-
text: doc.text,
|
|
246
|
-
timestamp: Date.now(),
|
|
247
|
-
category: 'AI'
|
|
248
|
-
}
|
|
249
|
-
});
|
|
250
|
-
|
|
251
|
-
console.log(`✅ Inserted: ${doc.id}`);
|
|
252
|
-
}
|
|
253
|
-
|
|
254
|
-
// Step 2.3: Search for similar vectors
|
|
255
|
-
// Create a query vector (in production, this would be from your search query)
|
|
256
|
-
const queryVector = new Float32Array(128).map(() => Math.random());
|
|
257
|
-
|
|
258
|
-
const results = await db.search({
|
|
259
|
-
vector: queryVector,
|
|
260
|
-
k: 5, // Return top 5 most similar vectors
|
|
261
|
-
threshold: 0.7 // Only return results with similarity > 0.7
|
|
262
|
-
});
|
|
263
|
-
|
|
264
|
-
console.log('\n🔍 Search Results:');
|
|
265
|
-
results.forEach((result, index) => {
|
|
266
|
-
console.log(`${index + 1}. ${result.id} - Score: ${result.score.toFixed(3)}`);
|
|
267
|
-
console.log(` Text: ${result.metadata.text}`);
|
|
268
|
-
});
|
|
269
|
-
|
|
270
|
-
// Step 2.4: Retrieve a specific vector
|
|
271
|
-
const retrieved = await db.get('doc1');
|
|
272
|
-
if (retrieved) {
|
|
273
|
-
console.log('\n📄 Retrieved document:', retrieved.metadata.text);
|
|
274
|
-
}
|
|
275
|
-
|
|
276
|
-
// Step 2.5: Get database statistics
|
|
277
|
-
const count = await db.len();
|
|
278
|
-
console.log(`\n📊 Total vectors in database: ${count}`);
|
|
279
|
-
|
|
280
|
-
// Step 2.6: Delete a vector
|
|
281
|
-
const deleted = await db.delete('doc1');
|
|
282
|
-
console.log(`\n🗑️ Deleted doc1: ${deleted ? 'Success' : 'Not found'}`);
|
|
283
|
-
|
|
284
|
-
// Final count
|
|
285
|
-
const finalCount = await db.len();
|
|
286
|
-
console.log(`📊 Final count: ${finalCount}`);
|
|
287
|
-
}
|
|
288
|
-
|
|
289
|
-
// Run the tutorial
|
|
290
|
-
tutorial().catch(console.error);
|
|
291
|
-
```
|
|
292
|
-
|
|
293
|
-
**Expected Output:**
|
|
294
|
-
```
|
|
295
|
-
✅ Database created successfully
|
|
296
|
-
✅ Inserted: doc1
|
|
297
|
-
✅ Inserted: doc2
|
|
298
|
-
✅ Inserted: doc3
|
|
299
|
-
|
|
300
|
-
🔍 Search Results:
|
|
301
|
-
1. doc2 - Score: 0.892
|
|
302
|
-
Text: Deep learning neural networks
|
|
303
|
-
2. doc1 - Score: 0.856
|
|
304
|
-
Text: Artificial intelligence and machine learning
|
|
305
|
-
3. doc3 - Score: 0.801
|
|
306
|
-
Text: Natural language processing
|
|
307
|
-
|
|
308
|
-
📄 Retrieved document: Artificial intelligence and machine learning
|
|
309
|
-
|
|
310
|
-
📊 Total vectors in database: 3
|
|
311
|
-
|
|
312
|
-
🗑️ Deleted doc1: Success
|
|
313
|
-
📊 Final count: 2
|
|
314
|
-
```
|
|
315
|
-
|
|
316
|
-
### Step 3: TypeScript Tutorial
|
|
317
|
-
|
|
318
|
-
Ruvector provides full TypeScript support with complete type safety. Here's how to use it:
|
|
319
|
-
|
|
320
|
-
```typescript
|
|
321
|
-
import { VectorDb, VectorEntry, SearchQuery, SearchResult } from 'ruvector';
|
|
322
|
-
|
|
323
|
-
// Step 3.1: Define your custom metadata type
|
|
324
|
-
interface DocumentMetadata {
|
|
325
|
-
title: string;
|
|
326
|
-
content: string;
|
|
327
|
-
author: string;
|
|
328
|
-
date: Date;
|
|
329
|
-
tags: string[];
|
|
330
|
-
}
|
|
331
|
-
|
|
332
|
-
async function typescriptTutorial() {
|
|
333
|
-
// Step 3.2: Create typed database
|
|
334
|
-
const db = new VectorDb({
|
|
335
|
-
dimensions: 384, // sentence-transformers/all-MiniLM-L6-v2
|
|
336
|
-
maxElements: 10000,
|
|
337
|
-
storagePath: './typed-vectors.db'
|
|
338
|
-
});
|
|
339
|
-
|
|
340
|
-
// Step 3.3: Type-safe vector entry
|
|
341
|
-
const entry: VectorEntry<DocumentMetadata> = {
|
|
342
|
-
id: 'article-001',
|
|
343
|
-
vector: new Float32Array(384), // Your embedding here
|
|
344
|
-
metadata: {
|
|
345
|
-
title: 'Introduction to Vector Databases',
|
|
346
|
-
content: 'Vector databases enable semantic search...',
|
|
347
|
-
author: 'Jane Doe',
|
|
348
|
-
date: new Date('2024-01-15'),
|
|
349
|
-
tags: ['database', 'AI', 'search']
|
|
350
|
-
}
|
|
351
|
-
};
|
|
352
|
-
|
|
353
|
-
// Step 3.4: Insert with type checking
|
|
354
|
-
await db.insert(entry);
|
|
355
|
-
console.log('✅ Inserted typed document');
|
|
356
|
-
|
|
357
|
-
// Step 3.5: Type-safe search
|
|
358
|
-
const query: SearchQuery = {
|
|
359
|
-
vector: new Float32Array(384),
|
|
360
|
-
k: 10,
|
|
361
|
-
threshold: 0.8
|
|
362
|
-
};
|
|
363
|
-
|
|
364
|
-
// Step 3.6: Fully typed results
|
|
365
|
-
const results: SearchResult<DocumentMetadata>[] = await db.search(query);
|
|
366
|
-
|
|
367
|
-
// TypeScript knows the exact shape of metadata
|
|
368
|
-
results.forEach(result => {
|
|
369
|
-
console.log(`Title: ${result.metadata.title}`);
|
|
370
|
-
console.log(`Author: ${result.metadata.author}`);
|
|
371
|
-
console.log(`Tags: ${result.metadata.tags.join(', ')}`);
|
|
372
|
-
console.log(`Similarity: ${result.score.toFixed(3)}\n`);
|
|
373
|
-
});
|
|
374
|
-
|
|
375
|
-
// Step 3.7: Type-safe retrieval
|
|
376
|
-
const doc = await db.get('article-001');
|
|
377
|
-
if (doc) {
|
|
378
|
-
// TypeScript autocomplete works perfectly here
|
|
379
|
-
const publishYear = doc.metadata.date.getFullYear();
|
|
380
|
-
console.log(`Published in ${publishYear}`);
|
|
381
|
-
}
|
|
382
|
-
}
|
|
383
|
-
|
|
384
|
-
typescriptTutorial().catch(console.error);
|
|
385
|
-
```
|
|
386
|
-
|
|
387
|
-
**TypeScript Benefits:**
|
|
388
|
-
- ✅ Full autocomplete for all methods and properties
|
|
389
|
-
- ✅ Compile-time type checking prevents errors
|
|
390
|
-
- ✅ IDE IntelliSense shows documentation
|
|
391
|
-
- ✅ Custom metadata types for your use case
|
|
392
|
-
- ✅ No `any` types - fully typed throughout
|
|
393
|
-
|
|
394
|
-
## 🎯 Platform Detection
|
|
395
|
-
|
|
396
|
-
Ruvector automatically detects the best implementation for your platform:
|
|
397
|
-
|
|
398
|
-
```javascript
|
|
399
|
-
const { getImplementationType, isNative, isWasm } = require('ruvector');
|
|
400
|
-
|
|
401
|
-
console.log(getImplementationType()); // 'native' or 'wasm'
|
|
402
|
-
console.log(isNative()); // true if using native Rust
|
|
403
|
-
console.log(isWasm()); // true if using WebAssembly fallback
|
|
404
|
-
|
|
405
|
-
// Performance varies by implementation:
|
|
406
|
-
// Native (Rust): <0.5ms latency, 50K+ ops/sec
|
|
407
|
-
// WASM fallback: 10-50ms latency, ~1K ops/sec
|
|
408
|
-
```
|
|
409
|
-
|
|
410
|
-
## 🔧 CLI Tools
|
|
411
|
-
|
|
412
|
-
Ruvector includes a full command-line interface for database management:
|
|
413
|
-
|
|
414
|
-
### Create Database
|
|
415
|
-
|
|
416
|
-
```bash
|
|
417
|
-
# Create a new vector database
|
|
418
|
-
npx ruvector create mydb.vec --dimensions 384 --metric cosine
|
|
419
|
-
|
|
420
|
-
# Options:
|
|
421
|
-
# --dimensions, -d Vector dimensionality (required)
|
|
422
|
-
# --metric, -m Distance metric (cosine, euclidean, dot)
|
|
423
|
-
# --max-elements Maximum number of vectors (default: 10000)
|
|
424
|
-
```
|
|
425
|
-
|
|
426
|
-
### Insert Vectors
|
|
427
|
-
|
|
428
|
-
```bash
|
|
429
|
-
# Insert vectors from JSON file
|
|
430
|
-
npx ruvector insert mydb.vec vectors.json
|
|
431
|
-
|
|
432
|
-
# JSON format:
|
|
433
|
-
# [
|
|
434
|
-
# { "id": "doc1", "vector": [0.1, 0.2, ...], "metadata": {...} },
|
|
435
|
-
# { "id": "doc2", "vector": [0.3, 0.4, ...], "metadata": {...} }
|
|
436
|
-
# ]
|
|
437
|
-
```
|
|
438
|
-
|
|
439
|
-
### Search Vectors
|
|
440
|
-
|
|
441
|
-
```bash
|
|
442
|
-
# Search for similar vectors
|
|
443
|
-
npx ruvector search mydb.vec --vector "[0.1,0.2,0.3,...]" --top-k 10
|
|
444
|
-
|
|
445
|
-
# Options:
|
|
446
|
-
# --vector, -v Query vector (JSON array)
|
|
447
|
-
# --top-k, -k Number of results (default: 10)
|
|
448
|
-
# --threshold Minimum similarity score
|
|
449
|
-
```
|
|
450
|
-
|
|
451
|
-
### Database Statistics
|
|
452
|
-
|
|
453
|
-
```bash
|
|
454
|
-
# Show database statistics
|
|
455
|
-
npx ruvector stats mydb.vec
|
|
456
|
-
|
|
457
|
-
# Output:
|
|
458
|
-
# Total vectors: 10,000
|
|
459
|
-
# Dimensions: 384
|
|
460
|
-
# Metric: cosine
|
|
461
|
-
# Memory usage: ~500 KB
|
|
462
|
-
# Index type: HNSW
|
|
463
|
-
```
|
|
464
|
-
|
|
465
|
-
### Benchmarking
|
|
466
|
-
|
|
467
|
-
```bash
|
|
468
|
-
# Run performance benchmark
|
|
469
|
-
npx ruvector benchmark --num-vectors 10000 --num-queries 1000
|
|
470
|
-
|
|
471
|
-
# Options:
|
|
472
|
-
# --num-vectors Number of vectors to insert
|
|
473
|
-
# --num-queries Number of search queries
|
|
474
|
-
# --dimensions Vector dimensionality (default: 128)
|
|
475
|
-
```
|
|
476
|
-
|
|
477
|
-
### System Information
|
|
478
|
-
|
|
479
|
-
```bash
|
|
480
|
-
# Show platform and implementation info
|
|
481
|
-
npx ruvector info
|
|
482
|
-
|
|
483
|
-
# Output:
|
|
484
|
-
# Platform: linux-x64-gnu
|
|
485
|
-
# Implementation: native (Rust)
|
|
486
|
-
# GNN Module: Available
|
|
487
|
-
# Node.js: v18.17.0
|
|
488
|
-
# Performance: <0.5ms p50 latency
|
|
489
|
-
```
|
|
490
|
-
|
|
491
|
-
### Install Optional Packages
|
|
492
|
-
|
|
493
|
-
Ruvector supports optional packages that extend functionality. Use the `install` command to add them:
|
|
494
|
-
|
|
495
|
-
```bash
|
|
496
|
-
# List available packages
|
|
497
|
-
npx ruvector install
|
|
498
|
-
|
|
499
|
-
# Output:
|
|
500
|
-
# Available Ruvector Packages:
|
|
501
|
-
#
|
|
502
|
-
# gnn not installed
|
|
503
|
-
# Graph Neural Network layers, tensor compression, differentiable search
|
|
504
|
-
# npm: @ruvector/gnn
|
|
505
|
-
#
|
|
506
|
-
# core ✓ installed
|
|
507
|
-
# Core vector database with native Rust bindings
|
|
508
|
-
# npm: @ruvector/core
|
|
509
|
-
|
|
510
|
-
# Install specific package
|
|
511
|
-
npx ruvector install gnn
|
|
512
|
-
|
|
513
|
-
# Install all optional packages
|
|
514
|
-
npx ruvector install --all
|
|
515
|
-
|
|
516
|
-
# Interactive selection
|
|
517
|
-
npx ruvector install -i
|
|
518
|
-
```
|
|
519
|
-
|
|
520
|
-
The install command auto-detects your package manager (npm, yarn, pnpm, bun).
|
|
521
|
-
|
|
522
|
-
### GNN Commands
|
|
523
|
-
|
|
524
|
-
Ruvector includes Graph Neural Network (GNN) capabilities for advanced tensor compression and differentiable search.
|
|
525
|
-
|
|
526
|
-
#### GNN Info
|
|
527
|
-
|
|
528
|
-
```bash
|
|
529
|
-
# Show GNN module information
|
|
530
|
-
npx ruvector gnn info
|
|
531
|
-
|
|
532
|
-
# Output:
|
|
533
|
-
# GNN Module Information
|
|
534
|
-
# Status: Available
|
|
535
|
-
# Platform: linux
|
|
536
|
-
# Architecture: x64
|
|
537
|
-
#
|
|
538
|
-
# Available Features:
|
|
539
|
-
# • RuvectorLayer - GNN layer with multi-head attention
|
|
540
|
-
# • TensorCompress - Adaptive tensor compression (5 levels)
|
|
541
|
-
# • differentiableSearch - Soft attention-based search
|
|
542
|
-
# • hierarchicalForward - Multi-layer GNN processing
|
|
543
|
-
```
|
|
544
|
-
|
|
545
|
-
#### GNN Layer
|
|
546
|
-
|
|
547
|
-
```bash
|
|
548
|
-
# Create and test a GNN layer
|
|
549
|
-
npx ruvector gnn layer -i 128 -h 256 --test
|
|
550
|
-
|
|
551
|
-
# Options:
|
|
552
|
-
# -i, --input-dim Input dimension (required)
|
|
553
|
-
# -h, --hidden-dim Hidden dimension (required)
|
|
554
|
-
# -a, --heads Number of attention heads (default: 4)
|
|
555
|
-
# -d, --dropout Dropout rate (default: 0.1)
|
|
556
|
-
# --test Run a test forward pass
|
|
557
|
-
# -o, --output Save layer config to JSON file
|
|
558
|
-
```
|
|
559
|
-
|
|
560
|
-
#### GNN Compress
|
|
561
|
-
|
|
562
|
-
```bash
|
|
563
|
-
# Compress embeddings using adaptive tensor compression
|
|
564
|
-
npx ruvector gnn compress -f embeddings.json -l pq8 -o compressed.json
|
|
565
|
-
|
|
566
|
-
# Options:
|
|
567
|
-
# -f, --file Input JSON file with embeddings (required)
|
|
568
|
-
# -l, --level Compression level: none|half|pq8|pq4|binary (default: auto)
|
|
569
|
-
# -a, --access-freq Access frequency for auto compression (default: 0.5)
|
|
570
|
-
# -o, --output Output file for compressed data
|
|
571
|
-
|
|
572
|
-
# Compression levels:
|
|
573
|
-
# none (freq > 0.8) - Full precision, hot data
|
|
574
|
-
# half (freq > 0.4) - ~50% savings, warm data
|
|
575
|
-
# pq8 (freq > 0.1) - ~8x compression, cool data
|
|
576
|
-
# pq4 (freq > 0.01) - ~16x compression, cold data
|
|
577
|
-
# binary (freq <= 0.01) - ~32x compression, archive
|
|
578
|
-
```
|
|
579
|
-
|
|
580
|
-
#### GNN Search
|
|
581
|
-
|
|
582
|
-
```bash
|
|
583
|
-
# Differentiable search with soft attention
|
|
584
|
-
npx ruvector gnn search -q "[1.0,0.0,0.0]" -c candidates.json -k 5
|
|
585
|
-
|
|
586
|
-
# Options:
|
|
587
|
-
# -q, --query Query vector as JSON array (required)
|
|
588
|
-
# -c, --candidates Candidates file - JSON array of vectors (required)
|
|
589
|
-
# -k, --top-k Number of results (default: 5)
|
|
590
|
-
# -t, --temperature Softmax temperature (default: 1.0)
|
|
591
|
-
```
|
|
592
|
-
|
|
593
|
-
### Attention Commands
|
|
594
|
-
|
|
595
|
-
Ruvector includes high-performance attention mechanisms for transformer-based operations, hyperbolic embeddings, and graph attention.
|
|
596
|
-
|
|
597
|
-
```bash
|
|
598
|
-
# Install the attention module (optional)
|
|
599
|
-
npm install @ruvector/attention
|
|
600
|
-
```
|
|
601
|
-
|
|
602
|
-
#### Attention Mechanisms Reference
|
|
603
|
-
|
|
604
|
-
| Mechanism | Type | Complexity | When to Use |
|
|
605
|
-
|-----------|------|------------|-------------|
|
|
606
|
-
| **DotProductAttention** | Core | O(n²) | Standard scaled dot-product attention for transformers |
|
|
607
|
-
| **MultiHeadAttention** | Core | O(n²) | Parallel attention heads for capturing different relationships |
|
|
608
|
-
| **FlashAttention** | Core | O(n²) IO-optimized | Memory-efficient attention for long sequences |
|
|
609
|
-
| **HyperbolicAttention** | Core | O(n²) | Hierarchical data, tree-like structures, taxonomies |
|
|
610
|
-
| **LinearAttention** | Core | O(n) | Very long sequences where O(n²) is prohibitive |
|
|
611
|
-
| **MoEAttention** | Core | O(n*k) | Mixture of Experts routing, specialized attention |
|
|
612
|
-
| **GraphRoPeAttention** | Graph | O(n²) | Graph data with rotary position embeddings |
|
|
613
|
-
| **EdgeFeaturedAttention** | Graph | O(n²) | Graphs with rich edge features/attributes |
|
|
614
|
-
| **DualSpaceAttention** | Graph | O(n²) | Combined Euclidean + hyperbolic representation |
|
|
615
|
-
| **LocalGlobalAttention** | Graph | O(n*k) | Large graphs with local + global context |
|
|
616
|
-
|
|
617
|
-
#### Attention Info
|
|
618
|
-
|
|
619
|
-
```bash
|
|
620
|
-
# Show attention module information
|
|
621
|
-
npx ruvector attention info
|
|
622
|
-
|
|
623
|
-
# Output:
|
|
624
|
-
# Attention Module Information
|
|
625
|
-
# Status: Available
|
|
626
|
-
# Version: 0.1.0
|
|
627
|
-
# Platform: linux
|
|
628
|
-
# Architecture: x64
|
|
629
|
-
#
|
|
630
|
-
# Core Attention Mechanisms:
|
|
631
|
-
# • DotProductAttention - Scaled dot-product attention
|
|
632
|
-
# • MultiHeadAttention - Multi-head self-attention
|
|
633
|
-
# • FlashAttention - Memory-efficient IO-aware attention
|
|
634
|
-
# • HyperbolicAttention - Poincaré ball attention
|
|
635
|
-
# • LinearAttention - O(n) linear complexity attention
|
|
636
|
-
# • MoEAttention - Mixture of Experts attention
|
|
637
|
-
```
|
|
638
|
-
|
|
639
|
-
#### Attention List
|
|
640
|
-
|
|
641
|
-
```bash
|
|
642
|
-
# List all available attention mechanisms
|
|
643
|
-
npx ruvector attention list
|
|
644
|
-
|
|
645
|
-
# With verbose details
|
|
646
|
-
npx ruvector attention list -v
|
|
647
|
-
```
|
|
648
|
-
|
|
649
|
-
#### Attention Benchmark
|
|
650
|
-
|
|
651
|
-
```bash
|
|
652
|
-
# Benchmark attention mechanisms
|
|
653
|
-
npx ruvector attention benchmark -d 256 -n 100 -i 100
|
|
654
|
-
|
|
655
|
-
# Options:
|
|
656
|
-
# -d, --dimension Vector dimension (default: 256)
|
|
657
|
-
# -n, --num-vectors Number of vectors (default: 100)
|
|
658
|
-
# -i, --iterations Benchmark iterations (default: 100)
|
|
659
|
-
# -t, --types Attention types to benchmark (default: dot,flash,linear)
|
|
660
|
-
|
|
661
|
-
# Example output:
|
|
662
|
-
# Dimension: 256
|
|
663
|
-
# Vectors: 100
|
|
664
|
-
# Iterations: 100
|
|
665
|
-
#
|
|
666
|
-
# dot: 0.012ms/op (84,386 ops/sec)
|
|
667
|
-
# flash: 0.012ms/op (82,844 ops/sec)
|
|
668
|
-
# linear: 0.066ms/op (15,259 ops/sec)
|
|
669
|
-
```
|
|
670
|
-
|
|
671
|
-
#### Hyperbolic Operations
|
|
672
|
-
|
|
673
|
-
```bash
|
|
674
|
-
# Calculate Poincaré distance between two points
|
|
675
|
-
npx ruvector attention hyperbolic -a distance -v "[0.1,0.2,0.3]" -b "[0.4,0.5,0.6]"
|
|
676
|
-
|
|
677
|
-
# Project vector to Poincaré ball
|
|
678
|
-
npx ruvector attention hyperbolic -a project -v "[1.5,2.0,0.8]"
|
|
679
|
-
|
|
680
|
-
# Möbius addition in hyperbolic space
|
|
681
|
-
npx ruvector attention hyperbolic -a mobius-add -v "[0.1,0.2]" -b "[0.3,0.4]"
|
|
682
|
-
|
|
683
|
-
# Exponential map (tangent space → Poincaré ball)
|
|
684
|
-
npx ruvector attention hyperbolic -a exp-map -v "[0.1,0.2,0.3]"
|
|
685
|
-
|
|
686
|
-
# Options:
|
|
687
|
-
# -a, --action Action: distance|project|mobius-add|exp-map|log-map
|
|
688
|
-
# -v, --vector Input vector as JSON array (required)
|
|
689
|
-
# -b, --vector-b Second vector for binary operations
|
|
690
|
-
# -c, --curvature Poincaré ball curvature (default: 1.0)
|
|
691
|
-
```
|
|
692
|
-
|
|
693
|
-
#### When to Use Each Attention Type
|
|
694
|
-
|
|
695
|
-
| Use Case | Recommended Attention | Reason |
|
|
696
|
-
|----------|----------------------|--------|
|
|
697
|
-
| **Standard NLP/Transformers** | MultiHeadAttention | Industry standard, well-tested |
|
|
698
|
-
| **Long Documents (>4K tokens)** | FlashAttention or LinearAttention | Memory efficient |
|
|
699
|
-
| **Hierarchical Classification** | HyperbolicAttention | Captures tree-like structures |
|
|
700
|
-
| **Knowledge Graphs** | GraphRoPeAttention | Position-aware graph attention |
|
|
701
|
-
| **Multi-Relational Graphs** | EdgeFeaturedAttention | Leverages edge attributes |
|
|
702
|
-
| **Taxonomy/Ontology Search** | DualSpaceAttention | Best of both Euclidean + hyperbolic |
|
|
703
|
-
| **Large-Scale Graphs** | LocalGlobalAttention | Efficient local + global context |
|
|
704
|
-
| **Model Routing/MoE** | MoEAttention | Expert selection and routing |
|
|
705
|
-
|
|
706
|
-
### ⚡ ONNX WASM Embeddings (v2.0)
|
|
707
|
-
|
|
708
|
-
RuVector includes a pure JavaScript ONNX runtime for local embeddings - no Python, no API calls, no build tools required.
|
|
709
|
-
|
|
710
|
-
```bash
|
|
711
|
-
# Embeddings work out of the box
|
|
712
|
-
npx ruvector hooks remember "important context" -t project
|
|
713
|
-
npx ruvector hooks recall "context query"
|
|
714
|
-
npx ruvector hooks rag-context "how does auth work"
|
|
715
|
-
```
|
|
716
|
-
|
|
717
|
-
**Model**: all-MiniLM-L6-v2 (384 dimensions, 23MB)
|
|
718
|
-
- Downloads automatically on first use
|
|
719
|
-
- Cached in `.ruvector/models/`
|
|
720
|
-
- SIMD-accelerated when available
|
|
721
|
-
|
|
722
|
-
**Performance:**
|
|
723
|
-
| Operation | Time | Notes |
|
|
724
|
-
|-----------|------|-------|
|
|
725
|
-
| Model load | ~2s | First use only |
|
|
726
|
-
| Embedding | ~50ms | Per text chunk |
|
|
727
|
-
| HNSW search | 0.045ms | 150x faster than brute force |
|
|
728
|
-
| Cache hit | 0.01ms | 40,000x faster than inference |
|
|
729
|
-
|
|
730
|
-
**Fallback Chain:**
|
|
731
|
-
1. Native SQLite → best persistence
|
|
732
|
-
2. WASM SQLite → cross-platform
|
|
733
|
-
3. Memory Cache → fastest (no persistence)
|
|
734
|
-
|
|
735
|
-
### 🧠 Self-Learning Hooks v2.0
|
|
736
|
-
|
|
737
|
-
Ruvector includes **self-learning intelligence hooks** for Claude Code integration with ONNX embeddings, AST analysis, and coverage-aware routing.
|
|
738
|
-
|
|
739
|
-
#### Initialize Hooks
|
|
740
|
-
|
|
741
|
-
```bash
|
|
742
|
-
# Initialize hooks in your project
|
|
743
|
-
npx ruvector hooks init
|
|
744
|
-
|
|
745
|
-
# Options:
|
|
746
|
-
# --force Overwrite existing configuration
|
|
747
|
-
# --minimal Minimal configuration (no optional hooks)
|
|
748
|
-
# --pretrain Initialize + pretrain from git history
|
|
749
|
-
# --build-agents quality Generate optimized agent configs
|
|
750
|
-
```
|
|
751
|
-
|
|
752
|
-
This creates `.claude/settings.json` with pre-configured hooks and `CLAUDE.md` with comprehensive documentation.
|
|
753
|
-
|
|
754
|
-
#### Session Management
|
|
755
|
-
|
|
756
|
-
```bash
|
|
757
|
-
# Start a session (load intelligence data)
|
|
758
|
-
npx ruvector hooks session-start
|
|
759
|
-
|
|
760
|
-
# End a session (save learned patterns)
|
|
761
|
-
npx ruvector hooks session-end
|
|
762
|
-
```
|
|
763
|
-
|
|
764
|
-
#### Pre/Post Edit Hooks
|
|
765
|
-
|
|
766
|
-
```bash
|
|
767
|
-
# Before editing a file - get agent recommendations
|
|
768
|
-
npx ruvector hooks pre-edit src/index.ts
|
|
769
|
-
# Output: 🤖 Recommended: typescript-developer (85% confidence)
|
|
770
|
-
|
|
771
|
-
# After editing - record success/failure for learning
|
|
772
|
-
npx ruvector hooks post-edit src/index.ts --success
|
|
773
|
-
npx ruvector hooks post-edit src/index.ts --error "Type error on line 42"
|
|
774
|
-
```
|
|
775
|
-
|
|
776
|
-
#### Pre/Post Command Hooks
|
|
777
|
-
|
|
778
|
-
```bash
|
|
779
|
-
# Before running a command - risk analysis
|
|
780
|
-
npx ruvector hooks pre-command "npm test"
|
|
781
|
-
# Output: ✅ Risk: LOW, Category: test
|
|
782
|
-
|
|
783
|
-
# After running - record outcome
|
|
784
|
-
npx ruvector hooks post-command "npm test" --success
|
|
785
|
-
npx ruvector hooks post-command "npm test" --error "3 tests failed"
|
|
786
|
-
```
|
|
787
|
-
|
|
788
|
-
#### Agent Routing
|
|
789
|
-
|
|
790
|
-
```bash
|
|
791
|
-
# Get agent recommendation for a task
|
|
792
|
-
npx ruvector hooks route "fix the authentication bug in login.ts"
|
|
793
|
-
# Output: 🤖 Recommended: security-specialist (92% confidence)
|
|
794
|
-
|
|
795
|
-
npx ruvector hooks route "add unit tests for the API"
|
|
796
|
-
# Output: 🤖 Recommended: tester (88% confidence)
|
|
797
|
-
```
|
|
798
|
-
|
|
799
|
-
#### Memory Operations
|
|
800
|
-
|
|
801
|
-
```bash
|
|
802
|
-
# Store context in vector memory
|
|
803
|
-
npx ruvector hooks remember "API uses JWT tokens with 1h expiry" --type decision
|
|
804
|
-
npx ruvector hooks remember "Database schema in docs/schema.md" --type reference
|
|
805
|
-
|
|
806
|
-
# Semantic search memory
|
|
807
|
-
npx ruvector hooks recall "authentication mechanism"
|
|
808
|
-
# Returns relevant stored memories
|
|
809
|
-
```
|
|
810
|
-
|
|
811
|
-
#### Context Suggestions
|
|
812
|
-
|
|
813
|
-
```bash
|
|
814
|
-
# Get relevant context for current task
|
|
815
|
-
npx ruvector hooks suggest-context
|
|
816
|
-
# Output: Based on recent files, suggests relevant context
|
|
817
|
-
```
|
|
818
|
-
|
|
819
|
-
#### Intelligence Statistics
|
|
820
|
-
|
|
821
|
-
```bash
|
|
822
|
-
# Show learned patterns and statistics
|
|
823
|
-
npx ruvector hooks stats
|
|
824
|
-
|
|
825
|
-
# Output:
|
|
826
|
-
# Patterns: 156 learned
|
|
827
|
-
# Success rate: 87%
|
|
828
|
-
# Top agents: rust-developer, tester, reviewer
|
|
829
|
-
# Memory entries: 42
|
|
830
|
-
```
|
|
831
|
-
|
|
832
|
-
#### Swarm Recommendations
|
|
833
|
-
|
|
834
|
-
```bash
|
|
835
|
-
# Get agent recommendation for task type
|
|
836
|
-
npx ruvector hooks swarm-recommend "code-review"
|
|
837
|
-
# Output: Recommended agents for code review task
|
|
838
|
-
```
|
|
839
|
-
|
|
840
|
-
#### AST Analysis (v2.0)
|
|
841
|
-
|
|
842
|
-
```bash
|
|
843
|
-
# Analyze file structure, symbols, imports, complexity
|
|
844
|
-
npx ruvector hooks ast-analyze src/index.ts --json
|
|
845
|
-
|
|
846
|
-
# Get complexity metrics for multiple files
|
|
847
|
-
npx ruvector hooks ast-complexity src/*.ts --threshold 15
|
|
848
|
-
# Flags files exceeding cyclomatic complexity threshold
|
|
849
|
-
```
|
|
850
|
-
|
|
851
|
-
#### Diff & Risk Analysis (v2.0)
|
|
852
|
-
|
|
853
|
-
```bash
|
|
854
|
-
# Analyze commit with semantic embeddings and risk scoring
|
|
855
|
-
npx ruvector hooks diff-analyze HEAD
|
|
856
|
-
# Output: risk score, category, affected files
|
|
857
|
-
|
|
858
|
-
# Classify change type (feature, bugfix, refactor, docs, test)
|
|
859
|
-
npx ruvector hooks diff-classify
|
|
860
|
-
|
|
861
|
-
# Find similar past commits via embeddings
|
|
862
|
-
npx ruvector hooks diff-similar -k 5
|
|
863
|
-
|
|
864
|
-
# Git churn analysis (hot spots)
|
|
865
|
-
npx ruvector hooks git-churn --days 30
|
|
866
|
-
```
|
|
867
|
-
|
|
868
|
-
#### Coverage-Aware Routing (v2.0)
|
|
869
|
-
|
|
870
|
-
```bash
|
|
871
|
-
# Get coverage-aware routing for a file
|
|
872
|
-
npx ruvector hooks coverage-route src/api.ts
|
|
873
|
-
# Output: agent weights based on test coverage
|
|
874
|
-
|
|
875
|
-
# Suggest tests for files based on coverage gaps
|
|
876
|
-
npx ruvector hooks coverage-suggest src/*.ts
|
|
877
|
-
```
|
|
878
|
-
|
|
879
|
-
#### Graph Analysis (v2.0)
|
|
880
|
-
|
|
881
|
-
```bash
|
|
882
|
-
# Find optimal code boundaries (MinCut algorithm)
|
|
883
|
-
npx ruvector hooks graph-mincut src/*.ts
|
|
884
|
-
|
|
885
|
-
# Detect code communities (Louvain/Spectral clustering)
|
|
886
|
-
npx ruvector hooks graph-cluster src/*.ts --method louvain
|
|
887
|
-
```
|
|
888
|
-
|
|
889
|
-
#### Security & RAG (v2.0)
|
|
890
|
-
|
|
891
|
-
```bash
|
|
892
|
-
# Parallel security vulnerability scan
|
|
893
|
-
npx ruvector hooks security-scan src/*.ts
|
|
894
|
-
|
|
895
|
-
# RAG-enhanced context retrieval
|
|
896
|
-
npx ruvector hooks rag-context "how does auth work"
|
|
897
|
-
|
|
898
|
-
# Enhanced routing with all signals
|
|
899
|
-
npx ruvector hooks route-enhanced "fix bug" --file src/api.ts
|
|
900
|
-
```
|
|
901
|
-
|
|
902
|
-
#### Hooks Configuration
|
|
903
|
-
|
|
904
|
-
The hooks integrate with Claude Code via `.claude/settings.json`:
|
|
905
|
-
|
|
906
|
-
```json
|
|
907
|
-
{
|
|
908
|
-
"env": {
|
|
909
|
-
"RUVECTOR_INTELLIGENCE_ENABLED": "true",
|
|
910
|
-
"RUVECTOR_LEARNING_RATE": "0.1",
|
|
911
|
-
"RUVECTOR_AST_ENABLED": "true",
|
|
912
|
-
"RUVECTOR_DIFF_EMBEDDINGS": "true",
|
|
913
|
-
"RUVECTOR_COVERAGE_ROUTING": "true",
|
|
914
|
-
"RUVECTOR_GRAPH_ALGORITHMS": "true",
|
|
915
|
-
"RUVECTOR_SECURITY_SCAN": "true"
|
|
916
|
-
},
|
|
917
|
-
"hooks": {
|
|
918
|
-
"PreToolUse": [
|
|
919
|
-
{
|
|
920
|
-
"matcher": "Edit|Write|MultiEdit",
|
|
921
|
-
"hooks": [{ "type": "command", "command": "npx ruvector hooks pre-edit \"$TOOL_INPUT_file_path\"" }]
|
|
922
|
-
},
|
|
923
|
-
{
|
|
924
|
-
"matcher": "Bash",
|
|
925
|
-
"hooks": [{ "type": "command", "command": "npx ruvector hooks pre-command \"$TOOL_INPUT_command\"" }]
|
|
926
|
-
}
|
|
927
|
-
],
|
|
928
|
-
"PostToolUse": [
|
|
929
|
-
{
|
|
930
|
-
"matcher": "Edit|Write|MultiEdit",
|
|
931
|
-
"hooks": [{ "type": "command", "command": "npx ruvector hooks post-edit \"$TOOL_INPUT_file_path\"" }]
|
|
932
|
-
}
|
|
933
|
-
],
|
|
934
|
-
"SessionStart": [{ "hooks": [{ "type": "command", "command": "npx ruvector hooks session-start" }] }],
|
|
935
|
-
"Stop": [{ "hooks": [{ "type": "command", "command": "npx ruvector hooks session-end" }] }]
|
|
936
|
-
}
|
|
937
|
-
}
|
|
938
|
-
```
|
|
939
|
-
|
|
940
|
-
#### How Self-Learning Works
|
|
941
|
-
|
|
942
|
-
1. **Pattern Recording**: Every edit and command is recorded with context
|
|
943
|
-
2. **Q-Learning**: Success/failure updates agent routing weights
|
|
944
|
-
3. **AST Analysis**: Code complexity informs agent selection
|
|
945
|
-
4. **Diff Embeddings**: Change patterns improve risk assessment
|
|
946
|
-
5. **Coverage Routing**: Test coverage guides testing priorities
|
|
947
|
-
6. **Vector Memory**: Decisions and references stored for semantic recall (HNSW indexed)
|
|
948
|
-
7. **Continuous Improvement**: The more you use it, the smarter it gets
|
|
949
|
-
|
|
950
|
-
## 📊 Performance Benchmarks
|
|
951
|
-
|
|
952
|
-
Tested on AMD Ryzen 9 5950X, 128-dimensional vectors:
|
|
953
|
-
|
|
954
|
-
### Native Performance (Rust)
|
|
955
|
-
|
|
956
|
-
| Operation | Throughput | Latency (p50) | Latency (p99) |
|
|
957
|
-
|-----------|------------|---------------|---------------|
|
|
958
|
-
| Insert | 52,341 ops/sec | 0.019 ms | 0.045 ms |
|
|
959
|
-
| Search (k=10) | 11,234 ops/sec | 0.089 ms | 0.156 ms |
|
|
960
|
-
| Search (k=100) | 8,932 ops/sec | 0.112 ms | 0.203 ms |
|
|
961
|
-
| Delete | 45,678 ops/sec | 0.022 ms | 0.051 ms |
|
|
962
|
-
|
|
963
|
-
**Memory Usage**: ~50 bytes per 128-dim vector (including index)
|
|
964
|
-
|
|
965
|
-
### Comparison with Alternatives
|
|
966
|
-
|
|
967
|
-
| Database | Insert (ops/sec) | Search (ops/sec) | Memory per Vector | Node.js | Browser |
|
|
968
|
-
|----------|------------------|------------------|-------------------|---------|---------|
|
|
969
|
-
| **Ruvector (Native)** | **52,341** | **11,234** | **50 bytes** | ✅ | ❌ |
|
|
970
|
-
| **Ruvector (WASM)** | **~1,000** | **~100** | **50 bytes** | ✅ | ✅ |
|
|
971
|
-
| Faiss (HNSW) | 38,200 | 9,800 | 68 bytes | ❌ | ❌ |
|
|
972
|
-
| Hnswlib | 41,500 | 10,200 | 62 bytes | ✅ | ❌ |
|
|
973
|
-
| ChromaDB | ~1,000 | ~20 | 150 bytes | ✅ | ❌ |
|
|
974
|
-
|
|
975
|
-
*Benchmarks measured with 100K vectors, 128 dimensions, k=10*
|
|
976
|
-
|
|
977
|
-
## 🔍 Comparison with Other Vector Databases
|
|
978
|
-
|
|
979
|
-
Comprehensive comparison of Ruvector against popular vector database solutions:
|
|
980
|
-
|
|
981
|
-
| Feature | Ruvector | Pinecone | Qdrant | Weaviate | Milvus | ChromaDB | Faiss |
|
|
982
|
-
|---------|----------|----------|--------|----------|--------|----------|-------|
|
|
983
|
-
| **Deployment** |
|
|
984
|
-
| Installation | `npm install` ✅ | Cloud API ☁️ | Docker 🐳 | Docker 🐳 | Docker/K8s 🐳 | `pip install` 🐍 | `pip install` 🐍 |
|
|
985
|
-
| Node.js Native | ✅ First-class | ❌ API only | ⚠️ HTTP API | ⚠️ HTTP API | ⚠️ HTTP API | ❌ Python | ❌ Python |
|
|
986
|
-
| Setup Time | < 1 minute | 5-10 minutes | 10-30 minutes | 15-30 minutes | 30-60 minutes | 5 minutes | 5 minutes |
|
|
987
|
-
| Infrastructure | None required | Managed cloud | Self-hosted | Self-hosted | Self-hosted | Embedded | Embedded |
|
|
988
|
-
| **Performance** |
|
|
989
|
-
| Query Latency (p50) | **<0.5ms** | ~2-5ms | ~1-2ms | ~2-3ms | ~3-5ms | ~50ms | ~1ms |
|
|
990
|
-
| Insert Throughput | **52,341 ops/sec** | ~10,000 ops/sec | ~20,000 ops/sec | ~15,000 ops/sec | ~25,000 ops/sec | ~1,000 ops/sec | ~40,000 ops/sec |
|
|
991
|
-
| Memory per Vector (128d) | **50 bytes** | ~80 bytes | 62 bytes | ~100 bytes | ~70 bytes | 150 bytes | 68 bytes |
|
|
992
|
-
| Recall @ k=10 | 95%+ | 93% | 94% | 92% | 96% | 85% | 97% |
|
|
993
|
-
| **Platform Support** |
|
|
994
|
-
| Linux | ✅ Native | ☁️ API | ✅ Docker | ✅ Docker | ✅ Docker | ✅ Python | ✅ Python |
|
|
995
|
-
| macOS | ✅ Native | ☁️ API | ✅ Docker | ✅ Docker | ✅ Docker | ✅ Python | ✅ Python |
|
|
996
|
-
| Windows | ✅ Native | ☁️ API | ✅ Docker | ✅ Docker | ⚠️ WSL2 | ✅ Python | ✅ Python |
|
|
997
|
-
| Browser/WASM | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
|
|
998
|
-
| ARM64 | ✅ Native | ☁️ API | ✅ Yes | ✅ Yes | ⚠️ Limited | ✅ Yes | ✅ Yes |
|
|
999
|
-
| Alpine Linux | ✅ WASM | ☁️ API | ⚠️ Build from source | ⚠️ Build from source | ❌ No | ✅ Yes | ✅ Yes |
|
|
1000
|
-
| **Features** |
|
|
1001
|
-
| Distance Metrics | Cosine, L2, Dot | Cosine, L2, Dot | 11 metrics | 10 metrics | 8 metrics | L2, Cosine, IP | L2, IP, Cosine |
|
|
1002
|
-
| Filtering | ✅ Metadata | ✅ Advanced | ✅ Advanced | ✅ Advanced | ✅ Advanced | ✅ Basic | ❌ Limited |
|
|
1003
|
-
| Persistence | ✅ File-based | ☁️ Managed | ✅ Disk | ✅ Disk | ✅ Disk | ✅ DuckDB | ❌ Memory |
|
|
1004
|
-
| Indexing | HNSW | Proprietary | HNSW | HNSW | IVF/HNSW | HNSW | IVF/HNSW |
|
|
1005
|
-
| Quantization | ✅ PQ | ✅ Yes | ✅ Scalar | ✅ PQ | ✅ PQ/SQ | ❌ No | ✅ PQ |
|
|
1006
|
-
| Batch Operations | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
|
|
1007
|
-
| **Developer Experience** |
|
|
1008
|
-
| TypeScript Types | ✅ Full | ✅ Generated | ⚠️ Community | ⚠️ Community | ⚠️ Community | ⚠️ Partial | ❌ No |
|
|
1009
|
-
| Documentation | ✅ Excellent | ✅ Excellent | ✅ Good | ✅ Good | ✅ Good | ✅ Good | ⚠️ Technical |
|
|
1010
|
-
| Examples | ✅ Many | ✅ Many | ✅ Good | ✅ Good | ✅ Many | ✅ Good | ⚠️ Limited |
|
|
1011
|
-
| CLI Tools | ✅ Included | ⚠️ Limited | ✅ Yes | ✅ Yes | ✅ Yes | ⚠️ Basic | ❌ No |
|
|
1012
|
-
| **Operations** |
|
|
1013
|
-
| Monitoring | ✅ Metrics | ✅ Dashboard | ✅ Prometheus | ✅ Prometheus | ✅ Prometheus | ⚠️ Basic | ❌ No |
|
|
1014
|
-
| Backups | ✅ File copy | ☁️ Automatic | ✅ Snapshots | ✅ Snapshots | ✅ Snapshots | ✅ File copy | ❌ Manual |
|
|
1015
|
-
| High Availability | ⚠️ App-level | ✅ Built-in | ✅ Clustering | ✅ Clustering | ✅ Clustering | ❌ No | ❌ No |
|
|
1016
|
-
| Auto-Scaling | ⚠️ App-level | ✅ Automatic | ⚠️ Manual | ⚠️ Manual | ⚠️ K8s HPA | ❌ No | ❌ No |
|
|
1017
|
-
| **Cost** |
|
|
1018
|
-
| Pricing Model | Free (MIT) | Pay-per-use | Free (Apache) | Free (BSD) | Free (Apache) | Free (Apache) | Free (MIT) |
|
|
1019
|
-
| Monthly Cost (1M vectors) | **$0** | ~$70-200 | ~$20-50 (infra) | ~$30-60 (infra) | ~$50-100 (infra) | $0 | $0 |
|
|
1020
|
-
| Monthly Cost (10M vectors) | **$0** | ~$500-1000 | ~$100-200 (infra) | ~$150-300 (infra) | ~$200-400 (infra) | $0 | $0 |
|
|
1021
|
-
| API Rate Limits | None | Yes | None | None | None | None | None |
|
|
1022
|
-
| **Use Cases** |
|
|
1023
|
-
| RAG Systems | ✅ Excellent | ✅ Excellent | ✅ Excellent | ✅ Excellent | ✅ Excellent | ✅ Good | ⚠️ Limited |
|
|
1024
|
-
| Serverless | ✅ Perfect | ✅ Good | ❌ No | ❌ No | ❌ No | ⚠️ Possible | ⚠️ Possible |
|
|
1025
|
-
| Edge Computing | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ⚠️ Possible |
|
|
1026
|
-
| Production Scale (100M+) | ⚠️ Single node | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Excellent | ⚠️ Limited | ⚠️ Manual |
|
|
1027
|
-
| Embedded Apps | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ⚠️ Possible | ✅ Good |
|
|
1028
|
-
|
|
1029
|
-
### When to Choose Ruvector
|
|
1030
|
-
|
|
1031
|
-
✅ **Perfect for:**
|
|
1032
|
-
- **Node.js/TypeScript applications** needing embedded vector search
|
|
1033
|
-
- **Serverless and edge computing** where external services aren't practical
|
|
1034
|
-
- **Rapid prototyping and development** with minimal setup time
|
|
1035
|
-
- **RAG systems** with LangChain, LlamaIndex, or custom implementations
|
|
1036
|
-
- **Cost-sensitive projects** that can't afford cloud API pricing
|
|
1037
|
-
- **Offline-first applications** requiring local vector search
|
|
1038
|
-
- **Browser-based AI** with WebAssembly fallback
|
|
1039
|
-
- **Small to medium scale** (up to 10M vectors per instance)
|
|
1040
|
-
|
|
1041
|
-
⚠️ **Consider alternatives for:**
|
|
1042
|
-
- **Massive scale (100M+ vectors)** - Consider Pinecone, Milvus, or Qdrant clusters
|
|
1043
|
-
- **Multi-tenancy requirements** - Weaviate or Qdrant offer better isolation
|
|
1044
|
-
- **Distributed systems** - Milvus provides better horizontal scaling
|
|
1045
|
-
- **Zero-ops cloud solution** - Pinecone handles all infrastructure
|
|
1046
|
-
|
|
1047
|
-
### Why Choose Ruvector Over...
|
|
1048
|
-
|
|
1049
|
-
**vs Pinecone:**
|
|
1050
|
-
- ✅ No API costs (save $1000s/month)
|
|
1051
|
-
- ✅ No network latency (10x faster queries)
|
|
1052
|
-
- ✅ No vendor lock-in
|
|
1053
|
-
- ✅ Works offline and in restricted environments
|
|
1054
|
-
- ❌ No managed multi-region clusters
|
|
1055
|
-
|
|
1056
|
-
**vs ChromaDB:**
|
|
1057
|
-
- ✅ 50x faster queries (native Rust vs Python)
|
|
1058
|
-
- ✅ True Node.js support (not HTTP API)
|
|
1059
|
-
- ✅ Better TypeScript integration
|
|
1060
|
-
- ✅ Lower memory usage
|
|
1061
|
-
- ❌ Smaller ecosystem and community
|
|
1062
|
-
|
|
1063
|
-
**vs Qdrant:**
|
|
1064
|
-
- ✅ Zero infrastructure setup
|
|
1065
|
-
- ✅ Embedded in your app (no Docker)
|
|
1066
|
-
- ✅ Better for serverless environments
|
|
1067
|
-
- ✅ Native Node.js bindings
|
|
1068
|
-
- ❌ No built-in clustering or HA
|
|
1069
|
-
|
|
1070
|
-
**vs Faiss:**
|
|
1071
|
-
- ✅ Full Node.js support (Faiss is Python-only)
|
|
1072
|
-
- ✅ Easier API and better developer experience
|
|
1073
|
-
- ✅ Built-in persistence and metadata
|
|
1074
|
-
- ⚠️ Slightly lower recall at same performance
|
|
1075
|
-
|
|
1076
|
-
## 🎯 Real-World Tutorials
|
|
1077
|
-
|
|
1078
|
-
### Tutorial 1: Building a RAG System with OpenAI
|
|
1079
|
-
|
|
1080
|
-
**What you'll learn:** Create a production-ready Retrieval-Augmented Generation system that enhances LLM responses with relevant context from your documents.
|
|
1081
|
-
|
|
1082
|
-
**Prerequisites:**
|
|
1083
|
-
```bash
|
|
1084
|
-
npm install ruvector openai
|
|
1085
|
-
export OPENAI_API_KEY="your-api-key-here"
|
|
1086
|
-
```
|
|
1087
|
-
|
|
1088
|
-
**Complete Implementation:**
|
|
1089
|
-
|
|
1090
|
-
```javascript
|
|
1091
|
-
const { VectorDb } = require('ruvector');
|
|
1092
|
-
const OpenAI = require('openai');
|
|
1093
|
-
|
|
1094
|
-
class RAGSystem {
|
|
1095
|
-
constructor() {
|
|
1096
|
-
// Initialize OpenAI client
|
|
1097
|
-
this.openai = new OpenAI({
|
|
1098
|
-
apiKey: process.env.OPENAI_API_KEY
|
|
1099
|
-
});
|
|
1100
|
-
|
|
1101
|
-
// Create vector database for OpenAI embeddings
|
|
1102
|
-
// text-embedding-ada-002 produces 1536-dimensional vectors
|
|
1103
|
-
this.db = new VectorDb({
|
|
1104
|
-
dimensions: 1536,
|
|
1105
|
-
maxElements: 100000,
|
|
1106
|
-
storagePath: './rag-knowledge-base.db'
|
|
1107
|
-
});
|
|
1108
|
-
|
|
1109
|
-
console.log('✅ RAG System initialized');
|
|
1110
|
-
}
|
|
1111
|
-
|
|
1112
|
-
// Step 1: Index your knowledge base
|
|
1113
|
-
async indexDocuments(documents) {
|
|
1114
|
-
console.log(`📚 Indexing ${documents.length} documents...`);
|
|
1115
|
-
|
|
1116
|
-
for (let i = 0; i < documents.length; i++) {
|
|
1117
|
-
const doc = documents[i];
|
|
1118
|
-
|
|
1119
|
-
// Generate embedding for the document
|
|
1120
|
-
const response = await this.openai.embeddings.create({
|
|
1121
|
-
model: 'text-embedding-ada-002',
|
|
1122
|
-
input: doc.content
|
|
1123
|
-
});
|
|
1124
|
-
|
|
1125
|
-
// Store in vector database
|
|
1126
|
-
await this.db.insert({
|
|
1127
|
-
id: doc.id || `doc_${i}`,
|
|
1128
|
-
vector: new Float32Array(response.data[0].embedding),
|
|
1129
|
-
metadata: {
|
|
1130
|
-
title: doc.title,
|
|
1131
|
-
content: doc.content,
|
|
1132
|
-
source: doc.source,
|
|
1133
|
-
date: doc.date || new Date().toISOString()
|
|
1134
|
-
}
|
|
1135
|
-
});
|
|
1136
|
-
|
|
1137
|
-
console.log(` ✅ Indexed: ${doc.title}`);
|
|
1138
|
-
}
|
|
1139
|
-
|
|
1140
|
-
const count = await this.db.len();
|
|
1141
|
-
console.log(`\n✅ Indexed ${count} documents total`);
|
|
1142
|
-
}
|
|
1143
|
-
|
|
1144
|
-
// Step 2: Retrieve relevant context for a query
|
|
1145
|
-
async retrieveContext(query, k = 3) {
|
|
1146
|
-
console.log(`🔍 Searching for: "${query}"`);
|
|
1147
|
-
|
|
1148
|
-
// Generate embedding for the query
|
|
1149
|
-
const response = await this.openai.embeddings.create({
|
|
1150
|
-
model: 'text-embedding-ada-002',
|
|
1151
|
-
input: query
|
|
1152
|
-
});
|
|
1153
|
-
|
|
1154
|
-
// Search for similar documents
|
|
1155
|
-
const results = await this.db.search({
|
|
1156
|
-
vector: new Float32Array(response.data[0].embedding),
|
|
1157
|
-
k: k,
|
|
1158
|
-
threshold: 0.7 // Only use highly relevant results
|
|
1159
|
-
});
|
|
1160
|
-
|
|
1161
|
-
console.log(`📄 Found ${results.length} relevant documents\n`);
|
|
1162
|
-
|
|
1163
|
-
return results.map(r => ({
|
|
1164
|
-
content: r.metadata.content,
|
|
1165
|
-
title: r.metadata.title,
|
|
1166
|
-
score: r.score
|
|
1167
|
-
}));
|
|
1168
|
-
}
|
|
1169
|
-
|
|
1170
|
-
// Step 3: Generate answer with retrieved context
|
|
1171
|
-
async answer(question) {
|
|
1172
|
-
// Retrieve relevant context
|
|
1173
|
-
const context = await this.retrieveContext(question, 3);
|
|
1174
|
-
|
|
1175
|
-
if (context.length === 0) {
|
|
1176
|
-
return "I don't have enough information to answer that question.";
|
|
1177
|
-
}
|
|
1178
|
-
|
|
1179
|
-
// Build prompt with context
|
|
1180
|
-
const contextText = context
|
|
1181
|
-
.map((doc, i) => `[${i + 1}] ${doc.title}\n${doc.content}`)
|
|
1182
|
-
.join('\n\n');
|
|
1183
|
-
|
|
1184
|
-
const prompt = `Answer the question based on the following context. If the context doesn't contain the answer, say so.
|
|
1185
|
-
|
|
1186
|
-
Context:
|
|
1187
|
-
${contextText}
|
|
1188
|
-
|
|
1189
|
-
Question: ${question}
|
|
1190
|
-
|
|
1191
|
-
Answer:`;
|
|
1192
|
-
|
|
1193
|
-
console.log('🤖 Generating answer...\n');
|
|
1194
|
-
|
|
1195
|
-
// Generate completion
|
|
1196
|
-
const completion = await this.openai.chat.completions.create({
|
|
1197
|
-
model: 'gpt-4',
|
|
1198
|
-
messages: [
|
|
1199
|
-
{ role: 'system', content: 'You are a helpful assistant that answers questions based on provided context.' },
|
|
1200
|
-
{ role: 'user', content: prompt }
|
|
1201
|
-
],
|
|
1202
|
-
temperature: 0.3 // Lower temperature for more factual responses
|
|
1203
|
-
});
|
|
1204
|
-
|
|
1205
|
-
return {
|
|
1206
|
-
answer: completion.choices[0].message.content,
|
|
1207
|
-
sources: context.map(c => c.title)
|
|
1208
|
-
};
|
|
1209
|
-
}
|
|
1210
|
-
}
|
|
1211
|
-
|
|
1212
|
-
// Example Usage
|
|
1213
|
-
async function main() {
|
|
1214
|
-
const rag = new RAGSystem();
|
|
1215
|
-
|
|
1216
|
-
// Step 1: Index your knowledge base
|
|
1217
|
-
const documents = [
|
|
1218
|
-
{
|
|
1219
|
-
id: 'doc1',
|
|
1220
|
-
title: 'Ruvector Introduction',
|
|
1221
|
-
content: 'Ruvector is a high-performance vector database for Node.js built in Rust. It provides sub-millisecond query latency and supports over 52,000 inserts per second.',
|
|
1222
|
-
source: 'documentation'
|
|
1223
|
-
},
|
|
1224
|
-
{
|
|
1225
|
-
id: 'doc2',
|
|
1226
|
-
title: 'Vector Databases Explained',
|
|
1227
|
-
content: 'Vector databases store data as high-dimensional vectors, enabling semantic similarity search. They are essential for AI applications like RAG systems and recommendation engines.',
|
|
1228
|
-
source: 'blog'
|
|
1229
|
-
},
|
|
1230
|
-
{
|
|
1231
|
-
id: 'doc3',
|
|
1232
|
-
title: 'HNSW Algorithm',
|
|
1233
|
-
content: 'Hierarchical Navigable Small World (HNSW) is a graph-based algorithm for approximate nearest neighbor search. It provides excellent recall with low latency.',
|
|
1234
|
-
source: 'research'
|
|
1235
|
-
}
|
|
1236
|
-
];
|
|
1237
|
-
|
|
1238
|
-
await rag.indexDocuments(documents);
|
|
1239
|
-
|
|
1240
|
-
// Step 2: Ask questions
|
|
1241
|
-
console.log('\n' + '='.repeat(60) + '\n');
|
|
1242
|
-
|
|
1243
|
-
const result = await rag.answer('What is Ruvector and what are its performance characteristics?');
|
|
1244
|
-
|
|
1245
|
-
console.log('📝 Answer:', result.answer);
|
|
1246
|
-
console.log('\n📚 Sources:', result.sources.join(', '));
|
|
1247
|
-
}
|
|
1248
|
-
|
|
1249
|
-
main().catch(console.error);
|
|
1250
|
-
```
|
|
1251
|
-
|
|
1252
|
-
**Expected Output:**
|
|
1253
|
-
```
|
|
1254
|
-
✅ RAG System initialized
|
|
1255
|
-
📚 Indexing 3 documents...
|
|
1256
|
-
✅ Indexed: Ruvector Introduction
|
|
1257
|
-
✅ Indexed: Vector Databases Explained
|
|
1258
|
-
✅ Indexed: HNSW Algorithm
|
|
1259
|
-
|
|
1260
|
-
✅ Indexed 3 documents total
|
|
1261
|
-
|
|
1262
|
-
============================================================
|
|
1263
|
-
|
|
1264
|
-
🔍 Searching for: "What is Ruvector and what are its performance characteristics?"
|
|
1265
|
-
📄 Found 2 relevant documents
|
|
1266
|
-
|
|
1267
|
-
🤖 Generating answer...
|
|
1268
|
-
|
|
1269
|
-
📝 Answer: Ruvector is a high-performance vector database built in Rust for Node.js applications. Its key performance characteristics include:
|
|
1270
|
-
- Sub-millisecond query latency
|
|
1271
|
-
- Over 52,000 inserts per second
|
|
1272
|
-
- Optimized for semantic similarity search
|
|
1273
|
-
|
|
1274
|
-
📚 Sources: Ruvector Introduction, Vector Databases Explained
|
|
1275
|
-
```
|
|
1276
|
-
|
|
1277
|
-
**Production Tips:**
|
|
1278
|
-
- ✅ Use batch embedding for better throughput (OpenAI supports up to 2048 texts)
|
|
1279
|
-
- ✅ Implement caching for frequently asked questions
|
|
1280
|
-
- ✅ Add error handling for API rate limits
|
|
1281
|
-
- ✅ Monitor token usage and costs
|
|
1282
|
-
- ✅ Regularly update your knowledge base
|
|
1283
|
-
|
|
1284
|
-
---
|
|
1285
|
-
|
|
1286
|
-
### Tutorial 2: Semantic Search Engine
|
|
1287
|
-
|
|
1288
|
-
**What you'll learn:** Build a semantic search engine that understands meaning, not just keywords.
|
|
1289
|
-
|
|
1290
|
-
**Prerequisites:**
|
|
1291
|
-
```bash
|
|
1292
|
-
npm install ruvector @xenova/transformers
|
|
1293
|
-
```
|
|
1294
|
-
|
|
1295
|
-
**Complete Implementation:**
|
|
1296
|
-
|
|
1297
|
-
```javascript
|
|
1298
|
-
const { VectorDb } = require('ruvector');
|
|
1299
|
-
const { pipeline } = require('@xenova/transformers');
|
|
1300
|
-
|
|
1301
|
-
class SemanticSearchEngine {
|
|
1302
|
-
constructor() {
|
|
1303
|
-
this.db = null;
|
|
1304
|
-
this.embedder = null;
|
|
1305
|
-
}
|
|
1306
|
-
|
|
1307
|
-
// Step 1: Initialize the embedding model
|
|
1308
|
-
async initialize() {
|
|
1309
|
-
console.log('🚀 Initializing semantic search engine...');
|
|
1310
|
-
|
|
1311
|
-
// Load sentence-transformers model (runs locally, no API needed!)
|
|
1312
|
-
console.log('📥 Loading embedding model...');
|
|
1313
|
-
this.embedder = await pipeline(
|
|
1314
|
-
'feature-extraction',
|
|
1315
|
-
'Xenova/all-MiniLM-L6-v2'
|
|
1316
|
-
);
|
|
1317
|
-
|
|
1318
|
-
// Create vector database (384 dimensions for all-MiniLM-L6-v2)
|
|
1319
|
-
this.db = new VectorDb({
|
|
1320
|
-
dimensions: 384,
|
|
1321
|
-
maxElements: 50000,
|
|
1322
|
-
storagePath: './semantic-search.db'
|
|
1323
|
-
});
|
|
1324
|
-
|
|
1325
|
-
console.log('✅ Search engine ready!\n');
|
|
1326
|
-
}
|
|
1327
|
-
|
|
1328
|
-
// Step 2: Generate embeddings
|
|
1329
|
-
async embed(text) {
|
|
1330
|
-
const output = await this.embedder(text, {
|
|
1331
|
-
pooling: 'mean',
|
|
1332
|
-
normalize: true
|
|
1333
|
-
});
|
|
1334
|
-
|
|
1335
|
-
// Convert to Float32Array
|
|
1336
|
-
return new Float32Array(output.data);
|
|
1337
|
-
}
|
|
1338
|
-
|
|
1339
|
-
// Step 3: Index documents
|
|
1340
|
-
async indexDocuments(documents) {
|
|
1341
|
-
console.log(`📚 Indexing ${documents.length} documents...`);
|
|
1342
|
-
|
|
1343
|
-
for (const doc of documents) {
|
|
1344
|
-
const vector = await this.embed(doc.content);
|
|
1345
|
-
|
|
1346
|
-
await this.db.insert({
|
|
1347
|
-
id: doc.id,
|
|
1348
|
-
vector: vector,
|
|
1349
|
-
metadata: {
|
|
1350
|
-
title: doc.title,
|
|
1351
|
-
content: doc.content,
|
|
1352
|
-
category: doc.category,
|
|
1353
|
-
url: doc.url
|
|
1354
|
-
}
|
|
1355
|
-
});
|
|
1356
|
-
|
|
1357
|
-
console.log(` ✅ ${doc.title}`);
|
|
1358
|
-
}
|
|
1359
|
-
|
|
1360
|
-
const count = await this.db.len();
|
|
1361
|
-
console.log(`\n✅ Indexed ${count} documents\n`);
|
|
1362
|
-
}
|
|
1363
|
-
|
|
1364
|
-
// Step 4: Semantic search
|
|
1365
|
-
async search(query, options = {}) {
|
|
1366
|
-
const {
|
|
1367
|
-
k = 5,
|
|
1368
|
-
category = null,
|
|
1369
|
-
threshold = 0.3
|
|
1370
|
-
} = options;
|
|
1371
|
-
|
|
1372
|
-
console.log(`🔍 Searching for: "${query}"`);
|
|
1373
|
-
|
|
1374
|
-
// Generate query embedding
|
|
1375
|
-
const queryVector = await this.embed(query);
|
|
1376
|
-
|
|
1377
|
-
// Search vector database
|
|
1378
|
-
const results = await this.db.search({
|
|
1379
|
-
vector: queryVector,
|
|
1380
|
-
k: k * 2, // Get more results for filtering
|
|
1381
|
-
threshold: threshold
|
|
1382
|
-
});
|
|
1383
|
-
|
|
1384
|
-
// Filter by category if specified
|
|
1385
|
-
let filtered = results;
|
|
1386
|
-
if (category) {
|
|
1387
|
-
filtered = results.filter(r => r.metadata.category === category);
|
|
1388
|
-
}
|
|
1389
|
-
|
|
1390
|
-
// Return top k after filtering
|
|
1391
|
-
const final = filtered.slice(0, k);
|
|
1392
|
-
|
|
1393
|
-
console.log(`📄 Found ${final.length} results\n`);
|
|
1394
|
-
|
|
1395
|
-
return final.map(r => ({
|
|
1396
|
-
id: r.id,
|
|
1397
|
-
title: r.metadata.title,
|
|
1398
|
-
content: r.metadata.content,
|
|
1399
|
-
category: r.metadata.category,
|
|
1400
|
-
score: r.score,
|
|
1401
|
-
url: r.metadata.url
|
|
1402
|
-
}));
|
|
1403
|
-
}
|
|
1404
|
-
|
|
1405
|
-
// Step 5: Find similar documents
|
|
1406
|
-
async findSimilar(documentId, k = 5) {
|
|
1407
|
-
const doc = await this.db.get(documentId);
|
|
1408
|
-
|
|
1409
|
-
if (!doc) {
|
|
1410
|
-
throw new Error(`Document ${documentId} not found`);
|
|
1411
|
-
}
|
|
1412
|
-
|
|
1413
|
-
const results = await this.db.search({
|
|
1414
|
-
vector: doc.vector,
|
|
1415
|
-
k: k + 1 // +1 because the document itself will be included
|
|
1416
|
-
});
|
|
1417
|
-
|
|
1418
|
-
// Remove the document itself from results
|
|
1419
|
-
return results
|
|
1420
|
-
.filter(r => r.id !== documentId)
|
|
1421
|
-
.slice(0, k);
|
|
1422
|
-
}
|
|
1423
|
-
}
|
|
1424
|
-
|
|
1425
|
-
// Example Usage
|
|
1426
|
-
async function main() {
|
|
1427
|
-
const engine = new SemanticSearchEngine();
|
|
1428
|
-
await engine.initialize();
|
|
1429
|
-
|
|
1430
|
-
// Sample documents (in production, load from your database)
|
|
1431
|
-
const documents = [
|
|
1432
|
-
{
|
|
1433
|
-
id: '1',
|
|
1434
|
-
title: 'Understanding Neural Networks',
|
|
1435
|
-
content: 'Neural networks are computing systems inspired by biological neural networks. They learn to perform tasks by considering examples.',
|
|
1436
|
-
category: 'AI',
|
|
1437
|
-
url: '/docs/neural-networks'
|
|
1438
|
-
},
|
|
1439
|
-
{
|
|
1440
|
-
id: '2',
|
|
1441
|
-
title: 'Introduction to Machine Learning',
|
|
1442
|
-
content: 'Machine learning is a subset of artificial intelligence that provides systems the ability to learn and improve from experience.',
|
|
1443
|
-
category: 'AI',
|
|
1444
|
-
url: '/docs/machine-learning'
|
|
1445
|
-
},
|
|
1446
|
-
{
|
|
1447
|
-
id: '3',
|
|
1448
|
-
title: 'Web Development Best Practices',
|
|
1449
|
-
content: 'Modern web development involves responsive design, performance optimization, and accessibility considerations.',
|
|
1450
|
-
category: 'Web',
|
|
1451
|
-
url: '/docs/web-dev'
|
|
1452
|
-
},
|
|
1453
|
-
{
|
|
1454
|
-
id: '4',
|
|
1455
|
-
title: 'Deep Learning Applications',
|
|
1456
|
-
content: 'Deep learning has revolutionized computer vision, natural language processing, and speech recognition.',
|
|
1457
|
-
category: 'AI',
|
|
1458
|
-
url: '/docs/deep-learning'
|
|
1459
|
-
}
|
|
1460
|
-
];
|
|
1461
|
-
|
|
1462
|
-
// Index documents
|
|
1463
|
-
await engine.indexDocuments(documents);
|
|
1464
|
-
|
|
1465
|
-
// Example 1: Basic semantic search
|
|
1466
|
-
console.log('Example 1: Basic Search\n' + '='.repeat(60));
|
|
1467
|
-
const results1 = await engine.search('AI and neural nets');
|
|
1468
|
-
results1.forEach((result, i) => {
|
|
1469
|
-
console.log(`${i + 1}. ${result.title} (Score: ${result.score.toFixed(3)})`);
|
|
1470
|
-
console.log(` ${result.content.slice(0, 80)}...`);
|
|
1471
|
-
console.log(` Category: ${result.category}\n`);
|
|
1472
|
-
});
|
|
1473
|
-
|
|
1474
|
-
// Example 2: Category-filtered search
|
|
1475
|
-
console.log('\nExample 2: Category-Filtered Search\n' + '='.repeat(60));
|
|
1476
|
-
const results2 = await engine.search('learning algorithms', {
|
|
1477
|
-
category: 'AI',
|
|
1478
|
-
k: 3
|
|
1479
|
-
});
|
|
1480
|
-
results2.forEach((result, i) => {
|
|
1481
|
-
console.log(`${i + 1}. ${result.title} (Score: ${result.score.toFixed(3)})`);
|
|
1482
|
-
});
|
|
1483
|
-
|
|
1484
|
-
// Example 3: Find similar documents
|
|
1485
|
-
console.log('\n\nExample 3: Find Similar Documents\n' + '='.repeat(60));
|
|
1486
|
-
const similar = await engine.findSimilar('1', 2);
|
|
1487
|
-
console.log('Documents similar to "Understanding Neural Networks":');
|
|
1488
|
-
similar.forEach((doc, i) => {
|
|
1489
|
-
console.log(`${i + 1}. ${doc.metadata.title} (Score: ${doc.score.toFixed(3)})`);
|
|
1490
|
-
});
|
|
1491
|
-
}
|
|
1492
|
-
|
|
1493
|
-
main().catch(console.error);
|
|
1494
|
-
```
|
|
1495
|
-
|
|
1496
|
-
**Key Features:**
|
|
1497
|
-
- ✅ Runs completely locally (no API keys needed)
|
|
1498
|
-
- ✅ Understands semantic meaning, not just keywords
|
|
1499
|
-
- ✅ Category filtering for better results
|
|
1500
|
-
- ✅ "Find similar" functionality
|
|
1501
|
-
- ✅ Fast: ~10ms query latency
|
|
1502
|
-
|
|
1503
|
-
---
|
|
1504
|
-
|
|
1505
|
-
### Tutorial 3: AI Agent Memory System
|
|
1506
|
-
|
|
1507
|
-
**What you'll learn:** Implement a memory system for AI agents that remembers past experiences and learns from them.
|
|
1508
|
-
|
|
1509
|
-
**Complete Implementation:**
|
|
1510
|
-
|
|
1511
|
-
```javascript
|
|
1512
|
-
const { VectorDb } = require('ruvector');
|
|
1513
|
-
|
|
1514
|
-
class AgentMemory {
|
|
1515
|
-
constructor(agentId) {
|
|
1516
|
-
this.agentId = agentId;
|
|
1517
|
-
|
|
1518
|
-
// Create separate databases for different memory types
|
|
1519
|
-
this.episodicMemory = new VectorDb({
|
|
1520
|
-
dimensions: 768,
|
|
1521
|
-
storagePath: `./memory/${agentId}-episodic.db`
|
|
1522
|
-
});
|
|
1523
|
-
|
|
1524
|
-
this.semanticMemory = new VectorDb({
|
|
1525
|
-
dimensions: 768,
|
|
1526
|
-
storagePath: `./memory/${agentId}-semantic.db`
|
|
1527
|
-
});
|
|
1528
|
-
|
|
1529
|
-
console.log(`🧠 Memory system initialized for agent: ${agentId}`);
|
|
1530
|
-
}
|
|
1531
|
-
|
|
1532
|
-
// Step 1: Store an experience (episodic memory)
|
|
1533
|
-
async storeExperience(experience) {
|
|
1534
|
-
const {
|
|
1535
|
-
state,
|
|
1536
|
-
action,
|
|
1537
|
-
result,
|
|
1538
|
-
reward,
|
|
1539
|
-
embedding
|
|
1540
|
-
} = experience;
|
|
1541
|
-
|
|
1542
|
-
const experienceId = `exp_${Date.now()}_${Math.random()}`;
|
|
1543
|
-
|
|
1544
|
-
await this.episodicMemory.insert({
|
|
1545
|
-
id: experienceId,
|
|
1546
|
-
vector: new Float32Array(embedding),
|
|
1547
|
-
metadata: {
|
|
1548
|
-
state: state,
|
|
1549
|
-
action: action,
|
|
1550
|
-
result: result,
|
|
1551
|
-
reward: reward,
|
|
1552
|
-
timestamp: Date.now(),
|
|
1553
|
-
type: 'episodic'
|
|
1554
|
-
}
|
|
1555
|
-
});
|
|
1556
|
-
|
|
1557
|
-
console.log(`💾 Stored experience: ${action} -> ${result} (reward: ${reward})`);
|
|
1558
|
-
return experienceId;
|
|
1559
|
-
}
|
|
1560
|
-
|
|
1561
|
-
// Step 2: Store learned knowledge (semantic memory)
|
|
1562
|
-
async storeKnowledge(knowledge) {
|
|
1563
|
-
const {
|
|
1564
|
-
concept,
|
|
1565
|
-
description,
|
|
1566
|
-
embedding,
|
|
1567
|
-
confidence = 1.0
|
|
1568
|
-
} = knowledge;
|
|
1569
|
-
|
|
1570
|
-
const knowledgeId = `know_${Date.now()}`;
|
|
1571
|
-
|
|
1572
|
-
await this.semanticMemory.insert({
|
|
1573
|
-
id: knowledgeId,
|
|
1574
|
-
vector: new Float32Array(embedding),
|
|
1575
|
-
metadata: {
|
|
1576
|
-
concept: concept,
|
|
1577
|
-
description: description,
|
|
1578
|
-
confidence: confidence,
|
|
1579
|
-
learned: Date.now(),
|
|
1580
|
-
uses: 0,
|
|
1581
|
-
type: 'semantic'
|
|
1582
|
-
}
|
|
1583
|
-
});
|
|
1584
|
-
|
|
1585
|
-
console.log(`📚 Learned: ${concept}`);
|
|
1586
|
-
return knowledgeId;
|
|
1587
|
-
}
|
|
1588
|
-
|
|
1589
|
-
// Step 3: Recall similar experiences
|
|
1590
|
-
async recallExperiences(currentState, k = 5) {
|
|
1591
|
-
console.log(`🔍 Recalling similar experiences...`);
|
|
1592
|
-
|
|
1593
|
-
const results = await this.episodicMemory.search({
|
|
1594
|
-
vector: new Float32Array(currentState.embedding),
|
|
1595
|
-
k: k,
|
|
1596
|
-
threshold: 0.6 // Only recall reasonably similar experiences
|
|
1597
|
-
});
|
|
1598
|
-
|
|
1599
|
-
// Sort by reward to prioritize successful experiences
|
|
1600
|
-
const sorted = results.sort((a, b) => b.metadata.reward - a.metadata.reward);
|
|
1601
|
-
|
|
1602
|
-
console.log(`📝 Recalled ${sorted.length} relevant experiences`);
|
|
1603
|
-
|
|
1604
|
-
return sorted.map(r => ({
|
|
1605
|
-
state: r.metadata.state,
|
|
1606
|
-
action: r.metadata.action,
|
|
1607
|
-
result: r.metadata.result,
|
|
1608
|
-
reward: r.metadata.reward,
|
|
1609
|
-
similarity: r.score
|
|
1610
|
-
}));
|
|
1611
|
-
}
|
|
1612
|
-
|
|
1613
|
-
// Step 4: Query knowledge base
|
|
1614
|
-
async queryKnowledge(query, k = 3) {
|
|
1615
|
-
const results = await this.semanticMemory.search({
|
|
1616
|
-
vector: new Float32Array(query.embedding),
|
|
1617
|
-
k: k
|
|
1618
|
-
});
|
|
1619
|
-
|
|
1620
|
-
// Update usage statistics
|
|
1621
|
-
for (const result of results) {
|
|
1622
|
-
const knowledge = await this.semanticMemory.get(result.id);
|
|
1623
|
-
if (knowledge) {
|
|
1624
|
-
knowledge.metadata.uses += 1;
|
|
1625
|
-
// In production, update the entry
|
|
1626
|
-
}
|
|
1627
|
-
}
|
|
1628
|
-
|
|
1629
|
-
return results.map(r => ({
|
|
1630
|
-
concept: r.metadata.concept,
|
|
1631
|
-
description: r.metadata.description,
|
|
1632
|
-
confidence: r.metadata.confidence,
|
|
1633
|
-
relevance: r.score
|
|
1634
|
-
}));
|
|
1635
|
-
}
|
|
1636
|
-
|
|
1637
|
-
// Step 5: Reflect and learn from experiences
|
|
1638
|
-
async reflect() {
|
|
1639
|
-
console.log('\n🤔 Reflecting on experiences...');
|
|
1640
|
-
|
|
1641
|
-
// Get all experiences
|
|
1642
|
-
const totalExperiences = await this.episodicMemory.len();
|
|
1643
|
-
console.log(`📊 Total experiences: ${totalExperiences}`);
|
|
1644
|
-
|
|
1645
|
-
// Analyze success rate
|
|
1646
|
-
// In production, you'd aggregate experiences and extract patterns
|
|
1647
|
-
console.log('💡 Analysis complete');
|
|
1648
|
-
|
|
1649
|
-
return {
|
|
1650
|
-
totalExperiences: totalExperiences,
|
|
1651
|
-
knowledgeItems: await this.semanticMemory.len()
|
|
1652
|
-
};
|
|
1653
|
-
}
|
|
1654
|
-
|
|
1655
|
-
// Step 6: Get memory statistics
|
|
1656
|
-
async getStats() {
|
|
1657
|
-
return {
|
|
1658
|
-
episodicMemorySize: await this.episodicMemory.len(),
|
|
1659
|
-
semanticMemorySize: await this.semanticMemory.len(),
|
|
1660
|
-
agentId: this.agentId
|
|
1661
|
-
};
|
|
1662
|
-
}
|
|
1663
|
-
}
|
|
1664
|
-
|
|
1665
|
-
// Example Usage: Simulated agent learning to navigate
|
|
1666
|
-
async function main() {
|
|
1667
|
-
const agent = new AgentMemory('agent-001');
|
|
1668
|
-
|
|
1669
|
-
// Simulate embedding function (in production, use a real model)
|
|
1670
|
-
function embed(text) {
|
|
1671
|
-
return Array(768).fill(0).map(() => Math.random());
|
|
1672
|
-
}
|
|
1673
|
-
|
|
1674
|
-
console.log('\n' + '='.repeat(60));
|
|
1675
|
-
console.log('PHASE 1: Learning from experiences');
|
|
1676
|
-
console.log('='.repeat(60) + '\n');
|
|
1677
|
-
|
|
1678
|
-
// Store some experiences
|
|
1679
|
-
await agent.storeExperience({
|
|
1680
|
-
state: { location: 'room1', goal: 'room3' },
|
|
1681
|
-
action: 'move_north',
|
|
1682
|
-
result: 'reached room2',
|
|
1683
|
-
reward: 0.5,
|
|
1684
|
-
embedding: embed('navigating from room1 to room2')
|
|
1685
|
-
});
|
|
1686
|
-
|
|
1687
|
-
await agent.storeExperience({
|
|
1688
|
-
state: { location: 'room2', goal: 'room3' },
|
|
1689
|
-
action: 'move_east',
|
|
1690
|
-
result: 'reached room3',
|
|
1691
|
-
reward: 1.0,
|
|
1692
|
-
embedding: embed('navigating from room2 to room3')
|
|
1693
|
-
});
|
|
1694
|
-
|
|
1695
|
-
await agent.storeExperience({
|
|
1696
|
-
state: { location: 'room1', goal: 'room3' },
|
|
1697
|
-
action: 'move_south',
|
|
1698
|
-
result: 'hit wall',
|
|
1699
|
-
reward: -0.5,
|
|
1700
|
-
embedding: embed('failed navigation attempt')
|
|
1701
|
-
});
|
|
1702
|
-
|
|
1703
|
-
// Store learned knowledge
|
|
1704
|
-
await agent.storeKnowledge({
|
|
1705
|
-
concept: 'navigation_strategy',
|
|
1706
|
-
description: 'Moving north then east is efficient for reaching room3 from room1',
|
|
1707
|
-
embedding: embed('navigation strategy knowledge'),
|
|
1708
|
-
confidence: 0.9
|
|
1709
|
-
});
|
|
1710
|
-
|
|
1711
|
-
console.log('\n' + '='.repeat(60));
|
|
1712
|
-
console.log('PHASE 2: Applying memory');
|
|
1713
|
-
console.log('='.repeat(60) + '\n');
|
|
1714
|
-
|
|
1715
|
-
// Agent encounters a similar situation
|
|
1716
|
-
const currentState = {
|
|
1717
|
-
location: 'room1',
|
|
1718
|
-
goal: 'room3',
|
|
1719
|
-
embedding: embed('navigating from room1 to room3')
|
|
1720
|
-
};
|
|
1721
|
-
|
|
1722
|
-
// Recall relevant experiences
|
|
1723
|
-
const experiences = await agent.recallExperiences(currentState, 3);
|
|
1724
|
-
|
|
1725
|
-
console.log('\n📖 Recalled experiences:');
|
|
1726
|
-
experiences.forEach((exp, i) => {
|
|
1727
|
-
console.log(`${i + 1}. Action: ${exp.action} | Result: ${exp.result} | Reward: ${exp.reward} | Similarity: ${exp.similarity.toFixed(3)}`);
|
|
1728
|
-
});
|
|
1729
|
-
|
|
1730
|
-
// Query relevant knowledge
|
|
1731
|
-
const knowledge = await agent.queryKnowledge({
|
|
1732
|
-
embedding: embed('how to navigate efficiently')
|
|
1733
|
-
}, 2);
|
|
1734
|
-
|
|
1735
|
-
console.log('\n📚 Relevant knowledge:');
|
|
1736
|
-
knowledge.forEach((k, i) => {
|
|
1737
|
-
console.log(`${i + 1}. ${k.concept}: ${k.description} (confidence: ${k.confidence})`);
|
|
1738
|
-
});
|
|
1739
|
-
|
|
1740
|
-
console.log('\n' + '='.repeat(60));
|
|
1741
|
-
console.log('PHASE 3: Reflection');
|
|
1742
|
-
console.log('='.repeat(60) + '\n');
|
|
1743
|
-
|
|
1744
|
-
// Reflect on learning
|
|
1745
|
-
const stats = await agent.reflect();
|
|
1746
|
-
const memoryStats = await agent.getStats();
|
|
1747
|
-
|
|
1748
|
-
console.log('\n📊 Memory Statistics:');
|
|
1749
|
-
console.log(` Episodic memories: ${memoryStats.episodicMemorySize}`);
|
|
1750
|
-
console.log(` Semantic knowledge: ${memoryStats.semanticMemorySize}`);
|
|
1751
|
-
console.log(` Agent ID: ${memoryStats.agentId}`);
|
|
1752
|
-
}
|
|
1753
|
-
|
|
1754
|
-
main().catch(console.error);
|
|
1755
|
-
```
|
|
1756
|
-
|
|
1757
|
-
**Expected Output:**
|
|
1758
|
-
```
|
|
1759
|
-
🧠 Memory system initialized for agent: agent-001
|
|
1760
|
-
|
|
1761
|
-
============================================================
|
|
1762
|
-
PHASE 1: Learning from experiences
|
|
1763
|
-
============================================================
|
|
1764
|
-
|
|
1765
|
-
💾 Stored experience: move_north -> reached room2 (reward: 0.5)
|
|
1766
|
-
💾 Stored experience: move_east -> reached room3 (reward: 1.0)
|
|
1767
|
-
💾 Stored experience: move_south -> hit wall (reward: -0.5)
|
|
1768
|
-
📚 Learned: navigation_strategy
|
|
1769
|
-
|
|
1770
|
-
============================================================
|
|
1771
|
-
PHASE 2: Applying memory
|
|
1772
|
-
============================================================
|
|
1773
|
-
|
|
1774
|
-
🔍 Recalling similar experiences...
|
|
1775
|
-
📝 Recalled 3 relevant experiences
|
|
1776
|
-
|
|
1777
|
-
📖 Recalled experiences:
|
|
1778
|
-
1. Action: move_east | Result: reached room3 | Reward: 1.0 | Similarity: 0.892
|
|
1779
|
-
2. Action: move_north | Result: reached room2 | Reward: 0.5 | Similarity: 0.876
|
|
1780
|
-
3. Action: move_south | Result: hit wall | Reward: -0.5 | Similarity: 0.654
|
|
1781
|
-
|
|
1782
|
-
📚 Relevant knowledge:
|
|
1783
|
-
1. navigation_strategy: Moving north then east is efficient for reaching room3 from room1 (confidence: 0.9)
|
|
1784
|
-
|
|
1785
|
-
============================================================
|
|
1786
|
-
PHASE 3: Reflection
|
|
1787
|
-
============================================================
|
|
1788
|
-
|
|
1789
|
-
🤔 Reflecting on experiences...
|
|
1790
|
-
📊 Total experiences: 3
|
|
1791
|
-
💡 Analysis complete
|
|
1792
|
-
|
|
1793
|
-
📊 Memory Statistics:
|
|
1794
|
-
Episodic memories: 3
|
|
1795
|
-
Semantic knowledge: 1
|
|
1796
|
-
Agent ID: agent-001
|
|
1797
|
-
```
|
|
1798
|
-
|
|
1799
|
-
**Use Cases:**
|
|
1800
|
-
- ✅ Reinforcement learning agents
|
|
1801
|
-
- ✅ Chatbot conversation history
|
|
1802
|
-
- ✅ Game AI that learns from gameplay
|
|
1803
|
-
- ✅ Personal assistant memory
|
|
1804
|
-
- ✅ Robotic navigation systems
|
|
1805
|
-
|
|
1806
|
-
## 🏗️ API Reference
|
|
1807
|
-
|
|
1808
|
-
### Constructor
|
|
1809
|
-
|
|
1810
|
-
```typescript
|
|
1811
|
-
new VectorDb(options: {
|
|
1812
|
-
dimensions: number; // Vector dimensionality (required)
|
|
1813
|
-
maxElements?: number; // Max vectors (default: 10000)
|
|
1814
|
-
storagePath?: string; // Persistent storage path
|
|
1815
|
-
ef_construction?: number; // HNSW construction parameter (default: 200)
|
|
1816
|
-
m?: number; // HNSW M parameter (default: 16)
|
|
1817
|
-
distanceMetric?: string; // 'cosine', 'euclidean', or 'dot' (default: 'cosine')
|
|
1818
|
-
})
|
|
1819
|
-
```
|
|
1820
|
-
|
|
1821
|
-
### Methods
|
|
1822
|
-
|
|
1823
|
-
#### insert(entry: VectorEntry): Promise<string>
|
|
1824
|
-
Insert a vector into the database.
|
|
1825
|
-
|
|
1826
|
-
```javascript
|
|
1827
|
-
const id = await db.insert({
|
|
1828
|
-
id: 'doc_1',
|
|
1829
|
-
vector: new Float32Array([0.1, 0.2, 0.3, ...]),
|
|
1830
|
-
metadata: { title: 'Document 1' }
|
|
1831
|
-
});
|
|
1832
|
-
```
|
|
1833
|
-
|
|
1834
|
-
#### search(query: SearchQuery): Promise<SearchResult[]>
|
|
1835
|
-
Search for similar vectors.
|
|
1836
|
-
|
|
1837
|
-
```javascript
|
|
1838
|
-
const results = await db.search({
|
|
1839
|
-
vector: new Float32Array([0.1, 0.2, 0.3, ...]),
|
|
1840
|
-
k: 10,
|
|
1841
|
-
threshold: 0.7
|
|
1842
|
-
});
|
|
1843
|
-
```
|
|
1844
|
-
|
|
1845
|
-
#### get(id: string): Promise<VectorEntry | null>
|
|
1846
|
-
Retrieve a vector by ID.
|
|
1847
|
-
|
|
1848
|
-
```javascript
|
|
1849
|
-
const entry = await db.get('doc_1');
|
|
1850
|
-
if (entry) {
|
|
1851
|
-
console.log(entry.vector, entry.metadata);
|
|
1852
|
-
}
|
|
1853
|
-
```
|
|
1854
|
-
|
|
1855
|
-
#### delete(id: string): Promise<boolean>
|
|
1856
|
-
Remove a vector from the database.
|
|
1857
|
-
|
|
1858
|
-
```javascript
|
|
1859
|
-
const deleted = await db.delete('doc_1');
|
|
1860
|
-
console.log(deleted ? 'Deleted' : 'Not found');
|
|
1861
|
-
```
|
|
1862
|
-
|
|
1863
|
-
#### len(): Promise<number>
|
|
1864
|
-
Get the total number of vectors.
|
|
1865
|
-
|
|
1866
|
-
```javascript
|
|
1867
|
-
const count = await db.len();
|
|
1868
|
-
console.log(`Total vectors: ${count}`);
|
|
1869
|
-
```
|
|
1870
|
-
|
|
1871
|
-
## 🎨 Advanced Configuration
|
|
1872
|
-
|
|
1873
|
-
### HNSW Parameters
|
|
1874
|
-
|
|
1875
|
-
```javascript
|
|
1876
|
-
const db = new VectorDb({
|
|
1877
|
-
dimensions: 384,
|
|
1878
|
-
maxElements: 1000000,
|
|
1879
|
-
ef_construction: 200, // Higher = better recall, slower build
|
|
1880
|
-
m: 16, // Higher = better recall, more memory
|
|
1881
|
-
storagePath: './large-db.db'
|
|
1882
|
-
});
|
|
1883
|
-
```
|
|
1884
|
-
|
|
1885
|
-
**Parameter Guidelines:**
|
|
1886
|
-
- `ef_construction`: 100-400 (higher = better recall, slower indexing)
|
|
1887
|
-
- `m`: 8-64 (higher = better recall, more memory)
|
|
1888
|
-
- Default values work well for most use cases
|
|
1889
|
-
|
|
1890
|
-
### Distance Metrics
|
|
1891
|
-
|
|
1892
|
-
```javascript
|
|
1893
|
-
// Cosine similarity (default, best for normalized vectors)
|
|
1894
|
-
const db1 = new VectorDb({
|
|
1895
|
-
dimensions: 128,
|
|
1896
|
-
distanceMetric: 'cosine'
|
|
1897
|
-
});
|
|
1898
|
-
|
|
1899
|
-
// Euclidean distance (L2, best for spatial data)
|
|
1900
|
-
const db2 = new VectorDb({
|
|
1901
|
-
dimensions: 128,
|
|
1902
|
-
distanceMetric: 'euclidean'
|
|
1903
|
-
});
|
|
1904
|
-
|
|
1905
|
-
// Dot product (best for pre-normalized vectors)
|
|
1906
|
-
const db3 = new VectorDb({
|
|
1907
|
-
dimensions: 128,
|
|
1908
|
-
distanceMetric: 'dot'
|
|
1909
|
-
});
|
|
1910
|
-
```
|
|
1911
|
-
|
|
1912
|
-
### Persistence
|
|
1913
|
-
|
|
1914
|
-
```javascript
|
|
1915
|
-
// Auto-save to disk
|
|
1916
|
-
const persistent = new VectorDb({
|
|
1917
|
-
dimensions: 128,
|
|
1918
|
-
storagePath: './persistent.db'
|
|
1919
|
-
});
|
|
1920
|
-
|
|
1921
|
-
// In-memory only (faster, but data lost on exit)
|
|
1922
|
-
const temporary = new VectorDb({
|
|
1923
|
-
dimensions: 128
|
|
1924
|
-
// No storagePath = in-memory
|
|
1925
|
-
});
|
|
1926
|
-
```
|
|
1927
|
-
|
|
1928
|
-
## 📦 Platform Support
|
|
1929
|
-
|
|
1930
|
-
Automatically installs the correct implementation for:
|
|
1931
|
-
|
|
1932
|
-
### Native (Rust) - Best Performance
|
|
1933
|
-
- **Linux**: x64, ARM64 (GNU libc)
|
|
1934
|
-
- **macOS**: x64 (Intel), ARM64 (Apple Silicon)
|
|
1935
|
-
- **Windows**: x64 (MSVC)
|
|
1936
|
-
|
|
1937
|
-
Performance: **<0.5ms latency**, **50K+ ops/sec**
|
|
1938
|
-
|
|
1939
|
-
### WASM Fallback - Universal Compatibility
|
|
1940
|
-
- Any platform where native module isn't available
|
|
1941
|
-
- Browser environments (experimental)
|
|
1942
|
-
- Alpine Linux (musl) and other non-glibc systems
|
|
1943
|
-
|
|
1944
|
-
Performance: **10-50ms latency**, **~1K ops/sec**
|
|
1945
|
-
|
|
1946
|
-
**Node.js 18+ required** for all platforms.
|
|
1947
|
-
|
|
1948
|
-
## 🔧 Building from Source
|
|
1949
|
-
|
|
1950
|
-
If you need to rebuild the native module:
|
|
1951
|
-
|
|
1952
|
-
```bash
|
|
1953
|
-
# Install Rust toolchain
|
|
1954
|
-
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
|
1955
|
-
|
|
1956
|
-
# Clone repository
|
|
1957
|
-
git clone https://github.com/ruvnet/ruvector.git
|
|
1958
|
-
cd ruvector
|
|
1959
|
-
|
|
1960
|
-
# Build native module
|
|
1961
|
-
cd npm/packages/core
|
|
1962
|
-
npm run build:napi
|
|
1963
|
-
|
|
1964
|
-
# Build wrapper package
|
|
1965
|
-
cd ../ruvector
|
|
1966
|
-
npm install
|
|
1967
|
-
npm run build
|
|
1968
|
-
|
|
1969
|
-
# Run tests
|
|
1970
|
-
npm test
|
|
1971
|
-
```
|
|
1972
|
-
|
|
1973
|
-
**Requirements:**
|
|
1974
|
-
- Rust 1.77+
|
|
1975
|
-
- Node.js 18+
|
|
1976
|
-
- Cargo
|
|
1977
|
-
|
|
1978
|
-
## 🌍 Ecosystem
|
|
1979
|
-
|
|
1980
|
-
### Related Packages
|
|
1981
|
-
|
|
1982
|
-
- **[ruvector-core](https://www.npmjs.com/package/ruvector-core)** - Core native bindings (lower-level API)
|
|
1983
|
-
- **[ruvector-wasm](https://www.npmjs.com/package/ruvector-wasm)** - WebAssembly implementation for browsers
|
|
1984
|
-
- **[ruvector-cli](https://www.npmjs.com/package/ruvector-cli)** - Standalone CLI tools
|
|
1985
|
-
- **[@ruvector/rvf](https://www.npmjs.com/package/@ruvector/rvf)** - RVF cognitive container SDK
|
|
1986
|
-
- **[@ruvector/rvf-wasm](https://www.npmjs.com/package/@ruvector/rvf-wasm)** - RVF WASM build for browsers, Deno, and edge
|
|
1987
|
-
- **[rvlite](https://www.npmjs.com/package/rvlite)** - Lightweight vector database with SQL, SPARQL, and Cypher
|
|
1988
|
-
|
|
1989
|
-
### Platform-Specific Packages (auto-installed)
|
|
1990
|
-
|
|
1991
|
-
- **[ruvector-core-linux-x64-gnu](https://www.npmjs.com/package/ruvector-core-linux-x64-gnu)**
|
|
1992
|
-
- **[ruvector-core-linux-arm64-gnu](https://www.npmjs.com/package/ruvector-core-linux-arm64-gnu)**
|
|
1993
|
-
- **[ruvector-core-darwin-x64](https://www.npmjs.com/package/ruvector-core-darwin-x64)**
|
|
1994
|
-
- **[ruvector-core-darwin-arm64](https://www.npmjs.com/package/ruvector-core-darwin-arm64)**
|
|
1995
|
-
- **[ruvector-core-win32-x64-msvc](https://www.npmjs.com/package/ruvector-core-win32-x64-msvc)**
|
|
1996
|
-
|
|
1997
|
-
---
|
|
1998
|
-
|
|
1999
|
-
## RVF Cognitive Containers
|
|
2000
|
-
|
|
2001
|
-
Ruvector integrates with [RVF (RuVector Format)](https://github.com/ruvnet/ruvector/tree/main/crates/rvf) — a universal binary substrate that stores vectors, models, graphs, compute kernels, and attestation in a single `.rvf` file.
|
|
2002
|
-
|
|
2003
|
-
### Enable RVF Backend
|
|
2004
|
-
|
|
2005
|
-
```bash
|
|
2006
|
-
# Install the optional RVF package
|
|
2007
|
-
npm install @ruvector/rvf
|
|
2008
|
-
|
|
2009
|
-
# Set backend via environment variable
|
|
2010
|
-
export RUVECTOR_BACKEND=rvf
|
|
2011
|
-
|
|
2012
|
-
# Or detect automatically (native -> rvf -> wasm fallback)
|
|
2013
|
-
npx ruvector info
|
|
2014
|
-
```
|
|
2015
|
-
|
|
2016
|
-
```typescript
|
|
2017
|
-
import { getImplementationType, isRvf } from 'ruvector';
|
|
2018
|
-
|
|
2019
|
-
console.log(getImplementationType()); // 'native' | 'rvf' | 'wasm'
|
|
2020
|
-
console.log(isRvf()); // true if RVF backend is active
|
|
2021
|
-
```
|
|
2022
|
-
|
|
2023
|
-
### RVF CLI Commands
|
|
2024
|
-
|
|
2025
|
-
8 RVF-specific subcommands are available through the ruvector CLI:
|
|
2026
|
-
|
|
2027
|
-
```bash
|
|
2028
|
-
# Create an RVF store
|
|
2029
|
-
npx ruvector rvf create mydb.rvf -d 384 --metric cosine
|
|
2030
|
-
|
|
2031
|
-
# Ingest vectors from JSON
|
|
2032
|
-
npx ruvector rvf ingest mydb.rvf --input vectors.json --format json
|
|
2033
|
-
|
|
2034
|
-
# Query nearest neighbors
|
|
2035
|
-
npx ruvector rvf query mydb.rvf --vector "[0.1,0.2,...]" --k 10
|
|
2036
|
-
|
|
2037
|
-
# File status and segment listing
|
|
2038
|
-
npx ruvector rvf status mydb.rvf
|
|
2039
|
-
npx ruvector rvf segments mydb.rvf
|
|
2040
|
-
|
|
2041
|
-
# COW branching — derive a child file
|
|
2042
|
-
npx ruvector rvf derive mydb.rvf --output child.rvf
|
|
2043
|
-
|
|
2044
|
-
# Compact and reclaim space
|
|
2045
|
-
npx ruvector rvf compact mydb.rvf
|
|
2046
|
-
|
|
2047
|
-
# Export to JSON
|
|
2048
|
-
npx ruvector rvf export mydb.rvf --output dump.json
|
|
2049
|
-
```
|
|
2050
|
-
|
|
2051
|
-
### RVF Platform Support
|
|
2052
|
-
|
|
2053
|
-
| Platform | Runtime | Backend |
|
|
2054
|
-
|----------|---------|---------|
|
|
2055
|
-
| Linux x86_64 / aarch64 | Node.js 18+ | Native (N-API) |
|
|
2056
|
-
| macOS x86_64 / arm64 | Node.js 18+ | Native (N-API) |
|
|
2057
|
-
| Windows x86_64 | Node.js 18+ | Native (N-API) |
|
|
2058
|
-
| Any | Deno | WASM (`@ruvector/rvf-wasm`) |
|
|
2059
|
-
| Any | Browser | WASM (`@ruvector/rvf-wasm`) |
|
|
2060
|
-
| Any | Cloudflare Workers | WASM (`@ruvector/rvf-wasm`) |
|
|
2061
|
-
|
|
2062
|
-
### Download Example .rvf Files
|
|
2063
|
-
|
|
2064
|
-
45 pre-built example files are available (~11 MB total):
|
|
2065
|
-
|
|
2066
|
-
```bash
|
|
2067
|
-
# Download a specific example
|
|
2068
|
-
curl -LO https://raw.githubusercontent.com/ruvnet/ruvector/main/examples/rvf/output/basic_store.rvf
|
|
2069
|
-
|
|
2070
|
-
# Popular examples:
|
|
2071
|
-
# basic_store.rvf (152 KB) — 1,000 vectors, dim 128
|
|
2072
|
-
# semantic_search.rvf (755 KB) — Semantic search with HNSW
|
|
2073
|
-
# rag_pipeline.rvf (303 KB) — RAG pipeline embeddings
|
|
2074
|
-
# agent_memory.rvf (32 KB) — AI agent memory store
|
|
2075
|
-
# self_booting.rvf (31 KB) — Self-booting with kernel
|
|
2076
|
-
# progressive_index.rvf (2.5 MB) — Large-scale HNSW index
|
|
2077
|
-
|
|
2078
|
-
# Generate all examples locally
|
|
2079
|
-
cd crates/rvf && cargo run --example generate_all
|
|
2080
|
-
```
|
|
2081
|
-
|
|
2082
|
-
Full catalog: [examples/rvf/output/](https://github.com/ruvnet/ruvector/tree/main/examples/rvf/output)
|
|
2083
|
-
|
|
2084
|
-
### Working Examples: Cognitive Containers
|
|
2085
|
-
|
|
2086
|
-
#### Self-Booting Microservice
|
|
2087
|
-
|
|
2088
|
-
A single `.rvf` file that contains vectors AND a bootable Linux kernel:
|
|
2089
|
-
|
|
2090
|
-
```bash
|
|
2091
|
-
# Build and run the self-booting example
|
|
2092
|
-
cd crates/rvf && cargo run --example self_booting
|
|
2093
|
-
# Output:
|
|
2094
|
-
# Ingested 50 vectors (128 dims)
|
|
2095
|
-
# Pre-kernel query: top-5 results OK (nearest ID=25)
|
|
2096
|
-
# Kernel: 4,640 bytes embedded (x86_64, Hermit)
|
|
2097
|
-
# Witness chain: 5 entries, all verified
|
|
2098
|
-
# File: bootable.rvf (31 KB) — data + runtime in one file
|
|
2099
|
-
```
|
|
2100
|
-
|
|
2101
|
-
```rust
|
|
2102
|
-
// The pattern: vectors + kernel + witness in one file
|
|
2103
|
-
let mut store = RvfStore::create("bootable.rvf", options)?;
|
|
2104
|
-
store.ingest_batch(&vectors, &ids, None)?;
|
|
2105
|
-
store.embed_kernel(KernelArch::X86_64 as u8, KernelType::Hermit as u8,
|
|
2106
|
-
0x0018, &kernel_image, 8080, Some("console=ttyS0 quiet"))?;
|
|
2107
|
-
// Result: drop on a VM and it boots as a query service
|
|
2108
|
-
```
|
|
2109
|
-
|
|
2110
|
-
#### Linux Microkernel Distribution
|
|
2111
|
-
|
|
2112
|
-
20-package Linux distro with SSH keys and kernel in a single file:
|
|
2113
|
-
|
|
2114
|
-
```bash
|
|
2115
|
-
cd crates/rvf && cargo run --example linux_microkernel
|
|
2116
|
-
# Output:
|
|
2117
|
-
# Installed 20 packages as vector embeddings
|
|
2118
|
-
# Kernel embedded: Linux x86_64 (4,640 bytes)
|
|
2119
|
-
# SSH keys: Ed25519, signed and verified
|
|
2120
|
-
# Witness chain: 22 entries (1 per package + kernel + SSH)
|
|
2121
|
-
# File: microkernel.rvf (14 KB) — immutable bootable system
|
|
2122
|
-
```
|
|
2123
|
-
|
|
2124
|
-
Features: package search by embedding similarity, Ed25519 signed SSH keys, witness-audited installs, COW-derived child images for atomic updates.
|
|
2125
|
-
|
|
2126
|
-
#### Claude Code AI Appliance
|
|
2127
|
-
|
|
2128
|
-
A sealed, bootable AI development environment:
|
|
2129
|
-
|
|
2130
|
-
```bash
|
|
2131
|
-
cd crates/rvf && cargo run --example claude_code_appliance
|
|
2132
|
-
# Output:
|
|
2133
|
-
# 20 dev packages (rust, node, python, docker, ...)
|
|
2134
|
-
# Kernel: Linux x86_64 with SSH on port 2222
|
|
2135
|
-
# eBPF: XDP distance program for fast-path lookups
|
|
2136
|
-
# Witness chain: 6 entries, all verified
|
|
2137
|
-
# Crypto: Ed25519 signature
|
|
2138
|
-
# File: claude_code_appliance.rvf (17 KB)
|
|
2139
|
-
```
|
|
2140
|
-
|
|
2141
|
-
#### CLI Full Lifecycle
|
|
2142
|
-
|
|
2143
|
-
```bash
|
|
2144
|
-
# Create → Ingest → Query → Derive → Inspect
|
|
2145
|
-
rvf create vectors.rvf --dimension 384
|
|
2146
|
-
rvf ingest vectors.rvf --input data.json --format json
|
|
2147
|
-
rvf query vectors.rvf --vector "0.1,0.2,..." --k 10
|
|
2148
|
-
rvf derive vectors.rvf child.rvf --type filter
|
|
2149
|
-
rvf inspect vectors.rvf
|
|
2150
|
-
|
|
2151
|
-
# Embed kernel and launch as microVM
|
|
2152
|
-
rvf embed-kernel vectors.rvf --image bzImage
|
|
2153
|
-
rvf launch vectors.rvf --port 8080
|
|
2154
|
-
|
|
2155
|
-
# Verify tamper-evident witness chain
|
|
2156
|
-
rvf verify-witness vectors.rvf
|
|
2157
|
-
rvf verify-attestation vectors.rvf
|
|
2158
|
-
```
|
|
2159
|
-
|
|
2160
|
-
#### Integration Tests (46 passing)
|
|
2161
|
-
|
|
2162
|
-
```bash
|
|
2163
|
-
cd crates/rvf
|
|
2164
|
-
cargo test --workspace
|
|
2165
|
-
# attestation .............. 6 passed
|
|
2166
|
-
# crypto ................... 10 passed
|
|
2167
|
-
# computational_container .. 8 passed
|
|
2168
|
-
# cow_branching ............ 8 passed
|
|
2169
|
-
# cross_platform ........... 6 passed
|
|
2170
|
-
# lineage .................. 4 passed
|
|
2171
|
-
# smoke .................... 4 passed
|
|
2172
|
-
# Total: 46/46 passed
|
|
2173
|
-
```
|
|
2174
|
-
|
|
2175
|
-
## 🐛 Troubleshooting
|
|
2176
|
-
|
|
2177
|
-
### Native Module Not Loading
|
|
2178
|
-
|
|
2179
|
-
If you see "Cannot find module 'ruvector-core-*'":
|
|
2180
|
-
|
|
2181
|
-
```bash
|
|
2182
|
-
# Reinstall with optional dependencies
|
|
2183
|
-
npm install --include=optional ruvector
|
|
2184
|
-
|
|
2185
|
-
# Verify platform
|
|
2186
|
-
npx ruvector info
|
|
2187
|
-
|
|
2188
|
-
# Check Node.js version (18+ required)
|
|
2189
|
-
node --version
|
|
2190
|
-
```
|
|
2191
|
-
|
|
2192
|
-
### WASM Fallback Performance
|
|
2193
|
-
|
|
2194
|
-
If you're using WASM fallback and need better performance:
|
|
2195
|
-
|
|
2196
|
-
1. **Install native toolchain** for your platform
|
|
2197
|
-
2. **Rebuild native module**: `npm rebuild ruvector`
|
|
2198
|
-
3. **Verify native**: `npx ruvector info` should show "native (Rust)"
|
|
2199
|
-
|
|
2200
|
-
### Platform Compatibility
|
|
2201
|
-
|
|
2202
|
-
- **Alpine Linux**: Uses WASM fallback (musl not supported)
|
|
2203
|
-
- **Windows ARM**: Not yet supported, uses WASM fallback
|
|
2204
|
-
- **Node.js < 18**: Not supported, upgrade to Node.js 18+
|
|
2205
|
-
|
|
2206
|
-
## 📚 Documentation
|
|
2207
|
-
|
|
2208
|
-
- 🏠 [Homepage](https://ruv.io)
|
|
2209
|
-
- 📦 [GitHub Repository](https://github.com/ruvnet/ruvector)
|
|
2210
|
-
- 📚 [Full Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)
|
|
2211
|
-
- 🚀 [Getting Started Guide](https://github.com/ruvnet/ruvector/blob/main/docs/guide/GETTING_STARTED.md)
|
|
2212
|
-
- 📖 [API Reference](https://github.com/ruvnet/ruvector/blob/main/docs/api/NODEJS_API.md)
|
|
2213
|
-
- 🎯 [Performance Tuning](https://github.com/ruvnet/ruvector/blob/main/docs/optimization/PERFORMANCE_TUNING_GUIDE.md)
|
|
2214
|
-
- 🐛 [Issue Tracker](https://github.com/ruvnet/ruvector/issues)
|
|
2215
|
-
- 💬 [Discussions](https://github.com/ruvnet/ruvector/discussions)
|
|
2216
|
-
|
|
2217
|
-
## 🤝 Contributing
|
|
2218
|
-
|
|
2219
|
-
We welcome contributions! See [CONTRIBUTING.md](https://github.com/ruvnet/ruvector/blob/main/docs/development/CONTRIBUTING.md) for guidelines.
|
|
2220
|
-
|
|
2221
|
-
### Quick Start
|
|
2222
|
-
|
|
2223
|
-
1. Fork the repository
|
|
2224
|
-
2. Create a feature branch: `git checkout -b feature/amazing-feature`
|
|
2225
|
-
3. Commit changes: `git commit -m 'Add amazing feature'`
|
|
2226
|
-
4. Push to branch: `git push origin feature/amazing-feature`
|
|
2227
|
-
5. Open a Pull Request
|
|
2228
|
-
|
|
2229
|
-
## 🌐 Community & Support
|
|
2230
|
-
|
|
2231
|
-
- **GitHub**: [github.com/ruvnet/ruvector](https://github.com/ruvnet/ruvector) - ⭐ Star and follow
|
|
2232
|
-
- **Discord**: [Join our community](https://discord.gg/ruvnet) - Chat with developers
|
|
2233
|
-
- **Twitter**: [@ruvnet](https://twitter.com/ruvnet) - Follow for updates
|
|
2234
|
-
- **Issues**: [Report bugs](https://github.com/ruvnet/ruvector/issues)
|
|
2235
|
-
|
|
2236
|
-
### Enterprise Support
|
|
2237
|
-
|
|
2238
|
-
Need custom development or consulting?
|
|
2239
|
-
|
|
2240
|
-
📧 [enterprise@ruv.io](mailto:enterprise@ruv.io)
|
|
2241
|
-
|
|
2242
|
-
## 📜 License
|
|
2243
|
-
|
|
2244
|
-
**MIT License** - see [LICENSE](https://github.com/ruvnet/ruvector/blob/main/LICENSE) for details.
|
|
2245
|
-
|
|
2246
|
-
Free for commercial and personal use.
|
|
2247
|
-
|
|
2248
|
-
## 🙏 Acknowledgments
|
|
2249
|
-
|
|
2250
|
-
Built with battle-tested technologies:
|
|
2251
|
-
|
|
2252
|
-
- **HNSW**: Hierarchical Navigable Small World graphs
|
|
2253
|
-
- **SIMD**: Hardware-accelerated vector operations via simsimd
|
|
2254
|
-
- **Rust**: Memory-safe, zero-cost abstractions
|
|
2255
|
-
- **NAPI-RS**: High-performance Node.js bindings
|
|
2256
|
-
- **WebAssembly**: Universal browser compatibility
|
|
2257
|
-
|
|
2258
|
-
---
|
|
2259
|
-
|
|
2260
|
-
<div align="center">
|
|
2261
|
-
|
|
2262
|
-
**Built with ❤️ by [rUv](https://ruv.io)**
|
|
2263
|
-
|
|
2264
|
-
[](https://www.npmjs.com/package/ruvector)
|
|
2265
|
-
[](https://github.com/ruvnet/ruvector)
|
|
2266
|
-
[](https://twitter.com/ruvnet)
|
|
2267
|
-
|
|
2268
|
-
**[Get Started](https://github.com/ruvnet/ruvector/blob/main/docs/guide/GETTING_STARTED.md)** • **[Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)** • **[API Reference](https://github.com/ruvnet/ruvector/blob/main/docs/api/NODEJS_API.md)** • **[Contributing](https://github.com/ruvnet/ruvector/blob/main/docs/development/CONTRIBUTING.md)**
|
|
2269
|
-
|
|
2270
|
-
</div>
|
|
1
|
+
# ruvector
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/ruvector)
|
|
4
|
+
[](https://opensource.org/licenses/MIT)
|
|
5
|
+
[](https://nodejs.org)
|
|
6
|
+
[](https://www.npmjs.com/package/ruvector)
|
|
7
|
+
[](https://github.com/ruvnet/ruvector)
|
|
8
|
+
[](https://github.com/ruvnet/ruvector)
|
|
9
|
+
[](https://github.com/ruvnet/ruvector)
|
|
10
|
+
|
|
11
|
+
**The fastest vector database for Node.js—built in Rust, runs everywhere**
|
|
12
|
+
|
|
13
|
+
Ruvector is a self-learning vector database with **enterprise-grade semantic search**, hybrid retrieval (sparse + dense), Graph RAG, FlashAttention-3, and DiskANN — all in a single npm package. Unlike cloud-only solutions or Python-first databases, Ruvector is designed for JavaScript/TypeScript developers who need **blazing-fast vector search** without external services.
|
|
14
|
+
|
|
15
|
+
> 🚀 **Sub-millisecond queries** • 🎯 **52,000+ inserts/sec** • 💾 **~50 bytes per vector** • 🌍 **Runs anywhere** • 🧠 **859 tests passing**
|
|
16
|
+
|
|
17
|
+
Built by [rUv](https://ruv.io) with production-grade Rust performance and intelligent platform detection—**automatically uses native bindings when available, falls back to WebAssembly when needed**.
|
|
18
|
+
|
|
19
|
+
🌐 **[Visit ruv.io](https://ruv.io)** | 📦 **[GitHub](https://github.com/ruvnet/ruvector)** | 📚 **[Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)**
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## 🧠 Claude Code Intelligence v2.0
|
|
24
|
+
|
|
25
|
+
**Self-learning intelligence for Claude Code** — RuVector provides optimized hooks with ONNX embeddings, AST analysis, and coverage-aware routing.
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
# One-command setup with pretrain and agent generation
|
|
29
|
+
npx ruvector hooks init --pretrain --build-agents quality
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
### Core Features
|
|
33
|
+
- 🎯 **Smart Agent Routing** — Q-learning optimized suggestions with 80%+ accuracy
|
|
34
|
+
- 📚 **9-Phase Pretrain** — AST, diff, coverage, neural, and graph analysis
|
|
35
|
+
- 🤖 **Agent Builder** — Generates optimized `.claude/agents/` configs
|
|
36
|
+
- 🔗 **Co-edit Patterns** — Learns file relationships from git history
|
|
37
|
+
- 💾 **Vector Memory** — HNSW-indexed semantic recall (150x faster)
|
|
38
|
+
|
|
39
|
+
### New in v2.1 — SOTA Vector Search
|
|
40
|
+
- **FlashAttention-3** — IO-aware tiled attention, O(N) memory instead of O(N^2)
|
|
41
|
+
- **Graph RAG** — Knowledge graph + community detection for multi-hop queries (30-60% improvement)
|
|
42
|
+
- **Hybrid Search** — Sparse + dense vectors with RRF fusion (20-49% better retrieval)
|
|
43
|
+
- **DiskANN / Vamana** — SSD-friendly ANN graph with PQ compression for large-scale search
|
|
44
|
+
- **ColBERT Multi-Vector** — Per-token late interaction retrieval (MaxSim)
|
|
45
|
+
- **Matryoshka Embeddings** — Adaptive-dimension search with funnel/cascade modes
|
|
46
|
+
- **MLA** — Multi-Head Latent Attention with ~93% KV-cache compression (DeepSeek-V2/V3)
|
|
47
|
+
- **Mamba SSM** — Selective State Space Models for linear-time sequence processing
|
|
48
|
+
- **TurboQuant** — 2-4 bit KV-cache quantization, 6-8x memory reduction
|
|
49
|
+
- **OPQ** — Optimized Product Quantization with learned rotation (10-30% error reduction)
|
|
50
|
+
- **GraphMAE** — Graph Masked Autoencoder for self-supervised node learning
|
|
51
|
+
|
|
52
|
+
### New in v2.0
|
|
53
|
+
- **ONNX WASM Embeddings** — all-MiniLM-L6-v2 (384d) runs locally, no API needed
|
|
54
|
+
- **AST Analysis** — Symbol extraction, complexity metrics, import graphs
|
|
55
|
+
- **Diff Embeddings** — Semantic change classification with risk scoring
|
|
56
|
+
- **Coverage Routing** — Test coverage-aware agent selection
|
|
57
|
+
- **Graph Algorithms** — MinCut boundaries, Louvain communities, Spectral clustering
|
|
58
|
+
- 🛡️ **Security Scanning** — Parallel vulnerability pattern detection
|
|
59
|
+
- 🎯 **RAG Context** — Semantic retrieval with HNSW indexing
|
|
60
|
+
|
|
61
|
+
### Performance
|
|
62
|
+
| Backend | Read Time | Speedup |
|
|
63
|
+
|---------|-----------|---------|
|
|
64
|
+
| ONNX inference | ~400ms | baseline |
|
|
65
|
+
| HNSW search | ~0.045ms | 8,800x |
|
|
66
|
+
| Memory cache | ~0.01ms | **40,000x** |
|
|
67
|
+
|
|
68
|
+
📖 **[Full Hooks Documentation →](https://github.com/ruvnet/ruvector/blob/main/npm/packages/ruvector/HOOKS.md)**
|
|
69
|
+
|
|
70
|
+
### MCP Server Integration
|
|
71
|
+
|
|
72
|
+
RuVector includes an MCP server for Claude Code with 103 tools:
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
# Add to Claude Code
|
|
76
|
+
claude mcp add ruvector -- npx ruvector mcp start
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
**Available MCP Tools:**
|
|
80
|
+
- `hooks_route`, `hooks_route_enhanced` — Agent routing with signals
|
|
81
|
+
- `hooks_ast_analyze`, `hooks_ast_complexity` — Code structure analysis
|
|
82
|
+
- `hooks_diff_analyze`, `hooks_diff_classify` — Change classification
|
|
83
|
+
- `hooks_coverage_route`, `hooks_coverage_suggest` — Test-aware routing
|
|
84
|
+
- `hooks_graph_mincut`, `hooks_graph_cluster` — Code boundaries
|
|
85
|
+
- `hooks_security_scan` — Vulnerability detection
|
|
86
|
+
- `hooks_rag_context` — Semantic context retrieval
|
|
87
|
+
- `hooks_attention_info`, `hooks_gnn_info` — Neural capabilities
|
|
88
|
+
- `brain_search`, `brain_share`, `brain_status` — Shared brain knowledge
|
|
89
|
+
- `brain_agi_status`, `brain_sona_stats`, `brain_temporal`, `brain_explore` — AGI diagnostics
|
|
90
|
+
- `brain_midstream`, `brain_flags` — Midstream platform + feature flags
|
|
91
|
+
- `midstream_status`, `midstream_attractor`, `midstream_scheduler` — Streaming analysis
|
|
92
|
+
- `midstream_benchmark`, `midstream_search`, `midstream_health` — Latency benchmarks + health
|
|
93
|
+
|
|
94
|
+
### Brain AGI Commands
|
|
95
|
+
|
|
96
|
+
Access all 8 AGI subsystems deployed at π.ruv.io:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
npx ruvector brain agi status # Combined AGI + midstream diagnostics
|
|
100
|
+
npx ruvector brain agi sona # SONA patterns, trajectories, ticks
|
|
101
|
+
npx ruvector brain agi temporal # Knowledge evolution velocity
|
|
102
|
+
npx ruvector brain agi explore # Meta-learning curiosity & regret
|
|
103
|
+
npx ruvector brain agi midstream # Scheduler, attractor, solver, strange-loop
|
|
104
|
+
npx ruvector brain agi flags # Feature flag state
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### Midstream Commands
|
|
108
|
+
|
|
109
|
+
Real-time streaming analysis platform:
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
npx ruvector midstream status # Platform overview
|
|
113
|
+
npx ruvector midstream attractor # Lyapunov attractor analysis
|
|
114
|
+
npx ruvector midstream scheduler # Nanosecond scheduler metrics
|
|
115
|
+
npx ruvector midstream benchmark # Latency benchmark (p50/p90/p99)
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## 🌟 Why Ruvector?
|
|
121
|
+
|
|
122
|
+
### The Problem with Existing Vector Databases
|
|
123
|
+
|
|
124
|
+
Most vector databases force you to choose between three painful trade-offs:
|
|
125
|
+
|
|
126
|
+
1. **Cloud-Only Services** (Pinecone, Weaviate Cloud) - Expensive, vendor lock-in, latency issues, API rate limits
|
|
127
|
+
2. **Python-First Solutions** (ChromaDB, Faiss) - Poor Node.js support, require separate Python processes
|
|
128
|
+
3. **Self-Hosted Complexity** (Milvus, Qdrant) - Heavy infrastructure, Docker orchestration, operational overhead
|
|
129
|
+
|
|
130
|
+
**Ruvector eliminates these trade-offs.**
|
|
131
|
+
|
|
132
|
+
### The Ruvector Advantage
|
|
133
|
+
|
|
134
|
+
Ruvector is purpose-built for **modern JavaScript/TypeScript applications** that need vector search:
|
|
135
|
+
|
|
136
|
+
🎯 **Native Node.js Integration**
|
|
137
|
+
- Drop-in npm package—no Docker, no Python, no external services
|
|
138
|
+
- Full TypeScript support with complete type definitions
|
|
139
|
+
- Automatic platform detection with native Rust bindings
|
|
140
|
+
- Seamless WebAssembly fallback for universal compatibility
|
|
141
|
+
|
|
142
|
+
⚡ **Production-Grade Performance**
|
|
143
|
+
- **52,000+ inserts/second** with native Rust (10x faster than Python alternatives)
|
|
144
|
+
- **<0.5ms query latency** with HNSW indexing and SIMD optimizations
|
|
145
|
+
- **~50 bytes per vector** with advanced memory optimization
|
|
146
|
+
- Scales from edge devices to millions of vectors
|
|
147
|
+
|
|
148
|
+
🧠 **Built for AI Applications**
|
|
149
|
+
- Optimized for LLM embeddings (OpenAI, Cohere, Hugging Face)
|
|
150
|
+
- Perfect for RAG (Retrieval-Augmented Generation) systems
|
|
151
|
+
- Agent memory and semantic caching
|
|
152
|
+
- Real-time recommendation engines
|
|
153
|
+
|
|
154
|
+
🌍 **Universal Deployment**
|
|
155
|
+
- **Linux, macOS, Windows** with native performance
|
|
156
|
+
- **Browser support** via WebAssembly (experimental)
|
|
157
|
+
- **Edge computing** and serverless environments
|
|
158
|
+
- **Alpine Linux** and non-glibc systems supported
|
|
159
|
+
|
|
160
|
+
💰 **Zero Operational Costs**
|
|
161
|
+
- No cloud API fees or usage limits
|
|
162
|
+
- No infrastructure to manage
|
|
163
|
+
- No separate database servers
|
|
164
|
+
- Open source MIT license
|
|
165
|
+
|
|
166
|
+
### Key Advantages
|
|
167
|
+
|
|
168
|
+
- ⚡ **Blazing Fast**: <0.5ms p50 latency with native Rust, 10-50ms with WASM fallback
|
|
169
|
+
- 🎯 **Automatic Platform Detection**: Uses native when available, falls back to WASM seamlessly
|
|
170
|
+
- 🧠 **AI-Native**: Built specifically for embeddings, RAG, semantic search, and agent memory
|
|
171
|
+
- 🔧 **CLI Tools Included**: Full command-line interface for database management
|
|
172
|
+
- 🌍 **Universal Deployment**: Works on all platforms—Linux, macOS, Windows, even browsers
|
|
173
|
+
- 💾 **Memory Efficient**: ~50 bytes per vector with advanced quantization
|
|
174
|
+
- 🚀 **Production Ready**: Battle-tested algorithms with comprehensive benchmarks
|
|
175
|
+
- 🔓 **Open Source**: MIT licensed, community-driven
|
|
176
|
+
|
|
177
|
+
## 🚀 Quick Start Tutorial
|
|
178
|
+
|
|
179
|
+
### Step 1: Installation
|
|
180
|
+
|
|
181
|
+
Install Ruvector with a single npm command:
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
npm install ruvector
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
**What happens during installation:**
|
|
188
|
+
- npm automatically detects your platform (Linux, macOS, Windows)
|
|
189
|
+
- Downloads the correct native binary for maximum performance
|
|
190
|
+
- Falls back to WebAssembly if native binaries aren't available
|
|
191
|
+
- No additional setup, Docker, or external services required
|
|
192
|
+
|
|
193
|
+
**Windows Installation (without build tools):**
|
|
194
|
+
```bash
|
|
195
|
+
# Skip native compilation, use WASM fallback
|
|
196
|
+
npm install ruvector --ignore-scripts
|
|
197
|
+
|
|
198
|
+
# The ONNX WASM runtime (7.4MB) works without build tools
|
|
199
|
+
# Memory cache provides 40,000x speedup over inference
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
**Verify installation:**
|
|
203
|
+
```bash
|
|
204
|
+
npx ruvector info
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
You should see your platform and implementation type (native Rust or WASM fallback).
|
|
208
|
+
|
|
209
|
+
### Step 2: Your First Vector Database
|
|
210
|
+
|
|
211
|
+
Let's create a simple vector database and perform basic operations. This example demonstrates the complete CRUD (Create, Read, Update, Delete) workflow:
|
|
212
|
+
|
|
213
|
+
```javascript
|
|
214
|
+
const { VectorDb } = require('ruvector');
|
|
215
|
+
|
|
216
|
+
async function tutorial() {
|
|
217
|
+
// Step 2.1: Create a new vector database
|
|
218
|
+
// The 'dimensions' parameter must match your embedding model
|
|
219
|
+
// Common sizes: 128, 384 (sentence-transformers), 768 (BERT), 1536 (OpenAI)
|
|
220
|
+
const db = new VectorDb({
|
|
221
|
+
dimensions: 128, // Vector size - MUST match your embeddings
|
|
222
|
+
maxElements: 10000, // Maximum vectors (can grow automatically)
|
|
223
|
+
storagePath: './my-vectors.db' // Persist to disk (omit for in-memory)
|
|
224
|
+
});
|
|
225
|
+
|
|
226
|
+
console.log('✅ Database created successfully');
|
|
227
|
+
|
|
228
|
+
// Step 2.2: Insert vectors
|
|
229
|
+
// In real applications, these would come from an embedding model
|
|
230
|
+
const documents = [
|
|
231
|
+
{ id: 'doc1', text: 'Artificial intelligence and machine learning' },
|
|
232
|
+
{ id: 'doc2', text: 'Deep learning neural networks' },
|
|
233
|
+
{ id: 'doc3', text: 'Natural language processing' },
|
|
234
|
+
];
|
|
235
|
+
|
|
236
|
+
for (const doc of documents) {
|
|
237
|
+
// Generate random vector for demonstration
|
|
238
|
+
// In production: use OpenAI, Cohere, or sentence-transformers
|
|
239
|
+
const vector = new Float32Array(128).map(() => Math.random());
|
|
240
|
+
|
|
241
|
+
await db.insert({
|
|
242
|
+
id: doc.id,
|
|
243
|
+
vector: vector,
|
|
244
|
+
metadata: {
|
|
245
|
+
text: doc.text,
|
|
246
|
+
timestamp: Date.now(),
|
|
247
|
+
category: 'AI'
|
|
248
|
+
}
|
|
249
|
+
});
|
|
250
|
+
|
|
251
|
+
console.log(`✅ Inserted: ${doc.id}`);
|
|
252
|
+
}
|
|
253
|
+
|
|
254
|
+
// Step 2.3: Search for similar vectors
|
|
255
|
+
// Create a query vector (in production, this would be from your search query)
|
|
256
|
+
const queryVector = new Float32Array(128).map(() => Math.random());
|
|
257
|
+
|
|
258
|
+
const results = await db.search({
|
|
259
|
+
vector: queryVector,
|
|
260
|
+
k: 5, // Return top 5 most similar vectors
|
|
261
|
+
threshold: 0.7 // Only return results with similarity > 0.7
|
|
262
|
+
});
|
|
263
|
+
|
|
264
|
+
console.log('\n🔍 Search Results:');
|
|
265
|
+
results.forEach((result, index) => {
|
|
266
|
+
console.log(`${index + 1}. ${result.id} - Score: ${result.score.toFixed(3)}`);
|
|
267
|
+
console.log(` Text: ${result.metadata.text}`);
|
|
268
|
+
});
|
|
269
|
+
|
|
270
|
+
// Step 2.4: Retrieve a specific vector
|
|
271
|
+
const retrieved = await db.get('doc1');
|
|
272
|
+
if (retrieved) {
|
|
273
|
+
console.log('\n📄 Retrieved document:', retrieved.metadata.text);
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
// Step 2.5: Get database statistics
|
|
277
|
+
const count = await db.len();
|
|
278
|
+
console.log(`\n📊 Total vectors in database: ${count}`);
|
|
279
|
+
|
|
280
|
+
// Step 2.6: Delete a vector
|
|
281
|
+
const deleted = await db.delete('doc1');
|
|
282
|
+
console.log(`\n🗑️ Deleted doc1: ${deleted ? 'Success' : 'Not found'}`);
|
|
283
|
+
|
|
284
|
+
// Final count
|
|
285
|
+
const finalCount = await db.len();
|
|
286
|
+
console.log(`📊 Final count: ${finalCount}`);
|
|
287
|
+
}
|
|
288
|
+
|
|
289
|
+
// Run the tutorial
|
|
290
|
+
tutorial().catch(console.error);
|
|
291
|
+
```
|
|
292
|
+
|
|
293
|
+
**Expected Output:**
|
|
294
|
+
```
|
|
295
|
+
✅ Database created successfully
|
|
296
|
+
✅ Inserted: doc1
|
|
297
|
+
✅ Inserted: doc2
|
|
298
|
+
✅ Inserted: doc3
|
|
299
|
+
|
|
300
|
+
🔍 Search Results:
|
|
301
|
+
1. doc2 - Score: 0.892
|
|
302
|
+
Text: Deep learning neural networks
|
|
303
|
+
2. doc1 - Score: 0.856
|
|
304
|
+
Text: Artificial intelligence and machine learning
|
|
305
|
+
3. doc3 - Score: 0.801
|
|
306
|
+
Text: Natural language processing
|
|
307
|
+
|
|
308
|
+
📄 Retrieved document: Artificial intelligence and machine learning
|
|
309
|
+
|
|
310
|
+
📊 Total vectors in database: 3
|
|
311
|
+
|
|
312
|
+
🗑️ Deleted doc1: Success
|
|
313
|
+
📊 Final count: 2
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
### Step 3: TypeScript Tutorial
|
|
317
|
+
|
|
318
|
+
Ruvector provides full TypeScript support with complete type safety. Here's how to use it:
|
|
319
|
+
|
|
320
|
+
```typescript
|
|
321
|
+
import { VectorDb, VectorEntry, SearchQuery, SearchResult } from 'ruvector';
|
|
322
|
+
|
|
323
|
+
// Step 3.1: Define your custom metadata type
|
|
324
|
+
interface DocumentMetadata {
|
|
325
|
+
title: string;
|
|
326
|
+
content: string;
|
|
327
|
+
author: string;
|
|
328
|
+
date: Date;
|
|
329
|
+
tags: string[];
|
|
330
|
+
}
|
|
331
|
+
|
|
332
|
+
async function typescriptTutorial() {
|
|
333
|
+
// Step 3.2: Create typed database
|
|
334
|
+
const db = new VectorDb({
|
|
335
|
+
dimensions: 384, // sentence-transformers/all-MiniLM-L6-v2
|
|
336
|
+
maxElements: 10000,
|
|
337
|
+
storagePath: './typed-vectors.db'
|
|
338
|
+
});
|
|
339
|
+
|
|
340
|
+
// Step 3.3: Type-safe vector entry
|
|
341
|
+
const entry: VectorEntry<DocumentMetadata> = {
|
|
342
|
+
id: 'article-001',
|
|
343
|
+
vector: new Float32Array(384), // Your embedding here
|
|
344
|
+
metadata: {
|
|
345
|
+
title: 'Introduction to Vector Databases',
|
|
346
|
+
content: 'Vector databases enable semantic search...',
|
|
347
|
+
author: 'Jane Doe',
|
|
348
|
+
date: new Date('2024-01-15'),
|
|
349
|
+
tags: ['database', 'AI', 'search']
|
|
350
|
+
}
|
|
351
|
+
};
|
|
352
|
+
|
|
353
|
+
// Step 3.4: Insert with type checking
|
|
354
|
+
await db.insert(entry);
|
|
355
|
+
console.log('✅ Inserted typed document');
|
|
356
|
+
|
|
357
|
+
// Step 3.5: Type-safe search
|
|
358
|
+
const query: SearchQuery = {
|
|
359
|
+
vector: new Float32Array(384),
|
|
360
|
+
k: 10,
|
|
361
|
+
threshold: 0.8
|
|
362
|
+
};
|
|
363
|
+
|
|
364
|
+
// Step 3.6: Fully typed results
|
|
365
|
+
const results: SearchResult<DocumentMetadata>[] = await db.search(query);
|
|
366
|
+
|
|
367
|
+
// TypeScript knows the exact shape of metadata
|
|
368
|
+
results.forEach(result => {
|
|
369
|
+
console.log(`Title: ${result.metadata.title}`);
|
|
370
|
+
console.log(`Author: ${result.metadata.author}`);
|
|
371
|
+
console.log(`Tags: ${result.metadata.tags.join(', ')}`);
|
|
372
|
+
console.log(`Similarity: ${result.score.toFixed(3)}\n`);
|
|
373
|
+
});
|
|
374
|
+
|
|
375
|
+
// Step 3.7: Type-safe retrieval
|
|
376
|
+
const doc = await db.get('article-001');
|
|
377
|
+
if (doc) {
|
|
378
|
+
// TypeScript autocomplete works perfectly here
|
|
379
|
+
const publishYear = doc.metadata.date.getFullYear();
|
|
380
|
+
console.log(`Published in ${publishYear}`);
|
|
381
|
+
}
|
|
382
|
+
}
|
|
383
|
+
|
|
384
|
+
typescriptTutorial().catch(console.error);
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
**TypeScript Benefits:**
|
|
388
|
+
- ✅ Full autocomplete for all methods and properties
|
|
389
|
+
- ✅ Compile-time type checking prevents errors
|
|
390
|
+
- ✅ IDE IntelliSense shows documentation
|
|
391
|
+
- ✅ Custom metadata types for your use case
|
|
392
|
+
- ✅ No `any` types - fully typed throughout
|
|
393
|
+
|
|
394
|
+
## 🎯 Platform Detection
|
|
395
|
+
|
|
396
|
+
Ruvector automatically detects the best implementation for your platform:
|
|
397
|
+
|
|
398
|
+
```javascript
|
|
399
|
+
const { getImplementationType, isNative, isWasm } = require('ruvector');
|
|
400
|
+
|
|
401
|
+
console.log(getImplementationType()); // 'native' or 'wasm'
|
|
402
|
+
console.log(isNative()); // true if using native Rust
|
|
403
|
+
console.log(isWasm()); // true if using WebAssembly fallback
|
|
404
|
+
|
|
405
|
+
// Performance varies by implementation:
|
|
406
|
+
// Native (Rust): <0.5ms latency, 50K+ ops/sec
|
|
407
|
+
// WASM fallback: 10-50ms latency, ~1K ops/sec
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
## 🔧 CLI Tools
|
|
411
|
+
|
|
412
|
+
Ruvector includes a full command-line interface for database management:
|
|
413
|
+
|
|
414
|
+
### Create Database
|
|
415
|
+
|
|
416
|
+
```bash
|
|
417
|
+
# Create a new vector database
|
|
418
|
+
npx ruvector create mydb.vec --dimensions 384 --metric cosine
|
|
419
|
+
|
|
420
|
+
# Options:
|
|
421
|
+
# --dimensions, -d Vector dimensionality (required)
|
|
422
|
+
# --metric, -m Distance metric (cosine, euclidean, dot)
|
|
423
|
+
# --max-elements Maximum number of vectors (default: 10000)
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
### Insert Vectors
|
|
427
|
+
|
|
428
|
+
```bash
|
|
429
|
+
# Insert vectors from JSON file
|
|
430
|
+
npx ruvector insert mydb.vec vectors.json
|
|
431
|
+
|
|
432
|
+
# JSON format:
|
|
433
|
+
# [
|
|
434
|
+
# { "id": "doc1", "vector": [0.1, 0.2, ...], "metadata": {...} },
|
|
435
|
+
# { "id": "doc2", "vector": [0.3, 0.4, ...], "metadata": {...} }
|
|
436
|
+
# ]
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
### Search Vectors
|
|
440
|
+
|
|
441
|
+
```bash
|
|
442
|
+
# Search for similar vectors
|
|
443
|
+
npx ruvector search mydb.vec --vector "[0.1,0.2,0.3,...]" --top-k 10
|
|
444
|
+
|
|
445
|
+
# Options:
|
|
446
|
+
# --vector, -v Query vector (JSON array)
|
|
447
|
+
# --top-k, -k Number of results (default: 10)
|
|
448
|
+
# --threshold Minimum similarity score
|
|
449
|
+
```
|
|
450
|
+
|
|
451
|
+
### Database Statistics
|
|
452
|
+
|
|
453
|
+
```bash
|
|
454
|
+
# Show database statistics
|
|
455
|
+
npx ruvector stats mydb.vec
|
|
456
|
+
|
|
457
|
+
# Output:
|
|
458
|
+
# Total vectors: 10,000
|
|
459
|
+
# Dimensions: 384
|
|
460
|
+
# Metric: cosine
|
|
461
|
+
# Memory usage: ~500 KB
|
|
462
|
+
# Index type: HNSW
|
|
463
|
+
```
|
|
464
|
+
|
|
465
|
+
### Benchmarking
|
|
466
|
+
|
|
467
|
+
```bash
|
|
468
|
+
# Run performance benchmark
|
|
469
|
+
npx ruvector benchmark --num-vectors 10000 --num-queries 1000
|
|
470
|
+
|
|
471
|
+
# Options:
|
|
472
|
+
# --num-vectors Number of vectors to insert
|
|
473
|
+
# --num-queries Number of search queries
|
|
474
|
+
# --dimensions Vector dimensionality (default: 128)
|
|
475
|
+
```
|
|
476
|
+
|
|
477
|
+
### System Information
|
|
478
|
+
|
|
479
|
+
```bash
|
|
480
|
+
# Show platform and implementation info
|
|
481
|
+
npx ruvector info
|
|
482
|
+
|
|
483
|
+
# Output:
|
|
484
|
+
# Platform: linux-x64-gnu
|
|
485
|
+
# Implementation: native (Rust)
|
|
486
|
+
# GNN Module: Available
|
|
487
|
+
# Node.js: v18.17.0
|
|
488
|
+
# Performance: <0.5ms p50 latency
|
|
489
|
+
```
|
|
490
|
+
|
|
491
|
+
### Install Optional Packages
|
|
492
|
+
|
|
493
|
+
Ruvector supports optional packages that extend functionality. Use the `install` command to add them:
|
|
494
|
+
|
|
495
|
+
```bash
|
|
496
|
+
# List available packages
|
|
497
|
+
npx ruvector install
|
|
498
|
+
|
|
499
|
+
# Output:
|
|
500
|
+
# Available Ruvector Packages:
|
|
501
|
+
#
|
|
502
|
+
# gnn not installed
|
|
503
|
+
# Graph Neural Network layers, tensor compression, differentiable search
|
|
504
|
+
# npm: @ruvector/gnn
|
|
505
|
+
#
|
|
506
|
+
# core ✓ installed
|
|
507
|
+
# Core vector database with native Rust bindings
|
|
508
|
+
# npm: @ruvector/core
|
|
509
|
+
|
|
510
|
+
# Install specific package
|
|
511
|
+
npx ruvector install gnn
|
|
512
|
+
|
|
513
|
+
# Install all optional packages
|
|
514
|
+
npx ruvector install --all
|
|
515
|
+
|
|
516
|
+
# Interactive selection
|
|
517
|
+
npx ruvector install -i
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
The install command auto-detects your package manager (npm, yarn, pnpm, bun).
|
|
521
|
+
|
|
522
|
+
### GNN Commands
|
|
523
|
+
|
|
524
|
+
Ruvector includes Graph Neural Network (GNN) capabilities for advanced tensor compression and differentiable search.
|
|
525
|
+
|
|
526
|
+
#### GNN Info
|
|
527
|
+
|
|
528
|
+
```bash
|
|
529
|
+
# Show GNN module information
|
|
530
|
+
npx ruvector gnn info
|
|
531
|
+
|
|
532
|
+
# Output:
|
|
533
|
+
# GNN Module Information
|
|
534
|
+
# Status: Available
|
|
535
|
+
# Platform: linux
|
|
536
|
+
# Architecture: x64
|
|
537
|
+
#
|
|
538
|
+
# Available Features:
|
|
539
|
+
# • RuvectorLayer - GNN layer with multi-head attention
|
|
540
|
+
# • TensorCompress - Adaptive tensor compression (5 levels)
|
|
541
|
+
# • differentiableSearch - Soft attention-based search
|
|
542
|
+
# • hierarchicalForward - Multi-layer GNN processing
|
|
543
|
+
```
|
|
544
|
+
|
|
545
|
+
#### GNN Layer
|
|
546
|
+
|
|
547
|
+
```bash
|
|
548
|
+
# Create and test a GNN layer
|
|
549
|
+
npx ruvector gnn layer -i 128 -h 256 --test
|
|
550
|
+
|
|
551
|
+
# Options:
|
|
552
|
+
# -i, --input-dim Input dimension (required)
|
|
553
|
+
# -h, --hidden-dim Hidden dimension (required)
|
|
554
|
+
# -a, --heads Number of attention heads (default: 4)
|
|
555
|
+
# -d, --dropout Dropout rate (default: 0.1)
|
|
556
|
+
# --test Run a test forward pass
|
|
557
|
+
# -o, --output Save layer config to JSON file
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
#### GNN Compress
|
|
561
|
+
|
|
562
|
+
```bash
|
|
563
|
+
# Compress embeddings using adaptive tensor compression
|
|
564
|
+
npx ruvector gnn compress -f embeddings.json -l pq8 -o compressed.json
|
|
565
|
+
|
|
566
|
+
# Options:
|
|
567
|
+
# -f, --file Input JSON file with embeddings (required)
|
|
568
|
+
# -l, --level Compression level: none|half|pq8|pq4|binary (default: auto)
|
|
569
|
+
# -a, --access-freq Access frequency for auto compression (default: 0.5)
|
|
570
|
+
# -o, --output Output file for compressed data
|
|
571
|
+
|
|
572
|
+
# Compression levels:
|
|
573
|
+
# none (freq > 0.8) - Full precision, hot data
|
|
574
|
+
# half (freq > 0.4) - ~50% savings, warm data
|
|
575
|
+
# pq8 (freq > 0.1) - ~8x compression, cool data
|
|
576
|
+
# pq4 (freq > 0.01) - ~16x compression, cold data
|
|
577
|
+
# binary (freq <= 0.01) - ~32x compression, archive
|
|
578
|
+
```
|
|
579
|
+
|
|
580
|
+
#### GNN Search
|
|
581
|
+
|
|
582
|
+
```bash
|
|
583
|
+
# Differentiable search with soft attention
|
|
584
|
+
npx ruvector gnn search -q "[1.0,0.0,0.0]" -c candidates.json -k 5
|
|
585
|
+
|
|
586
|
+
# Options:
|
|
587
|
+
# -q, --query Query vector as JSON array (required)
|
|
588
|
+
# -c, --candidates Candidates file - JSON array of vectors (required)
|
|
589
|
+
# -k, --top-k Number of results (default: 5)
|
|
590
|
+
# -t, --temperature Softmax temperature (default: 1.0)
|
|
591
|
+
```
|
|
592
|
+
|
|
593
|
+
### Attention Commands
|
|
594
|
+
|
|
595
|
+
Ruvector includes high-performance attention mechanisms for transformer-based operations, hyperbolic embeddings, and graph attention.
|
|
596
|
+
|
|
597
|
+
```bash
|
|
598
|
+
# Install the attention module (optional)
|
|
599
|
+
npm install @ruvector/attention
|
|
600
|
+
```
|
|
601
|
+
|
|
602
|
+
#### Attention Mechanisms Reference
|
|
603
|
+
|
|
604
|
+
| Mechanism | Type | Complexity | When to Use |
|
|
605
|
+
|-----------|------|------------|-------------|
|
|
606
|
+
| **DotProductAttention** | Core | O(n²) | Standard scaled dot-product attention for transformers |
|
|
607
|
+
| **MultiHeadAttention** | Core | O(n²) | Parallel attention heads for capturing different relationships |
|
|
608
|
+
| **FlashAttention** | Core | O(n²) IO-optimized | Memory-efficient attention for long sequences |
|
|
609
|
+
| **HyperbolicAttention** | Core | O(n²) | Hierarchical data, tree-like structures, taxonomies |
|
|
610
|
+
| **LinearAttention** | Core | O(n) | Very long sequences where O(n²) is prohibitive |
|
|
611
|
+
| **MoEAttention** | Core | O(n*k) | Mixture of Experts routing, specialized attention |
|
|
612
|
+
| **GraphRoPeAttention** | Graph | O(n²) | Graph data with rotary position embeddings |
|
|
613
|
+
| **EdgeFeaturedAttention** | Graph | O(n²) | Graphs with rich edge features/attributes |
|
|
614
|
+
| **DualSpaceAttention** | Graph | O(n²) | Combined Euclidean + hyperbolic representation |
|
|
615
|
+
| **LocalGlobalAttention** | Graph | O(n*k) | Large graphs with local + global context |
|
|
616
|
+
|
|
617
|
+
#### Attention Info
|
|
618
|
+
|
|
619
|
+
```bash
|
|
620
|
+
# Show attention module information
|
|
621
|
+
npx ruvector attention info
|
|
622
|
+
|
|
623
|
+
# Output:
|
|
624
|
+
# Attention Module Information
|
|
625
|
+
# Status: Available
|
|
626
|
+
# Version: 0.1.0
|
|
627
|
+
# Platform: linux
|
|
628
|
+
# Architecture: x64
|
|
629
|
+
#
|
|
630
|
+
# Core Attention Mechanisms:
|
|
631
|
+
# • DotProductAttention - Scaled dot-product attention
|
|
632
|
+
# • MultiHeadAttention - Multi-head self-attention
|
|
633
|
+
# • FlashAttention - Memory-efficient IO-aware attention
|
|
634
|
+
# • HyperbolicAttention - Poincaré ball attention
|
|
635
|
+
# • LinearAttention - O(n) linear complexity attention
|
|
636
|
+
# • MoEAttention - Mixture of Experts attention
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
#### Attention List
|
|
640
|
+
|
|
641
|
+
```bash
|
|
642
|
+
# List all available attention mechanisms
|
|
643
|
+
npx ruvector attention list
|
|
644
|
+
|
|
645
|
+
# With verbose details
|
|
646
|
+
npx ruvector attention list -v
|
|
647
|
+
```
|
|
648
|
+
|
|
649
|
+
#### Attention Benchmark
|
|
650
|
+
|
|
651
|
+
```bash
|
|
652
|
+
# Benchmark attention mechanisms
|
|
653
|
+
npx ruvector attention benchmark -d 256 -n 100 -i 100
|
|
654
|
+
|
|
655
|
+
# Options:
|
|
656
|
+
# -d, --dimension Vector dimension (default: 256)
|
|
657
|
+
# -n, --num-vectors Number of vectors (default: 100)
|
|
658
|
+
# -i, --iterations Benchmark iterations (default: 100)
|
|
659
|
+
# -t, --types Attention types to benchmark (default: dot,flash,linear)
|
|
660
|
+
|
|
661
|
+
# Example output:
|
|
662
|
+
# Dimension: 256
|
|
663
|
+
# Vectors: 100
|
|
664
|
+
# Iterations: 100
|
|
665
|
+
#
|
|
666
|
+
# dot: 0.012ms/op (84,386 ops/sec)
|
|
667
|
+
# flash: 0.012ms/op (82,844 ops/sec)
|
|
668
|
+
# linear: 0.066ms/op (15,259 ops/sec)
|
|
669
|
+
```
|
|
670
|
+
|
|
671
|
+
#### Hyperbolic Operations
|
|
672
|
+
|
|
673
|
+
```bash
|
|
674
|
+
# Calculate Poincaré distance between two points
|
|
675
|
+
npx ruvector attention hyperbolic -a distance -v "[0.1,0.2,0.3]" -b "[0.4,0.5,0.6]"
|
|
676
|
+
|
|
677
|
+
# Project vector to Poincaré ball
|
|
678
|
+
npx ruvector attention hyperbolic -a project -v "[1.5,2.0,0.8]"
|
|
679
|
+
|
|
680
|
+
# Möbius addition in hyperbolic space
|
|
681
|
+
npx ruvector attention hyperbolic -a mobius-add -v "[0.1,0.2]" -b "[0.3,0.4]"
|
|
682
|
+
|
|
683
|
+
# Exponential map (tangent space → Poincaré ball)
|
|
684
|
+
npx ruvector attention hyperbolic -a exp-map -v "[0.1,0.2,0.3]"
|
|
685
|
+
|
|
686
|
+
# Options:
|
|
687
|
+
# -a, --action Action: distance|project|mobius-add|exp-map|log-map
|
|
688
|
+
# -v, --vector Input vector as JSON array (required)
|
|
689
|
+
# -b, --vector-b Second vector for binary operations
|
|
690
|
+
# -c, --curvature Poincaré ball curvature (default: 1.0)
|
|
691
|
+
```
|
|
692
|
+
|
|
693
|
+
#### When to Use Each Attention Type
|
|
694
|
+
|
|
695
|
+
| Use Case | Recommended Attention | Reason |
|
|
696
|
+
|----------|----------------------|--------|
|
|
697
|
+
| **Standard NLP/Transformers** | MultiHeadAttention | Industry standard, well-tested |
|
|
698
|
+
| **Long Documents (>4K tokens)** | FlashAttention or LinearAttention | Memory efficient |
|
|
699
|
+
| **Hierarchical Classification** | HyperbolicAttention | Captures tree-like structures |
|
|
700
|
+
| **Knowledge Graphs** | GraphRoPeAttention | Position-aware graph attention |
|
|
701
|
+
| **Multi-Relational Graphs** | EdgeFeaturedAttention | Leverages edge attributes |
|
|
702
|
+
| **Taxonomy/Ontology Search** | DualSpaceAttention | Best of both Euclidean + hyperbolic |
|
|
703
|
+
| **Large-Scale Graphs** | LocalGlobalAttention | Efficient local + global context |
|
|
704
|
+
| **Model Routing/MoE** | MoEAttention | Expert selection and routing |
|
|
705
|
+
|
|
706
|
+
### ⚡ ONNX WASM Embeddings (v2.0)
|
|
707
|
+
|
|
708
|
+
RuVector includes a pure JavaScript ONNX runtime for local embeddings - no Python, no API calls, no build tools required.
|
|
709
|
+
|
|
710
|
+
```bash
|
|
711
|
+
# Embeddings work out of the box
|
|
712
|
+
npx ruvector hooks remember "important context" -t project
|
|
713
|
+
npx ruvector hooks recall "context query"
|
|
714
|
+
npx ruvector hooks rag-context "how does auth work"
|
|
715
|
+
```
|
|
716
|
+
|
|
717
|
+
**Model**: all-MiniLM-L6-v2 (384 dimensions, 23MB)
|
|
718
|
+
- Downloads automatically on first use
|
|
719
|
+
- Cached in `.ruvector/models/`
|
|
720
|
+
- SIMD-accelerated when available
|
|
721
|
+
|
|
722
|
+
**Performance:**
|
|
723
|
+
| Operation | Time | Notes |
|
|
724
|
+
|-----------|------|-------|
|
|
725
|
+
| Model load | ~2s | First use only |
|
|
726
|
+
| Embedding | ~50ms | Per text chunk |
|
|
727
|
+
| HNSW search | 0.045ms | 150x faster than brute force |
|
|
728
|
+
| Cache hit | 0.01ms | 40,000x faster than inference |
|
|
729
|
+
|
|
730
|
+
**Fallback Chain:**
|
|
731
|
+
1. Native SQLite → best persistence
|
|
732
|
+
2. WASM SQLite → cross-platform
|
|
733
|
+
3. Memory Cache → fastest (no persistence)
|
|
734
|
+
|
|
735
|
+
### 🧠 Self-Learning Hooks v2.0
|
|
736
|
+
|
|
737
|
+
Ruvector includes **self-learning intelligence hooks** for Claude Code integration with ONNX embeddings, AST analysis, and coverage-aware routing.
|
|
738
|
+
|
|
739
|
+
#### Initialize Hooks
|
|
740
|
+
|
|
741
|
+
```bash
|
|
742
|
+
# Initialize hooks in your project
|
|
743
|
+
npx ruvector hooks init
|
|
744
|
+
|
|
745
|
+
# Options:
|
|
746
|
+
# --force Overwrite existing configuration
|
|
747
|
+
# --minimal Minimal configuration (no optional hooks)
|
|
748
|
+
# --pretrain Initialize + pretrain from git history
|
|
749
|
+
# --build-agents quality Generate optimized agent configs
|
|
750
|
+
```
|
|
751
|
+
|
|
752
|
+
This creates `.claude/settings.json` with pre-configured hooks and `CLAUDE.md` with comprehensive documentation.
|
|
753
|
+
|
|
754
|
+
#### Session Management
|
|
755
|
+
|
|
756
|
+
```bash
|
|
757
|
+
# Start a session (load intelligence data)
|
|
758
|
+
npx ruvector hooks session-start
|
|
759
|
+
|
|
760
|
+
# End a session (save learned patterns)
|
|
761
|
+
npx ruvector hooks session-end
|
|
762
|
+
```
|
|
763
|
+
|
|
764
|
+
#### Pre/Post Edit Hooks
|
|
765
|
+
|
|
766
|
+
```bash
|
|
767
|
+
# Before editing a file - get agent recommendations
|
|
768
|
+
npx ruvector hooks pre-edit src/index.ts
|
|
769
|
+
# Output: 🤖 Recommended: typescript-developer (85% confidence)
|
|
770
|
+
|
|
771
|
+
# After editing - record success/failure for learning
|
|
772
|
+
npx ruvector hooks post-edit src/index.ts --success
|
|
773
|
+
npx ruvector hooks post-edit src/index.ts --error "Type error on line 42"
|
|
774
|
+
```
|
|
775
|
+
|
|
776
|
+
#### Pre/Post Command Hooks
|
|
777
|
+
|
|
778
|
+
```bash
|
|
779
|
+
# Before running a command - risk analysis
|
|
780
|
+
npx ruvector hooks pre-command "npm test"
|
|
781
|
+
# Output: ✅ Risk: LOW, Category: test
|
|
782
|
+
|
|
783
|
+
# After running - record outcome
|
|
784
|
+
npx ruvector hooks post-command "npm test" --success
|
|
785
|
+
npx ruvector hooks post-command "npm test" --error "3 tests failed"
|
|
786
|
+
```
|
|
787
|
+
|
|
788
|
+
#### Agent Routing
|
|
789
|
+
|
|
790
|
+
```bash
|
|
791
|
+
# Get agent recommendation for a task
|
|
792
|
+
npx ruvector hooks route "fix the authentication bug in login.ts"
|
|
793
|
+
# Output: 🤖 Recommended: security-specialist (92% confidence)
|
|
794
|
+
|
|
795
|
+
npx ruvector hooks route "add unit tests for the API"
|
|
796
|
+
# Output: 🤖 Recommended: tester (88% confidence)
|
|
797
|
+
```
|
|
798
|
+
|
|
799
|
+
#### Memory Operations
|
|
800
|
+
|
|
801
|
+
```bash
|
|
802
|
+
# Store context in vector memory
|
|
803
|
+
npx ruvector hooks remember "API uses JWT tokens with 1h expiry" --type decision
|
|
804
|
+
npx ruvector hooks remember "Database schema in docs/schema.md" --type reference
|
|
805
|
+
|
|
806
|
+
# Semantic search memory
|
|
807
|
+
npx ruvector hooks recall "authentication mechanism"
|
|
808
|
+
# Returns relevant stored memories
|
|
809
|
+
```
|
|
810
|
+
|
|
811
|
+
#### Context Suggestions
|
|
812
|
+
|
|
813
|
+
```bash
|
|
814
|
+
# Get relevant context for current task
|
|
815
|
+
npx ruvector hooks suggest-context
|
|
816
|
+
# Output: Based on recent files, suggests relevant context
|
|
817
|
+
```
|
|
818
|
+
|
|
819
|
+
#### Intelligence Statistics
|
|
820
|
+
|
|
821
|
+
```bash
|
|
822
|
+
# Show learned patterns and statistics
|
|
823
|
+
npx ruvector hooks stats
|
|
824
|
+
|
|
825
|
+
# Output:
|
|
826
|
+
# Patterns: 156 learned
|
|
827
|
+
# Success rate: 87%
|
|
828
|
+
# Top agents: rust-developer, tester, reviewer
|
|
829
|
+
# Memory entries: 42
|
|
830
|
+
```
|
|
831
|
+
|
|
832
|
+
#### Swarm Recommendations
|
|
833
|
+
|
|
834
|
+
```bash
|
|
835
|
+
# Get agent recommendation for task type
|
|
836
|
+
npx ruvector hooks swarm-recommend "code-review"
|
|
837
|
+
# Output: Recommended agents for code review task
|
|
838
|
+
```
|
|
839
|
+
|
|
840
|
+
#### AST Analysis (v2.0)
|
|
841
|
+
|
|
842
|
+
```bash
|
|
843
|
+
# Analyze file structure, symbols, imports, complexity
|
|
844
|
+
npx ruvector hooks ast-analyze src/index.ts --json
|
|
845
|
+
|
|
846
|
+
# Get complexity metrics for multiple files
|
|
847
|
+
npx ruvector hooks ast-complexity src/*.ts --threshold 15
|
|
848
|
+
# Flags files exceeding cyclomatic complexity threshold
|
|
849
|
+
```
|
|
850
|
+
|
|
851
|
+
#### Diff & Risk Analysis (v2.0)
|
|
852
|
+
|
|
853
|
+
```bash
|
|
854
|
+
# Analyze commit with semantic embeddings and risk scoring
|
|
855
|
+
npx ruvector hooks diff-analyze HEAD
|
|
856
|
+
# Output: risk score, category, affected files
|
|
857
|
+
|
|
858
|
+
# Classify change type (feature, bugfix, refactor, docs, test)
|
|
859
|
+
npx ruvector hooks diff-classify
|
|
860
|
+
|
|
861
|
+
# Find similar past commits via embeddings
|
|
862
|
+
npx ruvector hooks diff-similar -k 5
|
|
863
|
+
|
|
864
|
+
# Git churn analysis (hot spots)
|
|
865
|
+
npx ruvector hooks git-churn --days 30
|
|
866
|
+
```
|
|
867
|
+
|
|
868
|
+
#### Coverage-Aware Routing (v2.0)
|
|
869
|
+
|
|
870
|
+
```bash
|
|
871
|
+
# Get coverage-aware routing for a file
|
|
872
|
+
npx ruvector hooks coverage-route src/api.ts
|
|
873
|
+
# Output: agent weights based on test coverage
|
|
874
|
+
|
|
875
|
+
# Suggest tests for files based on coverage gaps
|
|
876
|
+
npx ruvector hooks coverage-suggest src/*.ts
|
|
877
|
+
```
|
|
878
|
+
|
|
879
|
+
#### Graph Analysis (v2.0)
|
|
880
|
+
|
|
881
|
+
```bash
|
|
882
|
+
# Find optimal code boundaries (MinCut algorithm)
|
|
883
|
+
npx ruvector hooks graph-mincut src/*.ts
|
|
884
|
+
|
|
885
|
+
# Detect code communities (Louvain/Spectral clustering)
|
|
886
|
+
npx ruvector hooks graph-cluster src/*.ts --method louvain
|
|
887
|
+
```
|
|
888
|
+
|
|
889
|
+
#### Security & RAG (v2.0)
|
|
890
|
+
|
|
891
|
+
```bash
|
|
892
|
+
# Parallel security vulnerability scan
|
|
893
|
+
npx ruvector hooks security-scan src/*.ts
|
|
894
|
+
|
|
895
|
+
# RAG-enhanced context retrieval
|
|
896
|
+
npx ruvector hooks rag-context "how does auth work"
|
|
897
|
+
|
|
898
|
+
# Enhanced routing with all signals
|
|
899
|
+
npx ruvector hooks route-enhanced "fix bug" --file src/api.ts
|
|
900
|
+
```
|
|
901
|
+
|
|
902
|
+
#### Hooks Configuration
|
|
903
|
+
|
|
904
|
+
The hooks integrate with Claude Code via `.claude/settings.json`:
|
|
905
|
+
|
|
906
|
+
```json
|
|
907
|
+
{
|
|
908
|
+
"env": {
|
|
909
|
+
"RUVECTOR_INTELLIGENCE_ENABLED": "true",
|
|
910
|
+
"RUVECTOR_LEARNING_RATE": "0.1",
|
|
911
|
+
"RUVECTOR_AST_ENABLED": "true",
|
|
912
|
+
"RUVECTOR_DIFF_EMBEDDINGS": "true",
|
|
913
|
+
"RUVECTOR_COVERAGE_ROUTING": "true",
|
|
914
|
+
"RUVECTOR_GRAPH_ALGORITHMS": "true",
|
|
915
|
+
"RUVECTOR_SECURITY_SCAN": "true"
|
|
916
|
+
},
|
|
917
|
+
"hooks": {
|
|
918
|
+
"PreToolUse": [
|
|
919
|
+
{
|
|
920
|
+
"matcher": "Edit|Write|MultiEdit",
|
|
921
|
+
"hooks": [{ "type": "command", "command": "npx ruvector hooks pre-edit \"$TOOL_INPUT_file_path\"" }]
|
|
922
|
+
},
|
|
923
|
+
{
|
|
924
|
+
"matcher": "Bash",
|
|
925
|
+
"hooks": [{ "type": "command", "command": "npx ruvector hooks pre-command \"$TOOL_INPUT_command\"" }]
|
|
926
|
+
}
|
|
927
|
+
],
|
|
928
|
+
"PostToolUse": [
|
|
929
|
+
{
|
|
930
|
+
"matcher": "Edit|Write|MultiEdit",
|
|
931
|
+
"hooks": [{ "type": "command", "command": "npx ruvector hooks post-edit \"$TOOL_INPUT_file_path\"" }]
|
|
932
|
+
}
|
|
933
|
+
],
|
|
934
|
+
"SessionStart": [{ "hooks": [{ "type": "command", "command": "npx ruvector hooks session-start" }] }],
|
|
935
|
+
"Stop": [{ "hooks": [{ "type": "command", "command": "npx ruvector hooks session-end" }] }]
|
|
936
|
+
}
|
|
937
|
+
}
|
|
938
|
+
```
|
|
939
|
+
|
|
940
|
+
#### How Self-Learning Works
|
|
941
|
+
|
|
942
|
+
1. **Pattern Recording**: Every edit and command is recorded with context
|
|
943
|
+
2. **Q-Learning**: Success/failure updates agent routing weights
|
|
944
|
+
3. **AST Analysis**: Code complexity informs agent selection
|
|
945
|
+
4. **Diff Embeddings**: Change patterns improve risk assessment
|
|
946
|
+
5. **Coverage Routing**: Test coverage guides testing priorities
|
|
947
|
+
6. **Vector Memory**: Decisions and references stored for semantic recall (HNSW indexed)
|
|
948
|
+
7. **Continuous Improvement**: The more you use it, the smarter it gets
|
|
949
|
+
|
|
950
|
+
## 📊 Performance Benchmarks
|
|
951
|
+
|
|
952
|
+
Tested on AMD Ryzen 9 5950X, 128-dimensional vectors:
|
|
953
|
+
|
|
954
|
+
### Native Performance (Rust)
|
|
955
|
+
|
|
956
|
+
| Operation | Throughput | Latency (p50) | Latency (p99) |
|
|
957
|
+
|-----------|------------|---------------|---------------|
|
|
958
|
+
| Insert | 52,341 ops/sec | 0.019 ms | 0.045 ms |
|
|
959
|
+
| Search (k=10) | 11,234 ops/sec | 0.089 ms | 0.156 ms |
|
|
960
|
+
| Search (k=100) | 8,932 ops/sec | 0.112 ms | 0.203 ms |
|
|
961
|
+
| Delete | 45,678 ops/sec | 0.022 ms | 0.051 ms |
|
|
962
|
+
|
|
963
|
+
**Memory Usage**: ~50 bytes per 128-dim vector (including index)
|
|
964
|
+
|
|
965
|
+
### Comparison with Alternatives
|
|
966
|
+
|
|
967
|
+
| Database | Insert (ops/sec) | Search (ops/sec) | Memory per Vector | Node.js | Browser |
|
|
968
|
+
|----------|------------------|------------------|-------------------|---------|---------|
|
|
969
|
+
| **Ruvector (Native)** | **52,341** | **11,234** | **50 bytes** | ✅ | ❌ |
|
|
970
|
+
| **Ruvector (WASM)** | **~1,000** | **~100** | **50 bytes** | ✅ | ✅ |
|
|
971
|
+
| Faiss (HNSW) | 38,200 | 9,800 | 68 bytes | ❌ | ❌ |
|
|
972
|
+
| Hnswlib | 41,500 | 10,200 | 62 bytes | ✅ | ❌ |
|
|
973
|
+
| ChromaDB | ~1,000 | ~20 | 150 bytes | ✅ | ❌ |
|
|
974
|
+
|
|
975
|
+
*Benchmarks measured with 100K vectors, 128 dimensions, k=10*
|
|
976
|
+
|
|
977
|
+
## 🔍 Comparison with Other Vector Databases
|
|
978
|
+
|
|
979
|
+
Comprehensive comparison of Ruvector against popular vector database solutions:
|
|
980
|
+
|
|
981
|
+
| Feature | Ruvector | Pinecone | Qdrant | Weaviate | Milvus | ChromaDB | Faiss |
|
|
982
|
+
|---------|----------|----------|--------|----------|--------|----------|-------|
|
|
983
|
+
| **Deployment** |
|
|
984
|
+
| Installation | `npm install` ✅ | Cloud API ☁️ | Docker 🐳 | Docker 🐳 | Docker/K8s 🐳 | `pip install` 🐍 | `pip install` 🐍 |
|
|
985
|
+
| Node.js Native | ✅ First-class | ❌ API only | ⚠️ HTTP API | ⚠️ HTTP API | ⚠️ HTTP API | ❌ Python | ❌ Python |
|
|
986
|
+
| Setup Time | < 1 minute | 5-10 minutes | 10-30 minutes | 15-30 minutes | 30-60 minutes | 5 minutes | 5 minutes |
|
|
987
|
+
| Infrastructure | None required | Managed cloud | Self-hosted | Self-hosted | Self-hosted | Embedded | Embedded |
|
|
988
|
+
| **Performance** |
|
|
989
|
+
| Query Latency (p50) | **<0.5ms** | ~2-5ms | ~1-2ms | ~2-3ms | ~3-5ms | ~50ms | ~1ms |
|
|
990
|
+
| Insert Throughput | **52,341 ops/sec** | ~10,000 ops/sec | ~20,000 ops/sec | ~15,000 ops/sec | ~25,000 ops/sec | ~1,000 ops/sec | ~40,000 ops/sec |
|
|
991
|
+
| Memory per Vector (128d) | **50 bytes** | ~80 bytes | 62 bytes | ~100 bytes | ~70 bytes | 150 bytes | 68 bytes |
|
|
992
|
+
| Recall @ k=10 | 95%+ | 93% | 94% | 92% | 96% | 85% | 97% |
|
|
993
|
+
| **Platform Support** |
|
|
994
|
+
| Linux | ✅ Native | ☁️ API | ✅ Docker | ✅ Docker | ✅ Docker | ✅ Python | ✅ Python |
|
|
995
|
+
| macOS | ✅ Native | ☁️ API | ✅ Docker | ✅ Docker | ✅ Docker | ✅ Python | ✅ Python |
|
|
996
|
+
| Windows | ✅ Native | ☁️ API | ✅ Docker | ✅ Docker | ⚠️ WSL2 | ✅ Python | ✅ Python |
|
|
997
|
+
| Browser/WASM | ✅ Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
|
|
998
|
+
| ARM64 | ✅ Native | ☁️ API | ✅ Yes | ✅ Yes | ⚠️ Limited | ✅ Yes | ✅ Yes |
|
|
999
|
+
| Alpine Linux | ✅ WASM | ☁️ API | ⚠️ Build from source | ⚠️ Build from source | ❌ No | ✅ Yes | ✅ Yes |
|
|
1000
|
+
| **Features** |
|
|
1001
|
+
| Distance Metrics | Cosine, L2, Dot | Cosine, L2, Dot | 11 metrics | 10 metrics | 8 metrics | L2, Cosine, IP | L2, IP, Cosine |
|
|
1002
|
+
| Filtering | ✅ Metadata | ✅ Advanced | ✅ Advanced | ✅ Advanced | ✅ Advanced | ✅ Basic | ❌ Limited |
|
|
1003
|
+
| Persistence | ✅ File-based | ☁️ Managed | ✅ Disk | ✅ Disk | ✅ Disk | ✅ DuckDB | ❌ Memory |
|
|
1004
|
+
| Indexing | HNSW | Proprietary | HNSW | HNSW | IVF/HNSW | HNSW | IVF/HNSW |
|
|
1005
|
+
| Quantization | ✅ PQ | ✅ Yes | ✅ Scalar | ✅ PQ | ✅ PQ/SQ | ❌ No | ✅ PQ |
|
|
1006
|
+
| Batch Operations | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Yes |
|
|
1007
|
+
| **Developer Experience** |
|
|
1008
|
+
| TypeScript Types | ✅ Full | ✅ Generated | ⚠️ Community | ⚠️ Community | ⚠️ Community | ⚠️ Partial | ❌ No |
|
|
1009
|
+
| Documentation | ✅ Excellent | ✅ Excellent | ✅ Good | ✅ Good | ✅ Good | ✅ Good | ⚠️ Technical |
|
|
1010
|
+
| Examples | ✅ Many | ✅ Many | ✅ Good | ✅ Good | ✅ Many | ✅ Good | ⚠️ Limited |
|
|
1011
|
+
| CLI Tools | ✅ Included | ⚠️ Limited | ✅ Yes | ✅ Yes | ✅ Yes | ⚠️ Basic | ❌ No |
|
|
1012
|
+
| **Operations** |
|
|
1013
|
+
| Monitoring | ✅ Metrics | ✅ Dashboard | ✅ Prometheus | ✅ Prometheus | ✅ Prometheus | ⚠️ Basic | ❌ No |
|
|
1014
|
+
| Backups | ✅ File copy | ☁️ Automatic | ✅ Snapshots | ✅ Snapshots | ✅ Snapshots | ✅ File copy | ❌ Manual |
|
|
1015
|
+
| High Availability | ⚠️ App-level | ✅ Built-in | ✅ Clustering | ✅ Clustering | ✅ Clustering | ❌ No | ❌ No |
|
|
1016
|
+
| Auto-Scaling | ⚠️ App-level | ✅ Automatic | ⚠️ Manual | ⚠️ Manual | ⚠️ K8s HPA | ❌ No | ❌ No |
|
|
1017
|
+
| **Cost** |
|
|
1018
|
+
| Pricing Model | Free (MIT) | Pay-per-use | Free (Apache) | Free (BSD) | Free (Apache) | Free (Apache) | Free (MIT) |
|
|
1019
|
+
| Monthly Cost (1M vectors) | **$0** | ~$70-200 | ~$20-50 (infra) | ~$30-60 (infra) | ~$50-100 (infra) | $0 | $0 |
|
|
1020
|
+
| Monthly Cost (10M vectors) | **$0** | ~$500-1000 | ~$100-200 (infra) | ~$150-300 (infra) | ~$200-400 (infra) | $0 | $0 |
|
|
1021
|
+
| API Rate Limits | None | Yes | None | None | None | None | None |
|
|
1022
|
+
| **Use Cases** |
|
|
1023
|
+
| RAG Systems | ✅ Excellent | ✅ Excellent | ✅ Excellent | ✅ Excellent | ✅ Excellent | ✅ Good | ⚠️ Limited |
|
|
1024
|
+
| Serverless | ✅ Perfect | ✅ Good | ❌ No | ❌ No | ❌ No | ⚠️ Possible | ⚠️ Possible |
|
|
1025
|
+
| Edge Computing | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ⚠️ Possible |
|
|
1026
|
+
| Production Scale (100M+) | ⚠️ Single node | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Excellent | ⚠️ Limited | ⚠️ Manual |
|
|
1027
|
+
| Embedded Apps | ✅ Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ⚠️ Possible | ✅ Good |
|
|
1028
|
+
|
|
1029
|
+
### When to Choose Ruvector
|
|
1030
|
+
|
|
1031
|
+
✅ **Perfect for:**
|
|
1032
|
+
- **Node.js/TypeScript applications** needing embedded vector search
|
|
1033
|
+
- **Serverless and edge computing** where external services aren't practical
|
|
1034
|
+
- **Rapid prototyping and development** with minimal setup time
|
|
1035
|
+
- **RAG systems** with LangChain, LlamaIndex, or custom implementations
|
|
1036
|
+
- **Cost-sensitive projects** that can't afford cloud API pricing
|
|
1037
|
+
- **Offline-first applications** requiring local vector search
|
|
1038
|
+
- **Browser-based AI** with WebAssembly fallback
|
|
1039
|
+
- **Small to medium scale** (up to 10M vectors per instance)
|
|
1040
|
+
|
|
1041
|
+
⚠️ **Consider alternatives for:**
|
|
1042
|
+
- **Massive scale (100M+ vectors)** - Consider Pinecone, Milvus, or Qdrant clusters
|
|
1043
|
+
- **Multi-tenancy requirements** - Weaviate or Qdrant offer better isolation
|
|
1044
|
+
- **Distributed systems** - Milvus provides better horizontal scaling
|
|
1045
|
+
- **Zero-ops cloud solution** - Pinecone handles all infrastructure
|
|
1046
|
+
|
|
1047
|
+
### Why Choose Ruvector Over...
|
|
1048
|
+
|
|
1049
|
+
**vs Pinecone:**
|
|
1050
|
+
- ✅ No API costs (save $1000s/month)
|
|
1051
|
+
- ✅ No network latency (10x faster queries)
|
|
1052
|
+
- ✅ No vendor lock-in
|
|
1053
|
+
- ✅ Works offline and in restricted environments
|
|
1054
|
+
- ❌ No managed multi-region clusters
|
|
1055
|
+
|
|
1056
|
+
**vs ChromaDB:**
|
|
1057
|
+
- ✅ 50x faster queries (native Rust vs Python)
|
|
1058
|
+
- ✅ True Node.js support (not HTTP API)
|
|
1059
|
+
- ✅ Better TypeScript integration
|
|
1060
|
+
- ✅ Lower memory usage
|
|
1061
|
+
- ❌ Smaller ecosystem and community
|
|
1062
|
+
|
|
1063
|
+
**vs Qdrant:**
|
|
1064
|
+
- ✅ Zero infrastructure setup
|
|
1065
|
+
- ✅ Embedded in your app (no Docker)
|
|
1066
|
+
- ✅ Better for serverless environments
|
|
1067
|
+
- ✅ Native Node.js bindings
|
|
1068
|
+
- ❌ No built-in clustering or HA
|
|
1069
|
+
|
|
1070
|
+
**vs Faiss:**
|
|
1071
|
+
- ✅ Full Node.js support (Faiss is Python-only)
|
|
1072
|
+
- ✅ Easier API and better developer experience
|
|
1073
|
+
- ✅ Built-in persistence and metadata
|
|
1074
|
+
- ⚠️ Slightly lower recall at same performance
|
|
1075
|
+
|
|
1076
|
+
## 🎯 Real-World Tutorials
|
|
1077
|
+
|
|
1078
|
+
### Tutorial 1: Building a RAG System with OpenAI
|
|
1079
|
+
|
|
1080
|
+
**What you'll learn:** Create a production-ready Retrieval-Augmented Generation system that enhances LLM responses with relevant context from your documents.
|
|
1081
|
+
|
|
1082
|
+
**Prerequisites:**
|
|
1083
|
+
```bash
|
|
1084
|
+
npm install ruvector openai
|
|
1085
|
+
export OPENAI_API_KEY="your-api-key-here"
|
|
1086
|
+
```
|
|
1087
|
+
|
|
1088
|
+
**Complete Implementation:**
|
|
1089
|
+
|
|
1090
|
+
```javascript
|
|
1091
|
+
const { VectorDb } = require('ruvector');
|
|
1092
|
+
const OpenAI = require('openai');
|
|
1093
|
+
|
|
1094
|
+
class RAGSystem {
|
|
1095
|
+
constructor() {
|
|
1096
|
+
// Initialize OpenAI client
|
|
1097
|
+
this.openai = new OpenAI({
|
|
1098
|
+
apiKey: process.env.OPENAI_API_KEY
|
|
1099
|
+
});
|
|
1100
|
+
|
|
1101
|
+
// Create vector database for OpenAI embeddings
|
|
1102
|
+
// text-embedding-ada-002 produces 1536-dimensional vectors
|
|
1103
|
+
this.db = new VectorDb({
|
|
1104
|
+
dimensions: 1536,
|
|
1105
|
+
maxElements: 100000,
|
|
1106
|
+
storagePath: './rag-knowledge-base.db'
|
|
1107
|
+
});
|
|
1108
|
+
|
|
1109
|
+
console.log('✅ RAG System initialized');
|
|
1110
|
+
}
|
|
1111
|
+
|
|
1112
|
+
// Step 1: Index your knowledge base
|
|
1113
|
+
async indexDocuments(documents) {
|
|
1114
|
+
console.log(`📚 Indexing ${documents.length} documents...`);
|
|
1115
|
+
|
|
1116
|
+
for (let i = 0; i < documents.length; i++) {
|
|
1117
|
+
const doc = documents[i];
|
|
1118
|
+
|
|
1119
|
+
// Generate embedding for the document
|
|
1120
|
+
const response = await this.openai.embeddings.create({
|
|
1121
|
+
model: 'text-embedding-ada-002',
|
|
1122
|
+
input: doc.content
|
|
1123
|
+
});
|
|
1124
|
+
|
|
1125
|
+
// Store in vector database
|
|
1126
|
+
await this.db.insert({
|
|
1127
|
+
id: doc.id || `doc_${i}`,
|
|
1128
|
+
vector: new Float32Array(response.data[0].embedding),
|
|
1129
|
+
metadata: {
|
|
1130
|
+
title: doc.title,
|
|
1131
|
+
content: doc.content,
|
|
1132
|
+
source: doc.source,
|
|
1133
|
+
date: doc.date || new Date().toISOString()
|
|
1134
|
+
}
|
|
1135
|
+
});
|
|
1136
|
+
|
|
1137
|
+
console.log(` ✅ Indexed: ${doc.title}`);
|
|
1138
|
+
}
|
|
1139
|
+
|
|
1140
|
+
const count = await this.db.len();
|
|
1141
|
+
console.log(`\n✅ Indexed ${count} documents total`);
|
|
1142
|
+
}
|
|
1143
|
+
|
|
1144
|
+
// Step 2: Retrieve relevant context for a query
|
|
1145
|
+
async retrieveContext(query, k = 3) {
|
|
1146
|
+
console.log(`🔍 Searching for: "${query}"`);
|
|
1147
|
+
|
|
1148
|
+
// Generate embedding for the query
|
|
1149
|
+
const response = await this.openai.embeddings.create({
|
|
1150
|
+
model: 'text-embedding-ada-002',
|
|
1151
|
+
input: query
|
|
1152
|
+
});
|
|
1153
|
+
|
|
1154
|
+
// Search for similar documents
|
|
1155
|
+
const results = await this.db.search({
|
|
1156
|
+
vector: new Float32Array(response.data[0].embedding),
|
|
1157
|
+
k: k,
|
|
1158
|
+
threshold: 0.7 // Only use highly relevant results
|
|
1159
|
+
});
|
|
1160
|
+
|
|
1161
|
+
console.log(`📄 Found ${results.length} relevant documents\n`);
|
|
1162
|
+
|
|
1163
|
+
return results.map(r => ({
|
|
1164
|
+
content: r.metadata.content,
|
|
1165
|
+
title: r.metadata.title,
|
|
1166
|
+
score: r.score
|
|
1167
|
+
}));
|
|
1168
|
+
}
|
|
1169
|
+
|
|
1170
|
+
// Step 3: Generate answer with retrieved context
|
|
1171
|
+
async answer(question) {
|
|
1172
|
+
// Retrieve relevant context
|
|
1173
|
+
const context = await this.retrieveContext(question, 3);
|
|
1174
|
+
|
|
1175
|
+
if (context.length === 0) {
|
|
1176
|
+
return "I don't have enough information to answer that question.";
|
|
1177
|
+
}
|
|
1178
|
+
|
|
1179
|
+
// Build prompt with context
|
|
1180
|
+
const contextText = context
|
|
1181
|
+
.map((doc, i) => `[${i + 1}] ${doc.title}\n${doc.content}`)
|
|
1182
|
+
.join('\n\n');
|
|
1183
|
+
|
|
1184
|
+
const prompt = `Answer the question based on the following context. If the context doesn't contain the answer, say so.
|
|
1185
|
+
|
|
1186
|
+
Context:
|
|
1187
|
+
${contextText}
|
|
1188
|
+
|
|
1189
|
+
Question: ${question}
|
|
1190
|
+
|
|
1191
|
+
Answer:`;
|
|
1192
|
+
|
|
1193
|
+
console.log('🤖 Generating answer...\n');
|
|
1194
|
+
|
|
1195
|
+
// Generate completion
|
|
1196
|
+
const completion = await this.openai.chat.completions.create({
|
|
1197
|
+
model: 'gpt-4',
|
|
1198
|
+
messages: [
|
|
1199
|
+
{ role: 'system', content: 'You are a helpful assistant that answers questions based on provided context.' },
|
|
1200
|
+
{ role: 'user', content: prompt }
|
|
1201
|
+
],
|
|
1202
|
+
temperature: 0.3 // Lower temperature for more factual responses
|
|
1203
|
+
});
|
|
1204
|
+
|
|
1205
|
+
return {
|
|
1206
|
+
answer: completion.choices[0].message.content,
|
|
1207
|
+
sources: context.map(c => c.title)
|
|
1208
|
+
};
|
|
1209
|
+
}
|
|
1210
|
+
}
|
|
1211
|
+
|
|
1212
|
+
// Example Usage
|
|
1213
|
+
async function main() {
|
|
1214
|
+
const rag = new RAGSystem();
|
|
1215
|
+
|
|
1216
|
+
// Step 1: Index your knowledge base
|
|
1217
|
+
const documents = [
|
|
1218
|
+
{
|
|
1219
|
+
id: 'doc1',
|
|
1220
|
+
title: 'Ruvector Introduction',
|
|
1221
|
+
content: 'Ruvector is a high-performance vector database for Node.js built in Rust. It provides sub-millisecond query latency and supports over 52,000 inserts per second.',
|
|
1222
|
+
source: 'documentation'
|
|
1223
|
+
},
|
|
1224
|
+
{
|
|
1225
|
+
id: 'doc2',
|
|
1226
|
+
title: 'Vector Databases Explained',
|
|
1227
|
+
content: 'Vector databases store data as high-dimensional vectors, enabling semantic similarity search. They are essential for AI applications like RAG systems and recommendation engines.',
|
|
1228
|
+
source: 'blog'
|
|
1229
|
+
},
|
|
1230
|
+
{
|
|
1231
|
+
id: 'doc3',
|
|
1232
|
+
title: 'HNSW Algorithm',
|
|
1233
|
+
content: 'Hierarchical Navigable Small World (HNSW) is a graph-based algorithm for approximate nearest neighbor search. It provides excellent recall with low latency.',
|
|
1234
|
+
source: 'research'
|
|
1235
|
+
}
|
|
1236
|
+
];
|
|
1237
|
+
|
|
1238
|
+
await rag.indexDocuments(documents);
|
|
1239
|
+
|
|
1240
|
+
// Step 2: Ask questions
|
|
1241
|
+
console.log('\n' + '='.repeat(60) + '\n');
|
|
1242
|
+
|
|
1243
|
+
const result = await rag.answer('What is Ruvector and what are its performance characteristics?');
|
|
1244
|
+
|
|
1245
|
+
console.log('📝 Answer:', result.answer);
|
|
1246
|
+
console.log('\n📚 Sources:', result.sources.join(', '));
|
|
1247
|
+
}
|
|
1248
|
+
|
|
1249
|
+
main().catch(console.error);
|
|
1250
|
+
```
|
|
1251
|
+
|
|
1252
|
+
**Expected Output:**
|
|
1253
|
+
```
|
|
1254
|
+
✅ RAG System initialized
|
|
1255
|
+
📚 Indexing 3 documents...
|
|
1256
|
+
✅ Indexed: Ruvector Introduction
|
|
1257
|
+
✅ Indexed: Vector Databases Explained
|
|
1258
|
+
✅ Indexed: HNSW Algorithm
|
|
1259
|
+
|
|
1260
|
+
✅ Indexed 3 documents total
|
|
1261
|
+
|
|
1262
|
+
============================================================
|
|
1263
|
+
|
|
1264
|
+
🔍 Searching for: "What is Ruvector and what are its performance characteristics?"
|
|
1265
|
+
📄 Found 2 relevant documents
|
|
1266
|
+
|
|
1267
|
+
🤖 Generating answer...
|
|
1268
|
+
|
|
1269
|
+
📝 Answer: Ruvector is a high-performance vector database built in Rust for Node.js applications. Its key performance characteristics include:
|
|
1270
|
+
- Sub-millisecond query latency
|
|
1271
|
+
- Over 52,000 inserts per second
|
|
1272
|
+
- Optimized for semantic similarity search
|
|
1273
|
+
|
|
1274
|
+
📚 Sources: Ruvector Introduction, Vector Databases Explained
|
|
1275
|
+
```
|
|
1276
|
+
|
|
1277
|
+
**Production Tips:**
|
|
1278
|
+
- ✅ Use batch embedding for better throughput (OpenAI supports up to 2048 texts)
|
|
1279
|
+
- ✅ Implement caching for frequently asked questions
|
|
1280
|
+
- ✅ Add error handling for API rate limits
|
|
1281
|
+
- ✅ Monitor token usage and costs
|
|
1282
|
+
- ✅ Regularly update your knowledge base
|
|
1283
|
+
|
|
1284
|
+
---
|
|
1285
|
+
|
|
1286
|
+
### Tutorial 2: Semantic Search Engine
|
|
1287
|
+
|
|
1288
|
+
**What you'll learn:** Build a semantic search engine that understands meaning, not just keywords.
|
|
1289
|
+
|
|
1290
|
+
**Prerequisites:**
|
|
1291
|
+
```bash
|
|
1292
|
+
npm install ruvector @xenova/transformers
|
|
1293
|
+
```
|
|
1294
|
+
|
|
1295
|
+
**Complete Implementation:**
|
|
1296
|
+
|
|
1297
|
+
```javascript
|
|
1298
|
+
const { VectorDb } = require('ruvector');
|
|
1299
|
+
const { pipeline } = require('@xenova/transformers');
|
|
1300
|
+
|
|
1301
|
+
class SemanticSearchEngine {
|
|
1302
|
+
constructor() {
|
|
1303
|
+
this.db = null;
|
|
1304
|
+
this.embedder = null;
|
|
1305
|
+
}
|
|
1306
|
+
|
|
1307
|
+
// Step 1: Initialize the embedding model
|
|
1308
|
+
async initialize() {
|
|
1309
|
+
console.log('🚀 Initializing semantic search engine...');
|
|
1310
|
+
|
|
1311
|
+
// Load sentence-transformers model (runs locally, no API needed!)
|
|
1312
|
+
console.log('📥 Loading embedding model...');
|
|
1313
|
+
this.embedder = await pipeline(
|
|
1314
|
+
'feature-extraction',
|
|
1315
|
+
'Xenova/all-MiniLM-L6-v2'
|
|
1316
|
+
);
|
|
1317
|
+
|
|
1318
|
+
// Create vector database (384 dimensions for all-MiniLM-L6-v2)
|
|
1319
|
+
this.db = new VectorDb({
|
|
1320
|
+
dimensions: 384,
|
|
1321
|
+
maxElements: 50000,
|
|
1322
|
+
storagePath: './semantic-search.db'
|
|
1323
|
+
});
|
|
1324
|
+
|
|
1325
|
+
console.log('✅ Search engine ready!\n');
|
|
1326
|
+
}
|
|
1327
|
+
|
|
1328
|
+
// Step 2: Generate embeddings
|
|
1329
|
+
async embed(text) {
|
|
1330
|
+
const output = await this.embedder(text, {
|
|
1331
|
+
pooling: 'mean',
|
|
1332
|
+
normalize: true
|
|
1333
|
+
});
|
|
1334
|
+
|
|
1335
|
+
// Convert to Float32Array
|
|
1336
|
+
return new Float32Array(output.data);
|
|
1337
|
+
}
|
|
1338
|
+
|
|
1339
|
+
// Step 3: Index documents
|
|
1340
|
+
async indexDocuments(documents) {
|
|
1341
|
+
console.log(`📚 Indexing ${documents.length} documents...`);
|
|
1342
|
+
|
|
1343
|
+
for (const doc of documents) {
|
|
1344
|
+
const vector = await this.embed(doc.content);
|
|
1345
|
+
|
|
1346
|
+
await this.db.insert({
|
|
1347
|
+
id: doc.id,
|
|
1348
|
+
vector: vector,
|
|
1349
|
+
metadata: {
|
|
1350
|
+
title: doc.title,
|
|
1351
|
+
content: doc.content,
|
|
1352
|
+
category: doc.category,
|
|
1353
|
+
url: doc.url
|
|
1354
|
+
}
|
|
1355
|
+
});
|
|
1356
|
+
|
|
1357
|
+
console.log(` ✅ ${doc.title}`);
|
|
1358
|
+
}
|
|
1359
|
+
|
|
1360
|
+
const count = await this.db.len();
|
|
1361
|
+
console.log(`\n✅ Indexed ${count} documents\n`);
|
|
1362
|
+
}
|
|
1363
|
+
|
|
1364
|
+
// Step 4: Semantic search
|
|
1365
|
+
async search(query, options = {}) {
|
|
1366
|
+
const {
|
|
1367
|
+
k = 5,
|
|
1368
|
+
category = null,
|
|
1369
|
+
threshold = 0.3
|
|
1370
|
+
} = options;
|
|
1371
|
+
|
|
1372
|
+
console.log(`🔍 Searching for: "${query}"`);
|
|
1373
|
+
|
|
1374
|
+
// Generate query embedding
|
|
1375
|
+
const queryVector = await this.embed(query);
|
|
1376
|
+
|
|
1377
|
+
// Search vector database
|
|
1378
|
+
const results = await this.db.search({
|
|
1379
|
+
vector: queryVector,
|
|
1380
|
+
k: k * 2, // Get more results for filtering
|
|
1381
|
+
threshold: threshold
|
|
1382
|
+
});
|
|
1383
|
+
|
|
1384
|
+
// Filter by category if specified
|
|
1385
|
+
let filtered = results;
|
|
1386
|
+
if (category) {
|
|
1387
|
+
filtered = results.filter(r => r.metadata.category === category);
|
|
1388
|
+
}
|
|
1389
|
+
|
|
1390
|
+
// Return top k after filtering
|
|
1391
|
+
const final = filtered.slice(0, k);
|
|
1392
|
+
|
|
1393
|
+
console.log(`📄 Found ${final.length} results\n`);
|
|
1394
|
+
|
|
1395
|
+
return final.map(r => ({
|
|
1396
|
+
id: r.id,
|
|
1397
|
+
title: r.metadata.title,
|
|
1398
|
+
content: r.metadata.content,
|
|
1399
|
+
category: r.metadata.category,
|
|
1400
|
+
score: r.score,
|
|
1401
|
+
url: r.metadata.url
|
|
1402
|
+
}));
|
|
1403
|
+
}
|
|
1404
|
+
|
|
1405
|
+
// Step 5: Find similar documents
|
|
1406
|
+
async findSimilar(documentId, k = 5) {
|
|
1407
|
+
const doc = await this.db.get(documentId);
|
|
1408
|
+
|
|
1409
|
+
if (!doc) {
|
|
1410
|
+
throw new Error(`Document ${documentId} not found`);
|
|
1411
|
+
}
|
|
1412
|
+
|
|
1413
|
+
const results = await this.db.search({
|
|
1414
|
+
vector: doc.vector,
|
|
1415
|
+
k: k + 1 // +1 because the document itself will be included
|
|
1416
|
+
});
|
|
1417
|
+
|
|
1418
|
+
// Remove the document itself from results
|
|
1419
|
+
return results
|
|
1420
|
+
.filter(r => r.id !== documentId)
|
|
1421
|
+
.slice(0, k);
|
|
1422
|
+
}
|
|
1423
|
+
}
|
|
1424
|
+
|
|
1425
|
+
// Example Usage
|
|
1426
|
+
async function main() {
|
|
1427
|
+
const engine = new SemanticSearchEngine();
|
|
1428
|
+
await engine.initialize();
|
|
1429
|
+
|
|
1430
|
+
// Sample documents (in production, load from your database)
|
|
1431
|
+
const documents = [
|
|
1432
|
+
{
|
|
1433
|
+
id: '1',
|
|
1434
|
+
title: 'Understanding Neural Networks',
|
|
1435
|
+
content: 'Neural networks are computing systems inspired by biological neural networks. They learn to perform tasks by considering examples.',
|
|
1436
|
+
category: 'AI',
|
|
1437
|
+
url: '/docs/neural-networks'
|
|
1438
|
+
},
|
|
1439
|
+
{
|
|
1440
|
+
id: '2',
|
|
1441
|
+
title: 'Introduction to Machine Learning',
|
|
1442
|
+
content: 'Machine learning is a subset of artificial intelligence that provides systems the ability to learn and improve from experience.',
|
|
1443
|
+
category: 'AI',
|
|
1444
|
+
url: '/docs/machine-learning'
|
|
1445
|
+
},
|
|
1446
|
+
{
|
|
1447
|
+
id: '3',
|
|
1448
|
+
title: 'Web Development Best Practices',
|
|
1449
|
+
content: 'Modern web development involves responsive design, performance optimization, and accessibility considerations.',
|
|
1450
|
+
category: 'Web',
|
|
1451
|
+
url: '/docs/web-dev'
|
|
1452
|
+
},
|
|
1453
|
+
{
|
|
1454
|
+
id: '4',
|
|
1455
|
+
title: 'Deep Learning Applications',
|
|
1456
|
+
content: 'Deep learning has revolutionized computer vision, natural language processing, and speech recognition.',
|
|
1457
|
+
category: 'AI',
|
|
1458
|
+
url: '/docs/deep-learning'
|
|
1459
|
+
}
|
|
1460
|
+
];
|
|
1461
|
+
|
|
1462
|
+
// Index documents
|
|
1463
|
+
await engine.indexDocuments(documents);
|
|
1464
|
+
|
|
1465
|
+
// Example 1: Basic semantic search
|
|
1466
|
+
console.log('Example 1: Basic Search\n' + '='.repeat(60));
|
|
1467
|
+
const results1 = await engine.search('AI and neural nets');
|
|
1468
|
+
results1.forEach((result, i) => {
|
|
1469
|
+
console.log(`${i + 1}. ${result.title} (Score: ${result.score.toFixed(3)})`);
|
|
1470
|
+
console.log(` ${result.content.slice(0, 80)}...`);
|
|
1471
|
+
console.log(` Category: ${result.category}\n`);
|
|
1472
|
+
});
|
|
1473
|
+
|
|
1474
|
+
// Example 2: Category-filtered search
|
|
1475
|
+
console.log('\nExample 2: Category-Filtered Search\n' + '='.repeat(60));
|
|
1476
|
+
const results2 = await engine.search('learning algorithms', {
|
|
1477
|
+
category: 'AI',
|
|
1478
|
+
k: 3
|
|
1479
|
+
});
|
|
1480
|
+
results2.forEach((result, i) => {
|
|
1481
|
+
console.log(`${i + 1}. ${result.title} (Score: ${result.score.toFixed(3)})`);
|
|
1482
|
+
});
|
|
1483
|
+
|
|
1484
|
+
// Example 3: Find similar documents
|
|
1485
|
+
console.log('\n\nExample 3: Find Similar Documents\n' + '='.repeat(60));
|
|
1486
|
+
const similar = await engine.findSimilar('1', 2);
|
|
1487
|
+
console.log('Documents similar to "Understanding Neural Networks":');
|
|
1488
|
+
similar.forEach((doc, i) => {
|
|
1489
|
+
console.log(`${i + 1}. ${doc.metadata.title} (Score: ${doc.score.toFixed(3)})`);
|
|
1490
|
+
});
|
|
1491
|
+
}
|
|
1492
|
+
|
|
1493
|
+
main().catch(console.error);
|
|
1494
|
+
```
|
|
1495
|
+
|
|
1496
|
+
**Key Features:**
|
|
1497
|
+
- ✅ Runs completely locally (no API keys needed)
|
|
1498
|
+
- ✅ Understands semantic meaning, not just keywords
|
|
1499
|
+
- ✅ Category filtering for better results
|
|
1500
|
+
- ✅ "Find similar" functionality
|
|
1501
|
+
- ✅ Fast: ~10ms query latency
|
|
1502
|
+
|
|
1503
|
+
---
|
|
1504
|
+
|
|
1505
|
+
### Tutorial 3: AI Agent Memory System
|
|
1506
|
+
|
|
1507
|
+
**What you'll learn:** Implement a memory system for AI agents that remembers past experiences and learns from them.
|
|
1508
|
+
|
|
1509
|
+
**Complete Implementation:**
|
|
1510
|
+
|
|
1511
|
+
```javascript
|
|
1512
|
+
const { VectorDb } = require('ruvector');
|
|
1513
|
+
|
|
1514
|
+
class AgentMemory {
|
|
1515
|
+
constructor(agentId) {
|
|
1516
|
+
this.agentId = agentId;
|
|
1517
|
+
|
|
1518
|
+
// Create separate databases for different memory types
|
|
1519
|
+
this.episodicMemory = new VectorDb({
|
|
1520
|
+
dimensions: 768,
|
|
1521
|
+
storagePath: `./memory/${agentId}-episodic.db`
|
|
1522
|
+
});
|
|
1523
|
+
|
|
1524
|
+
this.semanticMemory = new VectorDb({
|
|
1525
|
+
dimensions: 768,
|
|
1526
|
+
storagePath: `./memory/${agentId}-semantic.db`
|
|
1527
|
+
});
|
|
1528
|
+
|
|
1529
|
+
console.log(`🧠 Memory system initialized for agent: ${agentId}`);
|
|
1530
|
+
}
|
|
1531
|
+
|
|
1532
|
+
// Step 1: Store an experience (episodic memory)
|
|
1533
|
+
async storeExperience(experience) {
|
|
1534
|
+
const {
|
|
1535
|
+
state,
|
|
1536
|
+
action,
|
|
1537
|
+
result,
|
|
1538
|
+
reward,
|
|
1539
|
+
embedding
|
|
1540
|
+
} = experience;
|
|
1541
|
+
|
|
1542
|
+
const experienceId = `exp_${Date.now()}_${Math.random()}`;
|
|
1543
|
+
|
|
1544
|
+
await this.episodicMemory.insert({
|
|
1545
|
+
id: experienceId,
|
|
1546
|
+
vector: new Float32Array(embedding),
|
|
1547
|
+
metadata: {
|
|
1548
|
+
state: state,
|
|
1549
|
+
action: action,
|
|
1550
|
+
result: result,
|
|
1551
|
+
reward: reward,
|
|
1552
|
+
timestamp: Date.now(),
|
|
1553
|
+
type: 'episodic'
|
|
1554
|
+
}
|
|
1555
|
+
});
|
|
1556
|
+
|
|
1557
|
+
console.log(`💾 Stored experience: ${action} -> ${result} (reward: ${reward})`);
|
|
1558
|
+
return experienceId;
|
|
1559
|
+
}
|
|
1560
|
+
|
|
1561
|
+
// Step 2: Store learned knowledge (semantic memory)
|
|
1562
|
+
async storeKnowledge(knowledge) {
|
|
1563
|
+
const {
|
|
1564
|
+
concept,
|
|
1565
|
+
description,
|
|
1566
|
+
embedding,
|
|
1567
|
+
confidence = 1.0
|
|
1568
|
+
} = knowledge;
|
|
1569
|
+
|
|
1570
|
+
const knowledgeId = `know_${Date.now()}`;
|
|
1571
|
+
|
|
1572
|
+
await this.semanticMemory.insert({
|
|
1573
|
+
id: knowledgeId,
|
|
1574
|
+
vector: new Float32Array(embedding),
|
|
1575
|
+
metadata: {
|
|
1576
|
+
concept: concept,
|
|
1577
|
+
description: description,
|
|
1578
|
+
confidence: confidence,
|
|
1579
|
+
learned: Date.now(),
|
|
1580
|
+
uses: 0,
|
|
1581
|
+
type: 'semantic'
|
|
1582
|
+
}
|
|
1583
|
+
});
|
|
1584
|
+
|
|
1585
|
+
console.log(`📚 Learned: ${concept}`);
|
|
1586
|
+
return knowledgeId;
|
|
1587
|
+
}
|
|
1588
|
+
|
|
1589
|
+
// Step 3: Recall similar experiences
|
|
1590
|
+
async recallExperiences(currentState, k = 5) {
|
|
1591
|
+
console.log(`🔍 Recalling similar experiences...`);
|
|
1592
|
+
|
|
1593
|
+
const results = await this.episodicMemory.search({
|
|
1594
|
+
vector: new Float32Array(currentState.embedding),
|
|
1595
|
+
k: k,
|
|
1596
|
+
threshold: 0.6 // Only recall reasonably similar experiences
|
|
1597
|
+
});
|
|
1598
|
+
|
|
1599
|
+
// Sort by reward to prioritize successful experiences
|
|
1600
|
+
const sorted = results.sort((a, b) => b.metadata.reward - a.metadata.reward);
|
|
1601
|
+
|
|
1602
|
+
console.log(`📝 Recalled ${sorted.length} relevant experiences`);
|
|
1603
|
+
|
|
1604
|
+
return sorted.map(r => ({
|
|
1605
|
+
state: r.metadata.state,
|
|
1606
|
+
action: r.metadata.action,
|
|
1607
|
+
result: r.metadata.result,
|
|
1608
|
+
reward: r.metadata.reward,
|
|
1609
|
+
similarity: r.score
|
|
1610
|
+
}));
|
|
1611
|
+
}
|
|
1612
|
+
|
|
1613
|
+
// Step 4: Query knowledge base
|
|
1614
|
+
async queryKnowledge(query, k = 3) {
|
|
1615
|
+
const results = await this.semanticMemory.search({
|
|
1616
|
+
vector: new Float32Array(query.embedding),
|
|
1617
|
+
k: k
|
|
1618
|
+
});
|
|
1619
|
+
|
|
1620
|
+
// Update usage statistics
|
|
1621
|
+
for (const result of results) {
|
|
1622
|
+
const knowledge = await this.semanticMemory.get(result.id);
|
|
1623
|
+
if (knowledge) {
|
|
1624
|
+
knowledge.metadata.uses += 1;
|
|
1625
|
+
// In production, update the entry
|
|
1626
|
+
}
|
|
1627
|
+
}
|
|
1628
|
+
|
|
1629
|
+
return results.map(r => ({
|
|
1630
|
+
concept: r.metadata.concept,
|
|
1631
|
+
description: r.metadata.description,
|
|
1632
|
+
confidence: r.metadata.confidence,
|
|
1633
|
+
relevance: r.score
|
|
1634
|
+
}));
|
|
1635
|
+
}
|
|
1636
|
+
|
|
1637
|
+
// Step 5: Reflect and learn from experiences
|
|
1638
|
+
async reflect() {
|
|
1639
|
+
console.log('\n🤔 Reflecting on experiences...');
|
|
1640
|
+
|
|
1641
|
+
// Get all experiences
|
|
1642
|
+
const totalExperiences = await this.episodicMemory.len();
|
|
1643
|
+
console.log(`📊 Total experiences: ${totalExperiences}`);
|
|
1644
|
+
|
|
1645
|
+
// Analyze success rate
|
|
1646
|
+
// In production, you'd aggregate experiences and extract patterns
|
|
1647
|
+
console.log('💡 Analysis complete');
|
|
1648
|
+
|
|
1649
|
+
return {
|
|
1650
|
+
totalExperiences: totalExperiences,
|
|
1651
|
+
knowledgeItems: await this.semanticMemory.len()
|
|
1652
|
+
};
|
|
1653
|
+
}
|
|
1654
|
+
|
|
1655
|
+
// Step 6: Get memory statistics
|
|
1656
|
+
async getStats() {
|
|
1657
|
+
return {
|
|
1658
|
+
episodicMemorySize: await this.episodicMemory.len(),
|
|
1659
|
+
semanticMemorySize: await this.semanticMemory.len(),
|
|
1660
|
+
agentId: this.agentId
|
|
1661
|
+
};
|
|
1662
|
+
}
|
|
1663
|
+
}
|
|
1664
|
+
|
|
1665
|
+
// Example Usage: Simulated agent learning to navigate
|
|
1666
|
+
async function main() {
|
|
1667
|
+
const agent = new AgentMemory('agent-001');
|
|
1668
|
+
|
|
1669
|
+
// Simulate embedding function (in production, use a real model)
|
|
1670
|
+
function embed(text) {
|
|
1671
|
+
return Array(768).fill(0).map(() => Math.random());
|
|
1672
|
+
}
|
|
1673
|
+
|
|
1674
|
+
console.log('\n' + '='.repeat(60));
|
|
1675
|
+
console.log('PHASE 1: Learning from experiences');
|
|
1676
|
+
console.log('='.repeat(60) + '\n');
|
|
1677
|
+
|
|
1678
|
+
// Store some experiences
|
|
1679
|
+
await agent.storeExperience({
|
|
1680
|
+
state: { location: 'room1', goal: 'room3' },
|
|
1681
|
+
action: 'move_north',
|
|
1682
|
+
result: 'reached room2',
|
|
1683
|
+
reward: 0.5,
|
|
1684
|
+
embedding: embed('navigating from room1 to room2')
|
|
1685
|
+
});
|
|
1686
|
+
|
|
1687
|
+
await agent.storeExperience({
|
|
1688
|
+
state: { location: 'room2', goal: 'room3' },
|
|
1689
|
+
action: 'move_east',
|
|
1690
|
+
result: 'reached room3',
|
|
1691
|
+
reward: 1.0,
|
|
1692
|
+
embedding: embed('navigating from room2 to room3')
|
|
1693
|
+
});
|
|
1694
|
+
|
|
1695
|
+
await agent.storeExperience({
|
|
1696
|
+
state: { location: 'room1', goal: 'room3' },
|
|
1697
|
+
action: 'move_south',
|
|
1698
|
+
result: 'hit wall',
|
|
1699
|
+
reward: -0.5,
|
|
1700
|
+
embedding: embed('failed navigation attempt')
|
|
1701
|
+
});
|
|
1702
|
+
|
|
1703
|
+
// Store learned knowledge
|
|
1704
|
+
await agent.storeKnowledge({
|
|
1705
|
+
concept: 'navigation_strategy',
|
|
1706
|
+
description: 'Moving north then east is efficient for reaching room3 from room1',
|
|
1707
|
+
embedding: embed('navigation strategy knowledge'),
|
|
1708
|
+
confidence: 0.9
|
|
1709
|
+
});
|
|
1710
|
+
|
|
1711
|
+
console.log('\n' + '='.repeat(60));
|
|
1712
|
+
console.log('PHASE 2: Applying memory');
|
|
1713
|
+
console.log('='.repeat(60) + '\n');
|
|
1714
|
+
|
|
1715
|
+
// Agent encounters a similar situation
|
|
1716
|
+
const currentState = {
|
|
1717
|
+
location: 'room1',
|
|
1718
|
+
goal: 'room3',
|
|
1719
|
+
embedding: embed('navigating from room1 to room3')
|
|
1720
|
+
};
|
|
1721
|
+
|
|
1722
|
+
// Recall relevant experiences
|
|
1723
|
+
const experiences = await agent.recallExperiences(currentState, 3);
|
|
1724
|
+
|
|
1725
|
+
console.log('\n📖 Recalled experiences:');
|
|
1726
|
+
experiences.forEach((exp, i) => {
|
|
1727
|
+
console.log(`${i + 1}. Action: ${exp.action} | Result: ${exp.result} | Reward: ${exp.reward} | Similarity: ${exp.similarity.toFixed(3)}`);
|
|
1728
|
+
});
|
|
1729
|
+
|
|
1730
|
+
// Query relevant knowledge
|
|
1731
|
+
const knowledge = await agent.queryKnowledge({
|
|
1732
|
+
embedding: embed('how to navigate efficiently')
|
|
1733
|
+
}, 2);
|
|
1734
|
+
|
|
1735
|
+
console.log('\n📚 Relevant knowledge:');
|
|
1736
|
+
knowledge.forEach((k, i) => {
|
|
1737
|
+
console.log(`${i + 1}. ${k.concept}: ${k.description} (confidence: ${k.confidence})`);
|
|
1738
|
+
});
|
|
1739
|
+
|
|
1740
|
+
console.log('\n' + '='.repeat(60));
|
|
1741
|
+
console.log('PHASE 3: Reflection');
|
|
1742
|
+
console.log('='.repeat(60) + '\n');
|
|
1743
|
+
|
|
1744
|
+
// Reflect on learning
|
|
1745
|
+
const stats = await agent.reflect();
|
|
1746
|
+
const memoryStats = await agent.getStats();
|
|
1747
|
+
|
|
1748
|
+
console.log('\n📊 Memory Statistics:');
|
|
1749
|
+
console.log(` Episodic memories: ${memoryStats.episodicMemorySize}`);
|
|
1750
|
+
console.log(` Semantic knowledge: ${memoryStats.semanticMemorySize}`);
|
|
1751
|
+
console.log(` Agent ID: ${memoryStats.agentId}`);
|
|
1752
|
+
}
|
|
1753
|
+
|
|
1754
|
+
main().catch(console.error);
|
|
1755
|
+
```
|
|
1756
|
+
|
|
1757
|
+
**Expected Output:**
|
|
1758
|
+
```
|
|
1759
|
+
🧠 Memory system initialized for agent: agent-001
|
|
1760
|
+
|
|
1761
|
+
============================================================
|
|
1762
|
+
PHASE 1: Learning from experiences
|
|
1763
|
+
============================================================
|
|
1764
|
+
|
|
1765
|
+
💾 Stored experience: move_north -> reached room2 (reward: 0.5)
|
|
1766
|
+
💾 Stored experience: move_east -> reached room3 (reward: 1.0)
|
|
1767
|
+
💾 Stored experience: move_south -> hit wall (reward: -0.5)
|
|
1768
|
+
📚 Learned: navigation_strategy
|
|
1769
|
+
|
|
1770
|
+
============================================================
|
|
1771
|
+
PHASE 2: Applying memory
|
|
1772
|
+
============================================================
|
|
1773
|
+
|
|
1774
|
+
🔍 Recalling similar experiences...
|
|
1775
|
+
📝 Recalled 3 relevant experiences
|
|
1776
|
+
|
|
1777
|
+
📖 Recalled experiences:
|
|
1778
|
+
1. Action: move_east | Result: reached room3 | Reward: 1.0 | Similarity: 0.892
|
|
1779
|
+
2. Action: move_north | Result: reached room2 | Reward: 0.5 | Similarity: 0.876
|
|
1780
|
+
3. Action: move_south | Result: hit wall | Reward: -0.5 | Similarity: 0.654
|
|
1781
|
+
|
|
1782
|
+
📚 Relevant knowledge:
|
|
1783
|
+
1. navigation_strategy: Moving north then east is efficient for reaching room3 from room1 (confidence: 0.9)
|
|
1784
|
+
|
|
1785
|
+
============================================================
|
|
1786
|
+
PHASE 3: Reflection
|
|
1787
|
+
============================================================
|
|
1788
|
+
|
|
1789
|
+
🤔 Reflecting on experiences...
|
|
1790
|
+
📊 Total experiences: 3
|
|
1791
|
+
💡 Analysis complete
|
|
1792
|
+
|
|
1793
|
+
📊 Memory Statistics:
|
|
1794
|
+
Episodic memories: 3
|
|
1795
|
+
Semantic knowledge: 1
|
|
1796
|
+
Agent ID: agent-001
|
|
1797
|
+
```
|
|
1798
|
+
|
|
1799
|
+
**Use Cases:**
|
|
1800
|
+
- ✅ Reinforcement learning agents
|
|
1801
|
+
- ✅ Chatbot conversation history
|
|
1802
|
+
- ✅ Game AI that learns from gameplay
|
|
1803
|
+
- ✅ Personal assistant memory
|
|
1804
|
+
- ✅ Robotic navigation systems
|
|
1805
|
+
|
|
1806
|
+
## 🏗️ API Reference
|
|
1807
|
+
|
|
1808
|
+
### Constructor
|
|
1809
|
+
|
|
1810
|
+
```typescript
|
|
1811
|
+
new VectorDb(options: {
|
|
1812
|
+
dimensions: number; // Vector dimensionality (required)
|
|
1813
|
+
maxElements?: number; // Max vectors (default: 10000)
|
|
1814
|
+
storagePath?: string; // Persistent storage path
|
|
1815
|
+
ef_construction?: number; // HNSW construction parameter (default: 200)
|
|
1816
|
+
m?: number; // HNSW M parameter (default: 16)
|
|
1817
|
+
distanceMetric?: string; // 'cosine', 'euclidean', or 'dot' (default: 'cosine')
|
|
1818
|
+
})
|
|
1819
|
+
```
|
|
1820
|
+
|
|
1821
|
+
### Methods
|
|
1822
|
+
|
|
1823
|
+
#### insert(entry: VectorEntry): Promise<string>
|
|
1824
|
+
Insert a vector into the database.
|
|
1825
|
+
|
|
1826
|
+
```javascript
|
|
1827
|
+
const id = await db.insert({
|
|
1828
|
+
id: 'doc_1',
|
|
1829
|
+
vector: new Float32Array([0.1, 0.2, 0.3, ...]),
|
|
1830
|
+
metadata: { title: 'Document 1' }
|
|
1831
|
+
});
|
|
1832
|
+
```
|
|
1833
|
+
|
|
1834
|
+
#### search(query: SearchQuery): Promise<SearchResult[]>
|
|
1835
|
+
Search for similar vectors.
|
|
1836
|
+
|
|
1837
|
+
```javascript
|
|
1838
|
+
const results = await db.search({
|
|
1839
|
+
vector: new Float32Array([0.1, 0.2, 0.3, ...]),
|
|
1840
|
+
k: 10,
|
|
1841
|
+
threshold: 0.7
|
|
1842
|
+
});
|
|
1843
|
+
```
|
|
1844
|
+
|
|
1845
|
+
#### get(id: string): Promise<VectorEntry | null>
|
|
1846
|
+
Retrieve a vector by ID.
|
|
1847
|
+
|
|
1848
|
+
```javascript
|
|
1849
|
+
const entry = await db.get('doc_1');
|
|
1850
|
+
if (entry) {
|
|
1851
|
+
console.log(entry.vector, entry.metadata);
|
|
1852
|
+
}
|
|
1853
|
+
```
|
|
1854
|
+
|
|
1855
|
+
#### delete(id: string): Promise<boolean>
|
|
1856
|
+
Remove a vector from the database.
|
|
1857
|
+
|
|
1858
|
+
```javascript
|
|
1859
|
+
const deleted = await db.delete('doc_1');
|
|
1860
|
+
console.log(deleted ? 'Deleted' : 'Not found');
|
|
1861
|
+
```
|
|
1862
|
+
|
|
1863
|
+
#### len(): Promise<number>
|
|
1864
|
+
Get the total number of vectors.
|
|
1865
|
+
|
|
1866
|
+
```javascript
|
|
1867
|
+
const count = await db.len();
|
|
1868
|
+
console.log(`Total vectors: ${count}`);
|
|
1869
|
+
```
|
|
1870
|
+
|
|
1871
|
+
## 🎨 Advanced Configuration
|
|
1872
|
+
|
|
1873
|
+
### HNSW Parameters
|
|
1874
|
+
|
|
1875
|
+
```javascript
|
|
1876
|
+
const db = new VectorDb({
|
|
1877
|
+
dimensions: 384,
|
|
1878
|
+
maxElements: 1000000,
|
|
1879
|
+
ef_construction: 200, // Higher = better recall, slower build
|
|
1880
|
+
m: 16, // Higher = better recall, more memory
|
|
1881
|
+
storagePath: './large-db.db'
|
|
1882
|
+
});
|
|
1883
|
+
```
|
|
1884
|
+
|
|
1885
|
+
**Parameter Guidelines:**
|
|
1886
|
+
- `ef_construction`: 100-400 (higher = better recall, slower indexing)
|
|
1887
|
+
- `m`: 8-64 (higher = better recall, more memory)
|
|
1888
|
+
- Default values work well for most use cases
|
|
1889
|
+
|
|
1890
|
+
### Distance Metrics
|
|
1891
|
+
|
|
1892
|
+
```javascript
|
|
1893
|
+
// Cosine similarity (default, best for normalized vectors)
|
|
1894
|
+
const db1 = new VectorDb({
|
|
1895
|
+
dimensions: 128,
|
|
1896
|
+
distanceMetric: 'cosine'
|
|
1897
|
+
});
|
|
1898
|
+
|
|
1899
|
+
// Euclidean distance (L2, best for spatial data)
|
|
1900
|
+
const db2 = new VectorDb({
|
|
1901
|
+
dimensions: 128,
|
|
1902
|
+
distanceMetric: 'euclidean'
|
|
1903
|
+
});
|
|
1904
|
+
|
|
1905
|
+
// Dot product (best for pre-normalized vectors)
|
|
1906
|
+
const db3 = new VectorDb({
|
|
1907
|
+
dimensions: 128,
|
|
1908
|
+
distanceMetric: 'dot'
|
|
1909
|
+
});
|
|
1910
|
+
```
|
|
1911
|
+
|
|
1912
|
+
### Persistence
|
|
1913
|
+
|
|
1914
|
+
```javascript
|
|
1915
|
+
// Auto-save to disk
|
|
1916
|
+
const persistent = new VectorDb({
|
|
1917
|
+
dimensions: 128,
|
|
1918
|
+
storagePath: './persistent.db'
|
|
1919
|
+
});
|
|
1920
|
+
|
|
1921
|
+
// In-memory only (faster, but data lost on exit)
|
|
1922
|
+
const temporary = new VectorDb({
|
|
1923
|
+
dimensions: 128
|
|
1924
|
+
// No storagePath = in-memory
|
|
1925
|
+
});
|
|
1926
|
+
```
|
|
1927
|
+
|
|
1928
|
+
## 📦 Platform Support
|
|
1929
|
+
|
|
1930
|
+
Automatically installs the correct implementation for:
|
|
1931
|
+
|
|
1932
|
+
### Native (Rust) - Best Performance
|
|
1933
|
+
- **Linux**: x64, ARM64 (GNU libc)
|
|
1934
|
+
- **macOS**: x64 (Intel), ARM64 (Apple Silicon)
|
|
1935
|
+
- **Windows**: x64 (MSVC)
|
|
1936
|
+
|
|
1937
|
+
Performance: **<0.5ms latency**, **50K+ ops/sec**
|
|
1938
|
+
|
|
1939
|
+
### WASM Fallback - Universal Compatibility
|
|
1940
|
+
- Any platform where native module isn't available
|
|
1941
|
+
- Browser environments (experimental)
|
|
1942
|
+
- Alpine Linux (musl) and other non-glibc systems
|
|
1943
|
+
|
|
1944
|
+
Performance: **10-50ms latency**, **~1K ops/sec**
|
|
1945
|
+
|
|
1946
|
+
**Node.js 18+ required** for all platforms.
|
|
1947
|
+
|
|
1948
|
+
## 🔧 Building from Source
|
|
1949
|
+
|
|
1950
|
+
If you need to rebuild the native module:
|
|
1951
|
+
|
|
1952
|
+
```bash
|
|
1953
|
+
# Install Rust toolchain
|
|
1954
|
+
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
|
1955
|
+
|
|
1956
|
+
# Clone repository
|
|
1957
|
+
git clone https://github.com/ruvnet/ruvector.git
|
|
1958
|
+
cd ruvector
|
|
1959
|
+
|
|
1960
|
+
# Build native module
|
|
1961
|
+
cd npm/packages/core
|
|
1962
|
+
npm run build:napi
|
|
1963
|
+
|
|
1964
|
+
# Build wrapper package
|
|
1965
|
+
cd ../ruvector
|
|
1966
|
+
npm install
|
|
1967
|
+
npm run build
|
|
1968
|
+
|
|
1969
|
+
# Run tests
|
|
1970
|
+
npm test
|
|
1971
|
+
```
|
|
1972
|
+
|
|
1973
|
+
**Requirements:**
|
|
1974
|
+
- Rust 1.77+
|
|
1975
|
+
- Node.js 18+
|
|
1976
|
+
- Cargo
|
|
1977
|
+
|
|
1978
|
+
## 🌍 Ecosystem
|
|
1979
|
+
|
|
1980
|
+
### Related Packages
|
|
1981
|
+
|
|
1982
|
+
- **[ruvector-core](https://www.npmjs.com/package/ruvector-core)** - Core native bindings (lower-level API)
|
|
1983
|
+
- **[ruvector-wasm](https://www.npmjs.com/package/ruvector-wasm)** - WebAssembly implementation for browsers
|
|
1984
|
+
- **[ruvector-cli](https://www.npmjs.com/package/ruvector-cli)** - Standalone CLI tools
|
|
1985
|
+
- **[@ruvector/rvf](https://www.npmjs.com/package/@ruvector/rvf)** - RVF cognitive container SDK
|
|
1986
|
+
- **[@ruvector/rvf-wasm](https://www.npmjs.com/package/@ruvector/rvf-wasm)** - RVF WASM build for browsers, Deno, and edge
|
|
1987
|
+
- **[rvlite](https://www.npmjs.com/package/rvlite)** - Lightweight vector database with SQL, SPARQL, and Cypher
|
|
1988
|
+
|
|
1989
|
+
### Platform-Specific Packages (auto-installed)
|
|
1990
|
+
|
|
1991
|
+
- **[ruvector-core-linux-x64-gnu](https://www.npmjs.com/package/ruvector-core-linux-x64-gnu)**
|
|
1992
|
+
- **[ruvector-core-linux-arm64-gnu](https://www.npmjs.com/package/ruvector-core-linux-arm64-gnu)**
|
|
1993
|
+
- **[ruvector-core-darwin-x64](https://www.npmjs.com/package/ruvector-core-darwin-x64)**
|
|
1994
|
+
- **[ruvector-core-darwin-arm64](https://www.npmjs.com/package/ruvector-core-darwin-arm64)**
|
|
1995
|
+
- **[ruvector-core-win32-x64-msvc](https://www.npmjs.com/package/ruvector-core-win32-x64-msvc)**
|
|
1996
|
+
|
|
1997
|
+
---
|
|
1998
|
+
|
|
1999
|
+
## RVF Cognitive Containers
|
|
2000
|
+
|
|
2001
|
+
Ruvector integrates with [RVF (RuVector Format)](https://github.com/ruvnet/ruvector/tree/main/crates/rvf) — a universal binary substrate that stores vectors, models, graphs, compute kernels, and attestation in a single `.rvf` file.
|
|
2002
|
+
|
|
2003
|
+
### Enable RVF Backend
|
|
2004
|
+
|
|
2005
|
+
```bash
|
|
2006
|
+
# Install the optional RVF package
|
|
2007
|
+
npm install @ruvector/rvf
|
|
2008
|
+
|
|
2009
|
+
# Set backend via environment variable
|
|
2010
|
+
export RUVECTOR_BACKEND=rvf
|
|
2011
|
+
|
|
2012
|
+
# Or detect automatically (native -> rvf -> wasm fallback)
|
|
2013
|
+
npx ruvector info
|
|
2014
|
+
```
|
|
2015
|
+
|
|
2016
|
+
```typescript
|
|
2017
|
+
import { getImplementationType, isRvf } from 'ruvector';
|
|
2018
|
+
|
|
2019
|
+
console.log(getImplementationType()); // 'native' | 'rvf' | 'wasm'
|
|
2020
|
+
console.log(isRvf()); // true if RVF backend is active
|
|
2021
|
+
```
|
|
2022
|
+
|
|
2023
|
+
### RVF CLI Commands
|
|
2024
|
+
|
|
2025
|
+
8 RVF-specific subcommands are available through the ruvector CLI:
|
|
2026
|
+
|
|
2027
|
+
```bash
|
|
2028
|
+
# Create an RVF store
|
|
2029
|
+
npx ruvector rvf create mydb.rvf -d 384 --metric cosine
|
|
2030
|
+
|
|
2031
|
+
# Ingest vectors from JSON
|
|
2032
|
+
npx ruvector rvf ingest mydb.rvf --input vectors.json --format json
|
|
2033
|
+
|
|
2034
|
+
# Query nearest neighbors
|
|
2035
|
+
npx ruvector rvf query mydb.rvf --vector "[0.1,0.2,...]" --k 10
|
|
2036
|
+
|
|
2037
|
+
# File status and segment listing
|
|
2038
|
+
npx ruvector rvf status mydb.rvf
|
|
2039
|
+
npx ruvector rvf segments mydb.rvf
|
|
2040
|
+
|
|
2041
|
+
# COW branching — derive a child file
|
|
2042
|
+
npx ruvector rvf derive mydb.rvf --output child.rvf
|
|
2043
|
+
|
|
2044
|
+
# Compact and reclaim space
|
|
2045
|
+
npx ruvector rvf compact mydb.rvf
|
|
2046
|
+
|
|
2047
|
+
# Export to JSON
|
|
2048
|
+
npx ruvector rvf export mydb.rvf --output dump.json
|
|
2049
|
+
```
|
|
2050
|
+
|
|
2051
|
+
### RVF Platform Support
|
|
2052
|
+
|
|
2053
|
+
| Platform | Runtime | Backend |
|
|
2054
|
+
|----------|---------|---------|
|
|
2055
|
+
| Linux x86_64 / aarch64 | Node.js 18+ | Native (N-API) |
|
|
2056
|
+
| macOS x86_64 / arm64 | Node.js 18+ | Native (N-API) |
|
|
2057
|
+
| Windows x86_64 | Node.js 18+ | Native (N-API) |
|
|
2058
|
+
| Any | Deno | WASM (`@ruvector/rvf-wasm`) |
|
|
2059
|
+
| Any | Browser | WASM (`@ruvector/rvf-wasm`) |
|
|
2060
|
+
| Any | Cloudflare Workers | WASM (`@ruvector/rvf-wasm`) |
|
|
2061
|
+
|
|
2062
|
+
### Download Example .rvf Files
|
|
2063
|
+
|
|
2064
|
+
45 pre-built example files are available (~11 MB total):
|
|
2065
|
+
|
|
2066
|
+
```bash
|
|
2067
|
+
# Download a specific example
|
|
2068
|
+
curl -LO https://raw.githubusercontent.com/ruvnet/ruvector/main/examples/rvf/output/basic_store.rvf
|
|
2069
|
+
|
|
2070
|
+
# Popular examples:
|
|
2071
|
+
# basic_store.rvf (152 KB) — 1,000 vectors, dim 128
|
|
2072
|
+
# semantic_search.rvf (755 KB) — Semantic search with HNSW
|
|
2073
|
+
# rag_pipeline.rvf (303 KB) — RAG pipeline embeddings
|
|
2074
|
+
# agent_memory.rvf (32 KB) — AI agent memory store
|
|
2075
|
+
# self_booting.rvf (31 KB) — Self-booting with kernel
|
|
2076
|
+
# progressive_index.rvf (2.5 MB) — Large-scale HNSW index
|
|
2077
|
+
|
|
2078
|
+
# Generate all examples locally
|
|
2079
|
+
cd crates/rvf && cargo run --example generate_all
|
|
2080
|
+
```
|
|
2081
|
+
|
|
2082
|
+
Full catalog: [examples/rvf/output/](https://github.com/ruvnet/ruvector/tree/main/examples/rvf/output)
|
|
2083
|
+
|
|
2084
|
+
### Working Examples: Cognitive Containers
|
|
2085
|
+
|
|
2086
|
+
#### Self-Booting Microservice
|
|
2087
|
+
|
|
2088
|
+
A single `.rvf` file that contains vectors AND a bootable Linux kernel:
|
|
2089
|
+
|
|
2090
|
+
```bash
|
|
2091
|
+
# Build and run the self-booting example
|
|
2092
|
+
cd crates/rvf && cargo run --example self_booting
|
|
2093
|
+
# Output:
|
|
2094
|
+
# Ingested 50 vectors (128 dims)
|
|
2095
|
+
# Pre-kernel query: top-5 results OK (nearest ID=25)
|
|
2096
|
+
# Kernel: 4,640 bytes embedded (x86_64, Hermit)
|
|
2097
|
+
# Witness chain: 5 entries, all verified
|
|
2098
|
+
# File: bootable.rvf (31 KB) — data + runtime in one file
|
|
2099
|
+
```
|
|
2100
|
+
|
|
2101
|
+
```rust
|
|
2102
|
+
// The pattern: vectors + kernel + witness in one file
|
|
2103
|
+
let mut store = RvfStore::create("bootable.rvf", options)?;
|
|
2104
|
+
store.ingest_batch(&vectors, &ids, None)?;
|
|
2105
|
+
store.embed_kernel(KernelArch::X86_64 as u8, KernelType::Hermit as u8,
|
|
2106
|
+
0x0018, &kernel_image, 8080, Some("console=ttyS0 quiet"))?;
|
|
2107
|
+
// Result: drop on a VM and it boots as a query service
|
|
2108
|
+
```
|
|
2109
|
+
|
|
2110
|
+
#### Linux Microkernel Distribution
|
|
2111
|
+
|
|
2112
|
+
20-package Linux distro with SSH keys and kernel in a single file:
|
|
2113
|
+
|
|
2114
|
+
```bash
|
|
2115
|
+
cd crates/rvf && cargo run --example linux_microkernel
|
|
2116
|
+
# Output:
|
|
2117
|
+
# Installed 20 packages as vector embeddings
|
|
2118
|
+
# Kernel embedded: Linux x86_64 (4,640 bytes)
|
|
2119
|
+
# SSH keys: Ed25519, signed and verified
|
|
2120
|
+
# Witness chain: 22 entries (1 per package + kernel + SSH)
|
|
2121
|
+
# File: microkernel.rvf (14 KB) — immutable bootable system
|
|
2122
|
+
```
|
|
2123
|
+
|
|
2124
|
+
Features: package search by embedding similarity, Ed25519 signed SSH keys, witness-audited installs, COW-derived child images for atomic updates.
|
|
2125
|
+
|
|
2126
|
+
#### Claude Code AI Appliance
|
|
2127
|
+
|
|
2128
|
+
A sealed, bootable AI development environment:
|
|
2129
|
+
|
|
2130
|
+
```bash
|
|
2131
|
+
cd crates/rvf && cargo run --example claude_code_appliance
|
|
2132
|
+
# Output:
|
|
2133
|
+
# 20 dev packages (rust, node, python, docker, ...)
|
|
2134
|
+
# Kernel: Linux x86_64 with SSH on port 2222
|
|
2135
|
+
# eBPF: XDP distance program for fast-path lookups
|
|
2136
|
+
# Witness chain: 6 entries, all verified
|
|
2137
|
+
# Crypto: Ed25519 signature
|
|
2138
|
+
# File: claude_code_appliance.rvf (17 KB)
|
|
2139
|
+
```
|
|
2140
|
+
|
|
2141
|
+
#### CLI Full Lifecycle
|
|
2142
|
+
|
|
2143
|
+
```bash
|
|
2144
|
+
# Create → Ingest → Query → Derive → Inspect
|
|
2145
|
+
rvf create vectors.rvf --dimension 384
|
|
2146
|
+
rvf ingest vectors.rvf --input data.json --format json
|
|
2147
|
+
rvf query vectors.rvf --vector "0.1,0.2,..." --k 10
|
|
2148
|
+
rvf derive vectors.rvf child.rvf --type filter
|
|
2149
|
+
rvf inspect vectors.rvf
|
|
2150
|
+
|
|
2151
|
+
# Embed kernel and launch as microVM
|
|
2152
|
+
rvf embed-kernel vectors.rvf --image bzImage
|
|
2153
|
+
rvf launch vectors.rvf --port 8080
|
|
2154
|
+
|
|
2155
|
+
# Verify tamper-evident witness chain
|
|
2156
|
+
rvf verify-witness vectors.rvf
|
|
2157
|
+
rvf verify-attestation vectors.rvf
|
|
2158
|
+
```
|
|
2159
|
+
|
|
2160
|
+
#### Integration Tests (46 passing)
|
|
2161
|
+
|
|
2162
|
+
```bash
|
|
2163
|
+
cd crates/rvf
|
|
2164
|
+
cargo test --workspace
|
|
2165
|
+
# attestation .............. 6 passed
|
|
2166
|
+
# crypto ................... 10 passed
|
|
2167
|
+
# computational_container .. 8 passed
|
|
2168
|
+
# cow_branching ............ 8 passed
|
|
2169
|
+
# cross_platform ........... 6 passed
|
|
2170
|
+
# lineage .................. 4 passed
|
|
2171
|
+
# smoke .................... 4 passed
|
|
2172
|
+
# Total: 46/46 passed
|
|
2173
|
+
```
|
|
2174
|
+
|
|
2175
|
+
## 🐛 Troubleshooting
|
|
2176
|
+
|
|
2177
|
+
### Native Module Not Loading
|
|
2178
|
+
|
|
2179
|
+
If you see "Cannot find module 'ruvector-core-*'":
|
|
2180
|
+
|
|
2181
|
+
```bash
|
|
2182
|
+
# Reinstall with optional dependencies
|
|
2183
|
+
npm install --include=optional ruvector
|
|
2184
|
+
|
|
2185
|
+
# Verify platform
|
|
2186
|
+
npx ruvector info
|
|
2187
|
+
|
|
2188
|
+
# Check Node.js version (18+ required)
|
|
2189
|
+
node --version
|
|
2190
|
+
```
|
|
2191
|
+
|
|
2192
|
+
### WASM Fallback Performance
|
|
2193
|
+
|
|
2194
|
+
If you're using WASM fallback and need better performance:
|
|
2195
|
+
|
|
2196
|
+
1. **Install native toolchain** for your platform
|
|
2197
|
+
2. **Rebuild native module**: `npm rebuild ruvector`
|
|
2198
|
+
3. **Verify native**: `npx ruvector info` should show "native (Rust)"
|
|
2199
|
+
|
|
2200
|
+
### Platform Compatibility
|
|
2201
|
+
|
|
2202
|
+
- **Alpine Linux**: Uses WASM fallback (musl not supported)
|
|
2203
|
+
- **Windows ARM**: Not yet supported, uses WASM fallback
|
|
2204
|
+
- **Node.js < 18**: Not supported, upgrade to Node.js 18+
|
|
2205
|
+
|
|
2206
|
+
## 📚 Documentation
|
|
2207
|
+
|
|
2208
|
+
- 🏠 [Homepage](https://ruv.io)
|
|
2209
|
+
- 📦 [GitHub Repository](https://github.com/ruvnet/ruvector)
|
|
2210
|
+
- 📚 [Full Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)
|
|
2211
|
+
- 🚀 [Getting Started Guide](https://github.com/ruvnet/ruvector/blob/main/docs/guide/GETTING_STARTED.md)
|
|
2212
|
+
- 📖 [API Reference](https://github.com/ruvnet/ruvector/blob/main/docs/api/NODEJS_API.md)
|
|
2213
|
+
- 🎯 [Performance Tuning](https://github.com/ruvnet/ruvector/blob/main/docs/optimization/PERFORMANCE_TUNING_GUIDE.md)
|
|
2214
|
+
- 🐛 [Issue Tracker](https://github.com/ruvnet/ruvector/issues)
|
|
2215
|
+
- 💬 [Discussions](https://github.com/ruvnet/ruvector/discussions)
|
|
2216
|
+
|
|
2217
|
+
## 🤝 Contributing
|
|
2218
|
+
|
|
2219
|
+
We welcome contributions! See [CONTRIBUTING.md](https://github.com/ruvnet/ruvector/blob/main/docs/development/CONTRIBUTING.md) for guidelines.
|
|
2220
|
+
|
|
2221
|
+
### Quick Start
|
|
2222
|
+
|
|
2223
|
+
1. Fork the repository
|
|
2224
|
+
2. Create a feature branch: `git checkout -b feature/amazing-feature`
|
|
2225
|
+
3. Commit changes: `git commit -m 'Add amazing feature'`
|
|
2226
|
+
4. Push to branch: `git push origin feature/amazing-feature`
|
|
2227
|
+
5. Open a Pull Request
|
|
2228
|
+
|
|
2229
|
+
## 🌐 Community & Support
|
|
2230
|
+
|
|
2231
|
+
- **GitHub**: [github.com/ruvnet/ruvector](https://github.com/ruvnet/ruvector) - ⭐ Star and follow
|
|
2232
|
+
- **Discord**: [Join our community](https://discord.gg/ruvnet) - Chat with developers
|
|
2233
|
+
- **Twitter**: [@ruvnet](https://twitter.com/ruvnet) - Follow for updates
|
|
2234
|
+
- **Issues**: [Report bugs](https://github.com/ruvnet/ruvector/issues)
|
|
2235
|
+
|
|
2236
|
+
### Enterprise Support
|
|
2237
|
+
|
|
2238
|
+
Need custom development or consulting?
|
|
2239
|
+
|
|
2240
|
+
📧 [enterprise@ruv.io](mailto:enterprise@ruv.io)
|
|
2241
|
+
|
|
2242
|
+
## 📜 License
|
|
2243
|
+
|
|
2244
|
+
**MIT License** - see [LICENSE](https://github.com/ruvnet/ruvector/blob/main/LICENSE) for details.
|
|
2245
|
+
|
|
2246
|
+
Free for commercial and personal use.
|
|
2247
|
+
|
|
2248
|
+
## 🙏 Acknowledgments
|
|
2249
|
+
|
|
2250
|
+
Built with battle-tested technologies:
|
|
2251
|
+
|
|
2252
|
+
- **HNSW**: Hierarchical Navigable Small World graphs
|
|
2253
|
+
- **SIMD**: Hardware-accelerated vector operations via simsimd
|
|
2254
|
+
- **Rust**: Memory-safe, zero-cost abstractions
|
|
2255
|
+
- **NAPI-RS**: High-performance Node.js bindings
|
|
2256
|
+
- **WebAssembly**: Universal browser compatibility
|
|
2257
|
+
|
|
2258
|
+
---
|
|
2259
|
+
|
|
2260
|
+
<div align="center">
|
|
2261
|
+
|
|
2262
|
+
**Built with ❤️ by [rUv](https://ruv.io)**
|
|
2263
|
+
|
|
2264
|
+
[](https://www.npmjs.com/package/ruvector)
|
|
2265
|
+
[](https://github.com/ruvnet/ruvector)
|
|
2266
|
+
[](https://twitter.com/ruvnet)
|
|
2267
|
+
|
|
2268
|
+
**[Get Started](https://github.com/ruvnet/ruvector/blob/main/docs/guide/GETTING_STARTED.md)** • **[Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)** • **[API Reference](https://github.com/ruvnet/ruvector/blob/main/docs/api/NODEJS_API.md)** • **[Contributing](https://github.com/ruvnet/ruvector/blob/main/docs/development/CONTRIBUTING.md)**
|
|
2269
|
+
|
|
2270
|
+
</div>
|