@soulcraft/brainy 0.50.0 → 0.51.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +151 -1084
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -11,160 +11,39 @@
|
|
|
11
11
|
|
|
12
12
|
</div>
|
|
13
13
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
### 🎯 **v0.49: Filter Discovery & Performance Improvements**
|
|
17
|
-
|
|
18
|
-
**Discover available filters and scale to millions of items!**
|
|
19
|
-
|
|
20
|
-
```javascript
|
|
21
|
-
// Discover what filters are available - O(1) field lookup
|
|
22
|
-
const categories = await brainy.getFilterValues('category')
|
|
23
|
-
// Returns: ['electronics', 'books', 'clothing', ...]
|
|
14
|
+
---
|
|
24
15
|
|
|
25
|
-
|
|
26
|
-
// Returns: ['category', 'price', 'brand', 'rating', ...]
|
|
27
|
-
```
|
|
16
|
+
# The Search Problem Every Developer Faces
|
|
28
17
|
|
|
29
|
-
|
|
30
|
-
- ✅ **Improved Performance**: Removed deprecated methods, now uses pagination everywhere
|
|
31
|
-
- ✅ **Better Scalability**: Hybrid indexing with O(1) field access scales to millions
|
|
32
|
-
- ✅ **Smart Caching**: LRU cache for frequently accessed filters
|
|
33
|
-
- ✅ **Zero Configuration**: Everything auto-optimizes based on usage patterns
|
|
18
|
+
**"I need to find similar content, explore relationships, AND filter by metadata - but that means juggling 3+ databases"**
|
|
34
19
|
|
|
35
|
-
|
|
20
|
+
❌ **Current Reality**: Pinecone + Neo4j + Elasticsearch + Custom Sync Logic
|
|
21
|
+
✅ **Brainy Reality**: One database. One API. All three search types.
|
|
36
22
|
|
|
37
|
-
|
|
23
|
+
## 🔥 The Power of Three-in-One Search
|
|
38
24
|
|
|
39
25
|
```javascript
|
|
40
|
-
|
|
26
|
+
// This ONE query does what used to require 3 databases:
|
|
27
|
+
const results = await brainy.search("AI startups in healthcare", 10, {
|
|
28
|
+
// 🔍 Vector: Semantic similarity
|
|
29
|
+
includeVerbs: true,
|
|
30
|
+
|
|
31
|
+
// 🔗 Graph: Relationship traversal
|
|
32
|
+
verbTypes: ["invests_in", "partners_with"],
|
|
33
|
+
|
|
34
|
+
// 📊 Faceted: MongoDB-style filtering
|
|
41
35
|
metadata: {
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
brand: { $ne: "Generic" }
|
|
36
|
+
industry: "healthcare",
|
|
37
|
+
funding: { $gte: 1000000 },
|
|
38
|
+
stage: { $in: ["Series A", "Series B"] }
|
|
46
39
|
}
|
|
47
40
|
})
|
|
41
|
+
// Returns: Companies similar to your query + their relationships + matching your criteria
|
|
48
42
|
```
|
|
49
43
|
|
|
50
|
-
|
|
51
|
-
- ✅ **Automatic Indexing**: Zero configuration, maximum performance
|
|
52
|
-
- ✅ **Nested Fields**: Use dot notation for complex objects
|
|
53
|
-
- ✅ **100% Backward Compatible**: Your existing code works unchanged
|
|
54
|
-
|
|
55
|
-
### ⚡ **v0.46: Transformers.js Migration**
|
|
56
|
-
|
|
57
|
-
**Replaced TensorFlow.js for better performance and true offline operation!**
|
|
58
|
-
|
|
59
|
-
- ✅ **95% Smaller Package**: 643 kB vs 12.5 MB
|
|
60
|
-
- ✅ **84% Smaller Models**: 87 MB vs 525 MB models
|
|
61
|
-
- ✅ **True Offline**: Zero network calls after initial download
|
|
62
|
-
- ✅ **5x Fewer Dependencies**: Clean tree, no peer dependency issues
|
|
63
|
-
- ✅ **Same API**: Drop-in replacement, existing code works unchanged
|
|
64
|
-
|
|
65
|
-
### Migration (It's Automatic!)
|
|
66
|
-
|
|
67
|
-
```javascript
|
|
68
|
-
// Your existing code works unchanged!
|
|
69
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
70
|
-
|
|
71
|
-
const db = new BrainyData({
|
|
72
|
-
embedding: { type: 'transformer' } // Now uses Transformers.js automatically
|
|
73
|
-
})
|
|
74
|
-
|
|
75
|
-
// Dimensions changed from 512 → 384 (handled automatically)
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
**For Docker/Production or No Egress:**
|
|
79
|
-
|
|
80
|
-
```dockerfile
|
|
81
|
-
RUN npm install @soulcraft/brainy
|
|
82
|
-
RUN npm run download-models # Download during build for offline production
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
---
|
|
86
|
-
|
|
87
|
-
## 🏆 Industry First: True Vector + Graph Database
|
|
88
|
-
|
|
89
|
-
**Brainy is the only database that natively combines vector search and graph relationships in a single, unified system.**
|
|
90
|
-
|
|
91
|
-
Unlike other solutions that bolt vector search onto traditional databases or require multiple systems:
|
|
92
|
-
|
|
93
|
-
✅ **Native Vector + Graph Architecture** - Purpose-built for both semantic search AND knowledge graphs
|
|
94
|
-
✅ **Single API, Dual Power** - Vector similarity search AND graph traversal in one database
|
|
95
|
-
✅ **True Semantic Relationships** - Not just "similar vectors" but meaningful connections like "develops", "owns", "causes"
|
|
96
|
-
✅ **Zero Integration Complexity** - No need to sync between Pinecone + Neo4j or pgvector + graph databases
|
|
97
|
-
|
|
98
|
-
**Why This Matters:**
|
|
99
|
-
```javascript
|
|
100
|
-
// Other solutions: Manage 2+ databases
|
|
101
|
-
const vectors = await pinecone.search(query) // Vector search
|
|
102
|
-
const graph = await neo4j.run("MATCH (a)-[r]->(b)") // Graph traversal
|
|
103
|
-
// How do you keep them in sync? 😢
|
|
104
|
-
|
|
105
|
-
// Brainy: One database, both capabilities
|
|
106
|
-
const results = await brainy.search("AI models", 10, {
|
|
107
|
-
includeVerbs: true, // Include relationships
|
|
108
|
-
verbTypes: ["develops"] // Filter by relationship type
|
|
109
|
-
})
|
|
110
|
-
// Everything stays perfectly synchronized! 🎉
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
This revolutionary architecture enables entirely new classes of AI applications that were previously impossible or prohibitively complex.
|
|
114
|
-
|
|
115
|
-
## ✨ What is Brainy?
|
|
116
|
-
|
|
117
|
-
**One API. Every environment. Zero configuration.**
|
|
118
|
-
|
|
119
|
-
Brainy is the **AI-native database** that combines vector search and knowledge graphs in one unified API. Write your
|
|
120
|
-
code once, and it runs everywhere - browsers, Node.js, serverless, edge workers - with automatic optimization for each
|
|
121
|
-
environment.
|
|
122
|
-
|
|
123
|
-
```javascript
|
|
124
|
-
// This same code works EVERYWHERE
|
|
125
|
-
const brainy = new BrainyData()
|
|
126
|
-
await brainy.init()
|
|
127
|
-
|
|
128
|
-
// Vector search (like Pinecone) + Graph database (like Neo4j)
|
|
129
|
-
await brainy.add("OpenAI", { type: "company" }) // Nouns
|
|
130
|
-
await brainy.relate(openai, gpt4, "develops") // Verbs
|
|
131
|
-
const results = await brainy.search("AI", 10) // Semantic search
|
|
132
|
-
```
|
|
133
|
-
|
|
134
|
-
### 🆕 NEW: Distributed Mode (v0.38+)
|
|
135
|
-
|
|
136
|
-
**Scale horizontally with zero configuration!** Brainy now supports distributed deployments with automatic coordination:
|
|
137
|
-
|
|
138
|
-
- **🌐 Multi-Instance Coordination** - Multiple readers and writers working in harmony
|
|
139
|
-
- **🏷️ Smart Domain Detection** - Automatically categorizes data (medical, legal, product, etc.)
|
|
140
|
-
- **📊 Real-Time Health Monitoring** - Track performance across all instances
|
|
141
|
-
- **🔄 Automatic Role Optimization** - Readers optimize for cache, writers for throughput
|
|
142
|
-
- **🗂️ Intelligent Partitioning** - Hash-based partitioning for perfect load distribution
|
|
143
|
-
|
|
144
|
-
### 🚀 Why Developers Love Brainy
|
|
44
|
+
**Three search paradigms. One lightning-fast query. Zero complexity.**
|
|
145
45
|
|
|
146
|
-
|
|
147
|
-
environment and optimizes itself
|
|
148
|
-
- **🌍 True Write-Once, Run-Anywhere** - Same code runs in Angular, React, Vue, Node.js, Deno, Bun, serverless, edge
|
|
149
|
-
workers, and web workers with automatic environment detection
|
|
150
|
-
- **⚡ Scary Fast** - Handles millions of vectors with sub-millisecond search. GPU acceleration for embeddings, optimized
|
|
151
|
-
CPU for distance calculations
|
|
152
|
-
- **🎯 Self-Learning** - Like having a database that goes to the gym. Gets faster and smarter the more you use it
|
|
153
|
-
- **🔮 AI-First Design** - Built for the age of embeddings, RAG, and semantic search. Your LLMs will thank you
|
|
154
|
-
- **🎮 Actually Fun to Use** - Clean API, great DX, and it does the heavy lifting so you can build cool stuff
|
|
155
|
-
|
|
156
|
-
### 🚀 NEW: Ultra-Fast Search Performance + Auto-Configuration
|
|
157
|
-
|
|
158
|
-
**Your searches just got 100x faster AND Brainy now configures itself!** Advanced performance with zero setup:
|
|
159
|
-
|
|
160
|
-
- **🤖 Intelligent Auto-Configuration** - Detects environment and usage patterns, optimizes automatically
|
|
161
|
-
- **⚡ Smart Result Caching** - Repeated queries return in <1ms with automatic cache invalidation
|
|
162
|
-
- **📄 Cursor-Based Pagination** - Navigate millions of results with constant O(k) performance
|
|
163
|
-
- **🔄 Real-Time Data Sync** - Cache automatically updates when data changes, even in distributed scenarios
|
|
164
|
-
- **📊 Performance Monitoring** - Built-in hit rate and memory usage tracking with adaptive optimization
|
|
165
|
-
- **🎯 Zero Breaking Changes** - All existing code works unchanged, just faster and smarter
|
|
166
|
-
|
|
167
|
-
## 📦 Get Started in 30 Seconds
|
|
46
|
+
## 🚀 Install & Go
|
|
168
47
|
|
|
169
48
|
```bash
|
|
170
49
|
npm install @soulcraft/brainy
|
|
@@ -173,1028 +52,214 @@ npm install @soulcraft/brainy
|
|
|
173
52
|
```javascript
|
|
174
53
|
import { BrainyData } from '@soulcraft/brainy'
|
|
175
54
|
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
await brainy.init() // Auto-detects your environment
|
|
179
|
-
|
|
180
|
-
// 1️⃣ Simple vector search (like Pinecone)
|
|
181
|
-
await brainy.add("The quick brown fox jumps over the lazy dog", { type: "sentence" })
|
|
182
|
-
await brainy.add("Cats are independent and mysterious animals", { type: "sentence" })
|
|
55
|
+
const brainy = new BrainyData() // Auto-detects your environment
|
|
56
|
+
await brainy.init() // Auto-configures everything
|
|
183
57
|
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
// 2️⃣ Graph relationships (like Neo4j)
|
|
188
|
-
const openai = await brainy.add("OpenAI", { type: "company", founded: 2015 })
|
|
189
|
-
const gpt4 = await brainy.add("GPT-4", { type: "product", released: 2023 })
|
|
190
|
-
const sam = await brainy.add("Sam Altman", { type: "person", role: "CEO" })
|
|
191
|
-
|
|
192
|
-
// Create relationships between entities
|
|
58
|
+
// Add data with relationships
|
|
59
|
+
const openai = await brainy.add("OpenAI", { type: "company", funding: 11000000 })
|
|
60
|
+
const gpt4 = await brainy.add("GPT-4", { type: "product", users: 100000000 })
|
|
193
61
|
await brainy.relate(openai, gpt4, "develops")
|
|
194
|
-
await brainy.relate(sam, openai, "leads")
|
|
195
|
-
await brainy.relate(gpt4, sam, "created_by")
|
|
196
|
-
|
|
197
|
-
// 3️⃣ Combined power: Vector search + Graph traversal
|
|
198
|
-
const similar = await brainy.search("AI language models", 10) // Find by meaning
|
|
199
|
-
const products = await brainy.getVerbsBySource(openai) // Get relationships
|
|
200
|
-
const graph = await brainy.findSimilar(gpt4, { relationType: "develops" })
|
|
201
|
-
|
|
202
|
-
// 4️⃣ Advanced: Search with context
|
|
203
|
-
const contextual = await brainy.search("Who leads AI companies?", 5, {
|
|
204
|
-
includeVerbs: true, // Include relationships in results
|
|
205
|
-
nounTypes: ["person"], // Filter to specific entity types
|
|
206
|
-
})
|
|
207
62
|
|
|
208
|
-
//
|
|
209
|
-
const
|
|
210
|
-
metadata: {
|
|
211
|
-
|
|
212
|
-
year: { $gte: 2020 },
|
|
213
|
-
status: { $in: ["published", "peer-reviewed"] },
|
|
214
|
-
impact: { $gt: 100 }
|
|
215
|
-
}
|
|
63
|
+
// Search across all dimensions
|
|
64
|
+
const results = await brainy.search("AI language models", 5, {
|
|
65
|
+
metadata: { funding: { $gte: 10000000 } },
|
|
66
|
+
includeVerbs: true
|
|
216
67
|
})
|
|
217
|
-
// Filters DURING search for maximum performance!
|
|
218
68
|
```
|
|
219
69
|
|
|
220
|
-
|
|
70
|
+
**That's it. You just built a knowledge graph with semantic search and faceted filtering in 8 lines.**
|
|
221
71
|
|
|
222
|
-
|
|
72
|
+
## 🏆 Why Brainy Wins
|
|
223
73
|
|
|
224
|
-
- **
|
|
225
|
-
- **
|
|
226
|
-
- **
|
|
227
|
-
- **
|
|
74
|
+
- 🧠 **Triple Search Power** - Vector + Graph + Faceted filtering in one query
|
|
75
|
+
- 🌍 **Runs Everywhere** - Same code: React, Node.js, serverless, edge
|
|
76
|
+
- ⚡ **Zero Config** - Auto-detects environment, optimizes itself
|
|
77
|
+
- 🔄 **Always Synced** - No data consistency nightmares between systems
|
|
78
|
+
- 📦 **Truly Offline** - Works without internet after initial setup
|
|
79
|
+
- 🔒 **Your Data** - Run locally, in browser, or your own cloud
|
|
228
80
|
|
|
229
|
-
|
|
81
|
+
## 🔮 Coming Soon
|
|
230
82
|
|
|
231
|
-
|
|
83
|
+
- **🤖 MCP Integration** - Let Claude, GPT, and other AI models query your data directly
|
|
84
|
+
- **⚡ LLM Generation** - Built-in content generation powered by your knowledge graph
|
|
85
|
+
- **🌊 Real-time Sync** - Live updates across distributed instances
|
|
232
86
|
|
|
233
|
-
|
|
234
|
-
**🔍 Semantic Search Engines** - Search by meaning, not keywords. Find "that thing that's like a cat but bigger" →
|
|
235
|
-
returns "tiger"
|
|
236
|
-
**🎯 Recommendation Engines** - "Users who liked this also liked..." but actually good
|
|
237
|
-
**🧬 Knowledge Graphs** - Connect everything to everything. Wikipedia meets Neo4j meets magic
|
|
238
|
-
**👁️ Computer Vision Apps** - Store and search image embeddings. "Find all photos with dogs wearing hats"
|
|
239
|
-
**🎵 Music Discovery** - Find songs that "feel" similar. Spotify's Discover Weekly in your app
|
|
240
|
-
**📚 Smart Documentation** - Docs that answer questions. "How do I deploy to production?" → relevant guides
|
|
241
|
-
**🛡️ Fraud Detection** - Find patterns humans can't see. Anomaly detection on steroids
|
|
242
|
-
**🌐 Real-Time Collaboration** - Sync vector data across devices. Figma for AI data
|
|
243
|
-
**🏥 Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity
|
|
87
|
+
## 🎯 Perfect For
|
|
244
88
|
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
89
|
+
**🤖 AI Chat Applications** - ChatGPT-like apps with long-term memory and context
|
|
90
|
+
**🔍 Semantic Search** - Find "that thing like a cat but bigger" → returns "tiger"
|
|
91
|
+
**🧬 Knowledge Graphs** - Connect everything. Wikipedia meets Neo4j meets magic
|
|
92
|
+
**🎯 Recommendation Engines** - "Users who liked this also liked..." but actually good
|
|
93
|
+
**📚 Smart Documentation** - Docs that answer questions before you ask them
|
|
248
94
|
|
|
249
|
-
|
|
95
|
+
## 🌍 Works Everywhere - Same Code
|
|
250
96
|
|
|
251
97
|
```javascript
|
|
98
|
+
// This EXACT code works in ALL environments
|
|
252
99
|
import { BrainyData } from '@soulcraft/brainy'
|
|
253
100
|
|
|
254
|
-
// SAME CODE in React, Angular, Vue, Svelte, etc.
|
|
255
101
|
const brainy = new BrainyData()
|
|
256
|
-
await brainy.init()
|
|
257
|
-
|
|
258
|
-
// Add entities and relationships
|
|
259
|
-
const john = await brainy.add("John is a software engineer", { type: "person" })
|
|
260
|
-
const jane = await brainy.add("Jane is a data scientist", { type: "person" })
|
|
261
|
-
const ai = await brainy.add("AI Project", { type: "project" })
|
|
262
|
-
|
|
263
|
-
await brainy.relate(john, ai, "works_on")
|
|
264
|
-
await brainy.relate(jane, ai, "leads")
|
|
265
|
-
|
|
266
|
-
// Search by meaning
|
|
267
|
-
const engineers = await brainy.search("software developers", 5)
|
|
102
|
+
await brainy.init()
|
|
268
103
|
|
|
269
|
-
//
|
|
270
|
-
|
|
104
|
+
// Works in: React, Vue, Angular, Node.js, Deno, Bun,
|
|
105
|
+
// Cloudflare Workers, Vercel Edge, AWS Lambda, browsers, anywhere
|
|
271
106
|
```
|
|
272
107
|
|
|
273
|
-
|
|
274
|
-
<summary>📦 Full Angular Component Example</summary>
|
|
275
|
-
|
|
276
|
-
```typescript
|
|
277
|
-
import { Component, signal, OnInit } from '@angular/core'
|
|
278
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
279
|
-
|
|
280
|
-
@Component({
|
|
281
|
-
selector: 'app-search',
|
|
282
|
-
template: `<input (input)="search($event.target.value)" placeholder="Search...">`
|
|
283
|
-
})
|
|
284
|
-
export class SearchComponent implements OnInit {
|
|
285
|
-
brainy = new BrainyData()
|
|
108
|
+
Brainy automatically detects and optimizes for your environment:
|
|
286
109
|
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
110
|
+
| Environment | Storage | Optimization |
|
|
111
|
+
|-------------|---------|-------------|
|
|
112
|
+
| 🌐 Browser | OPFS | Web Workers, Memory Cache |
|
|
113
|
+
| 🟢 Node.js | FileSystem / S3 | Worker Threads, Clustering |
|
|
114
|
+
| ⚡ Serverless | S3 / Memory | Cold Start Optimization |
|
|
115
|
+
| 🔥 Edge | Memory / KV | Minimal Footprint |
|
|
291
116
|
|
|
292
|
-
|
|
293
|
-
const results = await this.brainy.search(query, 5)
|
|
294
|
-
// Display results...
|
|
295
|
-
}
|
|
296
|
-
}
|
|
297
|
-
```
|
|
117
|
+
## 🆚 Why Not Just Use...?
|
|
298
118
|
|
|
299
|
-
|
|
119
|
+
### vs. Multiple Databases
|
|
120
|
+
❌ **Pinecone + Neo4j + Elasticsearch** - 3 databases, sync nightmares, 3x the cost
|
|
121
|
+
✅ **Brainy** - One database, always synced, built-in intelligence
|
|
300
122
|
|
|
301
|
-
|
|
302
|
-
|
|
123
|
+
### vs. Traditional Solutions
|
|
124
|
+
❌ **PostgreSQL + pgvector + extensions** - Complex setup, performance issues
|
|
125
|
+
✅ **Brainy** - Zero config, purpose-built for AI, works everywhere
|
|
303
126
|
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
function Search() {
|
|
309
|
-
const [brainy, setBrainy] = useState(null)
|
|
310
|
-
const [results, setResults] = useState([])
|
|
311
|
-
|
|
312
|
-
useEffect(() => {
|
|
313
|
-
const init = async () => {
|
|
314
|
-
const db = new BrainyData()
|
|
315
|
-
await db.init()
|
|
316
|
-
// Add your data...
|
|
317
|
-
setBrainy(db)
|
|
318
|
-
}
|
|
319
|
-
init()
|
|
320
|
-
}, [])
|
|
321
|
-
|
|
322
|
-
const search = async (query) => {
|
|
323
|
-
const results = await brainy?.search(query, 5) || []
|
|
324
|
-
setResults(results)
|
|
325
|
-
}
|
|
127
|
+
### vs. Cloud-Only Vector DBs
|
|
128
|
+
❌ **Pinecone/Weaviate/Qdrant** - Vendor lock-in, expensive, cloud-only
|
|
129
|
+
✅ **Brainy** - Run anywhere, your data stays yours, cost-effective
|
|
326
130
|
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
131
|
+
### vs. Graph Databases with "Vector Features"
|
|
132
|
+
❌ **Neo4j + vector plugin** - Bolt-on solution, not native, limited
|
|
133
|
+
✅ **Brainy** - Native vector+graph architecture from the ground up
|
|
330
134
|
|
|
331
|
-
|
|
135
|
+
## 📦 Advanced Features
|
|
332
136
|
|
|
333
137
|
<details>
|
|
334
|
-
<summary
|
|
335
|
-
|
|
336
|
-
```vue
|
|
337
|
-
|
|
338
|
-
<script setup>
|
|
339
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
340
|
-
import { ref, onMounted } from 'vue'
|
|
341
|
-
|
|
342
|
-
const brainy = ref(null)
|
|
343
|
-
const results = ref([])
|
|
344
|
-
|
|
345
|
-
onMounted(async () => {
|
|
346
|
-
const db = new BrainyData()
|
|
347
|
-
await db.init()
|
|
348
|
-
// Add your data...
|
|
349
|
-
brainy.value = db
|
|
350
|
-
})
|
|
351
|
-
|
|
352
|
-
const search = async (query) => {
|
|
353
|
-
const results = await brainy.value?.search(query, 5) || []
|
|
354
|
-
setResults(results)
|
|
355
|
-
}
|
|
356
|
-
</script>
|
|
357
|
-
|
|
358
|
-
<template>
|
|
359
|
-
<input @input="search($event.target.value)" placeholder="Search..." />
|
|
360
|
-
</template>
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
</details>
|
|
364
|
-
|
|
365
|
-
### 🟢 Node.js / Serverless / Edge
|
|
138
|
+
<summary>🔧 <strong>MongoDB-Style Metadata Filtering</strong></summary>
|
|
366
139
|
|
|
367
140
|
```javascript
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
//
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
//
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
141
|
+
const results = await brainy.search("machine learning", 10, {
|
|
142
|
+
metadata: {
|
|
143
|
+
// Comparison operators
|
|
144
|
+
price: { $gte: 100, $lte: 1000 },
|
|
145
|
+
category: { $in: ["AI", "ML", "Data"] },
|
|
146
|
+
rating: { $gt: 4.5 },
|
|
147
|
+
|
|
148
|
+
// Logical operators
|
|
149
|
+
$and: [
|
|
150
|
+
{ status: "active" },
|
|
151
|
+
{ verified: true }
|
|
152
|
+
],
|
|
153
|
+
|
|
154
|
+
// Text operators
|
|
155
|
+
description: { $regex: "neural.*network", $options: "i" },
|
|
156
|
+
|
|
157
|
+
// Array operators
|
|
158
|
+
tags: { $includes: "tensorflow" }
|
|
385
159
|
}
|
|
386
160
|
})
|
|
387
161
|
```
|
|
388
162
|
|
|
389
|
-
**
|
|
390
|
-
|
|
391
|
-
Brainy automatically detects and optimizes for:
|
|
392
|
-
|
|
393
|
-
- 🌐 **Browser frameworks** → OPFS storage, Web Workers, memory optimization
|
|
394
|
-
- 🟢 **Node.js servers** → FileSystem or S3/R2 storage, Worker threads, cluster support
|
|
395
|
-
- ⚡ **Serverless functions** → S3/R2 or Memory storage, cold start optimization
|
|
396
|
-
- 🔥 **Edge workers** → Memory or KV storage, minimal footprint
|
|
397
|
-
- 🧵 **Web/Worker threads** → Shared storage, thread-safe operations
|
|
398
|
-
- 🦕 **Deno/Bun runtimes** → FileSystem or S3-compatible storage, native performance
|
|
399
|
-
|
|
400
|
-
### 🐳 NEW: Zero-Config Docker Deployment
|
|
401
|
-
|
|
402
|
-
**Deploy to any cloud with embedded models - no runtime downloads needed!**
|
|
403
|
-
|
|
404
|
-
```dockerfile
|
|
405
|
-
# One line extracts models automatically during build
|
|
406
|
-
RUN npm run download-models
|
|
163
|
+
**15+ operators supported**: `$gt`, `$gte`, `$lt`, `$lte`, `$eq`, `$ne`, `$in`, `$nin`, `$and`, `$or`, `$not`, `$regex`, `$includes`, `$exists`, `$size`
|
|
407
164
|
|
|
408
|
-
|
|
409
|
-
```
|
|
410
|
-
|
|
411
|
-
- **⚡ 7x Faster Cold Starts** - Models embedded in container, no downloads
|
|
412
|
-
- **🌐 Universal Cloud Support** - Same Dockerfile works everywhere
|
|
413
|
-
- **🔒 Offline Ready** - No external dependencies at runtime
|
|
414
|
-
- **📦 Zero Configuration** - Automatic model detection and loading
|
|
415
|
-
|
|
416
|
-
See [Docker Deployment Guide](./docs/docker-deployment.md) for complete examples.
|
|
417
|
-
|
|
418
|
-
```javascript
|
|
419
|
-
// Zero configuration - everything optimized automatically!
|
|
420
|
-
const brainy = new BrainyData() // Auto-detects environment & optimizes
|
|
421
|
-
await brainy.init()
|
|
422
|
-
|
|
423
|
-
// Caching happens automatically - no setup needed!
|
|
424
|
-
const results1 = await brainy.search('query', 10) // ~50ms first time
|
|
425
|
-
const results2 = await brainy.search('query', 10) // <1ms cached hit!
|
|
426
|
-
|
|
427
|
-
// Advanced pagination works instantly
|
|
428
|
-
const page1 = await brainy.searchWithCursor('query', 100)
|
|
429
|
-
const page2 = await brainy.searchWithCursor('query', 100, {
|
|
430
|
-
cursor: page1.cursor // Constant time, no matter how deep!
|
|
431
|
-
})
|
|
432
|
-
|
|
433
|
-
// Monitor auto-optimized performance
|
|
434
|
-
const stats = brainy.getCacheStats()
|
|
435
|
-
console.log(`Auto-tuned cache hit rate: ${(stats.search.hitRate * 100).toFixed(1)}%`)
|
|
436
|
-
```
|
|
165
|
+
</details>
|
|
437
166
|
|
|
438
|
-
|
|
167
|
+
<details>
|
|
168
|
+
<summary>🔗 <strong>Graph Relationships & Traversal</strong></summary>
|
|
439
169
|
|
|
440
170
|
```javascript
|
|
441
|
-
//
|
|
442
|
-
const
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
})
|
|
171
|
+
// Create entities and relationships
|
|
172
|
+
const company = await brainy.add("OpenAI", { type: "company" })
|
|
173
|
+
const product = await brainy.add("GPT-4", { type: "product" })
|
|
174
|
+
const person = await brainy.add("Sam Altman", { type: "person" })
|
|
446
175
|
|
|
447
|
-
//
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
})
|
|
176
|
+
// Create meaningful relationships
|
|
177
|
+
await brainy.relate(company, product, "develops")
|
|
178
|
+
await brainy.relate(person, company, "leads")
|
|
179
|
+
await brainy.relate(product, person, "created_by")
|
|
452
180
|
|
|
453
|
-
//
|
|
454
|
-
await
|
|
455
|
-
|
|
181
|
+
// Traverse relationships
|
|
182
|
+
const products = await brainy.getVerbsBySource(company) // What OpenAI develops
|
|
183
|
+
const leaders = await brainy.getVerbsByTarget(company) // Who leads OpenAI
|
|
184
|
+
const connections = await brainy.findSimilar(product, {
|
|
185
|
+
relationType: "develops"
|
|
456
186
|
})
|
|
457
187
|
|
|
458
|
-
//
|
|
459
|
-
const results = await
|
|
460
|
-
|
|
188
|
+
// Search with relationship context
|
|
189
|
+
const results = await brainy.search("AI models", 10, {
|
|
190
|
+
includeVerbs: true,
|
|
191
|
+
verbTypes: ["develops", "created_by"],
|
|
192
|
+
searchConnectedNouns: true
|
|
461
193
|
})
|
|
462
|
-
|
|
463
|
-
// Monitor health across all instances
|
|
464
|
-
const health = reader.getHealthStatus()
|
|
465
|
-
console.log(`Instance ${health.instanceId}: ${health.status}`)
|
|
466
194
|
```
|
|
467
195
|
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
### Core Capabilities
|
|
471
|
-
|
|
472
|
-
- **Vector Search** - Find semantically similar content using embeddings
|
|
473
|
-
- **MongoDB-Style Metadata Filtering** 🆕 - Advanced filtering with `$gt`, `$in`, `$regex`, `$and`, `$or` operators
|
|
474
|
-
- **Graph Relationships** - Connect data with meaningful relationships
|
|
475
|
-
- **JSON Document Search** - Search within specific fields with prioritization
|
|
476
|
-
- **Distributed Mode** - Scale horizontally with automatic coordination between instances
|
|
477
|
-
- **Real-Time Syncing** - WebSocket and WebRTC for distributed instances
|
|
478
|
-
- **Streaming Pipeline** - Process data in real-time as it flows through
|
|
479
|
-
- **Model Control Protocol** - Let AI models access your data
|
|
480
|
-
|
|
481
|
-
### Developer Experience
|
|
482
|
-
|
|
483
|
-
- **TypeScript Support** - Fully typed API with generics
|
|
484
|
-
- **Extensible Augmentations** - Customize and extend functionality
|
|
485
|
-
- **REST API** - Web service wrapper for HTTP endpoints
|
|
486
|
-
- **Auto-Complete** - IntelliSense for all APIs and types
|
|
487
|
-
|
|
488
|
-
## 📦 Installation
|
|
489
|
-
|
|
490
|
-
### Development: Quick Start
|
|
491
|
-
|
|
492
|
-
```bash
|
|
493
|
-
npm install @soulcraft/brainy
|
|
494
|
-
```
|
|
495
|
-
|
|
496
|
-
### ✨ Write-Once, Run-Anywhere Architecture
|
|
196
|
+
</details>
|
|
497
197
|
|
|
498
|
-
|
|
198
|
+
<details>
|
|
199
|
+
<summary>🌐 <strong>Universal Storage & Deployment</strong></summary>
|
|
499
200
|
|
|
500
201
|
```javascript
|
|
501
|
-
//
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
const brainy = new BrainyData()
|
|
506
|
-
await brainy.init() // Auto-detects environment and chooses optimal storage
|
|
507
|
-
|
|
508
|
-
// Vector + Graph: Add entities (nouns) with relationships (verbs)
|
|
509
|
-
const companyId = await brainy.addNoun("OpenAI creates powerful AI models", "company", {
|
|
510
|
-
founded: "2015", industry: "AI"
|
|
511
|
-
})
|
|
512
|
-
const productId = await brainy.addNoun("GPT-4 is a large language model", "product", {
|
|
513
|
-
type: "LLM", parameters: "1.7T"
|
|
202
|
+
// Development: File system
|
|
203
|
+
const dev = new BrainyData({
|
|
204
|
+
storage: { fileSystem: { path: './data' } }
|
|
514
205
|
})
|
|
515
206
|
|
|
516
|
-
//
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
// Vector search finds semantically similar content
|
|
520
|
-
const similar = await brainy.search("AI language models", 5)
|
|
521
|
-
|
|
522
|
-
// Graph operations: explore relationships
|
|
523
|
-
const relationships = await brainy.getVerbsBySource(companyId)
|
|
524
|
-
const allProducts = await brainy.getVerbsByType("develops")
|
|
525
|
-
```
|
|
526
|
-
|
|
527
|
-
### 🔍 Advanced Graph Operations
|
|
528
|
-
|
|
529
|
-
```javascript
|
|
530
|
-
// Vector search with graph filtering
|
|
531
|
-
const results = await brainy.search("AI models", 10, {
|
|
532
|
-
searchVerbs: true, // Search relationships directly
|
|
533
|
-
verbTypes: ["develops"], // Filter by relationship types
|
|
534
|
-
searchConnectedNouns: true, // Find entities connected by relationships
|
|
535
|
-
verbDirection: "outgoing" // Direction: outgoing, incoming, or both
|
|
207
|
+
// Production: S3/R2
|
|
208
|
+
const prod = new BrainyData({
|
|
209
|
+
storage: { s3Storage: { bucketName: 'my-vectors' } }
|
|
536
210
|
})
|
|
537
211
|
|
|
538
|
-
//
|
|
539
|
-
const
|
|
540
|
-
const incoming = await brainy.getVerbsByTarget(entityId) // What relates to this entity
|
|
541
|
-
const byType = await brainy.getVerbsByType("develops") // All relationships of this type
|
|
212
|
+
// Browser: OPFS
|
|
213
|
+
const browser = new BrainyData() // Auto-detects OPFS
|
|
542
214
|
|
|
543
|
-
//
|
|
544
|
-
const
|
|
545
|
-
|
|
546
|
-
direction: "both"
|
|
215
|
+
// Edge: Memory
|
|
216
|
+
const edge = new BrainyData({
|
|
217
|
+
storage: { memory: {} }
|
|
547
218
|
})
|
|
548
219
|
|
|
549
|
-
//
|
|
550
|
-
const
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
**Universal benefits:**
|
|
554
|
-
|
|
555
|
-
- ✅ **Auto-detects everything** - Environment, storage, threading, optimization
|
|
556
|
-
- ✅ **Framework-optimized** - Best experience with Angular, React, Vue bundlers
|
|
557
|
-
- ✅ **Runtime-agnostic** - Node.js, Deno, Bun, browsers, serverless, edge
|
|
558
|
-
- ✅ **TypeScript-first** - Full types everywhere, IntelliSense support
|
|
559
|
-
- ✅ **Tree-shaking ready** - Modern bundlers import only what you need
|
|
560
|
-
- ✅ **ES Modules architecture** - Individual modules for better optimization by modern frameworks
|
|
561
|
-
|
|
562
|
-
### Production: Add Offline Model Reliability
|
|
563
|
-
|
|
564
|
-
```bash
|
|
565
|
-
# For development (online model loading)
|
|
566
|
-
npm install @soulcraft/brainy
|
|
567
|
-
|
|
568
|
-
# For production (offline reliability)
|
|
569
|
-
npm install @soulcraft/brainy @soulcraft/brainy-models
|
|
570
|
-
```
|
|
571
|
-
|
|
572
|
-
**Why use offline models in production?**
|
|
573
|
-
|
|
574
|
-
- **🛡️ 100% Reliability** - No network timeouts or blocked URLs
|
|
575
|
-
- **⚡ Instant Startup** - Models load in ~100ms vs 5-30 seconds
|
|
576
|
-
- **🐳 Docker Ready** - Perfect for Cloud Run, Lambda, Kubernetes
|
|
577
|
-
- **🔒 Zero Dependencies** - No external network calls required
|
|
578
|
-
- **🎯 Zero Configuration** - Automatic detection with graceful fallback
|
|
579
|
-
- **🔐 Enhanced Security** - Complete air-gapping support for sensitive environments
|
|
580
|
-
- **🏢 Enterprise Ready** - Works behind corporate firewalls and restricted networks
|
|
581
|
-
- **⚖️ Compliance & Forensics** - Frozen mode for audit trails and legal discovery
|
|
582
|
-
|
|
583
|
-
The offline models provide the **same functionality** with maximum reliability. Your existing code works unchanged -
|
|
584
|
-
Brainy automatically detects and uses bundled models when available.
|
|
585
|
-
|
|
586
|
-
```javascript
|
|
587
|
-
import { createAutoBrainy } from 'brainy'
|
|
588
|
-
import { BundledUniversalSentenceEncoder } from '@soulcraft/brainy-models'
|
|
589
|
-
|
|
590
|
-
// Use the bundled model for offline operation
|
|
591
|
-
const brainy = createAutoBrainy({
|
|
592
|
-
embeddingModel: BundledUniversalSentenceEncoder
|
|
220
|
+
// Redis: High performance
|
|
221
|
+
const redis = new BrainyData({
|
|
222
|
+
storage: { redis: { connectionString: 'redis://...' } }
|
|
593
223
|
})
|
|
594
224
|
```
|
|
595
225
|
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
**Deploy Brainy to any cloud provider with embedded models for maximum performance and reliability.**
|
|
599
|
-
|
|
600
|
-
### Quick Docker Setup
|
|
226
|
+
**Extend with any storage**: MongoDB, PostgreSQL, DynamoDB - [see storage adapters guide](docs/api-reference/storage-adapters.md)
|
|
601
227
|
|
|
602
|
-
|
|
603
|
-
```bash
|
|
604
|
-
npm install @soulcraft/brainy-models
|
|
605
|
-
```
|
|
606
|
-
|
|
607
|
-
2. **Add to your Dockerfile:**
|
|
608
|
-
```dockerfile
|
|
609
|
-
# Extract models during build (zero configuration!)
|
|
610
|
-
RUN npm run download-models
|
|
611
|
-
|
|
612
|
-
# Include models in final image
|
|
613
|
-
COPY --from=builder /app/models ./models
|
|
614
|
-
```
|
|
615
|
-
|
|
616
|
-
3. **Deploy anywhere:**
|
|
617
|
-
```bash
|
|
618
|
-
# Works on all cloud providers
|
|
619
|
-
gcloud run deploy --source . # Google Cloud Run
|
|
620
|
-
aws ecs create-service ... # AWS ECS/Fargate
|
|
621
|
-
az container create ... # Azure Container Instances
|
|
622
|
-
wrangler publish # Cloudflare Workers
|
|
623
|
-
```
|
|
228
|
+
</details>
|
|
624
229
|
|
|
625
|
-
|
|
230
|
+
<details>
|
|
231
|
+
<summary>🐳 <strong>Docker & Cloud Deployment</strong></summary>
|
|
626
232
|
|
|
627
233
|
```dockerfile
|
|
234
|
+
# Production-ready Dockerfile
|
|
628
235
|
FROM node:24-slim AS builder
|
|
629
236
|
WORKDIR /app
|
|
630
237
|
COPY package*.json ./
|
|
631
238
|
RUN npm ci
|
|
632
239
|
COPY . .
|
|
633
|
-
RUN npm run download-models #
|
|
240
|
+
RUN npm run download-models # Embed models for offline operation
|
|
634
241
|
RUN npm run build
|
|
635
242
|
|
|
636
243
|
FROM node:24-slim AS production
|
|
637
244
|
WORKDIR /app
|
|
638
245
|
COPY package*.json ./
|
|
639
|
-
RUN npm ci --only=production
|
|
246
|
+
RUN npm ci --only=production
|
|
640
247
|
COPY --from=builder /app/dist ./dist
|
|
641
|
-
COPY --from=builder /app/models ./models #
|
|
248
|
+
COPY --from=builder /app/models ./models # Offline models included
|
|
642
249
|
CMD ["node", "dist/server.js"]
|
|
643
250
|
```
|
|
644
251
|
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
- **⚡ 7x Faster Cold Starts** - No model download delays
|
|
648
|
-
- **🌐 Universal Compatibility** - Same Dockerfile works on all clouds
|
|
649
|
-
- **🔒 Offline Ready** - No external dependencies at runtime
|
|
650
|
-
- **📦 Zero Configuration** - Automatic model detection
|
|
651
|
-
- **🛡️ Enhanced Security** - No network calls for model loading
|
|
652
|
-
|
|
653
|
-
**📖 Complete Guide:** See [docs/docker-deployment.md](./docs/docker-deployment.md) for detailed examples covering Google
|
|
654
|
-
Cloud Run, AWS Lambda/ECS, Azure Container Instances, Cloudflare Workers, and more.
|
|
655
|
-
|
|
656
|
-
### 📦 Modern ES Modules Architecture
|
|
657
|
-
|
|
658
|
-
Brainy now uses individual ES modules instead of large bundles, providing better optimization for modern frameworks:
|
|
659
|
-
|
|
660
|
-
- **Better tree-shaking**: Frameworks import only the specific functions you use
|
|
661
|
-
- **Smaller final apps**: Your bundled application only includes what you actually need
|
|
662
|
-
- **Faster development builds**: No complex bundling during development
|
|
663
|
-
- **Better debugging**: Source maps point to individual files, not large bundles
|
|
664
|
-
|
|
665
|
-
This change reduced the package size significantly while improving compatibility with Angular, React, Vue, and other
|
|
666
|
-
modern framework build systems.
|
|
667
|
-
|
|
668
|
-
## 🧬 The Power of Nouns & Verbs
|
|
669
|
-
|
|
670
|
-
Brainy uses a **graph-based data model** that mirrors how humans think - with **Nouns** (entities) connected by **Verbs
|
|
671
|
-
** (relationships). This isn't just vectors in a void; it's structured, meaningful data.
|
|
672
|
-
|
|
673
|
-
### 📝 Nouns (What Things Are)
|
|
674
|
-
|
|
675
|
-
Nouns are your entities - the "things" in your data. Each noun has:
|
|
676
|
-
|
|
677
|
-
- A unique ID
|
|
678
|
-
- A vector representation (for similarity search)
|
|
679
|
-
- A type (Person, Document, Concept, etc.)
|
|
680
|
-
- Custom metadata
|
|
681
|
-
|
|
682
|
-
**Available Noun Types:**
|
|
683
|
-
|
|
684
|
-
| Category | Types | Use For |
|
|
685
|
-
|---------------------|-------------------------------------------------------------------|-------------------------------------------------------|
|
|
686
|
-
| **Core Entities** | `Person`, `Organization`, `Location`, `Thing`, `Concept`, `Event` | People, companies, places, objects, ideas, happenings |
|
|
687
|
-
| **Digital Content** | `Document`, `Media`, `File`, `Message`, `Content` | PDFs, images, videos, emails, posts, generic content |
|
|
688
|
-
| **Collections** | `Collection`, `Dataset` | Groups of items, structured data sets |
|
|
689
|
-
| **Business** | `Product`, `Service`, `User`, `Task`, `Project` | E-commerce, SaaS, project management |
|
|
690
|
-
| **Descriptive** | `Process`, `State`, `Role` | Workflows, conditions, responsibilities |
|
|
691
|
-
|
|
692
|
-
### 🔗 Verbs (How Things Connect)
|
|
693
|
-
|
|
694
|
-
Verbs are your relationships - they give meaning to connections. Not just "these vectors are similar" but "this OWNS
|
|
695
|
-
that" or "this CAUSES that".
|
|
696
|
-
|
|
697
|
-
**Available Verb Types:**
|
|
698
|
-
|
|
699
|
-
| Category | Types | Examples |
|
|
700
|
-
|----------------|----------------------------------------------------------------------|------------------------------------------|
|
|
701
|
-
| **Core** | `RelatedTo`, `Contains`, `PartOf`, `LocatedAt`, `References` | Generic relations, containment, location |
|
|
702
|
-
| **Temporal** | `Precedes`, `Succeeds`, `Causes`, `DependsOn`, `Requires` | Time sequences, causality, dependencies |
|
|
703
|
-
| **Creation** | `Creates`, `Transforms`, `Becomes`, `Modifies`, `Consumes` | Creation, change, consumption |
|
|
704
|
-
| **Ownership** | `Owns`, `AttributedTo`, `CreatedBy`, `BelongsTo` | Ownership, authorship, belonging |
|
|
705
|
-
| **Social** | `MemberOf`, `WorksWith`, `FriendOf`, `Follows`, `Likes`, `ReportsTo` | Social networks, organizations |
|
|
706
|
-
| **Functional** | `Describes`, `Implements`, `Validates`, `Triggers`, `Serves` | Functions, implementations, services |
|
|
707
|
-
|
|
708
|
-
### 💡 Why This Matters
|
|
709
|
-
|
|
710
|
-
```javascript
|
|
711
|
-
// Traditional vector DB: Just similarity
|
|
712
|
-
const similar = await vectorDB.search(embedding, 10)
|
|
713
|
-
// Result: [vector1, vector2, ...] - What do these mean? 🤷
|
|
714
|
-
|
|
715
|
-
// Brainy: Similarity + Meaning + Relationships
|
|
716
|
-
const catId = await brainy.add("Siamese cat", {
|
|
717
|
-
noun: NounType.Thing,
|
|
718
|
-
breed: "Siamese"
|
|
719
|
-
})
|
|
720
|
-
const ownerId = await brainy.add("John Smith", {
|
|
721
|
-
noun: NounType.Person
|
|
722
|
-
})
|
|
723
|
-
await brainy.addVerb(ownerId, catId, {
|
|
724
|
-
verb: VerbType.Owns,
|
|
725
|
-
since: "2020-01-01"
|
|
726
|
-
})
|
|
727
|
-
|
|
728
|
-
// Now you can search with context!
|
|
729
|
-
const johnsPets = await brainy.getVerbsBySource(ownerId, VerbType.Owns)
|
|
730
|
-
const catOwners = await brainy.getVerbsByTarget(catId, VerbType.Owns)
|
|
731
|
-
```
|
|
732
|
-
|
|
733
|
-
## 🌍 Distributed Mode (New!)
|
|
734
|
-
|
|
735
|
-
Brainy now supports **distributed deployments** with multiple specialized instances sharing the same data. Perfect for
|
|
736
|
-
scaling your AI applications across multiple servers.
|
|
737
|
-
|
|
738
|
-
### Distributed Setup
|
|
739
|
-
|
|
740
|
-
```javascript
|
|
741
|
-
// Single instance (no change needed!)
|
|
742
|
-
const brainy = createAutoBrainy({
|
|
743
|
-
storage: { s3Storage: { bucketName: 'my-bucket' } }
|
|
744
|
-
})
|
|
745
|
-
|
|
746
|
-
// Distributed mode requires explicit role configuration
|
|
747
|
-
// Option 1: Via environment variable
|
|
748
|
-
process.env.BRAINY_ROLE = 'writer' // or 'reader' or 'hybrid'
|
|
749
|
-
const brainy = createAutoBrainy({
|
|
750
|
-
storage: { s3Storage: { bucketName: 'my-bucket' } },
|
|
751
|
-
distributed: true
|
|
752
|
-
})
|
|
753
|
-
|
|
754
|
-
// Option 2: Via configuration
|
|
755
|
-
const writer = createAutoBrainy({
|
|
756
|
-
storage: { s3Storage: { bucketName: 'my-bucket' } },
|
|
757
|
-
distributed: { role: 'writer' } // Handles data ingestion
|
|
758
|
-
})
|
|
759
|
-
|
|
760
|
-
const reader = createAutoBrainy({
|
|
761
|
-
storage: { s3Storage: { bucketName: 'my-bucket' } },
|
|
762
|
-
distributed: { role: 'reader' } // Optimized for queries
|
|
763
|
-
})
|
|
764
|
-
|
|
765
|
-
// Option 3: Via read/write mode (role auto-inferred)
|
|
766
|
-
const writer = createAutoBrainy({
|
|
767
|
-
storage: { s3Storage: { bucketName: 'my-bucket' } },
|
|
768
|
-
writeOnly: true, // Automatically becomes 'writer' role
|
|
769
|
-
distributed: true
|
|
770
|
-
})
|
|
771
|
-
|
|
772
|
-
const reader = createAutoBrainy({
|
|
773
|
-
storage: { s3Storage: { bucketName: 'my-bucket' } },
|
|
774
|
-
readOnly: true, // Automatically becomes 'reader' role (allows optimizations)
|
|
775
|
-
// frozen: true, // Optional: Complete immutability for compliance/forensics
|
|
776
|
-
distributed: true
|
|
777
|
-
})
|
|
778
|
-
```
|
|
779
|
-
|
|
780
|
-
### Key Distributed Features
|
|
781
|
-
|
|
782
|
-
**🎯 Explicit Role Configuration**
|
|
783
|
-
|
|
784
|
-
- Roles must be explicitly set (no dangerous auto-assignment)
|
|
785
|
-
- Can use environment variables, config, or read/write modes
|
|
786
|
-
- Clear separation between writers and readers
|
|
787
|
-
|
|
788
|
-
**#️⃣ Hash-Based Partitioning**
|
|
789
|
-
|
|
790
|
-
- Handles multiple writers with different data types
|
|
791
|
-
- Even distribution across partitions
|
|
792
|
-
- No semantic conflicts with mixed data
|
|
793
|
-
|
|
794
|
-
**🏷️ Domain Tagging**
|
|
795
|
-
|
|
796
|
-
- Automatic domain detection (medical, legal, product, etc.)
|
|
797
|
-
- Filter searches by domain
|
|
798
|
-
- Logical separation without complexity
|
|
799
|
-
|
|
800
|
-
```javascript
|
|
801
|
-
// Data is automatically tagged with domains
|
|
802
|
-
await brainy.add({
|
|
803
|
-
symptoms: "fever",
|
|
804
|
-
diagnosis: "flu"
|
|
805
|
-
}, metadata) // Auto-tagged as 'medical'
|
|
806
|
-
|
|
807
|
-
// Search within specific domains
|
|
808
|
-
const medicalResults = await brainy.search(query, 10, {
|
|
809
|
-
filter: { domain: 'medical' }
|
|
810
|
-
})
|
|
811
|
-
```
|
|
812
|
-
|
|
813
|
-
**📊 Health Monitoring**
|
|
814
|
-
|
|
815
|
-
- Real-time health metrics
|
|
816
|
-
- Automatic dead instance cleanup
|
|
817
|
-
- Performance tracking
|
|
818
|
-
|
|
819
|
-
```javascript
|
|
820
|
-
// Get health status
|
|
821
|
-
const health = brainy.getHealthStatus()
|
|
822
|
-
// {
|
|
823
|
-
// status: 'healthy',
|
|
824
|
-
// role: 'reader',
|
|
825
|
-
// vectorCount: 1000000,
|
|
826
|
-
// cacheHitRate: 0.95,
|
|
827
|
-
// requestsPerSecond: 150
|
|
828
|
-
// }
|
|
829
|
-
```
|
|
830
|
-
|
|
831
|
-
**⚡ Role-Optimized Performance**
|
|
832
|
-
|
|
833
|
-
- **Readers**: 80% memory for cache, aggressive prefetching
|
|
834
|
-
- **Writers**: Optimized write batching, minimal cache
|
|
835
|
-
- **Hybrid**: Adaptive based on workload
|
|
836
|
-
|
|
837
|
-
## ⚖️ Compliance & Forensics Mode
|
|
838
|
-
|
|
839
|
-
For legal discovery, audit trails, and compliance requirements:
|
|
840
|
-
|
|
841
|
-
```javascript
|
|
842
|
-
// Create a completely immutable snapshot
|
|
843
|
-
const auditDb = new BrainyData({
|
|
844
|
-
storage: { s3Storage: { bucketName: 'audit-snapshots' } },
|
|
845
|
-
readOnly: true,
|
|
846
|
-
frozen: true // Complete immutability - no changes allowed
|
|
847
|
-
})
|
|
848
|
-
|
|
849
|
-
// Perfect for:
|
|
850
|
-
// - Legal discovery (data cannot be modified)
|
|
851
|
-
// - Compliance audits (guaranteed state)
|
|
852
|
-
// - Forensic analysis (preserved evidence)
|
|
853
|
-
// - Regulatory snapshots (unchanging records)
|
|
854
|
-
```
|
|
855
|
-
|
|
856
|
-
### Deployment Examples
|
|
857
|
-
|
|
858
|
-
**Docker Compose**
|
|
859
|
-
|
|
860
|
-
```yaml
|
|
861
|
-
services:
|
|
862
|
-
writer:
|
|
863
|
-
image: myapp
|
|
864
|
-
environment:
|
|
865
|
-
BRAINY_ROLE: writer # Optional - auto-detects
|
|
866
|
-
|
|
867
|
-
reader:
|
|
868
|
-
image: myapp
|
|
869
|
-
environment:
|
|
870
|
-
BRAINY_ROLE: reader # Optional - auto-detects
|
|
871
|
-
scale: 5
|
|
872
|
-
```
|
|
873
|
-
|
|
874
|
-
**Kubernetes**
|
|
875
|
-
|
|
876
|
-
```yaml
|
|
877
|
-
# Automatically detects role from deployment type
|
|
878
|
-
apiVersion: apps/v1
|
|
879
|
-
kind: Deployment
|
|
880
|
-
metadata:
|
|
881
|
-
name: brainy-readers
|
|
882
|
-
spec:
|
|
883
|
-
replicas: 10 # Multiple readers
|
|
884
|
-
template:
|
|
885
|
-
spec:
|
|
886
|
-
containers:
|
|
887
|
-
- name: app
|
|
888
|
-
image: myapp
|
|
889
|
-
# Role auto-detected as 'reader' (multiple replicas)
|
|
890
|
-
```
|
|
891
|
-
|
|
892
|
-
**Benefits**
|
|
893
|
-
|
|
894
|
-
- ✅ **50-70% faster searches** with parallel readers
|
|
895
|
-
- ✅ **No coordination complexity** - Shared JSON config in S3
|
|
896
|
-
- ✅ **Zero downtime scaling** - Add/remove instances anytime
|
|
897
|
-
- ✅ **Automatic failover** - Dead instances cleaned up automatically
|
|
898
|
-
|
|
899
|
-
## 🤔 Why Choose Brainy?
|
|
900
|
-
|
|
901
|
-
### vs. Traditional Databases
|
|
902
|
-
|
|
903
|
-
❌ **PostgreSQL with pgvector** - Requires complex setup, tuning, and DevOps expertise
|
|
904
|
-
✅ **Brainy** - Zero config, auto-optimizes, works everywhere from browser to cloud
|
|
905
|
-
|
|
906
|
-
### vs. Vector Databases
|
|
907
|
-
|
|
908
|
-
❌ **Pinecone/Weaviate/Qdrant** - Cloud-only, expensive, vendor lock-in
|
|
909
|
-
✅ **Brainy** - Run locally, in browser, or cloud. Your choice, your data
|
|
910
|
-
|
|
911
|
-
### vs. Graph Databases
|
|
912
|
-
|
|
913
|
-
❌ **Neo4j** - Great for graphs, no vector support
|
|
914
|
-
✅ **Brainy** - Vectors + graphs in one. Best of both worlds
|
|
915
|
-
|
|
916
|
-
### vs. "Vector + Graph" Solutions
|
|
917
|
-
|
|
918
|
-
❌ **Pinecone + Neo4j** - Two databases, sync nightmares, double the cost
|
|
919
|
-
❌ **pgvector + graph extension** - Hacked together, not native, performance issues
|
|
920
|
-
❌ **Weaviate "references"** - Limited graph capabilities, not true relationships
|
|
921
|
-
✅ **Brainy** - Purpose-built vector+graph architecture, single source of truth
|
|
922
|
-
|
|
923
|
-
### vs. DIY Solutions
|
|
924
|
-
|
|
925
|
-
❌ **Building your own** - Months of work, optimization nightmares
|
|
926
|
-
✅ **Brainy** - Production-ready in 30 seconds
|
|
927
|
-
|
|
928
|
-
## 🚀 Getting Started in 30 Seconds
|
|
929
|
-
|
|
930
|
-
**The same Brainy code works everywhere - React, Vue, Angular, Node.js, Serverless, Edge Workers.**
|
|
931
|
-
|
|
932
|
-
```javascript
|
|
933
|
-
// This EXACT code works in ALL environments
|
|
934
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
935
|
-
|
|
936
|
-
const brainy = new BrainyData()
|
|
937
|
-
await brainy.init()
|
|
938
|
-
|
|
939
|
-
// Add nouns (entities)
|
|
940
|
-
const openai = await brainy.add("OpenAI", { type: "company" })
|
|
941
|
-
const gpt4 = await brainy.add("GPT-4", { type: "product" })
|
|
942
|
-
|
|
943
|
-
// Add verbs (relationships)
|
|
944
|
-
await brainy.relate(openai, gpt4, "develops")
|
|
945
|
-
|
|
946
|
-
// Vector search + Graph traversal
|
|
947
|
-
const similar = await brainy.search("AI companies", 5)
|
|
948
|
-
const products = await brainy.getVerbsBySource(openai)
|
|
949
|
-
```
|
|
950
|
-
|
|
951
|
-
<details>
|
|
952
|
-
<summary>🔍 See Framework Examples</summary>
|
|
953
|
-
|
|
954
|
-
### React
|
|
955
|
-
|
|
956
|
-
```jsx
|
|
957
|
-
function App() {
|
|
958
|
-
const [brainy] = useState(() => new BrainyData())
|
|
959
|
-
useEffect(() => brainy.init(), [])
|
|
960
|
-
|
|
961
|
-
const search = async (query) => {
|
|
962
|
-
return await brainy.search(query, 10)
|
|
963
|
-
}
|
|
964
|
-
// Same API as above
|
|
965
|
-
}
|
|
966
|
-
```
|
|
967
|
-
|
|
968
|
-
### Vue 3
|
|
969
|
-
|
|
970
|
-
```vue
|
|
971
|
-
|
|
972
|
-
<script setup>
|
|
973
|
-
const brainy = new BrainyData()
|
|
974
|
-
await brainy.init()
|
|
975
|
-
// Same API as above
|
|
976
|
-
</script>
|
|
977
|
-
```
|
|
978
|
-
|
|
979
|
-
### Angular
|
|
980
|
-
|
|
981
|
-
```typescript
|
|
982
|
-
|
|
983
|
-
@Component({})
|
|
984
|
-
export class AppComponent {
|
|
985
|
-
brainy = new BrainyData()
|
|
986
|
-
|
|
987
|
-
async ngOnInit() {
|
|
988
|
-
await this.brainy.init()
|
|
989
|
-
// Same API as above
|
|
990
|
-
}
|
|
991
|
-
}
|
|
992
|
-
```
|
|
993
|
-
|
|
994
|
-
### Node.js / Deno / Bun
|
|
995
|
-
|
|
996
|
-
```javascript
|
|
997
|
-
const brainy = new BrainyData()
|
|
998
|
-
await brainy.init()
|
|
999
|
-
// Same API as above
|
|
1000
|
-
```
|
|
1001
|
-
|
|
1002
|
-
</details>
|
|
1003
|
-
|
|
1004
|
-
### 🌍 Framework-First, Runs Everywhere
|
|
1005
|
-
|
|
1006
|
-
**Brainy automatically detects your environment and optimizes everything:**
|
|
1007
|
-
|
|
1008
|
-
| Environment | Storage | Optimization |
|
|
1009
|
-
|-----------------|-----------------|----------------------------|
|
|
1010
|
-
| 🌐 Browser | OPFS | Web Workers, Memory Cache |
|
|
1011
|
-
| 🟢 Node.js | FileSystem / S3 | Worker Threads, Clustering |
|
|
1012
|
-
| ⚡ Serverless | S3 / Memory | Cold Start Optimization |
|
|
1013
|
-
| 🔥 Edge Workers | Memory / KV | Minimal Footprint |
|
|
1014
|
-
| 🦕 Deno/Bun | FileSystem / S3 | Native Performance |
|
|
1015
|
-
|
|
1016
|
-
## 🌐 Deploy to Any Cloud
|
|
1017
|
-
|
|
1018
|
-
<details>
|
|
1019
|
-
<summary>☁️ See Cloud Platform Examples</summary>
|
|
1020
|
-
|
|
1021
|
-
### Cloudflare Workers
|
|
1022
|
-
|
|
1023
|
-
```javascript
|
|
1024
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
1025
|
-
|
|
1026
|
-
export default {
|
|
1027
|
-
async fetch(request) {
|
|
1028
|
-
const brainy = new BrainyData()
|
|
1029
|
-
await brainy.init()
|
|
1030
|
-
|
|
1031
|
-
const url = new URL(request.url)
|
|
1032
|
-
const results = await brainy.search(url.searchParams.get('q'), 10)
|
|
1033
|
-
return Response.json(results)
|
|
1034
|
-
}
|
|
1035
|
-
}
|
|
1036
|
-
```
|
|
1037
|
-
|
|
1038
|
-
### AWS Lambda
|
|
1039
|
-
|
|
1040
|
-
```javascript
|
|
1041
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
1042
|
-
|
|
1043
|
-
export const handler = async (event) => {
|
|
1044
|
-
const brainy = new BrainyData()
|
|
1045
|
-
await brainy.init()
|
|
1046
|
-
|
|
1047
|
-
const results = await brainy.search(event.query, 10)
|
|
1048
|
-
return { statusCode: 200, body: JSON.stringify(results) }
|
|
1049
|
-
}
|
|
1050
|
-
```
|
|
1051
|
-
|
|
1052
|
-
### Google Cloud Functions
|
|
1053
|
-
|
|
1054
|
-
```javascript
|
|
1055
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
1056
|
-
|
|
1057
|
-
export const searchHandler = async (req, res) => {
|
|
1058
|
-
const brainy = new BrainyData()
|
|
1059
|
-
await brainy.init()
|
|
1060
|
-
|
|
1061
|
-
const results = await brainy.search(req.query.q, 10)
|
|
1062
|
-
res.json(results)
|
|
1063
|
-
}
|
|
1064
|
-
```
|
|
1065
|
-
|
|
1066
|
-
### Vercel Edge Functions
|
|
1067
|
-
|
|
1068
|
-
```javascript
|
|
1069
|
-
import { BrainyData } from '@soulcraft/brainy'
|
|
1070
|
-
|
|
1071
|
-
export const config = { runtime: 'edge' }
|
|
1072
|
-
|
|
1073
|
-
export default async function handler(request) {
|
|
1074
|
-
const brainy = new BrainyData()
|
|
1075
|
-
await brainy.init()
|
|
1076
|
-
|
|
1077
|
-
const { searchParams } = new URL(request.url)
|
|
1078
|
-
const results = await brainy.search(searchParams.get('q'), 10)
|
|
1079
|
-
return Response.json(results)
|
|
1080
|
-
}
|
|
1081
|
-
```
|
|
252
|
+
Deploy to: Google Cloud Run, AWS Lambda/ECS, Azure Container Instances, Cloudflare Workers, Railway, Render, Vercel, anywhere Docker runs.
|
|
1082
253
|
|
|
1083
254
|
</details>
|
|
1084
255
|
|
|
1085
|
-
|
|
1086
|
-
|
|
1087
|
-
```dockerfile
|
|
1088
|
-
FROM node:24-slim
|
|
1089
|
-
USER node
|
|
1090
|
-
WORKDIR /app
|
|
1091
|
-
COPY package*.json ./
|
|
1092
|
-
RUN npm install brainy
|
|
1093
|
-
COPY . .
|
|
1094
|
-
|
|
1095
|
-
CMD ["node", "server.js"]
|
|
1096
|
-
```
|
|
1097
|
-
|
|
1098
|
-
```javascript
|
|
1099
|
-
// server.js
|
|
1100
|
-
import { createAutoBrainy } from 'brainy'
|
|
1101
|
-
import express from 'express'
|
|
1102
|
-
|
|
1103
|
-
const app = express()
|
|
1104
|
-
const brainy = createAutoBrainy()
|
|
1105
|
-
|
|
1106
|
-
app.get('/search', async (req, res) => {
|
|
1107
|
-
const results = await brainy.searchText(req.query.q, 10)
|
|
1108
|
-
res.json(results)
|
|
1109
|
-
})
|
|
1110
|
-
|
|
1111
|
-
app.listen(3000, () => console.log('Brainy running on port 3000'))
|
|
1112
|
-
```
|
|
1113
|
-
|
|
1114
|
-
### Kubernetes
|
|
1115
|
-
|
|
1116
|
-
```yaml
|
|
1117
|
-
apiVersion: apps/v1
|
|
1118
|
-
kind: Deployment
|
|
1119
|
-
metadata:
|
|
1120
|
-
name: brainy-api
|
|
1121
|
-
spec:
|
|
1122
|
-
replicas: 3
|
|
1123
|
-
template:
|
|
1124
|
-
spec:
|
|
1125
|
-
containers:
|
|
1126
|
-
- name: brainy
|
|
1127
|
-
image: your-registry/brainy-api:latest
|
|
1128
|
-
env:
|
|
1129
|
-
- name: S3_BUCKET
|
|
1130
|
-
value: "your-vector-bucket"
|
|
1131
|
-
```
|
|
1132
|
-
|
|
1133
|
-
### Railway.app
|
|
1134
|
-
|
|
1135
|
-
```javascript
|
|
1136
|
-
// server.js
|
|
1137
|
-
import { createAutoBrainy } from 'brainy'
|
|
1138
|
-
|
|
1139
|
-
const brainy = createAutoBrainy({
|
|
1140
|
-
bucketName: process.env.RAILWAY_VOLUME_NAME
|
|
1141
|
-
})
|
|
1142
|
-
|
|
1143
|
-
// Railway automatically handles the rest!
|
|
1144
|
-
```
|
|
1145
|
-
|
|
1146
|
-
### Render.com
|
|
1147
|
-
|
|
1148
|
-
```yaml
|
|
1149
|
-
# render.yaml
|
|
1150
|
-
services:
|
|
1151
|
-
- type: web
|
|
1152
|
-
name: brainy-api
|
|
1153
|
-
env: node
|
|
1154
|
-
buildCommand: npm install brainy
|
|
1155
|
-
startCommand: node server.js
|
|
1156
|
-
envVars:
|
|
1157
|
-
- key: BRAINY_STORAGE
|
|
1158
|
-
value: persistent-disk
|
|
1159
|
-
```
|
|
1160
|
-
|
|
1161
|
-
## Getting Started
|
|
1162
|
-
|
|
1163
|
-
- [**Quick Start Guide**](docs/getting-started/) - Get up and running in minutes
|
|
1164
|
-
- [**Installation**](docs/getting-started/installation.md) - Detailed setup instructions
|
|
1165
|
-
- [**Environment Setup**](docs/getting-started/environment-setup.md) - Platform-specific configuration
|
|
256
|
+
## 📚 Documentation & Resources
|
|
1166
257
|
|
|
1167
|
-
|
|
1168
|
-
|
|
1169
|
-
- [
|
|
1170
|
-
- [
|
|
1171
|
-
- [
|
|
1172
|
-
- [**Production Migration**](docs/guides/production-migration-guide.md) - Deployment best practices
|
|
1173
|
-
|
|
1174
|
-
### API Reference
|
|
1175
|
-
|
|
1176
|
-
- [**Core API**](docs/api-reference/) - Complete method reference
|
|
1177
|
-
- [**Configuration Options**](docs/api-reference/configuration.md) - All configuration parameters
|
|
1178
|
-
|
|
1179
|
-
### Optimization & Scaling
|
|
1180
|
-
|
|
1181
|
-
- [**Performance Features Guide**](docs/PERFORMANCE_FEATURES.md) - Advanced caching, auto-configuration, and
|
|
1182
|
-
optimization
|
|
1183
|
-
- [**Large-Scale Optimizations**](docs/optimization-guides/) - Handle millions of vectors
|
|
1184
|
-
- [**Memory Management**](docs/optimization-guides/memory-optimization.md) - Efficient resource usage
|
|
1185
|
-
- [**S3 Migration Guide**](docs/optimization-guides/s3-migration-guide.md) - Cloud storage setup
|
|
1186
|
-
|
|
1187
|
-
### Examples & Patterns
|
|
1188
|
-
|
|
1189
|
-
- [**Code Examples**](docs/examples/) - Real-world usage patterns
|
|
1190
|
-
- [**Integrations**](docs/examples/integrations.md) - Third-party services
|
|
1191
|
-
- [**Performance Patterns**](docs/examples/performance.md) - Optimization techniques
|
|
1192
|
-
|
|
1193
|
-
### Technical Documentation
|
|
1194
|
-
|
|
1195
|
-
- [**Architecture Overview**](docs/technical/) - System design and internals
|
|
1196
|
-
- [**Testing Guide**](docs/technical/TESTING.md) - Testing strategies
|
|
1197
|
-
- [**Statistics & Monitoring**](docs/technical/STATISTICS.md) - Performance tracking
|
|
258
|
+
- **[🚀 Quick Start Guide](docs/getting-started/)** - Get up and running in minutes
|
|
259
|
+
- **[📖 API Reference](docs/api-reference/)** - Complete method documentation
|
|
260
|
+
- **[💡 Examples](docs/examples/)** - Real-world usage patterns
|
|
261
|
+
- **[⚡ Performance Guide](docs/optimization-guides/)** - Scale to millions of vectors
|
|
262
|
+
- **[🔧 Storage Adapters](docs/api-reference/storage-adapters.md)** - Universal storage compatibility
|
|
1198
263
|
|
|
1199
264
|
## 🤝 Contributing
|
|
1200
265
|
|
|
@@ -1211,5 +276,7 @@ We welcome contributions! Please see:
|
|
|
1211
276
|
---
|
|
1212
277
|
|
|
1213
278
|
<div align="center">
|
|
1214
|
-
<strong>Ready to build
|
|
1215
|
-
|
|
279
|
+
<strong>Ready to build the future of search? Get started with Brainy today!</strong>
|
|
280
|
+
|
|
281
|
+
**[Get Started →](docs/getting-started/) | [View Examples →](docs/examples/) | [Join Community →](https://github.com/soulcraft-research/brainy/discussions)**
|
|
282
|
+
</div>
|