@soulcraft/brainy 0.47.0 → 0.49.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/README.md +268 -554
  2. package/dist/brainyData.d.ts +83 -2
  3. package/dist/brainyData.js +536 -66
  4. package/dist/brainyData.js.map +1 -1
  5. package/dist/coreTypes.d.ts +74 -12
  6. package/dist/distributed/configManager.d.ts +9 -0
  7. package/dist/distributed/configManager.js +129 -10
  8. package/dist/distributed/configManager.js.map +1 -1
  9. package/dist/hnsw/hnswIndex.d.ts +2 -2
  10. package/dist/hnsw/hnswIndex.js +48 -29
  11. package/dist/hnsw/hnswIndex.js.map +1 -1
  12. package/dist/hnsw/optimizedHNSWIndex.d.ts +1 -1
  13. package/dist/hnsw/optimizedHNSWIndex.js +3 -3
  14. package/dist/hnsw/optimizedHNSWIndex.js.map +1 -1
  15. package/dist/storage/adapters/baseStorageAdapter.d.ts +18 -2
  16. package/dist/storage/adapters/baseStorageAdapter.js +69 -4
  17. package/dist/storage/adapters/baseStorageAdapter.js.map +1 -1
  18. package/dist/storage/adapters/fileSystemStorage.d.ts +14 -8
  19. package/dist/storage/adapters/fileSystemStorage.js +90 -22
  20. package/dist/storage/adapters/fileSystemStorage.js.map +1 -1
  21. package/dist/storage/adapters/memoryStorage.d.ts +0 -8
  22. package/dist/storage/adapters/memoryStorage.js +26 -45
  23. package/dist/storage/adapters/memoryStorage.js.map +1 -1
  24. package/dist/storage/adapters/opfsStorage.d.ts +40 -8
  25. package/dist/storage/adapters/opfsStorage.js +195 -44
  26. package/dist/storage/adapters/opfsStorage.js.map +1 -1
  27. package/dist/storage/adapters/optimizedS3Search.js +4 -3
  28. package/dist/storage/adapters/optimizedS3Search.js.map +1 -1
  29. package/dist/storage/adapters/s3CompatibleStorage.d.ts +3 -10
  30. package/dist/storage/adapters/s3CompatibleStorage.js +41 -44
  31. package/dist/storage/adapters/s3CompatibleStorage.js.map +1 -1
  32. package/dist/storage/backwardCompatibility.d.ts +84 -0
  33. package/dist/storage/backwardCompatibility.js +141 -0
  34. package/dist/storage/backwardCompatibility.js.map +1 -0
  35. package/dist/storage/baseStorage.d.ts +33 -19
  36. package/dist/storage/baseStorage.js +116 -195
  37. package/dist/storage/baseStorage.js.map +1 -1
  38. package/dist/utils/distance.d.ts +4 -4
  39. package/dist/utils/distance.js +4 -4
  40. package/dist/utils/embedding.d.ts +10 -0
  41. package/dist/utils/embedding.js +81 -9
  42. package/dist/utils/embedding.js.map +1 -1
  43. package/dist/utils/metadataFilter.d.ts +79 -0
  44. package/dist/utils/metadataFilter.js +229 -0
  45. package/dist/utils/metadataFilter.js.map +1 -0
  46. package/dist/utils/metadataIndex.d.ts +148 -0
  47. package/dist/utils/metadataIndex.js +639 -0
  48. package/dist/utils/metadataIndex.js.map +1 -0
  49. package/dist/utils/metadataIndexCache.d.ts +60 -0
  50. package/dist/utils/metadataIndexCache.js +119 -0
  51. package/dist/utils/metadataIndexCache.js.map +1 -0
  52. package/package.json +4 -4
package/README.md CHANGED
@@ -11,28 +11,56 @@
11
11
 
12
12
  </div>
13
13
 
14
- ## 🔥 MAJOR UPDATE: TensorFlow.js Transformers.js Migration (v0.46+)
14
+ ## 🔥 MAJOR UPDATES: What's New in v0.49, v0.48 & v0.46+
15
15
 
16
- **We've completely replaced TensorFlow.js with Transformers.js for better performance and true offline operation!**
16
+ ### 🎯 **v0.49: Filter Discovery & Performance Improvements**
17
17
 
18
- ### Why We Made This Change
18
+ **Discover available filters and scale to millions of items!**
19
19
 
20
- **The Honest Truth About TensorFlow.js:**
20
+ ```javascript
21
+ // Discover what filters are available - O(1) field lookup
22
+ const categories = await brainy.getFilterValues('category')
23
+ // Returns: ['electronics', 'books', 'clothing', ...]
24
+
25
+ const fields = await brainy.getFilterFields() // O(1) operation
26
+ // Returns: ['category', 'price', 'brand', 'rating', ...]
27
+ ```
28
+
29
+ - ✅ **Filter Discovery API**: O(1) field discovery for instant filter UI generation
30
+ - ✅ **Improved Performance**: Removed deprecated methods, now uses pagination everywhere
31
+ - ✅ **Better Scalability**: Hybrid indexing with O(1) field access scales to millions
32
+ - ✅ **Smart Caching**: LRU cache for frequently accessed filters
33
+ - ✅ **Zero Configuration**: Everything auto-optimizes based on usage patterns
34
+
35
+ ### 🚀 **v0.48: MongoDB-Style Metadata Filtering**
36
+
37
+ **Powerful querying with familiar syntax - filter DURING search for maximum performance!**
38
+
39
+ ```javascript
40
+ const results = await brainy.search("wireless headphones", 10, {
41
+ metadata: {
42
+ category: { $in: ["electronics", "audio"] },
43
+ price: { $lte: 200 },
44
+ rating: { $gte: 4.0 },
45
+ brand: { $ne: "Generic" }
46
+ }
47
+ })
48
+ ```
49
+
50
+ - ✅ **15+ MongoDB Operators**: `$gt`, `$in`, `$regex`, `$and`, `$or`, `$includes`, etc.
51
+ - ✅ **Automatic Indexing**: Zero configuration, maximum performance
52
+ - ✅ **Nested Fields**: Use dot notation for complex objects
53
+ - ✅ **100% Backward Compatible**: Your existing code works unchanged
21
54
 
22
- - 📦 **Massive Package Size**: 12.5MB+ packages with complex dependency trees
23
- - 🌐 **Hidden Network Calls**: Even "local" models triggered fetch() calls internally
24
- - 🐛 **Dependency Hell**: Constant `--legacy-peer-deps` issues with Node.js updates
25
- - 🔧 **Maintenance Burden**: 47+ dependencies to keep compatible across environments
26
- - 💾 **Huge Models**: 525MB Universal Sentence Encoder models
55
+ ### **v0.46: Transformers.js Migration**
27
56
 
28
- ### What You Get Now
57
+ **Replaced TensorFlow.js for better performance and true offline operation!**
29
58
 
30
- - ✅ **95% Smaller Package**: 643 kB vs 12.5 MB (and it actually works better!)
31
- - ✅ **84% Smaller Models**: 87 MB vs 525 MB all-MiniLM-L6-v2 vs USE
32
- - ✅ **True Offline Operation**: Zero network calls after initial model download
33
- - ✅ **5x Fewer Dependencies**: Clean dependency tree, no more peer dep issues
34
- - ✅ **Same API**: Drop-in replacement - your existing code just works
35
- - ✅ **Better Performance**: ONNX Runtime is faster than TensorFlow.js in most cases
59
+ - ✅ **95% Smaller Package**: 643 kB vs 12.5 MB
60
+ - ✅ **84% Smaller Models**: 87 MB vs 525 MB models
61
+ - ✅ **True Offline**: Zero network calls after initial download
62
+ - ✅ **5x Fewer Dependencies**: Clean tree, no peer dependency issues
63
+ - ✅ **Same API**: Drop-in replacement, existing code works unchanged
36
64
 
37
65
  ### Migration (It's Automatic!)
38
66
 
@@ -58,9 +86,22 @@ RUN npm run download-models # Download during build for offline production
58
86
 
59
87
  ## ✨ What is Brainy?
60
88
 
61
- Imagine a database that thinks like you do - connecting ideas, finding patterns, and getting smarter over time. Brainy
62
- is the **AI-native database** that brings vector search and knowledge graphs together in one powerful, ridiculously
63
- easy-to-use package.
89
+ **One API. Every environment. Zero configuration.**
90
+
91
+ Brainy is the **AI-native database** that combines vector search and knowledge graphs in one unified API. Write your
92
+ code once, and it runs everywhere - browsers, Node.js, serverless, edge workers - with automatic optimization for each
93
+ environment.
94
+
95
+ ```javascript
96
+ // This same code works EVERYWHERE
97
+ const brainy = new BrainyData()
98
+ await brainy.init()
99
+
100
+ // Vector search (like Pinecone) + Graph database (like Neo4j)
101
+ await brainy.add("OpenAI", { type: "company" }) // Nouns
102
+ await brainy.relate(openai, gpt4, "develops") // Verbs
103
+ const results = await brainy.search("AI", 10) // Semantic search
104
+ ```
64
105
 
65
106
  ### 🆕 NEW: Distributed Mode (v0.38+)
66
107
 
@@ -78,7 +119,8 @@ easy-to-use package.
78
119
  environment and optimizes itself
79
120
  - **🌍 True Write-Once, Run-Anywhere** - Same code runs in Angular, React, Vue, Node.js, Deno, Bun, serverless, edge
80
121
  workers, and web workers with automatic environment detection
81
- - **⚡ Scary Fast** - Handles millions of vectors with sub-millisecond search. Built-in GPU acceleration when available
122
+ - **⚡ Scary Fast** - Handles millions of vectors with sub-millisecond search. GPU acceleration for embeddings, optimized
123
+ CPU for distance calculations
82
124
  - **🎯 Self-Learning** - Like having a database that goes to the gym. Gets faster and smarter the more you use it
83
125
  - **🔮 AI-First Design** - Built for the age of embeddings, RAG, and semantic search. Your LLMs will thank you
84
126
  - **🎮 Actually Fun to Use** - Clean API, great DX, and it does the heavy lifting so you can build cool stuff
@@ -103,41 +145,51 @@ npm install @soulcraft/brainy
103
145
  ```javascript
104
146
  import { BrainyData } from '@soulcraft/brainy'
105
147
 
148
+ // Same code works EVERYWHERE - browser, Node.js, cloud, edge
106
149
  const brainy = new BrainyData()
107
150
  await brainy.init() // Auto-detects your environment
108
151
 
109
- // Add some data
110
- await brainy.add("The quick brown fox jumps over the lazy dog")
111
- await brainy.add("A fast fox leaps over a sleeping dog")
112
- await brainy.add("Cats are independent and mysterious animals")
113
-
114
- // Vector search finds similar content
115
- const results = await brainy.search("speedy animals jumping", 2)
116
- console.log(results) // Finds the fox sentences!
117
- ```
152
+ // 1️⃣ Simple vector search (like Pinecone)
153
+ await brainy.add("The quick brown fox jumps over the lazy dog", { type: "sentence" })
154
+ await brainy.add("Cats are independent and mysterious animals", { type: "sentence" })
118
155
 
119
- **🎯 That's it!** You just built semantic search in 4 lines. Works in Angular, React, Vue, Node.js, browsers,
120
- serverless - everywhere.
156
+ const results = await brainy.search("fast animals", 5)
157
+ // Finds similar content by meaning, not keywords!
121
158
 
122
- ## 🚀 The Magic: Vector + Graph Database
159
+ // 2️⃣ Graph relationships (like Neo4j)
160
+ const openai = await brainy.add("OpenAI", { type: "company", founded: 2015 })
161
+ const gpt4 = await brainy.add("GPT-4", { type: "product", released: 2023 })
162
+ const sam = await brainy.add("Sam Altman", { type: "person", role: "CEO" })
123
163
 
124
- **Most databases do one thing.** Brainy does both vector similarity AND graph relationships:
164
+ // Create relationships between entities
165
+ await brainy.relate(openai, gpt4, "develops")
166
+ await brainy.relate(sam, openai, "leads")
167
+ await brainy.relate(gpt4, sam, "created_by")
125
168
 
126
- ```javascript
127
- // Add entities with relationships
128
- const companyId = await brainy.addNoun("OpenAI creates powerful AI models", "company")
129
- const productId = await brainy.addNoun("GPT-4 is a large language model", "product")
169
+ // 3️⃣ Combined power: Vector search + Graph traversal
170
+ const similar = await brainy.search("AI language models", 10) // Find by meaning
171
+ const products = await brainy.getVerbsBySource(openai) // Get relationships
172
+ const graph = await brainy.findSimilar(gpt4, { relationType: "develops" })
130
173
 
131
- // Connect them with relationships
132
- await brainy.addVerb(companyId, productId, undefined, { type: "develops" })
174
+ // 4️⃣ Advanced: Search with context
175
+ const contextual = await brainy.search("Who leads AI companies?", 5, {
176
+ includeVerbs: true, // Include relationships in results
177
+ nounTypes: ["person"], // Filter to specific entity types
178
+ })
133
179
 
134
- // Now you can do BOTH:
135
- const similar = await brainy.search("AI language models") // Vector similarity
136
- const products = await brainy.getVerbsByType("develops") // Graph traversal
180
+ // 5️⃣ NEW! MongoDB-style metadata filtering
181
+ const filtered = await brainy.search("AI research", 10, {
182
+ metadata: {
183
+ type: "academic",
184
+ year: { $gte: 2020 },
185
+ status: { $in: ["published", "peer-reviewed"] },
186
+ impact: { $gt: 100 }
187
+ }
188
+ })
189
+ // Filters DURING search for maximum performance!
137
190
  ```
138
191
 
139
- **Why this matters:** Find content by meaning AND follow relationships. It's like having PostgreSQL and Pinecone working
140
- together seamlessly.
192
+ **🎯 That's it!** Vector search + graph database + works everywhere. No config needed.
141
193
 
142
194
  ### 🔍 Want More Power?
143
195
 
@@ -162,335 +214,148 @@ returns "tiger"
162
214
  **🌐 Real-Time Collaboration** - Sync vector data across devices. Figma for AI data
163
215
  **🏥 Medical Diagnosis Tools** - Match symptoms to conditions using embedding similarity
164
216
 
165
- ## 🚀 Write-Once, Run-Anywhere Quick Start
217
+ ## 🚀 Works Everywhere - Same Code
166
218
 
167
- Brainy uses the same code across all environments with automatic detection. **Framework-optimized** for the best
168
- developer experience. Choose your environment:
219
+ **Write once, run anywhere.** Brainy auto-detects your environment and optimizes automatically:
169
220
 
170
- ### 🅰️ Angular (Latest)
221
+ ### 🌐 Browser Frameworks (React, Angular, Vue)
171
222
 
172
- ```bash
173
- npm install @soulcraft/brainy
223
+ ```javascript
224
+ import { BrainyData } from '@soulcraft/brainy'
225
+
226
+ // SAME CODE in React, Angular, Vue, Svelte, etc.
227
+ const brainy = new BrainyData()
228
+ await brainy.init() // Auto-uses OPFS in browsers
229
+
230
+ // Add entities and relationships
231
+ const john = await brainy.add("John is a software engineer", { type: "person" })
232
+ const jane = await brainy.add("Jane is a data scientist", { type: "person" })
233
+ const ai = await brainy.add("AI Project", { type: "project" })
234
+
235
+ await brainy.relate(john, ai, "works_on")
236
+ await brainy.relate(jane, ai, "leads")
237
+
238
+ // Search by meaning
239
+ const engineers = await brainy.search("software developers", 5)
240
+
241
+ // Traverse relationships
242
+ const team = await brainy.getVerbsByTarget(ai) // Who works on AI Project?
174
243
  ```
175
244
 
245
+ <details>
246
+ <summary>📦 Full Angular Component Example</summary>
247
+
176
248
  ```typescript
177
249
  import { Component, signal, OnInit } from '@angular/core'
178
250
  import { BrainyData } from '@soulcraft/brainy'
179
251
 
180
252
  @Component({
181
253
  selector: 'app-search',
182
- template: `
183
- <div class="search-container">
184
- <input [(ngModel)]="query"
185
- (input)="search($event.target.value)"
186
- placeholder="Search by meaning (try 'pets' or 'food')..."
187
- class="search-input">
188
-
189
- <div class="results">
190
- @for (result of results(); track result.id) {
191
- <div class="result-item">
192
- <strong>{{result.metadata?.category}}</strong>: {{result.metadata?.originalData}}
193
- <small>Similarity: {{result.score | number:'1.2-2'}}</small>
194
- </div>
195
- }
196
- </div>
197
- </div>
198
- `
254
+ template: `<input (input)="search($event.target.value)" placeholder="Search...">`
199
255
  })
200
256
  export class SearchComponent implements OnInit {
201
- private brainy: BrainyData | null = null
202
- results = signal<any[]>([])
203
- query = ''
257
+ brainy = new BrainyData()
204
258
 
205
259
  async ngOnInit() {
206
- // Auto-detects environment and uses OPFS storage in browsers
207
- this.brainy = new BrainyData({
208
- defaultService: 'my-app'
209
- })
210
260
  await this.brainy.init()
211
-
212
- // Add sample data
213
- await this.brainy.add("Cats are amazing pets", { category: "animals" })
214
- await this.brainy.add("Dogs love to play fetch", { category: "animals" })
215
- await this.brainy.add("Pizza is delicious food", { category: "food" })
261
+ // Add your data...
216
262
  }
217
263
 
218
264
  async search(query: string) {
219
- if (!query.trim() || !this.brainy) {
220
- this.results.set([])
221
- return
222
- }
223
-
224
- const searchResults = await this.brainy.search(query, 5)
225
- this.results.set(searchResults)
265
+ const results = await this.brainy.search(query, 5)
266
+ // Display results...
226
267
  }
227
268
  }
228
269
  ```
229
270
 
230
- ### ⚛️ React
271
+ </details>
231
272
 
232
- ```bash
233
- npm install @soulcraft/brainy
234
- ```
273
+ <details>
274
+ <summary>📦 Full React Example</summary>
235
275
 
236
276
  ```jsx
237
277
  import { BrainyData } from '@soulcraft/brainy'
238
278
  import { useEffect, useState } from 'react'
239
279
 
240
- function SemanticSearch() {
280
+ function Search() {
241
281
  const [brainy, setBrainy] = useState(null)
242
282
  const [results, setResults] = useState([])
243
- const [query, setQuery] = useState('')
244
- const [loading, setLoading] = useState(true)
245
283
 
246
284
  useEffect(() => {
247
- async function initBrainy() {
248
- // Auto-detects environment and uses OPFS storage in browsers
249
- const db = new BrainyData({
250
- defaultService: 'my-app'
251
- })
285
+ const init = async () => {
286
+ const db = new BrainyData()
252
287
  await db.init()
253
-
254
- // Add sample data
255
- await db.add("Cats are amazing pets", { category: "animals" })
256
- await db.add("Dogs love to play fetch", { category: "animals" })
257
- await db.add("Pizza is delicious food", { category: "food" })
258
-
288
+ // Add your data...
259
289
  setBrainy(db)
260
- setLoading(false)
261
290
  }
262
-
263
- initBrainy()
291
+ init()
264
292
  }, [])
265
293
 
266
- const search = async (searchQuery) => {
267
- if (!searchQuery.trim() || !brainy) return setResults([])
268
-
269
- const searchResults = await brainy.search(searchQuery, 5)
270
- setResults(searchResults)
294
+ const search = async (query) => {
295
+ const results = await brainy?.search(query, 5) || []
296
+ setResults(results)
271
297
  }
272
298
 
273
- if (loading) return <div>Initializing Brainy...</div>
274
-
275
- return (
276
- <div className="search-container">
277
- <input
278
- value={query}
279
- onChange={(e) => {
280
- setQuery(e.target.value)
281
- search(e.target.value)
282
- }}
283
- placeholder="Search by meaning (try 'pets' or 'food')..."
284
- className="search-input"
285
- />
286
-
287
- <div className="results">
288
- {results.map((result, i) => (
289
- <div key={result.id} className="result-item">
290
- <strong>{result.metadata?.category}</strong>: {result.metadata?.originalData}
291
- <small>Similarity: {result.score.toFixed(2)}</small>
292
- </div>
293
- ))}
294
- </div>
295
- </div>
296
- )
299
+ return <input onChange={(e) => search(e.target.value)} placeholder="Search..." />
297
300
  }
298
-
299
- export default SemanticSearch
300
301
  ```
301
302
 
302
- ### 🌟 Vue 3
303
+ </details>
303
304
 
304
- ```bash
305
- npm install @soulcraft/brainy
306
- ```
305
+ <details>
306
+ <summary>📦 Full Vue Example</summary>
307
307
 
308
308
  ```vue
309
309
 
310
- <template>
311
- <div class="search-container">
312
- <input
313
- v-model="query"
314
- @input="search"
315
- placeholder="Search by meaning (try 'pets' or 'food')..."
316
- class="search-input"
317
- />
318
-
319
- <div v-if="loading" class="loading">
320
- Initializing Brainy...
321
- </div>
322
-
323
- <div v-else class="results">
324
- <div
325
- v-for="result in results"
326
- :key="result.id"
327
- class="result-item"
328
- >
329
- <strong>{{ result.metadata?.category }}</strong>: {{ result.metadata?.originalData }}
330
- <small>Similarity: {{ result.score.toFixed(2) }}</small>
331
- </div>
332
- </div>
333
- </div>
334
- </template>
335
-
336
310
  <script setup>
337
311
  import { BrainyData } from '@soulcraft/brainy'
338
312
  import { ref, onMounted } from 'vue'
339
313
 
340
314
  const brainy = ref(null)
341
315
  const results = ref([])
342
- const query = ref('')
343
- const loading = ref(true)
344
316
 
345
317
  onMounted(async () => {
346
- // Auto-detects environment and uses OPFS storage in browsers
347
- const db = new BrainyData({
348
- defaultService: 'my-app'
349
- })
318
+ const db = new BrainyData()
350
319
  await db.init()
351
-
352
- // Add sample data
353
- await db.add("Cats are amazing pets", { category: "animals" })
354
- await db.add("Dogs love to play fetch", { category: "animals" })
355
- await db.add("Pizza is delicious food", { category: "food" })
356
-
320
+ // Add your data...
357
321
  brainy.value = db
358
- loading.value = false
359
322
  })
360
323
 
361
- const search = async () => {
362
- if (!query.value.trim() || !brainy.value) {
363
- results.value = []
364
- return
365
- }
366
-
367
- const searchResults = await brainy.value.search(query.value, 5)
368
- results.value = searchResults
324
+ const search = async (query) => {
325
+ const results = await brainy.value?.search(query, 5) || []
326
+ setResults(results)
369
327
  }
370
328
  </script>
371
329
 
372
- <style scoped>
373
- .search-container {
374
- max-width: 600px;
375
- margin: 0 auto;
376
- padding: 20px;
377
- }
378
-
379
- .search-input {
380
- width: 100%;
381
- padding: 12px;
382
- margin-bottom: 20px;
383
- border: 2px solid #ddd;
384
- border-radius: 8px;
385
- }
386
-
387
- .result-item {
388
- padding: 12px;
389
- border: 1px solid #eee;
390
- margin-bottom: 8px;
391
- border-radius: 6px;
392
- }
393
-
394
- .loading {
395
- text-align: center;
396
- color: #666;
397
- }
398
- </style>
399
- ```
400
-
401
- ### 🟢 Node.js Server
402
-
403
- ```bash
404
- npm install @soulcraft/brainy
330
+ <template>
331
+ <input @input="search($event.target.value)" placeholder="Search..." />
332
+ </template>
405
333
  ```
406
334
 
407
- ```javascript
408
- import { BrainyData } from '@soulcraft/brainy'
409
-
410
- // Auto-detects Node.js → FileSystem (local) or S3 (production), Worker threads
411
- const brainy = new BrainyData({
412
- defaultService: 'my-app',
413
- // Optional: Production S3 storage
414
- storage: {
415
- s3Storage: {
416
- bucketName: process.env.S3_BUCKET,
417
- region: process.env.AWS_REGION,
418
- accessKeyId: process.env.AWS_ACCESS_KEY_ID,
419
- secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
420
- }
421
- }
422
- })
423
- await brainy.init()
424
-
425
- // Same API everywhere
426
- await brainy.add("Cats are amazing pets", { category: "animals" })
427
- const results = await brainy.search("pets", 5)
428
- console.log('Search results:', results)
429
- ```
335
+ </details>
430
336
 
431
- ### Serverless (Vercel/Netlify)
337
+ ### 🟢 Node.js / Serverless / Edge
432
338
 
433
339
  ```javascript
434
340
  import { BrainyData } from '@soulcraft/brainy'
435
341
 
436
- export default async function handler(req, res) {
437
- // Auto-detects serverless S3/R2 storage for persistence, or Memory for temp
438
- const brainy = new BrainyData({
439
- defaultService: 'my-app',
440
- // Optional: Explicit S3-compatible storage
441
- storage: {
442
- r2Storage: {
443
- bucketName: process.env.R2_BUCKET,
444
- accessKeyId: process.env.R2_ACCESS_KEY_ID,
445
- secretAccessKey: process.env.R2_SECRET_ACCESS_KEY,
446
- accountId: process.env.R2_ACCOUNT_ID
447
- }
448
- }
449
- })
450
- await brainy.init()
451
-
452
- // Same API everywhere
453
- const results = await brainy.search(req.query.q, 5)
454
- res.json({ results })
455
- }
456
- ```
342
+ // SAME CODE works in Node.js, Vercel, Netlify, Cloudflare Workers, Deno, Bun
343
+ const brainy = new BrainyData()
344
+ await brainy.init() // Auto-detects environment and optimizes
457
345
 
458
- ### 🔥 Cloudflare Workers
346
+ // Add entities and relationships
347
+ await brainy.add("Python is great for data science", { type: "fact" })
348
+ await brainy.add("JavaScript rules the web", { type: "fact" })
459
349
 
460
- ```javascript
461
- import { BrainyData } from '@soulcraft/brainy'
350
+ // Search by meaning
351
+ const results = await brainy.search("programming languages", 5)
462
352
 
463
- export default {
464
- async fetch(request) {
465
- // Auto-detects edge → Minimal footprint, KV storage
466
- const brainy = new BrainyData({
467
- defaultService: 'edge-app'
468
- })
469
- await brainy.init()
470
-
471
- // Same API everywhere
472
- const url = new URL(request.url)
473
- const results = await brainy.search(url.searchParams.get('q'), 5)
474
- return Response.json({ results })
353
+ // Optional: Production with S3/R2 storage (auto-detected in cloud environments)
354
+ const productionBrainy = new BrainyData({
355
+ storage: {
356
+ s3Storage: { bucketName: process.env.BUCKET_NAME }
475
357
  }
476
- }
477
- ```
478
-
479
- ### 🦕 Deno
480
-
481
- ```typescript
482
- import { BrainyData } from 'https://esm.sh/@soulcraft/brainy'
483
-
484
- // Auto-detects Deno → Native compatibility, FileSystem storage
485
- const brainy = new BrainyData({
486
- defaultService: 'deno-app'
487
358
  })
488
- await brainy.init()
489
-
490
- // Same API everywhere
491
- await brainy.add("Deno is awesome", { category: "tech" })
492
- const results = await brainy.search("technology", 5)
493
- console.log(results)
494
359
  ```
495
360
 
496
361
  **That's it! Same code, everywhere. Zero-to-Smart™**
@@ -510,7 +375,7 @@ Brainy automatically detects and optimizes for:
510
375
 
511
376
  ```dockerfile
512
377
  # One line extracts models automatically during build
513
- RUN npm run extract-models
378
+ RUN npm run download-models
514
379
 
515
380
  # Deploy anywhere: Google Cloud, AWS, Azure, Cloudflare, etc.
516
381
  ```
@@ -577,6 +442,7 @@ console.log(`Instance ${health.instanceId}: ${health.status}`)
577
442
  ### Core Capabilities
578
443
 
579
444
  - **Vector Search** - Find semantically similar content using embeddings
445
+ - **MongoDB-Style Metadata Filtering** 🆕 - Advanced filtering with `$gt`, `$in`, `$regex`, `$and`, `$or` operators
580
446
  - **Graph Relationships** - Connect data with meaningful relationships
581
447
  - **JSON Document Search** - Search within specific fields with prioritization
582
448
  - **Distributed Mode** - Scale horizontally with automatic coordination between instances
@@ -684,6 +550,7 @@ npm install @soulcraft/brainy @soulcraft/brainy-models
684
550
  - **🎯 Zero Configuration** - Automatic detection with graceful fallback
685
551
  - **🔐 Enhanced Security** - Complete air-gapping support for sensitive environments
686
552
  - **🏢 Enterprise Ready** - Works behind corporate firewalls and restricted networks
553
+ - **⚖️ Compliance & Forensics** - Frozen mode for audit trails and legal discovery
687
554
 
688
555
  The offline models provide the **same functionality** with maximum reliability. Your existing code works unchanged -
689
556
  Brainy automatically detects and uses bundled models when available.
@@ -712,7 +579,7 @@ const brainy = createAutoBrainy({
712
579
  2. **Add to your Dockerfile:**
713
580
  ```dockerfile
714
581
  # Extract models during build (zero configuration!)
715
- RUN npm run extract-models
582
+ RUN npm run download-models
716
583
 
717
584
  # Include models in final image
718
585
  COPY --from=builder /app/models ./models
@@ -730,15 +597,15 @@ const brainy = createAutoBrainy({
730
597
  ### Universal Dockerfile Template
731
598
 
732
599
  ```dockerfile
733
- FROM node:24-alpine AS builder
600
+ FROM node:24-slim AS builder
734
601
  WORKDIR /app
735
602
  COPY package*.json ./
736
603
  RUN npm ci
737
604
  COPY . .
738
- RUN npm run extract-models # ← Automatic model extraction
605
+ RUN npm run download-models # ← Automatic model download
739
606
  RUN npm run build
740
607
 
741
- FROM node:24-alpine AS production
608
+ FROM node:24-slim AS production
742
609
  WORKDIR /app
743
610
  COPY package*.json ./
744
611
  RUN npm ci --only=production --omit=optional
@@ -876,7 +743,8 @@ const writer = createAutoBrainy({
876
743
 
877
744
  const reader = createAutoBrainy({
878
745
  storage: { s3Storage: { bucketName: 'my-bucket' } },
879
- readOnly: true, // Automatically becomes 'reader' role
746
+ readOnly: true, // Automatically becomes 'reader' role (allows optimizations)
747
+ // frozen: true, // Optional: Complete immutability for compliance/forensics
880
748
  distributed: true
881
749
  })
882
750
  ```
@@ -938,6 +806,25 @@ const health = brainy.getHealthStatus()
938
806
  - **Writers**: Optimized write batching, minimal cache
939
807
  - **Hybrid**: Adaptive based on workload
940
808
 
809
+ ## ⚖️ Compliance & Forensics Mode
810
+
811
+ For legal discovery, audit trails, and compliance requirements:
812
+
813
+ ```javascript
814
+ // Create a completely immutable snapshot
815
+ const auditDb = new BrainyData({
816
+ storage: { s3Storage: { bucketName: 'audit-snapshots' } },
817
+ readOnly: true,
818
+ frozen: true // Complete immutability - no changes allowed
819
+ })
820
+
821
+ // Perfect for:
822
+ // - Legal discovery (data cannot be modified)
823
+ // - Compliance audits (guaranteed state)
824
+ // - Forensic analysis (preserved evidence)
825
+ // - Regulatory snapshots (unchanging records)
826
+ ```
827
+
941
828
  ### Deployment Examples
942
829
 
943
830
  **Docker Compose**
@@ -1005,51 +892,41 @@ spec:
1005
892
 
1006
893
  ## 🚀 Getting Started in 30 Seconds
1007
894
 
1008
- ### React
895
+ **The same Brainy code works everywhere - React, Vue, Angular, Node.js, Serverless, Edge Workers.**
1009
896
 
1010
- ```jsx
1011
- import { createAutoBrainy } from 'brainy'
1012
- import { useEffect, useState } from 'react'
897
+ ```javascript
898
+ // This EXACT code works in ALL environments
899
+ import { BrainyData } from '@soulcraft/brainy'
1013
900
 
1014
- function SemanticSearch() {
1015
- const [brainy] = useState(() => createAutoBrainy())
1016
- const [results, setResults] = useState([])
901
+ const brainy = new BrainyData()
902
+ await brainy.init()
1017
903
 
1018
- const search = async (query) => {
1019
- const items = await brainy.searchText(query, 10)
1020
- setResults(items)
1021
- }
904
+ // Add nouns (entities)
905
+ const openai = await brainy.add("OpenAI", { type: "company" })
906
+ const gpt4 = await brainy.add("GPT-4", { type: "product" })
1022
907
 
1023
- return (
1024
- <input onChange={(e) => search(e.target.value)}
1025
- placeholder="Search by meaning..." />
1026
- )
1027
- }
908
+ // Add verbs (relationships)
909
+ await brainy.relate(openai, gpt4, "develops")
910
+
911
+ // Vector search + Graph traversal
912
+ const similar = await brainy.search("AI companies", 5)
913
+ const products = await brainy.getVerbsBySource(openai)
1028
914
  ```
1029
915
 
1030
- ### Angular
916
+ <details>
917
+ <summary>🔍 See Framework Examples</summary>
1031
918
 
1032
- ```typescript
1033
- import { Component, OnInit } from '@angular/core'
1034
- import { createAutoBrainy } from 'brainy'
919
+ ### React
1035
920
 
1036
- @Component({
1037
- selector: 'app-search',
1038
- template: `
1039
- <input (input)="search($event.target.value)"
1040
- placeholder="Semantic search...">
1041
- <div *ngFor="let result of results">
1042
- {{ result.text }}
1043
- </div>
1044
- `
1045
- })
1046
- export class SearchComponent implements OnInit {
1047
- brainy = createAutoBrainy()
1048
- results = []
921
+ ```jsx
922
+ function App() {
923
+ const [brainy] = useState(() => new BrainyData())
924
+ useEffect(() => brainy.init(), [])
1049
925
 
1050
- async search(query: string) {
1051
- this.results = await this.brainy.searchText(query, 10)
926
+ const search = async (query) => {
927
+ return await brainy.search(query, 10)
1052
928
  }
929
+ // Same API as above
1053
930
  }
1054
931
  ```
1055
932
 
@@ -1058,128 +935,66 @@ export class SearchComponent implements OnInit {
1058
935
  ```vue
1059
936
 
1060
937
  <script setup>
1061
- import { createAutoBrainy } from 'brainy'
1062
- import { ref } from 'vue'
1063
-
1064
- const brainy = createAutoBrainy()
1065
- const results = ref([])
1066
-
1067
- const search = async (query) => {
1068
- results.value = await brainy.searchText(query, 10)
1069
- }
938
+ const brainy = new BrainyData()
939
+ await brainy.init()
940
+ // Same API as above
1070
941
  </script>
1071
-
1072
- <template>
1073
- <input @input="search($event.target.value)"
1074
- placeholder="Find similar content...">
1075
- <div v-for="result in results" :key="result.id">
1076
- {{ result.text }}
1077
- </div>
1078
- </template>
1079
942
  ```
1080
943
 
1081
- ### Svelte
1082
-
1083
- ```svelte
1084
- <script>
1085
- import { createAutoBrainy } from 'brainy'
1086
-
1087
- const brainy = createAutoBrainy()
1088
- let results = []
1089
-
1090
- async function search(e) {
1091
- results = await brainy.searchText(e.target.value, 10)
1092
- }
1093
- </script>
1094
-
1095
- <input on:input={search} placeholder="AI-powered search...">
1096
- {#each results as result}
1097
- <div>{result.text}</div>
1098
- {/each}
1099
- ```
944
+ ### Angular
1100
945
 
1101
- ### Next.js (App Router)
946
+ ```typescript
1102
947
 
1103
- ```jsx
1104
- // app/search/page.js
1105
- import { createAutoBrainy } from 'brainy'
948
+ @Component({})
949
+ export class AppComponent {
950
+ brainy = new BrainyData()
1106
951
 
1107
- export default function SearchPage() {
1108
- async function search(formData) {
1109
- 'use server'
1110
- const brainy = createAutoBrainy({ bucketName: 'vectors' })
1111
- const query = formData.get('query')
1112
- return await brainy.searchText(query, 10)
952
+ async ngOnInit() {
953
+ await this.brainy.init()
954
+ // Same API as above
1113
955
  }
1114
-
1115
- return (
1116
- <form action={search}>
1117
- <input name="query" placeholder="Search..." />
1118
- <button type="submit">Search</button>
1119
- </form>
1120
- )
1121
956
  }
1122
957
  ```
1123
958
 
1124
- ### Node.js / Bun / Deno
959
+ ### Node.js / Deno / Bun
1125
960
 
1126
961
  ```javascript
1127
- import { createAutoBrainy } from 'brainy'
1128
-
1129
- const brainy = createAutoBrainy()
1130
-
1131
- // Add some data
1132
- await brainy.add("TypeScript is a typed superset of JavaScript", {
1133
- category: 'programming'
1134
- })
1135
-
1136
- // Search for similar content
1137
- const results = await brainy.searchText("JavaScript with types", 5)
1138
- console.log(results)
962
+ const brainy = new BrainyData()
963
+ await brainy.init()
964
+ // Same API as above
1139
965
  ```
1140
966
 
1141
- ### 🌍 Framework-First, Runs Everywhere
1142
-
1143
- **Brainy is designed for modern frameworks** with automatic environment detection and storage selection:
1144
-
1145
- **✨ Supported environments:**
967
+ </details>
1146
968
 
1147
- - ⚛️ **React/Vue/Angular** - Framework-optimized builds with proper bundling
1148
- - 🟢 **Node.js/Deno/Bun** - Full server-side capabilities
1149
- - ⚡ **Serverless/Edge** - Optimized for cold starts and minimal footprint
1150
- - 🧵 **Web/Worker threads** - Thread-safe, shared storage
1151
-
1152
- **🗄️ Auto-selected storage:**
969
+ ### 🌍 Framework-First, Runs Everywhere
1153
970
 
1154
- - 🌐 **OPFS** - Browser frameworks (persistent, fast)
1155
- - 📁 **FileSystem** - Node.js servers (local development)
1156
- - ☁️ **S3/R2/GCS** - Production, serverless, distributed deployments
1157
- - 💾 **Memory** - Edge workers, testing, temporary data
971
+ **Brainy automatically detects your environment and optimizes everything:**
1158
972
 
1159
- **🚀 Framework benefits:**
973
+ | Environment | Storage | Optimization |
974
+ |-----------------|-----------------|----------------------------|
975
+ | 🌐 Browser | OPFS | Web Workers, Memory Cache |
976
+ | 🟢 Node.js | FileSystem / S3 | Worker Threads, Clustering |
977
+ | ⚡ Serverless | S3 / Memory | Cold Start Optimization |
978
+ | 🔥 Edge Workers | Memory / KV | Minimal Footprint |
979
+ | 🦕 Deno/Bun | FileSystem / S3 | Native Performance |
1160
980
 
1161
- - **Proper bundling** - Handles dynamic imports and dependencies correctly
1162
- - ✅ **Type safety** - Full TypeScript integration and IntelliSense
1163
- - ✅ **State management** - Reactive updates and component lifecycle
1164
- - ✅ **Production ready** - Tree-shaking, optimization, error boundaries
981
+ ## 🌐 Deploy to Any Cloud
1165
982
 
1166
- **Note:** We focus on framework support for reliability. Vanilla JS had too many module resolution issues.
983
+ <details>
984
+ <summary>☁️ See Cloud Platform Examples</summary>
1167
985
 
1168
986
  ### Cloudflare Workers
1169
987
 
1170
988
  ```javascript
1171
- import { createAutoBrainy } from 'brainy'
989
+ import { BrainyData } from '@soulcraft/brainy'
1172
990
 
1173
991
  export default {
1174
- async fetch(request, env) {
1175
- const brainy = createAutoBrainy({
1176
- bucketName: env.R2_BUCKET
1177
- })
992
+ async fetch(request) {
993
+ const brainy = new BrainyData()
994
+ await brainy.init()
1178
995
 
1179
996
  const url = new URL(request.url)
1180
- const query = url.searchParams.get('q')
1181
-
1182
- const results = await brainy.searchText(query, 10)
997
+ const results = await brainy.search(url.searchParams.get('q'), 10)
1183
998
  return Response.json(results)
1184
999
  }
1185
1000
  }
@@ -1188,156 +1003,54 @@ export default {
1188
1003
  ### AWS Lambda
1189
1004
 
1190
1005
  ```javascript
1191
- import { createAutoBrainy } from 'brainy'
1006
+ import { BrainyData } from '@soulcraft/brainy'
1192
1007
 
1193
1008
  export const handler = async (event) => {
1194
- const brainy = createAutoBrainy({
1195
- bucketName: process.env.S3_BUCKET
1196
- })
1197
-
1198
- const results = await brainy.searchText(event.query, 10)
1199
-
1200
- return {
1201
- statusCode: 200,
1202
- body: JSON.stringify(results)
1203
- }
1204
- }
1205
- ```
1206
-
1207
- ### Azure Functions
1208
-
1209
- ```javascript
1210
- import { createAutoBrainy } from 'brainy'
1211
-
1212
- module.exports = async function(context, req) {
1213
- const brainy = createAutoBrainy({
1214
- bucketName: process.env.AZURE_STORAGE_CONTAINER
1215
- })
1216
-
1217
- const results = await brainy.searchText(req.query.q, 10)
1009
+ const brainy = new BrainyData()
1010
+ await brainy.init()
1218
1011
 
1219
- context.res = {
1220
- body: results
1221
- }
1012
+ const results = await brainy.search(event.query, 10)
1013
+ return { statusCode: 200, body: JSON.stringify(results) }
1222
1014
  }
1223
1015
  ```
1224
1016
 
1225
1017
  ### Google Cloud Functions
1226
1018
 
1227
1019
  ```javascript
1228
- import { createAutoBrainy } from 'brainy'
1020
+ import { BrainyData } from '@soulcraft/brainy'
1229
1021
 
1230
1022
  export const searchHandler = async (req, res) => {
1231
- const brainy = createAutoBrainy({
1232
- bucketName: process.env.GCS_BUCKET
1233
- })
1023
+ const brainy = new BrainyData()
1024
+ await brainy.init()
1234
1025
 
1235
- const results = await brainy.searchText(req.query.q, 10)
1026
+ const results = await brainy.search(req.query.q, 10)
1236
1027
  res.json(results)
1237
1028
  }
1238
1029
  ```
1239
1030
 
1240
- ### Google Cloud Run
1241
-
1242
- ```dockerfile
1243
- # Dockerfile
1244
- FROM node:20-alpine
1245
- USER node
1246
- WORKDIR /app
1247
- COPY package*.json ./
1248
- RUN npm install brainy
1249
- COPY . .
1250
- CMD ["node", "server.js"]
1251
- ```
1252
-
1253
- ```javascript
1254
- // server.js
1255
- import { createAutoBrainy } from 'brainy'
1256
- import express from 'express'
1257
-
1258
- const app = express()
1259
- const brainy = createAutoBrainy({
1260
- bucketName: process.env.GCS_BUCKET
1261
- })
1262
-
1263
- app.get('/search', async (req, res) => {
1264
- const results = await brainy.searchText(req.query.q, 10)
1265
- res.json(results)
1266
- })
1267
-
1268
- const port = process.env.PORT || 8080
1269
- app.listen(port, () => console.log(`Brainy on Cloud Run: ${port}`))
1270
- ```
1271
-
1272
- ```bash
1273
- # Deploy to Cloud Run
1274
- gcloud run deploy brainy-api \
1275
- --source . \
1276
- --platform managed \
1277
- --region us-central1 \
1278
- --allow-unauthenticated
1279
- ```
1280
-
1281
1031
  ### Vercel Edge Functions
1282
1032
 
1283
1033
  ```javascript
1284
- import { createAutoBrainy } from 'brainy'
1034
+ import { BrainyData } from '@soulcraft/brainy'
1285
1035
 
1286
- export const config = {
1287
- runtime: 'edge'
1288
- }
1036
+ export const config = { runtime: 'edge' }
1289
1037
 
1290
1038
  export default async function handler(request) {
1291
- const brainy = createAutoBrainy()
1292
- const { searchParams } = new URL(request.url)
1293
- const query = searchParams.get('q')
1039
+ const brainy = new BrainyData()
1040
+ await brainy.init()
1294
1041
 
1295
- const results = await brainy.searchText(query, 10)
1042
+ const { searchParams } = new URL(request.url)
1043
+ const results = await brainy.search(searchParams.get('q'), 10)
1296
1044
  return Response.json(results)
1297
1045
  }
1298
1046
  ```
1299
1047
 
1300
- ### Netlify Functions
1301
-
1302
- ```javascript
1303
- import { createAutoBrainy } from 'brainy'
1304
-
1305
- export async function handler(event, context) {
1306
- const brainy = createAutoBrainy()
1307
- const query = event.queryStringParameters.q
1308
-
1309
- const results = await brainy.searchText(query, 10)
1310
-
1311
- return {
1312
- statusCode: 200,
1313
- body: JSON.stringify(results)
1314
- }
1315
- }
1316
- ```
1317
-
1318
- ### Supabase Edge Functions
1319
-
1320
- ```typescript
1321
- import { createAutoBrainy } from 'brainy'
1322
- import { serve } from 'https://deno.land/std@0.168.0/http/server.ts'
1323
-
1324
- serve(async (req) => {
1325
- const brainy = createAutoBrainy()
1326
- const url = new URL(req.url)
1327
- const query = url.searchParams.get('q')
1328
-
1329
- const results = await brainy.searchText(query, 10)
1330
-
1331
- return new Response(JSON.stringify(results), {
1332
- headers: { 'Content-Type': 'application/json' }
1333
- })
1334
- })
1335
- ```
1048
+ </details>
1336
1049
 
1337
1050
  ### Docker Container
1338
1051
 
1339
1052
  ```dockerfile
1340
- FROM node:20-alpine
1053
+ FROM node:24-slim
1341
1054
  USER node
1342
1055
  WORKDIR /app
1343
1056
  COPY package*.json ./
@@ -1420,6 +1133,7 @@ services:
1420
1133
 
1421
1134
  - [**Search and Metadata**](docs/user-guides/) - Advanced search techniques
1422
1135
  - [**JSON Document Search**](docs/guides/json-document-search.md) - Field-based searching
1136
+ - [**Read-Only & Frozen Modes**](docs/guides/readonly-frozen-modes.md) - Immutability options for production
1423
1137
  - [**Production Migration**](docs/guides/production-migration-guide.md) - Deployment best practices
1424
1138
 
1425
1139
  ### API Reference