@soulcraft/brainy 5.7.13 → 5.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +13 -7
- package/README.md +6 -2
- package/dist/brainy.d.ts +1 -0
- package/dist/brainy.js +137 -126
- package/dist/graph/graphAdjacencyIndex.d.ts +76 -7
- package/dist/graph/graphAdjacencyIndex.js +94 -9
- package/dist/hnsw/typeAwareHNSWIndex.d.ts +3 -3
- package/dist/hnsw/typeAwareHNSWIndex.js +3 -3
- package/dist/query/typeAwareQueryPlanner.d.ts +4 -4
- package/dist/query/typeAwareQueryPlanner.js +4 -4
- package/dist/transaction/Transaction.d.ts +55 -0
- package/dist/transaction/Transaction.js +175 -0
- package/dist/transaction/TransactionManager.d.ts +67 -0
- package/dist/transaction/TransactionManager.js +145 -0
- package/dist/transaction/errors.d.ts +41 -0
- package/dist/transaction/errors.js +66 -0
- package/dist/transaction/index.d.ts +14 -0
- package/dist/transaction/index.js +14 -0
- package/dist/transaction/operations/IndexOperations.d.ts +172 -0
- package/dist/transaction/operations/IndexOperations.js +301 -0
- package/dist/transaction/operations/StorageOperations.d.ts +128 -0
- package/dist/transaction/operations/StorageOperations.js +253 -0
- package/dist/transaction/operations/index.d.ts +10 -0
- package/dist/transaction/operations/index.js +13 -0
- package/dist/transaction/types.d.ts +84 -0
- package/dist/transaction/types.js +8 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
|
|
4
4
|
|
|
5
|
+
### [5.8.0](https://github.com/soulcraftlabs/brainy/compare/v5.7.13...v5.8.0) (2025-11-14)
|
|
6
|
+
|
|
7
|
+
- feat: add v5.8.0 features - transactions, pagination, and comprehensive docs (e40fee3)
|
|
8
|
+
- docs: label all performance claims as MEASURED vs PROJECTED (NO FAKE CODE compliance) (52e9617)
|
|
9
|
+
|
|
10
|
+
|
|
5
11
|
### [5.7.13](https://github.com/soulcraftlabs/brainy/compare/v5.7.12...v5.7.13) (2025-11-14)
|
|
6
12
|
|
|
7
13
|
|
|
@@ -1666,11 +1672,11 @@ After upgrading to v3.50.2:
|
|
|
1666
1672
|
|
|
1667
1673
|
### ✨ Features
|
|
1668
1674
|
|
|
1669
|
-
**Phase 2: Type-Aware HNSW - 87% Memory Reduction @ Billion Scale**
|
|
1675
|
+
**Phase 2: Type-Aware HNSW - PROJECTED 87% Memory Reduction @ Billion Scale**
|
|
1670
1676
|
|
|
1671
1677
|
- **feat**: TypeAwareHNSWIndex with separate HNSW graphs per entity type
|
|
1672
|
-
- **87% HNSW memory reduction**: 384GB → 50GB (-334GB) @ 1B scale
|
|
1673
|
-
- **10x faster single-type queries**: search 100M nodes instead of 1B
|
|
1678
|
+
- **PROJECTED 87% HNSW memory reduction**: 384GB → 50GB (-334GB) @ 1B scale (calculated from architectural analysis, not yet benchmarked at billion scale)
|
|
1679
|
+
- **PROJECTED 10x faster single-type queries**: search 100M nodes instead of 1B (not yet benchmarked)
|
|
1674
1680
|
- **5-8x faster multi-type queries**: search subset of types
|
|
1675
1681
|
- **~3x faster all-types queries**: 31 smaller graphs vs 1 large graph
|
|
1676
1682
|
- Lazy initialization - only creates indexes for types with entities
|
|
@@ -1688,11 +1694,11 @@ After upgrading to v3.50.2:
|
|
|
1688
1694
|
- Maintains O(log n) performance guarantees
|
|
1689
1695
|
- Zero API changes for existing code
|
|
1690
1696
|
|
|
1691
|
-
### 📊 Impact @ Billion Scale
|
|
1697
|
+
### 📊 Impact @ Billion Scale (PROJECTED)
|
|
1692
1698
|
|
|
1693
|
-
**Memory Reduction (Phase 2):**
|
|
1699
|
+
**Memory Reduction (Phase 2) - PROJECTED:**
|
|
1694
1700
|
```
|
|
1695
|
-
HNSW memory: 384GB → 50GB (-87% / -334GB)
|
|
1701
|
+
HNSW memory: 384GB → 50GB (-87% / -334GB) - PROJECTED from architectural analysis, not benchmarked at 1B scale
|
|
1696
1702
|
```
|
|
1697
1703
|
|
|
1698
1704
|
**Query Performance:**
|
|
@@ -1758,7 +1764,7 @@ Part of the billion-scale optimization roadmap:
|
|
|
1758
1764
|
### 🎯 Next Steps
|
|
1759
1765
|
|
|
1760
1766
|
**Phase 3** (planned): Type-First Query Optimization
|
|
1761
|
-
- Query: 40% latency reduction via type-aware planning
|
|
1767
|
+
- Query: PROJECTED 40% latency reduction via type-aware planning (not yet benchmarked)
|
|
1762
1768
|
- Index: Smart query routing based on type cardinality
|
|
1763
1769
|
- Estimated: 2 weeks implementation
|
|
1764
1770
|
|
package/README.md
CHANGED
|
@@ -631,16 +631,20 @@ This comprehensive guide includes:
|
|
|
631
631
|
- Your primary resource for building with Brainy
|
|
632
632
|
- Every method documented with working examples
|
|
633
633
|
|
|
634
|
-
2. **[
|
|
634
|
+
2. **[Filter & Query Syntax Guide](docs/FIND_SYSTEM.md)**
|
|
635
|
+
- Complete reference for operators, compound filters, and optimization tips
|
|
636
|
+
|
|
637
|
+
3. **[Natural Language Queries](docs/guides/natural-language.md)**
|
|
635
638
|
- Master the `find()` method and Triple Intelligence queries
|
|
636
639
|
|
|
637
|
-
|
|
640
|
+
4. **[v4.0.0 Migration Guide](docs/MIGRATION-V3-TO-V4.md)**
|
|
638
641
|
- Upgrading from v3 (100% backward compatible)
|
|
639
642
|
|
|
640
643
|
### 🧠 Core Concepts & Architecture
|
|
641
644
|
|
|
642
645
|
- **[Triple Intelligence Architecture](docs/architecture/triple-intelligence.md)** — How vector + graph + document work together
|
|
643
646
|
- **[Noun-Verb Taxonomy](docs/architecture/noun-verb-taxonomy.md)** — The universal type system (42 nouns × 127 verbs)
|
|
647
|
+
- **[Transactions](docs/transactions.md)** — Atomic operations with automatic rollback
|
|
644
648
|
- **[Architecture Overview](docs/architecture/overview.md)** — System design and components
|
|
645
649
|
- **[Data Storage Architecture](docs/architecture/data-storage-architecture.md)** — Type-aware indexing and HNSW
|
|
646
650
|
|
package/dist/brainy.d.ts
CHANGED
package/dist/brainy.js
CHANGED
|
@@ -6,6 +6,7 @@
|
|
|
6
6
|
*/
|
|
7
7
|
import { v4 as uuidv4 } from './universal/uuid.js';
|
|
8
8
|
import { HNSWIndex } from './hnsw/hnswIndex.js';
|
|
9
|
+
import { HNSWIndexOptimized } from './hnsw/hnswIndexOptimized.js';
|
|
9
10
|
import { TypeAwareHNSWIndex } from './hnsw/typeAwareHNSWIndex.js';
|
|
10
11
|
import { createStorage } from './storage/storageFactory.js';
|
|
11
12
|
import { defaultEmbeddingFunction, cosineDistance } from './utils/index.js';
|
|
@@ -23,6 +24,8 @@ import { CommitBuilder } from './storage/cow/CommitObject.js';
|
|
|
23
24
|
import { NULL_HASH } from './storage/cow/constants.js';
|
|
24
25
|
import { createPipeline } from './streaming/pipeline.js';
|
|
25
26
|
import { configureLogger, LogLevel } from './utils/logger.js';
|
|
27
|
+
import { TransactionManager } from './transaction/TransactionManager.js';
|
|
28
|
+
import { SaveNounMetadataOperation, SaveNounOperation, AddToTypeAwareHNSWOperation, AddToHNSWOperation, AddToMetadataIndexOperation, SaveVerbMetadataOperation, SaveVerbOperation, AddToGraphIndexOperation, RemoveFromHNSWOperation, RemoveFromTypeAwareHNSWOperation, RemoveFromMetadataIndexOperation, RemoveFromGraphIndexOperation, UpdateNounMetadataOperation, DeleteNounMetadataOperation, DeleteVerbMetadataOperation } from './transaction/operations/index.js';
|
|
26
29
|
import { DistributedCoordinator, ShardManager, CacheSync, ReadWriteSeparation } from './distributed/index.js';
|
|
27
30
|
import { NounType } from './types/graphTypes.js';
|
|
28
31
|
/**
|
|
@@ -46,6 +49,7 @@ export class Brainy {
|
|
|
46
49
|
this.distance = cosineDistance;
|
|
47
50
|
this.embedder = this.setupEmbedder();
|
|
48
51
|
this.augmentationRegistry = this.setupAugmentations();
|
|
52
|
+
this.transactionManager = new TransactionManager();
|
|
49
53
|
// Setup distributed components if enabled
|
|
50
54
|
if (this.config.distributed?.enabled) {
|
|
51
55
|
this.setupDistributedComponents();
|
|
@@ -327,24 +331,6 @@ export class Brainy {
|
|
|
327
331
|
...(params.weight !== undefined && { weight: params.weight }),
|
|
328
332
|
...(params.createdBy && { createdBy: params.createdBy })
|
|
329
333
|
};
|
|
330
|
-
// v5.0.1: Save metadata FIRST so TypeAwareStorage can cache the type
|
|
331
|
-
// This prevents the race condition where saveNoun() defaults to 'thing'
|
|
332
|
-
await this.storage.saveNounMetadata(id, storageMetadata);
|
|
333
|
-
// Then save vector
|
|
334
|
-
await this.storage.saveNoun({
|
|
335
|
-
id,
|
|
336
|
-
vector,
|
|
337
|
-
connections: new Map(),
|
|
338
|
-
level: 0
|
|
339
|
-
});
|
|
340
|
-
// v5.4.0: Add to HNSW index AFTER entity is saved (fixes race condition)
|
|
341
|
-
// CRITICAL: Entity must exist in storage before HNSW tries to persist
|
|
342
|
-
if (this.index instanceof TypeAwareHNSWIndex) {
|
|
343
|
-
await this.index.addItem({ id, vector }, params.type);
|
|
344
|
-
}
|
|
345
|
-
else {
|
|
346
|
-
await this.index.addItem({ id, vector });
|
|
347
|
-
}
|
|
348
334
|
// v4.8.0: Build entity structure for indexing (NEW - with top-level fields)
|
|
349
335
|
const entityForIndexing = {
|
|
350
336
|
id,
|
|
@@ -362,8 +348,28 @@ export class Brainy {
|
|
|
362
348
|
// Only custom fields in metadata
|
|
363
349
|
metadata: params.metadata || {}
|
|
364
350
|
};
|
|
365
|
-
//
|
|
366
|
-
|
|
351
|
+
// v5.8.0: Execute atomically with transaction system
|
|
352
|
+
// All operations succeed or all rollback - prevents partial failures
|
|
353
|
+
await this.transactionManager.executeTransaction(async (tx) => {
|
|
354
|
+
// Operation 1: Save metadata FIRST (v5.0.1 - TypeAwareStorage caching)
|
|
355
|
+
tx.addOperation(new SaveNounMetadataOperation(this.storage, id, storageMetadata));
|
|
356
|
+
// Operation 2: Save vector data
|
|
357
|
+
tx.addOperation(new SaveNounOperation(this.storage, {
|
|
358
|
+
id,
|
|
359
|
+
vector,
|
|
360
|
+
connections: new Map(),
|
|
361
|
+
level: 0
|
|
362
|
+
}));
|
|
363
|
+
// Operation 3: Add to HNSW index (v5.4.0 - after entity saved)
|
|
364
|
+
if (this.index instanceof TypeAwareHNSWIndex) {
|
|
365
|
+
tx.addOperation(new AddToTypeAwareHNSWOperation(this.index, id, vector, params.type));
|
|
366
|
+
}
|
|
367
|
+
else {
|
|
368
|
+
tx.addOperation(new AddToHNSWOperation(this.index, id, vector));
|
|
369
|
+
}
|
|
370
|
+
// Operation 4: Add to metadata index
|
|
371
|
+
tx.addOperation(new AddToMetadataIndexOperation(this.metadataIndex, id, entityForIndexing));
|
|
372
|
+
});
|
|
367
373
|
return id;
|
|
368
374
|
});
|
|
369
375
|
}
|
|
@@ -557,32 +563,6 @@ export class Brainy {
|
|
|
557
563
|
...(params.confidence === undefined && existing.confidence !== undefined && { confidence: existing.confidence }),
|
|
558
564
|
...(params.weight === undefined && existing.weight !== undefined && { weight: existing.weight })
|
|
559
565
|
};
|
|
560
|
-
// v4.0.0: Save metadata FIRST (v5.1.0 fix: updates type cache for TypeAwareStorage)
|
|
561
|
-
// v5.1.0: saveNounMetadata must be called before saveNoun so that the type cache
|
|
562
|
-
// is updated before determining the shard path. Otherwise type changes cause
|
|
563
|
-
// entities to be saved in the wrong shard and become unfindable.
|
|
564
|
-
await this.storage.saveNounMetadata(params.id, updatedMetadata);
|
|
565
|
-
// Then save vector (will use updated type cache)
|
|
566
|
-
await this.storage.saveNoun({
|
|
567
|
-
id: params.id,
|
|
568
|
-
vector,
|
|
569
|
-
connections: new Map(),
|
|
570
|
-
level: 0
|
|
571
|
-
});
|
|
572
|
-
// v5.4.0: Update HNSW index AFTER entity is saved (fixes race condition)
|
|
573
|
-
// CRITICAL: Entity must be fully updated in storage before HNSW tries to persist
|
|
574
|
-
if (needsReindexing) {
|
|
575
|
-
// Update in index (remove and re-add since no update method)
|
|
576
|
-
// Phase 2: pass type for TypeAwareHNSWIndex
|
|
577
|
-
if (this.index instanceof TypeAwareHNSWIndex) {
|
|
578
|
-
await this.index.removeItem(params.id, existing.type);
|
|
579
|
-
await this.index.addItem({ id: params.id, vector }, newType); // v5.1.0: use new type
|
|
580
|
-
}
|
|
581
|
-
else {
|
|
582
|
-
await this.index.removeItem(params.id);
|
|
583
|
-
await this.index.addItem({ id: params.id, vector });
|
|
584
|
-
}
|
|
585
|
-
}
|
|
586
566
|
// v4.8.0: Build entity structure for metadata index (with top-level fields)
|
|
587
567
|
const entityForIndexing = {
|
|
588
568
|
id: params.id,
|
|
@@ -600,9 +580,32 @@ export class Brainy {
|
|
|
600
580
|
// Only custom fields in metadata
|
|
601
581
|
metadata: newMetadata
|
|
602
582
|
};
|
|
603
|
-
//
|
|
604
|
-
await this.
|
|
605
|
-
|
|
583
|
+
// v5.8.0: Execute atomically with transaction system
|
|
584
|
+
await this.transactionManager.executeTransaction(async (tx) => {
|
|
585
|
+
// Operation 1: Update metadata FIRST (v5.1.0 - updates type cache)
|
|
586
|
+
tx.addOperation(new UpdateNounMetadataOperation(this.storage, params.id, updatedMetadata));
|
|
587
|
+
// Operation 2: Update vector data (will use updated type cache)
|
|
588
|
+
tx.addOperation(new SaveNounOperation(this.storage, {
|
|
589
|
+
id: params.id,
|
|
590
|
+
vector,
|
|
591
|
+
connections: new Map(),
|
|
592
|
+
level: 0
|
|
593
|
+
}));
|
|
594
|
+
// Operation 3-4: Update HNSW index (remove and re-add if reindexing needed)
|
|
595
|
+
if (needsReindexing) {
|
|
596
|
+
if (this.index instanceof TypeAwareHNSWIndex) {
|
|
597
|
+
tx.addOperation(new RemoveFromTypeAwareHNSWOperation(this.index, params.id, existing.vector, existing.type));
|
|
598
|
+
tx.addOperation(new AddToTypeAwareHNSWOperation(this.index, params.id, vector, newType));
|
|
599
|
+
}
|
|
600
|
+
else {
|
|
601
|
+
tx.addOperation(new RemoveFromHNSWOperation(this.index, params.id, existing.vector));
|
|
602
|
+
tx.addOperation(new AddToHNSWOperation(this.index, params.id, vector));
|
|
603
|
+
}
|
|
604
|
+
}
|
|
605
|
+
// Operation 5-6: Update metadata index (remove old, add new)
|
|
606
|
+
tx.addOperation(new RemoveFromMetadataIndexOperation(this.metadataIndex, params.id, existing.metadata));
|
|
607
|
+
tx.addOperation(new AddToMetadataIndexOperation(this.metadataIndex, params.id, entityForIndexing));
|
|
608
|
+
});
|
|
606
609
|
});
|
|
607
610
|
}
|
|
608
611
|
/**
|
|
@@ -615,47 +618,37 @@ export class Brainy {
|
|
|
615
618
|
}
|
|
616
619
|
await this.ensureInitialized();
|
|
617
620
|
return this.augmentationRegistry.execute('delete', { id }, async () => {
|
|
618
|
-
//
|
|
619
|
-
|
|
620
|
-
|
|
621
|
-
const metadata = await this.storage.getNounMetadata(id);
|
|
622
|
-
if (metadata && metadata.noun) {
|
|
623
|
-
await this.index.removeItem(id, metadata.noun);
|
|
624
|
-
}
|
|
625
|
-
}
|
|
626
|
-
else {
|
|
627
|
-
await this.index.removeItem(id);
|
|
628
|
-
}
|
|
629
|
-
// Remove from metadata index
|
|
630
|
-
await this.metadataIndex.removeFromIndex(id);
|
|
631
|
-
// Delete from storage
|
|
632
|
-
await this.storage.deleteNoun(id);
|
|
633
|
-
// Delete metadata (if it exists as separate)
|
|
634
|
-
try {
|
|
635
|
-
await this.storage.saveMetadata(id, null); // Clear metadata
|
|
636
|
-
}
|
|
637
|
-
catch {
|
|
638
|
-
// Ignore if not supported
|
|
639
|
-
}
|
|
640
|
-
// Delete related verbs
|
|
621
|
+
// Get entity metadata and related verbs before deletion
|
|
622
|
+
const metadata = await this.storage.getNounMetadata(id);
|
|
623
|
+
const noun = await this.storage.getNoun(id);
|
|
641
624
|
const verbs = await this.storage.getVerbsBySource(id);
|
|
642
625
|
const targetVerbs = await this.storage.getVerbsByTarget(id);
|
|
643
626
|
const allVerbs = [...verbs, ...targetVerbs];
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
if (
|
|
652
|
-
|
|
627
|
+
// v5.8.0: Execute atomically with transaction system
|
|
628
|
+
await this.transactionManager.executeTransaction(async (tx) => {
|
|
629
|
+
// Operation 1: Remove from vector index
|
|
630
|
+
if (noun && metadata) {
|
|
631
|
+
if (this.index instanceof TypeAwareHNSWIndex && metadata.noun) {
|
|
632
|
+
tx.addOperation(new RemoveFromTypeAwareHNSWOperation(this.index, id, noun.vector, metadata.noun));
|
|
633
|
+
}
|
|
634
|
+
else if (this.index instanceof HNSWIndex || this.index instanceof HNSWIndexOptimized) {
|
|
635
|
+
tx.addOperation(new RemoveFromHNSWOperation(this.index, id, noun.vector));
|
|
653
636
|
}
|
|
654
637
|
}
|
|
655
|
-
|
|
656
|
-
|
|
638
|
+
// Operation 2: Remove from metadata index
|
|
639
|
+
if (metadata) {
|
|
640
|
+
tx.addOperation(new RemoveFromMetadataIndexOperation(this.metadataIndex, id, metadata));
|
|
657
641
|
}
|
|
658
|
-
|
|
642
|
+
// Operation 3: Delete noun metadata
|
|
643
|
+
tx.addOperation(new DeleteNounMetadataOperation(this.storage, id));
|
|
644
|
+
// Operations 4+: Delete all related verbs atomically
|
|
645
|
+
for (const verb of allVerbs) {
|
|
646
|
+
// Remove from graph index
|
|
647
|
+
tx.addOperation(new RemoveFromGraphIndexOperation(this.graphIndex, verb));
|
|
648
|
+
// Delete verb metadata
|
|
649
|
+
tx.addOperation(new DeleteVerbMetadataOperation(this.storage, verb.id));
|
|
650
|
+
}
|
|
651
|
+
});
|
|
659
652
|
});
|
|
660
653
|
}
|
|
661
654
|
// ============= RELATIONSHIP OPERATIONS =============
|
|
@@ -780,14 +773,18 @@ export class Brainy {
|
|
|
780
773
|
// CRITICAL FIX (v3.43.2): Check for duplicate relationships
|
|
781
774
|
// This prevents infinite loops where same relationship is created repeatedly
|
|
782
775
|
// Bug #1 showed incrementing verb counts (7→8→9...) indicating duplicates
|
|
783
|
-
|
|
784
|
-
const
|
|
785
|
-
|
|
786
|
-
|
|
787
|
-
|
|
788
|
-
|
|
789
|
-
|
|
776
|
+
// v5.8.0 OPTIMIZATION: Use GraphAdjacencyIndex for O(log n) lookup instead of O(n) storage scan
|
|
777
|
+
const verbIds = await this.graphIndex.getVerbIdsBySource(params.from);
|
|
778
|
+
// Check each verb ID for matching relationship (only load verbs we need to check)
|
|
779
|
+
for (const verbId of verbIds) {
|
|
780
|
+
const verb = await this.graphIndex.getVerbCached(verbId);
|
|
781
|
+
if (verb && verb.targetId === params.to && verb.verb === params.type) {
|
|
782
|
+
// Relationship already exists - return existing ID instead of creating duplicate
|
|
783
|
+
console.log(`[DEBUG] Skipping duplicate relationship: ${params.from} → ${params.to} (${params.type})`);
|
|
784
|
+
return verb.id;
|
|
785
|
+
}
|
|
790
786
|
}
|
|
787
|
+
// No duplicate found - proceed with creation
|
|
791
788
|
// Generate ID
|
|
792
789
|
const id = uuidv4();
|
|
793
790
|
// Compute relationship vector (average of entities)
|
|
@@ -815,40 +812,47 @@ export class Brainy {
|
|
|
815
812
|
metadata: params.metadata,
|
|
816
813
|
createdAt: Date.now()
|
|
817
814
|
};
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
822
|
-
|
|
823
|
-
sourceId: params.from,
|
|
824
|
-
targetId: params.to
|
|
825
|
-
});
|
|
826
|
-
await this.storage.saveVerbMetadata(id, verbMetadata);
|
|
827
|
-
// Add to graph index for O(1) lookups
|
|
828
|
-
await this.graphIndex.addVerb(verb);
|
|
829
|
-
// Create bidirectional if requested
|
|
830
|
-
if (params.bidirectional) {
|
|
831
|
-
const reverseId = uuidv4();
|
|
832
|
-
const reverseVerb = {
|
|
833
|
-
...verb,
|
|
834
|
-
id: reverseId,
|
|
835
|
-
sourceId: params.to,
|
|
836
|
-
targetId: params.from,
|
|
837
|
-
source: toEntity.type,
|
|
838
|
-
target: fromEntity.type
|
|
839
|
-
};
|
|
840
|
-
await this.storage.saveVerb({
|
|
841
|
-
id: reverseId,
|
|
815
|
+
// v5.8.0: Execute atomically with transaction system
|
|
816
|
+
await this.transactionManager.executeTransaction(async (tx) => {
|
|
817
|
+
// Operation 1: Save verb vector data
|
|
818
|
+
tx.addOperation(new SaveVerbOperation(this.storage, {
|
|
819
|
+
id,
|
|
842
820
|
vector: relationVector,
|
|
843
821
|
connections: new Map(),
|
|
844
822
|
verb: params.type,
|
|
845
|
-
sourceId: params.
|
|
846
|
-
targetId: params.
|
|
847
|
-
});
|
|
848
|
-
|
|
849
|
-
|
|
850
|
-
|
|
851
|
-
|
|
823
|
+
sourceId: params.from,
|
|
824
|
+
targetId: params.to
|
|
825
|
+
}));
|
|
826
|
+
// Operation 2: Save verb metadata
|
|
827
|
+
tx.addOperation(new SaveVerbMetadataOperation(this.storage, id, verbMetadata));
|
|
828
|
+
// Operation 3: Add to graph index for O(1) lookups
|
|
829
|
+
tx.addOperation(new AddToGraphIndexOperation(this.graphIndex, verb));
|
|
830
|
+
// Create bidirectional if requested
|
|
831
|
+
if (params.bidirectional) {
|
|
832
|
+
const reverseId = uuidv4();
|
|
833
|
+
const reverseVerb = {
|
|
834
|
+
...verb,
|
|
835
|
+
id: reverseId,
|
|
836
|
+
sourceId: params.to,
|
|
837
|
+
targetId: params.from,
|
|
838
|
+
source: toEntity.type,
|
|
839
|
+
target: fromEntity.type
|
|
840
|
+
};
|
|
841
|
+
// Operation 4: Save reverse verb vector data
|
|
842
|
+
tx.addOperation(new SaveVerbOperation(this.storage, {
|
|
843
|
+
id: reverseId,
|
|
844
|
+
vector: relationVector,
|
|
845
|
+
connections: new Map(),
|
|
846
|
+
verb: params.type,
|
|
847
|
+
sourceId: params.to,
|
|
848
|
+
targetId: params.from
|
|
849
|
+
}));
|
|
850
|
+
// Operation 5: Save reverse verb metadata
|
|
851
|
+
tx.addOperation(new SaveVerbMetadataOperation(this.storage, reverseId, verbMetadata));
|
|
852
|
+
// Operation 6: Add reverse relationship to graph index
|
|
853
|
+
tx.addOperation(new AddToGraphIndexOperation(this.graphIndex, reverseVerb));
|
|
854
|
+
}
|
|
855
|
+
});
|
|
852
856
|
return id;
|
|
853
857
|
});
|
|
854
858
|
}
|
|
@@ -858,10 +862,17 @@ export class Brainy {
|
|
|
858
862
|
async unrelate(id) {
|
|
859
863
|
await this.ensureInitialized();
|
|
860
864
|
return this.augmentationRegistry.execute('unrelate', { id }, async () => {
|
|
861
|
-
//
|
|
862
|
-
await this.
|
|
863
|
-
//
|
|
864
|
-
await this.
|
|
865
|
+
// Get verb data before deletion for rollback
|
|
866
|
+
const verb = await this.storage.getVerb(id);
|
|
867
|
+
// v5.8.0: Execute atomically with transaction system
|
|
868
|
+
await this.transactionManager.executeTransaction(async (tx) => {
|
|
869
|
+
// Operation 1: Remove from graph index
|
|
870
|
+
if (verb) {
|
|
871
|
+
tx.addOperation(new RemoveFromGraphIndexOperation(this.graphIndex, verb));
|
|
872
|
+
}
|
|
873
|
+
// Operation 2: Delete verb metadata (which also deletes vector)
|
|
874
|
+
tx.addOperation(new DeleteVerbMetadataOperation(this.storage, id));
|
|
875
|
+
});
|
|
865
876
|
});
|
|
866
877
|
}
|
|
867
878
|
/**
|
|
@@ -55,27 +55,96 @@ export declare class GraphAdjacencyIndex {
|
|
|
55
55
|
private ensureInitialized;
|
|
56
56
|
/**
|
|
57
57
|
* Core API - Neighbor lookup with LSM-tree storage
|
|
58
|
-
*
|
|
58
|
+
*
|
|
59
|
+
* O(log n) with bloom filter optimization (90% of queries skip disk I/O)
|
|
60
|
+
* v5.8.0: Added pagination support for high-degree nodes
|
|
61
|
+
*
|
|
62
|
+
* @param id Entity ID to get neighbors for
|
|
63
|
+
* @param optionsOrDirection Optional: direction string OR options object
|
|
64
|
+
* @returns Array of neighbor IDs (paginated if limit/offset specified)
|
|
65
|
+
*
|
|
66
|
+
* @example
|
|
67
|
+
* // Get all neighbors (backward compatible)
|
|
68
|
+
* const all = await graphIndex.getNeighbors(id)
|
|
69
|
+
*
|
|
70
|
+
* @example
|
|
71
|
+
* // Get outgoing neighbors (backward compatible)
|
|
72
|
+
* const out = await graphIndex.getNeighbors(id, 'out')
|
|
73
|
+
*
|
|
74
|
+
* @example
|
|
75
|
+
* // Get first 50 outgoing neighbors (new API)
|
|
76
|
+
* const page1 = await graphIndex.getNeighbors(id, { direction: 'out', limit: 50 })
|
|
77
|
+
*
|
|
78
|
+
* @example
|
|
79
|
+
* // Paginate through neighbors
|
|
80
|
+
* const page1 = await graphIndex.getNeighbors(id, { limit: 100, offset: 0 })
|
|
81
|
+
* const page2 = await graphIndex.getNeighbors(id, { limit: 100, offset: 100 })
|
|
59
82
|
*/
|
|
60
|
-
getNeighbors(id: string,
|
|
83
|
+
getNeighbors(id: string, optionsOrDirection?: {
|
|
84
|
+
direction?: 'in' | 'out' | 'both';
|
|
85
|
+
limit?: number;
|
|
86
|
+
offset?: number;
|
|
87
|
+
} | 'in' | 'out' | 'both'): Promise<string[]>;
|
|
61
88
|
/**
|
|
62
89
|
* Get verb IDs by source - Billion-scale optimization for getVerbsBySource
|
|
90
|
+
*
|
|
63
91
|
* O(log n) LSM-tree lookup with bloom filter optimization
|
|
64
92
|
* v5.7.1: Filters out deleted verb IDs (tombstone deletion workaround)
|
|
93
|
+
* v5.8.0: Added pagination support for entities with many relationships
|
|
65
94
|
*
|
|
66
95
|
* @param sourceId Source entity ID
|
|
67
|
-
* @
|
|
96
|
+
* @param options Optional configuration
|
|
97
|
+
* @param options.limit Maximum number of verb IDs to return (default: all)
|
|
98
|
+
* @param options.offset Number of verb IDs to skip (default: 0)
|
|
99
|
+
* @returns Array of verb IDs originating from this source (excluding deleted, paginated if requested)
|
|
100
|
+
*
|
|
101
|
+
* @example
|
|
102
|
+
* // Get all verb IDs (backward compatible)
|
|
103
|
+
* const all = await graphIndex.getVerbIdsBySource(sourceId)
|
|
104
|
+
*
|
|
105
|
+
* @example
|
|
106
|
+
* // Get first 50 verb IDs
|
|
107
|
+
* const page1 = await graphIndex.getVerbIdsBySource(sourceId, { limit: 50 })
|
|
108
|
+
*
|
|
109
|
+
* @example
|
|
110
|
+
* // Paginate through verb IDs
|
|
111
|
+
* const page1 = await graphIndex.getVerbIdsBySource(sourceId, { limit: 100, offset: 0 })
|
|
112
|
+
* const page2 = await graphIndex.getVerbIdsBySource(sourceId, { limit: 100, offset: 100 })
|
|
68
113
|
*/
|
|
69
|
-
getVerbIdsBySource(sourceId: string
|
|
114
|
+
getVerbIdsBySource(sourceId: string, options?: {
|
|
115
|
+
limit?: number;
|
|
116
|
+
offset?: number;
|
|
117
|
+
}): Promise<string[]>;
|
|
70
118
|
/**
|
|
71
119
|
* Get verb IDs by target - Billion-scale optimization for getVerbsByTarget
|
|
120
|
+
*
|
|
72
121
|
* O(log n) LSM-tree lookup with bloom filter optimization
|
|
73
122
|
* v5.7.1: Filters out deleted verb IDs (tombstone deletion workaround)
|
|
123
|
+
* v5.8.0: Added pagination support for popular target entities
|
|
74
124
|
*
|
|
75
125
|
* @param targetId Target entity ID
|
|
76
|
-
* @
|
|
77
|
-
|
|
78
|
-
|
|
126
|
+
* @param options Optional configuration
|
|
127
|
+
* @param options.limit Maximum number of verb IDs to return (default: all)
|
|
128
|
+
* @param options.offset Number of verb IDs to skip (default: 0)
|
|
129
|
+
* @returns Array of verb IDs pointing to this target (excluding deleted, paginated if requested)
|
|
130
|
+
*
|
|
131
|
+
* @example
|
|
132
|
+
* // Get all verb IDs (backward compatible)
|
|
133
|
+
* const all = await graphIndex.getVerbIdsByTarget(targetId)
|
|
134
|
+
*
|
|
135
|
+
* @example
|
|
136
|
+
* // Get first 50 verb IDs
|
|
137
|
+
* const page1 = await graphIndex.getVerbIdsByTarget(targetId, { limit: 50 })
|
|
138
|
+
*
|
|
139
|
+
* @example
|
|
140
|
+
* // Paginate through verb IDs
|
|
141
|
+
* const page1 = await graphIndex.getVerbIdsByTarget(targetId, { limit: 100, offset: 0 })
|
|
142
|
+
* const page2 = await graphIndex.getVerbIdsByTarget(targetId, { limit: 100, offset: 100 })
|
|
143
|
+
*/
|
|
144
|
+
getVerbIdsByTarget(targetId: string, options?: {
|
|
145
|
+
limit?: number;
|
|
146
|
+
offset?: number;
|
|
147
|
+
}): Promise<string[]>;
|
|
79
148
|
/**
|
|
80
149
|
* Get verb from cache or storage - Billion-scale memory optimization
|
|
81
150
|
* Uses UnifiedCache with LRU eviction instead of storing all verbs in memory
|