@soulcraft/cortex 1.4.0 → 1.4.1

package/README.md

@@ -1,97 +1,121 @@
-# @soulcraft/cortex
+# Cortex
 
-Native Rust acceleration for [Brainy](https://github.com/soulcraftlabs/brainy). Drop-in performance upgrade — zero configuration required.
+<p align="center">
+  <img src="https://raw.githubusercontent.com/soulcraftlabs/brainy/main/brainy.png" alt="Brainy Logo" width="200">
+</p>
 
-## Pricing
+<p align="center">
+  <strong>Native Rust acceleration for <a href="https://github.com/soulcraftlabs/brainy">Brainy</a></strong>
+</p>
 
-Cortex is a commercial add-on for Brainy. A license key is required after the 14-day trial.
+<p align="center">
+  <a href="https://www.npmjs.com/package/@soulcraft/cortex"><img src="https://badge.fury.io/js/%40soulcraft%2Fcortex.svg" alt="npm version"></a>
+  <a href="https://soulcraft.com/product"><img src="https://img.shields.io/badge/info-soulcraft.com-blue.svg" alt="Product Info"></a>
+</p>
 
-| Plan | Price | For |
-|------|-------|-----|
-| **Standard** | $199/year | Individuals and small teams |
-| **Enterprise** | $999/year | Companies, priority support |
+---
 
-**[Purchase a license at soulcraft.com/pricing](https://soulcraft.com/pricing?focus=pro)**
+## What is Cortex?
 
-**14-day free trial** — starts automatically on first use, no signup required.
+Cortex replaces Brainy's JavaScript internals with native Rust implementations. Same API, same data, dramatically faster.
 
-## Installation
+Install it, and Brainy automatically uses Rust for distance calculations, embeddings, metadata queries, graph operations, and more. No code changes needed.
 
 ```bash
 npm install @soulcraft/cortex
 ```
 
-That's it. Brainy auto-detects cortex during `init()` and uses native implementations where available.
+```typescript
+import Brainy from '@soulcraft/brainy'
 
-## License Setup
+const brain = new Brainy()
+await brain.init()
+// Cortex is auto-detected. That's it.
 
-After purchasing, set your license key via environment variable or file:
+// Verify what's accelerated
+const diag = brain.diagnostics()
+console.log(diag.providers)
+// { distance: { source: 'plugin' }, embeddings: { source: 'plugin' }, ... }
+```
 
-```bash
-# Option 1: Environment variable
-export SOULCRAFT_LICENSE=sc_cortex_<your-key>
+## Why Cortex?
 
-# Option 2: License file
-echo "sc_cortex_<your-key>" > ~/.soulcraft/license
-```
+Brainy works great on its own with pure JavaScript and WASM. Cortex is for when you need more:
+
+- **SIMD distance calculations** — every vector comparison gets faster
+- **Native ML inference** — Candle engine replaces WASM embeddings, with CPU, CUDA, and Metal support
+- **Rust metadata engine** — queries and mutations run in Rust with CRoaring bitmaps
+- **Native graph index** — 4 LSM-trees with verb tracking, all in Rust
+- **Batch embeddings** — single Rust forward pass for bulk operations
+- **Zero-copy storage** — mmap-backed SSTables for disk-heavy workloads
 
-License validation is offline-only — no network calls, no telemetry.
+Brainy's plugin system means cortex acceleration flows through every operation — `add()`, `find()`, `update()`, `delete()`, `similar()`, VFS path resolution, neural APIs, and everything in between.
 
 ## What Gets Accelerated
 
-| Subsystem | Native Implementation | Impact |
-|-----------|----------------------|--------|
-| Distance functions | SIMD cosine/euclidean/manhattan/dot product | Every vector search uses native distance |
-| Vector quantization | SIMD SQ8/SQ4 distance on compressed vectors | 4-8x memory reduction |
-| Product quantization | Native PQ codebook training and ADC lookup | 16-32x compression for large datasets |
-| Graph compression | Delta-varint encoded connection lists | 2-3x smaller graph storage |
-| Metadata index | Full Rust query/mutation engine with bitmap operations | Faster metadata filtering and mutations |
-| Graph adjacency | 4 LSM-trees with verb tracking in Rust | Faster relationship queries |
-| Embeddings | Candle ML (CPU, CUDA, Metal) | Native inference, no WASM compilation |
-| Roaring bitmaps | CRoaring bindings (binary-compatible) | Faster set operations on entity IDs |
-| Msgpack | Native encode/decode | Faster serialization |
-| Storage | MmapFileSystemStorage with zero-copy SSTables | Faster disk I/O for graph data |
+| Subsystem | What Cortex Provides | Impact |
+|-----------|---------------------|--------|
+| **Distance** | SIMD cosine, euclidean, manhattan, dot product | Every vector comparison |
+| **Embeddings** | Candle ML engine (CPU/CUDA/Metal) | No WASM compilation, GPU support |
+| **Batch embeddings** | Single forward pass for multiple texts | Bulk import and reindex |
+| **Metadata index** | Rust query/mutation engine with CRoaring | Faster filtering and type queries |
+| **Graph adjacency** | 4 LSM-trees with verb tracking | Faster relationship operations |
+| **Entity ID mapper** | Rust O(1) HashMap for UUID-integer mapping | Bitmap index operations |
+| **Vector quantization** | SIMD SQ8/SQ4 distance on compressed vectors | 4-8x memory reduction |
+| **Product quantization** | Native PQ codebook training and ADC | 16-32x compression |
+| **Graph compression** | Delta-varint encoded connections | 2-3x smaller graph storage |
+| **Roaring bitmaps** | CRoaring bindings | Faster set operations |
+| **Msgpack** | Native encode/decode | Faster serialization |
+| **Storage** | MmapFileSystemStorage with zero-copy SSTables | Faster disk I/O |
+
+## Getting Started
+
+### 1. Install
 
-## Platform Support
+```bash
+npm install @soulcraft/brainy @soulcraft/cortex
+```
 
-| Platform | Architecture | Status |
-|----------|-------------|--------|
-| Linux | x64 (glibc) | Supported |
-| Linux | arm64 (glibc) | Supported |
-| macOS | arm64 (Apple Silicon) | Supported |
-| macOS | x64 (Intel) | Supported |
-| Windows | x64 | Supported |
+### 2. Set up your license
+
+```bash
+# Environment variable
+export SOULCRAFT_LICENSE=sc_cortex_<your-key>
 
-## Usage
+# Or license file
+echo "sc_cortex_<your-key>" > ~/.soulcraft/license
+```
+
+License validation is offline — no network calls, no telemetry.
+
+### 3. Use Brainy as normal
 
 ```typescript
 import Brainy from '@soulcraft/brainy'
 
-// cortex is auto-detected — no import or config needed
 const brain = new Brainy({ storage: { type: 'filesystem', rootDirectory: './data' } })
 await brain.init()
-
-// Check if native acceleration is active
-console.log(brain.getActivePlugins())
-// → ['@soulcraft/cortex']
+// [brainy] Plugin activated: @soulcraft/cortex
+// [brainy] Providers: 10/10 native (@soulcraft/cortex)
 ```
 
-### Manual Registration
+### 4. Verify in production
 
-If auto-detection doesn't suit your setup, register the plugin manually:
+Use `requireProviders()` to fail fast if cortex isn't providing expected acceleration:
 
 ```typescript
-import Brainy from '@soulcraft/brainy'
-import cortexPlugin from '@soulcraft/cortex'
+brain.requireProviders(['distance', 'embeddings', 'metadataIndex', 'graphIndex'])
+```
 
-const brain = new Brainy({ storage: { type: 'memory' } })
-brain.use(cortexPlugin)
-await brain.init()
+Or inspect the full picture with diagnostics:
+
+```typescript
+console.log(brain.diagnostics())
 ```
 
-### MmapFileSystemStorage
+## MmapFileSystemStorage
 
-When cortex is installed, a new storage adapter becomes available:
+Cortex includes a storage adapter with mmap-backed binary blob support:
 
 ```typescript
 const brain = new Brainy({
@@ -99,17 +123,26 @@ const brain = new Brainy({
 })
 ```
 
-This extends the standard FileSystemStorage with binary blob support for zero-copy mmap access to LSM-tree SSTables.
+Zero-copy access to LSM-tree SSTables — useful for large graph datasets where disk I/O is the bottleneck.
+
+## Platform Support
+
+| Platform | Architecture | Status |
+|----------|-------------|--------|
+| Linux | x64 (glibc) | Supported |
+| Linux | arm64 (glibc) | Supported |
+| macOS | arm64 (Apple Silicon) | Supported |
+| macOS | x64 (Intel) | Supported |
+| Windows | x64 | Supported |
 
 ## Requirements
 
 - `@soulcraft/brainy` >= 7.0.0
 - Node.js >= 22 or Bun >= 1.3.0
-- Platform: Linux/macOS/Windows (see table above)
 
-## Direct Access
+## Direct Native Access
 
-For advanced use cases, you can access native bindings directly:
+For advanced use cases, access the Rust bindings directly:
 
 ```typescript
 import { loadNativeModule, isNativeAvailable } from '@soulcraft/cortex'
@@ -120,6 +153,13 @@ if (isNativeAvailable()) {
 }
 ```
 
+## Learn More
+
+- **[Product info and licensing](https://soulcraft.com/product)**
+- **[Brainy documentation](https://soulcraft.com/docs)**
+- **[Brainy on GitHub](https://github.com/soulcraftlabs/brainy)**
+- **[Plugin development guide](https://github.com/soulcraftlabs/brainy/blob/main/docs/PLUGINS.md)**
+
 ## License
 
 Commercial. See [LICENSE](./LICENSE) for details.

package/dist/hnsw/NativeHNSWWrapper.d.ts

@@ -0,0 +1,115 @@
+/**
+ * NativeHNSWWrapper — Thin TS wrapper around the Rust HNSW graph engine
+ *
+ * Bridges brainy's HNSWIndex API to the native Rust NativeHNSWIndex.
+ * The Rust engine handles all graph operations (insert, search, remove) in-memory
+ * while this wrapper handles async storage I/O for persistence.
+ *
+ * Key design:
+ * - Uses addItemFull() for single-FFI-call inserts (returns node + neighbors + system)
+ * - Persistence modes: 'immediate' (safe) or 'deferred' (30-50x faster for cloud)
+ * - COW via native fork() (Arc-based copy-on-write in Rust)
+ * - rebuild() restores pre-computed graph from storage (O(N) vs O(N log N) rebuild)
+ */
+import type { HNSWConfig, HNSWNoun, Vector, VectorDocument, DistanceFunction, StorageAdapter } from '@soulcraft/brainy';
+export declare class NativeHNSWWrapper {
+    private native;
+    private config;
+    private distanceFunction;
+    private storage;
+    private persistMode;
+    private dirtyNodes;
+    private dirtySystem;
+    private useParallelization;
+    private unifiedCache;
+    private cowEnabled;
+    constructor(config: (Partial<HNSWConfig> & {
+        distanceFunction?: DistanceFunction;
+    }) | undefined, distanceFunction: DistanceFunction, options?: {
+        storage?: StorageAdapter | null;
+        persistMode?: 'immediate' | 'deferred';
+    });
+    addItem(item: VectorDocument): Promise<string>;
+    removeItem(id: string): Promise<boolean>;
+    search(queryVector: Vector, k?: number, filter?: (id: string) => Promise<boolean>, options?: {
+        rerank?: {
+            multiplier: number;
+        };
+        candidateIds?: string[];
+    }): Promise<Array<[string, number]>>;
+    flush(): Promise<number>;
+    getDirtyNodeCount(): number;
+    getPersistMode(): 'immediate' | 'deferred';
+    rebuild(options?: {
+        lazy?: boolean;
+        batchSize?: number;
+        onProgress?: (loaded: number, total: number) => void;
+    }): Promise<void>;
+    enableCOW(parent: NativeHNSWWrapper): void;
+    setUseParallelization(useParallelization: boolean): void;
+    getUseParallelization(): boolean;
+    size(): number;
+    clear(): void;
+    getEntryPointId(): string | null;
+    getMaxLevel(): number;
+    getDimension(): number | null;
+    getConfig(): HNSWConfig;
+    getDistanceFunction(): DistanceFunction;
+    /**
+     * Get all nouns — builds HNSWNoun objects from native data.
+     * @deprecated Use getNounsPaginated() instead
+     */
+    getNouns(): Map<string, HNSWNoun>;
+    getNounsPaginated(options?: {
+        offset?: number;
+        limit?: number;
+        filter?: (noun: HNSWNoun) => boolean;
+    }): {
+        items: Map<string, HNSWNoun>;
+        totalCount: number;
+        hasMore: boolean;
+    };
+    getNodesAtLevel(level: number): HNSWNoun[];
+    getLevelStats(): Array<{
+        level: number;
+        nodeCount: number;
+        avgConnections: number;
+    }>;
+    getIndexHealth(): {
+        averageConnections: number;
+        layerDistribution: number[];
+        maxLayer: number;
+        totalNodes: number;
+    };
+    getCacheStats(): {
+        cachingStrategy: 'preloaded' | 'on-demand';
+        autoDetection: {
+            entityCount: number;
+            estimatedVectorMemoryMB: number;
+            availableCacheMB: number;
+            threshold: number;
+            rationale: string;
+        };
+        unifiedCache: {
+            totalSize: number;
+            maxSize: number;
+            utilizationPercent: number;
+            itemCount: number;
+            hitRatePercent: number;
+            totalAccessCount: number;
+        };
+        hnswCache: {
+            vectorsInCache: number;
+            cacheKeyPrefix: string;
+            estimatedMemoryMB: number;
+        };
+        fairness: {
+            hnswAccessCount: number;
+            hnswAccessPercent: number;
+            totalAccessCount: number;
+            fairnessViolation: boolean;
+        };
+        recommendations: string[];
+    };
+}
+//# sourceMappingURL=NativeHNSWWrapper.d.ts.map

package/dist/hnsw/NativeHNSWWrapper.js

@@ -0,0 +1,515 @@
+/**
+ * NativeHNSWWrapper — Thin TS wrapper around the Rust HNSW graph engine
+ *
+ * Bridges brainy's HNSWIndex API to the native Rust NativeHNSWIndex.
+ * The Rust engine handles all graph operations (insert, search, remove) in-memory
+ * while this wrapper handles async storage I/O for persistence.
+ *
+ * Key design:
+ * - Uses addItemFull() for single-FFI-call inserts (returns node + neighbors + system)
+ * - Persistence modes: 'immediate' (safe) or 'deferred' (30-50x faster for cloud)
+ * - COW via native fork() (Arc-based copy-on-write in Rust)
+ * - rebuild() restores pre-computed graph from storage (O(N) vs O(N log N) rebuild)
+ */
+import { loadNativeModule } from '../native/index.js';
+import { getGlobalCache, prodLog } from '@soulcraft/brainy/internals';
+const DEFAULT_CONFIG = {
+    M: 16,
+    efConstruction: 200,
+    efSearch: 50,
+    ml: 16,
+};
+export class NativeHNSWWrapper {
+    native;
+    config;
+    distanceFunction;
+    storage;
+    persistMode;
+    dirtyNodes = new Set();
+    dirtySystem = false;
+    useParallelization = true;
+    unifiedCache;
+    // COW support
+    cowEnabled = false;
+    constructor(config = {}, distanceFunction, options = {}) {
+        this.config = { ...DEFAULT_CONFIG, ...config };
+        this.distanceFunction = distanceFunction;
+        this.storage = options.storage || null;
+        this.persistMode = options.persistMode || 'immediate';
+        this.unifiedCache = getGlobalCache();
+        const bindings = loadNativeModule();
+        const nativeConfig = {
+            m: this.config.M,
+            efConstruction: this.config.efConstruction,
+            efSearch: this.config.efSearch,
+            ml: this.config.ml,
+        };
+        this.native = new bindings.NativeHNSWIndex(nativeConfig);
+        prodLog.info(`NativeHNSWWrapper initialized (M=${this.config.M}, ef=${this.config.efSearch}, ` +
+            `persist=${this.persistMode})`);
+    }
+    // ---------------------------------------------------------------------------
+    // Core CRUD
+    // ---------------------------------------------------------------------------
+    async addItem(item) {
+        if (!item)
+            throw new Error('Item is undefined or null');
+        if (!item.vector)
+            throw new Error('Vector is undefined or null');
+        const { id, vector } = item;
+        // Use addItemFull — single FFI call returns all persistence data
+        const result = this.native.addItemFull(id, vector);
+        if (this.persistMode === 'immediate' && this.storage) {
+            // Persist new node
+            await this.storage.saveHNSWData(result.id, result.nodeData).catch((e) => {
+                console.error(`[HNSW native] Failed to persist node ${result.id}:`, e);
+            });
+            // Persist modified neighbors
+            const neighborPromises = result.modifiedNeighbors.map(neighbor => this.storage.saveHNSWData(neighbor.id, {
+                level: neighbor.level,
+                connections: neighbor.connections,
+            }).catch((e) => {
+                console.error(`[HNSW native] Failed to persist neighbor ${neighbor.id}:`, e);
+            }));
+            await Promise.allSettled(neighborPromises);
+            // Persist system data
+            await this.storage.saveHNSWSystem(result.systemData).catch((e) => {
+                console.error('[HNSW native] Failed to persist system data:', e);
+            });
+        }
+        else if (this.persistMode === 'deferred') {
+            this.dirtyNodes.add(result.id);
+            for (const neighbor of result.modifiedNeighbors) {
+                this.dirtyNodes.add(neighbor.id);
+            }
+            this.dirtySystem = true;
+        }
+        return result.id;
+    }
+    async removeItem(id) {
+        if (!this.native.hasNode(id))
+            return false;
+        // Get neighbors before removal (they'll be modified)
+        const nodeData = this.native.getNodeData(id);
+        const neighborIds = new Set();
+        if (nodeData) {
+            for (const ids of Object.values(nodeData.connections)) {
+                for (const nid of ids) {
+                    neighborIds.add(nid);
+                }
+            }
+        }
+        const removed = this.native.removeItem(id);
+        if (!removed)
+            return false;
+        if (this.persistMode === 'immediate' && this.storage) {
+            // Persist updated neighbors (their connections changed)
+            const promises = Array.from(neighborIds).map(async (nid) => {
+                const nData = this.native.getNodeData(nid);
+                if (nData) {
+                    await this.storage.saveHNSWData(nid, nData).catch((e) => {
+                        console.error(`[HNSW native] Failed to persist neighbor ${nid} after removal:`, e);
+                    });
+                }
+            });
+            await Promise.allSettled(promises);
+            // Persist system data (entry point may have changed)
+            const sysData = this.native.getSystemData();
+            await this.storage.saveHNSWSystem(sysData).catch((e) => {
+                console.error('[HNSW native] Failed to persist system data after removal:', e);
+            });
+        }
+        else if (this.persistMode === 'deferred') {
+            for (const nid of neighborIds) {
+                this.dirtyNodes.add(nid);
+            }
+            // Remove the deleted node from dirty set (no need to persist it)
+            this.dirtyNodes.delete(id);
+            this.dirtySystem = true;
+        }
+        return true;
+    }
+    async search(queryVector, k = 10, filter, options) {
+        if (this.native.size() === 0)
+            return [];
+        if (!queryVector)
+            throw new Error('Query vector is undefined or null');
+        let results;
+        if (options?.candidateIds && options.candidateIds.length > 0) {
+            // Pre-filtered search: use native searchWithCandidates for optimal performance
+            // The Rust engine only traverses candidate nodes, avoiding unnecessary distance calculations
+            results = this.native.searchWithCandidates(queryVector, k, this.config.efSearch, options.candidateIds);
+        }
+        else if (filter) {
+            // Fallback: search with larger k and post-filter asynchronously
+            const overRetrieve = k * 5;
+            const rawResults = this.native.search(queryVector, overRetrieve, this.config.efSearch);
+            const filtered = [];
+            for (const r of rawResults) {
+                if (await filter(r.id)) {
+                    filtered.push(r);
+                    if (filtered.length >= k)
+                        break;
+                }
+            }
+            results = filtered;
+        }
+        else {
+            results = this.native.search(queryVector, k, this.config.efSearch);
+        }
+        // Convert NativeSearchResult[] to [string, number][] tuples
+        return results.map(r => [r.id, r.distance]);
+    }
+    // ---------------------------------------------------------------------------
+    // Persistence
+    // ---------------------------------------------------------------------------
+    async flush() {
+        if (!this.storage)
+            return 0;
+        if (this.dirtyNodes.size === 0 && !this.dirtySystem)
+            return 0;
+        const startTime = Date.now();
+        const nodeCount = this.dirtyNodes.size;
+        // Batch persist dirty nodes
+        if (this.dirtyNodes.size > 0) {
+            const batchSize = 50;
+            const nodeIds = Array.from(this.dirtyNodes);
+            for (let i = 0; i < nodeIds.length; i += batchSize) {
+                const batch = nodeIds.slice(i, i + batchSize);
+                const promises = batch.map(nodeId => {
+                    const nodeData = this.native.getNodeData(nodeId);
+                    if (!nodeData)
+                        return Promise.resolve(); // Node was deleted
+                    return this.storage.saveHNSWData(nodeId, nodeData).catch((e) => {
+                        console.error(`[HNSW native flush] Failed to persist node ${nodeId}:`, e);
+                    });
+                });
+                await Promise.allSettled(promises);
+            }
+            this.dirtyNodes.clear();
+        }
+        // Persist system data
+        if (this.dirtySystem) {
+            const sysData = this.native.getSystemData();
+            await this.storage.saveHNSWSystem(sysData).catch((e) => {
+                console.error('[HNSW native flush] Failed to persist system data:', e);
+            });
+            this.dirtySystem = false;
+        }
+        const duration = Date.now() - startTime;
+        if (nodeCount > 0) {
+            prodLog.info(`[HNSW native] Flushed ${nodeCount} dirty nodes in ${duration}ms`);
+        }
+        return nodeCount;
+    }
+    getDirtyNodeCount() {
+        return this.dirtyNodes.size;
+    }
+    getPersistMode() {
+        return this.persistMode;
+    }
+    // ---------------------------------------------------------------------------
+    // Rebuild from storage
+    // ---------------------------------------------------------------------------
+    async rebuild(options = {}) {
+        if (!this.storage) {
+            prodLog.warn('NativeHNSWWrapper rebuild skipped: no storage adapter configured');
+            return;
+        }
+        const batchSize = options.batchSize || 1000;
+        try {
+            // Step 1: Clear native index
+            this.native.clear();
+            // Step 2: Load system data
+            const systemData = await this.storage.getHNSWSystem();
+            if (systemData && systemData.entryPointId) {
+                this.native.setSystemData(systemData.entryPointId, systemData.maxLevel);
+            }
+            // Step 3: Load nouns from storage and restore into native engine
+            const storageType = this.storage?.constructor.name || '';
+            const isLocalStorage = storageType === 'FileSystemStorage' ||
+                storageType === 'MemoryStorage' ||
+                storageType === 'OPFSStorage' ||
+                storageType === 'MmapFileSystemStorage';
+            let loadedCount = 0;
+            let totalCount;
+            if (isLocalStorage) {
+                // Local: load all at once
+                const result = await this.storage.getNounsWithPagination({
+                    limit: 10000000
+                });
+                totalCount = result.totalCount || result.items.length;
+                for (const nounData of result.items) {
+                    try {
+                        const hnswData = await this.storage.getHNSWData(nounData.id);
+                        if (!hnswData)
+                            continue;
+                        this.native.restoreNode(nounData.id, nounData.vector, hnswData.level, hnswData.connections);
+                        loadedCount++;
+                    }
+                    catch (error) {
+                        console.error(`Failed to rebuild HNSW data for ${nounData.id}:`, error);
+                    }
+                }
+                if (options.onProgress && totalCount !== undefined) {
+                    options.onProgress(loadedCount, totalCount);
+                }
+                prodLog.info(`NativeHNSW: Loaded ${loadedCount.toLocaleString()} nodes at once (local storage)`);
+            }
+            else {
+                // Cloud: paginated loading
+                let hasMore = true;
+                let offset = 0;
+                while (hasMore) {
+                    const result = await this.storage.getNounsWithPagination({
+                        limit: batchSize,
+                        offset,
+                    });
+                    if (totalCount === undefined && result.totalCount !== undefined) {
+                        totalCount = result.totalCount;
+                    }
+                    for (const nounData of result.items) {
+                        try {
+                            const hnswData = await this.storage.getHNSWData(nounData.id);
+                            if (!hnswData)
+                                continue;
+                            this.native.restoreNode(nounData.id, nounData.vector, hnswData.level, hnswData.connections);
+                            loadedCount++;
+                        }
+                        catch (error) {
+                            console.error(`Failed to rebuild HNSW data for ${nounData.id}:`, error);
+                        }
+                    }
+                    if (options.onProgress && totalCount !== undefined) {
+                        options.onProgress(loadedCount, totalCount);
+                    }
+                    hasMore = result.hasMore;
+                    offset += batchSize;
+                }
+            }
+            // Step 4: Recover entry point if needed
+            if (this.native.size() > 0 && !this.native.getEntryPointId()) {
+                // Find highest level node from native
+                const allIds = this.native.getAllIds();
+                let bestId = null;
+                let bestLevel = -1;
+                for (const nid of allIds) {
+                    const nd = this.native.getNodeData(nid);
+                    if (nd && nd.level > bestLevel) {
+                        bestLevel = nd.level;
+                        bestId = nid;
+                    }
+                }
+                if (bestId) {
+                    this.native.setSystemData(bestId, bestLevel);
+                    prodLog.info(`NativeHNSW entry point recovered: ${bestId} at level ${bestLevel}`);
+                }
+            }
+            prodLog.info(`NativeHNSW index rebuilt: ${loadedCount.toLocaleString()} entities, ` +
+                `${this.native.getMaxLevel() + 1} levels, ` +
+                `entry point: ${this.native.getEntryPointId() || 'none'}`);
+        }
+        catch (error) {
+            prodLog.error('NativeHNSW rebuild failed:', error);
+            throw new Error(`Failed to rebuild NativeHNSW index: ${error}`);
+        }
+    }
+    // ---------------------------------------------------------------------------
+    // COW
+    // ---------------------------------------------------------------------------
+    enableCOW(parent) {
+        this.cowEnabled = true;
+        this.native = parent.native.fork();
+        this.config = parent.config;
+        this.distanceFunction = parent.distanceFunction;
+        this.useParallelization = parent.useParallelization;
+        this.unifiedCache = parent.unifiedCache;
+        prodLog.info(`NativeHNSW COW enabled: ${this.native.size()} nodes (Arc-based fork)`);
+    }
+    setUseParallelization(useParallelization) {
+        this.useParallelization = useParallelization;
+    }
+    getUseParallelization() {
+        return this.useParallelization;
+    }
+    // ---------------------------------------------------------------------------
+    // Info / Introspection
+    // ---------------------------------------------------------------------------
+    size() {
+        return this.native.size();
+    }
+    clear() {
+        this.native.clear();
+        this.dirtyNodes.clear();
+        this.dirtySystem = false;
+    }
+    getEntryPointId() {
+        return this.native.getEntryPointId() || null;
+    }
+    getMaxLevel() {
+        return this.native.getMaxLevel();
+    }
+    getDimension() {
+        return this.native.getDimension() ?? null;
+    }
+    getConfig() {
+        return { ...this.config };
+    }
+    getDistanceFunction() {
+        return this.distanceFunction;
+    }
+    /**
+     * Get all nouns — builds HNSWNoun objects from native data.
+     * @deprecated Use getNounsPaginated() instead
+     */
+    getNouns() {
+        const result = new Map();
+        const allIds = this.native.getAllIds();
+        for (const id of allIds) {
+            const nodeData = this.native.getNodeData(id);
+            const vector = this.native.getVector(id);
+            if (!nodeData || !vector)
+                continue;
+            const connections = new Map();
+            for (const [levelStr, ids] of Object.entries(nodeData.connections)) {
+                connections.set(parseInt(levelStr, 10), new Set(ids));
+            }
+            result.set(id, {
+                id,
+                vector,
+                connections,
+                level: nodeData.level,
+            });
+        }
+        return result;
+    }
+    getNounsPaginated(options = {}) {
+        const offset = options.offset || 0;
+        const limit = options.limit || 100;
+        const allIds = this.native.getAllIds();
+        const totalCount = allIds.length;
+        // Build noun objects for the requested page
+        const items = new Map();
+        let added = 0;
+        let skipped = 0;
+        for (const id of allIds) {
+            const nodeData = this.native.getNodeData(id);
+            const vector = this.native.getVector(id);
+            if (!nodeData || !vector)
+                continue;
+            const connections = new Map();
+            for (const [levelStr, ids] of Object.entries(nodeData.connections)) {
+                connections.set(parseInt(levelStr, 10), new Set(ids));
+            }
+            const noun = { id, vector, connections, level: nodeData.level };
+            if (options.filter && !options.filter(noun))
+                continue;
+            if (skipped < offset) {
+                skipped++;
+                continue;
+            }
+            items.set(id, noun);
+            added++;
+            if (added >= limit)
+                break;
+        }
+        return {
+            items,
+            totalCount,
+            hasMore: offset + limit < totalCount,
+        };
+    }
+    getNodesAtLevel(level) {
+        const ids = this.native.getNodesAtLevel(level);
+        const nodes = [];
+        for (const id of ids) {
+            const nodeData = this.native.getNodeData(id);
+            const vector = this.native.getVector(id);
+            if (!nodeData || !vector)
+                continue;
+            const connections = new Map();
+            for (const [levelStr, nids] of Object.entries(nodeData.connections)) {
+                connections.set(parseInt(levelStr, 10), new Set(nids));
+            }
+            nodes.push({ id, vector, connections, level: nodeData.level });
+        }
+        return nodes;
+    }
+    getLevelStats() {
+        const nativeStats = this.native.getLevelStats();
+        return nativeStats.map(s => ({
+            level: s.level,
+            nodeCount: s.nodeCount,
+            avgConnections: s.avgConnections,
+        }));
+    }
+    getIndexHealth() {
+        const health = this.native.getIndexHealth();
+        return {
+            averageConnections: health.averageConnections,
+            layerDistribution: health.layerDistribution,
+            maxLayer: health.maxLayer,
+            totalNodes: health.totalNodes,
+        };
+    }
+    getCacheStats() {
+        const cacheStats = this.unifiedCache.getStats();
+        const entityCount = this.native.size();
+        const vectorDimension = this.native.getDimension() || 384;
+        const bytesPerVector = vectorDimension * 4;
+        const estimatedVectorMemoryMB = (entityCount * bytesPerVector) / (1024 * 1024);
+        const availableCacheMB = (cacheStats.maxSize * 0.8) / (1024 * 1024);
+        const vectorsInCache = cacheStats.typeCounts.hnsw || 0;
+        const hnswMemoryBytes = cacheStats.typeSizes.hnsw || 0;
+        const hnswAccessCount = cacheStats.typeAccessCounts.hnsw || 0;
+        const totalAccessCount = cacheStats.totalAccessCount;
+        const hnswAccessPercent = totalAccessCount > 0 ? (hnswAccessCount / totalAccessCount) * 100 : 0;
+        const hnswCachePercent = cacheStats.maxSize > 0 ? (hnswMemoryBytes / cacheStats.maxSize) * 100 : 0;
+        const fairnessViolation = hnswCachePercent > 90 && hnswAccessPercent < 10;
+        const hitRatePercent = (cacheStats.hitRate * 100) || 0;
+        const cachingStrategy = estimatedVectorMemoryMB < availableCacheMB ? 'preloaded' : 'on-demand';
+        const recommendations = [];
+        if (cachingStrategy === 'on-demand' && hitRatePercent < 50) {
+            recommendations.push(`Low cache hit rate (${hitRatePercent.toFixed(1)}%). Consider increasing UnifiedCache size`);
+        }
+        if (fairnessViolation) {
+            recommendations.push(`Fairness violation: HNSW using ${hnswCachePercent.toFixed(1)}% cache with only ${hnswAccessPercent.toFixed(1)}% access`);
+        }
+        if (recommendations.length === 0) {
+            recommendations.push('All metrics healthy - no action needed');
+        }
+        return {
+            cachingStrategy,
+            autoDetection: {
+                entityCount,
+                estimatedVectorMemoryMB: parseFloat(estimatedVectorMemoryMB.toFixed(2)),
+                availableCacheMB: parseFloat(availableCacheMB.toFixed(2)),
+                threshold: 0.8,
+                rationale: cachingStrategy === 'preloaded'
+                    ? `Vectors in native Rust memory (${estimatedVectorMemoryMB.toFixed(1)}MB)`
+                    : `Adaptive on-demand loading (${estimatedVectorMemoryMB.toFixed(1)}MB > ${availableCacheMB.toFixed(1)}MB threshold)`
+            },
+            unifiedCache: {
+                totalSize: cacheStats.totalSize,
+                maxSize: cacheStats.maxSize,
+                utilizationPercent: parseFloat((cacheStats.utilization * 100).toFixed(2)),
+                itemCount: cacheStats.itemCount,
+                hitRatePercent: parseFloat(hitRatePercent.toFixed(2)),
+                totalAccessCount: cacheStats.totalAccessCount,
+            },
+            hnswCache: {
+                vectorsInCache,
+                cacheKeyPrefix: 'hnsw:vector:',
+                estimatedMemoryMB: parseFloat((hnswMemoryBytes / (1024 * 1024)).toFixed(2)),
+            },
+            fairness: {
+                hnswAccessCount,
+                hnswAccessPercent: parseFloat(hnswAccessPercent.toFixed(2)),
+                totalAccessCount,
+                fairnessViolation,
+            },
+            recommendations,
+        };
+    }
+}
+//# sourceMappingURL=NativeHNSWWrapper.js.map

package/dist/plugin.d.ts

@@ -8,15 +8,17 @@
  * can always access their data. Compute acceleration requires a valid license.
  *
  * Provider registration order:
- * 1. storage:mmap-filesystem — ALWAYS registered (data access)
+ * 1. storage:filesystem — mmap-enhanced FileSystemStorage (ALWAYS registered)
  * 2. distance    — SIMD-accelerated cosine (licensed)
- * 3. metadataIndex — Native Rust query/mutation engine (licensed)
- * 4. graphIndex    — Native 4 LSM-trees with verb tracking (licensed)
- * 5. embeddings    — Candle ML native engine (CPU/CUDA/Metal) (licensed)
- * 6. embedBatch    — Native batch embedding (single forward pass) (licensed)
- * 7. entityIdMapper — Native UUID ↔ integer mapping (licensed)
- * 8. roaring       — CRoaring bitmap bindings (licensed)
- * 9. msgpack       — Native encode/decode (licensed)
+ * 3. hnsw        — Native Rust HNSW graph engine (licensed)
+ * 4. cache       — Native Rust eviction engine with JS data storage (licensed)
+ * 5. metadataIndex — Native Rust query/mutation engine (licensed)
+ * 6. graphIndex    — Native 4 LSM-trees with verb tracking (licensed)
+ * 7. embeddings    — Candle ML native engine (CPU/CUDA/Metal) (licensed)
+ * 8. embedBatch    — Native batch embedding (single forward pass) (licensed)
+ * 9. entityIdMapper — Native UUID ↔ integer mapping (licensed)
+ * 10. roaring       — CRoaring bitmap bindings (licensed)
+ * 11. msgpack       — Native encode/decode (licensed)
  */
 import type { BrainyPlugin } from '@soulcraft/brainy/plugin';
 declare const cortexPlugin: BrainyPlugin;

package/dist/plugin.js

@@ -8,27 +8,33 @@
  * can always access their data. Compute acceleration requires a valid license.
  *
  * Provider registration order:
- * 1. storage:mmap-filesystem — ALWAYS registered (data access)
+ * 1. storage:filesystem — mmap-enhanced FileSystemStorage (ALWAYS registered)
  * 2. distance    — SIMD-accelerated cosine (licensed)
- * 3. metadataIndex — Native Rust query/mutation engine (licensed)
- * 4. graphIndex    — Native 4 LSM-trees with verb tracking (licensed)
- * 5. embeddings    — Candle ML native engine (CPU/CUDA/Metal) (licensed)
- * 6. embedBatch    — Native batch embedding (single forward pass) (licensed)
- * 7. entityIdMapper — Native UUID ↔ integer mapping (licensed)
- * 8. roaring       — CRoaring bitmap bindings (licensed)
- * 9. msgpack       — Native encode/decode (licensed)
+ * 3. hnsw        — Native Rust HNSW graph engine (licensed)
+ * 4. cache       — Native Rust eviction engine with JS data storage (licensed)
+ * 5. metadataIndex — Native Rust query/mutation engine (licensed)
+ * 6. graphIndex    — Native 4 LSM-trees with verb tracking (licensed)
+ * 7. embeddings    — Candle ML native engine (CPU/CUDA/Metal) (licensed)
+ * 8. embedBatch    — Native batch embedding (single forward pass) (licensed)
+ * 9. entityIdMapper — Native UUID ↔ integer mapping (licensed)
+ * 10. roaring       — CRoaring bitmap bindings (licensed)
+ * 11. msgpack       — Native encode/decode (licensed)
  */
 import { loadNativeModule, isNativeAvailable } from './native/index.js';
 import { validateLicense } from './license.js';
 const cortexPlugin = {
     name: '@soulcraft/cortex',
     async activate(context) {
-        // Storage adapters are ALWAYS registered — users must always be able
-        // to access their data, even if the license expires. Brainy falls back
-        // to JS compute but needs the storage adapter to read the files.
+        // Storage: Override built-in FileSystemStorage with mmap-enhanced version.
+        // MmapFileSystemStorage extends FileSystemStorage, adding binary blob methods
+        // (saveBinaryBlob/loadBinaryBlob/getBinaryBlobPath) that enable Rust native
+        // indexes to mmap SSTable files directly. Zero-config — users with
+        // type: 'filesystem' automatically get the enhanced version.
+        //
+        // ALWAYS registered (even without a license) so users can always access data.
         const { MmapFileSystemStorage } = await import('./storage/mmapFileSystemStorage.js');
-        context.registerProvider('storage:mmap-filesystem', {
-            name: 'mmap-filesystem',
+        context.registerProvider('storage:filesystem', {
+            name: 'filesystem',
             create: (config) => new MmapFileSystemStorage(config.rootDirectory, {
                 compression: config.compression,
                 compressionLevel: config.compressionLevel
@@ -72,6 +78,15 @@ const cortexPlugin = {
         });
         // Product Quantization: 16-32x compression for large datasets
         context.registerProvider('quantization:pq', native.NativePqCodebook);
+        // HNSW: Native Rust graph engine with SIMD distance and Arc-based COW
+        const { NativeHNSWWrapper } = await import('./hnsw/NativeHNSWWrapper.js');
+        context.registerProvider('hnsw', (config, distanceFn, options) => {
+            return new NativeHNSWWrapper(config, distanceFn, options);
+        });
+        // Cache: Native Rust eviction engine with JS data storage
+        // Registered before indexes so they use native cache for memory management
+        const { NativeUnifiedCacheWrapper } = await import('./utils/NativeUnifiedCache.js');
+        context.registerProvider('cache', new NativeUnifiedCacheWrapper());
         // Metadata index: Native Rust query/mutation engine
         const { MetadataIndexManager: NativeMetadataIndex } = await import('./utils/NativeMetadataIndex.js');
         context.registerProvider('metadataIndex', (storage) => new NativeMetadataIndex(storage));

package/dist/utils/NativeMetadataIndex.js

@@ -879,7 +879,7 @@ export class MetadataIndexManager {
             catch { }
             // Adaptive rebuild strategy
             const storageType = this.storage.constructor.name;
-            const isLocalStorage = storageType === 'FileSystemStorage' || storageType === 'MemoryStorage';
+            const isLocalStorage = storageType === 'FileSystemStorage' || storageType === 'MmapFileSystemStorage' || storageType === 'MemoryStorage';
             let totalNounsProcessed = 0;
             if (isLocalStorage) {
                 const result = await this.storage.getNouns({

package/dist/utils/NativeUnifiedCache.d.ts

@@ -0,0 +1,81 @@
+/**
+ * NativeUnifiedCacheWrapper — Native Rust eviction engine with JS data storage
+ *
+ * The Rust NativeUnifiedCache manages eviction metadata (key, type, size, cost,
+ * access counts) but does NOT store actual JS data. This wrapper:
+ * - Keeps a Map<string, any> for actual cached data
+ * - Delegates eviction decisions to the native engine
+ * - Implements request coalescing in JS (same as brainy's UnifiedCache)
+ * - Drop-in replacement for brainy's UnifiedCache
+ */
+import type { UnifiedCacheConfig } from '@soulcraft/brainy/internals';
+export declare class NativeUnifiedCacheWrapper {
+    private data;
+    private native;
+    private loadingPromises;
+    private readonly maxSize;
+    private readonly config;
+    private readonly memoryInfo;
+    private readonly allocationStrategy;
+    private memoryPressureCheckTimer;
+    private lastMemoryWarning;
+    constructor(config?: UnifiedCacheConfig);
+    get(key: string, loadFn?: () => Promise<any>): Promise<any>;
+    getSync(key: string): any | undefined;
+    set(key: string, data: any, type: 'hnsw' | 'metadata' | 'embedding' | 'other', size: number, rebuildCost?: number): void;
+    delete(key: string): boolean;
+    deleteByPrefix(prefix: string): number;
+    clear(type?: 'hnsw' | 'metadata' | 'embedding' | 'other'): void;
+    evictForSize(bytesNeeded: number): boolean;
+    private startFairnessMonitor;
+    private checkFairness;
+    private startMemoryPressureMonitor;
+    private checkMemoryPressure;
+    getStats(): {
+        totalSize: number;
+        maxSize: number;
+        utilization: number;
+        itemCount: number;
+        typeSizes: Record<string, number>;
+        typeCounts: Record<string, number>;
+        typeAccessCounts: Record<string, number>;
+        totalAccessCount: number;
+        hitRate: number;
+        memory: {
+            available: number;
+            source: "cgroup-v2" | "cgroup-v1" | "system" | "fallback";
+            isContainer: boolean;
+            systemTotal: number;
+            allocationRatio: number;
+            environment: "production" | "development" | "container" | "unknown";
+        };
+    };
+    getMemoryInfo(): {
+        memoryInfo: {
+            available: number;
+            source: "cgroup-v2" | "cgroup-v1" | "system" | "fallback";
+            isContainer: boolean;
+            systemTotal: number;
+            free: number;
+            warnings: string[];
+        };
+        allocationStrategy: {
+            cacheSize: number;
+            ratio: number;
+            minSize: number;
+            maxSize: number | null;
+            environment: "production" | "development" | "container" | "unknown";
+            modelMemory: number;
+            modelPrecision: "q8" | "fp32";
+            availableForCache: number;
+            reasoning: string;
+        };
+        currentPressure: {
+            pressure: "none" | "moderate" | "high" | "critical";
+            warnings: string[];
+        };
+    };
+    saveAccessPatterns(): Promise<any>;
+    loadAccessPatterns(patterns: any): Promise<void>;
+}
+//# sourceMappingURL=NativeUnifiedCache.d.ts.map

package/dist/utils/NativeUnifiedCache.js

@@ -0,0 +1,268 @@
+/**
+ * NativeUnifiedCacheWrapper — Native Rust eviction engine with JS data storage
+ *
+ * The Rust NativeUnifiedCache manages eviction metadata (key, type, size, cost,
+ * access counts) but does NOT store actual JS data. This wrapper:
+ * - Keeps a Map<string, any> for actual cached data
+ * - Delegates eviction decisions to the native engine
+ * - Implements request coalescing in JS (same as brainy's UnifiedCache)
+ * - Drop-in replacement for brainy's UnifiedCache
+ */
+import { loadNativeModule } from '../native/index.js';
+import { prodLog } from '@soulcraft/brainy/internals';
+import { getRecommendedCacheConfig, formatBytes, checkMemoryPressure, } from '@soulcraft/brainy/internals';
+// Cache type enum matching Rust (0-3)
+const CACHE_TYPE_MAP = {
+    hnsw: 0,
+    metadata: 1,
+    embedding: 2,
+    other: 3,
+};
+export class NativeUnifiedCacheWrapper {
+    data = new Map();
+    native;
+    loadingPromises = new Map();
+    maxSize;
+    config;
+    memoryInfo;
+    allocationStrategy;
+    memoryPressureCheckTimer = null;
+    lastMemoryWarning = 0;
+    constructor(config = {}) {
+        const recommendation = getRecommendedCacheConfig({
+            manualSize: config.maxSize,
+            minSize: config.minSize,
+            developmentMode: config.developmentMode,
+        });
+        this.memoryInfo = recommendation.memoryInfo;
+        this.allocationStrategy = recommendation.allocation;
+        this.maxSize = recommendation.allocation.cacheSize;
+        prodLog.info(`NativeUnifiedCache initialized: ${formatBytes(this.maxSize)} ` +
+            `(${this.allocationStrategy.environment} mode, ` +
+            `${(this.allocationStrategy.ratio * 100).toFixed(0)}% of ${formatBytes(this.allocationStrategy.availableForCache)} ` +
+            `after ${formatBytes(this.allocationStrategy.modelMemory)} ${this.allocationStrategy.modelPrecision.toUpperCase()} model)`);
+        for (const warning of recommendation.warnings) {
+            prodLog.warn(`NativeUnifiedCache: ${warning}`);
+        }
+        this.config = {
+            enableRequestCoalescing: true,
+            enableFairnessCheck: true,
+            fairnessCheckInterval: 30000,
+            persistPatterns: true,
+            enableMemoryMonitoring: true,
+            memoryCheckInterval: 30000,
+            ...config,
+        };
+        const bindings = loadNativeModule();
+        this.native = new bindings.NativeUnifiedCache(this.maxSize);
+        if (this.config.enableFairnessCheck) {
+            this.startFairnessMonitor();
+        }
+        if (this.config.enableMemoryMonitoring) {
+            this.startMemoryPressureMonitor();
+        }
+    }
+    // ---------------------------------------------------------------------------
+    // Core API
+    // ---------------------------------------------------------------------------
+    async get(key, loadFn) {
+        // Check JS data map
+        const cached = this.data.get(key);
+        if (cached !== undefined) {
+            this.native.recordAccess(key);
+            return cached;
+        }
+        if (!loadFn)
+            return undefined;
+        // Request coalescing
+        if (this.config.enableRequestCoalescing && this.loadingPromises.has(key)) {
+            return this.loadingPromises.get(key);
+        }
+        const loadPromise = loadFn();
+        if (this.config.enableRequestCoalescing) {
+            this.loadingPromises.set(key, loadPromise);
+        }
+        try {
+            const result = await loadPromise;
+            return result;
+        }
+        finally {
+            if (this.config.enableRequestCoalescing) {
+                this.loadingPromises.delete(key);
+            }
+        }
+    }
+    getSync(key) {
+        const cached = this.data.get(key);
+        if (cached !== undefined) {
+            this.native.recordAccess(key);
+            return cached;
+        }
+        return undefined;
+    }
+    set(key, data, type, size, rebuildCost = 1) {
+        const cacheType = CACHE_TYPE_MAP[type] ?? 3;
+        // Remove existing entry if present
+        if (this.data.has(key)) {
+            this.native.remove(key);
+        }
+        // Insert into native (may trigger eviction)
+        const evictionResult = this.native.insert(key, cacheType, size, rebuildCost);
+        // Remove evicted keys from JS data
+        for (const evictedKey of evictionResult.evictedKeys) {
+            this.data.delete(evictedKey);
+        }
+        // Store actual data in JS
+        this.data.set(key, data);
+    }
+    delete(key) {
+        const existed = this.data.has(key);
+        if (existed) {
+            this.native.remove(key);
+            this.data.delete(key);
+        }
+        return existed;
+    }
+    deleteByPrefix(prefix) {
+        const removedKeys = this.native.removeByPrefix(prefix);
+        for (const key of removedKeys) {
+            this.data.delete(key);
+        }
+        return removedKeys.length;
+    }
+    clear(type) {
+        if (!type) {
+            const clearedKeys = this.native.clear();
+            for (const key of clearedKeys) {
+                this.data.delete(key);
+            }
+            // Also clear any remaining data entries
+            this.data.clear();
+        }
+        else {
+            const cacheType = CACHE_TYPE_MAP[type] ?? 3;
+            const clearedKeys = this.native.clear(cacheType);
+            for (const key of clearedKeys) {
+                this.data.delete(key);
+            }
+        }
+    }
+    // ---------------------------------------------------------------------------
+    // Eviction
+    // ---------------------------------------------------------------------------
+    evictForSize(bytesNeeded) {
+        const result = this.native.evictForSize(bytesNeeded);
+        for (const key of result.evictedKeys) {
+            this.data.delete(key);
+        }
+        return result.bytesFreed >= bytesNeeded;
+    }
+    // ---------------------------------------------------------------------------
+    // Fairness
+    // ---------------------------------------------------------------------------
+    startFairnessMonitor() {
+        const timer = setInterval(() => {
+            this.checkFairness();
+        }, this.config.fairnessCheckInterval);
+        if (timer.unref)
+            timer.unref();
+    }
+    checkFairness() {
+        const result = this.native.checkFairness();
+        for (const key of result.evictedKeys) {
+            this.data.delete(key);
+        }
+    }
+    // ---------------------------------------------------------------------------
+    // Memory pressure
+    // ---------------------------------------------------------------------------
+    startMemoryPressureMonitor() {
+        const checkInterval = this.config.memoryCheckInterval || 30000;
+        this.memoryPressureCheckTimer = setInterval(() => {
+            this.checkMemoryPressure();
+        }, checkInterval);
+        if (this.memoryPressureCheckTimer.unref) {
+            this.memoryPressureCheckTimer.unref();
+        }
+    }
+    checkMemoryPressure() {
+        const stats = this.native.getStats();
+        const pressure = checkMemoryPressure(stats.totalSize, this.memoryInfo);
+        const now = Date.now();
+        const fiveMinutes = 5 * 60 * 1000;
+        if (pressure.warnings.length > 0 && now - this.lastMemoryWarning > fiveMinutes) {
+            for (const warning of pressure.warnings) {
+                prodLog.warn(`NativeUnifiedCache: ${warning}`);
+            }
+            this.lastMemoryWarning = now;
+        }
+        if (pressure.pressure === 'critical') {
+            const targetSize = Math.floor(this.maxSize * 0.7);
+            const bytesToFree = stats.totalSize - targetSize;
+            if (bytesToFree > 0) {
+                prodLog.warn(`NativeUnifiedCache: Critical memory pressure - forcing eviction of ${formatBytes(bytesToFree)}`);
+                this.evictForSize(bytesToFree);
+            }
+        }
+    }
+    // ---------------------------------------------------------------------------
+    // Statistics
+    // ---------------------------------------------------------------------------
+    getStats() {
+        const nativeStats = this.native.getStats();
+        const typeNames = ['hnsw', 'metadata', 'embedding', 'other'];
+        const typeSizes = {};
+        const typeCounts = {};
+        const typeAccessCounts = {};
+        for (let i = 0; i < typeNames.length; i++) {
+            typeSizes[typeNames[i]] = nativeStats.typeSizes[i] || 0;
+            typeCounts[typeNames[i]] = nativeStats.typeCounts[i] || 0;
+            typeAccessCounts[typeNames[i]] = nativeStats.typeAccessCounts[i] || 0;
+        }
+        const hitRate = nativeStats.totalAccessCount > 0
+            ? nativeStats.itemCount / nativeStats.totalAccessCount
+            : 0;
+        return {
+            totalSize: nativeStats.totalSize,
+            maxSize: nativeStats.maxSize,
+            utilization: nativeStats.maxSize > 0 ? nativeStats.totalSize / nativeStats.maxSize : 0,
+            itemCount: nativeStats.itemCount,
+            typeSizes,
+            typeCounts,
+            typeAccessCounts,
+            totalAccessCount: nativeStats.totalAccessCount,
+            hitRate,
+            memory: {
+                available: this.memoryInfo.available,
+                source: this.memoryInfo.source,
+                isContainer: this.memoryInfo.isContainer,
+                systemTotal: this.memoryInfo.systemTotal,
+                allocationRatio: this.allocationStrategy.ratio,
+                environment: this.allocationStrategy.environment,
+            },
+        };
+    }
+    getMemoryInfo() {
+        const stats = this.native.getStats();
+        return {
+            memoryInfo: { ...this.memoryInfo },
+            allocationStrategy: { ...this.allocationStrategy },
+            currentPressure: checkMemoryPressure(stats.totalSize, this.memoryInfo),
+        };
+    }
+    // ---------------------------------------------------------------------------
+    // Access pattern persistence
+    // ---------------------------------------------------------------------------
+    async saveAccessPatterns() {
+        if (!this.config.persistPatterns)
+            return;
+        const json = this.native.saveAccessPatterns();
+        return JSON.parse(json);
+    }
+    async loadAccessPatterns(patterns) {
+        if (!patterns)
+            return;
+        this.native.loadAccessPatterns(JSON.stringify(patterns));
+    }
+}
+//# sourceMappingURL=NativeUnifiedCache.js.map

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@soulcraft/cortex",
-  "version": "1.4.0",
+  "version": "1.4.1",
   "description": "Native Rust acceleration for Brainy — SIMD distance, vector quantization, zero-copy mmap, native embeddings. Commercial license required (14-day free trial).",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",